I’m packaging the prebuild libtensorflow for OS X.
For the Linux build I’m using patchelf to set the correct rpath in the .so. This however fails on OS X with “not an ELF executable” because the .so is a Mach-O 64-bit x86_64 dynamically linked shared library.
Is there a patchelf for Mach-O shared libraries?
Perhaps you are you looking for install_name_tool?
@mpickering thanks! install_name_tool is exactly what I’m looking for.
I’m currently using the following to patch the prebuilt libtensorflow:
Unfortunately when I build a package that depends on libtensorflow like haskell.packages.ghc822.tensorflow I get a run-time failure when running the test-suite (which doesn’t happen when I build on Linux):
$ nix-build -A haskell.packages.ghc822.tensorflow
these derivations will be built:
/nix/store/m7f1bapd3cm3q62f8wpxbp5lm09rkr7i-tensorflow-0.1.0.2.drv
building '/nix/store/m7f1bapd3cm3q62f8wpxbp5lm09rkr7i-tensorflow-0.1.0.2.drv'...
...
Test suite FFITest: RUNNING...
dyld: Library not loaded: @rpath/libtensorflow.so
Referenced from: /private/tmp/nix-build-tensorflow-0.1.0.2.drv-0/source/tensorflow/dist/build/FFITest/FFITest
Reason: image not found
Test suite FFITest: FAIL
Here’s some information about the current rpath of libtensorflow.so:
$ otool -l $(nix-build -A libtensorflow)/lib/libtensorflow.so
...
Load command 3
cmd LC_ID_DYLIB
cmdsize 48
name @rpath/libtensorflow.so (offset 24)
time stamp 1 Thu Jan 1 01:00:01 1970
current version 0.0.0
compatibility version 0.0.0
Load command 4
cmd LC_DYLD_INFO_ONLY
cmdsize 48
rebase_off 53649408
rebase_size 87208
bind_off 53736616
bind_size 40184
weak_bind_off 53776800
weak_bind_size 2480
lazy_bind_off 53779280
lazy_bind_size 103744
export_off 53883024
export_size 4488
Load command 5
cmd LC_SYMTAB
cmdsize 24
symoff 54053288
nsyms 241471
stroff 57965408
strsize 67297248
Load command 6
cmd LC_DYSYMTAB
cmdsize 80
ilocalsym 0
nlocalsym 239430
iextdefsym 239430
nextdefsym 217
iundefsym 239647
nundefsym 1824
tocoff 0
ntoc 0
modtaboff 0
nmodtab 0
extrefsymoff 0
nextrefsyms 0
indirectsymoff 57916824
nindirectsyms 12146
extreloff 0
nextrel 0
locreloff 0
nlocrel 0
Load command 7
cmd LC_UUID
cmdsize 24
uuid 45142FF3-C3B9-3409-AC92-9D4E1F4F9ED7
Load command 8
cmd LC_VERSION_MIN_MACOSX
cmdsize 16
version 10.12
sdk 10.12
Load command 9
cmd LC_SOURCE_VERSION
cmdsize 16
version 0.0
Load command 10
cmd LC_LOAD_DYLIB
cmdsize 64
name @rpath/libtensorflow_framework.so (offset 24)
time stamp 2 Thu Jan 1 01:00:02 1970
current version 0.0.0
compatibility version 0.0.0
Load command 11
cmd LC_LOAD_DYLIB
cmdsize 56
name /usr/lib/libSystem.B.dylib (offset 24)
time stamp 2 Thu Jan 1 01:00:02 1970
current version 1238.50.2
compatibility version 1.0.0
Load command 12
cmd LC_LOAD_DYLIB
cmdsize 104
name /System/Library/Frameworks/CoreFoundation.framework/Versions/A/CoreFoundation (offset 24)
time stamp 2 Thu Jan 1 01:00:02 1970
current version 1349.64.0
compatibility version 150.0.0
Load command 13
cmd LC_LOAD_DYLIB
cmdsize 96
name /System/Library/Frameworks/Security.framework/Versions/A/Security (offset 24)
time stamp 2 Thu Jan 1 01:00:02 1970
current version 57740.51.2
compatibility version 1.0.0
Load command 14
cmd LC_LOAD_DYLIB
cmdsize 88
name /System/Library/Frameworks/IOKit.framework/Versions/A/IOKit (offset 24)
time stamp 2 Thu Jan 1 01:00:02 1970
current version 275.0.0
compatibility version 1.0.0
Load command 15
cmd LC_LOAD_DYLIB
cmdsize 48
name /usr/lib/libc++.1.dylib (offset 24)
time stamp 2 Thu Jan 1 01:00:02 1970
current version 307.5.0
compatibility version 1.0.0
Load command 16
cmd LC_LOAD_DYLIB
cmdsize 96
name /System/Library/Frameworks/Foundation.framework/Versions/C/Foundation (offset 24)
time stamp 2 Thu Jan 1 01:00:02 1970
current version 1349.63.0
compatibility version 300.0.0
Load command 17
cmd LC_LOAD_DYLIB
cmdsize 56
name /usr/lib/libobjc.A.dylib (offset 24)
time stamp 2 Thu Jan 1 01:00:02 1970
current version 228.0.0
compatibility version 1.0.0
Load command 18
cmd LC_RPATH
cmdsize 32
path @loader_path/ (offset 12)
Load command 19
cmd LC_FUNCTION_STARTS
cmdsize 16
dataoff 53887512
datasize 156136
Load command 20
cmd LC_DATA_IN_CODE
cmdsize 16
dataoff 54043648
datasize 9640
Load command 21
cmd LC_RPATH
cmdsize 216
path /nix/store/yxl7yn6bx55j9wq58y88lvm126nkbadm-Libsystem-osx-10.11.6/lib:/nix/store/gypar7abrwdd13pmdal3wq0rcsgymrj2-clang-5.0.2-lib/lib:/nix/store/8yz1ml3h01rn4qni8nlxifaiq3acd0va-libtensorflow-1.8.0/lib (offset 12)
The non-nix-store paths like /System/Library/Frameworks/CoreFoundation.framework/Versions/A/CoreFoundation look a bit suspicious. Is that normal or should those be replaced by nix-store paths?
I’m clearly not well versed in Nix development on OS X. Anybody who can point me in the right direction?
Thanks.
The CoreFoundation paths are fine.
What about the entry for the FFITest executable?
You probably need to change the install name of lib/libtensorflow.so using -change.
Thanks, listing the RPATH of FFITest on OS X highlights the problem because it doesn’t contain nix/store/...-libtensorflow-1.8.0/lib:
$ nix-shell ../../nixpkgs -A haskell.packages.ghc822.tensorflow.env
$ cabal configure --enable-tests
$ cabal build
$ otool -l dist/build/FFITest/FFITest
dist/build/FFITest/FFITest:
...
Load command 7
cmd LC_LOAD_DYLINKER
cmdsize 32
name /usr/lib/dyld (offset 12)
Load command 8
cmd LC_UUID
cmdsize 24
uuid BC3F6925-DCDC-3E2D-AAC9-E668EB2292BC
Load command 9
cmd LC_VERSION_MIN_MACOSX
cmdsize 16
version 10.10
sdk 10.13
Load command 10
cmd LC_SOURCE_VERSION
cmdsize 16
version 0.0
Load command 11
cmd LC_MAIN
cmdsize 24
entryoff 5904
stacksize 0
Load command 12
cmd LC_LOAD_DYLIB
cmdsize 112
name /nix/store/yxl7yn6bx55j9wq58y88lvm126nkbadm-Libsystem-osx-10.11.6/lib/libSystem.B.dylib (offset 24)
time stamp 2 Thu Jan 1 01:00:02 1970
current version 1226.10.1
compatibility version 1.0.0
Load command 13
cmd LC_LOAD_DYLIB
cmdsize 48
name @rpath/libtensorflow.so (offset 24)
time stamp 2 Thu Jan 1 01:00:02 1970
current version 0.0.0
compatibility version 0.0.0
Load command 14
cmd LC_LOAD_DYLIB
cmdsize 104
name /nix/store/b2kgprf9lxyhg0shfjssr0fj8n642vdl-gmp-6.1.2/lib/libgmp.10.dylib (offset 24)
time stamp 2 Thu Jan 1 01:00:02 1970
current version 14.2.0
compatibility version 14.0.0
Load command 15
cmd LC_LOAD_DYLIB
cmdsize 112
name /nix/store/1zdjr2krpbxfdpax47r27v583dkn19ws-libiconv-osx-10.11.6/lib/libiconv.dylib (offset 24)
time stamp 2 Thu Jan 1 01:00:02 1970
current version 7.0.0
compatibility version 7.0.0
Load command 16
cmd LC_RPATH
cmdsize 96
path /nix/store/xfnj858xpavhbj0nyzgjhhcq868b4b4d-CF-osx-10.10.5/Library/Frameworks (offset 12)
Load command 17
cmd LC_FUNCTION_STARTS
cmdsize 16
dataoff 2168352
datasize 24064
Load command 18
cmd LC_DATA_IN_CODE
cmdsize 16
dataoff 2192416
datasize 200
while on Linux it does: (formatted for readability)
$ patchelf --print-rpath dist/build/FFITest/FFITest
/nix/store/zj40n2rdzb59c82z6p72017j69rsr8gg-interactive-tensorflow-0.1.0.2-environment/lib64:
/nix/store/zj40n2rdzb59c82z6p72017j69rsr8gg-interactive-tensorflow-0.1.0.2-environment/lib:
/nix/store/s3yvly4rfcig7hbli93yhwbbv2mwc8cs-gmp-6.1.2/lib:
/nix/store/5fk60xz8s6ssrj60n7lgqq6ksfzwx10w-libtensorflow-1.8.0/lib:
/nix/store/rpvyrx4ia53a9bqjw08jbfqb5q9bkdr0-glibc-2.27/lib:
/nix/store/2in0z8k88qrijnn4ck2f9pba5ybv8ls9-gcc-7.3.0-lib/lib
I do see that on OS X the Haskell builder is adding an RPATH to executables. However, this only happens when enableSharedExecutables is enabled (it defaults to false) and it only applies to installed executables, so not to executables for the test-suite.
@mpickering I don’t understand your suggestion about changing the install name of lib/libtensorflow.so. What should I change it into and why would that help?
Thanks a lot for your help so far!
We generally don’t use RPATH on darwin, adding $out/lib is disabled in the stdenv because we don’t use it. Dynamic libraries shouldn’t use @rpath since like you noticed this won’t work for packages that link against it unless rpath is configured manually there. CoreFoundation is a special case because we have both a pure and impure version, the RPATH entry enables us to switch to the impure version if necessary without having to rebuild the entire tree.
The way libraries get linked on darwin is using the install_name, see otool -D foo.dylib. The install_name is generally the absolute path to the final install location so in this case $out/lib/libtensorflow.so. This can be patched similar to library references using install_name_tool or our by including our fixDarwinDylibNames setup hook.
Note that if this is used during the check phase the library probably won’t be in place yet, some build systems use @rpath intermediately during the build and patch it before installing, but using the installCheckPhase instead should allow you to avoid that.
install_name_tool -id $out/lib/libfoo.dylib $out/lib/libfoo.dylib
Haskell libraries is another special case, but that’s only to avoid mach-o header limits that where reduced drastically in 10.12 or 10.12.
Using install_name_tool -id on libtensorflow I can successfully build haskell.packages.ghc822.tensorflow.
I will push this to master soon.
Thank you!