Tweag+Nix dev update #40

anonimasu · November 30, 2022, 8:37am

Any work planned on this:

Link to libraries through absolute paths?

opened 01:32PM - 12 Apr 17 UTC

1.severity: mass-rebuild 9.needs: community feedback significant

## Issue description `echo`'s dynamic section currently looks like this... ```… 0x0000000000000001 (NEEDED) Shared library: [libacl.so.1] 0x0000000000000001 (NEEDED) Shared library: [libattr.so.1] 0x0000000000000001 (NEEDED) Shared library: [librt.so.1] 0x0000000000000001 (NEEDED) Shared library: [libpthread.so.0] 0x0000000000000001 (NEEDED) Shared library: [libc.so.6] 0x000000000000001d (RUNPATH) Library runpath: [/nix/store/v0wcqsb6vpljx13vw8q60dvldf5pffma-acl-2.2.52/lib:/nix/store/gij6mgj1vixf7qcyb13h5aa5y15r2xxd-attr-2.4.47/lib:/nix/store/vn6fkjnfps37wa82ri4mwszwvnnan6sk-glibc-2.25/lib] <snip> ``` Because we don't have any caching as far as I understand, this means that when echo is run, a whole lot of fairly superfluous system calls are made: ``` $ strace -e open,stat echo hello <snip> open("/run/opengl-driver/lib/tls/x86_64/libacl.so.1", O_RDONLY|O_CLOEXEC) = -1 ENOENT (No such file or directory) stat("/run/opengl-driver/lib/tls/x86_64", 0x7ffddc7f01d0) = -1 ENOENT (No such file or directory) open("/run/opengl-driver/lib/tls/libacl.so.1", O_RDONLY|O_CLOEXEC) = -1 ENOENT (No such file or directory) stat("/run/opengl-driver/lib/tls", 0x7ffddc7f01d0) = -1 ENOENT (No such file or directory) open("/run/opengl-driver/lib/x86_64/libacl.so.1", O_RDONLY|O_CLOEXEC) = -1 ENOENT (No such file or directory) stat("/run/opengl-driver/lib/x86_64", 0x7ffddc7f01d0) = -1 ENOENT (No such file or directory) open("/run/opengl-driver/lib/libacl.so.1", O_RDONLY|O_CLOEXEC) = -1 ENOENT (No such file or directory) stat("/run/opengl-driver/lib", {st_mode=S_IFDIR|0555, st_size=4096, ...}) = 0 open("/run/opengl-driver-32/lib/tls/x86_64/libacl.so.1", O_RDONLY|O_CLOEXEC) = -1 ENOENT (No such file or directory) stat("/run/opengl-driver-32/lib/tls/x86_64", 0x7ffddc7f01d0) = -1 ENOENT (No such file or directory) open("/run/opengl-driver-32/lib/tls/libacl.so.1", O_RDONLY|O_CLOEXEC) = -1 ENOENT (No such file or directory) stat("/run/opengl-driver-32/lib/tls", 0x7ffddc7f01d0) = -1 ENOENT (No such file or directory) open("/run/opengl-driver-32/lib/x86_64/libacl.so.1", O_RDONLY|O_CLOEXEC) = -1 ENOENT (No such file or directory) stat("/run/opengl-driver-32/lib/x86_64", 0x7ffddc7f01d0) = -1 ENOENT (No such file or directory) open("/run/opengl-driver-32/lib/libacl.so.1", O_RDONLY|O_CLOEXEC) = -1 ENOENT (No such file or directory) stat("/run/opengl-driver-32/lib", {st_mode=S_IFDIR|0555, st_size=4096, ...}) = 0 open("/nix/store/v0wcqsb6vpljx13vw8q60dvldf5pffma-acl-2.2.52/lib/tls/x86_64/libacl.so.1", O_RDONLY|O_CLOEXEC) = -1 ENOENT (No such file or directory) stat("/nix/store/v0wcqsb6vpljx13vw8q60dvldf5pffma-acl-2.2.52/lib/tls/x86_64", 0x7ffddc7f01d0) = -1 ENOENT (No such file or directory) open("/nix/store/v0wcqsb6vpljx13vw8q60dvldf5pffma-acl-2.2.52/lib/tls/libacl.so.1", O_RDONLY|O_CLOEXEC) = -1 ENOENT (No such file or directory) stat("/nix/store/v0wcqsb6vpljx13vw8q60dvldf5pffma-acl-2.2.52/lib/tls", 0x7ffddc7f01d0) = -1 ENOENT (No such file or directory) open("/nix/store/v0wcqsb6vpljx13vw8q60dvldf5pffma-acl-2.2.52/lib/x86_64/libacl.so.1", O_RDONLY|O_CLOEXEC) = -1 ENOENT (No such file or directory) stat("/nix/store/v0wcqsb6vpljx13vw8q60dvldf5pffma-acl-2.2.52/lib/x86_64", 0x7ffddc7f01d0) = -1 ENOENT (No such file or directory) open("/nix/store/v0wcqsb6vpljx13vw8q60dvldf5pffma-acl-2.2.52/lib/libacl.so.1", O_RDONLY|O_CLOEXEC) = 3 open("/run/opengl-driver/lib/libattr.so.1", O_RDONLY|O_CLOEXEC) = -1 ENOENT (No such file or directory) open("/run/opengl-driver-32/lib/libattr.so.1", O_RDONLY|O_CLOEXEC) = -1 ENOENT (No such file or directory) open("/nix/store/v0wcqsb6vpljx13vw8q60dvldf5pffma-acl-2.2.52/lib/libattr.so.1", O_RDONLY|O_CLOEXEC) = -1 ENOENT (No such file or directory) open("/nix/store/gij6mgj1vixf7qcyb13h5aa5y15r2xxd-attr-2.4.47/lib/tls/x86_64/libattr.so.1", O_RDONLY|O_CLOEXEC) = -1 ENOENT (No such file or directory) stat("/nix/store/gij6mgj1vixf7qcyb13h5aa5y15r2xxd-attr-2.4.47/lib/tls/x86_64", 0x7ffddc7f01a0) = -1 ENOENT (No such file or directory) open("/nix/store/gij6mgj1vixf7qcyb13h5aa5y15r2xxd-attr-2.4.47/lib/tls/libattr.so.1", O_RDONLY|O_CLOEXEC) = -1 ENOENT (No such file or directory) stat("/nix/store/gij6mgj1vixf7qcyb13h5aa5y15r2xxd-attr-2.4.47/lib/tls", 0x7ffddc7f01a0) = -1 ENOENT (No such file or directory) open("/nix/store/gij6mgj1vixf7qcyb13h5aa5y15r2xxd-attr-2.4.47/lib/x86_64/libattr.so.1", O_RDONLY|O_CLOEXEC) = -1 ENOENT (No such file or directory) stat("/nix/store/gij6mgj1vixf7qcyb13h5aa5y15r2xxd-attr-2.4.47/lib/x86_64", 0x7ffddc7f01a0) = -1 ENOENT (No such file or directory) open("/nix/store/gij6mgj1vixf7qcyb13h5aa5y15r2xxd-attr-2.4.47/lib/libattr.so.1", O_RDONLY|O_CLOEXEC) = 3 open("/run/opengl-driver/lib/librt.so.1", O_RDONLY|O_CLOEXEC) = -1 ENOENT (No such file or directory) open("/run/opengl-driver-32/lib/librt.so.1", O_RDONLY|O_CLOEXEC) = -1 ENOENT (No such file or directory) open("/nix/store/v0wcqsb6vpljx13vw8q60dvldf5pffma-acl-2.2.52/lib/librt.so.1", O_RDONLY|O_CLOEXEC) = -1 ENOENT (No such file or directory) open("/nix/store/gij6mgj1vixf7qcyb13h5aa5y15r2xxd-attr-2.4.47/lib/librt.so.1", O_RDONLY|O_CLOEXEC) = -1 ENOENT (No such file or directory) open("/nix/store/vn6fkjnfps37wa82ri4mwszwvnnan6sk-glibc-2.25/lib/tls/x86_64/librt.so.1", O_RDONLY|O_CLOEXEC) = -1 ENOENT (No such file or directory) stat("/nix/store/vn6fkjnfps37wa82ri4mwszwvnnan6sk-glibc-2.25/lib/tls/x86_64", 0x7ffddc7f0170) = -1 ENOENT (No such file or directory) open("/nix/store/vn6fkjnfps37wa82ri4mwszwvnnan6sk-glibc-2.25/lib/tls/librt.so.1", O_RDONLY|O_CLOEXEC) = -1 ENOENT (No such file or directory) stat("/nix/store/vn6fkjnfps37wa82ri4mwszwvnnan6sk-glibc-2.25/lib/tls", 0x7ffddc7f0170) = -1 ENOENT (No such file or directory) open("/nix/store/vn6fkjnfps37wa82ri4mwszwvnnan6sk-glibc-2.25/lib/x86_64/librt.so.1", O_RDONLY|O_CLOEXEC) = -1 ENOENT (No such file or directory) stat("/nix/store/vn6fkjnfps37wa82ri4mwszwvnnan6sk-glibc-2.25/lib/x86_64", 0x7ffddc7f0170) = -1 ENOENT (No such file or directory) open("/nix/store/vn6fkjnfps37wa82ri4mwszwvnnan6sk-glibc-2.25/lib/librt.so.1", O_RDONLY|O_CLOEXEC) = 3 open("/run/opengl-driver/lib/libpthread.so.0", O_RDONLY|O_CLOEXEC) = -1 ENOENT (No such file or directory) open("/run/opengl-driver-32/lib/libpthread.so.0", O_RDONLY|O_CLOEXEC) = -1 ENOENT (No such file or directory) open("/nix/store/v0wcqsb6vpljx13vw8q60dvldf5pffma-acl-2.2.52/lib/libpthread.so.0", O_RDONLY|O_CLOEXEC) = -1 ENOENT (No such file or directory) open("/nix/store/gij6mgj1vixf7qcyb13h5aa5y15r2xxd-attr-2.4.47/lib/libpthread.so.0", O_RDONLY|O_CLOEXEC) = -1 ENOENT (No such file or directory) open("/nix/store/vn6fkjnfps37wa82ri4mwszwvnnan6sk-glibc-2.25/lib/libpthread.so.0", O_RDONLY|O_CLOEXEC) = 3 open("/run/opengl-driver/lib/libc.so.6", O_RDONLY|O_CLOEXEC) = -1 ENOENT (No such file or directory) open("/run/opengl-driver-32/lib/libc.so.6", O_RDONLY|O_CLOEXEC) = -1 ENOENT (No such file or directory) open("/nix/store/v0wcqsb6vpljx13vw8q60dvldf5pffma-acl-2.2.52/lib/libc.so.6", O_RDONLY|O_CLOEXEC) = -1 ENOENT (No such file or directory) open("/nix/store/gij6mgj1vixf7qcyb13h5aa5y15r2xxd-attr-2.4.47/lib/libc.so.6", O_RDONLY|O_CLOEXEC) = -1 ENOENT (No such file or directory) open("/nix/store/vn6fkjnfps37wa82ri4mwszwvnnan6sk-glibc-2.25/lib/libc.so.6", O_RDONLY|O_CLOEXEC) = 3 <snip> ``` This issue is more pronounced for programs that link to lots of libraries, as the linker will go through the RPATH for each library linked to. As far as I understand it, the `DT_NEEDED` entries in the dynamic relocation section could contain absolute paths, which would reduce the amount of searching necessary. Additionally, this ties the library reference more closely to the derivation's build inputs. Any thoughts? I haven't got any hard data on the performance impact this has (though the fact that most distros use the ld.so.cache mechanism to speed up loading suggests that it is relevant at least in some contexts), or any idea what other consequences of using absolute paths would be, but having executable loading be O(n²) for the number of referenced store components just somewhat bothers me. ## Technical details - System: 17.03.899.c87abfae79 (Gorilla) - Nix version: nix-env (Nix) 1.11.8 - Nixpkgs version: "17.03.899.c87abfae79"

aka the stat storm, i.e. the fact that library loading in nixos is O(n^2) vs O(1) in standard FHS distros.