Making devenv start fast, and the whole nixpkgs with it

Announcements
1

devenv hook-should-activate runs on every shell prompt and takes 70ms doing almost nothing.

That cost isn’t devenv’s, it’s nixpkgs’. Every program pays it: Nix gives each package its own store path and records a DT_RUNPATH with one dir per dependency, so the loader makes hundreds of failing openat() calls hunting for libraries. devenv version does ~486, imagemagick ~1225. It’s the decade old “stat storm.”

We compared five fixes at Tacosprint, each trading off LD_LIBRARY_PATH overrides, duplicate sonames, cross compilation, and disk:

  1. Absolute DT_NEEDED paths: fastest, but loses LD_LIBRARY_PATH and the GPU driver swap.
  2. RUNPATH symlink farm: keeps overrides, can’t handle duplicate sonames.
  3. Per DSO ELF note cache: preserves everything, but needs a glibc patch.
  4. Guix style ld.so.cache: proven, breaks cross compilation.
  5. Static linking (musl): we spiked it, 70ms to 16ms, but only works for a self contained CLI like devenv.

We landed on the ELF note cache, revived after a decade in limbo:

  • patchelf --build-resolution-cache shipped in patchelf 0.19.0
  • nixpkgs#535735 turns it on across the whole package set

Longer term, we want to upstream the loader patch into glibc so every store based package manager, Guix included, benefits.

Full post: Making devenv start fast, and the whole nixpkgs with it - devenv

Umbrella issue: Address /nix/store recursive lookup/syscall overhead · Issue #481620 · NixOS/nixpkgs · GitHub

25 Likes
2

As this was posted as an announcement, I am wondering what the state of this is? The linked blog post mentions the release of patchelf 0.19. Will it be part of NixOS once it reaches unstable (currently it is on 0.15.2). What about the needed glibc patch?

Can anyone clarify?

3 Likes
3

As I was reading the list of options and trade-offs I told myself “I hope they chose the per ELF note cache option” and was thrilled to hear that that was the case. I don’t know anything about this, but it sounds like you made the right choice. Seems like taco-driven-development works! Thank you for this significant contribution.

This seems like the PR to follow:

6 Likes