Optimizing patchShebangs (and other stdenv phases/hooks)

Development Nixpkgs Architecture
1

Quick disclaimer:

  • Yes, swapping a POSIX function with a compiled binary has a lot of complications
  • This discussion maybe already happened, but I can’t find it if it did.

I was digging through the internals of mkDerivation and realized how massive it is, both in terms of nix code and shell code.

For bootstrapping, yes, shell code in mkDerivation probably needs to remain as shell code (POSIX is the one true foundation). But, surely, for 95% of packages, a compiled version of patchShebangs, one that does parallelized edits, could be a dramatic performance improvement in runtime.

I know the logisitics of swapping out a “bottom brick” are insanely tough, but in concept, could I make a fastMkDerivation that uses a compiled version of patchShebangs? Is something like that already happening in nixpkgs bootstrapping and I just don’t know it? Any info appreciated.

6 Likes
2

Generally, before starting such an optimizing endeavor, I’d recommend trying to at least roughly measure how much time we spend with patchShebangs. (though I understand it’s hard, as it can vary a lot, especially by derivation)

2 Likes