When any of those reverse dependencies rebuild, they will have a reference to the new libinput’s path in them. It can produce an output that’s almost bit identical, except the path to libinput in the nix store in it somewhere is different, so it has a different hash and has been rebuilt. So now qtbase5 has a different hash and everything dependent on it still needs rebuilt, and this will continue all the way through the reverse deps. qtbase5 has 29 things referring to it, each of which will end up with a different hash even if all their content except for references into the nix store is identical.
When dependent packages get the same ca-hash, all their dependencies don’t need to be rebuilt though
I don’t see how ca derivations can ever stop this chain of reverse dependencies from needing fully rebuilt in the current implementation. If one dep at the start has a single bit different its hash will be different, and then anything that references it will have a different hash, and this will repeat all the way through the chain of dependencies.
I thought about this a lot for RFC 17 and in the end I decided it is impossible to generally shortcut these kinds of references, since a build can always decide to copy things from a dependency’s references. For example a static build.
However, we can make a store that de-duplicates content by patching out store paths before saving.
The problem could be solved at the nixpkgs level by having C(++) style derivations only take their dependencies’ headers as inputs, producing shared libraries that don’t actually use absolute paths to refer to other libraries, and having separate derivations up the chain take on the responsibility of linking together shared libraries correctly. This has a good number of drawbacks, so this is not a serious suggestion, but it would have the side benefit of allowing dependent packages to be built in parallel if they could output their headers in separate derivations.
Not too bad I’d say! It makes sense that substitution introduces quite a bit more overhead for CA drvs since they both induce more requests in total and they are interleaved with building.
A similar suggestion that should be compatible with more existing build scripts is to first transform input shared objects into stub libraries that contain nothing but the symbol table. The stub references could then be replaced with the actual libraries in a second derivation, which would be cheap to rebuild. Apparently LLVM now has a tool llvm-ifs that can produce ELF stub libraries as well as Apple .tbd stub files.
Well in the case of a compiler being rebuilt, if the new compiler binary has a different hash and therefore store path, it may still (and hopefully will be) functionally equivalent to the old one. So everything that uses that compiler directly will be rebuilt, but not anything depending on those, because these are then bit for bit identical.
Not sure how common a situation this is though, because most compilers will have a huge amount of direct reverse dependencies…
I think the real issue with the compiler “use case”, is that most programs that transitively depend on a compiler most likely also directly depend on it.
My current large problem with content-addressed derivations is how Nix itself mixes in variability indirectly with the hash input.
My expectations go like this: I compile GNU Make, Nix gives it an input-addressed hash Ai as $out. GNU Make hardcodes Ai/lib inside the binary as a constant string because GNU tools are insane. Then Nix comes and rewrites it to a content-addressed Ac/lib. If I change something insignificant in the buildscript, the output stays the same, gets rewritten to Ac/lib, rebuld gets cut short, the day is saved.
Reality is much more disappointing. When I recompile GNU Make with my insiginificant change. Nix gives it an input hash Bi, GNU Make hardcodes Bi/lib, but now the linker decides to place it at some different position in the binary’s rodata because… Ai and Bi were different, I suppose? And if they’re placed differently then no rewriting will save me, one new different Bc incoming. And here I am, recompiling all GNU Make reverse builddeps like in the old input-addressed days =(
Very interesting problem, so the linker is applying some sort of sorting?
Can we give the build always the same path, same length as the input hash but not available outside the sandbox?
Then path rewriting must succeed for builds to work, and since all the inputs are CA, all paths remain unchanged if you’re just compiling a small change.
Also, there’s now a --shrinkwrap option to patchelf, which might help and probably should be used in the CA builds?
It gets all the transitive dependencies of a binary (so including those of libraries that are used) and pins them all with their full path into the binary’s ELF header. Speeds up loading but might also solve that linker order?
Can we give the build always the same path, same length as the input hash but not available outside the sandbox?
Sure, but that’d break non-sandboxed builds. =)
–shrinkwrap
Not related to the string constants in rodata.
I think this is something to solve on the linker or autotools or project patch level, not in Nix per se. Builds with different prefix shouldn’t differ that much. If only I knew what determines the rodata string ordering, I need to look deeper into that.
Actually, would it? We could make the path for a build be /nix/store/0000000...000-name-version and then it would only be a problem if 2 builds were making the same name-version at the same time.
applications hardcode self-references into binaries as constant strings
to get stable results, hash rewriting in content-addressed derivation requires the linker to maintain a stable order of strings even in the face of $out hashes changing
the worlds wants new fast parallel linkers like lld or mold that will shard the work and process strings in parallel.
even if they merge it back in a stable order based, say, on a hash of a string, the order is going to be different for different $out values, so we’re screwed
(most depressing to me) even if we come to all the linker writers and they listen to us… what will we say? please maintain a stable order based on… what exactly?
This is gonna bite us and I’d really like to hear we have a way out.
EDIT: and I’m afraid it’s patching all linkers for all languages to “link slow” =(