Another one that I think you might be missing in there (I’m not sure it’s covered by any of your other categories): the Nix build sandbox is leaky. sysconf(_SC_PAGESIZE) for example can be different from system to system, and that’s not covered by any of the Nix system attributes.
Perhaps Nix could make a new type of alternative build for producing randomness so that this issue could be compartmentalised, but it is not clear if that would be an effective solution to any real problem.
scss has a “random” function that outputs a specific value in the resulting CSS that will be different every time it builds. Seems like a silly feature, but I’ve used it for making animations feel right without having to manually write out a bunch of numbers.
This tripped up my nix builds before I figured out how the function actually works (yeah, I know, but I figured maybe CSS with all its new math had a random nowadays). Forcing a fixed seed into /dev/random & co could be useful for weird tools like that (perhaps even through a different builder so people don’t accidentally build fixed-seed secrets), though I’m not sure this is possible?
This is great! Thanks for writing this down!
Another one that I think is worth mentioning is reproducibility when running resulting binaries. A lot of people probably think that dynamic libraries being absolute nix store paths is a property of Nix, when really Nix doesn’t guarantee anything like that. Instead it’s all Nixpkgs machinery that makes it work this way in at least most cases.
The strongest claim I want to make about how Nix works is:
Nix does the opposite of what you would do if you were deliberately trying to fuck things up.
In other words: Nix is the best chance I have to do builds in sane way.
I think about it as something like:
Nix isn’t magic (but it’s understandable where the perception comes from). It helps us build ever-expanding toolkits for solving packaging problems (and incentivizes doing so), but someone still has to run into those packaging problems, recognize them as such, run them to ground, figure out what to do about them, and write tooling for doing it.
When we’re lucky, the problem/solution are a good fit for near-global ~automagic fixes, and Nix ends up looking a little more like magic if you don’t need to peek under the hood. When we’re less lucky, the problem can have a simple local fix (a discovery problem, since individual packagers still have to recognize the problem and know about the solution–usually they’ll need to get bit at least once first).
One (somewhat untapped?) advantage I think we have is that (with enough computational resources) Nix and Nixpkgs can be a really big lever for experimenting on a massive package set. While working on resholve I came to suspect that we could be using this potential to trawl the package-set for issues with source/binary analysis and build some mechanism for collecting/surfacing those. The scale we could do this at should make it ~easy to iterate on heuristics (relative to developing and validating the heuristics by manually hunting for packages with similar issues).
(This isn’t a pure hypothetical. I haven’t had as much time/energy to work on it as I’d like, but I already do a weak/limited/local form of this in nixpkgs for resholve’s narrow purposes. I also used an approach similar to what I’m describing to iterate on the initial set of heuristics.)
Compiler flags such as -march=native or mtune=native also lead to non reproducible results. The compiler will pick optimizations based on the machine that builds the package.
I think the most irritating point is the “Resource no longer available” one. It can be addressed and we should be addressing it. I remember when a derivation I am maintaining just turned unbuildable because the download URL of a dependency was changed upstream.
As for the sources of randomness, well, I think there is little we can do that is also not very dangerous.
The non-determinism of concurrent builds is an issue with concurrent build systems themselves, therefore something that should be addressed outside Nixpkgs and in the respective upstream projects. Unless we decide to re-link everything deterministically with Nix, but I don’t think that’s universally applicable.
Can we dream of a future where all architectural characteristics are parameterized and stored in a persistent database in Nixpkgs? Things like CPU model and features. It hurts me to know that some software cannot be optimized reasonably well just because of reproducibility issues.
There is a way to handle optimizations for packages that use build-time optimizations and produce more reproducible packages.
stdenv.hostPlatform has CPU capability flags, such as avxSupport. We use that in
our overlay in combination with hostPlatfrom flags and an optimized stdenv to compile optimized packages. This solution could certainly be improved, but it works for now. The most tedious part is that one has find all the hidden -march/tune=native statements and remove them via patches.
See for example the ucc package
I wonder if it would be possible to set up https://www.softwareheritage.org/ as a mirror of some kind.