The reproducibility thing is a result of unfortunate marketing. Perhaps a better claim to make would be “Nix makes it easy to create reproducible builds”.
The strongest claim I want to make about how Nix works is:
Nix does the opposite of what you would do if you were deliberately trying to muck things up.
I dunno I feel like it’s pretty easy to do better than that. “Nix prioritizes reproducibility,” or “The Nix model is very conducive to reproducibility.” Or even change tack and talk about hermeticity instead: “The Nix build sandbox provides strong hermeticity guarantees.”
Also worth noting nixos hosts a web page talking about nix reproducibility https://reproducible.nixos.org
I appreciate the message behind this post (and similar posts, like NixOS is not reproducible), but I always end up asking myself the question: do people really think Nix can always guarantee reproducibility?
Prior to getting into it, most of the marketing around Nix(OS) felt pretty tame to me. I didn’t expect it to be a magic tool that’s able to remove nondeterminism from every build or download resources from dead links 20 years from now, only that it would have a lot of guarantees and tools for helping with them. More technical pages like Explore | Nix & NixOS also don’t really make any claims close to this
But to be fair, I did have a bit of background in packaging and reproducible builds already, so maybe that context helped me. I think the introduction paragraph of the reproducibility page linked above sums it up very well
When compiling from the same source on independent infrastructure yields bit-by-bit identical results, this gives confidence that the build infrastructure was not compromised and the artifact really does correspond to the source.
Because a Nix derivation has deterministic references to all of its dependencies, and the build happens in a sandbox, Nix is an excellent starting point for achieving Reproducible Builds.
However, this alone is not sufficient: builds may still leak timestamps or have other nondeterminisms. […]
It’s a shame it’s relegated to this somewhat obscure page, since it could help clear up this (what I hope to be small) issue
I would also avoid more technical jargon like this. Sticking with the “reproducible builds” term is something that many more people are familiar with, while getting the same basic point across. Talking about things like “hermeticity” and “evaluation purity” quickly gets confusing and kinda overwhelming outside of technical resources that can go much more in-depth into what they mean (or have a glossary, like the Nix manual :p)
Yeah, they really do, and they think it means that deployments, including all state can be repeated across systems, because a Youtuber told them nix was magic. Which makes sense, because frankly, most people (including most nixpkgs contributors) outside of say, research-related or security-conscious fields, don’t really care about bit-for-bit reproducibility nor find it useful. (I’d even say it’s not a priority for NixOS in general, since even our minimal ISO has mostly been non-reproducible over the last 2-3 years at least.)
By the way, there was also a relevant discussion last year about similar ideas:
Nix is “replicatible” is a better distinction if we are following ACM
Repeatability (Same team, same experimental setup)
The measurement can be obtained with stated precision by the same team using the same measurement procedure, the same measuring system, under the same operating conditions, in the same location on multiple trials. For computational experiments, this means that a researcher can reliably repeat her own computation.
Reproducibility (Different team, different experimental setup )*
The measurement can be obtained with stated precision by a different team using the same measurement procedure, the same measuring system, under the same operating conditions, in the same or a different location on multiple trials. For computational experiments, this means that an independent group can obtain the same result using the author's own artifacts.
Replicability (Different team, same experimental setup )*
The measurement can be obtained with stated precision by a different team, a different measuring system, in a different location on multiple trials. For computational experiments, this means that an independent group can obtain the same result using artifacts which they develop completely independently.
When I was first learning about Nix as a build system, I assumed that all derivations would be fixed-output.
I can see how some would be surprised to learn that Nix output isn’t guaranteed to always be reproducible.
So unless everything is made purely functional(all possible tools from scratch which have side effects like relying on timestamp, etc maybe for randomising output),
the side effects betray at some point and we won’t achieve reproducible builds?