A build is reproducible if given the same source code, build environment and build instructions, any party can recreate bit-by-bit identical copies of all specified artifacts.
I feel for the author when someone dismisses their work with âhey, didnât NixOS fix this a decade ago?â, but if we apply the strict definition of reproducibility then no distribution is reproducibile, and probably never will be.
What I mean is: even if at some point we hit the 100% reproducibility mark it still doesnât mean much: reproducibile according to what kind of testing? What If I inject randomness in the filesystem timestamps, user ids, build on different machines, is it still reproducible?
Even in the fantastically unlikely event of someone formally proving the entire source-to-build artifacts chain is a mathematical function of its millions(?) of inputs, thereâs literally no guarantee that in the next second an update is pushed breaking the reproducibility of a package.
Also, achieving and maintaining reproducibility is often in the hands of the developers of the compilers, build tools and the people using those tools: if a compiler orders outputs in an unpredictable manner, thereâs not much Arch or NixOS maintainers can do about it.
The author of this article has been on this hobby horse for years. I thought maybe the article would be interesting based on its edgy title, but less than halfway through it became clear who was writing and what the rest of the article would be. Frankly, itâs boring.
The truth is that Nix environments are reproducible in ways that tools for which the weaker designation ârepeatableâ is commonly used (e.g., Docker) are not. And in the same way, NixOS systems are reproducible in a way that conventional distros like Arch never have been; running a NixOS system as it existed 5 years ago is trivial and running an Arch system as it existed 5 years ago is virtually impossible. Running a friendâs NixOS system as a VM is a trivial reproduction-with-a-twist and running a friendâs Arch system as a VM is a cumbersome, manual clone-and-repair job at best. Changing the language here effaces more differences than it honors.
Unfortunately for the author, Nixâs form of reproducibility (âfunctional reproducibilityâ, âbehavioral reproducibilityâ, or âbuild environment reproducibilityâ) is just more exciting, interesting, and accessible to people than the narrowly focused âI can get the exact same bits if I run this build twice right now (even if I probably canât do it next month because I may or may not have the resources to reproduce the build environment by then)â notion of reproducibility that heâs interested in for supply chain security/integrity reasons. The kind of reproducibility that people are going to be excited about at Linux conferences and across the web is, predictably, the one that lets them do cool stuff.
The only distro that has really turned bit-for-bit reproducibility into a thing normal users can easily and conveniently do, afaik, is Guix, with guix challenge. Maybe if that changes, the kind of broad reproducibility Nix and NixOS offer will no longer overshadow the younger, narrower notion of reproducibility meant by the phrase âReproducible Buildsâ in the work of reproducible-builds dot org. That would be a cool Nix feature. The website could also probably use a section on kinds of reproducibility, how they work together and reinforce each other, what other projects are doing to advance them, how Nix is different from those projects, and how Nix benefits from other projectsâ reproducible builds efforts. There are other potential âaction itemsâ for the Nix community in measuring, tracking, and improving our bit-for-bit build reproducibility, too, as zimbatm points out in the Lobsters thread.
But ultimately I donât think anyone should worry about the clickbait title, or that Foxboronâs feelings about enthusiastic conference-goers mistaking him for a fellow Nixer is the Nix communityâs problem to solve.
Props to Foxboron for filing a PR against the NixOS website to get the changes he wanted, though, instead of just griping about it. Heâs obviously quite right that reproducible builds is an ongoing project and not a solved problem. Thatâs a message worth repeating.
Practical reproducibility is probably a term that helps to clearify the situation.
I do care about a certain kind of reproducibility, not about the complete fulfillment of a specificiation, that covers all theoretically imaginable usecases of the universe.
Even though nobody really uses all of that.
Arch is rebuilding distributed packages, and we have reproducible packages across several rebuilders.
Itâs interesting to know what exactly does it mean. Do they just make sure they have identical builders that output the same result? Or do they specifically make sure they have different builders that output the same result?
Checkout the section of âRebuilder Instanceâ in the ArchWiki, I believe thatâs used for the Arch Linux Reproducible Status site.
To me it seems that it is not something widely available. And more importantly awkward to use since pacman is not source-based.
This tool actually looks like it lets you do something similar to guix challenge with local builds. (Last commit is by the author of the OP.) Looks like it may be easier to reproduce a packageâs build environment on Arch than I realized. 
@L-asâs work on WASM derivations is neet be cause it points to how we could get correct-by-construction reproducibility in the authorâs sense of the word.
If I re-watch their talk will that be clear to me? I kinda skimmed it while I was working the other day and while the talk was exciting, that didnât come through for me. Alternatively, would you mind explaining that a bit?
Thanks for the feedback.
The execution of WASM code is tightly defined by a spec,
and has only one correct interpretation.
This means, modulo bugs, given the same input (derivation), you always get the same output (build result).
This is obviously quite useful, of course itâs also even more work to get it working.
Sounds like I should do more than âkinda skimâ it! 
Thanks for your patience here.