Need a good example for specifying versions in nixpkgs

Development
21

I don’t want to discourage you from using Nix, but can’t you just build a small CentOS (or whatever base system) chroot environment from a set of RPMs with fixed versions and some automated way of bootstrapping the chroot using these RPMs?

22

Force version eh… I’ll try it. Thanks.

The idea is to produce a build system that can produce completely auditable binaries.

nix expressions are auditable. RPMs and Debs lose auditability. There is no way to know for sure what the process or dependents of a deb are, or even what source and tools produced it. The best folks are able to come up with are outboard tools that have a correspondence, probably, but not a provable one.

23

Well, given that you want to look carefully at every dependency anyway, you can also just fork Nixpkgs and replace each expression with its copy from a commit having your preferred version.

It sounds like your preferred versions do not change quickly; and a single rebuild of a pretty large system is still OK-ish on a single powerful machine, even without a buildfarm.

To be fair, you can also take specs/SRPMs and rebuild DEBs/RPMs from there.

24

But without complete reproducibility, you have no way of knowing whether that particular package is derived from that source. With deb/debs you have two unrelated files that have similar file names. No way at all of knowing the deb is related to that debs except faith.

In a high security (hence reproducible and auditable) environment, faith and trust should be kept to a minimum. In nix, the expression is reproducible, and that can be verified with a command line option

 nix-build --substituters ''

should produce the correct deterministic result.

25

So, from what I can tell, patches isn’t an attribute. dontPatch is, and I suspect I can override the entire patchPhase but I that is profoundly large piece of code and I think the result would be fragile.

FWIW, override glibc’s version and the result ripples through every package in the repo. The build time exceeds 14 hours (so far) on a 12 core threadripper with ssd. So IMO we can call that impractical.

26

No, Nix is not deterministic yet in that sense. Most packages in the base system are because of the hard work of people who care about reproducibility. But many packages in nixpkgs are (probably) non-deterministic and will not result in bitwise identical outputs.

27

The packages that aren’t wont. But in the case of deb and rpm, none are.

28

The build time for what? And with what parallelism settings? Because your audit requirements sounded like you have a relatively restricted set of dependencies.

29

I thought Debian reached reproducibility from spec for a nontrivial subset of packages?

30

While the binaries may or may not be reproducible, the nature of the package process for debs is non-reproducible. (I can get into details, but you can’t get there from here for the deb itself. ).

That said, the clearest issue is auditability. There is nothing in the package process that allows you to prove the debs and the deb are in anyway related, and in fact in reality the debs and deb are often different in source or patch level. They certainly are different in build environment and tools.

31

Yea, glibc is essentially the “most fundamental” package in most linux systems… it will likely cause a rebuild of every other package.

Not sure how you’re getting 14 hours, I’m usually able to build ~40k packages when reviewing staging PRs in that amount of time.

$ time nix-build -A clang_7.cc --cores 128 --check
...
/nix/store/dy5h1ypzpdiyhiyf4qvvyb5fj5bbv87l-clang-7.1.0

real	2m6.461s
user	0m0.270s
sys	0m0.152s

although, a few packages like gcc, glibc, and chromium usually have very long sections which are single threaded (patchShebangs, linking, autoreconf, etc) which give you pretty diminished returns on parallel jobs.

what package are you trying to build?

32

FYI: For producing .rpm or .deb packages using nix, there is some infrastructure in nixpkgs to reproducibly spin up other Linux distros with certain packages installed and then build their native packages:
See e.g.

This way you can make use of the nice reproducibility guarantees that nix the tool gives you, while making sure that your library dependencies will match your target system.

33

https://openenclave/openenclave.git

34

it says the link is dead

35

Sorry

1 Like