Rémy Grünblatt - Nix and NixOS, my pain points


1 Like

it is very often necessary to read the source code (`.nix’ files) because there are no explanations in the manual, or the explanations are not very comprehensible.

That’s called documentation as code. At least it is always up to date and correct. :sweat_smile:
But jokes aside, documentation should be improved.

using a specific commit of Nixpkgs is not enough, it is also necessary to pay attention to the version of Nix used

Not if you are using nixos-rebuild because it uses the nix from the nixpkgs that is used to build the system. Also the nix version being used to build the system has next to no impact on the final result. Most bugs that occur are on the way to the final result and impact the doability of that but not the outcome.

two machines that have the same configuration files (/etc/nixos/configuration.nix)

Yes, obviously. You need to store the data to configure your system somewhere. In comparison to other distros it is in one place and not all over your system.

can differ widely due to dependency on other configuration files such as ~/.config/nixpkgs/config.nix, the ability to change Nix’s behaviour using environment variables, command line arguments or even due to using different versions of Nix.

No, not really. This mostly affects evaluation, binary caches, builders configuration, how to obtain sources and permissions, almost no setting has an effect on the output of a build. For example if you don’t set config.allowUnfree then your system won’t eval if you use unfree packages or removing a binary cache can forces you to build a package.
For NixOS systems this should be set in the configuration.nix. The config.nix is mostly useful for user settings and for systems that are using nix on some other distro.
You can create side effects based on envs but in nixpkgs they mostly only effect evaluation and can’t by default change the final output. This is fixed with flakes forbidding access to envs by default.

Guix has a better approach in my opinion, a single repository being used for the package manager and the package “recipes”.

That makes sense for Guix because everything is scheme. Mixing nixpkgs and nixos written in nix with nix written in c++ will only create a bigger mess.

The reproducibility of NixOS / Nix is also only assured if the sources used remain truly accessible. A recent example underlines the fragility of these sources, and the fragility of Nix which does not exploit solutions such as Software Heritage (where Guix does for at least three years).

There was effort in the past to archive many source downloads from nixpkgs Long-term reproducibility with Nix and Software Heritage .
Also the issue underlines how easy it is to fix cases where cache.nixos.org is not enough by acquiring a copy of the file and adding it back to the nix store.

This means, in a scientific context for example, the user, that is to say me, still has to care for source preservation if they wants to be sure five or ten years after they developed their code and used nix, it still runs.

Yes, of course. If you want perfect reproducibility even in 10 or 20 years you should create a bundle with nix bundle, build a docker image/VM/iso/etc. or export all relevant store path entries.

One of the problems with Nix / NixOS / Nixpkgs regarding the backup of these sources is simply the discovery of the sources to archive. Indeed, it is not possible to easily extract the set of dependencies (e.g. URLs used in the recipes of Nixpkgs packages) to archive them

I am going to make a bold claim: It is easier with NixOS than with any other distro. I can even program that and could make it generic to any nixos configuration even if it is going to take a while to be evaluated.

as a certain number of these URLs are only “computed” at evaluation time, which requires to evaluate the whole Nixpkgs set, in all its possible configurations, to make sure that one is actually archiving what needs to be archived

This sounds like nixpkgs can generate an infinite or unpredictable amount of source urls. This is not the case. If you evaluate nixpkgs with unfree packages and for each platform then you will get for the same result every time for the same commit.
Some parts of the URLs are substituted to make updates or multiple versions easier, so you can’t simply grep for them but nix can still generate them fully reproducible.

And that’s without even mentioning Fixed Output Derivation.

For short FODs which reference a store entry that could be generated by a fetcher like fetchurl. This makes sure that download servers are mirrored and cached to improve reproducibility in case a link no longer works.

This binary reproducibility of Nix / NixOS / Nixpkgs is indeed not really tested, at least not systematically.

As the author later points out it is on r13y.com for NixOS iso which only recently achieved 100% reproducibility which then dropped again because python had (again) a bug that their bytecode is not fully reproducible. I wonder how Debian achieves higher percents but maybe they are just updating so slowly that the bugs where already reported and fixed by other people.

but this is a check on a tiny percentage, less than 2%, of all packages

That would be a next step, to expand r13y.com to more and more packages but since achieving the last percents of reproducibility are always really tricky this is going to take a bit of time and the effort of some dedicated people.

Again, Guix is much better with stats on all packages in the repository.

Couldn’t checkout their website because it 504’ed on me.

And even more classical distributions such as Archlinux or Debian are better at this than Nix / NixOS / Nixpkgs !

I would be interested in knowing the numbers if the AUR was counted into that because only 11000 packages where checked, 1/8th of nixpkgs.

For example, for postgresql, the package version depends on system.stateVersion, unless overwritten manually :

Which is a very good thing! This way we make sure that the postgres package is up to date for new installs and does not break for existing. On other distros you might break your database on an upgrade with no easy way to fix it.

especially when the Github Search is broken

Yes, I am annoyed by this too in the past but thats on GitHub, not us but the new github code search works better. A better way would be to download the nixpkgs repository and manually check it with ripgrep.

which happens quite often

I doubt that. nixpkgs has the biggest package collection available right now. There is no other distro with more packages.

we have only one choice

That is wrong. The are a lot of tools to patch precompiled binaries, there are fhsenvs to run especially closed source binaries in a more classic file layout and you can always fall back to containers if all string break.

It is generally not possible to take a shortcut and use precompiled binaries

It is with at least 4 tools: autopatchelfHook, nix-ld, nix-autobahn, nix-alien
and with fhsenvs.

you can’t really mix them together in a single project and expect to have a good user experience.

You can still use tools like pip, npm and so on, just not apt, rpm, etc but thats also true for any other package manager.

Hosted by Flying Circus.