Hi! I’ve been trying to get familiar with NixOS in a VM for the past week or so, trying to re-create my Arch setup in a declarative way so I never have to manually set it up again.
Anyway my question is, in the Nix execution process (I’m using flakes by the way), which parts of the overall process get executed when? e.g. I was somewhat confused by what’s the difference (from a purely declaration/execution standpoint) between:
environment.systemPackages = [ pkgs.nginx ];
# and
services.nginx.enable = true;
Because the first one requires pkgs to exist, I’m explicitly referencing a package that I’m adding to a list that I presume a further stage iterates through to build the system.
But the second line doesn’t explicitly reference anything. It just sets an arbitrary value, which makes me assume that there’s a further stage that is looking for these values to set a configuration if it’s enabled.
From what I’ve gathered, the stages I’ve found so far are:
Flake input execution, to grab all the inputs
nixpkgs package list execution (triggered by nixpkgs.lib I assume), which collects all available package derivations and provides them under pkgs.
Configuration execution, e.g. configuration.nix that the flake references, which collects all the specified properties, e.g. the environment.systemPackages, but also any properties for future stages
Some further nixpkgs post-processing stages that do things like defining services based on properties, taking various options in the flake’s config and flattening them out into lower level constructs, e.g. more environment.systemPackages and systemd service declarations
Maybe some other stages?
Handing the resulting config over to the system builder
Am I understanding this right? How much is missing from here? I’m trying to fully get the hang of the whole process, as I plan to get really deep into nix, because I see a lot of great use in it.
Also just to note, I’m familiar with the concept of lazy evaluation, and I’ve played with Haskell before.
This is a decent place to start with the NixOS module system. I’d say you have some slight mis-assumptions/conceptions about how it works, but I suspect you’ll have a decent idea if you skim through this: NixOS modules - NixOS Wiki
This is sort of where you’re losing the thread. @colemickens is exactly right; from this point on, it’s basically just the module system, with the final step being running the switch-to-configuration script that it produces. The reason you don’t have to reference pkgs with services.nginx.enable = true; is simply because another module does that for you.
it’s all just data: environment and services and hardware and the rest are just attrs of config. The module system, as noted, provides a way of stacking these together to merge individual scopes into a larger one.
nix is lazy; lots of the data is, initially, unresolved or partially-resolved functions. The stages you talk about don’t exist in any particularly explicit fashion, they come about more because something wants a concrete result from the evaluation of the config. It’s not the greatest analogy, but it’s more of a “pull” style of execution on the giant stack of modules, where lots of it goes unused unless actually evaluated.
there’s an evaluation stage which includes loading and parsing all the nix code and your settings to realise all the derivations, a build stage where they get built (or substituted from cache) all as a dependency of the top-level drv, and then an (optional, maybe later) activation stage which mostly shuffles a bunch of symlinks and maybe updates a bootloader or restarts some services and doesn’t involve nix at all really.
Edit: oh, for a different perspective, there’s also a question about when all the various fragments of script (typically bash) that you see assembled by and embedded in the nix code run:
none during evaluation (IFD aside, but even then, ehhh… semantics)
some of it during build, all the build and test phase hooks for example
bits of it during activation, if specific hooks are used
more of it as systemd units start scripts and wrappers and similar, as part of the delivered closure. More stuff is moving out of activation to here as time goes on, too.
This will probably be completely unreadable to you for now though.
The big takeaways for me were that NixOS is not an inherent part of what nix does, but more like a framework for system config that happens to be written in nix. The entire nix build is effectively just building a giant, super-modular bash script, the modules are settings for the bash script generator, and “activation” (i.e. the thing that makes nixos-rebuild switch different from nixos-rebuild build) is just running said bash script to make changes on disk. Also said script runs every time your system boots, which is where the reproducibility comes from. Demystifying all of thay really made things click for me.
NixOS is honestly a far more complex beast than nix itself.
The module system in turn is just layers upon layers of abstraction that eventually go to more primitive things like environment.systemPackages and environment.etc, as you mention, and are then turned into little snippets in the bash script.
The reason using something like services.nginx.enable is not quite the same is that modules like it usually use many of those underlying primitives rather than just one - but it all ultimately boils down to just using some primitive options that are directly turned into snippets of bash and eventually files in /nix/store during the build.
In particular, a lot of it ends up in system.activationScripts, understanding how things get there will make the whole process make more sense.
@TLATER I’ve been reading the Nix pills, and what you said actually explains quite a lot, I was confused with how much bash plays a role in Nix but it makes sense now
I will try to rephrase what the NixOS module system is doing with your expectation in mind.
The NixOS module system is using a fix-point, a fix-point is basically an infinite loop which stops when nothing changes anymore. The number of stages in NixOS is basically the minimal number of stages needed for computing the configuration.
One way to think of it is that you have a set where all options are set to “undefined”, and at each stage, you compute the options where you have all the inputs to gather the result. When you toggle the services.nginx.enable, this value is a dependency of the systemd.services.nginx, which would be resolved at the next iteration, which would register the nginx service in systemd, which will then resolve the system which expect systemd to be fully configured.
While the previous paragraph is one way to get a good feeling of how fix-points are working, in a lazy language, what happens is the same process but in reverse. Stages are evaluated by the requirement to have them evaluated. The system requires systemd to be fully configured, to resolve systemd configuration we need nginx service configuration, which then resolve to the option you have defined using services.nginx.enable.
In practice it does not matter whether you think of it in a forward or backward way of execution, until you want to do crazy things such as peeling the fix-point last stage … So pick the one you feel the most comfortable with.