How is nixpkgs and the NixOS module system avoiding infinite recursion? (With specific code excerpt)

Help
1

I’m trying to understand the module system a bit better and digging into the code starting from the entry point at <nixpkgs/nixos/default.nix>, which then calls into <nixpkg/nixos/lib/eval-config>.

While trying to understand that I’ve come across numerous instances of what look like infinite recursion, but presumably are not because NixOS does in fact produce a working system. An example from nixpkgs/nixos/lib/eval-config.nix at 831e9487a51800dade243b7b08533a6202be1374 · NixOS/nixpkgs · GitHub

# excerpted from a let statement, for full context use the github link above
  noUserModules = evalModulesMinimal ({
    inherit prefix specialArgs;
    modules =
      baseModules
      ++ extraModules
      ++ [
        pkgsModule
        modulesModule
      ];
  });


  # Extra arguments that are useful for constructing a similar configuration.
  modulesModule = {
    config = {
      _module.args = {
        inherit
          noUserModules
          baseModules
          extraModules
          modules
          ;
      };
    };
  };

In this file, I think we are bootstraping some module lists which will be loaded as part of the config. Where I am very confused is that modulesModule seems to be defined in terms of inheriting noUserModules, but noUserModules is defined by calling evalModulesMinimal with a module list that was derived in part from modulesModule, thus seeming to create a circular dependency.

Why does this work? What bit of nix’s evaluation am I missing that lets this work? Is there a minimal bit of code I could put in a nix repl to understand this? Trying something like let x=z; z=x; in x does yell about infinte recursion. I’m at a loss. There are bigger questions that I’m grappling with, like how NixOS modules can rely on a fully consistent config argument, when the config is based on evaluating those same Nixos modules. I’m sure that lazy eval is involved in some way, but I’ve spent closing in on double digit hours thinking about this, reading docs and diving into the code and I’m stuck.

1 Like
2

This is also something I struggle with to, but what helped me get a better understanding was this video: https://www.youtube.com/watch?v=cZjOzOHb2ow Maybe it could also help you.

1 Like
3

Thanks so much for this link! I will definitely need some more days to feel like I really understand this, as the pace of this video definitely assumes a set of background knowledge that is more firmly integrated and rooted than the fuzzy understanding I currently have. However, your link is exactly the capstone for what I am trying to understand, so another “thank you” is warranted.

Lazy evaluation and functional programming are definitely breaking my brain. I am documenting the most important links as I build my understanding below. There is a lot of lambda calculus theoretical underpinning, but I’m trying to focus my links below on things that discuss things from within the context of nix itself.

  • Nix Evaluation Performance - NixOS Wiki - For a worked example of thunks and when they are created (i.e. some details on how laziness gets implemented)
  • Understanding Nix’s lib.fix - akavel - For a worked example of lib.fix that particularly shows how foo.bar only evaluates foo sufficiently to identify whether it has a child bar before evaluating bar, and why this aspect of laziness is critical to the implementation of lib.fix working correctly
    • one thing that confused me initially – lib.fix does not solve for the fixed point x satisfying the equation [math] f(x) = x. Rather, it evaluates x through successive substitution (beta-reduction). If that evaluation terminates successfully, then the value of x so determined will be a fixed point because of the definition [nixlang] x = f x
  • Tobi’s blog - Nix overlay evaluation example - for a worked example of how overlays can effectively “look into the future”, referencing variables and definitions from the “final object”.
  • The Nix Hour #19 [module system recursion, config vs config, common infinite recursion causes] - your link, showing how these come together to also create a toy evalModules that can evaluate Nixos modules (configuration.nix is one such NixOS module)

Hopefully this list will be helpful to someone else in the future, but minimally putting it all together in one place makes it easier for me to find in the future haha.

4

Simply laziness. The reference back to noUserModules is inside _module.args, so it only gets evaluated if/when a particular module actually uses the value, at which point the module list is determined. It can cause infrec if the other modules do the wrong thing, (such as using something extracted from the noUserModules arg to determine that same value), but it’s not inherently a problem, any more than modules accessing the config argument is.

Something like this, maybe?

nix-repl> let x = { foo = 1; inherit y; }; y = { bar = 2; inherit x; }; in x.y.x.foo
1
1 Like
5

Working through your example:

nix-repl> let x = { foo = 1; inherit y; }; y = { bar = 2; inherit x; }; in x.y.x.foo
1

definitely aligns with and solidifies the understanding I was developing of how evaluation of something like foo.bar works as I worked through the lib.fix stuff. Gratis!

1 Like
specialArgs outside of flakes