Infinite recursion in module with mkMerge

Help
1

I’ve been trying to create a module that will essentially merge two sets of attributes. Here’s the module:

{ config, lib, pkgs, ... }:
with lib;
let
  cfg = config.tilde.modules.configuration;
in {
  options.tilde.modules.configuration = {
    machine = mkOption {
      type = types.attrs;
      default = { };
    };

    user = mkOption {
      type = types.attrs;
      default = { };
    };
  };

  config = mkMerge [ cfg.machine cfg.user ];
}

And the usage of the module:

{
  imports = [
    ./modules/configuration
  ];

  tilde.modules.configuration = { };
}

When I try to build as part of a nix-darwin configuration, I get an infinite recursion encountered error:

building the system configuration...
error: while evaluating 'evalModules' at /nix/store/pci4ssg62nr8mg3kax4ccrpzbz4vn5lc-nixpkgs-21.03pre262944.f18ba0425d8/nixpkgs/lib/modules.nix:62:17, called from /nix/store/0882sb0p0lhf4b964apsbbfpijl94ngz-darwin/darwin/eval-config.nix:25:10:
while evaluating the attribute '_module.freeformType' at undefined position:
while evaluating 'g' at /nix/store/pci4ssg62nr8mg3kax4ccrpzbz4vn5lc-nixpkgs-21.03pre262944.f18ba0425d8/nixpkgs/lib/attrsets.nix:276:19, called from undefined position:
while evaluating anonymous function at /nix/store/pci4ssg62nr8mg3kax4ccrpzbz4vn5lc-nixpkgs-21.03pre262944.f18ba0425d8/nixpkgs/lib/modules.nix:139:72, called from /nix/store/pci4ssg62nr8mg3kax4ccrpzbz4vn5lc-nixpkgs-21.03pre262944.f18ba0425d8/nixpkgs/lib/attrsets.nix:279:20:
while evaluating the attribute 'value' at /nix/store/pci4ssg62nr8mg3kax4ccrpzbz4vn5lc-nixpkgs-21.03pre262944.f18ba0425d8/nixpkgs/lib/modules.nix:512:9:
while evaluating the option `_module.freeformType':
while evaluating the attribute 'mergedValue' at /nix/store/pci4ssg62nr8mg3kax4ccrpzbz4vn5lc-nixpkgs-21.03pre262944.f18ba0425d8/nixpkgs/lib/modules.nix:544:5:
while evaluating the attribute 'values' at /nix/store/pci4ssg62nr8mg3kax4ccrpzbz4vn5lc-nixpkgs-21.03pre262944.f18ba0425d8/nixpkgs/lib/modules.nix:538:9:
while evaluating the attribute 'values' at /nix/store/pci4ssg62nr8mg3kax4ccrpzbz4vn5lc-nixpkgs-21.03pre262944.f18ba0425d8/nixpkgs/lib/modules.nix:637:7:
while evaluating 'byName' at /nix/store/pci4ssg62nr8mg3kax4ccrpzbz4vn5lc-nixpkgs-21.03pre262944.f18ba0425d8/nixpkgs/lib/modules.nix:362:25, called from /nix/store/pci4ssg62nr8mg3kax4ccrpzbz4vn5lc-nixpkgs-21.03pre262944.f18ba0425d8/nixpkgs/lib/modules.nix:378:22:
while evaluating 'byName' at /nix/store/pci4ssg62nr8mg3kax4ccrpzbz4vn5lc-nixpkgs-21.03pre262944.f18ba0425d8/nixpkgs/lib/modules.nix:362:25, called from /nix/store/pci4ssg62nr8mg3kax4ccrpzbz4vn5lc-nixpkgs-21.03pre262944.f18ba0425d8/nixpkgs/lib/modules.nix:374:21:
while evaluating anonymous function at /nix/store/pci4ssg62nr8mg3kax4ccrpzbz4vn5lc-nixpkgs-21.03pre262944.f18ba0425d8/nixpkgs/lib/modules.nix:337:19, called from /nix/store/pci4ssg62nr8mg3kax4ccrpzbz4vn5lc-nixpkgs-21.03pre262944.f18ba0425d8/nixpkgs/lib/modules.nix:337:8:
while evaluating 'pushDownProperties' at /nix/store/pci4ssg62nr8mg3kax4ccrpzbz4vn5lc-nixpkgs-21.03pre262944.f18ba0425d8/nixpkgs/lib/modules.nix:576:24, called from /nix/store/pci4ssg62nr8mg3kax4ccrpzbz4vn5lc-nixpkgs-21.03pre262944.f18ba0425d8/nixpkgs/lib/modules.nix:337:73:
while evaluating 'pushDownProperties' at /nix/store/pci4ssg62nr8mg3kax4ccrpzbz4vn5lc-nixpkgs-21.03pre262944.f18ba0425d8/nixpkgs/lib/modules.nix:576:24, called from /nix/store/pci4ssg62nr8mg3kax4ccrpzbz4vn5lc-nixpkgs-21.03pre262944.f18ba0425d8/nixpkgs/lib/modules.nix:578:7:
while evaluating the module argument `config' in "/Users/eturkeltaub/tilde/modules/configuration":
while evaluating the attribute 'config' at /nix/store/pci4ssg62nr8mg3kax4ccrpzbz4vn5lc-nixpkgs-21.03pre262944.f18ba0425d8/nixpkgs/lib/modules.nix:130:21:
infinite recursion encountered, at /nix/store/pci4ssg62nr8mg3kax4ccrpzbz4vn5lc-nixpkgs-21.03pre262944.f18ba0425d8/nixpkgs/lib/modules.nix:130:21

It seems like I can’t refer to cfg inside of mkMerge, but I have no idea why that’s the case. Can anyone help me out?

2

In theory the result of mkmerge could contain new options for tilde.modules.configuration therefore it’s recursing infinitely. You need to make sure that this overlap can not happen.

1 Like
3

Is there any way to avoid this? The result of mkMerge won’t need to access the tilde.modules.configuration options.

4

As it potentially can have those keys, it would mean that those options would need to be merged again due to how the nodule system and final merging works.

Merge them under a key that is not tilde and it should work without infinite recursion.

5

Do you mean something like this?

{ config, lib, pkgs, ... }:
with lib;
let
  cfg = config.modules.configuration;
in {
  options.modules.configuration = {
    machine = mkOption {
      type = types.attrs;
      default = { };
    };

    user = mkOption {
      type = types.attrs;
      default = { };
    };
  };

  config = mkMerge [ (cfg.machine) (cfg.user) ];
}
6

No, I meant something like config.foo = mkMerhe ...

As long as you merge directly into the top-level config, you will keep the the infinite recursion as your merged results still could contain the keys you want to merge over.

1 Like
7

Is there any way to apply this sort of pattern then, where some configurations get merged into the top level? I could use an example, I’m a bit of a Nix newcomer :slight_smile:

8

No. Not that I am aware of. Unless you have it at the top-level from the very first moment.

What do you actually want? Mirror top-level configuration in your 2 sets?

9

Basically I’d like to have definitions for machines (nix-darwin or NixOS) and definitions for users (home-manager) and then use combinations of those to create a configuration. I could do this with just imports but I like the module pattern because it allows for a set of required options and merges applies that to a default configuration (so I can pass the networking hostname to a machine definition, and it will merge that in with the rest of the default machine configuration).

10

Common approach is to provide different entrypoint for the config using nixos-config entry in the nix path. Or even provide host and user entries to import from.

{
  imports = [ <host> <user> ];
}

This at least works when not using flakes.

11

I seem to have the same Problem:

{
  options.myModule = mkOption {
    type = types.attrsOf types.anything;
    default = {};
  };
  config = mkMerge (
    attrsets.mapAttrsToList (
      name: value: {}
    )
    config.myModule
  );
}

The actual module I was trying to make was larger, but I broke it down to this.

What I don’t really understand, why would it cause an infinite recursion error, if it could contain options for itself? Shouldn’t the infinite recursion be caused by the containing itself instead of the possibility?

I mean something like this:

{
  options.myModule.enable = mkEnableOption "myModule";
  config = mkIf config.myModule.enable {};
}

would literally be the same thing, wouldn’t it?
It could contain options, that would cause infinite recursion, but doesn’t.

12

no. the infinite recursion is caused by some config attribute depending, in some way, on itself in a way the module system cannot resolve. config (the argument) is (to first approximation) a fixed point of mkMerge, so try to imagine the config (attribute) you write as though every config (argument) had been replaced with the entire value bound to config (attribute). in your first case this process goes on forever, so you get an infinite recursion. in your second example it stops after one iteration, so you don’t get an infinite recursion.

importantly note that nix is a lazy language, so only values that are actually used count towards producing an infinite recursion. that’s why you get an infinite recursion out of

{ options.myModule = mkEnableOption "";
  config = mkMerge (mapAttrsToList (_: _: {}) config); }) ]; }

but not out of

{ options.myModule = mkEnableOption "";
  config = mkMerge (mapAttrsToList (_: _: {}) { a = config; }); }) ]; }

the first case uses the attribute names (and thus the value) of config, the second one does not use config at all. if we introduce a use of config with a trace we get an infinite recursion again:

{ options.myModule = mkEnableOption "";
  config = mkMerge (mapAttrsToList (_: c: trace c {}) { a = config; }); }) ]; }

mkIf gets some additional special handling on top of all this or it would create infinite recursion pretty much every time it’s used.

2 Likes
13

Just last week I held a Nix Hour relating to recursion in the module system, and it also covered getting infinite recursion with mkMerge and how to work around it. Feel free to check it out, comes with timestamps :slight_smile: https://www.youtube.com/live/cZjOzOHb2ow

And if you want something else covered in this format, feel free to ask a question directly by joining, or writing it down in GitHub - tweag/nix-hour: Questions for the weekly Nix Hour beforehand

5 Likes
14

Here’s an approach by @udf

This takes the keys on the first level and mkMerges their values.

However, it needs to get a list of keys to recurse into, why? Is it because listing all the keys triggers the recursion alert?

2 Likes