Custom NixOS module and infinite recursion encountered

Help
1

Hello,

While writing a custom NixOS module I faced the infinite recursion encountered error.

The module provides a configuration for multiple systemd units. It is split into 3 parts, I naively thought it was simpler to maintain like that : the module definition in default.nix, the options of one unit in options.nix and the configuration of one unit in module.nix.

I expected the module to work this way:

  1. Read the system configuration from config.services.custom-service,
  2. Evict disabled units,
  3. Generate an attr. set for each unit, merge them together and assign the final attr. set to the module config attribute.

See the nix code describing this module:

Custom service 
# /etc/nixos/services.nix
{ config, lib, pkgs, ... }:

{
  imports = [
    /etc/nixos/modules/custom-service/default.nix
  ];

  services.custom-service."foo" = {
    enable = true;
    content = "hello world";
  };
}

# /etc/nixos/modules/custom-service/default.nix
{
  lib,
  config,
}:

let
  inherit (lib)
  filterAttrs
  mapAttrsToList
  mergeAttrsList
  mkOption
  types
  ;

  mkModule = import ./module.nix;

  enabled = filterAttrs (_: cfg: cfg.enable) config.services.custom-service;
  result = mergeAttrsList (mapAttrsToList (name: cfg: mkModule { name = name; cfg = cfg; }) enabled);
in
{
  options.services.custom-service = mkOption {
    default = {};
    type = types.attrsOf (types.submodule (import ./options.nix { inherit lib; }));
  };

  config = result;
}

# /etc/nixos/modules/custom-service/options.nix
{ lib }:

let
  inherit (lib)
  mkEnableOption
  mkOption
  types
  ;
in
{
  options = {
    enable = mkEnableOption "";

    content = mkOption {
      type = types.str;
    };
  };
}

# /etc/nixos/modules/custom-service/module.nix
{
  name,
  cfg,
}:

{
  systemd.services."custom-service-${name}" = {
    description = "Custom service ${name}";

    wantedBy = [ "multi-user.target" ]; 

    serviceConfig = {
      DynamicUser = true;
    };

    script = ''
      cat <<EOF
        Service: ${name}.
        Content:
        ${cfg.content}
      EOF
    '';
  };

  systemd.timers."custom-service-${name}" = {
     wantedBy = [ "timers.target" ];
     timerConfig = {
        OnUnitActiveSec = "30s";
     };
  };
}

While nixos-rebuild dry-build fails to build this NixOS configuration, I can produce the attr. set of the module in nix repl:

nix-repl> lib = import <nixpkgs/lib>

nix-repl> config = { services = { custom-service = { foo = { enable = true; content = "hello world"; }; }; }; }

nix-repl> module = import ./default.nix { lib = lib; config = config; }

nix-repl> :p module.config.systemd
{
  services = {
    custom-service-foo = {
      description = "Custom service foo";
      script = "cat <<EOF\n  Service: foo.\n  Content:\n  hello world\nEOF\n";
      serviceConfig = { DynamicUser = true; };
      wantedBy = [ "multi-user.target" ];
    };
  };
  timers = {
    custom-service-foo = {
      timerConfig = { OnUnitActiveSec = "30s"; };
      wantedBy = [ "timers.target" ];
    };
  };
}

Alternatively I looked at different approaches and the most common I’ve seen in nixpkgs was to “merge manually” my content of module.nix in the top level module definition, I’ve adapted my original structure of the module and it now works. I’m not sure this is super clear and I hope the following code will capture the idea.

Custom service without `module.nix` 
{
  lib,
  config,
}:

let
  inherit (lib)
  filterAttrs
  mapAttrs'
  mkOption
  types
  ;

  enabled = filterAttrs (_: cfg: cfg.enable) config.services.custom-service;
in
{
  options.services.custom-service = mkOption {
    default = {};
    type = types.attrsOf (types.submodule (import ./options.nix { inherit lib; }));
  };

  config = {
    systemd.services = mapAttrs' (name: cfg: {
      name = "custom-service-${name}";
      value = {
        description = "Custom service ${name}";

        wantedBy = [ "multi-user.target" ]; 

        serviceConfig = {
          DynamicUser = true;
        };

        script = ''
          cat <<EOF
            Service: ${name}.
            Content:
            ${cfg.content}
          EOF
        '';
      };
    }) enabled;

    systemd.timers = mapAttrs' (name: cfg: {
      name = "custom-service-${name}";
      value = {
        wantedBy = [ "timers.target" ];
         timerConfig = {
           OnUnitActiveSec = "30s";
         };
       };
    }) enabled;
  };
}

From a programming perspective, I expect both solutions to produce the exact same configuration and it seems to be the case when tested with nix repl.
But the former approach fails against nixos-rebuild dry-build while the latter works.
Personally I prefer the former but I’m fine to adapt. Still I would like to understand what’s happening but my knowledge of NixOS is too limited to understand the issue, would anyone be able to explain it?
Also, I’m interested if there exists a method to debug from the trace produced by nixos-rebuild.

Thank you.

PS: I originally asked the question on IRC and thought it would be better documented on discourse.

2

This is a common issue. The problem is that you’re computing the value of config itself in the module, and computing using the values of options under config.

To be specific, the number of attributes in config.services.custom-service must be determined in order to determine the length of the list mergeAttrsList is operating over, which must be determined before it can give any output for config to be merged into the argument config, which is needed to index in and find config.services.custom-service. Loop complete.

1 Like
3

What you did in the REPL doesn’t replicate what the module system does. In the module system, config always refers to the final config value. We call it a ‘fixed point’. In the REPL, you have two different config values: the one constructed just from attrset literals, and the one that is found in module.config.

Infinite recursion is basically guaranteed any time you try to assign a non-literal, strictly-computed value to all of config. You can often avoid it by assigning only a subvalue of config, as you’ve done in your second example (‘manually merging’ indeed). Or you could use lib.mkMerge (instead of strict functions like mergeAttrsList) to combine multiple config values in a way the module system will understand as a suspended computation; that will often resolve infinite recursion issues too.

1 Like
4

Rarely helpful in practice for these, since the length of that list still needs to be totally independent of any option to avoid infrec.

2 Likes
5

Ah, you are correct!

6

I didn’t realize the config was composed using the fixed point combinator but it actually makes sense.

I think I can see the problem now while it’s not totally clear to me how the fixed point combinator is used in NixOS to compose the configuration and how the lazy evaluation works in nix.

Is there a world (i.e. maybe with a different evaluation system) where the analysis of the expression can detect result evaluates to { systemd.services.* = ...; systemd.timers.* = ...; }, detect config of the module does not have services.custom-service.* and so detect the expression is finite?

Thanks!

7

Short answer, no.

The nix language has less laziness than it theoretically could. As things stand now, the full set of keys in an attrset must be determined before any key’s value can be accessed. It’s possible to make the laziness more fine grained, which would help with some problems like this, but wouldn’t help with this one. Even with a finer grain, there’s simply no way for nix to know that the result of mergeAttrsList doesn’t set something in config.services.custom-service. Only some kind of proof engine could do something like that, and that would be rediculous to incorporate into a language’s semantic model…

1 Like