Can anyone Help me streamline stuff?

Help
1

So i’ve been using NixOs for half a year now. Reading up on the Nix language / library really proves to be hard to me. Best resource has been nixpkgs repo so far. But only having done some non-functional programming in my life, nix really messes with my head.

Currently i have this in my flake.nix for my client, so i can add more clients in the future.

nixosConfigurations =
        let
          mkHost =
            hostname:
            {
              system ? "x86_64-linux",
            }:
            nixpkgs.lib.nixosSystem {
              specialArgs = {
                inherit hostname system;
                unstable = import nixpkgs-unstable {
                  inherit system;
                  config.allowUnfree = allowUnfree;
                };
                pkgs = import nixpkgs {
                  inherit system;
                  config.allowUnfree = allowUnfree;
                };
              };
              modules = [ ./clients/${hostname} ];
            };
        in
        nixpkgs.lib.mapAttrs mkHost { tux = { }; }

This works perfectly fine. One thing i don’t understand about it though is how it nixosConfigrations.“…” is set with this mkHost function. Does it just apply the first function parameter? Also, is there a way to avoid having to inherit system everywhere?

No I’ve been trying to get a similar thing going for server infrastructure using deploy-rs, since i really don’t want to retype hostnames all the time, just set it in one place and forget about it. Currently I’m typing the hostname in 6 different places, which is way to many times.

nixosConfigurations.nc = nixpkgs.lib.nixosSystem {
        system = "x86-64-linux";
        specialArgs = {
          hostname = "nc";
        };
        modules = [ ./hosts/nc ];
      };
      deploy.nodes.nc = {
        hostname = "nc.${domain}";
        profiles.system = {
          user = "root";
          sshUser = "admin";
          path = deploy-rs.lib.x86_64-linux.activate.nixos self.nixosConfigurations.nc;
        };
      };

      checks = builtins.mapAttrs (system: deployLib: deployLib.deployChecks self.deploy) deploy-rs.lib;
    };

So how could i streamline this so that i only have to type the hostname once, like in the first example. I’m having a really hard time figuring this stuff out.

I’v Looked at the “Nix Language: Learning resources” and they helped me understand the basics, but this stuff just goes over my head. I also can’t comprehend what the “checks” value would possibly be evaluated to.

Any help is greatly appreciated.

Cheers!

BTW just made this account and didn’t expect to get my name, there’s always a first i guess :slight_smile:

1 Like
2

You could do a let binding:

let
  hostname = "myhostname";
in

And then you can use it either directly or with string interpolation: ${hostname}.

1 Like
3

Sorry of being somehow off topic, but these are my remarks about you post:

I’m not sure where about the source of this code. It looks to me that you copied it from somewhere and and now you don’t understand it. Maybe I’m wrong…

In my opinion you should setup you systems in a way that works for you. Ideally simple and at first maybe with redundancy and repetition. I think this is completely fine and part of the learning path. Later on, when you understand more of the nix way, you could add more abstractions and remove redundancy from you code.

Adding something like deploy-rs to you setup is quite easy, but without a basic understanding of the language, its scoping rules like let in, you will be lost.

Just my 2 cents.

2 Likes
4

Not sure which languages you are familiar but mapAttrs is pretty much like the common map function you can find e.g. in Python or JavaScript, except instead of lists, it works on attribute sets (similar to Objects in JS, dicts from Python).

mapAttrs expects a “function with two arguments”. Well, to be honest, Nix functions can only have a single argument. If you need to pass multiple arguments, you can either:

  • use multiple nested functions. For example, a: b: a + b is really a function that takes a as an argument, and returns a function (closure, to be precise) b: a + b, where a is set to the value passed as an argument to the outer function. When you call a function such as add 33 9, you actually have two function calls add 33 and then (add 33) 9. The benefit is that the partial application add 33 is a value so you can use and pass it around without providing the second argument, which is useful for passing data around.
  • use a composite data structure such as a list or, more commonly, an attribute set. Nix supports pattern matching (destructuring in JS) for attribute sets in function arguments. For example, { a, b }: a + b Is still a function that takes a single argument. Really, it could be equivalently written as args: args.a + args.b.

You always need to pass system when importing a Nixpkgs in a pure evaluation (Flakes) since Nixpkgs it needs to know for which platform the packages should build. You do not need to pass it to specialArgs, though, unless you need it to be included in the attribute set that is passed as an argument to NixOS modules.

Also note that pkgs is a special argument for NixOS, so you should really pass it as follows:

             nixpkgs.lib.nixosSystem {
+              pkgs = import nixpkgs {
+                inherit system;
+                config.allowUnfree = allowUnfree;
+              };
               specialArgs = {
                 inherit hostname system;
                 unstable = import nixpkgs-unstable {
                   inherit system;
                   config.allowUnfree = allowUnfree;
                 };
-                pkgs = import nixpkgs {
-                  inherit system;
-                  config.allowUnfree = allowUnfree;
-                };
               };
               modules = [ ./clients/${hostname} ];
             };

Or equivalently (the above is a shorthand for the below):

             nixpkgs.lib.nixosSystem {
               specialArgs = {
                 inherit hostname system;
                 unstable = import nixpkgs-unstable {
                   inherit system;
                   config.allowUnfree = allowUnfree;
                 };
-                pkgs = import nixpkgs {
-                  inherit system;
-                  config.allowUnfree = allowUnfree;
-                };
               };
-              modules = [ ./clients/${hostname} ];
+              modules = [
+                ./clients/${hostname}
+
+                {
+                  nixpkgs.pkgs = import nixpkgs {
+                    inherit system;
+                    config.allowUnfree = allowUnfree;
+                  };
+                }
+              ];
             };

Something like the following should work:

outputs = { nixpkgs, self, ... }:
  let
    inherit (nixpkgs) lib;

    makeNixosConfigurationAndDeploy =
      hostname:
      {
        system ? "x86_64-linux",
      }:
      {
        nixosConfigurations.${hostname} = lib.nixosSystem {
          # …
        };
        deploy.nodes.${hostname} = {
          # …
        };
      };

    hosts = {
      nc = { };
      tux = { };
    };

    # Here, we have the following list:
    #
    #     [
    #       {
    #         nixosConfigurations.nc = /* … */;
    #         deploy.nodes.nc = /* … */;
    #       }
    #
    #       {
    #         nixosConfigurations.tux = /* … */;
    #         deploy.nodes.tux = /* … */;
    #       }
    #     ]

    nixosConfigurationsAndDeploysList =
      lib.mapAttrsToList /* link visited  times */
        makeNixosConfigurationAndDeploy
        hosts;

    # Here, we have the following attribute set:
    #
    #     {
    #       nixosConfigurations = {
    #         nc = /* … */;
    #         tux = /* … */;
    #       };
    #       deploy.nodes = {
    #         nc = /* … */;
    #         tux = /* … */;
    #       };
    #     }
    #
    # This already matches the shape Flake `outputs` function should return.
    #
    # `foldl'` (called `reduce` in JavaScript) is basically a loop.
    # Here, specifically, we repeatedly recursively merge the attribute sets
    # from the `nixosConfigurationsAndDeploysList` into the result set,
    # starting with an empty attribute set.

    nixosConfigurationsAndDeploysMerged =
      lib.foldl'
        lib.recursiveUpdate /*link visited  times*/
        { }
        nixosConfigurationsAndDeploysList;

  in

  # Merge it all together:
  nixosConfigurationsAndDeploysMerged
  // {
    checks =
      lib.mapAttrs
        (system: deployLib: deployLib.deployChecks self.deploy)
        deploy-rs.lib;
  };
2 Likes
5

First of all hank you very much for taking your time to help me out!

So far as i understand mapAttrs takes a function with two inputs and an attribute set as input. Then it executes the function for every key from that attribute set and maps the respective key to the first input and the value to the second input, where the value can also be a attribute set. is that correct?

what i don’t understand is, how the hostname part of nixosConfigurations is set, it evaluates to nixosConfigurations.tux in my example, but i don’t seem to be setting it in any way. how does it evaluate this correctly?

Does my mkHost function return an attribute set like this which nixosConfigurations is set to?

# added nc as example not in original code
{
  tux = { /* ... */ };
  nc = { /* ... */  };
}

If so, why is the first input to mkHost returned as a key to the attribute set?

Your suggested change for the pkgs import results in following error:

Failed assertions:

  • Your system configures nixpkgs with an externally >created instance.
    nixpkgs.config options should be passed when creating >the instance instead.

The answers to this error on goolge just confuse me, so I’ll just leave it like it was before since that’s working for me so far. Or is there any issues with passing pkgs like that?

So foldl' applies the function lib.recursiveUpdate to every attribute set in the provided list and accumulates the result from lib.recursiveUpdate in the initially empty attribute set. But the accumulator of foldl' is also the first input of lib.recursiveUpdate which merges common parent sets and overwrites duplicate keys?

Is there a way i can load the flake into a repl and actually explore what everything evaluates to, so i don’t have to replicate everything for exploring purposes or is this unpractical.

again thanks, learned a bunch from your response. i’m really struggling to understand the functional programming paradigm.

6

you’re right i copied the mapAttrs example, but i actually played around with it trying to understand.

I started with a plain nix-configuration.nix and later on moved on to flakes, and now i got bothered by the extensive retyping of the hostname variable etc. which is why i want to streamline it.
the problem is that i won’t be able to understand the nix language by only using what i understand.

if you have any techniques or challenges for learning nix they’d be appreciated.

7

When we talk about mapping here, we mean that values are transformed while preserving the shape of the data structure. For example:

builtins.map someFunc [
  valueA
  valueB
  valueC
]

will evaluate to

[
  (someFunc valueA)
  (someFunc valueB)
  (someFunc valueC)
]

and similarly

lib.mapAttrs anotherFunc {
  a = valueA;
  b = valueB;
  c = valueC;
}

will evaluate to

{
  a = anotherFunc a valueA;
  b = anotherFunc b valueB;
  c = anotherFunc c valueC;
}

Notably, lib.mapAttrs cannot change the attribute names in the attribute set it maps over. It only passes the key to the function because it is often useful, and can be just ignored when not needed. It is less useful for lists, where items are deemed equal so if you want indexes, you will need lib.imap0.

The value of the attribute can be of any Nix type including an attribute set. Since it is an attribute set in the mkHost case, we can pattern match on the argument value.

No. It returns a NixOS system (which, coincidentally, is also an attribute set – Nix does not have that many types it could be – but that is not relevant here).

Right. You are setting nixpkgs.config NixOS option somewhere, which will not have any effect on the pkgs you create yourself. You can just remove the option, as you are already passing the config when creating pkgs.

Alternately, you can remove pkgs definition completely and let NixOS module create it implicitly, in which case nixpkgs.config is correct way to configure it.

Take care to remove the number, Discourse inserts it next to the link but it will break the expression.

Just about right, though note that values in Nix are immutable so really, a new attribute set is created each time. The evaluation would happen something like this

lib.foldl' lib.recursiveUpdate { } [ { a = aVal; } { b = bVal; } { c = cVal; } ]
lib.foldl' lib.recursiveUpdate (lib.recursiveUpdate { } { a = aVal; }) [ { b = bVal; } { c = cVal; } ]
lib.foldl' lib.recursiveUpdate (recursiveUpdate (lib.recursiveUpdate { } { a = aVal; }) { b = bVal; }) [ { c = cVal; } ]
lib.foldl' lib.recursiveUpdate (lib.recursiveUpdate (recursiveUpdate (lib.recursiveUpdate { } { a = aVal; }) { b = bVal; }) { c = cVal; }) [ ]
(lib.recursiveUpdate (recursiveUpdate (lib.recursiveUpdate { } { a = aVal; }) { b = bVal; }) { c = cVal; })
(lib.recursiveUpdate (recursiveUpdate { a = aVal; } { b = bVal; }) { c = cVal; })
(lib.recursiveUpdate { a = aVal; b = bVal; } { c = cVal; })
{ a = aVal; b = bVal; c = cVal; }

You can run :lf . in nix repl to bring the outputs of the flake in the current directory into scope.