What's the intended use for flakes in system configuration?

Help
1

Summary - I seem to be misunderstanding flakes. On to the rest:

I’ve been playing with NixOS the last few weeks and trying to get a feel for it. Love the concept and actually found it easier to do many of the things I was wanting to do (very minimal system with zfs root inside LUKS, unified kernel secure booting with no grub etc - basically a very minimal but very modern stack).

I have subsequently spent a lot of time chasing my tail trying to get a scalable config repo going. It’s quite possible that this is me overcomplicating things as I have sysadmin experience and have used other config management systems in the past (ansible/chef et al).

I had assumed that flakes was the preferred way to manage config fragments that you would then compose into a system, but that seems to be a really painful process. Perhaps naively I started pulling things out into seperate folders for things (eg one for zfs root related stuff, one for boot loader) one for home manager and then one for hosts - looks like this after a few evolutions:

.
├── README.md
├── configuration.nix
├── flake.lock
├── flake.nix
├── home
│   └── user
│       └── home.nix
├── hosts
│   └── test
│       └── default.nix
└── modules
    ├── default.nix
    ├── secure-boot
    │   ├── default.nix
    │   └── flake.nix
    └── zfs-root
        └── default.nix

Where the root flake is pretty minimal and just does this:

  description = "Barebones NixOS on ZFS config";

  inputs = {
    nixpkgs.url = "nixpkgs/nixos-23.11";
    #nixpkgs-unstable.url = "nixpkgs/master";
    home-manager = {
      url = "github:nix-community/home-manager/release-23.11";
      inputs.nixpkgs.follows = "nixpkgs";
    };
  };

  outputs = inputs@{ self, nixpkgs, home-manager }:
    let
      mkHost = hostName: system:
        nixpkgs.lib.nixosSystem {
          pkgs = import nixpkgs {
            inherit system;
          };

          specialArgs = {
            # make all inputs availabe in other nix files
            inherit inputs;
          };

          modules = [
            # Configuration shared by all hosts
            ./configuration.nix

            # Configuration per host
            ./hosts/${hostName}

            # make home-manager as a module of nixos
            # so that home-manager configuration will be deployed automatically when executing `nixos-rebuild switch`
            home-manager.nixosModules.home-manager
            {
              home-manager.useGlobalPkgs = true;
              home-manager.useUserPackages = true;

              home-manager.users.user = import ./home/user/home.nix;
            }
          ];
        };
    in {
      nixosConfigurations = {
        test = mkHost "test" "x86_64-linux";
      };
    };
}

So the intent is that the host config then pulls in various config fragments for features that I want (zfs, boot, etc) along with host specific settings (network, system packages etc).

But that means that the host nix file has to do something like this, which seems like it’s unsupported:

  # Import system config modules
  imports = builtins.concatMap import [
    "../../modules/zfs-root"
    "../../modules/secure-boot"
  ];

I’ve seen the issues open about relative paths, but lots of the discussion in there leads me to believe this really isn’t a pattern that’s on the golden path so to speak. I’ve found a few interesting projects that seem to work around things in various ways from the simple to the framework and the libraries (flake-parts and flakelight I guess) but all in all I can’t help but feel like I’m swimming against the tide somehow.

Seems like all my problems go away if each flake is its own repo and maybe that’s more the design use case? That would be an unpleasant workflow while I’m developing a first pass, but maybe I’m just overcomplicating things?

So I guess I’m asking advice - for someone reasonably experienced with the problem space and a few programming languages but new to nix (and not a super huge FP nerd) but looking to really “get” the tool, how should I approach something like this idiomatically in nix? Or am I already on the right track and have just hit the rough edges of something experimental and either accept the pain or just use modules?

2

The issues with relative paths are about having sub-flakes. This is not your problem. What you’re trying to do is very common and easily supported.

Your problem is just that this code is wrong

  imports = builtins.concatMap import [
    "../../modules/zfs-root"
    "../../modules/secure-boot"
  ];

It should be more like

  imports = [
    ../../modules/zfs-root
    ../../modules/secure-boot
  ];

Note that these are path literals, not strings; you don’t have to concatMap; and you don’t have to manually apply the import function

2 Likes
3

There are distinct but stacked concepts in Nix.

  • Nix Lang Builtins:
    • functions to work with nix files and create derivaritons packages,
    • deafult.nix, import, readFile, fromJSON, fromTOML, derivation, etc;
  • NixPkgs Lib:
    • lib to help create packages
    • callPackage, mkDerivation, stdEnv, etc;
  • Nix Modules:
    • Config framework writed in Nix lang.
    • Used by NixOS/SystemManager/HomeManager/DevEnv/etc configurations.
    • configuration.nix, imports, config, options;
      pause for breathing
  • Nix Flakes:
    • Nix Pinning tools and Schema
    • flakes.nix, flakes.lock, outputs, inputs;

Your problem is about Nix Modules imports, not nix lang import, nor nixpkgs lib callPackage, neither flakes inputs.

Yes it is. You can import a Module (using imports), another nix file (using import, callPackage), or read any file (using readFile). You could use local inputs but is mostly overkill in a monorepo.

Do you need a flake?
No, you only need Nix Module.
But flakes add tools for pinning.

Can you do pinning without flake?
Yes, you can do it your self fixing hashes in the code, or can use tools like npins or niv.

2 Likes
4

So yeah, skip flakes until I get the basics locked down is my takeaway. Flakes really seem like they solve distribution problems that just don’t exist in a monorepo. So my thinking is that you just avoid them until one of your modules becomes big enough that it’s worth distributing, or having its own lifecycle, and then when you pull it out into its own repo flakes help you manage that. Before that point they are just needless overhead?

I was trying to make the modules sub flakes, but when I found myself trying to do things like my example it set off my “you have obviously taken a wrong turn somewhere” sense! I’ve moved back to all the modules actually being modules and conceptually it hangs together better for the monorepo scenario I am in for the minute and it seems like pinning etc aren’t much use in this scenario anyway?

The one thing that flakes seem to do better (different?) is the inputs and outputs part to put some kind of contract of expectations around a fragment, but that seems like it’s just not normally done with modules? Modules are really just a lightweight thing to allow you to break up a file and have nix merge them all into a single blob for evaluation? but am I right to think that no matter how you structure things, without flakes there is just one giant evaluation at the end?

Thanks for the quick replies, and also thank you for those mind maps, super helpful!

5

Not a needless overhead, pinning is important.

Without pinning, your versions are not defined in your repository, if you lose your computer tomorrow, and reinstall your config in another machine, and have different behaviors (software versions). If you have more than one machine, the same issue.

Flakes is the official pinning method after channels and defining hashes in your code, it is experimental, but half community are using :person_shrugging:.

I think (subjective opinion) that flake is the easier way to do pinning.

Is not a requirement, but I use the flake “contract” (sometimes called schema) with modules

    nixosModules.I.imports =  [ ./hugosenari/default.nix ];
    nixosModules.os.imports = [ ./cfg.nix ./cache ./networking.nix ];
    nixosModules.BO.imports = [ self.nixosModules.os self.nixosModules.I ./bo ];
    nixosModules.HP.imports = [ self.nixosModules.os self.nixosModules.I ./hp ];
    nixosModules.T1.imports = [ self.nixosModules.os self.nixosModules.I ./t1 ];

    nixosConfigurations.BO  = mylib.os self.nixosModules.BO;
    nixosConfigurations.HP  = mylib.os self.nixosModules.HP;
    nixosConfigurations.T1  = mylib.os self.nixosModules.T1;

I could create a BO.nix module and use imports there :slight_smile:

Not just but yes.

Nix Modules is a framework for templating information with merge and type validation, is a “contract framework”, “Nix Schema” (like json schema).
I like to think it has 3 parts:

  1. options: contract definition, defines what kind of information we need (like someone creating the form).
  2. config: information definition (like someone filling the form). You and I, setting our OS config.
  3. config¹: use the information (single blob of evaluation) to create a config file or package. Again, is the community, but can be anyone.

¹ Is easy to make a confusion within 2 and 3 because they have same name, 2 is you defining information, 3 is someone using the single blob of evaluation (usaully to set 2).

example (source):

{ pkgs, lib, config, ...}:
             # ^- 3 the blob
{
  # 1 starts --------------------------------------------
  options.nixos-boot = {
    enable = lib.mkOption {
      default = false;
      type    = lib.type.bool;
    };
    theme  = lib.mkOption {
      default = "default";
      type    = lib.types.enum [ "load_unload" "default" ];
    };
    bgColor.red = lib.mkOption {
      default = 255;
      type    = lib.types.ints.between 0 255;
    };
    # ...
  };
  # 1 ends -----------------------------------------------

  # 2 starts ---------------------------------------------
  config.boot.plymouth = lib.mkIf config.nixos-boot.enable {
                                # ^----- 3 the blob -----^
    themePackages = [ pkgs.nixos-boot ];
    enable = true;
    theme  = config.nixos-boot.theme;
           # ^---- 3  the blob ----^
  };
  # ...
  # 2 ends -----------------------------------------------
}

With that our user (maybe we or another module creator) can set nixos-boot.enable, nixos-boot.theme and nixos-boot.bgColor.red, because of 1. And since we are referencing it (3) to set 2, it will enable plymouth for us, set its theme. But could be used to create a file or a package.

NixOS is just a giant evaluation of Nix Modules at end, so with or without flakes, yes.

There are other cool features for packages in flakes, like run, build and shell, templates.
Example: GeNix7000: Nix Project Logo Generator - #17 by hugosenari

6

The other thing they do much better are the inputs. Traditional nix uses channels, which are inherently stateful and a massive footgun. Having inputs and a flake.lock is an incredibly important improvement and almost the entire reason I use them personally.

The contract results in concepts that are easier to learn and understand, as well as improved CLI UX, but those are just side benefits.

From a learning perspective, personally I believe that flakes are much more transparent and ultimately result in a better understanding of NixOS, even if up-front they may be a bit harder to grok, mostly because official docs don’t cover them at all.

If you really struggle with flakes, though, you can alternatively get pinning with niv, too.

3 Likes