Understanding the import system in NixOS

This question is really specific to my way of doing things but I think it can be enlightening to ask here for visibility on the (I hope there is one) solution and the understanding of the inner workings.

The setup

To work on my NixOS setup I have the following tree hierarchy in the root of my configuration directory :

.
├── dotfiles
├── lib
├── machines
├── modules
├── overlays
├── packages
├── pkgs
├── secrets
└── sources

In the modules directory, I have file default.nix that looks like this :

{ lib, config, pkgs, ... }:

with lib; with builtins; with types;
let
  # Recursively constructs an attrset of a given folder, recursing on directories, value of attrs is the filetype
  getDir = dir:
    mapAttrs
      (file: type: if type == "directory" then getDir "${dir}/${file}" else type)
      (readDir dir);

  # Collects all files of a directory as a list of strings of paths
  files = dir:
    collect isString
      (mapAttrsRecursive (path: type: concatStringsSep "/" path) (getDir dir));

  # Filters out directories that don't end with .nix or are this file, also makes the strings absolute
  validFiles = dir:
    map (file: ./. + "/${file}")
      (filter (file: hasSuffix ".nix" file && file != "default.nix" && !hasSuffix "lib.nix" file) (files dir));

  overlays = (import ../overlays/overlays.nix { inherit lib; }).overlays;
  src-location = (import ../sources { inherit lib; inherit (pkgs) callPackage; });
in
{
  imports = validFiles ./.; # <=  HERE
  config.nix.nixPath = [
    "/nix/var/nix/profiles/per-user/root/channels"
    "nixos-config=/etc/nixos/configuration.nix"
  ] ++ (lib.attrsets.mapAttrsToList (name: value: "${name}=${value}")) src-location;
  config.nixpkgs.overlays = overlays;
}

This file allows me to organize my files in the module directory how I please (at least that’s what I thought) and let NixOS to do the import and evaluation thanks to imports marked “HERE”.

For a machine configuration in a subdirectory of the machine directory I have for example

{ lib, config, pkgs, ... }: {

  imports = [
    <nixos-hardware/raspberry-pi/2>
    <nixpkgs/nixos/modules/profiles/minimal.nix>
    ../../modules # The infamous import of the above file here
  ];

  networking.hostName = "ophion";

# Personal modules defined throughout the `module` directories
  perso = {
    ssh = true;
    rust-utils = true;
    type = "server";
    services = {
      cluster = {
        enable = true;
        role = "server";
      };
    };
  };
# NixOS defined module
  boot.cleanTmpDir = true;
}

It might not be the best and most efficient solution (I am more than interested in improvements so please comment ahead if you have any!) but it works well enough for my use case until…

The problem

I am in the process of setting up a Raspberry Pi from an x86_64 machine using nixos-generators to build the ISO image. When I pass the configuration file (the one above) with nixos-generate -f iso -c machines/ophion/default.nix, I get
error: path '/nix/store/qyinilkg5l70jp87mnvq9ncgg8hmbiw7-modules' is not valid

A keen eye will have noticed that this is one of the path from my configuration. Although this specific store path does not exist, many others (created by nixos-rebuild, I imagine) do exist and contain the content of the modules directory at the time of execution of nixos-rebuild.

The same configuration can be built without an issue by nixos-rebuild.

Why is it not just importing the files and evaluating them as nixos-rebuild is doing ?

There may be nix store corruption in this case. Specifically, the nix database and the store are out of sync. Nix thinks the store path already exists, and thus doesn’t copy it into place, but then it actually goes and looks, finds it doesn’t exist, and freaks out. That’s usually the cause of not valid errors. It would happen if, for example, you deleted a store path manually. Why it’s happening here is less clear, but you should at least verify your store.

1 Like

Thank you a lot for your answer !

That’s an interesting take ! After verifying its use I came up with the following command to check the Nix store content nix-store --verify --check-contents. After letting it run, the previous command to generate the ISO still sends back the same result with the same nix store path…

Given that you didn’t mention anything, I’m assuming the verification didn’t complain about anything?

Looking more closely at your code, I believe you’re copying things to the nix store that you really don’t need to be and probably didn’t mean to be.

The getDir implementation uses ${dir}, which, if dir is a path type, will copy it to the nix store, resulting in the next call being on a string instead of a path. You probably want (dir + "/${file}") instead of "${dir}/${file}". I think that should avoid the problem entirely by not involving that store path in the first place. I still don’t know why it was failing exclusively with nixos-generators, however.

For reference:

$ mkdir test
$ cd test
$ touch foo
$ nix repl
Welcome to Nix 2.11.0. Type :? for help.

nix-repl> "${./foo}"
"/nix/store/avva3pijyyj4bgrj1vcl3sj6hiadjxlj-foo"

nix-repl> "${./.}/foo"
"/nix/store/4f7080bgxv3hc5shk2dbw4vfazg341p2-test/foo"

nix-repl> ./foo
/mnt/persist/share/data/tejing/work/test/foo

nix-repl> ./. + "/foo"
/mnt/persist/share/data/tejing/work/test/foo

I must say that you puzzled me for a while because this is not true using the function as stated in my first message is sending back recursively all the files in the hierarchy and no directories.

Perhaps this makes the difference clearer?

$ mkdir test
$ cd test
$ mkdir foo
$ touch foo/bar
$ nix repl
Welcome to Nix 2.11.0. Type :? for help.

nix-repl> :p let getDir = dir: builtins.trace "getDir called on ${toString dir}" builtins.mapAttrs (file: type: if type == "directory" then getDir "${dir}/${file}" else type) (builtins.readDir dir); in getDir ./.
trace: getDir called on /mnt/persist/share/data/tejing/work/test
{ foo = trace: getDir called on /nix/store/ch913k9qn6zd9v9pn1ym0pbs1x6mhy99-test/foo
{ bar = "regular"; }; }

nix-repl> :p let getDir = dir: builtins.trace "getDir called on ${toString dir}" builtins.mapAttrs (file: type: if type == "directory" then getDir (dir + "/${file}") else type) (builtins.readDir dir); in getDir ./.
trace: getDir called on /mnt/persist/share/data/tejing/work/test
{ foo = trace: getDir called on /mnt/persist/share/data/tejing/work/test/foo
{ bar = "regular"; }; }

That is really weird because when I use validFiles I get a list of nix files that are not nix store paths…

The output is the same in both cases. It’s not about the output, it’s about strange and unnecessary copying going on in the process of getting that output. I’m using builtins.trace to show what’s happening differently. The first call is the same in both cases, but the recursive call implicitly copies the directory to the nix store in one case, and keeps working on the original in the other.

Ok I see your point now !

But my question is now why would folders that are not imported would be the cause of the issue outlined

Aside: I am trying to make your version work without success… :sweat_smile:

The unnecessary implicit copy to the nix store I’m talking about is the whole reason the /nix/store/qyinilkg5l70jp87mnvq9ncgg8hmbiw7-modules store object is being created in the first place. I still don’t know why there’s an issue with that store path, but this change should avoid the need to even deal with it, so hopefully can get you around the problem.

Thank you for your explanation ! I will try to make your version of getDir working then !

P.S. : you were exactly right about the whole issue : nixos-generators is now more than happy to generate the iso (modulo some module error :eyes: ) :confetti_ball: