Understanding hardware-config

[tcurdt]

I am currently trying nixos-anywhere to setup a hetzner server and realized that I don’t fully understand the process.

I use nixos-anywhere --flake #hetzner --target-host root@$SERVER_IP

When the machine restarts it gets stuck in stage 1.

Which seems like it’s missing the correct mapping of the filesystem to the root partition - although the mountpoint is specified.

Now I guess it needs:

  #   --generate-hardware-config nixos-generate-config ./hardware-configuration.nix \
  #   --generate-hardware-config nixos-facter ./facter.json  \

And now I am wondering…

Why is the hardware config a file - and not detected instead?

Here are the files I am using…

configuration.nix:

{ pkgs, lib, ... }:

let
  sshDir = ./ssh;
  pubKeyFiles = builtins.filter (name: lib.hasSuffix ".pub" name) (
    builtins.attrNames (builtins.readDir sshDir)
  );
  sshKeys = map (file: builtins.readFile (sshDir + "/${file}")) pubKeyFiles;
in

{
  imports = [
    ./disk-config.nix
  ];

  boot.loader.grub.enable = true;

  networking = {
    hostName = "nixos";
    useDHCP = false;
    interfaces.eth0.useDHCP = true;
  };

  services.openssh = {
    enable = true;
    settings = {
      PermitRootLogin = "prohibit-password";
      PasswordAuthentication = false;
    };
  };

  users.users.root.openssh.authorizedKeys.keys = sshKeys;

  environment.systemPackages = with pkgs; [
    vim
    git
    curl
    wget
  ];

  nix.settings.experimental-features = [
    "nix-command"
    "flakes"
  ];

  system.stateVersion = "25.05";
}

flake.nix:

{
  description = "nixOS";

  inputs = {
    nixpkgs.url = "github:nixos/nixpkgs/nixos-25.05";
    disko = {
      url = "github:nix-community/disko";
      inputs.nixpkgs.follows = "nixpkgs";
    };
  };

  outputs =
    {
      nixpkgs,
      disko,
      ...
    }:
    {
      nixosConfigurations = {
        hetzner = nixpkgs.lib.nixosSystem {
          system = "x86_64-linux";
          modules = [
            ./configuration.nix
            disko.nixosModules.disko
          ];
        };
      };
    };
}

disk-config.nix:

{
  disko.devices = {
    disk = {
      main = {
        type = "disk";
        device = "/dev/sda";
        content = {
          type = "gpt";
          partitions = {
            boot = {
              size = "1M";
              type = "EF02";
              priority = 1;
            };
            ESP = {
              size = "512M";
              type = "EF00";
              content = {
                type = "filesystem";
                format = "vfat";
                mountpoint = "/boot";
              };
            };
            root = {
              size = "100%";
              content = {
                type = "filesystem";
                format = "ext4";
                mountpoint = "/";
              };
            };
          };
        };
      };
    };
  };
}

[bme]

It’s a question of goals. Is the goal to make the computer like the nix expression or the nix expression like the computer. Nix picks the former over the later exclusively.

[tcurdt]

How is

nixos-generate-config ./hardware-configuration.nix \
nixos-facter ./facter.json

not making the expression like the computer?
Just at a different time.

[TLATER]

@tcurdt

Just at a different time.

That’s the point. If you generate the expression ad-hoc, it’s subject to random change and not reproducible. Hardware may change, bugs may cause changes in reporting, etc.

NixOS prioritizes reproducibility; taking that snapshot of what the hardware looks like and making it static ensures that your configuration is set up for a specific system and produces an OS precisely for that system, every time, rather than different systems producing different OSes every time you evaluate.

The theory is that it matters less that sometimes a misconfigured system will fail to boot than that you can’t reliably tell whether your configuration is broken or if the hardware generation layer is randomly failing to do its job.

I don’t think dynamic generation is really feasible with nix anyway, you’d have to do something very hacky with nixos-rebuild.

[waffle8946]

I don’t even think dynamic gen is possible at the nixos-rebuild stage, since hardware often gets switched out when the system is off. The generation would have to happen at boot.

[tcurdt]

@TLATER

That’s the point. If you generate the expression ad-hoc, it’s subject to random change and not reproducible. Hardware may change, bugs may cause changes in reporting, etc.

Just to be clear - I am not pushing for a dynamic nix expression rebuild that magically works at boot.

(Although I am wondering how different the expression is between systems really are for systems to boot. Basically how specialized the expression really is. But I digress…)

But shouldn’t it be possible to compare what the nix expression was built for with the expected status quo and show the differences if found?

[RaitoBezarius]

@tcurdt

But shouldn’t it be possible to compare what the nix expression was built for with the expected status quo and show the differences if found?

You could totally have nixos-rebuild have a step where it will print the generated HW config and compare the configuration fragment and warn you that the system may not boot again.
But it can only be an advisory warning as people can do extremely complicated things, e.g. boot on a system via kexec which cannot lead to simple heuristics as “are you going to reboot on the right disk?” works.

@tcurdt

(Although I am wondering how different the expression is between systems really are for systems to boot. Basically how specialized the expression really is. But I digress…)

Simple:

• You have a system which boots over the NVMe bus.
• You have a system which boots over the SATA bus.

This is going to lead in different kernel modules being required in the initrd.

[tcurdt]

@RaitoBezarius

You could totally have nixos-rebuild have a step where it will print the generated HW config and compare the configuration fragment and warn you that the system may not boot again.
But it can only be an advisory warning as people can do extremely complicated things, e.g. boot on a system via kexec which cannot lead to simple heuristics as “are you going to reboot on the right disk?” works.

In my case I used nixos-anyware to install the base system.

So it sounds like the during the build for the new system (before the reboot) it would be the ideal time. It is already on the machine in question - and it should be able to break the build and tell me this will not work.

The build could run the generate hardware config and compare what is contained in the nix expression. And fail if it is (too) different.

Did I understand that correctly?

@RaitoBezarius

@tcurdt

Although I am wondering how different the expression is between systems really are for systems to boot. Basically how specialized the expression really is.

Simple:

• You have a system which boots over the NVMe bus.
• You have a system which boots over the SATA bus.

This is going to lead in different kernel modules being required in the initrd.

Learning question:
Couldn’t both kernel modules be included in the nix expression?
And during runtime it just picks the one found/needed?
Isn’t that what happens via systemd-udevd?
Or is this different for the initrd?

I don’t see that necessarily changing the nix expression yet.

[RaitoBezarius]

@tcurdt

The build could run the generate hardware config and compare what is contained in the nix expression. And fail if it is (too) different.

Did I understand that correctly?

Yep, whatever you use doesn’t matter. It all trickle down to the same flavor of Nix magic that builds a toplevel and install it. Nix is made to inspect configurations, as long as you have the original .nix or some metadata left behind, you can “re-inspect” things and compare to what should have been the hardware configuration.

@tcurdt

Learning question:
Couldn’t both kernel modules be included in the nix expression?
And during runtime it just picks the one found/needed?
Isn’t that what happens via systemd-udevd?
Or is this different for the initrd?

I don’t see that necessarily changing the nix expression yet.

You can certainly take the union of every possible things you need, do note though there exists hardware configuration that can collide. An installer does exactly what you say, because its role is to work without knowing the target system hardware configuration in advance.

But do note that this can sometimes not work, more examples:

• kernel 6.18 has a regression that makes old systems not complete boot
• kernel 6.18 is required to boot newest AMD CPUs
• you can choose either nvidia or nouveau to boot the graphical stack of an nVidia graphical card, but both are mutually exclusive; some systems may live better with one or the other.

You cannot always get away with putting ALL the modules.

Yes, systemd-udevd, systemd-kmod does many of these things. But certain modules might need to be loaded before they exist and are ready. You cannot always use runtime detection if you need runtime detection to start the runtime detection process.