Issue building linux kernel modules after Flake update

Help
1

I’m unable to build my nixos config on any computer after updating the flake to the latest revision. Each build gets stuck at the same place with the same error:
“error: path ‘/nix/store/9y7z0wj024np8zrhwvpz6ph6ksckiwgy-linux-6.12.20-modules-shrunk/lib’ is not in the Nix store”

To perform the update I ran nix flake update then nixos-rebuild as below. I’ve tested three systems built from the same flake and all fail at the same point.

Nix version: nix (Nix) 2.18.5, flake channel is set to nixos-unstable

The path exists, and has a folder structure that looks more or less correct.

So far I’ve tried:

  • changing kernel versions to the latest in case it was package specific
  • nix-store --gc and then rebuilding
  • manually removing and adding the path to nix-store

Not sure what else to do, or if I’m simply missing a step.

More complete output:
❯ nixos-rebuild switch --flake ./nixos#desktop --verbose --upgrade
$ nix --extra-experimental-features nix-command flakes build --out-link /tmp/nixos-rebuild.sREbZC/nixos-rebuild ./nixos#nixosConfigurations.“desktop”.config.system.build.nixos-rebuild --verbose
$ exec /nix/store/4j5la8m214m57xs81bfq9ixivky507fs-nixos-rebuild/bin/nixos-rebuild switch --flake ./nixos#desktop --verbose --upgrade
building the system configuration…
Building in flake mode.
$ nix --extra-experimental-features nix-command flakes build ./nixos#nixosConfigurations.“desktop”.config.system.build.toplevel --verbose --out-link /tmp/nixos-rebuild.b5Dze6/result
these 4 derivations will be built:
/nix/store/p4ng98h6yvsc543125976rccy4ycl274-closure-info.drv
/nix/store/fyv39zyaxny6p49pkj8bi81518x2ffz0-initrd-linux-6.12.20.drv
/nix/store/dsmh6vzhfgpi5p61d5fjhnazg1y4k2c9-boot.json.drv
/nix/store/0xnmgs1754yaiv85nrf5bln8p55a9l33-nixos-system-desktop-25.05.20250327.5e5402e.drv
error: path ‘/nix/store/9y7z0wj024np8zrhwvpz6ph6ksckiwgy-linux-6.12.20-modules-shrunk/lib’ is not in the Nix store

1 Like
2

I also ran into this, fixed temporarily by switching nixpkgs to 24.11 in my flake but seems like there might be some issue with unstable.

I tried everything you listed as well as reimaging a new machine and building from scratch but that didn’t work either

edit: my configs are here for reference: GitHub - Samiser/nix-configs: my nix stuff

You should be able to reproduce the issue by trying to build the nix-lab nixosConfiguration, but weirdly garnix seemed to build it successfully last time I pushed changes (might have been built before a breaking change)

3

Thank you, switching to 24.11 does get the system to build. I’d still like to understand exactly what is causing the issue but at least I have a functional system.

4

I just switched back to nixpkgs-unstable and I’m no longer able to reproduce the issue, the build just works. Is it still an issue for you? Also can you post your flake/configs?

5

it’s to do with using the old version of nix, somehow i must have upgraded my version of nix resolving the issue

tested via podman run -it nixos/nix:2.18.5 sh

[nix-shell:~]$ cat flake.nix
{
  inputs = {
    nixpkgs.url = "github:NixOS/nixpkgs/nixpkgs-unstable";
  };

  outputs = {nixpkgs, ...}: {
    nixosConfigurations.minimal = nixpkgs.lib.nixosSystem {
      system = "x86_64-linux";
      modules = [
        {
          boot.loader.grub.enable = false;
          fileSystems."/" = {
            device = "none";
            fsType = "tmpfs";
          };
          system.stateVersion = "25.05";
        }
      ];
    };
  };
}

[nix-shell:~]$ nix --version
nix (Nix) 2.18.5

[nix-shell:~]$ nix build .#nixosConfigurations.minimal.config.system.build.toplevel --extra-experimental-features nix-command --extra-experimental-features flakes
error: path '/nix/store/8lbv65x6gmjmrigpn7xb37wxdhb08whw-linux-6.12.20-modules-shrunk/lib' is not in the Nix store
6

I guess the logical question is… how do I update nix before updating the rest of the system? I definitely suspected it might be the nix version.

7

if you have either a different host with a newer nix version or docker/podman you could build it in the environment with a newer nix version then copy out the build configuration and switch to it with ./result/bin/switch-to-configuration switch?

8

So, I was able to reproduce this with Nix 2.18.5, but not the latest 2.18.x which is 2.18.9. So whatever the bug is, it was fixed by then.

@eureka-cpu Reported something similar, and in the revision of nixpkgs they said they were using before, Nix 2.18.5 was the default Nix version. So presumably that’s why they were affected. (Sidenote, that nixpkgs version is from july, so it must have been a while since you updated!)

Nonetheless, clearly nixpkgs has recently made a change triggering this bug. So I decided to bisect nixpkgs, and found that this was the culprit: make-initrd: use closureInfo again by xaverdh · Pull Request #372931 · NixOS/nixpkgs · GitHub

So there must have been some bug with exportReferencesGraph that was fixed before 2.18.9. I’ve also tested it with Nix 2.3 (which is currently the oldest version of Nix that NixOS is required to remain compatible with) and it does not seem to exhibit the bug. So I don’t think that PR needs to be reverted. I think users simply need to be advised not to use obsolete versions of Nix, e.g. 2.18.5 when 2.18.9 exists.

3 Likes
9

This whole journey started when I realised I had misunderstood how nix updates work so yeah it had been a while.

Thank you for taking the time to replicate and help me understand the issue.

My understanding is that this is kind of a catch 22 scenario because in order to upgrade nix, I would need to be able to do a nixos-rebuild switch right? So in terms of avoiding this in the future, is there a way to run the latest version of nix regardless of whats in the store? For example with a ‘nix shell’ command or something?

10

I appreciate the follow up! Thanks very much. I was able to update to Warbler at least, I’m now at Nix 2.25.5, and I’m going to try to update to latest from here.

11

I’m not sure how well this would work but there is at least a latest version of nix the package in nixpkgs: nix - MyNixOS
Then you could make a shell.nix like so:

{ pkgs ? import (fetchTarball "https://github.com/NixOS/nixpkgs/archive/nixos-unstable.tar.gz") { } }:
pkgs.mkShell {
  packages = with pkgs; [
    nixVersions.latest
  ];
}

Just tried it locally. Without the shell I’m at nix 2.25.5, and with it I’m at 2.26.3

12

@ElvishJerricco after updating where my lock file was at on one of my other machines, I was able to update to latest unstable. Thanks again!

13

Thank you for the nix-shell thats a good work around and it works for me.

14

Hi, I’m running into this issue with Nix 2.28.1.

I’m getting the same error, it seems:

❯ nix build .#nixosConfigurations."nixos-work".config.system.build.toplevel
warning: Git tree '/home/felix/nixos' is dirty
error: path '/nix/store/pdi9qqvmgb8zwc0j7v1h0j7zpf43v424-linux-6.12.22-modules-shrunk/lib' is not in the Nix store
❯ nix --version
nix (Nix) 2.28.1

I got a shell with this nix version using

❯ nix shell github:NixOS/nixpkgs/nixpkgs-unstable#{nix,nixos-rebuild}

Without that, I have:

❯ nix --version
nix (Nix) 2.18.5

So quite a bit older, and, it seems, the broken version.

But using a shell with a newer nix version didn’t solve the issue for me, is there something I’m missing, or any other hints? Thanks!

15

The problem lies in your nix daemon, I believe, not the nix version in your shell. You’ll need to update the daemon. For instance, you can update to the latest version of 24.11, which should be on a newer version of 2.18 which doesn’t have the issue. Then you should be able to update to anything from there.

1 Like
16

Thanks! that was the missing piece! I hadn’t used this system in ~6 months, but had updated my config all the time because of other systems. So I checked when that commit you bisected (thanks so much! :pray:) was merged, which was March 10, and switched to a git rev of my config before that. Then I was able to switch to that, nix-daemon --version showed an updated version, and then I was able to switch to current nixos-unstable again :slight_smile: Thanks so much for writing all of this down, going the extra mile to bisect, and for helping me out! :slight_smile:

17

For me the issue occured on a thin client that had been installed freshly with an older 24.05 version of NixOS.

What I did to solve it was the following. First I switched to the unstable channel

sudo nix-channel --add https://nixos.org/channels/nixos-unstable nixos

after that I updated the system using

sudo nixos-rebuild-switch --upgrade

with that being done nixos-version reported the new version of NixOS was running now. After that

sudo nixos-rebuild switch --flake .

also worked again.