NixOS Flakes:Rebuild not using subsitution despite package being available in cache.nixos.org

Hey everyone,
I recently moved to flakes, but no matter what I do, NixOS keeps rebuilding over 1000 packages from source (which will take a few days, even on my beefy processor) despite Hydra already having binary substitutes for those packages.
With the following flake.nix:

{
  description = "NixOS System Configuration";

  inputs = {
    nixpkgs = { url = "github:NixOS/nixpkgs?rev=16448f9e76cf207a331e397b33b3e059b81fce0d"; };
    nur = { url = "github:nix-community/NUR"; };
    home-manager.url = "github:nix-community/home-manager";
    home-manager.inputs.nixpkgs.follows = "nixpkgs";
  };

  outputs = inputs@{ nixpkgs, home-manager, nur, ... }:
  {
    nix.registry.nixpkgs.flake = nixpkgs;
    nixosConfigurations = {
      laptop = inputs.nixpkgs.lib.nixosSystem {
        system = "x86_64-linux";
        modules = [
          ./configuration.nix
          home-manager.nixosModules.home-manager
          {
            home-manager.useGlobalPkgs = true;
            home-manager.useUserPackages = true;
            home-manager.users.prim = import ./home/prim/home.nix;
          }
        ];
        specialArgs = { inherit inputs; };
      };
    };
  };
}

yields to:
[15/33/1237 built, 5/34/43 copied (1036.4/1743.8 MiB), 544.0/755.0 MiB DL] building ghostscript-with-X-9.56.1
despite ghostscript and all most other mentioned packages being available on cache.nixos.org
(notice the revision of nixpkgs, this is from 8 days ago, but even if I use nixos-unstable it rebuilds everything from source)
The following nix config is in place (default):

accept-flake-config = false
access-tokens =
allow-dirty = true
allow-import-from-derivation = true
allow-new-privileges = false
allow-symlinked-store = false
allow-unsafe-native-code-during-evaluation = false
allowed-impure-host-deps =
allowed-uris =
allowed-users = *
auto-optimise-store = true
bash-prompt =
bash-prompt-prefix =
bash-prompt-suffix =
build-hook = /nix/store/gi0aq6w348vfxs3ggxl9jpg25ixp44kd-nix-2.11.0/bin/nix __build-remote
build-poll-interval = 5
build-users-group = nixbld
builders =
builders-use-substitutes = false
commit-lockfile-summary =
compress-build-log = true
connect-timeout = 0
cores = 0
diff-hook =
download-attempts = 5
download-speed = 0
enforce-determinism = true
eval-cache = true
experimental-features = flakes nix-command
extra-platforms = aarch64-linux i686-linux
fallback = false
filter-syscalls = true
flake-registry = https://channels.nixos.org/flake-registry.json
fsync-metadata = true
gc-reserved-space = 8388608
hashed-mirrors =
http-connections = 25
http2 = true
ignore-try = false
ignored-acls = security.csm security.selinux system.nfs4_acl
impersonate-linux-26 = false
keep-build-log = true
keep-derivations = true
keep-env-derivations = false
keep-failed = false
keep-going = false
keep-outputs = false
log-lines = 10
max-build-log-size = 0
max-free = 18446744073709551615
max-jobs = 20
max-silent-time = 0
min-free = 0
min-free-check-interval = 5
nar-buffer-size = 33554432
narinfo-cache-negative-ttl = 3600
narinfo-cache-positive-ttl = 2592000
netrc-file = /etc/nix/netrc
nix-path = nixpkgs=/nix/var/nix/profiles/per-user/root/channels/nixos nixos-config=/etc/nixos/configuration.nix /nix/var/nix/profiles/per-user/root/channels
plugin-files =
post-build-hook =
pre-build-hook =
preallocate-contents = false
print-missing = true
pure-eval = true
repeat = 0
require-sigs = true
restrict-eval = false
run-diff-hook = false
sandbox = true
sandbox-build-dir = /build
sandbox-dev-shm-size = 50%
sandbox-fallback = false
sandbox-paths = /bin/sh=/nix/store/b0ypzsib8ymwgi9a7rl3m4fxni83wdai-busybox-static-x86_64-unknown-linux-musl-1.35.0/bin/busybox /nix/store/wisbr6daj4ih1751wylkhf66bxfww9rs-qemu-aarch64-binfmt-P-x86_64-unknown-linux-musl /run/binfmt
secret-key-files =
show-trace = false
stalled-download-timeout = 300
store = auto
substitute = true
substituters = https://cache.nixos.org/
sync-before-registering = false
system = x86_64-linux
system-features = benchmark big-parallel kvm nixos-test
tarball-ttl = 3600
timeout = 0
trace-function-calls = false
trace-verbose = false
trusted-public-keys = cache.nixos.org-1:6NCHdD59X431o0gWypbMrAURkbJ16ZPMQFGspcDShjY=
trusted-substituters =
trusted-users = root
use-case-hack = false
use-registries = true
use-sqlite-wal = true
user-agent-suffix =
warn-dirty = true

Is your configuration.nix available anywhere? Most relevant would be what overlays if any that you’ve added.

Would post it, but heavy redaction would be needed,
I just noticed that using rev 2a04057664defeb0ce971b66a0da274b55d3ffbb from 15 days ago solves this. Now I am kinda confused, shouldn’t nixos-unstable branch only be updated once hydra cached most pkgs? 98% of my used pkgs and services are stock, only three overlays:

    package = with pkgs;
      zerotierone.overrideAttrs (old: {
        cargoDeps = rustPlatform.importCargoLock {
          lockFile = fetchurl {
            url =
              "https://raw.githubusercontent.com/zerotier/ZeroTierOne/${old.version}/zeroidc/Cargo.lock";
            sha256 = "sha256-pn7t7udZ8A72WC9svaIrmqXMBiU2meFIXv/GRDPYloc=";
          };
          outputHashes = {
            "jwt-0.16.0" =
              "sha256-P5aJnNlcLe9sBtXZzfqHdRvxNfm6DPBcfcKOVeLZxcM=";
          };
        };
      });
nixpkgs.overlays = [
    (self: super: {
      dbus = super.dbus.overrideAttrs (old: rec {
        version = "1.14.0";
        src = pkgs.fetchurl {
          url =
            "https://dbus.freedesktop.org/releases/dbus/dbus-${version}.tar.xz";
          sha256 = "sha256-zNfM43WW4KGVWP1mSNEnKrQ/AR2AyGNa6o/QutWK69Q=";
        };
      });
    })
  ];

  nixpkgs.config.packageOverrides = pkgs: {
    vaapiIntel = pkgs.vaapiIntel.override { enableHybridCodec = true; };
  };

That commit seems to be master only, I’d not sure if everything of it is built.

Please use a commit from an actual channel and try again.

As I wrote already, that does not fix it. Using latest nixpkgs/nixos-unstable commit 093268502280540a7f5bf1e2a6330a598ba3b7d0 triggers a rebuild of > 1000 packages for me

Yet again, GitHub says this commit was master.

How do you get those revs?


I checked it myself and indeed that commit is the most recent on nixos-unstable… Seems as if GH now only displays a single branch, not all anymore when viewing a commit…

Sorry for the noise, just ignore me…

Overlaying dbus would cause all of its direct and indirect dependencies to be rebuilt. I guess you can try removing that overlay and doing a dry-build to see if the build amount is cut down.

$ nix-store -q --referrers-closure /nix/store/...-dbus-1.14.0/ | wc -l
755

Uh, yea, overlaying dbus is going to rebuild a lot of packages lol.

1 Like

THANK YOU! Removing the overlay fixed it! I don’t even remember why I put it there …