With today’s update of NixOS 26.05 (34268251cf55), I observed something strange: While “nixos-rebuild” downloaded most packages from the binary cache as usual (many many packages, as today’s update contains a staging merge staging-next-26.05 iteration 1 - 2026-06-06 by vcunat · Pull Request #528906 · NixOS/nixpkgs · GitHub ), there are some packages that it insists on rebuilding locally, although they have already been built by Hydra. Some examples:
The store hashes of the “.drv” files of those Hydra builds are exactly the same as those that my Nix insists to build locally. Totally, it probably tries to build 10 or 30 packages that it should normally download from the binary cache. Most other packages (several thousands) are actually downloaded from the binary cache, so that should work in principal.
I have observed that problem one three machines now, and it seems to be the same packages that are affected (although those machines don’t need the exact same package set while updating).
All those machines use NixOS 26.05 as channel. They are not using flakes.
I’m wondering what’s going on here. Does anyone have an idea on how I could diagnose that problem further?
This is most likely consequence of bugs we had in the infra. We’ve been trying a rewritten queue-runner, and it’s got some rough edges so far. Thanks for the feedback
My understanding is that uploading of new results is hopefully reliable now, and we did restart lots of failures where other package was failing to build due to such a missing binary (dependencies). But this approach doesn’t cover (all) leaf packages.
Thanks, vcunat, for the explanation! If I understand correctly, the missing packages will likely stay missing, but updates will hopefully work again normally as soon as the next staging cycle caused a full rebuild.
In case I observe missing packages again that should have been rebuilt after today, I will report back again.
And thanks for keeping the infrastructure running, by the way!
I don’t know whether anyone is even looking at it. Probably open an issue? (and/or try finding the problem) Or let me ping @booxter here, as they’re maintaining the darwin variant.
