Nixos-rebuild takes forever

I would like to update my system, but the building does not terminate:

sudo nixos-rebuild test --no-build-output --show-trace --upgrade-all
...
building '/nix/store/nccd5y7kxa9dwqdb4mlzfzxj6a8cmvrr-X-Restart-Triggers-systemd-udevd.drv'...
building '/nix/store/x9256xqzxa527430ra2fiznsh56nlpd8-unit-cups.service.drv'...
building '/nix/store/r84hwllp589y0xy0g9395ajwiay7yi39-unit-systemd-udevd.service.drv'...

After a few hours I cancelled the update.

What can I do?

First step would be to enable build output so that you can actually see what it’s doing.

I’d remove the –no-build-outputoption and add -vL instead.

2 Likes

Thanks @R-VdP for your reply.
I have tried your suggestion:

sudo nixos-rebuild test -vL --show-trace --upgrade-all 1> nixosRebuild_test_upgrade.out 2> nixosRebuild_test_upgrade.err

Now I get an error to build zed-editor:

    zed::tests::test_setting_language_when_saving_as_single_file_worktree
    zed::tests::test_window_edit_state_restoring_disabled

test result: FAILED. 9 passed; 18 failed; 0 ignored; 0 measured; 4 filtered out; finished in 1.48s

error: test failed, to rerun pass `--lib`
error: Cannot build '/nix/store/p5l94zz05ljdzmiigk4q3kh3h91xcfw4-zed-editor-0.218.6.drv'.
       Reason: builder failed with exit code 101.
       Output paths:
         /nix/store/gk5lmfpkmfayg5acgb238pnrqgjnrhi8-zed-editor-0.218.6
         /nix/store/psl1bhanfm0ngjsh313h9d7fdmyc4wvd-zed-editor-0.218.6-remote_server

I will try again without zed.

Without zed the build works.

There should be more logs.

That said, if you’re building zed at all you’re most likely not following nixpkgs. You’re likely overriding something or using a branch that shouldn’t be used. Got the wrong package, sorry, @NobbZ is right.

zed is fine and cached, zed-editor though is currently broken on stable and has been on unstable for a day or 2.

1 Like

Having the same problem the build copiled for like 2-3 hours never ending….

If you are on stable, wait and update in a couple of days again, once the hydra issue is fixed. If you are on unstable make sure to update to the latest nixos-unstable.

PS: roll back your inputs till the next update!

1 Like

I am on stable using the newest inputs:

âžś  sudo nix-channel --list
home-manager https://github.com/nix-community/home-manager/archive/release-25.11.tar.gz
nixos https://nixos.org/channels/nixos-25.11

âžś nix-info               
system: "x86_64-linux", multi-user?: yes, version: nix-env (Nix) 2.31.2, channels(root): "home-manager-25.11.tar.gz, nixos-25.11", nixpkgs: /nix/store/vxz0kx0crh7k4mp2zcrjihb8myj4a9h2-nixos-25.11/nixos

Why are there unstable packages in stable?

I have no clue why you think that there were unstable packages in stable. You are in stable, where zed-editor is currently broken. Please roll back your inputs and wait for a couple of days, till the build is fixed.

Maybe my wording is not correct:
Why are there broken packages in stable?

I would expect that all packages in stable can be compiled without an issue, at the price of not having the newest version.
But maybe I do not understand what stable means in this context.

Would not rolling back inputs impact all packages?

The stability guarantee of stable is that there will be no “breaking changes”. Laypeople often misunderstand the term - this doesn’t mean that no bugs will be introduced, but that updates will not be backwards-incompatible.

I.e., if you update, you won’t have to change your configuration, do irreversible data migrations or suchlike. What precisely that means depends a bit on the software you use, browsers for example generally have no such stability guarantees, but changes to nixpkgs.lib are only applied on NixOS version updates (e.g. 25.11 → 26.05).

zed-editor specifically uses the rust ecosystem’s convention of saying that their software has no backwards-compatibility guarantees (for now) by setting the major version number to 0.

This does not say anything about whether package updates contain bugs that may or may not cause the software to be unable to be built or otherwise be unusable/break. It’s inherently impossible to know ahead of time if a software update will introduce bugs, if we could know this bugs would not exist.


In theory, PR authors could (and should!) at least check that their commits build, but in practice things slip through because many just create automatic backport PRs (for minor version bumps, such as this one). Unfortunately nixpkgs is so big that CI can’t be used cost-effectively to assert every package fully builds, so there are no real gates against this except code review, and there isn’t enough manpower around; committers typically just merge if a package build seems to have worked for reviewers.

Typically such issues are found and fixed within days, though, so you can just revert and run the last commit for a bit longer. This does affect all software; if there’s a critical CVE or something in other software you’re SOOL - you could isolate the package from an old commit if you desperately need to, though.

2 Likes

Stable means that there will be no breaking changes (unless to fix security issues).

It does not mean “builds at all times”. Human error happens, and even worse, it’s not always the merge of X that breaks X, sometimes it’s completely unrelated. Sometimes it’s the update of a library, which should still be compatible but X has a version check that now fails.

Nixpkgs is far too huge to not have problems of this kind.

Sometimes shit just hits the fan, and than someone has to clean up, everyone else stays out of the room for a while, or tries to help cleaning.

2 Likes

To be fair, in this case it was clearly just a version bump that nobody cared to actually build locally. If you don’t want this kind of stuff to happen, you could sign up to help maintain the package @klt :slight_smile:

1 Like

A bit of context for the full picture from one of the two zed-editor package maintainers:

  • we’ve recently begun backporting all updates for zed-editor to stable because of its collaboration features (that expect users to have a somewhat close version for compatibility) and reliance on external services
  • independent of that, 0.218.5 fixed two high severity CVEs, so it would have been backported in any case, and then we would also have backported 0.218.6 because it’s just a patch update (which caused the test failures on both unstable and stable)
  • the update PR breaking the package (0.218.5 → 0.218.6) was a patch update, these usually have so little changes that I don’t have much reason to believe they could break, and the PR was opened by someone who had submitted multiple update PRs for zed-editor in the past; the author stated it built on x86_64-linux and I took their word for it because building zed-editor takes a lot of resources
  • similarly, there wasn’t much reason to believe the backport wouldn’t build as release-25.11 and unstable are still on the same Rust version as of now, and the (somewhat trusted) author of the original update PR stated they had built it on x86_64-linux

The build failure itself on both unstable and 25.11 is due to upstream using a different test runner. We have fixed both channels now by switching the package to cargo-nextest, and the fix has already landed in nixos-unstable, though there was no eval to nixos-25.11 since then, so it will still take a bit of time to reach stable users.

In the end, a huge source of the current issues stems from the amount of resources needed to build zed-editor, which have grown immensely since we adopted the package in nixpkgs.

I have a very resourceful desktop machine (Ryzen 7950X3D with 64 GB DDR5) and that still needs more than an hour to pass build and checks. I stopped testing darwin on most zed-editor PRs because my Macbook only has 18 GB of RAM, thus leaning heavily into swap memory to build zed-editor, which causes the build and checks to take at least 2.5 hours on my M3 Pro. Maintainers of browser packages such as Firefox or Chromium could say that’s bearable, but I currently can’t expend the resources to keep up with these build times on my personal machines with the high update interval that zed-editor requires.

If someone wants to help in reviewing zed-editor PRs and has the resources for it (which ideally means 64gb of RAM, or 32gb + swap + patience), then please do step up and become a maintainer.

8 Likes

I currently can’t expend the resources to keep up with these build times on my personal machines with the high update interval that zed-editor requires.

You can request access to Community builders - Nix Community for this use case.

1 Like

Thanks for your help and yll your lengthy comments.
Now I understand a bit better (but still not completely) what happened.
nixos-rebuild works again with zed-editor.