[unstable] How come it's taking so long for LLVM to land in cache?

Every day since Thursday I’ve been trying to do a nixos-rebuild —upgrade on my 11inch notebook. It’s a low power Intel and I notice an unfamiliar compilation event immediately. Every day it tries to compile LLVM. That’s 3307 compilation steps on Cmake. After about 4.5 hours, we reach step 3304 or 3305 and we enter compilation for swift. That’s 1339 compilation units. And every time, after 2.5 hours

[1188/1339] Building swift module Basic
FAILED: bootstrapping1/SwiftCompilerSources/Basic.o /build/build/swift/bootstrapping1/SwiftCompilerSources/Basic.o
[…]
:0: warning: unable to perform implicit import of “_Concurrency” module: no such module found
/build/src/llvm-project/llvm/include/llvm/CAS/CASReference.h:12:2: note: in module ‘LLVM_Utils’ imported from /build/src/llvm-project/llvm/include/llvm/CAS/CASReference.h:12:
#include “llvm/ADT/ArrayRef.h”
^

Fixing this seems to me a high priority task.

Are you using the master or release branches? I’m too lazy to check if llvm is actually in the set of tests for nixos/nixpkgs, but it sounds like it’d be hard to avoid completely in such a closure.

If it isn’t included, well, that’s why, maybe it should be included :wink:

I’ve just copy-pasted from the manual, so it should be release, shouldn’t it? And the channel doesn’t get upgraded every day, just abt. one day out of two.

If you’re using a release branch you’ll indeed get constant rebuilds. It’s the most common way people achieve this, and why I ask.

Could you share what you copy pasted?

Does that answer your question?

sophroniscus > sudo nix-channel --list
nixos https://nixos.org/channels/nixos-unstable
sophroniscus > sudo nix-channel --list-generations
15 2025-04-01 19:54:44
16 2025-05-26 17:34:49
17 2025-07-21 14:16:43
18 2025-12-03 10:05:41
19 2025-12-03 22:18:57
20 2025-12-06 12:45:20 (current)

/nix/var/nix/profiles/per-user/root
lrwxrwxrwx 1 root root 60 Apr 1 2025 channels-15-link → /nix/store/dm1la82l9fvyxy1hjcabnghlgwxncpyv-user-environment
lrwxrwxrwx 1 root root 60 May 26 2025 channels-16-link → /nix/store/9l05jwx2xb4nwzgf7bny1k6xdvg8y2fw-user-environment
lrwxrwxrwx 1 root root 60 Jul 21 14:16 channels-17-link → /nix/store/gdssjbp45xrl57gb5lhkcs86mpzffmfm-user-environment
lrwxrwxrwx 1 root root 60 Dec 3 10:05 channels-18-link → /nix/store/5rawp7qc3qi87bjgwdfn9c9vrq61yzfg-user-environment
lrwxrwxrwx 1 root root 60 Dec 3 22:18 channels-19-link → /nix/store/010k3y354h9vim5rhv1aa85mi0pqnswq-user-environment
lrwxrwxrwx 1 root root 60 Dec 6 12:45 channels-20-link → /nix/store/gbf7fc508jzk9829z3p2gvp904ir8j4w-user-environment

You’re not using the master or release branch then :slight_smile:

Any flakes, overlays, cuda, or import *fetch* in your config? If not, yep, sounds like the llvm build may need to be added to the test, and I don’t know why it isn’t, sorry for jumping in, just wanted to make sure this isn’t yet another instance of cache misses that shouldn’t be cache misses.

I’ve never needed a cmake build, FWIW, but I also don’t use unstable, so maybe the churn is smaller.

You’re not using the master or release branch then

So what am I using then?
I keep things as vanilla as possible. Every package I use is in configuration.nix, no home profile, no flakes, overlays, cuda, etc.

Please share your config. llvm built succesfully on hydra for the last 6 months…

$ hydra-check --channel=unstable llvm
Build Status for nixpkgs.llvm.x86_64-linux on jobset nixos/trunk-combined
https://hydra.nixos.org/job/nixos/trunk-combined/nixpkgs.llvm.x86_64-linux
✔  llvm-21.1.2  2025-11-23  https://hydra.nixos.org/build/314198986
✔  llvm-21.1.2  2025-11-13  https://hydra.nixos.org/build/313183189
✔  llvm-21.1.2  2025-10-21  https://hydra.nixos.org/build/310412356
✔  llvm-21.1.1  2025-10-06  https://hydra.nixos.org/build/308851828
✔  llvm-19.1.7  2025-09-11  https://hydra.nixos.org/build/306930802
✔  llvm-19.1.7  2025-08-25  https://hydra.nixos.org/build/305612724
✔  llvm-19.1.7  2025-07-28  https://hydra.nixos.org/build/303391815
✔  llvm-19.1.7  2025-07-15  https://hydra.nixos.org/build/302419271
✔  llvm-19.1.7  2025-06-19  https://hydra.nixos.org/build/300614724
✔  llvm-19.1.7  2025-05-16  https://hydra.nixos.org/build/297176837

I am pretty sure, you have something that causes the recompilation.

Also, it might be better to not use --upgrade randomly, but manually advance roots nixos channel occasionally and immediately rebuilding.

Then you can at least rollback the channel easily.

The way you have it now, will advance the channel everytime, and it is hard to know which channel generation is actually still working.

I found what was causing the compilation. I thought I’d removed deadbeef from my package list, but it was in two places actually and last week I removed only one mention. Deadbeef uses a very old version of gtk.

The upgrade process still barfs, but very very close to the end it seems

The following new units were started: lastlog2-import.service,
NetworkManager-dispatcher.service,
sys-devices-LNXSYSTM:00-LNXSYBUS:00-MSFT0101:00-tpm-tpm0.device,
sysinit-reactivation.target, systemd-hostnamed.service,
systemd-tmpfiles-resetup.service

Done. The new configuration is Exception: arity mismatch: arguments
must be 1 value, but is 2 values

Any idea where I should look to fix this?
I keep a log and use dates to know the last working channel.

Thanks a lot guys.

The switch has finished, that error message happened too late to abort the process. It’s why nixos-rebuild boot is advisable, especially for more complex updates, since it is actually atomic, while switch can leave you in partially broken states.

Looks like a bug in nixos-rebuild of some kind. Seems odd to me, wasn’t nixos-rebuild-ng made the default, making this kinda output impossible?

Either way, I imagine this will be gone on future rebuilds after a reboot; the switch is complete, I’d attempt a reboot and check logs for any weirdness afterwards, thrn use it in blissful ignorance until I see the message again.

You’re using the nixos-unstable branch (and in fact channels, which actually makes a lot of this silliness impossible), which is different from master or release-25.11. The latter are the in-development branches, and get commits written to them before any of the software has been built (hydra picks up those commits, builds the world, and then fast-forwards the commits to nixos-unstable or nixos-25.11 after tests pass).

Often, when someone complains about rebuilds, it’s because they saw release-* and assumed it meant this is what they should update to, or because they just picked the master branch without understanding that there’s a bunch of build machinery around the branches.

You did in fact do things correctly, you just threw me a red herring by saying:

That was my first –upgrade since July so no surprise.

The switch has finished, that error message happened too late to abort the process. It’s why nixos-rebuild boot is advisable, especially for more complex updates, since it is actually atomic, while switch can leave you in partially broken states.

I just did nixos-rebuild boot and it took two seconds and the upgrade was done. That’s a really useful bit of information you just gave me.

Thanks a bunch

Note that the actual update won’t be complete until you also reboot :wink:

It only taking two seconds is down to nix already having cached everything, so there’s not much to do.

Note that the actual update won’t be complete until you also reboot :wink:

I always reboot after a major upgrade, since I logout to update the app menus. So why not also reboot?

Well I thought that [unstable] in the thread header said it all. But it seems that master is release’s very own unstable. Have I got this right?

There are two independent axes: stable–unstable and cached–not-yet-cached.

Cached Not yet cached
Stable nixos-XX.YY release-XX.YY
Unstable nixos-unstable master

Always pick a channel from the left column and submit PRs to the right column, regardless of which row you favor.

3 Likes