I’m fairly new to Nix OS, and I am having some trouble rebuilding my system. This has been an ongoing issue for me for months in various ways, and I suspect I’m just doing something altogether wrong. The general situation goes like this:
it’s been a month or so since I’ve updated my system, so I nix flake update and sudo nixos-rebuild switch –flake <my-flake-dir>.
the rebuild starts, and as I understand it, somewhere along the way a cache miss occurs, and my laptop starts trying to rebuild something (it’s often been librewolf, though it happened a bit ago for mpv. Most recently, I figured out it was triggering a build of python3 as well as numpy, pandas, and a handful of other python libraries all because I had services.pipewire.alsa.enable32bit set to true.
Over the course of several hours, my laptop gets extremely warm, freezes up completely (due to the memory and cpu’s being completely maxed out), and the whole thing crashes. No rebuild for me. Unfortunately, now that I’ve updated the flake.lock, even if I wanted to change something innocuous on my system (e.g. the timezone) any rebuild will now trigger this process and so can’t complete. As I understand it this can be rolled back somehow though I’ve not yet learned how to do so.
I clunkily figure out what package is causing the builds, say aloud "ugh…I guess I can live without that for a while?” remove it from my config, and pray that whenever I end up needing it whatever has been happening has been resolved.
What I’ve tried:
adding the community Cachix instance to my substituters. So far as I can tell, this did nothing.
EDIT: forgot to add these VV
adding different versions of the desired package (for example librewolf-unwrapped or librewolf-bin) to my permittedInsecurePackages.
the above-mentioned “just remove it for now and pray it is available later” strategy.
So, my questions:
Is there some way I can blacklist packages from rebuilding on my system? Or even forbid builds entirely? I don’t think my laptop can handle them.
Is all of this indicative of some misconfiguration in my system, or is this all intended behaviour? It seems very odd to me that a normal failure mode for a simple system rebuild is to, without prompting the user, take on an insane rebuild that your system cannot possibly complete.
Why is the option services.pipewire.alsa.enable32bit alone (and not any of the other pipewire options I also have set) causing a rebuild?
Thanks for any help. Links are of course welcome, and I’ve tried my best to read around about these topics, but there are so many competing wikis, websites, blog posts, and books, many of which often give me erroneous information, I’m not sure what sources I can trust for learning Nix.
I’d guess you have some nvidia overlay or such, but share your config and we can identify it.
librewolf was uncached for a while, and browser builds are rough. It should be cached as of your next update, though.
Maintain your config in a git repo, and commit often. Then you can just revert a bad commit easily. Nix will not help you with this, it’s just a set of text files that you need to manage externally.
That’s not a failure mode, because nix is a build system and therefore designed to administer builds. The fact that there’s a huge cache run by/for NixOS is tangential to that. You can limit cores/jobs to reduce the load on your system and make it more likely to complete a build, at the cost of speed.
We don’t know what it’s rebuilding without seeing your config.
Thanks for your reply. I’ve attached my config here. As you’ll see I’ve commented out various things to try and single out the problem, and I’ve labelled the problem line, which is in conf.nix. To be clear, as of now the rebuild goes through just fine with that line commented out, but triggers a whole pile of python builds if I uncomment.
The hyprland cachix isn’t doing you any good if you’re not using their flake. So you can remove that.
I also don’t see any overlay, but you also commented out all your packages, so hard to tell what would obviously get rebuilt.
Again, though, if the concern is librewolf then it will be cached soon, see Librewolf Marked as Insecure - #3 by mBornand. (Insecure packages are not cached by hydra, as a policy.)
Ok, I’ve edited my packages back in. Still, the problematic line is services.pipewire.alsa.support32Bit, which I figured had nothing to do with which packages were installed. It’s nice to see that librewolf will be cached soon, though at this point my concern is with this general issue as a pain point for nix as whole, because I’ve now experienced it for three seemingly unrelated packages, so I want to figure out what I can do if/when this happens again.
EDIT: on a related note, how do I check the status of caching and https://cache.nixos.org/? When I go there myself I only find a simple page describing the site, and https://status.nixos.org/ takes me to commits for the channel I choose, but that doesn’t seem to give any package-specific information.
An aspect of configuring an OS with a programming language is that it pushes you to develop compsci/dev skills in order to do so effectively. (Whether or not this is a desirable aspect is user-specific and best left to discuss in a different thread if so desired)
It turns out that this option does relate to which packages your system profile depends on (“install” has no coherent meaning in NixOS). For services.pipewire.alsa.support32Bit, it’s best to lookup the option source.
There are many ways to find the source, but man configuration.nix will tell you that this option is declared in <nixpkgs/nixos/modules/services/desktops/pipewire/pipewire.nix>.
The result of this investigation is that enabling this option is expected to force you to compile packages at this time due to infra limitation on the ability to support i686.
Because nix is input-addressed, you can know the output store path before the derivation is actually built. Practically speaking, since the build system already knows the output store path, it can then use this store path as the key for a cache lookup and simply substitute it rather than build.
It’s nice to see that librewolf will be cached soon, though at this point my concern is with this general issue as a pain point for nix as whole, because I’ve now experienced it for three seemingly unrelated packages, so I want to figure out what I can do if/when this happens again.
Something you could do if you encounter a package that has caching issues, but you still want to update the rest of your system, is add another input to your flake that points to the commit that you were using before the update, and then create an overlay so that you can refer to it in your configuration. I wish i could share an example, but i don’t have it on hand right now since haven’t had the need to do this for a while. plus i’m mainly using Guix right now, which handles it differently, but you should be able to find some examples online (or maybe someone else can chime in).
perhaps not, depends on how old your previous generation is. If you don’t want to or can’t build it yourself, you’re still running an outdated browser if you don’t update at all.
You should probably remove that entire block, not just the cachix substituter. This is:
Giving all your wheel users passwordless sudo with zero logging or asserting that an actual human is performing the task
Hard-coding a key for the NixOS cache that would best be invalidated through upstream definitions, not your own manual edits
Trusting two extra keys you clearly don’t actually have to trust because you’re not intentionally using packages from their associated caches
Slowing down almost everything you do with nix by adding additional http queries to extra caches you don’t need
Generally, you’re almost always better off using the nixpkgs versions of nix-community packages than using their flakes + the community caches.
Even if you insist on using the flakes (why are people so obsessed with auto updates for bleeding edge? You probably don’t even know what features are added, let alone make use of them, and you are gaining like a week or two at best), adding the community caches will slow down your overall builds, often sufficiently that any gain from avoiding a (typically pretty small) package build once every few weeks is outweighed by the extra http queries.
I just checked and i’ve had "@wheel” inside my config for the longest time now. What is the recommended way to go about this? should i just add my specific user to trusted-users instead?
instead, edit root /etc/nix/nix.conf or from nixos settings and
add binary cache public key to trusted-public-keys
add binary cache url to trusted-substituters (used only if additionally enabled via flake or user nix conf) or substituters (used every time, slows down lookup a bit)
CAUTION! Remove everything but root from trusted-users! It allows arbitrary code execution as root sometimes even in nix build, like that random flake you used last week.
Don’t set trusted-users at all, frankly - that just gives a user (and programs running as that user) passwordless and non-interactive root. That’s the easiest way to add a malicious cache, among other things.
Swap increases I/O thrashing and zram increases memory pressure as it reduces available ram (needing CPU cycles to flip back and forth between ram and zram). Just don’t use these at all, especially for a browser build.
You can also use systemd’s resource allocation to limit how much the nix daemon gets, e.g. by setting services.nix-daemon.serviceConfig.CPUQuota to set max cpu usage (note that the max is (# cores) * 100%), or services.nix-daemon.serviceConfig.OOMScoreAdjust to make it more likely to kill the nix daemon than other processes when under memory pressure. earlyoom may also be of interest.
Yeah, as @waffle8946 says, use sudo if you need to do something that can only be done by a trusted user. You don’t need to be a trusted user for most things you might want to do with nix.
I think there are two notable situations in which people set that setting:
If you use a flake that asks for permissions to toggle features that can only be toggled by trusted users through the nix-settings output, it tells you that you need to be a trusted user, so people add this without thinking.
Just… Don’t do that. It’s horrible UX that nix recommends such poor practice without telling you exactly what it implies.
The only use case for this feature is temporarily adding community caches, which again, you should probably not be using to begin with. People largely do this only because it’s possible; the only sensible use of that option is enabling additional experimental features.
Deploying to a remote with --target-host requires either trusting the ssh-ing user or for the derivations to be signed.
This is a kinda legit use case, but you can just sign your derivations instead to avoid having a second root user.
Hmm, i see, well i definitely removed my trusted-users stuff now that i know this, but @brainrake mentioned i could put my additional caches inside the system-wide nix config instead, but if i’m understanding correctly, you’re saying i shouldn’t use additional caches at all? At the moment i’m only using a cache from nix-community, which i think i added for NUR, but i’m not sure anymore tbh lol.