Help! I cannot update NixOS anymore because suddenly webkitgtk-2.42.3+abi=4.1 and webkitgtk-2.42.3+abi=6.0 needs to get manually compiled. My laptop does not have enough RAM (only 32 GB) for it to be compiled. What should I do? This started happening suddenly.
I am running a flake based system. Here is the whole flake repo.
When I update, I do:
nix flake update
sudo nixos-rebuild switch --flake .#asus_gnome
I have good luck using the --max-jobs (otherwise known as -j) flag to control the concurrency that Nix uses to rebuild.
Using -j 1 will usually get out of these jams, albeit slowly.
Let me try this. But I thought the limiting factor was memory, not CPU.
Reducing concurrency also reduces the amount of memory required. A ratio of about 2 GiB of RAM per assigned thread/core works well for most cases. I have no clue about WebKit though
Well, using --max-jobs 1 is also causing a failure.
(I did not press any key, it just died on its own)
Usually when something needs to be compiled downstream your nixpkgs version is simply not one for which hydra has artifacts:
url = "github:NixOS/nixpkgs/release-23.05";
Don’t use the release branches, use nixos-23.05. Afaik the release branches are what people apply patches to, you should not expect artifacts to be built for that branch, that happens on the nixos-* branches, along with all testing.
Those branches should maybe be renamed to dev-23.05 or something, then it will be more clear that you will from time to time hit things that haven’t been built by hydra yet.
Also, 23.11 is out now, I think 23.05 will be EOL next week, so upgrade. I don’t think artifacts for older nixpkgs versions are usually deleted, at least not currently, but still, 23.05 doesn’t really get security patches anymore.
Yes this did it. Thank you so much. Updated to 23.11 as well.
So, this issue has been solved by picking the right nixpkgs branch, but I still think this is worth commenting on. There’s an important difference between --max-jobs and --cores. The max-jobs setting controls how many individual derivations Nix will build at a time simultaneously. The cores setting will inform (not control) those derivations how many threads they ought to launch. For instance…
My personal Hydra configures its builder with cores = 16 and 2 max-jobs. Theoretically, a Threadripper 1950X is capable of more than this (because most builds don’t use every core they can), but containing it to 2GiB of RAM per thread is the only way to keep it from crashing when it builds a few webkits at the same time.
Sorry, I know this is not OP’s issue. Just some interesting stuff to think about.
Do you know how to specify the RAM limit?
There basically isn’t one. The best you can do is systemd restrictions on nix-daemon.service, but that’s not very nuanced. It helps though
Borrowing a term used elsewhere, releng-XX.YY would indicate this is the release engineering branch, not the release branch, and perhaps save a little of this confusion.
There is no RAM limit that you need to specify. The “2 GiB per thread” is a rule of thumb to get you an estimate of the allowed concurrency.
When we say this, we mean, that if you have 4 GiB of RAM, then you can have a concurrency of 2, if you have 16 GiB of 8, and so on. You should never increase this above your CPUs thread count though.
And as someone already sayd, you have 2 values to adjust the concurrency, and depending on the usecase, its worth to play with them under different circumstances.
Try mounting an external SSD as an extra swap partition. Your build will be slow, but you should be able to overcome your RAM limitations.
So why is cache.nixos.org missing a build of webkitgtk? The latest version is webkitgtk-2.42.4+abi=4.0
I am using unstable, but still I have this problem, since 24.05 was released. I try to go back to 23.11, it still happens. the package in 23.11 has also been updated to the newest version. I don’t want to go back to 23.05, what should I do?
Did you read my reply? This was about someone who was using a release branch, which doesn’t get updated after initial release of the NixOS version. I.e., replace Erm, sorry, clearly skimmed past the first bit of your reply.release with nixos.
If you have, which package is missing, and what are your flake inputs/channels? Do you have any overlays or overrides? Definitely don’t downgrade to an OS that has had no updates for a year.
Finally I found what caused this problem. The only overlay is a libwacom.
nixpkgs.overlays = [
(self: super: {
libwacom = self.callPackage (self.path + "/pkgs/development/libraries/libwacom/surface.nix") {
inherit (super) libwacom;
};
})
];
I don’t understand why a libwacom will cause this
Certainly unexpected:
$ nix why-depends --derivation nixpkgs#webkitgtk_4_1 nixpkgs#libwacom
/nix/store/qlssm5wfpp7m5cwga694svg4m2f5mkis-webkitgtk-2.44.2+abi=4.1.drv
└───/nix/store/lvk5ws5ps5qgivl7a7mpchz3hf69vs5d-gst-plugins-bad-1.24.2.drv
└───/nix/store/szmxz3zp6gvlk5nnwpijwljash59maii-openal-soft-1.23.1.drv
└───/nix/store/733lrv3nq01pls1g0v6glabvj0as1c4j-pipewire-1.0.6.drv
└───/nix/store/0nwsg3qp0pcpv07hqgcxnnas4nj6ilmm-ffado-2.4.8.drv
└───/nix/store/3pbldq7gyfynwpaaq3x190a4grzx379g-wrap-qt5-apps-hook.drv
└───/nix/store/7nax802rrslrxrk4cd0n5gckndq0b1y7-qtbase-5.15.12.drv
└───/nix/store/x5d3v4v9yz4359m787gz24z3pwikxckf-libinput-1.25.0.drv
└───/nix/store/43f0x1f197blniqgj6ka0i163j42xsin-libwacom-2.11.0.drv
(And there’s more if you use --all)
But it’s a build-time only dependency, so that’s unfortunate.
There is ffado: Remove Qt dependency by jtojnar · Pull Request #306407 · NixOS/nixpkgs · GitHub but I think I want to try asking upstream for something cleaner.