Nixos-rebuild builds same package twice at the same time

Help
1

I used sudo nixos-build boot –max-jobs 3 and found that it’s trying to build the same package in 2 jobs at the same time. Here is what I see in the /nix/var/nix/builds directory:


[root@nixos-laptop:/nix/var/nix/builds]# ls nix-162532-204011482*/build
nix-162532-2040114821/build:
cc-params.0sc9Q7  cc-params.bV9Do3  cc-params.KwGQXS  cc-params.QrPYtp  env-vars                response-3f5275.txt  response-86c79c.txt  response-d0dc4d.txt  response-ee5e9e.txt
cc-params.2pOvnT  cc-params.dOYob5  cc-params.NYQjSu  cc-params.sF4kh4  __nix_source_checksums  response-530cf0.txt  response-876cd0.txt  response-d1bd67.txt  response-ee734b.txt
cc-params.2qKwxY  cc-params.HY6Paq  cc-params.o0frTa  cc-params.TXUNQy  response-322fbe.txt     response-67ad45.txt  response-a79a74.txt  response-d94d4f.txt  webkitgtk-2.50.3
cc-params.4Ekgz0  cc-params.ilbeNb  cc-params.o66KdP  cc-params.VGXNAq  response-388449.txt     response-7105bf.txt  response-afaaed.txt  response-e9e739.txt
cc-params.9CkgKW  cc-params.KAcXMh  cc-params.PFOEFI  cc-params.yxvpn7  response-3c16e7.txt     response-8422d7.txt  response-cca326.txt  response-eda1f4.txt

nix-162532-2040114822/build:
cc-params.2gP44E  cc-params.C3AFGZ  cc-params.gURgQC  cc-params.JZkThK  cc-params.QysYW3  cc-params.vE7OVz  cc-params.xRenwp  __nix_source_checksums
cc-params.8gZD6C  cc-params.dWMnSD  cc-params.jgKQ1S  cc-params.Pq21ak  cc-params.umTUEo  cc-params.WHFUNY  cc-params.yAIvcf  webkitgtk-2.50.3
cc-params.9okGIZ  cc-params.fH45uD  cc-params.jqM5QG  cc-params.QGeJlo  cc-params.uxkqNh  cc-params.Xgx7Z8  env-vars

The package is webkitgtk-2.50.3 Let me know if this is normal.

2

How do you know it’s the same package? Do you have build logs?

3

webkitgtk shows in both build directories. and I looked at the __nix_source_checksums in both directories. they are exactly the same.

At the same time, the build process eats up all the swap space and forced me to reboot the build machine. I’m wondering if the oom issue was caused by this multi job rebuild bug.

4

That just means the sources are the same. How do you know the builds are the same?

pkgs.helloand pkgs.hello.overrideAttrs { env.BLAH = 1; } are two different packages, for example.

Ideally you would check the --dry-build output and see if webkitgtk shows up multiple times with different drvs. Or if you already built it and switched, check the requisites of the derivers of the current system.

There’s no evidence of a bug yet. Inspecting random folders won’t tell you much. But yes, browser builds are prone to OOMing with less than say, 64 gigs of RAM, unless you reduce cores (and that may not always work).

5

Or if you already built it and switched, check the requisites of the derivers of the current system.

I’m still trying to finish the build. Once it’s done, how do I check the requisites of the current system?

6

I checked the env-vars file both directories. They are actually different. Does that mean it’s all good?

The following are some of the differences:

93c93
< declare -x name="webkitgtk-2.50.3+abi=6.0"
---
> declare -x name="webkitgtk-2.50.3+abi=4.1"
95c95
< declare -x out="/nix/store/y3dhkm9f0nnkpii0l6z1b9m33wrgk21p-webkitgtk-2.50.3+abi=6.0"
---
> declare -x out="/nix/store/1rgh91nrqqgvd3d3vaqb4a5yyl1bhbh4-webkitgtk-2.50.3+abi=4.1"
101c101
< declare -x propagatedBuildInputs="/nix/store/vpdmhakvz5ypm2q2rcsjz6hrybbdiak6-gtk4-4.20.3-dev /nix/store/gzh8pg4ljrhwbanz8brdkm5b0wlvji9d-libsoup-3.6.5-dev"
---
> declare -x propagatedBuildInputs="/nix/store/r9qybwam7v4sj4b6jyavb7dm97hhprm9-gtk+3-3.24.51-dev /nix/store/gzh8pg4ljrhwbanz8brdkm5b0wlvji9d-libsoup-3.6.5-dev"
~
7

It means it’s expected to get built with each set of envvars, yes.
You may want to usenix-tree (3rd party tool) to narrow down why your config needs two versions of webkitgtk. With the --derivation flag you can inspect the drvs to see what else differs, too.

8

To be clear, are you actually trying to build webkit for academic reasons or would you be happy with nix grabbing the build from the hydra cache?

You don’t need to build anything if you just want to update your system, if that’s happening without you intending to that means your configuration is set up wrong.

1 Like
9

Good question. I was trying to re-compile libcamera by add the following

  nixpkgs.overlays = [
    (final: prev: {
      libcamera = prev.libcamera.overrideAttrs (oa: rec {
        version = "0.6.0";
        src = pkgs.fetchgit {
          url = "https://git.libcamera.org/libcamera/libcamera.git";
          rev = "v${version}";
          hash = "sha256-zGcbzL1Q2hUaj/s9NjBlp7hVjmSFb0GF8CnCoDS82Tw=";
        };
        buildInputs = oa.buildInputs ++ [
          final.libunwind
        ];
      });
    })
]

And this triggered a huge rebuild of everything.

10

I’d suggest waiting for libcamera: 0.5.2 -> 0.6.0 by r-burns · Pull Request #470093 · NixOS/nixpkgs · GitHub (status: https://nixpkgs-tracker.ocfox.me/?pr=470093) instead.