Nix wants to build everything from sources with the same nixpkgs. How to diagnose?

Help
1

I have multi-user setup of nix on RHEL7 and pinned nixpkgs in my .nix file that creates an environment of a large number of packages (python, many python packages, texlive, etc.). At some point trying to build the environment I noticed that nix-build all of a sudden wants to rebuild everything from scratch (libc, gcc, python, jdk, etc.) which takes days on the small VM I am using. I never ran GC so it doesn’t even need to go to the binary cache to get the compiled libc and gcc. The only change I know of is adding one more python package to the script.

How do I diagnose why nix-build would want to start building the whole world from scratch? I see that it builds, say libc, with a different hash than before. I tried comparing the drv files but quickly got lost. Is there anything more basic than libc which would help me pinpoint the reason for the full rebuild?

2

nix-diff is the best tool I’ve found to diagnose this.

The way I use it is like:

$ nix run nixpkgs.nix-diff -c nix-diff $(nix-instantiate ./old-nixpkgs -A my-attr) $(nix-instantiate ./new-nixpkgs -A my-attr)
1 Like
3

Thanks! My nixpkgs is the same in both cases. I ran it on the two libc derivations and basically got these few lines repeated for many inputs:

  • The arguments do not match
      • -e
      - /nix/store/9krlzvny65gdc8s7kpb6lkx8cd02c25b-default-builder.sh
      + /nix/store/4z3n8xqam4d8z0fma1kc5narcsgfsg2z-default-builder.sh
  • The input named `bootstrap-stage2-gcc-wrapper-` differs
    • These two derivations have already been compared
  • The input named `bootstrap-stage2-stdenv-linux` differs
    - /nix/store/9p9lv9fasriagylsja0qzw4ldg38jpci-bootstrap-stage2-stdenv-linux.drv:{out}
    + /nix/store/xi2fplqc1w7pgjligkld743paaywa3kz-bootstrap-stage2-stdenv-linux.drv:{out}
    • The builders do not match
        - /nix/store/n9acaakxahkv1q3av11l93p7rgd4xqsf-bootstrap-tools/bin/bash
        + /nix/store/qgf0p7p4in4pw0xl1grgxdnkd57p3l5l-bootstrap-tools/bin/bash
    • The arguments do not match
        • -e
        - /nix/store/dsyj1sp3h8q2wwi8m6z548rvn3bmm3vc-builder.sh
        + /nix/store/cc0p56545861ay7598a8q3y282fbk7xb-builder.sh

I have compared the pairs of shell scripts and they are all identical but the permissions differ, for instance

$ ls -l /nix/store/*unpack-bootstrap-tools.sh
-rwxrwxr-x 1 root root 2055 Dec 31  1969 /nix/store/c0sr4qdy8halrdrh5dpm7hj05c6hyssa-unpack-bootstrap-tools.sh
-r-xr-xr-x 2 root root 2055 Dec 31  1969 /nix/store/m1a45k9f64clp7q08r6swskj53bjhgw3-unpack-bootstrap-tools.sh

$ ls -l /nix/store/*default-builder.sh
-r-xr-xr-x 2 root root 34 Dec 31  1969 /nix/store/4z3n8xqam4d8z0fma1kc5narcsgfsg2z-default-builder.sh
-rwxrwxr-x 1 root root 34 Dec 31  1969 /nix/store/9krlzvny65gdc8s7kpb6lkx8cd02c25b-default-builder.sh

Is that suspicious?

4

I don’t think that’s an issue - that won’t go into the drv hash. Although it is a little weird.

There’s a few things that go into stage2 that you may have accidentally changed the Nix expressions for. For instance, perl, gcc, bison, glibc all go into that.

5

Wait what… Nix isn’t supposed to allow the write permission. Everything under /nix/store should be 555 or 444.