Cachix says "All Done", "nix-build" still wants to rebuild!

Help
1

I have a GHA:

  • it uses nix-build-uncached
  • it pushes the paths it builds to cachix when its done
  • I call nix-build manually with:
    • --option 'extra-binary-caches' 'https://cache.nixos.org https://colemickens.cachix.org https://nixpkgs-wayland.cachix.org https://arm.cachix.org https://thefloweringash-armv7.cachix.org'
    • --option 'trusted-public-keys' 'cache.nixos.org-1:6NCHdD59X431o0gWypbMrAURkbJ16ZPMQFGspcDShjY= colemickens.cachix.org-1:bNrJ6FfMREB4bd4BOjEN85Niu8VcPdQe4F4KxVsb/I4= nixpkgs-wayland.cachix.org-1:3lwxaILxMRkVhehr5StQprHdEo4IrE8sRho9R9HOLYA= arm.cachix.org-1:5BZ2kjoL1q6nWhlnrbAl+G7ThY7+HaBRD9PZzqZkbnM= thefloweringash-armv7.cachix.org-1:v+5yzBD2odFKeXbmC+OPWVqx4WVoIVO6UXgnSAWFtso='

I have another GHA that uses nix-build-uncached with -j0 to basically give a PASS/FAIL on whether or not my derivations are already built and cached.

Except that this … isn’t working.

In fact, this log is interesting: https://gist.githubusercontent.com/colemickens/9e56fd87a61fd582e5240b43fa79332e/raw/02401a58df2ca7f4fc61cd1952f92721e4f86ab4/l

  • it shows nix-build --dry-run SUCCESSFULLY downloading from my cachix:

    2021-05-25T16:18:09.9830837Z querying info about '/nix/store/4f39rrz40r9d0bs34wwv9n0ff24q70g6-linux-5.4.79-1.20201201-armv6l-unknown-linux-gnueabihf' on 'https://colemickens.cachix.org'...
2021-05-25T16:18:09.9834053Z downloading 'https://colemickens.cachix.org/4f39rrz40r9d0bs34wwv9n0ff24q70g6.narinfo'...

  • it shows nix-build --dry-run telling me it will need to build some paths, after listing the ones that it will download (again, some of which are coming from my cache):

     2021-05-25T16:18:11.3612574Z downloading 'https://arm.cachix.org/i2ibrxfggvjbs2f75r086vmi0g33zby1.narinfo'...
 2021-05-25T16:18:11.3614833Z querying info about '/nix/store/v6fshyx7kh54rzdlw15r9p1ixccvrh3x-libevent-2.1.12' on 'https://thefloweringash-armv7.cachix.org'...
 2021-05-25T16:18:11.3617367Z downloading 'https://thefloweringash-armv7.cachix.org/v6fshyx7kh54rzdlw15r9p1ixccvrh3x.narinfo'...
 2021-05-25T16:18:11.3619961Z querying info about '/nix/store/i2ibrxfggvjbs2f75r086vmi0g33zby1-keyutils-1.6.3-lib' on 'https://thefloweringash-armv7.cachix.org'...
 2021-05-25T16:18:11.3622850Z downloading 'https://thefloweringash-armv7.cachix.org/i2ibrxfggvjbs2f75r086vmi0g33zby1.narinfo'...
 2021-05-25T16:18:11.3624284Z these 169 derivations will be built:
 2021-05-25T16:18:11.3625652Z   /nix/store/01mw8hj18knxgymw7afi7hx64lb7sv15-etc-resolvconf.conf.drv
 2021-05-25T16:18:11.3627490Z   /nix/store/05qk6rys1x66nhb9j9agxln6n975rx42-unit-keys.target.drv
 2021-05-25T16:18:11.3629352Z   /nix/store/0d16db0vp4nj5xlk3yf690k49z7fkzbc-unit-nix-daemon.socket.drv
 2021-05-25T16:18:11.3631771Z   /nix/store/0jzgqdfr5cd06n7mrlsfvmr2x8hmnpdx-unit-network-online.target.drv

Here’s the thing… the build products for all of those derivations exist in my cachix… I’ve checked repeatedly. The output paths for those derivations… ALL return 200s when I query for their narinfos manually.

And of course, cachix push insists that it’s “All done” because these paths already exist.

I can repro this with stable + unstable nix, but… uh… I sort of assume I must be doing something wrong, somewhere. Any tips?

It’s not a cache TTL issue. Beyond overriding it on every call, this is occurring on GitHub Action Runners which are ephemeral.

EDIT: While I’ve techncally tested with the stable/unstable daemon, I’ve basically only exercised this with nix-build --dry-run from an unstable nix cli.

2

I really don’t think it’s a trust issue, given that the narinfo that it DID download from my cachix contains only a signature from my cachix key, so it’s not as if it pulled from my cachix and then validated with a different trusted signature.

3

shamelessly cc: @mic92 here for suggestions, I’m really out of sorts on this one.

4

Does this ever reproduce with nix-build or only nix-build-uncached?

5

I guess I’m not entirely sure. Since nix-build-uncached calls nix build and it awesomely doesn’t produce logs by default, I can’t actually tell if nix itself is performing a build or just fetching the remaining paths from cachix.\

EDIT: Actually, the part of my script that uploads to cachix does so by looking for unsigned paths… indicating that indeed nix is rebuilding these paths and not just having re-downloaded them from cachix (since they’d have a signature in that case).

But again, the path it builds for the system toplevel… it’s still hashing to the same thing because I’m on flakes and not changing my inputs… and its narinfo is in cachix.

6

You can pass -L to -build-flags I think to get more logs?