I’m testing a binary cache with the following script:
# Build package
MY_HELLO_OUT_PATH=$(nix build --no-link --print-out-paths .#my-hello)
# Generate keys
nix-store --generate-binary-cache-key my-bucket my-bucket-private-key.pem my-bucket-public-key.pem
PUBLIC_KEY=$(cat my-bucket-public-key.pem)
# Sign and verify
nix store sign --key-file my-bucket-private-key.pem "$MY_HELLO_OUT_PATH" --verbose
nix store verify --sigs-needed 1 "$MY_HELLO_OUT_PATH" --option trusted-public-keys "$PUBLIC_KEY" --verbose
nix path-info "$MY_HELLO_OUT_PATH" --sigs
# Copy to cache
SUBSTITUTER='''s3://my-bucket?scheme=http&endpoint=localhost:4566'''
nix copy --to "$SUBSTITUTER" "$MY_HELLO_OUT_PATH" --verbose
# Delete local package
nix store delete "$MY_HELLO_OUT_PATH"
# Fetch from cache
nix build --no-link --option extra-substituters "$SUBSTITUTER" --option trusted-public-keys "$PUBLIC_KEY" .#my-hello --verbose
This works as expected:
added 1 signatures
checking '/nix/store/k8qw2n8r032g88pdfzwiklqn9gzw39p1-my-hello'...
/nix/store/k8qw2n8r032g88pdfzwiklqn9gzw39p1-my-hello ultimate my-bucket:5Rak...
uploaded 's3://my-bucket/nix-cache-info' (21 bytes) in 219 ms
copying 5 paths...
copying path '/nix/store/ibw4h4lp57820gi5i46iwdnp9i58k8li-xgcc-12.3.0-libgcc' to 's3://my-bucket'...
copying path '/nix/store/p58fbj416fzm72h2h2v87h7a8brl55d5-libunistring-1.1' to 's3://my-bucket'...
uploaded 's3://my-bucket/nar/1qi9kgysh7rpfnj20zxiys6xg15265ws0sc5kybmw1zdj8asc37j.nar.xz' (50868 bytes) in 42 ms
uploaded 's3://my-bucket/ibw4h4lp57820gi5i46iwdnp9i58k8li.narinfo' (3221 bytes) in 46 ms
uploaded 's3://my-bucket/nar/1p5vyc1aiyi61b149w7vhjkcm7v3frcdhbycbhs79v31jf37cm4p.nar.xz' (427072 bytes) in 52 ms
uploaded 's3://my-bucket/p58fbj416fzm72h2h2v87h7a8brl55d5.narinfo' (3275 bytes) in 46 ms
copying path '/nix/store/7lihz4s5pg8irvp691x94argri346mls-libidn2-2.3.4' to 's3://my-bucket'...
uploaded 's3://my-bucket/nar/1rm417qkglyy6xngccjsrsqar440lj6imni4fxh471x33ay23jwi.nar.xz' (89480 bytes) in 47 ms
uploaded 's3://my-bucket/7lihz4s5pg8irvp691x94argri346mls.narinfo' (3314 bytes) in 46 ms
copying path '/nix/store/9la894yvmmksqlapd4v16wvxpaw3rg70-glibc-2.37-8' to 's3://my-bucket'...
uploaded 's3://my-bucket/nar/086lm8lagamhy44zdgx0wkr15yw18vd9wb6gsld3nli9hx9fnlpv.nar.xz' (6504812 bytes) in 117 ms
uploaded 's3://my-bucket/9la894yvmmksqlapd4v16wvxpaw3rg70.narinfo' (3364 bytes) in 47 ms
copying path '/nix/store/k8qw2n8r032g88pdfzwiklqn9gzw39p1-my-hello' to 's3://my-bucket'...
uploaded 's3://my-bucket/nar/0yqpi9w7jnqhgwc13v8hb25s35i943r4xmaa580arfy2k4v13s47.nar.xz' (34392 bytes) in 43 ms
uploaded 's3://my-bucket/k8qw2n8r032g88pdfzwiklqn9gzw39p1.narinfo' (579 bytes) in 48 ms
1 store paths deleted, 0.16 MiB freed
this path will be fetched (0.03 MiB download, 0.18 MiB unpacked):
/nix/store/k8qw2n8r032g88pdfzwiklqn9gzw39p1-my-hello
copying path '/nix/store/k8qw2n8r032g88pdfzwiklqn9gzw39p1-my-hello' from 's3://my-bucket'...
But when I delete and recreate the bucket/cache (and also rm -rf ~/.cache/nix/binary-cache-v6.sqlite* to force copying), then rerun the script, I get a different result:
added 1 signatures
checking '/nix/store/k8qw2n8r032g88pdfzwiklqn9gzw39p1-my-hello'...
/nix/store/k8qw2n8r032g88pdfzwiklqn9gzw39p1-my-hello my-bucket:5Rak... my-bucket:DxRA...
uploaded 's3://my-bucket/nix-cache-info' (21 bytes) in 210 ms
copying 5 paths...
copying path '/nix/store/ibw4h4lp57820gi5i46iwdnp9i58k8li-xgcc-12.3.0-libgcc' to 's3://my-bucket'...
copying path '/nix/store/p58fbj416fzm72h2h2v87h7a8brl55d5-libunistring-1.1' to 's3://my-bucket'...
uploaded 's3://my-bucket/nar/1qi9kgysh7rpfnj20zxiys6xg15265ws0sc5kybmw1zdj8asc37j.nar.xz' (50868 bytes) in 43 ms
uploaded 's3://my-bucket/ibw4h4lp57820gi5i46iwdnp9i58k8li.narinfo' (3221 bytes) in 47 ms
uploaded 's3://my-bucket/nar/1p5vyc1aiyi61b149w7vhjkcm7v3frcdhbycbhs79v31jf37cm4p.nar.xz' (427072 bytes) in 49 ms
uploaded 's3://my-bucket/p58fbj416fzm72h2h2v87h7a8brl55d5.narinfo' (3275 bytes) in 48 ms
copying path '/nix/store/7lihz4s5pg8irvp691x94argri346mls-libidn2-2.3.4' to 's3://my-bucket'...
uploaded 's3://my-bucket/nar/1rm417qkglyy6xngccjsrsqar440lj6imni4fxh471x33ay23jwi.nar.xz' (89480 bytes) in 43 ms
uploaded 's3://my-bucket/7lihz4s5pg8irvp691x94argri346mls.narinfo' (3314 bytes) in 46 ms
copying path '/nix/store/9la894yvmmksqlapd4v16wvxpaw3rg70-glibc-2.37-8' to 's3://my-bucket'...
uploaded 's3://my-bucket/nar/086lm8lagamhy44zdgx0wkr15yw18vd9wb6gsld3nli9hx9fnlpv.nar.xz' (6504812 bytes) in 145 ms
uploaded 's3://my-bucket/9la894yvmmksqlapd4v16wvxpaw3rg70.narinfo' (3364 bytes) in 43 ms
copying path '/nix/store/k8qw2n8r032g88pdfzwiklqn9gzw39p1-my-hello' to 's3://my-bucket'...
uploaded 's3://my-bucket/nar/0yqpi9w7jnqhgwc13v8hb25s35i943r4xmaa580arfy2k4v13s47.nar.xz' (34392 bytes) in 44 ms
uploaded 's3://my-bucket/k8qw2n8r032g88pdfzwiklqn9gzw39p1.narinfo' (683 bytes) in 45 ms
1 store paths deleted, 0.16 MiB freed
this path will be fetched (0.03 MiB download, 0.18 MiB unpacked):
/nix/store/k8qw2n8r032g88pdfzwiklqn9gzw39p1-my-hello
warning: ignoring substitute for '/nix/store/k8qw2n8r032g88pdfzwiklqn9gzw39p1-my-hello' from 's3://my-bucket', as it's not signed by any of the keys in 'trusted-public-keys'
building '/nix/store/8rjiyq1halbq7m47dj6zmi792aspmyfr-my-hello.drv'...
nix-path-info suggests that the new key was added to the signatures, so why does this not work?
Also, that fact that the new key was added to the old one suggests that some state persists on the client side. How do I remove signatures?
EDIT: I’m using nix (Nix) 2.13.3.
