Why does copying closure over `ssh-ng://` fail where `ssh://` succeeds?

debtquity · August 13, 2026, 9:20pm

I have run into this strange error when rebuilding system and copying closure to remote system:

# nixos-rebuild switch --log-format internal-json --verbose --flake .#remotetarget1 --target-host "ssh-ng://remotetarget1" |& nom --json
...
copying 39 paths
copying path '/nix/store/1bbq14qfhiglhm8i5dpw5ny3awwkn1b0-system-generators' to 'ssh-ng://remotetarget1'
...
error: cannot add path '/nix/store/1bbq14qfhiglhm8i5dpw5ny3awwkn1b0-system-generators' because it lacks a signature by a trusted key
...

However, switching to legacy/default of ssh:// will succeed without any issues:

# nixos-rebuild switch --log-format internal-json --verbose --flake .#remotetarget1 --target-host "remotetarget1" |& nom --json
...
copying 39 paths
copying path '/nix/store/1bbq14qfhiglhm8i5dpw5ny3awwkn1b0-system-generators' to 'ssh://remotetarget1'
...
(successful rebuild)

Any clues as to why this happens?

xokdvium · August 13, 2026, 9:55pm

ssh:// (aka nix-store —serve protocol) simply doesn’t have a notion of checking signatures and doesn’t pass the flag for checking signatures on the remote.

In fact, it always doesn’t check signatures – hence why it requires the remote user to be trusted by the daemon the nix-store —serve process is talking to on the remote machine (in case it’s not interacting with a local store directly).

github.com/NixOS/nix

src/nix/nix-store/nix-store.cc

6274126b2


      
              info.references = ServeProto::Serialise<StorePathSet>::read(*store, rconn);
              in >> info.registrationTime >> info.narSize >> info.ultimate;
              info.sigs = ServeProto::Serialise<std::set<Signature>>::read(*store, rconn);
              info.ca = ContentAddress::parseOpt(readString(in));
          
              if (info.narSize == 0)
                  throw Error("narInfo is too old and missing the narSize field");
          
              SizedSource sizedSource(in, info.narSize);
          
              store->addToStore(info, sizedSource, NoRepair, NoCheckSigs);
          
              // consume all the data that has been sent before continuing.
              sizedSource.drainAll();
          
              out << 1; // indicate success
          
              break;
          }
          
          default:

debtquity · August 13, 2026, 11:04pm

hi @xokdvium , if I am understanding your response correctly, ssh-ng by default checks the signatures whereas ssh does not. Thus the reason why remote build fails with ssh-ng.

In both cases (ssh:// and ssh-ng://), the authenticated user is root on remote machine (remotetarget1). root by default is a trusted-user of the nix-store daemon.

So in theory, it should build without the signature check error. But it does not.

xokdvium · August 13, 2026, 11:18pm

It’s a bit different (at least with upstream nix). Root doesn’t go through the daemon at all and accesses the local store directly.

You can pass —no-check-sigs if you really want to ignore signatures. With ssh:// stores that flag is completely ignored, but with ssh-ng it is not.

xokdvium · August 13, 2026, 11:19pm

Ah, I don’t actually know how one would pass this down the call stack from nixos-rebuild. I’d probably copy the built systems closure manually via nix copy

debtquity · August 14, 2026, 2:23am

Seems with nixos-rebuild, need to set require-sigs [1] to false in nix configuration on remote machine to get the equivalent of the --no-check-sigs flag on nix copy.

[1] nix.conf - Nix 2.34.9 Reference Manual

tree-sapii · August 14, 2026, 5:15am

I think nixos rebuild internally called nix copy, may just need to edit the python script and pass to calls.

bitbloxhub · August 14, 2026, 4:16pm

I have not used it, but I think nh actually uses --no-check-sigs when copying. Although I think nixos-rebuild should probably have an argument to do that too, but it does not.

Or use deploy-rs like me to deploy your machines.

debtquity · August 14, 2026, 7:53pm

I ended up using this process instead [1]:

generating key pairs on all of my remote builders
signing the remote builder stores
rebuild remote builders to auto-sign stores
distribute public key to the remote targets (setting trusted-public-keys) over ssh://
re-build using ssh-ng:// works as expected.

Using --no-check-sigs or disabling require-sigs “works” but weakens security posture.

[1] https://wiki.nixos.org/wiki/Signing_store_paths