How to perform remote evaluation when deploying from macOS to Linux with sops-nix secrets?

Help
1

Hi everyone,

I’m trying to deploy NixOS configurations from my macOS machine to a remote x86_64-linux server using nixos-rebuild-ng. I’m running into an issue where evaluation happens locally on macOS, but my configuration references sops-nix secrets that only exist on the target machine.

Current Setup

  • Controller: macOS with nix-darwin and flakes
  • Target: NixOS x86_64-linux server
  • Secrets: Managed with sops-nix, using age keys derived from SSH keys

secrets/default.nix:

{ ... }:

{
  sops = {
    defaultSopsFile = ./secrets.yaml;
    age.sshKeyPaths = [ "/root/.ssh/id_ed25519" ];
    age.generateKey = true;
    secrets  = {
      "pangolin/newt_id" = {};
      "pangolin/newt_secret" = {};
  
      "ssh_keys/dobrynikolov" = {};
      "ssh_keys/dobrynikolov.pub" = {};
      "ssh_keys/engineer" = {};
      "ssh_keys/engineer.pub" = {};
    };
  };
}

Deployment command:

nixos-rebuild-ng switch \
  --flake ".#engineer" \
  --target-host "root@192.168.1.6" \
  --build-host "root@192.168.1.6" \
  --sudo \
  --impure

The Problem

My NixOS configuration uses openssh.authorizedKeys.keyFiles that reference sops secrets:

users.users.engineer = {
  openssh.authorizedKeys.keyFiles = [
    config.sops.secrets."ssh_keys/dobrynikolov.pub".path  # /run/secrets/ssh_keys/dobrynikolov
    config.sops.secrets."ssh_keys/engineer.pub".path
  ];
};

When I run the deployment, evaluation happens locally on macOS (despite --build-host), and it fails with:

error: opening file '/private/var/run/secrets/ssh_keys/dobrynikolov.pub': No such file or directory

inb4 store .pub secrets in plain text
Issue comes up with other secrets aswell

This path (/run/secrets/...) only exists on the target NixOS machine after sops-nix decrypts secrets during activation.

What I’ve Tried

  1. Using --build-host - doesn’t change where evaluation happens
  2. Using --impure flag - still evaluates locally

My Questions

  1. Is there a way to make nixos-rebuild evaluate the configuration on the remote machine instead of locally?

  2. What’s the recommended pattern for using sops-nix secrets in configurations when deploying cross-platform (macOS → Linux)? Should I:

    • Use activation scripts instead of authorizedKeys.keyFiles?
    • Accept that I need to rsync the flake and run nixos-rebuild directly on the target?

  3. Is there an --eval-host or equivalent flag I’m missing that would solve this?

Workaround I’m Considering

The only solution I can think of is:

rsync -az ./ root@192.168.1.6:/tmp/deploy/
ssh root@192.168.1.6 "cd /tmp/deploy && nixos-rebuild switch --flake .#engineer"

But this defeats the purpose of nixos-rebuild’s remote deployment features.
Thanks in advance!

2

I’m using nixos-rebuild-ng to evaluate on machine A build on machine B deploy to machine C which works flawlessly with sops secrets. Evaluation always happens on the host where you run nixos-rebuild-ng as far as I know.

Can you please be a bit more specific where the error comes from and a few more lines what you did? Does the evaluation throw the error? That would be odd. I suspect that the error comes from the remote host and there is something wrong with the keys?

1 Like
3

You can reduce the hosts involved if you start the rebuild command on your build host. You’d only have to drop --build-host but otherwise would be fine (provided the flake was on that host too). This is the usual way I trigger remote builds, where the builder starts and pushes the closure. I also use sops-nix where the builder can’t decrypt all the end host’s secrets, but that hasn’t been any particular issue.

1 Like
4

You’re not using sops-nix correctly if you’re having this issue. Sops-nix doesn’t need any secrets at eval or build time if used correctly. That’s the whole point.

The issue here is that users.users.<name>.openssh.authorizedKeys.keyFiles is built to access the files at eval/build time, not runtime, as stated in the documentation for the option:

The contents of the files are read at build time and added to a file that the SSH daemon reads in addition to the the user’s authorized_keys file.

That’s also perfectly fine, because the contents of those options are not secrets. They’re public keys, and really don’t need to be protected by sops in the first place.

This kind of “looping through the activated system” is one of the mistakes flakes help keep you from making. It wouldn’t function at all from pure eval.

3 Likes
5

Thanks, that makes sense — will stop feeding /run/secrets into authorizedKeys.keyFiles and either inline the public keys with users.users..openssh.authorizedKeys.keys or reference committed .pub files, since keyFiles are read at build time by design. <3