Local deployment with NixOps

Help
1

Hey folks,

I’m new to Nix and NixOS, so bear with me please. I am trying to use NixOps to manage my bare-metal server running NixOS on my local network. According to the NixOps manual, this should just work if I set deployment.targetHost to the local IP address and leave deployment.targetEnv undefined. This is working as intended.

However, by default this means that NixOps will try to SSH to the server as root. According to the documentation, this can be overridden by setting deployment.targetUser to the username of the user that is configured to allow passwordless sudo and added as a trusted user to nix. I have set up my user correctly on my local server.

Finally, it should also be possible to use my own SSH keys and not have NixOps create new keys that I need to add to my server, but the documentation for this is hard to find. From the
NixOps source code, I’ve managed to find the following values:

resources.sshKeyPairs.ssh-key = {
  publicKey = builtins.readFile ~/.ssh/id_rsa.sage.pub;
  privateKey = builtins.readFile ~/.ssh/id_rsa.sage;
};

# or

sage = { config, pkgs, ...}: {
  ...
  deployment.none.sshPrivateKey = builtins.readFile ~/.ssh/id_rsa.sage;
  # deployment.none.sshPublicKey = builtins.readFile ~/.ssh/id_rsa.sage.pub;
  deployment.none.sshPublicKeyDeployed = true;
  deployment.provisionSSHKey = false;
...
};

Now to my problem: none of this seems to work! With this following configuration, if I execute NixOps, I get the following output:

 ~/code/nas.local/nixos
 > nixops create ./nixops/home/deploy.nix -d home
created deployment ‘1be0bee8-502c-11eb-acd1-35ca375e8ac2’
1be0bee8-502c-11eb-acd1-35ca375e8ac2

 ~/code/nas.local/nixos
 > nixops info -d home
nixops Network name: home
Network UUID: 1be0bee8-502c-11eb-acd1-35ca375e8ac2
Network description: Home network
Nix expressions: /home/jente/code/nas.local/nixos/nixops/home/deploy.nix

+------+---------+------+-------------+------------+
| Name |  Status | Type | Resource Id | IP address |
+------+---------+------+-------------+------------+
| sage | Missing | none |             |            |
+------+---------+------+-------------+------------+

 ~/code/nas.local/nixos
 > nixops deploy -d home
sage> generating new SSH keypair... done
root@sage.local: Permission denied (publickey,keyboard-interactive).
sage> could not connect to ‘root@sage.local’, retrying in 1 seconds...
root@sage.local: Permission denied (publickey,keyboard-interactive).
sage> could not connect to ‘root@sage.local’, retrying in 2 seconds...
^Cerror: interrupted

That is:

  1. NixOps is still trying to SSH as root, instead of my user, despite my following the manual. Did I miss anything?
  2. NixOps is still generating an SSH keypair, despite deployment.provisionSSHKey being set to false. Did I miss anything?
  3. Is my way of specifying my own SSH keys correct?
  4. Is there a better way of including the standard NixOS configuration.nix file in my NixOps deployment file? I’ve seen e.g. sage = { config, pkgs, ... }: (import ../../hosts/sage/configuration.nix { inherit config pkgs; }) // { .. } as well. I’d prefer to keep my deployment code separate from the system configuration.

Again, I’m new to all this so it is entirely possible that I have missed something. Thanks in advance!

2 Likes
2

I would probably not bother with the nixops on localhost and just use nixos-rebuild instead.
Your /etc/nixos/configuration.nix would look something like this:

{ lib, ... } @args:

let
  deployment-path = /home/jente/code/nas.local/nixos/nixops/home/deploy.nix;
  server-name = "sage";
in lib.filterAttrs (n: v: n != "deployment") ((import deployment-path).${server-name} args)
3

Excellent question! I particularly like how you present your progress so far. Now that will be the documentation (for a while at least) you were missing, thanks for sharing. Since I don’t know the answer, I can only generally help with troubleshooting.

Just a quick insanity check: did you read the manual and source for the exact version of NixOps you were running? Maybe the features are not yet built into what you have there.

This looks perfectly fine to me, and this is what I also do.

That is also true, you can just run nixos-rebuild —target-host <target> to rebuild a different machine with the chosen configuration. It will just use your ssh setup, which fits your use case nicely.

4

Thanks both for you answers! I wasn’t aware that nixos-rebuild can build on a remote host, that looks interesting! What are the options for secrets management when using this? I suppose I can rsync them into the same location as NixOps puts them, but perhaps there’s a standard way of doing this.

Just a quick insanity check: did you read the manual and source for the exact version of NixOps you were running? Maybe the features are not yet built into what you have there.

Good hunch! I missed the hint on the non-root feature that it applies to NixOps 2.0 (as of yet unreleased) only:

NixOps 2.0 allows for deploying as users other than root, as long as the deploying user meets two requirements

I can’t find in the documentation at least whether NixOps 1.7 supports using my own SSH keys, but I’m going to assume that’s also a NixOps 2.0 feature.

5

I have to add, a downside of using nixos-rebuild is that it seems to be difficult to install the it on a non-NixOS operating system. My main system does not (yet?) run NixOS, so that’s a bit of an issue for me.

6

I got everything to work using NixOps 2.0 from the master branch, with the nix-shell setup they provide. For a summary, here’s what ended up working for me:

  1. From the nix-shell in the NixOps repository on the master branch:
nixops create </path/to/nixops.nix> -d home
  1. In my NixOps deploy.nix file for the SSH access:
{
  network.description = "Home network";

  sage = { config, pkgs, ... }: {
    imports = [ ../../hosts/sage/configuration.nix ];

    deployment.targetHost = "sage";
    deployment.targetUser = "jente";
    deployment.provisionSSHKey = false;

    ...
  };
}

Note the targetHost = "sage" value. sage is what I configured in my ~/.ssh/config for this server:

Host sage
HostName 192.168.1.2
User jente
IdentityFile ~/.ssh/id_rsa.sage

Because NixOps shells out to SSH, using this hostname makes SSH automagically pick the correct keys!

With these changes, everything I posted above works as intended. The only annoyance is having to use NixOps from the repository in its nix-shell environment, but I can probably hold over until version 2.0 is released :slight_smile:

Out of curiosity, I will try to deploy with nixos-rebuild --target-host as well, but NixOps does have the advantage of managing secrets built-in and the fact that everything is written in Nix rather than a combination of Nix (the configuration) and Bash (the deployment).

4 Likes
7

A difference between nixos-rebuild --target-host and nixops is that the former can build on the target machine, whereas the latter can only build on the host machine. Whether that’s an advantage or disadvantage depends on your use case :slight_smile:

8

I use sops-nix and git-crypt to manage secrets when not using NixOps (both rely on GPG for encryption), and recently agenix is also an option if you don’t want to mess with GPG.

1 Like
9

Additionaly you can make NixOps build remotely, but this is managed through nix configuration. But unless you have exactly one remote builder and it is also your target, it seems not to be possible to tell it to build on the target machine specifically. NixOps sounds like the right way to approach things, but for my taste it’s by far not mature enough to be adopted lightheartedly.

Yes. :frowning: You have to fiddle around a lot, see here for running it on Darwin.

10

Thanks both! For now I’ll keep using NixOps building on my local machine. Once this machine also runs NixOS, I might switch to using nixos-rebuild with some scripting around secrets.