Hi, at work for our remote builder access, we started restrict users to only run the minimum required to speak to the nix daemon. However, we found that the command is different depending if people are using ssh:// or ssh-ng:// in the builder URL.
I came up with a wrapper that depending what you call on the remote ssh server, it will provide you the correct command to match the client, and stop if you try to do something else. It’s working fine so far, but I wonder if it’s something worth upstreaming as a NixOS module or in the Nix documentation.
# add this before each authorized key
ssh-restrict = "restrict,pty,command=\"${wrapper-dispatch-ssh-nix}/bin/wrapper-dispatch-ssh-nix\" ";
wrapper-dispatch-ssh-nix = pkgs.writeShellScriptBin "wrapper-dispatch-ssh-nix" ''
case $SSH_ORIGINAL_COMMAND in
"nix-daemon --stdio")
exec env NIX_SSL_CERT_FILE=${pkgs.cacert}/etc/ssl/certs/ca-bundle.crt ${pkgs.nix}/bin/nix-daemon --stdio
;;
"nix-store --serve --write")
exec env NIX_SSL_CERT_FILE=${pkgs.cacert}/etc/ssl/certs/ca-bundle.crt ${pkgs.nix}/bin/nix-store --serve --write
;;
*)
echo "Access only allowed for using the nix remote builder" 1>&2
exit
esac
'';
the “issue” is that even with that, this allows any user with access to the nix user to compromise the machine through the store if they want. So, it tighten the security, but still allow to compromise the machine…
true, but having a module makes it very easy to roll out future improvements as and when they appear. it might not be perfect, but it’s better than nothing? the module could conceivably gain a sandboxed feature at some point that spawns a container with a dedicated store rather than sharing the host store. or do it in a vm instead. or enable new safety features in nix as they appear. anything is probably better than cargo-culting setup snippets
yes, for instance, if you use a nix builder on a NixOS system that will be updated soon, you could poison the store by pushing there a bash derivation with the same hash of the one that will land in the update, but compromised.
Perhaps the wrapper script being in nixpkgs would be a good start for reuse. As for a NixOS option… I’m skeptical, but maybe. The “API” provided by SSH seems rather restrictive. A line in a file, that also contains a public key (which could be considered a secret). But I guess more awkward things have been nixified?
yes, for instance, if you use a nix builder on a NixOS system that will be updated soon, you could poison the store by pushing there a bash derivation with the same hash of the one that will land in the update, but compromised.
Sorry for the late question to this thread. But wouldn’t that require to still craft the malicious script with the same hash than the upcoming good one, hence require to craft a hash collision? Or did I understand something wrongly about how the Nix store works?
There are a variety of ways to contribute to DoS issues (which is one of the vectors which the ForceCommand is used for), including something like a fork bomb, which cannot be properly mitigated with sandboxing; restricting the number of processes does only so much and it may hinder the actual workload depending on thread counts and such, CPU cycles are irrelevant in that case since they happen on the kernel level, etc…
The best one can do is write a wrapper which does read a single line, check it far validity, and then print “started” or run the corresponding process as a child and on exit reap and repeat, which I assume winter wanted to do and which I’m now going to hack into a shell script real quick.
Huh, now that I’m going through everything again I somehow managed to absolutely miss the script at the very top of the thread.
There’s only so many ways to write that script anyway, and below is my take on it, in the form of a nixpkgs patch (nixos-24.11, but can be cleanly cherry-picked to master), which I’ll be maintaining out of tree for a bit.
Feel free to copy, modify, move it upstream.… whatever.
If you do move it upstream, it might be worth making some of this configurable though, and of course fix the formatting and general code quality.
In terms of functionality though, I’m running Lix main and this works (at least with the ssh:// protocol, but ssh-ng:// should too) both locally as well as from within a Hydra instance.
nixpkgs patch for ForceCommand
From c7e73502898afc540a824689a5e93081cf0d3b7e Mon Sep 17 00:00:00 2001
From: benaryorg <binary@benary.org>
Date: Wed, 14 May 2025 23:10:00 +0000
Subject: [PATCH] nixos/ssh-serve: wrapper for SSH ForceCommand
Due to complications of SSH handling some implementations require a "sanity check" before operating via the ssh protocol.
By providing a wrapper script which enforces one of the corresponding access modes the sanity check can be included.
For maximum compatibility, a command passed directly via SSH is also allowed.
Note that this is not a filter on which commands are allowed to run, or some sort of sandbox, instead this is a non-interactive shell providing only the specified commands.
In that sense there is no risk of using escape sequences or special shell characters, and there is no way to run malicious commands such as a fork bomb or similar.
Signed-off-by: benaryorg <binary@benary.org>
---
nixos/modules/services/misc/nix-ssh-serve.nix | 42 ++++++++++++++++---
1 file changed, 36 insertions(+), 6 deletions(-)
diff --git a/nixos/modules/services/misc/nix-ssh-serve.nix b/nixos/modules/services/misc/nix-ssh-serve.nix
index b27ef03dae6f..3b869c4d191f 100644
--- a/nixos/modules/services/misc/nix-ssh-serve.nix
+++ b/nixos/modules/services/misc/nix-ssh-serve.nix
@@ -6,11 +6,41 @@
}:
let
cfg = config.nix.sshServe;
- command =
- if cfg.protocol == "ssh" then
- "nix-store --serve ${lib.optionalString cfg.write "--write"}"
- else
- "nix-daemon --stdio";
+ template = input: command: " ${lib.escapeShellArg input})\n ${lib.escapeShellArgs command}\n ;;";
+ buildCommand = commands: pkgs.writeShellApplication {
+ name = "nix-serve-forcecommand";
+ text = ''
+ runCommand() {
+ case "$1" in
+ ${builtins.concatStringsSep "\n" (builtins.map ({ input, command }: template input command) commands)}
+ # some implementations use "exec", emulate this
+ "exec "*)
+ runCommand "''${1##"exec "}"
+ exit
+ ;;
+ # other commands are prohibited
+ *)
+ printf "not running unknown command '%s'\\n" "$1" >&2
+ exit 1
+ esac
+ }
+
+ # implementations may pass a command directly, however some implementations pass "bash" as the command to run interactively
+ if test -n "''${SSH_ORIGINAL_COMMAND:+x}" && test "$SSH_ORIGINAL_COMMAND" != bash; then
+ runCommand "$SSH_ORIGINAL_COMMAND"
+ else
+ while read -r line; do
+ runCommand "$line"
+ done
+ fi
+ '';
+ runtimeInputs = [ config.nix.package ];
+ };
+ command = buildCommand ([ { input = "echo started"; command = [ "echo" "started" ]; } ]
+ ++ lib.optional (cfg.protocol == "ssh") { input = "nix-store --serve"; command = [ "nix-store" "--serve" ]; }
+ ++ lib.optional (cfg.protocol == "ssh" && cfg.write) { input = "nix-store --serve --write"; command = [ "nix-store" "--serve" "--write" ]; }
+ ++ lib.optional (cfg.protocol == "ssh-ng") { input = "nix-daemon --stdio"; command = [ "nix-daemon" "--stdio" ]; }
+ );
in
{
options = {
@@ -68,7 +98,7 @@ in
PermitTTY no
PermitTunnel no
X11Forwarding no
- ForceCommand ${config.nix.package.out}/bin/${command}
+ ForceCommand ${command.out}/bin/nix-serve-forcecommand
Match All
'';
--
2.47.2
This patch worked for me, thanks! I’ve been trying to diagnose this for about a week since I thought the problem was with having to use the ssh ControlMaster/ControlPath options after upgrading to Lix 2.93.0.
It seems to work for me with both ssh:// and ssh-ng:// on Lix 2.93.0, although I had to enable trusted=true; write=true; before I could get nix store ping --store to respond positively (I used to run with these options false and protocol="ssh-ng"). If you’re looking to upstream, I’d really appreciate it!
