Running NixOS from any Linux Distro in systemd-nspawn Containers

9 Likes

This has been really useful to me.

I run Silverblue in my main workstation (I just enjoy automatic updates of the base system and a home-manager userland).

In my NixOS installation I had a couple of NixOS containers running some services and managing them with NixOS is wonderful.

Now I can port those containers easily to Silverblue (or any Linux).

Although the tarball creation was pretty slow and I didn’t like it since I’m only deploying those containers in the same machine that builds them, so I modified it to leverage of the local /nix/store and create only the required files (most of them symlinked to /nix/store), now the container just bind mounts the local /nix/store and re-deployments are pretty fast: GitHub - aorith/media-stack: Personal media stack running in a NixOS systemd-nspawn container

That’s nice! I’m trying to follow this guide, but I’m getting the error inside the container:

[root@nixos:~]# nixos-rebuild switch
error: this system does not support the kernel namespaces that are required for sandboxing; use '--no-sandbox' to disable sandboxing

Output of machinectl status:

% machinectl status nixos
nixos(3666d215b00e4c5088dcc818bfbf4311)
           Since: Mon 2024-01-08 15:22:34 CET; 2min 48s ago
          Leader: 1909041 (systemd)
         Service: systemd-nspawn; class container
            Root: /var/lib/machines/nixos
              OS: NixOS 23.11 (Tapir)
  UID Shift: 819658752
            Unit: systemd-nspawn@nixos.service
                  ├─payload
                  │ ├─init.scope
                  │ │ └─1909041 /run/current-system/systemd/lib/systemd/systemd
                  │ └─system.slice
                  │   ├─console-getty.service
                  │   │ └─1909375 agetty --login-program /nix/store/hlzi9rwycvpf907r5jhhl6v7090108sc-shadow-4.13/bin/login --noclear --keep-baud console 115200,38400,9600 vt220
                  │   ├─dbus.service
                  │   │ └─1909379 /nix/store/ai87d2awsm4xasaly144cjwk2k2b815l-dbus-1.14.8/bin/dbus-daemon --system --address=systemd: --nofork --nopidfile --systemd-activation --syslog-only
                  │   ├─nix-daemon.service
                  │   │ └─1909589 nix-daemon --daemon
                  │   ├─nscd.service
                  │   │ └─1909325 /nix/store/nd4yn9v9561ss4xcpr9166n02pddb0cg-nsncd-unstable-2022-11-14/bin/nsncd
                  │   ├─systemd-journald.service
                  │   │ └─1909316 /nix/store/sabybrrms75zv55a3nx2qsfyp9h5jbr3-systemd-253.6/lib/systemd/systemd-journald
                  │   └─systemd-logind.service
                  │     └─1909373 /nix/store/sabybrrms75zv55a3nx2qsfyp9h5jbr3-systemd-253.6/lib/systemd/systemd-logind
                  └─supervisor
                    └─1909038 systemd-nspawn --quiet --keep-unit --boot --link-journal=try-guest --network-veth -U --settings=override --machine=nixos

Jan 08 15:22:35 blueridge systemd-nspawn[1909038]: [  OK  ] Reached target Login Prompts.
Jan 08 15:22:35 blueridge systemd-nspawn[1909038]:          Starting D-Bus System Message Bus...
Jan 08 15:22:35 blueridge systemd-nspawn[1909038]: [  OK  ] Started User Login Management.
Jan 08 15:22:35 blueridge systemd-nspawn[1909038]: [  OK  ] Reached target Multi-User System.
Jan 08 15:22:35 blueridge systemd-nspawn[1909038]: [  OK  ] Started D-Bus System Message Bus.
Jan 08 15:22:36 blueridge systemd-nspawn[1909038]: 
Jan 08 15:22:36 blueridge systemd-nspawn[1909038]: 
Jan 08 15:22:36 blueridge systemd-nspawn[1909038]: <<< Welcome to NixOS 23.11.20230826.5237477 (x86_64) - console >>>
Jan 08 15:22:36 blueridge systemd-nspawn[1909038]: 
Jan 08 15:22:36 blueridge systemd-nspawn[1909038]: 

What’s missing here?

Running for the first time:

nixos-rebuild switch --option sandbox false

seems to solve the issue for me.

There are also some tips with mountpoints here: https://github.com/trofi/nix-guix-gentoo/blob/7a845f91f6f531faca670d79a24f0dc6a9b51878/README.md#missing-sandbox-support