Https://0pointer.net/blog/running-an-container-off-the-host-usr.html

Links
1

https://0pointer.net/blog/running-an-container-off-the-host-usr.html

2 Likes
2

Did you try it? I got:

❯ sudo systemd-nspawn \
        --directory=/ \
        --volatile=yes \
        -U \
        --set-credential=passwd.hashed-password.root:$(mkpasswd mysecret) \
        --set-credential=firstboot.locale:C.UTF-8 \
        --bind-user=bbigras \
        -b
Spawning container desktop on /.
Press ^] three times within 1s to kill container.
Selected user namespace base 1756692480 and range 65536.
Failed to set up ID mapped mounts: Device or resource busy
3

There are a few problems on the way to using this directly on NixOS:

  • /bin is a normal directory, so we’re not considered a “merged /usr distribution”. Well… that’s funny because we have almost no /bin nor /usr. Only have them for pragmatic/POSIX purposes. But meh… make a symlink to sh in /usr/bin and symlink /bin to /usr/bin. source Next…
  • Our users by default go into the “users” primary group. Instead this trick expects users to be in a self-named group. So a quick users.users.<name>.group = "<name>"; users.users.<name>.extraGroups = ["users"]; users.group.<name>={}; source Next…
  • Then systemd wants itself to live in /usr/ or /lib or /sbin (wait, aren’t these supposed to be merged?) source.

At this point I’d ask what is the point? From the blog post: “Oftentimes I want to run my development code on my PC but be reasonably sure it cannot destroy or otherwise negatively affect my host system.” So it sounds like you just want NixOS?

@lennart, if you ever read this: NixOS’ers have been doing essentially this for quite some time with nixos-containers. Using your systemd-nspawn since 2013! source

  • ephemeral (similar to the volatile): link
  • bindMount home: link
  • arbitrary credentials: use declarative containers link
  • user remapping… okay, that’s kinda cool, but should be clear that it would be relatively easy to implement/adopt.

Follow-on: should we symlink bin itself in NixOS instead of just /bin/sh ?

2 Likes
4

I think you have to set subuid and subgid to your user. That’s my output:

❯ sudo systemd-nspawn \
              --directory=/ \
              --volatile=yes \
              -U \
              --set-credential=passwd.hashed-password.root:$(mkpasswd mysecret) \
              --set-credential=firstboot.locale:C.UTF-8 \
              --bind-user=lennart \
              -b
Spawning container watchmen on /.
Press ^] three times within 1s to kill container.
Selected user namespace base 1709375488 and range 65536.
Sorry, --volatile=yes mode is not supported with OS images that have not merged /bin/, /sbin/, /lib/, /lib64/ into /usr/. Please work with your distribution and help them adopt the merged /usr scheme.
1 Like
5

Recently I wondered what it would take to make nixos-container work on non NixOS distros like Ubuntu. I’m guessing the best answer to that is: incorporate extra-container into nixos-container and support it better as a community?

2 Likes