Nix services separated by user - NAS

Help
1

Hello,

I would like to use NixOS for a NAS I plan to build. I have two main questions about using NixOS for this purpose, they are:

  1. Should I use containers, or packages/flakes?
    It seems like all the instructions online use containers and I know that NixOS has the ability to use containers, but it seems like unnecessary complexity when near 95% of the programs I want to use are in Nix packages. In addition, I want to be able to rollback easier and distrust containers more than the Nix way of doing things.

  2. How compartmentalized should I make the services?
    I read a great article entitled Paranoid NixOS Setup where the author made the following statement:

Each Service Gets its own User Account

I am going to use the word “service” annoyingly vague here. In this world, a “service” is a human-oriented view of “computer does the thing I want it to do”. This website you’re reading this post on could be one service, and it should have a separate account from other services. See here for more information on how to set this up.

If you read the other article in the link above, it seems to make sense, but is there additional maintenance that is required with this setup? I am a newer user and worry I am going to mess something up. Also, how would this be done with a flake instead? Would each service be put into a flake, and can a flake add a user account?

Should I have a seperate user account for the following (examples):

  • JellyFin
  • BookStack
  • Bitwarden

Thank you,

Ben

2

The NixOS configuration options remain the same whether you use flakes, or not. Usually one flake is enough to configure a NixOS system, or even several systems, unless you have a good reason to split it up to multiple flakes.

Yes.

Most services under NixOS already use their own separate (often dynamic) users: see Jellyfin, Bookstack, Vaultwarden.

You can also look into NixOS containers, but again, most NixOS service modules implement basic isolation / hardening measures, and you might not need containers.

2 Likes
3

To elaborate with a little more clarification, which I sense is needed.

Firstly, the “containers or flakes” part of the question is a bit off-target. Flakes are a way of organising your nix source, but don’t inherently change what is built and how it runs. It’s not really a distinction that matters (directly) to this question. It might matter indirectly, say if someone has published a service configuration for the thing you want to run, in the form of a flake separate from nixpkgs.

As for the containers part, I suggest a slightly different emphasis: what you’re looking for is isolation or containment of services in various ways, including cpu and memory limits, filesystem visibility, network access. In NixOS that comes in a few practical forms, building up in complexity:

  1. systemd hardening: configuration to restrict (contain) the running process(es)
  2. NixOS containers: building a smaller specialised instance of NixOS userspace in which to run the service
  3. Container images: building a smaller(?) specialised instance of an operating system userspace in which to run the service, perhaps using third-party images and configuration (like some project’s published docker)

The goal is to keep them apart, so that (for example) your JellyFin media indexer can’t see your personal documents file share or bitwarden data.

These all use the same mechanisms (cgroups, linux namespaces, bind mounting bits of shared data) and, to a first approximation have roughly the same isolation capabilities. The difference is in how they’re managed, and what the running code expects to see.

The third might be what you’re thinking of as “containers”, especially if you’re focusing on the contents rather than the stackable standardised shipping boxes when you say:

That sounds to me like you’re thinking of running someone’s docker image (option 3) when you think of containers. Option 2 gives you “the Nix way of doing things” in a container, for those things that have a NixOS services.* entry. And as pointed out, for some of those (it varies quite a bit, alas) there is already enough isolation / systemd hardening in the basic service config.

The nice thing is that you can use all three together where it makes sense. Sometimes it might be easiest to use (say) pi-hole as a packaged software appliance.

1 Like
4

Thanks @justinas and @uep! I did look and some of the services I want to use include a username and some do not, so I will look to use a combination of NixOS containers and the standard services where it makes sense.

Edit: on second thought, would it be safer to use nix containers everywhere?

5

“Safer” is not really something that can be answered without a good understanding of the threat model you need to defend against.

It will be (mildly) less convenient, though at least most of the hassle of other kinds of containers will be removed: for example, they all update together with the host. Most of the inconvenience will be a few extra lines of config, and perhaps a few more to be explicit about which host filesystem paths to expose to the service.

So that cost isn’t much, but it could be basically redundant (no more safe) if the systemd config is sufficient. It might be worth it anyway, to be more explicit about data paths, and more consistent with how you organise your config and specify those settings.

As a broad assumption, it can’t be less safe: the systemd service config is the same whether wrapped in a container or not.

But most of the caution comes from the fact that we can’t tell you which is safe enough for your needs.

There are limitations (that you can read about) to NixOS and systemd-nspawn containers, primarily that they (by default):

  • run privileged (root is the same as on the host)
  • share the host’s /nix/store, and so can learn a lot about the host and its config from there, and run binaries and tools they find there even if they wouldn’t normally be part of the container config

FWIW I found this article a good and fairly quick summary a while back: https://blog.beardhatcode.be/2020/12/Declarative-Nixos-Containers.html

And there’s also plenty to read about the systemd service config options that can be used to harden services as well. Settings like PrivateUsers= and DynamicUser= affect the “user id” even if a username is not listed explicitly, if that’s all your’re looking for.

1 Like
6

I appreciate the guidance. My threat model is low; the person that lives behind me is Amish :slight_smile: (true btw). I am more concerned with understanding security and using this as a learning experience.

1 Like
7

I tried to run home assistant in a non-root container and found it challenging. It seems like it would he possible, but seems not easy. Following this thread with interest.