Hello, I am losing my marbles about the question above. I am new-ish to Nixos and Nix, so have mercy. However I have read my fair share of tutorials, manuals and guides about Nix, modules, nixpkgs. I can find my way, so to speak.
However, I need to base my workflow on containers. And no, I don’t want/need docker. Systemd-nspawn is something I have liked for some time, even before choosing Nix.
The point is that defining containers in /etc/nixos/configuration.nix has a good number of downs. So I read about imperative containers and they looked great. Until I tried to build a container to run a bunch of programs, vscode, with audio and wayland. And here comes the problem, because it looks like I cannot create a flake.nix file for the container with bind mounts defined in it. Even Chatgpt and Gemini say I must define the containers declaratively in order to implement the bind mounts.
Is this true? And is this the reason why someone created extra-container?
The systemd-nspawn docs document how you set up a bind mount here.
So, yes, you can just call systemd-nspawn imperatively from the command line with whatever args you want to create containers ad-hoc, and this includes bind mounts because there is an arg for it.
That said, how did you get the idea that you couldn’t? What do flakes - that would be inherently declarative and therefore completely unable to create something imperative - have to do with this?
Also, yeah, read the upstream docs instead of ChatGPT, it just repeats your biases to you; This has been a dying skill ever since stackoverflow became popular and I really recommend being one of the healthy few who actually maintain it.
Besides using the CLI, you can also write .nspawn “units”, and this is likely more ergonomic most of the time. Those are basically just an alternative to the CLI, but they are neat because you end up with a ton of arguments that can be quite complex. This helps tie your containers to their actual root file systems - with that said, these files have some limitations, read the docs for details.
FWIW, containers to run applications like this are a ton of work, since exposing all the underlying system details needed for them to actually run isn’t trivial. You also likely defeat your purpose in the process, precisely because you’ll have to leak those details. You’d probably do better with Flatpak, which is actually designed to do this for user-facing applications and has appropriate protocols in place to provide the system details without exposing them fully to the whims of the containerized application.
I see your points. But containers are a way, for me, to segregate environments. So the point is not in running a single program inside a container, but a set of programs, for example a dev env. So vscode/neovim, rust toolchain, etc… However, yeah it looks like I need to dive deeper into how the nix mechanics work. For examples many things appear like magic, like having programs.firefox.enable = true in configuration.nix. But, if I understand correctly, this is because there is a module in nixpkgs called firefox.nix that at some point in its code has options.programs.firefox = { enable = lib.mkEnableOption … } …and I have found many blog posts that use the term “options tree”, so I suppose this options property is some big shot…oh and the firefox.nix loses me where there is cfg = config.programs.firefox; …I am puzzled, because this means that in the config there is already a firefox object under programs…but shouldn’t be firefox.nix to define it?
Sure. You’re not getting much segregation if your container ends up just bind-mounting / read-write because you need to expose your graphics libraries, though.
Taking this to the logical conclusion of making specific exceptions for all the things that are needed leads you to reinventing Flatpak (which uses cgroups under the hood, just like systemd-nspawn, docker, podman, bubblewrap, etc.). So just use Flatpak instead of systemd-nspawn. In fairness, it’s harder to create custom Flatpaks - so I’d just use the vscode Flatpak and install the other stuff into it imperatively.
Well, I personally actually just create nix shells for each project I work on and don’t encapsulate my editor into a specific environment for separate projects, as well as forego the “physical” isolation of containers for development environments because I don’t think it’s necessary when using nix - nix shells are isolated enough for non-malicious use cases, and containers aren’t better enough at containing malicious code that I’d generally trust them. But that’s besides the point.
I don’t think nix is particularly relevant to your question; all of this can be done completely without it. Hence why I’m asking why you think nix is involved at all.
Yep, precisely. You’re describing the NixOS module system - this is NixOS, not nix. NixOS is a declarative system deployment tool implemented in nix and bash (and I guess technically some other languages).
Basically, what happens is that nix takes all your .nix files and merges the options and settings found in there together to get a big attribute set (call it object, tree, whatever). It then builds a big ol’ bash script out of all of these options which then ultimately creates some symlinks on your system (the “activation script”). If you call nixos-rebuild switch, nix does all of that and executes the script.
In the process of creating the script nix has to build a bunch of “derivations”, and their locations in the nix store are ultimately interpolated into strings which end up in the activation script. One of these is, for example, Firefox, which will be symlinked to /run/current-system/bin - as part of a whole stack of derivations that are all in turn symlinked recursively.
It’s not all that complex once you wrap your head around it, but if that’s too much at once I wouldn’t worry too much about it for now; as a pure user you can largely ignore these details, and pick them up as you go.
Once/if you want to learn more about it, follow the tutorials to learn the correct terminology and stuff before delving too much further into this, I imagine a lot of what I’m saying makes zero sense to you just because you’re not used to the words.
Either way, yes, you’re completely correct, it looks like a paradox, but it works because nix is lazy, which means that cfg ends up being defined by the time it is used despite looking like it isn’t. It’s much clearer that this actually works if you walk through the evaluation of lib.fix manually.
I like its declarative nature. Yes it is true that with other distros I can do the same, but it becomes a mess very fast. In theory many tools starting with Distrobox can do the trick, but in practice I see nix organizational principles as a way to put order in to the chaos. Bonus point it starts very fast. I am testing it in a VM with Plasma and holy…it is a breeze! Also, sometimes I have been forces to compile and install software maually, with the end result of having a bunch of binaries, config files, etc… scattered around my home directory and sometimes even in /opt and /usr/local. With nix I could write a derivation and have everything well organized.
The nix shells have been the main reason why I have chosen to dig into nix/nixos, because it looks like the ideal solution to my use case. At some point I started to think that although segregated at the level of PATH, nix shells cannot stop noisy programs from contaminating other things beyond the isolated environment. This is why I started considering containers. And I liked the idea that even using containers, everything goes into the host’s nix store, so maximum deduplication.
As of now I am using VMs, but the keyboard lags, video playback too, real useful 3d acceleration needs GPU passthorugh…