Ephemeral containers for devs

Help
1

I have just read about using NixOS with tmpfs root and tmpfs home which looks very interesting to me, however I feel like it is too much to begin with. So my question is, can I somehow replicate it with containers, especially with regards to development environments?

My use-case is: I often contribute to projects just once or very sporadically. It is often that someone writes me Could you please have a look at this project and do this or that modification? I clone their project, understand their dev tools and set it up using nix-shell -p … or writing down shell.nix. So far it is just great and nix saved me a lot of time. However the problem is, many of the tools used a stateful. NPM/Composer manages its cache in the home directory for example. So after running npm install my disk space shrinks and is not released after being done with the project a deleting it.

Before switching to NixOS, I was using Docker to fight this problem. I just spawned a clean image and mounted the code there. I then installed all the deps otherwise filled the container with garbage. But thanks to the nature of containers, it was all gone when I deleted it.

Is there a better, pure Nix way? My idea is setting up an imperative container with the project mounted into it and having everything inside mounted with a tmpfs overlay or tmpfs over the home dir and everything else as read only or so. I could find some mentions about containers ephemeral option, but without any details.

I have also come across nixos-shell, however the last commit is 6 year ago and there seems to be no community nor any docs. Is is usable for this kind of problem?

How do you personally do development not to fill your machine with garbage?

Thanks in advance

2 Likes
2

Can you specify the exact problem you’re trying to solve? Is it just reclaiming disk space for unused Nix packages?

If that’s the case, the default behavior of garbage collection already has you covered. By default GC roots are not established for nix-shells, and thus running nix-collect-garbage -d will run garbage collection and should clear any Nix packages that you had downloaded, but are not part of your booted/current system configuration.

3

Nix is fine. My problem is with foreign package managers and dev tools, JS npm, PHP composer and similar. I have found a workaround with podman starting with en empty root fs, mounting the nix store and project dir and tmpfs over empty home. This way, I can keep developing in my well known environment while running these messy commands (composer install etc.) inside a well defined jail. It works, but isn’t there a more Nix way?

I am also very interested in any tips and tricks how you, Great Nixers, do development in Nix machines. I have already fell in love with Nix, however I believe there is still much more potential a haven’t utilized yet.

4

Could you elaborate a bit more on what the development environments/containers should look like. Since you mention npm, composer, etc. is the idea to get containers that have follow the FHS layout (for these third-party tools) but provide Nix to install binaries etc.?

5

The idea is to have an environment that provides access to Nix binaries and a specified project folder and discards modifications to any other location (or marks it as read-only). FHS is not needed for me.

To be very specific: I enter the environment. I run npm install. It creates node_modules in the project directory and a bunch of files in my home. The node_modules folder is immediately visible to processes outside of the environment. The modifications to my home dir are not. When I exit the environment, all of the modifications to my home dir are discarded. The node_modules folder persists.

1 Like
6

I understand now, but I don’t know of such a tool. It sounds like a great idea. There is a newer(/different?) nixos-shell, that @mic92 has here: GitHub - Mic92/nixos-shell: Spawns lightweight nixos vms in a shell but I still don’t think it quite does what you’re asking.

7

Your question touches on several things I’ve been meaning to look into, but I can only respond in an “I want this also!” manner.

The PR that introduced the ephemeral option to nixos containers may be a useful reference.

8

Have a look at my own project extra-container. I’m using it mainly for developing services and NixOS features, but it may also be useful for your task.

Here’s how to setup a container with private networking that bind-mounts your working directory to /src:

read -d '' src <<'EOF' || true
{ config, pkgs, ... }:
{
  containers.demo = {
    extra.addressPrefix = "10.250.0";
    bindMounts.src = {
      hostPath = toString <pwd>;
      mountPoint = "/src";
    };
    config = { /* add your config here */ };
 };
}
EOF
extra-container shell -E "$src" --run c ls -al /src

For interactive use, run it like this:

extra-container shell -E "$src" --no-destroy --run bash -c 'c; extra-container stop $name'

This starts a shell inside the container and stops the container on shell exit. The container contents are preserved for subsequent runs and can be removed with extra-container destroy demo.

1 Like