Best practices when building container images using nixpkgs tooling

tguzik · July 25, 2026, 10:07am

Hey,

I’ve run into a situation where in order to run some application in a reasonable way (e.g. run it 24x7 on a low-power host that did not run NixOS), the path of least resistance was to put it in a container.. but the issue was that the app required some odd system dependencies that were awful to wrangle into submission. The application was nothing to write home about - just some microservice with a GUI I made for myself ages ago, but I thought that instead of starting with a common base image like one of the Debian or Ubuntu ones I could use nixpkgs to trim down unnecessary cruft, started to build it, ran into some issues, yadda yadda yadda.

Long story short I thought I would put together a semi-copypasta-able recipe on how to structure the derivations to build a decent container image with the thing I wanted to run.

I think I’ve already got the table-stakes:

One derivation to put together the dependencies of the application (the recipe uses a placeholder binary, so my original usecase doesn’t pollute the scripts with too many specifics),
Second derivation to build the actual container image, including a shell and some utilities besides the app,
Including a default configuration file in the container,
Having the application run in a well-known directory (/app) in the container so that configuration can be easily changed without rebuilding the image,
Having a GitHub actions pipeline that would actually build the container using a public runner.

I’ve left out things like running with non-root user or limiting the CPU/RAM quotas to the runtime configuration - I’ve ran into issues with uidmapping on public GitHub runners, so I didn’t want to overcomplicate it with “deep magic” that can break whenever GH updates security policies on their runners again.

Anyway, the recipe is at: https://github.com/tguzik/containerize-app-using-nix

Did I miss anything important? Would adding basic derivation tests (start the app & verify it prints given string) be worth the effort? What would be the best practices for building containers using tooling from nixpkgs anyway?

I’d appreciate y’alls feedback. If you think this entire thing is a stupid idea, feel free to bash me too

bitbloxhub · July 25, 2026, 5:24pm

Public GitHub Actions runners do not support kvm.

is wrong if your repo is public, just FYI. They provide it for free on public repos (although it is undocumented AFAIK).

github.com/cachix/install-nix-action

install-nix.sh

630ae543e


      
          if [[ ($OSTYPE =~ linux) && ($INPUT_ENABLE_KVM == 'true') ]]; then
            enable_kvm() {
              echo 'KERNEL=="kvm", GROUP="kvm", MODE="0666", OPTIONS+="static_node=kvm"' | sudo tee /etc/udev/rules.d/99-install-nix-action-kvm.rules
              sudo udevadm control --reload-rules && sudo udevadm trigger --name-match=kvm
            }
          
            echo '::group::Enabling KVM support'
            enable_kvm && echo 'Enabled KVM' || echo 'KVM is not available'
            echo '::endgroup::'
          fi

buildLayeredImage is AFAIK the better way to build images with dockerTools in general, although you can run into the layer limit.

If you control the hosts and can run a Nix daemon on them (maybe not the case here, but in general) I recommend nix-snapshotter/nix-storage-plugin for building containers with Nix.

TLATER · July 26, 2026, 6:06am

I don’t understand why you build a separate derivation with dependencies. Just adding the package (“appBinary”) itself to the container’s contents (or even Cmd) should pull in the full closure and “just work”.

This isn’t so much “best practices” as I don’t understand your use case here. I can imagine a scenario where something like this would be handy, but you’ve not really presented a scenario with a need for this. Maybe you’re just doing extra work because you don’t understand something?

Also, if you just want to run something on a system with arbitrary libraries, nix bundle is a simpler approach than shipping around OCI containers (and it also supports turning arbitrary derivations into OCI containers, kinda forgot about that).