Hello! This is Edgar and Robbie, and we’re excited to share with you the public release of nix-snapshotter! nix-snapshotter brings native understanding of Nix packages to containerd, the industry standard container runtime.
We built this because Nix is a great fit for making efficient containers. They don’t need an OS because Nix captures all dependencies exactly. However, the current process of creating Nix images is subpar because one needs to transform Nix packages into a format that container runtimes understand.
Using nix-snapshotter, instead of downloading image layers, packages come directly from the Nix store. Packages can be fetched from a binary cache or built on the fly if necessary. All existing non-Nix images continue to be supported, and Nix layers can be interleaved with normal layers.
nix-snapshotter also provides a CRI image service, which allows Kubernetes to resolve image manifests from Nix directly too. This enables for the first time, fully declarative Kubernetes resources, all the way down to the image specification and its contents. With this, you can even run pure Nix images without a Docker Registry at all, if you wish.
nix run "github:pdtpartners/nix-snapshotter#vm"
nixos login: root # (Ctrl-a then x to quit)
Password: root
# Running `pkgs.hello` image with nix-snapshotter
nerdctl run ghcr.io/pdtpartners/hello
# Running `pkgs.redis` image with kubernetes & nix-snapshotter
kubectl apply -f /etc/kubernetes/redis/
# Wait a few seconds...
watch kubectl get pods
# And a kubernetes service will be ready to forward port 30000 to the redis
# pod, so you can test it out with a `ping` command
redis-cli -p 30000 ping
The link that probably everybody in here is looking for:
This is great stuff!
Have you been in contact with the OCI committee? About a year ago similar ideas where tossed around. If not, it might be worthwhile to present there. Folks seemed pretty interested and open to those ideas.
A long term goal could be to make this a recommended extension of the runtime spec.
Edit: I also think nlewo/nix2container may deserve some honorary mention in the “how is this different” section. That project move the goal post of more efficient OCI workflows by one important marginal step for ~2 years now.
Thank you! That’s a good idea, we’ll see if they are interested in a presentation. We have been to containerd meetings to give them a heads up and made several upstream contributions but that is separate from the OCI committee.
I think it will be beneficial to have it documented/recommended, but I don’t think we should modify the runtime spec. Adding a new media type suggested in your issue will require native support in all the container runtimes, whereas in our design it works with off the shelf containerd/Kubernetes.
Yes, the NixOS and home-manager modules both set up containerd with nix-snapshotter without Kubernetes, see installation steps. For example with the NixOS module, all you need is services.nix-snapshotter.enable = true and it’ll configure containerd for you as well.
I don’t recommend virtualisation.oci-containers.containers since it only works with docker or podman backend, and we don’t have support for either atm. See tracking issue
Have you read through the installation steps? You need to set the following:
services.nix-snapshotter = {
enable = true;
# Sets CONTAINERD_SNAPSHOTTER=nix
setContainerdSnapshotter = true;
};
# A CLI for containerd, so you can run `nerdctl run --rm ghcr.io/pdtpartners/hello`
environment.systemPackages = [ pkgs.nerdctl ];
Ah sorry, I misunderstood. We don’t have declarative containers yet, but please do file an issue we can track this. Welcome any ideas or contributions.
For this feature in particular, it’s more about containerd than nix-snapshotter. Perhaps this just involves adding containerd support to the virtualisation.oci-containers.containers backends.
Does running nerdctl run <image> automatically fetch nix store path from my configured substituters? It didn’t seem to be the case, but maybe I needed to restart something. (note that I built the image on another computer, so the files are missing from the nix store).
Yes, it’s meant to run nix build <store-path> under the hood, so it should use your configured substituter. You can check journalctl -u nix-snapshotter to see the logs if something is going wrong.