I’ve been working for a while on a couple of projects that are meant to come together to Nixify Kubernetes that I’d like to share. They’re not battle tested, docs are sparse and examples are rough.
easykubenix
easykubenix is very heavily inspired by kubenix. Both use the NixOS module system to render manifests. The main difference is that instead of codegen to replicate the full Kubernetes API surface in Nix I use (pkgs.formats.json {}).type to cover all possible resources, and to have API validation generate a script that spins up an ephemeral etcd+kube-apiserver and apply the manifests.
It bundles a deployment script using kluctl which is an amazing deployment tool if you like being able to work from the CLI and not just GitOps (and it’ll handle secrets so we don’t write them to store)
Supports rendering Helm charts, supports importing YAML files. Also supports setting _namedlist on an attrset to convert it into a “named list” in the rendering stage to allow you to reference containers as attrset rather than list.
easykubenix module example:
{
config.kubernetes.resources.namespace.Secret.name.stringData.secret = "nix is awesome";
}
Status: Usable
nix-csi
TL;DR: mounts /nix into pods, shares storage, doesn’t require NixOS nodes, /nix/store views are created per pod so it’s not global.
A “reimplementation” of nix-snapshotter but on the CSI “layer” instead of CRI layer, this means you can run it on any Kubernetes by just deploying the CSI driver, it doesn’t require the node OS to be NixOS. Underneath it uses hardlinks to create a “view” of a shared Nix store, if the volume is RO the hardlink folder is then bind-mounted into the pod, if the volume is RW it’s overlayfs mounted into the pod (so you can run nix commands in the pod). Just like nix-snapshotter this means in RO mode you share inodes all the way and therefore page-cache making this way of deploying more RAM efficient than container images.
It’ll link the root derivation to /nix/var/result so you can easily run binaries from it, also initializes the Nix database.
I’ve also implemented an optional cluster-local binary cache using dinix to run openssh and nix-serve-ng (More work needed)
Status: Works. Deployment docs are not there, image is on quay. It’s opinionated towards Lix currently but that’s just because I use Lix myself and it’ll be configureable soon
dinix
dinix uses the NixOS module system to render dinit configurations + start scripts. dinit is a great process supervisor that’s able to run as PID1 in containers or as a normal userspace supervisor from your terminal or systems or any other way to run processes. It’s similar to NixNG in a sense but laser focused on dinit only.
dinix module example:
config = {
services.boot = {
type = "internal";
depends-on = [ "nginx" ];
};
services.nginx = {
type = "process";
command =
pkgs.writeExeclineBin "nginx-launcher" # execline
''
${lib.getExe pkgs.nginx} -c ${nginxConfig} -e /dev/stderr
'';
restart = true;
options = [
"shares-console"
"pass-cs-fd"
];
};
};
Status: Works, renders all dinit options properly through my tests
A match made in heaven
I know this works even though I haven’t written these workflows yet: You can nix build a easykubenix manifest referencing a dinix startup script in a nix-csi volume, push the manifest derivation to your build cache then apply the manifest with the easykubenix/kluctl deployment script to have your full container lifecycle managed through Nix. This means you can run a COMPLETELY EMPTY container image (you still need one because Kubernetes) and have the power of Nix while doing it.
That being said, you can use only easykubenix, only nix-csi or only dinix depending on what your usecase is.
Final words
While I realize it might be a bit premature to announce software without stable releases I’ve been working on these on and off for awhile and would LOVE to hear from the community.
Other things I wanna do: terranix but for terragrunt long live the module system!
If you’re curious, clone the repositories and run nix repl --file . in each of them to discover, they don’t bite!
So with all that said, please ask questions, give feedback… Like and subscribe, ring the bell, join my membership, Patreon (/s). I’ll try to keep this post updated as things progress and questions are answered 
Come say hi on Matrix Nix: Cloud Native
Anything where you can run privileged containers will work 
I’ve still got to implement garbage collection (I do setup gcroots already) and several small improvements here and there.
