This thread is to gather interest for Kubernetes on Nix. So far I am aware of a few others who are using either k8s or k3s on nix and I think it would be useful to create a centralized place to discuss issues and ideas. Currently there are no actively developed nix tools for managing cluster state and I think it would be good to either start a new tool or pickup one of the existing ones.
A matrix channel would be a great way to start this initiative but I’m unsure of how to go about getting one in the Nix space.
I personally run a small bare-metal Kubernetes cluster on NixOS (via the k3s module), and I’m looking to improve my setup and develop some tooling around managing it.
What you’re talking about sounds pretty closely related to that.
I’m in the same boat as you as far as a bare-metal cluster. My idea for running applications is similar to helm where you define all your manifests and simply substitute variables where needed. Perhaps this is a good use case for nickel+nix?
@azazel75
Maybe discourse works for now while we’re still in the planning phase but when it comes time for active development synchronous chatting will be essential.
My idea for running applications is similar to helm where you define all your manifests and simply substitute variables where needed
Kubenix uses the module approach for configuring resources, which is nice, IMO. It does not appear to be maintained currently, and there apparently a few open PRs that need attention. But it still looks like an excellent starting point.
This seems like what I’m looking for. There’s that big PR that updates Kubernetes that looks like it’s been stalled . I’ll add this link to my original post for visibility.
This weekend I’ll see if I can get this running on my local system and maybe we can go from there. Kubernixos looks interesting too but I’m worried about creating some mangled state with it but I don’t know much about it right now.
Edit: hmm looks like I can’t edit my original post.
For me the terraform abstraction seems to be good enough for k8s cluster state to not look into nix-based solutions too much, also I miss the nix module system in terraform.
I currently do not use NixOS as OS for Kubernetes simply because I do not host anything more than a local Kind cluster, yet would be freaking cool to have Nix immutability alongside Kubernetes.
Depending on how things play out, I can try to help at least with some K8S knowledge and single host (aka my laptop) tests.
I am not too sold on the Kubernetes model, but I have clients who still want it, so I am definitely interested into how I can reuse Nix as much as possible, so sign me up for interest :).
I’ve recently moved from talos to k3s on nixos (on raspberry pis) and started playing around with a fork of kubenix as a way to learn nix. If nothing else, I offer the is-it-isn’t-it-slightly-misaligned-hexagon-in-septagon logo
. I’ll add this link to my original post for visibility.
