Kubernetes on Nix Community Organization

Development
#1

Hello,

This thread is to gather interest for Kubernetes on Nix. So far I am aware of a few others who are using either k8s or k3s on nix and I think it would be useful to create a centralized place to discuss issues and ideas. Currently there are no actively developed nix tools for managing cluster state and I think it would be good to either start a new tool or pickup one of the existing ones.

A matrix channel would be a great way to start this initiative but I’m unsure of how to go about getting one in the Nix space.

Pinging a few people who might be interested.
@euank
@superherointj
@Mic92
@bryanasdev000
@kalbasit

6 Likes
#2

I’d like to register interest :slight_smile:

I personally run a small bare-metal Kubernetes cluster on NixOS (via the k3s module), and I’m looking to improve my setup and develop some tooling around managing it.

What you’re talking about sounds pretty closely related to that.

#3

I’m using k3s as well! I would prefer to use this discourse as a central place to discuss… Having many matrix channels open is becoming …overwhelming.

#4

I’m in the same boat as you as far as a bare-metal cluster. My idea for running applications is similar to helm where you define all your manifests and simply substitute variables where needed. Perhaps this is a good use case for nickel+nix? :thinking:

@azazel75
Maybe discourse works for now while we’re still in the planning phase but when it comes time for active development synchronous chatting will be essential. :grinning_face_with_smiling_eyes:

#5

My idea for running applications is similar to helm where you define all your manifests and simply substitute variables where needed

Kubenix uses the module approach for configuring resources, which is nice, IMO. It does not appear to be maintained currently, and there apparently a few open PRs that need attention. But it still looks like an excellent starting point.

#6

There was Kubenix: call to rejuvenate last year.

2 Likes
#7

We use kubernixos https://github.com/DBCDK/kubernixos.

#8

This seems like what I’m looking for. There’s that big PR that updates Kubernetes that looks like it’s been stalled :frowning_face:. I’ll add this link to my original post for visibility.

This weekend I’ll see if I can get this running on my local system and maybe we can go from there. Kubernixos looks interesting too but I’m worried about creating some mangled state with it but I don’t know much about it right now.

Edit: hmm looks like I can’t edit my original post. :thinking:

1 Like
#9

For me the terraform abstraction seems to be good enough for k8s cluster state to not look into nix-based solutions too much, also I miss the nix module system in terraform.

#10

Maybe @jmgilman Nixago project could help here. Also @blaggacao old ambition to dig kubenix/terranix for a builtin JSON schema validator.

#11

First, Sorry for the late response :stuck_out_tongue:

I currently do not use NixOS as OS for Kubernetes simply because I do not host anything more than a local Kind cluster, yet would be freaking cool to have Nix immutability alongside Kubernetes.

Depending on how things play out, I can try to help at least with some K8S knowledge and single host (aka my laptop) tests.

I know that in the past, people brought up the discussion to have separate repos for K8S in Nix, see (https://github.com/NixOS/nixpkgs/issues/115179) as a way to fast iterate and simplify nixpkgs abstraction, I do not know how successful they were but is another starting point to discuss and organize a Nix Based Kubernetes install.

#12

I am not too sold on the Kubernetes model, but I have clients who still want it, so I am definitely interested into how I can reuse Nix as much as possible, so sign me up for interest :).

#13

Interested.

I’ve recently moved from talos to k3s on nixos (on raspberry pis) and started playing around with a fork of kubenix as a way to learn nix. If nothing else, I offer the is-it-isn’t-it-slightly-misaligned-hexagon-in-septagon logo :sweat_smile:

1 Like
#14

The cloud-native channel is fairly inactive; I say we just pile in there until (and if) someone kicks us to a separate channel.

#15

there are no actively developed nix tools for managing cluster state

What do you mean? Can you be more specific? Which ‘cluster state’ you refer?

#16

Hello Hall!
Nice to see you around. I hope you have a nice time here.
Feel free to ping me, mark for reviews in nixpkgs or whatever.

Hosted by Flying Circus.