Kubernetes controller for building Nix images

Development
1

Hey!

Ever since Graham’s blog post about the new Docker image builder, I’ve been thinking that it may be interesting to pursue the idea of having a Kubernetes controller that can build and serve (via the Docker registry protocol) images based on Nix expressions.

This would make it super easy to use images with any software packaged in nixpkgs in it, and also to deploy custom software built with Nix on k8s.

I’ve written down some more or less detailed thoughts about how to do this in a gist and would appreciate some feedback and/or general indication of whether this is interesting to anyone!

(cc: @moretea & @shlevy as suggested on IRC)

2 Likes
2

I really like this idea because it would allow us to abstract away the concept of even using nix. Are CRDs hard to implement in kubernetes?

3

Are CRDs hard to implement in kubernetes?

No, that part is basically trivial. You just tell Kubernetes that you have a new resource, and (if you want) you can tell it some basic properties/types to validate. After that it just gives you a CRUD API for your resource that immediately works with kubectl.

The difficult bits here are doing the registry protocol handling (IIRC the registry protocol requires two different forms of the content hash, which may be annoying to compute Edit: This has been fixed!) and figuring out where/when to actually perform builds. Both of those can be done in a simple way for a prototype, though.

Last time I tried to run a registry in a cluster it was also a bit annoying to address the registry in image names, but this may have improved … I’ll investigate :slight_smile:

4

Thanks. So I had a bit more time to think about this and I would like to ask what exact advantages would this have over say a Jenkins job that generates images from yaml templates. It seems to me this approach would be much simpler while integrating with other tooling that do not require kubernetes. Basically I’m asking if it really should be a task that kubernetes should be responsible for.

5

(sorry for the late response!)

while integrating with other tooling that do not require kubernetes

This idea is not useful for people who aren’t already using Kubernetes, it’s not intended to be a CI workflow for Nix projects. The goal is more like giving people who use Kubernetes an easy way in to benefit from Nix, that doesn’t require them to learn about / build Jenkins jobs. A sort of “plug&play Nix integration” for Kubernetes.