Firecracker doesn’t support virtio and doesn’t seem to plan to anytime soon. And I found that my old Kata work is still on GitHub, though indeed quite old.
So here’s what I’m after now:
- kata containers (a
containerd-compatible runtime that runs workloads under various KVM VMMs)
- a fork of the container nomad task driver
where fork of
nomad-containerd-driver does a few different things:
- reads the image tag, recognizes some special url format
_NIX_:/nix/store/..... (or something)
- don’t fetch any image
- instead, fetch the store path
- creates a GC root for that path, in some dir that containerd manages for the task
- adds a
/nix:/nix mount to the container configuration for the task
- tells containerd to setup an empty root for the container, with the /nix+other mounts
This gets a multi-node workfload scheduler, allows us to re-use existing tooling for maintaining signed nix caches on various cloud blob providers, avoids any images, avoids snapshots, theoretically leverage virtio for solid isolation and virtio-fs for performant sharing of the /nix/store with the guests.
- update all of my pkg/modules for the latest kata stuff
- figure out state of containerd in nixpkgs, see how hard it is to teach it about a new runtime
- evaluate how to run nomad in nixos
- bitte? (very “mature”/“complicated”, tied to AWS, maybe over-kill, but maybe great?)
- hack on the nomad-containerd-driver to add the small nix features
- if you’re interested in this, please reach out. Even if you’re just a potential user, the biggest thing I need right now is motivation. Someone to bounce ideas off of, or be excited with, or just asking “hey, did you make any progress”, would go a long way.
- if this seems dumb, useless, or inadvisable for some reason, also let me know .