A nix-native CI setup with buildbot

I just want to share my buildbot configuration setup that I created for nix.

  • It uses nix-eval-jobs to evaluate attributes from a flake.nix in parallel.
  • It then creates for each attribute its own pipeline step and executes those steps in parallel each with their own build log
  • Additionally I added cachix upload support and also a flake update pipeline that merges automatically if the build succeeds.

It turns out while buildbot’s configuration and API is a bit harder to grasp, it’s very dynamic and flexible. And I was able to perfectly fit in nix
into it. While many CIs have their configuration in the source repo these days, for a public nix CI setup, it’s actually beneficial to not allow users to modify the CI code. Instead they can only submit nix code and not access other secrets.
Unlike hydra, pull-request based workflows and integration into github/gitlab etc is much better with proper status updates for failed and succeeded builds.

11 Likes

That sounds amazing!

(I was looking at Buildbot just a few days ago, as I was frustrated with Jenkins being so difficult to configure.)

In the same space, it might be worth sharing also: https://github.com/input-output-hk/cicero

It is a nix-native workflow execution engine (currently) scheduled through nomad, though more advanced schedules might be dropped in in the future.

I now also deployed this setup in my university. It allows every user in our github org to access buildbot as well: doctor-cluster-config/modules/buildbot at master · TUM-DSE/doctor-cluster-config · GitHub

Hosted by Flying Circus.