A nix-native CI setup with buildbot

Guides
1

I just want to share my buildbot configuration setup that I created for nix.

  • It uses nix-eval-jobs to evaluate attributes from a flake.nix in parallel.
  • It then creates for each attribute its own pipeline step and executes those steps in parallel each with their own build log
  • Additionally I added cachix upload support and also a flake update pipeline that merges automatically if the build succeeds.

It turns out while buildbot’s configuration and API is a bit harder to grasp, it’s very dynamic and flexible. And I was able to perfectly fit in nix
into it. While many CIs have their configuration in the source repo these days, for a public nix CI setup, it’s actually beneficial to not allow users to modify the CI code. Instead they can only submit nix code and not access other secrets.
Unlike hydra, pull-request based workflows and integration into github/gitlab etc is much better with proper status updates for failed and succeeded builds.

14 Likes
2

That sounds amazing!

(I was looking at Buildbot just a few days ago, as I was frustrated with Jenkins being so difficult to configure.)

3

In the same space, it might be worth sharing also: GitHub - input-output-hk/cicero: event-driven automation for Nomad

It is a nix-native workflow execution engine (currently) scheduled through nomad, though more advanced schedules might be dropped in in the future.

4

I now also deployed this setup in my university. It allows every user in our github org to access buildbot as well: https://github.com/TUM-DSE/doctor-cluster-config/tree/master/modules/buildbot

5

I have now put more effort into the buildbot. It now has a project view that gets build based on repositories that have the build-with-buildbot topic: Buildbot

1 Like
6

The project now lives in GitHub - Mic92/buildbot-nix: A nixos module to make buildbot a proper Nix-CI. It’s still early days, so if you are already relying on it, please pin the commit and check what changed before updating. I might incorporate major changes in the beginning.

3 Likes