I have a few mildly complex projects that use nix as their build system (some using non-standard targets or compiler toolchains, some mixing build products from different architectures, some using Import-From-Derivation), and spent some time thinking about the ideal CI system.
Because of the custom dependencies, any CI system must use and fill a custom nix cache. But thanks to the great Cachix, that part is solved.
I looked at @roberth’s Hercules CI and set it up for one of my open source projects. The setup was smooth, I was able to quickly run a runner on one of my Debian servers, and it it mostly did its job. But I quickly noticed that while it was great at building derivations and filling the nix cache, I was rather limited in my control of what to build, and what to do after the build.
So I mostly moved back to plain Github Actions (with cachix, of course). This would allow me to do things like
- Run a set different set derivations depending on, say, Github labels.
- Do “differential CI”, where I compare the build output against the base branch (imagine a compiler project, and any PR with label
refactoringshould still produce identical output on the test files) - Have a wealth of Github Action to choose from to do something after the build (post comment, create Github releases, push code lint fixes to the branch, upload documentation etc.).
I saw that Hercules CI has a beta feature called Effects, but it seems like it would require a lot reengineering to be able to do what I can do with Github Actions.
So pragmatically, I am currently inclined to got with Github Actions.
Yet it pains me, because when it comes to the “build this derivation” part, it’s just so dumb:
- A simple
nix-build -A foowill, if the output is in the cache, still download it for no good reason, instead of just reporting success. (This part can probably be improved.) - Two jobs in parallel that build the same or overlapping derivations will both build it, instead of one waiting for the other.
- Parallelism is limited to the Github runner
- A dedicated Hercules CI runner will likely have a warm
/nix/store, making builds much faster
So what I think I really want is a Github Action step “Build on Hercules CI”, which builds a given path and attribute on Hercules CI, failing if it cannot be built. This would give me most of the benefits of Hercules CI, and allow me to combine them with most of the benefits of Github Actions. Would that be feasiable?
Taking this thought one step further, it seems that Hercules CI is, in a way, just a smarter remote builder. So maybe one can have a hci build command that works like nix build, but builds stuff on Hercules CI, which may be useful even for local development sometimes?
)