Define custom script for Hydra jobset

Help
1

Hi everybody,

I’m trying to make NixOS installations that I use, more reliable by creating CI/CD pipeline that will build an ISO for me with newest updates and validate if it works. After that, if everything is ok, system update on the production servers will be triggered.

I discovered Hydra as a most Nix-native option to make CI/CD pipelines, but from the docs I’ve seen, it’s only being used(or only suitable) for building NixOS packages.

So I’m wondering, is there’s a way of defining custom scripts for the Hydra jobsets?

1 Like
2

Hydra might be quite a heavy weight solution to this problem.This might be a good use for flakes…

seems an excellent guide on build iso’s with flakes… as flakes can continuously track upstream git repo’s and ‘follow’ them, then it should be trivial to script this.

It maybe trivial also build this job with hydra , which seems to be defined in

nixos/release.nix
41:  makeIso =
146:  iso_minimal = forAllSystems (system: makeIso {
152:  iso_plasma5 = forMatchingSystems [ "x86_64-linux" ] (system: makeIso {
158:  iso_gnome = forMatchingSystems [ "x86_64-linux" ] (system: makeIso {
166:  iso_minimal_new_kernel = forMatchingSystems [ "x86_64-linux" "aarch64-linux" ] (system: makeIso {

it may be as easy to hotwire hydra with these options, and make it build one or all of these from hydra checkout of nixpkgs.

the nixos/release.nix defines the job for , say here.

https://hydra.nixos.org/job/nixos/release-20.09/nixos.iso_minimal.x86_64-linux

it will require some nixfoo to disable other jobs from the release.nix , or there may be a simple way to do it i’ve not discovered yet.

maybe @grahamc can correct me on this, as i’m still learning…

3

Some notes:

  1. The NixOS ISOs are just a “package”. You build the nixos iso derivation using the same method you’d build the firefox derivation.
  2. Thus, you don’t need a “script” for this, you just need a regular old hydra job.
  3. We have other hydra jobs for the various NixOS isos that you can use for inspiration: Hydra - nixos:trunk-combined:nixos.iso_gnome.x86_64-linux
2 Likes
4

Sorry for my ignorance at this point, but I’ve tried running through the Hydra Job, that you provided link to, but failed to find a way to scrape a .nix config and spec.json file, describing the job. Could you please, if possible, advise, where should I search for it?

Thanks in advance

5

For what it’s worth, I don’t think you’re ignorant, I don’t think hydra is the easiest to use/comprehend or setup. The most helpful thing I can do is just give you pointers, since I just went through this:

Hope it helps!

Extra notes:

  1. Flakes is just some import syntax, it’s not necessary here. You can do declarative jobs, etc, all the same, the hydra config is just slightly different
  2. I think the hydra specs part, where the project is actually configured - I think I still use non-flakes for that (hydra-specs.nix → flakes-compat → #hydraSpecs), but I think hydra supports a flake ref here too, I haven’t tested yet.

Extra random notes, apropos of nothing:

  • It would be really cool if hydra could decompose flake references and track them. I’d love to see "all hydra jobs where nixcfg was an input, or “show me the inputs and their rev changes that occurred for this instance of a job”. You could see failures based on inputs pop out, etc.
  • shout out to clever from freenode for the hydralib that I stole to generate the jobspecs
  • shout out to nixpkgs ‘stdlib’ somewhere where I stole the beginnings of the hydra machine line builder and huge shout out to @grahamc for the pointer about configuring a specific identity key for a given machine (which is now incorporated in the machines-txt-builder helper), and some others I’ve forgotten.
1 Like