Enterprise Hydra Questions

Help
1

Hello again — I’m in the early stages of a nix migration for a large body of private sources and need to address the CI question. It comes down to Hydra vs more conventional systems like Jenkins, and I’m interested to understand more about Hydra:

  • How do I set up Hydra to sign and push outputs to a cache? Does this work with a cache that’s just a mount, like dav/nfs? Does it push each job as it finishes, or only when the whole jobset is complete?

  • If multiple overlapping jobsets are running at the same time, is Hydra smart enough to deduplicate the common dependencies, or will it just build each independently based on the cache state at the outset?

  • My org already has a snapshot versioning scheme in place for our code, but it seems my Hydra setup wants to track a moving branch for my flake at the project level. Instead of its arbitrary build numbering, can it trigger new evaluations within a project based on tags I feed it, and expose those tags in UI?

  • What’s the story for integrations? I know there’s an API, but what about webhooks? Are there a lot of consumers of the API so far? Anyone attempted a Jenkins/Hydra bridge for a hybrid process?

Thanks!

1 Like
2

I run my own Hydra instance, but it has a pretty basic config with only a local store, so I might not get some of these answers exactly right.

  • Hydra should theoretically work with any store that Nix supports, although I’m not sure how well tested some of these configurations are. The store path is configured using the store_uri option. For example, the official Hydra instance uses this config to sign and upload to S3: nixos-org-configurations/hydra.nix at 137e13bcf7039689cd567d88d41199bd1ee5fe0f · NixOS/nixos-org-configurations · GitHub. I’m pretty such each derivation is pushed as it finishes.

  • Hydra just runs the nix command to run builds, so most things that apply to normal Nix usage also apply to Hydra. This means that Hydra will never build the same derivation twice.

  • There is the GithubRefs plugin, which might work for you. Otherwise you might be able to write your own similar plugin that implements exactly what you want.

  • I don’t think Hydra supports webhooks, but you might be able to write a plugin to implement them.

1 Like
3

@lopsided98 Thanks for the tips! Regarding the cache, I will experiment with bringing up the mount and specifying it as a store_uri = file://mnt/thing affair.

I would agree about the never building twice, and I see now in the web UI that it shows a particular job is “part of evaluation xxxx (and yy others)” which definitely clarifies that it has that understanding built in.

I am also looking into what the story is with Hydra plugins and should be able to adapt what is there to my needs. Looks like the interfaces are defined in hydra/Plugin.pm at 2bb1ba22d3717f38a162e632471407f0c09923b0 · NixOS/hydra · GitHub and the fetchInput function that’s the basis of all the Pulls/Refs plugins is called from here:

I did a quick search across GitHub for hydraJobs definitions, but I’m having trouble finding good examples of what these plugins look like when they’re in use. Will continue to look and experiment, in any case.