Give automated feedback when a commit causes breakage

Hypothesis
A package changing from building to breaking in Hydra is often manually discovered by users of the package. The author of the commit causing breakage often does not know about this breakage until it is manually discovered by someone.

Idea/Solution: Tightening the feedback loop
git bisect (Git - git-bisect Documentation) + Hydra could be used to track down the exact commit causing a breakage. Once the exact commit has been found - a bot could comment on this commit with a link to the first failed Hydra build.

That way, the author of the commit will get automated feedback that their commit broke something as quickly as possible, which means the package might spend less time overall being broken - improving the overall “robustness” of nixpkgs unstable.

Potential downsides

  • This might not be as common a problem as I imagine, and not worth spending a lot of time on
  • Performing these bisections every time something breaks might be too resource intensive on the Hydra infrastructure

I’d love to hear your thoughts on this!

  • Is the hypothesis correct (is this worth solving)?
  • Would it be too resource intensive for Hydra to do by default?
  • Are there some other downsides I didn’t consider here?
1 Like

Hydra has a feature to notify committers (without bisection), but “we” haven’t used it in the past few years. IIRC it was considered (by Eelco) to generate too large amount of e-mail notifications in practice.

Hosted by Flying Circus.