We’re maintaining several Flake repos at work that contain derivations for both internal and third-party software. We have a cron job that does nix flake update and opens a PR/MR with the new flake.lock file. We’ve had quite a few occasions where bumping Nixpkgs (or a third part flake input like e.g. crane) breaks our build and most of the time it’s quite time consuming to find the root cause and fix it on our side. I am thinking that the best way would be to run git bisect over each individual flake input to find which of its commits caused the breakage, as finding the commit that caused the breakage usually provides the needed context (e.g. PR discussion) to understand the problem and figure out how to fix it on our side.
While doing git bisect over each flake input sounds like a combinatorial explosion, my hope is that since the Nix flake update happens every week, the search space should be relatively limited. Also, it’s often the case that in a given week Nixpkgs in the only flake input that has been updated.
Has anyone had the need to solve a similar problem? Are there any preexisting scripts that automate this?
I’m considering writing a script that does something along the lines of:
- Cloning the repo of each flake input
- Using
git rev-listto find the list of commits that need checking, the range between the last known good commit and the earliest known bad one - Making a
git worktreecheckout for the flake input where git bisect will run - Making a
git worktreefor the consumer repo - Running
git bisectin the flake input worktree and for each step:- Updating the flake lock file in the consumer worktree to point to the checked out version of the flake input worktree
- Running the build script of the consumer repo
P.S. I am not familiar with Nixpkg’s tools and processes, but I have a feeling it is a treasure trove. I’m hoping my particular problem has already been solved there upstream, and that by learning more about the Nixpkgs development tools, I can apply them to my specific use case at work.