What commits belong to which channel

1

I want to optimise bisection workflows. Perhaps with some script even.

Currently, for any bisection over nixpkgs, you usually quickly run into a commit that has a lot of dependencies involved, not built by hydra, because you get a staging-merge or end up on staging itself, due to the way how the bisection went.

So my idea was, to do a 2 pass bisection, to avoid the amount of necessary rebuilds.

In the first pass only consider commits that have been built by hydra and considered to be good enough to be a new channel push. git bisect skip all the commits that are not.

In the second pass consider all the commits between the last known “good” and “bad” commits.

To be able to implement this though, I’d need such a list of commits that hydra created a new channel push from, is there an easy way to get such a list of commits? I think it would still be sufficient for my purposes, if this is only available for the last year of the unstable channels and the last 2 “full” release channels.

Does someone have an idea?

My current approach is to nix eval some outPaths of well known and expensive derivations, which in itself costs some time, extract the hashes, query https://cache.nixos.org/$hash.narinfo for availability and skip if any of them is missing.

This list currently for me contains gcc, llvm, rust in various versions, as well as systemd. I include the very same checks in every bisection I do from the start, ignoring whether an individual from this list would make sense, after python regularly surprised me with 3 LLVM rebuilds regularly because of some cryptography dependency.

I hope to cut a lot of the intermediate evals when I know in advance whether or not a commit has been built by hydra or not.

1 Like
2

nix-bisect allows you to skip commits that would trigger more rebuilds than the max-rebuilds flag.

And there is also hydrasect, which sounds closer to what you want.

2 Likes
3

Nice feature! Not quite exactly what I want though. This might skip some stuff where the “builds” are basically no-ops but many of them, while the dreaded hour long builds of LLVM or a GCC bootstrap cycle can still happen.

I agree this is unlikely, but possible in general.

Yes, nice tool, I think I will at least take some inspiration from it.

Feels a bit weird, that there is no nix expression in the repo for the build…

Thank you!

4
1 Like
5

I usually just do a first-parent bisect on master. A commit being on master usually means that it’s built. There’s only very rarely >1 rebuild of some package in the couple hour window between hydra evals.

That way the pain only begins when the first pass lands you on a staging-next merge.

1 Like
6

I tend to hit them after the second to third bisection step, unless I do semi-intelligent skipping.

7

:heart: I didn’t know there was git bisect start --first-parent. @NobbZ: I assume you did not use that one.

8

I am aware, still hits staging merges quite often :frowning:

But thanks!

9

Well, of course. If the real root cause went to staging, you will obviously have to bisect staging. No way around that.

What the simple first-parent strategy does is entirely avoid commits of the staging branch if the cause went to master which is why I think it’s super useful.
What it also does is tell you which staging cycle the cause is part of; allowing a narrower frame for the costly staging bisect; the second phase.

2 Likes
10

Re-read your comment and there’s an uncertainty: What exactly do you mean by “hit”? Land on it as the cause or a staging commit being a bisection step?

With first-parent, you should never land on a staging commit as a bisection step, only staging-next merges (as in “merge staging-next into master”) as the root cause and that would still be cached.

11

I am talking about staging next, and didn’t yet had good experiences them being cached.

Or at least not what I needed from it.

12

Yes, most of the combinations aren’t built (on staging and staging-next). That’s the main reason why they were created in the first place.