Is there a fundamental difference between OfBorg and Hydra? I understand that OfBorg builds PRs before they get merged, while Hydra checks out the master branch regularly and build all the packages. But why do we need to build all the packages if each of them have been already built by OfBorg already?
3 Likes
OfBorg doesn’t build PRs. It runs some checks on them.
They only get built by OfBorg when a “trusted person” opened the PR or asks OfBorg to build them.
Hydra on the other hand is the instance that feeds cache.nixos.org and verifies that the commits build successfully and tests run successfully after merge, which is an important difference.
Let’s assume that there are two PRs, each updating only their own package respectively A and B. B needs A as a dependency.
B builds with the old version of A, but not with the new version.
Both PRs build and test successfully on their own, but the combination breaks.
OfBorg will probably not catch it, as it depends on the exact timing of the PRs and when they are merged.
Hydra will detect it and - assuming that software B is considered critical enough - will not advance the channel.
2 Likes
Another key difference is if you send a PR updating GNU Hello, ofborg will typically only build GNU hello. Hydra will build GNU Hello and the remaining 49,999 packages in Nixpkgs.
2 Likes
tests run successfully after merge, which is an important difference
Well, OfBorg could probably build and run tests after merge, like bors for example.
1 Like
Another key difference is if you send a PR updating GNU Hello, ofborg will typically only build GNU hello. Hydra will build GNU Hello and the remaining 49,999 packages in Nixpkgs.
But why building the remaining packages if they’ve been built by OfBorg already? Each of these remaining packages must have been built by OfBorg at some point (for example the last time they were updated). To me it make sense to rebuild all the packages that depends on the package that is updated in the PR, but I don’t understand why one need to rebuild all the packages.
Not really. I believe that most of rebuilding work on Hydra is due to transitive dependencies, i.e. something deeper among dependencies gets changed and that’s why the package is rebuilt. We now manage multiple mass-rebuild merges to master per month.
BTW, both Hydra and OfBorg read from the binary cache, so they don’t really rebuild stuff that’s there already (and all Hydra builds go immediately into the binary cache).
Does OfBorg also push to the binary cache? I think that’s what @smaret is asking. Because then Hydra doesn’t have to rebuild those closures that OfBorg has already built.
From the reactions I thought it was clear that it does not.
Yes it was kind of clear but I wanted to explicitly ask again.
Then the question is: Is there a reason why OfBorg shouldn’t also push to the binary cache?
1 Like
That’s how the whole service was built. Graham will know better details than me. Now it’s mostly an unprivileged service – basically anyone is allowed to run the build machines IIRC, so currently we can’t really trust the binaries. Also completely anyone can create a pull request and put malicious code into the expression, and the build machines would execute that automatically – yes, typically inside a sandbox, but I don’t think the namespaces are always foolproof against malicious code, and there are the fixed-output derivations that require at least internet access.