Dynamic Derivation Working Group - Meeting Notes #1

@roberth, here’s an example of building llc from LLVM across a few version bumps (at a derivation-per-sourcefile granularity):

┌───────────────────────────┬────────┬─────────┬─────────────┬────────────────┐
│           Rung            │ ninja  │ Shinobi │ ninja edges │ Shinobi chunks │
├───────────────────────────┼────────┼─────────┼─────────────┼────────────────┤
│ 21.1.1 cold               │ 207.0s │ 229.0s  │ 1776        │ 1773           │
├───────────────────────────┼────────┼─────────┼─────────────┼────────────────┤
│ → 21.1.2 (adjacent patch) │ 91.0s  │ 88.1s   │ 558         │ 537            │
├───────────────────────────┼────────┼─────────┼─────────────┼────────────────┤
│ → 21.1.8 (6 patches)      │ 84.5s  │ 89.7s   │ 567         │ 549            │
├───────────────────────────┼────────┼─────────┼─────────────┼────────────────┤
│ → 21.1.8 (from 21.1.7)    │ 9.8s   │ 15.0s   │ 102         │ 93             │
├───────────────────────────┼────────┼─────────┼─────────────┼────────────────┤
│ → 22.1.1 (major)          │ 200.4s │ 211.0s  │ 1809        │ 1807           │
└───────────────────────────┴────────┴─────────┴─────────────┴────────────────┘

You’re right that on a major bump it didn’t really help, but it was like 2.5x faster on the two patch bumps and one was 15x faster (apparently it didn’t touch any headers, so cut-off was quick. Now, there is still some overhead I have not pinnned down or that got lost in the forest of envvar variations, but if it can make a package bump evaluate ~2x faster on average, I think it would be a big to resource utilisation efficiency in Hydra?

Maybe you were also misled by:

because that was supposed to be 5:00 PM, actually. I assumed this one was correct and " CET (Central European Time): 7:00 PM Thursdays" was an hour off (we’re +2 right now). Timezones are hard xD

2 Likes