Dynamic Derivation Working Group - Meeting Notes #1

tomberek · August 13, 2026, 9:35pm

Dyn Drvs Working Group - Kickoff meeting

Result of Dynamic Derivations + Dynamic build graphs: next steps - #11 by tomberek. If interested; contact @tomberek for meetup invites, Ping @Ericson2314 for org invites, join Matrix: #nix-dyn-drv:ggpeti.com for discussion.

Status

@ericson2314:
- @artemis has been working on rust use case w/crates per derivation
- if using one of the libraries then end user doesn’t need to know
- basic primitives are feature complete
- build integrations
- have had dyn-drvs for a while, but depended on recursive-nix
- @artemis implemented builder-rpc-v0 (unix domain socket) to interact with store and register outputs, allows for references between outputs
- Haskell has Sandstone (need to investigate Darwin failure)
@mschwaig:
- Q: do we have to register the outputs?
- A: @artemis: only have to register final derivation, the tree is resolved by nix-daemon
- Q: can it be used as a caching mechanims, via CA?
- A: Yes if the fine-grained derivation collides. They become valid paths.
@tomberek: anything else needed in Nix itself?
- A: @ericson: allow for building to get started before entire plan is done. Can implement your integration without depending on it, but can get speed up for free. Looking to remove placeholders. More work needed on signature format.
@ericson2314:
- lang2nix tool development can work in parallel
@reiss:
- sandboxing costs 30-40% (few versions ago)
- @ericson: current design doesn’t inherently have the costs, this can be optimized over time. Current approach does O(n) mounting of dependencies. Can consider file-descriptor based approachs.
- @artemis: a firecracker-based VM can do snapshot/restore, or get reused
@artemis:
- Hydra work need.

Brainstorm

Things to work on

@tomberek: working on a gg-style set of shims. Currently working on something to replace things like ccache + distcc.
- GitHub - tomberek/nixgg · GitHub
@bryanhonof: we are currently investigating making Nix Bazel-like
- @tomberek: we have a similar project from pre-dyndrvs (GitHub - anduril/nixel · GitHub)
@bryanhonof: Buck2? They call it something else, but have a similar concept.
- @ericson: not yet explored
@bryanhonoh: intersted in a Linux kernel builder. Custom drivers and patches make rebuilding expensive.
- @tomberek: i’ll try it
- @ericson: this might work with nix-ninja: GitHub - vitkyrka/kninja: Ninja build file generator for the Linux kernel · GitHub Kernel becomes a blob, self-references not an issue
- @jaen: in my experience with OotB Nix Chromium isn’t really practical, so the kernel probably won’t be either (at least a fully granular build); I can throw kninja output at my fork and see what shakes out after I finish finding all the kinks in the Chromium build
@tomberek: Do we try to coordinate the user-facing design?
- @ericson: goal is to get this into Nixpkgs. Hydra work is for that purpose. Can focus on having this be the way various ecosystems.
- @tomberek: feature detect? (bulitins?outputOf) to turn it on/off
- @ericson: @roberth spoke of dynamic-inputs. Instead of needing a trampoline, can hook up the dependencies the other way around. Resolution of dynamic inputs happen before CA and scanning.
cachix is implementing support … in-progress. Needs to learn about realizations. There may be a development server available.
@ericson: @jaen is working on an NL project related, curious about it.
- @jaen: it is, roughly, a webscale nix-ninja — I tried to build AOSP with the upstream version, it didn’t work for various reasons: at first being N2’s Ninja parser was way too slow, then it turned out a lot of things in Nix just don’t work well at scale (e.g. Make the Worker not choke on 70k derivation builds by xokdvium · Pull Request #15611 · NixOS/nix · GitHub which I also came across); right now I’m fishing for optimisation at ~Chromium scale, I hope to make a document describing them soon, and we can triage what to implement and in what order

infra
- Hydra: @artemis: should work, needs further testing. A few PRs are in flight.
- @tomberek: should we setup shared infrastructure?
  - @ericson: having join infra and larger scale demo would help “live it”. Performance issues would be felt and prioritized.
  - @tomberek: I can try to get an Oxide rack. (no ARM, no KVM)
  - @artemis: considering Obsidian after some TBD work
  - @mschwaig: how much? (@tomberek: maybe Hydra + 2 builders)
  - @waern: nixbulid recently added CA-derivation, not yet added support for DynDrvs.
  - @ericson: draft of RFC to stablize the rpc format, derivation format, etc.
  - Currently rpc implements the protocol as a limitted form of the regular protocol, inheriting the daemon protocol’s warts. There is some thought on other formats (Cap’N’Proto,Varlink, etc) might reduce toil for integrations. It is possible that the format can change.
When do we start aggressive outreach?
- nixpkgs fork/overlay
- build farm
- cache: S3 might be best for now
admin
- video: Jitsi Meet
- shared GitHub org with fork/overlay
- Matrix: #nix-dyn-drv:ggpeti.com
  - todo move to nixos.org
documentation/tutorials/testimonials
benchmarks/perf

With Jaen post meeting

derivation formats for perf

Tasks

@tomberek: ask Oxide for a rack
@tomberek: establish meetup cadence
@bryanhonof: can get feedback from internal use
@ericson: Github org
@artemis: Hydra stand-up + infra
- @adeci happy to help here with infra
@bryanhonof: engage on cache
@mschwaig: build-trace implementation
@lyndeno: explore kernel build + writeup
all: send Github handles to @ericson

llakala · August 13, 2026, 10:41pm

It’s mentioned a little here, but any nixpkgs implementation will need feature detection that also works without the latest versions of Nix. Lix has all dynamic-derivations features removed, and compatibility needs to be maintained on that end.

fzakaria · August 13, 2026, 10:50pm

I was just talking to @xokdvium about this. I have other features I would like to adopt that also need feature detection…..

xokdvium · August 13, 2026, 11:24pm

FWIW, I don’t see dyn drvs being remotely ready for prime time in nixpkgs of course. Too many bugs to fix and a very long road to having it be competitive in terms of performance.

Language-side feature detection is one thing, and store layer is another. There isnt really an easy way for libstore-dependent pure feature detection that can be exposed in libexpr. So if anything, this stuff would have to stay opt-in and target out-of-tree use. That’s where the greatest (potential) benefits lie anyway.

llakala · August 13, 2026, 11:29pm

Agreed (at least from the outside). Thanks for keeping expectations tempered.

jaen · August 13, 2026, 11:46pm

The caveats about there still being many bugs to be fixed and improvements to be made is obviously self evident (trying to get Chromium to build with reasonable overhead using dyndrvs had been a challenge), but I’m not so sure about this one.

If we had granular, incremental builds of big and/or fundamental packages like Chromium or compilers or the bootstrap chain, wouldn’t it save a lot of resources in Hydra and time spent on waiting for staging to bake?

roberth · August 14, 2026, 7:10am

The incremental nature of it only works when the input is itself incremental, so its usefulness depends on how fine grained the input is, and how coupled the derivations are to the parts of the source that were modified. Or more simply the size of the reverse closure - closing over dependents.
I imagine it would help tremendously for patches, but not so much for release updates, which will have many changes that easily saturate the reverse closure.

Derivations are a final description of how stuff must be built, take it or leave it. That’s their purpose.
Evaluation is also meant to be pure, behaving like a pure function from sources to derivations, so under pure eval, we can’t have the evaluator decide anything based on properties of the builders.

So the only place where the opt-in or opt-out decision can be made is in the codomain, so in the expression sources.
That’s what we’ve been doing with nixpkgs.config flags like the timidly named config.contentAddressedByDefault.

If a sufficient number of people cares about Nix implementations without dyndrv support, Nixpkgs could use a similar flag to offer both builds, at least in leaf-ish packages, and users of dyndrv get the benefit of being able to patch their software without the wait.
Ideally channel updates can also take advantage of that, but iiuc that requires taking other hurdles anyway before it becomes significant.

SergeK · August 14, 2026, 10:03am

Huh, the jitsi room was empty yesterday . Will DM you for the invite.

I (and some others) feel skeptical that “dynamic derivations”, in the specific technical sense of “derivations producing ATerm (and one node, not a subgraph!) + speaking to the recursive-nix socket (renaming it to builder-rpc-v0 doesn’t substantially change the fact that it’s (filtered) recursive-nix)” ever go through in Nixpkgs, it just doesn’t feel realistic. The only reason we seem to be talking about dyndrvs is because our evaluator’s state is (currently) not representable, i.e. cannot be named, memoized, or distributed across multiple machines. If Nix-lang were more like Unison or Lisp (and we don’t know if it’s particularly unlike Lisp, it’s just too weird and ad hoc to easily judge, and idk if anyone has yet made a formal effort), we’d obviously just stick to the much-cleaner IFD model.

jaen · August 14, 2026, 4:22pm

@roberth, here’s an example of building llc from LLVM across a few version bumps (at a derivation-per-sourcefile granularity):

┌───────────────────────────┬────────┬─────────┬─────────────┬────────────────┐
│           Rung            │ ninja  │ Shinobi │ ninja edges │ Shinobi chunks │
├───────────────────────────┼────────┼─────────┼─────────────┼────────────────┤
│ 21.1.1 cold               │ 207.0s │ 229.0s  │ 1776        │ 1773           │
├───────────────────────────┼────────┼─────────┼─────────────┼────────────────┤
│ → 21.1.2 (adjacent patch) │ 91.0s  │ 88.1s   │ 558         │ 537            │
├───────────────────────────┼────────┼─────────┼─────────────┼────────────────┤
│ → 21.1.8 (6 patches)      │ 84.5s  │ 89.7s   │ 567         │ 549            │
├───────────────────────────┼────────┼─────────┼─────────────┼────────────────┤
│ → 21.1.8 (from 21.1.7)    │ 9.8s   │ 15.0s   │ 102         │ 93             │
├───────────────────────────┼────────┼─────────┼─────────────┼────────────────┤
│ → 22.1.1 (major)          │ 200.4s │ 211.0s  │ 1809        │ 1807           │
└───────────────────────────┴────────┴─────────┴─────────────┴────────────────┘

You’re right that on a major bump it didn’t really help, but it was like 2.5x faster on the two patch bumps and one was 15x faster (apparently it didn’t touch any headers, so cut-off was quick. Now, there is still some overhead I have not pinnned down or that got lost in the forest of envvar variations, but if it can make a package bump evaluate ~2x faster on average, I think it would be a big to resource utilisation efficiency in Hydra?

Maybe you were also misled by:

because that was supposed to be 5:00 PM, actually. I assumed this one was correct and " CET (Central European Time): 7:00 PM Thursdays" was an hour off (we’re +2 right now). Timezones are hard xD

Eveeifyeve · August 16, 2026, 9:40am

Where is the matrix room, I can’t join?