Improve Flake Performance in absurdly bloated Monorepo

I’ll preface this all by saying I don’t like monorepos, and I’ve been down the road of trying to convince our org to use submodules; for the purpose of this thread let’s just focus on “best practices and tricks to get flakes optimized in a monorepo”.

For context the tree is ~2Gb with nearly a thousand branches, and master sees about 50-100 commits/day and about 500 CI runs for PR checks.

I’ve isolated flakes for clusters of projects with much smaller sizes. But as you probably know, Nix still processes the top level git dir to do just about anything. For local dev in a dirty it’s painful. Any eval, even without builds ( nix flake show for example ) takes over a minute. I survive by using a REPL, nix eval, and nix build -f but from a UX perspective having nix build and nix flake check run at a reasonable speed is important for adoption.

In CI I’ve used locking and building from URIs with no checkout to help speed things up; but there’s no such luxury for local dev.

A goofy trick I’ve used in wrapper script is maintaining a shadow tree that will use git diff to override unchanged subtrees with the base; but this is a hideous workaround and the minor improvement isn’t worth the trouble.

What tricks have y’all picked up; and just for posterity : what’s your usual flake usage look like is subdirectories?

As an example let’s say as a local checkout with staged files wants to run consecutive evaluations. Maybe build 50 targets and tests in a loop. Currently I use linkFarmFromDrvs as a way to skip multiple executions, but I hit max open file on fresh VMs, so being able to do multiple runs without the lag would be really nice. How can I get Nix to stop attempting to copy or checksum between these runs?


Not really a solution for today, but it looks like it’s something being worked on in Nix:


Oh yeah. I’ve had this on my notifications for months now. Just waitin for the day to come! :sweat_smile:

But like til then, what? Should I just drop a shit load of individual default.nix files and do the giant web of overlays like the olden days? Flakes have me spoiled now, I don’t wanna go back to a tangled mess with no eval cache. I’ve got a modified callFlakeWith that I cribbed from flakes-compat that I could use but then I have to explain to my coworkers that run some bespoke Nix framework and you can’t use X, Y, Z.

1 Like

If you can live with some impurities and missing source info, using path: on the “subflake” would only pull that into the store.

I used a similar trick at my previous employer to get flake and lockfile in a nix hostile work environment.


So in subdir with a flake, do you mean nix build path://$PWD instead of nix build which I’m guessing is referring to the top level git dir?

I’m not at my box but I’ll give that a try. If it works I’m going to update the Nix build command docs because they ought to recommend it more prominently.

Edit : That fully works. Y’all run with nix CMD path://$PWD it was a million times faster. ~90 second eval ran in 8 seconds. I’m sure the eval cache is angry about it but copying the whole tree is definitely slower than any amount of eval cache savings could hope to win.


I wound up creating this super useful template that I drop into every directory with a flake.nix hopefully you find it useful : rime/inputs.nix at 5fd030440db2d2dceaf406a592d571c3a4c5ff38 · aakropotkin/rime · GitHub