Flake lockfile update loop when having dependent flakes in a monorepo

Development
1

This is the minimum example, I have two flakes a and b in a hypothetical git monorepo:

$ tree 
.
├── a.nix
│   ├── flake.lock
│   └── flake.nix
└── b.nix
    └── flake.nix

2 directories, 3 files

where

a.nix/flake.nix:

{
  inputs = {
    flake-utils.url = "github:numtide/flake-utils";
    nixpkgs.url = "/var/lib/nixpkgs.git";
    b = {
      url = "../b.nix";
      inputs.flake-utils.follows = "flake-utils";
      inputs.nixpkgs.follows = "nixpkgs";
    };
  };

  outputs = { flake-utils, nixpkgs, b, ... } :
    flake-utils.lib.eachDefaultSystem (system:
      let pkgs = import nixpkgs { inherit system; };
      in  {
        apps.default = {
          type = "app";
          program = nixpkgs.lib.getExe pkgs.${b.foo.${system}};
        };
      });
}

b.nix/flake.nix:

{
  outputs = { flake-utils, ... } :
    flake-utils.lib.eachDefaultSystem (system: {
      foo = "hello";
    });
}

But each time I run a.nix, it will trigger its input updates:

$ nix run a.nix
warning: input 'b' has an override for a non-existent input 'nixpkgs'
warning: updating lock file '/home/hellwolf/tmp/flakes-monorepo/a.nix/flake.lock':
• Updated input 'b':
    'path:/nix/store/7yx2x5jy2wqll0bf39153yk6d1fp9l6g-source/b.nix?lastModified=1&narHash=sha256-ZtbtLFgH1GZRrsgytRuq%2fp9YJcBi5aecXUDcAgZsLKA=' (1970-01-01)
  → 'path:/nix/store/cjc32bhj2r9c2y9mxbp9133garnfch08-source/b.nix?lastModified=1&narHash=sha256-ZtbtLFgH1GZRrsgytRuq%2fp9YJcBi5aecXUDcAgZsLKA=' (1970-01-01)
warning: Git tree '/home/hellwolf/tmp/flakes-monorepo' is dirty
Hello, world!

Then I commit the a.nix/flake.lock update, and run again, it gets updated again. It never gets settled to a stable lockfile in such setup.

It seems not ideal, any thoughts?

2

How about with path:

  b.url = "path:../b.nix";

In my best practice, I prefer to call a subflake through flakeCompat rather than path:./b;

1 Like
3

Using “path:” indeed changes the behavior to what I needed! But what’s the differences?

Thanks for the tips of using flakeCompat, I will need to take a deeper look to understand that.

4

Thanks for the tips of using flakeCompat, I will need to take a deeper look to understand that.

IMO, You don’t need to put all inputs into the main flake, especially for the trivial or dev inputs, such as pre-commit, devshell, etc. It’s an efficient way to avoid the flake inputs getting bloated and prevent an unbounded stretch of flake inputs. The downstream users will introduce a light flake, then load the flake’s functions lazily.(just a personal idea)

Yeh, I’m pretty sure the path would be the best solution for your case. Improve Flake Performance in absurdly bloated Monorepo.

1 Like
5

I cribbed from flakes-compat that I could use but then I have to explain to my coworkers that run some bespoke Nix framework and you can’t use X, Y, Z.

So, it has a risk for the flake-comapt? I’m using an optioned way and modified flake-compat to manage a monorepo. Hence, I probably wouldn’t recommend that for my use case without being sure of some real environment.

6

Hey I was the author of the linked thread. I wound up creating this incredibly useful template that I drop alongside every flake now. Strongly recommend it.

1 Like
7

While using a path: URL does fix the always-changing lock file issue, I now have a different problem: whenever I change something in the sub-flake these changes are not immediately “propagated” to the parent-flake. Instead I have to recreate the lock file (or, more specifically, update the lock file entry for the sub flake). While I suppose this happens because the sub-flake is getting locked properly in the lock file, I do not think that this is desired behavior most of the time. Is there any way to make this automatic?

Edit: I guess I am running into this: More intelligent subflake locking. · Issue #6352 · NixOS/nix · GitHub

8

I recommend using the call-flake for your case.

example: Add reproducible dev environment by blaggacao · Pull Request #107 · reinterpretcat/vrp · GitHub
or having overridable inputs with flake-compat and pop: https://github.com/GTrunSec/hive/blob/472b6ae3cda4cc9c0d7a5806a8789f2d48f14b80/cells/common/lib.nix#L15

9

Thanks, but after a quick test this does not seem to help in my case. My use case is this: I currently have one flake that provides nixosConfigurations for multiple hosts. I would like to split this into one sub-flake per host, primarily to isolate their inputs, e.g. the used nixpkgs. These nixosConfigurations from the sub-flakes should be “re-exported” in the parent flake. For ease of updating I would still like the inputs of the sub-flakes to be locked in the parent flake lock file, i.e. what’s happening with a path: URL. call-flake (and also get-flake) would require lock files for the sub-flakes, AFAICT.