Workflow for developing multiple related flakes

HI all,

I have a software development project that consists of several sub-projects that depend on each other. I’d like to build them as a set of separate nix flakes that depend on each other.

But how would I go about hacking on all of them at the same time?

Assume I have three flakes, A, B and C, B depends on A, C depends on B.

I’d want to check out all three git repositories on my PC, and have my local nix build or nix shell in flake C use my local versions of A and B.
I’d want to make a change to A, go to directory C and run nix build to rebuild all three. I do not really want to have to commit my changes to A first.

When I’m done, I want to commit my changes and push them to my github, and other people could just depend on my github:autc04/C flake, nix flake update and get the latest changes.

How do I achieve that, or something like it?

I didn’t manage to get anything working with --override-input; do I really have to edit my flake.nix files in order to hack on them locally, and then remember to change them back before I commit? Can the flake registry help in some way?

Are you able to depend on flakes by their paths?

As long as the repos are only on my hard disk, I could hard-code the flakes to use path: for dependencies. But I would not want to put paths to my local hard disk into the public github repositories. I’d expect the flake.nix files there to refer to each other by github paths, right?

So do I have to keep changing my flake.nix files back and forth between B.url = "path:../B" and B.url = "github:autc04/B";?

So do I have to keep changing my flake.nix files back and forth
between B.url = "path:../B" and B.url = "github:autc04/B";?

Are you unable to just use the former? Looks like inputs.foo.url = "path:../bar" works on my version.

I don’t think it can. To use a flake from the registry, you need an attribute in inputs like

inputs.nixpkgs = {
  type = "indirect";
  id = "nixpkgs";
};

or don’t define it at all. Both of these options modify the inputs attribute set, which you are trying to avoid.


--override-input seems to be working, at least with two projects.

I have a setup similar to yours but with only two projects. I was changing the url attributes to path://<path> while working on them. If I keep the original urls and do nix flake info --override-input <input-name> "path://<input-path>" the input seems to be overridden as expected. You should be able to use any flake sub command instead of info.

Doesn’t that stop working as soon as I push it to github?

Indeed.
It breaks down for the indirect dependencies :frowning:.
I can’t get it to override an indirect dependency.

In addition to --override-input, nix develop --redirect can be useful for development (though it might not be as well documented as what it would need and it’s a bit tricky to use well)

That’s worth opening an issue imho. I don’t think it has been intended to work with indirect dependencies, but that’s a natural thing to do so it would be a sensible addition

Doesn’t that stop working as soon as I push it to github?

If the two flakes are in different repos, it probably would. This sounds
like a good candidate for a mono-ish repo, though, if the two flakes are
inseperable in their dev environments.

I just figured that part out - I needed to use --override-input B/A blah to refer to input A of input B, in addition to an --override A blah to override the direct dependency. The documentation just needs some reading between the lines here, but clearly someone already thought of it.
So I just need to dump a complete list of --override-input options into an environment variable and make sure to use it on every flake-related command.

I’ve also found the following in the output of nix registry --help:

In addition, the flake registry allows you to redirect arbitrary flake references (e.g. github:NixOS/patchelf) to another location, such as a local fork.

… but I could not get that to work for nix flakes referred to by URLs, just for symbolic identifiers like nixpkgs.

Can you share your complete solution?

It’s not yet ‘complete’ in the sense that I’ve only tried it outwith trivial test flakes, and I haven’t actually set up anything that actually works yet.

I will outline my current plan below; I think it’s ‘good enough’ to address my use case, but not really ‘good’, so it could be better.

The flakes A, B & C live on github, and their dependencies to each other are set up accordingly.

B/flake.nix: { inputs.A.url = "github:autc04/A"; ... }
C/flake.nix: { inputs.A.url = "github:autc04/A"; inputs.B.url = "github:autc04/B"; ... }

To hack on all of them simultaneously, I’d clone all of them locally, and then set up:

export B_DEPS="--override-input A path:../A"
export C_DEPS="--override-input A path:../A --override-input B path:../B --override-input B/A path:../A"

and whenever I do something in directory B, I’d need to pass $B_DEPS to all nix commands, and whenever I do something in directory C, I’d need to pass $C_DEPS.

As a next step, I could probably use direnv to automate that to a certain degree (I’d still need to write the separate flag lists for B and C by hand), but I have yet to figure out the details.

The docs make me feel that there might be a more convenient solution using the flake registry, but I have yet to figure out how that works.

1 Like

Are you unable to just use the former? Looks like inputs.foo.url = "path:../bar" works on my version.

AFAIK, relative path support has been effectively removed last September: Fix relative path input handling by edolstra · Pull Request #5279 · NixOS/nix · GitHub
There is a PR trying to support relative paths in the simple case where the root flake is the parent flake: Fix flake relative paths by roberth · Pull Request #5437 · NixOS/nix · GitHub

2 Likes

AFAIK, relative path support has been effectively removed last
September

Ah, good to know!