Automatic Nix flake follows

Surprisingly simple to get it working and kind of neat.

9 Likes

This should be a setting in Nix. Running a python script on a lock file sounds like a tedious task since lock file gets rewritten after each flake update.

7 Likes

For now this is a good way to experiment whether it should be included but I agree.

I’m writing a script to also visualize your follows tree which is helpful. I’ll write about that shortly.

4 Likes

I disagree.
How would nix know what are similar-enough inputs and what aren’t?
And sometimes you explicitly don’t want to use follows due to breakage.

What might be better is allowing thunks in inputs (which nix would never permit), which would then allow using functions to generate that attrset.

I say most use of my own for follows falls into two camps:

  • I either care about being “correct” and nothing is followed
  • I don’t care so follow everything.

Rarely am I in-between.

I can give a concrete example: I used follows on everything except nixfmt (back when it was in an old repo using an older nixpkgs rev). Using follows would break it due to a different version of haskell, deps, etc. There are other projects that explicitly say not to use follows for similar reasons. Even using follows on HM leads to breakage since they’re downstream of the frequent changes in nixpkgs.

One person’s usecase does not inherently generalise into a good design. Thankfully nix isn’t a design-by-100000-user-committee type of project.

1 Like

I believe there are cases that you don’t want to follow; but overall it’s a pity that we even have to follow but it’s become a predominant pattern to mimic.

The tool could be smarter and you could annotate which ones to leave perhaps;
I just added a “check” tool to validate what you might be missing.

I just noticed my own NixOS config was bringing two home-managers which I didn’t intend.

2 Likes

cool stuff :+1:

didn’t @lheckemann talk a bit about this idea at NixCon last year? how someone could write other tools to do flake.lock management and come up with some interesting schemes? awesome to see someone acting on that!

1 Like

I am having a hard time groking what this does?

This has the same result as writing the follows, but doesn’t require me to write it?

Does it not get undone when you next run nix flake lock? I guess if nothing else it seems like it can be used as an optimization tool, I can use the validation mode for hints at further overrides I may or may not want to add.

Definitely an optimization tool.
If the lock file gets overwritten just run it again!

I am really enjoying the check command I added also.
I’ve found a couple of deeply nested follows in my own projects I didn’t fix.

Could this be slicker if it’s part of Nix ? Sure, but it’s also cool we can extend and do stuff outside of Nix :slight_smile:

Try it on your projects. Worst case you revert the changes.

3 Likes

@fzakaria cool stuff, check out GitHub - nix-community/nix-melt: A ranger-like flake.lock viewer [maintainer=@figsoda] as well.

1 Like

How would nix know what are similar-enough inputs and what aren’t?
And sometimes you explicitly don’t want to use follows due to breakage.

That’s why I suggest a setting instead of changing the default behavior. To me, any input pointing to the same repo is similar enough to attempt deduplication. The setting could be set at flake level and then overridden on input level.

Also, maybe nixd could suggest follows-optimizations. Nix syntax is far from ergonomic anyway, so as long as I don’t have to write boilerplate by hand, it would be an improvement.

3 Likes

/cc @inclyc @Aleksanaa

That’s right.

In fact the tool only dedupes inputs that are specified on the top flake. This is reasonable because the user already picked a main one they’d likely do follows on.

It will check and warn about duplicates below as well but if not specified at top level it won’t do any changes and only print out what the choices are. I leave it up to the user to specify which to consolidate to.

1 Like

Could you think of an exact feature request anyway? I don’t think it’s appropriate to guide users to use follows or avoid it. We can only add some completion when they have typed inputs.<>.follows = "<>"

I would really hope for a native implementation such as:

# flake.nix
# a _hypothetical_ native implementation for auto follows

inputs = { ... };
nixConfig.flake-auto-follows = true;

# ^ defaults to "false" for backward compatibility and correctness
# users may opt in for auto-follows
1 Like

An IDE/language server could warn/hint with a squiggly line about duplicate sources in flake imports and suggest a quick fix that would automatically add follows to the offending import. Duplicate is a transitive implicit dependency (input’s input) of one of the explicit imports that uses that same repo (but different branch/hash) as one of the explicitly declared inputs. A more complex use case would be to find a duplicate among implicit dependencies and suggest to declare it as a common explicit import and set it in follows.

1 Like

After reading some comments above, I’d like to pointing out that, follows is based on assumption that flake dependency in different versions won’t have effective changes and API compatible. But this is not true. Package versions, modules, they get changed among nixpkgs revs. Thus flake users do follow for all inputs, likely they won’t reproduce the result in upstream.

I agree but if you want to get extra pedantic; nixpkgs itself doesn’t build with the library versions a package author might have specified for a given version since everything must resolve to a few versions or a single one – so software isn’t as the author intended to begin with :stuck_out_tongue_winking_eye: