I’m a user of crane and saw that on the 0.15.0 release all public inputs, with the exception of nixpkgs were dropped from the root flake.nix. They were moved to another flake.nix not on the root of the project.
The release notes state:
All dependencies (outside of nixpkgs) have been dropped from the (main)
flake.lock file so they do not pollute downstream projects’ lock files.
It’s an interesting approach: the inputs needed by the application itself and by ci, tests etc. get split, staying the former on the mainflake.nix, while the latter on secondaryflake.nix files.
Any thoughts on this? Advantages, downsides, other ways of achieving the same result.
At that point there’s effectively no difference to just using say niv or npins to manage dependencies. Plus you get programmatically updatable inputs for free!
Okay, I’ll bite, what’s not programmatic about nix flake update [--commit-lock-file] and adjacent subcommands? Or what do you feel is superior about the DX of using those other projects?
Ah I guess I wrote it ambiguously. I meant that you can programmatically add/remove/update the input references. E.g. niv add foo/bar adds a GitHub dependency on github.com/foo/bar, or niv update nixpkgs --branch=nixos-unstable changes the branch.
Flakes can be programmatically modified :). We already support it with fh:
fh add ipetkov/crane
or
fh add github:ipetkov/crane
since the flake.nix’s format is predictable and restricted, it is fairly straightforward to parse the Nix expression sufficiently to do a nice job of this.
We achieve this in GitHub - juspay/services-flake: NixOS-like services for Nix flakes by having sub-flakes pointing to the parent flake, and then in the command line they receive a --override-input services-flake ../. to refer to the current parent flake (overriding what’s in flake.lock). nixci supports encoding this information the top-level flake.itself; see here.
The fh CLI makes it most convenient to work with FlakeHub, a centralised and proprietary service, as such I wouldn’t recommend it. Furthermore it doesn’t support updating existing inputs.
But even if we had an official and totally bug-free CLI for this, this is just poor design. As a general rule, you should never generate Nix code. Nix is not meant to be programmatically written. As a quick showcase, fh --input-name=in generates invalid Nix because in is a reserved identifier and needs to be escaped. There might be more such bugs lurking. This could not happen in niv or npins, because they generate JSON instead.
Furthermore, the only reason fh somewhat works at all is because of how flake.nixisn’t even a proper Nix file, a design decision I really can’t get behind. If/when this restriction gets lifted, fh will break. Alternatively a more proper solution is to have a file like flake.toml and specify your inputs there (you can’t do this currently because of the above issue, import is disallowed in inputs).
And relatedly, tools like niv/npins aren’t built on top of experimental features, so you can expect them to work for a long time
The FlakeHub CLI is one example. Thaw currently only operates on refs, but I’ve been thinking about adding support for managing inputs as a whole. The process would be trivial given the existing implementation: GitHub - snowfallorg/thaw: Semantic Versioning for Nix Flakes
There are certainly drawbacks to flake.nix not being a “real” nix file, but as it turns out there are some benefits. Modifications like Thaw makes are quite easy because of the assumptions made around static values in flake inputs. If these were normal expressions then we would have to evaluate the flake every time and track source locations for values. It would all get very complicated very quickly. Introducing another file to hold this information probably isn’t the right solution here.
I think “Nix is not meant to be programmatically written” is an overly broad statement that stifles potential tools from improving DX. There are certainly times where generating code is useful, where programmatically modifying code is useful, and where analyzing that code is useful.
Hello, I’m the author and I can share my own thoughts/reasoning around this!
Currently, Nix does not support the concept of “development-only” inputs, meaning that before this change, any dependencies used by the test suite end up showing up in all downstream flake.lock files in perpetuity. The test suite also requires a bunch of different inputs between compatibility checks and lots of detailed examples on working with other projects
Some people care about keeping their flake.lock lean/auditable, so cutting down bloat is nice but not a hardcore requirement IMO. Yes, moving test dependencies to their own flake doesn’t remove them (it only “launders” them from showing up in flake.lock) but it still cuts down on churn of downstream lock files (plus the inputs are effectively pinned/reproducible anyway based on the repo contents). This can be especially impactful if a flake transitively depends on different checkouts of crane
Even if you don’t care about bloating flake.lock, having lots of test dependencies still requires nix to hit the network and pull down their sources before any evaluations and lockfile updates, which is kind of wasteful (especially for test code that project isn’t even executing!). Yes, it’s possible to use follows clauses on the inputs to dedup or even omit the entries from the flake.lock, but that now requires downstream projects to a) realize they have to do that, b) repeat that work for independent flakes, and c) understand how crane uses those dependencies and whether it is safe to do this sort of pruning.
So, moving the inputs to a sub-flake allows for solving 1 and 3 nicely (with 2 as an added “bonus”) while still giving me the ability to automate updates with nix flake update and I don’t have to pull in a separate project like niv.
Downsides: a big downside of this approach is that you break overriding inputs (both from the CLI with nix build .#foo --override-input bar ... and from downstream flakes). In this particular scenario I don’t think this is a detriment (only test suite code that no one should be relying on downstream, I can wrangle my own overrides if necessary while developing the project). But I would strongly advise against doing something like this for inputs used by your flake’s non-development definitions!
So the name is still there, but I don’t consider it polluted, or bloat. It doesn’t pull in the input, or it’s inputs. It doesn’t update with nix flake update, and I as a consumer have full control over it. And people using my flake as an input don’t pull in hercules-ci-effects, even though the flake I am using as an input pulls it in.
, though for other readers – a reminder that 
.