What is the recommended way to sync dependencies in lock files across multiple flakes-based projects? I have a number of interrelated development projects, mostly using Haskell and Rust, which use libraries like haskell.nix and fenix, and without an effort to sync versions across projects I spend an eternity rebuilding different versions of libraries.
Currently I’ve settled on using a “proxy flake” repo on github that contains just a flake.nix and flake.lock with some common dependencies as inputs (nixpkgs, haskell.nix, fenix, etc). My other projects depend on this flake and use inputs.xxx.follows. It works, but feels ugly.
I’ve considered using nix registry. The problem is that afaik registry entries are machine local so this doesn’t work well when developing across multiple workstations.
nix develop has an --inputs-from option that I thought would help, but this seems to only work with inputs from the registry.
I don’t have a direct answer, but an idea that has come up before does use the registry, and goes like this:
My nixos system flake has inputs and a lock file. When the system is built, entries are added to the system registry of the new closure, pinning those inputs. I do this today, mostly to avoid ad-hoc nix shell and similar commands checking and re-downloading the sources.
What I would like is a version of nix flake update in a project workspace that takes the same revisions from the registry, even if they’re slightly out of date. This means that a number of projects will get updated all to the same versions of build tools, and those will already be on my system together with all their dependencies. If there happens to be a giant rebuild after a staging cycle lands, I won’t accidentally wind up with surprise downloads and different versions in the middle of the afternoon (unless I want to).
I don’t want something that builds an implicit / impure dependency on the registry, just regular locking but with a different selection mechanism at the time of update.
Keeping registries in sync across machines is then also as easy as keeping them up to date with the same system flake.
This doesn’t work so well for other inputs that aren’t part of the system flake, however.
At one point I was brainstorming and sketching out a CLI that I would’ve pitched as “cross-flake follows support”, but I ultimately found a solution that only has a dependency on jq and a predictable public flake URL. My own motivating use-case is and was to keep my GC roots and store usage somewhat under control when many disparate projects have their own nixpkgs inputs.
Right now it has stable, unstable and darwininputs.
As an aside, I genuinely wanted to be believe that --inputs-from was somehow related to this conceptual goal, but I never found a working invocation that suited my particular goals.
For a mildly more complicated use-case, this was pulled out of my dotfiles but based on the same source-of-truth, using Nushell which has slick structured-data support built-in:
#!/usr/bin/env nu
use std "log debug"
use std "log info"
def main [target: string = "github:shanesveller/nix-flake-lock-targets"] {
log info $"Fetching lock contents from ($target)"
let metadata = (nix flake metadata --json $target | from json)
let darwin = ($metadata.locks.nodes.darwin.locked.rev)
let stable = ($metadata.locks.nodes.stable.locked.rev)
let unstable = ($metadata.locks.nodes.unstable.locked.rev)
log debug $"Darwin: ($darwin)"
log debug $"Stable: ($stable)"
log debug $"Unstable: ($unstable)"
log info "Updating local lock file"
(nix flake update
--commit-lock-file
--override-input nixpkgs-darwin
$"github:nixos/nixpkgs/($darwin)"
--override-input nixpkgs
$"github:nixos/nixpkgs/($stable)"
--override-input unstable
$"github:nixos/nixpkgs/($unstable)")
}
(First-ever Nushell script for me, so if you have pointers please feel free to DM or reply about it.)
nix flake metadata nixpkgs prints the same data from the registry, and of course nu makes it easy to use with --json. Look at the inputs of the local flake, loop over them in registry, and call update as you do here.
I wasn’t particularly thinking of an extraneous script, and agree it feels like there’s a missing but of functionality where the concepts and use-case hadn’t quite yet gelled in the nix experimental cli, but yes this could work.
Sourcing information from the registry is not as fixed in time, IIUC, unless you’re using something to pin that on your machine. I do so here and I never intentionally use it in my workflows, so that’s just there for a safety net. If you do not do that kind of pinning, nix flake metadata <anything from the registry> would be unlikely to produce identical results when invoked from more than one machine spaced apart by hours or days, due to the TTL mechanics, right?
I came up with a “trick” a while back to obliviate the need to obsessively use follows in several work flakes where I wanted all of them to track the same versions of various flakes. follows was also insufficient in the first place because it doesn’t work more than 2 levels deep for some reason, which wasn’t good enough for my use-case.
So anyway, what you can do is either fork, or maintain your own registry and the set nixConfig.flake-registry = "https://some.remote/flake-registry.json"; in your consuming flakes to automatically pick it up.
You can use the same specificity in a registry entry that you would for a normal flake ref, so you can pin to commits or tags or whatever you want. Then in the consuming flakes you simply reference inputs by their indirect registry entries instead of locking them to a specific url.
From there, you manage all updates by bumping the entry in your registry json. You could use CI automation in the consumers to periodically check for registry updates and re-lock their flakes on a regular basis, say once a day or once a week. It requires a bit of descipline, but it works well enough without having to get obsessive with the follows or other override madness.
Only drawback is that you have no control over unowned flakes if they choose not to use indirect inputs.
Yeah, I’m setting the system registry entries to the revisions used at evaluation time, (see the linked other post). So they won’t change (or just get re-downloaded hourly because of the TTL) between workspaces on a given host, and are usually the same even between hosts (because they typically all get updated to the current version of my systems flake).
But of course what actually matters is the version locked in the workspace, this desired feature is just a convenience to minimise the variation between workspaces. If I or someone else checks out the repo on a machine running a release branch, or even gasp some other distro, of course they should have the same environment.
Sometimes I go update the locks in various workspaces, not because I’m actually doing anything particular in that workspace, but because the versions pinned there are old and I’d like to let them garbage-collect. A few times, I’ve just removed the profile links / .direnv cache, and of course a week or a month later the old versions just get re-downloaded next time I actually pick up the project, just in time to then get upgraded. So instead I want to update them to just be “the same”.