Just to be sure I’m understanding this correctly, and because I was somewhat confused by @roberth’s comment here…
Currently I have a shared flake between several systems. It works great for nixos-rebuild, darwin-rebuild, and nixops deploy, across x86_64-linux (local and cloud), aarch64-linux (an RPi3), and aarch64-darwin (my primary day-to-day device). Huzzah!
On the RPi, I decided to turn its previously (Raspbian, Ubuntu) working scanner functionailty into a flake.
I wrote a flake using a local path: inputs.scanner.url = "/path/to/repo";. I don’t refer to that input or flake anywhere in the other systems, and so I expected nix’s laziness to allow this to work (because this input is never used on those systems, it would never be evaluated).
Instead, none of the other systems will build. I then tried to remove the input and use getFlake, but then things are impure.
For the time being I just pushed the scanner flake to a private git repo.
I think I’m waiting for this to land before this will work as expected:
Values in an attribute set are computed lazily, so as long as you don’t use the value of an input, the input does not need to be fetched during evaluation.
If your flake isn’t locked or if the lock file needs to be updated because you’ve changed the inputs, you’re doing more than evaluation though, so maybe that could explain what you’re seeing. Otherwise, --show-trace might give a clue as to why your expression does load the input. Maybe you’re using a framework that scans the inputs for some attribute. That’s one way to kill the lazy fetching.
Lazy trees / Source tree abstraction does not change this behavior. It only makes the unpacking lazy. Accessing any single file inside an input (think flake.nix) will be enough to cause the whole input to be downloaded, but not necessarily written to the store.
For my test case, I’m using a locked flake which is up to date.
$ git status
On branch master
Your branch is up to date with 'origin/master'.
nothing to commit, working tree clean
Adding this input, with no other changes (and therefore not being utilized anywhere):
notexist.url = "/does-not-exist";
And then:
$ sudo nixos-rebuild --flake . switch
warning: Git tree '/home/n8henrie/git/nixos' is dirty
error: getting status of '/does-not-exist': No such file or directory
So if I understand you correctly, the issue is that flake.lock is now out of date, and it’s trying to automatically update the lockfile, and this step is causing the error (not the flake itself). Is that right?
I’m wondering if a POC script might make it easier for you to point out what I’m misunderstanding. (EDIT: If you have time of course, I’m grateful for the input you’ve already had here.)
This little script creates two flakes in /tmp, both using the trivial template (but x86_64-linux → aarch64-linux). For the second, it adds an input named unused pointed at the first one, which is thereafter only referenced in outputs = { ..., unused }, because otherwise it complains about being passed an unexpected argument.
It then builds flake2, showing that everything works, and then moves the path to which flake1 refers and rebuilds, and fails with
+ nix build │
error: getting status of '/tmp/flake1': No such file or directory
This is what I mean by the path not being used in the flake outputs at all and yet being required for the flake to build. I thought that lazy evaluation would mean that it wouldn’t matter that unused.url didn’t exist – because unused isn’t… used?
#!/usr/bin/env bash
set -Eeuf -o pipefail
set -x
readonly FLAKE2='
{
description = "A very basic flake";
inputs.unused.url = "/tmp/flake1";
outputs = { self, nixpkgs, unused }: {
packages.aarch64-linux.hello = nixpkgs.legacyPackages.aarch64-linux.hello;
packages.aarch64-linux.default = self.packages.aarch64-linux.hello;
};
}
'
main() {
rm -rf /tmp/flake{1,2,1.bak}
mkdir -p /tmp/flake{1,2}
cd /tmp/flake1
nix flake init
sed -i 's/x86_64/aarch64/g' ./flake.nix
cd /tmp/flake2
echo "$FLAKE2" >flake.nix
nix build
mv /tmp/flake1{,.bak}
nix build
}
main "$@"
Right, the locking part is not lazy and locking needs to run if you don’t have a lock file yet.
If you create the /tmp/flake1 path, then lock the flake, and then remove the path, consumers of your flake should not be able to tell that something is wrong unless they look inside the inputs attribute.
I added another nix build step in /tmp/flake1 (right after sed), and I see in the output that the lockfile is created, but the outcome is the same. Removing the set -x:
$ ./empty_input.sh
wrote: /tmp/flake1/flake.nix
warning: creating lock file '/tmp/flake1/flake.lock'
warning: creating lock file '/tmp/flake2/flake.lock'
error: getting status of '/tmp/flake1': No such file or directory
Current script:
#!/usr/bin/env sh
set -Eeuf -o pipefail
readonly FLAKE2='
{
description = "A very basic flake";
inputs.unused.url = "/tmp/flake1";
outputs = { self, nixpkgs, unused }: {
packages.aarch64-linux.hello = nixpkgs.legacyPackages.aarch64-linux.hello;
packages.aarch64-linux.default = self.packages.aarch64-linux.hello;
};
}
'
main() {
rm -rf /tmp/flake{1,2,1.bak}
mkdir -p /tmp/flake{1,2}
cd /tmp/flake1
nix flake init
sed -i 's/x86_64/aarch64/g' ./flake.nix
nix build
cd /tmp/flake2
echo "$FLAKE2" >flake.nix
nix build
mv /tmp/flake1{,.bak}
nix build
}
main "$@"
Also tried scattering some nix flake locks in there, tried --no-update-lock-file for the last nix build, but the behavior is the same.
Maybe I’m wrong and maybe flake-compat is the best implementation of flakes, or maybe local paths are treated specially. If this doesn’t work for git-based flakes either, I’d consider that a bug.
Wow, that helps! Using inputs.fake.url = "/tmp/testflake";, where /tmp/testflake is a git repo with a committed flake.lock, if I add that path to inputs and nix build, then move / remove /tmp/testflake, then nix build again, it fails (even though I’ve never used that input anywhere else in the flake).
However, if I change nothing about /tmp/testflake but change it to inputs.fake.url = "git+file:/tmp/testflake", I can nix build, then remove that file, and it still builds.
Not only that, but as long as I build and commit the parent flake.lock, I can successfully build on a different system (defined in the same parent flake) that does not have /tmp/testflake available.
So I think this solves the issue – I just need to use git+file:/tmp/testflake instead of /tmp/testflake.
I recently revisited this after adding a few additional local flake inputs.
I was surprised to discover that using any other prefix (such as path:, using the bare /path/to/input, git:), several of which I thought would be evaluated identically based on the flake docs, failed to work – other machines complained with errors about ssh, about git, even though the input was never used on those machines.
Using git+file: was the only way I could get it to lazily ignore those inputs when not needed.
If this sounds like the intended behavior, maybe I could contribute a documentation PR somewhere?
git+file isn’t that different from git+https on the Nix side of things, so I’d be surprised.
fetchTree, which is responsible for loading already locked flakes, already works quite well and lazily, so if you’re observing eager behavior it’s probably
locking, which is quite necessary if you’re making additions or changes in the inputs attribute.
expressions that really need those dependencies
flakes that scan all their inputs, e.g. filterAttrs someFunction inputs
In my case there were 3 different local inputs to consider (all directories local to a machine, with these inputs unused on other machines’ configurations):
1 local flake inputs from machine A, only used in its own nixosConfiguration
2 local flake inputs from machine B, only used in its configuration
0 local flakes inputs on machine C, which shares a flake but has its own configuration that does not use any of the above local inputs
My process was the same for all test cases. On machine B, I:
edited the inputs to be either git+file vs path:, git:, etc.
ran a nix flake update (or perhaps a lock --update-input?)
ran a rebuild switch, and when that worked
committed all changes including the flake.lock and pushed to remote
on machine C, I pulled the changes and attempted to rebuild switch
With every variation of step 1, step 5 failed with an error in fetchTree, except with git+file:, which worked (and continues to work as we speak).
There’s always a chance I missed some detail, I’ll see if I can come up with a reproducible example.