Nix flake inputs not lazy

Help
1

Just to be sure I’m understanding this correctly, and because I was somewhat confused by @roberth’s comment here…

Currently I have a shared flake between several systems. It works great for nixos-rebuild, darwin-rebuild, and nixops deploy, across x86_64-linux (local and cloud), aarch64-linux (an RPi3), and aarch64-darwin (my primary day-to-day device). Huzzah!

On the RPi, I decided to turn its previously (Raspbian, Ubuntu) working scanner functionailty into a flake.

I wrote a flake using a local path: inputs.scanner.url = "/path/to/repo";. I don’t refer to that input or flake anywhere in the other systems, and so I expected nix’s laziness to allow this to work (because this input is never used on those systems, it would never be evaluated).

Instead, none of the other systems will build. I then tried to remove the input and use getFlake, but then things are impure.

For the time being I just pushed the scanner flake to a private git repo.

I think I’m waiting for this to land before this will work as expected:

Does this all sound right? If so, @roberth could you clarify what you meant in the above-linked comment?

2

Internally, Nix does something along the lines of

result = flake.outputs inputs;
inputs =
  mapAttrs
    (name: lockItem: fetchTree lockItem)
    lockedInputs;

Values in an attribute set are computed lazily, so as long as you don’t use the value of an input, the input does not need to be fetched during evaluation.

If your flake isn’t locked or if the lock file needs to be updated because you’ve changed the inputs, you’re doing more than evaluation though, so maybe that could explain what you’re seeing. Otherwise, --show-trace might give a clue as to why your expression does load the input. Maybe you’re using a framework that scans the inputs for some attribute. That’s one way to kill the lazy fetching.

Lazy trees / Source tree abstraction does not change this behavior. It only makes the unpacking lazy. Accessing any single file inside an input (think flake.nix) will be enough to cause the whole input to be downloaded, but not necessarily written to the store.

3

Thanks for your time and the input.

For my test case, I’m using a locked flake which is up to date.

$ git status
On branch master
Your branch is up to date with 'origin/master'.

nothing to commit, working tree clean

Adding this input, with no other changes (and therefore not being utilized anywhere):

notexist.url = "/does-not-exist";

And then:

$ sudo nixos-rebuild --flake . switch
warning: Git tree '/home/n8henrie/git/nixos' is dirty
error: getting status of '/does-not-exist': No such file or directory

So if I understand you correctly, the issue is that flake.lock is now out of date, and it’s trying to automatically update the lockfile, and this step is causing the error (not the flake itself). Is that right?

4

Hmmm, either I’ve misunderstood or perhaps my config is evaluating the input somewhere I didn’t notice.

I’ve pushed the updated lockfile into the repo and pulled the changes into a system that doesn’t use that input (and doesn’t have the path available).

$ sudo nixos-rebuild --flake . switch --show-trace
error: getting status of '/home/n8henrie/git/scanbutton': No such file or directory
5

@roberth

I’m wondering if a POC script might make it easier for you to point out what I’m misunderstanding. (EDIT: If you have time of course, I’m grateful for the input you’ve already had here.)

This little script creates two flakes in /tmp, both using the trivial template (but x86_64-linux → aarch64-linux). For the second, it adds an input named unused pointed at the first one, which is thereafter only referenced in outputs = { ..., unused }, because otherwise it complains about being passed an unexpected argument.

It then builds flake2, showing that everything works, and then moves the path to which flake1 refers and rebuilds, and fails with

+ nix build                                                                                                            │
error: getting status of '/tmp/flake1': No such file or directory

This is what I mean by the path not being used in the flake outputs at all and yet being required for the flake to build. I thought that lazy evaluation would mean that it wouldn’t matter that unused.url didn’t exist – because unused isn’t… used?

#!/usr/bin/env bash

set -Eeuf -o pipefail
set -x

readonly FLAKE2='
{
  description = "A very basic flake";

  inputs.unused.url = "/tmp/flake1";

  outputs = { self, nixpkgs, unused }: {

    packages.aarch64-linux.hello = nixpkgs.legacyPackages.aarch64-linux.hello;

    packages.aarch64-linux.default = self.packages.aarch64-linux.hello;

  };
}
'

main() {
    rm -rf /tmp/flake{1,2,1.bak}
    mkdir -p /tmp/flake{1,2}

    cd /tmp/flake1
    nix flake init
    sed -i 's/x86_64/aarch64/g' ./flake.nix

    cd /tmp/flake2
    echo "$FLAKE2" >flake.nix
    nix build

    mv /tmp/flake1{,.bak}
    nix build
}
main "$@"
6

Right, the locking part is not lazy and locking needs to run if you don’t have a lock file yet.
If you create the /tmp/flake1 path, then lock the flake, and then remove the path, consumers of your flake should not be able to tell that something is wrong unless they look inside the inputs attribute.

7

I added another nix build step in /tmp/flake1 (right after sed), and I see in the output that the lockfile is created, but the outcome is the same. Removing the set -x:

$ ./empty_input.sh
wrote: /tmp/flake1/flake.nix
warning: creating lock file '/tmp/flake1/flake.lock'
warning: creating lock file '/tmp/flake2/flake.lock'
error: getting status of '/tmp/flake1': No such file or directory

Current script:

#!/usr/bin/env sh

set -Eeuf -o pipefail

readonly FLAKE2='
{
  description = "A very basic flake";

  inputs.unused.url = "/tmp/flake1";

  outputs = { self, nixpkgs, unused }: {

    packages.aarch64-linux.hello = nixpkgs.legacyPackages.aarch64-linux.hello;

    packages.aarch64-linux.default = self.packages.aarch64-linux.hello;

  };
}
'

main() {
    rm -rf /tmp/flake{1,2,1.bak}
    mkdir -p /tmp/flake{1,2}

    cd /tmp/flake1
    nix flake init
    sed -i 's/x86_64/aarch64/g' ./flake.nix
    nix build

    cd /tmp/flake2
    echo "$FLAKE2" >flake.nix
    nix build

    mv /tmp/flake1{,.bak}
    nix build
}
main "$@"

Also tried scattering some nix flake locks in there, tried --no-update-lock-file for the last nix build, but the behavior is the same.

8

Maybe I’m wrong and maybe flake-compat is the best implementation of flakes, or maybe local paths are treated specially. If this doesn’t work for git-based flakes either, I’d consider that a bug.

9

Wow, that helps! Using inputs.fake.url = "/tmp/testflake";, where /tmp/testflake is a git repo with a committed flake.lock, if I add that path to inputs and nix build, then move / remove /tmp/testflake, then nix build again, it fails (even though I’ve never used that input anywhere else in the flake).

However, if I change nothing about /tmp/testflake but change it to inputs.fake.url = "git+file:/tmp/testflake", I can nix build, then remove that file, and it still builds.

Not only that, but as long as I build and commit the parent flake.lock, I can successfully build on a different system (defined in the same parent flake) that does not have /tmp/testflake available.

So I think this solves the issue – I just need to use git+file:/tmp/testflake instead of /tmp/testflake.

Thanks @roberth!

1 Like