I have a NixOS configuation as defined as a flake output someSystem. It uses packages fooPackage defined in another flake that I import as an input. I’ve noticed that building that someSystem rebuilds the input package (fooPackage), even though I’ve built them locally with nix build .#fooPackage just a moment ago.
Looks like fooPackage has a different hash when built from inside it’s own flake, and when imported via git.
I checked git ref=... on the import is right, I checked that it’s not myPackages.input.nixpkgs.follows. The system in question is the same on both machines (just a normal x86_64-linux).
As a general rule, nix believes hashes over urls and commit ids. If it has something with the right hash around, it stops looking at that point, even if the url or commit id has changed. It takes a bit of getting used to, but it would actually take a somewhat significant architectural change to improve the situation without causing other problems.
I did change ref=... on the input to point to another git commit. In similar situation a building system like cargo would know that the input changed, so must be updated. In a similar way I would expect nix to effectively run nix flake lock --update-input input on the input I changed automatically and not require me to run it manually.
It’s an understandable expectation, but the current architecture of the nix store and database doesn’t have a way to save the relevant information, much less act on it. It’s somewhat complicated by the fact that the store functions as a cache for so many different kinds of things.
This isn’t a problem unique to flake inputs, either. When you change the url on a FOD and leave the hash, it silently reuses the previous result, despite the url change. I don’t know that anyone has proposed a solution that cuts appropriately across all the different aspects of the issue.
Also, I believe no flake inputs ever update automatically. The only thing that happens automatically is that newly defined inputs that haven’t been locked get automatically locked. Changing the semantics would be hard, since in most cases, there’s no sensible way to tell if the locked version of an input could correspond to the given source or not. Under exactly what circumstances should it change it, and under what circumstances should it leave it alone?
Personally, I would actually expect and prefer that nix absolutely never change the lock file except when explicitly requested to do so… that is, I would like it to fail with a complaint about an unlocked input rather than presumptively locking, even when an input is not yet locked at all.
and other input data is already in the flake.lock.
When the input is a locked via rev and it changed, it is clear that input must be updated.
It’s all exactly the same problem cargo has where dependencies can be "*" (kind of like Nix input being branch so the source can change underneath), release version (so it will not change) or a local or git path , including specific git branch and a commit. I’m sure it does not update on "*", doesn’t have to update on release version, but it does update when git commit is specified. I’m not sure about the exact rules, but it does the right thing from my experience.
I’d say that commit pinning change is a clear signal to auto-update, possibly even any source string change should trigger an auto-update, just like adding new input does.
The issue is evaluating the suitability of a given existing store path for use as that flake input. For that, you’d need to have saved information regarding the store path, which you just don’t have with the current data model. Right now, you just have a file tree and its hash, with no historical context, so the only decision you can make is to accept it if the hash matches (or to never accept it, but that’s rather wasteful).
This is also further complicated (in the general case) because FODs can run totally arbitrary code. How is nix supposed to know that it’s ok to substitute the output from before when the version of curl being used has changed, but not ok to do so if the url has changed? As far as nix is concerned, it’s just a big, opaque description of a thing to run.
I’m sorry, but I don’t get it. As an end user: If I changed input in my flake.nix (what’s in the flake.lock doesn’t equal what’s in the flake.nix, let’s just have nix flake lock --update-input <changed-input> once on a first nix flake-related command notices the mismatch, which will update flake.lock and that’s it.
Is the problem that more generically nix might be doing something involving my local flake, where it does not have flake.lock available or otherwise the “on a first nix flake-related command notices the mismatch” is not possible? It can do it when the input was added, so …
… I’m sorry, you’re right. Nix has more information in the case of flake inputs, and I didn’t think it through before answering. Because you can detect the mismatch between flake.nix and flake.lock, it is possible to do this without fixing the more general problem for FODs and builtin fetchers. flake.lock is essentially acting as the extra database that you would need.
I still would much rather that nix never change a lock file without explicit instructions to do so, but we both agree that it shouldn’t silently proceed with a mismatched value. (I would argue it should error out in that case, rather than presume it’s ok to alter the lock file, though that could also be configurable)
An error would be much better than silently producing wrong result, so that’s fine with me.
I don’t mind auto-updates because… if I really didn’t want updates, I would pin the version. But I am aware that it might lead to some problems in certain use-cases, and cargo even has some flags to tweak this behavior:
--frozen Require Cargo.lock and cache are up to date
--keep-going Do not abort the build as soon as there is an error (unstable)
--locked Require Cargo.lock is up to date
--offline Run without accessing the network
IIRC - these are relevant ones.
So should I open a github issue if it seems that this change seems possible and desirable?