Flake does not follow local nixpkgs anymore even with flake:nixpkgs

A while ago I could simply not put any input and it would grab nixpkgs from the system (really practical to avoid extra dependencies). But it seems like this won’t work with flake-parts:

       error: flake-parts: The flake does not have a `nixpkgs` input. Please add it, or set `perSystem._module.args.pkgs` yourself.

Instead I tried:

  inputs = {
    flake-parts.url = "github:hercules-ci/flake-parts";
    nixpkgs.url = "flake:nixpkgs";
  };

but it still tries to download a really recent version of nixpkgs (2 days old) instead of using my older nixpkgs. I thought that flake-part might rely on nixpkgs so I tried to make it follow nixpkgs but this fails. The only solution I found to rely on my system nixpkgs is to hardcode it like:

  inputs = {
    flake-parts.url = "github:hercules-ci/flake-parts";
    nixpkgs.url = "github:nixos/nixpkgs/c97c47f2bac4fa59e2cbdeba289686ae615f8ed4";
  };

but this is of course not really practical as update etc are not very natural to do. Any idea what’s going on? Is it a bug?

For reference here is my full flake:

{
  description = "A very basic flake";

  inputs = {
    flake-parts.url = "github:hercules-ci/flake-parts";
    nixpkgs.url = "github:nixos/nixpkgs/c97c47f2bac4fa59e2cbdeba289686ae615f8ed4";
  };
  
  outputs = { flake-parts, ... } @ inputs: flake-parts.lib.mkFlake { inherit inputs; } {
    perSystem = { config, self', inputs', pkgs, system, ... }: {
      devShells.default = pkgs.mkShell {
        nativeBuildInputs = with pkgs; [
          nodejs_22
          playwright-driver.browsers
        ];

        shellHook = ''
          export PLAYWRIGHT_BROWSERS_PATH=${pkgs.playwright-driver.browsers}
          export PLAYWRIGHT_SKIP_VALIDATE_HOST_REQUIREMENTS=true;
          export PLAYWRIGHT_NODEJS_PATH=${pkgs.nodejs_22}/bin/node
          playwright_chromium_revision="$(${pkgs.jq}/bin/jq --raw-output '.browsers[] | select(.name == "chromium").revision' ${pkgs.playwright-driver}/browsers.json)"
          export PLAYWRIGHT_LAUNCH_OPTIONS_EXECUTABLE_PATH=${pkgs.playwright-driver.browsers}/chromium-$playwright_chromium_revision/chrome-linux/chrome
          echo "Using $PLAYWRIGHT_LAUNCH_OPTIONS_EXECUTABLE_PATH"
        '';
      };
    };
    # Declared systems that your flake supports. These will be enumerated in perSystem
    systems = [ "x86_64-linux" "aarch64-linux" "x86_64-darwin" "aarch64-darwin" ];
  };
}

Flake inputs do not use the local registries by design.

Of course you can use --override-input nixpkgs nixpkgs while running commands, but it won’t be part of the flake.nix itself.

@waffle8946 what do you mean, I’ve been using local registry on a project with great success to specify the URL of a local project that was changing from computers to computers. And the documentation nix flake - Nix Reference Manual precisely mentions:

Here are some examples of flake references in their URL-like representation:

  • nixpkgs: The nixpkgs entry in the flake registry.

so I would expect nixpkgs.url = "nixpkgs"; to just work. The doc mentions explicit support for registry lookup:

indirect: Indirections through the flake registry. These have the form

[flake:]<flake-id>(/<rev-or-ref>(/rev)?)?

These perform a lookup of <flake-id> in the flake registry. For example, nixpkgs and nixpkgs/release-20.09 are indirect flake references. The specified rev and/or ref are merged with the entry in the registry; see nix registry for details.

Yes, this misbehaviour got fixed somewhere between nix 2.20 and 2.25 IIRC.

Sadly the global registry is still used, so you could in theory provide your pins through the global registry.

Though the better approach would be to just not rely on the registry at all when declaring flake inputs

3 Likes

What do you mean: nix flake is not anymore able to deal with registry and documentation is not up to date (if not why??), or do you mean that I experienced a bug that should be fixed in 2.25? (I have 2.31.2 so would be weird)

Considering the registry at all is a flaw in nix’ design and bad for reproducibility.

The creators of nix eventually saw that this is at least true for local and system registry entries and ignore those when resolving flake inputs. This is mentioned in nix’ release notes and happened somewhen between 2.20 and 2.25, IIRC.

As said, you can still use the global registry to do waht you want. Still, it is bad practice and you should simply not rely on flake:* inputs, use whatever is more appropriate, like channel tarballs or github:*!


Don’t get me wrong, the registry is nice on the CLI to have a shortcut, but its really counterproductive when used in flake-inputs.

3 Likes

Note that it didn’t really accomplish that either, because the commit is still frozen in time through flake.lock. Different projects will still in practice pin different versions of nixpkgs (though they will agree if they’re created around the same time).

You can implement your own “registry” by depending on a path flake and making extensive use of follows (in fact, if you make all your other on-system flakes follow your NixOS config’s inputs that explains what’s happening much, much better). That at least makes the dependency and intent clear, but it will still not quite do what you want in practice.

I don’t agree, it is bad for reproducibility just because of the way it is implemented now, but it does not need to be bad. Registry is a bit like a DNS, you can’t always hardcode the PATH/IP (because it may change, …). I started an issue report in Regression: Nix flake ignores local registry · Issue #15441 · NixOS/nix · GitHub proposing some (possibly naive?) fixes that would fix both my issues and the issues to this original problem, so we can maybe continue the discussion there? If you think my proposition is bad, then I’d love to hear your solution to the two applications I have in mind:

  • to automatically use the system nixpkgs, e.g., to reuse the original hash
  • to compose local flakes elegantly (if you hardcode the path you won’t be able to deploy to a different machine)

Well, it is a good thing to have the commit frozen, this way I can upgrade when needed and revert to the old commit if needed. What I mean here is that by providing the same original hash as the system I don’t need to redownload a different nixpkgs & all dependencies for any single project I’m starting.

Well if you upgrade your system every 6 months, it still gives you enough space to make sure many projects pin the same nixpkgs. And if one is not, you just type nix flake update nixpkgs and here we go. With the new system, you need to spend time to copy/paste hashes between projects, and upgrade is also not elegant. As I said, maybe we can continue the discussion in Regression: Nix flake ignores local registry · Issue #15441 · NixOS/nix · GitHub.

You… Really shouldn’t do that. CVEs happen on the scale of days, even weekly updates are a little slow.

But yeah, as I said, you can instead use the follows mechanism, that does exactly what you want.

Ahah sure, in practice it’s not always possible to lose 2h to update (and possibly deal with an unstable system) every week ^^

Not sure to understand, can you detail a bit? And anyway this seems fairly hacky, it might not be something I want to do when sharing it with others as this might make it work only on my system, or anyway feel weird.

I find weekly works well for me, and at least on stable I notice barely any instability, at worst the occasional build failure. But yes, I also don’t manage every week. 6 months is much too infrequent, though.

E.g.:

# /etc/nixos/flake.nix
{
  inputs.nixpkgs.url = "https://channels.nixos.org/nixos-25.11/nixexprs.tar.xz";
}
# ~/Projects/myProject/flake.nix
{
  inputs = {
    system.url = "/etc/nixos";
    nixpkgs.follows = "system/nixpkgs";
  };
}

Well, you’re the one relying on the local registry. The local registry is incredibly hacky and doesn’t work when you share it with others.

Any projects where you rely on either the registry or this system input shouldn’t be shared. At least the system input makes explicit what’s missing, and gives people a way to replace it cleanly.

1 Like

If it’s “misbehavior”, why is it documented like this?

You can use these identifiers on the command line (e.g. when you do nix run nixpkgs#hello) or in flake input specifications in flake.nix files. The latter are automatically resolved to full URLs and recorded in the flake’s flake.lock file.

1 Like

Your doc quote is referring to the global registry. We’re talking about local registries, which are different. To keep this on topic:

An intentional design change isn’t a regression, by definition.

If you allow local registries in flake inputs, then you’re basically allowing paths to sneak into the lock file without intentionally opting into it. And those paths will not exist in any other system that tries to use this flake, which in turn means that the lock file will be broken on any system except for the current one.

--override-input is your workaround, take it or leave it.

Though your presumably internet-connected system being 6 months out of date and therefore likely easily exploitable is a much bigger issue, best get that sorted first.

You make the flake explicitly depend on hardcoded paths that may not exist on the host + this must be created by the root account (otherwise different users will have different paths), that’s really not a great this since any machine that relies on this flake needs to write their own modification of the flake which is horrible when committing etc.

Ok, to make the discussion more productive, let’s give two precise applications I have in mind:

  1. local reusable flake: say that I write in /home/me/myflakeA a flake that I want to reuse in other local flakes, and possibly share with friends/servers. The strategy that I would like to follow (and that I was following) is to do something like that:
    $ nix registry add myflakeA /home/me/myflakeA
    
    this way in all others flakes around (say /home/me/myflakeB, /home/me/myflakeC) other registries I can simply type:
    inputs.myflakeA.url = "flake:myflakeA";
    
    and here I am. I can also very easily change the ref if one project needs a specific dev branch for instance with something like that:
    inputs.myflakeA.url = "flake:myflakeA?rev=mybranch";
    
    when sharing this config with friends/server/other user, I just need to put my flakes (A, B…) in any place I want, and simply type something like:
    $ nix registry add myflakeA /home/newuser/myflakeA
    
    Very elegant I think, and I am using this method already to deploy some local project to my server easily, use several branches etc.
    Why your proposed solution is not ideal: beside being very hacky, if I don’t want to change flakeB when applying the changes to a new user, I need system.url to point to a folder that is not in the home folder, i.e. it needs to involve the root user. Also, I don’t think I can easily make different flakeB/flakeC point to a different revision like I shown above.
  2. Space-efficient flake: the second application I have in mind is a regular flake that I want to widely share and that relies on nixpkgs, but during development I’d prefer it to follow my nixpkgs for efficiency reasons (don’t want to spend 10mn redownloading the world + waste 10Gb of space). Hence, I’d like to simply make inputs.nixpkgs.url = "nixpkgs" to be able to stay compatible with all existing flakes, except that by default when creating it should point to the same revision as my local nixpkgs. I could hardcode the commit in the url, sure, but then I find it less elegant since it might suggest that I need this precise commit for a precise important reason.

Where do you see a single reference to global registries in nix flake - Nix Reference Manual? These perform a lookup of <flake-id> in the flake registry. For example, nixpkgs and nixpkgs/release-20.09 are indirect flake references. clearly refers to a non-global registry.

All changes are intentional. It is a regression for me if it has unwanted side effects which I believe is the case here, and I believe we can get best of both worlds.

That’s not a very democratic/FOSS way to discuss, as I’m telling that I believe we can do better. --override-input might be a solution to my use case 2, but not really for my usecase 1 as it actually changes the flake.lock file of flakeB (and also do you just put a dummy URL before the override-input? seems a bit wrong), which means that two people cannot collaborate with git on flakeB since the modifications on the lock file will keep being modified and sent between parties. Also, I can’t easily express the fact that flakeB should rely on a different branch except by putting a bit warning in the readme hey make sure to change your override-input now, that also seems off.

Many things must be sorted sure, including https://www.cvedetails.com/cve/cve-2024-36050 that has been around forever and that is an issue for me but also for you, even with an up-to-date system. But aren’t we going out of topic here?

Yeah, this is exactly the problem with the local registry as well. It is just as not-guaranteed to exist or contain what your flake expects as /etc/nixos.

If users do not also have a flake in their /etc/nixos, they can use --override-input to supply an equivalent without editing the source code, so root accounts or modifications to the flake are not required.

Yes, this is a hack, but you shouldn’t be sharing flakes like this with other people anyway. If you want to lock to specific versions of nixpkgs to encourage local reuse in flakes without following antipatterns, you can use --override-input with nix flake update to make targeted lockfile modifications. This is pretty easily scriptable if need-be.


Anyway, since you insist on dragging back up the conversation that led to the removal of the local registry, instead of figuring out how to work around the limitation…

What you’re running into here is a mismatch in priorities. nix wants to make sure that your code can be reproducibly developed on, and not have any dependencies on the environment it’s evaluated in. This is impossible to achieve as long as the local registry concept exists, because your evaluation will fundamentally depend on your local registry, which can be edited per-host.

A pretty common pattern is to make the inputs.nixpkgs registry entry point to the system flake’s nixpkgs input so that nix run nixpkgs#<thing> and such resolve to your already-downloaded flake input, instead of updating every time the cache TTL expires. Any system which does this will make your local registry link resolve to path:/nix/store, which will not be present on any other systems. So your flakes, as they are defined currently, will break e.g. on my system.

In other words, the hack you’re using currently already has exactly all the same issues that you claim make my suggestion unworkable, you’re just not aware that they’re there. It’s more obvious to you with my suggestion because you understand the flake input pattern, but not the local registry.

This means that the nix change has had its intended effect; it’s exposing an anti-pattern that you’ve been unkowingly following. By continuing to argue this, you’re showing exactly why this change is good, and why it should not be reverted.

Flakes do also have an issue with heavy disk churn, aliasing, mirroring, etc., and I do think these problems need some kind of solution, but the registry isn’t a very good solution to those problems. The only good solution out there at the moment is detsys’ proprietary (hosted) flakehub, which I’ve argued before is intentionally there to stifle good solutions being found inside nix to carve out space for commercialization.

4 Likes

I mean “take it or leave it” given the current intentions of nix’s design.

I’m not a nix dev, you’re welcome to argue with the nix bdfl or team if you care to. Regardless, FOSS has nothing to do with democracy - it just means you get to do whatever you want with the code on your machine. You may also feel free to fork/patch nix if that’s what you’re up for (I certainly do). Of course, that means that users of your flake must also share this forked/patched nix depending on how far you diverge from mainline.

Hence --no-update-lock-file.

The remaining misconceptions can be cleared up via google search, so I’ll leave it there.

1 Like

Then you need to write --no-update-lock-file --override-input nixpkgs nixpkgs to all commands I need to write. Too error prone.

How should I design locally sharable & modular flakes then?

Yeah, I don’t think existing solutions will help me here, I want to see if we can create one that fits my needs and yours. I don’t think we say incompatible statements. I like the old syntax, you dislike the old behavior leading to reproducibility issues that I can understand. I’m not saying I want the old system back, I want to find a way to change nix to have best of both worlds (syntax & modularity & reproducibility).

I share the same priorities, just I think your two priorities collide partially in this case and I want to favor reproducibility while you seem to favor “no environment”.

If we want to go further let’s define terms (if you don’t agree let me know):

  1. reproducibility: same code should give same result (goal: no more “it works on my machine”)
  2. environment: everything around nix and that we can’t directly control: environment variables, configuration not stored in the flake, OS, but also internet, and file system.

So your hack does depend on the environment (hardcoded path outside the flake, even more in my opinion than by relying on registries), and actually environment is ALWAYS present: e.g. without internet you can’t do much.

I think that we can agree that the new behavior reduces the need to rely on environment (alone it is a good thing) and that it also sometimes helps with reproducibility because of the issues you mentioned about the user PATH being written in the lock-file (so also a good thing), but it sometimes reduces reproducibility:

  • for instance when composing flakes as it means that when changing the location of flakeA, the code of flakeB will break. So same code != same result.
  • with the current nix we also can’t define a nixpkgs mirror, or an alternative URL. This may needed, e.g., if we are in a country that forbids access to github… See for instance Censorship of GitHub - Wikipedia for countries that (partially) block/used to block github.
  • with the current system, it is not possible to change the location of nixpkgs, e.g. to move it to gitlab since it would break all systems.

And my point is that by asking optional help to the environment, we can have everything and maximize reproducibility.

The fundamental solution for me is to distinguish “what” (revision XXX, hash YYY), “who” (nixpkgs) and “where” (https://github.com/nixos/nixpkgs), while for now “who” and “where” are identified in nix. Reproducibility is (more or less) ensured if we can give to nix the “what”, while “who” (and possibly “where”) must be provided to nix to help it to find the file (“where” will always rely on the environment, typically at least internet, so we can’t hope an environment-free solution), but “where” is a mean, not a goal in itself. So introducing an additional mechanism (relying on the environment/registries) to help get the “what” would actually improve reproducibility since the same code will give the same result even in different environments that would not be usable without relying on the environment.

So IMHO it is a mistake to hardcode a specific PATH/URL inside the flake itself. Instead:

  • The flake.nix should only mention the “who”, i.e nixpkgs.url = "nixpkgs" or nixpkgs.url = "flakeA". Note that “who” may also be an actual uri like github:foo/bar or even github:nixos/nixpkgs, e.g., when there is no canonical way to refer to an entity, but this should be interpreted as a name and not directly as the only possibly URI to get this ressource.
  • the flake.lock should only mention the “what”, i.e. the revision and hash of the file needed to continue. Importantly flake.lock should not contain the where, i.e. the user PATH, this is the current design that I’m questioning
  • Then, nix should find the “where” using as many ways as possible to help with reproducibility. In any case, it should first check if the file is already present in the /nix/store. If not, if “who” is a canonical name like nixpkgs, it would try first with the user registry, if the file is not found there, it tries then with the system registry, then global registry (this one basically always contains nixpkgs’s github repository so eventually we will get the file). If “who” is an URI, it also tries this URI + the user/system/global registries until finding a place that provides the wanted file.

This way we fix all above issues:

  • we can compose local flakes easily, basically using the same syntax as before except that under the hook nix flake does not store the “where” in the
  • we can easily add nixpkgs mirrors
  • we don’t rely only on github’s availability
  • we can use by default our local registry if available when creating a new flake to avoid pulling lot’s of external dependencies…
  • we strictly increase reproducibility at the cost of a small cost in environment: by normalizing the environment, we can actually limit the issues of having different environment itself.

If I’m missing something I’d be happy to understand.

Flakes are simply not meant to be used the way you want to use them, they are designed to be monolithic sources of truth.

Maybe legacy nix suites your use case better.

1 Like

Given how people use flake to include a project into another, I think that composability is very much an issue that flake tries to solve, while “monolitic source of truth” could more or less be achievable with a simple pin. And history shows that composable stuff are way nicer to use (whole point of flake-part for instance, and IMHO flake itself), and I don’t think that legacy nix could be an elegant solution to my problem 2. Flake was sooo close to be the perfect tool.

I am aware of the previous discussions and have been using --override-input nixpkgs nixpkgs for some time now. I am still constantly annoyed by the length of these flags and the fact that, without them I may accidentally introduce a bunch of different nixpkgs copies on my tiny system (and this has happened multiple times).

I understand the reproducibility concern for the local registry but it is really helpful for usability. Maybe a good compromise is to add a --use-local-registries flag (and maybe a global option in nix.conf as well) to always respect local registry overrides.

For better reproducibility, we can force the CLI to always print a warning: using local registries for flake inputs when this flag is used, so that people are aware of the caveats.

My current workaround is to use direnv: create a .envrc file and put nix flake update --override-input nixpkgs nixpkgs within it, for projects that I want to keep in sync with my local registry.

1 Like