1000 instances of nixpkgs

My impression was that flakes should generally expose both an overlay and packages (and defaultPackage if applicable). The overlay is used for composition, and the packages set is used when consuming the flake directly. This way you can evaluate nixpkgs once with several overlays applied if you want, and you can still just nix run the flake or pull packages out of its packages set if there’s an applicable binary cache this will pull from.

There’s still a downside where each flake that redeclares nixpkgs will potentially require downloading and locking a separate copy of nixpkgs (e.g. if one is nixpkgs-unstable, another is master, another is nixos-21.11-small, etc). That’s annoying. But that can be mitigated by using inputs.foo.inputs.nixpkgs.follows = "nixpkgs" (though the repetition there is annoying). And flakes that don’t really care about what version they use can just avoid declaring it at all such that it comes from the nix registry.

6 Likes

Using a flake as input and setting it’s follows for nixpkgs to whatever you want, is effectively like an overlay, but without the computational overhead for re-evaluating nixpkgs.

At least from what I understand.

While at the same time, the way how most flake-overlays are written, still require a “follows” to be applied against your nixpkgs. At least in those flakes that expose both.

Since flake inputs are fetched lazily, I don’t think you need to worry about this if you are using a correctly created overlay.

The point of the article is that Flakes should never* import nixpkgs { inherit system; } but use nixpkgs.legacyPackages.${system} instead.

As a secondary option, you could also expose an overlay. But the primary mechanism should be to expose packages. Overlays, by design, cannot be used reliably without reading their implementation. Sometimes they have a dependency on another overlay, and the ordering can only be understood by reading the code. Same with knowing which attributes are going to be extended or overridden.

*: unless your project is not public. Or if nixpkgs needs to be parameterized.

3 Likes

Given legacyPackages is defined as:

Aren’t those expressions equivalent?

1 Like

They are currently, though nixpkgs might transition to a more efficient version.

1 Like

Let’s say you have a tree of flakes:

  • A inputs:
    • nixpkgs
    • B
  • B inputs:
    • nixpkgs

And assuming that you’re using the “follows” feature in A where:

{
  inputs.B.inputs.follows.nixpkgs = "nixpkgs";
}

Now if nixpkgs.legacyPackages gets accessed from A or B, they will both be using the same instances of nixpkgs. That’s the main mechanism we can use to avoid re-creating nixpkgs for each flake.

1 Like

Here is a follow-up discussion about how to use unfree packages with that requirement:

No, overlays are still far more powerful, in a way I don’t think we should be throwing away. By composing overlays, I can change a lower level dependency if, e.g. I need to fix a vulnerability in a common system library, and that gets applied to all packages in base nixpkgs and all overlays, without having to fork nixpkgs. Or if I don’t like the configure flags that you applied to your package, I can change that with override, and then I don’t have to manually go back and thread it through to all my other dependencies; the overlays just make sure it happens automatically.

It doesn’t require reevaluating nixpkgs once per overlay; it’s all just one evaluation of nixpkgs, with minimal overhead per-overlay.

The composition abilities of overlays is unmatched and we should definitely not be throwing it away.

That said, yes, please still use follows. We really don’t want a node_modules situation on our hands, with thousands of copies of the same repos, all for one project. (Sidenote: This is one of the things fixed by the fixed point style of nixpkgs; we don’t have this hierarchy of packages saying they want other packages. Instead, we have a flat set of known definitions of each specific thing. It’s one of my only gripes with the design of flakes)

7 Likes

As I said, in my opinion overlays should be only used for “conscious root level changes of the dependency graph”.

They are not a tool to add leaf packages, they are a tool to fine-tune the full tree.

As you said, security fixes you want to apply system, at the potential cost of recompiling everything.

Or if course if you want unsafe march=native or a safer explicit version of this.

This are legitimate use cases of overlays.

2 Likes

But a package distributed via flake is just as capable of being used as a non-leaf package; flake A can depend on flake B, and A’s package can depend on B’s package. Then flake C doesn’t have the ability to e.g. reasonably override the configure flags to A’s package when it uses B’s, unless all are using overlays. Flake A may never have been considered for use as an intermediary package, and yet there it is.

To simplify that point, I’m saying there is no reason for any particular package to believe that it is a “leaf package” in a dependency graph. It can always be thrown arbitrarily deep, so overlays are the only way to reasonably manage customizations to such things.

1 Like

exposing overlays is very powerful and sometimes the only way to accomplish stuff so I wouldn’t dismiss it. Also the “follows” bit has been broken for the longest time see `follows` attribute of flake inputs seems broken as is · Issue #4808 · NixOS/nix · GitHub so I wouldn’t recommand it. What could be nice is a tool that finds the nixpkgs version that is compatible with different flakes (i.e., a common revision that builds every flakes’ outputs ).

2 Likes

@zimbatm, thanks for bringing this up!
The article contains a (rather important and confusing) error:
Line
other-dep = inputs.nixpkgs.packages.${system};
should be
other-dep = inputs.other-dep.packages.${system};

Thanks, fixed!

Lots of good points about overlays. It’s true that today, nothing beats overlays for maximum flexibility.

From a beginner’s perspective, I don’t think it’s reasonable to ask to understand them to do common tasks. The infamous infinite recursion error messages are coming from the nixpkgs fix-point.

I haven’t thought deeply about this but I think that a better solution would be to extend the Flakes model so inputs can also be constructed programmatically. One obvious one is to make the list of systems open for extension:

{
  description = "...";
  // This would be a new flake attribute that makes the flake parametrizable.
  config = {
    systems = ["x86_64-linux"];
  };
  
  inputs = {
    nixpkgs.url = "github:NixOS/nixpkgs/nixos-unstable";
    # This would copy the systems from the current flake
    nixpkgs.inheritSystems = true;
  };

  // This would be some sort of constructor of flakes.
  // The point would be to control the inputs and outputs of each flake.
  // The implementation details are fuzzy here.
  overrides = { self, nixpkgs }: {
    // Propagate the current systems to nixpkgs.
    nixpkgs.config.systems = self.config.systems;
  };
  outputs = { self, nixpkgs, ... }:
    // This would be needed to make the flake open for changing the list of systems
    let eachSystem = nixpkgs.lib.genAttrs self.config.systems; in
    {
      packages = eachSystem (system: { hello = nixpkgs.legacyPackages.${system}; });
    };
}

Just an idea

2 Likes

I always thought providing overlays was a best practice to follow because of the following mention in the flake manpage:

Overrides and follows can be combined, e.g.
 | inputs.nixops.inputs.nixpkgs.follows = "dwarffs/nixpkgs";

sets the nixpkgs input of nixops to be the same as the nixpkgs input of dwarffs. It is worth noting, however, that it is generally not useful to eliminate transitive nixpkgs flake inputs in this way. Most flakes provide their functionality through Nixpkgs overlays or NixOS modules, which are composed into the top-level flake’s nixpkgs input; so their own nixpkgs input is usually irrelevant.

Without overlays it’s not even clear to me how to expose a flake’s package to NixOS. AFAICT I have to add a custom module in my flake.nix directly that either defines a config option I can read to get the package, or stuffs it into _module.args somewhere.

nix-darwin’s solution is to automatically take your flake inputs and set that as _module.args.inputs, so I can say inputs.foo to get at the foo input, but it’s still awkward to work with as I have to resolve it to the right system (e.g. inputs.foo.packages.${pkgs.stdenv.system}.bar, and I’m not even sure this is the “correct” way to get the system or how this interacts with cross-compiling).

Overlays make usage simple. I just define an inline module with config.nixpkgs.overlays = [ foo bar baz ] and now I can get the packages from those flakes, without worrying about what system, or having to set inputs.foo.inputs.nixpkgs.follows, or whatnot. The only concern is if they define keys that conflict with each other or with keys already in nixpkgs, there’s now an ordering issue as well as the flake potentially inadvertently replacing dependencies in the nixpkgs set.

I’m also irked by the fact that if I can nix run a flake and then want to use the same package in my own flake, I have to figure out what path nix run even ran from as there are multiple options, and have to deal with other complexity like that. I wish there was a simple way to say “I have an input, give me the primary package from that input” and have it abstract away the paths and systems and whatnot. This could be a nixpkgs lib function I suppose, that just takes the input directly (and probably an optional installable so I can name the package I want) and gives me the result. And NixOS itself should have a standard way of exposing inputs to modules, perhaps by doing what nix-darwin does and have the lib.nixosSystem function take an inputs arg (that should be set to your flake inputs) and stuffing that in _module.args.inputs so this stuff is standardized (and following nix-darwin’s approach means I can share modules too).

Also related, I filed an issue on home-manager about automatically forwarding the inputs arg along which got pushback, if NixOS did this too then that would make it much more compelling as a standard way of handling flakes.

https://github.com/nix-community/home-manager/issues/2679

4 Likes

I am currently thinking about the same issue and @zimbatm’s post and this discussion cleared a lot of things up for me already. However I am still struggling to arrive at a best practice to default to when creating a new flake. Basically I was wondering the same thing asked here (Good practice for Nix Flakes) by @techieAgnostic, to which it seems the exact opposite to what’s proposed here is recommended.

My current default is to

  1. expose on overlay
  2. (optionally) expose a nixos module applying that overlay, e.g.

(assume flake-utils is used in the examples)

...
{
  overlay.default = import ./overlay.nix;
  nixosModule = {
    imports = [ ./module.nix ];
    config.nixpkgs.overlays = [ self.overlay.default ];
  };
};
...
  1. expose a package applying that same overlay.

For 3. the natural thing to do is probably what @zimbatm proposes in (Good practice for Nix Flakes):

...
let
  pkgs = import nixpkgs {
    inherit system;
    overlays = [ self.overlay ];
  };
in   
{
  packages.default = pkgs.my-package;
}

From my point of view this has a few benefits:

  1. I do not need to make an assumption on how this flake is used. Downstream users can either choose to use the overlay to avoid the “1000 nixpkgs”-problem, or decide they don’t care (e.g. because usage is at the root of the dependency tree, like nix run) and use the provided package.
  2. It’s easy to expose the package to the provided nixos module. If I understand correctly using specialArgs requires extra ritual when constructing the system configuration, while with this approach the provided nixos module is more or less usable “standalone”.

Of course it comes with the usual drawbacks of using overlays already discussed here. Most importantly:

  • overlays are opaque, ie. I cannot know what packages are exposed by the overlay without reading the flake
  • some say it is a misuse of overlays to only provide leaf packages
  • it has the potential of accidentally overwriting existing attributes in nixpkgs. Even if a name is currently not taken, that might easily change in the future.

Another drawback is that it is not at all clear to newcomers when to use the overlay and when to use the packages output of my flake. With which we’re basically back to the 1000 nixpkgs problem.

Now I wonder if at least for simple cases (ie. packages that do not need to modify nixpkgs, which is probably the majority?) I could instead expose my package as

...
let
  pkgs = import nixpkgs.legacyPackages.${system};
in   
{
  packages.default = (self.overlay.default pkgs pkgs).my-package;
}

and only fall back to importing nixpkgs when really necessary. Is there something obviously wrong with that approach?

3 Likes

My experience is that the issue with follows was fixed quite some time ago, which seems to bear out in the linked issue (it’s seemingly been closed because the problem is no longer present).

I agree with you about not dismissing overlays, though.

A serious issue with the proposed approach that was not really discussed in this thread is that you non-negotiably need to expose package definitions as “bring your own pkgs” of some form if you want cross compilation to work, since flakes don’t support cross compilation (they merely barely tolerate it). To do cross compilation, you need to callPackage the package definition with a nixpkgs initialized with crossSystem, either by sticking it in the global set or doing pkgsCross.something.callPackage. Either way, though, as an upstream flake, you need to provide an unevaluated version of your packages.

The most reasonable way I can see to do this and not contribute to nixpkgs instance proliferation is to expose an overlay that doesn’t overwrite anything, and call it, as above, with pkgs pkgs, to obtain the packages to put in packages.${system}. Then, downstream consumers that need to cross compile your software can give the overlay to nixpkgs when importing it or call it with pkgs pkgs or suchlike.

4 Likes