But a package distributed via flake is just as capable of being used as a non-leaf package; flake A can depend on flake B, and A’s package can depend on B’s package. Then flake C doesn’t have the ability to e.g. reasonably override the configure flags to A’s package when it uses B’s, unless all are using overlays. Flake A may never have been considered for use as an intermediary package, and yet there it is.
To simplify that point, I’m saying there is no reason for any particular package to believe that it is a “leaf package” in a dependency graph. It can always be thrown arbitrarily deep, so overlays are the only way to reasonably manage customizations to such things.
1 Like
exposing overlays is very powerful and sometimes the only way to accomplish stuff so I wouldn’t dismiss it. Also the “follows” bit has been broken for the longest time see `follows` attribute of flake inputs seems broken as is · Issue #4808 · NixOS/nix · GitHub so I wouldn’t recommand it. What could be nice is a tool that finds the nixpkgs version that is compatible with different flakes (i.e., a common revision that builds every flakes’ outputs ).
2 Likes
@zimbatm, thanks for bringing this up!
The article contains a (rather important and confusing) error:
Line
other-dep = inputs.nixpkgs.packages.${system};
should be
other-dep = inputs.other-dep.packages.${system};
Thanks, fixed!
Lots of good points about overlays. It’s true that today, nothing beats overlays for maximum flexibility.
From a beginner’s perspective, I don’t think it’s reasonable to ask to understand them to do common tasks. The infamous infinite recursion error messages are coming from the nixpkgs fix-point.
I haven’t thought deeply about this but I think that a better solution would be to extend the Flakes model so inputs can also be constructed programmatically. One obvious one is to make the list of systems open for extension:
{
description = "...";
// This would be a new flake attribute that makes the flake parametrizable.
config = {
systems = ["x86_64-linux"];
};
inputs = {
nixpkgs.url = "github:NixOS/nixpkgs/nixos-unstable";
# This would copy the systems from the current flake
nixpkgs.inheritSystems = true;
};
// This would be some sort of constructor of flakes.
// The point would be to control the inputs and outputs of each flake.
// The implementation details are fuzzy here.
overrides = { self, nixpkgs }: {
// Propagate the current systems to nixpkgs.
nixpkgs.config.systems = self.config.systems;
};
outputs = { self, nixpkgs, ... }:
// This would be needed to make the flake open for changing the list of systems
let eachSystem = nixpkgs.lib.genAttrs self.config.systems; in
{
packages = eachSystem (system: { hello = nixpkgs.legacyPackages.${system}; });
};
}
Just an idea
2 Likes
I always thought providing overlays was a best practice to follow because of the following mention in the flake manpage:
Overrides and follows can be combined, e.g. | inputs.nixops.inputs.nixpkgs.follows = "dwarffs/nixpkgs";sets the nixpkgs input of nixops to be the same as the nixpkgs input of dwarffs. It is worth noting, however, that it is generally not useful to eliminate transitive nixpkgs flake inputs in this way. Most flakes provide their functionality through Nixpkgs overlays or NixOS modules, which are composed into the top-level flake’s nixpkgs input; so their own nixpkgs input is usually irrelevant.
Without overlays it’s not even clear to me how to expose a flake’s package to NixOS. AFAICT I have to add a custom module in my flake.nix directly that either defines a config option I can read to get the package, or stuffs it into _module.args somewhere.
nix-darwin’s solution is to automatically take your flake inputs and set that as _module.args.inputs, so I can say inputs.foo to get at the foo input, but it’s still awkward to work with as I have to resolve it to the right system (e.g. inputs.foo.packages.${pkgs.stdenv.system}.bar, and I’m not even sure this is the “correct” way to get the system or how this interacts with cross-compiling).
Overlays make usage simple. I just define an inline module with config.nixpkgs.overlays = [ foo bar baz ] and now I can get the packages from those flakes, without worrying about what system, or having to set inputs.foo.inputs.nixpkgs.follows, or whatnot. The only concern is if they define keys that conflict with each other or with keys already in nixpkgs, there’s now an ordering issue as well as the flake potentially inadvertently replacing dependencies in the nixpkgs set.
I’m also irked by the fact that if I can nix run a flake and then want to use the same package in my own flake, I have to figure out what path nix run even ran from as there are multiple options, and have to deal with other complexity like that. I wish there was a simple way to say “I have an input, give me the primary package from that input” and have it abstract away the paths and systems and whatnot. This could be a nixpkgs lib function I suppose, that just takes the input directly (and probably an optional installable so I can name the package I want) and gives me the result. And NixOS itself should have a standard way of exposing inputs to modules, perhaps by doing what nix-darwin does and have the lib.nixosSystem function take an inputs arg (that should be set to your flake inputs) and stuffing that in _module.args.inputs so this stuff is standardized (and following nix-darwin’s approach means I can share modules too).
Also related, I filed an issue on home-manager about automatically forwarding the inputs arg along which got pushback, if NixOS did this too then that would make it much more compelling as a standard way of handling flakes.
4 Likes
I am currently thinking about the same issue and @zimbatm’s post and this discussion cleared a lot of things up for me already. However I am still struggling to arrive at a best practice to default to when creating a new flake. Basically I was wondering the same thing asked here (Good practice for Nix Flakes) by @techieAgnostic, to which it seems the exact opposite to what’s proposed here is recommended.
My current default is to
- expose on overlay
- (optionally) expose a nixos module applying that overlay, e.g.
(assume flake-utils is used in the examples)
...
{
overlay.default = import ./overlay.nix;
nixosModule = {
imports = [ ./module.nix ];
config.nixpkgs.overlays = [ self.overlay.default ];
};
};
...
- expose a package applying that same overlay.
For 3. the natural thing to do is probably what @zimbatm proposes in (Good practice for Nix Flakes):
...
let
pkgs = import nixpkgs {
inherit system;
overlays = [ self.overlay ];
};
in
{
packages.default = pkgs.my-package;
}
From my point of view this has a few benefits:
- I do not need to make an assumption on how this flake is used. Downstream users can either choose to use the overlay to avoid the “1000 nixpkgs”-problem, or decide they don’t care (e.g. because usage is at the root of the dependency tree, like
nix run) and use the provided package. - It’s easy to expose the package to the provided nixos module. If I understand correctly using
specialArgsrequires extra ritual when constructing the system configuration, while with this approach the provided nixos module is more or less usable “standalone”.
Of course it comes with the usual drawbacks of using overlays already discussed here. Most importantly:
- overlays are opaque, ie. I cannot know what packages are exposed by the overlay without reading the flake
- some say it is a misuse of overlays to only provide leaf packages
- it has the potential of accidentally overwriting existing attributes in nixpkgs. Even if a name is currently not taken, that might easily change in the future.
Another drawback is that it is not at all clear to newcomers when to use the overlay and when to use the packages output of my flake. With which we’re basically back to the 1000 nixpkgs problem.
Now I wonder if at least for simple cases (ie. packages that do not need to modify nixpkgs, which is probably the majority?) I could instead expose my package as
...
let
pkgs = import nixpkgs.legacyPackages.${system};
in
{
packages.default = (self.overlay.default pkgs pkgs).my-package;
}
and only fall back to importing nixpkgs when really necessary. Is there something obviously wrong with that approach?
3 Likes
My experience is that the issue with follows was fixed quite some time ago, which seems to bear out in the linked issue (it’s seemingly been closed because the problem is no longer present).
I agree with you about not dismissing overlays, though.
A serious issue with the proposed approach that was not really discussed in this thread is that you non-negotiably need to expose package definitions as “bring your own pkgs” of some form if you want cross compilation to work, since flakes don’t support cross compilation (they merely barely tolerate it). To do cross compilation, you need to callPackage the package definition with a nixpkgs initialized with crossSystem, either by sticking it in the global set or doing pkgsCross.something.callPackage. Either way, though, as an upstream flake, you need to provide an unevaluated version of your packages.
The most reasonable way I can see to do this and not contribute to nixpkgs instance proliferation is to expose an overlay that doesn’t overwrite anything, and call it, as above, with pkgs pkgs, to obtain the packages to put in packages.${system}. Then, downstream consumers that need to cross compile your software can give the overlay to nixpkgs when importing it or call it with pkgs pkgs or suchlike.
4 Likes