Should nix flakes become the default installation method for packages?

[iFreilicht]

Following up on the discussion from Nixpkgs package badges as I think this is an interesting topic that would derail the topic there.

(To define the scope a little, I’m only talking about the vision for the future here, I know that flakes are not going to be stabilized for some time still and won’t become any sort of default before that)

@vcunat

Getting off topic, but I don’t think that flakes should become the default installation way. Or generally any way where the whole dependency tree is manually hard-pinned.

Say, will individual package maintainers really maintain and update their whole dependency trees? (and users pulling them) Or will it be common for these to contain security issues that could’ve been fixed for months?

I see what you mean. If you use flakes inside other flakes, this is resolved by .follows, but when installing on the command line, you don’t have access to that option directly. You could just fork the repo and nix flake update it, but that’s only an individual solution.

If the flake fits the notion of a package, it can get imported into nixpkgs pretty easily, and then .follows will work just fine again.

But generally, how much of a problem is this really? Isn’t this exactly the job of a package maintainer? They put out some software and maintain it so that it stays up to date?

I feel like your argument works for software that is outdated or abandoned, but in that case the argument applies with or without flakes.

This somewhat ties in to my impression that the user journey for using nix is not clearly defined yet. Granted, there’s a lot of ways to use nix and things to use it for, but it seems the most common use and the one we introduce users to first is installing a package with nix-env and hopefully nix profile in the future.

I hope we can find a way to quickly introduce users to the imperative interface of nix profile and then immediately transition to a declarative approach, and I assume that is what you’re suggesting when you talk about flakes as the “default installation way”?

[vcunat]

I see security updates of openssl as job for the distro. Not for upstreams of every package (transitively) using it, and not for inividual users.

IMO flakes were primarily designed for developer workflow. But there’s generally (outside nix*) a trend in lockfiles…

[ElvishJerricco]

@vcunat

I see security updates of openssl as job for the distro. Not for upstreams of every package (transitively) using it, and not for inividual users.

I think this is very important. Any given package effectively has an operating system’s worth of dependencies, and usually many more. This very much needs to come from an up-to-date and well curated package repo, such as nixpkgs. So once package A depends on package B, it’s going to be very desirable that B were just in nixpkgs instead, and maintained alongside updates to whatever parts of nixpkgs it depends on. The more granular the flake dependency graph, the more difficult it is not just to keep things up to date, but to ensure they’re actually curated to work well together.

[iFreilicht]

@vcunat

I see security updates of openssl as job for the distro. Not for upstreams of every package (transitively) using it, and not for inividual users.

IMO flakes were primarily designed for developer workflow. But there’s generally (outside nix*) a trend in lockfiles…

Ok sure, that makes sense if you’re on NixOS, but when using nix as a package manager on any other distro, you don’t really have that option.

We explicitly don’t want to rely on the distros version of any package as that would break purity and thus all the reproducibility guarantees you would usually get from nix, so everything has to come from nixpkgs.

Ideally, every repository with a flake.nix would just be imported into nixpkgs, then we’d have full control over the dependencies, but that’s not quite feasible.

@ElvishJerricco

This very much needs to come from an up-to-date and well curated package repo, such as nixpkgs. So once package A depends on package B, it’s going to be very desirable that B were just in nixpkgs instead, and maintained alongside updates to whatever parts of nixpkgs it depends on.

Right ok, so nixpkgs basically has the role of the “distro” in our scenario. That makes sense.

And I agree, the full dependency graph should be in nixpkgs if possible at all, and flakes should make this much easier, right? They already offer developers a great tool to manage their build environment, and that means adding their project to nixpkgs becomes trivial.

So as I understand you, while something like

nix profile install github:randomdude/someproject

is and should be possible to execute, we should actively discourage users from using this in favor of either creating their own flake that can use .follows to ensure the dependencies are up to date or submitting a packaging request to nixpkgs?

[vcunat]

@iFreilicht

Ok sure, that makes sense if you’re on NixOS, but when using nix as a package manager on any other distro, you don’t really have that option.

No, I see no difference in NixOS vs. non-NixOS. (except a couple parts like the kernel)

EDIT: I mean, the expressions, the whole binary closures – it’s the same, bit for bit.

[7c6f434c]

@iFreilicht

Isn’t this exactly the job of a package maintainer?

Non-bug-fixing maintenance is a problem, not a desirable state of affairs. Whenever the dependency upstreams (like OpenSSL) succeeds to maintain compatibility across a bugfix, the version record in Nixpkgs should be the only place that needs changing and distribution. Whatever takes us further from that state of affairs, is a degradation.

[TLATER]

@iFreilicht

If you use flakes inside other flakes, this is resolved by .follows, but when installing on the command line, you don’t have access to that option directly.

Maybe this is worth addressing in general. Flakes definitely have a habit of giving you dozens of nixpkgs versions if you use them from the CLI, with no mechanism to prevent this.

I agree that distros are the best way to maintain a secure and functional software stack, but realistically projects have been moving to fat binaries for a looong time. You won’t always be able to guarantee that all distros and platforms can run your project, so the ability to bundle your package with a platform that you have tested is often desirable. Flakes offer this, and projects will use it.

Software does become abandoned and insecure, but it’s sort of on the user when they go shopping outside the confines of their distro. As long as the messaging is clear that third party projects are not endorsed by NixOS, and that this may mean they’re insecure, I don’t think that’s necessarily a problem.

That’s not to say installing packages from flakes should be the “default”. But I do think they’re a use case that should be considered normal.

[iFreilicht]

@7c6f434c

Non-bug-fixing maintenance is a problem, not a desirable state of affairs. Whenever the dependency upstreams (like OpenSSL) succeeds to maintain compatibility across a bugfix, the version record in Nixpkgs should be the only place that needs changing and distribution. Whatever takes us further from that state of affairs, is a degradation.

Good point, I really wasn’t considering that. It really makes no sense for the package maintainer to do that work, and in fact in ecosystems where they do, like npm, this is a real problem that requires workarounds like explicit resolution overrides.

@vcunat

No, I see no difference in NixOS vs. non-NixOS. (except a couple parts like the kernel)

You’re right, there really isn’t one. In this scenario, nixpkgs really has the same function as any other distros package collection.

@TLATER

Maybe this is worth addressing in general. Flakes definitely have a habit of giving you dozens of nixpkgs versions if you use them from the CLI, with no mechanism to prevent this.

Hmm potentially? I could imagine something like a --optimize-follows option that automatically adds follows to all inputs and tries to match them to something in your profile or the registry, but that seems quite magical. Or there could be a --follows <input-name> <flakeref> you can specify multiple times. Not sure how this would interact with nix profile upgrade.

But anyway, I believe the solution would be to make the declarative approach (i.e. a flake.nix in your home directory or user specified location) so easy to use that it’s the default most users are drawn towards.

@TLATER

You won’t always be able to guarantee that all distros and platforms can run your project, so the ability to bundle your package with a platform that you have tested is often desirable. Flakes offer this, and projects will use it.

Yes, I believe so too, the value preposition is just so great for the project maintainers; you make development, contributing, troubleshooting and distribution much easier and more reliable in a single step.

To me, this is what Nix seemed to promise when I first heard about it, but the usability has always hindered the adoption. Flakes have the potential to resolve this.

@TLATER

That’s not to say installing packages from flakes should be the “default”. But I do think they’re a use case that should be considered normal.

Yeah good point. It is the first point of contact, the easiest to understand, and maybe useful in some cases, but generally the worst way to manage the packages in your profile.

@TLATER

Software does become abandoned and insecure, but it’s sort of on the user when they go shopping outside the confines of their distro. As long as the messaging is clear that third party projects are not endorsed by NixOS, and that this may mean they’re insecure, I don’t think that’s necessarily a problem.

Good point, yes. Some abandoned software still works perfectly fine as well, and what is insecure really depends on the context the software will run in.

[vcunat]

@iFreilicht

Yes, I believe so too, the value preposition is just so great for the project maintainers; you make development, contributing, troubleshooting and distribution much easier and more reliable in a single step.

Yes, if you maintain a project, it is a simplification when every user is running exactly the same binary (bit-for-bit), including all dependencies. Ideally they’d all use a small set of HW variants as well, because that also changes things. But… I kind-of think it’s a wrong mindset.

reminds me of Apple, but that might be incidental. You could see the same with upstream-distributed containers, flatpaks, snaps, etc.

[iFreilicht]

I think I see what you’re saying here, but I didn’t want to argue for this sort of homogenization in general at all. But that’s besides the point.

I concede, the default way of installation should always be nixpkgs. It is more secure and avoids common usage problems with flakes. Power-users might want to install flakes directly under some circumstances and it’s nice that they can do that, but we shouldn’t encourage maintainers to advertise that as the mode of installation and instead recommend to package the project in nixpkgs, which should be a minimal amount of work if they’re already using flakes to build their project.