Let's nudge for goreleaser nix flakes support

I’m a relative laymen. If you’re not and can push those argumentative buttons to nudge the goreleaser maintainers into supporting nix flakes, please feel free to comment on that issue.

Driven by the multiplicator potential of goreleaser…

An argument against it is that flakes is still an experimental feature. :grimacing:

Very true. Though a good one. :wink:

I expect this needs a considerable amount of time in downstream libraires to gain traction and be seriously evaluated, so I thought I’d throw an early thought anchor at them.

poetry for python might be a similar target, I knew of, for such nudging.

Maybe this particular use case in downstream release libraries might inform a common interface in flakes for language builders to make their job easy as possible. … since easy and fanned-out adoption might be a goal for flakes to achieve.

EDIT: One such newly discovered use case optimizing downstream maintainability Flakes don't discover git references?

I thought about this last night and still feel this way this morning – I love flakes, I’ve converted an extensive personal repo and two overlays to flakes, and have finally started another largish new Nix project because I finally feel like it makes sense with flakes…

But I don’t think the tooling is ready to start asking others to do work with flakes. I’m not convinced it even makes sense to use flakes for source code when it simply doesn’t really do the job for Go or Rust (or other-non-Git-scm packages afaict). And when the Nix 2 CLI tooling has so much room to grow still. I mean, you have to add --experimental-features 'nix-command flakes' to every single invocation starting with 2.4. There are bugs that block common scenarios from working with chaining calls together, there are commands that just are broken, etc.

Personally, if I were asked to start basically doing packaging work for another distro, I’d be pretty turned off if I waded in currently, simply given the state of docs, lack of examples, lack of nix builtins, lack of standard flake utils library, flake lock format changing, etc.

7 Likes

While I appreciate your effort, I have to agree with @colemickens, pushing the work of package maintenance (for a non-mainstream ecosystem) to an upstream repository is not a good idea (regardless of the stability of flakes).

Also, we don’t have a general agreement whether or not nixpkgs will be split into smaller chunks (I still prefer a monorepo approach for instance, but that’s another topic), so there’s still a chance that we’ll continue maintaining the majority of packages in a single repo even with stabilized flakes.

5 Likes

It is spinning around my head for two days now, and the picture starts getting clearer.

I get the understanding that central package discoverability is probably served by something like https://github.com/NixOS/flake-registry (either centrally or decentrally). I think this solves discoverability advantages of a monorepo.

Packages could also be discovered using the github api, see e.g https://github.com/casey/janus

EDIT: The answer was given. It’s pi: Flake discoverability

As for the reproducability guarantees, it probably depends on the language tooling, if - in a general sense - nix could offload reproducability trust to that tooling. For go, I can see how this is somewhat solved with go.sum. Go itself has the go get command with a hosting service aware syntax. So why not? nix shell github:bla/blubb is - in reality - just a language neutral abstraction to that.

However, not every language ecosystem is conformed in a way that it might become a candidate for offloading reproducability trust partly into this ecosystem’s tooling and use flake.nix as a generalized interface to the nix package manager.

In the letter case, nix would need an external source for content hashes (either hash or build instructions), which would necessarily reside within the nix community’s boundary.

1 Like

I think this solves discoverability advantages of a monorepo.

Well this is not just about package discoverability for end-users:

  • Updates affecting multiple packages are way easier to integrate in a single repository. For instance, I’ve been taking care of the last two glibc updates and having a single Hydra jobset and an integration branch (on top of staging) was fairly helpful. Having to contribute to tons of downstream-repos would make this way more complicated.
  • Having a single repo makes it easier to debug things (you simply need to grep over a single git checkout)

However, not every language ecosystem is conformed in a way that it might become a candiate for offloading reproducability trust partly into this ecosystem’s tooling and use flake.nix as a generalized interface to the nix package manager.

This isn’t just a question about the qualification of an ecosystem: why should we push the work of maintaining another build-system to upstream maintainers who probably don’t even know what NixOS is (or don’t care about it)?

2 Likes

Agree! I should have formulated better in the first place.

Having to contribute to tons of downstream-repos would make this way more complicated.

Agree, we might need to introduce a concise and meaningful distinction between leaf and core packages.

The majority of (go) packages not yet available in nix are leaf packages. They would just pin the nixpkgs version (in flakes.lock) that works for them and go along until they need a more recent go version.

Having a single repo makes it easier to debug things (you simply need to grep over a single git checkout)

I think that perspective is not a concern of the application developer role which just wants to ease as widespread adoption as possible for their brainchild, including nix.

why should we push the work of maintaining another build-system to upstream maintainers who probably don’t even know what NixOS is (or don’t care about it)?

The argument of this post is precisely to only target people whose goal is to support application developers to maintain yet another release system (ok, it says “binaries”, but at the same time it’s called go"releaser"). And thereby avoid bothering application developers directly.

To that end, I might paraphrase your question and make it an argument for leveraging those multipliers:

Why should we push work to the end application developer, if we can partner up with release specialists?

I think it’s important to think of flakes more broadly as a release system, rather than narrow it down to a build system. Sure, a release system has opinion about how to build. So let’s get this opinion out of the way for the application developer and allow him focus on the release aspect.

I made it through my hands on learning session: https://github.com/zegl/kube-score/pull/274 on nix flakes. :smile_cat:

However, out of gratefulness to be willing to include nix flakes builds (much more quickly than it would be possible on nixpkgs *), if feel obliged to provide the esteemed and brave package maintainer with an easy to update way for this vendorSha256 thing. Within intermediate time frame, for him, it all boils down to maintaining this vendorSha256 thing!

What can I tell @zegl, if he was actually about to pull this off?

See also: https://github.com/zegl/kube-score/pull/274#discussion_r456947380

That’s the kind of real (and very focused) use case for goreleaser, etc. support. QED.

* My real other option to seeking relieve in horizontal scaling of package distribution, is to contribute marginally to the already dire overload of nixpkgs maintainers: all nixpkgs maintainer tools optimizations and workflows only ever scale so much so until they don’t any more. Flakes are the solution, they scale with every marginal adopter. And as long as the reproducibility ethics of the nix package manager can be enforced it has the potential of developing a whole different dynamic of its own.

1 Like

Since not everyone is willing to click through all the topics I’m going to resume the conclusion of those interactions:

Let’s assume for a second that the flake concept has a strong intrinsic motivation to fan out and be adopted out in the wild.

To help it with that, people that maintain software that explicitly or implicitly plays a role as release multipliers such as goreleaser, poetry or […] would require from the nix ecosystem a flake generator & maintainer library with language binding for their implementation language.

Those projects could serve as prevursors and inspiration:

1 Like