Nixpkgs-multiverse: every version that ever existed

Links
1

Disclosure: AI helped for in tool and graph creation.

57 Likes
2

I got really excited when Farid showed me this!

He has fixed the oldest and most upvoted issue (62 upvotes) on devenv:

{ multiverse, ... }: {
  packages = [
    multiverse.cmake."3.16.5"
    multiverse.bun."0.7.0"
  ];
}

Docs

10 Likes
3

This is incredible :star_struck: I have a couple questions:

  • Why do you use nixos-unstable specifically instead of nixpkgs-unstable? Or just the union of both? Similarly for the stable branches, would it make sense to also have nixpkgs-*-darwin instead of just nixos-*?

  • Could you add a dev shell with all the necessary dependencies to run the scripts under tools/? For instance, I currently get this error when I try to run tools/build-index.sh:

    tools/build-index.sh: line 44: mapfile: command not found
Traceback (most recent call last):
  File "<string>", line 6, in <module>
BrokenPipeError: [Errno 32] Broken pipe
Exception ignored in: <_io.TextIOWrapper name='<stdout>' mode='w' encoding='utf-8'>
BrokenPipeError: [Errno 32] Broken pipe
1 Like
4

AFAIK nixpkgs-unstable vs nixos-* only differs in what they test. It might uncover a few more commits to walk both so maybe it’s worthwhile.

I definitely want to limit it to commits that were channel bumps so that they have access to the cache.

I will update the flake shortly. Family & dinner time right now :call_me_hand:or I welcome PRs.

(I will be wiring in the GitHub action I have as well)

2 Likes
5

From the article:

Every pin is a whole extra nixpkgs in the file. Flake inputs are fetched eagerly even if not used. A flake with three nixpkgs inputs whose output references only the first are all materialised.

I didn’t realize this was a thing with flakes – all inputs are always evaluated? Why do they break lazy evaluation?

For example in normal nix:

[user@host ~]$ nix repl -I nixpkgs=https://github.com/NixOS/nixpkgs/archive/b79c10888f9147c128e2f6c00607c02c6ee3208f.tar.gz
Nix 2.34.2
Type :? for help.
nix-repl> ({pkgs}: 1) {pkgs = import <nixpkgs> {};}
1

nix-repl> (import <nixpkgs> {}).hello
# does the download and copying here but does inline console output and then erases it
«derivation /nix/store/d46zpb9f4qfi85sn8n6zghdf67p7dhgq-hello-2.12.3.drv»

nix-repl>
1 Like
6

Is this still true after https://github.com/NixOS/nix/pull/15711 (included in 2.35.0)?

7

Fixed + added updating github action.

1 Like
8

I’m on 2.34.8 so potentially /shrug

9

Cool idea. I have to ask though: was an LLM involved in the creation of this project?

Also, what if version “1.0.0” of some package “foo” was broken in one nixpkgs revision and then removed in the next revision. IIUC, in that case versions.foo."1.0.0" would forever reference a broken derivation. I suppose, this could be improved by only bumping “versions” that were successfully built by hydra, but then that wouldn’t work with unfree packages.

10

Such an elegant solution! I’m almost surprised nobody has built this before

3 Likes
11

Please read the opening post. It is disclosed. I personally am uninterested in seeing any more LLM discussion in the thread.

5 Likes
12

My bad. I searched the linked blogpost and the github repo for any mentions of AI / LLMs, but somehow missed the disclaimer on the announcement post itself.

2 Likes
13

@fzakaria is a strong contender for the Nobel Prize in Nix. Every week a new amazing blog post.

6 Likes
14

While I do have opinions™ about mixing nixpkgs (too many glibc mixups debugged at different employers after they found out I know a thing or two about that :wink: ), I do get that sometimes that’s a very useful thing to have.

That being said, I do like the multiverse metaphor here because one big value nixpkgs IMHO has is that it’s a single package-set with packages working together (well, most of them) which is why I found all these ideas around splitting nixpkgs up aggressively into flakes extremely off-putting and with this metaphor you can fit both stances together, to a certain degree at least.

14 Likes
15

I think the way to use this is to pull in individual top-level packages only.
This does not by default override anything such that glibc will be mixed up.

Each package remains true to the graph at the commit it’s from by default which will always work unless you do .overrideAttr or something else.

2 Likes
16

Another thing you can often do — correctness not guaranteed, naturally; this goes directly against the whole package-set advantage — is something like:

pkgs.callPackage multiverse.foobar."X.Y.Z".override { }

in order to get a leaf package from X.Y.Z built that draws dependencies from your main Nixpkgs, preventing a lot of near-duplication. Not every package can support this; some packages involve wrapping things in ways that aren’t observationally equivalent to the standard callPackage pattern.

(I haven’t tried nixpkgs-multiverse yet, but I’ve done this sort of thing with manually-pinned Nixpkgs versions.)

8 Likes
17

Flake inputs are locked eagerly, but fetched lazily.

9 Likes
18

I wonder if you could use something like Z3 to minimize the number of nixpkgs revisions for a given set of package versions

3 Likes
19

Some updates:

image
image1816Ă—1350 157 KB

10 Likes
20

This now exists with mvs solve – it’s not even a SAT problem; it’s polynomial.

8 Likes