Nixpkgs's current development workflow is not sustainable

Development
89

Here’s a diagram that will hopefully explain the orignal idea better
(not a response to recent messages, just me catching up)

  • Pink and green are the main points/purpose
  • Every circle is a commit (unimportant commits are not visible though)
  • Curved dotted lines are dependencies.
    Purple/Blue = nixpkgs-depending-on-torch
    Gray = torch-depending-on-nixpkgs

There’s lots of possible variation I wanted to show, but the visualization gets complicated quick.
I’ll make more visualizations for edgescases we want to discuss. (And I talk about some of them below)

IMG_1303.PNG
IMG_1303.PNG1920×1439 123 KB

  • *Observations and Possibilities

    • The only edgecase visualized is the blue dot (wave-2-3), because I thought it was important.

      In theory, wave-2-3 was a python update that broke torch 1.10.0.
      In this hypothetical, the easiest way to get “all-green” again was just patch the old wave-1-5 tag (purple) and create the patched-commit (the blue dot itself).

      This is importatnt because its different from wave-2-5. Wave-2-5 is specifically trying to update torch to the latest version (1.10.3).
      (Wave-2-5 is the wave that “hits” pytorch)

    • The part that says “glibc, … (and downstream fixes)” the blue dot/wave-2-3 is an example of such a downstream fix

    • Notice nixpkgs never connects (purple line) to torch 1.10.2. That’s intentional.
      (In this example, torch 1.10.2 can only be obtained from the torch repo directly)

    • Torch 1.10.1, 1.10.2, 1.10.3, show all their dependencies (gray dotted lines) going to wave-1-5.
      This was visually convienient and is plausible, but is not required.
      In practice, I’d expect them to point at the latest finished wave. For example
      torch 1.10.3 => wave-2-1
      torch 1.10.2 => wave-1-10
      torch 1.10.1 => wave1-6
      “which wave?” would be a choice of the torch maintainer
      (and it would be done based what dependencies torch needs).

    • Multiple versions, like we have for LLVM, are fine (or at least are not worse than the current situation).
      For example, torch1_10_3 and torch1_10_0 could both be in nixpkgs.
      Instead of one purple to wave-2-5,
      there would be two purple lines,
      one pointing to the wave-2-5 tag (renamed to “wave-2-5__1.10.3”)
      and the 2nd line pointing at an older commit (with a wave-2-5__1.10.0 tag)

12 Likes