Nixpkgs's current development workflow is not sustainable

Development
53

I think that the community ought to consider packaging and pinning as separable tasks with separate workflows.

Correct me if I’m mistaken (?) but I feel that currently there’s an (implicit?) rule/guideline that there should be one version of every package. However this seems to lead to PRs that cause a lot of contention.

For example, take my recent PR bumping pytorch-bin from 1.10.0 to 1.11.0: pytorch-bin: 1.10.0 -> 1.11.0 by rehno-lindeque · Pull Request #164712 · NixOS/nixpkgs · GitHub

I’m an infrequent contributor to nixpkgs, and to be honest when I do contribute I’m kind of slow and can’t keep up with the rapid changes:

  • It’s pretty easy for me to contribute new/bump existing packages. E.g. adding pytorch-bin_1_11_0 when pytorch-bin = pytorch-bin_1_10_0; already exists
  • But you may not want me to do pinning because:
    A. I don’t really know much about some other packageset and how its build systems work. It takes a sizable effort to get up to speed.
    B. I’m just never really going to have a huge amount of motivation to dive into the downstream effects of my change that I’ll never experience first hand.
    C. Even when I do make the effort to make sure everything keeps building consistently I might not have enough experience to do a good job of testing and knowing what I’m missing.

The current situation seems to me leads to huge PRs which has the following problems:

  • long comment threads and lots of context that you have to read through
  • much of that context is already stale by the time you read it
  • there’s a cadence mismatch between authors, maintainers and reviewers - different people work at different times etc
  • large amounts of back-and-forth and hopping between tasks leads to mental fatigue
  • nixpkgs itself is a moving target which changes frequently

Instead I’d much rather contribute separate granular PRs for pytorch-bin_1_11_0, torchvision-bin_1_11_0, torchaudio-bin_1_11_0, etc. Then, finally, when all that work is done it’s possible to make a much more focussed effort to roll forward the entire pytorch based ecosystem, involving many more stakeholders.

(I know this “kicks the can down the road” so to speak, and I’m an infrequent contributor as I said. However, I personally think that you eat an elephant one bite at a time.)

9 Likes