What would you do if you were the BDFL of Nix?

1

Just for fun :slight_smile:

Imagine that you had all the powers on the NixOS project. You could decide the decide what to do and even direct the rest of the community to help you out.

What would be the top 5 things that you would change?

8 Likes
2

Here are the top things that pop in my head right now:

  1. Merge home-manager into nixpkgs, with a generic nix-rebuild script that would work across both and be installable from the pkgs/ packages.
  2. Re-create the file hierarchy in nixpkgs. It would be 4 levels deep at most. All the pkgs derivation names would have an enforced 1:1 mapping with their filenames to make lookups trivial.
  3. Create a new documentation system that follows the new nixpkgs file hierarchy. Each packages and modules can have their own README.md in their folder that would be collected into a central documentation.
  4. Something we discussed with @flokli today: create a new website that acts as a meta-layer on top of GitHub. Each package and nixos modules would have their own page that contain the maintainer, versions per channel, and open issues and PRs. It would be possible to create new issues for the selected package from that page.
  5. And finally, create a constitution that specifies an election system and governance model so that when I am done with all of this I can leave the position :stuck_out_tongue:
22 Likes
3

I’d probably do a similar thing (maybe not an election system) and try to remove myself as BDFL :grin:.
It just seems to have as many downsides as upsides:

  • probably stressful
  • people expect a lot from you
  • major changes have to get your approval
  • people care what I think too much
  • “with the power comes the great responsibility”

and there’s always the risk of coup d’état.

But with all that in mind :thinking:, I’m thinking I could do the most damage by ruining the Nix codebase because I don’t know C++.

3 Likes
4

@worldofpeace I feel like your answer was waaay too realistic :stuck_out_tongue:

One of my annoyances with the Nix ecosystem is that there is no formal leadership in a lot of nixpkgs. If you don’t have commit rights and you want something merged in, you basically just have to know who to bug to get it merged in.

There isn’t a single person who is responsible for making a yes/no call to merge something in.

For instance, there was recently a PR for bumping pytorch from @stites:

python3Packages.pytorch: 1.2.0 -> 1.4.1, python3Packages.ignite: 0.2.1 -> 0.3.0 by stites · Pull Request #75827 · NixOS/nixpkgs · GitHub

I think Sam put a lot of work into this, but there were small suggestions from a bunch of different people. Normally this would be okay, but no one ever said something along the lines of “If you do X, Y, Z, then we will merge this in.”

So it appears that Sam just got burnt out from the lack of a clear goal, and no one having explicit responsibility to decide yes/no for merging this in.

I think it is really unfortunate we don’t have someone in a BDFL position with the responsibility to make these yes/no calls.

This leads me to my answer. If I were BDFL of nixpkgs, I would setup some sort of clear responsibility hierarchy for deciding whether or not to merge PRs. There should be a given person who has the responsibility to say yes/no to any given PR. People contributing to nixpkgs should be able to easily figure out who is responsible for their PR.

However, to be honest, I’m not sure how this would work in practice:

  • I think it could be easy for some of the subsystems in nixpkgs. For instance, I think it would be relatively easy to decide on BDFLs for the Haskell subsystem or Python subsystem. But there are lots of packages in nixpkgs that don’t clearly fall into a given subsystem. How do you decide who is responsible for giving a decisive yes/no for them? Maybe with @zimbatm’s suggestion of a more clear nixpkgs package hierarchy, this would be easier.

  • What do you do when a subsystem BDFL starts making bad decisions? Or merging in things that don’t build?

  • What would be the interaction between package maintainers and the BDFL? Should package maintainers be removed automatically if they stop responding to PR for their packages?

  • I think there are some nixpkgs commiters that only review a small portion of all the nixpkgs PRs. For instance, I almost exclusively review PRs for the Haskell subsystem. However, there are other reviewers that review all sorts of PRs. Maybe this subsystem BDFL wouldn’t make sense for them?

  • Would other people be less likely to review PRs if they are not officially responsible for the final yes/no decision? In Sam’s pytorch PR from above, I think it is really great that so many people were willing to do a review. Reviewing PRs is often very challenging. I think it is amazing so many people were willing to give their time to review a non-trivial PR like this.

I’m not very familiar with it, but I think the Linux kernel development happens in a similar BDFL-fashion. I’d be interested if anyone here has experience for how this works in practice. (I feel like I’ve also heard that a lot of the core Linux kernel developers are paid, which is probably not the case for the nixpkgs developers.)

2 Likes
5

:joy_cat: I guess you’re right. Perhaps my interpretation of @zimbatm’s “just for fun” is creating chaos.

When I was release manager, it was probably the only position where I could see people looking to me for a clear yes/no because I was the gate keeper to the releases.
I thought it was weird, at first, but the more I did that the better things would get.

Contributing to nixpkgs, in my experience, is navigating a bunch of cliques.
While thankfully, these cliques are next level legit and let me sit with them, not everyone is going to work best with that. I’m not sure what’s better, usually if you’re in the clique you’re silently happy.

A growing thing, that was an idea @jtojnar and I shared (and was implemented by him), is maintainer teams https://github.com/NixOS/nixpkgs/blob/3a1e8bdcd9a9e13ccd3f298bf598acc8b35caa4d/maintainers/team-list.nix.
It’s not automated yet, but we’ve added them for GitHub too Sign in to GitHub · GitHub Sign in to GitHub · GitHub. I see the future, at least for subsystems, is that these teams make a yes/no collectively. So there isn’t one person, and I think that’s healthier at least for subsystems.

I think the answer is those people would form the nixpkgs packaging committee, or something. I happen to be one of those people who are fine reviewing whatever, and can find a happy medium of a yes/no call, or who to ask to make a more informed decision. I go :crying_cat_face: when we burn out contributors.

I’ve found that the social aspect of nixpkgs is not to turn away anyone as long as they interact fair and kindly.

3 Likes
6
  • NixOS/kNetBSD
  • Emacs integration all over the way! Including documentation in org mode (README in ORG)
  • Spend everything I can in documentation
  • Every big project being put in overlays, from X11 and Wayland to XFCE and KDE Now I prefer the monorepo approach
  • Create an init system, or some abstraction for an init system
    • Getting rid of systemd dependency
  • A nix evaluator in pure Zig!
  • Lisp Flavored Nix
  • A Parliament :slight_smile:
2 Likes
7

Probably not too much, I like Nix and NixOS and as an outsider, the RFC process seems to work well.

  1. Clearly separate dependencies from build options in derivations. I guess that a straightforward implementation would be to let each derivation take two attribute sets, one with dependencies and one with (usually compile-time) configuration options. Make package configuration options easy to query/document.
  2. Maintainer teams like @worldofpeace suggests. As an occasional contributor it is sometimes hard to know who to poke to get reviews/merges.
  3. Set up a clear overview on where nixpkgs stands in various areas. Something along the lines of Rust’s are we X yet, with clear fine-grained status + pointers. Examples: Wayland support, ROCm support, FreeBSD status, Video playback acceleration. Currently figuring these things our requires hunting through issues, PRs and the Wiki.
  4. Implement a basic curses installer a la Slackware or BSD. Most installs that I do start out pretty standard, an installer with some basic choices would make it simpler/faster.
  5. Controversial: make allowUnfree the default. People who are principled know where to find the knob, 95% of the users probably install something that is unfree and it will make life easier for them. (And would anyone care on a Mac, most of the system is unfree already :stuck_out_tongue: )
8 Likes
8

Point 4: I know there is someone trying it!

Point 5 above: I would make something like “warnUnfree”. It would point out if and what packages you are using that aren’t free, but would be installed anyway.

5 Likes
9

I’ve tried reviewing this PR several times, it can take >64GB of ram to do a nixpkgs-review, and it’s often broken because machine learning packages are notorious for breaking changes.

Edit: I agree that he put in a lot of hardwork, and did a lot of good. But the nature of the package makes it difficult to test pratically (not to mention the integration of mkl, cuda, and other optional libraries)

Edit2: Actually takes >100GB

Selection_024
Selection_0241802Ă—1012 1.38 MB

5 Likes
10

Two words: LTS releases.

1 Like
11

Controversial: make allowUnfree the default. People who are principled know where to find the knob, 95% of the users probably install something that is unfree and it will make life easier for them.

Where we probably agree: make allowUnfree actually mean anything when default is true…

I have some very specific unfree packages and want to control that list tightly, and as there are many slightly different contexts in which different things are evaluated, having a full guarantee that each of them except exactly one Nix expression gets nondefault allowUnfree = false is hard.

Obviously, the complicated interplay of details matters not only for unfree packages…

12

I assume that one cannot really magic resources out of thin air, but can do what amounts to convincing all the people to accept the core principle of some RFCs (which then get slowly debated on details as things do not get written perfectly the first time, and later accepted).

I guess I risk ending up driving an anti-perfectionism + anti-SPOF campaign…

Core principles:

  1. If something is important in principle and an ugly proposal works, accept it and use until it breaks (or until a cleaner one fully covers what the ugly version does).

This obviously applies to things like new test types, but also…

  1. If we can swap out a component and get the basics working, then we need this capability.

Be it NixOS/k*BSD, libc choice, alternative Nix evaluation options if they arise, init, service supervision… or running just a few services on another distribution without installing a full NixOS.

And I guess the way to do it without sinking everything into a complete mess is better override options and better separation of substantial from dependency-choice specific…

-=-=-=-

… and I guess if I started defining the BDFL powers in comparison to the current RFC process, maybe doing all this by abusing influence in RFC discussions, then becoming more careful to avoid abusing influence is a way to step down without even a clear step-down event.

1 Like
13
8 Likes
14

I’d broaden the userbase to attract different roles into the ecosystem and seak salvation in letting sort out complexity by itself.

Therefore, leveraging flakes:

  1. nix develop → the most notorious tool in the world to attract contributors and community to about any repository across any language. Strongly appeals to the maintainers.
  2. nix run → having done the above, the maintainer might choose to make his baby discoverable through the nix package manager
  3. All the rest.

Then I’d take the hat and resort to RFCs for governance.

15
  • auto-merge “trivial” pull requests after one week, except one maintainer says “veto”
    • “trivial” means the diff is shorter than 20 lines (for example), so it is cheap to review
    • rate-limit is 1 PR per user per week
    • if something breaks, we can always revert a merged PR, similar to wikipedia “edit wars”. this is better than stale PRs
    • this should encourage contributors to keep diffs small
  • more maintainers = more users with commit privilege
  • automate updating of packages
  • automate adding new packages, similar to github copilot (AI pair programmer)
  • integrate projects like python on nix which are just workarounds for the undermaintained nixpkgs
  • find a shorter way to say tensorflow.overridePythonAttrs (old: { version = "1.2.3"; }), for example tensorflow."v1.2.3". by default, maintain multiple versions per package. problem? we would have to say tensorflow.latest. maybe solve this in Nix, by implementing the @ operator. then tensorflow is the default version, and tensorflow@1.2.3 is a pinned version. the @ operator would override the derivation’s version attribute. related: Concept: use any package version
  • rename all pname/default.nix to pname/pname.nix so i dont have a million default.nix tabs in vscodium. upstream issue Provide API to access and change editor tab labels · Issue #41909 · microsoft/vscode · GitHub
  • fix the live DVD, so it has a dark mode splash screen. im still traumatized by that 90% white splash screen in the middle of the night *__*
1 Like
16

Yes.

I implemented something like this, nixos-up.

Lots of great suggestions already, I would add:

  • CODEOWNERS and integrations with meta.maintainers. If you are a maintainer, you are responsible for reviewing and merging the relevant PRs. If you don’t attend to PRs then you are removed so someone else can come in and take up the reins.
  • More focus on server-side usage for NixOS, less emphasis on desktop. Desktop users are great (I am one), but in my experience it’s on the server where NixOS really shines. We should leverage that as a beachhead towards more adoption. In a similar vein, Nix-on-macOS should be a continued area of investment as that’s the #1 most popular platform for developers these days.
  • home-manager should be part of NixOS.
  • Better Nvidia/CUDA support :slight_smile:
  • Moonshot: start moving Nix away from C++ and into Rust since that’s the direction the universe is headed now.
6 Likes
17

There is a PR that adds a graphical installer in the work. A basic install already works.

https://github.com/NixOS/nixpkgs/pull/161788

3 Likes