CI will soon enforce `pkgs/by-name` for new packages

Development Nixpkgs Architecture
1

Four months ago, the first milestone towards RFC 140 was achieved by introducing the pkgs/by-name directory for Nixpkgs, allowing new packages to be declared in that directory instead of the category-based file hierarchy, removing the need for all-packages.nix in most cases. Thanks to the efforts of all the contributors and reviewers, we’ve seen the number of packages in pkgs/by-name skyrocket to over 800!

Now’s time for the next step: Enforcing pkgs/by-name for all new packages (where applicable)! This is implemented in the following PR:

https://github.com/NixOS/nixpkgs/pull/275539

Unless significant problems are brought up, the PR is scheduled to have its final review and merge party on Monday, 2024-01-15T16:00:00Z in this Jitsi, anybody is free to join! :tada:

Edit: Now over! While we couldn’t merge during the meeting due to a test failure in CI, I was able to fix it fairly easily.

The PR is now merged, and will take effect once the nixos-unstable channel updates the next time!

Good to know

  • Only packages using pkgs.callPackage need to use pkgs/by-name, so this doesn’t apply for e.g. python3Packages.callPackage, callPackages (note the s!) or package aliases.
  • Whether pkgs/by-name should be used is automatically checked by CI, so there’s no need for reviewers to think about this: If CI is green, there’s no problems with pkgs/by-name and merging won’t break it.
  • This also applies to older PRs that still introduce new packages outside pkgs/by-name: They can still be merged as long as CI is green. CI will start failing if you push new code though, in which case it would have to be moved to pkgs/by-name.
  • There’s no need to migrate existing packages to pkgs/by-name. That will be done automatically as the next and final step of RFC 140. See also the guide on manual pkgs/by-name migration.

Sponsoring

Finally, a huge shout-out to mainly Tweag but also newly Antithesis for sponsoring this work! :sparkles:

50 Likes
2

Update: After merging the PR I discovered a problem, but there’s a fix ready:

https://github.com/NixOS/nixpkgs/pull/281390

It’s going to take a bit longer until it actually takes effect, but some very nice improvements came out of this, see

https://github.com/NixOS/nixpkgs/pull/281374

3 Likes
3

What is the plan for avoiding CI failures on unrelated packages due to packages not using pkgs/by-name? Apparently, staging checks for pkgs/by-name now, but new packages not using it were merged into that branch, which leads to CI failures: https://github.com/NixOS/nixpkgs/actions/runs/7600068348/job/20697863923?pr=282473

It seems as if these two could easily be fixed manually, but the documentation says we should wait for the automated migration, which is still a draft.

1 Like
4

@gm6k Oh that’s a very interesting case. See workflows/check-by-name: Trigger on base branch changes by infinisil · Pull Request #282707 · NixOS/nixpkgs · GitHub for a fix

2 Likes
5

Stupid question: what if a package is built with pkgs.libsForQt5.callPackage or qt5.callPackage… (which is, I expect, the case of all Qt packages?) Is there a way to turn it into a pkgs.callPackage entry, or should we just use the “legacy” way?

6

@tobiasBora Check out Documentation: Qt callPackage approach incompatible with pkgs/by-name · Issue #287015 · NixOS/nixpkgs · GitHub, looks like it can work with the standard pkgs.callPackage!

3 Likes