In the last Nixpkgs Architecture Team (NAT) meeting, the idea of announcing the planned agenda for the next meeting a day ahead on Discourse was well-received, so this is what this post is about.
The NAT is there to solve architectural nixpkgs issues that are too big in scope for any single person to undertake. There are regular 30 minute meetings every week (currently 13:30 to 14:00 UTC) in Jitsi Meet, open for anybody to join, recorded and live-streamed to YouTube, meeting notes are public.
The agenda for the NAT is generally created from discussions in the past week, mostly on Matrix, but you’re also free and encouraged to suggest additional topics by replying to this thread.
Agenda
Active RFC drafts
In addition to the agenda, this post can bring attention to the existing RFC drafts of the NAT. Feel free to open issues/PRs in these repositories if you want to bring something to attention relating to that RFC draft. Alternatively feel free to chat on Matrix or reply here on Discourse.
-
Auto-called packages: Make most attribute definitions in
pkgs/top-level/all-packages.nix be auto-generated from a single flat (or prefixed) directory instead, where the subdirectory corresponds to the attribute name. The ad-hoc category-based structure of packages will be removed.
4 Likes
Meeting notes
These meeting notes are also archived here. These notes are not cleaned up, usernames might not be correct.
Agenda
Grouped by agenda items
-
(@Growpotkin, @DavHau, @Profpatsch, @roberth) Using something like the module system for packages (on Matrix)
-
@David: Discussion quickly derails into “too inefficient”, “doesn’t work”. Just from user perspective, the code one needs to write is much better than overrides. So let’s not discuss implementation details, but start differently by collecting examples of what we do with overrides, and what it could look like with something like the module system. Created GitHub - DavHau/pkgs-modules: The Ideal interface for configuring packages for this.
-
@Profpatsch: Something to take into account, check-meta change started with the module system, did benchmarks, slowed down test evaluation by a bunch. There is some overhead, we need benchmarks before making.
-
@David: Agrees
-
@infinisil: I think it’s clear that the module system can configure everything we need. Shown with NixOS
-
@David: Create the interface that we’d like to have and that works first. Then implement, perhaps hacky using overlays underneath. Maybe very inefficient, but we can play with it. Later improve implementation.
-
@infinisil: Mostly concerned about a migration path for such a large change, which should also be backwards-compatible.
-
@David: The first example in GitHub - DavHau/pkgs-modules: The Ideal interface for configuring packages is already very convincing
-
@Profpatsch: In practice, what would it look like with config options. E.g.
git with pythonSupport = true, would come from the global config.pythonSupport?
-
(@Profpatsch, @Growpotkin, @hexa) Stability guarantees in nixpkgs, prompted by the type checking meta change (on Matrix)
-
@Profpatsch: Downstream was complaining about own meta attributes failing
-
@tomberek: Discussed before that we want a backwards compatible convention
-
@Profpatsch: Debugging sessions needed by third-party users for a change they didn’t cause. Change log isn’t sufficient. Breaks workflow
-
@infinisil: Recently updated NixOS, ran into removed attributes, was annoying
-
@Profpatsch: “Could have known it”
-
@mkaito: Nobody reads change logs, needs a low-noise way of announcing breaking changes, along with how to fix them. ArchLinux has a low-traffic mailing list with breaking changes, with precise steps for how to fix it. When you get one you know it’s important.
-
@fricklerhandwerk: What would be the optimal cadence for breaking changes? How often do people update NixOS?
-
@mkaito: Should be interactive. Given one has a 5 year old system, how much is it going to break?
-
@infinisil: From X to Y has combined breaking changes over those releases
-
@fricklerhandwerk: How much effort should we invest in this?
-
@mkaito: A lot, it’s important
-
@fricklerhandwerk: Once you install NixOS, shouldn’t break. But only works up to a point. Need improvements but also keep old stuff around.
-
@mkaito: Minimize breaking changes
-
@infinisil: Mailing list is last option when we can’t have a nice warning/error beforehand. A nice message is great
-
@infinisil: Inform community that we need warnings/errors for breaking changes
-
@mkaito: Improve searchability of breaking changes in release notes
-
(@ashkitten, @tomberek) Less duplication of packages with different feature sets (on Matrix)
-
@infinisil: Say feature is “don’t care whether enabled or not” allows flexibility
-
@infinisil: Another idea: Combine version requirements/specification with feature sets
-
@tomberek: Relates to global evaluation with module system, some global resolution required
-
@tomberek: Lattice structure could work for that.
-
@mkaito: Introduce some indirection to require specific versions/featuresets of packages. provider, consumer, when matches you get a thing
-
@tomberek: Late callPackage call, don’t use derivations, use “proto” (prototype) derivations. Similar to module system, don’t do multiple fixed-point resolution too early, only one at the end
-
@roberth (who can’t talk): sounds like we need some options with type “boolOr” which merges like || (and put the package in a global option that reads those)
-
@tomberek: Look into how spack works
-
@fricklerhandwerk: Might be very challenging to do. Pick most pressing pain points first instead. Unleash energy of more people
Post-agenda
-
@infinisil: Solve multiple solutions at once, from the ground up
-
@fricklerhandwerk: Might not work well, from experience with docs, have to go incrementally
-
@infinisil: Incremental might make the complexity worse
-
@fricklerhandwerk: List all different ways of doing things, remove least used one
-
@infinisil: We can’t remove something without also having another way of doing it
-
@fricklrehandwerk: E.g. replace one use case with another than is more well established
-
@David: In favor of bigger changes, some things are just not right and need to be fixed. We might not be able to achieve a good end result with just incremental changes
-
@infinisil: Incremental changes as in users updating slowly over time is alright. But we might need to make a lot of users update their third-party code. Rip off band-aid at some point
-
@fricklerhandwerk: Cannot make people migrate, needs to be
-
@David: Need smooth upgrade path
-
@mkaito: Don’t do band-aid ripping (break user code leaving them mostly alone). Backend and frontend code, incremental changes only apply to frontend code. On backend we can rip off everything as long as frontend stays the same.
-
@David: No clear separation between frontend/backend
-
@infinisil: Specify API surface/frontend, make it smaller
-
@mkaito: More decouplings
-
@fricklerhandwerk: Can always change stuff underneath. But to change surface we need to document it first
-
@mkaito: Have a small core part that everything else uses, clear API
-
@fricklerhandwerk: Document use cases that are well supported and intended
-
@John: Little risk of doing too much. We need migration paths yes, but doing anything is good already. Getting ahead of ourselves. RFC in progress is a good first step. Don’t need full vision yet. Over time we’ll get a better picture of the size of incremental steps.
-
@John: RFC’s idea of getting rid of all trivial all-packages.nix cases, leaving only non-trivial ones, giving a better idea
1 Like
It probably doesn’t square with the vision behind the package/module discussion, but I’ve wished/wondered in the past if more config/util(/maintenance?) functionality could be provided by functions in a package’s passthru.