2023-01-16 Nix team meeting minutes #24

Development Nix
1

Attendees: @edolstra @thufschmitt @roberth @tomberek @fricklerhandwerk @Ericson2314

Agenda

  • outline problem space and design space around flakes and CLI in order to decide with approach to follow incrementally in the future

Notes

  • @tomberek: flake-utils provide a new interface. is that something that we want to provide, is that the outcome that we want?
    • @roberth: probably. it’s not the most general way to deal with packages, but it doesn’t have to be
      • there are others, such as mkPackage, overlays, or the immutable way where all dependencies are defined by flake inputs
        • the latter is an actual “package”, where dependencies are finalised
        • supposedly the community will mostly depend on the variant where dependencies are final, which is not a function
          • also that’s what the CLI supports
        • @tomberek: we definitely have to provide both
          • @roberth: agreed. perSystem is a good solution for the finalised variant
  • @edolstra: don’t want to block stabilising flakes on this
    • @roberth: agreed. if this change turns out problematic, fine with postponing it
    • a couple of things have to be addressed:
      • show flake should show what a flake provides, perSystem does not allow doing that
        • @roberth: can pass the current system
        • @edolstra: that can’t enumerate all other systems
          • @tomberek: can’t present both in the same category then, these are different things
      • the evaluation cache doesn’t support it, but that can be fixed
        • @thufschmitt: the current implementation uses a compatibility layer, so it isn’t a problem here
        • @roberth: could make the cache more powerful
      • it shouldn’t change callFlake or anything in the flakes mechanism
        • could just add a packagesFor output with a system argument, at the CLI layer
          • this would not change much about the autocalling, and call the function instead of accessing the system attribute
          • @thufschmitt: that could be merged right away under an experimental flag
        • we should not bring the concept of a system into the flakes specification
  • @tomberek: would it be simpler to make the mapping over attributes less awkward? e.g. natively provide a library function
    • @edolstra: that is what flake-utils does. but what you want is to use a flake on any system type. but because evaluation is hermetic, you cannot actually have that
    • what if we had an explicit enumeration of known-good outputs by the flake author, and in addition a “system-less” variant which does not give guarantees?
      • this is a pattern that we already have in Nixpkgs, and similar to default.nix + release.nix
      • it already works and needs no additional machinery, except establishing it as a convention and making it more convenient
      • hesitant to bake flake-utils logic into callFlake
  • @thufschmitt: do we agree we want to merge this under an experimental flag?
    • @fricklerhandwerk: it’s an experimental flag on top of flakes, find that weird
    • @edolstra: the answer is yes, because otherwise we’d be stuck with this feature when we stabilise flakes
    • @tomberek: it’s fine since it’s implemented only as CLI syntactic sugar and doesn’t change callFlake
    • agreement, @roberth can spend some time on following up with the PR
  • @Ericson2314: it sounds there is agreement we want to stabilise some things but not others. @edolstra, would you like to stabilise flakes and the new CLI together?
    • @edolstra: yes, but it doesn’t have to be, because they are conceptually independent
  • @thufschmitt: shall we discuss the broader issue more or get back to the issue board?
    • @edolstra: no, this has been debated over and over, do not want to rehash this again. some people seemingly simply don’t agree with hermetic evaluation and discoverability
    • @fricklerhandwerk: the point is not to debate, but to converge on what we can do in order to reduce the cognitive dissonance produced by the long-standing experimental features
  • (some ruminations around RFC 49)
    • @fricklerhandwerk: not clear what prevented the RFC from getting merged. boiled down to its essence is that a flake is a function from flakes to attribute sets that can be adressed by a flake reference. all the stuff on top is what makes it open-ended and controversial
      • @Ericson2314: no one uses flakes on their own
        • @tomberek: the examples of home-manager and nix-darwin using getFlake independently of how the Nix CLI uses flakes is an indication that the flakes spec solves a more generic problem than Nix and Nixpkgs do, and that we should lean towards supporting that generic solution
    • @tomberek: it could continue almost unchanged. the only controversial piece would be follows
      • @edolstra: follows is not that controversial. but we may still leave override as an experimental feature, because it allows injecting configuration
      • @Ericson2314: follows doesn’t really work, we shouldn’t stabilise it
        • @edolstra: that’s a bug, it has nothing to do with the proposal
      • @tomberek: what if we took out all the weakest parts, would that alleviate the remaining hesitance?
        • @Ericson2314: no, it should instead be constructive, rebuilt bottom-up. the main criticism is that the flakes proposal in its current state is too big
          • @edolstra: it is already boiled down to the bare minimum
        • (assorted brainstorming)
  • @fricklerhandwerk: there seems to be consenus that we want to get closer to stabilisation. we lack resources to do multiple far-reaching things at once. what should we focus on first then? stabilising the flakes schema or the new CLI?
    • @Ericson2314: should do the new CLI paradigm without flakes, not an obscure feature of the language such as getFlake
    • @roberth: the new CLI existed before flakes. in favor of stabilising that without flakes-specifics. we can help a lot of people just with the CLI.
      • @edolstra: the new CLI is incomplete because it doesn’t have support for channels
      • @roberth: one can use NIX_PATH without channels
    • @tomberek: flakes are an attempt at standardising a lot of things people have been doing. where it doesn’t do it the way people are accustomed to, this is where its weaknesses are overemphasised
      • in favor of addressing the mechanism first, not the CLI. otherwise we’d be stabilising an informal understanding of the underlying mechanism from the outside. but the CLI and the flakes are not mutually exclusive and both go down the right path
    • @edolstra: would ideally focus on both of them – Nix 3.0 would require both – but they can be done separately
      • one issue is documentation. if we were to set everything up to be flakes-centric we don’t want to first show the new CLI with --file everywhere
      • technically we can just flip the switch to enable the new CLI without flakes, but from a presentation perspective it would be very unfortunate and produce lots of wasted effort
    • @roberth: problem: no clear separation between process and end goal. stabilising is a process. one could stabilise only individual nix commands which are independent from the others.
      • we don’t need Nix 3.0 for this, it would be just as well to have it a decade from now
      • focusing on the milestone is not productive, because it removes focus from what we can do right now
        • @fricklerhandwerk: agree
        • @edolstra: a milestone is important from a marketing perspective, because it allows communicating a significant change in a concise way
          • also deprecating the old CLI is something that you wouldn’t want to do in 2.x
          • @roberth: we can reconcile the two. Nix 3.0 could be defined as the point where everything is stabilised and the old CLI is deprecated. but we can have arbitrarily many steps in between. (credit to @Ericson2314)
  • consensus: we want to stabilise the new CLI incrementally, independently of developing flakes
    • @edolstra will make a draft PR for discussing this in detail
  • we did not come to a conclusion about how to proceed with the flakes proposal

Agenda for next time

  • discuss whether and, if, how to approach picking up the flakes proposal (60 min)
5 Likes
2

This is encouraging to see! I’m grateful for your collective work in wrangling a ton of experimental functionality into a stable release. This will be a huge boon to the whole community. I appreciate the conservative approach of disentangling the ergonomic changes to the CLI from the new built-in tools and interfaces for distributing and referring to Nix code— even though both are features that are important to me.

4 Likes