2023-10-30 Nix team meeting minutes #99

Development Nix
1

Attendees: @roberth @edolstra @ericson2314 @tomberek @infinisil (@fricklerhandwerk, second half)

Notes

  • Regarding fetchTree docs (see 2023-10-27 Nix team meeting minutes #98)

  • Issues with fetchTree:

  • Work on fetchTree docs

  • Clarification about fetchTree behavior of the url field while using the git type in attrset form: parameters passed in the url field SHOULD NOT affect the other attributes.

  • Regarding server,path, and params. These are not valid attributes passed to fetchTree, these are specifically portions of the URL form, not the attribute-style.

  • “+”-style syntax is expressed by the “type” field being the first component and the remaining portion after the “+” as the url attribute.

  • “dir” support

    • potentially a source reference attribute? (out-of-scope)
    • is currently supported by flake references as “subdir” #8440
    • currently out of scope, could be added later

  • “narHash”

    • generic parameter
    • will be used to perform substitution, and if succussful, this will mean other attribute parameters are ignored.

  • fetchTree returns an attribute set

    • outPath
    • narHash (same as what might have been specified as an argument, result is checked against that)
    • depends on the fetcher type:
      • github
        • lastModified
        • lastModifiedDate
        • shortRev
        • rev
      • git
        • lastModified
        • lastModifiedDate
        • shortRev
        • rev
        • revCount
          • if the tree is substituted rather than fetched from git, the output won’t contain that information
        • submodule

  • question: include type or other information in fetchTree? concern about including the URL, branch, tag, type, etc

  • fetchTree is intended to provide the same information as would end up in a lockfile

    • that information is supposed to make the remote reference reproducible

  • invariant: idempotency

    • if you feed fetchTree its own output (except outPath, which is only there to coerce the result to a store path), it will produce the same result again

  • tangential question: should source metadata live in the lock file or is it a separate concern?

    • @roberth: a git commit is more than a plain tree
      • we want to retain as much metadata about it as we can
      • we may want a component that verifies these metadata against the source
      • we should at least characterise each attribute, such as “revCount will be regenerated if not specified”
    • what about impure metadata like branch names or tags?
    • what exactly is the use case for that out-of-tree metadata?

  • what parts are stable now, which experimental?

    • fetchers and their attribute-style input scheme: file (includes URLs to plain files), tarball (also underlies fetchTarball), git (also underlies fetchGit)
      • @roberth asks to un-stabilise git due to unresolved reproducibility concerns
        • @edolstra will add a hack to keep fetchGit working
      • not stable: path, mercurial
    • URL syntax is still experimental

  • any consequences for implementation?

    • type field is missing in fetchTree output, needs to be added
    • @roberth: assuming one wants to write a fetcher, how to decide which behavior the input/output attributes should produce?

  • do we have enough information to document the current state?

    • @fricklerhandwerk: should result in something good enough given a few days for the review cycle

  • would that documentation be good enough to publish in a release?

  • clarify and/or fix output attribute classes?

    • @roberth finds a lot of issues with his code when writing documentation, and the design has some unknown unknowns, so it seems important enough to look at it in detail

  • decisions:

    • the git input type won’t be part of the release
      • reason: we want to make sure it’s reproducible and the behavior well-defined, and we don’t expect to implement that for the next release if we want to keep the regular release schedule
      • stabilising that will be the next milestone: address git smudging, and reproducibility, git-tree-based fetching
    • @edolstra will make the git fetcher unstable again
    • @fricklerhandwerk will open a PR to document the fetchTree input types as they are, @infinisil will review
    • @roberth and @fricklerhandwerk will pair to clarify output attribute semantics
      • deliverble: documentation on how attributes are handled by fetchTree, for review and consideration by the team
    • next time: review results and decide what to do next