2024-04-29 Nix team meeting minutes #142

Development Nix
1

Attendees: @edolstra @roberth @tomberek (later: @fricklerhandwerk)
Notes: @tomberek

Lazy `fetchTree` `outPath` path values by roberth · Pull Request #10252 · NixOS/nix · GitHub

Alternative lazy tree strings implementation

  • @edolstra attempt being made using a new type of context to return string with context w/ virtual path
  • only when used in derivation does it copy to store
  • not finished
  • fetchTree would return storePath w/context
  • @roberth: still has value to return a path
    • gap: when observed, it is fetched?
    • /nix/store/virtual/12321382857938 with context
    • semantics? substring?
      • @edolstra: cannot hide storepaths from nix
      • already have placeholders in evaluator
      • CA derivation set precedent
      • only restriction is user should not “hide the hash”, similar to compress/decompress trick
    • counter… (can be used to observe eval order!!!, should fix)
    • fingerprint is not yet stable between versions
    • ca-derivations are imperfect as well
  • @tomberek: user impact?
    • @edolstra: presentation should make this clearer
  • @roberth: example: flake is loaded, base directory is a “lazy path”
    • resolve a module using baseDir, produces another lazy path
    • toString is used to convert path → attribute name
  • @edolstra:
    • hope is that this simplifies other things
  • @roberth: to remove the nondeterminism, string can represted by list of potential thunks
    • single observable type, with multiple constructors underneath, like list1, list2, listN
      • tStringConcat
      • tStringThunk
      • tString
      • each has a “length” method, and relevant builtins can opt to use the low level representation instead of converting to a tString first
      • not useful for module system unless attrNames can be full strings instead of symbols
    • @tomberek: looked at list of thunk representation, potential memory savings and speed
    • thunk+length, can have placeholder
    • string ops will realize strings as needed, or kept in-representation
  • @tomberek: very interest in updating the string representation as well as lazy attr update
  • @roberth: zipAttrsWith performs such a task for the module system and is efficient
  • Summary
    • Requires further investigation to determine impact, regressions, unknowns.
    • Edolstra will work on it (is part of lazy-trees branch)
    • would be better as new branch
    • pros/cons: can be a benfit and a good abstraction / must take care to avoid being leaky

Actual outPath type semantics

  • @roberth still a reason to consider direct path values for the original issue
    • /. + "/thing" creates absolute path, can be made to work with the virtual paths, but is awkward because /. is meant to represent the file system root
    • if (fetchTree x).outPath isn’t a path value, we should add a builtin that returns a path value (e.g. toPathValue), so that we don’t have to use /. to make a path value. We may avoid this.
    • @edolstra: changing outPath type could cause compat issues?
    • fetchTree has few direct users, call-flake.nix can mitigate
    • solutions to different problems which interact

(@fricklerhandwerk brought up governance topics as preparation for the upcoming foundation board call)

1 Like