2022-11-14 Nixpkgs Architecture Team Meeting #17

Development Nixpkgs Architecture
2

Notes

intro

Auto-called packages RFC draft review

proposal: package.nix instead of default.nix, https://github.com/nixpkgs-architecture/auto-called-packages/pull/8

  • Benefit: Can’t accidentally nix-build these paths anymore

    • Leaves room open for a default.nix that can be nix-build’t

  • @Ericson2314:

    • File paths are an implementation detail
    • We can do extra migrations out the kindness of our heart.
    • But we shoudln’t feel “boxed in” by this. If we can’t do as good of a “bonus” thing for downstream coe snooping at unstable file layouts, that’s fine.

  • @roberth: PR’s need to be rebased, lots of them

    • @Ericson2314: PR’s change the implementation, is expected. Tool to automate PR rebases has a lot of flexibilities

  • zooming out, the choice is between

    • package-function/<name>
      • new stuff would be foo/<name>, bar/<name>.
    • <name>/package-function
      • new stuff would be <name>/foo <name>/bar -

  • Concretely:

    • Example: pkgs/auto/hello/package-function.nix
    • Example: auto/hello/package-function.nix ?

  • @Ericson2314: directory should imply structure; avoid exceptions (+1)

    • What do do with bootstrap, build-support, etc?
    • What to do with fetchgit (things that are libraries are not exactly packages)
    • lib?
    • @tomberek: dir structure should be a close match to attr structure
    • during transition will have new/old separation
    • @infinisil: Proposal: Change the RFC to only do the transition for packages, not functions like fetchers and other special values
    • @tomberek: The directory structure / attribute structure mismatch is currently confusing. Might be better to keep it the same

  • might be controversial

  • @growpotkin: Could use default.nix for filling out non-default arguments

    • @roberth: Doesn’t scale to nixpkgs’ scale. Can have default.nix be something like (import ../.. {}).hello instead though
    • @Ericson2314:
      • Agreed we should save default.nix for things that are not footguns that cannot be called. A good general rule.
      • A bit wary about this specific default.nix, because definitions should not know their own name.
      • But perhaps it is less bad because the package directories were already supposed to be 1-1 with attribute names.

File name proposals:

  • @roberth: Something with “fun”/“functions”

  • pkg-fun.nix, same length as default.nix :slight_smile:

  • Other ideas:

    • package-function.nix
    • package-fun.nix
    • package.nix
    • proto-derivation.nix
    • prototype.nix
    • blueprint.nix
    • default.nix
    • pkg.nix

  • @Ericson2314: Naming details shouldn’t block the RFC

Directory root

In nixpkgs root:

  • pkg/<name>/pkg-fun.nix

  • component/<name>/pkg-fun.nix, too long

  • part/<name>/pkg-fun.nix

  • thing/<name>/pkg-fun.nix

  • mod/<name>/pkg-fun.nix, we have modules already

  • pkg/<name>/pkg-fun.nix

  • comp

  • uniform

  • unit

    • +1 @infinisil
    • +1 @growpotkin
    • +1 @roberth
    • @Ericson2314 proposed pkgs/unit/<name>/*.nix
      • @roberth: pkgs not necessary.
      • @roberth: Implementation should include rename tool, shouldn’t do many renames manually
      • @roberth: Doesn’t make sense to move nixos modules into pkgs
      • Not disagreeing with pkgs/unit, for now only for packages
    • unit/<name>/*.nix
    • @infinisil: Just a name
    • @Ericson2314: Good to think ahead, less ossifying, can change things later as well. We’re trying to think ahead, but not set in stone.
    • @growpotkin strictness is flexibility!
      • @Ericson3214: +90000
    • @infinisil: Encouraging more evolution

@growpotkin’s sharding experiment

  • Ran into some performance issues with readDir.
  • Automatically generate single file with all paths, much faster.
    • @infinisil: builtins.listDir could be faster
      • @roberth: This is an implementation detail (and readDir attrValues can be made lazy)
  • 8000 nodejs packages, sharding node packages by scope + base name, plus one-letter sharding for unscoped
  • @Ericson2314: Attribute union should be faster in Nix, currently linear
  • @Ericson2314: Fundamentally, we should do things like sharding because limitations in things outside our control, like the filesystem. When the implementation is in Nix itself we should just fix it.
    • My guess is that sharding is good for the filesystem (and for git’s content addressing), and if Nix doesn’t like it we should fix Nix to get along with it.
  • @growpotkin: Beware of case-insensitive file systems!

outcomes, todos

2 Likes