Nixpkgs Architecture Team Meeting #23
- Past meeting notes
- Recording
- Matrix announcement
- Recorded by: @infinisil
- Lead by: @infinisil
- Meeting notes by: @Ericson2314
- Present members: @infinisil @tomberek @growpotkin @Ericson2314 @roberth
Notes
-
- Alex checked on macOS, also seems fine there
- Darwin and Linux both batch and include the file type in the read dir entry — perf is good.
- other issue might have been due to unrelated thing
- e.g. accidental coppying to the store.
- The PR for lazy
stat
-ing no longer need block simple package paths, or a Nixpkgs arch team concern, but it is still a fine PR Alex/John/Robert/etc. can persue separately.
- Alex checked on macOS, also seems fine there
-
Sharding questions
- Should measure
- Also listing file vs dir measure
- worst case, we can keep a file like all-packages.nix, but make it machine readable / machine editted / completely uniform.
-
Discuss Sharding and other updates by infinisil · Pull Request #20 · nixpkgs-architecture/simple-package-paths · GitHub a bit
- detailed design is more declarative
- questions on what should be CI
- exact rules about refined, “shadowing unit files by accident” is bad, but overriding them (e.g. for bootstrapping) fine.
- Don’t need to separately worry about
pkg-fun.nix
callPackage
-ing.
- John wrote a new alternative to demonstrate our overall process/methodology
-
For discussion
Action items
-
everbody: Go through the entire RFC draft in GitHub - nixpkgs-architecture/simple-package-paths: Nix RFC draft for auto-calling packages in nixpkgs and make a PR for anything that needs changing
-
@growpotkin to test Test `all-packages.nix`-like performance to `builtins.readDir` with and without sharding · Issue #23 · nixpkgs-architecture/simple-package-paths · GitHub
-
@infinisil will continue writing the tooling in Add a simple tool to print Nix path references by infinisil · Pull Request #22 · nixpkgs-architecture/simple-package-paths · GitHub
-
@Ericson2314 will talk to the Cargo people on Zulip about sharding. @infinisil will have an account so @Ericson2314 can ping and not be single point of failure.
- Ask git too, maybe. They do for
.git/objects
.-
@growpotkin: I’d be shocked if GitHub knew implementation details of
git
objects.
-
@growpotkin: I’d be shocked if GitHub knew implementation details of
- Ask git too, maybe. They do for
Unit CI Audits
This should get moved to a repo or something.
Just a draft for some CI checks for unit dirs.
#! /usr/bin/bash
set -eu;
set -o pipefail;
# All Shard Dirs must be 1-4 chars.
if [[ 0 -lt "$(
find ./unit -mindepth 1 -maxdepth 1 -type d -print \
|grep -qv '^[a-z][a-zA-Z0-9_-]\{1,4\}$' \
|wc -l;
)"
]]; then
echo "You blew it" >&2;
exit 1;
fi
# All members of shard dir must have prefix.
for d in $( find ./unit -maxdepth 1 -type d -print; ); do
if [[ 0 -lt "$(
find "$d" -mindepth 1 -maxdepth 1 -type d -print \
|grep -qv "${d//./\\.}.*" \
|wc -l;
)"
]]; then
echo "You blew it in shard dir: $d" >&2;
exit 2;
fi
done
# Eval must succeed without referring to `../` paths.
for p in $( find ./unit -type f -name pkg-fun.nix -print; ); do
# Something like:
nix eval --show-trace -f <( cat "$p"; ) --apply 'pf: let
nixpkgs = builtins.getFlake "nixpkgs";
system = builtins.currentSystem;
pkgsFor = nixpkgs.legacyPackages.${system}
in builtins.seq ( pkgsFor.callPackage pf {} ).drvAttrs true
';
done