2024-03-04 Nix team meeting minutes #130

thufschmitt · March 5, 2024, 7:41am

Triage

remote instantiation (`NIX_REMOTE`, `--store`) is an order of magnitude slower than local instantiation + `nix copy` · Issue #10119 · NixOS/nix · GitHub

How do we prevent this thing from happening?

@eelco: Construct the derivation graph in memory and stream the whole result to the store?
- Also would solve the issues with read-only mode
- Not a perfect fix because only works for drv files, but already covers the bulk of the problem
Could also make eval-store an option
Maybe also separate build-store
- Right now is build-and-dest-store
- Maybe just make it dest-store
- Add `target-store` and `eval-store` settings · Issue #10145 · NixOS/nix · GitHub
In-memory eval store could be either an overlay store or a normal store

`fetchTree` performs fetch even when a `narHash` is provided · Issue #10104 · NixOS/nix · GitHub

Assigned to @edolstra

@roberth: The fetch cache is quite confusing, stores a whole bunch of different stuff, no schema
@edolstra: It’s a very generic KV store, makes the implementation much easier
@roberth: Might be hiding bugs, might want to be a bit stricter
@roberth: Maybe have a set of methods (of the same form) that make clear what’s the purpose of each “virtual table”
- make sure to separate “pure” caching and “impure” caching

github.com/NixOS/nix

Make the fetcher cache more reliable and easier to reason upon

opened 04:18PM - 04 Mar 24 UTC

thufschmitt

feature fetching

**Is your feature request related to a problem? Please describe.** The fetche…r cache is a more-or-less generic untyped key-value cache. This makes it hard to reason about it, and might hide some tricky bugs. **Describe the solution you'd like** Give some structure, either to the cache itself or to the code that manipulates it - Add some typing to the fields - Add a set of helper methods to manipulate the cache **Describe alternatives you've considered** A clear and concise description of any alternative solutions or features you've considered. **Additional context** Add any other context or screenshots about the feature request here. **Priorities** Add :+1: to [issues you find important](https://github.com/NixOS/nix/issues?q=is%3Aissue+is%3Aopen+sort%3Areactions-%2B1-desc).

Installer bootstrapping issue due to nix 2.20.3 missing from nixpkgs · Issue #10109 · NixOS/nix · GitHub

@thufschmitt: Any way we could avoid that in the future?
@edolstra: In that instance, had to change an element from an array to an object, so hard to do in a forward-compatible way
@roberth: Could have kept both the object and the array under different names, and have the new version be able to handle both
For that, might jq a backwards migration script, although would be tricky to get Nix to rebuild a profile with all the right things

@thufschmitt will add a comment near the version field to warn people who’ll want to change it in the future.

Add a warning against carelessly changing the profile version number by thufschmitt · Pull Request #10155 · NixOS/nix · GitHub

Fix `__darwinAllowLocalNetworking` sandbox by szlend · Pull Request #10078 · NixOS/nix · GitHub

Merged

refactor: move the repl input code to its own file by lf- · Pull Request #10083 · NixOS/nix · GitHub

Looks good in theory, needs careful review

Assigned to @roberth

https://github.com/NixOS/nix/pull/10087

Assigned to @thufschmitt

Use SourcePath for reading flake.{nix,lock} by edolstra · Pull Request #10088 · NixOS/nix · GitHub

Suggestions by @Ericson2314, but can be a follow-up.
Merged

Improve support for subflakes by edolstra · Pull Request #10089 · NixOS/nix · GitHub

Makes subflakes actually usable.

@thufschmitt: Semantic changes?
- @edolstra: Some things are more permissive. Like allowing subflakes to access the parent directory
- @roberth: Too permissive?
- Maybe should only allow that explicitly?
To discuss (and discussed below)

Ignore references when importing signatures by raboof · Pull Request #10101 · NixOS/nix · GitHub

github.com/NixOS/nix

Comment by thufschmitt to Ignore references when importing signatures

NixOS:master ← raboof:ignore-references-when-importing-signatures

Discussed during the Nix maintainers meeting on 2024-03-04. We don't want to ju…st remove the check as it's a useful one for normal users. Three ways from there: - Store the needed information in your db (indeed somewhat wasteful, but I also see that pretty useful for debugging the source of descrepancies) - Work around the check by first getting the narSize and references from the local store (with `nix path-info --json` or equivalent) - Add a flag to disable these checks. But a bit ugly. <details> - This sanity check is useful to have - References can be different because of different builders configuration (`__discardReferences` not supported, or possibly some bug/improvement) - Can also get these infos from `nix path-info` - This is probably called from a script any way, so acceptable </details>

improve error reports somewhat by pennae · Pull Request #9874 · NixOS/nix · GitHub

Assigned to @thufschmitt.
Will wait at least until after the release (early next week) to merge as it can be a disruptive change.

Forbid nested debuggers by 9999years · Pull Request #9920 · NixOS/nix · GitHub

Assigned to @thufschmitt, can merge once CI is green

feat: small-string optimization by tomberek · Pull Request #9895 · NixOS/nix · GitHub

Waiting for when Tom is back

Remove the stack overflow handler by edolstra · Pull Request #9878 · NixOS/nix · GitHub

Might be too early to remove it.
Not having it on Darwin is annoying

Could it be replaced by Instrumentation Options (Using the GNU Compiler Collection (GCC))?

Discussion

Add nixos-unstable based outputs, bdwgc 8.2.4, Nixpkgs lib tests, cleanup by roberth · Pull Request #9900 · NixOS/nix · GitHub

github.com/NixOS/nix

Comment by thufschmitt to Add nixos-unstable based outputs, bdwgc 8.2.4, Nixpkgs lib tests, cleanup

NixOS:master ← hercules-ci:nixpkgs-lib-tests

Discussed during the Nix maintainers meeting on 2024-03-04. We won't track `nix…os-unstable` as the primary target, but we also want to test against it. Opened https://github.com/NixOS/nix/issues/10156 for that. <details> - Need to have a job that builds against `nixos-unstable` so that we don't spend three months updating to the latest release against - Can be a side job, not the main one - A bit tricky because hydra doesn't have `--override-input`. But will find a way </details>

Improve support for subflakes by edolstra · Pull Request #10089 · NixOS/nix · GitHub

github.com/NixOS/nix

Comment by thufschmitt to Improve support for subflakes

NixOS:master ← edolstra:relative-flakes

Discussed during the Nix maintainers meeting on 2024-03-04. - Idea approved - …Document that directly accessing `..` in subflakes is likely to break in the future - Use path literals as the url for subflakes <details> - Great overall - Right now, the PR allows `import ../.` in a subflake. Do we want that? - Directly accessing `..` is convenient, but breaks the encapsulation property of flakes - Relates to https://github.com/NixOS/nix/issues/4414 - We can however access other parts of the tree with relative inputs (that can go up until the root) - `inputs.something_else = ../foo` - From an implementation perspective, hard to forbid without lazy trees - Decision: - Don't formally forbid direct access to `..` since it's hard without lazy trees - Once lazy trees lands, make that conditional on a flag - Potentially make the default restrictive. Would be a breaking change, but - can be mitigated by only making that change effective after a `flake update` - small breakage as getting the old behavior back would only be a one-line change - Syntax for relative flakes? - Currently reuses `url = "path:./foo"` that already kinda works on master - A more natural syntax would be `url = ./foo` - This is actually what most people seem to try first when they want subflakes </details>