2024-11-20 Nix team meeting minutes #196

roberth · November 27, 2024, 8:00pm

Wednesday triage meeting

Attendees: @edolstra, @tomberek, @ericson2314, @mschwaig

`builtins.mergeMapAttrs` (or `builtins.concatMapAttrs`) · Issue #11887 · NixOS/nix · GitHub

Idea approved in principle. Benchmarking can see if it’s worth it.

github.com/NixOS/nix

`builtins.mergeMapAttrs` (or `builtins.concatMapAttrs`)

opened 01:25PM - 15 Nov 24 UTC

roberth

feature language performance idea approved

**Is your feature request related to a problem? Please describe.** Nixpkgs `l…ib` defines `concatMapAttrs f attrs`, which is a very versatile function. (To those it may concern: by virtue of being almost the monadic bind operation on attrsets. See [here](https://github.com/NixOS/nixpkgs/pull/201527#discussion_r1025207834). Almost => no associativity due to attribute name collisions iirc) **Describe the solution you'd like** Add to `builtins` a primop that behaves like Nixpkgs' `lib.concatMapAttrs`. It could be `builtins.concatMapAttrs` or a different name, such as `flatMap` attrs or `mergeMapAttrs`. Its implementation will be slightly more efficient than that in `lib`, and `lib` may reuse it, so that it becomes a polyfill. The added efficiency comes from having no intermediate Nix lists, and may build the attrset in one go. The current implementation in `lib` has the following behaviors, which could be optimized away (in order of increasing implementation work): 1. performs more function calls (significant, since we're an interpreter, and our calls perform allocations) 2. creates and sorts intermediate attrsets after each `f` call (except the final one which is the return value) 3. creates and sorts intermediate attrsets returned by `f` - (1) and (2) a trivially solved by implementing this in C++. - (3) may feel unavoidable, but can be improved with https://github.com/NixOS/nix/pull/11290 - or a variation that streams an insertion sort, "accumulating" individual attrs, rather than a bunch of attrsets. - this is a _nice to have_, because (1) and (2) are sufficient reason to implement the builtin. If we pick a new name, we could make attribute name **collisions** an **error**, which iirc would make it a lawful monad in all the successful cases. Errors are better than buggy behavior somewhere deep down in some expression, because how would you find the bug. **Describe alternatives you've considered** - Perhaps make it `flatMapAttrs` or `mergeMapAttrs`. `concat` kinda works by extrapolating from lists, but I'm not super happy about that. - `mergeMapAttrsWith (name: values: merge name values) f attrs` to handle the collisions. This is like `zipAttrsWith (name: values: if length values == 1 then head values else m name values)` but without allocating a whole bunch of singleton lists. - I feel like this isn't sufficiently distinct to warrant a separate builtin - => describe the connection in the docs instead. **Additional context** - https://github.com/NixOS/nixpkgs/pull/201527 **Priorities** Add :+1: to [issues you find important](https://github.com/NixOS/nix/issues?q=is%3Aissue+is%3Aopen+sort%3Areactions-%2B1-desc).

`--extra-experimental-features` must come before other arguments as of Nix 2.19 · Issue #11891 · NixOS/nix · GitHub

Closed issue

github.com/NixOS/nix

`--extra-experimental-features` must come before other arguments as of Nix 2.19

opened 05:41AM - 17 Nov 24 UTC

closed 09:41PM - 20 Nov 24 UTC

Enzime

bug regression settings

As of https://github.com/NixOS/nix/commit/89b39520630726b7c48e8cbc065643134352e4…3d, `--extra-experimental-features "nix-command flakes"` must come before `--override-input`: ``` $ nix run github:NixOS/nix/89b39520630726b7c48e8cbc065643134352e43d \ --extra-experimental-features "nix-command flakes" \ -- flake metadata github:edolstra/flake-compat \ --override-input nixpkgs nixpkgs/nixpkgs-unstable \ --extra-experimental-features "nix-command flakes" error: experimental Nix feature 'flakes' is disabled; add '--extra-experimental-features flakes' to enable it ``` ``` $ nix run github:NixOS/nix/89b39520630726b7c48e8cbc065643134352e43d \ --extra-experimental-features "nix-command flakes" \ -- flake metadata github:edolstra/flake-compat \ --extra-experimental-features "nix-command flakes" \ --override-input nixpkgs nixpkgs/nixpkgs-unstable warning: input '' has an override for a non-existent input 'nixpkgs' warning: the flag '--override-input nixpkgs flake:nixpkgs/nixpkgs-unstable' does not match any input Resolved URL: github:edolstra/flake-compat Locked URL: github:edolstra/flake-compat/0f9255e01c2351cc7d116c072cb317785dd33b33 Description: Allow flakes to be used with Nix < 2.4 Path: /nix/store/8kpx53qi52yhjai1vdw8zpa95iqa61bv-source Revision: 0f9255e01c2351cc7d116c072cb317785dd33b33 Last modified: 2023-10-05 00:37:54 ``` This is a regression as in the previous commit https://github.com/NixOS/nix/commit/bfe1308d3ff7f3dfd4facf4318adcb459e57ece9 this works fine: ``` $ nix run github:NixOS/nix/bfe1308d3ff7f3dfd4facf4318adcb459e57ece9 \ --extra-experimental-features "nix-command flakes" \ -- flake metadata github:edolstra/flake-compat \ --override-input nixpkgs nixpkgs/nixpkgs-unstable \ --extra-experimental-features "nix-command flakes" warning: input '' has an override for a non-existent input 'nixpkgs' warning: the flag '--override-input nixpkgs flake:nixpkgs/nixpkgs-unstable' does not match any input Resolved URL: github:edolstra/flake-compat Locked URL: github:edolstra/flake-compat/0f9255e01c2351cc7d116c072cb317785dd33b33 Description: Allow flakes to be used with Nix < 2.4 Path: /nix/store/8kpx53qi52yhjai1vdw8zpa95iqa61bv-source Revision: 0f9255e01c2351cc7d116c072cb317785dd33b33 Last modified: 2023-10-05 00:37:54 ``` **Steps to reproduce** 1. Ensure that `experimental-features = nix-command flakes` is not set in `/etc/nix/nix.conf` and `~/.config/nix/nix.conf` 2. Run the commands above **Priorities** Add :+1: to [issues you find important](https://github.com/NixOS/nix/issues?q=is%3Aissue+is%3Aopen+sort%3Areactions-%2B1-desc). cc @Ericson2314

@ericson2314: Oops, I realized I misunderstood this and my previous concerns were not relevant. Support fine-grained database schema migrations by edolstra · Pull Request #11668 · NixOS/nix · GitHub approved.

2024-11-20 Nix team meeting minutes #196

`builtins.mergeMapAttrs` (or `builtins.concatMapAttrs`) · Issue #11887 · NixOS/nix · GitHub

`--extra-experimental-features` must come before other arguments as of Nix 2.19 · Issue #11891 · NixOS/nix · GitHub

Make @nix json structured build log parsing warn instead of fail by roberth · Pull Request #11921 · NixOS/nix · GitHub

Merkle verification of non-content-addressed data · Issue #11919 · NixOS/nix · GitHub

Support fine-grained database schema migrations by edolstra · Pull Request #11668 · NixOS/nix · GitHub