2024-08-12 Nix team meeting minutes #168

Attendees: @edolstra @roberth @tomberek @L-as

downloadTarball() returns an uncachable accessor · Issue #11271 · NixOS/nix · GitHub

Assigned to @edolstra

Cannot bind and listen to port with macOS sandbox, even with __darwinAllowLocalNetworking · Issue #11269 · NixOS/nix · GitHub

Binding to a port should always work, regardless of this flag.

We don’t have a network namespace, so the port might be taken. This does not seem to be the case. Also the number is >1024 so probably not a privileged port.

Not related to the recent GHSA-q82p-44mg-mgh5 (diff) or GHSA-2ffj-w4mj-pg37 (diff)

Operation not permitted should be EPERM, which is not listed on the darwin bind manpage

Can probably be lifted by configuring the sandbox ((allow network-bind) as suggested). Does this have undesirable side effects?

Which version of macOS is this? It could be a regression caused by Apple.

@L-as: do we have a macOS user in the Nix team? Maybe ask on discourse?
Most of us have access to macOS, but do not use it daily.

Nested attribute merging may or may not work as intended · Issue #11268 · NixOS/nix · GitHub

The code that handles these syntax combinations is fairly complex and would probably need to be refactored in order to support this properly.
Most of the complexity comes from the combinations with dynamic attributes (${name} = v;)

If supporting this properly proves infeasible, we might deprecate some variations, but not remove it, because they’ve been supported for around 2 years.

I don’t know if this is a bug (if so, it should be fixable without any breaking changes) or if this is an as-intended aspect of the language

These restrictions are not intentional. It used to be that explicit attrsets in repeated attrs could not be merged at all. ~2 years ago we’ve started supporting that, but apparently the support is not complete.

Perhaps you could document it, but add a caveat that not all combinations are currently supported.

Would you be interested in improving support for these cases?

error: input compression not recognized (private cache using zstd compression) · Issue #11261 · NixOS/nix · GitHub

We have a test for zstd compression for a binary cache

If this were HTTP, I would ask the following. S3 should be analogous.
What does curl http(s)://your-cache.local/l59dvqj9lbm2n5agfx19hxbm2qlq4w0g.narinfo return?
It should provide you with a nar.zstd filename.
If you download that, is it in the correct file format?

Possibly related: advertise transport encoding in http transfers to by Mic92 · Pull Request #7712 · NixOS/nix · GitHub

Fetching paths from cache.nixos.org gets slow in certain Docker environments · Issue #11258 · NixOS/nix · GitHub

Possibly same cause: slow nix build in devcontainer · Issue #11249 · NixOS/nix · GitHub, but @edolstra could not reproduce.

Could this be libgit2 overhead of the new tarball-cache? In a Docker container we may be dealing with an overhead of many fs overlays, amplifying the overhead (normally on the order of 2× at most?).
This might also make reproducing dependent on the layer complexity of the image?

Hacking

Looked into alternate string representations with @tomberek

  • Symbol-backed string looks promising
  • Other representations tricky to get right; WIP

@roberth has proposed an optimization for attrset update operations, to show an implementation strategy that could also apply to strings, without having to introduce an extra internal value type. Draft PR: WIP: accumulate attrset updates, use k-way merge by roberth · Pull Request #11290 · NixOS/nix · GitHub.
Help wanted if k-way merge and performance optimization sound interesting to you!

2 Likes