Tweag + Nix dev update #24

Hey everyone, a new update from the Tweag Nix team (long overdue since the previous one was nearly a month ago. But Christmas came, and a baby − not the Christmas one, another − and I found myself… busy).

Flakes

  • A common pain for Flake users is that when working on a local flake, Nix would always start by copying the flake contents to the store, which can take a non-trivial amount of time (so that people often end-up falling back to “standard” evaluation when hacking on big repos like nixpkgs). This also caused a number of small user experience issues, like #5425 or #4507.
    @edolstra started changing the evaluation model to remove the need for this first copy but instead filter on-the-fly the files that shouldn’t be picked up (e.g. files ignored/unknown to the vcs system, files ignored by a .flakeignore if the related proposal gets accepted, etc…)
  • A nice thing with Flakes and the new CLI is that there’s now a notion of “project” for Nix.
    @balsoft and @Ravendii helped @tomberek take advantage of that to make Nix commands transparently do what you want when ran from a subdirectory of a flake (nix#5720, fixing nix#4612)

Nix Docker image

@garbas finished the migration of the Nix docker image from the NixOS/docker repository to NixOS/nix. This should make it easier (well, automatic) to get up-to-date docker images for new Nix releases.

He also took the opportunity to switch the build of the docker image from a plain old Dockerfile to pkgs.dockerTools (nix#5453). The context for this is explained in the parent pull-request: docker#31

Linked to that (and an other motivation thereof), it was discovered that Nix 2.5 wasn’t working properly inside a docker image (nix#5777). @edolstra fixed this in nix#5787 and released a 2.5.1 bugfix release.

This new docker image is now referenced as one of the official ways of installing Nix on the new download page (see below)

NixOS webpage

@garbas wrote or merged a number of small fixes to https://nixos.org (nixos-homepage#788, nixos-homepage#787, nixos-homepage#786, nixos-homepage#785, nixos-homepage#773, nixos-homepage#724, nixos-homepage#769, nixos-homepage#658)

He also re-vamped the download page by including some materials from https://nix.dev (nixos-homepage#780)

Performances improvements

@pennae (not from Tweag) demonstrated some wonderful skills in performance C++ programming by tweaking the evaluator code to squeeze a lot small performance issues (nix#5809, nix#5812, nix#5813, nix#5830, nix#5865, nix#5887, nix#5906). @edolstra and regnat spent a fair amount of time carefully reviewing this stream of really pleasant (but demanding) contributions, so that a number of them are now merged and several others are about to.

Misc

Removing the affinity hack

An old Nix issue (nix#3345) was reporting that using nix-shell on a Nix expression involving import-from-derivation was causing the shell to only see one CPU.
This was because the Nix evaluator was temporarily switching itself to always run on the same CPU as the daemon for performance reasons, but the “temporarily” bit was broken when IFD was added to the mix.
While looking at it, @edolstra noticed that the issue wasn’t present on master anymore. More investigations showed that this was because the “stick to one CPU” logic was broken… but that it didn’t have any performance impact anymore.
As a result, he simply removed all that logic, which definitely fixed the issue and cleaned the code (nix#5821)

nix copy logs

@edolstra added a new command to copy the build logs from one machine to another (nix#5929, fixing nix#5222). This feature was sponsored by Flox.

UX tweaks

  • @balsoft made it possible to show the stack trace in the repl (nix#5840)
  • @balsoft with Tomberek worked towards making the nix output more useful in non-interactive settings (nix#5918)
  • Regnat fixed a small glitch here nix why-depends was eating the first line of output (nix#5914)

Bugfixes

  • @yorickVP fixed a small performance issue when loading a big attr set in nix repl (nix#5835)
  • @balsoft fixed an internal incoherence causing different Nix commands to generate a different lockfile (nix#5839)

Nickel

Nickel didn’t see a lot of groundbreaking changes during this period. That’s because @yannham and @aspiwack have been working hard gathering and digesting feedback to freeze an infinity of small (or big for that matter) details before a first release. The bulk of the work is referenced from nickel#462

This didn’t mean of course that nothing was done (the git log speaks for itself on that). Most notably:

  • Ysndr kept making the language server more precise and awesome
  • Francois-caddet worked on pattern destructuring, so that one can write let {x, y} = foo in bar or fun { x, y } => foo
  • Litchipi hacked around the typechecker to both extend it and fix some minor issues

And that’s all, folks

14 Likes

The Docker changes sound great. But now I wonder how the official Docker images differ from GitHub - nix-community/docker-nixpkgs: docker images from nixpkgs [maintainer=@zimbatm].

I had switched to the community image a while back since, as stated in one of the linked PRs above, the official image was Alpine + Nix.

Does that mean someone’s planning to work on that (or already on it)? Asking because my comment seems to be the last one in #4097.

While I started hacking on this a few months ago I never managed to finish this .flakeignore-implementation (and can’t promise I’ll get to it soonish I’m afraid).

1 Like

Not that I know of, no. And actually the implementation might have to be rewritten since removing the copy will probably entail some big changes to that part of the code. So I wouldn’t be too keen on working on it right now :wink:

With Nix version 2.5.0 and onward the official docker image is built with just Nix (no more Alpine).

Currently the plan is to move docker.nix to nixpkgs to be able to use it to generate other images that require docker image with Nix inside. I think that could be a part that would be shared with nix-community/docker-nixpkgs.

The overlap with nix-community/docker-nixpkgs is that nixpkgs/nix and nixpkgs/nix-unstable are a “duplicate” of nixos/nix:latest and nixos/nix:master. I used duplicate in quotes since I’m not sure what is exactly included in nixpkgs/* images by default. The idea of nixos/nix images is to walk the line of being minimal while still useful to majority.

One difference with nix-community/docker-nixpkgs (right now) is that stable releases of Nix come with both amd64 and arm64 flavor. Hopefully we will be able to do the same soon for docker images from master (the arm github runner is missing right now).

But in nix-community/docker-nixpkgs there are other images which could be useful. Additionally to all of this I could see having nixery.nixos.org (or similar) serving on-demand docker images. But I’m not sure if there is enough man power in infra team to do this.

4 Likes