Determinate Nix 3.8.4: introducing the native Linux builder for macOS

Links
1
12 Likes
2

How does your builder differs from nix-darwin/modules/nix/linux-builder.nix at e04a388232d9a6ba56967ce5b53a8a6f713cdfcf · nix-darwin/nix-darwin · GitHub ?

1 Like
3

It differs in many ways. Most importantly:

  • The Linux builder in Nixpkgs needs to be manually started and stopped and you need to provide proper configuration for it in your nix.conf. The nix-darwin module does make it easier but it’s still something that it’d be tricky to get an entire team or org up and running with. With Determinate Nix, the configuration is handled for you and there’s no manual starting/stopping of any VM. You could trivially have 100 engineers on macOS laptops building Linux stuff in minutes.
  • Determinate Nix uses some pretty straightforward Swift code to interact with macOS’s native Virtualization framework, which runs builds in lightweight VMs, whereas the Nixpkgs Linux builder runs a full-fledged NixOS VM.

The Linux builder in Nixpkgs is great and I’ve used it a ton personally. But we wanted to take the usability story to a qualitatively different level and I think we’ve achieved that.

7 Likes
4

A couple other things:

  1. It uses virtio for the filesystem, so there is absolutely no copying to a remote builder. That also means there is no GC that needs to run, or long running remote process with its own state that needs management.
  2. It doesn’t require SSH keys, configuring IPs, managing a machines file, etc. It’s just … there! Ready to work.
7 Likes
5

Awesome! When do we expect this to be ready?

Also, any plans to make this available in nix-darwin without determinate nix? Asking because Nix Darwin is not fully compatible with Determinate Nix per the documentation here: GitHub - nix-darwin/nix-darwin: Manage your macOS using Nix

2 Likes
6

It’s ready now and we’re going to begin rolling it out to users, as described in the post.

And Determinate Nix is definitely compatible with nix-darwin! GitHub - DeterminateSystems/determinate: Determinate is Nix for the enterprise. An end-to-end experience of using Nix, from installation to collaboration to deployment.

To answer your question, it’s not clear to me how this functionality could be moved out of Determinate Nix.

7

You might want to have a look at GitHub - cpick/nix-rosetta-builder (can automatically start and stop Linux builder on demand, also supports Rosetta for x86_64-linux builds) or GitHub - quinneden/virby-nix-darwin: A vfkit-based linux builder for Nix-darwin (similar to nix-rosetta-builder but based on vfkit/Virtualization.framework instead of lima/qemu)

2 Likes
8

Good stuff! Does this work with non-Flakes Nix derivations, e.g. nix-build -A someLinuxDerivation ?

9

We intend to deprecate nix-build in Determinate Nix but I verified just now on macOS that yes, it does work.

cd nixpkgs
sudo nix-build -A jq --system x86_64-linux --option substitute false --option binary-caches ""
./result-bin/bin/jq
zsh: exec format error: ./result-bin/bin/jq
10

Sorry, lucperkins, would you mind adding additional details? Is this planned deprecation of nix-build in favor of nix build --file? Is this a general plan in Determinate Nix to deprecate anything that can be replaced via features that would have previously been gated under the nix-command flag?

1 Like
11

You can read more about it here:

We’re deprecating all of the nix-* commands in favor of the unified Nix CLI, which we believe is stable.

12

Got it, thanks for the clarification

1 Like
13

Thanks! Hadn’t seen the latter, but had built a very similar thing myself a while ago GitHub - phaer/nixos-vm-on-macos: Run #NixOS on darwin, including Rosetta 2 & virtiofs! :slight_smile:

14

Then do you know why the official nix darwin instructions state not to use Determinate Nix? This is a bit confusing. Perhaps you could work with the nix darwin maintainers to clarify the instructions

  • When run with the --determinateflag, it will install the Determinatedistribution. As Determinate manages the Nix installation itself, you will need to set nix.enable = false; in your configuration to disable nix-darwin’s own Nix management. Some nix-darwin functionality that relies on managing the Nix installation, like the nix.* options to adjust Nix settings or configure a Linux builder, will be unavailable.
1 Like
15

The nix-darwin project isn’t saying not to use Determinate Nix. What that means is that if you do use Determinate Nix, you need to set nix.enable = false in your nix-darwin configuration so that Determinate Nix can handle your nix.conf configuration for you. If you need to provide custom configs with Determinate Nix, you can do so in /etc/nix/nix.custom.conf.

1 Like
16

Thanks for clarifying!

BTW, is the native linux-builder still in developer preview and/or when will it be generally available?

1 Like
17

It’s still in developer preview but you can find instructions for early access here. We’re working through some issues and limitations with it so no specific timeline for GA but it’s coming along nicely!

1 Like
18

I’m trying to use native linux builder in determinate nix. I can run nix build --system x86_64-linux --option substitute false --option binary-caches “” nixpkgs#legacyPackages.x86_64-linux.jq and get an ELF executable that predictably fails to run on macos.

However I’m not able to build a nixos configuration from a flake. When running nix build .#nixosConfigurations.myhostname.config.system.build.toplevel I get errors like:

error: Cannot build '/nix/store/saypdak2k45zyl5wgbl3lcw5syi30mvg-nixos.conf.drv'.
       Reason: required system or feature not available
       Required system: 'aarch64-linux' with features {}
       Current system: 'aarch64-darwin' with features {apple-virt, benchmark, big-parallel, nixos-test}
19

how can I find out if I’m running this new builder?