Using Nix with optional build hosts

Help
1

I have two laptops (and maybe more hosts in the future) with a desktop all running NixOS. Both laptops will cook themselves when doing builds, so using remote building against the desktop would be nice if available. But these laptops will be in the wild and I’ll need to grab/build something at the spot sometimes, which means I won’t have a build host and I’ll have to awkwardly remove /etc/nix/machines and modify /etc/nix/nix.conf if I can’t nixos-rebuild with my remote build config commented out.

I tried searching before and people alleged that it’s possible, but then not provide any resources or examples of the configuration.

Anyways, my config that works within the home network, configured for distributed build and allow my build-host to serve builds it already completed (since they share common configs):

{ config, pkgs, lib, ... }:

let
  gammax = "gammax.domain.local";
in
{
  nix.buildMachines = [ {
    hostName = "${gammax}";
    systems = [ "x86_64-linux" "i686-linux" ];
    protocol = "ssh-ng";
    maxJobs = 32;
    speedFactor = 2;
    supportedFeatures = [ "nixos-test" "big-parallel" "kvm" ];
    mandatoryFeatures = [ ];
  }] ;

  nix.distributedBuilds = true;

  nix.extraOptions = ''
    builders-use-substitutes = true
  '';
  
  nix.settings = {
    trusted-substituters = lib.mkBefore [ "ssh-ng://${gammax}?priority=1" "https://cache.nixos.org?priority=50"];
    trusted-public-keys = lib.mkBefore [ "[gammax-public-key…]" ];
    substituters = lib.mkBefore [ "ssh-ng://${gammax}" ];
  };
  
}
2

Try adding --option builders ""

3

I use aggressive timeouts in case a builder is down, not the best solution, but better than just waiting for nothing

  programs.ssh.extraConfig = ''
    Host ${gammax}
      ConnectTimeout 3
  '';
4

At some point I used a builder configured as 127.0.0.1:22001 or whatever, with an SSH tunnel to 192.168.0.10 forwarding this port to remote 127.0.0.1:22. No builder → instant connection refused.

A general solution for «I want two setups always available» (e.g. with/without remote builders) is NixOS specialisation

5

I had the exact same problem…
I’ven just been writing --builders "" which to be honest is not great.

I almost want something like “profiles” rather than specifying individual builders and maybe Nix could fan-out try all build machines + locally so you always make progress even one has a long timeout.