Rebuild NixOS Offline

1

I use /etc/nixos/configuration.nix to mange my networks. The problem I have run into is that to connect to a new network after I add it to /etc/nixos/configuration.nix, I have to run nixos-rebuild switch, but this requires a network connection to run properly. Is there a way to rebuild nixos offine?

7 Likes
2

in general case, no, there isn’t. The easiest is to boot from installation drive (assuming you have it burned somewhere), where internet works and reinstall using existing configuration.nix.

If the only thing you do is change network (like, choose another WiFi network), it shouldn’t use network for rebuild. But there can be situations:

  • you may have updated channels, but didn’t rebuild system. Then (when network is off) you try to rebuild system with new config, but it now wants to redownload stuff, thus fails
  • you have impure derivations (import from remote derivation, fetchTarball without hash, etc), where caches do expire and you have to download stuff
  • you have import from derivation, but have run GC and build inputs for that IFD have to be redownloaded

So, the best you can do with this is NOT to manage network entirely declarative. New network connection should be possible to setup in CLI (and you should know “how”). I use NetworkManager and nmtui command, which helps a lot when I forget about different wpa_supplicant flags.

1 Like
3

Besides the fact that it is obviously dangerous to remotely update the network on a remote machine in case of configuration typos and mistakes, here are the steps I take to make it work (when I do not forget…).

  1. Run nixos-rebuild build to check that it actually builds correctly
  2. Enable persistent user instances in systemd [1].
  3. Run nixos-rebuild {switch/test/boot} inside a screen (or tmux ?) session to keep it running after the ssh session ends.
  4. remove the ./result symlink left by nixos-rebuild build

For a long discussion of users caught by the same issue, read [1] further :smiley: .

(1) https://github.com/NixOS/nixpkgs/issues/3702

PS: Running nixos-rebuild in a persistent session should have been implemented years ago. We are still waiting for someone to write the patch :wink:

4 Likes
4

I use /etc/nixos/configuration.nix to mange my networks. The problem
I have run into is that to connect to a new network after I add it to
/etc/nixos/configuration.nix, I have to run nixos-rebuild switch,
but this requires a network connection to run properly. Is there a way
to rebuild nixos offine?

If I understand correctly, then I think you want

nixos-rebuild switch --option substitute false

This will stop it trying to look up what paths are available on a
substituter, which should allow it to rebuild without an internet
connection as long as you have all of the dependencies and stuff.

18 Likes
5

Not the answer you want by if you want to manage network with nixos- I believe wpa_supplicant, iwd and nm all have ways to change wifi network without you having to rebuild . Also I just save multiple networks’ password on nixos config, the wpa_suppplicant option allows that. networking.wireless.networks.<ssidName>={psk=<password>;}

2 Likes
6

Easiest way I have found to rebuild locally: nixos-rebuild switch --option binary-caches “”

IMPORTANT: This will only work if you didn’t add any new packages since your last build. As soon as you add a new package, there’s no way around having internet.

Source: Manual networking on NixOS install ¡ Issue #26186 ¡ NixOS/nixpkgs ¡ GitHub

1 Like
7

You say there is no other way around it. Maybe carrying a copy of all the source tarballs.

Maybe space usage would be impractical, and getting nixos to use these files ‘off line’ for an say an external drive or storage device might be tricky.

Then again, I’m seeing second user slow 2TB spinning drives for sale for < $30 , so maybe it’s practical now.

1 Like
8

This will only work if you didn’t add any new packages since your last build.

Does it? I can’t rebuild my system even with an identical configuration.
When the caches are cold Nix always tries to fetch these packages:

c-ares-1.17.2
curl-7.79.1
libkrb5-1.18
libssh2-1.10.0
nghttp2-1.43.0
openssl-1.1.1m

which seem to be dependencies of curl. I’m confused because
nix why-depends --all /run/current-system on any of these says there is no dependency.

EDIT: I figured these are native build inputs of a fetchzip derivation I have.

It seems the core issue is that Nix tries to rebuild some (fixed output?) derivation X which needs Y at build time, however Y is not available and hasn’t been cached.
Can we just not rebuild X in this case?

2 Likes
9

To summarize the topic, either of the two options passed to nixos-rebuild switch --flake will allow to rebuilt the system offline offline, given all the dependencies are already present in the Nix store

  • --option substitute false
  • --option binary-caches ""
1 Like
10

There’s also --offline I believe.

11

FWIW, system.includeBuildDependencies is (now) a thing on NixOS 23.05 and unstable.

It keeps all your build-time closures, so you can rebuild your system as long as you don’t introduce totally new components (sources).

8 Likes
12

system.includeBuildDependencies works, but it’s crazy expensive: it adds several GB of dependencies for even a pretty minimal headless system, so pretty much unfeasible.

I really don’t understand why Nix couldn’t just reproduce the same system build, again, given nothing changed in either configuration.nix or the channel. This is extremely annoying.

2 Likes
13

It always irritated me that NixOS cannot be reconfigured offline, like, sometimes I have to change some configuration while on a train and it takes forever.

I get the feeling that this is one of those cases where we started taking for granted the constant availability of a network connection without thinking it through… Isn’t this also the issue that has led to the custom solution presented in the Nix in Space thread?

6 Likes
14

NixOS can be reconfigured offline though? Some Nix commands will accept --offline, and for others you can use --option substituters "" to disable checking the cache.

1 Like
15

Those don’t work, see the comments above for the problem. Also note that --offline is part of the experimental Nix CLI and is not accepted by nixos-rebuild.

16

It sounds like you’re garbage collecting a build-only time dependencies between rebuilds. Nix doesn’t keep build-only time dependencies alive by default. You can fix this with keep-outputs = true in nix.conf. This works because the system closure keeps the derivation closure alive (meaning the .drv files for build time deps), but the derivations don’t keep the build time outputs alive without keep-outputs = true.

17

I didn’t know about keep-outputs, I’m not sure I understand how it compare to system.includeBuildDependencies. If it won’t cause to download additional dependencies (like sources for every single package installed) it may be what I’m looking for. I wonder why it’s not the default…

18

It’s sort of a lite version of system.includeBuildDependencies, in that it doesn’t make them part of the closure; it just keeps them from being GC’d. So it doesn’t force things all the way back to the bootstrap compiler to be on your system; it just keeps them alive if they’re already there. And that’s why it isn’t the default. It has the potential to keep a very excessive amount of paths alive from the GC.

1 Like
19

And I assume I can’t enable it just for one build, right?

20

It’s not a “build” thing. It’s a GC thing. So no.