Running nixos-rebuild switch locally on a non-nix computer?

Hey!

I discovered that you can locally run nixos-rebuild switch on a non nix computer

nix-shell -p '(nixos{}).nixos-rebuild'

I am trying to figure out how I can levarage this to rebuild my entire os locally on a non-nixos computer and deploy it.

When I run nixos-rebuild --target-host root@machine switch -I nixos-config=my-folder, I get the error

building the system configuration...
warning: unknown experimental feature 'allow-import-from-derivation'
error:
       … while evaluating the attribute 'config.system.build.toplevel'

         at /nix/store/h0wjwsm9qzidk61yh5d17arr1a2rcf8d-nixpkgs/nixpkgs/lib/modules.nix:322:9:

          321|         options = checked options;
          322|         config = checked (removeAttrs config [ "_module" ]);
             |         ^
          323|         _module = checked (config._module);

       … while calling the 'seq' builtin

         at /nix/store/h0wjwsm9qzidk61yh5d17arr1a2rcf8d-nixpkgs/nixpkgs/lib/modules.nix:322:18:

          321|         options = checked options;
          322|         config = checked (removeAttrs config [ "_module" ]);
             |                  ^
          323|         _module = checked (config._module);

       (stack trace truncated; use '--show-trace' to show the full trace)

       error: opening file '/myconfig/default.nix': No such file or directory

I’m curious what I am missing here?

Two things jump out to me:

  1. You’re setting nixos-config to a directory. All of the examples I can see in the nixos directory of nixpkgs set nixos-config to a file which is a valid nixos module (for example, /etc/nixos/configuration.nix).
  2. The error message says the missing file is /myconfig/default.nix, which seems entirely unrelated to my-folder (what you’ve set nixos-config to). Is there a typo/copy-paste error here somewhere? Do any of these directories exist in the directory in which you’re calling nixos-rebuild? It’s unclear to me how/why nix would be looking under /myconfig, unless you’re running this from /.

Two things jump out to me:

  1. You’re setting nixos-config to a directory. All of the examples I can see in the nixos directory of nixpkgs set nixos-config to a file which is a valid nixos module (for example, /etc/nixos/configuration.nix).
  2. The error message says the missing file is /myconfig/default.nix, which seems entirely unrelated to my-folder (what you’ve set nixos-config to). Is there a typo/copy-paste error here somewhere? Do any of these directories exist in the directory in which you’re calling nixos-rebuild? It’s unclear to me how/why nix would be looking under /myconfig, unless you’re running this from /.

Hey! Thanks for the response.

I gave it another spin pointing to configuration.nix instead, and it did work better! Though I’m still getting errors

This time inputs seems to be missing, but I was able to remedy that by adding --flake . and it almost worked!

[nix-shell:~/builds/nixos]$ nixos-rebuild --target-host root@mycomp -I nixos-config=./configuration.nix dry-build --flake .

It runs

building the system configuration...
warning: unknown experimental feature 'allow-import-from-derivation'
these 365 derivations will be built:
...
warning: unknown experimental feature 'allow-import-from-derivation'
error: getting status of '/run/user/1000/nixos-rebuild.MTJQUR/result': No such file or directory

So it seems to struggle to store the result?

Hm… If your configuration is a flake, as --flake . would imply, I don’t think you need to supply -I nixos-config=... — your flake should reference the configuration directly (though I believe adding this path would, at worst, do nothing). Is it possible for you to share the configuration you’re trying to build? Regardless, can you share the version of nix and nixpkgs you’re using, as well as the contents of your system and user nix.conf files (see man nix.conf for the locations you would need to look for)? I find it strange that you’re getting warnings about 'allow-import-from-derivation' being an unknown experimental feature. That’s not an experimental feature in my version of nix (2.18.1), and it defaults to true.

Hm… If your configuration is a flake, as --flake . would imply, I don’t think you need to supply -I nixos-config=... — your flake should reference the configuration directly (though I believe adding this path would, at worst, do nothing). Is it possible for you to share the configuration you’re trying to build? Regardless, can you share the version of nix and nixpkgs you’re using, as well as the contents of your system and user nix.conf files (see man nix.conf for the locations you would need to look for)? I find it strange that you’re getting warnings about 'allow-import-from-derivation' being an unknown experimental feature. That’s not an experimental feature in my version of nix (2.18.1), and it defaults to true.

I tried upgrading nix (nix --version = 2.18.1), and it seems you are correct that --flake . is is all that is needed.

I get the same error though. Here is my nix.conf

experimental-features = nix-command flakes allow-import-from-derivation

In my flake I have the following input for my nixpkgs

  inputs.nixpkgs.url = "github:NixOS/nixpkgs?ref=nixpkgs-unstable";

I updated the flake lock the other day so it’s probably quite recent?

I also want to note that I tried just running nixos-rebuild dry-build --flake . and then it works without error. It only errors out if I add --target-host root@myserver.

Update here, I tried to just run it and despite the warning it seems to work just fine in practice. When I run it with nixos-rebuild switch --flake . --target-host X, it runs without issue. Very cool!

It’s a bit odd that on either side of the build, the result out-link is being written/read from a /run/user/1000 dir. But I’m guessing that’s just that the other distro has temp files configured differently.

But then, that makes me wonder if potentially there’s a “lingering” issue with whatever is getting written to /run/user/1000 getting deleted on ssh disconnect / linger-log-out, etc. (It’s handwavey, but throwing it out as a rough guess-idea)

Note that some third party tools like deploy-rs are capable of remote deployments from non-NixOS hosts.

Might be easier just to use one of them instead of trying to force nixos-rebuild to do something it’s not really intended for.

Utimately deployment just means running a copy-closure, ssh-ing and running am activation script though. Not too surprising that nixos-rebuild doesn’t have any host prerequisites for that.

Note that some third party tools like deploy-rs are capable of remote deployments from non-NixOS hosts.

Might be easier just to use one of them instead of trying to force nixos-rebuild to do something it’s not really intended for.

Hmm, I guess so. Though that makes me wonder, what is the intention of nixos-rebuild --remote-target if it is not to build locally and deploy the change?

The intention is to do that, but “locally” is expected to be on a NixOS host. Admittedly this is unlikely to ever become a problem, though, so I suppose whatever floats your boat.

1 Like