I’m trying to figure out how to build a system for my raspberry pi, using my x86 nixos laptop. I found this: NixOS on ARM - NixOS Wiki, which mentions this command:
$ nix-build '<nixpkgs/nixos>' -A config.system.build.sdImage -I nixos-config=./sd-image.nix --argstr system aarch64-linux
I think that the -I nixos-config=./sd-image.nix makes sense - I substitue this filename (sd-image.nix) with the configuration.nix that I currently have on my raspberry pi. The -A config.system.build.sdImage part however, I don’t understand. What is config.system.build.sdImage?
What I’m doing right now is to try the command without that part. Here are the files I currently have:
configuration.nix
hardware-configuration.nix
Both of these are copied from my raspberry pi over to my nixos laptop. I then try to build this with the command nix-build '<nixpkgs/nixos>' -I nixos-config=./configuration.nix --argstr system aarch64-linux. This seems to work at first, but fails in the end. Here’s the output:
building '/nix/store/g07m9j525g2lakhihzxvwsbfsl9lqkm8-etc.drv'...
building '/nix/store/7k3vjrsnp2cg22l13p8w6xiymqny1jkw-nixos-system-nixosraspberrypi42-22.05.3620.e06bd4b64bb.drv'...
building '/nix/store/islqwi6qrb088bnf7n5m4p8b6rsibz6a-nixos-system-nixosraspberrypi42-22.05.3620.e06bd4b64bb.drv'...
building '/nix/store/3cmgrakr934lsig156593jhphzfl8wkk-nixos-boot-disk.drv'...
Formatting '/nix/store/208iz3lqq3q320n216cpyil094qzk325-nixos-boot-disk/disk.img', fmt=qcow2 cluster_size=65536 extended_l2=off compression_type=zlib size=62914560 lazy_refcounts=off refcount_bits=16
building '/nix/store/0bq3588y05g8mjjqga0848gwnyspvmzv-closure-info.drv'...
kvm version too old
qemu-system-aarch64: failed to initialize kvm: Function not implemented
qemu-system-aarch64: falling back to tcg
building '/nix/store/fwzcx7vz914cqsg7j8bawaf48kplj24j-run-nixos-vm.drv'...
building '/nix/store/8r7sgwiai4kfb4dp0gpw0xiffv3yzklq-nixos-vm.drv'...
loading kernel modules...
[ 62.150509] Invalid ELF header magic: != ELF
[ 69.114861] Invalid ELF header magic: != ELF
mounting Nix store...
mounting host's temporary directory...
starting stage 2 (/nix/store/xpq2c17sncjmhnxknyiwcr397q4i04rv-vm-run-stage2)
Creating new GPT entries in memory.
Setting name!
partNum is 0
Setting name!
partNum is 1
The operation has completed successfully.
mkfs.fat 4.2 (2021-01-31)
Hidden (32) does not match sectors (63)
insmod: ERROR: missing filename.
insmod: ERROR: missing filename.
insmod: ERROR: missing filename.
cp: error writing '/boot/nixos/lbg2i0c8h8d167bx0kfdpfiqzy83a9xi-initrd-linux-5.15.72-initrd.tmp.187': No space left on device
[ 728.793811] reboot: Power down
error: builder for '/nix/store/3cmgrakr934lsig156593jhphzfl8wkk-nixos-boot-disk.drv' failed with exit code 1;
last 10 log lines:
> Setting name!
> partNum is 1
> The operation has completed successfully.
> mkfs.fat 4.2 (2021-01-31)
> Hidden (32) does not match sectors (63)
> insmod: ERROR: missing filename.
> insmod: ERROR: missing filename.
> insmod: ERROR: missing filename.
> cp: error writing '/boot/nixos/lbg2i0c8h8d167bx0kfdpfiqzy83a9xi-initrd-linux-5.15.72-initrd.tmp.187': No space left on device
> [ 728.793811] reboot: Power down
For full logs, run 'nix log /nix/store/3cmgrakr934lsig156593jhphzfl8wkk-nixos-boot-disk.drv'.
error: 1 dependencies of derivation '/nix/store/nkxhigw0ccjnhqwc5qii268aqwp8k64g-run-nixos-vm.drv' failed to build
error: 1 dependencies of derivation '/nix/store/wlxz7fwx28kmy3p01igd1z4mm16kgzih-nixos-vm.drv' failed to build
Any hints that may get me closer to be able to build the raspberry pi system on my more beefy laptop would be appreciated. Thanks!
config.system.build.sdImage is the path to the SD image derivation. You’re basically asking nix to build the SD image, and not e.g. the R-Pi’s NixOS configuration activation package (what you typically run/apply when installing/upgrading locally).
I’m guessing you’ve done the binfmt registration mentioned in the wiki right before the command line you’re trying out. You’d need that for your x86_64 machine to be able to run the aarch64 build toolchains. You can first check to see that you’ve got an aarch64 in /proc/sys/fs/binfmt (I think it is).
Edit: And if it’s not sufficiently clear from the wiki: the binfmt.emulatedSystems setting is for your build host (x86_64) machine. So that means a nixos-rebuild test|switch is required.
However, there’s also a kvm version too old error screaming us straight in the face there. Which I don’t recall ever having encountered personally, so that makes the rest of this speculation. I would guess that there’s some issue with kvm on your setup leading to your machine unable to virtualize aarch64, which then later on gives you those ELF errors because it’s trying to run aarch64 binaries natively, or something.
I have indeed added the boot.binfmt.emulatedSystems = [ "aarch64-linux" ]; to my host machine, and I’ve done a build&switch.
My host system is using flakes, so I hadn’t done a sudo nix-channel --update for quite some time. I’ll try doing that now, and then I’ll retry the nix-build '<nixpkgs/nixos>' -I nixos-config=./configuration.nix --argstr system aarch64-linux command.
building '/nix/store/ini3pyshch8i0h3xf34rk66fp9278jyz-closure-info.drv'...
building '/nix/store/clyynm666l7kj4x4hk7cwz5hg7ka97xz-nixos-boot-disk.drv'...
Formatting '/nix/store/y5x1xvxbn7sbgl6samcx9rxcnlk4kjpr-nixos-boot-disk/disk.img', fmt=qcow2 cluster_size=65536 extended_l2=off compression_type=zlib size=62914560 lazy_refcounts=off refcount_bits=16
kvm version too old
qemu-system-aarch64: failed to initialize kvm: Function not implemented
qemu-system-aarch64: falling back to tcg
building '/nix/store/yivvj0ba26bj839pz21qyylp18acdlym-run-nixos-vm.drv'...
building '/nix/store/93ajzqyl8b662g0sxhvqfs789p4bgv52-nixos-vm.drv'...
loading kernel modules...
[ 62.752553] Invalid ELF header magic: != ELF
[ 69.910141] Invalid ELF header magic: != ELF
mounting Nix store...
mounting host's temporary directory...
starting stage 2 (/nix/store/mcsdgb8wkrvpgg4j1g9zxa7q6gf3ff3h-vm-run-stage2)
Creating new GPT entries in memory.
Setting name!
partNum is 0
Setting name!
partNum is 1
The operation has completed successfully.
mkfs.fat 4.2 (2021-01-31)
Hidden (32) does not match sectors (63)
insmod: ERROR: missing filename.
insmod: ERROR: missing filename.
insmod: ERROR: missing filename.
cp: error writing '/boot/nixos/bzvf7pk9myqvv1wi42swy5d4vw5c485x-initrd-linux-5.15.79-initrd.tmp.188': No space left on device
[ 762.401315] reboot: Power down
error: builder for '/nix/store/clyynm666l7kj4x4hk7cwz5hg7ka97xz-nixos-boot-disk.drv' failed with exit code 1;
last 10 log lines:
> Setting name!
> partNum is 1
> The operation has completed successfully.
> mkfs.fat 4.2 (2021-01-31)
> Hidden (32) does not match sectors (63)
> insmod: ERROR: missing filename.
> insmod: ERROR: missing filename.
> insmod: ERROR: missing filename.
> cp: error writing '/boot/nixos/bzvf7pk9myqvv1wi42swy5d4vw5c485x-initrd-linux-5.15.79-initrd.tmp.188': No space left on device
> [ 762.401315] reboot: Power down
For full logs, run 'nix log /nix/store/clyynm666l7kj4x4hk7cwz5hg7ka97xz-nixos-boot-disk.drv'.
error: 1 dependencies of derivation '/nix/store/d23k3v1sif6xwnb77qkdjds2cmg0fqmj-run-nixos-vm.drv' failed to build
error: 1 dependencies of derivation '/nix/store/akhng24y9pixf5xik8b4vj1zjmywb2pl-nixos-vm.drv' failed to build
-A config.system.build.sdImage is the part that tells it what to build. The file is just the configuration; config.system.build.sdImage is the actual derivation that builds the image. You can’t just ignore that and not use it, because then you’ll build all the attributes returned by the <nixpkgs/nixos> expression, which is a lot of stuff including derivations you shouldn’t expect to work.
Thanks for responding! I thought that I had mistunderstood something like that. No wonder my build-attempt took so long, and no wonder it fails in the end
So how do these three parts play together?
<nixpkgs/nixos>
-A config.system.build.sdImage
-I nixos-config=./configuration.nix
I understand that I must get nixpkgs from somewhere. Is that #1?
Does the <nixpkgs/nixos> expression point to the current channel that I see when I run nixos-version?
I have the configuration.nix+hardware-configuration.nix from my raspberry pi. It’s this configuration.nix that I want to build. Is that not enough? Do I need a derivation (sdImage) that sits on top of that, somehow?
nix-build needs to know where to get the Nix code, what arguments to pass to that code to evaluate and which part of the evaluation result to build.
Free argument <nixpkgs/nixos> tells it to find the code at that path. Triangle brackets mean that it should find nixpkgs in NIX_PATH variable and use nixos subdirectory of that.
-I nixos-config=./configuration.nix tells it to add this line to NIX_PATH, so that <nixos-config> in code will resolve to ./configuration.nix (the code suggests that setting NIXOS_CONFIG environment variable should work just as well)
-A config.system.build.sdImage means that from the resulting attribute set nix-build should build this specific nested attribute config.system.build.sdImage.
To have a different example with building a package from local checkout of nixpkgs: nix-build <nixpkgs> -A hello -I nixpkgs=./nixpkgs will build hello package from nixpkgs from ./nixpkgs directory. You don’t have to use <nixpkgs> and -I there though.
Where is the config.system.build.* options documented? In particular, I’m interested in what other outputs can be built this way. I can’t even figure out how to reflect into that:
Personally I’ve traversed these attributes by :lf-ing a flake and tabbing through outputs.nixosConfigurations before. Not that that’s documentation, of course, but it helped me find what I needed. It’s probably all technically unstable internals though.
