Following the guide kindly provided, I have successfully built my own SD installer image for ARMv7 (see this issue for the reason why I needed to do this!) and it works great.
However, those instructions also explain how to customize the image with things like authorized ssh keys – super useful, because it sounds like that would allow me to put an SD card into a machine, power it on, and then be able to ssh in over the network without needing a serial console.
Now, using a modified sd-image.nix, the custom image builds (and boots) just fine. However, the /etc/nixos/configuration.nix on the resulting system is basically empty, and doesn’t contain any of my useful changes. I know the image itself was built with them, because (for example) it is running the hardkernel kernel as requested. But, sshd is not running, despite being added to services in the normal way.
Looking deeper, by mounting the installer immediately after it’s made, I cannot actually see any normal-looking root filesystem there. Certainly no configuration.nix, although there is the complete Nix store. So, questions:
When I boot my ARMv7 machine with this image, it has a root filesystem. Where does this come from?
How much should I expect from a configurable SD image? Is this a bug? It would be suuuuper neat if it could work the way I was hoping, because it would make it really easy to set up a fleet of ARM machines.
(1) Unless you went and removed the installation-device profile, it will be included in sd images. That profile enables sshd, but disables it from starting at boot.
(2) The image is, uh, rehydrated at first boot (for lack of a better term), which is why you’re seeing a weird husk of a nix system when mounting a fresh image.
@samueldr, thanks for your reply! This is really helpful.
Unless you went and removed the installation-device profile…
I did not! And upon investigating it further, I see that this is the place that was causing my configuration collisions (I was also trying to set sshd’s permitRootLogin to false in my sd-image-configuration.nix, giving multiple-definition errors. So, this makes sense, and it explains why there is an sshd systemd unit but it is not running.
Unfortunately, I am still a bit confused. The example in the instructions suggests adding some ssh authorized_keys to the root user. I opted instead for the configuration I want: adding an actual user and some authorized_keys for them. And the result is not what I expected – the user exists on the final system, and they have a home folder, but there is no .ssh folder in there and so, no authorized_keys file. Is this expected, do you think?
The image is, uh, rehydrated at first boot…
Yeah, this was my assumption. Excellent choice of word, btw However, I’m looking right at the code in sd-image.nix and I still can’t tell exactly which step is responsible for this rehydration process
My goal, I think, would be to add another step… somewhere… which will take the sd-image-configuration.nix I am building the image from and ensure it ends up on the final system as /etc/nixos/configuration.nix. Also, to close this gap whereby the authorized_keys are missing (and anything else that might also not quite work right). How tricky do you think this might be? I will try to dig into it now. Let me know if you have any suggestions!
I’m not sure, never tried it. I can at least show you where the “dummy” configuration.nix comes from. Later in the file, it is copied to /etc/nixos/configuration.nix through a postBootCommands.
The “tricky” bit with the sd_image as they are, is that they are not ready-to-use systems, but an installer on which you install the system itself. This means that it has all the “weird bits” from installer images. You might have to remove the installation-image things entirely if you intend to have a ready-to-use image instead.
However, I’m looking right at the code in