I see a lot of different ways to set things up on systems like headless RasPi, but none of them are quite as nice as what some OSes have.
if there was a way to just put files in a special folder in /boot(Maybe automatically only made available if disk encryption was turned off, otherwise you’d assume the user didn’t want such things), that would then be copied over to /etc/nixos and rebuilt if there was a change, then a lot of stuff would be much easier.
To make setup even simpler, there could be convenience tools to use a flash drive as a binary cache, so that setup could be done entirely by copying files.
That seems to be the current accepted solution, but it doesn’t seem like a good idea in a very small company, where you really don’t want to be the only one who knows how to do stuff, and convincing anyone else to learn new tech is sometimes a battle, especially if it can’t be done in Windows without the CLI, or involves budgeting more cloud space for images.
What might work is building one custom organization-wide image that could do the /boot based config, but if this is a problem other people have too then it would make sense to make it a reusable solution.
What you’re looking for is something similar to the ability to customise cloud images through host metadata. For instance, in AWS, you can put a nix expression in the host metadata (through the console, cloudformation / api, whatever). On startup, if a configuration is found there, a nixos-rebuild switch is done using that as input. There are other similar examples for different providers too, and probably something more generic via cloud-init (but I haven’t looked at that specifically).
So, the mechanism already exists, you’re mostly looking for a config delivery pattern. Whether that’s a specific file path that can somehow be managed externally, a configuration service that can be queried, a git deployment repo that can be pulled, or even just a flake that gets pulled and used based on hostname.
Do you have any examples of what you need? What do other OSs do?
You could make a generic image that pulled a system name from a file in /boot/, and then nixos-rebuild from a Git repo like the nixos-upgrade service does. For a lightweight device like a Pi, you could just put a URL in /boot that is fetched via nix-copy-closure and then activated.
The declarative nature of Nix makes putting actual configuration in /boot difficult. Although you could just put an entire Nix flake or config in /boot and have a generic image that just deploys the real NixOS configuration from that.
I think this could be done with just a small change to the nixos-upgrade service but I agree it would be helpful do document how to do that and build this modified minimal iso.
@peterhoeg A scenario where you need to make a fairly trivial change, and nobody else has a Linux machine or VM… A lot of my background is in companies where even things like version control aren’t really used. Lots of smart people who can do pretty much anything, but nobody who uses Linux enough to stay sharp with the commands.
@Princemachiavelli The solution I’m used to seeing is flashing a fresh Pi image, and either setting it up manually with an HDMI monitor(No management solution at all used, probably not even a setup script), or to use the Rpi flasher tool’s config options to enable SSH and Wifi, and set it up via SSH(Maybe with Ansible, probably manually).
Balena has a cloud solution, but I haven’t used it.
Putting the whole .nix config in /boot and having a generic image that could fetch everything from there was pretty much what I was thinking, with the closest other OS precedent I can think of being RasPi’s ability to customize in the flasher tool while flashing.
Yea there really aren’t any non-cli tools for configuring NixOS like that (there are a few GUI projects but I don’t know their status ATM). If you just need simple things like WiFI, then you could bake in a simple script/service that reads in variables from a file in /boot and configures WiFI at runtime/boot. Just have a customized ISO that includes the necessary services already enabled.
Personally, I’d just make a simple script that automated adding a new host to a flake and populated a simple vars.nix file with the WiFi info, hostname, etc… A custom module would use that file to configure the new nixosConfiguration. Then the script would build that host as an ISO. I’d have this script and flake repo checkout on a Nix server users can access via Git Bash. That server would just have the Git credentials pre-configured. The output ISO can be downloaded via SCP/SFTP from Git Bash as well.
Having a cloud build server seems like a pretty reasonable approach, but having a Nix server somewhere in the cloud is also a pretty big bus number risk…
I might have a go one of these days at creating a custom ISO and a GUI flasher tool that can fill in the config details.
Thinking more about it, a minimal solution would probably have:
A flutter package that imports config from /boot(Trivial bash or Python work)
A generic ISO that had the package
And a more polished solution that would be possibly too much work would have:
A cross-platform tool to configure a flashed card by putting together a configuration from a library, and a few text boxes
Which would save that configuration as a project folder for reproduction
And could remember what ssh authorized keys it gave to devices, to later update them remotely, open an SSH, etc.
And had a built in caching proxy for the binary cache, for setting up offline stuff
@peterhoeg A scenario where you need to make a fairly trivial change, and nobody else has a Linux machine or VM… A lot of my background is in companies where even things like version control aren’t really used. Lots of smart people who can do pretty much anything, but nobody who uses Linux enough to stay sharp with the commands.
I think it’s relevant to try to break this into 2 different issues to ensure we are discussing the same thing.
the choice of NixOS as the platform given your organizational constraints, and
the technical solution to support your desired workflows
The tl;dr: with a bus factor of one, do not pick NixOS. Without some level of proper engineering practices in place, do not pick NixOS. Generally speaking, do not pick $RELATIVELY_OBSCURE_TECHNOLOGY if you haven’t committed to maintaining it yourself internally or have a 3rd party you are paying to do it for you.
What is the point of going through the hassle/context-switch/unfamiliarity of something like NixOS when its biggest selling points are moot anyway since you cannot reason about what exactly was used to build a certain generation when nothing exists in source control?
Changes are trivial until they are not. And something is suddenly needed urgently and without at least a few people who know what is going on, that’s going to come back and bite you.
As a few people have mentioned there are existing similar ways (cloud-init as an example) that could be used to build something that does what you need. But then you would either have to build it yourself, convince someone else to do it or pay a 3rd party to build it for you. But if an existing solution exists that solves it “well enough” (that might not be the case of course), use that. Bring up a Linux VM somewhere and use that and keep everything in source control.
I’m definitely not planning on switching anything in production anytime soon, but it seems like NixOS is gaining ground, so I am interested in R&D to make that kind of switch possible in the future.
At the moment, the “Good enough” solution I see is to do all your setup by hand right on a RasPi (With stock Raspbian full) or VM with no reproducibility at all, and to not even try doing anything complex enough that you can’t use that approach well(Even something like web server with SSL is right out!).
And honestly, it seems to work just as well as a more proper workflow, if you don’t have a full time staff with tech specialists.
But the underlying model of NixOS seems to be far more suitable for just about anything than traditional Linux, so I would think that it will be a pretty good choice in the not too distant future.
One should always be careful dishing out advise when not having the full picture (and especially unsolicited advice), but if you are at the assessment stage, I would recommend trying some of the existing tooling/solutions rather than building too much on your own.