SkaraboxOS - opinionated NixOS installer

Announcements
1

SkaraboxOS provides a flake template which combines:

  • Creating a bootable ISO, installable on an USB key.
  • nixos-anywhere to install NixOS headlessly.
  • disko to format the drives.
  • sops-nix to handle secrets.
  • deploy-rs to deploy updates.

SkaraboxOS expects a particular hardware layout:

  • 1 SSD or NVMe drive for the OS.
  • 2 Hard drives that will store data. Capacity depends on the amount of data that will be stored. They will be formatted in Raid 1 (mirror) so each hard drive should have the same size.

WARNING: The 3 disks will be formatted and completely wiped out of data.

At the end of the process, the server will:

  • Have an encrypted ZFS root partition using the NVMe drive, unlockable remotely through ssh.
  • Have an encrypted ZFS data hard drives.
  • Be accessible through ssh for administration and updates.
10 Likes
2

seems like a cool tool, i will be sure to try it out when i install NixOS on a new machine.

1 Like
3

Obligatory “zfs native encryption is iffy”: How good is ZFS on root on NixOS? - #9 by ElvishJerricco

1 Like
4

You picked all the libraries I’ve been looking at recently! Thank you so much for putting this together.

1 Like
5

I just did something which I find really really cool. A 2 step demo to install NixOS with some goodies in a VM GitHub - ibizaman/skarabox: Opinionated and simplified headless NixOS installer.
It’s really just a demo, because it’s insecure since I store the ssh private key in the repo and nix store. That’s fine for a demo, the goal was to make it deterministic to be able to cache it.
But the extra cool factor is I use it in CI to make sure my changes do not impact negatively the installation skarabox/tests/default.nix at 2b8a8e3948357a7f802db6c1edd00a9bd948a410 · ibizaman/skarabox · GitHub

The repo’s initial goal is not to install on a VM but on a real server. You must burn an ISO on a USB key then boot on that. The normal path is thus to create a flake template and follow these instructions GitHub - ibizaman/skarabox: Opinionated and simplified headless NixOS installer. But I’ll be adding those to CI and use much less manual steps and then it will be really nice.

2 Likes
6

Another milestone passed for Skarabox, the NixOS installer Release Store secrets in SOPS file · ibizaman/skarabox · GitHub

The user’s password as well as the root and data ZFS pools’ passphrases are stored in the generated SOPS file. Thanks to generating the host key ourselves, we can encrypt the sops file upon template initialization, even before the first boot. This means the SOPS file is generated completely without user intervention, including the secrets it contains.

The full install process is now:


mkdir myskarabox
cd myskarabox
nix run github:ibizaman/skarabox#init

# Tweak settings to match installing on a target host
echo 127.0.0.1 > ip
echo x86_64-linux > system
nix run .#gen-knownhosts-file

# More tweaks to install on a VM (for testing)
echo 2222 > ssh_port
echo 2223 > ssh_boot_port

nix run .#beacon-vm &
nix run .#ssh -- -o StrictHostKeyChecking=no sudo nixos-facter > facter.json
nix run .#install-on-beacon .#skarabox
# VM will reboot.

# Installation is done!
1 Like
7

I’m super happy to announce the installer is now at version v1.0.0. I implemented all the features I wanted and I must say the result is pretty slick.

New features are:

  • Use flake-parts to create a flake module per host and reduce flake.nix to absolute minimum.
  • All variants (one or two OS drive, zero or two data drives) are tested in CI.
  • Static IP is tested in CI.
  • Add scripts to generate scaffolding and secrets for new host.
  • Add manual and reference readme to it, which allows to keep offline documentation always up to date.
  • Group ZFS options per pool.
  • Add WiFi hotspot to beacon.
  • Use static IP in beacon, allowing to use same IP for beacon and host.
  • Use optionally static IP on host and at initrd stage.
  • Use same username in beacon and host.
  • Handle copying extra secrets on installation.
  • Share values between flake module and nixos modules.
  • Add recovery instructions.
  • Add colmena as deployment tool.

I’m now stopping working on this actively and switching back to SelfHostBlocks but all contributions are very welcomed. I like the manual but not sure it’s great, so any hint on how to make it better are very appreciated!

8 Likes