I have an Arch Linux “bare metal” VPS hosted at https://ovhcloud.com. It’s running a Nextcloud server and a few other lightweight services for personal use. Back when I subscribed to their service, OVH were the only provider that supplied Arch Linux support (better then Debian IMO) and low prices for low specs:
Disk Space: 20GB
RAM Memory: 2 GB
Price: ~60$/Year = 5$/Month including VAT
I’d really love to make a transition to a Nix based setup - possibly via an alternative VPS provider. Even if I put aside the challenge of installation itself, I don’t think using simply NixOS is a good idea because large system upgrades will cost me too much disk space. Hence I’m looking for creative alternatives.
Is there a service out there that allows you to upload an image via an API, and that image can be generated according to a configuration.nix?
Alternatively, is it possible to use a configuration.nix to build a NixOS docker image? I’m pretty sure there is, but if so, is there a service out there that will give me a single docker container, that I can upload whatever image I want to it? Of course pricing and specs matter, and I need storage that will not change between upgrades…
I recommend using Hetzner Cloud, you can get a VPS starting from $3 a month. Pick any image, doesn’t matter which as long as you have SSH access, then run nixos-anywhere on it and enjoy your new NixOS-running VPS.
According to this image, for the same price you’re paying you could get a 2C/4GB/40GB box and probably not encounter that storage problem.
Contabo is pretty cheap, offers good specs (unless high bandwidth is needed) and supports custom ISOs. I never could get the official NixOS image to boot on their VMs but that might be just a me thing and you could get it to work. Their web console UX is pretty bad btw, it feels like straight out of a 90’s website.
Personally I use Hetzner VMs. They’re price competitive to Contabo and offer lots of storage (lowest VM comes with 20GB) but they don’t support custom built ISOs. They have a list of ISOs you can boot into a VM and the official NixOS 22.11 image is one of them. What I did was install NixOS once, create a snapshot and now I can instance as many VMs as I want from that one snapshot. I’ve written my notes here on the process if you’re interested.
In my opinion the days of creating entire snapshots/images (for a VPS or a dedicated server) are over now that we have something like nixos-anywhere which lets us install a fully configured system with a custom partition table via a single command in under a minute (depends on the interned speed of course) and on any previously booted OS (as long as it supports kexec and ssh).
What happens when NixOS is already installed on a your server? Do you run nixos-anywhere again? Or do you run nixos-rebuild there? I read this note on nixos-anywhere’s README:
Important Note: Never use a production server as the target. It will be completely overwritten and all data lost. This tool should only be used for commissioning a new computer or repurposing an old machine once all important data has been migrated.
I really want to maximize the storage capacity by not running nixos-rebuild for a “whole world” system upgrades.
nixos-anywhere is only used for the initial installation, after that you just use that machine normally and upgrade via nixos-rebuild. Running nixos-anywhere on a target formats and repartitions the drives (using disko), so you don’t wanna do that on a machine you don’t want to fully reinstall.
I don’t think you can avoid the storage capacity problem besides doing the following:
Not using NixOS ()
Not upgrading ()
Enabling automatic garbage cleaning
Getting a bigger storage disk for 2 bucks more (80GB)
Also, NixOS can be pretty damn small if you configure it properly.
What if I use a dedicated partition for /var/lib for instance, that will hold all the data that I don’t want to be overwritten between upgrades. And then I ask nixos-anywhere to forcefully replace the /nix/store partition, but not the /var/lib partition. Do you think it’s possible?
In anycase, I guess I need to run a few nixos-rebuild build tests to see how much storage my configuration takes - multiply it by 2 or 3 to calculate how much storage capacity I need for 2 or 3 generations of my configuration with zero overlap in /nix/store derivations (whole world upgrade scenario).
What if I use a dedicated partition for /var/lib for instance, that will hold all the data that I don’t want to be overwritten between upgrades. And then I ask nixos-anywhere to forcefully replace the /nix/store partition, but not the /var/lib partition. Do you think it’s possible?
I am not sure I am following along on what that’s getting us (besides a headache). I think you are just over-complicating it and should just pay the extra 2 bucks for 80GB and save your peace of mind. 80GB is probably sufficient considering the fact you are currently using 10GB, LOL. And just set up automatic garbage collection, just in case.
If you need help getting all of this set up, feel free to DM me on matrix.
I am not sure I am following along on what that’s getting us (besides a headache).
LOL . I was trying to describe a scheme where I don’t have to hold two generations of my system in /nix/store at the same time, thanks to the partitioning. In this scheme when I upgrade the system, I reformat the /nix/store partition and I put there afterwards all the derivations that I need for the 2nd generation - and thus probably saving 2$ a month!
After reading the nixos-anywhere source code I think this its not possible in it’s current state. But it’s still worth thinking IMO.
How about running nix-collect-garbage -d in the activation script so it GCs all the older generations on boot and every time you run nixos-rebuild switch? Also, you have to consider that only keeping 1 generation isn’t the safest thing in the world.
The point is, that as far as I can tell, it’s impossible to not hold at least two generations on disk at the same time. For comparison, my current (rather) slim Arch Linux VPS has a /usr of 3.1GB. I really have no clue how much a disk space a single NixOS generation will require - it’ll take me some time to test that, but for sure I’ll report here when done!
You could also get some free cloud storage (e.g. from Google), mount that as a read-only base with a writable overlay and after each update move store paths from the overlay to the cloud drive
The thing you need to consider is that most updates with NixOS don’t require that much additional space because you are only adding a few packages and some configuration changes. What requires most space is major upgrades like going from 22.11 to 23.05 since that changes a lot of packages.
I run some very slim NixOS servers with more or less no state and I can get by with between 4.5 - 5.5 GB of storage in those machines. That means I have just enough space to do a major upgrade of the machine but I have to reboot and run GC right after the upgrade.
As for the image building, the project nixos-generate seems to produce image derivation of various format. Since it’s just a derivation, you could build it on cloud services that could build Nix derivations such as nixbuild.net.
BTW, there’s another Nix-based distro called NixNG targeting container use.
These are certainly interesting options! - the nixos-generators page essentially lists also providers that accept images built that way - which is pretty much exactly what I was looking for! I still wonder though, whether it is possible to configure persistent storage with either of them?
Digital ocean seems like the best option so far - in terms of pricing and support by nixos-generators. But I wonder if it’d be feasible and cheap enough to have a persistent storage along with such a “droplet”…
This honestly sounds like just reprovisioning the / a new machine, with some persistent data stored / replicated elsewhere (a different volume, an S3-like storage service, embedded and provisioned via the nixos config, etc).
Perhaps looking at it like that suggests some different approach or options?
Correct, the next option that came to my mind is using AWS EC2 and EBS. The pricing they offer is also super flexible. I’m still a bit worried how well e.g a postgresql server will operate on that EC2 machine when it’s database is stored there. It seems they offer plans where you can pay after a year of trial period, which leaves me long enough time period to test it.
but that might be just a me thing and you could get it to work. Their web console UX is pretty bad btw, it feels like straight out of a 90’s website.
)
. I was trying to describe a scheme where I don’t have to hold two generations of my system in