One host, multiple nixos configurations

I am running two services, nicely packed as nixos configurations, and each deployed to their own virtual machine (on the Hetzner cloud) using nixos-switch build. So far so good.

But neither service really needs a lot of resources, and I expect they could just run side-by side on the same virtual machine, and it would feel less wasteful.

So of course I could combine them in a single configuration, maybe import the respective service configurations via flake imports, and deploy as such.

But I do like to keep them apart. Its good to be able to upgrade one’s nixpkgs pin without having to upgrade the other one, and to deploy one without affecting the other one.

Is there a way to have my cake and eat it too? A way to deploy to the same machine (or same virtual machine) somehow independently?

One idea might be to have a system running on the virtual machine that just defines two nixos containers, maybe somehow assigns them separate IPs, and then I can nixos-switch-deploy my services into these containers?

Have other thought about this and solved this before maybe?

NixOS Containers - NixOS Wiki might be what you are looking for, you can deploy multiple nixos configs side by side as containers on the host machine

1 Like

Thanks! I played with these before, but at least as described on the wiki the configuration for these containers is part of the host’s configuration – that’s the kind of tight coupling of the configuration that I want to avoid (else running all services on the same host would also be an option).

Maybe “imperative container managmenet” as described in https://nixos.org/manual/nixos/stable/#ch-containers is closer to what I would want. Or some middle ground – declaring the container in the host (name, networking), but configuring their content with separate nixos-switch or nixos-container update commands.

Hi, there is also microvm.nix that is pretty interesting for this kind of setup

1 Like

Looks like Hetzner cloud doesn’t support nested virtualisation, but other providers might. Not sure how efficient this is but you indicate light use, so perhaps the value proposition is in ongoing costs and management rather than bang for buck?

1 Like

I’ve been doing something similar to this:

{
  inputs.nixpkgs-one.url = "github:NixOS/nixpkgs/nixos-unstable";
  inputs.nixpkgs-two.url = "github:NixOS/nixpkgs/nixos-22.05";

  outputs = { self, nixpkgs-one, nixpkgs-two }: {
    nixosConfigurations.one = nixpkgs-two.lib.nixosSystem {
      system = "x86_64-linux";
      modules = [{ boot.isContainer = true; }];
    };
    nixosConfigurations.two = nixpkgs-two.lib.nixosSystem {
      system = "x86_64-linux";
      modules = [{ boot.isContainer = true; }];
    };
  };
}
$ sudo nixos-container run one -- nix --version
nix (Nix) 2.18.4
$ sudo nixos-container run two -- nix --version
nix (Nix) 2.8.1

I think I have used Hetzner at some point to run containerized services, but efficiency wasn’t a priority for that case at all.

3 Likes

Right, ongoing costs and the general sense of not having too much resources reserved and then unused.

1 Like