Managing a fleet of NixOS desktop machines

Hi all,
I was wondering if anyone had any suggestions on how to manage a fleet of employee laptops using NixOS.

Topics such as:

  • how to centrally manage the configuration on multiple machines?
  • how to do this based on role? various roles will need different setups, eg “devops” vs “ML dev” vs “mobile dev”
  • how to do this based on single user? centrally managing secrets for users such as vpn dial-in, ssh keyring for infrastructure access, etc.
  • enrolling new machines into the configuration and how to keep track of which machine should have which config
  • any external paid-for or free services that can be helpful here
  • any other software that could be useful in addition to NixOS
  • any resources, blog posts, etc
  • best practices
  • concepts to think about

Basically doing the IT thing with NixOS.

I would appreciate any suggestions, thanks!

3 Likes

This is an interesting topic that i would like to explore in the context of comin, a pull based deployment tool presented at the last Nixcon.

how to centrally manage the configuration on multiple machines?

There are a lot of deployment tools available, most are push based while some are pull based. Basically, you could have a Git repository containing your configurations and either a CICD for pushing configuration changes to machines or an agent running on all machines which pulls the repository (this is how comin is working).

how to do this based on role? various roles will need different setups, eg “devops” vs “ML dev” vs “mobile dev”

I think the easiest way would be to have a NixOS configuration per employee machine. They could then be built by composing NixOS modules, which could describe roles. Also, this would allow employees to submit configuration changes to their own machine. Note I don’t know how this could scale if you have hundreds of machines to deploy.

how to do this based on single user? centrally managing secrets for users such as vpn dial-in, ssh keyring for infrastructure access, etc.

For secret management, a lot of of people are using sops-nix. Providing yubikeys to employees could also be a way of managing their secrets (since a yubikey holds private keys).

enrolling new machines into the configuration and how to keep track of which machine should have which config

The deployment agent could report a status or expose prometheus metrics. There are also tooling such as osquery (recently added to NixOS) that can be used to build a global overview of your fleet.

or Clan: Fully-automated distributed NixOS management :: NixCon 2024 :: pretalx

If you use flakes there is already a pull mode built into NixOS.

https://search.nixos.org/options?channel=24.05&show=system.autoUpgrade.flake&from=0&size=50&sort=relevance&type=packages&query=autoUpgrade

As for the organisation of the files.
The profiles can just be a bunch of files in which you enable the related configuration.

Here you can see that I assign each machine a specific home manager profiles: nixos/flake.nix at abce15394c13dc46aacea1f3113e70e3a034ee35 - nixos - Gitea: Git with a cup of tea

That comes from here: nixos/profiles at master - nixos - Gitea: Git with a cup of tea

For the NixOS profiles I have done it a bit differently and written my own module to enable it with a flag. I can’t remember why I did differently with home-manager: nixos/default.nix at abce15394c13dc46aacea1f3113e70e3a034ee35 - nixos - Gitea: Git with a cup of tea

Which then gets used here: nixos/default.nix at abce15394c13dc46aacea1f3113e70e3a034ee35 - nixos - Gitea: Git with a cup of tea

None of my config is done with a special framework. One could certainly streamline it a bit but Nix is already quite capable for many tasks.