# Deployment tools: evaluating NixOps, deploy-rs, and vanilla nix-rebuild
**URL:** https://discourse.nixos.org/t/deployment-tools-evaluating-nixops-deploy-rs-and-vanilla-nix-rebuild/36388
**Category:** Deployments
**Created:** [December 3, 2023, 11:53am UTC](https://discourse.nixos.org/t/deployment-tools-evaluating-nixops-deploy-rs-and-vanilla-nix-rebuild/36388 "2023-12-03T11:53:43Z")
**Posts on this page:** 20
**Page:** 1
### Author:  [@bowmanjd](https://discourse.nixos.org/u/bowmanjd)
#### Post date: [December 3, 2023, 11:53am UTC](https://discourse.nixos.org/t/deployment-tools-evaluating-nixops-deploy-rs-and-vanilla-nix-rebuild/36388/1 "2023-12-03T11:53:43Z")
Our family is redoing our home lab to use NixOS. Currently, we SSH into the server and execute `nix-rebuild`, but we are exploring other deployment options.
Our top contenders are NixOps, deploy-rs, or just using `--target-host` with `nixos-rebuild`.
All three options seem pretty straightforward, so we may just try all three and see what we like. At present, my goal is to build locally, then deploy. All target and build architectures are currently NixOS and x86\_64, although it is not out of the question that we might throw some ARM targets in the mix at some point.
Have any of you tried 2 or more of these options, and have any reflections to add? What did you love about a particular tool?
---
### Author:  [@teto](https://discourse.nixos.org/u/teto)
#### Post date: [December 3, 2023, 2:19pm UTC](https://discourse.nixos.org/t/deployment-tools-evaluating-nixops-deploy-rs-and-vanilla-nix-rebuild/36388/2 "2023-12-03T14:19:14Z")
nixops is abandonware. It was kind of magical for aws deployements where it could do everything form scratch.
If you have physical access to the machines, nixos-rebuild is the simplest and most straightforward one.
I once locked out myself of a machine with nixos-rebuild because I had disabled the ssh server. Since then I use deploy-rs that automatically fallbacks to previous generations in case of issues but it adds complexity compared to nixos-rebuild, might not be worth it for you.
---
### Author:  [@bowmanjd](https://discourse.nixos.org/u/bowmanjd)
#### Post date: [December 3, 2023, 8:30pm UTC](https://discourse.nixos.org/t/deployment-tools-evaluating-nixops-deploy-rs-and-vanilla-nix-rebuild/36388/3 "2023-12-03T20:30:49Z")
> [@teto](#):
>
> nixops is abandonware… nixos-rebuild is the simplest and most straightforward
Thank you, this is very helpful!
Looking at the NixOps repo activity, it wouldn’t occur to me that this project is abandonware, other than the oddly recent transition to Python 3. What reasons cause you to brand it abandonware?
Using nixos-rebuild sounds like a good starting place, then. Much appreciated!
---
### Author:  [@aanderse](https://discourse.nixos.org/u/aanderse)
#### Post date: [December 3, 2023, 11:55pm UTC](https://discourse.nixos.org/t/deployment-tools-evaluating-nixops-deploy-rs-and-vanilla-nix-rebuild/36388/4 "2023-12-03T23:55:31Z")
we don’t want to hurt anyone’s feelings by saying that… but very unfortunately it is true
real shame too because to me nixops, conceptually speaking, is the perfect tool for the type of networks i manage
* * *
since you said homelab you might consider colmena… it has one nice feature where you can reference configuration from machine a on machine b, which can be really nice if you have a reverse proxy or whatever
---
### Author:  [@delroth](https://discourse.nixos.org/u/delroth)
#### Post date: [December 4, 2023, 12:14am UTC](https://discourse.nixos.org/t/deployment-tools-evaluating-nixops-deploy-rs-and-vanilla-nix-rebuild/36388/5 "2023-12-04T00:14:33Z")
+1 to colmena. For me the “reference other machines’ configs when generating a machine’s config” feature is a game changer. This is something that nixops did originally, and was carried over (almost identically) in later deployment tools like morph and colmena. There are a few minor things that I wish colmena did better (e.g. remote evaluation, nix-output-monitor support, etc.), but it’s pretty close to a local optimum for me.
A few things I do with this “cross-referencing other machines’ config” feature:
- Machines that I configure as being remote builders automatically get added to the /etc/nix/machines of all the other machines that need to perform nix builds.
- All the enabled Prometheus exporters on all my machines automatically get scraped by my Prometheus server without having to write any of the plumbing manually.
- I auto-generate a wireguard p2p mesh by having each machine look at the wireguard config of other machines in my network’s Nix config.
---
### Author:  [@bowmanjd](https://discourse.nixos.org/u/bowmanjd)
#### Post date: [December 4, 2023, 12:18am UTC](https://discourse.nixos.org/t/deployment-tools-evaluating-nixops-deploy-rs-and-vanilla-nix-rebuild/36388/6 "2023-12-04T00:18:00Z")
Excellent. Thanks so much to all of you! I am looking at colmena now; thanks for that tip, @aanderse .
I love your thoughts on referencing build hosts on each machine, @delroth .
---
### Author:  [@prescientmoon](https://discourse.nixos.org/u/prescientmoon)
#### Post date: [December 4, 2023, 12:40am UTC](https://discourse.nixos.org/t/deployment-tools-evaluating-nixops-deploy-rs-and-vanilla-nix-rebuild/36388/7 "2023-12-04T00:40:26Z")
I remember deploy-rs not playing well with having to type a password for remote sudo. I know 1-2 other tools I’ve tried also had similar problems. ot sure if they’ve been fixed since then.
---
### Author:  [@nixinator](https://discourse.nixos.org/u/nixinator)
#### Post date: [December 4, 2023, 2:08pm UTC](https://discourse.nixos.org/t/deployment-tools-evaluating-nixops-deploy-rs-and-vanilla-nix-rebuild/36388/8 "2023-12-04T14:08:27Z")
if you want to ‘learn’ how you can do this at the cmd line with bash, then try nixinate…
it’s simple and you’ll be able to find out how easy it is to deploy with nix.
> **[GitHub - MatthewCroughan/nixinate: Another NixOS Deployment Tool - Nixinate...](https://github.com/MatthewCroughan/nixinate)**
>
> Another NixOS Deployment Tool - Nixinate your systems 🕶️ - GitHub - MatthewCroughan/nixinate: Another NixOS Deployment Tool - Nixinate your systems 🕶️
---
### Author:  [@sirphobos](https://discourse.nixos.org/u/sirphobos)
#### Post date: [December 4, 2023, 4:51pm UTC](https://discourse.nixos.org/t/deployment-tools-evaluating-nixops-deploy-rs-and-vanilla-nix-rebuild/36388/9 "2023-12-04T16:51:20Z")
but in case the machine configurations are in flakes - referencing different configs is not a problem, in this case just doing `nixos-rebuild --target-host` will do the trick, no?
I’m just personally trying to keep all the “third-party” tools to the minimum
---
### Author:  [@nixinator](https://discourse.nixos.org/u/nixinator)
#### Post date: [December 5, 2023, 12:10am UTC](https://discourse.nixos.org/t/deployment-tools-evaluating-nixops-deploy-rs-and-vanilla-nix-rebuild/36388/10 "2023-12-05T00:10:27Z")
this is basically what nixinate does in a nutshell.
You can probably just run it up and reverse engineer the bash script it makes!!!
it designed to be to show you how to do it yourself.
It’s very popular in 2038.
---
### Author:  [@jasom](https://discourse.nixos.org/u/jasom)
#### Post date: [December 9, 2023, 12:04am UTC](https://discourse.nixos.org/t/deployment-tools-evaluating-nixops-deploy-rs-and-vanilla-nix-rebuild/36388/11 "2023-12-09T00:04:50Z")
> [@teto](#):
>
> nixops is abandonware
This was surprising for me to read, [but seems to be true](https://github.com/NixOS/teams-collaboration/blob/941a4b5139e8a594469a9ded74208f548036c378/meeting-notes/2023-11-10.md?plain=1#L88-L96). I’ve been happily using it for years to maintain my remote (some virtual) machines. I’d been meaning to look into colmena but haven’t had an impetus to, given that I haven’t had any issues with NixOps. If plans really are to sunset it, then that might be the impetus I need.
---
### Author:  [@hugosenari](https://discourse.nixos.org/u/hugosenari)
#### Post date: [December 9, 2023, 5:58am UTC](https://discourse.nixos.org/t/deployment-tools-evaluating-nixops-deploy-rs-and-vanilla-nix-rebuild/36388/12 "2023-12-09T05:58:25Z")
There is also [cachix agent](https://docs.cachix.org/deploy/deploying-to-agents/), has nice web interface.
But now I’m using [system.autoUpgrade](https://search.nixos.org/options?channel=unstable&from=0&size=50&sort=relevance&type=packages&query=system.autoUpgrade) with flake.
```nix
system.autoUpgrade.enable = true;
system.autoUpgrade.dates = "*-*-* *:20:00";
system.autoUpgrade.flake = "github:hugosenari/nixos-config#${config.networking.hostName}";
system.autoUpgrade.flags = ["--refresh"];
system.autoUpgrade.randomizedDelaySec = "5m";
```
Pools my config (flake) from GitHub every day.
Good: simple
Good: 546ms CPU time, received 9.6K IP traffic, sent 3.4K IP traffic. (every X time)
Bad: have to wait timer trigger.
Bad: Isn’t a tool to setup new instances (like NixOps).
edit: update link because 23.11 doesn’t exist anymore
---
### Author:  [@bowmanjd](https://discourse.nixos.org/u/bowmanjd)
#### Post date: [December 9, 2023, 1:05pm UTC](https://discourse.nixos.org/t/deployment-tools-evaluating-nixops-deploy-rs-and-vanilla-nix-rebuild/36388/13 "2023-12-09T13:05:57Z")
Cachix agent looks very nice; thanks for pointing that out.
Our current use case is a single deployment target that has decent compute resources. So we are finding that our original practice of connecting via SSH may not be so bad. Running `nix-rebuild --target-host` works, but doesn’t necessarily gain us anything over connecting first (and using tmux, which has advantages) then running `nix-rebuild`.
Given our scenario, your `system.autoUpgrade` idea, @hugosenari,is brilliant. Honestly, I didn’t know that option existed!
Gonna run that by the Ops team now… (the 16-year-old in the basement). We’ll see…
---
### Author:  [@rjpc](https://discourse.nixos.org/u/rjpc)
#### Post date: [December 9, 2023, 1:26pm UTC](https://discourse.nixos.org/t/deployment-tools-evaluating-nixops-deploy-rs-and-vanilla-nix-rebuild/36388/14 "2023-12-09T13:26:57Z")
FWIW I’ve been using deploy-rs as of late for most of my Pis and it’s been rather painless. I would like to give colmena a try. Would be nice to just use `nixos-rebuild --target-host` but I haven’t and deploy-rs offers those nice checks to avoid mistakes
---
### Author:  [@bowmanjd](https://discourse.nixos.org/u/bowmanjd)
#### Post date: [December 9, 2023, 3:20pm UTC](https://discourse.nixos.org/t/deployment-tools-evaluating-nixops-deploy-rs-and-vanilla-nix-rebuild/36388/15 "2023-12-09T15:20:00Z")
> [@hugosenari](#):
>
> But now I’m using [system.autoUpgrade](https://search.nixos.org/options?channel=23.11&from=0&size=50&sort=relevance&type=packages&query=system.autoUpgrade) with flake.
Curious: does this do a nix flake update as well? And does it also do a fresh `git pull`?
---
### Author:  [@hugosenari](https://discourse.nixos.org/u/hugosenari)
#### Post date: [December 11, 2023, 10:54pm UTC](https://discourse.nixos.org/t/deployment-tools-evaluating-nixops-deploy-rs-and-vanilla-nix-rebuild/36388/16 "2023-12-11T22:54:30Z")
> [@bowmanjd](#):
>
> Curious: does this do a nix flake update as well? And does it also do a fresh `git pull`?
Basically what it does is constantly asking your system to be reinstalled, pointing to a repository.
So I’m not expecting it to update your inputs. And you have to add --refresh or nix will use flake cache (not a fresh pull)
---
### Author:  [@uep](https://discourse.nixos.org/u/uep)
#### Post date: [December 11, 2023, 11:09pm UTC](https://discourse.nixos.org/t/deployment-tools-evaluating-nixops-deploy-rs-and-vanilla-nix-rebuild/36388/17 "2023-12-11T23:09:59Z")
Answering only for how _I_ would use this:
This would be run on hosts I log into less often, or that are used rarely and should be upgraded on boot/resume after a period of time. Previously, I had one or two of those with the auto upgrade service enabled, on channels. I disabled that when everything moved to a system flake.
It’s why I haven’t used any of the various deployment tools (though bento seems interesting), because they’re mostly push style, and this use-case needs something more pull based.
I, too, had missed that there was a flake option for the autoupdate service. For me, the idea here is that I update regularly on my active desktop, and push those revisions including the locked flake inputs to the repo. So those autoupgrades update inputs via the git pull of the lock file, and only update to revisions I have already used and built, rather than updating their lock file locally. This also means the content will already be in the store of another local system.
I might (if I can be bothered) even keep a separate branch for the known-good updates. That’s still less branch maintenance than I used to do before consolidating a common base and per-host branches into a single flake branch.
---
### Author:  [@bowmanjd](https://discourse.nixos.org/u/bowmanjd)
#### Post date: [December 11, 2023, 11:29pm UTC](https://discourse.nixos.org/t/deployment-tools-evaluating-nixops-deploy-rs-and-vanilla-nix-rebuild/36388/18 "2023-12-11T23:29:59Z")
Excellent detail. Thank you for helping me think through this!
---
### Author:  [@uep](https://discourse.nixos.org/u/uep)
#### Post date: [December 13, 2023, 10:55pm UTC](https://discourse.nixos.org/t/deployment-tools-evaluating-nixops-deploy-rs-and-vanilla-nix-rebuild/36388/19 "2023-12-13T22:55:34Z")
I set this up on one of those hosts shortly after posting here, and initial tests were good. But I had one suspicion, which turned out to be valid on testing again this morning:
The default config of the autoUpdate service has a timer with the persistent flag set, so it runs when the laptop is resumed after being suspended for a while (overnight, or for several months…)
But the problem I found this morning:
```plaintext
Dec 14 09:05:13 rocinante systemd[1]: Starting NixOS Upgrade...
Dec 14 09:05:13 rocinante nixos-upgrade-start[32301]: warning: you don't have Internet access; disabling some network-dependent features
Dec 14 09:05:13 rocinante nixos-upgrade-start[32298]: building the system configuration...
Dec 14 09:05:13 rocinante nixos-upgrade-start[32355]: warning: you don't have Internet access; disabling some network-dependent features
…
```
It runs _too soon_ after resume, before the wifi has had a chance to reconnect.
So it needs and additional dependency on `network-online.target` probably, and/or a delay. _Edit:_ that dependency is already there, but doesn’t seem to apply after resume, I’ll look further into this.
---
### Author:  [@firecat53](https://discourse.nixos.org/u/firecat53)
#### Post date: [December 14, 2023, 1:46am UTC](https://discourse.nixos.org/t/deployment-tools-evaluating-nixops-deploy-rs-and-vanilla-nix-rebuild/36388/20 "2023-12-14T01:46:07Z")
I use this snippet for fwupd-refresh, but it could be easily used for nixos-upgrade. It just allows the service to restart several times over several minutes without failing…which typically should allow the network time to be up on a laptop.
```auto
# Firmware updates - fwupd
services.fwupd.enable = true;
# Allow fwupd-refresh to restart if failed (after resume)
systemd.services.fwupd-refresh = {
serviceConfig = {
Restart = "on-failure";
RestartSec = "20";
};
unitConfig = {
StartLimitIntervalSec = 100;
StartLimitBurst = 5;
};
};
```
[Next page](https://discourse.nixos.org/t/deployment-tools-evaluating-nixops-deploy-rs-and-vanilla-nix-rebuild/36388.md?page=2)