Introducing bento, a NixOS deployment framework

Announcements
42

So, today I learned about nix-copy-closure, why not allow bento to also use a push model?

 sudo env NIX_SSHOPTS="-p2222" TARGET_IP=10.42.42.42 NAME=kikimora ~/dev/bento/bento build switch

:upside_down_face:

this is working, but won’t update the metadata information until the next timer kick in to send a log after building nothing (except if you just use nixos-rebuild build which will just activate it as normal).

I have three hosts with very tight memory and CPU requirements, it currently works but the one in the local network could be handled this way fine :thinking:

I think bento will need a rewrite in a proper language when I’m done writing it, this would allow more strictness and better user interface. But it’s cool to experiment :smiley:

43

Meanwhile, I got a funnier workflow :+1:t3: nix-copy-closure but async + pull based

Remote systems can depose the list of their nix store entries on the SFTP server, when generating a new configuration on the central / sftp server, a closure of the config can be generated, using comm and the nix store entries we can calculate missing store entries on the remote computer compared to the closure components. Then, using nix-store --export we can generate the closure diff and make it available on the SFTP, so when the remote computer will pick up the configuration files changes, it can also pick everything prepared for it to update :+1:t3:

Basically

  • on a remote system, run find /nix/store/ -maxdepth 1 | sort | gzip > current-closure.gz and copy the file to the central server (bento already offer SFTP)
  • on the central server, run nixos-rebuild build to generate the machine config, then nix-store --query -R $(readlink -f result) | sort > new-closure
  • copy the current-closure.gz from the sftp directory, uncompress and store it somewhere temporary
  • run nix-store --export $(comm -2 -3 new-closure current-closure) | gzip > closure-diff.gz
  • make closure-diff.gz available over SFTP and the remote system to use it with zcat closure-diff.gz | nix-store --import
1 Like
44

Even better :thinking:

As we know the remote configuration of the system, and the local (new) one, it’s possible to locally create a closure diff and make it available on the SFTP server.

The client would just fetch the result and import it into the store, the last line of nix-store --import is the NixOS system, in which there is the required scripts to switch to that configuration.

The diff will not take care of any store entry that would already be there though, but I think the tradeoff is fair.

1 Like
45

It could be implemented by users, but would it make sense to have some kind of integration so that a Hydra instance can build systems and then systems would only update once Hydra has finished building them?

46

It seems like after a couple of updates you will end up with all packages present on your SFTP server in different .gz files. Wouldn’t it be easier to use it as a substituter at this point? Publish new config there with nix copy and let servers fetch it from there with nix copy (or nix-copy-closure, of course).

47

Yeah, I’ve been wondering about the advantages compared to a substituters, and there are none.

However, I had in mind a single closure file to download and not multiple files, there would be only one closure file available.

48

I never used hydra, but if you start building the systemd and only publish the configuration files after it’s done, that will work.

That’s how Bento is currently working, when deploying new files, they are built locally first one by one, and you can use that machine as a substituter.

49

TIL about nix store diff-closures to have a nice digest of changes between two versions:

> nix store diff-closures  /nix/store/p50qql7f42rl0fccdwxw45k21pnqb9ii-nixos-system-x1-22.11.20220921.d6490a0/  /nix/store/0m339qjp4mvirnkyvpbx09gjqwscadpc-nixos-system-x1-22.11.20220927.7e52b35/
bind: 9.18.6 → 9.18.7
cpupower: 5.19.9, 5.19.9_fish → 5.19.11, 5.19.11_fish
gh: 2.15.0, 2.15.0_fish → 2.16.1, 2.16.1_fish
imagemagick: 7.1.0-48 → 7.1.0-49, +18.0 KiB
initrd-linux: 5.19.9 → 5.19.11
libblockdev: 2.26 → 2.28
libbytesize: 2.6 → 2.7
libdmtx: 0.7.5 → 0.7.7
linux: 5.19.9, 5.19.9-modules → 5.19.11, 5.19.11-modules, +126.6 KiB
man: -11.8 KiB
nixos: +12.5 KiB
nixos-system-x1: 22.11.20220921.d6490a0 → 22.11.20220927.7e52b35
opencv: 4.5.4 → 4.6.0, +1901.6 KiB
plasma-workspace: +62.4 KiB
root-authorized_keys: ∅ → ε
source: +701.9 KiB
systemsettings: +62.6 KiB

This could be used to display a nice summary of changes to an user and prompt them to update or not :+1:

1 Like
50

I was so enthusiastic about it that bento diff is now a thing :+1:t3: which comes with the tag 1.2.0.

https://github.com/rapenne-s/bento/commit/f14121f9bb88ae90b89cda2018613d404ab72c3a

2 Likes
51

So, bento received a new feature again, it can push configurations to a remote server :smiley:

It’s practical for setting up new services and have changes instantaneous, and it doesn’t break the pull model. When you push a config, the same config is available in the SFTP server, so when it’s picked up, bento will do nothing except reporting the current version to the server.

sudo env NIX_SSHOPTS="-p2222" TARGET_IP=10.43.43.1 NAME=ams bento build switch

https://github.com/rapenne-s/bento/commit/80c0de326b0d5653ebe0138d8c376abf07794d3b

2 Likes
52

New feature, the status display was done once or upon a timer, that was exceptionnaly boring to see it displayed infinitely without knowing if something changed.

Theses times are now over! bento status will display the current status and wait for a change (new version published, or an host running an update) to display the status again.

if the terminal isn’t interactive, the status is only displayed once. Like in bento status | cat or bento status > status.txt

https://github.com/rapenne-s/bento/commit/fcb949881a0b12e9ccc3edb750efa6146cdced0a

53

1.5.0 is out! It’s now using nvd when available to look at the diff of new published versions, otherwise it’s just more efficient.

I’m very happy of Bento, it’s working really well for me, I don’t have to care about updating my machines manually, and on the main server I always exactly know which NixOS version is running on each computer :star_struck:

4 Likes
54

Hi @Solene thanks for your app!

Do you have idea how to use bento to deploy home-manager configurations?

55

If you are not using NixOS, I have no idea how to deal with that :thinking:

56

I’m using NixOS. Right now I use deploy-rs which offers a way to deploy system profiles and home-manager profiles (e.g. homeConfigurations.jdoe = home-manager.lib.homeManagerConfiguration)

57

You could use home-manager as a NixOS module instead, unless you have a specific reason for deploying it as a separate profile.

4 Likes
58

New 1.5.1 version, this includes a ready-to-use reboot timer in case the kernel got updated. This is in addition of the REBOOT file used to trigger immediate reboot after a change (it’s exclusive obviously).

edit: new 1.5.2 which uses flake-utils to support all other architectures and not only x86_64-linux :slight_smile:

2 Likes
59

I am wondering if bento is suitable to run on a low powered arm server (like rpi4) to deploy to x86 laptops. You say that it is possible to use the server to built all configurations. But to me it is not clear if the opposite is practical.

60

You could use bento to publish the configuration and track the remote systems state, without having to build. This was allowed in the last commits IIRC.

1 Like