I am trying to update a nix os system through a 600 baud link.
The issue is, it could take a very long time, even if it only copies the paths that don’t exist on the remote.
I wrote a simple python script which traverses the files in a store given the previous system hash, and produces binary diffs using the excellent bsdiff tool.
The result is then compressed using tar and xz -9
Another script uses bspatch to restore the files on the remote side and it perfectly replicates the original store using 1/6-1/15th the data transferred.
I am looking at files < 1MB instead of 10MB, so a few hours to transfer instead of several days.
My issue now is, nix store import/export will not let me import paths outside the store, and nix realize is not working either.
How can I restore the files? Doesn’t it seem like this ability could be built into nix store import/export to begin with? It could also work with nix copy closure, since in that case it knows the remote paths, so it should be able to automatically do it.
Any thoughts? Pointers to how I can easily solve this?
OK, I’m intrigued now - have you somehow managed to launch a deep space probe running NixOS? :)
I’d be lying if I said I fully understood, but is the issue basically registering the newly restored files with the database? The installer uses nix-store --load-db to do this, and the format seems to be described in this ancient mailing list post (it doesn’t seem to have changed though.) There’s also an undocumented nix-store --register-validity operation, which I assume does something similar one path at a time.
this is quite relevant to my interests, although my case is much more prosaic: I want to copy just the “store diff” over an air gap. i.e. the bandwidth is ok, but latency and, uhh, packet size are not great.
would love to know how you deal with the import part, at least…