Hi there, as part of Developing a system that replaces nix remote build I’m looking into implementing enough of the nix-daemon protocol (or the nix-store --serve protocol) to call buildDerivation.
The reason is that just doing nix-store -r /nix/store/thing.drv requires the drv to be in the nix store, which requires all its ancestors all the way up, which is a very long download. Also it requires to upload all the drvs to the remote cache, which is also very slow (I suspect because it doesn’t do it in parallel, but have yet to look into that seriously… I did notice LegacySSHStore has “max-connection” but we’re using http:// so I suppose it would need to be HttpBinaryCacheStore and come to think of it I don’t know if the protocol even supports overlapping copies… like I said, haven’t researched yet).
Anyway, both of those protocols are ad-hoc imperative code scattered everywhere. I’m trying to reverse engineer it but it’s complicated so I’m wondering if there is any prior art out there? Or documentation? I did find GitHub - input-output-hk/nix-daemon-protocol: Experimental nix-daemon, but this is from the daemon’s perspective. Still I was able to learn from it, such as the daemon handshake expects an “unknown int” at the end, which I can’t find in the source for but does indeed seem to be necessary.
Specifically processStderr is pretty confusing.
Thanks in advance!
If I ever get this working, I’d be happy to publish code along with whatever documentation I’m able to make up (might be full of “must do this, don’t know why”).
4 Likes
There are no docs I’m aware of, but I’d like to be positively surprised.
@flokli @ericson2314 @thufschmitt hints?
Yes, unfortunately it is not adequately documented.
I have have some PRs cleaning up things and trying to separate the serialization logic from the rest some more, but they have no been reviewed yet.
Have you checked out https://github.com/haskell-nix/hnix-store/tree/master/hnix-store-remote? I hear it reasonably complete.
Ah yes indeed this is exactly the kind of thing I was hoping for, thanks! I will dig in.
I already notice it also demonstrates the same “mystery int”: “-- XXX: reason for this is unknown […] putInt 0” XD
1 Like
Hi,
A few colleagues started working on a Rust reimplementation of the protocol (mostly working as a proxy, so that should do both the client and the daemon) as a learning side-project. They aren’t too keen on making it public (yet) as they want to be able to shamelessly experiment and screw-up, but I’ve invited you to the repo so that you can have a look. And if you find this useful, don’t hesitate to tell them as it will help my lobbying to get them to publish it 
1 Like
Nice, thanks, I’ll reference this too. My eventual goal is to implement only buildDerivation, or the minimum amount necessary to get a build going without having to copy the drv (though that’s not the problem, but rather that nix also wants to copy all the drv’s dependencies, and that it’s very slow to upload them). So it’s not an ambitious as a full protocol implementation, but would work as another reference at least.
1 Like