We maintain our own hydra at our company that continually builds our software
and systems, to keep deploy times to a minimum. We have been using eris as a
binary cache for quite some time we got more and more incomplete downloads. To
solve this problem we internally had a discussion if we should either add
http accept-ranges to eris
or try to implement a binary cache in rust ourselves. We
went with the second option.
We’ve been using harmonia for over a month now and are now confident to
officially announce it here.
Harmonia is a binary cache that serves your local nix-store similar to
nix-serve. To make harmonia work, we first had to write rust bindings into libnixstore
, to make access to the store really simple. This makes part
of the project unsafe
but we accepted the drawback rather than
reimplementing a lot of the nix store internals. We have plans to make the
bindings in a separate crate and would accept PRs that improve the current
bindings
On downloading a nar
we currently still do nix-store --dump
as it was more
efficient than using the FdSink
directly, because we had to wait till it was
done writing to a file descriptor. I still think an export path with a Sink
could be more efficient and I was working on an implementation using
BufferedSink
but it was still not as fast as using an external command.
Some other noteworthy features include support for http accept-ranges and access to
build logs via /log/{drv}
.
While we already use harmonia in production, I don’t consider it done yet and
still have the following tasks on my todo list
- Compression, ideally
zstd
-
libnixstore
bindings cleanup and publish as separate crate so other rust
projects can use it. -
nar
streaming usinglibnixstore
-
eris
upstream
option is still missing - providing
.ls
files fornix-index
- nix integration tests
Contributors to the project are much appreciated.
Last but not least, I have to thank eris
, which was used as inspiration while
implementing. @das_j for answering all my nix questions, and @ajs124 for testing.