Another thing to keep in mind is that the NAR format is pretty wasteful on its own, no matter if we compress the NAR files or not.
A lot of the files inside a store path don’t actually change, and “exploding” the NAR file and the structure into smaller parts gives a lot of possibility for deduplication.
As part of my work on tvix, I’ve been spending quite some time on tvix-store, which is using a model similar to git trees/blobs (but not quite) as an underlying storage.
It is still able to produce NAR files on demand, just using another internal, content-adressed storage model for the underlying data.
We are already using all that logic in other places, but I also have some code wiring this up to a HTTP server handler, so Nix clients would still be able to “download NAR files” even though we don’t need to store them as such.
If there’s interest to explore this idea further, maybe start ingesting some store paths and see how much dedup we get, and potentially write some fetch-through migration tool, I’d be happy to discuss this further.