One of the most downloaded artifacts nixpkgs is currently served with gz compression as it comes from github instead of the more effective xz compression which is used for all other artifacts coming from cache.nixos.org.
The size of the current nixpkgs unstable archive:
gz compressed: 26.5 MB
xz compressed: 18 MB
For some of you sitting behind gigabit, it might not make a difference, but for many other people time waiting for nixpkgs to finish downloading accumulates during the day.
The interesting thing is, that a xz compressed version of each nixpkgs is already available from cache.nixos.org. If one imports a remote nixpkgs and access it’s path attribute, one can observe that nix re-downloads a xz version of that nixpkgs archive.
It is there, but it seems like there is no way of finding it without first downloading the badly compressed nixpkgs.tar.gz.
Is there a way to fix that? can we somehow make a cheap API that provides us with the xz URL for a given nixpkgs revision?
The interesting thing is, that a xz compressed version of each nixpkgs
is already available from cache.nixos.org. If one imports a remote
nixpkgs and access it’s path attribute, one can observe that nix
re-downloads a xz version of that nixpkgs archive.
As an aside, would you be able to provide an example or otherwise say
more about that process? I’m not quite sure what you mean and how you
obtain that xz-compressed package set after downloading the tar.gz.
Other than that, xz compression, especially with the numbers you
provided, would be very welcome on my internet connection :))
> curl https://cache.nixos.org/nar/1gf7v3819akxna3i9ffqaffbcb72kjg15ss2qdhx2q7x4z6lpcgw.nar.xz > /tmp/nixpkgs.nar.xz
% Total % Received % Xferd Average Speed Time Time Time Current
Dload Upload Total Spent Left Speed
100 17.7M 100 17.7M 0 0 2481k 0 0:00:07 0:00:07 --:--:-- 2728k
It seems like it. I just noticed that nixpkgs inputs locked via flake.lock will be downloaded from the cache in xz compression instead of from github. The flake.lock contains a narHash which allows nix to get the artifact from the cache.
Therefore the problem will basically be solved as soon as flakes is used.