Fetchurl with compressed files

Hi everyone! How can I decompress files downloaded using fetchurl? I’ve figured out that postFetch hook can be used for that, but how can I rename the file in the nix store?

Example: /nix/store/xxxxx-filename.json.zst/nix/store/xxxxx-filename.json

Thank you!

To decompress files you want fetchzip. If it does not deal with a particular compression format, add stuff to unpackPhase. Since usually these archive contains multiple files, i guess they will be created in a subfolder based on the pname?

Thank you for your suggestion. I’m dealing with regular files, not archives (e.g. “.txt.zst”). I just want to decompress them.

Then you can do something like:

stdenv.mkDerivation {name = "test.json"; src = ./test.json.zst; unpackPhase = ":"; nativeBuildInputs = [ zstd ]; installPhase = ''zstd -d "$src" -o "$out"'';} 


nix-repl> :b stdenv.mkDerivation {name = "test.json"; src = ./test.json.zst; unpackPhase = ":"; nativeBuildInputs = [ zstd ]; installPhase = ''zstd -d "$src" -o "$out"'';} 

This derivation produced the following outputs:
  out -> /nix/store/xrrwznjgy10pykb6ham5ch32dlgmgc0p-test.json

$ cat /nix/store/xrrwznjgy10pykb6ham5ch32dlgmgc0p-test.json
{a: 42}

And with the url:

stdenv.mkDerivation {
  name = "test.json";
  src = pkgs.fetchurl {
    url = "";
    hash = "sha256-wNX3RotfJ043JWRvfokCF3VY+/bArkakruoUxNiyMdI=";
  unpackPhase = ":";
  nativeBuildInputs = [ zstd ];
  installPhase = ''zstd -d "$src" -o "$out"'';

will output a derivation pointing to something like:

$ cat /nix/store/zg201kixdw2n0x9ljll3xz1dvax1wjxa-test.json
{a: 42}

Now, I’m not sure why you want to do that, but if you do some more steps, you might prefer to move the compression part to unpackPhase, and use installPhase/buildPhase to do other stuff on this json.

1 Like

Thank you! What does “:” in unpackPhase mean? I know that you can skip phases with “true”, but this is something I’ve never seen.

It is just replacing the unpackPhase with an empty phase doing nothing. Guess that you could equivalently use dontUnpack = true;

If I want to turn it into a function, can I use fetchzip, but skip unpacking and only decompress on “non-archives”? So, basically I guess it should be similar to your example but with unpackPhase from fetchzip?

I’ve seen the source code for fetchzip, can it be accomplished by using unpackPhase = "unpackFile ..."?