Unable to use gzip in derivation to package Crystal LSP server binary

Help
1

I’m attempting to install the binary for Crystalline, the LSP server for Crystal GitHub - elbywan/crystalline: A Language Server Protocol implementation for Crystal. 🔮. It’s offered as a pre-compiled, statically linked binary; installing it naively, using the initial method mentioned in Sander van der Burg's blog: Deploying prebuilt binary software with the Nix package manager should work.

However, I’m not having problems dealing with the linker, but simply, getting the derivation to allow the use of gzip.

The output is as follows:

these derivations will be built:
  /nix/store/n7kqp0jvshg9y438dl7bmby589xfi7bw-crystalline-v0.3.0.drv
building '/nix/store/n7kqp0jvshg9y438dl7bmby589xfi7bw-crystalline-v0.3.0.drv'...
unpacking sources
gzip: warning: GZIP environment variable is deprecated; use an alias or script
gzip: /nix/store/j6vy4jkp38na3d84h4k3jcr3fzfxbhxv-crystalline_x86_64-unknown-linux-gnu.gz has 1 other link  -- unchanged
builder for '/nix/store/n7kqp0jvshg9y438dl7bmby589xfi7bw-crystalline-v0.3.0.drv' failed with exit code 2
error: build of '/nix/store/n7kqp0jvshg9y438dl7bmby589xfi7bw-crystalline-v0.3.0.drv' failed

The derivation itself is straightforward:

{ stdenv, lib, fetchurl, gzip, zlib }:

stdenv.mkDerivation rec {
  pname = "crystalline";
  version = "v0.3.0";

  outputs = [ "out" ];

  src = fetchurl {
    url = "https://github.com/elbywan/crystalline/releases/download/v0.3.0/crystalline_x86_64-unknown-linux-gnu.gz";
    sha256 = "4dd9105b48d3b24e57106179f6c83c0f4321453a8a6f2c5db38d40309af6d07d";
  };

  nativeBuildInputs = [ zlib gzip ];
  buildInputs = [ ];

  unpackPhase = ''
    gzip $src
    chmod u+x crystalline
  '';

  installPhase = ''
    mkdir -p $out/bin
    cp crystalline_x86_64-unknown-linux-gnu/crystalline $out/bin
  '';

  meta = with lib; {
    homepage = https://github.com/elbywan/crystalline;
    description = "A Language Server for Crystal.";
    platforms = platforms.linux;
  };

}

I found some mention of an initial implementation for gzip and NixOS, but there’s nothing conclusive on how it works… Add gzip decompression support · Issue #3256 · NixOS/nix · GitHub

2

Hmm, don’t you want to use gunzip? Also, I think you would use it with ${pkgs.gzip}/bin/gunzip

1 Like
3

I was simply following the build instructions provided by Crystalline; which is as follows:

wget https://github.com/elbywan/crystalline/releases/latest/download/crystalline_x86_64-unknown-linux-gnu.gz -O crystalline.gz &&\
gzip -d crystalline.gz &&\
chmod u+x crystalline

I could find little to no mention of gzip (nor gunzip) in any of the Nix documentation that I went through.

I just tried gunzip though. Unfortunately, it’s still produces the same error.

these derivations will be built:
  /nix/store/g8qn7z46zvbzz0x1fnx25vndn8c73m1q-crystalline-v0.3.0.drv
building '/nix/store/g8qn7z46zvbzz0x1fnx25vndn8c73m1q-crystalline-v0.3.0.drv'...
unpacking sources
gzip: warning: GZIP environment variable is deprecated; use an alias or script
gzip: /nix/store/j6vy4jkp38na3d84h4k3jcr3fzfxbhxv-crystalline_x86_64-unknown-linux-gnu.gz has 1 other link  -- unchanged
builder for '/nix/store/g8qn7z46zvbzz0x1fnx25vndn8c73m1q-crystalline-v0.3.0.drv' failed with exit code 2
error: build of '/nix/store/g8qn7z46zvbzz0x1fnx25vndn8c73m1q-crystalline-v0.3.0.drv' failed

I did attempt to replicate the build instructions provided at NixOS - Nixpkgs 21.05 manual using the buildCrystalPackage function, but I guess crystal2nix hasn’t been maintained to work with newer version of Crystal.

As a last resort, I ended up manually downloading the binary archive, decompressing the gz, compressing it as a tar.gz, and using that for a locally-sourced derivation… how obnoxious… But whatever, Crystalline was installed successfully and is working without any issues.

4

The problem is that it tries to uncompress the file to /nix/store/j6vy4jkp38na3d84h4k3jcr3fzfxbhxv-crystalline_x86_64-unknown-linux-gnu. Copying it first to a writable location and extracting it then works better.
It also depends on Crystal, so I added a little wrapper around it so it doesn’t fail if there isn’t a crystal executable around, though one in your environment will take precedence.

{ stdenv, lib, fetchurl, gzip, crystal, makeWrapper }:

stdenv.mkDerivation rec {
  pname = "crystalline";
  version = "v0.3.0";

  nativeBuildInputs = [ gzip makeWrapper ];

  src = fetchurl {
    url = "https://github.com/elbywan/crystalline/releases/download/v0.3.0/crystalline_x86_64-unknown-linux-gnu.gz";
    sha256 = "4dd9105b48d3b24e57106179f6c83c0f4321453a8a6f2c5db38d40309af6d07d";
  };

  unpackPhase = ''
    cp $src crystalline.gz
    gzip -d crystalline.gz
  '';

  installPhase = ''
    mkdir -p $out/bin
    chmod +x crystalline
    cp crystalline $out/bin
    wrapProgram $out/bin/crystalline --suffix PATH : "${crystal}/bin"
  '';

  meta = with lib; {
    homepage = "https://github.com/elbywan/crystalline";
    description = "A Language Server for Crystal.";
    platforms = platforms.linux;
  };
}

That said, a better way would be to use crystal2nix, but that involves a bit more work.

5

Actually, the problem is that the mentioned derivation executes gzip $src vs either gunzip $src or gzip -d $src. But otherwise yes.

In any case, we definitely want the to build from source in nixpkgs but it turns out that I didn’t update the latest crystal2nix in nixpkgs (you need v0.1.1 for it to grok the shards format). Here’s a quick starting point for building it properly:

default.nix:

{ lib, fetchFromGitHub, crystal, llvm, coreutils, makeWrapper }:

crystal.buildCrystalPackage rec {
  pname = "crystalline";
  version = "0.3.0";

  src = fetchFromGitHub {
    owner = "elbywan";
    repo = pname;
    rev = "v${version}";
    sha256 = "sha256-HInvPO1hm+HyN5dqoEo/vg6g1BdHApXzYZ7BFd8gYMQ=";
  };

  format = "shards";

  buildInputs = [ llvm ];

  shardsFile = ./shards.nix;

  meta = with lib; {
    description = "Code analysis server for the Crystal programming language";
    homepage = "https://github.com/elvywan/crystalline";
  };
}

and shards.nix:

{
  bisect = {
    owner = "spider-gazelle";
    repo = "bisect";
    rev = "v1.2.0";
    sha256 = "0fljyn7abm3mwpca1j7f5zbny4zqs2dr87xm81i68g7lnd8mnq6y";
  };
  priority-queue = {
    owner = "spider-gazelle";
    repo = "priority-queue";
    rev = "bde47d11cb4b4d1ed7680a7f417ff707c434b1e4";
    sha256 = "1m9rmrrvsxy3yv8fgd7idd1x7gj1rw6vx5w3a46mi6ic5281c1gw";
  };
  sentry = {
    owner = "samueleaton";
    repo = "sentry";
    rev = "edcafd89be2658a4bd306ca1096ef9918c727edf";
    sha256 = "107spqjgddz8s55lhz5k6xc73qx2z9sgf48i8szk8pqpvals0f8v";
  };
  version_from_shard = {
    owner = "hugopl";
    repo = "version_from_shard";
    rev = "v1.2.2";
    sha256 = "1iwxp2ya4ryh5zyhhq0xzzcicbv5akwb18a8q22xb9biklrmqnvp";
  };
}