Shared libraries error with cabal repl in nix-shell

Development
1

Hi,

I’m trying to use a nix-shell with cabal using just a “bare” pkgs.haskell.compiler.ghc865, rather than ghcWithPackages.

This is my shell.nix:

{ pkgs ? import <nixpkgs> {} }:

let
  ghc = pkgs.haskell.compiler.ghc865;
  # ghc = haskell.compiler.ghc8102;
in
  pkgs.mkShell {
    buildInputs = with pkgs; [
      cabal-install 
      ghc
      pkgconfig
      zlib.dev
    ];
    shellHook = ''
      eval $(egrep ^export ${ghc}/bin/ghc)
    '';
  }

The cabal project is the template project created with cabal init, with the zlib package added as a dependency and Main.hs calling Codec.Compression.GZip.decompress. Full source here: Main.hs · GitHub.

The executable builds and runs fine with cabal v2-run. But when I try cabal v2-repl, it fails:

[nix-shell:~/dev/test/cabal-test]$ cabal v2-repl
Build profile: -w ghc-8.6.5 -O1
In order, the following will be built (use -v for more details):
 - cabal-test-nix-shell-0.1.0.0 (exe:cabal-test-nix-shell) (ephemeral targets)
Preprocessing executable 'cabal-test-nix-shell' for cabal-test-nix-shell-0.1.0.0..
GHCi, version 8.6.5: http://www.haskell.org/ghc/  :? for help
<command line>: can't load .so/.DLL for: libz.so (libz.so: cannot open shared object file: No such file or directory)
cabal: repl failed for exe:cabal-test-nix-shell from
cabal-test-nix-shell-0.1.0.0.

Interestingly, if I try with ghc-8.10.2, the error becomes a warning: [-Wmissed-extra-shared-lib]. But I would like to use ghc-8.6.5, and not have errors/warnings.

Any ideas? I’m sure there needs to be some environment variable set, or wrapped build tool in the PATH, but I’m not sure which one.

Cheers,

Rodney

1 Like
2

Have you tried zlib not zlib.dev?

1 Like
3

Yes, thanks for the suggestion. I’ve tried all combinations of zlib and zlib.dev with the same result.

4

What does work is:

nix-shell -p "haskell.packages.ghc865.ghcWithPackages (hp: [hp.zlib])"

But I need something that will work with any system library that I add to mkShell { buildInputs }.

I can’t seem to find which of the environment variables makes cabal repl able to find the shared library.

5

When you ask for help with stuff like this, you really need to pin your nixpkgs version so that people trying to help you know what version you’re having problems with.

I tried building the code you had in your gist, and it seemed to basically work for me. I could run cabal build and cabal run and it seemed to work. I tried on both NixOS and Debian, both worked.

Here’s the only change I made to your shell.nix, although I don’t think it caused it to start working:

diff --git a/shell.nix b/shell.nix
index f438a4e..d6ace10 100644
--- a/shell.nix
+++ b/shell.nix
@@ -1,17 +1,22 @@
 { pkgs ? import <nixpkgs> {} }:
 
 let
+  nixpkgs-src = builtins.fetchTarball {
+    # nixpkgs 20.03 as of 2020-09-10
+    url = "https://github.com/NixOS/nixpkgs/archive/4bd1938e03e1caa49a6da1ec8cff802348458f05.tar.gz";
+    sha256 = "0529npmibafjr80i2bhqg22pjr3d5qz1swjcq2jkdla1njagkq2k";
+  };
+
+  pkgs = import nixpkgs-src {};
+
   ghc = pkgs.haskell.compiler.ghc865;
-  # ghc = haskell.compiler.ghc8102;  # becomes warning: [-Wmissed-extra-shared-lib]
 in
-  pkgs.mkShell {
-    buildInputs = with pkgs; [
-      cabal-install 
-      ghc
-      pkgconfig
-      zlib.dev
-    ];
-    shellHook = ''
-      eval $(egrep ^export ${ghc}/bin/ghc)
-    '';
-  }
+
+pkgs.mkShell {
+  buildInputs = with pkgs; [
+    cabal-install
+    ghc
+    pkgconfig
+    zlib.dev
+  ];
+}
6

Thanks @cdepillabout I did mention that cabal v2-run works but cabal v2-repl does not.

Also related, if the project uses some template haskell functions (I don’t know which exactly), the build itself will also fail with the same error.

I am using the nixos-unstable channel, but it also happens on 20.03.

I am wondering whether perhaps this “works as designed”, and I am simply not using it properly by setting the correct environment variable(s).

7

Ah, so it does work with cabal build, but not cabal repl. That’s somewhat unexpected.

I’ve been able to reproduce this problem.

With the following shell.nix:

let
  nixpkgs-src = builtins.fetchTarball {
    # nixpkgs 20.03 as of 2020-09-10
    url = "https://github.com/NixOS/nixpkgs/archive/4bd1938e03e1caa49a6da1ec8cff802348458f05.tar.gz";
    sha256 = "0529npmibafjr80i2bhqg22pjr3d5qz1swjcq2jkdla1njagkq2k";
  };

  pkgs = import nixpkgs-src {};

  ghc = pkgs.haskell.compiler.ghc865;
in

pkgs.mkShell {
  buildInputs = with pkgs; [
    cabal-install
    ghc
    pkgconfig
    zlib.dev
  ];
}

$ nix-shell --command 'cabal repl --ghc-options -v'
Resolving dependencies...
Build profile: -w ghc-8.6.5 -O1
In order, the following will be built (use -v for more details):
 - zlib-0.6.2.2 (lib) (requires build)
 - cabal-test-nix-shell-0.1.0.0 (exe:cabal-test-nix-shell) (first run)
Starting     zlib-0.6.2.2 (lib)
Building     zlib-0.6.2.2 (lib)
Installing   zlib-0.6.2.2 (lib)
Completed    zlib-0.6.2.2 (lib)
Configuring executable 'cabal-test-nix-shell' for cabal-test-nix-shell-0.1.0.0..
Glasgow Haskell Compiler, Version 8.6.5, stage 2 booted by GHC version 8.2.2
Using binary package database: /nix/store/9wvsbqr57k9n6d8vv6b10d04j51f9ims-ghc-8.6.5/lib/ghc-8.6.5/package.conf.d/package.cache
package flags []
loading package database /nix/store/9wvsbqr57k9n6d8vv6b10d04j51f9ims-ghc-8.6.5/lib/ghc-8.6.5/package.conf.d
wired-in package ghc-prim mapped to ghc-prim-0.5.3
wired-in package integer-gmp mapped to integer-gmp-1.0.2.0
wired-in package base mapped to base-4.12.0.0
wired-in package rts mapped to rts
...
Loading package array-0.5.3.0 ... linking ... done.
Loading package deepseq-1.4.4.0 ... linking ... done.
Loading package bytestring-0.10.8.2 ... linking ... done.
*** gcc:
/nix/store/m6h7zh8w6s52clnyskffj5lbkakqgywn-gcc-wrapper-9.2.0/bin/cc -fno-stack-protector -DTABLES_NEXT_TO_CODE '-fuse-ld=gold' -B/home/illabout/.cabal/store/ghc-8.6.5/zlib-0.6.2.2-001db90e889d92c218c08aec62ce5feb0448efa46dc78b2956cd2db24e10ed61/lib --print-file-name libz.so
*** gcc:
/nix/store/m6h7zh8w6s52clnyskffj5lbkakqgywn-gcc-wrapper-9.2.0/bin/cc -fno-stack-protector -DTABLES_NEXT_TO_CODE '-fuse-ld=gold' -B/home/illabout/.cabal/store/ghc-8.6.5/zlib-0.6.2.2-001db90e889d92c218c08aec62ce5feb0448efa46dc78b2956cd2db24e10ed61/lib --print-file-name liblibz.so
*** gcc:
/nix/store/m6h7zh8w6s52clnyskffj5lbkakqgywn-gcc-wrapper-9.2.0/bin/cc -fno-stack-protector -DTABLES_NEXT_TO_CODE '-fuse-ld=gold' -B/home/illabout/.cabal/store/ghc-8.6.5/zlib-0.6.2.2-001db90e889d92c218c08aec62ce5feb0448efa46dc78b2956cd2db24e10ed61/lib --print-file-name z.lib
*** gcc:
/nix/store/m6h7zh8w6s52clnyskffj5lbkakqgywn-gcc-wrapper-9.2.0/bin/cc -fno-stack-protector -DTABLES_NEXT_TO_CODE '-fuse-ld=gold' -B/home/illabout/.cabal/store/ghc-8.6.5/zlib-0.6.2.2-001db90e889d92c218c08aec62ce5feb0448efa46dc78b2956cd2db24e10ed61/lib --print-file-name libz.lib
*** gcc:
/nix/store/m6h7zh8w6s52clnyskffj5lbkakqgywn-gcc-wrapper-9.2.0/bin/cc -fno-stack-protector -DTABLES_NEXT_TO_CODE '-fuse-ld=gold' -B/home/illabout/.cabal/store/ghc-8.6.5/zlib-0.6.2.2-001db90e889d92c218c08aec62ce5feb0448efa46dc78b2956cd2db24e10ed61/lib --print-file-name libz.dll.a
*** gcc:
/nix/store/m6h7zh8w6s52clnyskffj5lbkakqgywn-gcc-wrapper-9.2.0/bin/cc -fno-stack-protector -DTABLES_NEXT_TO_CODE '-fuse-ld=gold' -B/home/illabout/.cabal/store/ghc-8.6.5/zlib-0.6.2.2-001db90e889d92c218c08aec62ce5feb0448efa46dc78b2956cd2db24e10ed61/lib --print-file-name z.dll.a
*** gcc:
/nix/store/m6h7zh8w6s52clnyskffj5lbkakqgywn-gcc-wrapper-9.2.0/bin/cc -fno-stack-protector -DTABLES_NEXT_TO_CODE '-fuse-ld=gold' -B/home/illabout/.cabal/store/ghc-8.6.5/zlib-0.6.2.2-001db90e889d92c218c08aec62ce5feb0448efa46dc78b2956cd2db24e10ed61/lib --print-file-name libz.a
*** gcc:
/nix/store/m6h7zh8w6s52clnyskffj5lbkakqgywn-gcc-wrapper-9.2.0/bin/cc -fno-stack-protector -DTABLES_NEXT_TO_CODE '-fuse-ld=gold' -B/home/illabout/.cabal/store/ghc-8.6.5/zlib-0.6.2.2-001db90e889d92c218c08aec62ce5feb0448efa46dc78b2956cd2db24e10ed61/lib --print-file-name z.a
*** gcc:
/nix/store/m6h7zh8w6s52clnyskffj5lbkakqgywn-gcc-wrapper-9.2.0/bin/cc -fno-stack-protector -DTABLES_NEXT_TO_CODE '-fuse-ld=gold' -B/home/illabout/.cabal/store/ghc-8.6.5/zlib-0.6.2.2-001db90e889d92c218c08aec62ce5feb0448efa46dc78b2956cd2db24e10ed61/lib --print-file-name libz
*** gcc:
/nix/store/m6h7zh8w6s52clnyskffj5lbkakqgywn-gcc-wrapper-9.2.0/bin/cc -fno-stack-protector -DTABLES_NEXT_TO_CODE '-fuse-ld=gold' -B/home/illabout/.cabal/store/ghc-8.6.5/zlib-0.6.2.2-001db90e889d92c218c08aec62ce5feb0448efa46dc78b2956cd2db24e10ed61/lib --print-file-name z
Loading package zlib-0.6.2.2 ... *** Deleting temp files:
Deleting: 
*** Deleting temp dirs:
Deleting: 
<command line>: can't load .so/.DLL for: libz.so (libz.so: cannot open shared object file: No such file or directory)
cabal: repl failed for exe:cabal-test-nix-shell from
cabal-test-nix-shell-0.1.0.0.

It appears to be failing when loading the Haskell package zlib, as expected.

However, it is not clear to me why it would be failing, or how to get GHC to print why linking would fail.

Using ghcWithPackages works. shell.nix:

let
  nixpkgs-src = builtins.fetchTarball {
    # nixpkgs 20.03 as of 2020-09-10
    url = "https://github.com/NixOS/nixpkgs/archive/4bd1938e03e1caa49a6da1ec8cff802348458f05.tar.gz";
    sha256 = "0529npmibafjr80i2bhqg22pjr3d5qz1swjcq2jkdla1njagkq2k";
  };

  pkgs = import nixpkgs-src {};
in
pkgs.mkShell {
  buildInputs = [
    pkgs.cabal-install
    (pkgs.haskell.packages.ghc865.ghcWithPackages (p: [ p.zlib ]))
    pkgs.pkgconfig
  ];
}


$ rm -rf dist-newstyle ~/.cabal/store
$ nix-shell --run 'cabal repl --ghc-options "-v"'
Resolving dependencies...
Build profile: -w ghc-8.6.5 -O1
In order, the following will be built (use -v for more details):
 - cabal-test-nix-shell-0.1.0.0 (exe:cabal-test-nix-shell) (first run)
Configuring executable 'cabal-test-nix-shell' for cabal-test-nix-shell-0.1.0.0..
Glasgow Haskell Compiler, Version 8.6.5, stage 2 booted by GHC version 8.2.2
Using binary package database: /nix/store/fgc4yg5jqipa3hms436a4h62h91w9wsd-ghc-8.6.5-with-packages/lib/ghc-8.6.5/package.conf.d/package.cache
package flags []
loading package database /nix/store/fgc4yg5jqipa3hms436a4h62h91w9wsd-ghc-8.6.5-with-packages/lib/ghc-8.6.5/package.conf.d
wired-in package ghc-prim mapped to ghc-prim-0.5.3
wired-in package integer-gmp mapped to integer-gmp-1.0.2.0
wired-in package base mapped to base-4.12.0.0
wired-in package rts mapped to rts
...
Loading package ghc-prim-0.5.3 ... linking ... done.
Loading package integer-gmp-1.0.2.0 ... linking ... done.
Loading package base-4.12.0.0 ... linking ... done.
Loading package array-0.5.3.0 ... linking ... done.
Loading package deepseq-1.4.4.0 ... linking ... done.
Loading package bytestring-0.10.8.2 ... linking ... done.
Loading package zlib-0.6.2.1 ... linking ... done.
Search directories (user):
Search directories (gcc):
...
Ok, one module loaded.
*Main>

The only thing I really figured out from trying to debug this is that when using ghc directly (without ghcWithPackages), the package.db entry for the zlib Haskell package doesn’t contain any reference to the libz system package:

$ cat ~/.cabal/store/ghc-8.6.5/package.db/zlib-0.6.2.2-001db90e889d92c218c08aec62ce5feb0448efa46dc78b2956cd2db24e10ed61.conf | grep /nix/store

However, when using ghcWithPackages, it does:

$ cat /nix/store/fgc4yg5jqipa3hms436a4h62h91w9wsd-ghc-8.6.5-with-packages/lib/ghc-8.6.5/package.conf.d/zlib-0.6.2.1-EiCuzWuUbh1FOATBAoeQ3G.conf  | grep -C 4 -i zlib-1.2.11
library-dirs: /nix/store/9na6vymqyj88ip7d8rp3kmy9nqsgk6gy-zlib-0.6.2.1/lib/ghc-8.6.5/x86_64-linux-ghc-8.6.5/zlib-0.6.2.1-EiCuzWuUbh1FOATBAoeQ3G
              /nix/store/xhhkr936b9q5sz88jp4l29wljbbcg39k-ncurses-6.1-20190112/lib
              /nix/store/j2fdy70n25zaws892dc95yhj0gfhdxg6-libffi-3.3/lib
              /nix/store/447im4mh8gmw85dkrvz3facg1jsbn6c7-gmp-6.2.0/lib
              /nix/store/fkbpg2lx585gr28fmcs610zk88jl9bd7-zlib-1.2.11-dev/lib
              /nix/store/msp4hm62a75pdidlc3s2ymma2g5hsjjk-zlib-1.2.11/lib
dynamic-library-dirs: /nix/store/9na6vymqyj88ip7d8rp3kmy9nqsgk6gy-zlib-0.6.2.1/lib/ghc-8.6.5/x86_64-linux-ghc-8.6.5
                      /nix/store/xhhkr936b9q5sz88jp4l29wljbbcg39k-ncurses-6.1-20190112/lib
                      /nix/store/j2fdy70n25zaws892dc95yhj0gfhdxg6-libffi-3.3/lib
                      /nix/store/447im4mh8gmw85dkrvz3facg1jsbn6c7-gmp-6.2.0/lib
                      /nix/store/fkbpg2lx585gr28fmcs610zk88jl9bd7-zlib-1.2.11-dev/lib
                      /nix/store/msp4hm62a75pdidlc3s2ymma2g5hsjjk-zlib-1.2.11/lib
include-dirs: /nix/store/fkbpg2lx585gr28fmcs610zk88jl9bd7-zlib-1.2.11-dev/include

Maybe @peti would know how to get something like this to work.

edit: I even tried explicitly passing the library path of zlib and it doesn’t seem to help:

$ nix-shell # using the non-working shell.nix above
$ cabal repl -v --extra-lib-dirs /nix/store/msp4hm62a75pdidlc3s2ymma2g5hsjjk-zlib-1.2.11/lib --ghc-options -v
...
*** gcc:
/nix/store/m6h7zh8w6s52clnyskffj5lbkakqgywn-gcc-wrapper-9.2.0/bin/cc -fno-stack-protector -DTABLES_NEXT_TO_CODE '-fuse-ld=gold' -B/home/illabout/.cabal/store/ghc-8.6.5/zlib-0.6.2.2-001db90e889d92c218c08aec62ce5feb0448efa46dc78b2956cd2db24e10ed61/lib --print-file-name libz.so
*** gcc:
/nix/store/m6h7zh8w6s52clnyskffj5lbkakqgywn-gcc-wrapper-9.2.0/bin/cc -fno-stack-protector -DTABLES_NEXT_TO_CODE '-fuse-ld=gold' -B/home/illabout/.cabal/store/ghc-8.6.5/zlib-0.6.2.2-001db90e889d92c218c08aec62ce5feb0448efa46dc78b2956cd2db24e10ed61/lib --print-file-name liblibz.so
*** gcc:
/nix/store/m6h7zh8w6s52clnyskffj5lbkakqgywn-gcc-wrapper-9.2.0/bin/cc -fno-stack-protector -DTABLES_NEXT_TO_CODE '-fuse-ld=gold' -B/home/illabout/.cabal/store/ghc-8.6.5/zlib-0.6.2.2-001db90e889d92c218c08aec62ce5feb0448efa46dc78b2956cd2db24e10ed61/lib --print-file-name z.lib
*** gcc:
/nix/store/m6h7zh8w6s52clnyskffj5lbkakqgywn-gcc-wrapper-9.2.0/bin/cc -fno-stack-protector -DTABLES_NEXT_TO_CODE '-fuse-ld=gold' -B/home/illabout/.cabal/store/ghc-8.6.5/zlib-0.6.2.2-001db90e889d92c218c08aec62ce5feb0448efa46dc78b2956cd2db24e10ed61/lib --print-file-name libz.lib
*** gcc:
/nix/store/m6h7zh8w6s52clnyskffj5lbkakqgywn-gcc-wrapper-9.2.0/bin/cc -fno-stack-protector -DTABLES_NEXT_TO_CODE '-fuse-ld=gold' -B/home/illabout/.cabal/store/ghc-8.6.5/zlib-0.6.2.2-001db90e889d92c218c08aec62ce5feb0448efa46dc78b2956cd2db24e10ed61/lib --print-file-name libz.dll.a
*** gcc:
/nix/store/m6h7zh8w6s52clnyskffj5lbkakqgywn-gcc-wrapper-9.2.0/bin/cc -fno-stack-protector -DTABLES_NEXT_TO_CODE '-fuse-ld=gold' -B/home/illabout/.cabal/store/ghc-8.6.5/zlib-0.6.2.2-001db90e889d92c218c08aec62ce5feb0448efa46dc78b2956cd2db24e10ed61/lib --print-file-name z.dll.a
*** gcc:
/nix/store/m6h7zh8w6s52clnyskffj5lbkakqgywn-gcc-wrapper-9.2.0/bin/cc -fno-stack-protector -DTABLES_NEXT_TO_CODE '-fuse-ld=gold' -B/home/illabout/.cabal/store/ghc-8.6.5/zlib-0.6.2.2-001db90e889d92c218c08aec62ce5feb0448efa46dc78b2956cd2db24e10ed61/lib --print-file-name libz.a
*** gcc:
/nix/store/m6h7zh8w6s52clnyskffj5lbkakqgywn-gcc-wrapper-9.2.0/bin/cc -fno-stack-protector -DTABLES_NEXT_TO_CODE '-fuse-ld=gold' -B/home/illabout/.cabal/store/ghc-8.6.5/zlib-0.6.2.2-001db90e889d92c218c08aec62ce5feb0448efa46dc78b2956cd2db24e10ed61/lib --print-file-name z.a
*** gcc:
/nix/store/m6h7zh8w6s52clnyskffj5lbkakqgywn-gcc-wrapper-9.2.0/bin/cc -fno-stack-protector -DTABLES_NEXT_TO_CODE '-fuse-ld=gold' -B/home/illabout/.cabal/store/ghc-8.6.5/zlib-0.6.2.2-001db90e889d92c218c08aec62ce5feb0448efa46dc78b2956cd2db24e10ed61/lib --print-file-name libz
*** gcc:
/nix/store/m6h7zh8w6s52clnyskffj5lbkakqgywn-gcc-wrapper-9.2.0/bin/cc -fno-stack-protector -DTABLES_NEXT_TO_CODE '-fuse-ld=gold' -B/home/illabout/.cabal/store/ghc-8.6.5/zlib-0.6.2.2-001db90e889d92c218c08aec62ce5feb0448efa46dc78b2956cd2db24e10ed61/lib --print-file-name z
Loading package zlib-0.6.2.2 ... *** Deleting temp files:
Deleting: 
*** Deleting temp dirs:
Deleting: 
<command line>: can't load .so/.DLL for: libz.so (libz.so: cannot open shared object file: No such file or directory)
CallStack (from HasCallStack):
  die', called at ./Distribution/Client/ProjectOrchestration.hs:1035:55 in main:Distribution.Client.ProjectOrchestration
cabal: repl failed for
cabal-test-nix-shell-0.1.0.0-inplace-cabal-test-nix-shell.

I wish I knew a way to get more insight into what exactly GHC is doing when it is trying to load libz.so.

8

Thanks @cdepillabout - that may be a fix. If I set this this flag, then it works:

cabal configure --constraint='zlib +pkg-config'

Although I can’t see this flag enabled in nixpkgs. So I am probably still missing the correct environment variable/wrapped commands.

9

I think I have “solved” it after a chat with @angerman.

Here is a working shell.nix:

let
  nixpkgs-src = builtins.fetchTarball {
    # nixpkgs 20.03 as of 2020-09-10
    url = "https://github.com/NixOS/nixpkgs/archive/4bd1938e03e1caa49a6da1ec8cff802348458f05.tar.gz";
    sha256 = "0529npmibafjr80i2bhqg22pjr3d5qz1swjcq2jkdla1njagkq2k";
  };

  pkgs = import nixpkgs-src {};

in
  with pkgs;

  mkShell rec {
    buildInputs = [
      cabal-install
      haskell.compiler.ghc865
      pkgconfig
      zlib
    ];

    # Ensure that libz.so and other libraries are available to TH
    # splices, cabal repl, etc.
    LD_LIBRARY_PATH = lib.makeLibraryPath buildInputs;
  }

After a bit more searching, I found this was already answered by @peti on the nixos-dev mailing list, several years ago.

4 Likes
10

Thanks for the follow-up.

To summarize the email thread, it sounds like GCC is used directly by cabal configure, and GCC is patched in nixpkgs to looks at environment variables like NIX_LDFLAGS. nix-shell (or is it stdenv.mkDerivation?) automatically populates NIX_LDFLAGS based on buildInputs. This chain of events causes cabal build to work.

However, in the case of cabal repl, GCC is not used, and GHC is directly responsible for loading and linking libraries. GHC doesn’t know anything about the NIX_LDFLAGS environment variables, so it is not able to find zlib.

The solution you’re proposing is to just modify LD_LIBRARY_PATH so that GHC can find zlib. Given the above info, it makes sense that this would work.

I guess there is also an argument to be made that we should wrap GHC in nixpkgs to make it aware of things like NIX_LDFLAGS. However, in the email thread linked above, peti makes the argument that he would rather have users using ghcWithPackages than raw cabal + GHC.

2 Likes