Statically linking Haskell: fails to link libffi

1

Hey :slight_smile:

I develop a Haskell program for scientific calculations, that I want to link fully statically. It is supposed to run on a HPC cluster at some point. The cluster runs a very old CentOS version, the firewall blocks access to the outside, and there will probably never be Nix. Thus I basically only see the option to build a statically linked executable.

My expression is inspired by the static-haskell-nix example from @nh2

I have a nix like this (interesting parts only, full file is here)

{ mkDerivation, base, binary, hpack # and many more Haskell deps
# C dependencies
, gmp6, libffi
# Linking flags
, staticLinking ? false
}:

in mkDerivation {
  pname = "spicy";
  version = "0.0.1";

  prePatch = "hpack";

  isLibrary = true;
  isExecutable = true;

  libraryHaskellDepends = [ base, binary ]; # And many more ...

  libraryToolDepends = [ hpack ];

  executableHaskellDepends = [ base, binary ];

  testHaskellDepends = [ base, binary ];

  benchmarkHaskellDepends = [ base, binary ];

  # Create a fully static executable. Usually requires a GHC linked against Musl instead of Glibc.
  enableSharedExecutables = !staticLinking;
  enableSharedLibraries = !staticLinking;
  enableStaticLibraries = staticLinking;
  configureFlags = [
    "--ghc-option=-optl=-static"
    "--extra-lib-dirs=${gmp6.override { withStatic = true; }}/lib"
    "--extra-lib-dirs=${libffi.overrideAttrs (old: { dontDisableStatic = true; })}/lib"
  ];
}

add it to the set of Haskell packages via an overlay (see here) and then call it with the pkgsMusl set like this

spicyStatic = pkgsMusl.haskell.packages.ghc921.spicy.override {
  staticLinking = true;
};

The corresponding flake is here (nix build git+https://gitlab.com/theoretical-chemistry-jena/quantum-chemistry/Spicy?ref=feature/static-builds#spicyStatic)

Everything works fine until the linking phase, which fails to link against libffi.

spicy> [2 of 2] Compiling Paths_spicy      ( dist/build/calculations/autogen/Paths_spicy.hs, dist/build/calculations/calculations-tmp/Paths_spicy.o, dist/build/calculations/calculations-tmp/Paths_spicy.dyn_o )
spicy> Linking dist/build/calculations/calculations ...
spicy> /nix/store/y1mackrlqz0cwff5grlv18v1srk9mdrf-binutils-2.35.2/bin/ld: cannot find -lffi
spicy> collect2: error: ld returned 1 exit status
spicy> `cc' failed in phase `Linker'. (Exit code: 1)

Is there something I am missing that static-haskell-nix does, that I am not doing? Any idea?

Best wishes
Phillip

2

In my experience, “inspired by” static-haskell-nix often isn’t sufficient. There are a bunch of work-arounds in static-haskell-nix, and most(?) projects need them.

There are two other things I’d personally be concerned about when looking at your code:

  • It appears you’re building against Nixpkgs master. As of right now, the master branch from static-haskell-nix is targeting an old Nixpkgs (from maybe 8 months ago or so). I have a branch targeting nixos-21.11 as of December 2021: Bump to nixpkgs-21.11 by cdepillabout · Pull Request #111 · nh2/static-haskell-nix · GitHub. This should be relatively similar to Nixpkgs master from the same time period, but I’m not sure how much Nixpkgs master has progressed in the last 4 months.

  • It appears you’re building with GHC-9.2.1. I’ve personally never tested static-haskell-nix with anything other than the default GHC, so I’m not sure how well that will work for you.

If I were you, I would attempt to do the following things in this order:

  1. Double check that justStaticExecutables + patchelf doesn’t work for you. I’m guessing since you’re on a very old version of CentOS that this won’t work, but it would be good to check just in case. (This may be the easiest to maintain if it works.)

  2. If the above doesn’t work, give haskell: pkgsStatic fixes by rnhmjoj · Pull Request #162374 · NixOS/nixpkgs · GitHub a try. You might prefer backporting it to whatever Nixpkgs version you are using (since the current state of the haskell-updates branch is a little shakey: haskellPackages.ghc: 8.10.7 -> 9.0.2 by sternenseemann · Pull Request #160733 · NixOS/nixpkgs · GitHub).

  3. If neither of the above work for you, then try static-haskell-nix. Although you might have to spend time updating the repo to work with a recent Nixpkgs master. Also, I’m not sure if GHC-9.2.1 will give you any trouble.

3

It appears you’re building against Nixpkgs master

I’m building against nixpkgs-unstable. master is a little bit too fragile for me. Also switched to the default GHC for the static builds. Doesn’t change anything, unfortunately.

Double check that justStaticExecutables + patchelf doesn’t work for you.

Couldn’t get it to work unfortunately. The libc version 2.13 on the cluster …

If the above doesn’t work, give haskell: pkgsStatic fixes by rnhmjoj · Pull Request #162374 · NixOS/nixpkgs · GitHub a try.

tried with the haskell-updates branch (okay for a quick exploration at least). Made it even worse unfortunately :grimacing: Now basically no C-library can be found (-lc -lrt -lutil -ldl -lpthread -lm -lffi).

If the above doesn’t work, give haskell: pkgsStatic fixes by rnhmjoj · Pull Request #162374 · NixOS/nixpkgs · GitHub a try.

Will do when I find some time to figure out how this works with flakes and how to integrate it in my repo. I am a little bit lost how exactly it works.

4

I build my applications with pkgsStatic (unlike static-haskell-nix which uses pkgsMusl).
Assuming that:

  1. your haskell project doesn’t depend on TemplateHaskell (dependencies included)
  2. system libraries build under pkgsStatic
  3. you don’t mind building GHC

this is already working as of NixOS 21.11. Here’s a real world example.

5

I build my applications with pkgsStatic

Thank you for this idea :slight_smile: Also tried this but now libffi fails to compile :sweat_smile:

configureFlags: --verbose --prefix=/nix/store/d431jyr48jn2kvkwnwmgissya4zd1i4q-libffi-static-x86_64-unknown-linux-musl-0.1 --libdir=$prefix/lib/$compiler --libsubdir=$abi/$libname --docdir=/nix/store/9azkn5qnnb15xihfxf11hp8y18lvq2a0-libffi-static-x86_64-unknown-linux-musl-0.1-doc/share/doc/libffi-0.1 --with-gcc=x86_64-unknown-linux-musl-gcc --package-db=/build/package.conf.d --ghc-options=-j16 +RTS -A64M -RTS --disable-split-objs --enable-library-profiling --profiling-detail=exported-functions --disable-profiling --disable-shared --disable-coverage --enable-static --disable-executable-dynamic --disable-tests --disable-benchmarks --enable-library-vanilla --disable-library-for-ghci --ghc-option=-split-sections --configure-option=--host=x86_64-unknown-linux-musl --with-ghc=x86_64-unknown-linux-musl-ghc --with-ghc-pkg=x86_64-unknown-linux-musl-ghc-pkg --with-gcc=x86_64-unknown-linux-musl-cc --with-ld=x86_64-unknown-linux-musl-ld --with-ar=x86_64-unknown-linux-musl-ar --with-hsc2hs=x86_64-unknown-linux-musl-hsc2hs --with-strip=x86_64-unknown-linux-musl-strip --hsc2hs-option=--cross-compile --disable-shared --enable-static --disable-shared --disable-shared --enable-static --disable-shared --extra-lib-dirs=/nix/store/vp0r23qy9658dhxjgh53lb241qr542w1-ncurses-6.3-x86_64-unknown-linux-musl/lib --extra-lib-dirs=/nix/store/8kfpacnks7j9q07ygrwm2skv41mfm8s6-libffi-static-x86_64-unknown-linux-musl-3.4.2/lib --extra-lib-dirs=/nix/store/jgqmmg02ard63nzcz5wq1li9pj892yx3-gmp-static-x86_64-unknown-linux-musl-6.2.1/lib --extra-include-dirs=/nix/store/8q9mvv23phj21v6kkijfwxfrif6y5w7i-musl-iconv-1.2.2/include --extra-include-dirs=/nix/store/qw992iargm9ic725dbc10rs4dx8m0wiy-libffi-static-x86_64-unknown-linux-musl-3.4.2-dev/include --extra-lib-dirs=/nix/store/qw992iargm9ic725dbc10rs4dx8m0wiy-libffi-static-x86_64-unknown-linux-musl-3.4.2-dev/lib --extra-lib-dirs=/nix/store/8kfpacnks7j9q07ygrwm2skv41mfm8s6-libffi-static-x86_64-unknown-linux-musl-3.4.2/lib
       > Using Parsec parser
       > Configuring libffi-0.1...
       > Dependency base -any: using base-4.14.3.0
       > Dependency bytestring -any: using bytestring-0.10.12.0
       > Setup: The program 'pkg-config' version >=0.9.0 is required but it could not
       > be found.

I also depend on TemplateHaskell, to derive some instances.

6

TemplateHaskell has just been fixed in the haskell-updates branch in Nixpkgs, the real problem is libffi being broken. I’ve no idea why it can’t find pkg-config, but I can easily fix the build with:

  libffi = overrideCabal (drv: {
    buildTools = (drv.buildTools or []) ++ [ pkgs.pkg-config ];
  }) super.libffi;

Note: I haven’t checked if it’s actually usable, if it is tell me, please.

1 Like
7

I figured out the root cause of the build failure: it was a cross compilation issue.

https://github.com/NixOS/nixpkgs/pull/166781

2 Likes
8

Very nice, both together got it working! :+1: I’ve temporarily switched to your PR branch and enabled the libffi Haskell override and everything is working just fine now. Thank you so much!

1 Like