Specifying location of non-Haskell libraries in Haskell derivation

I’m new to nix, and am trying to get a Haskell program depending on inline-r running. During compliation, this requires libR.so, and I’m having some difficulty getting this to be found in the environment.

default.nix

let
  config = {
    packageOverrides = pkgs: rec {
      haskellPackages = pkgs.haskellPackages.override {
        overrides = haskellPackagesNew: haskellPackagesOld: rec {
          nix-mwe =
            haskellPackagesNew.callPackage ./cabal.nix {
              R = pkgs.rWrapper.override {
                    packages = with pkgs.rPackages; [ dplyr ];
                    };
            };
        };
      };
    };
  };

  pkgs = import <nixpkgs> { inherit config; };
in
  pkgs.haskellPackages.nix-mwe

cabal.nix (generated from package.yaml by cabal2nix)

{ mkDerivation, base, hpack, inline-r, pkgconfig, R, stdenv }:
mkDerivation {
  pname = "nix-mwe";
  version = "0.0.0";
  src = ./.;
  isLibrary = true;
  isExecutable = true;
  libraryHaskellDepends = [ base inline-r ];
  libraryPkgconfigDepends = [ R ];
  libraryToolDepends = [ hpack pkgconfig ];
  executableHaskellDepends = [ base ];
  prePatch = "hpack";
  license = "unknown";
  hydraPlatforms = stdenv.lib.platforms.none;
}

Running nix-build fails with the following message:

CallStack (from HasCallStack):
  die', called at libraries/Cabal/Cabal/Distribution/Simple/Configure.hs:1508:37 in Cabal-2.4.0.1:Distribution.Simple.Configure
  configurePkgconfigPackages, called at libraries/Cabal/Cabal/Distribution/Simple/Configure.hs:579:7 in Cabal-2.4.0.1:Distribution.Simple.Configure
  configure, called at libraries/Cabal/Cabal/Distribution/Simple.hs:596:20 in Cabal-2.4.0.1:Distribution.Simple
  confHook, called at libraries/Cabal/Cabal/Distribution/Simple/UserHooks.hs:67:5 in Cabal-2.4.0.1:Distribution.Simple.UserHooks
  configureAction, called at libraries/Cabal/Cabal/Distribution/Simple.hs:178:19 in Cabal-2.4.0.1:Distribution.Simple
  defaultMainHelper, called at libraries/Cabal/Cabal/Distribution/Simple.hs:115:27 in Cabal-2.4.0.1:Distribution.Simple
  defaultMain, called at Setup.hs:7:8 in main:Main
Setup: The pkg-config package 'libR' version >3.0 is required but it could not
be found.

I’ve tried introducing buildInputs = [ R ], but mkDerivation as called by haskellPackages doesn’t recognise this field. I can get around this problem by calling nix-shell with the R package directly

nix-shell -p R --run "cabal build --extra-lib-dirs='$(nix-build "<nixpkgs>" -A R)/lib'"

however this just punts the error another step down the line.

<command line>: can't load .so/.DLL for: /home/user/.cabal/store/ghc-8.6.5/inline-r-0.10.2-79ba24b2c29ca38b632c248984628abd2639c607774acf4f036765dfab08bf0f/lib/libHSinline-r-0.10.2-79ba24b2c29ca38b632c248984628abd2639c607774acf4f036765dfab08bf0f-ghc8.6.5.so (libR.so: cannot open shared object file: No such file or directory)

Adding more -p arguments including pkgconfig, haskellPackages.inline-r, and adding --extra-lib-dirs='$(nix-build "<nixpkgs>" -A haskellPackages.inline-r)/lib doesn’t change the result.

So my questions are:

  1. How can I help cabal find the relevant libraries inside the nix-shell
  2. How can I include this information in the derivation itself, so I don’t have to append a large number of --extra-lib-dirs to by cabal call every time.

More details:

  • I am using not on NixOS, and am just using nix as a build tool
  • I’m using nixpkgs-unstable, but I got the same results when I tried using nixos-19.09

In case it’s relevant, the Haskell code follows.

Summary
{-# LANGUAGE QuasiQuotes #-}
module MWE where

import Language.R
import Language.R.QQ

main :: IO ()
main = withEmbeddedR def $ void
    [r| library(dplyr)
        print("Test")
    |]

and the package.yaml is

name: nix-mwe

extra-source-files:
- package.yaml

library:
  dependencies:
  - base
  - inline-r >=0.9 && <0.11
  pkg-config-dependencies:
  - libR >3.0
  system-build-tools:
  - pkg-config
  source-dirs: library

executables:
  hledger-report:
    source-dirs: executable
    main: Main.hs
    dependencies:
    - base
    - nix-mwe

I think you want librarySystemDepends for R since it’s a library.

Unfortunately no combination of putting R in {library,executable}{Pkgconfig,System}Depends changes the result.

It bothers me that in my nix-shell it seems to be looking for the library in my .cabal directory, rather than some /nix/store directory. The file that it claims it can’t open is indeed there, but using it would seem to violate purity.

I have no clue about the Haskell stuff here, but have you tried plain pkgs.R?

That actually gets the nix-build command to complete, though nix-shell has the same error as before (I’m sure this is just a matter of writing a shell.nix sufficiently cleverly). Of course, the program now fails on execution because R can’t find the library which is imported (dplyr).

This suggests that perhaps the problem is different versions of R being visible to inline-r and my project: one with the imported package, and one without. Should I be doing a system-wide override to have the necessary libraries available for all packages, or is there a better way of doing this?

@xitian Sorry you’re having trouble with the Haskell infrastructure.

Could you throw up all your current code on GitHub and create a simple reproducible example?


If I had to guess what the problem was without actually testing anything, it is that you don’t actually need pkg-config-dependencies and system-build-tools in your cabal file, since that information should be propagated from the inline-r library.

@cdepillabout I’m new to this, so there’s no need to apologise. Thanks for your help!

Here’s the current code in GitHub form: https://github.com/Xitian9/nix-inline-r-mwe

@xitian I’ve sent a PR trying to get some of this stuff working:

https://github.com/Xitian9/nix-inline-r-mwe/pull/1

However, I couldn’t figure out how to get dplyr working. (If you comment out the line loading dplyr, then everything builds and runs fine, but I assume you need additional libraries like dplyr).

1 Like

@cdepillabout Thanks for your help. Unfortunately I’m still not having much luck getting it to work with libraries in R. Tweag has a few issues listed with similar problems:

Unfortunately, none of the listed solutions seems to be working for me. I’ll post a solution if I find one, but it feels like it might be a lot easier to just work around using inline-r.

1 Like

Aha! I believe I’ve found the solution. The problem seems to be that what the rWrapper call does is wrap the call to R with an export of the R_LIBS_SITE variable set to contain all the libraries. However, the program using inline-r doesn’t call the R executable, and so never gets the locations of the R libraries; they need to be set manually. I added the following to default.nix.

          postInstall = oldAttrs.postInstall or "" + ''
            wrapProgram $out/bin/nix-mwe \
              --prefix 'R_LIBS_SITE' ':' "${self.lib.getLib self.rPackages.dplyr}/library" \
              --prefix 'R_LIBS_SITE' ':' "${self.lib.getLib self.rPackages.R6}/library" \
              --prefix 'R_LIBS_SITE' ':' "${self.lib.getLib self.rPackages.Rcpp}/library" \
              --prefix 'R_LIBS_SITE' ':' "${self.lib.getLib self.rPackages.assertthat}/library" \
              --prefix 'R_LIBS_SITE' ':' "${self.lib.getLib self.rPackages.glue}/library" \
              --prefix 'R_LIBS_SITE' ':' "${self.lib.getLib self.rPackages.magrittr}/library" \
              --prefix 'R_LIBS_SITE' ':' "${self.lib.getLib self.rPackages.pkgconfig}/library" \
              --prefix 'R_LIBS_SITE' ':' "${self.lib.getLib self.rPackages.tibble}/library" \
              --prefix 'R_LIBS_SITE' ':' "${self.lib.getLib self.rPackages.pillar}/library" \
              --prefix 'R_LIBS_SITE' ':' "${self.lib.getLib self.rPackages.crayon}/library" \
              --prefix 'R_LIBS_SITE' ':' "${self.lib.getLib self.rPackages.vctrs}/library" \
              --prefix 'R_LIBS_SITE' ':' "${self.lib.getLib self.rPackages.tidyselect}/library" \
              --prefix 'R_LIBS_SITE' ':' "${self.lib.getLib self.rPackages.purrr}/library" \
              --prefix 'R_LIBS_SITE' ':' "${self.lib.getLib self.rPackages.rlang}/library"
          '';

This list of packages was arrived at by adding sequentially until it ran successfully, and there is surely a better way. Basically the R wrapper script just needs to be extended to be used for other executables too. It’s not immediately obvious to me whether there already exists a way to do this, or if it needs to be written separately.

1 Like

This works for me

{ nixpkgs ? import <nixpkgs> {},
  doBenchmark ? false }:

let

  inherit (nixpkgs) pkgs;

f = { mkDerivation, base, inline-r, R, stdenv
    , template-haskell }:
mkDerivation {
  pname = "mrp";
  version = "1.0.0";
  src = ./.;
  isLibrary = false;
  isExecutable = true;
  executableHaskellDepends = [
    base
    (nixpkgs.haskell.lib.dontCheck inline-r)
    template-haskell ];
  executableSystemDepends = [
    R
    pkgs.rPackages.dplyr
    pkgs.rPackages.ggplot2
  ];
  license = stdenv.lib.licenses.bsd3;
};

  haskellPackages = pkgs.haskellPackages;

  variant = if doBenchmark then pkgs.haskell.lib.doBenchmark else pkgs.lib.id;

  drv = variant (haskellPackages.callPackage f {});

in

  if pkgs.lib.inNixShell then drv.env else drv