Nix-build fails, but using nix-shell + cabal etc. works

Help
1

When I try building my project using nix-build, it fails. However, I can build it without any problems using nix-shell and cabal. So I have a way to build my project, but I am curious why nix-build is failing here. Could it be because I once accidentally ran cabal-update outside of a nix shell?

When I build using nix-build, it complains about missing dependencies: lens ==4.19.*

$ nix-build release.nix 
these derivations will be built:
  /nix/store/5bppyi384lyf2x4anfafkam3fcx4cynv-creatur-wains-9.7.drv
building '/nix/store/5bppyi384lyf2x4anfafkam3fcx4cynv-creatur-wains-9.7.drv'...
setupCompilerEnvironmentPhase
Build with /nix/store/vrdzir6ql35x1an44ln0vir3a3pra75f-ghc-8.4.4.
unpacking sources
unpacking source archive /nix/store/w3qndrqryhff2a5yl7x1zh9mqbw6yrwj-creatur-wains
source root is creatur-wains
patching sources
compileBuildDriverPhase
setupCompileFlags: -package-db=/build/setup-package.conf.d -j16 -threaded
[1 of 1] Compiling Main             ( Setup.hs, /build/Main.o )
Linking Setup ...
configuring
configureFlags: --verbose --prefix=/nix/store/vs6v0g2lw2p4gqx4gjyzkcaravmjm9zn-creatur-wains-9.7 --libdir=$prefix/lib/$compiler --libsubdir=$abi/$libname --docdir=/nix/store/dgkdbkiaj5f28xbk7gxsls01vj27n4vz-creatur-wains-9.7-doc/share/doc/creatur-wains-9.7 --with-gcc=gcc --package-db=/build/package.conf.d --ghc-option=-j16 --disable-split-objs --enable-library-profiling --profiling-detail=exported-functions --disable-profiling --enable-shared --disable-coverage --enable-static --disable-executable-dynamic --enable-tests --enable-library-vanilla --enable-library-for-ghci --ghc-option=-split-sections --extra-lib-dirs=/nix/store/9kr7pv6m220gk5518qvkbxdbwq5gzb80-ncurses-6.1-20181027/lib --extra-lib-dirs=/nix/store/rxaclc3vhxbzgfllv7601i85z854h48a-gmp-6.1.2/lib
Using Parsec parser
Configuring creatur-wains-9.7...
CallStack (from HasCallStack):
  die', called at libraries/Cabal/Cabal/Distribution/Simple/Configure.hs:958:20 in Cabal-2.2.0.1:Distribution.Simple.Configure
  configureFinalizedPackage, called at libraries/Cabal/Cabal/Distribution/Simple/Configure.hs:462:12 in Cabal-2.2.0.1:Distribution.Simple.Configure
  configure, called at libraries/Cabal/Cabal/Distribution/Simple.hs:596:20 in Cabal-2.2.0.1:Distribution.Simple
  confHook, called at libraries/Cabal/Cabal/Distribution/Simple/UserHooks.hs:67:5 in Cabal-2.2.0.1:Distribution.Simple.UserHooks
  configureAction, called at libraries/Cabal/Cabal/Distribution/Simple.hs:178:19 in Cabal-2.2.0.1:Distribution.Simple
  defaultMainHelper, called at libraries/Cabal/Cabal/Distribution/Simple.hs:115:27 in Cabal-2.2.0.1:Distribution.Simple
  defaultMain, called at Setup.hs:2:8 in main:Main
Setup: Encountered missing dependencies:
lens ==4.19.*

builder for '/nix/store/5bppyi384lyf2x4anfafkam3fcx4cynv-creatur-wains-9.7.drv' failed with exit code 1
error: build of '/nix/store/5bppyi384lyf2x4anfafkam3fcx4cynv-creatur-wains-9.7.drv' failed

When I build using nix-shell and cabal, everything works, and there’s no problem finding a suitable version of lens.

$ nix-shell
. . .
(nix-shell) $ cabal configure
. . .
 - lens-4.19.2 (requires download & build)
. . .
(nix-shell) $ cabal build
Build profile: -w ghc-8.4.4 -O1
In order, the following will be built (use -v for more details):
. . .
 - lens-4.19.2 (lib:lens) (requires download & build)
. . .
Downloaded   lens-4.19.2
. . .
Linking /home/amy/github/creatur-wains/dist-newstyle/build/x86_64-linux/ghc-8.4.4/creatur-wains-9.7/x/creatur-wains-teaching-test/build/creatur-wains-teaching-test/creatur-wains-teaching-test ...
(nix-shell) $ cabal test
Build profile: -w ghc-8.4.4 -O1
. . .
Running 1 test suites...
Test suite creatur-wains-test: RUNNING...
Test suite creatur-wains-test: PASS
Test suite logged to:
/home/amy/github/creatur-wains/dist-newstyle/build/x86_64-linux/ghc-8.4.4/creatur-wains-9.7/t/creatur-wains-test/test/creatur-wains-9.7-creatur-wains-test.log
1 of 1 test suites (1 of 1 test cases) passed.

Here are the relevant files.

$ cat shell.nix
(import ./release.nix).project.env

$ cat project.nix
{ mkDerivation, base, bytestring, cereal, containers, creatur
, deepseq, directory, filepath, lens, MonadRandom, mtl
, numeric-tools, QuickCheck, random, som, stdenv, test-framework
, test-framework-quickcheck2, unix
}:
mkDerivation {
  pname = "creatur-wains";
  version = "9.7";
  src = ./.;
  isLibrary = true;
  isExecutable = true;
  libraryHaskellDepends = [
    base bytestring cereal containers creatur deepseq directory
    filepath lens MonadRandom mtl random som unix
  ];
  executableHaskellDepends = [
    base cereal containers creatur deepseq lens MonadRandom mtl
    QuickCheck random som test-framework test-framework-quickcheck2
  ];
  testHaskellDepends = [
    base cereal containers creatur deepseq lens MonadRandom mtl
    numeric-tools QuickCheck random som test-framework
    test-framework-quickcheck2
  ];
  homepage = "https://github.com/mhwombat/creatur-wains#readme";
  description = "Artificial agent specialised for for pattern discovery";
  license = stdenv.lib.licenses.bsd3;
}

$ cat release.nix
let
  config = {
    packageOverrides = pkgs: rec {
      haskellPackages = pkgs.haskellPackages.override {
        overrides = haskellPackagesNew: haskellPackagesOld: rec {
          project =
            haskellPackagesNew.callPackage ./project.nix { };
          creatur =
            haskellPackagesNew.callPackage ./creatur.nix { };
          som =
            haskellPackagesNew.callPackage ./som.nix { };
          grid =
            haskellPackagesNew.callPackage ./grid.nix { };
          gray-extended =
            haskellPackagesNew.callPackage ./gray-extended.nix { };
        };
      };
    };
  };

  pkgs = import <nixpkgs> { inherit config; };

in
  { project = pkgs.haskellPackages.callPackage ./project.nix { };
  }

I can provide creatur.nix, gray-extended.nix, grid.nix and som.nix if anyone wants.

System info:

$ nix-shell -p nix-info --run "nix-info -m"
 - system: `"x86_64-linux"`
 - host os: `Linux 4.19.116, NixOS, 19.09.2431.cae3ac8c30b (Loris)`
 - multi-user?: `yes`
 - sandbox: `yes`
 - version: `nix-env (Nix) 2.3.3`
 - channels(amy): `"nixpkgs-18.09.1676.7e88992a8c7"`
 - channels(root): `"nixos-19.09.2431.cae3ac8c30b"`
 - nixpkgs: `/home/amy/.nix-defexpr/channels/nixpkgs`
2

The problem you’re seeing is caused because nixpkgs only contains one version of a given dependency (in some cases multiple versions), but cabal knows about all versions of your dependencies and has a solver built-in to figure out which version to use.

For example, nixpkgs might only contain a derivation for lens-4.18.0. Since that doesn’t meet the version you require in creatur-wains.cabal, nix-build just fails.

However, when building with cabal, cabal looks on Hackage and sees that there is a lens-4.19.0 (or whatever) available and downloads and uses that.

The easiest solution is to either relax the version bounds for the lens dependency in your creatur-wains.cabal file, or to override the nixpkgs version of lens in your release.nix file (with a function like callHackage).

This isn’t related to the problem you’re seeing above, but I’d also suggest you use the callCabal2nix function to build your own packages in your release.nix file, instead of generating the creatur.nix, etc files by hand with cabal2nix and using callPackage on them.

See Nix Haskell Development (2020) for some more suggestions.

2 Likes
3

Thank you so much for the very clear explanation, and for the suggestions.

1 Like