Still wrestling with the retrofitting nix to Haskell project

I’m still tying to understand how to get to an initial working condition for my Haskell project.
This was initially a stack project, which did build with the stack nix integration, but then I realized that I needed to refer to the built package somehow in my NixOS configuration.nix, which I think also means that it has to be built by the nixos-rebuild task that processes the configuration.nix, which means that I can’t use --option sandbox false, as one would do if just calling nix-build directly. (At least this is my understanding of how you get a reference to a derivation in the nix-store, for use in configuration.nix, but as my current nix understanding is so poor, I’m not even sure about that).

My project is a Haskell yesod app, and unfortunately it also has a number of dependencies included in it (with stack unpack) that I’ve had to patch up to make them compatible with ghc 9.2.5 and all its contemporary dependencies (at stackage LTS 20.11). Consequently, it has ‘copies’ of packages such as readable-0.3.1, with my own local hacks. As these are entirely local, I haven’t locally bumped the version numbers, merely included the package inside my project an fixed it it. In stack.yml I was referring to these packages in the -packages stanza, and I now have the same in my cabal.package (yes, I’ve converted to cabal too in a hope this would make my nix build easier!).

Reading the NisOS Wiki “How to develop with Haskell and Nix” page, I had some hope that I could have a single default.nix with:

let 
  pkgs = import <nixpkgs> { };
in 
  pkgs.haskellPackages.developPackage {
    root = ./.;
    source-overrides = {
      readable = ./readable-0.3.1;
      snap-server = ./snap-server-1.1.2.0;
      ekg-json = ./ekg-json-0.1.0.6;
      ekg = ./ekg-0.4.0.15;
    };
  }

Sadly though, this results in:
error: infinite recursion encountered

So I’m guessing that the enumeration of things to build in the outer project finds these references and the fact that they are declared as overrides, causes this loop.

There’s perhaps some simple magic that will prevent this (I’m hoping), in such a way that a regular cabal build would still work (i.e. leaving them in the cabal.project), but also such that haskellPackages.developPackage doesn’t go into this screaming loop.

At the moment I feel like I’m playing whack-a-mole to random-walk my way to a working nix build. I’ve also seen various examples of nix-flakes, but these look even more impenetrable atm, so my goal is to absolutely get the simplest thing working (notwithstanding the complications of the stack unpack’ed local packages).

maybe this can help you?

Thanks! I guess that’s yet another direction.

I decided to step right back and start with a basic cabal project, and then one with an embedded cabal project (just unpacked into the project root). I followed https://github.com/Gabriella439/haskell-nix as a template this time (which is yet another pattern, and seems more verbose than the haskellPackages.developPackage approach in my original post in this thread).

With this approach, my main project has a release.nix of:

{ compiler ? "ghc925" }:

let
  config = {
    packageOverrides = pkgs: rec {
      haskell = pkgs.haskell // {
        packages = pkgs.haskell.packages // {
          "${compiler}" = pkgs.haskell.packages."${compiler}".override {
            overrides = haskellPackagesNew: haskellPackagesOld: rec {
              # optparse-applicative =
              #   haskellPackagesNew.callPackage ./optparse-applicative-2.nix { };

              testnix =
                haskellPackagesNew.callPackage ./testnix.nix { };

              embed =
                haskellPackagesNew.callPackage ./embed.nix { };
            };
          };
        };
      };
    };
  };

  pkgs = import <nixpkgs> { inherit config; };

in
  { testnix = pkgs.haskell.packages.${compiler}.testnix;
  }

… and there are two other .nix files, both generated from cabal2nix: one for the main project, and one for the embedded ‘embed’ project.

I had not noticed that the cabal inits for the main and embedded project had resulted in different base package requirements for some reason. So this didn’t work initially, but because these are just toy projects, I just hacked the embed project’s base constraint to be the same as for the top level project and then nix-build release.nix worked fine.

At my level of understanding, the nix expression in release.nix is just a recipe and I’ll need to spend more time understanding it, as well as seeing if this basic approach can now be ported to the project I’ve actually being trying to nixify.

However, I’m still left puzzling over how many different approaches I’ve read about in various places and their corresponding different examples, and I’d prefer to be following an approach that is considered modern, stable and likely to persist for some reasonable time in the future. I seem to be reading that flakes are the new hotness too, but looking at a few examples of these, it seems they are even more complex and hence an even steeper learning curve.

While I don’t know if this conforms to my criteria for being currently thought of as a reasonable approach, the haskellPackages.developPackage approach did seem simpler, but I’m betting that this is only because it wasn’t doing half the things it needed to (such as actually building the embed project properly (and of course it has the problem of ‘infinite recursion’ which is the current impasse. So, if it’s possible to remediate the original config, that would still be great as a comparative.

There are many approaches to building haskell projects.

from simple ‘2nix’ converters… to full on haskell.nix from IOHK.

If you see an example a large production project. Maybe not a great example for beginner …, but if you get good with nix, you’ll be able to build quite complex projects, and keep them building over time.

I don’t know how upto date this page is, but it may help.

https://nixos.wiki/wiki/Haskell

There are a couple things to unpack here.

First, there are two widely-used approaches to building Haskell projects completely with Nix:

  1. The Haskell infrastructure in Nixpkgs
  2. haskell.nix

All of the projects that have come up as suggestions so far are all using the APIs provided by the Haskell infrastructure in Nixpkgs (well, other than haskell.nix). So while it seems like there are a bunch of different approaches, the question really boils down to “Do you want to use the Haskell infrastructure from Nixpkgs, or haskell.nix?”

My suggestion for you would be to use the Haskell infrastructure from Nixpkgs, as it seems like you’re trying to do.

The Haskell infrastructure from Nixpkgs provides a few widely-used functions you may have seen, like haskellPackages.mkDerivation, haskellPackages.callPackage, haskellPackages.callCabal2nix (and similar helper functions like haskellPackages.callHackage), haskellPackages.developPackage and haskellPackages.shellFor. I’ve tried to give an overview of these functions here (but this explanation is assuming some level of familiarity with Nix):

There are various projects that either show a certain way of working with these functions (like
GitHub - Gabriella439/haskell-nix: Nix and Haskell in production), or build light abstractions on top of these functions (like GitHub - srid/haskell-template: Haskell project template using Nix + Flakes + VSCode (HLS)), or build new functionality on top of these functions (like GitHub - cdepillabout/stacklock2nix: Easily build a Haskell project from a stack.yaml.lock file with Nix).

My recommendation for a beginner would be to go with haskellPackages.developPackage to get started, since it is relatively easy to use. But, it sounds like you’re having trouble with it. Would you be able to send a link to the repo of Haskell and Nix code you’re trying to use? For someone looking to help you, it is generally easiest just to get a link to a repo that can be cloned and locally run nix-build to see what the error is.

In a lot of ways, the choice of whether or not to use flakes is orthogonal to the choice of method for building Haskell. Most approaches for building Haskell projects will work with or without flakes. (That said, there are some libraries/templates that are built around flakes and would be hard to use without flakes, like GitHub - srid/haskell-template: Haskell project template using Nix + Flakes + VSCode (HLS))

My personal advice would be to avoid flakes for now, and only come back to it when you’re a little more comfortable with Nix. (Although there are also people that would give the opposite advice to beginners.)

1 Like

Thanks for the comprehensive remarks there cdepillabout.

My toy build with a single trivial embedded project in the root of my main project and following the ‘Gabriella439’ pattern I noted earlier seemed promising as it built fine. I’ve since applied this to my real project, where my ‘stack unpack’ derived embedded projects all have a cabal2nix derived .nix file (along with one for the main project), which are each referenced by a “release.nix” package.

The behaviour I’m getting in my mail project however, is:

building '/nix/store/qxd4sisl7ps9ibjblkjx3dyk68g70zmh-ekg-json-0.0.0.0-r7.cabal.drv'...

trying https://hackage.haskell.org/package/ekg-json-0.0.0.0/revision/7.cabal
  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                 Dload  Upload   Total   Spent    Left  Speed
  0    56    0     0    0     0      0      0 --:--:-- --:--:-- --:--:--     0
curl: (22) The requested URL returned error: 404
error: cannot download ekg-json-0.0.0.0-r7.cabal from any mirror
error: builder for '/nix/store/qxd4sisl7ps9ibjblkjx3dyk68g70zmh-ekg-json-0.0.0.0-r7.cabal.drv' failed with exit code 1;
       last 7 log lines:
       >
       > trying https://hackage.haskell.org/package/ekg-json-0.0.0.0/revision/7.cabal
       >   % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
       >                                  Dload  Upload   Total   Spent    Left  Speed
       >   0    56    0     0    0     0      0      0 --:--:-- --:--:-- --:--:--     0
       > curl: (22) The requested URL returned error: 404
       > error: cannot download ekg-json-0.0.0.0-r7.cabal from any mirror
       For full logs, run 'nix log /nix/store/qxd4sisl7ps9ibjblkjx3dyk68g70zmh-ekg-json-0.0.0.0-r7.cabal.drv'.
error: 1 dependencies of derivation '/nix/store/nsx2b7l65dznikgll3i912zc1bz4qnqn-ekg-json-0.0.0.0.drv' failed to build
error: 1 dependencies of derivation '/nix/store/qi9bjn2vrllfal9hshdkiyp32fdr1la6-scoti2-0.0.5.drv' failed to build

Ergo, it thinks that this dependency should be fetched from a mirror somewhere.
The obvious question is therefore: what makes nix think that a derivation should be fulfilled by checking various caches online, versus simply building it from the local sources, which is what I thought it would do by having the src property in mkDerivation set to a local subdirectory of my project (for this ekg-json package the cabal2nix generated nix file is):

{ mkDerivation, aeson, base, ekg-core, lib, text
, unordered-containers
}:
mkDerivation {
  pname = "ekg-json";
  version = "0.0.0.0";
  src = ./ekg-json;
  revision = "7";
  editedCabalFile = "19jkmlnw65kqpd0y8f6b260dvl5hkjw9aw5jxnwzr9niqdyyjazh";
  libraryHaskellDepends = [
    aeson base ekg-core text unordered-containers
  ];
  homepage = "https://github.com/tibbe/ekg-json";
  description = "JSON encoding of ekg metrics";
  license = lib.licenses.bsd3;
}

As you can probably tell I have experimented with setting the version to 0.0.0.0, which is an attempt to make nix think “ha, this can’t possibly be some package shared on the internet, but rather just some part of the parent project that the author has decided to wrap as a distinct package in the main project”. As far as I can tell, it is able to determine in my toy case not to go to the internet for a package called “embed”. So, at the moment, this difference (the condition that determines whether nix thinks this is a completely locally sourced package, or one that might be cached online) is my main line of enquiry. I didn’t really expect merely fudging the version numbers in the dependencies to change that, but I also wanted to be sure that it was the obvious dependency (from the local source) that it was trying to fulfil, rather than some other transitive dependency on the same package that I wasn’t aware of.

So, I suppose I feel like I might be close to a working pattern, because I think my ‘in vitro’ version works, but my ‘in vivo’ version doesn’t, so in theory I’m just looking for a difference. Nevertheless, differences can be subtle at times, so perhaps this is obvious to someone out there.

If I had to guess, you probably need to remove these two lines.

But it would be easier to debug if you could share a github repo with the full project you’re trying to build (or at least some sort of minimal reproducer). It is difficult to debug things without being able to see the full code.

Here’s a project:
https://bitbucket.org/levans/testnix/src/master/

The main project has a couple of local dependencies, which are embedded whole at the top of the main project directory tree, both supposed to be private dependencies with no requirement for the build to go out to a repo to lookup cached artifacts or anything. The two dependencies are “embed”, which was never a public package and “workdays” which was copied into the project from a public package and is an example of where I’ve had to copy in projects to make minor fixes, usually due to them not having been updated to later versions of their own dependencies.

Following the '‘Gabriella439’ pattern, I used cabal2nix to create nix files describing the derivations for the main project and the two embedded projects (embed and workdays), and a ‘release.nix’ which references these derivation building nix files and projects the final package.

Doing a ‘cabal build’ will build everything fine.
Doing a ‘nix-build’ will result in my problem… the configuration clearly causes nix-build to think that it should go out to mirrors to check if there’s are cached versions of the workdays artifacts, whereas I just want it to build from the local sources and never consider that these might be public dependencies that might have been built and cached before.

In the case of workdays nix-build reports:
error: Package ‘workdays-0.1.1’ in /nix/store/w7i4cmni72cmnqp35zs38q9i06nbg5q5-nixpkgs/nixpkgs/pkgs/development/haskell-modules/hackage-packages.nix:310996 is marked as broken, refusing to evaluate.

… which is exactly why I would have done a ‘stack unpack’ into my project in the first place, in order to ‘unbreak’ the package. But, of course the main problem here is simply that I don’t know how to assert that this package should be considered built from local sources and not considered for remote fetching.

The fact that the cabal2nix build the workdays.nix file to include “src = ./workdays;” ought (I thought think!) to be sufficient to tell nix-build to do this, and it seems happy to build the ‘embed’ dependency.

Anyway, hopefully this is sufficiently illustrative of my problem.

Alright, here’s a patch to fix your problems:

patch to get everything building
diff --git a/release.nix b/release.nix
index 562e950..dba7193 100644
--- a/release.nix
+++ b/release.nix
@@ -2,19 +2,19 @@
 
 let
   config = {
-    packageOverrides = pkgs: rec {
-      haskell = pkgs.haskell // {
-        packages = pkgs.haskell.packages // {
-          "${compiler}" = pkgs.haskell.packages."${compiler}".override {
-            overrides = haskellPackagesNew: haskellPackagesOld: rec {
+    packageOverrides = prev: rec {
+      haskell = prev.haskell // {
+        packages = prev.haskell.packages // {
+          "${compiler}" = prev.haskell.packages."${compiler}".override {
+            overrides = haskellPackagesNew: haskellPackagesOld: {
               testnix =
                 haskellPackagesNew.callPackage ./testnix.nix { };
 
               embed =
                 haskellPackagesNew.callPackage ./embed.nix { };
-                
-              worddays =
-                haskellPackagesNew.callPackage ./workdays.nix { };  
+
+              workdays =
+                prev.haskell.lib.dontCheck (haskellPackagesNew.callPackage ./workdays.nix { });
             };
           };
         };
@@ -22,9 +22,15 @@ let
     };
   };
 
-  pkgs = import <nixpkgs> { inherit config; };
+  nixpkgs-src = builtins.fetchTarball {
+    # Descriptive name to make the store path easier to identify
+    name = "nixpkgs-for-testnix";
+    # Commit hash for nixpkgs-unstable from 2023-03-07
+    url = "https://github.com/NixOS/nixpkgs/archive/21eda9bc80bef824a037582b1e5a43ba74e92daa.tar.gz";
+    sha256 = "0vqnwhdwwn5sklm0wv80v03l9p0aqdg4rdsmw3g8k7723ivrfnyb";
+  };
+
+  pkgs = import nixpkgs-src { inherit config; };
 
 in
-  { testnix = pkgs.haskell.packages.${compiler}.testnix;
-  }
-
+pkgs.haskell.packages.${compiler}.testnix

A couple things to note here:

  • It looks like your main problem was that workdays was misspelled as worddays. (Always hate these types of problems!)

    How could you have figured this out yourself? This is a tough one, since workdays is already available in the Haskell package set, and it is not obvious that it is not actually being overridden. The thing that tipped me off is that it is saying it is marked broken, but your workdays.nix file obvious does not have broken set anywhere.

  • Unless you have a very good reason, always, always, always pin your Nixpkgs. It is not reproducible otherwise!

  • The tests for workdays don’t appear to work with the dependencies in our package set, so I’ve just disabled the tests.

I also took the liberty of rewriting somethings how I’d actually write them. This patch should be applied on top of the previous one:

my suggestions
diff --git a/.gitignore b/.gitignore
new file mode 100644
index 0000000..0ed5c1a
--- /dev/null
+++ b/.gitignore
@@ -0,0 +1,3 @@
+
+# Nix result files
+result
diff --git a/embed.nix b/embed.nix
deleted file mode 100644
index 388da4a..0000000
--- a/embed.nix
+++ /dev/null
@@ -1,8 +0,0 @@
-{ mkDerivation, base, lib }:
-mkDerivation {
-  pname = "embed";
-  version = "0.1.0.0";
-  src = ./embed;
-  libraryHaskellDepends = [ base ];
-  license = lib.licenses.mit;
-}
diff --git a/release.nix b/release.nix
index dba7193..3f94009 100644
--- a/release.nix
+++ b/release.nix
@@ -7,14 +7,9 @@ let
         packages = prev.haskell.packages // {
           "${compiler}" = prev.haskell.packages."${compiler}".override {
             overrides = haskellPackagesNew: haskellPackagesOld: {
-              testnix =
-                haskellPackagesNew.callPackage ./testnix.nix { };
-
-              embed =
-                haskellPackagesNew.callPackage ./embed.nix { };
-
-              workdays =
-                prev.haskell.lib.dontCheck (haskellPackagesNew.callPackage ./workdays.nix { });
+              testnix = haskellPackagesNew.callCabal2nix "testnix" ./. { };
+              embed = haskellPackagesNew.callCabal2nix "embed" ./embed { };
+              workdays = prev.haskell.lib.dontCheck (haskellPackagesNew.callCabal2nix "workdays" ./workdays-0.1.1 { });
             };
           };
         };
diff --git a/testnix.nix b/testnix.nix
deleted file mode 100644
index 32a7510..0000000
--- a/testnix.nix
+++ /dev/null
@@ -1,11 +0,0 @@
-{ mkDerivation, base, embed, lib, workdays }:
-mkDerivation {
-  pname = "testnix";
-  version = "0.1.0.0";
-  src = ./.;
-  isLibrary = false;
-  isExecutable = true;
-  executableHaskellDepends = [ base embed workdays];
-  license = lib.licenses.mit;
-  mainProgram = "testnix";
-}
diff --git a/workdays.nix b/workdays.nix
deleted file mode 100644
index c494bd5..0000000
--- a/workdays.nix
+++ /dev/null
@@ -1,11 +0,0 @@
-{ mkDerivation, base, containers, doctest, hspec, lib, time }:
-mkDerivation {
-  pname = "workdays";
-  version = "0.1.1";
-  src = ./workdays-0.1.1;
-  libraryHaskellDepends = [ base containers time ];
-  testHaskellDepends = [ base containers doctest hspec ];
-  homepage = "https://github.com/stackbuilders/workdays";
-  description = "Workday calculations";
-  license = lib.licenses.mit;
-}

This uses callCabal2nix so that you don’t have to manually update files like testnix.nix with the cabal2nix command every time the .cabal file changes.

And then here is my suggested organization for code:

my suggested organization
diff --git a/default.nix b/default.nix
new file mode 100644
index 0000000..427cc99
--- /dev/null
+++ b/default.nix
@@ -0,0 +1,4 @@
+
+{...}:
+
+(import ./nixpkgs.nix).testnix
diff --git a/nixpkgs.nix b/nixpkgs.nix
new file mode 100644
index 0000000..306d8d2
--- /dev/null
+++ b/nixpkgs.nix
@@ -0,0 +1,27 @@
+
+let
+
+  project-overlay = final: prev: {
+    # Create a new haskell package set so that we could still easily get
+    # development tools from haskell.packages.ghc925.
+    testnix-haskell-packages = prev.haskell.packages.ghc925.override {
+      overrides = haskellPackagesNew: haskellPackagesOld: {
+        testnix = haskellPackagesNew.callCabal2nix "testnix" ./. { };
+        embed = haskellPackagesNew.callCabal2nix "embed" ./embed { };
+        workdays = final.haskell.lib.dontCheck (haskellPackagesNew.callCabal2nix "workdays" ./workdays-0.1.1 { });
+      };
+    };
+
+    testnix = final.haskell.lib.justStaticExecutables final.testnix-haskell-packages.testnix;
+  };
+
+  nixpkgs-src = builtins.fetchTarball {
+    # Descriptive name to make the store path easier to identify
+    name = "nixpkgs-for-testnix";
+    # Commit hash for nixpkgs-unstable from 2023-03-07
+    url = "https://github.com/NixOS/nixpkgs/archive/21eda9bc80bef824a037582b1e5a43ba74e92daa.tar.gz";
+    sha256 = "0vqnwhdwwn5sklm0wv80v03l9p0aqdg4rdsmw3g8k7723ivrfnyb";
+  };
+in
+
+import nixpkgs-src { overlays = [ project-overlay ]; }
diff --git a/release.nix b/release.nix
deleted file mode 100644
index 3f94009..0000000
--- a/release.nix
+++ /dev/null
@@ -1,31 +0,0 @@
-{ compiler ? "ghc925" }:
-
-let
-  config = {
-    packageOverrides = prev: rec {
-      haskell = prev.haskell // {
-        packages = prev.haskell.packages // {
-          "${compiler}" = prev.haskell.packages."${compiler}".override {
-            overrides = haskellPackagesNew: haskellPackagesOld: {
-              testnix = haskellPackagesNew.callCabal2nix "testnix" ./. { };
-              embed = haskellPackagesNew.callCabal2nix "embed" ./embed { };
-              workdays = prev.haskell.lib.dontCheck (haskellPackagesNew.callCabal2nix "workdays" ./workdays-0.1.1 { });
-            };
-          };
-        };
-      };
-    };
-  };
-
-  nixpkgs-src = builtins.fetchTarball {
-    # Descriptive name to make the store path easier to identify
-    name = "nixpkgs-for-testnix";
-    # Commit hash for nixpkgs-unstable from 2023-03-07
-    url = "https://github.com/NixOS/nixpkgs/archive/21eda9bc80bef824a037582b1e5a43ba74e92daa.tar.gz";
-    sha256 = "0vqnwhdwwn5sklm0wv80v03l9p0aqdg4rdsmw3g8k7723ivrfnyb";
-  };
-
-  pkgs = import nixpkgs-src { inherit config; };
-
-in
-pkgs.haskell.packages.${compiler}.testnix

A couple things to note here:

  • I’ve removed release.nix and split into nixpkgs.nix and default.nix. Having a default.nix makes it easy to run nix-build (instead of having to do nix-build release.nix).
  • I’m using overlays in nixpkgs.nix, since they are a little more flexible than config.packageOverrides.

And finally, I’d add a shell.nix:

add a shell.nix
diff --git a/nixpkgs.nix b/nixpkgs.nix
index 306d8d2..393dc24 100644
--- a/nixpkgs.nix
+++ b/nixpkgs.nix
@@ -13,6 +13,18 @@ let
     };
 
     testnix = final.haskell.lib.justStaticExecutables final.testnix-haskell-packages.testnix;
+
+    testnix-dev-shell = final.testnix-haskell-packages.shellFor {
+      withHoogle = false;
+      packages = hpkgs: [ hpkgs.testnix hpkgs.embed ];
+      buildInputs = [ ];
+      nativeBuildInputs = [
+        final.cabal-install
+        final.ghcid
+        final.haskell.packages.ghc925.haskell-language-server
+        final.haskell.packages.ghc925.fourmolu
+      ];
+    };
   };
 
   nixpkgs-src = builtins.fetchTarball {
diff --git a/shell.nix b/shell.nix
new file mode 100644
index 0000000..fab6356
--- /dev/null
+++ b/shell.nix
@@ -0,0 +1,4 @@
+
+{...}:
+
+(import ./nixpkgs.nix).testnix-dev-shell

This allows you to run nix-shell to get into an environment with cabal, ghcid, HLS, etc.

3 Likes

Thanks cdepillabout.

I’ll be back on this within a couple of days. Sorry about the ‘worddays’ issue - more haste, less speed as I whipped up the minimal example, Hope it didn’t take too much of the time you put in, but really appreciate your engagement and look forward to being able to have your working version as a springboard to further learning.

1 Like

I’m curious about this. How does it make it easier? Why not just replace the package set entirely? I assume there’s a good reason, I just don’t know what it is. :slight_smile:

1 Like

I talked about this a little bit here:

Basically, there are a set of Haskell tools that must be built with with the same version of GHC you’re using. A few that come to mind are HLS, weeder, and fourmolu/ormolu (maybe?).

If you override the top-level Haskell package set (like haskell.packages.ghc925 in the above example), you have a chance of accidentally changing dependencies of these developer tools, which can cause them to no longer compile.

Here’s a simple example. Imagine you have a local package that needs to be built with ghc-9.2.5, but uses aeson-1.5.6.0:

final: prev: {
  haskell = prev.haskell // {
    packages = prev.haskell.packages // {
      ghc925 = prev.haskell.packages.ghc925.override {
        overrides = haskFinal: haskPrev: rec {
          my-cool-pkg = haskFinal.callCabal2nix "my-cool-pkg" ./. { };
          aeson = haskFinal.callHackage "aeson" "1.5.6.0" { };
        };
      };
    };
  };
}

With this overlay, while you’ll be able to build haskell.packages.ghc925.my-cool-pkg, you likely wouldn’t be able to build haskell.packages.ghc925.haskell-language-server, since haskell-language-server requires aeson-2.0. Although, most users will want to use haskell.packages.ghc925.haskell-language-server in their development environment.

(This is just an example. In practice, aeson-1.5 might not compile with ghc-9.2.5, or haskell-language-server might be able to compile with older versions of aeson.)

Alternatively, if you create a separate package set, you can override whatever you want.

For example:

final: prev: {
  my-haskell-pkg-set = prev.haskell.packages.ghc925.override {
    overrides = haskFinal: haskPrev: rec {
      my-cool-pkg = haskFinal.callCabal2nix "my-cool-pkg" ./. { };
      aeson = haskFinal.callHackage "aeson" "1.5.6.0" { };
    };
  };
}

With just this overlay, you’ll be able to build my-haskell-pkg-set.my-cool-pkg, as well as haskell.packages.ghc925.haskell-language-server.

The takeaway here is that when working with Nixpkgs, it is very easy to create your own packages and package sets. It often helps to do so, so that you don’t accidentally override something that is used in another place.

1 Like

That’s excellent advice, thanks!

1 Like

I’ve applied all the recommended changes to by project and I think I’m on a good track now, with one small exception (see below). So, thank you once again.

I did have an issue with some of my ‘stack unpacked’ locally patched packages having an ‘x-revision’ property in their cabal file. This was causing a check against (presumably) a cached version from hackage:

building '/nix/store/g2k7r9093yhjm4gpz1sql0afr2qr680f-ekg-0.4.0.15-r8.cabal.drv'...

trying https://hackage.haskell.org/package/ekg-0.4.0.15/revision/8.cabal
  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                 Dload  Upload   Total   Spent    Left  Speed
100  2031    0  2031    0     0   4229      0 --:--:-- --:--:-- --:--:--  4231
error: hash mismatch in fixed-output derivation '/nix/store/g2k7r9093yhjm4gpz1sql0afr2qr680f-ekg-0.4.0.15-r8.cabal.drv':
         specified: sha256-CctSaZxaWnVBa20y1mNXsdfmY1tPhejZ7BYKTriTO1I=
            got:    sha256-1uSIWaifu+I0lvhxWB5EpB+X2sYnwrnbgfSbkvoGZRY=
error: 1 dependencies of derivation '/nix/store/9nl7hc0s99lbg40xx8i3s2fvzjpwfyz5-ekg-0.4.0.15.drv' failed to build
error: 1 dependencies of derivation '/nix/store/k4w0hdhliql1jy1030rqa9simmbkg76y-scoti2-0.0.6.drv' failed to build

It would be nice to know if there’s a way to even turn this off in the nix expression somehow, for the future, but it appears sufficient to simply comment out that line in the cabal file and the build proceeds from local sources without referencing the externally cached package.

I’ve run into a new problem however, which is that some downloaded dependency is failing its own tests and stopping the build cold. I’m sure it’s possible to add something to prevent any of the test phases from running in a build. Perhaps cdepillabout you would be able to suggest (what I’m hoping is) a small additional change to your recommendations for my test project to have if suppress the test phase in all packages (local and downloaded dependencies)? Perhaps it would also be nice to have turn the tests on again discretely for the main package though, even if the default is off for everything in the build.

You can use lib.dontCheck for this. Something like

my-cool-package = haskFinal.lib.dontCheck (haskFinal.callCabal2nix ...)
1 Like

Thanks robx. I have lib.dontCheck on all the top level referenced dependencies, and have just put it on my main project package too, but I still get the failed test stopping the build.

This test failure is in wai-middleware-metrics (which is a transitive dependency). See the output below.

I’m guessing that ‘lib.dontCheck’ only applies to a single specific directly-referenced package, rather than transitively (?).


Preprocessing test suite 'metrics-test' for wai-middleware-metrics-0.2.4..
Building test suite 'metrics-test' for wai-middleware-metrics-0.2.4..
[1 of 2] Compiling Network.Wai.Metrics ( Network/Wai/Metrics.hs, dist/build/metrics-test/metrics-test-tmp/Network/Wai/Metrics.o )
[2 of 2] Compiling Main             ( tests.hs, dist/build/metrics-test/metrics-test-tmp/Main.o )

tests.hs:83:30: warning: [-Wdeprecations]
    In the use of ‘assert’
    (imported from Test.Tasty.HUnit, but defined in tasty-hunit-0.10.0.3-4CvgRDeDGH53qr3ZijTKzh:Test.Tasty.HUnit.Orig):
    Deprecated: "This class or type seems dubious. If you have a good use case for it, please create an issue for tasty. Otherwise, it may be removed in a future version."
   |
83 |                              assert $ between 0.1 0.11 (Distribution.mean s)
   |                              ^^^^^^
Linking dist/build/metrics-test/metrics-test ...
running tests
Running 1 test suites...
Test suite metrics-test: RUNNING...
Metrics tests
  Request counter must be incremented in middleware:   OK
    +++ OK, passed 100 tests.
  Error counter must be incremented in middleware:     OK (0.01s)
    +++ OK, passed 100 tests.
  Request time average must be measured in middleware: FAIL (0.47s)
    ./Test/Tasty/HUnit/Orig.hs:203:

    Use -p '/Request time average must be measured in middleware/' to rerun this test only.

1 out of 3 tests failed (0.50s)
Test suite metrics-test: FAIL
Test suite logged to: dist/test/wai-middleware-metrics-0.2.4-metrics-test.log
0 of 1 test suites (0 of 1 test cases) passed.
error: builder for '/nix/store/hknn0w3p9zzimkwdld614hchab0xvw5q-wai-middleware-metrics-0.2.4.drv' failed with exit code 1;
       last 10 log lines:
       >     +++ OK, passed 100 tests.
       >   Request time average must be measured in middleware: FAIL (0.47s)
       >     ./Test/Tasty/HUnit/Orig.hs:203:
       >
       >     Use -p '/Request time average must be measured in middleware/' to rerun this test only.
       >
       > 1 out of 3 tests failed (0.50s)
       > Test suite metrics-test: FAIL
       > Test suite logged to: dist/test/wai-middleware-metrics-0.2.4-metrics-test.log
       > 0 of 1 test suites (0 of 1 test cases) passed.
       For full logs, run 'nix log /nix/store/hknn0w3p9zzimkwdld614hchab0xvw5q-wai-middleware-metrics-0.2.4.drv'.
error: 1 dependencies of derivation '/nix/store/qg3j2bm59r90qffmh48x6ga7awksdcd9-scoti2-0.0.6.drv' failed to build

Ah, I think I have it.
I found the following stack overflow.

This suggests my nixpkgs.nix file can be modified as follows:

let
  project-overlay = final: prev: {
    # Create a new haskell package set so that we could still easily get
    # development tools from haskell.packages.ghc925.
    testnix-haskell-packages = prev.haskell.packages.ghc925.override {
      overrides = haskellPackagesNew: haskellPackagesOld: {
        mkDerivation = args: haskellPackagesOld.mkDerivation (args // {
          doCheck = false;
        });
        testnix =

Probably all the redundant haddock generation can be suppressed in a similar way, which I’ll try sometime soon.

Glad you got everything working!

Let me try to answer a few of your questions.

I think I’ve seen that too. I’m not really sure what to do here, but it sounds like just manually removing the x-revision thing is probably the easiest thing to do.

However, I’ve never seen this problem when I use a function like callHackage or callCabal2nix. Since people generally don’t vendor dependencies in the Haskell community, I guess this x-revision thing isn’t too bad of a problem.

Yup, your thinking here is correct. haskell.lib.dontCheck doesn’t apply transitively.

It is still easy to add an override to any transitive dependency though. In your Haskell packages overlay, you’d just add a line like:

metrics-test = final.haskell.lib.dontCheck haskellPkgsOld.metrics-test;

You can do that for any dependency that is failing to build.

Although, your solution of overriding haskell.packages.ghc925.mkDerivation and setting doCheck = false also works. The downside of this though is that it changes all Haskell derivations in the package set, so you have to locally build everything. You won’t be able to get any of your Haskell packages from the NixOS cache.

1 Like