haskellPackages.stm-containers fails to build

4

@CMCDragonkai Sorry you’re having some trouble with the Haskell package set. Let me try to answer some of your questions.

It shows that stm-containers fails because stm-hamt fails, and stm-hamt fails because it’s dependency constraint is not met specifically primitive and primitive-extras which is still on an older version.

I believe your diagnosis here is correct:

https://hydra.nixos.org/build/98648183/nixlog/2

As a general question I’m quite confused how haskell packages get updated.

The main tool for updating the Haskell package set is hackage2nix.

Also, @peti has weekly streams where he goes through and actually updates the Haskell packages. I think he has uploaded some of them to YouTube as well, so I’d recommend actually watching through one:

I believe hackage2nix is run automatically at regular intervals (once a day)? It pushes an update to the haskell-updates branch. Here is an example of it running:

https://github.com/NixOS/nixpkgs/pull/76982/commits/3741b43294b4d807fc982fa671d3c52065d1ef02

hackage2nix basically looks at two separate sets of information to decide what version of each Haskell package to use:

  1. haskellPackages tracks package versions that are available in the latest Stackage LTS. You can see a list of packages and the corresponding versions here:

    nixpkgs/pkgs/development/haskell-modules/configuration-hackage2nix.yaml at 3741b43294b4d807fc982fa671d3c52065d1ef02 · NixOS/nixpkgs · GitHub

    For each of the packages in this list, the corresponding package in haskellPackages gets set to this same version.

    For instance, in the above list, we have - lens == 4.17.1. If you look at haskellPackages.lens.version for nixpkgs in that commit, you’ll see that it is at 4.17.1.

    hackage2nix ALSO creates a derivation for the latest package version on Hackage (if they are different).

    For instance, you’ll notice that the latest version of lens on Hackage is actually 4.18.1. hackage2nix helpfully creates haskellPackages.lens_4_18_1 that we can use as well.

    Haskell packages that are available in the latest Stackage LTS set are updated semi-automatically by @peti once a week. Here’s an example of this:

    https://github.com/NixOS/nixpkgs/pull/76982/commits/bae6c63431301926de5064754236f1e0eb0d3101

  2. For Haskell packages not in the latest Stackage LTS, hackage2nix just generates the package from the latest version on Hackage.

    For instance, stm-containers is not in the latest Stackage LTS, so hackage2nix just generates the latest version. haskellPackages.stm-containers.version is currently 1.1.0.4 on the haskell-updates branch.

You can probably see what the problem will be here. The primitive package is in Stackage LTS as version 0.6.4.0, so haskellPackages.primitive is also 0.6.4.0. However, stm-hamt is not in Stackage LTS, and requires primitive >= 7.0.

There are a couple ways to get stm-hamt to build. You could send a PR to the haskell-updates branch that does any of the following:

  1. Override stm-hamt to explicitly use the newest version of primitive. You might have to do something similar for stm-containers:

    In pkgs/development/haskell-modules/configuration-common.nix:

    stm-hamt = super.stm-hamt.override {
    primitive = self.haskellPackages.primitive_0_7_0_0;
    primitive-extras = self.haskellPackages.primitive-extras_0_8;
};

    This is basically what you have to do, but I haven’t tested this. (edit: As you can see below, this is not sufficient, although it should give you an idea of basically what needs to be done.)

    The bad thing about this method is that at some point, when a new version of primitive or primitive-extras is released to Hackage, hackage2nix will remove the primitive_0_7_0_0 and primitive-extras_0_8 derivations. This will cause nixpkgs to not be able to be evaluated, and @peti or I will have to fix it up. This is annoying to do.

  2. Create an older stm-hamt derivation.

    There is a list of packages that will be used to generate older versions of packages from Hackage:

    nixpkgs/pkgs/development/haskell-modules/configuration-hackage2nix.yaml at e116b4b331b855653be114b7a26e4a20fdc95b3e · NixOS/nixpkgs · GitHub

    You can see that - aeson < 0.8 is on this list. This means that hackage2nix will generate a derivation like aeson_0_7_0_6.

    You could do something similar for stm-hamt. Add a line like - stm-hamt ==1.2.0.3. This will cause hackage2nix to create the stm-hamt_1_2_0_3 package. Then you could create an override like above on stm-containers to use stm-hamt_1_2_0_3 instead of the normal stm-hamt.

    The bad thing about this method is that this extra-packages list rarely gets updated when it is no longer needed. For instance, you can see the comment on aeson < 0.8 is newer versions don't work with GHC 7.6.x or earlier. However, nixpkgs has long since dropped support for GHC-7.6. It is unlikely anyone is still using this.

    Unless you want to be proactive in supporting stm-hamt and stm-containers, it is likely that it will eventually be broken again in the future.

  3. Force stm-hamt (and stm-containers?) to be an older version.

    You can actually add a line like - stm-hamt ==1.2.0.3 right above the line # LTS Haskell 14.20.

    nixpkgs/pkgs/development/haskell-modules/configuration-hackage2nix.yaml at 3741b43294b4d807fc982fa671d3c52065d1ef02 · NixOS/nixpkgs · GitHub

    This will cause hackage2nix to generate the haskellPackages.stm-hamt as version 1.2.0.3 (and also create a haskellPackages.stm-hamt_1_2_0_4 package, since that is the latest on Hackage).

    stm-containers will then pick this up automatically, and hopefully be able to be built.

    The bad thing about this method is that there is no automatic process to remove the - stm-hamt ==1.2.0.3 line. This could also very easily break in the future.

  4. Get stm-hamt and stm-containers into Stackage. Then we can be sure that they will compile with the existing version of primitive in Stackage/nixpkgs.

    This is by far the easiest for the nixpkgs maintainers, since we often don’t have to do anything. Although it is often the most work for you. And it normally takes a couple months until you can actually use the package you want to use in nixpkgs.

As you can see, Haskell packages in nixpkgs often break if they are not in Stackage LTS.

1 Like