I’ve been using nix to develop Haskell packages for about 8 months now. While my overall experience with nixpkgs is great, there are also some frustrating problems with the nixpkgs Haskell infrastructure. I figured out that others might have had the same problems and a good start for improvement would be to identify the problems. So I decided to make a summary.
Nixpkgs tries to have only one version of each Haskell package at the same time – the version of every Haskell package in the package set is determined in the following way: If this package exists in stackage nightly then use the version from stackage, otherwise use the latest version from hackage. This works well if you are okay with the versions from the package set, you get prebuilt binaries from hydra, and all of the infrastructures (
developPackageetc.) work out of the box, but if the version does not satisfy your need then tough luck, you’ll have to override the dependency. For developers, this means that you’ll need to add an overlay to your
haskellPackages, and for maintainers, to override the inputs in
configuration-commons.nix. This is not only tedious but also requires constant maintenance: if the package updates then you’ll need to update the overrides as well, otherwise it will break again.
The Haskell infrastructure is really lacking in documentation, the Haskell section in nixpkgs manual points to cabal2nix’s documentation which hasn’t received any meaningful update for years, is really sparse and does not cover the functions in the library at all. This makes it really hard for newcomers to contribute to the Haskell infrastructure. The interaction between hydra and nixpkgs and the update process (which is usually done by peti) should also be documented.
While we currently have some amazing tools to automate the packaging process, there is still a lot of manual work required to keep the packages buildable. Most of the time, when a package gets marked as broken, it is because of one of the following reasons:
- The versions provided by nixpkgs does not satisfy the constraint of the package
- cabal2nix cannot find a derivation that provides the external dependencies (
pkgconfig-depends) required by the package
- The test broke because of sandboxing
- Its dependencies broke
Currently, all of these cases require manual intervention and it’s really tedious to unbreak packages.
Multi-package project support
Let cabal to do the heavy lifting
I view nixpkgs’s version fixing as inherently flawed, you are trying to create a huge global package set where every package works with every other package and it’s just not gonna work without immense efforts. On the contrary, we can use per-package package sets, where a different package set is generated for each package which is guaranteed to satisfy all the constraints. How do we do this, you might ask. Well, we can use cabal to do the heavy lifting. Cabal will write the build plan to
plan.jsonafter dependency resolution and we could generate the package set from it. We can also get multi-package project support for free. This is the approach used by haskell.nix.
Keeping it pure
Another problem arises when we chose to use cabal – how do we keep the build pure when cabal requires reading an index that changes over time? Well, we filter out all the entries that are created after a specific date so that the output is fixed no matter when do we download the index. We then configure cabal to use this truncated index as a local repository to resolve dependencies, this way we can keep the build pure and deterministic. We also need to generate nix expressions for all versions of the packages on hackage.
Mixing with version fixing
One problem with using per-package package sets is that depending on the package set it’s in, one package can potentially evaluate to multiple derivations. This will not only increase the workload on hydra but also the binary cache size. To solve this we could use per-package package sets only on a few packages that do not work well with the global package set but mixing version fixing with per-package package sets is gonna be complex as cabal does not permit packages to depend on multiple versions of the same package.