Nix and the haskell eco system; A match made in heaven or a difficult relationship?

maralorn · June 28, 2019, 4:03pm

I am not sure where exactly I am going with this post, but I hope it will be coherent.

Obviously there is a big overlap between the nix and Haskell communities and in fact a lot Haskell projects I came across lately rely on nix on some way or another for building, deploying or developing.

So I have two questions, which bother me, because I feel like those problems don’t have to exist.

1. Build enviroments

It feels like there is no consensus on how to use nix with Haskell. (Which admittedly is a general thing with Haskell and a strength from a different perspective.) And this leads to always having to understand anew how to do things.

My understanding is, when developing software with nix there is one reasonable way to do that.
You have a default.nix to build your project and a shell.nix to create your development environment. This has a lot of advantages (for someone familiar with nix), because you can understand this conceptually and can use it for example with direnv/lorri or any other tooling around this setup. Also you can easily use the default.nix to then install the software to your system.

For a simple Haskell project I had success with something like this. (It’s inspired by some nice blogposts and I think mainly Gabriel439/haskell-nix but I am not sure anymore.)

# default.nix
{ pkgs ? import <unstable> {} }:                                                                                                                               
     
let
  inherit (pkgs) haskellPackages;
  drv = haskellPackages.callCabal2nix "my-project" ./. {};    
     
in    
  {    
    my_project = drv;    
    shell = haskellPackages.shellFor {    
      withHoogle = true;    
      packages = p: [drv];    
      buildInputs = with haskellPackages; [ hlint ghcid cabal-install hindent stack ];    
    };    
  }

and

# shell.nix
(import ./. {}).shell

I realize that this might get a little bit more complicated once I need to pin a ghc version. But this looks to me how one would roughly want to do things. It sets up my development environment nicely (the rest are a few lines in my neovim config) and it should work with cabal alone or with cabal and stack.

Now I’d like to develop an app with reflex-frp, but to get a development environment for that it is recommend to either use obelisk where I don’t use nix-shell myself but a completely opaque to me ob run. Alternatively I can use reflex-platform where I am supposed to run ./try-reflex.
Same problem.

I can come up with 2 theories:

The developers want to make the entry as friction-less as possible and thereby try to abstract everything away, even the fact that nix is being used. This might be a benefit for persons complete foreign to nix, but I am not convinced it does. I feel like this shifts the need to understand nix a little later in the development process but makes it harder.
Often when things are more complicated than seems necessary to me, it means I haven‘t understood the difficulties of the problem they want to solve. So can it be that it is just not feasible to give an example default.nix and shell.nix? (And I mean it would be totally fine to include some extra derivations, nix-functions or tools which the user needs to install first or via those derivations. Things may be complicated for a project. (In this case ghcjs seems to be a culprit.) Also you can use this with extra binary caches.)

So: Why not just give an example default.nix and shell.nix and be done with it? (Or is this done everywhere else in the Haskell world and I picked a bad sample? Not my impression …)

2. Why are so many Haskell packages in nixpkgs broken?

You can see in my example above that I use hindent. I’d love to use brittany instead of hindent, but brittany is marked as broken on 19-03 and unstable. I had similar problems with IHaskell and quite a few other packages. Again I could have picked a bad sample, but still:
That these packages don’t work in nixpkgs baffles me because all this projects are build with nix. Which means there are working nix-expressions. So the transfer into nixpkgs shouldn’t be that hard. Obviously I am wrong about this, please tell me why.

Conclusion

I hope I am not stepping on anyone’s toes here. I realize that a lot of the contributions to all of this happen in the free time of people and I appreciated. I just had some frustrating experiences with Haskell and nix (and I know I am not alone with this). And I’d love to

a) learn why those problems are so frequent and
b) possibly understand how I can circumvent then at least for my purposes. (e.g. how can I get a nix-expression for a reflex project or integrate a working brittany build into my example derivation above.)

Thank you for your attention if you read up to here and I would be very thankful for any insights or corrections.

warbo · June 28, 2019, 8:31pm

My understanding is, when developing software with nix there is one
reasonable way to do that. You have a default.nix to build your
project and a shell.nix to create your development environment.

I think this is true for an application, but not necessarily for a
library, since the library user might want some freedom w.r.t. compiler
and dependency versions, etc. (Application users might want this too,
but calling ‘.override’ seems acceptable for that).

I like https://www.reddit.com/r/NixOS/comments/8tkllx/standard_project_structure
but usually adapt it a little depending on the project. For Haskell
projects I tend to have:

default.nix as a function (e.g. the output of cabal2nix; sometimes I
don’t bother with this file and use haskellSrc2nix instead)

overlay.nix to insert this package into an overlay, e.g. as an
override in haskellPackages and/or haskell.packages.ghcXXX. Any other
overrides that might be needed, e.g. dependencies, tooling, etc. will
be defined here too.

nixpkgs.nix imports a pinned “known-good” nixpkgs revision with
./overlay.nix applied

release.nix imports nixpkgs.nix and exposes concrete versions of the
package that we want to build on CI (e.g. for a particular GHC)

shell.nix imports nixpkgs.nix and provides a dev environment as normal

The nice thing about this is that any of our choices can be ignored,
e.g. if another project is importing our repo with fetchgit or whatever.
All our choices can be ignored by using default.nix (or haskellSrc2nix);
our nixpkgs choice can be ignored by using overlay.nix.

Now I’d like to develop an app with
reflex-frp, but to get a
development environment for that it is recommend to either use
obelisk where I don’t
use nix-shell myself but a completely opaque to me ob run. Alternatively I can use
reflex-platform where
I am supposed to run ./try-reflex. Same problem.

I’ve not used those projects myself, but two things spring to mind:

Hiding complexity from the user is nice, but providing access to the
underlying machinery is usually a good idea. If these projects are
using Nix internally, but I couldn’t use them from my own Nix
expressions, I would make a feature request asking for it.
Keep in mind that Nixpkgs is a (very) glorified pile of bash scripts:
if a project tells you to run some shell commands, try sticking those
in (import {}).runCommand and seeing what happens

As far as brittany is concerned, I’m pinning the version from nixpkgs
18.03. I have an overlay defining a bunch of nixpkgs versions, which
makes it easy to pluck out old packages:

github.com

Warbo/nix-helpers/blob/ea05a229ed56543cd49617de8a6218ff83383bad/nixpkgs.nix

# Pinned nixpkgs repos
self: super:
with builtins // super.lib // {
  inherit (self) die;
};
with rec {
  getNixpkgs = { rev, sha256 }: super.fetchFromGitHub {
    inherit rev sha256;
    owner = "NixOS";
    repo  = "nixpkgs";
  };

  repos = mapAttrs (_: getNixpkgs) repoData;

  repoData = {
    repo1603 = {
      rev    = "d231868";
      sha256 = "0m2b5ignccc5i5cyydhgcgbyl8bqip4dz32gw0c6761pd4kgw56v";
    };
    repo1609 = {

This file has been truncated. show original

mpickering · June 28, 2019, 10:08pm

As far as I am aware most Haskell developer who use nix just use nix to provide ghc and cabal and then use the cabal new-build workflow. It gets too annoying have to manually override everything when you start on a different project.

A promising new direction is the haskell.nix project. It uses the output of cabal new-configure to create a set of derivations which match the build plan that cabal created. This means that less packages are likely to be “broken” but you can still take advantage of a binary cache (if someone caches the precise version of the dependencies you need).

falsifian · July 12, 2019, 3:52pm

Thanks for the suggestion about cabal new-build, @mpickering. In case it’s helpful to anyone else, if you run into an error like this when using cabal new-install:

Configuring library for zeromq4-haskell-0.8.0..
cabal: The pkg-config package 'libzmq' version >=4.0 && <5.0 is required but
it could not be found.

then running a command like nix-shell -p pkgconfig zeromq4, and running the command inside the shell, might help.

Notes:

Nix might later garbage collect the dependencies provided by nix-shell. Adding the arguments --add-root (some dir) to the nix-shell command might help, but it looks like there’s an open issue about this so maybe not.
The command I’m using is cabal new-install ihaskell (ihaskell is broken in nixpkgs). I still don’t have it working, because ghc complains about a type error when building ihaskell itself, but reporting my progress anyway since it might be helpful to others.

mpickering · July 25, 2019, 7:36am

@falsifian The maintainer of IHaskell (@vaibhavsagar) uses nix so I imagine there are nix build scripts directly in the repo you can use rather than via cabal.

falsifian · July 26, 2019, 4:15pm

Thanks @mpickering, you were right, though I did quickly bump into another problem after following your suggestion. For anyone interested in following along, I filed nix: kernel complains "cannot satisfy -package-id HsOpenSSL-0.11.4.16-LYOI62EkAWe2BIzDG3VWh" · Issue #1080 · IHaskell/IHaskell · GitHub