Backport ghc-8.6.5 to 19.03

1

Is it reasonable to backport newer versions of GHC to older nixos releases?

Currently, ghc-8.6.5 is available in nixpkgs master, but it is not available in the release-19.03 branch. Currently only ghc-8.6.4 is available in the release-19.03 branch. I’d like to backport ghc-8.6.5 to release-19.03.

I feel like I saw somewhere that only a limited number of GHC versions were made available in nixpkgs, because they took so much time to compile.

The reason I would like ghc-8.6.5 available in 19.03 is because of the way stack works with nix. When using stack with a recent LTS, it assumes that newer versions of GHC will be available from your system-wide nix channel. Internally, it runs a command similar to the following:

$ nix-shell -E \
    "with (import <nixpkgs> {}); let inputs = [haskell.compiler.ghc865 git gcc gmp]; ... in runCommand ''myEnv'' { buildInputs = inputs; ... } \"\"" \
    --run "/nix/store/j34ijwzfhn8c2i87z11s6a1r46dyirnz-stack-1.9.3.1/bin/stack build"

You can see that it is trying to pull in haskell.compiler.ghc865 as a buildInput, which doesn’t exist, since I’m on the 19.03 channel.

It is possible to work around this by using a different NIX_PATH:

$ stack --nix-path nixpkgs=/some/path/to/master/branch/of/nixpkgs build

However, it would be nice if recent versions of GHC (really just 8.6.5 right now) were available in the stable nixpkgs channel.

I’d be willing to send a PR backporting GHC-8.6.5 if this would be accepted.

2

Disclaimer: I am not a haskell person.

While we could certainly do a backport why don’t we try to fix the Haskell tooling around nixpkgs? Can’t you declare in some stack config file that you’d like to use ghc864 instead? Maybe You could teach stack to specify the nixpkgs channel?

3

Thanks for the response @andir.

Can’t you declare in some stack config file that you’d like to use ghc864 instead?

Sort of. stack has a config file where I would be able to specify that I want to use a different “resolver”. A resolver is basically a specific set of known working Haskell packages with a corresponding compiler version. If I change the resolver to a different version (in order to get a different compiler version), then I have no assurance that the project will actually compile and continue to work. It is similar to a “lock file” that other programming languages use.

Ideally I’d like to use the compiler version specified by the resolver for this project, which uses ghc865.

Maybe You could teach stack to specify the nixpkgs channel?

I think this would be another possibility. And there are probably tons of other ways to improve the stack tooling to work better with Nix.

But I feel like it is somewhat orthogonal to the issue of backporting new GHC compiler releases to the stable nixpkgs.

In my first post, I tried to motivate having newer compiler releases in the stable channel with a specific problem I’m having with stack. But this actually affects any projects tracking the stable branch. Anyone building a Haskell package using nix tooling (instead of stack) and tracking 19.03 (instead of unstable) could benefit from having a newer GHC version in 19.03. In fact, this is the exact situation we’re in at work.

In our big project at work, we’re currently tracking 19.03. We’re using a stack resolver that uses ghc864, so no problems yet, but in the next month or so we’d like to update to a newer resolver that uses ghc865. Since ghc865 isn’t in 19.03, we’re going to have to get it from somewhere else (unstable?). It would be convenient for us if ghc865 was in 19.03.

I also think it makes sense to backport newer GHC releases to the current stable channel, since it doesn’t really hurt anything (other than putting more strain on hydra), and it is convenient for people doing Haskell development. I am somewhat surprised that it hasn’t already been backported, which is why I created this thread.

4

Having a stable version means focusing on fixing issues rather than updating programs, however unharmful the update may seem.

In my short experience, the haskell ecosystem is very sensitive to any change so I am not sure it’s reasonable to backport ghc8.6.5. And if we do I don’t see any reason not to udpate other packages.

Meanwhile you can pin nixpkgs to unstable, use your own nixpkgs with a ghc8.6.5 and cachix to cache the binaries. GitHub - typeable/nixpkgs-stackage: Stackage overlay for Nixpkgs may be of interest.

1 Like
5

One thing that could make this easier is if we provied aliases to the latest ghc patch version in haskell.compiler.ghc*. So for instance stack could refer to haskell.compiler.ghc86 instead of haskell.compiler.ghc865. I don’t think anyone has objections to this?

6

Let me try to sum up the questions and possible solutions from this thread, along with my own veiws.

I think there have been 3 possible courses of action suggested.

  1. Add (backport) a ghc865 compiler and package set to 19.03, leaving ghc864 as-is

    This would add a ghc865 compiler and package set to 19.03, in addition to the ghc864 package set. This would mean that ghc865 would be available at haskell.compiler.ghc865. There would be a corresponding package set available at haskell.packages.ghc865. ghc864 would continue to be available at haskell.compiler.ghc864 (as well as a top-level ghc, haskellPackages.ghc, etc).

    This was my original suggestion.

    I don’t think there is really any downside to this approach, other than making hydra build one more additional haskell compiler. We could tell hydra to NOT build any of the packages in the new haskell.packages.ghc865 package set, which would reduce the load on hydra if necessary.

    This would give stack users an easy way to use newer LTS versions that use ghc-8.6.5. This would be helpful to both me personally, and my company.

  2. Update the ghc-8.6 version in 19.03 from ghc-8.6.4 to ghc-8.6.5

    This would mean completely getting rid of the ghc-8.6.4 compiler in nixpkgs and replacing it with ghc-8.6.5. This would mean that we get a new haskell.compiler.ghc865, and the older haskell.compiler.ghc864 is removed.

    I think this is what @teto thought I was suggesting in my original post.

    The GHC minor releases (which, for example, would mean moving from ghc-8.6.4 to ghc-8.6.5) are always bug fixes. In general they fix bugs (sometimes major bugs) in the compiler, but don’t cause any Haskell package breakage. This is in contrast to major releases (which, for example, would mean moving from ghc-8.6.5 to ghc-8.8.1), which often cause a huge amount of breakage. I can’t recall ever seeing a minor release cause a Haskell package to stop compiling, unless it was a justifiable bug fix in the compiler.

    As a point of comparison, stackage LTS releases will bump compiler minor versions without bumping the LTS major version. For example, lts-13.11 requires ghc-8.6.3, while lts-13.12 requires ghc-8.6.4. Bumping compiler minor versions is generally seen as “safe”.

    I believe that in nixpkgs stable, it is generally okay to bump package minor versions for bug fixes. This would mean that it should generally be okay to bump GHC minor versions.

  3. Add an alias haskell.compiler.ghc86 that stack could refer to

    I think this might be a nice thing to do. It would make it a little easier to refer to GHC-8.6 when you don’t really care about the minor version.

    However, my concern with this is that it might cause people to accidentally use older compilers. For instance, some GHC releases have big bugs. For instance, ghc-8.6.1 had a bug that made it give logic errors in certain situations. I would be worried that by using haskell.compiler.ghc86, I might get into the situation where I would be mistakenly using one of these broken ghc releases. I definitely want to avoid that.

My opinion is that we should go with either (1) or (2). (1) is slightly safer (since everything in haskellPackages will continue working as-is), but puts slightly more load on hydra. (2) is more in-line with how the rest of nixpkgs operates, and is still pretty safe.

I’m neutral towards (3). I’d like to make sure I am using a compiler version that matches the LTS version when using stack (or at least a newer compiler version).

1 Like
7

@teto

https://github.com/typeable/nixpkgs-stackage may be of interest.

You may be well aware of this, but for anyone else interested, there are generally 3 ways I know of to reliably build a stack package with nix-build:

  1. stackage2nix (and the corresponding nixpkgs-stackage repo)

    This generates a nix Haskell package set from a stack.yaml file. I’ve never really played around with stackage2nix. I was under the impression that the community generally recommended stack2nix instead of stackage2nix.

  2. stack2nix

    Like stackage2nix, this is a CLI tool that generates a Haskell package set from a stack.yaml file. This Haskell package set can be imported in nix and built with nix-build. It can also be easily used with nix-shell and cabal new-build.

    We are using this at my current company. It works well.

    However, the lead developer (@domenkozar) is considering moving to haskell.nix, so stack2nix may not be maintained in the future.

  3. haskell.nix

    This is a complete re-thinking of the Haskell architecture in nixpkgs. I think the long-term goal for this is to get fully merged back into nixpkgs.

    I haven’t played around with haskell.nix too much, but I think they also provide a CLI tool for converting a stack.yaml into a Haskell package set, similar to stack2nix. However, it seems to me like there is more community interest in haskell.nix compared to stack2nix.

If I were starting a new Haskell project right now, I would recommend looking into either stack2nix or haskell.nix. As of right now, haskell.nix might be the smarter bet for the future, although you might see more breaking changes over the next couple months than you would with stack2nix.

1 Like
8

There is also option to use Mathew Pickering’s GHC overlay as it’s shown in Stack on nixos fails when ghc not available in nixpkgs, e.g. ghc802 · Issue #4539 · commercialhaskell/stack · GitHub for example.
Or you could just import GHC8.6.5 from unstable @cdepillabout

9

@qrilka Thanks for the additional option! Here’s a link to @mpickering’s repo:

GitHub - mpickering/old-ghc-nix: Old and New GHC

However, I probably should have made this more clear in the initial post, but this thread was originally to discuss whether or not ghc-8.6.5 belongs in nixpkgs-19.03.

In my previous post, I think I have convinced myself that it does, so I may open a PR to add it.

This thread was to see if there were any good reasons not to add it (other than the additional burden to Hydra).

10

I stand by the “any change, however minor, is a risk”. Security fixes make the risk worth backporting. I can understand the will of backporting on ubuntu/other distribs because setting up repositories is a pain but the great strength of nix(os) is the quick iteration cycle and ease to mix repositories.
What do you gain from it being backported you can’t have with the other solutions ?

NB: In the end, I don’t really mind, I just play the devil’s advocate (sorry :stuck_out_tongue: ). I guess ultimately the choice should go to haskell maintainers since they are the ones sinking time into this.