Why does Nix-shell take so long?

Cmdv · August 11, 2020, 9:56am

Hello,

I’ve been trying to use Nix in correlation with Haskell/Stack/Haskell-language-server.

It’s very much been hit and miss, but one thing that seems apparent is that it take 1-2 hours to comple an initial nix-shell!! I’m no on a slow machine either, I have a MBP 2.4GHz 8-core i9, 32GB ram.

As an example I’ve been trying to run this project and specifically this file:

github.com

jeremyjousse/mailparser/blob/0f2666d5bc7a4513e6cc3281521f57b7505f9f55/shell.nix

let

  # Must be the same as the Stack resolver we use!
  ghcCompiler = "ghc883";

  # By default, all functions/values come fron Nixpkgs
in with import nix/default.nix { };

let

  # Wrap Ormolu to add our default extensions that are not enabled by default in Ormolu
  # Run 'ormolu --manual-exts' to find out which extensions are not enabled by default
  # Inspired by https://nixos.wiki/wiki/Nix_Cookbook
  # If for whatever reason you want to reformat the whole code base, run:
  # for f in ./**/*.hs; do
  #   ormolu --mode inplace "$f"
  # done
  ormolu-wrapped = symlinkJoin {
    name = "ormolu";
    paths = [ ormolu ];

This file has been truncated. show original

I commented out docker related things, but I’m now coming up to 2 hours!!
Is there something wrong in the setup or is this just normal?

If that is normal could I have an explanation as to why too?

jonringer · August 11, 2020, 4:44pm

Do you have an overlay which is changing a “low-level” dependency, causing you rebuild everything?

Cmdv · August 11, 2020, 5:55pm

Not that I can see in that repo.

Is that generally a potential cause it would rebuild things to overwrite original with the overlay?

michaelpj · August 11, 2020, 6:03pm

Anything which isn’t cached by [cache.nixos.org](http://cache.nixos.org) you will need to build. What is cached? Ultimately, things that are transitive dependencies of the nixpkgs release jobs.

I suspect that, for example, haskell-language-server is not, and so is not cached. It is a big project with a lot of dependencies, so it will just take a long time (welcome to Haskell). On the plus side, after that you shouldn’t have to build it again.

However, I’m guessing here: it would help a lot if you told us what you end up building!

jonringer · August 11, 2020, 6:09pm

  haskellPackages = dontRecurseIntoAttrs haskell.packages.ghc884;

hydra doesn’t build anything under haskellPackages, so they never get cached

However, I ran the build on my server and was able to pull from cache:

$ nix-build -A haskellPackages.haskell-language-server
...
copying path '/nix/store/m0gkb4f5z1xyhwnvzfisnxznsaz9l92x-haskell-language-server-0.2.2.0' from 'https://cache.nixos.org'...
/nix/store/m0gkb4f5z1xyhwnvzfisnxznsaz9l92x-haskell-language-server-0.2.2.0

Looks like it get cached Hydra - Build 124973027 of job nixpkgs:trunk:haskellPackages.haskell-language-server.x86_64-linux I would point your channel to a different point in time where it is cached

jonringer · August 11, 2020, 6:10pm

I would recommend looking into GitHub - target/lorri: Your project's nix-env , it will add gcroots automatically, so you don’t accidentally clean up your nix-shell’s dependency graph.

This won’t remove the rebuild completely, but it should amortize the cost

Cmdv · August 11, 2020, 6:29pm

ah thank you for the replies, I do use cachix in other projects but I now assume they might still be building other non cached packages and why it felt like everything it taking much longer.

So say when I’ve used a library like Yesod in the past with just stack compilation with additional packages was never more than 5-10 minutes on fresh install. How is the way stack and nix differ?