Understand how overlays feature used to build Haskell project

Help
1

Hi, I am trying to understand how to use overlays over my haskell project. I have asked in the #nixos irc channel but then I thought it will be better to documented here (so I can get back here to learn more). Here it goes:

  1. I have haskell project consist of the following structure:

    backend
|-- auth-service
|   |-- auth-api
|   |   |-- auth-api.nix #this is a cabal2nix generated file
|   |-- auth-client
|   |   |-- auth-client.nix #this is a cabal2nix generated file
|   |-- auth-server
|   |   |-- auth-server.nix #this is a cabal2nix generated file
|   |-- default.nix
|-- backend.nix
|-- packages.nix
|-- other-service
|   |-- .... #another haskell project folder
|   |-- default.nix
|-- ..... #another folder and files

    There is also frontend, and other *-service folder.

  2. Inside default.nix, which live inside my auth-service folder I have the following code:

    newHaskellPackages: oldHaskellPackages:
{
  auth-api    = oldHaskellPackages.callPackage ./auth-api/auth-api.nix { };
  auth-server = oldHaskellPackages.callPackage ./auth-server/auth-server.nix { };
  auth-client = oldHaskellPackages.callPackage ./auth-client/auth-client.nix { };
}

    According to this nixpkgs-mozilla example repo, I follow the servo-overlay.nix example, which assuming that my default.nix will be overriding haskellPackages (I put the .nix code no. 3 below) and in this packages, auth-server and auth-client will depend on auth-api

  3. Inside backend folder, I have 2 .nix files which are:
    a. backend.nix which will be my main build for all *-service haskell project. I am planning to have default.nix which contains derivation that will have $out/services/{__name of package__} but I think for now I just want to build it using backend.nix. The code as follows:

    let
  # GHC compiler version
  compiler = "ghc822";

  # Import 3rd party dependencies from nixdeps folder
  beam          = import ../nixdeps/beam.nix;

  # Modifikasi paket yang ada pada variabel haskellPackages
  overlays = [
    (newPkgs: oldPkgs: {
      haskellPackages =  oldPkgs.haskell.packages.${compiler}.override {
        overrides = newHaskellPkgs: oldHaskellPkgs: {
          auth-service =  import ./auth-service {}; 
        } // (import ./packages.nix newHaskellPkgs oldHaskellPkgs beam); -- got error here probably.
      };
    })
  ];

  # Config declaration
  config = { allowUnfree = true; };

  # Nixpkgs pointing stable channel
  nixpkgs = import ../nixdeps/nixpkgs.nix; 
  pkgs    = import nixpkgs { inherit config overlays; };

in
  { inherit (pkgs.haskellPackages) auth-service; }

    note that all files in folder nixdeps are only a result of nix-prefetch-git function so I can call it either with callCabal2Nix or callPackage

  4. In addition to no. 3 above, what I want is that my auth-service and any other *-service project use upstream github packages like in this case I use beam package and try to override haskellPackages to use the upstream version of that package. I put it under packages.nix below:

    newHaskellPkgs: oldHaskellPkgs: beam:
let
  optparse-generic_1_3_0 =
    oldHaskellPkgs.callCabal2nix "text" (import ../nixdeps/optparsegeneric.nix) {};
  swagger2 =
    oldHaskellPkgs.callCabal2nix "swagger2" (import ../nixdeps/swagger2.nix) {};

  # --- Beam Related Package --- #
  beam-core =
    oldHaskellPkgs.callCabal2nix "beam-core" "${beam}/beam-core" {};
  beam-migrate =
    oldHaskellPkgs.callCabal2nix "beam-migrate" "${beam}/beam-migrate" {};
  beam-postgres =
    oldHaskellPkgs.callCabal2nix "beam-postgres" "${beam}/beam-postgres" {};

in
  { inherit optparse-generic_1_3_0 swagger2 beam-core beam-migrate beam-postgres }

So I run nix-build backend.nix and also try nix build -f backend.nix but it doesn’t say error and also not generated result folder.

I have tried this with the old ways modifiedHaskellPackages.callPackage style which modifiedHaskellPackages = haskellPackages.override { ... } and it works as it is intended to.

So now I’m a little bit confused how does the overlays pass through the arg newHaskellPkgs and oldHaskellPkgs to my haskell packages. Or did I miss some step above?

Any help to point me to the right direction is appreciated. I provide the gist here
BR,

2

There is a lot to take in for a casual response. At a quick glance it looks ok overall. What I would change is give the extended Haskell package set a new name like myHaskellPackages. That way it avoid unnecessary rebuilds if you want to also use say “stack” from hackellPackages.

After that change you should be able to run nix-build default.nix -A myHaskellPackages. auth-service

1 Like
3

Hi… thank you for your input. So my understanding with myHaskellPackages is that it will create new environment for my package right?

Also what i want to ask is re the above quoted code. As you can see that it only imports the following ./auth-service/default.nix:

newHaskellPackages: oldHaskellPackages:
{
  auth-api    = oldHaskellPackages.callPackage ./auth-api/auth-api.nix { };
  auth-server = oldHaskellPackages.callPackage ./auth-server/auth-server.nix { };
  auth-client = oldHaskellPackages.callPackage ./auth-client/auth-client.nix { };
}

Can I say that once we do auth-service = import ./auth-service {}; it will also pass newHaskellPkgs and oldHaskellPkgs to the auth-service so those two will be treated as newHaskellPackages and oldHaskellPackages inside my ./auth-service/default.nix function?

and when you mean nix-build default.nix -A myHaskellPackages. auth-service, it is nix-build backend.nix where my above code is backend.nix right?

4

First, create a default.nix that setups nixpkgs with an empty overlay. Eg:

let
  nixpkgs = import ./nixpkgs;
  overlay = self: pkgs; {}; # replace with `import ./overlay.nix` in the next step
in
  import nixpkgs {
    config = {};
    overlays = [overlay];
  }

This will be your entry point. Don’t call nix-build backend.nix but go through that file and use the -A attribute to access the right path in the returned attribute set. To test that it works, call nix-build -A hello and you should have the hello package.

Now the next thing is to create an overlay with your own packages. We take all of nixpkgs as add our own things + overrides:

overlays.nix

let
  compiler = "ghc822";
in
self: pkgs: {
  myHaskellPackages = pkgs.haskell.packages.${compiler}.override {
    overrides = newHaskellPkgs: oldHaskellPkgs: { };
  };
}

Now call nix-build -A myHaskellPackages.alex. If that works you can give the same treatment to the Haskell packages overlay.

,beam should be added to either the top-level or haskell overlay depending on what type of package it is. callPackage is then able to inject it as a dependency properly. callPackage also injects attributes that have been defined in the current overlay.

1 Like