Whenever I open Main.hs in VSCode and HLS starts, stuff starts to get downloaded into ~/.cabal/store/ghc-9.0.2 (into the cabal store!). This means things aren’t working at all. Nix should provide the dependencies. What am I doing wrong? Thank you.
app/Main.hs:
module Main where
-- If Nix/Haskell/HLS works, I should see an "unused"
-- warning here in vscode. Also, no packages should
-- have been downloaded into ~/.cabal/store
import qualified Data.Attoparsec.ByteString.Char8 as AP
main :: IO ()
main = putStrLn "Hello, Haskell!"
First, I think you need haskell-language-server and cabal-install in nativeBuildInputs.
Second, if you want to have the Haskell package dependencies provided by Nixpkgs, you need to load your package into the closure using callCabal2nix. The procedure depends on whether your Haskell package is part of the haskell packages in Nixpkgs or not. If you package is part of the Nixpkgs package set, I think the canonical way is to use an overlay for all Haskell package sets like so:
This is basically taken from the Serokell blog: Practical Nix Flakes But that was one of the things that also don’t work. So I even tried replacing buildInputs to nativeBuildInputs as you suggested, but that doesn’t work either.
Note: Doesn’t work = cabal still downloading packages into ~/.cabal/store.
@cdepillabout hope you can chime in. I came back to nixpkgs thanks to your comment here: Nix Haskell Development (2020) - #27 by NixZeus_2 I feel, after reading your comment, that haskell.nix would be overkill for me and I prefer to just be in nixpkgs.
But now we have flakes.
It’s a bit sad to be in the dark, just copying/pasting things without solid understanding of anything. At least the copying/pasting worked in the pre-flakes world.
I tried to put together a small example for you. This is how I would write a flake.nix if I were starting a Haskell project like you described:
There is also a README that describes how to use it:
Let me know if you have any problems with this.
Yeah, I feel you here. Nix’s learning curve is intense. Definitely feel free to ask any questions you have. Everybody here was in the same boat as you once. Any questions you ask will likely be of help to people coming after you and facing the same problems as you.
Can you explain a bit why it works? It seems like a very internal/esoteric reason having to do with buildInputs vs. nativeBuildInputs, prev vs. final, putting HLS under final.haskellPackages vs. final, forgetting to add anything under packages in myDevShell … or something else or combination of multiple factors.
(Not flake specific) In the nix develop -L shell, running cabal build works without putting anything into ~/.cabal/store - exactly as expected! By what mechanism can cabal do this? Is this some built-in Nix specific support inside of cabal or just some environment variable trickery that is supported by cabal?
To reply to precisely one of your questions: prev vs final is simply how overlays work. The overlays mechanism gives you access to two inputs: the “final” result of applying the overlay, and the "prev"ious object which is about to be overridden by the overlay. A section of the Wiki was what made it click for me.
A lot of people who are pretty familiar with Nixpkgs spend a lot of time looking through the Nixpkgs source code. If you want to build up a good understanding of how Nixpkgs works, and be able to look up answers to your own questions, you’ll likely want to get in the habit of grepping through Nixpkgs for things. I know this can be quite annoying at first, but it is quite common if you’re really trying to get a good handle on Nixpkgs.
I don’t know if I’d call things like buildInputs vs nativeBuildInputs, prev vs final, packages from the top-level vs from internal package sets, etc as internal or esoteric, but I get what you’re trying to say. Depending on what you’re trying to do, using either buildInputs or nativeBuildInputs may work. Using either prev or final may work. Pulling packages from the top-level vs internal package sets may work. But when myself and others write examples and code snippets, we try to do the “most correct” thing, even if it is a little more confusing. You’re of course always welcome to question why something is written like it is!
And now to answer your actual questions:
buildInputs is for libraries that are used a run-time. nativeBuildInputs is for compilers/tools that are run at build-time. It is sort of hand-wavey for something like HLS, which both needs to run at “build-time” (since you’ll be running it in your dev environment), but also needs to be able to link into your own Haskell code.
As long as you’re not cross-compiling, it is often okay to mix up buildInputs vs nativeBuildInputs. (But there are a non-insignificant number of people who use Nixpkgs for cross-compiling, so you will often get corrected in PRs and forum posts if you mess it up.)
Nixpkgs contains “top-level” derivations, that are all mostly defined in this single file:
If you grep through this file, you should find an entry for haskell-language-server. This is the “top-level” haskell-language-server. It is directly under final.
Nixpkgs also contains sub-package sets. haskellPackages is an example of a sub package set. This gets a little more complicated, but things in haskellPackages are effectively defined here:
(You should also be able to find haskellPackages defined in the above all-packages.nix file)
If you dig into how final.haskell-language-server is defined, vs how final.haskellPackages.haskell-language-server is defined, you’ll see that there are some minor differences, but it mostly doesn’t matter which one you use.
However, you do have to be sure to use haskell-language-server that was compiled with the same GHC version as you’re using for your project. So if your project uses haskell.packages.ghc94 instead of haskellPackages, you’ll need to make sure to use haskell.packages.ghc94.haskell-language-server instead of the top-level haskell-language-server, or haskellPackages.haskell-language-server.
This is just sort of how haskellPackages.shellFor works. There is an example in the source code:
Although now that you mention it, it is kind of confusing that this doesn’t throw an error, or at least a warning or something.
GHC has a concept of a package, and a package database:
Cabal and cabal-install work directly with GHC packages and package databases, but give users a high-level tool over-top of them. Most users aren’t really directly aware of GHC package databases, but just work with them through cabal-install.
Basically what Nix does is build a GHC package database, and directly hands it to Cabal. Cabal is able to use it without having to build any dependencies on its own. For your nix develop shell, if you run ghc-pkg list, you should see what this looks like.
Thank you, this is really helping me - and I’m sure over time countless others - getting up and running with Flakes! Some follow-up questions came to mind:
Where should I add system library dependencies? I did it like this but maybe there’s a better way:
let packageName = "xyz"; # Putting the name in variable.
# ...
myHaskellPackages = haskellPkgs.override {
overrides = hfinal: hprev: {
${packageName} =
final.haskell.lib.addBuildDepends
(hfinal.callCabal2nix "${packageName}" ./. { })
[final.libzip]; # system libraries
};
};
Related, I’m also adding buildInputs = [final.libzip]; (note: not under nativeBuildInputs) to get the library into the devShell. It’s a bit unfortunate I have to put two two places like that i.e. the devShell is not reusing what I already put into addBuildDepends. Is there a way around that?
Is it appropriate to fix the branch in flake.nix (e.g. nixpkgs.url = "github:NixOS/nixpkgs/nixos-22.11";). I feel unstable changes too frequently but that’s a personal choice and I wonder if I’m disrespecting others by making that decision for them.
Is it appropriate to fix a GHC version in flake.nix (e.g. haskellPkgs = final.haskell.packages."ghc8107";)? This is the GHC version I currently work with, but I wonder if I’m disrespecting others by baking this decision into flake.nix - or worse, breaking my library in haskellPackages.
I have sometimes seen callCabal2nix not re-evaluate anything even if the .cabal file changed. (I had to change the name argument to “invalidate” it to make it recompile.) Is this normal?
Is this name usually (by convention only) set to the Hackage name / cabal file name (without extention)? It doesn’t appear to be doing anything crucial.
I’m not sure how the .cabal file gets chosen - does it assume there is one cabal in the ./. directory and pick that?
You can scratch this for now. This may have been something about fetchFromGithub behavior where just changing the rev (revision) is not enough: you also have to invalidate it by putting in a bogus sha256 hash (i.e. it caches by the hash, not by the rev).
In general, you should add system library dependencies to your .cabal file (normally as pkgconfig-depends, or in similar fields). If you do this, then cabal2nix will generally know to pull in the system library for you. So a lot of times it won’t require any additional work for you on the Nix side.
If this doesn’t work for some reason, then you’re correct with using functions like haskell.lib.compose.addBuildDepends.
In general, shellFor should be automatically figuring out all the required system dependencies, and providing you a shell with them. Off the top of my head, I’m not sure why it might not be doing so.
Do you have your code up on GitHub that I (or someone else) could take a look at?
Yeah, pinning to nixos-22.11 is definitely a valid choice!
I had only pinned to nixos-unstable in my above example because I knew that haskellPackages in the current nixos-unstable is ghc927, which I thought would be the easiest to use and show off. Pinning to any other Nixpkgs version should be completely fine.
If any users wanted to depend on your flake in one of their projects, they can override the Nixpkgs version your project uses if they want. There is an easy way to do that with flakes.
I’d say this is also completely fine.
The only time I’d worry about this is if you wanted to provide a Nixpkgs Haskell overlay that end-users are expected to apply to their own Nixpkgs Haskell package set, and you want it to work with multiple versions of GHC. This type of thing can be a little complex, so if you decide you want to go down this route, you should probably think about exactly what you want your end users to do, and what sort of configurations to support.
(In practice, in the Haskell world, almost everyone releases their Haskell packages to Hackage. It is much less common for Haskell projects to share Nixpkgs Haskell overlays as a way of managing library dependencies).
If I remember correctly, the name used to actually be used somewhere in the code (maybe as a way to easily find the .cabal file?), but we removed that restriction at some point. You’re correct that now it doesn’t really do anything crucial.
That’s correct. (Although technically there is also logic for first converting a package.yaml with hpack if there is no .cabal file.)