I have integration tests in my Haskell project’s test suite that call cabal build (using the turtle Haskell scripting library (in case anyone cares)) on the project itself, followed by cabal exec.
This worked fine when calling cabal test without Nix. This also works fine when calling cabal test inside of a nix develop shell.
However when running nix build, the test suite fails with the error
The Nixpkgs Haskell builder uses Cabal (the library) through the Setup.hs file to build Haskell packages. It doesn’t use cabal-install. The details of exactly what happens are in the following file (if you’re interesting in reading a bunch of Nix code):
I’m not sure what exactly to suggest here. Most people don’t call cabal-install install from within nix-build, so I’m not sure what exactly you need to do to get this working.
One thing you could try is to instead see if the binary built from Setup.hs will work for you. You should be able to access this if you can figure out what ${setupCommand} from the above script evaluates to.
Another thing you could try is to unset GHC_PACKAGE_PATH (since the above build code also has to unset it at one point).
If I were to give you advice, it would be to not try to call cabal recursively. If you can achieve what you’re trying to do without having to call cabal recursively, you’ll likely have a better time if you’re writing tests you’d like to run from multiple build systems.
I wonder if trying to hack my way around this is worth the trouble.
Maybe a better question is: What is “the way” to do integration tests? Integration test: Where the test compiles the program itself and runs the executable in various ways to see if results are as expected.
I can’t be the only person doing this (I’m guessing). What is the way to do this properly?
One idea I came with is, put everything in main into a separate library function main_, and have main simply call main_. Then of course my tests could just run main_ and check the results (instead of bothering to compile any executable).
I’ve never heard of anything in the Haskell community for doing exactly this.
When you’re talking about “tests” in the Haskell sense, most people think about test components defined in .cabal files. But when you’re running these Cabal-based tests, I don’t think you’re guaranteed that the executables defined in the same package are built and available in the tests.
This is what I would do.
Alternatively, you could just define your tests in a shell-script. Although that might be quite a pain depending on exactly what you’re trying to test.