Benchmarking lapack/blas libraries and compilers with nix

So this has been on my mind for awhile and I was wondering how exactly it would be done and if others have done it. I would like to compare lapack vs. atlas vs. openblas vs. the new mkl. Additionally I would like to compare gcc vs clang vs. icc (which I believe was just added with mkl?).

What is the recommended way of doing this? I imagine a derivation would be made for each combination maybe even a matrix of derivations with all combinations of compilers and linear algebra libraries could be made.

Where I am confused is how to override the compiler for a given library. I am aware that you can substitute blas libraries quite easily.


You should be able to override the compiler, in the same way you would override a library, with stdenv=clangStdenv?

It would be nice to have some nix tooling for benchmarking apps (in similar way we have tests). I was thinking a little about it but I did not have a good idea yet. It could be something like mkBenchmarkDerivation, which

  • runs a benchmark script for a given package
  • sets up OMP_NUM_PROCS for the run
  • sets up mpi related environment variables to get reproducible environment
  • takes number of cpus and some other related parameters as an input
  • measures the total time for the run

Those are great ideas. I did not know how to override the compiler and that looks very simple. Once I get it working I plan to benchmark numpy with different linear algebra libraries. Currently I am wanting to get mkl working with numpy.

I will probably create a benchmark derivation that can form a matrix of the possible combinations. Nix seems like a perfect platform for these types of tests.

There is an ongoing effort to the Intel compiler going: Add Intel Parallel Studio stdenv · Issue #32434 · NixOS/nixpkgs · GitHub

I am not a Python expert. I’m afraid I won’t be of much help with the MKL/numpy topic.

1 Like