So this has been on my mind for awhile and I was wondering how exactly it would be done and if others have done it. I would like to compare lapack vs. atlas vs. openblas vs. the new mkl. Additionally I would like to compare gcc vs clang vs. icc (which I believe was just added with mkl?).
What is the recommended way of doing this? I imagine a derivation would be made for each combination maybe even a matrix of derivations with all combinations of compilers and linear algebra libraries could be made.
Where I am confused is how to override the compiler for a given library. I am aware that you can substitute blas libraries quite easily.
3 Likes
You should be able to override the compiler, in the same way you would override a library, with stdenv=clangStdenv?
It would be nice to have some nix tooling for benchmarking apps (in similar way we have tests). I was thinking a little about it but I did not have a good idea yet. It could be something like mkBenchmarkDerivation, which
- runs a benchmark script for a given package
- sets up
OMP_NUM_PROCS for the run
- sets up mpi related environment variables to get reproducible environment
- takes number of cpus and some other related parameters as an input
- measures the total time for the run
Those are great ideas. I did not know how to override the compiler and that looks very simple. Once I get it working I plan to benchmark numpy with different linear algebra libraries. Currently I am wanting to get mkl working with numpy.
I will probably create a benchmark derivation that can form a matrix of the possible combinations. Nix seems like a perfect platform for these types of tests.
There is an ongoing effort to the Intel compiler going: Add Intel Parallel Studio stdenv · Issue #32434 · NixOS/nixpkgs · GitHub
I am not a Python expert. I’m afraid I won’t be of much help with the MKL/numpy topic.
1 Like