Automated NixOS Benchmarks? (Phoronix Test Suite Issues...)

Help
1

I’m trying to evaluate performance of some NixOS configurations. Basically, I want to twiddle with Linux kernel settings (for example) and see how/whether this impacts performance of the system. Ideally, I’d like to use Phoronix Test Suite (PTS) and openbenchmarking.org. They have lots of really nice infrastructure that would be unfortunate to rebuild.

While PTS is itself packaged in Nixpkgs, it always needs additional dependencies to run benchmarks. For example, I haven’t been able to get its build-linux-kernel benchmark working. I’ve tried other people’s approaches and trying it in a FHS environment. No success. :frowning:

Now my questions are:

  • Did anyone get a couple of PTS benchmarks working on NixOS?
  • Is there another set of read-to-use benchmarks out there that works better on NixOS?
  • Are there other people out there that try to benchmark NixOS in an automated way?
3 Likes
2

I tried to use “impure derivations” last week to run phoronix-test-suite, but didn’t get very far. Basically, adding __impure = true; to a derivation allows it to access the network as though the sandbox is disabled, but it can only be depended on by other impure derivations or content-addressed derivations. I combined that with a weird little function I wrote a while back to build the derivation in an FHS env instead, but that only helped a little.

This was the last thing I tried before I got bored and gave up:

with import <nixpkgs> {};

let
  runInFHSEnv = drv: envArgs: let
    fhsWrapper = buildFHSUserEnv ({
      name = "${drv.name}-fhs-wrapper";
      runScript = "$@";
    } // envArgs);
  in lib.overrideDerivation drv (old: {
    builder = "${fhsWrapper}/bin/${fhsWrapper.name}";
    args = [old.builder] ++ old.args;
  });

in runInFHSEnv (runCommand "phoronix" {
  __impure = true;
} ''
  export PATH="/bin:/sbin:$PATH"
  phoronix-test-suite batch-benchmark pts/cpu
'') {
  targetPkgs = _: [
    phoronix-test-suite
    autoconf
    automake
    p7zip
  ];
}
2 Likes
3

Interesting. :slight_smile: I wouldn’t want to run PTS as a derivation, because it runs for its side effects.

I guess the next step is why PTS can’t find the dependencies it needs. Will report if I make any advances. :wink:

4

Is there any progress here?

I am also trying to benchmark my disks with Phoronix Test Suite.

1 Like
5

Unfortunately not. PTS is really not very compatible with NixOS. :frowning:

6

I found this: https://www.phoronix.com/review/docker-phoronix-pts
PTS in docker - it seams like a (partial) solution in NixOS.

7

Since the official docker image phoronix/pts doesn’t work for me either, I created my own docker image, which is working good in NixOS.

See github repo: https://github.com/torx-cz/phoronix-test-suite-in-docker

I also plan to put it on Docker Hub.

2 Likes
8

A docker image would only test the NixOS kernel since the userspace will be whatever comes with the container. That’s not very interesting for benchmarking NixOS itself.

9

You are right though.
However I am interested to test my hardware, like SSDs, CPUs etc., so for me it is working sufficiently.

1 Like