Nix-unit - A Nix unit testing runner compatible with runTests

1

I was a bit unhappy with the state of unit testing and wanted a better experience when hacking on GitHub - adisbladis/pyproject.nix so I ended up forking GitHub - nix-community/nix-eval-jobs: Parallel nix evaluator with a streamable json output [maintainers @Mic92, @adisbladis] and turned it into a test runner rather than a threaded evaluator.

My previous test suite was a Python one that first inspected the test attribute names and then fired off one Nix process per attribute. This instead uses the Nix C++ API and allows you to do the same per-attribute eval failure catching without the process overhead.

My test runs locally took 3.5s for 50 tests and are now down to 70ms.

5 Likes
2

@infinisil @yorickvp @roberth this is related to https://github.com/NixOS/nix/pull/8699

2 Likes
3

I’ve extended the schema to be able to test trees like:

{
  testPass = {
    expr = 1;
    expected = 1;
  };

  testFail = {
    expr = { x = 1; };
    expected = { y = 1; };
  };

  nested = {
    testFoo = {
      expr = "bar";
      expected = "bar";
    };
  };
}

This also means that you can test only a certain sub-tree using
$ nix-unit --flake .#libTests.someTestAttr

and added a few better README examples.