nixosTest vs runTest

I am having trouble understanding the big differences between runTest and nixosTest. The diffrences I know of are…

  • runTest allows including multiple tests and nixosTest can only be passed one test.
  • The various docs seem to learn towards runTest for any nixos related stuff (like modules) and nixosTest for packages. And the file paths also align with this (runTest is defined under nixos/lib/testing/default.nix and nixosTest at pkgs/build-support/testers/default.nix

But even with that, it seems like it would easy enough to use either function to run a single or multiple test (I am making the assumption that the code to use nixosTest to run multiple tests would be simple, someone correct me if I am wrong, especially if the purpose of runTest is to easily run multiple tests).

Also when trying to read threw the code for both, they seem to have significant amounts of different code (even if they do share some code). While I could not quite figure what the differences are, the code leads me to assume that they are likely to have significant differences.

I want to know the significant differences because I am looking to set up testing for my nixos system configs (the “Testing outside the NixOS project” use case from manual), and I can’t decided what I should use, or why to use one over the other.

4 Likes

Bumping this because I am interested in an answer as well.