Output from a nixos integration test

Help
1

Hello,

I’m writing an integration test for a project using pkgs.nixosTest. I’ve managed to get all my services spun up and the tests passing. But I’ve noticed that the output in the nix store is just an empty directory. I’d like to capture the output of my tests (I run an exe with machine.succeed("${exe}"). My tests also produce coverage information and other data that I’d like to get in the output in the nix store. I’ve dug through the code a bit and it’s not at all clear to me how I might do that, or if it is even possible.

Thanks,
Rich

2 Likes
2

Hi,

the Python testScript runs in the (empty) output directory you found as it’s working directory. So any file you produce in the testScript will be placed in this folder when the test finishes.

There are some helpers which let you transfer files between your test VM’s and the test host, which you can find here.

So depending on the filesize you want to transfer, you can either use the machine.succeed output and write it to a file (small files only):

with open('output.txt', 'w') as file:
    file.write(machine.cucceed("${exe}"))

Or create the file in the VM and copy it back to the host.

An example test which utilizes this is the systemd-analyze.nix test.

result
├── systemd-analyze
│   ├── blame.txt
│   ├── blame.txt.err
│   ├── critical-chain.txt
│   ├── critical-chain.txt.err
│   ├── dependencies.dot
│   ├── dependencies.dot.err
│   ├── systemd-analyze.svg
│   └── systemd-analyze.svg.err
└── systemd-analyze.svg

1 directory, 9 files
2 Likes
3

Oh my gosh! I can’t believe I missed those functions. Thanks!

1 Like
4

Ok, so this works great when the test succeeds. What about when the test fails? Normally I’d use --keep-failed and look in the build directory for test output. I’ve wrapped the test like this

try:
  machine.succeed("${exe}")
finally:
  machine.copy_from_vm("logs", "")

When the test succeeds, the logs are available in the nix store. But when the test fails, it seems like nothing is copied

$ sudo tree /nix/tmp/nix-build-vm-test-run-apitests-non-hypervisor.drv-1/
/nix/tmp/nix-build-vm-test-run-apitests-non-hypervisor.drv-1/
├── env-vars
├── nixos-test-vde-pz_sbs5t-vde1.ctl
├── shared-xchg
└── vm-state-machine
    ├── machine.qcow2
    ├── monitor
    ├── shell
    └── xchg

4 directories, 4 files
5

That’s unfortunate. Especially when a test fails it would be great to have access to the copied files.

What seem to happen is that files created in the testScript are directly written to $out:

But $out is not located in the build directory, which --keep-failed preserves on failure.

So to preserve files you would need to copy them to /tmp/xchg in the VM (vm-state-machine/xchg outside) or /tmp/shared (shared-xchg outside).

Another approach would be to run the test interactively. This way you can choose a output directory for the test driver and don’t have to rely on --keep-failed:

mkdir output
nix shell -f ./test.nix driverInteractive -c nixos-test-driver --no-interactive -o output

The interactive driver also has the advantage that you can directly interact with the test VM, if you remove the --no-interactive flag.

1 Like