When are PRs adding NixOS tests appropriate?

Development
1

I’ve recently extended a game package, and I think a NixOS test could be helpful here. But they seem somewhat “heavyweight” to me and I was wondering if there are guidelines or practices for when to use them.

If PRs testing previously untested packages are appreciated in general I’d have a couple of things I could add :slight_smile:

Related questions: suppose there’s a NixOS test for package xyz that suddenly fails. Does that block some of the automatic nixpkgs processes/workflows, or is xyz simply marked as broken and things move on?

2

I have been under an impression that the causal chain is: NixOS VM tests are heavy → they are slow → they are annoying to debug → there are not enough of them…

3

Hum, so then I’ll probably create a PR or two. But how about the “block” thing? I’d like to understand the “consequences” first.

1 Like
4

I am pretty sure all auto-mark-broken processes are manually triggered… How to make sure a test does not block any channel is a good question.

(It looks like it could be possible not to include the test in the channels at all, though)

5

No blocking by default, except for waiting for everything (including the test) to finish. Just look at the amount of failing tests: https://hydra.nixos.org/eval/1767792?filter=nixos.tests#tabs-still-fail

1 Like