How do you create your own flake check?

Hello!

As in the title; how do you create flake checks? I can’t find any examples anywhere.

Thank you kindly for the help!

A check is a derivation. If the derivation builds successfully, then the check is a success.

$ cat flake.nix
{
  outputs = { nixpkgs, ... }:
  {
    checks.x86_64-linux.math = with nixpkgs.legacyPackages.x86_64-linux;
      runCommand "math-test" {} ''
        test $(( 1 + 1 )) -eq  3
        touch $out
      '';
  };
}

$ nix flake check
error: builder for '/nix/store/zxjran6ax2dmfc65603h1qpj3984sgr4-math-test.drv' failed with exit code 1

Changing the command to test $(( 1 + 1 )) -eq 2 makes the check succeed.

2 Likes

Can we run individual nix commands within these checks, such as nix build?

Not exactly, they still need to be derivations. Your derivation can just be an indicidual command, however, as long as you produce some kind if $out file, e.g.:

checks.x86_64-linux.test = with nixpkgs.legacyPackages.x86_64-linux;
      runCommand "test" {} ''
        echo test > $out
      '';

nix build specifically won’t work, since it’s a nix command. You can do the exact same thing as nix build by just setting the check to equal one of your packages, though:

checks.x86_64-linux.flake-build = self.packages.x86_64-linux.default;

Or you could write a test for the package, which will also build it, since it’s needed to run the test:

checks.x86_64-linux.test =
    let
      package = self.packages.x86_64-linux.default;
    in
      with nixpkgs.legacyPackages.x86_64-linux;
        runCommand "test" {} ''
          ${package}/bin/package > $out
        '';
```:
2 Likes

Ah; got it. Thanks so much for the detailed explanation!

Hi thanks for this explanation, a furthur question maybe:

Is there some way to write a check to do proper evaluation of a nixosConfigurations output?

TIL the default nix flake check only verifies the type of nixosConfigurations is a ‘derivation’, but doesn’t actually eval nixosConfigurations thanks to @raphi

checks.x86_64-linux.myhost = self.nixosConfigurations.myhost.config.system.build.toplevel;
checks.x86_64-linux.myotherchecks = runCommand "test" {} ''echo ..";

If I do something like above, it’d build the host, which is time consuming and unwanted. What I want is nix flake check to run nix eval --raw .#nixosConfigurations.myhost.config.system.build.toplevel in essences.

There is nix flake check --no-build, but I do actually need to build in order for the some other hand written checks to run, so… :smiling_face_with_tear:

1 Like

Sounds like that’s worth reconsidering, I personally assumed it would actually evaluate the nixosConfigurations at least. Anyone know why it doesn’t?

1 Like