Force nix not to check/test without to override each package

Hello,

I’m wondering how to tell nix to omit checks/tests for packages without to override each pkg explicitly?

the following example will execute test for hug which I wouldn’t want (well, I would like to control it)

nix-shell test.nix

let
  overlay = (self: super: rec {
    python37 = super.python37.override {
      packageOverrides = self: super: {
        pytest = super.pytest.overrideAttrs (old: {
          doCheck = false; 
        });
      };
    };

    python37Packages = python37.pkgs;
  });

  myPythonPackages = ps: with ps; [
    hug 
  ];
in
{ pkgs ? import <nixpkgs> { overlays = [ overlay ]; } }:
pkgs.mkShell {  

  doCheck = false;

  buildInputs = with pkgs; 
  [
    (python37.withPackages myPythonPackages)

    libspatialindex   
  ];
}

Due to the nature of nix I don’t see how it would be possible without overriding. You can write a function that does this generically, and maybe even recursively for all dependencies, too. Then apply it to the package of interest. I think there’s a good chance someone has already done something like this, possibly even in the library.

Honestly, if the tests are failing, it’s a good indication that the package will also fail at runtime.

A downside of overriding doCheck like this is it means you can’t use the binary cache anymore. The check phase shouldn’t modify the output, but unless it’s a fixed-output derivation (and it almost certainly is not) the output path is based on the derivation hash, not the output contents, so any change busts the cache.

For this reason recursively turning off checks on dependencies is a rather poor idea as you’ll now have to rebuild a significant portion of the entire world.

FWIW if you are going to turn off checks in some automated fashion, at least make sure what you’re overriding actually specifies doCheck = true, because if it’s already false then checks are already disabled.

nobody said that the test would fail but I don’t have hours to wait for their execution by every run.

What else is doCheck=false (here for mkShell) if not a attribute that results in a generic function which (should) deactivate all test?

How long does pytorch takes when you build (from source) an environment and have you ever build pytorch without on failing test?

I cannot see what doCheck in mkShell does. If I add e.g. “pyodbc” or “mkl-service”, for both the binary get installed (and for mkl are running some test). Nevertheless doCheck is set to “false”. I cannot see an effect of setting doCheck.

doCheck in mkShell does nothing. Generally, doCheck just disables the build phase checkPhase for this one derivation (since from a package dependency perspective it doesn’t really matter if dependencies are tested or not). As a mkShell derivation can’t be built at all (it’s only used to pull in dependencies), doCheck does nothing.

It seems you have given several unrelated scattered examples here and it’s a bit unclear what you want. Perhaps you can give an example where overriding each package is not viable and we’ll see what can be done? For example in the first post you can just do something like:

  # ...
  dontCheckPython = drv: drv.overridePythonAttrs (old: { doCheck = false; });
  myPythonPackages = ps: with ps; [
    (dontCheckPython hug)
  ];

(Also you wouldn’t specifiy doCheck = false in either pytest or mkShell because they have no effect) But it seems like it’s not what you want and what you really need would be tedious when done in this way? It would be nice if we could see how it would be tedious for you to ‘override each package’.

thanks for the hints. I think I have to go a few steps back and read more docs.
In general I would like to use Nix for Data Science but like with conda it isn’t possible without further pip installations (-> but than it’s not clean and neither nix nor conda is reproducible). Perhaps with poetry and nix is better suited …

In addition to reading docs, I want to mention that it’s really easy to play around with stuff in nix repl. I just typed python37Packages.hug.override and pressed tab and it gave me overridePythonAttrs, whose functionality was more or less obvious once you see it.

The build times mostly depend on what needs to be done. If you don’t have binary dependencies, it’s just copying sources around and replacing some strings. That is why it’s a very sensible suggestion to only disable tests on pure Python packages. If they have them turned off already, copying stuff from the cache is of course a but faster. Otherwise you‘re obviously in for a long round of compilation - IIRC Python wheels are not compatible with nix.

on my personal machine, it takes a very long time, and a lot of RAM. Even my 3990X server gets clobbered when doing reviews which trigger a pytorch rebuild.

We might refactor that package so that tests are part of passthru.tests, and not part of the base derivation

EDIT: kind of cheating to use my 3990X, but here’s the raw times:

$ nix-build -A python3Packages.pytorch --check
...
/nix/store/654i8pbhha42852v35598cvsczmyjbcc-python3.8-pytorch-1.5.0

real 5m55.566s
user 0m0.476s
sys	 0m0.261s