Git bisect: how to detect if a commit can use the nix binary cache?

Help
1

I’m trying to use git bisect to find in nixpkgs the source of a regression… However I don’t know why but a huge fraction of the commit does not allow me to use the binary cache (and therefore I need to re-compile many things which is not viable to bisect the source).

So I have two questions:

  1. Is there an efficient way to quickly check if a commit is going to use the binary cache, notably in order to do git bisect more efficiently?
  2. Do you know why sometimes the binary cache is not usable? I thought that the cache was basically never removed from the nix server and that hydra basically compiles everything it can.

For reference here is the test I am using, using nix-build test.nix:

import nixos/tests/make-test-python.nix ({...} :

{
  name = "swapping-keyboard-keys";

  machine = { pkgs, ... }:
    {
      documentation.enable = false;
      users.users.root.password = "";
      services.udev.extraHwdb =
        ''
          evdev:name:AT Translated Set 2 keyboard:*
           KEYBOARD_KEY_2c=x  # z -> x
           KEYBOARD_KEY_2d=z  # x -> z
        '';
    };

  testScript =
    ''
      machine.start()
      machine.wait_for_unit("getty.target")
      machine.sleep(2)
      machine.send_chars("root\n")
      machine.sleep(2)
      machine.send_key("ret")
      machine.sleep(2)
      machine.send_chars("echo zx > /test\n")
      machine.wait_for_file("/test")
      machine.succeed("grep -q xz /test")
    '';
})

Note also that I am using --first-parent with bisect start.

2

As a rule of thumb: Prepare for massive rebuilds when bisecting over staging branches or master.

It is relatively safe to bisect the actual channel branches.

3

I think you’d either want to just run with --dry-run and check the number of builds it says it’ll need to build instead of fetch, or just git bisect skip any commits that aren’t in the channel’s history as reported here: https://channels.nix.gsc.io (and by “history” I mean commits that Hydra actually finished jobs for to the point that the channel was advanced to it)

4

I asked that question myself a few times.

  1. Is there an efficient way to quickly check if a commit is going to use the binary cache, notably in order to do git bisect more efficiently?

This should be possible by using Hydras API for the trunk jobset on https://hydra.nixos.org/,
but I don’t know any specifics.

Regarding bisection with Nix there is already a project easing the use
of bisect with Nix: GitHub - timokau/nix-bisect: Bisect nix builds. Status: alpha/proof of concept. You'll probably have to dig into the implementation if you want to use it. Built for personal use, lightly maintained. PRs welcome. Issues welcome, but I make no promises regarding responses or fix

They don’t seem to care about Hydra evaluations, but maybe that would be
a good place to build logic for that kind of feature?

5

Thanks a lot for your answers!

So regarding the --dry-run it does give some insight but the number of build is definitely not enough to judge… Usually when I see 900 builds it seems to imply a rebuild, but many times I had a build of a python lib (mentioning django??) with only 150-200 builds. On the other hand I got no build while --dry-run was giving me around 550 lines. Is there a more reliable way to use dry-run to know if I will be actually compiling stuff, like run a makefile or alike?

The https://channels.nix.gsc.io/ is actually interesting! But I guess I would need to script something to get the list of all commits, and it will certainly miss some (for instance I can build with commit 5c3e01fe1c5 but it does not appear to be in the list for what I tried). But at least it seems to provide a nice base to start.

Regarding the api is it documented somewhere?

I’m also really not sure to understand why sometimes strange things are built… for instance on d957d3ed9cf I build something that needs Sphinx-4.0.2 or on 58e8e3d5df3 it needs /nix/store/jw71y4017vv3gdagf3dsgrf7bvlipah5-firefox-78.15.0esr.source.tar.xz…

The tool of timokau seems interesting but as far as I can see I can’t see anything related to hydra evaluation… Maybe it would be a good addition tool!

6

Actually, I am thinking that most of the rebuild are actually recompiling python stuff… which means that it tries to change the test library and not the actual system. Do you know if somehow I could manage to decouple the test library to pin it to a given nixpkgs version and let it build the system coming from the current HEAD? (I’m trying to dig into the code but it does not seem to be trivial… but I’m surely missing something)

And it would even have an advantage: if I can pin the qemu version I can ensure that the problem does not come from a change in qemu!

7

There is a way to quickly check if an outpath exists in a binary cache:
nix path-info /nix/store/<hash>-package --store https://cache.nixos.org

If it exists the command will exit printing info about the outpath, if not the command will fail. If you want to ensure that the dependencies are also cached, you can pass the --recursive,-r flag to nix path-info as well.

path-info is still gated by the nix-command experimental feature irrc.