Nix-collect-garbage during builds

Help
1

Is it safe to run garbage collection during a build? Could it screw up the state of the store somehow?

2 Likes
2

It is safe, as access to the store is coordinated by the daemon, but I have seen race conditions where the build failed when inputs have been GC’d during the build.

1 Like
3

I got excited and tried it out and here’s what I found:

A derivation to test GC during build

flake.nix:

{
  inputs.nixpkgs.url = "github:nixos/nixpkgs";
  outputs = { self, nixpkgs }: {
    packages.x86_64-linux.default =
      let
        pkgs = nixpkgs.legacyPackages.x86_64-linux;
      in
      derivation {
        name = "myDrv";
        builder = "${pkgs.bash}/bin/bash";
        args = [ ./hello_builder.sh ];
        system = "x86_64-linux";
        inherit (pkgs) coreutils;
      };
  };
}

hello_builder.sh:

export PATH="$coreutils/bin"
mkdir $out
# Try deleting the outPath/drvPath while it is sleeping during the buildPhase
sleep 300
mkdir $out/sth

Now that we have the derivation, let’s get the outPath before starting the build

Get the drvPath first and then use it to find the outPath

> nix repl
Welcome to Nix 2.17.0. Type :? for help.

nix-repl> :lf .
Added 12 variables.

nix-repl> output

nix-repl> outputs.packages.x86_64-linux.default
«derivation /nix/store/qwg20gb07csnbb9v9a3cdpc268y7winl-myDrv.drv»

nix-repl> :q

outPath

nix show-derivation /nix/store/qwg20gb07csnbb9v9a3cdpc268y7winl-myDrv.drv

Output:

...
"outputs": {
      "out": {
        "path": "/nix/store/hwnwd47knmqch8461iwvb94bznz3lkw3-myDrv"
      }
    },
...

Trying ls on the outPath:

ls /nix/store/hwnwd47knmqch8461iwvb94bznz3lkw3-myDrv
ls: cannot access '/nix/store/hwnwd47knmqch8461iwvb94bznz3lkw3-myDrv': No such file or directory

Well, that’s obvious; we haven’t built the derivation yet!

Time to build and during the build try deleting both drvPath and outPath 

# Start the build
nix build

In a different terminal, try deleting the outPath

nix store delete /nix/store/hwnwd47knmqch8461iwvb94bznz3lkw3-myDrv

You will notice that you can’t do it, even though we are calling mkdir $out before sleep 300:

error: Cannot delete path '/nix/store/hwnwd47knmqch8461iwvb94bznz3lkw3-myDrv' since it is still alive. To find out why, use: nix-store --query --roots

Time to find out why:

nix-store --query --roots /nix/store/hwnwd47knmqch8461iwvb94bznz3lkw3-myDrv
{temp:511199} -> /nix/store/hwnwd47knmqch8461iwvb94bznz3lkw3-myDrv

It is because the build process is keeping the outPath alive.

You will observe the same when you try to delete the drvPath using nix store delete

Hope this helps!

Note: In rare conditions you might even face race conditions while trying to gc during a build.

1 Like