"source" store paths not being automatically tracked?

Help
1

See the following trivial flake:

{
  inputs = {
    nixpkgs.url = "github:NixOS/nixpkgs";
  };

  outputs = { nixpkgs, ... }:
  let
    system = "x86_64-linux";
    pkgs = import nixpkgs {
      inherit system;
    };
    drv = pkgs.runCommand "test" {} ''
      ${pkgs.coreutils}/bin/mkdir -p $out
      echo "path:/nix/store/xdij5xmkssfrqnwixyh65j7d1hb0d1j1-source" > $out/foo
      echo "path:/nix/store/3yhi12pgsmv2dz0x2ialvjw51lnv4jw8-zlib-1.2.13.tar.gz" > $out/bar
      echo "path:/nix/store/04c0b1rmi9r6k9sl69a4gw3mhp3b5q2n-zlib-1.2.13" > $out/baz
  '';

  in {
    defaultPackage.${system} = drv;
  };
}

When I build this, the only tracked reference is the last one, which is zlib’s outPath:

$ nix path-info -r ./result
/nix/store/y78s8i569g5w04pnili1z7bkg73gqgbl-libunistring-1.0
/nix/store/s99my0m4dcxi75m03m4qdr0hvlip3590-libidn2-2.3.2
/nix/store/yzjgl0h6a3qh1mby405428f16xww37h0-glibc-2.35-224
/nix/store/04c0b1rmi9r6k9sl69a4gw3mhp3b5q2n-zlib-1.2.13
/nix/store/pk3vqayhrdvhcih24zvlk7bs0mjgb6q8-test

The other two references are to source paths— one for zlib.src, and the other for the nixpkgs flake:

$ nix flake metadata github:NixOS/nixpkgs/deb85c6b174184459895c33569f84706fec2a4df | grep store
Path:          /nix/store/xdij5xmkssfrqnwixyh65j7d1hb0d1j1-source

Why does Nix treat source store paths differently? I thought maybe it had something to do with getContext/attachContext, but those seem to be specific to strings rather than paths, and when I used the getter with nix eval to inspect these strings, they all looked the same.

So I’d love to understand what the implementation difference is here, but more importantly to my use-case— is there a way to suppress this this behaviour and have Nix track these, so that I can nix copy a pool of related sources to a remote target and protect them from the garbage collector, without needing to deal with individual tracking?

FYI @zimbatm

2

The only reason the zlib ones worked was coincidence. Those happen to be transitive build inputs thanks to stdenv referring to them properly. You are not referring to these paths properly, so they aren’t added to your derivation’s inputDrvs. Check out nix show-derivation to see what I mean. Your source path isn’t in there, but the zlib paths are in there for the stdenv derivation. Nix only checks for output references to the (transitive) build inputs, so any paths that aren’t actually build inputs get ignored.

This is why you never never copy and paste store paths into Nix code. Always get the paths you need via nix expressions with proper string context, so that Nix knows to add them as inputs.

1 Like
3

Okay, thanks for that— I can confirm that when I change the zlib path to be ${pkgs.zlib.src}, then that path does indeed show up as expected in the resulting closure.

My understanding had been that Nix did some kind of post-build text-scanning trick to “look” for store paths in the generated outputs, but I guess it doesn’t actually look for arbitrary paths, but rather “candidate” paths that it knows came in as inputs.

Looks like I can get the flake source location just using ${nixpkgs.outPath}, so that works perfectly for my use-case. Thanks!

4

That’s right, it only scans the outputs for paths that were part of the inputs of the drv. And the drv inputs are those stored in the propagated string contexts.

1 Like