How to quickly find derivations with a specific build-time dependency

Help
1

tl;dr What are some quick ways to find, in a NixOS setup, all derivations that depend on a specific python package with a specific python runtime?

So my issue was that my NixOS build was failing because apparently a new CVE was discovered. The package was pypy2.7-setuptools-44.0.0, which is python2’s setuptools built on the pypy2.7 runtime:

error: Refusing to evaluate package 'pypy2.7-setuptools-44.0.0' in /nix/store/snzg7swkp0vsd2qch9izsbmxnyb13ca0-source/pkgs/development/python2-modules/setuptools/default.nix:79 because it is marked as insecure

The extended error message (not shown here) proved useful because it told me how I can temporarily permit the package and I could proceed to build my system.

After my system was built, I realized (and found out) that this package are only used during build-time. I can’t use nix why-depends to find out which packages I have that use this insecure package.

How do I do that?

Ultimately, what I want is to attempt to override the build dependency (if even possible). I also would like to find out why e.g. pypy2.7 is used as the default runtime.

Any pointers?

2 
nixos-rebuild build --show-trace |& grep 'while evaluating derivation'

or use lix which has it built-in.

You’ll need to remove it from the insecure packages allowlist for the error to show up again.

3

Does nix-store --query --referrers /nix/store/<hash>-pypy2.7-setuptools-44.0.0 provide what you are asking? If you need indirect as well, perhaps --referrers-closure?

Depending, you may need to find the intersection of this set and nix-store --query --requisites /var/run/current-system to limit it to just the store references that are part of your current generation, but if you just did this build probably everything is within the 1 generation.

4

In principal, without enabling that CVE, you should be able during eval to figure out the faulty system dependency with --show-trace passed to nixos-rebuild – that’s the original intention of the designers. nix why-depends has the --derivation flag you can use but it won’t help you here. If the --show-trace method doesn’t help you, I’d remove the CVE from pypy27Packages.setuptools and run:

nix why-depends --derivation /run/current-system -f. pypy27Packages.setuptools
5

Thanks for the pointers all.

This turned out to be a weirder issue.

I tried using --derivationto figure this out. Also with nix-tree. When I disabled the CVE, I could get my system to build. But Pypy2.7 is apparently not part of any build-time closure of any of the package.

Packages that have errorred out when building, with the CVE enabled, all use Python3.13. Furthermore, this showed up in seemingly random places (first it was neovim, then after commenting that out, it showed up in autorandr postswitch hook, then in i3).

After another long debugging session, I got a hunch that something is off with my Python setting in general.

I isolated it down to a custom overlay that I made in a custom flake I had. In this flake, I package some private Python packages into nix and I want to make it available under e.g. pkgs.Python314Packages.{mynamespace}.{mypackage}.

For some reason that I can’t explain, this overlay fails only in Python3.13. When I comment out that overlay, then it could build just fine even with the CVEs included. Perhaps it’s because those derivations that failed earlier (nvim, etc) uses Python3.13. I did not investigate enough further. For now, I just commented out the python313 overlay then and everything seems to work.

For reference, here’s the custom overlay. In the flake repo, this is under overlays/default.nix. This overlay also adds e.g. pkgs.{mynamespace}.{nonpythonpackage} to make non-Python packages available under mynamespace.

{ flake }:
let
  inherit (flake.inputs.nixpkgs) lib;
  supportedPythons = [
    "python310"
    "python311"
    "python312"
    # FIXME: Including this in the overlay breaks downstream builds.
    # "python313"
    "python314"
  ];
in
{
  default =
    let
      pythonOverrides = pyfinal: _pyprev: {
        mynamespace = {
          mypackage1 = pyfinal.callPackage ../packages/mypackage1 { };
          mypackage2 = pyfinal.callPackage ../packages/mypackage2 { };
        };
      };
      mkPythons =
        prev:
        lib.genAttrs supportedPythons (
          pyname: prev.${pyname}.override { packageOverrides = pythonOverrides; }
        );
    in
    final: prev:
    (
      {
        mynamespace = import ../packages {
          inherit flake;
          pkgs = final;
        };
      }
      // (mkPythons prev)
    );
}

I have checked to ensure that mynamespace does not cause any collisions (i.e. there is no e.g. pkgs.python313Packages{mynamespace} in nixpkgs.

6

I’m not sure if that’s the reason why it breaks things for you, but in general I wouldn’t create such a namespace inside python3Packages - it breaks cross compilation and python3Packages.callPackage is not designed to handle non-package arguments such as mynamespace when it defines packages.

Also, there’s no eed for a let in there, it will be much more readable if you’d generated the attributes as in mkPythons inline, and write the pythonOverrides inline.

As for the real cause for the eval error, I suspect something is wrong in ../packages, but I’m not sure.

7

I’m not sure if that’s the reason why it breaks things for you, but in general I wouldn’t create such a namespace inside python3Packages - it breaks cross compilation and python3Packages.callPackage is not designed to handle non-package arguments such as mynamespace when it defines packages.

It could be ~ the reason I set it up this way, is so that after applying the overlay, one can do e.g. this:

pkgs.python314.withPackages (pypkg: [ pypkg.{mynamespace}.{mypackage} ])

While at the same time decreasing the possibility of name clashes, since I don’t have to worry if {mypackage} is already what something else is called in nixpkgs. {mypackage} are internal Python packages that may or may not clash with actual, PyPI-published package names.

I kind of liked the API (i.e. being able to namespace packages this way), but this is mostly because I’m not aware of other ways to achieve this.

I can prepend {mynamespace} to {mypackage} to avoid namespacing. But then I’d be inventing a new name for these packages that users have to remember.

This is already an aside now, but do you have any pointers about what else can be done to achieve such a thing?

8

If these clash with Pypi-published package names, your override would break things.

If you choose something that no one will choose, like python3Packages._bow_string (pun partially intended :bow_and_arrow: ), then you won’t have to worry about name clashes (the _ prefix guaranties this).

Try to convert your overlay so it won’t use an inner attribute set, and check if the PyPy errors are still there with Python313 - this might give you a strong hint.