Error: rich should use `buildPythonPackage` or `toPythonModule` if it is to be part of the Python packages set

Help
1

Hello!

As in the title; with the following overlay:

overlay = final: prev: { python311 = prev.python311.override (super: {
  self = final.python311;
  packageOverrides = composeExtensions (super.packageOverrides or (_: _: {})) (new: old: {
    rich = inputs.nixos-unstable.legacyPackages.${prev.stdenv.targetPlatform.system}.python311.pkgs.rich.overridePythonAttrs (old:
      let python3Packages = final.Python3.pkgs;
      in rec {
        version = j.pyVersion old.format src;
        src = ./.;
        propagatedBuildInputs = (with python3Packages; [ hy ])
          ++ old.propagatedBuildInputs;
        meta.homepage = "https://github.com/syvlorg/${pname}";
        checkInputs = (with python3Packages; [
          pytest-cov
          pytest-custom_exit_code
          pytest-randomly
          pytest-sugar
        ]) ++ old.checkInputs;
        checkPhase =
          "TERM=unknown pytest --cov-report term-missing --cov=rich tests/ -vv";

        # I'm checking if `old' has `doCheck' so that, regardless of whether it's true or false,
        # `old.doCheck' will be used if it exists.
        passthru = old.passthru // {
          doCheck = if (old ? doCheck) then old.doCheck else true;
        };

      });
  });
}); };

I’m getting the error in the title. Thing is, the unstable version is still using buildPythonPackage, so I’m not entirely sure what’s happening here.

Any help in solving this would be greatly appreciated!

2

I would guess that even though it’s using buildPythonPackage, it’s not using the same buildPythonPackage, since the underlying python derivations are different. You probably can’t mix and match packages from python scopes built on different interpreter derivations.

3

So we can’t use python package overlays from different channels any more?

4

Mixing python packages like this is completely unsupported by the python team.

You should stay within one nixpkgs evaluation and use overlays there.

Some other random notes:

  • generating coverage inside the nix sandbox is a bit wasteful and only slows down test runs, we always disable it
  • using python3* in a python build usually means a good source for impureness and mixing of multiple python versions which does not work.
5

I could’ve sworn I was advised by others in the community that this was how to use older versions of packages when newer versions had bugs. How can I use newer versions, then? By overriding the source?

Regarding coverage, I was trying to follow rich's makefile test. Am I missing anything by omitting the coverage report?

And for the overlays, how do I access other packages, for, say, additional build inputs? I’d assume using the new and old keywords / arguments, but I split up the functions for easier modularity, so I don’t really have access to them most of the time.

6

You can try but you can also run into problems. I think it could work that you apply toPythonModule to the package but you might also run into the next problem.

Yes, see the python language docs on how to do that.

The coverage report which is mostly useless for end users.

self: super: {
  python = super.python.override {
    packageOverrides = python-self: python-super: {
      twisted = python-super.twisted.overrideAttrs (oldAttrs: {
        src = super.fetchPypi {
          pname = "Twisted";
          version = "19.10.0";
          hash = "sha256-c5S6fycq5yKnTz2Wnc9Zm8TvCTvDkgOHSKSQ8XJKUV0=";
          extension = "tar.bz2";
        };

        buildInputs = oldAttrs.buildInputs or [ ] ++ [ python-super.something ];
      });
    };
  };
}
7

You can try but you can also run into problems. I think it could work that you apply toPythonModule to the package but you might also run into the next problem.

The default toPythonModule does not allow me to overridePythonAttrs after the fact, such as to temporarily disable tests for a specific build; I had to modify it in an overlay to include the functions. Should that be raised as an issue on the nix / nixos report?

Yes, see the python language docs on how to do that.

I am aware of this, but the main reason I want to use the newer versions themselves is so that I don’t have to replicate any new build dependencies, ignored tests, check inputs, etc. Should I just grab anything I might need from from the new version, and override the source manually from there?

And got it; thanks for the confirmation on the overlay!

8

Also, quick note: when I override the src, both the old version and the new version seem to be being built (as shown in the build logs, where you can see both versions’ paths), and the old version is hitting the errors I’m trying to avoid with the new version.

Wait, no, sorry; my fault. Modularization can get extremely confusing at times.

Actually, wait; still happening.