Un-disabling a python package in an overlay

Help
1

Does anyone know how to make an overlay that changes the “disabled” attribute of a python module?

For example, in order to test a new upstream version of this package: https://github.com/NixOS/nixpkgs/blob/15ab96d5a7c61204be79d0588a4816a0f7969306/pkgs/development/python-modules/astroid/default.nix on python3.9, I need to both change the src attribute (easy with overrideAttrs) and the disabled attribute. But I can’t seem to actually change the disabled attribute.

With something like

{ python-self: python-super: {
      astroid = python-super.astroid.overrideAttrs (x: {
        disabled = false;
      });
}}

i seems as if the old disabled attribute is evaluated while trying to inject the override, and so it fails before the override is actually added.

1 Like
2

use overridePythonAttrs

3

Here’s my attempt with that. Still haven’t been able to get it to work

# shell.nix
let pkgs = (import
  (fetchTarball {
    # https://github.com/NixOS/nixpkgs/commit/253de1fcdb3b82a74a0bf128a7ae2a7b2fed0932
    # December 9, 2020
    url =
      "https://github.com/NixOS/nixpkgs/archive/253de1fcdb3b82a74a0bf128a7ae2a7b2fed0932.tar.gz";
    sha256 = "0za7fl0wy47hw2b41784cq1l2vj6wkgx4hl7y0k6h44wrgxnivmv";
  })
  {
    overlays = [
      (self: super: {
        pythonOverrides = python-self: python-super: {
          astroid = python-super.astroid.overridePythonAttrs (x: {
            disabled = builtins.trace (false) false;
          });
          python39 =
            super.python39.override
              {
                packageOverrides = self.pythonOverrides;
              };
        };
      })
    ];

  });
in
pkgs.python39.buildEnv.override {
  extraLibs = with pkgs.python39Packages; [
    astroid
  ];
}

Fails with

$ nix-shell shell.nix 
error: astroid-2.4.2 not supported for interpreter python3.9
(use '--show-trace' to show detailed location information)
4

I tried to do the same thing without success as of now.
My exact nix snippet is

{ pkgs ? import <nixpkgs> { } }:
let
  py = let
    packageOverrides = self: super: {
      azure-identity = super.azure-identity.overridePythonAttrs (oldattrs: {
          disabled = false;
          version = "newVersion";
          });};
  in (pkgs.python3.override { inherit packageOverrides; }).pkgs;
in py.azure-identity

And I have this error

$ nix-build x.nix --show-trace
error: while evaluating anonymous function at pwd/x.nix:1:1, called from undefined position:
while evaluating the attribute 'azure-identity' at pwd/x.nix:6:7:
while evaluating the attribute 'azure-identity.overridePythonAttrs' at /nix/store/q0v3m59iid3h0x8hrps2d8f5s5jk6m8r-nixpkgs-21.03pre262503.51894963cbd/nixpkgs/pkgs/top-level/python-packages.nix:574:3:
while evaluating 'callPackageWith' at /nix/store/q0v3m59iid3h0x8hrps2d8f5s5jk6m8r-nixpkgs-21.03pre262503.51894963cbd/nixpkgs/lib/customisation.nix:117:35, called from /nix/store/q0v3m59iid3h0x8hrps2d8f5s5jk6m8r-nixpkgs-21.03pre262503.51894963cbd/nixpkgs/pkgs/top-level/python-packages.nix:574:20:
while evaluating 'makeOverridable' at /nix/store/q0v3m59iid3h0x8hrps2d8f5s5jk6m8r-nixpkgs-21.03pre262503.51894963cbd/nixpkgs/lib/customisation.nix:67:24, called from /nix/store/q0v3m59iid3h0x8hrps2d8f5s5jk6m8r-nixpkgs-21.03pre262503.51894963cbd/nixpkgs/lib/customisation.nix:121:8:
while evaluating anonymous function at /nix/store/q0v3m59iid3h0x8hrps2d8f5s5jk6m8r-nixpkgs-21.03pre262503.51894963cbd/nixpkgs/pkgs/development/python-modules/azure-identity/default.nix:1:1, called from /nix/store/q0v3m59iid3h0x8hrps2d8f5s5jk6m8r-nixpkgs-21.03pre262503.51894963cbd/nixpkgs/lib/customisation.nix:69:16:
while evaluating 'makeOverridablePythonPackage' at /nix/store/q0v3m59iid3h0x8hrps2d8f5s5jk6m8r-nixpkgs-21.03pre262503.51894963cbd/nixpkgs/pkgs/top-level/python-packages.nix:28:37, called from /nix/store/q0v3m59iid3h0x8hrps2d8f5s5jk6m8r-nixpkgs-21.03pre262503.51894963cbd/nixpkgs/pkgs/development/python-modules/azure-identity/default.nix:18:1:
while evaluating 'makeOverridable' at /nix/store/q0v3m59iid3h0x8hrps2d8f5s5jk6m8r-nixpkgs-21.03pre262503.51894963cbd/nixpkgs/lib/customisation.nix:67:24, called from /nix/store/q0v3m59iid3h0x8hrps2d8f5s5jk6m8r-nixpkgs-21.03pre262503.51894963cbd/nixpkgs/pkgs/top-level/python-packages.nix:30:12:
while evaluating anonymous function at /nix/store/q0v3m59iid3h0x8hrps2d8f5s5jk6m8r-nixpkgs-21.03pre262503.51894963cbd/nixpkgs/pkgs/development/interpreters/python/mk-python-derivation.nix:30:1, called from /nix/store/q0v3m59iid3h0x8hrps2d8f5s5jk6m8r-nixpkgs-21.03pre262503.51894963cbd/nixpkgs/lib/customisation.nix:69:16:
azure-identity-1.5.0 not supported for interpreter python3.8

It seems the code at https://github.com/NixOS/nixpkgs/blob/05b5b5491e822ec005d9cf4589e9eb75e2499d90/pkgs/development/interpreters/python/mk-python-derivation.nix#L100-L102 is evaluated before applying overrides. I don’t understand how overrides work so I have no more solution.

A side-question is why this package is specifically disabled for python3.8 which is now default python3 in nixpkgs. This is strange as it seems to be supported upstream.

5

I have the feeling that overrideAttrs does not work as expected for complex packaging which use intermediate functions as in pythonBuildPackage or buildGoModule (as seen in "inconsistent vendoring" in buildGoModule when overriding source - #3 by jshholland)

There is something difficult to understand as a newbie as some attributes can be overriden and some others cannot.

1 Like
6

Your observation is correct. The throw is evaluated before any override can be applied, because overrides can only be applied if the initial function call (buildPythonPackage in this case) returns a value and does not throw.

There is also an open GitHub issue for this problem:

https://github.com/NixOS/nixpkgs/issues/48663

7

Thank you for the link. I will follow the issue.

8

I made a PR removing the disabled clause, as it’s no longer necessary.

https://github.com/NixOS/nixpkgs/pull/109375

9

on a related note, if anyone wants to help with the azure python packages, any help would be welcome.

I no longer work at microsoft, but still think that having azure support in nix is important.