Duplicate when installing extraPackage from overlay

Help
1

As described in a previous post, I am trying to install home-assistant with a python library that has not yet been packaged (pydeconz).

  service.home-assistant = {
    package = pkgs.home-assistant.override {
      extraPackages = ps: with ps; [ pydeconz ];

      packageOverrides = self: super: {
         pydeconz = pkgs.pythonPackages.pydeconz;
      };
    };
  ...

pydeconz itself was added via an overlay as follows:

self: super:
# Within the overlay we use a recursive set, though I think we can use `self` as well.
rec {
  # nix-shell -p python.pkgs.my_stuff
  python3 = super.python3.override {
    # Careful, we're using a different self and super here!
    packageOverrides = self: super: {
      pydeconz = super.buildPythonPackage rec {
        pname = "pydeconz";
        version = "71";
        # name = "${pname}-${version}";
        propagatedBuildInputs = [ super.pythonPackages.aiohttp ];
        doCheck = false;
        src = super.fetchPypi {
          inherit pname version;
          sha256 = "cd7436779296ab259c1e3e02d639a5d6aa7eca300afb03bb5553a787b27e324c";
        };
      };
    };
  };
  # nix-shell -p pythonPackages.my_stuff
  pythonPackages = python3.pkgs;
}

nioxos-rebuild switch now complains that it encountered two built derivatives for aiohttp:

Found duplicated packages in closure for dependency ā€˜aiohttpā€™:
aiohttp 3.6.2 (/nix/store/3n1hb7nm7fvp49dzy23w5510xgcfjnba-python3.7-aiohttp-3.6.2/lib/python3.7/site-packages)
aiohttp 3.6.2 (/nix/store/h4p8isyy84rzrd2bk43n8nml8x6b1ygq-python3.7-aiohttp-3.6.2/lib/python3.7/site-packages)

I already tried to remove both packages and rebuild, but it always ends up with this error message.

Of note, not only pydeconz, but also home-assistant requires aiohttp via propagatedBuildInputs.

Can anyone see what Iā€™m doing wrong here? Thank you in advance.

2

Installation (and usage) of the pydeconz package like this works:

let mypy= python3.withPackages ( python-pkgs: [ python-pkgs.pydeconz ] ); 
in environment.systemPackages = [ mypy ];

This also works in combination with installing home-assistant (of course only when pydeconz is not added to home-assistantā€™s extraPackages).

Furthermore, here are the dependers of the two aiohttp packages:


[moritz@monix:/nix/store]$ nix-store --query --referrers /nix/store/3n1hb7nm7fvp49dzy23w5510xgcfjnba-python3.7-aiohttp-3.6.2
/nix/store/3n1hb7nm7fvp49dzy23w5510xgcfjnba-python3.7-aiohttp-3.6.2
/nix/store/anvnwnjimjc3nxidc53ybpvr06nzkqv7-python3.7-aiohttp-cors-0.7.0
/nix/store/c0g5gx5gy0s613b165qw18p66wjfcsi9-python3.7-aiohue-1.9.2
/nix/store/dhlnz22h52bx8sidy167m5df3v3siifv-python3.7-pytest-aiohttp-0.3.0
/nix/store/fx76xj62ckxpffvpy58xdbp59dz9vb9i-homeassistant-0.100.3

[moritz@monix:/nix/store]$ nix-store --query --referrers /nix/store/h4p8isyy84rzrd2bk43n8nml8x6b1ygq-python3.7-aiohttp-3.6.2
/nix/store/h4p8isyy84rzrd2bk43n8nml8x6b1ygq-python3.7-aiohttp-3.6.2
/nix/store/zrsriwkk241frccgiq2w4flwhbfd9a2f-python3.7-pydeconz-71
/nix/store/rxd5yibpmdi98bay54ybrdbg31d91rk6-python3-3.7.5-env
3

seems like something is causing a different aiohttp derivation to be created, which it shouldnā€™t, can you try:

propagatedBuildInputs = [ super.aiohttp ];

self super in this regard should already be pointing to the package set.

4

Iā€™m stuck with the same error after applying your suggestion. My guess is that since there exist two aiohttp derivations, Iā€™d have to delete one of them. Unfortunately I canā€™t: Although I removed this

let mypy= python3.withPackages ( python-pkgs: [ python-pkgs.pydeconz ] ); 
in environment.systemPackages = [ mypy ];

from my configuration.nix (switch worked well), I am not able to delete the aiohttp derivation:

[moritz@monix:~]$ nix-store --delete /nix/store/h4p8isyy84rzrd2bk43n8nml8x6b1ygq-python3.7-aiohttp-3.6.2/
finding garbage collector roots...
0 store paths deleted, 0.00 MiB freed
error: cannot delete path '/nix/store/h4p8isyy84rzrd2bk43n8nml8x6b1ygq-python3.7-aiohttp-3.6.2' since it is still alive

[moritz@monix:/nix/store]$ nix-store --query --referrers /nix/store/h4p8isyy84rzrd2bk43n8nml8x6b1ygq-python3.7-aiohttp-3.6.2
/nix/store/h4p8isyy84rzrd2bk43n8nml8x6b1ygq-python3.7-aiohttp-3.6.2
/nix/store/zrsriwkk241frccgiq2w4flwhbfd9a2f-python3.7-pydeconz-71
/nix/store/rxd5yibpmdi98bay54ybrdbg31d91rk6-python3-3.7.5-env

[moritz@monix:~]$ python
python: command not found

[moritz@monix:~]$ nix-store --delete /nix/store/rxd5yibpmdi98bay54ybrdbg31d91rk6-python3-3.7.5-env/
finding garbage collector roots...
0 store paths deleted, 0.00 MiB freed
error: cannot delete path '/nix/store/rxd5yibpmdi98bay54ybrdbg31d91rk6-python3-3.7.5-env' since it is still alive

A reboot didnā€™t ā€œfreeā€ the liveliness of /nix/store/rxd5yibpmdi98bay54ybrdbg31d91rk6-python3-3.7.5-env either.

5

Upgrading my packages (nix-channel --update) fixed the problem. No idea why. Thank you for your support @jonringer.

1 Like
6

You are lucky. Did it happen again?

I have same issue and channel update doesnā€™t help.

Found duplicated packages in closure for dependency ā€˜tomlā€™:
toml 0.10.2 (/nix/store/p0bhxdaw59x6p5ikzqjcmsi31jmdcdws-python3.9-toml-0.10.2/lib/python3.9/site-packages)
toml 0.10.2 (/nix/store/gmv2cfzs5i4ssnk016rhy44dn9j8mh2k-python3.9-toml-0.10.2/lib/python3.9/site-packages)

7

I had the same issue, and narrowed it down.

I have a packageOverrides for certifi in home-assistant to add additional trusted certificates. After this update, the build for pyjwt failed, but only if I had the packageOverrides for certifi.

    ā€¢ Updated input 'unstable':
        'github:nixos/nixpkgs/34c5293a71ffdb2fe054eb5288adc1882c1eb0b1' (2022-10-09)
      ā†’ 'github:nixos/nixpkgs/285e77efe87df64105ec14b204de6636fb0a7a27' (2022-10-11)

python3.10-pyjwt> pythonCatchConflictsPhase
python3.10-pyjwt> Found duplicated packages in closure for dependency 'Sphinx':
python3.10-pyjwt>   Sphinx 5.1.1 (/nix/store/54q3v4ihj1wasvirb9jf2zh7xg89bc67-python3.10-sphinx-5.1.1/lib/python3.10/site-packages)
python3.10-pyjwt>   Sphinx 5.1.1 (/nix/store/xrv7hhjxvdmqvdpjwq4j8bmsq3ymx5m1-python3.10-sphinx-5.1.1/lib/python3.10/site-packages)
python3.10-pyjwt> Found duplicated packages in closure for dependency 'requests':
python3.10-pyjwt>   requests 2.28.1 (/nix/store/2rhqi7f02g36wzbh4ymq8dnms8r7fhwq-python3.10-requests-2.28.1/lib/python3.10/site-packages)
python3.10-pyjwt>   requests 2.28.1 (/nix/store/l1snk8ccljysyh7zza6mkfvmxp392czw-python3.10-requests-2.28.1/lib/python3.10/site-packages)
python3.10-pyjwt> Found duplicated packages in closure for dependency 'certifi':
python3.10-pyjwt>   certifi 2022.6.15 (/nix/store/akc8dsdxn1cy8pwxs0xy9dkxqj81ryyb-python3.10-certifi-2022.06.15/lib/python3.10/site-packages)
python3.10-pyjwt>   certifi 2022.6.15 (/nix/store/qpn27wlz3cdw9286cnvwf5ymr22bv42w-python3.10-certifi-2022.06.15/lib/python3.10/site-packages)
python3.10-pyjwt> Package duplicates found in closure, see above. Usually this happens if two packages depend on different version of the same dependency.

During that time period pyjwt stayed at 2.5.0, but the package was updated to include docs: python3Packages.pyjwt: Update various packaging details Ā· NixOS/nixpkgs@750da02 Ā· GitHub

My problem is some weird interaction with the added sphinxHook. After removing that input from pyjwt everything started to work again:

pyjwt = prev.pyjwt.overridePythonAttrs (oA: {
  outputs = lib.remove "doc" oA.outputs;
  nativeBuildInputs = lib.remove prev.sphinxHook oA.nativeBuildInputs;
});

Hopefully this helps the next person who comes across this after searching for ā€œPackage duplicates found in closure, see aboveā€ :slight_smile: