Collision between `python3` and `python3-env`

Help
1

I got these warnings when running nix-rebuild. After some digging, I found that python3-env is required by systemd-minimal then by libX11, and python3 from my own configuration.

Considering libX11 as essential package, I can’t find out a solution. Any help?

warning: collision between `/nix/store/8sb12yqrjrzrg4js62v0vnpskbslbnqm-python3-3.10.9-env/bin/2to3' and `/nix/store/65cp4izx3bllnwqn7c7dhrq9h9gmjkal-python3-3.10.9/bin/2to3'
warning: collision between `/nix/store/8sb12yqrjrzrg4js62v0vnpskbslbnqm-python3-3.10.9-env/bin/2to3-3.10' and `/nix/store/65cp4izx3bllnwqn7c7dhrq9h9gmjkal-python3-3.10.9/bin/2to3-3.10'
warning: collision between `/nix/store/8sb12yqrjrzrg4js62v0vnpskbslbnqm-python3-3.10.9-env/bin/idle' and `/nix/store/65cp4izx3bllnwqn7c7dhrq9h9gmjkal-python3-3.10.9/bin/idle'
warning: collision between `/nix/store/8sb12yqrjrzrg4js62v0vnpskbslbnqm-python3-3.10.9-env/bin/idle3' and `/nix/store/65cp4izx3bllnwqn7c7dhrq9h9gmjkal-python3-3.10.9/bin/idle3'
warning: collision between `/nix/store/8sb12yqrjrzrg4js62v0vnpskbslbnqm-python3-3.10.9-env/bin/idle3.10' and `/nix/store/65cp4izx3bllnwqn7c7dhrq9h9gmjkal-python3-3.10.9/bin/idle3.10'
warning: collision between `/nix/store/8sb12yqrjrzrg4js62v0vnpskbslbnqm-python3-3.10.9-env/bin/pydoc' and `/nix/store/65cp4izx3bllnwqn7c7dhrq9h9gmjkal-python3-3.10.9/bin/pydoc'
warning: collision between `/nix/store/8sb12yqrjrzrg4js62v0vnpskbslbnqm-python3-3.10.9-env/bin/pydoc3' and `/nix/store/65cp4izx3bllnwqn7c7dhrq9h9gmjkal-python3-3.10.9/bin/pydoc3'
warning: collision between `/nix/store/8sb12yqrjrzrg4js62v0vnpskbslbnqm-python3-3.10.9-env/bin/pydoc3.10' and `/nix/store/65cp4izx3bllnwqn7c7dhrq9h9gmjkal-python3-3.10.9/bin/pydoc3.10'
warning: collision between `/nix/store/8sb12yqrjrzrg4js62v0vnpskbslbnqm-python3-3.10.9-env/bin/python' and `/nix/store/65cp4izx3bllnwqn7c7dhrq9h9gmjkal-python3-3.10.9/bin/python'
warning: collision between `/nix/store/8sb12yqrjrzrg4js62v0vnpskbslbnqm-python3-3.10.9-env/bin/python-config' and `/nix/store/65cp4izx3bllnwqn7c7dhrq9h9gmjkal-python3-3.10.9/bin/python-config'
warning: collision between `/nix/store/8sb12yqrjrzrg4js62v0vnpskbslbnqm-python3-3.10.9-env/bin/python3' and `/nix/store/65cp4izx3bllnwqn7c7dhrq9h9gmjkal-python3-3.10.9/bin/python3'
warning: collision between `/nix/store/8sb12yqrjrzrg4js62v0vnpskbslbnqm-python3-3.10.9-env/bin/python3-config' and `/nix/store/65cp4izx3bllnwqn7c7dhrq9h9gmjkal-python3-3.10.9/bin/python3-config'
warning: collision between `/nix/store/8sb12yqrjrzrg4js62v0vnpskbslbnqm-python3-3.10.9-env/bin/python3.10' and `/nix/store/65cp4izx3bllnwqn7c7dhrq9h9gmjkal-python3-3.10.9/bin/python3.10'
warning: collision between `/nix/store/8sb12yqrjrzrg4js62v0vnpskbslbnqm-python3-3.10.9-env/bin/python3.10-config' and `/nix/store/65cp4izx3bllnwqn7c7dhrq9h9gmjkal-python3-3.10.9/bin/python3.10-config'
2

Python definitely isn’t normally in the PATH on a nixos system. Just because it’s in the closure isn’t enough for it to be causing a collision. That requires that it’s being put inside the same buildEnv or the like. You need to figure out where that’s happening. It would also help to have some more context on those erros, as nix would say when derivation is failing.

3

I’ve written a (kind of) small script to parse output of nix-store -q --tree /run/current-system, which prints out the dependency path of specified package name or path.

Here’s what I got with the python3-env package, one of the colliding two:

python3-3.10.9-env
  system-path
    nixos-system-nixos-22.11.2315.13fdd3945d8
python3-3.10.9-env [...]
  python3-3.10.9-env
    system-path
      nixos-system-nixos-22.11.2315.13fdd3945d8
python3-3.10.9-env [...]
  system-path
    nixos-system-nixos-alternate-22.11.2315.13fdd3945d8
      nixos-system-nixos-22.11.2315.13fdd3945d8

I belive that nixos-alternate is one of my specialisations. What confuses me is that python3-env is directly referenced by the system.

For the other side (python3), the result seems lengthy and not intuitive. In case I’ve pasted it on pastebin .

Additionally, I’m curious about the [...] mark at the end of nix store path, but I couldn’t find any information about it in the manual.

4

The experimental nix cli has a tool for this: nix why-depends

Regardless, it looks like both packages are in your system-path, and that’s where the collision is happening. Somehow you have both of them making their way into environment.systemPackages, while neither of them would be there on a normal system. The other references in the tree don’t matter.

5

Thank you for the confirmation. python3 is added to environment.systemPackages by myself. Seems it returns to the origin of finding out which brings in python3-env without any other clue.

6

OK I manage to find the left out reference to python3Full as builder of a derivation block(attr set). So does the conflict only occur in the building environment of that specific derivation, or it affects my shell environment?

Further, if by any chance I need a package that needs to be built in an environment with conflictions, could I temporarily remove or mask some packages from environment.systemPackages only for the builder of derivation, and still keep the packages in the final system environment?

7

You can extract the actual definitions of the option and where they come from, if you like. For a flake-based system it would be something like this:

nix eval .\#nixosConfigurations.tejingdesk.options.environment.systemPackages.definitionsWithLocations --apply builtins.toJSON --raw | jq -C

For a non-flake system it should be some variant of:

nix-instantiate --eval '<nixos/nixos>' -A options.environment.systemPackages.definitionsWithLocations --json

I can’t test that easily, however, as my system is flake based.

8

No, as I said, you have them both in environment.systemPackages. That’s where the conflict occurs.

9

I thought of a way to test it and worked out the kinks. This should work:

nix-instantiate --eval --expr 'builtins.toJSON (import <nixos/nixos> {}).options.environment.systemPackages.definitionsWithLocations' --json | jq 'fromjson' -C
10

Much appreciated for the detailed instruction. The ability to eval any expression under the current system environment really helps debugging.

However I can’t reproduce the confliction now. After I changed the builder = "${pkgs.python3Full}/bin/python3"; to builder = "${pkgs.python3}/bin/python3"; the confliction disappears, and never appear even if the builder is changed back to python3Full. I’ve checked the change history of all nix files to confirm I hadn’t touch other configurations (except for adding jq to environment.systemPackages).

The only chance it can happen I can imagine is, I used to have python3Full but not python3 in my environment.systemPackages, but the builder as python3. It should be when the confliction comes up, and at that time the number of conflicting files is times more than in the main post. After some blindly tweaking I finally got python3 in systemPackages and python3Full in builder, and reduced the number of confliction to ~10.

Unfortunately I can’t test with a individual nixos now. Maybe I can figure it out later.

Considering all information by hand, I have to think it as some bug, or non-purity, that does kept some influence from previous build. Anyway I do need to further investigate on it. And thank you again for all your support.

11

Well, best of luck getting to the bottom of it later, then.