Proper way to deal with plugins and circular depedencies?

Help
1

Hi, I’m reworking the Python Application devpi-server. It uses a plugin architecture where the plugins are python packages and depend on devpi-server itself. My navie approach leads to this problem:

Rewriting #!/nix/store/m1fw8l8y9ycxh5dzispbb7cwl6rra14l-python3-3.13.12/bin/python3.13 to #!/nix/store/m1fw8l8y9ycxh5dzispbb7cwl6rra14l-python3-3.13.12
wrapping `/nix/store/gf607lw8wmcz4gc0b7hgdd0slgas4gkb-python3.13-devpi-server-6.19.1/bin/devpi-gen-secret’…
Executing pythonRemoveTestsDir
Finished executing pythonRemoveTestsDir
Running phase: installCheckPhase
no Makefile or custom installCheckPhase, doing nothing
Running phase: pythonCatchConflictsPhase
Found duplicated packages in closure for dependency ‘devpi_server’:
devpi_server 6.19.1 (/nix/store/gf607lw8wmcz4gc0b7hgdd0slgas4gkb-python3.13-devpi-server-6.19.1)
dependency chain:
this derivation: /nix/store/gf607lw8wmcz4gc0b7hgdd0slgas4gkb-python3.13-devpi-server-6.19.1
devpi_server 6.19.1 (/nix/store/vvpg8z3v87r3yv8r98p357zrq03qqkfj-python3.13-devpi-server-6.19.1)
dependency chain:
this derivation: /nix/store/gf607lw8wmcz4gc0b7hgdd0slgas4gkb-python3.13-devpi-server-6.19.1
…depending on: /nix/store/ld3mh0mq9qvbyv9prs8k89ckyx2fyy7q-python3.13-devpi-web-5.0.1
…depending on: /nix/store/vvpg8z3v87r3yv8r98p357zrq03qqkfj-python3.13-devpi-server-6.19.1

Package duplicates found in closure, see above. Usually this happens if two packages depend on different version of the same dependency.
For full logs, run:
nix log /nix/store/pnr7l6xd14cdgjcy46hqrs7yxzm8mgn6-python3.13-devpi-server-6.19.1.drv
error: Cannot build ‘/nix/store/a8qlc0s2i9a4w16l342lsjiv4jzr4x6q-unit-devpi-server.service.drv’.
Reason: 1 dependency failed.
Output paths:
/nix/store/1wrim2461pavjqhj52b4z7bhyk3h47xk-unit-devpi-server.service

I.e. the plugins are build with the original devpi-server, but the new devpi-server that has the plugins as dependency collides with that.

What’s the proper way to build devpi-server interdependent with its packages here?

2

Usually the approach with software involving plugins would be to build it without the plugins, build the plugins, then make a wrapper derivation that plugs the two together, rather than rebuilding the software that uses the plugins.

3

Do you have an example for Python?

4

I’m not aware of one, off the top of my head. Sorry.

5

The nixpkgs Python builders are unable to deal with circular dependencies without patching packages. There is no “proper way” to deal with it in nixpkgs. It’s all ad-hoc hacking.

For some prior art on Python wrapping derivations see xonsh.

6

Thanks adis. I updated the PR and did this: devpi-web: init at 5.0.1 by con-f-use · Pull Request #495637 · NixOS/nixpkgs · GitHub I now,I get a devpi-server executable that uses the plugins.

python3Packages.callPackage ../../../development/python-modules/devpi-server/wrapper.nix {
    plugins = with python3Packages; [
      devpi-web
      devpi-ldap
    ];
  }

However, when I try to integrate that in the nixos-module, like in l.92 here, it doesn’t know of the plugin options and doesn’t seem to use the wrapper, but the unwrapped package.

error: function 'anonymous lambda' called with unexpected argument 'plugins' at /nix/store/4ldjwr6icpc0p79a8xj0gwj5d28zyidi-source/pkgs/by-name/de/devpi-server/package.nix:1:1: 1| { python3Packages }: | ^ 2|

@adisbladis , do you have an idea, why that is or how do use my wrapper in the module?

This is me:

image
image508×457 78.4 KB

7

Nevermind, I think I got it working. The xonsh tip was worth its weight in gold.