How does poetry2nix work?

If I understand this correctly, poetry2nix creates nix expressions from the poetry.lock file. Since this fails for some packages, overrides are used to patch these python packages.

However, there seems to be only one override for each python package. How does poetry2nix support multiple version of the same python package which each might need a different override?

1 Like

It doesn’t.

Within a packaged python application dependency graph, there can be only one version of each python package.

That’s what I feared. Then, the poetry2nix approach is damned to fail, as the user has to manually specify the overrides depending on the package version in poetry.lock.

I think there is a misunderstanding here. Yes, one environment will have only one version of a Python package; this is normal with Python.

If I am correct, the question is, how does one with poetry2nix specify version-dependent overrides. Well, when creating an override, you could check old.version. You can find some in poetry2nix/default.nix at b37a389df4977b9f9de1711297d732faaa73e648 · nix-community/poetry2nix · GitHub.

No, the correct version is chosen from the lockfile, no need to override.

@NobbZ That is the part I am trying to understand. Assume

  • foo in version 1.0.0 needs override A
  • foo in version 2.0.0 needs override B

in overrides/default.nix, there can be either override A or override B.

What if I have two poetry projects, one with dependency foo=1.0.0 and the other with foo=2.0.0 in the poetry.lock. How it the correct override selected?

The question is, how do these 2 projects belong together?

No, they are completely unrelated.

Then I do not see the problem. They are separate applications.

I would like to understand the technique behind. How is the override for a specific package version selected from overrides/default.nix?

If you specify an override it will be used, if you don’t it won’t.

So I have to specify the override as in my comment:

That’s what I feared. Then, the poetry2nix approach is damned to fail, as the user has to manually specify the overrides depending on the package version in poetry.lock.

For foo in version 1.0.0 I have to tell to use override A and for foo in version 2.0.0 I have to tell to use override B.

Assume I use numpy, scipy, matplotlib, … in different versions in different projects. Each one of these need different overrides for different version.

The override can depend on the package version such as for cryptography for instance

Why do they need overrides? And if you use different versions of them in different projects, what do you care? As long as you do not have a project that uses all those projects, all of these describe its own environment, so there is no problem.

@lewo Thank you, that was what I was looking for. Although, the approach may become difficult in a few years with many different versions.

@NobbZ Without overrides, the packages do not build. There is a reason why poetry2nix has them.

Thats fine, and each individual project gets its own overrides, thats fine then.

So my question remains, what is the actual problem?

Have you a single example project where you actually have a clash? And why can’t you solve it with just another override?

This one is especially hard and is not representative actually. I should have chosen another example (there are some discussions with upstream to solve it)!

If I understand correctly,

  • @zeec123 isn’t asking about custom overrides in the individual projects using poetry2nix, but about those overrides maintained within poetry2nix.
  • @zeec123 isn’t trying to solve a particular problem, but wants to understand how poetry2nix works “internally” if you will.