Python program can't invoke module in same package

Help
1

Hi all. I’m working on packaging a Python program using pyproject.nix as explained in the wiki. I’ve got the following in pyproject.toml, which declares the entrypoint:

[project.scripts]
snapcast-mqtt = "snapcast_mqtt.app:main"

This works fine in nix shell and the script has no problems finding its own modules. But, when the script tries to invoke one of its own modules as a subprocess using subprocess.Popen, Python complains it can’t find the module. Effectively, it’s running:

/nix/store/h3i0acpmr8mrjx07519xxmidv8mpax4y-python3-3.12.5/bin/python3.12 -m snapcast_mqtt.server

Despite using the same environment and interpreter as the one that launched the script itself, I get the following:

/nix/store/h3i0acpmr8mrjx07519xxmidv8mpax4y-python3-3.12.5/bin/python3.12: Error while finding module specification for 'snapcast_mqtt.server' (ModuleNotFoundError: No module named 'snapcast_mqtt')

I think the reason the parent process works is because of the magic wrapping that happens within buildPythonPackage to generate the entrypoint script. How can I similarly bootstrap the Python interpreter invocation?

2

You are in this scenario: Python: wrapped python packages don't propagate their PYTHONPATH · Issue #23676 · NixOS/nixpkgs · GitHub

One approach is a patch to the sources. Or, a PYTHONPATH needs to be set. For the latter, I’m not sure, but I think maybe this is your solution: Broken python path in custom python package - #2 by tobiasBora

It takes the value of $PYTHONPATH that’s set for the build shell, and puts it into a postFixup wrapper.

3

Thank you, that was very helpful! What I ended up doing was passing an explicit PYTHONPATH to the subprocess environment constructed via the sys.path of the parent:

subprocess.Popen(proc_cmd, env={"PYTHONPATH": ":".join(sys.path)})

Not ideal, but it shouldn’t break other distros.

1 Like