Platformio python issue: No module named 'pkg_resources'

Help
1

Hi!

I’m trying to use the espidf framework for a platformio project.
When trying to build / run using pio run, the dependencies start getting installed but it fails with this error:

ModuleNotFoundError: No module named 'pkg_resources':
  File "/nix/store/s3adyk2jm5rfhffsa3m9wfcgzaq0asci-platformio-6.1.6/lib/python3.10/site-packages/platformio/builder/main.py", line 187:
    env.SConscript("$BUILD_SCRIPT")
  File "/home/xxx/.platformio/packages/tool-scons/scons-local-4.4.0/SCons/Script/SConscript.py", line 597:
    return _SConscript(self.fs, *files, **subst_kw)
  File "/home/xxx/.platformio/packages/tool-scons/scons-local-4.4.0/SCons/Script/SConscript.py", line 285:
    exec(compile(scriptdata, scriptname, 'exec'), call_stack[-1].globals)
  File "/home/xxx/.platformio/platforms/espressif32/builder/main.py", line 312:
    target_elf = env.BuildProgram()
  File "/home/xxx/.platformio/packages/tool-scons/scons-local-4.4.0/SCons/Util.py", line 737:
    return self.method(*nargs, **kwargs)
  File "/nix/store/s3adyk2jm5rfhffsa3m9wfcgzaq0asci-platformio-6.1.6/lib/python3.10/site-packages/platformio/builder/tools/piobuild.py", line 60:
    env.ProcessProgramDeps()
  File "/home/xxx/.platformio/packages/tool-scons/scons-local-4.4.0/SCons/Util.py", line 737:
    return self.method(*nargs, **kwargs)
  File "/nix/store/s3adyk2jm5rfhffsa3m9wfcgzaq0asci-platformio-6.1.6/lib/python3.10/site-packages/platformio/builder/tools/piobuild.py", line 120:
    env.BuildFrameworks(env.get("PIOFRAMEWORK"))
  File "/home/xxx/.platformio/packages/tool-scons/scons-local-4.4.0/SCons/Util.py", line 737:
    return self.method(*nargs, **kwargs)
  File "/nix/store/s3adyk2jm5rfhffsa3m9wfcgzaq0asci-platformio-6.1.6/lib/python3.10/site-packages/platformio/builder/tools/piobuild.py", line 331:
    SConscript(env.GetFrameworkScript(name), exports="env")
  File "/home/xxx/.platformio/packages/tool-scons/scons-local-4.4.0/SCons/Script/SConscript.py", line 660:
    return method(*args, **kw)
  File "/home/xxx/.platformio/packages/tool-scons/scons-local-4.4.0/SCons/Script/SConscript.py", line 597:
    return _SConscript(self.fs, *files, **subst_kw)
  File "/home/xxx/.platformio/packages/tool-scons/scons-local-4.4.0/SCons/Script/SConscript.py", line 285:
    exec(compile(scriptdata, scriptname, 'exec'), call_stack[-1].globals)
  File "/home/xxx/.platformio/platforms/espressif32/builder/frameworks/espidf.py", line 29:
    import pkg_resources

This error appears to be related to the setuptools python module.
I have installed platformio, scons, as well as the python packages setuptools and pyserial (via (python3.withPackages my-python-packages)) in my configuration.nix in environment.systemPackages. So it cannot find the package?

In a second attempt, I have removed the packages above from my system config and tried to create a shell.nix as follows:

# shell.nix
{ pkgs ? import <nixpkgs> {} }:
let
  my-python-packages = ps: with ps; [
    setuptools
    pyserial
  ];
in 
pkgs.mkShell rec {

  packages = with pkgs; [
    gcc
    cmake
    scons
    platformio
    (python3.withPackages my-python-packages) # we have defined this in the installation section
  ];
}

Now it actually seems to compile! But a new error arises regarding the serial module (pyserial):

Pyserial is not installed for /nix/store/bc45k1n0pkrdkr3xa6w84w1xhkl1kkyp-python3-3.10.12/bin/python3.10. Check the README for installation instructions.
Traceback (most recent call last):
  File "/home/xxx/.platformio/packages/tool-esptoolpy/esptool.py", line 31, in <module>
    import esptool
  File "/home/xxx/.platformio/packages/tool-esptoolpy/esptool/__init__.py", line 41, in <module>
    from esptool.cmds import (
  File "/home/xxx/.platformio/packages/tool-esptoolpy/esptool/cmds.py", line 14, in <module>
    from .bin_image import ELFFile, ImageSegment, LoadFirmwareImage
  File "/home/xxx/.platformio/packages/tool-esptoolpy/esptool/bin_image.py", line 14, in <module>
    from .loader import ESPLoader
  File "/home/xxx/.platformio/packages/tool-esptoolpy/esptool/loader.py", line 30, in <module>
    import serial
ModuleNotFoundError: No module named 'serial'
*** [.pio/build/esp32doit-devkit-v1/bootloader.bin] Error 1

Not sure which README is meant but anyways, it cannot find the serial module even though I set it up in shell.nix…

So two questions:

  1. Why does the first version not work up to the same point as the second version (shell.nix), even though the same packages are installed via configuration.nix and shell.nix?
  2. Why is the second version unable to find the pyserial module?

One thing that I noticed is that even when I remove platformio from my system packages, it is still available from my shell. This is because platformio requires this line in configuration.nix to work properly:

services.udev.packages = [ pkgs.platformio-core.udev ];

Could this be the source of the issue? That it is using the “wrong” platformio package when using nix-shell? Can I somehow move this services configuration for the required udev rules into my shell.nix?