Poetry pandas issue, libz.so.1 not found

1

I am trying to setup a project using python3 and pandas.
A minimal example is here:

When i run poetry shell
and try to import pandas I get:

ImportError: Unable to import required dependencies:
numpy:
IMPORTANT: PLEASE READ THIS FOR ADVICE ON HOW TO SOLVE THIS ISSUE!
Importing the numpy C-extensions failed. This error can happen for
many reasons, often due to issues with your setup or how NumPy was
installed.
We have compiled some common reasons and troubleshooting tips at:
Troubleshooting — NumPy v2.0.dev0 Manual
Please note and check the following:

  • The Python version is: Python3.8 from “/home/kohlerm/.cache/pypoetry/virtualenvs/poetrytest-C7XS23X2-py3.8/bin/python3”
  • The NumPy version is: “1.22.1”
    and make sure that they are the versions you expect.
    Please carefully study the documentation linked above for further help.
    Original error was: libz.so.1: cannot open shared object file: No such file or directory

Anyone has an idea why that happens, and whether there is a workaround to fix it?

3 Likes
2

I’m currently having the same issue. The error output is the following:

>>> import pandas
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/home/victor/.cache/pypoetry/virtualenvs/allocation-wf-dg-KQpPHA9g-py3.9/lib/python3.9/site-packages/pandas/__init__.py", line 16, in <module>
    raise ImportError(
ImportError: Unable to import required dependencies:
numpy: 

IMPORTANT: PLEASE READ THIS FOR ADVICE ON HOW TO SOLVE THIS ISSUE!

Importing the numpy C-extensions failed. This error can happen for
many reasons, often due to issues with your setup or how NumPy was
installed.

We have compiled some common reasons and troubleshooting tips at:

    https://numpy.org/devdocs/user/troubleshooting-importerror.html

Please note and check the following:

  * The Python version is: Python3.9 from "/home/victor/.cache/pypoetry/virtualenvs/allocation-wf-dg-KQpPHA9g-py3.9/bin/python"
  * The NumPy version is: "1.22.2"

and make sure that they are the versions you expect.
Please carefully study the documentation linked above for further help.

Original error was: libz.so.1: cannot open shared object file: No such file or directory
3

In the project that I use pandas in, I have to regularly purge my poetry cache folders and run poetry install again from within the shell to get native/FFI code properly linked again.

Poetry/python will link to whatever is available when first built a package and cache it that way, though nixos moves on and eventually garbage collects what got linked by poetry building stuff.

I have not yet found out how to force poetry into not using its package cache but instead always rebuild from scratch.

4

I ended up using the packaged pandas.

python3.withPackages (ps: with ps; [ pandas ]);
1 Like
5

I gave up on this until now. Just tried with cleaning the cache in .~/cache/pypoetry but it still does not work, e.g. libz is somehow missing. Any suggestions?

6

The problem seems to be a dynamic linking problem that your python program fail to find the zlib artifact, adding the path to LD_LIBRARY_PATH will work.

Example shell.nix

{ pkgs ? import <nixpkgs> { } }:

pkgs.mkShell rec {

  buildInputs = [
    pkgs.python3
    pkgs.poetry
    pkgs.zlib
  ];

  shellHook = ''
    export LD_LIBRARY_PATH="${pkgs.lib.makeLibraryPath buildInputs}:$LD_LIBRARY_PATH"
    export LD_LIBRARY_PATH="${pkgs.stdenv.cc.cc.lib.outPath}/lib:$LD_LIBRARY_PATH"
  '';
}

The above shell can fail actually because I don’t know which nixpkgs channel you’re using. Would be nice if this can help you.

5 Likes