Python library is empty

Help
1

I’m trying to package the python binding of HaHyPar but I don’t know why, I can import the library but the library is emply:

$ nix-shell
$ python3
Python 3.9.13 (main, May 17 2022, 14:19:07) 
[GCC 11.3.0] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> import kahypar as kahypar
>>> kahypar.Hypergraph
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
AttributeError: module 'kahypar' has no attribute 'Hypergraph'
>>> kahypar.<TAB><TAB>
kahypar.__class__(          kahypar.__ge__(             kahypar.__lt__(             kahypar.__repr__(
kahypar.__delattr__(        kahypar.__getattribute__(   kahypar.__name__            kahypar.__setattr__(
kahypar.__dict__            kahypar.__gt__(             kahypar.__ne__(             kahypar.__sizeof__(
kahypar.__dir__(            kahypar.__hash__(           kahypar.__new__(            kahypar.__spec__
kahypar.__doc__             kahypar.__init__(           kahypar.__package__         kahypar.__str__(
kahypar.__eq__(             kahypar.__init_subclass__(  kahypar.__path__            kahypar.__subclasshook__(
kahypar.__file__            kahypar.__le__(             kahypar.__reduce__(         
kahypar.__format__(         kahypar.__loader__          kahypar.__reduce_ex__(

I also tried to import it with python in verbose mode:

>>> import kahypar as kahypar
# possible namespace for /nix/store/snlbkcd6kjyraad108fd0bbqc5q1l9fj-python3.9-hahypar/lib/python3.9/site-packages/kahypar
# possible namespace for /nix/store/ha04dvr9g4mmya714xjh3x6afd8hmdf7-python3-3.9.13-env/lib/python3.9/site-packages/kahypar
import 'kahypar' # <_frozen_importlib_external._NamespaceLoader object at 0x7f0e1ca64490>

However if I check the .so file (that I compiled and copied to the site package as explained in the doc) it seems to contain some strings involving hypergraph so I guess the compilation did something meaningful:

$ cat /nix/store/snlbkcd6kjyraad108fd0bbqc5q1l9fj-python3.9-hahypar/lib/libkahypar.so | grep Hypergraph
grep: (standard input): binary file matches

Any idea what I am missing?

For reference here is my shell.nix

# https://docs.python.org/3/extending/building.html
with (import (fetchTarball https://github.com/nixos/nixpkgs/archive/67e45078141102f45eff1589a831aeaa3182b41e.tar.gz) {});
let
  kahypar = python3Packages.buildPythonPackage {
    format = "other";
    name = "kahypar";
    version = "1.0";
    nativeBuildInputs = [ cmake doxygen git gtest python3 ];
    buildInputs = [ boost ];
    src = pkgs.fetchFromGitHub {
      owner = "kahypar";
      repo = "kahypar";
      rev = "9fa8bd0ac49c3b71fce89bdbe9804c45ffa6f6e3";
      sha256 = "sha256-6Bvv1v5BoJ0TKK7dwhPyiLCBCbhDceyyXi331rGglf8=";
      fetchSubmodules = true;
    };
    cmakeFlags = [
      "-DKAHYPAR_PYTHON_INTERFACE=1"
    ];

    postBuild = ''
      echo "Following the doc we compile the python software now"
      ls
      cd python
      make
      cd ..
      pwd
      ls
    '';

    postInstall = ''
      echo "Following the doc we install python/kahypar.so in site-package (to find the path I did 'rg site-package' in nixpkgs)"
      pwd
      mkdir -p $out/lib/${python3Packages.python.libPrefix}/site-packages/kahypar/
      cp python/*.so $out/lib/${python3Packages.python.libPrefix}/site-packages/kahypar/ #kahypar.so
    '';
  };
in
mkShell {
  packages = [
    kahypar
    (python3.withPackages (ps: [kahypar ps.requests]))
  ];
}
2

You created a package (folder) kahypar. Then, you copied in a module, kahypar.so. The full name of that module is then kahypar.kahypar and the class of interest kahypar.kahypar.Hypergraph.

Now, note that you did not put a __init__.py in the folder, so the package is what is called a namespace package. That shouldn’t really be any problem though. Note that tab completion will not show the nested kahypar folder.

1 Like
3

Awesome, thanks a lot! So is the proper fix simply to put instead kahypar.so at the root of the site-package? And if I put a kahypar/__init__.py file to have a non namespace package, should I also keep kahypar at the root of the site package?

4

The “proper” fix would be to use a build system that produces a wheel that then installs the library correctly, including metadata which is now missing.

It’s really up to you (or the package author) how they want to expose the symbols, that is, what the desired import should look like.

5

You mean something like a setup.py? But then I would be confused, how would I compile it in nix? For now I use format=''other'' to compile since I need cmake, so I’m not sure to see how I should call the setup.py in this case in the same derivation.

6

For example. You would need to dive more into learning Python packaging, this is not the right place for that.

If what you have works for you, then keep it that way.