How to package monorepo with autoconf + subfolder python package?

bravo · November 26, 2024, 12:17pm

So i would like to package MDSplus (GitHub - MDSplus/mdsplus: The MDSplus data management system), which is a big monorepo according to their documentation i would need to use autoconf to generate the makefile and also configure the python package which is contained in a subfolder afterwards i would need to build the python package in the subfolder. What would be the recommended way of doing this?
Currently I am doing the following:

{
  lib,
  fetchFromGitHub,
  buildPythonPackage,
  setuptools,
  wheel,
  numpy,
  readline70,
}:
buildPythonPackage  rec {
  pname = "mdsplus";
  version = "7-148-2";
  format = "pyproject";

  src = fetchFromGitHub {
    owner = "MDSplus";
    repo = "mdsplus";
    rev = "alpha_release-${version}";
    hash = "sha256-LEHDxPFPWJ4qVYcKs6gPGZ85v8yMocpZAto7r+tytFw=";
    fetchSubmodules = true;
  };
  pyversion = lib.replaceStrings ["-"] [","] version;
  postPatch = ''
    echo "version = (${pyversion})" > ./_version.py
  '';

  propagatedBuildInputs = [readline70];

  dependencies = [
    numpy
  ];

  sourceRoot = "${src.name}/python/MDSplus/";

  pythonImportsCheck = [
    "MDSplus"
  ];

  build-system = [setuptools wheel];
  meta = {
    description = "The MDSplus data management system";
    homepage = "https://github.com/MDSplus/mdsplus/tree/alpha/python/MDSplus";
    # license = lib.licenses.unfree; # FIXME: nix-init did not find a license
    maintainers = with lib.maintainers; [ ];
    mainProgram = "mdsplus";
    platforms = lib.platforms.all;
  };
}

but it seems like the configure step is doing more than just setting the version.

doronbehar · November 26, 2024, 12:46pm

This is the build error I get with this expression (not a bad start):

Sourcing python-remove-tests-dir-hook
Sourcing python-catch-conflicts-hook.sh
Sourcing python-remove-bin-bytecode-hook.sh
Sourcing pypa-build-hook
Using pypaBuildPhase
Sourcing python-runtime-deps-check-hook
Using pythonRuntimeDepsCheckHook
Sourcing pypa-install-hook
Using pypaInstallPhase
Sourcing python-imports-check-hook.sh
Using pythonImportsCheckPhase
Sourcing python-namespaces-hook
Sourcing python-catch-conflicts-hook.sh
Running phase: unpackPhase
@nix { "action": "setPhase", "phase": "unpackPhase" }
unpacking source archive /nix/store/q9c0743qq5qgdh9pn5mdf82yhy8k1wmj-source
source root is source/python/MDSplus
setting SOURCE_DATE_EPOCH to timestamp 315619200 of file source/python/MDSplus/wsgi/js/mdsScope.js
Running phase: patchPhase
@nix { "action": "setPhase", "phase": "patchPhase" }
Running phase: updateAutotoolsGnuConfigScriptsPhase
@nix { "action": "setPhase", "phase": "updateAutotoolsGnuConfigScriptsPhase" }
Running phase: configurePhase
@nix { "action": "setPhase", "phase": "configurePhase" }
no configure script, doing nothing
Running phase: buildPhase
@nix { "action": "setPhase", "phase": "buildPhase" }
Executing pypaBuildPhase
Creating a wheel...
pypa build flags: --no-isolation --outdir dist/ --wheel
* Getting build dependencies for wheel...
running egg_info
creating MDSplus.egg-info
writing MDSplus.egg-info/PKG-INFO
writing dependency_links to MDSplus.egg-info/dependency_links.txt
writing requirements to MDSplus.egg-info/requires.txt
writing top-level names to MDSplus.egg-info/top_level.txt
writing manifest file 'MDSplus.egg-info/SOURCES.txt'
reading manifest file 'MDSplus.egg-info/SOURCES.txt'
writing manifest file 'MDSplus.egg-info/SOURCES.txt'
* Building wheel...
running bdist_wheel
running build
running build_py
creating build/lib/MDSplus
copying ./__init__.py -> build/lib/MDSplus
copying ./_mdsshr.py -> build/lib/MDSplus
copying ./apd.py -> build/lib/MDSplus
copying ./connection.py -> build/lib/MDSplus
copying ./descriptor.py -> build/lib/MDSplus
copying ./event.py -> build/lib/MDSplus
copying ./magic.py -> build/lib/MDSplus
copying ./mdsarray.py -> build/lib/MDSplus
copying ./mdsdata.py -> build/lib/MDSplus
copying ./mdsdcl.py -> build/lib/MDSplus
copying ./mdsscalar.py -> build/lib/MDSplus
copying ./modpython.py -> build/lib/MDSplus
copying ./scope.py -> build/lib/MDSplus
copying ./setup.py -> build/lib/MDSplus
copying ./tree.py -> build/lib/MDSplus
copying ./version.py -> build/lib/MDSplus
copying ./_version.py -> build/lib/MDSplus
creating build/lib/MDSplus/widgets
copying widgets/__init__.py -> build/lib/MDSplus/widgets
copying widgets/mdsplusactionwidget.py -> build/lib/MDSplus/widgets
copying widgets/mdsplusdigchans.py -> build/lib/MDSplus/widgets
copying widgets/mdsplusdispatchwidget.py -> build/lib/MDSplus/widgets
copying widgets/mdsplusdtypeselwidget.py -> build/lib/MDSplus/widgets
copying widgets/mdspluserrormsg.py -> build/lib/MDSplus/widgets
copying widgets/mdsplusexpr.py -> build/lib/MDSplus/widgets
copying widgets/mdsplusexpressionwidget.py -> build/lib/MDSplus/widgets
copying widgets/mdsplusmethodwidget.py -> build/lib/MDSplus/widgets
copying widgets/mdsplusnidoptionwidget.py -> build/lib/MDSplus/widgets
copying widgets/mdsplusokbuttons.py -> build/lib/MDSplus/widgets
copying widgets/mdsplusonoff.py -> build/lib/MDSplus/widgets
copying widgets/mdsplusonoffxdbox.py -> build/lib/MDSplus/widgets
copying widgets/mdspluspathwidget.py -> build/lib/MDSplus/widgets
copying widgets/mdsplusrangewidget.py -> build/lib/MDSplus/widgets
copying widgets/mdsplusroutinewidget.py -> build/lib/MDSplus/widgets
copying widgets/mdsplussequentialwidget.py -> build/lib/MDSplus/widgets
copying widgets/mdsplustaskwidget.py -> build/lib/MDSplus/widgets
copying widgets/mdspluswidget.py -> build/lib/MDSplus/widgets
copying widgets/mdspluswindowwidget.py -> build/lib/MDSplus/widgets
copying widgets/mdsplusxdbox.py -> build/lib/MDSplus/widgets
copying widgets/mdsplusxdboxbutton.py -> build/lib/MDSplus/widgets
creating build/lib/MDSplus/wsgi
copying wsgi/__init__.py -> build/lib/MDSplus/wsgi
copying wsgi/do1darray.py -> build/lib/MDSplus/wsgi
copying wsgi/do1dsignal.py -> build/lib/MDSplus/wsgi
copying wsgi/doDir.py -> build/lib/MDSplus/wsgi
copying wsgi/doEvent.py -> build/lib/MDSplus/wsgi
copying wsgi/doFrame.py -> build/lib/MDSplus/wsgi
copying wsgi/doGetnid.py -> build/lib/MDSplus/wsgi
copying wsgi/doImage.py -> build/lib/MDSplus/wsgi
copying wsgi/doJavascripts.py -> build/lib/MDSplus/wsgi
copying wsgi/doMdsip.py -> build/lib/MDSplus/wsgi
copying wsgi/doPlot.py -> build/lib/MDSplus/wsgi
copying wsgi/doScope.py -> build/lib/MDSplus/wsgi
copying wsgi/doTreepath.py -> build/lib/MDSplus/wsgi
creating build/lib/MDSplus/tests
copying tests/__init__.py -> build/lib/MDSplus/tests
copying tests/_common.py -> build/lib/MDSplus/tests
copying tests/connection_case.py -> build/lib/MDSplus/tests
copying tests/data_case.py -> build/lib/MDSplus/tests
copying tests/dcl_case.py -> build/lib/MDSplus/tests
copying tests/devices_case.py -> build/lib/MDSplus/tests
copying tests/exception_case.py -> build/lib/MDSplus/tests
copying tests/segment_case.py -> build/lib/MDSplus/tests
copying tests/task_case.py -> build/lib/MDSplus/tests
copying tests/thread_case.py -> build/lib/MDSplus/tests
copying tests/tree_case.py -> build/lib/MDSplus/tests
creating build/lib/MDSplus/wsgi/html
copying wsgi/html/scope.html -> build/lib/MDSplus/wsgi/html
creating build/lib/MDSplus/wsgi/conf
copying wsgi/conf/25-mdsplus.conf -> build/lib/MDSplus/wsgi/conf
copying wsgi/conf/mdsplus.conf -> build/lib/MDSplus/wsgi/conf
copying wsgi/conf/mdsplus.wsgi -> build/lib/MDSplus/wsgi/conf
creating build/lib/MDSplus/wsgi/js
copying wsgi/js/mdsScope.js -> build/lib/MDSplus/wsgi/js
copying wsgi/colors1.tbl -> build/lib/MDSplus/wsgi
installing to build/bdist.linux-x86_64/wheel
running install
running install_lib
creating build/bdist.linux-x86_64/wheel
creating build/bdist.linux-x86_64/wheel/MDSplus
copying build/lib/MDSplus/__init__.py -> build/bdist.linux-x86_64/wheel/./MDSplus
copying build/lib/MDSplus/_mdsshr.py -> build/bdist.linux-x86_64/wheel/./MDSplus
copying build/lib/MDSplus/apd.py -> build/bdist.linux-x86_64/wheel/./MDSplus
copying build/lib/MDSplus/connection.py -> build/bdist.linux-x86_64/wheel/./MDSplus
copying build/lib/MDSplus/descriptor.py -> build/bdist.linux-x86_64/wheel/./MDSplus
copying build/lib/MDSplus/event.py -> build/bdist.linux-x86_64/wheel/./MDSplus
copying build/lib/MDSplus/magic.py -> build/bdist.linux-x86_64/wheel/./MDSplus
copying build/lib/MDSplus/mdsarray.py -> build/bdist.linux-x86_64/wheel/./MDSplus
copying build/lib/MDSplus/mdsdata.py -> build/bdist.linux-x86_64/wheel/./MDSplus
copying build/lib/MDSplus/mdsdcl.py -> build/bdist.linux-x86_64/wheel/./MDSplus
copying build/lib/MDSplus/mdsscalar.py -> build/bdist.linux-x86_64/wheel/./MDSplus
copying build/lib/MDSplus/modpython.py -> build/bdist.linux-x86_64/wheel/./MDSplus
copying build/lib/MDSplus/scope.py -> build/bdist.linux-x86_64/wheel/./MDSplus
copying build/lib/MDSplus/setup.py -> build/bdist.linux-x86_64/wheel/./MDSplus
copying build/lib/MDSplus/tree.py -> build/bdist.linux-x86_64/wheel/./MDSplus
copying build/lib/MDSplus/version.py -> build/bdist.linux-x86_64/wheel/./MDSplus
copying build/lib/MDSplus/_version.py -> build/bdist.linux-x86_64/wheel/./MDSplus
creating build/bdist.linux-x86_64/wheel/MDSplus/widgets
copying build/lib/MDSplus/widgets/__init__.py -> build/bdist.linux-x86_64/wheel/./MDSplus/widgets
copying build/lib/MDSplus/widgets/mdsplusactionwidget.py -> build/bdist.linux-x86_64/wheel/./MDSplus/widgets
copying build/lib/MDSplus/widgets/mdsplusdigchans.py -> build/bdist.linux-x86_64/wheel/./MDSplus/widgets
copying build/lib/MDSplus/widgets/mdsplusdispatchwidget.py -> build/bdist.linux-x86_64/wheel/./MDSplus/widgets
copying build/lib/MDSplus/widgets/mdsplusdtypeselwidget.py -> build/bdist.linux-x86_64/wheel/./MDSplus/widgets
copying build/lib/MDSplus/widgets/mdspluserrormsg.py -> build/bdist.linux-x86_64/wheel/./MDSplus/widgets
copying build/lib/MDSplus/widgets/mdsplusexpr.py -> build/bdist.linux-x86_64/wheel/./MDSplus/widgets
copying build/lib/MDSplus/widgets/mdsplusexpressionwidget.py -> build/bdist.linux-x86_64/wheel/./MDSplus/widgets
copying build/lib/MDSplus/widgets/mdsplusmethodwidget.py -> build/bdist.linux-x86_64/wheel/./MDSplus/widgets
copying build/lib/MDSplus/widgets/mdsplusnidoptionwidget.py -> build/bdist.linux-x86_64/wheel/./MDSplus/widgets
copying build/lib/MDSplus/widgets/mdsplusokbuttons.py -> build/bdist.linux-x86_64/wheel/./MDSplus/widgets
copying build/lib/MDSplus/widgets/mdsplusonoff.py -> build/bdist.linux-x86_64/wheel/./MDSplus/widgets
copying build/lib/MDSplus/widgets/mdsplusonoffxdbox.py -> build/bdist.linux-x86_64/wheel/./MDSplus/widgets
copying build/lib/MDSplus/widgets/mdspluspathwidget.py -> build/bdist.linux-x86_64/wheel/./MDSplus/widgets
copying build/lib/MDSplus/widgets/mdsplusrangewidget.py -> build/bdist.linux-x86_64/wheel/./MDSplus/widgets
copying build/lib/MDSplus/widgets/mdsplusroutinewidget.py -> build/bdist.linux-x86_64/wheel/./MDSplus/widgets
copying build/lib/MDSplus/widgets/mdsplussequentialwidget.py -> build/bdist.linux-x86_64/wheel/./MDSplus/widgets
copying build/lib/MDSplus/widgets/mdsplustaskwidget.py -> build/bdist.linux-x86_64/wheel/./MDSplus/widgets
copying build/lib/MDSplus/widgets/mdspluswidget.py -> build/bdist.linux-x86_64/wheel/./MDSplus/widgets
copying build/lib/MDSplus/widgets/mdspluswindowwidget.py -> build/bdist.linux-x86_64/wheel/./MDSplus/widgets
copying build/lib/MDSplus/widgets/mdsplusxdbox.py -> build/bdist.linux-x86_64/wheel/./MDSplus/widgets
copying build/lib/MDSplus/widgets/mdsplusxdboxbutton.py -> build/bdist.linux-x86_64/wheel/./MDSplus/widgets
creating build/bdist.linux-x86_64/wheel/MDSplus/wsgi
copying build/lib/MDSplus/wsgi/__init__.py -> build/bdist.linux-x86_64/wheel/./MDSplus/wsgi
copying build/lib/MDSplus/wsgi/do1darray.py -> build/bdist.linux-x86_64/wheel/./MDSplus/wsgi
copying build/lib/MDSplus/wsgi/do1dsignal.py -> build/bdist.linux-x86_64/wheel/./MDSplus/wsgi
copying build/lib/MDSplus/wsgi/doDir.py -> build/bdist.linux-x86_64/wheel/./MDSplus/wsgi
copying build/lib/MDSplus/wsgi/doEvent.py -> build/bdist.linux-x86_64/wheel/./MDSplus/wsgi
copying build/lib/MDSplus/wsgi/doFrame.py -> build/bdist.linux-x86_64/wheel/./MDSplus/wsgi
copying build/lib/MDSplus/wsgi/doGetnid.py -> build/bdist.linux-x86_64/wheel/./MDSplus/wsgi
copying build/lib/MDSplus/wsgi/doImage.py -> build/bdist.linux-x86_64/wheel/./MDSplus/wsgi
copying build/lib/MDSplus/wsgi/doJavascripts.py -> build/bdist.linux-x86_64/wheel/./MDSplus/wsgi
copying build/lib/MDSplus/wsgi/doMdsip.py -> build/bdist.linux-x86_64/wheel/./MDSplus/wsgi
copying build/lib/MDSplus/wsgi/doPlot.py -> build/bdist.linux-x86_64/wheel/./MDSplus/wsgi
copying build/lib/MDSplus/wsgi/doScope.py -> build/bdist.linux-x86_64/wheel/./MDSplus/wsgi
copying build/lib/MDSplus/wsgi/doTreepath.py -> build/bdist.linux-x86_64/wheel/./MDSplus/wsgi
creating build/bdist.linux-x86_64/wheel/MDSplus/wsgi/html
copying build/lib/MDSplus/wsgi/html/scope.html -> build/bdist.linux-x86_64/wheel/./MDSplus/wsgi/html
creating build/bdist.linux-x86_64/wheel/MDSplus/wsgi/conf
copying build/lib/MDSplus/wsgi/conf/25-mdsplus.conf -> build/bdist.linux-x86_64/wheel/./MDSplus/wsgi/conf
copying build/lib/MDSplus/wsgi/conf/mdsplus.conf -> build/bdist.linux-x86_64/wheel/./MDSplus/wsgi/conf
copying build/lib/MDSplus/wsgi/conf/mdsplus.wsgi -> build/bdist.linux-x86_64/wheel/./MDSplus/wsgi/conf
creating build/bdist.linux-x86_64/wheel/MDSplus/wsgi/js
copying build/lib/MDSplus/wsgi/js/mdsScope.js -> build/bdist.linux-x86_64/wheel/./MDSplus/wsgi/js
copying build/lib/MDSplus/wsgi/colors1.tbl -> build/bdist.linux-x86_64/wheel/./MDSplus/wsgi
creating build/bdist.linux-x86_64/wheel/MDSplus/tests
copying build/lib/MDSplus/tests/__init__.py -> build/bdist.linux-x86_64/wheel/./MDSplus/tests
copying build/lib/MDSplus/tests/_common.py -> build/bdist.linux-x86_64/wheel/./MDSplus/tests
copying build/lib/MDSplus/tests/connection_case.py -> build/bdist.linux-x86_64/wheel/./MDSplus/tests
copying build/lib/MDSplus/tests/data_case.py -> build/bdist.linux-x86_64/wheel/./MDSplus/tests
copying build/lib/MDSplus/tests/dcl_case.py -> build/bdist.linux-x86_64/wheel/./MDSplus/tests
copying build/lib/MDSplus/tests/devices_case.py -> build/bdist.linux-x86_64/wheel/./MDSplus/tests
copying build/lib/MDSplus/tests/exception_case.py -> build/bdist.linux-x86_64/wheel/./MDSplus/tests
copying build/lib/MDSplus/tests/segment_case.py -> build/bdist.linux-x86_64/wheel/./MDSplus/tests
copying build/lib/MDSplus/tests/task_case.py -> build/bdist.linux-x86_64/wheel/./MDSplus/tests
copying build/lib/MDSplus/tests/thread_case.py -> build/bdist.linux-x86_64/wheel/./MDSplus/tests
copying build/lib/MDSplus/tests/tree_case.py -> build/bdist.linux-x86_64/wheel/./MDSplus/tests
running install_egg_info
running egg_info
writing MDSplus.egg-info/PKG-INFO
writing dependency_links to MDSplus.egg-info/dependency_links.txt
writing requirements to MDSplus.egg-info/requires.txt
writing top-level names to MDSplus.egg-info/top_level.txt
reading manifest file 'MDSplus.egg-info/SOURCES.txt'
writing manifest file 'MDSplus.egg-info/SOURCES.txt'
Copying MDSplus.egg-info to build/bdist.linux-x86_64/wheel/./MDSplus-7.148.2-py3.12.egg-info
running install_scripts
creating build/bdist.linux-x86_64/wheel/MDSplus-7.148.2.dist-info/WHEEL
creating '/build/source/python/MDSplus/dist/.tmp-ar1tlb0e/MDSplus-7.148.2-py3-none-any.whl' and adding 'build/bdist.linux-x86_64/wheel' to it
adding 'MDSplus/__init__.py'
adding 'MDSplus/_mdsshr.py'
adding 'MDSplus/_version.py'
adding 'MDSplus/apd.py'
adding 'MDSplus/connection.py'
adding 'MDSplus/descriptor.py'
adding 'MDSplus/event.py'
adding 'MDSplus/magic.py'
adding 'MDSplus/mdsarray.py'
adding 'MDSplus/mdsdata.py'
adding 'MDSplus/mdsdcl.py'
adding 'MDSplus/mdsscalar.py'
adding 'MDSplus/modpython.py'
adding 'MDSplus/scope.py'
adding 'MDSplus/setup.py'
adding 'MDSplus/tree.py'
adding 'MDSplus/version.py'
adding 'MDSplus/tests/__init__.py'
adding 'MDSplus/tests/_common.py'
adding 'MDSplus/tests/connection_case.py'
adding 'MDSplus/tests/data_case.py'
adding 'MDSplus/tests/dcl_case.py'
adding 'MDSplus/tests/devices_case.py'
adding 'MDSplus/tests/exception_case.py'
adding 'MDSplus/tests/segment_case.py'
adding 'MDSplus/tests/task_case.py'
adding 'MDSplus/tests/thread_case.py'
adding 'MDSplus/tests/tree_case.py'
adding 'MDSplus/widgets/__init__.py'
adding 'MDSplus/widgets/mdsplusactionwidget.py'
adding 'MDSplus/widgets/mdsplusdigchans.py'
adding 'MDSplus/widgets/mdsplusdispatchwidget.py'
adding 'MDSplus/widgets/mdsplusdtypeselwidget.py'
adding 'MDSplus/widgets/mdspluserrormsg.py'
adding 'MDSplus/widgets/mdsplusexpr.py'
adding 'MDSplus/widgets/mdsplusexpressionwidget.py'
adding 'MDSplus/widgets/mdsplusmethodwidget.py'
adding 'MDSplus/widgets/mdsplusnidoptionwidget.py'
adding 'MDSplus/widgets/mdsplusokbuttons.py'
adding 'MDSplus/widgets/mdsplusonoff.py'
adding 'MDSplus/widgets/mdsplusonoffxdbox.py'
adding 'MDSplus/widgets/mdspluspathwidget.py'
adding 'MDSplus/widgets/mdsplusrangewidget.py'
adding 'MDSplus/widgets/mdsplusroutinewidget.py'
adding 'MDSplus/widgets/mdsplussequentialwidget.py'
adding 'MDSplus/widgets/mdsplustaskwidget.py'
adding 'MDSplus/widgets/mdspluswidget.py'
adding 'MDSplus/widgets/mdspluswindowwidget.py'
adding 'MDSplus/widgets/mdsplusxdbox.py'
adding 'MDSplus/widgets/mdsplusxdboxbutton.py'
adding 'MDSplus/wsgi/__init__.py'
adding 'MDSplus/wsgi/colors1.tbl'
adding 'MDSplus/wsgi/do1darray.py'
adding 'MDSplus/wsgi/do1dsignal.py'
adding 'MDSplus/wsgi/doDir.py'
adding 'MDSplus/wsgi/doEvent.py'
adding 'MDSplus/wsgi/doFrame.py'
adding 'MDSplus/wsgi/doGetnid.py'
adding 'MDSplus/wsgi/doImage.py'
adding 'MDSplus/wsgi/doJavascripts.py'
adding 'MDSplus/wsgi/doMdsip.py'
adding 'MDSplus/wsgi/doPlot.py'
adding 'MDSplus/wsgi/doScope.py'
adding 'MDSplus/wsgi/doTreepath.py'
adding 'MDSplus/wsgi/conf/25-mdsplus.conf'
adding 'MDSplus/wsgi/conf/mdsplus.conf'
adding 'MDSplus/wsgi/conf/mdsplus.wsgi'
adding 'MDSplus/wsgi/html/scope.html'
adding 'MDSplus/wsgi/js/mdsScope.js'
adding 'MDSplus-7.148.2.dist-info/METADATA'
adding 'MDSplus-7.148.2.dist-info/WHEEL'
adding 'MDSplus-7.148.2.dist-info/top_level.txt'
adding 'MDSplus-7.148.2.dist-info/RECORD'
removing build/bdist.linux-x86_64/wheel
Successfully built MDSplus-7.148.2-py3-none-any.whl
Finished creating a wheel...
Finished executing pypaBuildPhase
Running phase: pythonRuntimeDepsCheckHook
@nix { "action": "setPhase", "phase": "pythonRuntimeDepsCheckHook" }
Executing pythonRuntimeDepsCheck
Checking runtime dependencies for MDSplus-7.148.2-py3-none-any.whl
Finished executing pythonRuntimeDepsCheck
Running phase: installPhase
@nix { "action": "setPhase", "phase": "installPhase" }
Executing pypaInstallPhase
/nix/store/lr5n22jddw20xgzn899744gby37icmw8-python3.12-mdsplus-7-148-2/lib/python3.12/site-packages/MDSplus/widgets/mdsplusdigchans.py:100: SyntaxWarning: invalid escape sequence '\%'
  MDSplusErrorMsg('Error storing value', 'Error storing value %s for channel %d\n\n\%s' % (
/nix/store/lr5n22jddw20xgzn899744gby37icmw8-python3.12-mdsplus-7-148-2/lib/python3.12/site-packages/MDSplus/widgets/mdsplusdigchans.py:100: SyntaxWarning: invalid escape sequence '\%'
  MDSplusErrorMsg('Error storing value', 'Error storing value %s for channel %d\n\n\%s' % (
/nix/store/lr5n22jddw20xgzn899744gby37icmw8-python3.12-mdsplus-7-148-2/lib/python3.12/site-packages/MDSplus/wsgi/__init__.py:26: SyntaxWarning: invalid escape sequence '\i'
  """Use as a mod_wsgi handler in apache.
/nix/store/lr5n22jddw20xgzn899744gby37icmw8-python3.12-mdsplus-7-148-2/lib/python3.12/site-packages/MDSplus/wsgi/__init__.py:26: SyntaxWarning: invalid escape sequence '\i'
  """Use as a mod_wsgi handler in apache.
Successfully installed MDSplus-7.148.2-py3-none-any.whl
Finished executing pypaInstallPhase
Running phase: pythonOutputDistPhase
@nix { "action": "setPhase", "phase": "pythonOutputDistPhase" }
Executing pythonOutputDistPhase
Finished executing pythonOutputDistPhase
Running phase: fixupPhase
@nix { "action": "setPhase", "phase": "fixupPhase" }
shrinking RPATHs of ELF executables and libraries in /nix/store/lr5n22jddw20xgzn899744gby37icmw8-python3.12-mdsplus-7-148-2
checking for references to /build/ in /nix/store/lr5n22jddw20xgzn899744gby37icmw8-python3.12-mdsplus-7-148-2...
patching script interpreter paths in /nix/store/lr5n22jddw20xgzn899744gby37icmw8-python3.12-mdsplus-7-148-2
/nix/store/lr5n22jddw20xgzn899744gby37icmw8-python3.12-mdsplus-7-148-2/lib/python3.12/site-packages/MDSplus/wsgi/conf/mdsplus.wsgi: interpreter directive changed from "#!/usr/bin/env python" to "/nix/store/px2nj16i5gc3d4mnw5l1nclfdxhry61p-python3-3.12.7/bin/python"
stripping (with command strip and flags -S -p) in  /nix/store/lr5n22jddw20xgzn899744gby37icmw8-python3.12-mdsplus-7-148-2/lib
shrinking RPATHs of ELF executables and libraries in /nix/store/rw2rff8dvf8z78r06027j2lq92zxmn8d-python3.12-mdsplus-7-148-2-dist
checking for references to /build/ in /nix/store/rw2rff8dvf8z78r06027j2lq92zxmn8d-python3.12-mdsplus-7-148-2-dist...
patching script interpreter paths in /nix/store/rw2rff8dvf8z78r06027j2lq92zxmn8d-python3.12-mdsplus-7-148-2-dist
Executing pythonRemoveTestsDir
Finished executing pythonRemoveTestsDir
Running phase: installCheckPhase
@nix { "action": "setPhase", "phase": "installCheckPhase" }
no Makefile or custom installCheckPhase, doing nothing
Running phase: pythonCatchConflictsPhase
@nix { "action": "setPhase", "phase": "pythonCatchConflictsPhase" }
Running phase: pythonRemoveBinBytecodePhase
@nix { "action": "setPhase", "phase": "pythonRemoveBinBytecodePhase" }
Running phase: pythonImportsCheckPhase
@nix { "action": "setPhase", "phase": "pythonImportsCheckPhase" }
Executing pythonImportsCheckPhase
Check whether the following modules can be imported: MDSplus
Issues loading MdsShr, trying find_library
Traceback (most recent call last):
  File "<string>", line 1, in <module>
  File "<string>", line 1, in <lambda>
  File "/nix/store/px2nj16i5gc3d4mnw5l1nclfdxhry61p-python3-3.12.7/lib/python3.12/importlib/__init__.py", line 90, in import_module
    return _bootstrap._gcd_import(name[level:], package, level)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "<frozen importlib._bootstrap>", line 1387, in _gcd_import
  File "<frozen importlib._bootstrap>", line 1360, in _find_and_load
  File "<frozen importlib._bootstrap>", line 1331, in _find_and_load_unlocked
  File "<frozen importlib._bootstrap>", line 935, in _load_unlocked
  File "<frozen importlib._bootstrap_external>", line 995, in exec_module
  File "<frozen importlib._bootstrap>", line 488, in _call_with_frames_removed
  File "/nix/store/lr5n22jddw20xgzn899744gby37icmw8-python3.12-mdsplus-7-148-2/lib/python3.12/site-packages/MDSplus/__init__.py", line 64, in <module>
    class libs:
  File "/nix/store/lr5n22jddw20xgzn899744gby37icmw8-python3.12-mdsplus-7-148-2/lib/python3.12/site-packages/MDSplus/__init__.py", line 65, in libs
    MdsShr = _ver.load_library('MdsShr')
             ^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/nix/store/lr5n22jddw20xgzn899744gby37icmw8-python3.12-mdsplus-7-148-2/lib/python3.12/site-packages/MDSplus/version.py", line 99, in load_library
    raise ImportError("Could not find library: %s" % (name,))
ImportError: Could not find library: MdsShr

It is not a problem in the configurePhase so it seems to me. Note how without that pythonImportsCheck, you’d get a working build, but of course you won’t be able to actually use the library. You can read the Python code that you get in the output when pythonImportsCheck is disabled, and figure out what exactly fails. I guess it is a shared object that is missing.

If my guess is correct, and you’d also like to package other stuff from that monorepo, such as C libraries etc, I’d start there, and perhaps the solution in the end for the Python library will be to symlink the shared object from the derivation that builds this MdsShr thing to the the Python package’s output.

A somewhat interesting package to compare with, is lammps which also has a Python library that simply is linked to the shared object that the non-python derivation lammps is compiling.

bravo · November 26, 2024, 12:52pm

thx for the fast reply i will look into lammps

bravo · December 4, 2024, 1:35pm

I think i was able to build the c derivation and it builds some the libraries using the following derivation (IMPORTANT it doesn’t work on unstable only on the latest stable 24.05). However the result folder is quite crowded and not really usable. How should i prepare export such that the libraries are available when i build the python package? When I pass the derivation of the package to the python package it is still not able to find the MdsShr shared library. I assume this is due to the mangled output.

{
  lib,
  stdenv,
  fetchFromGitHub,
  gperf,
  jdk8,
  jre8,
  libxml2,
  readline,
  motif,
  perl,
  xorg,
  which,
  automake,
  autoconf,
  autoconf-archive,
  python3,
  git,
  flex,
  bison,
  gfortran,
  rsync,
  pkg-config,
  # xz, # is need according to them
  # jdk, # maybe could work also just using the newest jdk
  # libz, # we need zlib.h and it isn't included here
}:

stdenv.mkDerivation rec {
  pname = "mdsplus";
  version = "7-142-81";

  src = fetchFromGitHub {
    owner = "MDSplus";
    repo = "mdsplus";
    rev = "stable_release-${version}";
    hash = "sha256-GV2Ry6NoYTpHWPKqgzK2TNo3BbX9OHdkFExCVIO5oEg=";
    fetchSubmodules = true;
  };

  patchPhase = ''
    patchShebangs --build .
  '';
  preConfigure = ''
    ./bootstrap
  '';
    buildInputs = [ 
				gperf
				jdk8
				jre8
				libxml2.dev
				readline
				motif
				perl
				xorg.libXt
        # libz
			];
    nativeBuildInputs = [
				which
				automake
				autoconf
				autoconf-archive
				python3
				git
				flex
				bison
				gfortran
				rsync
        pkg-config
			];

  meta = {
    description = "The MDSplus data management system";
    homepage = "https://github.com/MDSplus/mdsplus";
    # license = lib.licenses.unfree; # FIXME: nix-init did not find a license
    maintainers = with lib.maintainers; [ ];
    mainProgram = "mdsplus";
    platforms = lib.platforms.all;
  };
}

ls ./result

bin  ChangeLog  desktop  epics  etc  idl  include  java  LabView  lib  local  matlab  MDSplus-License.rtf  MDSplus-License.txt  nodejs  php  pixmaps  pydevices  python  rpm  setup.csh  setup.sh  share  tdi  trees  uid  xml

could you give a direction i should look into. It seems that a python folder is also included in the out put where the _version.py is populated. I believe that

        return C.CDLL('lib%s.so' % name)

is not able to find the package.

bravo · December 5, 2024, 10:10am

After doing the c build i hijack now the build context by setting the source to mdsplus (c) build directory and choosing the sourceRoot to be the python subproject. Building this subproject now has still the same issue that the ld libraries aren’t found. Even though they exist in …/…/lib . I will make a separate post for that.

{
  lib,
  buildPythonPackage,
  setuptools,
  wheel,
  numpy,
  mdsplus,
}:
buildPythonPackage  rec {
  pname = "mdsplus";
  version = "7-142-81";
  format = "pyproject";

  src = mdsplus;

  # buildInputs = [ mdsplus ];

  nativeBuildInputs = [ mdsplus ];

  dependencies = [
    numpy
  ];

  sourceRoot = "${src.name}/python/MDSplus/";

  pythonImportsCheck = [
    "MDSplus"
  ];

  build-system = [setuptools wheel];
  meta = {
    description = "The MDSplus data management system";
    homepage = "https://github.com/MDSplus/mdsplus/tree/alpha/python/MDSplus";
    # license = lib.licenses.unfree; # FIXME: nix-init did not find a license
    maintainers = with lib.maintainers; [ ];
    mainProgram = "mdsplus";
    platforms = lib.platforms.all;
  };
}