Fixing ifcopenshell, libxml2 not found by linker at runtime

autra · May 21, 2024, 6:32am

I’m currently trying to fix ifcopenshell on nixpkgs.

I’ve made good progress and now with the change I made in the MR linked above, the library compiles just fine.

However, I have an error at runtime:

❯ ./result/bin/IfcConvert  --version
./result/bin/IfcConvert: error while loading shared libraries: libxml2.so.2: cannot open shared object file: No such file or directory

And sure enough, ldd says libxml2.so.2 => not found

It’s odd, because libxml2 is in my buildInputs and I had the same trouble with opencascade, and adding it to buildInputs fixed it.

Is there anything special with libxml2? What is going on here?

One intuition I have is that maybe I should separate the c++ package from the python part, but I’m not even sure it is relevant to this issue.

Thanks!

autra · May 24, 2024, 6:49am

I found a partial solution: this package was using buildPythonPackage, but it needed to use stdenv.mkDerivation. Using the latter fixes things. I think it is because it patches the elf, contrary to the former.

But now, I need to figure out how to create the python package from the build result (the build process creates a .so and a python folder containing the python bindings that uses this .so).

autra · July 26, 2024, 6:26pm

The real (?) solution was this:

github.com/NixOS/nixpkgs

ifcopenshell: 240611 -> 0.7.10, fix build and activate most tests

NixOS:master ← autra:fix_ifcopenshell_build

opened 07:21AM - 17 May 24 UTC

autra

+132 -15

## Description of changes Fix the ifcopenshell build. Fix #312380 …## Things done - Built on platform(s) - [x] x86_64-linux - [ ] aarch64-linux - [ ] x86_64-darwin - [ ] aarch64-darwin - For non-Linux: Is sandboxing enabled in `nix.conf`? (See [Nix manual](https://nixos.org/manual/nix/stable/command-ref/conf-file.html)) - [ ] `sandbox = relaxed` - [ ] `sandbox = true` - [ ] Tested, as applicable: - [NixOS test(s)](https://nixos.org/manual/nixos/unstable/index.html#sec-nixos-tests) (look inside [nixos/tests](https://github.com/NixOS/nixpkgs/blob/master/nixos/tests)) - and/or [package tests](https://github.com/NixOS/nixpkgs/blob/master/pkgs/README.md#package-tests) - or, for functions and "core" functionality, tests in [lib/tests](https://github.com/NixOS/nixpkgs/blob/master/lib/tests) or [pkgs/test](https://github.com/NixOS/nixpkgs/blob/master/pkgs/test) - made sure NixOS tests are [linked](https://nixos.org/manual/nixpkgs/unstable/#ssec-nixos-tests-linking) to the relevant packages - [ ] Tested compilation of all packages that depend on this change using `nix-shell -p nixpkgs-review --run "nixpkgs-review rev HEAD"`. Note: all changes have to be committed, also see [nixpkgs-review usage](https://github.com/Mic92/nixpkgs-review#usage) - [ ] Tested basic functionality of all binary files (usually in `./result/bin/`) - [24.05 Release Notes](https://github.com/NixOS/nixpkgs/blob/master/nixos/doc/manual/release-notes/rl-2405.section.md) (or backporting [23.05](https://github.com/NixOS/nixpkgs/blob/master/nixos/doc/manual/release-notes/rl-2305.section.md) and [23.11](https://github.com/NixOS/nixpkgs/blob/master/nixos/doc/manual/release-notes/rl-2311.section.md) Release notes) - [ ] (Package updates) Added a release notes entry if the change is major or breaking - [ ] (Module updates) Added a release notes entry if the change is significant - [ ] (Module addition) Added a release notes entry if adding a new NixOS module - [x] Fits [CONTRIBUTING.md](https://github.com/NixOS/nixpkgs/blob/master/CONTRIBUTING.md).  --- Add a :+1: [reaction] to [pull requests you find important]. [reaction]: https://github.blog/2016-03-10-add-reactions-to-pull-requests-issues-and-comments/ [pull requests you find important]: https://github.com/NixOS/nixpkgs/pulls?q=is%3Aopen+sort%3Areactions-%2B1-desc

famadorian · September 5, 2024, 8:05pm

Sorry for intruding here, but I am not sure if my issue is related to your issue. I enabled ifcopenshell in my config and I get this:

building '/nix/store/d1frarn98pn9s9pm0s2j7bpmj9qbhzbl-python3.11-ifcopenshell-240306.drv'...
Sourcing python-remove-tests-dir-hook
Sourcing python-catch-conflicts-hook.sh
Sourcing python-remove-bin-bytecode-hook.sh
Sourcing python-imports-check-hook.sh
Using pythonImportsCheckPhase
Sourcing python-namespaces-hook
Running phase: unpackPhase
unpacking source archive /nix/store/ns0ciy9a0lllgzwm5kl4bic6p5s7bvh2-source
source root is source
setting SOURCE_DATE_EPOCH to timestamp 315619200 of file source/win/vs-cfg.cmd
Running phase: patchPhase
Running phase: updateAutotoolsGnuConfigScriptsPhase
Running phase: configurePhase
fixing cmake files...
cmake flags: -DCMAKE_FIND_USE_SYSTEM_PACKAGE_REGISTRY=OFF -DCMAKE_FIND_USE_PACKAGE_REGISTRY=OFF -DCMAKE_EXPORT_NO_PACKAGE_REGISTRY=ON -DCMAKE_BUILD_TYPE=Release -DBUILD_TESTING=OFF -DCMAKE_INSTALL_LOCALEDIR=/nix/store/iwwkkx3qq3b2rr3mi9ab7yl4xsvm32n4-python3.11-ifcopenshell-240306/share/locale -DCMAKE_INSTALL_LIBEXECDIR=/nix/store/iwwkkx3qq3b2rr3mi9ab7yl4xsvm32n4-python3.11-ifcopenshell-240306/libexec -DCMAKE_INSTALL_LIBDIR=/nix/store/iwwkkx3qq3b2rr3mi9ab7yl4xsvm32n4-python3.11-ifcopenshell-240306/lib -DCMAKE_INSTALL_DOCDIR=/nix/store/iwwkkx3qq3b2rr3mi9ab7yl4xsvm32n4-python3.11-ifcopenshell-240306/share/doc/IfcOpenShell -DCMAKE_INSTALL_INFODIR=/nix/store/iwwkkx3qq3b2rr3mi9ab7yl4xsvm32n4-python3.11-ifcopenshell-240306/share/info -DCMAKE_INSTALL_MANDIR=/nix/store/iwwkkx3qq3b2rr3mi9ab7yl4xsvm32n4-python3.11-ifcopenshell-240306/share/man -DCMAKE_INSTALL_OLDINCLUDEDIR=/nix/store/iwwkkx3qq3b2rr3mi9ab7yl4xsvm32n4-python3.11-ifcopenshell-240306/include -DCMAKE_INSTALL_INCLUDEDIR=/nix/store/iwwkkx3qq3b2rr3mi9ab7yl4xsvm32n4-python3.11-ifcopenshell-240306/include -DCMAKE_INSTALL_SBINDIR=/nix/store/iwwkkx3qq3b2rr3mi9ab7yl4xsvm32n4-python3.11-ifcopenshell-240306/sbin -DCMAKE_INSTALL_BINDIR=/nix/store/iwwkkx3qq3b2rr3mi9ab7yl4xsvm32n4-python3.11-ifcopenshell-240306/bin -DCMAKE_INSTALL_NAME_DIR=/nix/store/iwwkkx3qq3b2rr3mi9ab7yl4xsvm32n4-python3.11-ifcopenshell-240306/lib -DCMAKE_POLICY_DEFAULT_CMP0025=NEW -DCMAKE_OSX_SYSROOT= -DCMAKE_FIND_FRAMEWORK=LAST -DCMAKE_STRIP=/nix/store/p1v5jnywcsm0fhffdifqkm55smcdsi1r-gcc-wrapper-10.5.0/bin/strip -DCMAKE_RANLIB=/nix/store/p1v5jnywcsm0fhffdifqkm55smcdsi1r-gcc-wrapper-10.5.0/bin/ranlib -DCMAKE_AR=/nix/store/p1v5jnywcsm0fhffdifqkm55smcdsi1r-gcc-wrapper-10.5.0/bin/ar -DCMAKE_C_COMPILER=gcc -DCMAKE_CXX_COMPILER=g++ -DCMAKE_INSTALL_PREFIX=/nix/store/iwwkkx3qq3b2rr3mi9ab7yl4xsvm32n4-python3.11-ifcopenshell-240306 -DUSERSPACE_PYTHON_PREFIX=ON -DOCC_INCLUDE_DIR=/nix/store/5kpc6zgvv3ngf393362g33chr1yd8j1p-opencascade-occt-7.6.2/include/opencascade -DOCC_LIBRARY_DIR=/nix/store/5kpc6zgvv3ngf393362g33chr1yd8j1p-opencascade-occt-7.6.2/lib -DOPENCOLLADA_INCLUDE_DIR=/nix/store/2cr55i4d3aym66wwpg3gf49as439znx5-opencollada-1.6.68/include/opencollada -DOPENCOLLADA_LIBRARY_DIR=/nix/store/2cr55i4d3aym66wwpg3gf49as439znx5-opencollada-1.6.68/lib/opencollada -DSWIG_EXECUTABLE=/nix/store/anr83k2ag19jm3dw7g7w9csfkd21977h-swig-3.0.12/bin/swig -DLIBXML2_INCLUDE_DIR=/nix/store/xlwfaizd8lm91yimfsx3r8k40b81vnhy-libxml2-2.12.7-dev/include/libxml2 -DLIBXML2_LIBRARIES=/nix/store/4ns3cjaf3vvkwp08g007hjqllfajnq23-libxml2-2.12.7/lib/libxml2.so 
-- The C compiler identification is GNU 10.5.0
-- The CXX compiler identification is GNU 10.5.0
-- Detecting C compiler ABI info
-- Detecting C compiler ABI info - done
-- Check for working C compiler: /nix/store/p1v5jnywcsm0fhffdifqkm55smcdsi1r-gcc-wrapper-10.5.0/bin/gcc - skipped
-- Detecting C compile features
-- Detecting C compile features - done
-- Detecting CXX compiler ABI info
-- Detecting CXX compiler ABI info - done
-- Check for working CXX compiler: /nix/store/p1v5jnywcsm0fhffdifqkm55smcdsi1r-gcc-wrapper-10.5.0/bin/g++ - skipped
-- Detecting CXX compile features
-- Detecting CXX compile features - done
CMake Deprecation Warning at CMakeLists.txt:28 (cmake_policy):
  The OLD behavior for policy CMP0078 will be removed from a future version
  of CMake.

  The cmake-policies(7) manual explains that the OLD behaviors of all
  policies are deprecated and that a policy should be set to OLD only under
  specific short-term circumstances.  Projects should be ported to the NEW
  behavior and not rely on setting a policy to OLD.


-- BINDIR: /nix/store/iwwkkx3qq3b2rr3mi9ab7yl4xsvm32n4-python3.11-ifcopenshell-240306/bin
-- INCLUDEDIR: /nix/store/iwwkkx3qq3b2rr3mi9ab7yl4xsvm32n4-python3.11-ifcopenshell-240306/include
-- LIBDIR: /nix/store/iwwkkx3qq3b2rr3mi9ab7yl4xsvm32n4-python3.11-ifcopenshell-240306/lib
-- Found Boost: /nix/store/z5xgz3gzpf7calynyvby44jhrzvmcbq5-boost-1.79.0-dev/lib/cmake/Boost-1.79.0/BoostConfig.cmake (found version "1.79.0") found components: system program_options regex thread date_time
-- Boost include files found in /nix/store/z5xgz3gzpf7calynyvby44jhrzvmcbq5-boost-1.79.0-dev/include
-- Boost libraries found in /nix/store/1f995fsvvj2x3vwd6j7mv9dr49zi2kwi-boost-1.79.0/lib
-- Found LibXml2: /nix/store/4ns3cjaf3vvkwp08g007hjqllfajnq23-libxml2-2.12.7/lib/libxml2.so (found version "2.12.7")
-- Looking for Open CASCADE include files in: /nix/store/5kpc6zgvv3ngf393362g33chr1yd8j1p-opencascade-occt-7.6.2/include/opencascade
-- Looking for Open CASCADE library files in: /nix/store/5kpc6zgvv3ngf393362g33chr1yd8j1p-opencascade-occt-7.6.2/lib
-- Required Open Cascade Library files found
-- OpenCOLLADA library files found
-- OpenCOLLADA header files found
-- No HDF5 include directory specified
-- No HDF5 library directory specified
-- IFC SCHEMA_VERSIONS that will be used for the build: 2x3;4;4x1;4x2;4x3_rc1;4x3_rc2;4x3_rc3;4x3_rc4;4x3;4x3_tc1;4x3_add1;4x3_add2.
-- Found Threads: TRUE
-- Found SWIG: /nix/store/anr83k2ag19jm3dw7g7w9csfkd21977h-swig-3.0.12/bin/swig (found version "3.0.12")
CMake Warning (dev) at /build/source/src/ifcwrap/CMakeLists.txt:38 (FIND_PACKAGE):
  Policy CMP0148 is not set: The FindPythonInterp and FindPythonLibs modules
  are removed.  Run "cmake --help-policy CMP0148" for policy details.  Use
  the cmake_policy command to set the policy and suppress this warning.

This warning is for project developers.  Use -Wno-dev to suppress it.

-- Found PythonLibs: /nix/store/4rf5qybw37b4lh1g0xczlv14sqdbmnpm-python3-3.11.9/lib/libpython3.11.so (found version "3.11.9")
CMake Warning (dev) at /build/source/src/ifcwrap/CMakeLists.txt:69 (FIND_PACKAGE):
  Policy CMP0148 is not set: The FindPythonInterp and FindPythonLibs modules
  are removed.  Run "cmake --help-policy CMP0148" for policy details.  Use
  the cmake_policy command to set the policy and suppress this warning.

This warning is for project developers.  Use -Wno-dev to suppress it.

-- Found PythonInterp: /nix/store/4rf5qybw37b4lh1g0xczlv14sqdbmnpm-python3-3.11.9/bin/python3.11 (found version "3.11.9")
-- Looking for CGAL include files in: /usr/include
-- Use CGAL_INCLUDE_DIR to specify another directory
CMake Error at /build/source/src/svgfill/CMakeLists.txt:58 (message):
  Unable to find GMP library files, aborting


-- Configuring incomplete, errors occurred!
error: builder for '/nix/store/d1frarn98pn9s9pm0s2j7bpmj9qbhzbl-python3.11-ifcopenshell-240306.drv' failed with exit code 1
error: 1 dependencies of derivation '/nix/store/3vrwjg6d1v372d9vz2vqznhd6fnbq4ma-system-path.drv' failed to build
error: 1 dependencies of derivation '/nix/store/70crhw6br5zrcimzi2p5s2h7wgcd9g1b-nixos-system-quasar-24.05.2310.7dca15289a1c.drv' failed to build

autra · September 5, 2024, 9:36pm

Yes, ifcopenshell build is currently broken on nixpkgs. I just need to rebase ifcopenshell: 240611 -> 0.7.10, fix build and activate most tests by autra · Pull Request #312381 · NixOS/nixpkgs · GitHub then merge it I’ll try to do quickly!

autra · September 6, 2024, 8:19am

then merge it

I mean: get it merged, because I don’t have that power!