Build platform CPython headers instead of Host platform headers used whenever a Python extension is cross-compiled

Help
1

Hello,

I am wondering if anyone had some idea on how to fix this cross-compilation issue, I am using httptools as a test package here, but the issue affects other Python packages that include extensions.

% nix-build https://github.com/NixOS/nixpkgs/archive/nixos-22.05.tar.gz -A pkgsCross.raspberryPi.python3Packages.httptools
this derivation will be built:
  /nix/store/fhszw0s2s48h53bkxqikfv3sxpdfndsh-python3.9-httptools-0.4.0-armv6l-unknown-linux-gnueabihf.drv
building '/nix/store/fhszw0s2s48h53bkxqikfv3sxpdfndsh-python3.9-httptools-0.4.0-armv6l-unknown-linux-gnueabihf.drv'...
Sourcing python-remove-tests-dir-hook
Sourcing python-remove-bin-bytecode-hook.sh
Sourcing setuptools-build-hook
Using setuptoolsBuildPhase
Using setuptoolsShellHook
Sourcing pip-install-hook
Using pipInstallPhase
Sourcing python-namespaces-hook
unpacking sources
unpacking source archive /nix/store/bbwsy3sgzqm25wllgvgi059pzaqjhxlj-httptools-0.4.0.tar.gz
source root is httptools-0.4.0
setting SOURCE_DATE_EPOCH to timestamp 1645555402 of file httptools-0.4.0/setup.cfg
patching sources
updateAutotoolsGnuConfigScriptsPhase
configuring
no configure script, doing nothing
building
Executing setuptoolsBuildPhase
running bdist_wheel
running build
running build_py
creating build
creating build/lib.linux-armv6l-3.9
creating build/lib.linux-armv6l-3.9/httptools
copying httptools/_version.py -> build/lib.linux-armv6l-3.9/httptools
copying httptools/__init__.py -> build/lib.linux-armv6l-3.9/httptools
creating build/lib.linux-armv6l-3.9/httptools/parser
copying httptools/parser/errors.py -> build/lib.linux-armv6l-3.9/httptools/parser
copying httptools/parser/__init__.py -> build/lib.linux-armv6l-3.9/httptools/parser
running egg_info
writing httptools.egg-info/PKG-INFO
writing dependency_links to httptools.egg-info/dependency_links.txt
writing requirements to httptools.egg-info/requires.txt
writing top-level names to httptools.egg-info/top_level.txt
reading manifest file 'httptools.egg-info/SOURCES.txt'
reading manifest template 'MANIFEST.in'
adding license file 'LICENSE'
writing manifest file 'httptools.egg-info/SOURCES.txt'
copying httptools/parser/parser.c -> build/lib.linux-armv6l-3.9/httptools/parser
copying httptools/parser/url_parser.c -> build/lib.linux-armv6l-3.9/httptools/parser
running build_ext
building 'httptools.parser.parser' extension
creating build/temp.linux-armv6l-3.9
creating build/temp.linux-armv6l-3.9/httptools
creating build/temp.linux-armv6l-3.9/httptools/parser
creating build/temp.linux-armv6l-3.9/vendor
creating build/temp.linux-armv6l-3.9/vendor/llhttp
creating build/temp.linux-armv6l-3.9/vendor/llhttp/src
armv6l-unknown-linux-gnueabihf-gcc -Wno-unused-result -Wsign-compare -DNDEBUG -g -fwrapv -O3 -Wall -fPIC -Ivendor/llhttp/include -Ivendor/llhttp/src -I/nix/store/7kx74vn1sna0y4158g45sbikdzslyp34-python3-3.9.13/include/python3.9 -c httptools/parser/parser.c -o build/temp.linux-armv6l-3.9/httptools/parser/parser.o -O2
In file included from /nix/store/7kx74vn1sna0y4158g45sbikdzslyp34-python3-3.9.13/include/python3.9/Python.h:50,
                 from httptools/parser/parser.c:22:
/nix/store/7kx74vn1sna0y4158g45sbikdzslyp34-python3-3.9.13/include/python3.9/pyport.h:741:2: error: #error "LONG_BIT definition appears wrong for platform (bad gcc/glibc config?)."
  741 | #error "LONG_BIT definition appears wrong for platform (bad gcc/glibc config?)."
      |  ^~~~~
httptools/parser/parser.c:227:41: warning: division by zero [8;;https://gcc.gnu.org/onlinedocs/gcc/Warning-Options.html#index-Wdiv-by-zero-Wdiv-by-zero8;;]
  227 |     enum { __pyx_check_sizeof_voidp = 1 / (int)(SIZEOF_VOID_P == sizeof(void*)) };
      |                                         ^
httptools/parser/parser.c:227:12: error: enumerator value for ‘__pyx_check_sizeof_voidp’ is not an integer constant
  227 |     enum { __pyx_check_sizeof_voidp = 1 / (int)(SIZEOF_VOID_P == sizeof(void*)) };
      |            ^~~~~~~~~~~~~~~~~~~~~~~~
httptools/parser/parser.c: In function ‘__pyx_pf_9httptools_6parser_6parser_10HttpParser_10feed_data’:
httptools/parser/parser.c:3452:23: warning: assignment discards ‘const’ qualifier from pointer target type [8;;https://gcc.gnu.org/onlinedocs/gcc/Warning-Options.html#index-Wdiscarded-qualifiers-Wdiscarded-qualifiers8;;]
 3452 |       __pyx_v_err_pos = llhttp_get_error_pos(__pyx_v_self->_cparser);
      |                       ^
error: command '/nix/store/sabnd3wv05clkmhlb5kv3hmcyyjzdwwv-armv6l-unknown-linux-gnueabihf-stage-final-gcc-debug-wrapper-10.3.0/bin/armv6l-unknown-linux-gnueabihf-gcc' failed with exit code 1
error: builder for '/nix/store/fhszw0s2s48h53bkxqikfv3sxpdfndsh-python3.9-httptools-0.4.0-armv6l-unknown-linux-gnueabihf.drv' failed with exit code 1;
       last 10 log lines:
       >   227 |     enum { __pyx_check_sizeof_voidp = 1 / (int)(SIZEOF_VOID_P == sizeof(void*)) };
       >       |                                         ^
       > httptools/parser/parser.c:227:12: error: enumerator value for ‘__pyx_check_sizeof_voidp’ is not an integer constant
       >   227 |     enum { __pyx_check_sizeof_voidp = 1 / (int)(SIZEOF_VOID_P == sizeof(void*)) };
       >       |            ^~~~~~~~~~~~~~~~~~~~~~~~
       > httptools/parser/parser.c: In function ‘__pyx_pf_9httptools_6parser_6parser_10HttpParser_10feed_data’:
       > httptools/parser/parser.c:3452:23: warning: assignment discards ‘const’ qualifier from pointer target type [8;;https://gcc.gnu.org/onlinedocs/gcc/Warning-Options.html#index-Wdiscarded-qualifiers-Wdiscarded-qualifiers8;;]
       >  3452 |       __pyx_v_err_pos = llhttp_get_error_pos(__pyx_v_self->_cparser);
       >       |                       ^
       > error: command '/nix/store/sabnd3wv05clkmhlb5kv3hmcyyjzdwwv-armv6l-unknown-linux-gnueabihf-stage-final-gcc-debug-wrapper-10.3.0/bin/armv6l-unknown-linux-gnueabihf-gcc' failed with exit code 1
       For full logs, run 'nix log /nix/store/fhszw0s2s48h53bkxqikfv3sxpdfndsh-python3.9-httptools-0.4.0-armv6l-unknown-linux-gnueabihf.drv'.
%

This fails because somehow setuptools does not generate an appropriate gcc command. It uses -I/nix/store/7kx74vn1sna0y4158g45sbikdzslyp34-python3-3.9.13/include/python3.9 which includes the CPython headers for the Python interpreter for the build platform, when the headers for the host platform should be used (path would be something like -I/nix/store/XXXXX-python3-armv6l-unknown-linux-gnueabihf-3.9.13/include/python3.9).

It is the same issue as the one described in https://github.com/NixOS/nixpkgs/issues/166053.

Interestingly, this was not an issue in 21.11, nix-build https://github.com/NixOS/nixpkgs/archive/nixos-21.11.tar.gz -A pkgsCross.raspberryPi.python3Packages.httptools builds fine, so I’ll see if some bisect can reveal something useful, but I was wondering if anyone else had some pointers.

Thank you

2

Alright, so I am done with that bisect, and we do have some clue:

% git bisect good                                               
5c09870c0244bbcf47a84f379e05bf10c7aa3f0d is the first bad commit
commit 5c09870c0244bbcf47a84f379e05bf10c7aa3f0d
Author: Martin Weinelt <hexa@darmstadt.ccc.de>
Date:   Tue Feb 15 00:26:26 2022 +0100

    python3Packages.setuptools: 57.2.0 -> 60.8.2

 pkgs/development/python-modules/setuptools/default.nix | 4 ++--
 1 file changed, 2 insertions(+), 2 deletions(-)
%

I’ll dig in later


3

The setuptools version that introduces the problem is v60.0.0:

Breaking Changes

  • #2896: Setuptools once again makes its local copy of distutils the default. To override, set SETUPTOOLS_USE_DISTUTILS=stdlib.

Hello @ris or @hexa, do you have any context on this change by any chance?

It doesn’t fix the issue, but fwiw, export SETUPTOOLS_USE_DISTUTILS=stdlib in preBuild in the httptools derivation does make it build.

4

These are dusty memories now, but we fought this issue mainly in https://github.com/NixOS/nixpkgs/pull/160067

The big change with setuptools 60 is it went (back) to including its own bundled distutils. The issue this can introduce is that any special patches/hacks that were made to cpython’s stdlib distutils are, of course, missing. So we needed to port our c++ support-adding patch from the stdlib distutils to the bundled distutils. This is where I would start looking - for “missing fixes”. I guess as you’ve got a “working version” (using SETUPTOOLS_USE_DISTUTILS=stdlib) and a “broken version” you can try and analyze the real differences between the two distutils.

It could, of course, be cause by a completely different change in setuptools: History - setuptools 62.6.0.post20220701 documentation would be useful here.

2 Likes
5

With the previous details in mind, I looked into the setuptools history and I found:

So I tried setuptools v62.6.0, but that didn’t fix the issue. However searching for the author in nixpkgs led me to:

Which fixes the build issue when I apply it over the release-22.05 branch.

So, thank you very much for the patch @lopsided98, my question at this point is: how can we get it merged into the release-22.05 branch?

I am also wondering if setuptools ≄ v62.4.0 was supposed to work out of the box? The changes in distutils#145 look pretty different than the changes in the nixpkgs patch.

6

It was supposed to work; the patch was reworked quite a bit but looks like it should provide the same functionality. That said, upstream didn’t request my feedback on the final version that was merged and I didn’t test it.