I’d like to nix build a manylinux compliant Python wheel including a compiled python extension. Or rather, I’d like to build such a wheel so that it runs on various Linux distributions. (How) is this possible?
Manylinux achieves portability by being old, e.g. old glibc version, old compilers. But this is hard to achieve with nix, isn’t it? The only way, going forward I see currently, are compiling statically linked shared objects based on musl.
The underlying mission here is to have one build system that builds every artifact from a multi-language project including C++ executables, python packages with pybind11 based python extensions and docker images. Exe and docker is easily possible, but I’m pondering about python.
It is possible to do, but some work. First you probably want to make sure that you only build the extension with older versions, building all transitive dependencies is expensive. I’ve also found that various parts of nixpkgs do not build with older gcc/glibc anymore.
The steps are:
Build gcc against your target glibc. This avoids that libgcc/libstdc++ uses symbols from newer glibc versions.
Create a stdenv that uses the new gcc/glibc.
Use the stdenv in your Python extension derivation.
What this is going to look like is going to be very specific to what you are building. E.g., we build Torch extensions compatible with Ubuntu >= 20.04. Steps 1-2 are here:
Step 3:
Step 3 is a bit different for us, because we not building a full Python package, but only the native extension bits. We also statically link libstdc++ for simplicity.
For a real Python package, you have to pass stdenv to buildPythonPackage, like e.g.:
However, like I said above, it’s going to be quite specific to what you are building. E.g. if you are linking in static libraries, you have to ensure that they are built with this stdenv as well. Or in our case, we are also building variants with ROCm and we have to rewrap the ROCm toolchain to ensure it uses the old glibc, etc.
Once you get it running it’s pretty nice . Otherwise, it can be easier to build it in a manylinux Docker container.
The question isn’t very specific about what output artifact is actually needed, let alone the response comment.
i.e., it’s one multi-language project, which just calls some python through the python C bindings. Likely the output artifact only needs to have those wheels available in a python env - it doesn’t sound like the idea is to get python libraries built that can be depended on by python in a dev env on another distro. More evidence comes in the next sentence, the other targets are:
… so it sounds like “natively” (read: without docker) targeting Linux with an exe-like format would meet the requirements, without having to do anything silly.
The wheels are probably a red herring from @clamydothinking they need to build wheels to accomplish what they want. In other words, I was responding to the XY problem that I think underpins this. I should have verbalized that, but hey.
I quite like your answer, though, it’s also a cool thing to figure out, and nice that expertise with this exists in the ecosystem
. Otherwise, it can be easier to build it in a manylinux Docker container.