I am trying to build some nixosSystem which pulls in Python twisted. I am building on an x86_64-linux system and in the configuration I have nixosSystem { system = "aarch64-linux"; } i.e. the target is an ARM system. I get the following error in the test phase:
error: builder for '/nix/store/mrl4ys33b2dqc1cg31zx3zkgc5argg5s-python3.10-twisted-22.4.0.drv' failed with exit code 1;
last 10 log lines:
> self.assertEqual(self.client.transport.getOutgoingInterface(), "0.0.0.0")
> File "/nix/store/zqpwv39d7jqh2l7p685pfgzqjv5bhllq-python3.10-twisted-22.4.0/lib/python3.10/site-packages/twisted/internet/udp.py", line 449, in getOutgoingInterface
> i = self.socket.getsockopt(socket.IPPROTO_IP, socket.IP_MULTICAST_IF)
> builtins.OSError: [Errno 92] Protocol not available
>
> twisted.test.test_udp.MulticastTests.test_interface
> -------------------------------------------------------------------------------
> Ran 11926 tests in 1185.231s
>
> FAILED (skips=1798, errors=1, successes=10127)
For full logs, run 'nix log /nix/store/mrl4ys33b2dqc1cg31zx3zkgc5argg5s-python3.10-twisted-22.4.0.drv'.
error: 1 dependencies of derivation '/nix/store/9fsvscvmcpfb8240cizd8zxkv3llz6b4-python3-3.10.9-env.drv' failed to build
...
I am building for aarch64-linux on x86_64-linux with the help of boot.binfmt.emulatedSystems = [ "aarch64-linux" ]; in a configuration.nix. I think the build succeeds if performed natively on the target platform. (Frankly I don’t know if I can claim the builds are the same… but I can tell you I find the same store path /nix/store/mrl4ys33b2dqc1cg31zx3zkgc5argg5s-python3.10-twisted-22.4.0.drv on both systems.)
I don’t really know if this is a twisted problem, a Python problem, a qemu problem, a Nix problem, an Arch (host x86 OS) problem, a host OS config problem, or other.
I found that the build makes calls like /usr/bin/qemu-aarch64-static /nix/store/0bdd19v2awzdwz3xr1rxdyc9qm219jhr-python3-3.10.9/bin/python3.10 ..., which I can do myself from a shell and then do things like import socket, maybe import the build of twisted as it enters the test phase to poke around.
Hoping someone with more knowledge in any related area might be able to get an easy assessment of what’s going on. Thanks
EDIT 1
I found that if I do /usr/bin/qemu-aarch64-static /nix/store/0bdd19v2awzdwz3xr1rxdyc9qm219jhr-python3-3.10.9/bin/python3.10 and run the test case from a pip version of twisted, the test fails in thes same way. If I just use my system Python (3.10.9, no qemu) and run the test case from the same twisted, it succeeds. Maybe qemu-related then? Not sure.
EDIT 2
I wrote a little C program with basically the same logic as the failed Python unit test. Basically just
sockfd = socket(AF_INET , SOCK_DGRAM , 0);
bind(sockfd, (struct sockaddr *) &sin, sizeof sin);
getsockopt(sockfd, IPPROTO_IP, IP_MULTICAST_IF, &optval, &optlen);
Couldn’t make it work with qemu-user on a couple of different systems, but it would work in a full qemu vm. I also found a post (albeit from 2017) on the qemu-discuss list saying that IP_MULTICAST_IF wasn’t supported yet.
This is a slightly older version of twisted pinned by a module in my flake. I do see newer versions in Hydra built for aarch64-linux, so I’m confused how those succeded, or what Hydra’s configuration is for aarch64 builds. Maybe Hydra doesn’t use qemu-user. Anyway I have decided to just do without the service that pulls in twisted as a dependency, for now.
EDIT 3
Have since learned that Hydra does its aarch64-linux builds on real aarch64 hardware.