Cross compiling rust to ARM

Help
1

Hi,
I’m trying to cross compile a simple rust hello-world to ARM (specifically armv7 for a raspberry-pi 2).
I have a setup here: GitHub - backuitist/wake-on-lan-rust-armv7: Experimenting cross-compiliation with Nix

Trying to build this with
nix build --arg crossSystem ' { config = "armv7l-unknown-linux-gnueabihf"; }'
would result in a ton of dependencies (including git and perl) to be built for ARM.

I looked a bit at my dependencies:

$ nix-instantiate default.nix --arg crossSystem ' { config = "armv7l-unknown-linux-gnueabihf"; }'
/nix/store/w5nb5hymc5jda35fjxpklzhd5mf1jhfn-raspberry-hello-world-armv7l-unknown-linux-gnueabihf.drv
$ nix-store -q --references /nix/store/w5nb5hymc5jda35fjxpklzhd5mf1jhfn-raspberry-hello-world-armv7l-unknown-linux-gnueabihf.drv
/nix/store/9krlzvny65gdc8s7kpb6lkx8cd02c25b-default-builder.sh
/nix/store/02ws9mxmqizbw50027xq7dqq5cskjms5-bash-4.4-p23.drv
/nix/store/sjd41awxa0h0xh9pbrlnqr9a775zmmgm-stdenv-linux.drv
/nix/store/cg3rvy8xn44m8cgpmkp8mv6vgfsnhh14-nss-cacert-3.38-armv7l-unknown-linux-gnueabihf.drv
/nix/store/nk6b2ckznjic5wj8ddw0wgdrn4mbz3lg-patch-registry-deps
/nix/store/vysvbmbynbmzyfk4s85612a78xi4q673-git-2.18.0-armv7l-unknown-linux-gnueabihf.drv
/nix/store/z5zicpii1i7ncbig7rxa9vv0sacpyq75-rustc-1.27.0-armv7l-unknown-linux-gnueabihf.drv
/nix/store/n09y5ww85b4gfz5n2sdsla1q6yxs1mis-cargo-1.27.0-armv7l-unknown-linux-gnueabihf.drv
/nix/store/q2hizgmxb681zhk4nql04qi8dwk6lk6d-hello-world
/nix/store/812y7jibpb8hbpvbcf45gmhzr8yw4wip-raspberry-hello-world-vendor.drv

As you can see Nix tries to build git for armv7? Would it be possible to use git for the host platform as I don’t think git for ARM would be of any use?

Another thing is that amongst the requisites is perl. It seems that git brings this to the party. I’ve briefly looked into the git derivation and it seems that perl could be removed from git’s dependencies (although I don’t understand how this option could affect the derivation…): nixpkgs/default.nix at master · NixOS/nixpkgs · GitHub

Trying to build this would anyway fail as down the line, nix would try to run the perl-for-arm on the host machine, resulting in an execution failure.

> nix build --arg crossSystem ' { config = "armv7l-unknown-linux-gnueabihf"; }'
builder for '/nix/store/0f9bczppi3ch6h3075gg05fy4qfkgb4c-perl-TermReadKey-2.37-armv7l-unknown-linux-gnueabihf.drv' failed with exit code 2; last 10 log lines:
  armv7l-unknown-linux-gnueabihf-gcc -c   -D_LARGEFILE_SOURCE -D_FILE_OFFSET_BITS=64 -fwrapv -fno-strict-aliasing   -DVERSION=\"2.37\" -DXS_VERSION=\"2.37\" -fPIC -Wno-unused-function "-I/nix/store/v11ji6vm0sxdl7rn7sq56ql5kjxdqgk6-perl-5.24.4-armv7l-unknown-linux-gnueabihf/lib/perl5/5.24.4/armv7l-linux/CORE"   ReadKey.c
  rm -f blib/arch/auto/Term/ReadKey/ReadKey.so
  armv7l-unknown-linux-gnueabihf-gcc  -shared ReadKey.o  -o blib/arch/auto/Term/ReadKey/ReadKey.so 	\
       	\
    
  chmod 755 blib/arch/auto/Term/ReadKey/ReadKey.so
  "/nix/store/30jw9gz12n7pkzqab1w9pyi1mi4nnlsk-perl-5.24.4-armv7l-unknown-linux-gnueabihf-dev/bin/perl" -MExtUtils::Command::MM -e 'cp_nonempty' -- ReadKey.bs blib/arch/auto/Term/ReadKey/ReadKey.bs 644
  "/nix/store/v11ji6vm0sxdl7rn7sq56ql5kjxdqgk6-perl-5.24.4-armv7l-unknown-linux-gnueabihf/bin/perl" "-Iblib/arch" "-Iblib/lib" ReadKey_pm.PL ReadKey.pm
  /nix/store/56nrxy58wbhvs2sy3rir1jqa68p0kkm5-bash-4.4-p23/bin/bash: /nix/store/v11ji6vm0sxdl7rn7sq56ql5kjxdqgk6-perl-5.24.4-armv7l-unknown-linux-gnueabihf/bin/perl: cannot execute binary file: Exec format error
  make: *** [Makefile:532: ReadKey.pm] Error 126
cannot build derivation '/nix/store/vysvbmbynbmzyfk4s85612a78xi4q673-git-2.18.0-armv7l-unknown-linux-gnueabihf.drv': 1 dependencies couldn't be built
cannot build derivation '/nix/store/w5nb5hymc5jda35fjxpklzhd5mf1jhfn-raspberry-hello-world-armv7l-unknown-linux-gnueabihf.drv': 1 dependencies couldn't be built
[0 built (1 failed)]

Not surprisingly:

file /nix/store/v11ji6vm0sxdl7rn7sq56ql5kjxdqgk6-perl-5.24.4-armv7l-unknown-linux-gnueabihf/bin/perl
/nix/store/v11ji6vm0sxdl7rn7sq56ql5kjxdqgk6-perl-5.24.4-armv7l-unknown-linux-gnueabihf/bin/perl: ELF 32-bit LSB executable, ARM, EABI5 version 1 (SYSV), dynamically linked, interpreter /nix/store/h0iynmqx9mq8dvghi28xp29p06659y69-glibc-2.27-armv7l-unknown-linux-gnueabihf/lib/ld-linux-armhf.so.3, for GNU/Linux 2.6.32, not stripped

Any suggestion?
Thanks,
Bruno

1 Like
2

It’s not ideal, but I’ve found that for any non-trivial system it is easier to do native compiling than cross-compiling and solving all the issues that come up.

3

I was just setting this up myself, but gave the rust-overlay a try. It seems to work out pretty well! Here is my shell.nix:

with (import <nixpkgs> {});
let arm = import <nixpkgs> { crossSystem = lib.systems.examples.raspberryPi; };
in
stdenv.mkDerivation {
  name = "hi-there";
  buildInputs = [
    (rustChannelOfTargets "stable" null
                          [ "x86_64-unknown-linux-gnu"
                            "arm-unknown-linux-gnueabihf" ])
    arm.stdenv.cc
  ];
};

and then here is my .cargo/config:

[target.arm-unknown-linux-gnueabihf]
linker = "armv6l-unknown-linux-gnueabihf-gcc"
rustflags = ["-C", "link-arg=-Wl,-dynamic-linker,/lib/ld-linux-armhf.so.3"]

That linker flag was a nuisance to sort out. The executables I built didn’t run on the target Raspberry Pi because the dynamic linker was set incorrectly. First, I fixed that with patchelf, but then I was trying to reduce the file size (which was over 4MB for a hello world) with strip, and strip was erroring out. This turns out to be a known problem with patchelf, but using the linker flags directly gets the job done and lets you use strip on the resulting executable (a 90% file size saving!).

4

Thanks! That’s great!
I’ve updated my repo: wake-on-lan-rust-armv7/shell.nix at master · backuitist/wake-on-lan-rust-armv7 · GitHub
Strip is working as well :slight_smile:

5

Rust cross compilation is currently broken. I’ve opened a PR to fix some of the obvious things, but it definitely needs some more work:

The patch shebangs thing is an issue still and effects lots of things. It was partially fixed in patch-shebangs: respect cross compilation by matthewbauer · Pull Request #43833 · NixOS/nixpkgs · GitHub but I think it ended up breaking other things and now has been reverted. Hopefully we can retry it soon though.