Cross compile for Raspberry PI

Help
1

Hi,

I’m looking for help with cross compiling for arm (target is raspberry pi, host is linux x86). essentially running into an issue where the ARM linker I built fails to even run. here’s a simplified shell I have:

let
  nixpkgs = import <nixpkgs> { crossSystem = { config = "armv7l-unknown-linux-gnueabihf"; }; };
in
  with nixpkgs;
  stdenv.mkDerivation {
    name = "raspberry-shell";
    buildInputs = [
      stdenv.cc
    ];
  }

That produces armv7l-unknown-linux-gnueabihf-ld binary that when run gives this error:

.../bin/armv7l-unknown-linux-gnueabihf-ld: undefined symbol: elf32_arm_fix_exidx_coverage

Any pointers would be useful, I’m new to cross-compiling and building on ARM in general.

Thanks
Mateusz

2

I ended up just modifying the LD_LIBRARY_PATH by hand:


  arm = import <nixpkgs> { crossSystem.config = "armv7l-unknown-linux-musleabihf"; };
  nixpkgs = import <nixpkgs> { overlays = [ ... ]; };

in
  with nixpkgs;
  stdenv.mkDerivation {
    name = "arm-shell";
    buildInputs = [
      arm.stdenv.cc
      ...
    ];
    shellHook =
    ''
      export LD_LIBRARY_PATH=${arm.binutils-unwrapped}/lib
    '';
  }

I am new to cross compilation but the fact I’m having to go through so many hooks seems fishy at best.

3

I’ve simplified your example to:

{
  pkgs ? import <nixpkgs> {
    crossSystem = { config = "armv7l-unknown-linux-gnueabihf"; };
  }
}:

pkgs.mkShell { }

A nix-shell already comes with the desired stdenv, no need to add any of its components.

$LD runs when I execute it. What files did you run this on? Could you provide a minimal reproducer?

mkDerivation/mkShell put nearly all of their args as environment variables into the build/shell. You don’t need to add shellHooks to set them. Better set them declaratively:

pkgs.mkShell {
  LD_LIBRARY_PATH = "${pkgs.binutils-unwrapped}/lib";
}
4

Thanks for simplifying my examples. However, I get the same error when trying your code as I did before. I ran the first example verbatim and got the same missing symbol error:

> cat ~/shells/raspberry-pi.nix
{
  pkgs ? import <nixpkgs> {
    crossSystem = { config = "armv7l-unknown-linux-gnueabihf"; };
  }
}:

pkgs.mkShell { }

> nix-shell ~/shells/raspberry-pi.nix --command fish
Welcome to fish, the friendly interactive shell
Type help for instructions on how to use fish
> echo $LD
armv7l-unknown-linux-gnueabihf-ld
> $LD
/nix/store/nx12g0hvhajddcz93zzh4m4lxf1pzsri-armv7l-unknown-linux-gnueabihf-binutils-2.38/bin/armv7l-unknown-linux-gnueabihf-ld: symbol lookup error: /nix/store/nx12g0hvhajddcz93zzh4m4lxf1pzsri-armv7l-unknown-linux-gnueabihf-binutils-2.38/bin/armv7l-unknown-linux-gnueabihf-ld: undefined symbol: elf32_arm_fix_exidx_coverage

I’m on nixos-22.05.

I was also expecting nix shell to handle library linkage correctly.

5 
[atemu@HEPHAISTOS ~]$ cat /tmp/repro.nix
{
  pkgs ? import <nixpkgs> {
    crossSystem = { config = "armv7l-unknown-linux-gnueabihf"; };
  }
}:

pkgs.mkShell { }
[atemu@HEPHAISTOS ~]$ nix-shell /tmp/repro.nix -I nixpkgs=Projects/nixpkgs-22.05/ --command fish
Welcome to fish, the friendly interactive shell
Type help for instructions on how to use fish
atemu@HEPHAISTOS ~ (master)> echo $LD
armv7l-unknown-linux-gnueabihf-ld
atemu@HEPHAISTOS ~ (master)> $LD
/nix/store/nx12g0hvhajddcz93zzh4m4lxf1pzsri-armv7l-unknown-linux-gnueabihf-binutils-2.38/bin/armv7l-unknown-linux-gnueabihf-ld: no input files

Could you try running the shell without fish?

What revision is the Nixpkgs in your $NIX_PATH? Mine is Merge pull request #196155 from NixOS/backport-196107-to-release-22.05 · NixOS/nixpkgs@be44bf6 · GitHub.

6

Running with or without the fish command didn’t make a difference. However, I had a thought and figured out what could be interfering with this - I have a nix-env setup where I install this:

      devtoolsEnv = super.lowPrio (super.buildEnv {
        name = "devtoolsEnv";
        paths = with self;
        [
          binutils-unwrapped
          ...
        ];
      });

The binutils-unwrapped here are for x86 and removing this env by running nix-env --remove-all fixed the issue! After that, running the shell (even with --command fish) produces a $LD that works.

I’m not sure if this is a bug. It feels like it? I would expect the nix-shell to take precedence over any nix-env that was setup earlier.

I suppose the moral of the story here is to be extremely careful about using nix-env for things that can bring libs into ones environment. That’s really annoying though because I was adding binutils for the sake of binaries/tools it packages, not the libs.

BTW thanks for your help on this!

7

To ignore the global environment as best as possible, use --pure.

Better just not use nix-env at all. See Depreciate the use of nix-env to install packages?

You can make it only include binaries using buildEnv’s pathsToLink.

Which of its tools do you need installed globally though? If you’re compiling things outside of Nix, use nix-shells.

8

It still seems like a bug, as long nix-env is available and nix-shell is un-pure by default. I am now made aware of this kludge but it’s not something I would have expected!

Ideally this behavior could be fixed w/o having to set --pure which is a big hammer and a nuisance to work with in many ways. It’s kind of nice to have access to basic things like e.g. text editor in the nix shell. Combine it with the fact many projects now provide their own development shells.

I’m thinking of filing a bug for this, I think this is likely to surprise many people.

Thanks for the pointer for pathsToLink. I wonder if the binutils tools will work with that though, I’m assuming something there did need the “libbfd” to start with? I can give it a try sometime later.