Cross-Compiling Problem with deviceTree on armv6

Help
1

Hi,
I’m currently trying to build an Image for my Raspberry Pi0W (so I’m on armv6l).
My config is based on this config from sorki and looks like this.

Everything works fine so far, except that I need a deviceTree overlay.
Adding the following config causes the build to fail:

  hardware.deviceTree = {
    enable = true;
    overlays = [
      {
        name = "hifiberry-dac";
        dtboFile = "${pkgs.device-tree_rpi.overlays}/hifiberry-dac.dtbo";
      }
    ];
  };

The error I’m getting is this:

[...]
checking whether armv6l-unknown-linux-gnueabihf-gcc accepts -Kpthread... no
checking whether armv6l-unknown-linux-gnueabihf-gcc accepts -Kthread... no
checking whether armv6l-unknown-linux-gnueabihf-gcc accepts -pthread... no
checking whether armv6l-unknown-linux-gnueabihf-g++ also accepts flags for thread support... no
checking for ANSI C header files... (cached) yes
    [CXX] base/output_file_requirements.cc
gcc: error: unrecognized argument in option '-mabi=aapcs-linux'
gcc: note: valid arguments to '-mabi=' are: ms sysv
gcc: error: unrecognized command line option '-mlittle-endian'
gcc: error: unrecognized command line option '-mapcs'
gcc: error: unrecognized command line option '-mno-sched-prolog'
gcc: error: unrecognized command line option '-mfpu=vfp'
make[1]: *** [Kbuild:21: kernel/bounds.s] Error 1
make: *** [Makefile:1102: prepare0] Error 2
  UPD     include/generated/timestamp_autogenerated.h
checking for canonicalize_file_name... uint64_t
builder for '/nix/store/k206l0wilb04fk1z5xjfxicf9hy0lffi-dtbs-with-symbols-armv6l-unknown-linux-gnueabihf.drv' failed with exit code 2
cannot build derivation '/nix/store/jlddxhy1jz9cb21grqlmzn76pw3bpp6k-device-tree-overlays.drv': 1 dependencies couldn't be built
building '/nix/store/bbgsmpmig48d93iymvcpcm26bdlv4nzc-libsepol-2.9-armv6l-unknown-linux-gnueabihf.drv'...
building '/nix/store/yz6b80xibiarav0cihc7bar469skqpcr-libunistring-0.9.10-armv6l-unknown-linux-gnueabihf.drv'...
building '/nix/store/0mndmsq2hhsz1saz8av9jnvdcxb90zsi-python3-3.7.9-armv6l-unknown-linux-gnueabihf.drv'...
building '/nix/store/5wbczfq33gqyrri34bqh76ldj2rabvx8-thin-provisioning-tools-0.9.0.drv'...
cannot build derivation '/nix/store/qs3rpkfr2j2kpz983kmch0pkxzcfh0s5-nixos-system-nixos-21.03pre244556.dfd2eeabd6e.drv': 1 dependencies couldn't be built
cannot build derivation '/nix/store/zkqysjkip74kyrczbg3rzg06bqlvhznf-ext4-fs.img.zst-armv6l-unknown-linux-gnueabihf.drv': 1 dependencies couldn't be built
cannot build derivation '/nix/store/cp7nrhz1g5whxndnn2aa8bscdlkk64v3-nixos-sd-image-21.03pre244556.dfd2eeabd6e-armv6l-linux.img-armv6l-unknown-linux-gnueabihf.drv': 1 dependencies couldn't be built
error: build of '/nix/store/cp7nrhz1g5whxndnn2aa8bscdlkk64v3-nixos-sd-image-21.03pre244556.dfd2eeabd6e-armv6l-linux.img-armv6l-unknown-linux-gnueabihf.drv' failed

The used nixos-unstable channel is on nixos-unstable-21.03pre244772.502845c3e31. I tried both linuxPackages_rpi1 and linuxPackages_latest for boot.kernelPackages.

I found a similar issue description by @sorki in this PR, but that is on aarch64.

Should I open an issue for this (which I so far did not because armv6 is afaik not really in scope), or am I doing something wrong?

2

Hello,

It should work on any architecture.

I’ve managed to reproduce and fix this on _rpi kernel variants, see nixos/device-tree: fix cross compilation for _rpi kernels by sorki · Pull Request #99378 · NixOS/nixpkgs · GitHub

It seems to apply the overlay correctly with

  hardware.deviceTree.filter = "bcm2708-rpi-zero*.dtb";
  hardware.deviceTree.overlays = [
    {
      name = "hifiberry-dac";
      dtboFile = "${pkgs.device-tree_rpi.overlays}/hifiberry-dac.dtbo";
    }
  ];
  • the filter is needed as without it it tries to apply it to bcm2835-rpi-zero.dtb and fails - I don’t quite understand why there’s both bcm2708-rpi-zero.dtb and bcm2835-rpi-zero.dtb in the kernel package.
  • raspberrypi-firmware package contains only bcm2708-rpi-zero.dtb.
  • You can test it according to instructions at GitHub - sorki/rpi-cross: Cross build images for Raspberry Pi from x86 (using hifiberry_pi0 branch)
  • I haven’t tested the image myself yet

Btw I can build armv6 image using linuxPackages_latest but the trouble is you can’t use binary overlays from pkgs.device-tree_rpi.overlays, their sources (from rPi kernel tree could work but most probably needs to be adapted to mainline kernel).

1 Like
3

Thank you very much. I was finally able to build the image when using your PR (#99378), even without using your repository :slight_smile:

I also have a similar problem with the overlay for the HifiBerry DAC+ADC, I hope it’s okay to ask for this here, too:

Right now, I’m trying to use this deviceTree configuration (again within the minimal working example from the first post):

  hardware.deviceTree.filter = "bcm2708-rpi-zero*.dtb"; # This line does not change anything in this case
  hardware.deviceTree.overlays = [
    {
      name = "hifiberry-dacplusadc";
      dtboFile = "${pkgs.device-tree_rpi.overlays}/hifiberry-dacplusadc.dtbo";
    }
  ];

Instead of failing to compile something, it fails to apply that overlay (?).

Failed to apply '/nix/store/dyhmndcicwvzqrgghxcdpm0105miivx6-raspberrypi-firmware-1.20200601-armv6l-unknown-linux-gnueabihf/share/raspberrypi/boot/overlays/hifiberry-dacplusadc.dtbo': FDT_ERR_NOTFOUND

While trying to find the reason, I found that the device-tree builder calls the following when failing to apply the dtbo

fdtoverlay \
  -o /nix/store/mzg1i62bfvj3xqkhzd1hk8gm8986m8wy-device-tree-overlays/bcm2708-rpi-zero-w.dtb \
  -i /nix/store/mzg1i62bfvj3xqkhzd1hk8gm8986m8wy-device-tree-overlays/bcm2708-rpi-zero-w.dtb.in \
  /nix/store/dyhmndcicwvzqrgghxcdpm0105miivx6-raspberrypi-firmware-1.20200601-armv6l-unknown-linux-gnueabihf/share/raspberrypi/boot/overlays/hifiberry-dacplusadc.dtbo

The output path does not exist (probably because it crashes before being created), but both the path to the .dtbo and the .dtb.in do exist. Sadly, I was not able to find any documentation about what the FDT_ERR_NOTFOUND could mean.

4

I’ve played with the binary overlays and rPi kernels and now I think the solution to this is to bring back the old way of applying overlay to all raspberrypi overlays without checking for compatible as the binary rpi overlays are all using compatible brcm,bcm2835.

I think I’ll reintroduce the old way as hardware.deviceTree.raspberrypi.overlays as it is quite rPi specific.

1 Like
5

I’ve also played around with the overlays some more and failed to get it running so far.

I think I’ll reintroduce the old way as hardware.deviceTree.raspberrypi.overlays as it is quite rPi specific.

Would you mind explaining what this old way is? Checking the version history of pkgs/os-specific/linux/device-tree/default.nix suggests that it just drops the compatibility check, but using that approach still fails with

Failed to apply '/nix/store/0w7w6m9iyj5gr4dhfz06yp1d0j74d5g9-raspberrypi-firmware-1.20200902-armv6l-unknown-linux-gnueabihf/share/raspberrypi/boot/overlays/hifiberry-dacplusadc.dtbo': FDT_ERR_NOTFOUND

I used this build command for that step:


    buildCommand = let
      overlays = toList overlays';
    in ''
      mkdir -p $out
      cd ${base}
      find . -type f -name '*.dtb' -print0 \
        | xargs -0 cp -v --no-preserve=mode --target-directory $out --parents

      for dtb in $(find $out -type f -name '*.dtb'); do
        ${flip (concatMapStringsSep "\n") overlays (o: ''
          echo "Applying overlay ${o.name} to $( basename $dtb )"
          mv $dtb{,.in}
          fdtoverlay -o "$dtb" -i "$dtb.in" ${o.dtboFile};
          rm $dtb.in
        '')}

      done
    '';

I also found PR #67989 from @kwohlfahrt, where the dto is applied by using dtmerge from raspberrypi-tools. Using that approach allows successfully building an image, but the hardware is not initialized. For testing, I used this buildCommand in applyOverlays:


    buildCommand = let
      overlays = toList overlays';
    in ''
      mkdir -p $out
      cd ${base}
      find . -type f -name '*.dtb' -print0 \
        | xargs -0 cp -v --no-preserve=mode --target-directory $out --parents

      for dtb in $(find $out -type f -name '*.dtb'); do
        ${flip (concatMapStringsSep "\n") overlays (o: ''
          echo "Applying overlay ${o.name} to $( basename $dtb )"
          mv $dtb{,.in}
          dtmerge -d $dtb.in $dtb ${escapeShellArg o.dtboFile}
          rm $dtb.in
        '')}

      done
    '';

In the PR, there is also a param option introduced, but I’m not really sure whether I need that for the hifiberry-dacplusadc. According to the documentation of it, there is only a dtoverlay added to the config.txt, but adding that option to the dtmerge command results in failure as the param is not know to dtmerge. Relaxing the filter to also use the bcm2835-rpi-zero-w.dtb results in this error, by the way:

Applying overlay hifiberry-dacplusadc to bcm2835-rpi-zero-w.dtb
DTOVERLAY[debug]: using platform 'bcm2835'
DTOVERLAY[debug]: overlay map loaded
DTOVERLAY[debug]: /__symbols__:i2c=i2c0
DTOVERLAY[debug]: /__symbols__:i2c_arm=i2c0
DTOVERLAY[debug]: /__symbols__:i2c_vc=i2c1
DTOVERLAY[error]: can't find symbol 'sound'

According to this Issue that is referenced in PR #67989, dtmerge is also the suggested method to apply dtos.

Any help here is appreciated, I’m kind of really stuck on this one (especially as I don’t really know how dtos work internally, so it’s a bit harder to debug for me).

6

Sorry for the late response, I’ve got kind-of exhausted by trying to figure this out.

Have you managed to get it working? IIRC what I found out was that not every overlay applies to every base device tree (like some work for pi0 but not pi0w which is weird).

My intention was to add something like hardware.deviceTree.raspberry which would use either old method or the one proposed in #67989 with dtmerge but I’m still not sure which way is the correct one.

I guess we have some time to figure this out since this didn’t make it into 20.09 but e.g. instructions on the wiki for Pi4 with GPU are currently broken with master.