Kernel Cross Compilation Overlay

Help
1

I have a device I have been trying to get NixOS to run on, but it requires a custom kernel. The device is relatively low powered (a rpi CM4 essentially), so it would take a while for it to build the kernel itself. I have attempted to solve this by pre-building an SD image, and using an overlay (specifically linuxKernel and linuxPackagesFor) to make the kernel cross-compile on my proper computer. The kernel compiles fine, but it also seems to make a bunch of other packages that I never specified need to be built natively (through qemu emulation), such as glib and python3/python3-env. Is there any way I can mitigate these other package rebuilds, a better way of doing cross-compilation, or am I simply doing something weird enough that I have to deal this?

2

Emulation and cross-comp are not the same thing. Can you share what you’re doing here using code?

3

I have an overlay set up with

    ienCrossPkgs = (import nixpkgs-unstable {
      localSystem = "x86_64-linux";
      crossSystem = "aarch64-linux";
    });
    crossKernelOverlay = final: prev: {
      inherit (ienCrossPkgs) linuxKernel linuxPackagesFor;
    };

and then on my build system I also have

boot.binfmt.emulatedSystems = [ "aarch64-linux" ];

I don’t mind using qemu to compile other things, but the kernel simply takes so long I would like to do that non-emulated.
The actual kernel build I am taking from someone else who seemed to have figured it out, but did not entirely respond to messages

{pkgs, ...}: let
  kernelPackagesCfg = {
    linuxPackagesFor,
    linux_rpi4,
    fetchFromGitHub,
  }: let
    # Version picked from the current (as of 8th Oct 2024) nixpkgs-unstable branch
    modDirVersion = "6.6.51";
    tag = "stable_20241008";
  in
    linuxPackagesFor (linux_rpi4.override {
      argsOverride = {
        version = "${modDirVersion}-${tag}-uc-cm4";
        inherit modDirVersion;

        src = fetchFromGitHub {
          owner = "raspberrypi";
          repo = "linux";
          rev = tag;
          hash = "sha256-phCxkuO+jUGZkfzSrBq6yErQeO2Td+inIGHxctXbD5U=";
        };
        ignoreConfigErrors = true;
      };
    });
  patches = [
    ./patches/001-OCP8178-backlight-driver.patch
    ./patches/002-drm-panel-add-clockwork-cwu50.patch
    ./patches/003-axp20x-power.patch
    ./patches/004-vc4_dsi-update.patch
    ./patches/005-bcm2835-audio-staging.patch
    ./patches/007-drm-panel-cwu50-expose-dsi-error-status-to-userspace.patch
    ./patches/008-driver-staging-add-uconsole-simple-amplifier-switch.patch
  ];
in {
  boot.kernelPackages = pkgs.callPackages kernelPackagesCfg {};

My end goal is only to have the SD image for this system built in a reasonable amount of time, and qemu emulation compiling the kernel seemed to be pushing that. I’m willing to sit through qemu compiling these ~180 packages if I have to, but I can’t tell why they’re different from the ones in cache, especially for such non-obscure things as glib, or why it’s forcing me to build things like fontforge at all. I can include the results of nix build --dry-run if that is a clue.

4

I switched from doing an overlay to running kernelPackagesCfg with a callPackages from the cross compilation overlay, and that seems to have done the trick. Now I just have to deal with the screen not working, but that’s a separate problem from doing weird things with cross compilation.