Home-Manger Cross Compilation?

I am partly asking this question so that future ppl hopefully don’t have to suffer as much as I do. I am more or less a nix beginner and currently I am trying to make an sd-card image for a raspberry pi 4 with my custom nixos config.

Pretty much everything works well, except that home manager fails to compile its packages (I am using home-manager as a nixos module in a flake).

I get this output:

error: Cannot build '/nix/store/46cy3f079mhzhbzrk17mgxhmvzzi8pll-home-manager-path.drv'.
       Reason: platform mismatch
       Required system: 'aarch64-linux'
       Current system: 'x86_64-linux'
error: Cannot build '/nix/store/m8shf11gd945zk87jhc6nq7xc0gz0fx5-home-manager-generation.drv'.
       Reason: 1 dependency failed.
       Output paths:
         /nix/store/9h5jzl21jwxrn2gi1qx73wzigigbdb2j-home-manager-generation
error: Cannot build '/nix/store/xdqg140pa356rgrcrjxy17dmfh55qhhm-unit-home-manager-enzy.service.drv'.
       Reason: 1 dependency failed.
       Output paths:
         /nix/store/y6gkk23xszrymfypqfyjm20glwhq6plg-unit-home-manager-enzy.service
error: Cannot build '/nix/store/2vnlbyw67v6hyqanxffhlhiwv6b04qqr-system-units.drv'.
...

Nothing that I tried enabled cross compilation in home manager:

nixpkgs.hostPlatform.system = "aarch64-linux";
nixpkgs.buildPlatform.system = "x86_64-linux";

# or setting

environment.systemPackages = [
  home-manager.packages."aarch64-linux".default
];

# or setting
programs.home-manager.package = pkgs.pkgsCross.raspberryPi.home-manager;

Generally, is it even possible to cross compile with home manager? I know that I can set boot.binfmt.emulatedSystems = [ "aarch64-linux" ] but that results in reaaaaly slow compilation times and i wanna avoid that.

Here is my current configuration

Cheers!

PS having both boot.binfmt.emulatedSystems = [ "aarch64-linux" ] set and

nixpkgs.hostPlatform.system = "aarch64-linux";
nixpkgs.buildPlatform.system = "x86_64-linux";

seems to result in most packages being built without emulation (fast) but the ones that are failing for me with emulation. This is acceptable but requires the host to set boot.binfmt.emulatedSystems = [ "aarch64-linux" ] in their configuration outside of e.g. my flake..