I wrote a flake that has both x86_64-linux and aarch64-linux versions of the same package.
Now I would like to cross-compile that package for aarch64-linux on an x86_64-linux machine.
Is there a good way to do that?
None of the flags/options I saw in the wiki or elsewhere online seem to apply. Is there already a (convenient) way do do this, or is that functionality just not available yet?
For context this is the flake I’m working on.
I can’t say I’ve tried it, but would an option like registering QEMU as a binfmt wrapper like on NixOS on ARM - NixOS Wiki work?
Oh wow. Looks like it’s building something now. Thanks!
I will update this topic with the results.
Following your suggestion I added
boot.binfmt.emulatedSystems = [ "aarch64-linux" ];
to the configuration of my x86_64-linux build system.
As soon as I did that I was able to build my flake using
nix build .#defaultPackage.aarch64-linux
That worked like a charm! 
I even copied over the closure to verify that the result runs on the actual aarch64-linux target, and it does. 
Just as a reference for others, before adding that option I only got the following error message:
nix build .#defaultPackage.aarch64-linux
warning: Git tree '/home/mschwaig/flakes/tensorflow-lite-hello-world' is dirty
error: --- Error ----------------------------------------------------- nix
a 'aarch64-linux' with features {} is required to build '/nix/store/my1h7yqcplmpihygkhq8s4i4syfcazpv-builder.pl.drv', but I am a 'x86_64-linux' with features {benchmark, big-parallel, kvm, nixos-test}
(use '--show-trace' to show detailed location information)
This is emulated native compilation, not cross compilation. I don’t think proper cross compilation is part of the flakes design, yet?
@mschwaig what did you do with the ./result exactly? After it was built. I mean, how did you copy it to the target machine?
Thanks.
You can do cross compilation with flakes:
$ uname -a
Linux jon-desktop 5.4.142 #1-NixOS SMP Wed Aug 18 06:57:05 UTC 2021 x86_64 GNU/Linux
$ nix build nixpkgs#legacyPackages.x86_64-linux.pkgsCross.aarch64-multiplatform.hello
$ file ./result/bin/hello
./result/bin/hello: ELF 64-bit LSB executable, ARM aarch64, version 1 (SYSV), dynamically linked, interpreter /nix/store/zh8wdxlvbaycfnnd3c9hzq07h5x7blaw-glibc-aarch64-unknown-linux-gnu-2.33-47/lib/ld-linux-aarch64.so.1, for GNU/Linux 2.6.32, not stripped
Using boot.binfmt.emulatedSystems = [ "aarch64-linux" ]; is still probably preferred, as the target system will likely expect to use aarch64 packages that were built on aarch64 (cross compilation produces different .drv’s). I’m not really sure how to “consume” cross compiled packages without referencing the store path directly.
You can use a nix copy command to copy the result to the target machine like this:
nix copy --to ssh://user@hostname .#defaultPackage.aarch64-linux
You can also use that command to copy to a binary cache and have the target get to them that way.
I am not convinced being able to do
nix build nixpkgs#legacyPackages.x86_64-linux.pkgsCross.aarch64-multiplatform.hello
means that you can do ‘real’ cross-compilation with Flakes, as an alternative to compiling under emulation. This seems helpful if you want to cross-compile a package from nixpkgs with the new nix command.
As far as I can see this only works with nxpkgs specifically and I don’t know what it would take to have any other Flake to offer an interface for cross-compilation like that. I have not read about anything like that yet.
apply an overlay adding your package, then any of the pkgsX cross-compilation overlays will also work.
@jonringer do you have an example of doing that? I am struggling to figure out how to get it to work based on my limited nix (and flake) experience, and I can’t seem to find any documentation showing how to do it with a flake.
I was also struggling with this and in the end, I assembled from all the hints here and after some tries the following flake file:
{
description = "flake description";
outputs = { self, flake-utils, nixpkgs }: {
overlay = final: prev: import ./pkgs { nixpkgs = prev; };
nixosModules = import ./nixos;
nixosModule = {
imports = builtins.attrValues self.nixosModules;
nixpkgs.overlays = [ self.overlay ];
};
} // flake-utils.lib.eachDefaultSystem (system: {
packages = flake-utils.lib.filterPackages system (import ./pkgs {
nixpkgs = nixpkgs.legacyPackages."${system}";
nixlib = nixpkgs.lib;
});
# The legacyPackages imported as overlay allows us to use pkgsCross to
# cross-compile those packages.
legacyPackages = import nixpkgs {
inherit system;
overlays = [ self.overlay ];
crossOverlays = [ self.overlay ];
};
});
}
The important thing here is that I used legacyPackages instead of packages as otherwise nix flake show would fail. This way I can list content with nix flake show . as well as do cross-compilation using nix build .#pkgsCross.aarch64-multiplatform.foo (as an example).
Is there a simple better way to do this? Adding overlay seems kinda hacky…
I am specifically looking to build a boot ISO for my raspberry Pi. So far I’ve been deploying to it using cross-compiling from my PC just fine, but for some reason ISO generation is getting overly complicated.
I get this
error: getting status of ‘/nix/store/xmgg331jcch5jxih6agbnbcr56v76f88-source/pkgs’: No such file or directory
My flake expects that packages are defined in either directory pkgs or file pkgs.nix. If that is how you have it then make sure that it is part of git. Nix won’t include files outside of git in case of flake. This smells like you either do not have pkgs or is just ignored by nix.
Feel free to browse this flake for example nixturris - NixOS support for Turris Omnia and Mox.
@payas here’s an SD image that I made for Raspberry Pi in case that’s helpful to you:
I’m sharing it here because of how simple it is to specify. I’m fairly sure this is compiled under emulation and not cross-compiled. I’m also wondering what it would take to make cross-compilation similarly ergonomic, maybe it’s possible to write a nice function for that.
packages are defined in flakes using an overlay which is applied to the Nixpkgs set. This allows your packages to be cross-compiled in other flakes.
This also limits how i write overlays
https://git.sr.ht/~ehmry/sigil/tree/post-mortem/item/overlay/default.nix
gemini://gemini.spam.works/~emery/sigil-report.gmi
At this point in time, flakes are designed to take away control of the nixpkgs invocation, while neglecting to support cross compilation and other Nixpkgs configuration.
By using overlays, you opt out of the way flakes suggest to do things, which is by using the inputs’ packages and legacyPackages attributes. This reduces your use of flakes to little more than a simple lock manager like niv, and you’ll be doing things the way we used to do them before flakes.
While I consider this a valid thing to do, I can’t call it “cross-compiling a flake” especially considering that you have to write your own flake in order to even invoke nixpkgs for cross compilation. True cross compilation support would allow any flake out there to be evaluated for cross compilation, without having to write your own. Barring any mistakes in the expressions or lack of cross support in packaged tools, this should just work.
It’s possible to cross compile in a kinda hacky way in flakes without using an overlay but it’s much less nice than nixpkgs’ infra for it.
dxvk and vkd3d-proton + installer? by LunNova · Pull Request #42 · fufexan/nix-gaming · GitHub here’s an example which cross compiles some packages.
DXVK is in nixpkgs. Is it not suitable for your needs?