Recommended style to cross-compile flake nixosSystems

Help
1

I’m currently tinkering with cross-compiling NixOS from an AMD machine (“x86_64-linux”) to Raspberry Pi 5 (“aarch64-linux”) and Starfive VisionFive 2 (“riscv-linux”), and would like to know about the latest best practice to recommend in guides. I found several similar styles, but i’m not sure which one is the latest and best:

 mySystem = nixpkgs.lib.nixosSystem {
        modules = [
          {
            nixpkgs.buildPlatform = "x86_64-linux";
            nixpkgs.hostPlatform = "riscv64-linux";
          }
        ];
      };

like it seems to be recommended in the official NixOS manual cross-compile section (but unclear as it mentions other styles as well) and in Jörg Thalheims blog post.

 mySystem = nixpkgs.lib.nixosSystem {
        modules = [
          {
            nixpkgs.localSystem.config = "x86_64-unknown-linux-gnu";
            nixpkgs.crossSystem.config = "riscv64-unknown-linux-gnu";
          }
        ];
      };

that seems to be recommended in the official nix.dev cross-compile guide (although that style is shown only for pkgsCross usage)

 mySystem = nixpkgs.lib.nixosSystem {
        modules = [
          {
            nixpkgs.localSystem.system = "x86_64-linux";
            nixpkgs.crossSystem.system = "riscv64-linux";
          }
        ];
      };

that seems to be used in multiple blogs like the fantastic recently published handbook guide by Ryan Yin.

 mySystem = nixpkgs.lib.nixosSystem {
       system = "x86_64-linux"
        modules = [
          {
            nixpkgs.crossSystem.system = "riscv64-linux";
          }
        ];
      };

I have seen this style many times in blog posts.

 mySystem = nixpkgs.lib.nixosSystem {
       system = "x86_64-linux"
        modules = [
          {
            nixpkgs.crossSystem.config = "riscv64-unknown-linux-gnu";
            nixpkgs.crossSystem.system = "riscv64-linux";
          }
        ];
      };

I have sometimes also have seem both system and config having been defined.

So which one is the latest, recommended way? (Or what are the trade-offs?). I could create a PR against the official documentation as well if you can point me to the right place.

5 Likes
2

Shameless bump as i stumbled upon this part-hilarious blog post https://ianthehenry.com/posts/how-to-learn-nix/cross-compilation/ about the state of the Nix manual on cross-compiling and it describes the same questions and confusion i have. I would like to improve the documentation and start with this low-hanging fruit here.

6 Likes
3

What did you decide on is ultimately the best currently?

4

I am also confused why we would encode the build platform.

That means if I wanted to do a nixos-rebuild switch on my other machine (i.e. Raspberry Pi) it would now fail?

5

Okay I could do:

        {
          # This is the architecture we build from (pkgs.system from above)
          nixpkgs.buildPlatform = builtins.currentSystem;
          # pkgsCross.<yourtarget>.system
          nixpkgs.hostPlatform = "aarch64-linux";
        }

but you have to pass --impure but i’m not sure otherwise how to make a nixosConfiguration that I can either build on my laptop or on my raspberry pi. I feel like I’m missin gsomething.

1 Like
6

I enabled this on my notebook to build the config for the Raspberry Pis:

I have two different “configs” however for x86 and aarch64, don’t know if there is a better way but it’s what I’ve been using for a while now.

7

That’s flake-specific. With flakes, you’re supposed to create additional outputs that have different system configurations.

8

I haven’t decided. I’m still waiting for some Nix experts to give recommendations. I think i remember all variants working, but it’s confusing for non-experts as every guide is slightly different.

9

That’s because, as with many things in Nix, there isn’t one proven best way of doing things. If you ask 4 experts, you’ll get 4 opinions on how things should be done.

You’ll have to think for yourself here.

10

I couldn’t get it to work. Something was failing.
I resorted back to binfmt sadly.

I also had to disable check phase for a few packages that were expensive to do in emulation.

11

@malteneuss
I’d like to throw in two more ways to declare cross compilation:

mySystem = nixpkgs.lib.nixosSystem {
  system = "x86_64-linux"
  modules = [
    {
      nixpkgs.buildPlatform = "x86_64-linux";
      nixpkgs.hostPlatform = nixpkgs.lib.systems.examples.armv7l-hf-multiplatform;
    }
  ];
};

mySystem = nixpkgs.legacyPackages.x86_64-linux.pkgsCross.armv7l-hf-multiplatform.nixos {
  imports = [
    ./my-module.nix
  ];
};

Thank you for sharing :slight_smile:

I wish there was a way to search for specific hydra built images and inspect the sources for all those artifacts. I am hoping this could help me overcome my current armv7l cross compilation issues.

2 Likes