Nix CPU global CPU flags

Help
1

It used to be that setting

{ config, pkgs, lib, ... }:
{
    nixpkgs.localSystem = {
      gcc.arch = "cpufamily";
      gcc.tune = "cpufamily";
      system = "x86_64-linux";
    };
}

within configuration.nix would enable CPU optimization for the whole system. It however isn’t the case anymore, gcc.arch and gcc.tune have been, from as far as I can tell, discontinued.

How would one set CPU flags for the C compiler within NixOS 22.11? From reading about it I think

    nixpkgs.localSystem = {
      cpu = "cpufamily";
      system = "x86_64-linux";
    };
}

Would work but I can’t quite tell.

Can you please confirm the information I have found?

Thank you for your aid.

4 Likes
2

I have struggled with something similar. I wanted to do a build optimized for the x86-64-v3 feature level. Here is what didn’t work for me.

I started with just this:

  nixpkgs.localSystem = {
    gcc.arch = "x86-64-v3";
    gcc.tune = "generic";
    system = "x86_64-linux";
  };

This complained about nix system features, and so I had to add gccarch feature:

nix.settings.system-features = [ "nixos-test" "benchmark" "big-parallel" "kvm" "gccarch-x86-64-v3" ];

This then complained about the architecture not being valid, so I tried again targeting skylake instead:

  nix.settings.system-features = [ "nixos-test" "benchmark" "big-parallel" "kvm" "gccarch-skylake" ];
  nixpkgs.localSystem = {
    gcc.arch = "skylake";
    gcc.tune = "skylake";
    system = "x86_64-linux";
  };

With these flags the build did start to run, but from what I could see with ps none of the invocations of gcc actually had the march or mtune flags set as I’d expect. I interrupted the build and have given up on this for the time being.

I would be very interested to hear if you figure out a solution to this, especially if you get the micro-architecture feature level build working.

4 Likes
3

@pauldoo @superfly.johnson did you guys figure out a way? i have looked all around but can’t find one.

4

Don’t you also either need to set crossSystem? I’m not on the same architecture, but this probably is working for me on zen3. There was a reddit post about this here.

To be clear, the currently works for me:

    nix.settings.system-features = [ "gccarch-znver3" ];
    import pkgs {
      inherit overlays;
      crossSystem = {
        system = "x86_64-linux";
        gcc.arch = "znver3";
        gcc.tune = "znver3";
        gcc.abi = "64";
      };
      localSystem = {
        system = "x86_64-linux";
        gcc.arch = "znver3";
        gcc.tune = "znver3";
        gcc.abi = "64";
      };
    };

I think there were a couple of tricky things:

  • I don’t know if -march=$YOURARCH is ABI incompatible with not having extra instruction sets enabled? That’s what the reddit post says, but I can’t think what the incompatibility would be.
  • The “chicken and egg” problem mentioned in the reddit post is tricky. The first time you compile the system on an architecture without -march=$YOURARCH, your localSystem is x86_64 without the -march=YOURARCH, but once you’ve nixos-switched to the config with -march=$YOURARCH, then you need to build your system config with -march=$YOURARCH set for both crossSystem and localSystem. (same line of reasoning applies to tune)

But does this work? I have to recompile the world every time I upgrade, so I think so? As a case study I checked out htop, and built it from source with and without -march=native. Then, I ran elfx86exts over it. The result without -march=native is:

File format and CPU architecture: Elf, X86_64
MODE64 (call)
CMOV (cmovge)
SSE2 (movd)
SSE1 (movups)
BMI (tzcnt)
Instruction set extensions used: BMI, CMOV, MODE64, SSE1, SSE2
CPU Generation: Haswell

And with -march=native:

File format and CPU architecture: Elf, X86_64
MODE64 (call)
CMOV (cmovge)
AVX (vmovd)
AVX2 (vinserti128)
NOVLX (vpxor)
BMI2 (shlx)
BMI (tzcnt)
Instruction set extensions used: AVX, AVX2, BMI, BMI2, CMOV, MODE64, NOVLX
CPU Generation: Unknown

And this matches what is in the nix store for htop built with -march=znver3 in my system config. So, I think it’s safe to say the -march=znver3 is being applied.

1 Like
5

An interesting follow up question (that maybe someone else knows the answer to?) is how does one add LTO globally? I haven’t been able to figure out how to even add -flto globally to CFLAGS. This documentation is broken. And there doesn’t seem to be a place to append to LDFLAGS AFAICT.