Hello! I’m fairly new to nix and I’m having a fantastic experience except for struggling to replicate one aspect of my arch system. I’m trying build linux-xanmod with march and mtune set and I’m running to the following error relating to bootstrapping glibc on the build.
error: a 'x86_64-linux' with features {gccarch-znver2} is required to build '/nix/store/rbnjbsshwxiv7ax29wphsxbsrgsc7mkd-bootstrap-stage0-glibc-bootstrap.drv', but I am a 'x86_64-linux' with features {benchmark, big-parallel, kvm, nixos-test}
I’ve attempted adding:
features = [ "gccarch-znver2" ];
to nixpkgs.localSystem, but there is no apparent change in build output.
GCC w/ march and mtune set works on the running systems as shown here:
gcc -march=znver2 -mtune=znver2 /dev/null
/nix/store/rq6bh3qfrqnyqwik0w3q6w180zg3w2pa-binutils-2.38/bin/ld: cannot open output file a.out: Permission denied
collect2: error: ld returned 1 exit status
Note that without nixpkgs.localSystem set the build works fine, but I’m then not getting the optimizations I’m looking for.
Nixpkgs needs to set requiredFeatures in order to guarantee that the machine that builds your system can really do so. This is necessary because we’ve had problems with cpu extensions on the NixOS build farm. You could either configure your NixOS machine to have this feature, in the nix.settings.supported-features option (nix.conf - Nix Reference Manual), or you could use cross compilation to allow your system config to be built on any x86_64-linux host, including ones with other cpus.
That’s an old post, but the recipe for actually building the kernel with some additional flags is still nowhere to be found. It took me a night, and I sort of did it without rebuilding the world — but it really does look suboptimal tbh.
boot.kernelPackages = with pkgs; let tune = "alderlake"; in (linuxKernel.packagesFor (linux_latest.override ({
stdenv = stdenvAdapters.addAttrsToDerivation {
env.KCPPFLAGS = "-march=${tune} -O2";
env.KCFLAGS = "-march=${tune} -O2";
} stdenv;
})));
Some rationale:
You can’t actually get KCPPFLAGS\KCFLAGS to work from extraMakeFlags override, because that crashes config derivation due to the silly way escapes handled there. It only crashes if you have spaces, and more than one parameter (so you can get -mtune=smth to work). If you try to escape spaces, then the kernel derivation build fails due to gcc thinking it’s one big parameter. I’ve hit my head on this thing for a while
Neither overriding gcc.tune/gcc.arch works — I guess configuration passthroughs are broken somewhere. I’ve tried tracing stdenv flag generation, and it always receives empty parameters. So stdenv.override { target/host/buildPlatform = elaborate { system = “…“; gcc = { … } } } will not lead you anywhere useful — at least at the task of building the kernel.
I think from what i’ve read march is enough, and mtune is not required if you aim to build for your cpu only.
There is this article — so I guess -O3 is not that hot. But given that in NixOS we also build for x86-64-v1 (for whatever reason we still do that) — building with march set to something more reasonable might actually yield some performance gains. I am too lazy rn to check that thought.
-O3 is probably within even just variance – even without considering the massive error bars that memory layout as a measurement bias imposes.
The article contradicts this prediction. The tests involving IO show quite significant regressions when using -march=native. They’re so significant that they probably cannot be explained by measurement bias either but even here I don’t think we can be sure.
no reliable data showing any benefit in switching to v2 that cannot be explained by known measurement biases (or demonstration that the measurement biases are less significant/relevant nowadays)
anecdata that people still use NixOS on v1-only CPUs and would no longer be able to
v3 (or higher) might improve perf significantly (don’t have an overview over the data rn) but it doesn’t matter as that would probably cut off too many users
dynamic alternatives to raising the march level globally exist in userspace and should be explored first
