Targeting particular x86_64 generation (haswell/ivybridge/skylake/...)

[volth]

https://github.com/NixOS/nixpkgs/pull/56649

[vcunat]

@volth

The only problem with the official cache is it might already have binaries which fail to run on

Yes, we need to find those particular cases and patch them. Hydra.nixos.org machines are quite a variety of machines, so the result wouldn’t even be always the same.

We could make a cc-wrapper variant that detects these flags, but that would be a bit complicated, as we surely do want to allow -march=whatever outside nixpkgs while using compiler from nixpkgs. I’m not sure it’s worth it.

[7c6f434c]

We could make a cc-wrapper variant that detects these flags, but that would be a bit complicated, as we surely do want to allow -march=whatever outside nixpkgs while using compiler from nixpkgs. I’m not sure it’s worth it.

I think there are already some limitations we enforce for most of Nixpkgs but allow lifting by a build-time environment variable; I don’t see why it would be unsuitable here. Reproducibility might be worth it…

(As for x86_64 subtypes, I would support it, if mainline Nixpkgs part of the change is a list of platforms plus a low number of changes to fix platform specification in compilers as needed; just what is enough to allow overlays to work sanely with generation-specific or vendor-specific package flavours)

[volth]

This also raises a problem that we cannot build on an older processor targeting a newer one (the compilation is ok, but small programs compiled on configurePhase, tests, … do crash).
So, remote builders should expose their gcc.arch

[vcunat]

@volth

So, remote builders should expose their gcc.arch

That’s already doable via supportedFeatures on machines and meta.requiredSystemFeatures on derivations, though it’s not “standardized” for this use case ATM. Well, so far we seemed to be mostly OK with generic binaries like all larger distros (AFAIK, discounting source-focused ones like Gentoo), at least for the official NixPkgs; IMHO it’s not too difficult for users to override some “leaf” packages for this purpose and keep rebuilding them.

[volth]

@vcunat

That’s already doable via supportedFeatures on machines and meta.requiredSystemFeatures on derivations

Yes, it would be perfect, assuming there is only ordered gcc.arch.

Although it won’t cover well the cases like Thinkpad X220’s

gcc = {
  arch = "sandybridge";
  extraFlags = ["-mavx"]; # -march=sandybridge alone does not enable AVX
};

[vcunat]

@volth

Although it won’t cover well the cases like Thinkpad X220’s

I don’t think it’s worth aiming for 100% accuracy, more like meaningfully splitting the “range” between plain x86_64 and new CPUs; just creating a few groups will probably be enough to get almost all the effect. Actually feature flags like -mavx feel more significant for this. The -mtune part of -march would be more relevant for old machines, I think, as the default/generic optimizer in GCC shifts in time towards the commonly used CPU types.

[kamilchm]

Is there a way to set things like gcc = { arch = ... } system wide?

[volth]

in /etc/nix/configuration.nix

  nixpkgs.localSystem.platform  = lib.systems.platforms.pc64 // {
    gcc.arch = "skylake";
    gcc.tune = "skylake";
  };

[t184256]

@vcunat

The usual approach is that upstream determines parts of code where it’s worth it, sets up the build system to compile multiple variants and adds runtime switches among them. It’s been common e.g. for multimedia packages (like ffmpeg), for many years IMHO, and it seems the best approach to me…

I just hope function multiversioning would just become fully automatic one day.

[kamilchm]

I get an error:

error: undefined variable 'lib' at /etc/nixos/configuration.nix

I also tried:

  nixpkgs.localSystem.platform  = pkgs.lib.systems.platforms.pc64 // {

without a success:

error: attribute 'system' missing, at <nixpkgs>/lib/systems/default.nix:21:81

Do you have any complete working /etc/nixos/configuration.nix example?

[volth]

@kamilchm

without a success:

error: attribute 'system' missing, at <nixpkgs>/lib/systems/default.nix:21:81

then

  nixpkgs.localSystem.system   = "x86_64-linux"; # or =builtins.currentSystem
  nixpkgs.localSystem.platform = pkgs.lib.systems.platforms.pc64 // {
    gcc.arch = "skylake";
    gcc.tune = "skylake";
  };

@kamilchm

Do you have any complete working /etc/nixos/configuration.nix example?

No, I am not using /etc/nixos/configuration.nix, I use fork of nixops even to manage localhost as though it is one of the remote machines.

[henrycrutcher]

How did you do that? I’d love to manage my machine with nixops! To be precise, I’d love to know how to use the fork of nixops you are using😀

[volth]

No, it was not designed as a detachable product so it has many hardcoded values and other assumption (for example, about a Tinc network connecting all the devices) on my infrastructure.

I think it should be easy to add this feature to mainline nixops (if not done yet), it is just about producing a system closure and calling switch-to-configuration.pl

[volth]

@kamilchm Does it work?

[OpenSourceAnarchist]

@volth If I disable access to the binary cache and add

 nixpkgs.localSystem.platform = pkgs.lib.systems.platforms.pc64 // {
    gcc.arch = "skylake";
    gcc.tune = "skylake";
  };

to my /etc/nixos/configuration.nix, will that cause every package to be rebuilt on my computer with these optimized settings? If not, how can this be achieved? Ideally, I want to make NixOS as Gentoo-like as possible and build every package on my system with “-O3 -march=native …”

I know this breaks reproducibility/purity but packages will only ever be on my system so I’m fine with that. Is this even possible?

[volth]

@OpenSourceAnarchist

If I disable access to the binary cache and add

 nixpkgs.localSystem.platform = pkgs.lib.systems.platforms.pc64 // {
    gcc.arch = "skylake";
    gcc.tune = "skylake";
  };

to my /etc/nixos/configuration.nix , will that cause every package to be rebuilt on my computer with these optimized settings?

Yes.
And you do not have to disable access to the binary cache, the packages just have other hashes.
So it does not even break purity

[OpenSourceAnarchist]

@volth Thank you so much! Learning to use NixOS is a fun exercise (that I’m falling in love with).

I’m still stumped on overriding default makeopts like CFLAGS, CXXFLAGS, etc. With Gentoo it is so easy… I found this in the nixpkgs manual:

makeFlagsArray

A shell array containing additional arguments passed to make . You must use this instead of makeFlags if the arguments contain spaces, e.g.

preBuild = 
'' makeFlagsArray+=(CFLAGS="-O0 -g" LDFLAGS="-lfoo -lbar")
 '';

Note that shell arrays cannot be passed through environment variables, so you cannot set makeFlagsArray in a derivation attribute (because those are passed through environment variables): you have to define them in shell code.

So how exactly can I set these for every package? It feels like this should be as easy as changing gcc.arch or gcc.tune!

Edit: Could I do it like this (copied from a Nix cheatsheet site)?
stdenv.userHook = '' NIX_CFLAGS_COMPILE+=" -march=native" '';

Or is something like this required and an overridden stdenv?

[volth]

@OpenSourceAnarchist

So how exactly can I set these for every package?

Probably, you cannot without patching nixpkgs.

In the initial proposal there was gcc.extraFlags (akin to gcc.arch) (https://github.com/NixOS/nixpkgs/search?q=gcc.extraFlags&type=Issues) but it did not get to the mainline

cc @matthewbauer

[matthewbauer]

I think the best way to do this now is like this:

let
  pkgs = import <nixpkgs> {
    crossOverlays = [
      (self: super: {
        stdenv = super.stdenvAdapters.impureUseNativeOptimizations super.stdenv;
      })
    ];
  };
in pkgs.hello

You can use “overlays” or “crossOverlays”, but overlays requires you to rebuild the entire stdenv, while you probably don’t need that. On the other hand, this will make sure that everything is built with -march=native.

Other “adapters” are possible, see https://github.com/NixOS/nixpkgs/blob/c7b4b3bb1677d4b50251605a428016c6b5ac1d60/pkgs/stdenv/adapters.nix

We probably should add some more for things like this.