Targeting particular x86_64 generation (haswell/ivybridge/skylake/...)

[volth]

@vcunat

That’s already doable via supportedFeatures on machines and meta.requiredSystemFeatures on derivations

Yes, it would be perfect, assuming there is only ordered gcc.arch.

Although it won’t cover well the cases like Thinkpad X220’s

gcc = {
  arch = "sandybridge";
  extraFlags = ["-mavx"]; # -march=sandybridge alone does not enable AVX
};

[vcunat]

@volth

Although it won’t cover well the cases like Thinkpad X220’s

I don’t think it’s worth aiming for 100% accuracy, more like meaningfully splitting the “range” between plain x86_64 and new CPUs; just creating a few groups will probably be enough to get almost all the effect. Actually feature flags like -mavx feel more significant for this. The -mtune part of -march would be more relevant for old machines, I think, as the default/generic optimizer in GCC shifts in time towards the commonly used CPU types.

[kamilchm]

Is there a way to set things like gcc = { arch = ... } system wide?

[volth]

in /etc/nix/configuration.nix

  nixpkgs.localSystem.platform  = lib.systems.platforms.pc64 // {
    gcc.arch = "skylake";
    gcc.tune = "skylake";
  };

[t184256]

@vcunat

The usual approach is that upstream determines parts of code where it’s worth it, sets up the build system to compile multiple variants and adds runtime switches among them. It’s been common e.g. for multimedia packages (like ffmpeg), for many years IMHO, and it seems the best approach to me…

I just hope function multiversioning would just become fully automatic one day.

[kamilchm]

I get an error:

error: undefined variable 'lib' at /etc/nixos/configuration.nix

I also tried:

  nixpkgs.localSystem.platform  = pkgs.lib.systems.platforms.pc64 // {

without a success:

error: attribute 'system' missing, at <nixpkgs>/lib/systems/default.nix:21:81

Do you have any complete working /etc/nixos/configuration.nix example?

[volth]

@kamilchm

without a success:

error: attribute 'system' missing, at <nixpkgs>/lib/systems/default.nix:21:81

then

  nixpkgs.localSystem.system   = "x86_64-linux"; # or =builtins.currentSystem
  nixpkgs.localSystem.platform = pkgs.lib.systems.platforms.pc64 // {
    gcc.arch = "skylake";
    gcc.tune = "skylake";
  };

@kamilchm

Do you have any complete working /etc/nixos/configuration.nix example?

No, I am not using /etc/nixos/configuration.nix, I use fork of nixops even to manage localhost as though it is one of the remote machines.

[henrycrutcher]

How did you do that? I’d love to manage my machine with nixops! To be precise, I’d love to know how to use the fork of nixops you are using😀

[volth]

No, it was not designed as a detachable product so it has many hardcoded values and other assumption (for example, about a Tinc network connecting all the devices) on my infrastructure.

I think it should be easy to add this feature to mainline nixops (if not done yet), it is just about producing a system closure and calling switch-to-configuration.pl

[volth]

@kamilchm Does it work?

[OpenSourceAnarchist]

@volth If I disable access to the binary cache and add

 nixpkgs.localSystem.platform = pkgs.lib.systems.platforms.pc64 // {
    gcc.arch = "skylake";
    gcc.tune = "skylake";
  };

to my /etc/nixos/configuration.nix, will that cause every package to be rebuilt on my computer with these optimized settings? If not, how can this be achieved? Ideally, I want to make NixOS as Gentoo-like as possible and build every package on my system with “-O3 -march=native …”

I know this breaks reproducibility/purity but packages will only ever be on my system so I’m fine with that. Is this even possible?

[volth]

@OpenSourceAnarchist

If I disable access to the binary cache and add

 nixpkgs.localSystem.platform = pkgs.lib.systems.platforms.pc64 // {
    gcc.arch = "skylake";
    gcc.tune = "skylake";
  };

to my /etc/nixos/configuration.nix , will that cause every package to be rebuilt on my computer with these optimized settings?

Yes.
And you do not have to disable access to the binary cache, the packages just have other hashes.
So it does not even break purity

[OpenSourceAnarchist]

@volth Thank you so much! Learning to use NixOS is a fun exercise (that I’m falling in love with).

I’m still stumped on overriding default makeopts like CFLAGS, CXXFLAGS, etc. With Gentoo it is so easy… I found this in the nixpkgs manual:

makeFlagsArray

A shell array containing additional arguments passed to make . You must use this instead of makeFlags if the arguments contain spaces, e.g.

preBuild = 
'' makeFlagsArray+=(CFLAGS="-O0 -g" LDFLAGS="-lfoo -lbar")
 '';

Note that shell arrays cannot be passed through environment variables, so you cannot set makeFlagsArray in a derivation attribute (because those are passed through environment variables): you have to define them in shell code.

So how exactly can I set these for every package? It feels like this should be as easy as changing gcc.arch or gcc.tune!

Edit: Could I do it like this (copied from a Nix cheatsheet site)?
stdenv.userHook = '' NIX_CFLAGS_COMPILE+=" -march=native" '';

Or is something like this required and an overridden stdenv?

[volth]

@OpenSourceAnarchist

So how exactly can I set these for every package?

Probably, you cannot without patching nixpkgs.

In the initial proposal there was gcc.extraFlags (akin to gcc.arch) (https://github.com/NixOS/nixpkgs/search?q=gcc.extraFlags&type=Issues) but it did not get to the mainline

cc @matthewbauer

[matthewbauer]

I think the best way to do this now is like this:

let
  pkgs = import <nixpkgs> {
    crossOverlays = [
      (self: super: {
        stdenv = super.stdenvAdapters.impureUseNativeOptimizations super.stdenv;
      })
    ];
  };
in pkgs.hello

You can use “overlays” or “crossOverlays”, but overlays requires you to rebuild the entire stdenv, while you probably don’t need that. On the other hand, this will make sure that everything is built with -march=native.

Other “adapters” are possible, see https://github.com/NixOS/nixpkgs/blob/c7b4b3bb1677d4b50251605a428016c6b5ac1d60/pkgs/stdenv/adapters.nix

We probably should add some more for things like this.

[OpenSourceAnarchist]

Thank you, I can try this on a new install.

Instead of adding more adapters, why not just create one where the user specifies the desired CFLAGS, CXXFLAGS, LDFLAGS, jobs (-j4 for ex.)? Maybe this is just a Gentoo thing, but I don’t know why these shouldn’t be easily specified by the user if desired (I understand not allowing -march=native for preserving purity but everything else like -O3 is host independent).

If I do use a stdenvAdapter, is setting gcc.arch/gcc.tune still necessary?

[repu]

Hello I’m having a problem with my NixOS 19.09
When specifying

nixpkgs.localSystem.system   = builtins.currentSystem;
  nixpkgs.localSystem.platform = pkgs.lib.systems.platforms.pc64 // {
    gcc.arch = "bdver2";
    gcc.tune = "bdver2";
  };

Also tried setting …system = “x86_64-linux” to no avail (nothing changed)
I can not rebuild system, and get errors:

$ nixos-rebuild test
error: infinite recursion encountered, at /nix/var/nix/profiles/per-user/root/channels/nixos/lib/modules.nix:336:9
(use '--show-trace' to show detailed location information)
building Nix...
error: infinite recursion encountered, at /nix/var/nix/profiles/per-user/root/channels/nixos/lib/modules.nix:336:9
(use '--show-trace' to show detailed location information)
building the system configuration...
error: infinite recursion encountered, at /nix/var/nix/profiles/per-user/root/channels/nixos/lib/modules.nix:336:9
(use '--show-trace' to show detailed location information)

Tried --show-trace but don’t get what broke from it.
Is this deprecated?

[volth]

@repu

Is this deprecated?

No, AMD support is not merged yet (https://github.com/NixOS/nixpkgs/pull/61019)

[repu]

Is there information about when it can be merged?
I would like to help if i can. What i could do?

[sepal]

Looks like localSystem.platform is not supported anywhere since 22.05 ( discussion on nixos.wiki.

The error message displayed on 23.05:

building Nix...
building the system configuration...
error:
       Failed assertions:
       - Your system configures nixpkgs with the platform parameter:
       nixpkgs.hostPlatform, with values defined in:
         - /etc/nixos/hardware-configuration.nix

       However, it also defines the legacy options:
       nixpkgs.system, with values defined in:
         - /nix/var/nix/profiles/per-user/root/channels/nixos/nixos/lib/eval-config.nix
       nixpkgs.localSystem, with values defined in:
         - /etc/nixos/configuration.nix

       For a future proof system configuration, we recommend to remove
       the legacy definitions.
(use '--show-trace' to show detailed location information)

Any idea how to target a particular CPU generation on recent NixOS releases ?