Targeting particular x86_64 generation (haswell/ivybridge/skylake/...)

That’s already doable via supportedFeatures on machines and meta.requiredSystemFeatures on derivations, though it’s not “standardized” for this use case ATM. Well, so far we seemed to be mostly OK with generic binaries like all larger distros (AFAIK, discounting source-focused ones like Gentoo), at least for the official NixPkgs; IMHO it’s not too difficult for users to override some “leaf” packages for this purpose and keep rebuilding them.

Yes, it would be perfect, assuming there is only ordered gcc.arch.

Although it won’t cover well the cases like Thinkpad X220’s

gcc = {
  arch = "sandybridge";
  extraFlags = ["-mavx"]; # -march=sandybridge alone does not enable AVX
};

I don’t think it’s worth aiming for 100% accuracy, more like meaningfully splitting the “range” between plain x86_64 and new CPUs; just creating a few groups will probably be enough to get almost all the effect. Actually feature flags like -mavx feel more significant for this. The -mtune part of -march would be more relevant for old machines, I think, as the default/generic optimizer in GCC shifts in time towards the commonly used CPU types.

Is there a way to set things like gcc = { arch = ... } system wide?

in /etc/nix/configuration.nix

  nixpkgs.localSystem.platform  = lib.systems.platforms.pc64 // {
    gcc.arch = "skylake";
    gcc.tune = "skylake";
  };
1 Like

I just hope function multiversioning would just become fully automatic one day.

I get an error:

error: undefined variable 'lib' at /etc/nixos/configuration.nix

I also tried:

  nixpkgs.localSystem.platform  = pkgs.lib.systems.platforms.pc64 // {

without a success:

error: attribute 'system' missing, at <nixpkgs>/lib/systems/default.nix:21:81

Do you have any complete working /etc/nixos/configuration.nix example?

then

  nixpkgs.localSystem.system   = "x86_64-linux"; # or =builtins.currentSystem
  nixpkgs.localSystem.platform = pkgs.lib.systems.platforms.pc64 // {
    gcc.arch = "skylake";
    gcc.tune = "skylake";
  };

No, I am not using /etc/nixos/configuration.nix, I use fork of nixops even to manage localhost as though it is one of the remote machines.

How did you do that? I’d love to manage my machine with nixops! To be precise, I’d love to know how to use the fork of nixops you are using😀

No, it was not designed as a detachable product so it has many hardcoded values and other assumption (for example, about a Tinc network connecting all the devices) on my infrastructure.

I think it should be easy to add this feature to mainline nixops (if not done yet), it is just about producing a system closure and calling switch-to-configuration.pl

@kamilchm Does it work?

@volth If I disable access to the binary cache and add

 nixpkgs.localSystem.platform = pkgs.lib.systems.platforms.pc64 // {
    gcc.arch = "skylake";
    gcc.tune = "skylake";
  };

to my /etc/nixos/configuration.nix, will that cause every package to be rebuilt on my computer with these optimized settings? If not, how can this be achieved? Ideally, I want to make NixOS as Gentoo-like as possible and build every package on my system with “-O3 -march=native …”

I know this breaks reproducibility/purity but packages will only ever be on my system so I’m fine with that. Is this even possible?

Yes.
And you do not have to disable access to the binary cache, the packages just have other hashes.
So it does not even break purity :slight_smile:

1 Like

@volth Thank you so much! Learning to use NixOS is a fun exercise (that I’m falling in love with).

I’m still stumped on overriding default makeopts like CFLAGS, CXXFLAGS, etc. With Gentoo it is so easy… I found this in the nixpkgs manual:

makeFlagsArray

A shell array containing additional arguments passed to make . You must use this instead of makeFlags if the arguments contain spaces, e.g.

preBuild = 
'' makeFlagsArray+=(CFLAGS="-O0 -g" LDFLAGS="-lfoo -lbar")
 '';

Note that shell arrays cannot be passed through environment variables, so you cannot set makeFlagsArray in a derivation attribute (because those are passed through environment variables): you have to define them in shell code.

So how exactly can I set these for every package? It feels like this should be as easy as changing gcc.arch or gcc.tune!

Edit: Could I do it like this (copied from a Nix cheatsheet site)?
stdenv.userHook = '' NIX_CFLAGS_COMPILE+=" -march=native" '';

Or is something like this required and an overridden stdenv?

Probably, you cannot without patching nixpkgs.

In the initial proposal there was gcc.extraFlags (akin to gcc.arch) (https://github.com/NixOS/nixpkgs/search?q=gcc.extraFlags&type=Issues) but it did not get to the mainline

cc @matthewbauer

I think the best way to do this now is like this:

let
  pkgs = import <nixpkgs> {
    crossOverlays = [
      (self: super: {
        stdenv = super.stdenvAdapters.impureUseNativeOptimizations super.stdenv;
      })
    ];
  };
in pkgs.hello

You can use “overlays” or “crossOverlays”, but overlays requires you to rebuild the entire stdenv, while you probably don’t need that. On the other hand, this will make sure that everything is built with -march=native.

Other “adapters” are possible, see https://github.com/NixOS/nixpkgs/blob/c7b4b3bb1677d4b50251605a428016c6b5ac1d60/pkgs/stdenv/adapters.nix

We probably should add some more for things like this.

1 Like

Thank you, I can try this on a new install.

Instead of adding more adapters, why not just create one where the user specifies the desired CFLAGS, CXXFLAGS, LDFLAGS, jobs (-j4 for ex.)? Maybe this is just a Gentoo thing, but I don’t know why these shouldn’t be easily specified by the user if desired (I understand not allowing -march=native for preserving purity but everything else like -O3 is host independent).

If I do use a stdenvAdapter, is setting gcc.arch/gcc.tune still necessary?

Hello I’m having a problem with my NixOS 19.09
When specifying

nixpkgs.localSystem.system   = builtins.currentSystem;
  nixpkgs.localSystem.platform = pkgs.lib.systems.platforms.pc64 // {
    gcc.arch = "bdver2";
    gcc.tune = "bdver2";
  };

Also tried setting …system = “x86_64-linux” to no avail (nothing changed)
I can not rebuild system, and get errors:

$ nixos-rebuild test
error: infinite recursion encountered, at /nix/var/nix/profiles/per-user/root/channels/nixos/lib/modules.nix:336:9
(use '--show-trace' to show detailed location information)
building Nix...
error: infinite recursion encountered, at /nix/var/nix/profiles/per-user/root/channels/nixos/lib/modules.nix:336:9
(use '--show-trace' to show detailed location information)
building the system configuration...
error: infinite recursion encountered, at /nix/var/nix/profiles/per-user/root/channels/nixos/lib/modules.nix:336:9
(use '--show-trace' to show detailed location information)

Tried --show-trace but don’t get what broke from it.
Is this deprecated?

No, AMD support is not merged yet (https://github.com/NixOS/nixpkgs/pull/61019)

Is there information about when it can be merged?
I would like to help if i can. What i could do?