Building packages with CFLAGS Gentoo-style

Help
1

I’d like to be able to recompile all packages from source with specific flags (-march=native -mtune=znver1 -O3, etc).

I’ve tried to override stdenv in configuration.nix, but I haven’t been able to get it to pick up the flags.

Has anyone done this and can guide me through the process?

2 Likes
2

How have you overriden stdenv?

The easiest way is probably doing it in an overlay loaded through nixpkgs.overlays.

2 Likes
3

I’ve tried to do it the following ways:

#1:

  nixpkgs = {
    overlays = [
      (self: super: {
        stdenv = super.impureUseNativeOptimizations super.stdenv;
      })
    ];
  };

#2:

let
  pkgs = import <nixpkgs> {
    crossOverlays = [
      (self: super: {
        stdenv = super.stdenvAdapters.impureUseNativeOptimizations super.stdenv;
      })
    ];
  };
in
[...]

#3:

stdenv = stdenv: stdenv //
    { mkDerivation = args: stdenv.mkDerivation (args // {
        NIX_CFLAGS_COMPILE = toString (args.NIX_CFLAGS_COMPILE or "") + " -O3 -march=native -mtune=znver1 -ftree-vectorize -pipe -fstack-protector-strong -fno-plt";
        NIX_CXXFLAGS_COMPILE = toString (args.NIX_CFXXLAGS_COMPILE or "") + " -O3 -march=native -mtune=znver1 -ftree-vectorize-pipe -fstack-protector-strong -fno-plt";
        NIX_ENFORCE_NO_NATIVE = false;
        preferLocalBuild = true;
        allowSubstitutes = false;
      });
    };

But none seems to work. The compiler still builds the packages with default flags and no optimizations for the target/host platform.

2 Likes
4

Hello.

I have just discovered NixOS. As a Gentoo user I really would like to globally declare the optimization flags.

I tried reading the Nix manuals to search for a solution. Unfortunaly they are so large, but still software optimization is suspiciously missing from them. I don’t have the time nor skills to reverse engineer the sources to achieve this goal.

Is there a hope for me? Could I ask for help on how to get what I want?

5

this may help you. Optimization levels

6

Thank you for replying.

Unfortunately, for someone new, like me, this is useless in current form.
Do I paste it directly into the configuration file?
Also, what about other languages? I am thinking of Gentoo’s CXXFLAGS and similar other counterparts. I hope NIX_CFLAGS_COMPILE is simply injected everywhere and the “C” part is just a misunderstanding.

I did try a leap of faith by simply entering the expression from the other topic, but it fails. This is what I have in configuration.nix:

{config, pkgs, ... }:

{
  imports = [ the other configuration files ];

  self: super: {
    stdenv = super.stdenvAdapters,addAttrsToDerivation {
      NIX_CFLAGS_COMPILE = "my flags";
      NIX_LDFLAGS = "";
    } super.stdenv;
  };

  more;
  expressions;
}

The error message is a syntax error, unexpected ‘:’ after the self keyword.

Is it because I am missing something in the first line? This is my current gut feeling.

7

That expression is an overlay so you would add it as an item to nixpkgs.overlays option:

{config, pkgs, ... }:

{
  imports = [ the other configuration files ];

  nixpkgs.overlays = [
    (final: prev: {
      stdenv = prev.stdenvAdapters.addAttrsToDerivation {
        NIX_CFLAGS_COMPILE = "my flags";
        NIX_LDFLAGS = "";
      } prev.stdenv;
    })
  ];

  …
}
3 Likes
8

Thank you.

If I were to move it into a separate file, would I have to open it with a { pkgs, ... }: pattern? I am still not familiar with how function patterns work.

EDIT:
I am happy to report, that gcc just complained about wrong znver2 being a wrong march. This means, that the expression seems to be correct.

9

Yes, you would write the content of the parenthesis in your overlay file:

overlay.nix:

final: prev:

{
  stdenv = ...;
}

and import it from your configuration:

configuration.nix:

{ config, ... }:

{
  ...
   nixpkgs.overlays = [
    (import ./overlay.nix)
   ];
   ...
}
1 Like
10

Excellent.

I am deeply grateful. I am now going to equip all of my machines with NixOS.

11

The drawback of this method is that it overrides the mkDerivation attributes that exist in packages already. And many nixpkgs packages do in fact utilize NIX_CFLAGS_COMPILE and NIX_LDFLAGS.

The definition is in pkgs/stdenv/adapters.nix:

addAttrsToDerivation = extraAttrs: stdenv: stdenv.override (old: {
  mkDerivationFromStdenv = extendMkDerivationArgs old (_: extraAttrs);
});

It doesn’t allow merging with previous attribute value.

I know there is a tricky alternative that modifies the compiler wrapper (as opposed to packages):

wrapCC = cc: prev.wrapCCWith {
  inherit cc;
  extraBuildCommands = ''
    echo "my cflags" >> "$out/nix-support/cc-cflags"
  '';
};

But it isn’t elegant as you cannot turn such flags off for individual packages or even in nix-shell. They become part of the compiler wrapper itself.
Does anyone know a better method?

12

I think it might be useful to deprecate addAttrsToDerivation and create overrideDerivationAttrs adapter that would accept a function.

1 Like
13

Did you ever figure it out? I am having the same issue - I want to apply global linker flags to all packages through an overlay.

For example: LDFLAGS=“-rtlib=compiler-rt -unwindlib=libunwind -stdlib=libc++ -lc++abi -Oz -flto=thin -fsanitize=cfi -fvisibility=hidden” with a full llvm toolchain.

14

https://www.shlomifish.org/humour/by-others/funroll-loops/Gentoo-is-Rice.html

3 Likes
15

I guess Package request: `llvmPackages.stdenv` should be an actual Clang/LLVM toolchain · Issue #277564 · NixOS/nixpkgs · GitHub would partially solve my use case of removing the need for gcc and libstdc++, however I still cannot find a good way to global compiler flags.

16

Maybe the withCflags adapter in combination with a global stdenv override could be helpful:

The adapters.nix file might also have some more pointers on how to customize other parts of the toolchain.