How to recompile a package with flags?

Help
1

I found this on stackoverflow and it is exactly what I want but I don’t understand how to implement it or use it.

I am using the nixos/nix docker image.

What files do I put these functions in?
What commands do I run to compile that package (say openssl)?
How do I then install it?

2

Does anyone know how this works?

3

I predict lots of us know, it’s just not very clear what you want to do exactly.

I can show you how to build/install patched openssl adhoc or how to do that through overlay. Which do you want? Which flags do you want to override?

4

I would like to do it through the overlay to be programmatic about it.

I want to be able to list packages such as openssl glibc that I want to compile with the flags listed in the stackoverflow post.

“-O3” “-march=native” “-fPIC”

I sort of understand the clips provided in the post, but I don’t know where to put them and what commands to run to build and install the results.

5 

self: super: {
  openssl_old = super.openssl;

  #openssl = super.openssl.overrideAttrs (old: {
  #  NIX_CFLAGS_COMPILE = "-march=native";
  #});

  openssl = super.openssl.override {
     stdenv = super.stdenvAdapters.impureUseNativeOptimizations super.stdenv;
  };
}

here are two examples how you can achieve it.

Build/install just like you do that for openssl - nix-env -iA nixpkgs.openssl (only if you place overlay in correct place - ~/.config/nixpkgs/overlays/)

6

This didn’t work for me. I put your snippet in:

~/.config/nixpkgs/overlays/openssl.nix

I ran:

nix-env -iA nixpkgs.openssl

and I got this:

$ nix-env -iA nixpkgs.openssl_1_1
replacing old 'openssl-1.1.1c'
installing 'openssl-1.1.1c'
these paths will be fetched (40.01 MiB download, 146.30 MiB unpacked):
  /nix/store/8ziw0r9hx4any7hd30a5r3wjcb3skzaf-openssl-1.1.1c-debug
  /nix/store/bniand9afisrgrsfi7kr093334iv3ibv-glibc-2.27-dev
  /nix/store/fh7ghpk8k7lppy6y1qk7vcgikymbbiw0-openssl-1.1.1c-dev
  /nix/store/kd87dwlhl0cpz09hkrg5z52klw39ipvx-linux-headers-4.19.16
  /nix/store/lcpf5mzc4f1scj67znarbd6kg64lykls-glibc-2.27-bin
  /nix/store/w3wqang215is14xqajycbxmd52b44qkw-zlib-1.2.11
  /nix/store/wr5dlcw1asw8dwgm4bwjmga8f52m5lfx-gcc-7.4.0
  /nix/store/x7vzi9s698q31vikcxxjcrrmppm5mh11-openssl-1.1.1c-doc
copying path '/nix/store/x7vzi9s698q31vikcxxjcrrmppm5mh11-openssl-1.1.1c-doc' from 'https://cache.nixos.org'...
copying path '/nix/store/lcpf5mzc4f1scj67znarbd6kg64lykls-glibc-2.27-bin' from 'https://cache.nixos.org'...
copying path '/nix/store/kd87dwlhl0cpz09hkrg5z52klw39ipvx-linux-headers-4.19.16' from 'https://cache.nixos.org'...
copying path '/nix/store/fh7ghpk8k7lppy6y1qk7vcgikymbbiw0-openssl-1.1.1c-dev' from 'https://cache.nixos.org'...
copying path '/nix/store/bniand9afisrgrsfi7kr093334iv3ibv-glibc-2.27-dev' from 'https://cache.nixos.org'...
copying path '/nix/store/w3wqang215is14xqajycbxmd52b44qkw-zlib-1.2.11' from 'https://cache.nixos.org'...
copying path '/nix/store/wr5dlcw1asw8dwgm4bwjmga8f52m5lfx-gcc-7.4.0' from 'https://cache.nixos.org'...
copying path '/nix/store/8ziw0r9hx4any7hd30a5r3wjcb3skzaf-openssl-1.1.1c-debug' from 'https://cache.nixos.org'...
building '/nix/store/nsi6j6pahbq3xrlzai5qkfwljl8b16x0-user-environment.drv'...
created 192 symlinks in user environment

It didn’t compile the package.

Ideally, I am trying to understand the post from StackOverflow though. So let me copy it here for clarity:

I managed to write a function which I can apply to the packages I want to compile with custom flags:

  optimizeWithFlags = pkg: flags:
    pkgs.lib.overrideDerivation pkg (old:
    let
      newflags = pkgs.lib.foldl' (acc: x: "${acc} ${x}") "" flags;
      oldflags = if (pkgs.lib.hasAttr "NIX_CFLAGS_COMPILE" old)
        then "${old.NIX_CFLAGS_COMPILE}"
        else "";
    in
    {
      NIX_CFLAGS_COMPILE = "${oldflags} ${newflags}";
    });
This function takes a package and a list of strings (which are the flags) and builds a new package with the existing one, but with the additional compiler flags.

I want to use this function.

What file do I put it in?

For convenience, I wrote another set of helper functions:

  optimizeForThisHost = pkg:
    optimizeWithFlags pkg [ "-O3" "-march=native" "-fPIC" ];

  withDebuggingCompiled = pkg:
    optimizeWithFlags pkg [ "-DDEBUG" ];

I want to use this function as well. Do I put it in the same file as the one above?

Now here is the actual override:

Now, I can override packages (here mutt and dmenu):

  muttWithoutThings = pkgs: (pkgs.mutt.override {
    sslSupport  = false;
    saslSupport = false;
    imapSupport = false;
    withSidebar = false;
  });

  mutt = pkgs:
    (utils pkgs).withoutConfigureFlag "--enable-debug"
    ((utils pkgs).optimizeForThisHost (muttWithoutThings pkgs)
    );

  dmenu = pkgs:
    (utils pkgs).optimizeForThisHost
    (pkgs.dmenu.override {
      patches = [ ./patches/dmenu-fuzzymatch-4.6.diff ];
    });

Here I would replace the mutt and dmenu overrides with openssl and glibc in my case.

This would go into ~/.config/nixpkgs/overlays/overrides.nix ?

Then in the last comment of the post:

In the above utils is utils = pkgs: import ./util.nix { pkgs = pkgs; }; and the util.nix file returns a function which spits out a set of functions.

Are the first two functions placed in a file called utils.nix?
Where does this file go?
It seems they are importing the utils.nix to be used in the overrides?

7

Well, you said you run

nix-env -iA nixpkgs.openssl

but the log says you run

nix-env -iA nixpkgs.openssl_1_1

These are different things. You should create overrides in overlays for openssl_1_1 as well.

I want to use this function.
What file do I put it in?

You can put it into overlay! It will look like:

self: super: {
  optimizeWithFlags = pkg: flags:
    pkg.overrideAttrs (old: {
      NIX_CFLAGS_COMPILE = [ (old.NIX_CFLAGS_COMPILE or "") ] ++ flags;
    });

  optimizeForThisHost = pkg:
    self.optimizeWithFlags pkg [ "-O3" "-march=native" "-fPIC" ];

  openssl = self.optimizeForThisHost super.openssl;
}

I’ve modernized the optimizeWithFlags function a bit. I’ve also tested this in Docker nixos/nix. See also https://github.com/NixOS/nixpkgs/blob/47b551c6a854a049045455f8ab1b8750e7b00625/pkgs/stdenv/adapters.nix#L203

The utils method is much more complicated compared to this. They create a custom file with functions, utils.nix, which looks like:

pkgs: rec {
  optimizeWithFlags = ,,,;

  optimizeForThisHost = pkg:
    optimizeWithFlags pkg [ "-O3" "-march=native" "-fPIC" ];

  withDebuggingCompiled = pkg:
    optimizeWithFlags pkg [ "-DDEBUG" ];
}

and do the overrides in config.nix (NOT in overlay), in packageOverrides attribute. And yes, there is utils = import ./utils.nix; somewhere in the code, but if you use the mine overlay, it is no longer needed.

1 Like