How to build libraries with custom compilers like Intel OneAPI or AMD aocc

Help
1

I’m trying to create a Nix shell with libraries built by the AMD AOCC compiler and the Intel OneAPI compiler. Both of these are modified LLVM compilers.

For example, AMD provides a .deb package for the AOCC compiler. On FHS distributions, after installation, one must execute source /opt/AMD/aocc-compiler-5.0.0/setenv_AOCC.sh to add the AOCC compilers to their PATH. The binaries for AOCC are named clang, clang++, and so on.

I currently have an aocc.nix file that creates a package for AOCC. When I use callPackage ./aocc.nix in an FHS environment, I can access files under /opt/AMD/aocc-compiler-5.0.0/.

Here’s the aocc.nix file:

{ stdenv
, fetchurl
, autoPatchelfHook
, dpkg
, libgcc
, libgccjit
, rocmPackages
, elfutils
, libffi
}:

stdenv.mkDerivation rec {
  pname = "aocc";
  version = "5.0.0";
  revision = "1";
  src = fetchurl {
    url = "https://download.amd.com/developer/eula/aocc/aocc-5-0/aocc-compiler-${version}_${revision}_amd64.deb";
    hash = "sha256-uTez8Z9ZrJAaLDRmqAmI4FRdU4J5AOqls8GtDNn98Mg=";
  };
  nativeBuildInputs = [
    autoPatchelfHook
    dpkg
    stdenv
    libgcc
    libgccjit
    rocmPackages.rocm-runtime
    elfutils
    libffi
  ];
  unpackPhase = "dpkg-deb -x $src unpack";
  preFixup = ''
    for lib in $out/opt/AMD/aocc-compiler-5.0.0/lib/*.so*; do
      patchelf --replace-needed libffi.so.6 libffi.so.8 "$lib" || true
    done
  '';
  installPhase =
    ''
      mkdir -p $out/opt
      cp -r unpack/opt/* $out/opt/
    '';
}

I’ve also created a flake.nix to set up a development shell via nix develop .#gcc, which works well since all libraries can be downloaded from Nix’s cache.

Here’s the flake.nix file:

{
  description = "Development environment for MPI and HDF5 applications";

  inputs = {
    nixpkgs.url = "github:nixos/nixpkgs/nixos-unstable";
  };

  outputs = { self, nixpkgs }:
    let
      pkgs = import nixpkgs {
        system = "x86_64-linux";
      };

      commonEnv = with pkgs; [
        bc
        python313
        perl
        cmake
        openssh
        file
      ];

      commonDependencies = with pkgs; [
        libz
        libxml2
        libfyaml
        lapack
        blas
      ];

      # aoccDependencies = ?

      commonShellHook = ''
        export LD_LIBRARY_PATH=/usr/lib
        export LIBRARY_PATH=/usr/lib
        export CPLUS_INCLUDE_PATH=/usr/include
        export CMAKE_LIBRARY_PATH=/usr/lib
        export CMAKE_INCLUDE_PATH=/usr/include
        export NETLIB_LAPACK_ROOT='${pkgs.lapack}'
        export ZLIB_ROOT='${pkgs.libz}'
        export XML2_ROOT='${pkgs.libxml2.dev}'
        export FYAML_ROOT='${pkgs.libfyaml.dev}'
      '';

    # aoccShellHook = ?

    in
    {
      devShells.x86_64-linux.gcc = pkgs.buildFHSUserEnv {
        name = "dev-env-gcc";
        nativeBuildInputs = commonEnv ++ commonDependencies ++ [ pkgs.gcc ];
        shellHook = commonShellHook;
      };

      # devShells.x86_64-linux.aocc = pkgs.buildFHSUserEnv {
      #   name = "dev-env-aocc";
      #   nativeBuildInputs = commonEnv ++ aoccDependencies ++ [ aocc ];
      #   shellHook = aoccShellHook;
      # };
    };
}

Now, I want to create another environment with all packages built by the AOCC compilers, which I believe is very similar to Clang. I wonder what steps I should take to achieve this goal.

I’ve tried using config.replaceStdenv and overrides on those libraries, but I haven’t succeeded in getting a working Nix script, because AOCC requires sourcing a specific .sh file to enable it, the same applies to Intel OneAPI compilers.

I’m new to Nix programming and using NixOS. Could you please provide some guidance on how to proceed?

Thank you very much!

1 Like
2

Haven’t tried this yet, but read recently about changing compilers in the dev shell: Using Nix to Fuzz Test a PDF Parser (Part One) · mtlynch.io either in this one or in part two.

Disclaimer: it’s not my blog or anything.

1 Like
3

Using config.replaceStdenv or $pkg.override { stdenv = $aoccStdEnv; } might work but you need to define this aoccStdEnv with working aocc compilers.

I have done this years ago in nur-packages/overlays/aocc-overlay/default.nix at 708b38fe80fcd8a2529316f5cd1edf5c163209b7 · dguibert/nur-packages · GitHub.

There are also overlays for other compilers: intel-compilers, nvhpc, …
Some of some need to be updated to support newer versions or newer installer.

Feel free to give a try (and made a PR).

3 Likes
5

Thank you very much! Your work is very helpful to me :wink:

6

I have a flake like this, and it gives me error: attribute 'libllvm' missing, and i don’t know is this the right way to do so or not. Could you please give me more hints :face_holding_back_tears:

{
  inputs = {
    nixpkgs.url = "github:nixos/nixpkgs/nixos-unstable";
  };

  outputs = { self, nixpkgs }:
    let
      pkgs = import nixpkgs {
        system = "x86_64-linux";
      };
      aoccStdenv = pkgs.clangStdenv.override {
        cc = pkgs.clangStdenv.cc.override {
          cc = pkgs.callPackage ./aocc { };
        };
      };
      aoccpkgs = import nixpkgs {
        system = "x86_64-linux";
        config.replaceStdenv = { pkgs, ... }: aoccStdenv;
      };
    in
    rec {
      commonDependencies = with aoccpkgs; [
        libz
        # ...
      ];
      devShells.x86_64-linux.aocc = aoccpkgs.buildFHSUserEnv {
        name = "dev-env-aocc";
        nativeBuildInputs = commonDependencies;
      };
    };
}

error:
       … while calling the 'derivationStrict' builtin
         at <nix/derivation-internal.nix>:34:12:
           33|
           34|   strict = derivationStrict drvAttrs;
             |            ^
           35|

       … while evaluating derivation 'dev-env-aocc'
         whose name attribute is located at /nix/store/lsy6c2f9alj2gkjj36h754kk63x6701l-source/pkgs/stdenv/generic/make-derivation.nix:336:7

       … while evaluating attribute 'buildCommand' of derivation 'dev-env-aocc'
         at /nix/store/lsy6c2f9alj2gkjj36h754kk63x6701l-source/pkgs/build-support/trivial-builders/default.nix:59:17:
           58|         enableParallelBuilding = true;
           59|         inherit buildCommand name;
             |                 ^
           60|         passAsFile = [ "buildCommand" ]

       (stack trace truncated; use '--show-trace' to show the full, detailed trace)

       error: attribute 'libllvm' missing
       at /nix/store/lsy6c2f9alj2gkjj36h754kk63x6701l-source/pkgs/development/interpreters/python/cpython/default.nix:163:5:
          162|   ] ++ optionals (stdenv.cc.isClang && (!stdenv.hostPlatform.useAndroidPrebuilt or false) && (enableLTO || enableOptimizations)) [
          163|     stdenv.cc.cc.libllvm.out
             |     ^
          164|   ];
7

See my updated branch: GitHub - dguibert/nur-packages at dg/aocc
aocc 3.2.0 and 5.0.0 are working there.

To use the overlay, look at: nur-packages/checks/aocc.nix at dg/aocc · dguibert/nur-packages · GitHub

2 Likes
9

:+1: Thanks a lot for helping me, Iill learn more about nix overlay and test it :wink: