Gnu/stubs-64.h not found, cannot find -lgcc | Cross-compiled clang fork

Hello everyone,

I am in the process of creating a derivation for a fork of clang I am developing. My goal is to create a stdenv that allows me to compile any other derivation with my fork.

There is a caveat, though. My fork, for now, applies my custom passes only when compiling for x86 (32-bit) but should work anyway on 64-bit as a vanilla clang compiler.

I would like to compile it and run it on x86-64 hosts hence the choice to cross-compile it.

Now: the compilation of the compiler itself succeeds but whenever I try to compile any C source file, I encounter some errors.

For example, I will be using this sample C program as possible test program:

#include <stdio.h>

int main () {
  return 0;

When compiling for 32-bit hosts:

$ clang -m32 /tmp/test.c -v                                                                                                                                           
clang version 10.0.1 
Target: i386-unknown-linux-gnu
Thread model: posix
InstalledDir: /nix/store/837ld5rmbjs3ifxvbd5awh3yg6amccrl-ropfuscator-i686-unknown-linux-gnu-0.1.0/bin
Found CUDA installation: /nix/store/bjzj90r170ja4zhqips98d32kawi7pq3-cudatoolkit-10.2.89, version 10.1
 "/nix/store/837ld5rmbjs3ifxvbd5awh3yg6amccrl-ropfuscator-i686-unknown-linux-gnu-0.1.0/bin/clang-10" -cc1 -triple i386-unknown-linux-gnu -emit-obj -mrelax-all -disable-free -disable-llvm-verifier -discard-value-names -main-file-name test.c -mrelocation-model static -mthread-model posix -mframe-pointer=all -fmath-errno -fno-rounding-math -masm-verbose -mconstructor-aliases -target-cpu pentium4 -dwarf-column-info -fno-split-dwarf-inlining -debugger-tuning=gdb -v -resource-dir /nix/store/837ld5rmbjs3ifxvbd5awh3yg6amccrl-ropfuscator-i686-unknown-linux-gnu-0.1.0/lib/clang/10.0.1 -idirafter /nix/store/nd5vf7a5xahfia7vashv4aq7qp37ki24-glibc-2.33-56-dev/include -internal-isystem /usr/local/include -internal-isystem /nix/store/837ld5rmbjs3ifxvbd5awh3yg6amccrl-ropfuscator-i686-unknown-linux-gnu-0.1.0/lib/clang/10.0.1/include -internal-externc-isystem /include -internal-externc-isystem /usr/include -fdebug-compilation-dir /home/giulio/dev/ropfuscator/ropfuscator -ferror-limit 19 -fmessage-length 0 -fgnuc-version=4.2.1 -fobjc-runtime=gcc -fdiagnostics-show-option -fcolor-diagnostics -faddrsig -o /tmp/test-381307.o -x c /tmp/test.c
clang -cc1 version 10.0.1 based upon LLVM 10.0.1 default target x86_64-unknown-linux-gnu
ignoring nonexistent directory "/usr/local/include"
ignoring nonexistent directory "/include"
ignoring nonexistent directory "/usr/include"
#include "..." search starts here:
#include <...> search starts here:
End of search list.
 "/nix/store/vnh5nzh0jzic413i1q3xam0pvfx58sp4-ropfuscator-wrapper-0.1.0/bin/ld" --eh-frame-hdr -m elf_i386 -dynamic-linker /lib/ -o a.out /nix/store/0m62dg76bjqnm2m7y0m8fci9na92i03i-glibc-2.33-56/lib/crt1.o /nix/store/0m62dg76bjqnm2m7y0m8fci9na92i03i-glibc-2.33-56/lib/crti.o crtbegin.o -L/nix/store/0m62dg76bjqnm2m7y0m8fci9na92i03i-glibc-2.33-56/lib -L/nix/store/837ld5rmbjs3ifxvbd5awh3yg6amccrl-ropfuscator-i686-unknown-linux-gnu-0.1.0/lib -L/nix/store/837ld5rmbjs3ifxvbd5awh3yg6amccrl-ropfuscator-i686-unknown-linux-gnu-0.1.0/bin/../lib -dynamic-linker=/nix/store/0m62dg76bjqnm2m7y0m8fci9na92i03i-glibc-2.33-56/lib/ /tmp/test-381307.o -lgcc --as-needed -lgcc_s --no-as-needed -lc -lgcc --as-needed -lgcc_s --no-as-needed crtend.o /nix/store/0m62dg76bjqnm2m7y0m8fci9na92i03i-glibc-2.33-56/lib/crtn.o
/nix/store/h0ybx86850hgjqpln8pppydv62ca4kj3-binutils-2.35.2/bin/ld: cannot find crtbegin.o: No such file or directory
/nix/store/h0ybx86850hgjqpln8pppydv62ca4kj3-binutils-2.35.2/bin/ld: cannot find -lgcc
/nix/store/h0ybx86850hgjqpln8pppydv62ca4kj3-binutils-2.35.2/bin/ld: cannot find -lgcc
clang-10: error: linker command failed with exit code 1 (use -v to see invocation)

When compiling for 64-bit hosts:

$ clang /tmp/test.c
In file included from /tmp/test.c:1:
In file included from /nix/store/nd5vf7a5xahfia7vashv4aq7qp37ki24-glibc-2.33-56-dev/include/stdio.h:27:
In file included from /nix/store/nd5vf7a5xahfia7vashv4aq7qp37ki24-glibc-2.33-56-dev/include/bits/libc-header-start.h:33:
In file included from /nix/store/nd5vf7a5xahfia7vashv4aq7qp37ki24-glibc-2.33-56-dev/include/features.h:497:
/nix/store/nd5vf7a5xahfia7vashv4aq7qp37ki24-glibc-2.33-56-dev/include/gnu/stubs.h:10:11: fatal error: 'gnu/stubs-64.h' file not found
# include <gnu/stubs-64.h>
1 error generated.

Here is the derivation I use:

{ pkgs, llvm, clang, lib, fmt, tinytoml }:
  pkgs32 = pkgs.pkgsi686Linux;
  python-deps = python-packages: with python-packages; [ pygments ];
  python = pkgs.python3.withPackages python-deps;
  ccache_path = "/nix/var/cache/ccache";

  derivation_function = { stdenv, llvmPackages_13, cmake, git, curl, pkg-config
    , z3, libxml2, ninja, ccache, glibc_multi, use_ccache ? false, debug ? false
    stdenv.mkDerivation {
      pname = "ropfuscator";
      version = "0.1.0";
      enableParallelBuilding = true;
      nativeBuildInputs =
        [ cmake git curl pkg-config ninja llvmPackages_13.bintools glibc_multi ]
        ++ lib.optional (use_ccache == true) [ ccache ];
      buildInputs = [ libxml2 python glibc_multi ];
      srcs = [ ./cmake ./src ./thirdparty ];
      patches = [ ./patches/ropfuscator_pass.patch ];
      postPatch = "patchShebangs .";
      dontStrip = debug;

      cmakeFlags = [
      ] ++ lib.optional (debug == true) [
      ] ++ lib.optional (use_ccache == true) [ "-DLLVM_CCACHE_BUILD=On" ];

      CCACHE_DIR = ccache_path;

      unpackPhase = ''
        runHook preUnpack

        cp --no-preserve=mode,ownership -r ${llvm}/* .

        # insert clang
        pushd tools
          mkdir clang
          cp --no-preserve=mode,ownership -r ${clang}/* clang

        # insert ropfuscator
        pushd lib/Target/X86
          mkdir ropfuscator
          for s in $srcs; do
            # strip hashes
            cp --no-preserve=mode,ownership -r $s ropfuscator/`echo $s | cut -d "-" -f 2`
          # manually copy submodules due to nix currently not having
          # proper support for submodules
          pushd ropfuscator/thirdparty
            mkdir -p {tinytoml,fmt}
            cp --no-preserve=mode,ownership -r ${tinytoml}/* tinytoml
            cp --no-preserve=mode,ownership -r ${fmt}/* fmt

        runHook postUnpack

      buildPhase = ''
        runHook preBuild

        cmake --build . -- clang

        runHook postBuild
in rec {
  ropfuscator-unwrapped =
    pkgs.pkgsCross.gnu32.callPackage derivation_function { };

  # release
  ropfuscator = pkgs32.wrapCCWith { cc = ropfuscator-unwrapped; };
  ropfuscatorCcache = pkgs32.wrapCCWith {
    cc = ropfuscator-unwrapped.override { use_ccache = true; };

  # debug
  ropfuscatorDebug = pkgs32.wrapCCWith {
    cc = ropfuscator-unwrapped.override { debug = true; };
  ropfuscatorCcacheDebug = pkgs32.wrapCCWith {
    cc = ropfuscator-unwrapped.override {
      debug = true;
      use_ccache = true;

  # stdenvs
  stdenv = pkgs32.overrideCC pkgs32.stdenv ropfuscator;
  stdenvCcache = pkgs32.overrideCC pkgs32.stdenv ropfuscatorCcache;
  stdenvDebug = pkgs32.overrideCC pkgs32.stdenv ropfuscatorDebug;
  stdenvCcacheDebug = pkgs32.overrideCC pkgs32.stdenv ropfuscatorCcacheDebug;

Any help would be appreciated! Any ideas on what I am doing wrong?

Thank you!

Looks familiar to


this is terrible because the gcc binutils wrapper will add things like -lgcc to LD_FLAGS which makes linking impossible as there isn’t a corresponding library available.

Thank you for the input.

I am trying to use wrapCCWith and adding LLVM’s bintools to no avail, unfortunately.

  ropfuscator = pkgs32.wrapCCWith { cc = ropfuscator-unwrapped; bintools = pkgs.llvmPackages_10.bintools; };

This is what I get when I invoke clang:

/nix/store/0jqjwk0d4fhfkypcc8vkwkgnj5x6g8hw-llvm-binutils-wrapper-10.0.1/nix-support/ line 29: NIX_DYNAMIC_LINKER_x86_64_unknown_linux_gnu: unbound variable