How to modify cmakeFlagsin llvmPackages.libunwind in a cross-compile nixpkgs so that all other packages that rely on it are using this overlayed version?
I have a flake for cross compiling to armv6m-none-eabi with clang, but libunwind fails to build (I think it is a dependency of libcxx which is a dependency of clang). I have already determined the flags to pass to libunwind to make it compile in (libunwindClangFlags). Note that I also need to set doFakeLibgcc to false llvmPackages.libunwind: condition doFakeLibgcc on !stdenv.hostPlatform.isStatic by pwaller · Pull Request #417354 · NixOS/nixpkgs
{
description = "A very basic flake";
inputs = {
nixpkgs.url = "github:nixos/nixpkgs?ref=nixos-unstable";
utils.url = "github:numtide/flake-utils";
};
outputs = { self, nixpkgs, utils }: utils.lib.eachDefaultSystem (system:
let
libunwindCmakeFlags = [
(pkgsCross.lib.cmakeBool "LIBUNWIND_IS_BAREMETAL" true)
(pkgsCross.lib.cmakeBool "LIBUNWIND_ENABLE_THREADS" false)
];
pkgsCross = import nixpkgs {
system = system;
crossSystem = {
config = "armv6m-none-eabi";
libc = "newlib-nano";
useLLVM = true;
isStatic = true;
gcc = {
cpu = "cortex-m0plus";
float-abi = "soft";
arch = "armv6m";
thumb = true;
};
};
config = {
allowUnsupportedSystem = true;
};
crossOverlays = [
];
overlays = [
(self: super: {
newlib-nano = super.newlib-nano.overrideAttrs (old: {
configureFlags = (old.configureFlags or []) ++ [
"--disable-multilib"
"--disable-libgloss"
];
});
})
];
};
in {
packages.pkgs = pkgsCross;
});
}
I was able to override newlib-nano.configureFlags so that it compiles, but when I try to override llvmPackages.libunwind.cmakeFlags it either doesn’t take or only applies to some instances of libunwind but other instances that are used by other packages stay unchanged and the build fails. I think I am running into the issue because there are multiple libunwind’s; llvmPackages_xx.libunwind ,llvmPackages_xx.libraries.libunwind,buildPackages.llvmPackages_xx.libunwind, … and llvmPackages uses targetPackages and targetLlvmLibraries in its derivation and I don’t know exactly how I should be overriding to cascade to all other instances.
Some, but not all, of the things I have tried, in both crossOverlays and overlays:
# does nothing?
llvmPackages = super.llvmPackages.override (prev: {
targetLlvmLibraries = prev.targetPackages.llvmPackages.libraries.extend (super: {
libunwind = super.libunwind.override (_: {
devExtraCmakeFlags = libunwindCmakeFlags;
});
});
});
# overrides llvmPackages.libunwind but not llvmPackages.libraries.libunwind
# and doesn't propogate to packages requiring it
llvmPackages = super.llvmPackages // {
libunwind = super.llvmPackages.libunwind // {
cmakeFlags = libunwindCmakeFlags;
};
};
# overrides llvmPackages.libraries.libunwind and propogrates so some packages maybe?
# `nix build .#pkgs.stdenv.cc --dry-run` shows multiple different libunwinds
llvmPackages = super.llvmPackages // {
libraries = super.llvmPackages.libraries // {
libunwind = super.llvmPackages.libraries.libunwind // {
cmakeFlags = super.llvmPackages.libraries.libunwind.cmakeFlags ++ libunwindCmakeFlags;
};
};
};
Any help on figuring out the proper incantation would be appreciated.
Also, what tools are available to help with tracking down the proper package/override? I have been relying on a mix of nix eval .#pkgs.<package> and nix build .#pkgs.<package> --dry-run to compare hashes and see if anything is actually changing.
Link to relevant nixpkgs: nixpkgs/pkgs/development/compilers/llvm at 0147c2f1d54b30b5dd6d4a8c8542e8d7edf93b5d · NixOS/nixpkgs
Hi, I actually have to do this to get a buildable LLVM kernel on aarch64-linux. If you look here, you can see that overriding libraries as well as the toplevel attribute of what you want is necessary.
However, this isn’t an ideal thing and the LLVM team (includes myself) are looking to make LLVM more override friendly. This PR would greatly make what you’re doing much easier. pkgs/development/compilers/llvm: add replacement entrypoint using mak… by RossComputerGuy · Pull Request #436350 · NixOS/nixpkgs · GitHub
1 Like
Thank you for responding! And I want to thank you for all of the work you do!
I modified my flake to:
outputs = { self, nixpkgs, utils }: utils.lib.eachDefaultSystem (system:
let
libunwindCmakeFlags = [
(pkgsCross.lib.cmakeBool "LIBUNWIND_IS_BAREMETAL" true)
(pkgsCross.lib.cmakeBool "LIBUNWIND_ENABLE_THREADS" false)
];
pkgsCross = import nixpkgs {
system = system;
crossSystem = {
config = "armv6m-none-eabi";
libc = "newlib-nano";
useLLVM = true;
isStatic = true;
gcc = {
cpu = "cortex-m0plus";
float-abi = "soft";
arch = "armv6m";
thumb = true;
};
};
config = {
allowUnsupportedSystem = true;
};
crossOverlays = [
];
overlays = [
(final: prev: {
llvmPackages = prev.llvmPackages // (
let
libraries = prev.llvmPackages.libraries.extend (f: p: {
libunwind = p.libunwind.override (p: {
devExtraCmakeFlags = libunwindCmakeFlags;
doFakeLibgcc = false;
});
});
in
libraries // { inherit libraries; }
);
newlib-nano = prev.newlib-nano.overrideAttrs (old: {
configureFlags = (old.configureFlags or []) ++ [
"--disable-multilib"
"--disable-libgloss"
];
});
})
];
};
in {
packages.pkgs = pkgsCross;
});
}
I believe this changed all of the libunwinds in llvmPackages and so is the closest so far.
But I still am running into the issue where the libunwind used by stdenv.cc.libcxx is not the same as is in llvmPackages.libunwind.
Do I need to somehow apply these changes to the stdenv? I would think the overlays would happen before the stdenv is created.
That build fail is because LIBUNWIND_IS_BAREMETAL=false
└─[$] nix build .#pkgs.stdenv.cc [17:40:48]
error: builder for '/nix/store/p28mka7c3n6ykkjxn9vvn0miskb990ni-libunwind-static-armv6m-none-eabi-19.1.7.drv' failed with exit code 1;
last 25 log lines:
> clang++: warning: argument unused during compilation: '-rtlib=compiler-rt' [-Wunused-command-line-argument]
> [2/10] Building C object libunwind/src/CMakeFiles/unwind_static_objects.dir/Unwind-wasm.c.obj
> clang: warning: argument unused during compilation: '-rtlib=compiler-rt' [-Wunused-command-line-argument]
> [3/10] Building C object libunwind/src/CMakeFiles/unwind_static_objects.dir/Unwind-sjlj.c.obj
> clang: warning: argument unused during compilation: '-rtlib=compiler-rt' [-Wunused-command-line-argument]
> [4/10] Building ASM object libunwind/src/CMakeFiles/unwind_static_objects.dir/UnwindRegistersRestore.S.obj
> clang: warning: argument unused during compilation: '-rtlib=compiler-rt' [-Wunused-command-line-argument]
> [5/10] Building ASM object libunwind/src/CMakeFiles/unwind_static_objects.dir/UnwindRegistersSave.S.obj
> clang: warning: argument unused during compilation: '-rtlib=compiler-rt' [-Wunused-command-line-argument]
> [6/10] Building C object libunwind/src/CMakeFiles/unwind_static_objects.dir/UnwindLevel1.c.obj
> clang: warning: argument unused during compilation: '-rtlib=compiler-rt' [-Wunused-command-line-argument]
> [7/10] Building C object libunwind/src/CMakeFiles/unwind_static_objects.dir/UnwindLevel1-gcc-ext.c.obj
> clang: warning: argument unused during compilation: '-rtlib=compiler-rt' [-Wunused-command-line-argument]
> [8/10] Building CXX object libunwind/src/CMakeFiles/unwind_static_objects.dir/libunwind.cpp.obj
> FAILED: [code=1] libunwind/src/CMakeFiles/unwind_static_objects.dir/libunwind.cpp.obj
> /nix/store/s3m4h7s5zf1h51cq0cymk0010z1qmpm3-armv6m-none-eabi-clang-wrapper-19.1.7/bin/armv6m-none-eabi-clang++ -D_FILE_OFFSET_BITS=64 -D_LARGEFILE_SOURCE -D_LIBCPP_HAS_NO_PRAGMA_SYSTEM_HEADER -D_LIBUNWIND_LINK_DL_LIB -D_LIBUNWIND_LINK_PTHREAD_LIB -D__STDC_CONSTANT_MACROS -D__STDC_FORMAT_MACROS -D__STDC_LIMIT_MACROS -I/build/libunwind-src-19.1.7/libunwind/include -fvisibility-inlines-hidden -Werror=date-time -Werror=unguarded-availability-new -Wall -Wextra -Wno-unused-parameter -Wwrite-strings -Wcast-qual -Wmissing-field-initializers -Wimplicit-fallthrough -Wcovered-switch-default -Wno-noexcept-type -Wnon-virtual-dtor -Wdelete-non-virtual-dtor -Wsuggest-override -Wno-comment -Wstring-conversion -ffunction-sections -fdata-sections -O3 -DNDEBUG -std=c++17 -Werror=return-type -funwind-tables -nostdinc++ -D_DEBUG -UNDEBUG -D_LIBUNWIND_IS_NATIVE_ONLY -Wall -Wextra -Wnewline-eof -Wshadow -Wwrite-strings -Wno-unused-parameter -Wno-long-long -Werror=return-type -Wextra-semi -Wundef -Wunused-template -Wformat-nonliteral -Wno-user-defined-literals -Wno-covered-switch-default -Wno-suggest-override -Wno-error -pedantic -fno-rtti -fstrict-aliasing -fno-exceptions -fno-rtti -MD -MT libunwind/src/CMakeFiles/unwind_static_objects.dir/libunwind.cpp.obj -MF libunwind/src/CMakeFiles/unwind_static_objects.dir/libunwind.cpp.obj.d -o libunwind/src/CMakeFiles/unwind_static_objects.dir/libunwind.cpp.obj -c /build/libunwind-src-19.1.7/libunwind/src/libunwind.cpp
> clang++: warning: argument unused during compilation: '-rtlib=compiler-rt' [-Wunused-command-line-argument]
> In file included from /build/libunwind-src-19.1.7/libunwind/src/libunwind.cpp:30:
> /build/libunwind-src-19.1.7/libunwind/src/AddressSpace.hpp:35:10: fatal error: 'dlfcn.h' file not found
> 35 | #include <dlfcn.h>
> | ^~~~~~~~~
> 1 error generated.
> [9/10] Building CXX object libunwind/src/CMakeFiles/unwind_static_objects.dir/Unwind-EHABI.cpp.obj
> clang++: warning: argument unused during compilation: '-rtlib=compiler-rt' [-Wunused-command-line-argument]
> ninja: build stopped: subcommand failed.
For full logs, run:
nix log /nix/store/p28mka7c3n6ykkjxn9vvn0miskb990ni-libunwind-static-armv6m-none-eabi-19.1.7.drv
error: 1 dependencies of derivation '/nix/store/r67zvxixc7ldjx52kwmc7v8snsrzd4d3-armv6m-none-eabi-clang-wrapper-19.1.7.drv' failed to build
┌─[kc@nixos] - [~/test] - [2356]
└─[$] nix eval .\#pkgs.llvmPackages.libunwind [17:40:58]
«derivation /nix/store/15qpl34qgv572i7kxf24zbw4ckzd2hr6-libunwind-static-armv6m-none-eabi-19.1.7.drv»
┌─[kc@nixos] - [~/test] - [2357]
└─[$] nix eval .\#pkgs.llvmPackages.libraries.libunwind [17:41:07]
«derivation /nix/store/15qpl34qgv572i7kxf24zbw4ckzd2hr6-libunwind-static-armv6m-none-eabi-19.1.7.drv»
┌─[kc@nixos] - [~/test] - [2358]
└─[$] nix eval .\#pkgs.llvmPackages.libraries.libcxx [17:41:14]
«derivation /nix/store/sqzk72y3fcwiznvib4yk8yl1p1f5zrji-libcxx-static-armv6m-none-eabi-19.1.7.drv»
┌─[kc@nixos] - [~/test] - [2359]
└─[$] nix eval .\#pkgs.llvmPackages.libraries.libcxx.propagatedBuildInputs [17:41:44]
[ null «derivation /nix/store/15qpl34qgv572i7kxf24zbw4ckzd2hr6-libunwind-static-armv6m-none-eabi-19.1.7.drv» ]
┌─[kc@nixos] - [~/test] - [2360]
└─[$] nix eval .\#pkgs.stdenv.cc.libcxx [17:41:57]
«derivation /nix/store/92319cxznaljxhgnj1r97a66dif52zdr-libcxx-static-armv6m-none-eabi-19.1.7.drv»
┌─[kc@nixos] - [~/test] - [2361]
└─[$] nix eval .\#pkgs.stdenv.cc.libcxx.propagatedBuildInputs [17:42:36]
[ null «derivation /nix/store/p28mka7c3n6ykkjxn9vvn0miskb990ni-libunwind-static-armv6m-none-eabi-19.1.7.drv» ]
┌─[kc@nixos] - [~/test] - [2362]
└─[$] nix build .#pkgs.stdenv.cc --dry-run [17:42:43]
these 6 derivations will be built:
/nix/store/p28mka7c3n6ykkjxn9vvn0miskb990ni-libunwind-static-armv6m-none-eabi-19.1.7.drv
/nix/store/92319cxznaljxhgnj1r97a66dif52zdr-libcxx-static-armv6m-none-eabi-19.1.7.drv
/nix/store/yl578f98j0y7ybxvr19xkikkq2rlq4lx-armv6m-none-eabi-clang-wrapper-19.1.7.drv
/nix/store/dccc8rnxidvdkj7dawqvybvjfvww6vq5-stdenv-linux.drv
/nix/store/4vmfwi1f9qadw6n491ff144d5ma5mkg0-compiler-rt-libc-static-armv6m-none-eabi-19.1.7.drv
/nix/store/r67zvxixc7ldjx52kwmc7v8snsrzd4d3-armv6m-none-eabi-clang-wrapper-19.1.7.drv```
It seems to be that the libcxx in clang is not updating with the changes to llvmPackages.libraries.libcxx:
┌─[kc@nixos] - [~/test] - [2390]
└─[$] nix eval .\#pkgs.llvmPackages.clang.libcxx [6:35:54]
«derivation /nix/store/92319cxznaljxhgnj1r97a66dif52zdr-libcxx-static-armv6m-none-eabi-19.1.7.drv»
┌─[kc@nixos] - [~/test] - [2391]
└─[$] nix eval .\#pkgs.llvmPackages.libcxx [6:36:02]
«derivation /nix/store/sqzk72y3fcwiznvib4yk8yl1p1f5zrji-libcxx-static-armv6m-none-eabi-19.1.7.drv»
┌─[kc@nixos] - [~/test] - [2392]
└─[$] nix eval .\#pkgs.llvmPackages.libraries.libcxx [6:36:08]
«derivation /nix/store/sqzk72y3fcwiznvib4yk8yl1p1f5zrji-libcxx-static-armv6m-none-eabi-19.1.7.drv»
I figured this out when I tried
packages.stdenv = (pkgsCross.overrideCC pkgsCross.stdenv pkgsCross.llvmPackages.clang);
And realized even that didn’t work. Do I need to force the re-evaluation of all of llvmPackages somehow? Maybe with llvmPackages.override?
I still have not been able to get this to propagate any of the changes into llvmPackages.tools or stdenv.cc. Could you send me a snippet of how I would override these flags with your PR#436350?