I’m switching to NixOS on my workstation (coming from Gentoo), and the nix-shell looks like an awesome tool. I’m trying to figure out how to integrate that with the way how we’re building some C and C++ projects at work. As a rule, our projects require a mixture of boring system packages (think PCRE, OpenSSL, etc) as well as some bleeding-edge libraries. For those bleeding-edge dependencies, we’re pinning their specific versions via this git submodule. We’re using CMake, and we’re almost exclusively relying on pkg-config to find paths to all dependencies. Here’s how a typical CI build script looks like if you’re curious.
On my development machine, I would like to be able to switch between different “build environments” – say, GCC 9, GCC 9 with ASAN, or CLANG 10 with TSAN. On my old Gentoo system I was doing this by using system packages for my boring dependencies, and installing my apps and bleeding-edge dependencies into a custom prefix, once per each “build environment”, say, ~/proj/gcc-9 and ~/proj/clang-10-tsan. This required setting up a few variables such as PKG_CONFIG_PATH and PATH so that they include my prefix. I tried to replicate this with nix-shell by doing roughly this:
#!/run/current-system/sw/bin/env nix-shell
#!nix-shell -i bash -p cmake ninja gcc pkg-config swig pcre bison flex libev openssl libssh python38 python3.pkgs.pytest
PREFIX=${TOP_BUILD_DIR}/target
export PATH=${PREFIX}/bin:$PATH
export PKG_CONFIG_PATH=${PREFIX}/lib64/pkgconfig${PKG_CONFIG_PATH:+:$PKG_CONFIG_PATH}
CMAKE_OPTIONS="-GNinja -DCMAKE_EXPORT_COMPILE_COMMANDS=1 \
-DCMAKE_INSTALL_RPATH:INTERNAL=${PREFIX}/lib64 \
-DCMAKE_INSTALL_RPATH_USE_LINK_PATH:INTERNAL=ON \
-DCMAKE_PREFIX_PATH=${PREFIX} \
-DCMAKE_INSTALL_PREFIX=${PREFIX}"
build_dep_cmake() {
BUILD_DIR=${TOP_BUILD_DIR}/$(basename $1)
mkdir -p ${BUILD_DIR}
pushd ${BUILD_DIR}
cmake ${CMAKE_OPTIONS} $1
ninja install
ctest --output-on-failure
popd
}
build_dep_cmake ${PROJECT_ROOT}/project/submodules/dependencies/A
build_dep_cmake ${PROJECT_ROOT}/project/submodules/dependencies/B
build_dep_cmake ${PROJECT_ROOT}/project/submodules/dependencies/C
In the code above, all of the A, B and C dependencies provide some *.pc files which will be needed by the later stages of the build. On a regular Linux system, this works fine because pkg-config can find systemwide packages (such as libev) as well as my custom A, B and C libraries which are coming from my $PREFIX. Under the nix-shell, however, the $PKG_CONFIG_PATH gets created as a union of all library dependencies, which makes sense. That variable eventually gets renamed to $PKG_CONFIG_PATH_x86_64_unknown_linux_gnu, and that variable is then used as-is for overriding my own user-provided env variables in the pkg-config wrapper. Here’s how the final pkg-config that is in my $PATH when under nix-shell looks like:
# ...
if (( ${#role_suffixes[@]} > 0 )); then
# replace env var with nix-modified one
PKG_CONFIG_PATH=$PKG_CONFIG_PATH_x86_64_unknown_linux_gnu exec /nix/store/52360vwgzlv2h27pyzidv542gk6dvp1c-pkg-config-0.29.2/bin/pkg-config "$@"
else
# pkg-config isn't a bonafied dependency so ignore setup hook entirely
exec /nix/store/52360vwgzlv2h27pyzidv542gk6dvp1c-pkg-config-0.29.2/bin/pkg-config "$@"
fi
#...
The TL;DR version is that Nix’ pkg-config wrapper overrides my modifications to $PKG_CONFIG_PATH. I can hack that around by setting $PKG_CONFIG_PATH_x86_64_unknown_linux_gnu in my own wrapper, but then pkg-config complains that it cannot find my systemwide libraries. Do I have to write my own pkg-config wrapper which calls something like this?
#!/bin/sh
# file: ${PREFIX}/bin/pkg-config
PKG_CONFIG_PATH=${PKG_CONFIG_PATH}:${PKG_CONFIG_PATH_x86_64_unknown_linux_gnu} \
pkg-config-unwrapped
…and stash that into my $PATH?
Perhaps the Nix way of working is to package all of my dependencies as derivations, and always enter an environment via nix-shell. That sounds awesome, and it will probably also allow me to build the whole environment with ASAN or clang’s libstdc++ (which only works on my old system because I just happened not to use any systemwide C++ library). That would be super cool. However, I’m often working on these libraries themselves, I’m editing them in my text editor, and for that it is very useful to have the compilation database so that my editor can offer tab completion and what not. Can I instruct nix-shell and/or nix-build to keep the build directory around, so that I still have access to the compile_commands.json? Should I put some src = stdenv.fetchGit { url = "file:///path/to/my/checkout"; } override blocks into the nix-shell file that I’m using? How do I inform nix-shell that I’ve changed these revisions on disk, and that the dependencies should be rebuilt?
How do I do this while I’m actively developing my dependencies? I guess I’ll have to commit every time I want to regenerate my shell environment, right?
How are people solving these problems on NixOS?