Nix Shells and Python Packages with C++ Extensions

Guides
1

TLDR

If you’re using a nix shell on macOS / darwin, and installing a Python package that has C++ extensions, then it may not compile because of missing shared standard / framework libraries. This is because Python confusingly uses clang instead of clang++ to compile C++ extensions, and Nix doesn’t include C++ standard libraries in the clang wrapper by default. To fix it, you can adjust your shell.nix like this:

{ pkgs ? import <nixpkgs> {}}:

with pkgs; mkShell {
  # Include C++ headers for regular clang calls:
  NIX_CFLAGS_COMPILE = lib.optionals stdenv.isDarwin [
    "-I${lib.getDev libcxx}/include/c++/v1"
  ];

  nativeBuildInputs = [
    ...
  ] ++ lib.optionals stdenv.isDarwin [
    # Add any Apple framework libraries your package needs, e.g.
    darwin.apple_sdk.frameworks.IOKit
  ];

}

Longer story

I ran into an issue that chewed up a couple hours of my time yesterday, and figured I’d drop the solution here in case anyone else would find it useful. I use direnv with nix shells to create automatic virtual environments for my Python projects along with some other dependencies from nixpkgs. Yesterday, I went to upgrade a few packages and ran into a weird error with ddtrace==1.9.3, which recently introduced a C++ extension:

clang -Wno-unused-result -Wsign-compare -Wunreachable-code -DNDEBUG -g -fwrapv -O3 -Wall -I/nix/store/4vfgllzm4wgpl5vf50zakd89l1gdx5jm-libxcrypt-4.4.33/include -I/usr/local/opt/openssl/include -I/Users/johnwhitman/projects/api.galileo.io/.direnv/python-3.10.10/include -I/nix/store/jx137r213yrklvvqd83dhhk01n671pas-python3-3.10.10/include/python3.10 -c ddtrace/appsec/iast/_taint_tracking/_taint_tracking.cpp -o /var/folders/3v/nn4ctvl12sl768gct5vt199h0000gn/T/tmp81l5u6lg.build-temp/ddtrace/appsec/iast/_taint_tracking/_taint_tracking.o -std=c++17
clang-11: warning: argument unused during compilation: '-fno-strict-overflow' [-Wunused-command-line-argument]
ddtrace/appsec/iast/_taint_tracking/_taint_tracking.cpp:2:10: fatal error: 'iostream' file not found
#include <iostream>
	   ^~~~~~~~~~
1 error generated.

I noticed it was using clang instead of clang++ to compile the source. That lead me to NixOS/nixpkgs#26531, which points out that Python confusingly uses cc to compile C++ (ref unixcompiler.py). Another comment in that thread provides the following hint, which leads to the first fix for missing iostream:

To hack around this problem you simply have to grep the pythonPackages for libcxx.

Sure enough, you can find a few examples of workarounds, like this one from kiwisolver to override some compile flags:

NIX_CFLAGS_COMPILE = lib.optionals stdenv.isDarwin [
  "-I${lib.getDev libcxx}/include/c++/v1"
];

Another attempt to install the package got further, but failed with an error about IOKit/IOKitLib.h missing:

ddtrace/vendor/psutil/_psutil_osx.c:37:10: fatal error: 'IOKit/IOKitLib.h' file not found
#include <IOKit/IOKitLib.h>
         ^~~~~~~~~~~~~~~~~~
1 error generated.

This is apparently an Apple framework library that isn’t available in the shell’s sandbox by default, so you simply have to include that as an input in your shell (StackOverflow ref):

nativeBuildInputs = [
  ...
] ++ lib.optionals stdenv.isDarwin [
  # Add any Apple framework libraries your package needs, e.g.
  darwin.apple_sdk.frameworks.IOKit
];

Hope this saves somebody else some time in the future.

6 Likes
2

@johnjameswhitman this sounds like almost a PR, but at least an addition to the Nixpkgs manual. Ping me if you need guidance where to put new documentation.

@ryantm this should be in Guides.

3

SG – Can definitely add…do you have a section and/or reference for where I can submit a PR?

4

A quick look at the Nixpkgs manual’s Python section doesn’t reveal a precise place where to put this particular piece of information. But you may find a nice spot either in Building packages and applications or FAQ. Here’s the source code for that part of the manual.

1 Like
5

Thanks for the pointers :pray: Finally got around to opening a PR in the docs whenever you have a free moment to review: doc/python: Adds workarounds for building extensions on Darwin by johnjameswhitman · Pull Request #225937 · NixOS/nixpkgs · GitHub

6

Wanted to drop a note that this tip worked very well and probably saved me a lot of head-banging.

For reference, I had to fully qualify pkgs in my shell.nix:

  nativeBuildInputs = [ pkgs.pkg-config ] ++ pkgs.lib.optionals pkgs.stdenv.isDarwin [
    # Add any Apple framework libraries your package needs, e.g.
    pkgs.darwin.apple_sdk.frameworks.IOKit
  ];
  NIX_CFLAGS_COMPILE = pkgs.lib.optionals pkgs.stdenv.isDarwin [
    "-I${pkgs.lib.getDev pkgs.libcxx}/include/c++/v1"
  ];

Overall it is impressive how well Python packaging works in native Nix … pytorch and friends compiled and ran perfectly the first time, and I only had issues trying to install a separate package from pip (hsnwlib)

3 Likes