Get clangd to find standard headers in nix-shell

Help
1

TLDR

I would like to come up with a reliable recipe that lets me hack on C++ projects with clangd as LSP server.

I have trouble getting clangd to find standard headers. For example, in a simple hello world program it fails to find <iostream>.

Some details

I’m not sure it’s worth while to spew details of all the things that I have tried, so here’s an outline of the basic problem.

Most basic problem

I understand that clangd needs some help in understanding how files in the project are compiled, including where headers can be found. One way of providing it that information is via the compile_commands.json file.

Let’s say I have the following main.cc:

#include <iostream>
int main() { std::cout << "Hi!\n"; }

then, opening this file in my editor causes LSP to complain with “‘iostream’ file not found”.

At some point in the past, I have managed to solve this problem by generating a compile_commands.json with some variation on the theme of

bear -- clang++ main.cc

which creates a compile_commands.json, in the presence of which (after a restart of the server) LSP now recognizes <iostream>. (Having just tried this procedure in a fresh toy project, it appears not to work at this time of day / this phase of the moon.)

Projects with dependencies

I would like to work on more complex projects than Hello World. These typically have dependencies, whose header files clangd also fails to find without an appropraite compile_commands.json, but bear usually solves this without much trouble. Thus I find myself in the bizarre situation where clangd understands all the headers from obscure libraries, but none of the standard ones.

cmake

invoking CMake with cmake -DCMAKE_EXPORT_COMPILE_COMMANDS=1 generates a compile_commands.json with which LSP fails to find <iostream>. I notice that this compile_commands.json specifies gcc-wrapper as the compiler. [edit: I can change this to clang-wrapper by using pkgs.llvmPackages_11.stdenv.mkDerivation in shell.nix.]

[Edit:

While cmake -DCMAKE_EXPORT_COMPILE_COMMANDS=1 generates a seemingly useless compile_commands.json, I get much more useful results with

cmake .
make clean
bear -- make

Without the make clean, any files that don’t need to be recompiled since some previous compilation, will be unknown to bear.

]

I doubt that further details of my stumbling around would be useful or interesting, so I’ll leave it at that.

The question

Do you have a reliably working solution for using clangd as an LSP server in nix-shell-based development of C++ projects?

7 Likes
2

I don’t know whether there’s already an easy way to do that, but I have the impression that it would be possible to wrap clangd the same way we wrap the compilers themselves to pass it the right set of flags.

Other than that, I think bear should indeed work (although it’s a bit of an occasional pain to use because of the make clean dance − but as I’ve only been using it in autotools-based projects where bear was the only solution I never worried about it too much).

3

I asked about this too recently

Some suggestions were proposed, but they didn’t work for my project.

4 
$ nix-shell --pure -p clangStdenv bear clang

$ bear -- clang++ main.cc

$ ./a.out
Hi!

for a shell.nix, I would do:

with import <nixpkgs> { };

clangStdenv.mkDerivation {
  name = "dev-shell";

  src = null;

  nativeBuildInputs = [ bear clang ];
  buildInputs = [ clang ];

  shellHook = ''
     # code ran when entering shell
  '';
}

why put clang in both inputs?

you get auto completion “for free” when they are in nativeBuildInputs, clang in buildInputs might be redundant with clangStdenv

2 Likes
5

The problem is that clangd currently is wrapped with the standard lib from clang instead of gcc. We need to fix this in nixpkgs.

6

I see

Which is used in the substituttion for the wrapper here:

Do we need to have separately wrapped clangd for gcc stdlib and for clangd stdlib?

7

And this is an attempt to fix that wrapping, right?

8

Yes I just did not get around to finish it. But feel free to pick it up.

9

I think your PR resolved my issue with not finding the gcc headers, but I’m still having issues finding generated headers. Thanks! I can continue looking into it a bit myself.

10

I seem to have found a solution to the cmake-failing-to-find-standard-C+±headers problem.

Firstly, adding

set(CMAKE_EXPORT_COMPILE_COMMANDS ON CACHE INTERNAL "")

to my CMakeLists.txt removes the need to specify -DCMAKE_EXPORT_COMPILE_COMMANDS=1 on the cmake CLI.

Secondly, adding

if(CMAKE_EXPORT_COMPILE_COMMANDS)
  set(CMAKE_CXX_STANDARD_INCLUDE_DIRECTORIES 
      ${CMAKE_CXX_IMPLICIT_INCLUDE_DIRECTORIES})
endif()

seems to make the cmake-generated compile_commands.json give clangd the required information to find standard headers, in addition to the project specific information.

This solution seems to be working quite well for me.

14 Likes
11

After a day of banging my head against the wall with clangd not working in my text editor but being able to compile the project fine your post has fixed everything for me, thank you!

2 Likes
12

I eventually got fed up with needing to solve this problem differently for each build system, so wrote this: GitHub - danielbarter/mini_compile_commands. it is similar to bear, but works more reliably on nixos (bear struggles to distinguish between our compiler wrappers and the compiler invocations themselves)

1 Like
13

From what I got, the reason clangd isn’t recognizing where standard libraries are is because it’s invoking the “raw” clang command. The clang command that nix provides is actually a wrapper script that attaches a bunch of compiler flags to the raw command, telling it where to find the libraries and other things.

Everybody said that above, but I didn’t really get it till I ran clang --version and which clang, and saw they were in two different nix store locations. You can run vim $(which clang) to read the wrapper script.

@danielbarter’s solution looks the nicest, but in the meantime found this hack that worked for a simple project. I ran

NIX_DEBUG=1 clang myfile.c

The wrapper script emits debug logs that are really nice – it outputs all the flags it used for clang and ld.

extra flags after to /nix/store/7kln4zar8cv0h7bxbw20crwai3mha9z7-clang-11.1.0/bin/clang:
  -B/nix/store/a64bcr4kndmx2kvpn46zq6h17n1bwly0-libSystem-11.0.0/lib/
  -idirafter
  /nix/store/a64bcr4kndmx2kvpn46zq6h17n1bwly0-libSystem-11.0.0/include
  -B/nix/store/nih4qpbwvasnz6syqicc8raa2kvfkqwm-clang-11.1.0-lib/lib
  ...

I copied the above block of flags (starting just under “extra flags after to …/clang”) to compile_flags.txt in my project directory, and clangd started working :tada:.

compile_flags.txt is a file like compile_commands.json that clangd recognizes. When compile_flags.txt is in the project directory, clangd will use the flags when invoking the raw version of the clang compiler.

3 Likes
14

clang-tools has been providing a working clangd for a while. Using that is the easiest solution.

There is a stale PR to fix the clangd provided by clang too.

5 Likes
15

I use the following as @tobim suggests: