I’m a C++ developer but a Nix newb. I’m trying to define a shell.nix that I can use for building a variety of C and C++ projects with autotools. I get an error on the first file compiled:
g++ -std=c++17 -DHAVE_CONFIG_H -I. -I./include -I/usr/include -I/nix/store/89f1my7szprsn9pdwr2h2ms845sxcna9-icu4c-68.2-dev/include -DU_USING_ICU_NAMESPACE=0 -DU_CHARSET_IS_UTF8=1 -DUNISTR_FROM_CHAR_EXPLICIT=explicit -DUNSTR_FROM_STRING_EXPLICIT=explicit -pthread -I/home/jon/code/make_world2/install/include -W -Wall -Wextra -Wnon-virtual-dtor -pedantic -pipe -O3 -mtune=intel -MT src/enc/encodings.o -MD -MP -MF $depbase.Tpo -c -o src/enc/encodings.o src/enc/encodings.cpp &&\
mv -f $depbase.Tpo $depbase.Po
In file included from /nix/store/sr0ci8f8pgby77fj4mpcl9bcgxji3676-gcc-10.2.0/include/c++/10.2.0/cstdio:42,
from /nix/store/sr0ci8f8pgby77fj4mpcl9bcgxji3676-gcc-10.2.0/include/c++/10.2.0/ext/string_conversions.h:43,
from /nix/store/sr0ci8f8pgby77fj4mpcl9bcgxji3676-gcc-10.2.0/include/c++/10.2.0/bits/basic_string.h:6535,
from /nix/store/sr0ci8f8pgby77fj4mpcl9bcgxji3676-gcc-10.2.0/include/c++/10.2.0/string:55,
from /nix/store/sr0ci8f8pgby77fj4mpcl9bcgxji3676-gcc-10.2.0/include/c++/10.2.0/stdexcept:39,
from ./include/basic.h:21,
from src/enc/encodings.cpp:19:
/usr/include/stdio.h:781:10: fatal error: bits/sys_errlist.h: No such file or directory
781 | #include <bits/sys_errlist.h>
| ^~~~~~~~~~~~~~~~~~~~
compilation terminated.
make: *** [Makefile:1753: src/enc/encodings.o] Error 1
I’m using Linux Mint 20.1 and invoking nix-shell with --pure. Are there some stunningly obvious packages or incantations I’m missing? Should I be using stdenv.mkShell instead?
I’d also be grateful for any links to articles about how autoconf and/or pkg-config interact with nix. I have a separate error in a configure script where ax_boost_filesystem.m4 is not finding libboost_filesystem.so, even though it’s installed as part of the boost package.
So, I need to do a bit more looking for the first issue, but I think the second issue is probably that you’re trying to compile in pure mode. Specifically, because libboost_filesystem.so is a shared object library file, and the shell is pure (meaning no external undefined dependencies), you would have to find a nix package that either contained the shared object library (which boost only contains headers), compile it statically, or build it in an impure shell.
Generally, I don’t know that autoconf or pkg-conf would interact well, since most tooling does make the assumption of the standard shared library system, which Nix specifically avoids.
(zulip.out) 1,216 r /nix/store/r7w37zn4ixf0g14i7fgd49nfv96yhxm2-glibc-2.31-74-dev/include/bits/sys_errlist.h
(libidn2.bin) 1,215 r /nix/store/p4s4jf7aq6v6z9iazll1aiqwb34aqxq9-bootstrap-tools/include-glibc/bits/sys_errlist.h
glibc_multi.dev 0 s /nix/store/pn6ah0chz6nc0q6zf3s9qlxn2d2vx82h-glibc-multi-2.31-74-dev/include/bits/sys_errlist.h
glibc.dev 1,216 r /nix/store/ch44wa8qqss0dm04lnvf5n7mfh23kmrv-glibc-2.31-74-dev/include/bits/sys_errlist.h
glibc_memusage.dev 1,216 r /nix/store/br3lav9gvkyhsjdhcvy0d9fpmmbz12wh-glibc-gd-2.31-74-dev/include/bits/sys_errlist.h
you may have to include one these in your shell.nix.
good luck, feedback how it goes.
i’ve solved a few pkg-config not finding libraries…
nativeBuildInputs = [ pkg-config ];
this default C/C++ builders do a lot of automatic patching of autoconf scripts to make them nix friendly, so be wary of that.
for my C/C++ projects i’ve used
stdenv.mkDerivation
which seems to have been quite successful so far , the genericBuilder is a monster shell script that automagically tries to the right thing for a project as far as i understand it.
is this project on a public git repo I can pull, i then test reproduce your nix-shell directly. rather than guessing.
RE: libboost_filesystem.so, I seem to have all the usual Boost libraries installed under /nix/store/qpmk4gqhsyv2b1rs69pmbjylafx4spjz-boost-1.69.0/lib/, including libboost_filesystem.so as a symbolic link to libboost_filesystem.so.1.69.0. So I think the boost package must have put it there.
When I check env, that boost nix store path is included in NIX_LDFLAGS, CMAKE_LIBRARY_PATH, and CMAKE_PREFIX_PATH.
Thank you for the pointers. What I’ve been trying is not from a public repo, but it probably makes sense for me to get something smaller working, so let me try that and I’ll report back.
I traced through the headers. /nix/store/sr0ci8f8pgby77fj4mpcl9bcgxji3676-gcc-10.2.0/ looks like the C++ stdlib, which is what I’d expect, but then tries to #include <stdio.h>, which is not to be found in that package. From the error, it’s picking up on /usr/include/stdio.h, and then that lacks bits/sys_errlist.h — but of course that’s the system gcc/glibc environment, not under /nix, and a different version to boot. So perhaps I do need to have glibc in my packages under shell.nix…
The default.nix for the ssdeep package is pretty simple (https://github.com/NixOS/nixpkgs/blob/7eda163eac253fd03cd8ff3d2ab08f96d2dab8d3/pkgs/tools/security/ssdeep/default.nix), but I think what stdenv.mkDerivation implies is that it will be built with the system compiler. I’d like to avoid the system compiler–the point is to specify the build chain in the nix-shell so that everyone on my team can build things in the same way, regardless of macOS, Linux distro choice, etc.
Here is the output:
[nix-shell:~/Downloads]$ tar -xzf ssdeep-release-2.14.1.tar.gz
[nix-shell:~/Downloads]$ cd ssdeep-release-2.14.1/
[nix-shell:~/Downloads/ssdeep-release-2.14.1]$ ll
total 408
drwxr-xr-x 3 jon jon 4096 Nov 6 2017 ./
drwxr-xr-x 3 jon jon 4096 Mar 22 22:29 ../
-rw-r--r-- 1 jon jon 403 Nov 6 2017 .gitignore
-rw-r--r-- 1 jon jon 68 Nov 6 2017 AUTHORS
-rw-r--r-- 1 jon jon 18002 Nov 6 2017 COPYING
-rw-r--r-- 1 jon jon 11145 Nov 6 2017 ChangeLog
-rw-r--r-- 1 jon jon 105718 Nov 6 2017 Doxyfile
-rw-r--r-- 1 jon jon 915 Nov 6 2017 FILEFORMAT
-rw-r--r-- 1 jon jon 15756 Nov 6 2017 INSTALL
-rw-r--r-- 1 jon jon 1762 Nov 6 2017 Makefile.am
-rw-r--r-- 1 jon jon 5654 Nov 6 2017 NEWS
-rw-r--r-- 1 jon jon 3097 Nov 6 2017 README
-rw-r--r-- 1 jon jon 259 Nov 6 2017 TODO
-rwxr-xr-x 1 jon jon 28 Nov 6 2017 bootstrap*
-rw-r--r-- 1 jon jon 2731 Nov 6 2017 configure.ac
-rw-r--r-- 1 jon jon 3266 Nov 6 2017 cycles.cpp
-rw-r--r-- 1 jon jon 14401 Nov 6 2017 dig.cpp
-rw-r--r-- 1 jon jon 2769 Nov 6 2017 edit_dist.c
-rw-r--r-- 1 jon jon 497 Nov 6 2017 edit_dist.h
-rw-r--r-- 1 jon jon 3144 Nov 6 2017 engine.cpp
-rw-r--r-- 1 jon jon 4197 Nov 6 2017 filedata.cpp
-rw-r--r-- 1 jon jon 2632 Nov 6 2017 filedata.h
-rw-r--r-- 1 jon jon 4335 Nov 6 2017 find-file-size.c
-rw-r--r-- 1 jon jon 26724 Nov 6 2017 fuzzy.c
-rw-r--r-- 1 jon jon 8004 Nov 6 2017 fuzzy.h
-rw-r--r-- 1 jon jon 8432 Nov 6 2017 helpers.cpp
drwxr-xr-x 2 jon jon 4096 Nov 6 2017 m4/
-rw-r--r-- 1 jon jon 9328 Nov 6 2017 main.cpp
-rw-r--r-- 1 jon jon 2121 Nov 6 2017 main.h
-rw-r--r-- 1 jon jon 9476 Nov 6 2017 match.cpp
-rw-r--r-- 1 jon jon 1214 Nov 6 2017 match.h
-rwxr-xr-x 1 jon jon 112 Nov 6 2017 normal.sh*
-rw-r--r-- 1 jon jon 3445 Nov 6 2017 sample.c
-rw-r--r-- 1 jon jon 5256 Nov 6 2017 ssdeep.1
-rw-r--r-- 1 jon jon 6395 Nov 6 2017 ssdeep.h
-rw-r--r-- 1 jon jon 26778 Nov 6 2017 sum_table.h
-rw-r--r-- 1 jon jon 1591 Nov 6 2017 tchar-local.h
-rwxr-xr-x 1 jon jon 426 Nov 6 2017 test-against-old.sh*
-rw-r--r-- 1 jon jon 2670 Nov 6 2017 ui.cpp
-rwxr-xr-x 1 jon jon 201 Nov 6 2017 win.sh*
-rwxr-xr-x 1 jon jon 114 Nov 6 2017 world.sh*
[nix-shell:~/Downloads/ssdeep-release-2.14.1]$ ./bootstrap
libtoolize: putting auxiliary files in '.'.
libtoolize: copying file './ltmain.sh'
libtoolize: putting macros in AC_CONFIG_MACRO_DIRS, 'm4'.
libtoolize: copying file 'm4/libtool.m4'
libtoolize: copying file 'm4/ltoptions.m4'
libtoolize: copying file 'm4/ltsugar.m4'
libtoolize: copying file 'm4/ltversion.m4'
libtoolize: copying file 'm4/lt~obsolete.m4'
configure.ac:5: warning: 'AM_CONFIG_HEADER': this macro is obsolete.
configure.ac:5: You should use the 'AC_CONFIG_HEADERS' macro instead.
./lib/autoconf/general.m4:2433: AC_DIAGNOSE is expanded from...
aclocal.m4:727: AM_CONFIG_HEADER is expanded from...
configure.ac:5: the top level
configure.ac:11: warning: The macro `AC_LIBTOOL_WIN32_DLL' is obsolete.
configure.ac:11: You should run autoupdate.
m4/ltoptions.m4:148: AC_LIBTOOL_WIN32_DLL is expanded from...
configure.ac:11: the top level
configure.ac:11: warning: AC_LIBTOOL_WIN32_DLL: Remove this warning and the call to _LT_SET_OPTION when you
configure.ac:11: put the 'win32-dll' option into LT_INIT's first parameter.
./lib/autoconf/general.m4:2433: AC_DIAGNOSE is expanded from...
m4/ltoptions.m4:148: AC_LIBTOOL_WIN32_DLL is expanded from...
configure.ac:11: the top level
configure.ac:12: warning: The macro `AM_PROG_LIBTOOL' is obsolete.
configure.ac:12: You should run autoupdate.
m4/libtool.m4:100: AM_PROG_LIBTOOL is expanded from...
configure.ac:12: the top level
configure.ac:9: installing './compile'
configure.ac:7: installing './config.guess'
configure.ac:7: installing './config.sub'
configure.ac:2: installing './install-sh'
configure.ac:2: installing './missing'
Makefile.am: installing './depcomp'
[nix-shell:~/Downloads/ssdeep-release-2.14.1]$ ./configure
checking for a BSD-compatible install... /nix/store/w21pgi6691202iafl0cnr7hk6wvvdz3n-coreutils-8.32/bin/install -c
checking whether build environment is sane... yes
checking for a race-free mkdir -p... /nix/store/w21pgi6691202iafl0cnr7hk6wvvdz3n-coreutils-8.32/bin/mkdir -p
checking for gawk... gawk
checking whether make sets $(MAKE)... yes
checking whether make supports nested variables... yes
checking build system type... x86_64-pc-linux-gnu
checking host system type... x86_64-pc-linux-gnu
checking for gcc... gcc
checking whether the C compiler works... yes
checking for C compiler default output file name... a.out
checking for suffix of executables...
checking whether we are cross compiling... no
checking for suffix of object files... o
checking whether the compiler supports GNU C... yes
checking whether gcc accepts -g... yes
checking for gcc option to enable C11 features... none needed
checking whether gcc understands -c and -o together... yes
checking whether make supports the include directive... yes (GNU style)
checking dependency style of gcc... gcc3
checking whether the compiler supports GNU C++... yes
checking whether g++ accepts -g... yes
checking for g++ option to enable C++11 features... unsupported
checking for g++ option to enable C++98 features... unsupported
checking dependency style of g++... gcc3
checking how to print strings... printf
checking for a sed that does not truncate output... /nix/store/n0516fv7vwjbl41nl4q58w9si80ab93i-gnused-4.8/bin/sed
checking for grep that handles long lines and -e... /nix/store/3mx947xcmsyk79izqc8ifv31qafp80pc-gnugrep-3.6/bin/grep
checking for egrep... /nix/store/3mx947xcmsyk79izqc8ifv31qafp80pc-gnugrep-3.6/bin/grep -E
checking for fgrep... /nix/store/3mx947xcmsyk79izqc8ifv31qafp80pc-gnugrep-3.6/bin/grep -F
checking for ld used by gcc... ld
checking if the linker (ld) is GNU ld... yes
checking for BSD- or MS-compatible name lister (nm)... nm
checking the name lister (nm) interface... BSD nm
checking whether ln -s works... yes
checking the maximum length of command line arguments... 1572864
checking how to convert x86_64-pc-linux-gnu file names to x86_64-pc-linux-gnu format... func_convert_file_noop
checking how to convert x86_64-pc-linux-gnu file names to toolchain format... func_convert_file_noop
checking for ld option to reload object files... -r
checking for objdump... objdump
checking how to recognize dependent libraries... pass_all
checking for dlltool... dlltool
checking how to associate runtime and link libraries... printf %s\n
checking for archiver @FILE support... @
checking for strip... strip
checking for ranlib... ranlib
checking command to parse nm output from gcc object... ok
checking for sysroot... no
checking for a working dd... /nix/store/w21pgi6691202iafl0cnr7hk6wvvdz3n-coreutils-8.32/bin/dd
checking how to truncate binary pipes... /nix/store/w21pgi6691202iafl0cnr7hk6wvvdz3n-coreutils-8.32/bin/dd bs=4096 count=1
checking for mt... no
checking if : is a manifest tool... no
checking for stdio.h... yes
checking for stdlib.h... yes
checking for string.h... yes
checking for inttypes.h... yes
checking for stdint.h... yes
checking for strings.h... yes
checking for sys/stat.h... yes
checking for sys/types.h... yes
checking for unistd.h... yes
checking for dlfcn.h... yes
checking for objdir... .libs
checking if gcc supports -fno-rtti -fno-exceptions... no
checking for gcc option to produce PIC... -fPIC -DPIC
checking if gcc PIC flag -fPIC -DPIC works... yes
checking if gcc static flag -static works... no
checking if gcc supports -c -o file.o... yes
checking if gcc supports -c -o file.o... (cached) yes
checking whether the gcc linker (ld -m elf_x86_64) supports shared libraries... yes
checking whether -lc should be explicitly linked in... no
checking dynamic linker characteristics... GNU/Linux ld.so
checking how to hardcode library paths into programs... immediate
checking whether stripping libraries is possible... yes
checking if libtool supports shared libraries... yes
checking whether to build shared libraries... yes
checking whether to build static libraries... yes
checking how to run the C++ preprocessor... g++ -E
checking for ld used by g++... ld -m elf_x86_64
checking if the linker (ld -m elf_x86_64) is GNU ld... yes
checking whether the g++ linker (ld -m elf_x86_64) supports shared libraries... yes
checking for g++ option to produce PIC... -fPIC -DPIC
checking if g++ PIC flag -fPIC -DPIC works... yes
checking if g++ static flag -static works... no
checking if g++ supports -c -o file.o... yes
checking if g++ supports -c -o file.o... (cached) yes
checking whether the g++ linker (ld -m elf_x86_64) supports shared libraries... yes
checking dynamic linker characteristics... (cached) GNU/Linux ld.so
checking how to hardcode library paths into programs... immediate
checking for /usr/local/include... yes
checking for /usr/local/lib... yes
checking for /opt/local/include... no
checking for /opt/local/lib... no
checking for /sw/include... no
checking for /sw/lib... no
checking whether byte ordering is bigendian... no
checking for special C compiler options needed for large files... no
checking for _FILE_OFFSET_BITS value needed for large files... no
checking for libgen.h... yes
checking for dirent.h... yes
checking for inttypes.h... (cached) yes
checking for fcntl.h... yes
checking for sys/types.h... (cached) yes
checking for sys/ioctl.h... yes
checking for sys/param.h... yes
checking for wchar.h... yes
checking for unistd.h... (cached) yes
checking for sys/stat.h... (cached) yes
checking for sys/disk.h... no
checking for inttypes.h... (cached) yes
checking for sys/mount.h... yes
checking for _LARGEFILE_SOURCE value needed for large files... no
checking that generated files are newer than configure... done
configure: creating ./config.status
config.status: creating Makefile
config.status: creating config.h
config.status: executing depfiles commands
config.status: executing libtool commands
[nix-shell:~/Downloads/ssdeep-release-2.14.1]$ make
make all-am
make[1]: Entering directory '/home/jon/Downloads/ssdeep-release-2.14.1'
g++ -DHAVE_CONFIG_H -I. -I/usr/local/include -g -O2 -MT main.o -MD -MP -MF .deps/main.Tpo -c -o main.o main.cpp
In file included from /nix/store/sr0ci8f8pgby77fj4mpcl9bcgxji3676-gcc-10.2.0/include/c++/10.2.0/ext/string_conversions.h:41,
from /nix/store/sr0ci8f8pgby77fj4mpcl9bcgxji3676-gcc-10.2.0/include/c++/10.2.0/bits/basic_string.h:6535,
from /nix/store/sr0ci8f8pgby77fj4mpcl9bcgxji3676-gcc-10.2.0/include/c++/10.2.0/string:55,
from ssdeep.h:14,
from main.cpp:11:
/nix/store/sr0ci8f8pgby77fj4mpcl9bcgxji3676-gcc-10.2.0/include/c++/10.2.0/cstdlib:75:15: fatal error: stdlib.h: No such file or directory
75 | #include_next <stdlib.h>
| ^~~~~~~~~~
compilation terminated.
make[1]: *** [Makefile:598: main.o] Error 1
make[1]: Leaving directory '/home/jon/Downloads/ssdeep-release-2.14.1'
make: *** [Makefile:408: all] Error 2
stdenv.mkDerivation is actually a wrapper for derivation that includes a couple of standard GNU autotools and compilers, as well as libraries, specifically gcc, so the builds are still pure.
Edit: Additionally, its due to these included tools that you may even want to consider building your project via stdenv.mkDerivation and simply using the bundler to export it for other non-nix platforms.
shell script that automagically tries to the right thing for a project as far as i understand it.
Hound 