I’d like to compile DeepStream, a Nvidia’s set of Gstreamer plugins, from sources as a Nix derivation.
There are sources in a Docker image nvcr.io/nvidia/deepstream:6.3-triton-multiarch, for example. Directory structure is following:
ls sources/gst-plugins/gst-nvinfer sources/includes/
In a flake https://gist.github.com/le-chat/46731da439b8a9765651db53eb11b1f6 there is .#deepstream, where I unpack a deb with DeepStream and try to recompile gst-plugins (patching Makefiles which contain many hard-coded paths). Several plugins fail to build, but I ignore them. nix develop .#gst gives a shell, where rm -r ~/.cache/gstreamer-1.0; gst-inspect-1.0 nvinfer sees a plugin.
There is also nix run .#ds-yolo-run -- ./yolov5.onnx ./video.mp4 who, being run inside devshell, tries to run a full pipeline.
However, an attempt to do this produces error messages like these:
log
GLib (gthread-posix.c): Unexpected error from C library during 'pthread_setspecific': Invalid argument. Aborting.
GLib (gthread-posix.c): Unexpected error from C library during 'pthread_setspecific': Invalid argument. Aborting.
(gst-plugin-scanner:3878334): GLib-GObject-CRITICAL **: 15:43:23.935: object GstNvInfer 0xd7df80 finalized while still in-construction
(gst-plugin-scanner:3878334): GLib-GObject-CRITICAL **: 15:43:23.935: object GstNvInfer 0xd7fe70 finalized while still in-construction
(gst-plugin-scanner:3878334): GLib-GObject-CRITICAL **: 15:43:23.935: object GstNvDsOsd 0xda57b0 finalized while still in-construction
...
(gst-plugin-scanner:3878334): GLib-GObject-CRITICAL **: 15:43:23.936: object GstNvMsgBroker 0xdbf5d0 finalized while still in-construction
(gst-plugin-scanner:3878334): GStreamer-WARNING **: 15:43:23.938: Failed to load plugin '/nix/store/qd4qy89axpd2siq6x5pjl7ga17b8fzpa-deepstream-7.1/lib/gstreamer-1.0/libnvdsgst_eglglessink.so': libnvidia-ml.so.1: cannot open shared object file: No such file or directory
(gst-plugin-scanner:3878334): GStreamer-WARNING **: 15:43:23.941: Failed to load plugin '/nix/store/qd4qy89axpd2siq6x5pjl7ga17b8fzpa-deepstream-7.1/lib/gstreamer-1.0/libnvdsgst_inferserver.so': libtritonserver.so: cannot open shared object file: No such file or directory
(gst-plugin-scanner:3878334): GStreamer-WARNING **: 15:43:24.007: Failed to load plugin '/nix/store/qd4qy89axpd2siq6x5pjl7ga17b8fzpa-deepstream-7.1/lib/gstreamer-1.0/libnvdsgst_udp.so': librivermax.so.1: cannot open shared object file: No such file or directory
Error: Could not get cuda device count (cudaErrorStubLibrary)
Failed to parse group property
** ERROR: <gst_nvinfer_parse_config_file:1392>: failed
WARNING: erroneous pipeline: no element "nvv4l2decoder"
Clearly there many problems.
First, from strace it looks like it loads a stub library libcuda.so.1 from cudatoolkit instead of true one from /run/opengl-driver/lib. Let’s start here.
I’ve updated the gist with some fixes. The main one is that auto-path hooks should go to nativeBuildInputs (were misplaced to buildInputs); and also make install didn’t seem to actually replace already existing libraries. I’m also try to use nvh264{dec,enc} instead of v4l2- versions (who doesn’t compile yet).
Now it warns only about nvh264enc, although it comes from gst-plugins-bad and moreover, it has been seen in an environment without deepstream, iirc.
Logs with GST_DEBUG=3 states it doesn’t see libnvidia-encode.so.1, libnvcuvid.so.1, but they both exist in /run/opengl-driver/lib. Gstreamer has a special patch to find them for https://github.com/NixOS/nixpkgs/issues/189217.
/tmp/nix-shell.ngIF3B/tmp.WY31TrQQYp
Using pipeline:
----
nvstreammux name=mux live-source=true width=1920 height=1080 batch-size=1
filesrc location=./video.mp4 ! qtdemux ! h264parse ! nvh264dec !
nvvideoconvert ! mux.sink_0
mux. !
nvinfer config-file-path=./infer.txt !
nvtracker
ll-lib-file=./deepstream/lib/libnvds_nvmultiobjecttracker.so
ll-config-file=./NvSORT.yml !
nvstreamdemux name=demux
demux.src_0 ! nvvideoconvert !
nvdsosd display-text=1 process-mode=1 display-bbox=true ! nvvideoconvert !
nvh264enc ! h264parse ! qtmux ! filesink location=./result.mp4
^^^^
0:00:00.030666281 1176038 0x70c8a0 WARN default gstjackloader.c:187:gst_jack_load_library: Could not open library libjack.so.0, libjack.so.0: cannot open shared object file: No such file or directory
0:00:00.030681861 1176038 0x70c8a0 WARN default gstjack.c:108:plugin_init: Failed to load jack library
0:00:00.112943290 1176038 0x70c8a0 WARN ladspa gstladspa.c:509:plugin_init:<plugin227> no LADSPA plugins found, check LADSPA_PATH
0:00:00.121344162 1176038 0x70c8a0 WARN nvenc gstnvenc.c:928:gst_nvenc_load_library: Could not open library libnvidia-encode.so.1, libnvcuvid.so.1: cannot open shared object file: No such file or directory
0:00:00.593088540 1176038 0x70c8a0 WARN cudanvrtc gstcudanvrtc.cpp:152:gst_cuda_nvrtc_load_library_once: Could not open nvrtc library libnvrtc.so: cannot open shared object file: No such file or directory
0:00:00.920784097 1176038 0x70c8a0 WARN vadisplay gstvadisplay_drm.c:142:gst_va_display_drm_create_va_display:<vadisplaydrm0> Failed to open /dev/dri/renderD128: Permission denied
0:00:00.920821461 1176038 0x70c8a0 WARN vadisplay gstvadisplay_drm.c:142:gst_va_display_drm_create_va_display:<vadisplaydrm1> Failed to open /dev/dri/renderD129: Permission denied
0:00:00.920833437 1176038 0x70c8a0 WARN vadisplay gstvadisplay_drm.c:142:gst_va_display_drm_create_va_display:<vadisplaydrm2> Failed to open /dev/dri/renderD130: Permission denied
0:00:00.927530495 1176038 0x70c8a0 WARN default gstsfelement.c:97:gst_sf_create_audio_template_caps: format 0x120000: 'AVR (Audio Visual Research)' is not mapped
0:00:00.927542665 1176038 0x70c8a0 WARN default gstsfelement.c:97:gst_sf_create_audio_template_caps: format 0x180000: 'CAF (Apple Core Audio File)' is not mapped
0:00:00.927563261 1176038 0x70c8a0 WARN default gstsfelement.c:97:gst_sf_create_audio_template_caps: format 0x100000: 'HTK (HMM Tool Kit)' is not mapped
0:00:00.927569592 1176038 0x70c8a0 WARN default gstsfelement.c:97:gst_sf_create_audio_template_caps: format 0xc0000: 'MAT4 (GNU Octave 2.0 / Matlab 4.2)' is not mapped
0:00:00.927574557 1176038 0x70c8a0 WARN default gstsfelement.c:97:gst_sf_create_audio_template_caps: format 0xd0000: 'MAT5 (GNU Octave 2.1 / Matlab 5.0)' is not mapped
0:00:00.927579543 1176038 0x70c8a0 WARN default gstsfelement.c:97:gst_sf_create_audio_template_caps: format 0x210000: 'MPC (Akai MPC 2k)' is not mapped
0:00:00.927585126 1176038 0x70c8a0 WARN default gstsfelement.c:97:gst_sf_create_audio_template_caps: format 0x230000: 'MPEG-1/2 Audio' is not mapped
0:00:00.927591102 1176038 0x70c8a0 WARN default gstsfelement.c:97:gst_sf_create_audio_template_caps: format 0xe0000: 'PVF (Portable Voice Format)' is not mapped
0:00:00.927596868 1176038 0x70c8a0 WARN default gstsfelement.c:97:gst_sf_create_audio_template_caps: format 0x160000: 'SD2 (Sound Designer II)' is not mapped
0:00:00.927603493 1176038 0x70c8a0 WARN default gstsfelement.c:97:gst_sf_create_audio_template_caps: format 0x190000: 'WVE (Psion Series 3)' is not mapped
0:00:00.940651804 1176038 0x70c8a0 WARN vadisplay gstvadisplay_drm.c:142:gst_va_display_drm_create_va_display:<vadisplaydrm3> Failed to open /dev/dri/renderD128: Permission denied
0:00:00.940670710 1176038 0x70c8a0 WARN vadisplay gstvadisplay_drm.c:142:gst_va_display_drm_create_va_display:<vadisplaydrm4> Failed to open /dev/dri/renderD129: Permission denied
0:00:00.940696826 1176038 0x70c8a0 WARN vadisplay gstvadisplay_drm.c:142:gst_va_display_drm_create_va_display:<vadisplaydrm5> Failed to open /dev/dri/renderD130: Permission denied
0:00:01.233419242 1176036 0x6fbdf0 WARN nvenc gstnvenc.c:928:gst_nvenc_load_library: Could not open library libnvidia-encode.so.1, libnvcuvid.so.1: cannot open shared object file: No such file or directory
0:00:01.729702696 1176036 0x6fbdf0 WARN cudanvrtc gstcudanvrtc.cpp:152:gst_cuda_nvrtc_load_library_once: Could not open nvrtc library libnvrtc.so: cannot open shared object file: No such file or directory
0:00:02.360230893 1176036 0x6fbdf0 ERROR GST_PIPELINE gst/parse/grammar.y:630:gst_parse_element_make: no element "nvh264enc"
0:00:02.360346878 1176036 0x6fbdf0 ERROR GST_PIPELINE gst/parse/grammar.y:1327:priv_gst_parse_yyparse: link has no sink [source=@0xff3410]
0:00:02.360626378 1176036 0x6fbdf0 ERROR GST_PIPELINE gst/parse/grammar.y:1327:priv_gst_parse_yyparse: link has no source [sink=@0xff4a10]
WARNING: erroneous pipeline: no element "nvh264enc"
g++ -c -o nvdsinfer_func_utils.o -fPIC -Wno-deprecated-declarations -std=c++14 -I /usr/local/cuda-12.2/include -I ../../includes -DNDEBUG nvdsinfer_func_utils.cpp
nvdsinfer_func_utils.cpp: In function 'std::string nvdsinfer::dataType2Str(nvinfer1::DataType)':
nvdsinfer_func_utils.cpp:164:34: error: 'kINT64' is not a member of 'nvinfer1::DataType'; did you mean 'kINT8'?
164 | case nvinfer1::DataType::kINT64:
| ^~~~~~
| kINT8
make: *** [Makefile:62: nvdsinfer_func_utils.o] Error 1
FAILED on /build/root/opt/nvidia/deepstream/deepstream-7.1/sources/libs/nvdsinfer
kINT64 comes from a newer TensorRT 10. So with a nixos-unstable channel the latest version of my gist a number of compiling plugins is enough to run a pipeline, perform some inference and draw bounding boxes on a video. These boxes look randomly, probably some format or postprocessing issues.
By the way, I haven’t managed out the problem with boxes. I gave it an ONNX model which just reshapes input and added printing results; it looks like input for a model is messed somehow.
Before going to debug their code it would be nice to compile with supported version of CUDA (12.6 instead of 12.3). When I have more time, hopefully I give a try to https://nixos.org/manual/nixpkgs/stable/#cuda for upgrading. But it begins to look too cumbersome…
It may also be an incompatibility with a newer Gstreamer, in this case I completely lost.