Jellyfin, CUDA & NixOS : Unable to transcode

I am posting here in the hopes that someone can see my silly mistake and go on my way (feeling silly but with one less problem). Any help is welcomed !

Description of the bug

I am faced with exactly the same bug as jellyfin: NVENC transcoding does not work · Issue #152008 · NixOS/nixpkgs · GitHub

Of course just as in jellyfin: NVENC ffmpeg fails when called by jellyfin but works as user · Issue #141298 · NixOS/nixpkgs · GitHub , if I sudo -u jellyfin <ffmpeg command bellow> everything runs so…

The transcoding logs are as follow (for the SEO gods if they still exist) :


ffmpeg version 7.1.4-Jellyfin Copyright (c) 2000-2026 the FFmpeg developers
  built with gcc 15.3.0 (GCC)
  configuration: --disable-static --prefix=/nix/store/eeeeeeeeeeeeeeeeeeeeeeeeeeeeeeee-jellyfin-ffmpeg-7.1.4-3 --target_os=linux --arch=x86_64 --pkg-config=pkg-config --enable-gpl --enable-version3 --disable-nonfree --disable-static --enable-shared --enable-pic --disable-thumb --disable-small --enable-runtime-cpudetect --enable-gray --enable-swscale-alpha --enable-hardcoded-tables --enable-safe-bitstream-reader --enable-pthreads --disable-w32threads --disable-os2threads --enable-network --enable-pixelutils --datadir=/nix/store/mbkxhdxm9jg7fm4x5zc3919nfki9q2rv-jellyfin-ffmpeg-7.1.4-3-data/share/ffmpeg --enable-ffmpeg --enable-ffplay --enable-ffprobe --bindir=/nix/store/eeeeeeeeeeeeeeeeeeeeeeeeeeeeeeee-jellyfin-ffmpeg-7.1.4-3-bin/bin --enable-avcodec --enable-avdevice --enable-avfilter --enable-avformat --enable-avutil --enable-postproc --enable-swresample --enable-swscale --libdir=/nix/store/eeeeeeeeeeeeeeeeeeeeeeeeeeeeeeee-jellyfin-ffmpeg-7.1.4-3-lib/lib --incdir=/nix/store/eeeeeeeeeeeeeeeeeeeeeeeeeeeeeeee-jellyfin-ffmpeg-7.1.4-3-dev/include --enable-doc --enable-htmlpages --enable-manpages --mandir=/nix/store/eeeeeeeeeeeeeeeeeeeeeeeeeeeeeeee-jellyfin-ffmpeg-7.1.4-3-man/share/man --enable-podpages --enable-txtpages --docdir=/nix/store/eeeeeeeeeeeeeeeeeeeeeeeeeeeeeeee-jellyfin-ffmpeg-7.1.4-3-doc/share/doc/ffmpeg --enable-alsa --enable-amf --enable-libaom --enable-libaribb24 --enable-libaribcaption --enable-libass --enable-avisynth --enable-libbluray --enable-libbs2b --enable-bzlib --enable-libcaca --enable-libcdio --enable-libcelt --enable-chromaprint --enable-libcodec2 --enable-cuda --enable-cuda-llvm --disable-cuda-nvcc --enable-cuvid --enable-libdav1d --enable-libdavs2 --enable-libdc1394 --enable-libdrm --enable-libdvdnav --enable-libdvdread --disable-libfdk-aac --enable-ffnvcodec --enable-libflite --enable-fontconfig --enable-libfontconfig --enable-libfreetype --enable-frei0r --enable-libfribidi --enable-libgme --enable-gmp --enable-gnutls --enable-libgsm --enable-libharfbuzz --enable-iconv --enable-libilbc --enable-libjack --enable-libjxl --enable-libkvazaar --enable-ladspa --enable-liblc3 --enable-liblcevc-dec --enable-lcms2 --enable-lzma --disable-metal --disable-libmfx --enable-libmodplug --enable-libmp3lame --enable-libmysofa --disable-libnpp --enable-nvdec --enable-nvenc --enable-openal --enable-opencl --enable-libopencore-amrnb --enable-libopencore-amrwb --enable-opengl --enable-libopenh264 --enable-libopenjpeg --enable-libopenmpt --enable-libopus --enable-libplacebo --enable-libpulse --enable-libqrencode --enable-libquirc --enable-librav1e --enable-librist --disable-librtmp --enable-librubberband --enable-libsmbclient --enable-sdl2 --enable-libshaderc --enable-libshine --enable-libsnappy --enable-libsoxr --enable-libspeex --enable-libsrt --enable-libssh --enable-librsvg --enable-libsvtav1 --disable-libtensorflow --enable-libtheora --enable-libtwolame --enable-libuavs3d --enable-libv4l2 --enable-v4l2-m2m --enable-vaapi --enable-vdpau --enable-libvpl --enable-libvidstab --enable-libvmaf --enable-libvo-amrwbenc --enable-libvorbis --enable-libvpx --enable-vulkan --enable-libvvenc --enable-libwebp --enable-libx264 --enable-libx265 --enable-libxavs --enable-libxavs2 --enable-libxcb --enable-libxcb-shape --enable-libxcb-shm --enable-libxcb-xfixes --enable-libxevd --enable-libxeve --enable-xlib --enable-libxml2 --enable-libxvid --enable-libzimg --enable-zlib --enable-libzmq --enable-libzvbi --disable-debug --enable-optimizations --disable-extra-warnings --disable-stripping --nvcc=/nix/store/eeeeeeeeeeeeeeeeeeeeeeeeeeeeeeee-clang-21.1.8/bin/clang --extra-version=Jellyfin --disable-ptx-compression
  libavutil      59. 39.100 / 59. 39.100
  libavcodec     61. 19.101 / 61. 19.101
  libavformat    61.  7.103 / 61.  7.103
  libavdevice    61.  3.100 / 61.  3.100
  libavfilter    10.  5.100 / 10.  5.100
  libswscale      8.  3.100 /  8.  3.100
  libswresample   5.  3.100 /  5.  3.100
  libpostproc    58.  3.100 / 58.  3.100
[AVHWDeviceContext @ 0x5d84b3b6ba80] cu->cuInit(0) failed -> CUDA_ERROR_NO_DEVICE: no CUDA-capable device is detected
Device creation failed: -542398533.
Failed to set value 'cuda=cu:0' for option 'init_hw_device': Generic error in an external library
Error parsing global options: Generic error in an external library

Dead ends (Troubleshooting attempts)

services.jellyfin = {
  enable = true;
  forceEncodingConfig = true;
  hardwareAcceleration = {
    enable = true;
    device = "/dev/dri/renderD128";
    type = "nvenc";
  };
  transcoding = {
    enableHardwareEncoding = true;
    hardwareDecodingCodecs = {
      h264 = true;
      hevc = true;
      hevc10bit = true;
      mpeg2 = true;
      vc1 = true;
      vp8 = true;
      vp9 = true;
    };
    hardwareEncodingCodecs.hevc = true;
  };
  openFirewall = true;
};

Still no difference !

System information

[user@host ~]$ nix-shell -p nix-info --run "nix-info -m" 
- system: `"x86_64-linux"`
 - host os: `Linux 6.18.44, NixOS, 26.11 (Zokor), 26.11pre-git`
 - multi-user?: `yes`
 - sandbox: `yes`
 - version: `nix-env (Nix) 2.35.2`
 - channels(root): `"nixos, nixpkgs"`
 - nixpkgs: `/nix/store/nglfj5i11zigpbgvw51n5g699y5gi6r9-nixpkgs/nixpkgs`

Relevant system configuration

Jellyfin

Jellyfin reported to be v.10.11.11 (latest to date on nixpkgs)

{ config, pkgs, lib, ... }:

with lib;

{
  config = {
    users.users.jellyfin.extraGroups = [ "video" "render" ];
    services.jellyfin = {
      enable = true;
      openFirewall = true;
      serviceConfig = {
          SystemCallFilter = mkForce [];
      };
    };
  };
}

GPU

{ config
, pkgs
, lib
, ...
}:
let
  inherit (lib) mkEnableOption mkIf;

  nvidia-offload = pkgs.writeShellScriptBin "nvidia-offload" ''
    export __NV_PRIME_RENDER_OFFLOAD=1
    export __NV_PRIME_RENDER_OFFLOAD_PROVIDER=NVIDIA-G0
    export __GLX_VENDOR_LIBRARY_NAME=nvidia
    export __VK_LAYER_NV_optimus=NVIDIA_only
    exec -a "$0" "$@"
  '';
in
{

  config = mkIf config.perso.hardware.gpu.nvidia.enable {
    services.xserver.videoDrivers = [ "nvidia" ];

    nixpkgs.config.allowUnfree = true;
    nixpkgs.config.cudaSupport = true;
    nix.settings = {
      substituters = [ "https://cache.nixos-cuda.org" ];
      trusted-public-keys = [ "cache.nixos-cuda.org:74DUi4Ye579gUqzH4ziL9IyiJBlDpMRn9MBN8oNan9M=" ];
    };

    environment.systemPackages = [ nvidia-offload pkgs.nvitop ];
    services.udev.packages = with pkgs; [ gnome-settings-daemon ];

    hardware = {
      nvidia = {
        open = true;
        package = config.boot.kernelPackages.nvidiaPackages.beta;
      };
      graphics = {
        enable = true;
        enable32Bit = true; # To enable Vulkan on 32 bit applications
      };
    };

    hardware.nvidia-container-toolkit.enable = true;

    };
  };
}

1 Like

Have you tried setting services.jellyfin.hardwareAcceleration as described in the wiki?

See also services.jellyfin.transcoding.hardwareEncodingCodecs.

Yes !

I did : you’re right I should have put it in the “Dead ends” section.
The problem is actually not that the transcoding is software, the logs seems to indicate that hardware encoding is trying to happen but fails !

It definitely looks like systemd sandbox settings not being open enough for some reason, since you say the ffmpeg command works at the command line.

You could just start temporarily turning flags off until you see movement.

I don’t have the impression that a lot of people test with CUDA (not running it myself), so I’d be suspicious of PRs like this one: https://github.com/NixOS/nixpkgs/pull/469282 If you roll back/force off those additions, maybe one of them un-jams you.

Good news ! I was able to make it work !.

TLDR

Here is the patch to have :

  systemd.services."jellyfin".serviceConfig = lib.mkIf config.perso.jellyfin {
    SystemCallFilter = lib.mkForce [
      "@system-service"
      "ioctl"
      "~@privileged"
    ];
    DeviceAllow = lib.mkForce [
      "/dev/dri/renderD128 rw"
      "/dev/nvidia0 rw"
      "/dev/nvidiactl rw"
      "/dev/nvidia-uvm rw"
      "/dev/nvidia-modeset rw"
    ];
  };

Process

I am documenting my process for any poor soul (like me) that might cross this issue :

  1. I used strace to find out what syscalls where used by the command sudo -u jellyfin strace -c /nix/store/skqbhw6kkd6nsc164b3zzybjfvx2y8zs-jellyfin-ffmpeg-7.1.2-2-bin/bin/ffmpeg <rest of command given by jellyfin transcoding log here> to see if SystemCallFilter might be overzealous in its deny-list, thank you for the pointer @miangraham, (spoiler alert, it was), you’ll be greated by this lovely list :
syscall
-----------------------
futex
ioctl
openat
read
mmap
write
newfstatat
access
close
fstat
pselect6
mprotect
execve
readlink
brk
clone3
lseek
getdents64
rt_sigprocmask
stat
munmap
madvise
unlink
mkdir
fcntl
fstatfs
connect
getpid
pipe2
symlink
rt_sigaction
dup
eventfd2
chmod
sched_getaffinity
socket
pread64
bind
getrusage
statfs
sysinfo
setsockopt
get_mempolicy
geteuid
prlimit64
uname
listen
sched_get_priority_max
sched_get_priority_min
getrandom
arch_prctl
set_tid_address
set_robust_list
rseq
  1. Now use your patience to recursively check that ioctl is actually nowhere to be found in the allow list

  2. Realize that in the configuration of the service, none of /dev/nvidia* are included in the DeviceAllow which must be the device used for the NVENC & NVDEC

  3. ??

  4. Profit ?

I am somewhat surprised that I am the only one struck by this issue but I will put that on the count of my own stupidity that I document then here to save time to others !

3 Likes

Nice work! It might be worth a PR to get required permissions back into the module defaults and save some future CUDA users the headache. Wiki additions with your findings are probably helpful too.

AFAICT needing to add a bunch of your own lines to DeviceAllow is expected since the exact list depends on card specifics, but service permissions are something we should be able to keep working for basically everyone.

1 Like