System: NixOS 26.05 x86_64
WM: Xfwm4 (X11)
CPU: Intel N150
GPU: Integrated Graphics
I wrote this intending to ask for help, but I eventually figured it out on my own. After a week of hunting through the wiki’s, a bunch of unofficial wikis on increasingly shady domains, youtube videos, artificial stupidity, and, as a last resort, Stackoverflow and Reddit. It’s less of a guide and more of a catalog of how I eventually found my answer. Keep in mind I tried my usual approach of skimming the reference manual before reading source code to try learning Nix. I was confident having played with functional paradims in Lua, Javascript, multiple Schemes and even a little Haskell.
I tried using ffmpeg to screen capture as I’ve done on my previous system. However, I received this error informing me that x11grab was unavailable.
$ ffmpeg -video_size 1366x768 -framerate 25 -f x11grab -i :0.0 output.mkv
ffmpeg version 8.1.1 Copyright (c) 2000-2026 the FFmpeg developers
built with gcc 15.2.0 (GCC)
configuration: --disable-static --prefix=/nix/store/eeeeeeeeeeeeeeeeeeeeeeeeeeeeeeee-ffmpeg-8.1.1 --target_os=linux --arch=x86_64 --pkg-config=pkg-config --enable-gpl --enable-version3 --disable-nonfree --disable-static --enable-shared --enable-pic --disable-thumb --disable-small --enable-runtime-cpudetect --disable-gray --enable-swscale-alpha --enable-hardcoded-tables --enable-safe-bitstream-reader --enable-pthreads --disable-w32threads --disable-os2threads --enable-network --enable-pixelutils --datadir=/nix/store/50jahvv3gz6awqi0vh90sbyz894j95xh-ffmpeg-8.1.1-data/share/ffmpeg --enable-ffmpeg --enable-ffplay --enable-ffprobe --bindir=/nix/store/eeeeeeeeeeeeeeeeeeeeeeeeeeeeeeee-ffmpeg-8.1.1-bin/bin --enable-avcodec --enable-avdevice --enable-avfilter --enable-avformat --enable-avutil --enable-swresample --enable-swscale --libdir=/nix/store/eeeeeeeeeeeeeeeeeeeeeeeeeeeeeeee-ffmpeg-8.1.1-lib/lib --incdir=/nix/store/eeeeeeeeeeeeeeeeeeeeeeeeeeeeeeee-ffmpeg-8.1.1-dev/include --enable-doc --enable-htmlpages --enable-manpages --mandir=/nix/store/eeeeeeeeeeeeeeeeeeeeeeeeeeeeeeee-ffmpeg-8.1.1-man/share/man --enable-podpages --enable-txtpages --docdir=/nix/store/eeeeeeeeeeeeeeeeeeeeeeeeeeeeeeee-ffmpeg-8.1.1-doc/share/doc/ffmpeg --enable-alsa --enable-amf --enable-libaom --disable-libaribb24 --disable-libaribcaption --enable-libass --disable-avisynth --enable-libbluray --disable-libbs2b --enable-bzlib --disable-libcaca --disable-libcdio --disable-libcelt --disable-chromaprint --disable-libcodec2 --disable-cuda --enable-cuda-llvm --disable-cuda-nvcc --enable-cuvid --enable-libdav1d --disable-libdavs2 --disable-libdc1394 --enable-libdrm --disable-libdvdnav --disable-libdvdread --disable-libfdk-aac --enable-ffnvcodec --disable-libflite --enable-fontconfig --enable-libfontconfig --enable-libfreetype --disable-frei0r --enable-libfribidi --disable-libgme --enable-gmp --enable-gnutls --disable-libgsm --enable-libharfbuzz --enable-iconv --disable-libilbc --disable-libjack --disable-libjxl --disable-libkvazaar --disable-ladspa --disable-liblc3 --disable-liblcevc-dec --disable-lcms2 --enable-lzma --disable-metal --disable-libmfx --disable-libmodplug --enable-libmp3lame --disable-libmysofa --disable-libnpp --enable-nvdec --enable-nvenc --disable-openal --enable-liboapv --enable-opencl --disable-libopencore-amrnb --disable-libopencore-amrwb --disable-opengl --disable-libopenh264 --enable-libopenjpeg --enable-libopenmpt --enable-libopus --disable-libplacebo --enable-libpulse --disable-libqrencode --disable-libquirc --disable-librav1e --enable-librist --disable-librtmp --disable-librubberband --disable-libsmbclient --enable-sdl2 --disable-libshaderc --disable-libshine --disable-libsnappy --enable-libsoxr --enable-libspeex --enable-libsrt --enable-libssh --disable-librsvg --enable-libsvtav1 --disable-libtensorflow --enable-libtheora --disable-libtwolame --disable-libuavs3d --enable-libv4l2 --enable-v4l2-m2m --enable-vaapi --enable-vdpau --disable-libvpl --enable-libvidstab --disable-libvmaf --disable-libvo-amrwbenc --enable-libvorbis --enable-libvpx --enable-vulkan --disable-libvvenc --enable-libwebp --disable-whisper --enable-libx264 --enable-libx265 --disable-libxavs --disable-libxavs2 --disable-libxcb --disable-libxcb-shape --disable-libxcb-shm --disable-libxcb-xfixes --disable-libxevd --disable-libxeve --disable-xlib --enable-libxml2 --enable-libxvid --enable-libzimg --enable-zlib --disable-libzmq --enable-libzvbi --disable-debug --enable-optimizations --disable-extra-warnings --disable-stripping
libavutil 60. 26.101 / 60. 26.101
libavcodec 62. 28.101 / 62. 28.101
libavformat 62. 12.101 / 62. 12.101
libavdevice 62. 3.101 / 62. 3.101
libavfilter 11. 14.101 / 11. 14.101
libswscale 9. 5.101 / 9. 5.101
libswresample 6. 3.101 / 6. 3.101
[in#0 @ 0x5e312e40ccc0] Unknown input format: 'xllgrab'
Error opening input file :0.0.
Error opening input files: Invalid argument
What stood out to me was the –disable-xlib flag. I don’t remember doing that. I assumed at the time that it was related to x11grab. I then checked devices with ffmpeg -devices…
Devices:
D. = Demuxing supported
.E = Muxing supported
---
DE alsa ALSA audio output
DE fbdev Linux framebuffer
D kmsgrab KMS screen capture
D lavfi Libavfilter virtual input device
DE oss OSS (Open Sound System) playback
DE pulse Pulse audio output
DE video4linux2,v4l2 Video4Linux2 output device
Never heard of kmsgrab before. Tried replacing it with x11grab and got a similar error. The ffmpeg manual tells me that kmsgrab needs CAP_SYS_ADMIN or DRM master to function. The former is a super user privilege, which even without the capabilities man page’s warning strikes me as a bad idea. Looking up the later leads me to the drm man page which is of little help since it has multiple broken links (even on mirror sites) with the one functioning link leading to a library page (3) instead of an executable program (1).
I then looked it up in the packages. Wasn’t sure what I was looking at as the descriptions were identical with each package. Swapped to options search and there’s nothing pertaining to ffmpeg itself, just other packages that depend on ffmpeg. Found a package called capture which seemed to rely on ffmpeg based on the source. grabbed it. Still no x11grab.
Looking at the package source, xlib is listed a few times as an optional for the full variant of ffmpeg. The only warning was to not use overrideAttr, so I tried using stuff like
environment.systemPackages = with pkgs;[
...
(ffmpeg.override{withfull = true;})
...
];
Not a package error. I thought maybe I was using it wrong, so I looked at the nixpkg manual. Ctrl+f gave 568 references. Ok, lets try “overrideAttr“ just to see what its about. 78 references. “overriding” at least had a name drop in the anemic table of contents, so I looked at that.
pkgs.foo.override {
arg1 = val1;
arg2 = val2; # ...
}
So I’m not supposed to use the surrounding parenthesis?
ffmpeg.override{xlib = true;}
Not a package error.
So back to trying to figure out how activate these feature flags. In the source I see enableFeature and optionals, both are part of lib according to the nixpkgs manual which I already have as an import. Neither fit my need, as I’m working with a package. The comments warned against using overrideAttr, so lets try just overide…
(ffmpeg.override{withXlib = true;})
Returned errors about this not being a package. I tried putting it inside the nixpkgs.config. No dice.
I took another look at the package search. This time I saw “fmpeg-full” and “ffmpeg-headless“ and the correlation was now obvious. While that technically satisfies my needs by granting xlib (thus x11grab), ffmpeg-full also comes with tons of baggage I don’t need like cuda support. Sides, even the package comments suggest only activating the options that I need rather than using full.
Eventually I had a completely different issue (lua couldn’t find the path for packages) and noticed the pkgs prefix in the solution. So I randomly decided to append pkgs…
(pkgs.ffmpeg.override{withXlib = true;})
It started building ffmpeg from source. This was a scope issue this whole time? I tried looking at the nix manual to figure out the scope rules…
A scope in the Nix language is a dictionary keyed by name, mapping each name to an expression and a definition type.
Excuse me? In every other language I’ve used, scope is where a variable or function definition is valid. So even if I thought to look this up earlier, I would have his another wall.
After waiting for ffmpeg to build from source, I ffmpeg -devices again and still no x11grab. I look it up and… it’s been deprecated since v3.3 nearly a decade ago? And it was the Xcb flags I should have been looking for? ¿Que demonios de pasa? How many programs rely on this functionality? I just used it on my last system install late last year. So I update my config again…
(pkgs.ffmpeg.override{withXlib = true; withXcb = true;})
Wait another 20 minutes. Ran ffmpeg -devices one more time. Saw my beloved x11grab. ran my one liner again and… now the mouse is missing. Looked at the output. –disable-libxcb-xfixes, which enables mouse recording. Looks like I misread this line:
withXcb ? withXcbShm || withXcbxfixes || withXcbShape, # X11 grabbing using XCB
As “if withXcb is true, then withXcbShm or withXcbxfixes or withXcbShape will be true“. This actually means “withXcb will be true if any of the following are true“. Coming from C I thought something looked weird about these if statements. One more change…
(pkgs.ffmpeg.override{
withXcbxfixes = true;
withXcbShape = true;
withXcbShm = true;
})
Rebuilt again, and with one more line:
$ ffmpeg -video_size 1366x768 -framerate 25 -f x11grab -i :0.0 please-work-for-the-love-of-odin.mkv
I finally got my screen recorder, and as I have a good idea of how to fix capture. After a nap. I haven’t slept in days dealing with this mess.
