Hardware acceleration on Firefox

There is an ongoing effort to enable hardware accelerated video playback on Firefox.

Has anybody tried to get this working on NixOS?

It appears that Firefox cannot find the appropriate libva libraries.

[Child 7022: Main Thread]: D/PlThis text will be hiddenatformDecoderModule VA-API support: Missing or old libva-wayland.so.2 library

This is on NixOS 20.03 with the firefox-wayland package from the nixos-20.03 channel , installed via either environment.systemPackages or nix-env as a regular user, but other firefox packages yield the same result.

See Accelerated Video Playback - NixOS Wiki and Hardware video acceleration - ArchWiki

Hi pasqui23, thanks for your suggestion.

Did this already… on e.g. Arch Linux or Fedora this is working as expected, on NixOS hardware accelerated video via e.g. mpv or vainfo is also working fine. It’s just Firefox in this case.

The arch wiki says that it works only on wayland and plasma on nixos is still struck on x11 Plasma5: Use upstream xsession/wayland-session files by bkchr · Pull Request #56988 · NixOS/nixpkgs · GitHub

Ah, i should have mentioned this in the first post… I don’t know much about Plasma and which compositor library KDE uses, but apparently to the relevant Bugzilla entries, GNOME’s Mutter and also, what i’m using here, sway with wlroots do support zwp_linux_dmabuf, which is necessary for DMABUF on Firefox. This very same setup is working just fine on Arch Linux.

Yes, unfortunately Firefox on NixOS currently doesn’t support VA-API as no one had enough time/interest to update the build expression so far. But firefox-bin already supports it (PR 81917) if that would work for you as well.

For firefox 915cb09e81c75e550c95568d24aa9cbefd926b9d should already do the job but IIRC it didn’t work in practice.

Update: Just re-read the OP:

[Child 7022: Main Thread]: D/PlThis text will be hiddenatformDecoderModule VA-API support: Missing or old libva-wayland.so.2 library

If Firefox is already looking for the library then the linked PR should be enough, but that change wasn’t backported to 20.03. You could test installing firefox-wayland from nixos-unstable if you want.

Firefox with VA-API on X11 is coming in version 80 Firefox 80 To Support VA-API Acceleration On X11 - Phoronix

2 Likes

You are right, thanks a bunch for the hint.
Firefox indeed seems to find the appropriate libs now and initializes VA-API decoding correctly.

libva info: VA-API version 1.7.0
libva info: User environment variable requested driver 'i965'
libva info: Trying to open /run/opengl-driver/lib/dri/i965_drv_video.so
..
libva info: Found init function __vaDriverInit_1_5
libva info: va_openDriver() returns 0
..
[Child 1494: MediaPDecoder #1]: D/PlatformDecoderModule VA-API FFmpeg init successful
...
[Child 1494: MediaPDecoder #3]: D/PlatformDecoderModule Got one VAAPI frame output with pts=80333 dts=80333 duration=40000 opaque=-9223372036854775808

But, Martin Stransky has this [1] blogpost up about VA-API on Firefox, where it states that Window Protocol should be “wayland/drm” instead of just “wayland”. Obviously, i don’t know about the implications of this, just wondering if other perhaps got different results. I’m on NixOS-20.03 and perhaps mesa 19.3 from release is just to old.

[1] Firefox on Fedora finally gets VA-API on Wayland. – Martin Stransky's Blog

1 Like

where it states that Window Protocol should be “wayland/drm” instead of just “wayland”.

I don’t know why it says wayland instead of wayland/drm. I’ve run across this multiple times and can’t seem to tell what difference it makes, or why.

BTW: VA-API seems to just work with firefox-wayland if you enable WR and VAAPI (gfx.webrender.enabled) and (widget.wayland-dmabuf-vaapi.enabled).

Also, I rebased and would like us to reconsidering merging this: firefox-wrapper: rename gdkWayland->forceWayland; always use libglvnd by colemickens · Pull Request #73365 · NixOS/nixpkgs · GitHub and make libglvnd available to firefox by default. In my opinion, we should encourage people to set MOZ_ENABLE_WAYLAND=1 and use firefox and keep firefox-wayland as legacy until MOZ_ENABLE_WAYLAND is deprecated in favor of an inverted flag.

It also has the benefit of making using firefox the same as using the other firefox derivations that respect the same convention. firefox-wayland is the odd-ball now.

I kind of wonder if others were using firefox like me and couldn’t get vaapi working because WR wasn’t actually working right…

3 Likes

I’m on Firefox 77.0.1 with firefox-wayland from nixos-unstable now and it appears that VAAPI is indeed used, where it applies, according to intel_gpu_top measurements.

Waiting for Firefox 78 hitting the shelves now, as there have been important performance fixes [1] gone into this release, especially around stuttering/flickering with UI-elements on-screen.

About the Window Protocol: This is just a guess, but perhaps VAAPI is indeed used to decode the frames, but Firefox still copies the decoded frames from the GPU to the CPU and back again without “wayland/drm”.

Anyhow, @primeos is correct in this context, as FF from unstable has now added libva to the wrapper libs, also, this whole FF hardware acceleration topic is still under heavy development and today still is dependend on many other factors.

@colemickens There have been issues [2] around opengl in firefox lately, which webrender is dependend upon.

[1] https://bugzilla.mozilla.org/show_bug.cgi?id=1619882
[2] Firefox no WebGL on Intel hardware: FEATURE_FAILURE_GLXTEST_FAILED · Issue #81514 · NixOS/nixpkgs · GitHub

Here’s the explanation and the fix for wayland vs wayland/drm. PTAL if you’re interested, I’d love some feedback to help confidence for merging. This change does NOT require rebuilding Firefox. firefox{,-bin}: add 'mesa', for wayland/drm by colemickens · Pull Request #92823 · NixOS/nixpkgs · GitHub

I’d love to get this PR and firefox-wrapper: rename gdkWayland->forceWayland; always use libglvnd by colemickens · Pull Request #73365 · NixOS/nixpkgs · GitHub merged and then do another pass on our Firefox builds. I’ve seen discussion that, in most distros, likely including NixOS, firefox-bin is better than firefox because Mozilla is better at producing optimized builds of Firefox. While I don’t doubt that, I’d like to close the gap, I’m in Nix, I want the source-built option! :slight_smile:

2 Likes

Awesome, I just tested VA-API with firefox-wayland again and it now works on my setup as well (including VP9) :slight_smile: My CPU usage is WAY lower now.

Thank you so much for all your work and awesome PRs @colemickens I really appreciate it! :heart:

So now that the PR are merged we should use firefox instead of firefox-wayland and just enable gfx.webrender.enabled and widget.wayland-dmabuf-vaapi.enabled in about:config and that’s it?

How do you test that it’s working? With the CPU usage? I don’t seem to see much of a difference.

“ VA-API FFmpeg init successful ” claims the VA-API is up and running, VP9 is the video format and “ Got one VAAPI frame output… ” line confirms that frame decoding works.
Firefox on Fedora finally gets VA-API on Wayland. – Martin Stransky's Blog

With that?

After these changes hit your nixpkgs:

  • sway, firefox-wayland = Firefox (Wayland)
  • sway, env MOZ_ENABLE_WAYLAND=1 firefox = Firefox (Wayland)
  • sway, unset MOZ_ENABLE_WAYLAND; firefox = Firefox (X11,Xwayland)

The only difference in firefox-wayland is that it sets MOZ_ENABLE_WAYLAND=1 in the wrapper. (And I guess we put “(Wayland)” into the Application title in the Desktop unit.)

As for confirming, I just repeated these instructions with a fresh profile:

Hopefully this can give you an idea if it’s working or not. Change the nix-build to account for your nixpkgs or NIX_PATH.

Important note: I have MOZ_ENABLE_WAYLAND set desktop-wide. You will need to set it as well, or modify the command below to use the firefox-wayland attribute. In fact, to be safe, I’m going to do that here:

  • export MOZ_ENABLE_WAYLAND=1
    export MOZ_LOG="PlatformDecoderModule:5";
    f="$(nix-build ~/code/nixpkgs/cmpkgs -A firefox)"
    $f/bin/firefox --no-remote -P vaapi 'http://demo.nimius.net/video_test/' &> /tmp/firefox 
    
  • Create a new profile vaapi for testing, that command launches that profile.

  • about:config, enable widget.wayland-dmabuf-vaapi.enabled=true, gfx.webrender.enabled=true, restart.

  • The launch command should’ve had fIrefox load a page that has a test video at the top that should work. Play it for a few seconds. Close firefox.

Then I open nvim /tmp/firefox/ and grep for:

  • VA-API FFmpeg init successful
  • D/PlatformDecoderModule Got one VAAPI frame output with

They are both present.

Note, Firefox Nightly users might not want to aggressively enable all dmabuf/drm options. My stable profile has all related options enabled, and it worked well, but Nightly has even more knobs available and some didn’t work out well with my system (though do note, we have libva, mesa and other related changes coming soon to nixos-unstable that might help too).

1 Like

Thanks! I guess there’s a difference:

vp09 (software)

avc1 (H.264) (hardware)

oh the scale is not the same. :laughing:

EDIT: my graph was with youtube. I’ll try your test with http://demo.nimius.net/video_test/ next.

Couple other tips:

  1. To force YouTube to play h264: enhanced-h264ify – Get this Extension for 🦊 Firefox (en-US)

  2. experiment with LIBVA_DRIVER_NAME={i965|iHD|etc}.

1 Like

Anyone got this to work with X11? I’m trying with two Thinkpad T520s, the other works with Arch Linux. With Arch, if I run with MOZ_LOG="PlatformDecoderModule:5" I can see VAAPI in the messages, when playing a video from Youtube. With NixOS I don’t see it. I use the enhanced-h264ify addon.
Edit: not working on a Slimbook with AMD Ryzen 4800H either.

Here is a summarised version of my configuration.nix:

{ config, pkgs, ... }:

{
  imports =
    [
      ./hardware-configuration.nix
     (import "${builtins.fetchTarball https://github.com/rycee/home-manager/archive/master.tar.gz}/nixos")
    ];

  boot.kernelPackages = pkgs.linuxPackages_latest;

  environment.systemPackages = with pkgs; [
    firefox
  ];

  system.stateVersion = "21.03";

  hardware.opengl = {
    enable = true;
    extraPackages = with pkgs; [
      vaapiIntel # LIBVA_DRIVER_NAME=i965
      vaapiVdpau
      libvdpau-va-gl
    ];
  };

  home-manager.users.myuser = {
    home.sessionVariables = {
      MOZ_X11_EGL = "1";
      LIBVA_DRIVER_NAME = "i965";
    };
    programs.firefox.enable = true;
    programs.firefox.profiles = {
      myuser = {
        id = 0;
        settings = {
            "media.ffmpeg.vaapi.enabled" = true;
            "media.ffvpx.enabled" = false;
            "media.av1.enabled" = false;
            "gfx.webrender.all" = true;
        };
      };
    };
  };
}

What version of firefox are you running? VA-API on X11 is only working from v81 onwards. You probably already have >v81, but worth a check?

Also seems that VA-API support is only included if the firefox package was built with wayland support. X11 VAAPI depends on Wayland

Settings Checklist:

  • Required prefs are: media.ffmpeg.vaapi.enabled=true (use hw video decoding), media.ffvpx.enabled=false (use system ffmpeg), gfx.webrender.all=true (use OpenGL rendering)
  • env vars: MOZ_X11_EGL=1 is correct and required for VAAPI. MOZ_LOG=“PlatformDecoderModule:5” is of course only needed if you want to check the log.
  • VAAPI is definitely broken in 80 (bug 1656436), technically the first VAAPI on X11 release. It was left broken for Wayland.
    Please try Beta 81 or Nightly 82.
  • enhanced-h264ify – Get this Extension for 🦊 Firefox (en-US) only works on YouTube, but disabling VP9 means video resolutions above 1080p are not available.
  • With sudo apt install intel-gpu-tools; sudo intel_gpu_top you can see hardware video decoding workload (“Video”).

Looks like you have all of these covered already though…

FF version is 81 on all machines and as I mentioned it is provably working on Arch Linux.

Interesting to see that bug report. They say it requires a “GTK built with Wayland support”. In the Firefox Nix package I see

assert forceWayland -> (browser ? gtk3); # Can only use the wayland backend if gtk3 is being used

So what is the deal here? Is it about the build setup or a runtime thing?