Running playwright tests

Hello,

I am unable to get playwright to run tests.

I am able to get the playwright cli command to run using the following commands:

export PLAYWRIGHT_BROWSERS_PATH=$(nix build --print-out-paths nixpkgs#playwright.browsers)
nix shell nixpkgs#playwright
playwright open nixos.org

However the playwright cli is missing the test command

Using playwright through npm/yarn also has problems:

yarn playwright install-deps expects apt, so it does not work

yarn playwright install
yarn playright test

produces errors like:

browserType.launch: 
╔══════════════════════════════════════════════════════╗
║ Host system is missing dependencies to run browsers. 
║ Missing libraries:                                   
║     libgobject-2.0.so.0                              
║     libglib-2.0.so.0                                 
export PLAYWRIGHT_BROWSERS_PATH=$(nix build --print-out-paths nixpkgs#playwright
.browsers)
yarn playwright test

produces errors like:

Executable doesn’t exist at /nix/store/21isr8iwrdkhdhdyjw4jfxmwv0cn5zja-playwright-browsers/chromium-1045/chrome-linux/chrome

Short of a fix, is there a common workaround for problems like this? I often run into these kinds of problems when using npm/yarn and beating my head against nix is hurting my productivity.

Any help would be much appreciated.

Sources:
https://github.com/microsoft/playwright/issues/5501
https://github.com/NixOS/nixpkgs/issues/215450

2 Likes

I’m facing same issues and after checking source code of the following packages : python39Packages.playwright , playwright and python310Packages.playwright I found all of them pointing into same source code https://github.com/NixOS/nixpkgs/blob/a3d745e701c337e65ef467d5a9400d9336a303a1/pkgs/development/python-modules/playwright/default.nix and it’s related to python only.

Also it still show the missed libs after including them:

  buildInputs = with nixpkgs; [
   ###### Playwright deps
    # libgobject-2.0.so.0
    # libglib-2.0.so.0
    # libgio-2.0.so.0
    glib
    # libnss3.so
    # libnssutil3.so
    # libsmime3.so
    nss
    # libnspr4.so
    nspr
    # libatk-1.0.so.0
    # libatk-bridge-2.0.so.0
    # libatspi.so.0
    at-spi2-core
    # libcups.so.2
    cups
    # libdrm.so.2
    libdrm
    # libdbus-1.so.3
    dbus
    # libX11.so.6
    xorg.libX11
    # libXcomposite.so.1
    xorg.libXcomposite
    # libXdamage.so.1
    xorg.libXdamage
    # libXext.so.6
    xorg.libXext
    # libXfixes.so.3
    xorg.libXfixes
    # libXrandr.so.2
    xorg.libXrandr
    # libgbm.so.1
    mesa
    # libexpat.so.1
    expat
    # libxcb.so.1
    xorg.libxcb
    # libxkbcommon.so.0
    libxkbcommon
    # libpango-1.0.so.0
    pango
    # libcairo.so.2
    cairo
    # libasound.so.2
    alsa-lib
    ######
   ....

It would be incredibly cool to be able to run playwright tests as part of nix flake check. The testScript attribute in pkgs.nixosTest is python code and playwright has python driver so it should be somehow possible no :slight_smile: ?

I am new to playwright so I might miss something but here is what I found Package request: playwright test · Issue #217693 · NixOS/nixpkgs · GitHub , ie., for the tests I think you must use the typescript version of playwright ?

At work we’re using python with playwright, though not in NixOS tests (yet). No problems.

but there is no playwright test command in hte python package:

➜ result/bin/playwright 
Usage: playwright [options] [command]

Options:
  -V, --version                          output the version number
  -h, --help                             display help for command

Commands:
  open [options] [url]                   open page in browser specified via -b, --browser
  codegen [options] [url]                open page and generate code for user actions
  install [options] [browser...]         ensure browsers necessary for this version of Playwright are installed
  install-deps [options] [browser...]    install dependencies necessary to run browsers (will ask for sudo permissions)
  cr [options] [url]                     open page in Chromium
  ff [options] [url]                     open page in Firefox
  wk [options] [url]                     open page in WebKit
  screenshot [options] <url> <filename>  capture a page screenshot
  pdf [options] <url> <filename>         save page as pdf
  show-trace [options] [trace...]        show trace viewer
  help [command]                         display help for command

what do you use it for ? do you write tests in python ? how do you launch them then ?

playwright is used as a library in python with pytest and will spawn a browser based on its python API.

We have a shell.nix that looks like this:

let
  python' = python3.withPackages (p: with p; [
    lxml
    mixins
    playwright
    pytest
    pytest-playwright
    pytest-pytestrail
  ]);
in
mkShell {
  name = "pythonEnv";

  packages = [
    python'
  ];

  shellHook = ''
    export PLAYWRIGHT_BROWSERS_PATH=${playwright.browsers}
  '';
}

From the source directory you enter the shell and run pytest with some arguments and the python code runs.

1 Like

I currently use this shell.nix for JS:

{ pkgs ? import <nixpkgs> {} }:
  pkgs.mkShell {
    buildInputs = [
      pkgs.nodejs-18_x
      pkgs.playwright
    ];

    PLAYWRIGHT_BROWSERS_PATH="${pkgs.playwright-driver.browsers}";
  }

Note that there is no playwright.browsers anymore, now it’s playwright-driver.browsers (see Build failure: playwright · Issue #215450 · NixOS/nixpkgs · GitHub).

I can then do a playwright open nixos.org, but I can’t run tests with npm or run a visual studio code instance for testing with the playwright extension. It always tells me to install the browsers with npx playwright install.

@pbek what if you try playwright-test: init at 1.31.1 by teto · Pull Request #227071 · NixOS/nixpkgs · GitHub ?

1 Like

Thank you, I’m on it!

Currently, this shell.nix is working for me:

{ pkgs ? import <nixpkgs> {} }:
  pkgs.mkShell {
    # nativeBuildInputs is usually what you want -- tools you need to run
    nativeBuildInputs = with pkgs; [
      playwright-driver.browsers
    ];

    shellHook = ''
      export PLAYWRIGHT_BROWSERS_PATH=${pkgs.playwright-driver.browsers}
      export PLAYWRIGHT_SKIP_VALIDATE_HOST_REQUIREMENTS=true
    '';
}
4 Likes

It took me forever to figure this out, but finally I managed to get Playwright to work in a Node.js project.

TL;DR:

  • replace playwright with playwright-core in your dependencies
  • set executablePath in you chromium launch options and point it to where it can find a chromium binary (run which chromium to figure this out)

Full explanation

Let’s take this simple script index.cjs as an example.

const { chromium } = require('playwright')

const main = async () => {
  const browser = await chromium.launch({
    headless: false
  })

  const ctx = await browser.newContext()
  const page = await ctx.newPage()
  await page.goto('https://www.reddit.com/r/NixOS/')
  await browser.close()
}

main()

If we try to execute the script with node index.cjs, we encounter an exception. It’s Playwright telling us that a bunch of shared objects are missing. To be more precise, when we are on any Linux distro it’s the validateLinuxDependencies that throws this exception.

By default, Playwright uses the chromium revision it bundles with. This is also what Puppeteer does.

Why bundle a chromium revision?

The Playwright team (like the Puppeteer team) bundles a specific chromium revision so then they can focus on supporting only the specific chromium revision they bundle with (the chromium revision can be found in the Playwright release notes).

Where can we find the browsers used by Playwright?

As they write in the documentation, on Linux all browsers bundled with Playwright are stored in ~/.cache/ms-playwright.

Double checking the required shared libraries

Now that we know where to find the browsers, we can double check which shared objects are required by the particular chromium revision used by Playwright:

ldd ~/.cache/ms-playwright/chromium-1091/chrome-linux/chrome

On my laptop (NixOS 24.05) I got this:

linux-vdso.so.1 (0x00007ffca03c5000)
libdl.so.2 => /nix/store/9y8pmvk8gdwwznmkzxa6pwyah52xy3nk-glibc-2.38-27/lib/libdl.so.2 (0x00007f2c90e98000)
libpthread.so.0 => /nix/store/9y8pmvk8gdwwznmkzxa6pwyah52xy3nk-glibc-2.38-27/lib/libpthread.so.0 (0x00007f2c90e93000)
libgobject-2.0.so.0 => not found
libglib-2.0.so.0 => not found
libnss3.so => not found
libnssutil3.so => not found
libsmime3.so => not found
libnspr4.so => not found
libdbus-1.so.3 => not found
libatk-1.0.so.0 => not found
libatk-bridge-2.0.so.0 => not found
libcups.so.2 => not found
libgio-2.0.so.0 => not found
libdrm.so.2 => not found
libexpat.so.1 => not found
libxcb.so.1 => not found
libxkbcommon.so.0 => not found
libatspi.so.0 => not found
libX11.so.6 => not found
libXcomposite.so.1 => not found
libXdamage.so.1 => not found
libXext.so.6 => not found
libXfixes.so.3 => not found
libXrandr.so.2 => not found
libgbm.so.1 => not found
libpango-1.0.so.0 => not found
libcairo.so.2 => not found
libasound.so.2 => not found
libm.so.6 => /nix/store/9y8pmvk8gdwwznmkzxa6pwyah52xy3nk-glibc-2.38-27/lib/libm.so.6 (0x00007f2c90da9000)
        libgcc_s.so.1 => /nix/store/ldrslljw4rg026nw06gyrdwl78k77vyq-xgcc-12.3.0-libgcc/lib/libgcc_s.so.1 (0x00007f2c90d88000)
libc.so.6 => /nix/store/9y8pmvk8gdwwznmkzxa6pwyah52xy3nk-glibc-2.38-27/lib/libc.so.6 (0x00007f2c81218000)
        /lib64/ld-linux-x86-64.so.2 => /nix/store/9y8pmvk8gdwwznmkzxa6pwyah52xy3nk-glibc-2.38-27/lib64/ld-linux-x86-64.so.2 (0x00007f2c90e9f000)

In fact, if we set PLAYWRIGHT_SKIP_VALIDATE_HOST_REQUIREMENTS = true in our nix shell and then try running node index.cjs, we bypass that nicely-formatted black box and get a more traditional stack trace. Here is the most relevant extract of the entire stack trace:

- <launched> pid=179437
- [pid=179437][err] Could not start dynamically linked executable: /home/jack/.cache/ms-playwright/chromium-1091/chrome-linux/chrome
- [pid=179437][err] NixOS cannot run dynamically linked executables intended for generic
- [pid=179437][err] linux environments out of the box. For more information, see:
- [pid=179437][err] https://nix.dev/permalink/stub-ld
- [pid=179437] <process did exit: exitCode=127, signal=null>
- [pid=179437] starting temporary directories cleanup

3 solutions and their pros and cons

Now that we have double-checked with ldd what validateLinuxDependencies already told us, we have a few options to make Playwright (and Nix) happy. I can think of at least three:

  1. install all shared objects in one way or another. For example, if we know that a particular nixpkgs package includes a shared object, we can declare that package in our nix shell packages. For example, we can declare pkgs.cairo and pkgs.pango to obtain libcairo.so.2 and libpango-1.0.so.0, respectively.
  2. adopt the solution proposed by @pbek, namely add pkgs.playwright-driver.browsers to the nativeBuildInputs and set export PLAYWRIGHT_BROWSERS_PATH=${pkgs.playwright-driver.browsers} in the shellHook (it doesn’t seem mandatory to set export PLAYWRIGHT_SKIP_VALIDATE_HOST_REQUIREMENTS = true in the shellHook).
  3. tell Playwright to use a version of chromium we explicitly specify, instead of the bundled one.

Option 1 seems too complicated to me. In some cases it might be easy to figure out how to get the shared object from a package (e.g. pkgs.cairo => libcairo.so.2), but others are more challenging. Also, having to list all of these packages in the nix shell is annoying.

Option 2 works, but it forces us to download and compile chromium (and maybe other browsers bundled with Playwright?). Downloading chromium is not a big deal, but compiling it takes forever (on my laptop it took an entire night).

Option 3 works and, as long as we have chromium on our machine, doesn’t require us to compile chromium. Also, in some cases we might need to rely on features not provided by the chromium revision bundled with Playwright, so we would need to use a custom chromium anyway. Let’s go for this option then!

Double checking the chromium revision installed via nixpkgs

I use Home Manager, and if I run which chromium I see that the chromium binary is at:

/home/jack/.nix-profile/bin/chromium

That location is actually a symlink, and if I run ls -l /home/jack/.nix-profile/bin/ I see that on my laptop chromium is somewhere in the nix store:

/nix/store/8595809xjaq1a04djljzp3r3h9ham4z4-chromium-120.0.6099.129/bin/chromium

What’s the difference between the chromium revision used by Playwright and the chromium revision found in my nix store? The former is dynamically linked, the latter is statically linked. Let’s double check using ldd:

ldd $(which chromium)

We get: not a dynamic executable.

Revisiting the JS script

Now that we have decided to not use the chromium revision bundled with Playwright, we can update the index.cjs script:

const { chromium } = require('playwright-core')

const main = async () => {
  const browser = await chromium.launch({
    // tip: use `which chromium` to figure out where the chromium binary is
    executablePath: '/home/jack/.nix-profile/bin/chromium',
    headless: false
  })

  const ctx = await browser.newContext()
  const page = await ctx.newPage()
  await page.goto('https://www.reddit.com/r/NixOS/')
  await browser.close()
}

main()

Note: by declaring playwright-core instead of playwright in our package.json, we avoid downloading the chromium revision bundled with Playwright. Another way to avoid downloading chromium is to set PLAYWRIGHT_SKIP_BROWSER_DOWNLOAD=1 before running npm install.

Finally, we can define a nix shell in this flake.nix:

{
  description = "Playwright demo";

  inputs = {
    nixpkgs.url = "github:nixos/nixpkgs/nixos-23.11";
    # nixpkgs.url = "github:nixos/nixpkgs/nixos-unstable";
  };

  outputs = {
    nixpkgs,
    self,
    ...
  } @ inputs: let
    overlays = [
      (final: prev: {
        nodejs = prev.nodejs_20;
      })
    ];
    supportedSystems = ["x86_64-linux" "aarch64-linux" "x86_64-darwin" "aarch64-darwin"];
    forEachSupportedSystem = f:
      nixpkgs.lib.genAttrs supportedSystems (system:
        f {
          pkgs = import nixpkgs {inherit overlays system;};
        });
  in {
    devShells = forEachSupportedSystem ({pkgs}: {
      default = pkgs.mkShell {
        packages = with pkgs; [nodejs];

        shellHook = ''
          echo "welcome to this nix shell"
          echo "- Node.js $(node --version)"
          echo "- npm $(npm --version)"
          echo "- $(chromium --version)"
        '';
      };
    });
  };
}

We can also add an .envrc whose content is just:

use flake

Note 1: there is no need to set PLAYWRIGHT_SKIP_VALIDATE_HOST_REQUIREMENTS = true in this case, since there are no dynamic dependencies to check (because the chromium binary we specified in executablePath is a static executable).

Note 2: I’m still pretty new to Nix and haven’t quite figure out how to work with Node.js projects in a purely nix way, so for now I create nix shells which are impure and require to install npm dependencies by manually typing npm install the first time.

4 Likes

Thank you @jackdbd, your explanation very usefull for me.

For dotnet developers (like me), it is very convenient to use env variables instead of editing code:

❯ echo $PLAYWRIGHT_LAUNCH_OPTIONS_EXECUTABLE_PATH
/nix/store/g04zli5glz02jiknj701rb75wcc7xvcv-playwright-browsers-chromium/chromium-1091/chrome-linux/chrome

❯ dotnet test -- Playwright.LaunchOptions.ExecutablePath=$PLAYWRIGHT_LAUNCH_OPTIONS_EXECUTABLE_PATH

Simple flake.nix with dev env:

# run with
# nix develop .
{
  description = "dev env";

  inputs = {
    nixpkgs.url = "github:nixos/nixpkgs/nixpkgs-unstable";
  };

  outputs = { self, nixpkgs }: let
    system = "x86_64-linux";
    pkgs = import nixpkgs { inherit system; };
  in {
      devShells.${system}.default = pkgs.mkShell {
      packages = with pkgs; [
        nodejs
        dotnet-sdk_8
        playwright-driver.browsers
      ];

      PLAYWRIGHT_NODEJS_PATH = "${pkgs.nodejs}/bin/node";
      PLAYWRIGHT_LAUNCH_OPTIONS_EXECUTABLE_PATH = "${pkgs.playwright-driver.browsers}/chromium-1091/chrome-linux/chrome";

      shellHook = ''
        echo "dev env started"
        fish && echo "exit dev env" && exit
      '';
    };
  };
}