Running playwright tests

Hi!

I use this config and everything works well :

# shell.nix
{
  pkgs ? import <nixpkgs> {},
  unstable ? import <unstable> { }
}:
  pkgs.mkShell {
    # nativeBuildInputs is usually what you want -- tools you need to run
    # Check last version of playwright https://search.nixos.org/packages?channel=unstable&from=0&size=50&sort=relevance&type=packages&query=playwright-driver
    nativeBuildInputs = with pkgs; [
      unstable.playwright-driver.browsers
    ];

    shellHook = ''
      export PLAYWRIGHT_BROWSERS_PATH=${unstable.playwright-driver.browsers}
      export PLAYWRIGHT_SKIP_VALIDATE_HOST_REQUIREMENTS=true
    '';

}

The version of playwright in unstable is 1.40.0. But I need to use the last version of playwright : 1.46.1
I wonder how to use the last version with my shell.nix ?

I have prepared a flake that provides a bunch of versions of playwright so that it will be possible to choose it independently of nixpkgs: GitHub - voidus/nix-playwright-browsers: Nix derivations for Playwright browsers in various versions

It’s very early stage but I think the idea is solid.

Since nixpkgs is not meant to contain lots of version of the same package, I put it in a separate flake.

4 Likes

I have tried the examples here, but non are working. Does anyone have a working flake example to work in the cli or vscode?

I’ve beeing using this setup with devenv.sh flake and using playwright as a nodejs dependency in a nextjs project:

{
  inputs = {
    nixpkgs.url = "github:NixOS/nixpkgs/nixos-23.05";
    systems.url = "github:nix-systems/default";
    devenv.url = "github:cachix/devenv";
  };

  nixConfig = {
    extra-trusted-public-keys = "devenv.cachix.org-1:w1cLUi8dv3hnoSPGAuibQv+f9TZLr6cv/Hm9XgU50cw=";
    extra-substituters = "https://devenv.cachix.org";
  };

  outputs = { self, nixpkgs, devenv, systems, ... } @ inputs:
    let
      forEachSystem = nixpkgs.lib.genAttrs (import systems);
    in
    {
      devShells = forEachSystem
        (system:
          let
            pkgs = nixpkgs.legacyPackages.${system};
            yarnVersion = "4.0.1";
          in
          {
            default = devenv.lib.mkShell {
              inherit inputs pkgs;
              modules = [
                {
                  # https://devenv.sh/reference/options/
                  packages = [
                    pkgs.postgresql
                    pkgs.playwright-driver.browsers
                    pkgs.nodejs
                  ];

                  env.YARN_VERSION = yarnVersion;
                  env.PLAYWRIGHT_BROWSERS_PATH = "${pkgs.playwright-driver.browsers}";
                  env.PLAYWRIGHT_SKIP_VALIDATE_HOST_REQUIREMENTS = true;
                  env.PLAYWRIGHT_NODEJS_PATH = "${pkgs.nodejs}/bin/node";
                  env.PLAYWRIGHT_LAUNCH_OPTIONS_EXECUTABLE_PATH = "${pkgs.playwright-driver.browsers}/chromium-1048/chrome-linux/chrome";


                  languages.javascript = {
                    enable = true;
                    corepack.enable = true;
                    package = pkgs.nodejs;
                  };
                  enterShell = ''
                    corepack prepare yarn@${yarnVersion}
                  '';

                }
              ];
            };
          });
    };
}

So you are using devenv.nix and a flake.nix file together? How does that work? It looks like what you’ve shared is the flake… is it possible to share the devenv.nix file too?

We are having some issues using just devenv.nix with our Playwright installation (pnpm dependency). It’s a TypeScript project, but when we run pnpm exec playwright test we are seeing this error:

Error: browserType.launch: Executable doesn't exist at /nix/store/lrrc9yfclwvck9q8ng1b3c7560nlv51a-playwright-browsers/chromium_headless_shell-1148/chrome-linux/headless_shell

That location doesn’t exist, but /nix/store/lrrc9yfclwvck9q8ng1b3c7560nlv51a-playwright-browsers/chromium_headless_shell-1148/chrome-linux/chrome does exist.

Yes, that’s a plain flake.nix file, but it is using the devenv repository. You should be able to use it just by running nix develop --impure. In this setup you don’t have a devenv.nix file.

I’m just not sure why you are getting this error with pnpm, for me using npm works just fine. Are you also using devenv? Could you share the flake file you are using?

We weren’t using a flake at all, was using a devenv.nix file. But I’ve converted it to a flake and get the exact same error as when I use the devenv.nix file.

Here is the flake:

{
  inputs = {
    nixpkgs.url = "github:NixOS/nixpkgs/nixpkgs-unstable/";
    systems.url = "github:nix-systems/default";
    devenv.url = "github:cachix/devenv";
    node.url = "github:NixOS/nixpkgs/42754a724d486c2bb4888c7602fb4f7d90772931";
  };

  nixConfig = {
    extra-trusted-public-keys = "devenv.cachix.org-1:w1cLUi8dv3hnoSPGAuibQv+f9TZLr6cv/Hm9XgU50cw=";
    extra-substituters = "https://devenv.cachix.org";
  };

  outputs = { self, nixpkgs, devenv, systems, node, ... } @ inputs:
    let
      forEachSystem = nixpkgs.lib.genAttrs (import systems);
    in
    {
      devShells = forEachSystem
        (system:
          let
            pkgs = nixpkgs.legacyPackages.${system};
            node_pkgs = import node { system = pkgs.stdenv.system; };
          in
          {
            default = devenv.lib.mkShell {
              inherit inputs pkgs;
              modules = [
                {
                  # https://devenv.sh/reference/options/
                  packages = [
                    node_pkgs.nodejs_20
                    pkgs.git
                    pkgs.playwright-driver.browsers
                    pkgs.pnpm
                  ];

                  env.PLAYWRIGHT_BROWSERS_PATH = "${pkgs.playwright-driver.browsers}";
                  env.PLAYWRIGHT_SKIP_VALIDATE_HOST_REQUIREMENTS = true;
                  env.PLAYWRIGHT_NODEJS_PATH = "${node_pkgs.nodejs_20}/bin/node";
                  env.PLAYWRIGHT_LAUNCH_OPTIONS_EXECUTABLE_PATH = "${pkgs.playwright-driver.browsers}/chromium-1134/chrome-linux/chrome";
                }
              ];
            };
          });
    };
}

I’ve tried your flake file and got the same error.
After looking into it, it seems that the path is actually ā€œl/nix/store/x2ih0irl5zby5wgyfrcfjkjfkqfgzg28-playwright-browsers/chromium-1134/chrome-linux/chromeā€

It’s in chromium-1134 but for some reason playwright is expecting at chromium-1148.

I just do not know why.

OK, I got past playwright expecting the wrong version of chromium. Looking about the internet, I found a similar issue discussed here [Bug] BrowserType.launch: Executable doesn't exist at /root/.cache/ms-playwright/chromium-1140/chrome-linux/chrome Ā· Issue #33673 Ā· microsoft/playwright Ā· GitHub.

Looking at that made me realize it was a mismatch between the playwright version requested in my package.json file (v1.49.0) and the related driver version supplied by nixpkgs-unstable (v1.47.0). So updating the package.json file resolved the browserType.launch: Executable doesn't exist error.

Unfortunately, the same tests are still failing. But now it’s due to:

Test timeout of 30000ms exceeded.

Another puzzle to solve!

2 Likes

This is what I am currently doing in a shell.nix:

{ pkgs ? import <nixpkgs> {} }:

let
  # We need to pin the version of playwright-driver, so we can match the version in package.json
  # See https://www.nixhub.io/packages/playwright-driver for hash
  playwrightPkgs = import
    (builtins.fetchTarball {
      # Currently pinned to: playwright-driver@1.47.0
      url = https://github.com/NixOS/nixpkgs/archive/34a626458d686f1b58139620a8b2793e9e123bba.tar.gz;
      sha256 = "sha256:1dm43gvl20wbl6j1d6l35y4bqjdspg3xv143ibnbzvv5anw6q2cw";
    }) {};
in
  pkgs.mkShell {
    nativeBuildInputs = with pkgs; [
      nodejs
      playwrightPkgs.playwright-driver.browsers
      jq
    ];

    shellHook = ''
      export PLAYWRIGHT_BROWSERS_PATH=${playwrightPkgs.playwright-driver.browsers}
      export PLAYWRIGHT_SKIP_VALIDATE_HOST_REQUIREMENTS=true
      export PLAYWRIGHT_NODEJS_PATH=${pkgs.nodejs}/bin/node

      playwright_chromium_revision="$(${pkgs.jq}/bin/jq --raw-output '.browsers[] | select(.name == "chromium").revision' ${playwrightPkgs.playwright-driver}/browsers.json)"
      export PLAYWRIGHT_LAUNCH_OPTIONS_EXECUTABLE_PATH=${playwrightPkgs.playwright-driver.browsers}/chromium-$playwright_chromium_revision/chrome-linux/chrome

      env | grep ^PLAYWRIGHT
    '';
}

I didn’t need PLAYWRIGHT_LAUNCH_OPTIONS_EXECUTABLE_PATH in the end, and I never got running the tests in VS Code working. The CLI and test UI are working fine.

1 Like

I tried many things, and for now I just expose the files where playwright expects them… With this solution, the chromium version used is not in sync, but it works for now
Also supports the playwright UI and IDE extensions

{pkgs, ...}: let
  # a nix derivation that contains the playwright browsers
  # with the same directory structure as expected by playwright
  pw-chromium = let
    rev.chrome.pkgs = "1134";
    rev.chrome.npm = "1140";

    rev.ffmpeg.pkgs = "1010";
    rev.ffmpeg.npm = "1010";
  in
    pkgs.stdenv.mkDerivation {
      name = "pw-chromium";
      src = pkgs.playwright-driver.browsers;
      installPhase = ''
        echo "Source directory: $src"
        echo "Content:"
        ls $src
        echo "Starting playwright chromium installation";

        if [ ! -f $src/chromium-${rev.chrome.pkgs}/chrome-linux/chrome ]; then
          # screaming because because cant see nothing in nixos output
          echo "======= ERROR playwright browsers nixpkgs revision has changed, please update the pkgsRevision variable";
          echo "======= ERROR playwright browsers nixpkgs revision has changed, please update the pkgsRevision variable";
          echo "======= ERROR playwright browsers nixpkgs revision has changed, please update the pkgsRevision variable";
          echo "======= ERROR playwright browsers nixpkgs revision has changed, please update the pkgsRevision variable";
          exit 1;
        fi

        echo "Copying ffmpeg binary from nixpkgs version ${rev.ffmpeg.pkgs} to npm version ${rev.ffmpeg.npm}";
        mkdir -p $out/ffmpeg-${rev.ffmpeg.npm}
        cp $src/ffmpeg-${rev.ffmpeg.pkgs}/ffmpeg-linux $out/ffmpeg-${rev.ffmpeg.npm}/ffmpeg-linux

        echo "Creating the folder structure expected by the npm package";
        mkdir -p $out/chromium-${rev.chrome.npm}/chrome-linux

        echo "Copying chromium binary from nixpkgs version ${rev.chrome.pkgs} to npm version ${rev.chrome.npm}";
        cp $src/chromium-${rev.chrome.pkgs}/chrome-linux/chrome $out/chromium-${rev.chrome.npm}/chrome-linux/chrome-bin

        echo "Creating a wrapper at the location expected by playwright, so we can pass additional parameters";
        echo '#!${pkgs.runtimeShell}' >> $out/chromium-${rev.chrome.npm}/chrome-linux/chrome
        echo "$out/chromium-${rev.chrome.npm}/chrome-linux/chrome-bin" --use-angle '$@' >> $out/chromium-${rev.chrome.npm}/chrome-linux/chrome
        chmod +x $out/chromium-${rev.chrome.npm}/chrome-linux/chrome

        echo "Making a copy for the playwright ui symlink (systemd service)";
        mkdir -p $out/bin
        cp $out/chromium-${rev.chrome.npm}/chrome-linux/chrome $out/bin/chrome
      '';
    };

  # a script that symlinks the playwright chromium to /opt/google/chrome/chrome
  # this is required to run the playwright ui as that path is hardcoded in playwright
  symlink-chrome = pkgs.writeShellScript "symlink-chrome" ''
    ${pkgs.coreutils}/bin/rm -rf /opt/google
    ${pkgs.coreutils}/bin/mkdir -p /opt/google/chrome
    ${pkgs.coreutils}/bin/ln -s ${pw-chromium}/bin/chrome /opt/google/chrome/chrome
    ${pkgs.coreutils}/bin/chmod 755 /opt
    ${pkgs.coreutils}/bin/chmod -R 755 /opt/google
  '';
in {
  # add pw-browsers to the list of packages on the system level
  # is this really necessary?
  environment.systemPackages = [
    pw-chromium
  ];

  systemd.services.playwright-ui-chromium-symlink = {
    description = "Symlink chromium to the path expected by playwright";
    wantedBy = ["multi-user.target"];
    serviceConfig = {
      ExecStart = "${symlink-chrome}";
      Type = "oneshot";
    };
  };

  environment.sessionVariables = {
    PLAYWRIGHT_BROWSERS_PATH = "${pw-chromium}";
    PLAYWRIGHT_SKIP_VALIDATE_HOST_REQUIREMENTS = "true";
  };
}

I just gave up on it.
For anyone else interested, I recommend using distrobox to wrap your application:

./install.sh

container_name="${1:-my-project}"
venvDir="${2:-./.venv}"

printf "Creating distrobox container...\n"
# you MUST set either --init or --unshare-all
distrobox create --name ${container_name} --image ubuntu --home "$(pwd)/.distrobox" --unshare-all

distrobox enter ${container_name} -- /bin/bash --login -c "
  printf \"Installing uv...\n\"
  curl -LsSf https://astral.sh/uv/install.sh | sh
"

distrobox enter ${container_name} -- /bin/bash --login -c "
  printf \"Creating virtual environment...\n\"
  uv venv --python 3.13 ${venvDir}
  source \"${venvDir}/bin/activate\"

  printf \"nstalling dependencies...\n\"
  uv pip install -r requirements.txt
"

distrobox enter ${container_name} -- /bin/bash --login -c "
  printf \"Installing playwright dependencies...\n\"
  sudo ${venvDir}/bin/playwright install-deps

  printf \"Installing playwright browsers...\n\"
  ${venvDir}/bin/playwright install
"

./run.sh

container_name="${1:-my-project}"
venvDir="${2:-./.venv}"

# Check if container and virtual environment exist
if ! distrobox list | grep -q "${name}" || [ ! -d "${venvDir}" ]; then
  printf "Container or virtual environment not found. Please run:\n"
  printf "\t./scripts/install.sh "${name}" "${venvDir}"\n"
  exit 1
fi

echo "Setting up virtual environment in container..."
distrobox enter "${name}" -- /bin/bash --login -c "
  printf \"Activating Python environment...\n\"
  source \"${venvDir}/bin/activate\"
  printf \"(Re)Installing dependencies...\n\"
  uv pip install -r requirements.txt

  printf \"Starting interactive container session...\n\"
  exec /bin/bash
"

If you like, the code inside ./run.sh can actually be the code of shellHook inside your shell.nix.

Running npm tests with karma and playwright was also an issue for me, so far I wasn’t able to make that work directly with NixOS. But using the playwright docker container worked!

docker run --rm -v "$(pwd):/app" -w /app mcr.microsoft.com/playwright:v1.49.1-noble npm run test

Recently, Microsoft has released playwright-mcp. Following this thread, I have packaged Playwright as a flake for MCP usage, so it can be run on NixOS: GitHub - akirak/nix-playwright-mcp: A Nix flake for using Playright MCP on NixOS. Any feedback is welcome.

2 Likes