Running playwright tests

Help
21

I’m glad my comment was helpful!

If you’re having trouble finding the nixpkgs’s commit in which the desired playwright version you might find it useful nixhub

How I use it:

{
  inputs = {
    nixpkgs.url = "github:nixos/nixpkgs/nixpkgs-unstable";
    nixpkgs-pw-v1_40.url = "github:nixos/nixpkgs/0a254180b4cad6be45aa46dce896bdb8db5d2930";
  };

  outputs = { self, nixpkgs, nixpkgs-pw-v1_40 }: let
    system = "x86_64-linux";
    pkgs = import nixpkgs { inherit system; };
    pkgs-pw-v1_40 = import nixpkgs-pw-v1_40 { inherit system; };
  in {
      devShells.${system}.default = pkgs.mkShell {
        packages = [
          pkgs.nodejs_20
          pkgs-pw-v1_40.playwright
        ];

        PLAYWRIGHT_NODEJS_PATH = "${pkgs.nodejs_20}/bin/node";
        PLAYWRIGHT_SKIP_VALIDATE_HOST_REQUIREMENTS = true;

        PLAYWRIGHT_BROWSERS_PATH="${pkgs-pw-v1_40.playwright-driver.browsers}";
    };
  };
}

For example: if I need to use pw 1.38 instead 1.40, I’ll just use commit a71323f68d4377d12c04a5410e214495ec598d4c (found it on nixhub).

What would need to change to make this work with direnv?

My .envrc in project folder:


if ! has nix_direnv_version || ! nix_direnv_version 3.0.4; then
    source_url "https://raw.githubusercontent.com/nix-community/nix-direnv/3.0.4/direnvrc" "sha256-DzlYZ33mWF/Gs8DDeyjr8mnVmQGx7ASYqA5WlxwvBG4="
fi
use flake /src/flakes/flake-for-this-project

I store flakes separately from projects (/src/flakes/*), but of course it’s easier to keep them in a project.

22

Same issue for me. I’ve been really careful with the versions:
pnpm list shows playwright 1.40.0 and echo ${pkgs.playwright-driver.browsers.version} in the shellHook shows 1.40.0. However pnpx playwright codegen results in

Error: t.parse: Executable doesn't exist at
/nix/store/xc7q18y0rw1yk40wrvqxzy87fa3vzccr-playwright-browsers-chromium/chromium-1105/chrome-linux/chrome

But

/nix/store/xc7q18y0rw1yk40wrvqxzy87fa3vzccr-playwright-browsers-chromium/chromium-1091/chrome-linux/chrome

does exist (chromium-1091, not 1105), which is kind of weird too…

Any help would be appreciated.

1 Like
23

Strange, if the versions of playwright are really the same, the same browser would have been used. :thinking:

24

Do these solutions work with Firefox as well? Or just with Chromium?

25

I was only able to do it with Chromium, Firefox failed.

26

Okay, it seems things are working now. As it turns out, it looks like playwright-driver.browsers only provides chromium, not firefox or webkit. Thus, I was getting the following stack trace, and focused on the “executable doesn’t exist” errors and missed the “1 passed” at the end.

It still sucks that I can’t test my software on firefox or webkit, but now I guess it’s the Playwright team’s responsibility to make their project work on Linux?

Anyway, thank you for your help!

❯ pnpm exec playwright test

Running 3 tests using 2 workers
  1) [firefox] › example.spec.ts:3:5 › test ────────────────────────────────────────────────────────

    Error: browserType.launch: Executable doesn't exist at /nix/store/jws56gk36j4akxcv4xisad8ak2wib5sd-playwright-browsers-chromium/firefox-1429/firefox/firefox
    ╔═════════════════════════════════════════════════════════════════════════╗
    ║ Looks like Playwright Test or Playwright was just installed or updated. ║
    ║ Please run the following command to download new browsers:              ║
    ║                                                                         ║
    ║     pnpm exec playwright install                                        ║
    ║                                                                         ║
    ║ <3 Playwright Team                                                      ║
    ╚═════════════════════════════════════════════════════════════════════════╝




  2) [webkit] › example.spec.ts:3:5 › test ─────────────────────────────────────────────────────────

    Error: browserType.launch: Executable doesn't exist at /nix/store/jws56gk36j4akxcv4xisad8ak2wib5sd-playwright-browsers-chromium/webkit-1944/pw_run.sh
    ╔═════════════════════════════════════════════════════════════════════════╗
    ║ Looks like Playwright Test or Playwright was just installed or updated. ║
    ║ Please run the following command to download new browsers:              ║
    ║                                                                         ║
    ║     pnpm exec playwright install                                        ║
    ║                                                                         ║
    ║ <3 Playwright Team                                                      ║
    ╚═════════════════════════════════════════════════════════════════════════╝




  2 failed
    [firefox] › example.spec.ts:3:5 › test ─────────────────────────────────────────────────────────
    [webkit] › example.spec.ts:3:5 › test ──────────────────────────────────────────────────────────
  1 passed (4.6s)

  Serving HTML report at http://localhost:9323. Press Ctrl+C to quit.
27

It DOES work on Linux on distros like Debian / Ubuntu… :wink: Playwright is downloading the browser binaries via NPM and running them directly. But NixOS of course has troubles with binaries that depend on all kind of paths that we don’t have…

1 Like
28

could you try and comment on Playwright: browser improvements, update by phaer · Pull Request #298944 · NixOS/nixpkgs · GitHub that brings firefox support ?

2 Likes
29

I managed to get this working through some of the suggestions, and one additional tweak. I do not have to rely on the versions being aligned as I explicitly set the executablePath for the launch options.

flake.nix

         packages = with pkgs; [
            nodejs_21

            # playwright
            playwright-driver.browsers
          ];

          # playwright
          PLAYWRIGHT_NODEJS_PATH = "${pkgs.nodejs_21}/bin/node";
          PLAYWRIGHT_LAUNCH_OPTIONS_EXECUTABLE_PATH = "${pkgs.playwright-driver.browsers}/chromium-1091/chrome-linux/chrome";
          PLAYWRIGHT_BROWSERS_PATH = "${pkgs.playwright-driver.browsers}";

package.json

    "@playwright/experimental-ct-svelte": "^1.43.0",
    "@playwright/test": "^1.43.0",

playwright.config.ts

projects: [
    {
      name: 'chromium',
      use: {
        ...devices['Desktop Chrome'],
        launchOptions: {
          executablePath: process.env.PLAYWRIGHT_LAUNCH_OPTIONS_EXECUTABLE_PATH as string,
        }
      },
    },
]
2 Likes
30

But doesn’t this mean that you potentially use a different browser version than the one coming with your @playwright/test version which may lead to problems.

When testing with playwright locally, I usually set the playwright version of the project to the one coming with Nixpkgs so that the browser version align.

31

Obviously the preference is to run the same versions. But playwright-driver is currently only set to 1.40 and my requirements were to use playwright-ct 1.43 due to some changes that were made. The only way to get around it was to force a browser version.

The issues i’ve run into are that ui and debug versions don’t seem to work.

32

If you want you can grab that 1091 programmatically:

shellHook = ''
  playwright_chromium_revision="$(${jq}/bin/jq --raw-output '.browsers[] | select(.name == "chromium").revision' ${playwright-driver}/package/browsers.json)"

  export PLAYWRIGHT_CHROMIUM_EXECUTABLE_PATH="${playwright-driver.browsers}/chromium-$playwright_chromium_revision/chrome-linux/chrome";
'';