Sometimes a Playwright test will run and pass correctly locally, but will fail when running in CI. We can debug the test using Playwright's verbose logging, and compare the output from both environments.
By setting the DEBUG env var to pw:api, Playwright runs the tests in verbose logging mode:
DEBUG=pw:api npm test
which produces output like this:
pw:api attempting click action +2ms
pw:api waiting for element to be visible, enabled and stable +0ms
pw:api element is visible, enabled and stable +24ms
pw:api scrolling into view if needed +1ms
pw:api done scrolling +1ms
pw:api performing click action +3ms
pw:api click action done +29ms
pw:api waiting for scheduled navigations to finish +0ms
We can compare the output from our successful local run with output of the failed CI run. We'll first need to remove the bits like +24ms, as these will otherwise show up in the diffs.
Here's an example regex to use for removing these tokens in your editor:
\+[0-9]*[a-z]*$
Then you can use a diff view to spot where the test diverged. In this screenshot, the left pane is the local test passing, while the right pane is the CI test failing:
In the specific case of this test, the "locator.click failed" bit was useful information from the CI logs. Before using the verbose logging, it seemed the "new tab" waiting to open was timing out, or there was some issue about opening the tab. But through this debugging process, we determined the "connect link" was not found, and thus was never clicked. The tests passed in CI after changing the link's selector to something more specific.