Five Playwright CLI features you should know

Thanks to Microsoft's Playwright, running end-to-end tests with real browsers is quickly done. Initialize a new Playwright project, install all the dependencies, and off you go! 

Then, any new headless browser test run is only one npx playwright test away. But have you checked all the test command's CLI options? playwright test includes a few real gems to help you create better tests faster. 

Let me share a mixed bag of my favorite CLI tricks in this post.

And trust me, you'll continue to use some of these "hidden features". Let's go!

Use --last-failed to run only failed tests

You know the situation: you run your test suite and have a few failing test cases. You investigate, tweak your app or test code, and want to rerun the failing tests. How do you do it?

You could use the spec file name when you run your tests (npx playwright test failed-test.spec.ts) or rely on test.only, but luckily, Playwright has the functionality of running failed tests built in. Say hello to the --last-failed CLI flag.

# Only run the tests that failed in the previous test run
$ npx playwright test --last-failed

Playwright writes a hidden JSON file in the test-results directory whenever it finishes a test run. This file keeps track of the test result and failed test cases.

{
  "status": "failed",
  "failedTests": [
    "ecb7ac7b365afffe0dcb-0992cf0d9df8b5e74dc4"
  ]
}

By relying on --last-failed, you can focus on and rerun the failing tests without manually specifying them.

But what if you don't want to run failing tests but only the tests you touched?

Use --only-changed to run only changed spec files

A standard practice in software development is to rely on Git branches. To release a new feature, you branch out, adjust your application and test code, and eventually merge everything back into main. 

During development, you probably want to run only changed and new tests — this is when --only-changed shines.

# During feature development and test creation:
# Files aren't committed yet, and the development of tests is still a work in progress
$ git status
On branch new-feature
Changes not staged for commit:
	modified:   tests/new-feature.spec.ts

# Only run the spec files that are uncommitted compared to the current branch
$ npx playwright test --only-changed

Using npx playwright test --only-changed, Playwright will only run the tests that aren't committed to version control. It will only run modified test files or new test cases.

Running and testing only the changed files is a handy shortcut when your development flow is based on version control, feature branches, and pull requests. Develop the new feature, only run the affected tests, and commit the changes to get ready to land. 

But if Playwright knows about Git, you might wonder if there's a way to run only code changed across branches — you bet!

Playwright will run all the uncommitted changes if you apply the --only-changed flag without additional parameters. But when you pass the flag a Git ref like the main branch, Playwright will run files that differ comparing two branches.

# After feature development and ready for pull request:
# New test files are committed and ready to merge
$ git status
On branch new-feature
nothing to commit, working tree clean

# Run all the spec files that are new or modified compared to the `main` branch
$ npx playwright test --only-changed=main

Let it be uncommitted and dirty code or changes across branches;  --only-changed is a handy option to run only changed test code.

Use --repeat-each to ensure tests are stable

Let's take a step back for a moment: before adding tests to an existing codebase, you should guarantee they're stable. You don't want to be the one bringing flakiness into the codebase, right?

Unfortunately, the problem with test case flakiness is that it's tough to pinpoint. How can you discover and fix a test case failing only once a hundred times? An easy and obvious answer is to run your tests over a hundred times.

And yet again, Playwright has this functionality built-in. If you want to run the same tests multiple times, use --repeat-each.

# Run all tests 100 times
npx playwright test --repeat-each 100

# Run all spec files matching `new-feature` 100 times
npx playwright test new-feature --repeat-each 100

# Run all uncommited spec files 100 times
npx playwright test --only-changed --repeat-each 100

If you run your new test cases a hundred times and they're passing the entire time, you can be confident you're committing a stable test to the code base!

Use --forbid-only to always run all your tests

I mentioned it earlier: if you want to run only changed test cases, you could also apply test.only to specific tests. I often use this option when I want to dive into one test case.

// Only run this test when using `npx playwright test`
test.only('Your test', async ({ page }) => {
 // Your test code...
});

But marking tests as only can be really risky business. Imagine that you accidentally merge a test.only into the main branch. Then, hundreds of tests could be ignored to run only your "priority" test case. And it could take a while until you notice your pipeline isn't testing what it is supposed to because nobody investigates a passing testing pipeline.

To prevent this from happening, instruct Playwright to alert you if your test cases include a test.only.

# Fail the test suite if it includes a `test.only`
$ npx playwright test --forbid-only
Error: item focused with '.only' is not allowed due to the '--forbid-only' CLI flag: "your-test.spec.ts your test

During development, the --forbid-only flag is unimportant, but if you're in charge of configuring and setting up CI/CD, I recommend turning it on to be safe.

Combine UI mode with --headed --workers 1 for more comfortable debugging

All the previous examples focused on running your end-to-end tests from the command line. But you might prefer developing your tests in UI mode because it provides more debug information and can turn watch mode — Playwright's UI mode is wonderful! 

I constantly switch between running my tests from the CLI and UI mode. UI mode is my tool of choice, especially when I have to debug a tricky test case. However, I don't use the "vanilla" UI mode and usually apply additional configuration to make my debugging session even more straightforward.

# Run Playwright UI mode but also open browsers 
# and limit the execution to one test at a time
$ npx playwright test --ui --headed --workers 1

By applying --headed, I can rely on UI mode's extra information (logs, network waterfall, etc.) and follow along to see all the test actions running in a browser. Seeing a test run in a real browser sometimes helps me to realize what's off with my test case.

And to avoid multiple browser windows from popping up, I also apply --workers 1 to run only one test at a time. I found this combination very comfortable when focussing on a tricky and flaky test case.

Conclusion

These are my five favorite Playwright CLI tricks. Do you have any other tips and tricks? If you do, please reach out in the Checkly Community Slack; there are quite some Playwright experts hanging out.

How so? We at Checkly enable you to take your existing Playwright test suite and run it on a schedule from anywhere in the world as synthetic monitoring. If you're worrying about production issues or — god forbid — outages, take your existing Playwright setup and run it against production. Then, you'll know that everything is always up and running. It's pretty cool, trust me!