Stefan Judis

Frankly, end-to-end testing and synthetic monitoring are challenging in today’s JavaScript-heavy world. There’s so much asynchronous JavaScript code running in modern applications that getting tests stable can be a real headscratcher.

That’s why many teams rely on testing mission-critical features and treat “testing the rest” as a nice to have. It’s a typical cost-effort decision. The downside of this approach is that if you’re not testing all functionality end-to-end, it’s very easy to miss production issues and regressions.

What can you do about the lack of test coverage besides writing more tests?

One valuable approach is implementing a safety net and making your end-to-end tests listen for obvious application problems. Let’s take JavaScript exceptions as an example. 

Just because your mission-critical tests pass doesn’t mean everything works as expected. By monitoring JavaScript exceptions while testing your application end-to-end increases the chance of catching problems without directly testing all the available features. 

In this article, I’ll show you how to set up Playwright and use its fixture feature to fail your tests if the JavaScript console goes red.

If you prefer video over reading this blog post, the content of this post is also available on YouTube.

The problem: a broken Frontend with succeeding end-to-end tests

Let’s look at an example: I broke the lazy loading on the Checkly blog a while ago. To prevent this from happening again, I wrote a Playwright end-to-end test that runs in Checkly. GitHub actions test every preview deployment with npx checkly test, and if everything passes, my tests are transformed into synthetic production monitoring with npx checkly deploy. 

Here’s the quick Playwright test in its entire glory.

And I thought I’d be safe with this approach (spoiler: I wasn’t).

This test navigates to the blog and tests the implemented lazy loading but does not monitor general site issues. Does functional lazy loading mean that everything on the page is working correctly? No. 

Even highly critical, user-facing issues go unnoticed as long as more blog posts are on the page after scrolling down. The test covers one feature and ignores the rest, including a blowing-up JavaScript exception. It’s not great!

Listening to thrown JavaScript exceptions is a safeguard against broken UI. 

As mentioned, you probably don’t have the time or capacity to test every feature end-to-end, but your end-to-end tests should listen and at least watch out for obvious problems.

How could you listen to thrown JS exceptions, then?

Listen for JavaScript exceptions in your Playwright tests

Listening to page events in Playwright is straightforward. The provided page object comes with a handy on function that allows you to listen to multiple page event types ("load", "pageerror", "crash", etc). For JavaScript exception tracking, we’re after the "pageerror" event. 

To consider thrown exceptions in your tests, attach an event listener and collect all the thrown exceptions in an array. Once the end-to-end test functionality passes, assert that your error array isn’t holding any entries. If it does, your test will fail.

Note, that you must attach the event listener before your tests interact with the page. Ideally, your page.on code comes first in your test case.

While the code snippet above works excellent for implementing exception tracking in a single test case, it isn’t maintainable in a large test code base. You don’t want to multiply the same event handling in every test case. 

Automatically fail end-to-end tests if JavaScript throws

You could rely on beforeEach or beforeAll hooks to run code before and after your tests, but an underrated Playwright feature deserves more attention!

Playwright fixtures allow you to structure your code in a Playwright-native way. They also enable you to run code before and after your tests, and you can even provide config and test data. Fixtures shine for many other use cases. 

You rely on Playwright's built-in fixtures whenever you write a standard Playwright test and use page, context, or request.

Test fixtures are objects that are isolated between tests. For example, when you use page, Playwright hands you a new and independent page object in every test case to avoid collision and a mixed-up state.

But there’s more — test fixtures are also a handy way to structure your code base and provide commonly used functionality. You can add, override and extend Playwright with your custom magic.

Let’s change the test example above and sprinkle on some fairy dust:

Extend the Playwright’s test object to be ready for custom fixtures.

Override the page fixture to listen to thrown JavaScript exceptions automatically. 

Sounds complicated? It’s not so bad, trust me!

Extend Playwright’s "test" object with custom fixtures

To write Playwright tests, you usually import test and expect from the @playwright/test package and get going, but guess what? Both objects are extendable.

To implement custom Playwright fixtures, you can extend the provided test object. 

The extend method returns a new test object that includes all the Playwright functionality enriched with your custom additions. Assign this new test object to a variable (myTest) and use it to register your tests.

myTest can now be extended with test data or objects that have an established state, such as a loggedInPage for your product. 

Let’s look at an extended test example. 

test.extend accepts an object with your fixture definitions. The property key becomes the fixture's name when used in your tests. To provide static data, define an object in your fixture configuration object (see testUser). It’s quick and easy.

If you want to add custom functionality and code that runs before and after your tests, pass in an async function (see loggedInPage). Functions will be called with a second parameter (use) that you must use to hand in objects to your tests. This approach lets you programmatically control what happens before and after your tests. 

Be sure to await your use() calls. Otherwise, it’s an asynchronous operation, and your test cases will fail if you don’t wait for it.

Custom fixtures for a testUser and loggedInPage make sense. Still, for constant error logging, I don’t like to include a new fixture (pageWithoutJSErrors 👎) but rather make JavaScript exception monitoring the enabled default in every test that uses page.

Luckily, overriding the built-in page object is also quickly done. Adjust the test.extend call and provide a new page function. And that’s pretty much it. 😅

Custom fixtures and overrides can still use Playwright’s built-in fixtures. See above how our new page fixture also uses the original page object.

Thanks to use(), custom logic can run before and after our test cases. We can attach the JavaScript error event listener, pass the page object to the tests, and when the tests succeed, check if the error array is empty for every test case. It’s a clean and tidy solution!

And all this functionality is automatically available when you register tests using the extended myTest.

But what about situations where you don’t want to fail your test cases when JS exceptions appear? Can you make fixtures configurable?

Make your fixture configurable with fixture options

Extending your test object isn’t only about providing custom functionality. You can also use fixture options to make your tests configurable. Fixture options enable you to tweak your test configuration via your playwright.config or per-file test.use() calls.

To configure the error tracking, introduce a new failOnJSError option in your test extension.

Do you see what we did there? There’s now a new failOnJSError fixture option, that’s used by the overridden page fixture. Your Playwright setup just became tailored to your needs, and you can now control if you want to fail your tests if the JavaScript throws or not. 💪

Here’s an example of a playwright.config that configures the new failOnJSError option.

You can also call test.use() in your spec files to set the option per-test.

Your entire Playwright setup just became more powerful and configurable.

Restructure your project to be ready for a scaleable fixture setup

At this stage, you have a single spec file that extends Playwright’s test object and a test case using it. Unfortunately, this approach is neither scalable nor reusable.

To make the fixture setup reusable, place the test.extend code into its own base.ts file and import it in your *.spec.ts files.

It includes the same functionality as our test file but now exports Playwright’s expect and the new extended test object. And because we have a base file with similar exports to @playwright/test, it can be used by all your *.spec.ts files by changing the import.

With this setup, you stop requiring test and expect from @playwright/test and import your custom Playwright setup from a base file. Provide data, reuse code, and make everything configurable in a Playwright-native and standardized way! 

It’s a win-win without spaghetti imports everywhere. And thanks to the overridden page fixture, you won’t miss thrown JavaScript exceptions when your end-to-end tests run.

Conclusion — test more with an extended Playwright setup

As mentioned at the beginning of this article, having 100% end-to-end test coverage is tough to achieve and most likely shouldn’t be your end goal. But if you’re not passively monitoring broken functionality while testing your core business, you’re simply missing out.

Are your network waterfalls plastered with red 500s?

You can passively monitor all these things while testing critical user flows.

And to be safe, ideally, you don’t rely on testing your preview or staging deployments but also monitor your production environment end-to-end. This environment is what your visitors see, after all. ;) 

If you value high quality (and I hope you do!), you should get the most out of your tests; and hopefully, this guide helped you do that.

If you have any questions or comments, say hi in the Checkly Community. 👋 We’re a lovely bunch, I promise!

Track Frontend JavaScript exceptions with Playwright fixtures

Detect

Communicate

Resolve

Monitoring as Code

Developers

Resources

Community

Track Frontend JavaScript exceptions with Playwright fixtures