Track Frontend JavaScript exceptions with Playwright fixtures

Frankly, end-to-end testing and synthetic monitoring are challenging in today’s JavaScript-heavy world. There’s so much asynchronous JavaScript code running in modern applications that getting tests stable can be a real headscratcher.

That’s why many teams rely on testing mission-critical features and treat “testing the rest” as a nice to have. It’s a typical cost-effort decision. The downside of this approach is that if you’re not testing all functionality end-to-end, it’s very easy to miss production issues and regressions.

What can you do about the lack of test coverage besides writing more tests?

One valuable approach is implementing a safety net and making your end-to-end tests listen for obvious application problems. Let’s take JavaScript exceptions as an example. 

Just because your mission-critical tests pass doesn’t mean everything works as expected. By monitoring JavaScript exceptions while testing your application end-to-end increases the chance of catching problems without directly testing all the available features. 

In this article, I’ll show you how to set up Playwright and use its fixture feature to fail your tests if the JavaScript console goes red.

Ready, steady, go!

If you prefer video over reading this blog post, the content of this post is also available on YouTube.

The problem: a broken Frontend with succeeding end-to-end tests

Let’s look at an example: I broke the lazy loading on the Checkly blog a while ago. To prevent this from happening again, I wrote a Playwright end-to-end test that runs in Checkly. GitHub actions test every preview deployment with npx checkly test, and if everything passes, my tests are transformed into synthetic production monitoring with npx checkly deploy. 

Here’s the quick Playwright test in its entire glory.

import { expect, test } from "@playwright/test"

test("Checkly blog lazy loading", async ({ page }) => {
  await page.goto("http://localhost:3000/blog/")
  // locate all blog articles
  const articles = page.locator('a[title="Visit this post"]')
  // count the number of initially included articles
  const articleCount = await articles.count()

  // scroll the last article into view to trigger lazy-loading
  await articles.last().scrollIntoViewIfNeeded()

  // wait for more articles to be loaded and rendered
  await expect(async () => {
    const newCount = await articles.count()
    expect(newCount).toBeGreaterThan(articleCount)
  }).toPass()
})

And I thought I’d be safe with this approach (spoiler: I wasn’t).

This test navigates to the blog and tests the implemented lazy loading but does not monitor general site issues. Does functional lazy loading mean that everything on the page is working correctly? No. 

Even highly critical, user-facing issues go unnoticed as long as more blog posts are on the page after scrolling down. The test covers one feature and ignores the rest, including a blowing-up JavaScript exception. It’s not great!

Listening to thrown JavaScript exceptions is a safeguard against broken UI. 

As mentioned, you probably don’t have the time or capacity to test every feature end-to-end, but your end-to-end tests should listen and at least watch out for obvious problems.

How could you listen to thrown JS exceptions, then?

Listen for JavaScript exceptions in your Playwright tests

Listening to page events in Playwright is straightforward. The provided page object comes with a handy on function that allows you to listen to multiple page event types ("load", "pageerror", "crash", etc). For JavaScript exception tracking, we’re after the "pageerror" event.

To consider thrown exceptions in your tests, attach an event listener and collect all the thrown exceptions in an array. Once the end-to-end test functionality passes, assert that your error array isn’t holding any entries. If it does, your test will fail.

test("Checkly blog lazy loading", async ({ page }) => {
  // set up a new Array to collect thrown errors
  const errors: Array<Error> = []

  // listen to exceptions during the test sessions
  page.on("pageerror", (error) => {
    errors.push(error)
  })
  
  // All your test code…
  // …

  // assert that there haven’t been any errors
  expect(errors).toHaveLength(0)
})

Note, that you must attach the event listener before your tests interact with the page. Ideally, your page.on code comes first in your test case.

While the code snippet above works excellent for implementing exception tracking in a single test case, it isn’t maintainable in a large test code base. You don’t want to multiply the same event handling in every test case.

Automatically fail end-to-end tests if JavaScript throws

You could rely on beforeEach or beforeAll hooks to run code before and after your tests, but an underrated Playwright feature deserves more attention!

Playwright fixtures allow you to structure your code in a Playwright-native way. They also enable you to run code before and after your tests, and you can even provide config and test data. Fixtures shine for many other use cases.

What is a Playwright fixture?

You rely on Playwright's built-in fixtures whenever you write a standard Playwright test and use page, context, or request.

import { test, expect } from '@playwright/test';

// This is the built-in Playwright `page` fixture
//                           👇
test('basic test', async ({ page }) => {
  await page.goto('https://playwright.dev/');

  await expect(page).toHaveTitle(/Playwright/);
});

Test fixtures are objects that are isolated between tests. For example, when you use page, Playwright hands you a new and independent page object in every test case to avoid collision and a mixed-up state.

But there’s more — test fixtures are also a handy way to structure your code base and provide commonly used functionality. You can add, override and extend Playwright with your custom magic.

Let’s change the test example above and sprinkle on some fairy dust:

1. Extend the Playwright’s test object to be ready for custom fixtures.
2. Override the page fixture to listen to thrown JavaScript exceptions automatically. 

Sounds complicated? It’s not so bad, trust me!

Extend Playwright’s "test" object with custom fixtures

To write Playwright tests, you usually import test and expect from the @playwright/test package and get going, but guess what? Both objects are extendable.

To implement custom Playwright fixtures, you can extend the provided test object. 

import { expect, test } from "@playwright/test"

// rely on the standard `test` setup from Playwright
test("The standard Playwright setup", async ({ page }) => {
  // ...
})

// —-----------------------

// extend `test` to configure your Playwright setup
const myTest = test.extend({ /* ... */ })
// use your custom Playwright setup
myTest('Your custom Playwright setup', async ({ page }) => {
  // ...
} )

The extend method returns a new test object that includes all the Playwright functionality enriched with your custom additions. Assign this new test object to a variable (myTest) and use it to register your tests.

myTest can now be extended with test data or objects that have an established state, such as a loggedInPage for your product.

Let’s look at an extended test example. 

const myTest = test.extend<{
 testUser: { name: String }
 loggedInPage: Page
}>({
  testUser: {
    name: "Jenny Fish",
  },
  async loggedInPage({ page }, use) {
    await page.goto("/login")
    // more log-in actions before the test
    // ...

    // pass the `page` to tests that use `loggedInPage`
    await use(page)

    // clean up steps after the test
    // ...
  },
})

// `testUser` and `loggedInPage` fixtures are available in your tests now
//                                                  👇          👇
myTest("Your custom Playwright setup", async ({ testUser, loggedInPage }) => {
  // …
})

test.extend accepts an object with your fixture definitions. The property key becomes the fixture's name when used in your tests. To provide static data, define an object in your fixture configuration object (see testUser). It’s quick and easy.

If you want to add custom functionality and code that runs before and after your tests, pass in an async function (see loggedInPage). Functions will be called with a second parameter (use) that you must use to hand in objects to your tests. This approach lets you programmatically control what happens before and after your tests. 

Be sure to await your use() calls. Otherwise, it’s an asynchronous operation, and your test cases will fail if you don’t wait for it.

Custom fixtures for a testUser and loggedInPage make sense. Still, for constant error logging, I don’t like to include a new fixture (pageWithoutJSErrors 👎) but rather make JavaScript exception monitoring the enabled default in every test that uses page.

How would this work? 

Override Playwright’s "page" fixture

Luckily, overriding the built-in page object is also quickly done. Adjust the test.extend call and provide a new page function. And that’s pretty much it. 😅

// extend the test object and assign it to a new `myTest` variable
const myTest = test.extend<{ page: void }>({
  // override and use Playwright’s base `page` fixture 
  page: async ({ page }, use) => {
    const errors: Array<Error> = []

    page.addListener("pageerror", (error) => {
      errors.push(error)
    })

    // pass the `page` object to tests using it
    await use(page)

    expect(errors).toHaveLength(0)
  },
})

Custom fixtures and overrides can still use Playwright’s built-in fixtures. See above how our new page fixture also uses the original page object.

Thanks to use(), custom logic can run before and after our test cases. We can attach the JavaScript error event listener, pass the page object to the tests, and when the tests succeed, check if the error array is empty for every test case. It’s a clean and tidy solution!

And all this functionality is automatically available when you register tests using the extended myTest.

// use the extended test object with a new `page` fixture
// the `page` object now has event listeners attached to it
// and will fail if there’ve been JavaScript exceptions
myTest("Checkly blog lazy loading", async ({ page }) => {
  await page.goto("https://www.checklyhq.com/blog/")
  // all your test code
  // …
})

But what about situations where you don’t want to fail your test cases when JS exceptions appear? Can you make fixtures configurable?

Make your fixture configurable with fixture options

Extending your test object isn’t only about providing custom functionality. You can also use fixture options to make your tests configurable. Fixture options enable you to tweak your test configuration via your playwright.config or per-file test.use() calls.

To configure the error tracking, introduce a new failOnJSError option in your test extension.

const myTest = test.extend<{ page: void; failOnJSError }>({
  // introduce a Boolean fixture option (default: true)
  // this option is also accessible in every test case or fixture
  failOnJSError: [true, { option: true }],
  page: async ({ page, failOnJSError }, use) => {
    const errors: Array<Error> = []

    page.addListener("pageerror", (error) => {
      errors.push(error)
    })

    // run the test
    await use(page)

    // don’t check thrown exceptions if `failOnJSError` was set to false
    if (failOnJSError) {
      expect(errors).toHaveLength(0)
    }
  },
})

Do you see what we did there? There’s now a new failOnJSError fixture option, that’s used by the overridden page fixture. Your Playwright setup just became tailored to your needs, and you can now control if you want to fail your tests if the JavaScript throws or not. 💪

Here’s an example of a playwright.config that configures the new failOnJSError option.

import { defineConfig, devices } from '@playwright/test';

export default defineConfig({
  projects: [
    {
      name: "Project with many JS exceptions",
      // override and set the `failOnJSError` fixture option to false
      use: { ...devices["Desktop Chrome"], failOnJSError: false },
    },
  ],
})

You can also call test.use() in your spec files to set the option per-test.

myTest.use({ failOnJSError: false })


myTest("Checkly blog lazy loading", async ({ page }) => {
  // ...
})

Your entire Playwright setup just became more powerful and configurable.

But it’s not perfect yet!

Restructure your project to be ready for a scaleable fixture setup

At this stage, you have a single spec file that extends Playwright’s test object and a test case using it. Unfortunately, this approach is neither scalable nor reusable.

import { expect, test } from "@playwright/test"

// Having everything in a single file works
// But the custom fixtures aren’t reusable right now.

const myTest = test.extend<{ page: void; failOnJSError: Boolean }>({
  failOnJSError: [true, { option: true }],
  page: async ({ page, failOnJSError }, use) => {
    const errors: Array<Error> = []

    page.addListener("pageerror", (error) => {
      errors.push(error)
    })

    await use(page)

    if (failOnJSError) {
      expect(errors).toHaveLength(0)
    }
  },
})


myTest.only("Checkly blog lazy loading", async ({ page }) => {
  await page.goto("https://www.checklyhq.com/blog/")
  // your test code
  // ...
})

To make the fixture setup reusable, place the test.extend code into its own base.ts file and import it in your *.spec.ts files.

Here’s the new base.ts fixture file:

import { test as base, expect } from "@playwright/test"

// export the extended `test` object
export const test = base.extend<{ page: void; failOnJSError: Boolean }>({
  failOnJSError: [true, { option: true }],
  page: async ({ page, failOnJSError }, use) => {
    const errors: Array<Error> = []

    page.addListener("pageerror", (error) => {
      errors.push(error)
    })

    await use(page)

    if (failOnJSError) {
      expect(errors).toHaveLength(0)
    }
  },
})

// export Playwright's `expect`
export { expect } from "@playwright/test"

It includes the same functionality as our test file but now exports Playwright’s expect and the new extended test object. And because we have a base file with similar exports to @playwright/test, it can be used by all your *.spec.ts files by changing the import.

// use your extended Playwright setup
import { expect, test } from "./base"

test("Checkly blog lazy loading", async ({ page }) => {
  await page.goto("https://www.checklyhq.com/blog/")
  // your test code
  // ...
})

With this setup, you stop requiring test and expect from @playwright/test and import your custom Playwright setup from a base file. Provide data, reuse code, and make everything configurable in a Playwright-native and standardized way! 

It’s a win-win without spaghetti imports everywhere. And thanks to the overridden page fixture, you won’t miss thrown JavaScript exceptions when your end-to-end tests run.

Conclusion — test more with an extended Playwright setup

As mentioned at the beginning of this article, having 100% end-to-end test coverage is tough to achieve and most likely shouldn’t be your end goal. But if you’re not passively monitoring broken functionality while testing your core business, you’re simply missing out.

• Are all images loading?
• Is your JavaScript throwing exceptions?
• Are your network waterfalls plastered with red 500s?

You can passively monitor all these things while testing critical user flows.

And to be safe, ideally, you don’t rely on testing your preview or staging deployments but also monitor your production environment end-to-end. This environment is what your visitors see, after all. ;)

If you value high quality (and I hope you do!), you should get the most out of your tests; and hopefully, this guide helped you do that.

If you have any questions or comments, say hi in the Checkly Community. 👋 We’re a lovely bunch, I promise!