E2E Testing in Production with Playwright and Checkly

We all do testing. We all test code as we write it, and before we release it. But somehow regressions, bugs and even downtime still happens. Code that worked perfectly on our laptop somehow fails when it meets production, and in ways so obscure that simple integration tests, even canary deploys, don’t catch every failure.

When we want to ensure reliable software, traditional testing methods often fall short at monitoring real-world scenarios. Enter Playwright and Checkly, two powerful tools that enable end-to-end (E2E) testing in production. In this guide, we’ll explore how these tools work together to provide robust monitoring and testing solutions, ensuring your application performs flawlessly for end users.

The Problem: Monitoring in a Complex World

Modern software systems are no longer simple. With the rise of cloud-native technologies, microservices, and distributed systems, applications have become more complex than ever. Front-end frameworks like React and Angular, each implementing modeling that’s more complex than an entire 2010 web application, have added further edge cases and wrinkles. Questions like ‘if this microservice goes down, will users still be able to add items to their cart?’ are harder to answer than ever.

The result? A flood of monitoring data—logs, metrics, and traces—that can be overwhelming. While observability tools have emerged to help manage this data, they often come with a hefty price tag and can be noisy, making it difficult to extract clear signals about whether your application is truly working.

The core question developers and operations teams need to answer is: “Does my app still work?” This means ensuring that the application provides the expected business function correctly and timely, not just in pre-production environments but in production, where real users interact with it.

Why Traditional QA Falls Short

Traditional QA processes, including unit tests and end-to-end tests, are essential but often stop at the pre-production stage. Once the application goes live, monitoring takes over, typically handled by a different team using different tools. This disconnect between QA and monitoring creates gaps in visibility and reliability.

In the ideal SDLC, Playwright helps you test code pre deployment, and make sure everything’s working

A simple thought experiment is to ask:

• Can we agree that no level of pre-deployment QA will catch every failure? And if so,
• Is it acceptable that everything not caught by QA will have to be found and reported by a user?

If your answers to these questions is ‘yes’ and ‘no’ respectively, it’s time to think about going beyond traditional QA.

Testing in production is often treated as a meme, but it shouldn’t be. The reality is that production environments are fundamentally different from pre-production environments. Differences in data, security configurations, third-party services, and geographic distribution mean that testing in staging or UAT environments doesn’t guarantee success in production.

In the real world, problems appear at lots of stages, and Playwright can also help you find these edge cases, with the power of Checkly

Enter Playwright: A Modern Browser Automation Tool

Playwright is Microsoft’s open-source browser automation framework that simplifies end-to-end testing. It supports multiple platforms and provides a robust API for interacting with web pages and APIs. Here’s why Playwright stands out:

1. Web-First Locators and Assertions: Playwright uses web-first locators, making it easier to interact with dynamic web elements without relying on brittle CSS selectors.
2. Automatic Retries and Waits: Playwright automatically retries actions and waits for elements to be visible, reducing flaky tests.
3. Powerful Debugging Tools: Playwright provides traces, videos, and screenshots to help diagnose issues quickly.
4. API Testing: Beyond browser automation, Playwright can also handle API testing, making it a versatile tool for end-to-end workflows.
5. Detailed Assertions: No more do we call every 200 OK response a passing test, Playwright can make complex, programmatic assertions about the responses it receives, checking things like ‘every loaded record has a connected ID’ or ‘all buttons are set to be a brand-approved color’

Testing in Production with Playwright and Checkly

While Playwright excels at browser automation, Checkly takes it a step further, automating runs of Playwright tests and enabling monitoring as code. Checkly enables you to run and schedule Playwright tests against production environments, providing real-time insights into your application’s health. Run a Playwright test every hour or every minute from locations all across the globe, all managed from a workflow that fits into your current CI/CD model, rather than requiring a cumbersome interface to manage monitors.

Key Features of Checkly:

1. Global Test Execution: Run your Playwright scripts from multiple geographic locations to ensure your application performs well for users worldwide.
2. Monitoring Cadence: Run tests every minute or every hour, and set a monitoring cadence that defends your SLA
3. Alerting and Notifications: Set up alerts via email, Slack, PagerDuty, or other channels to notify your team of failures.
4. Connect Backend Traces: With Checkly Traces, harness the power of OpenTelemetry to see backend traces connected to each testing session.
5. Code Workflow Integration: Manage your monitoring scripts as code, enabling version control, collaboration, and CI/CD integration.

A Practical Example: Monitoring an E-Commerce Workflow

Let’s say you’re running an e-commerce platform. Any regression, missing feature, or unavailability means users can’t check out on your site, so even a few minutes of a problem costs your business money. A critical user journey might involve:

1. Logging into the application.
2. Searching for a product.
3. Adding the product to the cart.
4. Checking out.

A typical web development process would have QA or developers run tests before deployment, and then ‘monitor’ deployments to make sure the site is still available after releases. Playwright enhances this pre-deployment testing by letting you simulate user behavior in detail. However, when features don’t work as expected on production, in a way that wasn’t detectable in a QA environment, you might end up relying on your users to tell you. By the time problems are identified, revenue (and trust) has already been lost.

With Playwright, you can automate the process of testing your site, removing the need for a human to click through your pages. And with Checkly, you can run your playwright tests on a cadence, meaning your monitoring will be the first place you’ll hear about problems with the critical user paths. Here’s how Checkly enhances the process of releasing new code:

1. Write the Playwright Script: Automate the login, product search, and checkout steps. Run initial tests through the Checkly network to make sure it’s working from all your users’ geo locations.
2. Configure Monitoring Settings: Define how often the script should run (e.g., every 15 minutes) and from which locations (e.g., Ireland, Frankfurt).
3. Set Up Alerts: Notify your team if the script fails, ensuring quick resolution of issues.
4. Deploy to Production: Use Checkly’s CLI to deploy the script and start monitoring your production environment.
5. Use Hooks to Clean Up After Tests: If you’ve run a real ecommerce site you’d know, you can’t have your database clogged with test run data, throwing off your analytics, but with Checkly and playwright you can use hooks to make sure that each test makes the necessary calls to clean up after itself.

With Checkly running your Playwright monitors, you can find and resolve issues before your users are affected.

Solving the Cultural Problem: DevOps in Practice

One of the biggest challenges in monitoring and testing is the cultural divide between development and operations teams. By adopting monitoring as code, you bridge this gap. Developers can write and maintain monitoring scripts using the same tools and workflows they use for application code. This approach aligns with the core principles of DevOps, where operational problems are solved using development practices.

Conclusion

Testing in production is no longer a meme—it’s a necessity. With tools like Playwright and Checkly, you can automate end-to-end testing, monitor your application in real-time, and ensure it delivers a seamless experience to users. By treating monitoring as code, you empower your team to proactively address issues, reduce downtime, and improve overall reliability.

Whether you’re running a small startup or a large enterprise, integrating Playwright and Checkly into your workflow can transform how you approach testing and monitoring. So, start testing in production today—your users will thank you!