Synthetic Monitoring: What is it, Challenges, and How to Get Started

What is Synthetic Monitoring?

Synthetic Monitoring, or ‘Synthetic User Monitoring’ is the process of sending an automated browser to your site and simulating basic user actions. This synthetic user then reports back how the site performs, possibly testing the output as to how it meets various predefined tests.

Before diving into synthetic monitoring, let's take a step back and consider automated testing.

Automated tests aren't new in software development. Synthetic tests focus on emulating user behavior, from pinging an HTTP endpoint to controlling real browsers to mimic a realistic user session with application transactions. Synthetic tests allow you to automate critical application flows such as adding a product to a shopping cart or logging in to an account. Once your tests fail, hold production deployments to figure out what broke and prevent a negative user experience in production. 

Synthetic tests are essential for high-velocity development teams, but unfortunately, they can't lead to complete confidence. Knowing that your application works during or shortly after a production deployment doesn't help you detect future user experience issues. Maybe your infrastructure is struggling under high load, or a third-party vendor is experiencing downtime; you must constantly monitor user experience to be safe.

Transforming and reusing your synthetic tests as monitors is the only way to know that your application always works! 

How Does Synthetic Monitoring Work?

When you adopt synthetic monitoring (sometimes also called active monitoring), you constantly run synthetic tests against your production environment — you create and run your headless browser or API tests during development, and ideally, you follow the Monitoring as Code workflow to reuse them to monitor your production application.

Emulating typical user behavior by testing your mission-critical transaction in high intervals helps you gain confidence that your entire application operates smoothly at any time. And if your application serves a global audience, running synthetic monitoring from multiple locations guarantees you won't miss regional issues.

Synthetic monitoring enables you to be on top of your game and notice issues before your customers do. But there's more to it!

Importance of Synthetic Monitoring in Application Performance Monitoring

At its core, synthetic monitoring is about simulating user interactions with an application to preemptively identify issues before they impact real users. Our users expect digital experiences to be seamless and uninterrupted. Synthetic monitoring allows developers and operations engineers to stay one step ahead, ensuring that potential problems are identified and resolved swiftly.

Synthetic monitoring plays a pivotal role in understanding an application's behavior under various scenarios. It allows teams to test how their application performs in different geographical locations, under various network conditions, and during peak load times. This insight is critical for applications that cater to a global audience, where performance can vary significantly based on a user's location and the time of day.

Synthetic Monitoring vs Real User Monitoring (RUM)

Real user monitoring, also called RUM or passive monitoring, is another way to analyze and monitor your application. RUM monitoring offers deep insights into user interactions, performance statistics, and your users' devices and locations. 

Unlike synthetic monitoring tools, RUM monitoring tools don't simulate user interactions. There's nothing for you to script or define. With RUM, an embedded JavaScript snippet records and reports the behavior of actual users back to you so that you can monitor and analyze your applications' performance and health. Your users are the data source!

But when should you use which? The answer is the usual, "it depends". An active monitoring approach enables you to notice and get alerted about issues proactively. Testing and monitoring before and after your production deployments help massively speed up development and prevent production issues. Your synthetic user session unveils and alerts you about production issues before a customer notices.

That said, though, passively monitoring real users provides insights into behavior and environments you probably aren't aware of! Maybe your users use low-end devices, or you have a customer base in a particular part of the world. Real user monitoring gives you these insights. Ideally, you bet on a combination of synthetic and real user monitoring to have all the user experience insights.

We at Checkly are big fans of synthetic monitoring, though; why should you invest in it?

Types of Synthetic Monitoring

There are a number of terms used around synthetic monitoring; everything from "Synthetic User Monitoring" to "Heartbeat Checks" can refer to the same test! However, in general, there are three types of monitoring that are most critical for monitoring your production services:

Availability Monitoring

Availability Monitoring is fundamental. Its primary objective is to verify whether a web service or application is accessible at any given time. This type of monitoring simulates user interactions to check the availability of websites, APIs, and servers. It's not just about confirming that a server is up and returning "200 OK" status messages; it's about ensuring that the application is operational and responding as expected.

Transaction Monitoring

Transaction Monitoring takes a step further. To know that every part is really working as expected, Transaction Monitoring means simulating complex user transactions or workflows to verify that critical processes, like checkout or login, are working as intended.

Scripts are designed to mimic the path a user would take through the application. This can be as simple as logging in or as complex as completing a multi-step transaction. The goal is to identify performance issues or bugs that might not be evident in other forms of testing. For example, this will reveal whether third-party services have failed, resulting in degraded performance on our own site.

Web Performance Monitoring

Web Performance Monitoring is all about speed and efficiency. It evaluates how quickly and smoothly your web application loads and operates from the user's perspective. This is a more analog measurement than the Availability or Transaction Monitoring, as exact loading times will vary and won't give simple "up or down" feedback. Web Performance Monitoring can identify technical debt, as small changes to your experience can add up to degraded (or improved!) performance for your end users.

Benefits of Investing in Synthetic Monitoring

Synthetic monitoring on the back of headless browser automation enables development teams to ship fast and confidently. 

Let's name more synthetic monitoring benefits.

1. Know That Your Application is Available Online

The most terrifying scenario when running an online business is being offline. Downtimes can be caused by DNS misconfiguration, hosting issues, developers deploying a bug, and countless other reasons. If you bet on synthetic monitoring, you'll know that your apps and their APIs are available online, and if they're not, you'll be the first one to be alerted in case of a failure.

But synthetic monitoring has way more to offer than classical uptime monitoring!

2. Quickly Detected Mission-Critical Issues

Considering your application's essential features, how long would it be acceptable for these to be broken? The answer to this question is your ideal monitoring interval. 

You can only fix production issues you know about. If making a purchase is your core business, you probably don't want to test this functionality only once a day after a production deployment. 

Synthetic monitoring enables you to test your core functionality daily, every hour, or even every minute. The shorter your synthetic monitoring interval, the quicker your mean time to detect (MTTD) will be. A short MTTD will enable you to fix production issues before your customers reach out to your support channels!

3. Less Noise and More Meaningful Alerts

Betting on user experience testing with synthetic monitoring leads to more meaningful alerts. A failed transaction might be an issue, but your infrastructure could handle it gracefully. Alerts based on a broken user experience tell you the entire story and must be treated critically and acted on immediately.

4. More Transparency with Monitored Third-Party Services and Vendors

Depending on your application, third-party providers could be essential to your offering. Be it a SaaS service to handle your logins or a cloud database storing your user data, all these services could run into issues at any time. If your third party's downtime can become your downtime, synthetic monitoring helps you detect these issues. 

By testing user experience, you'll get alerted when things are broken. Whether it's your or your third parties' fault, synthetic monitoring informs you about issues so you can act quickly.

5. Clear Performance Benchmarks

Another benefit of running synthetic monitoring with headless browsers is that you can monitor performance implications while testing core functionality. Is your web app fast enough for customers in Australia? Does it provide a good Core Web Vitals experience? Do core flows like the customer login become slower over time?

Many things can cause performance degradation, but synthetic monitoring will unveil a slower user experience with aggregated performance metrics. You can't fix things you don't know about!

But synthetic monitoring also comes with some challenges; let's look at them.

Challenges in Synthetic Monitoring

While the idea of synthetic monitoring is compelling, establishing it across an organization also comes with technical and organizational challenges.

1. Complexity

Building and maintaining modern applications is already complex. To establish well-running synthetic monitoring practices, it must also seamlessly integrate into your developers' software development lifecycle.

If synthetic monitoring is too hard or cumbersome to set up, and configuring it "just" adds another layer of complexity, you won't succeed because the development teams won't adopt it.

At Checkly, we believe in Monitoring as Code, which allows you to banish "ClickOps" and handle your monitoring needs the same way you handle your deployments — automated and in code!

But when your synthetic monitoring setup lives in code and can be deployed with a single command, who owns it?

2. Shared Ownership

Earlier, development and operations were two different disciplines. However, the DevOps era and its connected mindset, "You build it, you run it." changed the game.

The same teams were responsible for building and operating their features and applications in production. The same principles should apply to your synthetic monitoring efforts. Development teams must consider building and maintaining but also monitoring their features and applications.

3. Cost

Whenever we discuss the ideal setup for synthetic monitoring, experienced DevOps and SRE engineers will respond: "Sure, that would be great, if we had the budget for it." While it would be nice to simulate a complex user behavior every 30 seconds from 5 different geographic regions, the cost of most synthetic tools makes that cost-prohibitive.

4. False Alerts

Regarding synthetic monitoring with headless browsers, flaky tests are a significant challenge. Since your tests run on a schedule, failures won't just hold a production deployment; flaky results will lead to useless alerts waking up your on-call engineers.

Avoiding false positive results and creating a solid synthetic monitoring test suite must be one of your core priorities, or you won't succeed.

How Do You Get Started with Synthetic Monitoring?

To emulate real user interactions and test your mission-critical flows end-to-end, you must be able to control your users' environment. Headless browser automation is essential for this. But what tools can you use to automate browser operations?

Synthetic Monitoring Tools

To do synthetic monitoring right, you'll either need to run a service totally independently of your existing tech stack or use a SaaS tool. A system to monitor your production services shouldn't go down when your own system does! You'll also need a tool to store past performance data and present it within a dashboard.

Playwright - The Leader in Synthetic Test Scripting

Backed by Microsoft, Playwright quickly became one of the leaders in synthetic and end-to-end testing. Its ability to control headless browsers while providing a stellar developer experience convinced the developers, quality assurance, and DevOps communities.And we at Checkly believe it's the best tool for synthetic monitoring.

Checkly - The best platform for running Synthetic Tests

While multiple Observability vendors offer limited site pings as part of a larger toolset, Checkly offers the best experience for users focused on synthetics testing. While other teams use Playwright only for the occasional end-to-end checks, Checkly lets you run these detailed checks with complex scripted behavior at any cadence you need to maintain your SLA. And Checkly can cost less than half of what our competitors charge. 
Checkly offers rich options for scheduling tests on a cadence, including scheduling checks from multiple regions, and offers the most complete answer to ‘how is our service performing for all our users?’

Tips for Synthetic Monitoring Excellence

Fight Flaky Tests with Auto-Waiting

As mentioned earlier, avoiding flaky tests and false-positive alerts must be a core priority when setting up synthetic testing and monitoring. Playwright battles this problem with auto-waiting mechanisms and user-first actions and assertions. Focus on testing and validating your application features instead of monitoring application internals!

Time Travel Debugging with Traces

Another headless testing challenge is the "it works on my machine" problem. How can you debug failed tests that were executed in a remote environment? Playwright lets you record every test action and takes network and HTML snapshots of your application's state. All this information is put together in a Playwright trace file for your investigation.

Do your tests fail in CI/CD? Open the generated trace file, travel back in time, and inspect your application's state to see what caused the test failure.

Excellent Developer Experience

And lastly, to reach high synthetic testing and monitoring coverage, development teams must enjoy writing tests. Playwright's ecosystem provides valuable tools to generate tests, debug them in your favorite editor, and run/write your test cases side by side for immediate feedback!

In summary, Playwright is a valuable tool to level up your synthetic testing game.

But it doesn't provide monitoring capabilities. How could you schedule your tests, get alerted, and gather meaningful monitoring data over time?

Monitoring as Code — Synthetic Testing and Monitoring United in a Single Workflow

Previously, synthetic testing and monitoring were different practices. They were owned by different teams choosing various tools. At Checkly, we believe in a code-first monitoring approach that unites testing and monitoring.

Leverage Monitoring as Code (MaC) to run and develop your synthetic Playwright tests locally. Run them in the global Checkly infrastructure to test your preview deploy environments. If all your synthetic tests pass, reuse and transform the same test code to synthetic monitors in the Checkly cloud.

By adopting Monitoring as Code, development teams can rely on a single tool to test and monitor their app's user experience. This approach removes friction, nurtures team collaboration, and leads to faster issue discovery and resolution in production environments!

Conclusion

Synthetic monitoring forms the foundation of shipping a seamless user experience. It is a safety net for development and DevOps teams, allowing them to innovate confidently. By integrating synthetic monitoring with a MaC approach, we create a bridge between testing and monitoring, fostering a collaborative environment that enhances the overall health of your application.

Synthetic monitoring and Monitoring as Code anticipate and resolve issues before they impact users and streamlines the process of maintaining a high-performing, user-centric application. In the fast-paced world of development, synthetic monitoring and MaC are tools and essential practices that ensure your application consistently delivers an optimal user experience.

Because you have to remember you can only fix issues you know about.