Sara Miteva

OpenTelemetry (OTel) Metrics are part of the OpenTelemetry project, which provides tools, APIs, and SDKs for telemetry data collection. These metrics capture system performance data like request latency, error rates, resource usage, and throughput. OTel aims to standardize observability across languages and platforms, making it easier to use and integrate telemetry data.

Metrics are one of three core signals of OpenTelemetry along with logs and traces.

To start with OTel Metrics, familiarize yourself with the OpenTelemetry API and SDKs for your preferred language. Instrument your code using the API to create and record metrics with counters, gauges, and histograms. Set up an OpenTelemetry Collector to collect and process these metrics, then export the data to monitoring systems like Prometheus or Grafana. Using community-provided libraries and following OTel conventions will ensure your metrics are consistent and useful.

OpenTelemetry Metrics are helpful for various reasons. Here are some key points to understand their usefulness:

Track system performance metrics like latency and throughput. For example, monitor API request times to identify slow endpoints. Using OTel metrics, you can gain detailed insights into the performance of different parts of your system, helping you pinpoint bottlenecks. This allows for more effective tuning and optimization of your infrastructure and code.

Identify and quantify errors and failures. For instance, measure error rates to quickly spot and address issues in your application. With OTel, you can set up alerts based on these metrics to get notified of problems in real-time. This proactive approach can significantly reduce downtime and improve the reliability of your services.

Monitor resource usage such as CPU and memory. Example: Optimize server performance by tracking CPU usage spikes. By using OTel metrics, you can correlate resource usage with application performance and user load. This helps in making informed decisions about scaling and resource allocation, ensuring efficient use of infrastructure.

Consistent Observability with Synthetic Monitoring

If you’re already using synthetic monitoring, standardizing metrics collection across services and platforms could be particularly useful. For example, use OTel to gather consistent metrics from microservices written in different languages. This uniformity, enhanced by synthetic monitoring, simplifies the integration of metrics from diverse systems, providing a cohesive view of your entire architecture. 

Understanding the basics of OpenTelemetry Metrics concepts helps engineers gather and analyze data about their applications' performance. This knowledge empowers them to make smarter decisions and improve how their systems run.

The API (Application Programming Interface) of OpenTelemetry Metrics provides a way for developers to instrument their code and capture various metrics. It offers a set of functions and methods that developers can use to define and record metrics within their applications.

The SDK (Software Development Kit) is a collection of tools and libraries that implement the OpenTelemetry API. It provides the necessary components to integrate telemetry instrumentation into applications written in different programming languages, such as Java, Python, Go, and more.

OpenTelemetry defines a standard format for representing metric data. This format ensures consistency and interoperability across different telemetry systems and components. By adhering to this format, developers can easily exchange metric data between various parts of their application stack.

Exporters are responsible for sending metric data collected by OpenTelemetry to external monitoring systems or storage backends. OpenTelemetry supports a variety of exporters, including those for popular monitoring platforms like Prometheus, Jaeger, and Grafana.

Types of Metrics in OpenTelemetry + Examples 

OpenTelemetry offers various types of metrics for monitoring applications effectively, providing insights into performance and behavior. Understanding these metrics is essential for optimizing your monitoring strategy:

A Counter is a simple metric that keeps track of how many times something happens. For example, you can use a Counter to count the number of requests your application receives or the number of errors it encounters. Because of its intent as a steadily increasing number, counters only accept positive numbers.

This code sets up OpenTelemetry to export metrics to the console every 60 seconds using the PeriodicExportingMetricReader. The ConsoleMetricExporter is used to print the metrics to the console. The counters app_requests_total and app_errors_total are created to track the number of requests and errors, respectively. The handleRequest function simulates requests and increments the counters accordingly.

A Gauge is a metric that represents a single numerical value that can go up or down. For example, you can use a Gauge to track the current temperature or the number of users currently logged into your application.

Here we set up an OpenTelemetry MeterProvider and use the ConsoleMetricsExporter to output metrics to the console. We create a Gauge to track the current number of users logged into the application. The getCurrentUsers function serves as a callback to provide the current value of logged-in users, and it is registered with the gauge.

The functions userLogin and userLogout simulate user logins and logouts by incrementing and decrementing the currentUsers variable. Finally, the metrics are exported to the console. In a real application, you would configure an appropriate backend exporter and set up periodic exporting.

A histogram is a metric that divides a range of values into buckets and counts how many times each value falls into each bucket. This can be useful for understanding the distribution of values, such as response times or request sizes.

In this snippet, we create a Histogram to track the response times of requests in milliseconds.

The recordResponseTime function simulates recording response times by adding values to the histogram. We then simulate some response times by calling this function with various values. Finally, the metrics are exported to the console. In a real application, you would configure an appropriate backend exporter and set up periodic exporting.

A Summary is similar to a Histogram, but instead of counting values into buckets, it calculates things like the sum, count, and quantiles of the observed values. This can be useful for understanding the distribution of values and identifying outliers.from opentelemetry import metrics.

In this snippet, we create a Histogram to track and summarize the response times of requests in milliseconds.

The recordResponseTime function simulates recording response times by adding values to the histogram. We then simulate some response times by calling this function with various values. 

Note: OpenTelemetry API does not provide a direct "Summary" metric type like Prometheus. However, histograms with explicit bucket configurations can achieve a similar summarizing effect, which includes calculating sum, count, and providing insights into distribution.

In addition to these, here are some more OTel metrics: 

CounterObserver: Similar to a Counter, a CounterObserver also counts occurrences of events, but it allows you to attach additional information to each event. This can be useful for tracking things like the size of requests or the severity of errors.

UpDownCounter: An UpDownCounter is like a Counter, but it can increase or decrease its value. This can be useful for tracking things like the number of active connections to a server or the amount of memory being used by an application.

UpDownCounterObserver: Similar to an UpDownCounter, an UpDownCounterObserver can increase or decrease its value and allows you to attach additional information to each event. This can be useful for tracking changes in resource usage over time.

GaugeObserver: Similar to a Gauge, a GaugeObserver represents a single numerical value, but it allows you to attach additional information to each value. This can be useful for tracking things like the size of a database or the latency of network requests.

OpenTelemetry provides several types of instruments for capturing and recording metric data within applications. These instruments serve different purposes and are suited to various use cases:

Timeseries: Timeseries instruments track the evolution of a metric value over time. They are useful for monitoring trends and patterns in application behavior. Examples of timeseries metrics include CPU usage, memory consumption, and request latency.

Additive: Additive instruments accumulate metric values over time by adding new data points to the existing total. They are well-suited for measuring cumulative quantities, such as the total number of requests processed or the total amount of data transferred.

Synchronous: Synchronous instruments record metric data immediately when called within the application code. They provide real-time visibility into application performance but may introduce overhead due to the synchronous nature of data collection.

Asynchronous: Asynchronous instruments defer the recording of metric data to a separate thread or process, allowing the main application logic to proceed without waiting for metric collection to be completed. This approach minimizes performance impact but may result in slightly delayed metric reporting.

Choosing the right instrument type depends on the specific requirements and characteristics of your application. The following table outlines some considerations to help you decide:

Real-time visibility, suitable for continuous monitoring

May require significant storage space for long-term data retention

Simple accumulation of values, easy to interpret

May not provide detailed insights into short-term fluctuations

Immediate feedback on application performance

Potential performance overhead due to synchronous data collection

Minimal impact on application performance, can handle bursts of data

May introduce slight delays in metric reporting

By considering factors such as the frequency of metric updates, the importance of real-time visibility, and the trade-offs between performance and accuracy, you can determine the most appropriate instrument type for your monitoring needs.

Now, let's explore the different kinds of metrics you can keep track of using OpenTelemetry.

HTTP response time measures the duration between sending a request to a server and receiving a response. It's crucial for assessing the performance of web applications and detecting any anomalies that might affect user experience.

Error rate indicates the frequency of errors occurring within an application. Monitoring error rate helps identify bugs, misconfigurations, or other issues that can impact reliability and user satisfaction.

Throughput measures the rate at which an application processes requests or transactions within a given timeframe. It's essential for ensuring that the system can handle the expected workload efficiently.

Network latency refers to the time it takes for data to travel from one point to another over a network. High network latency can lead to delays in communication between components, affecting overall system performance.

Database query metrics track the performance of database operations, including execution time, throughput, and error rate. Monitoring these metrics helps optimize database performance and identify potential bottlenecks.

Memory utilization metrics provide insights into how effectively an application uses available memory resources. Monitoring memory usage helps prevent memory leaks and optimize resource allocation.

CPU usage metrics indicate the percentage of CPU capacity consumed by an application. Monitoring CPU usage helps identify performance bottlenecks and optimize resource allocation for better efficiency.

Now, let's look into some practical examples of OpenTelemetry Metrics to demonstrate their implementation.

In this example, we use a counter metric to track the number of emails sent by our application. We initialize the emailsSentMetric as a counter metric, and each time the sendEmail() function is called, we increment the counter by one using the .add() method. This allows us to monitor the volume of emails sent over time.

Here, we create a histogram metric to measure the latency of an operation within our application. The performOperation() function calculates the duration of the operation by measuring the time before and after its execution. The resulting latency is then recorded in the operationLatencyMetric histogram using the record() method. This allows us to analyze the distribution of operation latencies over time.

This example demonstrates the use of a gauge metric to track the cache hit rate of our application's caching system. The calculateCacheHitRate() function calculates the hit rate based on the number of cache hits and misses, then sets the value of the cacheHitRateMetric gauge accordingly using the observe() method. This allows us to monitor the effectiveness of our caching strategy over time.

In this example, we use a counter metric to track the error rate of our application. The handleError() function is called whenever an error occurs, and it increments the errorRateMetric counter by one using the inc() method. This allows us to monitor the frequency of errors and identify any patterns or trends over time.

Best Practices for Using OpenTelemetry Metrics

Understanding and implementing best practices for using OpenTelemetry Metrics is important for effectively monitoring the performance of your applications. By following these guidelines, you can ensure that your metrics accurately reflect the behavior of your systems and provide valuable insights into areas for optimization and improvement.

Use clear and consistent naming conventions for your metrics. Names should be descriptive and follow a standard format, such as component_metric_type (e.g., http_server_requests_total).

Consistent naming helps in easily identifying and understanding metrics across different services and components.

Labels and attributes provide crucial context to metrics, making them more informative. However, excessive labels can complicate analysis and increase storage needs. Select labels that add meaningful differentiation without overwhelming the dataset.

Using attributes wisely improves metric usefulness. For instance, adding a user role label to a login attempts counter allows for detailed analysis of login patterns, aiding in security monitoring and user behavior insights.

Utilize histograms, summaries, and other aggregation techniques to capture and analyze the distribution of your metrics. This approach provides a more comprehensive view of your application's performance by showing not just averages but the entire range and frequency of values. 

Aggregations help in identifying outliers and trends that might be missed with simple averages. For example, a histogram can reveal latency spikes in certain transactions, helping you pinpoint specific issues and optimize performance accordingly.

Continuously review and update your metrics to ensure they remain relevant and useful. As your application evolves, some metrics may become obsolete, while new metrics may be needed to monitor new features or components. 

Regular audits of your metrics will help you retire outdated ones and introduce new metrics that align with current business goals and technical requirements. This practice ensures your monitoring strategy adapts to changes and provides ongoing value in performance tracking and troubleshooting.

How to Get Started with OpenTelemetry Metrics and Checkly

With Monitoring as Code at the core of our synthetic monitoring offering, we enabled our customers to simulate user behaviors by setting up monitoring checks. Now, we’re integrating with OpenTelemetry to correlate synthetic checks with backend traces and pinpoint the reasons for the check failure. 

By correlating synthetic checks with tracing data, you can filter out unnecessary information and focus on the most critical data points. This ensures that both development and operations teams can zero in on the telemetry that directly impacts key business processes, reducing the noise and improving overall efficiency.

We’re starting our OTel integration with traces, now in BETA. This update offers two key benefits: easier trace-error correlation and simpler trace setup. Once configured, each check run provides an immediate view of your end-to-end traces, from the frontend or API endpoints to the backend. This end-to-end visibility helps you quickly pinpoint the root causes of failing check runs.

Enable tracing instrumentation on all HTTP requests with one click.

Instrument your code with the relevant OpenTelemetry (OTel) SDKs.

Send only traces "triggered" by Checkly checks to our backend and get the results in your check results and even live in the editor. This also works for test session triggered by our CLI.

Here’s a quick demo for a better understanding of the integration: 

Checkly Open Telemetry Tracing Demo

Instrumenting your code with OpenTelemetry SDKs

Ingesting traces into Checkly using the OpenTelemetry collector

If you’re a Coralogix user, check out our Coralogix integration

By starting to use our OTel integration, you can leverage the power of OpenTelemetry Metrics with Checkly to gain a deeper understanding of your applications' performance and behavior.

OpenTelemetry Metrics provide a standardized and flexible way of collecting, aggregating, and exporting metric data from your applications and services. By instrumenting your code with OpenTelemetry Metrics, you can gain insight into the performance and behavior of your systems, enabling you to identify issues, optimize performance, and deliver a better user experience. With Checkly's OpenTelemetry integration, getting started with metrics monitoring is easier than ever, allowing you to focus on building and delivering great software.

Get started with Checkly for free or schedule a customized demo. 

OpenTelemetry Metrics: Concepts, Types & Instruments

Detect

Communicate

Resolve

Monitoring as Code

Developers

Resources

Community

OpenTelemetry Metrics: Concepts, Types & Instruments