Table of contents

Writing assertions for API checks

The response of an API request can be checked for correctness and timeliness by using assertions on the response data. Assertions are flexible statements that combine preset modifiers with custom values to meet the needs of a broad set of use cases.

Assertions are statements you create that check one aspect of the HTTP response. You can create multiple assertions for one check that assert various aspects of a response, for example:

• HTTP response status equals 200.
• HTTP response body equals the text “success”.
• HTTP response time is lower than 2000 milliseconds.
• HTTP response header “X-Custom-Header” equals “SomeValue”.
• HTTP response JSON object has a key called “accountBalance” with a value greater than 9999

In each assertion, a source is connected to a comparison and a target.

In some cases a property is added, for example when asserting headers or JSON response bodies.

Assertions are executed from top to bottom. If one assertion fails, the full check is considered as failed.

On each API check assertion, the following sources are available:

• Status code: The HTTP response status, parsed as an integer.
• JSON body: The response body parsed as a JSON object.
• Text body: The response body as plain text.
• Headers: The response headers as an array of key/value pairs.
• Response time: The response time of the full API request in milliseconds, parsed as an integer value.

The property field is a free form text field used to identify the name of a header, the part of a JSON object using JSON path or part of a text body.

• With JSON response bodies, the property field accepts JSON path expressions in the form of dot-separated strings to target nested properties in an object, i.e. $.product.size or an item in an array, i.e. $.[1].key

• With text response bodies, the property field accepts a regular expression with a capture group to pick out parts, i.e. <!doctype (.{4}) would grab the word html from a body return <doctype html>

• With headers, you provide the header you want to assert in the property field, i.e. Content-Type. You can even add a regular expression after that to tease out a specific part of the header.

Read more about asserting JSON response bodies below.

Comparisons are the operators that work on the source data and target data, e.g.

• Response time is LESS THAN 150 milliseconds.
• Status code EQUALS 200.
• Header X-MY-HEADER CONTAINS the string some value.

The following comparisons are available. Note that some comparisons don’t make sense when paired with a specific source. Response time is empty? JSON Object is less than? We block out the comparisons when they are not applicable to the source.

• Equals / Not equals
• Is empty / Not empty
• Greater than
• Less than
• Contains / Not contains
• Is null / Not null
• Has key / Not has key (deprecated)
• Has value / Not has value (deprecated)

The target field is a free form text field that determines the desired outcome of your assertion.

You can use JSON path to specify which field of a JSON response body should be asserted. JSON path is a query language similar to Xpath for XML, but in general a lot more intuitive and simpler to use.

JSON path only uses a handful of operators in its queries. Not all of them are useful in the context of assertions. Here is an adapted set of examples based on Stefan Goessner’s 2007 introduction post.

Given this sample data set, see example expressions below:

{
  "store": {
    "book": [ 
      {
        "category": "reference",
        "author": "Nigel Rees",
        "title": "Sayings of the Century",
        "price": 8.95
      }, {
        "category": "fiction",
        "author": "Evelyn Waugh",
        "title": "Sword of Honour",
        "price": 12.99
      }, {
        "category": "fiction",
        "author": "Herman Melville",
        "title": "Moby Dick",
        "isbn": "0-553-21311-3",
        "price": 8.99
      }, {
         "category": "fiction",
        "author": "J. R. R. Tolkien",
        "title": "The Lord of the Rings",
        "isbn": "0-395-19395-8",
        "price": 22.99
      }
    ],
    "bicycle": {
      "color": "red",
      "price": 19.95
    }
  }
}

JSON path expressions using the store example above:

Use this online editor to play around, or look at the examples below. We use this jsonpath NPM module under the hood.

Asserting string, boolean and number values works exactly as you’d expect, e.g. the example below asserts the number value of the id property is greater than 2000.

You can traverse a JSON object using a dot notation. In the example below we are checking the string-based size property that is part of the product object in the JSON response.

This next example checks for a boolean value in the owner.site_admin property:

For response bodies with JSON arrays you use JSON path’s [] expressions.

In the first example below we check if the first item in our result array has a property title:

In the next example we pick the last item in the array and check if the customerId property has the value 123abc:

In this example we pick the item with index value 4. This is the 5th item as array indexes start at 0. We then assert that the responseTime property is less than 2000.

In the last example we check if the returned array has more than 10 items.

Before we released the feature to use JSON path queries, Checkly had a custom way of querying the contents of a JSON response body.

This method is still available for backwards compatibility reasons, but no longer advised as it is much less flexible and powerful.

Regular expressions give you the power to extract specific parts of text from a larger text using capturing groups. You can use regular expressions with two assertions sources:

1. Text body: Use the property field to add your regex.
2. Headers: First select the header you are interested in in the property field, then click “add regex”.

Under the hood, we use the stock Javascript regular expressions implementation. Specifically, we use the .match() method. We do not use the /g modifier and return the first matched group the expression finds.

Sounds more complicated than it is. Here is an example:

const paragraph = 'The quick brown fox jumps over the lazy dog. It barked.'
const regex = /quick (.*) fox/
const found = paragraph.match(regex)
console.log(found)

// [ 'quick brown fox',
//  'brown',
//  index: 4,
//  input: 'The quick brown fox jumps over the lazy dog. It barked.' ]

In the example above we return the string brown because it is the first capture group, the (.*) bit. The first item quick brown fox is the full match, which we do not return.

If our API returns HTML, there might be a lang="en" attribute in the <html> element. We can capture the two character country/language code value of that attribute with the following expression.

The expression lang="(.{2})" means ‘grab any of the first two characters between lang=" and the next "’. If we were sure there are only non-capital characters, we could tighten it up a bit with lang="([a-z]*)".

We can use regular expressions with headers too. In this example, we check if the max-age property of a Strict-Transport-Security response header is above a 100000

Variables and secrets API Response time limits

Last updated on December 13, 2024. You can contribute to this documentation by editing this page on Github