Tim Nolet

Update: this has been released as per 13:00 CET on 11 November. We delayed it a week due to some rescheduling of other items on our roadmap.

This post is to give you a heads up of a change we will release on Tuesday November 3rd that could impact a tiny percentage of your Checkly API monitoring. 

Take 5 minutes and read this through. There is a (very) large chance this will not impact you, but we are being extra careful here.

API check assertions on JSON bodies will apply to *each element* in a response array instead of to the array as a whole. This enables using JSON path expressions to check all array values.

If you are not asserting JSON bodies, please skip all of this and enjoy your day.

If you are confused, let's look at some examples...

Just to be 100% clear: this describes the new / upcoming situation. Let's say your API returns a JSON array as a response, i.e.

You want to make sure that each customer name has the word "Doe" in it. You can now check this with the $[*] JSON path operator and a simple Contains comparison

This will check if each name attribute in the array will contain the word "Doe". In this case the assertion will pass ✅

Let's extend the example with the active boolean attribute.

Say we want to assert that every customer in the array is active with the following assertion:

Again, this will check if each active attribute in the array is true and will therefore fail ❌ because Kate Doe is not active.

We've identified one specific corner case that will break under this new regime. Look at the following payload. We've added the tags array to each customer.

We want to check if the word "tag" is part of each tags array. We use the following assertion:

In the current / old situation, this assertion will pass ✅ as JSON path will yield the string [one_tag, another_one, i_love_tags] and clearly the word tag is in there. Twice even!

In the upcoming / new situation, this will fail ❌ as we check each item in this array separately and clearly the word another_one does not contain the word tag.

If you have such an assertion, I would advise to just remove it now and set a short reminder for yourself to add a similar one after 2 November.

JSON responses with arrays can already be asserted in many ways using JSON path queries. All of the examples below will not break. 

The reason is that the JSON path queries below all yield exactly one item from the array. The moment our comparison operator comes into play, we are not "looking at" an array anymore.

In the first example below we check if the first item in our result array has a property title:

In the next example we pick the last item in the array and check if the customerId property has the value 123abc:

In this example we pick the item with index value 4. This is the 5th item as array indexes start at 0. We then assert that the responseTime property is less than 2000.

In the last example we check if the returned array has more than 10 items.

Check if you are using API checks that assert JSON response arrays "as a whole".

If so, just remove them for now and set a short reminder to add a similar assertions after 2 November.

In the worst case, you will be alerted of a failing check on 2 November and the result will clearly state what is wrong. 

For any follow up questions, don't hesitate to reach out to support using the chat bubble!

Heads up: we are making a change to assertions for API checks

Detect

Communicate

Resolve

Monitoring as Code

Developers

Resources

Community

Heads up: we are making a change to assertions for API checks