The Advent of Monitoring, Day 4: Solving E2E Testing Challenges With Checkly's PWT Garbage Collector

This is the fourth part of our 12-day Advent of Monitoring series. In this series, Checkly's engineers will share practical monitoring tips from their own experience.

One of the biggest challenges in conducting end-to-end (E2E) testing is managing the artifacts created during the process. These artifacts are necessary for asserting specific functionalities.

For instance, we might want to verify a resource creation ability, but we do not want to:

1. Assert the deletion of the created entities afterwards, since this could compromise the isolation and single responsibility of the checks. For example, a check that creates and then deletes a resource might fail during the cleanup process. This could result in a false positive.
2. Keep the entities around. Imagine having monitors that execute this check every 10 minutes. That would mean six entities are created per hour, or 144 leftovers per day. These would keep piling up and require a specific external cron job to keep the room tidy.

In Checkly, it was our decision, as frontenders, that we wanted to keep not only the testing flow but also the maintenance flow as contained within Checkly as possible.

The conception of the method I will show you is an elaboration on top of the low-level API calls approach that we implemented a while back and Nico splendidly showcased in a past blog.

From this point onwards in the article, there will be some heavy TS nerdiness involved, so behold!

** cracks knuckles * —* let’s go!

1. Let’s declare an API module factory

This is a factory method that we will use for returning a wrapped API module that will include two extra methods: addToCleanupQueue and cleanup.

The first one is a method that we will invoke during checks to add IDs to an internal pile of pending-to-remove artifact IDs. The second method will be responsible for invoking the cleanupMethod declared module call sequentially with the IDs, present in the cleanup pile.

export type ChecklyApiModule <Methods extends Record<string, (...args: any) => any>> = Methods & {
  addToCleanupQueue: (id: string | number) => void
  cleanup: () => Promise<void>
}

function defineChecklyApiModule <Methods extends Record<string, (...args: any) => any>> ({
  methods,
  cleanupMethod,
}: {
  methods: Methods
  cleanupMethod: keyof Methods
}): ChecklyApiModule<Methods> {
  const cleanupIds: Set<string | number> = new Set()

  const module = {
    ...methods,
    addToCleanupQueue: (id: string | number) => {
      cleanupIds.add(id)
    },
    cleanup: async () => {
      const ids = [...cleanupIds]

      if (!ids.length) return

      console.log('Cleaning up IDs:', ids.join(', '))

      for (const id of [...cleanupIds]) {
        await methods[cleanupMethod](id)
      }
    },
  }

  return module
}

2. Let’s declare the API fixture

For showcasing purposes, I will only list a single module here. Note that the fixture would be configurable to pass any http client you might prefer (axios, fetch, or what have you):

export type HttpClient = {
  [key in 'get' | 'delete' | 'patch' | 'post' | 'put']: (
    url: string,
    config?: {
      // PWT request doesn’t tell apart methods with or without data.
      data?: any,
      params?: Record<string, any>
    }
  ) => Promise<any>
}

function createChecklyApiService ({ httpClient }: { httpClient: HttpClient }) {
  const moduleMap = {
    checks: defineChecklyApiModule({
      methods: {
        fetchAll: () => {
          return httpClient.get('/v1/checks', { params: { limit: 100 } })
        },
        create: (data) => {
          return httpClient.post('/v1/checks', { data })
        },
        remove: (checkId: string) => {
          return httpClient.delete(`/v1/checks/${checkId}`)
        },
      },
      cleanupMethod: 'remove',
    }),
  }

  const modules = Object.values(moduleMap)

  async function cleanup () {
    for (const module of modules) {
      try {
        await module.cleanup()
      } catch (e) {
        console.error(e)
      }
    }
  }

  return {
    ...moduleMap,
    cleanup,
  }
}

As you might have noticed, there are two interesting parts here:

1. The declaration of API modules gets full TS-type inference, so you cannot go wrong there

2. There is an API fixture global method for iterating the declared modules and invoking cleanup on every one of them.

3. Let’s use the API fixture

Reading through the PWT docs, you would notice that creating a fixture involves invoking use to tell Playwright to inject it into our tests and then having the ability to do some additional calls for whatever housekeeping would be needed. This is the cornerstone of our implementation since we leverage that to invoke our API fixture cleanup:

export const test = base.extend<ChecklyWebappFixtures>({
	// other fixturs
	api: async ({ request, auth }, use) => {
		/**
     * here you would build your http client by using axios,
     * wrapping request, fetch, or even mocking it if you needed so.
     */
    const api = createChecklyApiService({ httpClient })

    await use(api)
    await api.cleanup()
  },

4. Let’s use it in our checks

import { test } from './wrappedChecklyTest'

test('should create a check', async ({ page, api }) => {
  // perform our check logic by navigating through the UI

  const createdId = /** retrieve the created ID via page.waitForResponse */
	
	// this is where the magic happens
  api.checks.addToCleanupQueue(createdId)
})

As you can see, all the bloat that would be required to perform garbage collection at the end of each check is reduced to a clean and tidy oneliner to just add the created entity ID to the API module cleanup pile.

The fixture will take responsibility for invoking the clean-up after the check execution is finished.