Open Banking API Monitoring with Playwright and Checkly

Table of contents

Testing the whole flow end-to-end
Monitoring the flow with Checkly
Wrapping up

Open banking offers users a way to have easier access to their own bank account information, like via third-party applications. This is achieved by allowing third-party financial service providers access to the financial data of a bank's customers through the use of APIs, which enable secure communication between the different parties involved.

Some notable examples of banks and financial institutions that are leveraging open banking to offer enhanced services, increased transparency, and a more personalised banking experience for their customers:

Barclays has launched an open banking API platform, providing access to a range of APIs for account information, payments, and transactions.
Klarna has launched an innovative open banking platform called Kosma, revolutionising access to more than 15,000 banks in 27 countries
Wells Fargo has an API portal that provides a suite of tools and sample code for developers, along with a testing environment.
Monzo utilises open banking to make easy bank transfers, see all your accounts at once and even prove your income to get an overdraft.

Additional examples include Truelayer, Trading 212, Plaid, Revolut, and Mint. Each of these organizations utilizes open banking in various ways, from account aggregation to payment processing and personal finance management. Take a look at the Open Banking Tracker for more information.

Monitoring Open Banking APIs

As far as API journeys are concerned, the intricate, sequential interactions of open banking can be quite demanding when it comes to monitoring. The layers of authentication, authorization, encryption and data transfer mean that a single transaction will involve multiple steps and several API endpoints working together, each bound by the step before. This means that monitoring an isolated endpoint is not enough, and that we’ll rather have to look at the flow as a whole, while still surfacing the right information in case of failure to let us quickly understand what has broken in this complex exchange.

To better understand the challenges that monitoring open banking flows poses, let’s look at an example flow from the Klarna XS2A App.

The Open Banking XS2A App handles all consumer interactions. It is used whenever the API flow requires an input from the consumer, such as selecting the consumer bank or the strong customer authentication towards the bank. In addition, depending on the selected flow, the XS2A App may be used to select a specific bank account or to authorize an account-to-account payment.

Now, let’s break down the flow into its constituent steps:

Start Session: The process kicks off with a PUT request to Klarna's session initiation endpoint, where we set up the necessary parameters for an open banking session. Language preferences, bank details, user information, and the scope of consent for accessing various account services are all defined here so the session scope is defined from the start and cannot be hijacked for any other purpose.
Start Account Details Flow: Upon obtaining a session ID, we proceed with another PUT request, this time to initiate an account details flow. By further specifying account identifiers and cryptographic keys, we are ensuring that all communication is secure.
Select Test Bank: A POST request follows, aimed at selecting a bank for the user. This step simulates the customer's bank choice within the Klarna playground.
a. [Optional] Get Flow Configuration: Next, we perform a GET request to retrieve the current state of the flow. This step ensures that the transaction sequence is progressing as expected.
Encrypt Responses: This phase involves sending responses encrypted using both AES and RSA encryption algorithms. The encrypted payload is sent back to the API endpoint using a POST request, including the RSA-encrypted AES key and the AES-encrypted data.
This multi-layer encryption approach ensures that the encrypted data can only be decrypted by the API endpoint with the corresponding RSA private key, and the AES key remains protected throughout the transmission.
Complete Flow: This step ties up the transaction flow by posting the acquired redirect details back to Klarna's API, confirming the API actions and moving the process forward.
User Bank Account Selection: The account value fetched in step five is sent over. We have to encrypt the account numbers for security.
Get Consent: A POST request is made to secure consent for data access, a regulatory requirement for open banking services. The response includes URLs for account details and balances, which will be used in subsequent requests.
Get Account Details and Balance: The final POST request involves fetching the user's account details and balance, signifying the completion of the transaction flow. These requests validate the consent token and return the specific account information.

Testing a single API endpoint with Playwright

The flow we just analyzed is composed of 8 steps and 11 requests.

If we were to write a Playwright script to test the first request in isolation, it might look something like this:

const { test } = require('@playwright/test');

const auth_token = process.env.KOSMA_AUTH_TOKEN;
const psu_ua =
	'Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/80.0.3987.122 Safari/537.36';
const psu_ip = '10.20.30.40';

test.describe('Klarna Open Banking', () => {
	test('XS2A API Flow', async ({ request }) => {
		const startSession = await request.put(`https://authapi.openbanking.playground.klarna.com/xs2a/v1/sessions`, {
			data: {
				language: 'en',
				_selected_bank: {
					bank_code: '000000',
					country_code: 'GB',
				},
				psu: {
					ip_address: psu_ip,
					user_agent: psu_ua,
				},
				consent_scope: {
					accounts: {},
					account_details: {},
					balances: {},
					transactions: {},
					transfer: {},
					_insights_refresh: {},
					lifetime: 30,
				},
				_aspsp_access: 'prefer_psd2',
				_redirect_return_url: 'http://test/auth',
				keys: {
					hsm: '--- xxx ---',
					aspsp_data: '--- xxx ---',
				},
			},
			headers: {
				'Content-Type': 'application/json',
				Authorization: 'Token ' + auth_token,
			},
		});
		const startSessionResponse = await startSession.json();
		const session_id = startSessionResponse.data.session_id;
	});
});

Running this script on a schedule might give us precious information already, but given how interconnected these endpoints will be in the context of the open banking flow we are testing, we’ll want to chain each request to test the flow end-to-end.

Testing the whole flow end-to-end

Playwright can help us big time here, by allowing us to wrap each request and subsequent response manipulation in its own test.step() to keep our code cleaner and our reporting more understandable.

Here is what the whole flow would look like end to end (we’ve numbered the steps in the code to match our flow breakdown from above):

const { test, expect } = require('@playwright/test');
const { sendEncryptedResponse } = require('./snippets/functions.js')

const auth_token = process.env.KOSMA_AUTH_TOKEN
const psu_ua = "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/80.0.3987.122 Safari/537.36";
const psu_ip = "10.20.30.40"

test.describe('Klarna Open Banking', () => {

    test('XS2A API Flow', async ({ request }) => {

			/* --------------------------------------- 1 ----------------------------------------------- */

      const startSession = await test.step('Start Session', async () => {
        return request.put(`https://authapi.openbanking.playground.klarna.com/xs2a/v1/sessions`, {
        data: {
            language: "en",
            _selected_bank: {
              bank_code: "000000",
              country_code: "GB",
            },
            psu: {
              ip_address: psu_ip,
              user_agent: psu_ua,
            },
            consent_scope: {
              accounts: {},
              account_details: {},
              balances: {},
              transactions: {},
              transfer: {},
              _insights_refresh: {},
              lifetime: 30,
            },
            _aspsp_access: "prefer_psd2",
            _redirect_return_url: "http://test/auth",
            keys: {
              hsm: "--- xxx ---",
              aspsp_data: "--- xxx ---"
            }
        },
        headers: {
          "Content-Type": "application/json",
          "Authorization": "Token " + auth_token,
        }
      })
      });
      const startSessionResponse = await startSession.json();
      const session_id = startSessionResponse.data.session_id;
    
      /* --------------------------------------- 2 ----------------------------------------------- */

      const accountDetailsFlow = await test.step('Start Account Details Flow', async () => {
        return request.put(`https://authapi.openbanking.playground.klarna.com/xs2a/v1/sessions/` + session_id + `/flows/account-details`, {
        data: {
          "account_id": "",
          "iban": "",
          "keys": {
            "hsm": "",
            "aspsp_data": ""
          }
        },
        headers: {
          "Content-Type": "application/json",
          "Authorization": "Token " + auth_token,
        }
      })

        expect(client_token).toBeEmpty();
      });
      const accountDetailsFlowResponse = await accountDetailsFlow.json();
      const flow_id = accountDetailsFlowResponse.data.flow_id;
      var client_token = accountDetailsFlowResponse.data.client_token;

      /* --------------------------------------- 3 ----------------------------------------------- */

      const selectTestBank = await test.step('Select Test Bank (Germany)', async () => {
        return request.post(`https://authapi.openbanking.playground.klarna.com/branded/xs2a/v1/wizard/` + flow_id, {
        data: {
            "bank_code": "00000",
            "country_code": "DE",
            "keys": {
              "hsm": "",
              "aspsp_data": ""
            }
          },
        headers: {
          "Content-Type": "application/json",
          "Authorization": "Token " + auth_token,
        }
      })
      });
      const selectTestBankResponse = await selectTestBank.json();
      
      const getFlowConfig = await test.step('Get Flow Configuration', async () => {
        return request.get(`https://authapi.openbanking.playground.klarna.com/branded/xs2a/v1/wizard/` + flow_id, {
        headers: {
          "Content-Type": "application/json",
          "Authorization": "Token " + auth_token,
        }
      })
      });

      const getFlowConfigResponseJSON = await getFlowConfig.json();
      const getFlowConfigResponse = getFlowConfigResponseJSON.data;

      /* --------------------------------------- 4 ----------------------------------------------- */

      // Encrypt Responses Here

      const selectTransportForm = JSON.stringify(
      {
        "form_identifier": getFlowConfigResponse.result.form.form_identifier,
        "data": [
          { "key": "interface", "value": "de_testbank_bias" }
        ]
      });
    
      const selectTransportFormResponse = await sendEncryptedResponse(getFlowConfigResponse, selectTransportForm, auth_token);

      const userAndPasswordForm = JSON.stringify(
      {
        "form_identifier": selectTransportFormResponse.result.form.form_identifier,
        "data": [
          { "key": "bias.apis.forms.elements.UsernameElement", "value": "redirect" },
          { "key": "bias.apis.forms.elements.PasswordElement", "value": "123456" }
        ]
      });
    
      const userAndPasswordFormResponse = await sendEncryptedResponse(selectTransportFormResponse, userAndPasswordForm, auth_token);
      if (userAndPasswordFormResponse.result.context === "authentication"){
        var redirect_url = userAndPasswordFormResponse.result.redirect.url + "&result=success"
        var redirect_id = userAndPasswordFormResponse.result.redirect.id
      }

      /* --------------------------------------- 5 ----------------------------------------------- */

      const completeFlow = await test.step('Complete Flow', async () => {
        return request.post(`https://authapi.openbanking.playground.klarna.com/branded/xs2a/v1/wizard/` + flow_id, {
        data: {
            "redirect_id": redirect_id,
            "return_url": redirect_url,
        
            "keys": {
              "hsm": "",
              "aspsp_data": ""
            }
          },
        headers: {
          "Content-Type": "application/json",
          "Authorization": "Token " + auth_token,
        }
      })
      });
      const completeFlowResponseJSON = await completeFlow.json();
      const completeFlowResponse = completeFlowResponseJSON.data;

      if (completeFlowResponse.result.form.elements.options === null) {
        console.error("Log: " + "No accounts found for this user?");
      }

      const accounts = completeFlowResponse.result.form.elements[0].options;
      const first_account = accounts[0];

      /* --------------------------------------- 6 ----------------------------------------------- */

      const selectFirstAccountForm = JSON.stringify({
        "form_identifier": completeFlowResponse.result.form.form_identifier,
        "data": [
          { "key": "account_id", "value": first_account.value }
        ]
      });

      const accountSelectionForm = await sendEncryptedResponse(completeFlowResponse, selectFirstAccountForm);

      if (accountSelectionForm.state === "FINISHED"){
        console.log("Log: " + first_account.label + " selected." );
      }

      /* --------------------------------------- 7 ----------------------------------------------- */

      const getConsent = await test.step('Get Consent', async () => {
        return request.post(`https://api.openbanking.playground.klarna.com/xs2a/v1/sessions/${session_id}/consent/get`, {
        data: {
          "keys": {
            "hsm": "--- xxx ---",
            "aspsp_data": "--- xxx ---"
          }
          },
        headers: {
          "Content-Type": "application/json",
          "Authorization": "Token " + auth_token,
        }
      })
      });
      const getConsentResponseJSON = await getConsent.json();
      const getConsentResponse = getConsentResponseJSON.data;
      const balances_url = getConsentResponse.consents.balances;
      const account_details_url = getConsentResponse.consents.account_details;

      /* --------------------------------------- 8 ----------------------------------------------- */

      const getAccountDetails = await test.step('Get Account Details', async () => {
        return request.post(account_details_url, {
        data: {
          "consent_token": getConsentResponse.consent_token,
          "account_id": first_account.value,
      
          "psu": {
            "ip_address": psu_ip,
            "user_agent": psu_ua
          },
      
          "keys": {
            "hsm": "",
            "aspsp_data": ""
          }
          },
        headers: {
          "Content-Type": "application/json",
          "Authorization": "Token " + auth_token,
        }
      })
      });
      const getAccountDetailsResponseJSON = await getAccountDetails.json();
      const getAccountDetailsResponse = getAccountDetailsResponseJSON.data;

      const getAccountBalance = await test.step('Get Account Balance', async () => {
        return request.post(balances_url, {
        data: {
          "consent_token": getAccountDetailsResponse.consent_token,
          "account_id": getAccountDetailsResponse.result.account.id,
      
          "psu": {
            "ip_address": psu_ip,
            "user_agent": psu_ua
          },
      
          "keys": {
            "hsm": "",
            "aspsp_data": ""
          }
          },
        headers: {
          "Content-Type": "application/json",
          "Authorization": "Token " + auth_token,
        }
      })
      });

      const getAccountBalanceResponse = await getAccountBalance.json();
      const accountBalance = getAccountBalanceResponse.data.result.available.amount

      console.log("Log: Account Balance = €" + accountBalance );
      expect(accountBalance).toBeGreaterThanOrEqual(0);
    
    });
  });

Note that we’ll need to export the KOSMA_AUTH_TOKEN environment variable set to the value of Kosma’s demo auth token.

Our script comes with an additional dependency on top of just Playwright, functions.js, which looks as follows (and also includes jsbn.js.):

const { RSA } = require('./jsbn.js');
const axios = require('axios');
const crypto = require('crypto');
const CryptoJS = require("crypto-js");

function findModAndExp(xs2a_form_key) {

  // Base64 decoding function
  function b64Decode(str) {
    str = str.replace(/-/g, '+').replace(/_/g, '/');
    while (str.length % 4) {
      str += '=';
    }
    return Buffer.from(str, 'base64').toString('utf8');;
  }

  // Split JWT into its three parts
  const parts = xs2a_form_key.split('.');
  const header = JSON.parse(b64Decode(parts[0]));
  const payload = JSON.parse(b64Decode(parts[1]));
  const signature = parts[2];

  // Extract the modulus value from the JWK object
  const modulus = payload.modulus;
  const exponent = payload.exponent;

  return {
    'modulus': modulus,
    'exponent': exponent
  };
}

function generateRandomHexString(byteLength) {
  // Create a buffer with random bytes
  const buf = crypto.randomBytes(byteLength);

  // Convert buffer to hex string
  let res = '';
  for (let i = 0; i < buf.length; i++) {
    res += ('0' + (buf[i] & 0xff).toString(16)).slice(-2);
  }
  return res;
}

function encrypt(publicKey, plainText) {
  if (!publicKey) {
    throw new Error('No or wrongly formatted Public Key for Encryption given');
  }
  var { modulus, exponent } = findModAndExp(publicKey)
  const iv = generateRandomHexString(16);
  const keyHex = generateRandomHexString(256 / 8);
  const key = CryptoJS.enc.Hex.parse(keyHex);
  const encrypted = CryptoJS.AES.encrypt(plainText, key, { iv: CryptoJS.enc.Hex.parse(iv) });
  const ciphertext = encrypted.toString();

  const rsa = new RSA.key();
  rsa.setPublic(modulus, exponent);
  // Encrypt the data
  const encryptedByRsa = rsa.encrypt(keyHex);
  const encryptedKeyBytes = CryptoJS.enc.Hex.parse(encryptedByRsa);
  // Convert the encrypted key to base64
  const encryptedKey = encryptedKeyBytes.toString(CryptoJS.enc.Base64);
  return { ct: ciphertext, iv: iv, ek: encryptedKey };
}

async function sendEncryptedResponse(lastResponse, responseForm, auth_token) {
  const publicKey = lastResponse.result.key
  const encryptedData = encrypt(publicKey, responseForm);
  let data = JSON.stringify(encryptedData);

  try {
    const response = await axios({
      method: 'post',
      maxBodyLength: Infinity,
      url: lastResponse.next,
      headers: {
        "Content-Type": "application/json",
        Authorization:
          "Token " + auth_token,
      },
      data: data
    });
    return response.data.data
  } catch (err) {
    throw new Error(err);
  }
}

module.exports = { sendEncryptedResponse, generateRandomHexString, encrypt, findModAndExp }

Monitoring the flow with Checkly

To make sure the flow is reliably functioning in its entirety, we need to run our test at regular intervals, making it effectively a monitoring check. Our check will probably need to run from multiple locations, and surely will need to be tied to one or more alert channels. The Checkly CLI enables us to get started quickly defining all of this without leaving a code editor.

Now let’s initialize a Checkly CLI project and copy our script from above into it. To save you time, we’ve done this for you already.

Note how we are defining a Checkly multi-step API check, using the appropriate construct:

import * as path from 'path';
import { MultiStepCheck } from 'checkly/constructs';
import { emailChannel, callChannel } from './alertChannels';

new MultiStepCheck('xs2a-flow-check', {
	name: 'Klarna Open Banking - XS2A API Flow',
	alertChannels: [emailChannel, callChannel],
	muted: false,
	code: {
		entrypoint: path.join(__dirname, 'xs2a.spec.ts'),
	},
});

This is pointing to a xs2a.spec.ts which contains our Playwright spec testing the entire API flow end to end.

Whenever the check fails, Checkly will reach out to us using the linked emailChannel and callChannel:

import { EmailAlertChannel } from 'checkly/constructs';
import { PhoneCallAlertChannel } from 'checkly/constructs';

const sendDefaults = {
	sendFailure: true,
	sendRecovery: true,
	sendDegraded: false,
	sslExpiry: true,
	sslExpiryThreshold: 30,
};

export const emailChannel = new EmailAlertChannel('email-channel-1', {
	address: 'user@email.com', // Substitute with your email address
	...sendDefaults,
});

export const callChannel = new PhoneCallAlertChannel('call-channel-1', {
	phoneNumber: '+31061234567890', // Substitute with your phone number
});

These are of course examples; Checkly is able to alert on a wide variety of channels reaching from email and SMS, to Pagerduty and OpsGenie, through Slack and MSTeams.

Our check is also inheriting check defaults that are set at project level in our config.checkly.ts:

import { defineConfig } from 'checkly'
import { Frequency } from 'checkly/constructs'

const config = defineConfig({
  projectName: 'OpenBanking CLI Project',
  logicalId: 'openbanking-cli-project',
  repoUrl: 'https://github.com/checkly-solutions/checkly-open-banking',
  checks: {
    locations: ['us-east-1', 'us-east-2', 'us-west-1'],
    runParallel: true,
    frequency: Frequency.EVERY_1M,
    tags: ['open-banking'],
    runtimeId: '2023.09',
    checkMatch: '**/*.check.ts',
    browserChecks: {
      testMatch: '**/__checks__/*.spec.ts',
    },
  },
})

export default config

Note the locations the check will run from, the scheduling strategy runParallel and the frequency param. We’re ensuring our checks are running at high frequency and from multiple locations at once to thoroughly monitor what is an essential, customer-facing flow.

Code-wise everything is ready, but we still need to feed the KOSMA_AUTH_TOKEN environment variable from our Playwright spec into Checkly if we want the check to actually work. We can easily do that with:

npx checkly env add KOSMA_AUTH_TOKEN <my_token_value> -l

We are now ready to actually deploy our check to Checkly:

npx checkly deploy

Logging into our Checkly account, we can see our check is now running as scheduled:

For each call, we can now see detailed reports showing the timing, result and other request details for each request in our check:

Our linked alert channels have been deployed too:

Checkly will now email us and call us on our phone when our open banking flow is broken, allowing us to jump into a detailed report showing which step failed in this complex flow. That’s handy.

Wrapping up

Not only may poor API performance and difficulties affect end users, but they can also create concerns with industry standards organizations and regulators. As open banking APIs handle very sensitive data, constantly monitoring these flows is key to data security and user satisfaction.

In this blog post, we provided a detailed guide on using Playwright and Checkly to test and monitor the Klarna Open Banking XS2A API flow. We also showed you how to encrypt responses for security, set up monitoring checks, create alert channels for failure notifications, and deploy the check to Checkly for regular monitoring. To get you started, we've set up this GitHub repo.

By combining Checkly and Playwright, you can ensure your open banking flow is functioning correctly and be alerted promptly when issues arise.

Give Checkly a try today and experience seamless API flow testing and monitoring.