Checks & Assertions

In this guide you will learn how to use checks and assertions within your test cases to make your tests easier to debug and reason about.

As always, if you have any questions, feel free to get in contact any time!

Motivation

Test cases can get quite extensive and complex. When building up your cases or executing them, its good to see if certain assumptions are (still) correct.

Checks and assertions operate in the context of one client session. One example would be that within the user journey the login request has to be successful before you can continue with further steps. This can be done with checks and assertions.

You will see a summary of used checks and assertions in test run reports.

Checks

A check is a concept to see if a certain condition is true or not. Checks work like [session.if()][sessions-if]. They have

• a name
• a variable to check (e.g. session.getVar("csrfToken"))
• a comparator (=, !=, >, >=, < or <=) and
• an value to compare against.

If the check fails - read: the condition is not met - we will count a “KO” under its name and a “OK” otherwise. OKs and KOs will be summed up across all sessions.

Here is an example test case:

session.post("/login", {
  payload: {
    user: "road-runner",
    pass: "meep meep",
  },
  extraction: {
    jsonpath: {
      "accessToken": "authorization.token",
    }
  },
  tag: "login",
});

session.check("login", session.getVar("accessToken"), "!=", "");

The above example could generate the following “Checks” part in a test run report:

Assertions

Assertions behave very much like checks with one major difference: **After counting a KO assertions will terminate the current session. ** If the condition within an assertion passes, an OK is counted. Think of session.assert() as session.check() combined with session.abort().

We can now use session.assert() to improve the example from above:

session.post("/login", {
  payload: {
    user: "road-runner",
    pass: "meep meep",
  },
  extraction: {
    jsonpath: {
      "accessToken": "authorization.token",
    }
  },
  tag: "login",
});

session.assert("login", session.getVar("accessToken"), "!=", "");

// will only execute if "accessToken" is present!
session.get("/personal-profile");
// ...

If the scenario (like ours here) cannot be continued without a successful login, session.assert() can be used to make sure that the scenario is aborted for this user. This helps to minimize noise caused by lots of consequential errors arising because of a failed authentication step.

In addition to counting a KO, you will also get a summary of reasons (based on the assertions name) why a session was aborted in the test report:

Shorthands for HTTP Errors

If you are interested to see if the request was successful in terms of HTTP semantics, you can achieve something similar using the abort_on_error request option which is a bit more concise in this case:

session.get("/", {
  abort_on_error: true,
});

session.post("/login", {
  payload: {
    user: "road-runner",
    pass: "meep meep",
  },
  tag: "login",
  abort_on_error: true,
});

Using abort_on_error we will terminate the current session if the response code of the request is HTTP 4xx or 5xx. The abort reason (name) will be named after the requests tag - login in our case. If no tag is provided, http_error will be used.

There is also a way to switch this on automatically for all requests. This will basically add a abort_on_error: true for all subsequent requests:

session.defaults.setAbortOnError(true);

session.post("/login", {
  payload: {
    user: "road-runner",
    pass: "meep meep",
  },
  extraction: {
    jsonpath: {
      "accessToken": "authorization.token",
    }
  },
  tag: "login",
});

See also our reference on Session Checks