Checkpoints

A test is normally considered failed and the test execution is stopped after the very first failed test step. Most of the times, this is exactly what we want. However, if a test step is not critical in the flow of the test, it is sometimes more practical to continue the test execution. This means we can do more verifications in one go, to minimize execution time and conserve computing resources. This also helps with troubleshooting the errors since multiple failures will be shown nicely in the same test execution report. Of course, when a test step fails, the test execution result will still be considered as failed, even when we allow the test to continue.

The functionality described above can be achieved by using the "checkpoints" feature. Any test action can be transformed into a "checkpoint" by passing the global argument $checkpoint with a value of true:

- description: Verify visibility, but don't fail the test right away
  action: org.getopentest.selenium.AssertElementVisible
  args:
    locator: { name: q }
    $checkpoint: true

For example, let’s consider a test that navigates to the GitHub homepage and verifies that:

  • The search textbox is visible.

  • The search textbox is empty.

  • The "Explore" menu is visible

We will be marking the three verification steps as checkpoints, since we don’t want a failure in those steps to stop the test execution. Note the $checkpoint argument we added to these test actions:

description: Search for the "react" repo on the GitHub website
actors:
  - actor: WEB
    segments:
      - segment: 1
        actions:
          - description: Navigate to the GitHub homepage
            action: org.getopentest.selenium.NavigateTo
            args:
              url: https://github.com

          - description: Verify that the search textbox is visible
            action: org.getopentest.selenium.AssertElementVisible
            args:
              locator: { name: q }
              $checkpoint: true

          - description: Verify that the search textbox is empty
            action: org.getopentest.selenium.AssertElementText
            args:
              locator: { name: q }
              text: ""
              $checkpoint: true

          - description: Verify that the "Explore" menu is visible
            action: org.getopentest.selenium.AssertElementVisible
            args:
              locator: { css: "div.HeaderMenu a[href='/explore']" }
              $checkpoint: true

If we now change the three verification steps to make them fail by pointing their locator arguments to some nonexistent UI elements, the test will still continue until the end, but the execution report will look like this:

Test execution report