Checkpoints

A test is normally considered failed and the test execution is stopped after the very first failed test step. Most of the times, this is a good thing. However, when a test step doesn’t affect the steps that follow, it is sometimes more practical to continue the test execution, thus performing more verifications in one go, minimizing execution time and conserving computing resources. This can also help with troubleshooting failed test steps, since multiple failures will be shown nicely in one single test execution report. Of course, when a test step fails, the test will still be considered failed, even though we allow the test execution to continue.

The functionality described above can be achieved by using the "checkpoints" feature. Any test action can be transformed into a "checkpoint" by passing the global argument $checkpoint with a value of true:

# This verifies the element's visibility, but doesn't
# fail the test right away if the step fails

- description: Verify query box visibility
  action: org.getopentest.selenium.AssertElementVisible
  args:
    locator: { name: q }
    $checkpoint: true

For example, let’s consider a test that navigates to the GitHub homepage and verifies that:

  • The search textbox is visible.

  • The search textbox is empty.

  • The "Explore" menu is visible

We will be marking the three verification steps as checkpoints, since we don’t want a failure in those steps to stop the test execution. Note the $checkpoint argument we added to these test actions:

description: Search for the "react" repo on the GitHub website
actors:
  - actor: WEB
    segments:
      - segment: 1
        actions:
          - description: Navigate to the GitHub homepage
            action: org.getopentest.selenium.NavigateTo
            args:
              url: https://github.com

          - description: Verify that the search textbox is visible
            action: org.getopentest.selenium.AssertElementVisible
            args:
              locator: { name: q }
              $checkpoint: true

          - description: Verify that the search textbox is empty
            action: org.getopentest.selenium.AssertElementText
            args:
              locator: { name: q }
              text: ""
              $checkpoint: true

          - description: Verify that the "Explore" menu is visible
            action: org.getopentest.selenium.AssertElementVisible
            args:
              locator: { css: "div.HeaderMenu a[href='/explore']" }
              $checkpoint: true

If we now change the three verification steps to make them fail by pointing their locator arguments to some nonexistent UI elements, the test will still continue until the end, but it will still be marked as failed and the execution report will look like this:

Test execution report