Test session templates

Contents

Session templates overview

A test session identifies a set of tests to run, along with other details like the environment to run against, whether or not to repeat failed tests, etc. Manually creating test sessions can be done using the Web UI, which is good enough for ad-hoc test runs. However, this is also a repetitive, time consuming and error-prone process, especially when including a large number of tests. Furthermore, the manual input of these details is not possible when a test session is kicked off automatically from a CI system. To address these problems, OpenTest introduces the concept of a test session template.

The location and structure of template files

Test session templates are stored as YAML files in the templates subdirectory of the test repository. For example, to create a session template named My first session template, that contains two tests Test 1 and Test 2, the test repo will have to look as follows:

test-repo
    ├── templates
    │   └── My first session template.yaml
    └── tests
        ├── Test 1.yaml
        └── Test 2.yaml

The template file structure is as shown below. All properties are optional, so you can mix and match as needed.

test-repo/templates/My first session template.yaml
description: A test session template that runs two dummy tests
sessionLabel: Two tests
actorTags: []
environment: null
maxIterations: 1
includeTestsWithTags: (tag1 || tag2) && tag3
includeTestsFromTemplates:
    - name: template1
      path: dir1

    - name: template2
      path: dir2
tests:
    - name: Test 1
      path: .

    - name: Test 2
      path: .
Table 1. Properties of test session templates

description

Human-readable description of the template.

sessionLabel

The session label that will be assigned to test sessions created from this template.

actorTags

An array of strings representing the list of tags that will be required for the test actors executing the test session. Default: null.

environment

The environment the new session will be executed against. Default: null.

maxIterations

The maximum number of times to repeat execution of failed tests. Default: 1.

includeTestsWithTags

An expression that identifies the tests to include in the session based on the tags that were assigned to them using the tags property in the test file. This is a boolean expression, where each tag is used as an operand of type boolean. The value of the operand will be true if a test contains a certain tag and false otherwise. For example, if a test contains the tags smoke and flaky, the expression smoke && !flaky will translate to true && false, which is false. In consequence, the test will not be selected to be run in this test session.

includeTestsFromTemplates

A list of template names that will be used as the source for the list of tests to include in the session. Make sure to specify the template names, along with their path, relative to the templates directory.

tests

An array that stores the list of tests to include in the test session. Each test is specified as an object with the properties name and path. The name property specifies the file name of the test without the ".yaml" extension, and the path is the test file path relative to the tests directory in the test repo.

Creating a test session based on a template

There are two ways to create a new test session based on a template file:

  1. Using the server’s web UI, from the menu, select Session > Create Session From Template. Select the template file you want and click Create session.

  2. Using the OpenTest server API.