Test structure
Overview
By convention, tests are defined as individual files located in the tests subdirectory of the test repo and are expressed in YAML. YAML is a human-friendly data format which makes it very easy to create, read and modify the tests in any decent text editor and with minimal training. YAML is also source-control friendly, as opposed to XML, JSON, Excel or other formats. This is a vital requirement, especially with large teams. To learn more about the YAML syntax, read the YAML primer page.
A simple test example
The code snippet below is a Selenium test that performs three actions (test steps):
Navigates to the GitHub homepage.
Searches for the text "react" and presses ENTER.
Validates that the React repo exists in the search results.
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23 description: Check the React repo on the GitHub website
tags: [smoke-tests]
actors:
- actor: WEB
segments:
- segment: 1
actions:
- description: Navigate to GitHub
action: org.getopentest.selenium.NavigateTo
args:
url: https://github.com
- description: Type "react" in the search box
action: org.getopentest.selenium.SendKeys
args:
locator: { name: q }
text: react
sendEnter: true
- description: Verify that the React repo is in the results
action: org.getopentest.selenium.AssertElementVisible
args:
locator: { xpath: "//a[@href='/facebook/react']" }
The test actor section
Line 3 in the sample test (- actor: WEB) defines the start of a new test actor section and specifies the type of test actor that will execute the work defined in this section. It’s necessary to identify the actor type because complex projects might involve multiple types of test actors running simultaneously (e.g. a mobile actor, a web actor, an API actor, etc.). The test actor type is just a plain string whose value can be set to anything that is relevant for the application under test or the specific test automation project. Typically, it is set to an uppercase three-letter abbreviation like "WEB", "MOB" or "API". The only requirement is that you need to use the exact same string in the test file as was specified for the actorType parameter in the test actor configuration file.
The test segment section
Line 5 in the sample test (- segment: 1) defines the start of a new test segment. Test segments are executed sequentially, meaning that segment number 2 will only start executing after segment 1 has finished. This feature is only needed for distributed testing, to synchronize the work between multiple test actors running on different machines. However, typical test files only contain one single test segment, same as in our sample test.
The test actions section
The lines 7 through 22 from the sample test define the work that the test actor needs to do (the important stuff). This section consists of three blocks of text, each one of them representing one single test action that we want performed.
Let’s take a closer look at the first test action:
- description: Navigate to GitHub
action: org.getopentest.selenium.NavigateTo
args:
url: https://github.comThe elements that make up this test action block are:
The
-(dash) character. The dash is used to indicate that we are adding a new test action into the test flow. In the YAML format, the dash is used to add a new element to an array (theactionsarray, in our case).The
descriptionproperty. This is an optional human-readable description of the test action that explains what the action is trying to accomplish. The action description is used in logs and test execution reports, so you should try to make it concise and meaningful.The
actionproperty. The full name of the action (keyword) that we want to run. In this case, we are running theNavigateTokeyword from theorg.getopentest.seleniumpackage, which opens a browser window and navigates to the specified URL.The
argsproperty. This is the list of arguments passed to the test action, whose value is a collection of name-value pairs. TheNavigateToaction, in particular, takes one single argument namedurl, which points to the URL to navigate to.
A slightly more complex test
Here’s an example of a test that’s a little more involved, since it requires validating the response returned by an API call. Whenever you need to execute more advanced logic, like loops and some more complex verifications, you can always achieve this by embedding JavaScript code in the flow of the test:
description: Get repo branches from the GitHub API
tags: [smoke-tests]
actors:
- actor: API
segments:
- segment: 1
actions:
- description: Perform the API call
action: org.getopentest.actions.HttpRequest
args:
url: https://api.github.com/repos/facebook/react/branches
verb: GET
- description: Verify that the branches array exists and is not empty
script: |
var branches = $output.body;
if (!branches || (branches.length == 0)) {
$fail("Response body invalid or contains no branch elements.");
} else {
$log($format(
"The GitHub API returned {0} branches",
branches.length));
}Final observations
The structure of the test definition file is controlled by indentation, as opposed to JSON, where the same goal is achieved using curly braces. If two properties start at the same column number, it means they belong to the same object. For example, it’s important that the
description,actionandargsproperties are perfectly aligned horizontally, otherwise you’ll end up with an invalid YAML file.Indentation is always achieved using space characters and never TABs. With any decent text editor, the TAB key will automatically produce the right number of space characters, so this is not something that you have to keep track of yourself. We recommend setting the indent size at 2 spaces for the best compromise between readability and ease of use.