Distributed testing

Contents

Overview

Most automated tests execute on a single machine and only ever interact with a single web browser window, mobile app or back-end service. That’s perfectly fine, since we should always aim to keep our tests simple and with as few dependencies as possible. There are also times when your test scenario will need to execute individual steps on two or more machines, either sequentially or in parallel. This scenario calls for building a distributed test. OpenTest has very good support for building distributed tests, this being one of the early design goals of the tool.

The typical challenges when doing distributed testing are:

  • Controlling which test steps execute on each particular machine.

  • Synchronizing the execution of test steps to ensure they run in the correct sequence between the multiple machines.

  • Sharing dynamically generated data across machines.

OpenTest answers these challenges in a way that allows test engineers to focus on the logic and flow of the test, rather than the complexities of distributed testing.

Test actors setup

A distributed test involves at least two test actors. The actors typically run on different machines but you may also run all actors on a single machine. This is convenient because it means the same test can be run unchanged in both a dev environment where everything runs on the same machine, as well as in a staging, performance testing or production environment where individual software components are deployed on separate machines.

The diagram below shows the more complex scenario, where the OpenTest server and the two test actors run on three separate machines:

This is how the configuration files for the two test actors should look like:

MACHINE 2, C:\opentest\actor1\actor.yaml
actorType: ACTOR1
syncServerUrl: http://10.0.0.1:3000

# continue with other parameters here
MACHINE 3, C:\opentest\actor2\actor.yaml
actorType: ACTOR2
syncServerUrl: http://10.0.0.1:3000

# continue with other parameters here

The two most important configuration parameters we need to set in the actor.yaml configuration file for each test actor are:

  • actorType, which indicates the role the actor will play in the test.

  • syncServerUrl, which indicates the endpoint that the OpenTest server is listening at, so the test actor knows how to communicate with the server.

Distributed test syntax

Let’s consider a very simple distributed test involving the two test actors described in the previous section:

description: A simple distributed test
actors:
  - actor: ACTOR1
    segments:
      - segment: 1
        actions:
          - script: $log("This runs first, since it's included in segment 1")

  - actor: ACTOR2
    segments:
      - segment: 2
        actions:
          - script: $log("This runs second, as it's included in segment 2")

Here are a few important observations about this distributed test:

  • For the test to start executing, it requires one test actor of type ACTOR1 and one test actor of type ACTOR2.

  • The test contains two segments: segment 1 and segment 2.Test segments execute sequentially, so the test actions in segment 2 will only execute after segment 1 was completed.

  • ACTOR1 is only involved in segment 1 and ACTOR2 is only involved in segment 2. Any actor can execute test steps in any test segment and multiple actors can execute test steps in the same segment, if necessary.

Sharing data between test actors

More often than not, the test actors involved in a distributed test need to "talk" to each other in order to share various pieces of data (a dynamically generated identifier, an account activation code, etc.) The proper way to share this data is by using the shared data store.

Below you can find an example of a simple distributed test that uses the shared data feature. The first test actor creates a random number in a script block and writes its value in shared data under the name someRandomNumber. The second test actor reads this value, and outputs it to the test session log.

description: A simple distributed test
actors:
  - actor: ACTOR1
    segments:
      - segment: 1
        actions:
          - script: |
              var randomNumber = 1 + Math.floor(Math.random() * 100);
              $sharedData.someRandomNumber =  randomNumber;

  - actor: ACTOR2
    segments:
      - segment: 2
        actions:
          - script: |
              $log("The random number is " + $sharedData.someRandomNumber)

You can find more information on the shared data store here.