2017 October Release

How to Run Multiple Tests From Command LinePermanent link for this heading

It is possible to run multiple test from a predefined list on a defined pool of Fabasoft app.test Agents using the Fabasoft app.test Starter. The Fabasoft app.test Starter takes care of scheduling jobs onto the agents based on the OS on which the test should be executed and in which browser the test should run. To be able to schedule the tests according to those properties Fabasoft app.test Starter needs to know the OS and the installed browser of each agent that should run tests. The scheduling supports multiple backends to test against with a specified number of scopes.

The Fabasoft app.test Starter jar file needs to be in the same directory as the jar of Fabasoft app.test Console Player as it uses the Console Player classes to execute the tests.

The Fabasoft app.test Starter uses a template configuration where specific parts are overwritten at runtime (e.g. Agent hostname, used scope, …).

How to Start the Fabasoft app.test StarterPermanent link for this heading

The Fabasoft app.test Starter is started with the following command line:

java -jar <path to apptest-starter.jar> -config:<path to template config> -agentlist:<path to agent list> -testlist:<path to test list> -scopes:<first scope>-<last scope>

The parameters are described in the following table:

Parameter

Definition

config

Here you enter the path to the template configuration file which specifies the settings that are used for each test.

agentlist

Here you enter the path to the list of agents that should be used to execute the tests defined in the test list. An example agent list can be found below.

testlist

Here you enter the path to the list of tests that should be executed. An example test list can be found below.

scopes

Here you define the scopes to use for the tests. Scopes a given in the min-max format (e.g. 1-10) and are the same for all backends found in the test list.

useprocess

By using the useprocess parameter you can define that each test should be run inside a separate Console Player process instead of a thread. Per default it is set to false. The usage of processes results in much higher resource consumption and is not recommended.

Example agent listPermanent link for this heading

The agent list is formatted as CSV file and defines the agents that are used to execute the tests. To be able to schedule the tests correctly Fabasoft app.test Starter needs to know the hostname, OS and the supported browsers of each agent. Each line of the list corresponds to one Fabasoft app.test Agent and is constructed as follows:

<hostname or ip>[:port];<OS>;<browser1> <browser2>

The possible OS and browser values are predefined:

Parameter

Possible Values

OS

WINDOWS, LINUX, MAC

Browser

MOZILLA_FIREFOX, MICROSOFT_INTERNET_EXPLORER, GOOGLE_CHROME, APPLE_SAFARI

The following example shows a complete agent list:

win1.example.com;WINDOWS;MOZILLA_FIREFOX MICROSOFT_INTERNET_EXPLORER GOOGLE_CHROME

win2.example.com;WINDOWS;MOZILLA_FIREFOX MICROSOFT_INTERNET_EXPLORER GOOGLE_CHROME

lin1.example.com;LINUX;MOZILLA_FIREFOX

lin2.example.com;LINUX;MOZILLA_FIREFOX

lin2.example.com:1234;LINUX;MOZILLA_FIREFOX

mac1.example.com;MAC;GOOGLE_CHROME MOZILLA_FIREFOX APPLE_SAFARI

mac2.example.com;MAC:1122;GOOGLE_CHROME MOZILLA_FIREFOX APPLE_SAFARI

If no port is specified, the port defined in the given config is used.

Example test listPermanent link for this heading

The test list is like the agent list formatted as CSV file and represents one test per line that should be executed by app.test Starter. Each test entry holds the path to the .ducx-test file, the OS on which the test should run, the browser that should be used and the backend URL to test against. Optionally it is possible to define a delay before the test should be started. While this delay is active no other tests are run or scheduled. This leads to the following line format:

<path to .ducx-test>;<OS>;<browser>;<backend URL>[;<delay in ms>]

For OS and browser the same predefined values are used as defined in 5.5.1.1 Example agent list.

The following example shows a complete test list:

D:\path\to\example.ducx-test;WINDOWS;MOZILLA_FIREFOX;https://backend1.ex.com/folio

D:\path\to\example.ducx-test;WINDOWS;GOOGLE_CHROME;https://backend2.ex.com/folio

D:\path\to\example.ducx-test;WINDOWS;MICROSOFT_INTERNET_EXPLORER;https://backend1.ex.com/folio;10000

D:\path\to\example.ducx-test;LINUX;MOZILLA_FIREFOX;https://backend2.ex.com/folio

D:\path\to\example.ducx-test;MAC;GOOGLE_CHROME;https://backend2.ex.com/folio

D:\path\to\example.ducx-test;LINUX;GOOGLE_CHROME;https://backend1.ex.com/folio

D:\path\to\example.ducx-test;MAC;APPLE_SAFARI;https://backend1.ex.com/folio;1500

D:\path\to\example.ducx-test;WINDOWS;MOZILLA_FIREFOX;https://backend1.ex.com/folio

D:\path\to\example.ducx-test;WINDOWS;GOOGLE_CHROME;https://backend2.ex.com/folio

D:\path\to\example.ducx-test;WINDOWS;MICROSOFT_INTERNET_EXPLORER;https://backend1.ex.com/folio

Example Output of Fabasoft app.test StarterPermanent link for this heading

0/10 - EXAMPLE_TEST - win1.example.com - WINDOWS/MOZILLA_FIREFOX - backend1.ex.com - 0001

0/10 - EXAMPLE_TEST - win2.example.com - WINDOWS/GOOGLE_CHROME - backend2.ex.com - 0001

0/10 - EXAMPLE_TEST - lin1.example.com - LINUX/MOZILLA_FIREFOX - backend2.ex.com - 0002

Agent: mac1.example.com is not alive

Agent: mac2.example.com is not alive

2016-09-02 12:21:17.068:INFO::Thread-1113: Logging initialized @17900ms

  NOK - EXAMPLE_TEST - lin1.example.com - LINUX/MOZILLA_FIREFOX - backend2.ex.com - 0002

Agent: mac1.example.com is not alive

Agent: mac2.example.com is not alive

  NOK - EXAMPLE_TEST - win2.example.com - WINDOWS/GOOGLE_CHROME - backend2.ex.com - 0001

2/8  - EXAMPLE_TEST - win2.example.com - WINDOWS/MICROSOFT_INTERNET_EXPLORER - backend1.ex.com - 0002

   OK - EXAMPLE_TEST - win1.example.com - WINDOWS/MOZILLA_FIREFOX - backend1.ex.com - 0001

2/7  - EXAMPLE_TEST - win1.example.com - WINDOWS/MOZILLA_FIREFOX - backend1.ex.com - 0003

Agent: mac1.example.com is not alive

   OK - EXAMPLE_TEST - win2.example.com - WINDOWS/MICROSOFT_INTERNET_EXPLORER - backend1.ex.com - 0002

Agent: mac2.example.com is not alive

2/6  - EXAMPLE_TEST - win2.example.com - WINDOWS/GOOGLE_CHROME - backend2.ex.com - 0003

   OK - EXAMPLE_TEST - win1.example.com - WINDOWS/MOZILLA_FIREFOX - backend1.ex.com - 0003

2/5  - EXAMPLE_TEST - win1.example.com - WINDOWS/MICROSOFT_INTERNET_EXPLORER - backend1.ex.com - 0004

  NOK - EXAMPLE_TEST - win2.example.com - WINDOWS/GOOGLE_CHROME - backend2.ex.com - 0003

   OK - EXAMPLE_TEST - win1.example.com - WINDOWS/MICROSOFT_INTERNET_EXPLORER - backend1.ex.com 0004

Finished: 7 - NOK: 3 - OK: 4 - Unschedulable: 3 - Duration: 00:05:44

When a test is started on an agent, Fabasoft app.test Starter prints the current status of the execution list (<failed tests>/<number of tests>) and information about the started test (<test name> - <agent hostname> - <OS>/<Browser> - <backend hostname> - <used scope>). When a test finishes the result of the test (OK or NOK) is printed alongside with the same information about the test as described before.

If a Fabasoft app.test Agent becomes unresponsive Fabasoft app.test Starter starts polling the agent in increasing periods of time until it becomes responsive again or all tests are finished. If there are tests that were scheduled but could not be started because of an agent becoming unavailable, a notification is printed at the end of the run.

After all tests are finished a summary is printed including the duration.