2017 October Release

Testing Generic HTML PagesPermanent link for this heading

In addition to supporting various versions and editions of Fabasoft Folio based solutions, this release of Fabasoft app.test includes a technology preview for testing arbitrary HTML pages. Using this technology allows the execution of tests which are not based on hardwired knowledge of UI structures. Instead, it is possible to declare which structural patterns are to be recognized by Fabasoft app.test. This is accomplished by writing a specification file and applying it while recording and running tests (see chapter “How to Use a Control Specification”).

The standard folder for specification files is located at resources\controlspecifications inside the Fabasoft app.test installation directory (by default C:\Program Files (x86)\Fabasoft\apptest). As part of this technology preview Fabasoft app.test provides examples of specification files:

Note that the syntax and semantics of specification files are experimental and will be changed in future versions of Fabasoft app.test.

How to Create a Specification FilePermanent link for this heading

A specification file is created as a project resource:

Use the context menu on the resources folder as shown above to create a new "Control Specification" resource. This will create an empty specification file. After entering a name and pressing the Return key, the specification file will be opened in the Fabasoft app.test Studio.

Overall Structure of a Specification FilePermanent link for this heading

A specification file has to contain UTF-8 encoded text. This text will be parsed by Fabasoft app.test into the following structure:

  • Pattern
    • Section
    • Section
      ...
  • Pattern
    • Section
    • Section
      ...
  • Pattern
    ...

By declaring patterns, the author of a control specification specifies the structure of a test candidate. This principle is designed to be applicable to all kinds of testing scenarios. As of this preview, we focus on user interface testing of applications which generate HTML. Therefore, the specification author refers to the HTML Document Object Model (DOM) in order to declare patterns.

A pattern declaration consists of one or more sections. Each section declares an aspect of the pattern. Depending on the semantics of this aspect, different formal languages can by used by the specification author to express those semantics:

  • Pattern
    • Section   [ a formal language
      is used here to describe
      an aspect of the pattern ]

This openness requires tight and simple rules (see chapter “hierarchy”) for the recognition of patterns and sections in order to prevent various formal languages from interfering with the overall pattern/section structure of a control specification file.

A pattern specifies something a test author will refer to. Because we focus on user interface tests for now, a pattern usually declares a unit recognized by the user of an application as a control. Those controls are structured as a tree, the control tree. The specification author declares every aspect which is relevant for the test author, such as the structural location of a control or which actions are applicable to test the control.

This is the general structure of a pattern declaration:

Syntax

# <pattern name> ( <parameters> )
  <comment>
: <section name> ( <parameters> )
  <section content>
: <section name> ( <parameters> )
  <section content>

PatternsPermanent link for this heading

A pattern defines for example a control on a web page. With a pattern you can define the elements that should be displayed in the control tree of Fabasoft app.test which means the elements, where possibly actions can be carried out within a test. A patterns definition is identified in the control specification by the symbol # in the first column of a row. Afterwards you define the name of the pattern.

Example

#Button

Note: It is necessary that the hash character is located in the first column that it is recognized as a pattern.

You can define parameters with a pattern. These parameters are defined in brackets whereby the opening bracket must be located in the same row as the pattern name. If several parameters are set, they are separated by a comma. These parameters can be set:

  • parent
  • category
  • role

Example

#Subnavigation (category=BUTTONBAR, parent=Navigation)

SectionsPermanent link for this heading

Sections are used for describing the pattern such as where it can be found as well as the text of this pattern in the control tree and the actions for this pattern. A section is identified by a colon in the first column of a row. These sections can be defined:

  • expression
  • label
  • include

Note: Please note that a section can only be identified as a section if the colon is really located in the first column of the row!

In a section you can define the parameter language (see chapter “language”) for defining the programming language used in this section. Most sections use XPath as the default language.

expressionPermanent link for this heading

The expression section defines where the pattern can be found (in the HTML of a web page or in the application to be tested).

Example

#Button
:expression descendant::INPUT[@type="button" or @type="submit" or @type="reset" or @type="image" or @type="link"]

You can define a programming language to be used in this section by using this parameter. The default language is XPath.

cssexpressionPermanent link for this heading

The cssexpression section defines a CSS selector expression where the pattern can be found (in the HTML of a web page). CSS expressions work like XPath expressions, but with different syntax and a little performance benefit.

Note: In Internet Explorer such expressions are only supported if the document mode which the web page runs with is greater or equal to 8. Internet Explorer 7 is not supported.
In Firefox CSS selector expressions are supported from version 4.0. Safari is supported from version 5.

Example

#MyControl
:cssexpression DIV.myclass

idPermanent link for this heading

The id section defines where the pattern can be found via the id of an HTML element.

Note: The expression is executed globally. An Element is always matched when it has the specified id, regardless of the parent of the pattern.

Example

#MyControl
:id myid

classPermanent link for this heading

The class section defines where the pattern can be found via a class name of an HTML element

Example

#MyControl
:class myclass

labelPermanent link for this heading

The label section defines the text of this pattern in the control tree. This section must return a text node. If no text is returned, the value of the node is taken. You can define several label sections within one pattern. In that case, the first text is used that is returned (and all label sections afterwards are ignored). You can also just define a string for this pattern.

Note: In the whole control specification, text between double quotation marks is identified as a string.

Example

#Button
:expression ...
:label attribute::name
:label text()
:label "button without a name"

Note: The XPath variable $this refers to the result of the :expression section, i.e. to the main node of the current pattern.

includePermanent link for this heading

In this section you can enter a path that should be included in the pattern (relative to the control specification).

Example

#Button
....
:include <path>

CommentsPermanent link for this heading

A comment in a control specification can only be inserted between the pattern definition and the first section of a pattern. All text before the first section is identified as a comment (either after the pattern name or - if parameters are defined - after the closing bracket).

Example

#Button
All text in here is interpreted as a comment
the comment is ended when the first section starts with a ":" in the first column of a row
:<section>

ParametersPermanent link for this heading

Parameters can be defined for a section or a pattern. Parameters are defined in brackets, the opening bracket must alsways be located in the same row as the pattern/section. Afterwards you can insert line breaks as you want, the parameters are taken till the closing bracket occurs.

Example

#Button (
   parameter1 = value1,
   parameter2 = value2
)

The value of a parameter can also be set between double quotation marks. If a closing bracket occurs in the value of your parameter, this absolutely necessary.

Example

Example

#Button (
   parameter1 = "1) value",
)

Note:If the value was not between quotation marks, the value of the parameter would be "1" and " value" would be identified as a comment.

parentPermanent link for this heading

The parent parameter can be set for a pattern and defines if this patters is subordinated to another one. This means that the pattern, which is currently defined, is only searched within the defined parent pattern. If you want to define several parent patterns, just set the parent parameter several times.

Example An option is only searched in a Select element.

Example

#Option (parent = Select)

rolePermanent link for this heading

The role parameter defines if the pattern should have a specific role. These roles are currently available:

  • error - This pattern represents an error, which can then be handled by Fabasoft app.test.
  • warning - This pattern represents a warning.
  • information - This pattern represents an information.
  • busy - If this pattern is displayed, the browser/application is currently busy. This can for example be a JavaScript variable.

Example

#Error (role = error)
...

categoryPermanent link for this heading

The category parameter defines if the pattern belongs to a specific category (e.g. those known from the control tree of Fabasoft Folio). This has the advantage that you can summarize several patterns, which are defined within the control specification, can be displayed in the control tree of Fabasoft app.test under one category.

Example

#Navigation (category=BUTTONBAR)
...

#Subnavigation (category=BUTTONBAR, parent=Navigation)
...

languagePermanent link for this heading

The language parameter is used for defining the programming language of a section. Usually the default language of a section is XPath - so if you do not set the language parameter, you have to use XPath in this section.

Example

:expression (language = JavaScript) <programming code>

hierarchyPermanent link for this heading

The hierarchy parameter is used to control how the result of an expression is used to build the control tree.

There are two possibilities:

  • DOM
    “Marks” the elements in the original DOM tree. The hierarchy of the controls in the resulting control tree corresponds to the hierarchy in the DOM.
  • PARENT
    “Creates” new controls for each matched element. The controls are flat added to the parent of the control specification pattern (or the root, if the pattern has no parent declared).
    With that option, the control-tree can be restructured, e.g. sibling elements in the DOM tree can be inserted as children of a control.
    Note: Controls created in this way cannot be touched by other expressions. Expressions use always the original DOM tree as search base.

Defaults:

  • :expression/XPath: DOM
  • :expression/JavaScript: PARENT
  • :cssexpression: DOM
  • :id: DOM
  • :class: DOM

Expression performance (most performant listed first):

  • :id with PARENT
  • :class with PARENT (IE document mode >= 8, Firefox >= 4.0)
  • :cssexpression with PARENT
  • :expression/JavaScript with PARENT
  • :id with DOM
  • :class with DOM (IE document mode >= 8, Firefox >= 4.0)
  • :cssexpression with DOM
  • :class with DOM (IE document mode < 8, Firefox < 4.0)
  • :class with PARENT (IE document mode < 8)
  • :expression/XPath with DOM
  • :expression/XPath with PARENT
  • :expression/JavaScript with DOM

Example

#Select

:expression (hierarchy=DOM)

descendant::DIV[@class="selectcontrol"]

#SelectOption (parent=Select)

:expression (hierarchy=PARENT)

following-sibling::DIV[@class="selectoption"]

Lexical and Syntactical Rules for Specification FilesPermanent link for this heading

The information contained in a specification file is structured as follows:

  • Pattern
    • Section   [ a formal language
      is used here to describe
      an aspect of the pattern ]
    • Section   [ ... ]
      ...
  • Pattern
    • Section   [ ... ]
    • Section   [ ... ]
      ...
  • Pattern
    • ...

Each section content [ ... ] is intended for interpretation by a formal language processor (like an XPath or JavaScript engine). The set of applicable formal languages is not limited by the design of the specification mechanism.

As a corollary to this openness, Fabasoft app.test is unable to parse section contents itself. It rather has to extract each content in order to hand it over to the appropriate processor. To handle this extraction safely, it is critical to exactly define the lexical rules which govern the embedding of an open set of formal languages into a single text file.

Content of a Specification FilePermanent link for this heading

A specification file contains a sequence of bytes which have to correctly encode a character sequence. The maximum number of bytes is 131072 (128 KB). The only supported encoding is UTF-8. Fabasoft app.test decodes this byte sequence into a sequence of UNICODE code points (characters). This character sequence constitutes the input for tokenizing and line counting.

Tokenizing and Line CountingPermanent link for this heading

The input character sequence is tokenized in order to

  • Find and parse section and pattern headers
  • Extract each section content

Tokenizing combines consecutive input characters into

  • Whitespace sections
    • Spaces
    • Tabs
    • Line breaks
  • Tokens
    • Strings
    • Identifiers
    • Other tokens

Special Tokens (Markers)Permanent link for this heading

  • Marker for patterns: #
  • Marker for sections: :
  • Marker for ignored patterns: ##
  • Marker for ignored sections: ::

Header SyntaxPermanent link for this heading

Syntax

header = marker identifier [ parameterlist ].

marker = "#" [ "#" ] | ":" [ ":" ].

parameterlist = "(" [ parameter { "," parameter } ] ")".

parameter = identifier "=" ( string | identifier).

identifier = letter { letter | digit }.

string = '"' { char | escape } '"'.

How to Use a Control SpecificationPermanent link for this heading

If you have created a control specification, you have to reference it in your test. This happens in the source code of the test file.

Therefore, create a new test (see chapter “How to Create a Test File”) and then change to the "Source" view of the test.

A control specification is referenced in the ControlSpecification tag. The path and the name of the control specification file is defined in the filename attribute.

Example

<ControlSpecification filename="<path to configuration file>" />

Note: If you just enter a name (and no path), Fabasoft app.test searches for the control specification first in the "Configuration Files" folder of the current project (in here you usually save the configuration files you have written). If the control specification is not found in here, Fabasoft app.test looks in the standard folder for control specification files (in here you find the control specifications, which are provided by Fabasoft app.test) of Fabasoft app.test for the defined specification file.

You can change the path (after adding the ControlSpecification statement to the test) in the "Properties" view.

Start the Fabasoft app.test Recorder by clicking the "Start/Stop Recorder" button in the "Outline" view. You can test the following:

  • Web pages
    For testing a generic web page, enter the address of the web site into the Location field and select a browser in the Client Type field.
  • SWT applications
    For testing an SWT application (such as Eclipse), enter the path to the application to be tested into the Location field and select "SWT Application" in the Client Type field.
  • Applications by using the accessibility interface of Microsoft Windows
    For testing an application by using the accessibility interface of Microsoft Windows (e.g. Notepad or Adobe Acrobat), enter the path to the application (or to a document) to be tested into the Location field and select "Accessibility" in the Client Type field.

After starting the Fabasoft app.test Recorder, you can load the control specification you referenced before by selecting this statement in the "Outline" view and clicking "Run Step".

After loading the specification (which is only valid within the sequence where it has been loaded), the control tree of the tested web site resp. application is shown and you can now record statements by using the control tree (see chapter “How to Record a Test using the Control Tree”).

How to Integrate External Java Applications into Your TestPermanent link for this heading

Fabasoft app.test provides interfaces to implement and call external Java applications.
To implement such an application the following libraries are required:

  • app.test-adaptorinterfaces - Provides interfaces to the Fabasoft app.test Client Adaptor host and contains the required interfaces.
  • app.test-base - Provides basic Fabasoft app.test functionality and utilities.
  • app.test-runtime - Provides additional basic Fabasoft app.test functionality and utilities.
  • app.test-sdk - Provides basic Fabasoft app.test functionality and utilities with dependencies to the SWT library.
  • SWT 4.3 - This is required for the graphical user interface.

To call an external Fabasoft app.test Client Adaptor, a new context has to be established. These attributes must be defined:

Example

<Context clienttype="External" jarfile="{~resources~}/Adaptors/DUCXtestMailAdaptor.jar" classname="src.adaptor.MailAdaptor">
   <Execution action="Set" parameter="server" value="<Mail-Server>"/>
   <Execution action="Set" parameter="username" value="<Username>"/>
   <Execution action="Set" parameter="password" value="<Password>"/>
   <Execution action="Connect" location="Mailbox"/>
   <Set parameter="mail_exists" location="<Subject>" eval="exists"/>
   <Validation ok="{~mail_exists~}"/>
</Context>