Logo QF-Test

The testing tool for people who
are excited by perfection.

 

Free Trial  Download  Buy

Martina Schmid, QFS Director

Subscribe to our Release Notes: Send "Subscribe newsletter" to newsletter@remove-this.qfs.de.

Martina Schmid,
QFS Director

Manual | Tutorial | Standard Library | Release Notes

Release Notes

33.2
Test and Sequence nodes

Tests and sequences are the main structural elements of a test-suite. 'Test-case' nodes represent logical test-cases and are implemented as specialized sequences. A sequence is a container that executes its child nodes one by one. It can define variables (see chapter 7) that remain bound during the execution of the children.

Other kinds of sequences differ either in the way their child nodes are executed, or in the restrictions they impose on their child or parent nodes.

33.2.1
'Test-case'

A 'Test-case' node represents one or more logical test cases. In a sense it is the most important of all QF-Test nodes and everything else only serves to lend structure to 'Test-cases' or to implement their logic. Functionally it is a highly specialized 'Sequence' with a number of important extensions and special attributes.

A 'Test-case' should focus on the actual test it supposed to perform. Setup and cleanup tasks required to ensure that the 'Test-case' executes in the required environment and does not interfere with subsequent tests should be implemented in the form of 'Dependencies' as described in section 9.6. Alternatively - or in addition to 'Dependencies' - a 'Test-case' can have 'Setup' and 'Cleanup' nodes to be executed before and after the 'Test-case'.

Because a 'Test-case' can be called from a 'Test call' node it is also somewhat similar to a 'Procedure' in that its 'Name' attribute is mandatory and that it has a list of 'Parameter default values' that can be overridden in the calling node.

'Test-cases' also play a central role in run-logs and test reports. During a test-run a 'Test-case' node can be executed several times in different contexts and with different parameters. Logically these executions may represent the same or different test-cases. By defining a set of 'Characteristic variables' you can specify which variable values are used to differentiate between executions, thus characterizing the run-time environment of the test. The values of these variables at the time of entry to the 'Test-case' are stored in the run-log. To emphasize the fact that each execution of a 'Test-case' node may represent a separate logical test-case there is an alternative name attribute called 'Name for run-log and report'. Its value may contain references to the 'Characteristic variables' of the test. In the run-log or report the test will then be listed with this name, including the expanded run-time variable values.

Finally, there are situations in which a test cannot or should not be executed for specific variable settings. If the 'Condition' attribute of the 'Test-case' is defined, the 'Test-case' will only be executed if that expression expands to a true value. If the 'Test-case' is not executed due to the 'Condition' it will be listed as skipped in the report.

Contained in: 'Test-suite', 'Test-set'.

Children: Optional 'Dependency' or 'Dependency reference' as the first element. A 'Setup' may be the next and a 'Cleanup' the last node with an arbitrary number of normal child nodes in between. A 'Test-case' that does not contain normal child nodes will be listed as not implemented in the report.

Execution: First the 'Variable definitions' of the 'Test-case' are bound on the primary and its 'Parameter default values' on the fallback variable stack (see chapter 7). With these in place the 'Condition' is evaluated and the node will be skipped if a non-empty 'Condition' evaluates to false. Next the dependency of the 'Test-case' - possibly inherited from its parent node - is determined and resolved as described in section 9.6. Then the optional 'Setup' is executed once, followed by the normal child nodes and a single execution of the optional 'Cleanup'. An exception raised during the course of the 'Test-case' will be caught and passed to the 'Dependency' for handling in a 'Catch' node. Even if the exception is not handled by a 'Dependency' it is not propagated beyond the 'Test-case' to prevent aborting the whole test-run. The error state is duly noted in run-log and report however.

Attributes:

Test-case attributes
Figure 33.2:  'Test-case' attributes
'Name'

A 'Test-case' is identified by its name and the names of its 'Test-set' ancestors, so you should assign a name with a meaning that is easy to recognize and remember.

Variable: No

Restrictions: Must not be empty or contain the characters '.' or '#'.

'Name for run-log and report'

A separate name to be used for run-log and report. This is useful to differentiate between multiple executions with potentially different values for the 'Characteristic variables'.

Variable: Yes

Restrictions: None

'Characteristic variables'

These variables are part of the characteristics of a 'Test-set' or 'Test-case'. Two executions of a 'Test-case' are considered to represent the same logical test-case if the run-time values of all 'Characteristic variables' are identical. The run-time values of the 'Characteristic variables' are stored in the run-log.

Variable: No

Restrictions: None

'Name for separate run-log'

If this attribute is set it marks the node as a breaking point for split run-logs and defines the filename for the partial log. When the node finishes, the respective log entry is removed from the main run-log and saved as a separate, partial run-log. This operation is completely transparent, the main run-log retains references to the partial logs and is fully controllable. Please see subsection 8.1.4 for further information about split run-logs.

This attribute has no effect if the option Create split run-logs is disabled or split run-logs are explicitly turned off for batch mode via the -splitlog command line argument.

There is no need to keep the filename unique. If necessary, QF-Test appends a number to the filename to avoid collisions. The filename may contain directories and, similar to specifying the name of a run-log in batch mode on the command line, the following placeholders can be used after a '%' or a '+' character:

Character Replacement
% Literal '%' character.
+ Literal '+' character.
i The current runid as specified with -runid <ID>.
r The error level of the partial log.
w The number of warnings in the partial log.
e The number of errors in the partial log.
x The number of exceptions in the partial log.
t The thread index to which the partial log belongs (for tests run with parallel threads).
y The current year (2 digits).
Y The current year (4 digits).
M The current month (2 digits).
d The current day (2 digits).
h The current hour (2 digits).
m The current minute (2 digits).
s The current second (2 digits).
Table 33.1:  Placeholders for the 'Name for separate run-log' attribute

Variable: Yes

Restrictions: None, characters that are illegal for a filename will be replaced with '_'.

'Inherit dependency from parent node'

This option allows inheriting the 'Dependency' from the parent node as a replacement for or in addition to specifying a 'Dependency' for the node itself.

Variable: No

Restrictions: None

'Condition'

If the condition is non-empty it will be evaluated and if the result is false the execution of the current node is aborted. In this case the node will be reported as skipped.

Like the 'Condition' of an 'If' node, the 'Condition' is evaluated by the Jython interpreter, so the same rules apply.

Variable: Yes

Restrictions: Valid syntax

'Expected to fail if...'

This attribute allows specifying a condition under which the 'Test-case' is expected to fail. This is helpful to distinguish new problems from already known ones. Of course the latter should be fixed, but if and when that happens may be outside the tester's sphere of influence.

Most of the time it is sufficient to simply set this attribute to 'true' to mark the 'Test-case' as an expected failure. But if the 'Test-case' is executed multiple times, e.g. within a 'Data driver' or on multiple systems and fails only in specific cases, the condition should be written so that it evaluates to true for exactly those cases, e.g. ${qftest:windows} for a test that fails on Windows but runs fine on other systems.

If this attribute evaluates to true and the 'Test-case' fails with an error, it will be listed separately in the run-log, report and on the status line. It still means that there is an error in the application, so the overall percentage of successful tests is not changed.

It is treated as an error if this attribute evaluates to true and the 'Test-case' does not fail because this means that either the test is not able to reproduce the problem reliably or that the problem has been fixed and the 'Test-case' must be updated accordingly.

Variable: Yes

Restrictions: Valid syntax

'Variable definitions'

These variables are bound on the direct bindings stack (see chapter 7). They remain valid during the execution of the 'Test-case's' child nodes and cannot be overridden by a 'Test call' node. See subsection 2.2.5 about how to work with the table.

Variable: Variable names no, values yes

Restrictions: None

'Parameter default values'

Here you can define default or fallback values for the 'Test-case's' parameters (see chapter 7). Defining these values also serves as documentation and is a valuable time-saver when using the dialog to select the 'Test-case' for the 'Test name' attribute of a 'Test call'. See subsection 2.2.5 about how to work with the table.

Variable: Variable names no, values yes

Restrictions: None

'Maximum error level'

When a warning, error or exception occurs during a test run, the state of the corresponding node of the run-log is set accordingly. This state is normally propagated to the parent node in a way that ensures that the error state of a run-log node represents the worst of its child nodes' states. Using this attribute, the maximum error state that the run-log node for a sequence will propagate, can be limited.

Note This value has no effect on the way exceptions are handled. It only affects the error states of the run-log nodes and by that the exit code of QF-Test when run in batch mode (see. section 1.7). It also has no effect on the creation of compact run-logs (see command line argument -compact). The node for a sequence in which a warning, error or exception occurs is never removed from a compact log, even if the error is not propagated due to the setting of this attribute.

Variable: No

Restrictions: None

'Execution timeout'

Time limit for the node's execution in milliseconds. If that limit expires the execution of that node will get interrupted.

Variable: Yes

Restrictions: >= 0

'Border for relative calls'

This flag determines whether relative procedure calls, test calls or dependency references are allowed within that certain node. Relative calls passing that border are not allowed. If that attribute is not specified in the hierarchy, no relative calls are allowed.

Variable: No

Restrictions: None

'QF-Test ID'

When using the command line argument -test <n>|<ID> for execution in batch mode you can specify the 'QF-Test ID' of the node as an alternative to its qualified name.

Variable: No

Restrictions: Must not contain any of the characters '\', '#', '$', '@', '&', or '%' or start with an underscore ('_').

'Delay before/after'

These attributes cause a delay before or after the execution of the node. If a value is empty, the Default delay from the global options is used.

Variable: Yes

Restrictions: Valid number >= 0

'Comment'

Here you can enter a comment that explains the purpose of this node. This is the preferred way of documenting the test-suite.

Note For detailed documentation, especially for 'Test-set', 'Test-case' or 'Procedure' nodes, this text area might not be the right place. There are many excellent editors that are much better suited to this task. The option External editor command lets you define an external editor in which comments can be edited conveniently by pressing [Alt-Return] or by clicking the Edit row button.

You can trigger a special behavior of some nodes using doctags, please see Doctags.

Variable: Yes

Restrictions: None

33.2.2
'Test-set'

The main purpose of a 'Test-set' is to give structure to the 'Test-cases' of a test-suite. 'Test-sets' can be nested. The 'Name' of a 'Test-set' is part of the 'Test-case's' fully qualified name to which a 'Test call' node refers. 'Test-sets' are also callable themselves and thus have a set of 'Parameter default values' that can be overridden in the 'Test call'.

One way of structuring 'Test-cases' is to collect those with similar setup and cleanup requirements in the same 'Test-set' which can provide a 'Dependency' or 'Dependency reference' node to be inherited by the 'Test-case' or nested 'Test-set' child nodes. If the 'Inherit dependency from parent node' attribute of the 'Test-set' is set, the 'Dependency' nodes of the 'Test-set's' parent(s) are also inherited. See section 9.6 for information about QF-Test's dependency mechanism. Alternative or additional setup and cleanup is available in the form of 'Setup' and 'Cleanup' nodes which will be executed before and after each of the 'Test-set' and 'Test-case' child nodes.

Another important feature of a 'Test-set' is data-driven testing. This can be achieved by adding a 'Data driver' node with one or more 'Data binders' as described in chapter 15.

Like a 'Test-case', a 'Test-set' plays an important role for reports. As it can also be executed several times with different parameter settings, it has a set of 'Characteristic variables' and an alternative 'Name for run-log and report' that work just like for a 'Test-case'. The same is true for the 'Condition' which can be used to skip an entire 'Test-set' depending on current variable values.

Contained in: 'Test-suite', 'Test-set'.

Children: Optional 'Dependency' or 'Dependency reference' as the first child, followed by an optional 'Data driver'. A 'Setup' may come next and a 'Cleanup' at the end with an arbitrary number of 'Test-set', 'Test-case' and 'Test call' nodes in between.

Execution: First the 'Parameter default values' of the 'Test-set' are bound on the fallback variable stack (see chapter 7). With these in place the 'Condition' is evaluated and the node will be skipped if a non-empty 'Condition' evaluates to false. The 'Dependency' or 'Dependency reference' node will not be resolved at this time unless its 'Always execute, even in test-suite and test-set nodes' attribute is set. If there is a 'Data driver' node, it is executed to create a data driving context, bind one or more 'Data binders' and then repeatedly execute the other child nodes as described in chapter 15. For each loop iteration - or once in case no 'Data driver' is present - the 'Test-set' executes each of its 'Test-set' or 'Test-case' child nodes. If an optional 'Setup' or 'Cleanup' node are present, they are executed before and after each of these child nodes respectively.

Attributes:

Test-set attributes
Figure 33.3:  'Test-set' attributes
'Name'

The name of a 'Test-set' is part of its own identification and of that of the 'Test-case' and 'Test-set' nodes it contains, so you should assign a name with a meaning that is easy to recognize and remember.

Variable: No

Restrictions: Must not be empty or contain the characters '.' or '#'.

'Name for run-log and report'

A separate name to be used for run-log and report. This is useful to differentiate between multiple executions with potentially different values for the 'Characteristic variables'.

Variable: Yes

Restrictions: None

'Characteristic variables'

These variables are part of the characteristics of a 'Test-set' or 'Test-case'. Two executions of a 'Test-case' are considered to represent the same logical test-case if the run-time values of all 'Characteristic variables' are identical. The run-time values of the 'Characteristic variables' are stored in the run-log.

Variable: No

Restrictions: None

'Name for separate run-log'

If this attribute is set it marks the node as a breaking point for split run-logs and defines the filename for the partial log. When the node finishes, the respective log entry is removed from the main run-log and saved as a separate, partial run-log. This operation is completely transparent, the main run-log retains references to the partial logs and is fully controllable. Please see subsection 8.1.4 for further information about split run-logs.

This attribute has no effect if the option Create split run-logs is disabled or split run-logs are explicitly turned off for batch mode via the -splitlog command line argument.

There is no need to keep the filename unique. If necessary, QF-Test appends a number to the filename to avoid collisions. The filename may contain directories and, similar to specifying the name of a run-log in batch mode on the command line, the following placeholders can be used after a '%' or a '+' character:

Character Replacement
% Literal '%' character.
+ Literal '+' character.
i The current runid as specified with -runid <ID>.
r The error level of the partial log.
w The number of warnings in the partial log.
e The number of errors in the partial log.
x The number of exceptions in the partial log.
t The thread index to which the partial log belongs (for tests run with parallel threads).
y The current year (2 digits).
Y The current year (4 digits).
M The current month (2 digits).
d The current day (2 digits).
h The current hour (2 digits).
m The current minute (2 digits).
s The current second (2 digits).
Table 33.2:  Placeholders for the 'Name for separate run-log' attribute

Variable: Yes

Restrictions: None, characters that are illegal for a filename will be replaced with '_'.

'Inherit dependency from parent node'

This option allows inheriting the 'Dependency' from the parent node as a replacement for or in addition to specifying a 'Dependency' for the node itself.

Variable: No

Restrictions: None

'Condition'

If the condition is non-empty it will be evaluated and if the result is false the execution of the current node is aborted. In this case the node will be reported as skipped.

Like the 'Condition' of an 'If' node, the 'Condition' is evaluated by the Jython interpreter, so the same rules apply.

Variable: Yes

Restrictions: Valid syntax

'Parameter default values'

Here you can define default or fallback values for the 'Test-set's' parameters (see chapter 7). Defining these values also serves as documentation and is a valuable time-saver when using the dialog to select the 'Test-set' for the 'Test name' attribute of a 'Test call'. See subsection 2.2.5 about how to work with the table.

Variable: Variable names no, values yes

Restrictions: None

'Maximum error level'

When a warning, error or exception occurs during a test run, the state of the corresponding node of the run-log is set accordingly. This state is normally propagated to the parent node in a way that ensures that the error state of a run-log node represents the worst of its child nodes' states. Using this attribute, the maximum error state that the run-log node for a sequence will propagate, can be limited.

Note This value has no effect on the way exceptions are handled. It only affects the error states of the run-log nodes and by that the exit code of QF-Test when run in batch mode (see. section 1.7). It also has no effect on the creation of compact run-logs (see command line argument -compact). The node for a sequence in which a warning, error or exception occurs is never removed from a compact log, even if the error is not propagated due to the setting of this attribute.

Variable: No

Restrictions: None

'Execution timeout'

Time limit for the node's execution in milliseconds. If that limit expires the execution of that node will get interrupted.

Variable: Yes

Restrictions: >= 0

'Border for relative calls'

This flag determines whether relative procedure calls, test calls or dependency references are allowed within that certain node. Relative calls passing that border are not allowed. If that attribute is not specified in the hierarchy, no relative calls are allowed.

Variable: No

Restrictions: None

'QF-Test ID'

When using the command line argument -test <n>|<ID> for execution in batch mode you can specify the 'QF-Test ID' of the node as an alternative to its qualified name.

Variable: No

Restrictions: Must not contain any of the characters '\', '#', '$', '@', '&', or '%' or start with an underscore ('_').

'Delay before/after'

These attributes cause a delay before or after the execution of the node. If a value is empty, the Default delay from the global options is used.

Variable: Yes

Restrictions: Valid number >= 0

'Comment'

Here you can enter a comment that explains the purpose of this node. This is the preferred way of documenting the test-suite.

Note For detailed documentation, especially for 'Test-set', 'Test-case' or 'Procedure' nodes, this text area might not be the right place. There are many excellent editors that are much better suited to this task. The option External editor command lets you define an external editor in which comments can be edited conveniently by pressing [Alt-Return] or by clicking the Edit row button.

You can trigger a special behavior of some nodes using doctags, please see Doctags.

Variable: Yes

Restrictions: None

33.2.3
'Test call'

With this node a call to some other executable test node can be made. Possible targets are 'Test-suite', 'Test-set' and 'Test-case' nodes in the same or a different test-suite. Execution will continue in the called node and when finished return to the 'Test call' and thus to its parent node.

The name of a 'Test-case' or 'Test-set' to call is determined by its 'Name' and the 'Names' of its 'Test-set' ancestors. These are concatenated with a dot ('.') as separator, starting with the outermost 'Test-set' and ending in the 'Test-case's' name. Thus to call a 'Test-case' named nodeTest in a 'Test-set' named Tree that is itself a child of a 'Test-set' named Main, the 'Test name' attribute would be set to 'Main.Tree.nodeTest'. A node in a different test-suite is addressed by prepending the filename of the test-suite followed by a '#' to the 'Test name'. A 'Test-suite' node is addressed by a single '.', so calling a whole test-suite is done with a 'Test name' attribute of the form 'suitename.qft#.'. It is generally easiest to pick the target node interactively from a dialog by clicking on the button above the 'Test name' attribute.

See also section 18.1 for further information about cross-suite calls.

Contained in: All kinds of sequences.

Children: None

Execution: The 'Variable definitions' of the 'Test call' are bound, the target node is determined and execution is passed to it. After the 'Test call' returns, the 'Test call's' variables are unbound again.

Attributes:

Test call Attributes
Figure 33.4:  'Test call' Attributes
'Name'

In case of a 'Test-case' or 'Test-set' it is the full name, created from the 'Name' of its 'Test-set' parents and its own 'Name', joined by a dot. A 'Test-suite' node is addressed by a single '.' character. The button above the attribute brings up a dialog in which you can select the target node interactively. You can also get to this dialog by pressing [Shift-Return] or [Alt-Return] when the focus is in the text field. By selecting the "Copy parameters" checkbox you can adopt the 'Test-case's' or 'Test-set's' 'Parameter default values' as parameters for the 'Test call' node to save typing.

Variable: Yes

Restrictions: Must not be empty.

'Name for separate run-log'

If this attribute is set it marks the node as a breaking point for split run-logs and defines the filename for the partial log. When the node finishes, the respective log entry is removed from the main run-log and saved as a separate, partial run-log. This operation is completely transparent, the main run-log retains references to the partial logs and is fully controllable. Please see subsection 8.1.4 for further information about split run-logs.

This attribute has no effect if the option Create split run-logs is disabled or split run-logs are explicitly turned off for batch mode via the -splitlog command line argument.

There is no need to keep the filename unique. If necessary, QF-Test appends a number to the filename to avoid collisions. The filename may contain directories and, similar to specifying the name of a run-log in batch mode on the command line, the following placeholders can be used after a '%' or a '+' character:

Character Replacement
% Literal '%' character.
+ Literal '+' character.
i The current runid as specified with -runid <ID>.
r The error level of the partial log.
w The number of warnings in the partial log.
e The number of errors in the partial log.
x The number of exceptions in the partial log.
t The thread index to which the partial log belongs (for tests run with parallel threads).
y The current year (2 digits).
Y The current year (4 digits).
M The current month (2 digits).
d The current day (2 digits).
h The current hour (2 digits).
m The current minute (2 digits).
s The current second (2 digits).
Table 33.3:  Placeholders for the 'Name for separate run-log' attribute

Variable: Yes

Restrictions: None, characters that are illegal for a filename will be replaced with '_'.

'Variable definitions'

This is where you define the parameter values for the target node. These variables are bound on the primary variable stack (see chapter 7) so they override any 'Parameter default values'. See subsection 2.2.5 about how to work with the table.

4.2+ In case you want to re-set the order of the parameters like they are sorted in the called test-case or test-set, you can select »Re-set parameter order«.

Variable: Variable names no, values yes

Restrictions: None

'Execution timeout'

Time limit for the node's execution in milliseconds. If that limit expires the execution of that node will get interrupted.

Variable: Yes

Restrictions: >= 0

'QF-Test ID'

At the moment the 'QF-Test ID' attribute has no meaning for this type of node.

Variable: No

Restrictions: Must not contain any of the characters '\', '#', '$', '@', '&', or '%' or start with an underscore ('_').

'Delay before/after'

These attributes cause a delay before or after the execution of the node. If a value is empty, the Default delay from the global options is used.

Variable: Yes

Restrictions: Valid number >= 0

'Comment'

Here you can enter a comment that explains the purpose of this node. This is the preferred way of documenting the test-suite.

Note For detailed documentation, especially for 'Test-set', 'Test-case' or 'Procedure' nodes, this text area might not be the right place. There are many excellent editors that are much better suited to this task. The option External editor command lets you define an external editor in which comments can be edited conveniently by pressing [Alt-Return] or by clicking the Edit row button.

You can trigger a special behavior of some nodes using doctags, please see Doctags.

Variable: Yes

Restrictions: None

33.2.4
'Sequence'

This is the most general kind of sequence. Its children are executed one by one and their number or type is not limited in any way.

Contained in: All kinds of sequences.

Children: Any

Execution: The 'Variable definitions' of the 'Sequence' are bound and its child nodes executed one by one. After the execution of the last child is complete, the variables are unbound again.

Attributes:

Sequence attributes
Figure 33.5:  'Sequence' attributes
'Name'

The name of a sequence is a kind of short description. It is displayed in the tree view, so it should be concise and tell something about the function of the sequence.

Variable: No

Restrictions: None

'Variable definitions'

This is where you define the values of the variables that remain bound during the execution of the sequence's child nodes (see chapter 7). See subsection 2.2.5 about how to work with the table.

Variable: Variable names no, values yes

Restrictions: None

'Maximum error level'

When a warning, error or exception occurs during a test run, the state of the corresponding node of the run-log is set accordingly. This state is normally propagated to the parent node in a way that ensures that the error state of a run-log node represents the worst of its child nodes' states. Using this attribute, the maximum error state that the run-log node for a sequence will propagate, can be limited.

Note This value has no effect on the way exceptions are handled. It only affects the error states of the run-log nodes and by that the exit code of QF-Test when run in batch mode (see. section 1.7). It also has no effect on the creation of compact run-logs (see command line argument -compact). The node for a sequence in which a warning, error or exception occurs is never removed from a compact log, even if the error is not propagated due to the setting of this attribute.

Variable: No

Restrictions: None

'QF-Test ID'

At the moment the 'QF-Test ID' attribute has no meaning for this type of node.

Variable: No

Restrictions: Must not contain any of the characters '\', '#', '$', '@', '&', or '%' or start with an underscore ('_').

'Delay before/after'

These attributes cause a delay before or after the execution of the node. If a value is empty, the Default delay from the global options is used.

Variable: Yes

Restrictions: Valid number >= 0

'Comment'

Here you can enter a comment that explains the purpose of this node. This is the preferred way of documenting the test-suite.

Note For detailed documentation, especially for 'Test-set', 'Test-case' or 'Procedure' nodes, this text area might not be the right place. There are many excellent editors that are much better suited to this task. The option External editor command lets you define an external editor in which comments can be edited conveniently by pressing [Alt-Return] or by clicking the Edit row button.

You can trigger a special behavior of some nodes using doctags, please see Doctags.

Variable: Yes

Restrictions: None

33.2.5
'Test-step'

A 'Test-step' is a special 'Sequence' that serves to divide a 'Test-case' into steps that can be documented individually and will show up in the report or testdoc documentation. In contrast to 'Test-cases', which should not be nested, 'Test-steps' can be nested to any depth.

Contained in: All kinds of sequences.

Children: Any

Execution: The 'Variable definitions' of the 'Test-step' are bound and its child nodes executed one by one. After the execution of the last child is complete, the variables are unbound again.

Attributes:

Test-step attributes
Figure 33.6:  'Test-step' attributes
'Name'

The name of a sequence is a kind of short description. It is displayed in the tree view, so it should be concise and tell something about the function of the sequence.

Variable: No

Restrictions: None

'Name for run-log and report'

A separate name to be used for run-log and report. This is useful to differentiate between multiple executions of the surrounding 'Test-case' with potentially different values for its 'Characteristic variables'.

Variable: Yes

Restrictions: None

'Name for separate run-log'

If this attribute is set it marks the node as a breaking point for split run-logs and defines the filename for the partial log. When the node finishes, the respective log entry is removed from the main run-log and saved as a separate, partial run-log. This operation is completely transparent, the main run-log retains references to the partial logs and is fully controllable. Please see subsection 8.1.4 for further information about split run-logs.

This attribute has no effect if the option Create split run-logs is disabled or split run-logs are explicitly turned off for batch mode via the -splitlog command line argument.

There is no need to keep the filename unique. If necessary, QF-Test appends a number to the filename to avoid collisions. The filename may contain directories and, similar to specifying the name of a run-log in batch mode on the command line, the following placeholders can be used after a '%' or a '+' character:

Character Replacement
% Literal '%' character.
+ Literal '+' character.
i The current runid as specified with -runid <ID>.
r The error level of the partial log.
w The number of warnings in the partial log.
e The number of errors in the partial log.
x The number of exceptions in the partial log.
t The thread index to which the partial log belongs (for tests run with parallel threads).
y The current year (2 digits).
Y The current year (4 digits).
M The current month (2 digits).
d The current day (2 digits).
h The current hour (2 digits).
m The current minute (2 digits).
s The current second (2 digits).
Table 33.4:  Placeholders for the 'Name for separate run-log' attribute

Variable: Yes

Restrictions: None, characters that are illegal for a filename will be replaced with '_'.

'Variable definitions'

This is where you define the values of the variables that remain bound during the execution of the sequence's child nodes (see chapter 7). See subsection 2.2.5 about how to work with the table.

Variable: Variable names no, values yes

Restrictions: None

'Maximum error level'

When a warning, error or exception occurs during a test run, the state of the corresponding node of the run-log is set accordingly. This state is normally propagated to the parent node in a way that ensures that the error state of a run-log node represents the worst of its child nodes' states. Using this attribute, the maximum error state that the run-log node for a sequence will propagate, can be limited.

Note This value has no effect on the way exceptions are handled. It only affects the error states of the run-log nodes and by that the exit code of QF-Test when run in batch mode (see. section 1.7). It also has no effect on the creation of compact run-logs (see command line argument -compact). The node for a sequence in which a warning, error or exception occurs is never removed from a compact log, even if the error is not propagated due to the setting of this attribute.

Variable: No

Restrictions: None

'Execution timeout'

Time limit for the node's execution in milliseconds. If that limit expires the execution of that node will get interrupted.

Variable: Yes

Restrictions: >= 0

'QF-Test ID'

At the moment the 'QF-Test ID' attribute has no meaning for this type of node.

Variable: No

Restrictions: Must not contain any of the characters '\', '#', '$', '@', '&', or '%' or start with an underscore ('_').

'Delay before/after'

These attributes cause a delay before or after the execution of the node. If a value is empty, the Default delay from the global options is used.

Variable: Yes

Restrictions: Valid number >= 0

'Comment'

Here you can enter a comment that explains the purpose of this node. This is the preferred way of documenting the test-suite.

Note For detailed documentation, especially for 'Test-set', 'Test-case' or 'Procedure' nodes, this text area might not be the right place. There are many excellent editors that are much better suited to this task. The option External editor command lets you define an external editor in which comments can be edited conveniently by pressing [Alt-Return] or by clicking the Edit row button.

You can trigger a special behavior of some nodes using doctags, please see Doctags.

Variable: Yes

Restrictions: None

33.2.6
'Sequence with time limit'

This node extends the basic 'Sequence' node with time-constraint checking. Child nodes are executed as usual, but upon completion of the sequence the elapsed time is compared to the time-limit. Exceeding the time limit will result in a warning, error or exception, depending on the value of the attribute 'Error level if time limit exceeded'. Explicit delays like 'Delay before/after' or user interrupts are deducted from the duration before the constraints are checked, unless 'Check realtime' is activated.

For report generation, time-constraints are treated like checks. If the 'Comment' attribute starts with an '!' character, the result will be logged in the report.

Note The function of this node is to check time constraints in the user-acceptance range, i.e. between a few hundred milliseconds and a few seconds. Real-time constraints of a few milliseconds or less are beyond its limits.

Contained in: All kinds of sequences.

Children: Any

Execution: The 'Variable definitions' of the 'Sequence with time limit' are bound and its child nodes executed one by one. After the execution of the last child is complete, the variables are unbound again. The elapsed time is compared to the given time limit.

Attributes:

Sequence with time limit attributes
Figure 33.7:  'Sequence with time limit' attributes
'Name'

The name of a sequence is a kind of short description. It is displayed in the tree view, so it should be concise and tell something about the function of the sequence.

Variable: No

Restrictions: None

'Time limit for execution'

The time (in milliseconds) allowed for the execution of the sequence.

Variable: Yes

Restrictions: Must not be negative.

'Check realtime'

Normally explicit delays like 'Delay before/after' or user interrupts are deducted from the duration, before the time constraints are checked. To prevent this deduction and therefore check the real-time, this attribute can be activated.

Variable: No

Restrictions: None

'Error level if time limit exceeded'

This attribute determines what happens in case the time limit is exceeded. If set to "exception", a CheckFailedException will be thrown. Otherwise a message with the respective error-level will be logged in the run-log.

Variable: No

Restrictions: None

'Variable definitions'

This is where you define the values of the variables that remain bound during the execution of the sequence's child nodes (see chapter 7). See subsection 2.2.5 about how to work with the table.

Variable: Variable names no, values yes

Restrictions: None

'Maximum error level'

When a warning, error or exception occurs during a test run, the state of the corresponding node of the run-log is set accordingly. This state is normally propagated to the parent node in a way that ensures that the error state of a run-log node represents the worst of its child nodes' states. Using this attribute, the maximum error state that the run-log node for a sequence will propagate, can be limited.

Note This value has no effect on the way exceptions are handled. It only affects the error states of the run-log nodes and by that the exit code of QF-Test when run in batch mode (see. section 1.7). It also has no effect on the creation of compact run-logs (see command line argument -compact). The node for a sequence in which a warning, error or exception occurs is never removed from a compact log, even if the error is not propagated due to the setting of this attribute.

Variable: No

Restrictions: None

'Execution timeout'

Time limit for the node's execution in milliseconds. If that limit expires the execution of that node will get interrupted.

Variable: Yes

Restrictions: >= 0

'QF-Test ID'

At the moment the 'QF-Test ID' attribute has no meaning for this type of node.

Variable: No

Restrictions: Must not contain any of the characters '\', '#', '$', '@', '&', or '%' or start with an underscore ('_').

'Delay before/after'

These attributes cause a delay before or after the execution of the node. If a value is empty, the Default delay from the global options is used.

Variable: Yes

Restrictions: Valid number >= 0

'Comment'

Here you can enter a comment that explains the purpose of this node. This is the preferred way of documenting the test-suite.

Note For detailed documentation, especially for 'Test-set', 'Test-case' or 'Procedure' nodes, this text area might not be the right place. There are many excellent editors that are much better suited to this task. The option External editor command lets you define an external editor in which comments can be edited conveniently by pressing [Alt-Return] or by clicking the Edit row button.

You can trigger a special behavior of some nodes using doctags, please see Doctags.

Variable: Yes

Restrictions: None

33.2.7
'Extras'

This node is a kind of clipboard or playground where the usual restrictions on the parent of a node don't apply. You can add any kind of node here to assemble and try out some test sequences.

Contained in: Root node

Children: Any

Execution: Cannot be executed

Attributes:

Extras attributes
Figure 33.8:  'Extras' attributes
'QF-Test ID'

At the moment the 'QF-Test ID' attribute has no meaning for this type of node.

Variable: No

Restrictions: Must not contain any of the characters '\', '#', '$', '@', '&', or '%' or start with an underscore ('_').

'Comment'

Here you can enter a comment that explains the purpose of this node. This is the preferred way of documenting the test-suite.

Note For detailed documentation, especially for 'Test-set', 'Test-case' or 'Procedure' nodes, this text area might not be the right place. There are many excellent editors that are much better suited to this task. The option External editor command lets you define an external editor in which comments can be edited conveniently by pressing [Alt-Return] or by clicking the Edit row button.

You can trigger a special behavior of some nodes using doctags, please see Doctags.

Variable: Yes

Restrictions: None

Videos Downloads Documentation Buy Free Trial