Variables

Video Video: 'Variables' .

Variables are the primary means to add flexibility to a test suite. Though they are used mainly as parameters for 'Procedures', they are also useful in many other cases.

Variable syntax can be used in almost every attribute of the nodes of a suite, except for boolean values (check boxes). There are three versions of variable references:

  • $(varname) expands to the value of some previously defined variable.
  • ${group:name} accesses external data from a resource bundle or property file. The groups system and qftest are always defined and have a special meaning (see section 6.5).
  • $[expression] evaluates some mathematical expressions.

Variable lookup

To understand the reasons of why and how variables are defined in multiple places, you first have to learn about how the values of variables are determined.

Each set of variable definitions, called bindings, is placed on one of two stacks of bindings. One stack is used for actual - or direct - definitions and one for fallback bindings or default values. When a variable's value is requested via $(...), QF-Test first searches the stack of direct bindings from top to bottom, then the stack of fallbacks, also top down. The first value found is used. If there is no binding at all for a name, an UnboundVariableException is thrown unless you use the special syntax ${default:varname:defaultvalue} to provide a default value for this case as described in section 6.5.

Topmost bindings
(highest precedence)
...
Bottommost bindings
(lowest precedence)
Topmost bindings
(highest precedence)
...
Bottommost bindings
(lowest precedence)
Primary stack
(Direct bindings)
Secondary stack
(Default values)
Figure 6.1:  Direct and fallback bindings

This mechanism supports recursive or self-referencing variable definitions. For example, setting a variable named classpath to the value some/path/archive.jar:$(classpath) will extend a binding for classpath with lower precedence. If no such binding exists, a RecursiveVariableException is thrown.

Defining variables

Variables are defined in various places with different orders of precedence. You define a variable by adding a row to the appropriate table with the name and value of the variable in the respective columns.

System variables
Figure 6.2:  System variables

Four sets of variable definitions are accessible from the global options:

System variables
The most basic set of bindings, commonly used for system-specific definitions like path names, JDK version, operating system stuff etc. These are placed at the bottom of the fallback stack so they have the least possible precedence. The system bindings are saved in the system configuration file together with the other QF-Test system options.
Suite variables
These are general definitions for a test suite and saved as part of the suite. The bindings of the suite from which a test run is started, are placed at the bottom of the direct bindings stack. When a procedure from a different suite is called, or a different suite's window or component is accessed, the bindings of that suite are temporarily placed on top of the fallback stack.
Variables from the command line
You can define variables on the command line when starting QF-Test with the -variable <name>=<value> argument which can be used multiple times. These bindings are placed above the suite bindings on the direct bindings stack, so they override the bindings of the suite. For example, a variable count=1 can be defined for the suite and used as $(count) for the 'Number of iterations' of a 'Loop' node for quick trial runs. Then you can use qftest -batch -variable count=100 ... for the actual testing. The command line bindings are accessible mainly for your information, but you can also edit them for experimentation.
Global variables
Placed above the command line variables on the direct call stack, these bindings are a means to pass values between unrelated parts of a test run. Values read from the SUT by a 'Fetch text' or a 'Fetch index' node are bound here, as well as definitions from a 'Set variable' node. The globals are not cleared in between test runs as a convenience and you can edit them for the same reason. To simulate a real test run you should clear them via the »Run«-»Clear global variables« menu first. When run in batch mode (see section 1.7) QF-Test clears the global variables before the execution of each test given with the -test <n>|<ID> command line argument.

The rest of the variable definitions are part of the test suite itself:

  • Each sequence type node defines a set of bindings that are placed on top of the direct bindings stack when the test run enters the node and removed when the node is exited.
  • A 'Procedure call's' bindings are placed on top of the direct bindings stack before executing the 'Procedure'. The bindings defined by a 'Procedure' are placed on top of the fallback stack during execution of the 'Procedure'. Think of them as default values for the expected parameters. Once the procedure returns, both sets of bindings are removed.

Variable example

Consider the following example:

Variable example
Figure 6.3:  Variable example

The 'Sequence' "Login" contains a 'Procedure call' of the 'Procedure' "login" which expects two parameters: user and password. The 'Procedure's' default bindings are user=username and password=pwd. The 'Procedure call' overrides these with user=myname and password=mypassword.

The "login" 'Procedure' itself holds 'Procedure calls' of 'Procedures' in another test suite called "lib.qft" to write the user and password to some GUI components. We assume that the 'Procedures' in this library have most parameters in common, so they don't define default values themselves but in the suite bindings of "lib.qft" where they are set to user=libuser and password=libpwd. This is useful for creating and editing the procedures in the library since they can all be executed and tried out separately without having to define 'Procedure calls'.

The following diagram shows the State of the binding stacks during the execution of the 'Procedure' "lib.qft#setUser":

'Procedure call' "lib.qft#setUser"
'Procedure call' "login"
user=myname
password=mypassword
'Sequence' "Login"
Global bindings
Command line bindings
Suite bindings
'Procedure' "lib.qft#setUser"
Test suite "lib.qft"
user=libuser
password=libpwd
'Procedure' "Login"
user=user
password=pwd
System bindings
Primary stack
(Direct bindings)
Secondary stack
(Default values)
Figure 6.4:  Variable bindings example

The important thing to note here is that the 'Procedure call' to "lib.qft#setUser" inside the 'Procedure' "login" does not need to define the user parameter again, the parameter is "passed through". As a rule of thumb, when calling one 'Procedure' from another, define a parameter value in the 'Procedure call' if and only if the value has not been explicitly defined yet or you want to pass a different value.

Fetching data from the GUI

Often it is necessary to fetch some kind of data from the SUT's GUI to use it as test input.

QF-Test offers a special set of query nodes for this kind of task, available via »Insert«-»Miscellaneous«:

The retrieved values are assigned to local or global variables that can be declared in the fetch node.

Instead of inserting fetch nodes by hand they can easily be created by first recording a mouse click node the the respective component and then use the transform operation to convert the same to the fetch node you need.

External data and special groups

External data is made available by the 'Load resources' and 'Load properties' nodes. These assign a group name to a set of definitions from a resource bundle or properties file. To access the value of the definition for name, use the syntax ${group:name}.

When run in batch mode (see section 1.7) QF-Test clears the resources and properties before the execution of each test given with the -test <n>|<ID> command line argument. In normal mode QF-Test keeps them around to ease building a suite, but for a true trial run you should clear them via the »Run«-»Clear resources and properties« menu first.

Some special group names are predefined and always available:

system
The group system gives access to the system properties of the Java VM (for programmers: java.lang.System.getProperties()), e.g. ${system:user.home} for the user's home directory or ${system:java.class.path} for the CLASSPATH with which QF-Test was started. Which names are defined in the group system depends on the utilised JDK.
The group always refers to the VM QF-Test was started with, because variable expansion takes place there.
env
On operating systems which support environment variables like PATH, CLASSPATH or JAVA_HOME (practically all systems QF-Test runs on), these environment variables can be accessed with the help of the group env.
default
3.4+ You can specify a default value for a variable with the group default. The syntax is ${default:varname:defaultvalue}. This is extremely useful for things like generic components or in almost every place where there is a reasonable default for a variable because the default value is then tightly coupled with the use of the variable and doesn't have to be specified at 'Sequence' or test suite level. Of course you should only use this syntax if the variable lookup in question is more or less unique. If you are using the same variable with the same default in different places it is preferable to use normal syntax and explicitly set the default, so that the default for all values can be changed in a single place.
id
3.1+ The group id can be used to reference QF-Test component IDs. Values in this group simply expand to themselves, i.e. "${id:whatever}" expands to "whatever". Though QF-Test component IDs can be referenced without the help of this group, its use increases the readability of tests. Most notably however, QF-Test component ID references in this group will be updated automatically in case the referenced target component gets moved or its QF-Test ID changed.
idlocal
4.2.3+ The group idlocal is similar to the id group but includes the path to the current test suite, i.e. "${idlocal:x}" expands to "path/to/current/suite/suite.qft#x". This enforces use of the component referenced in the suite that is current at the time of expansion, irrespective of whether there is a component with the same %attId; in the target suite of a procedure call.
quoteitem
4.0+ Via the quoteitem group you can conveniently escape special characters like '@', '&' and '%' in the name of a textual sub-item index to prevent it from being treated as several items, e.g. "${quoteitem:user@host.org}" will result in "user\@host.org".
quoteregex, quoteregexp
4.0+ The group quoteregex with its alias quoteregexp can be used to escape characters with special meaning in regular expressions. This is often useful when building regular expressions dynamically or when referencing subitems with special characters in their name by a regular expression index, e.g. "componentid%${quoteregex:foo(baa)}.*" allows you to address the first occurrence of items beginning with 'foo(baa)'.
quotesmartid
6.0.1+ The quotesmartid group is similar to quoteitem. In addition to the item syntax special characters '@', '&' and '%' it also escapes the characters ':', '=', '<' and '>' that have special meaning in SmartIDs, e.g. "${quotesmartid:Name: A & B}" will result in "Name\: A \& B".
qftest
The special group named qftest provides miscellaneous values that may be useful during a test run. The following tables list the values currently defined.
NameMeaning
32 or 32bittrue if QF-Test is running in a 32bit Java VM - which is not the same as running on a 32bit Operating System - false otherwise.
64 or 64bittrue if QF-Test is running in a 64bit Java VM, false otherwise.
batchtrue if QF-Test is running in batch mode, false for interactive mode.
client.exitcode.<name> The exit-code of the last process started with the 'Client' attribute set to <name>. In case the process is still alive the result is the empty string.
client.output.<name> The output of the last process started with the 'Client' attribute set to <name>. The maximum size for buffered output is defined by the option Maximum size of client terminal (kB).
client.stdout.<name> The output on the standard output stream (stdout) of the last process (started with the 'Client' attribute set to <name>). The maximum size for buffered output is defined by the option Maximum size of client terminal (kB).
client.stderr.<name> The output on the standard error stream (stderr) of the last process (started with the 'Client' attribute set to <name>). The maximum size for buffered output is defined by the option Maximum size of client terminal (kB).
clients A list of the names of all active process clients, separated by a newline.
clients.all A list of the names of all process clients, separated by a newline. This includes live clients as well as the recent dead clients similar to those listed in the "Clients" menu.
count.exceptions Number of exceptions in the current test run.
count.errors Number of errors in the current test run.
count.warnings Number of warnings in the current test run.
count.testcases Total number of total test cases (run and skipped) in the current test run.
count.testcases.exception Number of test cases with exceptions in the current test run.
count.testcases.error Number of test cases with errors in the current test run.
count.testcases.expectedtofail Number of test cases expected to fail in the current test run.
count.testcases.ok Number of successful test cases in the current test run.
count.testcases.ok.percentage Percentage of successful test cases in the current test run.
count.testcases.skipped Number of skipped test cases in the current test run.
count.testcases.notimplemented Number of not implemented test cases in the current test run.
count.testcases.run Number of run test cases in the current test run.
count.testsets.skipped Number of skipped test sets in the current test run.
dir.cacheCache directory of QF-Test
dir.groovyDirectory of Groovy
dir.javascriptDirectory of JavaScript
dir.jythonDirectory of Jython
dir.logLog directory of QF-Test
dir.pluginPlugin directory of QF-Test
dir.rootRoot directory of QF-Test
dir.runlogRun log directory of QF-Test
dir.system System-specific configuration directory of QF-Test.
dir.userUser-specific configuration directory of QF-Test
dir.versionVersion-specific directory of QF-Test
engine.<componentId>Retrieves the GUI engine responsible for the given component (see chapter 43).
languageThe language in which QF-Test displays its graphical user interface.
licenseThe path to the license file
systemcfgThe path to the system configuration file
usercfgThe path to the user specific configuration file
executable The qftest executable matching the currently running QF-Test version, including the full path to its bin directory and with .exe appended on Windows. Useful if you need to run QF-Test from QF-Test for example to call a daemon or create reports.
isInRerun"true", if current execution is in rerun mode, "false" otherwise, see subsection 23.3.2.
isInRerunFromLog"true", if test run has been re-started from run log, "false" otherwise, see subsection 23.3.1.
javaStandard Java program (javaw under Windows, java under Linux/Unix) or the explicit Java argument if QF-Test is started with -java <executable>(deprecated)
java.mainversion The major version of the JRE that QF-Test currently runs on, using 8 for Java 1.8 so the result is something like 8, 11 or 17.
java.subversion The sub-version of the JRE that QF-Test currently runs on. For Java 8 the sub-version taken from after the '_', so for java.version 1.8.0_302 this results in 302. For Java 9 or higher this is the minor version, e.g. 9 in case of java.version 11.0.9.
linux"true" under Linux, "false" otherwise
macos"true" under macOS, "false" otherwise
os.fullversionThe whole version of the operating system
os.mainversionThe main version of the operating system, e.g. "10" for Windows 10
os.nameThe name of the operating system
os.versionThe version of the operating system. In some cases that's not the whole one then you should use os.fullversion instead.
project.dirThe directory to the current project. This variable is not defined in case the current test suite is not part of a project.
reruncounterNumber of current rerun attempt, default is 0, for details see subsection 23.3.2.
return The most recent value returned from a 'Procedure' through a 'Return' node.
runid The runid of the current test run. See section 22.1 for further information about the runid.
screen.heightScreen height in pixels
screen.widthScreen width in pixels
skipnode This magic value is not for the casual user. It causes QF-Test to skip execution of the current node. Its primary use is as the value for a variable defined in the 'Text' attribute of a 'Text input' node which also has its 'Clear target component first' attribute set. An empty value would clear the field whereas $_{qftest:skipnode} leaves the field unchanged. But skipnode is also applicable for fine-grained execution control by placing a variable in the comment of a node and selectively passing $_{qftest:skipnode} to that variable. Please note that you almost always want to use lazy syntax '$_' with this variable. Otherwise its expansion as the parameter in a 'Procedure call' node would cause skipping the whole call.
suite.dirDirectory of the current suite
suite.fileFile name of the current suite without directory
suite.pathFile name of the current suite including directory
suite.nameGet the name of the current test suite.
testcase.name The name of the current 'Test case', empty if no 'Test case' is currently being executed.
testcase.id The QF-Test ID of the current 'Test case', empty if no 'Test case' is currently being executed.
testcase.qname The qualified name of the current 'Test case', including the names of its parent 'Test sets'. Empty if no 'Test case' is currently being executed.
testcase.reportname The expanded report name of the current 'Test case', empty if no 'Test case' is currently being executed.
testcase.splitlogname The qualified name of the current 'Test case' converted to a filename, including the names of its parent 'Test sets' as directories. Empty if no 'Test case' is currently being executed.
testset.name The name of the current 'Test set', empty if no 'Test set' is currently being executed.
testset.id The QF-Test ID of the current 'Test set', empty if no 'Test set' is currently being executed.
testset.qname The qualified name of the current 'Test set', including the names of its parent 'Test sets'. Empty if no 'Test set' is currently being executed.
testset.reportname The expanded report name of the current 'Test set', empty if no 'Test set' is currently being executed.
testset.splitlogname The qualified name of the current 'Test set' converted to a filename, including the names of its parent 'Test sets' as directories. Empty if no 'Test set' is currently being executed.
teststep.name The name of the current 'Test step', empty if no 'Test step' is currently being executed.
teststep.qname The qualified name of the current 'Test step', including the names of its parent 'Test steps', but not including 'Test cases' or 'Test sets'. Empty if no 'Test step' is currently being executed.
teststep.reportname The expanded report name of the current 'Test step', empty if no 'Test step' is currently being executed.
thread The index of the current thread. Always 0 except if QF-Test is started with the argument -threads <number>.
threads The number of parallel threads. Always 1 except if QF-Test is started with the argument -threads <number>.
versionQF-Test version
version.buildQF-Test build number
windows"true" under Windows, "false" otherwise
Table 6.1:   Definitions in the special group qftest

Expressions

The $[...] syntax for mathematical expressions supports operations like +, -, *, / and % (modulo) as well as braces. It comes in handy when calculating coordinates, e.g. in a loop.

In fact these expressions are far more powerful, since they are evaluated by the Jython interpreter. You can use arbitrary constructs that are legal syntax for the Jython method eval. See chapter 11 for details about Jython scripting.

Note Accessing QF-Test variables in an expression follows the same rules as in Jython scripts (see subsection 11.2.3). You can use the standard QF-Test syntax $(...) and ${...:...} for numeric or Boolean values. String values should be accessed with rc.lookup(...).

3.0+6.7
Immediate and lazy binding

There is a very subtle issue in using QF-Test variables that requires further explanation:

When a new set of variable bindings is pushed on one of the variable stacks, there are two possibilities for handling variable references in the value of a binding, for example when the variable named 'x' is bound to the value '$(y)'. The value '$(y)' can be stored literally, in which case it will be expanded some time in the future when '$(x)' is referenced somewhere, or it can be expanded immediately, so that the value of the variable 'y' is bound instead. The first approach is called 'lazy' or 'late binding', the second approach 'immediate binding'.

The difference, of course, is the time and thus the context in which a variable is expanded. In most cases there is no difference at all, but there are situations where it is essential to use either lazy or immediate binding. Consider the following two examples:

A utility test suite contains a procedure for starting the SUT with different JDK versions. The variable 'jdk' is passed as a parameter to this procedure. For ease of use, the author of the test suite defines some additional useful variables at test suite level, for example a variable for the java executable named 'javabin' with the value '/opt/java/$(jdk)/bin/java'. At the time 'javabin' is bound in the test suite variables, 'jdk' may be undefined, so immediate binding would cause an exception. But even if 'jdk' were bound to some value, immediate binding would not have the desired effect, because the java executable is supposed to be the one from the JDK defined later by passing the parameter 'jdk' to a procedure. Thus lazy binding is the method of choice here.

Imagine another utility test suite with a procedure to copy a file. Two parameters called 'source' and 'dest' specify the source file and destination directory. The caller of the procedure wants to copy a file called 'data.csv' from the same directory as the calling test suite to some other place. The natural idea is to bind the variable 'source' to the value '${qftest:suite.dir}/data.csv' in the procedure call. With immediate binding, '${qftest:suite.dir}' will indeed expand to the directory in which the calling suite resides. However, if lazy binding were used, the actual expansion would take place inside the procedure. In that case, '${qftest:suite.dir}' would expand to the directory of the utility suite, which most likely is not what the caller intended.

In versions of QF-Test up to and including 2.2 all variable expansion was lazy. As the examples above show, both variants are sometimes necessary. Since immediate binding is more intuitive it is now the default. This can be changed with the option When binding variables, expand values immediately. The option Fall back to lazy binding if immediate binding fails complements this and helps to ease migration of old test suites to the use of Immediate Binding. The warnings issued in this context help locating the few spots where you should use explicit lazy binding as described below. Except for very rare cases where lazy binding is required but immediate binding also works so that the fallback is not triggered, all tests should work out of the box.

In the few cases where it makes a difference whether a variable is expanded immediately or lazily, the expansion of choice can be selected individually, independent of the setting of the above option, by using an alternative variable syntax. For immediate binding use '$!' instead of just '$'. Lazy binding is selected with '$_'. For example, to define a variable at test suite level that specifies a file located in this test suite's directory, use '$!{qftest:suite.dir}/somefile'. If immediate binding is the default and you require lazy binding as in the 'jdk' example above, use '$_(jdk)'.

Note With lazy binding the order of variable or parameter definitions in a node or a data driver did not matter because nothing was expanded during the binding stage. With immediate bindings, variables are expanded top-to-bottom or, in a data driver, left-to-right. This means that if you define x=1 and y=$(x) it will work, with y being set to 1, if x is defined first. If y is defined first the definition will either fail or trigger the lazy definition fallback described above.