Logo QF-Test

Completely documented in two languages.
Manual = The QF-Test reference work

 

Free Trial  Download  Buy

Uwe Klüh, Senior Sales Manager, QFS

Find the information you need in the whole documentation (Manual, Tutorial, Mailing list, Standard library) by using the online search above.

Uwe Klüh, Sr. Sales Manager, QFS

Help is given in the software itself, too.

You can access the whole documentation directly from within QF-Test.

Manual

16
Data-driven testing

Data-driven testing is a very important aspect of test automation. In short, the goal is to run a given test or set of tests multiple times with different sets of input data and expected results. QF-Test has various means to store data or load external data for use in data-driven tests. The most convenient is based on a 'Data driver' node which sets up an environment for iterating over the sets of data combined with one or more 'Data binder' nodes to bind the variables for test execution. Note that there is no 'Data binder' node as such. The name serves as a generic term for the specific nodes like a 'Data table' or a 'CSV data file'. This is best explained through some examples. A demo test-suite with simple and advanced examples named datadriver.qft is provided in the directory doc/tutorial below QF-Test's root directory. Please take care to store modified test-suites in a project-related folder.

16.1
Data-driver examples
A simple data-driven test
Figure 16.1:  A simple data-driven test

The image above shows a 'Test-set' with a 'Data driver' node that contains a single 'Data binder' in the form of a 'Data table' node. The contents of the 'Data table' are as follows:

Data table example
Figure 16.2:  'Data table' example

When the 'Test-set' is executed, it will iterate over the rows of the 'Data table' shown above. For each of the three iterations the values of the respective row in the table are bound to the variable named in the matching column header. Thus, during the first iteration the variable named "Model" is bound to "Rolo", "Variant" to "None" and "Price" to "19000". During the second iteration, "Model" is set to "I5" and to "Minigolf" during the third and last iteration. For each iteration, all 'Test-case' child nodes of the 'Test-set' are run.

The following image shows a run-log for the above 'Test-set'.

Run-log of a data-driven test
Figure 16.3:  Run-log of a data-driven test

The next example shows that data-driven testing is not limited to a single loop:

Data-driven test with nested loops
Figure 16.4:  Data-driven test with nested loops

The 'Data driver' now contains a second 'Data table' node with the following contents:

Second data table example
Figure 16.5:  Second data table example

The 'Test-set' will now go through a total of six iterations because for each of the three iterations of the outer loop "cars", both iterations of the inner loop "accessories" will be run as shown in the following run-log.

Run-log of a data-driven test with nested loops
Figure 16.6:  Run-log of a data-driven test with nested loops

Note The extremely useful dynamic names of the loop nodes in the run-logs are obtained by setting the attribute 'Name for loop pass in the run-log' to the value "car Model: $(Model)"in the first and to "car Model: $(Model), accessory Name: $(Accessory)" in the second example. As you can see, that name is expanded individually for each iteration, so you can make use of the variables bound for that iteration.

16.2
General use of 'Data drivers'

As seen in the example above the 'Data driver' node must be placed in a 'Test-set' node, between the optional 'Dependency' and 'Setup' nodes. When the 'Test-set' is executed it will check for 'Data driver' and run it. The contents of the 'Data driver' node are not limited to 'Data binders'. Like a normal 'Sequence' the 'Data driver' node can hold any executable node to be able to perform any setup that may be required to retrieve the data. Thus it is also possible to share 'Data binders' by putting them inside a 'Procedure' and calling the 'Procedure' from inside the 'Data driver'.

Conceptually, a 'Data binder' represents a loop where a different set of variables is bound for each iteration. A 'Data binder' must be registered with a name in the 'Data driver' context of a 'Test-set'. This ensures that the loop can be interrupted by a 'Break' node with the same name. Once the 'Test-set' has run the 'Data driver' node, it will iterate over the registered data loops and perform the tests.

In case of nested loops the 'Data binder' that was registered first represents the outermost loop. Its variables are bound first and have lesser precedence than the variables from the inner loop(s).

16.3
Examples for 'Data drivers'

We provide a couple of examples for reading CSV or Excel files in the test-suite doc/tutorial/datadriver.qft.

16.4
Advanced use

Besides the 'Data table' node there are various other means for binding data in a data driver. The 'Excel data file', 'CSV data file', 'Database' and 'Data loop' nodes are all explained in detail in section 34.4.

It is also possible to bind data by calling the 'Procedures' qfs.databinder.bindList or qfs.databinder.bindSets in the standard library qfs.qft. These take as parameters strings with lists or sets of values to split and iterate over. Please see tutorial chapter 8 for information about the standard library.

And finally, data can be bound directly from Jython (and analogous from Groovy and JavaScript) with the help of the databinder module, which offers the following methods:

 
 
bindList(Object rc, String loopname, String varname, Object values, String separator=None, String counter=None, String intervals=None)
Create and register a databinder that binds a list of values to a variable.
Parameters
rcThe current run-context.
loopname The name under which to bind the data, equivalent to the 'Name' attribute of a 'Data binder' node.
varnameThe name of the variable to bind to.
values The values to bind. Either a sequence type or a string to split.
separator Optional separator character to split the values at in case they're a string. Default is whitespace.
counter An optional variable name for the iteration counter.
intervals Optional ranges of indices, separated by comma, e.g. "0,2-3".
 
bindSets(Object rc, String loopname, Object varnames, Object values, String separator=None, String counter=None, String intervals=None)
Create and register a databinder that binds a list of value-set to a set of variables.
Parameters
rcThe current run-context.
loopname The name under which to bind the data, equivalent to the 'Name' attribute of a 'Data binder' node.
varnames The names of the variables to bind to. Either a sequence type or a string to split.
values The value-sets to bind. Either a sequence of sequences - each inner sequence being one set of data to bind - or a string to split.
separator Optional separator character to split the varnames and the values of a value-set at in case they're a string. Default is whitespace. Value-sets are separated by line-breaks.
counter An optional variable name for the iteration counter.
intervals Optional ranges of indices, separated by comma, e.g. "0,2-3".
 
bindDict(Object rc, String loopname, dictionary dict, String counter=None, String intervals=None)
Create and register a databinder that binds data from a dictionary. The keys of the dictionary are the names of the variables and the values are sequences of values to be bound.
Parameters
rcThe current run-context.
loopname The name under which to bind the data, equivalent to the 'Name' attribute of a 'Data binder' node.
dictThe dictionary to bind.
counter An optional variable name for the iteration counter.
intervals Optional ranges of indices, separated by comma, e.g. "0,2-3".
 
 

Some examples:

import databinder

# Three iterations with the values "spam", "bacon" and "eggs"
# bound to the variable named "ingredient"
databinder.bindList(rc, "meal", "ingredient", ["spam", "bacon", "eggs"])
# Same with string values
databinder.bindList(rc, "meal", "ingredient", "spam bacon eggs")
# Same with string values and special separator
databinder.bindList(rc, "meal", "ingredient", "spam|bacon|eggs", "|")

# Two iterations, the first with item="apple" and number="5",
# the second with item="orange" and number="3"
databinder.bindSets(rc, "fruit", ["item", "number"],
                    [["apple",5], ["orange",3]])
# Same with string values, note the linebreak
databinder.bindSets(rc, "fruit", "item number", """apple 5
orange 3""")

# Same as before with the data stored in a dict
databinder.bindDict(rc, "fruit",
                    {"item": ["apple", "orange"],
                     "number": [5,3]})
Example 16.1:  Examples for use of the databinder module
Videos Downloads Documentation Buy Free Trial