38

Test execution

This chapter gives some hints about how to implement your tests to get stable and reliable test execution.

38.1

Dependencies

The 'Dependencies' concept of QF-Test provides functionality to guarantee that all prerequisites for a test case are fulfilled before running it. It is also capable of reacting to unexpected behavior, e.g. closing an error dialog, which pops up and blocks your tests.

The concept is described in section 40.3 and a use case can be found in the tutorial in the chapter 'Dependencies'.

You should at least implement a 'Dependency' which is responsible for launching the SUT, containing a 'Setup' for launching, a 'Cleanup' for a normal exit and a 'Catch' to react on any unexpected behavior.

Note If you implement a 'Cleanup', try to close the SUT normally first and only if the SUT does not terminate correctly, kill it via 'Stop client'.

For SWING and SWT applications please use the procedures qfs.cleanup.swing.closeAllModalDialogs and qfs.cleanup.swt.closeAllModalDialogsAndShells from the standard library qfs.qft for closing unexpected error dialogs.

38.2

Timeout vs. delay

Instead of using the 'Delay before' and 'Delay after' attributes you should try to use QF-Test's synchronization nodes to optimize test execution time.

The first kind of synchronization nodes are the 'waiter' nodes like 'Wait for component to appear', 'Wait for client to connect', 'Wait for document to load' and 'Wait for process to terminate'. You can specify the 'Timeout' attribute to wait for a component, process or document. The 'Wait for component to appear' node even provides the functionality to wait for the absence of a component.

The second kind are the 'check' nodes which allow you to specify the 'Timeout' attribute as well. Those nodes can be used to continue the test when a GUI element of your SUT has reached a defined state.

38.3

What to do if the run log contains an error

If the test report contains an error message or exceptions, the following steps should be performed to find the source of that failure very fast:

1. Analyze the run log, especially the screenshots and any other messages.
2. If you cannot find the cause immediately, jump to the failing location in your test suite by typing [Ctrl-T] in the run log.
3. Set a breakpoint before or at the failing step.
4. Ensure that the debugger of QF-Test is enabled.
5. Run the failing test.
6. When QF-Test reaches the breakpoint and stops, open the debugger window and check the active variable bindings to wee whether they contain any wrong values.
7. Perhaps at that time you can also see the error immediately in your SUT.
8. If you cannot see any source of that error, run the failing step.
9. If you still encounter errors you might have to re-debug some steps executed before the failing step. Use the 'Continue execution from here' menu entry to jump to previous steps instead of rerunning the whole test again.

3.1+ Since QF-Test version 3.1 it is possible to mark nodes via the context menu item »Set mark« or setting bookmarks for specific nodes via the menu item »Add bookmark«. These features enable you to find important nodes very fast again.

If you encounter problems with component recognition, please see section 5.10 and section 5.3.

Last update: 5/31/2023 Copyright © 1999-2023 Quality First Software GmbH