Capture and replay

Once the SUT is up and running under QF-Test, the next step is to record sequences of events and play them back.

Video The video 'Capture and Replay' explains capturing and replaying of sequences.

The video 'Creating a test case' shows how to create a test case.

Recording sequences

In order to record a sequence of events in the SUT, the SUT must have been run from QF-Test (see chapter 3) and the connection between QF-Test and the SUT must be established. A visual indicator of this is the color of the record button which turns red when it is enabled.

Disabled Record buttonEnabled Record button
Figure 4.1:  Disabled and enabled Record button

To record a sequence, simply start recording be pressing the record button Record or selecting »Record«-»Start«. Then switch to the SUT, execute a few commands, switch back to QF-Test and stop the recording with the stop button Stop or »Record«-»Stop«. The recorded events will be added to the test suite, either directly at the position indicated by the insertion marker or as a new 'Sequence' under the 'Extras' node, depending on the setting of the Insert recording at current selection option. Pause the recording with »Record«-»Pause« or the pause button Pause recording if you need to execute some steps in the SUT that should not be recorded and you don't want to stop and restart the recording.

Recording mode can be started and stopped directly in the SUT by use of the Hotkey for recording. Default key is [F11].

Any 'Components' referred to by the newly recorded events are added automatically to the 'Windows and components' node if they are not there already.

There are many options that influence the way QF-Test records events and how it treats the components of the GUI. All of these are explained in detail in section 39.2 of the reference manual. Once you are familiar with QF-Test you should take the time to skim through it.

Here's some general advice for recording:

  • Record short sequences at a time.
  • After recording, take a look at the sequence, try to understand what you got and whether it represents the actions you took.
  • Edit the sequence to remove unnecessary events, especially those at the beginning and end caused by switching windows. QF-Test has excellent filters that should catch nearly all of these, but some might remain and have to be removed manually.
  • Finally, try out the new sequence to see whether it replays OK. Then you can cut/copy/paste as needed to integrate it into larger parts.

MacFor SUT's running on macOS QF-Test disables use of the screen menu bar and activates normal menu bar behavior like on other platforms. This is due to the fact that QF-Test cannot fully access the screen menu bar which prevents proper capture/replay of menu actions. In case the typical Mac screen menu bar behavior is necessary for any reason, this can be forced by adding the line to the file that is located in QF-Test's root directory. After restarting the SUT the screen menu bar is supposed to work as normal on Mac.

Running tests

To run some tests, select the node or nodes to execute and press [Return] or the play button Play or select »Run«-»Start«. QF-Test will mark each node with a small arrow as it is executed and also show progress messages in the status bar. This can slow down execution a little and can be turned off with the options Mark nodes during replay and Show replay messages in status line.

When the test is finished, the result is shown in the status bar. If things are fine you should see "No errors", otherwise the number of warnings, errors and exceptions is shown. Additionally, a message dialog is shown in case of errors or exceptions to make sure you don't miss these.

As for recording there are many options that influence the replay of tests. Some of these are only for convenience while others have a major impact on the outcome of the tests. Be sure to read section 39.3 some time to familiarize yourself with these.

To abort execution before the test is finished, press the stop button Stop or select »Run«-»Stop«. You can also suspend execution temporarily via the pause button Pause or by selecting »Run«-»Pause«. This will also enable the debugger (see chapter 7). To continue, press pause again.

While a test is run at full speed it can be tricky to stop or interrupt it, especially when the mouse cursor is actually moved across the screen or the SUT's windows are raised on every event. To regain control, press the Hotkey for pausing test run ("Don't Panic" key) (the default is [Alt-F12]). This will pause all running tests immediately. To continue, press the same combination again.

While building a test suite you will often want to execute some sequences to get the SUT to a point where you can continue recording. Sometimes you may want to skip certain nodes at this stage because they don't get you where you want, but you don't want to delete them or move them to some other place. In that case use the »Edit«-»Toggle disabled state« menu item to disable the node(s). When you want to use them again later you can re-enable them.

The current error state during replay as well as the final result is shown in the status line at the bottom of the QF-Test main window. The visibility of this status line can be controlled via »View«-»Show status line«.

In case 'Test set' or 'Test case' nodes (section 8.2 describes their usage) are executed the status line also contains relevant result counters from the following list.

Counter IconDescription
Total number of test cases. This counter value starts with a '>' symbol in case there are skipped test sets.
Number of test cases with exceptions.
Number of test cases with errors.
Number of test cases with expected errors. 'Expected to fail if...' marks a test case expected to fail.
Number of successful test cases.
Number of skipped test cases. A test case is skipped when its (optional) 'Condition' fails. This counter value starts with a '>' symbol in case there are skipped test sets.
Number of skipped test sets. A test set is skipped when its (optional) 'Condition' fails.
Number of not implemented test cases. A test case is not implemented when it doesn't contain nodes that were executed during the test run.
Number of executed test cases.
Percent test cases passed.
Table 4.1:  Test result counter in the status line

The final test result counts also appear in the report which can be created for any test run. Reports are discussed in chapter 22.

NoteThe counter values above can also be accessed as variables during the test run. A TestRunListener can help to keep track of counter values and trigger dependent actions.

Recording checks

Though it can be quite entertaining to record sequences and watch the magic dance of the SUT as they are played back, the task at hand is to find out whether the SUT actually works as expected. This is where checks come into play. The most common check, the 'Check text' node, reads the text displayed by a component, e.g. a text field, and compares it to a given value. If the values differ, an error is signaled.

How checks work is explained in detail in the Reference manual. There is a range available from simple text check to advance 'Check image' and even custom check types can be implemented. Here we are going to concentrate on the most convenient way to create checks, which is to record them.

While recording, the SUT is in record mode, which means that all events are collected and sent to QF-Test. With the help of the check button Check or by selecting »Record«-»Check« you can bring it into check mode, recognizable through the different mouse cursor. In this mode, recording events is suspended. Instead, the mouse cursor is tracked and the component under it is highlighted. When you click on the component, a check for the component is recorded using the value that is currently displayed. To get back to record mode, select the check button or menu item again.

There are different kinds of checks that can be performed. Which kinds of checks are applicable depends on the selected component. Some components don't display any text, so a 'Check text' node doesn't make sense for, say, a scroll bar. Clicking on a component with the right mouse button while in check mode brings up a menu of applicable checks for this component. Select one of the items to create the respective check node. Clicking with the left mouse button always records the default check, which is the topmost one in the popup menu.

If you hold down the [Shift] or [Ctrl] key while clicking with the right mouse button, the check menu will stay open after making a selection. That way, you can easily record multiple kinds of checks for the same component.

Checks integrate well with events and you'll soon develop a recording style a'la click, click, type, click, check, click, click, check... Having to switch back and forth between QF-Test and the SUT every time you want to create a check can be a real pain. That is where the Hotkey for checks option comes into play. It defines a key which toggles the SUT between record mode and check mode. The default value is [F12], but if this key has some defined meaning for your application you can change it to whatever you like. To record a sequence of interspersed events and checks, simply start recording, switch to the SUT and record the sequence. Whenever you want to add some checks, just press [F12] (or whatever you defined), record the checks, then switch back to record mode by pressing [F12] again and continue. This way you can work with the SUT for the whole sequence and need to switch back to QF-Test only to stop the recording.

One word of warning should be repeated: Don't let this convenience tempt you into recording extremely long sequences. When something changes that causes such a sequence to fail you will be hard put to find out what went wrong and how to cope.

Recording components

As already described component information is automatically stored when recording events or checks. Nevertheless there are situations where capturing of just components proves useful.

To activate component recording mode you simply need to press the record components button Record         components or select »Record«-»Record components« from the menu. Then switch to the SUT window where you will notice that the component below the mouse cursor is now highlighted.

Clicking with the left mouse button on a component will record the single component while pushing the right button instead will pop up a menu with choices to record the nested components as well or all components in the window. Multiple components can be captured in this way. Then switch back to QF-Test and release the record components button or deactivate »Record«-»Record components«. Now the recorded component information is stored in shape of respective 'Component' nodes under the 'Windows and components' node.

Component recording mode can be alternatively controlled by a configurable hotkey. Default binding is [F11], the option that configures it is Hotkey for components.

Pressing [F11] (default setting) in the SUT window starts component recording. Further details can be found in the documentation for the option Hotkey for components.

Note Only one test suite at a time can receive the recorded components. If more than one test suite is open and each is shown in an individual window, i.e. workbench mode is deactivated, either the test suite in which the recording is stopped (toolbar button or menu) or - when using [F11], the test suite that can be selected via the menu item »Record«-»Suite is receiver for recording« will receive the components.

The component recording feature can also be used to quickly locate a component independent of whether it has been recorded before. When creating event or checks by hand or changing the target component, the 'QF-Test component ID' attribute needs to be specified. When a component is recorded, its 'QF-Test ID' is saved in the clipboard and can be pasted directly into the 'QF-Test component ID' field with [Ctrl-V]. You can also jump directly to the 'Component' node with [Shift-Ctrl-Backspace] or by choosing the »Edit«-»Select next node« menu item or clicking the respective toolbar button.

The context menu which appears when clicking with the right mouse button on a component also contains an entry »Show methods« which allows of some kind of component inspection (see section 5.12).

Components play a central role in the structure of a test suite which is explained further in chapter 5.

Recording of HTTP requests (GET/POST)

In order to record a (GET/POST) request sent by the SUT, the SUT must have been launched from QF-Test (see chapter 3) and the connection between QF-Test and the SUT must be established.

While recording, the SUT is in record mode, which means that all events are collected and sent to QF-Test. By selecting »Record«-»Record HTTP Requests« you can bring it into request recording mode. In contrast to Recording sequences all GET/POST-request send by the web browser are saved as http-request nodes in this special recording mode. To get back to record mode, select the menu item again.

In section Web options the ability to change the type of the recorded request is described. By default 'Browser HTTP request' is recorded. This Request type is likely used to automate large web form inputs, the use of separate input nodes will be avoided. The form data will be submitted within the browser, so that the response will be shown as well. At this point the test execution could be continued. In opposition to this the 'Server HTTP request' will be directly submitted through QF-Test without the need of a running browser. The response is also only available in QF-Test and a eventually running browser will not be affected.

All attributes of an accordingly recorded request node as well as the parametrization of requests are explained in detail in the HTTP Requests section of the reference part of this manual.