Logo QF-Test

Tutorial as a practical
introduction to QF-Test


Free Trial  Download  Buy

Yann Spöri, QF-Test development & support

The QF-Test examples, demos and practical exercises give you a quick start.

Yann Spöri,
Software Engineer, QFS

Uwe Klüh, Senior Sales Manager, QFS

Search for the information you need in the whole documentation (Manual, Tutorial, Mailing list, Standard library) by using the onsite-search.

Uwe Klüh, Sr. Sales Manager, QFS


Using the Debugger [30-45 min]

In this chapter we will learn how to run a test-suite with QF-Test's built-in and intuitive debugger. For those of you familiar with debugging programs in Java or other programming languages, you will find this debugger similar in function and usefulness.

For the debugger tutorial we will use a test-suite which you are most likely already familiar with, Options.qft. If this test-suite is not already running on your system, refer to section chapter 1 above to get it running.

Starting the Debugger

The QF-Test debugger can be started either by selecting a node (or nodes) to execute and pressing the step-in Step-In or step-over Step-Over buttons, or by using the »Debugger«-»Step In« or »Debugger«-»Step Over« menu items. The sections below describe the function of these operations in more detail .

Figure 3.1:   Select a node in the test-suite Options.qft and start the debugger.

Note that if you are running tests on your test-suite and use the play button to start execution, the debugger will normally not be seen. If you activate the debugger with the »Debugger«-»Enable debugger« menu item the debugger will be activated automatically if an uncaught exception is thrown. If a user-defined breakpoint is reached (q.v.) the debugger is activated unconditionally. In either case, execution of the test-suite will be stopped and the node which stopped execution will be indicated with an arrow.

The Debugger Window

The debugger is run either from within the normal test-suite view, or by opening the dedicated debugger window by selecting »Debugger«-»Show Debugger Window« once the debugger has been started.

You can also cause the debugger window to open automatically when the debugger starts by changing the global setting through the »Debugger«-»Options«-»Always Show Debugger Window« menu option.

The debugger window itself looks similar to the normal test-suite view which you are by now familiar with. It contains, however, only a subset of the QF-Test operations which are needed for debugging. The figure below shows the debugger window after starting the debugger for the Options.qft test-suite.

Figure 3.2:  The debugger window.
The Current Node versus the Selected Node

When the debugger is running, an indicator is used to mark the "current" node, which simply shows you where the debugger is while a test or sequence is being executed. Note that QF-Test marks the hierarchy of execution, from entry point down to the current node, with a small pointer as seen in figure 3.2.

If you cannot immediately see the current node at any time while the debugger is running, you can press the "Locate Current Node" Locate Current Node button or select the »Debugger«-»Locate Current Node« menu option to cause the debugger to "select" the current node.

A "selected" node is a node that is highlighted as when you click on it with your mouse. This can sometimes cause confusion, because the current node during debugging is often selected as well. Learn to recognize the distinction though!

Stepping Through a Test or Sequence

You are now ready to begin stepping through nodes of the test-suite to see the behavior of the debugger. With the current node set to "Sequence: Table" as in figure 3.2, press the step-in Step-In button now.

You will see that the debugger causes the current node to expand to reveal the child nodes beneath it, and that the first child node is now selected as the current node, as shown in the figure below:

Figure 3.3:  After a step-in operation.

Try pressing the step-in button a few more times. The debugger will execute each node, reset the current node to the next available node, and wait in a paused state for your next command.

Now after seeing the functionality of the step-in operation, press the "Continue test-run" Continue button to allow this particular sequence to run to completion.

After executing the first sequence of the test-suite, your debugger view should now show something similar to the figure below:

Figure 3.4:  Preparing for the step-over command.

If this "Sequence: Tab" node is not selected in your window as shown in the figure, then select it now. When you're ready, press the step-over Step-Over button.

This time, the debugger reacts by starting execution of sequence, but unlike the step-in operation, you will not see the individual steps. While the sequence is running, you can expand the "Sequence: Tab" node to see that there several child nodes underneath it, indeed there are several other sequences located here.

The step-over operation is therefore a way for you to execute a node without having to concern yourself with the details of that node's individual steps.

When execution of the "Sequence: Tab" node has completed, the "Sequence: Text Check" node will now be selected. Press the step-in button, so that your debugger view shows the following:

Figure 3.5:  Preparing for the step-out command.

Now press the step-out Step-Out button. You will see that the debugger causes the rest of the nodes in the sequence to be executed and the next "Sequence: Selected test" node is now selected.

What does the step-out operation do? Very simply, it executes all nodes found at the same level in the hierarchy as the current node and stops when a node that is higher in the hierarchical structure is found. Any child nodes found during execution of "step-out" are, of course, executed as would be expected.

From this example, the functionality of step-out may look similar to that of the "Continue test-run" operation. This is partly true, but there is one distinct difference: The step-out operation stops at the next node that is higher in the hierarchy, whereas "Continue test-run" works until it finds a node that is at the same level as the entry point node where you began. If this sounds complicated, simply try playing around with the two operations within the test-suite. You'll quickly get the hang of it!

Skipping Execution of Nodes

The "skip" functions expand the QF-Test debugger's capabilities in a powerful way which is not typically possible for a debugger in a standard programming environment. In short, they allow you to jump over one or more nodes without having to execute them at all. This may be helpful due to different reasons, e.g. to navigate fastest to a certain position in your test run or skip a node which currently leads to an error.

Set up the test-suite again as you did in the beginning of the previous section, and press the step-in button again until the current node is inside of the sequence as seen in figure 3.3. Now press the skip-out Skip-Out button. You see immediately that QF-Test simply jumped out of the sequence in which you were located without executing the remaining nodes in that sequence!

Your test-suite should now look like the image shown now in figure 3.4. Leaving the set-up as is, now press the skip-over Skip-Over button. Here we see a similar behavior as with skip-out, QF-Test jumped over the selected node without executing any child nodes within it. (If you expand the node which you just skipped over, you see there are indeed several other sequences, none of which were executed).

One note about skip-over and skip-out: use them cautiously. Skipping out of a sequence before its completion could cause your SUT to be left in an unknown state that other sequences or tests in your test-suite cannot react to.

Setting Breakpoints

Setting a breakpoint is a deterministic way of allowing your test-suite to execute until it reaches the node you specify breakpoint, at which time the debugger will be active and allow you to continue as you wish.

To set a breakpoint, simply click on a node and select the »Debugger«-»Breakpoint on/off« menu item, or alternatively use the [Control-F8] shortcut. The breakpoint is indicated with a (B) before the name of the node, as shown in the following figure:

Figure 3.6:  Setting a breakpoint.

To see for yourself how a breakpoint works, select the top-level node of your test suite and press the replay button.

To turn off a breakpoint, simply click on the node where the breakpoint was set, and select the »Debugger«-»Breakpoint on/off« menu option or [Control-F8] shortcut again. The menu option »Debugger«-»Clear all breakpoints« is also useful to remove all breakpoints set in your test-suite.

There is no limit to the number of breakpoints you can set in your test-suite, but note that breakpoints are not saved with your session. So when you close the test-suite and open it again, the breakpoints will be gone.

Resolving Run-Time Problems

As you begin to create and run your own test-suites, you might face a problem like an unknown 'QF-Test component ID' or procedure name. This is a common problem and is typically caused by an incorrectly entered 'QF-Test ID' or name, or perhaps a change in the name or 'QF-Test ID' which you are referencing. During run-time, you will see the manifestation of this problem as a halt in the execution of your test-suite, with an appropriate message from QF-Test relating to the unknown element.

This gives us a chance to demonstrate one of QF-Test's most powerful features in the debugger: the ability to correct such problems on the fly during run-time execution of your test-suite.

Let's first cause your test-suite to throw an exception when it runs into an unknown 'QF-Test component ID'. Take any node within the test-suite and edit the 'QF-Test component ID' field so that it is obviously wrong. For this example, we took the first node under the "Table" sequence we used in figure 3.6

Figure 3.7:  Select a node to modify.

and modified the 'QF-Test component ID' entry to look like this:

Figure 3.8:  The modified node.

To edit the 'QF-Test component ID' field, click on the node and then select the »Edit«-»Edit...« menu option, or simply edit the field in the Details window (»View«-»Show Details« if the Details window is not visible on your screen). Click OK in the Edit dialog or the Details window after making your change. At this point you will see a warning that the 'QF-Test component ID' you entered is not valid. That's good! This is QF-Test's way of attempting to find problems before they occur at run-time. However, for this example we want to force an incorrect reference, so just leave the modified entry as it is.

We can now let the modified test-suite run. Select the "Test: Clickstream" node and press the play button to start execution. The test run will quickly stop with an error message dialog box:

Figure 3.9:  No component for QF-Test ID.

Click OK to close the dialog box. You will now see that execution of your test-suite is paused with the modified node selected as the current node. The debugger at this point is simply waiting for your directions. Do you want to skip over the offending node, completely halt the execution of your test-suite to investigate the problem further, or modify the node on the fly and continue execution? In this case, we want the latter option.

So with the debugger still paused, you can now edit the 'QF-Test component ID' field again to change it back to its original state. If you've forgotten what the original QF-Test ID was, select the »Edit«-»Undo« menu option to restore it (assuming you've not made any other changes to the test-suite).

With the correct QF-Test ID restored, press the "Continue test-run" Continue button to allow execution to continue where it left off. You'll now see that the test-suite runs just like before!

Jump to Run Log

At times, tracking down errors in your test-suite is not so simple as in the example we just showed. The actual cause for some problems may lie farther back in a sequence of events which led to the eventual exception or error which you see in front of you.

Under such circumstances, the run-log can be of immense use for debugging purposes, as it shows in detail the steps QF-Test used to execute each node and how it resolved QF-Test component IDs, procedure names, variables or other elements.

Since the run-log can be tedious to search, especially if your test-suite became quite complex, the debugger provides you with a simple mechanism to jump immediately to the region of interest in the run-log when an error occurs.

We'll demonstrate this by using the same simple example we used in section 3.8, which caused an exception to be thrown due to an unknown QF-Test component ID. Repeat the steps from that section and execute the test-suite as before, except that now instead of correcting the 'QF-Test component ID' as we did before when the error occurred, we will first inspect the run log. To jump to the appropriate place in the run log, select the »Debugger«-»Jump to Run-Log« menu option, or use the [Control-J] shortcut. The run-log will now be opened, and the offending node will be automatically selected as shown below:

Figure 3.10:  Jump to run-log.

Take some time now to look at the various elements presented for you in the run-log. Although the example we are using now is quite readily understandable, becoming familiar with this powerful tool will help you greatly when your test-suites become more complicated.

When you're finished, you can continue with the example as before or simply stop the test run. However, remember not to save the changes to the test-suite if you did not correct the error.

Videos Downloads Documentation Buy Free Trial