Logo QF-Test

Tutorial as a practical
introduction to QF-Test

 

Free Trial  Download  Buy

Yann Spöri, QF-Test development & support

The QF-Test examples, demos and practical exercises give you a quick start.

Yann Spöri,
Software Engineer, QFS

Uwe Klüh, Senior Sales Manager, QFS

Search for the information you need in the whole documentation (Manual, Tutorial, Mailing list, Standard library) by using the onsite-search.

Uwe Klüh, Sr. Sales Manager, QFS

Tutorial

10
Working with a Sample Test-suite [30-45 min]
10.1
Starting QF-Test and Loading the Test-suite

After starting up QF-Test, you can immediately bring up our first example test-suite. You can locate the test-suite file by bringing up the open-file dialog window with the »File«-»Open« menu option. Select the directory qftest-4.1.6/doc/tutorial/web from your QF-Test installation. From this directory, select the file webdemo.qft. QF-Test will then load the indicated test-suite which should look as follows:

Figure 10.1:  The Test-suite webdemo.qft

The left part of the main window contains the tree structure that represents the test-suite. The right side displays the details of a selected node. (In case you don't see the details view, you can switch it on with the »View«-»Details« menu option.) At the bottom of the main window you'll see the terminal output area, which displays standard messages and communications in between your test-suite and the client application you are testing. Displaying the terminal output can be toggled on or off using the »View«-»Terminal«-»Show« menu option.

With the help of the tree structure in the main window, you can navigate and select individual nodes of the test-suite. When a node is selected, you'll see its properties displayed in the details area on the right.

The top level of the tree consists of following node types:

  • "Test-suite" - This is the root node of every test-suite. All other node are subordinated to it.
  • "Test-set" - This node type allows collecting and structuring of test-cases.
  • "Procedures" - This node is meant to contain the procedures of the test-suite.
  • "Extras" - This node serves as a playground for experimenting and assembling of test-cases. Here you can insert arbitrary nodes independent of their usual restrictions.
  • "Windows and Components" - At this position QF-Test collects the information on the elements and structure of the document to be tested. Those is gathered automatically during recording and necessary for finding the proper components when replaying the tests.

Click on the node "Test-set: Web demo" to expand the nodes underneath it. You'll see that basically three test-cases are contained in this test-set: "Clickstream", "Text check" and "Selected test". Those test-case nodes are enclosed by a "Setup"/"Cleanup" pair to ensure a proper system as base to run the test-cases on.

Figure 10.2:  The Content of the Node "Test-set: Web demo"

In the following sections we'll describe the purpose and function of these individual nodes.

10.2
Starting the browser

Our first step will be to examine the "Setup" node. Expand this node now, so that you see the contents as shown in the following diagram:

Figure 10.3:  The "Setup: Start browser"

Within the "Setup" node you'll see four main sequences:

  1. "Set global variables" - define variables for the client name, the browser type and the optional browser directory.
  2. "Start browser without window" - start of the browser process to allow configuration
  3. "Perform browser settings" - configure the browser
  4. "Open browser window with URL" - open the browser window and wait for the URL to be loaded

This might look a bit circumstantial in the first place but leaves room for possible required adaptations later on in a convenient way.

At this point, we're ready to start the SUT. Click on the "Setup" node again (so that it is selected), but still expanded (the child-nodes still visible). Now click on the "Start test run" button Play. This button causes the selected node - "Setup" in our case - to be executed.

When a node executes, QF-Test will use the tree structure to show you what is happening. An active node will be marked with an arrow pointer "->" that indicates which node is being executed. You'll also see messages from the SUT client in the terminal display.

When the "Setup" sequence is completed, the browser window showing the demo page will appear on your screen. This is the SUT, fully under the control of the watchful eye of QF-Test.

Note If the browser window is not visible on your screen or only appears for a short moment, it is probably covered by the main window of QF-Test. The best solution is to place both windows side by side so you can watch them at the same time.

Figure 10.4:  The Web Demo Page in the Browser Window
10.3
Clickstream Test-case

Our next task is to take a first look at how a test-case for the SUT in QF-Test is structured. The test-case node labeled "Clickstream" contains a sequence of mouse-clicks to all menu items in the web site menu.

When you expand the "Clickstream" test-case node, you'll see the sequence as shown below:

Figure 10.5:  The Clickstream Test-case Node

Before executing the test please ensure the "Welcome" page is selected within the web demo. Then mark the "Clickstream" test-case node and click the replay button Play. The sequence is supposed to pass through without problems.

The test result is indicated during and after the test run in the status line at the bottom of the QF-Test main window and says "No errors". Next to it there are counters for the number and results of the executed test-cases. In our case it was just one that performed error-free which means a success rate of 100%.

Hint: Let the mouse cursor hover above a counter icon to see its description. A listing of all possible counters can be found in chapter Capture and replay of the manual.

For a better understanding of what you saw during replay, let's take a look at the sequence "Menu items" within the "Clickstream" test-case:

Figure 10.6:  The "Menu items" Test Sequence

Here we see that the sequence starts with a mouse click. Next to the "Mouse click" label you'll see the actual coordinates used to implement the mouse click relative to the component listed in the brackets next to the coordinates. The first mouse click node is, for example, a click on the second menu item "Text input".

After the first mouse click there is a "Wait for component" node that causes the execution of the sequence to wait for the appearance of the new web page. After the page is loaded successfully, execution of the sequence will continue with the next mouse click. It works the same way for selecting all remaining menu items until we are back at the "Welcome" page.

10.4
A Few Tips

We take a short break in this section to give a few tips that might prove helpful once you continue on with the tutorial.

While working with various nodes in QF-Test you may, for example, require some assistance in remembering what purpose they are used for. To serve this need, QF-Test comes with context-sensitive help, driven simply by your mouse. Move the mouse pointer over an element you would like to have assistance with, then click the right mouse-button. In the now appearing popup menu, you'll see an entry labeled "What's this?" By selecting this option from the popup menu, the appropriate section in the user's manual will be brought up and displayed in your standard browser. You'll hopefully find the manual quite helpful and complete in answering questions that may arise.

We should also mention our online search, available via the »Help«-»Online search... « menu option. It allows you to search all documentation at once and filter results afterward according to its source (manual, tutorial, etc.).

10.5
Text Check

One of the most important concepts in QF-Test is that of a check; that is, a query of certain elements in the SUT. A 'text check' queries the existence or absence of text within a component of the SUT, such as a text field.

For example, let's make a check of a field in the SUT client. In our web demo page please select the second entry in the menu bar which is "Input text". Within the respective page you'll see a text field labeled "Name:" For our example, we'll do a check of this label text.

Figure 10.7:  The "Name:" Label to Check

Now return to your test-suite and expand the "Text check" test-case node. Within you will see the following test sequence:

Figure 10.8:  The Text Check Test-case

It consists of one click, aiming to open the "Input text" page and a "Check text" node for the label text.

First return to the "Welcome" page within the web demo. Then with the "Text check" test-case node selected, click on the replay button to execute it. When the sequence completes, a dialog-box will appear indicating that one error and one warning occurred:

Figure 10.9:  Error in String Check Node

What happened? Whenever such incidents occur, taking a look at the QF-Test run-log can prove very useful for diagnosis. You can open it directly via the "Open run-log" button on the error dialog. Alternatively the latest run-log can be opened from the test-suite via the »Run«-»1. ...« menu item or by pressing [Control-L]. A new window will then appear displaying the recorded contents of the sequence you just executed:

Figure 10.10:  Run-Log for the Text Check Sequence

The run-log is similar in structure to the tree structure you are already familiar with from the normal test-suite view. The tree on the left side represents the time-recorded events of the test run, with the first event at the top, the last at the bottom. When you click on one of the event nodes on the left side, the properties of the event will be displayed on the right.

As you expand the nodes in the tree structure on the left, you'll notice immediately that some nodes are surrounded by a suspicious-looking red box. As you may suspect, this is an indication by QF-Test locating where a problem occurred in a child-node. If you keep expanding the red nodes, you'll eventually come to the actual problem node. An easier way to find the error is to simply use the »Edit«-»Find next error« menu option of the run-log's menu. This leads to the following:

Figure 10.11:  Diagnosis of Error in the Text Check Sequence.

When you click on the final red node, you'll see in the details of this node the deviation between the expected and found label text. The difference is actually a missing ":" at the end. Of course we have intentionally introduced this error into the test for demonstration.

The typical next step is to jump to the corresponding node in the test-suite and correct it. Provided that the appropriate node in the run-log is selected, you can use »Edit«-»Find node in test-suite« or simply type [Control-T]. One step beyond goes a functionality provided by the run-log node titled "Failed: Check text" in figure 10.11 available from the context menu, to be opened via a right click - »Update check node with current data« or [Control-U].

Another useful feature for error diagnosis is the tree node labeled "screenshot" in the run-log. Its details contain a full screenshot taken at the time when the error happened. Being able to see the state of the SUT at that moment is a valuable aid in determining the cause of the error. The following image shows a screenshot node:

Figure 10.12:  Screenshot node showing the error situation

Beside a screenshot of the whole screen, QF-Test also saves independent images of all SUT windows. For our example it is the protocol node "Screenshot of window: Web demo". Such can be helpful when the SUT is hidden by another application window at the time when the error occurs.

Note The information gathered from a long test-run accumulates and can devour enormous amounts of memory. Therefore QF-Test is typically configured to create a special form of compact run-log. Herein only the last 100 run-log nodes are kept, the rest is discarded, except for information relevant for report generation and for error diagnosis which is always retained. This functionality is configurable through the option "Create compact run-log" within »Edit«-»Options «-»Run-logs«-»Content«. The type of a run-log is also shown in its root node. The number of screenshots stored in the run-log can be configured as well.

Now a remark about the warning which occurred beside the error during the test-run. In the run-log of figure 10.11 it is marked by a yellow frame around the respective node. It's the one just above the erroneous check text node. The warning says: "Component 'textLast_name:' has no name". For the moment we don't want to track this warning any further. But naming of components is an important topic concerning test stability.

As a simple exercise, you may wish to modify the expected text in the "String check" node of your test-suite. Now when you execute the sequence, no error will occur. Please reverse the text to its previous state by »Edit«-»Undo« or [Control-Z] once the test has finished, because we will need the error for demonstration purpose later on.

10.6
Checking a Radio Button

The third test sequence "Selected test" performs queries about the state of a specific radio button in the SUT in order to determine if that radio button is selected. On the "Radio buttons & check boxes" page you can interactively select an number radio buttons. We'll use these radio buttons to implement a test that includes a successful check as well as a check that leads to an error.

Figure 10.13:  The "Selected test" Node

The node "Selected test" contains one mouse click, two checks and a selection in the middle. The click selects the "Radio buttons & check boxes" page. Afterwards, you see a "Check boolean: selected" node that queries the selected state of the first radio button to determine that it is selected. The "Selection" sets the combo box at the top of the page to the "not selected" option. Finally, another "Check boolean: selected" is performed just as before. You'll see that this last check leads to an error.

Figure 10.14:  Radio Buttons in the Web Demo

Try running the "Selected test" sequence now so that you can see the actual execution of the description we just provided. Don't forget to return to the "Welcome" page first.

10.7
Stopping the Browser

The function of the "Cleanup" sequence is important to understand. Inside a test-set a "Cleanup" node will be executed after every test-case located in the same level of the tree-structure, just as the "Setup" sequence will be executed before each test. The "Cleanup" node serves the purpose of leaving the SUT in a specific or 'clean' state so that further tests can be executed deterministically. This step is equivalent to the "tear-down" method within unit testing.

Expand the "Cleanup: Stop browser" node now to take a look at its child-nodes. For this cleanup we choose the final way and simply terminate the browser.

Figure 10.15:  The Cleanup Sequence "Stop Browser"

The "Stop client" node is followed by a "Wait for process to terminate" node to ensure the next test will wait until the browser is fully terminated.

10.8
The Complete Test

In the previous sections we've taken you step-by-step through many elements of our example test-suite. At this point, you're now ready to execute the entire test in one fell swoop.

First, close the browser in case it is still running. Then select the "Test-set: Web demo" node and execute it with the replay button. The entire set of tests will last about one minute, due to the delays we built into certain nodes of the test-suite so that you can better follow what's going on. If you look in the details of the "Test-set: Web demo" node, you'll see a variable called "delay" defined in the parameter default values section under "Defaults". You can change this value if you wish to shorten or lengthen the total run time.

When the tests complete, you should end up with two failures, both of which we discussed in the previous sections. Open up the run-log again to take a closer look:

Figure 10.16:  Run-Log for the Completed Test-set "Web demo"

Here you'll see in a visible fashion the process we attempted to explain in the last section concerning the behavior of the test-set node in which execution of the "Setup" and "Cleanup" nodes take place before and after each test. Thus a clearly defined "clean" state is ensured.

10.9
Report Generation

In the world of quality assurance the documentation and archiving of test results are of vital importance. In order to achieve this, QF-Test offers an automated report-generation feature. Since you've just finished a complete test run, we're at a good point to show you the capabilities of this feature.

Make sure the run-log for your test-run is open. Now use the »File«-»Create HTML/XML report...« option from the run-log's menu to bring up the dialog window used to specify the nature of your desired report.

Figure 10.17:  Report Generation Properties

In the first field, you can specify the file-name of the report. Following this, you can decide what type of report you want. QF-Test offers two kinds of reports, HTML or XML format. An XML report is useful if you've written your own XSLT stylesheets to shape the report into the style you'd prefer.

There are further options to control handling of HTML and doctags used inside comment fields as well as viewing the report in your browser after creation. You can just leave those fields unchanged. Further details can be found in chapter Reports and test documentation of the user manual.

Let's try generating a simple HTML report from the results of the last test-run. After possibly adapting the fields we indicated above and confirming the generation with the "OK" button the report will then be generated and generally look as follows in your browser:

Figure 10.18:  An HTML Report

The report begins with a summary containing informational data from your system on the top left side, a legend describing the meaning of icons used in the report on the top right side and the overall test result below. In our case as result we see the two well-known errors again.

Following the summary, an error overview lists the errors with its location where they occurred (the test-case) and the message describing the error details. Part three is a test-set overview just containing the "Web demo" test-set, as our test-suite only contains that one. Finally all test-cases contained in the "Web demo" test-set are listed with details like description, result and execution time.

The report generation feature proves very useful for creating an overview over the test-run and a document for presentation and archiving purposes.

Videos Downloads Documentation Buy Free Trial