Logo QF-Test

Tutorial as a practical
introduction to QF-Test


Free Trial  Download  Buy

Yann Spöri, QF-Test development & support

The QF-Test examples, demos and practical exercises give you a quick start.

Yann Spöri,
Software Engineer, QFS

Uwe Klüh, Senior Sales Manager, QFS

Search for the information you need in the whole documentation (Manual, Tutorial, Mailing list, Standard library) by using the onsite-search.

Uwe Klüh, Sr. Sales Manager, QFS


Creating a Test-suite [45-60 min]

This chapter will take you through the steps of creating your first test-suite. The application to test will be the FileChooserDemo included in the Java software development kit. It is also included in the distribution of QF-Test together with the source-code, in the directory qftest-4.1.6/doc/tutorial/FileChooserDemo.

The FileChooserDemo application is a very nice demonstration of how you can customize the FileChooser class in Swing, to create your own "Open", "Save" and customized file dialog for your application. We will use it as the SUT (System Under Test) for the creation of your own test-suite.

This chapter of the tutorial has been designed to illustrate the following concepts:

  • Starting an SUT in QF-Test
  • Recording components and organizing simple tests
  • Creation of tests which check the dependencies of functions (business logic) within the SUT.
Starting the Application

Open a new (empty) test-suite from QF-Test with the »File«-»New Test-suite...« menu option. Please be aware the "old" test-suite remains open as a different tab.

Make sure that the detailed view is turned on, it can be toggled with the »View«-»Show Details« menu option. You now see the divided window with a tree on the left-hand side and a pane on the right-hand side representing the details of a selected node. Using the tree you can navigate through the test-suite's nodes; when you select a node, you'll see its properties appear within the details pane on the right.

To begin you need to start the application. As we mentioned, our application to test will be the FileChooserDemo from the Java software development kit. The QF-Test Quickstart Wizard will help us to create an appropriate setup sequence.

Please open the Quickstart Wizard via the »Extras« menu or by pressing the record button in case it still shows the question mark. The Wizard should welcome you as shown in the figure below. After saying a short hello please press the "Next" button.

Figure 2.1:  The Quickstart Wizard

Next you can choose the type of application to be tested. Please select "A Java archive (java -jar <archive>)" and continue.

Figure 2.2:  Select type of SUT

Now you are asked about the Java executable. You can leave the proposed value ${qftest:java} untouched which means the java installation QF-Test itself uses. Simply proceed to the next wizard step.

Figure 2.3:  Specification of the Java executable.

On the next pane you are asked for an optional working directory for the application. As such is not required for our demo you can simply proceed to the next wizard step.

Figure 2.4:  Working directory.

It's time now to select the Java archive to use. For our example it will be the FileChooserDemo.jar archive in the doc/tutorial/FileChooserDemo directory of your QF-Test installation. You can pick the file by using the "Select jar file" Select jar file button or directly type into the text field. Another option (which works both on Windows and Unix) is to fill in ${qftest:dir.version}/doc/tutorial/FileChooserDemo/FileChooserDemo.jar where the first part is a variable that will expand to the version specific installation directory of QF-Test.

Figure 2.5:  Jar file selection.

The next step of an optional SWT instrumentation can be simply skipped by "Next" as our FileChooserDemo is not based SWT but on Swing.

Figure 2.6:  SWT instrumentation

It's nearly done. We have just to assign a name to our client as a reference. Not quite unexpected let's call it "FileChooserDemo".

Figure 2.7:  Client name

Finally some information is given about what to expect when the wizard finishes its task of creating a setup sequence and where to find help in case of troubles. Please disable the "Start automatically" option, as we first want to take a look at the generated setup sequence and start it by hand. Now please press "Finish".

Figure 2.8:  Final Information

The generated setup sequence appears under the "Extras" and contains three steps:

  • Set the global client variable
  • Start the Java SUT
  • Wait for the client to connect
Figure 2.9:  Generated Setup Sequence

You can have a look at the details of the nodes if you like. Basically you will find the values you provided during the Wizard steps assigned to the respective node attributes.

Now we want to see some action. Please ensure the "Setup" node is selected. Then click the Play replay button or simply hit "Enter". You should see the FileChooserDemo application appear on your screen soon.

Figure 2.10:  The FileChooserDemo Window

If the FileChooserDemo fails to come up after a few seconds, take a look at the terminal output display, which typically shows helpful messages.

Adding a Clickstream Test

You're now ready to add a simple clickstream test. Press the "Start recording" button Record and switch to the application window of the SUT. From now on every mouse and keyboard action performed within the SUT window will be recorded. Click on some buttons and switch back to your QF-Test test-suite window. Press "Stop recording" Stop. You'll find the recorded sequence placed under "Extras" node on the left side of the main window, as seen below:

Figure 2.11:  The Recorded Clickstream

The name of the recorded sequence will be entered in a standard fashion by QF-Test as the time and date of the recording. You can change this name as you see fit by simply clicking on the node and changing its properties in the details view on the right.

Select your recorded sequence now and execute it using the Play button. You should now see your exact sequence of mouse and keyboard events replayed within the window of the SUT. (Of course you may need to restart the SUT to wipe the slate clean.)

This type of test can be quite spectacular and can be used for demonstrations of your application, but it mostly tests the underlying Swing implementation and not the business logic of your application. In the last section of this chapter we will implement tests that check the content of fields and also check for causal connections within the FileChooserDemo (business logic), but first we will structure the recorded sequences into a test-suite.

Building a Test-suite

The basic structure of a test-suite below the 'root' node is fixed through the following nodes:

  • An arbitrary number of "Test-set" and "Test-case" nodes to specify and structure functional tests.
  • "Procedures" contains reusable sequences that can be organized into modular functions.
  • "Extras" is the scratch pad for experimentation.
  • "Windows and components" contain the all-important elements of the test-suite: registered elements of your SUT such as windows, menus and buttons. Each element within the "Windows and components" section contains properties of the element so that QF-Test can find the component easily when replaying a test or sequence.

Functional test-cases are represented by "Test-case" nodes and can be grouped and structured with help of "Test-set" nodes. "Setup" and "Cleanup" nodes are intended to contain the steps to ensure a well-defined state before and after a test-case.

One possible implementation of the "Setup"/"Cleanup" pair is to start and stop the SUT application; this way the SUT is always in a known state when a test runs. This pairing can, however, be used at any level and for any purpose you see necessary, like opening and closing a dialog, for example.

Let's start by renaming our top-level test-set node from "unnamed" to "FileChooser Tests". A dialog is popping up asking whether to update references, which we can simply confirm with "Yes".

The second step is to move the "Setup" node generated by the Quickstart Wizard from the "Extras" node into the "Test-set" node - on the first position preceding to the "Test-case" node. Moving the "Setup" node can now be accomplished with mouse (Drag&Drop), context-menu (right mouse-button copy/paste) or with the [Control-X] and [Control-V] keyboard commands.

Note When using Drag&Drop a target node can be expanded by having the mouse cursor hovered above the "+" left of the target node.

You might want to assign a more specific description to the setup sequence like "Start FileChooserDemo".

Figure 2.12:  Start organizing the Test-suite

When this is done, add a "Wait for client to connect" node with the »Insert«-»Process nodes«-»Wait for client« menu option. This node causes QF-Test to wait for the SUT client getting started and ensure that you get a successful connection to the SUT.

We need one more thing before the "Setup" node is complete. After the SUT has connected to QF-Test it can take quite a while until the first window is shown on the screen. We must make sure that the actual test doesn't start before that so we will need to wait for that window to appear.

To that end insert a "Wait for component" node after the "Wait for client" node with the »Insert«-»Miscellaneous«-»Wait for component« menu option. If the SUT is running, the "Client" attribute should already be filled in with the name of the SUT client, "$(client)" in our case. To set the 'QF-Test component ID', click on the Select component button above the field to bring up a dialog with the available "Window" and "Component" nodes. Select the "Window" node labeled "JFrame frameFileChooserDemo".

Figure 2.13:  Component selection dialog

Now you can stop the FileChooserDemo and restart it by executing the "Setup" node. The result of your efforts should look like this:

Figure 2.14:  Setup sequence of the FileChooserDemo

The next step is to make a test-case from the previously recorded clickstream sequence. For this we have to move the sequence from the "Extras" into the unnamed test-case. You'll need to open the "Test-case" node before moving the "Sequence" node inside. Let's rename the test-case to 'Clickstream' and the sequence to 'Some clicks'.

Lastly you can create a "Cleanup" sequence to stop the application. This sequence contains two nodes: One to stop the client and one to ensure that it actually terminated. From the figure below, you should be able to see what nodes are necessary to carry out this little exercise. Your test-suite should now look like this:

Figure 2.15:  The Organization of your Test-suite

Now you've completed the most important steps of structuring a test-suite. In the following section you'll expand your test-suite by introducing a check of a specific text-field

Adding a Text-Check

To monitor the client's behavior we use check-nodes, which query certain states and properties of elements within the SUT. The first check that we'll introduce is a "Text-check," which verifies that a text-field contains the predicted string. Our check will be performed on a button within the FileChooser window.

If the SUT is not running, start it up now using your "Setup" sequence. Within the FileChooserDemo window, click the "Show FileChooser" button.

To begin recording of a check, click the "Record a check" Record a check button and switch to the FileChooser window. Now when you move your mouse over the components you will notice a blue border indicating the current selection. To record the check, move to the "Open" button, and right-click. You'll then see a popup menu appear which gives you the choice of possible standard checks for the button component, as seen below:

Figure 2.16:  Recording a Check in the FileChooser Window

Choose "Text" from the popup menu, return to your test-suite window and stop recording using the Stop button.

Note Instead of returning to the test-suite to activate the check mode by pressing the Record a check button, you can also utilize [F12] while remaining in the SUT. This hotkey activates/deactivates the record check mode.

Afterwards, record another arbitrary clickstream sequence using the Record button, and then go ahead and close the FileChooser window.

You should now have the know-how to organize the recorded sequences into a test-case. You can compare your results with the way we organized the suite and renamed the sequences. Additionally we transformed sequences into "Test-step" nodes which specify them more clearly and make them also visible in the test report.

Figure 2.17:  The Text-Check Test-case Organized into the Tree Structure

Note Transformation of nodes can be easily done via menu »Operations«»Transform node into« or also a nodes context menu.

At this point, you can give your new test a shakedown run. Stop the SUT client now if it is running, click on the 'FileChooser Tests' test-set node, and start execution by hitting the Play replay button.

The result of the test-run is stored in the run-log. To view it just make use of the error dialogs "Open run-log" button.

NoteFrom within the a test-suite the last run-log can be opened by use of the Open last run-log toolbar button or the »Run«-»1. ...« menu option with its short-cut [Control-L]

Figure 2.18:   The Run-Log of Your Test-set

Within the run-log you see that the "Setup"/"Cleanup" pair is executed before and after each test-case.

Note Starting and stopping the SUT before and after each test-case is a safe way for a proper state but a time consuming one. Typically you will try to start the SUT only once and use others methods for resetting it to a defined status.

In this test there were no errors or exceptions in the test-run. But you can see yellow frames around some nodes, which indicate warnings. In case of the FileChooser they are caused by components which have no name assigned. We will not dicuss those warnings here any further, but you will find detailed information on the importance of component names in the user manual.

Upon closing the run-log window you will be prompted if you want to save the run-log (since it will only be available until QF-Test ends).

Let's modify the test now so that we cause an error in the text-check. Click on your "Check-text" node so that you see its details displayed as follows:

Figure 2.19:  Properties of the "Check Text" Node

You see here that in the "Text" field a value has been entered ("Open") which represents the value that QF-Test expects. Change this text to some other arbitrary value and re-run the test.

Now when you replay the test, a dialog box appears at the end indicating that there were "0 Exceptions, 1 Error, 0 Warnings."

If you open the run-log, you'll see that red-fields now enclose certain nodes, indicating that some error occurred inside the child-nodes. You can use the run-log menu option »Edit«-»Find next error« to jump directly to the node which caused the error or use the [Control-N] keyboard shortcut.

We'll now go on by expanding your test-suite to include a test-case that checks the business logic of the SUT.

Checking Business Logic

Up to this point in the tutorial, we've primarily concentrated on the Swing implementation of windows and components within the SUT application. What we've not touched on is the idea of testing the actual business logic of the application, which will be our focus for this section.

To check the business logic we need a causal connection between some action of the user which leads to a reaction from the application. Within the FileChooserDemo, we've located one such feature which nicely demonstrates this type of connection in which a text field switches state from disabled to enabled upon clicking the "Custom" radio button (located in the left part of the window).

For our test, we've chosen to record a clickstream that activates the text field, then checks that the field really is enabled, and finally resets the application through a click on the "Open" button. This sequence is a true action-reaction test of the application since these two components are not connected in Swing. To just check the content of the text field is not a test of the logic but of the initial state of the whole application.

The steps for this test should be quite similar to those you did for the "Check text" test, with the clickstream sequence and the check recorded separately. We've provided our test-suite solution in the following image, to give you some assistance along the way:

Figure 2.20:  The Check Enabled Status Test-case

Here's a summary of what we did:

  • The first sequence clicks the "Custom" button, thus activating the text field.
  • The second checks that the text field is really active. Note here again that there is no direct connection in Swing between the button and the textfield, thus we are testing actual business logic of the application.
  • The third sequence resets the application by pressing the "Open" button.

With that we wrap up this chapter of the tutorial. In the next chapter, we'll concentrate more on error diagnosis by diving into the QF-Test debugger.

Videos Downloads Documentation Buy Free Trial