To test your application, from now on referred to as System Under Test (SUT), with QF-Test the first thing that needs to be decided is how to start it from QF-Test. As described in the previous chapter, prerequisites differ depending on the kind of application.

QF-Test connects to AWT/Swing applications automatically unless you explicitly turn off the option Connect without JDK instrumentation (Swing) or use a non-standard JDK. In such a special case you must instrument the JDK first.

Note To instrument a JDK you need write permission for some of its directories, so you may have to run QF-Test with administrator privileges to perform this step. The accessibility interface and all the technical details of the modifications to the JDK are described in detail in chapter 33. This section concentrates on how to instrument or de-instrument JDKs and how QF-Test maintains information about which JDKs have been instrumented.

Having started QF-Test, please select the menu item »Extras«-»Manage JDK instrumentation...« to bring up the following dialog:

		Figure 4.1:  Dialog for JDK instrumentation

Note If you get a warning that the file used for storing information about JDK instrumentation is currently locked, this typically means that some other user working with the same QF-Test installation is currently instrumenting JDKs. In that case, try again a little bit later. If the lock doesn't go away even though nobody seems to be working on the instrumentation, you can ignore it.

The instrumentation dialog shows a list of JDKs and JREs which QF-Test "knows" about. Initially the list will only contain the JDK with which QF-Test itself is running. If your SUT will be started with a different JDK or JRE you need to tell QF-Test about it. To do so, use the "Search JDK/JRE" button to bring up the standard file selection dialog and choose a directory. QF-Test will search that directory and its direct or indirect sub-directories and add all JDKs it finds to the list.

If you don't know which JDK your SUT is going to run with, first instrument the one JDK that's always shown, then try if it works. If it doesn't work, the record button will remain deactivated when you start the SUT from QF-Test. In that case you can either ask your developers or search for the JDK or JRE yourself. On Windows, C:\Program Files\Java is the most likely candidate. On Unix systems locations vary. Possible candidates are /usr/lib/java, /usr/share/java, /usr/local/java, etc. It is also possible that your application comes with its own JRE so letting QF-Test search the directory where the SUT is installed may also turn up something.

The 'Type' displayed in the first column of the list is usually JDK or JRE, '---' denotes an invalid path. This type information is only given for completeness and you don't really need to concern yourself with it.

The value in the 'State' column is important. The values in this column can be any of the following:

Not instrumented

The JDK is in its initial, unmodified state.

Instrumentation current

The JDK has been instrumented and the installed jar file corresponds the current QF-Test version.

Instrumentation outdated

The JDK has been instrumented but the current QF-Test version has a newer version of the installed jar file, so the JDK should be instrumented again.

Instrumentation inconsistent

The JDK has been partly instrumented, either because something has gone wrong during (de)instrumentation or because somebody (or some other tool) has modified the JDK. You should instrument or de-instrument such a JDK as required.

Path invalid

Either the JDK no longer exists, in which case you can remove it safely from the list, or it belongs to some other machine as explained below, in which case you should ignore it.

To instrument, de-instrument or remove one or more JDKs from the list, select the entry or entries in the list and press the respective button. When done, save the list and close the dialog.

The list of JDKs QF-Test knows about is stored in the file qfconnect.properties in QF-Test's root directory. If QF-Test is installed on a network file system which is shared by multiple computers, possibly with different operating systems, this single file will be shared by all systems. As you can see in the image above, this is not a problem. When you bring up the dialog, QF-Test will check all the saved JDKs to see whether they exist on the current system. If not, they will be marked with the state "Path invalid" and can be safely ignored. In short, you can run the instrumentation first from a Windows system and instrument your Windows JDKs, then from a Unix system to instrument those JDKs and that information can safely coexist.

In case you don't have write permissions for this file, QF-Test will not be able to save the list of instrumented JDKs, but that shouldn't cause any trouble with the instrumentation itself.

Some special setup is required for testing SWT based applications with QF-Test/swt. Because SWT was not written with testability in mind, applications need to be run with a slightly modified SWT version to enable testing. In this version we have added the necessary hooks for event filtering and component tracking. The changes are transparent so that the behavior of an application is not changed, regardless of whether it is run inside or outside of QF-Test.

If you use QF-Test's Quickstart Wizard to create the start sequence for your SUT (see chapter 3), it will take care of SWT instrumentation as well. For those with an aversion to wizard dialogs, the manual way is described next.

The standard library qfs.qft, which is part of the QF-Test distribution and described in detail in the tutorial, contains a 'Procedure' with which to perform the SWT instrumentation. It is named setup and located in the 'Package' qfs.swt.instrument. Insert a 'Procedure call' node before the start node for your SUT in your setup sequence. Set its 'Procedure name' attribute to qfs.qft#qfs.swt.instrument.setup and in the 'Variable definitions' set the parameter sutdir to the installation directory of your application. The plugin parameter can be left empty except when you are testing an Eclipse/RCP application that does not follow the standard plugin directory layout. In that case you can specify the plugin file to instrument directly via the plugin parameter. That's all. In case you want to know what goes on behind the scenes, all manual steps are described further on in this section.

Supported architectures for SWT testing are 32 and 64 bit Windows and 32 and 64 bit Linux with Gtk. The instrumentation files are provided in directories called .../qftest-3.4.4/swt/$ARCH/$VERSION where $ARCH is either win32, win32-64, linux-gtk or linux-gtk-64 and $VERSION is one of the supported SWT versions.

First you need to determine whether your application is a standalone SWT application or is based on eclipse. To do so, simply take a look at the directory structure of your application. If you find a directory called plugins containing a file called org.eclipse.swt.win32.win32.x86_X.Y.Z.jar (on Windows) or org.eclipse.swt.gtk.linux.x86_X.Y.Z.jar (on Linux), with X.Y.Z representing a version number like 3.2.0, your application is based on eclipse. For a standalone SWT application you should find a file called swt.jar, typically inside a directory called lib.

Simply replace the SWT plugin jar with one instrumented by QF-Test. To create the instrumented plugin you must run the 'Procedure' qfs.qft#qfs.swt.instrument.setup described above once with your original plugin (or a copy thereof) specified in the plugin parameter. QF-Test will create a backup copy of the original jar named _org.eclipse.swt....jar.orig. Next copy the instrumented plugin to the plugin directory of your application.

Finally, start your application once from the command line with the -clean command line argument to have it rebuild its plugin cache, e.g.

eclipse -clean

Your application's binary name may be different from eclipse, but all eclipse based applications should support the -clean argument.

For standalone SWT applications, replace the swt.jar file with the one provided with QF-Test. You may want to create a backup of the original first.

LinuxOn Linux systems you must also copy the file called libqfswt-gtk-xxxx.so where xxxx is a version number. to the directory that contains the shared library for your application. This directory is typically called bin and can easily be identified by the other shared SWT libraries it contains which are all of the form libswt-...-xxxx.so.

Note If you don't know the version of the SWT libraries on which your standalone SWT application is based, search for the directory with the shared libraries as described above. On Windows systems these are of the form swt-...-win32-xxxx.dll. The first two digits of the xxxx version number represent the major and medium version for the respective libraries provided with QF-Test. For example, if the version number is 3139, the replacement libraries will be found in the directory .../qftest-3.4.4/swt/win32/3.1 for Windows or .../qftest-3.4.4/swt/linux-gtk/3.1 for Linux.

With the Quickstart Wizard QF-Test offers a uitility to guide you throught the steps of creating a start sequence for your SUT. Please refer to chapter 3 for more information about the Quickstart Wizard.

Nevertheless we also want to describe how to create a start sequence for your application manually. There are basically two ways to start a Java application as an SUT from QF-Test. The first one is based on the standard java ... command line with its two variants for either starting a class or a jar file. The alternative is running a script or executable file which will then start the Java program. Indirect methods like launching the SUT through ant also fall into this category, as do Java WebStart and applets run in the Java plugin of a web browser.

The following examples show some typical setups. To get more detailed information about the required attributes, please follow the respective links to the reference manual. The tutorial also includes a number of examples.

Independent of how the SUT is started, the respective node should typically be followed immediately by a 'Wait for client to connect' node with an identical 'Client' attribute. Again, see the reference manual for further details.

If your application is started through a script or a binary executable, create a 'Start SUT client' as follows:

		Figure 4.2:  Starting the SUT from a script or executable

• Create a 'Start SUT client' node.
• Assign a name to the client in the 'Client' attribute.
• Set the 'Executable' to the script or executable that starts your application. If the program is not located in a directory on the PATH, the full path is required.
• Set the 'Directory' attribute to the working directory for your application.
• One thing to watch out for in scripts is redirection of the standard output and error streams (e.g. >myapp.log) which you may want to remove so that the output of the SUT reaches QF-Test and is captured in the run-log. Similarly, the start command in Windows batch files causes the SUT to detach and keeps the output away from QF-Test.

Using the new connection mechanism, an application launched through Java WebStart can be started directly from QF-Test without the need to modify any JNLP files (so do not use »Extras«-»Create WebStart SUT client starter...«). Instead create a 'Start SUT client' node as follows:

		Figure 4.3:  Starting the SUT through Java WebStart

• Create a 'Start SUT client' node.
• Assign a name to the client in the 'Client' attribute.
• Set the 'Executable' attribute to the Java WebStart executable which is typically called javaws and located somewhere inside the JDK or JRE. You'll probably have to specify the full path.
• For Java WebStart the 'Directory' attribute typically is of no consequence except that Java WebStart is looking in that directory for a file named .javaws which can contain settings like debug levels.
• Create an entry in the 'Parameters' for the executable to specify the URL for the application's JNLP descriptor.

Using the new connection mechanism, applets can be tested directly in a web browser provided that a recent Java plugin (version 1.5 or above) is installed in the browser. Applets can also be tested in the applet viewer, the setup for which is described in section 35.3. To start the SUT as an applet in a browser, create a 'Start SUT client' node as follows:

		Figure 4.4:  Starting the SUT as an applet in a web browser

• Create a 'Start SUT client' node.
• Assign a name to the client in the 'Client' attribute.
• Set the 'Executable' attribute to the browser that you want to start, i.e. firefox, iexplore, mozilla, netscape, opera to name just a few. You may have to specify the full path.
• For starting of an applet, the 'Directory' attribute is of no consequence.
• Create an entry in the 'Parameters' for the executable to specify the URL for the applet's web page.

Windows Internet Explorer on Windows sometimes causes problems that prevent the applet from from connecting to QF-Test. This is related to a registry setting that determines whether Windows should start a new process for each Internet Explorer instance or whether to use one shared process. The latter doesn't work with QF-Test because it initializes the Java plugin only once during system startup.

Internet Explorer is supposed to set the default for this option during installation depending on available system memory. If more than 32MB are available, the "new process" setting is supposed to be active - which would mean practically every machine larger than a PDA - but unfortunately that is not always the case. To fix it, the registry key

HKEY_LOCAL_MACHINE\Software\Microsoft\Windows\CurrentVersion\Explorer\BrowseNewProcess

should contain a string binding named "BrowseNewProcess" with the value "yes". We provide a small registry file named ...\qftest\qftest-3.4.4\misc\HKLM_BrowseNewProcess.reg that you can simply double-click to change this setting. Note that you may need to reboot the machine after changing the setting.

This may not be sufficient if either you don't have the rights to change HKEY_LOCAL_MACHINE settings or if there is an additional similar setting with the value "no" in HKEY_CURRENT_USER that overrides this one, i.e. under the key

HKEY_CURRENT_USER\Software\Microsoft\Windows\CurrentVersion\Explorer\BrowseNewProcess

If the HKEY_LOCAL_MACHINE setting is correct, you can remove the HKEY_CURRENT_USER setting. If you don't have sufficient permission or don't know what all this is about, simply double-click the second registry file which we provide named ...\qftest\qftest-3.4.4\misc\HKCU_BrowseNewProcess.reg which will change the HKEY_CURRENT_USER setting to "yes". This should always work (possibly after a reboot), but only for yourself. Other users using the same machine might still face the problem and may need to change the registry as well.

Note Firefox 2.0 has a built-in session restore feature that causes conflicts with QF-Test. When QF-Test terminates a Java applet and thus the browser, Firefox treats this as a crash and offers to restore the session on the next start through a dialog that cannot be controlled by QF-Test so that the test cannot continue unobserved.

To work around this problem, first create a separate user profile as described at http://kb.mozillazine.org/Profile_Manager and always start Firefox from QF-Test with this profile using the command firefox -P <profile>. Then disable session restore for this user by editing the preferences as described at http://kb.mozillazine.org/Editing_configuration. You need to add an option called browser.sessionstore.resume_from_crash and set it to false.

If your application is normally launched through a command of the form java -jar <archive>, create a 'Start Java SUT client' node as follows:

		Figure 4.5:  Starting the SUT from a jar archive

• Create a 'Start Java SUT client' node.
• Assign a name to the client in the 'Client' attribute.
• If necessary, change the 'Executable' attribute. Its default value ${qftest:java} is the java executable that QF-Test was started with.
• Set the 'Directory' attribute to the working directory for your application.
• Create two entries in the 'Parameters' table for the executable. Set the first to -jar and the second to the name of the archive. Unless the archive is located in the 'Directory' selected above, its full path is required.

If your application is normally launched through a command of the form java -classpath <classpath> <class>, create a 'Start Java SUT client' node as follows:

		Figure 4.6:  Starting the SUT via the main class

• Create a 'Start Java SUT client' node.
• Assign a name to the client in the 'Client' attribute.
• If necessary, change the 'Executable' attribute. Its default value ${qftest:java} is the java executable that QF-Test was started with.
• Set the 'Directory' attribute to the working directory for your application.
• Set the 'Class name' attribute to the fully qualified name of the application's starter class (the class with the main() method), just like for java.
• Create two entries in the 'Parameters' table for the executable. Set the first to -classpath and the second to the list of jar files and directories that constitute the classpath. The full path is required for jar archives not located in the 'Directory' selected above. This argument can get very long and hard to edit directly in the table. See subsection 2.2.5 about how to pop up a dialog for more convenient editing.

Note Currently web testing with QF-Test requires a 32bit Browser. Since there is little need for more than 4GB of memory in a Browser, 64bit Browsers are still the exception but on 64bit Linux systems the default browser may be 64bit in which case you need to install an additional 32bit Firefox. Also, the default JDK/JRE on 64bit Systems is often a 64bit one. In that case you must install an additional 32bit JDK or JRE and run QF-Test with that. To do so, either make the 32bit JDK the default when installing QF-Test or via the Java configuration dialog, or run QF-Test from the command line with -java <executable>. If you have a 32bit browser and a 32bit JDK and the browser still won't start from QF-Test, please check if the JDK is an original from Sun or an openJDK.

Like Swing or SWT clients, a web-based SUT - i.e. a browser - is started as a separate process from within QF-Test. In order to gain access to the internals of the browser and the web page shown with its Document Object Model (DOM), QF-Test embeds a standard browser like Internet Explorer or Mozilla in a special wrapper application. The technology for embedding and accessing those standard browsers enables efficient access to the DOM beyond the browsers' standard interfaces and a unified interface that hides browser differences and enables QF-Test - and thus you - to focus on test automation with a single set of tests for all supported browsers on multiple platforms.

		Figure 4.7:  Starting a web-based SUT in a browser

A 'Start browser' node can be used to launch a browser. One SUT web client process can display multiple browser windows, which can even be a mix of Internet Explorer and Mozilla windows. Additional browser windows in an already running process can be opened via a 'Start browser' node with the attribute 'Open new browser window in existing client' set or as the result of clicking a link in a web page that opens a popup window.

Note Generally QF-Test works with all 32bit Mozilla based browsers from version 1.9, i.e. Firefox from Version 3. On a 64bit machine you must install a 32bit variant of the browser you want to test with.

Note When setting up the startup sequence with the quickstart wizard or defining your own 'Directory of Mozilla installation' attribute, try pointing QF-Test to a current Firefox or SeaMonkey installation. If you get a message saying that the Gecko Runtime Environment (GRE) cannot be determined, please make sure that you have got a 32bit browser installed. On Linux, the standard browser for your distribution may be installed in various places. One common place to look for is /usr/lib/xulrunner-<version>.

This one is tricky and wasn't possible before QF-Test version 1.7. If a second Java VM is started from an already connected SUT, QF-Test will recognize this as an indirect connection attempt from a child process of the first SUT and automatically assign an artificial client name to this new SUT. The name is created by appending ':2' to the client name of the parent SUT, signifying that this is the second process for this client. Yet another Java VM started by either of these SUTs would get ':3' appended to the original client name unless the second process was already terminated so the ':2' was again free for use.

To summarize, the sequence for connecting to an indirectly started SUT typically consists of an event node that triggers something like a button click or menu selection, causing the SUT to launch the second SUT, followed by a 'Wait for client to connect' node for the ':2' extended client name.

The standard output and error streams of all processes started by QF-Test are captured and stored in the run-log under the node that represents the respective starter node. In this QF-Test does not distinguish between SUT clients and arbitrary processes or shell scripts started with a 'Start process' or 'Execute shell command' node.

The main window contains a shared terminal view that shows the output of all processes started by a test that was run from this window. The »View«-»Terminal« sub-menu holds items to configure whether this terminal is visible, whether long lines are wrapped and whether it is automatically scrolled to the end when new output arrives. Other items let you clear the terminal or save its contents to a file. The maximum amount of text that the terminal holds is configurable in the option Maximum size of shared terminal (kB).

In addition to the shared terminal, for each active or recently terminated process there is an individual terminal window that shows its output. These individual terminal windows can be opened from the »Clients« menu. The shared terminal's intention is to provide visual feedback whenever new output arrives, while the individual terminals are better suited for actually studying that output.

Active processes can also be stopped with the help of the »Clients« menu, either individually in the respective sub-menu or all at once with »Clients«-»Stop all«.

The number of terminated clients that are kept in the »Clients« menu is set with the option Number of terminated clients in menu. If your processes generate lots of output and you are low on memory you may want to reduce that number.

Last update: 01/27/2012 Copyright © 1999-2012 Quality First Software GmbH