|Testing native Windows applications (preview)|
This chapter covers automation and testing of Windows desktop applications, namely
All these kinds of applications support Microsoft UI Automation or Microsoft Active Accessibility (MSAA) for software test automation, joined together in the Windows Automation API, please see section 15.2 for background information.
As a precondition of testing, QF-Test first needs a connection with the respective process. An appropriate 'Setup' sequence can be created by help of the Setup sequence creation from the »Extras« menu. Choose 'A native Windows application' as application type then.
If your application is already up and running, just specify a regular expression for the
title of its (main) window, for example
.*- Editor for the Windows Notepad
application. Otherwise give the path to your Windows executable (.exe) so that QF-Test can
start the program, please see Launching/Attaching to an application for details.
Once the application is connected to QF-Test (as 'GUI engine'
win), capture and
replay can be performed as described in chapter 4. However,
due to the nature of UI Automation, you should observe the recording rules listed in
A few example test-suites can be found in the QF-Test installation folder, namely
Also have a look at the (Current) Limitations, most of which are expected to be fixed or improved in future releases of QF-Test.
A common framework for all Windows-based applications is the Windows Automation API
consisting of Microsoft's Active Accessibility and its successor, Microsoft UI
Automation. Being it the heart of the
win engine, QF-Test is now able to
control virtually all kinds of Windows applications.
A Windows application has to expose so-called
Providers in order to follow
the rules of UI Automation. This is done automatically when a framework like WPF is used
to develop a program. This is also done for Win32 applications via proxy providers. That
means, how good an application can be controlled and tested depends on the quality of
the respective providers, i.e. usually on the framework used in application development.
For example, as UI Automation was introduced along with WPF, these kinds of applications
are supposed to be testable quite well. And when you wonder what cannot be tested via UI
Automation, try out a Java Swing application. However, there is already a pretty good
tool available when you want to deal with Swing ...
A test application that wants to control a program via UI
Automation can get hold of so called
Automation Elements which represent the actual UI
elements in the SUT (System Under Test). Though every automation element has a control
MenuItem, etc.), its actual functionality
- for example, setting a value in a text field - depends on
Patterns implemented by the respective providers.
To deal with the UI Automation framework, QF-Test starts a special Java program which serves as UI Automation client application. That program can access all UI Automation elements in a given process and handle them according to the rules of QF-Test (e.g. create a snapshot of an element as 'Component').
|Launching/Attaching to an application|
Testing a native Windows application does not require you to launch that application from QF-Test. You can also connect to a running process and that way even control parts of the operating system, for example the Windows Taskbar.
In order to connect to a process you can specify a regular expression for the window
title or the respective process ID or the window's UI Automation class name. Strictly
speaking, that window must not be a
Window but could also be a
Pane or a
Menu in terms of UI Automation control types.
Whatever feature is used for attaching, QF-Test will eventually determine the respective
process ID and treat exactly that process as the actual client application (SUT).
For example, by specifying
.*- Editor you can attach to a running Windows
Notepad application, while
-class Shell_TrayWnd will address the Windows
Besides attaching to a running process, it is also possible to launch a program from the 'Start Windows client' node. To this end, specify the path to the respective executable in the 'Windows application' attribute.
In some cases, it can also be useful to define both the 'Windows application' and the 'Window title (RegExp)' attribute. QF-Test will then first try to attach. If that fails, the given program will be started and connected via its process ID. If that fails too - the process may launch a child process and terminate itself or may not display a (graphical) user interface - another attempt to attach is made.
When you terminate a
win client in QF-Test (either via the 'Stop client'
node or from the »Clients« menu), the respective UI Automation
client process will be stopped along with its sub-processes. That is, your actual SUT
will terminate as well, if you started it from QF-Test. On the other hand, the SUT will not
be stopped when it was running before you attached to it.
When you close the SUT, the UI Automation client will terminate as well.
To attach to an elevated processes (presenting the UAC prompt), you have to launch QF-Test as administrator.
However, as the communication between the SUT and the QF-Test UI Automation client is handled by Windows (the UI Automation core), accessing elements is not quite as fast as you may know it from the QF-Test Java automation. Furthermore, in contrast to Java and Web testing (QF-Driver), events are processed asynchronously, i.e. you cannot expect that an application's dispatch thread is blocked while QF-Test is handling an event.
That makes recording more difficult, because a target element might be destroyed by the action to be recorded, for example when selecting an entry from a ComboBox or clicking on a button that closes its parent window.
So you'd best get into the habit of following a few recording rules:
Sometimes check recording (and transforming the node afterwards) may work better than event recording, for example when a click on a button (like OK, Cancel) closes the respective dialog or when a mouse down event recreates elements (for example the accessory table in the CarConfiguratorNet WPF demo application). In check recording mode QF-Test covers the SUT with an (almost) invisible window to prevent mouse clicks from triggering an action in the client application.
In QF-Test an automation element is recorded (or can be inserted manually, of course) as
'Window' or 'Component' and stored within the 'Windows and components' node. The
QF-Test (generic) 'Class name' often corresponds to the type of the UI Automation element,
Button. To be able to differentiate between the UI Automation
type and the generic class name, QF-Test adds a prefix
Uia. to the type.
Similarly, the UI Automation framework specifier is used as prefix for the automation
element's class. So you may for example see a
classname: WPF.DataGrid in
the 'Extra features' of a Table component recorded in a WPF application.
QF-Test does not strictly follow the hierarchy of the UI Automation elements. That is often the case with dialogs (like Notepad's Font dialog) which are usually listed below the main application window in the UI Automation tree. From the Win32 perspective as well as what QF-Test users would expect, such dialogs are also top-levels and thus listed as a sibling of the main window below 'Windows and components'. On the other hand, a context menu can be a top-level in the UI Automation tree, but may be a window's child in QF-Test.
|Playback and Patterns|
win engine usually records mouse clicks with the 'Replay as "hard" event'
attribute. That is the safe way, i.e. a "hard" mouse click is likely to actually trigger
the desired action.
However, UI Automation supports various "soft" actions which do not rely on mouse events. For example, to trigger a button's action you can play back
+Select: invoke [myButtonID]
The effect should be the same as with
+Mouse click [myButtonID]
but no mouse is involved when using the 'Selection' node. Instead, the UI
Automation core will trigger the execution of a provider's
in the SUT.
The 'Selection' node does support the following actions in its 'Detail' attribute:
invoke: Usually equivalent with a mouse click.
expand, collapse: Should expand/collapse a ComboBox, MenuItem or TreeItem.
select: Should select an item in a list (also use 0 as 'Detail'). 1 and -1 are meant to extend or reduce the selection when multi-selection is allowed.
toggle[:on|off]: Change the state of a CheckBox element.
scroll:horiz%,vert%: Values between 0 and 100 are possible, defining the position of the scroll location in percent; specify -1 when you do not want to change a position (horizontally or vertically).
The actions actually supported depend on an automation element's patterns. They are recorded among the 'Extra features' of a component or can be determined in an 'SUT script' (see below).
What exactly a pattern means can vary from application to application. If, for example,
Invoke patterns are supported,
Invoke might be preferable because
may only highlight the element but not trigger the respective action (e.g. Notepad Fonts).
Even worse, the formal support of a pattern does not mean that applying it has any
effect, for example scrolling an (invisible) entry in the list of Windows Calculator's
modes into view (
ScrollItem pattern). To get around the problem, you can
simply play back
select here, whether or not the entry is currently
As already mentioned above, because "soft" playback may simply not work (due to the
provider implementation) or even block when a modal dialog is to be displayed (due to
COM method invocation), QF-Test records "hard" mouse clicks by default. If you deactivate
the 'Replay as "hard" event' attribute,
invoke is played back instead (only if the
respective pattern is supported by the element).
Regarding 'Key events', a text can only be set directly by a 'Text input' node if
Value pattern is supported. Otherwise single key events have to be played back.
win engine represents automation elements by a class
WinControl. To obtain an element in a Groovy 'SUT script' node,
||Example 15.1: Retrieving a |
with the respective 'QF-Test component ID'. The most important WinControl methods are
getType(), getClassName(), getFramework(), getName(), getId(), getHwnd(), getLocation(), getSize(), getLocationOnScreen(), getPatterns(), hasPattern()to retrieve UI Automation properties of the element
getChildren(), getParent(), getChildrenOfType(), getAncestorOfType(), getElementsByClassName()to traverse the element hierarchy
getControl()to retrieve the respective control from the mmarque ui-automation library. A snapshot of that library is part of the QF-Test releases.
The behavior of the
win engine can be influenced via a set of QF-Test options
and additionally by defining preferences which affect the native part of the UI
Automation Client. Those options and preferences can be set in an 'SUT script'
respectively. To reset an option, use
As the display resolution increased over the years, Windows allows to define a scale factor so that application windows and controls are enlarged and text becomes more readable. Usually, UWP, WPF and Windows Forms application do scale automatically, but especially Win32 programs may retain the size of its controls or scale differently.
QF-Test works with physical display coordinates by default so that geometry values will change when an application is scaled. Say the scale factor is set to 125%, a button which originally (100%) resides at location (24, 40) with size (100, 20) will be moved to location (30, 50) within its container and grow to an area of 125 times 25 pixels. The consequences are
To make QF-Test work with logical coordinates (as seen with a scale factor of 1), you can
true. QF-Test then uses the
scale factor of the primary connected monitor to adapt the geometry of components and
mouse event coordinates. Note that rounding errors may occur when calculating new
integer coordinates so that the mouse may not hit a given point exactly.
You sometimes may want to play back an event on an element that is actually not
visible (it may not be scrolled into view). To perform an
then, you may need to get rid of the visibility test which is usually part of the
This can be achieved by setting
false. After playing back the event, you should reset the option to
re-enable the visibility test.
|Attaching to a window with a given class|
If you attach to an application via
-class <class name>, QF-Test
by default ignores all toplevels in the application which do not have the given class
name. That way, you can for example deal solely with the Windows Taskbar (and set
apart the desktop and its icons which run in the same process).
To be able to access all toplevels in the process, you can set the preference
There are a number of limitations in the current implementation status of the Windows testing functionality. We will try to further improve things within the next versions, but possibly not all of the following points will be resolved soon.
As the support for UI Automation depends on the framework used for application development, the recording in QF-Test may not always be consistent. For example, a 'Wait for component to appear' node may or may not be recorded when opening a dialog.
Dealing with applications consisting of several processes requires several
win clients and can be tricky.
Further limitations / not yet implemented features (August 2019) are among other things
The Windows Automation API is described here: https://docs.microsoft.com/en-US/windows/desktop/WinAuto/windows-automation-api-portal.
More about Mark Humphrey's ui-automation Java library can be found on https://github.com/mmarquee.
|Last update: 12/12/2019
Copyright © 1999-2019 Quality First Software GmbH