Though they often go unnoticed, at least until the first
ComponentNotFoundException occurs, the 'Component' nodes
are the heart of a test-suite. Everything else revolves around
them. Explaining this requires a little side-tracking:
|GUI component hierarchy|
The GUI of an application consists of one or more windows which hold a number of components. The components are nested in a hierarchical structure. Components that hold other components are called containers. As QF-Test itself is a complex application, its main window should serve well as an example:
||Figure 5.1: Components of a GUI|
The window contains a menu bar which holds the menus for QF-Test. Below that is the toolbar with its toolbar buttons. The main area employs a split pane to separate the tree view from the details. The tree view consists of a label ("Test-suite") and the tree itself. The detail view contains a complex hierarchy of various components like text fields, buttons, a table, etc. Actually there are many more components that are not obvious. The tree, for example, is nested in a scroll pane which will show scroll bars if the tree grows beyond the visible area. Also, various kinds of panes mainly serve as containers and background for other components, like the region that contains the "OK" and "Cancel" buttons in the detail view.
SWT In SWT the main GUI components are called
Item. Unless explicitly stated otherwise the term
"component", as used in this manual, also applies to these and not only to AWT/Swing/JavaFX
Web The internal representation of an HTML page is based on the Document
Object Model (DOM) as defined by the W3C, a tree structure consisting of nodes. The
root node, a
Document can contain
Frame nodes with further
Document nodes and/or a root
Element with a tree structure of
Element nodes. Though an HTML page with its DOM is quite different
from a Swing, JavaFX or SWT interface, the abstractions QF-Test uses work just as well and the
general term "component" also applies to DOM nodes.
VideoThe video at https://www.qfs.de/en/yt/web-test-automation-40.html gives you a good idea of how QF-Test handles a deeply nested DOM structure.
Actions by the end-user of an application are transformed into events by the Java VM. Every event has a target component. For a mouse click this is the component under the mouse cursor, for a key press it is the component that has the keyboard focus. When an event is recorded by QF-Test, the component information is recorded as well, so that the event can later be replayed for the same component.
This may sound trivial and obvious, but component recognition is actually the most complex part of QF-Test. The reason for this is the necessity to allow for change. QF-Test is a tool designed for regression testing, so when a new version of the SUT is released, tests should continue to run, ideally unchanged. So when the GUI of the SUT changes, QF-Test needs to adapt. If, for example, the "OK" and "Cancel" buttons were moved from the bottom of the detail view to its top, QF-Test would still be able to replay events for these buttons correctly. The extent to which QF-Test is able to adapt varies and depends on the willingness of developers to plan ahead and assist a little bit in making the SUT well-suited to automated testing. But more on that later (section 5.6 and section 5.7).
|'Components' in QF-Test|
The recorded components are transformed into 'Window' and 'Component' nodes which form a hierarchy that represents the actual structure of the GUI. These nodes are located under the 'Windows and components' node. The following image shows part of the 'Components' representing QF-Test's main window.
||Figure 5.2: Component hierarchy of a Swing SUT|
Web Instead of a 'Window', the root
Document of a web page
is represented as a 'Web page' node. Nested
Frames are represented as 'Component' nodes.
Every time a sequence is recorded, nodes are generated for components that are not yet represented. When the sequence is discarded later on, the 'Components' remain, hence 'Component' nodes have a tendency to proliferate. The popup menu (right button click) for 'Window' and 'Component' nodes has two items, »Mark unused components...« and »Remove unused components«, which will mark or remove those 'Component' nodes that are no longer being referred to. Be careful though if you are referencing 'Components' across test-suite boundaries or use variable values in 'QF-Test component ID' attributes as these are not taken into account unless the test-suites belong to the same project or the 'Dependencies (reverse includes)' attribute of the 'Test-suite' root node is set correctly .
Note 4.0+ Besides this way of representing components as nodes it is also possible to address components as multi-level sub-items with an XPath-like syntax called QPath as explained in subsection 6.3.2
The attributes of 'Components' and the algorithm for component recognition are explained in detail in section 44.2. Here we will concentrate on the association between 'Component' nodes and the rest of the test-suite.
In order to control native Windows applications QF-Test provides some procedures in
qfs.autowin of the standard library.
You need to determine the criteria for the identification of the windows components
manually, using the procedures provided in the package
qfs.autowin.helpers, and then pass them as parameters
to the procedures performing the action on the components.
For details please refer to chapter 28.
Every node of the test suite has a 'QF-Test ID' attribute which is secondary for most kinds of nodes. For 'Component' nodes however, the 'QF-Test ID' has an important function. It is the unique identifier for the 'Component' node by which events, checks and other nodes that have a target component refer to it. Such nodes have a 'QF-Test component ID' attribute which is set to the 'Component's' 'QF-Test ID'. This level of indirection is important. If the GUI of the SUT changes in a way that QF-Test cannot adapt to automatically, only the 'Component' nodes for the unrecognized components need to be updated to reflect the change and the test will run again.
It is essential to understand that the 'Component's' 'QF-Test ID' is an artificial concept for QF-Test's internal use and should not be confused with the 'Name' attribute, which serves for identifying components in the SUT and is explained in detail in the following section. The actual value of the 'QF-Test ID' is completely irrelevant, except for the requirement to be unique, and it bears no relation whatever to the actual component in the GUI of the SUT. However, the 'QF-Test ID' of the 'Component' is shown in the tree of the test-suite, for 'Component' nodes as well as for events and other nodes that refer to a 'Component'. For this reason, 'Components' should have expressive 'QF-Test IDs' that allude to the actual GUI component.
When creating a 'Component' node, QF-Test has to assign a 'QF-Test ID' automatically. It does its best to create an expressive value from the information available. The option Prepend parent QF-Test ID to component QF-Test ID controls part of this process. If the generated 'QF-Test ID' doesn't suit you, you can change it. QF-Test will warn you if you try to assign a 'QF-Test ID' that is not unique and if you have already recorded events that refer to the 'Component', it will change their 'QF-Test component ID' attribute to reflect the change. Note that this will not cover references with a variable 'QF-Test component ID' attribute.
Note A common mistake is changing the 'QF-Test component ID' attribute of an event instead
of the 'QF-Test ID' itself. This will break the association between the event and the
'Component', leading to an
UnresolvedComponentIdException. Therefore you should
not do this unless you want to change the actual target component of the event.
Experienced testers with a well-structured concept for automated testing will find the component recording feature described in section 4.4 useful. It can be used to record the component hierarchy first in order to get an overview over the structure of the GUI and to assign 'QF-Test IDs' that suit you. Then you can continue to record the sequences and build the test-suite around the components.
The class of a component is a very important attribute as it describes the type of the recorded component. Once QF-Test records a button, it will only look for a button on replay, not for a table or a tree. Thus the component class conveniently serves to partition the components of a GUI. This improves performance and reliability of component recognition, but also helps you associate the component information recorded by QF-Test with the actual component in the GUI.
Besides its role in component identification, the class of a component is also important for registering various kinds of resolvers that can have great influence on the way QF-Test handles components. Resolvers are explained in detail in subsection 47.1.6.
Each toolkit defines its own system-specific classes for components like Buttons or Tables. In case of Buttons, that definition could be javax.swing.JButton for Java Swing or org.eclipse.swt.widgets.Button for Java SWT or javafx.scene.control.ButtonBase For JavaFX or INPUT:SUBMIT for web applications. In order to allow your tests to run independently of the actually utilized technology QF-Test unifies those classes via so-called generic classes, e.g. all buttons are simply called Button now. This approach provides a certain degree of independence from the dedicated technical classes and will allow you to create tests without taking care about the specific technology. You can find a detailed description of generic classes at chapter 54. In addition to generic classes QF-Test records system-specific classes as 'Extra features' with the state "Ignore". In case of component recognition problems due to too many similar components these can be activated to have a stricter component recognition at the expense of flexibility.
Another reason for generic classes is that dedicated technical classes could get changed during development, e.g. due to introduction of a new base framework or even another technology. In such cases QF-Test needs to be quite flexible in order to recognize a proper class. Here the concept of generic classes allows you to be able to cope with those changes and for the most part to re-use existing tests. You can find more details at subsection 5.4.3.
|Class hierarchy for web applications|
For Swing, FX and SWT QF-Test works with the actual Java GUI classes whereas a pseudo class hierarchy is used for web applications as follows:
||Figure 5.3: Pseudo class hierarchy for web elements|
As shown, "NODE" is at the root of the pseudo class hierarchy. It matches any kind of element in the DOM. Derived from "NODE" are "DOCUMENT", "FRAME", "DOM_NODE" and "DIALOG", the types of nodes implementing the pseudo DOM API explained in section 46.1. "DOM_NODE" is further sub-classed according to the tag name of the node, e.g. "H1", "A" or "INPUT" where some tags have an additional subclass like "INPUT:TEXT".
|Settings for class recording|
QF-Test can record classes of component in various ways, therefore it organizes component classes in various categories. Those categories are called as the specific class, the technology-specific system class, the generic class and the dedicated type of the generic class. Each category is recorded at 'Extra features'.
The option Record generic class names for components is checked by default. Using this option allows you to record generic classes in order to share and re-use your tests when testing a different technology with just minor changes to the existing tests.
In case you work with one Java engine only and you prefer to work with the "real" Java classes, you could also work without the generic class recording. But in this case you should consider to check the option Record system class only. This option makes QF-Test to record the technology-specific system class instead of the derived class. If you switch off this option you will get the derived class which enables you to make a very well targeted recognition but could cause maintenance efforts in case of changes coming from refactoring.
Web In web applications QF-Test records classes as described in the previous chapter subsection 5.4.2. In case you have to work with a supported AJAX toolkit (see section 46.3), QF-Test records generic classes as well. You shouldn't modify the default options for this technology.
Depending on its class a component has a set of (public) methods and fields which can be used in an 'SUT script' once you have a reference to the object (see subsection 12.2.4). Select the entry »Show the component's methods...« from the context menu of a node under the 'Windows and components' branch to display the methods and fields of the corresponding class or right click on a component in the SUT while you are in component recording mode (see section 4.4).
The methods and fields displayed for (HTML) elements in a browser cannot be used
directly with an object returned by
rc.getComponent(). These are at
(cf. section 46.1).
|The importance of naming components|
Test automation can be improved tremendously if the developers of the SUT have either planned ahead or are willing to help by defining names for at least some of the components of the SUT. Such names have two effects: They make it easier for QF-Test to locate components even after significant changes were made to the SUT and they are highly visible in the test-suite because they serve as the basis for the 'QF-Test IDs' QF-Test assigns to components. The latter should not be underestimated, especially for components without inherent features like text fields. Nodes that insert text into components called "textName", "textAddress" or "textAccount" are far more readable and maintainable than similar nodes for "text", "text2" or "text3". Indeed, coordinated naming of components is one of the most deciding factors for the efficiency of test automation and the return of investment on QF-Test. If development or management is reluctant to spend the little effort required to set names, please try to have them read this chapter of the manual.
Note Please note that recorded names are stored in the 'Name' attribute of 'Component' nodes. Because they also serve as the basis for the 'QF-Test ID' for the same node, 'Name' and 'QF-Test ID' are often identical. But always keep in mind that the 'QF-Test ID' is used solely within QF-Test and that the 'Name' is playing the critical part in identifying the component in the SUT. If the name of a component changes, it is the 'Name' attribute that must be updated, there is no need to touch the 'QF-Test ID'.
The technique to use for setting names during development depends on the kind of SUT:
Swing All AWT and Swing components are derived from the AWT class
Component, so its method
setName is the natural standard for
Swing SUTs and some developers make good use of it even without test automation in mind,
which is a great help.
JavaFX For JavaFx
setId is the pendant of Swing's
method to set identifiers for components (called 'Nodes'). Alternatively IDs can be set via
the FXML attribute
fx:id. While the ID of a 'Node' should
be unique within the scene graph, this uniqueness is not enforced. This is analogous to
the 'ID' attribute on an HTML element.
SWT Unfortunately SWT has no inherent concept for naming components. An
accepted standard convention is to use the method
setData(String key, Object
value) with the String
"name" as the key and the designated name as
the value. If present, QF-Test will retrieve that data and use it as the name for the
component. Obviously, with no default naming standard, very few SWT applications today
have names in place, including Eclipse itself.
Fortunately QF-Test can derive names for the major components of Eclipse/RCP based applications from the underlying models with good results - provided that IDs were specified for those models. See the Automatic component names for Eclipse/RCP applications option for more details.
WebIn case you want to test a web application using a supported AJAX toolkit, please take a look at subsection 46.3.3 for details about assigning IDs.
If developers have implemented some other consistent naming scheme not based on the above
methods, those names can still be made accessible to QF-Test by implementing a
NameResolver as described in subsection 47.1.6.
The reason for the tremendous impact of names is the fact that they make component recognition reliable over time. Obviously, locating a component that has a unique name assigned is trivial. Without the help of a name, QF-Test uses lots of different kinds of information to locate a component. The algorithm is fault-tolerant and configurable and has been fine-tuned with excellent results. However, every other kind of information besides the name is subject to change as the SUT evolves. At some time, when the changes are significant or small changes have accumulated, component recognition will fail and manual intervention will be required to update the test-suite.
Another aspect of names is that they make testing of multi-lingual applications independent of the current language because the name is internal to the application and does not need to be translated.
|Considerations for setting names|
There is one critical requirement for names: They must not change over time, not from one
version of the SUT to another, not from one invocation of the SUT to the next and not
while the SUT executes, for example when a component is destroyed and later created anew.
Once a name is set it must be persistent. Unfortunately there is no scheme for setting
names automatically that fulfills this requirement. Such schemes typically create names
based on the class of a component and an incrementing counter and invariably fail because
the result depends on the order of creation of the components. Because names play such a
central role in component identification, non-persistent names, specifically automatically
generated ones, can cause a lot of trouble. If development cannot be convinced to replace
them with a consistent scheme or at least drop them, such names can be suppressed with the
help of a
NameResolver as described in subsection 47.1.6.
QF-Test does not require ubiquitous use of names. In fact, over-generous use can even be counter-productive because QF-Test also has a concept for components being "interesting" or not. Components that are not considered interesting are abstracted away so they can cause no problem if they change. Typical examples for such components are panels used solely for layout. If a component has a non-trivial name QF-Test will always consider it interesting, so naming trivial components can cause failures if they are removed from the component hierarchy in a later version.
Global uniqueness of names is also not required. Each class of components has its own namespace, so there is no conflict if a button and a text field have the same name. Besides, only the names of components contained within the same window should be unique because this gives the highest tolerance to change. If your component names are unique on a per-window basis, set the options Name override mode (replay) and Name override mode (record) to "Override everything". If names are not unique per window but identically named components are at least located inside differently named ancestors, "Hierarchical resolution" is the next best choice for those options.
Two questions remain: Which components should have names assigned and which names to use?
As a rule of thumb, all components that a user directly interacts with should have a name,
for example buttons, menus, text fields, etc. Components that are not created directly,
but are automatically generated as children of complex components don't need a name, for
example the scroll bars of a
JScrollPane, or the list of a
JComboBox. The component itself should have a name, however.
If components were not named in the first place and development is only willing to spend
as little effort as possible to assign names to help with test automation, a good strategy
is to assign names to windows, complex components like trees and tables, and to panels
that comprise a number of components representing a kind of form. As long as the structure
and geometry of the components within such forms is relatively consistent, this will
result in a good compromise for component recognition and useful 'QF-Test ID' attributes.
Individual components causing trouble due to changing attributes can either be named by
development when identified or taken care of with a
Since QF-Test "knows" the components for which
setName is most useful, it comes
with a feature to locate and report these components. QF-Test even suggests names to assign,
though these aren't necessarily useful. This feature is similar to component recording and
is explained in the documentation for the option Hotkey for components.
Unavoidably the components of the SUT are going to change over time. If names are used consistently this is not really a problem, since in that case QF-Test can cope with just about any kind of change.
Without names however, changes tend to accumulate and may reach a point where component recognition fails. To avoid that kind of problem, QF-Test's representation of the SUT's components should be updated every now and then to reflect the current state of affairs. This can be done with the help of the »Update component(s)« menu-item in the context menu that you get by right-clicking on any node under the 'Windows and components' node.
Note This function can change a lot of information in your test-suite at once and it may be difficult to tell whether everything went fine or whether some components have been misidentified. To avoid problems, always create a backup file before updating multiple components. Don't update too many components at once, take things 'Window' by 'Window'. Make sure that the components you are trying to update are visible except for the menu-items. After each step, make sure that your tests still run fine.
Provided that you are connected to the SUT, this function will bring up the following dialog:
||Figure 5.4: Update components dialog|
If you are connected to multiple SUT clients, you must choose one to update the components for.
Select whether you only want to update the selected 'Component' node or all its child nodes as well.
You can choose to include components that are not currently visible in the SUT. This is mostly useful for menu-items.
The 'QF-Test ID' for an updated node is left unchanged if "Use QF-Test component ID of original node" is selected. Otherwise, updated nodes will receive a 'QF-Test ID' generated by QF-Test. If the 'QF-Test ID' of a node is changed, all nodes referring to that node via their 'QF-Test component ID' attribute will be updated accordingly. QF-Test also checks for references to the component in all suites of the same project and in those suites that are listed in the 'Dependencies (reverse includes)' attribute of the 'Test-suite' node. Those suites are loaded automatically and indirect dependencies are resolved as well.
Note In this case, QF-Test will open modified test-suites automatically, so you can save the changes or undo them.
After pressing "OK", QF-Test will try to locate the selected components in the SUT and fetch current information for them. Components that are not found are skipped. The 'Component' nodes are then updated according to the current structure of the SUT's GUI, which may include moving nodes to different parents.
Note For large component hierarchies this very complex operation can take a while, in extreme cases even a few minutes.
This function is especially useful when names have been set for the first time in the SUT. If you have already generated substantial test-suites before convincing the developers to add names, you can use this function to update your 'Components' to include the new names and update their 'QF-Test IDs' accordingly. This will work best if you can get hold of an SUT version that is identical to the previous one except for the added names.
Note Very important note: When updating whole windows or component hierarchies of significant size you may try to update components that are not currently visible or available. In that case it is very important to avoid false-positive matches for those components. You may want to temporarily adjust the bonus and penalty options for component recognition described in subsection 37.3.4 to prevent this. Specifically, set the 'Feature penalty' to a value below the 'Minimum probability', i.e. to 49 if you have not changed the default settings. Don't forget to restore the original value afterwards.
If you need to change the setting of the options Name override mode (replay) and Name override mode (record) because, for example, component names turned out not to be unique after all, change only the setting for the recording options before updating the components. When finished, change the replay option accordingly.
|Troubleshooting component recognition problems|
If your SUT has changed in a way that makes it impossible for QF-Test to locate a component,
your test will fail with a
ComponentNotFoundException. This should not be confused
UnresolvedComponentIdException which is caused by removing a 'Component'
node from the test-suite or changing the 'QF-Test component ID' attribute of an 'Event' node
to a non-existing 'QF-Test ID'.
Video There are two videos available that explain in detail how to deal with a
ComponentNotFoundException. A simple case video can be found at https://www.qfs.de/en/yt/cnfes40.html, a
more complex case is discussed in the video https://www.qfs.de/en/yt/cnfec40.html.
When you get a
ComponentNotFoundException, rerun the test with QF-Test's debugger
activated so that the test gets suspended and you can look at the node that caused the
problem. Here it pays if your 'QF-Test ID' attributes are expressive because you need to
understand which component the test tried to access. If you cannot figure out what this
node is supposed to do, try to deactivate it and rerun the test to see if it runs through
now. It could be a stray event that was not filtered during recording. In general your
tests should only contain the minimum of nodes required to achieve the desired effect.
If the node needs to be retained, take a look at the SUT to see if the target component is currently visible. If not, you need to modify your test to take that situation into account. If the component is visible, ensure that it was already showing at the time of replay by checking the screenshot in the run-log and try to re-execute the failed node by single-stepping. If execution now works you have a timing problem that you need to handle by either modifying the options for default delays (subsection 37.3.5) or with the help of a 'Wait for component to appear' node or a 'Check' node with a 'Timeout'. As a last resort you can work with a fixed delay.
If the component is visible and replay fails consistently, the cause is indeed a change in the component or one of its parent components. The next step is identifying what changed and where. To do so, re-record a click on the component, then look at the old and new 'Component' node in the hierarchy under 'Windows and components'.
Note You can jump directly from the 'Event' node to the corresponding 'Component' node by pressing [Ctrl-W] or right-clicking and selecting »Locate component«. You can jump back via [Ctrl-Backspace] or »Edit«-»Select previous node«. A clever trick is to mark the 'Component' nodes to compare by setting breakpoints on them to make them easier to spot.
The crucial point is where the hierarchy for those two components branches. If they are located in different 'Window' nodes, the difference is in the 'Window' itself. Otherwise the old and new 'Component' have a common ancestor just above the branching point and the crucial difference is in the respective nodes directly below that branch. When you have located those nodes, examine their attributes top-to-bottom and look for differences.
Note You can open a second QF-Test window via »View«-»New window...« so as to place the detail views of the nodes to compare side to side.
The only differences that will always cause recognition failures are 'Class name' and 'Name'. Differences in 'Feature', structure or geometry attributes can usually be compensated unless they accumulate.
A change in the 'Class name' attribute can be caused by refactoring done by development, in which case you need to update your 'Class name' attribute(s) to reflect the change(s). Another possible cause is obfuscation, a technique for making the names of the application classes illegible for protection against prying eyes. This poses a problem because the class names can then change with each version. You can prevent both refactoring and obfuscation problems by activating the option Record system class only.
If the 'Name' has changed things get more difficult. If the change is apparently
intentional, e.g. a typo was fixed, you can update the 'Name' attribute accordingly.
More likely the cause is some automatically generated name that may change again anytime.
As explained in the previous section, your options in this case are discussing things with
development or suppressing such names with the help of a
described in subsection 47.1.6.
Changes to the 'Feature' attribute are common for 'Window' nodes, where the 'Feature' represents the window title. When combined with a significant change in geometry such a change can cause recognition to break. This can be fixed by updating the 'Feature' to match the new title or, preferably, by turning it into a regular expression that matches all variants.
Depending on the kind and amount of changes to accommodate there are two ways to deal with the situation:
Note Automatic updates for references from other test-suites require that the suites belong to the same project or the correct setting the 'Dependencies (reverse includes)' attribute of the 'Test-suite' root node.
|Accessing hidden fields on a web page|
Hidden fields are not captured by default and therefore not stored under the 'Windows and components' node.
In case you frequently need to access hidden fields you can deactivate the Take visibility of DOM nodes into account option.
Another way to get hidden fields recorded is the following:
To access a hidden field's attributes (e.g. the 'value' attribute) you can create a simple 'SUT script' as shown below. Details on scripting in general, the used methods and parameters can be found in Scripting, Run-context API and Pseudo DOM API respectively.
||Example 5.1: Accessing the value attribute of a hidden field|
|Last update: 02/27/2019
Copyright © 1999-2019 Quality First Software GmbH