This chapter talks about the most important aspects that should be considered before you start to use QF-Test widely in your testing project. It mostly raises questions and gives general answers with references to more detailed information.
The aim of this chapter is to provide hints about issues which you should take care of in order to make your GUI tests reliable, stable, repeatable and especially maintainable.
Before you start creating and running automated tests you should think about some general matters pertaining to the environment where the tests have to run. In order to make tests reliable and repeatable you have to take into account that you must be able to bring your SUT into a well-defined state, which includes the state of its backend, e.g. a server and/or a database. If you do not think about such aspects it might become very difficult and sometimes quite tricky to re-run a test or simply to analyze test-results and maintenance of tests can become a nightmare.
Please consider the following topics:
Of course there will be more issues to take care about which are specific for your project. Try to figure them out.
You should also think about following aspects of saving or installing files:
qftest.cfg? Among other things that file contains information about how QF-Test should recognize components or what should go into the run-log. These options have to be the same for every QF-Test user, otherwise you cannot share tests in your team. To ensure that you can either use a shared network installation for QF-Test or specify the config file via command-line parameters when launching QF-Test. Make sure the shared config file is write-protected unless you explicitly want to change it. For details, see section 1.6.
license? You should put the license file to a central place in order to update the license only once when you receive an update for it. Again, you can either have a shared network installation for QF-Test or can use command-line parameters to specify the location of that file when launching QF-Test. For details, see section 1.5.
If you plan to install QF-Test on a shared network drive you have to take care about some specific things.
The main source of conflict is the system settings file
qftest.cfg. It is
actually a good (and necessary) thing to have all users use the same system settings,
especially for the recognition options. Sharing the system settings file facilitates
this. However, this file should be made read-only so that one user will not
inadvertently change the system settings for everyone. If the file is read-only, QF-Test
will not save the system settings upon exit. Any change to these settings will have
to be made by explicitly making that file writable, then exiting QF-Test and then making
it read-only again. Alternatively each user could specify a different system settings
-systemcfg <file> but that's not advisable.
The running QF-Test instances will also share the log directory (internal logging, not a problem) and the Jython package cache which can occasionally cause problems so that QF-Test cannot initialize its Jython interpreter. This doesn't happen often and can be fixed by clearing (not removing) the Jython cachedir.
For Windows, each user should also execute the
setup.exe for the primary
QF-Test version, located in the installed qftest-x.y.z directory, to get proper registry
settings and documentation links on his machine.
In the rare case when a QF-Test patch overwrites existing jar files of QF-Test, running instances based on those jars may crash on Windows.
The most important aspect of a GUI testing tool is a stable and reliable recognition of the graphical components. In that area QF-Test is very flexible and can be configured in several ways. In most cases the default configuration for the component recognition works well, but sometimes you may have to change it.
If you change the component recognition options after creating lots of test-cases, those test-cases may break. Therefore you should try to find the most appropriate settings for your project as early as possible. It is worth spending time in that area before starting to implement a huge amount of tests because in the worst case you might have to re-record or at least update all or most of the existing test-cases after a critical change of the recognition options.
Best start by recording some demo test-cases and figure out how QF-Test recognizes your SUT's components. The recognition mechanism is described in chapter 5 and section 5.9. If you re-run those demo test-cases - ideally on different versions of your SUT - and run into recognition problems, you have to ask yourself following questions about those tests:
Another important aspect besides synchronization points is the correct approach of recognizing components. You have to ask yourself the following questions to determine, which recognition approach might be the most appropriate one:
In some cases it is sufficient to change the default configuration. Let us assume the developers have set unique and stable names for the target components, i.e. buttons, textfields, checkboxes etc. In such cases it may be sufficient to just change the 'Name override mode' setting of QF-Test to 'Name overrides everything'. This setting tells QF-Test to ignore any changes in the component hierarchy and just work with the target components and the window directly.
Note You have to change this option in two places: Once at 'Record' -> 'Components' -> 'Name override mode' and at 'Replay' -> 'Recognition' -> 'Name override mode'. See section 5.3 for more details.
|Last update: 6/9/2022|
Copyright © 1999-2022 Quality First Software GmbH