From Winrunner to QF-Test

QF-Test at CoreMedia:
Robustness of tests important

Excerpt from a facebook community article from Mark Michaelis, Software Engineer Quality Assurance, CoreMedia AG, Hamburg, Germany:

About the poor GUI Automation Tools: I also think that GUI Test Automation is perhaps one of the most challenging things to do in testing. It takes a long evaluation to get the right automation tool, especially to get one accepted by the whole team.

For Java (Swing Applications) we switched from WinRunner (used for years before) to QF-Test. WinRunner has not been accepted by the team because of its proprietary language TSL. In contrast QF-Test is developed in Java and uses Jython as scripting language. A great start for Java developers. There are many more benefits of QF-Test, but I will only state three of them here:

• As QF-Test is written in Java it has no problems dealing with the latest Java releases. It's been a piece of cake to switch from testing with Java 1.5 to testing with Java 1.6
• Tests can be written in that way that also managers can easily read what the test actually does without going into details.
• Great support for Data Driven Testing.Never got so many test cases done in such a short amount of time..

About the robustness of GUI Test Automation: I think there are two things which need to be done to get the tests as robust as possible. One is already mentioned in the StickyMinds article you quoted: Involve developers. It's for sure needed to add test-hooks into the application. The most simple one is to give GUI elements a unique ID (in Java: using setName()).

The next thing is to have GUI Maps (a term I learned from WinRunner but I also adopted it for QF-Test): Place the rules to identify GUI components in an own database/file. This way you only have to touch this file for slight GUI changes.

But the best thing to do is to create a testing framework. E. g. for every menu action (every in XP sense: every currently used one) I created a library function. Developers just need to call this library function and don't need to care about if there are radio-buttons or a drop-down list behind. If the GUI element changes its type only the framework needs to be adjusted.

Even more complex functions moved into the framework. E. g. for the CMS I created framework functions which automatically create documents (using the GUI). So it's very easy for developers to get some test fixtures to work with.

Finally, back to the code, I added a GUI-Test-Facade to the application. This has two advantages:

• I can access the very inner of the application.
• Developers can easily see that I do this and how and thus can take care of it during refactoring of the code.

Having these three things: GUI Maps, GUI Automation Test Framework, GUI Test Facade I finally got a very robust set of GUI Tests.

AdditionalEvaluation report of CoreMedia