iSYS Software GmbH

iSYS Software GmbH, based in Munich, is a software and consulting company that has established itself as a specialist in the field of software project management for more than 25 years. iSYS develops individual software solutions and offers consulting in the entire project environment. By designing, implementing, operating and maintaining IT solutions based on future-proof platforms and standards, iSYS increases the value and competitiveness of its customers and partners.


Erzsébet Nicole Harmat , working in the quality management department at iSYS, designed a methodology for the systematic selection of a test automation tool for a software development project as part of her bachelor thesis. The core of the methodology is the creation of an individual, project-related criteria catalog. This serves as a basis for the evaluation of test automation tools.  The results, which are calculated on the basis of a formula designed in the methodology, show to what extent a test automation tool meets the requirements placed on a test automation tool by the project.

Evaluation and decision matrix

CriteriaWeight-ingQuestionQF‑TestSeleniumTestCafeTestCafe Studio
Evaluation in the operational environment2%Can the evaluation take place on site and on the object to be tested?2222
Market presence2%Has the TaT (test automation tool) been on the market for at least 5 years?2222
Community3%Is there any activity in the community in the last week or month?2222
Support4%Is there support possibility via e-mail and telephone?2011
Update4%Has there been an update in the last four months or year?2122
Documentation4%Is sufficient TaT documentation (manuals/tutorials) available for self-study in German and/or English?2211
Video tutorials4%Are there at least 10h video tutorials on the Tats website or on YouTube in German and/or English?2221
Human resources3%Are there at least three staff members who can work with the evaluated TaT?2000
Record/Playback functionality6%Does the TaT have a record/playback function?2202
Exception handling6%Does exception handling exist?2222
Waiting for events6%Is there a possibility to wait for an event (e.g. document loaded or component exists)?2222
Test-suite structure6%Is there a possibility to build the test suite modularly?2222
Log file2%Is it possible to create a configurable log file?2111
Reporting3%Does the TaT also have a report generator with its own design options?2000
User interface (UI)6%Does the TaT have a GUI?2202
Usability_012%Is it possible to insert elements or lines with different functionalities in the test case?2002
Usability_022%Is it possible to move lines in the test-suite?2002
Usability_032%Is it possible to undo the deletion in the test suite?2220
Maintainability4%Is there a log in tree view?2000
Version Control System (VCS)6%Are the TaT project artifacts manageable with git?2220
Interface to Jira2%Does the TaT have a Jira interface?2200
PDF check2%Can PDF files be checked for content and design?    2000
Developer license costs in the deployment period6%Are the developer license costs in the deployment period below 1,500 EUR/year per developer?1221
Floating licenses2%Is there an easy way (e.g. via license file) to unlock any workstations for the TaT?2220
Runtime license costs2%Are the runtime license costs within the specified range?1221
Runtime environment license duration2%Are there runtime licensing options per month?0220
Runtime environment3%Is there any way to run the automated tests independent of the development environment?2222
Virtualized environment4%Can the test cases run in Docker?2220
Result100% 94%77%65%59%

(page 34-39 and 50f)


The QF-Test test automation tool is recommended for use in the project.

The whole extract from the bachelor's thesis concerning QF-Test can be found here (German PDF only).

Bachelor's thesis: Conception of a methodology for the systematic selection of a test automation tool for a software development project using the example of real estate management, August 2021 - Erzsébet Nicole Harmat, Faculty of Computer Science and Mathematics, Hochschule München, Munich, Germany.

(Original German texts and citations are translated into English.)