SilkTest vs. Squish vs. Marathon vs.

back to the overview

Java automation testing

QF-Test vs. SilkTest vs. Squish vs. Marathon

Evaluation results

Realization:

Olav Krapp and Matthias Besemer, SBE network solutions GmbH, Heilbronn Products tested: Borland SilkTest, Froglogic Squish, QFS QF-Test, Marathon

Conclusion:

The decision fell in favour of „QF-Test“ by QFS.

Justification:

The main reason for this decision is the fact that QF-Test is much less developer-oriented than the other products. However, QF-Test still grants the necessary and also sufficient flexibility regarding changes in the GUI in test. Another decisive advantage of QFTest to be mentioned is its clearly arranged view and its - after some (although slightly longer) familiarization - also intuitive interface. The display of the test results and the available reports completed the overall impression positively.

Comparison matrix

Product	SilkTest	Squish	QF-Test	Marathon
Advantages	smooth support of obfuscating web testing already integrated screenshots can easily be recorded comprehensive integration in requirements-based testing / test management extensive tutorials, documentation and support	component picker for e.g. verification unlimited Linux support user has not to be logged in for testing on remote computers utilization of common script languages three runtime licenses included in a user license extensive tutorials, documentation and support web testing available (though as a separate edition with additional cost)	non-developer-oriented solution* (though very flexible) modern looking and basically user-friendly user interface (very clearstructured display of test suites, tests and other items) unlimited Linux support dependencies to avoid redundancies component picker for e.g. verification smooth support of obfuscating connection to SBE application without problems screenshots can easily be recorded reports for the test results floating license included extensive tutorials, documentation and support	available at no charge open source software component picker for e.g. verification connection to SBE application without problems
Disadvantages	developer-oriented solution* outdated appearing user interface, handling not very intuitive only limited Linux support (RedHat Enterprise) connection to SBE application faulty utilization of a proprietary script language user must be logged in for testing on remote computers expensive runtime license floating licenses for additional charge only	developer-oriented solution* outdated appearing user interface, handling partially laborious (e.g. forced naming of test cases) connection to SBE application faulty no simple support of obfuscating no easy provision of screenshots in case of errors cumbersome concept of „verification points“ (no renaming possible, redundancy) renaming of symbolic component names is very laborious, manual interaction necessary limited integration in test management floating licenses for additional charge only	slightly raised familiarization effort regarding handling, some icons aren’t very intuitive web testing not yet available (announced as separate edition at additional costs) user must be logged in for testing on remote computers limited integration in test management	developer-oriented solution* product obviously not mature web testing not available documentation and support not on sufficient level

*Remark: A developer-oriented solution (mainly script-based) is not necessarily a disadvantage, but we have been looking for a product usable without advanced programming skills if possible.

Evaluation of GUI-test tools for Java Swing: QFS QF-Test, Borland SilkTest, Froglogic Squish and Marathon - November 2007, O. Krapp, M.Besemer, SBE network solutions GmbH, Heilbronn, Germany.

(Original German texts and citations are translated into English.)

Free Download

Free Trial

Evaluation of GUI test tools for Java Swing

QF-Test vs. SilkTest vs. Squish vs. Marathon

Realization:

Conclusion:

Justification:

Comparison matrix