We have evaluated Quick Test Pro, but QF-Test was much more user friendly and customizable. The criteria were : interface, customization, development, factoring, reports quality.
Abdi Khalid, Project Analyst, Brussels Regional Informatics Centre, Belgium:
The advantages I found with QF-Test: easy to use, good documentation, excellent recognition of our Swing components. I would definitely recommend your tool in case someone is looking for automated testing.
My QF-Test evaluation feedback:
1. GUI interface:
It's a very professional testing software that combines the easiness of operating with the complexity and high capacity to cover the test cases and test scenarios required in the testing procedure.
Alexander Masis, Sr. Software Test Automation Engineer, Analogic Corp., Peabody, USA:
We at Analogic using QF-Test to automate our complex test environment. QF-Test is a powerful tool, it is very robust, has a lot of different capabilities. We did a long evaluation of other tools including TestComplete and Rational Functional Tester - QF-Test is a clear winner. It has nice debugging, reporting and most important - excellent capabilities to develop Jython scripts to extend the value and power of tests. It works equally well on Linux and Windows.
As with any complex tool - Manuals are very important and QF-Test has excellent manuals and reference sources. Occasionally users will have complex questions and call support... I've contacted support for about 10+ times over the last 2 months and a got fast and correct resolutions on any of my questions. The last support issue was not trivial, it involved data driven test development effort. Greg Schmid was helping me resolve this problem. He offered a solution after a very thorough understanding of my goal. He then offered to develop a test example/procedure that is used in our project.
Sonia Luo, Software engineer, JDS Uniphase Corporation, Shenzhen, China:
We have evaluated some of JAVA GUI automated test tools, and finished evaluation 3 months ago. Though we comparing many aspects of these products, I think the most competitive advantage of QF-test is that it can support our customized components well. Because many of components are not standard and other tools can't support all of them. Besides, QF-test performs professionally in nearly all the aspects.
Jérémie Abeilhou, System Engineer, SAFRAN Labinal / IT Engineering School 3iL Rodez, Toulouse, France:
I think your report logs are excellent! Particularly the screenshot of the application in case of error is a very good feature. The xml based file format for the script is also an awesome feature, offering great possibilities of automatic generation or customization and it is always convenient to be able to read the file with any text editor. In addition, your product is the only one I have tested able to deal with none standard swing classes present in my java application.
I have evaluated several tools. As a tools engineer for close to 20 years I have used many of the test tools out there. I found QF-Test because it was the only tool I could get to successfully test a Java based GUI application. After testing more than half a dozen tools with no success, I was faced with having to tell my customer that I could not find a tool that would work with their application.
Boy was I relieved when I found that QF-Test did work. It saved the project for me. That is why I recommended this company purchase the tool. I am well aware of QFS' wonderful support and have used it many times.
Pece Stanoev, Java developer, Skopje, Macedonia:
From my experience with QF-Test I find it to be very useful tool for automatic testing. Here are the pros from my perspective:
I enjoined the use of QFS pretty much and I can say it is the best tool for automatic testing that I've tried.
Thanks for this great product, Kind regards, Pece
Leonard Shi, Tester, DRS Signal Solutions, Columbia, USA:
I evaluated Squish other than QF-Test. What make us choose QF-Test was it's excellent customer support and it uses non-proprietary scripting language.
→ Klaus Berg, Senior Engineer, Siemens Corporate Technology, Munich, Germany:
For my evaluation criteria as well as dedicated test scenarios for the real system under test I have to say that QF-Test succeeded in fulfilling all my requirements with a score of more than 90%. That result placed QF-Test above all other evaluated commercial and open source tools -- at least for my specific requirements and target. However, I think you will hardly find another Java Swing GUI test tool that gives you a better value for your money. (PDF Version with more details)
When selecting a GUI testing system, we had several crucial requirements:
Tore Felix Munck, Lead Quality Assurance Specialist, SPT Group AS, Kjeller, Norway:
Question 1: Have you evaluated other tools as well?
Yes, Rational Robot from IBM
Question 2: Which criteria were crucial when deciding in favor of QF-Test?
Philippe Heredia, Infiniscale Test Manager, Montbonnot Saint Martin, France:
I have evaluated other tools before choosing QF-Test, the most important thing, for me, is that QF-Test has an easy to launch process under test method (just use the normal way to launch the process) and it's working with encrypted classes. The other tools I have tested are not so easy to use.
QF-Test also has a batch mode, and I am interested in, but I have not tested the batch for now... One other reason is that I have worked with QF-Test version 1.0, so I know QFS a little and the tool too...
Antti Sinisalo, Project Manager, Oikeat Oliot Oy, Helsinki, Finnland:
I started the evaluation with four different tools. I was evaluating general technical specifications/features and general look/feel of these tools. As a result we have decided to move forward with QF-Test. The main reason for the decision was web start support. This has turned out to be a difficult feature in the project for which we are currently looking for a new tool for test automation (we started the project with a different tool but web start support has generated number of problems).
The main reason we chose QF-Test was its compliance with Java and NetBeans. I must say I have composed a set of tests and hardly need any support. The tool is intuitive, and the manual usually helps me. This is indeed a great tool to work with. We are saving lots of testing time using the automation scripts.
Damian Barnes, Software Design Engineer, Tecnomen LTD, Shannon, Ireland:
Which criteria were crucial when deciding in favor of QF-Test?
Important criteria were:
Matt Ezren, QA developer, Creditex, Inc., New York, USA:
In 2005, I evaluated about 10 tools for automated Java GUI testing. Our table models were very complex therefore for evaluation of the tools I designed few tests/criteria which dealt with cells' content and images. Neither of the tools could perform all tests. I contacted specialists from technical support, they either admitted that their tools could not do some specific tasks, or they tried to offer workarounds, which would be an option if I would not find a tool that "does it all."
When I evaluated QF-Test, then qftestJUI, it passed all my tests. Still I was a little bit hesitant because its scripting language was Jython, and learning a new language is a considerable effort, though an extensive array of standard nodes was very simple to use. Some of other tools had scripting language which I already knew; nevertheless eventually I made my mind and selected QF-Test reasoning that reliable performance was more important than my learning efforts.
Bojan Lozinsek, Project Manager, Hermes Softlab, Maribor, Slovenia:
After evaluating a number of Java Swing test automation tools we found QF-Test to be the only one that managed to execute Java GUI tests on multiple platforms without platform dependency issues.
Let me add three sentences about your support: "This is what I call a response time! Great job! No wonder that your company is such a success :)."
Sean Kane, QA Manager, Intervoice, Dallas, USA:
This past year we looked at several test tools for eclipse. Your new eclipse support is far better than the competition. Keep the improvements coming. Your team has been very supportive when ever we have needed it. I wish you and your team continued success.
As part of its offer, our company proposes suite of products for airline IT named Altéa. Graphical interface use complex graphical components based on Java Swing and are deployed on several tens of thousands of workstations. Deployment is costly and time consuming thus it is primordial for us to ensure high level of quality. We have then conducted a study to find suitable test tool; 7 tools have been evaluated including QF-Test and the best known products.
QF-Test has been selected for following reasons:
We are utilising QF-Test very successfully in various projects since 2004. Some of the tested applications have gone through a number of release cycles in the meantime.
Generalising I can state that the tests were stable across all projects and easy to maintain, provided they are suitably modularized. Due to the good coverage of our automated tests we are able to conduct revision tests within a short period.
Our primary reasons leading to our decision for QF-Test were its maintainability gained through separation of components and events, the ability to test complex structures (e.g. JTree, JTable), its lucid structuring of the test-suite (considering our very large applications) and the possibility of distributed test development.
Dario Zanotti, Siemens S.p.A., Italy:
I performed a technical evaluation on a Linux machine, and I see that QF-Test is quite user friendly, versatile and environment independent.
When I had troubles, the answers from QFS were fast and centered on the problem. Documentation is useful and quite well organized.