Logo QF-Test

Test automation and ROI

 

Free Trial  Download  Buy

Logo Brocade

This full feature tool cut much of the manual regression testing time down, considerably.

Derrik Ammons
Software QA Test Engineer, Brocade Communications Systems, USA

Test automation and return on investment (ROI)

When should be automated?

When executing regression tests solely manually, this can surpass the effort of programming quickly when there are lots of changings and time spent for the testing of existing functionalities for quality assurance.
Besides unit tests by the developers and brief manual tests, test automation offers the following advantages (For testers and the whole project):

  • Focus on new/difficult test scenarios (Test architecture, test case creation and improvement of test quality)
  • No boring routine tests
  • Tests runs: more regular, more often, faster, any numbers of times, reproducible, reliable, less prone to errors due to human errors (without any extra costs, nightly, in batch mode without further license- or personal costs) to get fast feedback about quality in increasing projects quickly (agile approach)
  • To automate or not to automate
Controlling test expenses by automation

Automated tests and Return on Inverstment (ROI)

Handling the increasing testing effort

Due to the consecutive development cycles more and more functionalities must be tested. The increasing test effort can just be handled with test automation.

Agile software development has lots of iterations and that's the reason why test automation is a necessity - in addition to unit tests.


Image manual vs. automated tests

The initial effort is different, this means that the time spent for automating tests is usually greater than doing manual tests. With test automation (e.g. with QF-Test) after the third test cycle a positive balance can be reached already concerning test expenses. The individually reached ROI can vary of course.

In agile software development the ROI is reached faster due to shorter iteration cycles (daily at best) than in classical development and QA approaches.

"One hour of manual testing is automated in 1,5 to 2 hours.", says Ralph van Roosmalen, QA Manager, Planon, Netherlands.

Time savings due to automation in practice:
Practical examples of QF-Test users

Time before automationTime after automationCitation
8 weeks1 weekSean Kane, Manager of Test Engineering, Intervoice Inc., Dallas, USA:
But now we can regression test the application in 1 week. It used to take 8 weeks.
2 weeks5 daysGadi Goldbarg, Development Tools, QA Team Leader, Zend, Ramat Gan, Israel:
The usage of QF-Test has reduced my test cycle from 2 weeks (plus/minus) to 5 days (since not all is automated yet).
1 day3 hoursDenis Gauthier Software Integration, Thales Australia, Melbourne:
Regarding the time saving aspect, it usually took me a full day to perform the Non-Regression Tests. It has now been reduced to three hours. (see complete case study).
1 dayless than 1 hourPhil Cross, Lockheed Martin, Owego New York, USA:
Regression testing of our Java application that previously took a full day to execute is now done in less than an hour with QF-Test. 
7-8 hours1,5 hoursHeidi Klade, Logistik Pur Software GmbH, Koppl near Salzburg, Austria:
Our tests run for about 90 minutes. I suppose that a manual test run would last about 7-8 hours of concentrated work. 
Reduction of almost 50%CertiCon, Prague, Czech Republic:
In a matter of months we succeeded in reducing the test processing time by almost 50%, adding hundreds of additional test cases. (see complete case study).

You can find the complete citations here: Advantages for decision-makers


Phases of the test process with high influence on the ROI

The three pink phases of the test process (see image on the left) Test case development, test case execution and maintenance of test cases influence the return on investment (ROI) during the automation of tests the most.

Phases of the test process

Test case development

The implementation of the test cases with the tool need time, money and resources. Therefore this initial investment recoup during the test cycles. When you test manually, you should create here instructions for the testers costly.

Test case execution

In comparison to manual testing, that is slow and causes high costs for hardware and personnel, automated testing is fast and uses the hardware optimally. The test tool is very reliable in test execution.

Maintenance of test cases

The test cases must be adapted to GUI changes (Maintenance). However, the advantage of QF-Test is, that you have to make these changes just once. Furthermore modularization and a good recognition of the components is possible. With manual tests you would just adapt the instructions after fundamental changes.


Check it out for yourself:


Performance for testers

Software testers 
report their
Experiences with QF-Test

Security for developers

GUI developers
report their
Experiences with QF-Test

Advantages for deciders

Decision-makers
report their
Experiences with QF-Test

Videos Downloads Documentation Buy Free Trial