QF-Test at the Austrian insurance for the public sector

Initial situation

In 2013 the Österreichische Beamtenversicherung – ÖBV (Austrian insurance for the public sector) decided to introduce an IT system for the core processes of insurance inventory management and insurance benefit processing tailored to their needs. The Java Swing/AWT based application was implemented in cooperation with an external software service provider and had its go live in July 2016.

 

The plan was to have short release cycles after the go live to be able to react quickly to changes and new functionalities.

Short release cycles generate substantial test effort:

  • Approval tests for implemented tickets
  • Regression tests for quality assurance of existing functionality
  • Sanity & smoke tests after deployments in test and quality assurance environments as well as  in production.

The specialist departments cannot permanently provide the necessary resources for full test coverage to take on all tests.

The IT department is responsible for operation of the test automation tool QF-Test.

Objective

Due to the highly insurance-specific knowledge required for the design of the automated tests, the question arose: which department was to implement the tests?

  • Should the specialist department acquire knowledge about the test tool or
  • Should the IT department involve employees with specialist insurance knowledge?

The ÖBV chose the following solution: The IT knowledge stays within the IT department and the specialist knowledge within the specialist department coordinated by the test automation coordinator. The handling of the interface has to be very easily on the specialists’ side.

Implementation

Due to scarce resources the test automation coordinator is supported by Q_PERIOR for test management and Erler Consulting International for test engineering.

Test automation coordinator between specialist department and IT

The specialist insurance department designs test cases including test parameters that are forwarded to the test automation coordinator. Based on these he creates the tests in Excel files that are processed by the test automation tool QF-Test executing the keywords with their parameters.

Technical realization

A technical solution was chosen that

  • enables non-technical users to implement test cases in Excel, to copy them etc.
  • covers, respectively handles, all screens of the insurance software
  • can switch easily between the test environments.

The tests are keyword driven. All user actions within the application (e.g. mouse clicks, text input, text checks) are mapped in QF-Test to procedures. Test engineering provides the test automation coordinator with keywords who then puts them together in test cases.

The single test cases are assembled in further Excel files to test scenarios. These test control files define which test case is executed in which test scenario.

The keyword specific parameters can be entered either directly with the keyword in the test case file or in a separate excel file fora whole dialog of the application.

For the result file, date, time and the test result of the each test case are added to the test control file. Additionally,  QF-Test creates a HTML report with graphics of the overall result. Detailed QF-Test run logs can be used by test engineering for error analysis.

Conclusion

This solution helps every department fulfil its specialized tasks. In the first step all smoke tests were automated (tests for basic functionality coverage of the application), which up to then had been done manually by the specialist departments.

The advantages for ÖBV are:

  • Savings of 5-10 man-days per release compared to manual testing
  • Stable quality
  • Automated generation of test data

Step by step, further tests will be implemented to be runas regression tests after each software deployment.

 

Test management: Q_PERIOR AG 
The IT consulting house Q_PERIOR supported concerning test management. This covered the deduction of the test strategy from the risk view, the creation of the test concept, all tasks such as design of test plans and their implementation, planning and control of all tests as well as coordination of the test case design, test execution and test resources.

Test engineering: Erler Consulting International GmbH
Erler Consulting International GmbH, represented by Ute Erler, was responsible for test engineering. This included design and implementation of the keyword processing, programming of the keywords and structuring of the test scenarios. In ongoing operation she is assisting with error analysis of the executed test cases.

(Original German texts and citations are translated into English.)