Regression tests of the surface of the editor should be executed by a capture replay tool. The central issue of an interface test is the verification if the program reacts correctly to user’s inputs and if the editor is integrated correctly in Eclipse. On this purpose first of all an overview on the tools for Java UI tests is given. Two of these tools will be chosen and evaluated. Afterwards a description of the testsuite created with the tool chosen will follow.
In total the commercial tool seems much more comfortable and robust. The remarkably fast and competent support via email shouldn’t be neglected. This is something that you cannot expect from an open source tool and that you generally speaken not find. An exception on this are maybe the forums of huge open source products like open office or Eclipse, but the SWTBot’s community isn’t this big. Finally, in table 3.2 all relevant differences between the two tools are summarized.
The result of the evaluation and in consideration of the focus areas that are set by the task was that QF-Test should be used as tool for the UI test. This decision was made in accordance with ubigrade and due to the detected differences - especially regarding to the different linearity in test creation.
| Criteria | SWTBot | QF-Test |
|---|---|---|
| Capture-Replay | No Capture component | Yes |
| API knowledge necessary | Extensive (SWTBot.*, Hamcrest) | Limited (Resolvers) |
| Injection method | Own target platform plugin | Instrumented JDK, modified SWT |
| Test creation effort | Very high (Line-by-line and transition between test realization and SUT) | Low (Capture, unrecognizable components as worst case) |
| Weaknesses in test creation | Line-by-line scripting, nonlinear work flow | Less control on SUT internals |
| Execution speed | Fast (In situ on SUT level | Fast (Direct instrumentation of the JDK) |
| Strength in test execution | Introspection | Deploy neutrality |
| Setup/Teardown | Just on SUT-level or via ant etc. | Procedures, SUT can be restarted between test runs |
| Customizability | A customized Hamcrest-Matcher usable | Customized resolvers usable |
| GEF-support | Bad [Mer09] | Mature, activatable via import |
| SWTBot for Eclipse/RCP (SWT) | QF-Test for Eclipse/RCP (SWT) |
The whole extract from the evaluation report concerning QF-Test can be found here (German PDF only).
1. Functional cookies
We use functional cookies to ensure the basic functionality of the website.
2. Performance and statistic cookies
We use Matomo for analyzing and optimizing our website. Cookies permit an anonymous collection of information that help us offering you a clear and user-friendly visit of our web pages.
This cookie contains a unique, pseudonymized visitor ID internal to Matomo for recognizing returning visitors.
This cookie is used to track from which website the anonymized user proceeded to our website.
The Matomo session cookie is used to track the visitor's page requests during the session.
is created and should be then directly deleted (used to check whether the visitor’s browser supports cookies).
short lived cookies used to temporarily store data for the visit.
short lived cookies used to temporarily store data for the visit.
