Mailing List - Entries of 2012
[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]
Re: [QF-Test] Test case design in QF-Test -- How far is too far?
Hello Klaus-Peter, JUnit inspired the original design of the nested hierarchy of 'Test' nodes with their 'Setup' and 'Cleanup' nodes. The concept is basically sound and useful for unit testing, but it falls short for GUI test automation because it is inflexible and can be a real pain when you don't want to run a test-suite as a whole. For example running just a single 'Test' to get your SUT started and navigate to a certain point where you want to continue test development or creating a test-suite that calls various isolated 'Test' nodes are difficult with this design. When 'Test-set' and 'Test-case' nodes entered the picture they alone didn't change that situation, but with the addition of QF-Test's 'Dependency' nodes, one of its most complex but also most powerful features, things changed. Dependencies answer all the (implicit) questions you asked: - They can take care of setUp and tearDown for each and every Test-case. - Their use of 'Dependency reference' nodes is similar to class inheritance in OOP so you can create more specialized Dependencies by deriving from (referring to) more basic ones. - Dependencies are inherited through the hierarchy of 'Test-sets' and 'Test-cases' so you avoid duplication of code - Dependencies are smart - they calculate the difference between the dependencies that were set up for the test-case that just finished and the dependencies that are required for the next test and they tear down only what is no longer needed. This is a _very_ important aspect. In unit testing you can afford to set up and tear down objects many times, but you don't want to restart your SUT for every test-case. - With the use of characteristic variables Dependencies can be fine-tuned for switching between different setups. Besides all that, Dependencies also offer generic error and exception handling. Yes, they are complex. But the effort of creating a small set of the main dependencies will pay off thousands of times. Best regards, Greg "Berg, Klaus-Peter" <klaus-peter.berg@?.com> writes: > > Hi all, > > in Unit Testing (e.g. with a xUnit framework) we have the "mantra" of "self-contained" tests, and so called "test fixtures" [http://en.wikipedia.org/wiki/Test_fixture]: > > Test fixture refers to the fixed state used as a baseline for running tests in software testing. The purpose of a test fixture is to ensure that there is a well known and fixed environment in which tests are run so that results are repeatable. > Frequently fixtures are created by handling setUp() and tearDown() events of the unit testing framework. In setUp() one would create the expected state for the test, and tearDown() would clean up everything what had been set up. > > This way, we really have independent tests. > > However, is this mantra also valid (and needed) for end-user GUI-based tests done with QF-Test? > > In JUnit, e.g., a Test Class could be seen as the equivalent of a Test-set, where the test class is a container for related test methods (=test-cases). > Does this mean for our QF-Test suites that EVERY Test-set has to have a SETUP node as a starting point and a corresponding CLEANUP node at the end? > > Setup and Cleanup are executed for each test contained in the Test-set, as it is done with setUp() and tearDown() for every test method in a JUnit test class. > However, JUnit goes even one step further: JUnit provides a *new instance* of the fixture objects for each test method! > > So, how far is too far? Starting the SUT from the ground up in every Test-set's Setup node? And stopping the SUT (often done with a "File->Exit" menu click) in the CLEANUP node? > > > Does the QF-Test community has some guidelines for this aspect of test design, including the role of QF-Test dependency usage? > And: Is it valid that a test pre-condition (either expressed by a Setup node or a dependency) is a "test in itself, e.g., dependency "isProjectViewOpen" is a *check* (i.e., a kind of test) that this window has been opened correctly and is showing to the user... > > Would be interesting for me to get the opinion of others in order to add "new dimensions" to my test design ;-) > > Best regards > Klaus Peter -- Gregor Schmid Gregor.Schmid@?.de Quality First Software GmbH http://www.qfs.de Tulpenstr. 41 Tel: +49 8171 38648-0 DE-82538 Geretsried Fax: +49 8171 3864816 GF: Gregor Schmid, Karlheinz Kellerer HRB München 140833