2018 up to now | 2017 | 2016 | 2015 | 2014 | 2013 | 2012

Mailing List - Entries of 2012


[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

Re: [QF-Test] Test case design in QF-Test -- How far is too far?


  • Subject: Re: [QF-Test] Test case design in QF-Test -- How far is too far?
  • From: Gregor Schmid <Gregor.Schmid@?.de>
  • Date: Tue, 14 Feb 2012 21:56:52 +0100

Hello Klaus-Peter,

JUnit inspired the original design of the nested hierarchy of 'Test'
nodes with their 'Setup' and 'Cleanup' nodes. The concept is basically
sound and useful for unit testing, but it falls short for GUI test
automation because it is inflexible and can be a real pain when you
don't want to run a test-suite as a whole. For example running just a
single 'Test' to get your SUT started and navigate to a certain point
where you want to continue test development or creating a test-suite
that calls various isolated 'Test' nodes are difficult with this
design.

When 'Test-set' and 'Test-case' nodes entered the picture they alone
didn't change that situation, but with the addition of QF-Test's
'Dependency' nodes, one of its most complex but also most powerful
features, things changed. Dependencies answer all the (implicit)
questions you asked:

- They can take care of setUp and tearDown for each and every
  Test-case.

- Their use of 'Dependency reference' nodes is similar to class
  inheritance in OOP so you can create more specialized Dependencies
  by deriving from (referring to) more basic ones.

- Dependencies are inherited through the hierarchy of 'Test-sets' and
  'Test-cases' so you avoid duplication of code

- Dependencies are smart - they calculate the difference between the 
  dependencies that were set up for the test-case that just finished
  and the dependencies that are required for the next test and they
  tear down only what is no longer needed. This is a _very_ important
  aspect. In unit testing you can afford to set up and tear down
  objects many times, but you don't want to restart your SUT for every
  test-case.

- With the use of characteristic variables Dependencies can be
  fine-tuned for switching between different setups.

Besides all that, Dependencies also offer generic error and exception
handling.

Yes, they are complex. But the effort of creating a small set of the
main dependencies will pay off thousands of times.

Best regards,
    Greg

"Berg, Klaus-Peter" <klaus-peter.berg@?.com> writes:

>  
> Hi all,
>
> in Unit Testing (e.g. with a xUnit framework) we have the "mantra" of "self-contained" tests, and so called "test fixtures" [http://en.wikipedia.org/wiki/Test_fixture]: 
>
> Test fixture refers to the fixed state used as a baseline for running tests in software testing. The purpose of a test fixture is to ensure that there is a well known and fixed environment in which tests are run so that results are repeatable. 
> Frequently fixtures are created by handling setUp() and tearDown() events of the unit testing framework. In setUp() one would create the expected state for the test, and tearDown() would clean up everything what had been set up.
>
> This way, we really have independent tests.
>
> However, is this mantra also valid (and needed) for end-user GUI-based tests done with QF-Test?
>
> In JUnit, e.g., a Test Class could be seen as the equivalent of a Test-set, where the test class is a container for related test methods (=test-cases).
> Does this mean for our QF-Test suites that EVERY Test-set has to have a SETUP node as a starting point and a corresponding CLEANUP node at the end? 
>
> Setup and Cleanup are executed for each test contained in the Test-set, as it is done with setUp() and tearDown() for every test method in a JUnit test class. 
> However, JUnit goes even one step further: JUnit provides a *new instance* of the fixture objects for each test method!
>
> So, how far is too far? Starting the SUT from the ground up in every Test-set's Setup node? And stopping the SUT (often done with a "File->Exit" menu click) in the CLEANUP node?
>
>
> Does the QF-Test community has some guidelines for this aspect of test design, including the role of QF-Test dependency usage?
> And: Is it valid that a test pre-condition (either expressed by a Setup node or a dependency) is a "test in itself, e.g., dependency "isProjectViewOpen" is a *check* (i.e., a kind of test) that this window has been opened correctly and is showing to the user...
>
> Would be interesting for me to get the opinion of others in order to add "new dimensions" to my test design ;-)
>  
> Best regards
> Klaus Peter

-- 
Gregor Schmid                                Gregor.Schmid@?.de
Quality First Software GmbH                     http://www.qfs.de
Tulpenstr. 41                               Tel: +49 8171 38648-0
DE-82538 Geretsried                         Fax: +49 8171 3864816
GF: Gregor Schmid, Karlheinz Kellerer          HRB München 140833