2018 bis heute | 2017 2016 2015 2014 | 2013 | 2012 | 2011 2010 2009 | 2008 | 2007

(ältere Archiveinträge vor 2007 nicht dargestellt, aber in der Suche enthalten)

Mailingliste - Einträge 2008

[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

[QF-Test] Feature suggestions: expected pass/fail, time profiling

  • Subject: [QF-Test] Feature suggestions: expected pass/fail, time profiling
  • From: Jon Bright <jbright@?.de>
  • Date: Mon, 28 Jul 2008 11:27:20 +0200


1. I'm not sure if we previously suggested this, but it would be a huge
enhancement for us if we could mark tests as being an expected pass or
expected fail.  We often create tests for known issues significantly in
advance of those issues actually being fixed.  Our test reports
therefore always contain some number of failures.  At the moment, it's
difficult to differentiate between an "expected fail" and an "unexpected
fail".  Similarly, information about an "unexpected pass" is obviously
of interest.  Being able to easily differentiate these categories would
be of great value to us.

2. We've now moved to more automated testing.  Whenever a commit occurs,
our tester's testing machine automatically builds the app, reinstalls
the server it connects to, starts the testsuite and collects the
resulting report, runlog, Jython output and list of commits which
triggered this run together in a timestamped directory.  In doing this,
we've noticed that our tests are nowadays taking a good 90 minutes to
run in their totality.  The report shows us some elapsed-time stuff, but
it would be good if we could see some sort of graph or profiling
information for where the most time was taken.  Since it's running
automatically, 90 minutes isn't a huge deal, but I strongly suspect it
would be possible to get this much closer to 30 minutes and thereby get
results more quickly.  Doing this would be far easier if we had better
information about where the suite is spending its time.


Jon Bright

Software-Entwickler                tick Trading Software AG
Tel: +49-2324-90448-11             Niederlassung Sprockhövel
Fax: +49-2324-90448-21             Hattinger Straße 35
jbright@?.de                 D-45549 Sprockhövel

Hauptsitz: tick Trading Software AG,
Ernst-Schneider-Platz 1, D-40212 Düsseldorf
Tel: +49-211-3611-1111, Fax: +49-211-3611-1112
Handelsregister Düsseldorf HRB 54679
Vorstand: Oliver Wagner (Vors.), Matthias Hocke, Ingo Hillen
Aufsichtsratsvorsitzender: Dr. Joachim Schneider