The call syntax for interactive and batch mode varies widely since some
command line arguments are specific to interactive mode or batch mode or even sub-modes
of batch mode.
Note that all of the arguments have sensible default values which you only
need to override for special cases. In most cases you'll only need to
execute either qftest [<suite> | <run-log>]*
to run QF-Test in interactive mode, or qftest -batch
[-runlog [<file>]] [-report <directory>] <suite> to execute a test in batch mode.
WindowsThe program qftest.exe is a Windows GUI application.
When started from a command shell, it will not wait for QF-Test to terminate but return
immediately. Thus, when executing a test in batch mode, you cannot see whether QF-Test has
finished or not (you may put the command into a .bat file to deal with this behaviour).
Furthermore you won't see any output from QF-Test in the console window when using
qftest.exe.
For both reasons you may prefer to utilize the qftestc.exe Console application
when launching QF-Test from a command shell: It waits for QF-Test to terminate and print output
from 'Server scripts' will be displayed in the console window.
Apart from that, everything said about qftest.exe in this chapter holds true
for qftestc.exe too.
- Interactive mode
-
The full call syntax for interactive mode is:
qftest [-dbg]
[-java <executable>] [-noconsole] [-plugindir <directory>]
[-J<java-argument>]*
[-daemon]
[-daemonport <port>]
[-engine <engine>]
[-help]
[-jythonport <number>]
[-libpath <path>]
[-license <file>]
[-options <file>]
[-port <number>]
[-reuse]
[-runtime]
[-serverhost <host>]
[-shell <executable>]
[-shellarg <argument>]
[-systemcfg <file>]
[-tempdir <directory>]
[-usercfg <file>]
[-variable <name>=<value>]*
[-version]
[<suite> | <run-log>]*
There are several sub-modes for running QF-Test in batch mode. The default is to execute
one or more test-suites. Alternatively QF-Test can be invoked to create test documentation
from test-suites or reports from run-logs or to instrument one or more JDKs or JREs.
QF-Test can also be run in daemon mode where it sits in the background waiting for calls
from the outside telling it what to do (see chapter 40 for further
information about the daemon mode). Finally, showing help or version information can
also be seen as separate sub-modes.
- Test execution
-
To execute one or more test-suites and create a run-log and/or report as a result,
use:
qftest -batch [-run]
[-dbg] [-java <executable>] [-noconsole] [-plugindir <directory>]
[-J<java-argument>]*
[-compact]
[-engine <engine>]
[-exitcodeignoreexception]
[-exitcodeignoreerror]
[-exitcodeignorewarning]
[-jythonport <number>]
[-libpath <path>]
[-license <file>]
[-nolog]
[-nomessagewindow]
[-options <file>]
[-port <number>]
[-report <directory>]
[-report.checks]
[-report.doctags]
[-report.errors]
[-report.exceptions]
[-report.html <directory>]
[-report.ignorenotimplemented]
[-report.ignoreskipped]
[-report.junit <directory>]
[-report.name <name>]
[-report.nodeicons]
[-report.passhtml]
[-report.scale-thumbnails <percent>]
[-report.teststeps]
[-report.thumbnails]
[-report.warnings]
[-report.xml <directory>]
[-runid <id>]
[-runlog [<file>]]
[-runtime]
[-serverhost <host>]
[-shell <executable>]
[-shellarg <argument>]
[-splitlog]
[-systemcfg <file>]
[-test <n>|<id>]*
[-threads <number>]*
[-variable <name>=<value>]*
[-verbose [<level>]]
<suite>+
- Test execution via QF-Test daemon
-
The following parameters can be specified when executing a test-case
by calling a daemon:
qftest -batch -calldaemon
[-cleanup]
[-dbg] [-java <executable>] [-noconsole] [-J<java-argument>]*
[-daemonhost <host>]
[-daemonport <port>]
[-nomessagewindow]
[-ping]
[-options <file>]
[-runid <id>]
[-runlog [<file>]]
[-startclean]
[-startsut]
[-stopclean]
[-stoprun]
[-suitedir <dir>]
[-terminate]
[-timeout <milliseconds>]
[-variable <name>=<value>]*
[-verbose [<level>]]
<suite#test-case>
- Create test documentation
-
Package or test-case documentation can be create for one or more test-suites or whole
directories. This is described further in chapter 15. The
command line syntax is:
qftest -batch -gendoc
[-dbg] [-java <executable>] [-noconsole]
[-J<java-argument>]*
[-license <file>]
[-nomessagewindow]
[-options <file>]
[-pkgdoc <directory>]
[-pkgdoc.dependencies]
[-pkgdoc.doctags]
[-pkgdoc.html <directory>]
[-pkgdoc.includelocal]
[-pkgdoc.nodeicons]
[-pkgdoc.passhtml]
[-pkgdoc.sortpackages]
[-pkgdoc.sortprocedures]
[-pkgdoc.xml <directory>]
[-sourcedir <directory>]
[-testdoc <directory>]
[-testdoc.doctags]
[-testdoc.followcalls]
[-testdoc.html <directory>]
[-testdoc.nodeicons]
[-testdoc.passhtml]
[-testdoc.sorttestcases]
[-testdoc.sorttestsets]
[-testdoc.teststeps]
[-testdoc.xml <directory>]
(<suite> | <directory>)+
- Create a report from run-logs
-
To create a report from one or more run-logs or whole directories use:
qftest -batch -genreport
[-dbg] [-java <executable>] [-noconsole]
[-J<java-argument>]*
[-license <file>]
[-nomessagewindow]
[-options <file>]
[-report <directory>]
[-report.checks]
[-report.doctags]
[-report.errors]
[-report.exceptions]
[-report.html <directory>]
[-report.ignorenotimplemented]
[-report.ignoreskipped]
[-report.junit <directory>]
[-report.name <name>]
[-report.nodeicons]
[-report.passhtml]
[-report.scale-thumbnails <percent>]
[-report.teststeps]
[-report.thumbnails]
[-report.warnings]
[-report.xml <directory>]
[-runlogdir <directory>]
(<run-log> | <directory>)+
- Instrument one or more JDKs or JREs
-
This is useful for automating tests that include clean installation of an SUT that
comes with its own JRE, so it needs to be instrumented before the test on the SUT
can be run. This can be done as follows:
qftest -batch -instrument
[-dbg] [-java <executable>] [-noconsole]
[-J<java-argument>]*
[-license <file>]
[-nomessagewindow]
[-options <file>]
<path to JDK or JRE>+
- Daemon mode
-
To run QF-Test in daemon mode as described chapter 40 use:
qftest -batch -daemon
[-dbg] [-java <executable>] [-noconsole] [-plugindir <directory>]
[-J<java-argument>]*
[-daemonport <port>]
[-engine <engine>]
[-jythonport <number>]
[-libpath <path>]
[-license <file>]
[-nolog]
[-nomessagewindow]
[-options <file>]
[-port <number>]
[-runtime]
[-serverhost <host>]
[-shell <executable>]
[-shellarg <argument>]
[-splitlog]
[-systemcfg <file>]
[-variable <name>=<value>]*
- Import one test-suite into another
-
qftest -batch -import
[-import.from <test-suite>] [-import.into <test-suite>]
[-import.components] [-import.procedures] [-import.tests]
- Analyze references of a test-suite
-
qftest -batch -analyze
[-analyze.target <directory>]
[-suitedir <dir>]
[-analyze.components]
[-analyze.procedures]
[-analyze.dependencies]
[-analyze.tests]
[-analyze.transitive]
(<suite> | <directory>)+
- Get version information
-
qftest -batch -version
- Get help
-
qftest -batch -help
Command line arguments for QF-Test fall in three categories. They
can be mixed freely.
These arguments are evaluated directly by the
qftest shell script or executable and override
settings determined during installation. Under Unix these settings
are stored in the file launcher.cfg in QF-Test's root
directory, under Windows they are stored in the registry.
-
-batch
-
Run QF-Test in batch mode. This causes QF-Test to load and execute a
test-suite directly and finish with an exit code that represents
the result of the test run.
-
-dbg
-
Turn on debugging output for the starter script. The same effect is achieved by
setting the environment variable
QFTEST_DEBUG to a non-empty value. On
Windows this causes QF-Test to open a console window to display the output that would
otherwise be invisible unless the argument -noconsole is also given. This
also turns on debugging output for the qfclient and java
helper programs when using the old connection mechanism (see chapter 4).
-
-java <executable>
-
The Java executable used to run QF-Test. The default is
java on Unix and javaw.exe on Windows,
unless a different value was set during installation.
-
-noconsole (Windows only)
-
On Windows this argument suppresses the console window that would otherwise be
opened in case
-dbg is specified.
-
-plugindir <directory>
-
This argument overrides the default location of the directory
for plugins, jar files that should be made accessible to
scripts. The default directory called
plugin is located under QF-Test's root directory
qftest. See section 37.2 for more
information about plugins.
You can pass arguments to the Java VM through the starter script
by prepending them with -J,
e.g. -J-Duser.language=en to set a system
property. To set the classpath, prepend -J only to
the -cp or -classpath argument, not to
the actual value, e.g. -J-classpath
myclasses.jar. When setting the classpath this way, QF-Test's
own jar archives need not to be taken into account.
The rest of the arguments are handled by QF-Test itself when it is
executed by the Java virtual machine. These arguments can also
be placed in a file using the syntax
<name>=<value> for arguments with
parameters or <name>=true or
<name>=false to turn a simple argument on or
off. By default this file is called qftest.options,
located in the bin directory of QF-Test and used only for internal logging
purposes. If you
change anything in this file, you can move it to QF-Test's root
directory so your changes will still apply after
upgrading. Alternatively you can use the -options <file> argument
to specify a different file. Arguments given on the command line
override arguments from an option file except for those which can
be given multiple times. In the latter case the arguments are
merged.
-
-analyze (batch mode only)
-
Analyze a given test-suite or all test-suite from a given
directory. The result is stored in a given file.
-
-analyze.target <directory> (batch mode only)
-
The target folder, where the result file should be created.
-
-analyze.components (batch mode only)
-
Switch for analyzing components of the given test-suite.
-
-analyze.dependencies (batch mode only)
-
Switch for analyzing dependency references of the given test-suite.
-
-analyze.procedures (batch mode only)
-
Switch for analyzing procedure calls of the given test-suite.
-
-analyze.tests (batch mode only)
-
Switch for analyzing test calls of the given test-suite.
-
-analyze.transitive (batch mode only)
-
Switch for analyzing references and calls transitively,
i.e. follow the calls and analyze its content also.
-
-calldaemon
(batch mode only)
-
Connect to a running QF-Test daemon to execute a test-case.
-
-cleanup
(calldaemon mode only)
-
With this argument all TestRunDaemons belonging to a daemon are cleaned up and
all clients killed before running the test.
-
-compact (batch mode only)
-
Create a compact run-log which retains only those branches in
which a warning, error or exception occurred. Ignored if
log file creation is suppressed with
-nolog.
-
-daemon
-
Run QF-Test in daemon mode. Further information is provided in chapter 40.
-
-daemonhost <host>
(calldaemon mode only)
-
Specify the host where to locate the QF-Test daemon. The default is localhost.
-
-daemonport <port>
-
Specify the registry port for the QF-Test daemon to listen on and to connect to
respectively. The default is 3543 or the port defined with
-port <number>.
-
-engine <engine>
-
Specify which engine license(s) to use. This option is only useful in case the QF-Test
license contains a mix of GUI engine(s) with different numbers of engine licenses.
In that case it may be necessary to specify the engine license(s) to use in order to
prevent license conflicts with colleagues using the same license. Possible values
are "all" to use all supported licenses, "ask" to bring up a dialog for engine
selection or any combination of "awt" for AWT/Swing, "swt" for SWT or "web" for Web,
e.g. "awt,web". This is explained in more detail in subsection 29.1.7.
-
-exitcodeignoreexception
(batch mode only)
-
For exit code calculation exceptions, errors and warnings are ignored. This means if
exceptions, errors or warnings occurred during a test-run the exit code will be 0. This option
is helpful when integrating QF-Test with build tools that rate a build as failed
dependent on the the exit code.
-
-exitcodeignoreerror
(batch mode only)
-
For exit code calculation errors and warnings are ignored. This means if just
errors and warnings occurred during a test-run the exit code will be 0. This option
is helpful when integrating QF-Test with build tools that rate a build as failed
dependent on the the exit code.
-
-exitcodeignorewarning
(batch mode only)
-
For exit code calculation warnings are ignored. This means if just
warnings occurred during a test-run the exit code will be 0. This option
is helpful when integrating QF-Test with build tools that rate a build as failed
dependent on the the exit code.
-
-gendoc
(batch mode only)
-
Tell QF-Test that this batch run serves to create test documentation from test-suites.
-
-genreport
(batch mode only)
-
Tell QF-Test that this batch run serves to create a report from run-logs.
-
-help
-
Show help about available command line arguments.
-
-import (batch mode only)
-
Import a given test-suite into another one.
This mode can be used to merge two test-suites.
-
-import.from <test-suite> (batch mode only)
-
The source test-suite, which should be imported into another one.
-
-import.into <test-suite> (batch mode only)
-
The target tests-suite for importing.
-
-import.components (batch mode only)
-
Switch for merging components of two given test-suites.
-
-import.procedures (batch mode only)
-
Switch for merging packages and procedures of two given test-suites.
-
-import.tests (batch mode only)
-
Switch for merging test-cases and test-sets of two given test-suites.
-
-instrument (batch mode only)
-
Instrument one or more JDKs or JREs automatically. Instrumentation and running
of a subsequent test cannot be combined in a single call. They have to be executed
as separate commands.
-
-jythonport <number>
-
Tell the embedded Jython interpreter to listen for TCP
connections at the specified port. You can then use
telnet to connect to that port and get an
interactive Jython command line.
-
-libpath <path>
-
Override the library path option (Directories holding test-suite libraries). The directories of the library path should
be separated by the standard path separator character fot the
system, i.e. ';' for Windows and ':' for Unix. QF-Test's
include directory will automatically be appended to
the path.
-
-license <file>
-
Set the location of the license file (see section 1.3).
-
-nolog (batch mode only)
-
Suppress the automatic creation of a run-log. If any of
-runlog [<file>],
-report <directory>, -report.html <directory>, -report.xml <directory> or -report.junit <directory>
is given, this argument is ignored. This option is
reatined for backwards compatibility only. To keep memory use manageable, split
run-logs should be used instead (see -splitlog).
-
-nomessagewindow (batch mode only)
-
In case of a fatal error i batch mode QF-Test prints an error message to the console
and for improved visibility also brings up an error dialog for about 30 seconds.
That dialog can be suppressed with the help of this argument.
-
-options <file>
-
Override the location of the file used to specify additional
command line arguments. This argument can be given more than once
to use several sources of command line arguments.
-
-ping
(calldaemon mode only)
-
Use this option, if you want to check whether a daemon is up and running.
-
-pkgdoc <directory> (batch mode only)
-
With this argument QF-Test creates both HTML and XML pkgdoc documentation. If no
directory is given, it is created from the basename of the suite.
-
-pkgdoc.dependencies
(batch mode only)
-
Whether to list dependencies when creating the pkgdoc documentation.
Default is true, use
-pkgdoc.dependencies=false to disable.
-
-pkgdoc.doctags
(batch mode only)
-
Whether to use the QFS doctag extensions when creating the pkgdoc documentation.
Default is true, use
-pkgdoc.doctags=false to disable.
-
-pkgdoc.html <directory> (batch mode only)
-
With this argument QF-Test creates HTML pkgdoc documentation. If no directory is given,
it is created from the basename of the suite.
-
-pkgdoc.includelocal
(batch mode only)
-
Whether to include local packages and procedures (those whose names begin with an
'_'). Default is false.
-
-pkgdoc.nodeicons
(batch mode only)
-
Whether to show icons for nodes in the pkgdoc documentation. Default is true, use
-pkgdoc.nodeicons=false to disable.
-
-pkgdoc.passhtml
(batch mode only)
-
Whether to pass HTML tags in comments through to the HTML pkgdoc.
Default is true, use
-pkgdoc.passhtml=false to disable.
-
-pkgdoc.sortpackages
(batch mode only)
-
Whether to sort packages alphabetically.
Default is true, use
-pkgdoc.sortpackages=false to disable.
-
-pkgdoc.sortprocedures
(batch mode only)
-
Whether to sort procedures alphabetically.
Default is true, use
-pkgdoc.sortprocedures=false to disable.
-
-pkgdoc.xml <directory> (batch mode only)
-
With this argument QF-Test creates XML pkgdoc documentation. If no directory is given,
it is created from the basename of the suite.
-
-port <number>
-
The TCP port on which QF-Test communicates with the SUT. By
default QF-Test uses an arbitrary dynamic port where it creates its
own RMI registry. A specific port should only be requested if
it must be hard-coded when starting the SUT.
-
-report <directory>
(batch mode only)
-
Create a combined XML/HTML report. The directory name may contain placeholders as
explained in subsection 32.2.4.
-
-report.checks
(batch mode only)
-
Whether to list checks in the report. Default is false.
-
-report.doctags
(batch mode only)
-
Whether to use the QFS doctag extensions when creating the report.
Default is true, use
-report.doctags=false to disable.
-
-report.errors
(batch mode only)
-
Whether to list errors in the report. Default is true, use
-report.errors=false to disable.
-
-report.exceptions
(batch mode only)
-
Whether to list exceptions in the report. Default is true, use
-report.exceptions=false to disable.
-
-report.html <directory> (batch mode only)
-
Create an HTML report. The directory name may contain placeholders as explained in
subsection 32.2.4.
-
-report.ignorenotimplemented
(batch mode only)
-
Whether to ignore nodes that are not implemented in the report in which case the
legend and respective columns for not implemented tests are also not shown. Default
is false.
-
-report.ignoreskipped
(batch mode only)
-
Whether to ignore skipped nodes in the report in which case the legend and
respective columns for skipped tests are also not shown. Default is false.
-
-report.junit <directory> (batch mode only)
-
Create a report in JUnit XML format as understood by many Continuous Integration
Tools. The directory name may contain placeholders as explained in subsection 32.2.4.
-
-report.name <name>
(batch mode only)
-
Specify the name for the report, meaning its identifier, not a file name. Default is
the runid. The name may contain placeholders as explained in subsection 32.2.4.
-
-report.nodeicons
(batch mode only)
-
Whether to show icons for nodes in the report. Default is true, use
-report.nodeicons=false to disable.
-
-report.passhtml
(batch mode only)
-
Whether to pass HTML tags in comments through to the HTML report.
Default is true,
-report.passhtml=false to disable.
-
-report.scale-thumbnails <percent>
(batch mode only)
-
How to scale thumbnail images for screenshots in the error listings
of the report, an integer value interpreted as percent of the original image size.
Default is 20.
-
-report.teststeps
(batch mode only)
-
Whether to list test-steps in the report. Default is true, use
-report.teststeps=false to disable.
-
-report.thumbnails
(batch mode only)
-
Whether to display thumbnail images for screenshots in the error listings
of the report. Default is false.
-
-report.warnings
(batch mode only)
-
Whether to show warning information in the report. Default is true.
-
-report.xml <directory>
(batch mode only)
-
Create an XML report. The directory name may contain placeholders as explained in
subsection 32.2.4.
-
-reuse (interactive mode only)
-
This argument is used mainly when launching QF-Test from a desktop
icon or the Windows explorer through a file association. It
tells the newly started QF-Test instance to search for an already
running version of QF-Test and ask that to open the given file(s).
If another instance can be reused in that way, the newly started
program will terminate immediately and new windows for the
file(s) will be opened by the old instance.
-
-run
(batch mode only)
-
Explicitely tell QF-Test that this batch run is for actual test execution as opposed to
generating documentation or a report. As this is the default operation for batch
mode this argument can be omitted.
-
-runid <id>
(batch and calldaemon mode)
-
Specify an ID for the test-run. The ID may contain placeholders as
explained in subsection 32.2.4 and will itself serve as a
replacement for the placeholder
%i/+i.
-
-runlog [<file>]
(batch and calldaemon mode)
-
Save the run-log in the given file. The optional filename may contain placeholders
as explained in subsection 32.2.4. If no filename is given, it is
composed of the basename of the suite and a timestamp. If missing, the extension
.qrz is added automatically and the run-log is saved compressed.
Otherwise the extension .qrl or .qrz determines
compression. Even without this argument a run-log is created unless suppressed with
-nolog or when a report is generated. The default value is
%p%b.qrz.
In calldaemon mode, a run-log will be stored only if a (local) filename is specified.
-
-runlogdir <directory>
(batch mode only)
-
This argument serves as the target directory for saving run-logs unless the filename
of the run-logs specified with
-runlog [<file>] is an absolute path. If this argument
is given and a report is generated, the files in the report will be laid out
according to the structure of the run-logs relative to this directory. The directory
name may contain placeholders as explained in subsection 32.2.4.
-
-runtime
-
Use a runtime license only. In batch mode, QF-Test will normally use a runtime license
(or multiple runtime licenses if
-threads <number> is given). If not enough free
runtime licenses are available, full development licenses will be used instead
unless -runtime is given in which case no development license is used and QF-Test
will fail with an error message. In interactive mode, if -runtime is given, QF-Test
will use a runtime license instead of a full development license. In that mode, any
test-suite can be loaded and tests can be run interactively as usual, including
debugging support. Saving of test-suites will be disabled, however, though
test-suites can be modified for temporary experiments.
-
-shell <executable>
-
The shell to use when executing a 'Execute shell command'
node. Default for Unix is
/bin/sh, for Windows
COMMAND.COM or cmd.exe.
-
-shellarg <argument>
-
The argument that causes the shell specified with
-shell <executable>
to execute the following argument and then exit. For Unix shells
this is typically -c, COMMAND.COM and
cmd.exe expect /c. If you have Unix
tools installed on Windows and specify sh or
bash as the shell to use, don't forget to change
-shellarg <argument> to -c.
-
-serverhost <host>
-
Set the hostname for the machine QF-Test is running on. This is
passed to the SUT and usually determined automatically. You
may need to set it when running QF-Test and the SUT on
different machines and you experience troubles with reverse name
lookup.
-
-sourcedir <directory>
(batch mode only)
-
If this argument is given and a report is generated, the files in the report will be
laid out according to the structure of the test-suites relative to this directory
unless
-runlogdir <directory> is also specified. In any case, the directory of a
test-suite listed in the report will only be listed if this argument is specified
and the test-suite is located below this directory.
-
-splitlog (batch mode only)
-
In batch mode split run-logs (see subsection 9.1.4) are enabled by
default and can be turned off via
-splitlog=false. If -splitlog is
explicitely specified without parameter, the default extension for run-logs is
changed from .qrz to .qzp so as to create split run-logs
in ZIP format. The same can be achieved by specifying the name of the run-log
explicitely with the desired extension.
-
-startclean
(calldaemon mode only)
-
With this argument all contexts of the shared TestRunDaemon are cleaned up and released
before running the test.
-
-startsut
(only for internal use)
-
This argument is used to start a client application on a remote host. You should
not work with this argument directly. The standard library
qfs.qft contains
a procedure qfs.daemon.startRemoteSUT which can be use for this
purpose.
-
-stopclean
(calldaemon mode only)
-
With this argument all contexts of the shared TestRunDaemon are cleaned up and released
after running the test.
-
-stoprun
(calldaemon mode only)
-
Stop a running test executed by the Daemon on the given host and port. This
argument can be combined with
-cleanup or -stopclean.
-
-suitedir <dir>
(calldaemon mode only)
-
Specify the (remote) directory where the QF-Test Daemon looks for test-suites.
Use an absolute path when specifying the test to execute, if this argument
is not given.
-
-systemcfg <file>
-
Set the location of the system configuration file (see section 1.4).
-
-tempdir <directory> (interactive mode only)
-
Can be used to specify a temporary directory which is needed on
Windows only for the context sensitive help system. By default
the values of the environment variables
TEMP and
TMP are tried.
-
-terminate
(calldaemon mode only)
-
Use this option, if you want to terminate a running QF-Test daemon.
-
-test <n>|<id> (batch mode only)
-
Without this argument all the top-level tests of the suite
are executed one after the other. Using
-test
<n>|<id>, you can select specific tests. An
arbitrary node located anywhere in a test-suite can be accessed
by its 'Id'. 'Test-case' or 'Test-set' nodes can be
referenced also by its qualified name. Tests on the top level
of the suite can also be selected by their index, the index
for the first 'Test' being 0. You can use this argument multiple
times, even for the same test.
-
-testdoc <directory> (batch mode only)
-
With this argument QF-Test creates both HTML and XML testdoc documentation. If no
directory is given, it is created from the basename of the suite.
-
-testdoc.doctags
(batch mode only)
-
Whether to use the QFS doctag extensions when creating the testdoc documentation.
Default is true, use
-testdoc.doctags=false to disable.
-
-testdoc.followcalls
(batch mode only)
-
By default QF-Test ignores 'Test call' nodes during testdoc creation. With this
argument the target 'Test-case', 'Test-set' or whole test-suite are processed as
if they were part of the original test-suite. Thus it is possible to create a
subset testdoc documentation by creating a dedicated test-suite with 'Test calls' to
the required parts.
-
-testdoc.html <directory> (batch mode only)
-
With this argument QF-Test creates HTML testdoc documentation. If no directory is
given, it is created from the basename of the suite.
-
-testdoc.nodeicons
(batch mode only)
-
Whether to show icons for nodes in the testdoc documentation. Default is true, use
-testdoc.nodeicons=false to disable.
-
-testdoc.passhtml
(batch mode only)
-
Whether to pass HTML tags in comments through to the HTML testdoc.
Default is true, use
-testdoc.passhtml=false to disable.
-
-testdoc.sorttestsets
(batch mode only)
-
Whether to sort test-sets alphabetically.
Default is true, use
-testdoc.sorttestsets=false to disable.
-
-testdoc.sorttestcases
(batch mode only)
-
Whether to sort test-cases alphabetically.
Default is true, use
-testdoc.sorttestcases=false to disable.
-
-testdoc.teststeps
(batch mode only)
-
Whether to list test-steps in the testdoc documentation. Default is true, use
-testdoc.teststeps=false to disable.
-
-testdoc.xml <directory> (batch mode only)
-
With this argument QF-Test creates XML testdoc documentation. If no directory is given,
it is created from the basename of the suite.
-
-threads <number> (batch mode only)
-
Run the same test-suite in a number of parallel threads. Typically used for the
purpose of load testing. One license is required per thread, so normally
-runtime should be specified as well. See chapter 21 for
more information about load tests.
-
-timeout <milliseconds>
(calldaemon mode only)
-
Give a timeout value in milliseconds for the test run when executing a test
through the QF-Test daemon. Default is infinite.
-
-usercfg <file> (interactive mode only)
-
Set the location of the user configuration file (see section 1.4).
-
-variable <name>=<value>
-
To override a system or suite variable definition (see chapter 8) use this argument to set the variable
named
<name> to the value
<value>. Using this argument multiple times
you can define more than one variable.
-
-verbose [<level>]
-
Print progress and status information during a test-run to the console. This is in
particular useful when driving a test via
-calldaemon, because the actual test
usually runs on a different host where you may not be able to observe it. Note
that on Windows you need to use qftestc.exe (instead of
qftest.exe) to see the output. Specifying a level of verbosity is
optional, possible values are all (print all nodes) and
tests (default, only 'Test-set', 'Test-case' and 'Test' nodes are
printed out). Each level can be combined with errors (print error and
exception messages) like tests,errors.
-
-version
-
Print version information and exit.
The filename given in any of the command line arguments -runid <id>,
-runlog [<file>], -runlogdir <directory>, -report <directory>, -report.html <directory>,
-report.name <name>, -report.xml <directory> or -report.junit <directory>,
may contain placeholders of the form %X or +X (the latter must
be used on Windows where '%' is a special character) where X may be any of the
characters listed in the table below. QF-Test will fill in the respective value when
creating the run-log or report. All time values refer to the time the test was
started.
Note When executing multiple test-suites, be sure to include the
base name of the suite in the filename by specifying %b. Otherwise only a single
run-log or report may be written that represents only the test-run of the last
test-suite.
|
|
| Character |
Replacement |
| % |
Literal '%' character. |
| + |
Literal '+' character. |
| i |
The current runid as specified with -runid <id>. |
| p |
The directory of the test-suite relative to -sourcedir <directory>. Expands to the
absolute directory in case -sourcedir <directory> is unspecified and is empty if
-sourcedir <directory> is specified but the test-suite is not located below it.
|
| P |
The absolute directory of the test-suite. May only be given at the beginning.
|
| b |
The basename of the test-suite, exclusive directory and .qft
extension.
|
| r |
The return value (or "exit code") of the test-run (-runlog only). |
| w |
The number of warnings in the test-run (-runlog only). |
| e |
The number of errors in the test-run (-runlog only). |
| x |
The number of exceptions in the test-run (-runlog only). |
| y |
The current year (2 digits). |
| Y |
The current year (4 digits). |
| M |
The current month (2 digits). |
| d |
The current day (2 digits). |
| h |
The current hour (2 digits). |
| m |
The current minute (2 digits). |
| s |
The current second (2 digits). |
|
|
| | Table 32.1: Placeholders in filename parameters | |
So if, for example, you want to save the run-log in a subdirectory
of your test-suite directory called logs and want to
include a timestamp and the exit code, use
-runlog %p/logs/%b-%y%M%d-%h%m%s-%r.qrl
Note It is possible to use %b, %p and %P for collective parameters
like runid or report. This makes sense only when processing a single test-suite. When
processing multiple test-suites, the name of the first test-suite is used.
When run in interactive mode, the exit code of QF-Test is not very useful. It is either
negative if QF-Test fails to start or 0.
In batch mode however the exit code expresses the result of the test
run. Negative values represent fatal errors that prevent the test
from being executed, while positive values stand for errors during
the test run. Note that many systems only support exit codes between
0 and 255, so every exit code may have to be calculated modulo 256,
i.e. -1=255, -2=254 and so on.
The following exit codes are currently defined:
|
|
| Value |
Meaning |
| 0 |
Everything OK |
| 1 |
Warnings occurred during the test run |
| 2 |
Errors occurred during the test run |
| 3 |
Exceptions occurred during the test run |
| -1 |
Unexpected Exception |
| -2 |
Bad command line arguments |
| -3 |
Missing or invalid license |
| -4 |
Errors while setting up the RMI connection |
| -5 |
Errors while loading the test-suite |
| -6 |
The test-suite doesn't contain any tests |
|
|
| | Table 32.2: Exit codes for QF-Test | |
Besides, there are special exit codes when running QF-Test with the
-calldaemon argument:
|
|
| Value |
Meaning |
| -7 |
Daemon could not be found |
| -8 |
Failed to get or create a TestRunDaemon |
| -9 |
Failed to get or create a run context |
| -10 |
The test could not be started |
| -11 |
The test has not ended within the given timeout |
|
|
| | Table 32.3: calldaemon exit codes for QF-Test | |