Jenkins/Hudson (http://jenkins-ci.org or http://hudson-ci.org) are continuous integration build tools. They are meant to control and monitor the build process within a software project. One important step in this build process is automated testing.
There are number of benefits to be gained when integrating QF-Test with Jenkins/Hudson:
As Jenkins and Hudson share a common history, following sections will use Jenkins as representative.
|Install and start Jenkins|
Note For GUI tests, Jenkins must not be configured to run as a service but
within a real user session. On Windows the
.msi installer unfortunately
directly installs Jenkins as service without any further inquiry. Please beware of it
therefore and ensure Jenkins is started as real user process as described below.
To install Jenkins download the
war Archive (which can be found
and start it via
java -jar jenkins.war.
As soon as Jenkins is started its web interface can be accessed via http://localhost:8080. It should look like the following:
||Figure 21.4: Jenkins after start-up.|
|Requirements for GUI tests|
GUI testing requires an unlocked, active desktop. That is the only way to ensure that the SUT behaves the same as if a normal user interacts with it. Chapter Hints on setting up test-systems contains useful tips and tricks to set-up the Jenkins/Hudson process.
Jenkins allows execution of tasks on remote machines. This is of course also relevant for GUI testing. Due to its nature GUI tests are typically not intended to run on the central build server. In addition, tests might need to be executed for different environments, operating systems and SUT versions.
On a remote machine, a Jenkins slave agent needs to be launched in order to connect to the Jenkins server and wait for jobs to be processed. As described in the Jenkins documentation, there are several options to launch this agent, but for the GUI tests to properly work the only possible launch method is to use Java Web Start.
For GUI tests it is vital to have an active, unlocked user session. Therefore it is not possible to start the agent via a windows service but a real (test) user must be logged in (e.g. via auto login) using Windows Autostart to launch the Jenkins agent. Furthermore screen locking needs to be disabled.
NotePlease see also FAQ 14 for more technical background details.
|Install QF-Test Plugin|
The QF-Test Plugin enables QF-Test to interact with Jenkins. To install the plugin open the Jenkins dashboard and navigate via "Manage Jenkins" to "Manage Plugins". Select the QF-Test Plugin from the "Available" tab. When installing the QF-Test Plugin the JUNIT and HTML-Publisher Plugin will also be downloaded automatically, in case they were not already installed. Finally restart Jenkins to complete the installation. Now the QF-Test Plugin will show up under the Installed tab, as shown in Figure 20.2.
NoteFor an in-depth look at the plugin and all of its available features, please have a look at the Youtube Video about the installation and configuration of the plugin.
NoteJenkins will automatically use the latest installed version of QF-Test. In case you want to use a different version, you can provide its path under the QF-Test section in the Jenkins configuration (Manage Jenkins -> Configure System).
||Figure 21.5: Install QF-Test Plugin.|
|Configure a job for QF-Test|
To create a new job open the Jenkins dashboard and click "New Item". The standard project type is "Freestyle project". Once the job is created it has to be configured.
The first thing that needs to be done is to change the workspace of this job to the
directory that contains your test-suites. It can be changed in the "Advanced Project
Options" under "Advanced..." where you check "Use custom workspace" and specify the
In the "Display Name" field you have the option to provide a name to be shown instead of your job name, which is typically left empty.
||Figure 21.6: Set custom workspace.|
Now proceed to the "Build" section and add the "Run QF-Test" build step by choosing
it from the "Add build step" drop down. In the left field you can either enter
the name of a test-suite or a folder containing test-suites (e.g. your workspace
directory being "C:\MySuites\" and the path of the test-suite you want to
run being "C:\MySuites\qftest\mysuite.qft", simply enter
qftest/mysuite.qft in the first field). In case you enter a
directory, every test-suite in this directory will be run. In the right field
you can enter additional Command line arguments.
||Figure 21.7: Add build step to run QF-Test|
The advanced options of the "Run QF-Test" build step are optional.
The first option allows you to provide a dedicated QF-Test version for this job.
The next option lets you change the directory for the temporary reports. The QF-Test Plugin will create a folder called "qftestJenkinsReports" in the selected workspace and save the reports and logs for the post-build actions.
Finally, there is an option to run your test-suites on a daemon. You need to enter the deamonhost and port to do so. To learn more about the daemon mode please see section 17.2 of the manual.
||Figure 21.8: Configure build step advanced options.|
Next the post build actions need to be configured to process test results and archive
QF-Test run-logs and HTML reports.
To archive the run-logs add the post-build action "Archive the artifacts" and for "Files to archive" enter this path:
the HTML Reports add the post-build action "Publish HTML reports". In the HTML directory
enter this path:
qftestJenkinsReports/$JOB_NAME/$BUILD_NUMBER/html/ . Also
change the index page from
report.html. You can
keep past HTML reports if you check the respective box.
Finally add the
post-build action "Publish JUnit test result report" and for "Test report XMLs" enter
NoteThere will be warnings that the path you entered doesn't match anything, because of the environment variables in the path. These warnings can be ignored.
||Figure 21.9: Configure post build steps.|
NoteThis plugin supports concurrent builds, as long as your licence allows enough QF-Test instances.
NoteYou can use the additional command line argument "-dbg" to get more debug information in the console output of your build.
|Last update: 01/09/2018
Copyright © 1999-2019 Quality First Software GmbH