Manual
 

Bugs fixed:

• Web Some websites containing custom HTML elements were not testable with CDP connection mode.
• Web In CDP connection mode whitespace was missing from fetched text in some cases.
• Web In CDP connection mode, the keyCode property of key events generated from text is now set correctly.
• Web In CDP connection mode the location of elements in nested IFrames was calculated wrongly.
• Web The 'Name of the browser window' attribute of 'Wait for document to load' nodes was ignored.

New features:

• Support was added for testing applications based on Java 17.
• SWT QF-Test now supports tests for applications based on Eclipse/SWT 4.20 alias "2021-06".
• Performance in CDP connection mode with dynamic content updates has been improved.
• Support for JxBrowser 7.16 was added.
• Electron QF-Test now supports applications built with Electron 14 or newer, if the @electron/remote is bundled with the app.
• QF-Test now supports testing with Opera 77.

Bugs fixed:

• Web Validity of text input to web applications via single events has been improved through additional explicitly defined keycodes.

• Jython issues with character encoding have been reduced and it is now possible to treat Jython literals as 16-bit unicode string which is the natural representation for Java and thus QF-Test. For compatibility reasons the new option Literal Jython strings are unicode (16-bit as in Java) is turned off by default if QF-Test encounters an existing system configuration.

  Please see subsection 12.3.4 for detailed information about why the option should be turned on and how to trouble-shoot possible issues. Chances are high that your Jython scripts will simply work and string handling will become much cleaner. If not, either undo the option change or fix the resulting incompatibilities. We had to do the latter in only a handful of places in our over 1600 test-suites, some of which date back over 20 years. The section Trouble shooting Jython encoding issues explains the most common pitfalls and of course our support is always there to help.

• The default folder name of the Firefox profile when executed in WebDriver connection-mode has been renamed from mozProfile to firefoxProfile and it is now used in place instead of copying it to a temporary directory. This behaviour is now consistent with using QF-Driver but has the side-effect that preferences from one test execution are preserved for the next execution and might have to be overwritten during the next browser start. To restore the previous behavior, set the OPT_WEBDRIVER_COPY_MOZPROFILE option to true before starting the browser.
• The procedures in the package qfs.utils.ssh of the standard library qfs.qft and the underlying Jython module ssh have been updated to default to RSA public key authentication with the default private key file /.ssh/id_rsa instead of DSA which is no longer supported by most current ssh servers.
• The option Create compact run-log is now deactivated by default in interactive mode. Existing system configurations are not affected and the option has no effect in batch mode where compactification is controlled via the command line attribute -compact.

New features:

• Web The newly added browser connection mode CDP-Driver supplements QF-Driver and WebDriver for controlling Chromium based browsers via the Chrome DevTools Protocol. By talking directly to the browser without the intervening WebDriver protocol, speed, stability and feature set of CDP-Driver are on par with QF-Driver (and that's after the QF-Driver performance boost, see below). In addition, while QF-Driver is limited to Chrome on Windows, CDP-Driver now supports Google Chrome, Microsoft Edge and Opera on Windows, Linux and macOS, so this is a real game-changer for web test automation with QF-Test.
• Web The performance of web tests with QF-Driver for Chrome has been significantly improved. Observed speed-up ranges from 10% to over 500%.
• The user interface of QF-Test has been cleaned up and streamlined, using a uniform flat look with fewer lines and beautiful new icons that still maintain the existing image language and are immediately recognizable. The HTML manual and tutorial as well as report and test documentation have also received a face-lift.
• Use of international character sets in Jython scripts is now straightforward. If the new option Literal Jython strings are unicode (16-bit as in Java) is turned on, literal strings (explicitly specified string constants like "abc") in Jython scripts are treated as 16-bit unicode and are thus equivalent to strings in Java and the other QF-Test scripting languages. Please see subsection 12.3.4 for detailed information.
• Web Detection of errors in the browser console has been improved and, depending on the option How to handle errors in a web application, they are logged in the QF-Test run-log. Besides, the new procedure qfs.web.browser.settings.setTerminalLogs in the standard library qfs.qft can be used to define if and how messages from the browser console are to be shown in the QF-Test terminal.
• Web The embedded Chrome browser used for QF-Driver mode has been updated to CEF version 89.
• Web The bundled GeckoDriver was updated to version 0.29.1.
• Web Support for JxBrowser 7.14 and 7.15 was added.
• Web QF-Test now supports testing with Opera 76.
• The JRE ditributed with QF-Test has been updated to Zulu OpenJDK version 8_292.
• The quickstart wizard now has its own toolbar button. This - or any other unwanted toolbar button - can now be removed from the toolbar via a right-click popup menu.
• The 'Wait for absence' node now has a dedicated entry in the »Insert« menu and its logic when applied to sub-items has been simplified: Execution of the node is successful if either the parent component of the sub-item is absent or the sub-item itself.
• When running a test with the command line argument -verbose [<level>], QF-Test now expands variables in node names for console output also.
• The default setting for the available Java memory for QF-Test has been increased to 1024 MB. The configuration of existing QF-Test installations is not affected.
• Swing Sub-items of Swing JComboBox components can now be addressed relative to the JComboBox without requiring identification of the popup list.
• Linux The ability to bring windows of the SUT to the foreground when needed and set the input focus is crucial for automated testing. On Linux QF-Test now uses an updated, more reliable method to bring a window on top regardless of desktop settings if the option Force window to the top when raising is not deactivated.
• Exception messages in the run-log or an error dialog are now displayed using word-wrap to break long lines. This can be turned off via the option Wrap lines in exception messages.

Bugs fixed:

• Installing a resolver via the generic method resolvers.addResolver() did not work in SUT scripts with the JavaScript language.
• Web The browser zoom level is now reset to 100% when clearing the browser cache.
• The procedure qfs.swing.startup.startWebstartSUT now ensures proper quoting of the jnlp argument for use on the command line of Linux systems in order to avoid side-effects from special characters it might contain.
• Windows-Tests The qfs:label extra feature for elements within a TabPanel of a native WPF application was not determined correctly.
• Web Since QF-Test version 5.2.2 in very special cases elements in a web page were mistakenly considered to be invisible.
• Mac QF-Test now runs again on macOS versions older than 10.14.
• Resolution of the qfs:label extra feature for a label located above the target component is now slightly more tolerant about horizontal alignment.
• Swing Fast replay of several mouse clicks onto the same location of a Webswing application could accidentally create double clicks when redirecting events through the browser.

New features:

• SWT QF-Test now supports tests for applications based on Eclipse/SWT 4.19 alias "2021-03".

Bugs fixed:

• In special cases, unluckily placed comment nodes could lead to unwanted side-effects during test execution like a 'Setup' or 'Cleanup' node run just for a comment.
• Improved timing for client shutdown when ending batch mode execution.
• Web In WebDriver connection mode mutations on a website might have gone unnoticed in case the page contained too many elements.

New features:

• Support was added for testing applications based on Java 16.
• Web QF-Test now supports testing with Opera 74.
• Web The bundled GeckoDriver was updated to version 0.29.0.
• The new shortcuts [Ctrl-/] or [Ctrl-7] can be used to insert a new comment node in the test-suite tree.

Bugs fixed:

• Web Several performance bottlenecks for web tests have been fixed, most notably for Firefox on Linux with WebDriver.
• Web In very rare situations, Chrome was closed in case a JavaScript-execution intervened with a frame reload.
• Web Delayed attachment of a shadow DOM is now recognized correctly.
• Swing In special cases the improved event synchronization for Swing might have missed some events, causing slower test execution.
• If a 'Test-case' with the 'Expected to fail if...' attribute set to true does not fail, it should be treated as an error. That error was incorrectly reported as an expected error itself.
• The sort order of parameters is now also automatically applied when changing the target procedure of a 'Procedure call' node via the chooser dialog.

New features:

• SWT QF-Test now supports tests for applications based on Eclipse/SWT 4.18 alias "2020-12".
• Support for JxBrowser 7.12 was added.

Bugs fixed:

• Due to wrong file permissions on Linux machines, Jython scripts would fail when running QF-Test as a user other than the one that installed it.
• The JRE distributed with QF-Test has been changed back to Zulu OpenJDK. The version remains 8_275.
• In some cases, showing an alert box on a web page lead to a browser deadlock.
• Several details for the Webswing integration have been improved, including correct filtering of KeyEvents, focus handling for embedded JavaFX components and cleaner separation of client processes in the demo test-suites.
• Daemon connection might have failed during handshake if different java version where used on client and server side.

• Testing applications running on Java 7 is no longer supported.
• Windows-Tests Many procedures in the standard library package qfs.qft.autowin have been deprecated in favor of the much better suited Windows engine.
• Due to the updated JRE in the QF-Test installation, graphical elements and graphs in the PDF client may be painted with slightly different anti-aliasing. This can lead to errors in 'Check image' nodes. Given that such problems cannot be ruled out for future JRE updates you should set the 'Algorithm for image comparison' attribute of affected nodes to "algorithm=similarity;expected=0.98".

New features:

• QF-Test now supports integrated testing of Swing and JavaFX applications that are displayed in a browser using the technologies Webswing or JPro. See chapter 17 for an explantion of the concepts and the demo test-suite for Webswing, accessible via the menu »Help«-»Explore example test-suites...«, entry "Webswing SwingSet Suite".
• Mac QF-Test is now notarized by Apple and thus starts on modern macOS systems without showing a warning message.
• Web QF-Test now also supports testing with the Microsoft Edge browser on Linux.
• The JRE distributed with QF-Test has been updated to Liberica OpenJDK version 8_275.
• Web The embedded Chrome browser used for QF-Driver mode has been updated to CEF version 85.
• Groovy has been updated to version 3.0.6.
• Jython has been updated to version 2.7.2.
• Web Support for JxBrowser 7.11 was added.
• Web The embedded GeckoDriver has been updated to version 0.28.0.
• Web QF-Test now supports testing with Opera 72.
• Web Device specifications for many current mobile devices have been added to mobile emulation mode.
• The JUnit library has been updated to version 5.7.0.
• Web On Windows systems with a scaled display QF-Test now starts QF-Driver browsers in compatibility mode so that scaling is transparently handled by Windows and tests work very similar to unscaled mode except for image checks.
• It is now possible to specify options on the command line via the argument -option <name>=<value>.
• QF-Test command line arguments can now contain "." and "-" characters in arbitrary places and upper or lower case characters at will.
• When testing Java applications, QF-Test can now intercept calls that open a native browser window in order to launch a browser controlled by QF-Test for the given URL. An example is provided in the demo test-suites "CarConfig test project" and "CarConfig JavaFX test project", accessible via the menu item »Help«-»Explore sample test-suites...«.
• The new procedure qfs.utils.waitForClientOutput in the standard library qfs.qft assists in synchronizing with terminal output of the SUT.
• Several more node conversions are now possible.
• The 'Server HTTP request' step now also supports the PATCH method.
• The two new procedures qfs.utils.sendKey and qfs.utils.sendText in the standard library qfs.qft can be used to enter text into the currently focused element of the active window.
• The 'No events were recorded' dialog can now be suppressed via the new option Show message if no events were recorded.
• When merging run-logs in batch mode, the command line argument -mergelogs-masterlog [<file>] can now be combined with -mergelogs-mode [<mode>] set to "append". The appended run-logs will be stored as externalized thus minimizing memory use both during merging and when opening the resulting run-log.
• Similar to Jython, script steps for Groovy and JavaScript can now use common exceptions without an explicit import.
• When propagating the parameters of a callable node to its callers, there are now explicit choices for whether to add missing parameters, remove extraneous parameters and/or update the sort order.

Bugs fixed:

• Opening a run-log with an automatic re-run still in progress could lead to an exception.
• Encrypted connections to the QF-Test daemon are now also supported by the external daemon-API.
• When generating reports, thumbnail images were created even if -report-thumbnails was not specified.
• The 'Unit test' step now correctly supports the self.assertEqual call in Jython scripts.
• Text input on Swing and JavaFX components was slowed down if a browser embedded into Java was detected.
• Swing Event synchronization under heavy load for Swing based applications has been improved.
• Swing Text input with single events on a Swing JTextArea now handles newline characters correctly.
• Windows-Tests Elements of Windows applications may not have been scrolled visible correctly for hard events and image checks.
• Web With a browser in WebDriver mode a failed frame focus switch could lead to a StackOverflowException.
• Web In some cases the MSEdgeDriver was not downloaded correctly.
• Web Checks on elements inside a shadow DOM could not be recorded.
• Web Soft (invisible) hyphen characters are now implicitly ignored.
• Electron In some cases, dialog boxes from Electron where displayed empty.
• JavaFX The visibility of JavaFX components embedded in Swing was sometimes not determined correctly.
• SWT For SWT version 4.17 on Windows highlight rectangles on Menus were not restored correctly.

New features:

• SWT QF-Test now supports tests for applications based on Eclipse/SWT 4.17 alias "2020-09".

Bugs fixed:

• In rare cases QF-Test could crash during image compression if memory was tight.

New features:

• Web QF-Test now supports testing with Opera 70.
• Web The embedded GeckoDriver has been updated to version 0.27.0.
• Web For WebDriver based tests with Chrome/Chromium, site isolation is automatically deactivated.
• Web Support for JxBrowser 7.10 was added.
• A link to the JavaScript documentation was added to the help menu.

Bugs fixed:

• The 'Unit test' node now also searches for JUnit 5 (Jupiter) tests on the classpath.
• Electron In some cases, native menu clicks on Electron applications were not properly recorded.
• Web The cache of Chromium based browsers might not have been cleared properly.
• The option Enable 'Local variable' attribute by default will now be taken into account when pasting a copied 'Procedure' node as a 'Procedure call', for node conversions in general and also when recording checks.
• Web A deadlock could occur if embedded browser containers (e.g. JxBrowser) were removed and added at the same time.
• Adding 'Comment' nodes to a procbuilder configuration file could break procedure recording.
• Mac On macOS, JVM options (starting with "-J-") are now handled correctly.
• Windows-Tests Text input in Windows applications may not have worked properly when the AltGr key was involved.
• Web When working with dialogs in headless browser tests, sometimes the invisible dialog was not closed properly.
• Web A ClassNotFoundException could be triggered when an SWTBrowser was under test.
• When creating procedures via the Procbuilder using FORCECREATION the separation dots for the package structure were replaced by underscores.
• When generating procedures via the Procbuilder values from the 'Extra features' of parent and grand parent nodes can now be used as fallback.

• Due to the updated JRE in the QF-Test installation, graphical elements and graphs in the PDF client may be painted with slightly different anti-aliasing. This can lead to errors in 'Check image' nodes. Given that such problems cannot be ruled out for future JRE updates you should set the 'Algorithm for image comparison' attribute of affected nodes to "algorithm=similarity;expected=0.98".

  The JRE update can also cause communication problems between QF-Test and the QF-Test license server in case the license server is run with a very old Java version that cannot cope with the key length required for SSL in current Java versions. In that case it is best to update the license server to the current QF-Test version and use its included JRE.

• The library jniwrapper is no longer loaded by default because our old jniwrapper version crashes QF-Test on newer JDKs. Modules with native dependencies like autowin have been rewritten to no longer depend on it and all references to jniwrapper have been removed from the standard library qfs.qft.

  If you still have script nodes in your test-suites that depend on jniwrapper you should try to reimplement these in order to remove that dependency. Please get in touch with our support if you need help.

  As an interim solution you can get such scripts to work again (on older JDKs where jniwrapper does not crash) as follows:

  • Copy the files from misc/jniwrapper in the QF-Test installation directory to qftest in the QF-Test plugin directory. To locate those directories, open the »Help«-»Info« dialog and look for dir.version and dir.plugin on the 'System info' tab.
  • Add either a Jython server script to your startup sequence with
    

    from com.jniwrapper import DefaultLibraryLoader
    from java.io import File
    DefaultLibraryLoader.getInstance().addPath \
      (File(rc.lookup("qftest", "dir.plugin") + "/qftest"))

    or the following Groovy variant
    

    import com.jniwrapper.DefaultLibraryLoader
    DefaultLibraryLoader.getInstance().addPath
      (new File(rc.lookup("qftest", "dir.plugin") +
           "/qftest"))

• The ChromeDriver library for old Chrome versions (older than 72) is not bundled wit QF-Test anymore.
• Testing applications running on Java 7 ist still supported in this QF-Test version. However, support for Java 7 has been deprecated and will be removed in QF-Test version 5.2.

Video: QF-Test 5.1.0.

New features:

• Support was added for testing applications based on Java 15.
• The JRE ditributed with QF-Test has been updated to Zulu OpenJDK version 8_252.
• Web Recording and replay of tests for embedded browsers has been significantly improved.
• JxBrowser is now supported in version 7, embedded into Swing, JavaFX or Eclipse/SWT applications.
• Electron Support was added for handling native dialogs in Electron applications.
• Web QF-Test now supports testing with Opera 69.
• Web Support for the web framework Qooxdoo has been updated for Qooxdoo version 6.
• HTML reports can now be customized via JavaScript in the form of a user.js. See subsection 21.1.4 for details.
• The automac module now provides methods for simulating keyboard and mouse events. See chapter 48 for further information.
• The root node of a test-suite now also has a 'Name' attribute that is shown in the tree.
• The new option Enable 'Local variable' attribute by default determines, whether the attribute 'Local variable' gets pre-activated in newly created nodes.
• The procedure qfs.utils.dragAndDrop in the standard library qfs.qft has a new optional parameter eventDelay to control replay speed.
• It is now possible to convert a 'CSV data file' node into an 'Excel data file' node and vice versa.
• Test-suite tabs can be moved left or right using the keyboard shortcuts [Shift-Ctrl-Page up] and [Shift-Ctrl-Page down].

Bugs fixed:

• Web The procedure qfs.web.browser.settings.setLocale now also works for WebDriver connections to Chromium based browsers.

New features:

• SWT QF-Test now supports tests for applications based on Eclipse/SWT 4.16 alias "2020-06".
• The included jsch.jar library used by the qfs.utils.ssh package in the standard library qfs.qft has been updated to version 0.1.55 in order to add support for modern Linux systems like Ubuntu 20.

Bugs fixed:

• The included WebP image compression library was rolled back to version 1.0.0 to avoid incompatibilities.
• Web Component recognition might have failed when web components had non-integer sizes.
• In rare cases calling rc.callProcedure inside the parameters of a 'Procedure call' node could lead to the global variables in the variable stack getting lost.
• The special syntax ${qftest:engine.<componentid>} that can be used to determine the engine of a component now also works if <componentid> contains a '@', '%' or '&' character.
• Selecting a single value in a run-log's error list and using "Set value as filter" twice caused an ArrayIndexOutOfBoundsException.

New features:

• The WebP image compression library has been updated to version 1.1.0.
• Keyboard event input in JX Browser is now more stable.
• Contrast of toolbar icons has been improved especially for disabled buttons.
• QF-Test now supports testing with Opera 68.

Bugs fixed:

• Angular 9 is now auto-detected correctly.
• The 'CSV data file' node now correctly handles UTF-8 encoded files with BOM that start with an encapsulated complex expression.
• Windows-Tests The 'Start windows application' node can again attach to a client via a given class name (-class) in the 'Window title' attribute.
• Error handling and retry for automatic downloads of WebDriver libraries has been improved.
• Windows-Tests Recording elements with a flat hierarchy did not work.
• The PDF client is now able to check a Text component which contains only null "\u0000" characters and treats it as an empty String.

New features:

• Windows-Tests A new demo test-suite was added for the Windows 10 Calculator.
• QF-Test now supports Opera 67 with Operadriver 80.0.3987.100.
• SWT QF-Test now supports tests for applications based on Eclipse/SWT 4.15 alias "2020-03".

Bugs fixed:

• Windows-Tests When recording components for the whole window, elements within a WPF TabPanel were omitted.
• The qfs.database.executeSelectStatement procedure now works again for databases requiring an explicit db.commit() statement.
• When executing a 'Server HTTP request' node, the returned body was mistakenly not stored in a variable in case of a server error.
• JavaFX Processing JavaFX images in order to calculate a hash value could cause a NullPointerException; to get printed to the terminal.
• Windows-Tests Fixed a bug where a TextField in a Windows application might not get cleared before input.
• Windows-Tests Occasionally a ModalDialogException might get incorrectly thrown in WPF Windows applications.

Note For a detailed list of new features please see the release notes below for QF-Test versions 5.0.0.

The following major new features have been implemented for QF-Test version 5:

Changes that can affect test execution:

• The 'Server HTTP request' node now throws an exception if the status code is greater than or equal to 400. This behavior can be changed with the new attribute 'Error level if status code >= 400'.
• The options Connect via QF-Test agent and Instrument AWT EventQueue are no longer saved in the system config file and can only be changed at runtime via a script. See section 37.9 for detail.

Software that is no longer supported:

Note Please see section 1.1 for a detailed list of system requirements and supported technology versions.

• Testing of applications based on Java 6 is no longer supported.

New features:

• With the new Windows engine QF-Test can now test native Windows applications.
• Support was added for testing applications based on Java 14.
• The new 'Comment' node can be used to improve the structure and readability of test-suites and run-logs.
• It is now possible to download a file via the new 'Save response to file' attribute of the 'Server HTTP request' node.
• Windows-Tests A package for Windows applications has been added to the standard library qfs.qft.
• On Windows 10 QF-Test is now correctly displayed at scaled high resolution displays.
• Web In the manual the chapter Web testing has been revised and a section (subsection 46.1.3) describing the procedure qfs.web.ajax.installCustomWebResolver of the standard library has been added.
• The option Show message dialog after now has a setting to show a message dialog also when a test run finishes successfully.
• If the result dialog gets shown after a search the search dialog is now closed automatically.
• The new procedure qfs.util.click in the standard library qfs.qft can be used to click at an arbitrary position on the screen.
• You can now create an electron start sequence in the quickstart wizard that automatically detects the required ChromeDriver.
• When copying a 'Procedure call', 'Test call' or 'Dependency reference' node the name of the target node is now also copied as text to the clipboard.
• Web The bundled GeckoDriver has been updated to version 0.26.0.
• Web On Windows, tests with the Microsoft Edge 78 and newer are also possible in headless mode.
• Web QF-Test now supports Opera 66 with Operadriver 79.0.3945.79.
• The new variable engine.$(componentId) in the qftest special group makes it possible to find out which GUI-engine a component belongs to.
• The project tree view in QF-Test now uses a natural sort order, respecting indexes and cases.
• Data for several new mobile devices was added to the Mobile Emulation setup in the quickstart wizard.

Bugs fixed:

• An image might have been removed from the run-log in low-memory situations.
• Fixed a sporadic exception that could appear when creating a test-suite from a run-log.
• 'Finally' nodes inside a 'Try' now get executed even when an instant rerun gets triggered from within the 'Try' node.
• ImageWrapper methods now log warnings whenever the method fails.
• In very rare cases the [Ctrl] key might accidentally have stayed in pressed state after finishing replay.

B.5	QF-Test version 4

B.6	QF-Test version 3

Last update: 7/21/2021 Copyright © 1999-2021 Quality First Software GmbH