Checklist: Requirements GUI Test tool for Java and/or Web Requirements Weight Other tool 1 Other tool 2 Other tool 3 Features Supported platforms Multiple technolgies in one system Regression and load tests for Java and Web GUIs Java: Swing, JavaFX. AWT, SWT, Eclipse plugins, RCP, Applets, JavaWebStart, RIA, ULC, CaptainCasa, JavaFX SubScene components. Web: Internet Explorer, Firefox, Chrome, Safari, Edge (HTML 5, AJAX: Angular (up to Angular 5), Smart (GWT), GXT/ExtGWT, ExtJS, ICEfaces, jquery UI, jqueryeasyui, Kendo UI, PrimeFaces, Qooxdoo, RAP, RichFaces, Vaadin, ZK.). Further toolkits can be integrated with little effort if required, e.g. SAP UI5, Siebel Open UI and Salesforce. Headless browser for Chrome and Firefox. Suitable for hybrid systems with several GUI technologies as well as embedded browser components (JavaFX, WebView, JXBrowser, SWT-Browser by "generic components") Your rating PDF documents Supported OS Testing principle Testing PDF documents like a normal application (with textual and graphical checks for individual elements). Swing and JavaFX: Windows, Linux, Unix, macos. SWT: Windows, Linux-GTK; Solaris-GTK on request. Web: Windows, Linux, macos. Capture/Replay for quick test creation. Extensive possibilities for sequence control, modularization and advanced scripting. Checkliste_QF-Test 1
Test structuring Component recognition, robustness of tests, timing synchronization Reusability, maintenance effort Data-driven testing Keyword driven testing, behavior driven testing Run-logs, test documentation and reports Error analysis Clear and concise due to graphical depiction of the test sequence as tree structure. The layered test suite architecture enables modularization in large tests. Masters complex elements like dynamic trees and tables. Sophisticated recognition mechanism leads to extraordinary maintainability, especially via naming. Tests tolerant to GUI changes. Tests should be as fast as possible, not not faster - otherwise the SUT will be overwhelmed and produce errors that are not. High degree of reusability (due to stable component recognition). Low maintenance effort because of automatic creation of basic procedures for basic component access and modular structure. Data drivers allow for flexible integration of internal or external data, e.g. from CSV, Excel files or database access. Other data sources can be integrated via script extension. So that the test case is executable in multiple test data variants. Execution of test cases from a planning document or the specification, e.g. Excel or another test management tool (Test creation not directly in QF- Test). Creation at the push of a button or via command line execution. Detailed run-logs contain all information for an effective error analysis (including screen shots of the application once an error occurs). Summary reports in HTML and XML. Detailed run-log. Debugger functionality (including breakpoints and analysis of variables). Checkliste_QF-Test 2
Expandability, scripting Test management Continuous integration Maturity Usability Accessibility Installation Duration of orientation Previous knowledge required Embedded Jython, Groovy and JavaScript including comprehensive libraries allow direct access to the Java API and make function extensions and complex checks/actions realizable. Full access to all objects of application (SUT). Execution of Java code and Javascript codes directly in the browser. Pragmatic small scale solution included. Dedicated integrations are available for HP ALM QualityCenter, Imbus TestBench, Qmetry, SQS- TEST/Professional, Klaros (from Verit) and Testlink (Open source). Flexible integration with build tools: Ant, Bamboo, Bugzilla, ClearCase, ClearQuest, CruiseControl, CVS, Git, Hudson, JIRA, Mantis, Maven, Mercurial, Quality Manager, load tests with NeoLoad, Scapa TPP, Subversion, SVN, TestTrack Pro (Plugin for Jenkins available) High degree of maturity, available at the market since 2001. More than 1,000 customers worldwide with over 7,000 licenses. Anonymous download of free demo version. Free evaluation license also extensible including fee support and webinar. Upgrade to commercial version via license file. Very fast, small download files available for Windows and Unix systems. No license registration required for demo version. Good and clearly structured self-starter tutorial, first tests can be run after about one hour. Programming skills are not necessary for standard usage, but helpful for advanced script implementation (Standard scripting languages Jython, Groovy and JavaScript). Checkliste_QF-Test 3
User-friendliness Documentation Test execution Test preparation Easy to use, highly intuitive. Clearly structured tree view for comfortable test-case handling, e.g. Copy/Paste and Drag&Drop. Capture/Replay for a quick start. Extensive manual, selfstarter-tutorial, videos, standard procedures, fulltext search, online help via right-click in QF-Test, general and technical FAQ. Quickstart wizard for easy application integration which is only necessary for applications with SWT components, AJAX-toolkit is recognized automatically, no browser plug-ins necessary. Test case preparation Test scripts Verification points Image comparison / testing Object mapping Tests in batch mode Dependency management for test preparation enables independently executable test cases including automatic error handling. Concise and comprehensible, effortlessly extendable even during debugging. Recurring action can be modularized in procedures, also parametrizable. Maintainability increased, adaption just required in procedures. Structuration of tests in different test cases. Merge into test sets and constuction of different test scenarios possible. Direct recording of standard checks. Customized checks possible via scripting. Multiple algorithms also for fuzzy image comparison - convenient diff view for validation on deviations. Component information is saved in a specific area of the test suite during recording and can be edited anytime. Mechanism for automatic updates available. Yes, extensive options for configuration via command line parameter, can be integrated into build environments. Option to start additional steps after a test run through a return value. Checkliste_QF-Test 4
Remote execution Error handling Test modification Teamwork Formats and version management Generic components Licenses and prices Product variants License types License price License server Support and service Languages Test execution on a remote machine which might run in a daemon mode. Automatic error handling, which grants continued execution of the whole test. Tests can be edited fully and modified at arbitrary position. Test debugger offers valuable assistance. Modular structure of tests and an import mechanism for the component information enable smooth team work. Both test suites and run-logs are XML files and thus well-suited for e.g. version management. Recognition of technical components (buttons, test fields ) is optimally abstracted, helps to reuse tests over more technologies and to use common actions without the necessity of capturing every single field. QF-Test is available in different product variants of the four GUIs (Swing, FX, SWT and Web). Choose just thse GUI technologies that are actually required. Developer and runtime licenses: all licenses are floating (freely transferable within a projetct) and cannot expire. Purchase, leasing and upgrading possible. License server for cross network usage available. Developer license from 1,995 EUR, runtime license from 995 EUR. There are considerably dicounts for higher license or technology numbers or later additional purchase. Available or license management can be handled decentrally. Completely in English and German (Documentation, videos, training, support). Checkliste_QF-Test 5
Support During evaluation Support with maintenance (commercial support) Training, consulting Community / discussion forum Directly from the developers and testers of QF- Test. Free support via telephone / email, active contact and support offer, free evaluation webinar. Maintenance contract (support + upgrades) annually extendable: very short response times - individual support via email, telephone, webinar. Available, both at customers site and regular training sessions at QFS, also via webinar. Mailing-List available. Short response times, also from QF-Test developers - large list archive. Total result Checkliste_QF-Test 6