One Digital Integrated Digital Assurance Automation Framework Vinod Sundararaju Antony, Director Projects Vinoth Balasubramanian, Senior Manager Projects Ankur Joshi, Manager Business Development Cognizant Technology Solutions 1
Defining One Digital One Digital - Integrated Digital Assurance Automation Framework provides a common automation framework leveraging open source toolsets across digital platforms (web &mobile) & testing phases (functional, performance, security and accessibility). Optimized Team Reduced Effort Intelligence Faster Time to Market Eliminates need for Reduced script Brings in intelligence in the Reduced test cycle time due separate teams across design and form of self-diagnostics and to parallel execution (cross testing types maintenance effort self-recovery browser, across testing types) 2
Before One Digital Implementation Sprint N Sprint N + 1 Day 1 to Day 15 Hardening Sprint Release Coding, Unit Testing & Build Deploy Defect Fixing, Unit Testing & Build Deploy Retrospection & Demo Test Script Design Functional Test Execution Fix only functional defects Sequential Execution Defect Re-Testing Test Script Design Hardening Team (performance, security, accessibility ) Performance Testing Security Testing Accessibility Testing Release - Day 20 Separate and duplicated effort due to different tools Wait Time for Non-Functional Execution Separate Teams Legend Performance Security Functional Accessibility 3
After One Digital Implementation Sprint N Sprint N + 1 Day 1 to Day 15 Hardening Sprint Release Coding, Unit Testing & Build Deploy Post Build Stability Test Script Design Functional Test Execution Defect Fixing, Unit Testing & Build Deploy Fix only functional Fix only defects functional Fix Functional defects & Non- Functional Defects Sequential Execution Sequential Execution Parallel Execution Defect Re-Testing Test Script Design Hardening Team (performance, security, accessibility ) Separate and duplicated Separate and duplicated effort due to different tools effort due to different tools Single Test Script Design Performance Testing Security Testing Accessibility Testing Eliminate/ Minimize Hardening Sprint to Reduce Cycle Time Retrospection & Demo Release - Day 15 Wait Time for Non-Functional Execution Wait Time for Non-Functional Execution Early Defect Detection Separate Teams Separate Teams One QA Team Legend Performance Functional QA Team Security Accessibility 4
One Digital - idaaf Architecture Digital Package CI Based Execution Execution platforms Accessibility Driver Script Security Digital Controller Event Handler Test Script Package Test Script Performance Support Libraries Common Libraries for Web, Web service, Mobile Service Validation Libraries Mobile Error & Exception Handler Reporting Libraries Application Under Test Self Diagnostic and Self Healing Module Reports Enriched UI Report using Extent Report API 5
One Digital Behind the scenes Digital Package Execution Platforms Accessibility, performance, security, mobile automation methods are automatically called after driver navigates to new page Method Digital Controller Event Handler Sample of event handler methods. Method to check the accessibility guide line afterclickon(webelement arg0, WebDriver arg1) { accessibility1(); accessibility2(); runjmeter(); securitytest();// ZAPProxy settings,run runwebservice(); } Method: Test Package Test Script 1. Method for Webdriver initiation and Eventfiring driver initiation(for web/mobile) 2. Passing the control of webdriver to eventfiring driver so that event handlers can fire the action based on the event signal eventdriver = new EventFiringWebDriver(driver); eventdriver.get("http://newtours.demo aut.com/"); handler = new EventHandler(); eventdriver.register(handler); Reports 6
7
Sample Reports 8
One Digital Coverage What is covered? Limitations of the Framework Workarounds Functional / Regression Mobile Web Testing Early Performance Testing Single User Performance Test Load Test Security Testing (Vulnerability Assessment) Accessibility Testing (WCAG guidelines) Accessibility Verifying that text alternatives are meaningful can only be accomplished by manual review Keyboard access can only be confirmed by a tester taking the steps to evaluate those functions Performance Jmeter is not designed to long-polling application testing however you can test it with certain restrictions Accessibility Using NVDA speech viewer, content can be verified manually Quick keys and NVDA Keyboard layouts can be used Performance Solutions such as Blazemeter to be leveraged for running Jmeter script Open Source Tool Stack for the Framework 9
One Digital in Action PROBLEM STATEMENT Separate QA cycles for functional and non-functional - need to operate two teams, one aligned to sprint and a full capacity shared team for release level execution Wait time for the release level non-functional testing led to increased cycle time Additional effort for managing different tools and scripts SOLUTION Functional and non-functional use cases critical for the release were identified Integrated Automation Framework was established to bring together all testing types, for non-functional testing open source tools were leveraged to bring in specific libraries Shared capacity of 9 resources was reduced to 7 and the release level activities as part of shared team were running with a smaller team OUTCOMES Defects pertaining to non-functional aspects were detected in-sprint due to early performance and security test execution (On an average 3 out of 6 NFT defects uncovered during sprints) Release duration was reduced to 15 Days from 20 days due to minimized hardening sprint ~20% REDUCTION IN TEST EFFORT ENHANCED QUALITY THROUGH EARLY DEFECT DETECTION ~ 25% REDUCTION IN CYCLE TIME 10
Bibliography https://go.forrester.com/blogs/16-11-01-2017_predictions_mobile_is_the_face_of_digital/ The Forrester Wave : Modern Application Functional Test Automation Tools, Q4 2016 IDC FutureScape: Worldwide IT Industry 2017 Predictions http://pages.cs.wisc.edu/~swift/classes/cs736-fa06/papers/autonomic-computing.pdf Hootsuite Device Usage (Jan 2017) Infographicsme.com The Digital Revolution Thank You!!! 11
Question & Answers 12