ROTATE TO THE NEW: FROM TESTING TO QUALITY ENGINEERING

Size: px

Start display at page:

Download "ROTATE TO THE NEW: FROM TESTING TO QUALITY ENGINEERING"

Vanessa Moore
5 years ago
Views:

1 ROTATE TO THE NEW: FROM TESTING TO QUALITY ENGINEERING

INDUSTRY TRENDS AND PRESSURES IMPACTING QA IT organisations are being impacted by the convergence of three disciplines that were once operated as independent disciplines.

and performance automation tools MORE AUTOMATION Key enablers: DevOps pipeline integration for all code propagation functions and test execution Codeless automation DIFFERENT METRICS

2 INDUSTRY TRENDS AND PRESSURES IMPACTING QA IT organisations are being impacted by the convergence of three disciplines that were once operated as independent disciplines. MORE TESTING IN DEVLOPMENT/LOWER- TIER ENVIRONMENTS Key enablers: Service Virtualisation THE CONVERGENCE IS DRIVING THE TRANSFORMATION OF QUALITY ORGANISATIONS Functional, security, and performance automation tools MORE AUTOMATION Key enablers: DevOps pipeline integration for all code propagation functions and test execution Codeless automation DIFFERENT METRICS Engineering metrics such as volatility, predictability, solution readiness, and code coverage. Analytics-driven intelligent automated decision making Copyright 2017 Accenture All rights reserved. 2

Future Traditional Plan / Analyze Design Build Test Requirements to Repository Input Quality Code to Repository Repository to non- QA

3 THE TESTING BUBBLE Popping the bubble of testing by moving to a filtering approach to quality which starts at the beginning of the development life cycle. Future Traditional Plan / Analyze Design Build Test Requirements to Repository Input Quality Code to Repository Repository to non- QA Deployment Testing Repository to QA/Stage Deployment Product Pipeline (Powered by DevOps) Production Deployment Repository to Prod. Deployment Market Ship Confirm Input quality Feedback Loop Monitor Market Copyright 2017 Accenture All rights reserved. 3

4 INTELLIGENT AUTOMATION A key component of Quality Engineering is Intelligent Automation. Intelligent tools enable functional SME s to create and maintain automation without engineers. Total Testing Effort Full Application Stack Automation Coverage % Manual testing Remaining manual testing (exploratory) < 15% > 85% DevOps Support Environment Services & Service Virtualization % Automated Testing Current State Automated API Tests Target State Test Data Management Automation Coverage Optimization Copyright 2017 Accenture All rights reserved. 4

5 OPERA NEXT GENERATION TESTING AUTOMATION ACCELERATION Worksoft Certify, Tricentis Tosca where have we been in 2016? Objectives: Prove out thesis of strong differentiation and enable the foundation of journey to Intelligent Automation What we did: Conducted 20 Proofs of Concept New Operating Model Results: Consistent commercial data points emerged: Productivity Run Rate Improvements ranging from 45% to 90% (median +70%) ROI projections ranging from 130% to 290% (median +200%) Coverage attainment, speed to build, and maintenance improvements Copyright 2017 Accenture All rights reserved. 5

6 EXPANDED QUALITY-FOCUSED ROLES There are a changing set of roles within the quality space to better enable the automate everything vision within an Agile delivery model: In-Sprint Quality Focus: Cross-Sprint (Program-Level) Quality Focus: Quality Engineer (SDET*) Automation Architect In-Sprint Quality / UAT Specialist Regression / Release / Integration / UAT Tester Test Architect *SDET = Software Development Engineer in Test Test Manager Copyright 2017 Accenture All rights reserved. 6

7 TOMORROW S QUALITY CENTERS OF EXCELLENCE We anticipate the Quality Engineering Center of Excellence of the future will build and nurture Quality Engineering skills centrally that provide supply of skilled resources for projects and programs in each portfolio (vertical): Quality Governance / Operations Quality Lead Service Consumers (Portfolios by business solution) Application Portfolio #1 Application Portfolio #2 Tool Owner(s) Process Owner(s) Relationship Mgr(s) Application Portfolio #3 Application Portfolio n * Portfolios will be defined by similar technology and/or business service areas Quality Engineering Services Service Delivery Mgr(s) Service Coordinator(s) Management Architecture/Planning Functional Test Automation Performance Non-Functional Environment Test Data Defect Management Reporting Level of domain knowledge required Copyright 2017 Accenture All rights reserved. 7

8 APPROACH - TYPES OF TESTS Different levels of test artifacts help focus testing on the full set of functionality being delivered, from the granular unit level all the way up to the core business process. Agile Artifact Test Artifact In-Sprint? SDET Role Epic E2E / Integration Tests N Support Feature Feature / Capability Acceptance Tests Partial Build and Support User Story Story Acceptance Tests Y Build public Code Unit Tests & Code Quality Scans Y Review and Validate class { } User Story, Feature, or Epic (L1, L2, L3 Performance Test) Performance Tests Partial (L1 Performance) Build (PE-focused SDET) or Support Copyright 2017 Accenture All rights reserved. 8

Legend In-Sprint Testing Cross-Sprint Testing APPROACH TYPES OF TESTS WITHIN SPRINTS The delivery model implied below covers both in-sprint testing and cross-sprint testing, with different types of

9 Legend In-Sprint Testing Cross-Sprint Testing APPROACH TYPES OF TESTS WITHIN SPRINTS The delivery model implied below covers both in-sprint testing and cross-sprint testing, with different types of tests happening in each: Sprint 1 Sprint 1 Sprint 1 Sprint 2 Sprint 2 Sprint 2 Sprint N Sprint N Sprint N public class { } Sprint-Level Regression (defined as subsets of both) System Integration Testing and Release Integration Testing Regression Smoke Tests (defined as subsets of both) public class { } Unit Tests and Code Quality Activities Story Acceptance Tests Feature and Capability Acceptance Tests E2E / Integration Tests Copyright 2017 Accenture All rights reserved. 9

10 TEST AUTOMATION JOURNEY This journey typically starts with an assessment and strategy/roadmap exercise to identify specific opportunity areas, but the journey generally follows an establish, expand, and run/mature pattern: Focus Areas Along the Journey: Roles Approach Tools & Supporting Disciplines Establish Automation Cross-train Expand Automation Roll-out Resolve blockers Run and Mature Automation Increase lower layer automation Refine practices Mature infrastructure & performance Strategy Platform Pilot Start Milestone 1 Milestone 2 Milestone Months 6-12 Months Months Copyright 2017 Accenture All rights reserved. 10

Test Automation Strategies in Continuous Delivery. Nandan Shinde Test Automation Architect (Tech CoE) Cognizant Technology Solutions

Test Automation Strategies in Continuous Delivery Nandan Shinde Test Automation Architect (Tech CoE) Cognizant Technology Solutions The world of application is going through a monumental shift.. Evolving