Using Layered Model-Based Requirements to achieve Continuous Testing

Transcription

1 Using Layered Model-Based to achieve Continuous Testing Alex Martins CTO Continuous Testing CA Technologies October

2 11 years ago In A Galaxy Not Far Away

3 Gap in CD Pipeline Acceleration Initiatives idea Requirement Test Cases Mostly manual to go from Idea to Code (for TDD) and to Test Cases and Test Automation scripts How to automate the CD Pipeline Intake process? We typically focus on accelerating code-to-deployment activities The Intake process usually remains a bottleneck and mostly manual Test Data Virtual Services Test Automation Code Integrate Build BUILD Extreme acceleration possible through better tools and infrastructure that support the cultural change on the team Functional Testing TEST Performance Testing DEPLOY Customer Experience Deploy to Production OPERATE Developers Testers Release Team Operations

4 Challenges to Achieving Continuous Delivery & Testing IDEA 64% of total defect cost originate in the requirements analysis and design phase 1. Development 50% of developers time is spent finding and fixing defects 2 X? 80% of teams experience delays in development and QA due to unavailable dependencies 3 X? Security QA / Testing Release 70x required manual pen test scan cost vs. automation 10 70% of all testing is still manual 4 X X X 63% of testers admit they 50% of time can t test across all the different spent looking for devices and OS versions 5 test data 6! 30% of teams only security scan once per year 9 57% are dissatisfied with the time it takes to deploy new features 7 50% more time spent on security defects in lower-performing teams 8 79% of teams face prohibitive restrictions, time limits or access fees on needed 3rd party services 3 Operations! Greater than 50% production outages from configuration management! Ave. cost of downtime per minute = $4,700 53% percent of visits are abandoned if a mobile site takes more than three seconds to load

5 Why are Traditional Testing Activities a Bottleneck? Testing hasn t changed in the past 30 years Define an Idea or Customer Needs Write Manual Processes Enable Interfaces (real or virtual) Find/Create Test Data Automate Tests (usually Regression) Identify Interfaces (internal and 3P) Identify Test Data Create Detailed Test Cases Create Traceability Matrix Read Execute Test Cases Create Defects Fix Defects

6 Achieving Speed Through Continuous Testing Define an Idea or Customer Needs MODEL Enable Interfaces (real or virtual) Find/Create Test Data Automate Tests (usually Regression) NOW ALL AUTOMATED Identify Interfaces (internal and 3P) Identify Test Data Create Detailed Test Cases Create Traceability Matrix Read Execute Test Cases Create Defects Fix Defects

7 BUT HOW 7

8 Today we model our vision in diagrams on the whiteboard to describe expected behavior, rules, designs, plans Executives Project Managers Developers Testers Product Owners

9 Then we spend time on converting the visual diagram into words 9

10 Only to read those words and create yet other set of artifacts (i.e. app code, test cases, test data, automated test scripts, etc. 10

11 Too Many Unnecessary Manual Steps Only Slows You Down Why not skip the intermediate steps? Create a diagram Convert it to text Interpret text Create app code, tests, data, scripts, etc. Intermediate activities are impediments to speed and quality 11

12 Business Analysts/POs Focus on What An Example for a Shipping Business Process

13 While Quality Engineers/Testers Focus on How An Example of Detailed Subflows with Rules

14 And QA Engineers/Testers Aim at... Foundation & Layers, Collaboratively Enhancing the Model Model (Foundational Layer) Test Data Layer Services Layer Test Automation Layer

15 And QA Engineers/Testers Aim at... Foundation & Layers, Collaboratively Enhancing the Model Test Data Layer Model (Foundational Layer) Services Layer Test Automation Layer

16 So, Quality Engineers/Testers... End Up With The Desired Output

17 THANK YOU 17