Disruptive Technology

Transcription

1 Continuous Delivery & The Changing Role of the Tester Dr Kelvin Ross ANZTB 2015, Auckland 22 May 2015 Disruptive Technology 2 1

2 Disruptive Technology 3 Disruptive Technology 4 2

3 Disruptive Technology 5 CLOUD SOCIAL DISRUPTIVE FORCES MOBILE BIG DATA 3

4 Biggest Disruptors The Last 5 Years? Agile Offshoring Virtualisation Disruptive Forces DevOps Customer Testing 8 Big Data 4

5 Startup Corporate 10 You can't just ask customers what they want and then try to give that to them. By the time you get it built, they'll want something new. Steve Jobs 5

6 12 14 Scientific Method for Customer Experiments to Guide Software Development 6

7 FAIL FAST

8 3 6 Months weeks 18 8

9 You can't just ask customers what they SPEED want and then try to give that to them. By the time you get it built, they'll want something new. is the new black Steve Jobs

10 21 DEV OPS 10

11 No Silos 24 CI CD Continuous Integration Continuous Deployment CD 25 Continuous Delivery 11

12 Rapid Releases Source: Principles and Practices in Continuous Deployment Mike Brittain,Engineering Director, mikebrittain.com/talks Source: 12

13 Source Control Automated Deployment System Build System Test infrastructure CD Platform Pipeline Management & Visualization Automated Configuration Management (IaC) Artifact/ Package Repository Source: Continuous Delivery at Scale John Esser, Director Engineering Productivity, Ancestry.com ALM Forum Seattle, 2014 Version & Configuration Management Git Flow 13

14 Version Everything Test Data Static Data Application Code Infrastructure 30 Data Versioning Infrastructure/Platform Versioning 31 14

15 New Strategies Cloud Computing Duplicate test environments Copy of production Grid based test automation Blue/Green Deployments Resilience testing

16 Elastic Computing Time Which is better? 34 Release Pipeline Source: Principles and Practices in Continuous Deployment Mike Brittain,Engineering Director, mikebrittain.com/talks 16

17 Testing Pyramid Right Way SYSTEM Slower UI focused Later in the processes INTEGRATION Focused on linkages Harnesses, Mocks and Stubs UNIT Fast Isolated Aligned with Code 36 Release Pipeline Source: Steve ALM Forum Seattle,

18 Deployment Gating Source: Advanced Topics in Continuous Deployment Mike Brittain,Engineering Director, mikebrittain.com/talks Production Testing Source: Principles and Practices in Continuous Deployment Mike Brittain,Engineering Director, mikebrittain.com/talks 18

19 Failure Monitoring 40 Source: Principles and Practices in Continuous Deployment Mike Brittain,Engineering Director, mikebrittain.com/talks FAIL FAST 41 19

20 FAIL FAST MTBF 42 FAIL FAST MTTR 43 20

21 Customer Testing 45 Testing In The Wild Applause Secured $80m in VC funding 21

22 Elastic Resourcing Time Which is better? 47 Customer Testing 48 22

23 A / B Tests E.g. Google 49 Analytics Content Experiments Source: Advanced Topics in Continuous Deployment Mike Brittain,Engineering Director, mikebrittain.com/talks Feature Toggles 50 23

24 Feature Toggles 51 Source: Advanced Topics in Continuous Deployment Mike Brittain,Engineering Director, mikebrittain.com/talks Feature Toggles Progressive Delivery To Minimise Risk 52 Source: Advanced Topics in Continuous Deployment Mike Brittain,Engineering Director, mikebrittain.com/talks 24

25 Early Customer Testing 53 Can We Skip Some Pre-Prod Testing? BIG DATA Volume Velocity Variety 55 25

26 How To Test Prediction 56 Big Data Challenges Staging test data sets Test data privacy Variation of test data sets Production data variability Controllability & Observability Algorithm Evaluation Heuristic Testing Exploratory Testing 57 A/B Testing Customer Sampling & Monitoring 26

27 New QA Skills Technical Influence Test Environment Provisioning CI / CD Process Experimentation Automation Data Analytics Feature Architecture & Deployment Planning Production Monitoring Canary Releasing 27

28 Blue Green Deployments Feature Toggles 28

29 Feature Flippers A / B Tests E.g. Google 64 Analytics Content Experiments Source: Advanced Topics in Continuous Deployment Mike Brittain,Engineering Director, mikebrittain.com/talks 29

30 Dark Launch 30

31 68 In 5 Years??? 31

32 Skill Changes 69 DevOps Teams Embedded, Self Organising, Functional Business Tester Automated Tester DevOps Tester Non-Technical Functional Tester 70 32

33 Reduced Test Management Coordinate Resourcing Practice Support Consultants Coaches Governance Test Managers Test Leads 71 Crowd Test Coordination Customer Test Coordination Feature Deployment Planning User Experience and Failure Monitoring 72 33

34 Q&A 73 34