The Future of Testing

Transcription

1 The Future of Testing Wolfgang Platz, Founder & Chief Strategy Officer.

2 awards Leader Gartner MQ founded 2008 Leader Forrester Wave locations Austria Germany Switzerland United Kingdom The Netherlands Poland USA Australia Singapore India customers 4 Forbes Top 10 Global 2000 Companies 6 investment 4 Australian banks computer services 4 life insurance 5 major banks 4 diversified insurance support ASP Award 2016 & 2017 employees 390+

3 Back in 1999, this is how I felt 3

4 Vision To be the leader in software testing with the most innovative testing products that enable our customers to deliver high quality software. 4

5

6 Nov 2017 Dec 2015 Aug 2014 Jan 2011

7 It should be considered by enterprises that have struggled to make test automation work, and by those seeking to support agile continuous automation practices.

8 1 Market Macros 2 Testing is ready for Disruption 3 Testing the hype 8

9 81% of enterprises have adopted or plan to adopt DevOps. 9

10 Toda y The reliance on manual testing is the #1 technical challenge in app development 1. > 80% Manual Testing 35% IT Budget Spend on QA & Testing 1 $32B Annual Spend on Testing Services < 20% Automated Testing UI API Legacy platforms require weeks to execute testing. 1 Sou r ce: World Quality Report

11 60 Billion USD 50 12% Annual Growth (CAGR) So u r ce Software Testing Market (total) Application Testing Product Testing

12 Testing is still on the rise 2 Testing is ready for Disruption 3 Testing the hype 12

13 Today s reporting is based on numbers of test cases. Version ,278 10, What does it tell us? Is 53,274 the right number of test cases? Can I go productive? 13

14 Rear wing Engine Cassis & Monocoque Gear box Front wing Telemetry Software Exhaust Brakes Steering Wheel Tires 14

15 15

16 The law (almost) 80:20 we cover 80% of our business risk. With 30% of test cases 16

17 40% average risk coverage achieved in comprehensive enterprise test portfolios 1 67% average level of redundancy in enterprise test portfolios 1 60% of risk is secured through hope & pray. 2 out of 3 test cases never yield defects. 1 Source: Customer projects, Tricentis 17

18 Risk Coverage [%] Core Bank 66% 9% 15% 10% We need to introduce a new currency in testing. Risk Coverage 18

19 18% 8% Really? (official) overall test automation degree (official) enterprise test automation degree? X 19

20 Continuous Testing requires Automation Today Future Exploratory Tests 80% Manual Testing Automated UI Tests API Tests +85% 20% Automated Testing UI API Orchestrated Service Virtualization - key enabler for high automation rates - 20

21 Comparing learning curves 100 pax 3 months Go train them on Selenium based scripting for a certain framework you got 3 months! Head of QA Services 21

22 Comparing learning curves 100 pax 100 pax 3 months 3 weeks Go train them on test automation with Tricentis Tosca you got 3 weeks! Head of QA Services 22

23 Comparing learning curves 100 pax 100 pax 3 months How many made it? 3 weeks 85% 15% 23

24 Delivering on the promise of extreme automation requires a new paradigm. 24

25 Model-Based Browser Script-Based Browser Code Synchronization Behavior Code Recovery Handling Code Data Driven Framework Code Keyword Driven Framework Assert.AreEqual( 331,00", driver.findelement(by.xpath("//table[@id='pricelist']/t body/tr[10]/td[2]/b")).getattribute("value")); 25

26 Continuous testing is change It s way more than just UI automation

27 The three nightmares of test automation

28 Test Portfolio Maintenance Test Data Management Test System Provisioning Escape it also for E2E systems & APIs Needs to be stateful & fully integrated Requires service virtualization

29 Risk-Based Testing Test Management Manual Testing Analytics Exploratory Testing Robotics Service Virtualization Model-Based Test Automation 35+ Technologies Packaged Applications BI & B ig Data Dev Ops Development Impact Analysis Tosca Flood Test Data Management Integrations Operations

30 Testing is still on the rise Testing is ready for Disruption 3 Testing the hype 30

31 IoT Platform & Intelligent Things (Connected Home, etc.) (Deep) Machine Learning & AI Autonomous Drive Blockchain 2017 Plateau to be reached 2 5 years 5 10 years > 10 years Expectations Extracted from Gartner Hype-Cycle, As of July 2017 Innovation Trigger Peak of Inflated Expectations Through of Disillusionment Slope of Enlightenment Plateau of Productivity Tim e

32 Blockchain Trusted Transactions Blockchain will become a Service Bitcoin s blockchain Ethereum No trusted 3rd party 25+ significant Blockchain projects Bottom Line. We need to test the applications using that service

33 Internet of Things Machine-to-Machine communication through Intelligent Things 15 Billion Devices, Billion Devices, 2025 Smart health, smart energy, connected homes Bottom Line. Testing needs to be ready for mini- and micro-services

34 Innovations 200,000 B.C. Fire 7,000 B.C. Wheel 2007 iphone that changed mankind

35

36 Mobile First Google-CEO Sundar Pichai, Google I/O, 2017

37 AI First Google-CEO Sundar Pichai, Google I/O, 2017

38 No need to test AI per se, but applications using the outcomes of artificial intelligence

39 Today, there is a tremendous amount of noise associated with AI in the software industry.

40 We need to become smarter in testing, even if there is no AI involved.

41 Testing is still on the rise Testing is ready for Disruption Testing is ready for modern Technologies 41

42 Thank You for your attention Stay in touch! Host: Wolfgang Platz Follow us on 42