IBM Testing & Quality Solutions

Transcription

1 IBM Testing & Quality Solutions Shift Left for Higher Quality at Greater Speed Mehmet Cagri Elibol WW Sales Leader for Testing & Quality Middle East & Africa and Asia Pacific 2016 IBM Corporation

2 Agenda Digitization and DevOps is fundamentally changing solution delivery DevOps speed and automation requires new thinking about testing and quality, affecting: How and what to test The test processes to follow The automation needed The IBM Continuous Testing solution helps you: Start testing earlier Test continuously 1

3 Innovate. Disrupt. IBM New challenges delivering innovation Transform. Scale. for the enterprise, driven by cloud and mobile technology How to rapidly deliver new, innovative applications, leveraging modern architectures? How to modernize existing applications to enable them for faster delivery and innovation? How to adapt culture, tools and processes to succeed? 80% Partners Connected Appliances Internet TVs Websites APIs Trillions Tablets Smartphones 2 Connected Cars Game Consoles 2015 IBM Corporation of CxOs are experimenting with different business models or thinking of doing so

4 IBM DevOps Enterprise capability for continuous software delivery Continuous Customer Feedback & Optimization Continuous Monitoring Operate Continuous Business Planning Steer DevOps Continuous Feedback Deploy Develop/ Test Continuous Release and Deployment Collaborative Development Continuous Testing Speeding continuous innovation 3 Enabling continuous delivery of innovations Providing feedback for continuous learning IBM Corporation 3

5 DevOps is key in digital transformation Greater Automation, Reduced Time to Market, and Improved Quality Rapid iterations Develop Build Test Deploy Innovative Edge Digital Applications Production API Catalog Industrialized Core Enterprise Applications Slower iterations Develop Build Test Deploy Production Achieving Continuous Delivery Balancing Multi-Speed IT 4

6 Modern testing needs to adapt to the DevOps world Before DevOps manual testing could keep up Dev QA Dev Test In the DevOps world manual testing can t keep up! Dev QA Dev Test Testing needs to change with automation and collaboration Dev QA Dev Test

7 The traditional approach to test automation Probably waits for the user interface to become available and stable Writes manual tests and/or records automated tests via the user interface Passes or fails test based on some on screen application response UI Layer Service / API Layer Component Layer Performance and security testing may be overlooked May do some level of Service or API testing Unit tests at the component layer are probably a nice to have Custom stubs or mocks may have been written to simulate dependencies 6

8 This perspective can hide the true complexity of problems. UI Layer Service / API Layer Component Layer

9 And hide the biggest risks of failure Most testing effort Biggest technical challenges and risks Do you know what s under the surface? 8

10 Find the right balance across test practices How are you spending your testing time? Running manual tests Creating automated UI tests Refactoring/debugging automated tests Configuring test environments Verifying defects Creating/managing test data Reporting on test progress Benchmarking third-party apps Management Defects Automation Analytics Test Effort Environments Service virtualization Data 9

11 A smarter approach to automating tests Exploratory Testing UI Layer Understand the complexity from the bottom up Use service virtualization to eliminate delays Automate the high-risk areas first Create API layer tests earlier and for reuse Service / API Layer Component Layer 10

12 Traditionally, enterprise applications are integrated all at once With development teams delivering at different velocities: Big Bang! Big Bang! Presentation layer Middleware component Integrate Integrate Mainframe component typically 6 Weeks 1 week typically 6 Weeks 1 week 11

13 Benefits of test automation hampered by big bang integration 12

14 Continuous testing can help achieve software quality earlier Test earlier Find architectural issues sooner Reduce costs Decrease risk 13

15 Increase quality with continuous testing Accelerate testing in production-like test labs Service Virtualization Deployment Automation Dynamic Infrastructure Automate comprehensive test execution API/Integration Performance User Interface (UI) Provide project teams with continuous feedback on software quality to reduce business risk

16 The solution: Service Virtualization and Test Automation Test Automation Test repeatedly and continuously Author tests that execute automatically at component, service, and system levels. Service Virtualization Start testing earlier Simulate behavior of missing components Decouple at the interface level Browser UI / Manual / Integration Testing Your system Interfaces to other systems (e.g. Payment) virtual component 15

17 Virtualization enables you to create virtual services Simulate the behavior of an entire application or system during testing Can be run on commodity hardware, private cloud, public cloud Gives each developer or tester their own test environment Works with existing testing tools (Manual, Web performance, UI test automation) Routing Service Public Cloud Collaboration App Under Test Content Providers Private Cloud EJB Business Partners App Under Test Third-party Services Data Warehouse Portals Directory Identity File systems Mainframe Archives Heterogeneous Environments Enterprise Service Bus Shared Services Messaging Services Capture & Model Databases Mainframe applications Third-party Services Packaged apps, messaging services, etc. Virtual Services

18 Test Automation: reuse and execute tests repeatedly System Virtual Services enable Functional Tests reuse Performance Tests Sub-system Subsystem enable reuse Component Component Component enable reuse IBM Continuous Testing Solution 17

19 The Continuous Testing Solution includes: IBM Rational Test Workbench Enable functional, performance and API testing throughout your project lifecycle IBM Rational Test Virtualization Server Accelerate testing, reduce the infrastructure costs of testing environments IBM Rational Performance Test Server Save time and effort by reusing integration tests for performance testing Test Workbench Test Servers QA Test Lab Infrastructure DEV UI Testing API Testing Performance Testing Load Agents SaaS Load Agents Virtualization Agents 2016 IBM Corporation Page 18 IBM and Business Partner Use Only

20 DevOps Continuous Test and Deploy A Winning Combination: Faster Delivery with Highest Quality Configures and deploys: Applications Middleware Databases Infrastructure Unit Test Performance Testing Component Test Integration Test System Test User Acceptance Test IBM Rational Test Workbench Continuous Testing Solution

21 Solve challenges with continuous test and deploy Challenge + Opportunity = Innovation Insufficient testing; not enough resources; testing takes too long Continuously test functionality and performance in production-like labs IBM Rational Test Workbench Incomplete and inadequate test labs due to missing software Virtualize unavailable yet dependent software IBM Rational Test Virtualization Server Defect escape; insufficient test coverage; lack of visibility into software quality Manage quality across the lifecycle IBM Rational Quality Manager Lack of available env. for testing; poor utilization Error prone, manual deployment to production environments Repeatable, reliable and consistent deployments to test stages through to production IBM UrbanCode Deploy 20

22 IBM DevOps Continuous Testing Shift Left testing for higher quality at greater speed Increase Collaboration and Productivity Automate Testing Functional Testing API and Integration Testing Performance Testing Deliver Software with Speed and Quality Continuous Testing Accelerate Innovation and Experimentation Scale to the Enterprise Service Virtualization Load Agents SaaS Load Agents 13x ROI In 3 Years! Used by 50% of Top 20 Banks 60% Faster Timeto-Market Container Shipping Comp. 25% Improved Productivity Testing Innovations European Bank 233% Improved Testing Program Productivity Telco Provider

23 Key References for Functional and Performance Testing America First credit Union Improve the speed at which it delivered changes to its customers. Needed to invest in omni-channel retailing and improve customersatisfaction levels. Solution: Deployed performance testing solution supporting IBM WAS to implement customer-centric branch system Results: increased branch staff utilization rate by 30 percent and employee efficiency while reducing operating costs Asobancaria Excessive manual processes impedes productivity of teams and quality of business applications Needed to reduce time spent managing test results and relationship between sw artifacts Solution: Worked closely with IBM Lab Services to deploy Functional and Performance Tester along with RQM Results: Established a centralized, collaborative hub for testing and automated labor-intensive recurring testing tasks US Department of Agriculture Respond to increasing demands from business and direct customers (100+ web based apps) Develop a virtual test infrastructure to drive adoption of automated testing across teams Deploying integrated Rational tool stack (incl. CLM, RTW WebUI, RPT) to handle smoke tests, functional and P&L testing Goal to free up QA engineers from running P&L tests on maintenance releases so they can focus on new features and integrations. Credit Agricole Technologies Need to deploy an end to end Testing infrastructure to supporting the needs of teams working across various apps domains. Conducted elaborate evaluation and PoC, in competition with HP and OSS Leverage successful displacement of HP-QC by RQM for Test Management Following successful PoC, CAT acquired and deployed RTW Web UI Over 100+ Test Cases automated On-going discussion for additional licenses of RTW, RTW WebUI and RIT

24 IBM Rational Test Virtualization Solution has proven measureable results and business value Major telecom carrier Multiple test releases required additional test resources, increasing testing costs Solution: Virtualized services and created reusable test cases across multiple environments Results: Reduced testing time by 50%. A leading global financial services firm with assets of over $2 trillion Many disparate legacy format to be migrated to next gen payments system Solution: Virtualized third-party systems otherwise unavailable for testing Results: Reduced manual testing from 10 days to 10 minutes, saving over $7 million to date Major U.S. insurer Recognized manual efforts insufficient for complex SOA and web services Solution: Agile middleware solution created to match the legacy systems functionality Results: Reduced testing by 95% to 2 hours; reduced rate filing validation by 94% to 320 hours Global manufacturer acquires competitor Migrate off rented infrastructure onto company s standardized middleware platform Solution: Virtualized critical, unavailable systems during migration Results: Fully integrated in 6 months 2 months ahead of schedule; saved significant rental costs and dependencies on third-party systems

25 IBM ranks #1 in 3 Automated Software Quality Analyst reports Gartner MQ for Integrated Software Quality Suites

26 IBM Resources on DevOps and Continuous Testing IBM Continuous Testing Community Click here to download the ebooks! 25

27 Thank You Copyright IBM Corporation All rights reserved. The information contained in these materials is provided for informational purposes only, and is provided AS IS without warranty of any kind, express or implied. IBM shall not be responsible for any damages arising out of the use of, or otherwise related to, these materials. Nothing contained in these materials is intended to, nor shall have the effect of, creating any warranties or representations from IBM or its suppliers or licensors, or altering the terms and conditions of the applicable license agreement governing the use of IBM software. References in these materials to IBM products, programs, or services do not imply that they will be available in all countries in which IBM operates. Product release dates and/or capabilities referenced in these materials may change at any time at IBM s sole discretion based on market opportunities or other factors, and are not intended to be a commitment to future product or feature availability in any way. IBM, the IBM logo, Rational, the Rational logo, and other IBM products and services are trademarks of the International Business Machines Corporation, in the United States, other countries or both. Other company, product, or service names may be trademarks or service marks of others IBM Corporation

28 Additional Material

29 IBM Continuous Testing Rational Test Workbench Starter Editions Get up and running quickly, at no cost, with Rational Test Workbench. Rational Test Workbench Web UI Tester Starter Edition Create and run functional tests for modern web applications. Rational Integration Tester Starter Edition Record and test the APIs that your systems use every day. Rational Performance Tester Starter Edition Examine how your systems perform under load. Get the Rational Test Workbench Starter Editions now on IBM developerworks

30 IBM Continuous Testing Tool integrations Configuration Management RTC SVN Continuous Integration ANT RQM Git Jenkins Test Management Test Drive HP Quality Center API Management IBM APIm ClearCase Hudson z/os Connect * Analysis ITCAM for App. Diag. Purify Plus CVS Eclipse Team Provider Rational Performance Tester Rational Functional Tester Rational Integration Tester Rational Test Virtualization Server Rational Test Workbench Web UI Tester Rational Service Tester Rational Test Workbench Mobile Edition SOA Governance Centrasite Continuous Deployment UrbanCode Deploy Defect Management JIRA CQ RTC OSLC WSRR Bugzilla Team Foundation Server Modelling Rational System Architect Production Monitoring ITCAM for Transactions IBM Tivoli Monitoring Eclipse IDEs Visual Studio IBM Worklight Studio

31 Rational Test Workbench: API Testing Ensure sound integration of complex systems with a wide range of technologies Plan tests: Visualize connections and dependencies between services and components Create tests: Use predefined transports and message formats Customize tests: Build your own transports and formatters 30

32 Rational Test Workbench: Performance Testing Identify and manage the risk of application performance failures Capture and track Business SLAs in your test plans Create realistic user workloads that exercise the key business transactions Automatically deploy the environment and schedule execution of your workload Identify the root cause of performance problems Communicate results and areas of non-compliance to the stakeholders Rational Performance Tester System Under Test Test Authoring and Control Load Generators Web Server App Server DB Server 31

33 Rational Test Workbench: Web UI Testing Provides automated testing for HTML5-based browser applications Supports Internet Explorer, Firefox, Chrome, Safari, and Microsoft Edge on Windows, Linux, or Mac OS Relies on graphical scripting (no programming knowledge needed) 32

34 Rational Test Workbench: Functional Testing Functional Tester Capture Enhance Execute Reports Test Plans Defects Test Datapools Keyword Library Reusable Test Scripts Web Applications Terminal Based Applications Packaged Applications Java Applications.Net Applications Automated regression testing for traditional GUI clients Simple record enhance execute model Multiple scripting options (Java, VB.Net & Storyboard) System z Powerful custom scripting and extensibility SDK System i 33 33

35 Rational Test Virtualization Server Test and virtualize middle-tier and back-end applications and services Examine integration protocols and messages to verify end-to-end communication Reuse integration tests to verify the scalability of your integration and services infrastructure Virtualize application services to enable continuous integration Leverage Rational Test Workbench extensive testing features to address nonmobile components of your environment Virtual component or service Presentation Layer Backend Data and Services Middle Tier 34

36 Rational Performance Test Server Prevent the costly impact of poor application performance Create complex performance scenarios with a visual workload model Reuse your functional integration tests to assess the impact of real loads on your integrated infrastructure Correlate server resource statistics with response time measurements to quickly assess server health and identify resource bottlenecks Accelerate the analysis of test results with a rich set of customizable, out of the box reports 35

37 Testers want to spend their time on what matters Creating automated tests Performing exploratory tests Executing automated tests Designing tests Planning tests Reviewing test results Running predefined tests Maintaining automated tests Creating reports Maintaining automated test tools Configuring test tools Creating test data Rerunning tests Investigating & submitting defects Setting up labs [includes deployment] Techwell, The Future of Testing, Where do Testers Spend Their Time? November 2015

38 Kick off your DevOps transformation Start with a business objective Identify actions to fix pain points Execute on the actions / measure results Iterate for continuous improvement Plan Deliver Deploy Operate DevOps Improve Establish Business Objectives Assess current capabilities using the DevOps maturity model Identify pain points and sources of friction that impede capabilities Execute/Measure improvement actions to reduce or eliminate pain points Continuously Improve, Identify actions, pain points, new milestones. Progress as an agile project 37

39 IBM can help! Assess your current DevOps capability IBM DevOps Practice Self Assessment Establish Business Objectives Assess current capabilities using the DevOps maturity model Assess & Plan Plan Deliver Deploy Operate DevOps Improve Identify pain points and sources of friction that impede capabilities Execute & Measure Execute/Measure improvement actions to reduce or eliminate pain points Request an IBM DevOps Innovation & Optimization workshop Identify bottlenecks in a Value Mapping Stream interactive discussion Engage the local IBM team to: Deliver deep-dive demonstrations Lead an ROI discussion with management Execute a Proof of Concept Kick off a DevOps pilot project 38