Quality Engineering in DevOps world a Strategic Enabler

Transcription

1 Unsolicited Distribution is Restricted. Copyright , Cigniti Technologies Quality Engineering in DevOps world a Strategic Enabler

2 » Analyst Speak» DevOps in a nutshell» DevOps vs DevTestOps» Continuous Testing & Delivery leveraging QE» Unit Testing & SAST» Automation, Performance & Security Agenda» Service Virtualization» Continuous Integration, Deployments & Delivery» Operations Stabilization in Devops» Continuous Automation & Delivery Pipeline» Collaborative Platform & How does it help? Distribution is Restricted. Copyright , Cigniti Technologies 2

3 Analyst Speak To date, Gartner has had more than 3,500 client interactions (inquiries, best-practice calls and conference one-on-one meetings) discussing DevOps awareness, initiatives, successes and failures Forrester - Development and operations (DevOps), the purview of unicorns, is now transitioning to the mainstream. Business technology (BT) leaders are realizing that Agile is no longer enough; they need to involve the full technology management organization to drive competitiveness in an era when customer centricity is pivotal to success Distribution is Restricted. Copyright , Cigniti Technologies 3

4 Analyst Speak Distribution is Restricted. Copyright , Cigniti Technologies 4

5 Key Challenges DevOps toolchains are often built from discrete and sometimes disconnected tools, making it difficult to understand where bottlenecks are in the application delivery pipeline Orchestrating the DevOps toolchain can be compelling, but it can introduce another set of challenges DevOps initiatives continue to grow in complexity, driving many IT organizations to seek tools that provide greater control and visibility over the entire DevOps process Toolchain orchestration promises to simplify the DevOps toolchain; however, one tool is not able to support the entire toolchain resulting in additional integration, ownership and handoff challenges when using multiple orchestration tools Distribution is Restricted. Copyright , Cigniti Technologies 5

6 DevOps adoption Is no longer Just For unicorns Culture and people come First; process follows Release velocity is pivotal There s no easy button Source: Forrester s Q Global DevOps Benchmark Online Survey Source: Forrester Data Global Business Technographics Developer Survey, 2016 & Forrester Data Global Business Technographics Infrastructure Survey, Distribution is Restricted. Copyright , Cigniti Technologies 6

7 DevOps Adoption Distribution is Restricted. Copyright , Cigniti Technologies 7

8 DevOps Adoption Distribution is Restricted. Copyright , Cigniti Technologies 8

9 Adoption and Strategies vary across Vertical Segments Public Sector And Healthcare cling To Manual Audit And compliance Procedures Compliance-wary companies need not fear automation Utilities And Telecommunications Lead The Pack To Drive Business Transformation Financial Services Accelerate To compete With Fintechs DevOps offers key benefits in fraud detection, time-to-market, and scalability $ Data must drive strategy, with services that emphasize connectivity Media, entertainment, & Leisure embark On The Journey To Differentiate With DevOps Manufacturing Transitions Beyond Automation To Show DevOps Momentum Business Services And construction Ponder The Leap Into Automation Technology is becoming part of the business DevOps drives to improve customer experience Manufacturers improve efficiency and insight by expanding the role of technology Retail And Wholesale DevOps Adoption Drive Differentiated Interactions With clients Get ready for the next generation of digital stores Distribution is Restricted. Copyright , Cigniti Technologies 9

10 DevOps vs. DevTestOps D e v O p s D e v Te st O p s Equal Importance for Dev, Operations and Testing All actions Development --> Operations Continuous Testing Emphasis on Automation & Analytics Continuous Integration + Continuous Deployment (CI/CD) Dev --> Ops Process Reduced from Months to Days/Hours Continuous Feedback & Improvement Supported Maturing Processes and Organizations Focus End to End Automation Overall Cost Reduction Automation, Reduce Cost of Quality Fail First, Fail - Fast, Fail- Often Early Defects Detection Collaborative Communication utilizing Dashboard Distribution is Restricted. Copyright , Cigniti Technologies 10

11 Devops in a nutshell Automated Unit Testing & SAST Build Packing & Automated Deployments Continuous Testing Functional Automation Non Functional Automation Build Dev Virtual Server(s) Production Deployments CI Server Infrastructure Automation & operations Physical server(s) Hosted Tools Post Production Monitoring & Feedback Amplification Distribution is Restricted. Copyright , Cigniti Technologies 11

12 Leverage Continuous Testing & Delivery Leveraging QE elements Efficient Tools Centralised data management through automated TDM tools & other mechanisms for on demand testing Data Engineering Third party system Enabling continuous testing by overcoming the dependency factors involved Service Virtualization Dashboard Leveraging a collaborative dashboard to monitor application release activities & health Target System Ensuring faster feedback for issues remediation through continuous infrastructure & application monitoring in production & test environments Client Layer Web Server App Server Component 1 Component 2 Component 3 Continuous Monitoring Health Check Application Infrastructure {Through scripts} {Through metrics configuration} Application Under Test UI Services App Component CI Platform Ensuring quality during rapid and frequent build deployments by leveraging robust CI tools integrated with test automation frameworks Continuous Testing Functional Automation Non - Functional Automation Target System Adopting Quality Engineering approach to provide end to end automated functional and non functional test coverage for each release Distribution is Restricted. Copyright , Cigniti Technologies 12

13 DevOps Journey Cost Elements DevOps Journey Automation Maturity ROI Factors» Cost of Maintenance» Technology Adoption DevOps Build - Deployment Release Automation» Build Reliability» Uninterrupted Service Delivery» DevOps Process» Org. Change Mgmt. (People, Process and Systems) Matured Collaborative Agile ( Dev, QA and Ops) Implemented Dual Shift Approach Quality Engineering approach for early testing (Functional and Non Functional)» Tools and process for CI and CD» Infra Maintenance Transformed to TDD / ATDD / BDD Adopted CI and CD tools» Automation Stabilization» Continuous Testing Continuous Feedback» On Demand Test Envts.» End-to-End Automation and Test Coverage» Tools for Automation and Non Functional Tests» Process Standardization Matured Agile Organization Partial Agile Functional and Non Functional Automation No Automation in place» Tool optimization and consolidation» Quality Check gates» Unified Automation Framework» Best suited tools Distribution is Restricted. Copyright , Cigniti Technologies 13

14 Key QE elements In Detail Distribution is Restricted. Copyright , Cigniti Technologies 14

15 Unit Testing & SAST I D E (Technology Agnostic) 1 1 JUNIT/NUNIT Unit Testing Open Source/Commercial Tools 2 Static Code Analysis 3 Results Analysis & Vulnerable identification Manual Analysis for false positive removal CI Server Code Repository Final Summary 2 Code Commit Build Server» Unit testing is performed using respective open source tools that integrated with the IDE used for code development» CI Server picks the build from the code repository/version controlling tool once the developer commits the build» CI Server initiates the static code analysis by triggering a request to the configured open source / commercial tools Secure Coding Guidelines & Standards» Input Validation» Output Encoding» Secure handling of credentials» Session Management» Cryptographic practices» Error handling / Logging» Data Protection» Communication Security» Memory management Distribution is Restricted. Copyright , Cigniti Technologies 15

16 Automation, Performance & Security Test Automation Test Automation Framework Test Management Tools Performance Testing Test Automation Tools Trigger Automated Test Execution Performance Testing Tools Performance Testing Scenarios Build Version Controlling Tool CI Platform Trigger Monitoring High Level Analysis Report Security Testing (DAST) DAST Tools Monitoring Tools Automated Trigger Monitoring Tools Results Analysis & Vulnerabilities Identification Manual Analysis (False Positives) Final Summary Report Distribution is Restricted. Copyright , Cigniti Technologies 16

17 T D D Finalized Feature D e v O p s TDD blended with DevOps Feature Spec file Automated Configuration Management frameworks (Ansible/Chef/Puppet) Continuous Deployment DevOps Analytics Platform DevOps Specific input to feature requirements Feature broken into functions CI/CD framework (Continuous Integration/ Continuous Delivery) Continuous Feedback Function 1- Unit Test case Function 2- Unit Test case Unit Testing, Integration Tests and Performance Tests in reference to development under DevOps practices. Pass Add a Developer test Run Developer tests at the unit level Pass, Functionality Incomplete Fail Development Ends Function 3 - Unit Test case Pass, Functionality complete Run the Developer tests Make code changes Distribution is Restricted. Copyright , Cigniti Technologies 17 Fail

18 Service Response Service Request Service Request Service Response Service Virtualization in AWS Cloud Advantages DevTest Deploy SV tool on cloud Create Virtual Services» Automatically Capture realistic behavior» Build VS Image, collection services requests and Service responses» Optimize models as desired Service Request Service Response Service Request Service Response Configurations Configurations Configurations Configurations Legacy ERP Databases Mainframe Accessible from Multiple Geographies (Cloud & On-premise) Deploy & Bring Up/down to Optimize Resources & Cloud Spend Easily Customized as the UI or Business Needs Change Leverage Docker Containers, enabling Continuous Integration of Code Changes and Parallel Test Instances Reduce CapEx (Capital Expenditure) by Hosting in AWS (Amazon Web Services) Cloud T Dynamically Support Infrastructure & Policy as Code, with Fugue Typical DevTest SV Usage Scenario Test (Manual, Automated) Exercise SUT System Under Test (SUT) Playback against Virtual Service SOAP Request SOAP Response JMS Publish Target Service Virtual Service DevTest Distribution is Restricted. Copyright , Cigniti Technologies 18

19 Continuous- Integration vs. Delivery vs. Deployment C o n t i n u o u s - I n t e g r a t i o n v s. D e l i v e r y v s. D e p l o y m e n t V1.1 Continuous Integration Continuous Delivery Continuous Deployment Approve Deploy Automatic Deploy Automated Automated Source Control Commit Changes Build Run Build And Unit Tests Staging Deploy to test Environment Run Integration Tests, Load Test, & Other Tests Production Deploy to Production Environment Distribution is Restricted. Copyright , Cigniti Technologies 19

20 What tools/technologies should QE team get trained Though Selenium continue to dominate, the following tools have been adopted faster Web Driver IO Protractor Concourse REPL is used to build web driver component Brief list of various Test Automation tools that have been in faster adoption With DevOps, The need for Microservices testing with spring server technologies will take higher precedence Server 1 Server 2 Server 3 Service - This is better than Jenkins - Shell script - runs on a FLOW. automate just about anything with a minimum of effort Alternate to Maven Selenium runner - reports in Allure - very powerful one for integration. - Supports Jasmine, Mocha - BDD tools - This is better than Cucumber Webdriver CS Visual Regression Concourse as a server. Enables multiple deployment for seamless test execution By 2019, Spring Server will dominate the server market space ( GitHub will become the global platform and open source will be the future. Distribution is Restricted. Copyright , Cigniti Technologies 20

21 Operations Stabilization & Alignment with DevOps DevOps Enabled Test Lab Release Automation 1 Automated Deployments Staging QA Dev Deployments Smoke Tests Virtual Server(s) CI Server Configuration Management 2 Rapid Configuration Management Multiple Nodes across technologies Physical server(s) Hosted Tools Environment Management 3 Build Environments Manage Virtualized & Maintain Non - Virtualized Monitor Staging QA Dev Continuous Monitoring 4 Health Check Infrastructure Application Threshold evaluation Distribution is Restricted. Copyright , Cigniti Technologies 21

22 Delivery Pipeline Continuous Automation Continuous Automation & Delivery Pipeline Automated Testing Test Automation Scripts Regression & Risk Based CI Enabled Execution Results Requirements Continuous Development Coding Compile & Build Automated Performance DAST Code Quality SAST Unit testing Continuous Deployment Release Deploy Apps Management Operations Enablement Collaborative Dashboard Continuous Monitoring On Demand Infra ORCESTRAED CONTINIOUS INTEGRATION PIPE LINE Requirements Document & verify SCM Repo Adoption Test Build Package Deploy Host / Stage Jenki ns Container as a SCM service using Rancher/Docker DevOps Pipeline Environment Environment Resources Resources Private Container Service User Directory Container Registry Ops Tools Enterprise Micro service Stack On demand test bed using containers Test Automation Tools Web & Mobile browsers Reporting & Monitoring Advance Analytics Dashboards Tableau Distribution is Restricted. Copyright , Cigniti Technologies 22

23 Live Dashboard Collaborative DevTest Platform & How does it help? Build Integrate Deploy Operate Release Progression Build Status Test Status Predict Analysis Application Health Release Readiness Quality Metrics Go No Go Decision Enabler Monitoring Tools DAST Tools Test Automation Tools Test Management Tools Performance Testing Tools Distribution is Restricted. Copyright , Cigniti Technologies 23

24 Q&A Distribution is Restricted. Copyright , Cigniti Technologies 24