Marc Hornbeek DevOps-the-Gray Principal DevOps Consultant, Trace3 Author, DevOps Test Engineering Course The DevOps Institute

Transcription

1

2 HOST EXPERT PANEL Shashi Kiran CMO Quali Marc Hornbeek DevOps-the-Gray Principal DevOps Consultant, Trace3 Author, DevOps Test Engineering Course The DevOps Institute Pascal Joly Director, Technology Partnerships Quali 2

3 Agenda 1 Introduction 2 Principles of Continuous Testing 3 Implementing CT 4 Demo: Continuous Testing 5 Q&A Continuous Testing The Missing Link 3

4 About Quali Venture-funded start-up focused on Cloud-centric DevOps and BizOps automation Delivers Sandboxes - Replicas of production environments Strong Innovation DNA Trusted by customers worldwide customers Cloud providers, Telcos, Enterprises across FSI, Retail, Healthcare and Government ~115 employees in USA, Israel and Europe focused on serving customers worldwide 4

5

6 Infographic Download here:

7 The Rising Importance of (Continuous) Testing SPEED QUALITY TEST

8 Traditional Testing Siloed development and testing Waterfall approach Long lead times between identifying and solving bugs High risk when bugs are missed Rework and delays Source:

9 What is Continuous Testing (CT)? Quality assessment strategy Incremental changes tested continuously Automated tests (large %) DevOps Toolchain integration Production equivalent test environment stages End-to-end testing across the pipeline

10 CT A sign of DevOps Maturity Manual Adhoc Tests Process is unpredictable, poorly controlled and reactive Test Framework pipeline Projects tailor processes from organizations development methodology Dynamic Test and Environment Blueprints Focus on process improvement Automated Test Execution Processes characterized for specific projects. Organization is often reactive Test-driven Release Tools Processes are measured and controlled

11 Hurdles to Implement Continuous Testing People Leadership Culture Training and Expertise Collaboration Dev/Sec/Ops Budget Rewards and Incentives CI CT Product Low % test automation Budget Missing/fragmented toolchain Production equivalent test environment stages Process Continuous Test planning End-to-end Strategy End-to-end testing across pipeline CD

12 Collaborative Culture is the door to DevOps, but Continuous Testing is the key to Continuous Delivery DevOps-the-Gray

13 CT provides Assessment of CD Stage Artifacts

14 Continuous Testing Implementation Strategy Adopt 5 test Tenets Org. and Culture Streamline Process and Workflow Integrating CT Infra. Environment Orchestration and Test Automation

15 1. Five Tenets of Continuous Testing Arrange tests so most likely problems are found in earlier stage of pipeline. Focus on most important tests and results Fail Early Relevance Shift 3 Test Left Fast Conduct each test as early in the pipeline as possible. Arrange tests to run in quick cycles 5 Fail Often Run tests frequently and with many different conditions

16 2. Build Organizations and Culture DevOps does not mandate roles Teams decide role assignments Checklist help ensure tasks are covered. Multiple checklists may be needed to cover different stages of the pipeline such as Pre-Flight, Integration, Delivery, Deployment, etc.

17 3. Process and Workflow Practices Streamline Procurement, install and support: o Infrastructure o Tools o Integration of test tools into tool frameworks o Operating the DevOps pipeline Align Rewards and Incentives Train on Automation Skills and Tools

18 4. Integrated DevOps Test Infrastructure PLAN TEST ASSESS Collaboratively plan integration into each stage of DevOps Infrastructure Automatically trigger tests from master DevOps framework. Run DevOps test assessment for each stage to provide feedback and support workflow decisions. Stage 2

19 4A. Testing Triggers Infrastructure Glue Continuous Test framework is integrated with the DevOps master framework. DevOps test automation can be performed by the DevOps master framework directly or may be implemented using special purpose DevOps test framework tools. Example: DevOps testing for the build stage is triggered when the DevOps master framework triggers the DevOps test framework. TEST

20 4B.Asssessment Actions Processing ASSESS CT test executions produce verdicts. Verdicts provide decision support data for analytics that determine if a change is good enough to proceed to the next stage in the DevOps pipeline or needs to be remediated before proceeding. Analytics may be basic verdict reports or more sophisticated verdict trend analysis. Verdicts Verdict Trends Analytics

21 5. Test Environment Orchestration and Automation Environment Orchestration Automatically setup the test environment and resources (physical and virtual) to match the requirements of a test. Test Automation Execute test tasks without manual work required.

22 5A. Environment Orchestration Test resources vary depending on the complexity and risk of change being tested. A flexible lab environment employs orchestration tools to elastically scale physical and virtual resources for a specific test.

23 5B. Test Automation Continuous testing uses automation to execute tests quickly and consistently and to keep up with the frequent changes that are processed through DevOps pipelines. Create new automated tests at the same time changes are created. Run automated tests matched to specific content being changed. Produce test results automatically to support in-process DevOps test assessments.

24 DevOps Orchestration Stack Testing Orchestration simplifies management of test resource topologies through test environment blueprints. Dev & Integration Build Automation Packaging Config Mgmt. Test Orchestration (Dev) Packaging Delivery CI/CD Pipeline: Application Release Automation & ALM Systems Test Orchestration (Acceptance) Test Orchestration (Staging and Production) Test Resources Cloud

25 CI/CD with CT Leveraging Cloud Sandboxes Pipeline Automic CI/CD Pipeline: Application Release Automation & ALM Systems Dev Load Security Integration Staging/ Production Quali Sandbox Software

26 ENVIRONMENT Quali CloudShell -Sandboxes FEATURES INTERACE BUILD ANY ENVIRONMENT. DEPLOY AS-A-SERVICE DevOps Plug-Ins REST API EMBEDDED GUI CLOUD SANDBOX Applications Data Test Tools Network Storage Compute Security Physical Infra. Virtual Infra. Services Blueprint Modeling Automation & Orchestration Reservation & Scheduling Self-Service Catalog Built-In Networking BI & Analytics TO ANY CLOUD* Bare Metal

27 Test Strategies for Continuous Deployment A/B Testing Canary Testing Blue/Green Testing Reference:

28 Continuous Testing - Benefits CONTINUOUS TESTING Reduced test Time and test wait time Improved SLAs Improved Test Results Analysis Rapid Test Environment Recovery Cost Efficiency of Shared Resources

29 Summary CONTINUOUS TESTING Don t compromise between speed and quality. Support Continuous Delivery Initiative with Continuous Testing Cloud Sandboxes can be orchestrate and automate environments for continuous test From Missing Link to Glue

30 For More Information: Quali Trace3 DevOps Institute

31 THANK YOU 31