In This Webinar TODAY S PRESENTERS Tom Chavez: SOASTA, Sr. Product MarkeEng Manager Andy Pemberton: CloudBees, Senior Director, SoluEons Architecture & ConsulEng TWO COMPANIES, A SINGLE SOLUTION SOASTA: The leader in Cloud and Mobile TesEng CloudBees: The Enterprise Jenkins Company Together, we offer a complete cloud-based service: ConEnuous Load TesEng & ConEnuous Delivery with Jenkins 1
QuesEons? Ask in QuesEon window
Poll QuesEon What best describes your status with ConEnuous IntegraEon / ConEnuous Delivery (CI/CD)? (single answer) a) Determining if CI/CD is a good opeon a) ImplemenEng CI/CD a) PracEcing CI b) PracEcing CD c) What is CI/CD? 3
Agenda Load teseng: why? Why don t we load test earlier? ConEnuous Load and Performance TesEng CloudBees and the Enterprise Jenkins PlaYorm ConnecEng load tests to your CD Pipeline Q&A QuesEons: Submit in the queseon box during event 4
Why Should We Load Test? o Ensure that site will perform well at load Test hardware configuraeons Test applicaeon, database, components Test third party partners o To be ready for peak season Black Friday, Cyber Monday, Christmas Tax filing, Insurance filing, SuperBowl commercial 5
Why Should We Load Test? 6
Why Don t We Load Test Earlier? Copyright 2014 SOASTA, Inc. - All rights reserved.7 2015 SOASTA, Inc.-All rights reserved.
Why DON T We Load Test Earlier? Because it s hard. o Tools require coding skills we don t have (C, VB?) o We don t have a scalable test environment o We don t have hardware for load generation o It s a different tool o There are too many other tests to do 8
Why DON T We Load Test Earlier? Because it s hard. o Tools require coding skills we don t have (C, VB?) o We don t have a scalable test environment o We don t have hardware for load generation o It s a different tool o There are too many other tests to do Because it takes too long. o Scripting takes forever o Results are slow o Fast setup of environments is impossible o We re dependent on IT or other teams o We re too agile 9
Why DON T We Load Test Earlier? Because it s hard. o Tools require coding skills we don t have (C, VB?) o We don t have a scalable test environment o We don t have hardware for load generation o It s a different tool o There are too many other tests to do Because it takes too long. o Scripting takes forever o Results are slow o Fast setup of environments is impossible o We re dependent on IT or other teams o We re too agile Because no one ever told us to. o There are no performance coverage requirements o There isn t a performance baseline for reference o There isn t a defined performance target o No one is managing to a performance metric 10
Why DON T We Load Test Earlier? Because it s hard. o Tools require coding skills we don t have (C, VB?) o We don t have a scalable test environment o We don t have hardware for load generation o It s a different tool o There are too many other tests to do Because it takes too long. o Scripting takes forever o Results are slow o Fast setup of environments is impossible o We re dependent on IT or other teams o We re too agile Because no one ever told us to. o There are no performance coverage requirements o There isn t a performance baseline for reference o There isn t a defined performance target o No one is managing to a performance metric Until Now. 11
Poll QuesEon What do you see holding back a more iteraeve load teseng approach? (check all that apply) a) Skills gaps b) Lack of test environment(s) c) Slow test development d) Test tools limitaeons e) 3rd Party Dependencies f) Undefined Performance Requirements g) Lack of Management Push h) Other 12
Load TesEng Has Been an Elite Event Iteration 1 Automated unit tests Iteration 2 Automated unit & functional tests Iteration 3 Regression test suite Iteration n Regression test suite Load & Performance Testing Deploy Issues? SThe Issue: High expectaeons from tests run with diminishing Eme windows Code-level performance bugs found late set enere project back Separate team and schedule conflict with delivery goals Major issues escape to produceon 13
ConEnuous Load TesEng Validate performance piece by piece Iteration 1 Automated unit tests Small perf test Iteration 2 Automated unit & functional tests Small perf tests Iteration 3 Regression test suite Perf smoke Incl: tests Iteration n Regression test suite Perf smoke Incl: tests End to End Production Scale Test Deploy Production perf tests Development/Test team DevOps Performance/Ops team The SoluEon: Run incremental performance tests automaecally with each iteraeon Establish a trend line for performance early Developers find code and architecture issues early Performance teams focus on end-to-end infrastructure and user related teseng Minimize produceon escapes and implement produceon teseng 14
ConEnuous Performance (for Web & Mobile) The Path to a Performance Culture Manage with a new metric! I JUST MADE A CODE CHANGE Jenkins launches suitable cloudbased test environment Identify bottlenecks & inform teams, just like unit and functional tests in Jenkins Daily (or more frequent) updates keep teams performance focused CloudTest runs load tests via Jenkins 15
Poll QuesEon What types of Load TesEng are you doing today? (Single Answer) a) Not doing any load teseng b) Non-Web Protocols (not mobile) c) Primarily web protocols d) Primarily mobile web traffic e) Both web and mobile f) All web, mobile & non-web protocols 16
Enabling ConEnuous Performance TesEng with Jenkins Pipeline Andy Pemberton Twitter: @apemberton Email: apemberton@cloudbees.com 2015 SOASTA, Inc. All rights reserved. 17
Have you met Jenkins? #1 CI/CD Automation Engine Open Source Highly Extensible About 8 years old Easy to install/use 1000+ plugins Widely adopted 100K+ installations 18
Jenkins is the #1 Automation Platform Used for CI Continuous Integration Commit Build Test Code Commit Code Scan Compile /Build Tests Developer Source Code Control System Results Report 19
Jenkins is Also Your Key to Continuous Delivery Dev Continuous Delivery Commit Build Test Stage Deploy Prod Code Commit Code Scan Compile /Build Tests Developer Source Code Control System Deploy Stage Deploy Prod Results Report Complex Delivery Pipelines Delivery of App and Config 20
Over 1000 Jenkins Plugins for Tool Integration Automated API-based testing frameworks, static code analysis tools. Automated Regression Tests Load & Security Tests Load Testing Tools and Security Scanning technology Release management and release automation software Release (to Test) User Acceptance Tests Automated and manual UAT tools Automated API-based testing frameworks. Source code control systems and related asset management tools. Unit Tests Build & Integrate Commit Release (to Prod) Release management, release automation, and change control software. Integrated Development Environments (IDEs), editors/compilers/debuggers, code review and collaboration tools. Develop Environment and Configuration Management Software 21
Jenkins is the Hub of the CD Ecosystem Commit Build Test Stage Deploy Run Plug-ins for all your tools On-premise or in the cloud 22
Are You Using Today? Chances are someone in your organization is. 23 23
24 24
How Do You Manage Jenkins and Testing at Enterprise Scale? 25 25
About CloudBees Jenkins founder, Kohsuke Kawaguchi, CloudBees CTO We optimize Jenkins for the enterprise Jenkins Operations Center Jenkins OSS Support Enterprise-class plugins Commit code and support the Jenkins Community Jenkins training and consulting Jenkins User Conferences (JUC) and newsletters we are the Jenkins Enterprise experts! 26
CloudBees Jenkins Enterprise Professional support High availability Large installations Security Optimized utilization Continuous Delivery 27 27
Additional Jenkins Services from CloudBees Jenkins Certified Training Delivered via CloudBees partners Jenkins Consulting Delivered via CloudBees Professional Services Team Jenkins Newsletter Community news and expert tips Jenkins User Conferences Learn from other community members 28 28
How Can CloudBees Jenkins Enterprise Help with Your CD Strategy? Jenkins Pipeline Created by CloudBees to provide Jenkins with a way to define flexible delivery pipelines that allow for a rich set of new delivery scenarios Manage and share testing-as-code Deployment Tracking and Verfiication Track the history of your deployments including who kicked off a deployment and what code changes were included in it Create pipelines that leverage verification tests to determine the quality of a new deployment Restart builds, tests, and deployments where they break Leverage pipeline checkpoints to restart delivery pipelines without a full (expensive) rebuild PipelineStage View Provides deep insight into the execution of your workflow stages so you can detect problems and performance issues Manage complex level-of-detail testing pipelines Dynamic Environment Creation Spin up new test/deployment environments automatically as part of your delivery pipeline Jenkins Analytics Monitor and analyze performance across all your Jenkins instances Monitor, measure and manage adoption of testing, CI and CD best practices And more...central source of orchestration to bridge dev, test, and ops alike 29 29
#Jenkins2 Greatly Enhanced UI/UX for easier startup and beoer security 30
#Jenkins2 Jenkins Pipeline-as-Code front and center 31
Jenkins Pipeline and Pipeline Stage View Enable Full Continuous Delivery Pipelines Dev Continuous Delivery Prod Code Commit Developer Source Code Control System 32
CloudBees Jenkins Pipeline Stage View Restartable Checkpoints Artifact Downloads 33 33
Monitoring, Visualization and Analytics Jenkins Pipeline Stage View Visibility into Pipeline status Jenkins Analytics Cross-Team Build Analytics among many other features, tailored for enterprise organizations using Jenkins @ scale 34
35
ConEnuous Load TesEng What are we going to show o SOASTA Cloud Test Components / Architecture o Demo: CloudTest UI Overview Jenkins Setup Downloading and installing CloudTest plug-in CreaEng a Performance Job in Jenkins CreaEng a Pipeline in Jenkins Reviewing Results 36
Architecture: US Grid1 CloudTest Manager Always up at: Cloudtestmanager.soasta.com CloudTest Main server 2 AWS Virginia 2x Google Iowa DevOps Team 1 Jill Jenkins Up as needed at any IP Address/ Domainname CloudTest server Up as needed at any IP Address/ Domainname 2 Azure California WW Grid1 2 AWS Virginia 2x Google Iowa dev.acme.com CPU,DISK, Memory,I/O Etc. Raj Sri 2 Azure California 2015 SOASTA, Inc. All rights reserved. 37
Demo Preview: CloudTest Manager Always up at: Cloudtestmanager.soasta.com US Grid1 2 AWS Virginia Jenkins Enterprise on CloudBees DevOps Team 1 CloudTest Main server Up as needed at any IP Address/ Domainname 2x Google Iowa 2 Azure California dev.acme.com Jill Raj Sri 2015 SOASTA, Inc. All rights reserved. 38
Demo 2015 SOASTA, Inc. All rights reserved. 39
QuesEons? Ask in QuesEon window
Resources www.soasta.com Knowledge Center White Papers Webinar Recordings Case Studies CloudLink Community Support Tutorials Video Contact SOASTA: www.soasta.com/cloudtest/ info@soasta.com 866.344.8766 Follow us: twioer.com/soastainc facebook.com/cloudtest www.cloudbees.com Try it @ www.cloudbees.com/get-started Documenta]on CloudBees Jenkins Platform Workflow in Jenkins White Papers: Making the Leap from Continuous Integration to Continuous Delivery The Business Value of Continuous Delivery Contact CloudBees www.cloudbees.com info@cloudbees.com 1.781.404.5100 Follow us: twioer.com/cloudbees facebook.com/cloudbees 41
How to Introduce Cloud Based Load TesEng to your Jenkins ConEnuous Delivery Pipeline April 20, 2016 Tom Chavez, SOASTA @TomChavez Andy Pemberton, CloudBees @apemberton