Overview of load testing with Taurus in Jenkins pipeline how to get Taurus installed what a Taurus test script looks like how to configure Taurus to accurately represent use cases Actions in this session: dynamically capture session based values able to replay test at scale feeding data into test more realistic, user-oriented test cases provision a cloud environment accurately reflects production circumstances allows for most realistic test possible bring it all together into Jenkins pipeline build job to be automated Agenda: load testing continuous delivery pipeline in cloud load testing refresher and why we want to do this In functional testing, "We should fail the build if the test fails...load testing is more of a human decision." Taurus (CLI tool) as a testing tool what a file looks like installing the tool elements of test structure cloning production environments for testing can do so at little cost can automate it - game-changer for conducting tests in automation pipelines Jenkins pipeline how to configure Results, Reporting, and Cohesion Load Testing starts with simple idea: users working with applications expectations / criteria of response time and behavior of app under load "it's about scripting and understanding the steps a user will take through the (app), and ensuring they happen with the responsiveness and behavioral dynamics that we expect from the (app)" going toward user satisfaction and customer retention starts with user request and getting a response
turns into a script able to use virtual users to mimic and be tracked number can be multiplied and scaled based on size of app Challenges with Load Scripts finding dynamic, session-based values tokens that prevent certain exploits in your apps differs by user, which can break your test realistic user data avoiding repeating same request in testing - narrow scope validate correct behavior ensure that correct page is being returned production environments "The larger the scale gets, the more complicated the infrastructure gets, the weirder things happen." access to clouds allow you to duplicate your production environment Example Test Architecture: running against an application in AWS, has elastic load balancer running w/ 4 individual instances behind it, in an Amazon virtual private cloud, in an AWS network tool chain: GitHub, Visual Studio for editor, Packer for build, Fugue to deploy infrastructure into AWS, Taurus to run load tests, BlazeMeter for graphing results Getting Started: download Taurus at www.gettaurus.org platform-specific Python app; BZT package check dependencies; some XML libraries are idiosyncratic and tricky Taurus is triggered with BZT command YML or JSON file for argument; works well for integration Test case: sample app - Bootstrap framework, modeled after a medical benefit site login feature with a Taurus script, want to capture whole process and result in a "hello, user" banner at the top Test steps: 1) hit main home page, login page 2) send data to the form a) need to find "request verification" token embedded in page, in HTML source 3) send requests with embedded token 4) check for success by validating email address / username is present in return HTML Constructing the Taurus test and YML file:
* Taurus can use different execution engines to drive load - we're using JMeter for default 1) create execution conditions defines how test should unfold concurrency is number of virtual users going through the steps defined in test logic ramp up period helps deliver load in a gradual way duration is how long test should run; varies by app and intention iterations is how many times the test should run through the logic cycle the scenario is a bridge to the next section of the test that defines the web activity 2) start with default address (replace later with dynamic value from environment) think time creates small delay between requests "Users never move as fast as a computer." 3) data sources first row contains the names of the variables used in the requests (i.e. email, pass) CSV file 4) requests (steps that a user will take in application) homepage URL value matching for token found in HTML source (sometimes found in headers) * finding and capturing these values for reuse is one of the hardest things about load testing; if your test is failing when it shouldn't be, check the source code first!* login URL regular expression with greedy operator becomes "RVT" in subsequent steps and in post-ops post login (same end point, different verb) reusing email and pass in CSV file, which is now being used to send data into the app, plus token verify banner is correct ("Hello, ") logoff operation reuse token and post Taurus Modules: running scripts setting up criteria BlazeMeter reports Console output data is very readable average times Percentile Failures tracks local performance Run the test Locally first good for debugging see if the test works
see what performance is like allows to create idea of baseline very different from real-world environment BlazeMeter reports www.blazemeter.com captures graphs for CIDC tests put metrics in CSV or XML file for download and storage can be useful for Jenkins plugin data arranged visually hits per second average throughput errors in real time, sorted by response code and assertion name overall and average response times percentiles table of request stats Taurus Test in Jenkins Pipeline Packer creates Amazon image machine deploy using Fugue Pipeline feature is a scripted build in a Jenkinsfile getflow method is a great way build branches before pull requests are merged (multi-branch pipeline) different people working on different branches GitHub repo Jenkins will index and look at different branches Jenkinsfile using Groovy file works with concept of a stage graphs out build process in a user-friendly way when preparing environment, first step is to check out sourcecode use if conditions to guide different branches prepares a production-clone environment executes load test in real-world conditions upon passing, tears test down Production-Clone environment in AWS building a network creating an auto-scaling group creating a load balancer launch AMI AWS reports as "in service" when the instance is ready to receive requests export public address of load balancer
execute Taurus test teardown infrastructure