Introduction 2 Load Testing

Size: px

Start display at page:

Download "Introduction 2 Load Testing"

MargaretMargaret Lee
5 years ago
Views:

1 Introduction 2 Load Testing Gus Björklund, Progress Software Exchange Australia, October 18, 20, 22, 2010 Melbourne, Brisbane, Sydney

2 Abstract Load testing your application is well worth the time and effort. By load testing you can determine if your system is sized properly to handle the expected workload, what will happen when you add more users, whether application changes you make will have adverse effects, and prove to prospects that your application can meet their performance and scalability requirements. Now you can learn how in this session. ~40 minutes 2

3 Please ask questions as we go sometimes I do not explain something well enough, or you may want to know more 3

4 Summary Load testing an application is a bit more work, a bit more time consuming, and a bit more expensive than you might think but well worth doing. 4

5 What? 5

6 What is (software) load testing? Process of subjecting (your) software to a known workload, observing, and measuring its response + 6

7 Why? 7

8 Capacity Planning Transactions Per Second Number of Users if you increase workload, can system still do its job? 8

System Sizing system 3 system 2 system 1 0 200 400 600

9 System Sizing system 3 system 2 system max users Will your intended hardware be sufficient? 9

10 Regression testing TPS version 4.1 version 5.0 beta Did your changes cause unintended consequences? 10

11 Stress testing Transactions per Hour Avg, Response (Seconds) Number of Users How far can you push it before it breaks? 11

12 Sales scales to 10,000 users! Prove performance, scalability, and headroom to a prospect 12

13 How? 13

14 The Basic Idea Run the application With requisite number of users With specified input data Each performing some number of operations For a suitable amount of time Collect data Response times Transaction rates Correct outputs 14

15 Q: But where am I going to get 500 people to use the application? 15

16 Q: But where am I going to get 500 people to use the application? A: You can do that, but you don't have to. There is another way. 16

17 Q: But where am I going to get 500 people to use the application? A: You can do that, but you don't have to. There is another way. Use software to simulate the people using your application. 17

18 Load Testing Software Virtual users run the application in place of humans Parameterized test scripts and scenarios Provide user inputs and randomized data Different scripts for different operations Order entry Shipping Payment processing Etc. Data files Contain variable inputs for data values for many transactions 18

19 Some Commercial Load Testing Tools HP LoadRunner Silk Performer Facilita Forecast Scapa TPP Microfocus QALoad IBM Rational Performance Tester All have similar capabilities. Details and depth differ: Supported operating systems Protocols UI types 19

20 20

21 1000 User LoadRunner Setup Used by a Customer LoadRunner Controller Character LoadRunner Injector (4 Cores) HP UX Character Clients HP UX Database Server 21 GUI LoadRunner Injector (32 Cores) Windows GUI Clients

22 LoadRunner Character UI Virtual User LoadRunner Controller scripts scripts data LoadRunner Analyzer reports results data LoadRunner Injector Terminal Emulator Char UI ABL Application A Virtual User 22

23 LoadRunner GUI Virtual User LoadRunner Controller scripts scripts data LoadRunner Analyzer results data LoadRunner Injector Remote Desktop Protocol A Virtual User reports GUI ABL Application 23

24 Windows GUI Script Example # Type in account number in the Account field account = 100; set_window ("ATM"); edit_set ("Account", account); # Enter the amount to be deposited in the amount field. amount = 50; set_window ("ATM"); edit_set ("Amount", amount); # Press the Deposit button. button_press ("Deposit"); # Close client application. menu_select_item ("File; Exit"); 24

25 Script Parameterization # Type in account number in the Account field account = {Account_number}; set_window ("ATM"); edit_set ("Account", account); # Enter the amount to be deposited in the amount field. amount = {Amount}; set_window ("ATM"); edit_set ("Amount", amount); # Press the Deposit button. button_press ("Deposit"); # Close client application. menu_select_item ("File; Exit"); {} values read from file 25

26 Preparing 26

27 Different Solutions for Different Application Architectures What kind of UI do you have? Character Windows GUI.NET GUI Web browser Etc. Can you test back-end without a UI? App server calls from 4GL test program Batch jobs 27

28 What's Involved? Planning Workload model definition Resource allocation and procurement Scenario and script development Data generation Test execution 28

29 Model Your Workload What are the important operations performed by your application? What is NOT important? Number of users? How many of each type? Nightly batch runs? Month-end, quarter-end, year-end processing? 29

30 Sample Workload Model Transaction type User mix Transaction time (sec) User login/logout 100 % 0.2 OE order entry 45 % OE order shipping 10 % OE order inquiry 5 % AP costing 20 % AR cash receipts 15 % PO order entry 5 % OE pick ticket printing Batch login/logout 1 4 AI archiving 1 30 Transactions / hour / user 30

31 What Should We Measure? Time required to execute fixed amount of work Amount of work executed in a fixed time Response times for various operations Resource consumption Memory CPU Disk I/O rate Disk space Etc. Network traffic 31

32 Planning, Part 1 Determine need Establish goals and success criteria Determine what data will be needed to establish success or failure Allocate resources Create test plan and procedures Run small-scale tests Identify problems Correct errors Repeat as needed 32

33 Planning, Part 2 Procure hardware Use own hardware Rent or borrow Use a vendor's benchmark center o IBM, HP, Sun, Dell, etc Use Amazon EC2 Set up software Save configured, clean VM's if using Plan for things to go wrong!!! Allow extra time for the unexpected. 33

34 Doing 34

35 Load Test Execution Run the tests Multiple times are results reproducible?? You may have to fix something Archive data and test configuration Everything needed to do it again to get same results Analyze and report results Post-mortem What went well? What did we do wrong? What should we do better next time? 35

36 2 Quick Examples 36

37 example 0: verify capacity & headroom on new hardware 37

38 0: A Customer's Load Test Project Goals: Can we consolidate servers by using VMware ESX? Will proposed new server machine be sufficient? Will it handle 3 years expected growth? 38

39 0: A Customer's Load Test Project Goals: Can we consolidate servers by using VMware ESX? Will proposed new server machine be sufficient? Will it handle 3 years expected growth? Test up to 560 users Response time 1 sec or less for all transactions Scenarios for normal load, peak load, month-end load Must include all High-volume processes High business risk processes Resource-intensive processes 39

40 0: Where was Time Spent? Activity Planning Proof of concept trial Create test scripts Design test scenarios Prepare test data files Test environment setup Run the tests Archive test environment and data Analyze the results Write test report Update documents to reflect reality Contingency time (10 %) duration 30 days 3 days 10 days 2 days 2 days 3 days 9 days 1 days 2 days 3 days 3 days 7 days total 75 days 40

41 example 1: prove performance & scalability to a prospect 41

42 1: ACE hardware IBM Pseries P690+ (32 way) : 1,7 GHz POWER4+ architecture, 128 GB RAM AIX 5.2 with dynamic logical partitioning IBM ESS storage server model channel 2 GB fibre connections between server and disk array (128 disks) 8x 2 GB FC/AL 42

43 1: ACE Load Test Project Labor Summary Application tuning 150 man days Smaller scaled benchmarks 30 man days Loadrunner adapter 50 man days Benchmark scenario 25 man days Benchmark setup 5 man days Actual benchmark (2 weeks) 60 man days + Benchmark centre (machines + infrastructure) $ TOTAL: 320 man days 43

44 3 Things 0) Takes commitment 1) Requires budget and resources 2) Well worth doing Sell more stuff!! Earn more money 44

45 Questions 45

46 Bonus Material 46

47 LoadRunner protocols/interfaces FTP, HTTP, HTTPS, LDAP, DNS, SOAP, COM, RMI, IIOP, SAPGUI, IMAP, POP 3, MAPI, SMTP, SNMP Terminal emulation Windows GUI Windows RDP Citrix ICA Sockets ODBC JDBC and others 47

Oracle s JD Edwards EnterpriseOne IBM POWER7 performance characterization

Oracle s JD Edwards EnterpriseOne IBM POWER7 performance characterization Diane Webster IBM Oracle International Competency Center January 2012 Copyright IBM Corporation, 2012. All Rights Reserved. All