For. Rupinder 240 Singh 251 Virk 202. Dheeraj Chahal. Title and Content. Light 1. Accent 1. Dark 2. Accent 2. Dark 1. Light 2. Hyperlink.

Transcription

1 Title and Content Dark 1 Light 1 Dark 2 Light 2 Accent 1 Accent Trace Accent Replay 3 Accent Based 4 Accent 5 I/O Accent Performance 6 Hyperlink Followed Studies Hyperlink For Enterprise 238 Application Migration Tata Blue 50% Tata Blue 25% Purple 50 % Purple 25 % Yellow 50 % Yellow 25 % Rupinder 240 Singh 251 Virk Dheeraj Chahal Brown 50 % Brown 25 % Green 50 % Green 25 % Light Green 50% Light Green 25%

2 Outline Introduction Problem Statement State of the Art Our Methodology Implementation and Challenges Results Conclusion

3 Introduction CPU % DISK % MEM % NET % APPLICATION OF INTEREST APPLICATION SERVER DATABASE SERVER Source Environment Target Environment

4 Introduction Actual Application Deployment Perform Load Testing Collect Performance Data Target Environment - 3 -

5 Outline Introduction Problem Statement State of the Art Our Methodology Implementation and Challenges Results Conclusion

6 Problem Statement To develop a method for predicting the performance of an IO intensive multithreaded enterprise application workload on target storage systems before migrating and deploying it

7 Approaches Available for Prediction Running the synthetic workload generated by IO subsystem. Using characterization tools like IOmeter. IO trace replay is another popular technique that can be used for reproducing the application characteristics on a target platform

8 Benefits of trace-replay Traces are portable. Trace and replay does not require to copy the actual data because data access pattern is important than the actual data itself. Traces are deterministic and prove to be better than other methods like modeling techniques in some situation. 7

9 Outline Introduction Problem Statement State of the Art Our Methodology Implementation and Challenges Results Conclusion

10 State of the Art PseudoApp B. C. Tak, C. Tang, H. Huang, and L. Wang, Pseudoapp: Performance prediction for application migration to cloud, in Integrated Network Management (IM 2013), 2013 IFIP/IEEE International Symposium on, May 2013, pp ROOT Z. Weiss, T. Harter, A. C. Arpaci-Dusseau, and R. H. ArpaciDusseau, Root: Replaying multithreaded traces with resourceoriented ordering, in Proceedings of the Twenty-Fourth ACM Symposium on Operating Systems Principles, ser. SOSP 13. New York, NY, USA: ACM, 2013, pp CloudProphet A. Li, X. Zong, M. Zhang, S. Kandula, and X. Yang, Cloudprophet: predicting web application performance in the cloud, ACM SIGCOMM Poster,

11 Outline Introduction Problem Statement State of the Art Our Methodology Implementation and Challenges Results Conclusion

12 Methodology Record Trace Generate IO trace on DB system for varying Concurrency Replay Trace Replay traces on the target machine and collect performance data Extrapolate Extrapolate the data collected on the target machine using extrapolation tool

13 Methodology Trace of IO related Sytem Calls System call is how a program requests a service from an operating system's kernel. open() close() read() write() pread() pwrite() System Calls futex() send() recv() mkdir() seek() creat()

14 Trace Example for "vi mysql.log" execve("/bin/vi", [...], [/* 26 vars */]) = 0 < > open("mysql.log", O_RDONLY) = 3 < > TIMESTAMP FILENAME (microseconds) read(3, ""..., 8192) = 2899 < > read(3, "", 65536) = 0 < > close(3) = 0 < > Display it on the console PERMISSIONS FILE DESCRIPTOR open("mysql.log", O_WRONLY O_CREAT O_TRUNC, 0644) = 3 < > write(3, ""..., 2899) = 2899 < > fsync(3) = 0 < > close(3) = 0 < > Flush the cache The data hits the disk's platters TIME TAKEN

15 Trace Sample

16 System Call execution path for Multi-tier Application CLIENT APPLICATION SERVER DATABASE SERVER OPEN OPEN OPEN OPEN SEND open("./ibdata1", READ O_RDWR) = 3 < > open("./ib_logfile0", RECV O_RDWR) = 8 < > open("./ib_logfile1", SEND O_RDWR) = 9 RECV < > OPEN PREAD RECV SEND RECV PREAD SEND WRITE WRITE WRITE

17 Outline Introduction Problem Statement State of the Art Our Methodology Implementation and Challenges Results Conclusion

18 Implementation STRACE IOReplay PerfExt Source Environment Target Environment Prediction On Target 500, 1000?

19 Record Trace Perform Load testing on the Source Environment. Record the trace on Source Database Server using Strace

20 Record Trace with Single Strace Process 1. strace -f ttt s o mysqltrace vi mysql.log -f -ttt -e -s T execve ("/bin/vi", [...], [/* 26 vars */]) = execve ("/bin/vi", ["vi", "mysql.log"], [/*27 vars */]) = 0 < > < > Profile Mysql 2. strace tttt s0 -f -o mysqltrace /usr/sbin/mysqld 3. strace tttt s0 -f -o mysqltrace p

21 Strace is single threaded Challenges with Strace Equiz For 50 user.

22 Record Trace with Multiple strace Processes 1. nohup strace tttt s0 -f -o v1000_start /usr/sbin/mysqld Nohup strace Mysql Total Mysql Thread = Manually Login with one user in order to create connection pool (say 1000) Total Mysql thread = = Kill 3 <nohup strace pid> It creates nohup.out. It contains pid of all mysql threads. 4. A automated script will parse nohup.out and attach strace process without " f " option strace tttt s0 -o pid.straceout -p <mysql thread pid>

23 Record Trace with Multiple strace Processes 4. Run the Load Test. 5. Shutdown Mysql. It will kill all strace processes 6. Merge v1000_start and All pid.straceout files into one file v1000_trace 7. Sort the above file according to the timestamp

24 Record Trace with Multiple strace Processes

25 Replay Trace Move the trace to new Target Database Server. Replay trace using IOReplay Collect Performance Data

26 Replay Trace 1. Move the Mysql Directory to Target Database server Ibdata1 Ib_logfile0 & Ib_logfile1 *.ibd files MYI.proc 2. Change file paths to the current directory 3. Ioreplay t exact r f v1000_trace exact - Try to make calls in same time relatively from start of the program. 4. Use atop and iostat utility for capturing utilization data

27 Modifications To IOReplay 1. Implementation for replay of fsync() command. 2. Modification to perform multithreaded execution of the trace

28 Introduction to PerfExt N X Test for Small Users Resource Utilization

29 Load Testing Results Input List All the Servers Gather at least two test scenarios Test for 150 users Throughput X = 16.3 pages/sec Disk % = 5

30 Extrapolated Results of PerfExt Bottleneck resource and server Maximum number of users supported N* = 3700 Maximum Throughput X Max = 94 pages/sec 29

31 Output of PerfExt Maximum number of users supported N* = 4500 CPU% = 94 % Can limit the maximum utilization at bottleneck resource by controlling the number of users. 30

32 Outline Introduction Problem Statement State of the Art Our Methodology Implementation and Challenges Results Conclusion

33 Experimental Setup Application of Interest Equiz jpetstore Description Automatic code evaluation framework ecommerce J2EE application that deals with pets. Parameter Values Application Server Apache Tomcat Database Sever Mysql 5.6 Load Injector Grinder 3.2 Thinktime 5 seconds Test Duration 20 minutes

34 System configurations Storage Type Low-end HDD High-end HDD SSD Disk Model WD Caviar SE Serial ATA drive RPM No. of Disk s IO Sched ular File Syste m CFQ ext4 HP- GEN CGQ ext4 Virident. FlashMAX Drive Micron-slc PCI e Defau lt ext3 System Config 8 Core Xeon 2.6 GHz,,6MB L2 cache 16 Core Xeon 2.4 GHz,12MB L2 cache 16 Core Xeon 2.4 GHz,,12MB L2 cache Linux Kernel CentOS 6.5, CentOS 6.6, CentOS 6.6,

35 Results Testing Scenarios Low-End to High-End Disk High-End to SSD Comparison of Disk Utilization Actual versus Trace Replay on Target Environment. Source versus Target Environment Predicted Result on Target Environment

36 Results Actual versus Trace Replay on Target Environment

37 Disk Utilization : Equiz MID-HDD

38 Disk Utilization : Equiz HDD-SSD

39 Disk Utilization : JPET MID-HDD

40 Disk Utilization : JPET SSD

41 Results Source versus Target Environment

42 Disk Utilization Deveopment Vs Production Equiz HDD-HDD Equiz HDD-SSD JPET HDD-HDD JPET HDD-SSD

43 Results Predicted Results on Target Environment 42

44 Equiz Low-HDD to Mid-HDD

45 Equiz High-HDD to SSD

46 JPET Low-HDD to Mid-HDD

47 JPET HDD to SSD

48 Conclusion The primary objective is cross platform application performance prediction without delpoying the application. We have successfully predicted most of the performance metrics of an application with high prediction accuracy when application database is migrated from a low level HDD to a high level HDD and SSD. We capture only the IO trace of the applications on a database server while ignoring CPU and network traces on database or application server. Currently we are doing POCs and a tool is being developed to automate the prediction process

49 Title and Content Dark 1 Light 1 Dark 2 Light 2 Accent 1 Accent Accent Accent Accent 5 Accent 6 Thank You Hyperlink Followed Hyperlink Tata Blue 50% Tata Blue 25% Purple 50 % Purple 25 % Yellow 50 % Yellow 25 % Brown 50 % Brown 25 % Green 50 % Green 25 % Light Green 50% Light Green 25%