Adaptive Instrumentation of Java-Applications for Experiment-Based Performance Analysis

Size: px

Start display at page:

Download "Adaptive Instrumentation of Java-Applications for Experiment-Based Performance Analysis"

Tabitha Norris
5 years ago
Views:

1 Adaptive Instrumentation of Java-Applications for Experiment-Based Performance Analysis Henning Schulz, Albert Flaig, Alexander Wert and André van Hoorn Symposium on Software Performance KIT SOFTWARE DESIGN AND QUALITY GROUP UNIVERSITY OF STUTTGART RELIABLE SOFTWARE SYSTEMS GROUP KIT University of the State of Baden-Wuerttemberg and National Research Center of the Helmholtz Association University of Stuttgart

2 Motivation Application Performance? Application Performance Engineer Instrumentation 2/ Adaptive Instrumentation of Java-Applications for Experiment-Based Performance Analysis

3 Instrumentation Overhead start end (no instrumentation) end (full instrumentation) methoda() methoda() methodb() methodb() methodd() methodd() methodc() methodc() time 3/ Adaptive Instrumentation of Java-Applications for Experiment-Based Performance Analysis

4 Adaptive Instrumentation start end (no instrumentation) end (full instrumentation) methoda() methoda() methodb() methodb() methodd() methodd() methodc() time 4/ Adaptive Instrumentation of Java-Applications for Experiment-Based Performance Analysis

5 Kieker vs. AIM Kieker ( Instrumentation at application start-up Statically adaptive instrumentation Extensible (writer, etc.) Analysis infrastructure Focus on production Regexps for adaptive monitoring AIM ( Instrumentation on demand Dynamically adaptive Extendable (writer, etc.) No analysis at all Focus on experiments Instrumentation description language Use dynamic adaptive instrumentation from AIM Use instrumentation description language Use Kieker s monitoring & analysis infrastructure 5/ Adaptive Instrumentation of Java-Applications for Experiment-Based Performance Analysis

AIM IN EXPERIMENT-BASED CONTEXT Henning Schulz: Adaptive Instrumentation of Java- Applications for Experiment-Based Performance Analysis, Bachelor s

6 AIM IN EXPERIMENT-BASED CONTEXT Henning Schulz: Adaptive Instrumentation of Java- Applications for Experiment-Based Performance Analysis, Bachelor s Thesis, Karlsruhe Institute of Technology, / Adaptive Instrumentation of Java-Applications for Experiment-Based Performance Analysis

7 AIM :Instrumentation Description :Instrumentation Entity instrumentation entity Instrumentation Description Measurement Data Java Process Target Application how :Response Time Probe probe AIM Agent Injected Probes scope :Method Scope pattern = my.package.*(int) where JVM 7/ Henning Schulz - AIM in Experiment-Based Context

8 Evaluation AIM is comparable to existing approaches Compare to Kieker and DiSL Instrumentation process: fast for < 500 methods comparatively slow for > 500 methods Instrumentation overhead: smaller than Kieker s Benefits of adaptapable instrumentation Automated Palladio Resource Demand Estimation 8/ Henning Schulz - AIM in Experiment-Based Context

9 Automated Estimation of Palladio Resource Demands Derive Call Tree TPC-W Deployment enter A methoda() Palladio Model of TPC-W methodb() methodc() methodd() enter B enter C exit C exit B enter D exit D exit A Measure Response Times of All Methods in the Call Tree methoda() methodb() Insert Resource Demands into PCM Model methodc() methodd() enter A enter B enter D enter C exit C exit B exit D exit A Pr 100% 90% 80% 70% 60% 50% 40% 30% 20% 10% Derive Resource Demands by Subtraction of Response Times Simulate Compare Simulation Results response time 9/ Henning Schulz - AIM in Experiment-Based Context

10 Comparing PRD Qualities mean [ms] Reference AIM (adaptive) AIM (full) Kieker (adaptive) Kieker (full) mean error [%] Measurement Simulation Legend: R: Reference A a : AIM adaptive A f : AIM full K a : Kieker adaptive K f : Kieker full 10/ Henning Schulz - AIM in Experiment-Based Context

11 INTEGRATION OF KIEKER & AIM Albert Flaig: Dynamic Instrumentation in Kieker Using Runtime Bytecode Modification, Bachelor s Thesis, University of Stuttgart, / Adaptive Instrumentation of Java-Applications for Experiment-Based Performance Analysis

12 Statically Adaptive Monitoring in Kieker Expensive Checks all Instrumentation Patterns Result is cached Illustration based on Waller et al. (2012) 12/ Albert Flaig - Integration AIM/Kieker

13 Dynamically Adaptive Monitoring in Kieker MonitoredClass monitoredmethod() Illustration based on Waller et al. (2012) 13/ Albert Flaig - Integration AIM/Kieker

14 AIM Integration in Kieker JMX Client Kieker Monitoring Controller Instrumentation Patterns Instrumentation Pattern Process of External Tool Pattern List Loaded Classes Data Flow Component Piece of Data Java Process JVM Instr. API Instrumentation Execution Probe Record Kieker Instrumentor Target Application AIM Probe Snippet BCInjector Methods to instrument Hotswap Javassist Instrumented Classes 14/ Albert Flaig - Integration AIM/Kieker

15 Evaluation Bytecode instrumentation vs. Manual 1. Overhead RQ1: How does it fare in terms of performance? 2. Turnaround RQ2a: How does it fare in terms of turnaround time? RQ2b: Does it scale well? 3. Lost Transactions RQ3: Is it reliable? MooBench 15/ Albert Flaig - Integration AIM/Kieker

16 Evaluation 1: Overhead Comparison Execution Time (µs) Manual Instrumentation Bytecode Instrumentation No Probe Deactivated Probe Collecting Data Writer 16/ Albert Flaig - Integration AIM/Kieker

17 Evaluation 2: Turnaround (Setting) Bytecode Instrumentation Turnaround = Activating probe + first execution of probe Activating probe invokes BCI Manual Instrumentation Turnaround = Activating probe + first execution of probe First execution of probe invokes caching Instrumenting 1,000 dummy classes Comparison runtime BCI vs. caching 17/ Albert Flaig - Integration AIM/Kieker

18 Evaluation 2a: Turnaround Time Bytecode Instrumentation Manual Instrumentation 18/ Albert Flaig - Integration AIM/Kieker

Evaluation 2b: Scalability Comparison Turnaround Time (ms) 500 1000 1500 Bytecode Instrumentation Turnaround Time (ms) 0,3 500 1000

19 Evaluation 2b: Scalability Comparison Turnaround Time (ms) Bytecode Instrumentation Turnaround Time (ms) 0, , ,7 Manual Instrumentation # of classes 19/ Albert Flaig - Integration AIM/Kieker

20 Core Results synergies Small overhead reduction and higher flexibility High turnaround time relevant for production needs optimization Future Work Kieker supporting the instrumentation description language Optimize Performance of AIM s instrumentation step Usage in bigger context (e.g., performance problem diagnostics) Integration into Kieker release 20/ Adaptive Instrumentation of Java-Applications for Experiment-Based Performance Analysis

21 References [Dmitriev, 2004] M. Dmitriev, Profiling java applications using code hotswapping and dynamic call graph revelation [Ehlers et al, 2011] J. Ehlers, A. van Hoorn, J. Waller, and W. Hasselbring, Self-adaptive software system monitoring for performance anomaly localization [Moret et al, 2011] P. Moret, W. Binder, and E. Tanter, Polymorphic bytecode instrumentation [Marek et al, 2012] L. Marek, A. Villazon, Y. Zheng, D. Ansaloni, W. Binder, and Z. Qi, Disl: A domainspecific language for bytecode instrumentation [van Hoorn et al, 2009] A. van Hoorn, M. Rohr, W. Hasselbring, J. Waller, J. Ehlers, S. Frey, and D. Kieselhorst, Continuous monitoring of software services: Design and application of the kieker framework 21/ Adaptive Instrumentation of Java-Applications for Experiment-Based Performance Analysis

22 AIM Integration in Kieker Detailed JMX Client Kieker Monitoring Controller Instrumentation Patterns Instrumentation Pattern JMX Controller Probe Controller Process of External Tool Pattern List Class Loaded Loaded Classes Data Flow Component Piece of Data Java Process Kieker Class Loader JVM Instr. API Instrumentation Execution Probe Record Kieker Instrumentor Target Application AIM Probe Snippet BCInjector Methods to instrument Hotswap Classes to instrument Probe Snippets Javassist Instrumented Classes 22/ Albert Flaig - Integration AIM/Kieker

23 Evaluation 1: Overhead Comparison Manual Instrumentation Bytecode Instrumentation Execution Time (µs) / Albert Flaig - Integration AIM/Kieker

24 Instrumentation Duration (TPC-W) (8 methods) (251 methods) (380 methods) Legend: A: Instr. with AIM K: Instr. with Kieker D: Instr. With DiSL 26/ Adaptive Instrumentation of Java-Applications for Experiment-Based Performance Analysis

25 Instrumentation Duration (Liferay) (6 methods) (2810 methods) (3270 methods) Legend: A: Instr. with AIM K: Instr. with Kieker D: Instr. With DiSL 27/ Adaptive Instrumentation of Java-Applications for Experiment-Based Performance Analysis

26 Overhead (MooBench & TPC-W) Single Method (8 methods) (251 methods)(380 methods) Legend: N: No Instr. A: Instr. with AIM K: Instr. with Kieker 28/ Adaptive Instrumentation of Java-Applications for Experiment-Based Performance Analysis

27 Overhead (Liferay) Legend: N: No Instr. A: Instr. with AIM K: Instr. with Kieker 29/ Adaptive Instrumentation of Java-Applications for Experiment-Based Performance Analysis

A Framework for Application Performance Monitoring and Dynamic Software Analysis

A Framework for Application Performance and Dynamic Software André van Hoorn, J. Waller, and W. Hasselbring Software Engineering Group University of Kiel, Germany April 24, 2012 @ ICPE 12, Boston, MA A.