Welcome to the session...

Transcription

1 Welcome to the session... Copyright 2013, Oracle and/or its affiliates. All rights reserved. 02/22/2013 1

2 The following is intended to outline our general product direction. It is intended for information purposes only, and may not be incorporated into any contract. It is not a commitment to deliver any material, code, or functionality, and should not be relied upon in making purchasing decisions. The development, release, and timing of any features or functionality described for Oracle s products remains at the sole discretion of Oracle. Basically, don t trust anything I say... Copyright 2013, Oracle and/or its affiliates. All rights reserved. 02/22/2013 2

3 Java Flight Recorder - Next generation diagnostics and profiling Rickard Bäckman HotSpot JVM Compiler Team, Java Platform

4 public class Example { public void savetransaction(transaction tn) { synchronized (somelock) { // save to some shared data structure } } } 4

5 public class Example { public void savetransaction(transaction tn) { System.err.println(threadId + Waiting for lock ); synchronized (somelock) { System.err.println(threadId + Got lock ); // save to some shared data structure } System.err.println(threadId + Lock released ); } } 5

6 ... few million lines of output [T1] Waiting for lock [T3] Waiting for lock [T2] Waiting for lock [T2] Got lock [T1] Got lock [T1] Lock released [T3] Got lock [T3] Lock released [T2] Lock released... another million lines of output 6

7 Java Flight Recorder - Overview Released in 7u40 7

8 Java Flight Recorder - Overview 8

9 Java Flight Recorder - Overview Event Recorder & Profiler 9

10 Java Flight Recorder - Overview Implemented in the JVM 10

11 Java Flight Recorder - Overview No restart required 11

12 Java Flight Recorder - Overview After-the-facts analysis - What went wrong? 12

13 Event Recorder JVM tracks lots of information 13

14 Event Recorder JVMTI? 14

15 Event Recorder Instrumented JVM & JDK - Make that information visible 15

16 Profiler What is my CPU doing?!? 16

17 Inside the JVM Core in the JVM High-level things in Java 17

18 Inside the JVM Access to JVM internals & subsystems 18

19 Inside the JVM Class Unloading & Safepoints 19

20 Non-intrusive Designed to run all the time. In your production environment 20

21 Non-intrusive Low overhead Captures information that is already there 21

22 Non-intrusive Enable & disable at runtime 22

23 After-the-facts Problem in the environment 23

24 After-the-facts What happened before? 24

25 After-the-facts Keeps history of information 25

26 After-the-facts Java Mission Control - Dump the data on an SLA breach 26

27 Things JFR captures Compilation statistics Socket read/write VM Operations Context switches Object Count Exceptions Class Unload Garbage Collection Method Compilation CPU Load Class Load Object allocation Threads Locks Metaspace Statistics CodeCache Profiling Thread.sleep File read/write Garbage Collection statistics 27

28 Sampling Profiler How to implement a profiler? 28

29 Sampling Profiler JVMTI 29

30 Sampling Profiler Byte code instrumentation 30

31 Sampling Profiler Stop the thread, walk the stack, collect the methods. 31

32 Demo 32

33 Enable Java Flight Recorder -XX:+UnlockCommercialFeatures -XX:+FlightRecorder 33

34 Start recording -XX:StartFlightRecording=filename=<...>,duration=5m 34

35 Start on demand Java Mission Control or jcmd <pid> JFR.start <options> 35

36 Some interesting options stackdepth=<nr> dumponexit=<true/false> 36

37 Events Abstraction of captured information 37

38 Events Captures something interesting Latency? Performance? 38

39 Instant Events Happens instantly - Thread start, Thread stop, Class unload 39

40 Duration Events Happens over a period of time - Compilation, Garbage Collection, Wait for a lock 40

41 Requestable Events Something that should be done repeatedly - CPU Load, Profiling Events Context switches 41

42 Settings & Filtering Enable & disable 42

43 Settings & Filtering Threshold 43

44 Settings & Filtering Frequency / Period 44

45 Settings & Filtering Stack traces 45

46 Settings & Filtering Control signal / noise 46

47 Settings & Filtering Filter early! 47

48 Recording Start a recording 48

49 Recording Generates a stream of events 49

50 Recording Applies a filter and captures the remaining 50

51 Recording One stream of Events 51

52 Recording Multiple recordings 52

53 Recording The Union of Events 53

54 Profiles A preconfigured settings for a recording 54

55 Profiles default & profile 55

56 Profiles Create your own 56

57 Java Flight Recorder Implementation 57

58 Java Flight Recorder Challenges Assembly! File format Implementation Events C++ Low level Memory buffers 58

59 Challenges Overhead must be low 59

60 Challenges Memory footprint 60

61 Challenges The JVM environment is changing 61

62 Challenges Scalability 62

63 Events in detail Header Optional Size Event ID End Time Start Time Thread StackTrace Payload 63

64 Events in detail Self-describing Events 64

65 Event Metadata Name, Path, Description 65

66 Event Payload Metadata Name, Type, Description & Content Type 66

67 Content Type Semantics of a Value Content Type Displayed as Bytes 42 MB Percentage 42% Address 0xDEADBEEF Millis 242 ms 67

68 Content Type Can be multiple Values Content Type Class Displayed Class Loader Name Modifiers (public,final) 68

69 Event definition in HotSpot <event id="threadsleep" path="java/thread_sleep" label="java Thread Sleep" has_thread="true" has_stacktrace="true" is_instant="false"> <value type="millis" field="time" label="sleep Time"/> </event> What does it actually mean? path is a identifier for the UI, label is a description of the Event has_thread means each event is tied to a Thread has_stacktrace means get a stack trace for the Event is_instant decides whether it is instant event or not Finally the payload with a type name and a description. 69

70 Event in C++ EventThreadSleep event; if (event.should_commit()) { event.set_time(millis); event.commit(); } An example of an event in the HotSpot source code. This event repots the time a thread slept. Thread.sleep() So the event is created and when the thread returns from sleep we check if the event was enabled and should be committed. If so we save the time and commit. 70

71 Or in assembler 281: cmp BYTE [rbp-0x60],0x0 # should commit 285: je 300 # jump if zero 287: mov QWORD rbp-0x58],rbx # save time 291: lea rdi,[rbp-0x70] # get pointer 295: call 0x101b17dc4 # commit This is basically the assembler code generated for the same code (for amd64). Reserve some memory on the stack, initialize it. Check if the event was enabled, otherwise jump away. Save the time, get the pointer and call commit. The overhead of an event that is disabled is very small. Compare a value to zero and jump away. 71

72 Buffers Thread-local buffers 72

73 Buffers Promoted to Global buffers - Global buffers are circular Global Buffer Global Buffer Global Buffer Global Buffer 73

74 File Format Binary proprietary 74

75 File Format Fast writing 75

76 File Format Self Contained Header Event Records Event Meta Data 76

77 Low-level Enough High-Level Problems & Solutions 77

78 Starting Problem Classes are referenced by Events 78

79 Starting Problem java.lang.string 79

80 Starting Problem org.springframework.security.ui.preauth.preaut henticatedgrantedauthoritieswebauthenticatio ndetails 80

81 Starting Problem 99 characters! 81

82 Starting Problem class Klass { u8 _trace_id; } Introduce a unique ID event id class count 17 java.lang.integer 2 event id class count java.lang.integer java.lang.string java.lang.math java.util.list 82

83 New problem Mapping 83

84 Class List Must be synchronized 84

85 Class List Class Load? 85

86 Class List Class Load? - Startup performance! 86

87 Class List End of a Recording? 87

88 Class List End of a Recording? - Classes can unload! 88

89 Class List Piggyback Class Unloading! 89

90 Class List The Class List can grow Big 90

91 Class List Lots of classes + Long class names = Lots of wasted memory 91

92 Tagged Classes Reserve a bit in the ID 92

93 Tagged Classes Tag referenced classes! #define CLASS_USED 1 void use_class_id(klass* const klass) { klass->_trace_id = CLASS_USED; } 93

94 Class List Classes come & go. Don t waste memory 94

95 Checkpoints Special Event 95

96 Checkpoints Writes the Class List, Resets the tags & clears the Class List class_pool.lookup(242) java.lang.integer method_pool.lookup(314) java.lang.math:pow() 96

97 Constant Pool Classes, Methods, Stack Traces, Threads, Thread Groups, Strings 97

98 Checkpoints - revisit Events + Constant Pools + Event Metadata = Checkpoint 98

99 Checkpoints - revisit checkpoint events constant pools meta-data... 99

100 Checkpoints - revisit Contain everything required to parse the events prior to the checkpoint 100

101 More information Whitepaper User Guide Forum java_hotspot_virtual_machine/java_mission_control 101

102 Q&A 102

103 Thank you 103