2011 Oracle Corporation and Affiliates. Do not re-distribute!

Transcription

1

2 <Insert Picture Here> How to Write Low Latency Java Applications Charlie Hunt Java HotSpot VM Performance Lead Engineer

3 Who is this guy? Charlie Hunt Lead JVM Performance Engineer at Oracle 12+ years of Java performance experience 20+ years of (general) performance experience Lead author of Java Performance book, hot off the presses!!!

4 The following is not intended to outline our general product direction. It is intended for information purposes only, and may not be incorporated into any contract. It is not a commitment to deliver any material, code, or functionality, and should not be relied upon in making purchasing decisions. The development, release, and timing of any features or functionality described for Oracle s products remains at the sole discretion of Oracle.

5 Agenda What you need to know about GC <Insert Picture Here> What you need to know about JIT compilation Tools to help achieve low latency

6 What you need to know about GC Java HotSpot VM Heap Layout Eden Survivor Survivor The Heap Old Generation: for older / longer living objects Permanent Generation : for VM & class meta-data 6

7 What you need to know about GC Java HotSpot VM Heap Layout New allocations Retention of young objects during minor GCs Eden Survivor Survivor Promotions of longer lived objects during minor GCs The Heap Old Generation: for older / longer living objects Permanent Generation : for VM & class meta-data 7

8 What you need to know about GC Important Concepts (1 of 3) Frequency of minor GC is dictated by Application object allocation rate Size of the Eden Space

9 What you need to know about GC Important Concepts (1 of 3) Frequency of minor GC is dictated by Application object allocation rate Size of the Eden Space Frequency of object promotion into old generation is dictated by Frequency of minor GCs (how quickly objects age) Size of the survivor spaces (large enough to age effectively) Ideally you want to promote as little as possible

10 What you need to know about GC Important Concepts (1 of 3) Frequency of minor GC is dictated by Application object allocation rate Size of the Eden Space Frequency of object promotion into old generation is dictated by Frequency of minor GCs (how quickly objects age) Size of the survivor spaces (large enough to age effectively) Ideally you want to promote as little as possible Object retention impacts latency more than object allocation GC only visits live objects GC duration is a function of live objects and graph complexity

11 What you need to know about GC Important Concepts (2 of 3) Object allocation is very cheap! 10 CPU instructions in common case

12 What you need to know about GC Important Concepts (2 of 3) Object allocation is very cheap! 10 CPU instructions in common case Reclamation of new objects is also very cheap! Remember, only live objects are visited in a GC

13 What you need to know about GC Important Concepts (2 of 3) Object allocation is very cheap! 10 CPU instructions in common case Reclamation of new objects is also very cheap! Remember, only live objects are visited in a GC So, Do not be afraid to allocate short lived objects especially for immediate results GCs love small immutable objects and short-lived objects especially those that seldom survive a minor GC

14 What you need to know about GC Important Concepts (3 of 3) However, Do not do needless allocations

15 What you need to know about GC Important Concepts (3 of 3) However, Do not do needless allocations more frequent allocations means more frequent GCs more frequent GCs imply faster object aging faster promotions more frequent needs for either; concurrent Old Generation collection, or Old Generation compaction (i.e. Full GC)

16 What you need to know about GC Important Concepts (3 of 3) However, Do not do needless allocations more frequent allocations means more frequent GCs more frequent GCs imply faster object aging faster promotions more frequent needs for either; concurrent Old Generation collection, or Old Generation compaction (i.e. Full GC) In short

17 What you need to know about GC Important Concepts (3 of 3) However, Do not do needless allocations more frequent allocations means more frequent GCs more frequent GCs imply faster object aging faster promotions more frequent needs for either; concurrent Old Generation collection, or Old Generation compaction (i.e. Full GC) In short It is better to use short-lived immutable objects than longlived mutable objects btw, there's other advantages of immutable objects

18 What you need to know about GC Ideal Situation After application initialization phase, only experience minor GCs and old generation growth is negligible Ideally, never experience need for Old Generation collection Minor GCs are (generally) the fastest GC Start with Parallel GC (i.e. -XX:+UseParallelOldGC or -XX:+UseParallelGC) Parallel GC offers the fastest minor GC times Move to CMS if Old Generation collection is needed Minor GC times will be slower due to promotion into free lists But, Old Generation compaction (via Full GC) delayed by concurrent collection Hopefully for the duration of program execution

19 What you need to know about GC A tidbit about Concurrent GCs Concurrent collectors require a write barrier to track potential hidden live objects The write barrier tracks all writes to objects and records the creation and removal of references between objects Wanna know more about write barriers? Google it search for Java GC write barrier

20 What you need to know about GC A tidbit about Concurrent GCs Concurrent collectors require a write barrier to track potential hidden live objects The write barrier tracks all writes to objects and records the creation and removal of references between objects Wanna know more about write barriers? Google it search for Java GC write barrier The point is...

21 What you need to know about GC A tidbit about Concurrent GCs Concurrent collectors require a write barrier to track potential hidden live objects The write barrier tracks all writes to objects and records the creation and removal of references between objects Wanna know more about write barriers? Google it search for Java GC write barrier The point is... Write barriers introduce overhead The amount of overhead depends on the implementation STW GCs, i.e. don't require write barriers Hence, ideal situation is... use Parallel GC or ParallelOld GC and avoid Old Gen collection, i.e. avoid Full GC

22 What you need to know about GC GC Friendly Programming (1 of 3) Large objects

23 What you need to know about GC GC Friendly Programming (1 of 3) Large objects Expensive to allocate (perhaps not so much via fast path ) Expensive to initialize (remember Java Spec... zeroing)

24 What you need to know about GC GC Friendly Programming (1 of 3) Large objects Expensive to allocate (perhaps not so much via fast path ) Expensive to initialize (remember Java Spec... zeroing) Large objects of different sizes can cause Java heap fragmentation Applicable to non-compacting, or partially compacting GCs So,

25 What you need to know about GC GC Friendly Programming (1 of 3) Large objects Expensive to allocate (perhaps not so much via fast path ) Expensive to initialize (remember Java Spec... zeroing) Large objects of different sizes can cause Java heap fragmentation Applicable to non-compacting, or partially compacting GCs So, Avoid large object allocations if you can Especially frequent large object allocations during application steady state

26 What you need to know about GC GC Friendly Programming (2 of 3) Data Structure Re-sizing

27 What you need to know about GC GC Friendly Programming (2 of 3) Data Structure Re-sizing Avoid re-sizing of array backed container objects Use the object constructor that takes an explicit size

28 What you need to know about GC GC Friendly Programming (2 of 3) Data Structure Re-sizing Avoid re-sizing of array backed container objects Use the object constructor that takes an explicit size Re-sizing leads to unnecessary object allocation Can also contribute to fragmentation (again for noncompacting GCs)

29 What you need to know about GC GC Friendly Programming (2 of 3) Data Structure Re-sizing Avoid re-sizing of array backed container objects Use the object constructor that takes an explicit size Re-sizing leads to unnecessary object allocation Can also contribute to fragmentation (again for noncompacting GCs) Object pooling potential issues

30 What you need to know about GC GC Friendly Programming (2 of 3) Data Structure Re-sizing Avoid re-sizing of array backed container objects Use the object constructor that takes an explicit size Re-sizing leads to unnecessary object allocation Can also contribute to fragmentation (again for noncompacting GCs) Object pooling potential issues Contributes to number of live objects visited during a GC Remember GC duration is a function of live objects Access to the pool requires some kind of locking Potential for a scalability issue

31 What you need to know about GC GC Friendly Programming (3 of 3) Finalizers

32 What you need to know about GC GC Friendly Programming (3 of 3) Finalizers PPP-lleeeaa-ssseee don't do it! Requires at least 2 GCs cycles and GC cycles are slower If possible, add a method to explicitly free resources when done with an object

33 What you need to know about GC GC Friendly Programming (3 of 3) Finalizers PPP-lleeeaa-ssseee don't do it! Requires at least 2 GCs cycles and GC cycles are slower If possible, add a method to explicitly free resources when done with an object Reference Objects Use as an alternative to finalizers, (as a last resort)

34 What you need to know about GC GC Friendly Programming (3 of 3) Finalizers PPP-lleeeaa-ssseee don't do it! Requires at least 2 GCs cycles and GC cycles are slower If possible, add a method to explicitly free resources when done with an object Reference Objects Use as an alternative to finalizers, (as a last resort) Important note on SoftReferences

35 What you need to know about GC GC Friendly Programming (3 of 3) Finalizers PPP-lleeeaa-ssseee don't do it! Requires at least 2 GCs cycles and GC cycles are slower If possible, add a method to explicitly free resources when done with an object Reference Objects Use as an alternative to finalizers, (as a last resort) Important note on SoftReferences Referent is cleared by the GC, how aggressive it is at clearing is at the mercy of the GC's implementation The aggressiveness dictates the degree of object retention

36 What you need to know about GC Subtle Object Retention Consider the following: class MyImpl extends ClassWithFinalizer { private byte[] buffer = new byte[1024 * 1024 * 2];... What's the consequences if ClassWithFinalizer has a finalizer?

37 What you need to know about GC Subtle Object Retention Consider the following: class MyImpl extends ClassWithFinalizer { private byte[] buffer = new byte[1024 * 1024 * 2];... What's the consequences if ClassWithFinalizer has a finalizer? At least 2 GC cycles to free the byte[] buffer How to fix it?

38 What you need to know about GC Subtle Object Retention Consider the following: class MyImpl extends ClassWithFinalizer { private byte[] buffer = new byte[1024 * 1024 * 2];... What's the consequences if ClassWithFinalizer has a finalizer? At least 2 GC cycles to free the byte[] buffer How to fix it? class MyImpl { private ClassWithFinalier classwithfinalizer; private byte[] buffer = new byte[1024 * 1024 * 2];...

39 What you need to know about GC Subtle Object Retention What about inner classes?

40 What you need to know about GC Subtle Object Retention What about inner classes? Remember that inner classes have an implicit reference to the outer instance Potentially can increase object retention

41 What you need to know about GC Subtle Object Retention What about inner classes? Remember that inner classes have an implicit reference to the outer instance Potentially can increase object retention Again, increased object retention more live objects at GC time increased GC duration

42 Agenda What you need to know about GC <Insert Picture Here> What you need to know about JIT compilation Tools to help achieve low latency

43 What to know about JIT compilation The Basics Optimization decisions are made based on Classed that have been loaded and code paths executed

44 What to know about JIT compilation The Basics Optimization decisions are made based on Classed that have been loaded and code paths executed JIT compiler does not have full knowledge of entire program

45 What to know about JIT compilation The Basics Optimization decisions are made based on Classed that have been loaded and code paths executed JIT compiler does not have full knowledge of entire program Rather, it has knowledge of only what has been class loaded and code paths executed

46 What to know about JIT compilation The Basics Optimization decisions are made based on Classed that have been loaded and code paths executed JIT compiler does not have full knowledge of entire program Rather, it has knowledge of only what has been class loaded and code paths executed Hence, optimization decisions makes assumptions about how a program has been executing it knows nothing about what has not been class loaded or executed

47 What to know about JIT compilation The Basics Optimization decisions are made based on Classed that have been loaded and code paths executed JIT compiler does not have full knowledge of entire program Rather, it has knowledge of only what has been class loaded and code paths executed Hence, optimization decisions makes assumptions about how a program has been executing it knows nothing about what has not been class loaded or executed Note, assumptions may turn out (later) to be wrong...

48 What to know about JIT compilation The Basics Optimization decisions are made based on Classed that have been loaded and code paths executed JIT compiler does not have full knowledge of entire program Rather, it has knowledge of only what has been class loaded and code paths executed Hence, optimization decisions makes assumptions about how a program has been executing it knows nothing about what has not been class loaded or executed Note, assumptions may turn out (later) to be wrong it must keep information around to recover which (may) limit types of optimization(s)

49 What to know about JIT compilation The Basics Optimization decisions are made based on Classed that have been loaded and code paths executed JIT compiler does not have full knowledge of entire program Rather, it has knowledge of only what has been class loaded and code paths executed Hence, optimization decisions makes assumptions about how a program has been executing it knows nothing about what has not been class loaded or executed Note, assumptions may turn out (later) to be wrong it must keep information around to recover which (may) limit types of optimization(s) New class loading or code path possible de-opt/re-opt

50 What to know about JIT compilation Competing Forces: Inlining & Virtualization Biggest optimization impact from method inlining

51 What to know about JIT compilation Competing Forces: Inlining & Virtualization Biggest optimization impact from method inlining Virtualized methods are the biggest barrier to inlining

52 What to know about JIT compilation Competing Forces: Inlining & Virtualization Biggest optimization impact from method inlining Virtualized methods are the biggest barrier to inlining Good news

53 What to know about JIT compilation Competing Forces: Inlining & Virtualization Biggest optimization impact from method inlining Virtualized methods are the biggest barrier to inlining Good news JIT compiler can de-virtualize methods if it only sees 1 implementation of a virtualized method effectively makes it a mono-morphic call

54 What to know about JIT compilation Competing Forces: Inlining & Virtualization Biggest optimization impact from method inlining Virtualized methods are the biggest barrier to inlining Good news JIT compiler can de-virtualize methods if it only sees 1 implementation of a virtualized method effectively makes it a mono-morphic call Bad news

55 What to know about JIT compilation Competing Forces: Inlining & Virtualization Biggest optimization impact from method inlining Virtualized methods are the biggest barrier to inlining Good news JIT compiler can de-virtualize methods if it only sees 1 implementation of a virtualized method effectively makes it a mono-morphic call Bad news if JIT compiler later discovers an additional implementation it must de-optimize, re-optimize for 2 nd implementation now we have a bi-morphic call

56 What to know about JIT compilation Competing Forces: Inlining & Virtualization Biggest optimization impact from method inlining Virtualized methods are the biggest barrier to inlining Good news JIT compiler can de-virtualize methods if it only sees 1 implementation of a virtualized method effectively makes it a mono-morphic call Bad news if JIT compiler later discovers an additional implementation it must de-optimize, re-optimize for 2 nd implementation now we have a bi-morphic call This type of de-opt & re-opt will likely lead to lesser peak performance, especially true when / if you get to the 3 rd implementation because now its a mega-morphic call

57 What to know about JIT compilation Competing Forces: Inlining & Virtualization Important Point(s)

58 What to know about JIT compilation Competing Forces: Inlining & Virtualization Important Point(s) Discovery of additional implementations of virtualized methods during steady state will likely slow down your application (impacts application throughput, predictability and likely response times)

59 What to know about JIT compilation Competing Forces: Inlining & Virtualization Important Point(s) Discovery of additional implementations of virtualized methods during steady state will likely slow down your application (impacts application throughput, predictability and likely response times) A mega-morphic call can limit or inhibit inlining capabilities

60 What to know about JIT compilation Competing Forces: Inlining & Virtualization Important Point(s) Discovery of additional implementations of virtualized methods during steady state will likely slow down your application (impacts application throughput, predictability and likely response times) A mega-morphic call can limit or inhibit inlining capabilities Should I write JIT Compiler Friendly Code?

61 What to know about JIT compilation Competing Forces: Inlining & Virtualization Important Point(s) Discovery of additional implementations of virtualized methods during steady state will likely slow down your application (impacts application throughput, predictability and likely response times) A mega-morphic call can limit or inhibit inlining capabilities Should I write JIT Compiler Friendly Code? Ahh, that's a premature optimization!

62 What to know about JIT compilation Competing Forces: Inlining & Virtualization Important Point(s) Discovery of additional implementations of virtualized methods during steady state will likely slow down your application (impacts application throughput, predictability and likely response times) A mega-morphic call can limit or inhibit inlining capabilities Should I write JIT Compiler Friendly Code? Ahh, that's a premature optimization! Advice?

63 What to know about JIT compilation Competing Forces: Inlining & Virtualization Important Point(s) Discovery of additional implementations of virtualized methods during steady state will likely slow down your application (impacts application throughput, predictability and likely response times) A mega-morphic call can limit or inhibit inlining capabilities Should I write JIT Compiler Friendly Code? Ahh, that's a premature optimization! Advice? Write code in its most natural form, let the JIT compiler figure out how to best optimize it Do analysis (using tools) to identify the problem areas and make code changes (only) for those problem areas

64 Agenda What you need to know about GC <Insert Picture Here> What you need to know about JIT compilation Tools to help achieve low latency

65 Tools for GC Analysis Observing GC Behavior Two Modes Offline, after the fact, GC Histo, a java.net project (just search for GCHisto ) It's a GC log visualizer Logs should include -XX:+PrintGCTimeStamps or -XX:+PrintGCDateStamps along with -XX:+PrintGCDetails

66 Tools for GC Analysis Observing GC Behavior Two Modes Offline, after the fact, GC Histo, a java.net project (just search for GCHisto ) It's a GC log visualizer Logs should include -XX:+PrintGCTimeStamps or -XX:+PrintGCDateStamps along with -XX:+PrintGCDetails Online while the application is running VisualGC plug-in for VisualVM (distributed in the JDK its in the bin directory, launched as 'jvisualvm')

67 Tools for GC Analysis Observing GC Behavior Two Modes Offline, after the fact, GC Histo, a java.net project (just search for GCHisto ) It's a GC log visualizer Logs should include -XX:+PrintGCTimeStamps or -XX:+PrintGCDateStamps along with -XX:+PrintGCDetails Online while the application is running VisualGC plug-in for VisualVM (distributed in the JDK its in the bin directory, launched as 'jvisualvm') VisualVM's Memory Profiler for unnecessary object allocation and object retention

68 Tools for GC Analysis Observing GC Behavior Two Modes Offline, after the fact, GC Histo, a java.net project (just search for GCHisto ) It's a GC log visualizer Logs should include -XX:+PrintGCTimeStamps or -XX:+PrintGCDateStamps along with -XX:+PrintGCDetails Online while the application is running VisualGC plug-in for VisualVM (distributed in the JDK its in the bin directory, launched as 'jvisualvm') VisualVM's Memory Profiler for unnecessary object allocation and object retention In future a JDK 7 Update release Mission Control

69 Tools for JIT Compilation Oracle Solaris Studio Performance Analyzer

70 Tools for JIT Compilation Oracle Solaris Studio Performance Analyzer Works with both Solaris and Linux But,

71 Tools for JIT Compilation Oracle Solaris Studio Performance Analyzer Works with both Solaris and Linux But, better experience on Solaris (more productive gets you to things you want to look at the quickest)

72 Tools for JIT Compilation Oracle Solaris Studio Performance Analyzer Works with both Solaris and Linux But, better experience on Solaris (more productive gets you to things you want to look at the quickest) Method profiler, lock profiler, CPU hardware counter profiler and you can see generated JIT compiler code and Java source code

73 Tools for JIT Compilation Oracle Solaris Studio Performance Analyzer Works with both Solaris and Linux But, better experience on Solaris (more productive gets you to things you want to look at the quickest) Method profiler, lock profiler, CPU hardware counter profiler and you can see generated JIT compiler code and Java source code Did I mention its free? :-)

74 Tools for JIT Compilation Oracle Solaris Studio Performance Analyzer Works with both Solaris and Linux But, better experience on Solaris (more productive gets you to things you want to look at the quickest) Method profiler, lock profiler, CPU hardware counter profiler and you can see generated JIT compiler code and Java source code Did I mention its free? :-) Let's take a (quick) look

75 Tools for JIT Compilation Oracle Solaris Studio Performance Analyzer Works with both Solaris and Linux But, better experience on Solaris (more productive gets you to things you want to look at the quickest) Method profiler, lock profiler, CPU hardware counter profiler and you can see generated JIT compiler code and Java source code Did I mention its free? :-) Let's take a (quick) look Similar tools Intel VTune AMD CodeAnalyst

76 One last plug ;-) More goodies and fun

77 Acknowledgments A special thanks to Tony Printezis and John Coomes. Much of the GC related material, especially the GC friendly, is material originally drafted by Tony and John.

78

79