New Java performance developments: compilation and garbage collection

Transcription

1 New Java performance developments: compilation and garbage collection Jeroen #jfall17

2 Part 1: New in Java compilation Part 2: New in Java garbage collection 2

3 Part 1 New in Java compilation 3

4 New in Java compilation #jfall17 Compilation basics for the JVM AOT-compilation added Advantages of AOT-compilation Examples using AOT Current limitations Conclusions 4

5 Java compilation basics interpretation and JIT-compilation in the JVM

6 Interpretation versus compilation #jfall17.php file.cpp file.pl file compiler interpreter executable file each line native code OS Hardware 6

7 Interpretation versus compilation #jfall17.php file.cpp file portable source code.pl file compiler interpreter executable file each line native code OS Hardware 7 exe non-portable optimizations faster execution

8 JVM: Interpretation and JIT-compilation #jfall17.java file javac.class file byte code: portable JVM interpreter JIT-compiler native code OS Hardware 8

9 JVM: Interpretation and JIT-compilation #jfall17.java file javac.class file byte code JVM interpreter each line JIT-compiler method code cache native code OS Hardware 9

10 Profile guided JIT-compilation #jfall17.java file javac.class file byte code JVM interpreter profiling JIT-compiler hot method code cache native code OS Hardware 10

11 Profile guided JIT-compilation with adaptive optimizations.class file byte code JVM interpreter profiling JIT-compiler hot method code cache native code focus on hot code: speculatively optimize method inlining branch prediction loop unrolling de-optimization OS Hardware 11

12 C1: client or C2: server compiler #jfall17.class file byte code JVM interpreter profiling client - C1 server - C2 hot code cache C1: quick startup C2: best performance eventually by more optimizations 12 hot: > C1:1500 > C2: invocations + iterations

13 Tiered compilation: Best of C1 and C2 default in Java 8.class file byte code interpreter client - C1 hotter server - C2 JVM profiling hot profiling code cache C1: quick startup C2: best performance eventually by more optimizations 13

14 interpretation versus compilation flags -Xint -Xcomp -Xmixed JIT-compilation default 14

15 Java 9 test code: HelloJUG 15

16 HelloJUG Which is quickest? java HelloJUG java -Xcomp HelloJUG java -Xint HelloJUG 16

17 HelloJUG Real times, average of 5 runs java HelloJUG java -Xcomp java -Xint 252 ms 2272 ms 214 ms 17

18 Tiered compilation levels #jfall17 Interpreter profiling C1 no profiling C1 profiling C2 no profiling code speed common Level 0 hot hotter Level 3 Level 4 hot stop at L1 Level 0 Level 1 18

19 HelloJUG Real times, average of 5 runs java HelloJUG 252 ms java -Xcomp 2272 ms java -Xint 214 ms java -XX:TieredStopAtLevel=1 244 ms 19

20 AOT-compilation added #jfall17 byte code.class file JVM interpreter JIT-compiler native code 20

21 AOT-compilation added #jfall17 byte code.class file Graal jaotc libelf liby.so file native code non-portable -XX:AOTLibrary= JVM interpreter JIT-compiler native code 21

22 AOT in 2 flavors: tiered (with JIT) and non-tiered (no JIT) Graal libelf jaotc byte code.class file liby.so file native code -XX:AOTLibrary= JVM AOT code JIT-compiler native code 22

23 Non-tiered AOT-compilation: no JIT #jfall17 Graal libelf jaotc byte code.class file liby.so file native code -XX:AOTLibrary= JVM AOT code JIT-compiler native code 23

24 Non-tiered compilation with AOT #jfall17 AOT C1 no profiling no profiling profiling no profiling code speed C1 C2 common aot footprint more important than peak performance no JIT-code cache, no compiler threads more predictable behavior constant speed of code, no JIT-compilation taking CPU 24

25 HelloJUG Real times, average of 5 runs java HelloJUG java -Xcomp java -Xint java -XX:TieredStopAtLevel=1 java -XX:AOTLibrary=lib..nont.so ms 2272 ms 214 ms 244 ms 214 ms

26 Tiered AOT-compilation #jfall17 Graal libelf byte code.class file jaotc liby.so file native code -XX:AOTLibrary= JVM AOT code hotter client - C1 server - C2 profiling profiling hot code cache 26

27 Tiered compilation levels with AOT #jfall17 AOT profiling C1 no profiling C1 profiling C2 no profiling code speed common AOT hot hotter Level 3 Level 4 27

28 Why AOT-compilation? #jfall17 Faster startup time compiled/optimized methods immediately available JIT optional, for best peak-performance Reach peak-performance quicker Potentially less memory usage Class data sharing (CDS) No JIT-compiler overhead Less CPU usage less interpreting, profiling, compiling 28

29 AOT-compiling HelloJUG jaotc 29

30 Run HelloJUG using AOT with -XX:AOTLibrary= 30

31 Run HelloJUG using AOT with -XX:AOTLibrary and -XX:PrintAOT 31

32 CreateCalendars micro-benchmark Average of 5 runs, 2 CPU s non-aot tiered AOT non-tiered AOT -Xint > 29 s. real run time (s) C1 time C2 time 0 0,5 1 1,5 2 2,5 3 Shorter is better 32

33 Current limitations #jfall17 For JDK 9 only supported on Linux x64 JDK 10 (18.3) also on MacOS and Windows No use of profiling data during AOT (yet) Only supported for G1 GC and Parallel GC 33

34 AOT-compilation conclusions Promising technique for quicker startup times Most noticeable when user waiting for startup / execution Short run: non-tiered - without JIT-compiler Small devices with little resources IDE s, unit tests, code generation, coding checks,.. Can be 10% to 2x better for short execution or startup AOT-compile and bundle only touched methods of java.base etc. should help more training run in future versions 34

35 Part 1: New in Java compilation Part 2: New in Java garbage collection 35

36 Part 2: New garbage collectors Shenandoah and Epsilon GC

37 New garbage collectors #jfall17 GC basics Serial, Parallel, Concurrent GC Region based: G1 GC Shenandoah GC Epsilon GC Conclusions 37

38 Garbage collection basics Mark&Sweep, Compaction and Mark&Copy

39 Mark and Sweep #jfall17 OOPs GC-roots (stacks..) jpinpoint 39

40 Mark #jfall17 OOPs GC-roots (stacks..) jpinpoint 40

41 Sweep #jfall17 OOPs GC-roots jpinpoint 41

42 Mark-Sweep: Fragmentation #jfall17 After compaction: Compaction is expensive jpinpoint 42

43 Mark-Copy: no fragmentation 43

44 Generational GC: Young and Old Mark Sweep - Compact Eden Survi vor 1 Survi vor 2 Old space Mark Copy 44

45 Serial GC: stop-the-world pauzes GC thread eg. 7 s. 45

46 Serial vs Parallel GC #jfall17 GC thread eg. 7 s. GC threads eg. 2 s. 46

47 Concurrent Mark Sweep GC - mostly concurrent to app GC threads eg. 2x100 ms. STW 10 s. concurrent 47

48 Parallel GC vs CMS throughput vs responsiveness GC threads eg. 2 s. GC threads eg. 2x100 ms. STW 10 s. concurrent 48

49 Parallel GC vs CMS Young and Old GC threads 300 ms. GC threads 2 s. GC threads 300 ms. GC threads 2x100 ms. STW 10 s. concurrent 49

50 CMS problem 1: failures #jfall17 GC threads 300 ms. Concurrent failure GC threads 2x100 ms. STW 10 s. concurrent GC thread 7 s. 50

51 CMS problem 2: not scalable 4 GB heap x4 GC Conc GC 16 GB heap GC GC threads 1200 ms. Concurrent failure GC threads 2x400 ms. STW 40 s. concurrent 51 GC thread 28 s.

52 Solution: regionalize #jfall17 Mark Sweep - Compact Eden Survi vor 1 Survi vor 2 Old space Mark Copy 52

53 G1: Garbage first: young + old with most garbage % %-live objects 90% 40% 90% 80% 20% 15% 50% 53

54 G1: Garbage first: young + old regions with most garbage Concurrent mark in Old Mark-copy: no fragmentation Limit copy #regions to meet -XX:MaxPauseTimeMillis 90% 40% 90% Solves scalability and failures Still stw pauses 54 80% 20% 15% 50%

55 G1 stw-pauses: Mark young + Copy Y&O RSet CSet 55

56 G1 stw-pauses: Mark young + Copy Y&O + Update refs RSet CSet 56

57 G1 pauses: Mark young + Copy Y&O + Update refs CSet 57 RSet

58 Shenandoah GC Ultra low pause times

59 Stw-pause times in G1 how to beat them? Mark Young Copy Young & Old Update refs 59

60 pause times in G1 vs Shenandoah Mark Young Copy Young & Old Update refs 60

61 removed pause times in Shenandoah Mark Young Copy Young & Old concurrent Update refs concurrent 61

62 How to copy Old and update refs concurrently? RSet CSet 62

63 How to copy Old and update refs concurrently? add RSet CSet 63

64 All problems in computer science can be solved by? Jim Coplien - David Wheeler

65 Another level of indirection forwarding pointer 65

66 Copy Old and update refs concurrently by fwd pointer CSet 66

67 Application thread can write objects being evacuated add CSet 67

68 Application thread writes to new copy of object add CSet 68

69 Concurrent update refs #jfall17 CSet 69

70 Concurrent cleanup #jfall17 CSet 70

71 Shenandoah pause times 4 short stw pauses init mark, final mark, init UR, final UR GC(3) Pause Init Mark 0.771ms GC(3) Concurrent marking 76480M->77212M(102400M) ms GC(3) Pause Final Mark 1.821ms GC(3) Concurrent cleanup 77224M->66592M(102400M) 3.112ms GC(3) Concurrent evacuation 66592M->75640M(102400M) ms GC(3) Pause Init Update Refs 0.084ms GC(3) Concurrent update references 75700M->76424M(102400M) ms GC(3) Pause Final Update Refs 0.409ms GC(3) Concurrent cleanup 76244M->56620M(102400M) ms 71

72 Shenandoah pause times 4 short stw pauses init mark, final mark, init UR, final UR GC(3) Pause Init Mark 0.771ms GC(3) Concurrent marking 76480M->77212M(102400M) ms GC(3) Pause Final Mark 1.821ms GC(3) Concurrent cleanup 77224M->66592M(102400M) 3.112ms GC(3) Concurrent evacuation 66592M->75640M(102400M) ms GC(3) Pause Init Update Refs 0.084ms GC(3) Concurrent update references 75700M->76424M(102400M) ms GC(3) Pause Final Update Refs 0.409ms GC(3) Concurrent cleanup 76244M->56620M(102400M) ms 72

73 Elastic search benchmark (Oct 2, 2017) Run time (x100 s) Total pause time (s) Average (x10 ms) Max (x100 ms) Shorter is better G1 73 Shenandoah

74 Parallel GC: max = 161 ms 74

75 G1 GC: max = 38 ms #jfall17 75

76 Shenandoah GC: max = 5,9 ms 76

77 G1 with -XX:MaxGCPauseTimeMillis=6: 29 ms 77

78 Epsilon GC #jfall17 JEP Draft (by Aleksey Shipilev) stw pauzes = 0.0 ms! How? 78

79 Epsilon GC #jfall17 Garbage non-collector JVM shutdown when heap exhausted java -XX:+UnlockExperimentalVMOptions XX:+UseEpsilonGC Binary builds of patched JDK10 available 79

80 No-op GC use cases #jfall17 Performance testing compare with real garbage collectors Minimize overhead garbage free apps short-lived apps active failover before heap exhausted 80

81 Conclusions new GC s #jfall17 Shenandoah GC clearly beats G1 for short pauses Likely to replace G1 as default after JDK9 Can try it out now on JDK8+! Epsilon GC eliminates all GC-overhead (and comfort) if you can avoid GC last-drop performance improvement 81

82 Questions? #jfall17 want to learn more? resources: accelerating Java applications training covers Java 8 & March 2018 thanks for the attention! 82

83 Please rate my talk in the official J-Fall app