It s Good to Have (JVM) Options

Transcription

1 It s Good to Have (JVM) Options Chris Hansen / Sr Engineering Manager Tori Wieldt / Technical Evangelist JavaOne 2017

2 Safe Harbor This presentation and the information herein (including any information that may be incorporated by reference) is provided for informational purposes only and should not be construed as an offer, commitment, promise or obligation on behalf of New Relic, Inc. ( New Relic ) to sell securities or deliver any product, material, code, functionality, or other feature. Any information provided hereby is proprietary to New Relic and may not be replicated or disclosed without New Relic s express written permission. Such information may contain forward-looking statements within the meaning of federal securities laws. Any statement that is not a historical fact or refers to expectations, projections, future plans, objectives, estimates, goals, or other characterizations of future events is a forward-looking statement. These forward-looking statements can often be identified as such because the context of the statement will include words such as believes, anticipates, expects or words of similar import. Actual results may differ materially from those expressed in these forward-looking statements, which speak only as of the date hereof, and are subject to change at any time without notice. Existing and prospective investors, customers and other third parties transacting business with New Relic are cautioned not to place undue reliance on this forward-looking information. The achievement or success of the matters covered by such forward-looking statements are based on New Relic s current assumptions, expectations, and beliefs and are subject to substantial risks, uncertainties, assumptions, and changes in circumstances that may cause the actual results, performance, or achievements to differ materially from those expressed or implied in any forward-looking statement. Further information on factors that could affect such forward-looking statements is included in the filings New Relic makes with the SEC from time to time. Copies of these documents may be obtained by visiting New Relic s Investor Relations website at ir.newrelic.com or the SEC s website at New Relic assumes no obligation and does not intend to update these forward-looking statements, except as required by law. New Relic makes no warranties, expressed or implied, in this presentation or otherwise, with respect to the information provided. 2

3 Too Many Options.x10 Source: Source: 3

4 4

5 Focus 5

6 Visualization software courtesy wordle.net 6

7 Whence Data? New Relic monitors 500,000 JVMs running apps and services We report JVM options to aid in troubleshooting and discovery The JVM option and other data here is fully anonymized Filtered for Oracle Hotspot JVM only 7

8 What do we count as the use of an option? The key... -Xmx512M -Xloggc:/var/log/jvmgc.log -XX:MaxPermSize=64M -XX:+PrintGCDetails Value, sort of 8

9 Most Common JVM Options 9

10 Categories of Options Memory sizing Diagnostics GC tuning Other tuning 10

11 Basics of Java Heap and GC Source: Jörge Prante 11

12 Object Tenure req sesh Eden sesh S0 sesh S1 sesh Tenured (i.e. Old) 12

13 Source: 13

14 Resource Limits (Object Heap) -Xmx* : Heap max (87%) -Xms* : Heap initial (64%) -Xss* : Per-thread stack size (11%) -XX:MaxHeapFreeRatio=* : When to shrink heap (6%) -XX:MinHeapFreeRatio=* : When to expand heap (5%) -Xmn* : Size of young generation (6%) -XX:SurvivorRatio=* : Eden : Survivor (6%) 14

15 Resource Limits (PermGen/Metaspace) -XMaxPermSize=* : PermGen max (29%) -XPermSize=* : PermGen threshold for GC (12%) -XX:MaxMetaspaceSize=* : Class metadata max [unlimited] (13%) -XX:MetaspaceSize=* : Class metadata threshold for GC (7%) 15

16 Garbage Collector Diagnostics -Xloggc:* : Log GC events to a file, which sets... (23%) -XX:+PrintGC, -verbose:gc : Log at a basic level (18%) -XX:+PrintGCDetails : Log at a detailed level (19%) -XX:+PrintGCTimeStamps : Include relative timestamp (18%) -XX:+PrintGCDateStamps : Include absolute timestamp (16%) -XX:+PrintHeapAtGC : Include more heap detail (10%) -XX:+PrintTenuringDistribution : Include post-gc tenure summary (8%) -XX:+PrintGCApplicationStoppedTime : Log safepoint time (6%) 16

17 -XX:+PrintGC (-Xloggc:*, -verbose:gc) Java HotSpot(TM) 64-Bit Server VM ( b01) for bsd-amd64 JRE (1.8.0_144-b01), built on Jul :07:42 by "java_re" with gcc (Based on Apple Inc. build 5658) (LLVM build ) Memory: 4k page, physical k(780800k free) /proc/meminfo: CommandLine flags: -XX:MaxHeapSize= XX:MaxNewSize= XX:+PrintGC -XX:+PrintGCTimeStamps -XX:SurvivorRatio=1 -XX:TargetSurvivorRatio=2 -XX:+UseParallelGC 1.087: [GC (Allocation Failure) 58880K->8704K(466432K), secs] 1.515: [GC (Allocation Failure) 67584K->12147K(466432K), secs] 2.197: [GC (Allocation Failure) 71027K->19407K(466432K), secs] 2.644: [GC (Allocation Failure) 78287K->31108K(525312K), secs] 4.221: [GC (Metadata GC Threshold) K->18311K(525312K), secs] 4.232: [Full GC (Metadata GC Threshold) 18311K->17905K(525312K), secs] 17

18 -XX:+PrintTenuringDistribution 1.112: [GC (Allocation Failure) Desired survivor size bytes, new threshold 7 (max 15) 58880K->8719K(466432K), secs] 1.627: [GC (Allocation Failure) Desired survivor size bytes, new threshold 7 (max 15) 67599K->12121K(466432K), secs] 2.368: [GC (Allocation Failure) Desired survivor size bytes, new threshold 7 (max 15) 71001K->19519K(466432K), secs] 2.808: [GC (Allocation Failure) Desired survivor size bytes, new threshold 7 (max 15) 78399K->31713K(525312K), secs] 4.377: [GC (Metadata GC Threshold) Desired survivor size bytes, new threshold 6 (max 15) K->18620K(525312K), secs] 4.388: [Full GC (Metadata GC Threshold) 18620K->18060K(525312K), secs] 18

19 -XX:+PrintGCDetails 1.076: [GC (Allocation Failure) [ PSYoungGen: 58880K->8689K(116736K)] 58880K->8689K(466432K), sec s] [Times: user=0.01 sys=0.00, real=0.01 secs] 1.601: [GC (Allocation Failure) [PSYoungGen: 67569K->12219K(116736K)] 67569K->12219K(466432K), s ecs] [Times: user=0.02 sys=0.00, real=0.01 secs] 2.248: [GC (Allocation Failure) [PSYoungGen: 71099K->19431K(116736K)] 71099K->19431K(466432K), secs] [Times: user=0.03 sys=0.01, real=0.01 secs] 2.667: [GC (Allocation Failure) [PSYoungGen: 78311K->31397K(175616K)] 78311K->31397K(525312K), secs] [Times: user=0.04 sys=0.01, real=0.03 secs] 4.247: [GC (Metadata GC Threshold) [PSYoungGen: K->18470K(175616K)] K->18470K(525312K), secs] [Times: user=0.02 sys=0.00, real=0.01 secs] 4.259: [Full GC (Metadata GC Threshold) [PSYoungGen: 18470K->0K(175616K)] [ParOldGen: 0K->17922K(349696K)] 18470K->17922K(525312K), [Metaspace: 20851K->20847K( K)], secs] [Times: user=0.11 sys=0.01, real=0.04 secs] 19

20 -XX:+PrintHeapAtGC {Heap before GC invocations=2 (full 0): PSYoungGen total K, used 67610K [0x b , 0x baa80000, 0x c ) eden space 58880K, 100% used [0x b ,0x b ,0x b ) from space 57856K, 15% used [0x b ,0x b42069b0,0x b ) to space 57856K, 0% used [0x b ,0x b ,0x baa80000) ParOldGen total K, used 0K [0x , 0x , 0x b ) object space K, 0% used [0x ,0x ,0x ) Metaspace used 16469K, capacity 16654K, committed 16768K, reserved K class space used 2078K, capacity 2139K, committed 2176K, reserved K 1.553: [GC (Allocation Failure) 67610K->11771K(466432K), secs] Heap after GC invocations=2 (full 0): PSYoungGen total K, used 11771K [0x b , 0x be400000, 0x c ) eden space 58880K, 0% used [0x b ,0x b ,0x b ) from space 57856K, 20% used [0x b ,0x b7d7ed80,0x baa80000) to space 57856K, 0% used [0x b ,0x b ,0x b ) ParOldGen total K, used 0K [0x , 0x , 0x b ) object space K, 0% used [0x ,0x ,0x ) Metaspace used 16469K, capacity 16654K, committed 16768K, reserved K class space used 2078K, capacity 2139K, committed 2176K, reserved K } 20

21 Garbage Collector Diagnostics (Files) -XX:+UseGCLogFileRotation (8%) -XX:NumberOfGCLogFiles=* (9%) -XX:GCLogFileSize=* (9%) 21

22 Heap Dump -XX:+HeapDumpOnOutOfMemoryError : hprof heap dump (20%) -XX:HeapDumpPath=* : Path and file name for.hprof file (20%) Analyze hprof file with VisualVM, jhat, or your favorite profiler 22

23 Choosing an Alternate Garbage Collector ParallelGC: Optimized for throughput (60% default) CMS (Concurrent Mark Sweep): Optimized for low pause (28%) G1: Low pause and high throughput (12%) Serial GC: Good for batch processes 23

24 Other Optimizations -XX:+UseCompressedOops : Compress 64-bit ordinary object pointers (12%) -XX:+DisableExplicitGC : Ignore System.gc() (11%) -XX:+AggressiveOpts : Use experimental runtime optimizations (5%) -XX:-OmitStackTraceInFastThrow [5] : Java 5 JIT optimization (7%) -XX:-TieredCompilation [7-] : Use simple JIT compilation (5%) 24

25 What about the least used options? -ea, -enableassertions (<1%) Watch for typos! Quoting/spacing Use proper key-value delimiter: :, =,, etc. Case sensitive JVM won t always warn that these are ignored Watch for multiple with the same key! (Last one wins) 25

26 Test and Monitor the Effects! -XX:SurvivorRatio=1 [v 8] -XX:TargetSurvivorRatio=2 [v 50] -XX:MaxNewSize=256M [v ergo] 26

27 Conclusion People want diagnostics, especially for object allocation and collection Know your garbage collector. Go to CON4479 this afternoon. Consider this MOOC. Defaults are good! Test and monitor your option changes. 27

28 Bonus Content! Crunching the Docker Numbers Full details 28

29 Glamour Stats New Relic has been using and monitoring containers since 2014 Monitored 418 million containers in the last two years Monitor 1.5 million containers average a day Average customer has more than 800 containers running at any given moment Our top ten customers (by Docker container usage) spin up more than 92,000 containers a day 29

30 Java and Docker Docker is used on all kinds of applications: PHP (33%) Java (31%) Ruby (29%) Node (28%) Python (11%).NET (8%) 30

31 Docker Growth Average Number of Containers Average # of containers per company for the year Greatest number of containers in a single day for a single company Greatest number of containers in a year for a single company 28,000 1, ,000 4,800,000 31

32 Pets to Cattle to Bacteria * Average life of containers: 13 hours (2015) to 9.25 hours (2016) * Average life of containers running less than an hour: 10 minutes (2015) to 8 minutes (2016). * 20% of all containers run for less than one minute. 32

33 Conclusions Container usage continues to grow From VM replacement to ephemeral Paradigms are shifting 33

34 fin This page is intentionally left blank 34