Fast Java Programs. Jacob

Transcription

1 Fast Java Programs

2 Outline Intro The Stack Measuring Architecture Patterns & APIs Q&A

3 Outline Intro The Stack Measuring Architecture Patterns & APIs Q&A

4

5

6

7 Why?

8 Premature optimization (is not a free ticket to write dumb code)

9 Key points Perform science Mechanical sympathy Performance comes from architecture Build an arsenal of known unknowns

10 Outline Intro The Stack Measuring Architecture Patterns & APIs Q&A

11

12 The Java Language JVM OS Hardware

13 The Java Language JVM OS Hardware

14 CPU PU PU PU PU L/S L/S S S

15 long sum = 0; for( int i=0; i<vals.length; i++ ) { sum += vals[i]; } CPU 0.5ns L1 Cache ~64k cost: 128 loop iterations 5.0ns L2 Cache ~256k cost: 1280 loop iterations 15.0ns LLC 70.0ns Main Memory 20M cost: loop iterations

16 CPU L1 Cache L2 Cache LLC Main Memory

17 store x = 12 store y = 14 Can be eliminated store x = 10

18 CPU L1 Cache L2 Cache LLC Main Memory

19 CPU L1 Cache L2 Cache LLC Main Memory

20 64b (16 java ints) LLC

21 CPU L1 Cache L2 Cache LLC Main Memory

22 CPU CPU L1 Cache L1 Cache L2 Cache L2 Cache LLC Main Memory

23 CPU CPU L1 Cache L1 Cache L2 Cache L2 Cache LLC Main Memory

24 CPU CPU CPU CPU L1 Cache L1 Cache L1 Cache L1 Cache L2 Cache L2 Cache L2 Cache L2 Cache LLC LLC Main Memory Main Memory

25 CPU CPU CPU CPU L1 Cache L1 Cache L1 Cache L1 Cache L2 Cache L2 Cache L2 Cache L2 Cache LLC LLC QPI Main Memory QPI Main Memory

26 CPU CPU CPU CPU L1 Cache L1 Cache L1 Cache L1 Cache L2 Cache L2 Cache L2 Cache L2 Cache LLC LLC QPI Main Memory SSD PCIe QPI Network Main Memory

27 L1 Cache L1 Cache L1 Cache L1 Cache L2 Cache L2 Cache L2 Cache L2 Cache LLC LLC QPI 70.0ns CPU CPU CPU CPU Main Memory SSD PCIe QPI Network Main Memory

28 CPU CPU CPU CPU L1 Cache L1 Cache L1 Cache L1 Cache L2 Cache L2 Cache L2 Cache L2 Cache LLC LLC QPI Main Memory SSD PCIe QPI Main Memory Network 300.0ns

29

30 Core

31 LLC 20MB

32 Memory Bus

33 Other stuff - PCIe, QPI(?) etc.

34 CPU CPU CPU CPU L1 Cache L1 Cache L1 Cache L1 Cache L2 Cache L2 Cache L2 Cache L2 Cache LLC LLC QPI Main Memory SSD PCIe QPI Network Main Memory

35

36 ns

37 CPU PU PU PU PU L/S L/S S S

38 CPU L1 Cache L2 Cache LLC Main Memory

39 CPU CPU L1 Cache L1 Cache L2 Cache L2 Cache LLC Main Memory

40 CPU CPU CPU CPU L1 Cache L1 Cache L1 Cache L1 Cache L2 Cache L2 Cache L2 Cache L2 Cache LLC LLC QPI Main Memory SSD PCIe QPI Network Main Memory

41

42 Key points Modern hardware is fractal Concurrent software is distributed system Memory is major choke point

43 The Java Language JVM OS Hardware

44 Concurrency! Eat lunch Do the dishes Walk the dog Read a book Time

45 Concurrency! Time

46

47 ~ ns Time

48 Time

49 Lunch Dishes Dog Book

50 Key points Cost of mode switch is high Thread scheduling is key part of architecture Java locks can cause context switches Work in batches

51 The Java Language JVM OS Hardware

52 Managed Memory (GC) VM/Compiler (HotSpot) OS Abstraction (StdLib)

53 Managed Memory (GC) VM/Compiler (HotSpot) OS Abstraction (StdLib)

54 Managed Memory (GC) VM/Compiler (HotSpot) OS Abstraction (StdLib)

55 Interpreter int add( int a, int b ) { return a + b; } javac Client JIT (C1) mov add add pop %rdx,%rax %edx,%eax $0x30,%rsp %rbp Server JIT (C2) CPU

56 int dothing( int var ) { return stuff(var); } # int stuff( int ) mov %rdx,%rax add %edx,%eax jmp %idunno # void baz() pop %rbp.. jmp %idunno # int bar() mov %rdx,%rax.. jmp %idunno # void mov add add pop jmp add( int ) %rdx,%rax %edx,%eax $0x30,%rsp %rbp %idunno

57 Managed Memory (GC) VM/Compiler (HotSpot) OS Abstraction (StdLib)

58

59 new Object()

60 Key points HotSpot is fantastic, help it help you new is not free, not even close the first time you meet a new JVM, turn on GC logging, and keep it on.

61 Outline Intro The Stack Measuring Architecture Patterns & APIs Q&A

62

63

64 There are.. Lies Damn lies Statistics Dynamically profiled code that rewrites itself while it executes

65 ops / ns public void bench() { Math.log(Math.PI); }

66 ops / ns public double bench() { return Math.log(Math.PI); }

67 ops / ns public int bench(int reps) { int s = 0; for(int i = 0; i < reps; i++) { s += (x + y); } return s; }

68

69 @State( Scope.Thread ) class MyBenchmark { private double pi = Math.PI; ops / public double bench() { return Math.log(pi); } }

70

71 Angry Users Average

72

73 Credit: Azul Systems

74

75 Outline Intro The Stack Measuring Architecture Patterns & APIs Q&A

76 HTTP Parser Calls Router Processors Services Calls Endpoint

77 HTTP Parser Parsing Uses HTTP Parser Calls Router Routing Uses Router Calls Endpoint Endpoint Execution

78 Thread A HTTP Parser Parsing Calls Routing Uses HTTP Parser Uses Router Router Calls Endpoint Endpoint Execution Thread B

79 Thread A Owns Parsing HTTP Parser s ad Re Uses Router ns w O Routing Uses Endpoint Execution RAM Thread B

80 interface Locks { void lock( long processid, long resourceid ); }

81 interface Locks { void lock( long processid, long resourceid ); } interface Locks { interface Client extends AutoCloseable { void lock( long resourceid ); } Client newclient(); }

82 Key points Architecture > micro optimizations Design with processors and services Single-writer principle Make multi-client services explicit

83 Outline Intro The Stack Measuring Architecture Patterns & APIs Q&A

84 Patterns to help GC

85 // Iterator interface MyEvents extends Iterator<Event> { boolean hasnext(); Event next(); } // Cursor interface MyEvents extends Event { boolean next(); }

86 // Regular heap new byte[..] // "Off-heap" ByteBuffer.allocateDirect(..); // malloc unsafe.allocatememory(..); // realloc unsafe.reallocatememory(..); // free unsafe.freememory(..);

87 Benefits of "raw" memory: - No impact on GC*. Remember, GC time is (usually) relative to heap size. Significantly faster in some cases (1-100x, test!) Channels API pass direct BB straight through, no copying *as long as you keep the memory around!

88 // Classic thing.handle( new MyEvent(.. ) ); // Reset-pattern thing.handle( event.reset(.. ) );

89 class Foo { private final Thing thing =..; public void sendevent() { thing.handle( new Event(.. ) ); } }

90 class Foo { private final Thing thing =..; private final Event event = new Event(); public void sendevent() { thing.handle( event.reset(.. ) ); } }

91 Pooling: Now you have two problems! Or?

92

93 Map<Integer, Object> map =..; int mykey = 12; // Ends up on stack ( = ~free ) map.put( 12,.. ); // Creates Object(!= free )

94 public void myhelpfulmethod( int... vals ) {.. } myhelpfulmethod( 12 ); // Creates array

95 Other tips & tricks

96 int readbyreaderthread; int writtenbywriterthread; // With writer thread running, reader performance: reads/ns // Without writer thread running reads/ns (!)

97 CPU X CPU Y X Cache Cache X Y Y LLC Main Memory

98

99 // Java int writtenbywriterthread; // Java 7 long p01, p02, p03, p04, p05, p06, p07, p08; int writtenbywriterthread;

100 The power of final & class

101 -XX:MinInliningThreshold -XX:MaxInlineSize

102 Collection libraries

103 Outline Intro The Stack Measuring Architecture Patterns & APIs Q&A