High Performance Managed Languages. Martin Thompson

Transcription

1 High Performance Managed Languages Martin Thompson

2 Really, what is your preferred platform for building HFT applications?

3

4 Why do you build low-latency applications on a GC ed platform?

5

6 Agenda 1. Let s set some Context 2. Runtime Optimisation 3. Garbage Collection 4. Algorithms & Design

7 Some Context

8 Let s be clear A Managed Runtime is not always the best choice

9 Latency Arbitrage?

10

11 Two questions

12 Why build on a Managed Runtime?

13 Can managed languages provide good performance?

14 We need to follow the evidence

15

16 Are native languages faster?

17 Time? Skills & Resources?

18 What can, or should, be outsourced?

19 CPU vs Memory Performance

20 How much time to perform an addition operation on 2 integers?

21 1 CPU Cycle < 1ns

22 Sequential Access - Average time in ns/op to sum all longs in a 1GB array?

23 Access Pattern Benchmark Benchmark Mode Score Error Units testsequential avgt ± ns/op ~1 ns/op

24 Really??? Less than 1ns per operation?

25

26 Random walk per OS Page - Average time in ns/op to sum all longs in a 1GB array?

27 Access Pattern Benchmark Benchmark Mode Score Error Units testsequential avgt ± ns/op testrandompage avgt ± ns/op ~3 ns/op

28 Data dependant walk per OS Page - Average time in ns/op to sum all longs in a 1GB array?

29 Access Pattern Benchmark Benchmark Mode Score Error Units testsequential avgt ± ns/op testrandompage avgt ± ns/op testdependentrandompage avgt ± ns/op ~7 ns/op

30 Random heap walk - Average time in ns/op to sum all longs in a 1GB array?

31 Access Pattern Benchmark Benchmark Mode Score Error Units testsequential avgt ± ns/op testrandompage avgt ± ns/op testdependentrandompage avgt ± ns/op testrandomheap avgt ± ns/op ~20 ns/op

32 Data dependant heap walk - Average time in ns/op to sum all longs in a 1GB array?

33 Access Pattern Benchmark Benchmark Mode Score Error Units testsequential avgt ± ns/op testrandompage avgt ± ns/op testdependentrandompage avgt ± ns/op testrandomheap avgt ± ns/op testdependentrandomheap avgt ± ns/op ~90 ns/op

34 Then ADD 40+ ns/op for NUMA access on a server!!!!

35 Data Dependent Loads aka Pointer Chasing!!!

36 Performance 101

37 Performance Memory is transported in Cachelines

38 Performance Memory is transported in Cachelines 2. Memory is managed in OS Pages

39 Performance Memory is transported in Cachelines 2. Memory is managed in OS Pages 3. Memory is pre-fetched on predictable access patterns

40 Runtime Optimisation

41 Runtime JIT 1. Profile guided optimisations

42 Runtime JIT 1. Profile guided optimisations 2. Bets can be taken and later revoked

43 Branches void foo() { // code if (condition) { // code } } // code

44 Branches void foo() { // code if (condition) { // code Block A } } // code

45 Branches void foo() { // code if (condition) { // code Block A Block B } } // code

46 Branches void foo() { // code if (condition) { // code Block A Block B } } // code Block C

47 Branches void foo() { // code } if (condition) { // code } // code Block A Block B Block C Block A Block C

48 Branches void foo() { // code } if (condition) { // code } // code Block A Block B Block C Block A Block C Block B

49 Subtle Branches int result = (i > 7)? a : b;

50 Subtle Branches int result = (i > 7)? a : b; CMOV vs Branch Prediction?

51 Method/Function Inlining void foo() { // code bar(); } // code

52 Method/Function Inlining void foo() { // code bar(); Block A } // code

53 Method/Function Inlining void foo() { // code bar(); Block A } // code bar()

54 Method/Function Inlining void foo() { // code bar(); Block A } // code bar()

55 Method/Function Inlining void foo() { // code bar(); Block A } // code Block B bar()

56 Method/Function Inlining void foo() { // code bar(); Block A Block A } // code Block B bar()

57 Method/Function Inlining void foo() { // code } bar(); // code Block A Block B Block A bar() bar()

58 Method/Function Inlining void foo() { // code } bar(); // code Block A Block B Block A bar() Block B bar()

59 Method/Function Inlining void foo() { // code bar(); i-cache & code bloat? } // code

60 Method/Function Inlining Inlining is THE optimisation. - Cliff Click

61 Bounds Checking void foo(int[] array, int length) { // code for (int i = 0; i < length; i++) { bar(array[i]); } } // code

62 Bounds Checking void foo(int[] array) { // code for (int i = 0; i < array.length; i++) { bar(array[i]); } } // code

63 Subtype Polymorphism void draw(shape[] shapes) { for (int i = 0; i < shapes.length; i++) { shapes[i].draw(); } } void bar(shape shape) { bar(shape.isvisible()); }

64 Subtype Polymorphism void draw(shape[] shapes) { for (int i = 0; i < shapes.length; i++) { shapes[i].draw(); } } void bar(shape shape) { bar(shape.isvisible()); } Class Hierarchy Analysis & Inline Caching

65 Runtime JIT 1. Profile guided optimisations 2. Bets can be taken and later revoked

66 Garbage Collection

67 Generational Garbage Collection Only the good die young. - Billy Joel

68 TLAB TLAB Generational Garbage Collection Young/New Generation Eden Survivor 0 Survivor 1 Virtual Old Generation Tenured Virtual

69 Modern Hardware (Intel Sandy Bridge EP) C 1 C n Registers/Buffers <1ns C 1 C n L1 L1 ~4 cycles ~1ns L1 L1 L2 L2 ~12 cycles ~3ns L2 L L3 ~40 cycles ~15ns ~60 cycles ~20ns (dirty hit) L3 PCI-e 3 MC QPI QPI MC PCI-e 3 QPI ~40ns DRAM DRAM 40X IO DRAM DRAM ~65ns DRAM DRAM 40X IO DRAM DRAM * Assumption: 3GHz Processor

70 Broadwell EX 24 cores & 60MB L3 Cache

71 TLAB TLAB Thread Local Allocation Buffers Young/New Generation Eden

72 TLAB TLAB Thread Local Allocation Buffers Young/New Generation Eden Affords locality of reference Avoid false sharing Can have NUMA aware allocation

73 TLAB TLAB Object Survival Young/New Generation Eden Survivor 0 Survivor 1 Virtual

74 TLAB TLAB Object Survival Young/New Generation Eden Survivor 0 Survivor 1 Virtual Aging Policies Compacting Copy NUMA Interleave Fast Parallel Scavenging Only the survivors require work

75 TLAB TLAB Object Promotion Young/New Generation Eden Survivor 0 Survivor 1 Virtual Old Generation Tenured Virtual

76 TLAB TLAB Object Promotion Young/New Generation Eden Survivor 0 Survivor 1 Virtual Old Generation Tenured Virtual Concurrent Collection String Deduplication

77 Compacting Collections

78 Compacting Collections Depth first copy

79 Compacting Collections

80 Compacting Collections OS Pages and cache lines?

81 G1 Concurrent Compaction E E E Eden O S O S S O O S O H Survivor Old Humongous H O O E Unused O E O O O O H

82 Azul Zing C4 True Concurrent Compacting Collector

83 Where next for GC?

84 Object Inlining/Aggregation

85 GC vs Manual Memory Management Not easy to pick clear winner

86 GC vs Manual Memory Management Not easy to pick clear winner Managed GC GC Implementation Card Marking Read/Write Barriers Object Headers Background Overhead in CPU and Memory

87 GC vs Manual Memory Management Not easy to pick clear winner Managed GC GC Implementation Card Marking Read/Write Barriers Object Headers Background Overhead in CPU and Memory Native Malloc Implementation Arena/pool contention Bin Wastage Fragmentation Debugging Effort Inter-thread costs

88 Algorithms & Design

89 What is most important to performance?

90 Avoiding cache misses Strength Reduction Avoiding duplicate work Amortising expensive operations Mechanical Sympathy Choice of Data Structures Choice of Algorithms API Design Overall Design

91 In a large codebase it is really difficult to do everything well

92 It also takes some uncommon disciplines such as: profiling, telemetry, modelling

93 If I had more time, I would have written a shorter letter. - Blaise Pascal

94 The story of Aeron

95 Aeron is an interesting lesson in time to performance

96 Lots of others exists such at the C# Roslyn compiler

97 Time spent on Mechanical Sympathy vs Debugging Pointers???

98 Immutable Data & Concurrency

99 Functional Programming

100 In Closing

101 What does the future hold?

102 Remember Assembly vs Compiled Languages

103 What about the issues of footprint, startup time, GC pauses, etc.???

104

105

106

107 Questions? Blog: Any intelligent fool can make things bigger, more complex, and more violent. It takes a touch of genius, and a lot of courage, to move in the opposite direction. - Albert Einstein