High Performance Managed Languages. Martin Thompson

Transcription

1 High Performance Managed Languages Martin Thompson

2 Really, what s your preferred platform for building HFT applications?

3

4 Why would you build low-latency applications on a GC ed platform?

5

6 Some Context

7 Let s be clear A Managed Runtime is not always a good choice

8 Latency Arbitrage?

9

10 Are native languages faster?

11 Time? Skills & Resources?

12 CPU vs Memory Performance

13 Time to add 2 integers?

14 1 CPU Cycle : < 1ns

15 Average time per element to sum the values in an array of integers?

16 Access Pattern Benchmark Benchmark Score Error Units =========================================== sequential ± ns/op

17 Really??? Less than 1ns per operation?

18

19 What if the access pattern is different?

20 Access Pattern Benchmark Benchmark Score Error Units =========================================== sequential ± ns/op randompage ± ns/op dependentrandompage ± ns/op randomheap ± ns/op dependentrandomheap ± ns/op

21 Data Dependent Loads aka Pointer Chasing!!!

22 Runtime Optimisation

23 Runtime JIT 1. Profile guided optimisations

24 Runtime JIT 1. Profile guided optimisations 2. Bets can be taken and later revoked

25 Branches void foo() { // code if (condition) { // code } } // code

26 Branches void foo() { // code if (condition) { // code Block A } } // code

27 Branches void foo() { // code if (condition) { // code Block A Block B } } // code

28 Branches void foo() { // code if (condition) { // code Block A Block B } } // code Block C

29 Branches void foo() { // code } if (condition) { // code } // code Block A Block B Block C Block A Block C

30 Branches void foo() { // code } if (condition) { // code } // code Block A Block B Block C Block A Block C Block B

31 Subtle Branches int result = (i > 7)? a : b;

32 Subtle Branches int result = (i > 7)? a : b; CMOV vs Branch Prediction

33 Method/Function Inlining void foo() { // code bar(); } // code

34 Method/Function Inlining void foo() { // code bar(); Block A } // code

35 Method/Function Inlining void foo() { // code bar(); Block A } // code bar()

36 Method/Function Inlining void foo() { // code bar(); Block A } // code bar()

37 Method/Function Inlining void foo() { // code bar(); Block A } // code Block B bar()

38 Method/Function Inlining void foo() { // code bar(); Block A Block A } // code Block B bar()

39 Method/Function Inlining void foo() { // code } bar(); // code Block A Block B Block A bar() bar()

40 Method/Function Inlining void foo() { // code } bar(); // code Block A Block B Block A bar() Block B bar()

41 Method/Function Inlining void foo() { // code bar(); i-cache & code bloat? } // code

42 Method/Function Inlining Inlining is THE optimisation. - Cliff Click

43 Loops void foo(int[] array, int length) { for (int i = 0; i < length; i++) { bar(integer.bitcount(array[i])); } }

44 Loops void foo(int[] array, int length) { for (int i = 0; i < length; i += 4) { bar(integer.bitcount(array[i])); bar(integer.bitcount(array[i + 1])); bar(integer.bitcount(array[i + 2])); bar(integer.bitcount(array[i + 3])); } }

45 Loops void foo(int[] array, int length) { for (int i = 0; i < length; i++) { bar(integer.bitcount(array[i])); } } Intrinsics

46 Subtype Polymorphism void draw(shape[] shapes) { for (int i = 0; i < shapes.length; i++) { shapes[i].draw(); } }

47 Subtype Polymorphism void draw(shape[] shapes) { for (int i = 0; i < shapes.length; i++) { shapes[i].draw(); } } void bar(shape shape) { foo(shape.isvisible()); }

48 Subtype Polymorphism void draw(shape[] shapes) { for (int i = 0; i < shapes.length; i++) { shapes[i].draw(); } } void bar(shape shape) { foo(shape.isvisible()); } Class Hierarchy Analysis & Inline Caching

49 Garbage Collection

50 Generational Garbage Collection Only the good die young - Billy Joel

51 TLAB TLAB Generational Garbage Collection Young/New Generation Eden Survivor 0 Survivor 1 Virtual Old Generation Tenured Virtual

52 Modern Hardware (Intel Sandy Bridge) C 1 C n Registers/Buffers <1ns C 1 C n L1 L1 ~4 cycles ~1ns L1 L1 L2 L2 ~12 cycles ~3ns L2 L L3 ~45 cycles ~15ns ~75 cycles ~25ns (dirty hit) L3 PCI-e 3 MC QPI QPI MC PCI-e 3 QPI ~40ns DRAM DRAM 40X IO DRAM DRAM ~65ns DRAM DRAM 40X IO DRAM DRAM * Assumption: 3GHz Processor

53 Broadwell EX 24 cores & 60MB L3 Cache

54 TLAB TLAB Thread Local Allocation Buffers Young/New Generation Eden

55 TLAB TLAB Thread Local Allocation Buffers Young/New Generation Eden Affords locality of reference Avoid false sharing Can have NUMA aware allocation

56 TLAB TLAB Object Survival Young/New Generation Eden Survivor 0 Survivor 1 Virtual

57 TLAB TLAB Object Survival Young/New Generation Eden Survivor 0 Survivor 1 Virtual Aging Policies Compacting Copy NUMA Interleave Fast Parallel Scavenging Only the survivors require work

58 TLAB TLAB Object Promotion Young/New Generation Eden Survivor 0 Survivor 1 Virtual Old Generation Tenured Virtual

59 TLAB TLAB Object Promotion Young/New Generation Eden Survivor 0 Survivor 1 Virtual Old Generation Tenured Virtual Concurrent Collection String Deduplication

60 Compacting Collections

61 Compacting Collections Depth first copy

62 Compacting Collections

63 Compacting Collections OS Pages and cache lines?

64 Stop the World!

65 G1 Concurrent Compaction? E E E Eden O S O S S O O S O H Survivor Old Humongous H O O E Unused O E O O O O H

66 Azul Zing C4 True Concurrent Compacting Collector

67 GC vs Manual Memory Management Not easy to pick a clear winner

68 GC vs Manual Memory Management Not easy to pick a clear winner Managed GC GC Implementation Card Marking Read/Write Barriers Object Headers Background Overhead on CPU and Memory

69 GC vs Manual Memory Management Not easy to pick a clear winner Managed GC GC Implementation Card Marking Read/Write Barriers Object Headers Background Overhead on CPU and Memory Native Malloc Implementation Arena/pool contention Bin Wastage Fragmentation Debugging Effort Inter-thread costs

70 Where next for GC?

71 Object Inlining/Aggregation

72 Algorithms & Design

73 What s most the most important thing for achieving good performance?

74 Time

75 If I had more time, I would have written a shorter letter. - Blaise Pascal

76 Avoiding duplicate work Minimising cache misses Avoiding contention Strength reduction Amortising expensive operations Mechanical Sympathy Choice of algorithms & data structures API design

77 In a large codebase it is really difficult to do everything well

78 The story of Aeron

79 Aeron is an interesting lesson in time to performance

80 Lots of others exist such as the C# Roslyn compiler

81 What Really Matters? Models Codecs Algorithms Data Structures Mechanical Sympathy

82 GC Immutable Data & Concurrency

83 In Closing

84 What does the future hold?

85 Remember Assembly vs Compiled Languages?

86

87

88

89 Questions? Blog: Any intelligent fool can make things bigger, more complex, and more violent. It takes a touch of genius, and a lot of courage, to move in the opposite direction. - Albert Einstein