JCTools: Pit Of Performance Nitsan Copyright - TTNR Labs Limited 2018

Transcription

1 JCTools: Pit Of Performance Nitsan 1

2 You Look Familiar... 2

3 Why me? Main JCTools contributor Performance Eng. Find me on: GitHub : nitsanw Blog : psy-lob-saw.blogspot.com Twitter : nitsanw Also: Cape Town JUG Organizer 3

4 JCTools? Concurrent Queues: SPSC/MPSC/SPMC/MPMC Linked/Array/LinkedArray/Compound And MOAR!!! 4

5 What is this about? Optimizations as found in the code Design pressures/choices Novel algorithm: MPSC linked queues 5

6 From the codebase... Layout Unsafe Concurrency specialization Bits and bobs 6

7 Layout abstract class...l1pad<e> extends...coldfield<e> { long p,p1,p2,p3,p4,p5,p6,p7, p8,p9,p1,p11,p12,p13,p14; } abstract class...producerindexfields<e> extends...l1pad<e> { private volatile long producerindex; protected long producerlimit; } abstract class...l2pad<e> extends...producerindexfields<e> { long p,p1,p2,p3,p4,p5,p6,p7, p8,p9,p1,p11,p12,p13,p14; } 7

8 Layout abstract class...l1pad<e> extends...coldfield<e> { } long p,p1,p2,p3,p4,p5,p6,p7, p8,p9,p1,p11,p12,p13,p14; abstract class...producerindexfields<e> extends...l1pad<e> { private volatile long producerindex; protected long producerlimit; } abstract class...l2pad<e> extends...producerindexfields<e> { } long p,p1,p2,p3,p4,p5,p6,p7, p8,p9,p1,p11,p12,p13,p14; 8

9 Measure with/with out: With padding: ~95 ns/op Sans padding: ~5 ns/op *QueueBurst, 1 messages benchmark 9

10 Java Object Layout 8b aligned Object header (8/12/16b) Sub-classes fields after parent fields Field order within class can change 1

11 Let s use JOL!!! # Running 64-bit HotSpot VM. # Using compressed oop with 3-bit shift. # Using compressed klass with 3-bit shift. # WARNING Compressed references base/shifts are guessed by the experiment! # WARNING Therefore, computed addresses are just guesses, and ARE NOT RELIABLE. # WARNING Make sure to attach Serviceability Agent to get the reliable addresses. # Objects are 8 bytes aligned. # Field sizes by type: 4, 1, 1, 2, 2, 4, 4, 8, 8 [bytes] # Array element sizes: 4, 1, 1, 2, 2, 4, 4, 8, 8 [bytes] Instantiated the sample instance via public org.jctools.queues.ffbuffer(int) org.jctools.queues.spscarrayqueue object internals: OFFSET SIZE TYPE DESCRIPTION VALUE 4 (object header) 1 (1 ) (1) 4 4 (object header) ( ) () 8 4 (object header) f8 ( ) ( ) 12 4 (alignment/padding gap) 16 8 long ConcurrentCircularArrayQueueLPad.p 24 8 long ConcurrentCircularArrayQueueLPad.p long ConcurrentCircularArrayQueueLPad.p2 4 8 long ConcurrentCircularArrayQueueLPad.p long ConcurrentCircularArrayQueueLPad.p long ConcurrentCircularArrayQueueLPad.p long ConcurrentCircularArrayQueueLPad.p long ConcurrentCircularArrayQueueLPad.p7 8 8 long ConcurrentCircularArrayQueueLPad.p long ConcurrentCircularArrayQueueLPad.p long ConcurrentCircularArrayQueueLPad.p long ConcurrentCircularArrayQueueLPad.p long ConcurrentCircularArrayQueueLPad.p long ConcurrentCircularArrayQueueLPad.p long ConcurrentCircularArrayQueueLPad.p long ConcurrentCircularArrayQueue.mask java.lang.object[] ConcurrentCircularArrayQueue.buffer [null, null, null, null] int SpscArrayQueueColdField.lookAheadStep long SpscArrayQueueL1Pad.p 16 8 long SpscArrayQueueL1Pad.p long SpscArrayQueueL1Pad.p long SpscArrayQueueL1Pad.p long SpscArrayQueueL1Pad.p long SpscArrayQueueL1Pad.p5 2 8 long SpscArrayQueueL1Pad.p long SpscArrayQueueL1Pad.p long SpscArrayQueueL1Pad.p long SpscArrayQueueL1Pad.p long SpscArrayQueueL1Pad.p long SpscArrayQueueL1Pad.p long SpscArrayQueueL1Pad.p long SpscArrayQueueL1Pad.p long SpscArrayQueueL1Pad.p long SpscArrayQueueProducerIndexFields.producerIndex 28 8 long SpscArrayQueueProducerIndexFields.producerLimit long SpscArrayQueueL2Pad.p long SpscArrayQueueL2Pad.p long SpscArrayQueueL2Pad.p long SpscArrayQueueL2Pad.p long SpscArrayQueueL2Pad.p long SpscArrayQueueL2Pad.p long SpscArrayQueueL2Pad.p long SpscArrayQueueL2Pad.p long SpscArrayQueueL2Pad.p long SpscArrayQueueL2Pad.p long SpscArrayQueueL2Pad.p long SpscArrayQueueL2Pad.p long SpscArrayQueueL2Pad.p long SpscArrayQueueL2Pad.p long SpscArrayQueueL2Pad.p long SpscArrayQueueConsumerIndexField.consumerIndex long SpscArrayQueueL3Pad.p long SpscArrayQueueL3Pad.p long SpscArrayQueueL3Pad.p long SpscArrayQueueL3Pad.p long SpscArrayQueueL3Pad.p long SpscArrayQueueL3Pad.p long SpscArrayQueueL3Pad.p long SpscArrayQueueL3Pad.p long SpscArrayQueueL3Pad.p long SpscArrayQueueL3Pad.p long SpscArrayQueueL3Pad.p long SpscArrayQueueL3Pad.p long SpscArrayQueueL3Pad.p long SpscArrayQueueL3Pad.p long SpscArrayQueueL3Pad.p14 Instance size: 536 bytes 11

12 Let s use JOL!!! # Running 64-bit HotSpot VM. # Using compressed oop with 3-bit shift. # Using compressed klass with 3-bit shift. # Objects are 8 bytes aligned. # Field sizes by type: 4, 1, 1, 2, 2, 4, 4, 8, 8 [bytes] # Array element sizes: 4, 1, 1, 2, 2, 4, 4, 8, 8 [bytes] 12

13 Let s use JOL!!! org.jctools.queues.spscarrayqueue object internals: OFFSET SIZE TYPE DESCRIPTION 12 (object header) 12 4 (alignment/padding gap) long ConcurrentCircularArrayQueueLPad.p [... p14] long ConcurrentCircularArrayQueue.mask java.lang.object[] ConcurrentCircularArrayQueue.buffer int SpscArrayQueueColdField.lookAheadStep long SpscArrayQueueL1Pad.p [... p14] long SpscArrayQueueProducerIndexFields.producerIndex 28 8 long SpscArrayQueueProducerIndexFields.producerLimit long SpscArrayQueueL2Pad.p [... p14] 48 8 long SpscArrayQueueConsumerIndexField.consumerIndex long SpscArrayQueueL3Pad.p [... p14] Instance size: 536 bytes 13

14 Let s use JOL!!! org.jctools.queues.spscarrayqueue object internals: OFFSET SIZE TYPE DESCRIPTION (object header) (alignment/padding gap) 12 long ConcurrentCircularArrayQueueLPad.p [... p14] 8 long ConcurrentCircularArrayQueue.mask 4 java.lang.object[] ConcurrentCircularArrayQueue.buffer 4 int SpscArrayQueueColdField.lookAheadStep 12 long SpscArrayQueueL1Pad.p [... p14] 8 long SpscArrayQueueProducerIndexFields.producerIndex 8 long SpscArrayQueueProducerIndexFields.producerLimit 12 long SpscArrayQueueL2Pad.p [... p14] 8 long SpscArrayQueueConsumerIndexField.consumerIndex 12 long SpscArrayQueueL3Pad.p [... p14] Instance size: 536 bytes 14

15 Let s use JOL!!! org.jctools.queues.spscarrayqueue object internals: OFFSET SIZE TYPE DESCRIPTION 12 (object header) 12 4 (alignment/padding gap) long ConcurrentCircularArrayQueue.mask java.lang.object[] ConcurrentCircularArrayQueue.buffer int SpscArrayQueueColdField.lookAheadStep long ConcurrentQueueLPad.p [... p14] long SpscArrayQueueProducerIndexFields.producerIndex long SpscArrayQueueProducerIndexFields.producerLimit 12 8 long SpscArrayQueueL1Pad.p [... p14] 12 long SpscArrayQueueL2Pad.p [... p14] long SpscArrayQueueConsumerIndexField.consumerIndex long SpscArrayQueueL3Pad.p [... p14] Instance size: 536 bytes 15

16 Let s use JOL!!! org.jctools.queues.spscarrayqueue object internals: OFFSET SIZE TYPE DESCRIPTION 12 (object header) 12 4 (alignment/padding gap) long ConcurrentCircularArrayQueueLPad.p [... p14] long ConcurrentCircularArrayQueue.mask java.lang.object[] ConcurrentCircularArrayQueue.buffer int SpscArrayQueueColdField.lookAheadStep long SpscArrayQueueL1Pad.p [... p14] long SpscArrayQueueProducerIndexFields.producerIndex 28 8 long SpscArrayQueueProducerIndexFields.producerLimit long SpscArrayQueueL2Pad.p [... p14] 48 8 long SpscArrayQueueConsumerIndexField.consumerIndex long SpscArrayQueueL3Pad.p [... p14] Instance size: 536 bytes 16

17 Why bother with layout? False sharing Minimize load misses Offheap? Atomicity Alignment requirements (line/page/sector) Different method (see Aeron) 17

18 Sharing? header header pindex cindex mask buffer 18

19 False Sharing header header pindex cindex Mask buffer 19

20 False Sharing header header pindex cindex Mask buffer 2

21 Pad to resolve...??? header header pad12 pad13 pad14 pad15 pad16 pad17 pad2 pad21 pad22 pad23 pad24 pad25 pad26 pindex pmask pbuffer pad1 pad11 pad12 pad13 pad14 pad15 pad16 pad17 pad2 pad21 pad22 pad23 pad24 pad25 pad26 pad27 cindex cmask cbuffer pad1 pad11 21

22 Alternatives To Method? Rely on field (field/class level, -XX:-Restrict) Go Offheap 22

23 What would be // state alignment preference Class X { long field1; // still type aligned byte[12] pad; // allocate presized array inline long field2; } 23

24 Unsafe 24

25 Looks safe enough... abstract class...producerindexfields<e> extends SpscArrayQueueL1Pad<E> { final static long P_INDEX_OFFSET = fieldoffset(...fields.class, "producerindex"); private volatile long producerindex; protected long producerlimit; long lvproducerindex() { return producerindex; } long lpproducerindex() { return } UNSAFE.getLong(this, P_INDEX_OFFSET); void soproducerindex(long newvalue) { } } UNSAFE.putOrderedLong(this, P_INDEX_OFFSET, newvalue); 25

26 Why do we need Unsafe? Better than a poke in the eye with a sharp stick? Performance? Compatibility????!??!?!?!? 26

27 Unsafe for Compatibility? Just Works! (JDK6,7,8,11) Shouldn t we use VarHandle? 27

28 Unsafe vs. A*FU (pre-8u11) With Unsafe : ~95 ns/op With A*FU@7 : ~135 ns/op * Tested with JDK 7u8 as an example 28

29 Unsafe vs. A*FU (post 8u11/11) With Unsafe : ~95 ns/op With A*FU@8 : ~15 ns/op With A*FU@11 : ~115 ns/op See: 29

30 Unsafe vs VarHandles (>=11) With Unsafe : ~95 ns/op With VarHandle : ~115 ns/op 3

31 Unsafe for performance? With : ~95 ns/op With : ~13 ns/op With : ~15 ns/op With : ~115 ns/op With : ~115 ns/op 31

32 Unsafe vs Alternatives Pre-JDK11? Volatile AtomicBoolean/Integer/Long/*Array Atomic*FieldUpdater JDK11+? Should move to VarHandle? Multi-version jars Offheap? GO UNSAFE!!! 32

33 Concurrency Specialization 33

34 Generic vs. Specialized JDK? MPMC Lock based (ArrayBlockingQ/LinkedBlockingQ) Lock-less (ConcurrentLinkedQ) Allocation? JCTools? MPMC/SPMC/MPSC/SPSC Array backed/linked/both! Lock-less/lock-free/wait-free Zero allocation options 34

35 Generic vs. Specialized Load: plain, opaque, volatile Store: plain, ordered, volatile compareandswap getandadd 35

36 Single Writer Load: plain/volatile Store: ordered 36

37 Multi-Writer Load: plain/volatile Store: getandadd/compareandswap 37

38 Generic vs. Specialized SPSC 95 ns/op MPSC 65 ns/op MPMC 55 ns/op CLQ 138 ns/op ABQ 36 ns/op 38

39 Lock less/lock free/wait free SPSC 95 ns/op WF MPSC 5/65 ns/op LF/LL MPMC 5/55 ns/op LF/LL CLQ 138 ns/op LL ABQ 36 ns/op Locks... 39

40 Allocations CLQ 24b per op(1 messages) LBQ ~3b ABQ? 4

41 Allocations CLQ 24b per op(1 messages) LBQ ~3b ABQ ~15b!!! JCTools array backed - b 41

42 Helping the Compiler 42

43 Const vs Final private final long size = <some power of 2>; long offset = REF_ARRAY_BASE + ((consumerindex % size) * REF_ELEMENT_SCALE); long offset = REF_ARRAY_BASE + ((consumerindex & (size-1)) * REF_ELEMENT_SCALE); 43

44 Const vs Final long offset = REF_ARRAY_BASE + ((consumerindex & (size-1)) * REF_ELEMENT_SCALE); long offset = REF_ARRAY_BASE + << REF_ELEMENT_SHIFT); ((consumerindex & mask) 44

45 Const vs Final long offset = REF_ARRAY_BASE + ((consumerindex & mask) << REF_ELEMENT_SHIFT); long offset = calcelementoffset(consumerindex, mask); 45

46 Loads Across an Volatile Load public E poll() { long consumerindex = lpconsumerindex(); buffer buffer E[] = this. ; long offset = calcelementoffset(consumerindex, mask); E e = (E) UNSAFE.getObjectVolatile( if (null == e) return null; // EMPTY UNSAFE.putOrderedObject( } buffer, buffer, offset); offset, null); soconsumerindex(consumerindex + 1); return e; 46

47 A Novel Algorithm 47

48 Typical Offerings Array backed Pre-allocated, fixed sized Linked Allocate cell per element No pre-allocation Bounded/Unbounded 48

49 Actor Use Case Lots of queues Some full & busy Most empty MPSC 49

50 Linked Array Queues? Middle Ground? Allocate a multi-element cell/chunk Stay in cell/chunk? no allocation Resizing on the fly 5

51 Challenges? Lock less algo? Linking new array Sizing 51

52 Hanging on a Bit Steal producerindex parity bit Fixup code to match CAS parity to block ALL other producers Let them spin 52

53 JUMP indicator Notify consumer needs to jump Use extra cell in array as next pointer 53

54 Performance? Slightly slower than array backed MPSC Very similar throughput Allocates as needed: Stay in chunk no allocation Growable allocate until capacity Chunked bounded, fixed chunks Unbounded blow yer heap, in fixed increments! 54

55 In Closing... 55

56 Thanks! 56