Concurrency and High Performance Reloaded

Transcription

1 Concurrency and High Performance Reloaded

2 Disclaimer Any performance tuning advice provided in this presentation... will be wrong! 2

3 Me Work as independent (a.k.a. freelancer) performance tuning services benchmarking Java performance tuning course Nominated Sun Java Champion Other stuff 3

4 single-threaded, single-core

5 5

6 how did we get better performance?

7 concurrent programming is the norm

8

9 9

10 10

11 multi-core is a fact of life!

12 we need to deliver twice as much concurrency every 18 mounts

13 hardware components are notsharable

14 access to shared data must be serialized

15 databases offer access to shared data

16 serialization limits scalability

17 Maths to explain relationship between serialized execution and processor utilization 1 F + ( 1 - F ) N F -> 0 number of utilized CPU -> N F -> 1 number of utilized CPU -> 1 17 Amdahl s Law

18 serialization limits throughput

19 Maths explaining the relationship between locking and throughput barrier barrier λ =1 / μ μ = 10ms, λ = 100 tps μ = 100ms, λ = 10 tps 19 Little s Law

20 locking is pessimistic

21 getting better concurrency in the JVM

22 Java and system level locks

23 StringBuffer sb = new StringBuffer(); sb.append( a ); sb.append( b ); sb.append( c );... Lock Coarsening

24 StringBuffer sb = new StringBuffer(); sb.append( a ); sb.append( b ); sb.append( c );... Lock Coarsening

25 { } StringBuffer sb = new StringBuffer(); sb.append( a ); sb.append( b ); sb.append( c ); Lock Elision

26 CAS Lock Biased Locking

27 do these optimizations work?

28 Baseline A B C D E F A EliminateLocks B UseBiasedLocking (working) C UseBiasedLocking (not working) D EliminateLocks with UseBiasedLocking E DoEscapeAnalysis F EliminateLocks with UseBiasedLocking and DoEscapeAnalysis StringBuffer StringBuilder

29 techniques we can use

30 Atomics to reduce lock contention

31 private int counter = 0; Runnable mutator = new Runnable() { public void run() { long localcount = 0; while ( running) { counter++; counter--; localcount++; } addtototalcount( localcount); } };

32 Baseline Volatile Synchronized Lock Atomic

33 Lock striping

34 Thread lock lock lock { { { ConcurrentHashMap

35 HashMap (no sync) HashMap (sync) HashTable ConcurrentHashMap Blackboard

36 teaching threads to steal

37 Fork-Join

38 Dequeue

39 Units of work Work splitting

40

41

42

43

44

45 Degrees of Scalability

46 Lock free concurrency

47 Parallel reads, serialized writes

48 Reader/Writer lock with only readers will not scale beyond 100 cpus

49 large arrays for concurrent

50 arrays to hold all data

51 resize cannot block

52 fully concurrent lock-less hashmap

53 Things we need Large array to hold all data alternating array of key value pairs state machine for pair of words CAS to manage state transistion Tombstone to mark deleted words Box to mark a resize allows read access but prevents update No single point of contention 53

54 0/0 Initial Inserting K/V pair Already probed table, missed Found proper empty K/V slot Ready to claim slot for this Key

55 0/0 claim slot insert key K/0 naked key

56 0/0 insert key K/0 insert V K/V active initial set of value

57 0/0 insert key K/0 insert V K/V delete K/T deleted Tombstone marks delete, key remains

58 over-write V 0/0 insert key K/0 insert V K/V K/T re-insert delete deleted operations use same key slot

59 over-write V 0/0 insert key K/0 insert V K/V K/T re-insert delete 0/0 resize triggered new array created

60 over-write V 0/0 insert key K/0 insert V K/V K/T re-insert delete box K/[V] boxed V 0/0 boxing prevents further changes

61 over-write V 0/0 insert key K/0 insert V K/V K/T re-insert delete box K/[V] insert key 0/0 K/0 bare key claim key slot in new table

62 over-write V 0/0 insert key K/0 insert V K/V K/T re-insert delete box K/[V] insert 0/0 K/0 key copy K/V active copy V without box

63 over-write V 0/0 insert key K/0 insert V K/V K/T re-insert delete box K/[V] memory fence between arrays insert 0/0 K/0 key copy K/V active fence after writing new array and before copy done

64 over-write V 0/0 insert key K/0 insert V K/V K/T re-insert delete box K/[V] K/[T] copy done memory fence between arrays copy done insert 0/0 K/0 key copy K/V fence after writing new array and before copy done

65 over-write V 0/0 insert key K/0 copy stops partial insert insert V K/V K/T re-insert delete box K/[V] nothing to copy K/[T] copy done 0/0 memory fence between arrays

66 over-write V 0/0 insert key K/0 copy stops partial insert insert V K/V K/T re-insert delete box K/[V] K/[T] copy done memory fence between arrays insert key copy 0/0 K/0 K/V

67 HashMap (no sync) HashMap (sync) Hashtable ConcurrentHashMap NonBlockingHashMap Blackboard Reloaded

68 HashMap (no sync) HashMap (sync) Hashtable ConcurrentHashMap NonBlockingHashMap Blackboard Reloaded

69

70 scales linerarly up too 1000 CPUs

71 Fully concurrent lock-less FIFO?

72 Stripe on queues and randomly pick one

73 stripe ad-absurdum

74 insert searchs for null CAS down value

75 read searchs for value and CAS down null

76 too large read spin, too small inserts spin

77 resize is earier, promote when entire array is tombstoned

78 Questions?