A Skiplist-based Concurrent Priority Queue with Minimal Memory Contention

Transcription

1 A Skiplist-based Concurrent Priority Queue with Minimal Memory Contention Jonatan Lindén and Bengt Jonsson Uppsala University, Sweden December 18, 2013 Jonatan Lindén 1

2 Contributions Motivation: Improve performance of concurrent Discrete Event Simulator. Outcome: New lock-free skiplist-based priority queue. New representation for logically deleted nodes. Minimizes the contention. Improved performance over existing algorithms by 30 80% on multiprocessors. Linearizable. Jonatan Lindén 2

3 Outline Contributions Background Priority queue Skiplist Standard solution to increase concurrency The problem: contention Our algorithm Correctness Evaluation Jonatan Lindén 3

4 Priority Queues Priority Queue - A set of (key,value) pairs with two operations: Insert(key, value) Applications: Discrete Event Simulation, Numerical algorithms. Jonatan Lindén 4

5 Priority Queues Priority Queue - A set of (key,value) pairs with two operations: Insert(key, value) Applications: Discrete Event Simulation, Numerical algorithms. Implementations: Traditionally, implemented on top of heaps or tree data structures. Skiplists [Pugh:1990] have been used for several parallel implementations. Jonatan Lindén 4

6 Skiplist layered linked list lowest-level list defines logical state, ordered higher-level lists are shortcuts probabilistic guarantee of logarithmic search time Insert(4) H T Jonatan Lindén 5

7 Skiplist layered linked list lowest-level list defines logical state, ordered higher-level lists are shortcuts probabilistic guarantee of logarithmic search time Insert(4) Peek H T Jonatan Lindén 5

8 Skiplist layered linked list lowest-level list defines logical state, ordered higher-level lists are shortcuts probabilistic guarantee of logarithmic search time Insert(4) H T Jonatan Lindén 5

9 Skiplist layered linked list lowest-level list defines logical state, ordered higher-level lists are shortcuts probabilistic guarantee of logarithmic search time Insert(4) H T Jonatan Lindén 5

10 Skiplist layered linked list lowest-level list defines logical state, ordered higher-level lists are shortcuts probabilistic guarantee of logarithmic search time Insert(4) H T Jonatan Lindén 5

11 Skiplist layered linked list lowest-level list defines logical state, ordered higher-level lists are shortcuts probabilistic guarantee of logarithmic search time Insert(4) H T Jonatan Lindén 5

12 Skiplist layered linked list lowest-level list defines logical state, ordered higher-level lists are shortcuts probabilistic guarantee of logarithmic search time Insert(4) H T Jonatan Lindén 5

13 Skiplist layered linked list lowest-level list defines logical state, ordered higher-level lists are shortcuts probabilistic guarantee of logarithmic search time Insert(4) H T Jonatan Lindén 5

14 Skiplist layered linked list lowest-level list defines logical state, ordered higher-level lists are shortcuts probabilistic guarantee of logarithmic search time Insert(4) H T Jonatan Lindén 5

15 Skiplist layered linked list lowest-level list defines logical state, ordered higher-level lists are shortcuts probabilistic guarantee of logarithmic search time Insert(4) H T Jonatan Lindén 5

16 Skiplist layered linked list lowest-level list defines logical state, ordered higher-level lists are shortcuts probabilistic guarantee of logarithmic search time Smallest element at the beginning of the lowest-level list. DeleteMin entry point H T Jonatan Lindén 5

17 Concurrent skiplist-based priority queues Skiplists are easy to make concurrent Skiplists scale well when concurrent threads access different parts of the structure. Jonatan Lindén 6

18 Concurrent skiplist-based priority queues Skiplists are easy to make concurrent Skiplists scale well when concurrent threads access different parts of the structure. Jonatan Lindén 6

19 Concurrent skiplist-based priority queues Skiplists are easy to make concurrent Skiplists scale well when concurrent threads access different parts of the structure. Bottleneck: concurrent DeleteMin operations in priority queues try to remove the same element. Jonatan Lindén 6

20 Standard solution Standard solution to increase concurrency: logical deletion by setting a delete flag. physical deletion unlinks the node after the logical deletion has succeeded. Delete flag. H T Jonatan Lindén 7

21 Standard solution Standard solution to increase concurrency: logical deletion by setting a delete flag. physical deletion unlinks the node after the logical deletion has succeeded. H T Jonatan Lindén 7

22 Standard solution Standard solution to increase concurrency: logical deletion by setting a delete flag. physical deletion unlinks the node after the logical deletion has succeeded. H T Jonatan Lindén 7

23 Standard solution Standard solution to increase concurrency: logical deletion by setting a delete flag. physical deletion unlinks the node after the logical deletion has succeeded. H T Jonatan Lindén 7

24 Standard solution Standard solution to increase concurrency: logical deletion by setting a delete flag. physical deletion unlinks the node after the logical deletion has succeeded. H T Jonatan Lindén 7

25 Standard solution Standard solution to increase concurrency: logical deletion by setting a delete flag. physical deletion unlinks the node after the logical deletion has succeeded. H T Jonatan Lindén 7

26 Standard solution Standard solution to increase concurrency: logical deletion by setting a delete flag. physical deletion unlinks the node after the logical deletion has succeeded. H T Jonatan Lindén 7

27 Standard solution Standard solution to increase concurrency: logical deletion by setting a delete flag. physical deletion unlinks the node after the logical deletion has succeeded. Lock-free: Perform all writes using Compare-and-Swap (CAS) Colocate delete flag together with each next pointer, e.g., in the lowest order bit [Harris 2001], to make physical deletion safe. H T Jonatan Lindén 7

28 Contention Bottleneck: CAS in DeleteMin Several types of contention: (i) Multiple CASes compete H T Jonatan Lindén 8

29 Contention Bottleneck: CAS in DeleteMin Several types of contention: (i) Multiple CASes compete modified by CAS H T Jonatan Lindén 8

30 Contention Bottleneck: CAS in DeleteMin Several types of contention: (i) Multiple CASes compete (ii) Updates must be propagated to reads H T Jonatan Lindén 8

31 Contention Bottleneck: CAS in DeleteMin Several types of contention: (i) Multiple CASes compete (ii) Updates must be propagated to reads Insert(6) serialize H T Jonatan Lindén 8

32 Our algorithm Jonatan Lindén 9

33 Our solution Key idea: No physical deletion after logical deletion! Instead, delete nodes in batches. H T Jonatan Lindén 10

34 Our solution Key idea: No physical deletion after logical deletion! Instead, delete nodes in batches. By updating the pointers in head node. H T Jonatan Lindén 10

35 Our solution Key idea: No physical deletion after logical deletion! Instead, delete nodes in batches. By updating the pointers in head node. Requires that logically deleted nodes form a prefix. H T Jonatan Lindén 10

36 Our solution Key idea: No physical deletion after logical deletion! Instead, delete nodes in batches. By updating the pointers in head node. Requires that logically deleted nodes form a prefix. Insert(1) H T Jonatan Lindén 10

37 Our solution Key idea: No physical deletion after logical deletion! Instead, delete nodes in batches. By updating the pointers in head node. Requires that logically deleted nodes form a prefix. Insert(1) H T Jonatan Lindén 10

38 Our solution Key idea: No physical deletion after logical deletion! Instead, delete nodes in batches. By updating the pointers in head node. Requires that logically deleted nodes form a prefix. Insert(1) H T Jonatan Lindén 10

39 Our solution Key idea: No physical deletion after logical deletion! Instead, delete nodes in batches. By updating the pointers in head node. Requires that logically deleted nodes form a prefix. H T Does not work! Jonatan Lindén 10

40 Our solution Key idea: No physical deletion after logical deletion! Instead, delete nodes in batches. By updating the pointers in head node. Requires that logically deleted nodes form a prefix. H T should be inserted here Jonatan Lindén 10

41 Our solution To guarantee prefix property, Store delete flag together with the predecessor s next pointer. H T Jonatan Lindén 11

42 Our solution To guarantee prefix property, Store delete flag together with the predecessor s next pointer. Insert(1) H T Jonatan Lindén 11

43 Our solution To guarantee prefix property, Store delete flag together with the predecessor s next pointer. Insert(1) H T Jonatan Lindén 11

44 Our solution To guarantee prefix property, Store delete flag together with the predecessor s next pointer. Insert(1) H T Jonatan Lindén 11

45 Our solution To guarantee prefix property, Store delete flag together with the predecessor s next pointer. Insert(1) H T Still a prefix! Jonatan Lindén 11

46 Our solution Resolving conflicts between Insert and DeleteMin. Insert(1) H T Jonatan Lindén 12

47 Our solution Resolving conflicts between Insert and DeleteMin. Insert(1) H T 1 Jonatan Lindén 12

48 Our solution Resolving conflicts between Insert and DeleteMin. Insert(1) H T 1 Jonatan Lindén 12

49 Our solution Resolving conflicts between Insert and DeleteMin. Insert(1) H T 1 Jonatan Lindén 12

50 Our solution Resolving conflicts between Insert and DeleteMin. Insert(1) H T Jonatan Lindén 12

51 Our solution Resolving conflicts between Insert and DeleteMin. Insert(1) H T 1 Jonatan Lindén 12

52 Our solution Resolving conflicts between Insert and DeleteMin. H T 1 Jonatan Lindén 12

53 Our solution Resolving conflicts between Insert and DeleteMin. H T 1 Jonatan Lindén 12

54 Physical batch deletion Physical batch deletion: update pointers in the head Done by DeleteMin, when the prefix of deleted nodes exceeds a threshold H T Jonatan Lindén 13

55 Physical batch deletion Physical batch deletion: update pointers in the head Done by DeleteMin, when the prefix of deleted nodes exceeds a threshold H T Jonatan Lindén 13

56 Physical batch deletion Physical batch deletion: update pointers in the head Done by DeleteMin, when the prefix of deleted nodes exceeds a threshold H T Jonatan Lindén 13

57 Correctness linearizable concurrent priority queue Correctness proof based on assertional reasoning in the paper Follows rather easily after establishing that the lowest level list consists of a deleted prefix followed by a sorted list which defines the logical state of the queue. We have also modeled the algorithm in SPIN and performed extensive state-space exploration Jonatan Lindén 14

58 Evaluation Jonatan Lindén 15

59 Comparison of maximal throughput Compared against other lock-free skiplist-based priority queues: Sundell & Tsigas (ST): Only a single logically deleted node allowed in the lowest-level list. Herlihy & Shavit (HS): Lock-free adaptation of [Lotan & Shavit]. Logically deleted nodes need not form a prefix. Jonatan Lindén 16

60 Comparison of maximal throughput Benchmark: 50% Insert, 50% DeleteMin, 4 socket Intel sandybridge machine. M operations/s New HS ST Number of threads 30 80% improvement in comparison to HS 1-8 cores: single socket (shared L3 cache) Jonatan Lindén 17

61 Evaluation Benchmark: DES workload, 4-socket AMD bulldozer machine. M operations/s New HS ST Number of threads 30 80% improvement in comparison to HS 1 2 cores: shared L2 cache Jonatan Lindén 18

62 More resources BSD licensed implementation SPIN model extended technical report Performance bug in the version in the proceedings: in the Restructure algorithm which updates the head pointers please see the extended technical report. Jonatan Lindén 19

63 Conclusions a new linearizable, lock-free, skiplist-based priority queue algorithm. new representation of logical deletion. This reduces the number of CASes to critical shared memory locations % performance improvement over existing such algorithms. Jonatan Lindén 20

64 Conclusions a new linearizable, lock-free, skiplist-based priority queue algorithm. new representation of logical deletion. This reduces the number of CASes to critical shared memory locations % performance improvement over existing such algorithms. Thank you. Jonatan Lindén 20