A Skiplist-based Concurrent Priority Queue with Minimal Memory Contention Jonatan Lindén and Bengt Jonsson Uppsala University, Sweden December 18, 2013 Jonatan Lindén 1
Contributions Motivation: Improve performance of concurrent Discrete Event Simulator. Outcome: New lock-free skiplist-based priority queue. New representation for logically deleted nodes. Minimizes the contention. Improved performance over existing algorithms by 30 80% on multiprocessors. Linearizable. Jonatan Lindén 2
Outline Contributions Background Priority queue Skiplist Standard solution to increase concurrency The problem: contention Our algorithm Correctness Evaluation Jonatan Lindén 3
Priority Queues Priority Queue - A set of (key,value) pairs with two operations: Insert(key, value) Applications: Discrete Event Simulation, Numerical algorithms. Jonatan Lindén 4
Priority Queues Priority Queue - A set of (key,value) pairs with two operations: Insert(key, value) Applications: Discrete Event Simulation, Numerical algorithms. Implementations: Traditionally, implemented on top of heaps or tree data structures. Skiplists [Pugh:1990] have been used for several parallel implementations. Jonatan Lindén 4
Skiplist layered linked list lowest-level list defines logical state, ordered higher-level lists are shortcuts probabilistic guarantee of logarithmic search time Insert(4) H 1 2 5 7 T Jonatan Lindén 5
Skiplist layered linked list lowest-level list defines logical state, ordered higher-level lists are shortcuts probabilistic guarantee of logarithmic search time Insert(4) Peek H 1 2 5 7 T Jonatan Lindén 5
Skiplist layered linked list lowest-level list defines logical state, ordered higher-level lists are shortcuts probabilistic guarantee of logarithmic search time Insert(4) H 1 2 5 7 T Jonatan Lindén 5
Skiplist layered linked list lowest-level list defines logical state, ordered higher-level lists are shortcuts probabilistic guarantee of logarithmic search time Insert(4) H 1 2 5 7 T Jonatan Lindén 5
Skiplist layered linked list lowest-level list defines logical state, ordered higher-level lists are shortcuts probabilistic guarantee of logarithmic search time Insert(4) H 1 2 5 7 T Jonatan Lindén 5
Skiplist layered linked list lowest-level list defines logical state, ordered higher-level lists are shortcuts probabilistic guarantee of logarithmic search time Insert(4) H 1 2 5 7 T Jonatan Lindén 5
Skiplist layered linked list lowest-level list defines logical state, ordered higher-level lists are shortcuts probabilistic guarantee of logarithmic search time Insert(4) H 1 2 5 7 T Jonatan Lindén 5
Skiplist layered linked list lowest-level list defines logical state, ordered higher-level lists are shortcuts probabilistic guarantee of logarithmic search time Insert(4) H 1 2 5 7 T Jonatan Lindén 5
Skiplist layered linked list lowest-level list defines logical state, ordered higher-level lists are shortcuts probabilistic guarantee of logarithmic search time Insert(4) H 1 2 5 7 T Jonatan Lindén 5
Skiplist layered linked list lowest-level list defines logical state, ordered higher-level lists are shortcuts probabilistic guarantee of logarithmic search time Insert(4) H 1 2 4 5 7 T Jonatan Lindén 5
Skiplist layered linked list lowest-level list defines logical state, ordered higher-level lists are shortcuts probabilistic guarantee of logarithmic search time Smallest element at the beginning of the lowest-level list. DeleteMin entry point H 1 2 4 5 7 T Jonatan Lindén 5
Concurrent skiplist-based priority queues Skiplists are easy to make concurrent Skiplists scale well when concurrent threads access different parts of the structure. Jonatan Lindén 6
Concurrent skiplist-based priority queues Skiplists are easy to make concurrent Skiplists scale well when concurrent threads access different parts of the structure. Jonatan Lindén 6
Concurrent skiplist-based priority queues Skiplists are easy to make concurrent Skiplists scale well when concurrent threads access different parts of the structure. Bottleneck: concurrent DeleteMin operations in priority queues try to remove the same element. Jonatan Lindén 6
Standard solution Standard solution to increase concurrency: logical deletion by setting a delete flag. physical deletion unlinks the node after the logical deletion has succeeded. Delete flag. H 1 2 4 5 7 T Jonatan Lindén 7
Standard solution Standard solution to increase concurrency: logical deletion by setting a delete flag. physical deletion unlinks the node after the logical deletion has succeeded. H 1 2 4 5 7 T Jonatan Lindén 7
Standard solution Standard solution to increase concurrency: logical deletion by setting a delete flag. physical deletion unlinks the node after the logical deletion has succeeded. H 1 2 4 5 7 T Jonatan Lindén 7
Standard solution Standard solution to increase concurrency: logical deletion by setting a delete flag. physical deletion unlinks the node after the logical deletion has succeeded. H 1 2 4 5 7 T Jonatan Lindén 7
Standard solution Standard solution to increase concurrency: logical deletion by setting a delete flag. physical deletion unlinks the node after the logical deletion has succeeded. H 1 2 4 5 7 T Jonatan Lindén 7
Standard solution Standard solution to increase concurrency: logical deletion by setting a delete flag. physical deletion unlinks the node after the logical deletion has succeeded. H 1 2 4 5 7 T Jonatan Lindén 7
Standard solution Standard solution to increase concurrency: logical deletion by setting a delete flag. physical deletion unlinks the node after the logical deletion has succeeded. H 1 2 4 5 7 T Jonatan Lindén 7
Standard solution Standard solution to increase concurrency: logical deletion by setting a delete flag. physical deletion unlinks the node after the logical deletion has succeeded. Lock-free: Perform all writes using Compare-and-Swap (CAS) Colocate delete flag together with each next pointer, e.g., in the lowest order bit [Harris 2001], to make physical deletion safe. H 1 2 4 5 7 T Jonatan Lindén 7
Contention Bottleneck: CAS in DeleteMin Several types of contention: (i) Multiple CASes compete H 1 2 4 5 7 T Jonatan Lindén 8
Contention Bottleneck: CAS in DeleteMin Several types of contention: (i) Multiple CASes compete modified by CAS H 1 2 4 5 7 T Jonatan Lindén 8
Contention Bottleneck: CAS in DeleteMin Several types of contention: (i) Multiple CASes compete (ii) Updates must be propagated to reads H 1 2 4 5 7 T Jonatan Lindén 8
Contention Bottleneck: CAS in DeleteMin Several types of contention: (i) Multiple CASes compete (ii) Updates must be propagated to reads Insert(6) serialize H 1 2 4 5 7 T Jonatan Lindén 8
Our algorithm Jonatan Lindén 9
Our solution Key idea: No physical deletion after logical deletion! Instead, delete nodes in batches. H 2 4 5 7 T Jonatan Lindén 10
Our solution Key idea: No physical deletion after logical deletion! Instead, delete nodes in batches. By updating the pointers in head node. H 2 4 5 7 T Jonatan Lindén 10
Our solution Key idea: No physical deletion after logical deletion! Instead, delete nodes in batches. By updating the pointers in head node. Requires that logically deleted nodes form a prefix. H 2 4 5 7 T Jonatan Lindén 10
Our solution Key idea: No physical deletion after logical deletion! Instead, delete nodes in batches. By updating the pointers in head node. Requires that logically deleted nodes form a prefix. Insert(1) H 2 4 5 7 T Jonatan Lindén 10
Our solution Key idea: No physical deletion after logical deletion! Instead, delete nodes in batches. By updating the pointers in head node. Requires that logically deleted nodes form a prefix. Insert(1) H 2 4 5 7 T Jonatan Lindén 10
Our solution Key idea: No physical deletion after logical deletion! Instead, delete nodes in batches. By updating the pointers in head node. Requires that logically deleted nodes form a prefix. Insert(1) H 1 2 4 5 7 T Jonatan Lindén 10
Our solution Key idea: No physical deletion after logical deletion! Instead, delete nodes in batches. By updating the pointers in head node. Requires that logically deleted nodes form a prefix. H 1 2 4 5 7 T Does not work! Jonatan Lindén 10
Our solution Key idea: No physical deletion after logical deletion! Instead, delete nodes in batches. By updating the pointers in head node. Requires that logically deleted nodes form a prefix. H 1 2 4 5 7 T should be inserted here Jonatan Lindén 10
Our solution To guarantee prefix property, Store delete flag together with the predecessor s next pointer. H 2 4 5 7 T Jonatan Lindén 11
Our solution To guarantee prefix property, Store delete flag together with the predecessor s next pointer. Insert(1) H 2 4 5 7 T Jonatan Lindén 11
Our solution To guarantee prefix property, Store delete flag together with the predecessor s next pointer. Insert(1) H 2 4 5 7 T Jonatan Lindén 11
Our solution To guarantee prefix property, Store delete flag together with the predecessor s next pointer. Insert(1) H 2 4 1 5 7 T Jonatan Lindén 11
Our solution To guarantee prefix property, Store delete flag together with the predecessor s next pointer. Insert(1) H 2 4 1 5 7 T Still a prefix! Jonatan Lindén 11
Our solution Resolving conflicts between Insert and DeleteMin. Insert(1) H 2 4 5 7 T Jonatan Lindén 12
Our solution Resolving conflicts between Insert and DeleteMin. Insert(1) H 2 4 5 7 T 1 Jonatan Lindén 12
Our solution Resolving conflicts between Insert and DeleteMin. Insert(1) H 2 4 5 7 T 1 Jonatan Lindén 12
Our solution Resolving conflicts between Insert and DeleteMin. Insert(1) H 2 4 5 7 T 1 Jonatan Lindén 12
Our solution Resolving conflicts between Insert and DeleteMin. Insert(1) H 2 4 5 7 T Jonatan Lindén 12
Our solution Resolving conflicts between Insert and DeleteMin. Insert(1) H 2 4 5 7 T 1 Jonatan Lindén 12
Our solution Resolving conflicts between Insert and DeleteMin. H 2 4 5 7 T 1 Jonatan Lindén 12
Our solution Resolving conflicts between Insert and DeleteMin. H 2 4 5 7 T 1 Jonatan Lindén 12
Physical batch deletion Physical batch deletion: update pointers in the head Done by DeleteMin, when the prefix of deleted nodes exceeds a threshold H 2 4 1 5 7 T Jonatan Lindén 13
Physical batch deletion Physical batch deletion: update pointers in the head Done by DeleteMin, when the prefix of deleted nodes exceeds a threshold H 2 4 1 5 7 T Jonatan Lindén 13
Physical batch deletion Physical batch deletion: update pointers in the head Done by DeleteMin, when the prefix of deleted nodes exceeds a threshold H 2 4 1 5 7 T Jonatan Lindén 13
Correctness linearizable concurrent priority queue Correctness proof based on assertional reasoning in the paper Follows rather easily after establishing that the lowest level list consists of a deleted prefix followed by a sorted list which defines the logical state of the queue. We have also modeled the algorithm in SPIN and performed extensive state-space exploration Jonatan Lindén 14
Evaluation Jonatan Lindén 15
Comparison of maximal throughput Compared against other lock-free skiplist-based priority queues: Sundell & Tsigas (ST): Only a single logically deleted node allowed in the lowest-level list. Herlihy & Shavit (HS): Lock-free adaptation of [Lotan & Shavit]. Logically deleted nodes need not form a prefix. Jonatan Lindén 16
Comparison of maximal throughput Benchmark: 50% Insert, 50% DeleteMin, 4 socket Intel sandybridge machine. M operations/s 10 9 8 7 6 5 4 3 2 1 New HS ST 4 8 12 16 20 24 28 Number of threads 30 80% improvement in comparison to HS 1-8 cores: single socket (shared L3 cache) Jonatan Lindén 17
Evaluation Benchmark: DES workload, 4-socket AMD bulldozer machine. M operations/s 3 2 1 New HS ST 0 4 8 12 16 20 24 28 Number of threads 30 80% improvement in comparison to HS 1 2 cores: shared L2 cache Jonatan Lindén 18
More resources http://user.it.uu.se/~jonli208/priorityqueue BSD licensed implementation SPIN model extended technical report Performance bug in the version in the proceedings: in the Restructure algorithm which updates the head pointers please see the extended technical report. Jonatan Lindén 19
Conclusions a new linearizable, lock-free, skiplist-based priority queue algorithm. new representation of logical deletion. This reduces the number of CASes to critical shared memory locations 30 80 % performance improvement over existing such algorithms. Jonatan Lindén 20
Conclusions a new linearizable, lock-free, skiplist-based priority queue algorithm. new representation of logical deletion. This reduces the number of CASes to critical shared memory locations 30 80 % performance improvement over existing such algorithms. Thank you. Jonatan Lindén 20