Conflict-free Replicated Data Types (CRDTs)

Transcription

1 Conflict-free Replicated Data Types (CRDTs) for collaborative environments Marc Shapiro, INRIA & LIP6 Nuno Preguiça, U. Nova de Lisboa Carlos Baquero, U. Minho Marek Zawirski, INRIA & UPMC

2 Conflict-free objects for largescale distribution Shared Mutable data Read replicate Updates? Novel, principled approach: Conflict-free objects Large, dynamic graph Incremental, parallel, asynchronous: - updates - processing Can we design useful object types without any synchronisation whatsoever? Can we build practical systems from such objects? 2

3 Conflict-free objects for largescale distribution Shared Mutable data Read replicate Updates? Novel, principled approach: Conflict-free objects Large, dynamic graph Incremental, parallel, asynchronous: - updates - processing Can we design useful object types without any synchronisation whatsoever? Can we build practical systems from such objects? 2

4 Conflict-free objects for largescale distribution Shared Mutable data Read replicate Updates? Novel, principled approach: Conflict-free objects Large, dynamic graph Incremental, parallel, asynchronous: - updates - processing Can we design useful object types without any synchronisation whatsoever? Can we build practical systems from such objects? 2

5 Conflict-free objects for largescale distribution Shared Mutable data Read replicate Updates? Novel, principled approach: Conflict-free objects Large, dynamic graph Incremental, parallel, asynchronous: - updates - processing Can we design useful object types without any synchronisation whatsoever? Can we build practical systems from such objects? 2

6 Conflict-free objects for largescale distribution Shared Mutable data Read replicate Updates? Novel, principled approach: Conflict-free objects Large, dynamic graph Incremental, parallel, asynchronous: - updates - processing Can we design useful object types without any synchronisation whatsoever? Can we build practical systems from such objects? 2

7 Replication for beginners

8 Replicated data Share data Replicate at many locations Performance: local reads Availability: immune from network failure Fault-tolerance: replicate computation Scalability: load balancing Updates Push to all replicas Conflicts: Consistency? Fault tolerance and parallelism too? Conflict!! 4

9 Strong consistency Preclude conflicts All replicas execute updates in same total order Any deterministic object Simultaneous N- way agreement Consensus Serialisation bottleneck Tolerates < n/2 faults Very general Correct Doesn't scale 5

10 Strong consistency Preclude conflicts All replicas execute updates in same total order Any deterministic object 1 Simultaneous N- way agreement Consensus Serialisation bottleneck Tolerates < n/2 faults Very general Correct Doesn't scale 5

11 Strong consistency Preclude conflicts All replicas execute updates in same total order Any deterministic object 12 Simultaneous N- way agreement Consensus Serialisation bottleneck Tolerates < n/2 faults Very general Correct Doesn't scale 5

12 Strong consistency Preclude conflicts All replicas execute updates in same total order Any deterministic object 123 Simultaneous N- way agreement Consensus Serialisation bottleneck Tolerates < n/2 faults Very general Correct Doesn't scale 5

13 Strong consistency Preclude conflicts All replicas execute updates in same total order Any deterministic object 1234 Simultaneous N- way agreement Consensus Serialisation bottleneck Tolerates < n/2 faults Very general Correct Doesn't scale 5

14 Strong consistency Preclude conflicts All replicas execute updates in same total order Any deterministic object Simultaneous N- way agreement Consensus Serialisation bottleneck Tolerates < n/2 faults Very general Correct Doesn't scale 5

15 Strong consistency Preclude conflicts All replicas execute updates in same total order Any deterministic object Simultaneous N- way agreement Consensus Serialisation bottleneck Tolerates < n/2 faults Very general Correct Doesn't scale 5

16 Eventual Consistency Update local + propagate No foreground synch Expose tentative state Eventual, reliable delivery On conflict Arbitrate Roll back Availability ++ Parallelism++ Latency -- Complexity ++ Consensus moved to background 6

17 Eventual Consistency Update local + propagate No foreground synch Expose tentative state Eventual, reliable delivery On conflict Arbitrate Roll back Availability ++ Parallelism++ Latency -- Complexity ++ Consensus moved to background 6

18 Eventual Consistency Update local + propagate No foreground synch Expose tentative state Eventual, reliable delivery On conflict Arbitrate Roll back Availability ++ Parallelism++ Latency -- Complexity ++ Consensus moved to background 6

19 Eventual Consistency Update local + propagate No foreground synch Expose tentative state Eventual, reliable delivery On conflict Arbitrate Roll back Availability ++ Parallelism++ Latency -- Complexity ++ Consensus moved to background 6

20 Eventual Consistency Update local + propagate No foreground synch Expose tentative state Eventual, reliable delivery On conflict Arbitrate Roll back Availability ++ Parallelism++ Latency -- Complexity ++ Consensus moved to background 6

21 Eventual Consistency Update local + propagate No foreground synch Expose tentative state Conflict! Eventual, reliable delivery On conflict Arbitrate Roll back Availability ++ Parallelism++ Latency -- Complexity ++ Consensus moved to background 6

22 Eventual Consistency Update local + propagate No foreground synch Expose tentative state Eventual, reliable delivery On conflict Arbitrate Roll back Availability ++ Parallelism++ Latency -- Complexity ++ Consensus moved to background 6

23 Eventual Consistency Update local + propagate No foreground synch Expose tentative state Eventual, reliable delivery On conflict Arbitrate Roll back Availability ++ Parallelism++ Latency -- Complexity ++ Consensus moved to background 6

24 Eventual Consistency Update local + propagate No foreground synch Expose tentative state Eventual, reliable delivery On conflict Arbitrate Roll back Availability ++ Parallelism++ Latency -- Complexity ++ Consensus moved to background 6

25 Strong Eventual Consistency Available, responsive More parallelism No conflicts No rollback Update local + propagate No synch Expose intermediate state Eventual, reliable delivery No conflict Deterministic outcome of concurrent updates No consensus: n-1 faults Not universal Solves the CAP problem 7

26 Strong Eventual Consistency Available, responsive More parallelism No conflicts No rollback Update local + propagate No synch Expose intermediate state Eventual, reliable delivery No conflict Deterministic outcome of concurrent updates No consensus: n-1 faults Not universal Solves the CAP problem 7

27 Strong Eventual Consistency Available, responsive More parallelism No conflicts No rollback Update local + propagate No synch Expose intermediate state Eventual, reliable delivery No conflict Deterministic outcome of concurrent updates No consensus: n-1 faults Not universal Solves the CAP problem 7

28 Strong Eventual Consistency Available, responsive More parallelism No conflicts No rollback Update local + propagate No synch Expose intermediate state Eventual, reliable delivery No conflict Deterministic outcome of concurrent updates No consensus: n-1 faults Not universal Solves the CAP problem 7

29 Strong Eventual Consistency Available, responsive More parallelism No conflicts No rollback Update local + propagate No synch Expose intermediate state Eventual, reliable delivery No conflict Deterministic outcome of concurrent updates No consensus: n-1 faults Not universal Solves the CAP problem 7

30 The challenge: What interesting objects can we design with no synchronisation whatsoever?

31 Portfolio of CRDTs Register Last-Writer Wins Multi-Value Set Grow-Only 2P Observed-Remove Map Set of Registers Counter Unlimited Non-negative Graphs Directed Monotonic DAG Edit graph Sequence Edit sequence 9

32 Set design alternatives Sequential specification: {true} add(e) {e S} {true} remove(e) {e S} linearisable: sequential order equivalent to real-time order Requires consensus {true} add(e) remove(e) {????} linearisable? add wins? remove wins? last writer wins? error state? 10

33 Observed-Remove Set s s1 s2 s3 {} {} {} Can never remove more tokens than exist Op order removed tokens have been previously added 11

34 Observed-Remove Set s s1 s2 s3 {} {} {} add(a) S {aα} Can never remove more tokens than exist Op order removed tokens have been previously added 11

35 Observed-Remove Set s {} add(a) s1 s2 s3 {} {} S {aα} add(a) S {aβ} Can never remove more tokens than exist Op order removed tokens have been previously added 11

36 Observed-Remove Set s s1 s2 s3 {} {} {} add(a) S {aα} add(a) S {aβ} {aβ} M {aβ} Can never remove more tokens than exist Op order removed tokens have been previously added 11

37 Observed-Remove Set s s1 s2 s3 {} {} {} add(a) S {aα} add(a) S {aβ} {aβ} M {aβ} {aα} M {aβ, aα} Can never remove more tokens than exist Op order removed tokens have been previously added 11

38 Observed-Remove Set s s1 s2 s3 {} {} {} add(a) S {aα} add(a) S {aβ} {aβ} M rmv (a) {aβ} S {aα} {aα} M {aβ, aα} Can never remove more tokens than exist Op order removed tokens have been previously added 11

39 Observed-Remove Set s s1 s2 s3 {} {} {} add(a) S {aα} add(a) S {aβ} {aβ} rmv (a) S {aα} Can never remove more tokens than {aα} {aα} exist Op order removed M M M {aβ} {aβ, aα} {aβ, aα} tokens have been previously added 11

40 Observed-Remove Set s s1 s2 s3 {} {} {} add(a) S {aα} add(a) S {aβ} {aβ} rmv (a) S {aβ, aα} M {aα} {aβ} Can never remove more tokens than {aα} {aα} exist Op order removed M M M {aβ} {aβ, aα} {aβ, aα} tokens have been previously added 11

41 Observed-Remove Set s s1 s2 s3 {} {} {} add(a) S {aα} add(a) S {aβ} {aβ} rmv (a) S {aβ, aα} M {aα} {aβ} Can never remove more tokens than {aα} {aα} exist Op order removed M M M {aβ} {aβ, aα} {aβ, aα} tokens have been previously added Payload: added, removed (element, unique-token) add(e) = A A {(e, α)} Remove: all unique elements observed remove(e) = R R { (e, ) A} lookup(e) = (e, ) A \ R merge (S, S') = (A A', R R') {true} add(e) remove(e) {e S} 11

42 OR-Set Set: solves Dynamo Shopping Cart anomaly Optimisations Just mark tombstones Garbage-collect tombstones Operation-based approach 12

43 Graph design alternatives Graph = (V, E) where E V! V Sequential specification: {v,v' V} addedge(v,v') { } { (v,v') E} removevertex(v) { } Concurrent: removevertex(v') addedge(v,v') linearisable? addedge wins? removevertex wins? etc. for our Web Search Engine application, removevertex wins Do not check precondition at add/remove 13

44 Graph Payload = OR-Set V, OR-Set E Updates add/remove to V, E addvertex(v), removevertex(v) addedge(v,v'), removeedge(v,v') Do not enforce invariant a priori lookupedge(v,v') = (v,v') E v V v' V removevertex(v') addedge(v,v') remove wins " 14

45 Graph Payload = OR-Set V, OR-Set E Updates add/remove to V, E addvertex(v), removevertex(v) addedge(v,v'), removeedge(v,v') Do not enforce invariant a priori lookupedge(v,v') = (v,v') E v V v' V removevertex(v') addedge(v,v') remove wins " 14

46 Graph Payload = OR-Set V, OR-Set E Updates add/remove to V, E addvertex(v), removevertex(v) addedge(v,v'), removeedge(v,v') Do not enforce invariant a priori lookupedge(v,v') = (v,v') E v V v' V removevertex(v') addedge(v,v') remove wins " 14

47 Graph Payload = OR-Set V, OR-Set E Updates add/remove to V, E addvertex(v), removevertex(v) addedge(v,v'), removeedge(v,v') Do not enforce invariant a priori lookupedge(v,v') = (v,v') E v V v' V removevertex(v') addedge(v,v') remove wins " 14

48 Graph Payload = OR-Set V, OR-Set E Updates add/remove to V, E addvertex(v), removevertex(v) addedge(v,v'), removeedge(v,v') Do not enforce invariant a priori lookupedge(v,v') = (v,v') E v V v' V removevertex(v') addedge(v,v') remove wins " 14

49 Deep (internal) view: DAG representing insert order Co-operative editing {x,z graphi x < z} add-betweeni (x, y, z) {y graphi x<y<z} add: between alreadyordered elements strong order takes preceden ce over weak begin and end sentinels Local constraint implies globally acyclic This spec is implemented directly by WOOT Clean Surface view: summarises total order ensure consistent ordering at all replicas II NR RI A α ββ γ γδ ε

50 Deep (internal) view: DAG representing insert order Co-operative editing {x,z graphi x < z} add-betweeni (x, y, z) {y graphi x<y<z} I δ add: between alreadyordered elements strong order takes preceden ce over weak begin and end sentinels Local constraint implies globally acyclic This spec is implemented directly by WOOT Clean Surface view: summarises total order ensure consistent ordering at all replicas II NR RI A α ββ γ γδ ε

51 Co-operative editing Deep (internal) view: DAG representing insert order ensure consistent ordering at all replicas {x,z graphi x < z} add-betweeni (x, y, z) {y graphi x<y<z} I δ add: between alreadyordered elements strong order takes preceden ce over weak A ε begin and end sentinels II NR RI A Local constraint implies globally acyclic This spec is implemented directly by WOOT Clean Surface view: summarises total order α ββ γ γδ ε

52 Co-operative editing Deep (internal) view: DAG representing insert order N β ensure consistent ordering at all replicas {x,z graphi x < z} add-betweeni (x, y, z) {y graphi x<y<z} I δ add: between alreadyordered elements strong order takes preceden ce over weak A ε begin and end sentinels II NR RI A Local constraint implies globally acyclic This spec is implemented directly by WOOT Clean Surface view: summarises total order α ββ γ γδ ε

53 Co-operative editing Deep (internal) view: DAG representing insert order N β ensure consistent ordering at all replicas {x,z graphi x < z} add-betweeni (x, y, z) {y graphi x<y<z} R γ I δ add: between alreadyordered elements strong order takes preceden ce over weak A ε begin and end sentinels II NR RI A Local constraint implies globally acyclic This spec is implemented directly by WOOT Clean Surface view: summarises total order α ββ γ γδ ε

54 Co-operative editing Deep (internal) view: DAG representing insert order I α N β ensure consistent ordering at all replicas {x,z graphi x < z} add-betweeni (x, y, z) {y graphi x<y<z} R γ I δ add: between alreadyordered elements strong order takes preceden ce over weak A ε begin and end sentinels II NR RI A Local constraint implies globally acyclic This spec is implemented directly by WOOT Clean Surface view: summarises total order α ββ γ γδ ε

55 Co-operative editing Deep (internal) view: DAG representing insert order I α N β ensure consistent ordering at all replicas {x,z graphi x < z} add-betweeni (x, y, z) {y graphi x<y<z} R γ I δ add: between alreadyordered elements strong order takes preceden ce over weak A ε begin and end sentinels II NR RI A Local constraint implies globally acyclic This spec is implemented directly by WOOT Clean Surface view: summarises total order α ββ γ γδ ε

56 Co-operative editing Deep (internal) view: DAG representing insert order I α N β ensure consistent ordering at all replicas {x,z graphi x < z} add-betweeni (x, y, z) {y graphi x<y<z} R γ I δ add: between alreadyordered elements strong order takes preceden ce over weak A ε begin and end sentinels II NR RI A Local constraint implies globally acyclic This spec is implemented directly by WOOT Clean Surface view: summarises total order α ββ γ γδ ε

57 Co-operative editing Deep (internal) view: DAG representing insert order I α N β ensure consistent ordering at all replicas {x,z graphi x < z} add-betweeni (x, y, z) {y graphi x<y<z} R γ I δ add: between alreadyordered elements strong order takes preceden ce over weak A ε begin and end sentinels II NR RI A Local constraint implies globally acyclic This spec is implemented directly by WOOT Clean Surface view: summarises total order α ββ γ γδ ε

58 Co-operative editing Deep (internal) view: DAG representing insert order I α N β ensure consistent ordering at all replicas {x,z graphi x < z} add-betweeni (x, y, z) {y graphi x<y<z} R γ I δ add: between alreadyordered elements strong order takes preceden ce over weak A ε begin and end sentinels II NR RI A Local constraint implies globally acyclic This spec is implemented directly by WOOT Clean Surface view: summarises total order α ββ γ γδ ε

59 Continuum I N Assign each element a unique real number position Real numbers not appropriate approximate by tree 16

60 Continuum I N A Assign each element a unique real number position Real numbers not appropriate approximate by tree 16

61 Continuum I N I A Assign each element a unique real number position Real numbers not appropriate approximate by tree 16

62 Continuum L I N R I A Assign each element a unique real number position Real numbers not appropriate approximate by tree 16

63 Treedoc binary tree 0 R 1 I 1 N I 0 A Compact, lowarity tree In the following slides, will Low arity: binary, quaternary = L I N R I Binary naming tree: compact, self-adjusting logarithmic: Logarithmic properties assuming tree add appends leaf non-destructive, IDs don t change remove: tombstone, IDs don't change 17

64 Treedoc binary tree 0 R 1 L 0 I 1 N I 0 A Compact, lowarity tree In the following slides, will Low arity: binary, quaternary = L I N R I Binary naming tree: compact, self-adjusting logarithmic: Logarithmic properties assuming tree add appends leaf non-destructive, IDs don t change remove: tombstone, IDs don't change 17

65 Treedoc binary tree Low arity: binary, quaternary Binary naming tree: compact, self-adjusting Logarithmic properties L 0 I 0 1 N R I 0 1 = L I N R I A logarithmic: assuming tree Compact, lowarity tree In the following slides, will add appends leaf non-destructive, IDs don t change remove: tombstone, IDs don't change 17

66 Treedoc binary tree Low arity: binary, quaternary Binary naming tree: compact, self-adjusting Logarithmic properties L 0 I 0 1 N R I 0 1 = L I N R I A logarithmic: assuming tree Compact, lowarity tree In the following slides, will add appends leaf non-destructive, IDs don t change remove: tombstone, IDs don't change 17

67 Treedoc binary tree Low arity: binary, quaternary Binary naming tree: compact, self-adjusting Logarithmic properties L 0 I 0 1 N R I 0 1 = L I N R I A logarithmic: assuming tree Compact, lowarity tree In the following slides, will add appends leaf non-destructive, IDs don t change remove: tombstone, IDs don't change 17

68 Layered Treedoc binary tree 18

69 Layered Treedoc sparse ary tree Site 34 Site 79 binary tree 18

70 Layered Treedoc sparse ary tree Site 34 Site 79 Site 22 binary tree 18

71 Layered Treedoc sparse ary tree Site 34 Site 79 Site 34 Site 22 binary tree 18

72 Layered Treedoc sparse ary tree Site 34 Site 79 Site 34 Site 22 Site 34 Site 66 Site 79 binary tree 18

73 Layered Treedoc sparse ary tree Site 34 Site 79 Site 34 Site 22 Site 34 Site 66 Site 79 binary tree Edit: Binary tree Concurrency: Sparse tree 18

74 The theory Two simple conditions for correctness without synchronisation

75 Query client s s1 s2 s3 Local at source replica Client's choice Example: Amazon shopping cart is replicated unspecified client, e.g., Web frontend One or more load-balancer, failures may direct client to different replicas 20

76 Query s s1 client S s1.q(a) s2 s3 Local at source replica Client's choice Example: Amazon shopping cart is replicated unspecified client, e.g., Web frontend One or more load-balancer, failures may direct client to different replicas 20

77 Query client s s1 s2.q(b) S s1.q(a) s2 S s3 Example: Amazon shopping cart is replicated unspecified client, e.g., Web frontend One or more load-balancer, failures may direct client to different replicas Local at source replica Client's choice 20

78 State-based replication client s s1 s2 s3 Local at source s1.u(a), s2.u(b), Compute Update local payload Convergence: Episodically: send si payload On delivery: merge payloads m merge two valid states produce valid state no historical info available Inefficient if payload is large 21

79 State-based replication client s s1 S s1.u(a) s2 s3 Local at source s1.u(a), s2.u(b), Compute Update local payload Convergence: Episodically: send si payload On delivery: merge payloads m merge two valid states produce valid state no historical info available Inefficient if payload is large 21

80 State-based replication client s s1 s2.u(b) S s1.u(a) s2 S s3 Local at source s1.u(a), s2.u(b), Compute Update local payload Convergence: Episodically: send si payload On delivery: merge payloads m merge two valid states produce valid state no historical info available Inefficient if payload is large 21

81 State-based replication s s1 s2.u(b) S s1.u(a) s2 S s3 Local at source s1.u(a), s2.u(b), Compute Update local payload Convergence: Episodically: send si payload On delivery: merge payloads m merge two valid states produce valid state no historical info available Inefficient if payload is large 21

82 State-based replication s s1 s2.u(b) S s1.u(a) s1 s2 S M s2.m(s1) s3 Local at source s1.u(a), s2.u(b), Compute Update local payload Convergence: Episodically: send si payload On delivery: merge payloads m merge two valid states produce valid state no historical info available Inefficient if payload is large 21

83 State-based replication s s1 s2.u(b) S s1.u(a) s1 s2 S M s2.m(s1) s2 s3 M s3.m(s2) Local at source s1.u(a), s2.u(b), Compute Update local payload Convergence: Episodically: send si payload On delivery: merge payloads m merge two valid states produce valid state no historical info available Inefficient if payload is large 21

84 State-based replication s s1 s2 s2.u(b) S S s1.u(a) s1 M s2.m(s1) s2 s1.m(s2) M s2 s3 M s3.m(s2) Local at source s1.u(a), s2.u(b), Compute Update local payload Convergence: Episodically: send si payload On delivery: merge payloads m merge two valid states produce valid state no historical info available Inefficient if payload is large 21

85 State-based: monotonic semilattice CRDT s s1 s2 s3 s1.u(a) S s2.u(b) S s1 M s2.m(s1) s2 M s3.m(s2) s1.m(s2) M s2 = Least Upper Bound LUB = merge If payload type forms a semi-lattice updates are increasing merge computes Least Upper Bound then replicas converge to LUB of last values Example: Payload = int, merge = max no reference to history 22

86 Operation-based replication s s1 s2 s2.u(b) s1.u(a) S S push to all replicas eventually push small updates - more efficient than state-based s3 At source: prepare broadcast to all replicas Eventually, at all replicas: update local replica 23

87 Operation-based replication s s1 s2 s2.u(b) s1.u(a) S S b b s1.u(b) D push to all replicas eventually push small updates - more efficient than state-based s3 D s3.u(b) At source: prepare broadcast to all replicas Eventually, at all replicas: update local replica 23

88 Operation-based replication s s1 s2 s2.u(b) s1.u(a) S S b b a s1.u(b) D a s2.u(a) D push to all replicas eventually push small updates - more efficient than state-based s3 D s3.u(b) D s3.u(a) At source: prepare broadcast to all replicas Eventually, at all replicas: update local replica 23

89 Op-based: commute CRDT s s1 s1.u(a) S b s1.u(b) D a s2.u(a) s2 s2.u(b) S b a D s3 D s3.u(b) D s3.u(a) Delivery order ensures downstream precondition happened-before or weaker If:!! (Liveness) all replicas execute all operations!!! in delivery order (Safety) concurrent operations all commute Then: replicas converge 24

90 Monotonic semi-lattice commutative Systematic transformation Inefficient Hand-crafted op-based implementation 1. A state-based object can emulate an operation-based object, and vice-versa 2. State-based emulation of a CvRDT is a CmRDT 3. Operation-based emulation of a CvRDT is a CmRDT 25

91 Operation-based OR-Set Payload: S = { (e,α), (e, β), (e', γ), }!! where α, β, unique Operations: lookup(e) = α: (e, α) S Set of IDs associated with a in the old state As observed by source! 26

92 Operation-based OR-Set Payload: S = { (e,α), (e, β), (e', γ), }!! where α, β, unique Operations: lookup(e) = α: (e, α) S add(e) = S S {(e, α)} where α fresh Set of IDs associated with a in the old state As observed by source! 26

93 Operation-based OR-Set Payload: S = { (e,α), (e, β), (e', γ), }!! where α, β, unique Operations: lookup(e) = α: (e, α) S add(e) = S S {(e, α)} where α fresh remove (e) =! (at source) R = {(e, α) S}! (downstream) S S \ R Set of IDs associated with a in the old state As observed by source! 26

94 Operation-based OR-Set Payload: S = { (e,α), (e, β), (e', γ), }!! where α, β, unique Operations: lookup(e) = α: (e, α) S add(e) = S S {(e, α)} where α fresh remove (e) =! (at source) R = {(e, α) S}! (downstream) S S \ R Set of IDs associated with a in the old state As observed by source! No tombstones 26

95 Operation-based OR-Set Payload: S = { (e,α), (e, β), (e', γ), }!! where α, β, unique Operations: lookup(e) = α: (e, α) S add(e) = S S {(e, α)} where α fresh remove (e) =! (at source) R = {(e, α) S}! (downstream) S S \ R Set of IDs associated with a in the old state As observed by source! No tombstones {true} add(e) remove(e) {e S} 26

96 Ongoing work

97 CRDTs for P2P & Cloud Computing ConcoRDanT: ANR Systematic study of conflict-free design space Theory and practice Characterise invariants Library of data types Not universal Conflict-free vs. conflict semantics Move consensus off critical path, non-critical ops 28

98 CRDT + dataflow Web site 1 Web site 2 Set Whitelist crawl extract links Graph Graph spam detector Web site 3 Content DB Map URL to last 2 versions extract words Words Map word to Set of URLs Incremental, asynchronous processing Replicate, shard CRDTs near the edge Propagate updates dataflow Throttle according to QoS metrics (freshness, availability, cost, etc.) Scale: sharded Synchronous processing: snapshot, at centre 29

99 OR-Set + Snapshot Read consistent snapshot Despite concurrent, incremental updates Unique token = time (vector clock) α = Lamport (process i, counter t) UIDs identify snapshot version Snapshot: vector clock value Retain tombstones until not needed lookup(e, t) = (e, i, t') A : t'>t (e, i, t') R: t'>t 30

100 Sharded OR-Set Very large objects Independent shards Static: hash (Dynamic: requires consensus to rebalance) Statically-Sharded CRDT Each shard is a CRDT Update: single shard No cross-object invariants The combination remains a CRDT Statically Sharded OR-Set Combination of smaller OR-Sets Snapshot: clock across shards 31

101 Take aways Principled approach Strong Eventual Consistency Two sufficient conditions: State: monotonic semi-lattice Operation: commutativity Useful CRDTs Register, Counter, Set, Map (KVS), Graph, Monotonic DAG, Sequence Future work Snapshot, sharding, dataflow A wee bit of synchronisation 32

102 Portfolio of CRDTs Register Last-Writer Wins Multi-Value Set Grow-Only 2P Observed-Remove Map Set of Registers Counter Unlimited Non-negative Graphs Directed Monotonic DAG Edit graph Sequence Edit sequence 33