SWIFTCLOUD: GEO-REPLICATION RIGHT TO THE EDGE

Transcription

1 SWIFTCLOUD: GEO-REPLICATION RIGHT TO THE EDGE Annette Bieniusa T.U. Kaiserslautern Carlos Baquero HSALab, U. Minho Marc Shapiro, Marek Zawirski INRIA, LIP6 Nuno Preguiça, Sérgio Duarte, Valter Balegas CITI / U. Nova de Lisboa Copyright Nuno Preguiça CITI DI / FCT / UNL / 1

2 CONTEXT Latency to access cloud- based web services Commonly 10~100ms from the client- DC + poten7al DC- DC cost An increasing number of web applica3ons run code in clients for: responsiveness: avoid client- DC latency fault- tolerance: support opera7on on disconnec7on/server fault Industry catalyst HTML 5, Ajax Mobile devices NUNO PREGUIÇA CITI DI / FCT / UNL / 2

3 PROBLEM How to support data sharing web applications running in clients? NUNO PREGUIÇA CITI DI / FCT / UNL / 3

4 PROBLEM How to support data sharing web applications running in clients? Fast reads & fast writes Consistent data and session guarantees Low latency with useful consistency Fault-tolerance Support disconnection Tolerate DC faults / connectivity problems NUNO PREGUIÇA CITI DI / FCT / UNL / 4

5 OUTLINE Introduction Low latency with useful consistency Fault-tolerance Final remarks NUNO PREGUIÇA CITI DI / FCT / UNL / 5

6 LOW LATENCY for reads Caching data in client machines for writes Weak consistency Requires mergeable writes => CRDT Support fast writes => asynchronous writes; guarantees for app? NUNO PREGUIÇA CITI DI / FCT / UNL / 6

7 LOW LATENCY WITH USEFUL SEMANTICS Execute sequence of operations with atomic semantics Support for transactions Writes: mergeable, atomic visibility Reads: different isolation levels SI, repeatable reads, etc. AddFriend( X, Y) {!!X.add(Y)!!Y.add(X)! }! NUNO PREGUIÇA CITI DI / FCT / UNL / 7

8 LOW LATENCY WITH USEFUL SEMANTICS Client must observe coherent database state Observe her previous writes (read your writes) Successive reads reflect a non-decreasing set of writes (monotonic reads) Writes must be applied after observed reads (writes follow reads) Causal consistency at client Challenge: do it efficiently (small dep. info) NUNO PREGUIÇA CITI DI / FCT / UNL / 8

9 TRANSACTION EXECUTION ON SERVER seq [0 0 0] (C,1)è (IE,1 [1 0 0] IE (EU) x y x 000 = 0 y 000 = 0 T(IE,1): add(2) sur T(C,1)è (IE,1) : y.add(2) : y.add(2) Commit T(C,1) c T(C,1): y.add(2) NUNO PREGUIÇA CITI DI / FCT / UNL / 9

10 For SI, read version dep For read committed, read most recent version TRANSACTION EXECUTION ON CLIENT seq [0 0 0] [1 0 0] [1 1 0] IE (EU) x y x 000 = 0 y 000 = 0 T(IE,1): add(2) T(CA,1): sub(3) sur T(C,1)è (IE,1) : y.add(2) x.get(); dep=[1 0 0 ] y 000 = 0 Y 100 = 2 c T(C,1): y.add(2) T2.start dep = [1 0 0] T: y.get()=2 T: x.get()=0 NUNO PREGUIÇA CITI DI / FCT / UNL / 10

11 TRANSACTION EXECUTION ON CLIENT: ASYNC COMMIT seq [0 0 0] Keep cache fresh relying on event dissemination of data changes [1 0 0] [1 1 0] IE (EU) x y x 000 = 0 y 000 = 0 T(IE,1): add(2) T(CA,1): sub(3) x.get(); dep=[0 0 0] sur T(C,1)è (IE,1) : y.add(2) y 000 = 0 Y 000+T(C,1) = 2 c T(C,1): y.add(2) T2.start dep=[0 0 0]+T(C,1) T2: y.get()=2 T2: x.get()=0 NUNO PREGUIÇA CITI DI / FCT / UNL / 11

12 EXPERIMENTAL SETUP Swiftsocial application modeled after WalterSocial DC with single node running all components Amazon EC2 (Europe, US West, US East) clients planetlab NUNO PREGUIÇA CITI DI / FCT / UNL / 12

13 LATENCY IMPROVEMENT: SYNCHRONOUS COMMIT How moving the data close to the client improves latency? 100 Synchronous commit Cumulative Ocurrences (%) Transaction Execution Time (ms) Clt/Dc RTT Clt/Cdn/Dc RTT writes@client reads@dc reads@client Clt/Cdn RTT writes@cdn reads@cdn writes@dc NUNO PREGUIÇA CITI DI / FCT / UNL / 13

14 LATENCY IMPROVEMENT: ASYNCHRONOUS COMMIT How moving the data close to the client improves latency? 100 Asynchronous commit Cumulative Ocurrences (%) Transaction Execution Time (ms) Clt/Dc RTT Clt/Cdn/Dc RTT writes@client reads@dc reads@client Clt/Cdn RTT writes@cdn reads@cdn writes@dc NUNO PREGUIÇA CITI DI / FCT / UNL / 14

15 OUTLINE Introduction Low latency with useful semantics Fault-tolerance Final remarks NUNO PREGUIÇA CITI DI / FCT / UNL / 15

16 FAULT-TOLERANCE Support temporary disconnection Immediate: support execution from client cache Note: operating in a disconnected replica gives causal consistency Support DC-failure or client-dc network failure NUNO PREGUIÇA CITI DI / FCT / UNL / 16

17 SUPPORT DC-FAILURES OR CLIENT/DC NETWORK FAILURE Problem: on fail-over, new DC may not know some observed updates Leads to blocking or breaking session guarantees IE (EU) T(IE,1) X=2 T(IE,2) Y=5 client CA (US) T(IE,1) X=2 Potential solutions Operations only complete after being stable in f+1 DCs Slow writes Involve clients in fail-over: clients keep enough information to ensure session guarantees T(C,1) get(y)=5 dep=[2 0] T(C,2) get(y)? NUNO PREGUIÇA CITI DI / FCT / UNL / 17

18 CLIENT-ASSISTED FAILOVER Own updates Keep a log of own updates Observable state Union of own updates and stable updates On fail-over Replay log of own updates => may lead to double delivery Guarantee idempotence rely on CRDT properties; use client identifiers in operation execution NUNO PREGUIÇA CITI DI / FCT / UNL / 18

19 DC-FAULT TOLERANCE How often are reads stale? 3.5 Individual Reads Transactions 3.0 Stale Reads [ % ] Think Time [ s ] NUNO PREGUIÇA CITI DI / FCT / UNL / 19

20 FINAL REMARKS Extend geo-replication up to the client machine Low latency with adequate semantics Mergeable atomic transactions Causal+ consistency Partial replication Efficient implementation DC fault-tolerance Replay own updates Read stable updates NUNO PREGUIÇA CITI DI / FCT / UNL / 20

21 GEO-REPLICATION SCALABILITY How does SwiftCloud scale with the number of DCs? DC 1 DC 2 DC Latency [ ms ] ,000 2,000 3,000 4,000 5,000 6,000 Throughput [ tps ] NUNO PREGUIÇA CITI DI / FCT / UNL / 21