Corbett et al., Spanner: Google s Globally-Distributed Database

Transcription

1 Corbett et al., : Google s Globally-Distributed Database MIMUW

2 ACID transactions

3 ACID transactions SQL queries

4 ACID transactions SQL queries Semi-relational data model

5 ACID transactions SQL queries Semi-relational data model Lock-free distributed transactions

6 ACID transactions SQL queries Semi-relational data model Lock-free distributed transactions Global scale

7 ACID transactions SQL queries Semi-relational data model Lock-free distributed transactions Global scale Externally consistent

8 Consistency matters Unfriend untrustworthy person X Post: My government is repressive...

9 External consistency Linearisability: if T 1 commits before T 2 starts, then T 1 s commit timestamp is smaller than T 2 s

10 External consistency Linearisability: if T 1 commits before T 2 starts, then T 1 s commit timestamp is smaller than T 2 s The first system to provide the guarantee at global scale

11 External consistency Linearisability: if T 1 commits before T 2 starts, then T 1 s commit timestamp is smaller than T 2 s The first system to provide the guarantee at global scale Allows:

12 External consistency Linearisability: if T 1 commits before T 2 starts, then T 1 s commit timestamp is smaller than T 2 s The first system to provide the guarantee at global scale Allows: consistent reads in the past

13 External consistency Linearisability: if T 1 commits before T 2 starts, then T 1 s commit timestamp is smaller than T 2 s The first system to provide the guarantee at global scale Allows: consistent reads in the past consistent backups

14 External consistency Linearisability: if T 1 commits before T 2 starts, then T 1 s commit timestamp is smaller than T 2 s The first system to provide the guarantee at global scale Allows: consistent reads in the past consistent backups consistent MapReduce executions

15 External consistency Linearisability: if T 1 commits before T 2 starts, then T 1 s commit timestamp is smaller than T 2 s The first system to provide the guarantee at global scale Allows: consistent reads in the past consistent backups consistent MapReduce executions atomic schema updates

16 Organisation A universe consists of zones

17 Organisation A universe consists of zones Zone has:

18 Organisation A universe consists of zones Zone has: a zonemaster

19 Organisation A universe consists of zones Zone has: a zonemaster spanservers that serve data to clients

20 Organisation A universe consists of zones Zone has: a zonemaster spanservers that serve data to clients location proxies

21 Organisation A universe consists of zones Zone has: a zonemaster spanservers that serve data to clients location proxies Global:

22 Organisation A universe consists of zones Zone has: a zonemaster spanservers that serve data to clients location proxies Global: universe master

23 Organisation A universe consists of zones Zone has: a zonemaster spanservers that serve data to clients location proxies Global: universe master placement driver responsible for data transfer across zones

24 Organisation A universe consists of zones Zone has: a zonemaster spanservers that serve data to clients location proxies Global: universe master placement driver responsible for data transfer across zones Bucketing abstraction: directories

25 Spanserver tablets

26 Spanserver tablets key, timestamp string

27 Spanserver tablets key, timestamp string stored on Colossus as B-trees and WAL

28 Spanserver tablets key, timestamp string stored on Colossus as B-trees and WAL Paxos state machine

29 Spanserver tablets key, timestamp string stored on Colossus as B-trees and WAL Paxos state machine Leader

30 Spanserver tablets key, timestamp string stored on Colossus as B-trees and WAL Paxos state machine Leader long-lived leader leases

31 Spanserver tablets key, timestamp string stored on Colossus as B-trees and WAL Paxos state machine Leader long-lived leader leases lock table

32 Spanserver tablets key, timestamp string stored on Colossus as B-trees and WAL Paxos state machine Leader long-lived leader leases lock table transaction manager

33 TrueTime Idea: expose clock uncertainty

34 TrueTime Idea: expose clock uncertainty Time masters: GPS or atomic clocks

35 TrueTime Idea: expose clock uncertainty Time masters: GPS or atomic clocks (Armageddon masters)

36 TrueTime Idea: expose clock uncertainty Time masters: GPS or atomic clocks (Armageddon masters) Timeslave daemon polls a variety of masters

37 TrueTime Idea: expose clock uncertainty Time masters: GPS or atomic clocks (Armageddon masters) Timeslave daemon polls a variety of masters Marzullo s algorithm used to detect liars

38 TrueTime Idea: expose clock uncertainty Time masters: GPS or atomic clocks (Armageddon masters) Timeslave daemon polls a variety of masters Marzullo s algorithm used to detect liars Eviction of malfunctioning masters and clients

39 TrueTime Idea: expose clock uncertainty Time masters: GPS or atomic clocks (Armageddon masters) Timeslave daemon polls a variety of masters Marzullo s algorithm used to detect liars Eviction of malfunctioning masters and clients Assumed upper bound on clock drift: 200 µs s.

40 Transactions Operation Concurrency control Replica Required RW trans. pessimistic leader RO trans. lock-free leader (timestamp), any Snapshot read lock-free any

41 RW transactions Two-phase locking, timestamps assigned when all locks are being held

42 RW transactions Two-phase locking, timestamps assigned when all locks are being held Disjoint leader lease intervals

43 RW transactions Two-phase locking, timestamps assigned when all locks are being held Disjoint leader lease intervals Start: Coordinator leader assigns timestamp s TT.now().latest after receiving the commit request, and greater than all prepare timestamps previously issued

44 RW transactions Two-phase locking, timestamps assigned when all locks are being held Disjoint leader lease intervals Start: Coordinator leader assigns timestamp s TT.now().latest after receiving the commit request, and greater than all prepare timestamps previously issued Commit wait: Clients cannot see any data commited by the transaction until TT.after(s) is true

45 RW transactions Two-phase locking, timestamps assigned when all locks are being held Disjoint leader lease intervals Start: Coordinator leader assigns timestamp s TT.now().latest after receiving the commit request, and greater than all prepare timestamps previously issued Commit wait: Clients cannot see any data commited by the transaction until TT.after(s) is true Wound-wait

46 RW transactions Two-phase locking, timestamps assigned when all locks are being held Disjoint leader lease intervals Start: Coordinator leader assigns timestamp s TT.now().latest after receiving the commit request, and greater than all prepare timestamps previously issued Commit wait: Clients cannot see any data commited by the transaction until TT.after(s) is true Wound-wait Client drives two-phase commit using the identity of the coordinator

47 Snapshot reads Safe time Maximum timestamp at which the replica is up to date Minimum of: timestamp of the highest-applied Paxos write

48 Snapshot reads Safe time Maximum timestamp at which the replica is up to date Minimum of: timestamp of the highest-applied Paxos write prepare timestamps of prepared (but not commited) transactions

49 RO transactions A timestamp needs to be assigned

50 RO transactions A timestamp needs to be assigned Scope expression required to negotiate timestamp between all Paxos groups involved

51 RO transactions A timestamp needs to be assigned Scope expression required to negotiate timestamp between all Paxos groups involved Either TT.now().latest...

52 RO transactions A timestamp needs to be assigned Scope expression required to negotiate timestamp between all Paxos groups involved Either TT.now().latest or the timestamp of the last commited write at a Paxos group

53 Q&A