PhxSQL: A High-Availability & Strong-Consistency MySQL Cluster. Ming

Transcription

1 PhxSQL: A High-Availability & Strong-Consistency MySQL Cluster Ming CHEN@WeChat

2 Why PhxSQL Highly expected features for MySql cluster

3 Availability and consistency in MySQL cluster Master-slaves replication client SVR A MySQL (master) Replication solutions Fully synchronous Semi-synchronous Asynchronous Third-party aided failover SVR B MySQL (slave) Binlog 3: z 2: y 1: x pull Binlog 3: z 2: y 1: x Innodb pull SVR C MySQL (slave) Binlog 3: z 2: y 1: x Zookeeper/Admin/ Innodb Innodb 3

4 Call for both high-availability and strongconsistency Critical Applications demand both high-availability and strong-consistency accounts/financial transactions/ Support for both high-availability and strong-consistency greatly simplifies system design makes correctness-reasoning easier 4

5 A new MySQL cluster supports the same highavailability and strong-consistency as Zookeeper does PhxSQL 5

6 What is PhxSQL High-availability & strong-consisteny MySQL cluster

7 Key PhxSQL features Support full compatibility with MySQL Support high availability and linearizable consistency Support deployment in wide-area network Support online reconfiguration of cluster membership 7

8 Support full compatibility with MySQL Transparent to MySQL clients Support full features of MySQL Even serializable level of transaction isolation Minimum intrusive change to MySQL source code modify after_flush add before_recoverty 8

9 Support high availability Fully automatic failover within configurable seconds A cluster works well when more than half of cluster servers still function 9

10 Support linearizable consistency A cluster of PhxSQL seems to be a single MySQL server to MySQL clients concurrently accessing it PhxSQL supports the same consistency level as strong as Zookeeper does! 10

11 Support online reconfiguration of cluster membership Add a new server, remove an old server, or replace an old server with a new one in an atomic fashion while still serving read/write requests Be easier for maintenance and more friendly to clients 11

12 How PhxSQL Works MySQL cluster powered by Paxos

13 Paxos maintains consistent states across servers by enforcing same operations on all servers when more than half of servers work SVR A SVR B SVR C 1: R1: Foo(1, 2) 2: T1: Foo(3, 4) 3: S1: Bar(1, 10) 4: R2: Bar(3, 30) God int X=0; int Y=0; void Foo(int a, int b); bool Bar(int a, int b); invoke: R1, T1, S1, R2, int X=0; int Y=0; void Foo(int a, int b); bool Bar(int a, int b); invoke: R1, T1, S1, R2, int X=0; int Y=0; void Foo(int a, int b); bool Bar(int a, int b); invoke: R1, T1, S1, R2, Paxos Paxos Paxos Foo(1, 2); Bar(3, 30) Bar(1, 10) Foo(3, 4) 13 Client R Client S Client T

14 PhxSQL enforces a simple but effective constraint by Paxos Constraint: two MySQL servers have SAME states if and only if they have SAME binlog Enforcement PhxSQL maintains a GLOBAL consistent binlog by Paxos Every MySQL server aligns its LOCAL binlog to the GLOBAL one. 14

15 MySQL cluster MySQL (master) SVR A client pull Binlog 3: z 2: y 1: x SVR B SVR C MySQL (slave) Innodb pull MySQL (slave) Binlog 3: z 2: y 1: x Binlog 3: z 2: y 1: x Innodb Innodb 15

16 PhxSQL client forward MySql (master) Binlog 3: z 2: y 1: x Phx Proxy SVR A Phx BinlogSvr Paxos 1. Paxos detects failure and elects new master by leasing and periodic heartbeat 2. Clients access master MySQL transparently via PhxProxy who forwards requests to current master 3. MySql master appends local binlog to global consistent PhxBinlog maintained by Paxos 4. MySql slaves pull binlog from global consistent PhxBinlog MySql (slave) Phx Proxy SVR B Phx BinlogSvr Phx Plugin Innodb Phx Binlog 3: z 2: y 1: x MySql (slave) Phx Proxy SVR C Phx BinlogSvr Binlog 3: z 2: y 1: x Paxos Binlog 3: z 2: y 1: x Paxos pull pull Phx Plugin Innodb Phx Binlog 3: z 2: y 1: x Phx Plugin Innodb Phx Binlog 3: z 2: y 1: x 16

17 Paxos maintains consistent states in the cluster PhxBinlogSvr A PhxBinlogSvr B PhxBinlogSvr C 1: R1: C1(1, R) 2: T1: C1(1, T) 3: S1: C1(1, S) master_ver master_id binlog_queue SM master_ver master_id binlog_queue SM master_ver master_id binlog_queue SM 4: R2: C2() C1: bool change_master(ver, new_master); C2: int app_binlog(binlog_id, binlog); C1: bool change_master(ver, new_master); C2: int app_binlog(binlog_id,binlog); C1: bool change_master(ver, new_master); C2: int app_binlog(binlog_id,binlog); God Paxos Paxos Paxos C1(1, R); C2( R_xxx, insert foo=2 into table_1 ) C1(1, S) C1(1, T) 17 PhxPlugin:Client R PhxPlugin:Client S PhxPlugin:Client T

18 How to Use PhxSQL Easy to integrate into existing system

19 Case 1: non-intrusive change to existing system Assume there some kinds of naming service that tells the current master in existing system Zookeeper/DNS/configuration file/... Develop a new daemon program to learn the change of master in PhxSQL cluster and update information in the naming service accordingly Safety: PhxSQL ensures consistency even the master information in naming service is stale or MySQL clients connects to a slave daemon II. update I. learn new master master PhxSQL client 2. request 1. get ip of master naming service master ip: slave slave

20 Case 2: minimal intrusive change to existing system Traditional invocation scenario 1. Read the IP of master from configuration file 2. Get MySQL handler by calling mysql_real_connect(mysql *mysql, const char * IP, ) with the IP 3. Invoke other MySQL client API with the handler PhxSQL invocation scenario 1. Read the IP list of PhxSQL cluster servers from configuration file 2. Get MySQL handler by calling PhxSQLClientBase::Connect() with the IP list and then PhxSQLClientBase::GetMySQLFD() 3. Invoke other MySQL client API with the handler 20

21 Performance On par with MySQL semi-sync replication

22 Performance test settings: PhxSQL vs. MySQL Semi-sync 3 servers: Intel(R) Xeon(R) CPU 1.90GHz * 2, 32G memory, SSD Raid10, 1000M NIC Network ping latency: master->slave: 3ms; client->master: 4ms Percona Master of MySQL semi-sync replication waits for only ONE ack Test tool and parameters: sysbench --oltp-tables-count=10 --oltp-tablesize= num-threads=500 --max-requests= report-interval=1 -- max-time=200 22

23 QPS on par 200 concurrent client threads PhxSQL MySQL insert.lua (100% write) select.lua (0% write) OLTP.lua (20% write) 500 concurrent client threads PhxSQL MySQL insert.lua (100% write) select.lua (0% write) OLTP.lua (20% write) 23

24 Response time (ms) on par 200 concurrent client threads PhxSQL MySQL insert.lua (100% write) select.lua (0% write) OLTP.lua (20% write) concurrent client threads PhxSQL MySQL insert.lua (100% write) select.lua (0% write) OLTP.lua (20% write) 24

25 Current progress Deployed in WeChat account system WeChat account: 889M monthly active users Support for Percona 5.7 in progress Open source: 25