MySQL High Availability Solutions. Alex Poritskiy Percona

Transcription

1 MySQL High Availability Solutions Alex Poritskiy Percona

2 The Five 9s of Availability Clustering & Geographical Redundancy Clustering Technologies Replication Technologies Well-Managed disasters power failures network failures hardware failures software failures maintenance operations hardware upgrades software upgrades Unmanaged 35 days 4 days 8 hours 50 mins 5 mins Small Business ISPs & Mainstream Business Data Centers Banking Medical Telco Military Defense %!2

3 MySQL High Availability Solutions MySQL Replication MHA/Orchestrator by Github and MySQL Replication Percona XtraDB Cluster/Galera MySQL InnoDB Cluster (Group Replication) MySQL NDB Cluster!3

4 MySQL Replication %!4

5 MySQL Replication Topologies Master > Slave Master > Slaves Masters > Slave (Multi-Source) Master > Slave > Slaves Master < > Master (Multi-Master) Circular (Multi-Master)!5

6 Inside MySQL Replication Writes & Reads Web/App Server Writes relay binlog mysqld index & binlogs I/O Thread SQL Thread data Replication binlog data MySQL Master mysqld MySQL Slave!6

7 MySQL Replication (Scale-Out) Web/App Server Writes & Reads Reads Master Server Slave Server Backups Writes Replication Writes Index & Bin Log Rotation Possible Roles Fail over server Used for performing backups Read load balancing Additional slaves allow Scale-Out!7

8 MySQL Replication(Fail-Over) Web/App Server Master Server X X Manual Fail Over Writes & Reads Slave Server Fail Over Writes X X Replication Writes Index & Bin Log Rotation!8

9 MySQL Replication Pros Mature, standard feature of MySQL. Compatible with many topologies. Supports multi-datacenter clusters. Easy to configure. Continues to gain new features: Semi-synchronous Row-based binary logs Parallel SQL threads Global Transaction ID (GTID)!9

10 MySQL Replication Cons Slave lag. Use row-based binary logs and other strategies. Slave drift. Use read-only, pt-table-checksum, pt-table-sync. No automatic failover. Use haproxy and/or keepalived or other solutions.!10

11 Topology managers + MySQL replication %!11

12 MHA toolkit!12

13 MHA toolkit!13

14 MHA toolkit Note that MHA never rebuilds an existing multi-master replication link, it always reconfigures topology using masterslave. MHA only takes care of the MySQL failover but has the ability to add custom code to make the application and any load balancers aware of the change.!14

15 MHA toolkit: Pros Uses traditional MySQL replication. Easy to deploy for an existing replication group. Subscribes slaves to the correct binlog position. Strong command line support Very good point in time recovery via binary logs!15

16 MHA toolkit: Cons Disconnects after failover No GUI Not updated, long term stable release Doesn t crawl and auto discover topology Cannot manage PXC, co-masters or active/active Failover isn t guaranteed instantly - can cause delays as logs are scanned. Still async replication (Consistency may be compromised) Monitor can be a single point of failure (monitor is only needed for failover)!16

17 Orchestrator by Github Orchestrator crawls through your replication topology and updates its GUI based on your current topology and health. Orchestrator also knows about binary log position, GTID, and Binlog Servers. Refactoring replication topologies can be a matter of dragging and dropping a replica under another master.!17

18 Orchestrator by Github!18

19 Orchestrator: Pros Integrated into PMM GUI in addition to command line Various options for managing replication (GTID, psuedo-gtid etc) Built for GTID Can manage backups and other tasks Currently in development with a dedicated team Auto Discovers topology Works with co-masters!19

20 Orchestrator: Cons Does not recognise PXC nodes as part of a single cluster (sees them as separate masters) Point in time recovery relies on good slaves, doesn t do binary log discovery!20

21 PXC, MySQL InnoDB Cluster %!21

22 Percona XtraDB Cluster(PXC) Synchronous replication every node is writable.!22

23 Percona XtraDB Cluster(PXC)!23

24 Percona XtraDB Cluster(PXC) 5 nodes in the 2 Datacenters(example) Main Data Center Secondary Data Center MySQL WAN MySQL gmcast.segment=1 gmcast.segment=2!24

25 Percona XtraDB Cluster(PXC) 3 nodes in the single datacenter + Async Slave Main Data Center Secondary Data Center WAN MySQL Slave Server Percona XtraDB Cluster Asynchronous MySQL Replication slave (not part of PXC)!25

26 Percona XtraDB Cluster(PXC) : Pros Writing to any node updates all other nodes synchronously (like semi-sync replication). All nodes can be writers. Adding a new node, or recovering an offline node, initialises data automatically. Multi-datacenter (assuming low latency) Quorum-based selection of nodes that can take queries. Nodes not in quorum will be shut down and must copy data before they can rejoin the cluster.!26

27 Percona XtraDB Cluster(PXC): Cons Not suitable for large transactions. Not suitable for heavy write traffic, especially rapid writes to a hotspot don't rely on multiple writers. Reinitialising a node can be expensive.!27

28 MySQL InnoDB Cluster Synchronous replication every node is writable as well.!28

29 MySQL InnoDB Cluster: Pros Writing to any node updates all other nodes synchronously (like semi-sync replication). All nodes can be writers. Single-Primary and Multi-Primary modes available Quorum-based selection of nodes that can take queries.!29

30 MySQL InnoDB Cluster: Cons Not suitable for large transactions. Not suitable for heavy write traffic, especially rapid writes to a hotspot don't rely on multiple writers. No SST concept - all nodes needs to start with same data Replication error just stop replication to the node Inconsistencies can/should be fixed manually!30

31 Percona XtraDB Cluster and MySQL Cluster similar limitations Large & Long running transactions higher chance on failures non-efficient replication of large transactions Workload hotspots (updating same set of rows in parallel) Often writing to 1 node is the best solution!31

32 %!32

33 MySQL NDB Cluster Shared-Nothing Synchronous (2-phase commit) In memory store Traditionally used by: Telecom operators Gaming companies!33

34 MySQL NDB Cluster Basic Architecture Web/App Server Web/App Server MySQL Server or NDB API for all Writes & Reads Memory & Disk Data Node NDB API NDB Storage Engine Data Node Management Server Management Server MySQL Cluster!34

35 MySQL NDB Cluster Components Data Nodes Store redundant data fragments Combined to form of Node Groups Management Nodes Used at startup Re-configuration API Nodes Standard SQL interface NDB API(The NDB API is an object-oriented application programming interface for NDB Cluster that implements indexes, scans, transactions, and event handling)!35

36 MySQL NDB Cluster Data node Architecture ID 1 2 Capital Copenhagen Berlin Country Denmark Germany UTC 2 2 Partition 1 Data Node Data Node New York City Tokyo Athens Moscow Oslo Beijing USA Japan Greece Russia Norway China Partition 2 Partition 3 Partition 4 P1-Primary P1-Secondary P2-Secondary P2-Primary Node Group 1 Four Data Nodes Two Replicas Two Node Groups Data Node P3-Primary P4-Secondary Data Node P3-Secondary P4-Primary Node Group 2!36

37 MySQL NDB Cluster Data node Architecture ID Capital Country UTC X Data Node Data Node Copenhagen Berlin New York City Tokyo Athens Moscow Oslo Beijing Denmark Germany USA Japan Greece Russia Norway China Partition 1 Partition 2 Partition 3 Partition 4 P1-Primary P1-Secondary P2-Secondary P2-Primary Node Group 1 data still available Four Data Nodes Two Replicas Two Node Groups X Data Node P3-Primary P4-Secondary Data Node P3-Secondary P4-Primary Node Group 2!37

38 MySQL NDB Cluster Scale-Out NDB API Data Node NDB Storage Engine Data Node Management Server Data Node Data Node Management Server MySQL Cluster!38

39 MySQL NDB Cluster Failover X X X X NDB API Data Node NDB Storage Engine Data Node Management Server X Data Node X Data Node X Management Server MySQL Cluster!39

40 MySQL NDB Cluster + Replication!40

41 MySQL NDB Cluster: Pros Every component has redundancy for failover. Exceptionally high performance for reads and writes, if queries are simple, single-row. Ultimate HA add new nodes with no downtime.!41

42 MySQL NDB Cluster: Cons Not a general purpose system You probably need to redesign your schema. NDB is a distinct storage engine; it is not InnoDB. Queries that does not hit primary keys can be much slower and transfer lots of data temporarily to SQL nodes. Not suitable for multi-datacenter. (only can be achieved by async replication between the clusters) Increased hardware costs All components are redundant. Need enough RAM!42

43 How to choose right HA solution? Know your availability goals and requirements Know you workload(what? How many? What ratio? How complex?,etc) Know considered solutions limitations Know your application requirements and logic internals!43

44 How to choose right HA solution? Do not fully rely on the HA solution (be ready to handle some scenarios on the application side, since usually there is no solution that could cover any use case and/or scenario ) Once implemented : develop and follow code/infrastructure deployment guideline in order to make sure capability with implemented solution (What can do? What can t do? How we are doing stuff now?, etc ) Test, test, test!44

45 Demo: MySQL NDB Cluster Let s build small MySQL NDB Cluster: 4 Pre-setup Ubuntu VM s: mgm_sql Management node node Data node1 node Data node2 mgm_sql SQL node!45

46 Thank you!!46

47 Thank you Sponsors!!47