MySQL High Availability

Transcription

1 MySQL High Availability InnoDB Cluster and NDB Cluster Ted Wennmark Copyright 2016, Oracle and/or its its affiliates. All All rights reserved.

2 Safe Harbor Statement The following is intended to outline our general product direction. It is intended for information purposes only, and may not be incorporated into any contract. It is not a commitment to deliver any material, code, or functionality, and should not be relied upon in making purchasing decisions. The development, release, and timing of any features or functionality described for Oracle s products remains at the sole discretion of Oracle. 2

3 100% Almost all organizations require their most critical systems to be highly available 3

4 InnoDB Cluster or NDB Cluster 4

5 How Do The Two Compare I MySQL InnoDB Cluster Easy HA built into MySQL 5.7+ Write consistency Read Scalability App failover using MySQL Router Management via mysqlsh InnoDB storage engine MySQL NDB Cluster In memory database Automatic sharding Native access via several API Read/write consistency Read/write scalability NDB storage engine

6 How Do The Two Compare II MySQL InnoDB Cluster MySQL NDB Cluster Storage Engine InnoDB NDBCLUSTER Distributed Architecture Shared nothing Shared nothing Consistency Model Weak Consistency Strong Consistency Sharding No Yes Arbitration No Yes Load Balancing No Yes NoSQL APIs MySQL Document Store Native NDB API Operational Complexity Medium High Administration Standard (MySQL) Custom (MySQL + NDB)

7 InnoDB Cluster 7

8 InnoDB Cluster High Availability becomes a core first class feature of MySQL! MySQL Shell Setup, Manage, Orchestrate App Servers with MySQL Router MySQL Group Replication 8

9 MySQL InnoDB Cluster: Vision A single product MySQL with high availability and scaling features baked in; providing an integrated end-to-end solution that is easy to use. 9

10 MySQL InnoDB Cluster: Goals One Product: MySQL All components created together Tested together Packaged together Flexible and Modern SQL and NoSQL together Protocol Buffers Developer friendly Easy to Use One client: MySQL Shell Easy packaging Integrated orchestration Homogenous servers Support Read/Write Scale Out Sharded clusters Federated system of N replica sets Each replica set manages a shard 10

11 MySQL InnoDB Cluster: High Level Architecture Application Application Application Application MySQL Connector MySQL Connector MySQL Connector MySQL Connector MySQL Router MySQL Router MySQL Router MySQL Router MySQL Shell MySQL InnoDB cluster MySQL Enterprise Monitor

12 MySQL Group Replication Natively distributed and highly available replica sets

13 MySQL Group Replication: What Is It? Group Replication library Implementation of Replicated Database State Machine theory MySQL GCS is based on Paxos (variant of Mencius) Provides virtually synchronous replication for MySQL 5.7+ Supported on all MySQL platforms Linux, Windows, Solaris, OSX, FreeBSD MySQL Shell Setup, Manage, Orchestrate App Servers with MySQL Router Multi-master update anywhere replication plugin for MySQL with built-in conflict detection and resolution, automatic distributed recovery, and group membership. MySQL Group Replication 13

14 MySQL Group Replication: What Does It Provide? A highly available distributed MySQL database service Removes the need for manually handling server fail-over Provides distributed fault tolerance Enables Active/Active update anywhere setups Automates reconfiguration (adding/removing nodes, crashes, failures) Automatically detects and handles conflicts 14

15 MySQL Group Replication: Architecture Node Types R: Traffic routers/proxies: mysqlrouter, haproxy, sqlproxy,... M: mysqld nodes participating in Group Replication 15

16 MySQL Server: Full Stack 16

17 MySQL Router Transparent application connection routing

18 MySQL Router: 2.1 Native support for InnoDB clusters Understands Group Replication topology Utilizes metadata schema stored on each member Bootstraps itself and sets up client routing for the InnoDB cluster Allows for intelligent client routing into the InnoDB cluster Supports multi-master and single primary modes Core improvements Logging Monitoring Performance Security MySQL Shell Setup, Manage, Orchestrate labs.mysql.com App Servers with MySQL Router MySQL Group Replication 18

19 MySQL Shell Single tool for development, setup, management, orchestration, and monitoring

20 MySQL Shell A single unified client for all administrative and operations tasks Multi-Language: JavaScript, Python, and SQL Naturally scriptable Supports both Document and Relational models Exposes full Development and Admin API MySQL Shell provides the developer and DBA with a single intuitive, flexible, and powerfull interface for all MySQL related tasks! MySQL Shell Setup, Manage, Orchestrate App Servers with MySQL Router MySQL Group Replication 20

21 MySQL Shell: Admin API Database Administration Interface mysql-js> dba.help() The global variable 'dba' is used to access the MySQL AdminAPI Perform DBA operations Manage MySQL InnoDB clusters Create clusters Deploy MySQL instances Get cluster info Start/Stop MySQL Instances Validate MySQL instances MySQL Shell Setup, Manage, Orchestrate labs.mysql.com App Servers with MySQL Router MySQL Group Replication

22 MySQL Shell Deploy MySQL Instances shell> mysqlsh mysql-js> dba.deploylocalinstance(3306) mysql-js> dba.deployinstance( hanode2:3306 ) mysql-js> dba.deployinstance( hanode3:3306 ) 22

23 MySQL Shell Create an InnoDB Cluster mysql-js> \connect mysql-js> cluster = dba.createcluster( NewAppCluster') mysql-js> cluster.addinstance('root@hanode2:3306') mysql-js> cluster.addinstance('root@hanode3:3306') 23

24 MySQL Shell Add a MySQL Router shell> mysqlrouter --bootstrap hanode1:3306 shell> mysqlrouter & shell> mysqlsh --uri root@localhost:

25 What s Next MySQL Document Store Relational & Document Models S1 Read Scale-Out Async Replication + Auto Failover S3 MySQL HA Out-Of-Box HA S2 Write Scale-Out Sharding S4 * Download InnoDB Cluster Preview Release from labs.mysql.com 25

26 MySQL Enterprise Monitor Native holistic support for InnoDB clusters Topology views Detailed metrics and graphs Best Practice advice Monitoring of MySQL Routers Monitoring of Group Replication 26

27 NDB Cluster 27

28 Driving Database Requirements In-Memory Real-Time Performance Extreme Read & Write Scalability Rock Solid Availability Elasticity 9th April 2015 Copyright 2015, oracle and/or its affiliates. All rights reserved 28

29 No Trade-Offs: Cellular Network HLR / HSS Location Updates AuC, Call Routing, Billing Pre & Post Paid Billing, AuC, VLR Massive volumes of write traffic < 3ms database response Downtime & lost transactions = lost $ Extremly cost sensitive market MySQL Cluster in Action:

30 No Trade-Offs: Massive Parallel Online Games User Session Location Updates, Character movements,... Session management Session management Massive volumes of write traffic < 3ms database response Downtime & lost transactions = lost $ Extremly sensitive crowd

31 No Trade-offs: Hadoop NDB Cluster powering the world s fastest and most scalable Hadoop filesystem. Scaling Hadoop 20x! MySQL NDB Cluster Data Nodes ClusterJ NameNodes Leader HDFS Client DataNodes Hops.io Confidential Oracle Internal/Restricted/Highly Restricted 31

32 MySQL Cluster Overview REAL-TIME HIGH SCALE, READS + WRITES FULLY ELASTIC % AVAILABILITY SQL + NoSQL LOW TCO Memory optimized tables with durability Predictable Low-Latency, Bounded Access Time Auto-Sharding, Active-Active ACID Compliant, OLTP + Real-Time Analytics Add and remove storage and performance capacity in seconds Fully cloud enabled Active-Active, Shared nothing, no Single Point of Failure Self Healing + On-Line Operations Key/Value + Complex, Relational Queries SQL + Memcached + JavaScript + Java + HTTP/REST & C++ Open Source + Commercial Editions, Easy to use and deploy Commodity hardware + Management, Monitoring Tools 9th April 2015 Copyright 2015, oracle and/or its affiliates. All rights reserved 32

33 NoSQL Access to MySQL Cluster data Apps Apps Apps Apps Apps Apps Apps Apps Apps Apps Apps Apps JPA Cluster JPA PHP Perl Python Ruby JDBC Cluster J JS Apache Memcached MySQL JNI Node.JS mod_ndb ndb_eng NDB API (C++) MySQL Cluster Data Nodes

34 MySQL Cluster Architecture Clients Application Layer Data Layer MySQL Cluster Data Nodes 9th April 2015 Copyright 2015, oracle and/or its affiliates. All rights reserved 34

35 Data Partitioning / Auto-sharding User-id (PK) Service (PK) Data chat xxx chat xxx photos xxx photos xxx reminders xxx location xxx MySQL Cluster Data Nodes 28/09/2016 Copyright 2015, oracle and/or its affiliates. All rights reserved 35

36 Data Partitioning / Auto-sharding Shard Key User-id (PK) Service (PK) Data chat xxx chat xxx photos xxx photos xxx reminders xxx location xxx MySQL Cluster Data Nodes 28/09/2016 Copyright 2015, oracle and/or its affiliates. All rights reserved 36

37 Data Partitioning / Auto-sharding Shard Key User-id (PK) Service (PK) Data chat xxx chat xxx photos xxx photos xxx reminders xxx location xxx MySQL Cluster Data Nodes 28/09/2016 Copyright 2015, oracle and/or its affiliates. All rights reserved 37

38 Data Partitioning / Auto-sharding Data Node 1 Data Node 2 Data Node 3 Data Node 4 Node Group Node Group Data transparently sharded between Node Groups Each fragment active in one Data Node with synchronous replication to 2 nd Data Node in same Node Group 28/09/2016 Copyright 2015, oracle and/or its affiliates. All rights reserved 38

39 Data Partitioning / Auto-sharding Shard Key User-id (PK) Service (PK) Data chat xxx chat xxx photos xxx photos xxx reminders xxx location xxx DBA chooses which part of Primary Key to use as shard key Fragment for each row decided by hashing the sharding key Data Node 1 Data Node 2 Data Node 3 Data Node 4 Node Group Node Group 28/09/2016 Copyright 2015, oracle and/or its affiliates. All rights reserved 39

40 On-line Scaling and Elasticity Re-partioning Data Node 1 Data Node 2 Data Node 3 Data Node 4 Data Node 5 Data Node 6 Node Group Node Group Node Group Data automatically rearranged to use new capacity Designed to be a slow background process not impacting real-time performance. 28/09/2016 Copyright 2015, oracle and/or its affiliates. All rights reserved 40

41 Millions of UPDATEs per Second MySQL Cluster 7.2: 1.2 Billion UPDATEs per Minute MySQL Cluster Data Nodes NoSQL C++ API, flexasynch benchmark 30 x Intel E Intel Servers, 2 socket, 64GB ACID Transactions, with Synchronous Replication 9th April 2015 Copyright 2015, oracle and/or its affiliates. All rights reserved 41

42 Reads per second MySQL Cluster 7.4 NoSQL Performance 200 Million NoSQL Reads/Second Memory optimized tables Durable Mix with disk-based tables Massively concurrent OLTP Distributed Joins for analytics Parallel table scans for non-indexed searches MySQL Cluster 7.4 FlexAsych 200M NoSQL Reads/Second 250,000, ,000, ,000, ,000,000 50,000,000 - FlexAsync Reads Data Nodes 28/09/2016 Copyright 2015, oracle and/or its affiliates. All rights reserved 42

43 SQL Statements/sec MySQL Cluster 7.4 SQL Performance 2.5M SQL Statements/Second Memory optimized tables Durable Mix with disk-based tables Massively concurrent OLTP Distributed Joins for analytics Parallel table scans for non-indexed searches MySQL Cluster 7.4 DBT2 BM 2.5M SQL Statements/Second 3,000,000 2,500,000 2,000,000 1,500,000 1,000, ,000 - DBT2 SQL Statements per Second Data Nodes 28/09/2016 Copyright 2015, oracle and/or its affiliates. All rights reserved 43

44 MySQL NDB Cluster 7.5 RC Read Optimized Tables Fully Replicated Capacity and Scale Out MySQL 5.7 JSON Data Type Generated Columns Records-Per-Key Optimization Improved SQL Improved Reporting Improved Logging Improved Visibility Improved Restore Management 44

45 MySQL NDB 7.5: Reading from backup Data Node 1 Data Node 2 Reading from backup allows to read from any copy Previously all reads were directed towards the primary fragment only Node Group Confidential Oracle Internal/Restricted/Highly Restricted 45

46 MySQL NDB Cluster 7.5: Fully replicated Data Node 1 Data Node 2 Data Node 3 Data Node 4 Node Group Node Group Fully replicated allows a table to be read/written locally on any node Ideal for static data, faster join performance Confidential Oracle Internal/Restricted/Highly Restricted 46

47 Next Steps Learn More Authentic MySQL Curriculum: Try it Out dev.mysql.com/downloads/cluster/ 7.4 GA and 7.5 DMR 1 Let us know what you think forums.mysql.com/list.php?25 9th April 2015 Copyright 2015, oracle and/or its affiliates. All rights reserved 47

48 Thank you!