MySQL High available by design

Transcription

1 MySQL High available by design Carsten Thalheimer Sales Consultant MySQL GBU EMEA

2 Safe Harbor Statement The following is intended to outline our general product direction. It is intended for information purposes only, and may not be incorporated into any contract. It is not a commitment to deliver any material, code, or functionality, and should not be relied upon in making purchasing decisions. The development, release, and timing of any features or functionality described for Oracle s products remains at the sole discretion of Oracle. Oracle Nov 2016 Slide 2 Copyright 2015, Oracle 2

3 Agenda MySQL today MySQL 5.7 ( Fall 2015) Recap MySQL 5.7 and MySQL Cluster 7.5 ( Fall 2016) X-Protocol // Group Replication & InnoDB Cluster MySQL What s next? MySQL 8.0 Oracle Nov 2016 Slide 3

4 2.4BN USERS 40% DATA GROWTH PER YEAR 5700 TWEETS PER SECOND 8x DATA GROWTH IN 5 YRS 70+ NEW DOMAINS EVERY 60 SECONDS 600 NEW VIDEOS EVERY 60 SECONDS $1TR BY 2014 $700BN IN 2011 >1B USERS 6.7BN MOBILE SUBS IN % HANDSETS SHIPPED WITH A BROWSER Oracle Nov 2016 Slide 4

5 Logarithmic plotting! Source: Oracle Nov 2016 Slide 5

6 MySQL The view from the moon MySQL < 5.0/5.1 [pre Oracle] - MyISAM default, InnoDB optional Before Very often defaults in Linux distributions MySQL 5.5 more SQLish - First release under Oracle Dec InnoDB default (MVCC, ACID), MyISAM optional - MS Windows optimized MySQL 5.6 Replication revamp - InnoDB+, GTID, NoSQL, Performance SchemaIn Feb Used by some of the biggest Websites - Introduction of MySQL Fabric MySQL 5.7 New ways - InnoDB++, GIS Support, QRP, Instrinc Tables Oct 2015 MySQL 8.0??? - Check Labs! Two Years Cycle? - MultiThreaded Sls, MultiSourceRpl, Group Rpl - MySQL Fabric, SemiSync+ Oracle Nov 2016 Slide 6

7 MySQL 5.7 Oracle Nov 2016 Slide 7

8 MySQL 5.7 is GA (August 2015) Performance & Scalability 3 X Faster than MySQL 5.6 Enhanced InnoDB: faster online & bulk load operations Replication Improvements (incl. multisource, multi-threaded slaves...) New Optimizer Cost Model: greater user control & better query performance Manageability Native JSON Support Improved Security: safer initialization, setup & management Performance Schema Improvements MySQL SYS Schema MySQL Oracle Nov 2016 Slide 8

9 Core New JSON features in MySQL 5.7 Native JSON data type Native internal binary format for efficient processing & storage. Up to 10x faster than storing as text Built-in JSON functions Allowing you to efficiently store, search, update, and manipulate Documents Indexing of Documents using Generated Columns Automatically uses the best functional index available for even faster results JSON Comparator & New inline syntax for easy SQL integration Allows for easy integration of Document data within your SQL queries Oracle Nov 2016 Slide 9

10 JSON Functions 5.7 supports functions to CREATE, SEARCH, MODIFY and RETURN JSON values: JSON_ARRAY_APPEND() JSON_ARRAY_INSERT() JSON_ARRAY() JSON_CONTAINS_PATH() JSON_CONTAINS() JSON_DEPTH() JSON_EXTRACT() JSON_INSERT() JSON_KEYS() JSON_LENGTH() JSON_MERGE() JSON_OBJECT() JSON_QUOTE() JSON_REMOVE() JSON_REPLACE() JSON_SEARCH() JSON_SET() JSON_TYPE() JSON_UNQUOTE() JSON_VALID() Oracle Nov 2016 Slide 10

11 Agenda MySQL today MySQL 5.7 ( Fall 2015) Recap MySQL 5.7 and MySQL Cluster 7.5 ( Fall 2016) X-Protocol // Group Replication & InnoDB Cluster MySQL What s next? MySQL 8.0 Oracle Nov 2016 Slide 11

12 MySQL 5.7 [2016] - DocumentStore Oracle Nov 2016 Slide 12

13 Architecture SQL API Std Protocol CRUD API X Protocol MySQL Shell Std Protocol 3306 X Protocol X and Std Protocols MySQL Core X Protocol Plugin Plugins Memcached Plugin Oracle Nov 2016 Slide 13

14 MySQL as a Document Store a FULL Stack Store, Retrieve, Search and Managing JSON documents Native JSON Datatype, Indexes on JSON Documents Native Collections with Key Value Semantics Interactive Shell MySQL Shell Javascript, Python, SQL modes Connectors include NoSQL CRUD APIs Java, NodeJS, NET, C++/C, PHP, Python Method Chaining and Pipelining Supports Combined Document and Relational Oracle Nov 2016 Slide 14

15 MySQL Group Replication (RC) Oracle Nov 2016 Slide 15

16 Group Replication Plugin, what Is It? and it is the foundation for the MySQL InnoDB Cluster! Multi-master update anywhere replication plugin for MySQL with built-in conflict detection and resolution, automatic distributed recovery, and group membership. Group Replication library Implementation of Replicated Database State Machine theory MySQL GCS is based on Paxos (variant of Mencius) Provides virtually synchronous replication for MySQL/InnoDB 5.7+ Supported on all MySQL platforms Linux, Windows, Solaris, OSX, FreeBSD App Servers with MySQL Router MySQL Group Replication Oracle Nov 2016 Slide 16 16

17 Oracle/MySQL Group Replication Distributed as part of the official MySQL binaries from edelivery.oracle.com or support.oracle.com Starting MySQL and MySQL (next MySQL) it is available in all MySQL versions by default. Is/will be integrated with other products of the MySQL Productsuite MySQL Workbench, MySQL Backup and MySQL Monitor Easy to monitor (Performance Schema tables for group and node status/stat). Predictable roadmap and backuped by Oracle MySQL Premier Support Oracle Nov 2016 Slide 17 1

18 The Theory Behind It Implementation is based on Replicated Database State Machines Group Communication primitives resemble general properties of Databases Distributed systems meet Databases: Pedone, Guerraoui, and Schiper paper Deferred update replication: before committing locally we certify it on all nodes In order to implement it one needs Atomic Broadcast the change occurs everywhere or nowhere This is necessary to ensure data consistency across all nodes Membership Service Group Membership: it allows one to know that at a given moment in time all the members that are participating in the protocol are associated with the same logical identifier (view id) View Synchrony: ensure that messages from past views are all delivered before a new view is installed Oracle Nov 2016 Slide 18 1

19 MySQL Group Replication Multi-master update anywhere replication plugin for MySQL with built-in conflict detection and resolution, automatic distributed recovery, and group membership. App Servers with MySQL Router Active/Active update anywhere Distributed fault tolerance Automatic server fail-over Automatic reconfiguration Automatic conflict detection & resolution MySQL Shell Setup, Manage, Orchestrate MySQL Group Replication Oracle Nov 2016 Slide 19 1

20 MySQL Group Replication What does the MySQL Group Replication plugin do for the user? Removes the need for handling server fail-over. Provides fault tolerance. Enables update everywhere setups. Automates group reconfiguration (handling of crashes, failures, re-connects). Provides a highly available replicated database. Automatic distributed coordination (protects against split-brain and message loss). Less admin overhead, means more fun time! Oracle Nov 2016 Slide 20 2

21 MySQL Group Replication: Architecture Node Types R: Traffic routers/proxies: mysqlrouter, haproxy, sqlproxy,... M: mysqld nodes participating in Group Replication Oracle Nov 2016 Slide 21 2

22 Multi-Master Update Anywhere! Any two transactions on different servers can write to the same row Conflicts will automatically be detected and handled First committer wins rule UPDATE t1 SET a=3 WHERE a=1 OK M M M M M UPDATE t1 SET a=4 WHERE a=2 OK UPDATE t1 SET a=3 WHERE a=1 UPDATE t1 SET a=2 WHERE a=1 OK M M M M M Oracle Nov 2016 Slide 22 2

23 MySQL Router... Motivation Primary Master Mode by default (Single Master) (Mulit-Master possible) Do not need to know which server is the master Transparent fail-over App Router M M M Generic and versatile framework: MySQL Router. Oracle Nov 2016 Slide 23

24 MySQL InnoDB Cluster (DMR) Oracle Nov 2016 Slide 24 2

25 MySQL InnoDB Cluster: Vision A single product MySQL with high availability and scaling features baked in; providing an integrated end-to-end solution that is easy to use. Oracle Nov 2016 Slide 25 2

26 MySQL InnoDB Cluster: Goals One Product: MySQL All components created together Tested together Packaged together Flexible and Modern SQL and NoSQL together Protocol Buffers Developer friendly Easy to Use One client: MySQL Shell Easy packaging Integrated orchestration Homogenous servers Support Read/Write Scale Out Sharded clusters Federated system of N replica sets Each replica set manages a shard Oracle Nov 2016 Slide 26 2

27 MySQL InnoDB Cluster: Architecture Step 2 RC Application MySQL Connector Application MySQL Connector MySQL Router MySQL Router M HA MySQL Shell M M Group Replication Oracle Nov 2016 Slide 27

28 MySQL InnoDB Cluster: Architecture Step 3 Labs Application MySQL Connector Application MySQL Connector MySQL Router MySQL Router M HA MySQL Shell M M Group Replication S1 S2 S3 S4 S Read-Only Slaves Oracle Nov 2016 Slide 28

29 ReplicaSet 1 ReplicaSet 2 ReplicaSet 3 MySQL InnoDB Cluster: Architecture Step 4 Labs Application MySQL Connector Application MySQL Connector Application MySQL Connector Application MySQL Connector MySQL Router MySQL Router MySQL Router MySQL Router M HA M HA M HA MySQL Shell M M M M M M S1 S2 S3 S4 S S1 S2 S3 S4 S S1 S2 S3 S4 S Oracle Nov 2016 Slide 29

30 MySQL InnoDB Cluster: High Level Architecture Application Application Application Application MySQL Connector MySQL Connector MySQL Connector MySQL Connector MySQL Router MySQL Router MySQL Router MySQL Router MySQL Shell MySQL InnoDB cluster MySQL Enterprise Monitor Oracle Nov 2016 Slide 30

31 MySQL Cluster 7.5 Oracle Nov 2016 Slide 31 3

32 MySQL NDB Cluster Overview REAL-TIME HIGH SCALE, READS + WRITES FULLY ELASTIC % AVAILABILITY SQL + NoSQL LOW TCO Memory optimized tables with durability Predictable Low-Latency, Bounded Access Time Auto-Sharding, Active-Active ACID Compliant, OLTP + Real-Time Analytics Add and remove storage and performance capacity in seconds Fully cloud native Active-Active, Shared nothing, no Single Point of Failure Self Healing + On-Line Operations Key/Value + Complex, Relational Queries SQL + Memcached + JavaScript + Java + HTTP/REST & C++ Open Source + Commercial Editions, Easy to use and deploy Commodity hardware + Management, Monitoring Tools Oracle Nov 2016 Slide 32 3

33 MySQL Cluster 7.5 RC Node Sizes of 128TB Read Optimized Tables Global Tables Capacity and Scale Out MySQL 5.7 JSON Data Type Generated Columns Records-Per-Key Optimization Improved SQL Improved Reporting Improved Logging Improved Visibility Improved Debugging Improved Restore Management Oracle Nov 2016 Slide 33 3

34 Oracle Nov 2016 Slide 34 Confidential Oracle 3 MySQL Cluster 7.5 NDB Cluster powering the world s fastest and most scalable Hadoop filesystem. Scaling Hadoop 20x!

35 MySQL Cluster 7.5 Based on MySQL 5.7 Native JSON datatype Generated Columns Function Based Indexes Cost Based Optimizer New Spatial engine New hint syntax with additional hints Memory instrumentation Secure by default Much more! NEW! Oracle Nov 2016 Slide 35 Copyright 2016, oracle 3

36 MySQL Cluster 7.5 Continue Improving Supportability Overload controls via thread idle wait time Improve event buffer memory allocation Better and more descriptive error messages Dump command improvements Improved Lock reporting Continue Improving Performance Faster backup & restore NEW! Oracle Nov 2016 Slide 36 Copyright 2016, oracle 3

37 MySQL Cluster 7.5 Continue Improving Availability Convert ndb_binlog_index to ACID storage Make use of dynamic columns everywhere Support for online defragmentation and space reclamation Support for row transformations encryption and compression Continue Improving Quality Refactor binlog injector in order to address many known issues Increase the test base More system correctness and fault injection tests NEW! Oracle Nov 2016 Slide 37 Copyright 2016, oracle 3

38 MySQL Cluster 7.5 NEW! Continue Improving Capacity Enable individual fragments larger than 16GB (now up to 128TB) Improved Read Scale-Out The introduction of read from backup for tables Localized reads on every node in a node group The introduction of global tables (fully replicated) Localized reads on every node in the Cluster Support for Increased Redundancy Support for up to 4 nodes per node group (1-3 replicas per shard) Oracle Nov 2016 Slide 38

39 How Do The Two Compare? MySQL Group Replication MySQL NDB Cluster Storage Engine InnoDB NDBCLUSTER Distributed Architecture Shared nothing Shared nothing Consistency Model Weak Consistency Strong Consistency Sharding No Yes Arbitration No Yes Load Balancing No Yes NoSQL APIs MySQL Document Store Native NDB API Operational Complexity Medium High Administration Standard (MySQL) Custom (MySQL + NDB) Oracle Nov 2016 Slide 39

40 Agenda MySQL today MySQL 5.7 ( Fall 2015) Recap MySQL 5.7 and MySQL Cluster 7.5 ( Fall 2016) X-Protocol // Group Replication & InnoDB Cluster MySQL What s next? MySQL 8.0 Oracle Nov 2016 Slide 40

41 MySQL 8.0 Oracle Nov 2016 Slide 41 4

42 MySQL 8.0 (DMR), a new star is born Labs MySQL 8.0: Native Data Dictionary No FRM/DB.OPT/TRG/TRN/PAR - MyISAM not required User Management DDLs Atomic ( 100% InnoDB) Security Roles Support for the latest Unicode 9.0 UTF8MB4 as the default character set Performance Schema Indexes Implementation of indexes tricks the optimizer into better execution plan Oracle Nov 2016 Slide 42

43 MySQL Enterprise Edition

44 The Oracle MySQL business Licensing Support Tools Cloud Oracle Nov 2016 Slide 44

45 MySQL Enterprise Edition - Tools Clients and Applications Online Backup NoSQL Simple access patterns SQL Complex queries with joins Firewall Monitoring Integration Compromise on consistency for performance Ad-hoc data format Simple operation ACID transactions Well defined schemas Rich set of tools Audit Encryption/TDE 3 rd Party Tools 3 rd Party Tools 3 rd Party Tools 3 rd Party Tools 3 rd Party Tools Support InnoDB Storage Engine Authentication mysqld process * All Plugins and Tools are free as part of the Support Subscription Oracle Nov 2016 Slide 45 Copyright 2015, Oracle 4

46 MySQL Enterprise and Cloud Service Support Largest MySQL engineering and support organization Backed by the MySQL developers World-class support, in 29 languages Hot fixes & maintenance releases 24x7x365 Unlimited incidents Consultative support Global scale and reach Oracle Nov 2016 Slide 46 46

47 Oracle MySQL there s much more ( Oracle Premier Support 24x7 Support Unlimited Support Incidents Knowledge Base Maintenance Releases MySQL Consultative Support MySQL Community Edition MySQL Cloud Service MySQL Features MySQL Database Server MySQL Connectors MySQL Replication MySQL Fabric MySQL Router MySQL Partitioning MySQL Utilities MySQL Workbench Storage Engine: MyISAM Storage Engine: InnoDB Storage Engine: NDB Oracle Product Certifications Certified with Oracle Linux Certified with Oracle VM Certified with Oracle Solaris Certified with Oracle Enterprise Manager Certified with Oracle GoldenGate Certified with Oracle Data Integrator Certified with Oracle Fusion Middleware Certified with Oracle Secure Backup Certified with Oracle Audit Vault Certified with Oracle Database Firewall MySQL Enterprise & Security Tools Oracle Enterprise Manager for MySQL MySQL Enterprise Monitor MySQL Enterprise Dashboard MySQL Enterprise Advisors MySQL Query Analyzer MySQL Replication Monitor MySQL Enterprise Backup Hot backup for InnoDB Full, Incremental, Partial, Optimistic Backups Full, Partial, Selective, Hot Selective restore Encryption and Compression Point-In-Time-Recovery MySQL Enterprise Authentication MySQL Enterprise TDE MySQL Enterprise Encryption MySQL Enterprise Firewall MySQL Enterprise Audit MySQL Enterprise Scalability Thread Pool MySQL Enterprise High-Availability HA using Oracle Clusterware HA using Solaris Clustering MySQL Community Edition MySQL Cloud Service Oracle Nov 2016 Slide 47 4

48 The Winning Combination + Simple, Integrated and Enterprise Ready Oracle MySQL Cloud Service enables you to easily deploy modern applications powered by the World s Most Popular Open Source Database = Integrated with Oracle Cloud Services for quick development and deployment Based on Oracle s proven MySQL Enterprise Edition for performance, security & uptime, all MySQL Enterprise Tools, Plug-Ins, Certification and Support are included Oracle Nov 2016 Slide 48 4

49 Agenda MySQL within Oracle Overview of MySQL architecture MySQL Replication and MySQL Fabric Commercial MySQL vs. Open Source MySQL Subscription / MySQL License A MySQL Reference Questions and Answers Oracle Nov 2016 Slide 49 4