Maximizing MySQL Availability, Data Integrity, Performance, and Scalability

Transcription

1 Maximizing MySQL Availability, Data Integrity, Performance, and Scalability Dr John R Busch Founder and CTO Schooner Information Technology JohnBusch@SchoonerInfoTechcom 2011 Schooner Information Technology, p1 Schooner Information Technology October 25, 2011

2 Maximizing MySQL Availability, Data Integrity, Performance, and Scalability Abstract: First, we discuss the use of synchronous replication of InnoDB to eliminate data loss and to provide cluster-wide data consistency We then discuss instant automatic fail-over providing five 9s to maximize service availability Next we discuss the use of high parallelism and granular concurrency control with synchronous replication to fully exploit commodity servers and storage to maximize performance Next we discuss extending base synchronous clusters with tightly-coupled, parallel asynchronous replication to achieve auto-failover and high consistency across geographically dispersed data centers Bio: Dr John R Busch is Founder and CTO of Schooner Information technology, focusing on high performance and highly available databases for high volume web facing, cloud, and enterprise services Prior to Schooner, John was director of computer system research at Sun Microsystems Laboratories, VP of Engineering with Diba, founder of Clarity Software, and R&D director of computer systems at Hewlett Packard John holds a PhD in Computer Science from UCLA, MS in Mathematics from UCLA, MS in Computer Science from Stanford University, and attended the Stanford Sloan program 2011 Schooner Information Technology, p2

3 Business Most Valuable Asset: Its Data Data Most important and valuable component of modern applications and websites Driving revolutionary changes in computing and the internet Ø New opportunities for generating revenue Ø More efficient use of current business processes and infrastructure Data access downtime or poor performance has a major cost to a business bottom line 2011 Schooner Information Technology, p3

4 The Mission-Critical Imperative Let me tell you the difference between Facebook and everyone else, we don't crash EVER! If our service is down for even a minute, our entire reputation is irreversibly destroyed Facebook and Google invest hundreds of millions of dollars every year on custom software and hardware infrastructure to optimize availability, performance, administration, and cost 2011 Schooner Information Technology, p4

5 The Mission-Critical Imperative Providing high data availability, excellent response time is critical for key classes of businesses Ø Web 20 Ø ecommerce Ø High-volume websites Ø Telecommunications They require a mission critical database 2011 Schooner Information Technology, p5

6 Mission-Critical Database Requirements High Availability High Performance and Scalability Simple and Powerful Administra<on Data Integrity Cost Effec<ve Standards and Compa<bility 2011 Schooner Information Technology, p6

7 Mission-Critical Database Goals and Metrics High Availability Goals High Data Integrity Metrics Service unavailability (minutes/year) from failures, disaster recovery, or during planned administration Probability of data loss or corruption; data consistency levels High Performance and Scalability Transaction throughput, response time; performance scalability; performance stability Simple and powerful administration Ease of cluster administration; fail-over automation; monitoring and optimization tools Cost effective Total cost of ownership (TCO); return on investment (ROI) Standards and Compatibility Level of standards compliance and certification 2011 Schooner Information Technology, p7

8 Loosely-Coupled Asynchronous and Semi-Synchronous Replication Read version based on tx=50 Master mysqld Slave mysqld TxCommit(101) Replapply(51) tx=101 tx=101 tx=51 tx=51 DB InnoDB Tx log MySQL Bin log Log events pulled by Slave Relay log InnoDB Tx log DB Last tx=100 Last tx=100 Last tx=70 (asynch Last tx=100 (synch) Last tx=50 Example Products : MySQL Enterprise 51 Asynchronous and 55/56 Semi-Synchronous Replication 2011 Schooner Information Technology, p8

9 Loosely-Coupled Asynchronous and Semi-Synchronous Replication Master mysqld Read version based on tx=50 Inconsistent (Stale) Data Slave mysqld DB TxCommit(101) InnoDB Tx log MySQL Bin log Last tx=100 Last tx=100 Slow Recovery Complex Management Slow ExecuCon Log events pulled by Slave Relay log Last tx=70 (asynch Last tx=100 (synch) Replapply(51) tx=101 tx=101 tx=51 tx=51 PotenCal Data Loss InnoDB Tx log Last tx=50 DB Example Products : MySQL Enterprise 51 Asynchronous and 55/56 Semi-Synchronous Replication 2011 Schooner Information Technology, p9

10 Loosely-Coupled Asynchronous and Semi-Synchronous Replication Limited Service Availability Master fail-over, re-synch of slaves Limited Data Integrity Lost data; inconsistent Data Limited Performance and Utilization Low throughput and low utilization Complex Administration Manual processes, slave re-synch High Cost of Ownership High capital expense from server sprawl Increased operating expense from power, space, admin Reduced revenue and customer satisfaction from service downtime 2011 Schooner Information Technology, p10

11 Tight Coupling and Fully Synchronous Replication Cluster Admin MySQL clients MySQL clients SchoonerSQL (Master) Concurrently Execu<ng Transac<ons Op<mized Parallel Execu<on Threads in SchoonerSQL MySQL Parallel SchonerSQL Replica<on Threads SchoonerSQL (Read Master) Concurrently Execu<ng Transac<ons Op<mized Parallel Execu<on Threads in SchoonerSQL Parallel SchoonerSQL Replica<on Threads InnoDB InnoDB Red Hat, CentOS Linux Standard X86 Server Red Hat, CentOS Linux Standard X86 Server Parallel Synchronous Replica<on During Transac<on Execu<on Slaves in lock-step with Master At master transaction commit, all Slaves guaranteed to have received and committed the changes 2011 Schooner Information Technology, p11

12 Tight-Coupling and Synchronous Replication No Data Loss SchoonerSQL Master Cluster- Wide Consistent Reads Log for tx=100 pushed to Slave SchoonerSQL ReadMaster TxCommit(101) Slave ACK for tx=100 Replapply(100) tx=101 Eliminates Service InterrupCons Fast, Transparent Fail- Over (VIPs) Fast incremental recovery tx=101 Easy Management DB InnoDB Tx log High Performance High UClizaCon InnoDB Tx log DB Last tx=100 Last tx=100 Last tx=100 Last tx= Schooner Information Technology, p12

13 Tight Coupling and Synchronous Replication with Transparent Automatic Fail-over Improves Service Availability by 90% Tightly-coupled MySQL synchronous replication with fast, automated application transparent fail-over and can provide much higher service availability than that achievable with asynchronous or semi-synchronous replication Availability Improvement from Synchronous ReplicaCon MySQL 55 (% CumulaCve Down Time ReducCon) SchoonerSQL Auto- Failover Schooner SQL On- Line Upgrades 2011 Schooner Information Technology, p13

14 Tight Coupling and Synchronous Replication Provides Much Higher Performance Throughput per Server Synchronous TransacCon Throughput per Server can be Much greater Than Asynchronous or Semi- Synchronous (with hard disc drives (HDDs)) Transac<on Throughput with Hard Drives (ktpm) MySQL 55 asynch Measurement Configuration 2 node Master-Slave configuration 2 socket Westmere 72GB DRAM MySQL 55 semi- synch SchoonerSQL DBT2 open-source OLTP version of TPC-C 1000 warehouses, 32 connections 0 think-time Result metric: TPM (new order) 2011 Schooner Information Technology, p14

15 Tight Coupling and Synchronous Replication Enables Vertical Scaling with Commodity Flash Memory, Cores DBT2 open-source OLTP version of TPC-C 1000 warehouses, 32 connections 0 think-time Result metric: TPM (new order) Measurement Configuration 2 node Master-Slave configuration 2 socket Westmere 72GB DRAM TransacCon Throughput with Flash Drives TransacCon Throughput with Hard Disc Drives MySQL 55 asynch MySQL 55 semi- synch SchoonerSQL 20 0 MySQL 55 asynch MySQL 55 semi- synch SchoonerSQL 2011 Schooner Information Technology, p15

16 Tight Coupling and Synchronous Replication Lowers Response Times Response Time (ms) 7 Performance : Transac<on Response Time MySQL 55 asynch MySQL 55 semi- synch SchoonerSQL 2011 Schooner Information Technology, p16

17 Tight Coupling and Synchronous Replication Provides High Performance Stability MySQL 55 Asynchronous MySQL 55 Semi-synchronous SchoonerSQL 2011 Schooner Information Technology, p17

18 Tight Coupling and Synchronous Replication Lowers Total Cost of Ownership Lower Cost Reduced capital and operating costs through reduction in servers, power, space, admin Savings from increased service availability and associated revenue and customer retention Total Cost of Ownership (rela<ve) 0 MySQL 55 SchoonerSQL TCO and ROI models are customer and workload specific Function (throughput/server; server, rack, and network costs, software license and support costs, admin costs; space and power costs; cost of downtime) 2011 Schooner Information Technology, p18

19 Tight Coupling and Synchronous Replication with Transpaent Automated Failover and Admin GUI Greatly Simplifies Administration Fail-over can be completely automatic and instant requiring no administrator intervention or service interruption Cluster Administrator GUI and CLI can provide a single point for cluster-wide management single click slave creation and database migration 2011 Schooner Information Technology, p19

20 Powerful Administration : Monitoring, Trouble-shooting, Tuning, Alerts 2011 Schooner Information Technology, p20

21 Extending Synchronous Replication with Parallel Asynch and Auto-Failover in WAN Geographic Distribution/DR WAN/geographically dispersed data centers Requires Asynchronous replication Can t add additional ~100ms with high potential variance to query response time for synchronous replication Data Integrity Requirement : Remote consistency lag and recovery time should be ~ WAN latency Maximize WAN data consistency Minimize disaster recovery time Requires high performance asynchronous replication Must have multi-threaded asynchronous parallelizing updates 2011 Schooner Information Technology, p21

22 Extending Synchronous Replication with Parallel Asynch and Auto-Failover in WAN Geographic Distribution/DR HA Requirement: Must automatically fail-over when synchronous master fails-over or asynch slave fail-over occurs, requires WAN asynchronous replication tight coupling with synch replication group 2011 Schooner Information Technology, p22

23 Unlimited Query Scaling SchoonerSQL Throughput (sysbench OLTP) Requests/s Reads/s Writes/s Nodes in Cluster Query Scaling in a Synchronous Replication Group Fully replicated Master/Slave cluster No cluster overhead for adding queries to a slave Can add synchronous query read master nodes linearly Unlimited query scaling extension with parallel asynchronous replication (no data loss, auto-failover, high data integrity) With shared nothing architectures, there is potentially severe cross node query degradation 2011 Schooner Information Technology, p23

24 Scaling Updates Database Update Scalability Vertically scale with commodity : flash memory, more cores, higher frequency Schooner Synch Cluster: Hard Drives Schooner Synch Cluster : Flash Compelling option exploiting low cost, high performance commodity technology 2011 Schooner Information Technology, p24

25 Unlimited Update Scaling : Optimized Transparent Sharding Database Update Scalability After Optimal Vertical Scaling: Unlimited Query Scaling : Horizontally Scale Through Transparent Workload Aware Database Partitioning and Replication Database workload aware Administrator analysis and configuration tools Allows optimal layout, including replicated structures, for query data access optimization Application Transparent High performance, highly available dynamic query execution across shards 2011 Schooner Information Technology, p25

26 MySQL Database Alternatives for Mission-Critical Deployments MySQL 55 MySQL NDB Cluster Clustrix Linux DRBD ConCnuent Tungsten Golden Gate Schooner SQL Fail- Over Down<me Minutes- hours seconds seconds minutes seconds Minutes- hours seconds Automated Fail- over No Yes Yes No Yes No Yes Data Loss Yes No yes No Yes Yes No Data Consistency No Yes Yes No No No Yes Performance Med Med/Low Med/Low Med Low Low High Scalability Low Med/Low Med/Low Low Low Low High Ease of Management WAN perf and auto- fail- over 2011 Schooner Information Technology, p26 Low High High Low Med Med High No No No No No No Yes InnoDB Compa<ble High Med/Low Med/Low High High High High Custom Hardware No No Yes No No No No Cost (TCO) High Med High High High High Low Heterogeneous Databases No No No No Yes Yes No

27 SchoonerSQL Advantage Highest Availability No service interrup<on for planned or unplanned database down<me Instant automa<c fail- over On- line upgrade and migra<on 90% less down<me vs MySQL 55 Full WAN support with master auto- failover Highest Performance and Scalability 4-20x more throughput/server vs MySQL 55 High performance synchronous and asynchronous replica<on Compelling Economics Cut server capex (consolida<on) Cut opex (power, pipe, DBA <me) Increase revenue (eliminate service interrup<ons) TCO 70% cheaper than MySQL % MySQL Enterprise InnoDB CompaCble Highest Data Integrity No lost data Cluster- wide data consistency Visibility and Control Easy cluster administra<on No error- prone manual processes Monitoring and Op<miza<on Out- of- the- box Product Full MySQL + InnoDB: not a toolkit Free your staff to build your business, not a custom database Broad Industry Deployment ecommerce, Social Media, Telco, Financial Services, Education High volume web sites Geographically distributed websites! 2011 Schooner Information Technology, p27

28 Evaluating the Options and Trade-offs for Your Data Center? Let Schooner Help! Try SchoonerSQL! We guarantee SchoonerSQL will increase your Availability, Performance, and Scalability Contact us: Schooner Information Technology, Inc 501 Macara Avenue, Suite 101 Sunnyvale, CA USA Tel: wwwschoonerinfotechcom Schooner Information Technology, p28

29 Thank You! 2011 Schooner Information Technology, p29