Architecture and Design of MySQL Powered Applications. Peter Zaitsev CEO, Percona Highload Moscow, Russia 31 Oct 2014

Transcription

1 Architecture and Design of MySQL Powered Applications Peter Zaitsev CEO, Percona Highload Moscow, Russia 31 Oct 2014

2 About Percona 2 Open Source Software for MySQL Ecosystem Percona Server Percona XtraDB Cluster (PXC) Percona Xtrabackup Percona Toolkit Solutions to succeed with MySQL Consulting Support Managed Services

3 About Presentation 3 What's in your Toolbox? What Questions you should ask? Common MySQL Architecture Patterns

4 LiveJournal Architecture 10 Years Ago 4

5 10 Years ago we see 5 Load Balancers FrontEnd vs BackEnd Web Server Separation MemcacheD (Caching) Global Database for Shared Data Read Write Splitting Sharding Database Clusters

6 History Lesson 6 MySQL is quite Mature Battle tested Architecture Patterns Most Problems have proven solutions

7 7 Well certain things have changed YOUR TOOLBOX NOW

8 Very Powerful Hardware 8 Lots of Memory Very Fast Storage Many CPU Cores 16GB to 512GB 1000 IOPS to IOPS 10ms latency to 0.1ms IO Latency 4 Cores to 32 Cores

9 Reasonable Software 9 Can use this Hardware Pretty Well MySQL Linux Kernel File Systems Etc..

10 Cloud 10 Dynamic Scalability Throw Away Server Mentality Agility Less involved Operations

11 Automation 11 Development Testing Deployment Operations

12 New Languages and Frameworks 12 More Agile Less system code More Client Side Development Less Database Dependence ORM

13 Multiple Clients 13 Desktop Web Browsers Mobile Web Browsers Apps API

14 Not Only MySQL 14 Right Tools for Right Job! Memcache, Redis RabbitMQ Cassandra, MongoDB Hadoop Column Store Engines, Vertica

15 More HA Options 15 Mature MySQL Replication Percona XtraDB Cluster, Galera MySQL Cluster Proprietary cloud HA

16 Ready solutions for Replication Management 16 MHA PRM Continuent Tungsten

17 Making Sharding Better 17 Not Quite MySQL Proxy Solutions OpenSource Frameworks Clustrix MemSQL ScaleDB ScaleArc, ScaleBase, Tesora MySQL Proxy, MaxScale, ProxySQL Vitess JetPants MySQL Fabric

18 ARCHITECTURE QUESTIONS 18

19 Basics 19 Make some Right decisions Early Your first Architecture will not be perfect Some poor choices are very expensive to fix

20 Consultants are like Lawyers 20 You can waste a lot of money with them Getting right advice on the right topics will save you a lot of trouble in the end

21 Dimensions to consider 21 Application Scale High Availability Team Experience

22 Approach to Scaling 22 Avoid Over- Engineering Give yourself some runway Do Some Capacity Planning

23 High Availability 23 Real High Availability is Really Expensive Medium Availability is getting more affordable People are often leading cause of downtime

24 Team Experience 24 Chose Solutions which your Team can Support Often Developers know little about Databases Right Development, Operational Process Is a Key Tools to Support your Process Percona Toolkit, Percona Cloud Tools Systematically find and review new queries

25 Simplicity 25 Use fewer components when possible Each components needs experience to There is safety in numbers Less complexity lower costs Monitor Troubleshoot Follow the Crowd Unless you have a reason not to

26 World of Small Applications 26 Majority of Applications are Small May well run on single database instance Such Small Applications can be significant Really Especially with support of caching Active intranet site for company with 200K employees Ecommerce site supporting $10M+/month in Sales

27 Lets do Some Math 27 Modern MySQL Instance Handle 100K QPS 20 Queries per User Interaction 5K User Interactions Per Second 144M User Interactions a Day (Considering 3x peak to average) 10M Daily Active users (with Medium Engagement)

28 28 Basic Architecture Patterns PRACTICAL CHOICES

29 Baseline 29 Single MySQL Instance No Caching No HA No Supplemental Technologies

30 Simple High Availability 30 HA with DBaaS (Amazon RDS) HA with MySQL Replication HA with Percona XtraDB Cluster

31 What do you need to Scale? 31 Reads Data Size Writes

32 Load Management Tricks 32 Move the load In Time (Doing Batch Processing at Night) In Space (Use Slave for Reports)

33 Queueing 33 Used by Most high volume architectures Deal with Load Spikes Increase Reliability Scale parts of the system which need to Many solutions RabbitMQ Redis Gearman

34 Scaling Reads 34 Replication (and Read Write Splitting) Caching (Memcache, Redis etc) Percona XtraDB Cluster (Read Anywhere)

35 Scaling Writes 35 Functional Partitioning Using Different Servers for Different Parts of Application Sharding Horizontal Partitioning over many servers

36 Scaling Data Size 36 How Much We Can Practically Store in Single MySQL Instance these days Depends on the Application and Needs 10TB good rule of thumb

37 How Is it Possible 37 Online Schema Change Fast Backup Fast Network High Performance Storage Compression MySQL 5.6 Percona Xtrabackup 10Gbps affordable Flash/SSD Innodb Pt-online-schemachange (Percona Toolkit) MySQL Enterprise Backup TokuDB Modern Filesystem/Volume Managers

38 Summary 38 MySQL is Mature. Solutions Available for Many Problems Make few choices right in the beginning Hardware and Software advances allow to go a long way with simple architectures

39 Thank You! Peter Zaitsev 39