Five Common Myths About Scaling MySQL

Transcription

1 WHITE PAPER Five Common Myths About Scaling MySQL

2 Five Common Myths About Scaling MySQL In this age of data driven applications, the ability to rapidly store, retrieve and process data is incredibly important. A leading cause of application failures is the inability to effectively architect a data tier that is capable of meeting application requirements. Let s face it; scaling the data tier is hard! MySQL is arguably the most widely used relational database and many well-known companies like Facebook, Google and Twitter use MySQL or one of its derivatives like MariaDB or Amazon s RDS. But there are many myths and misconceptions about MySQL that make it difficult for someone to choose the right database technology for their application. As a result many companies end up adopting the wrong one, which can be catastrophic for their applications. Here are five leading myths about MySQL and relational databases in general. MySQL cannot scale-out; NoSQL is the only viable scale-out database solution that provides high availability MySQL can t scale cost-effectively in the cloud MySQL can t ingest data at high rates; you need NoSQL for that MySQL cannot handle large volumes of data, especially with queries like joins and aggregations I cannot use MySQL because I need a specialized database for each use-case in my application; NoSQL for data ingest, Hadoop for analytics, White Paper Five Common Myths About Scaling MySQL 2

3 Myth #1: MySQL cannot scale-out; NoSQL is a scale-out database solution While it is entirely true that MySQL is not natively a scale-out database, nothing prevents it from being used in a scale-out configuration. Applications that implement techniques like sharding operate MySQL in a scale-out configuration very effectively. Some very large scale-out MySQL configurations include the ones used by box.com, Pinterest and Etsy. Transparent database virtualization is another example of a technology that provides applications with all the benefits of elastic MySQL scale-out without the disruptions and cost of sharding. The CAP theorem is often cited as the reason why traditional relational databases like MySQL cannot be both scale-out and HA at the same time, but this is a misinterpretation of what the CAP theorem actually states. In reality, the CAP theorem applies equally to NoSQL databases as it does to MySQL, and it is perfectly possible for a MySQL based database system to be both scale-out and Highly Available at the same time. NoSQL solutions provide less functionality than traditional relational databases; for example there is no notion of a foreign key, nor the ability to perform a join. Those limitations make it simpler for NoSQL solutions to scale-out in a highly available manner. It is possible, however, to get these advantages without giving up these capabilities entirely. A sharded MySQL database can allow you to partition data in much the same way a NoSQL database would. On the other hand, database virtualization technologies provide all of the standard database capabilities including referential integrity, schema normalization and joins across partitions while still providing both scale-out and high availability for access within a partition. White Paper Five Common Myths About Scaling MySQL 3

4 Myth #2: MySQL can t scale cost-effectively in the cloud Traditionally relational databases are monolithic and built for scale-up. In other words, a more powerful database requires more powerful hardware. Unfortunately, hardware that is twice as expensive does not necessarily provide twice the database power. With the limitations on the sizes of hardware in the cloud, and limitations on the capabilities of servers available in the cloud, it is certainly the case that a single MySQL server instance may not be cost effective approach. However, MySQL databases can in fact be scaled-out effectively in the cloud using techniques such as replication, sharding or database virtualization. With these techniques, one can provision several inexpensive MySQL instances in the cloud to provide cost effective scaling and, in the case of database virtualization, make them perform together as a single logical database server of enormous size. Myth #3: MySQL can t ingest data at high rates; you need NoSQL for that MySQL database servers are very effective at both storing and retrieving data, and they do this while providing standard ACID (Atomicity, Consistency, Isolation and Durability) guarantees. Some NoSQL technology vendors have been responsible for the perpetration of the myth that the inability to scale writes or high throughput ingestion is a fundamental limitation of relational databases and that the only way to solve this is with a NoSQL solution. This is entirely false. Databases that respect ACID properties will naturally incur higher costs to store data than those that risk data loss by relaxing transactional guarantees. The same technique of scaling out writes across a number of servers can be applied to MySQL as it is to NoSQL databases, however. With database virtualization technologies sustained data ingestion rates in excess of 1,000,000 rows per second have been demonstrated using standard MySQL in the public cloud. There is absolutely no reason therefore to believe that one must necessarily ingest data into a NoSQL solution and live with all of the inherent limitations of those products! White Paper Five Common Myths About Scaling MySQL 4

5 Myth #4: MySQL cannot handle large volumes of data, especially with queries like joins and aggregations; I must use Hadoop! This particular myth is a bit more difficult to dispel than the ones above. In the cases above, application developers could achieve high degrees of performance by scaling out their usage of MySQL, even if it comes with some limitations. It is generally much more difficult, however, for them to achieve scalable query performance across extremely large data sets. The application can t simply use SQL to filter and process data to find the appropriate subset of data to match some criteria. If an application must do this, it will often require procedural code to be written to perform data aggregation, correlation and other forms of data processing. This is done in the application s memory based on intermediate results returned by a number of database servers. This code can be much more complex and hard to maintain than using a declarative like SQL. Of course this is exactly what Hadoop presents. While it provides a framework to make structuring this kind of processing easier, it still requires an application developer to build mappers and reducers to perform data manipulation. Other NoSQL solutions don t help here either. Because the data models presented by key-value or document stores partitions the data, there is really no provision to perform complex queries across the entire dataset for reporting or analysis purposes. There is a solution, however! The path is based on the application of the classic Massively Parallel Processing (MPP) database architecture that has been widely deployed in solutions such as Netezza, Teradata and other parallel database products. By using database virtualization, a collection of standard relational database servers can be addressed as a single MPP database. Using this technique, MySQL is perfectly capable of handling very large tables and queries against very large tables of data. Data can be transparently distributed across a collection of MySQL servers with queries being processed in parallel to achieve linear performance across extremely large data sets. Some of the world s largest databases follow the MPP paradigm. The scale-out capabilities are now available to MySQL through the use of database virtualization technologies that dramatically improve the utility of MySQL for very large and complex databases. These systems have been demonstrated to show linear performance improvements for most operations, and better than linear performance improvements for some operations like joins and aggregations. White Paper Five Common Myths About Scaling MySQL 5

6 Myth #5: I cannot use MySQL because I need a specialized database for each use-case in my application; NoSQL for data ingest, Hadoop for analytics, Falling victim to many of the preceding myths, some companies have ended up with complex pipelines where data is first ingested into one database technology and then progressively moved to other database technologies for specific processing. In one particularly unfortunate case, a company ingested data into a NoSQL database, then moved it into a number of MySQL databases, each used for one specific part of the application before finally moving it into an analytics database technology for more complex query processing. Each technology has its own special purpose API s, data representation, administrative tools, required skills and general quirks. Some have no formal query language while others supported SQL. As shown above, however, there is no need to live this horrible nightmare. You can ingest data directly into a collection of MySQL databases. Data is loaded using standard SQL statements. The schema can be normalized and data anomalies can be minimized. Queries are processed against the data using standard SQL. As data volumes grow, additional database servers can be added to the system without interruption and disruption. Conclusion There is an enormous amount of misinformation in the marketplace extoling the virtues of NoSQL and NewSQL solutions, and casting doubt on the power, flexibility and capabilities of traditional relational databases like MySQL. Before adopting a non-standard, highly proprietary approach from one of these solutions consider your business needs and if a relational database like MySQL is the right approach. Many of the supposed shortcomings of MySQL are due to the fact that it does not inherently support scale out. While many well known companies have been very successful scaling out the data tier on top of MySQL using techniques such as sharding and replication, this can be a challenging task for smaller organizations. It also typically still comes with the same limitations that are inherent to distributed NoSQL systems, namely that each slice of the data is treated independently for queries and transactions. White Paper Five Common Myths About Scaling MySQL 6

7 Scaling MySQL with Tesora The Tesora Database Virtualization Engine provides a transparent interface that makes a number of unmodified MySQL servers appear as a single server to the application and dramatically simplifies the task of assembling a scale-out data tier. Tesora distributes the data in the application across a number of standard MySQL servers without the pain of manual sharding and also allowing this distributed data set to be addressed as a single coherent unit. In addition, the system supports standard database semantics like foreign keys, joins, indexes, high availability, replication that are standard capabilities of the MySQL database. Key benefits of this approach include: Linear scalability for reads and writes, with demonstrated insert speeds over a million records per second. No need for changes in the application code No compromises on ACID transactional guarantees The power of the standard SQL query language (yes, I know that s redundant.:) The ability to use standard tools and experience in managing MySQL deployments In short, Tesora s Database Virtualization Engine gives applications the same scale-out and horizontal scalability benefits of NoSQL without giving up all the benefits associated with traditional MySQL. With Tesora, there is simply no good reason to discount the traditional MySQL database for your high volume web application. White Paper Five Common Myths About Scaling MySQL 7

8 For more information about Tesora, please contact us at 125 CambridgePark Drive, Suite 400 Cambridge, MA ext Tesora Corporation. This document is provided for informational purposes only, and the contents hereof are subject to change without notice. While reasonable efforts have been made to ensure correctness, this document is not warranted to be error-free, nor subject to any other warranties or conditions, whether expressed orally or implied in law, including implied warranties and conditions of merchantability or fitness for a particular purpose. We specifically disclaim any liability with respect to this document, and no contractual obligations are formed either directly or indirectly by this document. This document may not be reproduced or transmitted in any form or by any means, electronic or mechanical, for any purpose, without our prior written permission. The Tesora logo is a trademark of Tesora Corporation. Other names may be trademarks of their respective owners.