Accelerate MySQL for Demanding OLAP and OLTP Use Cases with Apache Ignite. Peter Zaitsev, Denis Magda Santa Clara, California April 25th, 2017

Transcription

1 Accelerate MySQL for Demanding OLAP and OLTP Use Cases with Apache Ignite Peter Zaitsev, Denis Magda Santa Clara, California April 25th, 2017

2 About the Presentation Problems Existing Solutions Denis Magda will show the power of Apache Ignite

3 My Conviction There is no silver bullet in technology!

4 Why? All design decisions comes with their own benefits and drawbacks

5 Technologies not Technology Large Scale applications tend to use more than one technology on data layer

6 Works especially well with Open Source! Additional Components do not require hefty license fees Easy to prototype and test out Open Source Community is good at building bridges

7 Balance is Needed Use as many technologies as you need, but no more

8 MySQL MySQL is no Exception. It is not Great for Everything.

9 Some of the Problems Hot Data Highly Volatile Data Large Data Volume Analytical Processing Full Text Search

10 Hot Data For example Cache Large volume of simple requests High overhead due to SQL No good Memory focused Engine Not Designed for very high Concurrency

11 Solutions MySQL MemcacheD interface Thread Pool External MemcacheD Redis

12 Highly Volatile Data Lots of updates, especially to a single row Design around full Transactional ACID semantics Disk Log based durability Pessimistic Logging

13 Solutions MySQL Data Design Configuration Tuning Parallel Replication External MemcacheD Redis

14 Large Data Volume MySQL is designed as single node system Limited in CPU, Memory Manual Sharding solutions are painful Especially with complex queries

15 Solutions MySQL Manual Sharding Vitess ProxySQL External Shading for MemcacheD and Redis MongoDB Cassandra

16 Analytics (OLAP) MySQL does not support column based storage MySQL optimizer is limited for complex queries MySQL does not do parallel query execution MySQL does not do distributed query execution

17 Solutions MySQL Configuration and Schema Design (Limited) External Hadoop & Spark Vertica ClickHouse

18 Full Text Search Can handle basic Full Text Search Does not scale well with data volume No parallel processing Limited search relevance options Hard To do GIS searches; Facets No language processing

19 Solutions MySQL Small Scale search applications only Supported with Innodb tables since MySQL 5.6 External Elastic Solr Sphinx

20 New Solutions constantly appear Always be on lookout for a better solutions!

21 Apache Ignite - In-Memory Data Fabric Beyond the Data Grid DENIS MAGDA Product Manager, Apache PMC Chair #apacheignite

22 In-Memory Data Fabric Advanced Clustering In-Memory Compute Grid In-Memory Data Grid In-Memory SQL Grid In-Memory Service Grid In-Memory Streaming & CEP Plug-n-Play Hadoop Accelerator

23 IN-MEMORY DATA FABRIC STRATEGIC APPROACH TO IMC

24 IN-MEMORY COMPUTING WHY NOW?

25 ADVANCED CLUSTERING & DEPLOYMENT Ease of Getting Started Automatic Discovery Any Environment Public Cloud Private Cloud Hybrid Cloud Local Laptop Zero-Deployment Auto-Deploy Code Full Cluster Management Pluggable Design

26 IN-MEMORY COMPUTE GRID Direct API for MapReduce Zero Deployment Cron-like Task Scheduling State Checkpoints Load Balancing Automatic Failover Full Cluster Management Pluggable SPI Design

27 IN-MEMORY DATA GRID Distributed In-Memory Key-Value Store Replicated and Partitioned data TBs of data, of any type On-Heap and Off-Heap Storage Highly Available In-Memory Replicas Automatic Failover Distributed ACID Transactions SQL queries and JDBC driver Collocation of Compute and Data

28 IN-MEMORY SQL GRID ANSI-99 SQL Compliant Aggregations, group by, sorting Cross-cache joins, unions, etc. DML and DDL JDBC, ODBC, Native APIs Distributed Always consistent Fault tolerant

29 IN-MEMORY SERVICE GRID Distribute Any Data Structure Available Anywhere on the Grid Automatic Remote Access via Proxies Controlled Deployment Support for Cluster Singleton Support for Node Singleton Support for Custom Topology Load Balanced Guaranteed Availability Auto Redeployment in Case of Failures

30 IN-MEMORY STREAMING AND CEP Streaming Data Never Ends Branching Pipelines Pluggable Routing Sliding Windows Real Time Analysis

31 IN-MEMORY HADOOP ACCELERATOR Plug and Play installation 2x to Nx Acceleration In-Memory Native MapReduce In-Process Data Colocation IgniteFS In-Memory File System Read-Through from HDFS Write-Through to HDFS Sync and Async Persistence

32 SPARK INTEGRATION SHARED RDD & IN-MEMORY FILE SYSTEM

33 Turbocharge your SQL queries in-memory with Apache Ignite 25 April - 4:50 PM - 5:15 Room 210

34 THANK YOU! #apacheignite

35 Thank You to All of Our Sponsors!!