Traditional RDBMS Wisdom is All Wrong -- In Three Acts. Michael Stonebraker

Transcription

1 Traditional RDBMS Wisdom is All Wrong -- In Three Acts Michael Stonebraker

2 The Stonebraker Says Webinar Series The first three acts: 1. Why main memory is the answer for OLTP Recording available at VoltDB.com 2. Main memory and ACID, How to Go Fast - Today! 3. OLTP RDBMSs are the Right Tool for Fast + Big Data Apps - June 10 Extensions to main memory Support for streaming Larger-than-memory applications

3 Our Speaker Dr. Mike Stonebraker of MIT, co-founder of VoltDB

4 Overview of Today Trends Why ACID is essential HA Concurrency Control Summary and Q&A

5 VoltDB The Company 400 Customers Magic Quadrant Operational Databases Mike Stonebraker Founder, Advisor One of the Top Companies that Matter Most in Data Version 4.2 launched in March 2014 Offices in Bedford, MA & Santa Clara, CA

6 Trends or Why is it important to go fast? Internet: people + content + things Data: Fast + Big Smart: raw data + intelligence

7 The RDBMS is Getting Crushed Mobile M2M Market Data Clickstreams Web Interactions Social Media Internet of Things Bottleneck RDBMS Making do? SSD or Fusion IO cards Sharding Cache/Grid NoSQL, No ACID Reports Analytics Dashboards Alerts (In Real-Time) vs. Batch

8 VoltDB: A Modern DB Architecture In-Memory performance NewSQL - ACID & SQL & Java Elastic Scaling Reliability and fault tolerance Real-time analytics

9 Preliminaries. Some people argue that you don t need ACID Some people argue that eventual consistency is the best thing since sliced bread

10 Who Needs ACID Anybody with integrity constraints Back out if fails Anybody for whom usually ships in 24 hours is not an acceptable outcome Anybody with a multi-record state E.g. move and shoot

11 Give Up ACID If you need data accuracy, giving up ACID is a decision to tear your hair out by doing database heavy lifting in user code Can you guarantee you won t need ACID tomorrow? ACID = goodness, in spite of what the critics say Even Jeff Dean (Google) agrees

12 Eventual Consistency and Replication Do the transaction at the primary, then send to secondary asynchronously If a failure, then make secondary the primary Guarantee eventual delivery of transactions to all replicas When the dust settles, the claim is that both replicas are the same Example of two people buying a widget

13 Fails to Work. Two people buying a widget when only one widget is in inventory. What goes wrong? Eventual consistency depends on transactions running in either order at the various replicas Fails if you cannot oversell inventory More generally Fails unless transactions are commutative

14 Non Commutative Updates For example, + and * don t commute 10% raise $ bonus Anybody with integrity constraints Can t sell the last item twice. In these cases, eventual consistency means creates garbage In the rest of this talk, we assume ACID

15 The Three Big Decisions How to do crash recovery How to do replication How to do concurrency control

16 Reality Check High Availability (HA) Requirement in today s OLTP systems Nobody will take down time Must be solved through replication

17 Reality Check Main Memory Performance TPC-C CPU cycles On the Shore DBMS prototype Elephants should be similar

18 Some Data From Nirmesh Malvaiya Implemented Aries in VoltDB Compared against the VoltDB scheme Asynchronous transaction-consistent checkpoints Command logging

19

20

21 In Round Numbers Command logging 50% faster at run time But data logging 50% faster at recovery time However.. You only stop and recover on cluster-wide failures, e.g. power failures Which are not very common (every few months???)

22 How to Implement HA Active-Passive As in the traditional wisdom Active-Active Send update transactions to all copies Each executes transaction logic

23 How to Implement HA Active-Passive Effectively requires you to write a log One of the four pie slices Active-Active Send only the transaction, not the effect of the transaction Allows read-queries to be sent to any replica

24 Some Data From Chaomin Yu (CMU) Extended Nirmesh s VoltDB code to replicas 1.5 becomes 1.9 at run time Roll forward stays at 1.5 i.e. active-active wins!

25 Concurrency Control MVCC popular (NuoDB, Hekaton) Time stamp order popular (H-Store/VoltDB) Lightweight combinations of time stamp order and dynamic locking (Calvin, Dora) I don t know anybody who is doing normal dynamic locking It s too slow!!!!

26 VoltDB Scheme Tables are replicated or fragmented Physical data base design cuts up fragmented tables horizontally and allocates them to partitions Stored procedures can be classified into Single-partition ones Multi-partition ones one shots Multi-step xacts

27 Concurrency Control Single partition transactions Sent directly from user code to the controller for the correct partition Who executes them serially (single threaded)

28 Concurrency Control Multi-partition one shots Sent to a multi-partition controller Who then sends the various pieces to the correct singlepartition controller Who inserts the piece somewhere in the stream of single-partition transactions Doesn t matter where With some cleverness, VoltDB does parallel one-shot reads

29 Concurrency Control Summary Single-partition transactions Parallelism determined by number of partitions Essentially no overhead Multi-partition one-shot reads Extensive parallelism Essentially no overhead

30 Concurrency Control Summary Multi-partition one shot writes One-at-a-time Parallelized with single-partition transactions Multi-partition multi-step transactions Currently slow Can be dramatically accelerated using speculative execution

31 Concurrency Control Advice VoltDB is unbeatable on a workload on single partition and one-shot transactions Multi-partition multi-step transactions are slow specex is part of the VoltDB roadmap discussion If you have high conflict, then you should redesign your applications Nobody will do well

32 The Nail in the Coffin VoltDB scheme compatible with active-active As are any deterministic schemes Locking and MVCC are not Give up a factor of 2!

33 Net-Net on Main memory ACID Deterministic concurrency control VoltDB has one scheme Crash recovery using command logging X 1.5 HA/replication via active-active X 1.9 Traditional wisdom (and essentially all DBMS textbooks) are all wrong!

34 34 Summary Data is growing in volume and velocity Smart data requires adding intelligence in real time New architectures are required, but compromises are not check out VoltDB

35 Questions? Use the chat window to type in your questions. Try VoltDB yourself: Free trial of the Enterprise Edition: Open source version is available on github.com Register for the June 10 th webinar

36 Thank You