CockroachDB on DC/OS. Ben Darnell, CTO, Cockroach Labs
|
|
- Barnard Hubbard
- 6 years ago
- Views:
Transcription
1 CockroachDB on DC/OS Ben Darnell, CTO, Cockroach Labs
2 Agenda A cloud-native database CockroachDB on DC/OS Why CockroachDB Demo!
3 Cloud-Native Database
4 What is Cloud-Native? Horizontally scalable Individual machines aren t special Built to handle failures
5 Why? Adapt to Change Grow and shrink based on demands Easy, rapid deployment Reorganize to reduce load and latency Load follows the sun
6 Traditional Databases More comfortable scaling vertically than horizontally Primaries vs secondaries vs read-only replicas Failover is often manual Asynchronous replication may lose data
7 Cloud-Native CockroachDB Add nodes with automatic rebalancing Any node can serve any role Automatic failover with consistent replication
8 CockroachDB on DC/OS
9 DC/OS Provides an Ideal Environment Elastic allocation and scheduling Discovery, VIPs, and load balancing Weakly persistent local storage
10 Storage Dilemma RAID or network-attached storage is expensive CockroachDB supplies its own redundancy Naive container deployments could lose all local volumes at once DC/OS SDK provides persistent scheduling
11 DC/OS Package dcos package install cockroachdb Starts a 3-node cluster Add more later by changing NODE_COUNT variable Or one-click install through the web
12 Why CockroachDB? An open source SQL database for global cloud services.
13 Why CockroachDB? Distributed SQL Data integrity at global scale Multi-active availability Consistent transactions
14 Distributed SQL A database that grows with your application
15 What is Distributed SQL? SQL with ACID semantics Runs across multiple database servers Distributed storage Distributed compute
16 Distributed Storage Tables scale to any size No manual partitioning necessary Automatic rebalancing as machines are added
17 Distributed Execution Nodes with data compute their portion of results SELECT SUM(duration) FROM sessions
18 Flexible Scheduling Schedule CockroachDB nodes......near your application servers for lower latency...or on your application nodes when you have underutilized disks
19 Global Distribution Span three datacenters to survive major outages Put data close to your customers
20 Multi-Active Availability Survive disasters
21 Disaster Recovery, Version 1.0 DC1 DC2 microservices{ Remote Backup
22 Recovery 2.0: Active/Passive DC1 DC2 async Primary Secondary
23 Recovery 3.0: Active/Active DC1 DC2
24 Multi-Active Availability DC1 DC2 consensus consensus consensus DC3
25 Consistent Transactions Data Integrity at Global Scale
26 Consensus Replication A majority of replicas must agree to commit Failovers never lose committed data Raft consensus algorithm
27 Distributed Transactions Transactions can span rows, tables, even databases ACID semantics Serializable isolation
28 Demo
29
30
31
32
33
34
35
36
37
38 Status
39 Current Version: First production-ready release in May 2017 Distributed SQL Multi-active availability Enterprise: Incremental, distributed backup/restore
40 Very Soon: 1.1 Ruggedization Diagnostic & debugging tools Performance SQL coverage Fast data import (CSV)
41 Up Next Performance SQL feature parity (JSONB) Change data capture (via Kafka) Global data architecture Enterprise: row-level partitioning
42 The End
43 Appendix
44 Target Customers Is CockroachDB right for you?
45 Fastest growing companies in the European Gaming Company world are using CockroachDB to solve their global data challenges $2B+ gaming company that uses CockroachDB to improve performance and availability for their global customer base. $60B+ web services company that uses CockroachDB to power their knowledge graph Real-time gaming platform that uses CockroachDB as the enabling technology to deliver their PaaS Proprietary & Confidential
46 Is CockroachDB right for you? Global, scalable cloud services We hope this eventually means everyone Not for: Latency-sensitive writes Unstructured data Efficient OLAP processing (for now) Analytics in a non-sql language (mapreduce)
47 Correctness or Performance? We believe it is better to have application programmers deal with performance problems due to overuse of transactions as bottlenecks arise, rather than always coding around the lack of transactions Authors of Google s Spanner paper
48 CockroachDB Internals Thorax, wings, eyes, legs,...
49 Layered architecture SQL Transactional KV Distribution Replication Storage
50 Data Distribution SQL Transactional KV Distribution Replication Storage
51 Data Distribution Data is a large collection of Key:Value pairs Data split into Ranges of Key:Value pairs Ranges are ~64 MB of data CockroachDB/Bigtable/HBase/Spanner
52 Data Distribution Options for finding data Hashing Order-Preserving
53 Order-preserving data distribution Pro: efficient scans over total ordering of data Con: requires additional indexing
54 Keyspace as Monolithic, Sorted Map
55 Ranges Form the Keyspace
56 Ranges Map to Nodes
57 Data Lookup
58 Consensus SQL Transactional KV Distribution Replication Storage
59 Consensus Replication Replicate to N (where N >= 3) nodes Commit happens when a quorum have written the data CockroachDB/Etcd/Spanner/Aurora/Zookeeper/...
60 Consensus Replication Raft is our consensus protocol of choice. Run one consensus group per range of data
61 Consensus Replication Put cherry Leader apricot banana blueberry cherry grape Follower apricot banana blueberry Follower apricot banana blueberry grape grape
62 Consensus Replication Put cherry Leader apricot Put cherry banana blueberry cherry grape Follower apricot banana blueberry Follower apricot banana blueberry grape grape
63 Consensus Replication Put cherry Leader apricot Put cherry banana blueberry cherry grape Write committed when 2 out of 3 nodes have written data Follower apricot banana blueberry cherry grape Follower apricot banana blueberry grape
64 Consensus Replication Put cherry Leader Put cherry apricot Ack banana blueberry cherry grape Ack Follower apricot banana blueberry Follower apricot banana blueberry cherry grape grape
65 Consensus Replication Put cherry Leader apricot Put cherry banana blueberry cherry grape Ack Follower apricot banana blueberry Follower apricot banana blueberry cherry cherry grape grape
66 Consensus Replication What happens during failover?
67 Consensus Replication: Failover Put cherry Leader apricot banana blueberry cherry grape Follower apricot banana blueberry Follower apricot banana blueberry grape grape
68 Consensus Replication: Failover Put cherry Leader Elect a new Leader apricot banana blueberry cherry grape Follower apricot banana blueberry Follower apricot banana blueberry grape grape
69 Consensus Replication: Failover Follower apricot Leader apricot banana blueberry On failover, only data written to a quorum is considered present Follower apricot banana grape banana blueberry blueberry cherry grape grape
70 Consensus Replication: Failover Read cherry Key not found Follower apricot Leader apricot banana blueberry Follower apricot banana grape banana blueberry blueberry cherry grape grape
71 Consensus Replication Consensus provides atomic writes But only for each range What about operations that hit multiple ranges?
72 Distributed Transactions SQL Transactional KV Distribution Replication Storage
73 Transactions Supports ACID semantics All-or-nothing Serializable [default] & Snapshot isolation levels
74 Distributed Transactions Transaction record Keyed by a transaction UUID Located on first written range Atomic commit or abort via write to txn record One phase commit fast path
75 Distributed Transactions 1. Begin Txn 1 Put cherry Txn-1 Record Status: PENDING apricot cherry (txn 1) grape
76 Distributed Transactions 1. Begin Txn 1 Put cherry 2. Put mango Txn-1 Record Status: PENDING apricot cherry (txn 1) grape lemon mango (txn 1) orange
77 Distributed Transactions 1. Begin Txn 1 Put cherry 2. Put mango 3. Commit Txn 1 Txn-1 Record Status: COMMITTED apricot cherry (txn 1) grape lemon mango (txn 1) orange
78 Distributed Transactions 1. Begin Txn 1 Put cherry 2. Put mango 3. Commit Txn 1 4. Clean up intents Txn-1 Record Status: COMMITTED apricot cherry grape lemon mango orange
79 Distributed Transactions 1. Begin Txn 1 Put cherry 2. Put mango 3. Commit Txn 1 4. Clean up intents 5. Remove Txn 1 apricot cherry grape lemon mango orange
80 Distributed Transactions That s the happy case What about conflicting transactions?
81 Distributed Transactions (read conflict) 1. Begin Txn 1 Put cherry Txn-1 Record Status: PENDING apricot cherry (txn 1) grape
82 Distributed Transactions (read conflict) 1. Begin Txn 1 Put cherry Txn-1 Record Status: PENDING apricot cherry (txn 1) 1. Begin Txn 2 2. Get cherry grape
83 Distributed Transactions (read conflict) 1. Begin Txn 1 Put cherry Txn-1 Record Status: PENDING apricot cherry (txn 1) 1. Begin Txn 2 2. Get cherry Check Txn-1 record grape
84 Distributed Transactions (read conflict) 1. Begin Txn 1 Put cherry Txn-1 Record Status: PENDING apricot cherry (txn 1) 1. Begin Txn 2 2. Get cherry Check Txn-1 record Wait while PENDING grape
85 Distributed Transactions (read conflict) 1. Begin Txn 1 Put cherry 2. Commit Txn1 Txn-1 Record Status: COMMITTED apricot cherry (txn 1) 1. Begin Txn 2 2. Get cherry Check Txn-1 record Wait while PENDING grape
86 Distributed Transactions (read conflict) 1. Begin Txn 1 Put cherry 2. Commit Txn1 Txn-1 Record Status: COMMITTED apricot cherry (txn 1) 1. Begin Txn 2 2. Get cherry Check Txn-1 record Wait while PENDING 3. Commit Txn2 grape
87 Distributed Transactions What about write / write conflicts?
88 Distributed Transactions (write conflict) 1. Begin Txn 1 Put cherry Txn-1 Record Status: PENDING apricot cherry (txn 1) grape
89 Distributed Transactions (write conflict) 1. Begin Txn 1 Put cherry Txn-1 Record Status: PENDING Txn-2 Record Status: PENDING 1. Begin Txn 2 2. Put cherry apricot cherry (txn 1) grape
90 Distributed Transactions (write conflict) 1. Begin Txn 1 Put cherry Txn-1 Record Status: PENDING Txn-2 Record Status: PENDING 1. Begin Txn 2 2. Put cherry Check Txn 1 record apricot cherry (txn 1) grape
91 Distributed Transactions (write conflict) 1. Begin Txn 1 Put cherry Txn-1 Record Status: PENDING Txn-2 Record Status: PENDING 1. Begin Txn 2 2. Put cherry Check Txn 1 record Wait while PENDING apricot cherry (txn 1) grape
92 Distributed Transactions (write conflict) 1. Begin Txn 1 Put cherry 2. Commit Txn 1 Txn-1 Record Status: COMMITTED Txn-2 Record Status: PENDING 1. Begin Txn 2 2. Put cherry Check Txn 1 record Wait while PENDING apricot cherry (txn 1) grape
93 Distributed Transactions (write conflict) 1. Begin Txn 1 Put cherry 2. Commit Txn 1 Txn-1 Record Status: COMMITTED Txn-2 Record Status: PENDING 1. Begin Txn 2 2. Put cherry Check Txn 1 record Resolve cherry (txn 1) Put cherry apricot cherry (txn 2) cherry grape
94 Click to edit title Click to edit text Second level Third level Fourth level» Fifth level
Data Modeling and Databases Ch 14: Data Replication. Gustavo Alonso, Ce Zhang Systems Group Department of Computer Science ETH Zürich
Data Modeling and Databases Ch 14: Data Replication Gustavo Alonso, Ce Zhang Systems Group Department of Computer Science ETH Zürich Database Replication What is database replication The advantages of
More informationGridGain and Apache Ignite In-Memory Performance with Durability of Disk
GridGain and Apache Ignite In-Memory Performance with Durability of Disk Dmitriy Setrakyan Apache Ignite PMC GridGain Founder & CPO http://ignite.apache.org #apacheignite Agenda What is GridGain and Ignite
More informationJargons, Concepts, Scope and Systems. Key Value Stores, Document Stores, Extensible Record Stores. Overview of different scalable relational systems
Jargons, Concepts, Scope and Systems Key Value Stores, Document Stores, Extensible Record Stores Overview of different scalable relational systems Examples of different Data stores Predictions, Comparisons
More informationDistributed PostgreSQL with YugaByte DB
Distributed PostgreSQL with YugaByte DB Karthik Ranganathan PostgresConf Silicon Valley Oct 16, 2018 1 CHECKOUT THIS REPO: github.com/yugabyte/yb-sql-workshop 2 About Us Founders Kannan Muthukkaruppan,
More informationHow we build TiDB. Max Liu PingCAP Amsterdam, Netherlands October 5, 2016
How we build TiDB Max Liu PingCAP Amsterdam, Netherlands October 5, 2016 About me Infrastructure engineer / CEO of PingCAP Working on open source projects: TiDB: https://github.com/pingcap/tidb TiKV: https://github.com/pingcap/tikv
More informationA Brief Introduction of TiDB. Dongxu (Edward) Huang CTO, PingCAP
A Brief Introduction of TiDB Dongxu (Edward) Huang CTO, PingCAP About me Dongxu (Edward) Huang, Cofounder & CTO of PingCAP PingCAP, based in Beijing, China. Infrastructure software engineer, open source
More informationSQL, NoSQL, MongoDB. CSE-291 (Cloud Computing) Fall 2016 Gregory Kesden
SQL, NoSQL, MongoDB CSE-291 (Cloud Computing) Fall 2016 Gregory Kesden SQL Databases Really better called Relational Databases Key construct is the Relation, a.k.a. the table Rows represent records Columns
More informationMegastore: Providing Scalable, Highly Available Storage for Interactive Services & Spanner: Google s Globally- Distributed Database.
Megastore: Providing Scalable, Highly Available Storage for Interactive Services & Spanner: Google s Globally- Distributed Database. Presented by Kewei Li The Problem db nosql complex legacy tuning expensive
More informationCISC 7610 Lecture 2b The beginnings of NoSQL
CISC 7610 Lecture 2b The beginnings of NoSQL Topics: Big Data Google s infrastructure Hadoop: open google infrastructure Scaling through sharding CAP theorem Amazon s Dynamo 5 V s of big data Everyone
More informationIntroduction to NoSQL Databases
Introduction to NoSQL Databases Roman Kern KTI, TU Graz 2017-10-16 Roman Kern (KTI, TU Graz) Dbase2 2017-10-16 1 / 31 Introduction Intro Why NoSQL? Roman Kern (KTI, TU Graz) Dbase2 2017-10-16 2 / 31 Introduction
More informationShen PingCAP 2017
Shen Li @ PingCAP About me Shen Li ( 申砾 ) Tech Lead of TiDB, VP of Engineering Netease / 360 / PingCAP Infrastructure software engineer WHY DO WE NEED A NEW DATABASE? Brief History Standalone RDBMS NoSQL
More informationDistributed Computation Models
Distributed Computation Models SWE 622, Spring 2017 Distributed Software Engineering Some slides ack: Jeff Dean HW4 Recap https://b.socrative.com/ Class: SWE622 2 Review Replicating state machines Case
More informationCPS 512 midterm exam #1, 10/7/2016
CPS 512 midterm exam #1, 10/7/2016 Your name please: NetID: Answer all questions. Please attempt to confine your answers to the boxes provided. If you don t know the answer to a question, then just say
More informationLast time. Distributed systems Lecture 6: Elections, distributed transactions, and replication. DrRobert N. M. Watson
Distributed systems Lecture 6: Elections, distributed transactions, and replication DrRobert N. M. Watson 1 Last time Saw how we can build ordered multicast Messages between processes in a group Need to
More informationHow do we build TiDB. a Distributed, Consistent, Scalable, SQL Database
How do we build TiDB a Distributed, Consistent, Scalable, SQL Database About me LiuQi ( 刘奇 ) JD / WandouLabs / PingCAP Co-founder / CEO of PingCAP Open-source hacker / Infrastructure software engineer
More informationDistributed Data on Distributed Infrastructure. Claudius Weinberger & Kunal Kusoorkar, ArangoDB Jörg Schad, Mesosphere
Distributed Data on Distributed Infrastructure Claudius Weinberger & Kunal Kusoorkar, ArangoDB Jörg Schad, Mesosphere Kunal Kusoorkar Director Solutions Engineering, ArangoDB @neunhoef Jörg Schad Claudius
More informationCMU SCS CMU SCS Who: What: When: Where: Why: CMU SCS
Carnegie Mellon Univ. Dept. of Computer Science 15-415/615 - DB s C. Faloutsos A. Pavlo Lecture#23: Distributed Database Systems (R&G ch. 22) Administrivia Final Exam Who: You What: R&G Chapters 15-22
More informationPerformance and Forgiveness. June 23, 2008 Margo Seltzer Harvard University School of Engineering and Applied Sciences
Performance and Forgiveness June 23, 2008 Margo Seltzer Harvard University School of Engineering and Applied Sciences Margo Seltzer Architect Outline A consistency primer Techniques and costs of consistency
More informationDatabase Architectures
Database Architectures CPS352: Database Systems Simon Miner Gordon College Last Revised: 4/15/15 Agenda Check-in Parallelism and Distributed Databases Technology Research Project Introduction to NoSQL
More informationMySQL High Availability Solutions. Alex Poritskiy Percona
MySQL High Availability Solutions Alex Poritskiy Percona The Five 9s of Availability Clustering & Geographical Redundancy Clustering Technologies Replication Technologies Well-Managed disasters power failures
More informationCSE 544 Principles of Database Management Systems. Magdalena Balazinska Winter 2015 Lecture 14 NoSQL
CSE 544 Principles of Database Management Systems Magdalena Balazinska Winter 2015 Lecture 14 NoSQL References Scalable SQL and NoSQL Data Stores, Rick Cattell, SIGMOD Record, December 2010 (Vol. 39, No.
More informationCSE 444: Database Internals. Section 9: 2-Phase Commit and Replication
CSE 444: Database Internals Section 9: 2-Phase Commit and Replication 1 Today 2-Phase Commit Replication 2 Two-Phase Commit Protocol (2PC) One coordinator and many subordinates Phase 1: Prepare Phase 2:
More informationDistributed systems. Lecture 6: distributed transactions, elections, consensus and replication. Malte Schwarzkopf
Distributed systems Lecture 6: distributed transactions, elections, consensus and replication Malte Schwarzkopf Last time Saw how we can build ordered multicast Messages between processes in a group Need
More informationEBOOK. FROM DISASTER RECOVERY TO ACTIVE-ACTIVE: NuoDB AND MULTI-DATA CENTER DEPLOYMENTS
FROM DISASTER RECOVERY TO ACTIVE-ACTIVE: NuoDB AND MULTI-DATA CENTER DEPLOYMENTS INTRODUCTION Traditionally, multi-data center strategies were deployed primarily to address disaster recovery scenarios.
More informationBuilding Consistent Transactions with Inconsistent Replication
Building Consistent Transactions with Inconsistent Replication Irene Zhang, Naveen Kr. Sharma, Adriana Szekeres, Arvind Krishnamurthy, Dan R. K. Ports University of Washington Distributed storage systems
More informationModern Database Concepts
Modern Database Concepts Basic Principles Doc. RNDr. Irena Holubova, Ph.D. holubova@ksi.mff.cuni.cz NoSQL Overview Main objective: to implement a distributed state Different objects stored on different
More informationBeyond Relational Databases: MongoDB, Redis & ClickHouse. Marcos Albe - Principal Support Percona
Beyond Relational Databases: MongoDB, Redis & ClickHouse Marcos Albe - Principal Support Engineer @ Percona Introduction MySQL everyone? Introduction Redis? OLAP -vs- OLTP Image credits: 451 Research (https://451research.com/state-of-the-database-landscape)
More informationNoSQL systems. Lecture 21 (optional) Instructor: Sudeepa Roy. CompSci 516 Data Intensive Computing Systems
CompSci 516 Data Intensive Computing Systems Lecture 21 (optional) NoSQL systems Instructor: Sudeepa Roy Duke CS, Spring 2016 CompSci 516: Data Intensive Computing Systems 1 Key- Value Stores Duke CS,
More information10. Replication. CSEP 545 Transaction Processing Philip A. Bernstein. Copyright 2003 Philip A. Bernstein. Outline
10. Replication CSEP 545 Transaction Processing Philip A. Bernstein Copyright 2003 Philip A. Bernstein 1 Outline 1. Introduction 2. Primary-Copy Replication 3. Multi-Master Replication 4. Other Approaches
More informationLarge-Scale Key-Value Stores Eventual Consistency Marco Serafini
Large-Scale Key-Value Stores Eventual Consistency Marco Serafini COMPSCI 590S Lecture 13 Goals of Key-Value Stores Export simple API put(key, value) get(key) Simpler and faster than a DBMS Less complexity,
More informationCopyright 2013, Oracle and/or its affiliates. All rights reserved.
1 Oracle NoSQL Database: Release 3.0 What s new and why you care Dave Segleau NoSQL Product Manager The following is intended to outline our general product direction. It is intended for information purposes
More informationNext-Generation Cloud Platform
Next-Generation Cloud Platform Jangwoo Kim Jun 24, 2013 E-mail: jangwoo@postech.ac.kr High Performance Computing Lab Department of Computer Science & Engineering Pohang University of Science and Technology
More informationRule 14 Use Databases Appropriately
Rule 14 Use Databases Appropriately Rule 14: What, When, How, and Why What: Use relational databases when you need ACID properties to maintain relationships between your data. For other data storage needs
More informationCISC 7610 Lecture 5 Distributed multimedia databases. Topics: Scaling up vs out Replication Partitioning CAP Theorem NoSQL NewSQL
CISC 7610 Lecture 5 Distributed multimedia databases Topics: Scaling up vs out Replication Partitioning CAP Theorem NoSQL NewSQL Motivation YouTube receives 400 hours of video per minute That is 200M hours
More informationMDCC MULTI DATA CENTER CONSISTENCY. amplab. Tim Kraska, Gene Pang, Michael Franklin, Samuel Madden, Alan Fekete
MDCC MULTI DATA CENTER CONSISTENCY Tim Kraska, Gene Pang, Michael Franklin, Samuel Madden, Alan Fekete gpang@cs.berkeley.edu amplab MOTIVATION 2 3 June 2, 200: Rackspace power outage of approximately 0
More information10. Replication. CSEP 545 Transaction Processing Philip A. Bernstein Sameh Elnikety. Copyright 2012 Philip A. Bernstein
10. Replication CSEP 545 Transaction Processing Philip A. Bernstein Sameh Elnikety Copyright 2012 Philip A. Bernstein 1 Outline 1. Introduction 2. Primary-Copy Replication 3. Multi-Master Replication 4.
More informationIntegrity in Distributed Databases
Integrity in Distributed Databases Andreas Farella Free University of Bozen-Bolzano Table of Contents 1 Introduction................................................... 3 2 Different aspects of integrity.....................................
More informationDistributed System. Gang Wu. Spring,2018
Distributed System Gang Wu Spring,2018 Lecture4:Failure& Fault-tolerant Failure is the defining difference between distributed and local programming, so you have to design distributed systems with the
More informationCS /15/16. Paul Krzyzanowski 1. Question 1. Distributed Systems 2016 Exam 2 Review. Question 3. Question 2. Question 5.
Question 1 What makes a message unstable? How does an unstable message become stable? Distributed Systems 2016 Exam 2 Review Paul Krzyzanowski Rutgers University Fall 2016 In virtual sychrony, a message
More informationHow to Scale MongoDB. Apr
How to Scale MongoDB Apr-24-2018 About me Location: Skopje, Republic of Macedonia Education: MSc, Software Engineering Experience: Lead Database Consultant (since 2016) Database Consultant (2012-2016)
More information@joerg_schad Nightmares of a Container Orchestration System
@joerg_schad Nightmares of a Container Orchestration System 2017 Mesosphere, Inc. All Rights Reserved. 1 Jörg Schad Distributed Systems Engineer @joerg_schad Jan Repnak Support Engineer/ Solution Architect
More informationReplication in Distributed Systems
Replication in Distributed Systems Replication Basics Multiple copies of data kept in different nodes A set of replicas holding copies of a data Nodes can be physically very close or distributed all over
More informationCSE 544 Principles of Database Management Systems. Alvin Cheung Fall 2015 Lecture 14 Distributed Transactions
CSE 544 Principles of Database Management Systems Alvin Cheung Fall 2015 Lecture 14 Distributed Transactions Transactions Main issues: Concurrency control Recovery from failures 2 Distributed Transactions
More informationConceptual Modeling on Tencent s Distributed Database Systems. Pan Anqun, Wang Xiaoyu, Li Haixiang Tencent Inc.
Conceptual Modeling on Tencent s Distributed Database Systems Pan Anqun, Wang Xiaoyu, Li Haixiang Tencent Inc. Outline Introduction System overview of TDSQL Conceptual Modeling on TDSQL Applications Conclusion
More informationVirtual protection gets real
Virtual protection gets real How to protect virtual machines from downtime and data loss 5 must-have features for VM backup Businesses virtualize to consolidate resources, reduce costs and increase workforce
More informationrelational Relational to Riak Why Move From Relational to Riak? Introduction High Availability Riak At-a-Glance
WHITEPAPER Relational to Riak relational Introduction This whitepaper looks at why companies choose Riak over a relational database. We focus specifically on availability, scalability, and the / data model.
More informationMySQL Replication Options. Peter Zaitsev, CEO, Percona Moscow MySQL User Meetup Moscow,Russia
MySQL Replication Options Peter Zaitsev, CEO, Percona Moscow MySQL User Meetup Moscow,Russia Few Words About Percona 2 Your Partner in MySQL and MongoDB Success 100% Open Source Software We work with MySQL,
More information10. Replication. Motivation
10. Replication Page 1 10. Replication Motivation Reliable and high-performance computation on a single instance of a data object is prone to failure. Replicate data to overcome single points of failure
More informationTechnical Note. Dell/EMC Solutions for Microsoft SQL Server 2005 Always On Technologies. Abstract
Technical Note Dell/EMC Solutions for Microsoft SQL Server 2005 Always On Technologies Abstract This technical note provides information on the Dell/EMC storage solutions, based on the Microsoft SQL Server
More informationBest practices for OpenShift high-availability deployment field experience
Best practices for OpenShift high-availability deployment field experience Orgad Kimchi, Red Hat, Christina Wong, NuoDB, Ariff Kassam, NuoDB Traditional disaster recovery means maintaining underutilized
More informationA Distributed System Case Study: Apache Kafka. High throughput messaging for diverse consumers
A Distributed System Case Study: Apache Kafka High throughput messaging for diverse consumers As always, this is not a tutorial Some of the concepts may no longer be part of the current system or implemented
More informationDrRobert N. M. Watson
Distributed systems Lecture 15: Replication, quorums, consistency, CAP, and Amazon/Google case studies DrRobert N. M. Watson 1 Last time General issue of consensus: How to get processes to agree on something
More informationBecome a MongoDB Replica Set Expert in Under 5 Minutes:
Become a MongoDB Replica Set Expert in Under 5 Minutes: USING PERCONA SERVER FOR MONGODB IN A FAILOVER ARCHITECTURE This solution brief outlines a way to run a MongoDB replica set for read scaling in production.
More informationMicrosoft SQL AlwaysOn and High Availability
Course 55153A: Microsoft SQL AlwaysOn and High Availability Course details Course Outline Module 1: Course Overview This module explains how the class will be structured and introduces course materials
More informationNew Oracle NoSQL Database APIs that Speed Insertion and Retrieval
New Oracle NoSQL Database APIs that Speed Insertion and Retrieval O R A C L E W H I T E P A P E R F E B R U A R Y 2 0 1 6 1 NEW ORACLE NoSQL DATABASE APIs that SPEED INSERTION AND RETRIEVAL Introduction
More informationEverything You Need to Know About MySQL Group Replication
Everything You Need to Know About MySQL Group Replication Luís Soares (luis.soares@oracle.com) Principal Software Engineer, MySQL Replication Lead Copyright 2017, Oracle and/or its affiliates. All rights
More informationDatabase Architectures
Database Architectures CPS352: Database Systems Simon Miner Gordon College Last Revised: 11/15/12 Agenda Check-in Centralized and Client-Server Models Parallelism Distributed Databases Homework 6 Check-in
More informationThere Is More Consensus in Egalitarian Parliaments
There Is More Consensus in Egalitarian Parliaments Iulian Moraru, David Andersen, Michael Kaminsky Carnegie Mellon University Intel Labs Fault tolerance Redundancy State Machine Replication 3 State Machine
More informationDISTRIBUTED DATABASES CS561-SPRING 2012 WPI, MOHAMED ELTABAKH
DISTRIBUTED DATABASES CS561-SPRING 2012 WPI, MOHAMED ELTABAKH 1 RECAP: PARALLEL DATABASES Three possible architectures Shared-memory Shared-disk Shared-nothing (the most common one) Parallel algorithms
More informationDistributed Data Analytics Partitioning
G-3.1.09, Campus III Hasso Plattner Institut Different mechanisms but usually used together Distributing Data Replication vs. Replication Store copies of the same data on several nodes Introduces redundancy
More informationDistributed Systems. Characteristics of Distributed Systems. Lecture Notes 1 Basic Concepts. Operating Systems. Anand Tripathi
1 Lecture Notes 1 Basic Concepts Anand Tripathi CSci 8980 Operating Systems Anand Tripathi CSci 8980 1 Distributed Systems A set of computers (hosts or nodes) connected through a communication network.
More informationDistributed Systems. Characteristics of Distributed Systems. Characteristics of Distributed Systems. Goals in Distributed System Designs
1 Anand Tripathi CSci 8980 Operating Systems Lecture Notes 1 Basic Concepts Distributed Systems A set of computers (hosts or nodes) connected through a communication network. Nodes may have different speeds
More informationExam 2 Review. October 29, Paul Krzyzanowski 1
Exam 2 Review October 29, 2015 2013 Paul Krzyzanowski 1 Question 1 Why did Dropbox add notification servers to their architecture? To avoid the overhead of clients polling the servers periodically to check
More informationDifferentiating Your Datacentre in the Networked Future John Duffin
Differentiating Your Datacentre in the Networked Future John Duffin Managing Director, South Asia. Uptime Institute July 2017 2017 Uptime Institute, LLC The Global Datacentre Authority 2 2017 Uptime Institute,
More informationDistributed Coordination with ZooKeeper - Theory and Practice. Simon Tao EMC Labs of China Oct. 24th, 2015
Distributed Coordination with ZooKeeper - Theory and Practice Simon Tao EMC Labs of China {simon.tao@emc.com} Oct. 24th, 2015 Agenda 1. ZooKeeper Overview 2. Coordination in Spring XD 3. ZooKeeper Under
More information4 Myths about in-memory databases busted
4 Myths about in-memory databases busted Yiftach Shoolman Co-Founder & CTO @ Redis Labs @yiftachsh, @redislabsinc Background - Redis Created by Salvatore Sanfilippo (@antirez) OSS, in-memory NoSQL k/v
More informationDistributed Systems 8L for Part IB
Distributed Systems 8L for Part IB Handout 3 Dr. Steven Hand 1 Distributed Mutual Exclusion In first part of course, saw need to coordinate concurrent processes / threads In particular considered how to
More informationMySQL HA Solutions Selecting the best approach to protect access to your data
MySQL HA Solutions Selecting the best approach to protect access to your data Sastry Vedantam sastry.vedantam@oracle.com February 2015 Copyright 2015, Oracle and/or its affiliates. All rights reserved
More information<Insert Picture Here> Oracle NoSQL Database A Distributed Key-Value Store
Oracle NoSQL Database A Distributed Key-Value Store Charles Lamb The following is intended to outline our general product direction. It is intended for information purposes only,
More informationNewSQL Without Compromise
NewSQL Without Compromise Everyday businesses face serious challenges coping with application performance, maintaining business continuity, and gaining operational intelligence in real- time. There are
More informationDistributed Data Management Transactions
Felix Naumann F-2.03/F-2.04, Campus II Hasso Plattner Institut must ensure that interactions succeed consistently An OLTP Topic Motivation Most database interactions consist of multiple, coherent operations
More informationParallel DBs. April 25, 2017
Parallel DBs April 25, 2017 1 Sending Hints Rk B Si Strategy 3: Bloom Filters Node 1 Node 2 2 Sending Hints Rk B Si Strategy 3: Bloom Filters Node 1 with
More informationMySQL High Availability. Michael Messina Senior Managing Consultant, Rolta-AdvizeX /
MySQL High Availability Michael Messina Senior Managing Consultant, Rolta-AdvizeX mmessina@advizex.com / mike.messina@rolta.com Introduction Michael Messina Senior Managing Consultant Rolta-AdvizeX, Working
More informationCassandra, MongoDB, and HBase. Cassandra, MongoDB, and HBase. I have chosen these three due to their recent
Tanton Jeppson CS 401R Lab 3 Cassandra, MongoDB, and HBase Introduction For my report I have chosen to take a deeper look at 3 NoSQL database systems: Cassandra, MongoDB, and HBase. I have chosen these
More informationAzure File Sync. Webinaari
Azure File Sync Webinaari 12.3.2018 Agenda Why use Azure? Moving to the Cloud Azure Storage Backup and Recovery Azure File Sync Demo Q&A What is Azure? A collection of cloud services from Microsoft that
More informationMQ High Availability and Disaster Recovery Implementation scenarios
MQ High Availability and Disaster Recovery Implementation scenarios Sandeep Chellingi Head of Hybrid Cloud Integration Prolifics Agenda MQ Availability Message Availability Service Availability HA vs DR
More informationDatacenter replication solution with quasardb
Datacenter replication solution with quasardb Technical positioning paper April 2017 Release v1.3 www.quasardb.net Contact: sales@quasardb.net Quasardb A datacenter survival guide quasardb INTRODUCTION
More informationA Guide to Architecting the Active/Active Data Center
White Paper A Guide to Architecting the Active/Active Data Center 2015 ScaleArc. All Rights Reserved. White Paper The New Imperative: Architecting the Active/Active Data Center Introduction With the average
More informationDon t Give Up on Serializability Just Yet. Neha Narula
Don t Give Up on Serializability Just Yet Neha Narula Don t Give Up on Serializability Just Yet A journey into serializable systems Neha Narula MIT CSAIL GOTO Chicago May 2015 2 @neha PhD candidate at
More informationSpanner: Google's Globally-Distributed Database. Presented by Maciej Swiech
Spanner: Google's Globally-Distributed Database Presented by Maciej Swiech What is Spanner? "...Google's scalable, multi-version, globallydistributed, and synchronously replicated database." What is Spanner?
More informationReal-time Protection for Microsoft Hyper-V
Real-time Protection for Microsoft Hyper-V Introduction Computer virtualization has come a long way in a very short time, triggered primarily by the rapid rate of customer adoption. Moving resources to
More informationSCALABLE CONSISTENCY AND TRANSACTION MODELS
Data Management in the Cloud SCALABLE CONSISTENCY AND TRANSACTION MODELS 69 Brewer s Conjecture Three properties that are desirable and expected from realworld shared-data systems C: data consistency A:
More informationBig Data Analytics. Rasoul Karimi
Big Data Analytics Rasoul Karimi Information Systems and Machine Learning Lab (ISMLL) Institute of Computer Science University of Hildesheim, Germany Big Data Analytics Big Data Analytics 1 / 1 Outline
More informationHOW CLUSTRIXDB RDBMS SCALES WRITES & READS
Scaling out a SQL RDBMS while maintaining ACID guarantees in realtime is a very large challenge. Most scaling DBMS solutions relinquish one or many realtime transactionality requirements. ClustrixDB achieves
More informationRecall: Primary-Backup. State machine replication. Extend PB for high availability. Consensus 2. Mechanism: Replicate and separate servers
Replicated s, RAFT COS 8: Distributed Systems Lecture 8 Recall: Primary-Backup Mechanism: Replicate and separate servers Goal #: Provide a highly reliable service Goal #: Servers should behave just like
More informationIntra-cluster Replication for Apache Kafka. Jun Rao
Intra-cluster Replication for Apache Kafka Jun Rao About myself Engineer at LinkedIn since 2010 Worked on Apache Kafka and Cassandra Database researcher at IBM Outline Overview of Kafka Kafka architecture
More informationCA485 Ray Walshe NoSQL
NoSQL BASE vs ACID Summary Traditional relational database management systems (RDBMS) do not scale because they adhere to ACID. A strong movement within cloud computing is to utilize non-traditional data
More informationSQL Server Availability Groups
A r c h i t e c t i n g SQL Server Availability Groups Without Losing Your S A N I T Y Edwin Sarmiento Microsoft MVP/Microsoft Certified Master: SQL Server http://www.edwinmsarmiento.com edwin@edwinmsarmiento.com
More informationHIGH-AVAILABILITY & D/R OPTIONS FOR MICROSOFT SQL SERVER
SQL SATURDAY # 91 - OMAHA HIGH-AVAILABILITY & D/R OPTIONS FOR MICROSOFT SQL SERVER 8/27/11 Tim Plas, Virteva tim.plas@virteva.com 1 The Presenter Tim Plas, Principal Consultant at Virteva (Mpls) Operational
More informationOutline. Spanner Mo/va/on. Tom Anderson
Spanner Mo/va/on Tom Anderson Outline Last week: Chubby: coordina/on service BigTable: scalable storage of structured data GFS: large- scale storage for bulk data Today/Friday: Lessons from GFS/BigTable
More informationCLUSTERING HIVEMQ. Building highly available, horizontally scalable MQTT Broker Clusters
CLUSTERING HIVEMQ Building highly available, horizontally scalable MQTT Broker Clusters 12/2016 About this document MQTT is based on a publish/subscribe architecture that decouples MQTT clients and uses
More informationDistributed Data Management Replication
Felix Naumann F-2.03/F-2.04, Campus II Hasso Plattner Institut Distributing Data Motivation Scalability (Elasticity) If data volume, processing, or access exhausts one machine, you might want to spread
More informationScaling for Humongous amounts of data with MongoDB
Scaling for Humongous amounts of data with MongoDB Alvin Richards Technical Director, EMEA alvin@10gen.com @jonnyeight alvinonmongodb.com From here... http://bit.ly/ot71m4 ...to here... http://bit.ly/oxcsis
More informationMySQL Enterprise High Availability
MySQL Enterprise High Availability A Reference Guide A MySQL White Paper 2018, Oracle Corporation and/or its affiliates Table of Contents MySQL Enterprise High Availability... 1 A Reference Guide... 1
More informationCS5412: TRANSACTIONS (I)
1 CS5412: TRANSACTIONS (I) Lecture XVII Ken Birman Transactions 2 A widely used reliability technology, despite the BASE methodology we use in the first tier Goal for this week: in-depth examination of
More informationThere is a tempta7on to say it is really used, it must be good
Notes from reviews Dynamo Evalua7on doesn t cover all design goals (e.g. incremental scalability, heterogeneity) Is it research? Complexity? How general? Dynamo Mo7va7on Normal database not the right fit
More information@unterstein #bedcon. Operating microservices with Apache Mesos and DC/OS
@unterstein @dcos @bedcon #bedcon Operating microservices with Apache Mesos and DC/OS 1 Johannes Unterstein Software Engineer @Mesosphere @unterstein @unterstein.mesosphere 2017 Mesosphere, Inc. All Rights
More informationLeader or Majority: Why have one when you can have both? Improving Read Scalability in Raft-like consensus protocols
Leader or Majority: Why have one when you can have both? Improving Read Scalability in Raft-like consensus protocols Vaibhav Arora, Tanuj Mittal, Divyakant Agrawal, Amr El Abbadi * and Xun Xue, Zhiyanan,
More informationEvaluating Cloud Storage Strategies. James Bottomley; CTO, Server Virtualization
Evaluating Cloud Storage Strategies James Bottomley; CTO, Server Virtualization Introduction to Storage Attachments: - Local (Direct cheap) SAS, SATA - Remote (SAN, NAS expensive) FC net Types - Block
More informationBIG DATA AND CONSISTENCY. Amy Babay
BIG DATA AND CONSISTENCY Amy Babay Outline Big Data What is it? How is it used? What problems need to be solved? Replication What are the options? Can we use this to solve Big Data s problems? Putting
More information