Architecture of a Real-Time Operational DBMS
|
|
- Anthony Scott
- 5 years ago
- Views:
Transcription
1 Architecture of a Real-Time Operational DBMS Srini V. Srinivasan Founder, Chief Development Officer Aerospike CMG India Keynote Thane December 3, 2016 [ CMGI Keynote, Thane, India Aerospike Inc. All rights reserved. 1 ]
2 Real-Time Workloads [ CMGI Keynote, Thane, India Aerospike Inc. All rights reserved. 2 ]
3 Reliability at Massive Scale Developments Internet growth High rate of transactions Millions/second Storage advances expansion of DRAM, rise of SSDs New distributed consensus algorithms e.g., CAP Theorem, Paxos Application developers prefer not using SQL Python, Go, PHP, Traditional DB Guarantee strong consistency to replicated data Limited in scalability and availability Cannot handle network partitions NoSQL DB Focus on massive scalability and high availability Use concepts from Operating Systems and Distributed Systems [ CMGI Keynote, Thane, India Aerospike Inc. All rights reserved. 3 ]
4 SQL è NoSQL SQL databases are architected for Disk oriented storage and indexing structures Multithreading to hide latency Locking-based concurrency control mechanisms Log-based recovery NoSQL or NewSQL databases are architected for In-Memory Incremental upgrades (no fork-lift upgrade) High Availability Self-managing (self-healing, self-maintaining, self-tuning) [ CMGI Keynote, Thane, India Aerospike Inc. All rights reserved. 4 ]
5 Next Generation of Database Systems Speed at Scale Many Choices Scaling up affordably TCO Restricted Functionality Real-time Problem Set Observed TCO Speed TPS Rich functionality nonreal-time use cases TCO ($) Desired TCO Scale TB Scale TB TCO - Total cost of ownership Goal: Deliver Predictable Performance, Highest Availability, and Lowest TCO [ CMGI Keynote, Thane, India Aerospike Inc. All rights reserved. 5 ]
6 Use Cases [ CMGI Keynote, Thane, India Aerospike Inc. All rights reserved. 6 ]
7 Billion Dollar Advertising Market uses RTB 1 to 6 billion cookies tracked Auctions at about 3.0M / sec in North America 100ms ad rendering, 50ms real-time bidding Low Latency, High Throughput, High Uptime [ CMGI Keynote, Thane, India Aerospike Inc. All rights reserved. 7 ]
8 RTB Tech Stack [ CMGI Keynote, Thane, India Aerospike Inc. All rights reserved. 8 ]
9 Beyond Ad Tech FinServ Marketing Tech Telco AdTech Gaming [ CMGI Keynote, Thane, India Aerospike Inc. All rights reserved. 9 ]
10 Financial Services Tech Stack [ CMGI Keynote, Thane, India Aerospike Inc. All rights reserved. 10 ]
11 Fraud Detection Tech Stack [ CMGI Keynote, Thane, India Aerospike Inc. All rights reserved. 11 ]
12 Telco Tech Stack [ CMGI Keynote, Thane, India Aerospike Inc. All rights reserved. 12 ]
13 Operational Scale in Enterprises Decisioning Engine BUSINESS TRANSACTIONS LEGACY RDBMS HDFS BASED Web views ( Payments ) ( Mobile Queries ) ( Recommendation ) ( And More ) High Performance NoSQL XDR REAL-TIME BIG DATA DECISIONING 500 Business Trans per sec LEGACY DATABASE (Mainframe) DATA WAREHOUSE/ DATA LAKE X 5000 = 2.5 M Calculations per sec Database Transactions per sec [ CMGI Keynote, Thane, India Aerospike Inc. All rights reserved. 13 ]
14 Technology [ CMGI Keynote, Thane, India Aerospike Inc. All rights reserved. 14 ]
15 Architecture Overview 1) No Hotspots Distributed Hashing simplifies data partitioning 2) Smart Client 1 hop to data, load balancing 3) Shared Nothing Architecture every node is identical 4) Smart Clustering auto-sharding, auto-failover, auto-rebalancing, rack aware, rolling upgrades 5) Transactions and long-running tasks prioritized in realtime 6) XDR sync replication across data centers ensures near Zero Downtime [ CMGI Keynote, Thane, India Aerospike Inc. All rights reserved. 15 ]
16 Cluster Formation Say N1 is seed node and N3 is the paxos principal N2 and N3 send themselves in list to N1; N1 discovers them N1 sends adjacency list [N1, N2, N3] to newly discovered node N3 (and also N2) N3 discovers N2 and starts sending the cluster node list [N3, N2, N1] to N1 and N2 [ CMGI Keynote, Thane, India Aerospike Inc. All rights reserved. 16 ]
17 Distributed Hash Based Partitioning Distributed Hashing with No Hotspots Every key hashed with RIPEMD160 into an ultra efficient 20 byte (fixed length) string Hash + additional (fixed 64 bytes) data forms index entry in RAM Some bits from hash value are used to calculate the Partition ID (4096 partitions) Partition ID maps to Node ID in the cluster [ CMGI Keynote, Thane, India Aerospike Inc. All rights reserved. 17 ]
18 Data Distribution Index and data are colocated 1. Distribute workload uniformly 2. Provide predictable read/write performance 3. Scale up and down by simply adding cluster nodes 4. Rebalance data non-disruptively and efficiently Partition assignment objectives 1. Deterministic, so each node can operate by itself 2. Uniform distribution of partitions across nodes 3. Minimize partition moves during cluster changes [ CMGI Keynote, Thane, India Aerospike Inc. All rights reserved. 18 ]
19 Partition Assignment Algorithm function REPLICATION_LIST_ASSIGN(partitionid) node_hash = empty map for nodeid in succession_list: node_hash[nodeid] = NODE_HASH_COMPUTE(nodeid, partitionid) replication_list = sort_ascending(node_hash using hash) return replication_list function NODE_HASH_COMPUTE(nodeid, partitionid): nodeid_hash = fnv_1a_hash(nodeid) partition_hash = fnv_1a_hash(partitionid) return jenkins_one_at_a_time_hash(<nodeid_hash, partition_hash>) [ CMGI Keynote, Thane, India Aerospike Inc. All rights reserved. 19 ]
20 Real-Time Prioritization master replica Writing with Immediate Consistency Adding a Node transactions continue 1. Write sent to row master 2. Latch against simultaneous writes 3. Apply write to master and replica synchronously 4. Queue operations to disk 5. Signal completed transaction 6. Master merges duplicate copies (if any) 1. Cluster discovers new node via gossip protocol 2. Paxos vote determines new data organization 3. Partition migrations scheduled (only deltas copied) 4. When a partition migration starts, write journal starts on destination 5. Partition moves atomically 6. Journal is applied and source data deleted [ CMGI Keynote, Thane, India Aerospike Inc. All rights reserved. 20 ]
21 Intelligent Client The Aerospike Client is implemented as a library, JAR or DLL, and consists of 2 parts: Operation APIs These are the operations that you can execute on the cluster CRUD+ etc. First class observer of the Cluster Monitoring the state of each node and aware of new nodes or node failures. 1 Hop to data Smart Client simply calculates Partition ID to determine Node ID Client performs load balancing [ CMGI Keynote, Thane, India Aerospike Inc. All rights reserved. 21 ]
22 Designed for Wire-Line Speed Multi-core architecture Optimized C based DB kernel 1. Multi-threaded data structures 2. Nested locking model for synchronization 3. Lockless data structures 4. Partitioned single threaded data structures 5. Index entries are aligned to cache line (64 bytes) 6. Custom memory management (arenas) Memory Arena Assignment [ CMGI Keynote, Thane, India Aerospike Inc. All rights reserved. 22 ]
23 In-Memory and Hybrid-Memory Storage Architecture 1. Direct device access 2. Large Block Writes 3. Indexes in DRAM 4. Highly Parallelized Highlights 5. Log-structured FS copy-on-write 6. Fast restart with shared memory Storage Layout [ CMGI Keynote, Thane, India Aerospike Inc. All rights reserved. 23 ]
24 Benchmarks [ CMGI Keynote, Thane, India Aerospike Inc. All rights reserved. 24 ]
25 Hybrid-Memory Performance HIGH THROUGHPUT LOW LATENCY 350, ,000 Balanced Workload Read Latency Throughput, ops/second 250, , , ,000 50,000 0 Balanced Read-Heavy Average Latency, ms , , , ,000 Throughput, ops/sec Aerospike Cassandra Aerospike Balanced 50/50 read-write ratio Read-Heavy 95/5 read-write ratio [ CMGI Keynote, Thane, India Aerospike Inc. All rights reserved. 25 ]
26 In-Memory performance YCSB Benchmark 50 million records YCSB Workload A (50/50 R/W) YCSB Workload B (95/5 R/W) Zipfian key distribution 8 Core Dual Socket Intel Xeon CPU E5-2665@2.4GHz 32GB DRAM with 16 queues [ CMGI Keynote, Thane, India Aerospike Inc. All rights reserved. 26 ]
27 DRAM Vs SSD on GCE GCE instance n1-standard-8 10 node cluster 150 byte record with 3 columns 100 million records [ CMGI Keynote, Thane, India Aerospike Inc. All rights reserved. 27 ]
28 Predictable Performance During Failures Phases 1) 100KTPS 4 nodes 2) Clients at Max 3) 400KTPS 4 nodes 4) 400KTPS 3 nodes 5) 400KTPS 4 nodes Aerospike Node Specs: CentOS 6.3 Intel i5-2400@ 3.1 GHz (Quad core) 16 GB RAM@1333 MHz [ CMGI Keynote, Thane, India Aerospike Inc. All rights reserved. 28 ]
29 TCO: In-Memory Vs Hybrid-Memory Actual deployment analysis. Deployment requires 500K TPS, 10 TB of storage, with 2x replication factor. In-Memory System 186 SERVERS REQUIRED Hybrid-Memory System ONLY 14 SERVERS REQUIRED 14 SERVERS REQUIRED Storage per server 180 GB (196 GB Server) 2.4 TB (4 x 700 GB) TPS per cluster 500, ,000 Cost per server $8,000 $11,000 Server costs $1,488,000 $154,000 Power/server 0.9 kw 1.1 kw Power (2 years) $0.12 per kwh ave. US $352,000 $32,400 Maintenance (2 years) $3,600 per server $670,000 $50,400 Total $2,510,000 $236,800 [ CMGI Keynote, Thane, India Aerospike Inc. All rights reserved. 29 ]
30 Future Work Software Application Requirements New Hardware Linearize with CP Mode Eventual consistency with conflict detection and resolution Pipelined execution of client transactions for increased performance Security enhancements Customers demand Real-time decisions based on recent data High Consistency Security 3D XPoint High core CPUs NVMe Multi-queue network cards Virtualized IO 30 [ CMGI Keynote, Thane, India Aerospike Inc. All rights reserved. 30 ]
31 Thank You Questions? [ CMGI Keynote, Thane, India Aerospike Inc. All rights reserved. 31 ]
Flash-Optimized, High-Performance NoSQL Database for All
Flash-Optimized, High-Performance NoSQL Database for All Khosrow Afroozeh Engineer Aerospike Aerospike Built for the Age of Scale The Gold Standard 6 of top 16 powered by Aerospike (after Google, FB, from
More informationVOLTDB + HP VERTICA. page
VOLTDB + HP VERTICA ARCHITECTURE FOR FAST AND BIG DATA ARCHITECTURE FOR FAST + BIG DATA FAST DATA Fast Serve Analytics BIG DATA BI Reporting Fast Operational Database Streaming Analytics Columnar Analytics
More informationHave your cake, and eat it too. Strong Consistency and High Performance
Have your cake, and eat it too Strong Consistency and High Performance Brian Bulkowski, CTO & Founder March 7, 2018 Qcon London Aerospike in a nutshell Hybrid Memory Enables Digital Transformation Fast
More informationNew Oracle NoSQL Database APIs that Speed Insertion and Retrieval
New Oracle NoSQL Database APIs that Speed Insertion and Retrieval O R A C L E W H I T E P A P E R F E B R U A R Y 2 0 1 6 1 NEW ORACLE NoSQL DATABASE APIs that SPEED INSERTION AND RETRIEVAL Introduction
More information<Insert Picture Here> Oracle NoSQL Database A Distributed Key-Value Store
Oracle NoSQL Database A Distributed Key-Value Store Charles Lamb The following is intended to outline our general product direction. It is intended for information purposes only,
More informationAerospike Scales with Google Cloud Platform
Aerospike Scales with Google Cloud Platform PERFORMANCE TEST SHOW AEROSPIKE SCALES ON GOOGLE CLOUD Aerospike is an In-Memory NoSQL database and a fast Key Value Store commonly used for caching and by real-time
More informationCIS 601 Graduate Seminar. Dr. Sunnie S. Chung Dhruv Patel ( ) Kalpesh Sharma ( )
Guide: CIS 601 Graduate Seminar Presented By: Dr. Sunnie S. Chung Dhruv Patel (2652790) Kalpesh Sharma (2660576) Introduction Background Parallel Data Warehouse (PDW) Hive MongoDB Client-side Shared SQL
More informationOracle Exadata X7. Uwe Kirchhoff Oracle ACS - Delivery Senior Principal Service Delivery Engineer
Oracle Exadata X7 Uwe Kirchhoff Oracle ACS - Delivery Senior Principal Service Delivery Engineer 05.12.2017 Oracle Engineered Systems ZFS Backup Appliance Zero Data Loss Recovery Appliance Exadata Database
More informationDell PowerEdge R730xd Servers with Samsung SM1715 NVMe Drives Powers the Aerospike Fraud Prevention Benchmark
Dell PowerEdge R730xd Servers with Samsung SM1715 NVMe Drives Powers the Aerospike Fraud Prevention Benchmark Testing validation report prepared under contract with Dell Introduction As innovation drives
More informationJargons, Concepts, Scope and Systems. Key Value Stores, Document Stores, Extensible Record Stores. Overview of different scalable relational systems
Jargons, Concepts, Scope and Systems Key Value Stores, Document Stores, Extensible Record Stores Overview of different scalable relational systems Examples of different Data stores Predictions, Comparisons
More informationVoltDB for Financial Services Technical Overview
VoltDB for Financial Services Technical Overview Financial services organizations have multiple masters: regulators, investors, customers, and internal business users. All create, monitor, and require
More informationVoltDB vs. Redis Benchmark
Volt vs. Redis Benchmark Motivation and Goals of this Evaluation Compare the performance of several distributed databases that can be used for state storage in some of our applications Low latency is expected
More informationMySQL Cluster Web Scalability, % Availability. Andrew
MySQL Cluster Web Scalability, 99.999% Availability Andrew Morgan @andrewmorgan www.clusterdb.com Safe Harbour Statement The following is intended to outline our general product direction. It is intended
More informationWhen, Where & Why to Use NoSQL?
When, Where & Why to Use NoSQL? 1 Big data is becoming a big challenge for enterprises. Many organizations have built environments for transactional data with Relational Database Management Systems (RDBMS),
More informationConceptual Modeling on Tencent s Distributed Database Systems. Pan Anqun, Wang Xiaoyu, Li Haixiang Tencent Inc.
Conceptual Modeling on Tencent s Distributed Database Systems Pan Anqun, Wang Xiaoyu, Li Haixiang Tencent Inc. Outline Introduction System overview of TDSQL Conceptual Modeling on TDSQL Applications Conclusion
More informationFusion iomemory PCIe Solutions from SanDisk and Sqrll make Accumulo Hypersonic
WHITE PAPER Fusion iomemory PCIe Solutions from SanDisk and Sqrll make Accumulo Hypersonic Western Digital Technologies, Inc. 951 SanDisk Drive, Milpitas, CA 95035 www.sandisk.com Table of Contents Executive
More informationCISC 7610 Lecture 2b The beginnings of NoSQL
CISC 7610 Lecture 2b The beginnings of NoSQL Topics: Big Data Google s infrastructure Hadoop: open google infrastructure Scaling through sharding CAP theorem Amazon s Dynamo 5 V s of big data Everyone
More informationIntroduction to Database Services
Introduction to Database Services Shaun Pearce AWS Solutions Architect 2015, Amazon Web Services, Inc. or its affiliates. All rights reserved Today s agenda Why managed database services? A non-relational
More information10. Replication. Motivation
10. Replication Page 1 10. Replication Motivation Reliable and high-performance computation on a single instance of a data object is prone to failure. Replicate data to overcome single points of failure
More informationNext-Generation Cloud Platform
Next-Generation Cloud Platform Jangwoo Kim Jun 24, 2013 E-mail: jangwoo@postech.ac.kr High Performance Computing Lab Department of Computer Science & Engineering Pohang University of Science and Technology
More informationOracle TimesTen Scaleout: Revolutionizing In-Memory Transaction Processing
Oracle Scaleout: Revolutionizing In-Memory Transaction Processing Scaleout is a brand new, shared nothing scale-out in-memory database designed for next generation extreme OLTP workloads. Featuring elastic
More informationAccelerating Big Data: Using SanDisk SSDs for Apache HBase Workloads
WHITE PAPER Accelerating Big Data: Using SanDisk SSDs for Apache HBase Workloads December 2014 Western Digital Technologies, Inc. 951 SanDisk Drive, Milpitas, CA 95035 www.sandisk.com Table of Contents
More informationOracle NoSQL Database Enterprise Edition, Version 18.1
Oracle NoSQL Database Enterprise Edition, Version 18.1 Oracle NoSQL Database is a scalable, distributed NoSQL database, designed to provide highly reliable, flexible and available data management across
More information4 Myths about in-memory databases busted
4 Myths about in-memory databases busted Yiftach Shoolman Co-Founder & CTO @ Redis Labs @yiftachsh, @redislabsinc Background - Redis Created by Salvatore Sanfilippo (@antirez) OSS, in-memory NoSQL k/v
More informationMemory-Based Cloud Architectures
Memory-Based Cloud Architectures ( Or: Technical Challenges for OnDemand Business Software) Jan Schaffner Enterprise Platform and Integration Concepts Group Example: Enterprise Benchmarking -) *%'+,#$)
More informationBENCHMARK: PRELIMINARY RESULTS! JUNE 25, 2014!
BENCHMARK: PRELIMINARY RESULTS JUNE 25, 2014 Our latest benchmark test results are in. The detailed report will be published early next month, but after 6 weeks of designing and running these tests we
More informationImproving Altibase Performance with Solarflare 10GbE Server Adapters and OpenOnload
Improving Altibase Performance with Solarflare 10GbE Server Adapters and OpenOnload Summary As today s corporations process more and more data, the business ramifications of faster and more resilient database
More informationMySQL Cluster for Real Time, HA Services
MySQL Cluster for Real Time, HA Services Bill Papp (bill.papp@oracle.com) Principal MySQL Sales Consultant Oracle Agenda Overview of MySQL Cluster Design Goals, Evolution, Workloads,
More informationDATABASE SCALE WITHOUT LIMITS ON AWS
The move to cloud computing is changing the face of the computer industry, and at the heart of this change is elastic computing. Modern applications now have diverse and demanding requirements that leverage
More informationGridGain and Apache Ignite In-Memory Performance with Durability of Disk
GridGain and Apache Ignite In-Memory Performance with Durability of Disk Dmitriy Setrakyan Apache Ignite PMC GridGain Founder & CPO http://ignite.apache.org #apacheignite Agenda What is GridGain and Ignite
More informationAccelerate Database Performance and Reduce Response Times in MongoDB Humongous Environments with the LSI Nytro MegaRAID Flash Accelerator Card
Accelerate Database Performance and Reduce Response Times in MongoDB Humongous Environments with the LSI Nytro MegaRAID Flash Accelerator Card The Rise of MongoDB Summary One of today s growing database
More informationOracle Exadata: Strategy and Roadmap
Oracle Exadata: Strategy and Roadmap - New Technologies, Cloud, and On-Premises Juan Loaiza Senior Vice President, Database Systems Technologies, Oracle Safe Harbor Statement The following is intended
More informationMapR Enterprise Hadoop
2014 MapR Technologies 2014 MapR Technologies 1 MapR Enterprise Hadoop Top Ranked Cloud Leaders 500+ Customers 2014 MapR Technologies 2 Key MapR Advantage Partners Business Services APPLICATIONS & OS ANALYTICS
More informationNVMFS: A New File System Designed Specifically to Take Advantage of Nonvolatile Memory
NVMFS: A New File System Designed Specifically to Take Advantage of Nonvolatile Memory Dhananjoy Das, Sr. Systems Architect SanDisk Corp. 1 Agenda: Applications are KING! Storage landscape (Flash / NVM)
More informationCS 655 Advanced Topics in Distributed Systems
Presented by : Walid Budgaga CS 655 Advanced Topics in Distributed Systems Computer Science Department Colorado State University 1 Outline Problem Solution Approaches Comparison Conclusion 2 Problem 3
More informationCISC 7610 Lecture 5 Distributed multimedia databases. Topics: Scaling up vs out Replication Partitioning CAP Theorem NoSQL NewSQL
CISC 7610 Lecture 5 Distributed multimedia databases Topics: Scaling up vs out Replication Partitioning CAP Theorem NoSQL NewSQL Motivation YouTube receives 400 hours of video per minute That is 200M hours
More informationHyperDex. A Distributed, Searchable Key-Value Store. Robert Escriva. Department of Computer Science Cornell University
HyperDex A Distributed, Searchable Key-Value Store Robert Escriva Bernard Wong Emin Gün Sirer Department of Computer Science Cornell University School of Computer Science University of Waterloo ACM SIGCOMM
More informationDistributed File Systems II
Distributed File Systems II To do q Very-large scale: Google FS, Hadoop FS, BigTable q Next time: Naming things GFS A radically new environment NFS, etc. Independence Small Scale Variety of workloads Cooperation
More informationHP NonStop Database Solution
CHOICE - CONFIDENCE - CONSISTENCY HP NonStop Database Solution Marco Sansoni, HP NonStop Business Critical Systems 9 ottobre 2012 Agenda Introduction to HP NonStop platform HP NonStop SQL database solution
More informationTools for Social Networking Infrastructures
Tools for Social Networking Infrastructures 1 Cassandra - a decentralised structured storage system Problem : Facebook Inbox Search hundreds of millions of users distributed infrastructure inbox changes
More informationCrescando: Predictable Performance for Unpredictable Workloads
Crescando: Predictable Performance for Unpredictable Workloads G. Alonso, D. Fauser, G. Giannikis, D. Kossmann, J. Meyer, P. Unterbrunner Amadeus S.A. ETH Zurich, Systems Group (Funded by Enterprise Computing
More informationHighly Scalable, Non-RDMA NVMe Fabric. Bob Hansen,, VP System Architecture
A Cost Effective,, High g Performance,, Highly Scalable, Non-RDMA NVMe Fabric Bob Hansen,, VP System Architecture bob@apeirondata.com Storage Developers Conference, September 2015 Agenda 3 rd Platform
More informationMigrating Oracle Databases To Cassandra
BY UMAIR MANSOOB Why Cassandra Lower Cost of ownership makes it #1 choice for Big Data OLTP Applications. Unlike Oracle, Cassandra can store structured, semi-structured, and unstructured data. Cassandra
More informationIntroduction to Oracle NoSQL Database
Introduction to Oracle NoSQL Database Anand Chandak Ashutosh Naik Agenda NoSQL Background Oracle NoSQL Database Overview Technical Features & Performance Use Cases 2 Why NoSQL? 1. The four V s of Big Data
More informationTrafficDB: HERE s High Performance Shared-Memory Data Store Ricardo Fernandes, Piotr Zaczkowski, Bernd Göttler, Conor Ettinoffe, and Anis Moussa
TrafficDB: HERE s High Performance Shared-Memory Data Store Ricardo Fernandes, Piotr Zaczkowski, Bernd Göttler, Conor Ettinoffe, and Anis Moussa EPL646: Advanced Topics in Databases Christos Hadjistyllis
More informationOracle NoSQL Database Enterprise Edition, Version 18.1
Oracle NoSQL Database Enterprise Edition, Version 18.1 Oracle NoSQL Database is a scalable, distributed NoSQL database, designed to provide highly reliable, flexible and available data management across
More informationTITLE. the IT Landscape
The Impact of Hyperconverged Infrastructure on the IT Landscape 1 TITLE Drivers for adoption Lower TCO Speed and Agility Scale Easily Operational Simplicity Hyper-converged Integrated storage & compute
More informationHow do we build TiDB. a Distributed, Consistent, Scalable, SQL Database
How do we build TiDB a Distributed, Consistent, Scalable, SQL Database About me LiuQi ( 刘奇 ) JD / WandouLabs / PingCAP Co-founder / CEO of PingCAP Open-source hacker / Infrastructure software engineer
More informationMaximizing Fraud Prevention Through Disruptive Architectures Delivering speed at scale.
Maximizing Fraud Prevention Through Disruptive Architectures Delivering speed at scale. January 2016 Credit Card Fraud prevention is among the most time-sensitive and high-value of IT tasks. The databases
More informationCMU SCS CMU SCS Who: What: When: Where: Why: CMU SCS
Carnegie Mellon Univ. Dept. of Computer Science 15-415/615 - DB s C. Faloutsos A. Pavlo Lecture#23: Distributed Database Systems (R&G ch. 22) Administrivia Final Exam Who: You What: R&G Chapters 15-22
More informationTable of contents. OpenVMS scalability with Oracle Rdb. Scalability achieved through performance tuning.
OpenVMS scalability with Oracle Rdb Scalability achieved through performance tuning. Table of contents Abstract..........................................................2 From technical achievement to
More informationMegastore: Providing Scalable, Highly Available Storage for Interactive Services & Spanner: Google s Globally- Distributed Database.
Megastore: Providing Scalable, Highly Available Storage for Interactive Services & Spanner: Google s Globally- Distributed Database. Presented by Kewei Li The Problem db nosql complex legacy tuning expensive
More informationThe Google File System. Alexandru Costan
1 The Google File System Alexandru Costan Actions on Big Data 2 Storage Analysis Acquisition Handling the data stream Data structured unstructured semi-structured Results Transactions Outline File systems
More informationNoSQL BENCHMARKING AND TUNING. Nachiket Kate Santosh Kangane Ankit Lakhotia Persistent Systems Ltd. Pune, India
NoSQL BENCHMARKING AND TUNING Nachiket Kate Santosh Kangane Ankit Lakhotia Persistent Systems Ltd. Pune, India Today large variety of available NoSQL options has made it difficult for developers to choose
More informationA Non-Relational Storage Analysis
A Non-Relational Storage Analysis Cassandra & Couchbase Alexandre Fonseca, Anh Thu Vu, Peter Grman Cloud Computing - 2nd semester 2012/2013 Universitat Politècnica de Catalunya Microblogging - big data?
More informationAgenda. AWS Database Services Traditional vs AWS Data services model Amazon RDS Redshift DynamoDB ElastiCache
Databases on AWS 2017 Amazon Web Services, Inc. and its affiliates. All rights served. May not be copied, modified, or distributed in whole or in part without the express consent of Amazon Web Services,
More informationNVMe SSDs Future-proof Apache Cassandra
NVMe SSDs Future-proof Apache Cassandra Get More Insight from Datasets Too Large to Fit into Memory Overview When we scale a database either locally or in the cloud performance 1 is imperative. Without
More informationChanging Requirements for Distributed File Systems in Cloud Storage
Changing Requirements for Distributed File Systems in Cloud Storage Wesley Leggette Cleversafe Presentation Agenda r About Cleversafe r Scalability, our core driver r Object storage as basis for filesystem
More informationVEXATA FOR ORACLE. Digital Business Demands Performance and Scale. Solution Brief
Digital Business Demands Performance and Scale As enterprises shift to online and softwaredriven business models, Oracle infrastructure is being pushed to run at exponentially higher scale and performance.
More informationPRESENTATION TITLE GOES HERE. Understanding Architectural Trade-offs in Object Storage Technologies
Object Storage 201 PRESENTATION TITLE GOES HERE Understanding Architectural Trade-offs in Object Storage Technologies SNIA Legal Notice The material contained in this tutorial is copyrighted by the SNIA
More informationState of the Dolphin Developing new Apps in MySQL 8
State of the Dolphin Developing new Apps in MySQL 8 Highlights of MySQL 8.0 technology updates Mark Swarbrick MySQL Principle Presales Consultant Jill Anolik MySQL Global Business Unit Israel Copyright
More informationIBM System Storage DCS3700
IBM System Storage DCS3700 Maximize performance, scalability and storage density at an affordable price Highlights Gain fast, highly dense storage capabilities at an affordable price Deliver simplified
More informationExadata Implementation Strategy
Exadata Implementation Strategy BY UMAIR MANSOOB 1 Who Am I Work as Senior Principle Engineer for an Oracle Partner Oracle Certified Administrator from Oracle 7 12c Exadata Certified Implementation Specialist
More informationPivot3 Acuity with Microsoft SQL Server Reference Architecture
Pivot3 Acuity with Microsoft SQL Server 2014 Reference Architecture How to Contact Pivot3 Pivot3, Inc. General Information: info@pivot3.com 221 West 6 th St., Suite 750 Sales: sales@pivot3.com Austin,
More informationNoSQL Databases MongoDB vs Cassandra. Kenny Huynh, Andre Chik, Kevin Vu
NoSQL Databases MongoDB vs Cassandra Kenny Huynh, Andre Chik, Kevin Vu Introduction - Relational database model - Concept developed in 1970 - Inefficient - NoSQL - Concept introduced in 1980 - Related
More informationMySQL High Availability. Michael Messina Senior Managing Consultant, Rolta-AdvizeX /
MySQL High Availability Michael Messina Senior Managing Consultant, Rolta-AdvizeX mmessina@advizex.com / mike.messina@rolta.com Introduction Michael Messina Senior Managing Consultant Rolta-AdvizeX, Working
More informationA Brief Introduction of TiDB. Dongxu (Edward) Huang CTO, PingCAP
A Brief Introduction of TiDB Dongxu (Edward) Huang CTO, PingCAP About me Dongxu (Edward) Huang, Cofounder & CTO of PingCAP PingCAP, based in Beijing, China. Infrastructure software engineer, open source
More informationMySQL & NoSQL: The Best of Both Worlds
MySQL & NoSQL: The Best of Both Worlds Mario Beck Principal Sales Consultant MySQL mario.beck@oracle.com 1 Copyright 2012, Oracle and/or its affiliates. All rights Safe Harbour Statement The following
More informationCourse Content MongoDB
Course Content MongoDB 1. Course introduction and mongodb Essentials (basics) 2. Introduction to NoSQL databases What is NoSQL? Why NoSQL? Difference Between RDBMS and NoSQL Databases Benefits of NoSQL
More information<Insert Picture Here> MySQL Web Reference Architectures Building Massively Scalable Web Infrastructure
MySQL Web Reference Architectures Building Massively Scalable Web Infrastructure Mario Beck (mario.beck@oracle.com) Principal Sales Consultant MySQL Session Agenda Requirements for
More informationAchieving the Potential of a Fully Distributed Storage System
Achieving the Potential of a Fully Distributed Storage System HPCN Workshop 2013, DLR Braunschweig, 7-8 May 2013 Slide 1 Scality Quick Facts Founded 2009 Experienced management team HQ in the San Francisco,
More informationA Gentle Introduction to Ceph
A Gentle Introduction to Ceph Narrated by Tim Serong tserong@suse.com Adapted from a longer work by Lars Marowsky-Brée lmb@suse.com Once upon a time there was a Free and Open Source distributed storage
More informationDataON and Intel Select Hyper-Converged Infrastructure (HCI) Maximizes IOPS Performance for Windows Server Software-Defined Storage
Solution Brief DataON and Intel Select Hyper-Converged Infrastructure (HCI) Maximizes IOPS Performance for Windows Server Software-Defined Storage DataON Next-Generation All NVMe SSD Flash-Based Hyper-Converged
More informationCloud Computing with FPGA-based NVMe SSDs
Cloud Computing with FPGA-based NVMe SSDs Bharadwaj Pudipeddi, CTO NVXL Santa Clara, CA 1 Choice of NVMe Controllers ASIC NVMe: Fully off-loaded, consistent performance, M.2 or U.2 form factor ASIC OpenChannel:
More informationNoSQL Performance Test
bankmark UG (haftungsbeschränkt) Bahnhofstraße 1 9432 Passau Germany www.bankmark.de info@bankmark.de T +49 851 25 49 49 F +49 851 25 49 499 NoSQL Performance Test In-Memory Performance Comparison of SequoiaDB,
More informationCopyright 2013, Oracle and/or its affiliates. All rights reserved.
1 Oracle NoSQL Database: Release 3.0 What s new and why you care Dave Segleau NoSQL Product Manager The following is intended to outline our general product direction. It is intended for information purposes
More informationSão Paulo. August,
São Paulo August, 28 2018 A Modernização das Soluções de Armazeamento e Proteção de Dados DellEMC Mateus Pereira Systems Engineer, DellEMC mateus.pereira@dell.com Need for Transformation 81% of customers
More informationDEMYSTIFYING BIG DATA WITH RIAK USE CASES. Martin Schneider Basho Technologies!
DEMYSTIFYING BIG DATA WITH RIAK USE CASES Martin Schneider Basho Technologies! Agenda Defining Big Data in Regards to Riak A Series of Trade-Offs Use Cases Q & A About Basho & Riak Basho Technologies is
More informationApril 21, 2017 Revision GridDB Reliability and Robustness
April 21, 2017 Revision 1.0.6 GridDB Reliability and Robustness Table of Contents Executive Summary... 2 Introduction... 2 Reliability Features... 2 Hybrid Cluster Management Architecture... 3 Partition
More informationNewSQL Databases. The reference Big Data stack
Università degli Studi di Roma Tor Vergata Dipartimento di Ingegneria Civile e Ingegneria Informatica NewSQL Databases Corso di Sistemi e Architetture per Big Data A.A. 2017/18 Valeria Cardellini The reference
More informationSpanner: Google's Globally-Distributed Database. Presented by Maciej Swiech
Spanner: Google's Globally-Distributed Database Presented by Maciej Swiech What is Spanner? "...Google's scalable, multi-version, globallydistributed, and synchronously replicated database." What is Spanner?
More informationDatabase Architecture 2 & Storage. Instructor: Matei Zaharia cs245.stanford.edu
Database Architecture 2 & Storage Instructor: Matei Zaharia cs245.stanford.edu Summary from Last Time System R mostly matched the architecture of a modern RDBMS» SQL» Many storage & access methods» Cost-based
More information5 Fundamental Strategies for Building a Data-centered Data Center
5 Fundamental Strategies for Building a Data-centered Data Center June 3, 2014 Ken Krupa, Chief Field Architect Gary Vidal, Solutions Specialist Last generation Reference Data Unstructured OLTP Warehouse
More informationNEC Express5800 A2040b 22TB Data Warehouse Fast Track. Reference Architecture with SW mirrored HGST FlashMAX III
NEC Express5800 A2040b 22TB Data Warehouse Fast Track Reference Architecture with SW mirrored HGST FlashMAX III Based on Microsoft SQL Server 2014 Data Warehouse Fast Track (DWFT) Reference Architecture
More informationAn Intelligent & Optimized Way to Access Flash Storage Increase Performance & Scalability of Your Applications
An Intelligent & Optimized Way to Access Flash Storage Increase Performance & Scalability of Your Applications Saeed Raja Director Product Management Enterprise Storage Solutions Corporation August 6,
More informationPerformance Benefits of Running RocksDB on Samsung NVMe SSDs
Performance Benefits of Running RocksDB on Samsung NVMe SSDs A Detailed Analysis 25 Samsung Semiconductor Inc. Executive Summary The industry has been experiencing an exponential data explosion over the
More informationCascade Mapping: Optimizing Memory Efficiency for Flash-based Key-value Caching
Cascade Mapping: Optimizing Memory Efficiency for Flash-based Key-value Caching Kefei Wang and Feng Chen Louisiana State University SoCC '18 Carlsbad, CA Key-value Systems in Internet Services Key-value
More informationBig and Fast. Anti-Caching in OLTP Systems. Justin DeBrabant
Big and Fast Anti-Caching in OLTP Systems Justin DeBrabant Online Transaction Processing transaction-oriented small footprint write-intensive 2 A bit of history 3 OLTP Through the Years relational model
More informationRIGHTNOW A C E
RIGHTNOW A C E 2 0 1 4 2014 Aras 1 A C E 2 0 1 4 Scalability Test Projects Understanding the results 2014 Aras Overview Original Use Case Scalability vs Performance Scale to? Scaling the Database Server
More informationDrilling Through The Stack. Burkhard Neidecker-Lutz Technical Director, SAP Research SAP AG
Drilling Through The Stack Burkhard Neidecker-Lutz Technical Director, SAP Research SAP AG Talk Outline Where I come from Layered stacks Violating layers for fun and profit The mother of all stacks: Future
More informationCA485 Ray Walshe Google File System
Google File System Overview Google File System is scalable, distributed file system on inexpensive commodity hardware that provides: Fault Tolerance File system runs on hundreds or thousands of storage
More informationWindows Servers In Microsoft Azure
$6/Month Windows Servers In Microsoft Azure What I m Going Over 1. How inexpensive servers in Microsoft Azure are 2. How I get Windows servers for $6/month 3. Why Azure hosted servers are way better 4.
More informationCIT 668: System Architecture. Distributed Databases
CIT 668: System Architecture Distributed Databases Topics 1. MySQL 2. Concurrency 3. Transactions and ACID 4. Database scaling 5. Replication 6. Partitioning 7. Brewer s CAP Theorem 8. ACID vs. BASE 9.
More informationIntroduction to Distributed Data Systems
Introduction to Distributed Data Systems Serge Abiteboul Ioana Manolescu Philippe Rigaux Marie-Christine Rousset Pierre Senellart Web Data Management and Distribution http://webdam.inria.fr/textbook January
More informationSCYLLA: NoSQL at Ludicrous Speed. 主讲人 :ScyllaDB 软件工程师贺俊
SCYLLA: NoSQL at Ludicrous Speed 主讲人 :ScyllaDB 软件工程师贺俊 Today we will cover: + Intro: Who we are, what we do, who uses it + Why we started ScyllaDB + Why should you care + How we made design decisions to
More informationIBM Spectrum NAS. Easy-to-manage software-defined file storage for the enterprise. Overview. Highlights
IBM Spectrum NAS Easy-to-manage software-defined file storage for the enterprise Highlights Reduce capital expenditures with storage software on commodity servers Improve efficiency by consolidating all
More informationYCSB++ benchmarking tool Performance debugging advanced features of scalable table stores
YCSB++ benchmarking tool Performance debugging advanced features of scalable table stores Swapnil Patil M. Polte, W. Tantisiriroj, K. Ren, L.Xiao, J. Lopez, G.Gibson, A. Fuchs *, B. Rinaldi * Carnegie
More informationEMC Virtual Infrastructure for Microsoft Applications Data Center Solution
EMC Virtual Infrastructure for Microsoft Applications Data Center Solution Enabled by EMC Symmetrix V-Max and Reference Architecture EMC Global Solutions Copyright and Trademark Information Copyright 2009
More informationFAWN. A Fast Array of Wimpy Nodes. David Andersen, Jason Franklin, Michael Kaminsky*, Amar Phanishayee, Lawrence Tan, Vijay Vasudevan
FAWN A Fast Array of Wimpy Nodes David Andersen, Jason Franklin, Michael Kaminsky*, Amar Phanishayee, Lawrence Tan, Vijay Vasudevan Carnegie Mellon University *Intel Labs Pittsburgh Energy in computing
More informationPerformance comparisons and trade-offs for various MySQL replication schemes
Performance comparisons and trade-offs for various MySQL replication schemes Darpan Dinker VP Engineering Brian O Krafka, Chief Architect Schooner Information Technology, Inc. http://www.schoonerinfotech.com/
More information