Building High Performance Apps using NoSQL. Swami Sivasubramanian General Manager, AWS NoSQL
|
|
- Elvin Brooks
- 6 years ago
- Views:
Transcription
1 Building High Performance Apps using NoSQL Swami Sivasubramanian General Manager, AWS NoSQL
2 Building high performance apps There is a lot to building high performance apps Scalability Performance at high percentiles Availability Database choice on has disproportionate impact on these
3 History of Databases in Amazon Amazon.com is a big startup Composed of thousands of service Each service is developed and operated independently No central mandate or coordination (slows down execution..) A vast ecosystem.. Survival of the fittest! We have gone through multiple iterations on the following question What is the right database architecture for Amazon apps?
4 Relational Era Amazon.com page composed of responses from 1000s of independent services Query patterns for different service are different Catalog service is usually heavy key-value Ordering service is very write intensive (key-value) Catalog search has a different pattern for querying However: Usually, a relational database is the default database choice!
5 Relational Era (contd..) How did it go? Poor availability Poor Scalability (Q4/Christmas was a big project) Exorbitantly high costs for hardware, software and administration
6 Lessons Relational Database is used even when they are not the right tool! Didn t need all the query capabilities RDBMS provide Need a database that can provide: Extreme availability Seamless scalability: No more re-architecture for planning Embrace failures and make it part of normal operations (Hey, this was early 2000s when these were not obvious)
7 Distributed Systems Era: Amazon Dynamo Replicated DHT with consistency management Consistent hashing Optimistic replication Sloppy quorum Anti-entropy mechanisms Object versioning Specialist tool: Limited query capabilities Simpler consistency
8 Amazon Dynamo Usage Higher Availability Incremental Scalability Lower costs.. however less query capability Adopted by services for which scale and availability are most important Dynamo inspired many other internal variants for distributed caching, messaging etc.. Services that needed complex queries used RDBMS
9 Amazon Dynamo: Lessons learned What could have been better? Lack of strong consistency Forced a model which may not fit every app We forced every engineer to learn distributed systems Version clocks, sloppy quorum, anti-entropy, Cluster balancing, Operational complexity Required each service to carry pagers Manage their fleet Deal with performance tuning In the end, Amazon developers wanted Dynamo as a service not a product!
10 Cloud Era Time when AWS was just starting Developers loved: Why? Amazon S3 for storage Amazon EC2 for compute Lets them focus on their app Not deal with operations They wanted equivalent of S3 for databases Seamless scalability No operational overhead
11 Cloud Era: Amazon DynamoDB Non- Rela)onal Fast & Predictable Performance Seamless Scalability Easy Administra)on
12 We built DynamoDB to make developers life easier
13 Where is Amazon.com right now? NoSQL (DynamoDB) has been a huge central piece for Amazon Most of the online workloads are using DynamoDB No other solution meets our scale, availability and cost needs We use other cloud databases too! RDS for relational workloads ElastiCache for caching Redshift for warehouse applications EMR for analytics
14 So much for Amazon.com, what about AWS and its customers?
15 State of NoSQL in AWS: Brief Recap
16 DynamoDB: Looking ahead We will continue to invest in making sure DynamoDB continues to be Secure Extremely reliable Three datacenter replication Synchronous replication Extremely well tested replication pipeline No compromise on reliability for costs or performance Seamlessly scalable Cost effective Launched in April: 75% price drop for storage + 35% drop for throughput Reserved capacity options: 1-year = 53% discount; 3-year = 76% discount 4KB read capacity units
17 DynamoDB: Looking ahead (contd..) We will continue to improve query capabilities Launched Local secondary indexes (April 2013) Launched Parallel Scan API (May 2013) Launched geospatial indexing library today! Lot more to come.. We will continue to reduce your operational overhead Example: Dynamic DynamoDB, autoscale-dynamodb, etc.. We will continue to integrate with other AWS services seamlessly EMR integration One click copy to Redshift (Feb 2013) Data Pipeline template to backup/restore (Mar 2013) More to come..
18 ElastiCache Managed caching service Offered memcache as a service Added Redis support yesteday! Lookout for more caching features here..!
19 Run your own database on EC2 There is a rich ecosytem of NoSQL solutions in EC2 MongoDB Cassandra Riak Graph databases Pick the right solutions based on your needs.
20 Getting back to original question
21 How do I choose the right database for my app?
22 So many choices, what to pick? Choose the right tool for each job.
23 Redux.. Decision point #1: Optimize Query patterns Decision point #2: Plan for (business) success Decision point #3: Plan for (infrastructure) failures Decision point #4: What is the operational expense for my pick?
24 Decision #1: Choosing right query patterns Understand your apps s query pattern carefully Identify which queries need to scale linearly with growth in user base For those queries, pick a database architecture that scales linearly Perf should be same for 10MB table or 10GB table or 10TB table. If your db does not grow with your business growth Signing up for operational hell Don t think about sharding as after thought
25 Decision #1: Choosing right query patterns (contd..) Separate query patterns carefully Interactive part of your apps need to perform well and scale Avoid non-scalable queries in interactive user workflow Good real-time query Example: Load user preferences, set user preferences Bad real-time query Example: Compute all friends of friends for user A who are interested in X Perform complex queries, pre-compute and store in a cache Example: Compute recommendations for user-a and store in a cache
26 Optimize Query Patterns For time series data Separate cold data from hot data Enables you to separate read heavy workload from write heavy workload Example: Ordering application is a great example for time series data Past few days orders are hot 6 month old orders are cold Recommendation: Create an ordering table every week Store recent orders in this week s table Archive the old tables or dial down their read throughput You can query across tables
27 Decision #2: Plan for success Understand scale needs Talk to your CFO/product visionary/business owner What does success look like? Don t postpone tough decisions until you are successful Re-architecting while dealing with growth is a pain Pick query flexibility vs. scalability carefully Don t take shortcuts Plan to sleep well for other 51 weekends
28 Decision #2: Plan for success (contd..) Test for scale You will find strange bottlenecks in these tests Connection timeouts Cluster reconfiguration issues Load balancing.. Test how system scales More throughput capacity (for DynamoDB) More cache nodes (for elasticache) More ec2 instances (for run your own database)
29 Decision #3: Plan for failure Do not treat failure as a special case Replication and redundancy is key! Pick replication technology carefully Synchronous vs. Asynchronous Hint: If you care about your data, pick synchronous replication Multi-AZ vs. Single-AZ Hint: If you care about availability, pick Multi-AZ replication Pick replication factor carefully Two is a terrible number in distributed systems Three is better (and is not a crowd)
30 Decision #3: Test for failures.. Plans are only good intentions.. In DynamoDB, we test for failures Unit tests Mock tests Cluster tests Performance tests Datacenter failure tests Network degradation tests Dependency failure tests Also, we use strong theoretical foundation when necessary.. Fault injection testing is key!
31 Decision #4: What is the operational overhead? Understand the operational costs of your app Don t underestimate the cost of Managing hardware Maintaining and patching software Configuring and keeping multi-az replication Plan for repeated game days and hardware upgrades Plan for optimizing costs Plan for operations staff If a cloud service works for you and meets your needs (#1 to #3) great! If not, do it your own but plan accordingly.
32 Simple rule of thumb.. When you need seamless scale and super availability: DynamoDB Complex query workloads and need relational capabilities Choose Amazon RDS Usually MySQL is a good choice Caching ElastiCache - memcached for scaling key-value ElastiCache Redis for advanced datastructures For data warehousing: Choose Amazon Redshift Cases where these services are not the right fit: Build your own on EC2!
33 Thank you!
Agenda. AWS Database Services Traditional vs AWS Data services model Amazon RDS Redshift DynamoDB ElastiCache
Databases on AWS 2017 Amazon Web Services, Inc. and its affiliates. All rights served. May not be copied, modified, or distributed in whole or in part without the express consent of Amazon Web Services,
More informationIntroduction to Database Services
Introduction to Database Services Shaun Pearce AWS Solutions Architect 2015, Amazon Web Services, Inc. or its affiliates. All rights reserved Today s agenda Why managed database services? A non-relational
More informationSpotify. Scaling storage to million of users world wide. Jimmy Mårdell October 14, 2014
Cassandra @ Spotify Scaling storage to million of users world wide! Jimmy Mårdell October 14, 2014 2 About me Jimmy Mårdell Tech Product Owner in the Cassandra team 4 years at Spotify
More informationManaging IoT and Time Series Data with Amazon ElastiCache for Redis
Managing IoT and Time Series Data with ElastiCache for Redis Darin Briskman, ElastiCache Developer Outreach Michael Labib, Specialist Solutions Architect 2016, Web Services, Inc. or its Affiliates. All
More informationA Journey to DynamoDB
A Journey to DynamoDB and maybe away from DynamoDB Adam Dockter VP of Engineering ServiceTarget Who are we? Small Company 4 Developers AWS Infrastructure NO QA!! About our product Self service web application
More informationARCHITECTING WEB APPLICATIONS FOR THE CLOUD: DESIGN PRINCIPLES AND PRACTICAL GUIDANCE FOR AWS
ARCHITECTING WEB APPLICATIONS FOR THE CLOUD: DESIGN PRINCIPLES AND PRACTICAL GUIDANCE FOR AWS Dr Adnene Guabtni, Senior Research Scientist, NICTA/Data61, CSIRO Adnene.Guabtni@csiro.au EC2 S3 ELB RDS AMI
More information/ Cloud Computing. Recitation 10 March 22nd, 2016
15-319 / 15-619 Cloud Computing Recitation 10 March 22nd, 2016 Overview Administrative issues Office Hours, Piazza guidelines Last week s reflection Project 3.3, OLI Unit 4, Module 15, Quiz 8 This week
More information/ Cloud Computing. Recitation 8 October 18, 2016
15-319 / 15-619 Cloud Computing Recitation 8 October 18, 2016 1 Overview Administrative issues Office Hours, Piazza guidelines Last week s reflection Project 3.2, OLI Unit 3, Module 13, Quiz 6 This week
More informationCloud Analytics and Business Intelligence on AWS
Cloud Analytics and Business Intelligence on AWS Enterprise Applications Virtual Desktops Sharing & Collaboration Platform Services Analytics Hadoop Real-time Streaming Data Machine Learning Data Warehouse
More informationDeep Dive Amazon Kinesis. Ian Meyers, Principal Solution Architect - Amazon Web Services
Deep Dive Amazon Kinesis Ian Meyers, Principal Solution Architect - Amazon Web Services Analytics Deployment & Administration App Services Analytics Compute Storage Database Networking AWS Global Infrastructure
More informationLarge-Scale Web Applications
Large-Scale Web Applications Mendel Rosenblum Web Application Architecture Web Browser Web Server / Application server Storage System HTTP Internet CS142 Lecture Notes - Intro LAN 2 Large-Scale: Scale-Out
More informationAmazon Aurora Relational databases reimagined.
Amazon Aurora Relational databases reimagined. Ronan Guilfoyle, Solutions Architect, AWS Brian Scanlan, Engineer, Intercom 2015, Amazon Web Services, Inc. or its affiliates. All rights reserved Current
More informationCloud Storage with AWS: EFS vs EBS vs S3 AHMAD KARAWASH
Cloud Storage with AWS: EFS vs EBS vs S3 AHMAD KARAWASH Cloud Storage with AWS Cloud storage is a critical component of cloud computing, holding the information used by applications. Big data analytics,
More informationBeating the Final Boss: Launch your game!
Beating the Final Boss: Launch your game! Ozkan Can Solutions Architect, AWS @_ozkancan ERROR The servers are busy at this time. Please try again later. (Error Code: 42 OOPS) Retry READY FOR LAUNCH?! WORST-CASE
More information2013 AWS Worldwide Public Sector Summit Washington, D.C.
2013 AWS Worldwide Public Sector Summit Washington, D.C. EMR for Fun and for Profit Ben Butler Sr. Manager, Big Data butlerb@amazon.com @bensbutler Overview 1. What is big data? 2. What is AWS Elastic
More informationBERLIN. 2015, Amazon Web Services, Inc. or its affiliates. All rights reserved
BERLIN 2015, Amazon Web Services, Inc. or its affiliates. All rights reserved Amazon Aurora: Amazon s New Relational Database Engine Carlos Conde Technology Evangelist @caarlco 2015, Amazon Web Services,
More informationAccelerate MySQL for Demanding OLAP and OLTP Use Case with Apache Ignite December 7, 2016
Accelerate MySQL for Demanding OLAP and OLTP Use Case with Apache Ignite December 7, 2016 Nikita Ivanov CTO and Co-Founder GridGain Systems Peter Zaitsev CEO and Co-Founder Percona About the Presentation
More informationGabriel Villa. Architecting an Analytics Solution on AWS
Gabriel Villa Architecting an Analytics Solution on AWS Cloud and Data Architect Skilled leader, solution architect, and technical expert focusing primarily on Microsoft technologies and AWS. Passionate
More informationCS 655 Advanced Topics in Distributed Systems
Presented by : Walid Budgaga CS 655 Advanced Topics in Distributed Systems Computer Science Department Colorado State University 1 Outline Problem Solution Approaches Comparison Conclusion 2 Problem 3
More informationDocument Sub Title. Yotpo. Technical Overview 07/18/ Yotpo
Document Sub Title Yotpo Technical Overview 07/18/2016 2015 Yotpo Contents Introduction... 3 Yotpo Architecture... 4 Yotpo Back Office (or B2B)... 4 Yotpo On-Site Presence... 4 Technologies... 5 Real-Time
More informationAmazon ElastiCache 8/1/17. Why Amazon ElastiCache is important? Introduction:
Amazon ElastiCache Introduction: How to improve application performance using caching. What are the ElastiCache engines, and the difference between them. How to scale your cluster vertically. How to scale
More informationCluster-Level Google How we use Colossus to improve storage efficiency
Cluster-Level Storage @ Google How we use Colossus to improve storage efficiency Denis Serenyi Senior Staff Software Engineer dserenyi@google.com November 13, 2017 Keynote at the 2nd Joint International
More informationCIB Session 12th NoSQL Databases Structures
CIB Session 12th NoSQL Databases Structures By: Shahab Safaee & Morteza Zahedi Software Engineering PhD Email: safaee.shx@gmail.com, morteza.zahedi.a@gmail.com cibtrc.ir cibtrc cibtrc 2 Agenda What is
More informationAWS 101. Patrick Pierson, IonChannel
AWS 101 Patrick Pierson, IonChannel What is AWS? Amazon Web Services (AWS) is a secure cloud services platform, offering compute power, database storage, content delivery and other functionality to help
More informationAurora, RDS, or On-Prem, Which is right for you
Aurora, RDS, or On-Prem, Which is right for you Kathy Gibbs Database Specialist TAM Katgibbs@amazon.com Santa Clara, California April 23th 25th, 2018 Agenda RDS Aurora EC2 On-Premise Wrap-up/Recommendation
More informationWhen, Where & Why to Use NoSQL?
When, Where & Why to Use NoSQL? 1 Big data is becoming a big challenge for enterprises. Many organizations have built environments for transactional data with Relational Database Management Systems (RDBMS),
More informationCloud Computing & Visualization
Cloud Computing & Visualization Workflows Distributed Computation with Spark Data Warehousing with Redshift Visualization with Tableau #FIUSCIS School of Computing & Information Sciences, Florida International
More informationBig Data on AWS. Peter-Mark Verwoerd Solutions Architect
Big Data on AWS Peter-Mark Verwoerd Solutions Architect What to get out of this talk Non-technical: Big Data processing stages: ingest, store, process, visualize Hot vs. Cold data Low latency processing
More informationCIT 668: System Architecture. Amazon Web Services
CIT 668: System Architecture Amazon Web Services Topics 1. AWS Global Infrastructure 2. Foundation Services 1. Compute 2. Storage 3. Database 4. Network 3. AWS Economics Amazon Services Architecture Regions
More informationAmazon Linux: Operating System of the Cloud
Amazon Linux: Operating System of the Cloud Chris Schlaeger Director, Kernel and Operating Systems Managing Director, Amazon Development Center Germany GmbH How did Amazon get into Cloud Computing? We
More informationAbout Intellipaat. About the Course. Why Take This Course?
About Intellipaat Intellipaat is a fast growing professional training provider that is offering training in over 150 most sought-after tools and technologies. We have a learner base of 600,000 in over
More informationCACHE ME IF YOU CAN! GETTING STARTED WITH AMAZON ELASTICACHE. AWS Charlotte Meetup / Charlotte Cloud Computing Meetup Bilal Soylu October 2013
1 CACHE ME IF YOU CAN! GETTING STARTED WITH AMAZON ELASTICACHE AWS Charlotte Meetup / Charlotte Cloud Computing Meetup Bilal Soylu October 2013 2 Agenda Hola! Housekeeping What is this use case What is
More informationNext-Generation Cloud Platform
Next-Generation Cloud Platform Jangwoo Kim Jun 24, 2013 E-mail: jangwoo@postech.ac.kr High Performance Computing Lab Department of Computer Science & Engineering Pohang University of Science and Technology
More information/ Cloud Computing. Recitation 9 March 17th and 19th, 2015
15-319 / 15-619 Cloud Computing Recitation 9 March 17th and 19th, 2015 Overview Administrative issues Tagging, 15619Project Last week s reflection Project 3.2 This week s schedule Project 3.3 Unit 4 -
More informationHow can you implement this through a script that a scheduling daemon runs daily on the application servers?
You ve been tasked with implementing an automated data backup solution for your application servers that run on Amazon EC2 with Amazon EBS volumes. You want to use a distributed data store for your backups
More informationWhite Paper Amazon Aurora A Fast, Affordable and Powerful RDBMS
White Paper Amazon Aurora A Fast, Affordable and Powerful RDBMS TABLE OF CONTENTS Introduction 3 Multi-Tenant Logging and Storage Layer with Service-Oriented Architecture 3 High Availability with Self-Healing
More informationLean & Mean on AWS: Cost-Effective Architectures. Constantin Gonzalez, Solutions Architect, AWS
Lean & Mean on AWS: Cost-Effective Architectures Constantin Gonzalez, Solutions Architect, AWS What you ll get out of this session A lower AWS bill A more scalable, robust, dynamic architecture More time
More informationSQL, NoSQL, MongoDB. CSE-291 (Cloud Computing) Fall 2016 Gregory Kesden
SQL, NoSQL, MongoDB CSE-291 (Cloud Computing) Fall 2016 Gregory Kesden SQL Databases Really better called Relational Databases Key construct is the Relation, a.k.a. the table Rows represent records Columns
More informationAmazon Web Services 101 April 17 th, 2014 Joel Williams Solutions Architect. Amazon.com, Inc. and its affiliates. All rights reserved.
Amazon Web Services 101 April 17 th, 2014 Joel Williams Solutions Architect Amazon.com, Inc. and its affiliates. All rights reserved. Learning about Cloud Computing with AWS What is Cloud Computing and
More informationAWS Solutions Architect Associate (SAA-C01) Sample Exam Questions
1) A company is storing an access key (access key ID and secret access key) in a text file on a custom AMI. The company uses the access key to access DynamoDB tables from instances created from the AMI.
More informationNoSQL Databases MongoDB vs Cassandra. Kenny Huynh, Andre Chik, Kevin Vu
NoSQL Databases MongoDB vs Cassandra Kenny Huynh, Andre Chik, Kevin Vu Introduction - Relational database model - Concept developed in 1970 - Inefficient - NoSQL - Concept introduced in 1980 - Related
More informationTour of Database Platforms as a Service. June 2016 Warner Chaves Christo Kutrovsky Solutions Architect
Tour of Database Platforms as a Service June 2016 Warner Chaves Christo Kutrovsky Solutions Architect Bio Solutions Architect at Pythian Specialize high performance data processing and analytics 15 years
More informationDEMYSTIFYING BIG DATA WITH RIAK USE CASES. Martin Schneider Basho Technologies!
DEMYSTIFYING BIG DATA WITH RIAK USE CASES Martin Schneider Basho Technologies! Agenda Defining Big Data in Regards to Riak A Series of Trade-Offs Use Cases Q & A About Basho & Riak Basho Technologies is
More informationAmazon. Exam Questions AWS-Certified-Solutions-Architect- Professional. AWS-Certified-Solutions-Architect-Professional.
Amazon Exam Questions AWS-Certified-Solutions-Architect- Professional AWS-Certified-Solutions-Architect-Professional Version:Demo 1.. The MySecureData company has five branches across the globe. They want
More informationBest Practices and Performance Tuning on Amazon Elastic MapReduce
Best Practices and Performance Tuning on Amazon Elastic MapReduce Michael Hanisch Solutions Architect Amo Abeyaratne Big Data and Analytics Consultant ANZ 12.04.2016 2016, Amazon Web Services, Inc. or
More information4 Myths about in-memory databases busted
4 Myths about in-memory databases busted Yiftach Shoolman Co-Founder & CTO @ Redis Labs @yiftachsh, @redislabsinc Background - Redis Created by Salvatore Sanfilippo (@antirez) OSS, in-memory NoSQL k/v
More informationEXAM - AWS-Solution-Architect- Associate. AWS Certified Solutions Architect - Associate. Buy Full Product
Amazon EXAM - AWS-Solution-Architect- Associate AWS Certified Solutions Architect - Associate Buy Full Product http://www.examskey.com/aws-solution-architect- Associate.html Examskey Amazon AWS-Solution-Architect-Associate
More informationCS / Cloud Computing. Recitation 11 November 5 th and Nov 8 th, 2013
CS15-319 / 15-619 Cloud Computing Recitation 11 November 5 th and Nov 8 th, 2013 Announcements Encounter a general bug: Post on Piazza Encounter a grading bug: Post Privately on Piazza Don t ask if my
More information5 Fundamental Strategies for Building a Data-centered Data Center
5 Fundamental Strategies for Building a Data-centered Data Center June 3, 2014 Ken Krupa, Chief Field Architect Gary Vidal, Solutions Specialist Last generation Reference Data Unstructured OLTP Warehouse
More informationUnlimited Scalability in the Cloud A Case Study of Migration to Amazon DynamoDB
Unlimited Scalability in the Cloud A Case Study of Migration to Amazon DynamoDB Steve Saporta CTO, SpinCar Mar 19, 2016 SpinCar When a web-based business grows... More customers = more transactions More
More informationSAA-C01. AWS Solutions Architect Associate. Exam Summary Syllabus Questions
SAA-C01 AWS Solutions Architect Associate Exam Summary Syllabus Questions Table of Contents Introduction to SAA-C01 Exam on AWS Solutions Architect Associate... 2 AWS SAA-C01 Certification Details:...
More informationAerospike Scales with Google Cloud Platform
Aerospike Scales with Google Cloud Platform PERFORMANCE TEST SHOW AEROSPIKE SCALES ON GOOGLE CLOUD Aerospike is an In-Memory NoSQL database and a fast Key Value Store commonly used for caching and by real-time
More informationScaling on AWS. From 1 to 10 Million Users. Matthias Jung, Solutions Architect
Berlin 2015 Scaling on AWS From 1 to 10 Million Users Matthias Jung, Solutions Architect AWS @jungmats How to Scale? lot of results not the right starting point What is the right starting point? First
More informationAmazon AWS-Solution-Architect-Associate Exam
Volume: 858 Questions Question: 1 You are trying to launch an EC2 instance, however the instance seems to go into a terminated status immediately. What would probably not be a reason that this is happening?
More informationSTATE OF MODERN APPLICATIONS IN THE CLOUD
STATE OF MODERN APPLICATIONS IN THE CLOUD 2017 Introduction The Rise of Modern Applications What is the Modern Application? Today s leading enterprises are striving to deliver high performance, highly
More informationCSE 544 Principles of Database Management Systems. Magdalena Balazinska Winter 2015 Lecture 14 NoSQL
CSE 544 Principles of Database Management Systems Magdalena Balazinska Winter 2015 Lecture 14 NoSQL References Scalable SQL and NoSQL Data Stores, Rick Cattell, SIGMOD Record, December 2010 (Vol. 39, No.
More informationPractical MySQL Performance Optimization. Peter Zaitsev, CEO, Percona July 02, 2015 Percona Technical Webinars
Practical MySQL Performance Optimization Peter Zaitsev, CEO, Percona July 02, 2015 Percona Technical Webinars In This Presentation We ll Look at how to approach Performance Optimization Discuss Practical
More informationOverview of AWS Security - Database Services
Overview of AWS Security - Database Services June 2016 (Please consult http://aws.amazon.com/security/ for the latest version of this paper) 2016, Amazon Web Services, Inc. or its affiliates. All rights
More informationDynamo: Amazon s Highly Available Key-Value Store
Dynamo: Amazon s Highly Available Key-Value Store DeCandia et al. Amazon.com Presented by Sushil CS 5204 1 Motivation A storage system that attains high availability, performance and durability Decentralized
More informationMigrating to Cassandra in the Cloud, the Netflix Way
Migrating to Cassandra in the Cloud, the Netflix Way Jason Brown - @jasobrown Senior Software Engineer, Netflix Tech History, 1998-2008 In the beginning, there was the webapp and a single database in a
More informationReactive Microservices Architecture on AWS
Reactive Microservices Architecture on AWS Sascha Möllering Solutions Architect, @sascha242, Amazon Web Services Germany GmbH Why are we here today? https://secure.flickr.com/photos/mgifford/4525333972
More informationTCO REPORT. NAS File Tiering. Economic advantages of enterprise file management
TCO REPORT NAS File Tiering Economic advantages of enterprise file management Executive Summary Every organization is under pressure to meet the exponential growth in demand for file storage capacity.
More informationPerformance Evaluation of NoSQL Databases
Performance Evaluation of NoSQL Databases A Case Study - John Klein, Ian Gorton, Neil Ernst, Patrick Donohoe, Kim Pham, Chrisjan Matser February 2015 PABS '15: Proceedings of the 1st Workshop on Performance
More informationCloud Computing. What is cloud computing. CS 537 Fall 2017
Cloud Computing CS 537 Fall 2017 What is cloud computing Illusion of infinite computing resources available on demand Scale-up for most apps Elimination of up-front commitment Small initial investment,
More informationSCALABLE DATABASES. Sergio Bossa. From Relational Databases To Polyglot Persistence.
SCALABLE DATABASES From Relational Databases To Polyglot Persistence Sergio Bossa sergio.bossa@gmail.com http://twitter.com/sbtourist About Me Software architect and engineer Gioco Digitale (online gambling
More informationMySQL In the Cloud. Migration, Best Practices, High Availability, Scaling. Peter Zaitsev CEO Los Angeles MySQL Meetup June 12 th, 2017.
MySQL In the Cloud Migration, Best Practices, High Availability, Scaling Peter Zaitsev CEO Los Angeles MySQL Meetup June 12 th, 2017 1 Let me start. With some Questions! 2 Question One How Many of you
More informationWhy NoSQL? Why Riak?
Why NoSQL? Why Riak? Justin Sheehy justin@basho.com 1 What's all of this NoSQL nonsense? Riak Voldemort HBase MongoDB Neo4j Cassandra CouchDB Membase Redis (and the list goes on...) 2 What went wrong with
More informationMega-scale Postgres How to run 1,000,000 Postgres Databases
Mega-scale Postgres How to run 1,000,000 Postgres Databases Program What is Heroku & Heroku Postgres? Organizing principles for mega-scale operations Heroku Postgres Code deployment is good, but what
More informationDistributed Key-Value Stores UCSB CS170
Distributed Key-Value Stores UCSB CS170 Overview Key-Value Stores/Storage Architecture Replication management Key Value Stores: Important system service on a cluster of machines Handle huge volumes of
More informationScaling Pinterest. Marty Weiner Level 83 Interwebz Geek
Scaling Pinterest Marty Weiner Level 83 Interwebz Geek Evolution Growth March 2010 Page views per day RackSpace 1 small Web Engine 1 small MySQL DB 1 Engineer + 2 Founders Mar 2010 Jan 2011 Jan 2012 May
More informationVOLTDB + HP VERTICA. page
VOLTDB + HP VERTICA ARCHITECTURE FOR FAST AND BIG DATA ARCHITECTURE FOR FAST + BIG DATA FAST DATA Fast Serve Analytics BIG DATA BI Reporting Fast Operational Database Streaming Analytics Columnar Analytics
More informationHuge market -- essentially all high performance databases work this way
11/5/2017 Lecture 16 -- Parallel & Distributed Databases Parallel/distributed databases: goal provide exactly the same API (SQL) and abstractions (relational tables), but partition data across a bunch
More informationExadata Implementation Strategy
Exadata Implementation Strategy BY UMAIR MANSOOB 1 Who Am I Work as Senior Principle Engineer for an Oracle Partner Oracle Certified Administrator from Oracle 7 12c Exadata Certified Implementation Specialist
More informationMicroservices on AWS. Matthias Jung, Solutions Architect AWS
Microservices on AWS Matthias Jung, Solutions Architect AWS Agenda What are Microservices? Why Microservices? Challenges of Microservices Microservices on AWS What are Microservices? What are Microservices?
More informationHyperDex. A Distributed, Searchable Key-Value Store. Robert Escriva. Department of Computer Science Cornell University
HyperDex A Distributed, Searchable Key-Value Store Robert Escriva Bernard Wong Emin Gün Sirer Department of Computer Science Cornell University School of Computer Science University of Waterloo ACM SIGCOMM
More informationData-Intensive Distributed Computing
Data-Intensive Distributed Computing CS 451/651 431/631 (Winter 2018) Part 5: Analyzing Relational Data (1/3) February 8, 2018 Jimmy Lin David R. Cheriton School of Computer Science University of Waterloo
More informationHighly Available Database Architectures in AWS. Santa Clara, California April 23th 25th, 2018 Mike Benshoof, Technical Account Manager, Percona
Highly Available Database Architectures in AWS Santa Clara, California April 23th 25th, 2018 Mike Benshoof, Technical Account Manager, Percona Hello, Percona Live Attendees! What this talk is meant to
More informationMigrating Oracle Databases To Cassandra
BY UMAIR MANSOOB Why Cassandra Lower Cost of ownership makes it #1 choice for Big Data OLTP Applications. Unlike Oracle, Cassandra can store structured, semi-structured, and unstructured data. Cassandra
More informationScaling Massive Content Stores in the Cloud. CloudExpo New York June Alfresco Founder & CTO
Scaling Massive Content Stores in the Cloud CloudExpo New York June 2016 @johnnewton Alfresco Founder & CTO Alfresco Customers Government Financial Services Healthcare Manufacturing Corporate Somewhere
More informationAWS Database Migration Service
AWS Database Migration Service Database Modernisation with Minimal Downtime John Winford Sr. Technical Program Manager May 18, 2017 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
More informationAmazon Web Services. Foundational Services for Research Computing. April Mike Kuentz, WWPS Solutions Architect
Amazon Web Services Foundational Services for Research Computing Mike Kuentz, WWPS Solutions Architect April 2017 2015, Amazon Web Services, Inc. or its Affiliates. All rights reserved. AWS Global Infrastructure
More informationBig Data on AWS. Big Data Agility and Performance Delivered in the Cloud. 2015, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
Big Data on AWS Big Data Agility and Performance Delivered in the Cloud 2015, Amazon Web Services, Inc. or its Affiliates. All rights reserved. Big Data Technologies and techniques for working productively
More informationBERLIN. 2015, Amazon Web Services, Inc. or its affiliates. All rights reserved
BERLIN 2015, Amazon Web Services, Inc. or its affiliates. All rights reserved Building Multi-Region Applications Jan Metzner, Solutions Architect Brian Wagner, Solutions Architect 2015, Amazon Web Services,
More informationContainers or Serverless? Mike Gillespie Solutions Architect, AWS Solutions Architecture
Containers or Serverless? Mike Gillespie Solutions Architect, AWS Solutions Architecture A Typical Application with Microservices Client Webapp Webapp Webapp Greeting Greeting Greeting Name Name Name Microservice
More informationUnder the Covers of DynamoDB. Steffen Krause Technology
Under the Covers of DynamoDB Steffen Krause Technology Evangelist @AWS_Aktuell skrause@amazon.de Overview 1. Getting started 2. Data modeling 3. Partitioning 4. Replication & Analytics 1 Getting started
More informationA Guide to Architecting the Active/Active Data Center
White Paper A Guide to Architecting the Active/Active Data Center 2015 ScaleArc. All Rights Reserved. White Paper The New Imperative: Architecting the Active/Active Data Center Introduction With the average
More informationNOSQL OPERATIONAL CHECKLIST
WHITEPAPER NOSQL NOSQL OPERATIONAL CHECKLIST NEW APPLICATION REQUIREMENTS ARE DRIVING A DATABASE REVOLUTION There is a new breed of high volume, highly distributed, and highly complex applications that
More informationRunning MySQL on AWS. Michael Coburn Wednesday, April 15th, 2015
Running MySQL on AWS Michael Coburn Wednesday, April 15th, 2015 Who am I? 2 Senior Architect with Percona 3 years on Friday! Canadian but I now live in Costa Rica I see 3-10 different customer environments
More informationHigh Performance NoSQL with MongoDB
High Performance NoSQL with MongoDB History of NoSQL June 11th, 2009, San Francisco, USA Johan Oskarsson (from http://last.fm/) organized a meetup to discuss advances in data storage which were all using
More informationCloud Computing. DB Special Topics Lecture (10/5/2012) Kyle Hale Maciej Swiech
Cloud Computing DB Special Topics Lecture (10/5/2012) Kyle Hale Maciej Swiech Managing servers isn t for everyone What are some prohibitive issues? (we touched on these last time) Cost (initial/operational)
More informationSQL, Scaling, and What s Unique About PostgreSQL
SQL, Scaling, and What s Unique About PostgreSQL Ozgun Erdogan Citus Data XLDB May 2018 Punch Line 1. What is unique about PostgreSQL? The extension APIs 2. PostgreSQL extensions are a game changer for
More information/ Cloud Computing. Recitation 9 March 15th, 2016
15-319 / 15-619 Cloud Computing Recitation 9 March 15th, 2016 Overview Administrative issues Office Hours, Piazza guidelines Last week s reflection Project 3.2, OLI Unit 4, Module 14, Quiz 7 This week
More informationHow to go serverless with AWS Lambda
How to go serverless with AWS Lambda Roman Plessl, nine (AWS Partner) Zürich, AWSomeDay 12. September 2018 About myself and nine Roman Plessl Working for nine as a Solution Architect, Consultant and Leader.
More informationMongoDB - a No SQL Database What you need to know as an Oracle DBA
MongoDB - a No SQL Database What you need to know as an Oracle DBA David Burnham Aims of this Presentation To introduce NoSQL database technology specifically using MongoDB as an example To enable the
More informationUsing MySQL for Distributed Database Architectures
Using MySQL for Distributed Database Architectures Peter Zaitsev CEO, Percona SCALE 16x, Pasadena, CA March 9, 2018 1 About Percona Solutions for your success with MySQL,MariaDB and MongoDB Support, Managed
More informationA NoSQL Introduction for Relational Database Developers. Andrew Karcher Las Vegas SQL Saturday September 12th, 2015
A NoSQL Introduction for Relational Database Developers Andrew Karcher Las Vegas SQL Saturday September 12th, 2015 About Me http://www.andrewkarcher.com Twitter: @akarcher LinkedIn, Twitter Email: akarcher@gmail.com
More informationQLIK INTEGRATION WITH AMAZON REDSHIFT
QLIK INTEGRATION WITH AMAZON REDSHIFT Qlik Partner Engineering Created August 2016, last updated March 2017 Contents Introduction... 2 About Amazon Web Services (AWS)... 2 About Amazon Redshift... 2 Qlik
More informationStages of Data Processing
Data processing can be understood as the conversion of raw data into a meaningful and desired form. Basically, producing information that can be understood by the end user. So then, the question arises,
More informationCloud Gaming Architectures
Cloud Gaming Architectures From Social to Mobile to MMO Mark Bate, AWS Solutions Architect 2015, Amazon Web Services, Inc. or its Affiliates. All rights reserved. Gratuitous logo slide Scale to what you
More informationHow to Scale Out MySQL on EC2 or RDS. Victoria Dudin, Director R&D, ScaleBase
How to Scale Out MySQL on EC2 or RDS Victoria Dudin, Director R&D, ScaleBase Boston AWS Meetup August 11, 2014 Victoria Dudin Director of R&D, ScaleBase 15 years of product development experience Previously
More information