2013 AWS Worldwide Public Sector Summit Washington, D.C.
|
|
- Rosa Manning
- 6 years ago
- Views:
Transcription
1 2013 AWS Worldwide Public Sector Summit Washington, D.C. EMR for Fun and for Profit Ben Butler Sr. Manager, Big
2 Overview 1. What is big data? 2. What is AWS Elastic MapReduce? 3. What data is available? 4. How to use AWS EMR to support my agency s mission? 2013 AWS Worldwide Public Sector Summit
3 What is Big Data?
4 Storage Big Data Compute So what is it? When your data sets become so large that you have to start innovating around how to collect, store, organize, analyze and share it
5 Storage Big Data Compute 100 GB Challenges start at relatively small volumes 1,000 PB
6 Storage Big Data Unconstrained data growth Compute EB ZB 95% of the 1.2 zettabytes of data in the digital universe is unstructured 70% of of this is usergenerated content GB TB PB Unstructured data growth explosive, with estimates of compound annual growth (CAGR) at 62% from Source: IDC
7 Storage Big Data Where does it come from? Compute Web sites Blogs/Reviews/ s/Pictures Social Graphs Facebook, Linked-in, Contacts Application server logs Web sites, games Sensor data Weather, water, smart grids Images/videos Traffic, security cameras Twitter 50m tweets/day 1,400% growth/year
8 Why AWS and big data? Storage Innovation Compute Amazon Amazon DynamoDB RedShift Amazon S3 HPC Spot EMR
9 Storage Big Data How do you get your slice of it? Compute AWS Direct Connect AWS Import/Export Queuing Amazon Storage Gateway Dedicated low latency bandwidth Physical media shipping Highly scalable event buffering Sync local storage to the cloud
10 Storage Big Data Where do you put your slice of it? Compute AWS Relational Database Service Fully managed database (MySQL, Oracle, MSSQL) AWS SimpleDB NoSQL, Schema-less Smaller datasets AWS DynamoDB NoSQL, Schema-less, Provisioned throughput database Amazon S3 Object datastore up to 5TB per object % durability
11 Storage Big Data Where do you put your slice of it? Compute Amazon Glacier Long term cold storage From $0.01 per GB/Month % durability
12 Storage Big Data How quick do you need to read it? Compute Single digit ms 10s-100s ms <5 hours AWS DynamoDB AWS S3 AWS Glacier Social scale applications Provisioned throughput performance Flexible consistency models Any object, any app % durability Objects up to 5TB in size Performance Media & asset archives Extremely low cost S3 levels of durability Scale Price
13 Storage Big Data Operate at any scale Compute Unlimited data Performance Scale Price
14 Storage Big Data Data has gravity Compute App Data App
15 Storage Big Data and inertia at volume Compute Data
16 Storage Big Data easier to move applications to the data Compute Data
17 Storage Big Data Bring compute capacity to the data Compute Very large dataset seeks strong & consistent compute for short term relationship, possibly longer
18 Storage Big Data Flexible compute resources, on demand Compute Amazon Elastic Compute Cloud (EC2) Basic unit of compute capacity Range of CPU, memory & local disk options 17 Instance types available, from micro through cluster compute to SSD backed Vertical Scaling From $0.02/hr Feature Flexible Scalable Machine Images Full control VM Import/Export Monitoring Inexpensive Secure Details Run Windows or Linux distributions Wide range of instance types from micro to cluster compute Configurations can be saved as machine images (AMIs) from which new instances can be created Full root or administrator rights Import and export VM images to transfer configurations in and out of EC2 Publishes metrics to Cloud Watch On-demand, Reserved and Spot instance types Full firewall control via Security Groups
19 Storage Big Data Elastic capacity as you need it Compute On and Off Fast Growth Variable peaks Predictable peaks
20 Storage Big Data Compute Elastic capacity as you need it WASTE On and Off Fast Growth Variable peaks CUSTOMER DISSATISFACTION Predictable peaks
21 Storage Big Data Elastic capacity as you need it Compute Capacity Traditional IT capacity Time Elastic cloud capacity Your IT needs
22 Storage Big Data Elastic capacity as you need it Compute On and Off Fast Growth Variable peaks Predictable peaks
23 Storage Big Data From one instance Compute
24 Storage Big Data to thousands Compute
25 Why AWS and big data? Storage Innovation Compute DynamoDB S3 RedShift HPC Spot EMR
26 Why AWS and big data? Storage Innovation Compute DynamoDB S3 RedShift HPC Spot EMR
27 AWS EMR Elastic MapReduce
28 Amazon Elastic MapReduce A key tool in the toolbox to help with Big Data challenges Makes possible analytics processes previously not feasible Cost effective when leveraged with EC2 spot market Broad ecosystem of tools to handle specific use cases 2013 AWS Worldwide Public Sector Summit
29 Hadoop-as-a-service Map-Reduce engine Integrated with tools What is EMR? Massively parallel Integrated to AWS services Cost effective AWS wrapper
30 Very large click log (e.g TBs)
31 Very large click log (e.g TBs) Lots of actions by John Smith
32 Lots of actions by John Smith Very large click log (e.g TBs) Split the log into many small pieces
33 Lots of actions by John Smith Process in an EMR cluster Very large click log (e.g TBs) Split the log into many small pieces
34 Lots of actions by John Smith Process in an EMR cluster Very large click log (e.g TBs) Split the log into many small pieces Aggregate the results from all the nodes
35 Lots of actions by John Smith Process in an EMR cluster Very large click log (e.g TBs) Split the log into many small pieces Aggregate the results from all the nodes What John Smith did
36 Very large click log (e.g TBs) Insight in a fraction of the time What John Smith did
37 How does it work? 1. Put the data into S3 (or HDFS) 3. Get the results Amazon S3 Amazon EMR Cluster EMR 2. Launch your cluster. Choose: Hadoop distribution How many nodes Node type (hi-cpu, hi-memory, etc.) Hadoop apps (Hive, Pig, HBase) 2013 AWS Worldwide Public Sector Summit
38 How does it work? Amazon EMR Cluster You can easily resize the cluster Amazon S3 EMR 2013 AWS Worldwide Public Sector Summit
39 How does it work? Amazon EMR Cluster Amazon S3 EMR Use Spot nodes to save time and money 2013 AWS Worldwide Public Sector Summit
40 How does it work? Amazon S3 Amazon EMR Clusters EMR Launch parallel clusters against the same data source (tune for the workload) 2013 AWS Worldwide Public Sector Summit
41 How does it work? Amazon S3 Amazon EMR Cluster When the work is complete, you can terminate the cluster (and stop paying) 2013 AWS Worldwide Public Sector Summit
42 How does it work? Amazon EMR Cluster You can store everything in HDFS (local disk) High Storage nodes = 48 TB/node 2013 AWS Worldwide Public Sector Summit
43 How does it work? Amazon EMR Cluster Launch in a Virtual Private Cloud for extra security 2013 AWS Worldwide Public Sector Summit
44 Start an Amazon EMR cluster using AWS Management Console or AWS Command Line Interface tools Amazon EMR cluster
45 Master instance group created that controls the cluster (runs MySQL) Amazon EMR cluster Master instance group
46 Core instance group created for life of cluster Amazon EMR cluster Master instance group Core instance group
47 Core instances run DataNode and TaskTracker daemons Amazon EMR cluster Master instance group HDFS HDFS Core instance group
48 Optional task instances can be added or subtracted to perform work Amazon EMR cluster Master instance group HDFS HDFS Core instance group Task instance group
49 Amazon S3 can be used as underlying file system for input/output data Amazon EMR cluster Master instance group HDFS HDFS Core instance group Task instance group Amazon S3
50 Master node coordinates distribution of work and manages cluster state Amazon EMR cluster Master instance group HDFS HDFS Core instance group Task instance group Amazon S3
51 Core and Task instances read-write to Amazon S3 Amazon EMR cluster Master instance group HDFS HDFS Core instance group Task instance group Amazon S3
52 Amazon EC2 instance Input file map Shuffle reduce Output file
53 Amazon EC2 instance Input file map reduce Output file EC2 instance Input file map reduce Output file EC2 instance Input file map Shuffle reduce Output file
54 Pig HDFS Amazon EMR
55 HDFS Amazon EMR Amazon S3 Amazon DynamoDB
56 Data management HDFS Amazon EMR Amazon S3 Amazon DynamoDB
57 Data management Analytics languages Pig HDFS Amazon EMR Amazon S3 Amazon DynamoDB
58 Data management Analytics languages Pig HDFS Amazon EMR Amazon RDS Amazon S3 Amazon DynamoDB
59 Data management Analytics languages Pig HDFS Amazon EMR Amazon RDS Amazon RedShift AWS Data Pipeline Amazon S3 Amazon DynamoDB
60 Amazon s Public Data Sets
61 Amazon Public Data Sets 2013 AWS Worldwide Public Sector Summit
62 1000 Genomes Project 270+ TB and growing dataset, hosted for free in AWS cloud Researchers no longer need massive onpremises storage and compute Collaboration revolution: not just shared data but executable papers 2013 AWS Worldwide Public Sector Summit
63 Using EMR for your Mission
64 Cloud Democratizes Big Data/HPC Lack of constraints leads to new usage models Gives control back to individual development teams Fail-fast (and fail-cheap) opens up exploratory style Many customers create 100s of Amazon EMR clusters per day Classic burst-y workload perfect for the cloud Big data / HPC clusters themselves are parallelized resources Can you build a faster on-premises cluster? Yes, but Usually a shared/contented resource; in cloud, each user/workgroup gets their own cluster Cloud is often the fastest platform based on MTTJC (Mean Time To Job Completion)
65 Cloud and Big Data/HPC No-obligation use allows for experimentation, prototypes and operational/business pilots Faster time from inception of idea to solution Provides a platform that can scale to meet the massive needs of large data sets Bottom line: Enables experimentation and innovation without large capital investments Improves ROI for Big Data projects
66 Try the tutorial: aws.amazon.com/articles/2855 Find out more: aws.amazon.com/big-data
67 Thank you! Ben Butler, Sr. Manager, AWS Marketing
Cloud Analytics and Business Intelligence on AWS
Cloud Analytics and Business Intelligence on AWS Enterprise Applications Virtual Desktops Sharing & Collaboration Platform Services Analytics Hadoop Real-time Streaming Data Machine Learning Data Warehouse
More informationBig Data on AWS. Big Data Agility and Performance Delivered in the Cloud. 2015, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
Big Data on AWS Big Data Agility and Performance Delivered in the Cloud 2015, Amazon Web Services, Inc. or its Affiliates. All rights reserved. Big Data Technologies and techniques for working productively
More informationCIT 668: System Architecture. Amazon Web Services
CIT 668: System Architecture Amazon Web Services Topics 1. AWS Global Infrastructure 2. Foundation Services 1. Compute 2. Storage 3. Database 4. Network 3. AWS Economics Amazon Services Architecture Regions
More informationWhat is Cloud Computing? What are the Private and Public Clouds? What are IaaS, PaaS, and SaaS? What is the Amazon Web Services (AWS)?
What is Cloud Computing? What are the Private and Public Clouds? What are IaaS, PaaS, and SaaS? What is the Amazon Web Services (AWS)? What is Amazon Machine Image (AMI)? Amazon Elastic Compute Cloud (EC2)?
More informationBig Data on AWS. Peter-Mark Verwoerd Solutions Architect
Big Data on AWS Peter-Mark Verwoerd Solutions Architect What to get out of this talk Non-technical: Big Data processing stages: ingest, store, process, visualize Hot vs. Cold data Low latency processing
More informationAWS: Basic Architecture Session SUNEY SHARMA Solutions Architect: AWS
AWS: Basic Architecture Session SUNEY SHARMA Solutions Architect: AWS suneys@amazon.com AWS Core Infrastructure and Services Traditional Infrastructure Amazon Web Services Security Security Firewalls ACLs
More informationAgenda. AWS Database Services Traditional vs AWS Data services model Amazon RDS Redshift DynamoDB ElastiCache
Databases on AWS 2017 Amazon Web Services, Inc. and its affiliates. All rights served. May not be copied, modified, or distributed in whole or in part without the express consent of Amazon Web Services,
More informationBest Practices and Performance Tuning on Amazon Elastic MapReduce
Best Practices and Performance Tuning on Amazon Elastic MapReduce Michael Hanisch Solutions Architect Amo Abeyaratne Big Data and Analytics Consultant ANZ 12.04.2016 2016, Amazon Web Services, Inc. or
More informationAWS Solutions Architect Associate (SAA-C01) Sample Exam Questions
1) A company is storing an access key (access key ID and secret access key) in a text file on a custom AMI. The company uses the access key to access DynamoDB tables from instances created from the AMI.
More informationAmazon AWS-Solution-Architect-Associate Exam
Volume: 858 Questions Question: 1 You are trying to launch an EC2 instance, however the instance seems to go into a terminated status immediately. What would probably not be a reason that this is happening?
More informationCloud Computing 1. CSCI 4850/5850 High-Performance Computing Spring 2018
Cloud Computing 1 CSCI 4850/5850 High-Performance Computing Spring 2018 Tae-Hyuk (Ted) Ahn Department of Computer Science Program of Bioinformatics and Computational Biology Saint Louis University Learning
More informationIntroduction to Amazon Web Services. Jeff Barr Senior AWS /
Introduction to Amazon Web Services Jeff Barr Senior AWS Evangelist @jeffbarr / jbarr@amazon.com What Does It Take to be a Global Online Retailer? The Obvious Part And the Not-So Obvious Part How Did
More information8/3/17. Encryption and Decryption centralized Single point of contact First line of defense. Bishop
Bishop Encryption and Decryption centralized Single point of contact First line of defense If working with VPC Creation and management of security groups Provides additional networking and security options
More informationAWS Storage Optimization. AWS Whitepaper
AWS Storage Optimization AWS Whitepaper AWS Storage Optimization: AWS Whitepaper Copyright 2018 Amazon Web Services, Inc. and/or its affiliates. All rights reserved. Amazon's trademarks and trade dress
More informationIntroducing Amazon Elastic File System (EFS)
Introducing Amazon Elastic File System (EFS) Danilo Poccia, Technical Evangelist, AWS @danilop 2015, Amazon Web Services, Inc. or its affiliates. All rights reserved Goals and expectations for this session
More informationWhat s New at AWS? A selection of some new stuff. Constantin Gonzalez, Principal Solutions Architect, Amazon Web Services
What s New at AWS? A selection of some new stuff Constantin Gonzalez, Principal Solutions Architect, Amazon Web Services Speed of Innovation AWS Pace of Innovation AWS has been continually expanding its
More informationWerden Sie ein Teil von Internet der Dinge auf AWS. AWS Enterprise Summit 2015 Dr. Markus Schmidberger -
Werden Sie ein Teil von Internet der Dinge auf AWS AWS Enterprise Summit 2015 Dr. Markus Schmidberger - schmidbe@amazon.de Internet of Things is the network of physical objects or "things" embedded with
More informationCS15-319: Cloud Computing. Lecture 3 Course Project and Amazon AWS Majd Sakr and Mohammad Hammoud
CS15-319: Cloud Computing Lecture 3 Course Project and Amazon AWS Majd Sakr and Mohammad Hammoud Lecture Outline Discussion On Course Project Amazon Web Services 2 Course Project Course Project Phase I-A
More informationQLIK INTEGRATION WITH AMAZON REDSHIFT
QLIK INTEGRATION WITH AMAZON REDSHIFT Qlik Partner Engineering Created August 2016, last updated March 2017 Contents Introduction... 2 About Amazon Web Services (AWS)... 2 About Amazon Redshift... 2 Qlik
More informationIntroduction to Database Services
Introduction to Database Services Shaun Pearce AWS Solutions Architect 2015, Amazon Web Services, Inc. or its affiliates. All rights reserved Today s agenda Why managed database services? A non-relational
More informationAmazon Web Services. For Government, Education, and Nonprofit Organizations
Amazon Web Services For Government, Education, and Nonprofit Organizations Max Peterson GM EMEA, LATAM and Global Contracts maxpete@amazon.co.uk +44 (0)7342 079563 2015, Amazon Web Services, Inc. or its
More informationBuilding High Performance Apps using NoSQL. Swami Sivasubramanian General Manager, AWS NoSQL
Building High Performance Apps using NoSQL Swami Sivasubramanian General Manager, AWS NoSQL Building high performance apps There is a lot to building high performance apps Scalability Performance at high
More informationAWS Storage Gateway. Not your father s hybrid storage. University of Arizona IT Summit October 23, Jay Vagalatos, AWS Solutions Architect
AWS Storage Gateway Not your father s hybrid storage University of Arizona IT Summit 2017 Jay Vagalatos, AWS Solutions Architect October 23, 2017 The AWS Storage Portfolio Amazon EBS (persistent) Block
More informationAbout Intellipaat. About the Course. Why Take This Course?
About Intellipaat Intellipaat is a fast growing professional training provider that is offering training in over 150 most sought-after tools and technologies. We have a learner base of 600,000 in over
More informationCloud Storage with AWS: EFS vs EBS vs S3 AHMAD KARAWASH
Cloud Storage with AWS: EFS vs EBS vs S3 AHMAD KARAWASH Cloud Storage with AWS Cloud storage is a critical component of cloud computing, holding the information used by applications. Big data analytics,
More informationIntro to Big Data on AWS Igor Roiter Big Data Cloud Solution Architect
Intro to Big Data on AWS Igor Roiter Big Data Cloud Solution Architect Igor Roiter Big Data Cloud Solution Architect Working as a Data Specialist for the last 11 years 9 of them as a Consultant specializing
More informationEmbedded Technosolutions
Hadoop Big Data An Important technology in IT Sector Hadoop - Big Data Oerie 90% of the worlds data was generated in the last few years. Due to the advent of new technologies, devices, and communication
More informationIntroduction: Is Amazon Web Service (AWS) cloud supports best cost effective & high performance modern disaster recovery.
Is Amazon Web Service (AWS) cloud supports best cost effective & high performance modern disaster recovery. Name: Rajib Das Employee ID- tcs 231462 ISU-CSP Email-rajib2.d@tcs.com Introduction: Disaster
More informationCloud Computing 2. CSCI 4850/5850 High-Performance Computing Spring 2018
Cloud Computing 2 CSCI 4850/5850 High-Performance Computing Spring 2018 Tae-Hyuk (Ted) Ahn Department of Computer Science Program of Bioinformatics and Computational Biology Saint Louis University Learning
More informationTopics. Big Data Analytics What is and Why Hadoop? Comparison to other technologies Hadoop architecture Hadoop ecosystem Hadoop usage examples
Hadoop Introduction 1 Topics Big Data Analytics What is and Why Hadoop? Comparison to other technologies Hadoop architecture Hadoop ecosystem Hadoop usage examples 2 Big Data Analytics What is Big Data?
More informationBUSINESS DATA LAKE FADI FAKHOURI, SR. SYSTEMS ENGINEER, ISILON SPECIALIST. Copyright 2016 EMC Corporation. All rights reserved.
BUSINESS DATA LAKE FADI FAKHOURI, SR. SYSTEMS ENGINEER, ISILON SPECIALIST 1 UNSTRUCTURED DATA GROWTH 75% 78% 80% 2015 71 EB 2016 106 EB 2017 133 EB Total Capacity Shipped, Worldwide % of Unstructured Data
More informationNext Generation Storage for The Software-Defned World
` Next Generation Storage for The Software-Defned World John Hofer Solution Architect Red Hat, Inc. BUSINESS PAINS DEMAND NEW MODELS CLOUD ARCHITECTURES PROPRIETARY/TRADITIONAL ARCHITECTURES High up-front
More informationWhat s New at AWS? looking at just a few new things for Enterprise. Philipp Behre, Enterprise Solutions Architect, Amazon Web Services
What s New at AWS? looking at just a few new things for Enterprise Philipp Behre, Enterprise Solutions Architect, Amazon Web Services 2016, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
More informationTour of Database Platforms as a Service. June 2016 Warner Chaves Christo Kutrovsky Solutions Architect
Tour of Database Platforms as a Service June 2016 Warner Chaves Christo Kutrovsky Solutions Architect Bio Solutions Architect at Pythian Specialize high performance data processing and analytics 15 years
More informationCLOUD ECONOMICS: HOW TO QUANTIFY THE BENEFITS OF MOVING TO THE CLOUD
CLOUD ECONOMICS: HOW TO QUANTIFY THE BENEFITS OF MOVING TO THE CLOUD Matt Johnson, Solutions Architect, Amazon Web Services 30 October 2017 2017, Amazon Web Services, Inc. or its Affiliates. All rights
More informationAWS & Intel: A Partnership Dedicated to fueling your Innovations. Thomas Kellerer BDM CSP, Intel Central Europe
AWS & Intel: A Partnership Dedicated to fueling your Innovations Thomas Kellerer BDM CSP, Intel Central Europe The Digital Service Economy Growth in connected devices enables new business opportunities
More informationThoughtSpot on AWS Quick Start Guide
ThoughtSpot on AWS Quick Start Guide Version 4.2 February 2017 Table of Contents Contents Chapter 1: Welcome to ThoughtSpot...3 Contact ThoughtSpot... 4 Chapter 2: Introduction... 6 About AWS...7 Chapter
More informationAmazon Web Services. Foundational Services for Research Computing. April Mike Kuentz, WWPS Solutions Architect
Amazon Web Services Foundational Services for Research Computing Mike Kuentz, WWPS Solutions Architect April 2017 2015, Amazon Web Services, Inc. or its Affiliates. All rights reserved. AWS Global Infrastructure
More informationEnergy Management with AWS
Energy Management with AWS Kyle Hart and Nandakumar Sreenivasan Amazon Web Services August [XX], 2017 Tampa Convention Center Tampa, Florida What is Cloud? The NIST Definition Broad Network Access On-Demand
More informationProject Presentation
Project Presentation Saad Arif Dept. of Electrical Engineering and Computer Science University of Central Florida - Orlando, FL November 7, 2013 1 Introduction 1 Introduction 2 Gallery 1 Introduction 2
More informationBig Data Programming: an Introduction. Spring 2015, X. Zhang Fordham Univ.
Big Data Programming: an Introduction Spring 2015, X. Zhang Fordham Univ. Outline What the course is about? scope Introduction to big data programming Opportunity and challenge of big data Origin of Hadoop
More informationSpatial Analytics Built for Big Data Platforms
Spatial Analytics Built for Big Platforms Roberto Infante Software Development Manager, Spatial and Graph 1 Copyright 2011, Oracle and/or its affiliates. All rights Global Digital Growth The Internet of
More informationDeep Dive on Amazon Elastic File System
Deep Dive on Amazon Elastic File System Yong S. Kim AWS Business Development Manager, Amazon EFS Paul Moran Technical Account Manager, Enterprise Support 28 th of June 2017 2015, Amazon Web Services, Inc.
More informationAUTOMATING IBM SPECTRUM SCALE CLUSTER BUILDS IN AWS PROOF OF CONCEPT
AUTOMATING IBM SPECTRUM SCALE CLUSTER BUILDS IN AWS PROOF OF CONCEPT By Joshua Kwedar Sr. Systems Engineer By Steve Horan Cloud Architect ATS Innovation Center, Malvern, PA Dates: Oct December 2017 INTRODUCTION
More informationAmazon Web Services 101 April 17 th, 2014 Joel Williams Solutions Architect. Amazon.com, Inc. and its affiliates. All rights reserved.
Amazon Web Services 101 April 17 th, 2014 Joel Williams Solutions Architect Amazon.com, Inc. and its affiliates. All rights reserved. Learning about Cloud Computing with AWS What is Cloud Computing and
More informationAmazon Linux: Operating System of the Cloud
Amazon Linux: Operating System of the Cloud Chris Schlaeger Director, Kernel and Operating Systems Managing Director, Amazon Development Center Germany GmbH How did Amazon get into Cloud Computing? We
More informationLeveraging Software-Defined Storage to Meet Today and Tomorrow s Infrastructure Demands
Leveraging Software-Defined Storage to Meet Today and Tomorrow s Infrastructure Demands Unleash Your Data Center s Hidden Power September 16, 2014 Molly Rector CMO, EVP Product Management & WW Marketing
More informationAccelerate Big Data Insights
Accelerate Big Data Insights Executive Summary An abundance of information isn t always helpful when time is of the essence. In the world of big data, the ability to accelerate time-to-insight can not
More informationMiddle East Technical University. Jeren AKHOUNDI ( ) Ipek Deniz Demirtel ( ) Derya Nur Ulus ( ) CENG553 Database Management Systems
Middle East Technical University Jeren AKHOUNDI (1836345) Ipek Deniz Demirtel (1997691) Derya Nur Ulus (1899608) CENG553 Database Management Systems * Introduction to Cloud Computing * Cloud DataBase as
More informationAutomating Elasticity. March 2018
Automating Elasticity March 2018 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved. Notices This document is provided for informational purposes only. It represents AWS s current product
More informationEMC STORAGE STRATEGY. Copyright 2015 EMC Corporation. All rights reserved.
EMC STORAGE STRATEGY 1 CREATING A MAJOR PAIN POINT FOR CIO S(BI-MODAL IT) MANAGE RISK INVEST REDUCE COST WORKLOAD CHARACTERISTICS Performance Traditional Apps Sequential Stability Reliability Greater Impact
More informationSoftNAS Cloud Data Management Products for AWS Add Breakthrough NAS Performance, Protection, Flexibility
Control Any Data. Any Cloud. Anywhere. SoftNAS Cloud Data Management Products for AWS Add Breakthrough NAS Performance, Protection, Flexibility Understanding SoftNAS Cloud SoftNAS, Inc. is the #1 software-defined
More informationWhite Paper. Platform9 ROI for Hybrid Clouds
White Paper Platform9 ROI for Hybrid Clouds Quantifying the cost savings and benefits of moving from the Amazon Web Services (AWS) public cloud to the Platform9 hybrid cloud. Abstract Deciding whether
More informationDATABASE SCALE WITHOUT LIMITS ON AWS
The move to cloud computing is changing the face of the computer industry, and at the heart of this change is elastic computing. Modern applications now have diverse and demanding requirements that leverage
More informationBenefits of Extending your Datacenters with Amazon Web Services
Benefits of Extending your Datacenters with Amazon Web Services Xavier Prélat Business Development Manager @aws_actus How did Amazon.. get into cloud computing? What is AWS? Amazon Web Services offers
More informationMixApart: Decoupled Analytics for Shared Storage Systems. Madalin Mihailescu, Gokul Soundararajan, Cristiana Amza University of Toronto and NetApp
MixApart: Decoupled Analytics for Shared Storage Systems Madalin Mihailescu, Gokul Soundararajan, Cristiana Amza University of Toronto and NetApp Hadoop Pig, Hive Hadoop + Enterprise storage?! Shared storage
More informationLambda Architecture for Batch and Real- Time Processing on AWS with Spark Streaming and Spark SQL. May 2015
Lambda Architecture for Batch and Real- Time Processing on AWS with Spark Streaming and Spark SQL May 2015 2015, Amazon Web Services, Inc. or its affiliates. All rights reserved. Notices This document
More informationCloud Computing & Visualization
Cloud Computing & Visualization Workflows Distributed Computation with Spark Data Warehousing with Redshift Visualization with Tableau #FIUSCIS School of Computing & Information Sciences, Florida International
More informationHow can you implement this through a script that a scheduling daemon runs daily on the application servers?
You ve been tasked with implementing an automated data backup solution for your application servers that run on Amazon EC2 with Amazon EBS volumes. You want to use a distributed data store for your backups
More informationBackup & Recovery on AWS
EBOOK: Backup & Recovery on AWS 2017, Amazon Web Services, Inc. or its affiliates. All rights reserved. Contents Backup and Recovery on AWS AWS Object Storage Services RDX APN Storage Partner Benefits
More informationIntegrating Splunk with AWS services:
Integrating Splunk with AWS services: Using Redshi+, Elas0c Map Reduce (EMR), Amazon Machine Learning & S3 to gain ac0onable insights via predic0ve analy0cs via Splunk Patrick Shumate Solutions Architect,
More informationTowards a Real- time Processing Pipeline: Running Apache Flink on AWS
Towards a Real- time Processing Pipeline: Running Apache Flink on AWS Dr. Steffen Hausmann, Solutions Architect Michael Hanisch, Manager Solutions Architecture November 18 th, 2016 Stream Processing Challenges
More informationWhat is the maximum file size you have dealt so far? Movies/Files/Streaming video that you have used? What have you observed?
Simple to start What is the maximum file size you have dealt so far? Movies/Files/Streaming video that you have used? What have you observed? What is the maximum download speed you get? Simple computation
More informationLean & Mean on AWS: Cost-Effective Architectures. Constantin Gonzalez, Solutions Architect, AWS
Lean & Mean on AWS: Cost-Effective Architectures Constantin Gonzalez, Solutions Architect, AWS What you ll get out of this session A lower AWS bill A more scalable, robust, dynamic architecture More time
More informationCISC 7610 Lecture 2b The beginnings of NoSQL
CISC 7610 Lecture 2b The beginnings of NoSQL Topics: Big Data Google s infrastructure Hadoop: open google infrastructure Scaling through sharding CAP theorem Amazon s Dynamo 5 V s of big data Everyone
More informationAmazon Web Services Training. Training Topics:
Amazon Web Services Training Training Topics: SECTION1: INTRODUCTION TO CLOUD COMPUTING A Short history Client Server Computing Concepts Challenges with Distributed Computing Introduction to Cloud Computing
More informationBig Data with Hadoop Ecosystem
Diógenes Pires Big Data with Hadoop Ecosystem Hands-on (HBase, MySql and Hive + Power BI) Internet Live http://www.internetlivestats.com/ Introduction Business Intelligence Business Intelligence Process
More informationSTATE OF MODERN APPLICATIONS IN THE CLOUD
STATE OF MODERN APPLICATIONS IN THE CLOUD 2017 Introduction The Rise of Modern Applications What is the Modern Application? Today s leading enterprises are striving to deliver high performance, highly
More informationAWS 101. Patrick Pierson, IonChannel
AWS 101 Patrick Pierson, IonChannel What is AWS? Amazon Web Services (AWS) is a secure cloud services platform, offering compute power, database storage, content delivery and other functionality to help
More informationNext-Generation Cloud Platform
Next-Generation Cloud Platform Jangwoo Kim Jun 24, 2013 E-mail: jangwoo@postech.ac.kr High Performance Computing Lab Department of Computer Science & Engineering Pohang University of Science and Technology
More informationCloud Computing and Hadoop Distributed File System. UCSB CS170, Spring 2018
Cloud Computing and Hadoop Distributed File System UCSB CS70, Spring 08 Cluster Computing Motivations Large-scale data processing on clusters Scan 000 TB on node @ 00 MB/s = days Scan on 000-node cluster
More informationAdvanced Architectures for Oracle Database on Amazon EC2
Advanced Architectures for Oracle Database on Amazon EC2 Abdul Sathar Sait Jinyoung Jung Amazon Web Services November 2014 Last update: April 2016 Contents Abstract 2 Introduction 3 Oracle Database Editions
More informationCSE6331: Cloud Computing
CSE6331: Cloud Computing Leonidas Fegaras University of Texas at Arlington c 2019 by Leonidas Fegaras Cloud Computing Fundamentals Based on: J. Freire s class notes on Big Data http://vgc.poly.edu/~juliana/courses/bigdata2016/
More informationIBM Power Systems: Open innovation to put data to work Dexter Henderson Vice President IBM Power Systems
IBM Power Systems: Open innovation to put data to work Dexter Henderson Vice President IBM Power Systems 2014 IBM Corporation Powerful Forces are Changing the Way Business Gets Done Data growing exponentially
More informationAWS Storage Gateway. Amazon S3. Amazon EFS. Amazon Glacier. Amazon EBS. Amazon EC2 Instance. storage. File Block Object. Hybrid integrated.
AWS Storage Amazon EFS Amazon EBS Amazon EC2 Instance storage Amazon S3 Amazon Glacier AWS Storage Gateway File Block Object Hybrid integrated storage Amazon S3 Amazon Glacier Amazon EBS Amazon EFS Durable
More informationAWS Database Migration Service
AWS Database Migration Service Database Modernisation with Minimal Downtime John Winford Sr. Technical Program Manager May 18, 2017 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
More informationIntroduction to Amazon Cloud & EC2 Overview
Introduction to Amazon Cloud & EC2 Overview 2015 Amazon Web Services, Inc. and its affiliates. All rights served. May not be copied, modified, or distributed in whole or in part without the express consent
More informationTechnical Brief. Adding Zadara Storage to VMware Cloud on AWS
Technical Brief Adding Zadara Storage to VMware Cloud on AWS Revision History Row Version Date Description 1 1.0 26 Nov 2018 Initial release 2 1.1 15 Jan 2019 Updates for VMware style guide compliance,
More informationPass4test Certification IT garanti, The Easy Way!
Pass4test Certification IT garanti, The Easy Way! http://www.pass4test.fr Service de mise à jour gratuit pendant un an Exam : SOA-C01 Title : AWS Certified SysOps Administrator - Associate Vendor : Amazon
More informationDemystifying the Cloud With a Look at Hybrid Hosting and OpenStack
Demystifying the Cloud With a Look at Hybrid Hosting and OpenStack Robert Collazo Systems Engineer Rackspace Hosting The Rackspace Vision Agenda Truly a New Era of Computing 70 s 80 s Mainframe Era 90
More informationAmazon Elastic File System
Amazon Elastic File System Choosing Between the Different Throughput & Performance Modes July 2018 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved. Notices This document is provided
More informationAmazon Web Services (AWS) Solutions Architect Intermediate Level Course Content
Amazon Web Services (AWS) Solutions Architect Intermediate Level Course Content Introduction to Cloud Computing A Short history Client Server Computing Concepts Challenges with Distributed Computing Introduction
More informationEBOOK: VMware Cloud on AWS: Optimized for the Next-Generation Hybrid Cloud
EBOOK: VMware Cloud on AWS: Optimized for the Next-Generation Hybrid Cloud Contents Introduction... 3 What is VMware Cloud on AWS?... 5 Customer Benefits of Adopting VMware Cloud on AWS... 6 VMware Cloud
More informationIntroduction to Cloud Computing
You will learn how to: Build and deploy cloud applications and develop an effective implementation strategy Leverage cloud vendors Amazon EC2 and Amazon S3 Exploit Software as a Service (SaaS) to optimize
More informationIBM Spectrum NAS, IBM Spectrum Scale and IBM Cloud Object Storage
IBM Spectrum NAS, IBM Spectrum Scale and IBM Cloud Object Storage Silverton Consulting, Inc. StorInt Briefing 2017 SILVERTON CONSULTING, INC. ALL RIGHTS RESERVED Page 2 Introduction Unstructured data has
More informationBringing OpenStack to the Enterprise. An enterprise-class solution ensures you get the required performance, reliability, and security
Bringing OpenStack to the Enterprise An enterprise-class solution ensures you get the required performance, reliability, and security INTRODUCTION Organizations today frequently need to quickly get systems
More informationAWS Agility + Splunk Visibility = Cloud Success. Splunk App for AWS Demo. Laura Ripans, AWS Alliance Manager
AWS Agility + Splunk Visibility = Cloud Success Splunk App for AWS Demo Laura Ripans, AWS Alliance Manager Disruptive innovation and business transformation starts with data I HAVE BEEN GIVEN AN AWS ACCOUNT!!!
More informationBig Data Technology Ecosystem. Mark Burnette Pentaho Director Sales Engineering, Hitachi Vantara
Big Data Technology Ecosystem Mark Burnette Pentaho Director Sales Engineering, Hitachi Vantara Agenda End-to-End Data Delivery Platform Ecosystem of Data Technologies Mapping an End-to-End Solution Case
More informationCPET 581 Cloud Computing: Technologies and Enterprise IT Strategies
CPET 581 Cloud Computing: Technologies and Enterprise IT Strategies Lecture 8 Cloud Programming & Software Environments: High Performance Computing & AWS Services Part 2 of 2 Spring 2015 A Specialty Course
More informationManaging IoT and Time Series Data with Amazon ElastiCache for Redis
Managing IoT and Time Series Data with ElastiCache for Redis Darin Briskman, ElastiCache Developer Outreach Michael Labib, Specialist Solutions Architect 2016, Web Services, Inc. or its Affiliates. All
More informationHadoop 2.x Core: YARN, Tez, and Spark. Hortonworks Inc All Rights Reserved
Hadoop 2.x Core: YARN, Tez, and Spark YARN Hadoop Machine Types top-of-rack switches core switch client machines have client-side software used to access a cluster to process data master nodes run Hadoop
More informationCloud Computing 4/17/2016. Outline. Cloud Computing. Centralized versus Distributed Computing Some people argue that Cloud Computing. Cloud Computing.
Cloud Computing By: Muhammad Naseem Assistant Professor Department of Computer Engineering, Sir Syed University of Engineering & Technology, Web: http://sites.google.com/site/muhammadnaseem105 Email: mnaseem105@yahoo.com
More informationApache Hadoop Goes Realtime at Facebook. Himanshu Sharma
Apache Hadoop Goes Realtime at Facebook Guide - Dr. Sunny S. Chung Presented By- Anand K Singh Himanshu Sharma Index Problem with Current Stack Apache Hadoop and Hbase Zookeeper Applications of HBase at
More informationSecurity & Compliance in the AWS Cloud. Vijay Rangarajan Senior Cloud Architect, ASEAN Amazon Web
Security & Compliance in the AWS Cloud Vijay Rangarajan Senior Cloud Architect, ASEAN Amazon Web Services @awscloud www.cloudsec.com #CLOUDSEC Security & Compliance in the AWS Cloud TECHNICAL & BUSINESS
More informationAmazon Web Services (AWS) Training Course Content
Amazon Web Services (AWS) Training Course Content SECTION 1: CLOUD COMPUTING INTRODUCTION History of Cloud Computing Concept of Client Server Computing Distributed Computing and it s Challenges What is
More informationPrepAwayExam. High-efficient Exam Materials are the best high pass-rate Exam Dumps
PrepAwayExam http://www.prepawayexam.com/ High-efficient Exam Materials are the best high pass-rate Exam Dumps Exam : SAA-C01 Title : AWS Certified Solutions Architect - Associate (Released February 2018)
More informationTESTING BIG DATA WORLD RIGA. by Konstantin Pletenev OCTOBER, 2017, TAPOST GROW CONFIDENTLY
RIGA TESTING BIG DATA WORLD by Konstantin Pletenev OCTOBER, 2017, TAPOST GROW CONFIDENTLY BIG DATA IS NOT ABOUT THE DATA THE REVOLUTION IS NOT THAT THERE S MORE DATA AVAILABLE THE REVOLUTION IS THAT WE
More informationEXAM - AWS-Solution-Architect- Associate. AWS Certified Solutions Architect - Associate. Buy Full Product
Amazon EXAM - AWS-Solution-Architect- Associate AWS Certified Solutions Architect - Associate Buy Full Product http://www.examskey.com/aws-solution-architect- Associate.html Examskey Amazon AWS-Solution-Architect-Associate
More informationWelcome to the New Era of Cloud Computing
Welcome to the New Era of Cloud Computing Aaron Kimball The web is replacing the desktop 1 SDKs & toolkits are there What about the backend? Image: Wikipedia user Calyponte 2 Two key concepts Processing
More informationBig Data Hadoop Stack
Big Data Hadoop Stack Lecture #1 Hadoop Beginnings What is Hadoop? Apache Hadoop is an open source software framework for storage and large scale processing of data-sets on clusters of commodity hardware
More information