Resiliency at Scale in the Distributed Storage Cloud
|
|
- Dortha Poole
- 6 years ago
- Views:
Transcription
1 Resiliency at Scale in the Distributed Storage Cloud Alma Riska Advanced Storage Division EMC Corporation In collaboration with many at Cloud Infrastructure Group
2 Outline Wi topic but this talk will focus on Architecture Resiliency Failures Redundancy schemes Policies to differentiate services 2
3 Digital Content Creation & Investment 3
4 Scaled-out Storage Systems Large amount of hardware Thousands of disks Tens to hundreds of servers Significant amount of networking Wi range of applications Internet Service Provirs On-line Service Provirs Private cloud Up-to million of users 4
5 Storage Requirements Store amount of data (Tens) PetaBytes massive Direct attached high capacity nearline HDDs Highly available Minimum down time Reliably stored Beyond the traditional 5 nines Ubiquitous access Cross geographical boundaries 5
6 Scaled-out Storage Architecture Hardware organized in nos / racks / geographical sites LAN / WAN o o s,. Services/ no Services/ no 6
7 Scalability in Scaled-out Storage Inpennce between components no single point of failure Hardware disks, nos, racks, sites Software services such as metadata Seamlessly add/remove storage vices or nos Isolation of failures Sustaining performance Shared-nothing architecture Elasticity / resilience / performance 7
8 EMC Atmos Architecture Shared nothing architecture s large capacity SAS HDDs s up to 8 nos or 480 4TB HDDs (>1PByte) At least two sites o LAN / WAN o 8
9 Storage Resiliency Data Reliability Data is stored persistently in vice(s) like HDDs Data Availability Data is available inpenntly of the failures of hardware Data Consistency and Accuracy Returned data is what the user has stored in the system 9
10 Failures Data vices (HDDs) Other components o o Hardware Network Power outages Cooling outages LAN / WAN Software Drivers o Services (metadata) o 10
11 Transient Failures Many failures are transient Temporary interruption of operation of a component o o Variability in component response time can be seen as a transient failure Particularly network lays LAN / WAN System load causes transient failures o o Transient failures occur much more often than hardware component failures 11
12 Impact of Failures Reliability Disk failures directly o o But all other failures too Availability Directly impacted by any failure, particularly transient Consistency o LAN / WAN o Service failures Metadata Transient failures 12
13 Criticality of Failures in the Cloud Large scale, e.g., no failure Make unavailable large amount of data and other components simultaneously Since there are more components in the system, failures happen more often System needs to be sign with high component unavailability in mind Even if the unavailability is transient 13
14 Challenges of Handling Failures Correct intification of failures Many failures have similar symptoms Disk unreachable (disk failure, controller failure, power failure, network failure) Effective isolation of failures Limit the cases when a single component failure becomes a no or site failure Timely tection of failures In a large system failures may go untected Particularly transient failures and their impact 14
15 Example of System Alerts HDD events are overwhelming Event do not necessarily indicate disk failures Rather temporary unreachable HDDs Various reasons Majority, transient 15
16 Fault Tolerance in Cloud Storage Transparency toward failures Disks / s / s Services Even entire sites Transparency varies by system Goal or targets o o X LA N / W AN X x x o o 16
17 Fault Tolerance in Cloud Storage Transparency toward failures Disks / s / s o o Services Even entire sites Transparency varies by system Goal or targets o LA N / W AN o Resilience goals termine fault domains 17
18 Fault Domains The hierarchy of the set of resources whose failure can be tolerated in a system o o Example: Tolerate a site failure Two racks or 16 nos or 240 disks o LA N / W AN o Determines distribution Data Services 18
19 Fault Tolerance and Redundancy Fault tolerance is primarily achieved via redundancy More hardware and software than need Achieving a fault tolerant goal pends Amount of redundancy (storage capacity) Traditionally parity (RAID) Often in the cloud is replication Erasure coding Pro-active measures Monitoring/analysis/prediction of system s health Background tection of failures 19
20 Fault Tolerance and Data Replication Replicate data (including metadata) up to 4 times Pros High reliability High availability Good performance and accessibility Easy to implement Cons High capacity overhead Up to 300% in a 4-way replication 20
21 Replication in Scale-Out Cloud Storage Average case in a cloud storage system Several tens (up to hundred) of raw PBytes capacity Multiple tens of user PBytes capacity Does not scale well with regard to Cost Resilience With only 3 replicas it is not always possible to tolerate multi-no and site failure 21
22 Erasure Coding Generalization of parity-based fault tolerance RAID schemes Replication is a special case Out of n fragments of information m are actual data k are additional cos (n=m+k) k missing fragments of data can be tolerated Co is referred to as m/n co 22
23 Erasure Coding Capacity overhead k/n Overhead reduces as n increases Same protection Complexity computational and management Increases as n increases As network lays dominate performance erasure coding becomes feasible approach Tra-off between protection, complexity, overhead Common EMC Atmos cos are 9/12, 10/16 23
24 EC vs. Other redundancy schemes 24
25 Erasure Coding at Scale Data fragments distributed based on the system fault domains Placement of these fragments is crucial Round-robin placement ensures uniform distribution of fragments Assumed in previous calculations Placement of data fragments pends on User requirements with regard to Performance Priorities 25
26 EC data placement in the Cloud We velop a mol to see penncies between EC fragment placement and system size/architecture Determine Tolerance toward site failures as a function of Number of sites m/n erasure co parameters Additional no failure tolerance
27 EC data placement in the Cloud Assumptions: Homogeneous geographically distributed sites Equal number of nos and disks Equal network lays between any pair of sites Equal data priority Round robin distribution of the fragments across s / nos / disks Failures on disks / nos / sites (power, network)
28 Failure Tolerance in 2 System In a two site system there is only one site failure tolerance Each site has 6 nos available The numbers insi each (x,y) tuple are the number of nos tolerated in addition to the sites tolerated
29 Failure Tolerance in 4 System In a four site system there are one, two and three site failure tolerance Each site has 6 nos available The numbers insi each (x,y) tuple are the number of nos tolerated in addition to the sites tolerated
30 Heterogeneous Protection Policies As system evolve their resources become heterogeneous Different no or site sizes Different network bandwidth Different data priority location origin In such a case Uniformity of data distribution not a requirement The above factors (including performance) should termine data fragment placement 30
31 Abstraction of Heterogeneous Cloud Storage Group components based on affinity criteria o o Network bandwidth Create homogeneous subcluster Determine redundancy for each sub-cluster Handle each sub-cluster inpenntly o LAN / WAN o Combine outcome for system-wi placement
32 Abstraction of Heterogeneous Cloud Storage - Example Two sites are close (e.g. on the same US coast) Fast network connection o o Data can be placed in any of the nos in both sites and retrieving it will not suffer extra network lay If an 6/12 redundancy scheme is used If data primary location is the upper two-site subcluster then 6 data fragments can be placed in its two sites and the 6 cos in the other remote sites o LAN / WAN o Accessing the data is not affected by network bandwidth One site failure is tolerated
33 Differentiate Protection via Policy Flexible policy settings for grouping resources and isolating applications /tenants Easily managing a large heterogeneous system Hybrid protection schemes that combine multiple replication schemes E.g., a two replications policy where First replica is the original data (stored in the closest site to tenant) Second replica is a 9/12 EC scheme that distributes the data in the rest of the sites for resilience 33
34 Protection Policies in the Field Tenants s 2 replicas >= 3 replicas 1 EC replica >= 2 EC replica Mix regular/ec /2; 9/3 4 1 sync; async 2 2 sync; async async 3 2 sync async 4 2 sync; async 9/3 9/3; 10/6; sync; async 2 2 sync; async sync; async 2 2 9/3; sync; async 2 sync; async sync 3 2 sync 9:3; sync; async 2 4 sync; async async 10/6; 9/3 9/3; sync; async 9:/3; async :2 2 2 sync 9:3; sync; async 9/3 async 2 1 sync 9:3 2 2 sync async 9/3 async 1 6 9:3 async 2 2 async 2 2 sync 9:3 9:3 async 3 2 sync async 3 3 sync async 2 1 sync 34
35 Proactive Failure Detection Monitoring the health of vices and services Logging events Taking corrective measures before failures happen Strengthen the resilience Address without the redundancy affected by failure Example Use of SMART logs to termine health of drives Replace HDDs that are about to fail rather than failed 35
36 Proactive Failure Detection Verify in the background the validity of data, services and health of hardware Critical aspect of resiliency in the cloud System are large and some portions maybe idle for extend periods of time Failures and issues may go untected Ensure timely failure tection Improve resilience for a given amount of redundancy 36
37 Conclusions Resilience at scale = reliability+availability+consistency Wi range of large scale failures Redundancy aids resiliency at scale Erasure coding efficient scaling of resiliency Proactive measures to ensure resiliency at scale 37
HDFS Architecture. Gregory Kesden, CSE-291 (Storage Systems) Fall 2017
HDFS Architecture Gregory Kesden, CSE-291 (Storage Systems) Fall 2017 Based Upon: http://hadoop.apache.org/docs/r3.0.0-alpha1/hadoopproject-dist/hadoop-hdfs/hdfsdesign.html Assumptions At scale, hardware
More informationCloudian Sizing and Architecture Guidelines
Cloudian Sizing and Architecture Guidelines The purpose of this document is to detail the key design parameters that should be considered when designing a Cloudian HyperStore architecture. The primary
More informationMellanox InfiniBand Solutions Accelerate Oracle s Data Center and Cloud Solutions
Mellanox InfiniBand Solutions Accelerate Oracle s Data Center and Cloud Solutions Providing Superior Server and Storage Performance, Efficiency and Return on Investment As Announced and Demonstrated at
More informationCold Storage: The Road to Enterprise Ilya Kuznetsov YADRO
Cold Storage: The Road to Enterprise Ilya Kuznetsov YADRO Agenda Technical challenge Custom product Growth of aspirations Enterprise requirements Making an enterprise cold storage product 2 Technical Challenge
More informationNext Generation Erasure Coding Techniques Wesley Leggette Cleversafe
Next Generation Erasure Coding Techniques Wesley Leggette Cleversafe Topics r What is Erasure Coded Storage? r The evolution of Erasure Coded storage r From first- to third-generation erasure coding r
More informationElastic Cloud Storage (ECS)
Elastic Cloud Storage (ECS) Version 3.1 Administration Guide 302-003-863 02 Copyright 2013-2017 Dell Inc. or its subsidiaries. All rights reserved. Published September 2017 Dell believes the information
More informationBusiness Continuity and Disaster Recovery. Ed Crowley Ch 12
Business Continuity and Disaster Recovery Ed Crowley Ch 12 Topics Disaster Recovery Business Impact Analysis MTBF and MTTR RTO and RPO Redundancy Failover Backup Sites Load Balancing Mirror Sites Disaster
More informationDecentralized Distributed Storage System for Big Data
Decentralized Distributed Storage System for Big Presenter: Wei Xie -Intensive Scalable Computing Laboratory(DISCL) Computer Science Department Texas Tech University Outline Trends in Big and Cloud Storage
More informationCONFIGURATION GUIDE WHITE PAPER JULY ActiveScale. Family Configuration Guide
WHITE PAPER JULY 2018 ActiveScale Family Configuration Guide Introduction The world is awash in a sea of data. Unstructured data from our mobile devices, emails, social media, clickstreams, log files,
More informationRecording at the Edge Solution Brief
CCTV IP Video Solution Brief Recording at the Edge Recording at the Edge Solution Brief Reliability Recording is independent of the network s health and degree of congestion. Even if the network grinds
More informationTIBCO StreamBase 10 Distributed Computing and High Availability. November 2017
TIBCO StreamBase 10 Distributed Computing and High Availability November 2017 Distributed Computing Distributed Computing location transparent objects and method invocation allowing transparent horizontal
More informationCommvault Backup to Cloudian Hyperstore CONFIGURATION GUIDE TO USE HYPERSTORE AS A STORAGE LIBRARY
Commvault Backup to Cloudian Hyperstore CONFIGURATION GUIDE TO USE HYPERSTORE AS A STORAGE LIBRARY CONTENTS EXECUTIVE SUMMARY... 2 SOLUTION OVERVIEW... 3 USE CASES... 4 SOLUTION COMPONENTS... 5 Commvault
More informationWhy Datrium DVX is Best for VDI
Why Datrium DVX is Best for VDI 385 Moffett Park Dr. Sunnyvale, CA 94089 844-478-8349 www.datrium.com Technical Report Introduction Managing a robust and growing virtual desktop infrastructure in current
More informationEMC Forum EMC ViPR and ECS: A Lap Around Software-Defined Services
EMC Forum 2014 Copyright 2014 EMC Corporation. All rights reserved. 1 EMC ViPR and ECS: A Lap Around Software-Defined Services 2 Session Agenda Market Dynamics EMC ViPR Overview What s New in ViPR Controller
More informationDatacenter replication solution with quasardb
Datacenter replication solution with quasardb Technical positioning paper April 2017 Release v1.3 www.quasardb.net Contact: sales@quasardb.net Quasardb A datacenter survival guide quasardb INTRODUCTION
More informationSystems Infrastructure for Data Science. Web Science Group Uni Freiburg WS 2014/15
Systems Infrastructure for Data Science Web Science Group Uni Freiburg WS 2014/15 Lecture X: Parallel Databases Topics Motivation and Goals Architectures Data placement Query processing Load balancing
More informationStep into the future. HP Storage Summit Converged storage for the next era of IT
HP Storage Summit 2013 Step into the future Converged storage for the next era of IT 1 HP Storage Summit 2013 Step into the future Converged storage for the next era of IT Karen van Warmerdam HP XP Product
More informationDisclaimer This presentation may contain product features that are currently under development. This overview of new technology represents no commitme
STO1926BU A Day in the Life of a VSAN I/O Diving in to the I/O Flow of vsan John Nicholson (@lost_signal) Pete Koehler (@vmpete) VMworld 2017 Content: Not for publication #VMworld #STO1926BU Disclaimer
More informationModern hyperconverged infrastructure. Karel Rudišar Systems Engineer, Vmware Inc.
Modern hyperconverged infrastructure Karel Rudišar Systems Engineer, Vmware Inc. 2 What Is Hyper-Converged Infrastructure? - The Ideal Architecture for SDDC Management SDDC Compute Networking Storage Simplicity
More informationDatabase Architectures
Database Architectures CPS352: Database Systems Simon Miner Gordon College Last Revised: 4/15/15 Agenda Check-in Parallelism and Distributed Databases Technology Research Project Introduction to NoSQL
More informationThe Google File System
The Google File System Sanjay Ghemawat, Howard Gobioff and Shun Tak Leung Google* Shivesh Kumar Sharma fl4164@wayne.edu Fall 2015 004395771 Overview Google file system is a scalable distributed file system
More informationHigh Availability for Citrix XenDesktop
WHITE PAPER Citrix XenDesktop High Availability for Citrix XenDesktop Enhancing XenDesktop Availability with NetScaler Reference Architecture www.citrix.com Contents Contents... 2 Introduction... 3 Desktop
More information5 reasons why choosing Apache Cassandra is planning for a multi-cloud future
White Paper 5 reasons why choosing Apache Cassandra is planning for a multi-cloud future Abstract We have been hearing for several years now that multi-cloud deployment is something that is highly desirable,
More informationChapter 18: Parallel Databases
Chapter 18: Parallel Databases Introduction Parallel machines are becoming quite common and affordable Prices of microprocessors, memory and disks have dropped sharply Recent desktop computers feature
More informationGoogle File System. Arun Sundaram Operating Systems
Arun Sundaram Operating Systems 1 Assumptions GFS built with commodity hardware GFS stores a modest number of large files A few million files, each typically 100MB or larger (Multi-GB files are common)
More informationThe Google File System
The Google File System Sanjay Ghemawat, Howard Gobioff, and Shun-Tak Leung Google SOSP 03, October 19 22, 2003, New York, USA Hyeon-Gyu Lee, and Yeong-Jae Woo Memory & Storage Architecture Lab. School
More informationThe Google File System
The Google File System Sanjay Ghemawat, Howard Gobioff, and Shun-Tak Leung SOSP 2003 presented by Kun Suo Outline GFS Background, Concepts and Key words Example of GFS Operations Some optimizations in
More informationActiveScale Erasure Coding and Self Protecting Technologies
WHITE PAPER AUGUST 2018 ActiveScale Erasure Coding and Self Protecting Technologies BitSpread Erasure Coding and BitDynamics Data Integrity and Repair Technologies within The ActiveScale Object Storage
More informationBUSINESS CONTINUITY: THE PROFIT SCENARIO
WHITE PAPER BUSINESS CONTINUITY: THE PROFIT SCENARIO THE BENEFITS OF A COMPREHENSIVE BUSINESS CONTINUITY STRATEGY FOR INCREASED OPPORTUNITY Organizational data is the DNA of a business it makes your operation
More informationScality RING on Cisco UCS: Store File, Object, and OpenStack Data at Scale
Scality RING on Cisco UCS: Store File, Object, and OpenStack Data at Scale What You Will Learn Cisco and Scality provide a joint solution for storing and protecting file, object, and OpenStack data at
More informationHadoop File System S L I D E S M O D I F I E D F R O M P R E S E N T A T I O N B Y B. R A M A M U R T H Y 11/15/2017
Hadoop File System 1 S L I D E S M O D I F I E D F R O M P R E S E N T A T I O N B Y B. R A M A M U R T H Y Moving Computation is Cheaper than Moving Data Motivation: Big Data! What is BigData? - Google
More informationSAP HANA. HA and DR Guide. Issue 03 Date HUAWEI TECHNOLOGIES CO., LTD.
Issue 03 Date 2018-05-23 HUAWEI TECHNOLOGIES CO., LTD. Copyright Huawei Technologies Co., Ltd. 2019. All rights reserved. No part of this document may be reproduced or transmitted in any form or by any
More informationECONOMICAL, STORAGE PURPOSE-BUILT FOR THE EMERGING DATA CENTERS. By George Crump
ECONOMICAL, STORAGE PURPOSE-BUILT FOR THE EMERGING DATA CENTERS By George Crump Economical, Storage Purpose-Built for the Emerging Data Centers Most small, growing businesses start as a collection of laptops
More informationCPSC 426/526. Cloud Computing. Ennan Zhai. Computer Science Department Yale University
CPSC 426/526 Cloud Computing Ennan Zhai Computer Science Department Yale University Recall: Lec-7 In the lec-7, I talked about: - P2P vs Enterprise control - Firewall - NATs - Software defined network
More informationLecture 9: MIMD Architectures
Lecture 9: MIMD Architectures Introduction and classification Symmetric multiprocessors NUMA architecture Clusters Zebo Peng, IDA, LiTH 1 Introduction A set of general purpose processors is connected together.
More informationChapter 20: Database System Architectures
Chapter 20: Database System Architectures Chapter 20: Database System Architectures Centralized and Client-Server Systems Server System Architectures Parallel Systems Distributed Systems Network Types
More informationSoftware-defined Storage: Fast, Safe and Efficient
Software-defined Storage: Fast, Safe and Efficient TRY NOW Thanks to Blockchain and Intel Intelligent Storage Acceleration Library Every piece of data is required to be stored somewhere. We all know about
More informationChapter 3. Design of Grid Scheduler. 3.1 Introduction
Chapter 3 Design of Grid Scheduler The scheduler component of the grid is responsible to prepare the job ques for grid resources. The research in design of grid schedulers has given various topologies
More informationIntegrated hardware-software solution developed on ARM architecture. CS3 Conference Krakow, January 30th 2018
Integrated hardware-software solution developed on ARM architecture CS3 Conference Krakow, January 30th 2018 Why Object Storage Data doubles every 2 year...growing at a faster pace and is mainly unstructured
More informationNovember 7, DAN WILSON Global Operations Architecture, Concur. OpenStack Summit Hong Kong JOE ARNOLD
November 7, 2013 DAN WILSON Global Operations Architecture, Concur dan.wilson@concur.com @tweetdanwilson OpenStack Summit Hong Kong JOE ARNOLD CEO, SwiftStack joe@swiftstack.com @joearnold Introduction
More informationAvailability for the modern datacentre Veeam Availability Suite v9.5
Availability for the modern datacentre Veeam Availability Suite v9.5 Jan van Leuken System Engineer Benelux, Veeam Software jan.vanleuken@veeam.com +31 (0)615 83 50 64 Robin van der Steenhoven Territory
More informationTake Back Lost Revenue by Activating Virtuozzo Storage Today
Take Back Lost Revenue by Activating Virtuozzo Storage Today JUNE, 2017 2017 Virtuozzo. All rights reserved. 1 Introduction New software-defined storage (SDS) solutions are enabling hosting companies to
More informationDifferentiating Your Datacentre in the Networked Future John Duffin
Differentiating Your Datacentre in the Networked Future John Duffin Managing Director, South Asia. Uptime Institute July 2017 2017 Uptime Institute, LLC The Global Datacentre Authority 2 2017 Uptime Institute,
More informationBusiness Benefits of Policy Based Data De-Duplication Data Footprint Reduction with Quality of Service (QoS) for Data Protection
Data Footprint Reduction with Quality of Service (QoS) for Data Protection By Greg Schulz Founder and Senior Analyst, the StorageIO Group Author The Green and Virtual Data Center (Auerbach) October 28th,
More informationActiveScale Erasure Coding and Self Protecting Technologies
NOVEMBER 2017 ActiveScale Erasure Coding and Self Protecting Technologies BitSpread Erasure Coding and BitDynamics Data Integrity and Repair Technologies within The ActiveScale Object Storage System Software
More informationEMC Forum 2014 EMC ViPR and ECS: A Lap Around Software-Defined Services. Magnus Nilsson Blog: purevirtual.
EMC Forum 2014 EMC ViPR and ECS: A Lap Around Software-Defined Services Magnus Nilsson magnus.nilsson@emc.com Twitter: @swevm Blog: purevirtual.eu 1 Session Agenda Market Dynamics EMC ViPR Overview What
More informationThe Google File System
October 13, 2010 Based on: S. Ghemawat, H. Gobioff, and S.-T. Leung: The Google file system, in Proceedings ACM SOSP 2003, Lake George, NY, USA, October 2003. 1 Assumptions Interface Architecture Single
More informationLecture 23 Database System Architectures
CMSC 461, Database Management Systems Spring 2018 Lecture 23 Database System Architectures These slides are based on Database System Concepts 6 th edition book (whereas some quotes and figures are used
More informationComputer Architecture 计算机体系结构. Lecture 6. Data Storage and I/O 第六讲 数据存储和输入输出. Chao Li, PhD. 李超博士
Computer Architecture 计算机体系结构 Lecture 6. Data Storage and I/O 第六讲 数据存储和输入输出 Chao Li, PhD. 李超博士 SJTU-SE346, Spring 2018 Review Memory hierarchy Cache and virtual memory Locality principle Miss cache, victim
More informationSecurely Access Services Over AWS PrivateLink. January 2019
Securely Access Services Over AWS PrivateLink January 2019 Notices This document is provided for informational purposes only. It represents AWS s current product offerings and practices as of the date
More informationCloud Computing and Hadoop Distributed File System. UCSB CS170, Spring 2018
Cloud Computing and Hadoop Distributed File System UCSB CS70, Spring 08 Cluster Computing Motivations Large-scale data processing on clusters Scan 000 TB on node @ 00 MB/s = days Scan on 000-node cluster
More informationDELL EMC DATA DOMAIN BOOST AND DYNAMIC INTERFACE GROUPS
WHITE PAPER DELL EMC DATA DOMAIN BOOST AND DYNAMIC INTERFACE GROUPS Maximize the efficiency of multiple network interfaces Abstract Dell EMC delivers dynamic interface groups to simplify the use of multiple
More informationVMware vsphere Clusters in Security Zones
SOLUTION OVERVIEW VMware vsan VMware vsphere Clusters in Security Zones A security zone, also referred to as a DMZ," is a sub-network that is designed to provide tightly controlled connectivity to an organization
More informationECS High Availability Design
ECS High Availability Design March 2018 A Dell EMC white paper Revisions Date Mar 2018 Aug 2017 July 2017 Description Version 1.2 - Updated to include ECS version 3.2 content Version 1.1 - Updated to include
More informationWhat's New in vsan 6.2 First Published On: Last Updated On:
First Published On: 07-07-2016 Last Updated On: 08-23-2017 1 1. Introduction 1.1.Preface 1.2.Architecture Overview 2. Space Efficiency 2.1.Deduplication and Compression 2.2.RAID - 5/6 (Erasure Coding)
More informationThe Google File System
The Google File System Sanjay Ghemawat, Howard Gobioff, and Shun-Tak Leung December 2003 ACM symposium on Operating systems principles Publisher: ACM Nov. 26, 2008 OUTLINE INTRODUCTION DESIGN OVERVIEW
More informationEMC DATA DOMAIN OPERATING SYSTEM
EMC DATA DOMAIN OPERATING SYSTEM Powering EMC Protection Storage ESSENTIALS High-Speed, Scalable Deduplication Up to 31 TB/hr performance Reduces requirements for backup storage by 10 to 30x and archive
More informationSolidFire and Pure Storage Architectural Comparison
The All-Flash Array Built for the Next Generation Data Center SolidFire and Pure Storage Architectural Comparison June 2014 This document includes general information about Pure Storage architecture as
More informationvsan Remote Office Deployment January 09, 2018
January 09, 2018 1 1. vsan Remote Office Deployment 1.1.Solution Overview Table of Contents 2 1. vsan Remote Office Deployment 3 1.1 Solution Overview Native vsphere Storage for Remote and Branch Offices
More informationChapter 18: Database System Architectures.! Centralized Systems! Client--Server Systems! Parallel Systems! Distributed Systems!
Chapter 18: Database System Architectures! Centralized Systems! Client--Server Systems! Parallel Systems! Distributed Systems! Network Types 18.1 Centralized Systems! Run on a single computer system and
More informationRED HAT CEPH STORAGE ROADMAP. Cesar Pinto Account Manager, Red Hat Norway
RED HAT CEPH STORAGE ROADMAP Cesar Pinto Account Manager, Red Hat Norway cpinto@redhat.com THE RED HAT STORAGE MISSION To offer a unified, open software-defined storage portfolio that delivers a range
More informationOutline. Definition of a Distributed System Goals of a Distributed System Types of Distributed Systems
Distributed Systems Outline Definition of a Distributed System Goals of a Distributed System Types of Distributed Systems What Is A Distributed System? A collection of independent computers that appears
More informationThe Google File System
The Google File System Sanjay Ghemawat, Howard Gobioff, and Shun-Tak Leung Google* 정학수, 최주영 1 Outline Introduction Design Overview System Interactions Master Operation Fault Tolerance and Diagnosis Conclusions
More informationLecture 9: MIMD Architectures
Lecture 9: MIMD Architectures Introduction and classification Symmetric multiprocessors NUMA architecture Clusters Zebo Peng, IDA, LiTH 1 Introduction MIMD: a set of general purpose processors is connected
More informationVendor: EMC. Exam Code: E Exam Name: Cloud Infrastructure and Services Exam. Version: Demo
Vendor: EMC Exam Code: E20-002 Exam Name: Cloud Infrastructure and Services Exam Version: Demo QUESTION NO: 1 In which Cloud deployment model would an organization see operational expenditures grow in
More informationCLOUD-SCALE FILE SYSTEMS
Data Management in the Cloud CLOUD-SCALE FILE SYSTEMS 92 Google File System (GFS) Designing a file system for the Cloud design assumptions design choices Architecture GFS Master GFS Chunkservers GFS Clients
More informationSoftware Defined Storage
Software Defined Storage Riyadh Forum 2016 By: Ayman El Marazky Sr. System Engineer software defined solutions The CIO Dilemma IT Budget $ By 2020, the digital universe of data will go to 44 ZB Modern
More informationvsan Security Zone Deployment First Published On: Last Updated On:
First Published On: 06-14-2017 Last Updated On: 11-20-2017 1 1. vsan Security Zone Deployment 1.1.Solution Overview Table of Contents 2 1. vsan Security Zone Deployment 3 1.1 Solution Overview VMware vsphere
More informationDistributed System. Gang Wu. Spring,2018
Distributed System Gang Wu Spring,2018 Lecture7:DFS What is DFS? A method of storing and accessing files base in a client/server architecture. A distributed file system is a client/server-based application
More informationThe Google File System (GFS)
1 The Google File System (GFS) CS60002: Distributed Systems Antonio Bruto da Costa Ph.D. Student, Formal Methods Lab, Dept. of Computer Sc. & Engg., Indian Institute of Technology Kharagpur 2 Design constraints
More informationNative vsphere Storage for Remote and Branch Offices
SOLUTION OVERVIEW VMware vsan Remote Office Deployment Native vsphere Storage for Remote and Branch Offices VMware vsan is the industry-leading software powering Hyper-Converged Infrastructure (HCI) solutions.
More informationDon t Run out of Power: Use Smart Grid and Cloud Technology
Don t Run out of Power: Use Smart Grid and Cloud Technology Bruce Naegel Sr. Product Manager Symantec Corp. Presentation Overview Overview of IT Power Challenges SMART Grid as Part of the Solution Cloud
More informationBalancing storage utilization across a global namespace Manish Motwani Cleversafe, Inc.
Balancing storage utilization across a global namespace Manish Motwani Cleversafe, Inc. Agenda Introduction What are namespaces, why we need them Compare different types of namespaces Why we need to rebalance
More informationVortex Whitepaper. Simplifying Real-time Information Integration in Industrial Internet of Things (IIoT) Control Systems
Vortex Whitepaper Simplifying Real-time Information Integration in Industrial Internet of Things (IIoT) Control Systems www.adlinktech.com 2017 Table of Contents 1. Introduction........ P 3 2. Iot and
More informationChapter 4. Fundamental Concepts and Models
Chapter 4. Fundamental Concepts and Models 4.1 Roles and Boundaries 4.2 Cloud Characteristics 4.3 Cloud Delivery Models 4.4 Cloud Deployment Models The upcoming sections cover introductory topic areas
More informationGeorgia Institute of Technology ECE6102 4/20/2009 David Colvin, Jimmy Vuong
Georgia Institute of Technology ECE6102 4/20/2009 David Colvin, Jimmy Vuong Relatively recent; still applicable today GFS: Google s storage platform for the generation and processing of data used by services
More informationDell EMC CIFS-ECS Tool
Dell EMC CIFS-ECS Tool Architecture Overview, Performance and Best Practices March 2018 A Dell EMC Technical Whitepaper Revisions Date May 2016 September 2016 Description Initial release Renaming of tool
More informationVirtual Security Server
Data Sheet VSS Virtual Security Server Security clients anytime, anywhere, any device CENTRALIZED CLIENT MANAGEMENT UP TO 50% LESS BANDWIDTH UP TO 80 VIDEO STREAMS MOBILE ACCESS INTEGRATED SECURITY SYSTEMS
More informationOracle E-Business Availability Options. Solution Series for Oracle: 2 of 5
Oracle E-Business Availability Options Solution Series for Oracle: 2 of 5 Table of Contents Coping with E-Business Hours Oracle E-Business Availability Options.....1 Understanding Challenges to Availability...........................2
More informationSwiftStack and python-swiftclient
SwiftStack and python-swiftclient Summary Storage administrators around the world are turning to object storage and particularly SwiftStack s enterprise-ready object storage built on OpenStack Swift for
More informationThe Google File System
The Google File System By Ghemawat, Gobioff and Leung Outline Overview Assumption Design of GFS System Interactions Master Operations Fault Tolerance Measurements Overview GFS: Scalable distributed file
More informationDISTRIBUTED SYSTEMS [COMP9243] Lecture 8a: Cloud Computing WHAT IS CLOUD COMPUTING? 2. Slide 3. Slide 1. Why is it called Cloud?
DISTRIBUTED SYSTEMS [COMP9243] Lecture 8a: Cloud Computing Slide 1 Slide 3 ➀ What is Cloud Computing? ➁ X as a Service ➂ Key Challenges ➃ Developing for the Cloud Why is it called Cloud? services provided
More informationStaggeringly Large Filesystems
Staggeringly Large Filesystems Evan Danaher CS 6410 - October 27, 2009 Outline 1 Large Filesystems 2 GFS 3 Pond Outline 1 Large Filesystems 2 GFS 3 Pond Internet Scale Web 2.0 GFS Thousands of machines
More informationSolace JMS Broker Delivers Highest Throughput for Persistent and Non-Persistent Delivery
Solace JMS Broker Delivers Highest Throughput for Persistent and Non-Persistent Delivery Java Message Service (JMS) is a standardized messaging interface that has become a pervasive part of the IT landscape
More informationDell Technologies IoT Solution Surveillance with Genetec Security Center
Dell Technologies IoT Solution Surveillance with Genetec Security Center Surveillance December 2018 H17436 Sizing Guide Abstract The purpose of this guide is to help you understand the benefits of using
More informationBIG DATA AND HADOOP ON THE ZFS STORAGE APPLIANCE
BIG DATA AND HADOOP ON THE ZFS STORAGE APPLIANCE BRETT WENINGER, MANAGING DIRECTOR 10/21/2014 ADURANT APPROACH TO BIG DATA Align to Un/Semi-structured Data Instead of Big Scale out will become Big Greatest
More informationDatabase Architectures
Database Architectures CPS352: Database Systems Simon Miner Gordon College Last Revised: 11/15/12 Agenda Check-in Centralized and Client-Server Models Parallelism Distributed Databases Homework 6 Check-in
More informationAdvanced Databases: Parallel Databases A.Poulovassilis
1 Advanced Databases: Parallel Databases A.Poulovassilis 1 Parallel Database Architectures Parallel database systems use parallel processing techniques to achieve faster DBMS performance and handle larger
More informationA Thorough Introduction to 64-Bit Aggregates
Technical Report A Thorough Introduction to 64-Bit Aggregates Shree Reddy, NetApp September 2011 TR-3786 CREATING AND MANAGING LARGER-SIZED AGGREGATES The NetApp Data ONTAP 8.0 operating system operating
More informationHow to Protect SAP HANA Applications with the Data Protection Suite
White Paper Business Continuity How to Protect SAP HANA Applications with the Data Protection Suite As IT managers realize the benefits of in-memory database technology, they are accelerating their plans
More informationAddressing Data Management and IT Infrastructure Challenges in a SharePoint Environment. By Michael Noel
Addressing Data Management and IT Infrastructure Challenges in a SharePoint Environment By Michael Noel Contents Data Management with SharePoint and Its Challenges...2 Addressing Infrastructure Sprawl
More informationDistributing Software in a Massively Parallel Environment
Distributing Software in a Massively Parallel Environment LISA 2014 Dinah McNutt Release Engineer, Google, Inc. November 12, 2014 Problem: Reliably and consistently distributing software in a Laaaaaaaaaaaarge
More informationCloud Computing. What is cloud computing. CS 537 Fall 2017
Cloud Computing CS 537 Fall 2017 What is cloud computing Illusion of infinite computing resources available on demand Scale-up for most apps Elimination of up-front commitment Small initial investment,
More information! Design constraints. " Component failures are the norm. " Files are huge by traditional standards. ! POSIX-like
Cloud background Google File System! Warehouse scale systems " 10K-100K nodes " 50MW (1 MW = 1,000 houses) " Power efficient! Located near cheap power! Passive cooling! Power Usage Effectiveness = Total
More informationNew HPE 3PAR StoreServ 8000 and series Optimized for Flash
New HPE 3PAR StoreServ 8000 and 20000 series Optimized for Flash AGENDA HPE 3PAR StoreServ architecture fundamentals HPE 3PAR Flash optimizations HPE 3PAR portfolio overview HPE 3PAR Flash example from
More informationSurveillance Dell EMC Storage with Digifort Enterprise
Surveillance Dell EMC Storage with Digifort Enterprise Configuration Guide H15230 REV 1.1 Copyright 2016-2017 Dell Inc. or its subsidiaries. All rights reserved. Published August 2016 Dell believes the
More informationArchitecting Storage for Semiconductor Design: Manufacturing Preparation
White Paper Architecting Storage for Semiconductor Design: Manufacturing Preparation March 2012 WP-7157 EXECUTIVE SUMMARY The manufacturing preparation phase of semiconductor design especially mask data
More informationIntroduction to Distributed Systems. INF5040/9040 Autumn 2018 Lecturer: Eli Gjørven (ifi/uio)
Introduction to Distributed Systems INF5040/9040 Autumn 2018 Lecturer: Eli Gjørven (ifi/uio) August 28, 2018 Outline Definition of a distributed system Goals of a distributed system Implications of distributed
More informationDeduplication has been around for several
Demystifying Deduplication By Joe Colucci Kay Benaroch Deduplication holds the promise of efficient storage and bandwidth utilization, accelerated backup and recovery, reduced costs, and more. Understanding
More informationDELL EMC DATA DOMAIN OPERATING SYSTEM
DATA SHEET DD OS Essentials High-speed, scalable deduplication Up to 68 TB/hr performance Reduces protection storage requirements by 10 to 30x CPU-centric scalability Data invulnerability architecture
More information