Welcome to Copyright 2016 EMC Corporation. All Rights Reserved. Published in the USA. EMC believes the information in this publication is accurate as of its publication date. The information is subject to change without notice. THE INFORMATION IN THIS PUBLICATION IS PROVIDED AS IS. EMC CORPORATION MAKES NO REPRESENTATIONS OR WARRANTIES OF ANY KIND WITH RESPECT TO THE INFORMATION IN THIS PUBLICATION, AND SPECIFICALLY DISCLAIMS IMPLIED WARRANTIES OF MERCHANTABILITY OR FITNESS FOR A PARTICULAR PURPOSE. Use, copying, and distribution of any EMC software described in this publication requires an applicable software license. The trademarks, logos, and service marks (collectively "Trademarks") appearing in this publication are the property of EMC Corporation and other parties. Nothing contained in this publication should be construed as granting any license or right to use any Trademark without the prior written permission of the party that owns the Trademark. EMC, EMC², the EMC logo, AccessAnywhere Access Logix, AdvantEdge, AlphaStor, AppSync ApplicationXtender, ArchiveXtender, Atmos, Authentica, Authentic Problems, Automated Resource Manager, AutoStart, AutoSwap, AVALONidm, Avamar, Aveksa, Bus-Tech, Captiva, Catalog Solution, C-Clip, Celerra, Celerra Replicator, Centera, CenterStage, CentraStar, EMC CertTracker. CIO Connect, ClaimPack, ClaimsEditor, Claralert,cLARiiON, ClientPak, CloudArray, Codebook Correlation Technology, Common Information Model, Compuset, Compute Anywhere, Configuration Intelligence, Configuresoft, Connectrix, Constellation Computing, CoprHD, EMC ControlCenter, CopyCross, CopyPoint, CX, DataBridge, Data Protection Suite. Data Protection Advisor, DBClassify, DD Boost, Dantz, DatabaseXtender, Data Domain, Direct Matrix Architecture, DiskXtender, DiskXtender 2000, DLS ECO, Document Sciences, Documentum, DR Anywhere, DSSD, ECS, elnput, E-Lab, Elastic Cloud Storage, EmailXaminer, EmailXtender, EMC Centera, EMC ControlCenter, EMC LifeLine, EMCTV, Enginuity, EPFM. eroom, Event Explorer, FAST, FarPoint, FirstPass, FLARE, FormWare, Geosynchrony, Global File Virtualization, Graphic Visualization, Greenplum, HighRoad, HomeBase, Illuminator, InfoArchive, InfoMover, Infoscape, Infra, InputAccel, InputAccel Express, Invista, Ionix, Isilon, ISIS,Kazeon, EMC LifeLine, Mainframe Appliance for Storage, Mainframe Data Library, Max Retriever, MCx, MediaStor, Metro, MetroPoint, MirrorView, Mozy, MultiBandDeduplication,Navisphere, Netstorage, NetWitness, NetWorker, EMC OnCourse, OnRack, OpenScale, Petrocloud, PixTools, Powerlink, PowerPath, PowerSnap, ProSphere, ProtectEverywhere, ProtectPoint, EMC Proven, EMC Proven Professional, QuickScan, RAPIDPath, EMC RecoverPoint, Rainfinity, RepliCare, RepliStor, ResourcePak, Retrospect, RSA, the RSA logo, SafeLine, SAN Advisor, SAN Copy, SAN Manager, ScaleIO Smarts, Silver Trail, EMC Snap, SnapImage, SnapSure, SnapView, SourceOne, SRDF, EMC Storage Administrator, StorageScope, SupportMate, SymmAPI, SymmEnabler, Symmetrix, Symmetrix DMX, Symmetrix VMAX, TimeFinder, TwinStrata, UltraFlex, UltraPoint, UltraScale, Unisphere, Universal Data Consistency, Vblock, VCE. Velocity, Viewlets, ViPR, Virtual Matrix, Virtual Matrix Architecture, Virtual Provisioning, Virtualize Everything, Compromise Nothing, Virtuent, VMAX, VMAXe, VNX, VNXe, Voyence, VPLEX, VSAM-Assist, VSAM I/O PLUS, VSET, VSPEX, Watch4net, WebXtender, xpression, xpresso, Xtrem, XtremCache, XtremSF, XtremSW, XtremIO, YottaYotta, Zero-Friction Enterprise Storage. Revision Date: 04/2016 Revision Number: MR-1WN-SIOINTRO.2.0.1 1
This course provides a sales and marketing overview of the ScaleIO and VxRack products. A high-level architecture review, positioning, features, and competitive products are covered. Product positioning is presented by use cases and interactions. 2
This module presents and defines the business challenges that a storage administrator may encounter. We also list and discuss the benefits of ScaleIO and the data center. Finally, we look at competition and alternative products in the SDS space. 3
As businesses grow, IT systems need to grow with them. Storage scalability and server performance are integral pieces to any business. Regardless of size, one of the biggest challenge in company growth is scaling up to the increasing number of computers connected to the system, and how to store the data in a way that is easily accessible without bogging the system down. With growing demand from business users for greater performance and agility from the supporting IT infrastructure, companies must transform IT. Businesses create new technology and tools which address the growth of data and invest huge money to buy hardware storage appliance leading to high CAPEX. With traditional storage, managing the storage infrastructure is complex and requires large number of IT Admins, wide variety of management protocols leading to higher OPEX. Customers are looking for ways to move to a hyper converged architecture to optimize TCO, reduce complexity and transform their data center to support traditional and next generation applications. As the 3rd platform evolves, a wave of innovation accelerates such as Internet of Things, robotics, and cognitive computing are radically expanding the portfolio of next generation applications. The IT infrastructure that supports such 3rd platform technologies must be agile, resilient, scalable on demand, and manageable in a CAPEX-friendly fashion. The current IT infrastructure falls short in its ability to support the design goals of the 3rd platform applications. This necessitates a switch to newer service-oriented computing, networking, and storage paradigms. 4
ScaleIO addresses these current business challenges. ScaleIO is a software-only solution that uses the local disks and LAN of existing servers to create a virtual SAN that has all the benefits of external storage but at a fraction of cost and complexity. ScaleIO utilizes the existing local storage devices and turns them into shared block storage. For many workloads, ScaleIO storage is comparable to, or better than external shared block storage. The lightweight ScaleIO software components are installed on the application servers and communicate via a standard LAN to handle the application I/O requests sent to ScaleIO block volumes. An extremely efficient decentralized block I/O flow, combined with a distributed, sliced volume layout, results in a massively parallel I/O system that can scale up to thousands of nodes. ScaleIO is designed and implemented with enterprise-grade resilience. It is hardware agnostic, the software works efficiently with various types of disks, including: magnetic (HDD) and solid-state disks (SSD), flash PCI Express (PCIe) cards, networks, and hosts. 5
SDS solution can be consumed as a pure downloadable software or as an appliance itself so that the customer can implement different persistency models and various use cases wherever they are at in their business cycles. If the IT organization is looking to acquire hardware and software independently, then the SDS can be used as a downloadable software. If organizations don t have the capability to marry hardware with software, then we can also have the appliance version of our SDS. It can be downloaded directly from EMC or as an OpenStack version of SDS. Customers will still have three choices in how they wish to consume ScaleIO: ScaleIO Software: Leverages server resources and use ScaleIO software to transform them into an intelligent server SAN. VxRack Node: Provides hardware as well as software to simplify the purchasing and support process. VxRack System 1000 FLEX: A fully engineered, hyper-converged solution that runs ScaleIO software. 6
There are 5 key benefits customers can see when implementing ScaleIO. Massive Scalability: ScaleIO scales from 3 to over 1000 servers while providing one large pool of storage. Customers can adopt a pay-as-you-grow buying model so that storage is aligned with application needs. Extreme Performance: The parallel I/O architecture of ScaleIO has multiple data paths so workloads are evenly shared among all servers. There are no bottlenecks and performance scales linearly with negligible impact on applications. Unparalleled Flexibility: ScaleIO provides the unique ability to choose the server operating environment, media type, and deployment mode. The ability to dynamically deploy different resources provides vastly improved measures of its agility for demanding environment. ScaleIO will run with just about any commodity hardware-any server or operating systems, bare metal or virtualizes, with any storage media (HDDs, SSDs, or PCIe cards). Supreme Elasticity: With ScaleIO, capacity is added (or removed) on demand as per business needs, on the fly in small or large increments with no interruption to the ongoing operation. It auto-rebalances when resources are added and auto-rebuilds when resources fail or are removed. This means that capacity planning and migration are practically eliminated. Compelling Economics: With ScaleIO, the use of commodity hardware drives down your CAPEX, while its innate elasticity and elimination of data migration lower OPEX by as much as 60% TCO savings compared to traditional IT. 7
ScaleIO competes with traditional storage vendors as an alternative to SAN block storage platforms; as well as against vendors with pure SDS or hyper-converged server SAN block appliance vendors such as Nutanix and Simplicity. In addition, ScaleIO indirectly competes with AWS and Windows Azure in that ScaleIO can be used by Cloud Service Providers and large enterprises as a hyper scale-out storage system to create Public Clouds (IaaS) and even private and hybrid clouds. Since ScaleIO is software defined, it competes with CEPH, Atlantis, HP StoreVirtual VSA and even potentially Scality and Cleversafe, who both (like CEPH) provide support for block data storage as well as object data. Traditional SAN solutions are typically offered by large, incumbent vendors such as IBM, HP, NetApp, HDS and, of course, EMC itself. ScaleIO offers improved flexibility and deployment. The hyper-converged technology offers datacenter in a box. Most solutions (excluding ScaleIO!) are tightly coupled with virtualization software (hypervisors) such as VMWare, and can only operate in such environments (no bare-metal). Most companies in this space are newcomers. Notable examples are Nutanix and Simplivity. Software Defined:The most notable example in this space is CEPH, which is offered by Red Hat based on the Inktank technology. At present, CEPH is complex and difficult to deploy and maintain. It carries a lot of hidden costs, which makes it a more expensive solution. Public Cloud Vendors: Block storage is offered as a service by public cloud IaaS providers. A notable example is Amazon s AWS Elastic Block Store (EBS). In general, ScaleIO offers much better performance, control, compliance and security parameters than IaaS solutions at a competitive cost. 8
The traditional dedicated SAN and dedicated workloads cannot always provide the scale and flexibility needed. A storage array can t borrow capacity from another SAN if demand increases and can lead to data bottlenecks. When delivering Infrastructure as a Service (IaaS) or high performance applications, delays in response are simply not acceptable to customers or users. By using the ScaleIO method of converging host-based storage we can reduce the cost associated with the traditional storage arrays. And as new servers are added, the number of IOPS and storage will continue to grow. The scalability of performance is linear with regard to the growth of the deployment. Whenever the need arises, additional storage and compute resources (i.e., additional servers and/or drives) can be added modularly so that resources can grow individually or together to maintain balance. Separate servers are combined to deliver the highest amount of IOPS and TBs of storage across the cluster. ScaleIO cluster of servers are used in the processing of I/O operations, making all I/O and throughput accessible to any application within the cluster. Such massive I/O parallelism eliminates bottlenecks. Throughput and IOPS scale in direct proportion to the number of servers and local storage devices added to the system, improving cost/performance rates with growth. EMC ScaleIO takes the datacenter to the next level by converging the storage, compute and application in a single layer. It brings all of the benefits of greater cost and operational efficiency due to convergence. This not only lowers TCO but also creates an infrastructure that is easier to manage, agile and more efficient. It achieves CAPEX/OPEX efficiency by providing a single-layer infrastructure on commodity hardware which means massive cost savings for large scale private cloud environments. 9
EMC ScaleIO provides many features when compared to it s competitors. CEPH is a software defined solution, which is offered by Red Hat based on the Inktank technology. CEPH is an open source software with support to Ubuntu operating system, which can be downloaded free of cost. But the code used in CEPH is unstable with poor performance and has the complexity to install and configure when compared to ScaleIO. VSAN(Virtual SAN) tight integration with vsphere supports hyper-converged infrastructure. But the solution is tightly coupled with virtualization software (hypervisors), and works with only VMWare, at maximum it can scale up to 64 nodes with weak performance whereas ScaleIO can scale from 3 to over 1000 nodes with linear performance. NetApp traditional SAN is an enterprise features and availability solution with good stability code, also provides disaster recovery to it s customer. The weakness are NetApp SAN is very expensive and comes with a pre-packaged array without any choice to the customer leading to complex deployment of solution. EMC ScaleIO has massive scalability feature along with flexible deployment options. It provides lower total cost of ownership by providing free and friction less downloads. ScaleIO provides a high performance range of tens of millions IOPS. But it does not provide support to Ubuntu operating system, compression and deduplication technology. 10
This module reviewed the business challenges that a storage administrator may encounter. We also listed and discussed the benefits of ScaleIO and the data center. Finally, we looked at competition and alternative products in the SDS space. 11
This module presents and defines the technical overview of ScaleIO. It reviews the technical aspects and primary technical components of ScaleIO. You will connect the technology and features to product benefits. 12
ScaleIO Data Client is a lightweight block device driver that exposes ScaleIO shared block volumes to applications. The SDC runs on the same server as the application. This enables the application to issue an I/O request and the SDC fulfills it regardless of where the particular blocks physically reside. The SDC communicates with other nodes (beyond its own local server) over TCP/IP-based protocol, so it is fully routable. ScaleIO Data Server contributes local storage to an aggregated pool of storage within the ScaleIO virtual SAN. An instance of the SDS runs on every host that contributes some or all of its local storage space (HDDs, SSDs, or PCIe flash cards) to the aggregated pool of storage within the ScaleIO virtual SAN. Local storage may be disks, disk partitions or even files. The role of the SDS is to actually perform I/O operations as requested by an SDC on the local or another host within the cluster. ScaleIO s control component is known as the Metadata Manager, or MDM. The MDM serves as the monitoring and configuration agent. It holds the cluster-wide mapping information and is responsible for decisions regarding migration, rebuilds, and all systemrelated functions. The ScaleIO monitoring dashboard communicates with the MDM to retrieve system information for display. The MDM is not on the ScaleIO datapath. That is, reads and writes never traverse the MDM. The MDM may communicate with other ScaleIO components within the cluster in order to perform system maintenance/management operations but never to perform data operations. This means that the MDM does not represent a bottleneck for data operations and is never an issue in the scaling up of the overall cluster. It consumes resources that are not needed by applications and/or datapath activities. It does not preempt users operations and does not have any impact on the overall cluster performance and bandwidth. To support high availability, three instances of MDM can be run on different hosts. This is also known as the MDM cluster. An MDM may run on hosts that also run SDCs and/or SDSs. The MDM may also run on a separate host. During installation, the user decides where MDM instances reside. If a primary MDM fails (due to host crash, for example), another MDM takes over and functions as primary. The third instance is as a tie-breaker in case of conflicts. 13
A Protection Domain is a set of SDSs. Each SDS belongs to one (and only one) Protection Domain. Each Protection Domain is a unique set of SDSs. The administrator can divide SDSs into multiple protection domains of various sizes, designating volume to domain assignments. As the name implies, data protection (redundancy, balancing, etc.) is established within a protection domain. Any application on any SDC can access all the volumes, regardless of protection domain assignment. Using a Protection Domain reduces the impact of simultaneous multiple failures in large clusters, isolates performance when needed and helps to fit network constraints. The recommended number of nodes in a Protection Domain is around 128. This yields optimal performance and reduces theoretical Mean Time Between Failures (MTBF) issues. Storage Pools allow the generation of different performance tiers in the ScaleIO system. A Storage Pool is a set of physical storage devices in a Protection Domain. Each storage device belongs to one (and only one) Storage Pool. When a Protection Domain is generated, it has one Storage Pool by default. The most common use of storage pools is to establish performance tiering. For example, within a protection domain, you can combine all the flash devices into one pool and all the disk drives into another pool. By assigning volumes, you can guarantee that frequently accessed data resides on low-latency flash devices while the less frequently accessed data resides on high-capacity HDDs. Thus, you can establish a performance tier and a capacity tier. When a ScaleIO volume is configured over the virtualization layer, the volume contents are distributed over all devices residing in the same Storage Pool. Since a Storage Pool can withstand the loss of one of its members, having two failures in two different Storage Pools will not cause data loss. 14
I/Os from the application are serviced by the SDC that runs on the same server as the application. The SDC fulfills the I/O request regardless of where any particular block physically resides. When the I/O is a Write, the SDC sends the I/O to the SDS where the primary copy is located. The primary SDS will send the I/O to the local drive and in parallel, another I/O is sent to the secondary mirror. Only after an acknowledgment is received from the secondary SDS, the primary SDS will acknowledge the I/O to the SDC, and an acknowledgement is sent to the application. 15
A Read I/O from the application will trigger the SDC to issue the I/O to the SDS with the Primary chunk. A Read Hit is a read to the ScaleIO system (SDS) where it finds the requested data already in the server Read Cache space. Therefore, RHs run at memory speeds, not disk speeds, and there are no disk operations required. A Read Miss is a read to the ScaleIO system when requested data is not in cache and must be retrieved from physical disks (HDD or SSD). In terms of resources consumed, one host Write I/O will generate two I/Os over both the network and back-end drives. A read will generate one network I/O and one back-end I/O to the drives. For example, if the application is issuing an 8 KB Write, the network and drives will get 2x8 KB I/Os. For an 8 KB Read, there will be only one 8 KB I/O on the network and drives. 16
A snapshot is a volume that is a copy of another volume. The ScaleIO storage system enables users to take snapshots of existing volumes. Snapshots do not consume much space initially because they are thinly provisioned. You can create snapshots of snapshots any number of them. Once a snapshot is generated, it becomes a new unmapped volume in the system. Users manipulate snapshots in the same manner as any other volume exposed to the ScaleIO storage system. Note that a snapshot can grow in one of two ways: The user writes to the original volume. Since the snapshot needs to preserve the original state, it must therefore grow. The user writes directly to the snapshot volume. 17
This module reviewed the technical overview of ScaleIO. It reviewed the technical aspects and primary technical components of ScaleIO. You connected the technology and features to product benefits. 18
This module presents the ScaleIO use cases and describes how ScaleIO features support the use cases. 19
The dependency on traditional SAN and siloed infrastructures has many inherent issues. Your business and applications depend upon dedicated storage arrays, with dedicated workloads and a complex network of Fiber Channel and HBAs. Dedicated SANs and dedicated workloads cannot always provide the scale, elasticity and flexibility needed to meet the demands of applications and the business as a whole. Storage architectures built for.now cannot always meet the demands of today, let alone tomorrow. To compensate for this you try to plan for future demand and purchase today more than you need in hopes you will be ready for two years or more from now. How can you plan for three years from now, when you cannot even plan what you will need 6 months from now? In the end, you purchase more capacity than you need today and end up incurring additional costs in the area of power and cooling. What about your applications? What will they need in the future? How flexible is hardware in meeting the needs of future applications or services and meeting supporting SLAs? How will what you have planned for today impact your ability to bring these services to market faster than your competition when you do not even know what new innovations await? Not to mention how these decisions now will impact your operational efficiency. Your staff must deal with forklift data migrations, multiple refreshes and provisioning to meet the ever changing demands. This is time that would be better served working on innovative ways to support and improve the business. It s time to leave the pains of.now. It s time for.next! 20
If you are looking to address these.now challenges and are ready to redesign your storage environment, then you are ready for SAN.Next. SAN.Next is about moving towards a software-defined, scale-out, SAN infrastructure using commodity hardware. You can leave your applications running the way they are today, on independent servers, and transform the way you handle your block storage. SAN.Next enables you to transform your SAN-based IT infrastructure to a next-generation, software-defined Server SAN to solve the problems associated with.now environments. There are many benefits in moving to SAN.Next including: Scale-out storage: with high reliability and performance Simplify capacity planning Add storage capacity & performance independently: without disruption Remove expensive FC Infrastructures Remove storage silos, migrations and over provisioning Create an architecture that aligns to IT organization structure and responsibilities 21
You can also choose to run your applications directly on your storage server nodes, bringing compute and storage together within the architecture. We call this Infrastructure.Next. Infrastructure.Next takes your datacenter to the next level by converging the storage, compute, and application in a single layer and can be achieved with EMC ScaleIO. It brings all of the benefits of SAN.Next plus even greater cost and operational efficiency due to convergence. This not only lowers TCO but also creates an infrastructure that is easier to manage, agile, and more efficient. Achieve Web-scale CAPEX/OPEX Efficiency: Single-layer infrastructure on commodity hardware means massive cost savings for large scale private cloud environments Enhance/Seamless Self-Service Provisioning: Launch services without any intervention or interruptions from IT infrastructure Improved Quality of Service: Improved SLAs via extreme performance provided by software defined storage Simplified Management: Easy to use GUI for managing cloud infrastructures Public Cloud Scale & Elasticity: Scale out private cloud infrastructures to 1000s of nodes and scale back as needed Infrastructure.Next also supports Infrastructure-as-a-Service (IaaS) initiatives. You are able to deploy a backend IT infrastructure (servers/compute, storage/das and networking, if you wish) which is designed to mirror the IaaS consumption based model pay as you go, pay as you grow. This addresses many of the issues faced when delivering IaaS using a traditional.now infrastructure. 22
However, this journey does not have to be a two-step process. If you are ready to make a full business, processes, people, and technology change today to experience even greater efficiency and performance from your infrastructure, then you are ready for Infrastructure.Next 23
Deltares is an institute for applied research. Its work focuses on deltas, coastal regions, and river basins. It is valued by governments and private companies. A few challenges that face the company are: Considering the incoming amount of huge data, research institute needed to improve scalability while keeping low cost. Existing storage does not provide required flexibility or performance. Deltares research relies on processing massive amounts of data with compromised performance. 24
To meet the challenges in the long term, Deltares opted for EMC ScaleIO which helped: Deliver the scalability needed to handle annual data growth of 100 percent. Enhanced the I/O performance are almost two times faster, delivering fast results and advice to customers. Moving towards a software-defined, scale-out, server SAN infrastructure which supports the organization for maximum choice on hardware by considering the IT costs down to maximize the resources directed at research. With Automated management there is a less risk of data loss or errors. System is also much more efficient and there is no risk periods of downtime. 25
A NoSQL database provides a mechanism for storage and retrieval of data which is nonrelational. Challenges that are faced by the organization are: The database has to support broad array of applications, that requires low latency to power on-line. The continuous availability of data is a critical challenge, with modern applications and data types. The hardware infrastructures that are used to store and process applications and data are expensive. The need to handle new, multi-structured datatypes scale beyond the capacity constraints of existing system. 26
EMC ScaleIO validates the challenges and supports the customers to transform to differentiated business model. NoSQL environments requires 99.999% availability. New nodes cans be added to the cluster with near zero impact to downtime or application SLA. ScaleIO increases infrastructure resiliency and faster recovery of the database by preventing simple failures. ScaleIO as a converged deployment consolidates all NoSQL applications and clusters on to a single platform, contributing to a lowered TCO. ScaleIO can scale compute and performance resources linearly or independently to support NoSQL requirements. ScaleIO s snapshot feature is used for roll-back purposes. This technique is tuned for database performance to rebuild missing data ensuring efficient redundancy. 27
Amsio focuses on infrastructure-as-a-service and managed hosting. It supports customers through three certified data centers and its own IP network. Challenges: Exponential growth in data volumes was putting a strain on existing storage resources. The fixed capacity and traditional storage made it difficult to meet the highly fluctuating demand. Requires a hardware-agnostic solution in order to leverage price/performance advantages of different server and media products delivering the best quality and services to their customers. 28
Customers are looking for ways to move to a hyper-converged architecture to transform their data centers. EMC ScaleIO is the only product that gives the ability for the transformation with a single software solution which provides: Software-defined solution scales without the need to add physical servers. ScaleIO feature of hardware-agnostic runs on any server and with any storage media. This will avoid vendor lock-in. ScaleIO provides the ability to match performance demands, by simply adding disks. ScaleIO automatically recognizes the extra storage, places in the pool, and rebalances the data over the new disks. Scalability can be done by removing the depreciated servers and replacing them from the cluster and replace them with new hardware, ScaleIO will recognize them and put data on the server. Supports Amsio s multi-tier storage strategy. 29
This module reviewed the ScaleIO use cases and described how ScaleIO features support the use cases. 30
If you are interested in learning more about ScaleIO solutions, go to these helpful places http://www.emc.com/storage/scaleio/index.htm http://www.emc.com/storage/scaleio/index.htm?vid=4823012763001 31
This course reviewed an introduction to the ScaleIO product. 32