JEUDI 19 NOVEMBRE 2015

Transcription

1 L AGILITE DU CLOUD PUBLIC DANS VOTRE DATACENTER AVEC ECS & NEUTRINO Copyright 2015 EMC Corporation. All rights reserved JEUDI 19 NOVEMBRE 2015 Denis JANNOT EMEA Principal Systems Engineer

2 SOFTWARE IS EATING THE WORLD WARE A COMPANIES EFFECTIVELY USING SOFTWARE DEVELOPMENT TO ACHIEVE COMPETITIVE ADVANTAGE ARE MORE PROFITABLE THAN THEIR PEERS 2

3 SOFTWARE DEVELOPMENT MODELS ARE EVOLVING FROM A TRADITIONAL WATERFALL MODEL TO AN AGILE MODEL Requirements Analysis Design Build Test Roll Out Project Span Planned Start Date: 1 st Jan 2014 Current Date: 24 th October 2014 Planned End Date: 31 st Dec 2014 Weeks or months available to provide the infrastructure Infrastructure needed in days or hours 3

4 NEW DATA SOURCES ARE EMERGING Traditional Data Sources Emerging Data Sources Structured Data Unstructured Data Public records Social Networks, UGC Dark Data Internet Of Things Location Data AND THEY CANNOT BE PROCESSED WITH TRADITIONAL SYSTEMS 4

5 AWS SERVICES ON PREMISE Cloud storage / Active archive Cold archive Cloud gateway Block storage IaaS PaaS Data analytics Amazon S3 Amazon Glacier AWS Storage Amazon EBS Amazon EC2 Elastic Beanstalk Amazon EMR Gateway NTP Software Syncplicity ECS CloudArray CloudBoost CloudPool ScaleIO Neutrino Pivotal Cloud Foundry Pivotal HD 5

6 ECS OVERVIEW 6

7 THE ECS JOURNEY REMOTE OFFICES INNOVATE 56% cost reduction Cloudarray USER DATA NEXT GEN APPLICATIONS Up to 83% cost reduction BIG DATA 50% faster time to market REDUCE COST 71% cost reduction 7

8 SIMPLIFY MANAGEMENT OF THE REMOTE OFFICES 8

9 TODAY S CHALLENGES WITH REMOTE OFFICES NAS appliance File server NAS appliance Backup software Backup software Backup software Deduplication appliance Deduplication appliance A file server or a NAS is located on each remote office Deduplication appliance Replication Deduplication appliance File server Backup software Deduplication appliance $ 0.021/GB/month $ 0.089/GB/month $ 0.042/GB/month The File servers and NAS appliances are then backed up and the backups are replicated CAPACITY PLANNING IS COMPLEX PRIMARY STORAGE AND BACKUP COSTS ARE HIGH 9

10 CLOUDARRAY & ECS SIMPLIFY REMOTE OFFICES MANAGEMENT 10 TB per site average capacity More recent 3 TB cached locally Replication 100 x Cloudarray VE with 3 TB cache 2 x ECS appliances U1500 $ 0.053/GB/month 56% cost reduction Protection included Hypervisors & Load balancers included For 1 PB stored, the cost reduction over 3 years is greater than $3,200,000 10

11 ENABLE COLLABORATION 11

12 USER DATA IS GENERALLY STORED IN A NAS 12

13 TODAY S CHALLENGES WITH USER DATA I hope they won t say today When do you need it? Today We need a CIFS share for our new project Users send s with attachments Infrastructure team NAS appliance NAS storage must be backed up Users 13

14 WHY SYNCPLICITY? - Save a file in any folder and it s accessible from everywhere - Share it easily - Send links instead of attachments WINDOWS 8 IPHONE ANDROID MS SURFACE MACBOOK AIR IPAD IPHONE ENABLE COLLABORATION / REDUCE PRIMARY STORAGE AND BACKUP COSTS BY 50% 14

15 ARCHIVE COLD DATA 15

16 SOME CUSTOMERS ARE BACKING UP USER DATA 16

17 TODAY S CHALLENGES WITH USER DATA NAS appliance Backup software Deduplication appliance Replication Users are storing their documents in network shares Replication NAS appliance Deduplication appliance $ 0.042/GB/month $ 0.089/GB/month $ 0.042/GB/month THE NAS APPLIANCE IS THEN BACKED UP AND THE BACKUPS ARE REPLICATED PRIMARY STORAGE AND BACKUP COSTS ARE HIGH 17

18 OTHERS ARE PROTECTING THE DATA WITH SNAPSHOTS 18

19 TODAY S CHALLENGES WITH USER DATA NAS appliance Snapshots Replication NAS appliance Users are storing their documents in network shares Snapshots $ 0.042/GB/month $ 0.01/GB/month PRIMARY STORAGE AND SNAPSHOT COSTS ARE HIGH THE NAS APPLIANCE IS THEN PROTECTED WITH SNAPSHOTS 19

20 IN BOTH CASES, INTRODUCING ARCHIVING HELPS REDUCING COSTS 20

21 NTP SOFTWARE & ECS Archive cold data NAS appliance Replication NAS appliance 2 x ECS appliances U x NTP bundle for ECS U1500 Replication $ 0.032/GB/month 39 to 82% cost reduction Protection included Load balancers included For 1 PB archived, the cost reduction over 3 years is up to $5,300,000 21

22 OTHER TRADITIONAL USE CASES 22

23 EMC CLOUDBOOST Long-Term Retention to the Cloud ROBO Desktops Laptops Files NAS/NDMP DB LAN CloudBoost virtual appliance LAN/WAN VMware & Hyper-V Databases Applications Included in the Data Protection Suite! 23

24 ISILON CLOUDPOOLS Clients SMB NFS REST HDFS SWIFT OneFS SmartPools -> CloudPools Smart Tiering to Object Store Service Provider ECS Public Cloud Key Features Stub to Cloud of choice Extending SmartPools workflow to CloudPools Ability to send encrypted data to the cloud Compression for efficient transport Simple policy based management Benefits Combine file & object store benefits Use stubs to optimize local storage space, with offsite archive protection Seamless placement and availability of data per policy One Accessible namespace 24

25 MODERNIZE APPLICATIONS WITH ECS 25

26 TODAY S CHALLENGES WITH APPLICATIONS I hope they won t say today How much capacity do you need? We need some storage for our new application Good question. We don t know how many users we ll have. Probably 5 TB. 10 TB, I think Infrastructure team When do you need it? Today Application team 26

27 TODAY S CHALLENGES WITH APPLICATIONS Access Method Access From Provisioning WAN friendly Developer friendly User Metadata One site Needed NFS / CIFS The same application can only run on one site Disaster Recovery is complex The application team needs to tell how much storage they ll need in advance The application and the storage must reside on the same site More lines of code needed to manage the file system structure and perform many checks (capacity available, share mounted, ) Any information about the files must be stored in a database 27

28 WHY OBJECT STORAGE? Access Method Access From Any site HTTP The same application can run on any site which provides native high availability and faster response time for the users Provisioning No The application team can access storage on demand WAN friendly Developer friendly User Metadata The application and the storage can be on different sites which allows many new use cases like archiving data on many sites to a central location A developer just need to download the SDK corresponding to the language he wants to use and doesn t have to manage any file system structure Information about the objects can be stored as a metadata instead of being stored in an expensive database Reduce the lines of code and the development time by at least 50% 28

29 WHY ECS? Support many APIs Provide performance and efficiency with objects of any size Enable in place data analytics Store objects in chunks A A A Delete original chunks Add parity 29

30 API EXTENSIONS TO LEVERAGE THE ECS UNIQUE CAPABILITIES Updating a byte range within an object Overwriting part of an object Appending data to an object Reading multiple byte ranges within an object Apply a retention to an object 30

31 CENTERA-LEVEL COMPLIANCE COMPLIANCE ARCHIVING AND LONG-TERM RETENTION Utilizes the CAS API to enable compliance features for Centera data Now also available for the other APIs Enabled at object or bucket level Time-based retention enforcement Immutability and checksum Privileged delete Audit logging 31

32 EXTRACT VALUE FROM YOUR DATA 32

33 TODAY S CHALLENGES WITH DATA ANALYTICS Primary storage Primary storage Replication Copy Analytics Hadoop cluster Data is then copied to an Copy Original data Hadoop is stored cluster on where 3 traditional copies storage of each file are created Hadoop cluster $ 0.042/GB/month $ 0.02/GB/month $ 0.02/GB/month Manual copy jobs are executed to protect the data in a remote Hadoop cluster Time needed to copy the data is high Duplication of data increases costs Data protection is complex and expensive 33

34 HADOOP ENVIRONMENT COMPONENTS WHAT MAKES UP A TRADITIONAL HADOOP ENVIRONMENT? DataNode(s) Metadata catalog 3 GB of RAM / million files Store the HDFS data Compute Map-Reduce job output Ethernet Network NameNode(s) Active-passive mode for HA Client(s) Enable access to HDFS Many options exist for client systems 34

35 GLOBAL DATA ANALYTICS WITH ECS Analyze the data Analyze the data Read local data Local Pool Local Pool Read local and remote data Hadoop cluster Write the output in a global pool Global Pool Site 1 Site 2 Write the output in a global pool Hadoop cluster $ 0.024/GB/month 71% cost reduction Protection included For 2 PB capacity, the cost reduction over 3 years is greater than $4,450,000 35

36 EMC ECS NFS DATA SERVICE Active / Active Scale out File 36

37 ECS U MODELS Model U300 U700 U1100 U1500 U1800 U2200 U2500 U3000 Nodes Disks Raw TB Max Power kva Max Hear BTU/h Weight lbs Height in Width in Depth in

38 BENCHMARKS More than 400 millions objects written or read per day More than 50 TB written per day More than 150 TB read per day 38

39 NEUTRINO OVERVIEW 39

40 INTRODUCING NEUTRINO TURNKEY, OPEN SOURCE PRIVATE CLOUD iphone of Cloud-Native platforms Commodity Hardware Neutrino Delivers an onpremise, private cloud in a single day Provisioning, management UI Software Services IaaS PaaS 40

41 WHY NEUTRINO? Public cloud offerings set the expectations for cloud infrastructure But they are not always possible Open source is the foundation for building cloud infrastructure But it is complex and very hard to use Cloud infrastructure is flexible to run many services But it requires many specialists to configure and tune 41

42 THE VALUE PROPOSITION Turn-key platform The iphone of CloudNative platforms Services: CloudCompute, Developer platform Stable and validated infrastructure stack Integrated monitoring, planning, showback, tenancy Single support EMC support hw and software stacks in Neutrino(not customer VMs and their developed apps) Low TCO Integrated patching and upgrades 42

43 NEUTRINO USE CASES Web and mobile applications Cloud-native applications Elastically scaling Self-healing Devops Internal development environment for cloud-native applications Hadoop batch analytics Parallel processing engine for large amounts of data 43

44 NEUTRINO PORTFOLIO Neutrino is a private cloud platform that enables customers to rapidly deploy and simply manage cloud services offerings for their tenants and users. 1. Neutrino Big Data: Big Data PaaS Hadoop-based big data service (similar to EMR) 2. Neutrino Cloud Compute: OpenStack operating environment (similar to EC2) 3. Neutrino Developer Platform: CloudFoundry on OpenStack (similar to EB) 44

45 Orchestration / Reporting NEUTRINO ARCHITECTURE Developer Platform (Cloud Foundry) Cloud Compute (OpenStack) (ScaleIO) (ECS) Big Data (Pivotal, Hortonworks) (ECS). Provisioning Engine Hardware Dense / High-Cap Add-on brick 45

46 NEUTRINO DASHBOARD 46

47 NEUTRINO ELASTICITY DEPLOY NEW NODES IN LESS THAN 15 MINUTES 47

48 NEUTRINO CLOUD COMPUTE THE FULL OPENSTACK USER EXPERIENCE 48

49 NEUTRINO CLOUD COMPUTE POWERED BY SCALEIO 49

50 NEUTRINO LOG MANAGEMENT POWERED BY ELASTICSEARCH & KIBANA 50

51 Keystone Account Service NEUTRINO ACCOUNTS Keystone Users and groups AuthN and authz Domains and projects Account service Aggregation of domains or user sets Multiple domains/idp per customer Base for account level reporting Cloud Compute, Big Data, and Object resources Get created and associated to accounts, domains within accounts Account wide management for M&R cross domain reports Hadoop Cluster 1 IDP Domain HR Object Namespace Project sandbo x Project dev Project secret Account IDP Cloud Compute Resource Pool A Logical resources created and linked to accounts Users Groups Domain Finance Users Groups Project stats Project results Project quarter s Cloud Compute Resource Pool B Keystone Default Domain Cloud Admin 51

52 NEUTRINO CHARGEBACK 52

53 NEUTRINO DEVELOPER PLATFORM AN AGILE SOFTWARE DEVELOPMENT MODEL DEVELOPERS OPERATORS Dramatically improve developer experience Agile teams, rapid iteration Ingest and incubate open source advancements and new data services Continuous delivery, no planned downtime Instant scaling of apps and data services Automation and deployment consistency at every step 53

54 NEUTRINO DEVELOPER PLATFORM TO TAKE THE APPLICATION LIFECYCLES FROM MONTHS TO HOURS AND MINUTES DEVELOPERS OPERATORS Auto-select runtimes Deploy app Select and bind middleware Scale app App Deployment: seconds Establish infrastructure Provision Add capacity Control Auth, Policies, HA Cloud Deployment: 2-4 hours 54

55 Si vous avez des questions sur cette présentation, n hésitez pas à prendre directement contact avec : Denis JANNOT denis.jannot@emc.com Copyright 2015 EMC Corporation. All rights reserved

56