CDMI Support to Object Storage in Cloud K.M. Padmavathy Wipro Technologies

Transcription

1 CDMI Support to Object Storage in Cloud K.M. Padmavathy Wipro Technologies

2 Abstract Object storage, a leading emerging technology is mainly intended to handle exponential growth of unstructured data. Unlike traditional storage of files in NAS or blocks in SAN, it uses data objects. Each object is assigned a unique object ID and each object contains its own meta data along with actual data thereby removing centralized indexing. Thus it enables massive scalability, geographic independence under reasonable costs. Pitfalls in IOPs Performance, latency and proprietary interface makes object storage more suitable for archiving & backup operations instead of primary storage. Hence it has become more ideal for cloud storage. CDMI, industry standard is meant for complete life cycle management of objects in cloud. How it can be leveraged/enhanced to support Object storage will be addressed here. 2

3 Agenda Object Storage Key features of Object Storage Apt for Cloud Storage Why? Challenges in Cloud Storage CDMI How CDMI can be leveraged to address the challenges? 3

4 Object Storage Paradigm shift in data landscape Exponential growth of data (i.e.) growth in terms of petabytes, Exabytes, zettabytes over years or billions of files at an org level. Data to be accessible over network (i.e. intranet and internet) in concurrent mode with sources dispersed geographically. Large amounts of Unstructured data due to Web 2.0 What Traditional storage do? Block based and file based storage handles structured data focused on high processing and high speed network within a data center. Well suited for high performance operations in primary storage. 4

5 Object Storage What limitations does Traditional storage face? Scalability Design limits can be reached soon on no. of nodes in hierarchical structures in NAS or LBA in SAN Ineffective data management without extended metadata. Less useful for inactive, read only, fixed type of data as it adds unnecessary cost and complexity. What comes for remedy? OBJECT STORAGE 5

6 Object Storage 6

7 Key features of Object Storage Objects are assigned a unique Object Id Unique Object ID makes the need to know physical location immaterial. Manage Unstructured data Each object is a combination of data and its associated metadata. Meta data helps to manage any type of data effectively. Tremendous scalability Flat Address space paves way for very high scalability. Accessible over internet Web 2.0 and cloud storage mandate data to be accessible over internet. Object storage supports HTTP/ HTTPs protocols for RESTful data and in-flight data. 7

8 Key Features of Object Storage Custom Metadata Meta data can be customized. ILM can be handled efficiently with rich metadata. Cost Management Reduce the storage cost by storing data in suitable storage tier. Storage cost can be reduced with minimal downtime. Very suitable for Archival Enable compliance through retention policies, data protection & authentication. Easy data migration Data migration has become easy because of Object Id. 8

9 What is Cloud Storage? Manage customers data and make it accessible over n/w usually internet. Accommodate structured/ unstructured data (e.g.) file, video, image, s Scalability of data is high Multi-tenancy Eliminate disruptive data migration Reduce cost associated with DR More ideal for less frequently accessible, Backup & archive data 9

10 Object storage apt for Cloud Storage High Scalability Handle Unstructured data Cost Efficiency Exceptional control with custom metadata Multi-tenancy enabled Non-disruptive data migration More suitable for Backup & Archival Reasonable IOPS Performance Relative scalability of Storage options 10

11 Perception on Cloud Storage What applications are you most likely to store in storage cloud? Primary reason for storing data in storage cloud Source: Report from Storage Strategies NOW 11

12 Challenges in Cloud Storage 12

13 Challenges in Cloud Storage Proprietary Interface Vendor lock-in Each cloud provider defines their own standard and exposes an interface for handling data in the cloud. Interoperability Application vendors need to support plethora of APIs provided by different cloud providers. Integration of multiple s/w or supporting many interfaces has become a tedious job in bringing interoperability. Portability As migration of data to a new cloud is found cumbersome, users are continuing with same provider even if they prefer to opt for new. 13

14 Challenges in Cloud Storage Data retention Today, Cloud storage is used mostly for backup & archival purpose; as tertiary storage; data retention period is high. If the data is spanned across multiple clouds, or private & public cloud, above operations require significant administrator effort. Security of data As data in the cloud is accessible by multiple users, secure segregation of each user s data is vital. Controlled access to namespace, objects based on user privileges is very essential. 14

15 CDMI Cloud Data Management Interface Provides a standardized way to exchange data between clients and clouds, manage resident data, and transfer data between clouds Defines the standard interface that applications will use to handle entire life cycle management of data in the Cloud. To discover the capabilities of the cloud storage offering (underlying storage and data services). Manage containers and the data that is embedded in them. Setting metadata for containers and their contained data elements is possible. Manage containers, accounts, security access, monitoring information even for storage that is accessible by other protocols. 15

16 How CDMI can be leveraged? To overcome Vendor Lock in CDMI defines object model and standard interface for every operation that can be performed in the cloud. (e.g.) Creation of data object, read/write data, deletion of object etc. If cloud(storage) providers start adopting CDMI, then One management interface is sufficient for managing any no. of cloud storage Single set of APIs can be exposed to application vendors. As CDMI has good handle on metadata (created by cloud system/ cloud user), Life cycle management of data objects in the cloud is easier. As CDMI is evolving with more enhancements(features) like support for exported protocols namely NFS/CIFS, snapshot etc., it has become ideal standard for cloud storage. 16

17 CDMI to eradicate Vendor lock-in 17

18 How CDMI can be leveraged? For Data migration between, into, out of clouds. Bulk data movement is required during Data migration between clouds Data migration during upgrades of cloud implementations and Backup CDMI is interoperable to move data & associated meta data between storage cloud providers. Data migration is handled with proper serialization /deserialization options, framing the data into canonical format. 18

19 How CDMI can be leveraged for Data Migration? 19

20 How CDMI can be leveraged To promote security Transport security User & entity authentication Authorization and Access controls Data-at rest encryption Data Integrity Data Retention Security capabilities 20

21 How CDMI can be leveraged For Data Retention CDMI Retention Management is comprised of Retention Hold Deletion Object Retention 21

22 CDMI Retention Object on Hold Cloud system may automatically delete a CDMI object after the retention time and hold criteria have been met. 22

23 CDMI Adoption ISO ratification of CDMI is expected in NetApp is the first major storage company to include support for CDMI Source: Report from Storage Strategies NOW Cisco, Dell, EMC, Hewlett- Packard, IBM and other industry heavyweights are also involved in CDMI support. 23

24 Thank you Reference:

25 Backup 25

26 How different from Traditional Storage? Object based Storage Block-based Storage File-based Storage Data Unit Objects Blocks Files Protocol REST & SOAP over HTTPs SCSI/FC CIFS / NFS Support for Unstructured data Yes No No Custom Metadata Yes No No Inline Update No Yes Yes Strength Scalability and distributed access Higher performance Accessibility to shared files in NAS/SAN Coupling of data to location No coupling Tightly coupled Tightly coupled Limitations Not suitable for frequently changing data IOPs with higher performance is feasible within DC High performance File operations is feasible within DC 26

27 Benefits of CDMI Multi-vendor support Data mobility Rapid development Interoperability Portability Foundation for Innovation 27