CEPH APPLIANCE Take a leap into the next generation of enterprise storage

Transcription

1 CEPH APPLIANCE Take a leap into the next generation of enterprise storage

2 1 Software Defined Storage An unstoppable trend Riding on the mega wave of cloud computing, big data analytics, and Internet of Things (IoT), it has become a major challenge for organizations in the public and private sectors to cope with the exponential growth of data. With previously predominant storage solutions such as RAID, NAS or SAN, IT administrator are facing problems of scaling complexities and budget constraints when overcoming explosive data storage demands. Since its rise over the past recent years, SDS (software-defined storage) systems have gained tremendous amount of market recognition in global enterprise-class storage. The market size is expected to grow at a rate of about 20% each year from $706 billion in 2015 to $1,349 billion in According to technology research firm Gartner, about 70% to 80% of unstructured data will be held on lower-cost storage managed by SDS environments by It is also predicted that, by 2019, 70% of existing storage array products will be available as software-only versions 2. Uprising demand for Ceph Amongst the SDS systems available on the market today, Ceph has earned the most popularity through its hyper-scalability, supreme flexibility and cost efficiency benefits, along with a thriving open source community to support its project development. As more and more companies have moved forward to deploy virtual environment for their IT department, they are seeking for a solution that embeds monitoring and alert functions to effectively manage their cloud platform and storage systems. Organizations are switching from VMWare systems to the widely popular OpenStack cloud platforms to save the cost on license fees, while Ceph is considered the optimal choice for backend storage of OpenStack. Enterprises that use file services such as NFS or CIFS are also looking for high-capacity and stable storage like Ceph to save their enormous amount of document archives and rich media files. Finally, for data protection purposes, business institutions are pursuing a secure fully and incremental backup and restore solution for their virtual machines. A total package to solve it all This paper introduces Ceph Appliance, a product which combines standard Ceph software with designated hardware and customizable configurations to provide a

3 2 comprehensive enterprise-standard storage solution. Ceph Appliance offers the flexibility to integrate with OpenStack platform, Amazon S3 storage service, and big data applications. With reliability, agility, manageability and cost efficiency all under one package, Ceph Appliance is built ideally for organizations pursuing high performance, maximum data security, and rapid growing storage needs. With Ceph Appliance, users are placed in an environment where they could store and manage gigantic amount of data with ease and security at all times. By eliminating hassles of storage management, upgrades and downtime issues faced in traditional systems, Ceph Appliance drastically reduces the time and cost on IT administration and improves workflow efficiency. Ceph Appliance comes with a bundled hardware and software package, hence the additional time spent to evaluate, setup, and adjust hardware on client side is eliminated, providing greater ease for installation and deployment. Ceph Appliance is built to be optimized in performance and stability, allowing enterprises to stay focused on their business performance. Other noticeable benefits of Ceph Appliance includes (but not limited to): Fully-Featured Extra plug-in functions on top of enterprise features for all-around storage solution Seamless scalability With its software-oriented characteristic, the system is massively scalable, achieving up to petabytes or even exabytes of capacity. Distributed system Data are widely distributed across storage nodes and disks to ensure no single point of failure and higher data availability. Unified platform Object, block, and file system all accessed through one unified autonomous platform for simpler management and better efficiency. Commodity hardware Ceph runs on X86 servers that are economical, easily scalable and replaceable. Users are not restricted to any proprietary hardware component.

4 3 Features List Default Feature Description Scale-out storage Capacity and performance can be dynamically increased by simply adding more nodes Data Replication Data are duplicated and stored on multiple nodes to ensure data availability Thin provisioning Storage utilization optimized by virtually adding more physical resources than are actually available Snapshot Technology Instant backup and recovery for continuous data protection Load Balancing Data requests and delivery are distributed across multiple storage nodes for resource optimization Dashboard Cluster health status at a glance with easy-to-read interface Cache Tiering SSD caching for data store to enhance lightning-fast data delivery (See Figure 3 for details) Storage Pool Data could be classified and stored in the most appropriate pool (SATA, SAS, or SSD) for efficient data management Erasure Coding A method of data protection that saves more space for data store while consuming more CPU power Plug-ins Feature Description Portal integration with Openstack and Amazon Provides API for Openstack Swift and Amazon S3 storage

5 4 S3 interfaces for object storage Integration with VMWare Provides integration with VMWare environment through NFS and CIFS protocol NFS & CIFS Service Provides file service of high availability through NFS and CIFS protocol support Ceph Monitoring app Built-in calamari app for easy monitoring of health status of Ceph system on multi-platform devices Alert Mechanism Users receive real-time and automatic notifications on their devices when error occurs or gets fixed Backup / Restore Full and incremental backup/restore solution by deploying backup and rely server for maximum data protection Big Data Integration Big data storing and crunching through Hadoop integration Ceph Accounting Usage info from each user is calculated and integrated with billing system for resource analysis and operational efficiency Performance Tuning Performance improvement on default features such as cache tiering DR Solution Data are replicated and synchronized to multiple sites to ensure business continuity Ceph Architecture The core component of Ceph is the Ceph Storage Cluster, which is based upon RADOS (Reliable Autonomous Distributed Object Store) and is composed of a group of compute and storage resources known as Ceph nodes. These nodes may consist of two types of daemons: Ceph OSD (Object Storage Daemon) Where data are managed and stored as objects on a storage node. OSDs interact with storage disks and handle read/write operations.

6 5 Ceph MON (Monitor) Maintains and distribute the cluster state to OSDs and Ceph clients before read/write operations. Normally a cluster of monitors are implemented to prevent single point of failure. The Ceph Storage Cluster accommodates a large number of nodes that communicate with each other to store, replicate and redistribute data dynamically. Data are stored within pools, which are logical groups for storing objects. Pools manage the number of replicas and the ruleset for the pool. To store data in a pool, one must have be an authenticated user with permission to access the pool. Ceph Appliance offers distributed object, block, and file storage all under one unified structure. Object storage - Accessed through Ceph RADOS Gateway (RGW) interface for cloud applications. Compatibility with S3 and Swift is also available through the RESTful API provided by Ceph RGW. Block storage - Provided by Ceph RADOS Block Device (RBD) for physical hosts or virtual machines. Ceph RBD is a distributed block device with a Linux kernel client and QEMU/KVM driver which supports features such as snapshots, thin provisioning, and replication. File storage - Provided to custom clients by CephFS, a POSIX-compliant file system with a Linus kernel client and support for FUSE. Ceph RGW, RBD, and FS are clients built on top of LIBRADOS, a library that allows apps to gain access to RADOS, with support for C, C++. Java, Python, Ruby, and PHP. Aside from these Ceph clients, apps can directly go through a custom client created using LIBRADOS and transfer data to the Ceph Storage Cluster. An architectural overview of Ceph is as follows (Figure 1):

7 6 Figure 1. Overview of Ceph architecture The workflow of Ceph is shown in Figure 2. By default, Ceph deploys three Ceph Monitors (for high availability) and three OSDs. The number of storage nodes can be infinitely scalable depending on user s needs. To store data, Ceph clients (object storage, block device, file system, or custom client created using LIBRADOS) first receive from Ceph monitor the information that determines what data to be stored. These data are then written into each OSD from Ceph clients by assigning hash code to each object data. Each Ceph OSD daemon checks its own state and state of other OSDs and reports back to Ceph monitors. Figure 2. Ceph Workflow

8 7 Efficient Data Management with Storage Pooling To meet ever-evolving storage challenges, a desired solution should be efficient, flexible, and easily scalable when necessary. Ceph Appliance achieves this by creating storage pools formed by a number of same or different types of hard disks. For example, depending on the needs of deployment, an IT admin can create a SSD pool made up of SSDs, SAS pool made up of SAS disks, etc. (Figure 3). Each pool can be composed of disks from different OSD nodes, and can be dynamically expanded by simply adding more disks to cope with ongoing data growth. An SSD pool is suitable for high speed or ultra-high speed applications, such as an online video streaming provider, online transaction processing, and financial analytic systems. A SAS-based pool is usually applied to host systems such as a web server, large web portal, or music streaming provider. A SATA pool is commonly used for archived data that are used less often but needs high storage capacity. Figure 3. Storage Pools composed of 3 types of hard disks Accelerate Peformance with Cache Tiering Ceph Appliance supports cache tiering (Figure 4) function and provides performance tuning option to suit the user s needs and preferences. This function creates a pool of fast or expensive storage devices (in this case SSD) set to act as a cache tier, and a backing pool of relatively slow or cheaper devices (in this case SATA) configured to act as an economical base storage tier. The most frequently used data (hot/active) are placed in the cache tier while the less accessed data (cold/inactive) live on the base

9 8 tier. When the data required by the user are missing in the cache tier, those data are promoted from the base tier to the cache tier and stay there as hot data for frequent and quicker access. Hot data are periodically and automatically moved out of the cache and flushed to the base tier by the tiering agent once detected inactive. Cache tiering ensures data availability and storage resilience, while effectively improving overall I/O performance and reducing latency. Figure 4. Cache Tiering using solid state drives Data Protection at Lower Cost with Erasure Coding In Ceph, stored data is generally copied and distributed among replicated pools inside the Ceph Storage Cluster to ensure high availability. The number of replicas set, depending on the accepted risk level of the user, is normally 2 to 3. When a drive fails, data is rebuilt on another location from the replica(s). The process of replication is simple, thus giving quick data write and rebuild performance. The drawback to this is that it requires 2 times or more the initial storage space. To effectively reduce the storage space and cost generated from data replications, an alternative technique known as erasure coding

10 9 is commonly used to protect stored data. The concept of it is similar to that of RAID, but further to the extent that fragments of data are encoded with a configurable number of redundant pieces and stored across different locations 3 such as hard drives, storage nodes, or even geographical locations. As opposed to replication, erasure coding (figure 5) solely requires 1.5X storage space and thus less cost, but suffers from slower data writes and rebuilds due to higher CPU intensity and latency. Generally, replication is recommended for active primary and secondary data whereas erasure coding is designed for archival data which does not require relatively high performance. Figure 5. How data is copied in a replicated pool VS an erasure coded pool Conclusion As we witness the era of gigantic data growth and transformation into software based storages, it is clear that a solution encompassing integration flexibility, hyper scalability, maximum data security, and lower TCO in the long run will play a large role in mainstream adoption. Such a solution is brought by inwinstack Ceph Appliance, which is not only a product but also a service provided by a group of top-notch cloud experts particularly focused in OpenStack and Ceph technologies. As a major contributor and active member in OpenStack ecosystem and Ceph community worldwide, inwinstack is constantly making efforts to offer professional deployment, relentless customer support and certified training services for enterprises to pave their way into the future of cloud computing and software storage.

11 10 References 1. Software Defined Storage No More Limits. Wojciech Furmankiewicz Gartner Innovation Insight: Separating Hype From Hope for Software-Defined Storage. Dave Russell and Arun Chandrasekaran. October About inwinstack inwinstack is Taiwan s leading contributor and solution provider in OpenStack/Ceph services. Started off as a group of cloud experts and senior engineers in IaaS/SDS development, we have been focusing on research and development of OpenStack/Ceph, the most popular cloud platform and SDS (software-defined storage) adopted by today s world-class tech giants. By adapting our OpenStack/ Ceph services to enterprises of any industry, we assist our clients by transforming their conventional IT infrastructure into a more stable and efficient cloud-enabled environment, allowing clients to focus on their business goals and stay competitive. For more information on our product and services, visit Terms of Use This document is Copyright 2016 inwinstack Inc. It may be freely duplicated and distributed in its entirety on an individual one to one basis, either electronically or in hard copy form. It may not, however, be disassembled or modified in any way as part of the duplication process. Hosting of the entire report for download and/or mass distribution by any means is prohibited unless express permission is obtained from inwinstack Inc. The contents contained herein are provided for your general information and use only, and neither inwinstack nor any third party provide any warranty or guarantee as to its suitability for any particular purpose. Copyright 2016 inwinstack Inc. All rights reserved. inwinstack and the inwinstack logo are trademarks of inwinstack Inc.