Introduction to Ceph Speaker : Thor

Transcription

1 Introduction to Ceph Speaker : Thor

2 Outline What s Ceph? Ceph Architecture Ceph Functions Ceph at CERN Ceph UI Ceph Solution Architectures

3 What is Ceph?. Distributed storage system - Fault tolerant, no SPoF. X86 Commodity hardware - Saving you costs, giving you flexibility. Large scale Incremental expansion - 10s to 1000s of nodes. Unified storage platform Scalable object, Block, File system.. Open source No vender lock-in. Automatically balance the file system. Data security -with 2 or more copies in different physical store media.

4 CEPH Architecture APP ( C/C++/Python/Java/ Ruby/PHP) APP OpenStack Host Hypervisor CIFS/NFS Direct S3 API Keystone API Swift API Cinder API Glance API RADOSGW ( Object ) RBD ( Block ) CEPH FS ( File ) LIBRADOS RADOS ( Reliable Autonomic Distributed Object Store ) Remote site RADOS Monitor node Monitors Monitor node Monitor node Storage node 1 Object Storage Demons (OSD) Storage node n 1. Export to remote site( Full backup ) 2. Snapshot

5 Ceph Functions RADOSGW ( Object ) RBD ( Block ) CEPH FS ( File ) CEPH OBJECT STORE CEPH BLOCK DEVICE CEPH FILESYSTEM RESTful Interface Thin-provisioned POSIX-compliant semantics S3- and Swift-compliant APIs Copy-on-write cloning Separates metadata from data User management In-memory caching Dynamic rebalancing Usage tracking Snapshots NFS/CIFS deployable Disaster recovery Copy-on-write cloning Use with Hadoop (replace HDFS) KVM/libvirt support Incremental backup

6 Ceph at CERN

7 Storage node Planning Real Config. using Ceph at CERN

8 Ceph at CERN In seven months with a 3PB cluster, we ve not had any disasters. Actually it s working quite well. Scaling Ceph at CERN - D. van der Ster

9 IOPS test report for OpenStack CERN test Ceph with OpenStack : Glance : in production for > 6 month Cinder : in production for > 1 year ~ 400 volumes: >100TB allocated, 45TB used, ~200TB including replicas

10 Ceph UI 目前 Cluster 整體狀態 : 包含使用量及 IOPS OSD MON 狀態等訊息

11 Ceph UI 呈現各主機包含 CPU 網路等相關監控資訊

12 Ceph Solution Architectures

13 Scenario 1 Ceph RBD integrate with OpenStack 1. Ceph Provide Storage space by RBD protocol for OpenStack 2. Each computing and storage resource is scalable 3. Ceph Monitor is possible to integrate with OpenStack Controller in some cases

14 Scenario 2 Ceph RGW integrate with Amazon S3 1. Ceph Provide Storage space by Rados Gateway 2. Ceph can integrate with Amazon S3 3. Ceph Monitor is scalable for large scale and can provide good performance

15 Scenario 3 Ceph with File Server & VMWare NFS CIFS provide file sharing services File Server NFS CIFS HA File Server NFS CIFS NFS integrate with VMWare 1. Ceph can provide CIFS and NFS service integrate with RBD protocol 2. Ceph can provide Storage space for VMWare solution 3. File service can provide high performance and high availability

16 Scenario 4 Ceph DR Solution 1. Ceph can modify crash map to write data to three sites for DR solution 2. DR Solution can also integrate with File Service 3. Need dedicate network line with latency less than 10 ms Cephmon Ceph- OSD Site A Ceph- OSD Cephmon Cephmon Ceph- OSD Ceph- OSD Ceph- OSD Ceph- OSD Site B Site C

17 Scenario 5 Ceph provide storage tiering solution 1. Ceph can provide storage tiering solution 2. The OSD for Ceph pool can be combined from different OSD nodes SSD Pool Tier 1 Ceph-Mon Node RGW ceph-mon SAS Pool Tier 2 Storage Node Storage Node Storage Node SSD SAS SSD SAS SSD SAS SATA Pool Tier 3 SATA SATA SATA

18 Scenario 6 Ceph integrate with BigData Solution 1. Alluxio is an open source which can integrate Ceph and Hadoop by Rados Gateway rather than CephFS 2. Alluxio can help Ceph integrate with several kinds of BigData solutions such as Spark, Hadoop, and Hbase Ceph-Mon Node RGW ceph-mon Server Storage Node ceph-osd ceph-osd ceph-osd Storage Node ceph-osd ceph-osd ceph-osd Storage Node ceph-osd ceph-osd ceph-osd

19 Architecture for Ceph integrate with BigData Solution

20 迎棧科技股份有限公司 Thank You!