Ceph, Xen, and CloudStack: Semper Melior. Xen User Summit New Orleans, LA 18 SEP 2013

Transcription

1 Ceph, Xen, and CloudStack: Semper Melior Xen User Summit New Orleans, LA 18 SEP 2013

2 2 C est Moi Accept no substitutes Patrick McGarry Community monkey Inktank / Ceph /. > ALU > patrick@inktankcom

3 3 Welcome! The plan, Stan Ceph in <30s Ceph, a little bit more Ceph in the wild Orchestration Community status What s Next? Questions

4 4 What is Ceph? besides wicked-awesome? Software All-in-1 CRUSH Scale On commodity hardware Object, block, and file Awesomesauce Huge and beyond Ceph can run on any infrastructure, metal or virtualized to provide a cheap and powerful storage cluster. Low overhead doesn t mean just hardware, it means people too! Infrastructure-aware placement algorithm allows you to do really cool stuff. Designed for exabyte, current implementations in the multi-petabyte. HPC, Big Data, Cloud, raw storage.

5 5 That WAS fast but you can find out more Find out more! Ceph.com Use it today Dreamhost.com/cloud/DreamObjects Get Support Inktank.com

6 6 OBJECTS VIRTUAL DISKS FILES & DIRECTORIES CEPH GATEWAY A powerful S3- and Swiftcompatible gateway that brings the power of the Ceph Object Store to modern applications CEPH BLOCK DEVICE A distributed virtual block device that delivers highperformance, cost-effective storage for virtual machines and legacy applications CEPH FILE SYSTEM A distributed, scale-out filesystem with POSIX semantics that provides storage for a legacy and modern applications CEPH OBJECT STORE A reliable, easy to manage, next-generation distributed object store that provides storage of unstructured data for applications

7 7

8 8

9 CRUSH Pseudo-random placement algorithm Ensures even distribution Repeatable, deterministic Rule-based configuration Replica count Infrastructure topology Weighting 9

10 hash(object name) % num pg CRUSH(pg, cluster state, rule set)

11

12 CLIENT 12

13 13

14 14

15 15

16 CLIENT 16??

17 Ceph in the Wild with Marty Stouffer 17

18 Linux Distros No incendiary devices please 18

19 19 OpenStack Our BFF Object && Block Via RBD and RGW (Swift API) Identity Via Keystone More coming! Work continues with updates in Havana and Icehouse.

20 20 CloudStack Community maintained Block Alternate primary, and secondary Community Wido from 42on.com More coming in 4.2! Snapshot & backup support Cloning (layering) support No NFS for system VMs Secondary/Backup storage (s3)

21 21 Primary Storage Flow A blatent ripoff! The mgmt server never talks to the Ceph cluster One mgmt server can manage 1000s of hypervisors Mgmt server can be clustered Multiple Ceph clusters/pools can be added to CloudStack cluster

22 22 Other Cloud So many delicious flavors SUSE Cloud Ganeti Proxmox OpenNebula A pretty package RADOS for Archipelago RBD backed BBC territory A commercially packaged OpenStack solution back by Ceph. Virtual server management software tool on top of Xen or KVM. Complete virtualization management with KVM and containers. Talk next week in Berlin

23 23 Project Intersection You can always use more friends Kernel Wireshark STGT VMWare Since Love me! iscsi ahoy! Getting creative Kernel clients for RBD and CephFS. Active development as a Linux file system. Slightly out-of-date. Some work has been done, but could use some love. One of the Linux iscsi target frameworks. Emulates: SBC (disk), SMC (jukebox), MMC (CD/DVD), SSC (tape), OSD. Creative community member used Ceph to back their VMWare infrastructure via fibre channel.

24 24 Project Intersection MOAR projects Hadoop Samba Ganesha CephFS Upstream CephFS or RBD XenServer Recently Open Source CephFS can serve as a drop-in replacement for HDFS. Ceph vfs module upstream samba. Reexporting CephFS or RBD for NFS/CIFS. Commercially supported product from Citrix. Recently Open Sourced. Still a bit of a tech preview.

25 25 Doing it with Xen* Don t let the naming fool you, it s easy Support for libvirt XenServer can manipulate Ceph! Blktap{2,3,asplode} Qemu; new boss, same as the old boss (but not really) What s in a name? Ceph :: XenServer :: Libvirt Block device :: VDI :: storage vol Pool :: Storage Repo :: storage pool

26 26 XenServer host arch Thanks David Scott! Client (CloudStack, OpenStack, XenDesktop) Xapi, XenAPI xenopsd S M adapters libvirt libxl ceph ocfs2 libxenguest libxc qemu xen

27 27 Gateway Drug No matter what you use! Come for the block Stay for the object and file Reduced Overhead Easier to manage one cluster Other Stuff CephFS prototypes fast development profile ceph-devel lots of partner action

28 28 Blocks are delicious But what does that mean? Squash Hotspots Multiple hosts = parallel workload Instant Clones No time to boot for many images Live migration Shared storage allows you to move instances between compute nodes transparently.

29 29 Objects can juggle And less filling! Flexible APIs Native support for swift and s3 Secondary Storage Coming with 4.2 Horizontal Scaling Easy with HAProxy or others

30 30 Files are tricksy You can dress them up, but you can t take them anywhere Neat prototypes Image distribution to hypervisors Still early You can fix that! Outside uses Great way to combine resources.

31 Deploying this stuff Where the metal meets the software 31

32 32 Orchestration The new hotness Chef Puppet Ansible Salt Procedural, Ruby Model-driven Agentless, whole stack Fast, 0MQ Written in Ruby, this is more of the devside of DevOps. Once you get past the learning curve it s powerful though. Aimed more at the sysadmin, this procedural tool has a very wide penetration (even on Windows!). Using the built-in OpenSSH in your OS, this super easy tool goes further up the stack than most. Using ZeroMQ this tool is designed for massive scale and fast, fast, fast. Unfortunately 0MQ has no built in encryption.

33 33 Orchestration Cont d MOAR HOTNESS Juju Crowbar ComodIT Ceph-deploy Canonical Unleashed Dell has skin in the game Others are joining in Doing it w/o a tool Being language agnostic, this tool can completely encapsulate a service. Can also handle provisioning all the way down to hardware. Complete operations platform that can dive all the way down to BIOS/RAID level. Custom provisioning and orchestration, just one example of how busy this corner of the market is. If you prefer not to use a tool, Ceph gives you an easy way to deploy your cluster by hand.

34 Ceph Community All your space are belong to us 34

35 35

36 Code Contributions Up and to the right! 36

37 Commits Up and to the right! 37

38 List Participation Up and to the right! 38

39 What s Next? This Ceph thing sounds hot. 39

40 40 The Ceph Train Hop on board! Geo-Replication Erasure Coding Tiering Governance An ongoing process Reception efficiency Headed to dynamic Making it open-er While the first pass for disaster recovery is done, we want to get to built-in, worldwide replication. Currently underway in the community! Can already do this in a static pool-based setup. Looking to get to a use-based migration. Been talking about it forever. The time is coming!

41 41 Get Involved! Open Source is Open! CDS Ceph Day IRC Lists Quarterly Online Summit Not just for NYC Geek-on-duty makes the world go Online summit puts the core devs together with the Ceph community. More planned, including Santa Clara and London. Keep an eye out: During the week there are times when Ceph experts are available to help. Stop by oftc.net/ceph Our mailing lists are very active, check out ceph.com for details on how to join in!

42 42 Projects Patches welcome Wiki Redmine IRC Lists velopment/project_ideas Lists, blueprints, sideboard, paper cuts, etc. All the things! New #ceph-devel Splitting off developer chatter to make it easier to filter discussions. /mailing-list-irc/ Our mailing lists are very active, check out ceph.com for details on how to join in!

43 43 Questions? Comments? Anything for the good of the cause? WEBSITE Facebook.com/cephstorage