Building a flexible and large-scale softwaredefined storage platform for OpenStack. OPENSTACK DAY Italy May 30 th 2014, Milan

Transcription

1 Building a flexible and large-scale softwaredefined storage platform for OpenStack OPENSTACK DAY Italy May 30 th 2014, Milan Michael Kienle, CEO it-novum GmbH michael.kienle@it-novum.com Seite 1

2 Open Clouds are picking up right now! Source: RED HAT Seite 2

3 What is this all about? Open Stack: Open cloud operating system supported by the OpenStack Foundation backed up by ### well-known companies Want to learn more: listen to the other tracks Major Design Guidelines are OPEN: Open source (apache), Open design (blueprints), Open development (branches), Open community (tech leads) Although OpenStack is purposely open it partly relies on underlying proprietary storage products. To build your cloud upon a truly OPEN software-defined storage platform (SDS) you might achieve greater flexibility and $$$$savings. openattic is a SDS cloud storage platform optimized for OpenStack's Cinder component. Seite 3

4 Agenda Introduction & Agenda OpenStack Focus on Storage within OpenStack, esp. Cinder Requirements & Optimization of current approaches SDS software-defined storage How it works SDS as part of the software-defined datacenter (SDDC) Why now? Advantages openattic How to bring SDS and OpenStack together Video/Examples Seite 4

5 Open Business Solutions. Delivered! Leading provider of open source business solutions 75 employees currently; approx. 20% growth p.a. Headquarters in Fulda/DE; Branch office in Vienna/AT Hundreds of mid-sized and large corporate clients Operating public listed parent company s data center for 20+ years and migrated most of this to open source Mission: Help customers to build and run a 100% open-source softwaredefined datacenter Proven Expertise in Business Open Source Infrastructure and Applications SAP Extended Business Partner (for integrating SAP into open source solutions) Seite 5

6 OpenStack: Value Proposition Generic Cloud Drivers: Abstraction, Automation, Self-Service, efficient use of resource pools, dynamic & massive scale, common APIs Manage Virtualization across different Hypervisors (KVM, VMware, Xen, Hyper-V etc.) Support the whole IaaS stack (Storage, Security, Networking etc.) Scale far beyond the limitations of an old virtualized landscape Enable the software-defined datacenter (SDDC) Seite 6

7 OpenStack: Value Proposition Generic Cloud Drivers: Abstraction, Automation, Self-Service, efficient use of resource pools, dynamic & massive scale, common APIs Manage Virtualization across different Hypervisors (KVM, VMware, Xen, Hyper-V etc.) Support the whole IaaS stack (Storage, Security, Networking etc.) Scale far beyond the limitations of an old virtualized landscape Enable the software-defined datacenter (SDDC) Seite 7

8 Customers Perspective on Storage Storage costs optimization ($$$$) Data growth of >>40% p.a. needs to be managed automatically and way cheaper Flexibility and reduced vendor lock-in (open) Incumbent players are perfect in securing their huge margins Using innovative technology without renewing existing assets Cloud Computing has characterized the next generation IT delivery model, easy orchestration/automation of (storage) services is essential need scale-up/scale-out of underlying storage at lesser costs Software-Defined DataCenter (SDDC) becomes a major disruptive IT trend (SDS beeing part of this) Standardisation enables automation & abstraction of software from hardware -> Commodity hardware and SDS gained strength Seite 8

9 Storage in OpenStack Cinder / Block Storage Objectives Storage for running VM disk volumes on a host Ideal for performance sensititve apps Use Cases Production Applications Traditional IT Systems Database-driven Apps Dev-/Test Landscape Workloads High chance content Smaller, Random R/W Higher / Bursty IO Swift / Object Storage Ideal for cost effective, scale-out storage Backup & Archive Dropbox-like services ISO Image- and VM- Templates Snapshots Backup & Archive Repository More static content Larger, sequential r/w Lower IOPS Seite 9

10 Cinder - Details Architecture to provide block-level resources to OpenStack ( block storage as a service ) Persistent block-level storage volumes for use with OpenStack Nova compute instances (e.g. iscsi target) Manages creation, attach/detach, clone of these storage volumens Intention of Cinder: keep you from beeing locked-in: Ability to change the backend (the storage system) easily Migrate data & expect the same Nova base level of functionality VM instance Many vendor/storage plug-ins for Cinder available; /dev/vda the choice between functionality and $$$$ Cinder Note that iscsi is just an example several additional protocols are supported (e.g., FC, NFS) iscsi initiator KVM iscsi target Legend Presistent volume control Presistent volume data Seite 10

11 Choose your underlying storage system! Limitations of Commodity Storage (like a simple Linux server or comparable) because of lacking features like: node redundancy & clustering & HA, scaling QoS, Deduplication/Compression, Replication Example: Image Cloning (e.g. image from Glance) could be faster than block-based copy&paste Limitations of Enterprise Storage (the big names): Vendor lock-in $$$$$ Consider Open Source & SDS! Seite 11

12 Software-defined Storage (SDS) SDS is the abstraction of the software from the hardware. Benefits: Independence from big-name commercial developer of proprietary systems (bundle of hardware and storage os) for better pricing and freedom of choice Hardware is treated as a pool, scalable across multiple storage systems Innovative functionality (e.g. top-down policy driven), easier to administrate, faster to implement (software can be extended and maintained more easily than hardware) Part of the Software-defined DataCenter (SDDC) By 2016 the avail market for OpenSource-Storage is 2,5$B (Gartner, Tech Go-to-Market: Storage Providers Should Focus on Three Growth Opportunities Through 2016, Nov 2013), by 2018 OpenSource Storage will gain 20% of the market, approx. 10$B (Gartner, 2014 Strategic Roadmap for Storage, March 2014). Seite 12

13 The Big Picture: SDDC with 100% open source User Cloud Self Service Network Operation Center Service Desk Applications Service Design Service Transition Service Operation Capacity Management Software-defined Storage Risk Management Cloud Management Virtualization Release Management Configuration Management System (CMS) CMDB Discovery Change Management System Problem Management Application Event Correlation Process End-2-End Monitoring Incident Management SAP Other Monitors Reporting Service Level Management Infrastructure Seite 13

14 openattic Framework - Architecture Services & Tools Back end Front end Access Samba, NFS, AFP, iscsi, FC Status Checking Web Availability DRBD, Pacemaker, Corosync,... Efficiency ZFS, LVM, BTFRS, XFS Application Management Core API CLI Backup Snapshotting, Tape- Backup,... Analysis openitcockpit, Nagios,... Configuration Deployment Apps Interfaces Cloud Management OpenStack Umbrella Monitoring openitcockpit ITSM i-doit & OTRS Reporting Pentaho Billing, Backup,... SAP, SEP sesam Seite 14

15 Current Open API, Features Scalable, Cinder-ready Central Configuration Web Frontend Command Line Interface Protection Integration Tape-Systems Snapshotting Consistent Snapshotting Mirroring Block- & File-based Synchronous & asynchronous Object-based Dynamic Monitoring Availability Performance Capacity Efficiency Deduplication Compression Performance Config-Set Config-Set for Virtualization Clustering Automated Failover Multi Node Support Cloud Support OpenStack/Cinder openqrm Umbrella Management Master/Slave Architecture Unified and versatile SAN / Block Storage (FC, iscsi, FCoE) NAS / File-based (CIFS, NFS) Object Storage Support & Services Training & Optimization Integration & Development Guaranteed Response Time Guaranteed Restore Time (Appliance only) Seite 15

16 How it works: Cinder API (Icehouse) Volume create/delete Snapshot create/delete Volume attach/detach (called by Nova) Create Volume from Snapshot Copy Image to Volume / Volume to Image Get Volume Stats, Clone Volume Extend Volume (mandatory since Icehouse) openattic Cinder plug-in: Video We show you some examples how this works Seite 16

17 OpenStack Cinder / openattic: Status Unit tests Cinder driver certification file Release hopefully next one More Info: Integration in other cloud platforms has been done already Seite 17

18 Summary: openattic within OpenStack Open and comprehensive API achivies great flexibility for integration into different cloud platforms Optimization of huge data through intelligent openattic functionalities Consistent snapshots relieve backup, accelerate desaster recovery and reduce downtimes Deduplication, Compression, efficient mirroring and redundancy Huge cost savings: openattic runs on commodity hardware reducing vendor lock-in Open Source approach saves license costs Are there limitations of software-defined storage / software-defined datacenter? Seite 18

19 Summary: openattic within OpenStack Open and comprehensive API achivies great flexibility for integration into different cloud platforms Optimization of huge data through intelligent openattic functionalities Consistent snapshots relieve backup, accelerate desaster recovery and reduce downtimes Deduplication, Compression, efficient mirroring and redundancy Huge cost savings: openattic runs on commodity hardware reducing vendor lock-in Open Source approach saves license costs However, the bad news is: even software-defined datacenters still need some underlying hardware Seite 19

20 Thank you! Questions? - More than welcome! Power your cloud & scale out your data center with rock-solid Software- Defined Storage (SDS) truly on open source, join the community & more about the openattic project at Cloud Storage Intelligent SDS Unified Storage Power your cloud Rock-solid storage Scale out data center Seite 20