All you need to know about OpenStack Block Storage in less than an hour. Dhruv Bhatnagar NirendraAwasthi

Transcription

1 All you need to know about OpenStack Block Storage in less than an hour Dhruv Bhatnagar NirendraAwasthi

2 Agenda 1 What is Cinder 2 Architecture & Services 3 Concepts, Configuration and CLI 4 Service Interactions 5 Advanced Features 6 Cinder Drivers 7 Veritas HyperScale and Cinder 8 Q&A 2

3 OpenStack Architecture 3

4 Different types of Storage in OpenStack Ephemeral Non-persistent Life Cycle coincides with an instance Usually local FS/QCOW file Shared FS Example: NFS Manilla Object Typically cheap and deep Commonly SWIFT Use cases: photos, mp4 etc Block Foundation for the other types Think raw disk Typically higher performance Cinder 4

5 What is Cinder. short description of Cinder is that it virtualizes the management of block storage devices and provides end users with a self service API to request and consume those resources without requiring any knowledge of where their storage is actually deployed or on what type of device 5

6 What is Cinder used for. Enables user to manage their storage Create volume Create Snapshot Backup Attach/Detach Cinder provides commands and API s to interact with vendors storage backend persistent storage to VM s Exposes vendors storage hardware to the cloud 6

7 Agenda 1 What is Cinder 2 Architecture & Services 3 Concepts, Configuration and CLI 4 Service Interactions 5 Advanced Features 6 Cinder Drivers 7 Veritas HyperScale and Cinder 8 Q&A 7

8 Cinder Architecture Rest API Cinder Scheduler Cinder Backup Cinder API Messaging Bus (AMQP) Cinder Volume (Backend) Cinder Volume (Backend) Cinder Volume (Backend) 8

9 Agenda 1 What is Cinder 2 Architecture & Services 3 Concepts, Configuration and CLI 4 Service Interactions 5 Advanced Features 6 Cinder Drivers 7 Veritas HyperScale and Cinder 8 Q&A 9

10 CLI Commands List Cinder services openstack volume service list 10

11 Add Cinder Backend (cinder.conf) Enable Multi Backend Support by adding different sections for each cinder driver enabled_backends parameter used to set which backends to enable Any changes to cinder configuration needs cinder services to be restarted afterwards 11

12 CLI Commands List Volume types openstack volume type list 12

13 Volume Types Created by Admin After creating a new Volume Type Update volume_backend_name key to hold the value of the Backend Name Create a volume out of this new volume type to enable Vendor provided backend features 13

14 CLI Commands Show Details of a volume type openstack volume type show <Volume-Type> List Volumes openstack volume list 14

15 Volume status Status creating available reserved attaching detaching in-use maintenance deleting awaiting-transfer error error_deleting backing-up restoring-backup error_backing-up error_restoring error_extending downloading uploading retyping extending Description The volume is being created. The volume is ready to attach to an instance. The volume is reserved for attaching or shelved. The volume is attaching to an instance. The volume is detaching from an instance. The volume is attached to an instance. The volume is locked and being migrated. The volume is being deleted. The volume is awaiting for transfer. A volume creation error occurred. A volume deletion error occurred. The volume is being backed up. A backup is being restored to the volume. A backup error occurred. A backup restoration error occurred. An error occurred while attempting to extend a volume. The volume is downloading an image. The volume is being uploaded to an image. The volume is changing type to another volume type. The volume is being extended. 15

16 CLI Commands Create Volumes openstack volume create test-vol-1 --size 50 openstack help volume create Attach Cinder volume to a nova instance openstack server add volume <instance-id> <vol-id> 16

17 Agenda 1 What is Cinder 2 Architecture & Services 3 Concepts, Configuration and CLI 4 Service Interactions 5 Advanced Features 6 Cinder Drivers 7 Veritas HyperScale and Cinder 8 Q&A 17

18 Cinder Interactions (Creating a Volume) Cinder Client Cinder Scheduler Cinder API Messaging Bus (AMQP) Cinder Volume (HyperScale) Cinder Volume (HyperScale) Cinder Volume (HyperScale) 18

19 Cinder Interactions (Attach a volume) iscsi is just an example. Several other protocols are supported as well VM Nova VM Cinder /dev/vda Storage Controller KVM iscsi Initiator iscsi Target 19

20 Volume API 20

21 Agenda 1 What is Cinder 2 Architecture & Services 3 Concepts, Configuration and CLI 4 Service Interactions 5 Advanced Features 6 Cinder Drivers 7 Veritas HyperScale and Cinder 8 Q&A 21

22 Advanced features Snapshot A snapshot is a point-in-time copy of the data that a volume contains A snapshot would live on the same storage backend as the active volume Quota Admins set this limit on volume, backup and snapshot capacity depending on policy settings Volume transfer Transfer a volume from one user to another user Encryption Encryption is done by Nova using dm-crypt which is a transparent disk encryption subsystem in Linux kernel Backup Full and incremental backups are supported Supported providers are Swift, Ceph, NFS, GlusterFS, POSIX, Google Cloud and IBM TSM 22

23 Volume Migration Move data between two backends with same volume type Two backends can be located on same or different cinder nodes # cinder migrate [--force-host-copy [<True False>]] [--lock-volume [<True False>]] <volume> <host> 23

24 Volume Retype Move between two backends with different volume types Two backends can be located on same or different Cinder Volume node # cinder retype [--migration-policy <never on-demand>] <volume> <volume-type> 24

25 Volume Groups Concept introduced in Newton Different volumes used by the same applications / workloads to be grouped and managed together Operations supported Create, delete, update, show and list groups 25

26 Glance Store and Create a VM 26

27 Glance Image Volume Cache When Image is downloaded, a volume cache is created Every create boot volume call will clone a volume instead of downloading the image again Configuration: image_volume_cache_enabled = True image_volume_cache_max_size = SIZE_IN_GB image_volume_cache_max_count = MAX_COUNT And then Launch VMs via boot from volume option 27

28 Using Cinder as Backing Store for Glance Backend should support fast cloning capabilities Every Create Volume from Image call checks for availability of image in backend If image is not present, it is downloaded and converted to raw If image is present, image is cloned to new volume Performance is improved 30 times when compared to downloading the same image 2 nd time Backends supporting de-duplication will perform even better 28

29 Agenda 1 What is Cinder 2 Architecture & Services 3 Concepts, Configuration and CLI 4 Service Interactions 5 Advanced Features 6 Cinder Drivers 7 Veritas HyperScale and Cinder 8 Q&A 29

30 Cinder Drivers What do they do? Map requests from Cinder to external Storage Platform LVM Various other Storage Controllers Distributed File Systems Currently about 100 drivers in github

31 Cinder Drivers Block Device Driver (local) Blockbridge (iscsi) CloudByte (iscsi) Coho (NFS) Datera (iscsi) Dell Equallogic (iscsi) Dell Storage Center (iscsi/fc) Disco (disco) DotHill (iscsi/fc) DRBD (DRBD/iSCSI) EMC VMAX (iscsi/fc) EMC VNX (iscsi/fc) EMC XtremIO (iscsi/fc) EMC ScaleIO (scaleio) Fujitsu ETERNUS (iscsi/fc) GlusterFS (GlusterFS) HGST (NFS) HPE 3PAR (iscsi/fc) HPE LeftHand (iscsi) HPE MSA (iscsi/fc) HPE XP (FC) Hitachi HBSD (iscsi/fc) Hitachi HNAS (iscsi/nfs) Huawei (iscsi/fc) IBM DS8000 (FC) IBM Flashsystem (iscsi/fc) IBM GPFS (GPFS) IBM Storwize SVC (iscsi/fc) IBM XIV (iscsi/fc) Infortrend (iscsi/fc) Lenovo (iscsi/fc) LVM (iscsi) - Reference NetApp ONTAP (iscsi/nfs/fc) NetApp E Series (iscsi/fc) Nexenta (iscsi/nfs) NFS (NFS) Reference Nimble Storage (iscsi) Oracle Zfssa (iscsi/nfs) Pure Storage (iscsi/fc) ProphetStor (iscsi/fc) Quobyte (quobyte) RBD (Ceph) - Reference Scality SOFS (scality) Sheepdog (sheepdog) SMBFS (SMB) SolidFire (iscsi) Tegile (iscsi/fc) Tintri (NFS) Violin (FC) VMware (VMDK) Virtuozzo Storage (NFS) Windows (SMB) X-IO technologies (iscsi/fc) Veritas HyperScale Veritas Access 31

32 Minimum driver features Drivers must implement support for the core features: Volume Create/Delete Volume Attach/Detach Snapshot Create/Delete Create Volume from Snapshot Copy Image to Volume Copy Volume to Image Clone Volume Extend Volume 32

33 Commodity Storage Drawbacks / Limitations HA Scale QoS Data Dedup Efficient Tiering 33

34 Vendor specific features Different vendor backend enable different use cases Backend are selected by cinder scheduler Every backend is created while thinking of specific set of capabilities The vendor specific features are thus exposed via Custom volume types Extensions 34

35 Provides ability to choose Plugin architecture, multiple choices for backend storage Don t have to choose just one Use the storage that fits your needs Mix of proprietary and open-source choices 35

36 Agenda 1 What is Cinder 2 Architecture & Services 3 Concepts, Configuration and CLI 4 Service Interactions 5 Advanced Features 6 Cinder Drivers 7 Veritas HyperScale and Cinder 8 Q&A 36

37 Integrating Storage Management with OpenStack 37

38 HyperScale: OpenStack Integration Volume extension Compute Scheduler Collector Volume Dashboard 41

39 HyperScale and Cinder High Availability with DAS Automated snapshot management Predictable Performance Hyper Scale Zero Backup Window Real Time Storage Insight Storage Cost Saving 39

40 Key Takeaways Manage Block Storage via self service APIs Cinder provides CLI to create and manage OpenStack volumes Cinder backends can be configured through cinder.confconfigfile VM spawn time can be optimized using either image volume cache or cinder as glance store Cinder volume drivers are pluggable interfaces to map Cinder requests to storage platform 40