Multiattach, Image Caching, and Other New Block Storage Features

Transcription

1 Multiattach, Image Caching, and Other New Block Storage Features

2 Hello! Peter Penchev a software developer who likes to play with things Misspent my youth on books and computers Worried each and every 8-, 16-, 32-, and 64-bit platform, programming language, and operating system that came my way until it broke, then put it back together until it almost worked again Lost count of the number of times when just that last fix to a free-software project would finally happen at dawn Masqueraded as a university lecturer in several programming and security courses, while actually gleefully helping others misspend their own youth in turn Currently helping build the best distributed storage system in the world at StorPool

3 OpenStack Block Storage (Cinder) A lot of data in the cloud, it has to live somewhere in the end Block storage: organize data in big volumes, then present them to the other components of OpenStack to do with them as they will: disk drives for the Nova VM instances actual storage for the Glance images anything else that wants big blobs to store data in With a properly configured backend, consumers don t need to worry about data availability, replication, reliability, or I/O performance

4 So what s new in Liberty? A lot, both on the surface and under the hood Major visible changes: Glance image caching Non-disruptive backup Major not-so-visible changes: Snapshot attachment Vendor properties for drivers and volumes Attaching a volume to multiple hosts at the same time

5 Before: shuffling data around Upload an OS root disk image to Glance Create a VM instance with a volume from this image: create a Cinder volume as large as needed transfer all the data from Glance to Cinder and write it out Transfer all the data for each new instance! What if you want to create dozens or hundreds or thousands of VM instances?

6 Maybe there s another way? Or be a bit more circumspect: create a Cinder volume transfer the data create a snapshot of the volume use the snapshot for the VM instances Not trivial to do by default or to integrate in third-party systems that make API calls to OpenStack

7 Enter the Image Cache Upload an OS root disk image to Glance Configure the Cinder image cache Create a VM instance with a volume from the image: create a Cinder volume as large as needed transfer the data from Glance to Cinder and write it out NEW: Cinder clones an internal cache volume Create a second VM instance from the same image: Cinder finds the image in the cache and clones its volume Create a third, fourth, VM instance: all Cinder does is clone a volume, no data transfer at all!

8 Backing up a Cinder volume Basically, read all the data and write it somewhere else another Cinder volume a Glance image Swift Until now (all done by Cinder): make the volume available at the Cinder volume host (attach it) start reading the data and sending it to the recipient read all the data detach the volume If the data changes, the backup is inconsistent (broken filesystem, broken database files, etc.)

9 Consistent backups? The data must not change while it is being backed up so nobody is allowed to write to the volume so any services/instances that have it attached should be suspended at the very least if not outright stopped so as to ensure consistent data on the volume itself (unmounted filesystem, properly shut down database, etc.) Kind of inconvenient, isn t it?

10 Non-disruptive backups Another way to make sure no one writes to the volume: suspend services for a moment (freeze the FS or similar), then make a snapshot! Not all Cinder backends allow snapshots to be attached Backing up a volume properly (all done by Cinder): if the Cinder volume backend allows a snapshot to be attached: create a temporary snapshot attach it to the Cinder volume host otherwise: clone the volume to a temporary new volume attach the new volume to the Cinder volume host read the data from the snapshot/volume at our leisure detach and remove the temporary snapshot/volume

11 Improvements under the hood Snapshot attachment Vendor properties: useful for third-party extensions obtaining information from the backend driver about the general state of the cloud or about particular volumes Multiattach: implemented as a general mechanism, but not supported by all drivers and not really used by other OpenStack components yet Various improvements to the abstract base class / backend driver class hierarchy

12 Thanks! For any questions: Peter Penchev StorPool integration with OpenStack: StorPool: