vsphere Virtual Volumes

vsphere Virtual Volumes Technical Overview Josh Atwell Andy Banta Special Thanks to Rawlinson Rivera and Cormac Hogan

Presenters Josh Atwell Solutions Architect, SolidFire Andy Banta Storage Janitor, SolidFire 2

App-centric delivery of storage objects Today s Infrastructure-centric Today Model App-centric Automation Static pre-allocation of shared storage container (Volume) Data services tied to storage container Vendor specific management Dynamic delivery of storage service levels when needed Fine control of data services at the VM level Common management across heterogeneous devices Rapid provisioning No overprovisioning of resources QoS automation Simple change management 3

vsphere Virtual Volumes Management & Integration Framework for External Storage vsphere Virtual Volumes The Basics Virtual disks are natively represented on arrays Enables VM granular storage operations using array-based data services Storage Policy-Based Management enables automated consumption at scale Supports existing storage I/O protocols (Block or NFS) Industry-wide initiative supported by major storage vendors Included with vsphere 4

vsphere Virtual Volumes Architecture Storage Policies vsphere Admin Protocol Endpoint vsphere Virtual Volumes PE Data Path Control Path Capacity Vendor Provider (VASA) Access Virtual Datastore Published Capabilities Snapshot Replication Storage Admin Deduplication QoS VVOLs Control Path SAN / NAS 5

VVols Components Objects Virtual Volumes Storage Containers Protocol Endpoints vstorage/vsphere API for Storage Awareness (VASA) Provider Concepts Out of Band Configuration Bindings Policy-Based Management VVols Capabilities 6

Vvols Just Stuff 7

vsphere Virtual Volumes Virtual Volumes Virtual machine objects stored natively on the array. No Filesystem on-disk formatting required vsphere Web Client View There are five different types of recognized Virtual Volumes: CONFIG vmx, logs, nvram, log files, etc DATA VMDKs MEM Memory Snapshots vvol SWAP Swap files Other Vendor solution specific

Storage Containers Where Stuff Lives VM VM VM VM 9

Storage Container Storage Containers vsphere Virtual Volumes Logical storage constructs for grouping of virtual volumes. Defined and setup by storage administrators on the array. Storage Containers Capacity is based on physical storage capacity Logically partition or isolate VMs with diverse storage needs and requirement Storage policy settings based on data service capabilities Minimum one storage container per array Maximum depends on the array SAN / NAS 10

Differences between Storage Containers and LUs Storage Containers Size based on array capacity Max number of SCs depend only on the array ability Size of SC can be extended Can distinguish heterogeneous capabilities for different VMs (Virtual Volumes) provisioned in that SC Simply an attribute of the VVols for SF LUN Fixed size mandates more number of LUs Needs a FileSystem Can only apply homogeneous capability on all VMs (VMDKs) provisioned in that LU. Managed by In-band FileSystem commands

Storage Container (SC) vsphere Web Client Datastore What do the Admins see? How are the storage containers setup? What does the vsphere Admins see? Why are we still creating datastores in this new model? Storage Management UI Storage Container 12

Virtual Volumes vsphere Web Client vsphere Admin View Storage Admin View Storage Management UI VM objects view from a datastore VM objects view from a storage container on an array

Common Questions Are Vvols required for vsphere 6? How does this help with performance/capacity management? Do I lose anything with virtual volumes? 14

Protocol Endpoints How You Get the Stuff 15

Protocol Endpoints Protocol Endpoint vsphere Virtual Volumes Virtual Datastore PE Storage Container Data Path Protocol Endpoints Access points that enables communication between ESXi hosts and storage array systems. Part of the physical storage fabric Created by Storage administrators Scope of Protocol Endpoints Compatible with all SAN and NAS Protocols: - iscsi - NFS v3 - FC - FCoE Existing multi-path policies can be applied to the PE SAN / NAS 16

Protocol Endpoints Today, there are different types of logical management constructs to store VMDKs/objects: NFS Mount Points IP or block based datastores Datastores serve two purposes today: Endpoints receive SCSI reads, write commands Storage Container for large number of VMs metadata and data files Differences between Protocol Endpoints and Datastores: PEs no longer stores VMDKs but it only becomes the access point. Now you wont need as many datastores as before Certain offloading operation will be done via VASA and other will be done using the standard protocol commands protocol endpoint SCSI: proxy LUN 1 VVol = VMDK vsphere storage fabric PE datastore = storage container One entity on the fabric storage system

Discovery Procedures Protocol Endpoint Protocol Endpoint vsphere Virtual Volumes Virtual Datastore PE Data Path Protocol Endpoint discovery process SCSI PEs are discovered during an ESX rescan ESX will identify PEs and maintain all discovered PEs in a database. Storage Container SAN / NAS 18

VASA Provider 19

VASA Provider (VP) vsphere Virtual Volumes Virtual Datastore Control Path Software component developed by Storage Array Vendors ESX and vcenter Server connect to VASA Provider Protocol Endpoint PE Data Path Vendor Provider (VASA) Provides Storage awareness services Single VASA Provider can manage multiple arrays Storage Container Supports VASA APIs exported by ESX Control Path VASA Provider is implemented within the cluster s management server or firmware Responsible for creating Virtual Volumes SAN / NAS

Discovery Procedures Storage Container vcenter Virtual Volumes Snapshot Replication VASA Provider Storage Container Discovery Process Storage admin sets up Storage Container Capabilities are applied to the Storage Containers VASA Provider discovers Storage Container and reports to vcenter Any new VMs that are created will subsequently be provisioned in the Storage Container

Provisioning Workflow Under the Covers vsphere Virtual Volumes Provisioning operations are translated into VASA API calls in order to create the individual virtual volumes. vsphere Admin 1. Create Virtual Machines 2. Assign a VM Storage Policy 3. Choose a suitable Datastore Virtual Datastore PE offload to array Under the Covers vvol DATA vvol CONF vvol SWAP Storage Container storage arrays vvol vvol vvol SWAP CONF DATA Provisioning operations are offloaded to the array for the creation of virtual volumes on the storage container that match the capabilities defined in the VM Storage Policies

VVOL/SPBM Provisioning Workflow 2 Capabilities are used to build storage policies 3 User selects available storage policy at provisioning Virtual Machines 1 Virtual Volumes Storage Container(s) Storage presents capabilities of the storage container to VASA Provider VASA Provider Storage policies 6 Virtual Volume 4 5 Virtual Volume is provisioned through VASA provider Administrators can monitor policy compliance Virtual Disk Virtual disk resides on Virtual Volume aligned with storage policy

Storage Capabilities and VM Storage Policies Storage Capabilities are array based features and data services specifications that capture storage requirements that can be satisfied by a storage arrays advertised as capabilities. Storage capabilities define what an array can offer to storage containers as opposed to what the VM requires. Arrays Storage Capabilities are advertised to vsphere through the Vendor Provider and VASA APIs In vsphere Storage Capabilities are consumed via VM Storage Policy constructs. VM Storage Policies is a component of the vsphere Storage Policy-based management framework (SPBM) VM Storage Policy Storage Capabilities for Storage Array Capacity Access Published Capabilities Snapshot Replication Deduplication QoS vsphere VM Storage Policy Management Framework SPBM Datastore Profile object manager virtual disk Virtual Datastore

Storage Policy Based Management (SPBM) VM Policies

Storage Policy Based Management (SPBM) Array Capabilities Storage Policy-Based Mgmt. Virtual Volumes APIs Publish Capabilities Storage admin CV CV CV Array based features and data services Defines what an Array can offer Advertised to ESX through VASA APIs 26

Storage Policy Based Management (SPBM)

SolidFire Implementation Data path Control path RPC calls vsphere PE CM PE PE PE MGR VP TS TS TS VVOL VVOL VVOL VVOL 28

Bindings Truck A Truck B Truck C 29

Binding Operations Virtual Volume Lifecycle Bindings are data path coordinating mechanisms that occurs between VASA providers and ESXi hosts for accessing virtual volume. Different Binding Mechanism: Binding allows array create I/O channels for a virtual volume Unbind destroys the I/O channel for a virtual volume to a given ESXi host Rebind provides the ability to change the I/O channel (PE) for a given virtual volumes run time using events. VM Creation create VP rebalance delete VM destroy REBIND VM power-on open (2) bind I/O unbind VM power-off close (2) I/O read (2) I/O write (2)

Binding Mechanism Workflow Bindings mechanism workflow ESXi hosts reference virtual volumes via a particular binding ID and communicates with the VASA providers via API calls with the given virtual volume ID. VASA provider return communication with a protocol endpoint ID as well as a secondary ID. A single virtual volume can be bound to multiple PE protocols simultaneously: FC NFS FCoE iscsi VASA Provider may choose to unbind a virtual volumes to a PE for several reasons in order to provide vendor specific capabilities. Virtual Volume Lifecycle VM Creation create VP rebalance delete VM destroy REBIND VM power-on open (2) bind I/O unbind VM power-off close (2) I/O read (2) I/O write (2)

Bind Requests Virtual Volumes PE Bind VASA Provider IO path is established through a VASA Bind request VASA Provider does two things upon receiving a Bind request Returns a PE ID to which the VVol is bound A unique secondary ID to be used for IO between the bound VVol and PE For SCSI, the secondary ID is the secondary LUN ID VVol VVol M:M Relationship between VVOL and PE For SolidFire, VVOL is bound to the PE on the node of the primary slice. 32

Virtual Volumes Other Bind Scenarios UNBIND Virtual Volumes PE VVol VVol UNBIND VASA Provider Delete binding of a VVol to the PE VASA Provider may choose to delete a binding for several reasons REBIND Virtual Volumes PE PE REBIND VASA Provider Move VVols across different PE VASA Provider may choose to issue a rebind for several reasons Eg., Slice migration VVol VVol

Backup Information 34

Migration Scenario: with VAAI vs Virtual Volumes vsphere Admins Migrate VM from VMFS to VVOL datastore 1 vsphere 2 software data mover implementation VMFS VVOLs VASA API Compatible Array

vsphere Virtual Volumes Offload Operations vsphere Admins offload to array vsphere Virtual Machine Operation Offloaded Virtual Machine provisioning Virtual Machine deletes Virtual Machine full clones offload to array Virtual Machine Linked Clones Virtual Machine Snapshots Storage vmotion VMFS VVOLs 36

Migration Scenario: with VAAI vs Virtual Volumes vsphere Admins Migrate VM from VMFS to VVOL datastore 1 vsphere 2 vmkernel data mover use VAAI APIs XCOPY /WRITE_SAME primitive for cloning operation VMFS VVOLs VAAI & VASA API Compatible Array

VAAI vs VVol VAAI Block and NAS how they apply on a VVol vsphere Admins clone VM onto same VVol container 1 vsphere Admins clone VM from VMFS to VVol container 1 If you have it, what are the use cases: vsphere vsphere cloning operation offload to array 2 2 vmkernel data mover use VAAI APIs XCOPY /WRITE_SAME primitive for cloning operation 3 vendor native clone utilized SC-A VMFS VVol VAAI & VASA API Compatible Array VAAI & VASA API Compatible Array

vsphere Admins VAAI vs VVol vsphere Admins 1 clone VM from VVol container to different VVol container 1 clone VM from VVol container to different VVol container 2 offload to array vsphere cloning operation 3 SC-A SC-B always attempt default operation with VASA API primitives 1 default fail back 2 If default operations fails VAAI API primitives are used SC-A vsphere 3 vmkernel data mover uses VAAI primitives for cloning operation SC-B offload to array 2 vendor native clone utilized with VASA primitives Fully VAAI & VASA APIs Compatible Array Fully VAAI & VASA APIs Compatible Array

Snapshots Snapshots are a point in time copy on write image of a Virtual Volume with a different ID from the original. Virtual Volumes snapshots are useful in the contexts of creating: a quiesced copy for backup or archival purposes, creating a test and rollback environment for applications, instantly provisioning application images, and so on. Managed Snapshot - vsphere Unmanaged Snapshot - Array Two type of snapshots supported: Managed Snapshot Managed by ESX. A maximum of 32 snapshot are supported for fast clones Unmanaged Snapshot Manage by the storage array. Maximum snapshot dictated by the storage array

vsphere Virtual Volumes Supported Features Storage Policy-Based Mgmt. vsphere Virtual Volumes Storage Policy Capacity Availability Security Performance Data Protection Supported vsphere Features SPBM Thin Provisioning Linked Clones Native Snapshots Protocols: NFS3, iscsi, FC, FCoE View Storage Accelerator (CBRC) PE PE VASA Provider vmotion SvMotion DRS XvMotion Published Capabilities vsphere SDK (VC APIs) Snapshot Replication Deduplication VDPA/VDP View vrealize Operations vrealize Automation Encryption Stateless / Host Profiles 41

Overview VVols Motivations Separation of VMware and storage management Make use of existing storage protocols Separate concepts of datastore and logical collection of storage Better use of available storage Less contention between ESX hosts Better use of storage network bandwidth The upshot: better scalability and control Configuration Examples Memory overprovisioning Multi-tenant use of VVols Disaster Recovery configurations

Motivations Ease of management Built on top of existing protocols Datastore scalability Less wasted storage space No filesystem contention Logical Storage Container concept as a replacement Scalability through less SAN bandwidth No vsphere data mover for normal operations Less Storage vmotion Less need for SDRS and SIOC 43

vsphere Virtual Volumes Is An Industry-wide Initiative And Many More 29 Partners in the Program Multiple Ready at GA Unique capabilities 44

Common Questions How do I configure the VASA provider and protocol endpoints? What happens if the VASA provider is unavailable? Is there anything I can start doing TODAY that will help with the transition to virtual volumes? 45