VMWARE VIRTUAL SAN: ENTERPRISE-GRADE STORAGE FOR HYPER- CONVERGED INFRASTRUCTURES CHRISTOS KARAMANOLIS RAWLINSON RIVERA

Agenda 1 Virtual SAN Overview 2 Architecture and Data Services 3 Predictable Performance / Scalability and Availabilities 4 Disaster Recovery & Data Protection 5 Operations 6 vsphere Platform Interoperability 7 Use Cases 8 Roadmap 3

VMware Virtual SAN, what is it? Software-Defined Storage Distributed, Scale-out Architecture Hyper-Converged Infrastructure vsphere & Virtual SAN Integrated with vsphere platform Ready for today s vsphere use cases 4

Why VMware Hyper-Converged Software? VMware HCS Radically Simple Highest Performance Lowest Cost vcenter Server vsphere 1 Integrated software stack >100K IOPS per node $1/GB On all-flash Virtual SAN Most Flexible Deployment Options x86 Server Hardware Engineered Appliance Virtual SAN Ready Nodes 5

Radically Simple Configuration Virtual SAN is a cluster level feature deployed, configured and manage from vcenter Server: UI + API 6

Storage Policy-Based Management App centric automation Virtual Machine Storage Policy Number of disk stripes per object Flash Read Cache Reservation % Number of failures to tolerate Force Provisioning Object Space Reservation % 1 10 2 No 100 Storage Policy-Based Management vsphere + Virtual SAN Operational Benefits Stable, Robust Automation Platform Intelligent placement Fine control of services at VM level Automation at scale through policy Single Self-tuning Datastore No more management of LUNs Virtual SAN Datastore 7

Tiered Hybrid and All-Flash Architectures Hybrid All-Flash SSD PCIe Ultra DIMM Read and Write Cache Caching SSD PCIe Ultra DIMM Writes cached first, Reads from capacity tier Virtual SAN Capacity Tier SAS / NL-SAS / SATA 40K IOPS per Host Data Persistence Capacity Tier Flash Devices Reads go directly to capacity tier 90K IOPS per Host + sub-millisecond latency 8

Virtual SAN Architecture esxi-01 esxi-02 esxi-03 Minimum cluster size: 3 >>> 2 hosts + data center witness Maximum cluster size: 64 hosts vsphere & Virtual SAN Uniform hosts recommended Disks: Locally attached disks o Flash: SSD, PCIe, NVMe, UltraDIMM vsandatastore o Magnetic: SAS, SATA VMDK Network o 1GB Ethernet OR o 10GB Ethernet (preferred) Resource consumption o CPU: < 10% typical workloads REPLICA-1 VMDK VMDK REPLICA-2 o Memory: function of storage; up to 32 GB for fully populated storage h/w 9

Virtual SAN Hyper-converged Architecture Typical Hyper-Converged Solution VMware Virtual SAN Storage VM vsphere Efficiency of a Single Software Stack vsphere + Virtual SAN 2x CPU efficiency 3x memory efficiency vsphere Client APIs Third Party Simplicity of Native Integration vsphere Client Native vmotion Native DRS Native HA Single pane glass Pre-Defined Flexibility of Hardware Agnostic Latest hardware Form factors Flexible Config 10

Virtual SAN Storage Consumption Compelling Capex => 2 tiers of storage Combine properties of technologies Device claiming on ESX hosts Each host: 5 disk groups max. Each disk group: 1 flash device + 1-7 capacity devices disk group disk group disk group disk group disk group Automated or (semi) manual Disk Groups Up to 5 disk groups / host 1 Caching device / disk group 1 7 Capacity devices / disk group Controllers Pass-through or RAID-0 mode Attn: Queue depth Attn: # drives per controller 11

Two Tiers of Storage: Hybrid All writes and the vast majority of reads are served by flash storage Optimize cost of storage HDDs: cheap capacity ($0.10/GB), expensive IOPS ($5/IOPS) Flash: expensive capacity ($2/GB), cheap IOPS ($0.01/IOPS) Combine best of both worlds Performance from Flash, capacity from HDD Read Cache 70% + Write-back Buffer 30% ARC Protocol Optimize workload per tier Random IO to Flash (high IOPS) Sequential IO to HDD (high throughput) A performance tier tuned for virtualized workloads High IOPS, low $/IOPS Low, predictable latency Achieved with modest capacity: ~10% of HDD 12

Two Tiers of Storage: All-Flash Optimize cost of flash storage High-endurance (2-3 TBWPD), expensive capacity ($2-$5/GB) Low-endurance (0.2 TBWPD), cheap capacity (<$1/GB) Most cost-efficient model Small expensive high-endurance write cache Large inexpensive low-endurance capacity tier Optimize workload per tier Keep frequently written data in write cache Lots of Read IOPS from capacity tier LRU Protocol 13

Virtual SAN Objects esxi-01 esxi-02 esxi-03 vsphere & Virtual SAN Virtual SAN manages data in the form of flexible data containers called objects. A Virtual Machine consists of multiple objects: VM Home VM swap VMDK Snapshots Memory Virtual machine objects are split into multiple components based on performance and availability requirements defined in VM Storage profile. Virtual SAN Datastore 14

Object and Components Layout /vmfs/volumes/vsandatastore/foo/ foo.vmx,.log, etc foo2.vmdk The VM Home directory object is formatted with VMFS to allow a VM s configuration files to be stored on it. Mounted under the root dir vsandatastore VMFS foo1.vmdk R1 Virtual SAN Storage Objects Availability policy refelcted on number of replicas R0 R0 R0 Performance policy may include a stripe width per replica disk group disk group disk group disk group disk group Object components may reside in different disks and/or hosts VSAN network VSAN network VSAN network VSAN network VSAN network 15

Distributed Caching esxi-01 esxi-02 esxi-03 Hybrid caching 70% Read Cache, 30% Write-back buffer VMDK vsphere & Virtual SAN vsandatastore No caching on local flash where VM runs Flash latencies 100x network latencies No data transfers, no perf hit during VM migration Better overall Flash utilization (most expensive resource) Use local cache when it matters In-memory CBRC (RAM << Network latency) Lots of block sharing (VDI) More options in the (near) future 16

Consistent Performance with VM mobility Orders per minute 5-minute moving average vmotion vmotion Orders per minute Consistent performance throughout Time (seconds) 17

Virtual SAN Data Services VSAN 6.5 Q3 2016 iscsi Support Software Encryption VSAN 6.2 Q1 2016 RAID5 and RAID6 over the Network Deduplication and Compression Software Checksum QoS IOPS Limits VSAN 6.0 March 2015 All Flash 64 Node Cluster X2 Hybrid Performance VSAN Snapshots VSAN Clones Rack Awareness VSAN 6.1 Sept. 2015 Stretched Cluster vrops Management Pack Replication - 5 Minutes RPO Health and Performance Monitoring Root Cause Analysis & Guided Remediation VSAN 5.5 March 2014 18

Beta coming! RAID-5 (erasure coding) over the network With FTT=1 availability RAID-5 3+1 (4 host minimum) 1.33x instead of 2x overhead 20GB disk normally takes 40GB, now just ~27GB RAID-5 parity data data data data parity data data data data parity data data data data parity ESXi Host ESXi Host ESXi Host ESXi Host 19

Beta coming! RAID-6 (erasure coding) over the network With FTT=2 availability RAID-6 4+2 (6 host minimum) 1.5x instead of 3x overhead 20GB disk normally takes 60GB, now just ~30GB RAID-6 parity parity data data data data data data parity parity data data data data data data parity parity ESXi Host ESXi Host ESXi Host ESXi Host ESXi Host ESXi Host 20

Beta Deduplication and compression for Space Efficiency Deduplication and compression per disk group level, up to 8x data reduction Will be called Space Efficiency Space Efficiency enabled on a cluster level Deduplicated when de-staging from cache tier to capacity tier Fixed block length deduplication (4KB Blocks) esxi-01 esxi-02 esxi-03 vsphere & Virtual SAN Compressed after deduplication 4KB 2KB on capacity layer vmdk vmdk vmdk 21

All-Flash Architectures esxi-01 esxi-02 esxi-03 vsphere & Virtual SAN Becoming default deployment Cost effective Best overall results Predictable performance Predictable cost Compatible with all Virtual SAN Data Service RAID5 and RAID6 over the Network Deduplication and Compression Software Checksum QoS IOPS Limits iscsi Support Software Encryption Virtual SAN Datastore 22

Why Virtual SAN All-Flash SnS SW HW $121,696 $14,970 $148,618 $23,970 $31,960 A customer needs ~25TB usable On Hybrid you ll need: 50TB Raw ~10x1.2TB (on 4-node cluster) $19,960 2x400gb Caching SSD ~100K IOPS On All Flash w/ 2x Dedupe & EC: $86,766 $92,688 19.2TB Raw ~6x800gb Capacity SSDs (on 4-node cluster) 1 x 400gb Cache SSD ~180K IOPS VSAN Hybrid on Dell 730xd VSAN AF on Dell FC630 (w/ Space Efficency) 23

VSAN 6.0 Performance: 8 hosts All-flash 1 VM per host, 8 IOMeter workers per VM, 100% Random, 70% Read 1/2x 400 GB P3700, 3/6 800 GB S3500 per host

Enterprise-Class Scalability Virtual SAN 5.5 Virtual SAN 6.x Hybrid Virtual SAN 6.x All-Flash Hosts per Cluster 32 64 64 VMs per Host 100 200 200 IOPS per Host 20K 40K 90K Snapshot depth per VM 2 32 32 Virtual Disk size 2TB 62TB 62TB 2x 2x 4.5x 16x 31x Datastore Capacity ~ 4PT ~ 8PT ~ 8PT 2x 25

Virtual SAN Scalable Capabilities esxi-01 esxi-02 esxi-03 vsphere & Virtual SAN Scale up and Scale out architecture granular and linearly storage, performance and compute scaling capabilities Per capacity disks for capacity Per caching device for caching / capacity Per disk group for caching and capacity Per node for compute capacity (up to 64 nodes per cluster for both hybrid and all-flash architectures) 26

Virtual SAN Availability Capabilities Availability zones logically defined as failure domains vsandatastore Zero data loss and zero downtime despite hardware failures: vsphere & Virtual SAN Disk devices, Disk Controllers Hosts, Network VMDK witness VMDK Power Failure, Rack, Site (building, data center) Automated and controlled through VM-level policies Interoperate with vsphere HA and Maintenance Mode rack a rack b rack c 27

Fault Domains, increasing availability through awareness Create fault domains to increase availability Four defined fault domains FD1 = esxi-01, esxi-02 FD3 = esxi-05, esxi-06 FD2 = esxi-03, esxi-04 FD4 = esxi-7, esxi-08 To protect against one rack failure only 2 replicas are required and a witness across 3 failure domains! FD1 FD2 FD3 FD4 raid-1 esxi-01 esxi-03 esxi-05 esxi-07 esxi-02 esxi-04 esxi-06 esxi-08 vmdk vmdk witness 28

Virtual SAN Network Partition With HA restart Partition 1 Partition 2 esxi-01 esxi-02 esxi-03 esxi-04 raid-1 HA restart vsan network vmdk vmdk witness vsphere HA restarts VM on ESXi-02 / ESXi-03, they own > 50% of components!

Virtual SAN Stretch Clusters Active-Active data centers witness Virtual SAN cluster split across 2 failure domains Site-level protection with zero data loss and near-instantaneous recovery Support for up to 5ms RTT Witness VM can reside anywhere 200ms RTT latency Automated (default) or manual failover VMware vsphere & Virtual SAN 5ms RTT, 10GbE 30

Today Virtual SAN Disaster Recovery witness Site Recovery Manager vsphere & Virtual SAN Virtual SAN uses vsphere Replication as a DR replication mechanism Replication between Virtual SAN datastores enables RPO as low as 5 minutes Exclusively available to Virtual SAN 6.x, leverages vsphere Replication Leverage Site Recovery Manager for disaster recovery orchestration Stretched across metro distance, replicated across geo! Site Recovery Manager VMware vsphere & Virtual SAN 5ms RTT, 10GbE 31

Today Virtual SAN Data Protection Data protection provided through our public VADP API Data protection produces by most industry data protection vendors. VMware vsphere & Virtual SAN 32

Roadmap Unified Data Protection Overview One data protection platform for multiple use cases Nearline Protection Portable efficient snapshots that can be replicated across various topologies Portable Snapshots Multiple use cases Offline Protection Disaster Recovery Global catalog of snapshots Flexible and granular recovery options Files and folders VMs and/or VMDKs Multiple VMs (whole applications) Test & Development Robust recovery workflows Recover on-premises or in vcloud Air 33

Operations Data services are systematically and individually delivered and controlled by a policy framework Pools of Resources Capacity VSCSI Abstraction Disks Disks Disks Reserve thick 10 GB vsphere + Virtual SAN Availability 2 Failures to tolerate Clustered Physical Servers Performance Flash Read Cache 10 % Virtual SAN Datastore

VMware Virtual SAN: Storage Consumption and Capabilities esxi-01 esxi-02 esxi-03 vsphere & Virtual SAN Storage allocation and data services are individually applied and delivered per object. Redundancy Performance Availability Capacity Space Efficiency Enables efficient operating models simplified provisioning management troubleshooting Virtual SAN Datastore

Virtual Machine Provisioning Operations Virtual SAN Datastore understands the capabilities in the VM Storage Policy, it will be displayed as a matching resource. 39

Virtual Machine Provisioning Operations If the VSAN Datastore can satisfy the VM Storage Policy, the VM Summary tab will display the VM as compliant. If not, due to failures, or the force provisioning capability, the VM will be shown as non-compliant. 40

Maintenance Mode Planned downtime 3 Maintenance mode options: Ensure accessibility Full data migration No data migration

New in 6.1 Virtual SAN Health Check Plug-in Tool designed to deliver troubleshooting and health reports about Virtual SAN subsystems Cluster Health Network Health Data Health Limits Health Physical Disk Health 42

Serviceability: Performance Monitoring Cluster Level Host Level Disk Group Level Disk Level 43

Failure Handling Philosophy Traditional SANs Physical drive needs to be replaced to get back to full redundancy Hot-spare disks are set aside to take role of failed disks immediately In both cases: 1:1 replacement of disk Virtual SAN Entire cluster is a hot-spare, we always want to get back to full redundancy When a disk fails, many small components (stripes or mirrors of objects) fail New copies of these components can be spread around the cluster for balancing Replacement of the physical disk just adds back resources Resync Operations Performed in background May impact application performance due to I/O traffic contention Virtual SAN automatically balances the load for fair distribution

vsphere Platform and Products Compatibility, Interoperability esxi-01 esxi-02 esxi-03 vsphere & Virtual SAN vsphere Features: HA, DRS, Distributed Switches, vmotion VMware Products: Horizon View vrealize Suite vrealize Operations vrealize Orchestration vrealize Automation vrealize Log Insight Site Recovery Manager vsphere Replication NSX Virtual SAN Datastore 45

Advanced Monitoring and Troubleshooting with vrealize Ops Introduced with Virtual SAN 6.1 Comprehensive global view across multiple Virtual SAN cluster Hundred of KPIs simplified to an easy to consume dashboard Smart alerts deliver insight and information correlate symptoms across the stack 46

Virtual SAN Use Cases Business Critical Apps End User Computing DR / DA Test/Dev DMZ Management Staging ROBO VMware vsphere + Virtual SAN 47

Virtual SAN Support Beyond vsphere OpenStack and Containers Containers vsphere + Virtual SAN vsphere & Virtual SAN Extend into the cloud with OpenStack End-to-end policy automation Frictionless access to infrastructure Deploy Docker containers with persistent Virtual SAN storage Easily move Docker applications Failover Docker applications Speedup time to recovery Virtual SAN Datastore 48

TECHNOLOGY DIRECTION: Virtual SAN for VMware Photon Designing the Best Storage for DevOps Cloud Native Applications Persistent storage for cloud native and next generation applications Cluster Manager Photon Controller Tightly integrated with cluster managers (Mesos, Kubernetes, Swarm, etc.) Virtual SAN for VMware Photon Photon Machine Photon Photon Machine Machine Photon Machine Virtual SAN for Photon Virtual SAN for VMware Photon DevOps Focus: Promotes the developer to first class user with control of storage provisioning Agile Storage Operations: Managed solely via REST APIs FOR agile, scalable storage operations Virtual SAN Datastore for Photon 49

Virtual SAN Roadmap Q3 15 Q1 16 Q3 16 2-Node (ROBO) IPV6 Software Encryption UlltraDIMM(Diablo, etc.) Software Checksum iscsi support VSAN Core NVMe 512e/4K Drives support Remove multi-cast Tighter integration with NSX Data Protection Stretch Clusters vsphere Replication with 5- min RPO SMP-FT Support Snap/Replicate local/remote/blob VSAN Data Services Near-line Dedupe Near-line Data Compression Erasure Coding ( RAID5/6 ) QoS: IOPS Limits QoS: IOPS Guarantee Usability & Manageability HealthUI SNMP traps, stretched cluster support vr Ops Monitoring & Management VSAN Assessment Tool HealthUI - performance monitoring, detail space reporting vr Ops Proactive Rebalance HealthUI Global view, Integrated cert suite, built-in HCL cert 50

VMware Virtual SAN: Generic Object Storage Platform VMFS Block File Rest vsphere or Photon & Virtual SAN

A Distributed File System for Cloud-Native Apps Why a file system? Why distributed? Key requirements Metadata for cloud-wide access Scalable data path Clones-R-Us Multi-tenancy Multi-purpose! CONFIDENTIAL 52

A Distributed File System for Cloud-Native Apps VDI files NoSQL Distributed File System Distributed Storage Management Service VSAN (Object) store VSAN (Object) store Hyper-converged scale-out file system Relies on Object Storage backend Hardware management Resource provisioning, discovery Distributed parallel data path High availability Backend: VSAN, others Can span multiple backend clusters Design Requirements POSIX file system Cloud scale: files, clients, clones Per file / directory access control O(1) snapshot / clone creation