Building Your Own Robust and Powerful Software Defined Storage with VMware vsan. Tips on Choosing Hardware for vsan Deployment

Transcription

1 Building Your Own Robust and Powerful Software Defined Storage with VMware vsan Tips on Choosing Hardware for vsan Deployment

2 Agenda 1 Overview of VSAN 2 VSAN VCG at a Glance 3 VSAN Hardware Guidance (Ready Nodes, Post Deployment Experience) 4 Ready Node Configurator Demo 5 VSAN Certification Under the Hood 6 Q & A 2

3 Overview of VSAN

4 Powered by VMware VSAN and vsphere Runs on any standard x86 server Pools HDD/SSD into a shared datastore vsphere VSAN Delivers enterprise-grade scale and performance Managed through per-vm storage policies VSAN Datastore Deeply integrated with the VMware stack 4

5 Supporting a Broad Variety of Use Cases Business Critical Apps Virtual Desktops (VDI) DMZ Test/Dev Virtual San DR / DA Staging Management ROBO 5

6 VSAN is the Market Leader 5,000 VSAN Customers >200% Y-O-Y Growth 40% % Penetration in Fortune 1000 (US) Countries have VSAN Presence Industry Verticals use VSAN 6 20 of top 25 Finance Companies in US 20 of top 25 High Tech Companies in US 12 of top 15 Retail companies in US 5 of top 7 Healthcare companies in US

7 vsan VCG at a Glance

8 VSAN Offers the Widest Diversity of Server Choices 164 Ready Nodes 65 Unique Server Platforms 15 Server Partners 8 8

9 VSAN Offers the Most Flexibility of Configuration Choices 164 Ready Nodes 65 Unique Server Platforms 15 Server Partners Widest Diversity of Platform Choices Most Flexibility 9 9

10 We are Continuously Refreshing and Keeping Up to Date. 164 Ready Nodes 65 Unique Server Platforms 15 Server Partners 70 Ready Nodes AUGUST

11 We are Continuously Refreshing and Keeping Up to Date. 163 NVMe? NVMe August 2016 August

12 We are Continuously Refreshing and Keeping Up to Date. 163 NVMe 0 NVMe August 2016 August

13 We support the latest NVMe Devices P3500, 3600, 3700 SN150 PM

14 Broadest Deployment Options Broadest Deployment Choices VxRail Ready Node Build Your Own Deployment Choices 164 Ready Nodes 65 Unique Server Platforms 15 Server Partners 163/ 220 NVMe / PCIe devices 122 IO controllers 14 1

15 Broadest Deployment Options Software Flexibility Ready to Run Ready Nodes Broadest Deployment Choices VxRail Ready Node Build Your Own Factory Install 164 Ready Nodes 65 Unique Server Platforms 15 Server Partners 163 / 220 NVMe / PCIe devices 122 IO controllers 15 1

16 Broadest Deployment Options Out of Box Experience Ready to Run Ready Nodes + Auto CONFIG Broadest Deployment Choices Agility VxRail Ready Node Build Your Own 164 Ready Nodes 65 Unique Server Platforms 15 Server Partners 163 / 220 NVMe / PCIe devices 122 IO controllers 16 1

17 Broadest Deployment Options Real Time Validation Ready Node Configurator 2.0 Out of Box Experience Ready to Run Ready Nodes + Auto CONFIG Broadest Deployment Choices VxRail Ready Node Build Your Own Choice Flexibility 164 Ready Nodes 65 Unique Server Platforms 15 Server Partners 163 / 220 NVMe / PCIe devices 122 IO controllers 17 1

18 Hardware Guidance

19 HCI is Pushing the Server IOs like Never Before! CPU DRAM DRAM PCIe Bus VSAN CERT Compute / Memory IO Subsystem 19 1

20 1. NVMe and PCIe Devices on the PCIe bus (Extension of Memory) CPU DRAM DRAM PCIe Bus NVMe PCIe Compute / Memory 10s ns 1µs IO Subsystem 10 s µs 20 2

21 2. Network Adaptors Inter Cluster Communication CPU PCIe Bus 10 s µs DRAM DRAM Compute / Memory 10 s ns 1µs NVMe IO Subsystem PCIe NIC 10GbE 21 2

22 3. IO Controllers to Propagate the IOs to the Drives CPU PCIe Bus 10 s µs DRAM NVMe PCIe DRAM NIC Compute / Memory 100µs 1ms 1 ns 1µs IO Controller SATA / SAS >10s ms 22 IO Subsystem 2

23 Design Guidance 1. All Components must be on the HCL/VCG 2. Turn off caching for HDDs ; turn off local drive cache 23 2

24 HCI is pushing the IO Subsystem like Never Before! VSAN CERT 1 NVMe / PCIe devices 2 IO Controllers VSAN CERT 24 VSAN CERT 3 4 Drives NIC IO CERT 2

25 VSAN Hardware Guidance Focus on 3 Components 1 NVMe / PCIe devices 2 HBA / IO Controllers 3 Drives 25 2

26 Considerations for NVMe and PCIe devices 1 Choosing the right form factor - HHHL, 2.5 and M.2 PCIe Bus NVMe PCIe 2 Better performance with NVMe as VSAN cache layer for higher IOPs 26 2

27 Considerations for NVMe and PCIe devices (and SSDs) 3 Choose the correct Endurance and Performance Rating PCIe Bus NVMe PCIe * Endurance: TB written in 5 years * Performance: Writes per second

28 Considerations for NVMe and PCIe devices PCIe Bus NVMe Search 3 for VSAN Quick Reference Guide PCIe 4 Choose the correct Endurance and Performance Rating Endurance: TB written in 5 years Performance: Writes per second

29 3. IO Controllers to Propagate the IOs to the Drives CPU PCIe Bus DRAM DRAM Compute / Memory NVMe PCIe NIC IO Controller IO Subsystem SATA / SAS 29 2

30 IO Controller Modes of Operation PCIe Interface PCIe Host Interface RAID Acceleration Block Firmware Commands (1)Passthrough (2)RAID Link 0 Link 1 SAS / SATA Interface 30 3

31 VSAN prefers Passthrough Over RAID PCIe Interface PCIe Host Interface RAID Block Firmware Commands 1 VSAN prefers passthrough mode over RAID mode Link 0 Link 1 SAS SATA Interface 2 For best performance use controllers with Queue Depth of 512 or higher 31

32 Why Passthrough is Preferred 1 Most Optimized VSAN Experience 2 Simplicity of Initial set up and Ongoing Operations 3 Consistency of Experience Passthrough mode is also called HBA mode 32

33 Do Not Share non VSAN and VSAN Drives on Same Controller 1 Boot Devices and Log Devices, should not be behind the same controller backing VSAN drives vsphere Virtual SAN 2 VMFS (other no VSAN file systems) have different IO completion expectation 33

34 Make sure you use the latest Driver Firmware Listed? 1. Running the correct combination of driver and firmware is critical for any IO Device 2. Don t use a later version of driver and firmware if it is not on the VSAN VCG no matter what anyone tells you! 34

35 Boot Options SD / USB 4 8 GB SATADOM Min: 16GB SSD / HDD Direct to Syslog server or use network dump Collector Persist or use network dump collector Persist or use network dump collector Endurance 384 TBW for VSAN 6.2 and beyond Separate Controller compared to VSAN 35

36 Drives Connect to the PHYs and Arranged into Disk Groups PCIe Interface PCIe Host Interface PHYs RAID Acceleration Block Firmware Commands Disk Group:2 Link 0 Link 1 Disk Group:1 CONFIDENTIAL 36

37 Max Disk Group is 5 per HOST Host 1 Host 2 Host 3 Host 64 Disk Group 1 Disk Group 2 Disk Group 5 10GbE Cluster: 2-64 hosts SSD or NVMe or PCIe 1 Cache 1 Host: 1-5 disk groups Disk Group 1 SSD / NVMe / PCIe for cache 1-7 SSD or HDD for capacity SSD or HDD 1 SSD or HDD 2 SSD or HDD 7 Capacity 7 37

38 NVMe Caching Tier with SSD Capacity Tier PCIe Interface PCIe Interface PCIe Host Interface RAID Acceleration Block Firmware Commands NVMe / PCIe Link 0 Link 1 Disk Group 38

39 Disk Groups for VSAN 40 Max Drives per Host 3-4 Disk Groups More Drives / Disk Group Balanced Disk Groups 39

40 Read / Write Patterns for All Flash vs. Hybrid SSD or NVMe or PCIe 1 Cache CACHE R/ W Split All Flash 100% Writes Hybrid 70% Reads, 30% Writes NVMe or SSD NVMe or SSD NVMe or SSD HDD HDD HDD Capacity Reads Capacity Tier Both Tiers Writes Cache Tier Both Tiers 40 Much lower endurance Capacity Device

41 Ready Nodes

42 Ready Nodes Abstracts the Complexities PCIe Interface PCIe Host Interface Ready Nodes abstract the complexities and provide a simple consumption model RAID Processing Unit Firmware Processing Unit Chassis FW Drives SAS Expander Boot Options Link 0 Link 1 Expanded Backplane BYO doesn t factor in additional server hardware and Firmware 42

43 Controller + Expander Certifications only available on Ready Nodes Expanded Backplane 43 Chassis FW Drives SAS Expander Boot Options

44 Post Deployment Experience

45 Errors Do Happen Don t Worry! Health Check Burn In Tests Flag incompatible Firmware, Drivers Network, Performance, Capacity 45

46 Errors Do Happen Don t Worry! Future Release Auto Config Health Check Burn In Tests Update Controller s Firmware and Driver 46

47 Ready Node Configurator

48 48 DEMO

49 vsan Certification Under the Hood

50 Principle 1 Identify the different types of I/O: Application IO APPLICATION I/O HOUSEKEEPING I/O 50

51 Principle 1 Identify the different types of I/O : Rebuild / Resync IO APPLICATION I/O RESYNC / REBUILD I/O 51

52 Principle 1 Identify the different types of I/O APPLICATION I/O RESYNC / REBUILD I/O ESX MANAGEMENT I/O HOUSEKEEPING I/O 52

53 Principle 2 Account for Worst Case Scenario APPLICATION I/O RESYCN / REBUILD I/O Worst case load on the system HOUSEKEEPING I/O 53

54 Principle 3 Initiate Controller Task Management Activities during IO fulfillment Virtual SAN IO controller may initiate its own Error Handling Operations IO Controller This is over an above the regular IO Operations 54

55 Certification Suite Tests Performance Steady State Analysis Stress Worst Case Behavior Hot Plug Drive or Disk Group Failure Constantly Evolving and Enhancing our Certification Suite Different combinations of Read / Write profiles* Sequential / Random *100r0w, 70r30w, 0r100w, 50r50w Sustained period of time with Heavy I/O Planned and Unplanned tests Both drive and entire disk group failure Formulating Rigorous Equivalency model to reduce certification efforts 55

56 Q & A

57