STO1053BES Redefine vsan Deployments with Next Generation Intel Xeon processor, Intel Optane and Intel 3D NAND SSDs VMworld 2017 Kathryn Vandiver, vsan ReadyLabs Engineering, VMware Vivek Sarathy, Non-Volatile Memory (NVM) Solutions Group, Intel Content: Not for publication #VMworld2017 #STO1053BES
Disclaimer This presentation may contain product features that are currently under development. This overview of new technology represents no commitment from VMware to deliver these features in any generally available product. Features are subject to change, and must not be included in contracts, purchase orders, or sales agreements of any kind. Technical feasibility and market demand will affect final delivery. Pricing and packaging for any new technologies or features discussed or presented have not been determined. 2
Legal Notices and Disclaimers Intel technologies features and benefits depend on system configuration and may require enabled hardware, software or service activation. Learn more at intel.com, or from the OEM or retailer. No computer system can be absolutely secure. Tests document performance of components on a particular test, in specific systems. Differences in hardware, software, or configuration will affect actual performance. Consult other sources of information to evaluate performance as you consider your purchase. For more complete information about performance and benchmark results, visit http://www.intel.com/performance. Cost reduction scenarios described are intended as examples of how a given Intel-based product, in the specified circumstances and configurations, may affect future costs and provide cost savings. Circumstances will vary. Intel does not guarantee any costs or cost reduction. This document contains information on products, services and/or processes in development. All information provided here is subject to change without notice. Contact your Intel representative to obtain the latest forecast, schedule, specifications and roadmaps. No license (express or implied, by estoppel or otherwise) to any intellectual property rights is granted by this document. Intel does not control or audit third-party benchmark data or the web sites referenced in this document. You should visit the referenced web site and confirm whether referenced data are accurate. Intel, the Intel logo, Xeon, Intel Optane, and 3D XPoint are trademarks of Intel Corporation in the U.S. and/or other countries. *Other names and brands may be claimed as the property of others. 2017 Intel Corporation. VMworld 2017 Content: Not for publication 3
Agenda 1 Business Overview 2 Emerging Technology and Trends 3 vsan All Flash Overview and Customer Adoption 4 vsan on Next Gen Platform Intel Optane Technology 5 Manageability Improvements for Next Gen Intel based Platform 6 Summary 4
Market Segmentation for Hardware 1. General Purpose Rack Servers - General purpose workloads BCA, Database, VDI, ROBO, DR, Test/Dev 2. Storage Dense Rack Servers - Storage Heavy workloads Archiving, video streaming, analytics VMworld 2017 3. Compute Intensive Blade Servers - Compute Heavy workloads Web apps, HPC, real time analytics, In-memory DB 4. Composable Infrastructure Scaling storage & compute independently for disaggregated workloads Data warehouses, Search engines databases, Log aggregation Content: Not for publication vsan for various customer workloads 5
vsan Ready Node: Many Platform Choices to Maximize Flexibility Many Platform Choices 15 server vendors 164 platform choices Flexible Support Individual vendor support Single support from partner VMworld 2017 Software Flexibility Bring your own software Factory install software Content: Not for publication Licensing Options Bring your own license Purchase license with platform 6 https://www.vmware.com/resources/compatibility/search.php?devicecategory=vsan
vsan Hardware Enhancements in 2016 & 2017 Overview Enable high density vsan ReadyNode with support for 512e Drives vsan 6.5 Certified Intel Optane SSD as caching tier for vsan LED Management for controllers & NVMe Revised caching guidelines for usage of larger capacity cache drives in Q1, 2017 Hardware Lifecycle Management: Automated controller driver/firmware updates for major OEMs in Q1, 2017 Benefits Larger capacity on high density server configs supported only in 512e/4Kn format Better All Flash TCO* with improved caching guidelines Simplified and automated HW life cycle management * https://vsansizer.vmware.com/vsansizing/html/dcscaledeploymentsoptions.html 7
Impact of Key Technology Trends on vsan Technology Trends CPU densities continues to increase Hi-Density Flash and NVMe will dominate Enterprise Storage High speed interconnect to keep up with fast storage All Flash Market is growing at a rapid rate 38% CAGR over last 5 years VMworld 2017 Content: Not for publication Enterprise customers are adopting slowly moving away from using magnetic drives Newer high speed storage & network interconnects are disrupting the HCI market. Leverage hardware trends to deliver All Flash vsan at the best $/IOP & $/Gig 8
Intel Xeon Scalable Processors optimized for Storage Platform Performance Server Agility Efficiency at Scale DDR4 Intel Xeon Processor VMD UPI X16 PCIe Integrated Networking RDMA Intel Xeon Processor VMD DDR4 ISA-L SPDK VROC Memory Drive Platform to drive storage transformation 9
Intel Technology Innovations Fill the Memory and Storage Gap Intel 3D NAND Technology Lower cost & higher density VMworld 2017 Intel Optane Technology Content: Not for publication Higher performance LOWER HIGHER LESS MORE Cost Delay 10
vsan All Flash Today vsan SSD NVMe NVDIMM Writes cached first, Reads from capacity tier Capacity Tier Flash Devices Reads primarily from capacity tier Caching Data Persistence 2 Tier Architecture: Tier 1 Caching: High performance, high endurance flash for caching writes Tier 2 Data Persistence: Read intensive, low endurance drives for capacity Customer Adoption: Gaining significant momentum, aligned with enterprise adoption of flash, particularly NVMe 11
NVMe Use Cases for vsan All Flash NVMe/PCIe High Performance Caching & Capacity Device NVMe Configurations for vsan Advantages: NVMe as Caching Tier for vsan (2-4 NVMe as caching tier per server) NVMe as Capacity Tier for vsan (7 NVMe capacity tier per cache device) ALL NVMe (caching & capacity) platforms (with PCIe switches, up to 24 drives per server) General trend of customers moving toward NVMe better TCO*, more power efficiency, compact form factors and better error handling Good for Latency sensitive workloads such as OLTP/SQL, VDI Full Clones Good for write intensive workloads requiring high IOPS analytics, database, media streaming as an example * https://estimator.intel.com/sddc 12
Customer Adoption of vsan All Flash Based on Telemetry Data &Customer Surveys Today: 65% 84% of customers are expected to adopt Purley within the next year Higher adoption of OEM s who are offering all NVMe ReadyNodes Increase in VCG Listings and ReadyNodes with All Flash over Hybrid 13
COOP Supermarket chain across Denmark and Europe 1200+ stores Challenges Competitive retail industry meant constant changes, IT need to respond fast Traditional storage, networking, maintenance very costly with limited staff Use Case vsan All-Flash supporting over 1200+ stores Tier-1 Apps including store inventory management, customer portals, credit card transaction processing, customer loyalty systems BCA like Microsoft SQL Server, Exchange, SharePoint 1.2 PB, 4000 VMs Benefits Standardization: Single, integrated VMware stack with full flexibility to grow Best TCO Compared to Any Other Vendor: best price/performance ratio compared to storage vendors and Azure Cloud Performance Exceeding Expectations: Latency-sensitive business-critical applications far exceeded performance goals 14
Largest Airline in Europe Tier-1 Applications on vsan Plans to move entire Tier-1 Apps to vsan Apps are low compute/high capacity For airline apps, more important than IOPS is reliably low latency, which we could achieve with vsan All-Flash - Philipp Schwarzhuber, Director of SDDC Challenges Standardization: SDDC initiative to standardize on Dell Meet demanding performance needs of billing and reservation systems Need cost savings and ease of scale with increasing application usage Use Case Tier-1 OLTP applications like mission-critical billing and reservation systems on SQL 800TB 1PB with 48 nodes across 4 clusters All-Flash for latency sensitive workloads Hardware Platform Dell R730XD servers Benefits Convergence of teams: team is less siloed, more productive and builds new skillsets OPEX improvement: Day-to-day operations are lean and efficient Easy to scale: grow with unpredictable airline business 15
Large Financial Institute Leading Global Financial Services Firm Leading global financial services firm VMworld 2017 Providing Investment banking, securities, wealth management, and services Challenges Standardization: SDDC initiative to standardize on All Flash Financial Apps (business critical) & Desktop Virtualization Tier 1 workloads requiring high performance / low latency for DB2 and VDI Use Case SDDC Initiative to go 100% HCI with All-Flash 48 Node Cluster managing 500TB capacity All-Flash for latency sensitive Financial Apps running DB2 Content: Not for publication Benefits VSAN All Flash delivers highest performance on NVMe at the lowest price Ease of management Scalable All Flash VDI solution for Full Clones 16
Intel Optane and Intel 3D NAND SSDs for vsan
Intel Optane Breakthrough Performance VMworld 2017 Content: Not for 5-8x faster at low Queue Depths 1 publication Vast majority of applications generate low QD storage workloads 1. Common Configuration - Intel 2U Server System, OS CentOS 7.2, kernel 3.10.0-327.el7.x86_64, CPU 2 x Intel Xeon E5-2699 v4 @ 2.20GHz (22 cores), RAM 396GB DDR @ 2133MHz. Configuration Intel Optane SSD DC P4800X 375GB and Intel SSD DC P3700 1600GB. Performance measured under 4K 70-30 workload at QD1-16 using fio-2.15. Tests document performance of components on a particular test, in specific systems. Differences in hardware, software, or configuration will affect actual performance. 18
Intel Optane Storage Performance Characterization Latency vs. Load: NAND SSD vs. Intel Optane SSD (Intel SSD DC P3700 vs. Intel Optane SSD DC P4800X) Lower is better QD ~12 10x latency reduction < 10usec latency 100x QoS improvement < 200usec 99.999th r/w Higher is better vs. NAND based SSD 19
Intel 3D NAND SSDs. Optimized for Capacity and Cost Array Architectural efficiency (CMOS under array) Smaller FG cell VMworld 2017 Content: Not for Advantages Scale to bigger capacities up to 20% publication higher areal density 1 Manufacturing efficiency More GB per wafer 20
Intel 3D NAND SSDs. Optimized for Power-Efficient Capacity Compute and Networking 48U Open Rack with 12 kw Budget 50% of rack space * 50% of rack space * U.2 15mm 2 TB at 25 W 720 TB, 9 kw 25% of power budget * 75% of power budget * OR U.2 7mm 4 TB Intel SSD DC P4501 Series at 10W, 1.44 PB, 3.6 kw 25% of power budget * 45% of power budget SAVED! * 30% of power budget * Sources Intel and Facebook. Facebook Lightening JBOF: https://code.facebook.com/posts/989638804458007/introducing-lightning-a-flexible-nvme-jbof/. Intel Specifications for Intel SSD DC P3520 and Intel SSD DC P4501. Each rack populates 24U with 15 4TB drives in each rack unit. Storage on the P3520 48U rack consumes 9kW (24U x 15 SSDs/U x 25W/drive max power of U.2 form factor for power budget) while the P4501 rack consumes 3.6kW (24U x 15 SSDs/U x 10W/drive). 21
Caching and Persistent Tiers with All Flash vsphere NVMe SSD vsan NVMe SSD vsan Datastore Cache Capacity All Writes are served by the Cache Tier Absorbs the bulk of the write rate from the application Data is de-staged or drained from Write Cache to Capacity Tier only when needed, extending the life of the capacity tier Deduplication / compression occurs in the de-staging process Reads are served directly from the Capacity Tier Read performance from capacity flash is more than sufficient. Intel Optane SSD provides benefit for low latency and high performing transactional workloads 22
Evaluator Group Study with Intel Optane SSD and Intel Xeon Scalable Processor Tests utilized Application Workloads Real-world application profiles Two Different Application Workload Profiles IOmark-VM = General Database and Server VM HammerDB TPC-C = High Performance Database 23
Up to 10x Performance w/ Latest HCI Technology on vsan IOmark-VM Performance: 4-Node HC 1200 1000 800 600 400 200 0 80 288 480 IOmark-VM s 800 800 1120 SSD + HDD NVMe + SSD All-NVMe Optane + S3520 Optane + P4500 Optane + P4500 ( No Dedupe) Hybrid HCI 1 st Gen NVME Optane + NVMe Hybrid HCI 1 st Gen NVMe Optane + NVMe https://www.evaluatorgroup.com/document/intel-next-generation-technology-powering-new-performance-levels/ 24
Up to 9x Better Price/Performance with Latest HCI Technology on vsan Price / Performance: 4-Node HC $2,500 $2,000 $1,500 $1,000 $500 $0 $2,048 $675 $545 $ / IOmark-VM-HC $293 $302 $237 SSD + HDD NVMe + SSD All-NVMe Optane + S3520 Optane + P4500 Optane + P4500 ( No Dedupe) Hybrid HCI 1 st Gen NVMe Optane NVMe Hybrid HCI 1 st Gen NVMe Optane NVMe https://www.evaluatorgroup.com/document/intel-next-generation-technology-powering-new-performance-levels/ 25
vsan Database Results (Running a TPC-C like workload) 10,000,000 9,000,000 8,000,000 7,000,000 6,000,000 5,000,000 4,000,000 3,000,000 2,000,000 1,000,000 - Total TPM for 4-Node vsan 9,116,024 4,828,595 P4600+P4500 Total TPM Nearly 2X better performance and 2X lower price / performance 89% higher TPM Performance 44% lower cost per TPM Optane+P4500 $0.05 $0.05 $0.04 $0.04 $0.03 $0.03 $0.02 $0.02 $ / TPM 4-Node vsan $0.05 $0.01 $0.01 $- $ / TPM HC $0.03 P4600+P4500 Optane+P4500 https://www.evaluatorgroup.com/document/intel-next-generation-technology-powering-new-performance-levels/ 26
HCIBench Tests: Up to 3x lower latency with Latest HCI Technology : Server: Supermicro* 2028U-TN24R4T with 2 x Intel Xeon E5 Cache Tier: 2x Intel SSD DC P3700 @ 400 GB Capacity Tier: 8x Intel SSD DC P3520 @ 2TB A Transition to New Intel Technology based vsan Server: Supermicro 2029U-TN24R4T with 2 x Intel Xeon Scalable Processor Cache Tier: 2x Intel SSD DC P4800x @ 375 GB Capacity Tier: 8x Intel SSD DC P4500 @ 4TB Increase Storage Efficiency B Lower Latency up to 2x up to 3x System Configuration: 4 Node vsan* Cluster. Per Node configuration: A:Supermicro* SuperServer 2028U-TN24R4T+ Dual Intel Xeon E5-2687Wv4 (12 Core @ 3.0 Ghz), B. Supermicro* SuperServer 2029U-TN24R4T+ Dual Intel Xeon scalebale processor, Supermicro* Server Board, 256 GB DDR4 RAM, Boot Drive, 1x Intel SSD DC S3710 Series (200 GB, 2.5 ), vsan Cluster: Virtual SAN SSDs - 2 Disk Groups comprised of 2x Intel SSD DC P4800x Series (375GB, 2.5 SFF), 8x Intel SSD DC P4500 Series (4 TB, 2.5 SFF), vsan Cluster: Virtual SAN SSDs - 2 Disk Groups comprised of 2x Intel SSD DC P3700 Series (400GB, 2.5 SFF), 8x Intel SSD DC P3520 Series (2 TB, 2.5 SFF), *Other names and brands names may be claimed as the property of others 27
Manageability Improvements for Next Gen Intel based platform VMworld 2017 Content: Not for publication
Storage at Scale Requires Efficiency Storage inspired Intel Xeon Platforms enable efficiency to manage data at scale Simple, Reliable and Manageable IT Efficiency starts with reliable and serviceable solutions 29
Efficiency Challenge: Lack of Reliability & Serviceability Did your vsan node PSOD due to surprise PCIe* SSD removal? 30
Efficiency Challenge: Lack of Reliability & Serviceability Which SSD Needs Replacement? Whoops! These tiny LEDs are critical important. or distribution 31
Intel Volume Management Device Before Intel VMD Operating System Intel Xeon E5v4 Processor PCIe Root Complex PCIe SSDs After Intel VMD Operating System Intel Xeon Scalable Processor PCIe Root Complex Intel VMD PCIe SSDs Intel VMD is a new technology to enhance solutions with PCIe* storage Benefit for VMware* vsan Intel VMD enables: Isolating fault domains for device surprise hot-plug and error handling Provide consistent framework for managing LEDs Simplify PCIe storage software stacks Intel VMD enables customers to simplify and harden solutions using PCIe storage. 32
vsan NVMe SSDs Hot Plug and LED Management Without Intel VMD Supported in vsan: Hot Insert: Supported if the server platform supports PCIe Hot Plug Prepared Hot Removal: Needs user management operations to hot remove NVMe SSDs Not Supported in vsan: VMworld 2017 Surprise Hot Plug Removal Surprise Hot Insert LED Management for NVMe SSDs With Intel VMD Benefit to vsan: Surprise Hot Plug Removal Content: Not for publication Surprise Hot Insert LED Management for NVMe SSDs 33
Intel NVMe SSDs Firmware Auto Update vsphere Intel SSD Data Center Tool vsan NVMe NVMe.... NVMe vsan Datastore 34
35
What s next with Intel NVMe SSDs and vsan?
Ruler form factor for Intel SSDs. Designed from the ground up to optimize rack efficiency, the new Ruler Form Factor delivers unparalleled Space-Efficient Capacity, Operationally- Efficient Design and Scalable Manageability. Ruler form factor for Intel SSD DC P4500 available by end of 2017. 37
Enabling up to 1PB of storage in a 1U server rack VMworld 2017 Content: Not for Storage Capacity publication 1PB in 1U Intel 3D NAND SSD, 32TB ruler in 2018 1PB IN 42u W/2 TB HDDs Opening up new use cases in datacenter storage with disruptive total cost of ownership 38
Ruler form factor for Intel SSDs with vsan - (Future) Next Gen NVMe/PCIe NVMe Over Fabric Scale compute and storage independently (disaggregated) Expand storage (dynamically) to serve compute blades Workloads for disaggregated storage Compute Blades Data warehouses, Search databases, Log aggregation, analytics NVMeOF Ruler form factor for Intel SSDs Ethernet Fabric PCIe Switch 39
Summary: Intel Optane SSD and Intel SSD DC P4500 VMworld 2017 Jointly committed to delivering vsan HCI solutions built on cutting-edge Intel Flash Technology Intel Optane and Intel Xeon Scalable Processors Delivers All Flash vsan at the best $/IOP, $/Gig and low latency Content: Not for publication Breakthrough performance for HCI Intel Xeon Scalable Platforms Higher responsiveness to meet today s modern workload needs https://www.evaluatorgroup.com/document/intel-next-generation-technology-powering-new-performance-levels/ 40