Disclaimer This presentation may contain product features that are currently under development. This overview of new technology represents no commitme

Transcription

1 STO1206BU Interpreting performance metrics in your vsan environment Pete Koehler Bradley Mott #VMworld #STO1206BU

2 Disclaimer This presentation may contain product features that are currently under development. This overview of new technology represents no commitment from VMware to deliver these features in any generally available product. Features are subject to change, and must not be included in contracts, purchase orders, or sales agreements of any kind. Technical feasibility and market demand will affect final delivery. Pricing and packaging for any new technologies or features discussed or presented have not been determined. #STO1206BU CONFIDENTIAL 2

3 Agenda 1 Why to Metrics Matter? 2 The vsan Performance Service 3 Metrics to Look at with Performance Service 4 Fact vs Fiction. Workloads, Metrics, and Technology 5 Supplementary Tools 6 Q&A #STO1206BU CONFIDENTIAL 3

4 Why Do Metrics Matter? A look at how looking at the right metrics in the right way is more important than ever before

5 Perception is everything VMworld 2017 Content: Not for Cannot define the problem if interpreted incorrectly What you measure, how you measure, and where you measure all matter publication 5

6 Performance measurement other architectures Storage Networking Three-tier architecture vsphere VMworld 2017 Measures here Virtual appliance based HCI Storage VM ( per server) vsphere Measures here Other architectures suffer when analyzing storage. What is measured How it is measured Where it is measured Leads to inconsistency in results Content: Not for publication Storage 6

7 Performance measurement the benefits of integration HCI powered by vsan VM VM VM VM VM VM Managed by vcenter vsphere vsan VMworld 2017 Measures here Can see all traffic from end to end Kernel level statistics Consistent method of measurement Content: Not for publication Appropriate levels of granularity Cluster Hosts Disk groups and disks VMs VMDKs 7

8 The vsan Performance Service Built-in performance metrics for your vsan environment

9 What it is CLI vsphere Perf Svc Power CLI vsan API Perf Svc SDKs vsan Datastore Stats DB Object vsan vcenter Perf Svc Purpose built collector of vsan performance metrics Native to ESXi 90 day retention of metrics Accessible via GUI (vcenter), CLI, and API Stats DB Object Lives on vsan datastore as an object Uses an SPBM policy Can be copied or renamed for historical retention. 9

10 What it can see vsphere vsan Datastore vsan VMs & VMDKs Cluster Hosts Disks and Disk groups Layered visibility of vsan metrics at: VMs and VMDKs Cluster Hosts Disk groups and disks All common I/O metrics IOPS Throughput Latency Other hypervisor and vsan data VMkernel traffic Congestions 10

11 What types of storage traffic it can distinguish vsphere vsan Datastore vsan Front end VM traffic Back end storage traffic Front end VM storage metrics Metrics as seen by the VM Per VMDK level Back end storage metrics Replica traffic Synchronization traffic 11

12 Detailed activity of resynchronization traffic vsphere vsan Datastore vsan Back end traffic resynchronization Object policy changes Host or disk group evacuations Object or component rebalancing Object or component repairs or NEW vsan 6.6 distribution 12

13 Metrics Provided in the Performance Service Where to find the performance service data in a vsan cluster

14 Providing performance metrics at three levels Cluster (and at all levels) IOPS Throughput Latency Congestions Outstanding I/O Host vsan disk group vsan disk Physical and VMkernel adapters VM Aggregated VM metrics VMDK metrics NEW vsan

15 Cloud-connected Performance Diagnostics Analysis Detect issues Visible to GSS HCIBench vsphere vsan Cloud Analytics Feedback Site results Links to KBs VMware Customer Experience Improvement Program vsan NEW vsan Provides diagnostics for benchmarking & PoCs Specify one of three predefined areas of focus for benchmarks: Max IOPS Max Throughput Min Latency Integration into HCIBench Output automatically sent to cloud for analysis Provides results of analysis in UI Detects issues and suggests remediation steps by tying to specific KB articles

16 Tip: Simplify your starting point VM/App Host level VMworld 2017 Content: Not for Disk / disk group Isolate source more quickly Viewing one layer below contending resource can mask real issue Viewing cluster view later can help identify influencing factors and context publication 4 Cluster 16

17 Tip: Learn how to identify abnormalities VMworld 2017 Content: Not for Establish what is normal for your environment Helps identify statistical outliers Helps identify systemic issues Observe real workloads publication Each workload may exhibit unique behavior Look for repetition of behavior Don t forget native vcenter VM metrics higher sampling rates short lived 17

18 Fact vs. Fiction Workloads, Metrics, and Technology Considerations to be aware of when measuring performance VMworld 2017 Content: Not for publication

19 Application demands vary! Databases (SQL/Oracle) Business Critical Apps VMworld 2017 Monitoring & Log Aggregation Virtual Desktops Applications have unique, dynamic signature of I/O activity Read/Write distributions I/O sizes (aka block sizes) Working sets Duty cycles I/O profile can change in the course of minutes, or seconds Content: Not for publication Synthetic I/O generators do NOT mimic real workloads Appropriate for some needs Limited in capabilities 19

20 Sampling rates - How you measure matters 200 IOPS 270 IOPS VMworld1,800 IOPS 5 minute sampling 1 minute sampling Defines how often data is collected and rendered Can present same data differently All are accurate for how the measurement is defined One may be more helpful than the other 2017 Content: Not for publication 20 second sampling 20

21 Clearing up confusion on blocks and I/Os KB 64KB 128KB 256KB 512KB An I/O is a single unit of storage payload in flight Sometimes referred to as I/O size, I/O Length, or block size NOT referring to on-disk filesystem structures IOPS x block size = Throughput Wide mix of block sizes that are being used at any given time on a single VM Dictated by OS and app processes Non dominate block sizes can dramatically impact performance 21

22 How I/O sizes can be different for reads and writes VMworld 2017 Content: Not for Same amount of data (throughput) for reads and writes. Different number of commands issued (IOPS) publication No static rule. Constantly changing R/W ratios based on IOPS can be misleading! 22

23 Impact on latency as a result of I/O size IOPS Latency VMworld 2017 Content: Not for Lower IOPS on writes as the result of increased latency Increased latency on writes a result of much larger I/O sizes publication Less than 1% of payload for writes transmitted by I/Os 4KB or smaller Less than 4% of payload for writes transmitted by I/Os 256KB or smaller I/O Distribution - Reads I/O Distribution - Writes 23

24 Interesting storage performance impacts VMworld 2017 Content: Not for An example of how higher CPU utilization can be a good thing Principles apply to any type of storage system Constrained storage can mislead the real demands of a workload publication Partial metrics can be misleading (ex: DAVG) No metric lives on an island 24

25 What faster hardware can (and can t) do vsphere vsan Applications Compute Storage Network VMworld 2017 Content: Not for Flash does some things well, and struggles with other demands Higher latency on large I/O writes May demonstrate lack of consistency publication Faster hardware may expose bottlenecks elsewhere in hardware stack and architecture Compute Storage Network Faster hardware may expose bottlenecks in applications 25

26 Supplementary Tools for Monitoring Performance Other ways to monitor performance data

27 Augment Performance Service with Analytics vsphere vsan Datastore vsan VCSA vrops LI Native vsan Integration vrealize Operations Native vsan Integration vrealize Log Insight Analytics Monitoring/Analytics Event history Alerting Insight/Trending Log Mgmt & Analytics Log aggregation Log data mining and intelligence Root cause analysis 27

28 IOInsight and vscsistats IOInsight Fling on labs.vmware.com Easy to use High sampling rate I/O sizes vscsistats Native to ESXi. on-disk placement level stats. Seek distance, I/O Size or distribution Platform independent No built-in sampling rate 28

29 vsan Observer VMworld 2017 Content: Not for Deprecated for consumer use May still see it be used on GSS/support cases Hints at what's next? See session: publication STO1378BU vsan Mgmt today and in the future 29

30 Questions? #STO1206BU CONFIDENTIAL 30

31

32