vrealize Operations Management Pack for Storage Devices Guide

Transcription

1 vrealize Operations Management Pack for Storage Devices Guide This document supports the version of each product listed and supports all subsequent versions until the document is replaced by a new edition. To check for more recent editions of this document, see EN

2 vrealize Operations Management Pack for Storage Devices Guide You can find the most up-to-date technical documentation on the VMware Web site at: The VMware Web site also provides the latest product updates. If you have comments about this documentation, submit your feedback to: Copyright 2015 VMware, Inc. All rights reserved. Copyright and trademark information. VMware, Inc Hillview Ave. Palo Alto, CA VMware, Inc.

3 Contents About This Book 5 1 Introduction to the vrealize Operations Management Pack for Storage Devices 7 2 Installing and Configuring the Management Pack 9 Install the Management Pack 9 Add Credentials and Fabric Servers for the Management Pack for Storage Devices 10 Verify That Your Management Pack for Storage Devices Is Connected and Collecting Data 11 3 Management Pack for Storage Devices Dashboards 15 4 Management Pack for Storage Devices Alert Definitions 17 5 Management Pack for Storage Devices Metrics 21 Statistical Metrics of the Management Pack for Storage Devices 21 Object Properties 28 6 User Scenario: An Object in Your Storage Network Has a Performance Problem 33 User Scenario: Use the Troubleshooting Dashboard to Locate the Real Problem Source 33 User Scenario: Examine Metrics to Find the Root Cause 34 Index 35 VMware, Inc. 3

4 vrealize Operations Management Pack for Storage Devices Guide 4 VMware, Inc.

5 About This Book The vrealize Operations Management Pack for Storage Devices Guide provides you with a complete view of your entire storage topology from your host, through your storage network, and out to the storage array. With this solution, you can use vrealize Operations Manager to monitor and troubleshoot capacity and performance problems on different components of your storage area network. Intended Audience The information in this guide is intended for storage administrators, data center architects, and IT operators. VMware Technical Publications Glossary VMware Technical Publications provides a glossary of terms that might be unfamiliar to you. For definitions of terms as they are used in VMware technical documentation, go to VMware, Inc. 5

6 vrealize Operations Management Pack for Storage Devices Guide 6 VMware, Inc.

7 Introduction to the vrealize Operations Management Pack for Storage Devices 1 The Management Pack for Storage Devices provides you with a complete view of your entire storage topology from your host, through your storage network, and out to the storage array. With this solution, you can use vrealize Operations Manager to monitor and troubleshoot capacity and performance problems on different components of your storage area network. The Management Pack for Storage Devices has the following features: Provides an end-to-end view of topology, statistics, and events at every affected level of the storage area network. Enables a vsphere administrator working with a virtual environment to isolate problems caused by elements in the physical storage stack, such as the Host Bus Adapter (HBA), storage switches, and array. The vsphere administrator uses this information to hand off the problem to the domain administrator for further analysis. Captures and analyzes information for throughput and latency on the HBA and mount objects. Captures throughput on the switch ports. Captures IOPS and queue depth at the HBA and switch ports for read and write components. Discovers the storage switches and applies the credentials required to collect data from the objects in the storage network. It uses the Common Information Model (CIM) to exchange information with objects managed by the following management systems: Cisco Data Center Network Manager (DCNM) Brocade Network Advisor (BNA) VMware, Inc. 7

8 vrealize Operations Management Pack for Storage Devices Guide 8 VMware, Inc.

9 Installing and Configuring the 2 Management Pack When you install a management pack, you install the PAK file that you downloaded as part of the solution, and you configure the management pack instance. This chapter includes the following topics: Install the Management Pack, on page 9 Add Credentials and Fabric Servers for the Management Pack for Storage Devices, on page 10 Verify That Your Management Pack for Storage Devices Is Connected and Collecting Data, on page 11 Install the Management Pack If your environment includes multiple vcenter Server systems on the same storage network, you must add fabric switch CIM servers for each adapter instance. The adapter instance provides you with a holistic view of the storage network connected to the vcenter Server system. The adapter discovers the storage devices only for the ESXi 5.1 update 2 and later. NOTE Any adapter credentials you add are shared with other adapter administrators and vrealize Operations Manager collector hosts. Other administrators might use these credentials to configure a new adapter instance or to move an adapter instance to a new host. Prerequisites Verify that the following conditions are met: You can provide credentials with sufficient privileges to connect and collect data from the vcenter Server host. If the user account has limited access to objects in the vcenter, you can only collect data from objects for which you have permission. Credentials must include at least read permissions. The CIM agent is enabled on all storage network switches from which you are collecting data, and that you can provide credentials with sufficient privileges to connect and collect data. If the user account has limited access to objects in the storage environment, you can only collect data from objects for which you have permission. Credentials must include at least read permissions. All storage network switches with the CIM agent enabled are added to DCNM or BNA solutions. The fabric CIM service supports switch profile and fabric profile. The supported versions for the Fabric Profile are 1.1 to 1.6. VMware, Inc. 9

10 vrealize Operations Management Pack for Storage Devices Guide A vsphere API for Storage Awareness (VASA) provider is registered as a storage provider with the vcenter Server host for your adapter instance. VASA providers gather information from arrays that support the environment and provide the data to vcenter Server system. If the VASA provider is not registered with vcenter Server host, the Management Pack for Storage Devices cannot discover objects in the array. The minimum support version is VASA 1.0. See information regarding storage providers in the ESXi and vcenter Server 5.5 Documentation. Download and install the Management Pack for Storage Devices. Procedure 1 Log in to the vrealize Operations Manager user interface with administrator privileges. 2 In the left pane of vrealize Operations Manager, click the Administration icon and click Solutions. 3 On the Solutions tab, click the plus sign. 4 On the Solutions tab, select Management Pack for Storage Devices and click Configure. 5 Browse to locate the temporary folder and select the PAK file. For example, managementpack_name-buildnumber.pak. 6 Click Upload. The upload might take several minutes. 7 Read and accept the EULA, and click Next. Installation details appear in the window during the process. 8 Ensure that you install the PAK file on the master node. 9 When the installation is complete, click Finish. What to do next Add credentials and add the fabric servers Add Credentials and Fabric Servers for the Management Pack for Storage Devices When configuring an adapter instance for the Management Pack for Storage Devices, you add credentials for a vcenter Server. Procedure 1 In the vcenter Server text box, enter the FQDN or IP address of the vcenter Server instance to which you are connecting. The vcenter Server FQDN or IP address must be reachable from all nodes in the vrealize Operations Manager cluster. 2 To add credentials on the Manage Solution page, click the plus sign. a b c In the Credential name text box, enter the name by which you are identifying the configured credentials. Type the User name and Password for the vcenter Server instance. Click OK. You configured credentials to connect to a vcenter Server instance. 3 Click Test Connection to validate the connection with your vcenter Server instance. 4 Accept the vcenter Server security certificate. 10 VMware, Inc.

11 Chapter 2 Installing and Configuring the Management Pack 5 Enter the SNMP Community Strings for your SNMP v2 enabled IP Network Switches. 6 To add fabric servers, click Add Fabric Server. a b c Click the plus sign. Enter a name and description for the fabric server. Enter the fabric switch CIM server IP or URL of the host on which the CIM server is installed. The Management Pack for Storage Devices supports CIM servers that manage Fibre Channel switches only. It does not support CIM servers that manage arrays. Option Brocade fabric Cisco fabric Action Enter the IP address Enter the service URL in the following format: Namespace>, for example, d e f To add credentials, click the plus sign. In the Credential name text box, enter the name by which you are identifying the configured credentials. Enter the user name and password for the Brocade or Cisco Fabric Switch CIM server. For the Cisco Fabric Switch CIM server, the user name and password are for the Cisco Data Center Network Manager (DCNM) and not the Cisco switch. g h Accept all the security certificates from the Fabric Switch CIM Server. Click OK. 7 Click Save Settings. The adapter is added to the Adapter Instance list and is active. What to do next To verify that the adapter is configured and collecting data from the objects in the storage topology, wait a few collection cycles, then view application-related data. Environment Overview Inventory. Verify that all the objects related to the Storage Devices Instance are listed. Objects should be in the collecting state and receiving data. Dashboards. Verify that the Storage Components Heatmap, the Storage Components Usage, and the Storage Troubleshooting are added to the default dashboards. Verify That Your Management Pack for Storage Devices Is Connected and Collecting Data You configured an adapter instance of the Management Pack for Storage Devices with credentials for a vcenter Server. Now you want to verify that your adapter instance can retrieve information from objects in the host, switch, and array domains of your environment. To view the object types, from the left pane of the vrealize Operations Manager user interface, click Administration > Environment Overview > Adapter Instances > Storage device Instances > <User_Created_Instance>. VMware, Inc. 11

12 vrealize Operations Management Pack for Storage Devices Guide Table 2 1. Object Types that the Management Pack for Storage Devices Discovers Object Type Active Zone Set ESX pnic Fabric Fabric Switch CIM Server Host Adapter IP Switch IP Switchport Logical Switch Magnetic Disks Mount NFS Client NFS Servers No Existing Object Container SAN (Storage Area Network) Solid State Device Storage Array Storage Devices Instance Storage LUN Storage Processor Switch Switch Port Target Port VirtualSAN Cluster VirtualSAN Datastore Zone Description A collection of all active zones in the storage network. The physical Network Interface Cards (NICs) on the ESXi host. In the Brocade storage network, fabric is a collection of interconnected switches. There is typically a one-to-one relationship between the fabric and the SAN. In the Cisco storage network, fabric is a virtual SAN which is collection of one or more logical switches. There is typically a one-to-many relationship between the SAN and the fabric. CIM server that manages the switches. It provides the username, password and URL information that the Management Pack for Storage Devices uses to discover storage network objects such as fabric, switches, and switch ports. The Fibre Channel (FC) or Fibre channel over Ethernet (FcoE) host adapters in the host system. The Simple Network Management Protocol (SNMP) v2 enabled (IP) network switches The switch ports on your SNMP v2 enabled IP network switches A logical switch abstraction for the Cisco storage network. It is a collection of one or more switch ports that are accessible to one other. Magnetic disks on the ESXi hosts. A storage LUN abstraction that is available on the host system. Network File System (NFS) clients on the ESXi hosts. NFS servers registered with vcenter via VASA. A container that includes all the objects that are no longer discovered in the storage devices topology. In the Brocade storage network, the SAN is a collection of fabrics. There is typically a one-toone relationship between the SAN and the fabrics. In the Cisco storage network, the SAN is a collection of one or more Virtual Storage Area Networks (VSAN). Solid State Devices (SSDs) on the ESXi hosts. The storage array registered to the VASA provider in the vcenter for the adapter instance. The Management Pack for Storage Devices instance. The storage LUN of a storage array. The storage controller on the storage array. The switch on the storage network that connects the host to the storage array. The port on a switch. The target port of a storage array. VirtualSAN clusters in your data center. VirtualSAN data stores in your data center. A collection of target ports or host adapters in the storage network. Procedure 1 In the left pane of vrealize Operations Manager, click the Administration tab and click Environment Overview. 2 In the list of tags, expand Adapter Instances and expand Storage Devices Instance. 3 Select the adapter instance name to display the list of objects discovered by your adapter instance. 12 VMware, Inc.

13 Chapter 2 Installing and Configuring the Management Pack 4 Slide the display bar to the right to view the object status. Object Status Collection State Collection Status Description If green, the object is connected. If green, the adapter is retrieving data from the object. 5 Deselect the adapter instance name and expand the Object Types tag. Each Object Type name appears with the number of objects of that type in your environment. 6 Verify that you have at least one object in each of the network areas that the Management Pack for Storage Devices discovers. What to do next If objects are missing or not transmitting data, you might need to investigate further. If an object is not connected or not transmitting data, search for the object and check for related alerts. If the adapter does not discover at least one object on the host system, in the storage network, and in the storage array, verify that your environment is configured properly. Common configuration problems such as invalid credentials, Fabric Switch CIM servers not reachable, or VASA providers not registered in the vcenter Server instance are raised as alerts for the Management Pack for Storage Devices instance. VMware, Inc. 13

14 vrealize Operations Management Pack for Storage Devices Guide 14 VMware, Inc.

15 Management Pack for Storage 3 Devices Dashboards When the Management Pack for Storage Devices is installed, it adds preconfigured dashboards and alert definitions that you use to monitor and troubleshoot the components in your storage area network. Table 3 1. Management Pack for Storage Devices Dashboards Dashboard for protocols such as FC-FCoE, iscsi, and NFS Troubleshooting Component Usage Heatmap Purpose View end-to-end storage topology, health, and alerts on storage components. Provides an overview of the most frequently used objects on the storage network and tracks the error count on these objects. Provides an overview of object performance against its predefined limits. VMware, Inc. 15

16 vrealize Operations Management Pack for Storage Devices Guide 16 VMware, Inc.

17 Management Pack for Storage 4 Devices Alert Definitions When the Management Pack for Storage Devices is installed, it adds preconfigured dashboards and alert definitions that you use to monitor and troubleshoot the components in your storage area network. vrealize Operations Manager generates an alert If a problem occurs with the components in the storage area network that the Management Pack for Storage Devices is monitoring. Table 4 1. Management Pack for Storage Devices Alert Definitions Alert (ID name) Alert Level Affected Object Type Description Adapter Configuration failed (AdapterNotConfigured) Critical Storage Devices Adapter Instance Triggered when the Management Pack for Storage Devices instance is incorrectly configured because of invalid credentials or inaccessible Fabric CIM or vcenter Server instances. Clears when the adapter is reconfigured and can successfully communicate with data sources, such as Fabric CIM or vcenter Server. All Paths Down on some Mounts (APD) Critical VMWARE: Host System Triggered by an All Paths Down (APD) situation on one or more mounts on the host system. This generic alert applies when the root cause for APD cannot be attributed to a zone configuration change, host adapter link failure, or switch port failure. Clears when the APD situation is resolved on the host system. All Paths Down on some Mounts because HBA is Link down (APDHostAdapterLinkDown) Critical VMWARE: Host System Triggered by an APD situation on one or more mounts on the host system because of a problem with the host adapter. Clears when the APD situation is resolved on the host system. All Paths Down On Some Mounts because the physical NIC status is down (APDPnicDown) Critical VMWARE: Host System Triggered by an APD situation on one or more mounts on the host system, because of the problem with the physical Network Interface Card (NIC) connection to the host adapter. Clears when the APD situation is resolved on the host system. All Paths Down On Some Mounts Because Switch Ports connected to the HBAs is Down (APDInitiatorSwitchPortDown) Critical VMWARE: Host System Triggered by an APD situation on one or more mounts on the host system, because of a problem with the switch port connection to the host adapter. Clears when the APD situation is resolved on the host system. VMware, Inc. 17

18 vrealize Operations Management Pack for Storage Devices Guide Table 4 1. Management Pack for Storage Devices Alert Definitions (Continued) Alert (ID name) Alert Level Affected Object Type Description All Paths Down On Some Mounts Because Switch Port connected to the Target is Down (APDTargetSwitchPortDown) Critical VMWARE: Host System Triggered by an APD situation on one or more mounts on the host system, because of a problem with the switch port connection to the target port on the array side. Clears when the APD situation is resolved on the host system. All Paths Down on some mounts because HBAs are zoned out (APDZoneConfiguration) Critical VMWARE: Host System Triggered by an APD situation on one or more mounts on the host system. This alert indicates a loss of connectivity between the host adapters on the host side and target ports on the array side because of zone configuration changes. Clears when the APD situation is resolved on the host system. Congestion due to high network bandwidth usage (NetworkBandwidthCongestion) Critical EsxPnic Triggered when the physical NIC bandwidth is over-utilized. Another symptom when the physical NIC capacity is less than 10 GB is also checked, if applicable. Clears when the physical NIC bandwidth usage comes back to normal. Datacenter has some HBAs that are not operating at maximum supported speed (HostAdapterSpeedNotOptimal) Info VMWARE: Datacenter Triggered if a connected HBA is not operating at its maximum supported speed. Clears when the HBA returns to its maximum operating speed. Fabric CIM server certificate is not trusted (FabricCIMServerUntrusted) Critical Fabric CIM Server Triggered if the Management Pack for Storage Devices cannot authenticate the fabric CIM server certificate. Clears when the adapter can authenticate the fabric CIM server certificate. Fabric CIM Server Configuration failed (FabricCredentialsWrong) Critical Fabric Switch CIM Server Triggered if the Management Pack for Storage Devices cannot communicate with the fabric switch CIM server using the login credentials or URL information provided. Clears when the adapter can communicate with the fabric switch CIM server. HBA Link Failure (HostAdapterLinkDown) HBA Speed dropped (HostAdapterSpeedDropped) High Virtual LUN Resets Or Aborts (HighLunAbortsOrLunResets) Critical Host Adapter Triggered when the HBA status changes from Online to LINK DOWN. Clears when the HBA status returns to Online. Critical Host Adapter Triggered when the operating speed of the host adapter drops compared to a previous reading in the data collection cycle. Not automatically cleared. A user must manually clear the alert. Critical Mount Triggered if 50 or more virtual LUN resets or aborts occur per minute on the mount object during two consecutive collection cycles. Clears when the LUN resets or aborts drop to fewer than 50 per minute for two consecutive collection cycles. 18 VMware, Inc.

19 Chapter 4 Management Pack for Storage Devices Alert Definitions Table 4 1. Management Pack for Storage Devices Alert Definitions (Continued) Alert (ID name) Alert Level Affected Object Type Description IO Latency is above normal (MountIoLatencyHigh) Multiple ports on the Switch are down (SwitchPortsDown) Critical Mount Triggered if the HostSpecificMetrics IOLatency metric exceeds the dynamic threshold. vrealize Operations Manager sets the dynamic threshold based on the historic data values of this metric. Clears when the metric values fall below the dynamic threshold. Critical Switch Triggered when 50 percent or more switch ports become disabled or display the Enabled but offline status. Clears when the percentage of unavailable switch ports falls below 50 percent. No VASA providers registered to vcenter. Cannot discover array objects (NoVasaProvidersRegistered) Warnin g Storage Devices Adapter Instance Triggered when no VASA providers are registered to the vcenter Server. Clears when at least one VASA provider is registered to the vcenter Server. Number of active paths is low (PathHealthRatioLow) Automa tic Mount Triggered when the HostSpecificMetrics PathHealthRatio metric for the mount object falls below thresholds. vrealize Operations Manager sets the alert level based on the value of pathhealthratio: 75<=pathHealthRatio<100 indicates Immediate 50<=pathHealthRatio<75 indicates Warning pathheathratio< 50 indicates Critical Clears when pathhealthratio value becomes 100 for all the host system instanced metric groups of the mount object. Permanent Device Loss on a Mount/Storage LUN (PDL) Critical VMWARE: Host System Triggered when one or more mounts on the host system is unavailable. This situation occurs if the underlying storage LUN is unmasked or removed from the host system. If the storage LUN was unmasked without first detaching it from the host system, the alert clears when the mount object becomes available. If the storage LUN was permanently removed, the alert does not automatically clear. A user must manually clear the alert. Some NFS volumes are not reachable from the host because the physical NIC may be down (APDOnNFSClientBecauseOfPnic) Critical VMWARE: Host System Triggered by an APD situation on one or more Network File System (NFS) volumes on the host system, because the connected physical NIC status is Down. Clears when the APD situation is resolved on the host system. Some NFS volumes are not reachable from the host because the switch port connecting to the physical NIC on the host may be down (APDOnNFSClientBecauseOfSwitch Port) Critical VMWARE: Host System Triggered by an APD situation on one or more NFS volumes on the host system because of the switch port connected to the physical NIC is Disabled. Clears when the APD situation is resolved on the host system. Some NFS volumes are unreachable from the host (APDOnNFSClientGeneric) Critical VMWARE: Host System Triggered by a APD situation on one or more NFS volumes on the host system. Clears when the APD situation is resolved on the host system. VMware, Inc. 19

20 vrealize Operations Management Pack for Storage Devices Guide Table 4 1. Management Pack for Storage Devices Alert Definitions (Continued) Alert (ID name) Alert Level Affected Object Type Description Storage LUN Thin Provisioned Capacity At Critical Levels (ThinProvisionedThresholdExceede d) Critical Storage LUN Triggered when the thin provisioning capacity on the storage LUN reaches a critical level. Thresholds for thin provisioning are set in the vcenter Server. Clears when the vcenter Server sends the clear event to the Management Pack for Storage Devices. Storage LUN Thin Provisioned Capacity Threshold At Warning Level (ThinProvisionedThresholdInitialize d) Warnin g Storage LUN Triggered when the thin provisioning capacity on the storage LUN reaches a warning level. The thresholds for thin provisioning are set in the vcenter. Clears when the vcenter sends the clear event to the Management Pack for Storage Devices. Switch Port Down Failure (SwitchPortLinkDown) Critical Switch Port Triggered when the status of switch port changes to Disabled or Enabled but Offline. Clears when the switch port status returns to Enabled. 20 VMware, Inc.

21 Management Pack for Storage 5 Devices Metrics The Management Pack for Storage Devices collects metrics for objects. It also displays object properties in the vrealize Operations Manager user interface. This chapter includes the following topics: Statistical Metrics of the Management Pack for Storage Devices, on page 21 Object Properties, on page 28 Statistical Metrics of the Management Pack for Storage Devices The Management Pack for Storage Devices collects statistical metrics of objects. You can view these metrics from the vrealize Operations Manager user interface. From the vrealize Operations Manager user interface, click Environment > Storage Devices > Troubleshooting > All Metrics tab. Disk Group Metrics The Management Pack for Storage Devices collects metrics about disk groups. Table 5 1. Disk Group Metrics Metric Name DerivedStatistics.DiskGroupCapacity DerivedStatistics.DiskGroupDemand DerivedStatistics.DiskGroupUsage VsanStatistics.Disk Capacity (GB) User Interface Name Disk Group Capacity Disk Group Demand Disk Group Usage Disk Capacity (GB) VsanStatistics.Disk Capacity Used (%) Disk Capacity Used (%) VsanStatistics.Capacity Used (GB) Capacity Used (GB) ESXi Physical NIC Metrics The Management Pack for Storage Devices collects metrics about the physical Network Interface Cards (NICs) on the ESXi host. Table 5 2. ESXi Physical NIC Metrics Metric Name esx_pnic_statistics.packetsreceived esx_pnic_statistics.packetstransmitted User Interface Name Packets Received Packets Transmitted VMware, Inc. 21

22 vrealize Operations Management Pack for Storage Devices Guide Table 5 2. ESXi Physical NIC Metrics (Continued) Metric Name esx_pnic_statistics.received esx_pnic_statistics.transmitted esx_pnic_statistics.droppedreceived esx_pnic_statistics.droppedtransmitted esx_pnic_statistics.bytesreceived esx_pnic_statistics.bytestransmitted esx_pnic_statistics.broadcastreceived esx_pnic_statistics.broadcasttransmitted DerivedStatistics.Capacity DerivedStatistics.Demand DerivedStatistics.Usage DerivedStatistics.UsageCapacityPercent User Interface Name Received Transmitted Dropped Received Dropped Transmitted Bytes Received Bytes Transmitted Broadcast Received Broadcast Transmitted Capacity (MBps) Demand (MBps) Usage (MBps) Used Capacity Percent Host Adapter Metrics The Management Pack for Storage Devices collects metrics about the host adapters in the host system. Table 5 3. Host Adapter Metrics Metric Name AdapterErrorStatistics.FailedCommands AdapterErrorStatistics.FailedBlocksRead AdapterErrorStatistics.FailedBlocksWritten AdapterErrorStatistics.FailedReadOperations AdapterErrorStatistics.FailedWriteOperations AdapterErrorStatistics.LipCount AdapterErrorStatistics.ErrorFrames AdapterErrorCountPerMin.LinkFailureCount AdapterErrorCountPerMin.LossOfSignalCount AdapterErrorCountPerMin.InvalidTxWordCount AdapterErrorCountPerMin.InvalidCRCCount ReadStatistics.BlocksRead ReadStatistics.ReadOperations WriteStatistics.BlocksWritten WriteStatistics.WriteOperations TaskManagementStatistics.Aborts TaskManagementStatistics.LunResets FCStatistics.TxFrames FCStatistics.RxFrames DerivedStatistics.CongestionRatio DerivedStatistics.HostAdapterCapacity User Interface Name Failed Commands Failed Blocks Read Per Second Failed Blocks Written Per Second Failed Read Operations Per Second Failed Write Operations Per Second Lip Count Error Frames Per Second Link Failure Count Loss Of Signal Count Invalid Tx Word Count Invalid CRC Count Blocks Read Per Second Read Operations Per Second Blocks Written Per Second Write Operations Per Second Aborts Per Minute Lun Resets Per Minute Tx Frames Per Second Rx Frames Per Second Congestion Ratio Host Adapter Capacity (MBps) 22 VMware, Inc.

23 Chapter 5 Management Pack for Storage Devices Metrics Table 5 3. Host Adapter Metrics (Continued) Metric Name DerivedStatistics.HostAdapterDemand DerivedStatistics.HostAdapterUsage DerivedStatistics.HostAdapterInferredLatency DerivedStatistics.IOPs User Interface Name Host Adapter Demand (MBps) Host Adapter Usage (MBps) Input/Output latency observed on the host adapter. Aggregation of the read and write operations per second. AdapterErrorStatistics and AdapterErrorCountPerMin metrics are displayed only in Fibre Channel (FC) and Fibre Channel over Ethernet (FCoE) environments. Magnetic Disk, Solid State Device, and Disk Group Metrics The Management Pack for Storage Devices collects metrics about the magnetic disks, solid state devices, and disk groups on the ESXi hosts. Table 5 4. Magnetic Disk, Solid State Device, and Disk Group Metrics Metric Name busresets capacity.usage commandsaborted devicelatency devicereadlatency devicewritelatency maxqueuedepth numberread numberreadaveraged numberwrite numberwriteaveraged read scsireservationconflicts totallatency totalreadlatency totalwritelatency usage write HDDErrors IOPS User Interface Name Bus Resets Capacity Usage Commands Aborted Device Latency Device Read Latency Device Write Latency Max Queue Depth Number Read Number Read Averaged Number Write Number Write Averaged Read SCSI Reservation Conflicts Total Latency Total Read Latency Total Write Latency Usage Write HDD Errors IOPS VMware, Inc. 23

24 vrealize Operations Management Pack for Storage Devices Guide Magnetic Disk and Solid State Device Smart Metrics The Management Pack for Storage Devices collects SMART metrics about magnetic disks and solid state devices on the ESXi hosts. Table 5 5. Magnetic Disk and Solid State Device SMART Metrics Metric Name scsi_smart_statistics.healthstatus.value scsi_smart_statistics.mediawearoutindicator.value scsi_smart_statistics.mediawearoutindicator.threshold csi_smart_statistics.writeerrorcount.value scsi_smart_statistics.writeerrorcount.threshold scsi_smart_statistics.readerrorcount.value scsi_smart_statistics.readerrorcount.threshold scsi_smart_statistics.poweronhours.value scsi_smart_statistics.reallocatedsectors.value scsi_smart_statistics.reallocatedsectors.threshold scsi_smart_statistics.rawreaderrorrate.value scsi_smart_statistics.rawreaderrorrate.threshold scsi_smart_statistics.drivetemperature.value scsi_smart_statistics.drivetemperature.worst scsi_smart_statistics.drivetemperature.threshold scsi_smart_statistics.driveratedmaxtemperature.value scsi_smart_statistics.driveratedmaxtemperature.threshold scsi_smart_statistics.writesectorstotcount.value csi_smart_statistics.writesectorstotcount.threshold scsi_smart_statistics.readsectorstotcount.value scsi_smart_statistics.readsectorstotcount.threshold scsi_smart_statistics.initialbadblockcount.value scsi_smart_statistics.initialbadblockcount.threshold User Interface Name Health Status Value Media Wearout Indicator Value Media Wearout Indicator Threshold Write Error Count Value Write Error Count Threshold Read Error Count Value Read Error Count Threshold Power on Hours value Re-allocated sectors value Re-allocated sectors threshold Raw Read Error Rate Value Raw Read Error Rate Threshold Drive Temperature Value Drive Temperature Worst Drive Temperature Threshold Drive Rated Max Temperature Value Drive Rated Max Temperature Threshold Write Sectors TOT Count Value Write Sectors TOT Count Threshold Read Sectors TOT Count Value Read Sectors TOT Count Threshold Initial Bad Block Count Value Initial Bad Block Count Threshold Mount Metrics The Management Pack for Storage Devices collects metrics about a storage LUN abstraction that is available on the host system. Table 5 6. Mount Metrics Metric Name DeviceErrorStatistics.FailedCommands DeviceErrorStatistics.FailedReadOperations DeviceErrorStatustics.FailedWriteOperations DeviceErrorStatustics.FailedBlocksWritten ReadStatistics.BlocksRead User Interface Name Failed Commands Failed Read Operations Failed Write Operations Failed Blocks Written Blocks Read 24 VMware, Inc.

25 Chapter 5 Management Pack for Storage Devices Metrics Table 5 6. Mount Metrics (Continued) Metric Name ReadStatistics.ReadOperations WriteStatistics.BlocksWritten WriteStatistics.WriteOperations TaskManagementStatistics.Aborts TaskManagementStatistics.LunResets DerivedStatistics.CongestionRatio HostSpecificMetrics.PathHealthRatio DerivedStatistics.MountCapacity DerivedStatistics.MountDemand DerivedStatistics.MountUsage DerivedStatistics.MountInferredLatency DerivedStatistics.MountAggregatedErrors DerivedStatistics.IOPS User Interface Name Read Operations Blocks Written Write Operations Task Management Statistics LUN Resets Ratio of the active commands to the queued commands on the mount. Path Health Ratio Mount Throughput Capacity (MBps) Mount Throughput Demand (MBps) Mount Throughput Usage (MBps) Mount Inferred Latency Mount errors IOPS NFS Volume Metrics The Management Pack for Storage Devices collects metrics about Network File System (NFS) volumes. Table 5 7. NFS Volume Metrics Metric Name DerivedStatistics.ThroughputCapacity DerivedStatistics.ThroughputUsage DerivedStatistics.InferredLatency NfsVolumePerformanceMetrics.datastoreIops NfsVolumePerformanceMetrics.datastoreWriteIops NfsVolumePerformanceMetrics.read NfsVolumePerformanceMetrics.write NfsVolumePerformanceMetrics.totalReadLatency NfsVolumePerformanceMetrics.datastoreReadBytes NfsVolumePerformanceMetrics.datastoreWriteBytes NfsVolumePerformanceMetrics.datastoreReadIops NfsVolumePerformanceMetrics.totalWriteLatency User Interface Name Throughput (MBps) Current Throughput (MBps) Inferred Latency Aggregate IOPS Writes per second Read Rate Write Rate Total Read Latency Bytes Read Bytes Written Reads per second Total Write Latency Storage LUN Metrics The Management Pack for Storage Devices collects metrics about the storage LUN of a storage array. Table 5 8. Storage LUN Metrics Metric Name lun.percentused lun.luncapacity User Interface Name Percent Used LUN Capacity VMware, Inc. 25

26 vrealize Operations Management Pack for Storage Devices Guide Switch Port Metrics The Management Pack for Storage Devices collects metrics about the port on a switch. Table 5 9. Switch Port Metrics Metric Name switch_port_statistics.addresserrors switch_port_statistics.bytesreceived switch_port_statistics.bytestransmitted switch_port_statistics.crcerrors switch_port_statistics.delimitererrors switch_port_rate_statistics.rxframerate switch_port_rate_statistics.rxrate switch_port_rate_statistics.txframerate switch_port_rate_statistics.txrate DerivedStatistics.SwitchPortCapacity DerivedStatistics.SwitchPortUsage DerivedStatistics.AggregatedErrors User Interface Name Address Errors Bytes Received Bytes Transmitted CRC Errors Delimiter Errors Rx Frame Rate Rx Rate Tx Frame Rate Tx Rate Switch Port Capacity (MBps) Switch Port Capacity NA Virtual SAN Data Store (Performance Metrics) The Management Pack for Storage Devices collects performance metrics about the Virtual SAN data stores in your datacenter. Table Virtual SAN Data Store (Performance Metrics) Metric Name PerformanceStatistics.opIngressCongestionCount PerformanceStatistics.readCachedBytes PerformanceStatistics.readCachedCount PerformanceStatistics.writeCongestionSum PerformanceStatistics.recoveryWriteCount PerformanceStatistics.readCount PerformanceStatistics.recoveryWriteBytes PerformanceStatistics.writeBytes PerformanceStatistics.recoveryWriteCongestionSum PerformanceStatistics.prepareLimitRetryCount PerformanceStatistics.readCachedLatencyMaxUs PerformanceStatistics.readCongestionSum PerformanceStatistics.ioCount PerformanceStatistics.objIngressCongestionCount PerformanceStatistics.recoveryWriteLatencyMaxUs PerformanceStatistics.writeLatencyMaxUs PerformanceStatistics.writeCount User Interface Name opingresscongestioncount Total Read Cached Bytes Read Cached Count Write Congestion Sum Total Recovery Write Count Total Read Count Recovery Write Bytes Total Write Bytes Recovery Write Congestion Sum Prepare Limit Retry Count Maximum Read Cached Latency (ms) Total Read Congestion Total Input Output Count Object Ingress Congestion Count Maximum Recovery Write Latency (ms) Maximum Write Latency (ms) Total Write Count 26 VMware, Inc.

27 Chapter 5 Management Pack for Storage Devices Metrics Table Virtual SAN Data Store (Performance Metrics) (Continued) Metric Name PerformanceStatistics.readLatencyMaxUs PerformanceStatistics.readCachedCongestionSum User Interface Name Maximum Read Latency (ms) Total Read Cached Congestion Virtual SAN Data Store Metrics The Management Pack for Storage Devices collects metrics about the Virtual SAN data stores in your datacenter. Table Virtual SAN Data Store Metrics Metric Name DerivedStatistics.totalCapacity DerivedStatistics.provisionedCapacity DerivedStatistics.usedCapacity DerivedStatistics.latency DerivedStatistics.throughput DerivedStatistics.maxThroughput DerivedStatistics.avgTotalCapacityPerDiskGroup DerivedStatistics.avgUsedCapacityPerDiskGroup DerivedStatistics.avgThroughputPerDiskGroup DerivedStatistics.avgMaxThroughputPerDiskGroup HostSpecificMetrics.readCachedLatency HostSpecificMetrics.readLatency HostSpecificMetrics.recoveryWriteLatency HostSpecificMetrics.writeLatency HostSpecificMetrics.PerHostThroughput HostSpecificMetrics.MaxPerHostThroughput User Interface Name Total Capacity (MB) Demand (MB) Usage (MB) Inferred Latency Throughput (MBps) Maximum Throughput (MBps) Average Capacity Per Disk Group (GB) Average Used Capacity Per Disk Group (GB) AverageThroughput Per Disk Group (MBps) Average Max Throughput Per Disk Group (MBps) Read Cached Latency (ms) Read Latency (ms) Recovery Write Latency (ms) Write Latency (ms) Throughtput (KBps) Maximum Throughtput (KBps) Virtual SAN Host Metrics The Management Pack for Storage Devices collects metrics about the Virtual SAN host in your datacenter. Table Virtual SAN Host Metrics Metric Name User Interface Name VsanStatistics.ComponentUsage Component usage (%) VsanStatistics.capacity VsanStatistics.totalUsedCapacity DerivedStatistics.ThroughputCapacity DerivedStatistics.ThroughputUsage VsanStatistics.failedCommands VsanStatistics.latency.totalTime Capacity (GB) Capacity Used (GB) Throughput (MBps) Current Throughput (MBps) Failed Commands Latency total time VsanStatistics.capacityUsed Disk Capacity Used (%) VMware, Inc. 27

28 vrealize Operations Management Pack for Storage Devices Guide Table Virtual SAN Host Metrics (Continued) Metric Name VsanStatistics.capacity User Interface Name Disk Capacity (GB) VsanStatistics.Read Cache Hit Rate (%) Read Cache Hit Rate (%) Object Properties The Management Pack for Storage Devices displays object properties. You can view the object properties from the vrealize Operations Manager user interface. From the vrealize Operations Manager user interface, click Environment > Storage Devices > Analysis tab. Disk Group Properties The Management Pack for Storage Devices displays the disk group object properties. Table Disk Group Properties Name Type Display Name Description HostUuid String Host UUID The UUID of the host. ESXi Physical Network Interface Card Properties The Management Pack for Storage Devices displays the properties of the ESXi Physical Network Interface Card (NIC) object. Table ESXi Physical NIC Properties Name Type Display Name Description Link Speed long Link Speed The speed of the link. Link State string Link State Indicates the state of the link. PCI Device string PCI Device Indicates the type of PCI used. Driver Name string Driver Name Indicates the name of the driver. Host Adapter Properties The Management Pack for Storage Devices displays the properties of the host adapter. Table Host Adapter Properties Name Type Display Name FirmwareVersion string Firmware Version The host bus adapter (HBA) firmware version. DriverName string Driver Name The HBA driver name in ESXi. DriverVersion string Driver Version The HBA drvier version in ESXi. PortSpeed string Port Speed The configured port speed. SupportedPortSpeeds string Supported Port Speeds The list of supported port speeds. Vendor string Vendor The vendor's name for the HBA. Model string Model The model name of the HBA. 28 VMware, Inc.

29 Chapter 5 Management Pack for Storage Devices Metrics Table Host Adapter Properties (Continued) Name Type Display Name HostPortName string Host Port Name The World Wide Port Name (WWPN) of the HBA. HostNodeName string Host Node Name The World Wide node name of the HBA. HardwareVersion string Hardware Version The hardware version of the HBA. OptionROMVersion string Option ROM Version The read-only memory of the HBA. MaxFrameSize string Max Frame Size The maximum size of a frame. PortType string Port Type The port type such as N port and F port. FabricName string Fabric Name The WWPN of the fabric the HBA connects. Name string Name The HBA name in ESXi. IP Switch Properties The Management Pack for Storage Devices displays the properties of the IP switch objects. Table IP Switch Properties Name Type Display Name Description SNMP Enabled string SNMP Enabled Indicates if the IP switch is SNMP enabled. IP Switch Port Properties The Management Pack for Storage Devices displays the properties of the IP switch port object. Table IP Switch Port Properties Name Type Display Name Description Port State string Port State Indicates the state of the port. Storage Array Properties The Management Pack for Storage Devices displays the properties of the storage array object. Table Storage Array Properties Name Type Display Name Description Fimware string Storage Array Firmware The firmware version of the array. ModelId string Storage Array Model Id The model number. VendorId string Vendor ID The vendor identifier. SupportedBlockInterfaces string Supported Block Interfaces The list of supported block interfaces like Fibre Channel (FC) and iscsi. SupportedFileSystemInterf aces string Supported File System Interfaces The list of supported file system interfaces like Network File System (NFS) and Common Internet File System (CIFS). Name string Array Name The array name. VMware, Inc. 29

30 vrealize Operations Management Pack for Storage Devices Guide Table Storage Array Properties (Continued) Name Type Display Name Description UUID string SMS Array Uuid The UUID of the array. AlternateNames string Alternate Names - SupportedProfile string Supported Profile The list of supported VASA profiles. Magnetic Disk Properties The Management Pack for Storage Devices displays the properties of the magnetic disk object. Table Magnetic Disk Properties Name Type Display Name Description Name string Name The name of the magnetic disk. Size string Size The size of the magnetic disk. Vendor string Vendor The name of the manufacturer of the magnetic disk. is Virtual SAN Entity string is Virtual SAN Entity - QueueDepth string Queue Depth The number of pending input/output requests. Mount Properties TheManagement Pack for Storage Devices displays the properties of the mount object. Table Mount Properties Name Type Display Name Description Vendor string Vendor The vendor name of the mount device. Model string Model The number of the mount device. Size long Mount Size in KB The capacity of the mount device. Name string Name The device identifier of the mount device. Solid State Device Properties The Management Pack for Storage Devices displays the properties of the solid state device object. Table Solid State Device Properties Name Type Display Name Description Name string Name The name of the solid state device. Size string Size The size of the solid state device. Vendor string Vendor The name of the manufacturer of the solid state device. is Virtual SAN Entity string is Virtual SAN Entity - QueueDepth string QueueDepth The number of pending input/output requests. 30 VMware, Inc.

31 Chapter 5 Management Pack for Storage Devices Metrics Switch Properties The Management Pack for Storage Devices displays the properties of the switch object. Table Switch Properties Name Type Display Name Description Manufacturer string Manufacturer The name of the manufacturer of the switch. Model string Model The model of the switch. PartNumber string Part Number The part number of the switch. SerialNumber string Serial Number The serial number of the switch. ManufacturerSerialNumber string Manufacturer Serial Number The serial number of the manufacturer of the switch. VersionString string Version String The version number. ClassificationDescriptions string Classification Descriptions The version class (operating system and firmware). Switch Port Properties The Management Pack for Storage Devices displays the properties of the switch port object. Table Switch Port Properties Name Type Display Name Description Speed long Speed The configured speed. MaxSpeed long Max Speed The maximum speed. PortType string Port Type The type of port. PortNumber string Port Number The port number. PermanentAddress string Permanent Address The WWPN of the switch port. EnabledState string Enabled State Indicates if the port is enabled. VMware, Inc. 31

32 vrealize Operations Management Pack for Storage Devices Guide 32 VMware, Inc.

33 User Scenario: An Object in Your Storage Network Has a Performance Problem 6 A user reports that a virtual machine is running slow. She is not sure how long the problem has existed, but needs to process data overnight and wants to know if she can start the task before she leaves for the day. As the system administrator, you realize that the problem might not be with the virtual machine itself. Rather than investigate the state of only the virtual machine, you begin by reviewing the dashboard that shows the state of all storage network objects associated with the virtual machine. Procedure 1 See the topic called User Scenario: Use the Troubleshooting Dashboard to Locate the Real Problem Source As the system administrator for the data center, you must locate the user's virtual machine in the network storage topology so that you can determine if the problem is with the host system or somewhere else in the storage network. 2 See the topic called User Scenario: Examine Metrics to Find the Root Cause To determine if the virtual machine performance problem is temporary or indicates a more serious problem in the storage network, you create metric charts to identify the root cause of the problem. This chapter includes the following topics: User Scenario: Use the Troubleshooting Dashboard to Locate the Real Problem Source, on page 33 User Scenario: Examine Metrics to Find the Root Cause, on page 34 User Scenario: Use the Troubleshooting Dashboard to Locate the Real Problem Source As the system administrator for the data center, you must locate the user's virtual machine in the network storage topology so that you can determine if the problem is with the host system or somewhere else in the storage network. The fastest way to inspect all of the components connected to the virtual machine is to look at the Troubleshooting dashboard. Start by locating the problematic object. Procedure 1 Click the Troubleshooting dashboard. The dashboard displays badges for all the objects that the Management Pack for Storage Devices discovered in the storage area network. 2 In the group of virtual machines displayed, point to each badge icon to see the object name. VMware, Inc. 33

34 vrealize Operations Management Pack for Storage Devices Guide 3 Identify the virtual machine that the user reported as slow and select the badge. This action highlights all the objects in the storage area network path from the host system to the storage mount that are associated with the virtual machine. 4 Review highlighted objects that appear with a red, orange, or yellow condition. 5 Select an object to inspect problems or alerts reported in the widgets on the right. Problems such as low throughput do not generate alerts but can affect performance. You identified storage components associated with the virtual machine that have problems. You can now select one of the components to examine faults, anomalies, and metrics on that object and determine if its problems are the root cause of problems seen on the virtual machine. User Scenario: Examine Metrics to Find the Root Cause To determine if the virtual machine performance problem is temporary or indicates a more serious problem in the storage network, you create metric charts to identify the root cause of the problem. You used the Troubleshooting dashboard to find an object networked to the problematic virtual machine that is not operating at an optimum. Now you want to investigate the performance of that object over a period of time. Procedure 1 If you are on the Troubleshooting dashboard with the problematic virtual machine selected, select a highlighted object that appears with a red, orange, or yellow condition. 2 Double-click the object to access collected metrics. The left pane displays the object name and the related objects. The main pane displays the Summary tab. 3 Click the Troubleshooting tab, and click the All Metrics tab. Throughput, latency, and error metrics are the most useful troubleshooting metrics. a b c Select the error statistics and double-click an entry to determine if errors occurred over time. Errors can indicate when the component had trouble accessing the network. Select the derived statistics and double-click the use to see a historical graph of the component use over time. Slow throughput can occur during times of high workload. Select the derived statistics and double-click the latency to see a historical graph of the time required to process IO requests. High latency can indicate that the component had trouble receiving data from the network. What to do next If the error statistics show no problems and derived statistics show a cyclical use pattern on the component, you can call the user and explain that the problem might be temporary and when IO traffic throughout the network drops, the virtual machine will become more responsive. If the error statistics show no problems and derived statistics show consistently low use of the component but the latency is suddenly high, another component on the network might be causing connectivity to time out. Check for a faulty switch or other faulty component on the network. 34 VMware, Inc.