vrealize Operations Management Pack for Storage Devices Guide

Transcription

1 vrealize Operations Management Pack for Storage Devices Guide This document supports the version of each product listed and supports all subsequent versions until the document is replaced by a new edition. To check for more recent editions of this document, see EN

2 vrealize Operations Management Pack for Storage Devices Guide You can find the most up-to-date technical documentation on the VMware Web site at: The VMware Web site also provides the latest product updates. If you have comments about this documentation, submit your feedback to: Copyright 2015 VMware, Inc. All rights reserved. Copyright and trademark information. VMware, Inc Hillview Ave. Palo Alto, CA VMware, Inc.

3 Contents About This Book 5 1 Introduction to the vrealize Operations Management Pack for Storage Devices 7 2 Installing and Configuring the Management Pack 9 Install the Management Pack 9 Add Credentials and Fabric Servers for the Management Pack for Storage Devices 10 Verify That Your Management Pack for Storage Devices Is Connected and Collecting Data 12 Ports and Log Files 13 3 Management Pack for Storage Devices Dashboards 15 4 Management Pack for Storage Devices Alert Definitions 17 5 Management Pack for Storage Devices Metrics 25 Statistical Metrics of the Management Pack for Storage Devices 25 Object Properties 33 6 User Scenario: An Object in Your Storage Network Has a Performance Problem 39 User Scenario: Use the Troubleshooting Dashboard to Locate the Real Problem Source 39 User Scenario: Examine Metrics to Find the Root Cause 40 Index 41 VMware, Inc. 3

4 vrealize Operations Management Pack for Storage Devices Guide 4 VMware, Inc.

5 About This Book The vrealize Operations Management Pack for Storage Devices Guide provides you with a complete view of your entire storage topology from your host, through your storage network, and out to the storage array. With this solution, you can use vrealize Operations Manager to monitor and troubleshoot capacity and performance problems on different components of your storage area network. Intended Audience The information in this guide is intended for storage administrators, data center architects, and IT operators. VMware Technical Publications Glossary VMware Technical Publications provides a glossary of terms that might be unfamiliar to you. For definitions of terms as they are used in VMware technical documentation, go to VMware, Inc. 5

6 vrealize Operations Management Pack for Storage Devices Guide 6 VMware, Inc.

7 Introduction to the vrealize Operations Management Pack for Storage Devices 1 The Management Pack for Storage Devices provides you with a complete view of your entire storage topology from your host, through your storage network, and out to the storage array. With this solution, you can use vrealize Operations Manager to monitor and troubleshoot capacity and performance problems on different components of your storage area network. The Management Pack for Storage Devices has the following features: Provides an end-to-end view of topology, statistics, and events at every affected level of the storage area network. Enables a vsphere administrator working with a virtual environment to isolate problems caused by elements in the physical storage stack, such as the Host Bus Adapter (HBA), storage switches, and array. The vsphere administrator uses this information to hand off the problem to the domain administrator for further analysis. Captures and analyzes information for throughput and latency on the HBA and mount objects. Captures throughput on the switch ports. Captures IOPS and queue depth at the HBA and switch ports for read and write components. Discovers the storage switches and applies the credentials required to collect data from the objects in the storage network. It uses the Common Information Model (CIM) to exchange information with objects managed by the following management systems: Cisco Data Center Network Manager (DCNM) Brocade Network Advisor (BNA) VMware, Inc. 7

8 vrealize Operations Management Pack for Storage Devices Guide 8 VMware, Inc.

9 Installing and Configuring the 2 Management Pack When you install a management pack, you install the PAK file that you downloaded as part of the solution, and you configure the management pack instance. This chapter includes the following topics: Install the Management Pack, on page 9 Add Credentials and Fabric Servers for the Management Pack for Storage Devices, on page 10 Verify That Your Management Pack for Storage Devices Is Connected and Collecting Data, on page 12 Ports and Log Files, on page 13 Install the Management Pack If your environment includes multiple vcenter Server systems on the same storage network, you must add fabric switch CIM servers for each adapter instance. The adapter instance provides you with a holistic view of the storage network connected to the vcenter Server system. The adapter discovers the storage devices only for the ESXi 5.1 update 2 and later. NOTE Any adapter credentials you add are shared with other adapter administrators and vrealize Operations Manager collector hosts. Other administrators might use these credentials to configure a new adapter instance or to move an adapter instance to a new host. Prerequisites Verify that the following conditions are met: Read-only credentials are not adequate to connect and collect data from the vcenter Server and fabric CIM servers. You must provide credentials with sufficient privileges. If the user account has limited access to objects in vcenter and the fabric CIM server, you can only collect data from objects for which you have permission. The CIM agent is enabled on all storage network switches from which you are collecting data, and that you can provide credentials with sufficient privileges to connect and collect data. If the user account has limited access to objects in the storage environment, you can only collect data from objects for which you have permission. Credentials must include at least read permissions. All storage network switches with the CIM agent enabled are added to DCNM or BNA solutions. The fabric CIM service supports switch profile and fabric profile. The supported versions for the Fabric Profile are 1.1 to 1.6. VMware, Inc. 9

10 vrealize Operations Management Pack for Storage Devices Guide A vsphere API for Storage Awareness (VASA) provider is registered as a storage provider with the vcenter Server host for your adapter instance. VASA providers gather information from arrays that support the environment and provide the data to vcenter Server system. If the VASA provider is not registered with vcenter Server host, the Management Pack for Storage Devices cannot discover objects in the array. The minimum support version is VASA 1.0. See information regarding storage providers in the ESXi and vcenter Server 5.5 Documentation. Specify the search domain value on vrealize Operations Manager to reach vcenter, hosts, and switches using their Domain Name System (DNS) names instead of IP addresses. The resolv.conf file is used by vrealize Operations Manager to connect to remote machines over the network. You must configure the file correctly so that vrealize Operations Manager can reach the destination machines, such as vcenter, ESXi hosts, or Fibre Channel (FC) switches. As a workaround, you can specify the search domain in the /etc/resolve.conf file on vrealize Operations Manager and then restart the collector. Download and install the Management Pack for Storage Devices. Procedure 1 Log in to the vrealize Operations Manager user interface with administrator privileges. 2 In the left pane of vrealize Operations Manager, click the Administration icon and click Solutions. 3 On the Solutions tab, click the plus sign. 4 On the Solutions tab, select Management Pack for Storage Devices and click Configure. 5 Browse to locate the temporary folder and select the PAK file. For example, managementpack_name-buildnumber.pak. 6 Click Upload. The upload might take several minutes. 7 Read and accept the EULA, and click Next. Installation details appear in the window during the process. 8 Ensure that you install the PAK file on the master node. 9 When the installation is complete, click Finish. What to do next Add credentials and add the fabric servers Add Credentials and Fabric Servers for the Management Pack for Storage Devices When configuring an adapter instance for the Management Pack for Storage Devices, you add credentials for a vcenter Server. Procedure 1 In the vcenter Server text box, enter the FQDN or IP address of the vcenter Server instance to which you are connecting. The vcenter Server FQDN or IP address must be reachable from all nodes in the vrealize Operations Manager cluster. 10 VMware, Inc.

11 Chapter 2 Installing and Configuring the Management Pack 2 To add credentials on the Manage Solution page, click the plus sign. a b c In the Credential name text box, enter the name by which you are identifying the configured credentials. Type the User name and Password for the vcenter Server instance. Click OK. You configured credentials to connect to a vcenter Server instance. 3 Click Test Connection to validate the connection with your vcenter Server instance. 4 Accept the vcenter Server security certificate. 5 Enter the SNMP Community Strings for your SNMP v2 enabled IP Network Switches. 6 To add fabric servers, click Add Fabric Server. a b c Click the plus sign. Enter a name and description for the fabric server. Enter the fabric switch CIM server IP or URL of the host on which the CIM server is installed. The Management Pack for Storage Devices supports CIM servers that manage Fibre Channel switches only. It does not support CIM servers that manage arrays. Option Brocade fabric Cisco fabric Action Enter the IP address Enter the service URL in the following format: Namespace>, for example, d e f To add credentials, click the plus sign. In the Credential name text box, enter the name by which you are identifying the configured credentials. Enter the user name and password for the Brocade or Cisco Fabric Switch CIM server. For the Cisco Fabric Switch CIM server, the user name and password are for the Cisco Data Center Network Manager (DCNM) and not the Cisco switch. g h Accept all the security certificates from the Fabric Switch CIM Server. Click OK. 7 Click Save Settings. The adapter is added to the Adapter Instance list and is active. What to do next To verify that the adapter is configured and collecting data from the objects in the storage topology, wait a few collection cycles, then view application-related data. Inventory Explorer. Verify that all the objects related to the Storage Devices Instance are listed. Objects should be in the collecting state and receiving data. Dashboards. Verify that the Storage Components Heatmap, the Storage Components Usage, and the Storage Troubleshooting are added to the default dashboards. VMware, Inc. 11

12 vrealize Operations Management Pack for Storage Devices Guide Verify That Your Management Pack for Storage Devices Is Connected and Collecting Data You configured an adapter instance of the Management Pack for Storage Devices with credentials for a vcenter Server. Now you want to verify that your adapter instance can retrieve information from objects in the host, switch, and array domains of your environment. To view the object types, from the left pane of the vrealize Operations Manager user interface, click Administration > Inventory Explorer > Adapter Instances > Storage device Instances > <User_Created_Instance>. Table 2 1. Object Types that the Management Pack for Storage Devices Discovers Object Type Active Zone Set ESX pnic Fabric Fabric Switch CIM Server Host Adapter IP Switch IP Switchport Logical Switch Magnetic Disks Mount NFS Client NFS Servers No Existing Object Container SAN (Storage Area Network) Solid State Device Storage Array Storage Devices Instance Storage LUN Storage Processor Switch Switch Port Target Port VirtualSAN Cluster Description A collection of all active zones in the storage network. The physical Network Interface Cards (NICs) on the ESXi host. In the Brocade storage network, fabric is a collection of interconnected switches. There is typically a one-to-one relationship between the fabric and the SAN. In the Cisco storage network, fabric is a virtual SAN which is collection of one or more logical switches. There is typically a one-to-many relationship between the SAN and the fabric. CIM server that manages the switches. It provides the username, password and URL information that the Management Pack for Storage Devices uses to discover storage network objects such as fabric, switches, and switch ports. The Fibre Channel (FC) or Fibre channel over Ethernet (FcoE) host adapters in the host system. The Simple Network Management Protocol (SNMP) v2 enabled (IP) network switches The switch ports on your SNMP v2 enabled IP network switches A logical switch abstraction for the Cisco storage network. It is a collection of one or more switch ports that are accessible to one other. Magnetic disks on the ESXi hosts. A storage LUN abstraction that is available on the host system. Network File System (NFS) clients on the ESXi hosts. NFS servers registered with vcenter via VASA. A container that includes all the objects that are no longer discovered in the storage devices topology. In the Brocade storage network, the SAN is a collection of fabrics. There is typically a one-toone relationship between the SAN and the fabrics. In the Cisco storage network, the SAN is a collection of one or more Virtual Storage Area Networks (VSAN). Solid State Devices (SSDs) on the ESXi hosts. The storage array registered to the VASA provider in the vcenter for the adapter instance. The Management Pack for Storage Devices instance. The storage LUN of a storage array. The storage controller on the storage array. The switch on the storage network that connects the host to the storage array. The port on a switch. The target port of a storage array. Virtual SAN clusters in your data center. 12 VMware, Inc.

13 Chapter 2 Installing and Configuring the Management Pack Table 2 1. Object Types that the Management Pack for Storage Devices Discovers (Continued) Object Type VirtualSAN Datastore VirtualSAN Diskgroups Zone Description Virtual SAN datastores in your data center. A collection of SSDs and magnetic disks used by Virtual SAN. A collection of target ports or host adapters in the storage network. Procedure 1 In the left pane of vrealize Operations Manager, click the Administration tab and click Inventory Explorer. 2 In the list of tags, expand Adapter Instances and expand Storage Devices Instance. 3 Select the adapter instance name to display the list of objects discovered by your adapter instance. 4 Slide the display bar to the right to view the object status. Object Status Collection State Collection Status Description If green, the object is connected. If green, the adapter is retrieving data from the object. 5 Deselect the adapter instance name and expand the Object Types tag. Each Object Type name appears with the number of objects of that type in your environment. 6 Verify that you have at least one object in each of the network areas that the Management Pack for Storage Devices discovers. What to do next If objects are missing or not transmitting data, you might need to investigate further. If an object is not connected or not transmitting data, search for the object and check for related alerts. If the adapter does not discover at least one object on the host system, in the storage network, and in the storage array, verify that your environment is configured properly. Common configuration problems such as invalid credentials, Fabric Switch CIM servers not reachable, or VASA providers not registered in the vcenter Server instance are raised as alerts for the Management Pack for Storage Devices instance. Ports and Log Files Some services and ports must be available to communicate with the rest of the data center. External Interfaces, Ports, and Services required by the Management Pack for Storage Devices For monitoring Fiber Channel (FC) and Fiber Channel over Ethernet (FCoE) using CIM protocol (SMI-S standard): Port 5989 must be open on each ESXi host. Port 5988 must be open on each FC Switch using http or port 5989 must be open on each FC Switch using https. For monitoring NFS and iscsi topologies over ethernet switches using CDPv2, LLDPv2, and SNMPv2: Ports 161 and 162 must be open on ethernet switches for SNMP. CDP and LLDP advertisements/tlvs must be permitted on the network. VMware, Inc. 13

14 vrealize Operations Management Pack for Storage Devices Guide Log Files and Security Updates Log files are available in the vrealize Operations Manager platform in the /usr/lib/vmwarevcops/user/log/adapters/universalstorageadapter3/ folder. Administrator privileges are assigned to service users. You can download and apply the latest security update from Solutions Exchange. 14 VMware, Inc.

15 Management Pack for Storage 3 Devices Dashboards When the Management Pack for Storage Devices is installed, it adds preconfigured dashboards and alert definitions that you use to monitor and troubleshoot the components in your storage area network. Table 3 1. Management Pack for Storage Devices Dashboards Dashboard for protocols such as FC-FCoE, iscsi, NFS, and Virtual SAN Troubleshooting Component/Entity Usage Heatmap VirtualSAN Device Insights VirtualSAN Cluster Insights Purpose View end-to-end storage topology, health, and alerts on storage components. Provides an overview of the most frequently used objects on the storage network and tracks the error count on these objects. Provides an overview of object performance against its predefined limits. Provides insights into device level details including capacity, usage, and errors of devices used by Virtual SAN. Provides insights into cluster level details for disk groups and Virtual SAN datastores. VMware, Inc. 15

16 vrealize Operations Management Pack for Storage Devices Guide 16 VMware, Inc.

17 Management Pack for Storage 4 Devices Alert Definitions When the Management Pack for Storage Devices is installed, it adds preconfigured dashboards and alert definitions that you use to monitor and troubleshoot the components in your storage area network. vrealize Operations Manager generates an alert if a problem occurs with the components in the storage area network that the Management Pack for Storage Devices is monitoring. Table 4 1. Management Pack for Storage Devices Alert Definitions Alert (ID name) Alert Level Affected Object Type Description Adapter Configuration failed (AdapterNotConfigured) Critical Storage Devices Adapter Instance Triggered when the Management Pack for Storage Devices instance is incorrectly configured because of invalid credentials or inaccessible Fabric CIM or vcenter Server instances. Clears when the adapter is reconfigured and can successfully communicate with data sources, such as Fabric CIM or vcenter Server. All Paths Down on some Mounts (APD) Critical VMWARE: Host System Triggered by an All Paths Down (APD) situation on one or more mounts on the host system. This generic alert applies when the root cause for APD cannot be attributed to a zone configuration change, host adapter link failure, or switch port failure. Clears when the APD situation is resolved on the host system. All Paths Down on some Mounts because HBA is Link down (APDHostAdapterLinkDown) Critical VMWARE: Host System Triggered by an APD situation on one or more mounts on the host system because of a problem with the host adapter. Clears when the APD situation is resolved on the host system. All Paths Down On Some Mounts because the physical NIC status is down (APDPnicDown) Critical VMWARE: Host System Triggered by an APD situation on one or more mounts on the host system, because of the problem with the physical Network Interface Card (NIC) connection to the host adapter. Clears when the APD situation is resolved on the host system. VMware, Inc. 17

18 vrealize Operations Management Pack for Storage Devices Guide Table 4 1. Management Pack for Storage Devices Alert Definitions (Continued) Alert (ID name) Alert Level Affected Object Type Description All Paths Down On Some Mounts Because Switch Ports connected to the HBAs is Down (APDInitiatorSwitchPortDown) Critical VMWARE: Host System Triggered by an APD situation on one or more mounts on the host system, because of a problem with the switch port connection to the host adapter. Clears when the APD situation is resolved on the host system. All Paths Down On Some Mounts Because Switch Port connected to the Target is Down (APDTargetSwitchPortDown) Critical VMWARE: Host System Triggered by an APD situation on one or more mounts on the host system, because of a problem with the switch port connection to the target port on the array side. Clears when the APD situation is resolved on the host system. All Paths Down on some mounts because HBAs are zoned out (APDZoneConfiguration) Critical VMWARE: Host System Triggered by an APD situation on one or more mounts on the host system. This alert indicates a loss of connectivity between the host adapters on the host side and target ports on the array side because of zone configuration changes. Clears when the APD situation is resolved on the host system. Congestion due to high network bandwidth usage (NetworkBandwidthCongestion) Critical EsxPnic Triggered when the physical NIC bandwidth is over-utilized. Another symptom when the physical NIC capacity is less than 10 GB is also checked, if applicable. Clears when the physical NIC bandwidth usage comes back to normal. Datacenter has some HBAs that are not operating at maximum supported speed (HostAdapterSpeedNotOptimal) Info VMWARE: Datacenter Triggered if a connected HBA is not operating at its maximum supported speed. Clears when the HBA returns to its maximum operating speed. Fabric CIM server certificate is not trusted (FabricCIMServerUntrusted) Critical Fabric CIM Server Triggered if the Management Pack for Storage Devices cannot authenticate the fabric CIM server certificate. Clears when the adapter can authenticate the fabric CIM server certificate. Fabric CIM Server Configuration failed (FabricCredentialsWrong) Critical Fabric Switch CIM Server Triggered if the Management Pack for Storage Devices cannot communicate with the fabric switch CIM server using the login credentials or URL information provided. Clears when the adapter can communicate with the fabric switch CIM server. HBA Link Failure (HostAdapterLinkDown) Critical Host Adapter Triggered when the HBA status changes from Online to LINK DOWN. Clears when the HBA status returns to Online. 18 VMware, Inc.

19 Chapter 4 Management Pack for Storage Devices Alert Definitions Table 4 1. Management Pack for Storage Devices Alert Definitions (Continued) Alert (ID name) Alert Level Affected Object Type Description HBA Speed dropped (HostAdapterSpeedDropped) High Virtual LUN Resets Or Aborts (HighLunAbortsOrLunResets) IO Latency is above normal (MountIoLatencyHigh) Multiple ports on the Switch are down (SwitchPortsDown) Critical Host Adapter Triggered when the operating speed of the host adapter drops compared to a previous reading in the data collection cycle. Not automatically cleared. A user must manually clear the alert. Critical Mount Triggered if 50 or more virtual LUN resets or aborts occur per minute on the mount object during two consecutive collection cycles. Clears when the LUN resets or aborts drop to fewer than 50 per minute for two consecutive collection cycles. Critical Mount Triggered if the HostSpecificMetrics IOLatency metric exceeds the dynamic threshold. vrealize Operations Manager sets the dynamic threshold based on the historic data values of this metric. Clears when the metric values fall below the dynamic threshold. Critical Switch Triggered when 50 percent or more switch ports become disabled or display the Enabled but offline status. Clears when the percentage of unavailable switch ports falls below 50 percent. No VASA providers registered to vcenter. Cannot discover array objects (NoVasaProvidersRegistered) Warning Storage Devices Adapter Instance Triggered when no VASA providers are registered to the vcenter Server. Clears when at least one VASA provider is registered to the vcenter Server. Number of active paths is low (PathHealthRatioLow) Automatic Mount Triggered when the HostSpecificMetrics PathHealthRatio metric for the mount object falls below thresholds. vrealize Operations Manager sets the alert level based on the value of pathhealthratio: 75<=pathHealthRatio<100 indicates Immediate 50<=pathHealthRatio<75 indicates Warning pathheathratio< 50 indicates Critical Clears when pathhealthratio value becomes 100 for all the host system instanced metric groups of the mount object. VMware, Inc. 19

20 vrealize Operations Management Pack for Storage Devices Guide Table 4 1. Management Pack for Storage Devices Alert Definitions (Continued) Alert (ID name) Alert Level Affected Object Type Description Permanent Device Loss on a Mount/Storage LUN (PDL) Critical VMWARE: Host System Triggered when one or more mounts on the host system is unavailable. This situation occurs if the underlying storage LUN is unmasked or removed from the host system. If the storage LUN was unmasked without first detaching it from the host system, the alert clears when the mount object becomes available. If the storage LUN was permanently removed, the alert does not automatically clear. A user must manually clear the alert. Some NFS volumes are not reachable from the host because the physical NIC may be down (APDOnNFSClientBecauseOfPnic) Critical VMWARE: Host System Triggered by an APD situation on one or more Network File System (NFS) volumes on the host system, because the connected physical NIC status is Down. Clears when the APD situation is resolved on the host system. Some NFS volumes are not reachable from the host because the switch port connecting to the physical NIC on the host may be down (APDOnNFSClientBecauseOfSwitchPort) Critical VMWARE: Host System Triggered by an APD situation on one or more NFS volumes on the host system because of the switch port connected to the physical NIC is Disabled. Clears when the APD situation is resolved on the host system. Some NFS volumes are unreachable from the host (APDOnNFSClientGeneric) Critical VMWARE: Host System Triggered by a APD situation on one or more NFS volumes on the host system. Clears when the APD situation is resolved on the host system. Storage LUN Thin Provisioned Capacity At Critical Levels (ThinProvisionedThresholdExceeded) Storage LUN Thin Provisioned Capacity Threshold At Warning Level (ThinProvisionedThresholdInitialized) Critical Storage LUN Triggered when the thin provisioning capacity on the storage LUN reaches a critical level. Thresholds for thin provisioning are set in the vcenter Server. Clears when the vcenter Server sends the clear event to the Management Pack for Storage Devices. Warning Storage LUN Triggered when the thin provisioning capacity on the storage LUN reaches a warning level. The thresholds for thin provisioning are set in the vcenter. Clears when the vcenter sends the clear event to the Management Pack for Storage Devices. 20 VMware, Inc.

21 Chapter 4 Management Pack for Storage Devices Alert Definitions Table 4 1. Management Pack for Storage Devices Alert Definitions (Continued) Alert (ID name) Alert Level Affected Object Type Description Switch Port Down Failure (SwitchPortLinkDown) Critical Switch Port Triggered when the status of switch port changes to Disabled or Enabled but Offline. Clears when the switch port status returns to Enabled. VsanClusterPartitionedGeneric Critical VsanCluster Triggered when the Virtual SAN cluster is network partitioned and not all Virtual SAN hosts in the cluster can talk to each other due to the absence of network connectivity. Clears when the network connectivity between all Virtual SAN hosts in the cluster is restored. VsanClusterPartitionedDueToPnic Critical VsanCluster Triggered when the Virtual SAN cluster is network partitioned because the configured Physical NICs on one or more hosts is not available. Clears when the network connectivity between all the Virtual SAN hosts in the cluster is restored. VsanClusterPartitionedDueToSwitchPort Critical VsanCluster Triggered when the Virtual SAN cluster is network partitioned because the configured network switch ports on one or more IP switches are not available. Clears when the network connectivity between all Virtual SAN hosts in the cluster is restored. VsanClusterPartitionedDueToMulticastIssue Critical VsanCluster Triggered when the Virtual SAN cluster is network partitioned because the multicast addresses on one or more NICs on Virtual SAN hosts is not compatible with the rest of the cluster. Clears when all Virtual SAN hosts in the cluster have compatible multicast addresses. VirtualSANHostComponentCountAlert Automatic HostSystem Triggered when the number of Virtual SAN components on a Virtual SAN host exceeds 80 percent of the maximum supported components per host. Clears when the number of Virtual SAN components on a Virtual SAN host is below 80 percent of the maximum supported components per host. VMware, Inc. 21

22 vrealize Operations Management Pack for Storage Devices Guide Table 4 1. Management Pack for Storage Devices Alert Definitions (Continued) Alert (ID name) Alert Level Affected Object Type Description HostSystemMagneticDiskDegraded Critical HostSystem Triggered when the magnetic disk reports a problem via SMART status, when the bad sector count on the magnetic disk is high, when Virtual SAN reports degraded components on the magnetic disk or when the magnetic disk is unreachable from the host. Clears if the symptom disappears. MagneticDiskCapacityWarning Immediate VsanDisk Triggered when the Virtual SAN's usage of the magnetic disk on the host exceeds 80 percent of its capacity. Clears when the Virtual SAN's usage of the magnetic disk on the host is below 80 percent of its capacity. HostSystemSSDDegraded Automatic HostSystem Triggered when the SSD reports a problem by using the SMART status when the media wearout indicator count on the SSD is high or when the SSD is unreachable from the host. Clears if the symptom disappears. MagneticDiskReallocatedSectorsWarning Warning VsanDisk Triggered when the bad sector count on the magnetic disk is high. Clears if the symptom disappears. SSDMediaWearoutWarning Warning SSD Triggered when the media wearout indicator count on the SSD is high. Clears if the symptom disappears. VirtualSANHostHBAMissing Critical HostSystem Triggered when the previously discovered Virtual SAN storage controller on a Virtual SAN host is no longer accessible. Clears when the Virtual SAN storage controller is accessible. VirtualSANHostMisconfiguredStorageAlert Critical HostSystem Triggered when the Virtual SAN host provides no storage to the Virtual SAN datastore. Clears when the Virtual SAN host provides some storage to the Virtual SAN datastore. NetworkBandwidthCongestion Critical EsxPnic Triggered when the physical NICs on ESXi host are saturated with the network traffic. Clears when the physical NICs on ESXi host are not saturated with the network traffic. 22 VMware, Inc.

23 Chapter 4 Management Pack for Storage Devices Alert Definitions Table 4 1. Management Pack for Storage Devices Alert Definitions (Continued) Alert (ID name) Alert Level Affected Object Type Description spbmvsanprofileincompatible3 Critical VirtualMachine Triggered when the virtual machines are not compliant with their specified storage profile. Cleared when the virtual machines are compliant with their specified storage profile. VsanComponentsNotActiveForVM Warning VirtualMachine Triggered when some Virtual SAN components are not active for a virtual machine. This could indicate a possible performance degradation of the virtual machine. Clears when the inactive components on the virtual machine are removed or are active again. VMware, Inc. 23

24 vrealize Operations Management Pack for Storage Devices Guide 24 VMware, Inc.

25 Management Pack for Storage 5 Devices Metrics The Management Pack for Storage Devices collects metrics for objects. It also displays object properties in the vrealize Operations Manager user interface. This chapter includes the following topics: Statistical Metrics of the Management Pack for Storage Devices, on page 25 Object Properties, on page 33 Statistical Metrics of the Management Pack for Storage Devices The Management Pack for Storage Devices collects statistical metrics of objects. You can view these metrics from the vrealize Operations Manager user interface. From the vrealize Operations Manager user interface, click Environment > Storage Devices > Troubleshooting > All Metrics tab. Disk Group Metrics The Management Pack for Storage Devices collects metrics about disk groups. Table 5 1. Disk Group Metrics Metric Name DerivedStatistics.DiskGroupCapacity DerivedStatistics.DiskGroupDemand DerivedStatistics.DiskGroupUsage VsanStatistics.Disk Capacity (GB) User Interface Name Disk Group Capacity Disk Group Demand Disk Group Usage Disk Capacity (GB) VsanStatistics.Disk Capacity Used (%) Disk Capacity Used (%) VsanStatistics.Capacity Used (GB) Capacity Used (GB) ESXi Physical NIC Metrics The Management Pack for Storage Devices collects metrics about the physical Network Interface Cards (NICs) on the ESXi host. Table 5 2. ESXi Physical NIC Metrics Metric Name esx_pnic_statistics.packetsreceived esx_pnic_statistics.packetstransmitted User Interface Name Packets Received Packets Transmitted VMware, Inc. 25

26 vrealize Operations Management Pack for Storage Devices Guide Table 5 2. ESXi Physical NIC Metrics (Continued) Metric Name esx_pnic_statistics.received esx_pnic_statistics.transmitted esx_pnic_statistics.droppedreceived esx_pnic_statistics.droppedtransmitted esx_pnic_statistics.bytesreceived esx_pnic_statistics.bytestransmitted esx_pnic_statistics.broadcastreceived esx_pnic_statistics.broadcasttransmitted DerivedStatistics.Capacity DerivedStatistics.Demand DerivedStatistics.Usage DerivedStatistics.UsageCapacityPercent User Interface Name Received Transmitted Dropped Received Dropped Transmitted Bytes Received Bytes Transmitted Broadcast Received Broadcast Transmitted Capacity (MBps) Demand (MBps) Usage (MBps) Used Capacity Percent Host Adapter Metrics The Management Pack for Storage Devices collects metrics about the host adapters in the host system. Table 5 3. Host Adapter Metrics Metric Name User Interface Name Fiber Channel Metrics adapter_performance_metrics.outstandingios adapter_performance_metrics.queuedepth adapter_performance_metrics.queuelatency adapter_performance_metrics.queued adapter_performance_metrics.read ReadStatistics.BlocksRead ReadStatistics.ReadOperations WriteStatistics.BlocksWritten WriteStatistics.WriteOperations TaskManagementStatistics.Aborts TaskManagementStatistics.LunResets DerivedStatistics.CongestionRatio DerivedStatistics.IOPs DerivedStatistics.HostAdapterCapacity DerivedStatistics.HostAdapterDemand DerivedStatistics.HostAdapterUsage DerivedStatistics.HostAdapterInferredLatency AdapterErrorStatistics.FailedCommands AdapterErrorStatistics.FailedBlocksRead AdapterErrorStatistics.FailedBlocksWritten Storage Adapter Outstanding I/Os Storage Adapter Queue Depth Storage Adapter Queue Command Latency Storage Adapter Number Queued Reads Per Sec Blocks Read Per Second Read Operations Per Second Blocks Written Per Second Write Operations Per Second Aborts Per Minute Lun Resets Per Minute Congestion Ratio Aggregation of the read and write operations per second. Host Adapter Capacity (MBps) Host Adapter Demand (MBps) Host Adapter Usage (MBps) Input/Output latency observed on the host adapter. Failed Commands Failed Blocks Read Per Second Failed Blocks Written Per Second 26 VMware, Inc.

27 Chapter 5 Management Pack for Storage Devices Metrics Table 5 3. Host Adapter Metrics (Continued) Metric Name AdapterErrorStatistics.FailedReadOperations AdapterErrorStatistics.FailedWriteOperations AdapterErrorStatistics.LipCount AdapterErrorStatistics.ErrorFrames AdapterErrorCountPerMin.LinkFailureCount AdapterErrorCountPerMin.LossOfSignalCount AdapterErrorCountPerMin.InvalidTxWordCount AdapterErrorCountPerMin.InvalidCRCCount adapter_performance_metrics.commandsaveraged adapter_performance_metrics.numberreadaveraged adapter_performance_metrics.numberwriteaveraged adapter_performance_metrics.totalreadlatency adapter_performance_metrics.totalwritelatency adapter_performance_metrics.write FCStatistics.TxFrames FCStatistics.RxFrames User Interface Name Failed Read Operations Per Second Failed Write Operations Per Second Lip Count Error Frames Per Second Link Failure Count Loss Of Signal Count Invalid Tx Word Count Invalid CRC Count Commands Averaged Number Read Averaged Number Write Averaged Read latency Write latency Writes per sec Tx Frames Per Second Rx Frames Per Second Local Storage Controller Metrics DerivedStatistics.HostAdapterCapacity DerivedStatistics.HostAdapterDemand DerivedStatistics.HostAdapterUsage DerivedStatistics.HostAdapterInferredLatency Host Adapter Capacity (MBps) Host Adapter Demand (MBps) Host Adapter Usage (MBps) Input/Output latency observed on the host adapter. AdapterErrorStatistics and AdapterErrorCountPerMin metrics are displayed only in Fibre Channel (FC) and Fibre Channel over Ethernet (FCoE) environments. Magnetic Disk Metrics The Management Pack for Storage Devices collects magnetic disk metrics about the about the Virtual SAN datastores in your datacenter. Table 5 4. Magnetic Disk Metrics Metric Name VsanStatistics.failedCommands VsanStatistics.latency.totalTime User Interface Name Failed Commands Latency total time VsanStatistics.capacityUsed Disk Capacity Used (%) VMware, Inc. 27

28 vrealize Operations Management Pack for Storage Devices Guide Magnetic Disk, Solid State Device, and Disk Group Metrics The Management Pack for Storage Devices collects metrics about the magnetic disks, solid state devices, and disk groups on the ESXi hosts. Table 5 5. Magnetic Disk, Solid State Device, and Disk Group Metrics Metric Name busresets commandsaborted devicelatency devicereadlatency devicewritelatency numberread numberreadaveraged numberwrite numberwriteaveraged Read Rate scsireservationconflicts NOTE This metric is applicable only to fibre channel mounts. totallatency totalreadlatency totalwritelatency Write Rate HDDErrors IOPS NOTE This metric is applicable only to magnetic disks. User Interface Name Bus Resets Commands Aborted Device Latency Device Read Latency Device Write Latency Number Read Number Read Averaged Number Write Number Write Averaged Read rate SCSI Reservation Conflicts Total Latency Total Read Latency Total Write Latency Write Rate HDD Errors IOPS Magnetic Disk and Solid State Device Smart Metrics The Management Pack for Storage Devices collects SMART metrics about magnetic disks and solid state devices on the ESXi hosts. Table 5 6. Magnetic Disk and Solid State Device SMART Metrics Metric Name scsi_smart_statistics.healthstatus.value scsi_smart_statistics.mediawearoutindicator.value scsi_smart_statistics.mediawearoutindicator.threshold scsi_smart_statistics.writeerrorcount.value scsi_smart_statistics.writeerrorcount.threshold scsi_smart_statistics.readerrorcount.value scsi_smart_statistics.readerrorcount.threshold scsi_smart_statistics.poweronhours.value scsi_smart_statistics.reallocatedsectors.value User Interface Name Health Status Value Media Wearout Indicator Value Media Wearout Indicator Threshold Write Error Count Value Write Error Count Threshold Read Error Count Value Read Error Count Threshold Power on Hours value Re-allocated sectors value 28 VMware, Inc.

29 Chapter 5 Management Pack for Storage Devices Metrics Table 5 6. Magnetic Disk and Solid State Device SMART Metrics (Continued) Metric Name scsi_smart_statistics.reallocatedsectors.threshold scsi_smart_statistics.rawreaderrorrate.value scsi_smart_statistics.rawreaderrorrate.threshold scsi_smart_statistics.drivetemperature.value scsi_smart_statistics.drivetemperature.worst scsi_smart_statistics.drivetemperature.threshold scsi_smart_statistics.driveratedmaxtemperature.value scsi_smart_statistics.driveratedmaxtemperature.threshold scsi_smart_statistics.writesectorstotcount.value scsi_smart_statistics.writesectorstotcount.threshold scsi_smart_statistics.readsectorstotcount.value scsi_smart_statistics.readsectorstotcount.threshold scsi_smart_statistics.initialbadblockcount.value scsi_smart_statistics.initialbadblockcount.threshold User Interface Name Re-allocated sectors threshold Raw Read Error Rate Value Raw Read Error Rate Threshold Drive Temperature Value Drive Temperature Worst Drive Temperature Threshold Drive Rated Max Temperature Value Drive Rated Max Temperature Threshold Write Sectors TOT Count Value Write Sectors TOT Count Threshold Read Sectors TOT Count Value Read Sectors TOT Count Threshold Initial Bad Block Count Value Initial Bad Block Count Threshold Mount Metrics The Management Pack for Storage Devices collects metrics about a storage LUN abstraction that is available on the host system. Table 5 7. Mount Metrics Metric Name DeviceErrorStatistics.FailedCommands DeviceErrorStatistics.FailedReadOperations DeviceErrorStatistics.FailedWriteOperations DeviceErrorStatistics.FailedBlocksWritten ReadStatistics.BlocksRead ReadStatistics.ReadOperations WriteStatistics.BlocksWritten WriteStatistics.WriteOperations TaskManagementStatistics.Aborts TaskManagementStatistics.LunResets DerivedStatistics.CongestionRatio HostSpecificMetrics.PathHealthRatio DerivedStatistics.MountCapacity DerivedStatistics.MountDemand DerivedStatistics.MountUsage DerivedStatistics.MountInferredLatency DerivedStatistics.MountAggregatedErrors DerivedStatistics.IOPS User Interface Name Failed Commands Failed Read Operations Failed Write Operations Failed Blocks Written Blocks Read Read Operations Blocks Written Write Operations Task Management Statistics LUN Resets Ratio of the active commands to the queued commands on the mount. Path Health Ratio Mount Throughput Capacity (MBps) Mount Throughput Demand (MBps) Mount Throughput Usage (MBps) Mount Inferred Latency Mount errors IOPS VMware, Inc. 29

30 vrealize Operations Management Pack for Storage Devices Guide Table 5 7. Mount Metrics (Continued) Metric Name User Interface Name The following metrics are relevant only for ISCSI Mounts DiskStatistics.busResets DiskStatistics.capacity.contention DiskStatistics.capacity.provisioned DiskStatistics.capacity.usage DiskStatistics.commands DiskStatistics.commandsAborted DiskStatistics.commandsAveraged DiskStatistics.deviceLatency DiskStatistics.deviceReadLatency DiskStatistics.deviceWriteLatency DiskStatistics.maxQueueDepth DiskStatistics.maxTotalLatency DiskStatistics.numberRead DiskStatistics.numberWrite DiskStatistics.queueLatency DiskStatistics.read DiskStatistics.scsiReservationConflicts DiskStatistics.totalLatency DiskStatistics.write Bus resets Capacity contention on mount Capacity provisioned on mount Capacity used on mount Commands on mount Aborted Commands on mount Average number of commands on mount Latency Read Latency Write Latency Maximum Queue Depth on Mount Max latency seen on mount Number of reads Number of writes Queued Latency Read Throughput Number of SCSI Reservation Conflicts Kernel Latency + Device Latency Write Throughput NFS Volume Metrics The Management Pack for Storage Devices collects metrics about Network File System (NFS) volumes. Table 5 8. NFS Volume Metrics Metric Name DerivedStatistics.ThroughputCapacity DerivedStatistics.ThroughputUsage DerivedStatistics.InferredLatency NfsVolumePerformanceMetrics.datastoreIops NfsVolumePerformanceMetrics.datastoreWriteIops NfsVolumePerformanceMetrics.read NfsVolumePerformanceMetrics.write NfsVolumePerformanceMetrics.totalReadLatency NfsVolumePerformanceMetrics.datastoreReadBytes NfsVolumePerformanceMetrics.datastoreWriteBytes NfsVolumePerformanceMetrics.datastoreReadIops NfsVolumePerformanceMetrics.totalWriteLatency User Interface Name Throughput (MBps) Current Throughput (MBps) Inferred Latency Aggregate IOPS Writes per second Read Rate Write Rate Total Read Latency Bytes Read Bytes Written Reads per second Total Write Latency 30 VMware, Inc.

31 Chapter 5 Management Pack for Storage Devices Metrics Solid State Devices Metrics The Management Pack for Storage Devices collects solid state devices metrics about the about the Virtual SAN datastores in your datacenter. Table 5 9. Solid State Devices Metrics Metric Name VsanStatistics.failedCommands VsanStatistics.latency.totalTime User Interface Name Failed Commands Latency total time VsanStatistics.Read Cache Hit Rate (%) Read Cache Hit Rate (%) Storage LUN Metrics The Management Pack for Storage Devices collects metrics about the storage LUN of a storage array. Table Storage LUN Metrics Metric Name lun.percentused lun.luncapacity User Interface Name Percent Used LUN Capacity Switch Port Metrics The Management Pack for Storage Devices collects metrics about the port on a switch. Table Switch Port Metrics Metric Name switch_port_statistics.addresserrors switch_port_statistics.bytesreceived switch_port_statistics.bytestransmitted switch_port_statistics.crcerrors switch_port_statistics.delimitererrors switch_port_rate_statistics.rxframerate switch_port_rate_statistics.rxrate switch_port_rate_statistics.txframerate switch_port_rate_statistics.txrate DerivedStatistics.SwitchPortCapacity DerivedStatistics.SwitchPortUsage DerivedStatistics.AggregatedErrors User Interface Name Address Errors Bytes Received Bytes Transmitted CRC Errors Delimiter Errors Rx Frame Rate Rx Rate Tx Frame Rate Tx Rate Switch Port Capacity (MBps) Switch Port Capacity Aggregated switch port error count including address errors, CRC errors, and delimiter errors. VMware, Inc. 31

32 vrealize Operations Management Pack for Storage Devices Guide Virtual SAN Datastore (Performance Metrics) The Management Pack for Storage Devices collects performance metrics about the Virtual SAN datastores in your datacenter. Table Virtual SAN Datastore (Performance Metrics) Metric Name PerformanceStatistics.opIngressCongestionCount PerformanceStatistics.readCachedBytes PerformanceStatistics.readCachedCount PerformanceStatistics.writeCongestionSum PerformanceStatistics.recoveryWriteCount PerformanceStatistics.readCount PerformanceStatistics.recoveryWriteBytes PerformanceStatistics.writeBytes PerformanceStatistics.recoveryWriteCongestionSum PerformanceStatistics.prepareLimitRetryCount PerformanceStatistics.readCachedLatencyMaxUs PerformanceStatistics.readCongestionSum PerformanceStatistics.ioCount PerformanceStatistics.objIngressCongestionCount PerformanceStatistics.recoveryWriteLatencyMaxUs PerformanceStatistics.writeLatencyMaxUs PerformanceStatistics.writeCount PerformanceStatistics.readLatencyMaxUs PerformanceStatistics.readCachedCongestionSum User Interface Name opingresscongestioncount Total Read Cached Bytes Read Cached Count Write Congestion Sum Total Recovery Write Count Total Read Count Recovery Write Bytes Total Write Bytes Recovery Write Congestion Sum Prepare Limit Retry Count Maximum Read Cached Latency (ms) Total Read Congestion Total Input Output Count Object Ingress Congestion Count Maximum Recovery Write Latency (ms) Maximum Write Latency (ms) Total Write Count Maximum Read Latency (ms) Total Read Cached Congestion Virtual SAN Datastore Metrics The Management Pack for Storage Devices collects metrics about the Virtual SAN datastores in your datacenter. Table Virtual SAN Datastore Metrics Metric Name DerivedStatistics.totalCapacity DerivedStatistics.provisionedCapacity DerivedStatistics.usedCapacity DerivedStatistics.latency DerivedStatistics.throughput DerivedStatistics.maxThroughput DerivedStatistics.avgTotalCapacityPerDiskGroup DerivedStatistics.avgUsedCapacityPerDiskGroup DerivedStatistics.avgThroughputPerDiskGroup User Interface Name Total Capacity (MB) Demand (MB) Usage (MB) Inferred Latency Throughput (MBps) Maximum Throughput (MBps) Average Capacity Per Disk Group (GB) Average Used Capacity Per Disk Group (GB) AverageThroughput Per Disk Group (MBps) 32 VMware, Inc.

33 Chapter 5 Management Pack for Storage Devices Metrics Table Virtual SAN Datastore Metrics (Continued) Metric Name DerivedStatistics.avgMaxThroughputPerDiskGroup HostSpecificMetrics.readCachedLatency HostSpecificMetrics.readLatency HostSpecificMetrics.recoveryWriteLatency HostSpecificMetrics.writeLatency HostSpecificMetrics.PerHostThroughput HostSpecificMetrics.MaxPerHostThroughput User Interface Name Average Max Throughput Per Disk Group (MBps) Read Cached Latency (ms) Read Latency (ms) Recovery Write Latency (ms) Write Latency (ms) Throughput (KBps) Maximum Throughput (KBps) Virtual SAN Host Metrics The Management Pack for Storage Devices collects metrics about the Virtual SAN host in your datacenter. Table Virtual SAN Host Metrics Metric Name User Interface Name VsanStatistics.ComponentUsage Component usage (%) VsanStatistics.capacity VsanStatistics.totalUsedCapacity DerivedStatistics.ThroughputCapacity DerivedStatistics.ThroughputUsage Capacity (GB) Capacity Used (GB) Throughput (MBps) Current Throughput (MBps) Object Properties The Management Pack for Storage Devices displays object properties. You can view the object properties from the vrealize Operations Manager user interface. From the vrealize Operations Manager user interface, click Environment > Storage Devices > Analysis tab. Disk Group Properties The Management Pack for Storage Devices displays the disk group object properties. Table Disk Group Properties Name Type Display Name Description HostUuid String Host UUID The UUID of the host. ESXi Physical Network Interface Card Properties The Management Pack for Storage Devices displays the properties of the ESXi Physical Network Interface Card (NIC) object. Table ESXi Physical NIC Properties Name Type Display Name Description Link Speed long Link Speed The speed of the link. Link State string Link State Indicates the state of the link. VMware, Inc. 33