Technical Brief GRANITE SNMP ALARMS AND RECOMMENDED ACTIONS. Technical Brief. Riverbed Technical Marketing

Transcription

1 Technical Brief GRANITE SNMP ALARMS AND RECOMMENDED ACTIONS Technical Brief Riverbed Technical Marketing

2 2012 Riverbed Technology. All rights reserved. Riverbed, Cloud Steelhead, Granite, Interceptor, RiOS, Steelhead, Think Fast, Virtual Steelhead, Whitewater, Mazu, Cascade, Cascade Pilot, Shark, AirPcap, SkipWare, TurboCap, WinPcap, Wireshark, and Stingray are trademarks or registered trademarks of Riverbed Technology, Inc. in the United States and other countries. Riverbed and any Riverbed product or service name or logo used herein are trademarks of Riverbed Technology. All other trademarks used herein belong to their respective owners. The trademarks and logos displayed herein cannot be used without the prior written consent of Riverbed Technology or their respective owners. Akamai and the Akamai wave logo are registered trademarks of Akamai Technologies, Inc. SureRoute is a service mark of Akamai. Apple and Mac are registered trademarks of Apple, Incorporated in the United States and in other countries. Cisco is a registered trademark of Cisco Systems, Inc. and its affiliates in the United States and in other countries. EMC, Symmetrix, and SRDF are registered trademarks of EMC Corporation and its affiliates in the United States and in other countries. IBM, iseries, and AS/400 are registered trademarks of IBM Corporation and its affiliates in the United States and in other countries. Linux is a trademark of Linus Torvalds in the United States and in other countries. Microsoft, Windows, Vista, Outlook, and Internet Explorer are trademarks or registered trademarks of Microsoft Corporation in the United States and in other countries. Oracle and JInitiator are trademarks or registered trademarks of Oracle Corporation in the United States and in other countries. UNIX is a registered trademark in the United States and in other countries, exclusively licensed through X/Open Company, Ltd. VMware, ESX, ESXi are trademarks or registered trademarks of VMware, Incorporated in the United States and in other countries Riverbed Technology. All rights reserved. 1

3 Introduction This document lists the Riverbed Granite SNMP alarms and associated recommended actions. SNMP v1, SNMP v2, and SNMP v3 are supported, although some MIB items may only be accessible through SNMP v2. This document does not cover how Granite appliances can be managed and monitored with SNMP, SNMP traps and notification, but can be found in Steelhead SNMP and Notification Technical Brief This document moreover does not cover additional management tasks on the Steelhead appliance such as configuring scheduled jobs, setting-up log server notification, and upgrading software, but can be found in Chapter 8 of the Steelhead Management Console User s Guide. Information on managing the Steelhead via the CMC is also not covered here, but can be found in the CMC Users Guide. Audience This document may act as a guide for network administrators who are familiar with deploying Steelhead appliances and want to take advantage of SNMP and alerts for seamless integration into existing network management systems. Additional Resources For a complete list and the most current version of Riverbed documentation log in to the Riverbed Support Web site located at The Riverbed Knowledge Base is a database of known issues, how-to documents, system requirements, and common error messages. You can browse titles or search for key words and strings. To access the Riverbed Knowledge Base, log in to the Riverbed Support site located at Riverbed Technology. All rights reserved. 2

4 Granite SNMP Alarms and Recommended Actions The below tables list all of the Granite related SNMP alarms along with suggested priority, description, and recommended actions for both Granite Core and Granite Edge appliances. For completeness Storage Profile Switch Failed, VSP and RSP alarms are also listed. Granite Core Alarms Category Sub Alarm Priority Description Recommendation Granite Core Failover Failover 1 The peer Granite Core appliance in HA configuration has gone down. Check the other Granite Core for health and status. Granite Core Service Granite Core Service is Not Running 1 The Granite Core service is not running. Check the service page to ensure that the Granite Core service is running and has not been suspended/paused; check the logs for any system crashes. Edge Connectivity 3 Granite Core does not have an active connection with one of the Edges present in its configuration. Edge and the Granite Core; recheck the Edge configuration on the Granite Core for correctness. Granite Edge Inner Channel Down 3 Data channel between Granite Core and the Edge is down. Edge and the Granite Core. Keep-Alive Timeout 3 The connection between the Granite Core and the Edge has stalled. Edge and the Granite Core. SAN Unavailable 1 The Granite Core has lost connection with the backend SAN. Granite Core and the backend SAN. iscsi Reservation Unsupported 3 The backend SAN does not support iscsi reservations. Remove the lun or do not use iscsi reservations. Setting up HA is not recommended. Granite iscsi iscsi Reservation Lost 3 iscsi Reservation Preempted 3 Unexpected loss of iscsi reservation on the backend SAN. Unexpected loss of iscsi reservation since another initiator preempted it. Check the network connection with the backend SAN. Check heartbeat connections between Granite Core HA peers. Check heartbeat connections between Granite Core HA peers. iscsi Reservation delayed 3 Granite core is unable to secure an iscsi reservation on the backend SAN for this lun. Check the LUN initiator security on the backend SAN, if the Granite Core initiator is allowed. Lost iscsi IO Channel (MPIO enabled) Lost iscsi IO Channel (MPIO disabled) Granite LUN Unable to Optimize LUN An iscsi MPIO path to backend storage is down iscsi connection to backend storage is down Unable to accelerate a particular LUN. Check the network path as reported in the alarm. Check the network path as reported in the alarm. Incorrect sizing of Granite Core. Should be cleared once enough memory (used for representing inactive files/dir) is freed. Ideally the Granite Core should be sized properly Riverbed Technology. All rights reserved. 3

5 Category Sub Alarm Priority Description Recommendation Granite Snapshot Licensing Unable to Optimize LUN 3 Unable to Optimize VM 2 Unavailable LUN 1 High Read/Write Latency 3 VSS Error 2 Insufficient Space Reservation 2 Invalid Credentials 2 Granite Core License 1 Granite Model License 1 NEEDS ATTENTION Unable to accelerate a particular LUN. Detected corruption in VMFS and hence cannot optimize the LUN and any VM/VMDKs on it. Unable to accelerate a particular VM. A LUN has become unavailable. High read/write SAN latency detected VSS has errored during a snapshot operation Failed to commit a snapshot to SAN due to insufficient space Failed to commit a snapshot to SAN due to invalid credentials Unable to start Granite Core Service due to missing or expired license. Unable to start Granite Core Service due to missing or expired license. We currently are unaware of any fsck utility for VMFS. Although, we have observed that ESXi sometimes continues to function with a corrupted VMFS. If snapshots of the entire VMFS have been taken, one can recover the entire VMFS from it. The Granite Core logs will also indicate which VMFS file/dir is corrupted and such file/dir can also be individually restored from a LUN snapshot and/or VM snapshot. A VM has too many snapshots causing us to take a lot of time to build the metadata for it. Currently, this requires a service restart to take the VM out of the blacklist. In the coming releases, we ll have an automatic way to take the VM out of the blacklist. Check to see if a LUN has been offlined while IO operations were in progress. The SAN might be overloaded or the network between Granite Core & SAN might have high latency. The end user performance will be degraded The system encountered an unknown error. Check the logs for more information Clear older snapshots or allocate more space to the snapshot reserve Check credentials. Check your licenses for validity. Check your licenses for validity. Granite Edge Alarms Granite Blockstore Disk Space Low 2 Disk Space Full 1 Running out of space on the Blockstore. Out of disk space on the Blockstore. Check your WAN connection as well as connectivity to the Granite Core. This may also happen if clients write more data that can be sent over the WAN for a prolonged period of time. Check your WAN connection as well as connectivity to the Granite Core. This may also happen if clients write more data that can be sent over the WAN for a prolonged period of time Riverbed Technology. All rights reserved. 4

6 Granite Core Memory Low 3 Read Error 2 Critical Read Error 1 Startup Failed 1 Startup Wrong Version 1 Write Error 1 Unknown Edge 3 Granite Core Connectivity 3 Inner Channel Down 3 Keep-Alive Timeout 3 Running out of memory for the Blockstore. Could not read data that was already replicated to the DC. Clients won t see any error since Granite Edge will fetch the data from the DC. Could not read data that is not yet replicated to the DC. Blockstore failed to start due to disk errors or incorrect configuration. Granite Edge software version is incompatible with the Blockstore version on disk. Could not save data to disk due to media error. The Edge device has connected to a Granite Core that does not recognize the Edge device. Granite Edge does not have an active connection with the Granite Core. Data channel between Granite Core and the Edge is down. The connection between the Granite Core and the Edge has stalled. Temporary condition caused by too much IO. Limit the number of active prepop sessions. Check if IOPS is more than what is recommended for the model. Check system logs to determine the root cause. Replace any disks that have gone bad. This alarm will clear only on service restart. Check system logs to determine the root cause. Replace any disks that have gone bad. This alarm will clear only on service restart. Check system logs to determine the root cause. This resulted from upgrading or downgrading the software to an incompatible version. Revert to the software version that was running before. Check system logs for the root cause. Replace any disks that have gone bad. This alarm will clear only on service restart. Most likely missing an entry for the Edge inside the configuration present on the Granite Core; check that the Edge is supplying the proper Edge ID by looking at the Branch Storage configuration on the Edge device. Edge and the Granite Core; recheck the Edge configuration on the Granite Core for correctness. Edge and the Granite Core. Edge and the Granite Core. Granite Edge Service Granite Service is Not Running 1 The Granite service on the Edge is not running. Check the logs for service crash. Snapshot Error 2 Granite Snapshot Timeout 3 Granite LUN Unavailable LUN 1 Granite Uncommitted Data Uncommitted Data 3 Snapshot has failed to be committed to the SAN Failed to complete a snapshot to due to Windows timing out A LUN has become unavailable. The Edge has a high amount of uncommitted data. Check The Granite Core logs for details. Retry the windows snapshot Check to see if a LUN has been offlined while IO operations were in progress. Check your WAN connection as well as connectivity to the Granite Core Riverbed Technology. All rights reserved. 5

7 Storage Profile Switch Failed Alarm Storage Profile Switch Failed Storage profile switch failed 1 or 3 Either Critical or Needs Attention, depending on: 1 If RiOS is unable to revert the Steelhead appliance to the previous storage profile, the alarm status becomes critical. 3 If RiOS successfully reverts the Steelhead appliance to the previous storage profile, the alarm status displays needs attention. An error has occurred while repartitioning the disk drives during a storage profile switch. When you encounter this error, try to switch the storage profile again. If the switch succeeds, the error clears. If it fails, RiOS reverts the Steelhead appliance to the previous storage profile. VSP Alarms (For Steelhead-EX release 1.0.2) VSP Service Error 1 VSP service is not running Check logs for service crash Memory allocated to slots is overcommitted Reduce the slot memory allocation VSP General Error Watchdog failed on slot(s) 2 Verify the slot watchdog configuration and check logs cause of watchdog failure Slot enabled but not powered Verify slot is not powered off by the guest VM, check the logs for cause of the slot powered failure. RSP Alarms (For Steelhead release 7.0.2) RSP Service Error RSP General Error Installed image version is not compatible with this RiOS release Memory allocated to slots is overcommitted Watchdog failed on slot(s) 1 2 RSP service is not running Check logs for service crash Install the compatible RSP image and re-enable RSP service Reduce the slot memory allocation Verify the slot watchdog configuration and check logs cause of watchdog failure Slot enabled but not powered Verify slot is not powered off by the guest VM, check the logs for cause of the slot powered failure Riverbed Technology. All rights reserved. 6

8 About Riverbed Riverbed delivers performance for the globally connected enterprise. With Riverbed, enterprises can successfully and intelligently implement strategic initiatives such as virtualization, consolidation, cloud computing, and disaster recovery without fear of compromising performance. By giving enterprises the platform they need to understand, optimize and consolidate their IT, Riverbed helps enterprises to build a fast, fluid and dynamic IT architecture that aligns with the business needs of the organization. Additional information about Riverbed (NASDAQ: RVBD) is available at Riverbed Technology, Inc. 199 Fremont Street San Francisco, CA Tel: (415) Riverbed Technology Ltd. One Thames Valley Wokingham Road, Level 2 Bracknell. RG42 1NG United Kingdom Tel: Riverbed Technology Pte. Ltd. 391A Orchard Road #22-06/10 Ngee Ann City Tower A Singapore Tel: Riverbed Technology K.K. Shiba-Koen Plaza Building 9F 3-6-9, Shiba, Minato-ku Tokyo, Japan Tel: Riverbed Technology. All rights reserved. 7