Cisco MDS 9000 Family SAN Volume Controller Configuration Guide. Cisco MDS SAN-OS Release 1.3 September, 2004

Transcription

1 Cisco MDS 9000 Family SAN Volume Controller Configuration Guide Cisco MDS SAN-OS Release 1.3 September, 2004 Corporate Headquarters Cisco Systems, Inc. 170 West Tasman Drive San Jose, CA USA Tel: NETS (6387) Fax: Customer Order Number: DOC = Text Part Number:

2 THE SPECIFICATIONS AND INFORMATION REGARDING THE PRODUCTS IN THIS MANUAL ARE SUBJECT TO CHANGE WITHOUT NOTICE. ALL STATEMENTS, INFORMATION, AND RECOMMENDATIONS IN THIS MANUAL ARE BELIEVED TO BE ACCURATE BUT ARE PRESENTED WITHOUT WARRANTY OF ANY KIND, EXPRESS OR IMPLIED. USERS MUST TAKE FULL RESPONSIBILITY FOR THEIR APPLICATION OF ANY PRODUCTS. THE SOFTWARE LICENSE AND LIMITED WARRANTY FOR THE ACCOMPANYING PRODUCT ARE SET FORTH IN THE INFORMATION PACKET THAT SHIPPED WITH THE PRODUCT AND ARE INCORPORATED HEREIN BY THIS REFERENCE. IF YOU ARE UNABLE TO LOCATE THE SOFTWARE LICENSE OR LIMITED WARRANTY, CONTACT YOUR CISCO REPRESENTATIVE FOR A COPY. The Cisco implementation of TCP header compression is an adaptation of a program developed by the University of California, Berkeley (UCB) as part of UCB s public domain version of the UNIX operating system. All rights reserved. Copyright 1981, Regents of the University of California. NOTWITHSTANDING ANY OTHER WARRANTY HEREIN, ALL DOCUMENT FILES AND SOFTWARE OF THESE SUPPLIERS ARE PROVIDED AS IS WITH ALL FAULTS. CISCO AND THE ABOVE-NAMED SUPPLIERS DISCLAIM ALL WARRANTIES, EXPRESSED OR IMPLIED, INCLUDING, WITHOUT LIMITATION, THOSE OF MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT OR ARISING FROM A COURSE OF DEALING, USAGE, OR TRADE PRACTICE. IN NO EVENT SHALL CISCO OR ITS SUPPLIERS BE LIABLE FOR ANY INDIRECT, SPECIAL, CONSEQUENTIAL, OR INCIDENTAL DAMAGES, INCLUDING, WITHOUT LIMITATION, LOST PROFITS OR LOSS OR DAMAGE TO DATA ARISING OUT OF THE USE OR INABILITY TO USE THIS MANUAL, EVEN IF CISCO OR ITS SUPPLIERS HAVE BEEN ADVISED OF THE POSSIBILITY OF SUCH DAMAGES. CCIP, CCSP, the Cisco Arrow logo, the Cisco Powered Network mark, Cisco Unity, Follow Me Browsing, FormShare, and StackWise are trademarks of Cisco Systems, Inc.; Changing the Way We Work, Live, Play, and Learn, and iquick Study are service marks of Cisco Systems, Inc.; and Aironet, ASIST, BPX, Catalyst, CCDA, CCDP, CCIE, CCNA, CCNP, Cisco, the Cisco Certified Internetwork Expert logo, Cisco IOS, the Cisco IOS logo, Cisco Press, Cisco Systems, Cisco Systems Capital, the Cisco Systems logo, Empowering the Internet Generation, Enterprise/Solver, EtherChannel, EtherSwitch, Fast Step, GigaStack, Internet Quotient, IOS, IP/TV, iq Expertise, the iq logo, iq Net Readiness Scorecard, LightStream, MGX, MICA, the Networkers logo, Networking Academy, Network Registrar, Packet, PIX, Post-Routing, Pre-Routing, RateMUX, Registrar, ScriptShare, SlideCast, SMARTnet, StrataView Plus, Stratm, SwitchProbe, TeleRouter, The Fastest Way to Increase Your Internet Quotient, TransPath, and VCO are registered trademarks of Cisco Systems, Inc. and/or its affiliates in the U.S. and certain other countries. All other trademarks mentioned in this document or Web site are the property of their respective owners. The use of the word partner does not imply a partnership relationship between Cisco and any other company. (0304R) Cisco MDS 9000 Family SAN Volume Conroller Configuration Guide Copyright 2003 Cisco Systems, Inc. All rights reserved.

3 New and Changed Information Table 1 summarizes the new and changed features for the Cisco MDS 9000 Family SAN Volume Controller Configuration Guide, and tells you where they are documented. If a feature has changed in Release 1.3, a brief description of the change appears in the Description column, and that release is shown in the Changed in Release column. Table 1 Documented Features for the Cisco MDS 9000 Family SAN Volume Controller Configuration Guide Feature Dual fabric configuration CSM modules Quorum configuration SVC configuration Description CSM modules in combination with the Inter-VSAN Routing (IVR) feature operate independently across two isolated fabrics. The SVC functionality is only available when CSM modules are present in the switch. Using the qorum command to overwrite the system-assigned quorum disk and pick a particular set of managed disks to be a quorum disk. Introduction of SVC configuration using Cisco MDS switches Changed in Release 1.3(5m) 1.3(5m) 1.3(5m) Where Documented Configuring a Dual Fabric SAN Environment Getting Started Managing Back-End Storage 1.3(1) This guide Table 2 contains the history of the changes to the Cisco MDS 9000 Family SAN Volume Controller Configuration Guide, Release 1.3. When the document is updated for the next release, these changes are incorporated into the new revision and will no longer appear in this table. Table 2 Documentation Changes for Cisco MDS 9000 Family SAN Volume Controller Configuration Guide, Release 1.3 Date Description of Change Where Changed 11/21/2003 Document created /19/2004 Configuring iscsi Hosts sample procedure added See Chapter 6, Configuring Hosts iii

4 New and Changed Information iv

5 CONTENTS New and Changed Information iii Preface ix Audience ix Organization ix Document Conventions x Related Documentation xi Obtaining Documentation xi Cisco.com xi Documentation CD-ROM xii Ordering Documentation xii Documentation Feedback xii Obtaining Technical Assistance xiii Cisco.com xiii Technical Assistance Center xiii Cisco TAC Website xiv Cisco TAC Escalation Center xiv Obtaining Additional Publications and Information xiv CHAPTER 1 SVC Product Overview 1-1 About SVC 1-1 SVC Features 1-2 CHAPTER 2 Getting Started 2-1 Preparing the Cisco MDS Switch 2-2 Setting Up the Cisco MDS Switch 2-2 Setting Up a New Cisco MDS Switch 2-2 Updating an Existing Cisco MDS Switch 2-3 SVC Role Authorization 2-4 Understanding SVC Terminology 2-5 Node 2-5 I/O group 2-5 Cluster 2-5 MDisk 2-5 v

6 Contents VDisk 2-5 MDisk Group 2-6 Host 2-6 Separating Hosts and Storage Devices 2-6 Verifying Interface Connectivity 2-8 Assigning VSAN Numbers 2-9 Multiple Initiators and Targets 2-9 CHAPTER 3 Creating and Managing Clusters 3-1 About CSM Nodes 3-2 About Clusters 3-2 Physical Topology 3-2 Selecting Nodes for a Cluster 3-3 Isolating Management Traffic 3-4 Creating a Cluster 3-5 Adding Nodes to Clusters 3-6 Verifying Nodes in a Cluster 3-7 Deleting a node from a Cluster 3-8 Deleting a Cluster 3-9 CHAPTER 4 Managing Back-End Storage 4-1 About Managed Disks 4-2 About MDisk Groups 4-2 Extents 4-2 MDisk Modes 4-2 Isolating Back-end Storage Traffic 4-3 Verifying Traffic Isolation 4-4 Configuring LUN Masking 4-4 Identifying MDisks 4-5 Configuring MDisk Groups 4-5 Configuring Quorum Disks 4-7 CHAPTER 5 Managing Virtual Disks 5-1 Virtualization Policies 5-2 Striped 5-2 Sequential 5-2 vi

7 Contents Image 5-2 Licensing Requirements 5-2 Configuring VDisks 5-3 CHAPTER 6 Configuring Hosts 6-1 About Hosts 6-1 Isolating Host Traffic 6-1 Creating Hosts 6-4 Mapping VDisks to Hosts 6-4 Configuring iscsi Hosts in SVC 6-5 CHAPTER 7 Configuring Copy Services 7-1 Data Migration 7-2 Data Migration Application 7-2 Data Migration Operation 7-2 Migrating VDisks between MDisk groups 7-2 FlashCopy 7-4 FlashCopy Applications 7-4 FlashCopy Mapping 7-4 FlashCopy Consistency Groups 7-5 Starting the FlashCopy 7-6 Stopping FlashCopy 7-7 Remote Copy 7-8 Disaster Recovery 7-8 Remote Copy Relationships 7-8 Remote Copy Consistency Groups 7-8 Configuring Remote Copy 7-9 Starting Remote Copy 7-10 Stopping Remote Copy 7-11 Failover and Recovery Process 7-12 CHAPTER 8 Upgrading CSM Software 8-1 Upgrading Clusters 8-2 Cluster Upgrade Prerequisites 8-2 Recognizing Failure Situations 8-2 Cluster Upgrade Guidelines 8-3 Performing the Cluster Upgrade 8-3 Sample Cluster Upgrade 8-4 vii

8 Contents Automatic Upgrade during Node Addition 8-5 Upgrading in Service Mode 8-5 Sample Service Mode Upgrade 8-6 Upgrading the Switch Software 8-7 Performing the Switch Upgrade 8-7 Sample Switch Upgrade 8-7 Replacing CSMs 8-9 Prerequisites to Replacing a CSM 8-10 Replacing a CSM in the Same Slot 8-10 Replacing a CSM in a Different Slot 8-11 Post-Replacement Verification 8-12 CHAPTER 9 Configuring SPAN on SVC Interfaces 9-1 About SVC as a SPAN Source 9-2 Configuring SVC Interfaces as SPAN Sources 9-2 Using VSAN Filters to Specify Traffic 9-3 CHAPTER 10 Configuring a Dual Fabric SAN Environment 10-1 Overview 10-2 Basic SVC Requirements 10-2 Dual Fabric Prerequisites 10-2 Sample Configuration 10-2 viii

9 Preface This preface describes the audience, organization, and conventions of the Cisco MDS 9000 Family Configuration Guide. It also provides information on how to obtain related documentation. Audience This Cisco MDS 9000 Family SAN Volume Controller User Guide is for system administrators who configure and maintain network and storage systems using the Cisco CSM and the SAN-OS Command-Line Interface (CLI). The information contained in this guide assumes that you have a: Basic understanding of: multilayer switches and related hardware system, storage, and some IP network administration Working knowledge using a host-based volume management tool General understanding of volume management in a SAN environment Working knowledge of the MDS 9000 Family switch administration and configuration Organization This guide is organized as follows: Chapter Title Description Chapter 1 Product Overview Presents an overview of the Cisco MDS 9000 Family of multilayer switches and directors. Chapter 2 Getting Started Explains the tasks required to set up and configure each building block of your system to run SVC for Switches. Chapter 3 Creating and Managing Clusters Provides details on the cluster-creation process. Chapter 4 Managing Back-End Storage Provides details on managed disks, managed disk groups, and the back-end traffic isolation process. ix

10 Document Conventions Preface Chapter Title Description Chapter 5 Managing Virtual Disks Explains the virtualization policies and provides details on the virtual disk configuration process. Chapter 6 Configuring Hosts Provides detail on isolating host traffic, creating hosts, and mapping VDisks to hosts. Chapter 7 Configuring Copy Services Discusses the copy services, including data migration, FlashCopy, and Remote Copy available in Cisco MDS 9216 Switches and in Cisco MDS 9500 Directors. Chapter 8 Upgrading CSM Software Explains the procedures and processes for the different upgrade possibilities when a Caching Services Module (CSM) is used. Document Conventions Command descriptions use these conventions: boldface font Commands and keywords are in boldface. italic font Arguments for which you supply values are in italics. [ ] Elements in square brackets are optional. [ x y z ] Optional alternative keywords are grouped in brackets and separated by vertical bars. Screen examples use these conventions: screen font Terminal sessions and information the switch displays are in screen font. boldface screen font Information you must enter is in boldface screen font. italic screen font Arguments for which you supply values are in italic screen font. < > Nonprinting characters, such as passwords are in angle brackets. [ ] Default responses to system prompts are in square brackets.!, # An exclamation point (!) or a pound sign (#) at the beginning of a line of code indicates a comment line. This document uses the following conventions: Note Means reader take note. Notes contain helpful suggestions or references to material not covered in the manual. Caution Means reader be careful. In this situation, you might do something that could result in equipment damage or loss of data. x

11 Preface Related Documentation Related Documentation The documentation set for the Cisco MDS 9000 Family includes the following documents: Cisco MDS 9000 Family Release Notes for Cisco MDS SAN-OS Releases Cisco MDS 9000 Family Interoperability Support Matrix Cisco MDS SAN-OS Release Compatibility Matrix for IBM SAN Volume Controller Software for Cisco MDS 9000 Regulatory Compliance and Safety Information for the Cisco MDS 9000 Family Cisco MDS 9500 Series Hardware Installation Guide Cisco MDS 9216 Switch Hardware Installation Guide Cisco MDS 9100 Series Hardware Installation Guide Cisco MDS 9000 Family Configuration Guide Cisco MDS 9000 Family Command Reference Cisco MDS 9000 Family Fabric Manager Configuration Guide Cisco MDS 9000 Family MIB Quick Reference Cisco MDS 9000 Family CIM Programming Reference Guide Cisco MDS 9000 Family System Messages Guide Cisco MDS 9000 Family Troubleshooting Guide Cisco MDS 9000 Family Port Analyzer Adapter 2 Installation and Configuration Note Cisco MDS 9000 Family Port Analyzer Adapter Installation and Configuration Note For information on IBM TotalStorage SAN Volume Controller Storage Software for the Cisco MDS 9000 Family, refer to the IBM TotalStorage Support website: Obtaining Documentation Cisco provides several ways to obtain documentation, technical assistance, and other technical resources. These sections explain how to obtain technical information from Cisco Systems. Cisco.com You can access the most current Cisco documentation on the World Wide Web at this URL: You can access the Cisco website at this URL: International Cisco websites can be accessed from this URL: xi

12 Obtaining Documentation Preface Documentation CD-ROM Cisco documentation and additional literature are available in a Cisco Documentation CD-ROM package, which may have shipped with your product. The Documentation CD-ROM is updated regularly and may be more current than printed documentation. The CD-ROM package is available as a single unit or through an annual or quarterly subscription. Registered Cisco.com users can order a single Documentation CD-ROM (product number DOC-CONDOCCD=) through the Cisco Ordering tool: All users can order monthly or quarterly subscriptions through the online Subscription Store: Ordering Documentation You can find instructions for ordering documentation at this URL: You can order Cisco documentation in these ways: Registered Cisco.com users (Cisco direct customers) can order Cisco product documentation from the Networking Products MarketPlace: Nonregistered Cisco.com users can order documentation through a local account representative by calling Cisco Systems Corporate Headquarters (California, U.S.A.) at or, elsewhere in North America, by calling NETS (6387). Documentation Feedback You can submit comments electronically on Cisco.com. On the Cisco Documentation home page, click Feedback at the top of the page. You can your comments to mdsfeedback-doc@cisco.com. You can submit comments by using the response card (if present) behind the front cover of your document or by writing to the following address: Cisco Systems Attn: Customer Document Ordering 170 West Tasman Drive San Jose, CA We appreciate your comments. xii

13 Preface Obtaining Technical Assistance Obtaining Technical Assistance Note If you purchased this product through a Cisco reseller, contact the reseller directly for technical support. If you purchased this product directly from Cisco, contact Cisco Technical Support at this URL: Cisco provides Cisco.com, which includes the Cisco Technical Assistance Center (TAC) website, as a starting point for all technical assistance. Customers and partners can obtain online documentation, troubleshooting tips, and sample configurations from the Cisco TAC website. Cisco.com registered users have complete access to the technical support resources on the Cisco TAC website, including TAC tools and utilities. Cisco.com Cisco.com offers a suite of interactive, networked services that let you access Cisco information, networking solutions, services, programs, and resources at any time, from anywhere in the world. Cisco.com provides a broad range of features and services to help you with these tasks: Streamline business processes and improve productivity Resolve technical issues with online support Download and test software packages Order Cisco learning materials and merchandise Register for online skill assessment, training, and certification programs To obtain customized information and service, you can self-register on Cisco.com at this URL: Technical Assistance Center The Cisco TAC is available to all customers who need technical assistance with a Cisco product, technology, or solution. Two types of support are available: the Cisco TAC website and the Cisco TAC Escalation Center. The type of support that you choose depends on the priority of the problem and the conditions stated in service contracts, when applicable. We categorize Cisco TAC inquiries according to urgency: Priority level 4 (P4) You need information or assistance concerning Cisco product capabilities, product installation, or basic product configuration. There is little or no impact to your business operations. Priority level 3 (P3) Operational performance of the network is impaired, but most business operations remain functional. You and Cisco are willing to commit resources during normal business hours to restore service to satisfactory levels. Priority level 2 (P2) Operation of an existing network is severely degraded, or significant aspects of your business operations are negatively impacted by inadequate performance of Cisco products. You and Cisco will commit full-time resources during normal business hours to resolve the situation. xiii

14 Obtaining Additional Publications and Information Preface Priority level 1 (P1) An existing network is down, or there is a critical impact to your business operations. You and Cisco will commit all necessary resources around the clock to resolve the situation. Cisco TAC Website Cisco TAC Escalation Center The Cisco TAC website provides online documents and tools to help troubleshoot and resolve technical issues with Cisco products and technologies. To access the Cisco TAC website, go to this URL: All customers, partners, and resellers who have a valid Cisco service contract have complete access to the technical support resources on the Cisco TAC website. Some services on the Cisco TAC website require a Cisco.com login ID and password. If you have a valid service contract but do not have a login ID or password, go to this URL to register: If you are a Cisco.com registered user, and you cannot resolve your technical issues by using the Cisco TAC website, you can open a case online at this URL: If you have Internet access, we recommend that you open P3 and P4 cases online so that you can fully describe the situation and attach any necessary files. The Cisco TAC Escalation Center addresses priority level 1 or priority level 2 issues. These classifications are assigned when severe network degradation significantly impacts business operations. When you contact the TAC Escalation Center with a P1 or P2 problem, a Cisco TAC engineer automatically opens a case. To obtain a directory of toll-free Cisco TAC telephone numbers for your country, go to this URL: Before calling, please check with your network operations center to determine the Cisco support services to which your company is entitled: for example, SMARTnet, SMARTnet Onsite, or Network Supported Accounts (NSA). When you call the center, please have available your service agreement number and your product serial number. Obtaining Additional Publications and Information Information about Cisco products, technologies, and network solutions is available from various online and printed sources. The Cisco Product Catalog describes the networking products offered by Cisco Systems, as well as ordering and customer support services. Access the Cisco Product Catalog at this URL: Cisco Press publishes a wide range of networking publications. Cisco suggests these titles for new and experienced users: Internetworking Terms and Acronyms Dictionary, Internetworking Technology Handbook, Internetworking Troubleshooting Guide, and the Internetworking Design Guide. For current Cisco Press titles and other information, go to Cisco Press online at this URL: xiv

15 Preface Obtaining Additional Publications and Information Packet magazine is the Cisco quarterly publication that provides the latest networking trends, technology breakthroughs, and Cisco products and solutions to help industry professionals get the most from their networking investment. Included are networking deployment and troubleshooting tips, configuration examples, customer case studies, tutorials and training, certification information, and links to numerous in-depth online resources. You can access Packet magazine at this URL: iq Magazine is the Cisco bimonthly publication that delivers the latest information about Internet business strategies for executives. You can access iq Magazine at this URL: Internet Protocol Journal is a quarterly journal published by Cisco Systems for engineering professionals involved in designing, developing, and operating public and private internets and intranets. You can access the Internet Protocol Journal at this URL: Training Cisco offers world-class networking training. Current offerings in network training are listed at this URL: xv

16 Obtaining Additional Publications and Information Preface xvi

17 CHAPTER 1 SVC Product Overview The Cisco MDS 9000 Family SAN Volume Controller User Guide provides information on how to set up and configure the SAN Volume Controller (SVC) storage software using a Cisco MDS 9000 Family Caching Services Module (CSM). For more information about IBM TotalStorage TM SAN Volume Controller (SVC) Storage Software, refer to the IBM TotalStorage SAN Volume Controller Storage Software for Cisco MDS 9000 Configuration Guide or the IBM TotalStorage SAN Volume Controller Storage Software for Cisco MDS 9000 Command-Line Interface User's Guide. This chapter describes the SVC and explains the supported features. It includes the following sections: About SVC, page 1-1 SVC Features, page 1-2 About SVC IBM and Cisco combine the IBM TotalStorage TM SAN Volume Controller (SVC) Storage Software with the Cisco MDS 9000 Caching Services Module (CSM) to reduce complexity and to reduce the cost of managing SAN-based storage. This solution implements a cache-based, clustered architecture and provides a highly available, scalable alternative that is necessary in today s demanding storage environments. The combined SVC storage software along with the Cisco MDS 9000 Series CSM is delivered as a feature of the Cisco MDS 9000 Family. The SVC software runs on a clustered pair of CSMs within the switch. Based on virtualization technology, this solution is designed to support a virtualized pool of storage from the storage subsystems attached to a SAN. It manges the combined storage volumes from a central point, avoids downtime for planned outages, increases capacity utilization, and implements copy services from a single license across multiple storage devices. This storage pool helps tap unused storage capacity by increasing efficiency. It is designed as an integrated solution supporting high performance and continuous availability in open-system environments. Storage volumes are represented to applications as virtual disks (VDisks). These VDisks are created from the pool of managed disks residing behind the storage engines. Storage administrators can scale performance by adding storage engines and scale capacity by adding disks to the managed storage pool. 1-1

18 SVC Features Chapter 1 SVC Product Overview The IBM Cisco SVC storage solution offers the following benefits and advantages: Reduces complexity Lowers the cost of managing SAN-based storage Creates a single pool of storage from disparate storage devices to increase capacity utilization Easier to manage Implementing a cache-based, clustered architecture to provide a highly available solution. Provides the scalability and performance required in today s demanding storage environments. Provides simple migration of storage. Provides a single set of copy services. SVC Features This section explains the features provided by the combined SVC storage software and the Cisco MDS 9000 Series CSM solution. A central point for volume management control Through virtualization, the Cisco MDS 9000 Family s SAN-OS software helps create pools of managed disks spanning multiple storage subsystems. These managed disks are mapped to virtual disks used by server applications thus making better use of existing storage. This simple interface incorporates the Storage Management Initiative Specification (SMIS) application programming interface (API), and further demonstrates IBM s focus on open standards. Dynamic data migration The Cisco MDS 9000 SAN-OS software includes a dynamic data-migration function that helps administrators migrate storage from one device to another, without taking it offline. This allows administrators to reallocate and scale storage capacity without disrupting applications. And the solution supports both local area network (LAN) free and server-free backups while a clustered configuration designed to support high availability allows for non-disruptive software upgrades. SVC for Cisco MDS 9000 also leverages the IBM TotalStorage Enterprise Storage Server multipathing software. Improved resource utilization This solution enables more efficient use of personnel and technology resources. It helps increase administrator productivity by empowering central management of volumes under disparate storage controllers from a single user interface. It also increases the amount of available storage capacity by pooling storage across multiple devices. Designed to manage up to two petabytes (PB) of total usable storage capacity, SVC for Cisco MDS 9000 will support even higher performance by adding storage engine pairs. All storage engines within a cluster jointly manage the entire capacity of a storage pool. Advanced copy services With conventional SAN disk arrays, copy operations are limited to in-box or like-box-to-like-box environments. But the SVC software moves copy services from individual storage controllers to the SAN. Administrators can apply copy services across disparate storage devices within the network. Advanced copy services such as FlashCopy and Peer-to-Peer Remote Copy (PPRC) are supported across the managed storage. 1-2

19 Chapter 1 SVC Product Overview SVC Features Cisco MDS 9000 CSM The CSM integrates two high performance processing nodes which, when combined with the Cisco MDS 9000 SAN-OS software, delivers network hosted virtualization and replication services. Each CSM includes 8 GB of local cache used to hold recently accessed data blocks. On-board dual redundant batteries and hard disk drives protect cached data in the event of a power failure. To further ensure data availability and integrity, nodes are paired with nodes on other CSMs in high availability clusters. Distributed cache-based virtualization The Cisco MDS 9000 CSM virtualization architecture overcomes the inherent bottlenecks associated with other virtualization architectures. Virtualization performance can be easily scaled to the level required by adding CSMs. Application I/O response time is improved over other virtualization architectures through the use of local caching of disk blocks. Because Cisco MDS 9000 virtualization is switch-based, any host can access any virtual volume from anywhere in the fabric, independent of the host s attachment point in the SAN. In addition to virtualization and replication services, the Cisco MDS 9000 CSM leverages all of the advanced SAN-OS features available on the Cisco MDS 9000 platform, simplifying security, diagnostics and management. Fabric-based virtualization using intelligent networking services Cisco MDS 9000 fabric-based virtualization provides a level of integration with intelligent SAN services unavailable to host or virtualization appliance-based solutions. The Cisco MDS 9000 platform with integrated CSM delivers the intelligence and advanced features required to make multilayer intelligent storage area networks a reality including hardware-enabled innovations designed to dramatically improve scalability, availability, security, and manageability of storage networks, resulting in increased utility and lower total cost of ownership (TCO). Virtual SAN Virtual SANs (VSANs) allow more efficient SAN utilization by creating hardware-based isolated environments within a single SAN fabric. Each VSAN maintains its own fabric services for added scalability and resilience. Zoning within a SAN allows additional configuration flexibility. VSANs allow the cost of SAN infrastructure to be shared among more users, while assuring absolute segregation and security of traffic and retaining independent control of configuration on a VSAN-by-VSAN basis. VSANs provide a protective barrier between application hosts and physical storage, enhancing data integrity in a virtualized storage environment. Comprehensive security The Cisco CSM and the SAN-OS software integrate seamlessly to apply extensive security measures at all possible points of attack. SSH, RADIUS, SNMPv3, and role-based access control are used to block unauthorized access. To guard against compromising control traffic, Fibre Channel Security Protocol (FC-SP) provides confidentiality, data origin authentication, and connectionless integrity across the fabric. Data plane traffic is secured with VSANs, guaranteeing segregation of traffic across shared fabrics, and with zoning to satisfy traffic segregation requirements within a VSAN. Hardware-based ACLs provide further granularity for advanced security options. High availability Like all other Cisco MDS 9000 Family modules, the CSM is hot-swappable and fully integrates into the Cisco MDS 9000 high availability architecture. The Cisco MDS 9000 SAN-OS software architecture offers an unparalleled level of availability including automatic restart of failed supervisor processes and fabric level availability via Cisco PortChannel capability which allows 16 physical links to be aggregated into one logical interface. The logical interface remains active in the event of a port, ASIC, or module failure, and can sustain the failure of any physical link without causing a reset. Additionally, Fabric Shortest Path First (FSPF) multipathing provides the 1-3

20 SVC Features Chapter 1 SVC Product Overview intelligence to load balance across up to 16 equal cost paths and, in the event of a switch failure, to dynamically reroute traffic. When deployed in clustered pairs and combined with SAN-OS software, availability is extended to the volume level ensuring maximum uptime. Management options The Cisco MDS 9000 Family CSM provides three principal modes of management of your virtual storage environment: Cisco MDS 9000 Family Command Line Interface (CLI), IBM s SVC for Cisco MDS 9000 CLI, and IBM s ICAT management GUI. For users who prefer a common interface for both SAN and Volume management, the Cisco SAN-OS CLI includes the full suite of capabilities necessary to manage your virtual storage environment from the SAN-OS command line. 1-4

21 CHAPTER 2 Getting Started The chapter explains the tasks required to set up and configure each building block of your system to run SVC for Switches. You will be performing these tasks using one or more Cisco MDS 9000 Family switches for setup and configuration. You may wish to have your Cisco and IBM documentation handy for reference to more detailed procedures. This chapter includes the following sections: Preparing the Cisco MDS Switch, page 2-2 Setting Up the Cisco MDS Switch, page 2-2 Understanding SVC Terminology, page 2-5 Separating Hosts and Storage Devices, page 2-6 Verifying Interface Connectivity, page 2-8 Assigning VSAN Numbers, page 2-9 Multiple Initiators and Targets, page

22 Preparing the Cisco MDS Switch Chapter 2 Getting Started Preparing the Cisco MDS Switch To prepare for both hardware and software setup, be sure you have the following in place before getting started: Installed and functioning Cisco MDS 9000 Family switch fabric. Each Cisco MDS 9000 Family switch needs a switch IP (mgmt0, one for each switch). Note Refer to the Cisco MDS [9216 Switch or the Cisco MDS 9500] Series Hardware Installation Guides, and the Cisco MDS 9000 Family Configuration Guide. One IP address for each SVC cluster for management purposes. This should be in the same subnet as the management IP address of the switches across which the cluster spans. Setting Up the Cisco MDS Switch This section explains the process to set up a new, or update an existing MDS 9000 Family switch in preparation for the SVC. Before setting up your hardware, ensure you have correctly installed the Cisco MDS chassis and its components as specified in Chapter 2: Installing the Cisco MDS 9000 Family Switch in the Cisco MDS [9216 Switch or the Cisco MDS 9500] Series Hardware Installation Guides. Caution Be sure to save your work frequently using the copy running-config startup-config command. Warning The SVC functionality is only available when CSM modules are present in the switch. If you issue a copy running startup command when all CSM modules are removed (or powered-down), then SVC supervisor configurations associated with the CSM module, including world wide name (WWNs), may be discarded. Setting Up a New Cisco MDS Switch To set up a new Cisco MDS 9000 Family switch, follow these steps. Step 1 Step 2 Step 3 Step 4 Follow instructions for preinstallation, installing the chassis in the rack, grounding the chassis, installing modules, installing CompactFlash cards, installing Power Supplies, and installing the Fan Assembly as specified in the Cisco MDS [9216 Switch or the Cisco MDS 9500] Series Hardware Installation Guides. Connect to the supervisor module as specified in Chapter 3: Connecting the Cisco MDS 9000 Family Switch in the Cisco MDS [9216 Switch or the Cisco MDS 9500] Series Hardware Installation Guides. Login to the Cisco MDS 9000 Family switch using the Cisco MDS 9000 Family CLI. Configure the switch as specified in Chapter 3: Initial Configuration in the Cisco MDS 9000 Family Configuration Guide. a. Perform the initial setup routine. b. Assign a switch name. 2-2

23 Chapter 2 Getting Started Setting Up the Cisco MDS Switch c. Access the switch. After reviewing the default configuration, you can change it or perform other configuration or management tasks. The initial setup can only be performed at the Command-Line Interface (CLI). However, you can continue to configure other software features, or access the switch after initial configuration by using either the CLI, or the Fabric Manager GUIs. Tip The Cisco Fabric Manager provides an alternative to the CLI for most switch configuration commands. To use the Cisco MDS 9000 Fabric Manager, refer to the Cisco MDS 9000 Family Fabric Manager User Guide. To use the CLI, refer to the Cisco MDS 9000 Family Configuration Guide. Note The rest of this procedure uses the Cisco CLI to configure the switch. Step 5 Verify the module status as specified in Chapter 3: Initial Configuration in the Cisco MDS 9000 Family Configuration Guide. Step 6 Configure the management port as specified in Chapter 3: Initial Configuration in the Cisco MDS 9000 Family Configuration Guide. Updating an Existing Cisco MDS Switch Note These steps assume you have a remote FTP, TFTP, SFTP, or SCP server that contains the SVC image. Caution Be sure to save your work frequently using the copy running-config startup-config command. To update a existing Cisco MDS switch, follow these steps. Step 1 Login to the Cisco MDS switch using the Cisco MDS 9000 Family CLI. Note Step 2 Refer to the web site to verify compatibility issues, or the appropriate MDS 9000 Family release notes to ensure your system and setup meets the minimum requirements, or images will not install properly. Install the new image on each SVC node in each module in the fabric. Each module has two nodes. switch# install module 2 node 1 image svc-system ftp:// /m9000-ek9-csm-svc-mz bin For ftp:// , please enter user name:user For ftp://user@ , please enter password: SVC reimage going on. Please wait m9000-ek9-csm-svc-mz bin 100% ***************************** KB 00:53 svc 2/1 software reimage succeeded switch# install module 2 node 2 image svc-system ftp:// /m9000-ek9-csm-svc-mz bin For ftp:// , please enter user name:user 2-3

24 SVC Role Authorization Chapter 2 Getting Started For ftp://user@ , please enter password: SVC reimage going on. Please wait m9000-ek9-csm-svc-mz bin 100% ***************************** KB 00:55 svc 2/2 software reimage succeeded Step 3 All prior information on the node is lost with each install. Verify that the CSM node booted successfully, by issuing the show module command. switch# show module Mod Ports Module-Type Model Status Caching Services Module DS-X9560-SMC ok < CSM 4 8 IP Storage Services Module DS-X9308-SMIP ok 5 0 Supervisor/Fabric-1 DS-X9530-SF1-K9 active * 6 0 Supervisor/Fabric-1 DS-X9530-SF1-K9 ha-standby Mod Sw Hw World-Wide-Name(s) (WWN) (1) (1) :c1:00:05:30:00:a7:9e to 20:c8:00:05:30:00:a7:9e 5 1.3(1) (1) Mod Application Image Description Application Image Version svc-node1 1.3(1) < Node one on the CSM 2 svc-node2 1.3(1) < Node two on the CSM Mod MAC-Address(es) Serial-Num e2 to e6 JAB06xxxx d-de to d-ea JAB064605aa f2 to f e to * this terminal session Tip The initial setup can only be performed at the CLI. You can continue to configure other software features, or access the switch after initial configuration by using either the CLI or the Device Manager and Fabric Manager GUIs. To use the Cisco MDS 9000 Fabric Manager, refer to the Cisco MDS 9000 Family Fabric Manager User Guide. SVC Role Authorization By default, two roles exist in all Cisco MDS switches: SVC administrator (svc-admin) Has permission to view the entire configuration and make SVC-specific configuration changes within the switch(svc) prompt. SVC operator (svc-operator) Has permission to view the entire configuration. The operator cannot make any configuration changes. The two default roles cannot be changed or deleted. 2-4

25 Chapter 2 Getting Started Understanding SVC Terminology Understanding SVC Terminology This section defines frequently-used SVC terms that are necessary to proceed with the configuration process (see Figure 2-1). Figure 2-1 Graphical Representation of SVC Terms I/O Group Node 1 CSM 1 Node 2 I/O Group Node 1 CSM 2 Node 2 Vdisk 1 Vdisk 2 Vdisk 3 Mdisk group Mdisk group Node An instance of SVC that is running in the CSM. A CSM consists of two completely independent nodes. Each node is represented by an interface. I/O group SVC nodes are always used in pairs. A pair of SVC nodes is called an I/O group. The nodes in a given I/O group must reside in two separate CSMs. An I/O group acts as a storage controller in the fabric. Cluster A cluster can contain multiple I/O groups. All nodes in the cluster must run the same SVC image version. The IP address for a cluster must be assigned in the same subnet as the management interface. MDisk A representation of back end storage. Each cluster is configured so all the nodes in a cluster see the same set of MDisks. VDisk A virtual representation of a LUN that is exposed by the cluster to the hosts in a SAN. Each VDisk is independently associated with a single I/O group. 2-5

26 Separating Hosts and Storage Devices Chapter 2 Getting Started MDisk Group A set of MDisks form a MDisk group. Storage for a VDisk originates from MDisks in a single MDisk group. Host One of more initiator Fibre Channel ports (pwwns) form a host. A host is mapped to one or more VDisks. Hosts cannot directly access a MDisk. Separating Hosts and Storage Devices In order for SVC to virtualize the back-end storage to hosts, you must ensure that the hosts do not directly access the storage. You can separate hosts or disks using the concept of VSANs. The VSAN feature is specific to Cisco MDS 9000 Family switches. Alternatively, you can also use zones to separate hosts or disks. Refer to the Cisco MDS 9000 Family Configuration Guide for further information on VSANs or zones. Each SVC interface is assigned a nwwn. A SVC interface consists of three N-ports: Target N-ports: used for traffic between SVC nodes and hosts. Initiator N-ports: used for traffic between SVC nodes and disks Management N-ports: used for traffic to and from other SVC nodes. Each N-port within an SVC interface is assigned a port World Wide Name (pwwn). The pwwns and node World Wide Names (nwwns) are preserved across switch reboots (see Figure 2-2). Figure 2-2 Logical Representation of an SVC Interface VSAN 1 VSAN 2 Node 1 Host H Node 2 Node 3 Host T Node 4 Node 5 Node 6 VSAN 3 Target B Mgmt C Initiator A Node In Figure 2-3, provides a logical view of four SVC nodes in a SAN. These nodes are configured so the hosts do not have direct access to the disks. 2-6

27 Chapter 2 Getting Started Separating Hosts and Storage Devices By default, all N-ports reside in VSAN 1. You must explicitly remove them when necessary. Figure 2-3 Logical Representation of a SAN Zone 1 Zone 2 Host VSAN VSAN 1 Target Target Target Target Node 3 Initiator Mgmt Node 1 Initiator Mgmt Management VSAN VSAN 2 Mgmt Node 2 Initiator Mgmt Node 4 Initiator Disk VSAN VSAN To configure a SVC interface and N-port VSANs in a Cisco MDS switch, follow these steps: Command Step 1 switch# config t switch(config)# Step 2 switch(config)# interface svc 2/1 switch(config-if)# Purpose Enters configuration mode. Enters the configuration mode for SVC interface 2/1. Step 3 switch(config-if)# initiator vsan 3 Configures the initiator VSAN 3 for disks. switch(config-if)# no initiator vsan 1 Removes VSAN 1 Step 4 switch(config-if)# target vsan 1 Configures the target VSAN 1 for hosts. Step 5 switch(config-if)# mgmt vsan 2 Configures the management VSAN 2 for nodes. Step 6 switch(config-if)# no mgmt vsan 1 Removes VSAN 1. Step 7 switch(config-if)# no shutdown Enables the SVC interface. 2-7

28 Verifying Interface Connectivity Chapter 2 Getting Started Verifying Interface Connectivity This section lists the show commands that help verify the SVC interface state and the N-port association with the SVC interface. Example 2-1 Displays Information for A Specified SVC Interface switch# show interface svc 2/1 svc2/1 is up Node WWN is 2e:ab:00:05:30:00:1a:e0 Fabric WWN is 20:01:00:05:30:00:1a:de Target N-port WWN is 2e:a5:00:05:30:00:1a:e0, vsan is 1, FCID is 0xe80003 Initiator N-port WWN is 21:2e:00:05:30:00:00:21, vsan is 3, FCID is 0xea0004 Mgmt N-port WWN is 2f:af:00:05:30:00:1a:e0, vsan is 2, FCID is 0xe minutes input rate 2392 bits/sec, 299 bytes/sec, 0 frames/sec 5 minutes output rate 2240 bits/sec, 280 bytes/sec, 0 frames/sec 272 frames input, bytes 0 discards, 0 errors 232 frames output, bytes 0 discards, 0 errors Example 2-2 Displays N-port Connections for a Specified SVC Interface switch# show svc session svc 2/1 svc2/1: Target N-port WWN is 2e:a5:00:05:30:00:1a:e0, vsan is 1, FCID is 0xe80003 pwwn 21:01:00:e0:8b:31:20:31, nwwn 20:01:00:e0:8b:31:20:31, FCID 0xe80200 pwwn 21:00:00:e0:8b:11:29:31, nwwn 20:00:00:e0:8b:11:29:31, FCID 0xe80300 Initiator N-port WWN is 21:2e:00:05:30:00:00:21, vsan is 3, FCID is 0xea0004 pwwn 50:05:07:63:00:c8:9c:f9, nwwn 50:05:07:63:00:c0:9c:f9, FCID 0xea0000 pwwn 50:05:07:63:00:c8:9c:fa, nwwn 50:05:07:63:00:c0:9c:fa, FCID 0xea0001 Mgmt N-port WWN is 2f:af:00:05:30:00:1a:e0, vsan is 2, FCID is 0xe80000 pwwn 2f:b9:00:05:30:00:1a:e0, nwwn 2f:b7:00:05:30:00:1a:e0, FCID 0xe80002 pwwn 2f:ba:00:05:30:00:1a:e0, nwwn 2e:b3:00:05:30:00:1a:e0, FCID 0xe80003 pwwn 2f:b8:00:05:30:00:1a:e0, nwwn 2e:ac:00:05:30:00:1a:e0, FCID 0xe80001 Example 2-3 Displays the FCNS Database switch# show fcns database VSAN 1: FCID TYPE PWWN (VENDOR) FC4-TYPE:FEATURE xe80000 N 2e:a8:00:05:30:00:1a:e0 (Cisco) scsi-fcp:target svc 0xe80003 N 2e:a5:00:05:30:00:1a:e0 (Cisco) scsi-fcp:target svc 0xe80200 N 21:01:00:e0:8b:31:20:31 (QLogic) scsi-fcp:init 0xe80300 N 21:00:00:e0:8b:11:29:31 (QLogic) scsi-fcp:init Total number of entries = 4 VSAN 2: FCID TYPE PWWN (VENDOR) FC4-TYPE:FEATURE xe80000 N 2f:af:00:05:30:00:1a:e0 (Cisco) scsi-fcp:both svc 0xe80001 N 2f:b8:00:05:30:00:1a:e0 (Cisco) scsi-fcp:both svc Total number of entries = 2 VSAN 3: FCID TYPE PWWN (VENDOR) FC4-TYPE:FEATURE 2-8

29 Chapter 2 Getting Started Assigning VSAN Numbers xea0000 N 50:05:07:63:00:c8:9c:f9 (IBM) scsi-fcp:target fc.. 0xea0001 N 50:05:07:63:00:c8:9c:fa (IBM) scsi-fcp:target fc.. 0xea0002 N 21:2e:00:05:30:00:00:23 (Cisco) scsi-fcp:init svc Total number of entries = 3 Assigning VSAN Numbers When configuring SVC interfaces and N-port VSANs, the VSAN number can be any number from 1 to Only 64 VSANs for all initiator/mgmt/target (in total) are allowed you can have initiator in VSANs 1-30, target in VSANs 31-60, and mgmt in VSANs 61-64). If the target, inititator, and mgmt overlap in any VSAN, each overlap is also included in the total VSAN count. A mgmt N-port can only exist in 4 of these 64 VSANs. Multiple Initiators and Targets You can create multiple N-ports for any SVC interface in different VSANs (see Figure 2-1). Figure 2-4 Graphical Representation of SVC Terms Host VSAN VSAN 1 Host VSAN VSAN 5 Target Target Target Target Target Target Target Target Node 3 Initiator Mgmt Node 1 Initiator Mgmt Management VSAN VSAN 2 Mgmt Node 2 Initiator Mgmt Node 4 Initiator Disk VSAN VSAN

30 Multiple Initiators and Targets Chapter 2 Getting Started 2-10

31 CHAPTER 3 Creating and Managing Clusters This section explains the steps required to create clusters. To configure other SVC features or to access the switch after initial configuration you can use one of the following CLI or Graphical User Interface (GUI) options: Cisco MDS 9000 Family CLI to use the Cisco MDS CLI, follow the procedure specified in this guide. Note The rest of this procedure uses the Cisco CLI to configure the Cisco MDS switch. IBM TM SAN Volume Controller CLI to use the IBM CLI, refer to the IBM TotalStorage SAN Volume Controller Storage Software for Cisco MDS 9000 Command-Line Interface User's Guide. IBM SAN Volume Controller GUI to use the IBM web-based GUI, refer to the IBM TotalStorage SAN Volume Controller Storage Software for Cisco MDS 9000 Configuration Guide. This chapter includes the following sections: About CSM Nodes, page 3-2 About Clusters, page 3-2 Selecting Nodes for a Cluster, page 3-3 Isolating Management Traffic, page 3-4 Creating a Cluster, page 3-5 Adding Nodes to Clusters, page 3-6 Verifying Nodes in a Cluster, page 3-7 Deleting a node from a Cluster, page

32 About CSM Nodes Chapter 3 Creating and Managing Clusters About CSM Nodes A node provides virtualization, caching, migration and copy services to the SAN. Nodes are deployed in pairs with each pair of nodes forming an I/O group. The nodes belonging to the same I/O group MUST have different power domains. This entails that nodes of the same I/O group should come from two different CSMs. When a node fails within an I/O group then the other node in the I/O group will take over the responsibilities of the failed node. Data loss during a node failure is prevented by mirroring the IO read/write cache info across both the nodes in the I/O group. About Clusters Nodes are grouped into clusters of up to 2 pairs of nodes. These nodes are managed as a set (cluster), and present a single point of control for the user for configuration and service activity. For I/O purposes, so as to avoid a single point of loss of availability nodes will be grouped into pairs (I/O groups), with a single pair being responsible for serving I/O on a given VDisk. I/O traffic for a particular VDisk is, at any one time, handled exclusively by the nodes in a single I/O group. Thus, although a cluster may have many nodes within it, the nodes handle i/o in independent pairs. This means that the i/o capability of a cluster scales well, since additional throughput can simply be obtained by adding additional I/O Groups. There are some circumstances when all the nodes in the cluster do act together rather than in pairs. At any one time, a single node in the cluster is used to manage configuration activity. The node at which the cluster got created will start off as the configuration node. This configuration node manages a cache of the configuration information that describes the cluster configuration and provides a focal point for configuration commands. Similarly, at any one time, a single node acts as the managing node for overall management of the cluster. If the configuration node or managing node fails, another node in the cluster will take over their responsibilities. The nodes also act together to implement the data migration function described in Chapter 7, Configuring Copy Services. There are several advantages to managing a set of nodes as a cluster. All the cluster related configuration operations happen on the config node. Individual node operations like node addition, deletion, shutdown, can be done at the config node. All the nodes in the cluster run the same software version. Software upgrade can be initiated for the whole cluster instead of having to do this on a per-node basis. Physical Topology In Figure 3-1, CSMs reside in slots 3 and 7 in a Cisco MDS 9500 Series switch. CSM 3 has two nodes identified as interface svc 3/1 and interface svc 3/2. CSM 7 has two nodes identified as interface svc 7/1 and interface svc 7/2. These four interfaces are configured to form a 4-node cluster. I/O group 1 includes interface svc 3/1 (Node1) and interface svc 7/1 (Node3). I/O group 2 is made up of interface svc 3/2 (Node 2) and interface svc 7/2 (Node 4). These two I/O groups form a SVC cluster. So SVC interfaces 3/1, 3/2, 7/1, and 7/2 belong to one cluster Figure 3-1 also shows two hosts and a back-end storage device. This physical topology serves as an example in the following sections to understand SVC configurations. 3-2

33 Chapter 3 Creating and Managing Clusters Selecting Nodes for a Cluster Figure 3-1 Sample SVC Configuration Using Cisco MDS Switches Host 1 Host 2 Cisco MDS 9500 Switch ESS Node 1 CSM 3 Node 2 Node 3 CSM 7 Node Selecting Nodes for a Cluster To configure a 4-node sample configuration, follow these steps. Step 1 Change to SVC configuration mode. switch1# svc-config Step 2 Display the available nodes in the local switch (switch 1). switch1(svc)# show nodes local Node cluster config cluster node sw node status status version svc3/1 No unconfigured free 1.3(1) svc3/2 No unconfigured free 1.3(1) svc7/1 No unconfigured free 1.3(1) svc7/2 No unconfigured free 1.3(1) 3-3

34 Isolating Management Traffic Chapter 3 Creating and Managing Clusters Isolating Management Traffic Management traffic is isolated for CSM nodes that are part of the same cluster using a traffic domain separator (for example VSANs or zones). To create a separate traffic domain using VSANs, follow these steps. Step 1 Step 2 Step 3 Step 4 Exit the SVC configuration mode and enter the configuration mode. switch1(svc)# exit switch1# config t switch1(config)# Configure VSAN 2 for CSM management traffic switch1(config)# vsan database switch1(config-vsan-db)# vsan 2 switch1(config-vsan-db)# exit switch1(config)# Configure the management N-ports on VSAN 2 for all 4 SVC nodes. switch1(config)# interface svc3/1 switch1(config-if)# mgmt vsan 2 switch1(config-if)# no mgmt vsan 1 <--- You have to explicitly remove from vsan 1 switch1(config-if)# no shut switch1(config-if)# exit switch1(config)# interface svc3/2 switch1(config-if)# mgmt vsan 2 switch1(config-if)# no mgmt vsan 1 <--- You have to explicitly remove from vsan 1 switch1(config-if)# no shut switch1(config-if)# exit switch1(config)# interface svc7/1 switch1(config-if)# mgmt vsan 2 switch1(config-if)# no mgmt vsan 1 <--- You have to explicitly remove from vsan 1 switch1(config-if)# no shut switch1(config-if)# exit switch1(config)# interface svc7/2 switch1(config-if)# mgmt vsan 2 switch1(config-if)# no mgmt vsan 1 <--- You have to explicitly remove from vsan 1 switch1(config-if)# no shut switch1(config-if)# exit switch1(config)# Show the FC name server database switch1# show fcns database vsan 2 VSAN 2: FCID TYPE PWWN (VENDOR) FC4-TYPE:FEATURE x N 22:32:00:05:30:00:11:69 (Cisco) scsi-fcp:both svc 0x N 22:33:00:05:30:00:11:69 (Cisco) scsi-fcp:both svc 0x N 22:34:00:05:30:00:11:69 (Cisco) scsi-fcp:both svc 0x N 22:35:00:05:30:00:11:69 (Cisco) scsi-fcp:both svc Total number of entries = 4 Step 5 Default zone configuration is permit for VSAN 2. switch1(config)# zone default-zone permit vsan 2 switch1(config)# 3-4

35 Chapter 3 Creating and Managing Clusters Creating a Cluster Creating a Cluster Create a cluster called SampleCluster using one node for the cluster creating This example uses interface svc3/1 to begin the cluster creation process. It also uses the IP address that is in the same subnet as switch 1 s management s IP network. Note If the cluster spans multiple switches, the switch management IP address should be in the same subnet as the cluster IP address, because the cluster IP address can move to any switch (based on the SVC config node). To create a cluster, follow these steps. Step 1 Step 2 Step 3 Create a cluster using the cluster add command in SVC configuration mode. switch1# svc-config switch1(svc)# cluster add SampleCluster ip node svc3/1 Cluster creation going on. Please wait...---> This process takes a few seconds. Verify the newly-created cluster for the configured node. switch1(svc)# show nodes local Node cluster config cluster node sw node status status version svc3/1 SampleCluster Yes active active 1.3(1) svc3/2 No unconfigured free 1.3(1) svc7/1 No unconfigured free 1.3(1) svc7/2 No unconfigured free 1.3(1) Display the configured cluster. switch1(svc)# show cluster SampleCluster nodes Node node1 is online(3) Node WWN is 22:26:00:05:30:00:11:69 Serial number is JAB072006AQ Unique id is 01:00:07:20:30:36:41:51 Node is in config mode Node is part of iogroup id 1 name io_grp0 The configured node is the only node in this cluster. 3-5

36 Adding Nodes to Clusters Chapter 3 Creating and Managing Clusters Adding Nodes to Clusters Once the initial node is used to create a cluster, you can add other required nodes to the same cluster. You can determine which nodes are available by issuing the show cluster cluster-name nodes candidate command. To add other nodes to a cluster, follow these steps. Step 1 Add the other nodes to the cluster. a. Verify which nodes can be added to the cluster. This step is important if you have several SVC nodes in the SAN,. switch1(svc)# show cluster SampleCluster nodes candidate NODE NWWN switch :28:00:05:30:00:11:69 switch :26:00:05:30:00:11:69 switch :2a:00:05:30:00:11:69 This example has 3 other SVC nodes in the SAN that are candidate nodes for this cluster. The node name is an encoding of the <switch-name>.<slot-number>.<node-id>. For example: switch1.7.2 is in the switch named switch1 at slot 7 node 2. Caution Do not add two nodes from the same CSM to the same I/O group of a cluster. Cisco MDS SVC does not allow this configuration as both nodes will be contained in one power domain. If both nodes are configured in the same I/O group of one cluster and a power failure occurs, both nodes will fail. b. Add more nodes (switch 1-slot 3-node 2, switch 1-slot 7-node 1, and switch 1-slot 7-node 2) to the newly-created cluster by entering the configuration submode for the selected cluster (SampleCluster) switch1(svc)# cluster config SampleCluster switch1(svc-cluster)# node nwwn 21:28:00:05:30:00:11:69 iogroup 1 switch1(svc-cluster)# node nwwn 21:26:00:05:30:00:11:69 iogroup 2 switch1(svc-cluster)# node nwwn 21:2a:00:05:30:00:11:69 iogroup 2 switch(svc-cluster)# exit 3-6

37 Chapter 3 Creating and Managing Clusters Verifying Nodes in a Cluster Verifying Nodes in a Cluster After the cluster is created with the required nodes, you can verify the status of each node in the cluster, status of each node, and the associated I/O group to ensure the configuration is functioning as desired. The node state transitions from adding to pending to online during the cluster creation process. To verify the nodes in a cluster, follow these steps. Step 1 Verify the nodes in the cluster. switch1(svc)# show cluster SampleCluster nodes Node node1 is online(3) Node WWN is 22:26:00:05:30:00:11:69 Serial number is JAB072006AQ Unique id is 01:00:07:20:30:36:41:51 Node is in config mode Node is part of iogroup id 1 name io_grp0 Node node2 is online(3) Node WWN is 21:28:00:05:30:00:11:69 Serial number is JAB076607H8 Unique id is 01:00:07:66:30:37:48:38 Node is in non config mode Node is part of iogroup id 1 name io_grp0 Node node3 is pending(2) Node WWN is 21:26:00:05:30:00:11:69 Serial number is JAB071007H8 Unique id is 01:00:07:10:30:37:48:38 Node is in non config mode Node is part of iogroup id 2 name io_grp1 Node node4 is adding(6) Node WWN is 00:00:00:00:00:00:00:00 Serial number is JAB076606AQ Unique id is 01:00:07:66:30:36:41:51 Node is in non config mode Node is part of iogroup id 2 name io_grp1 Step 2 Verify the I/O group status switch1(svc)# show cluster SampleCluster iogroup ID Name Node count VLUN count io_grp io_grp io_grp io_grp recovery_io_grp 0 0 Note The recovery_io_group is an internal iogroup used during cluster recovery. 3-7

38 Deleting a node from a Cluster Chapter 3 Creating and Managing Clusters Step 3 Display the status of all nodes in the switch. switch(svc)# show nodes local Config Cluster Node Sw Node Cluster node status status version svc3/1 SampleCluster Yes Active Active 1.3(1) svc3/2 SampleCluster No Active Active 1.3(1) svc7/1 SampleCluster No Active Active 1.3(1) svc7/2 SampleCluster No active Active 1.3(1) The 4-node cluster is now created with the nodes communicating with each other in VSAN 2. All nodes in the switch are active and are part of cluster named SampleCluster. The SVC config node is svc3/1 (see Figure 3-2). Figure 3-2 Creating a 4-Node Cluster Target Target Target Target Node 3 Initiator Mgmt Node 1 Initiator Mgmt Management VSAN VSAN 2 Mgmt Node 2 Initiator Mgmt Node 4 Initiator Deleting a node from a Cluster A node has to be removed from a cluster using the no node command in the cluster config mode. To delete a node that is online, follow these steps. Step 1 Step 2 Enter the cluster configuration mode for the required cluster. switch1# svc-config switch1(svc)# cluster config SampleCluster Issue the no node nwwn command. switch1(svc-cluster)# no node nwwn 21:28:00:05:30:00:11:69 iogroup 1 To delete a node that is in an offline state, follow these steps. Step 1 Step 2 Enter the cluster configuration mode for the required cluster. switch1# svc-config switch1(svc)# cluster config SampleCluster Issue the no node name command. switch1(svc-cluster)# no node name node3 3-8

39 Chapter 3 Creating and Managing Clusters Deleting a node from a Cluster When you delete a node in a cluster, the node is removed from the cluster state. In addition, the local state of the deleted node is also updated to indicate that it is no longer a part of any cluster. If the node is offline, the local state of the deleted node should be explicitly updated using the node svc x/y delete command. Deleting a Cluster The MDS CLI does not use an explicit command to delete a cluster. The cluster is automatically deleted when the last node in the cluster is deleted. 3-9

40 Deleting a node from a Cluster Chapter 3 Creating and Managing Clusters 3-10

41 CHAPTER 4 Managing Back-End Storage The nodes in a cluster view the back-end storage controllers as individual disks, known as managed disks (MDisks). This chapter includes the following sections: About Managed Disks, page 4-2 About MDisk Groups, page 4-2 Extents, page 4-2 MDisk Modes, page 4-2 Isolating Back-end Storage Traffic, page 4-3 Verifying Traffic Isolation, page 4-4 Configuring LUN Masking, page 4-4 Configuring LUN Masking, page 4-4 Configuring MDisk Groups, page 4-5 Configuring Quorum Disks, page

42 About Managed Disks Chapter 4 Managing Back-End Storage About Managed Disks SVC nodes see the back-end storage controllers presented to SANs as a number of individual disks, known as Managed Disks (MDisks). SVC does not attempt to provide recovery from physical disk failures within the back-end controllers. For this reason, a MDisk is usually, but not necessarily, a RAID array. All nodes in a cluster must be able to see the same set of back-end storage ports on each back-end controller. Operation in a mode where two nodes see a different set of ports on the same controller is degraded and the system logs errors requesting a repair action. This occurs if inappropriate zoning is configured in the fabric or if inappropriate LUN masking is used. This rule has important implications for back-end storage such as FastT which impose exclusivity rules on the HBA WWNs that a storage partition can be mapped to. For such storage devices, the initiator N-port pwwns of all nodes in a cluster must be mapped to the same set of LUNs About MDisk Groups MDisk Groups are collections of MDisks. A MDisk is contained within exactly one MDisk Group.The MDisk Group acts as a container into which MDisks contribute Extents (Chunks of disk blocks) and from which VDisk (logical disks associated with nodes in an I/O group) consume these extents of storage. Extents An extent is the unit of allocation of storage in a MDisk. Each MDisk is broken up logically into a number of extents. A MDisk does not need to be a integer multiple of extent size. SVC supports a partial extent at the end of the MDisk. However, a VDisk occupies an integer number of extents even if the VDisk size is not an integer multiple of the extent size. The remaining space at the last extent in the VDisk remains unused. MDisk Modes The three MDisks modes are Image, Managed, or Unmanaged. Image mode Image Mode provides a direct block-for-block translation from a MDisk to a VDisk virtualization. This mode allows virtualization of MDisks which already contain data. It allows a customer to insert SVC into the data path of an existing storage configuration with minimal downtime. Once SVC is inserted into the data path using image mode, you can use SVC s migration facilities to migrate the data to managed mode and re-arrange the data while an application is accessing the data. Managed mode Disks operating in managed mode allow an arbitrary relationship between the VDisk extents and the MDisk extents. The actual mapping of the extent is based on the VDisk creation policy. The unused extents in a MDisk are available for use in creating new VDisks data migration. Unmanaged mode Mdisks in this mode do not belong to any Mdisk group 4-2

43 Chapter 4 Managing Back-End Storage Isolating Back-end Storage Traffic Isolating Back-end Storage Traffic Setup the traffic isolation for all initiator N-ports in the SampleCluster into the same VSAN or zone as the back-end storage ports. To isolate initiator N-ports, follow these steps. Step 1 Step 2 Create the target N-ports that this cluster can and should access. In this example, the N-ports to be accessed by the SampleCluster are 20:12:00:05:30:00:8d:e0 (interface fc 1/7) and 20:22:00:05:30:00:8d:e0 (interface fc 1/8). Create VSAN 3 for the target traffic for the SampleCluster. switch# conf t switch(config)# vsan database switch(config-vsan-db)# vsan 3 Step 3 Add the two Fibre Channel N-ports connected to targets into VSAN 3. switch(config-vsan-db)# vsan 3 interface fc1/7 switch(config-vsan-db)# vsan 3 interface fc1/8 switch(config-vsan-db)# exit switch(config)# Step 4 Add the CSM node's initiator N-port in the SampleCluster in VSAN 3. Note Each initiator must be explicitly removed from VSAN 1. switch(config)# interface svc3/1 switch(config-if)# initiator vsan 3 switch(config-if)# no initiator vsan 1 switch(config-if)# exit switch(config)# interface svc3/2 switch(config-if)# initiator vsan 3 switch(config-if)# no initiator vsan 1 switch(config-if)# exit switch(config)# interface svc7/1 switch(config-if)# initiator vsan 3 switch(config-if)# no initiator vsan 1 switch(config-if)# exit switch(config)# interface svc 7/2 switch(config-if)# initiator vsan 3 switch(config-if)# no initiator vsan 1 switch(config-if)# exit Step 5 Configure the default zone configuration as permit for VSAN3. switch(config)# zone default-zone permit vsan 3 4-3

44 Verifying Traffic Isolation Chapter 4 Managing Back-End Storage You have now created a disk VSAN (see Figure 4-1). Figure 4-1 Creating Traffic Isolation Using a Target VSAN Target Target Target Target Node 3 Initiator Mgmt Node 1 Initiator Mgmt Management VSAN VSAN 2 Mgmt Node 2 Initiator Mgmt Node 4 Initiator Disk VSAN VSAN Verifying Traffic Isolation To verify that the target initiator ports are configured and traffic isolation has been implemented issue the show fcns database command for VSAN 3. switch# show fcns database vsan 3 VSAN 3: FCID TYPE PWWN (VENDOR) FC4-TYPE:FEATURE x N 50:05:07:63:00:c5:9c:f9 (IBM) scsi-fcp:target fc.. 0x N 50:05:07:63:00:cf:9c:f9 (IBM) scsi-fcp:target fc.. 0x N 22:36:00:05:30:00:11:69 (Cisco) scsi-fcp:init svc 0x N 22:37:00:05:30:00:11:69 (Cisco) scsi-fcp:init svc 0x N 22:38:00:05:30:00:11:69 (Cisco) scsi-fcp:init svc 0x N 22:39:00:05:30:00:11:69 (Cisco) scsi-fcp:init svc Total number of entries = 6 Configuring LUN Masking The initiator N-port pwwn of all four nodes in the cluster must be used to configure LUN masking in the backend storage controller. switch# show interface svc 3/1 svc2/1 is up Node WWN is 2e:ab:00:05:30:00:1a:e0 Fabric WWN is 20:01:00:05:30:00:1a:de Target N-port WWN is 2e:a5:00:05:30:00:1a:e0, vsan is 1, FCID is 0xe80003 Initiator N-port WWN is 21:2e:00:05:30:00:00:21, vsan is 3, FCID is 0xea0004 Mgmt N-port WWN is 2f:af:00:05:30:00:1a:e0, vsan is 2, FCID is 0xe

45 Chapter 4 Managing Back-End Storage Identifying MDisks 5 minutes input rate 2392 bits/sec, 299 bytes/sec, 0 frames/sec 5 minutes output rate 2240 bits/sec, 280 bytes/sec, 0 frames/sec 272 frames input, bytes 0 discards, 0 errors 232 frames output, bytes 0 discards, 0 errors Identifying MDisks To identify MDisks, follow these steps: Step 1 Step 2 Display the details of each MDisk using the show cluster cluster-name mdisk command. This command lists the back-end controller ports that can be accessed by the MDisk. The MDisk still do not belong to any MDisk group. switch# svc-config switch(svc)# show cluster SampleCluster mdisk id nwwn mdisk-grp capacity status :05:07:63:00:c0:9c:f9 953 MB online 2 50:05:07:63:00:c0:9c:f GB online 3 50:05:07:63:00:c0:9c:f GB online 4 50:05:07:63:00:c0:9c:f GB online 5 50:05:07:63:00:c0:9c:f GB online 6 50:05:07:63:00:c0:9c:f9 953 MB online 7 50:05:07:63:00:c0:9c:f GB online 8 50:05:07:63:00:c0:9c:f9 953 MB online 9 50:05:07:63:00:c0:9c:f GB online 10 50:05:07:63:00:c0:9c:f9 953 MB online Display details of the required MDisk using the show cluster cluster-name mdisk id command. switch(svc)# show cluster SampleCluster mdisk id 1 mdisk id 1 is online Is unmanaged Controller node WWN is 50:05:07:63:00:c0:9c:f9 --> IBM ESS storage device's nwwn Controller port WWN is 50:05:07:63:00:cf:9c:f9, LUN 00:00:00:00:00:00:00:00 Controller port WWN is 50:05:07:63:00:c5:9c:f9, LUN 00:00:00:00:00:00:00:00 Controller serial number is 07B24417 Capacity is 953 MB Configuring MDisk Groups To configure MDisk groups, follow these steps: Step 1 Obtain a list of candidate MDisks using the show cluster cluster-name mdisk candidate command. Select the group(s) to add the configured MDisk. switch(svc)# show cluster SampleCluster mdisk candidate id nwwn capacity :05:07:63:00:c0:9c:f9 953 MB 4-5

46 Configuring MDisk Groups Chapter 4 Managing Back-End Storage 2 50:05:07:63:00:c0:9c:f GB 3 50:05:07:63:00:c0:9c:f GB 4 50:05:07:63:00:c0:9c:f GB 5 50:05:07:63:00:c0:9c:f GB 6 50:05:07:63:00:c0:9c:f9 953 MB 7 50:05:07:63:00:c0:9c:f GB 8 50:05:07:63:00:c0:9c:f9 953 MB 9 50:05:07:63:00:c0:9c:f GB 10 50:05:07:63:00:c0:9c:f9 953 MB Step 2 Step 3 Step 4 Step 5 Step 6 Step 7 Step 8 Create a MDisk group called finance. switch(svc)# cluster config SampleCluster switch(svc-cluster)# mdisk-grp add finance extent 16 switch(svc-cluster)# mdisk-grp name finance switch(svc-cluster-mdisk-grp)# Add MDisks 1 to 5 to the finance MDisk group. switch(svc-cluster-mdisk-grp)# mdisk id 1 switch(svc-cluster-mdisk-grp)# mdisk id 2 switch(svc-cluster-mdisk-grp)# mdisk id 3 switch(svc-cluster-mdisk-grp)# mdisk id 4 switch(svc-cluster-mdisk-grp)# mdisk id 5 Exit to the SVC cluster configuration mode. switch(svc-cluster-mdisk-grp)# exit switch(svc-cluster)# Create a MDisk group called marketing. switch(svc-cluster)# mdisk-grp name marketing switch(svc-cluster)# mdisk-grp add marketing extent 32 switch(svc-cluster-mdisk-grp)# Add MDisks 6 to 10 to the marketing MDisk group. switch(svc-cluster-mdisk-grp)# mdisk id 6 switch(svc-cluster-mdisk-grp)# mdisk id 7 switch(svc-cluster-mdisk-grp)# mdisk id 8 switch(svc-cluster-mdisk-grp)# mdisk id 9 switch(svc-cluster-mdisk-grp)# mdisk id 10 Exit to the SVC configuration mode. switch(svc-cluster-mdisk-grp)# exit switch(svc-cluster)# exit switch(svc)# Verify that all configured MDisks are allocated to each MDisk group. switch(svc)# show cluster SampleCluster mdisk id nwwn mdisk-grp capacity status :05:07:63:00:c0:9c:f9 finance 953 MB online 2 50:05:07:63:00:c0:9c:f9 finance 1.86 GB online 3 50:05:07:63:00:c0:9c:f9 finance 1.86 GB online 4 50:05:07:63:00:c0:9c:f9 finance 1.86 GB online 5 50:05:07:63:00:c0:9c:f9 finance 1.86 GB online 6 50:05:07:63:00:c0:9c:f9 marketing 953 MB online 7 50:05:07:63:00:c0:9c:f9 marketing 1.86 GB online 8 50:05:07:63:00:c0:9c:f9 marketing 953 MB online 9 50:05:07:63:00:c0:9c:f9 marketing 1.86 GB online 10 50:05:07:63:00:c0:9c:f9 marketing 953 MB online 4-6

47 Chapter 4 Managing Back-End Storage Configuring Quorum Disks Step 9 Verify the details of each MDisk group and confirm that each MDisk group has 5 MDisks. VDisks have not been assigned at this point. switch(svc)# show cluster SampleCluster mdisk-grp name Capacity free extent number number status size(mb) of mdisks of vdisks finance 7.56 GB 7.56 GB online marketing 6.48 GB 6.48 GB online Note Step 10 The storage capacity (953 MB) is not an integer multiple of the extent size, the last partial extent is unused. To identify the number of free extents for MDisk 1, use the show cluster cluster-name mdisk id command. switch(svc)# show cluster SampleCluster mdisk id 1 mdisk id 1 is online Is member of mdisk-grp finance Controller node WWN is 50:05:07:63:00:c0:9c:f9 Controller port WWN is 50:05:07:63:00:cf:9c:f9, LUN 00:00:00:00:00:00:00:00 Controller port WWN is 50:05:07:63:00:c5:9c:f9, LUN 00:00:00:00:00:00:00:00 Controller serial number is 07B24417 Capacity is 953 MB Number of free extents is 42 Is quorum disk number 2 Note MDisk ID 1 (one) is a quorum disk and SVC nodes reserve some storage on the quorum disks for cluster management. Configuring Quorum Disks Quorum disks are used when the cluster loses half of the nodes that make up the cluster. The half of the cluster that contains the majority of the quorum disks continues to process the I/O transactions. The other half stops processing the I/O transactions. By establishing quorum disks, you can ensure that both halves of the cluster do not continue to operate. Quorum disks are set automatically by the software. You can use the quorum command to overwrite the system-assigned quorum disk and pick a particular set of managed disks to be a quorum disk. The managed disk that is currently assigned the quorum index number is automatically set to be a non-quorum disk. You can assign one of 3 possible quorum index numbers in any desired order. The quorum index number ranges from 1 to 3. Tip We recommend that you set quorum disks on multiple controllers to avoid the possibility of losing all of the quorum disks with a single failure. 4-7

48 Configuring Quorum Disks Chapter 4 Managing Back-End Storage To configure quorum disk allocation, follow these steps: Step 1 Step 2 Create a cluster called SampleCluster. switch(svc)# cluster config SampleCluster switch(svc-cluster)# Sets the quorum disk ID for the specified MDisk in this cluster. switch(svc-cluster)# quorum disk 2 mdisk 1 When you issue this command, the Mdisk that was previously-assigned the quorum index 2 will no longer be a quorum disk. Caution If the quorum command fails due to the lack of sufficient extents in the new quorum disk, the old quorum disk may no longer be operational. 4-8

49 CHAPTER 5 Managing Virtual Disks A VDisk is a virtual representation of a LUN that is exposed by the cluster to the hosts in a SAN. Each VDisk is independently associated with a single I/O group. This chapter includes the following sections: Virtualization Policies, page 5-2 Licensing Requirements, page 5-2 Configuring VDisks, page

50 Virtualization Policies Chapter 5 Managing Virtual Disks Virtualization Policies Virtualization is the process of creating a pool of storage that can be split into VDisks. VDisks are visible to the host systems that use them and provide a common way to manage SAN storage. VDisks in the Cisco MDS 9000 Family use one of three virtualization policies: striped, sequential, or image. Striped When a VDisk is created using a striped policy its extents are allocated from the specified ordered list of MDisks. The allocation algorithm starts with the first MDisk in the ordered list and attempts to allocate an extent to it and then moves on to the next disk. MDisk in turn process allocation if a specified MDisk has no free extents then it misses its turn and the turn passes to the next MDisk in the list. When the end of the list is reached the turn loops back to the first disk in the list. The disk allocation proceeds until all required extents are allocated. A specific MDisk can appear more than once in the list. This causes two extents to be allocated from the disk on each pass of the list. This might be useful when striping across MDisks of different sizes. Sequential When a VDisk is created using a sequential policy its extents are allocated from a single specified MDisk. The target MDisk is searched for regions containing free extents which are sequential such that the region is large enough to allocate the VDisk from completely sequential extents. If it finds more than once such region, it chooses the smallest region which satisfies this condition. If it finds no such regions, the VDisk creation fails. Image Image mode provides support to import existing data from a disk that was previously not managed by SVC. Licensing Requirements The total virtualized capacity that is licensed is the number of Gigabytes (GB) of VDisk capacity that are exported by the cluster. By default, this capacity is set to zero (0). The required amount of virtualization capacity must be licensed and configured using the feature enable command before creating any VDisk. When you reach 90% capacity, any attempt to create or extend VDisks results in a warning messages. The software does not stop you from creating and expanding VDisks. Instead, errors are placed in the featurization log when your usage reaches or exceeds 100%. 5-2

51 Chapter 5 Managing Virtual Disks Configuring VDisks Configuring VDisks To configure VDisks, follow these steps. Step 1 Create and identify three (3) VDisks from the marketing MDisk group and one (1) VDisk for the finance group. switch1(svc)# cluster config SampleCluster switch1(svc-cluster)# vdisk add crm-log iogroup 1 mdisk-grp marketing capacity 2 gb Warning: licensed virtualisation capacity has been exceeded Tip The official purchased virtualization capacity must be configured before any VDisk is created. Step 2 This warning is used if you are exceed the amount of virtualization for which you have a license. Use the feature enable capacity command to configure the amount of purchased virtualization capacity. Configure the licensed virtualization capacity to be 200GB and continue to create the VDisks for the marketing and finance MDisk groups. switch(svc-cluster)# vdisk add crm-data iogroup 1 mdisk-grp marketing capacity 2 gb clean switch(svc-cluster)# vdisk add crm-idx iogroup 1 mdisk-grp marketing capacity 1 gb switch(svc-cluster)# vdisk add fn-1 iogroup 2 mdisk-grp finance capacity 2 gb Note Step 3 Step 4 The clean option initializes the entire VDisk to 0. Until the cleaning is done, the VDisk stays in the offline state. Exit to the SVC configuration mode. switch(svc-cluster)# exit switch(svc)# Verify the VDisk configuration using the show cluster cluster-name vdisk command. switch(svc)# show cluster SampleCluster vdisk name capacity iogroup mdisk-grp name policy status crm-idx 1.00 GB 1 marketing striped online crm-log 1.00 GB 1 marketing striped online crm-data 2.00 GB 1 marketing striped offline fn GB 2 finance striped online Tip Step 5 The crm-data VDisk is offline due to the use of the clean option during the VDisk creation process in Step 2. Clearing a disk takes time please wait for this process to complete. Reissue the show cluster cluster-name vdisk command to ensure that all VDisks are online. switch(svc)# show cluster SampleCluster vdisk name capacity iogroup mdisk-grp name policy status crm-idx 1.00 GB 1 marketing striped online crm-log 1.00 GB 1 marketing striped online crm-data 2.00 GB 1 marketing striped online fn GB 2 finance striped online 5-3

52 Configuring VDisks Chapter 5 Managing Virtual Disks Note Step 6 Step 7 The online status for each VDisk indicates that formatting is complete. Use the show cluster cluster-name mdisk-group command to verify the number of VDisks created in each MDisk. switch(svc)# show cluster SampleCluster mdisk-grp name Capacity free extent number number status size(mb) of mdisks of vdisks finance 7.56 GB 5.56 GB online marketing 6.48 GB 2.48 GB online Use the show cluster cluster-name iogroup command to verify the number of VDisks available for each I/O group. switch(svc)# show cluster SampleCluster iogroup ID Name Node count Vdisk count io_grp io_grp io_grp io_grp recovery_io_grp 0 0 Note The recovery-io-group is an internal SVC I/O group created for cluster recovery processes. 5-4

53 CHAPTER 6 Configuring Hosts To continue configuring the using the SVC application for a Cisco MDS 9216 switch or for any switch in the Cisco MDS 9500 Family, you must determine the number of hosts, isolate host traffic to VSAN 1, and map VDisks to hosts. This chapter includes the following sections: About Hosts, page 6-1 Isolating Host Traffic, page 6-1 Creating Hosts, page 6-4 Mapping VDisks to Hosts, page 6-4 Configuring iscsi Hosts in SVC, page 6-5 About Hosts Hosts are identified to the cluster through user configuration. The LUN mapping feature controls which VDisks are accessible by which hosts. A host may contain multiple ports that connect to the SAN. To ease the configuration process, all host ports can be configured in a group. Host ports are referred to by their pwwns. The specific LUN number is optionally specified when the LUN map is configured. Otherwise, the cluster chooses a LUN number automatically. Isolating Host Traffic The SVC target N-ports are in VSAN 1 along with the two hosts. To isolate hosts and targets, this example uses the SAN-OS zoning feature. To isolate hosts using the zoning feature, follow these steps. Step 1 Issue the show fcns database vsan 1command to view the hosts and targets in VSAN 1. switch# show fcns database vsan 1 VSAN 1: FCID TYPE PWWN (VENDOR) FC4-TYPE:FEATURE x6a0200 N 21:00:00:e0:8b:09:e7:04 (QLogic) scsi-fcp:init 0x6a0300 N 21:00:00:e0:8b:09:f0:04 (QLogic) scsi-fcp:init 0x6a0003 N 22:20:00:05:30:00:11:69 (Cisco) scsi-fcp:target svc 0x6a0006 N 21:23:00:05:30:00:11:69 (Cisco) scsi-fcp:target svc 6-1

54 Isolating Host Traffic Chapter 6 Configuring Hosts 0x6a0009 N 22:23:00:05:30:00:11:69 (Cisco) scsi-fcp:target svc 0x6a000c N 21:20:00:05:30:00:11:69 (Cisco) scsi-fcp:target svc Total number of entries = 6 Step 2 Enter the MDS configuration mode and create a zone called host-finance in VSAN 1. switch# config t switch(config)# zone name host-finance vsan 1 switch(config-zone)# Step 3 Assign members to the host-finance zone. switch(config-zone)# member pwwn 21:00:00:e0:8b:09:e7:04 --> Host finance switch(config-zone)# member pwwn 22:20:00:05:30:00:11:69 --> SVC target N-port switch(config-zone)# member pwwn 21:23:00:05:30:00:11:69 --> SVC target N-port switch(config-zone)# member pwwn 22:23:00:05:30:00:11:69 --> SVC target N-port switch(config-zone)# member pwwn 21:20:00:05:30:00:11:69 --> SVC target N-port Step 4 Create a zone called host-marketing in VSAN 1. switch(config)# zone name host-marketing vsan 1 switch(config-zone)# Step 5 Assign members to the host-marketing zone. switch(config-zone)# member pwwn 21:00:00:e0:8b:09:f0:04 --> Host marketing switch(config-zone)# member pwwn 22:20:00:05:30:00:11:69 --> SVC target N-port switch(config-zone)# member pwwn 21:23:00:05:30:00:11:69 --> SVC target N-port switch(config-zone)# member pwwn 22:23:00:05:30:00:11:69 --> SVC target N-port switch(config-zone)# member pwwn 21:20:00:05:30:00:11:69 --> SVC target N-port Step 6 Exit to the MDS configuration mode and create a zone set called main-zset in VSAN 1 switch(config-zone)# exit switch(config)# zoneset name main-zset vsan 1 switch(config-zoneset)# Step 7 Step 8 Assign the host-finance zone and the host-marketing zone as members of the main-zset zone set. switch(config-zoneset)# member host-finance switch(config-zoneset)# member host-marketing Exit to the MDS configuration mode. switch(config-zoneset)# exit switch(config-zone)# exit switch(config)# Step 9 Activate the main-zset zoneset in VSAN 1. switch(config)# zoneset activate name main-zset vsan 1 Zoneset activation initiated. check zone status Step 10 Exit to the MDS EXEC mode and verify the active zoneset in VSAN 1. switch(config)# exit switch# show zoneset active vsan 1 zoneset name main-zset vsan 1 zone name host-finance vsan 1 * fcid 0x6a0200 [pwwn 21:00:00:e0:8b:09:e7:04] * fcid 0x6a0003 [pwwn 22:20:00:05:30:00:11:69] * fcid 0x6a0006 [pwwn 21:23:00:05:30:00:11:69] * fcid 0x6a0009 [pwwn 22:23:00:05:30:00:11:69] * fcid 0x6a000c [pwwn 21:20:00:05:30:00:11:69] zone name host-marketing vsan 1 6-2

55 Chapter 6 Configuring Hosts Isolating Host Traffic * fcid 0x6a0300 [pwwn 21:00:00:e0:8b:09:f0:04] * fcid 0x6a0003 [pwwn 22:20:00:05:30:00:11:69] * fcid 0x6a0006 [pwwn 21:23:00:05:30:00:11:69] * fcid 0x6a0009 [pwwn 22:23:00:05:30:00:11:69] * fcid 0x6a000c [pwwn 21:20:00:05:30:00:11:69] zone name $default_zone$vsan 1 You have now created the host VSAN and two zones host-finance and host-marketing (see Figure 6-1). Figure 6-1 Identifying the Target s host-marketing host-finance Host VSAN VSAN 1 Target Target Target Target Node 3 Initiator Mgmt Node 1 Initiator Mgmt Management VSAN VSAN 2 Mgmt Node 2 Initiator Mgmt Node 4 Initiator Disk VSAN VSAN

56 Creating Hosts Chapter 6 Configuring Hosts Creating Hosts To create a host, follow these steps. Step 1 Step 2 Use the show cluster cluster-name host candidate command to obtain a list of candidate hosts. switch# svc-config switch(svc)# show cluster SampleCluster host candidate id pwwn :00:00:e0:8b:09:e7:04 Use the host candidate s pwwn to identify the host to be mapped. Enter the cluster configuration submode. switch(svc)# cluster config SampleCluster switch(svc-cluster)# Step 3 Create a host called Finance1 with the pwwn identified in Step 1. switch(svc-cluster)# host add Finance1 hostport 21:00:00:e0:8b:09:e7:04 Step 4 Step 5 Exit to the SVC configuration mode. switch(svc-cluster)# exit switch(svc)# Use the show cluster cluster-name host command to verify that the newly added host displays the number of configured ports. switch(svc)# show cluster SampleCluster host name number of ports Finance1 1 Mapping VDisks to Hosts To map VDisks to hosts, follow these steps. Step 1 Step 2 Step 3 Enter the cluster configuration submode. switch(svc)# cluster config SampleCluster switch(svc-cluster)# Enter the host configuration submode for Finance1. switch(svc-cluster)# host name Finance1 switch(svc-cluster-host)# Map the three VDisks to Finance1 switch(svc-cluster-host)# map vdisk crm-data SCSI-lun 10 switch(svc-cluster-host)# map vdisk crm-idx switch(svc-cluster-host)# map vdisk crm-log 6-4

57 Chapter 6 Configuring Hosts Configuring iscsi Hosts in SVC Note The optional SCSI-lun 10 option allows the customer to specify the LUN value that is mapped to this VDisk, otherwise the cluster automatically picks the lowest available value. Step 4 Exit to the SVC configuration mode. switch(svc-cluster-host)# exit switch(svc-cluster)# exit switch(svc)# Step 5 Verify that Finance1 has one port with the configured pwwn switch(svc)# show cluster SampleCluster host Finance1 host Finance1: Number of port is 1 Port WWN is 21:00:00:e0:8b:09:e7:04 LUN0: vdik crm-idx LUN1: vdisk crm-log LUN10: vdisk crm-data Configuring iscsi Hosts in SVC This section provides the configuration procedure to allow an iscsi host to access VDisks exported by an SVC cluster. Tip Linux users should configure the /etc/iscsi.conf file using the following parameters: Multipath=no HostIPsforMP=<ip address of NIC1>,<ip address of NIC2> ConnFailTimeout=50 Refer to the Cisco MDS 9000 Family Configuration Guide for further details on iscsi concepts and configuration options. Note Before configuring the iscsi Hosts, be sure to configure the required level of iscsi authentication. Refer to the Cisco MDS 9000 Family Configuration Guide for further information on the available iscsi authentication option. The following example uses the null authentication option. It also displays the configuration for two iscsi hosts. To configure two iscsi hosts to access VDisks exported by an SVC cluster, follow these steps. Step 1 Configure iscsi to dynamically import all Fibre Channel targets into the iscsi SAN using auto-generated iscsi target names. switch# conf t switch(config)# iscsi import target fc Step 2 Configure the Gigabit Ethernet interface in slot 4 port 1 with an IP address and enable the interface. switch(config)# int gigabitethernet 4/1 6-5

58 Configuring iscsi Hosts in SVC Chapter 6 Configuring Hosts switch(config-if)# ip address switch(config-if)# no shut switch(config-if)# exit switch(config)# Step 3 Step 4 Configure the iscsi interface in slot 4 port 1 and enable the interface. switch(config)# int iscsi 4/1 switch(config-if)# no shut switch(config-if)# exit switch(config)# Configure static pwwns for each iscsi initiator. The first iscsi initiator is identified using the IQN name one nwwn and one pwwns from the switch's Fibre Channel WWN pool are allocated in the SVC target N-port VSAN (in this example, VSAN 1 See Figure 6-1) keep the mapping permanent: switch(config)# iscsi initiator name iqn com.cisco:01.e41695d16b1a switch(config-(iscsi-init))# vsan 1 switch(config-(iscsi-init))# static pwwn system-assign 1 switch(config-(iscsi-init))# static nwwn system-assign switch(config-(iscsi-init))# exit switch(config)# The second iscsi initiator is identified using the IP address one pwwn from the switch s Fibre Channel WWN pool is assigned in the SVC target N-port VSAN (in this example, VSAN 1 See Figure 6-1): Step 5 switch(config)# iscsi initiator ip address switch(config-(iscsi-init))# vsan 1 switch(config-(iscsi-init))# static pwwn system-assigned 1 switch(config-(iscsi-init))# end switch# View the configured initiators. The WWNs are automatically assigned by the system. switch# show iscsi initiator configured iscsi Node name is iqn com.cisco:01.e41695d16b1a Member of vsans: 1 Node WWN is 20:03:00:0b:fd:44:68:c2 No. of PWWN: 1 Port WWN is 21:00:00:e1:8b:09:e7:04 iscsi Node name is Member of vsans: 1 No. of PWWN: 1 Port WWN is 20:06:00:0b:fd:44:68:c2 Step 6 Step 7 Add the host to the same zone as the SVC target s. Zone membership for the iscsi initiator can either be the iscsi symbolic node name or the pwwn. In this case, the pwwn can be used since it is statically created. switch# conf t switch(config)# zone name host-finance vsan 1 switch(config-zone)# Assign the membership. The following example is based on the symbolic node name. switch(config-zone)# member symbolic-nodename iqn com.cisco:01.e41695d16b1a The following example is based on the persistent pwwn assigned to the initiator. You can obtain the pwwn from the output of the show iscsi initiator command. 6-6

59 Chapter 6 Configuring Hosts Configuring iscsi Hosts in SVC switch(config-zone)# member pwwn 20:06:00:0b:fd:44:68:c2 switch(config-zone)# end switch# Step 8 Create a SVC host using this pwwn switch# svc-config switch(svc)# show cluster SampleCluster host candidate id pwwn :00:00:e1:8b:09:e7: :06:00:0b:fd:44:68:c2 switch(svc)# cluster config SampleCluster switch(svc-cluster)# host add iscsi1 hostport 21:00:00:e1:8b:09:e7:04 switch(svc-cluster)# host add iscsi2 hostport 20:06:00:0b:fd:44:68:c2 Step 9 Step 10 Map VDisks to both hosts. switch(svc-cluster)# host name iscsi1 switch(svc-cluster-host)# map vdisk crm-data switch(svc-cluster-host)# exit switch(svc-cluster)# host name iscsi2 switch(svc-cluster-host)# map vdisk crm-data switch(svc-cluster-host)# end switch# Start the iscsi clients on both hosts and verify that the sessions come up using the show iscsi session command. 6-7

60 Configuring iscsi Hosts in SVC Chapter 6 Configuring Hosts 6-8

61 CHAPTER 7 Configuring Copy Services The SVC copy services function available in all Cisco MDS 9216 switches and directors in the Cisco MDS 9500 Family enables you to copy virtual Disks (VDisks). These copy services include data migration, FlashCopy TM, and Remote Copy. This chapter includes the following sections: Data Migration, page 7-2 FlashCopy, page 7-4 Remote Copy, page

62 Data Migration Chapter 7 Configuring Copy Services Data Migration The Data Migration feature allows you to change the mapping of VDisk extents to MDisk extents without interrupting a host s access to that VDisk. Data Migration Application You can perform data migration on any VDisk managed by the SVC for the following purposes: Redistributing workload within a Cluster across back-end storage. Moving workload onto newly installed storage. Moving workload off old/failing storage Moving workload to rebalance a changed workload Migrating in data from legacy back-end storage to SVC managed storage Data Migration Operation After a data migration, the virtualisation policy of the VDisk is set to striped. When you request an operation which requires some migration activity, SVC checks that sufficient suitable free extents exist to complete the requested migration operation. If this is not the case then the migration will not be started and the configuration command fails. However, once the operation starts, SVC does not reserve free extents for the future use of the migration activity. This means that another migration activity can be started to contend for free extents with existing migrations. If this happens, and no suitable free extent is available to proceed with the migration, then the migration operation fails. The implications of this are that migration between MDisk groups will normally be suspended and all other types of migration is normally stopped. Migrating VDisks between MDisk groups To migrate VDisks from one managed MDisk group to another MDisk group, follow these steps. Step 1 Step 2 Step 3 Step 4 Create a cluster and the required VDisks (see Chapter 3, Creating and Managing Clusters ). Display the configured VDisks in the newly-created cluster (called SampleCluster). switch(svc)# show cluster SampleCluster vdisk Name Capacity iogroup mdisk-grp name Policy Status fn-data 1.00 GB 1 finance1 striped online fn-log 1.00 GB 1 finance1 striped online switch(svc)# Create the new MDisk group to migrate the VDisks (see Chapter 4, Managing Back-End Storage ). Display the configured MDisk group (called finance2). switch(svc)# show cluster SampleCluster mdisk-grp finance2 mdisk-grp finance2 is online Total capacity is GB Free capacity is GB Extent size is 16 MB 7-2

63 Chapter 7 Configuring Copy Services Data Migration Number of mdisks is 1 Number of vdisks using this group is 0 Tip In order to migrate a VDisk from its existing MDisk group to a new one MDisk group, both the MDisk groups must have the same extent size. Step 5 Step 6 Step 7 Step 8 Step 9 Enter the cluster configuration submode for the cluster called SampleCluster. switch(svc)# cluster config SampleCluster switch(svc-cluster)# Issue the migration command to the new MDisk group. switch(svc-cluster)# migrate vdisk fn-data new-mdisk-grp finance2 switch(svc-cluster)# migrate vdisk fn-log new-mdisk-grp finance2 switch(svc-cluster)# Exit to the svc-config submode switch(svc-cluster)# exit switch(svc)# Check the progress of the migration by issuing the status command. switch(svc)# show cluster SampleCluster status migration migrating vdisk fn-data to mdisk grp finance2 : 68% migrating vdisk fn-log to mdisk grp finance2 : 0% switch(svc)# Verify that the migration has completed, by using the show VDisk command. The mdisk-grp name column should have the new MDisk group name. switch(svc)# show cluster SampleCluster vdisk Name Capacity iogroup mdisk-grp name Policy Status fn-data 1.00 GB 1 finance2 striped online fn-log 1.00 GB 1 finance2 striped online switch(svc)# 7-3

64 FlashCopy Chapter 7 Configuring Copy Services FlashCopy FlashCopy copies a set of source VDisks to a set of target VDisks. The original contents of the target VDisks are lost. After the copy operation is completed, the target VDisks have the contents of the source VDisks as they existed at a single point-in-time, when the FlashCopy was started. Although the copy operation takes a finite amount of time to complete, the resulting data at the target appears as if the copy was made instantaneously. FlashCopy is sometimes described as an instance of a Time-Zero copy (T0) or point-in-time copy technology. This time is much less than the time required to copy the same data using conventional techniques. Point-in-time copy techniques are used to help solve the problem of making a consistent copy of a data set which is being constantly updated. If a copy of a data set is taken using a technology which did not provide point-in-time semantics and the data set changes during the copy operation then the resulting copy may contain data which is not consistent with the latest version. FlashCopy Applications The following list provides some examples of using FlashCopy services: An important use of FlashCopy service is for taking consistent backups of changing data. In this usage, a FlashCopy is created in order to capture a point-in-time and the resulting image is backed up to tertiary storage such as tape. In this case the FlashCopy target is primarily treated as read only although a few writes are occasionally involved. Another use of FlashCopy is for application testing. In a business setting it is often important to test a new version of an application on real business data prior to updating the production copy of the software. This reduces the risk of the software upgrade failing because it is incompatible with the actual data in use at the time of the update. This application requires read/write access to the FlashCopy target. Other uses of FlashCopy in the business environment include creating copies for auditing purposes and for data mining. In the scientific and technical arena one way in which FlashCopy is employed is to create restart points for long running batch jobs. This means that if a batch job fails many days into its run it may be possible to restart the job from a saved copy of its data rather than re-running the entire multi-day job. FlashCopy Mapping FlashCopy mapping is done between a source VDisk and a target VDisk. The VDisks must be the same size. FlashCopying part of a VDisk is not supported. The source and target VDisks must both be managed by the same SVC cluster but may be in different I/O groups within that cluster. Each VDisk may be a member of only one FlashCopy mapping. VDisks participating in FlashCopy mapping cannot have their size increased or decreased while they remain participants of the FlashCopy mapping. 7-4

65 Chapter 7 Configuring Copy Services FlashCopy FlashCopy Consistency Groups A FlashCopy consistency group contains a number of FlashCopy mappings. A consistency group can contain an arbitrary number of FlashCopy mappings up to the maximum number of FlashCopy mappings supported by a SVC cluster. When the start command is issued for a consistency group, all of the FlashCopy mappings in the consistency group are started at the same time, resulting in a point-in-time copy which is consistent across all of the FlashCopy mappings which are contained in the consistency group. The FlashCopy function accepts one of two mode to copy the source VDisk contents to the destination VDisk full or copy-on-write. Full mode The source VDisk is copied to the destination VDisk even if the source VDisk is not changed after the FlashCopy operation is started. The copy process is implemented at the rate specified by the rate parameter (see Table 7-1). Table 7-1 Copy Rate Translation Copy-rate KB/sec to to MB Copy-on-write mode The source VDisk is only copied to the destination VDisk if the source VDisk is changed (by a write operation) after the FlashCopy operation is started. To configure consistency groups, follow these steps. Step 1 Step 2 Create the source, and target VDisks for FlashCopy (Chapter 5, Managing Virtual Disks ). Verify the VDisk configuration. switch(svc)# show cluster SampleCluster vdisk Name Capacity iogroup mdisk-grp name Policy Status fndata-snapshot 1.00 GB 1 finance striped online fnlog 2.00 GB 1 finance striped online fnlog-snapshot 2.00 GB 1 finance striped online fndata 1.00 GB 1 finance striped online Note Step 3 Step 4 The target VDisk, and the source VDisk in a FlashCopy mapping need to be of same size. Enter the cluster configuration submode switch(svc)# cluster config SampleCluster switch(svc-cluster)# Enable the FlashCopy feature. switch(svc-cluster)# feature enable flash-copy 7-5

66 FlashCopy Chapter 7 Configuring Copy Services Note Step 5 Step 6 Step 7 Step 8 Step 9 Step 10 Step 11 The FlashCopy feature must be licensed and enabled before configuring any FlashCopy mappings. Create a FlashCopy consistency group. switch(svc-cluster)# flash-copy add fcgrp Enter the FlashCopy consistency group submode switch(svc-cluster)# flash-copy name fcgrp switch(svc-cluster-flash-copy)# Map the source VDisk to the target VDisk. switch(svc-cluster-flash-copy)# map src-vdisk fnlog dst-vdisk fnlog-snapshot switch(svc-cluster-flash-copy)# map src-vdisk fndata dst-vdisk fndata-snapshot Refer to Table 7-1 and obtain the required copy rate. The copy rate specifies the rate of background copy. It is expressed as a percentage, and its translation to bandwidth is given below. If the optional copy rate is not configured, a default rate of 50 is configured. This example uses the full mode with a copy rate of 90. Configure the copy rate for the FlashCopy group. switch(svc-cluster-flash-copy)# mode full rate 90 Exit to the SVC configuration submode switch(svc-cluster-flash-copy)# exit switch(svc-cluster)# exit switch(svc)# Verify the FlashCopy configuration. switch(svc)# show cluster SampleCluster flash-copy fcgrp Flash-copy mapping 1: src vdisk is fnlog dest vdisk is fnlog-snapshot state is idle_or_copied copy rate is 90 progress 0% done Flash-copy mapping 2: src vdisk is fndata dest vdisk is fndata-snapshot state is idle_or_copied copy rate is 90 progress 0% done Starting FlashCopy Step 12 Refer to the Starting the FlashCopy section on page 7-6 to start the FlashCopy configuration. Starting the FlashCopy You can only start the FlashCopy process if you have configured a consistency group as specified in the FlashCopy Consistency Groups section on page 7-5. To start the FlashCopy process, follow these steps: 7-6

67 Chapter 7 Configuring Copy Services FlashCopy Step 1 Step 2 Step 3 Step 4 Step 5 Check the status of the FlashCopy consistency group. switch(svc)# show cluster SampleCluster flash-copy Name Status fccstgrp0 idle_or_copied fcgrp idle_or_copied Issue the cluster name cluster-name flash-copy fcgrp prepare command to prepare the source and target VDisks for FlashCopy. This will flush the cache of any data destined for the source VDisk and force the cache into write through until the FlashCopy is started. switch(svc)# cluster name SampleCluster flash-copy fcgrp prepare Check the status of the FlashCopy consistency group, to make sure that the prepare operation is completed. If the prepare operation is completed, the status for the group will be prepared. If the prepare operation is not completed, wait till it completes. switch(svc)# show cluster SampleCluster flash-copy Name Status fccstgrp0 idle_or_copied fcgrp prepared Once, the FlashCopy consistency group is prepared for FlashCopy, issue the cluster name cluster-name flash-copy fcgrp start command to start the FlashCopy. This makes a point-in-time copy of the source VDisk the moment the command is executed. switch(svc)# cluster name SampleCluster flash-copy fcgrp start Check the progress of the FlashCopy operation. switch(svc)# show cluster SampleCluster status flash-copy fcgrp src vdisk dest vdisk progress fnlog fnlog-snapshot 46% fndata fndata-snapshot 6% If the progress fields indicate 100%, for all the mappings in the FlashCopy consistency group, then the FlashCopy is completed. Stopping FlashCopy A FlashCopy, once started, can be stopped by issuing the cluster name cluster-name flash-copy fcgrp stop command. Once stopped, use the cluster name cluster-name flash-copy fcgrp prepare command for the FlashCopy group before it is started. 7-7

68 Remote Copy Chapter 7 Configuring Copy Services Remote Copy Remote copy is a feature that seeks to maintain two copies of a data set. The relationship between the two copies is not symmetric. One copy of the data set is considered the primary copy or the source. This copy provides the reference for normal run-time operation. Updates to this copy are shadowed to a secondary copy or the auxiliary. The secondary copy is not normally referenced for performing I/O. This release of SVC supports synchronous remote copy. Synchronous remote copy ensures that updates are committed at both primary and secondary before the application is given completion to an update. This ensures that the secondary is fully up-to-date should it be needed in a failover. However, this means that the application is fully exposed to the latency and bandwidth limitations of the communication link to the secondary. Disaster Recovery If the primary copy fails, you can enable the secondary copy for I/O operation. A typical use of this function may involve two sites where the first provides service during normal running and the second is only activated when a failure of the first site is detected. The secondary copy is not accessible for application I/O other than the I/Os that are performed for the remote copy process itself. Enabling the secondary copy for active operation require some SVC, operating system and possibly application specific configuration. Enabling the secondary copy needs to be performed as part of the entire failover process. The SVC cluster at the secondary must be instructed to Stop the relationship which will have the affect of making the secondary logical unit accessible for normal I/O access. The operating system might need to mount file systems. The application might have some log of work to recover. Remote Copy Relationships A remote copy relationship consists of two VDisks a source VDisk and an auxiliary VDisk. In the most common use of remote copy the source VDisk contains a production copy of the data and is the VDisk that normally used by an application. The auxiliary VDisk contains a backup copy of the data and is used in disaster recovery scenarios. Both VDisks in a remote copy relationship must be of the same size. Both VDisks can be in the same cluster (and consequently, the same I/O group). Optionally, the VDisks can come from two clusters that are configured to recognize each other and can be in any I/O group in each of those two clusters. Each VDisk in a remote copy relationship takes on a specific role, behaving as a primary or a secondary. A primary VDisk contains a valid copy of application data and is accessible for application write I/O. A secondary VDisk contains a valid copy of application data but is not available for application write I/O. Remote Copy Consistency Groups Certain uses of remote copy require the manipulation of more than one relationship. Remote copy provides the ability to group relationships so that they are manipulated in unison. For some uses it might be that the relationships share a loose association and that the grouping simply provides a convenience for the administrator. But a more significant use arises where the relationships contain VDisks that have 7-8

69 Chapter 7 Configuring Copy Services Remote Copy a tighter association. For example, if an application s data is spread across more than one VDisk. A more complex example is where multiple applications run on different host systems, where each application has data on different VDisks, and these applications exchange data with each other. A consistency group can contain zero or more relationship. All relationships in a consistency group must have matching source and auxiliary clusters. Configuring Remote Copy To configure remote copy, follow these steps. Note In this example, we choose separate local, and remote clusters, although they can be the same cluster. Step 1 Step 2 Create the local, and remote cluster (see Chapter 3, Creating and Managing Clusters ). Create the Virtual Disks in the local, and remote cluster that form part of the remote copy relationship (Chapter 4, Managing Back-End Storage ). local-switch(svc)# show cluster local-cluster vdisk Name Capacity iogroup mdisk-grp name Policy Status fndata-src 2.00 GB 1 finance striped online fnlog-src 1.00 GB 1 finance striped online remote-switch(svc)# show cluster remote-cluster vdisk Name Capacity iogroup mdisk-grp name Policy Status fnlog-aux 1.00 GB 1 finance striped online fndata-aux 2.00 GB 1 finance striped online Step 3 Step 4 Enter the cluster config submode of the local cluster local-switch(svc)# cluster config local-cluster local-switch(svc-cluster)# Enable the remote copy feature in the local cluster local-switch(svc-cluster)# feature enable remote-copy local-switch(svc-cluster)# Note Step 5 Step 6 Step 7 The remote copy feature must be licensed and enabled before configuring remote copy. Enter the cluster config submode of the remote cluster remote-switch(svc)# cluster config remote-cluster remote-switch(svc-cluster)# Enable the remote copy feature in the remote cluster remote-switch(svc-cluster)# feature enable remote-copy remote-switch(svc-cluster)# Establish a remote copy partnership with the remote cluster at the local cluster. local-switch(svc-cluster)# remote-copy cluster remote-cluster local-switch(svc-cluster)# 7-9

70 Remote Copy Chapter 7 Configuring Copy Services Step 8 Step 9 Step 10 Establish a remote copy partnership with the local cluster at the remote cluster. remote-switch(svc-cluster)# remote-copy cluster local-cluster remote-switch(svc-cluster)# Create a remote copy consistency group in the local cluster. local-switch(svc-cluster)# remote-copy add rcgrp cluster remote local-switch(svc-cluster)# Enter the remote copy consistency group submode in the local cluster. local-switch(svc-cluster)# remote-copy name rcgrp local-switch(svc-cluster-remote-copy)# Step 11 Step 12 Step 13 Create remote copy relationships between the source VDisks, and auxiliary VDisks, under the remote copy consistency group. local-switch(svc-cluster-remote-copy)# map src-vdisk fndata-src aux-vdisk fndata-aux local-switch(svc-cluster-remote-copy)# map src-vdisk fnlog-src aux-vdisk fnlog-aux local-switch(svc-cluster-remote-copy)# Return to the svc-config submode in the local switch. local-switch(svc-cluster-remote-copy)# exit local-switch(svc-cluster)# exit local-switch(svc)# Verify the remote copy consistency group configuration. local-switch(svc)# show cluster local-cluster remote-copy rcgrp Remote-copy mapping 1: master cluster is local-cluster master vdisk is fndata-src aux cluster is remote-cluster aux vdisk is fndata-aux status is inconsistent_stopped progress 0% done Remote-copy mapping 2: master cluster is local-cluster master vdisk is fnlog-src aux cluster is remote-cluster aux vdisk is fnlog-aux status is inconsistent_stopped progress 0% done local-switch(svc)# Starting Remote Copy Once the remote copy configuration is completed for a consistency group, start (activate) the remote copy relationships by issuing the start command. This command triggers the background copy from the corresponding source VDisks to the auxiliary VDisks respectively. Any subsequent writes issued to a particular source VDisk are also mirrored in the corresponding auxiliary VDisk, in the remote copy relationship. To start remote copy relationships, follow these steps: 7-10

71 Chapter 7 Configuring Copy Services Remote Copy Step 1 Step 2 Step 3 Step 4 Enter the svc-config submode in the local switch. local-switch# svc-config local-switch(svc)# Issue the start command for the remote copy consistency group to activate the remote copy relationships in the consistency group. local-switch(svc)# cluster name local-cluster remote-copy rcgrp start local-switch(svc)# Check the progress of the remote copy by issuing the Status command. local-switch(svc)# show cluster local-cluster status remote-copy rcgrp src vdisk dest vdisk progress fndata-src fndata-aux 8% fnlog-src fnlog-aux 16% local-switch(svc)# Once, the background copy of the data in source VDisks to auxiliary VDisks are completed, the status of all the relationships in the consistency group will be consistent_synchronized, and the auxiliary VDisks will be up to date with the corresponding source VDisks. local-switch(svc)# show cluster local-cluster remote-copy rcgrp Remote-copy mapping 1: master cluster is local-cluster master vdisk is fndata-src aux cluster is remote-cluster aux vdisk is fndata-aux status is consistent_synchronised progress 100% done Remote-copy mapping 2: master cluster is local-cluster master vdisk is fnlog-src aux cluster is remote-cluster aux vdisk is fnlog-aux status is consistent_synchronised progress 100% done local-switch(svc)# Stopping Remote Copy Use the stop command to suspend the remote copy relationships in a consistency group. Subsequently, the writes issued to a source VDisk, in a relationship, are not reflected at the corresponding auxiliary VDisk. local-switch(svc)# cluster name local-cluster remote-copy rcgrp stop local-switch(svc)# 7-11

72 Remote Copy Chapter 7 Configuring Copy Services Failover and Recovery Process When remote copy relationships are active during a normal operation, the source VDisks in the relationships are fully accessible, and the auxiliary VDisks are not available for write operations. After a failure in the local (master) cluster, you must stop the remote copy relationship using the aux-enable option, to allow write access to the auxiliary VDisks in the remote cluster. remote-cluster(svc)# cluster name remote-cluster remote-copy rcgrp stop aux-enable remote-cluster(svc)# If the local cluster comes back up, the administrator can choose to resume the remote copy relationships in one of two ways. Resume the remote copy relationships with the local cluster acting as the primary or master of the relationships. Resume the remote relationships with the remote cluster acting as the primary or master of the relationships. In either case, the force option is required when you resume the remote copy background copy is required to make the source VDisks, and auxiliary VDisks up to date. The following command resumes the remote copy relationships in the consistency group with the local cluster as the primary. local-cluster(svc)# cluster name local-cluster remote-copy rcgrp start force local-cluster(svc)# The following commands resume the remote copy relationships in the consistency group with the remote cluster as the primary, by enabling the aux option in the start command. remote-cluster(svc)# cluster name remote-cluster remote-copy rcgrp start aux force remote-cluster(svc)# Verify the primary configuration for a remote copy consistency group using the show remote-copy command. The Primary column indicate whether the auxiliary VDisks or source VDisks have the primary (or master) role. remote-cluster(svc)# show cluster remote-cluster remote-copy Name Remote Cluster Mappings Primary Status rcgrp remote 2 aux consistent_synchronised remote-cluster(svc)# 7-12

73 CHAPTER 8 Upgrading CSM Software When CSMs are present in any switch in the Cisco MDS 9000 Family, several kinds of upgrade may be performed as required a cluster software upgrade, an automatic upgrade when nodes are added, a service mode upgrade, or a switch software upgrade. This section also explains the process to manage pwwns and nwwns when CSM modules are replaced in any switch in the Cisco MDS 9000 Family switch. This chapter includes the following sections: Upgrading Clusters, page 8-2 Automatic Upgrade during Node Addition, page 8-5 Upgrading in Service Mode, page 8-5 Upgrading the Switch Software, page 8-7 Replacing CSMs, page

74 Upgrading Clusters Chapter 8 Upgrading CSM Software Upgrading Clusters Cluster software upgrades are done concurrently with user I/O operations and selected management activities. Note When software upgrades are in progress, not all nodes in the cluster are operational depending on the upgrade phase. Consequently, the cache operates in write through mode. Tip Configuration changes are disallowed from the time the upgrade is started until the upgrade operation has terminated successfully. In case of failure, the upgraded and/or failed nodes must be reverted to the original software version before configuration changes are permitted. If any configuration change is attempted during this time, an error message is issued to indicate that an upgrade is in progress. Cluster Upgrade Prerequisites To prepare a switch for cluster software upgrade, follow these steps. Step 1 Wait for all data migration to complete. The time taken for the data migration depends on the size of the VDisks being migrated. Caution Step 2 Step 3 Data migration, once started, cannot be stopped. Wait for all FlashCopy mappings to complete or stop them. If you choose to stop the FlashCopy mapping, this will result in the FlashCopy targets going offline. The FlashCopy must be prepared and started again in order to restart FlashCopy. This procedure results in the FlashCopy point-in-time being lost. Stop all remote copy relationships. Recognizing Failure Situations Cluster software upgrade will fail if one of the following situations apply: Any node configured to be a member of the cluster is not present in order to upgrade the software the node must either be deleted from the cluster or must be brought online. All the nodes configured into the cluster do not currently have the same software level may happen as result of a failed back out or a service mode upgrade. This behavior cannot be overridden using the force option. The new SVC software image is not compatible with the current switch or the SVC software level. This behavior cannot be overridden using the force option. A node has been deleted from the cluster such that any I/O group has only one member the upgrade process would result in loss of access to data if this were allowed. The force option can be used to override this restriction if you do not mind losing access to data during the upgrade. 8-2

75 Chapter 8 Upgrading CSM Software Upgrading Clusters Cluster Upgrade Guidelines Be aware of the following guidelines when upgrading clusters: Nodes are updated one at a time. One node from each I/O group is updated before the second node in any I/O group is updated. During the update of a node it does not participate in I/O activity in the I/O group. Thus all I/O activity for the VDisks in the I/O group is directed to the other node in the I/O group by the host multipathing software. During a node upgrade, the other node in the I/O group will attempt to flush the cache and change operations to write-through mode. This flush is not guaranteed to be successful or complete. Should the remaining node in an I/O group experience a failure when its partner is being upgraded, then the only valid copy of the modified data in the cache could be lost. If this risk is unacceptable, then the customer must take steps to ensure that no I/O activity takes place during the software upgrade. A 30 minute delay is inserted into the procedure between updating the first and second nodes in each I/O group. This allows time for the host multipathing software to rediscover paths to the nodes that were updated first so that when the second set of nodes are updated loss of access does not result. The update is not committed until all nodes in the cluster have been successfully updated to the new code level. When code upgrade starts an entry is made in the event log and another entry is made when the upgrade completes or fails. The config node goes down to upgrade to the new software level. This will cause a different node to become the new config node. At this point, you need to reconnect to the cluster IP address. Performing the Cluster Upgrade Step 1 Issue the cluster name cluster-name upgrade svc-system command at the configuration node of the cluster. Tip If the cluster contains an I/O group with a single node, use the optional force keyword at the end of this command. switch(svc)# cluster name SampleCluster upgrade svc-system scp://userc@ /auto/andusr/userc/hkrel/vegassw/build/gdb.sb-avanti/isan/targetfs /m9000-ek9-csm-svc-mzg.1.3.x.bin userc@ 's password: m9000-ek9-csm-svc-mz 100% *************************************** 108 MB 01:23 This process may take few minutes as the config node performs the following checks: If the new SVC software image is compatible with the config node s running switch software and running SVC software. If any compatibility checks fail, cluster upgrade fails and an appropriate error message is issued. If the SVC software image is compatible with the software running on each of the switches where the other nodes in the cluster reside. The config node collects the compatibility check result from each node in the cluster and makes sure that all of them have succeeded. If any of nodes report a check failure, then an error message is issued with a table displaying all the check failures across all the nodes in the cluster. 8-3

76 Upgrading Clusters Chapter 8 Upgrading CSM Software Step 2 If all the nodes in the cluster are running the same level of code and all nodes are present. If the new SVC software image has been successfully transmitted to all the nodes in the cluster. If all the checks succeed in Step 1 the upgrade process begins. Verify that cluster upgrade has succeeded by issuing one of two commands in SVC configuration mode: the show node local command Ensure that the node software version corresponds to the new node software version. the show cluster SampleCluster nodevpd command Ensure that the software upgrade complete event message is in the supervisor syslog. Sample Cluster Upgrade switch(svc)# show node local Config Cluster Node Sw Node Cluster node status status version svc3/1 SampleCluster No active active 1.3(1) svc3/2 SampleCluster No active active 1.3(1) svc7/1 SampleCluster Yes active active 1.3(1) svc7/2 SampleCluster No active active 1.3(1) switch(svc)# cluster name SampleCluster upgrade svc-system scp://userc@ /auto/andusr/userc/hkrel/vegassw/build/gdb.sb-avanti/isan/targetfs /m9000-ek9-csm-svc-mzg.1.3.x.bin userc@ 's password: m9000-ek9-csm-svc-mz 100% *************************************** 108 MB 01:23 Note The cluster upgrade process has initiated successfully when the prompt returns. switch(svc)# show node local Config Cluster Node Sw Node Cluster node status status version svc3/1 SampleCluster No active active 1.3(x) svc3/2 SampleCluster No active active 1.3(1) svc8/1 SampleCluster Yes active active 1.3(1) svc8/2 SampleCluster No active active 1.3(1) In this example, node svc4/1 has completed upgrade to the new version of the svc software. 8-4

77 Chapter 8 Upgrading CSM Software Automatic Upgrade during Node Addition Automatic Upgrade during Node Addition If a node is running a SVC software version which is different from the cluster version, and that node is added to the cluster, the config node automatically downloads the cluster software to the new node. If the cluster software is not compatible with the software running on the switch that contains the new node, the add node operation will fail. Upgrading in Service Mode Service mode software upgrade provides a recovery alternative if the cluster cannot accept a normal software upgrade. A single node is placed in service access mode to perform a service mode software upgrade. This is in contrast to the normal method for upgrading all the nodes in the cluster. Caution This upgrade process is only to be used by experienced switch administrators under the care of customer support representatives. To upgrade a single node in service mode, follow these steps. Step 1 Step 2 Step 3 Step 4 Issue the node svc slot/node servicemode command on the required node to place the node in service mode. switch(svc)# node svc 7/2 servicemode switch(svc)# Issue the show node local command in SVC configuration mode to verify that the required node is in servicemode. switch1(svc)# show nodes local Node cluster config cluster node sw node status status version svc3/1 SampleCluster No active active 1.3(1) svc3/2 SampleCluster No active active 1.3(1) svc7/1 SampleCluster Yes active active 1.3(1) svc7/2 SampleCluster No unconfigured servicemode 1.3(1) Issue the node svc command to begin the upgrade.' switch(svc)# node svc 7/2 upgrade svc-system scp://userc@ /auto/andusr/userc/hkrel/vegassw/build/gdb.sb-avanti/isan/targetfs /m9000-ek9-csm-svc-mzg.1.3.x.bin This command checks if the new SVC software image is compatible with the running switch software and the running SVC software. If any of these compatibility checks fail, then node upgrade fails with the appropriate error message to the user. Issue the show node local command in SVC configuration mode to verify that the required node is running with new node software. switch1(svc)# show nodes local Node cluster config cluster node sw node status status version 8-5

78 Upgrading in Service Mode Chapter 8 Upgrading CSM Software svc3/1 SampleCluster No active active 1.3(1) svc3/2 SampleCluster No active active 1.3(1) svc7/1 SampleCluster Yes active active 1.3(1) svc7/2 SampleCluster No active active 1.3(x) Note The node automatically exits the service mode when the upgrade is completed45. Sample Service Mode Upgrade switch(svc)# show nodes local Node cluster config cluster node sw node status status version svc3/1 SampleCluster No active active 1.3(1) svc3/2 SampleCluster No active active 1.3(1) svc7/1 SampleCluster Yes active active 1.3(1) svc7/2 SampleCluster No active active 1.3(1) switch(svc)# node svc 7/2 servicemode switch(svc)# show nodes local Node cluster config cluster node sw node status status version svc3/1 SampleCluster No active active 1.3(1) svc3/2 SampleCluster No active active 1.3(1) svc7/1 SampleCluster Yes active active 1.3(1) svc7/2 SampleCluster No unconfigured servicemode 1.3(1) switch(svc)# node svc 7/2 upgrade svc-system scp://userc@ /auto/andusr/userc/hkrel/vegassw/build/gdb.sb-avanti/isan/targetfs /m9000-ek9-csm-svc-mzg.1.3.x.bin switch(svc)# show nodes local Node cluster config cluster node sw node status status version svc3/1 SampleCluster No active active 1.3(1) svc3/2 SampleCluster No active active 1.3(1) svc7/1 SampleCluster Yes active active 1.3(1) svc7/2 SampleCluster No active active 1.3(x) 8-6

79 Chapter 8 Upgrading CSM Software Upgrading the Switch Software Upgrading the Switch Software This upgrade requires a dual-supervisor MDS 9500 director. This procedure only upgrades the switch software not the SVC software that is running on the nodes in the cluster. Use the install all command to do switch software upgrade irrespective of the modules present in the switch. Performing the Switch Upgrade To perform an automated software upgrade on any switch, follow these steps: Step 1 Step 2 Log into the switch through the console, Telnet, or SSH port of the active supervisor. Perform the upgrade by issuing the install all command. switch# install all system bootflash:system-image kickstart bootflash:kickstart-image At this point, the following events happen: The installer checks if the new switch system software is compatible with the SVC software for both nodes in a CSM and for all CSMs in the switch. You will receive the following descriptive information on the intended changes to your system: a compatibility assessment for each module in the switch a upgrade assessment a module version comparison See the Sample Switch Upgrade section on page 8-7 for an example of this information. Once the effects of the command are presented, you can choose to continue or cancel when you see this question (the default is no): Do you want to continue y/n? [n] :y Step 3 Step 4 Select y if you choose to continue or n if you choose to cancel the upgrade. y = each CSM module is upgraded one at a time, with a gap of 30 minutes between each to ensure that only one node in the I/O group is down at any time. This allows time for the host multipathing software to rediscover paths to the modules containing nodes that were upgraded first. If incompatibility warnings exist, the incompatible CSM nodes are shut down after the upgrade. If incompatibilities do not exist, the CSM nodes are upgraded. n = the installation process is aborted. Exit the switch console and open a new terminal session to view the upgraded supervisor module using the show module command. Sample Switch Upgrade switch# install all system scp://usery@ /auto/vwsvkd/usery/m9500-sf1ek9-mzg.1.3.x.bin For scp://usery@ , please enter password: 8-7

80 Upgrading the Switch Software Chapter 8 Upgrading CSM Software Copying image from scp://usery@ /auto/vwsvkd/usery/m9500-sf1ek9-mzg.1.3.x.bin to bootflash:///m9500-sf1ek9-mz.1.3.x.bin. [####################] 100% -- SUCCESS Verifying image bootflash:///b96d [####################] 100% -- SUCCESS Verifying image bootflash:///m9500-sf1ek9-mzg.1.3.x.bin [####################] 100% -- SUCCESS Extracting svclc version from image bootflash:///m9500-sf1ek9-mzg.1.3.x. in. [####################] 100% -- SUCCESS Extracting slc version from image bootflash:///m9500-sf1ek9-mzg.1.3.x.bi. [####################] 100% -- SUCCESS Extracting system version from image bootflash:///m9500-sf1ek9-mzg.1.3.x bin. [####################] 100% -- SUCCESS Extracting kickstart version from image bootflash:///b96d. [####################] 100% -- SUCCESS Extracting loader version from image bootflash:///b96d. [####################] 100% -- SUCCESS Compatibility check is done: Module bootable Impact Install-type Reason yes disruptive reset To be installed sys image is incompatible with node 1 running image 2 yes disruptive reset To be installed sys image is incompatible with node 2 running image Images will be upgraded according to following table: Module Image Running-Version New-Version Upg-Required slc 1.3(1) 1.3(x) yes 1 bios v1.0.8(08/07/03) v1.1.0(10/24/03) yes 2 svclc 1.3(1) 1.3(x) yes 2 svcsb 1.3(3) 1.3(3) no 2 svcsb 1.3(3) 1.3(3) no 2 bios v1.0.8(08/07/03) v1.1.0(10/24/03) yes 3 svclc 1.3(1) 1.3(x) yes 3 svcsb 1.3(1) 1.3(1) no 3 svcsb 1.3(1) 1.3(1) no 3 bios v1.0.8(08/07/03) v1.1.0(10/24/03) yes 5 system 1.3(1) 1.3(x) yes 5 kickstart 1.3(1) 1.3(1) no 5 bios v1.0.8(08/07/03) v1.1.0(10/24/03) yes 5 loader 1.2(2) 1.2(2) no 6 system 1.3(1) 1.3(x) yes 6 kickstart 1.3(1) 1.3(1) no 6 bios v1.0.8(08/07/03) v1.1.0(10/24/03) yes 6 loader 1.2(2) 1.2(2) no 8-8

81 Chapter 8 Upgrading CSM Software Replacing CSMs 7 svclc 1.3(1) 1.3(x) yes 7 svcsb 1.3(1) 1.3(1) no 7 svcsb 1.3(1) 1.3(1) no 7 bios v1.0.8(08/07/03) v1.1.0(10/24/03) yes Do you want to continue with the installation (y/n)? [n] y Caution If you type yes at this point, the switch upgrade will proceed. Since the software running on the nodes 2/1 and 2/2 are not compatible, after the switchover the nodes 2/1, 2/2 are shutdown. Nodes 3/1, 3/2, 7/1, and 7/2 on the other hand will be up and running since there were no incompatibility warning messages. Tip If the cluster spans multiple switches, we recommend that all switches run the same version of the switch software. When upgrading switch software in a multi-switch environment, be sure to update one switch at a time. Replacing CSMs When you replace the CSM, you must ensure that the new CSM is using valid nwwns and pwwns. You may choose to install the new CSM in a different slot or in the same slot. The process to replace the CSM differs based on this decision. Tip To avoid the need to reconfigure servers and controllers, we recommend that you configure the replacement nodes with the same nwwns and pwwns as the replaced nodes. The procedure provide in this section follows this recommendation. Caution If nodes being replaced are given the same nwwns or pwwns as previous nodes that were participating in a cluster, they must be added to same I/O group and the same cluster as the nodes being replaced. Adding nodes with the same nwwns or pwwns (as the replaced nodes) to a different I/O group or cluster, can result in data corruption. Refer to the IBM TotalStorage Subsystem Device Driver User's Guide for more information. If the nodes being replaced are given new nwwns or pwwns, then perform the following additional steps after adding the nodes back to the cluster: Step 1 Step 2 At the end of the recovery process, follow the SDD procedure to discover the new paths and to check that each VPath is presenting the correct number of paths. Refer to the IBM TotalStorage Subsystem Device Driver User's Guide for more information. You may also need to modify the configuration of your disk controllers. If your controller uses a mapping technique to present its RAID arrays or partitions to the cluster, you must modify the port groups that belong to the cluster because the nwwn or pwwn's of the node have changed. 8-9

82 Replacing CSMs Chapter 8 Upgrading CSM Software Prerequisites to Replacing a CSM Step 1 Step 2 Step 3 Step 4 Use the show nodes local and show interface svc slot/node commands to obtain the following information for each of the nodes in the CSM: The slot number in which the CSM is located The node number within the CSM (1 or 2) The node name If the default name was used, then you cannot keep the same node name. The cluster (name) to which this node belongs The I/O group to which this node belongs The applicable nwwn and pwwns. The current software version on the node Document this information in an easily accessible location you will be using this information to upgrade the software after replacing a CSM. Verify that the node is not functional. Verify that the other node in its I/O group is operational before deleting this node. Delete the nodes to be replaced from their cluster(s). Replacing a CSM in the Same Slot If a CSM is replaced by another CSM in the same slot in any switch in the Cisco MDS 9500 Series or in a Cisco MDS 9216 Switch, the same nwwns and pwwns are automatically assigned for both interfaces on the new CSM. No further configuration is required. Caution If the replacement nodes are assigned the same nwwns and pwwns as the replaced nodes, be sure to assign the nodes to the same I/O group and cluster as before. Otherwise, data corruption may occur. If the information on which I/O group and cluster the previous nodes were part of is not available, contact your reseller (if applicable) or customer service for assistance. If a CSM is replaced by another CSM in the same slot on the same chassis, and you do not wish to retain the same nwwns and pwwns, follow this process. Step 1 Step 2 Remove the CSM from the slot as directed in the Cisco MDS 9216 Switch or the Cisco MDS 9500 Series Hardware Installation Guide. Use the svc purge-wwn module command to erase the nwwns and pwwns from the original slot. Issue the command after the module has been removed or when it is in the powered-down state. switch# svc purge-wwn module <slot-num> 8-10

83 Chapter 8 Upgrading CSM Software Replacing CSMs The old nwwns and pwwns will be lost from the system and never reassigned for any purpose in that chassis. Step 3 Replace the new CSM in the same slot as directed in the Cisco MDS 9216 Switch or the Cisco MDS 9500 Series Hardware Installation Guide. The interfaces on this new CSM will have brand new nwwn and pwwns. Replacing a CSM in a Different Slot If a CSM is replaced by another CSM in a different slot on the same or different chassis, and the same nwwns and pwwns are to be retained, follow this process. Step 1 Step 2 Step 3 Step 4 Step 5 Step 6 Step 7 Document the nwwn and pwwn values for the CSM before removing it. Remove the CSM from the slot as directed in the Cisco MDS 9216 Switch or the Cisco MDS 9500 Series Hardware Installation Guide. Use the following command to erase the nwwns and pwwns from the original slot. Issue the command when the module has already been removed or when it is in the powered-down state. switch# svc purge-wwn module <slot-num> The old nwwns and pwwns will be lost from the system and never reassigned for any purpose in that chassis. Replace the new CSM in the desired slot as directed in the Cisco MDS 9216 Switch or the Cisco MDS 9500 Series Hardware Installation Guide. Wait for the module to initialize. Configure the VSAN for the initiator, target, and management N-ports to match the replaced nodes: switch# config t switch (config)# interface svc new-slot-num / node-num switch (config-if)# initiator vsan vsan-id switch (config-if)# target vsan vsan-id switch (config-if)# mgmt vsan vsan-id Set the nwwn and pwwn values for the two interfaces within the module with the following commands: switch (config-if)# shutdown switch (config-if)# nwwn saved-nwwn-value switch (config-if)# initiator vsan vsan-id pwwn saved-pwwn-value switch (config-if)# target vsan vsan-id pwwn saved-pwwn-value switch (config-if)# mgmt vsan vsan-id pwwn saved-pwwn-value Tip Step 8 The related SVC interface must be shut down before setting the WWNs. Reload the CSM using the reload module slot-number command for the new nwwns to take effect. 8-11

84 Replacing CSMs Chapter 8 Upgrading CSM Software Post-Replacement Verification To perform a post-replacement verification, follow these steps. Step 1 Step 2 Step 3 Use the show nodes local command to verify that the nodes in the CSM have initialized without errors. If node does not initialize, the software version of the nodes on the replacement CSM may not be compatible with the version running on the switch. Wait for the nodes to initialize. Add the nodes back to their clusters (see the Chapter 8, Upgrading CSM Software ). Tip If the previous node used the default name, you cannot reassign the default name to the new node. If you assigned a name to the old node, the new node can be assigned the same name. Note Step 4 The Node ID of the replacement node is different from the node ID of the replaced node. Use the SDD management tool on the host systems to verify that all paths are online. 8-12

85 CHAPTER 9 Configuring SPAN on SVC Interfaces Effective SAN-OS Release 1.3(x) Switched Port Analyzer (SPAN) capabilities are also available on the Caching Services Module (CSM) module. The SPAN feature is specific to switches in the Cisco MDS 9000 Family. It monitors network traffic though a interface. Traffic through any SVC interface can be replicated to a special port called the SPAN destination port (SD port). This chapter includes the following sections: About SVC as a SPAN Source, page 9-2 Configuring SVC Interfaces as SPAN Sources, page

86 About SVC as a SPAN Source Chapter 9 Configuring SPAN on SVC Interfaces About SVC as a SPAN Source SPAN sources refer to the interfaces from which traffic can be monitored. You can also specify CSM nodes as a SPAN source, in which case, the SVC virtual interfaces (N-ports) are included as SPAN sources. You can choose the SPAN traffic in the ingress direction, the egress direction, or both directions for any source interface (see Figure 9-1). Figure 9-1 SVC Interfaces as SPAN Sources Initiator Host VSAN Management VSAN Disk VSAN rx tx FC traffic that can be spanned rx tx tx rx Target Mgmt Initiator Caching Services Node (svc1/1) You can also specify the traffic type (initiator traffic, target traffic, or management traffic) in both the ingress and egress directions or in either direction. To specify this option, use the traffic-type parameter. Refer to the Multiple Initiators and Targets section on page 2-9 for additonal information on configuring multiple N-port VSANs. Configuring SVC Interfaces as SPAN Sources To create SVC interfaces as SPAN sources, follow these steps. Step 1 Step 2 Step 3 Step 4 Enter the MDS configuration mode. switch# config t switch(config)# Configure a SPAN session. switch(config)# span session 2 switch(config-span)# If the session does not exist, it will be created. Configure the destination interface for the SPAN session. switch(config-span)# destination interface fc1/9 Configure the source interface svc1/1 for the initiator, target, and mgmt traffic-types in both directions. 9-2

87 Chapter 9 Configuring SPAN on SVC Interfaces Using VSAN Filters to Specify Traffic switch(config-span)# source interface svc1/1 Note Step 5 Step 6 Step 7 The traffic-type option is specific to SVC interfaces and cannot be used with any other SPAN source interfaces type. Configure the source interface svc2/1 for all initiator and mgmt traffic in the rx direction, and for all target traffic in the tx direction. switch(config-span)# source interface svc2/1 rx traffic-type initiator switch(config-span)# source interface svc2/1 rx traffic-type mgmt switch(config-span)# source interface svc2/1 tx traffic-type target Configure the source interface in the rx direction for the initiator, target, and mgmt traffic-types in the rx direction. sw(config-span)# source interface svc1/1 rx Display the configured SPAM session to verify the configuration. switch# show span session 2 Session 2 (admin suspended) Destination is fc1/9 No session filters configured Ingress (rx) sources are svc1/1(inititator), svc1/1 (target), svc1/1 (mgmt) svc2/1(initiator), svc2/1(mgmt) Egress (tx) sources are svc1/1(inititator), svc1/1 (target), svc1/1 (mgmt) svc2/1(target)... You have now configured SVC interfaces as SPAN sources. Refer to the Cisco MDS 9000 Family Configuration Guide for further details on the SPAN feature. Using VSAN Filters to Specify Traffic By default, traffic on all N-ports are spanned. You can monitor traffic on a specific N-port using the VSAN filtering SPAN feature. For example, if a SVC interface (3/1) with three initiator N-ports. Each N-port is in VSAN 3, 4, and 5 respectively. The following procedure configures the steps to SPAN the initiator N-port in VSAN 3. Refer to the Cisco MDS 9000 Family Configuration Guide for further details on using VSAN filters. To configure a SPAN filter, follow these steps: Step 1 Step 2 Step 3 Enter the MDS configuration mode. switch# config t switch(config)# Configure a SPAN session. switch(config)# span session 3 switch(config-span)# Configures the source svc 3/1 interface in the egress (tx) direction for all initiator traffic. 9-3

88 Using VSAN Filters to Specify Traffic Chapter 9 Configuring SPAN on SVC Interfaces switch(config-span)# source interface svc 3/1 tx traffic-type initiator Step 4 Step 5 Configures VSAN 2 as a session filter to only span traffic in VSAN 2. This will specifically monitor traffic on the initiator N-port traffic in VSAN 2. switch(config-span)# source filter vsan 2 Configures VSAN 4 and 5 as a session filter to additionally span initiator N-port traffic in these two VSANS. switch(config-span)# source filter vsan

89 CHAPTER 10 Configuring a Dual Fabric SAN Environment Dual fabric SAN environments are an important configuration requirement. You can use CSM modules in combination with the Inter-VSAN Routing (IVR) feature to operate across two isolated fabrics. This chapter includes the following sections: Overview, page 10-2 Basic SVC Requirements, page 10-2 Dual Fabric Prerequisites, page 10-2 Sample Configuration, page