Cisco MDS 9000 Family Cookbook for SAN-OS 1.x

Transcription

1 Cisco MDS 9000 Family Cookbook for SAN-OS 1.x 12/7/04 Corporate Headquarters Cisco Systems, Inc. 170 West Tasman Drive San Jose, CA USA Tel: NETS (6387) Fax:

2 CCSP, the Cisco Square Bridge logo, Cisco Unity, Follow Me Browsing, FormShare, and StackWise are trademarks of Cisco Systems, Inc.; Changing the Way We Work, Live, Play, and Learn, and iquick Study are service marks of Cisco Systems, Inc.; and Aironet, ASIST, BPX, Catalyst, CCDA, CCDP, CCIE, CCIP, CCNA, CCNP, Cisco, the Cisco Certified Internetwork Expert logo, Cisco IOS, Cisco Press, Cisco Systems, Cisco Systems Capital, the Cisco Systems logo, Empowering the Internet Generation, Enterprise/Solver, EtherChannel, EtherFast, EtherSwitch, Fast Step, GigaDrive, GigaStack, HomeLink, Internet Quotient, IOS, IP/TV, iq Expertise, the iq logo, iq Net Readiness Scorecard, LightStream, Linksys, MeetingPlace, MGX, the Networkers logo, Networking Academy, Network Registrar, Packet, PIX, Post-Routing, Pre-Routing, ProConnect, RateMUX, Registrar, ScriptShare, SlideCast, SMARTnet, StrataView Plus, SwitchProbe, TeleRouter, The Fastest Way to Increase Your Internet Quotient, TransPath, and VCO are registered trademarks of Cisco Systems, Inc. and/or its affiliates in the United States and certain other countries. All other trademarks mentioned in this document or Website are the property of their respective owners. The use of the word partner does not imply a partnership relationship between Cisco and any other company. (0406R) Cisco MDS 9000 Family Cookbook Copyright 2004 Cisco Systems, Inc. All rights reserved. Comments: mds-cookbook@cisco.com

3 Preface This document provides a reference guide to the configuration and implementation of fabrics using the MDS-9000 family of switches. The configurations and components used herein have been tested and validated by Cisco s Solution-Interoperability Engineering for supporting risk-free deployment of fabrics using Cisco s MDS 9000 family of Fibre Channel Switch and Director Class products. Audience This reference document is designed for use by Cisco TAC, Sales, Support Engineers, Professional Service Partners, Systems Administrators and others, responsible for the design and deployment of Storage Area Networks in the data center environment. Inquiries Please send your comments via to: mds-cookbook@cisco.com State the document name, page number, and details of the request. Thank you. Latest version of this document is available on Cisco s Partner website at: THE SPECIFICATIONS AND INFORMATION REGARDING THE PRODUCTS IN THIS MANUAL ARE SUBJECT T CHANGE WITHOUT NOTICE. ALL STATEMENTS, INFORMATION, AND RECOMMENDATIONS IN THIS MANUAL ARE BELIEVED TO BE ACCURATE BUT ARE PRESENTED WITHOUT WARRANTY OF ANY KIND, EXPRESS OR IMPLIED. USERS MUST TAKE FULL RESPONSIBILITY FOR THEIR APPLICATION OF ANY PRODUCTS. THE SOFTWARE LICENSE AND LIMITED WARRANTY FOR THE ACCOMPANYING PRODUCT ARE SET FORTH IN THE INFORMATION PACKET THAT SHIPPED WITH THE PRODUCT AND ARE INCORPORATED HEREIN BY THIS REFERENCE. IF YOU ARE UNABLE TO LOCATE THE SOFTWARE LICENSE OR LIMITED WARRANTY, CONTACT YOUR CISCO REPRESENTATIVE FOR A COPY. The Cisco implementation of TCP header compression is an adaptation of a program developed by the University of California, Berkeley (UCB) as part of UCB s public domain version of the UNIX operating system. All rights reserved. Copyright 1981, Regents of the University of California. NOTWITHSTANDING ANY OTHER WARRANTY HEREIN, ALL DOCUMENT FILES AND SOFTWARE OF THESE SUPPLIERS ARE PROVIDED AS IS WITH ALL FAULTS. CISCO AND THE ABOVE-NAMED SUPPLIERS DISCLAIM ALL WARRANTIES, EXPRESSED ORIMPLIED, INCLUDING, WITHOUT LIMITATION, THOSE OF MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT OR ARISING FROM A COURSE OF DEALING, USAGE, OR TRADE PRACTICE. IN NO EVENT SHALL CISCO OR ITS SUPPLIERS BE LIABLE FOR ANY INDIRECT, SPECIAL, CONSEQUENTIAL, OR INCIDENTAL DAMAGES, INCLUDING, WITHOUT LIMITATION, LOST PROFITS OR LOSS OR DAMAGE TO DATA ARISING OUT OF THE USE OR INABILITY TO USE THIS MANUAL, EVEN IF CISCO OR ITS SUPPLIERS HAVE BEEN ADVISED OF THE POSSIBILITY OF SUCH DAMAGES. iii

4 Preface iv

5 Introduction While deploying or configuring the MDS platform, a task may come up that the administrator does not know how to perform and requires a simple recipe to complete it. This guide won t replace the MDS 9000 Family Configuration Guides, but compliment them with simple, concise procedures to accomplish a specific task. Procedures range from the simple, Configuring FC ports, page 51 to more complex procedures like Configuring IVR between two Switches, using a transit VSAN, page 80. Even non-technical procedures such as Steps to Perform Before Calling TAC, page 49 are included. This is field driven book, meaning that the intended audience, the storage administrators, technical support engineers, SEs and CEs are the source for these procedures. Their requirements for a procedure to perform XYZ are what determine the content. If there are procedures that should be covered in this book, please notify by mds-cookbook@cisco.com. Within this book, there are sections that look like this: Tip These tips refer to a MDS best practice towards implementing the MDS. They come from in depth knowledge of the platform as well as years of experience of implementing SANs. v

6 Introduction vi

7 CONTENTS CHAPTER 1 Account Management 11 Creating User Accounts for CLI Access 11 Creating User Accounts for SNMP Access 12 Creating a MDS User Role 12 Configuring TACACS+ with Cisco SecureACS 16 Authentication and Authorization with TACACS+ 16 Accounting with TACACS+ 22 Providing Passwordless Access Leveraging SSH 25 CHAPTER 2 Miscellaneous Management Tasks 29 Saving the MDS s Configuration 29 Copying files to or from the MDS 30 Managing Files on the Standby Supervisor 31 Upgrading MDS Firmware 32 Upgrading Switch Firmware using CLI 33 Uprading Switch Firmware Using Fabric Manager 36 Password Recovery 39 Installing a License 41 Using the CLI to Install a License: 42 Using Fabric Manager to Install a License 43 Copying core files from the MDS 45 Configuring an NTP Server 46 Fixed Switch Config Restoration 47 Steps to Perform Before Calling TAC 49 CHAPTER 3 Physical Interfaces 51 Configuring FC ports 51 Port Description 51 Port speed 51 Port mode (auto) 52 Port mode (E) 52 Port mode (F) 52 Port mode (FL) 53 vii

8 Contents Port mode (Fx) 53 Port mode (SD) 53 Port mode (ST) 53 Port mode (TL) 54 Configuring Trunking E ports 54 Enabling port beaconing 55 Configuring Gigabit Ethernet Ports 55 Configuring VRRP 55 CHAPTER 4 Logical Interfaces 59 Port Channels 59 Creating a Port-Channel 59 Adding a New Member to a Port-Channel 61 Removing a Member From a Port-channel 63 Modifying the VSAN Allowed List 64 CHAPTER 5 VSANs 65 Creating a VSAN on a single switch and adding an Interface 65 Setting VSAN Interop Mode 65 Changing Load-balancing scheme 66 Sequence Level load-balancing (Source_ID, Destination_ID) 66 Exchange level load balancing (S_ID, D_ID, OX_ID) 67 Configuring a static Domain_ID and persistent FC_ID 67 Restarting a VSAN 68 Assigning a Predetermined FCID to a PWWN 68 CHAPTER 6 Intra-VSAN Routing (IVR) 71 Configuring IVR on one Switch between two VSANs 71 Configuring IVR between two Switches, using a transit VSAN 80 IVR with Brocade and McData switches using Interop Modes 90 CHAPTER 7 Zoning 93 Zones 93 Creating a zone and adding it to a zoneset (standalone method) 94 Creating a zone and adding it to a zoneset (inline method) 95 Creating a FC Alias based zone 96 Creating an Interface based zone 97 ZoneSets 98 viii

9 Contents Zoneset Distribution 98 CHAPTER 8 iscsi 101 Enabling iscsi 101 Configuring iscsi in transparent mode 101 Configuring the iscsi Host 106 iscsi clients on Windows Platform 106 iscsi clients on Linux 109 Configuring iscsi in Proxy initiator mode 110 CHAPTER 9 FCIP 117 Enabling FCIP 117 Configuring FCIP 117 Tuning FCIP 121 TCP Tuning: Latency and Available Bandwidth 121 FCIP Write Acceleration 123 FCIP Compression 123 ix

10 Contents x

11 CHAPTER 1 Account Management This chapter provides recipes for managing users and their accounts. In MDS firmware versions prior to 2.0, a separate account was required for both SNMP and CLI access. After 2.0, a single username grants access to both CLI and SNMP. Tip The admin account should only be used during initial setup. Afterwards, other user accounts should be created. Each administrator should have their own individual account. Always change the admin password from the factory default value. Users should be granted the minimum amount of rights or abilities required to perform their job function. Creating User Accounts for CLI Access In order to access the MDS switch via console, ssh or telnet, a username with CLI access must be created. To create such a user, the following procedure can be used: Step 1 Enter the configuration submode: switch# config terminal Step 2 Create the CLI user with using the following syntax: username <username> password <password> role <role> switch(config)# username user1 password admin123 role network-admin At this point the user user1 can access the switch using the password admin123 via console, ssh or telnet. 1-11

12 Creating User Accounts for SNMP Access Chapter 1 Account Management Creating User Accounts for SNMP Access In order to access the MDS switch using SNMP, a user with SNMP access must be created. To create such a user, the following procedure can be used: Step 1 Enter the configuration submode: switch# config terminal Step 2 Create the SNMP user with using the following syntax: snmp-server user <username> <role> auth <encryption method> <password> switch(config)# snmp-server user user1 network-admin auth md5 admin123 At this point the user user1 can access the switch using the password admin123 via an SNMP based product such as Cisco s Device Manager or Fabric Manager. Tip Create a unique user account for each user to aid with troubleshooting and accounting. Use both privacy and authentication passwords for increased security during SNMPv3 based sessions Creating a MDS User Role The MDS switch has two default roles, network-admin and network-operator. Network-admin has write privileges to all parts of the MDS while network-operator has read-only access to the switch. It may be required that a user with write access to specific areas of the MDS is required while eliminating write access to other areas. The MDS switch comes with two predefined roles: Network-admin is the role assigned to the predefined user admin. It has the ability to perform any modification to the MDS platform. There are no restrictions on this user. Network-operator is a predefined read-only role. It does not have the ability to make modifications to the MDS. There are no predefined users assigned to this role. Tip Provide each user with a role that provides them with the minimum amount of abilities to perform their job function. Leverage the read only role network-operator for those users who do not require the ability to modify the MDS. 1-12

13 Chapter 1 Account Management Creating a MDS User Role VSAN based role based access can allow administrators to have complete control over their VSANs while having read only or no access to other VSANs. In this example, a role will be created providing the ability to only modify the zoning configuration on the switch. Step 1 From Device Manager, select Security and then Common Roles: Figure 1-1 Common Roles Step 2 Select Create: Figure 1-2 Create Common Roles Fill in a name and description (do not use spaces) for the role. Step 3 Optionally, a VSAN scope can be specified limiting this specific role to a subset of VSANs. Such that a zoning admin role could be created for zone admins that could only modify VSANs This example will not specify a VSAN scope. Select Rules 1-13

14 Creating a MDS User Role Chapter 1 Account Management Figure 1-3 Create Rules Step 4 Step 5 Select the show checkbox at the top and all of the zone and zoneset commands. Additionally the copy command should be checked so that the zoning admin can save the configuration. Select Apply. 1-14

15 Chapter 1 Account Management Creating a MDS User Role Figure 1-4 Create Common Roles Step 6 Select Create and this will bring up the original Roles screen: Figure 1-5 Display Role At this point any user that is assigned to the role of ZoningAdmin will only be able to make zoning changes or the command copy running-configuration startup-configuration. To replicate this role to other systems, the startup-configuration can be examined for the relevant information, and the following CLI commands can be used: switch# config terminal switch(config)#role name ZoningAdmin switch(config-role)# description Zoneset_Administrator switch(config-role)# rule 1 permit show switch(config-role)# rule 2 permit config feature zoneset switch(config-role)# rule 3 permit exec feature zoneset switch(config-role)# rule 4 permit clear feature zone switch(config-role)# rule 5 permit config feature zone switch(config-role)# rule 6 permit debug feature zone switch(config-role)# rule 7 permit exec feature zone switch(config-role)# rule 8 permit exec feature copy To create a user, zoning_user, with the new role: 1-15

16 Configuring TACACS+ with Cisco SecureACS Chapter 1 Account Management switch# config terminal switch(config)# username zoning_user password admin123 role ZoningAdmin Configuring TACACS+ with Cisco SecureACS Cisco s SecureACS product can enhance the MDS s management security, and provide centralized authentication, authorization and accounting of users. Tip It is recommended that a tacacs+ server be used for both authentication/authorization and accounting. Authentication and Authorization with TACACS+ Configuring an MDS switch to use tacacs+ will allow centralized account management of the MDS. This centralized management will allow an admin to not have to create and maintain usernames and passwords on individual switches. The SecureACS server will provide the authentication to a switch login as well as assigning the role that the user is a member of. A shared secret key is used to provide encryption and authentication between the tacacs client (MDS-9500) and the tacacs server (Cisco SecureACS). In this procedure: The switch s ip address is The tacacs+ server s ip address is The tacacs+ shared secret key is: WarEagle Configure SecureACS Server Step 1 Prior to configuring the MDS, the SecureACS server must be configured. The first step is to configure SecureACS to allow the modification of advanced tacacs settings. a. On the main screen, on the left hand pane, select Interface Configuration, b. In the resulting screen select TACACS+ (Cisco IOS). c. At the bottom of the screen, check Advanced TACACS+ Features and Display a window...attributes. This should result in the following screen. d. Select the Submit button at the bottom to save the changes. 1-16

17 Chapter 1 Account Management Configuring TACACS+ with Cisco SecureACS Figure 1-6 SecureACS Configure Display Step 2 Next is to define the MDS-9506 to the tacacs+ server, so that the MDS can be authenticated by the tacacs+ server. On the left hand pane, select Network Configuration and then Add Entry. a. Fill in the MDS s ip address, , and shared secret key WarEagle. b. Select Submit to save the information. 1-17

18 Configuring TACACS+ with Cisco SecureACS Chapter 1 Account Management Figure 1-7 SecureACS Client Setup Step 3 Next is to define a group. These groups should be used to provide an easy way to assign the same role to multiple users without having to modify the attributes of each user individually. On the left hand pane, select Group Setup. 1-18

19 Chapter 1 Account Management Configuring TACACS+ with Cisco SecureACS Figure 1-8 SecureACS: Group Setup Step 4 Select an available group and select Rename Group. In the resulting box, choose a new name for this group. Tip Step 5 Step 6 Use the same SecureACS group name as the MDS role, this will ease creation of tacacs based users. Click Submit to save the name change. On the left hand pane, select Group Setup, then select the newly renamed group and click Edit Settings. Scroll to the section labeled TACACS+ Settings, check Shell and Custom attributes. In the Custom attributes field, input the av-pair string corresponding to the role that is defined on the MDS for users. The syntax is: cisco-av-pair=shell:roles= <role> Select Submit + Restart to save and apply the configuration. 1-19

20 Configuring TACACS+ with Cisco SecureACS Chapter 1 Account Management Figure 1-9 SecureACS Adding MDS Role Step 7 Step 8 Next is to define a user. Select User Setup from the left hand pane. Then type in a new or existing username and select Add/Edit. In the resulting window, fill in the fields for Password, Confirm Password and Group to which the user is assigned as illustrated below. 1-20

21 Chapter 1 Account Management Configuring TACACS+ with Cisco SecureACS Figure 1-10 SecureACS Creating tacacs+ user Configuring the SecureACS server has been completed and he remaining steps involve configuring the MDS Switch itself. It can be done from the CLI or SNMP. Configure TACACS+ on the MDS Step 1 Enable tacacs+ ca-9506# conf t ca-9506(config)# tacacs+ enable Step 2 Define the tacacs+ server and the shared secret key to use with it. ca-9506(config)# tacacs-server host key WarEagle Step 3 Define a group of authentication servers to use and add the tacacs+ server to the group. 1-21

22 Configuring TACACS+ with Cisco SecureACS Chapter 1 Account Management ca-9506(config)# aaa group server tacacs+ tacacs-group1 ca-9506(config-tacacs+)# server Step 4 Define the methods for the switch to perform authentication for the telnet/ssh/snmp access. ca-9506(config)# aaa authentication login default group tacacs-group1 Below are some show commands to display the configuration: ca-9506# show tacacs-server timeout value:5 total number of servers:1 following TACACS+ servers are configured: : available on port:49 TACACS+ shared secret:******** ca-9506# show aaa authentication default: group tacacs-group1 console: local iscsi: local dhchap: local ca-9506# show user-account user:admin this user account has no expiry date roles:network-admin user:seth expires on Fri Jun 18 23:59: roles:network-admin account created through REMOTE authentication Local login not possible Note The user seth is not available locally on the switch and yet is a member of the group/role network-admin. This means the user was authenticated by the tacacs+ server and not by the switch. Accounting with TACACS+ Cisco s SecureACS server can be leveraged to provide a command history of what users performed which actions. This information is similar to the CLI command show accounting log. However, by placing it on a remote system, the logs can be independently examined and are available in case the MDS is inaccessible. This configuration will build upon the configuration defined in Authentication and Authorization with TACACS+, page Configuring the MDS To configure the MDS to leverage a tacacs+ server for accounting the following procedure should be used: 1-22

23 Chapter 1 Account Management Configuring TACACS+ with Cisco SecureACS Step 1 Configure the MDS to use the tacacs-group1 server group. The local keyword denotes to log locally on the switch if all servers listed in the server group are unavailable. If the server group is available, commands/events will not be logged locally. switch# conf t switch(config)# aaa accounting default group tacacs-group1 local Configuring SecureACS Since this procedure builds on the configuration defined in Authentication and Authorization with TACACS+, page 1-16, only small modifications need to be made: Step 1 Step 2 Step 3 Configure the SecureACS server to monitor for Update/Watchdog packets, by modifying the clients configuration. In SecureACS, in the left pane, click on Network Configuration. Select the client to be modified. Check the Log Update/Watchdog Packets from this AAA Client checkbox. Figure 1-11 Enabling Accounting on the SecureACS server Step 1 Step 2 Step 3 Step 4 Next is to configure the SecureACS to display the commands. Click on System Configuration in the left pane. Click on Logging. Click on CSV TACACS+ Accounting. Add the column err_msg, and check the Log to CSV TACACS+ Accounting report box.and click submit at the bottom. 1-23

24 Configuring TACACS+ with Cisco SecureACS Chapter 1 Account Management Figure 1-12 Add MDS command logging to report. Step 1 Step 2 Step 3 To view the accounting report, click Reports and Activity in the left pane. Click TACACS+ Accounting. In the right hand pane, select the day to view. The current day is called TACACS+ Accounting active.csv. 1-24

25 Chapter 1 Account Management Providing Passwordless Access Leveraging SSH Figure 1-13 SecureACS Accounting Log Providing Passwordless Access Leveraging SSH In some instances it may be required to allow access to the MDS without using a password, such as automated scripts or agents. Providing a null password or hard coding the password into the script or agent could be considered a weak security practice. However, leveraging the private/public key infrastructure associated with ssh maintains a solid secure environment. The procedure will include creating the appropriate key on a host and then adding it to a new user. Since SSH leverages a private/public key exchange, the MDS will know only the public key which the host will know both the public and private keys. Tip Passwordless logins should be assigned to either a read only role like network-operator or to one with a minimal set of privileges. Warning Only having the public key will not cause the MDS to grant access to a user the private key is required to be on the host. The private key should be guarded or treated like a password. This procedure will cover the process of setting up a read-only (network-operator) based account which will only allow access if the user comes from a host that knows both the public and private keys. Step 1 On the host, create a ssh rsa1 public/private key: $ /usr/bin/ssh-keygen -t rsa1 Generating public/private rsa1 key pair. Enter file in which to save the key (/users/testuser/.ssh/identity): /users/setmason/.ssh/identity already exists. 1-25

26 Providing Passwordless Access Leveraging SSH Chapter 1 Account Management Overwrite (y/n)? y Enter passphrase (empty for no passphrase): Enter same passphrase again: Your identification has been saved in /users/testuser/.ssh/identity. Your public key has been saved in /users/testuser/.ssh/identity.pub. The key fingerprint is: c2:4d:6d:26:21:9d:79:9b:c3:86:dc:a5:07:d2:62:d4 testuser@host On the host, the file /users/testuser/.ssh/identity.pub is the ssh public key that is encrypted using the rsa1 algorithm. The contents of this file will be used in the creation of the MDS user. In this example, the file looks like this: $ cat /users/testuser/.ssh/identity.pub testuser@host Step 2 On the MDS create all of the ssh keys, even though in this case the client is using rsa # conf t (config)# ssh key rsa1 generating rsa1 key(1024 bits)... generated rsa1 key ca-9506(config)# ssh key dsa generating dsa key(1024 bits)... generated dsa key ca-9506(config)# ssh key rsa generating rsa key(1024 bits)... generated rsa key Step 3 Enable ssh on the MDS (config)# ssh server enable Step 4 On the MDS create the user, pasting in the contents of the identity.pub file after the sshkey parameter: # conf t (config)# username testuser role network-operator warning: password for user:setmason not set. S/he cannot login currently (config)# username testuser sshkey testuser@host (config)# end Step 5 The configuration of the user can be listed with the following: 1-26

27 Chapter 1 Account Management Providing Passwordless Access Leveraging SSH # show user-account testuser user: testuser this user account has no expiry date roles:network-operator no password set. Local login not allowed Remote login through RADIUS/TACACS+ is possible ssh public key: testuser@host Step 6 Test the log in process from the host: $ ssh testuser@ Warning: Remote host denied X11 forwarding. Cisco Storage Area Networking Operating System (SAN-OS) Software TAC support: Copyright (c) , Cisco Systems, Inc. All rights reserved. The copyrights to certain works contained herein are owned by other third parties and are used and distributed under license. Some parts of this software are covered under the GNU Public License. A copy of the license is available at # If the same user tries logging in from another host which does not have both the private key file (/users/testuser/.ssh/identity) and the public key (/users/testuser/.ssh/identity), then it will be denied access to the MDS. The fact that the public key has testuser@host at the end of it, does not tie it to a specific host but allows an admin to determine which host it was generated from. Tip One simple method to leverage this feature is to set up a scheduled, for example using cron, to backup the switch s configuration on a nightly basis using ssh and tftp. This could be done with the following commands executed on a host, provided the user that is used has the privileges to issue the copy command: $ ssh testuser@ "copy running-config startup-config" $ ssh testuser@ "copy startup-config tftp://backup-server/switch1.config.bak 1-27

28 Providing Passwordless Access Leveraging SSH Chapter 1 Account Management 1-28

29 CHAPTER 2 Miscellaneous Management Tasks Saving the MDS s Configuration It is always a good idea to save the configuration after making changes to the MDS. Whether it is creating users, or configuring ports, the configuration should be saved so that if the switch is rebooted, the current configuration is reapplied to the switch. Optionally, the configuration should also be saved to a file server for purposes of archival, disaster recovery or version control. The MDS has two configuration files, the running-configuration which is how the MDS is currently configured, and the startup-configuration which is what configuration will be applied to the switch the next time the switch is reloaded. Both can be viewed using the show running-configuration or show startup-configuration command. Tip Commands that are listed in the running or startup configuration are valid CLI commands and can be used within the config terminal submode on the MDS. Adding conf t to the beginning of a file containing running-configuration or startup-configuration derived MDS commands will have the shell enter the config submode. Step 1 To save the running-configuration, copy it to the startup-configuration: ca-9506# copy running-config startup-config [########################################] 100% Step 2 To copy the startup-configuration to a remote server (in this example using SCP), just modify the destination filename, by providing a filename to use on the fileserver (switch1.startupconfig ). ca-9506# copy startup-config scp://user@fileserver/switch1.startup setmason@dino's password: sysmgr_system.cfg 100% ***************************** :00 Now the file can be viewed on the file in the filename switch1.startup

30 Copying files to or from the MDS Chapter 2 Miscellaneous Management Tasks Copying files to or from the MDS It may be required to move files to or from an MDS switch. These files may include log, configuration or firmware files. There are two methods for copying files to/from the MDS: CLI and FM. The first procedure will cover the CLI: The CLI offers a broad range of protocols to use for copying to or from the MDS. Note that the MDS always acts as a client, such that an ftp/scp/tftp session will always originate from the MDS and either push files to an external system or pull files from an external system. File Server: File to be copied to the switch: /etc/hosts The MDS s copy command supports 4 transfer protocols and 12 different sources for files. ca-9506# copy? bootflash: Select source filesystem core: Select source filesystem debug: Select source filesystem ftp: Select source filesystem licenses Backup license files log: Select source filesystem modflash: Select source filesystem nvram: Select source filesystem running-config Copy running configuration to destination scp: Select source filesystem sftp: Select source filesystem slot0: Select source filesystem startup-config Copy startup configuration to destination system: Select source filesystem tftp: Select source filesystem volatile: Select source filesystem To use scp (Secure copy) as the transfer mechanism, the following syntax would be used: "scp:[//[username@]server][/path]" To copy /etc/hosts from using the user user1, and the destination would be hosts.txt use: switch# copy scp://user1@ /etc/hosts bootflash:hosts.txt user1@ 's password: hosts 100% ***************************** :00 To backup the startup-configuration to a sftp server: switch# copy startup-config sftp://user1@ /mds/startup-configuration.bak1 Connecting to User1@ 's password: switch# 2-30

31 Chapter 2 Miscellaneous Management Tasks Managing Files on the Standby Supervisor Tip Backing up the startup-configuration to a server should be done on a daily basis and prior to any changes. A short script could be written to be run on the MDS to perform a save and then backup of the configuration. The script only needs to contain two commands: copy running-configuration startup-configuration and then copy startup-configuration tftp://<server>/<name>. To execute the script use: run-script <filename>. Managing Files on the Standby Supervisor Oftentimes a file may need to be copied to, off or deleted from the supervisor, or even deleted from the standby supervisor. To do this, attach to the standby supervisor and use the normal dir: and delete commands. Note This recipe is most often invoked when a firmware upgrade fails because there is not be enough free bootflash: capacity on the standby supervisor for the firmware images. Step 1 Determine which supervisor is the standby. In this case it is module 6 as displayed below. switch# show module Mod Ports Module-Type Model Status /2 Gbps FC Module DS-X9016 ok /2 Gbps FC Module DS-X9016 ok 3 8 IP Storage Services Module DS-X9308-SMIP ok 4 0 Caching Services Module DS-X9560-SMAP ok 5 0 Supervisor/Fabric-1 DS-X9530-SF1-K9 active * 6 0 Supervisor/Fabric-1 DS-X9530-SF1-K9 ha-standby Step 2 Next, connect to the standby supervisor using the attach command. Note how the prompt displays the word standby. ca-9506# attach module 6 Attaching to module 6... To exit type 'exit', to abort type '$.' Cisco Storage Area Networking Operating System (SAN-OS) Software TAC support: Copyright (c) , Cisco Systems, Inc. All rights reserved. The copyrights to certain works contained herein are owned by Andiamo Systems, Inc. and/or other third parties and are used and distributed under license. Some parts of this software are covered under the GNU Public License. A copy of the license is available at ca-9506(standby)# Step 3 List the files on the bootflash to be deleted: ca-9506(standby)# dir bootflash: Jun 30 21:11: boot-1-3-4a 2035 Jun 17 16:30: hosts.txt Jun 30 21:11: isan-1-3-4a Dec 31 17:13: lost+found/ 2-31

32 Upgrading MDS Firmware Chapter 2 Miscellaneous Management Tasks Jun 23 17:02: m9500-sf1ek9-kickstart-mz.1.3.4b.bin Jun 23 17:02: m9500-sf1ek9-mz.1.3.4b.bin 99 Apr 07 19:28: security_cnv.log Usage for bootflash://sup-local bytes used bytes free bytes total Step 4 Delete the file with the delete command: ca-9506(standby)# delete bootflash:hosts.txt Step 5 To return to the active supervisor, type exit, and the prompt will also return to the active supervisors prompt: ca-9506(standby)# exit rlogin: connection closed. ca-9506# Upgrading MDS Firmware To obtain new features and functionality of an MDS switch, the firmware may have to be upgraded. It can be upgraded using either the CLI or Fabric Manager. Firmware images can be downloaded from the Cisco software center located at the following URL: A CCO login account is required to download all software images. Tip On single supervisor MDS switches like the 9100 and 9200 series, the switch will reboot and therefore persistent FCID and static domainids should be enabled. Configuring this values is described in Configuring a static Domain_ID and persistent FC_ID, page In this procedure the firmware images have been downloaded from the Cisco website and are located on a local fileserver. File server: testhost System image: m9500-sf1ek9-mz.1.3.4b.bin Kickstart image: m9500-sf1ek9-kickstart-mz.1.3.4b.bin The location of the firmware images may either be on the switch s bootflash: filesystem or on another server accessible via ftp/tftp/sftp/scp. To upgrade the firmware of a MDS switch using scp the following CLI process can be followed: 2-32

33 Chapter 2 Miscellaneous Management Tasks Upgrading MDS Firmware Upgrading Switch Firmware using CLI Step 1 First, determine what the upgrade impact will be on the system using the show install all impact system command. This first optional command will also verify the image integrity as well as provide the details of the upgrade. It will not actually perform the upgrade. ca-9506# show install all impact system scp://setmason@testhost/tftpboot/rel/qa/1_3_4b/final/m95 00-sf1ek9-mz.1.3.4b.bin kickstart scp://setmason@testhost /tftpboot/rel/qa/1_3_4b/final/m9500-sf1ek9-kickstart-mz.1.3.4b.bin For scp://setmason@testhost, please enter password: For scp://setmason@testhost, please enter password: Copying image from scp://setmason@testhost /tftpboot/rel/qa/1_3_4b/final/m9500-sf1ek9-kickstart-mz.1.3.4b.bin to bootflash:///m9500-sf1ek9-kickstart-mz.1.3.4b.bin. [####################] 100% -- SUCCESS Copying image from scp://setmason@testhost /tftpboot/rel/qa/1_3_4b/final/m9500-sf1ek9-mz.1.3.4b.bin to bootflash:///m9500-sf1ek9-mz.1.3.4b.bin. [####################] 100% -- SUCCESS Verifying image bootflash:///m9500-sf1ek9-kickstart-mz.1.3.4b.bin [####################] 100% -- SUCCESS Verifying image bootflash:///m9500-sf1ek9-mz.1.3.4b.bin [####################] 100% -- SUCCESS Extracting "slc" version from image bootflash:///m9500-sf1ek9-mz.1.3.4b.bin. [####################] 100% -- SUCCESS Extracting "ips" version from image bootflash:///m9500-sf1ek9-mz.1.3.4b.bin. [####################] 100% -- SUCCESS Extracting "svclc" version from image bootflash:///m9500-sf1ek9-mz.1.3.4b.bin. [####################] 100% -- SUCCESS Extracting "system" version from image bootflash:///m9500-sf1ek9-mz.1.3.4b.bin. [####################] 100% -- SUCCESS Extracting "kickstart" version from image bootflash:///m9500-sf1ek9-kickstart-mz.1.3.4b.bin. [####################] 100% -- SUCCESS Extracting "loader" version from image bootflash:///m9500-sf1ek9-kickstart-mz b.bin. [####################] 100% -- SUCCESS Compatibility check is done: Module bootable Impact Install-type Reason yes non-disruptive rolling 2 yes non-disruptive rolling 3 yes non-disruptive rolling 4 yes non-disruptive rolling 5 yes non-disruptive reset 6 yes non-disruptive reset 2-33

34 Upgrading MDS Firmware Chapter 2 Miscellaneous Management Tasks Other miscellaneous information for installation: Module info Images will be upgraded according to following table: Module Image Running-Version New-Version Upg-Required slc 1.3(4a) 1.3(4b) yes 1 bios v1.1.0(10/24/03) v1.0.8(08/07/03) no 2 slc 1.3(4a) 1.3(4b) yes 2 bios v1.0.8(08/07/03) v1.0.8(08/07/03) no 3 ips 1.3(4a) 1.3(4b) yes 3 bios v1.0.8(08/07/03) v1.0.8(08/07/03) no 4 svclc 1.3(4a) 1.3(4b) yes 4 svcsb 1.3(4m) 1.3(4m) no 4 svcsb 1.3(4) 1.3(4) no 4 bios v1.1.0(10/24/03) v1.0.8(08/07/03) no 5 system 1.3(4a) 1.3(4b) yes 5 kickstart 1.3(4a) 1.3(4b) yes 5 bios v1.1.0(10/24/03) v1.0.8(08/07/03) no 5 loader 1.2(2) 1.2(2) no 6 system 1.3(4a) 1.3(4b) yes 6 kickstart 1.3(4a) 1.3(4b) yes 6 bios v1.1.0(10/24/03) v1.0.8(08/07/03) no 6 loader 1.2(2) 1.2(2) no Step 2 Upgrade the firmware using the install all command and the appropriate file locations.: ca-9506# install all system scp://setmason@testhost/tftpboot/rel/qa/1_3_4b/final/m95 00-sf1ek9-mz.1.3.4b.bin kickstart scp://setmason@testhost /tftpboot/rel/qa/1_3_4b/final/m9500-sf1ek9-kickstart-mz.1.3.4b.bin For scp://setmason@testhost, please enter password: For scp://setmason@testhost, please enter password: Copying image from scp://setmason@testhost /tftpboot/rel/qa/1_3_4b/final/m9500-sf1ek9-kickstart-mz.1.3.4b.bin to bootflash:///m9500-sf1ek9-kickstart-mz.1.3.4b.bin. [####################] 100% -- SUCCESS Copying image from scp://setmason@testhost /tftpboot/rel/qa/1_3_4b/final/m9500-sf1ek9-mz.1.3.4b.bin to bootflash:///m9500-sf1ek9-mz.1.3.4b.bin. [####################] 100% -- SUCCESS Verifying image bootflash:///m9500-sf1ek9-kickstart-mz.1.3.4b.bin [####################] 100% -- SUCCESS Verifying image bootflash:///m9500-sf1ek9-mz.1.3.4b.bin [####################] 100% -- SUCCESS Extracting "slc" version from image bootflash:///m9500-sf1ek9-mz.1.3.4b.bin. [####################] 100% -- SUCCESS Extracting "ips" version from image bootflash:///m9500-sf1ek9-mz.1.3.4b.bin. [####################] 100% -- SUCCESS Extracting "svclc" version from image bootflash:///m9500-sf1ek9-mz.1.3.4b.bin. [####################] 100% -- SUCCESS Extracting "system" version from image bootflash:///m9500-sf1ek9-mz.1.3.4b.bin. 2-34

35 Chapter 2 Miscellaneous Management Tasks Upgrading MDS Firmware [####################] 100% -- SUCCESS Extracting "kickstart" version from image bootflash:///m9500-sf1ek9-kickstart-mz.1.3.4b.bin. [####################] 100% -- SUCCESS Extracting "loader" version from image bootflash:///m9500-sf1ek9-kickstart-mz b.bin. [####################] 100% -- SUCCESS Compatibility check is done: Module bootable Impact Install-type Reason yes non-disruptive rolling 2 yes non-disruptive rolling 3 yes non-disruptive rolling 4 yes non-disruptive rolling 5 yes non-disruptive reset 6 yes non-disruptive reset Other miscellaneous information for installation: Module info Images will be upgraded according to following table: Module Image Running-Version New-Version Upg-Required slc 1.3(4a) 1.3(4b) yes 1 bios v1.1.0(10/24/03) v1.0.8(08/07/03) no 2 slc 1.3(4a) 1.3(4b) yes 2 bios v1.0.8(08/07/03) v1.0.8(08/07/03) no 3 ips 1.3(4a) 1.3(4b) yes 3 bios v1.0.8(08/07/03) v1.0.8(08/07/03) no 4 svclc 1.3(4a) 1.3(4b) yes 4 svcsb 1.3(4m) 1.3(4m) no 4 svcsb 1.3(4) 1.3(4) no 4 bios v1.1.0(10/24/03) v1.0.8(08/07/03) no 5 system 1.3(4a) 1.3(4b) yes 5 kickstart 1.3(4a) 1.3(4b) yes 5 bios v1.1.0(10/24/03) v1.0.8(08/07/03) no 5 loader 1.2(2) 1.2(2) no 6 system 1.3(4a) 1.3(4b) yes 6 kickstart 1.3(4a) 1.3(4b) yes 6 bios v1.1.0(10/24/03) v1.0.8(08/07/03) no 6 loader 1.2(2) 1.2(2) no Do you want to continue with the installation (y/n)? [n] y Install is in progress, please wait. Syncing image bootflash:///m9500-sf1ek9-kickstart-mz.1.3.4b.bin to standby. [####################] 100% -- SUCCESS Syncing image bootflash:///m9500-sf1ek9-mz.1.3.4b.bin to standby. [####################] 100% -- SUCCESS Setting boot variables. [####################] 100% -- SUCCESS Performing configuration copy. 2-35

36 Upgrading MDS Firmware Chapter 2 Miscellaneous Management Tasks [####################] 100% -- SUCCESS Module 5: Waiting for module online. -- SUCCESS At this point the switch performs a hitless supervisor switchover. A new telnet/cli session will need to be established to the new supervisor. Note If the images fail to copy to the standby supervisor, there may be insufficient room for the new images and some old images or files may need to be removed. See Managing Files on the Standby Supervisor, page 2-31 for a recipe on removing files from the standby supervisor. To view the status of the current upgrade from the new supervisor issue show install all status. switch# show install all status This is the log of last installation. Continue on installation process, please wait. The login will be disabled until the installation is completed. Module 5: Waiting for module online. -- SUCCESS Module 1: Non-disruptive upgrading. -- SUCCESS Module 2: Non-disruptive upgrading. -- SUCCESS Module 3: Non-disruptive upgrading. -- SUCCESS Module 4: Non-disruptive upgrading. -- SUCCESS Install has been successful. Uprading Switch Firmware Using Fabric Manager To upgrade the firmware of one or more MDS switches Fabric Manager can be leveraged. Step 1 Select the Software Install Wizard from the toolbar in Fabric Manager. 2-36

37 Chapter 2 Miscellaneous Management Tasks Upgrading MDS Firmware Figure 2-1 Image Installation with Fabric Mgr Step 2 Choose the switches to upgrade and select Next: Figure 2-2 Select Switches to Upgrade Step 3 Specify the location of the firmware images a. Fill in the file information to transfer the file from the server to the switch. b. If the files are to be downloaded during the install, the path and filename of the images are required to be filled in as well. c. By selecting Skip Image Download, an upgrade can be performed using images which are already located on the supervisor s bootflash. 2-37

38 Upgrading MDS Firmware Chapter 2 Miscellaneous Management Tasks Figure 2-3 Specify Firmware Images Step 4 Select Next. Depending on the method for installing (already downloaded to bootflash or download during the install), the wizard may prompt for additional file locations. The fourth and final screen will provide a summary and enable the user to start the install. During the install, a compatibility screen will pop-up displaying the same version compatibility information that was displayed in the CLI upgrade. The user must select Yes to continue with the upgrade. Note Unlike the CLI, the FM maintains connectivity to the switch and provides detailed information during the entire upgrade sequence without requiring the user to manually reestablish connectivity to the switch during the supervisor switchover. If there is a failure, the last screen will display any reasons for a failed upgrade. 2-38

39 Chapter 2 Miscellaneous Management Tasks Password Recovery Password Recovery If there are no accounts accessible on the MDS that have either network-admin or user account creation privileges, due to lost passwords, it may be required to perform a password recovery on the admin account. Warning This procedure requires console access to the switch and will require the switch to be rebooted. Tip It is possible for another CLI user with network-admin privileges to change the password of the admin user, which can alleviate reloading the switch. To recover the admin account s password the following procedure can be used: Step 1 If possible, save the current configuration: switch# copy running-config startup-config [########################################] 100% Step 2 Connect a console cable to the active supervisor of the MDS Switch: 2-39

40 Password Recovery Chapter 2 Miscellaneous Management Tasks Figure 2-4 Console Connection on 9500 and 9200 series MDS switches. Step 3 Step 4 Attach the RS-232 end of the console cable to a PC. Configure Hyperterm or similar terminal emulation software for 9600 baud, 8 data bits, no parity, 1 stop bit and no flow control: Figure 2-5 HyperTerm Terminal Settings. Step 5 Establish a connection to the switch if possible or enough to display the login prompt if no user accounts are available. 2-40

41 Chapter 2 Miscellaneous Management Tasks Installing a License Step 6 Step 7 Step 8 Step 9 For a multi supervisor switch, MDS-9509 or MDS-9506, physically remove the standby supervisor. It is not necessary to remove it from the chassis, just enough such that it does not make contact with the backplane. Reboot the switch either by cycling the power or issuing the reload command. Press the Ctrl-] key sequence (when the switch begins its SAN-OS software boot sequence) to enter the switch(boot)# prompt. Enter configuration mode: switchboot# config terminal Step 10 Issue the command admin-password <new password> switch(boot-config)# admin-password temppassword switch(boot-config)# exit Step 11 Load the system image to finish the boot sequence. switch(boot)# load bootflash: m9500-sf1ek9-mz.1.3.4b.bin Step 12 Log into the switch using the admin account and the temporary password. switch login: admin Password: Cisco Storage Area Networking Operating System (SAN-OS) Software TAC support: Copyright (c) , Cisco Systems, Inc. All rights reserved. The copyrights to certain works contained herein are owned by Andiamo Systems, Inc. and/or other third parties and are used and distributed under license. Some parts of this software are covered under the GNU Public License. A copy of the license is available at switch# Step 13 Change the admin password to a new permanent password: ca-9506# config terminal ca-9506(config)# username admin password g05ox Step 14 Save the configuration which will include the new password. switch# copy running-config startup-config [########################################] 100% Installing a License To install a license key there are two methods available, using the CLI and Fabric Manager respectfully. 2-41

42 Installing a License Chapter 2 Miscellaneous Management Tasks Using the CLI to Install a License: Step 1 Copy the license file to the bootflash of the supervisor switch# copy scp://user1@ /tmp/fm_server.lic bootflash:fm_server.lic user1@ 's password: FM_Server.lic 100% ***************************** :00 Step 2 Verify the license file: switch# show license file FM_Server.lic lic.lic: SERVER this_host ANY VENDOR cisco INCREMENT FM_SERVER_PKG cisco 1.0 permanent uncounted \ VENDOR_STRING=MDS HOSTID=VDH=FOX X \ NOTICE="<LicFileID>lic_template</LicFileID><LicLineID>0</LicLineID> \ <PAK>dummyPak</PAK>" SIGN=D8CF07EA26C2 Step 3 Cross reference the switch s host-id (VDH=FOX X) with that which is listed in the license file: ca-9506# show license host-id License hostid: VDH=FOX X Step 4 Install the license file: switch# install license bootflash:fm_server.lic Installing license..done Step 5 Verify license has been installed: switch# show license lic.lic: SERVER this_host ANY VENDOR cisco INCREMENT FM_SERVER_PKG cisco 1.0 permanent uncounted \ VENDOR_STRING=MDS HOSTID=VDH=FOX X \ NOTICE="<LicFileID>lic_template</LicFileID><LicLineID>0</LicLineID> \ <PAK>dummyPak</PAK>" SIGN=D8CF07EA26C2 To display a summary of the installed licenses use the command show license summary: switch# show license usage Feature Insta License Status Expiry Date Comments lled Count FM_SERVER_PKG Yes - In use never - MAINFRAME_PKG No - Unused - ENTERPRISE_PKG Yes - In use never - SAN_EXTN_OVER_IP Yes 2 In use never - SAN_EXTN_OVER_IP_IPS4 No 0 Unused

43 Chapter 2 Miscellaneous Management Tasks Installing a License To display which features within a license package are being used, specify the package name. In this case QOS is using the Enterprise package.: ca-9506# show license usage ENTERPRISE_PKG Application Qos Manager Using Fabric Manager to Install a License Step 1 To install the licenses launch the License Installation Wizard: Figure 2-6 License Installation Wizard Step 2 This will bring up the following screen which will allow the user to specify how to install the keys based upon whether or not the user has already obtained the license key files or if they only have a Product Authorization Key (PAK). If the user has a PAK then the license file can be downloaded and installed from Cisco s website. 2-43

44 Installing a License Chapter 2 Miscellaneous Management Tasks Figure 2-7 License Installation Method a. If the keys already exist on a server then the following dialog box will be provided to specify the name and location of the license key files: Figure 2-8 License File Location b. If the license files are not already available, and the user only has the PAK numbers, then Fabric Manager can obtain the license files directly from Cisco.com. 2-44

45 Chapter 2 Miscellaneous Management Tasks Copying core files from the MDS Figure 2-9 Install License using PAK At this point, the license keys can be installed and the licensable feature can be used. Copying core files from the MDS If an MDS process crashes it may create a core file which would then be sent to Cisco TAC for further troubleshooting. This procedure will cover how to copy a core file off of the MDS switch. Step 1 Before copying a core file to another server, the pid of the core file must be identified. switch# show cores Module-num Process-name PID Core-create-time fspf 1524 Jul 15 03:11 Step 2 Then to copy the core file using, for example, ftp issue the following command syntax: "core://<module-number>/<process-id>" switch# copy core://5/1524 ftp:// /tmp/fspfcore The file can now be sent to Cisco TAC according to the TAC engineer s directions. 2-45

46 Configuring an NTP Server Chapter 2 Miscellaneous Management Tasks Configuring an NTP Server Network Time Protocol (NTP) is a protocol used by devices to synchronize their internal clocks with other devices. The MDS can only be used as an NTP client and would talk to other NTP systems which are considered to have a higher stratum (or authority). NTP is hierarchical in nature such that the lower stratum numbers are closer to the source of the time authority. Devices that are at the same stratum can be configured as peers so that they will work together to determine the correct time by making minute adjustments. Normally, MDSs would be configured as peers while a router or other dedicated machine would be used as a NTP server. Note NTP will not set the timezone (or offset from UTP) for the switch, it must be set manually using for example for Eastern Standard Time and Eastern Daylight-Savings Time: clock timezone EST -5.0 clock summer-time EDT 1 Sunday Apr 02:00 5 Sunday Oct 02:00 60 For this example: Switch #1 IP Address: Switch #2 IP Address: NTP Server: To configure ntp for switch1, the following procedure would be used: Step 1 Enter configuration mode and add the ntp server switch1# conf t switch1(config)# ntp server Step 2 Add the ntp peer switch switch1(config)# ntp peer switch1(config)# end At this point, NTP is configured and the switch will slowly adjust to the new time. To view the NTP configuration: switch1# show ntp peers Peer IP Address Serv/Peer Server Peer 2-46

47 Chapter 2 Miscellaneous Management Tasks Fixed Switch Config Restoration Fixed Switch Config Restoration This procedure will cover the process for backing up and restoring a switch configuration for one of the MDS 9000 Family switches that have a fixed configuration. This includes the 9216 and 9100 series fabric switches. This procedure will leverage the following resources: Old Switch: switch1: ( ) New Switch: switch2 File Server: host1 Note Only restore a switch configuration to a switch which has the exact same firmware version on it as which was used to create the switch configuration. If an upgrade is required, restore the configuration then upgrade the firmware. Step 1 Step 2 Save the running configuration: switch1# copy running-config startup-config [########################################] 100% Copy the startup-configuration to the file server using any of the available methods on the MDS (ftp, tftp, sftp, scp): switch1# copy startup-config scp://user@host1/switch1.config user@switch1's password: sysmgr_system.cfg 100% ***************************** :00 switch1# Step 3 Capture the port assignments using the FLOGI database. This will be used to verify that all of the cables are placed in the correct locations. switch1# show flogi database INTERFACE VSAN FCID PORT NAME NODE NAME fc1/ x7c :05:07:63:00:ce:a2:27 50:05:07:63:00:c0:a2:27 fc1/ xef :06:0e:80:03:4e:95:13 50:06:0e:80:03:4e:95:13 fc1/ x7c :06:0b:00:00:13:37:ae 50:06:0b:00:00:13:37:af Note At this point the old switch is no longer needed and its mgmt0 port should be disconnected from the LAN. Step 4 Step 5 Log on to the new switch using the console connection and clear the switch s configuration. Do not run the setup script, if prompted. The write erase command will erase the switch s configuration. switch2# write erase Warning: This command will erase the startup-configuration. Do you wish to proceed anyway? (y/n) [n] y Reload the switch. switch2# reload 2-47

48 Fixed Switch Config Restoration Chapter 2 Miscellaneous Management Tasks This command will reboot the system. (y/n)? [n] y Step 6 Step 7 Step 8 Step 9 The switch will come up in its factory default mode and will prompt for the Basic System Configuration Dialog, skip it, as all the configuration options are contained in the old switch s startup-config. Manually configure the IP address. switch2# config terminal switch2(config)# int mgmt 0 switch2(config-if)# ip address switch2(config-if)# no shut If interface (fcx/y) based zoning was done, obtain the new switch s wwn. Otherwise, this step can be skipped. switch2# show wwn switch Switch WWN is 20:00:00:0d:ec:02:1d:40 On the file server, make a copy of the configuration file and then open it in a text editor (notepad, vi). a. Remove the lines that contain the snmp user accounts, as the encrypted passwords are tied to the MAC address of the chassis.: $ cp switch1.config switch1.config.orig $ vi switch1.config The user accounts are all grouped together and begin with the command snmp-server user snmp-server user admin network-admin auth md5 0x46694cac2585d39d3bc00c8a4c7d48a6 localizedkey snmp-server user guestadmin network-admin auth md5 0xcae40d bc57ee1df348 26b51 localizedkey b. If interface (fcx/y) zoning was not done skip this step. Otherwise, replace the old switch s wwn in the zone member commands with the the new switch s wwn: zone name Z_1 vsan 9 member interface fc1/9 swwn 20:00:00:0d:ec:02:1d:40 c. Save and exit the config file. From the new switch, copy the modified config file from the fileserver onto the new switch s running configuration. As the file is copied it will be executed on the switch as the configuration is applied. The commands being applied with be contained in single-quotes. Any errors due to applying the commands will be displayed immediately after the error causing command is executed. The prompt will change to reflect the new switchname. switch2# copy scp://user@host1/switch1.config running-config user@host1's password: switch1.config 100% ***************************** :00 Step 10 Save the configuration, by copy startup-config to running-config switch1# copy running-config startup-config [########################################] 100%: Step 11 At this point, thecan be accessed via the CLI and the following remaining items will need to be performed: a. SNMP user accounts will need to be recreated. Also, b. If the switch is accessed via ssh, the host s known_hosts file will need to have the MDS switch s entry removed from it, because the switch s public key is different. c. License keys, if required, will need to be installed. 2-48

49 Chapter 2 Miscellaneous Management Tasks Steps to Perform Before Calling TAC Step 12 Step 13 Step 14 Move the cables from the old switch to the new switch, using the old switch s show flogi database as a reference to verify that each cable is in the correct location. Verify that all devices have logged in and all features are running is as they are supposed to be, save the running-configuration to the startup-config with the command copy running-config startup-config. Reload the switch to verify that it boots correctly with the configuration. Steps to Perform Before Calling TAC At some point, the administrator may need to contact the Cisco TAC or their OSM for some additional assistance. This section outlines the steps that the admin should perform prior to contacting their next level of support, as this will reduce the amount of time to resolve the issue. Step 1 Step 2 Step 3 Do not reload the line card or the switch until atleast Step 2has been done. Some logs and counters are kept in volatile storage and will not survive a reload. Collect switch information and configuration. This should be done before and after the issue has been resolved. The three methods below each provide the same information. a. CLI: Configure the telnet/ssh application to log the screen output to a text file and issue the command show tech-support details. b. CLI: Issue the command tac-pac <filename> (ex: tac-pac bootflash://showtech.switch1). The tac-pac command will redirect the output of show tech-support details to a file then gzip the file. If no filename is specified the file created is volatile:show_tech_out.gz). The file should then be copied off the MDS using the procedure outlined in Copying files to or from the MDS on page 30. c. Fabric Manager: Select Tools, then select Show tech support. Fabric manager can capture switch configuration information from multiple switches simultaneously. The file can be saved on the local PC. Capture the exact error codes: a. If the error occurs in Fabric Manager, take a screen shot of the error. In windows, to capture the active window use ALT+PrintScreen, and to capture the entire desktop hit just PrintScreen. Then paste this into a new MSpaint.exe (or similiar program) session. b. Copy the error from the message log which can be displayed using either show logging log or to view the last X lines of the log show logging last # The following questions should be answered prior to placing the call to TAC: 1. Which switch, hba, or storage port is the problem occurring? a. List MDS firmware, driver versions, operating systems versions and storage device firmware. 2. What is the network topology? (FM/tools ->show tech & save map) 3. Were any changes being made to the environment (zoning, adding linecards, upgrades) prior to or at the time of this event? 4. Are there other similarly configured devices that could have this problem but do not have? 5. Where this problematic device connected to (MDS switch Z, interface x/y)? 2-49

50 Steps to Perform Before Calling TAC Chapter 2 Miscellaneous Management Tasks 6. When did this problem first occur? 7. When did this problem last occur? 8. How often does this problem occur? 9. How many devices have this problem? 10. Were any traces or debug outputs captured during the problem time? What troubleshooting steps have already been done? Using such tools as: a. Fcanalyzer, PAA-2, Ethereal, local or remote SPAN b. CLI debug commands c. FC traceroute, FC ping d. FM/DM 2-50

51 CHAPTER 3 Physical Interfaces The MDS is a multi protocol switch, and in this section various options will be used to configure the FibreChannel (FC) and Gigabit Ethernet ports. The recipes below show how to configure various parameters and modes for a physical port on the MDS Configuring FC ports Port Description The port description allows a user to provide a description of plain text name for the interface. In the below example the fibre channel interface fc 1/1 is given a description storage array 17 port 1. This is just naming the port on the switch. switch# conf t switch(config)# interface fc 1/1 switch(config-if)# switchport description storage array 17 port 1 switch(config-if)# end switch# Port speed This example show how to the set the port speed for fc 1/1 to 1Gb, 2Gb or auto negotiate speed. Note A port can be set to only one speed. The default is auto-negotiate. switch# conf t switch(config)# interface fc 1/1 3-51

52 Configuring FC ports Chapter 3 Physical Interfaces switch(config-if)# switchport speed 1000 <- port speed set to 1 GB switch(config-if)# switchport speed 2000 <- port speed set to 2 GB switch(config-if)# switchport speed auto <- port speed set to auto negotiate switch(config-if)# ^Z switch# Port mode (auto) Note A FC port can be set to only one port mode at any given time. The default is auto on the 16 port linecards and FX on the 32 port linecards. Setting the port mode to auto, allows the port to negotiate to either a F, FL or E port. fc 1/1 is set to auto port mode. It can not negotiate to either ST, SD or TL port modes. This is the default setting for all ports on a 16 port linecard. switch# conf t switch(config)# interface fc 1/1 switch(config-if)# switchport mode auto switch(config-if)# end switch# Port mode (E) Setting the port mode to E, restricts the port to coming up as an Eport (can be either trunking or non-trunking depending on the trunking port mode) fc 1/1 is set to E port mode. E port mode is used when the port function as one end of an ISL. switch# conf t switch(config)# interface fc 1/1 switch(config-if)# switchport mode E switch(config-if)# end Port mode (F) Setting the port mode to F, restricts the port to coming up as an F port. fc 1/1 is set to F port mode. F port mode is used for end devices which can only communicate in point to point mode or to a switch. switch# conf t switch(config)# interface fc 1/1 switch(config-if)# switchport mode F switch(config-if)# end switch# 3-52

53 Chapter 3 Physical Interfaces Configuring FC ports Port mode (FL) Setting the port mode to FL, restricts the port to coming up as an FL port. fc 1/1 is set to FL port mode. FL port mode is used for end devices which can only communicate as a public loop device. switch# conf t switch(config)# interface fc 1/1 switch(config-if)# switchport mode FL switch(config-if)# end switch# Port mode (Fx) Setting the port mode to Fx, restricts the port to coming up as an F or FL port. fc 1/1 is set to Fx port mode. Fx port mode is used exclusively for end devices and prevents a port from autonegotiating to an Eport. switch# conf t switch(config)# interface fc 1/1 switch(config-if)# switchport mode Fx switch(config-if)# end switch# Port mode (SD) Setting the port mode to SD, configures the port for usage with a span session as the span destiation (SD) port. fc 1/1 is set to SD port mode. This is used in conjunction with the PAA to span a port and obtain FC traces without a FC analyzer. switch# conf t switch(config)# interface fc 1/1 switch(config-if)# switchport mode SD switch(config-if)# end Port mode (ST) Setting the port mode to ST, configures the port for usage with a remote span session as the span tunnel (ST) port. fc 1/1 is set to ST port mode. This is used to set up a remote SPAN session to a remote switch in which a PAA or protocol analyzer is connected. switch# conf t switch(config)# interface fc 1/1 switch(config-if)# switchport mode ST switch(config-if)# end switch# 3-53

54 Configuring FC ports Chapter 3 Physical Interfaces Port mode (TL) Setting the port mode to TL, restricts the port to coming up as a TL port. fc 1/1 is set to TLport mode. TL port mode is used exclusively for end devices which can only communicate as a private loop device. switch# conf t switch(config)# interface fc 1/1 switch(config-if)# switchport mode TL switch(config-if)# end switch# Configuring Trunking E ports A Trunking port is used to carry VSAN enabled frames between switches. The section below shows the various configuration options for a trunking port. Note These same principals apply to port-channels. Just specify the port channel interface, int port-channel 1, rather than an individual link, interface fc 1/1. Trunk port mode This example shows how to the set the trunk port mode for fc 1/1 to auto, on and off. The default mode is auto. One end of an ISL should be set to on when connected between two MDS switches, while the other end can be either on or auto. It needs to be off when talking to non-mds switches. switch# conf t switch(config)# interface fc 1/1 switch(config-if)# switchport trunk mode auto < auto negotiates trunk port mode switch(config-if)# switchport trunk mode on < sets trunk port mode to on switch(config-if)# switchport trunk mode off < sets trunk port mode to off switch(config-if)# ^Z switch# Configuring trunk ports to filter specific VSANs This example shows how to configure the allowed VSAN traffic through the interface fc 1/1. The all keyword allows all VSAN traffic to go through the port. The add 2 adds VSAN 2 to the list of allowed VSANs through the port. The add 2-4 adds VSANs 2 to 4 to the list of allowed VSANs through the port. The default mode is to allow all the VSAN traffic to pass through the port. 3-54

55 Chapter 3 Physical Interfaces Configuring Gigabit Ethernet Ports switch# conf t switch(config)# interface fc 1/1 switch(config-if)# switchport trunk allowed vsan all < all VSAN traffic switch(config-if)# switchport trunk allowed vsan add 2 < only VSAN 2 traffic switch(config-if)# switchport trunk allowed vsan add 2-4 < VSAN 2 to 4 traffic switch(config-if)# ^Z switch# Enabling port beaconing This example causes the LEDs below the port fc 1/1 to start flashing. This is useful in identifying a port for physical cabling or trouble shooting. switch# conf t switch(config)# interface fc 1/1 switch(config-if)# switchport beacon switch(config-if)# end Configuring Gigabit Ethernet Ports Configuring VRRP Virtual Router Redundancy Protocol (VRRP) allows two gigabit Ethernet interfaces to provide failover capability for an IP address. The two interfaces will form an active/passive or master/backup state in which one interface will service requests for the shared IP address while the other will remain in a backup or standby state. It is ideal for providing port level redundancy in iscsi configurations. A gigabit Ethernet port can still have its own IP address while partaking in a VRRP configuration. A VRRP session has an ID assigned to it for which the two interfaces will communicate to identify its peer. The same ID must be used on both switches. The procedure for having both members of the VRRP pair on the same switch would be the same as if the two members were on different switches. Note To have one interface become the master interface whenever it is online, also known as premption, set the gigabit Ethernet interface to have the same IP address as the VRRP IP address. In this example, the configuration is as follows: VRRP ID: 1 VRRP IP address: Switch 1: Interface gige3/3 ( ) 3-55

56 Configuring Gigabit Ethernet Ports Chapter 3 Physical Interfaces Switch 2: Interface gige4/1 ( ) Step 1 Configure the IP address on the two Gige interfaces: Switch1# config terminal Switch1(config)# interface gigabitethernet 3/3 Switch1(config-if)# ip address Switch1(config-if)# no shut Switch2# config terminal Switch2(config)# interface gigabitethernet 4/1 Switch2(config-if)# ip address Switch2(config-if)# no shut At this point it is a good idea to verify that a host on the local subnet can ping both IP addresses ( and ). Alternatively, the ips measure-rtt command could be used to ping from one GigE port to the other. Switch1# ips measure-rtt interface gigabitethernet 3/3 Round trip time is 172 micro seconds (0.17 milli seconds) Step 2 Configure the VRRP session on both switches using the VRRP id (1) Switch1# conf t Switch1(config)# interface gigabitethernet 3/3 Switch1(config-if)# vrrp 1 Switch1(config-if-vrrp)# address Switch1(config-if-vrrp)# no shut Switch1(config-if-vrrp)# end Switch2# conf t Switch2(config)# interface gigabitethernet 4/1 Switch2(config-if)# vrrp 1 Switch2(config-if-vrrp)# address Switch2(config-if-vrrp)# no shut Switch2(config-if-vrrp)# end Step 3 Verify that the VRRP session is up and which interface has become the master perform the following: Switch2# show vrrp vr 1 Interface VR Status GigabitEthernet3/3 1 backup switch1# show vrrp vr 1 interface gig3/3 status vr id 1 status MAC address 00:00:5e:00:01:01 Operational state: master Up time 8 sec 3-56

57 Chapter 3 Physical Interfaces Configuring Gigabit Ethernet Ports To view the configuration: Switch1# show vrrp vr 1 interface gigabitethernet 3/3 configuration vr id 1 configuration admin state up priority 100 associated ip: no authentication advertisement-interval 1 preempt no protocol IP 3-57

58 Configuring Gigabit Ethernet Ports Chapter 3 Physical Interfaces 3-58

59 CHAPTER 4 Logical Interfaces Port Channels Port-Channels allow for the aggregation of multiple FC or FCIP links into a single, higher speed, fault tolerant FC or FCIP ISL.It has the same configuration options as a single link FC or FCIP ISL as far as configuration is done, however, this section will discuss the procedures associated with building, modifying and reducing a port-channel. Creating a Port-Channel There are two basic methods to create a port-channel, either using the Fabric Manager wizard or with the CLI. This procedure will illustrate using the CLI. The following resources will be used: Switch1: Channel Group 2 Interfaces: fc1/1 and fc2/1 Switch2: Channel Group 2 Interfaces: fc1/1 and fc2/1 Allowed VSANs: 1,

60 Port Channels Chapter 4 Logical Interfaces Tip A port-channel should use interfaces on multiple linecards to protect the port-channel against line card failure. The same channel group number should be used on both ends of a port-channel. This will aid in troubleshooting and identifying the corresponding channel group on the other switch. A port-channel, like all other interfaces, can have a description. Use the description field to specify exactly where the port-channel goes. Port-channels can use any port on the switch and connect to any other port on a switch. Set the initial VSAN allowed list prior to bringing up the port-channel. This will prevent any VSANs from merging during the initial bringup. To create a port-channel: Step 1 On switch1: a. Create the Port-Channel switch1# config terminal switch1(config)# interface fc1/1, fc2/1 switch1(config-if)# channel-group 2 fc1/1 fc2/1 added to port-channel 2 and disabled please do the same operation on the switch at the other end of the port-channel, then do "no shutdown" at both ends to bring them up switch1(config-if)# switchport description "To switch2 PortChannel2" b. Enable trunking (TE) and set the VSAN allowed list on switch1 switch1# config terminal switch1(config)# int port-channel 2 switch1(config-if)# switchport trunk mode on switch1(config-if)# switchport trunk allowed vsan 1 switch1(config-if)# switchport trunk allowed vsan add 10 Step 2 On switch2 a. Create the port-channel on switch2 including setting the description: switch2# config terminal switch2(config)# interface fc1/1, fc2/1 switch2(config-if)# channel-group 2 fc1/1 fc2/1 added to port-channel 2 and disabled please do the same operation on the switch at the other end of the port-channel, then do "no shutdown" at both ends to bring them up switch2(config-if)# switchport description "To switch1 PortChannel2" b. Enable trunking (TE) and set the VSAN allowed list on switch1 switch2# config terminal switch2(config)# int port-channel 2 switch2(config-if)# switchport trunk mode on switch2(config-if)# switchport trunk allowed vsan 1 switch2(config-if)# switchport trunk allowed vsan add

61 Chapter 4 Logical Interfaces Port Channels Step 3 Step 4 Enable the interfaces to bring up the port-channel a. Switch1: switch1# config terminal switch1(config)# interface fc1/1, fc2/1 switch1(config-if)# no shut a. Switch2: switch2# config terminal switch2(config)# interface fc1/1, fc2/1 switch2(config-if)# no shut Verify that the port-channel has come up: switch1# show interface port-channel 2 port-channel 2 is trunking Port description is To switch2 PortChannel2 Hardware is Fibre Channel Port WWN is 24:02:00:0c:85:e9:d2:c0 Admin port mode is E, trunk mode is on Port mode is TE Port vsan is 1 Speed is 4 Gbps Trunk vsans (admin allowed and active) (1,10) Trunk vsans (up) (1,10) Trunk vsans (isolated) () Trunk vsans (initializing) () 5 minutes input rate 64 bits/sec, 8 bytes/sec, 0 frames/sec 5 minutes output rate 56 bits/sec, 7 bytes/sec, 0 frames/sec frames input, bytes 0 discards, 0 errors 0 CRC, 0 unknown class 0 too long, 0 too short frames output, bytes 0 discards, 0 errors 0 input OLS, 2 LRR, 0 NOS, 0 loop inits 4 output OLS, 2 LRR, 0 NOS, 0 loop inits Member[1] : fc1/2 Member[2] : fc2/1 iscsi authentication: None Adding a New Member to a Port-Channel This procedure details how to add a member to an existing port-channel. The following resources are used: Switch1: Channel Group 2 Existing Interfaces: fc1/1 and fc2/1 New Interfaces: fc3/1 Switch2: Channel Group

62 Port Channels Chapter 4 Logical Interfaces Existing Interfaces: fc1/1 and fc2/1 New Interface: fc3/1 Step 1 Step 2 Step 3 To add the new member to switch1, use the force keyword to have the new link inherit the parameters of the existing link.: switch1# conf t switch1(config)# int fc3/1 switch1(config-if)# channel-group 2 force fc3/1 added to port-channel 2 and disabled please do the same operation on the switch at the other end of the port-channel, then do "no shutdown" at both ends to bring them up switch1(config-if)# no shut To add the new member to switch2, use the force keyword to have the new link inherit the parameters of the existing link.: switch2# conf t switch2(config)# int fc3/1 switch2(config-if)# channel-group 2 force fc3/1 added to port-channel 2 and disabled please do the same operation on the switch at the other end of the port-channel, then do "no shutdown" at both ends to bring them up switch3(config-if)# no shut Verify that the port-channel now has all three members: switch1# show interface port-channel 2 port-channel 2 is trunking Port description is To switch2 PortChannel2 Hardware is Fibre Channel Port WWN is 24:02:00:0c:85:e9:d2:c0 Admin port mode is E, trunk mode is on Port mode is TE Port vsan is 1 Speed is 6 Gbps Trunk vsans (admin allowed and active) (1,10) Trunk vsans (up) (1,10) Trunk vsans (isolated) () Trunk vsans (initializing) () 5 minutes input rate 64 bits/sec, 8 bytes/sec, 0 frames/sec 5 minutes output rate 56 bits/sec, 7 bytes/sec, 0 frames/sec frames input, bytes 0 discards, 0 errors 0 CRC, 0 unknown class 0 too long, 0 too short frames output, bytes 0 discards, 0 errors 0 input OLS, 2 LRR, 0 NOS, 0 loop inits 4 output OLS, 2 LRR, 0 NOS, 0 loop inits Member[1] : fc1/2 Member[2] : fc2/1 Member[3] : fc3/1 iscsi authentication: None 4-62

63 Chapter 4 Logical Interfaces Port Channels Removing a Member From a Port-channel To remove a member from a port-channel the following procedure can be used: Tip Using the quiesce command will allow seamless reduction in the port-channel, and no in flight frames will be dropped. The following configuration is used in this procedure: Switch1: Channel Group 2 Existing Interfaces: fc1/1, fc2/1, fc3/1 Remove Interface: fc3/1 Switch2: Channel Group 2 Existing Interfaces: fc1/1, fc2/1, fc3/1 Remove Interface: fc3/1 Step 1 Quiesce the interfaces to be removed a. On switch1: switch1# quiesce interface fc3/1 WARNING: this command will stop forwarding frames to the specified interfaces. I t is intended to be used to gracefully shutdown interfaces in a port-channel. Th e procedure is: 1. quiesce the interfaces on both switches. 2. shutdown the interfaces administratively. Do you want to continue? (y/n) [n] y fc3/1: quiesced Please quiesce the corresponding interfaces on the other switch and then shut do wn them administratively. b. On switch2: switch2# quiesce interface fc3/1 WARNING: this command will stop forwarding frames to the specified interfaces. I t is intended to be used to gracefully shutdown interfaces in a port-channel. Th e procedure is: 1. quiesce the interfaces on both switches. 2. shutdown the interfaces administratively. Do you want to continue? (y/n) [n] y fc3/1: quiesced Please quiesce the corresponding interfaces on the other switch and then shut do wn them administratively. Step 2 Shut the physical interfaces on both switches: a. On Switch1 switch1# config terminal switch1(config)# int fc3/1 4-63

64 Port Channels Chapter 4 Logical Interfaces switch1(config-if)# shut b. On Switch2 switch2# config terminal switch2(config)# int fc3/1 switch2(config-if)# shut Modifying the VSAN Allowed List To modify the VSAN allowed list for a port-channel is the same as for a standard, single link TE port. To add VSAN 17 to port-channel 2: switch2# config terminal switch2(config)# int port-channel 2 switch2(config-if)# switchport trunk allowed vsan add 17 To remove VSAN 17 from port-channel 2: switch2# config terminal switch2(config)# int port-channel 2 switch2(config-if)# no switchport trunk allowed vsan add

65 CHAPTER 5 VSANs VSANs (Virtual SAN) are a logical fabric. It is a logical grouping of ports in a single switch or across multiple switches that function like a single fabric. Each VSAN has all the required fabric services independent of the other VSANs configured on the same switch or set of switches. VSANs also provide SAN island consolidation on a higher port density physical switch, traffic isolation and increased security. VSANs can be numbered from VSAN 1 and VSAN 4094 are predefined and have very specific roles. VSAN 1 is the default VSAN which holds all the ports by default and the VSAN 4094 is the isolated VSAN into which orphaned ports are assigned. Creating a VSAN on a single switch and adding an Interface This recipe shows the steps to create and name a VSAN on a single switch. VSAN 200 is created with the name TapeVSAN and fibre channel interface fc 1/1 is added. switch1# conf t switch(config)# vsan database switch(config-vsan-db)# vsan 200 name TapeVSAN switch(config-vsan-db)# vsan 200 interface fc 1/1 switch(config-vsan-db)# ^Z switch# Setting VSAN Interop Mode Interop mode is set for VSANs that need to interact with other 3 rd party switches. Interop mode one is required when all vendor switches are set in their respective interop modes. In interop mode only Domain ids 97 to 127 are allowed. Interop 2 is required when a VSAN has to work with a brocade 2800/3800 switch in its native mode. Interop mode 3 is required when the VSAN has to work with brocade 3900 or a switch. For more information please refer to the interop guide. The MDS Switch to Switch Interoperability Configuration Guide, should be consulted prior to performing any interoperability work, explains the different interop modes and can be found on CCO or in the following location:

66 Changing Load-balancing scheme Chapter 5 VSANs This recipe shows the steps to set the interop modes 1, 2, 3 for a VSAN. Interop mode 1 (ensure that the domain ID of the VSAN is between for this mode to work). To change the interop mode of VSAN 200 to interop mode 1: switch# conf t switch(config)# vsan database switch(config-vsan-db)# vsan 200 interop 1 switch(config-vsan-db)# ^Z switch# Interop mode 2 (with brocade 2800/ 3800 switches). To change the interop mode of VSAN 200 to interop mode 2. switch# conf t switch(config)# vsan database switch(config-vsan-db)# vsan 200 interop 2 switch(config-vsan-db)# ^Z switch# Interop mode 3 (with brocade switches 3900 / 12000). To change the interop mode of VSAN 200 to interop mode 3. switch# conf t switch(config)# vsan database switch(config-vsan-db)# vsan 200 interop 2 switch(config-vsan-db)# ^Z switch# Changing Load-balancing scheme The load-balancing scheme can be configured per VSAN. S_ID (source id), D_ID (destination id) based load balancing and the exchange level (S_ID, D_ID, OX_ID) load balancing can be configured on the MDS This recipe shows the steps to configure load-balancing schemes for VSAN 200. Sequence Level load-balancing (Source_ID, Destination_ID) To change the load-balancing scheme for VSAN 200 to S_ID, D_ID mode. switch# conf t 5-66

67 Chapter 5 VSANs Configuring a static Domain_ID and persistent FC_ID switch(config)# vsan database switch(config-vsan-db)# vsan 200 loadbalancing src-dst-id switch(config-vsan-db)# ^Z switch# Exchange level load balancing (S_ID, D_ID, OX_ID) To change the load-balancing scheme for VSAN 200 to S_ID, D_ID, and OX_ID mode. This is the default load balancing scheme. switch# conf t switch(config)# vsan database switch(config-vsan-db)# vsan 200 loadbalancing src-dst-ox-id switch(config-vsan-db)# ^Z switch# Configuring a static Domain_ID and persistent FC_ID Within a VSAN, the domain manager process on the principal switch in a fabric is responsible for assigning a domain_id to switch that is joining the fabric. When a switch boots up or joins a new fabric it can request a specific domain_id or take any available domain_id. A domain_id can be configured in one of three ways: Dynamic (default): The new switch will not request any domain_id from the principal switch and accept any domain_id that is assigned. Preferred: The new switch will request a specific domain_id, however, if it receives a different domain_id it will accept it. Static: The new switch will request a specific domain_id, however, if it receives a different domain_id, it will isolate itself from the fabric. This is the case when the same domain_id must be maintained under all circumstances. After obtaining the domain_id from the principal switch in the VSAN, the local switch will assign Fibre Channel Identifiers (FC_IDs) to each device as they log in to the Fabric. A process known as FLOGI (Fabric Login). Some devices require that the same FC_ID be assigned to a device as the FC_ID is used in the host s device path. HPUX and AIX are two operating systems that utilize the FC_ID in the device path to the storage. In order to allow have the switch always assign the same FC_ID to a device, persistent FC_ID must be configured for the VSAN. By default, the switch will assign the same FC_ID to a device, however, if the switch is rebooted this database of pwwn/fc_id mapping is not maintained. Enabling persistent FC_IDs will make this database persistent. When persistent FC_ID is enabled, the MDS will make persistent all of the devices in that VSAN, therefore the admin is not required to manually type in devices entering that VSAN. This recipe shows the steps to configure static Domain_ID for a VSAN and also how to enable persistent FC_ID for the same VSAN. 5-67

68 Restarting a VSAN Chapter 5 VSANs This procedure sets the Domain_ID of VSAN 200 to 22 and then enables persistent FC_ID. switch# conf t switch(config)# fcdomain domain 22 static vsan 200 switch(config)# fcdomain fcid persistent vsan 200 switch(config)# ^Z switch# Note If the domain ID of VSAN 200 is different than what is currently running (22 in this case) then the VSAN will have to be restarted for the configuration changes to the Domain_ID and FC_ID persistence to take effect. Changing Domain_IDs and hence FC_IDs for a device is disruptive as an end device will have to relogin to the fabric (FLOGI) to obtain a new FCID. Warning Changing Domain_IDs and therefore FC_IDs for a device is disruptive, as an end device will have to relogin to the fabric (FLOGI) to obtain a new FCID. However, making a Domain_ID static without changing its value is not disruptive. Restarting a VSAN Sometimes the VSAN on a switch will need to be restarted. For example, after changing the Domain_ID of a VSAN, the VSAN should be restarted for the new Domain_ID to take effect. The recipe below shows how a VSAN (200) can be restarted. switch# conf t switch(config)# vsan database switch(config-vsan-db)# vsan 200 suspend switch(config-vsan-db)# no vsan 200 suspend switch(config-vsan-db)# end Assigning a Predetermined FCID to a PWWN When performing a migration or hba replacement, the same FCID as previously used may need to be assigned to the new pwwn. This recipe show steps to assign pre-determined FC_ID to a specific PWWN. FC_ID 0x will be assigned to pwwn 50:06:0b:82:bf:d1:db:cd permanently. Therefore when the pwwn logs into the switch (FLOGI) it will get this assigned FC_ID. 5-68

69 Chapter 5 VSANs Assigning a Predetermined FCID to a PWWN Note The FC_ID to be assigned (0x160000) should contain the same Domain_ID (0x16) as the currently running domain in the VSAN. switch# conf t switch(config)# fcdomain fcid database switch(config-fcid-db)# vsan 22 wwn 50:06:0b:82:bf:d1:db:cd fcid 0x dynamic switch(config-fcid-db)# ^Z switch# 5-69

70 Assigning a Predetermined FCID to a PWWN Chapter 5 VSANs 5-70

71 CHAPTER 6 Intra-VSAN Routing (IVR) Cross VSAN communication is not permitted on an MDS by default. Let us assume that there are 2 VSANS (VSAN 2112 and VSAN 3999 on a single switch. One or more of the of the initiators in VSAN 2112 needs to communicate with one or more of the targets in VSAN 3999, it can be done using Inter VSAN Routing (IVR). IVR facilitates the communication between the target and initiator in different VSANS using IVR zones and IVR zonesets. Configuring IVR on one Switch between two VSANs The recipe below details the steps needed to configure IVR on a single switch. In this example shown below VSANS 2112 and 3999 are used. Before IVR can be configured it needs to be enabled. Figure 6-1 Single Switch IVR Topology Step 1 In the Fabric Manager select All Vsans IVR as shown in the image below. Then select the switch on which IVR needs to be enabled and enable IVR and save the changes as shown in the images below. 6-71

72 Configuring IVR on one Switch between two VSANs Chapter 6 Intra-VSAN Routing (IVR) Figure 6-2 Enabling IVR of the Switch It can also be enabled from the command prompt of the switch as shown below. sjc # conf t Sjc (config)# ivr enable sjc (config)#^z sjc # Figure 6-3 IVR enable status on the switch Step 2 The next step is to configure the local IVR topology. To do this select the Local Topology tab Fabric manager. Then select the create row as shown in the figure below. 6-72

73 Chapter 6 Intra-VSAN Routing (IVR) Configuring IVR on one Switch between two VSANs Figure 6-4 Inserting IVR topology The create row brings up a window which lists all the switches that have IVR enabled in the topology. In this window select the tab Switch Wwn and pick the switch wwn on which IVR has to be configured. Then add the VSANs that need to be involved in this IVR configuration in the VSAN list box, the image below shows the switch sjc selected and its switch wwn and the VSANs (2112 and 3999) involved in the IVR configuration is filled in. 6-73

74 Configuring IVR on one Switch between two VSANs Chapter 6 Intra-VSAN Routing (IVR) Figure 6-5 Adding switch wwn and VSANs to create the IVR topology Step 3 Once this is done the configuration will be updated as shown below. The next step is to activate the created IVR topology. Then in the action tab select the Activate Local check for the switch on which the topology needs to be activated. Figure 6-6 Activating IVR topology The active topology can be seen by selecting the Active Topology tab. The Discrepancies tab shows the discrepancies if any that need to be fixed for IVR to function properly. Ideally the discrepancies tab should not have any entries. The snap below shows the current active topology. 6-74

75 Chapter 6 Intra-VSAN Routing (IVR) Configuring IVR on one Switch between two VSANs Figure 6-7 View configured active topology Step 4 The next step is to create a IVR zoneset and zones so that the initiator and the target can communicate with each other. Select IVR Edit Local Full Zone Database. This will launch the zone creation window. Figure 6-8 Edit IVR zoneset on the switch Step 5 In the zone creation window create a zone and the required members to the zone. Right click on zone and select insert. This brings up the IVR zone name dialogue box. 6-75

76 Configuring IVR on one Switch between two VSANs Chapter 6 Intra-VSAN Routing (IVR) Figure 6-9 Insert IVR zones Leave the initial IVR in the name of the zone and give the zone a name. In the recipe it is IVR_Z_sjc7-hp

77 Chapter 6 Intra-VSAN Routing (IVR) Configuring IVR on one Switch between two VSANs Figure 6-10 Naming a IVR zone 6-77

78 Configuring IVR on one Switch between two VSANs Chapter 6 Intra-VSAN Routing (IVR) Figure 6-11 Members on the IVR zone Step 6 Add the member ports into zone IVR_Z_sjc7-hp In this case target port is in VSAN 3999 and the initiator port is in VSAN Then create a IVR zoneset and add the above zone to the zone set. 6-78

79 Chapter 6 Intra-VSAN Routing (IVR) Configuring IVR on one Switch between two VSANs Figure 6-12 Inserting IVR zoneset Step 7 This is done by selecting Zone Sets in the Edit IVR Local Full Database window and then selecting insert. Figure 6-13 IVR zoneset Again as with the zone name retain the IVR in front of the Zoneset name to designate it as an IVR zoneset. Once the zone has been added to the zone set, the zone set has to be activated. Check to see that the target and initiator can see each other. Run the appropriate host commands to check that the host can see the LUNs through the target. 6-79

80 Configuring IVR between two Switches, using a transit VSAN Chapter 6 Intra-VSAN Routing (IVR) Figure 6-14 Activating IVR zoneset Configuring IVR between two Switches, using a transit VSAN When two or more switches are involved in an IVR configuration it may require one or more transit VSANs to do IVR. In this recipe switches sjc and sjc are used. The target (storage) port is on switch sjc in VSAN 3999 and the initiator a HP-UX server in on switch sjc VSAN VSAN 3999 is only available on switch sjc and VSAN 3460 is only available on switch sjc The transit VSAN used is The topology is as show below. Figure 6-15 IVR topology with two MDS switches As with any IVR configuration the switches involved need to have IVR enabled on them. In this case switches sjc and sjc need to have IVR enabled. Currently from the figure below IVR is disabled on both switches 6-80

81 Chapter 6 Intra-VSAN Routing (IVR) Configuring IVR between two Switches, using a transit VSAN Figure 6-16 Enabling IVR on the switch Step 1 In the Fabric Manager select All Vsans IVR as shown in the image below. Then on the switches that require IVR enabled from the command pull down menu select enabled as shown below. The images below show the process on switch sjc and sjc Figure 6-17 Enabling IVR in sjc Figure 6-17 shows the enabling of IVR on switch sjc Similarly enable VR for switch sjc Then the changes need to be applied. This is show in Figure

82 Configuring IVR between two Switches, using a transit VSAN Chapter 6 Intra-VSAN Routing (IVR) Figure 6-18 Save IVR changes Step 2 Once IVR has been enabled the IVR topology needs to be created. To do this select the Local Topology tab Fabric manager. Then select the create row as shown in the image below. 6-82

83 Chapter 6 Intra-VSAN Routing (IVR) Configuring IVR between two Switches, using a transit VSAN Figure 6-19 Create IVR local topology The create row brings up a window which lists all the switches that have IVR enabled in the topology. In this window select the tab Switch Wwn and pick the switch wwn on which IVR has to be configured. In the configuration below the switches sjc and sjc are both selected in the Switches dialogue box. In the Switch Wwn dialogue box the wwn of switch sjc is selected. In the VSAN lists the VSANs need to complete the IVR topology are added. In this case it is VSAN 1234 (transit VSAN) and 3460 (host VSAN) are added. This will create the first half of the topology required to make IVR work. 6-83

84 Configuring IVR between two Switches, using a transit VSAN Chapter 6 Intra-VSAN Routing (IVR) Figure 6-20 Phase 1 of the IVR Topology configuration Step 3 To make IVR work the VSANs on switch sjc needs to be added to the topology. This adds both switches, the wwn of sjc and VSANs 1234 (transit VSAN) and 3999 (storage VSAN) to the topology as shown in Figure

85 Chapter 6 Intra-VSAN Routing (IVR) Configuring IVR between two Switches, using a transit VSAN Figure 6-21 Phase 2 of the IVR Topology configuration After the configuration the local topology is as shown in Figure Figure 6-22 Configured IVR Local Topology Step 4 The configured IVR topology needs to be activated for it to take effect. From the above image both the switches now have knowledge of all the VSANs involved in the IVR configuration. In the window select the Action tab. Then select Activate local and Create Active Zoneset if None Present check boxes. This is shown in Figure

86 Configuring IVR between two Switches, using a transit VSAN Chapter 6 Intra-VSAN Routing (IVR) Figure 6-23 Activating IVR topology Tip Step 5 If any of the VSANs does not have an active zoneset, then the IVR zoneset activation will fail if the Create Active Zoneset if None Present checkbox is not selected. The next step is to create a IVR zoneset and zones so that the initiator and the target can communicate with each other. Select IVR Edit Local Full Zone Database. This launches the zone creation window. Figure 6-24 Edit IVR zoneset on the switch In the zone creation window create a zone and add the required members to the zone. To add zone right click on zone and select insert. This brings up the IVR zone name dialogue box. 6-86

87 Chapter 6 Intra-VSAN Routing (IVR) Configuring IVR between two Switches, using a transit VSAN Figure 6-25 Insert IVR zones Step 6 The next step is to give the zone a name. In the recipe the IVR zone is called IVR_Z_sjc7-hp The IVR in the name help to identify that the zone is an IVR zone and the name also contains the host name and the HBA hardware location the system. Figure 6-26 Naming a IVR zone Step 7 Then add the initiator and the target to the above zone. This will enable them to communicate with each other. This is show in Figure

88 Configuring IVR between two Switches, using a transit VSAN Chapter 6 Intra-VSAN Routing (IVR) Figure 6-27 Members of the IVR zone Step 8 The next step is to create the IVR zoneset and activate the zoneset to complete the IVR configuration. Figure 6-28 Inserting IVR zoneset This is done by selecting Zone Sets in the Edit IVR Local Full Database window and then selecting insert. 6-88

89 Chapter 6 Intra-VSAN Routing (IVR) Configuring IVR between two Switches, using a transit VSAN Figure 6-29 IVR zoneset In the recipe the zone set is IVR_ZS_1234_3460_3999. The name is IVR to classify it as an IVR zoneset and contains the VSAN number to show what VSANs are involved in the conifg. Then the zone IVR_Z_sjc7-hp is added to the zoneset. Figure 6-30 Add IVR zone to IVR zoneset Step 9 Once the zone has been added to the zone set, the zone set has to be activated. After the activation the zoneset check to see that the target and initiator can see each other. Run the appropriate host commands to check that the host can see the LUNs from the target. The activation of the zoneset is shown in Figure

90 IVR with Brocade and McData switches using Interop Modes Chapter 6 Intra-VSAN Routing (IVR) Figure 6-31 Activating IVR zoneset The two switch IVR configuration is now complete. IVR with Brocade and McData switches using Interop Modes IVR configuration procedure in interop mode with 3rd party switchs is same as described in the above 2 recipes. IVR can be used in conjunction with the other 3rd party switches like Brocade and McData switches. When a 3rd party switch is used in the topology they can be located either in the transit VSAN or as an edge switch connecting to a MDS switch enabled for IVR. 3rd party switches cannot be used as IVR gateways. Topologies such as those illustrated in Figure For the 3rd party switches to function properly in an IVR configuration they have to be configured in an interop mode VSAN. IVR works with all three interop modes (1, 2, and 3). For more details on interoperability please refer to the MDS Switch to Switch Interoperability Configuration Guide located on at When a McData switch is involved the destination switch s Domain ID needs to be in the interop range. I.e. the domain ID of the switches / VSAN has to be between 97 and 127 for IVR to function properly. Thus a device on a McData switch can not send a frame to a device in another VSAN, irregardless of the interop mode, that is outside of the Domain ID range. 6-90

91 Chapter 6 Intra-VSAN Routing (IVR) IVR with Brocade and McData switches using Interop Modes Figure 6-32 Sample IVR Topology with Interop Modes 6-91

92 IVR with Brocade and McData switches using Interop Modes Chapter 6 Intra-VSAN Routing (IVR) 6-92

93 CHAPTER 7 Zoning Zones and zonesets are the basic form of datapath security within a fibre channel environment. A zoneset is a collection of zones which in turn have individual members in them. Only those members within the same zone can communicate with each other. A device can be a member of multiple zones and those devices not in a zone are in the default-zone. The policy for the default-zone can either be permit (devices can see each other) or deny (devices in the default-zone cannot see each other). This chapter will focus on the creation of zones, zonesets and how to manipulate them. Zones In order for two devices to communicate, they must be in the same zone. Valid members of a zone can be: Port WWN FC Alias FCID FWWN (WWN of a fc interface) Switch Interface (fc X/Y) Symbolic nodename. The three most common types of zone members are the pwwn, FC alias and the switch interface. Tip It is recommended that pwwn (or an FC Alias representing a pwwn) be used for zoning as it provides the most security and ties a zone member to a specific hba rather than to the switch port. Equally important is the name of the zone. Many environments use different zone names, however, all name formats should provide relevant information as to their contents. Names like Zone1 or TapeZone do not provide sufficient information about their contents. Tip A zone name, should contain two members and within the zone name contain identifiers related to the two devices (eg: Z_testhost_fcaw0_symm13FA3aa). It may be longer than Z_testhost_hba0, but provides detailed information about the contents without having to consult further documentation. 7-93

94 Zones Chapter 7 Zoning Creating a zone and adding it to a zoneset (standalone method) This procedure will demonstrate how to create a single zone with a solaris host and a disk storage port in it, then add it to the zoneset ZS_Engr_primary. This will utilize the standalone method, which will not automatically add the zone to the zoneset upon creation of the zone. This procedure also can be used to to determine how to add an existing zone to a zoneset. This example will use pwwns as the zone members, which can be obtained either from the device itself or from the show flogi database vsan 804 command. Zoneset: ZS_Engr_primary Solaris host1, hba instance fcaw0: 22:35:00:0c:85:e9:d2:c2 Symmetrix 78, FA port 03ab: 10:00:00:00:c9:32:8b:a8 Figure 7-1 Standalone Zoning Topology Step 1 Create the zone, building a zone name that reflects the names of the members. ca-9506# config terminal ca-9506(config)# zone name Z_host1_fcaw0_symm78FA03ab vsan 804 ca-9506(config-zone)# member pwwn 22:35:00:0c:85:e9:d2:c2 ca-9506(config-zone)# member pwwn 10:00:00:00:c9:32:8b:a8 Step 2 Then add the zone to the zoneset ca-9506(config)# zoneset name ZS_Engr_primary vsan 804 ca-9506(config-zoneset)# member Z_host1_fcaw0_symm78FA03ab Step 3 Display the zoneset: ca-9506# show zoneset name ZS_Engr_primary vsan 804 zoneset name ZS_Engr_primary vsan 804 zone name Z_host1_fcaw0_symm78FA03ab vsan 804 pwwn 22:35:00:0c:85:e9:d2:c2 pwwn 10:00:00:00:c9:32:8b:a8 Step 4 Finally, to put the zoneset into production activate the zoneset using zoneset activate name ZS_Engr_primary vsan 804. This will activate all the zones in the zoneset, not just the one that was just added. 7-94

95 Chapter 7 Zoning Zones Creating a zone and adding it to a zoneset (inline method) This procedure will demonstrate how to create a single zone with a solaris host and a disk storage port in it, then add it to the zoneset ZS_Engr_primary. This will utilize the inline method, which will automatically add the zone to the zoneset upon creation of the zone. This example will use pwwns as the zone members, which can be obtained either from the device itself or from the show flogi database vsan 804 command. Zoneset: ZS_Engr_primary Solaris host1, hba instance fcaw0: 22:35:00:0c:85:e9:d2:c2 Symmetrix 78, FA port 03ab: 10:00:00:00:c9:32:8b:a8 Figure 7-2 Inline Zoning Topology Step 1 Step 2 Step 3 Step 4 Enter the submode of the zoneset ca-9506# config terminal ca-9506(config)# zoneset name ZS_Engr_primary vsan 804 Create the zone ca-9506(config-zoneset)# zone name Z_host1_fcaw0_symm78FA03ab Add the members ca-9506(config-zoneset-zone)# member pwwn 22:35:00:0c:85:e9:d2:c2 ca-9506(config-zoneset-zone)# member pwwn 10:00:00:00:c9:32:8b:a8 Display the zoneset: ca-9506# show zoneset name ZS_Engr_primary vsan 804 zoneset name ZS_Engr_primary vsan 804 zone name Z_host1_fcaw0_symm78FA03ab vsan 804 pwwn 22:35:00:0c:85:e9:d2:c2 pwwn 10:00:00:00:c9:32:8b:a8 Step 5 Finally, to put the zoneset into production activate the zoneset using zoneset activate name ZS_Engr_primary vsan 804. This will activate all the zones in the zoneset, not just the one that was just added. 7-95

96 Zones Chapter 7 Zoning Creating a FC Alias based zone Fibre Channel aliases allow the administrator toassign a plain text, human readable name to a pwwn, fcid, interface, ip-address, nwwn or symbolic-nodename. FC Aliases are restricted to the VSAN for which they were created in. The most common and recommended is the pwwn, which will be the basis for this procedure. Tip Aliases are distributed with the full zoneset database, thus if zoning is going to be editted on multiple switches, full zoneset distribution should be enabled. An alias can be mapped to more than one device, however it is recommended that a one to one mapping be used. The following resources will be used: Zoneset: ZS_Engr_primary Solaris host1, hba instance fcaw0: 22:35:00:0c:85:e9:d2:c2 Symmetrix 78, FA port 03ab: 10:00:00:00:c9:32:8b:a8 Figure 7-3 Alias Zoning Topology Step 1 Create the FC Alias to pwwn mappings. ca-9506# config terminal ca-9506(config)# fcalias name host1_fcaw0 vsan 804 ca-9506(config-fcalias)# member pwwn 22:35:00:0c:85:e9:d2:c2 ca-9506(config-fcalias)# exit ca-9506(config)# fcalias name symm78_fa03ab vsan 804 ca-9506(config-fcalias)# member pwwn 10:00:00:00:c9:32:8b:a8 ca-9506(config-fcalias)# end Step 2 Display the newly created FC Aliases. ca-9506# show fcalias vsan 804 fcalias name host1_fcaw0 vsan 804 pwwn 22:35:00:0c:85:e9:d2:c2 fcalias name symm78_fa03ab vsan

97 Chapter 7 Zoning Zones pwwn 10:00:00:00:c9:32:8b:a8 Step 3 Create the zone using the FC Aliases ca-9506# config terminal ca-9506(config)# zoneset name ZS_Engr_primary vsan 804 ca-9506(config-zoneset)# zone name Z_host1_fcaw0_symm78FA03ab ca-9506(config-zoneset-zone)# member fcalias host1_fcaw0 ca-9506(config-zoneset-zone)# member fcalias symm78_fa03ab Step 4 Optionally, display the zoneset. ca-9506# show zoneset vsan 804 zoneset name ZS_Engr_primary vsan 804 zone name Z_host1_fcaw0_symm78FA03ab vsan 804 fcalias name host1_fcaw0 vsan 804 pwwn 22:35:00:0c:85:e9:d2:c2 fcalias name symm78_fa03ab vsan 804 pwwn 10:00:00:00:c9:32:8b:a8 Step 5 Activate the zoneset ca-9506# conf t ca-9506(config)# zoneset activate name ZS_Engr_primary vsan 804 Zoneset activation initiated. check zone status Creating an Interface based zone This procedure will demonstrate how to create a zone based upon the physical interface(fcx/y) of the switch. Tip Interface based zoning should be done when a zone needs to be created prior to the hba being connected to the fabric. After, the hba is connected to the fabric, the zone member should be converted to a pwwn based member. 7-97

98 ZoneSets Chapter 7 Zoning Figure 7-4 Interface Zoning Topology Step 1 Create the zone using the interfaces ca-9506# config terminal ca-9506(config)# zoneset name ZS_Engr_primary vsan 804 ca-9506(config-zoneset)# zone name Z_host1_fcaw0_symm78FA03ab ca-9506(config-zoneset-zone)# member interface fc1/1 ca-9506(config-zoneset-zone)# member interface fc1/2 Step 2 Optionally, display the zoneset. ca-9506# show zoneset vsan 804 zoneset name ZS_Engr_primary vsan 804 zone name Z_host1_fcaw0_symm78FA03ab vsan 804 interface fc1/1 swwn 20:00:00:0c:85:e9:d2:c0 interface fc1/2 swwn 20:00:00:0c:85:e9:d2:c0 Note Step 3 The swwn is the switch s wwn as displayed by the show wwn switch command: ca-9506# show wwn switch Switch WWN is 20:00:00:0c:85:e9:d2:c0 Activate the zoneset ca-9506# conf t ca-9506(config)# zoneset activate name ZS_Engr_primary vsan 804 Zoneset activation initiated. check zone status ZoneSets Zonesets are containers of zones and of which there are two types on the MDS platform: 1. Active Zoneset: is the rules by which the MDS platform enforces its zoning security policy.it cannot be modified and is distributed to all switches in the VSAN. There are specific rules to merging the active zoneset when two switches are connected by an ISL as set by the FC standards. 2. Local Zoneset: The local zonesets are contained in the full zoneset database on the switch. The zonesets are editted directly and then activated to become the active zoneset. They can optionally be distributed to other switches, either manually or when a zoneset is activated. Zoneset Distribution Automatic Zoneset Distribution To enable the switch to distribute the local zoneset to all other switches in the VSAN when a zoneset is activated, use the following command: ca-9506# conf t 7-98

99 Chapter 7 Zoning ZoneSets ca-9506(config)# zoneset distribute full vsan 804 Tip This feature should be enabled on all switches in the fabric, and can be initially specified in the initial setup script. Manual Zoneset Distribution: To distribute the full zoneset database to other switches without activating a zoneset: This can be effective when a new switch is brought into the fabric and the zoneset with its zones and FC aliases need to be distributed. This command will overwrite the exiting zoneset database in the target switch. ca-9506# zoneset distribute vsan 804 Zoneset distribution initiated. check zone status 7-99

100 ZoneSets Chapter 7 Zoning 7-100

101 CHAPTER 8 iscsi iscsi is a transport protocol which is used to transport SCSI packets over TCP/IP. It is an internet protocol based standard that allows hosts to connect and to access storage over a network interface card. iscsi is used to facilitate data transfers over intranets and to manage storage over long distances. Since iscsi runs over IP it can be used to tramsmit data over LAN, WAN, MAN etc there by enabling data access independent of the location of the storage sub system. The MDS 9200 and 9500 series switches support iscsi using the IPS-8, IPS-4 and the 14+2 blades. Enabling iscsi This command has to be run before attempting to configure iscsi on the switch. Warning The iscsi enable command is required prior to performing any iscsi configuration using either the CLI or via Device/Fabric Manager. MDS1# conf t MDS1(config)# iscsi enable MDS1(config)#^Z MDS1# Configuring iscsi in transparent mode The recipe below shows the configuration of iscsi on the MDS. Transparent mode configures an equivilant fibre channel initiator for each iscsi initiator. In this process, no lun masking or reassignment is done on the MDS. For larger installations, iscsi should be configured using the proxy initiator mode, Configuring iscsi in Proxy initiator mode, page This recipie uses the transparent mode to configure iscsi on MDS. The topology is as shown in Figure

102 Configuring iscsi in transparent mode Chapter 8 iscsi Figure 8-1 iscsi Topology The topology consists of a w2k server with a dedicated gige NIC for iscsi. It is connected to a port on the catalyst The iscsi interface on the host is assigned the ip address The IPS port 8/1 on the mds is also connected to the Catalyst and has an ip address The storage port from the array is connected to the FC port 1/4 on MDS This example shows how to configure an iscsi initiator using iqn ( iscsi qualified name). Warning The ipaddress for the ports on the ips blade should be in a different subnet and ethernet segment than the management interface. This is critical to get iscsi working on the MDS. The following needs to be done to make iscsi initiator see drives from the array. Step 1 Step 2 Step 3 Configure the gige interface on the MDS switch The gige interface on the MDS switch is given an ip address, a subnetmask. This allows the gige interface to communicate with the network. sjc # conf t sjc (config)# interface gigabitethernet 8/1 sjc (config-if)# ip address sjc (config-if)# ^Z sjc # Configure ip route if required IP route needs to be configured to be able to communicate with the initiator is the initiator is in another subnet. In the above recipie both the initiator and the iscsi interface on the MDS are in the same subnet. This does not require the configuration of an explicit route on the MDS. Syntax for configuring a route is shown below. Here an initiator is in the subnet net hence a route is added to reach that subnet from the MDS switch. sjc # conf t sjc (config)# ip route sjc (config-if)# ^Z sjc # Configure the ip address on the iunterface on the win2k server. Configure the ip address on the network interface dedicated for iscsi on the win2k server. IP route may need to be added on the host to reach the gige port on the MDS. In this recipie it is not needed as both the initiator and the gige port are on the same subnet. Once this is done ping the gige port on the MDS to confirm connectivity. Similarly ping the host NIC from the MDS gige port. Note Step 4 It is critical to check the connectivity between the host NIC card and the gige port on the MDS IPS blade before proceeding further. A successful ping test should be sufficient. Enable the iscsi interface on the MDS

103 Chapter 8 iscsi Configuring iscsi in transparent mode Step 5 The iscsiinterface 8/1 (same port as the gige interface) needs to be enabled. Along with the enabling of the iscsi interface additional iscsi relate TCP tunning can also be done. In this recepie the default values are used. sjc # conf t sjc (config)# interface iscsi 8/1 sjc (config-if)# no shut sjc (config-if)# ^Z sjc # Configure the initiator on MDS9509. This can be done in multiple ways. It can be done using the ip address of the initiator or as a proxy initiator or using an iqn ( iscsi Qualified Name), This example will use iqn name. Most iscsi drivers / clients can automatically assign an iqn name to the on the host. The iqn name has to be atleast 16 charecters long. The iqn name can also be manually assigned. If it is being manually assigned care has to taken to ensure that the iqn name is unique. The win2k server used in the The microsoft driver pre configures the iqn name at the time of install. This can be viewed as shown below. In the iscsi utility select Initiator Settings, this shows the iqn name for the host. The driver assigned initiator node name is iqn com.microsoft:sjc7-pc-1.the Figure 8-2 shows the iqn name in the iscsi driver interface. For linux OS this can be found in the /etc/initiatorname.iscsi file. Figure 8-2 iscsi initiator Properties Step 6 Configure the iscsi initiator on MDS

104 Configuring iscsi in transparent mode Chapter 8 iscsi a. Specify the pwwn manually sjc # conf t sjc (config)# iscsi initiator name iqn com.microsoft:sjc7-pc-1 sjc (config-(iscsi-init))# static pwwn dec022d82 <-- manually assigned sjc (config-(iscsi-init))# vsan 3003 <-- Must be a member in the Targets VSAN sjc (config-(iscsi-init))# ^Z sjc # b. Allow the MDS to determine the pwwn (preferred) sjc # conf t sjc (config)# iscsi initiator name iqn com.microsoft:sjc7-pc-1 sjc (config-(iscsi-init))# static pwwn system-assign 1 <-- system assigned sjc (config-(iscsi-init))# vsan 3003 <-- Must be a member in the Targets VSAN sjc (config-(iscsi-init))# ^Z sjc # In the recipe the iqn assigned by the driver is used. If it needs to be changed make sure that it is unique and is at least 16 charecters long. Optionally a pwwn can also be assigned to the initiator. The pwwn can be statically assigned by the administrator as shown above or the system can automatically assign one. The initiator can be part of multiple VSANs. In order to access the Target it has to be a member of the target s VSAN. In the above example the target belongs to VSAN Note Alternatively the IP address of the iscsi initiator can be used for the configuration. Also assigning a static pwwn is also optional. While doing zoning an iscsi initerface, its ip address can be used in place of its pwwn or the iqn name. An example of the using ip address is shown below. sjc # conf t sjc (config)# iscsi initiator ipaddress sjc (config-(iscsi-init))# static pwwn system-assign 1 <-- system assigned sjc (config-(iscsi-init))# vsan 3003 <-- Must be a member in the Targets VSAN sjc (config-(iscsi-init))# ^Z sjc # Tip For the iscsi initiator to communicate with a target port, the iscsi initiator has to made a member of the target port s VSAN. iscsi initiators can be members of multiple VSANs. Step 7 Configure virtual target on MDS 9509 sjc # conf t sjc (config)# iscsi virtual-target name sym2127-fa4ba-sjc7 sjc (config-(iscsi-tgt))# pwwn 50:06:04:82:bf:d1:db:d3 sjc (config-(iscsi-tgt))# ^Z sjc # Step 8 The virtual target is given a name. Again the name has to at least 16 charecters long. Then the pwwn of the storage port is assigned to the virtual target as shown above. This configures a virtual target. Allow the initiator to communicate with the virtual target 8-104

105 Chapter 8 iscsi Configuring iscsi in transparent mode Step 9 Then the virtual target can permit all initiators or selected initiator to communicate with it. this is configured as shown below. sjc # conf t sjc (config)# iscsi virtual-target name sym2127-fa4ba-sjc7 sjc (config-(iscsi-tgt))#initiator iqn com.microsoft:sjc7-pc-1 permit sjc (config-(iscsi-tgt))# ^Z sjc # This permits the initator iqn com.microsoft:sjc7-pc-1 to communicate with the virtual target sym2127-fa4ba-sjc7. Create a zone with the initiator and the virtual target To create a zone with the iscsi initiator and the virtual targets are members.this enables the initiator and the target to communicate. The zone can be created either with the iqn name or with the ip address or with the pwwn that was assigned to the initiator. a. zoning with the pwwn of the target and initator. sjc # conf t sjc (config)# zone name Z_iscsi_sjc7-pc-1 vsan 3003 sjc (config-zone)# member pwwn 50:06:04:82:bf:d1:db:d3 sjc (config-zone)# member pwwn 21:05:00:0d:ec:02:2d:82 sjc (config-zone)# ^Z sjc # b. Zoning with the pwwn of the virtual target and the iqn of the iscsi initiator. sjc # conf t sjc (config-zone)# member pwwn 50:06:04:82:bf:d1:db:d3 sjc (config-zone)# member symbolic-nodename iqn com.microsoft:sjc7-pc-1 sjc (config-zone)# ^Z sjc # Step 10 Activate the zoneset The zone then has to be made a member of the active zoneset in vsan sjc (config)# zoneset name ZS_iSCSI vsan 3003 ssjc (config-zoneset)# member Z_iscsi_sjc7-pc-1 sjc (config-zoneset)# exit sjc (config)# zoneset activate name ZS_iSCSI vsan 3003 Zoneset activation initiated. check zone status sjc (config-zone)# ^Z Then check the active zone set to see if they are both active.this completes the configuration on the MDS. sjc # sh zoneset active vsan 3003 zoneset name ZS_iSCSI vsan 3003 zone name Z_iscsi_sjc7-pc-1 vsan 3003 * fcid 0xd90002 [symbolic-nodename iqn com.microsoft:sjc7-pc-1] * fcid 0xd90000 [pwwn 50:06:04:82:bf:d1:db:d3] sjc # 8-105

106 Configuring the iscsi Host Chapter 8 iscsi Configuring the iscsi Host iscsi clients on Windows Platform This section details the configuration of a Microsoft Windows iscsi driver 1.05a configuration. The steps to configure the iscsi client are as shown below. Figure 8-3 iscsi add target portals Step 1 Add target portal Select Target Portals in the iscsi client interface and the Add button to add a target portal on the w2k host. This shown in Figure 8-3. This will bring up the Add Target Portal dialogue. In the Add Target Portal diaglogue add the ip address assigned to gige port on MDS9509. In this case it is This is shown in Figure

107 Chapter 8 iscsi Configuring the iscsi Host Figure 8-4 Add Target Portal window Figure 8-5 iscsi interface showing the added target Step 2 This then adds the target portal address to the list of target portals that the iscsi client can logon to. This is shown in Figure 8-5. Log on to the target Select the Available Targets tab in the iscsi client window. If the client can see the target then the virtual configured and zoned to this initiator should be visible under this tab. Some times it takeas a refresh to see this window populated. This is shown in Figure 8-6. If the target still does not appear check the configuration on the switch and the iscsi driver sub sybsystem on the host

108 Configuring the iscsi Host Chapter 8 iscsi Figure 8-6 iscsi available Targets view A seen in the Figure 8-6 the status of the target is inactive. Now highlight the target and select Log On in the window. This brings up the iscsi logon to target window shown in Figure 8-7. Here there is an option of making the system automatically log on to the target automatically and also to enable multipath if multipathing software is installed on the server. Clicking on OK starts the iscsi login and storage discovery process for the target listed in the window.an iscsi initiator can see multiple targets. The status of the target in the Available Targets then changes to connected once the initiator successfully logs on to the target.this is shown in Figure 8-8. Figure 8-7 Log On to Target Dialogue At this stage the host should be able to see the storage made available ( directly assigned ot made available through LUN masking or otherwise) to the iscsi initiator

109 Chapter 8 iscsi Configuring the iscsi Host Figure 8-8 Successful iscsi log On The Active Sessions tab and the details button in that tab shows the LUNs visible to the initiator. Now from the MSwindows Start -->Settings -->Control Panel --> Administrative Tools --> Computer management -->Disk Management the user can scan for new disk and initialize them for use on the system. iscsi clients on Linux This section talks about the configuration that needs to be performed on a linux client to get iscsi client going on a Lunux server. To enable iscsi to work on a linux host the iscsi driver has to download and configured. The link hosts the iscsi drivers. In some instance the driver may need to be compiled. Once the driver is instaled on the host it can be configured by editing the iscsi.conf file. This usually present in the /etc directory on the linux server. Again as in case of the windows configuration the target portal has to be added and the iscsi initiator needs to be configured on the MDS. Then the iscsi initiator on the linux host has to be restarted to log on to the target to see the LUNs. As in the case of the Windows iscsi driver the iscsi initiator s name is auto generated by the driver subsystem. The auto generated name is found in the /etc/initiatorname.iscsi file. Step 1 Configuring the iscsi initiator on the host The /etc/iscsi.conf file has to be edited to make iscsi work on a linux server. The following changes have to mage in the iscsi.conf file. Under the DiscoverAddress Settings section add the address of the target portal. in this case it would be

110 Configuring iscsi in Proxy initiator mode Chapter 8 iscsi # DiscoveryAddress Settings # # Add "DiscoveryAddress=xxx" entries for each iscsi router instance. # The driver will attempt to discover iscsi targets at that address # and make as many targets as possible available for use. # 'xxx' can be an IP address or a hostname. A TCP port number can be # specified by appending a colon and the port number to the address. # All entries have to start in column one and must not contain any # whitespace. # # Example: # DiscoveryAddress= Step 2 This is all that is needed to changed to get basic iscsi configured on the linux server. Restaring the iscsi process. Once the target portal address has been updated the iscsi processes have to be restarted to force the iscsi client to log on to the target and to discover the luns. This is done using the rc scripts. The rc scripts are located in /etc/rc3.d directory. It is S25iscsi script. This script needs to be run with a restart option after the changes to the /etc/iscsi.conf are made. The script also have a status option to check the status of the iscsi processes on the system [root@sjc7-pc-6 rc3.d]# /etc/rc3.d/s25iscsi restart Stopping iscsi: sync umount sync iscsid iscsi Starting iscsi: iscsi iscsid [root@sjc7-pc-6 rc3.d]# /etc/rc3.d/s25iscsi status iscsi driver is loaded [root@sjc7-pc-6 rc3.d]# At this stage the iscsi initiator should log on to the target and discover the luns that have been assigned to it. Configuring iscsi in Proxy initiator mode The recipie below details the proxy mode configuration for iscsi on MDS switch. This method is the prefred mode for configuring a large number of iscsi clients to work with the switch. In proxy initiator mode the one fibre channel initiator is used for all iscsi clients that access the MDS via the same iscsi interface (iscsi3/3 for example). The initiators will use the PWWN assigned to the iscsi interface. The iscsi interface to which an iscsi client will logon to is configured in the client and must be permited by the virtual target configured for that initiator. Proxy mode is advantageous over transparent mode when the configuration requires multiple iscsi initiators to access the same fibre channel target. For example if 20 iscsi initiators need to comunicate with a fibre channel disk, in transparent mode 20 iscsi initators, 20 zones need to be created and also array based lun masking has to be updated for all 20 initiator instances. On the other hand, proxy initiator mode is far easier to manage as it allows for centralized management of the iscsi configuration, as all iscsi clients accessing the same switch interface will use a single fibre channel initiator

111 Chapter 8 iscsi Configuring iscsi in Proxy initiator mode First, a pwwn is assigned to iscsi interface. Then this one pwwn, is zoned with the fibre channel target so that the proxy initiator can see the luns presented by the virtual target. All the lun masking and zoning need to be performed only with the proxy initiator. As new hosts (iscsi clients) are added they only need to be exposed to the luns that they need to see. As no new zones nor modifications to the array s lun masking needs to be done. A typical practice is to create a virtual target for each hosts and configure the virtual target in such a way that they only expose the required luns to the iscsi initiator. The proxy initiator is not restricted to a single VSAN, as iscsi clients are configured and given access to different VSANs, the proxy initiator is created in the new VSAN. Therefore the maximum number of initiators that would need to be zoned would be the number of proxy initiators that have iscsi clients in a VSAN. This is far less than under transparent mode, whereby a fibre channel initiator is created for every iscsi client. The topology used for the iscsi proxy initiator recepie is shown in Figure 8-9. It has 2 w2k hosts on two different subnets. The first host;s iscsi interface is on network and the secont interface is on network. Figure 8-9 iscsi proxy initiatortopology. The configuration below shows a proxy initiator configuration. Step 1 Step 2 Configure the gige interface on the MDS switch The gige interface on the MDS switch is given an ip address, a subnetmask. This allows the gige interface to communicate with the network. sjc # conf t sjc (config)# interface gigabitethernet 8/1 sjc (config-if)# ip address sjc (config-if)# ^Z sjc # Configure ip route if required 8-111

112 Configuring iscsi in Proxy initiator mode Chapter 8 iscsi IP route needs to be configured to be able to communicate with the initiator is the initiator is in another subnet. In the above recipie both the initiator and the iscsi interface on the MDS are in the same subnet. This does not require the configuration of an explicit route on the MDS. Syntax for configuring a route is shown below. Here an initiator is in the subnet net hence a route is added to reach that subnet from the MDS switch. sjc # conf t sjc (config)# ip route sjc (config-if)# ^Z sjc # Step 3 Step 4 Enable and configure the iscsi interface on the MDS. The iscsi interface 8/1 (same port as the gige interface) needs to be enabled. Along with the enabling of the iscsi interface additional iscsi relate TCP tunning can also be done. In this recepie the default values are used. The switchport command is used to enable proxy-initiator mode for the iscsi interface 8/1. It is also used to assign pwwn to the interface as shown in the recepie below. sjc # conf t sjc (config)# interface iscsi 8/1 sjc (config-if)# switchport proxy-initiator sjc (config-if)# no shut sjc (config-if)# ^Z sjc # Configure the iscsi interface to access multiple VSANs. This needs to be done to allow the iscsi interface to communicate with the virtual target to see the luns.the commands below add the iscsi interface 8/1 into VSAN 1. All clients not explicity configured for a VSAN will end up in VSAN 1. Tip By using VSAN 1 as a default, non-production VSAN, prevents unconfigured devices from being able to access a devices in a production VSAN. In this case, the production VSAN is sjc # conf t sjc (config)# iscsi interface vsan-membership sjc (config)# vsan database sjc (config-vsan-db)# vsan 1 interface iscsi 8/1 sjc (config-vsan-db)#^z sjc # Step 5 Configure the iscsi initiator. With proxy-initiator, the proxy-initiator has a default VSAN in which all iscsi clients which are not explicitly configured for a VSAN will go into. The default value is VSAN 1. This ability allows a single iscsi client to access fibre channel targets in multiple VSANs. To add multiple VSANs, just specify more VSANs in the command. sjc # conf t sjc (config)# iscsi initiator ipaddress sjc (config-(iscsi-init))# static pwwn system-assign 1 <-- system assigned sjc (config-(iscsi-init))# vsan 3003 <-- Must be a member in the Targets VSAN sjc (config-(iscsi-init))# ^Z sjc # 8-112

113 Chapter 8 iscsi Configuring iscsi in Proxy initiator mode sjc # conf t sjc (config)# iscsi initiator ipaddress sjc (config-(iscsi-init))# static pwwn system-assign 1 <-- system assigned sjc (config-(iscsi-init))# vsan 3003 <-- Must be a member in the Targets VSAN sjc (config-(iscsi-init))# ^Z sjc # Note The command iscsi interface vsan-membership is required to make the iscsi interface as a part of the a VSAN. Step 6 Configure virtual target on MDS 9509 sjc # conf t sjc (config)# iscsi virtual-target name sym2127-fa4ba-sjc7 ssjc (config-(iscsi-tgt))# pwwn 50:06:04:82:bf:d1:db:d3 sjc (config-(iscsi-tgt))# ^Z sjc # Step 7 The virtual target is assigned a name. The name has to be at least 16 charecters long. Then the pwwn of the storage port is assigned to the virtual target as shown above. This configures a virtual target. Create a zone with the initiator and the virtual target The next step is to create a zone with the iscsi proxy interface and the virtual targets are members.this enables the initiator and the target to communicate. The zone can be created either with the interface iscsi 8/1 or with the ip address or with the pwwn that was assigned to the initiator. The pwwn of the proxy-initiator can be obtained by viewing the iscsi interface sjc # show int iscsi 3/3 iscsi8/1 is up Hardware is GigabitEthernet Port WWN is 20:89:00:0c:85:e9:d2:c0 Admin port mode is ISCSI Port mode is ISCSI Port vsan is 1 Speed is 1 Gbps iscsi initiator is identified by name Number of iscsi session: 0, Number of TCP connection: 0 Configured TCP parameters Local Port is 3260 PMTU discover is enabled, reset timeout is 3600 sec Keepalive-timeout is 60 sec Minimum-retransmit-time is 300 ms Max-retransmissions 4 Sack is enabled QOS code point is 0 Maximum allowed bandwidth is kbps Minimum available bandwidth is kbps Estimated round trip time is 1000 usec Send buffer size is 4096 KB Congestion window monitoring is enabled, burst size is 50 KB Configured maximum jitter is 500 us Forwarding mode: pass-thru TMF Queueing Mode : disabled Proxy Initiator Mode : enabled nwwn is 21:04:00:0d:ec:02:2d:82 (system-assigned) pwwn is 21:05:00:0d:ec:02:2d:82 (system-assigned) 8-113

114 Configuring iscsi in Proxy initiator mode Chapter 8 iscsi a. zoning with the pwwn of the target and proxy-initator. sjc # conf t sjc (config)# zone name Z_iscsi_sjc7-pc-1 vsan 3003 sjc (config-zone)# member pwwn 50:06:04:82:bf:d1:db:d3 sjc (config-zone)# member pwwn 21:05:00:0d:ec:02:2d:82 sjc (config-zone)# ^Z sjc # b. Zoning with the pwwn of the virtual target and the ip address of the iscsi interface. sjc # conf t sjc (config)# zone name Z_iscsi_sjc7-pc-1 vsan 3003 sjc (config-zone)# member pwwn 50:06:04:82:bf:d1:db:d3 sjc (config-zone)# member member ip-address sjc (config-zone)# ^Z sjc # Step 8 Activate the zoneset The zone then has to be made a member of the active zoneset in vsan 3003, and then activated. sjc (config)# zoneset name ZS_iSCSI vsan 3003 sjc (config-zoneset)# member Z_iscsi_sjc7-pc-1 sjc (config-zoneset)# exit sjc (config)# zoneset activate name ZS_iSCSI vsan 3003 Zoneset activation initiated. check zone status sjc (config-zone)# ^Z sjc # show zoneset active vsan 3003 zoneset name ZS_iscsi vsan 3003 zone name Z_iscsi_proxy_8-1 vsan 3003 * fcid 0xd90002 [pwwn 21:05:00:0d:ec:02:2d:82] * fcid 0xd90000 [pwwn 50:06:04:82:bf:d1:db:d3] sjc # Step 9 Configure a virtual target for each initiator and configure lun masking for the initiator Once the zone is successfully activated the luns made available on the storage port should now be visible to the iscsi interface. As this interface could be a proxy iscsi interface for many iscsi initiators some form of lun security needs to enabled. To achieve lun security a virtual target with access to specific luns needs to be created for each initiator. In this recepie the iscsi interface can see 10 luns (lun 11 to lun 20 in decimal). a. The configuration below will allow the host sjc7-pc-1 to see luns ( decimal) on the array. sjc # config t sjc (config)# iscsi virtual-target name sym2127-fa4ba-sjc7-1 sjc (config-(iscsi-tgt))# pwwn 50:06:04:82:bf:d1:db:d3 fc-lun b iscsi-lun 1 sjc (config-(iscsi-tgt))# pwwn 50:06:04:82:bf:d1:db:d3 fc-lun b iscsi-lun 2 sjc (config-(iscsi-tgt))# pwwn 50:06:04:82:bf:d1:db:d3 fc-lun d iscsi-lun 3 sjc (config-(iscsi-tgt))# pwwn 50:06:04:82:bf:d1:db:d3 fc-lun e iscsi-lun 4 sjc (config-(iscsi-tgt))# initiator ip address permit sjc (config-(iscsi-tgt))# end sjc # b. Similarly for the hosts sjc7-pc-2, the following needs to be done to make it see luns ( decimal)

115 Chapter 8 iscsi Configuring iscsi in Proxy initiator mode sjc # config t sjc (config)# iscsi virtual-target name sym2127-fa4ba-sjc7-2 sjc (config-(iscsi-tgt))# pwwn 50:06:04:82:bf:d1:db:d3 fc-lun 10 iscsi-lun 1 sjc (config-(iscsi-tgt))# pwwn 50:06:04:82:bf:d1:db:d3 fc-lun 11 iscsi-lun 2 sjc (config-(iscsi-tgt))# pwwn 50:06:04:82:bf:d1:db:d3 fc-lun 12 iscsi-lun 3 sjc (config-(iscsi-tgt))# pwwn 50:06:04:82:bf:d1:db:d3 fc-lun 13 iscsi-lun 4 sjc (config-(iscsi-tgt))# pwwn 50:06:04:82:bf:d1:db:d3 fc-lun 14 iscsi-lun 5 sjc (config-(iscsi-tgt))# initiator ip address permit sjc (config-(iscsi-tgt))# end sjc # After the above configuration changes both the hosts should be able to see the luns allocated to them through the virtual target created for each. As can be seen there is no need to create additional zones when new iscsi client are added and only need to see the already zoned in target. If the iscsi clients need access additional targets the iscsi interface needs to be zoned to the other targets as shown in step 6 and seven above. The client side iscsi configuration is as shown in the Configuring the iscsi Host, page The same is very true for this configuration also

116 Configuring iscsi in Proxy initiator mode Chapter 8 iscsi 8-116

117 CHAPTER 9 FCIP FCIP (Fibre Channel over IP) is a IETF standards based protocol for connecting Fibre Channel SANs over IP based networks. FCIP encapsulates the FCP frames in a TCP/IP packet which is then sent across an IP networks. FCIP is used to interconnect geographically dispersed SANs using the IP backbone network. In short FCIP creates a FC tunnel over an existing IP network. The MDS 9200 and 9500 series switch support FCIP. This is done using the IPS-8, IPS-4 and the 14+2 blade. Enabling FCIP This command has to be run before attempting to configure FCIP on the switch. Warning If the fcip enable command is not run, further FCIP configuration is not possible. This command will enable further FCIP configuration options in the CLI. MDS1# conf t MDS1(config)# FCIP enable MDS1(config)#^Z MDS1# Configuring FCIP The recipe below shows the configuration of FCIP on the MDS. The topology is as shown in Figure 9-1. Figure 9-1 FCIP Topology 9-117

118 Configuring FCIP Chapter 9 FCIP The topology consists of each with one IPS-8 blased in them. The gige port 8/8 on the switch sjc is connected to a catalyst The interface has an ip address with a subnet mask of and the gateway address being Another switch sjc , the gige port 8/3 is also connected to a catalyst This interface has an ip address with a subnet and the gateway address being In the recipie the a FCIP tunnel will be establisged between the switch sjc (gige 8/8) and sjc (gige 8/3). Warning The ipaddress for the ports on the IPS blade must be on a different subnet and ethernet segment than the management interface. The different segment can be either physical separation or using VLANs. This is critical to get FCIP working on the MDS. The following configuration needs to be done to make FCIP operational between the switches. Step 1 Configure the gige interface on the MDS switches The gige interface on the MDS switch sjc is given an ip address, a subnetmask. This allows the gige interface to communicate with the network. sjc # conf t sjc (config)# interface gigabitethernet 8/3 sjc (config-if)# ip address sjc (config-if)# no shut sjc (config-if)# ^Z sjc # Step 2 Configure ip routes so that the 2 gige interaces can communicate with each other IP routes needs to be configured to allow the two gige ports on switches sjc and sjc to communicate with each other. In the above recipe the gige ports in two different subnets. This does require explicit reoutes to be able to communicate with each other. Note The recommended best practice is just to have a host route to each of the two gige interfaces. I.e. only allow the two gige interfaces to communicate with each other, This is done by having a subnet mask of when the route is added. Syntax for configuring a route is shown below. For the gige port 8/8 on the switch sjc to communicate with the port gige 8/3 on switch sjc the follwoing is the route configuration. sjc # conf t sjc (config)# ip route interface gigabitethernet 8/8 sjc (config)# ^Z sjc # The above configuration means, in order to reach use the gateway and interface gige 8/8 on switch sjc Similarly for the gige port 8/3 on the switch sjc to communicate with the port gige 8/8 on switch sjc the follwoing is the route configuration. sjc # conf t 9-118

119 Chapter 9 FCIP Configuring FCIP Step 3 sjc (config)# ip route interface gigabitethernet 8/3 sjc (config-if)# ^Z sjc # The above configuration means, in order to reach use the gateway and interface gige 8/3 on switch sjc Ping the gige interfaces to ensure that the gige ports can communicating with each other From the switch sjc ping the ip address of the gige interface 8/3 on switch sjc Similarly ping the ip address of the gige interface 8/8 on switch sjc from switch sjc This can be done from the switch prompt. sjc # ping PING ( ) 56(84) bytes of data. 64 bytes from : icmp_seq=1 ttl=254 time=1.04 ms 64 bytes from : icmp_seq=2 ttl=254 time=0.624 ms 64 bytes from : icmp_seq=3 ttl=254 time=0.678 ms 64 bytes from : icmp_seq=4 ttl=254 time=0.580 ms ping statistics packets transmitted, 4 received, 0% packet loss, time 3008ms rtt min/avg/max/mdev = 0.580/0.732/1.046/0.184 ms sjc # sjc # ping PING ( ): 56 data bytes 64 bytes from : icmp_seq=0 ttl=254 time=0.7 ms 64 bytes from : icmp_seq=1 ttl=254 time=0.6 ms 64 bytes from : icmp_seq=2 ttl=254 time=0.5 ms 64 bytes from : icmp_seq=3 ttl=254 time=0.6 ms ping statistics packets transmitted, 4 packets received, 0% packet loss round-trip min/avg/max = 0.5/0.6/0.7 ms sjc # Note Step 4 Step 5 It is critical to check the connectivity between the two gige ports on each MDS IPS blades before proceeding further. A successful ping test should be sufficient. Measure the round trip time between the two gige interfaces. It is important to measure the round trip time between the interfaces. This value is required for later use in the configuration. sjc # ips measure-rtt interface gigabitethernet 8/8 Round trip time is 691 micro seconds (0.69 milli seconds) sjc # sjc # ips measure-rtt interface gigabitethernet 8/3 Round trip time is 743 micro seconds (0.74 milli seconds) sjc # Configure the FCIP profile on both the switches. FCIP profile needs to be created. The profile defines the characteristics for the FCIP tunnel that is setup. The round trip time measured is needed during the profile configuration. It is rounded to the nearest whole number. In this case the 690 micro seconds is rounded to 1 mili second. The ip address used in the config below is teh ip address assigned to the gige interface on the switch

120 Configuring FCIP Chapter 9 FCIP Tip It is recommended that the FCIP profile and FCIP interface numbers number at both ends of a FCIP tunnel be the same for easier management. sjc # conf t sjc (config)# fcip profile 10 sjc (config-profile)# ip address sjc (config-profile)# tcp max-bandwidth-mbps 1000 min-available-bandwidth-mbps 200 round-trip-time-ms 1 sjc (config-if)# ^Z sjc # sjc # conf t sjc (config)# fcip profile 10 sjc (config-profile)# ip address sjc (config-profile)# tcp max-bandwidth-mbps 1000 min-available-bandwidth-mbps 200 round-trip-time-ms 1 sjc (config-if)# ^Z sjc # Step 6 Configure the fcip interface on both the switches. The fcip interfaces need to be configured on both the switches. The configuration is as shown below.in the fcip interface configuration the profile to used and the peer information ( remote gige s ip address are specified.)are specified. Additionally compression and write acceleration can also be configured. sjc # conf t sjc (config)# interface fcip 1 sjc (config-if)# use-profile 10 sjc (config-if)# peer-info ipaddr sjc (config-if)# no shut sjc (config-if)# ^Z sjc # sjc # conf t sjc (config)# interface fcip 1 sjc (config-if)# use-profile 10 sjc (config-if)# peer-info ipaddr sjc (config-if)# no shut sjc (config-if)# ^Z sjc # The FCIP tunnel should be up and running now. a show interface fcip 1 brief should show the status of the FCIP link between the two switches. sjc # show interface fcip 1 brief Interface Vsan Admin Admin Status Oper Profile Eth Int Port-channel Mode Trunk Mode Mode fcip1 1 auto on trunking TE 10 GigabitEthernet8/8 -- sjc # sjc # show interface fcip 1 brief

121 Chapter 9 FCIP Tuning FCIP Interface Vsan Admin Admin Status Oper Profile Eth Int Port-channel Mode Trunk Mode Mode fcip1 1 auto on trunking TE 10 GigabitEthernet8/3 -- sjc # Tuning FCIP Implementing a FCIP tunnel is more than just creating the FCIP link and modifying the VSAN allowed list. In order to achieve the greatest efficiency out of the link some parameters may need to be tuned which are specific to the underlying connection. Therefore, if the FCIP link is running over a slow, 1Mb connection, the FCIP link should be tuned differently than one running over a low latency 1Gb connection. This section will look to provide insight into what parameters can be used and how to use them for achieving increased efficiency and utilization out of the FCIP connection. Note Your invidual results may vary due to network conditions (existing utilization) as well as storage array and host type. The following topology will be used throughout this chapter: Figure 9-2 FCIP Topology TCP Tuning: Latency and Available Bandwidth The latency of the link is the amount of time that it takes a packet to go from one end of the FCIP link to the other. This could be due to many factors including distance as well as the number of devices that it must traverse. Even the fastest routers and switches due incurr some amount of latency. Latency cannot be eliminated, however protocols can be tuned and MDS features such as FCIP Write Acceleration (FCIP Write Acceleration, page 9-123)can be enabled to eliminate the effects of it. These features are modified in the FCIP profile. Available Bandwidth is the amount of bandwidth that the FCIP link can use on the network. You will need to define a maximum and a minimum value for the FCIP link to use in the FCIP profile

122 Tuning FCIP Chapter 9 FCIP The maximum available bandwidth value is the maximum amount of bandwidth that the FCIP link should use on the network. The minimum available bandwidth value is used as a guideline for the minimum value. Of course, if there are serious problems on the network (dropped packets, congestion), the link will go slower than the minimum value. It is recommended that the minimum value be set to what the the minimum acceptable bandwidth that the application (EMC SRDF, IBM PPRC etc..) requires. Tip For dedicated links, the minimum and maximum available bandwidth values should be set the same. Table 9-1 contains some common WAN links and their speeds. These circuits are most often used as the underlying network for a FCIP link. For example, the underlying network may be a OC3 but you may only be able to use 100Mb of that link. Tip When deploying FCIP, you should always involve the LAN and WAN teams to find out about the connectivity that they are providing you. If there are performance issues, they can often help you troubleshoot them from the network s standpoint. Involve them earlier rather than later. Circuit Name T1 T3 OC3 OC12 OC48 OC192 Speed megabits per second megabits per second 155 megabits per second 622 megabits per second 2.5 gigabits per seconds 9.6 gigabits per second Table 9-1 Common WAN Circuit Speeds To specify the minimum and maximum available bandwidth as well as the network latency perform the following procedure. For this example, we ll assume that the LAN/WAN links are a dedicated gigabit ethernet link. Step 1 Measure the round trip time between the two FCIP end points sjc # ips measure-rtt interface gigabitethernet 8/8 Round trip time is 5691 micro seconds (5.69 milli seconds) sjc # ips measure-rtt interface gigabitethernet 8/3 Round trip time is 5743 micro seconds (5.74 milli seconds) Step 2 Modify the profile, and in this command the min and max bandwidth is also specified. sjc # conf t sjc (config)# fcip profile 2 sjc (config-profile)# tcp max-bandwidth-mbps 1000 min-available-bandwidth -mbps 1000 round-trip-time-us

123 Chapter 9 FCIP Tuning FCIP sjc # conf t sjc (config)# fcip profile 2 sjc (config-profile)# tcp max-bandwidth-mbps 1000 min-available-bandwidth -mbps 1000 round-trip-time-us 5743 FCIP Write Acceleration To alleaviate the affects of latency the IPS-8 and IPS-4 blades support software compression. The configuration is shown below. Note Only in MDS version 2.0.1b and higher should write acceleration be used with Port-Channels. Step 1 Enable Write Acceleration. sjc # conf t sjc (config)# interface fcip 1 sjc (config)# write-accelerator sjc (config)# ip-compression mode high-throughput sjc (config)# ^Z sjc # sjc # conf t sjc (config)# interface fcip 1 sjc (config)# write-accelerator sjc (config)# ip-compression mode high-throughput sjc (config)# ^Z sjc # FCIP Compression FCIP based compression should be enabled when the bandwidth requirements of the application (example, SRDF, TrueCopy or PPRC) exceed the bandwidth that the LAN or WAN link can provide. For SAN-OS versions prior to 2.0, software compression have been madeavailable on the IP Services Module using two compression modes, high compression ratio and high throughput. For environments where the bandwidth speed of the WAN is relatively low (such as T3/DS 3,45Mbps and lower speeds), the high compression ratio option should be enabled For WAN links up to OC3,155Mbps, the high throughput option should be enabled. In SAN-OS version 2.0, three new modes of compression were introduced along with hardware based compression using the 14+2 module. These modules are summarized in Figure 9-3 on page Note The numbers displayed in Figure 9-3 are derived using the Canterbury Corpus test suite. Your actual performance number may vary depending on device type and data pattern. However, these should be used as a guide to determine which compression mode is most appropriate

124 Tuning FCIP Chapter 9 FCIP Figure 9-3 Approximate Application Throughput with MDS 2.0 Compression 9-124