5 CHAPTER MDS 9000 Switch and McData Dual Core Topology (Interop Mode 1) This chapter describes how to set up a basic dual-core topology with an MDS 9000 switch configured for interop mode 1 and a McData 6064 switch. Devices are connected to both core switches and all traffic must flow through both cores to reach its destination. This chapter includes the following sections: Specifications, page 5-1 Expected Topology Behavior, page 5-2 Configuration, page 5-3 Verification, page 5-7 Zoning, page 5-15 Specifications The following switches and code levels were used for this example configuration: MDS 9509 running NX-OS Release 5.2(1) McData 6064 Version 09.09.09 2 Figure 5-1 shows the topology used for this example configuration. 5-1
Expected Topology Behavior Chapter 5 MDS 9000 Switch and McData Dual Core Topology (Interop Mode 1) Figure 5-1 MDS 9509 Switch and McData Switch Dual Core Topology Target JBOD VZ1 Host 1 330693 1/31 1/37 13/1 4 28 27 Cisco MDS 9513 McData 6064 2 Gigabit ISLs Note The VZ1 JBOD is an arbitrated loop with 12 drives. Expected Topology Behavior This section covers the Fibre Channel services and features that act differently in this topology (Figure 5-1) as compared to a homogeneous, single-vendor implementation. It contains the following topics: Zoning, page 5-2 FSPF, page 5-3 Trunking and PortChannels, page 5-3 Domain IDs, page 5-3 Zoning In the core-core topology, zone members are all pwwns, because the McData domain/port nomenclature is not a valid form according to the FC standard. When a zone set activation is made at one core switch, the zone set activation is sent to the other core switch. Either core switch may make zoning changes and have them be propagated to the other core switch. 5-2
Chapter 5 MDS 9000 Switch and McData Dual Core Topology (Interop Mode 1) Configuration Both core switches provide all of the zone security for the device-attached ports because the Cisco MDS 9000 switch and the McData switches do not check the source and destination of the frame when traversing E ports. The Cisco MDS 9000 switch only provides hardware-enforced security on the ingress ports to the fabric, while the McData switch only provides soft zoning. Note After two active zone sets successfully merge, always copy the active zone set to the full zone set database prior to modifying it on the Cisco MDS 9000 switch. FSPF All links within the topology show the link cost of 500, because the McData 6064 switch used in this example only supports 2 Gbps. The McData switches load balance their routes using source/destination; the ingress edge switch uses the same ISL for all traffic that has the same source/destination pair. If the McData switch can load balance using source/destination/ox-id, then it can choose either of the ISLs for the route through the fabric. The MDS switch continues to load balance across ISLs using source/destination/ox-id of the frame. Trunking and PortChannels The lack of MDS 9000 switch-to-mds 9000-switch connections prohibits the topology from containing TE ports or PortChannels. Only standard E ports are used for the ISLs. Domain IDs The domain IDs are limited to the 97 to 127 range due to a restriction imposed by McData's inability to handle IDs outside of that range. While the MDS 9000 switch can handle domain IDs outside of this range in its normal mode of operation, the implementation of interoperability mode includes this limitation. Domain ID modifications can be handled in two ways, disruptively or nondisruptively: Disruptive This event impacts the entire switch. When changing domain IDs, McData requires the entire switch to be taken offline and/or rebooted. Nondisruptive This event is limited to the VSAN where the event is taking place. Only the MDS 9000 switch can perform this action, as the domain manager process for this VSAN is restarted and not the entire switch. This restart requires any device logged into the VSAN to log into the fabric again to obtain a new FC ID. Configuration This section describes the configuration process and includes the following topics: Configuring the McData 6064 Switch, page 5-4 Configuring the MDS 9000 Switch, page 5-7 5-3
Configuration Chapter 5 MDS 9000 Switch and McData Dual Core Topology (Interop Mode 1) Configuring the McData 6064 Switch In this example, the McData 6064 switch is configured for interop mode using Telnet and the CLI. The switch is placed in an offline state, the default zone is set to deny, the domain ID is set to 0x61 (97), and timers are altered to match the fabric. To configure the McData 6064 switch, follow these steps: Step 1 Step 2 Display current switch information. Root> show switch State: Online R_A_TOV: 100 <=== must match other fabric switches E_D_TOV: 20 <=== must match other fabric switches Preferred Domain ID: 2 Domain ID Offset: 96 Switch Priority: Never Principal Speed: 2 Gb/sec Rerouting Delay: Interop Mode: McDATA Fabric 1.0 <=== McData Fabric mode Active Domain ID: 2 World Wide Name: 10:00:08:00:88:A0:7A:74 Insistent Domain ID: Domain RSCN: Zoning RSCN: FC Address Domain: 62 (hex) Limited Fabric RSCN: Zone FlexPars: Fabric Filtering RSCN Zone Isolation: Fabric Filtering Safe Zoning: ISL FSPF Cost: By Port Speed Web : Frame Resize State: Block Dup WWN State: API : API Management IP: 172.25.242.164 Password Exp. Time: 0 Alter the timers, RA_TOV and ED_TOV, if needed. These options are listed in the config switch menu. Root> config switch Config.Switch>.. - Move back one level FrameResizeState - Set the FRAME_RESIZE enabled state RSCNZoneIsolation - Set the RSCN Zone Isolation state apistate - Set the api enabled state banner - Configure the banner blockdupwwn - Set Block duplicate WWN state commadelim - Toggle comma-delimited display mode domainidoffset - Set the domain ID offset for the switch domainrscn - Set the switch state for generation of domain RSCN's edtov - Set the E_D_TOV for the switch insistdomainid - Set the Insistent domain ID state for the switch interopmode - Set the Interoperability mode for the switch islfspfcost - Sets the FSPF cost of ISLs login - Login to CLI under different access rights logout - Logout of CLI ltdfabrscn - Set the switch state for limiting Fabric RSCN's prefdomainid - Set the preferred domain ID for the switch priority - Set the switch priority ratov - Set the R_A_TOV for the switch reroutedelay - Enable or disable the reroute delay 5-4
Chapter 5 MDS 9000 Switch and McData Dual Core Topology (Interop Mode 1) Configuration root safezoning setipv6 show speed webstate zoneflexpars zoningrscn - Move back to root level - Set the fabric state for prohibiting Default Zoning - Sets the enabled state of IPv6 - Display the switch configuration - Set the switch operating speed - Set the web enabled state - Set the zone FlexPars state - Set the Zoning RSCN state for the switch Step 3 Set the switch offline. The switch must be offline before changing the mode to interop. Config.Switch> interopmode open Error 045: Not Allowed While Switch Online Note To change to interop mode, the switch must be set to offline. Config.Switch> root Root> maint system Maint.System> setonlinestate 0 Maint.System> root Root> show switch State: Offline <==== switch is now offline R_A_TOV: 100 E_D_TOV: 20 Preferred Domain ID: 2 Domain ID Offset: 96 Switch Priority: Never Principal Speed: 2 Gb/sec Rerouting Delay: Interop Mode: McDATA Fabric 1.0 Active Domain ID: 2 World Wide Name: 10:00:08:00:88:A0:7A:74 Insistent Domain ID: Domain RSCN: Zoning RSCN: FC Address Domain: 62 (hex) Limited Fabric RSCN: Zone FlexPars: Fabric Filtering RSCN Zone Isolation: Fabric Filtering Safe Zoning: ISL FSPF Cost: By Port Speed Web : Frame Resize State: Block Dup WWN State: API : API Management IP: 172.25.242.164 Password Exp. Time: 0 Step 4 Set the mode to interop. Root> config switch Config.Switch> interopmode open Warning: Zone members configured by domain and port will be ignored. Config.Switch> root Root> show switch State: Offline <==== still offline R_A_TOV: 100 E_D_TOV: 20 Preferred Domain ID: 2 Domain ID Offset: 96 5-5
Configuration Chapter 5 MDS 9000 Switch and McData Dual Core Topology (Interop Mode 1) Switch Priority: Never Principal Speed: 2 Gb/sec Rerouting Delay: Interop Mode: Open Fabric 1.0 <=== now in interop mode Active Domain ID: 2 World Wide Name: 10:00:08:00:88:A0:7A:74 Insistent Domain ID: Domain RSCN: Zoning RSCN: FC Address Domain: 62 (hex) Limited Fabric RSCN: Zone FlexPars: Fabric Filtering RSCN Zone Isolation: Fabric Filtering Safe Zoning: ISL FSPF Cost: By Port Speed Web : Frame Resize State: Block Dup WWN State: API : API Management IP: 172.25.242.164 Password Exp. Time: 0 Step 5 Set the preferred domain ID, with an offset of 0x60. In this example, the McData switch is domain ID 0x61 (decimal 97). The FC IDs for nodes on this switch are 61xxxx. Root> config switch Config.Switch> prefdo Config.Switch> prefdomainid 1 Config.Switch> root Root> maint system Maint.System> setonlinestate 1 Maint.System> root Root> show switch State: Online <==== now online R_A_TOV: 100 E_D_TOV: 20 Preferred Domain ID: 1 <==== Domain ID 97 Domain ID Offset: 96 Switch Priority: Never Principal Speed: 2 Gb/sec Rerouting Delay: Interop Mode: Open Fabric 1.0 <==== in interop mode Active Domain ID: 1 World Wide Name: 10:00:08:00:88:A0:7A:74 Insistent Domain ID: Domain RSCN: Zoning RSCN: FC Address Domain: 61 (hex) Limited Fabric RSCN: Zone FlexPars: Fabric Filtering RSCN Zone Isolation: Fabric Filtering Safe Zoning: ISL FSPF Cost: By Port Speed Web : Frame Resize State: Block Dup WWN State: API : API Management IP: 172.25.242.164 Password Exp. Time: 0 5-6
Chapter 5 MDS 9000 Switch and McData Dual Core Topology (Interop Mode 1) Verification Configuring the MDS 9000 Switch See Chapter 3, MDS 9000 Core with Brocade Edge Topology (Interop Mode 1), for examples of how to set the MDS 9000 switch to interop mode. Verification This section highlights the commands used to verify that the fabric is up and running in interoperability mode. It includes the following topics: Verifying the McData 6064 Switch, page 5-7 Verifying the MDS 9513 Switch, page 5-9 Verifying the McData 6064 Switch The following examples show the commands used to verify the configuration of the McData 6064 switch: Root> show switch State: Online R_A_TOV: 100 E_D_TOV: 20 Preferred Domain ID: 1 Domain ID Offset: 96 Switch Priority: Never Principal Speed: 2 Gb/sec Rerouting Delay: Interop Mode: Open Fabric 1.0 Active Domain ID: 1 World Wide Name: 10:00:08:00:88:A0:7A:74 Insistent Domain ID: Domain RSCN: Zoning RSCN: FC Address Domain: 61 (hex) Limited Fabric RSCN: Zone FlexPars: Fabric Filtering RSCN Zone Isolation: Fabric Filtering Safe Zoning: ISL FSPF Cost: By Port Speed Web : Frame Resize State: Block Dup WWN State: API : API Management IP: 172.25.242.164 Password Exp. Time: 0 Root> show system Name: M6064 Description: McData-6064 Contact: Field Support Location: LabD-Interop Date/Time: 08/23/2011 18:53:21 Serial Number: Type Number: 006064 Model Name: ED-64M Model Number: 001 5-7
Verification Chapter 5 MDS 9000 Switch and McData Dual Core Topology (Interop Mode 1) EC Level: 1050211 Firmware Version: 09.09.09 2 Beaconing: Issue the show port status command to verify that the correct ports are online and are type E port or F port. Root> show port status Port State Type Attached WWN Beaconing Reason ---- ------------------ ------ ----------------------- --------- ------ 0 No Light gport None 1 No Light gport None 2 No Light gport None 3 No Light gport None 4 Online eport 20:01:00:05:9B:01:B4:B3 5 No Light gport None 6 No Light gport None 7 No Light gport None 8 No Light gport None 24 No Light gport None 25 No Light gport None 26 No Light gport None 27 Online fport 10:00:00:00:C9:D0:92:E6 28 No Light gport None 29 No Light gport None 30 No Light gport None 31 No Light gport None 32 Online eport 20:01:00:05:9B:01:B4:B3 33 No Light gport None 60 No Light gport None 61 No Light gport None 62 No Light gport None 63 No Light gport None The show nameserver command shows that the Netfinity host has logged into the McData switch: Root> show nameserver DID Type PortId Port Name Node Name COS FC4 --- ---- ------ ----------------------- ----------------------- --- --- 1 N 611F13 10:00:00:00:C9:D0:92:E6 20:00:00:00:C9:D0:92:E6 2-3 Unknown FC4 types decode information can be viewed through the Show nameserverfc4types CLI command: Root> show nameserverfc4types 0. ISO/IEC 8802-2 LLC 1. ISO/IEC 8802-2 LLC/SNAP 2. SCSI-FCP 3. SCSI-GPP 4. IPI-3 Master 5. IPI-3 Slave 6. IPI-3 Peer 7. CP IPI-3 Master 8. CP IPI-3 Slave 9. CP IPI-3 Peer 10. SBCCS-Channel 11. SBCCS-Cntrl Unit 12. FC Srvcs 13. FC-FG 14. FC-XS 15. FC-AL 16. SNMP 17. HIPPI-FP 18. Vendor Unique 5-8
Chapter 5 MDS 9000 Switch and McData Dual Core Topology (Interop Mode 1) Verification Root> show loginserver Port BB Crdt RxFldSz COS Port Name Node Name ---- ------- ------- --- ----------------------- ----------------------- 27 40 2048 2-3 10:00:00:00:C9:D0:92:E6 20:00:00:00:C9:D0:92:E6 To view the FSPF routes used by the McData 3032 switch, use the Web GUI tool (Figure 5-2). There is no command line entry available to show this information. Figure 5-2 McData Core Displaying Routes Verifying the MDS 9513 Switch The following examples show the commands used to verify the configuration of the MDS 9513 switch: MDS9513(config)# show version Cisco Nexus Operating System (NX-OS) Software TAC support: http://www.cisco.com/tac Documents: http://www.cisco.com/en/us/products/ps9372/tsd_products_support_series_home.html Copyright (c) 2002-2011, Cisco Systems, Inc. All rights reserved. The copyrights to certain works contained herein are owned by other third parties and are used and distributed under license. Some parts of this software are covered under the GNU Public License. A copy of the license is available at http://www.gnu.org/licenses/gpl.html. Software BIOS: version 1.0.10 loader: version N/A kickstart: version 5.2(1) system: version 5.2(1) BIOS compile time: 01/08/09 kickstart image file is: bootflash:///m9500-sf2ek9-kickstart-mz.5.2.1.bin.s74 kickstart compile time: 12/25/2020 12:00:00 [07/16/2011 22:48:52] system image file is: bootflash:///m9500-sf2ek9-mz.5.2.1.bin.s74 system compile time: 6/7/2011 13:00:00 [07/17/2011 00:15:24] Hardware cisco MDS 9513 (13 Slot) Chassis ("Supervisor/Fabric-2a") Motorola, 7447A, altivec with 2071296 kb of memory. Processor Board ID JAE141501UD Device name: MDS9513 bootflash: 1000440 kb 5-9
Verification Chapter 5 MDS 9000 Switch and McData Dual Core Topology (Interop Mode 1) slot0: 0 kb (expansion flash) Kernel uptime is 20 day(s), 2 hour(s), 51 minute(s), 44 second(s) Last reset Reason: Unknown System version: 5.2(1) Service: plugin Core Plugin MDS9513(config)# show interface brief ------------------------------------------------------------------------------- Interface Vsan Admin Admin Status SFP Oper Oper Port Mode Trunk Mode Speed Channel Mode (Gbps) ------------------------------------------------------------------------------- fc1/1 1 auto on down swl -- -- fc1/2 1 FX on down swl -- -- fc1/3 1 FX on down swl -- -- fc1/28 1 FX on sfpabsent -- -- -- fc1/29 1 FX on sfpabsent -- -- -- fc1/30 1 FX on sfpabsent -- -- -- fc1/31 1 FX on up swl FL 2 -- fc1/32 1 FX on sfpabsent -- -- -- fc1/33 1 FX on sfpabsent -- -- -- fc1/34 1 FX on sfpabsent -- -- -- fc1/35 1 FX on sfpabsent -- -- -- fc1/36 1 FX on sfpabsent -- -- -- fc1/37 1 auto on up swl E 2 -- fc1/38 1 FX on sfpabsent -- -- -- fc1/39 1 FX on sfpabsent -- -- -- fc1/40 1 FX on sfpabsent -- -- --... fc13/1 1 auto on up swl E 1 -- fc13/2 1 FX on sfpabsent -- -- -- fc13/3 1 FX on sfpabsent -- -- -- fc13/4 1 FX on sfpabsent -- -- -- MDS9513(config)# show vsan 1 vsan 1 information name:vsan0001 state:active interoperability mode:1 loadbalancing:src-id/dst-id/oxid operational state:up MDS9513(config)# show fcdomain vsan 1 The local switch is the Principal Switch. Local switch run time information: State: Stable Local switch WWN: 20:01:00:05:9b:01:b4:b3 Running fabric name: 20:01:00:05:9b:01:b4:b3 Running priority: 2 Current domain ID: 0x64(100) Local switch configuration information: State: FCID persistence: Auto-reconfiguration: Contiguous-allocation: 5-10
Chapter 5 MDS 9000 Switch and McData Dual Core Topology (Interop Mode 1) Verification Configured fabric name: 20:01:00:05:30:00:28:df Optimize Mode: Configured priority: 128 Configured domain ID: 0x64(100) (preferred) Principal switch run time information: Running priority: 2 Interface Role RCF-reject ---------------- ------------- ------------ fc1/37 Downstream fc13/1 Non-principal ---------------- ------------- ------------ MDS9513(config)# show fcdomain domain-list vsan 1 Number of domains: 2 Domain ID WWN --------- ----------------------- 0x64(100) 20:01:00:05:9b:01:b4:b3 [Local] [Principal] 0x61(97) 10:00:08:00:88:a0:7a:74 @@@@ MDS9513(config-if)# sh fspf internal route vsan 1 FSPF Unicast Routes --------------------------- VSAN Number Dest Domain Route Cost Next hops ----------------------------------------------- 1 0x61(97) 500 fc1/37 fc13/1 MDS9513(config)# show fcns database vsan 1 VSAN 1: -------------------------------------------------------------------------- FCID TYPE PWWN (VENDOR) FC4-TYPE:FEATURE -------------------------------------------------------------------------- 0x611f13 N 10:00:00:00:c9:d0:92:e6 (Emulex) scsi-fcp:init 0x6401d4 NL 50:00:00:e0:10:32:d1:61 (Fujitsu) scsi-fcp:target 0x6401d5 NL 50:00:00:e0:10:32:d0:f1 (Fujitsu) scsi-fcp:target 0x6401d6 NL 50:00:00:e0:10:32:da:a1 (Fujitsu) scsi-fcp:target 0x6401d9 NL 50:00:00:e0:10:33:04:01 (Fujitsu) scsi-fcp:target 0x6401da NL 50:00:00:e0:10:33:fc:d1 (Fujitsu) scsi-fcp:target 0x6401dc NL 50:00:00:e0:10:5c:3c:21 (Fujitsu) scsi-fcp:target 0x6401e0 NL 50:00:00:e0:10:4b:b3:61 (Fujitsu) scsi-fcp:target 0x6401e1 NL 50:00:00:e0:10:33:06:51 (Fujitsu) scsi-fcp:target 0x6401e2 NL 50:00:00:e0:10:34:a4:c1 (Fujitsu) scsi-fcp:target 0x6401e4 NL 50:00:00:e0:10:32:d3:61 (Fujitsu) scsi-fcp:target 0x6401e8 NL 50:00:00:e0:10:33:01:d1 (Fujitsu) scsi-fcp:target 0x6401ef NL 50:00:00:e0:10:32:d8:81 (Fujitsu) scsi-fcp:target Total number of entries = 13 5-11
Verification Chapter 5 MDS 9000 Switch and McData Dual Core Topology (Interop Mode 1) switch# show fcns database vsan 99 VSAN 99: -------------------------------------------------------------------------- FCID TYPE PWWN (VENDOR) FC4-TYPE:FEATURE -------------------------------------------------------------------------- 0x610713 N 21:00:00:e0:8b:07:a4:36 (QLogic) 0x7800d5 NL 22:00:00:20:37:0e:65:da (Seagate) scsi-fcp:target 0x7800d6 NL 22:00:00:20:37:14:74:e8 (Seagate) scsi-fcp:target 0x7800d9 NL 22:00:00:20:37:04:f8:a1 (Seagate) scsi-fcp:target 0x7800da NL 22:00:00:20:37:0c:6f:a9 (Seagate) scsi-fcp:target 0x7800dc NL 22:00:00:20:37:0d:b6:4b (Seagate) scsi-fcp:target 0x7800e0 NL 22:00:00:20:37:0e:65:44 (Seagate) scsi-fcp:target 0x7800e1 NL 22:00:00:20:37:0c:75:d5 (Seagate) scsi-fcp:target 0x7800e2 NL 22:00:00:20:37:04:ea:2b (Seagate) scsi-fcp:target 0x7800e4 NL 22:00:00:20:37:14:74:49 (Seagate) scsi-fcp:target 0x7800e8 NL 22:00:00:20:37:14:74:fc (Seagate) scsi-fcp:target Total number of entries = 11 You can use Cisco DCNM-SAN and Device Manager to verify the current state of the fabric. Figure 5-3 shows the fabric topology from Cisco DCNM-SAN. Figure 5-3 MDS 9000 Switch and McData Dual Core Displayed in Cisco DCNM-SAN Figure 5-4 shows Device Manager on the MDS 9513 switch in the example topology. It shows which ports are active as E ports. 5-12
Chapter 5 MDS 9000 Switch and McData Dual Core Topology (Interop Mode 1) Verification Figure 5-4 MDS 9513 Switch Device Manager Showing Active Ports Figure 5-5 shows the status of the E ports that verifies the link speed. Figure 5-5 Cisco DCNM-SAN Verifying Port Status and Type 5-13
Verification Chapter 5 MDS 9000 Switch and McData Dual Core Topology (Interop Mode 1) Figure 5-6 shows the domain ID of the switches in the fabric. Figure 5-6 Cisco DCNM-SAN Verifying Domains Figure 5-7 shows that VSAN 1 is operating in interop mode. Figure 5-7 Verifying Interop Mode is Set Figure 5-8 shows the MDS 9513 switch name server. This shows the edge devices that have registered. 5-14
Chapter 5 MDS 9000 Switch and McData Dual Core Topology (Interop Mode 1) Zoning Figure 5-8 MDS 9513 Switch Name Server Entries Figure 5-9 shows the FSPF routes as calculated by this switch. Figure 5-9 FSPF Calculations in Cisco DCNM-SAN Zoning In this example topology, Cisco DCNM-SAN was used to create zones and zone sets. Cisco DCNM-SAN displays are also used to verify the zone and zone set information. Figure 5-10 shows the zones and zone sets displayed in Cisco DCNM-SAN. 5-15
Zoning Chapter 5 MDS 9000 Switch and McData Dual Core Topology (Interop Mode 1) Figure 5-10 Zoning Configuration for MDS 9000 Switch and McData Dual Core Topology Figure 5-11 shows the zone VZ1. Figure 5-11 Zone VZ1 Displayed in Cisco DCNM-SAN Figure 5-12 shows the FSPF traffic flow in this multivendor fabric. 5-16
Chapter 5 MDS 9000 Switch and McData Dual Core Topology (Interop Mode 1) Zoning Figure 5-12 MDS 9000 Switch and McData Dual Core FSPF Data Flow Target JBOD VZ1 Host 1 330694 1/31 1/37 13/1 4 28 27 Cisco MDS 9513 McData 6064 2 Gigabit ISLs 5-17
Zoning Chapter 5 MDS 9000 Switch and McData Dual Core Topology (Interop Mode 1) 5-18