IM System Storage N series Universal SS and P abling Guide G27-3919-02
ontents Preface................................ v Supported features............................. v Websites................................ v Getting information, help, and service...................... vi efore you call.............................. vi Using the documentation.......................... vi Hardware service and support......................... vi Firmware updates............................. vii How to send your comments......................... vii SS cabling for new storage systems...................... 1 SS cabling rules............................. 1 General SS cabling rules......................... 1 Onboard SS port rules.......................... 2 Shelf-to- connection rules........................ 3 ontroller-to--connection rules...................... 4 ontroller-to-controller rule (N3150, N3220, and N3240 system models only)........ 9 abling SS connections.......................... 10 ompleting the SS cabling worksheet..................... 10 SS cabling worksheet rules and template.................... 11 Sample SS cabling worksheet for a N6200 series................. 12 Sample SS cabling worksheet for an H pair (non N6200 series, N3150, N3220, and N3240, or N3400 )............................... 14 abling SS ports............................ 16 SS cabling examples........................... 19 Examples of SS cabling for H pairs with quad-port Hs............. 19 Examples of SS cabling for single-controller configurations with quad-port Hs...... 25 Examples of SS cabling for N6200 series systems................. 28 Examples of SS cabling for N3150, N3220, and N3240 system models.......... 31 Examples of SS cabling for N3600 systems................... 34 Examples of SS cabling for N3400 systems................... 39 P cabling for new storage systems..................... 43 P cabling rules............................. 43 General P cabling rules......................... 43 P connectivity rules.......................... 43 abling P connections.......................... 44 abling P on systems with external SS storage................ 44 abling P on systems with no external SS storage............... 49 P cabling examples........................... 50 Examples of P cabling for H pairs..................... 50 Examples of P cabling for single-controller configurations............. 53 P cabling examples for all single-controller configurations except N3400 systems..... 53 P cabling examples for single-controller N3400 systems............. 55 Examples of P cabling for N3150, N3220, and N3240 system models with no external storage 57 opyright IM orp. 2012 iii
Glossary............................... 59 ocument update record.......................... 63 opyright and trademark information..................... 65 Trademark information........................... 66 Notices................................ 69 iv IM System Storage N series: Universal SS and P abling Guide
Preface Supported features IM System Storage N series storage systems are driven by Netpp ata ONTP software. Some features described in the product software documentation are neither offered nor supported by IM. Please contact your local IM representative or reseller for further details. Information about supported features can also be found on the N series support website (accessed and navigated as described in Websites). Websites IM maintains pages on the World Wide Web where you can get the latest technical information and download device drivers and updates. The following web pages provide N series information: v listing of currently available N series products and features can be found at the following web page: www.ibm.com/storage/nas/ v The IM System Storage N series support website requires users to register in order to obtain access to N series support content on the web. To understand how the N series support web content is organized and navigated, and to access the N series support website, refer to the following publicly accessible web page: www.ibm.com/storage/support/nseries/ This web page also provides links to utosupport information as well as other important N series product resources. v IM System Storage N series products attach to a variety of servers and operating systems. To determine the latest supported attachments, go to the IM N series interoperability matrix at the following web page: www.ibm.com/systems/storage/network/interophome.html v For the latest N series hardware product documentation, including planning, installation and setup, and hardware monitoring, service and diagnostics, see the IM N series Information enter at the following web page: publib.boulder.ibm.com/infocenter/nasinfo/nseries/index.jsp opyright IM orp. 2012 v
Getting information, help, and service If you need help, service, or technical assistance or just want more information about IM products, you will find a wide variety of sources available from IM to assist you. This section contains information about where to go for additional information about IM and IM products, what to do if you experience a problem with your IM N series product, and whom to call for service, if it is necessary. efore you call efore you call, make sure you have taken these steps to try to solve the problem yourself: v heck all cables to make sure they are connected. v heck the power switches to make sure the system is turned on. v Use the troubleshooting information in your system documentation and use the diagnostic tools that come with your system. v Refer to the N series support website (accessed and navigated as described in Websites) for information on known problems and limitations. Using the documentation The latest versions of N series software documentation, including ata ONTP and other software products, are available on the N series support website (accessed and navigated as described in Websites). urrent N series hardware product documentation is shipped with your hardware product in printed documents or as PF files on a documentation. For the latest N series hardware product documentation PFs, go to the N series support website. Hardware documentation, including planning, installation and setup, and hardware monitoring, service, and diagnostics, is also provided in an IM N series Information enter at the following web page: publib.boulder.ibm.com/infocenter/nasinfo/nseries/index.jsp Hardware service and support You can receive hardware service through IM Integrated Technology Services. Visit the following web page for support telephone numbers: www.ibm.com/planetwide/ vi IM System Storage N series: Universal SS and P abling Guide
Firmware updates IM N series product firmware is embedded in ata ONTP. s with all devices, ensure that you run the latest level of firmware. ny firmware updates are posted to the N series support website (accessed and navigated as described in Websites). Note: If you do not see new firmware updates on the N series support website, you are running the latest level of firmware. Verify that the latest level of firmware is installed on your machine before contacting IM for technical support. How to send your comments Your feedback helps us to provide the most accurate and high-quality information. If you have comments or suggestions for improving this document, please send them by email to starpubs@us.ibm.com. e sure to include the following: v Exact publication title v Publication form number (for example, G26-1234-02) v Page, table, or illustration numbers v detailed description of any information that should be changed Preface vii
viii IM System Storage N series: Universal SS and P abling Guide
SS cabling for new storage systems For new storage system installations, you must understand the cabling rules before cabling the SS connections for the disk shelves. Note: abling information for hot-adding or replacing SS disk shelves is in the Installation and Service Guide for your disk model on the N series support website (accessed and navigated as described in Websites) at www.ibm.com/storage/support/nseries/. SS cabling rules You can cable SS disk shelves to storage systems that have SS Hs and/or onboard SS ports by applying the SS cabling rules. General SS cabling rules You need to know the general SS cabling rules governing supported configurations, cables, and SS port connections on the disk shelves. v ll H pair configurations must use multipath H, with the following exceptions: N3400 systems use single-path H. N3150, N3220, and N3240 system models with no external SS storage can use single-path H, which requires no external cabling. However, the recommended configuration is to enable multipath H for the internal disks by connecting the SS ports of one controller module to the SS ports of the partner. Note: When a filer is ordered without external shelves, the two QSFP SS cables and an P cable, required to enable multipath H configuration, can be ordered as F 9563. However, if the filer is ordered with external shelves, the cables will come with external shelves instead. Note: multipath H configuration having external shelves requires two ports on each controller for each stack of disk shelves. Even if the stack consists of one disk, two ports on each controller are required. v ll single-controller configurations must use dual-path, with the following exceptions: N3400 systems use single-path. N3150, N3220, and N3240 system models have the internal disks cabled as single-path, but the system can use either single-path or dual-path connectivity to the external shelves; both are supported. opyright IM orp. 2012 1
ual-path provides greater resiliency, but when dual-path is used, ata ONTP 8.1.x issues occasional warnings to the console because mixed paths are detected. To avoid the warnings, single-path connections to the external SS storage are a supported option in these configurations. v ables used for SS connections include the following: QSFP to QSFP SS cables are used to connect disk shelves together and to connect disk shelves to the SS ports on the controller. The following illustration shows the QSFP cable connector: QSFP to mini-ss cables are used for N3600 configurations only, and connect disk shelves to the dual-port mini-ss Hs on the controller. The following illustration shows the QSFP to mini-ss cable connectors: v SS ports on the disk are designated by a square symbol or a circle symbol. Square ports are always cabled to circle ports, and circle ports are always cabled to square ports. Never cable square ports to square ports or circle ports to circle ports. The following illustration shows square and circle port labeling on an IOM. IOM P SS Onboard SS port rules Storage systems that have controllers with onboard SS ports must follow a specific port identification rule. For all systems except N3400, an onboard H is defined as occupying PI slot 0, with each port named with sequentially increasing letters (0a, 0b, 0c, and so on). This is to ensure that the SS cabling rules apply to all SS Hs on a controller, whether in a physical PI slot (slot 1-N) or on board the base controller. 2 IM System Storage N series: Universal SS and P abling Guide
The following example shows a controller with one onboard SS H in slot 0 and three SS Hs in physical PI slots 1, 2, and 3. Slot 0 Slot 2 Slot 3 For N3400 systems, the onboard SS port labeled 0d should be treated as if it were slot 0 port a. Treating the port as 0a enables you to apply all the SS cabling rules. (ata ONTP continues to recognize this port as 0d.) The following example shows the N3400 exception: 2040 controller 0a 0d Shelf-to- connection rules When you have more than one disk in a stack of disk shelves, the shelves in the stack must be connected to each other (daisy-chained). The disk shelves are daisy-chained in the following way: v IOM circle port is connected to the next IOM square port. v IOM circle port is connected to the next IOM square port. Note: Square ports are always cabled to circle ports, and circle ports are always cabled to square ports. Never cable square ports to square ports or circle ports to circle ports. The following illustration shows a daisy-chained stack of disk shelves with the IOMs arranged side-by-side, similar to how a EXN3500 disk is oriented. SS cabling for new storage systems 3
Stack of three shelves P SS P SS IOM IOM The following illustration shows a daisy-chained stack of disk shelves with the IOMs arranged one above the other, similar to how a EXN3000 disk is oriented. Stack of three shelves P SS IOM IOM ontroller-to--connection rules Each controller connects to each stack of disk shelves in the system through the controller SS ports. These ports can be,,, and, and can be on a SS H in a physical PI slot (slot 1-N) or on board the base controller. ontrollers connect to the first and last shelves in a stack. ontroller ports and connect to the first in a stack and controller ports and connect to the last in a stack. For quad-port SS Hs, the controller-to- connection rules ensure resiliency for the storage system based on the SI chip design. Ports and are on one SI chip, and ports and are on a second SI chip. y using ports and to connect to the first in each stack, and using ports 4 IM System Storage N series: Universal SS and P abling Guide
and to connect to the last in each stack, the controllers maintain connectivity to the disk shelves if an SI chip fails. The following illustration shows a quad-port SS H with the two SI chips and their designated ports: Quad-port SS H PI-e chip SI chip SI chip port port port port and port connections v ontroller SS ports and are connected to the first disk square port in each stack of disk shelves. The and ports on controller 1/ connect to IOM. The and ports on controller 2/ connect to IOM. For example, the following illustration shows how controller ports and are connected for disk shelves that have the IOMs arranged side-by-side, similar to how a EXN3500 disk is oriented: ontroller 1/ ontroller 2/ Stack 1 Stack 2 P SS P SS P SS P SS IOM IOM IOM IOM For example, the following illustration shows how controller ports and are connected for disk shelves that have the IOMs arranged one above the other, similar to how a EXN3000 disk is oriented: SS cabling for new storage systems 5
ontroller 1/ ontroller 2/ Stack 1 P SS Stack 2 P SS IOM IOM v ontroller SS ports and are connected to the disk stacks in sequential order of the PI slots (slot 0, slot 1, slot 2, and so on) on the controller. For example, a controller with slots 1, 2, and 3 occupied with quad-port SS Hs is cabled in order of slots and ports 1a, 2a, 3a, 1c, 2c, 3c: ontroller Slot 2 Slot 3 For example, a controller with slots 0, 1, and 2 occupied (slot 0 being an onboard port and slot 1 and 2 being quad-port SS Hs) is cabled in order of slots and ports 0a, 1a, 2a, 1c, 2c: ontroller Slot 0 Slot 2 6 IM System Storage N series: Universal SS and P abling Guide
and port connections v ontroller SS ports and are connected to the last disk circle port in each stack of disk shelves. The and ports on controller 1/ connect to IOM. The and ports on controller 2/ connect to IOM. For example, the following illustration shows how controller ports and are connected for disk shelves that have the IOMs arranged side-by-side, similar to how a EXN3500 disk is oriented: ontroller 1/ ontroller 2/ Stack 1 Stack 2 P SS P SS P SS P SS IOM IOM IOM IOM For example, the following illustration shows how controller ports and are connected for disk shelves that have the IOMs arranged one above the other, similar to how a EXN3000 disk is oriented: SS cabling for new storage systems 7
ontroller 1/ ontroller 2/ Stack 1 P SS IOM IOM Stack 2 P SS v ontroller SS ports and are connected to the disk stacks by offsetting the order of the PI slots by one so that the first port on the first slot is cabled last. The cabling offset ensures system resiliency. For example, a controller with three quad-port SS Hs in slots 1, 2, and 3 is cabled in order of slots and ports 2b, 3b, 1d, 2d, 3d, 1b: ontroller Slot 2 Slot 3 For example, a controller with slots 0, 1, and 2 occupied (slot 0 being an onboard port and slot 1 and 2 being quad-port SS Hs) is cabled in order of slots and ports 1b, 2b, 1d, 2d, 0b: 8 IM System Storage N series: Universal SS and P abling Guide
ontroller Slot 0 Slot 2 controller with one quad-port H is cabled in order of port 1d and then 1b: ontroller 1/ Stack cabling sequence ontrollers are cabled to the stacks in sequence of the stacks, such as stack 1, stack 2, stack 3, and so on. N3400 configurations For single-controller N3400 configurations, the controller is in Slot (the bottom slot); therefore, the controller is cabled as if it were controller 2/. If a second controller is added to make this an H pair configuration, the new controller is cabled as controller 1/. ontroller-to-controller rule (N3150, N3220, and N3240 system models only) On N3150, N3220, and N3240 system models in H pairs with no external storage, external cables should connect the SS ports on the two controller modules to enable multipath H for the internal disks. Note: When a filer is ordered without external shelves, the two QSFP SS cables and an P cable, required to enable multipath H configuration, can be ordered as F 9563. However, if the filer is ordered with external shelves, the cables will come with external shelves instead. SS cabling for new storage systems 9
abling SS connections abling the SS connections involves completing the SS cabling worksheet to identify the SS ports that need to be cabled to the disk shelves, and applying the worksheet when following the cabling procedure. ompleting the SS cabling worksheet y using the SS cabling worksheet to list the slots and SS ports for each controller, you can simplify the cabling task. bout this task You can reference the appropriate Sample SS cabling worksheet when recording your information. Procedure 1. Print out a copy of the SS cabling worksheet template to record your information. 2. Read the cabling worksheet rules and be sure to understand them. 3. Make a list of all SS ports and then all SS ports on each controller in your configuration. controller with three quad-port SS Hs is listed as 1a, 2a, 3a, 1c, 2c, 3c. controller with one quad-port SS H is listed as 1a, 1c. N3600 with one dual-port SS H is listed as 1a. N3400 is listed as 0a. N6200 series dual enclosure (with two onboard SS ports and one quad-port) is listed as 0a, 1a, 1c. 4. If you have a N3400 go to Step 5; otherwise, list the remaining ports by completing the following substeps: a. Make a list of all SS ports and then all SS ports on each controller in your configuration (in the grayed-out cells of the worksheet). controller with three quad-port SS Hs is listed as 1b, 2b, 3b, 1d, 2d, 3d. controller with one quad-port SS H is listed as 1b, 1d. N3600 with one dual-port SS H is listed as 1b. N6200 series dual enclosure (with two onboard SS ports and one quad-port) is listed as 0b, 1b, 1d. b. Rewrite the list (on the line below the grayed-out cells) so that the first 1b, 2b, 3b, 1d, 2d, 3d, 1b port in your list is moved to the end of your list: Offsetting the list by one (so that the cabling begins on the second X 10 IM System Storage N series: Universal SS and P abling Guide
slot/h) ensures system resiliency when cabling your storage system. controller with three quad-port SS Hs is listed as 2b, 3b, 1d, 2d, 3d, 1b. controller with one quad-port SS H is listed as 1d, 1b. N3600 with one dual-port SS H is listed as 1b. N6200 series dual enclosure (with two onboard SS ports and one quad-port) is listed as 1b, 1d, 0b. c. ross out the information in the grayed-out cells. You don't need this information anymore. You will use the offset list below the grayed-out cells to cable your system. 5. Go to abling SS ports to cable your storage system. SS cabling worksheet rules and template y understanding the rules for filling out the worksheet template, you can record your controller SS slot and port information correctly. Use the following rules when recording your information in the SS cabling worksheet. Samples of a completed worksheets are shown in the Sample cabling worksheet sections. v Ports are listed in sequence of the slot numbers to which they belong and include the slot number. For example, onboard SS ports belong to slot 0. SS port of slot 0 is written as 0a. single quad-port SS H in a physical PI slot belongs to slot 1. dditional quad-port SS Hs belong to slot 2, slot 3, and so on. SS port of slot 1 is written as 1a. SS port of slot 2 is written as 2a, and so on. v If your controller has onboard ports, they are always listed first (since they belong to slot 0) and then followed by the quad-port SS H slots (slot 1, 2, and so on). v If you have an H pair, the port list is the same for controller 1/ and controller 2/. v Only one controller slot and SS port is listed in a table cell under a stack number. v ontroller slots and SS ports are listed in sequence of the stack numbers. Note: This worksheet provides space for eight stacks. o not skip any table cells. v Only fill in controller slots and SS ports for the number of stacks you have in your configuration. For example, if you have only four stacks of disk shelves in your configuration, the columns for stack numbers 5 through 8 remain blank. SS cabling for new storage systems 11
v The controller SS ports and rows show two lines for controller 1/ and controller 2/. Use the first line (grayed-out) for recording the port list prior to offsetting the list by one. Use the second line for recording the list with the ports offset by one. v The worksheet has you connect square ports first, but you can cable circle ports first. However, square ports must only be connected to circle ports and circle ports must only be connected to square ports. v If your configuration has a single stack, the references to the first and last apply to the one. v If your configuration has a N3400 with a single controller, list the slot and port information on the ontroller 2/ line. Single controller N3400 systems have the controller in the bottom (2/) slot of the chassis. ontroller SS ports and ontroller ontroller 1/ ontroller 2/ ontroller 1/ ontroller SS ports and stack cabling sequence Stack number (1-8) 1 2 3 4 5 6 7 8 able to disk IOM To first IOM To first IOM To last IOM and ontroller 2/ To last IOM Sample SS cabling worksheet for a N6200 series You can reference this sample SS cabling worksheet when filling in your own worksheet. The sample worksheet shows a N6200 series dual enclosure configuration (two onboard SS ports and one quad-port SS H) with two stacks of disk shelves. Note: The slots and SS ports shown in gray text for stack 3 are unused because of the limited number of stacks in this configuration. If there were three stacks, all three slots and SS ports listed would be used. Note: The two rows which are grayed-out and crossed-out are the original port lists that were made prior to offsetting the lists by one. The lines below that show the ports offset by one and would be used to cable the system. 12 IM System Storage N series: Universal SS and P abling Guide
ontroller SS ports and ontroller ontroller 1/ ontroller 2/ ontroller SS ports and stack cabling sequence Stack number (1-8) 1 2 3 4 5 6 7 8 0a 1a 1c 0a 1a 1c able to disk IOM To first IOM To first IOM and ontroller 1/ ontroller 2/ 0b 1b 0b 1b 1b 1d 1b 1d 1d 0b 1d 0b To last IOM To last IOM The following illustrates a cabled N6200 series using the information in the sample worksheet. Two examples are shown. One example shows disk shelves with the IOMs arranged side by side, similar to how a EXN3500 disk is oriented. The second example shows disk shelves with the IOMs arranged one above the other, similar to how a EXN3000 disk is oriented. Multipath H configuration ontroller 1/ ontroller 2/ 1. Slot 0 5. 2. 6. 3. Slot 0 4. 7. 8. Stack 1 Stack 2 P SS P SS P SS P SS IOM IOM IOM IOM SS cabling for new storage systems 13
1. ontroller 1/ ontroller 2/ Slot 0 Multipath H onfiguration Slot 0 2. 3. 4. 5. Stack 1 6. Stack 2 7. 8. P SS P SS IOM IOM Sample SS cabling worksheet for an H pair (non N6200 series, N3150, N3220, and N3240, or N3400 ) You can reference this sample SS cabling worksheet and illustration when filling in your own worksheet. The sample worksheet shows an H pair configuration with three quad-port SS Hs and three stacks of disk shelves. Note: The slots and SS ports shown in gray text for stacks 4, 5, and 6 are unused because of the limited number of stacks in this configuration. If there were six stacks, all six slots and SS ports listed would be used. Note: The two rows which are grayed-out and crossed-out are the original port lists made prior to offsetting the lists by one. The lines below that show the ports offset by one and would be used to cable the system. 14 IM System Storage N series: Universal SS and P abling Guide
ontroller SS ports and and ontroller ontroller 1/ ontroller 2/ ontroller 1/ ontroller 2/ ontroller SS ports and stack cabling sequence Stack number (1-8) 1 2 3 4 5 6 7 8 1a 2a 3a 1c 2c 3c 1a 2a 3a 1c 2c 3c 1b 2b 3b 1d 2d 3d 2b 3b 1d 2d 3d 1b 1b 2b 3b 1d 2d 3d 2b 3b 1d 2d 3d 1b able to disk IOM To first IOM To first IOM To last IOM To last IOM The following illustrates a cabled H pair (non N6200 series, N3150, N3220, and N3240, or N3400 ) using the information in the sample worksheet. Two examples are shown. One example shows disk shelves with the IOMs arranged side by side, similar to how a EXN3500 disk is oriented. The second example shows disk shelves with the IOMs arranged one above the other, similar to how a EXN3000 disk is oriented. Mutipath H configuration ontroller 1/ ontroller 2/ 1. Slot 2 Slot 3 Slot 2 Slot 3 6. 4. 9. 8. 11. 12. 7. 2. 3. Stack 1 Stack 2 5. 10. Stack 3 P SS P SS P SS P SS P SS P SS IOM IOM IOM IOM IOM IOM SS cabling for new storage systems 15
Multipath H configuration ontroller 1/ ontroller 2/ 1. Slot 2 Slot 3 4. Slot 2 Slot 3 6. 9. 7. 8. 3. 11. Stack 1 2. Stack 2 12. Stack 3 IOM IOM P SS P SS 5. P SS 10. abling SS ports abling the SS ports involves cabling the -to- connections and the -to-controller connections from the first and last in a stack, and verifying that the SS connections are correct. efore you begin v You should have filled out the SS cabling worksheet. v Your storage system must be running the minimum software release required to support your model of SS disk shelves. The IM System Storage N series Introduction and Planning Guide on the N series support website (accessed and navigated as described in Websites) at www.ibm.com/storage/support/nseries/ contains support information. v If your storage system was not shipped in a system cabinet, the controllers and disk shelves must already be installed in a rack. v If your storage system will be using one or more PIe SS Hs, they must already be installed and available for cabling the disk shelves. v If you are mixing different disk models in the same stack, you should read the Storage Subsystem onfiguration Guide TR-3838 and Storage Subsystem Technical FQ on the N series support website (accessed and navigated as described in Websites). 16 IM System Storage N series: Universal SS and P abling Guide
bout this task The SS cable QSFP connectors are keyed; when oriented correctly into a SS port the QSFP connector clicks into place and the SS port link LE, LNK, illuminates green. o not force a connector into a port. Procedure 1. If you have more than one disk in a stack, cable the -to- SS connections by completing the following substeps; otherwise, go to Step 2: You use the QSFP to QSFP SS cables that came with your system. a. onnect IOM circle port to the next IOM square port. b. onnect IOM circle port to the next IOM square port. The following examples show a stack of three shelves connected -to-. The first example shows shelves that have the IOMs arranged side-by-side, similar to how a EXN3500 disk is oriented. The second example shows shelves that have the IOMs arranged one above the other, similar to how a EXN3000 disk is oriented. Stack of three shelves P SS P SS IOM IOM Stack of three shelves P SS IOM IOM SS cabling for new storage systems 17
2. Use your completed worksheet to cable the SS and SS connections for each controller. Note: If you are cabling a N3600 storage system, use the QSFP to mini-ss cables for all controller to connections; otherwise, use the QSFP to QSFP SS cables. 3. Use your completed worksheet to cable the SS and SS connections for each controller. Note: For quad-port SS Hs, ports and never connect to the same stack of disk shelves and ports and never connect to the same stack of disk shelves. This means that you might have unused H ports, depending on the number of stacks in your configuration. For example, a single-controller with one quad-port SS H and one stack of shelves only uses H ports and. 4. Verify the SS connections by entering the applicable command: v For 7-Mode, enter the following command at the system console: sasadmin expander_map For H pairs, you can run this command on either node. 5. The next step depends on the output. If... Then... The output lists all IOMs The IOMs have connectivity; go to Step 6. ny IOMs are not shown (either the output does not show an IOM because it is cabled incorrectly, or the output does not show all the IOMs downstream from the incorrectly cabled IOM.) Repeat Steps 1 through 5 to correct cabling errors, then go to Step 6. 6. heck whether your system has disk autoassignment enabled by using the options disk command at the console of either controller. If disk autoassignment is enabled, the output shows disk.auto_assign on. 7. If your system does not have disk autoassignment enabled, assign disk drive ownership. The following substeps are for basic disk ownership assignment for 7-Mode. To specify the disk drives to be assigned or the system to own the disk drives, see the ata ONTP Storage Management Guide for 7-Mode. For 7-Mode H pairs, you can run these commands from the system console of either node. a. Enter the following command to see all of the unassigned disk drives: disk show -n The disk drives in the hot-added have the ownership state of Not Owned. 18 IM System Storage N series: Universal SS and P abling Guide
b. Enter the following command to assign ownership of the disk drives: disk assign all If you need to assign the disk drives to a non-local storage system, you can specify a storage system by using the -o owner_name option. To determine the owner_name, you can run the hostname command at the console of either controller. c. Enter the following command to verify the assignments that you made: disk show -v 8. The next step depends on whether you are using the P capability. If you are... Using the P capability Not using the P capability Then... Go to abling P connections. oot the storage system and begin setup. ttention: fter your system is up and serving data, you cannot move SS cables nondisruptively. SS cabling examples You can use the SS cabling examples to see how the SS cabling rules are applied for common configurations. Examples of SS cabling for H pairs with quad-port Hs You can reference these examples for multipath H configurations with quad-port SS Hs. The numbered cables show the sequence in which the system should be cabled. Two examples are shown for each configuration type. One example shows disk shelves with the IOMs arranged side by side, similar to how a EXN3500 disk is oriented. The second example shows disk shelves with the IOMs arranged one above the other, similar to how a EXN3000 disk is oriented. Note: These examples do not apply to N3400, N3150, N3220 and N3240, or N6200 series systems. SS cabling for new storage systems 19
One quad-port SS H and a single stack The SS port cabling sequence for each controller is 1a, 1d. Multipath H configuration ontroller 1/ ontroller 2/ 1. 2. 4. Stack 1 3. P SS P SS IOM IOM Multipath H configuration 1. ontroller 1/ ontroller 2/ 4. 2. P SS 3. IOM IOM 20 IM System Storage N series: Universal SS and P abling Guide
One quad-port SS H and two single stacks The SS port cabling sequence for each controller is 1a, 1c, 1d, 1b. Multipath H configruation ontroller 1/ ontroller 1/ 2. 5. 7. 3. 8. 4. 1. 6. P SS P SS P SS P SS IOM IOM IOM IOM Multipath H configruation ontroller 1/ ontroller 1/ 2. 3. 5. 6. 8. 4. 7. 1. P SS P SS IOM IOM SS cabling for new storage systems 21
Two quad-port SS Hs and two stacks of disk shelves The SS port cabling sequence for each controller is 1a, 2a, 2b, 1d. Multipath H configuration ontroller 1/ ontroller 2/ Slot 2 1. 2. 3. Slot 2 4. 8. 7. Stack 1 Stack 2 P SS P SS P SS P SS 5. IOM IOM IOM IOM 6. 22 IM System Storage N series: Universal SS and P abling Guide
Multipath H onfiguration ontroller 1/ ontroller 2/ Slot 2 2. Slot 2 4. 1. 3. 7. Stack 1 Stack 2 5. P SS P SS 6. IOM IOM 8. Three quad-port SS Hs and three stacks of disk shelves The SS port cabling sequence for each controller is 1a, 2a, 3a, 2b, 3b, 1d. Mutipath H configuration ontroller 1/ ontroller 2/ 1. Slot 2 Slot 3 Slot 2 Slot 3 6. 4. 9. 8. 11. 12. 7. 2. 3. Stack 1 Stack 2 5. 10. Stack 3 P SS P SS P SS P SS P SS P SS IOM IOM IOM IOM IOM IOM SS cabling for new storage systems 23
Multipath H configuration ontroller 1/ ontroller 2/ 1. Slot 2 Slot 3 4. Slot 2 Slot 3 6. 9. 7. 8. 3. 11. Stack 1 2. Stack 2 12. Stack 3 IOM IOM P SS P SS 5. P SS 10. Four quad-port SS Hs and four stacks of disk shelves The SS port cabling sequence for each controller is 1a, 2a, 3a, 4a, 2b, 3b, 4b, 1d. Four quad-port SS Hs and eight stacks of disk shelves The SS port cabling sequence for each controller is 1a, 2a, 3a, 4a, 1c, 2c, 3c, 4c, 2b, 3b, 4b, 1d, 2d, 3d, 4d, 1b. Five quad-port SS Hs and five stacks of disk shelves The SS port cabling sequence for each controller is 1a, 2a, 3a, 4a, 5a, 2b, 3b, 4b, 5b, 1d, 2d, 3d, 4d, 1b. Five quad-port SS Hs and 10 stacks of disk shelves The SS port cabling sequence for each controller is 1a, 2a, 3a, 4a, 5a, 1c, 2c, 3c, 4c, 5c, 2b, 3b, 4b, 5b,1d, 2d, 3d, 4d, 5d,1b. 24 IM System Storage N series: Universal SS and P abling Guide
Examples of SS cabling for single-controller configurations with quad-port Hs You can reference these SS cabling examples for single-controller configurations with quad-port SS Hs (dual-path configurations). The numbered cables show the sequence in which the system should be cabled. Two examples are shown for each configuration type. One example shows disk shelves with the IOMs arranged side by side, similar to how a EXN3500 disk is oriented. The second example shows disk shelves with the IOMs arranged one above the other, similar to how a EXN3000 disk is oriented. Note: These examples do not apply to N3400, N3220 and N3240, or N6200 series systems. One quad-port SS H and one stack of disk shelves The SS port cabling sequence is 1a, 1d. ual-path configuration ontroller 1/ 1. Stack 1 2. P SS P SS IOM IOM SS cabling for new storage systems 25
ual-path configuration 1. IOM IOM ontroller 1/ Stack 1 P SS 2. One quad-port SS H and two stacks of disk shelves The SS port cabling sequence is 1a, 1c,1d, 1b. ual-path configurations 3. 1. ontroller 1/ 4. 2. Stack 1 Stack 2 P SS P SS P SS P SS IOM IOM IOM IOM 26 IM System Storage N series: Universal SS and P abling Guide
ual-path configuration 1. ontroller 1/ 3. 2. 4. Stack 1 Stack 2 P SS P SS IOM IOM Two quad-port SS Hs and two stacks of disk shelves The SS port cabling sequence is 1a, 2a, 2b, 1d. 4. ual-path configurations 1. ontroller 1/ Slot 2 2. 3. Stack 1 Stack 2 P SS P SS P SS P SS IOM IOM IOM IOM SS cabling for new storage systems 27
ual-path configuration ontroller 1/ Slot 2 1. 2. 3. 4. Stack 1 Stack 2 P SS P SS IOM IOM Examples of SS cabling for N6200 series systems You can reference these SS cabling examples for N6200 series dual enclosure H and single enclosure H configurations. The numbered cables show the sequence in which the system should be cabled. Two examples are shown for each configuration type. One example shows disk shelves with the IOMs arranged side by side, similar to how a EXN3500 disk is oriented. The second example shows disk shelves with the IOMs arranged one above the other, similar to how a EXN3000 disk is oriented. Note: You can also view the FSN6200 series Series System abling Examples document a more real-world cabling example of a N6200 series system installed in a cabinet on the N series support website (accessed and navigated as described in Websites) at www.ibm.com/storage/support/ nseries. 28 IM System Storage N series: Universal SS and P abling Guide
N6200 series dual enclosure H configuration The SS port cabling sequence for each controller is 0a,1a, 1b,1d. Multipath H configuration ontroller 1/ ontroller 2/ 1. Slot 0 5. 2. 6. 3. Slot 0 4. 7. 8. Stack 1 Stack 2 P SS P SS P SS P SS IOM IOM IOM IOM ` 1. ontroller 1/ ontroller 2/ Slot 0 Multipath H onfiguration Slot 0 2. 3. 4. 5. Stack 1 6. Stack 2 7. 8. P SS P SS IOM IOM SS cabling for new storage systems 29
N6200 series single enclosure H configuration The SS port cabling sequence for each controller is 0a, 0b. Multipath H configuration 3. 1. Slot 0 Slot 0 Stack 1 P SS P SS ontroller 1/ ontroller 2/ 2. 4. IOM IOM Multipath H configuration Slot 0 1. ontroller 1/ 3. Slot 0 2. ontroller 2/ 4. Stack 1 P SS IOM IOM 30 IM System Storage N series: Universal SS and P abling Guide
Examples of SS cabling for N3150, N3220, and N3240 system models You can reference these SS cabling examples for N3200 series H pair and single-controller configurations. The numbered cables show the sequence in which the system should be cabled. For configurations with external disk shelves, two examples are shown for each configuration type. One example shows disk shelves with the IOMs arranged side by side, similar to how a EXN3500 disk is oriented. The second example shows disk shelves with the IOMs arranged one above the other, similar to how a EXN3000 disk is oriented. Note: These examples show a N3150 or a N3220, but the cabling also applies to the N3240, even though the controllers are in a different position in the N3240. N3150, N3220, and N3240H pair configuration with one stack of disk shelves (multipath H) The SS port cabling sequence for each controller is 0a, 0b. SS SS 0b 0a 0b 0a 1. 2. 3. P SS P SS IOM IOM 4. P SS P SS P SS P SS SS cabling for new storage systems 31
SS SS 0b 0a 0b 0a 1. 2. P SS 3. IOM IOM 4. N3150, N3220, and N3240 single-controller configuration with one stack of disk shelves (dual-path) The SS port cabling sequence is 0a, 0b. Note: ual-path provides greater resiliency, but when dual-path is used ata ONTP 8.1.x issues occasional warnings to the console because mixed paths are detected. To avoid the warnings, single-path connections to the external SS storage are a supported option in these configurations. SS P 0b 0a P SS P SS IOM IOM P SS P SS P SS P SS 32 IM System Storage N series: Universal SS and P abling Guide
SS 0b 0a 1. P SS 2. IOM IOM N3150, N3220, and N3240 single-controller configuration with one stack of disk shelves (single-path) The SS port cabling sequence is 0a. SS P 0b 0a P SS P SS IOM IOM P SS P SS P SS P SS SS cabling for new storage systems 33
SS 0b 0a P SS IOM IOM N3150 and N3220 H pair configuration with no external disk shelves (multipath H) The SS port cabling sequence for each controller is 0a, 0b. SS SS 0b 0a 0b 0a Examples of SS cabling for N3600 systems You can reference these SS cabling examples for N3600 H pair (multipath H) and single-controller (dual-path) configurations. The numbered cables show the sequence in which the system should be cabled. Two examples are shown for each configuration type. One example shows disk shelves with the IOMs arranged side by side, similar to how a EXN3500 disk is oriented. The second example shows disk shelves with the IOMs arranged one above the other, similar to how a EXN3000 disk is oriented. Note: For single-controller N3400 storage systems, the controller is in Slot (the bottom slot); therefore, the controller is cabled as if it were controller 2/. If a second controller is added to make this an H pair configuration, the new controller is cabled as controller 1/. 34 IM System Storage N series: Universal SS and P abling Guide
Note: ecause N3600 systems only have one dual-port SS H on each controller, a maximum of one stack of disk shelves is supported in a multipath H or dual-path configuration. N3600 multipath H configuration The SS port cabling sequence for each controller is 1a,1b. ontroller 1/ H pair configuration ontroller 2/ 1. 2. Stack 1 4. 3. P SS P SS IOM IOM SS cabling for new storage systems 35
ontroller 1/ H pair configuration ontroller 2/ 1. 2. Stack 1 P SS 3. IOM IOM 4. N3600 multipath H configuration with one disk The SS port cabling sequence for each controller is 1a,1b. 1. H pair configuration ontroller 1/ ontroller 2/ 4. Stack 1 2. 3. P SS P SS IOM IOM 36 IM System Storage N series: Universal SS and P abling Guide
1. H pair configuration ontroller 1/ ontroller 2/ 3. IOM P SS 4. 2. IOM N3600 dual-path configuration The SS port cabling sequence is 1a, 1b. Single-controller configuration ontroller 2/ 1. 2. Stack 1 P SS P SS IOM IOM SS cabling for new storage systems 37
Single-controller configuration 1. ontroller 2/ 2. IOM IOM Stack 1 P SS N3600 dual-path configuration with one disk The SS port cabling sequence is 1a,1b. Single-controller configuration ontroller 2/ 2. 1. Stack 1 P SS P SS IOM IOM 38 IM System Storage N series: Universal SS and P abling Guide
Single-controller configuration ontroller 2/ 1. 2. P SS IOM IOM Examples of SS cabling for N3400 systems You can reference these SS cabling examples for N3400 H pair (single-path H) and single-controller (single-path) configurations. The numbered cables show the sequence in which the system should be cabled. Two examples are shown for each configuration type. One example shows disk shelves with the IOMs arranged side by side, similar to how a EXN3500 disk is oriented. The second example shows disk shelves with the IOMs arranged one above the other, similar to how a EXN3000 disk is oriented. Note: For single-controller N3400 storage systems, the controller is in Slot (the bottom slot); therefore, the controller is cabled as if it were controller 2/. If a second controller is added to make this an H pair configuration, the new controller is cabled as controller 1/. Note: The N3400 storage system s onboard SS port labeled 0d is treated as slot 0 port a (0a) for cabling purposes. When the onboard port is treated as port 0a, all the SS cabling rules can be applied. Note: ecause N3400 systems only have one SS port on each controller, a maximum of one stack of disk shelves is supported. SS cabling for new storage systems 39
N3400 single-path H configuration The SS port cabling sequence for each controller is 0a. 1. ontroller 1/ 0a 0d H pair configuration e0p 0a 0d ontroller 2/ e0p Stack 1 2. P SS P SS IOM IOM 1. ontroller 1/ 0a 0d H pair configuration e0p 0a 0d ontroller 2/ e0p 2. P Stack 1 SS IOM IOM 40 IM System Storage N series: Universal SS and P abling Guide
N3400 single-path configuration The SS port cabling sequence is 0a. Single-controller configuration ontroller 2/ 0a 0d e0p 1. Stack 1 P SS P SS IOM IOM Single-controller configuration ontroller 2/ 0a 0d 1. e0p Stack 1 P SS IOM IOM SS cabling for new storage systems 41
42 IM System Storage N series: Universal SS and P abling Guide
P cabling for new storage systems fter you cable the SS connections, you have the option to cable the P connections. It is recommended that you use the P protocol because it enables ata ONTP to manage and control the SS disk storage subsystem. P cabling rules You can cable P connections on your storage system by applying the P cabling rules. General P cabling rules General P cabling rules apply to all supported storage systems using SS storage. v You must use T6 Ethernet cables with RJ-45 connectors for P connections. v If your storage system does not have a dedicated onboard network interface for each controller, you must dedicate one for each controller at system setup. You can use a quad-port Ethernet card. v ll P connections to the disk are cabled through the P ports, which are designated by a square symbol or a circle symbol. P connectivity rules P connectivity rules apply to all supported storage systems using SS storage. Shelf-to- connections v ll the disk shelves in a stack are daisy-chained when you have more than one in a stack. v IOM circle port is connected to the next IOM square port. IOM circle port is connected to the next IOM square port. Intrastack connections v The last and first in each stack are connected to provide resiliency within each stack of disk shelves. v The last IOM circle port is connected to the first IOM square port. This applies to stacks containing one or more disk shelves. opyright IM orp. 2012 43
ontroller-to-stack connections v Each storage system controller is connected to each stack of disk shelves through a dedicated. v ontroller 1/ always connects to the first IOM square port in a stack. ontroller 2/ always connects to the last IOM circle port in a stack. ontroller-to-controller connections (N3150, N3220, and N3240 system models with no external SS shelves) Each storage system controller is connected to the partner controller through a dedicated wrench icon. This rule applies to N3150, N3220, and N3240 storage systems only. N3400 storage system connections For single-controller N3400 storage systems, the controller is in Slot (the bottom slot); therefore, the controller is cabled as if it were controller 2/. If a second controller is added to make this an H pair, the new controller is cabled as controller 1/. abling P connections How you cable your system for P depends on whether you are cabling a storage system with external SS storage or a system with no external SS storage. abling P on systems with external SS storage abling P on a system with external SS disk shelves involves connecting disk shelves within a stack, connecting the first and last disk shelves in a stack, connecting the stacks (if you have more than one stack), and connecting the controllers to the stacks. efore you begin If your storage system does not have a native onboard P port, you must be prepared to designate a network interface for each controller at system setup. If you are using a four-port Ethernet card for this purpose, this card must already be installed and available in each controller. Two examples are shown for each configuration type. One example shows disk shelves with the IOMs arranged side by side, similar to how a EXN3500 disk is oriented. The second example shows disk shelves with the IOMs 44 IM System Storage N series: Universal SS and P abling Guide
arranged one above the other, similar to how a EXN3000 disk is oriented. bout this task To cable the P connections, you use the Ethernet cables that came with your system. Procedure 1. If you have more than one disk in a stack, cable your -to- P connections; otherwise, go to Step 2. a. onnect each IOM circle port to the next IOM square port until all shelves in each stack are connected. b. onnect each IOM circle port to the next IOM square port until all shelves in each stack are connected. aisy-chain connections P SS P SS IOM IOM aisy-chain connections P SS IOM IOM 2. able the intrastack connection. P cabling for new storage systems 45
If you have... More than one disk in a stack of disk shelves Then... onnect the last IOM circle port to the first IOM square port and repeat for all stacks. Intrastack connection P SS P SS IOM IOM Intrastack connection P SS IOM IOM Only one disk in a stack of disk shelves onnect the IOMs within the disk by connecting the IOM circle port to the IOM square of the disk. Intrastack connection P SS P SS IOM IOM P SS IOM IOM 46 IM System Storage N series: Universal SS and P abling Guide
3. If you have more than one stack of disk shelves, cable the stack-to-stack connections; otherwise, go to Step 4. a. eginning with Stack 1, connect the last IOM circle port to the next stack's first IOM square port. Stack 1 P SS P SS IOM IOM Stack-to-stack connections Stack 2 P SS P SS IOM IOM Stack 3 P SS P SS IOM IOM Stack 1 Stack-to-stack connections Stack 2 Stack 3 P SS P SS P SS IOM IOM b. Repeat Substep a until all stacks are connected. 4. able controller-to-stack connections. If your system has a controller... Then... 1/ able controller 1/ to the first stack first IOM square port. 2/ able controller 2/ to the last stack last IOM circle port. n H pair with two stacks of disk shelves has the controller-to-stack connections cabled as follows: P cabling for new storage systems 47
Note: N3400 storage systems use the e0p port on each controller. ontroller 1/ ontroller 2/ Stack 1 Stack 2 P SS P SS P SS P SS IOM IOM IOM IOM ontroller 1/ ontroller 2/ IOM IOM P Stack 1 SS Stack 2 P SS 5. Verify that the P cabling is correct by entering the applicable command: v For 7-Mode, enter the following command from the system console:storage show acp If you have an H pair, run this command on both nodes. 6. The next step depends on what the command output is for each node. If the result in the output shows P connectivity status... Full onnectivity Then... P cabling is correct. The number of IOMs detected out-of-band (through P) and in-band (through SS) are the same. 48 IM System Storage N series: Universal SS and P abling Guide
If the result in the output shows P connectivity status... Then... No onnectivity Repeat Step 4. Partial onnectivity dditional onnectivity No P ports on the IOMs are connected to the storage controllers. Repeat Steps 1 through 6 to see what P connections were missed. Fewer IOMs are detected out-of-band (through P) than in-band (through SS). Repeat the abling SS ports procedure to see what SS connections were missed. More IOMs are detected out-of-band (through P) than in-band (through SS). 7. oot the storage system and begin setup. See the Installation and Setup Instructions that came with your storage system. Note: If your storage system does not have a dedicated P onboard on each controller for P, you will designate them at setup. ttention: fter your storage system is up and serving data, you cannot move SS cables nondisruptively. abling P on systems with no external SS storage abling your N3150, N3220, and N3240 H pair with internal SS storage for P involves one connection between the two controller modules. Procedure able the locked wrench P port on one controller module to the same port on the partner controller module. The following example shows a N3150 or a N3220, but the cabling also applies to the N3240, even though the controllers are in a different position in the N3240. SS P (Locked wrench) SS P (Locked wrench) 0b 0a 0b 0a P cabling for new storage systems 49