BICSI Northeast Regional Meeting June 17, Storrs CT. Considerations in Data Center Design. Thomas McNamara, RCDD

Transcription

1 BICSI Northeast Regional Meeting June 17, Storrs CT Considerations in Data Center Design Thomas McNamara, RCDD

2 AGENDA Purpose is to allow the ITS professional to be aware of the issues and therefore pose the right questions and seek the answers from the owner (I.T) during the design process. Participants are encouraged to share ideas on specific codes and standards that apply/don t apply and what their experiences have proven to work.

3 Codes and Standards There are specific Codes (NEC 250/392/645/800), Standards (ANSI/TIA/EIA-942) requirements and practices (BICSI and The Uptime Institute) that can apply. All of these factors will determine the ultimate costs and functionality of the Data Center ITS. Some of the design choices that need to be made are: Overhead vs. under-floor cable tray? Plenum vs. non-plenum cabling? Shielded vs. non-shielded cabling? Single-mode vs. multimode fiber Optic cabling and termination methods? Information Transport System (ITS) grounding (TMGB)?

4 ITS Infrastructure and Cabling Choices Overhead Tray vs. Under-floor?

5 Cable Tray Options

6 Underfloor Tray

7 Overhead Single-Tier

8 Overhead Multi-Tier

9 ITS Infrastructure and Cabling Choices Backbone cable distribution for ER, MDA, HDA and NOC?

10 ITS Infrastructure and Cabling Choices Server/Host/SAN Cabling Media (i.e. Copper Cat.5E and Lazer Optimized Fiber)

11 Main Distribution Frame Fiber Optic Distribution (Backbone) versus Copper UTP (Horizontal)

12 Data Center Room Adjacencies TR/Cabling for Offices, Labs and other Soft Support Areas

13 Cabling for Offices, Labs and Other Soft Support Areas Medium Data Center

14 Redundancy Requirements Early agreement on the Tier Level (TI,TII,TIII,TIV) is the most important factor in designing the facility Tier I - composed of a single path for power and cooling distribution, without redundant components, providing % availability. Tier II - composed of a single path for power and cooling distribution, with redundant components, providing % availability. Tier III - composed of multiple active power and cooling distribution paths, but only one path active, has redundant components, and is concurrently maintainable, providing % availability. Tier IV - composed of multiple active power and cooling distribution paths, has redundant components, and is fault tolerant, providing % availability.

15 Redundancy Requirements Decision will dictate the level of redundancy required, including Information Transport System Start at the beginning (ER) if redundancy is a requirement (typical today is Tier III/Tier IV Combination)

16 Entrance Room Separate ER for Multiple Carrier s Demarcation OK to combine ER and MDA if environment is more suitable If the system must be designed to T-IV, all services should be fed from multiple (redundant) network carrier components

17 Cabling Type Requirements per NEC Non-Plenum (PVC) versus Plenum-Rated (CMP) Cabling NEC Article 645 (NFPA) Computer Room Cabling The NEC is the governing document for the installation of electrical cable in all situations. Code permits non-plenum cable under a raised floor that meets certain conditions per Article of the NEC. This supersedes Article 800 which requires the use of Plenum-Rated cables in air handling spaces. In summary, Article describes certain HVAC and disconnect features that must be included in your computer room design in order for the Article to apply. Using Non-Plenum Cable Under a Raised Floor and Meeting Code There are provisions in the National Electric Code that clearly permit the installation of non-plenum cable under a raised floor. Even though the National Electric Code permits non-plenum cable under a raised floor under certain circumstances, the final decision on which cable can be used is up to the Authority Having Jurisdiction (AHJ) which should be contacted prior to writing a cabling RFP and seek written permission to use non-plenum communications cable under the raised floor.

18 ITS grounding per NEC/TIA Grounding requirements with regards to the Data Center Grid: Reference BICSI TDDM Chapter 8 Version11 (Grounding, Bonding and Protection ANSI J-STD-607-A (Commercial Building Grounding and Bonding) IEEE 1100 (Power and Grounding Sensitive Electronic Equipment) TIA 942 (Telecommunications Infrastructure Standard for Data Centers) NEC Article 250 / Parts A-G (System Grounding) Fundamental concept of grounding: The prevention of "objectionable current flow over the grounding conductors or grounding paths."

19 Horizontal Distribution Area IS Strategy: Are the network (Core & Distribution) electronics going to be centralized (MDA only)?

20 Horizontal Distribution Area Core (MDA) versus Distribution Switching: Edge Switch placed at each Row (HDA) Fiber Optic Backbone to MDA

21 Horizontal Distribution Area Horizontal (Edge) Server cabinet electronics: Edge Switch placed in each cabinet Fiber Optic Backbone to MDA

22 Horizontal Distribution Area So how does this effect the Horizontal Distribution Area (HDA)? Cable tray, cabling and wire-management will vary by degree (size & quantity) based on the strategy chosen by the IS designer.

23 Server Cabinet Distribution Fiber Optic versus Copper or combination? Decision based on server interface (1Gb) and future deployments (10G)? If the system must be designed to T-IV, should all servers be fed from multiple (redundant) network components? How does this effect the ITS cabling?

24 SAN/Hosts Connectivity SAN Protocol cabling requirements: SAN Area Fiber Optics (Cabinet-to-Cabinet Patching) vs. MDA Distribution? 50-Micron Multi-Mode Laser Optimized Fiber (LOF)?YES!! Category 6(A) / CAT 7 (STP) for 10G IP YES!! Connectivity for: iscsi; a protocol that allows clients (called initiators ) to send SCSI commands ( CDBs ) to SCSI storage devices ( targets ) on remote servers. It is a popular Storage Area Network (SAN) protocol, allowing organizations to consolidate storage into data center storage arrays while providing hosts (such as database and web servers) with the illusion of locally-attached disks.

25 Sample SAN Equipment Positioning Tool SSR Data Center - SAMPLE Link Loss Budget Calculations End-Point shown above has an estimated (MAX) db Link-Loss as follows: = SAN A 2 = LAN B 3 = SAN C 4 = SAN D = LAN E ### ### ### ### ### ### ### ### ### ### ### ### ### ### ### ### R4 ### ### ### ### ### ### #### ### ### ### ### ### ### ### ### ### ### ### R4 6 = SAN F ### ### ### ### ### ### ### ### 7 = SAN G ### ### ### ### ### ### ### ### ### #### ### ### ### #### ### ### ### ### ### ### ### ### ### ### ### ### ### ### ### 8 = LAN H 38 ### ### ### ### 19 9 = SAN I ### ### ### ### ### ### ### ### ### ### ### ### ### ### ### ### R3 ### ### ### ### ### ### ### #### ### ### ### ### ### ### ### ### ### ### ### R3 10 = SAN J ### ### ### ### ### ### ### ### ### 11 = LAN K ### ### ### ### ### ### ### ### ### #### ### ### ### #### ### ### ### ### ### ### ### ### ### ### ### ### ### ### ### 12 = SAN L 39 ### ### ### ### = SAN M ### ### ### ### ### ### ### ### ### ### ### ### ### ### ### ### R2 ### ### ### ### ### ### ### #### ### ### ### ### ### ### ### ### ### ### ### R2 14 = LAN N ### ### ### ### ### ### ### ### ### 15 = SAN O ### ### ### ### ### ### ### ### ### #### ### ### ### #### ### ### ### ### ### ### ### ### ### ### ### ### ### ### ### 16 = SAN P 40 ### ### ### ### 1 17 = LAN Q ### ### ### ### ### ### ### ### ### ### ### ### ### ### ### ### R1 ### ### ### ### ### ### ### #### ### ### ### ### ### ### ### ### ### ### ### R1 18 = SAN R ### ### ### ### ### ### ### ### ### 19 = SAN S ### ### ### ### ### ### ### ### ### #### ### ### ### #### ### ### ### ### ### ### ### ### ### ### ### ### ### ### ### 20 = LAN T ### ### ### ### 21 = SAN U ### ### ### ### 22 = SAN V ### ### ### ### 23 = LAN W ### ### ##### #### ### ### ### ### ### ### ### ### ### ### #### ### ### ### ### ### #### ### ### #### ### ### ### ### ### ### ### ### ### ### ### ### ### ### ### 24 = SAN X ### ### 25 = SAN Y ### ### 26 = LAN Z KEY: ACTIVE SERVER CABINET ### ### ### ### ### ### ### ### ### ### ### ### ### 27 = SAN AA MDF 28 = SAN BB FUTURE SERVER CABINET 29 = LAN CC 30 = SAN HOST/TAPE/ETC General, Notes are as follows: 31 = SAN 32 = LAN SAN AREA (Brocade M6140 / 48000) Overall 10G Ethernet/SAN Spec allows for total 2.6dB using Laser Optimized (LO) 50um/125 cable ( nm VCSel). 33 = SAN 34 = SAN DCN CABINET 35 = LAN BC will require to goto ELO due to the number of connections and patch cables and BC must locate the SAN in the MDF (TBD). 36 = SAN ### FIBER - MTP/LC CASSETTE 37 = SAN (AREA) Each Cassette (MPO/LC) consumes.75db +.5 MPO: 1.25 db 38 = SAN (AREA) ### FIBER - MPO/MPO ADAPTERS Each Adapter Panel (MPO/MPO) in SAN area and MDF consumes.75db (MPO Mated Pai0.75 db 39 = SAN (AREA) The 12-Fiber Ribbon cable consumes.0011db per foot 850nm) Note: Based on EL0.001 db per foot 40 = SAN (AREA) ### LC - CONNECTOR (PATCHCORD) ### 50um ENHANCED LASER-OPTIMIZED (ELO) RIBBON CABLE NOTE: If BC does not locate SAN in MDF, to determine total channel (patch-thru), pick any two (2) SAN End-Points and add together NOTE: These calculations do not include Patch Cord Loss (Length), or equipment Gain

26 Sample Fiber Calculators SSR Example SSR FO Link Loss (TFM ).xls Siemon Company Online Tool Siemon Company - Fiber Loss Calculator ( ).xls

27 Questions and Follow-up Final Q&A Request for presentation attachments to