Structured Cabling Design for Large IT/Service Provider Data Centres. Liang Dong Yuan, Technical Sales manager, CDCE

Size: px

Start display at page:

Download "Structured Cabling Design for Large IT/Service Provider Data Centres. Liang Dong Yuan, Technical Sales manager, CDCE"

Bruce Newton
6 years ago
Views:

1 Structured Cabling Design for Large IT/Service Provider Data Centres Liang Dong Yuan, Technical Sales manager, CDCE

2 Structured Cabling Design for Large IT/Service Provider DCs Outline Section Industry Terms and Definitions Fiber Count Cause and Effect Deployment Methods Key areas Cloud Grows at the Expense of Enterprise Types of Facilities ANSI/TIA-942-A Network Speeds Network Architectures Oversubscription Switch Configuration Data Centre Cabling Areas Cabling Choices High Fiber Count Trunks Example Design Parameters Determine Fiber Count Mapping Logical Architecture - Structured Cabling Options Value Cable Tray Value Would You Rather

3 Structured Cabling Design for Large IT/Service Provider DCs Outline Section Industry Terms and Definitions Fiber Count Cause and Effect Deployment Methods Key areas Cloud Grows at the Expense of Enterprise Types of Facilities ANSI/TIA-942-A Network Speeds Network Architectures Oversubscription Switch Configuration Data Centre Cabling Areas Cabling Choices High Fiber Count Trunks Example Design Parameters Determine Fiber Count Mapping Logical Architecture - Structured Cabling Options Value Cable Tray Value Would You Rather

4 Industry Terms and Definitions Cloud Grows at the Expense of Enterprise Equilibrium is Maintained Light Counting Thoughts Less Data Centres Dell Oro Bigger Data Centres Structured cabling requirements will change

Industry Terms and Definitions Types of Facilities DC Types Description

Organizations Design, build and operate their own facilities Colocation

maintenance A multi-tenant data centre, colocation space can be sold to

Large Space Supplies Facilities Maintenance Sell data centre space in

Colo IT/Service Provider Dedicated Hosting Servers are NOT Shared

capacity to single customers Managed Hosting Hardware Owned by Customer

5 Industry Terms and Definitions Types of Facilities DC Types Description Notes In-House (Enterprise) Private Ownership Enterprise Large Organizations Design, build and operate their own facilities Colocation Customers Own Hardware Outsource facility and internal systems maintenance A multi-tenant data centre, colocation space can be sold to enterprises by the rack, cabinet or cage Wholesale Data Centre Sell Large Space Supplies Facilities Maintenance Sell data centre space in larger capacities vs. Colo IT/Service Provider Dedicated Hosting Servers are NOT Shared Customer Controls Server The provider operates and/or rents server capacity to single customers Managed Hosting Hardware Owned by Customer or Provider Many Services Provided The provider operates servers and storage for its customers Shared Hosting Multi-tenant Applications Customers share server capacity

6 Industry Terms and Definitions ANSI/TIA-942-A ANSI/TIA-942-A Telecommunications Infrastructure Standard for Data Centre Key Areas Insight Source ANSI/TIA Architecture Cross Connect vs. Interconnect Redundancy Definitions Zone Architectures Energy efficiency Recommends a star topology architecture MDA, IDA, HDA, ZDA, EDA Tiers (1-4) Reduced topologies and consolidation points Examples of routing cables and air flow contention

7 Structured Cabling Design for Large IT/Service Provider DCs Outline Section Industry Terms and Definitions Fiber Count Cause and Effect Deployment Methods Key areas Cloud Grows at the Expense of Enterprise Types of Facilities ANSI/TIA-942-A Network Speeds Network Architectures Oversubscription Switch Configuration Data Centre Cabling Areas Cabling Choices High Fiber Count Trunks Example Design Parameters Determine Fiber Count Mapping Logical Architecture - Structured Cabling Options Value Cable Tray Value Would You Rather

8 Fiber Count Cause and Effect Network Speeds Physical Logical Speed 2F 2F 2F Sw5 Sw4 Sw3 Sw2 2F 10G Thoughts 2/8F 2/8F 2/8F Sw5 Sw4 Sw3 Sw2 2/8F 2/8/20F 2/8/20F2/8/20F Sw5 Sw4 Sw3 Sw2 2/8/20F Sw2 Sw3 Sw4 Sw5 Sw2 Sw3 Sw4 Sw5 40G 100G Network speeds affect Fiber counts Check IEEE roadmaps for Ethernet Check ANSI roadmaps for Fiber Channel Sw2 Sw3 Sw4 Sw5

9 Fiber Count Cause and Effect Network Architectures Physical Logical Speed 8F 8F 8F Sw5 Sw4 Sw3 Sw2 8F Pt to Pt Sw2 Sw3 Sw4 Sw5 40G 8F 32F 32F 32F Sw5 Sw4 Sw3 Sw2 32F Sw2 Sw3 Full Mesh Sw5 40G 8f Thoughts Redundancy increases Fiber count Sw4 16F 16F 16F Sw5 Sw4 Sw3 Sw2 Sw2 Spine + Leaf 40G 8F Sw3 Sw4 Sw5

10 24 Servers 24 Servers 24 Servers Fiber Count Cause and Effect Oversubscription Physical Logical Speed Over 12F 12F 12F Sw5 Sw4 Sw3 Sw2 12F 10G 4:1 6 out 24F 24F 24F Sw5 Sw4 Sw3 Sw2 24F Sw2 Sw3 Sw4 Sw5 10G 24 in 2:1 12 out Thoughts The lower the oversubscription ratio the higher the Fiber count 48F 48F 48F Sw5 Sw4 Sw3 Sw2 48F Sw2 Sw3 Sw4 Sw5 10G 24 in 1:1 24 out Driven by network traffic requirements Sw2 Sw3 Sw4 Sw5 24 in

11 Fiber Count Cause and Effect Switch Configuration Physical Logical Sw Config 16F 16F 16F Sw5 Sw4 Sw3 Sw2 16F 40G QSFP 32F 32F 32F Sw5 Sw4 Sw3 Sw2 32F 64F 64F 64F 64F Sw5 Sw4 Sw3 Sw2 Sw2 Sw3 Sw4 Sw5 Sw2 Sw3 Sw4 Sw5 10G SFP (servers) 10G SFP (servers) 40G Breakout (servers) 40G QSFP 40G 10G SFP QSFP Thoughts Check Switch Config Check Mfg Recommended Procedures Understand Connector Form Factor Sw2 Sw3 Sw4 Sw5 10G SFP (servers)

12 Structured Cabling Design for Large IT/Service Provider DCs Outline Section Industry Terms and Definitions Fiber Count Cause and Effect Deployment Methods Key areas Cloud Grows at the Expense of Enterprise Types of Facilities ANSI/TIA-942-A Network Speeds Network Architectures Oversubscription Switch Configuration Data Centre Cabling Areas Cabling Choices High Fiber Count Trunks Example Design Parameters Determine Fiber Count Mapping Logical Architecture - Structured Cabling Options Value Cable Tray Value Would You Rather

Deployment Methods Data Centre Cabling Areas Key Areas Insight Meet Me Room Main Distribution Area Demarcation X-Connect Racks/Cabinets X-Connect

13 Deployment Methods Data Centre Cabling Areas Key Areas Insight Meet Me Room Main Distribution Area Demarcation X-Connect Racks/Cabinets X-Connect Indoor Cabling Plenum Rated Indoor/Outdoor Cabling OSE (Optical Splice Enclosures) Plenum/Riser Armored Cable Transition from Indoor to Outdoor Cables

14 Deployment Methods Cabling Choices High Fiber Count Trunks Method Insight Preterm Cables Pigtail Cables Cables that are factoryterminated on both ends with MTP connectors (MTP Trunk Assemblies) Cables that are factory terminated on one end with MTP connectors Field terminated at the blunt cable end Bulk Cable Field terminated on both ends with MTP Splice-On Connectors

Deployment Methods Cabling Choices High Fiber Count Trunks Method Insight Preterm Cables Cables that are factoryterminated on both ends with MTP connectors (MTP Trunk Assemblies) MTP Trunk Assemblies

15 Deployment Methods Cabling Choices High Fiber Count Trunks Method Insight Preterm Cables Cables that are factoryterminated on both ends with MTP connectors (MTP Trunk Assemblies) MTP Trunk Assemblies are used where the entire Fiber count is being deployed at a single location at each end of the link. Main Distribution Area (MDA) to the Horizontal Distribution Area (HDA) or to the Equipment Distribution Area (EDA). Typical deployment for indoor cabling

Deployment Methods Cabling Choices High Fiber Count Trunks Method Insight Pigtail Cables Cables that are factory terminated on one end with MTP connectors Field terminated at the blunt cable end

16 Deployment Methods Cabling Choices High Fiber Count Trunks Method Insight Pigtail Cables Cables that are factory terminated on one end with MTP connectors Field terminated at the blunt cable end Environments where the pathway will not allow for a pre-terminated end with a pulling grip to fit through, such as a small conduit space Environments where high Fiber count assembly is deployed to consolidate inter-building Fiber connectivity Deployments when the exact pathway or route is not fully known prior to ordering of the assembly.

Deployment Methods Cabling Choices High Fiber Count Trunks Method Insight Bulk Cable Field terminated on both ends with MTP Splice-On Connectors Field terminated solutions where cable designs

17 Deployment Methods Cabling Choices High Fiber Count Trunks Method Insight Bulk Cable Field terminated on both ends with MTP Splice-On Connectors Field terminated solutions where cable designs require extremely high Fiber counts, such as 1728 Fibers Deployments where a centre-pull cable installation is required, bulk cable may be needed in order to meet pathway installation challenges

Deployment Methods Cabling Choices High Fiber Count Trunks Method Envir Connector Counts Trunk Type Fiber Type Preterm Cables Pigtail Cable Indoor MTP - MTP 144 192 216 288 432 576

18 Deployment Methods Cabling Choices High Fiber Count Trunks Method Envir Connector Counts Trunk Type Fiber Type Preterm Cables Pigtail Cable Indoor MTP - MTP Non-Armored MMF SMF Preterm Cables Pigtail Cable Indoor/ Outdoor MTP - Fiber Armored Non-Armored MMF SMF Bulk Cable All Fiber - Fiber 144 to 1728 Armored Non-Armored MMF SMF

19 Structured Cabling Design for Large IT/Service Provider DCs Outline Section Industry Terms and Definitions Fiber Count Cause and Effect Deployment Methods Key areas Cloud Grows at the Expense of Enterprise Types of Facilities ANSI/TIA-942-A Network Speeds Network Architectures Oversubscription Switch Configuration Data Centre Cabling Areas Cabling Choices High Fiber Count Trunks Example Design Parameters Determine Fiber Count Mapping Logical Architecture - Structured Cabling Options Value Cable Tray Value Would You Rather

20 Example Design Parameters Cisco Spine Configuration/Device Count 4-way Spine Logical Architecture Number of Nexus 70xx Spine Switches 4x 7009 or 4x 7010 Cisco Nexus 7000 Number of N7K-F248XP-25 Blades per Chassis 7009: : 8 Number of Ports Used for Leaf Switches per Chassis Number of Nexus 3064 Leaf Switches 7009: : Cisco Nexus Servers 24 Number of Nexus 3064 Ports Facing Fabric 32 Number of Nexus 3064 Ports Facing Servers 32 Key Design Parameters: All 10G Ethernet, no 40G Spine Switches: Cisco Nexus 7000 series with 48-port blades Leaf Switches Cisco Nexus Ports Facing Fabric, 32 Ports Facing Servers

Example Determine Fiber Count Fiber Count / Rack Fiber Count Variables Cisco S+L Rules Cisco Leaf Rules Details 4 Spine SW 48 Leaf SW 32 Fabric 32 Server 64F Required per Switch or 72F Trunk

21 Example Determine Fiber Count Fiber Count / Rack Fiber Count Variables Cisco S+L Rules Cisco Leaf Rules Details 4 Spine SW 48 Leaf SW 32 Fabric 32 Server 64F Required per Switch or 72F Trunk Cable per Switch Fabric A Fabric B Spine Switches Cisco Nexus 7000 Architecture S+L A + B Fabrics Network Speed 10G Oversubscription 1:1 (32) 2-F Jumpers (32) 2-F Jumpers Standard Fiber Counts 12F Div

22 Example Mapping Logical Architecture - Structured Cabling Options Logical Architecture Cabling Option (1) Cisco Nexus 7000 Cisco Nexus 7000 Cisco Nexus Servers 24 3,072 (2F) Jumpers Cisco Nexus Servers 24 Rack Row

23 Example Mapping Logical Architecture - Structured Cabling Options Logical Architecture Cabling Option (2) Cisco Nexus 7000 Cisco Nexus 7000 MDA 48 (72F) Cables Backbone Cable Cisco Nexus 3064 HDA 32 Servers 24 Cisco Nexus Servers 24 Rack Row

24 Example Mapping Logical Architecture - Structured Cabling Options Logical Architecture Cabling Option (3) Cisco Nexus 7000 Cisco Nexus 7000 MDA Cisco Nexus Servers 24 6 (576F) Cables Backbone Cable HDA Cisco Nexus Servers 24 Rack Row

Fill Ratio Fill Ratio Example Value Cable Tray Fill Ratio Option 1 3,072 (2F Jumpers) Option 2 48 (72F Trunks) Option 3 6 (576F Trunks) Tray Size TIA 569 100% 93% Cable Fill not good Planned for

25 Fill Ratio Fill Ratio Example Value Cable Tray Fill Ratio Option 1 3,072 (2F Jumpers) Option 2 48 (72F Trunks) Option 3 6 (576F Trunks) Tray Size TIA % 93% Cable Fill not good Planned for initial max calculated fill ratio of 25% 50% 0% 100% 40% Cable Fill not good 27% 4x4 A calculated fill ratio of 50% for 4 pair & similar cable will physically fill entire tray due to spaces between cables 50% 62% 0% 27% 18% 4x6 4x6

26 Example Value Would You Rather Would You Rather Test and Clean Document and Label Pull and Install Option 1 3,072 (2F Jumpers) 6,144 (2F) Duplex LC Connectors 3,072 Jumpers + 6,144 Connectors 3,072 Jumpers (Both Ends) Option 2 48 (72F Trunks) 576 (12F) MTP Connectors 48 Trunks Connectors Option 3 6 (576F Trunks) 576 (12F) MTP Connectors 6 Trunks Connectors 48 Trunks (Both Ends) 6 Trunks (Both Ends) Purchase 3,072 Jumpers 48 (72F Trunks) 6 (576F Trunks) Troubleshoot 3,072 Links, >6000 Connectors 48 Links, 576 Connectors 6 Links, 576 Connectors Move, Add or Change 1 Jumper at a Time. Pt-Pt Config Create Cross-connect, use short jumper Create Cross-connect, use short jumper

27 Structured Cabling Design for Large IT/Service Provider DCs Outline Section Industry Terms and Definitions Fiber Count Cause and Effect Deployment Methods Key areas Cloud Grows at the Expense of Enterprise Types of Facilities ANSI/TIA-942-A Network Speeds Network Architectures Oversubscription Switch Configuration Data Centre Cabling Areas Cabling Choices High Fiber Count Trunks Example Design Parameters Determine Fiber Count Mapping Logical Architecture - Structured Cabling Options Value Cable Tray Value Would You Rather

Structured Cabling Design for Large IT/Service Provider Data Centers

Structured Cabling Design for Large IT/Service Provider Data Centers Author: Dave Kozischek, Marketing Applications Manager, Data Center and In-Building Networks Introduction Structured cabling is defined