Application Note. Data Center Standards and Layouts

Transcription

1 Application Note Data Center Standards and Layouts

2 1. Data Center Standards In earlier times, a room in which one or more servers, a telephone system and active network components were located was often called a computer room. However, because of the development of cloud computing, virtualization technologies and an increased tendency to use outsourcing, computer room requirements became more complex. The one room was replaced by a room structure, whose individual rooms are assigned defined functions. The data center is now divided up into an entrance area, the actual computer room and a work area for administrators, right through to separate rooms for UPS batteries, emergency power generators and cooling. In addition, attention must be given to matters like access control, video monitoring and alarm systems. An emphasis on active and passive components requires an improved infrastructure that includes such equipment as a cooling system and power supply, which in turn affect installation and construction as well as the data cabling structure. The point-to-point connections that were used previously are being replaced by a structured cabling system, which allows for rapid restoration in case of a fault, effortless system expansion and easy administration Overview of relevant standards Increasing demands on data centers have led standardization bodies to take this development into account and to take a close look as this topic. Cabling standards issued from international and national committees describe the structure and characteristics of a cabling system almost identically, but differ in terms of content. There are three big organizations in the world which are concerned with the standardization of data centers. The ISO/IEC (International Organization for Standardization / International Electrotechnical Commission) develops international standards, the CENELC (European Committee for Electrotechnical Standardization) European standards and the ANSI (American National Standards Institute) American standards. There are other organizations for the area of data centers, though these play a subordinate role to the ones above. The standardization bodies also remain in contact with one another in an attempt to avoid differing interpretations on the same topic, but this is not always successful. To achieve harmonization in Europe, CENELEC and ISO exchange information with national committees (SEV/SNV, VDE/DIN, etc.). Data center cabling is covered by the following key standards: ISO/IEC EN TIA-942 All three standards focus on cabling, but describe the requirements for a data center in different ways. New standards were developed for this area so as not to change the general (generic) cabling structure all too greatly. The central focus of these standards is to cover the structure and performance of data center cabling. The goal was to move away from the general point-to-point connections normally used in data centers and construct a structure that is flexible, scalable, clear and permits changes, troubleshooting and expansion. Application Note Data Center Standards and Layouts v1.0 en Thomas Glaettli, Roland Diethelm 2

3 Where are the differences between the relevant standards for a data center? A cabling planner designing a generic building cabling system must consider space requirements, the connection for potential equalization and other factors, but in a data center the planner must also consider things like low space availability, air conditioning, high power consumption, redundancy and failure safety including access restriction. The type of the data center and the future prospect for transmission protocols and data rates have an effect on the development of standards. These requirements and the interfaces to other standards are examined differently, depending upon the standard. The most important points in the different standards are classified in the following table. Criteria ISO/IEC EN TIA-942 Structure 2 Cabling performance 2 Redundancy 2 Grounding / potential equalization IEC EN Tier classification.. Cable routing IEC EN /A1. 2 Ceilings and raised floor IEC EN /A1 6 Floor load 2 Space requirements (ceiling height, door width) IEC EN /A1 3 2 Power supply / UPS 2 Fire protection / safety EN /A1 4 4 Cooling 2 Lighting 2 Administration / labeling IEC EN /A1 5 1 Temperature / humidity 2 1 not data center-specific, 2 refers to TIA-607, 3 only door widths and ceiling height, 4 refers to local standards, 5 refers to complexity level, 6 refers to TIA-569 On first glance, we see that TIA-942 devotes much more attention to the subject of data centers than EN or ISO/IEC Many important terms and their descriptions can be found in additional documents in ISO/IEC and EN, and are kept at a very general level. Some IEC documents were created a few years ago and are therefore not at the current state of technology. There are also differences in terminology in the central areas of this cabling standard. Application Note Data Center Standards and Layouts v1.0 en Thomas Glaettli, Roland Diethelm 3

4 1.2. ISO/IEC The configuration and structure of an installation per ISO/IEC must be carried out in accordance with this standard and/or ISO Testing methods for copper cabling must follow IEC and for optical fiber cabling the IEC standard must be followed. In addition, a quality plan and installation guidelines in accordance with IEC must be observed for any compliant installation. Since IEC is an older document, ISO/IEC must be consulted for cable routing issues. This standard not only contains descriptions for cable routing, but also rudimentary instructions concerning failure safety. Nevertheless, neither IEC nor ISO/IEC is explicitly designed for data center requirements. The cabling system is arranged in a tree structure that starts out from a central distributor. Point-to-point connections must be avoided wherever possible. Exceptions are allowed as long as active devices are located very closely to one another or cannot communicate over the structured cabling system. Not included in this structure are cabling in a local distributor per ISO and cabling at the interface to external networks. Application Note Data Center Standards and Layouts v1.0 en Thomas Glaettli, Roland Diethelm 4

5 Evolution of the basic computer room into a data center involves increased layout requirements. As mentioned at the start, it has become necessary to separate the different areas of the data center into rooms. It must be clearly indicated that the network structure in the data center is separate from building cabling. In addition, the connection to the internal network must be established over a separate distributor that is spatially detached from the data center. The interface to the external network (ENI) is created either within or outside of the data center. All other functional elements must exist in the data center permanently and be always accessible. Cabling components, patch cords and connection cables as well as distributors for the building s own network are not components of the standard. As the table below shows, ISO has terminology that is different from that of ISO/IEC ISO/IEC ISO External network interface (ENI) Campus distributor (CD) Network access cable Primary cable Main distributor (MD) Building distributor (BD) Main distributor cable Secondary cable Zone distributor (ZD) Floor distributor (FD) Area distributor cable Tertiary/horizontal cable Local distribution point (LDP) Consolidation point (CP) Local distribution point cable Consolidation point cable Equipment outlet (EO) Telco outlet (TO) Different distributor functions may be combined, depending upon the size of the data center. However, at least one main distributor must exist. Redundancy is not explicitly prescribed, but is referred to under failure safety. The standard gives information on options for redundant connections, cable paths and distributors. Like the consolidation point, the local distribution point (LDP) is located in either the ceiling or in the raised floor. Patching is not permitted, since the connection must be wired through. ISO/IEC refers to ISO regarding performance. The properties described in detail in ISO apply for all transmission links or channels and components. The difference is that for the data center minimum requirements for main and area distributor cabling apply. A cabling class that satisfies the applications listed in ISO is permitted for the network access connection. Cabling Infrastructure: Copper cables: Category 6 A, Category 7, Category 7 A Copper transmission link or channel: Class E A, Class F, Class F A Multi-mode optical fiber cables: OM3, OM4 Single-mode optical fiber cables: OS1, OS2 Application Note Data Center Standards and Layouts v1.0 en Thomas Glaettli, Roland Diethelm 5

6 Connectors for Copper Cabling: IEC or IEC for Category 6 A for shielded and unshielded respectively IEC /71 for Category 7 and 7 A IEC for Category 7 and 7 A Connectors for Optical Fibers: LC duplex per IEC for GA and ENS MPO/MTP multi-fiber connectors per IEC for 12 or 24 fibers It is important to consider the appropriate polarity in multi-fiber connections so the transmitter and receiver can be correctly connected. ISO/IEC references ISO with regard to this. Due to the age of the document the polarity of multi-fiber connections is not yet described EN This document is essentially the same as ISO/IEC 24764, but has a few minor differences. The last changes were established through an amendment. Therefore EN as well as EN /A1 are necessary for the complete overview. This also applies for all other standards in the EN series. EN also uses the hierarchical structure per EN & -2. A standard-compliant installation is achieved through both application of the EN series and application of the EN series and its amendments. In particular, an entire section about data centers has been added to EN /A1. The following topics are covered in this section: Design proposals Cable routing Separation of power and data cables Raised floors and ceilings Use of pre-assembled cables Entrance room Room requirements As opposed to ISO 11801, this standard uses optical fiber classes other OF-300, OF-500 und OF-2000; possible classes are OF-100 to OF Cabling performance is determined by EN and EN However, minimum requirements for data center cabling were also established in the standard. A cabling class that satisfies the applications listed in ISO is also permitted for the network access connection in EN Application Note Data Center Standards and Layouts v1.0 en Thomas Glaettli, Roland Diethelm 6

7 Cabling Infrastructure: Copper cables: Category 6 A, Category 7, Category 7 A Copper transmission link or channel: Class E A, Class F, Class F A Multi-mode optical fiber cables: OM3, OM4 Single-mode optical fiber cables: OS1, OS2 Connectors for Copper Cabling: IEC or IEC for Category 6 A for shielded and unshielded respectively IEC /71 for category 7 and 7 A IEC for category 7 and 7 A Connectors for Optical Fibers: LC duplex per IEC for GA and ENS MPO/MTP multi-fiber connectors per IEC for 12 or 24 fibers Though different options are listed for the use of multi-fiber connections, polarity is not covered in any of these standards TIA-942 Without a doubt, the American standard is currently the most comprehensive body of standards for data center. Not only is cabling described, but also the entire data center environment including the geographical location, as well as a discussion of seismic activities. Among other things, the data center is divided up into so-called tiers, or different classes of availability The data center must satisfy specific requirements for each class in order to guarantee availability. What differences from ISO/IEC exist, apart from cabling performance? The interface to the external network may not be in the computer room with TIA-942. The maximum length of horizontal cabling may not exceed 300 meters, even with glass fibers. No minimum requirements are provided for cabling. The limits for Category 6A ( 6 A ) are less strict than those of ISO/IEC or EN Application Note Data Center Standards and Layouts v1.0 en Thomas Glaettli, Roland Diethelm 7

8 The smallest common denominator of the data center standards listed is cabling. TIA-942 references the EIA/TIA-568 series in the area of performance. TIA-942 itself is divided into three sections; TIA-942, TIA and TIA All data center details are covered in the main standard. The use of coaxial cables is covered in TIA-942-1, and the latest version of the Cat6A (copper) performance class, laser-optimized fibers (OM3 and higher), changed temperature and ambient humidity as well as other changes in content are introduced in TIA Cabling is defined as follows in TIA. Cabling Infrastructure: Copper cable: Category 3 6A Copper transmission path: Category 3 Category 6A (Category 6A is recommended, TIA doesn t use classes) Multi-mode optical fiber cable: OM1, OM2, OM3 (OM3 recommended) Single-mode optical fiber cable: 9/125 µm Coaxial cables: 75 ohm Connectors for Copper Cabling: ANSI/TIA/EIA-568-B.2 Plug Connectors for Optical Fibers: SC duplex (568 SC) per TIA MPO/MTP multi-fiber connector per TIA (only 12-fiber) Other duplex connectors like LC may be used. A new version of TIA-942 (TIA-942A) was in the planning stages at the time this document was published. Since this change includes important areas, a preview of the new version is meaningful. Combining TIA-942, TIA and TIA into one document Harmonization with TIA-568.C0, TIA-568.C2, TIA-568.C3 Grounding is removed and integrated with TIA-607-B Administration is integrated with TIA-606-B Distribution cabinets and separation of data and power cables is integrated with TIA-569-C The limit restriction for glass fiber paths in horizontal cabling is rescinded. Category 3 and Category 5 are no longer allowed for horizontal cabling. The glass fiber for backbone cabling is OM3 and OM4. OM1 and OM2 are no longer allowed. LC and MPO plugs are a mandatory requirement for the data center. Adaptation of ISO/IEC terminology Application Note Data Center Standards and Layouts v1.0 en Thomas Glaettli, Roland Diethelm 8

9 1.5. Conclusion None of the bodies of standards is complete. A body will either omit an examination of the overall data center, or its permitted performance class will not allow for scaling to higher data rates. It is therefore recommended that all relevant standards be consulted when planning a data center. The best standard should be used, depending upon specific requirements. One example of this would be to use the cabling performance capability from the standard which provides the highest performance requirement for cabling components. In the case of other properties, other standards should be used if they handle these properties in a better way. 2. Data Center Layout In spite of the virtualization trends currently in the software market keyword server consolidation there is a high demand on the horizon for renewing data center and server rooms. It is estimated that over 85 percent of all data centers in Germany are over ten years old. A study by Actionable Research that was recently commissioned by Avocent, is also examining the global situation in this area. Approximately 35 percent of all data centers worldwide are therefore currently undergoing a redesign Standard Requirements Even if small differences between generic and data center cabling are uncovered between the American TIA-942 standard and international and European standards, these bodies of standards essentially differ only in how they name functional elements. ISO/IEC ISO TIA-942 External Network Interface ENI (External Network Interface) Campus Distributor (CD) (ENI) ER (Entrance Room) Network Access Cable Primary Cable Backbone Cabling Main Distributor (MD) Building Distributor (BD) MC (Main Cross Connect) MDA (Main Distribution Area) Main Distributor Cable Secondary Cable Backbone Cabling Zone Distributor (ZD) Floor Distributor (FD) HC (Horizontal Cross Connect) HDA (Horizontal Distribution Area) Zone Distribution Cable Tertiary/Horizontal Cable Horizontal Cabling Local Distribution Point (LDP) Consolidation Point (CP) CP* (Consolidation Point) ZDA (Zone Distribution Area) Local Distribution Point Cable Consolidation Point Cable Horizontal Cabling Equipment Outlet (EO) Telecommunications Outlet (TO) EO (Equipment Outlet) EDA (Equipment Distribution Area) CP* = LDP (Local Distribution Point) Application Note Data Center Standards and Layouts v1.0 en Thomas Glaettli, Roland Diethelm 9

10 Unique assignment exists of functional elements to rooms, though in the TIA-942 it is data center areas rather than the functional elements that are at the forefront. Given a detailed examination, we see that this topology is the same for ISO/IEC and EN Individual areas in the data center are therefore assigned specific functions. Entrance Room (External Network Interface): This is the entrance area to the network in the data center, which provides for access to the public network (the Internet provider) and can be connected multiple times, depending upon the tier level that is selected. In smaller networks, the External Network Interface can be connected directly to the Horizontal Distribution Area (Zone Distributor). Main Distribution Area (Main Distributor): This area represents the core of the data center and therefore forms the lifeblood of the network, which is why the redundant connections and components just in this area alone are of crucial importance. All data traffic in the backbone is therefore controlled here, which is why this point in the network is known as the Core Layer. However, the Aggregation Layer (or Distribution Layer), whose aggregation/distribution switches bundle and forward Access Layer data traffic to the core, is also located in this area. Application Note Data Center Standards and Layouts v1.0 en Thomas Glaettli, Roland Diethelm 10

11 Horizontal Distribution Area (Zone Distributor): In this interface between backbone and horizontal cabling, the data traffic of the access switches which control the data exchange with terminal devices is passed over to the aggregation layer. This area in the network is known as the Access Layer. Zone Distribution Area (Local Distribution Point): This area is for interim distribution to the Equipment Distribution Area, which can be used for reasons of space and is placed in the raised floor, for example. The Raised Floor Solution from R&M that was developed for this purpose provides the ideal option for this process, since it allows up to 288 connections per box and is freely configurable, thanks to its modular design. Telecom Room: This is the location where the connection to the internal network is located. The Operation Center, Support Room and Offices are rooms for data center personnel. Backbone cabling is preferably laid with fiber optic cables, and horizontal cabling with copper cables. Possible transmission protocols and the associated maximum transmission rates are defined at the same time that cable types are selected, which is why this is a significant decision which determines the future viability of the data center. Application Note Data Center Standards and Layouts v1.0 en Thomas Glaettli, Roland Diethelm 11