Ethernet: Convergence, Choices, Complexities By: Shawn Adams, PANDUIT Global Solutions Manager Introduction Ethernet is penetrating ever deeper into distributed control systems to provide real-time control with direct connectivity to a wider range of devices while the insatiable demand for speed (e.g. 10/40/100G) from commercial applications are driving continual advancements in Ethernet technologies. Terabit: 1 Trillion bits per second will be seen within the next decade. D'Ambrosia says even though 40/100G Ethernet products haven't arrived yet, he's already thinking ahead to Terabit Ethernet standards and products by 2015. "We are going to see a call for a higher speed much sooner than we saw the call for this generation" of 10/40/100G Ethernet By Jim Duffy, Network World, 04/20/2009 This expanded deployment of Ethernet in manufacturing has triggered convergence of disparate network technologies over a single Ethernet-based infrastructure, and this trend has left the controls engineer with new and more choices regarding how to specify and install the necessary network physical infrastructure. It becomes clear that network convergence continues unabated, thereby generating new and expanded network physical infrastructure options. PANDUIT s Unified Physical Infrastructure SM (UPI) based solutions are designed to help customers manage risk within the physical infrastructure by intelligently converging systems at the physical and logical levels (see Figure 1). This article discusses the ways that holistic UPI-based solutions enable distributed control systems to be used across manufacturing areas to help stakeholders capitalize on the business benefits of convergence. Figure 1. The network is separated into a physical and logical level to serve the needs of the control and enterprise levels. Converged Industrial Networks = More Infrastructure Choices Rockwell Automation indentifies three distinct levels of plant floor networking: Information Level, Control Level & Device Level. Manufacturing convergence currently is enabling controls engineers to deploy a single network technology and even a single network which addresses all three levels. That network technology is trending toward Ethernet. Page 1
Ethernet networks for enterprise environments are pervasive, reliable and extensively understood. However, when deploying the same technology across multiple manufacturing/cell zones, control panels and control rooms, the task can be daunting. Network stakeholders have a multitude of choices for connecting and managing these systems across the factory: High Speed Data Transport Systems: As Ethernet speeds continue to grow, media selection is critical for optimizing system performance. Media options now include Category 5e, Category 6, and Category 6A copper (unshielded or shielded) for shorter connections, and multimode and singlemode fiber for longer reach applications. Topology: Options have grown from a single bus/drop configuration to star, extended star, redundant star, ring, dual ring, mesh and partial mesh. Networking zones can be active or inactive, made possible by patch panels, bridges, switches (managed and unmanaged), and routers. Environment: Control engineers are additionally challenged with specifying an infrastructure that withstands the varying environmental demands across the factory floor (i.e., extreme temperatures, humidity, immunity to noise, exposure to fluid, intrinsic safety, shock and vibration, continuous motion and inter-building exposure). Maintenance: As Ethernet technology continues to penetrate the most basic layers of a control system, workers with hybrid skill sets are increasingly in demand to troubleshoot, maintain, and secure these converged industrial networks. Unlike other network assets, the physical infrastructure can transcend several product generations. For example Ethernet connectivity on PLCs can be traced back to the PLC-5E and now on the latest ControlLogix platform. This longevity reinforces the need to deploy a reliable, secure, and interoperable network physical infrastructure that ensures today s performance with the ability to support future technology demands. Applying UPI Principles for Reliable Converged Industrial Networks Effective integration of Ethernet into a manufacturing or processing facility can be challenging. Network stakeholders now need to factor variables such as interoperability, performance, reliability, and security into network infrastructure design and deployment as well as protection from the rigors of the manufacturing environment. Networks designed using Unified Physical Infrastructure SM (UPI) principles intelligently converge physical and logical systems to improve productivity, increase agility, and support sustainability initiatives (see Figure 2). Specifically, UPI-based solutions support the variety of innovative logical layout and design approaches in the industrial space for deployment of converged network architectures. Page 2
Figure 2. UPI-based solutions span all core systems necessary to run a business from data center and facilities operations to next-generation intelligent buildings and across the factory floor. Three core UPI principles for applying Ethernet in industrial applications are: 1. Use of Zone Cabling Topologies for distributing network cables, which enhances network flexibility and scalability to achieve greater operational efficiencies. 2. Apply Structured Environmental Analysis (SEA) to deploy a network physical infrastructure that mitigates the effects found in a manufacturing environment.. 3. Take a comprehensive Defense in Depth (DiD) approach to network security, which allows production and enterprise networks to safely and securely exchange data. Zone Cabling Topology Distributed and highly distributed control systems that are spread out over large geographical areas benefit from zone cabling topologies that consolidate network infrastructure elements closer to the devices they support. With this approach, distributed devices for manufacturing, enterprise networking, and building infrastructure may be efficiently supported from a strategically positioned zone cabling enclosure (also referred to as a network consolidation point). For example, a manufacturing zone that contains a control panel with multiple PACs, several drives, and an HMI may share the same zone cabling enclosure with a nearby shop office that contains PCs and printers connected to the enterprise network. By routing these devices through a single consolidation point, network engineers can eliminate multiple cabling home runs to reduce installation costs and increase network flexibility. In addition, building infrastructure systems such as security cameras, HVAC environmental controls, and power monitoring may be linked to this same enclosure in order to fully leverage the zone cabling backbone (see Figure 3). Page 3
Figure 3. A zone cabling topology enhances the flexibility and scalability of Industrial Ethernet networks to achieve greater design flexibility and operational efficiency. A zone cabling topology can help facilitate a network design that complies with Rockwell Automation and Cisco recommendations for segmenting networks in automation applications. However, without attention to additional core UPI principles to properly connect and secure the infrastructure, this approach can result in performance outages and security breaches. Structured Environmental Analysis Industrial environment conditions can vary greatly depending on the type of manufacturing, type of equipment, installation standards, and building construction -- so there are no hard rules about the level of protection required for each environmental zone. Another UPI core principle recommends that network stakeholders perform a Structured Environmental Analysis (SEA) of the environments in which their network infrastructure will be situated, and then identify cost effective cabling and connectivity options that will withstand these conditions while ensuring optimal network performance. To help stakeholders address environmental concerns, a subcommittee of the Telecommunications Industry Association (TIA) developed the Mechanical, Ingress Rating, Climatic, Electromagnetic (MICE) classification system. This system provides a structured method of categorizing the environmental classes to determine the level of hardening required for network infrastructure elements: media, connectors, pathways and enclosures (see Figure 4). In certain cases (e.g. final assembly areas) commercial grade components can be used. Page 4
Figure 4. An end-to-end channel solution often cuts across several MICE environments, ranging from environmentally controlled control rooms to more rugged settings. Defense in Depth (DiD) Approach for Effective Network Security Security is a vital network issue. Unauthorized access or damage to any facet of the network (devices, software or physical infrastructure) can cause disruptions of real time systems and can potentially introduce viruses which reduce network reliability. Logical and physical segmentation of production and enterprise networks allow the safe and secure sharing of data, services, and access from the production floor without creating additional risk to control systems. DiD is a multilayered approach for addressing network vulnerabilities. At the physical layer, the DiD approach integrates security features into infrastructure components to secure networks while maintaining effective segmentation at the physical level. A DiD-based physical security plan should include: Color-coded connectivity to help indicate different network functions Keyed solutions provide further depth to network physical security, as positive and negative keying features on jacks and plugs distinguish discrete networks and prevent the insertion of non-matching connectors that could compromise security (see Figure 5) Lock-in devices that can be used to secure dedicated connections, and blockout devices restrict access to selected ports Lockable server enclosures and lockable data access port with keyed or secured connections for heightened security Page 5
Figure 5. Color-coding on PANDUIT Keyed solutions visually distinguishes discrete networks to help prevent unauthorized connections across the physical infrastructure. Lock-In and Blockout devices further mitigate the risk of connectors becoming accidentally dislodged or otherwise compromised. Conclusion As Ethernet becomes more pervasive in the industrial automation space -- supplanting legacy networks, the need for a robust, manageable network continues. Unified Physical Infrastructure based solutions embody the next wave of systems integration and risk management by converging and harmonizing critical systems power, communication, computing, security, and control throughout the enterprise. This approach takes a broad, systems level view of Ethernet networks that spans from the office to the factory and within the building infrastructure. Strengthened by deep engagements with technology leaders including Rockwell Automation and Cisco, PANDUIT has developed solutions to design, install and maintain the physical infrastructure for the entire enterprise network. Our Physical Layer Reference Architecture, jointly developed with Rockwell Automation, includes clear definitions on best practices for specifying, installing, testing, and documenting the converged manufacturing network infrastructure. It also covers how enterprise IT and controls engineers can properly connect, manage, identify, and secure cabling throughout the physical infrastructure. Ultimately, a well-planned, UPI-driven infrastructure serves as a strategic asset to streamline operational efficiency and helps support manufacturing convergence. Page 6