IP/Ethernet Drivers and Landscape: Smart Grid in Iberdrola NETWORK BUSINESS SPAIN Control Systems and Telecommunications Miguel. A. Sánchez Fornié Alberto Sendín July 2013
Table of Contents Iberdrola Communication Network and Architecture in Iberdrola Evolution towards Ethernet MPLS as Extended LAN basis 2
Index Iberdrola 3
Iberdrola http://www.iberdrola.com, May 2013, August 2011 4
Iberdrola USA UK Coordinated strategies and architectures across Iberdrola Group Brazil Spain 5
Index Communication Network and Architecture in Iberdrola 6
Communications in Iberdrola. Infrastructure = opportunity Utilities rely on assets that makes them different from many other companies when facing Telecommunication challenges. Poles and ducts for fiber optic cables Poles for Base Stations Carriers carrier model in place Substations as nodal premises Radio station sites 7
Communications in Iberdrola. Mission Critical Telecommunication Services are fundamental for the Operational activities of the company. However, not all telecommunications are well suited for Operational activities. Operational activities have more demanding requirements than Corporate ones: A network that is prepared for operational services is ready for corporate use (but not necessarily the other way around!) Not all operational communication services have the same criticallity: Data is not the same as voice. Cost of the solution is biased by initial investment. Not all data related communications in operational areas have the same relevance and requirements: Remote control is critical to improve electricity grid availability. Remote metering of certain points of supply is not time critical. 8
Communications in Iberdrola. Network Perspective Type Substations (Primary ST&STR) Quantity Thousands (1,100, IBD, Spain) Distribution Substations (Surface&Pole mounted) Tens of Thousands (36,000 + 26,000, IBD, Spain) Distribution Substations (Underground) Tens of Thousands (26,000, IBD, Spain) Points of Supply Millions (10,800,000, IBD, Spain) 9
Communications in Iberdrola. Architecture Telecommunications Network BACKBONE - Hundreds - Ethernet ACCESS NETWORK - Thousands - Ethernet/IP Secondary Substation PRIME -Millions -Not IP (narrowband) LV The Smart Grid 10
Communications in Iberdrola. Challenges And how does the Smart Grid affect us in terms of telecoms? Pervasive reach: due to the distributed nature of electricity assets. Bandwidth: depending on the services needed (i.e. Smart Grids), bandwidth will be different. Two way comms. and interactivity: the communications have to be from the system to the devices, and vice versa. Scalability and flexibility: subject to changing demands and concepts, solutions must be flexible. Reliability, Security and Safety: the critical nature of the assets need such an approach. Electricity utilities own assets that configure a privileged situation for the provision of telecommunication services However 11
Communications in Iberdrola. Concerns Which elements condition telecom technologies planning and programming: Who is going to pay for the Remote Management? Who is going to pay for the Smart Grid? Which regulatory changes will affect the evolution of Smart Grid? Technologies almost fine-tuned. Integration is the biggest concern. Large amounts of devices from manufacturers. Changes are going to take place in all parts of the network, and day-today activities will be modified. Changes in processes. Deployment works will be taking place in a non-greenfield environment. Economic deviations must be closely controlled. 12
Communications in Iberdrola. Integration approach Efficiency ( ) is the key when selecting the proper solution Head End Private, Fixed / Mobile Network Private, Fixed / Mobile Network SS PLC: PRIME MPLS/SDH/FO PLC: MV (BPL) IP free Public Network PLC: PRIME Public Network PUBLIC NETWORK Public Network S S SS PLC: PRIME Fixed/Mobile Public Network SS 13
Communications in Iberdrola. IP everywhere? IP (Internet Protocol) plays a fundamental role in Communications Systems, due to its wide adoption. However, there are some communication environments (narrowband PLC channels, narrowband VHF&UHF radio channels) where IP might not be the best solution: Because channels are usually narrowband: IP was born in a broadband world. Because its overhead both in terms of payload, and in terms of message exchange, cannot be supported and at the same time provide the best performance. Because IP protocol might not be providing any added value (just cost). In these cases, end to end IP solutions are not probably the most cost efficient solutions. Communication Systems can provide gateways and/or adaptation layers to external networks using IP: There is no need to use IP internally in some systems as adaptation layers can get interoperability. Narrowband systems can avoid the unnecessary burden of the broadband-engineered IP protocol message exchange. 14
Communications in Iberdrola. Access Technologies Telecommunications Network MV BPL ACCESS NETWORK Eth / IP Based 15
Communications in Iberdrola. Access Techn: BPL vs. Public Radio L2 (BPL) L3 (GPRS) 16
Communications in a Utility. Technology Comparison BPL vs. GPRS comparison: GPRS BPL (x 10, x 100) Throughput 10 kbps 20 kbps 100 kbps 2 000 kbps Latency 400 700 ms 80 150 ms Performance example: Firmware Update (FW). Data Concentrator: FW image = 1,5 MB; Minimum throughput GPRS = 20 min.; BPL = 2 min. Maximum throughput GPRS = 10 min.; BPL = 6 s. Data Concentrator web page: Web page = 0,05 MB; Minimum throughput GPRS = 40 s.; BPL = 4 s. Telecontrol status update (with data) : Minimum throughput / Maximum Latency GPRS = 4,88 s.; BPL = 0,81 s. Maximum throughput / Minimum Latency GPRS = 2,64 s.; BPL = 0,33 s. 17
METERING & DSM NETWORK IT & APPLICATIONS Communications in Iberdrola. Increasing Data Volumes Data exchange Mobile Wf Scada Planning Maintenance Meter Managemnt Points of Supply Retailers TSO Datawarehouse Serv Comm. Serv Comm. Telecommunication Network Gbps Telecomm. Secondary Substation Mbps RTU IP Camara Sensors Actuators Mbps kbps Home Area Network Detail PLC 18
Index Evolution towards Ethernet 19
Evolution towards Ethernet. Transport Backbone technology evolution: Transport PDH: 34 Mbps SDH: STM 16 (2.5 Gbps) STM 64 (10 Gbps) DWDM: 40 λ (1.6 Tbps) λ =40, 10, 2.5, 1 Gbps 20
Number of services Number of services Evolution towards Ethernet. Transport Circuit Evolution PDH A huge investment to take into consideration: Ethernet transport 1000 900 800 700 Number of provisioned services per year 2005 600 500 400 300 200 100 0 1999 2000 2001 2002 2003 2004 2005 2006 2007 2008 2009 2010 2011 2012 Total 520 107 115 191 516 532 912 344 335 219 202 112 104 98 SDH 600 500 Number of provisioned services per year 2008 400 300 TYPES OF SERVICES TELECONTROL 200 100 0 2002 2003 2004 2005 2006 2007 2008 2009 2010 2011 2012 External 31 120 178 205 137 167 57 61 42 44 16 Internal 21 37 53 107 120 235 434 369 339 302 363 TELEPHONE TELEPROTECTION DWDM 30 25 Number of services per year 2010 OTHERS, non critical services 20 15 10 5 0 2008 2009 2010 2011 2012 External 0 18 20 18 0 Internal 6 6 4 8 4 21
Evolution towards Ethernet. Protocols Protocol towers Fibre Fibre Fibre Fibre Fibre Fibre Time Level 2 Level 1 Fibre 22
Evolution towards Ethernet. Switching Backbone technology evolution: Switching 15 years 6 years Point to point: 34 Mbps Frame Relay/ATM: 155 Mbps MPLS: 622 Mbps, 1/2/10 Gbps 23
Evolution towards Ethernet. Extended Local Area Networks VPLS Services 24
Evolution towards Ethernet. Management MPLS Network Management: Transport-like system 25
Number of VPLS and Access Evolution towards Ethernet. MPLS Services Layer 2 based MPLS core. Types of service offered: VPLS (Virtual Private LAN Service) Ethernet Multipoint Wide Area Network. VLL (Virtual Leased Line): Ethernet point to point circuit using MPLS network. Number of Access Points and VPLS 1400 1200 1000 800 600 400 200 0 2004 2005 2006 2007 2008 2009 2010 2011 2012 VPLS Services 12 2 35 28 40 42 90 81 395 VPLS Access Point 46 19 124 485 904 1062 949 1279 988 26
Index MPLS as Extended LANs basis 27
MPLS and Extended LANs IBERDROLA MPLS NETWORK 28
MPLS and Extended LANs. Keep it simple Direct Ethernet connectivity: Fiber Optics and Ethernet over SDH or DWDM. Ethernet extension from the core to the border: EthoSDH, and optical switches. Iberdrola matches VLANs with VPLS (administrative purposes): each VLAN is a service (VPLS). Simplification MPLS allows QoS. Iberdrola supports QoS: Quality to specific Services. Mapping with DSCP (IP field) and Quality (internal customers). Single device type: spares, training, Just Ethernet services (no PDH, RS232 over Eth encapsulation, ). L2 MPLS provides the provision simplicity we need. 29
MPLS in Iberdrola. Control Systems Main and Backup Systems Criteria: Infrastructure for Redundancy 30
MPLS in Iberdrola. Data Centers Corporate Systems Data Center Connectivity Substation Connectivity Generation Control Centers Connectivity Renewable Premises Connectivity 31
Number of devices MPLS in Iberdrola. MPLS to the Border? Switching and IP network: Switches: (LANs are straight-forward to provision) Routers: (Smart Grid derived concentrations: scalability reasons) 4000 3500 Switches and Routers Switch Providers NSN ZIV Ruggedcom 3000 2500 2000 24% 36% 40% 1500 1000 Router Providers 500 0 Switches Routers Teldat 1452 Ruggedcom 764 0 ZIV 1146 2178 NSN 1251 ZIV 40% Teldat 60% 32
MPLS in Iberdrola. And what next? EthoSDH,oTDM & L2MPLS Transprt & Reuse & Simple Oper Network Evolution Transport & L2 Transport & Fiber Optic Network Evolution 33
Conclusions Iberdrola takes profit from carriers carrier model activities: (1) see outside the box & (2) develop private infrastructure. MPLS has been the technology that, together with transport capabilities, has allowed Iberdrola to provide a simple implementation of Ethernet service provision anywhere: Iberdrola soon realized that PDH platforms, even if appropriate for low speed services, would not allow to take benefit from mass market prices. This fact drove us to install SDH, and soon NGN-SDH (Eth). Iberdrola soon realized that Switching platforms would allow to simplify operational models and service provision: low number of premises, but widely spread. There is a big question mark on the evolution of MPLS networks together with the evolution of Transport Systems. Smart Grids are changing the status quo with tens of thousands of premises, and millions of smart meters. Cost is a critical concern when we need to change the scope from hundreds, to thousands of premises. 34