Smart grid communication networks: The case for reusing telco infrastructures

Transcription

1 Smart grid communication networks: The case for reusing telco infrastructures Radovan Sernec, Telekom Slovenija, Ljupčo Jorgušeski, TNO, Netherlands Aleš Švigelj, Inštitut Jožef Stefan, Slovenia Jurij Jurše, Elektro Primorska, Slovenia Zhong Fan, Toshiba Reasearch Europe, UK 1 INTSIKT2014, SUNSEED Project Grant agreement number: Radovan SERNEC

2 Partners in smart grid arms (adapted from Dire Straits) 2 INTSIKT2014, SUNSEED Project Grant agreement number: Radovan SERNEC

3 Emotion driven design... 3 INTSIKT2014, SUNSEED Project Grant agreement number: Radovan SERNEC

4 Smart grid networks: energy and comms (NIST) 4 INTSIKT2014, SUNSEED Project Grant agreement number: Radovan SERNEC

5 Smart grid changing electricity landscape 1. Integration of distributed, renewable energy sources, improving efficiency via demand side management, e-car proliferation. New forms of organisation: microgrids, VPP, hybrid 2. Disruptive change in electricity distribution. Info: 1.2 M small PV producers in Germany in Changes in whole value chain. 4. Telecommunications are fundamental and vital part of smart grids. 5. Connect M of devices at end users and 100 K substations. Reliably! 6. efficiency value is deciding factor! ROI in utility sector years case for infrastructure reuse 7. Solutions: All by telco Utility by itself Mix of solutions services from both 5 INTSIKT2014, SUNSEED Project Grant agreement number: Radovan SERNEC

6 Energy utilities diverse needs (and views) 1. We want to control comms networks! All? physical layer? QoS? Management of network and data flows or only information from nodes? 2. Network resilience, fault tolarance, but: Low TCO Low per node ARPU 6 INTSIKT2014, SUNSEED Project Grant agreement number: Radovan SERNEC

7 Comms networks in utilities: Status 1. Local vs. Central Teleprotection, autonomous Substation management 2. Utilities are blind in distribution Substations from 0 to SCADA End users data sampling from 0 (homes) to 1 min (industry) GPRS connected smart meter generations behind SOTA in telecoms PLC, narrow band FR, ATM SDH, with resilient ring failover 4. Backend systems Non real time billing, dual tarif, 30 day period 5. ROI very long Low investment/year burden 7 INTSIKT2014, SUNSEED Project Grant agreement number: Radovan SERNEC

8 Comms nets: Issues from utility perspective 1. Low latencies for teleprotection (3-10 ms latency) 2. Vs. Financial markets low latency requirements in 100s seconds. for smart grid, too. Not doable in wireless (LTE) MPLS L2 Ethernet over fiber (aka FCoE) 3. Vertical only vs horizontal & vertical comms: Substations-substations DR-substation-management center 4. Self healing MV/LV networks necessary for mission critical Error bounds, QoS and teleprotection on PDH to IP defined by ITU (G.821, G.826, G.828, G.829, G.8201, Y.1540, Y.1541, Y.1560 and Y.1561) Fully separate systems, doubled Totaly different design and implementation reliability 5. Power autonomy h at substation and critical locations contrary to recent telco trend. 6. Information security by design must be designed in by telco! 8 INTSIKT2014, SUNSEED Project Grant agreement number: Radovan SERNEC

9 Smart grid: Latency requirements Problematic or potentially problematic Types L2 fiber marginally LTE no 9 INTSIKT2014, SUNSEED Project Grant agreement number: Radovan SERNEC

10 Comms nets: Issues from utility perspective 1. Low latencies for teleprotection (3-10 ms latency) 2. Vs. Financial markets low latency requirements in 100s seconds. for smart grid, too. Not doable in wireless (LTE) MPLS L2 Ethernet over fiber (aka FCoE) 3. Vertical only vs horizontal & vertical comms: Substations-substations DR-substation-management center 4. Self healing MV/LV networks necessary for mission critical Error bounds, QoS and teleprotection on PDH to IP defined by ITU (G.821, G.826, G.828, G.829, G.8201, Y.1540, Y.1541, Y.1560 and Y.1561) Fully separate systems, doubled Totaly different design and implementation reliability 5. Power autonomy h at substation and critical locations contrary to recent telco trend. 6. Information security by design must be designed in by telco! 10 INTSIKT2014, SUNSEED Project Grant agreement number: Radovan SERNEC

11 Some ideas for implementation: High level 1. Who-Where: Service vs. Control vs. Ownership End smart meter nodes Energy network control (substation automation) Energy network management Backend operations (e.g. data center, GIS, BSS) 2. Private vs. public comms network, but: 3. Let utility have complete control, visibility 4. Telcos to adapt extension of MVNO model 5. Do not reeinvent the wheel: Reuse freqency bands in your region (e.g. LTE EU) Fwd looking technologies (e.g. 4G 5G, SDN) TCO with known solutions Abundance of engineering talent 11 INTSIKT2014, SUNSEED Project Grant agreement number: Radovan SERNEC

12 Some ideas for implementation: Phy - App 1. Fiber laid with or inside energy cables 2. DWDM offers plenty of bandwidth and/or functional separation (e.g. per data traffic, management) 3. IP/MPLS backbone 4. Isolated data centers from outside data traffic 5. Smart meters direct connection to telecom operator 6. Share fiber DWDM with telecom operator 12 INTSIKT2014, SUNSEED Project Grant agreement number: Radovan SERNEC

13 Details on the wireless/cellular case Facts 1. < 4G cellular not suitable for mission critical implementations. 2. Bandwidth sharing not desirable. 4G LTE guarantees fixed bw per end node. 3. Target packet loss << 1 % 4G LTE Implementation hints 1. Coverage of MV/LV substation, from dual telecom operators 2. Not MVNO 3. Require separate physical infrastructure: base stations, backhauling, mobile center 4. Switch paths on: packet loss, QoS. 5. Smart grid end node (e.g. WAMS, PMU, SCADA) require dual SIM hw. 6. Allow for wireline access (3rd option) at critical locations: BPLC, xdsl, fiber. 13 INTSIKT2014, SUNSEED Project Grant agreement number: Radovan SERNEC

14 Why LTE: OFDMA t & f partitioning 14 INTSIKT2014, SUNSEED Project Grant agreement number: Radovan SERNEC

15 Details on the wireless/cellular case Facts 1. < 4G cellular not suitable for mission critical implementations. 2. Bandwidth sharing not desirable. 4G LTE guarantees fixed bw per end node. 3. Target packet loss << 1 % 4G LTE Implementation hints 1. Coverage of MV/LV substation, from dual telecom operators 2. Not MVNO 3. Require separate physical infrastructure: base stations, backhauling, mobile center 4. Switch paths on: packet loss, QoS. 5. Smart grid end node (e.g. WAMS, PMU, SCADA) require dual SIM hw. 6. Allow for wireline access (3rd option) at critical locations: BPLC, xdsl, fiber. 15 INTSIKT2014, SUNSEED Project Grant agreement number: Radovan SERNEC

16 Telecoms adaptation to smart grid comms 1. New SLA for smart grid Multi level, QoS based 4G LTE QCI M2M/IoT device types with bw guarantee Mission critical 2. Energy autonomy on comms network nodes Substation Teleprotection 3. New segments for smart grid telecom Mid bw nodes (e.g. WAMS, PMU) Low bw, high resilience, QoS (substation automation) 16 INTSIKT2014, SUNSEED Project Grant agreement number: Radovan SERNEC

17 Smart environment: Thinking beyond grid 1. Smart meter +? meters at home (water, gas, heat, electricity) Single access point 2. UK, Telefonica case for smart environment 53 M by 2020 Hybrid cellular + RF mesh ZigBee based Single point access backhaul to local mesh network Separate HLR for smart meter SIM cards and traffic Unified M2M platform 17 INTSIKT2014, SUNSEED Project Grant agreement number: Radovan SERNEC

18 SUNSEED: Main objectives DSO-telecom converged network for DEG smart grid guidelines: design deploy operate Large scale field trial, ~ 1000 nodes Develop advanced measurement & control sensor node WAMS for DEG smart grid Use intelligent analytics & visualisation tools for real time smart grid management 18 INTSIKT2014, SUNSEED Project Grant agreement number: Radovan SERNEC

19 Analogy: Smart grids comms higher aims 19 INTSIKT2014, SUNSEED Project Grant agreement number: Radovan SERNEC

20 Workflow: Design space exploration 20 INTSIKT2014, SUNSEED Project Grant agreement number: Radovan SERNEC

21 Use case driven design 1 1. State estimation and voltage profile formation Measurement of energy grid key parameters at nodes (U, I, P, ) Derivation of state and voltage grid graph in real time Automatic control and stability purposes Key comms requirements: high bw smart grid communication nodes (> 1 Mbps) of low density (< 100/cell sector). 2. Massive scale prosumer smart grid Renewable distributed energy sources (photovoltaics, wind, cogeneration) and storage (battery, fuel cell, e-car battery) Demand response and management on household level Key comms requirements: high density of smart grid communication nodes (> 1000/cell sector), each with moderate bandwidth requirement (< 100 kbps). 21 INTSIKT2014, SUNSEED Project Grant agreement number: Radovan SERNEC

22 Use case driven design 2 3. Fault identification and localization using external (non-grid) data sources Enhance reliability of power system service Identification and localization of power outages (faults) through monitoring of comms networks, components, CPE 4. Survivability of power distribution network Worst case scenarios based on massive natural disasters (e.g. floods, sleet, snow) with multiple faults leading to cascading failures of critical infrastructures. Establish robust and reliable wide area energy distribution network with self-healing abilities using real time monitor/control Key comms requirements: high availability through diverse communications paths, dynamic data rerouting over heterogeneous infrastructures (e.g. wireless, cellular, fiber, copper, PLC). 22 INTSIKT2014, SUNSEED Project Grant agreement number: Radovan SERNEC

23 Why survivability focus, too? 23 INTSIKT2014, SUNSEED Project Grant agreement number: Radovan SERNEC

24 Ultimate challenge: What to do when % of infrastructure is down? % of all MV/LV energy grid network is down? % of base stations are down? % of population is without electricity? 24 INTSIKT2014, SUNSEED Project Grant agreement number: Radovan SERNEC

25 Energy Comms networks: Overlap 25 INTSIKT2014, SUNSEED Project Grant agreement number: Radovan SERNEC

26 MPLS with SDN for smart grid 1. SDN towards MPLS: converged solution OpenFlow V1.3 standardised 26 INTSIKT2014, SUNSEED Project Grant agreement number: Radovan SERNEC

27 Thank You Radovan Sernec, Telekom Slovenija 27 INTSIKT2014, SUNSEED Project Grant agreement number: Radovan SERNEC

28 BACKUP SLIDES Radovan Sernec, Telekom Slovenija 28 INTSIKT2014, SUNSEED Project Grant agreement number: Radovan SERNEC

29 SUNSEED as an enabler of EU 2030 targets European parliament voted (5th February 2014) for three binding targets by 2030: % cut in greenhouse gases (compared to 1990) 2. > 30 % of energy to come from renewable sources % improvement in energy efficiency 29 INTSIKT2014, SUNSEED Project Grant agreement number: Radovan SERNEC

30 Convergence of comms networks 1 1. Present status and architectures Energy utilies: DSO, TSO Telecom operator 2. Information sources to converge (Smart) metering data Automatic control, SCADA WAMS (PMU-like) data Management & control global 30 INTSIKT2014, SUNSEED Project Grant agreement number: Radovan SERNEC

31 Convergence of comms networks 2 3. Means to converge OSI stack Phys Protocols Security policies Availability, survivability (e.g. Nx 9) Management & control 4. Constraints Energy grid specific limitations (e.g. IEEE 1646) Higher density of WAMS for network observability Mimc future high density prosumer smart grids 31 INTSIKT2014, SUNSEED Project Grant agreement number: Radovan SERNEC

32 Smart grid comms: Protocol stacks (Cisco) 32 INTSIKT2014, SUNSEED Project Grant agreement number: Radovan SERNEC

33 On failures and disruptions: Slo experienced ALL simultaneously! 1. Cascading failure: Disruption in one infrastructure causes a disruption in a second infrastructure 2. Escalating failure: Disruption in one infrastructure exacerbates an independent disruption of a second infrastructure (e.g., the time for recovery or restoration of an infrastructure increases, because another infrastructure is not available). 3. Common cause failure: Disruption of two or more infrastructures at the same time, because of a common cause (e.g. natural disaster). 33 INTSIKT2014, SUNSEED Project Grant agreement number: Radovan SERNEC

34 Network survivability: Preliminaries 1. A network is said to be survivable if all of the demands can be met under the failure of (m)any (one) of its links. 2. Given an undirected graph G = (N, E), where N is the set of nodes and E is the set of links, i.e., tuples of nodes, and demand of each node from every other node, the communication network design problem is to install integer multiples of a capacity unit on the links and route the flow of demands so that the total capacity installation cost is minimized. 34 INTSIKT2014, SUNSEED Project Grant agreement number: Radovan SERNEC

35 Network survivability: SOTA 1. HA or Self healing rings 3. Directed p cycle protection (pre configured) 35 INTSIKT2014, SUNSEED Project Grant agreement number: Radovan SERNEC

36 Network survivability for smart grids 1. Some directions for design DSO + TSO + telco nets: best from all Architecture Topology Integration of many phys (e.g. Eth, xdsl, PLC, WiFi, LTE) Dynamic autonomous decisions about active paths taken 36 INTSIKT2014, SUNSEED Project Grant agreement number: Radovan SERNEC

37 Complex systems interdependencies Can survive with automatic, deeper data analysis/ics 37 INTSIKT2014, SUNSEED Project Grant agreement number: Radovan SERNEC

38 Network survivability: KPI ideas 1. Throughput vs. density of WAMS 2. Delay vs. density of WAMS 3. Nodes survived vs all nodes (per each net level) 4. Scalability 5. Graceful degradation of performance, but still usable Delivering energy Transporting information 38 INTSIKT2014, SUNSEED Project Grant agreement number: Radovan SERNEC

39 Smart grid: BW & reliability requirements 39 INTSIKT2014, SUNSEED Project Grant agreement number: Radovan SERNEC

40 Methodology for smart grid data collection and evaluation 1. System Average Interruption Duration Index (SAIDI), extended SAIDI 2. Customer Average Interruption Duration Index (CAIDI), extended CAIDI 3. Momentary Average Interruption Frequency Index (MAIFI) 40 INTSIKT2014, SUNSEED Project Grant agreement number: Radovan SERNEC

41 Telco networks: MPLS is the norm 1. MPLS: today 41 INTSIKT2014, SUNSEED Project Grant agreement number: Radovan SERNEC

42 SDN potential 1. Software Defined Networks Open Net Foundation [ Commercial (e.g. Cisco, IBM, NEC) Experiemental [ low/sdn] 42 INTSIKT2014, SUNSEED Project Grant agreement number: Radovan SERNEC

43 Synchronisation: f, ph, time 43 INTSIKT2014, SUNSEED Project Grant agreement number: Radovan SERNEC

44 Time synchronisation: LTE Requirement 36: The NGMN Backhaul solution MUST support clock distribution to the e NB for frequency synchronization and SHOULD support phase/time alignment. Methods: Physical methods (e.g. Synchronous Ethernet) Long term stable oscillator (e.g. Rubidium) Protocol based methods (e.g. NTP, IEEE 1588v2) GPS 44 INTSIKT2014, SUNSEED Project Grant agreement number: Radovan SERNEC

45 Time synchronisation: Implications Allowed LTE phase jitter ±[1,5-5] s. LTE can tunnel NTPv4 to measurement nodes. Fiber tunneling non-issue. Precisely allign measurements within 20 ms period. RFC5905, NTPv4, IPv6 compliant Precise within a few tens of ms, poll intervals < 36 h 45 INTSIKT2014, SUNSEED Project Grant agreement number: Radovan SERNEC

46 KPI: Classical 1. System Average Interruption Duration Index (SAIDI), extended SAIDI 2. Customer Average Interruption Duration Index (CAIDI), extended CAIDI 3. Momentary Average Interruption Frequency Index (MAIFI) 46 INTSIKT2014, SUNSEED Project Grant agreement number: Radovan SERNEC

47 KPI: Details 1. Cumulative outage per customer Energy & comms networks building block, DER, WAMS node, home. 2. Momentary outage, < 1 min (MAIFI) 3. Extended versions Notation: C sections, N customers, outage duration, kth failure at section j D. Menasche, Survivability Analysis of Power Distribution in Smart Grids with Active and Reactive Power Modeling, INTSIKT2014, SUNSEED Project Grant agreement number: Radovan SERNEC

48 Key performance indicators 1. Comm. network availability (up/down, distribution by region), data traffic characteristics, service level parameters of WAMS nodes (density per base station/substation, time distribution, packet latency-jitter-loss-round trip) on {PLC, fibre, WiFi, GPRS, LTE}. 2. Reliability and availability of the existing vs. overlayed comms. solutions (resilience to node/link failures). 3. Energy network reliability parameters end2end on energy graph (S/CAIDI, MAIFI). 4. Energy flow measurements within energy grid graph of field trial setup to show losses in distribution network. 5. Investment cost & TCO, for smart grid comms network, Capex, Opex (design, maintenance, network operation, customer support). 48 INTSIKT2014, SUNSEED Project Grant agreement number: Radovan SERNEC

49 KPI: Beyond the obvious 1. Data traffic characteristics and service level parameters of WAMS nodes (e.g. density, density per mobile base station, density per substation region, time distribution, packet latency-jitter-loss-round trip). 2. Reliability and availability of the existing communication infrastructure for smart grid vs. converged solutions, with parameters (e.g. resilience to node and link failures, network attacks and jamming probability for communication link failure, data outage probability). 3. Energy flow measurements within energy grid graph of field trial setup, per vertex on graph, to show losses in distribution network. 4. Investment, operational, total costs for smart grid communication network (converged, DSO only, telecom only) and savings thereof. 5. Cost parameters (e.g. per bit per WAMS node, per bit per substation, per bit per DEG node) of communication network for smart grid (converged, DSO only, telecom only). 49 INTSIKT2014, SUNSEED Project Grant agreement number: Radovan SERNEC