Sharing Direct Fiber Channels Between Protection and Enterprise Applications Using Wavelength Division Multiplexing

Transcription

1 Sharing Direct Fiber Channels Between Protection and Enterprise Applications Using Wavelength Division Multiplexing Jonathan Sykes, Dewey Day, and Kevin Fennelly Pacific Gas and Electric Company Veselin Skendzic and Normann Fischer Schweitzer Engineering Laboratories, Inc PG&E and SEL

2 Pacific Gas & Electric Investor-owned utility 70,000 square-mile service territory 5.4 million electric customers 4.3 million gas customers

3 PG&E Motivations Constant work to improve system reliability Openness to latest technological advancements Careful selection process Communications as a key part of every evaluation Communications technology laboratory in San Ramon, California

4 PG&E Communications Network Company-wide Optical Transport Network (OTN) Substations serving as network nodes 61 nodes in service North-side extension with additional 10 nodes Central management Extra capacity available

5 Communications-Based Protection Protection assumes limited communications and is optimized for legacy communications technologies 64 kbps digital voice channels are still popular Higher-speed packet-based communications slowly gaining hold Direction of Current Relay 1 Current In = Current Out Relay 2

6 New World TW87 Dedicated Fiber Communications 3 phase voltages and 3 phase currents 145 Mbps stream payload Sampling at 1 MHz with 18 bit resolution 1 Gbps Ethernet physical layer Relay Relay Port 5 Port 6 TW87 Dedicated Port 6 Port 5 Point-to-Point Fiber Engineering Access Communications Network Communications Network Fiber Engineering Access Communications Network

7 TW87 Link Requirements 1 Gbps Ethernet physical layer Industry-standard SFP module-based fiber interface Low latency (<3 ms) Low frame jitter (<25 ns) No link asymmetry (<100 ns) No third-party traffic Constant link delay (no protected path switching)

8 Point-to-Point Communications Link Options Wavelength division multiplexing (different light colors) is a great way to share a fiber Transport Technology Single-mode fiber pair CWDM DWDM OTN Acceptability Yes Yes Yes Yes SONET No Jitter Ethernet No Jitter MPLS No Jitter SDN-based Ethernet Sometimes Comment Difficult to justify for sole use by protection Affected by device construction and network setup Amplitude Wavelength

9 Wavelength Division Multiplexing (WDM) Multiplexer (optical filters) Demultiplexer (optical filters) TX l 1 l 1 RX TX l 2 Fiber l 1, l 2,...l n l 2 RX TX l n l n RX

10 Fiber Capacity Today Capacity is affected by distance Optical amplifiers are used extensively Capacity (Tbps) Modulation 4 to 7 PM-BPSK 5,000 8 to 15 PM-QPSK 3, to 21 PM-8QAM 1, to 27 PM-16QAM 500 Typical Distance (km) Fiber can carry many optical wavelengths (88+) Advanced modulation schemes are required Capacity and distance records are broken daily

11 Conceptual Structure of a QPSK Transmitter

12 Conceptual Structure of a QPSK Receiver Matched Filter f 1 (t) Decision Device QPSK Signal Sampling Time Interval (T S ) Multiplexer Binary Bitstream Matched Filter f 2 (t) Decision Device

13 Example: MAREA Submarine Cable Laid in September ,000 miles long (Virginia Beach to Bilbao, Spain) Designed with 160 Tbps capacity (8 fiber pairs, 20 Tbps per fiber) Cofinanced by Facebook, Microsoft, and Telxius Operational by end of 2018

14 Optical Transport Network Physical layer (Layer 1) optical transport Very high speeds (1, 10, 100 Gbps, and above) Many wavelengths (88+) Diverse tributaries (SONET, MPLS, and so on) Integrated optical plane (DWDM, amplification, reconfigurable add-drop, wavelength management) Scalability

15 OTN Multiplexing High speed helps multiplex many tributaries Flexible bit mapping preserves each tributary Tributaries never see each other Signal ODU0 ODU1 ODU2 ODU3 ODU4 ODUflex Approximate Data Rate Gbps Gbps Gbps Gbps Gbps Any configured rate

16 Dual Redundant OTN System Hardware

17 OTN Multiplexer Latency Forward error correction (FEC) increases latency FEC level is user selectable Direct optical system access (dedicated wavelength) has no additional latency Line-Side FEC Tributary Rate (GHz) Line Rate (GHz) Latency (µs) None RS FEC EFEC EFEC

18 TW87 Link Test at PG&E Test Goals Verify TW87 link compatibility with PG&E network Determine best interface method Introduce OTN capabilities to protection engineers

19 Test Scenarios Alien wavelength test Uses dedicated wavelength Is potentially wasteful (>100 Gbps resource) ODU2 tributary test Is multiplexed with SONET, MPLS, and other services Uses more modest 1 Gbps resource to transport 145 Mbps

20 Alien Wavelength Test DWDM Relay SFP Port 5 Port 6 Engineering Access Operational Data Network TW87 Port Fiber TW87 Port Fiber DWDM SFP Relay Port 6 Port 5 Engineering Access Operational Data Network SONET System 1 Gbps 1 Gbps SONET System Other Business Systems 1 Gbps 1 Gbps Other Business Systems 2 Gbps 2.4 Gpbs MVAC Card (alien wavelength) MVAC Card (alien wavelength) 2.4 Gpbs 2 Gbps 11DPM12 Card - ODU2 Mapping (tributary multiplexing onto a 10 Gbps channel) 11DMP12 Card - ODU2 Mapping (tributary multiplexing onto a 10 Gbps channel) Other Services 10 Gbps 10 Gbps Other Services Tunable WDM Laser and Modulator Tunable WDM Laser and Modulator Tunable WDM Laser and Modulator Tunable WDM Laser and Modulator Tunable WDM Laser and Modulator Tunable WDM Laser and Modulator OTN/ROADM DWDM Core Optical Network (multiple wavelengths on the same fiber) OTN/ROADM OTN/ROADM

21 ODU2 Tributary Test Engineering Access Relay Port 5 Port 6 Operational Data Network TW87 Port Fiber TW87 Port Fiber Relay Port 6 Port 5 Engineering Access Operational Data Network SONET System 1 Gbps 1 Gbps SONET System Other Business Systems 1 Gbps 1 Gbps Other Business Systems 2 Gbps 2.4 Gpbs 2.4 Gpbs 2 Gbps 11DPM12 Card - ODU2 Mapping (tributary multiplexing onto a 10 Gbps channel) 11DMP12 Card - ODU2 Mapping (tributary multiplexing onto a 10 Gbps channel) Other Services 10 Gbps 10 Gbps Other Services Tunable WDM Laser and Modulator Tunable WDM Laser and Modulator Tunable WDM Laser and Modulator Tunable WDM Laser and Modulator Tunable WDM Laser and Modulator Tunable WDM Laser and Modulator OTN/ROADM DWDM Core Optical Network (multiple wavelengths on the same fiber) OTN/ROADM OTN/ROADM

22 Test Results Latency is in line with error correction level used Jitter is below relay measurement limit Test Error Correction Measured Latency (µs) Measured Jitter (ns) Alien wavelength None 1.6 < 8 ODU2 EFEC 60.8 < 8

23 Relay Design Opportunities Communications are omnipresent Digital communications are developing very fast Fiber capacity exceeds protection system needs by 5 9 orders of magnitude

24 Relay Design Opportunities What is the best way to use network communications? How should we build the next generation of relays to embrace network communications?

25 Relay Design Challenges Guaranteed network-wide time synchronization (IEEE 1588 PTP as a wide-area network service) Guaranteed frame delivery (quality of service) Maximum latency guarantees Mature network management tools Cybersecurity

26 Questions?