State of Reliability Report 2013

State of Reliability Report 2013 Jessica Bian, Director of Performance Analysis Reliability Assessment and Performance Analysis (RAPA), NERC Risk Issues Steering Committee Meeting, July 11-12, 2013

State of Reliability Report NERC s annual, independent assessment of reliability performance Provides an integrated view of reliability performance The report assesses 18 metrics that address the characteristics of an adequate level of reliability Guidance that enables industry and policy makers to take effective steps and make sound decisions supporting reliability The Report is a step forward and it delivers analytical information which can be used for Project Prioritization Process Improvement Characterization and Identification of Risk Development of coordinated and multifunctional solutions 2

Metrics and Performance Analysis History 2008 - Reliability Metrics Working Group formed, reporting to NERC Operating Committee (OC) and Planning Committee (PC) 2009 9 reliability metrics approved by OC and PC, details at: http://www.nerc.com/pa/rapa/pa/performance%20analysis%20dl/rmw G_Metric_Report-09-08-09.pdf 2010 Another 9 metrics approved by OC and PC, first annual performance report published: http://www.nerc.com/pa/rapa/pa/performance%20analysis%20dl/rmw G_AnnualReport6.1.pdf 2011 Second reliability performance report published: http://www.nerc.com/pa/rapa/pa/performance%20analysis%20dl/2011_ RARPR_FINAL.pdf 3

Performance Analysis History (continued) 2012 First annual state of reliability report 2013 Second state of reliability report New in 2013 Advanced risk identification methods Consistent and predicable pattern Correlation between initiating factors and transmission outage events Used in other industries (NASA, NRC, etc.) extremely effective 4

2013 State of Reliability Report Prepared by NERC with guidance from the Performance Analysis Subcommittee (PAS) Significant Contributions from: Operating Committee (OC) o Resources Subcommittee (RS) o Frequency Working Group (FWG) o Event Analysis Subcommittee (EAS) o Operating Reliability Subcommittee (ORS) Planning Committee (PC) o Reliability Assessment Subcommittee (RAS) o System Protection and Control Subcommittee (SPCS) o Transmission Availability Data System Working Group (TADSWG) o Generating Availability Data System Working Group (GADSWG) o Demand Response Availability Data System Working Group (DADSWG) Compliance and Certification Committee (CCC) 5

Events BPS Reliability Remains Adequate Bulk power system reliability stable within ALR conditions For 2008 to 2012, no significant upward or downward trends Consistent 97% AC circuit availability 16 Bulk Power System Transmission Related Events Resulting in Load Loss 14 12 10 8 6 4 2 0 14 14 12 12 11 9 9 9 8 6 2 2002 2003 2004 2005 2006 2007 2008 2009 2010 2011 2012 Year 6

BPS Reliability Remains Adequate NERC Annual Daily Severity Risk Index (SRI) Sorted Descending with Historic Benchmark Days 7

Highest 8 Stress Days (2008-2012) NERC Annual Daily Severity Risk Index (SRI) Sorted Descending with Historic Benchmark Days 8

KCMI (%) BAL CIP COM EOP FAC INT IRO MOD NUC PER PRC TOP TPL VAR Key Compliance Monitoring Indicator Trends Improving NERC Confirmed Violations by Assessed Risk (June 18, 2007 December 31, 2012) Assessed Risk ALL Minimal 80 1806 56 242 515 26 64 34 1 99 853 193 125 268 4362 85.3% Moderate 2 410 13 27 24 1 4 11 116 34 3 27 672 13.1% Serious 31 2 3 6 2 3 23 7 4 81 1.6% Total 82 2247 71 272 544 27 71 34 1 113 992 234 128 299 5115 100 95 90 85 80 75 70 65 60 55 50 Key Compliance Monitoring Index Trend (2008-2012) Time Period 9

(MW/0.1Hz) Frequency Response (FR) Stable Consistent data collection/procedures implemented in 2011 Further analysis for EI FR events with less than 1,500 MW/0.1Hz Eastern Interconnection 44 49 70 ERCOT Interconnection 51 67 65 10

Misoperations Significant Risk Issue Protection System Misoperations are a Significant Contributor to Disturbance Events and Automatic Transmission Outage Severity Largest positive correlation with 2012 automatic transmission outage severity Statistically significant Highest relative risk Industry Actionable Steps Recommended 65% from three leading causes Summarized in Protection System Misoperations Task Force (PSMTF) report Performance Measure Implemented 11

90 101 89 125 143 148 141 152 165 147 131 161 214 201 187 229 Misoperations Count 247 254 277 272 306 Misoperations Trend 350 Misoperations by Month (2011Q2-2012Q4) 300 250 200 150 100 50 0 Jan Feb Mar Apr May Jun Jul Aug Sep Oct Nov Dec 2011 Month 2012 12

Misoperation Rate (# Misoperations/# Operations) Misoperation Rate by Regional Entity (2012Q4) Misoperation Rate by Regional Entity in 2012Q4 18% 16% 14% 12% 10% 8% 16.8% 6% 11.7% 13.9% 4% 9.6% 9.7% 7.2% 2% 0 FRCC MRO NPCC* RFC SERC SPP TRE WECC* Regional Entity *NPCC and WECC Misoperation Rate Not Included. 13

Protection System Misoperations Task Force (PSMTF) Analysis Reviewed historical (pre-2011) Regional Entity misoperations Analyzed 1,500+ misoperations from January 1, 2011 to April 1, 2012 submitted in NERC misoperations template Focused on top 3 misoperation causes: Incorrect settings/logic/design Relay failure/malfunction Communication failures Classified misoperations in top 3 misoperation causes by sub causes Based on the analysis of top issues, developed suggestions for improvement to reduce top misoperation areas Recommended data monitoring and data improvements 14

Misoperations by Cause Code NERC-wide Misoperations by Cause Code (Jan 1, 2011 April 1, 2012) Communication failures, 396, 17% AC system, 193, 8% As-left personnel error, 201, 9% DC system, 112, 5% Relay failures / malfunctions, 465, 20% Unknown / unexplainable, 273, 12% Incorrect setting / logic / design errors, 628, 28% Other, 11, 1% 15

Suggestions for Improvement Proper Application of Relay Elements Settings Errors Microprocessor-based Relay Firmware Application of Power Line Carrier Communication Aided Protection Performance Monitoring and Data Improvements 16

Settings Errors Misoperations due to setting errors can potentially be reduced. Techniques that could be used to reduce the application of incorrect settings include: Peer reviews Increased training More extensive fault studies Standard templates for setting standard schemes using complex relays Periodic review of existing settings when there is a change in system topography The IEEE Power System Relaying Subcommittee, IEEE PSRC, published a working group report to provide additional technical guidance for protective relay setting quality control. http://www.pes-psrc.org/reports/processes_issues_trends_and_quality_control_of_relay_settings.pdf 17

Risk Patterns/Event Analysis NERC Event Analysis 2012 o 116 Category 1-5 Events o 42 Events (36%) with misoperations ALR1-4 - Transmission outage events led to firm load loss Only Two events in 2012 o Oct 15, 2012, 665 MW firm load loss due to relay misoperations o Dec 6, 2012, 300 MW+ firm load loss due to SPS/RAS misoperation Preliminary 2013 Event Analysis as of 4/30/2013 33 events, 10 with misoperations 18

NERC/Industry Actions One of High Priority Risk Issues from RISC 15+ Lessons Learned distributed since 2010 PSMTF Report Approved by PC in April 8, 2013 Developed targeted, actionable solutions for registered entities to reduce misoperations Enhance data reporting, including subcause codes NERC Industry Webinar May 7, 2013 Highlighted three identified primary causes Suggested registered entities to apply the solutions based on their particular circumstances. 19

NERC/Industry Actions Training Modules for Each Primary Cause Target to complete by July 2013 Provide further assistance to the entities that need guidance the most Share industry example of excellences On-Going Performance Monitoring (ALR4-1) PC directed its Protection System and Control Subcommittee (SPCS) at the June 11-12, 2013 meeting o Enhance reporting accuracy, including subcause codes for the three primary causes o Reduce percentage of misoperation reporting that does not have sufficient information Misoperation rate data became available first time for the 2012Q4 reporting period NERC Protection System Misoperations Dashboard http://www.nerc.com/pa/rapa/ri/pages/protectionsystemmisoperations.aspx 20

AC Substation Equipment Failure Second Significant Risk Issue AC Substation Equipment Failures are a Second Significant Contributor to Disturbance Events and Automatic Transmission Outage Severity Second largest positive correlation with 2012 automatic transmission outage severity Statistically significant Second highest relative risk Recommendations Form a technical group Investigate root causes Performance measure implemented 21

NERC/Industry Actions PC approved the formation of the AC Substation Equipment Task Force at the June 11-12, 2013 meeting Conduct a thorough investigation into the root causes of failed AC substation equipment failure that contribute to disturbance events Provide risk control solutions to improve performance Final report by June 2014 o Endorsement from OC o Approval from PC Collaborate with EAS/TWG and PAS/TADSWG Avoid duplication regarding related work by the TWGS and TADSWG 22

Risk Pattern/Event Analysis NERC Event Analysis 2011Q2 2012Q3Q02 o 121 Category 1-5 Events o 25 Events (21%) with AC substation equipment failures 26 AC Substation Equipment Failures in 25 Events Reactor 4% Transformer 15% Circuit Breaker 77% Disconnect Switch 4% 23

Transmission Events with Unknown Cause Warrant Analysis 19% automatic transmission outage events with Unknown initiating cause 10% common/dependent mode outages 34% sustained outages with Unknown Recommendations Form a technical group Enhance reporting clarifications and consistency PC directed TADSWG to address the issue at the June 11-12, 2013 meeting 24

Q/A Questions and Answers Additional background material is located at: State of Reliability Report 2013: http://www.nerc.com/pa/rapa/pa/performance%20analysis%20dl/2013_sor _May%2015.pdf NERC Reliability Indicators: http://www.nerc.com/pa/rapa/pages/reliabilityindicators.aspx Approved Metrics: http://www.nerc.com/comm/pc/pages/performance%20analysis%20subcommi ttee%20(pas)/approved-metrics.aspx Contacts Jessica Bian, 404-446-9702 Jessica.Bian@nerc.net Naved Khan, 404-446-9730 Naved.Khan@nerc.net 25

Background Materials - NERC 18 Reliability Performance Metrics

Objective Provide a high-level overview of the 2008-2012 reliability metrics and trends Analysis of reliability performance and trends will help develop quantitative risk control measures to better prioritize and inform reliability management business decisions. 27

Background NERC Planning Committee (PC) endorsed the formation of Reliability Metric Working Group at its December 2007 meeting Now known as Performance Analysis Subcommittee A slate of metrics which can yield an overall assessment of the North American Bulk Power System reliability Identify and track key reliability indicators as a means of benchmarking reliability performance Within the context of the Adequate Level of Reliability (ALR) framework Adequate Level of Reliability describes the fundamental concepts and six characteristics needed to achieve an adequate level of reliability Identify indication of potential reliability concerns 28

Metrics Number of Metrics Approved - 18 ALR * Characteristics Boundary Contingencies Integrity Protection Restoration Adequacy ALR1-3 ALR6-1 ALR1-5 ALR1-12 ALR1-4 ALR2-4 ALR2-5 ALR3-5 ALR4-1 ALR2-3 ALR5-1 ALR6-11 ALR6-12 ALR6-13 ALR6-14 ALR6-15 ALR6-16 ALR6-2 ALR6-3 * PAS is in the process of realigning current metrics with new ALR definition 29

Data Sources ES&D OE-417 Event Reports RCIS TADS Metrics Development DATA reporting Template 30

Metric Trend Rating ALR Boundary Trend Rating 1-5 System Voltage Performance 1-12 Interconnection Frequency Response ** Contingencies 1-4 BPS Transmission Related Events Resulting in Loss of Load 2-4 Average Percent Non-Recovery Disturbance Control Standard Events 2-5 Disturbance Control Events Greater than Most Severe Single Contingency * Integrity 3-5 Interconnected Reliability Operating Limit/ System Operating Limit (IROL/SOL) Exceedances Protection 2-3 Activation of Underfrequency Load Shedding 4-1 Automatic Transmission Outages Caused by Failed Protection System Equipment Adequacy 1-3 Planning Reserve Margin * 6-1 Transmission Constraint Mitigation 6-2 Energy Emergency Alert 3 (EEA3) 6-3 Energy Emergency Alert 2 (EEA2) 6-11 Automatic AC Transmission Outages Initiated by Failed Protection System Equipment 6-12 Automatic AC Transmission Outages Initiated by Human Error 6-13 Automatic AC Transmission Outages Initiated by Failed AC Substation Equipment 6-14 Automatic AC Transmission Outages Initiated by Failed AC Circuit Equipment 6-15 Element Availability Percentage 6-16 Transmission System Unavailability * ** Significant Improvement Slight Improvement No Change Trend Rating Symbols Inconclusive/Mixed * Slight Deterioration Significant Deterioration New Data 31 **

Improving Trend ALR Title 1-4 BPS Transmission Related Events Resulting in Loss of Load 2-4 Average Percent Non-Recovery Disturbance Control Standard Events 32

Events Total Annual Load Loss (MW) ALR1-4 Positive Trend Indication 70,000 60,000 64,850 ALR1 4 BPS Transmission-Related Events Resulting in Load Loss (2002 2012) Colors represent individual events. 50,000 40,000 30,000 15 20,000 10,000 0 8,942 11,045 10,212 7,085 4,950 3,763 2,249 4,432 4,078 1,055 2002 2003 2004 2005 2006 2007 Year 2008 2009 2010 2011 2012 Bulk Power System Transmission Related Events Resulting in Load Loss 10 5 0 33 14 14 12 11 12 8 9 9 9 6 2 2002 2003 2004 2005 2006 2007 2008 2009 2010 2011 2012 Year

Percentage ALR2-4 Positive Trend Indication ALR2-4 : Average Percent Non-Recovery of DCS Events (2007-2012) 4 3.5 3 2.5 2 1.5 1 0.5 0 Quarter TRE SERC NPCC FRCC FRCC MRO NPCC RFC SERC SPP TRE WECC 34

No Clear Trends/Mixed Trend ALR Title 2-5 Disturbance Control Events Greater than Most Severe Single Contingency 3-5 Interconnected Reliability Operating Limit/ System Operating Limit (IROL/SOL) Exceedances 1-3 Planning Reserve Margin 35

Count ALR2-5 Not Clear/Mixed ALR2-5 : Disturbance Control Events Greater Than Most Severe Single Contingency (2007-2012) 7 6 5 4 3 2 1 0 TRE SERC NPCC FRCC FRCC MRO NPCC RFC SERC SPP TRE WECC Quarter 36

ALR3-5 IROL/SOL Exceedance Not Clear/Mixed 37

ALR1-3: Planning Reserve Margin 38

No Change ALR Title 1-12 Interconnection Frequency Response 2-3 Activation of Under frequency Load Shedding 6-1 Transmission Constraint Mitigation 6-2 Energy Emergency Alert 3 (EEA3) 6-3 Energy Emergency Alert 2 (EEA2) 6-11 Automatic AC Transmission Outages Initiated by Failed Protection System Equipment 6-12 Automatic AC Transmission Outages Initiated by Human Error 6-13 Automatic AC Transmission Outages Initiated by Failed AC Substation Equipment 6-14 Automatic AC Transmission Outages Initiated by Failed AC Circuit Equipment 6-15 Element Availability Percentage 6-16 Transmission System Unavailability 39

No Data/New Data ALR Title 1-5 System Voltage Performance 4-1 Automatic Transmission Outages Caused by Failed Protection System Equipment 40

NERC Metrics Dashboard http://www.nerc.com/pa/rapa/pages/reliabilityindicators.aspx 41