Risk Implications of Digital RPS Operating Experience

Size: px

Start display at page:

Download "Risk Implications of Digital RPS Operating Experience"

Dulcie Stokes
6 years ago
Views:

1 Risk Implications of Digital RPS Operating Experience For Presentation at IAEA Technical Meeting on Common-Cause Cause Failures in Digital Instrumentation and Control Systems of Nuclear Power Plants June 19-21, 2007 Bethesda, Maryland, USA Dr. John H. Bickel 1

2 Motivations for this work: No prior risk or importance analysis of existing digital RPS failure experience exists Prior NRC Research reports concluded LER data too sparse to use Only found: 18 microprocessor failures, 4 software failures Suggested need to consider data from aerospace, medical, transport systems Lack of data implied: inability to risk-inform digital I&C applications and issues My belief: Much more data actually exists on CE CPCS Risks from CPCS experience should be assessed 2

3 CE Digital Core Protection Calculator Basics: CE High LPD, Low DNBR RPS design switched from analog Thermal Margin/Low Pressure Trip to digital Core Protection Calculators in mid 1970 s Used 6 specially qualified minicomputers running stored computer software and addressable constants CPCS performs static/dynamic projections of local power density and DNBR based upon: Ex-core neutron flux Pressurizer pressure Reactor Tcold, Thot RCP pump speed Control rod positions CPCS generates: alarms, pre-trip, and trip safety actions Original system was licensed on ANO-2 2 in 1978 Subsequently utilized at: SONGS-2/3, Waterford-3, Palo Verde-1/2/3. and Korean Standard NPPs 3

4 CE Digital Core Protection Calculator Basics: CPCS credited for reactor trip for following events: Uncontrolled Control Rod withdrawal from critical (>10-4 power) Uncontrolled Boron Dilution from critical (>10-4 power) Uncontrolled Control Rod withdrawal from power operation Dropped, or mis-positioned Control Rods Ejected Control Rods Single RCP loss of flow Single RCP shaft seizure 4-RCP loss of flow Electrical grid under-frequency Excess secondary steam flow (including turbine bypass valve malfunction) Excess feedwater flow Loss of feedwater heater Steam line break Single MSIV closure Rapid increase in local power 4

5 CE Digital CPCS Software Basics: Software Design: One Good Version not N-Version 5

CE Digital CPCS Interchannel Communications Basics: 4 CPC computers evaluate: LPD, DNBR using neutron flux, temperature, RCS flow and control rod position inputs in each quadrant 2 CEA

6 CE Digital CPCS Interchannel Communications Basics: 4 CPC computers evaluate: LPD, DNBR using neutron flux, temperature, RCS flow and control rod position inputs in each quadrant 2 CEA computers (CEACs) monitor all quadrants for CEA deviations within groups and generate Penalty Factors transmitted to all 4 CPCs CEACs communicate to CPCs via one-way simplex communication links 6

7 CE Reactor Protection System PRA Basics: PRA Assessments of overall CE RPS have existed for some time (2001) Component unavailabilities based on time averaged values NUREG/CR-5500 Vol.10: Q RPS = 7.2E-6 6 (Digital CPCS, w/o Operator Action) Q RPS = 1.6E-6 6 (Digital CPCS, w/ Operator Action) Relay and breaker CCF dominates predicted Q RPS : CCF of master trip relays (K-1 1 through K-4) K CCF of reactor trip breaker is not as significant on CE design due to configuration 7

8 How This Study was Carried Out: Failure experience from on-line NRC LER data base currently goes back to 1984 (NOTE: misses first 6 years ANO-2 2 experience) Post-1984 CPC LERs on CE plants were evaluated CPCS Failure experience categorized by subsystem Size of operating experience pool: 141 LERs ( ) ~145.5 Rx years (or: 1.27x10 6 Rx hr) 70 actual CPC reactor trip demands 26 events involving latent CCF (including: 1 latent software CCF) Subsystem failure rates calculated via Bayesian estimation using Jeffrey s non-informative prior CCDP risk estimated via ASP approach Method highlights CCDP impact of higher than average unavailability ility 8

component inventory per plant Total CPCS operating time

9 How Component Population Was Estimated: Total CPCS subsystem operating time estimation was based upon above component inventory per plant Total CPCS operating time (for 4/4 Channel CCF estimation) was simply s total plant operating time. 9

How Subsystem Operating Time Was Estimated Each of 4 CPC Computers and 2 CEAC Computers contain: 1 processor board, 1 memory board, 1 multiplexer board, 1

10 How Subsystem Operating Time Was Estimated Each of 4 CPC Computers and 2 CEAC Computers contain: 1 processor board, 1 memory board, 1 multiplexer board, 1 external Watchdog Timer Each of 4 CPC Channels contains: 1 PZR pressure sensor, 3 ex-core neutron flux inputs, 4 RCP speed sensors, 2 Tcold and 2 Thot inputs 10

11 Subsystem failure rates were calculated via Bayesian estimation using Jeffrey s non-informative prior Technique allows bounding failure rate estimation for 0 observed ed failures 11

12 Failure Rate and Unavailability Estimation Issues Data Needs for Risk Estimation Process: Ability to estimate CCDP given specific event demands and event-conditional system unavailabilities (such as RPS) Includes conditional unavailability due to specific combinations of input conditions to digital system Certain software bugs only triggered by unusual input sets Overall RPS unavailability must consider combinations of random and CCF events Operating experience estimates failure rates: λ Conversion to RPS unavailability uses estimate of time to detect and restore: P = λ x (fault duration) In many cases for latent Digital CCFs fault durations are many months 12

13 Actual Design Basis CPCS Trip Demands 13

14 Estimated CPCS Single Subsystem Failure Rates 14

15 CPCS Single Subsystem Failure Rates Also important to note: Failure modes of recent regulatory concern which have not occurred in population exposure time Recall failure rates can be estimated as: λ ~ 0.5/T Faults propagated via inter-channel communication: 2 events noted involving loss of CPCS -> > Plant Computer communications link that resulted in failure to perform Tech. Spec. required cross oss-checks, λ = 2.5 / ( 6 x 1.27x10 6 hours) = 3.3 x10-7 /hr Other events in which communication link failure occurred without operation impairment likely occurred but not reported in LER data a base Events involving a failure propagating to CPC or CEAC would be in LER data base if they occurred 0 events noted in which a communication link failure caused corruption to CPC or CEAC channel, λ ~ 0.5 / ( 6 x 1.27x10 6 hours), or: ~ 6.6 x10-8 /hr 15

16 Estimated CPCS Double Event Failure Rates 16

17 Estimated CPCS System CCF Failure Rates 17

18 Results: CPCS System CCF Failure Rates Breakdown of Common Mode Failures 4% 4%4% 4% 4% 8% 11% 8% 4% 8% Computer Technicians insert Wrong Data Sets to all 4 CPCS Channels Reactor Vendor supplies Erroneous Data Sets input to all 4 CPCS Channels Reactor Vendor Supplies Software Update Containing Latent Software Error Operators Fail to Confirm ASI in all four CPCS Channels when Reactor Power > 20% Incorrect Acceptance Criteria Used for Excore Data Set Calibration Checks >80% Inaccurate Cross Calibration of Excore Data Sets (Cross Channel, COLSS, etc.) High Log Power Bypass Removal Setpoints (1E-4) Incorrect 11% 26% 4% Inaccurate Cross Calibration of RCS Flow Data Sets (Cross Channel, COLSS, etc.) Operators Fail to Perform 12hr Auto-RESTART Surveillance on all CPCS Channels Operators Fail to Perform Refueling Interval Surveillance on all CPCS Channels Communication Data Link Failure to Plant Computer results in Missed Surveillances on both CEAC Channels 2 of 2 CEACs Inoperable 3 of 4 CPCS Neutron Flux Cross Channel Calibrations OOT The issue of latent software CCF represents only 4% of the CCF experience Calibration, generating, loading of incorrect data sets are the dominant sources of CCF 18

19 Types of Observed CCF Events: Inaccurate cross-calibration calibration of all Ex-core neutron flux (7 events) or all RCS flow channels (2 events) Computer technicians insert wrong addressable constant data sets into all 4 CPCS channels (3 events) Swapping addressable data sets between units CE supplies erroneous data sets (2 events) Software update provided to plant with incorrect logic for processing of indicated failed sensors (1 event) 19

20 Risk significance of this failure experience? None of actual CCF events resulted in core damage (all were latent faults missing triggering event ) Need to consider CCDP implications of specific failure modes Intent: apply risk screening process similar to NRC ASP program which focuses on higher than average values of system unavailability Use: ASP-type failure rate data, SPAR plant specific risk models, actual observed unavailability CCDP = Σ λ i x P CPCS-CCF CCF x HEP NR P CPCS-CCF CCF = λ CPCS-CCF CCF x (duration of latent fault) First: How sensitive is CCDP to RPS Logic CCF? 20

21 How sensitive is CCDP to RPS Logic CCF? RPS failure considers: Mechanical CCF jamming of control rods Relay/Breaker CCF failure RPS Logic CCF Operators fail to manually trip Operators fail to trip MG sets Loss of Offsite Power generates reactor trip without RPS Sensitivity studies conducted using NRC SPAR PRA models 21

22 How sensitive is CCDP to RPS Logic CCF? Variations in RPS-LOGIC LOGIC-CCF CCF are not risk significant until > 1x

23 Some example risk assessments of actual Digital CCF events 23

24 1995 SONGS Addressable Data Swapped Rod shadowing constants (on data disks) were swapped between adjacent SONGS units for 10,968 hours. Units at different power and burnup history, rod shadowing corrections thus different. Rod shadowing constants only impact power density predictions when control rods dropped, or partially inserted. P CPCS-CCF = 2.75x10-6 /hr x 10,968 hr = 3.0 x10-2 Summing over all initiating events involving dropped control rods and rod cycling tests, yields: CCDP < 0.488/yr x 3.0 x10-2 x 0.01 = 1.5 x 10-4 This represents bounding conservative estimate because better knowledge of duty cycle of rod cycling tests would likely reduce by factor of 10 or more. 24

25 1984 Erroneous F x,y factors supplied by CE and uploaded to SONGS-2 Incorrect F x,y factors generated by CE and used for CPCS LPD calculations from to (1,032 hrs). Events such as this have occurred twice. P CPCS-CCF = 1.96x10-6 /hr x 1,032 hr = 2.0 x10-3 CCDP = 0.488/yr x 2.0 x10-3 x 0.01 = 1.5 x

26 2005 Software Design Error in Software Upgrade at Palo Verde 2 for 2,736 hrs. Original software design: Trip CPC channel if sensor detected to be Failed Out of Range Software hardware upgrade: Use inputs from two sets of instruments and multiplexers (primary y and secondary) Out of Range Sensor Failure: Primary detected sensor failure results in switchover to secondary. Out of Range Failure on secondary reverts to last stored good value v CCF of all sensors of one type could result in continuous use of last good value in all 4 CPCS channels rather than TRIP. P CPCS-CCF = 8 x P Sensor-CCF x 2.75x10-6 /hr x 2,736 hr =8 x 8.4 x10-4 x 2.75x10-6 /hr x 2,736 hr = 5.0 x10-5 Given CCF of instruments, no credit for operators, HEP=1.0 CCDP = 0.289/yr x 5.0 x10-5 x 1.0 = 1.44 x

27 PRPS-CCF values from single events span many decades Fault duration times drive PRPS-CCF values Latent data uploading errors are dominant unavailability contributors Data uploading errors larger than relay and breaker CCF found in NUREG/CR-5500 (which used time-averaged values) 27

28 Event specific CCDP also dominated by data uploading errors Latent software CCF event is smaller due to unlikelihood of triggering condition. 28

29 Observations from this Total Picture of RPS : Designers of Digital I&C not particularly surprised by relative dominance of: Calibration problems and human errors uploading wrong data sets CCF due to errors by vendor in generating data sets These failure modes also existed in NPPs with Analog I&C CCF Unavailability and event CCDP estimates from operating experience are dominated by latent events with very long fault duration intervals Software-related CCF, while important, isn t dominant CCF source when actual operating experience is evaluated Likely because: software V&V processes more rigorous than operational controls after deployment at NPP Most-obvious software bugs generally caught by burn-in testing and qualification programs Software bugs triggered by highly unlikely input combinations are not key sources of RPS unavailability or CCDP risk 29

30 What is Concluded from all this? To assess Digital I&C risk it s necessary to view Total Picture of RPS not just software or : microprocessors : Final trip relays and trip breakers - will still be there Problems cross calibrating nuclear with thermal - will still be there Human errors inputting set-points and coefficients - will still be there When this is done - Total Picture of RPS risk emerges NPPs with CPCS have been operating since 1978 in typical, controlled, nuclear operations environment, which includes: Vendor generation of cycle specific constants, set-points Routine hardware, software upgrades developed and installed Routine operation, trouble alarms, and alarm response Impact of Technical Specifications, Testing, Calibrations Actual nuclear field reliability experience is better source of data than non-nuclear nuclear sources or theoretical models Ability to estimate, or bound risks of specific Digital I&C CCF failure modes thus: clearly exists 30

Common Technical Issue On I&C

Common Technical Issue On I&C Chashma Unit-1 and 2 Pakistan by M.Ramzan Tahir IAEA Technical Meeting on Instrumentation and Control in SMRs At VIC Vienna, Austria (21 ~ 24 May, 2013) Contents Plants Design