DGLR-Workshop 11.Oktober 2006

Transcription

1 Zertifizierung von generierter Software für f Kernkraftwerke J. Märtz ISTec GmbH, Forschungsgelaende, D Garching

2 Assessment Report BE-SECBS Benchmark Exercise of Safety Evaluation of Computer Based Systems Project-Consortium: - Industrial Partner providing the Reference-System (MADTEB-System) Assessment-Teams: - IRSN (Frankreich) - ISTec (BRD) - VTT/STUK (Finnland) Project-Coordinator: - JRC-IE 2

3 Assessment Object Description MADTEB System Part of the reactor limitation system to limit the allowed range of process variables (coolant pressure, pressurizer level) of the primary coolant loop of the reactor Consisting of 8 limitation functions implemented in 4 redundant trains Implemented in TELEPERM XS (TXS) Technology Implementation comprising Data-Acquisition, Limitation-Functions, Priority-Functions, Output- Functions for all 4 trains consisting of about 300 Function Diagrams of TXS 3

4 TXS BASIC STRUCTURE Specification based on a graphical user interface (SPACE) I&C functions (functional diagrams FD) are constructed from prefabricated, normed basic elements (function blocks FB) and are organized as groups (functional diagram groups FDG) which are executed cyclically on single processing units. The graphical specification is stored in a database. Code generation is based only on database tables, and uses and includes only prefabricated FB modules and declaration files. Function Diagram 4

5 ASSESSMENT OBJECT RELEVANT PROPERTIES Properties of Test Case due to Implementation in TXS-Technology: Strict formal unambiguous character already in the first specification steps by use of Function Diagrams Function Diagrams can be automatically checked for consistence. Resulting C-code is so called Normed Source Code with some predefined properties Strictly cyclic and data independent execution of the code within a fixed time interval Strictly linear control flow structure, i. e. the execution path ( and executiontiming) is predefined and not dependent on input data Resulting C-code is based on a code-library, the so called Function-Blocks. Function-Blocks are type-tested software components that allow to rely on their compliance with the specification of their functional properties in a data sheet. 5

6 BASIS FOR ASSESSMENT Standards: IEC KTA 3503 ( Type-Testing of electrical modules of the reactor protection system ) National Regulatory Rule Type-Test: Pre-developed components (Function-Blocks, RTE) are tested application-independent Subsequent applications of these components can refer to this Type- Test and do not need to assess these components again. 6

7 ASSESSMENT ACTIVITIES Adaptation of the Assessment Methodology to the Test Object being Normed Code (TXS) Assessment Steps performed, according to Life-Cycle-Steps of IEC and Development Documentation: Requirement-Spec System Specification Detail Design Coding / RETRANS-Analysis Testing 7

8 ADAPTATION OF ASSESSMENT METHODOLOGY Impact of TXS-Properties (Normed Code) on Assessment Methodology: No need for static analysis (e.g. LDRA-Testbed, CANTATA) of the generated code in order to identify complexity of code structure, control flow paths or some other properties supporting the assessment activities, as it is the case with conventional (not normed) software No need for dynamic analysis for measuring of test-coverage, because the test-coverage is predefined by the structure of the automatically generated code. There is only one main path that is executed in each cycle. Amount of Testing determined by functional aspects 8

9 OVERALL STRUCTURE OF ASSESSMENT ACTIVITIES Basic Documents: Development documents Function-Diagrams in database Assessment-Method Desk inspection of the respective documents Analysis of the functionality of the Function-Diagrams in the database by Application of SPACE-Tool for analysis of the Function-Diagrams (navigating through the system / tracing the signal-paths) Application of RETRANS (developed at ISTec) checking the functional equivalence of automatically generated source code with its underlying specification. Assessment-Output / LOPs Basic Framework for Assessment-Steps Document containing a List of Open Points (LOPs) LOPs handed out for audit and clarification to the developer Paper with comments on the LOPs sent back to Assessor 9

10 List of Open Points / LOPs Problems addressed in the LOPs Inconsistencies between different Specification Levels especially between descriptions and Function Diagrams Completeness of documentation Unambigousness of descriptions and of technical details Main Benefits of the LOPs Identification of errors insufficiencies and weak points of the product improving safety and quality of product. Identification of inconsistencies and insufficiencies of the documentation initiating revisions and updating of documentation renders a consistent and complete status of documentation 10

11 FINDINGS OF THE ASSESSMENT-STEPS STEPS Majority of the LOPs referring to changes and modifications not explicitly documented or explained, during translation of Process Requirements into FD-Templates Identification of a Fault by the LOP-Procedure System Specification vs. Requirement-Spec Check of a consistent and correct translation of the I&C functions description of the Process Requirements, into Prototype Function-Diagrams Assessor-Question: Scrutinising the opening- and closing-behaviour of a specific injection valve in various plant-status-transitions the assessor demands further explanations within his LOPs. Developer-Answer: Developer approves that for the operating of this injection valve a fault would be introduced due to an inappropriate aspect in the Requirement Specification. Reason for this fault: The validation of the Requirement-Spec. by process-engineering was out of the scope of this benchmark test-case! For real systems the Requirement Spec. is validated by process-engineering. 11

12 FINDINGS OF THE ASSESSMENT-STEPS STEPS Detail-Design vs. System-Spec Majority of LOPs referring to undocumented modifications between Template Function Diagrams and final implementation. Also one not intended modification was identified. Redefinition of logic during Detail Design (shifting of tasks between different Function- Diagrams) Modifications due to Results of Simulation-Tool during design of Function-Diagrams Priority Logic Assessment identified insufficient implementation of Priority Logic within the Software of TXS Function-Diagrams According to developer-response priority is implemented by an external logic downstream TXS outputs Inconsistent Modification Not intended inconsistency introduced while creating the Function-Diagrams from FD-Template by inhomogeneous usage of Flip-Flop-types No impact on functional behaviour of the integrated system (RCP-Speed). 12

13 RETRANS ANALYSIS TOOL-STRUCTURE FDG - Module FD - Module1 FD - Modulen ANSI-C source codes reverse transformation FDG - Module FD - Module FD - Module 1 n SPACEreverse transformed information comparison deviations, analysisprotocol data base 13

14 RETRANS ANALYSIS ANALYSIS RESULTS RETRANS Essential results of the software analysis tool: Automatic comparison of the graphic specification of the application programs stored in a database with the functionality of the automatically generated C-Source Code. Hints for the analyser with respect to the plausibility of FB parameters in redundant channels. Hints concerning inconsistencies in the database resp. in the C-Source Code (e.g. signalling, parameterisation) 14

15 RESULTS OF CODE-ANAYSIS BY RETRANS RETRANS-Analysis Comparison and plausibility check for the whole C-Code of Benchmark- Testcase with the content of corresponding BE-SECBS database About 300 FDs, about 1000 pages of Diagrams. Results and Findings The C-Code of the Benchmark-Test-Object is in accordance with the function diagrams in the related SPACE data base. Some non-uniform sequentialisations, yet without impact on correct functionality RETRANS plausibility check identified minor deficiencies of the benchmark test-case: deviations within extensions of some block-numbering incorrect explanatory text for some signal identifiers. Both deficiencies don t violate the correctness of code, yet they have to be avoided within real safety applications. The plausibility control in real applications yields more findings which finally are errors. 15

16 ASSESSMENT-STEP STEP TESTING Testing The amount of testing required for the assessment of this TXS-Code (Normed Code) only depending on the functionality of the system under test independent of any code-coverage-measuring. Basic testing strategy performed by the developer (Functional Tests) refer to the basic functionality of the system specific properties of the functionality interface to its environment concerning I&C and process system behaviour under failure condition for relevant ranges and combinations of input signals Comprehensiveness and sufficiency of Testing: Validation of process engineering aspects was not foreseen and not performed within this benchmark-exercise. 16

17 SUMMARY of BENCHMARK- -EXERCISE at ISTec Summary Normed Code, which is automatically generated, has an essential impact on the type of the required assessment activities. Assessment method applied in the benchmark test-case provides the following important benefits for the V&V process: Identification of inconsistencies and insufficiencies within and between the various development documents Complete and comprehensive documentation representing the actual status of development Tracing of all the modifications implemented during the development process Assessment as a diverse and supplementary method to testing is capable to find errors, weak points or insufficiencies that pass the software development process. 17

18 CONCLUSION Normed-Code-Systems support the Assessment Activities and thus contribute to SAFETY Formal method (Function Diagrams) for the description of the system from the very beginning of the development Approved components (Type-Testing) Approved structure and properties (Type Testing) Concentration of assessment on application-specific aspects COST-EFFECTIVENESS Some Assessment-Tasks are performed once for all by Type-Testing Reduction of aspects to check for an individual implementation Checks can be reasonably automated (RETRANS) due to the Normed Structure of the Code. 18