Finding bugs in the LHC: Verification methods for PLC programs

Transcription

1

2 Dániel Darvas (CERN) Finding bugs in the LHC: Verification methods for PLC programs Alpine Verification Meeting 18 21/09/2017, Visegrád Contains joint work with B. Fernández, E. Blanco, Gy. Sallai, I. Majzik, R. Speroni, M. Lettrich

3 CERN

4 Programmable Logic Controllers Crucial parts of industrial control systems PLCs: robust industrial computers for control tasks Special domain Special programming languages Different background knowledge Smaller user base Siemens AG 2014, All rights reserved

5 Special and numerous PLC languages

6 We need good quality! PLCs at CERN Critical and/or expensive systems are operated Cryogenics Vacuum Gas Detector control Better quality Higher availability Reasonable effort development, training Photos: CERN

7 We are working with prototypes anyways Not everything can be anticipated Slide of E. Bravin and S. Redaelli, CERN Daily LHC operation meeting, May 2016 Slide of B. Auchmann et al., CERN 6 th Evian Workshop, 2015

8 State of the art: Verification of PLC programs

9 Options for PLC program verification Subjective list of possible verification methods: Static code analysis (code smells, ) Testing (module tests, integration tests, ) Formal verification (model checking)

10 State of the practice Static analysis Lack of static analysis tools Old development environments Some commercial tools (e.g. PLC Checker) High FP ratio, many missed problems

11 State of the practice Testing Practically the only verification method used ISA/ANSI/IEC 62381: FAT: Factory Acceptance Test SAT: Site Acceptance Test Typically no module testing, no specific tool support (Module testing required for SIL1+ safety systems)

12 State of the practice Formal verification

13 State of the practice Formal verification

14 PLCverif: New verification methods for PLCs at CERN

15 Goals (originally)

16 What do we want? Formal verification! How do we want it? Without much effort Without understanding model checkers Without writing CTL/LTL formulae Without using command line interface Without reading the manual Quickly

17 Goals Lightweight verification Keep it simple, stupid and usable Flexible tool Model checking, testing, static analysis Reusable language infrastructure Lowering the entry barrier Reduced development effort Integrating external tools

18 Overall idea Lightweight verification Flexible tool Reusable lang. infrastructure Reduced development effort PLC code Formal model External verification tools Verification report Requirements

19 Overall idea Lightweight verification Flexible tool Reusable lang. infrastructure Reduced development effort Static analysis PLC code Formal model External verification tools Verification report Requirements Visualization

20 Overall idea Lightweight verification Flexible tool Reusable lang. infrastructure Reduced development effort Static analysis PLC code Formal model External verification tools Verification report Requirements Visualization

21 Overall idea Lightweight verification Flexible tool Reusable lang. infrastructure Reduced development effort STL code Static analysis SCL PLC code nuxmv Req. pattern Formal model UPPAAL External verification CBMC tools Verification report Requirements Assertions Reduction JUnit Visualization

22 PLCverif features Testing Helps unit and regression testing PLC code Formal verification (CBMC) Input scenario (CSV) C code (simulation) Actual output (CSV) Expected output (CSV) Report Work done together with Gyula Sallai

23 PLCverif features PLC code Requirement patterns Fixed English sentences with gaps to fill Control Flow Automata Reduction Support for multiple model checkers ITS-GAL representation nuxmv representation Making verification feasible Formal verification Report Human-readable verification report Fully automated workflow Read more: D. Darvas et al. PLCverif: A tool to verify PLC programs based on model checking techniques. ICALEPCS doi: /JACoW-ICALEPCS2015-WEPGF092

24 Example results

25 Results Static analysis Experimental phase, very preliminary results Issues found in our well-established library Read, but not written variables Incorrect logic expressions

26 Results Automated unit testing using Jenkins for our base object library Formal verification with CBMC, nuxmv, too Testing

27 Based on a still from Willy Wonka & the Chocolate Factory Paramount Pictures Results Testing

28 Results Model checking is more and more accepted and used at CERN Of course, performance may still be a problem Environment modelling is a challenge too Formal verification Use cases Library of base blocks Deeply hidden problems found in code used for 10+ years in production, in 200+ PLCs Various magnet testing safety controllers Several safety issues found ITER collaboration Ongoing verification of a critical communication protocol s implementation for their fusion reactor Read more: D. Darvas et al. Formal verification of safety PLC based control software. ifm doi: / _32

29 Results tooling Formal verification SCL code and verification case editor One-click verification Multiple model checkers under hood Verification report Read more: D. Darvas et al. PLCverif: A tool to verify PLC programs based on model checking techniques. ICALEPCS doi: /JACoW-ICALEPCS2015-WEPGF092

30 Future work What is next? More development of PLCverif (KT-funded project) Goal: Making the tool production-ready More stable, more generic, more open Integrating automated unit testing in the real development workflow Analysis of static analysis rules

31 Conclusion Static analysis: preliminary work, but very promising Testing: dedicated support for automated unit testing Model checking: often feasible, requiring acceptable resources and knowledge Big impact on PLC verification by introducing lightweight verification methods Industrial application of model checking is interesting, desired and feasible Important to specifically target usage domains

32

33 Get the presentation!

34 Model checking at CERN D. Darvas et al. Formal verification of complex properties on PLC programs. Formal Techniques for Distributed Objects, Components, and Systems (LNCS 8461), pp , Springer, B. Fernández et al. Bringing automated model checking to PLC program development A CERN case study. Proc. of the 12th Int. Workshop on Discrete Event Systems, pp , D. Darvas et al. PLCverif: A tool to verify PLC programs based on model checking techniques. Proc. of the 15th Int. Conf. on Accelerator & Large Experimental Physics Control Systems, pp , JaCoW, B. Fernández et al. Applying model checking to industrial-sized PLC programs. IEEE Transactions on Industrial Informatics, 11(6): , D. Darvas et al. Formal verification of safety PLC based control software. Integrated Formal Methods (LNCS 9681), pp , Springer,