Formal Verification of Flight Control Applications along a Model- Based Development Process A Case Study

Transcription

1 Formal Verification of Flight Control Applications along a Model- ased Development Process A Case Study München, 5 th of October, 2016 Hochstrasser Markus, Hornauer Markus, Holzapfel Florian Examples provided by: Christoph Krause, Simon Schatz, Agnes Gabrys

2 Introduction / Objectives Goal: Evaluation of the capabilities of formal method tools provided by MathWorks R2016b applied on latest FSD s modular flight control algorithms and the benefits regarding certification Presentation of the results of a case study on formal verification, focusing on 3 different applications Providing hands-on experience in the use of MathWorks Simulink Design Verifier and Polyspace Code Prover Discussion of risks and benefits when proposing formal methods in an aerospace certification workflow

3 Agenda Formal Verification Results A. Inner Loop System model verification by Model Checking (Design Error Detection with Simulink Design Verifier). System Automation Design Model verification by Model Checking (Property Proving with Simulink Design Verifier) Source Code verification by Model Checking + Abstract Interpretation (Polyspace Code Prover) System Level Software Level C. Trajectory Controller Source Code verification by Model Checking + Abstract Interpretation (Polyspace Code Prover) Formal Methods and Certification Aspects System Model vs. Software Design Model Tool Qualification considerations enefits and risks when applying formal verification activities / tools

4 Modular Flight Control Algorithm Architecture C Model Checking + Abstract Interpretation for Code Analysis Model Checking with A embedded properties HMI (Mode Control Panel) External Guidance System Medium Level Medium Level High Level Input Processing System Automation Trajectory Generation ATOL Auto Flight System Vertical/Lateral Path, Trajectory Speed Command Trafo ody Axis, Load Factor, ank Angle and Rate Commands Inner Loop Controller ody Axis Load Factor ank Angle ATHR Thrust Control Output Processing Control Surface Commands Throttle Lever Commands Model Checking with user-defined properties Model Checking + Abstract Interpretation for Code Analysis

5 Agenda Formal Verification Results a Case Study A. Inner Loop System model verification by model checking (Design Error Detection with Simulink Design Verifier) System Level. System Automation Design Model verification by Model Checking (Property Proving with Simulink Design Verifier) Source Code verification by MC + Abstract Interpretation (Polyspace Code Prover) Software Level C. Trajectory Controller Source Code verification by MC + Abstract Interpretation (Polyspace Code Prover) Formal Methods and Certification Aspects System Model vs. Software Design Model Tool Qualification considerations enefits and risks when applying formal verification activities / tools A SYSTEM Inner Loop Model Checking (Design Error Detection)

6 Case Study A: Inner Loop aseline Controller Innerloop Structure Turn Compensation aseline Controller Maneuver portion + Inertia Coupling portion + Control Allocation Notch Filter Plant + Actuators Roll- off Filter Aero Trim portion - Sensors System Model not for code generation, but algorithm development Characteristics: Trigonometric calculations, LookUp Tables Metrics collected with Simulink Metrics Tool Metric Value SL lock Count 1239 SF Chart Count 0 Subsys Count 152 Hierarchy Depth 5 Models 1 A SYSTEM Inner Loop Model Checking (Design Error Detection)

7 SLDV Design Error Detection (DED) / Model Checking Model Checking expands the state space of an automata and searches for violations of properties SLDV DED uses embedded properties Static Run-Time Error Detection Division by Zero Integer overflow Array index out of bounds Dead Logic / Active Logic Design range checking SLDV applies different abstractions / approximations Floating point numbers are converted to rational numbers Stubbing of nonlinear functions Linear interpolation of 2-D Lookup Tables A SYSTEM Inner Loop Model Checking (Design Error Detection)

8 Design Error Detection for Inner Loop (1st Iteration) Run-Time Error Detection proved 146 objectives valid 1 objective undecided Dead Logic 621 objectives valid 39 dead logic Input limitation [ ] ignored due to stubbing of trigonometric functions A SYSTEM Inner Loop Model Checking (Design Error Detection)

9 Dead Logic Detection for Inner Loop (1st run) 39/660 objectives are identified as Dead Logic Library Functions FSD Common Library blocks have own robustness design (e.g. the protected division) These blocks are inlined library subsystems Solution: Create block replacement Protected division from FSD Common Library lock replacement for protected division library block lock replacement rule setting A SYSTEM Inner Loop Model Checking (Design Error Detection)

10 Dead Logic Detection for Inner Loop (2nd run) 9/596 objectives are identified as Dead Logic 2 issues are to due missing parameter range Parameter ranges for SLDV must be specified separately in the configuration (Simulink.Parameter Min/Max is not considered) 7 are real Dead Logic due to over-protection in the same model A SYSTEM Inner Loop Model Checking (Design Error Detection)

11 Lessons Learned (Design Error Detection) Advantages Fast run-time error identification Identification of dead logic early in design Run-time error detection are low-hanging fruits limited effort to get the results Application Challenges Inconvenient parameter range specification lock replacement for library functions required ( compare to concepts of Code Prover with main-generator) Approximations A SYSTEM Inner Loop Model Checking (Design Error Detection)

12 Agenda Formal Verification Results a Case Study A. Inner Loop System model verification by model checking (Design Error Detection with Simulink Design Verifier) System Level. System Automation Design Model verification by Model Checking (Property Proving with Simulink Design Verifier) Source Code verification by MC + Abstract Interpretation (Polyspace Code Prover) Software Level C. Trajectory Controller Source Code verification by MC + Abstract Interpretation (Polyspace Code Prover) Formal Methods and Certification Aspects System Model vs. Software Design Model Tool Qualification considerations enefits and risks when applying formal verification activities / tools System Automation Property Proving

13 Case Study : System Automation STANDY XIT OPERATIONAL Controls the operational modes of the flight control system dependent on sensor information, data link availability and commands from the flight operator 1 Low Level Control Ground Parking External Pilot (EP) Flight Operator (FO) Low Level Control Airborne Medium Level Control Metric Value SL lock Count 415 SF Chart Count 15 AUTO TAKEOFF AND LANDING Takeoff Takeoff Abort Landing Go Around High Level Control Return to ase GPS HDG Subsys Count 89 Hierarchy Depth 10 Ground External Pilot Link Loss Airborne Models 6 Characteristics: Mealy state machines, bus copying, enums 1 Krause, C., and Holzapfel, F., Designing a System Automation for a novel UAV Demonstrator (submitted for publication), Proc. of 14th International Conference on Control, Automation, Robotics and Vision, Flight Operator Link Loss Ground ank HDG GPS Level 1 Level 2 Level 3 Level 4 System Automation Property Proving

14 Design Model Development Rules All Design Models are developed according to and verified against strict Modeling Guidelines including MathWorks High-Integrity Guidelines Subset of MAA Guidelines Subset of Code Generation Guidelines FSD-specific restrictions, e.g. single rate, no embedded MATLA, no variant subsystems, no function calls, mealy charts only, Design Models and Configuration Settings have high compatibility with Simulink Code Inspector System Automation Property Proving

15 Model Checking Property Proving Traditionally, properties are specified in behavioral logic The most general language is CTL*, with the subsets CTL and LTL CTL Example: AG(key_turned=>AX(auto_started)) At all times and paths, the auto must be started in the next step after the key is turned. Typical CTL operators are Monin, J. F., and Hinchey, M. G., Understanding formal methods, Springer, London, New York, AG all states on all paths satisfy Simulink Design Verifier Property Proving AF all paths eventually satisfy EG at least on one path all states satisfy EF at least one path along which some state satisfies System Automation Property Proving

16 Model Checking Property Proving II Properties can be categorized as follows Reachability EF φ Safety AG φ Liveness AF φ Fairness AG EF φ Dead Lock Freedom AG EF φ A particular situation can be reached. Under a certain condition, an event never occurs. Under a certain condition, an event will ultimately occur. Under a certain condition, an event will occur (or will fail to occur) infinitely often. The system can never be in a situation in which no progress is possible SLDV Property Proving supports Safety Properties Safety Properties are expressible in LTL PAST per definition 1 LTL withholds the path operators A, E of CTL* LTL PAST uses operators referring to the past, e.g., X (next) X 1 (previous) G G 1 (historically) F S 1 (sometime) 1 érard,., Systems and software verification. Model-checking techniques and tools, Springer, erlin, New York, System Automation Property Proving

17 Simulink Design Verifier for Property Proving Property specification not with temporal logic formulas, but by constructing semantically equivalent observers with blocks Additionally any other basic Simulink block can be used, observers can be executed and simulated System Automation Property Proving

18 Property Proving for System Automation (Example) Link Properties Modeling of linked requirements in TestHarness System Automation Property Proving

19 Lessons Learned (Property Proving) Advantages Requirement specification rules might differ from testing (Property Proving is exhaustive, so always and only can be proven which is hard to test on the other hand) Property Proving is extremely helpful to get a deep understanding of the developed model (especially edge cases that are hard to consider in testing) Graphical view is easier to interpret Application Challenges Theoretical background necessary to understand, what can be verified and what not Although the block set is limited, it is not straight-forward to model the right properties, e.g., The car never starts unless the key is turned. AG( sta t AW key_tu ned) AG(sta t AF 1 key_tu ned) To minimize the risk of invalid properties, provide a catalogue with common patterns System Automation Property Proving

20 Agenda Formal Verification Results a Case Study A. Inner Loop System model verification by model checking (Design Error Detection with Simulink Design Verifier) System Level. System Automation Design Model verification by Model Checking (Property Proving with Simulink Design Verifier) Source Code verification by MC + Abstract Interpretation (Polyspace Code Prover) Software Level C. Trajectory Controller Source Code verification by MC + Abstract Interpretation (Polyspace Code Prover) Formal Methods and Certification Aspects System Model vs. Software Design Model Tool Qualification considerations enefits and risks when applying formal verification activities / tools System Automation MC + Abstract Interpretation

21 Code Generation Settings (some) Generate code as model reference from a single model reference slbuild <modelname> ModelReferenceRTWTargetOnly Interface Generate Disable function and Reset function No terminate function Reusable Code Packaging Pass Root Level IO as Structure Reference Multiple instances of model references allowed No C main function With zero initialization of internal and IO data System Automation MC + Abstract Interpretation

22 Polyspace Code Prover Abstract interpretation (over estimation) to ensure scalability and automation No user specification of properties necessary (DO-333 wording: embedded properties ) Polyspace ug Finder uses same theoretic foundation, but only includes formal methods with low false positive rates System Automation MC + Abstract Interpretation

23 Code Prover Results (1st Iteration) Code Prover Results for System Automation Code Metric Value.h/.c Files 19/6 Executable Lines of Code Functions Main-Generator and call hierarchy issue Dead Code 2870 Proven error free 0 Undecidable 0 Errors 10 Unreachable code 250 MISRA Violations Non-unique typedef identifier System Automation MC + Abstract Interpretation

24 Resolution of MISRA Violations I Checks configured for MISRA C:2012 Mandatory/required Generated code requirements (Appendix E, former MISRA AC AGC) Enums and buses (structures) are used across the module and across model references Default approach of the coder is repeating guarded type definitions in each model reference code #ifndef DEFINED_TYPEDEF_FOR_ENUM_sysauto_level4_ #define DEFINED_TYPEDEF_FOR_ENUM_sysauto_level4_ typedef enum { ENUM_sysauto_level4_no_mode = 0, /* Default value */ ENUM_sysauto_level4_rtb_gps, ENUM_sysauto_level4_rtb_hdg } ENUM_sysauto_level4; #endif System Automation MC + Abstract Interpretation

25 Resolution of MISRA Violations II Solution: Specify a separate header file for enum and structure typedefs that are shared across model references classdef ENUM_system_modes < Simulink.IntEnumType... enumeration powering_up(0) standby(1) maintenance(2) test(3) operating(4) preparing_shutdown(5) end function dscope = getdatascope() % GETDATASCOPE Specifies whether the data type % definition should be imported from, % or exported to, a header file % during code generation. dscope = 'Exported'; end function filename = getheaderfile() % GETHEADERFILE Returns path to header file if non-empty. filename = 'sa_enum_types.h'; end System Automation MC + Abstract Interpretation

26 Code Prover Results (2nd Iter.) Code Prover Results for System Automation Code Metric Value.h/.c Files 21/6 Executable Lines of Code Functions Main-Generator and call hierarchy issue Dead Code 2870 Proven error free 0 Undecidable 0 Errors 10 Unreachable code 0 MISRA Violations System Automation MC + Abstract Interpretation

27 Uncalled Functions I y default, detection of uncalled functions is disabled Set uncalled-function-checks in the configuration to all or never called to activate their detection Uncalled functions are not analyzed System Automation MC + Abstract Interpretation

28 Uncalled Functions II Polyspace requires a main function to start the analysis If a main function is not provided, Polyspace offers a main-generator to automatically generate a specified main function Structure of the main function has significant influence on the result The user has to decide beforehand if the code shall be verified Function based (call every function by main) Model Reference based (call model reference by model reference) Module based (call functions of main module model reference(s)) Main based (call functions of top-level model reference only) Here, System Automation is considered as module with a single model reference as entry point Call hierarchy must be adapted accordingly (e.g. not only call initialize( ), but also Init() and Start() System Automation MC + Abstract Interpretation

29 Code Prover Results (3rd Iteration) Code Prover Results for System Automation Code Metric Value.h/.c Files 21/6 Main-Generator and call hierarchy issue Dead Code Executable Lines of Code Functions 2451 TD 2997 Proven error free 0 Undecidable 0 Errors 4 Unreachable code 0 MISRA Violations System Automation MC + Abstract Interpretation

30 Reset and Disable Functions Some model references are in a conditionally executed context Reset and disable functions are required However, other model references are always executed (not in a conditionally executed context) Unused reset and disable functions Generation of reset and disable functions is controlled via config set Problem cannot be solved: Different config sets (with and without reset and disable functions generated) cannot be used (config parameter mismatch error). When code is generated for each single model reference, the coder cannot determine whether it will ever be used in a conditional context => Mixing conditionally executed and always executed model references should be avoided, since the uncalled functions cannot be removed System Automation MC + Abstract Interpretation

31 Unreachable Code 1 Source: Data flow paths in model and code are not necessarily identical Well-known problem from preserving test coverage between model and source code Simulink Code Inspector (SLCI) compliance helps preserving data flow tc.atol_x() && tc.x_atol_unconfirmed mutually exculsive SLCI not compliant SLDV no dead logic PCP unreachable code switch(mode) { case fo_atol: if(tc.atol_x()) { if (tc.x_atol_unconfirmed) { } elseif(tc.x_fomlc())

32 Unreachable Code 2 Modified, SLCI compliant model (explicit transitions) SLCI compliant SLDV dead logic PCP unreachable code &&

33 Code Prover Results Code Prover Results for System Automation Code Metric Value.h/.c Files 21/6 Main-Generator and call hierarchy issue Executable Lines of Code Functions Proven error free 0 Undecidable 0 Errors 3 Unreachable code 0 MISRA Violations System Automation MC + Abstract Interpretation

34 Agenda Formal Verification Results a Case Study A. Inner Loop System model verification by model checking (Design Error Detection with Simulink Design Verifier) System Level. System Automation Design Model verification by Model Checking (Property Proving with Simulink Design Verifier) Source Code verification by MC + Abstract Interpretation (Polyspace Code Prover) Software Level C. Trajectory Controller Source Code verification by MC + Abstract Interpretation (Polyspace Code Prover) Formal Methods and Certification Aspects System Model vs. Software Design Model Tool Qualification considerations enefits and risks when applying formal verification activities / tools C LEVEL ALGORITHM TOOL RESULTS LESSONS LEARNED Trajectory Controller MC + Abstract Interpretation

35 Case Study C: Trajectory Controller Trajectory/path controller derived from nonlinear dynamic inversion of nonlinear error dynamics between a trajectory and the aircraft Metric Value SL lock Count 1748 SF Chart Count 0 Subsys Count 238 Models 61 Schatz, S. P., and Holzapfel, F., Modular trajectory / path following controller using nonlinear error dynamics, Aerospace Electronics and Remote Sensing Technology (ICARES), 2014 IEEE International, IEEE, 2014, pp C LEVEL ALGORITHM TOOL RESULTS LESSONS LEARNED Trajectory Controller MC + Abstract Interpretation

36 Code Prover for Embedded Coder Configure and start Code Prover from Simulink Highly recommended to generate the Code Prover Project from Simulink Shared utility functions can be automatically set as stubbed functions Design ranges from Simulink are reused Additional configurations settings for code generated by Embedded Coder Manual setting is hard due to the specifics of the auto-generated code (e.g. global structure reference as input) C LEVEL ALGORITHM TOOL RESULTS LESSONS LEARNED Trajectory Controller MC + Abstract Interpretation

37 Code Prover Results (1st Iteration) Code Prover Results for Trajectory Control Metric Value Design issues.h/.c Files 191/75 Executable Lines of Code Functions Proven error free 11 Undecidable 0 Errors 14 Unreachable code 57 MISRA Violations Imported DRS from Simulink! Empty initialize functions Directive C LEVEL ALGORITHM TOOL RESULTS LESSONS LEARNED Trajectory Controller MC + Abstract Interpretation

38 Review of MISRA Violations Unused code Embedded Coder creates empty initialize functions No fix in R2016, but Required MISRA rules can be justified formally Directives A directive is a guideline for which it is not possible to provide the full description necessary to perform a check for compliance. Additional information [ ] is required [ ]. (MISRA: Guideline classification ) Directly reviewable C LEVEL ALGORITHM TOOL RESULTS LESSONS LEARNED Trajectory Controller MC + Abstract Interpretation

39 Review of Remaining Issues Inlined parameters not specified with a range Input design ranges shadows saturation Oranges trace back to a single nonresolvable matrix-vector multiplication From pooled parameter 0 gain is only applicable for this configuration, however the controller is designed for a range of gains C LEVEL ALGORITHM TOOL RESULTS LESSONS LEARNED Trajectory Controller MC + Abstract Interpretation

40 Lessons Learned (Code Prover) Advantages Stateflow models with few arithmetic operations are verifiable without effort Embedded Coder with right settings produces MISRA:2012-compliant code with a few exceptions for mandatory and required content High integration in Simulink with Code Prover for Embedded Coder Correct settings for auto-generated code are hard to find Code Prover for Embedded Coder preserves design ranges Avoids redundancies and additional work Application Challenges Simulink controller models need design ranges a consistent approach for design ranges must be found on all levels Code Prover settings require expert knowledge C LEVEL ALGORITHM TOOL RESULTS LESSONS LEARNED Trajectory Controller MC + Abstract Interpretation

41 Agenda Formal Verification Results a Case Study A. Inner Loop System model verification by model checking (Design Error Detection with Simulink Design Verifier) System Level. System Automation Design Model verification by Model Checking (Property Proving with Simulink Design Verifier) Source Code verification by MC + Abstract Interpretation (Polyspace Code Prover) Software Level C. Trajectory Controller Source Code verification by MC + Abstract Interpretation (Polyspace Code Prover) Formal Methods and Certification Aspects System Model vs. Software Design Model Tool Qualification considerations enefits and risks when applying formal verification activities / tools