Beyond Static Code Analysis

Transcription

1 Beyond Static Code Analysis Dr. Michael Whalen July 23, 2009 Traditional Domains of Concern Mats Heimdahl. Tool Intensive Software Development, FAA Software Tools Forum, Daytona Beach, FL, May,

2 How we Develop Software Concept Formation Requirements Specification Is object code: - free from runtime errors? - correct w.r.t. functional requirements? - free from unintended functionality? System Test - fast enough? Test System Integration Test Design Integration Implementation Unit Test Object Code Analysis Mats Heimdahl. Tool Intensive Software Development, FAA Software Tools Forum, Daytona Beach, FL, May, How Static Analysis Helps Concept Formation Requirements Specification Is object code: - correct w.r.t. functional requirements? System - free from unintended functionality? System Test Design Test Integration Integration Test Implementation Unit Test Analysis:Timing Runtime Errors Object Code Analysis: Runtime Errors Adapted from: Mats Heimdahl. Tool Intensive Software Development, FAA Software Tools Forum, Daytona Beach, FL, May,

3 How we Will Develop Software Concept Formation Is model and/or code: - correct w.r.t. functional requirements? - free from unintended functionality? Requirements PropertiesAnalysis Specification/Model System System Test Specification Test Integration Integration Test Object Code Analysis:Timing Runtime Errors Adapted from: Mats Heimdahl. Tool Intensive Software Development, FAA Software Tools Forum, Daytona Beach, FL, May, Presentation Overview Overview Formalizing Requirements Formal Methods Examples of Applying Formal Methods Challenges and Future Directions 6 3

4 Requirements as Shall Statements 7 Encoding Requirements as Properties If the mode is COOKING, then the microwave door shall be CLOSED SPEC AG(mode = COOKING -> door_closed) ; CTL Simulink assert (!(mode == COOKING) door_closed); Java Code 8 4

5 Encoding Requirements as Properties 9 Uses of Formal Requirements Level of scale Subsystem System System of Systems Requirements Design Code Test Field Properties can be checked for completeness and consistency Verification tools prove whether design models meet properties Refutation tools can quickly find violations of properties Autogenerate code Focus of this talk Automated test generation directly from properties Properties used as test oracles for unit and integration test Properties as runtime monitors to recover from failures at runtime 10 5

6 Presentation Overview Overview Formalizing Requirements Formal Methods Examples of Applying Formal Methods Challenges and Future Directions 11 What are Formal Methods? Mathematically-based techniques for the specification, development and verification of software and hardware systems. Specification Temporal Logics (LTL, CTL, PSL) Textual notations (Z, B, VDM, PVS, SMV, ) Tabular notations (Parnas Tables, SCR, RSML, ) Graphical notations (SCADE, Simulink, Statecharts ) Wikipedia, 8 April 2008 Development Stepwise refinement with proofs of correctness Model-Based Development Automated code generation Verification Lightweight static analysis Model-checking (SMV, SAL, Prover, ) Theorem proving (ACL2, PVS, HOL, ) Test case generation 12 6

7 Verification Model Checking Breakthrough Technology of the 1990 s Widely used in hardware verification (Intel, IBM, ) Several Different Types of Model Checkers Explicit, symbolic, bounded, infinite bounded (SMT), Exhaustive Analysis of the Global State Space Consider all combinations of inputs and states Equivalent to exhaustive testing of the model Produce a counter example if a property is not true Easy to Use Push Button formal methods Very little effort (except at the tool s limits) Limitations Symbolic model checkers State space explosion ( states) SMT-based model checkers Skill of the user infinite state models possible 13 Verification Model Checking Testing Checks Only the Values We Select Model Checker Tries Every Possible Value! Even Small Systems Have Trillions (of Trillions) of Possible Tests! Finds every exception to the property being checked! 14 7

8 Model or Code (Simulink, SCADE, C, Java) Verification Model Checking Automatic Translation Analysis Model (SMV, Prover, SAL, ) Does the system have property X? Yes! Counter Example Automated Check Model Checker (NuSMV, PROVER, SAL, Translation Engineer Requirements (Properties) Properties (CTL, LTL, PSL, ) 15 Verification Theorem Proving Available Since the 1980 s Heavily used on security systems Use Rules of Inference to Prove New Properties Also consider all combinations of inputs and states Equivalent to exhaustive testing of the model Generate an unprovable proof obligation if a property is false Not Limited by State Space Applicable to almost any formal specification Limitations Skill of user about six months to become proficient Time - constructing proofs is labor intensive 16 8

9 Verification Theorem Proving Model or Code (Simulink, SCADE, C, Java) Automatic Translation Analysis Model (PVS ACL2, HOL, ) Does the system have property X? Yes! Guru Why not? Automated Check Theorem Prover (PVS, ACL2, HOL, Translation Engineer Requirements (Properties) Properties (Lemmas, Theorems, ) 17 Verification Automated Test Generation Available Since the 1990s Supported by several commercial tools T-VEC, Reactive Systems, LDRA, The Mathworks, etc. Generate tests up to some level of coverage In Avionics: MCDC is common Not a proof! Not Limited by State Space Applicable to source code, object code, or design models Low cost to generate thousands or millions of tests Limitations No oracle! Nothing to describe whether test passes. Properties are required to define notion of correctness May be unable to generate all tests for given level of coverage Once again, not a proof! Only possible to refute properties. 18 9

10 Verification Automated Testing Model or Code (Simulink, SCADE, C, Java) Automatic Translation TCG Model (T-VEC, Reactis, ) Does the system have property X? Tests OK Test Cases Counter Example Automated Check Autotest Tool (T-VEC, Reactis) Translation Engineer Requirements (Properties) Properties (CTL, LTL, PSL, ) 19 Presentation Overview Overview Formalizing Requirements Formal Methods? Examples of Applying Formal Methods Challenges and Future Directions 20 10

11 Rockwell Collins Translation Framework Support a wide variety of back end tools Very straightforward to add new tools E.g. Prover support: 4 days effort Allows the right tool for the right job 21 Examples of Using Formal Methods FCS 5000 Flight Control Mode Logic Mode Controller A Modeled in Simulink Translated to NuSMV 6.8 x Reachable States Mode Controller B Example Requirement Mode A1 => Mode B1 Counterexample Found in Less than Two Minutes Found 27 Errors 22 11

12 1 [trigger] sync<> sync 2 [A] input_a 3 [B] input_b 4 [C] input_c 5 [status_a] status_a 6 [status_b] status_b 7 [status_c] status_c 8 [DSTi] dst_index DOC Text [trigger] [A] [B] [C] trip_level trip_level1 persist_lim persistence limit [MS] [DSTi] DST Data Store Read trip_level persist_lim Index Vector input_a input_b input_c trip_level Extract Bits [0 3] Extract Bits failreport failreport double pc persistence_cnt<pc> 2 persistence_cnt persist_lim [trigger] tc 3 totalizer_cnt<tc> [A] totalizer_cnt MS [B] [C] triplex_input_monitor [DSTi] [MS] [status_a] [status_b] [status_c] [prev_sel] [A] [B] [C] mon_f ailure_report status_a status_b status_c f ailure_report prev_sel input_a input_b input_c Failure_Isolation pc trigger input_a input_sel input_b input_c DST_index triplex_input_selector 4 input_sel 1 failure_report [prev_sel] [DSTi] f ailure_report dst_index Failure_Processing Examples of Using Formal Methods ADGS-2100 Adaptive Display & Guidance System Modeled in Simulink Translated to NuSMV 4,295 Subsystems 16,117 Simulink Blocks Over Reachable States Example Requirement: Drive the Maximum Number of Display Units Given the Available Graphics Processors Counterexample Found in 5 Seconds Checked 573 Properties - Found and Corrected 98 Errors in Early Design Models 23 Examples of Formal Methods CerTA FCS Phase I Sponsored by AFRL Wright Patterson VA Directorate Compare FM & Testing Testing team & FM team Lockheed Martin UAV Adaptive Flight Control System Redundancy Management Logic Modeled in Simulink Translated to NuSMV model checker Subsystem/ Blocks Charts / Transitions / TT Cells Reachable State Space Properties Triplex voter 10 / 96 3 / 35 / * Failure processing 7 / 42 0 / 0 / * Reset manager 6 / 31 2 / 26 / * Totals 23 / / 61 / 198 N/A 62 for each of ten control surfaces Phase I Results Effort (% total) Errors Found Testing 60% 0 Model-Checking 40%

13 Examples of Formal Methods CerTA FCS Phase I Can We Eliminate Testing? Errors are always made during development ~ 1 errror / 1,000 SLOC for CMM 5. Testing can be used everywhere it verifies the final product but isn t very good at finding errors. Model-checking is very good at finding errors early in the design process but it doesn t work everywhere. System Use model-checking where it works now technology is improving rapidly and will be even better in the future. But some testing will always be necessary. 25 Examples of Formal Methods Can Model-Checking be Used on Large, Non-linear Systems? Lockheed Martin Adaptive UAV Flight Control System Extensive Use of matrix arithmetic Inputs 33 floating point inputs (including one 3 x 6 matrix) Outputs 6 floating point values 166 Simulink subsystems basic Simulink blocks Translated to Prover model checker CerTA FCS Phase II Sponsored by the AFRL - Wright Patterson VA Directorate Challenges Verification of floating point matrix arithmetic Verification of Stateflow flowcharts with cycles Compositional Verification Final Results Identified five previously unknown errors Identified several implementation errors that were being masked by defensive programming 26 13

14 Examples of Using Formal Methods AAMP7G Certified Microprocessor Formal proof of the MILS security partitioning implemented in the AAMP7G microprocessor Example of the industrial use of theorem proving using ACL2 Developed formal description of separation for uniprocessor, multipartition system (GWV) Modeled trusted AAMP7G microcode in ACL2 Constructed machine-checked proof of separation of the AAMP7G model Model subject of intensive code-to-spec review with AAMP7G microcode Satisfied formal methods requirements for NSA AAMP7G certification awarded in May verified using Formal Methods techniques as specified by the EAL-7 level of the Common Criteria - capable of simultaneously processing unclassified through Top Secret Codeword Information 27 One more fun example Empty Sudoku board has ~10 70 = (9 ^ 9 ^ 9) possible states After filling in given numbers for a board, considerably fewer states. [ July 22, 2009] What are Sudoku requirements? Must have exactly one of 1-9 in each row, column, and 3x3 square. Conversely, a bad non-solution will have two (or no) elements along one of these vectors Can we trick a model checker into solving Sudoku? State that all solutions for a set of givens are bad Model checker will demonstrate that this property is false and provide a counterexample (solution) 28 14

15 Sudoku encoding in SAL sudoku2: CONTEXT = BEGIN BLOCK : TYPE = [ 0.. 2] ; BLOCK_RANGE : TYPE = [1.. 3] ; RANGE : TYPE = [ 1..9 ]; BOARD : TYPE = array RANGE of array RANGE of RANGE ; sudoku : MODULE = BEGIN INPUT b : BOARD OUTPUT bad : BOOLEAN DEFINITION bad = % bad column (EXISTS (x: RANGE, y1: RANGE, y2: RANGE) : (y1 /= y2) AND (b[x][y1] = b[x][y2])) OR % bad row (EXISTS (y: RANGE, x1: RANGE, x2: RANGE) : (x1 /= x2) AND (b[x1][y] = b[x2][y])) OR % bad 3x3 block (EXISTS (xb: BLOCK, yb: BLOCK, x1: BLOCK_RANGE, y1: BLOCK_RANGE, x2: BLOCK_RANGE, y2: BLOCK_RANGE) : (x1 /= x2) AND (y1 /= y2) AND b[3*xb + x1][3*yb + y1] = b[3*xb + x2][3*yb + y2] END ; % generate a board that meets the % sudoku constraints test_th: THEOREM sudoku - G(bad); % solve a specific hard sudoku % according to wikipedia test_bf_hard: THEOREM sudoku - G ((b[6][2] = 3 AND b[8][2] = 8 AND b[9][2] = 5 AND b[3][3] = 1 AND b[5][3] = 2 AND b[4][4] = 5 AND b[6][4] = 7 AND b[3][5] = 4 AND b[7][5] = 1 AND b[2][6] = 9 AND b[1][7] = 5 AND b[8][7] = 7 AND b[9][7] = 3 AND b[3][8] = 2 AND b[5][8] = 1 AND b[5][9] = 4 AND b[9][9] = 9) => bad) ; END ; 29 SAL Sudoku Demo 30 15

16 Presentation Overview Overview Formalizing Requirements Formal Methods Examples of Applying Formal Methods Challenges and Future Directions 31 Classes of software systems Maturity of V&V Tools Ready for use Nonlinear Systems Linear Systems Finite-State Systems examples: mode logic, protocols, display logic characteristics: no reals or large-domain integers tools: symbolic model checking (NuSMV) example: linear controller characteristics: reals and integers; no trigonometry or non-linear arithmetic tools: k-induction model checkers (SAL, Prover) Current Research example: nonlinear or adaptive controller characteristics: trigonometry, nonlinear arithmetic tools: theorem provers (ACL2, PVS), abstract interpretation (ASTREE) 32 16

17 Challenges and Future Directions Compositional Verification Typical Model-Based Specification Models are organized in a hierarchy of subsystems several levels deep Most of the complexity is in the leaf models Leaf models can often be verified through model checking P1 1 In1 P2 & P3 2 In2 P2 & P3 -> Q1 In_A1 Out_A In_A2 Subsystem A Q1 P1 & Q1 -> Q2 In_B1 Out_B In_B2 Subsystem B Q2 Q 1 Out1 Composition of Subsystems Tends to be simple Well suited for theorem proving P2 & P3 => Q1 P1 & Q1 => Q2 => P1 & P2 & P3 => Q Issues Lack of a unified automated verification system Use model-checking to verify leaf models and theorem proving for composition Avoid circular reasoning to ensure soundness Can be ensured by eliminating cyclic dependencies between atomic subsystems Identifying the right leaf level invariants to support composition Complexity of the proof obligations for the intermediate levels 33 Challenges and Future Directions System Architectural Modeling & Analysis System Architecture Model Logical Abstracts Security Analysis Performance Analysis ADL Level B Classified Implements Auto Generate Safety Analysis Simulink Model C Code IMA Cabinet VAPS Model C Code Common Computing Resource 3 Common Computing Resource 2 Common Computing Resource 1 Level C Unclassified Ada Code App A App B App C Sys Specific Middleware (Schedule, Communication Routes) Reusable Trusted Middleware (RTOS, I/O, RT-CORBA) Separation Kernel Target Hardware Level A Top Secret Software Component Development Physical IMA BUS System Architecture Development 34 17

18 Challenges and Future Directions Applying Tools to Medical Software Embedded Medical Software is often finite-state or linear Complex mode logic Many feature interactions Implementing protocols for (say) pacing Off-the-shelf model checkers scale very well to similar problems in Avionics Likely that analysis tools will work well in this domain Requirements must be formalized Formal analysis can be performed on models or directly on source code high-quality code model checkers: C/C++: CBMC, FocusCheck, SLAM, BLAST Java: Java Pathfinder Ada: SPARK Tool suite 35 Challenges and Future Directions Conclusions Formal Methods Are Practical and Are Being Widely Used Model Based Development is the industrial face of formal methods The engineers get to pick the modeling tools! Semantics of some of the commercial tools could be improved Formal Verification Tools Are Being Used in Industry Key is to verify the things the engineers are already building Large portions of existing systems can be verified with model checkers Model checkers are only going to get better Theorem proving can be used on stable systems Directions for the Future Work Making verification tools more powerful and easier to use Addressing scalability through compositional verification Integration of theorem proving and model checking Modeling and analysis of system architectural models 36 18

19 For More Information Whalen, M., Cofer, D., Miller, S., Krogh, B., Storm, W.: Integration of Formal Analysis into a Model-Based Software Development Process. In 12th International Workshop on Formal Methods for Industrial Critical Systems (FMICS2007), Berlin, Germany (2007). Whalen, M., Innis, J., Miller, S., Wagner, L.: ADGS-2100 Adaptive Display & Guidance System Window Manager Analysis, CR , NASA (2006). Miller, S., Tribble, A., Whalen, M., Heimdahl, M., Proving the Shalls, International Journal on Software Tools for Technology Transfer (STTT), Feb Miller, S., Anderson, E., Wagner, L., Whalen, M., Heimdahl, M.: Formal Verification of Flight Critical Software. In AIAA Guidance, Navigation and Control Conference and Exhibit, AIAA , American Institute of Aeronautics and Astronautics (2005). Greve, D., Wilding, M., Vanfleet, W. M.: A Separation Kernel Formal Security Policy. In Fourth International Workshop on the ACL2 Prover and Its Applications (ACL2-2003) (2003). Greve, D., Richards, R., Wilding, M.: A Summary of Intrinsic Partitioning Verification. In Fifth International Workshop on the ACL2 Prover and Its Applications (ACL2-2004) (2004). Greve, D., Wilding, M., Richards, R., Vanfleet, W. M.: Formalizing Security Policies for Dynamic and Distributed Systems. In Systems and Software Technology Conference (SSTC 2005), Utah State University, (2005). 37 Backup Slides 38 19

20 What are Formal Methods? Textual (Lustre, PVS, SAL, ) node Thrust_Required( FG_Mode : FG_Mode_Type ; Airborne : bool ; In_Flare : bool ; Emergency_Descent : bool; Windshear_Warning : bool ; In_Eng_Accel_Zone : bool ; On_Ground : bool) returns (IsTrue : bool) ; let Specification Tabular (RSML -e, SCR) IsTrue = (FG_Thrust_Mode(FG_Mode) and Airborne) or (Airborne and Emergency_Descent) or Windshear_Warning or ((FG_Mode = ThrottleRetard) and In_Flare) or (In_Eng_Accel_Zone and On_Ground) ; tel ; Graphical (SCADE, Simulink) 39 Company Product Tools Specified & Autocoded Benefits Claimed Airbus A340 SCADE With Code Generator Eurocopter GE & Lockheed Martin Schneider Electric US Spaceware PSA CSEE Transport Honeywell Commercial Aviation Systems EC-155/135 Autopilot What are Formal Methods? FADEDC Engine Controls Nuclear Power Plant Safety Control SCADE With Code Generator 70% Fly-by-wire Controls 70% Automatic Flight Controls 50% Display Computer 40% Warning & Maint Computer 90 % of Autopilot 20X Reduction in Errors Reduced Time to Market 50% Reduction in Cycle Time ADI Beacon Not Stated Reduction in Errors 50% Reduction in Cycle Time Decreased Cost SCADE With Code Generator 200,000 SLOC Auto Generated from 1,200 Design Views 8X Reduction in Errors while Complexity Increased 4x DCX Rocket MATRIXx Not Stated 50-75% Reduction in Cost Reduced Schedule & Risk Electrical SCADE 50% SLOC Auto Generated 60% Reduction in Cycle Time Management With Code 5X Reduction in Errors System Generator Subway Signaling System Primus Epic Flight Control System Model-Based Development SCADE With Code Generator MATLAB Simulink 80,000 C SLOC Auto Generated 60% Automatic Flight Controls Improved Productivity from 20 to 300 SLOC/day 5X Increase in Productivity No Coding Errors Received FAA Certification 40 20

21 Examples of Using Formal Methods Integrity-178B Real-Time OS Evaluation Formal proof of the MILS security partitioning implemented in the Integrity-178B Real-Time OS Example of the industrial use of theorem proving using ACL2 Generalized the formal description of separation to describe the more dynamic scheduling managed by the OS (GWVr2) Modeled in ACL2 the target-independent C code implementing the Integrity-178B kernel. Constructed machine-checked proof of separation for the Integrity-178B kernel Model, analysis approach and proofs subject to intensive multi-national review Satisfied US Government SKPP (EAL6+), as well as Common Criteria v2.3 EAL7 ADV requirements Final certification pending NSA penetration testing K P1 P3 P