Changing the way the world does software

Automated Certification from Soup to Nuts Nick Tudor njt@drisq.com

Introducing D-RisQ D-RisQ is a SME based in Malvern, UK. All personnel have a background in mathematics, engineering and computer science Huge experience in analysis of complex systems and software across many sectors Safety and security critical systems, automotive, aerospace, robotics Build automated formal analysis tools Focus on cost/time reduction Builds upon the use of commercially available and widely used development tools

D-RisQ FM Ecosystem English CSP Verification Design CSP Protocols Packet Analysis CSP Formal Methods Design Networks CSP Simulink/ Stateflow Z SysML CSP CSP Object Code Source Code CSP Viruses HOL EOC Vulnerability CSP HOL CSP Ada Z Z C HOL

D-RisQ FM Ecosystem English CSP Verification Design CSP Protocols Packet Analysis CSP Formal Methods Modelworks CLawZ Design Networks CSP Simulink/ Stateflow Z SysML CSP CSP Object Code FEVER Source Code CSP Viruses HOL EOC Vulnerability CSP HOL CSP Ada Z Z C HOL

The USMOOTH Problem Project led by ASV How to design a decision making system that can support Unmanned Safe Maritime Operations Over The Horizon for extended durations (weeks)? How to provide assurance that the software does what is required and nothing else? How could we support a safety argument What standards could be used? Not necessarily maritime standards At what cost and can the process be easily [cheaply] repeated? NB This is safety critical software

Acceptable Behaviour Behaviour Boundary Defined by: User requirements Regulations Physics Environment 0 1 Everything in here must be acceptable The more complex, Acceptability demonstrated by: the more cost to Probing the boundary and verify conventionally By design/development processes Aiming to ensure: Nothing can penetrate or leak from the boundary And everything inside is done correctly

Engineering, Formal Methods and Certification

Verification Objectives DO-178C Soup A-3.2 Accuracy & Consistency A-3.3 HW Compatibility A-3.4 Verifiability A-3.5 Conformance A-3.7 Algorithm Accuracy A-4. 8 Architecture Compatibility System (A-2: 1, 2) High-Level A-4.1 Compliance A-4.6 Traceability (A-2: 3, 4, 5) A-3.1 Compliance A-3.6 Traceability A-6.1 Compliance A-6.2 Robustness A-7.1 Procedures are correct A-7.2 Results are Correct A-7.3 Test Coverage of HLRs A-7.4 Test Coverage of LLRs A-7.5 MC/DC A-7.6 Decision Coverage A-7.7 Statement Coverage A-7.8 Coupling Coverage A-4.9 Consistency A-4.10 HW Compatibility A-4.11 Verifiability A-4.12 Conformance A-4.13 Partition Integrity Architecture Low-Level A-4.2 Accuracy & Consistency A-4.3 HW Compatibility A-4.4 Verifiability A-4.5 Conformance A-4.7 Algorithm Accuracy A-5.2 Compliance A-5.3 Verifiability A-5.4 Conformance A-5.6 Accuracy & Consistency A-5.8 PDI Complete & Correct A-5.9 PDI Verified (A-2: 6) Source Code A-5.1 Compliance A-5.5 Traceability A-6.3 Compliance A-6.4 Robustness A-7.9 Untraceable object code Nuts A-5.7 Complete & Correct (A-2: 7) Executable Object Code A-6.5 Compatible With Target Compliance: with requirements Conformance: with standards

Verification Objectives DO-333 FM.A-3.8-11 Additional Objectives FM.A-3.2 Accuracy & Consistency FM.A-3.3 HW Compatibility FM.A-3.4 Verifiability FM.A-3.5 Conformance FM.A-3.7 Algorithm Accuracy FM.A-4. 8 Architecture Compatibility System (FM.A-2: 1, 2) High-Level FM.A-4.1 Compliance FM.A-4.6 Traceability (FM.A-2: 3, 4, 5) FM.A-3.1 Compliance FM.A-3.6 Traceability FM.A-6.1 Compliance FM.A-6.2 Robustness FM.A-7.1 Procedures are Correct FM.A-7.2 Results are Correct FM.A-7.3 Coverage of HLRs FM.A-7.4 Coverage of LLRs FM.A-7.5-8 Structural Coverage FM.A-4.9 Consistency FM.A-4.10 HW Compatibility FM.A-4.11 Verifiability FM.A-4.12 Conformance FM.A-4.13 Partition Integrity FM.A-4..14-17 Additional Objectives Architecture Low-Level FM.A-4.2 Accuracy & Consistency FM.A-4.3 HW Compatibility FM.A-4.4 Verifiability FM.A-4.5 Conformance FM.A-4.7 Algorithm Accuracy FM.A-5.2 Compliance FM.A-5.3 Verifiability FM.A-5.4 Conformance FM.A-5.6 Accuracy & Consistency FM.A-5.8 PDI Complete & Correct FM.A-5.9 PDI Verified FM.A-5.10-13 Additional Objectives FM.A-7.9 Property Preservation FM.A-7.10 Additional Objectives FM.A-5.7 Complete & Correct (FM.A-2: 6) Source Code (FM.A-2: 7) Executable Object Code FM.A-5.1 Compliance FM.A-5.5 Traceability FM.A-6.3 Compliance FM.A-6.4 Robustness FM.A-6.5 Compatible With Target Compliance: with requirements Conformance: with standards

Verification Objectives DO-333, Certification and Costs System System (FM.A-2: 1, 2) High-Level (FM.A-2: 3, 4, 5) Architecture Design Low-Level (FM.A-2: 6) Source Code Source Code (FM.A-2: 7) Executable Object Code Executable Object Code

, Certification and Costs System Development cost: 20-50% DO-333 Design Table FM.A-2 Objectives Table FM.A-3 Objectives Table FM.A-4 Objectives Table FM.A-5 Objectives Table FM.A-6 Objectives Table FM.A-7 Objectives Source Code Verification cost: 50-80% Executable Object Code Processor

IT STARTS WITH REQUIREMENTS D-RisQ

We Need Unambiguous Must be able to bound the required behaviour in a form that: Enables it to be agreed that it is the required behaviour Language understood by regulator and developer The language must enable formalisation Enable checks for conflicting requirements language should allow abstraction Minimal assumptions about architecture Enables a wider possibility of solution

and Certification - USMOOTH System LRE System Document (SRD) v1.0 Design DO-333 Table FM.A-2 Objectives Table FM.A-3 Objectives LRE Specification (SRS) v1.0 Source Code Executable Object Code Processor

and Certification - USMOOTH System Reviewed LRE System Document (SRD) v1.0 Design DO-333 Table FM.A-2 Objectives Table FM.A-3 Objectives LRE Specification (SRS) v1.0 Source Code Executable Object Code Processor

SOFTWARE DESIGN AND REQUIREMENTS VERIFICATION D-RisQ

Systems, and Certification System Reviewed LRE Specification (SRS) v1.0 Design DO-333 Table FM.A-2 Objectives Table FM.A-3 Objectives Table FM.A-4 Objectives LRE Simulink/ Stateflow Design v1.0 Source Code Executable Object Code Processor

Manual Manual Automatic D-RisQ Modelworks for Properties Description English D-RisQ Language Automatic Communicating Sequential Processes (CSP) Formal Language Simulink/Stateflow Formal Model check (FDR4) Systems Description Simulink/ Stateflow Modelworks Formal Language Communicating Sequential Processes (CSP) Formal Language Pre-defined Stability, reachability & exit Properties Manual design effort Automatic Automatic Automatic

Systems, and Certification System Reviewed Modelworks LRE Specification (SRS) v1.0 Design DO-333 Table FM.A-2 Objectives Table FM.A-3 Objectives Table FM.A-4 Objectives LRE Simulink/ Stateflow Design v1.0 Source Code Executable Object Code Processor

Hours Independent Experiments mid Feb 17 Now can being analysed by MW ~60% ~40% ~60% ~60% ~30% ~30% More automation of Modelworks due shortly

SOFTWARE SOURCE CODE VERIFICATION AGAINST DESIGN D-RisQ

Source Code System Reviewed Modelworks LRE Simulink/ Stateflow Design v1.0 Design DO-333 Table FM.A-2 Objectives Table FM.A-3 Objectives Table FM.A-4 Objectives Table FM.A-5 Objectives Source Code Autocoded using dspace TargetLink to C Executable Object Code Processor

Overview of the D-RisQ CLawZ Process Autocoder Simulink Code Development Z Producer Z User Interface Proof using Theorem Prover ProofPower Verification

Source Code System Reviewed Modelworks LRE Simulink/ Stateflow Design v1.0 Design DO-333 Table FM.A-2 Objectives Table FM.A-3 Objectives Table FM.A-4 Objectives Table FM.A-5 Objectives CLawZ Source Code CLawZ for C not yet fully developed; Verification not done Autocoded using dspace TargetLink to C Executable Object Code Processor

EXECUTABLE OBJECT CODE VERIFICATION

Executable Object Code System Reviewed Modelworks Autocoded using dspace TargetLink to C Design DO-333 Table FM.A-2 Objectives Table FM.A-3 Objectives Table FM.A-4 Objectives Table FM.A-5 Objectives Table FM.A-6 Objectives CLawZ Source Code Executable Object Code Compiled to ARM Processor

D-RisQ FEVER Formal Executable object code VERification Source Code Compiler Executable Object Code Processor Instruction Set Architecture User Interface Formalised in HOL Formalised in HOL Work in progress Proof using Theorem Prover ProofPower (HOL) Target is currently ARM

COVERAGE

DO-333 FM.6.7.f Options for verification activities of Executable Object Code include: (1) Formal analysis of Executable Object Code can be used to satisfy the objectives if all of the following conditions are satisfied: The requirements are formally expressed. A formal model of the Executable Object Code is defined. Formal evidence demonstrates that the formal model of the Executable Object Code satisfies the requirements. (2) Formal analysis of Source Code can be used to satisfy the objectives if all of the following conditions are satisfied: The requirements are formally expressed. A formal model of the Source Code is defined. Formal evidence demonstrates that the formal model of the Source Code satisfies the requirements. Complementary analysis shows property preservation between the Source Code and the EOC.

FMA-6 Our Approach System High-Level Development Activity Review/Analysis Activity Test Activity Formal Analysis Formal Representation Compliance Robustness Modelworks Architecture Design Low-Level Compliance Robustness CLawZ Source Code Executable Object Code Compliance Robustness FEVER Formal Proof of Property Preservation

Executable Object Code System Reviewed Modelworks Design DO-333 Table FM.A-2 Objectives Table FM.A-3 Objectives Table FM.A-4 Objectives Table FM.A-5 Objectives Table FM.A-6 Objectives CLawZ Source Code Table FM.A-7 Objectives FEVER Executable Object Code Processor

COSTS

Locking in IP and Reducing Cost System DO-333 Design Table FM.A-2 Objectives Table FM.A-3 Objectives Table FM.A-4 Objectives Table FM.A-5 Objectives Table FM.A-6 Objectives IP effort here Source Code Table FM.A-7 Objectives Automatic = Significant cost reductions Executable Object Code Processor

Summary Unmanned systems have to have a high integrity autonomous decision making system The business case for wider adoption is based on fast, cost effective assurance Automated formal proof from requirements to binary tools being built USMOOTH is showing that this is possible Regulatory approval of safety case and approach using DO-333 as we propose, still required Demonstration to follow in 2017

Changing the way the world does software