Injecting Model-Based Diagnosis Thinking into the Design Process. Johan de Kleer

Transcription

1 Injecting Model-Based Diagnosis Thinking into the Design Process Johan de Kleer DX2014 Sept 8, 2014

2 Development Time AVM Goals and Scope Goal: 5x Cost Reduction Aerospace Defense projects Integrated circuits Automotive Complexity

3

4 Current Industry State For almost all industries There are > 2000 tools BAE uses probably 400. Not integrated Solution tool chains (PTC, Dassault, Siemens, ESI,,Open Source CyPhy) Model-Based Engineering is progressing very slowly. Standards are progressing (but too slowly) No common languages.

5 Challenges to MBD Almost no thinking about reliability early. No thinking about diagnostics early. Lucky to get nominal models, never get fault models. There are some nominal model libraries No faulted model libraries

6 Bringing (reliability) analysis closer to design Push design changes to the conceptual stage Keep technical effort low Source: NASA Source: NASA Source: American Suppliers Institute 6

7

8 This is not the leverage point (from DX 2013)

9 Our Strategy Adopt the least-bad modeling language (by far) Develop fault model libraries (open source) (MSL) Focus on industries in which occasional failures are expected (Forget diagnosis for the moment) Focus on industry pain points Warranty costs Excess cost due to over-engineering Predict (reasonably accurately) the reliability of a design early in the design phase. Allow the designer to make more changes, not less. Inject into an industry tool chain.

10 Best Modeling Language: Modelica Modelica is an object-oriented acausal modeling language Industry and University support Multi-physics Open source tools (e.g., OpenModelica) Standard Libraries MSL multi-domain models Domain specific model libraries (e.g., buildings)

11 11

12 As complexity increases # of possible faults climbs # of faults # of components 12

13 But Number of root causes is constrained # of faults # of root causes # of components 13

14 Analyzing component physics Clutches and brakes have different functions but fail the same way model Clutch equation // Normal force and friction torque for w_rel=0 fn = fn_max*inport.signal[1]; tau0 = mue0*cgeo*fn; end Clutch model Brake equation // Torque equilibrium, normal force and friction torque for w_rel=0 fn = fn_max*inport.signal[1]; tau0 = mue0*cgeo*fn; end Brake Friction wear Fracture Fatigue Hydraulic pressure loss Source: Modelica Standard Library 14

15 Augmenting models with faults Friction wear model Clutch equation // Normal force and friction torque for w_rel=0 fn = fn_max*inport.signal[1]; if mode==nominal tau0 = mue0*cgeo*fn; elseif mode==worn sliding_dist = n_shift*t_shift*w_rel_clutch; slip_factor = wear_rate*fn*sliding_dist/a; tau0 = mue0*(1-slip_factor)*cgeo*fn; end if; end Clutch Clutch-specific factors Same physics of failure model Brake equation // Torque equilibrium, normal force and friction torque for w_rel=0 if mode==nominal tau0 = mue0*cgeo*fn; elseif mode==worn sliding_dist = n_braking*t_braking*w_wheel; slip_factor = wear_rate*fn*sliding_dist/a; tau0 = mue0*(1-slip_factor)*cgeo*fn; end if; end Brake Brake-specific factors 15

16 FAME Model Library Creation Process Validated with operational data Used Mechanical Engineers (OSU) to enumerate fault mechanisms and modes Focused on two types of faults (>> 80%). Port failures: open, short, leaky Parameter shift failures Port failure analogies don t work. JustAdd (from Modelon s Jmodelica) rules to rewrite Modelica We have (mostly) automatically converted most of the models in MSL. (some fluids and multibody models don t work yet).

17

18 FAME conceptual schematic Fault-Augmented Model Extension (FAME) nominal Fault-augment component models fault in torque converter Derive probabilities from physics-offailure Wrapping elements Read in design data Simulate design with faults Combine for fault analyses 18

19 Fault analysis Web-based flexible implementation 1. Select system configuration (numbers indicate reliability FOM) 2. Pick fault mode Set number of missions 4. Set required probability for meeting requirements 5. Select graphs to gain insight Tool publicly available at: 19

20 Fault analysis Find critical damage amount (requirement not met) Determine usage distribution corresponding to critical damage amount Plot (1 cdf) vs. usage as probability of success (requirement met) desired achievable 20

21 Overall Reliability Aggregate component failure probabilities into overall probability of mission failure 1 Probability of mission success under all single faults 21

22 Design feedback Component-level inferences Which Component Failure Modes causes critical performance loss? Why is a particular Component Failure Mode critical? How do these factors vary with Number of Missions? Aggregated system-level inferences Which System Configurations are most reliable? What maintenance strategy is best for chosen System Configurations? 22

23 Modelica Model Failure Mechanisms Compiler Modelica to Modelica fault compiler. Fault Augmented Models Component CAD Conditioning Posterior Probabilities Generic Design Damage Accumulation Simulator Conditional Probabilities Terrain

24 Modelica Model Failure Mechanisms Compiler Fault Augmented Models Component CAD Conditioning Posterior Probabilities Generic Design Damage Accumulation Simulator Conditional Probabilities Preconstructed Terrain

25 Archard s Law (for wear)

26 Working on fitting parameter models Use Interpolation (w,f,s,a)

27 Modelica Model Failure Mechanisms Compiler Fault Augmented Models PDFs Component CAD Conditioning Posterior Probabilities Generic Design Damage Accumulation Simulator Conditional Probabilities Terrain

28 Diagnostics Using FAME Model based diagnostics using Bayesian inference Use FAME to perform Monte Carlo simulations of nominal and fault modes Diagnose faults by comparing observed data against expected data All behaviors

29 ABCsampler ABCestimator Diagnostics Using FAME (non-gde like) Experimental Setup Draw FAME Parameter Values from Prior Distribution Get Actual Measurement Compute Summary Statistics Perform FAME Simulation Compute Summary Statistics Save Parameter Values and Summary Statistics to File Compute Posterior Distribution and Perform Model Selection

30 Some Bad News: Numerical Instabilities Some numerical simulations suffer from numerical instability (ill conditioned Jacobians) Simulations appear to be frozen (in fact they advance very slowly ) Mechanism to measure the simulation time and end the simulation when a threshold is reached (externally through process monitoring or internally through Dymola scripting) model test_bench parameter Real simulationtime = 10; equation when time>0 then starttime = systime(0.0); end when; elapsedtime = systime(starttime); when (elapsedtime > simulationtime) then terminate("exceeded allowable simulation t ime"); end when; end test_bench; instability happens at the beginning of the simulation instability happens at the end of the simulation 30

31 31

32 Accelera on Time to 10 kph (s) Simulation Outlier Removal Fault Amount 32

33 Modelica folks just rewrite their models when they have problems 33

34 Vision: QR Provides Support for Designers Engineers create and evaluate designs at multiple levels of abstraction? Current behavioral analysis tools require fully specified designs Qualitative reasoning (QR) enables reasoning at multiple levels of abstraction Qualitative reasoning can Determine if desired function is realizable Identify dangerous unexpected interactions Guide detailed design and redesign 34 8/28/2012

35 Basic Qualitative Reasoning Qualitative Values Q0 x = 0 Q+ x > 0 Q- x < 0 Qualitative Arithmetic: + Q- Q0 Q+ Q- Q- Q-? Type equation here. Q0 Q- Q0 Q+ Q+? Q+ Q+ Integration and Continuity: x: dx/dt: Q- Q0 Q+ Q- Q- Q- Q0* Q0 Q- Q0 Q+ Q+ Q0* Q+ Q+ * Q- Q0 Q+ Q- Q+ Q0 Q- Q0 Q0 Q0 Q0 Q+ Q- Q0 Q+ * Indicates continuity duration Q- Q0 Q+ Q- y y n Q0 y y y Q+ n y y 01/12/2011 BAE Systems All rights reserved. See title page for handling instructions. 35 x 1 Landmarks Higher-Order Derivatives Limitations x 2

36 Envisionment Summarizes Possible Behaviors Multitrajectory simulation <1> off, Q+ <105> off, Q+ <185> off, Q0 <169> off, Q+ <229> on, Diode1.OnV = 1.5, Battery.V =.6 Diode1.OnV =.6, Battery.V = 1.5 <269> on, Q+ <330> on, Q0

37 Modelica Model XML Equations Envisionment QRM

38 Simulation Tracker Highlighted trajectory conforms to parameter settings PARC 38

39 View simulation Simulation parameters Velocity Brake applied Rollback Position PARC 39

40 Modelica Challenges Encountered Guidance to the solver conflated with the model. Imperative Code

41 Modelica.math.IsEqual result := abs(s1-s2) <= eps; (1) Increases complexity by [7/3]^n (2) Introduces errors a b c d e f g -eps 0 eps Solution: Use AOP or meta-comments. Note: Unfortunately there are many other ones like this. E.g.,Modelica.Fluid.Utilities.regStep, PartialFriction, many tables.

42 Imperative Code in Models

43 Modelon to the Rescue We defined the evil constructs in Modelica DARPA is paying Modelon to develop an opensource declarative version of MSL. (and ensure its equivalent ) This is EXACTLY the kind of models we need to do consistency-based diagnosis. They are truly declarative, can use inference vs. simulation Peter will want this

44 Modelica in AVM Modelica was the right choice, but Modelica solvers need to become a lot better. Modelica is powerful enough to write good declarative models. Maybe Modelica needs a model curation process? Many MSL Models look like they were written by students. Unfortunately, people write some incredibly bad models. Most egregious: Arbitrary code can execute at run-time. Models can contain help for the simulator. (Partial solution) AVM is funding Ricardo, Modelon and Linkoping to build better models. Purely declarative Asserts for domain of applicability. Remove all help for the simulator. What should the gold standard be? Its definitely not Modelica. Graph of Models? Aspects for Models?

45 What do we want from a model? Modelica Model Textbook Model Simulation Validity Draw correct inferences cheaply Understandable by machine Understandable by human Poem What is the gold standard? World

46 What do we want from a model? Modelica Model Textbook Model QR/RA Analysis Validity Draw correct inferences cheaply Understandable by machine Understandable by human Poem What is the gold standard? World

47 Maybe there is no gold standard. But how then do we define whether a model is good or bad? How do we traffic in models?