The Design and Implementation of a Modeling Package

Transcription

1 The Design and Implementation of a Modeling Package p.1/20 The Design and Implementation of a Modeling Package Hossein ZivariPiran hzp@cs.toronto.edu Department of Computer Science University of Toronto (part of my PhD thesis under the supervision of professor Wayne Enright)

2 Outline The Design and Implementation of a Modeling Package p.2/20 The Modeling Process Why Modeling Packages? Difficulties of Designing a Modeling Package A Proposed Framework Based on Experiment

3 The Modeling Process The Design and Implementation of a Modeling Package p.3/20 Physical/Biological Phenomenon Mathematical Model Physical Laws / Empirical Rules Observed Data Sensitivity Analysis Refined Mathematical Model Parameter Estimation Practical Mathematical Model

4 The Modeling Process - Parameterized Models The Design and Implementation of a Modeling Package p.4/20 y(t) y(t;p) y(x) y(x;p) y(x,t) y(x,t;p) Parameters are used to: Make the model applicable in similar situations. Analyze the effect of uncertainties. Represent unknown quantities.

5 The Modeling Process - Simulation The Design and Implementation of a Modeling Package p.5/20 Computing a numerical approximation for fixed values of parameters and some initial/boundary conditions. Traditionally considered as the core of numerical studies. Many researchers are working on developing faster/more reliable methods. Generalized Methods - vs - Specialized Methods

6 The Modeling Process - Sensitivity Analysis The Design and Implementation of a Modeling Package p.6/20 Forward Sensitivity Analysis The (first order) solution sensitivity with respect to the model parameter p i is defined as the vector s i (t;p) = { p i }y(t;p), (i = 1,...,L) The second order solution sensitivity with respect to the model parameters p i and p j is defined as the vector 2 r ij (t;p) = { }s i (t;p) = { }y(t;p), p j p j p i (i, j = 1,...,L)

7 The Modeling Process - Sensitivity Analysis The Design and Implementation of a Modeling Package p.6/20 Sensitivity information can be used to: Estimate which parameters are most influential in affecting the behavior of the simulation. Such information is crucial for Experimental Design Data Assimilation Reduction of complex nonlinear models Study of Dynamical Systems : Periodic orbits, the Lyapunov exponents, chaos indicators, and bifurcation analysis are fundamental objects for the complete study of a dynamical system, and they require computation of the sensitivities with respect to the initial conditions of the problem. Evaluate optimization gradients and Jacobians in the setting of Dynamic Optimization Parameter Estimation

8 The Modeling Process - Parameter Estimation The Design and Implementation of a Modeling Package p.7/20 Parameter Estimation Problem A Parameterized Model y(t; p) A Set of Data (Observations/Measurements) {Y (γ i ) y(γ i ;p )} Estimate p by minimizing an objective function. e.g. W(p) = i [Y (γ i ) y(γ i ;p)] 2.

9 The Modeling Process - Parameter Estimation The Design and Implementation of a Modeling Package p.7/20 Parameter Estimation/Fitting is not easy, like almost any other optimization problem, The parameters may not be identifiable Numerical Difficulties The efficiency may depend strongly on the place/number of data points Strategies for Collecting the Best Data

10 Why Modeling Packages? The Design and Implementation of a Modeling Package p.8/20 Sensitivity analyzer and parameter estimator have become as essential as simulator for studying a phenomenon using a mathematical model. Increase of the processing power Use of more complex/detailed models sensitivity analysis and parameter estimation become more complicated. Sensitivity analysis and parameter estimation are highly dependent on the simulator An integrated design could lead to efficient communications and prevent possible redundancies.

11 Difficulties of Designing a Modeling Package The Design and Implementation of a Modeling Package p.9/20 We need to develop an efficient simulator, a sensitivity analyzer and a parameter estimator Time Consuming/Expensive How to deal with the "Generality - vs - Efficiency" problem? Want to have a manageable design with little effort be able to change some algorithms or add new algorithms. Want to have a user-friendly interface Easy enough for nonspecialists and controllable enough for specialist. Ideally Parallelizable At least thread safe No Global Variables. Easily incorporable into other packages/programms Easy to do experiments, for instance, comparing different algorithms No Assumption for Global Modules.

12 A Proposed Framework Based on Experiment - DDEs The Design and Implementation of a Modeling Package p.10/20 An Initial Value Problem (IVP) for Ordinary Differential Equations (ODEs) y (t) = f(t, y(t)) y(t 0 ) = y 0 Retarded Delay Differential Equations (RDDEs) y (t) = f(t, y(t), y(t σ 1 ),, y(t σ ν )) for t 0 t t F y(t) = φ(t), for t t 0 σ i = σ i (t, y(t)) 0 delay (constant / time dependent / state dependent) φ(t) history function (constant / time dependent) Neutral Delay Differential Equations (NDDEs) y (t) = f(t, y(t), y(t σ 1 ),, y(t σ ν ), y (t σ ν+1 ),, y (t σ ν+ω )) for t 0 t t F y(t) = φ(t), y (t) = φ (t), for t t 0,

13 DDEs - An Example A Parameterized DDE y (t;p) = f(t, y(t;p), y(t σ(t;p));p) for t 0 (p) t y(t;p) = φ(t;p), for t t 0 (p) For Example, in the neutral delay logistic Gause-type predator-prey system [Kuang 1991] y 1(t) = y 1 (t)(1 y 1 (t τ) ρy 1(t τ)) y 2(t)y 1 (t) 2 y 1 (t) ( y 2(t) y1 (t) 2 ) = y 2 (t) y 1 (t) α where α = 1/10, ρ = 29/10 and τ = 21/50, for t in [0, 30]. The history functions are for t 0. Parameters are φ 1 (t) = t φ 2 (t) = t p = [τ, ρ, α, a = , b = 1 10, c = , d = 1 10 ] The Design and Implementation of a Modeling Package p.11/20

14 DDEs - Numerical Simulation The Design and Implementation of a Modeling Package p.12/20 Classical Theory of Step by Step Integration for IVPs. Runge-Kutta (RK). Linear Multistep (LM). + Continuous Solution using Polynomial Approximation. Continuous Runge-Kutta (CRK). Linear Multistep methods have natural approximating polynomials. DDEs : Combining an interpolation method (for evaluating delayed solution values) with an ODE integration method (for solving the resulting ODE ).

15 DDEs - Special Difficulties The Design and Implementation of a Modeling Package p.13/20 Derivative Discontinuities In general φ (t 0 ) f(t 0, φ(t 0 ), φ(t 0 σ 1 ),, φ(t 0 σ ν )) Due to the existence of delays, discontinuities propagate along the integration interval. Solution is smoothed for RDDEs but in general not for NDDEs. The RK and LM methods fail in presence of discontinuities. Treatment : Tracking Discontinuities and forcing them to be mesh points.

16 DDEs - Special Difficulties The Design and Implementation of a Modeling Package p.14/20 The predator-prey model y t

17 DDEs - Sensitivity Analysis The Design and Implementation of a Modeling Package p.15/20 Internal Differentiation Approach for IVPs y (t;p) = f(t, y(t;p);p), y(t 0 ) = y 0 (p) Differentiation + Chain Rule + Clairaut s Theorem s i = f y s i + f p i, s i (t 0 ) = y 0(p) p i, (i = 1,... L)

18 DDEs - Sensitivity Analysis The Design and Implementation of a Modeling Package p.15/20 Adapted Internal Differentiation Approach for DDEs y (t;p) = f(t, y(t;p), y(α(t, y;p);p), y (α(t, y;p);p);p) y(t;p) = φ(t;p), for t t 0 (p) Differentiation + Chain Rule + Clairaut s Theorem s i(t) = f y s i(t) + f y(α k ) f + y (α) + f p ( ( α y (α k ) ( y (α) y s i(t) + α ) p ( α y s i(t) + α ) p ) + s i (α) ) + s i(α)

19 DDEs - Proper Handling of Sensitivity Jumps The Design and Implementation of a Modeling Package p.16/20 The predator-prey model y t y 1 τ t

20 DDEs - Proper Handling of Sensitivity Jumps The Design and Implementation of a Modeling Package p.16/20 Using the Theory of Hybrid ODE systems [Tolsma & Barton] Sensitivity Update Equation, where y (λ + r+1 p ) = y (λ r+1 l p ) + ( y (λ r+1 ) y (λ + r+1 )) λ r+1 (p) l p l λ r+1 (p) p l = α y λ 1 (p) p l = t 0(p) p l y p l + α p l λ r(p) p l α y y + α t for r 1

21 DDEs - Parameter Estimation The Design and Implementation of a Modeling Package p.17/20 Algorithms for Nonlinear Least-Squares Unconstrained min p W(p) = i [Y (γ i ) y(γ i ;p)] 2. Levenberg-Marquardt Variations of Sequential Quadratic Programming (SQP) Constrained min p W(p) = i [Y (γ i ) y(γ i ;p)] 2, c j (p) = 0, j E, c j (p) 0, j I. Sequential Quadratic Programming (SQP) The smoothness of functions involved in the problem, the objective function and constraints, is a necessary assumption.

22 DDEs - Parameter Estimation The Design and Implementation of a Modeling Package p.17/20 Computing the Gradient/Hessian ( 2 ) W(p) p l p m ±± = 2 i ( ) W(p) p l ± = 2 i [ ( ) y(γi ;p) p l ± ( ) y(γi ;p) [Y (γ i ) y(γ i ;p)] p l ( ) y(γi ;p) p m Recall the jump equation for sensitivities ± ( 2 ) y(γ i ;p) [Y (γ i ) y(γ i ;p)] p l p m y p l (λ + r+1 ) = y p l (λ r+1 ) + ( y (λ r+1 ) y (λ + r+1 )) λ r+1 (p) p l The General Rule : A jump occurs in W(p) when a discontinuity point λ r+1 (p) passes a data point γ i. ± ±± ]

23 DDEs - Parameter Estimation The Design and Implementation of a Modeling Package p.17/20 Avoiding The Non-smoothness y true solution continiously extended solution discontinuity point data point λ r (p) γ i λ r+1 (p) t y λ r (p) γ i λ r+1 (p) t Force the ordering by adding λ r (p) γ i λ r+1 (p) to the set of constraints. The partial derivatives (gradient) of the new constraints, λ r+1(p), can be computed recursively. p

24 Software Design The Design and Implementation of a Modeling Package p.18/20 User Calls

25 Software Design The Design and Implementation of a Modeling Package p.18/20 Simulator

26 Software Design The Design and Implementation of a Modeling Package p.18/20 Sensitivity Analyzer

27 Software Design The Design and Implementation of a Modeling Package p.18/20 Parameter Estimator

28 User Interface The Design and Implementation of a Modeling Package p.19/20 Simulation

29 User Interface The Design and Implementation of a Modeling Package p.19/20 Sensitivity Analysis

30 User Interface The Design and Implementation of a Modeling Package p.19/20 Parameter Estimation

31 Difficulties of Designing a Modeling Package - Revisited The Design and Implementation of a Modeling Package p.20/20 We need to develop an efficient simulator, a sensitivity analyzer and a parameter estimator Time Consuming/Expensive How to deal with the "Generality - vs - Efficiency" problem? Want to have a manageable design with little effort be able to change some algorithms or add new algorithms. Want to have a user-friendly interface Easy enough for nonspecialists and controllable enough for specialist. Ideally Parallelizable At least thread safe No Global Variables. Easily incorporable into other packages/programms Easy to do experiments, for instance, comparing different algorithms No Assumption for Global Modules.