Investigating an automated method for the sensitivity analysis of functions

Transcription

1 Investigating an automated metod for te sensitivity analysis of functions Sibel EKER Jill SLINGER Delft University of Tecnology 2628 BX, Delft, te Neterlands Gonenc YUCEL ABSTRACT Automated sensitivity analysis approaces in system dynamics focus primarily on model parameters. Altoug table functions are often subjectively approximated, tey do not form te focus of most sensitivity analyses. Recently, a promising approac tat allows automation of sensitivity analysis on functions was proposed by Hearne (2), but te applicability of tis metod to system dynamics table functions as not been studied, yet. In tis study, te metod is applied to a simple system dynamics model. In te ligt of te observations a number of sortcomings are identified and a set of extensions to address tese are proposed and ten tested. Te results of experiments wit te original and te extended metod demonstrate tat te metod can be used easily and efficiently for table functions. Te extensions are sown to be valuable in creating a more compreensive metod, but tey also raise te researc issue of te tradeoff between teir added value and te cost of dealing wit increased complication. Apart from our experimental results, te article also puts fort a set of directions along wic te approac can be improved furter. Despite te issues requiring furter researc, te metod olds promise for routine implementation. KEY WORDS: function sensitivity analysis, automated sensitivity analysis, epidemic model, triangular functions, uncertainty, table functions, system dynamics. INTRODUCTION Te problems modeled using system dynamics are caracterized by uncertainty, arising from a lack of information on te system itself or due to conflicting opinions of te actors involved. Tis real life uncertainty is reflected in model building as uncertainty in te selection of parameter values or in expressions for variable interactions. Despite te difficulties associated wit formulating tem, models ave to be robust/insensitive to suc uncertainties to be considered valuable and useful in solving real life problems. Sensitivity analysis is broadly defined as te study of model responses to model canges (Tank-Nielsen 98), and provides us wit a way to deal wit uncertainty. Elaborately, te importance of sensitivity analysis stems mainly from four factors. Firstly, it enances understanding about bot te structure-beavior relationsip in te

2 model and real world. Secondly, and as te main reason for its common usage, it sows te effects of uncertainties in te model, e.g. estimated parameters, on te conclusions derived from tis model. Tirdly, it determines te parameters wic affect te beavior strongly and so facilitates te devotion of limited resources to estimate tem. Lastly, te parameters wic ave a strong effect on te beavior may be te key points upon wic policies can be built (Sterman 2, 83, Tank-Nielsen 98). However, system dynamics as been criticized in te past owing to te absence of a precise teory or metod to conduct sensitivity analysis (Meadows 98). Following te explication and initial refutation of tis critique by Meadows, various approaces ave been developed. Several autors addressed te importance of sensitivity analysis, and provided basic non-automated approaces to conduct it. In teir early work, Ford, Amlin and Backus (983) empasized te difficulties of sensitivity analysis suc as large number of parameters to cange and large number of state variables wose responses are to be observed; and listed te approaces developed by tat time. Later on, studies are focused on coping wit tese problems. One of te two main approaces was to use optimization to determine parameter values wic cause maximum deviation from te original model beavior (J. Hearne 987, Miller 998). Te second approac wic attracted more attention was te use of statistics to determine combinations of parameter values in multivariate sensitivity analysis, and to interpret te responses of state variables (Kleijnen 995, Clemson, et al. 995). Anoter example of use of statistics was presented by Ford and Flynn (25) were tey screened te model and utilized te correlation coefficients to find te parameters most influential on te beavior. Meanwile, te progress in software tecnology allowed system dynamics and simulation packages to include automated sensitivity analysis tools. Tis reduced te burden of a compreensive sensitivity analysis to some extent, but te abovementioned statistical approaces are still needed. In addition to tose, since te sensitivity is defined as beavioral sensitivity rater tan numerical sensitivity in system dynamics, metods to recognize canges in te beavior patterns ave been developed and incorporated wit sensitivity analysis. Recent examples of suc studies can be seen in (Yucel and Barlas 2) and (Hekimoglu and Barlas 2). One common aspect of tese studies is tat tey deal primarily wit sensitivity of te model to canges in parameter values or to te canges in some model structure features suc as te model boundary or te level of aggregation (Sterman 2, 884). In system dynamics, nonlinear relationsips between two variables are usually specified by a lookup or table function, wic sows ow te dependent variable nonlinearly varies as te independent variable canges (Sterman 2, 552). In te context of model building, table functions are sometimes considered as parameters (Tank-Nielsen 98), but te attention paid to tem even in parameter sensitivity analysis is very limited. However, model robustness can depend on te coices of table functions as well because te sapes of tem are only approximations of real nonlinear relationsips, and due to tat sensitivity analysis is indeed te final step of formulating a table function (Sterman 2, ). Currently, sensitivity analysis activities, particularly automated analyses, focus on parameter sensitivity. Even if a modeler wonders about te outcome of is coices and carries out a manual sensitivity analysis for functions, tis effort will be limited because varying te functions manually, especially in combinations of multiple functions, is cumbersome. Clearly, an automated metod for function sensitivity analysis is required, before te analysis of te effects of uncertainty in functions can join parameter sensitivity analysis as a routinely applied tool. 2

3 Recently, Hearne (2) proposed a novel metod to conduct sensitivity analysis of model functions. Tis metod is based on systematic and parametric perturbation of te grapical functions. Despite being a promising proposal, tis approac as not yet been tested torougly nor as its wider applicability been establised. Te objective of tis study is two-fold. First, tis study aims to explore te applicability of Hearne s recent work on te sensitivity analysis of system dynamics functions. Tis is undertaken by implementing te metod on a simple model wit a single table function, and identifying te concerns and issues requiring attention. Second, extensions to te metod are proposed in order to overcome te identified sortcomings. Te delineated extensions are ten implemented and tested on te same model, and teir usefulness is assessed. Wit tis intention, before diving into te sensitivity analysis metod, te model tat is used in tis study is first introduced in te next section. Tereafter, te tird section deals wit Hearne s metod and its application on te cosen model. In Section 4, te proposed extensions are described and implemented. Te paper ends wit a discussion of te results and a conclusion regarding te promise of te metod. 2. THE SAMPLE MODEL Te model selected for testing te metod for te sensitivity analysis of functions is te basic form of te epidemics model (SIR model) wit an alternative infection rate formulation. In te original model (Sterman 2, 33) were S stands for susceptible population and I is te infected population. Te infection rate (IR) is defined as Te beavior yielded by tis formulation wen te contact rate is 6 people per person per day, te infectivity is.25, te initial number of Infected is and te Total Population is is depicted in Figure. person S I R 2. W2 W3 W4 W5 W6 Non-commercial use only! Figure : Te beavior of original SIR model over time In te alternative formulation te infection rate defined as 3

4 and f depicts te non-linear relationsip between te ratio of I to te total population and te infection rate. Tis relationsip is linear in te original formulation. To ensure similarity to te original formulation, te initial sape of tis table function is specified as depicted in Figure 2, wile te model beavior obtained wit tis function is depicted in Figure 3. Figure 2: Te table function f used in te alternative SIR model person S I R 2. W2 W3 W4 W5 W6 Non-commercial use only! Figure 3: Te beavior of alternative SIR model over time Two assumptions embedded in tis table function will be adopted in tis study, too. First, te function starts at (,), wic means tat wen tere are no infected people in te community, ten tere is no infection. In oter words, tis disease is transmitted only by contacts between people, and tis community is closed to interactions wit te outside world. Secondly, te maximum effect of te infected fraction on te infection rate is. Tis means tat, te infection rate cannot be iger tan te susceptible population. It is important to note tat tis claim does not imply tat te end point will be (,). Wen all or almost all people are infected, te effect of infected on infection rate may be smaller, and may even decline to zero. 4

5 3. APPLICATION OF HEARNE S METHOD TO THE ALTERNATIVE SIR MODEL In tis section, te metod of Jon Hearne will be explained, and ten te results obtained by using it on te abovementioned SIR model will be presented. 3.. Hearne s metod for function sensitivity analysis Te basis of Hearne s metod for function sensitivity analysis is te multiplication of a model function wit anoter function of specific form but variable parameterization in order to distort te model function. In is study, Hearne used triangular functions as distortion functions. Tecnically speaking, tese triangular functions ave te analytical and grapical forms sown below. For a model function r(y) defined on te interval [a, b], perturbation function (y, p, m) were m and p are parameters is given by: Here, it is important to note tat m stands for te maximum deviation from, wereas p is te point were tis deviation occurs. Since tis study does not aim at exploring te effect of different end points of model functions on te beavior, in tis paper te distortion function is formulated in suc a way tat no end point distortion occurs. Below, Figure 4 sows an example table function wic as an exponentially growing sape. If tis table function could ave an s-sape, almost linear or a less steep form, as R, R2 and R3 in Figures 6, 8 and respectively, ten suc sapes can be obtained by multiplying te original table function wit te distortion functions sown in Figures 5, 7 and 9. Wit eac possible combination of parameter values of te distortion function, a different table function sape can be obtained and a large span of possibilities can be explored.,6,4,2,8,6 r,4,2,,2,3,4,5,6,7,8,9 Figure 4: An example of a table function, r 5

6 4,5 4 3,5 3 2,5 2,5,5 Figure 5: Distortion Function wit m=,8 and p=,6 Figure 6: R, te distorted form of te table function r, distorted by 4,5 4 3,5 3 2,5 2,5,5,,2,3,4,5,6,7,8,9,,2,3,4,5,6,7,8,9 Figure 7: Distortion Function 2 wit m= and p=,5 Figure 8: R2, te distorted form of te table function r, distorted by 2,2,8,6,4,2,,2,3,4,5,6,7,8,9,,2,3,4,5,6,7,8,9 2 3 Figure 9: Distortion Function 3 wit m=-,64 and p=,8 Figure : R3, te distorted form of te table function r, distorted by 3,8,6,4,2,8,6,4,2,6,4,2,8,6,4,2,6,4,2,8,6,4,2,,2,3,4,5,6,7,8,9,,2,3,4,5,6,7,8,9 R r R2 r R3 r To find te minimum distortion to te model function wic causes te undesired model beavior, Hearne formulates te problem as follows: In te solution (m *, p * ) to tis optimization problem, m * represents te magnitude of te distortion, and p * is te point at wic te function manifests most sensitivity to te canges. 6

7 Tis formulation provides te reason for terming tis approac an automated metod. Instead of making manual canges to te function, te sape of te function wic gives rise to undesired beavior can easily be found wit te aid of an optimization tool Experiments wit te original approac For te modified SIR model, tree types of model beavior are specified as te criteria by wic sensitivity will be determined. If tese beaviors are generated by minor canges in te table function, ten te model will be said to be igly sensitive to te sape of te table function. Te tree criteria and te rationale for coosing eac of tem are described subsequently. i. Maximum number of Infected people greater tan 3 in two monts In te original SIR model, te infected population reaces its maximum value of around 3 in te second week; wereas te peak value of around 2 in te alternative model wic includes te table function. Te table function sape wic will yield a similar maximum for infected people is sougt by defining te criterion as te maximum number of infected people sould exceed 3 witin two monts. ii. Maximum number of Infected people greater tan 6 in two monts Given te duration of infectivity specified in te model, recovery is quite rapid and te maximum number of people infected at any time over a two mont period is not very ig. Te ambitious criterion of exceeding 6 infected people in two monts is cosen to figure out if any cange in te table function can cause suc extreme beavior. iii. Number of susceptible people still greater tan 99 after year Tis model is intended to model te dynamics of an epidemic. Terefore, if tere is no infection occurring, te model cannot be considered to acieve its purpose. Tis criterion is cosen in an attempt to understand if te sape of te table function could cause te model to be invalid. Tree optimization problems, one for eac of tese criteria, are formulated as proposed by Hearne. An additional constraint is added to ensure tat te values of te table function lie between and, preventing infection rates tat are negative or iger tan te susceptible population. Teoretically, m can take any value; owever, since te table function values cannot exceed, or go below, it is unnecessary to searc for m in a large interval. Terefore, m values are assumed to be between -5 and 5. Te matematical models specific to eac optimization problem are included in Appendix I Results To find te function sapes wic create undesired model beaviors, te optimization problems described above ave been solved using te optimization tool of Powersim Studio 7. Optimal solutions are presented in Table. 7

8 Table : Results of applying Hearne's metod to te alternative SIR model CRITERION CRITERION 2 CRITERION 3 m,69 4,695 -,47 p,,238,,2,8,6,4,2,,2,3,4,5,6,7,8,9 2,5 2,5,5,,2,3,4,5,6,7,8,9 Figure 2: Table function distorted to meet criterion 2 Figure : Table function distorted to meet criterion,2,8,6,4,2,,2,3,4,5,6,7,8,9 Figure 3: Table function distorted to meet criterion 3 Figures, 2 and 3 sow ow te original table function is distorted to meet eac criterion wit te function defined by te respective parameters in Table. In Figure, it can be seen tat if te table function is canged to ave a convex sape rater tan a linearly increasing one, it ten causes te maximum infected population to be greater tan 3. Te relationsip depicted in Figure 2 means tat to ave maximum infected population greater tan 6, infection rate sould be more tan, even twice as muc as, te susceptible population wen te infected people are between 5-% of te total population. In Figure 3, we see tat infection does not occur and te susceptible population remains ig if te table function values are sligtly decreased. Te results generally indicate tat te model beavior is more sensitive to te table function in terms of te tird criterion, because te absolute maximum deviation from unity (m) is smaller in tat case. A feasible solution for te second optimization problem couldn t be found; instead, te result obtained after removing te constraint on te table function is given. Since suc a distortion does not ave a real life meaning, it can be said tat te beavior is insensitive to te table function in terms of criterion Discussion on te observations and identifying potential improvements Te undesired beavior in te first and tird cases was obtained by a distortion of te table function made using a triangular function. However, tis does not mean tat tese are te only function sape canges wic can generate te undesired beavior. For example, an s-saped pattern wic requires te original function to increase initially ten to decrease, or vice versa, could potentially ave created te same undesired beavior. As Hearne pointed out, triangular functions are not capable of making different canges on te model functions in two different intervals. Wit teir end 8

9 points at, tey cannot decrease te function on one side, wile increasing it on te oter. However, in order to treat uncertainty in functional form torougly, suc variations in perturbations are necessary. Terefore, an improvement to Hearne s metod lies in determining alternative forms of distortion functions tat are able to create suc two-sided distortions. Furtermore, it is observed tat tis metod unnecessarily makes distortions on te regions of table function wic are not used during te simulation. For te first criterion, te segment after.3 is not used, because te igest infected population is 3. Similarly, only te part before. is used for te tird criterion. Tis indicates tat te distortions after tose points proved unnecessary, but ad to be made owing to te sape of te distortion function. A furter improvement lies in avoiding tese unnecessary distortions. Tere is anoter reason for using suc functions for te distortion of table functions: Tis table function ends at (, ) and single-extreme triangular functions wit stable end points at can be used to perturb tem. On te oter and, for table functions passing troug (, ) and going furter, distorted functions sould also pass troug (, ) and to ensure tis, distortion function as to pass troug (, ) as well. It is not possible to acieve tis by a partial function wit two segments wose end points are. Terefore, it is inevitable to use functions wic can provide different distortions in different intervals. As it was said before, tis table function assumes tat te effect of infected population ratio on infection rate increases till. However, it may not be logical to expect all te susceptible people to become infected even if infected ratio is very ig. Terefore, tis table function may saturate below. Tis opinion in fact points out te general fact tat end points of table functions may also be subject to uncertainty. Also, variable end points increase te variety of function sapes tat can be obtained after perturbation. Hence, tis function sensitivity analysis metod sould include te possible variations at te end points as well. 4. EXTENSIONS TO HEARNE S METHOD In tis section, te formulation and implementation of te proposed extensions on te model will be explained. Ten, te results of te implementation will be described. 4.. Extensions 4... Double-extreme triangular functions In selecting distortion functions tere is a trade-off between teir ability to provide te desired variety of perturbations and te number of parameters tey ave. Te variety of perturbations is important for te reasons previously described. As for te number of parameters, te iger te number, te longer it takes to solve te optimization problem and te more difficult it is to interpret te impact of te parameters on te sape of te function. Considering tis trade-off, te simplest extension tat can be made to triangular functions in order to obtain two-sided distortions is to use double-extreme triangular functions, tat is piece-wise functions wit two extreme points instead of one. Suc functions can be defined using four parameters wit meanings similar to te parameters 9

10 of triangular functions, and tey yield two-sided distortions wen m and m 2 ave opposite signs. If tey ave te same sign, ten tey just increase te variety. Te definition and an example of tese functions, for a model function r(x) defined in interval [, ] are given below (see Figures 4 and 5): 3 2,5 2,5,5 Double Single,,2,3,4,5,6,7,8,9 Figure 4: An example of a double-extreme triangular distortion function wit m=.2, p=.2, m2=-.3 and p2=.7 ( Double is in Figure 5),8,6,4,2,8,6,4,2 R Double R Single,,2,3,4,5,6,7,8,9 Figure 5: Te function in Figure 4 distorted by te single and double-extreme functions in Figure 4 r Variable end points Wen te end points are allowed to vary, te distortion functions defined above will ave to be modified as follows, were l and u are te values of te table function at and, respectively: For single extreme triangular functions: For double extreme triangular functions:

11 It is wortwile to mention tat in case of variable end points, parameter m represents te maximum deviation from te lower or te closer end point, not te deviation from unity anymore Experiments wit te extended metod Te experimental setup for implementing te extensions is te same as explained in section 3.2. An optimization problem for eac criterion and an extension is defined. For double-extreme distortion functions, bot te cases in wic te end points are stable and in wic tey are variable are considered. Because te population is assumed isolated from external infection sources (see section 2), even te distorted table function is assumed to start at (,) and no cange is made to tis point. Yet, te lower end point of te distortion function is still assumed to be a variable and take values different from, because te beginning point also affects te sape of te function. Te matematical models specific to eac optimization problem are detailed in Appendix I. To ensure two-sided distortion using double triangular functions, wic was te reason for introducing tem in te first place, m and m 2 can be forced to ave opposite signs Results Te optimization problems are solved as before. Te optimal solution of eac problem can be seen in Table 2 below: Table 2: Results of applying Hearne's metod wit extensions to te alternative SIR model CRITERION CRITERION 2 CRITERION 3 SINGLE- EXTREME DISTORTION FUNCTION DOUBLE- EXTREME DISTORTION FUNCTION END POINTS STABLE AT (Original metod) VARIABLE END POINTS END POINTS STABLE AT VARIABLE END POINTS m,69 4,695 -,47 p,,238, m,725 4,8, p,,364,55 l,,,295 u,874,69,428 m,74 4,745 -,446 p,,42, m2 -,5,,2 p2,,,366 m 4,47 2,383, p,484,,23 m2 2,257 2,782, p2,957,729,556 l,736,,3 u,3,,869

12 Tere is no feasible solution wic makes te maximum infected population greater tan 6 in any of te cases. Te solutions above (in bold typeface) belong to te modified optimization problem witout te constraint on te values of te distorted table function. Te graps of te distortion functions wit tese parameters and ow tey perturb te table function can be seen in Appendix II. In te context of te modified SIR model, te results above, supported by te visualizations in Appendix II, are very similar to te results of te original metod. Tey indicate tat te model is moderately sensitive to canges in te sape of table function f wen te sensitivity criterion is tat te maximum value of infected population exceeds 3% of te total population. If tis tresold is increased, tat is, if te model is expected to demonstrate even less desirable beavior, ten it can be said to be totally insensitive, because te distortions required to produce suc beavior no longer ave real life interpretations. However, if te undesired beavior is tat te final susceptible population is greater tan 99% of te total population, ten te beavior can be said to be igly sensitive to te canges in te sape of f, because sligt decreases at te initial points of te table function are sufficient to keep te infection rate very low. In te context of metod extension, it is seen tat on tis model and for tese criteria, te single-extreme triangular function wit stable end points was sufficient to carry out te function sensitivity analyses. Te double-extreme functions cosen by te optimization algoritm act very similarly to te single-extreme ones, because two of teir four intervals are negligibly narrow. Wen te end points are allowed to vary, tey are cosen different from, but tese canges are not really effective, because tese parts of te table function are not used. Still, te extensions proposed ere are teoretically necessary because tey enlarge te solution space and so facilitate a fuller consideration of uncertainty. Furtermore, tese experiments are useful in determining te utility of te metod, delineating necessary improvements and demonstrating teir effects Discussion on te observations For some of te optimal solutions, te end point deviates from unity. However, tis final segment of te table function is never even used. A u value lower tan one is cosen by te optimization algoritm eiter arbitrarily since it as no effect, or due to te ease of creating lower values for oter parts of te function wit a small end point. Terefore, before deriving te conclusion tat te beavior is sensitive to te canges in te end point of te table function if it falls below u, one needs to verify weter tis part of te function is used or weter tere is a significant effect on te part actually used. Furter, te variable end points add anoter level of complication to te problem. It is indeed straigtforward to include tem in te problem and tey are expected to stay at if variations don t cause a significant cange in te model beavior. However, as in te case of te double-extreme distortion function and te first criterion, altoug it is known tat tere is a feasible solution wen te end points are, te searc algoritm does not find tis. To overcome tis difficulty, eiter te searc process can be extended, wic brings te dilemma of quality versus time and effort; or anoter optimization tool can be used, wic decreases te efficiency of te analysis procedure by separating te environments in wic te optimization and simulation are executed. 2

13 Moreover, altoug double-extreme triangular functions are able to generate two-sided distortions, teir perturbation variety is still limited. To increase tis variety, furter functional forms, suc as cubic polynomials or sinusoidal functions, can be tried out; and te added value of aving tis variety against te complications and difficulty of interpretation caused by te iger number of parameters, can be assessed. 5. DISCUSSION AND CONCLUSION In tis study, te applicability of Hearne s automated sensitivity analysis metod for system dynamics functions as been tested by applying te metod to a simple model containing a table function. Extensions to te metod ave been developed and implemented. Specific sortcomings of bot te original metod and te extended version are delineated in sections 3.4 and 4.4. In addition to tese, tere are some general issues tat need to be addressed. First, te claims made about te table function s range in use sould be generalized, because it is not unusual for models to use only a limited part of te table functions under given initial conditions and parameter values. However, it is not possible to capture te exact end points of te domain interval used in runtime, and to make distortions only in tis interval. Terefore, it is necessary to ceck te active domain of te table functions, and ow muc of te distortions are really used in te simulation before deriving conclusions from te results obtained wit tis metod. Second, in tis study, undesired beaviors were generated by single-extreme functions in only one direction. To fully understand te added value of double-extreme functions, different undesired beavior types sould be determined and tested. Also, one may argue tat sensitivity in tis study is defined more in terms of numerical canges ere, rater tan in terms of significant beavioral canges. Tis claim is undeniably true; but wit an optimization tool in wic te constraints can be applied only once during te simulation, not continuously, it is not straigtforward to capture beavioral canges. Indeed, automated beavioral cange analysis metods suc as pattern recognition would need to be incorporated in te extended Hearne metod if te goal of creating an automated sensitivity analysis metod for model functions is to be attained. Tird, in tis paper te metod is applied on a simple model wit a single table function, and undesired beaviors are selected on te basis of te beavior of a single stock variable. If te metod is used on a larger model to create simultaneous distortions on multiple table functions and criteria dependent on combinations of several stock variables are defined, ten te value of tis automated metod can be establised. Bot te original and te extended version of Hearne s function sensitivity analysis metod were applied using te optimization tools available in system dynamics software, demonstrating te promise of te metod to deliver automated sensitivity analysis of functions in system dynamics models. A number of points wic need attention if tis metod is to be used routinely for table functions sensitivity analysis provide te points of departure in future work on tis issue. 3

14 REFERENCES Clemson, Barry, Yongming Tang, James Pyne, and Resit Unal. "Efficient metods for sensitivity analysis." System Dynamics Review, no. (995). Ford, A, J.S. Amlin, and G. Backus. "A Practical Approac to Sensitivity Testing of System Dynamics Models." Proceedings of te 983 International System Dynamics Conference. Cestnut Hill, MA, 983: Ford, A., and H Flynn. "Statistical Screening of System Dynamics Models." System Dynamics Review, 25. Hearne, J, W. "An Automated Metod for Extending Sensitivity Analysis to Model Functions." Natural Resource Modleing, 2. Hearne, J.W. "An approac to resolving parameter sensitivity problem is system dynamics metodology." Applied Matematical Modelling, 987. Hekimoglu, Mustafa, and Yaman Barlas. "Sensitivity Analysis of System Dynamics Models by Beavior Pattern Measures." Proceedings of te 28t International Conference of te System Dynamics Society. 2. Kleijnen, J.P.C. "Sensitivity analysis and optimization of system dynamics models: regression analysis and statistical design of experiments." System Dynamics Review, 995. Meadows, D. "Te Unavoidable a Priori." In Elements of te System Dynamics Metod, by Jørgen Randers. Cambridge, Massacusetts, USA: Productivity Press, 98. Miller, J.H. "Active Nonlinear Tests (ANTs) of Complex Simulation Models." Management Science, 998. Sterman, J. Business Dynamics: Systems Tinking and Modeling for a Complex World. Irwin/McGraw-Hill, 2. Tank-Nielsen, T. "Sensitivity Analysis in System Dynamics." In Elements of te System Dynamics Metod, by Jørgen Randers. Cambridge, Massacusetts, USA: Productivity Press, 98. Yucel, Gonenc, and Yaman Barlas. "Automated Parameter Specification in Dynamic Feedback Models Based on Beavior Pattern Features." System Dynamics Review 27, no. 2 (2):

15 APPENDIX I: OPTIMIZATION PROBLEMS. Single extreme, stable end points, model beavior () i s.t. (2) x (3) (4) (5) were InfectionFunction[i] = InfectionFunction[i] * [i] InfectionFunction[] = {,.9,.2,.29,.46,.5,.66,.69,.794,.96, } were () Objective function: Te optimization problem for sensitivity analysis is stated s fi di g te i i u distortio wic cre tes te u desired odel be vior. Terefore, te distortio is e sured by te squ re of xi u deviation of te distortion function from unity, to ensure tat absolute distortion is taken into account. (2) Undesired model beavior constraint: Te first undesired model beavior criterion is formulated as a constraint wic restricts te maximum value of te Infected population over te simulation time to be greater tan 3. In Powersim, tis constraint was defined by using RUNMAX() function. (3) Feasibility constraint: It is known tat te values below and above ave no real life meaning for te table function used in tis study. However, it is not possible to set a constraint to ceck te distorted values at eac time point. Terefore, an array of discrete values of te distorted table function, namely InfectionFunction[], is created and restricted to te interval of [, ], altoug it is known tat te distortions are continuous. Eac element of InfectionFunction[] is formed by multiplying eac element of te original distortion function array (InfectionFunction[]) wit te corresponding value of te distortion function([i]). (4) Searc range of p: As it can be recalled, parameter p represents te point were te maximum distortion from unity occurs. Terefore, it as to witin te domain of table function, wic is [, ] in tis case. (5) Searc range of m: Tere is no limit on te values of m, but it is anticipated tat ig values cause infeasible distortions. Terefore, te searc range is specified as [-5, 5] to include downward distortions as well. 2. Single extreme, stable end points, model beavior 2 Problem, except te first constraint wic is replaced by 3. Single extreme, stable end points, model beavior 3 Problem, except te first constraint wic is replaced by 5

16 4. Single extreme, variable end points, model beavior () s.t. (2) (3) (4) (5) (6) (7) were InfectionFunction[i] = InfectionFunction[i] * [i] InfectionFunction[] = {,.9,.2,.29,.46,.5,.66,.69,.794,.96, } (6) Searc range of l: In addition to te problem, searc ranges for te end point parameters are specified in tis case. For l, te end point at, tis range is [, 2] because it is tougt tat te values iger tan 2 would easily cause infeasibility. (7) Searc range of u: Te end point of te table function at is already, and since multiplying it wit values iger tan would cause infeasible values, te searc range of u is kept between and. 5. Single extreme, variable end points, model beavior 2 Problem 4, except te first constraint wic is replaced by 6. Single extreme, variable end points, model beavior 3 Problem 4, except te first constraint wic is replaced by 7. Double extreme, stable end points, model beavior () (2) (3) (4) (5) (6) were s.t. InfectionFunction[i] = InfectionFunction[i] * [i] 6

17 InfectionFunction[] = {,.9,.2,.29,.46,.5,.66,.69,.794,.96, } () Objective function: Tis time, two parameters, bot m and m 2 contribute to te maximum distortion, ence tey are bot included in te objective function. (4) Searc range of p and p 2: Te searc range [,] applies to bot p parameters. (5) Searc range of m and m 2: As specified in te first problem, te searc range [- 5, 5] applies to bot m parameters. (6) Difference of p s: Te distortion function is defined based on te assumption tat p precedes p 2. Wit tis constraint, it is guaranteed tat p is smaller tan p Double extreme, stable end points, model beavior 2 Problem 7, except te first constraint wic is replaced by 9. Double extreme, stable end points, model beavior 3 Problem 5, except te first constraint wic is replaced by. Double extreme, variable end points, model beavior () i s.t. (2) x (3) (4) (5) (6) (7) (8) were InfectionFunction[i] = InfectionFunction[i] * [i] InfectionFunction[] = {,.9,.2,.29,.46,.5,.66,.69,.794,.96, } 7

18 (7) and (8) Searc ranges of end points: Te searc range of bot of te end points is specified as te interval [,] because te values out of tis interval are infeasible.. Double extreme, variable end points, model beavior 2 Problem, except te first constraint wic is replaced by 2. Double extreme, variable end points, model beavior 3 Problem, except te first constraint wic is replaced by 8

19 ,,2,3,4,5,6,7,8,9,,2,3,4,5,6,7,8,9,,2,3,4,5,6,7,8,9 APPENDIX II: EXPERIMENT RESULTS. Single-Extreme Triangular Distortion Function a. Stable End Points i. Criterion (Model beavior: Max(I)>3) m,69 p,,8,6,4,2,,8,6,4,2,,2,8,6,4,2,,2,3,4,5,6,7,8,9 6, ii. Criterion 2 (Model beavior: Max(I)>6 ) m 4,695 p,238 2,5 5, 2 4, 3, 2,,,,5,5,,2,3,4,5,6,7,8,9 iii. Criterion 3 (Model beavior: Final S>99) m -,47 p,,2,2,,8,6,4,2,,8,6,4,2,,2,3,4,5,6,7,8,9 9

20 ,,2,3,4,5,6,7,8,9,,2,3,4,5,6,7,8,9,,2,3,4,5,6,7,8,9 b. Variable End Points i. Criterion (Model beavior: Max(I)>3) m,725 p, l, u,874,8,6,4,2,,8,6,4,2,,2,8,6,4,2,,2,3,4,5,6,7,8,9 ii. Criterion 2 (Model beavior: Max(I)>6 ) m 4,8 p,364 l, u,69 6, 3 5, 2,5 4, 3, 2,,, 2,5,5,,2,3,4,5,6,7,8,9 iii. Criterion 3 (Model beavior: Final S>99) m, p,55 l,295 u,428,45,4,35,3,25,2,5,,5,,2,8,6,4,2,,2,3,4,5,6,7,8,9 2

21 ,,2,3,4,5,6,7,8,9,,2,3,4,5,6,7,8,9,,2,3,4,5,6,7,8,9 2. Double-Extreme Triangular Distortion Function a. Stable End Points i. Criterion (Model beavior: Max(I)>3) m,74 p, m2 -,5 p2,,8,6,4,2,,8,6,4,2,,2,8,6,4,2,,2,3,4,5,6,7,8,9 ii. Criterion 2 (Model beavior: Max(I)>6 ) m 4,745 p,42 m2, p2, 7, 3 6, 5, 4, 3, 2,, 2,5 2,5,5,,,2,3,4,5,6,7,8,9 iii. Criterion 3 (Model beavior: Final S>99) m -,446 p, m2,2 p2,366,2,2,,8,8,6,4,2,6,4,2,,,2,3,4,5,6,7,8,9 2

22 ,,2,3,4,5,6,7,8,9,,2,3,4,5,6,7,8,9 b. Variable End Points i. Criterion (Model beavior: Max(I)>3) m 4,47 p,484 m2 2,257 p2,957 l,736 u,3 2,,2,5,,5,8,6,4,2 Origin al Distor ted,,,2,3,4,5,6,7,8,9,,2,3,4,5,6,7,8,9 ii. Criterion 2 (Model beavior: Max(I)>6 ) m 2,383 p, m2 2,782 p2,729 l, u, 4, 3,5 3, 2,5 2,,5,,5, 3 2,5 2,5,5,,2,3,4,5,6,7,8,9 iii. Criterion 3 (Model beavior: Final S>99) m, p,23 m2, p2,556 l,3 u,869,,2,8,6,4,2,8,6,4,2,,,2,3,4,5,6,7,8,9 22