Package CompModSA. February 14, 2012

Transcription

1 Package CompModSA February 14, 2012 Type Package Title Sensitivity Analysis for Complex Computer Models Version Date Author Curtis Storlie with contributions from Alexandre Janon Maintainer Curtis Storlie Alexandre Janon Depends R (>= 2.4.1), quadprog, tgp, gam, locfit, rpart, mda, corpcor,randomforest, gbm, MASS, mlegp, polspline, fields, methods,lars Description Uses regression surface approximations to calculate variance decomposition and total variance sensitivity indices. This package is useful for conducting sensitivity analysis of complex computer codes when model evaluations are somewhat expensive (e.g. take longer than a couple seconds to run) but a reasonable number (50 or more) of model evaluations can be obtained at sampled input values. License GPL (>= 2) Repository CRAN Date/Publication :08:34 R topics documented: CompModSA-package predict.sensitivity print.sensitivity sensitivity Index 9 1

2 2 predict.sensitivity CompModSA-package Sensitivity Analysis for Complex Computer Models Description Details Uses regression surface approximations to calculate sensitivity measures. This package is useful for conducting sensitivity analysis of complex computer codes when model evaluations are somewhat expensive but a reasonable number (~50 or more) of model evaluations can be obtained at sampled input values. The primary function in this package is sensitivity. Type?sensitivity for information on its usage. predict.sensitivity Prediction at New Input Values Description Usage This function is used to predict the computer model output at unobserved input locations using an surrogate model approximation; see sensitivity for more details. predict.sensitivity(object, X.new, control,...) Arguments Value object X.new control a sensitivity object a dataframe or matrix. Each row is a new input value. Each column represents an input variable. Columns should be the same variables as those in x.loc from the original call to sensitivity. list of control parameters as in sensitivity.... further arguments passed to or from other methods. a list with length(y.loc) elements corresponding to each y variable used in the original call to sensitivity. Each element of the list is a vector of length nrow(x.new) containing the predicted y values. Author(s) Curtis Storlie <storlie@stat.unm.edu>

3 print.sensitivity 3 See Also sensitivity Examples ## Not run: ## Read in some data for illustration ex.dat <- read.table(" header = TRUE) ans.rs <- sensitivity(ex.dat, x.pos=1:10, y.pos=11, surface= rs.reg ) ## Generate some new X values to predict at X.new <- matrix(runif(100), nrow=10) yhat <- predict(ans.rs, X.new) ## End(Not run) print.sensitivity Print a Sensitivity Object Description Generate and/or export a summary table for a sensitivity object to a file. Usage print.sensitivity(x, fname, control,...) Arguments x a sensitivity object fname the file name to export the summary to. The default will print to the screen. control control parameters as in sensitivity.... further arguments passed to or from other methods. Author(s) Curtis Storlie <storlie@stat.unm.edu> See Also sensitivity

4 4 sensitivity Examples ## Not run: ## Read in some data for illustration ex.dat <- read.table(" header = TRUE) ## run a sensitivity analysis using quadratic regression ## on the inputs in columns 1-10 and output in columns of ex.dat. ans.rs <- sensitivity(ex.dat, x.pos=1:10, y.pos=11:12, surface= rs.reg ) ## Print the results print(ans.rs) ## End(Not run) sensitivity Sensitivity Analysis for Complex Computer Models Description Usage Perform a sensitivity analysis by fitting regression surface(s) (of varying complexity) to output from a complex model. This package is useful for conducting sensitivity analysis of complex computer codes when model evaluations are somewhat expensive but a reasonable number (~50 or more) of model evaluations can be obtained at sampled input values. sensitivity(data, x.pos, y.pos, surface= auto, control) Arguments data x.pos y.pos surface control a matrix or data frame where the columns contain the input variables and the output variables of an analysis. Each row corresponds to a run of a computer code for example. a vector containing the column locations in sens.dat of the input variables used to generate model outputs. a vector containing the column locations in sens.dat of the output variables given by the model. which regression surface method to fit to the model output. Options are reg (linear regression), rank (rank regression), rs.reg (quadratic response surface regression), loc.reg (local regression), ppr (projection pursuit regression), mars (Multivariate Adaptive Regression Splines), tree (Recursive Partitioning Regression), acosso (Adaptive COSSO), gp (Bayesian Gaussian Process), and tgp (Treed Gaussian Process). The value can also be specified as a character vector of length(y.pos), giving the desired surface to be fit to each of the outputs. The default is auto which successively fits models in the order given by the try.surface control parameter until R^2 > min.rsq or the last model is fit. a list with values that control the surface fitting and sensitivity index calculation. Elements of control are descibed below:

5 sensitivity 5 maxterms- the maximum number of input variables allowed into the model; default = 20. crit- the criterion to use for which variables enter or leave the model during stepwise construction. Options are gcv (the default) or pval. Only applicable when surface equals reg, rs.reg, loc.reg, or ppr. alpha- the cut-off to use if crit== pval ; default =.02. gcvpen- the penalty to use for each degree of freedom when calculating GCV score. Only applicable when surface = mars or tree, default = 2. maxsize- the maximum number of basis functions allowed in mars model; default = 200. minsplit- Argument passed to rpart to build a regression tree model; the minimum number of observations that must exist in a node, in order for a split to be attempted; default = 10. int.order- The order of interactions to consider for acosso and mars models. Currently supports int.order = 1 (additive model) and int.order = 2 (two-way interaction model); default = 2. wt.pow- the power given to the intial estimate of L2 norm for acosso ; default = 2. acosso.cv- the criterion used for smoothing parameter selection in acosso. Options are bic (the default), gcv, and 5cv. BTE- 3-vector of MCMC parameters for gp and tgp. (B)urn in, (T)otal, and (E)very. Predictive samples are saved every E MCMC rounds starting at round B, stopping at T; default = c(1000, 4000, 2). n.deriv- a smoothness parameter for the prior distribution of the surface for gp and tgp. The resulting surface will be n.deriv times differentiable (within each node); default = 2. distn- The distribution to assume on the input variables for Monte Carlo (MC) calculation of S.index and T.index. Currently supports emp (use empirical distribution of the inputs) and unif (use uniform distribution). Either way, inputs are currently assumed independent for the purposes of calculation; default = emp. n.mc.s- number of MC sample points to calculate variance in the quantitiy Var[E(y x_1,...,x_j-1,x_- j+1,...x_p)] required to obtain S_j; default = n.mc.t- number of MC sample points to calculate variance in the quantitiy Var[E(y x_1,...,x_j-1,x_- j+1,...x_p)] required to obtain T_j; default = n.samples- MC integration to calculate of S_j and T_j is repeated n.samples times and the average is used. This allows for better accuracy while using less memory than simply using larger values of n.mc.t for example; default = 10. CI- TRUE or FALSE; should bootstrap confidence intervals (or credible sets for gp and tgp ) be calculated for the T_j s or the S_j s (see CI.S ) ; default = FALSE. Other options are control$ci = 1 which will use the nonparametric or "naive" bootstrap. control$ci = 2 or control$ci=true will use a parametric bootstrap with resampling on the x s as well. control$ci = "none" or control$ci=false will not do CI s.

6 6 sensitivity Details CI.S- TRUE or FALSE; is TRUE if bootstrap confidence intervals should be calculated for the main indices S_j s, or FALSE for confidence intervals to be for the total indices T_j s; CI.S=TRUE implies CI=TRUE; default = FALSE. n.ci- the number of bootstrap samples to use if creating confidence intervals (or number of realizations to use for gp and tgp ); default = 100. n.mc.ci- Same as n.mc.t, but applies to the calculation of T_j for each bootstrap sample (realization); default = n2.ci- Same as n2.t, but applies to the calculation of T_j for each bootstrap sample (realization); default = 20. alpha.ci- The confidence level used for the CI s is 100(1-alpha.CI)%; default =.05 max.disp.var- the maximum number of displayed variables in the output; default = 15. min.tul- the minimum value to display in the output for the upper confidence limit (UCL) on T. Variables with UCL < min.tul will not be displayed; default =.05 min.rsq- when using surface= auto, methods in try.surfaces will be fit until the model R^2 value is greater than min.rsq. If no method in try.surfaces has R^2 greater than min.rsq, the last surface to be fit is used; default =.90. try.surfaces- a list containing the surfaces to try (in order) when using surface= auto ; default = c( rank, rs.reg, tree, acosso ). categorical- a vector containing the column locations of data which correspond to categorical variables; default = auto which treats a variable as categorical if it has no more than min.distinct distinct values. min.distinct- see categorical above; default = 10. n.screen- If the number of input variables is greater than n.screen, then the methods listed in screen.surfaces are fit to the entire set of input variables. The collection of input variables to be used in the analysis is reduced to only those that show up in at least one of the models obtained. Using n.screen can be useful for gp or tgp for example since they perform much better when the set of input variables can be reduced; default = 15. screen.surfaces- a list containing the surfaces to use to screen variables before fitting the final regression surface; default = c( rs.reg, acosso ). Note: acosso and mars are applied with additive = TRUE. Performs a sensitivity analysis of a complex computer model by using a data set containing model evaluations at several sampled input values. If surface = reg, a stepwise regression is fit to the data and the corresponding standardized regression coefficients (src) s and partial correlation coefficients (pcc) s are obtained. If surface = rank stepwise regression is fit to the rank transformed output variables and the corresponding standardized rank regression coefficients (srrc) s and partial rank correlation coefficients (prcc) s are obtained. For all other surface options a surrogate model approximation is fit to the data set using the specified surface. For surface= rs.reg or loc.reg variable selection is achieved in a stepwise manner.

7 sensitivity 7 Variable selection is inherently part of the regression procedure when surface = tree mars, or acosso. Surfaces gp and tgp use all input variables when fitting the model. The following sensitivity indices are obtained using the surrogate model. The total variance index, T_j = Var(y)-Var[E(y x_1,...,x_j-1,x_j+1,...x_p)] / Var(y), is calulated. We also calculate a stepwise variance contribution, which we call S_j. This is calculated as follows. The first variable to "enter" the model is defined by x_j for the j that maximizes S_j,1 = Var[E(y x_j)] / Var(y). Call the x_j that maximized S_j,1, x_(1). maximizer of S_j,2 = Var[E(y x_(1),x_j)] / Var(y). The second variable to "enter" is then given by the Call the x_j that maximized S_j,2, x_(2). In general, the k^th variable to "enter" the model is then given by the maximizer of S_j,k = Var[E(y x_(1),x_(2),...,x_(k-1),x_j)] / Var(y). The corresponding S_j s provide a measure of the incremental increase in the percentage of variance explained at each step by including the uncertainty of the j^th variable in the model. The S_j s and T_j s defined above are evaluated by Monte Carlo (MC) sampling on the input (x) distribution while the output (y) at each sampled x value is evaluated via the surrogate model. These MC calculations can be done with a good deal of accuracy because the surrogate model is very fast to evaluate. Confidence intervals for the sensitivity indices (which include uncertainty from MC calculation of T_j and uncertainty in the surrogate model) can also be generated by bootstrapping (or from the posterior distribution of the gp or tgp methods) by setting the control parameter CI = TRUE. Value An object of type sensitivity with generic functions print and predict. Author(s) Curtis B. Storlie, <storlie@stat.unm.edu>. References CB Storlie, EA Baldwin, LP Swiler, JC Helton, and C Sallaberry (2007). Calculating Variance- Based Sensitivity Measures Using a Limited Number of Model Runs. CB Storlie and JC Helton (2007). Multiple Predictor Smoothing Methods for Sensitivity Analysis: Description of Techniques. Reliability Engineering and System Safety. A Saltelli, K Chan, and EM Scott (2000). Sensitivity analysis. Wiley. See Also predict.sensitivity, print.sensitivity

8 8 sensitivity Examples ## Not run: ## Read in some data for illustration ex.dat <- read.table(" header = TRUE) ## run a sensitivity analysis using quadratic regression ## on the inputs in columns 1-10 and output in columns of ex.dat. ans.rs <- sensitivity(ex.dat, x.pos=1:10, y.pos=11:12, surface= rs.reg ) print(ans.rs) ## run the same SA as above with bootstrap CI s for the T index. ans.boot <- sensitivity(ex.dat, x.pos=1:10, y.pos=11:12, surface= rs.reg, control=list(ci=true, n.ci=25, n.mc.ci=1000)) print(ans.boot) ## Automatically select surface(s) and specify control params for ## T.index calculation. ans.auto <- sensitivity(ex.dat, x.pos=1:10, y.pos=11:12, surface= auto, control=list(min.rsq=.95, CI=TRUE, n.ci=25)) print(ans.auto) ## End(Not run)

9 Index Topic smooth CompModSA-package, 2 predict.sensitivity, 2 print.sensitivity, 3 sensitivity, 4 CompModSA (CompModSA-package), 2 CompModSA-package, 2 predict.sensitivity, 2 print.sensitivity, 3 sensitivity, 3, 4 9