MIPE: Model Informing Probability of Eradication of non-indigenous aquatic species. User Manual. Version 2.4

Transcription

1 MIPE: Model Informing Probability of Eradication of non-indigenous aquatic species User Manual Version 2.4 March

2 Table of content Introduction 3 Installation 3 Using MIPE 3 Case study data 3 Input window 4 Main result window 5 ROC curve 6 Model selection 7 Main effects 8 Plots of main effects 9 Exporting the results 10 License information 10 2

3 Introduction MIPE v2.4 is an evidence-based tool designed to help management of aquatic non-indigenous species (ANIS). It provides a manager with an estimated probability of eradicating an ANIS given the characteristics of an infestation and the details of the planned intervention. MIPE uses a statistical model, fitted to a review of case studies, to predict the outcome of planned intervention with associated uncertainty. The output of MIPE can be used to refine the precision of cost-benefit analyses evaluating various intervention scenarios. Installation MIPE runs in a Windows environment and requires Microsoft Excel and ~1 Gb of hard drive space. MIPE was developed using MATLAB r2012b and uses several functions included in the statistical toolbox. Although it was compiled as a single executable file, it is not a stand-alone application if the appropriate MATLAB applications are not installed on a computer. Therefore, most users will need to install MATLAB Compiler Runtime (MCR). It is important to select the proper version, which is release R2012b (8.0) 32-bit (this is true even if the computer runs a 64- bit version of Windows). The MCR is available for free at the following URL: Once MCR is installed, there is no need to open it; it should run in the background automatically. Once this is done, simply open MIPE by double-clicking the MIPE_v2_4.exe file. NOTE: on the first use, the application can take a long time to open. Using MIPE Case study data set MIPE uses a statistical model as a backbone; it therefore relies on a data set of case studies. The data set is provided on the download page and is essential for the application to run. Every time a user hits the run button, the software will fit and evaluate the most parsimonious model from the case studies included in the.xls file. It is thus possible for a user to modify the data set by adding of removing case studies as long as the structure of the data file is not altered. In other words, 3

4 users can add rows as long as the columns are not changed. The application only uses the numerical variable columns; the rest of the columns (reference, location, notes, etc ) are there for transparency and replicability. Input window This is the window where the user inputs the information required by the software to perform the calculations. First, the user is asked to retrieve the Excel sheet containing the data. This is done by pressing the browse button and locating the file (A). Then the user needs to input the characteristics of the situation being faced by either selecting a variable using the drop menus (e.g. B) or inputting a number for continuous variables (e.g. C). If a parameter is unknown, the variable can be left out of the calculations entirely by unchecking the checkbox next to each variable (e.g. D). 4

5 Main result window Once the user inputted the data, hitting the run button will trigger the model selection and evaluation routines. Once the calculations are done, the main result window will pop up. In the upper panel (A), the probability of eradication and 95% confidence limits are returned. The middle panel returns the information about the predictive power of the model. This is done for the jackknife evaluation (B) and the receiver operating characteristics (ROC) evaluation (C). The jackknife evaluation returns the predicted probability of eradication of each case study (that either resulted in eradication or non-eradication), evaluated by a model fitted using every datum except the pertaining case study. The red lines represent the current spectrum of probability of eradication. The area under the ROC curve (AUC), representing the probability that an eradication event is ranked higher than a non-eradication event, is returned. From the main result window, the user has access to several other options described below. 5

6 A) ROC curve This option returns the ROC curve for the final model used for prediction. The blue stepped line represents the true positive rate (y-axis; sensitivity: the proportion of case studies that were properly assigned to the eradication category) and the false positive rate (x-axis; 1-specificity: the proportion of case studies that were wrongfully assigned to the eradication category) at varying thresholds (predicted probability). The diagonal dashed line represents the reference when a model has no predictive power, i.e. a model that assigns case studies to categories randomly. The blue dot represents the optimal probability threshold, i.e. the probability threshold that minimizes the false positive and false negative rates. This probability threshold is returned to the left (A). Among the case studies included in the data set, when the predicted probability is above the threshold, they resulted in eradication in a proportion given by the true positive rate (B). When the predicted probability is below the threshold, the proportion that resulted in eradication is given by the false positive rate (C). 6

7 B) Model selection The AIC info button returns the evaluation of every possible model (every combination of factors) and the factors included in the final model. The top table (A) returns the daic value (difference in AIC value from the top model), AIC weight (likelihood that model is the best model) and AUC (area under the ROC curve) for each combination of factors (1 if included and 0 if excluded). Models are ranked based on AIC value from lowest to highest. The bottom table (B) returns the factors included in the final model (1 if included and 0 if excluded). The final model is the most parsimonious and the one used for prediction. 7

8 C) Main effects The main effects button returns a table with the parameters and statistics for each variable retained in the final model. For each factor, we report the estimated parameter, its standard error, a t-value and a p-value. Note that for categorical variables the first category is excluded and is used as a reference category (parameter set at 0). Thus the p-value for the other categories is in comparison to the reference. 8

9 D) Plots of main effects The Plot button provides the possibility to produce a plot of any main effect. The button will make a window pop up, in which the user has to select the variable of interest (A). Checking one variable (only one at the time), and hitting OK will make the plot appear (B). The dependent variable for each plot is the deviance (a relative of residuals) from a partial model including every factor (of those retained in the final model) except the factor of interest. The horizontal lines in plots of categorical variables represent the mean deviance for a category. If the mean value is smaller than zero for a particular category, this means that this category has a lower probability of eradication when compared to the overall mean (and vice-versa). For continuous variables, the solid line represents the line of best fit between the independent variable and the deviance. 9

10 E) Exporting the results The Export button allows the user to produce an Excel spreadsheet to facilitate communication with collaborators. The spreadsheet contains the most important information including the factors retained in the final model, the input of characteristics selected by user, and the predicted probability of eradication and associated confidence intervals. Note that an error message may appear if Excel is already opened. If this happens, just close Excel and hit the button again. Once opened, the spreadsheet can be saved under a different name. License information MIPE is free to use, share, or modify. 10