Package truncgof. R topics documented: February 20, 2015

Transcription

1 Type Package Title GoF tests allowing for left truncated data Version Date Author Thomas Wolter Package truncgof February 20, 2015 Maintainer Renato Vitolo Depends R (>= 2.4.0),MASS Goodness-of-fit tests and some adjusted eploratory tools allowing for left truncated data License GPL (>= 2) Repository CRAN Date/Publication :11:28 NeedsCompilation yes R topics documented: ad.test ad2.test ad2up.test adup.test cdens dplot edf ks.test mctest qplot v.test w2.test Inde 19 1

2 2 ad.test ad.test Supremum Class Anderson-Darling test Supremum class version of the Anderson-Darling test providing a comparison of a fitted distribution with the empirical distribution. ad.test(,, fit, = NA, alternative = c("two.sided", "less", "greater"), sim = 100, tol = 1e-04, estfun = NA) fit alternative sim tol estfun a numeric vector of data values character string naming the null distribution function list of null distribution parameters indicates the alternative hypothesis and must be one of "two.sided" (default), "less", or "greater". Initial letter must be specified only. maimum number of szenarios in the Monte-Carlo simulation if the difference of two subsequent p-value calculations is lower than tol the Monte-Carlo simulation stops an function as character string or NA (default). See mctest. The supremum class Anderson-Darling test compares the null distribution with the empirical distribution of the observed data. The test statistic is given by AD + = n sup{ z + j n (1 z ) z j j (zj z )(1 z j ) } AD = n sup{ z j (z + j 1 n (1 z )) } j (zj z )(1 z j ) AD = ma{ad +, AD }. with z = F θ () and z j = F θ ( j ), where 1,..., n are the ordered data values. ere, F θ is the null distribution.

3 ad2.test 3 A list with class "mchtest" containing the following components statistic treshold p.value data.name method alternative sim.no the value of the Supremum Class Anderson-Darling statistic the treshold value the p-value of the test a character string giving the name of the data the character string "Supremum Class Anderson-Darling test" the alternative number of simulated szenarios in the Monte-Carlo simulation References Chernobay, A., Rachev, S., Fabozzi, F. (2005), Composites goodness-of-fit tests for left-truncated loss samples, Tech. rep., University of Calivornia Santa Barbara ks.test, v.test, adup.test for other supremum class tests and ad2.test, ad2up.test, w2.test for quadratic class tests. For more details see mctest. c <- rlnorm(100, 2, 2) # complete sample t <- c[c >= treshold] # left truncated sample ad.test(t, "plnorm", list(meanlog = 2, sdlog = 2), = 10) ad2.test Quadratic Class Anderson-Darling test Quadratic class Anderson-Darling test providing a comparison of a fitted distribution with the empirical distribution. ad2.test(,, fit, = NA, sim = 100, tol = 1e-04, estfun = NA)

4 4 ad2.test fit sim tol estfun a numeric vector of data values character string naming the null distribution list of null distribution parameters maimum number of szenarios in the Monte-Carlo simulation if the difference of two subsequent p-value calculations is lower than tol the Monte-Carlo simulation is discontinued an function as character string or NA (default). See mctest. The Anderson-Darling test compares the null distribution with the empirical distribution function of the observed data, where left truncated data samples are allowed. The test statistic is given by AD 2 = n + 2n log(1 z ) 1 n n (1 + 2(n j)) log(1 z j ) + 1 n j=1 n (1 2j) log(z j z ) j=1 with z = F θ () and z j = F θ ( j ), where 1,..., n are the ordered data values. ere, F θ is the null distribution. A list with class "mchtest" containing the following components statistic treshold p.value data.name method sim.no the value of the Quadratic Class Anderson-Darling statistic the treshold value the p-value of the test a character string giving the name of the data the character string "Quadratic Class Anderson-Darling test" number of simulated szenarios in the Monte-Carlo simulation References Chernobay, A., Rachev, S., Fabozzi, F. (2005), Composites goodness-of-fit tests for left-truncated loss samples, Tech. rep., University of Calivornia Santa Barbara ad2up.test, w2.test for other quadratic class tests and ks.test, v.test, adup.test, ad.test for supremum class tests. For more details see mctest.

5 ad2up.test 5 c <- rlnorm(1000, 2, 2) # complete sample t <- c[c >= treshold] # left truncated sample ad2.test(t, "plnorm", list(meanlog = 2, sdlog = 2), = 10) ad2up.test Quadratic Class Upper Tail Anderson-Darling test Quadratic Class Upper Tail Anderson-Darling test providing a comparison of a fitted distribution with the empirical distribution. ad2up.test(,, fit, = NA, sim = 100, tol = 1e-04, estfun = NA) fit sim tol estfun a numeric vector of data values character string naming the null distribution list of null distribution parameters maimum number of szenarios in the Monte-Carlo simulation if the difference of two subsequent p-value calculations is lower than tol the Monte-Carlo simulation is discontinued an function as character string or NA (default). See mctest. The Anderson-Darling test compares the null distribution with the empirical distribution function of the observed data, where left truncated data samples are allowed. The test statistic is given by AD 2 up = 2n log(1 z ) + 2 n j=1 log(1 z j ) + 1 z n n 1 (1 + 2(n j)) 1 z j j=1 with z = F θ () and z j = F θ ( j ), where 1,..., n are the ordered data values. ere, F θ is the null distribution.

6 6 adup.test A list with class "mchtest" containing the following components statistic treshold p.value data.name method sim.no the value of the Quadratic Class Upper Tail Anderson-Darling statistic the treshold value the p-value of the test a character string giving the name of the data the character string "Quadratic Class Upper Tail Anderson-Darling Test" number of simulated szenarios in the Monte-Carlo simulation References Chernobay, A., Rachev, S., Fabozzi, F. (2005), Composites goodness-of-fit tests for left-truncated loss samples, Tech. rep., University of Calivornia Santa Barbara ad2.test, w2.test for other quadratic class tests and ks.test, v.test, adup.test, ad.test for supremum class tests. For more details see mctest. c <- rlnorm(100, 2, 2) # complete sample t <- c[c >= treshold] # left truncated sample ad2up.test(t, "plnorm", list(meanlog = 2, sdlog = 2), = 10) adup.test Supremum Class Upper Tail Anderson-Darling test Supremum class version of the Upper Tail Anderson-Darling test providing a comparison of a fitted distribution with the empirical distribution. adup.test(,, fit, = NA, alternative = c("two.sided", "less", "greater"), sim = 100, tol = 1e-04, estfun = NA)

7 adup.test 7 fit alternative sim tol estfun a numeric vector of data values character string naming the null distribution list of null distribution parameters indicates the alternative hypothesis and must be one of "two.sided" (default), "less", or "greater". Initial letter must be specified only. maimum number of szenarios in the Monte-Carlo simulation if the difference of two subsequent p-value calculations is lower than tol the Monte-Carlo simulation is discontinued an function as character string or NA (default). See mctest. The supremum class Upper Tail Anderson-Darling test compares the null distribution with the empirical distribution function of the observed data. The test statistic is given by ADup + = n sup{ j j n z j } 1 z j ADup = n sup{ z j j 1 n } j 1 z j ADup = ma{adup +, ADup }, with z = F θ () and z j = F θ ( j ), where 1,..., n are the ordered data values. ere, F θ is the null distribution. A list with class "mchtest" containing the following components statistic treshold p.value data.name method alternative sim.no the value of the Supremum Class Upper Tail Anderson-Darling statistic the treshold value the p-value of the test a character string giving the name of the data the character string "Supremum Class Upper Tail Anderson-Darling test" the alternative number of simulated szenarios in the Monte-Carlo simulation References Chernobay, A., Rachev, S., Fabozzi, F. (2005), Composites goodness-of-fit tests for left-truncated loss samples, Tech. rep., University of Calivornia Santa Barbara

8 8 cdens ks.test, v.test, ad.test for supremum class tests and ad2.test, w2.test for other quadratic class tests. For more details see mctest. c <- rlnorm(100, 2, 2) # complete sample t <- c[c >= treshold] # left truncated sample adup.test(t, "plnorm", list(meanlog = 2, sdlog = 2), = 10) cdens Build a conditional density function For a given distribution function cdens builds a conditional density function with respect to a relevant treshold. cdens(, ) character string naming the distribution function for which the conditional density is to be built For the conditional density f of a density f is given by f θ () = f( ) = f θ() 1 F θ (), with θ the parameters of the distribution, F the cumulative distribution function and the treshhold value. For <, f disappear. The conditional density of the specified density function with arguments, the relevant parameters and the treshold predefined as the value of cdens argument. can be a numeric value or numeric vector, but must be greater or equal to. density functions, e.g. dlnorm, dgamma, etc.

9 dplot 9 require(mass) c <- rlnorm(100, 2, 2) # complete sample t <- c[c >= treshold] # left truncated sample clnorm <- cdens("plnorm", = treshold) args(clnorm) # mle fitting based on the complete sample start <- list(meanlog = 2, sdlog = 1) fitdistr(c, dlnorm, start = start) # mle fitting based on the truncated sample fitdistr(t, clnorm, start = start) # in contrast fitdistr(t, dlnorm, start = start) dplot Plot of the distribution functions Plot the empirical against the theoretical distibution function. dplot(,, parm, = NA, verticals = FALSE,...) a numeric vector of data samples character string naming the theoretical distribution function parm list of theoretical distribution parameters verticals see plot.stepfun... graphical parameters can be given as arguments to plot The empirical and the theoretical distribution function specified by the arguments and parm are plotted in one single graphic. For truncated data values it is important to assign the treshold value.

10 10 edf ecdf, plot.ecdf, plot.stepfun c <- rnorm(25) t <- c[c >= 0] # complete sample # left truncated sample # df of the complete sample dplot(c, "pnorm", list(0,1), vertical = TRUE) # df of the left truncated sample dplot(t, "pnorm", list(0,1), = 0, vertical = TRUE) edf Empirical distribution function Empircal distribution function of left truncated data with known distribution. edf(, = NA, parm = NA, = NA) parm a numerical vector of data values character string naming the distribution function list of distribution parameters edf is a version of ecdf allowing left truncated data. If is not assigned all other arguments ecept are ignored and the result is eactly the same as of ecdf. A function of class "stepfun". ecdf, dplot

11 ks.test 11 c <- rlnorm(30, meanlog = 2, sdlog = 1) t <- c[c >= treshold] # complete sample # truncated sample # the results are identical: plot(edf(c)) plot(ecdf(c)) # considering truncated samples: plot(edf(t)) # wrong plot plot(edf(t, "plnorm", list(meanlog = 2, sdlog = 1), = 10)) ks.test Kolmogorov-Smirnov test Kolmogorov-Smirnov test providing a comparison of a fitted distribution with the empirical distribution. ks.test(,, fit, = NA, alternative = c("two.sided", "less", "greater"), sim = 100, tol = 1e-04, estfun = NA) fit alternative sim tol estfun a numeric vector of data values character string naming the null distribution list of null distribution parameters indicates the alternative hypothesis and must be one of "two.sided" (default), "less", or "greater". Initial letter must be specified only. maimum number of szenarios in the Monte-Carlo simulation if the difference of two subsequent p-value calculations is lower than tol the Monte-Carlo simulation is discontinued an function as character string or NA (default). See mctest.

12 12 ks.test The Kolmogorov-Smirnov test compares the null distribution with the empirical distribution function of the observed data, where left truncated data samples are allowed. The test statistic is given by n KS + = sup{z + j 1 z j n (1 z ) z j } n KS = sup{z j (z + j 1 1 z j n (1 z ))} KS = ma{ks +, KS }, with z = F θ () and z j = F θ ( j ), where 1,..., n are the ordered data values. ere, F θ is the null distribution. A list with class "mchtest" containing the following components statistic treshold p.value data.name method alternative sim.no the value of the Kolmogorov-Smirnov statistic the treshold value the p-value of the test a character string giving the name of the data the character string "Kolmorov-Smirnov test" the alternative number of simulated szenarios in the Monte-Carlo simulation References Chernobay, A., Rachev, S., Fabozzi, F. (2005), Composites goodness-of-fit tests for left-truncated loss samples, Tech. rep., University of Calivornia Santa Barbara ad.test, v.test, adup.test for other supremum class tests and ad2.test, ad2up.test, w2.test for quadratic class tests. For more details see mctest. c <- rlnorm(100, 2, 2) # complete sample t <- c[c >= treshold] # left truncated sample ks.test(t, "plnorm", list(meanlog = 2, sdlog = 2), = 10)

13 mctest 13 mctest Monte-Carlo simulation based GoF test Performs Monte-Carlo based Goodness-of-Fit tests. mctest is called by the GoF tests defined in this package. For internal use only. mctest(,, parm,, sim, tol, STATISTIC, estfun) parm sim tol STATISTIC estfun numerical vector of data values character string specifying the null distribution parameters of the null distribution maimum number of szenarios within the Monte-Carlo-Simulation if the difference of two subsequent p-value calculations is lower than tol the Monte-Carlo simulation stops function of the test statistic an function as character string or NA, see details. From the fitted null distribution mctest draws samples each with length of the observed sample and with treshold. The random numbers are taken from the conditional distribution with support [, ). The maimum number of samples is specified by sim. For each of these samples the conditional distribution is fitted and the statistic given in STATISTIC is calculated. The p-value is the proportion of times the sample statistics values eceed the statistic value of the observed sample. For each szenario sample mctest uses a Maimum-likelihood fitting of the distribution as default. This is done by direct optimization of the log-likelihood function using optim. Alternativly the fitting parameters for the szenario samples might be estimated with a user-specified function assigned in estfun. It must be a function with argument (and if desired) which can be parsed. The return value of the evaluated function must be a list with the parameters which should be fitted. Inside mctest the evaluation of estfun is performed with as assigned in the call of mctest and the szenario sample. By assigning a function to estfun, the fitting procedure can be done faster and more appropriate to a given problem. The evir package for eample defines a function gpd to fit the Generalized Pareto Model. To start a test it is more reasonable to set estfun = gpd(, y), where y must be a defined numeric value.

14 14 mctest named list of TS p.value sim value of the test statistic for Monte-Carlo simulation based p-value of the statistic number of simulated szenarios in the Monte-Carlo simulation References Chernobay, A., Rachev, S., Fabozzi, F. (2005), Composites goodness-of-fit tests for left-truncated loss samples, Tech. rep., University of Calivornia Santa Barbara Ross, S. M. (2002), Simulation, 3rd Edition, Academic Press. Pages ad2.test, ad2up.test, w2.test for quadratic class GoF tests and ks.test, v.test, adup.test, ad.test for supremum class GoF tests. c <- rgamma(100, 20, 2) t <- c[c > treshold] # complete sample # left truncated sample ## function for parameter fitting estimate <- function(, ){ cgamma <- cdens("pgamma", ) ll <- function(p, y) { res <- -sum(do.call("cgamma", list(c(y), p[1], p[2], log = TRUE))) if (!is.finite(res)) return(-log(.machine$double.min)*length()) return(res) } est <- optim(c(1,1), ll, y =, lower = c(.machine$double.eps, 0), method = "L-BFGS-B") as.list(est$par) } fit <- estimate(t, treshold) cat("fitting parameters:", unlist(fit), "\n") ## calculate p-value with fitting algorithm defined in mctest... ad2up.test(t, "pgamma", fit, = treshold, estfun = NA, tol = 1e-02) ##... or with the function estimate ad2up.test(t, "pgamma", fit, = treshold, estfun = "estimate(, )", tol = 1e-02) ## not run: ## if the evir package is loaded: ## ad.test(t, "pgpd", list(2,3), = treshold,

15 qplot 15 ## estfun = "as.list(gpd(, 0)$par.ests)", tol = 1e-02) qplot QQ-Plot Adjusted QQ-Plot allowing for left-truncated data values. qplot(,, parm, = NA, plot.it = TRUE, main = "QQ-Plot", lab = "empirical quantiles", ylab = "theoretical quantiles",...) parm a numeric vector of data values character string of the distribution list of distribution parameters plot.it logical. TRUE (default) if the result is to be plotted. lab, ylab, main plot labels... further graphical parameters dplot, cdens c <- rlnorm(100, 2, 2) # complete sample t <- c[c >= treshold] # left truncated sample # for not assigned treshold the folliwing # graphics are identical but not usefull par(mfrow = c(2,1)) y <- qlnorm(ppoints(length(t)), 2, 2) qplot(t, "plnorm", list(2,2)) qqplot(t, y); abline(0,1) # fot trucated data rather use qplot(t, "plnorm", list(2,2), = 10)

16 16 v.test v.test Kuiper test Kuiper test providing a comparison of a fitted distribution with the empirical distribution. v.test(,, fit, = NA, sim = 100, tol = 1e-04, estfun = NA) fit sim tol estfun a numeric vector of data values character string naming the null distribution list of distribution parameters maimum number of szenarios in the Monte-Carlo simulation if the difference of two subsequent p-value calculations is lower than tol the Monte-Carlo simulation stops an function as character string or NA (default). See mctest. The Kolmogorov-Smirnov test compares the null distribution with the empirical distribution of the observed data, where left truncated data samples are allowed. The test statistic (see ks.test) is given by V = ma{ks +, KS }. A list with class "mchtest" containing the following components statistic treshold p.value data.name method sim.no the value of the Kuiper statistic the treshold value the p-value of the test a character string giving the name of the data the character string "Kuiper test" number of simulated szenarios in the Monte-Carlo simulation References Chernobay, A., Rachev, S., Fabozzi, F. (2005), Composites goodness-of-fit tests for left-truncated loss samples, Tech. rep., University of Calivornia Santa Barbara

17 w2.test 17 ks.test, ad.test, adup.test for other supremum class tests and ad2.test, ad2up.test, w2.test for quadratic class tests. For more details see mctest. c <- rlnorm(100, 2, 2) # complete sample t <- c[c >= treshold] # left truncated sample v.test(t, "plnorm", list(meanlog = 2, sdlog = 2), = 10) w2.test Cram/ er-von Mises test Cram/ er-von Mises test providing a comparison of a fitted distribution with the empirical distribution. w2.test(,, fit, = NA, sim = 100, tol = 1e-04, estfun = NA) fit sim tol estfun a numeric vector of data values character string naming the null distribution list of null distribution parameters maimum number of szenarios in the Monte-Carlo simulation if the difference of two subsequent p-value calculations is lower than tol the Monte-Carlo simulation is discontinued an function as character string or NA (default). See mctest. The Cram/ er-von Mies test compares the null distribution with the empirical distribution function of the observed data, where left truncated data samples are allowed. The test statistic is given by W 2 = n 3 + nz z n(1 z ) n (1 2j)z j + j=1 1 (1 z ) 2 n (z j z ) 2 with z = F θ () and z j = F θ ( j ), where 1,..., n are the ordered data values. ere, F θ is the null distribution. j=1

18 18 w2.test A list with class "mchtest" containing the following components statistic treshold p.value data.name method sim.no the value of the Cram\ er-von Mies statistic the treshold value the p-value of the test a character string giving the name of the data the character string "Cramer-von Mises test" number of simulated szenarios in the Monte-Carlo simulation References Chernobay, A., Rachev, S., Fabozzi, F. (2005), Composites goodness-of-fit tests for left-truncated loss samples, Tech. rep., University of Calivornia Santa Barbara ad2up.test, ad2.test for other quadratic class tests and ks.test, v.test, adup.test, ad.test for supremum class tests. For more details see mctest. c <- rlnorm(100, 2, 2) # complete sample t <- c[c >= treshold] # left truncated sample w2.test(t, "plnorm", list(meanlog = 2, sdlog = 2), = 10)

19 Inde Topic aplot dplot, 9 qplot, 15 Topic distribution cdens, 8 edf, 10 Topic dplot edf, 10 Topic htest ad.test, 2 ad2.test, 3 ad2up.test, 5 adup.test, 6 ks.test, 11 mctest, 13 v.test, 16 w2.test, 17 w2.test, 3, 4, 6, 8, 12, 14, 17, 17 ad.test, 2, 4, 6, 8, 12, 14, 17, 18 ad2.test, 3, 3, 6, 8, 12, 14, 17, 18 ad2up.test, 3, 4, 5, 12, 14, 17, 18 adup.test, 3, 4, 6, 6, 12, 14, 17, 18 cdens, 8, 15 dgamma, 8 dlnorm, 8 dplot, 9, 10, 15 ecdf, 10 edf, 10 ks.test, 3, 4, 6, 8, 11, 14, mctest, 3, 4, 6, 8, 12, 13, 17, 18 plot.ecdf, 10 plot.stepfun, 10 qplot, 15 v.test, 3, 4, 6, 8, 12, 14, 16, 18 19