Manuel Oviedo de la Fuente and Manuel Febrero Bande

Size: px

Start display at page:

Download "Manuel Oviedo de la Fuente and Manuel Febrero Bande"

Shana Gilbert
5 years ago
Views:

Bande Universidade de Santiago de Compostela CNTG (Centro

1 Supervised classification methods in by fda.usc package Manuel Oviedo de la Fuente and Manuel Febrero Bande Universidade de Santiago de Compostela CNTG (Centro de Novas Tecnoloxías de Galicia). Santiago de Compostela, 3 y 4 de octubre de 04

2 Introduction Multivariate Real Data Example Iris data. This data set gives the measurements in centimeters of the variables sepal length and width and petal length and width, respectively, for 50 flowers from each of 3 species of iris. The species are Iris setosa, versicolor, and virginica. Sepal.Length Sepal.Width Petal.Length Petal.Width Species October, 04 / 9

3 Introduction Multivariate Simulation Data Example Scatterplot of two uniform samples. October, 04 3 / 9

4 Introduction Functional Real Data Example Tecator data. 5 spectrometric curves of meat with Fat, Water and Protein contents. Goal: Explain the fat content through spectrometric curves. data(tecator) par(mfrow=c(,3)) fat5<-ifelse((y<-tecator$y$fat)<5,,4) boxplot(y,main="fat") plot((x<-tecator$absorp),col=fat5,main="spectrometric: X") plot((x.d<-fdata.deriv(tecator$absorp,)),col=fat5,main="derviative: X.d") Fat Spectrometric: X Derviative: X.d Absorbances d(absorbances,) Wavelength (mm) Wavelength (mm) October, 04 4 / 9

5 Introduction Functional Simulation Data Example Model, k=. Model, k=. Model 3, k=. X(t) m m X(t) m m X(t) m m t t t A sample of 40 functions (Ornstein Uhlenbeck process with Gaussian error) for every simulation model along with the means of each sub-group (G s (black lines) and G s (red line)). October, 04 5 / 9

6 Introduction Some definitions of Functional Data Functional data analysis is a branch of statistics that analyzes data providing information about curves, surfaces or anything else varying over a continuum. The continuum is often time, but may also be spatial location, wavelength, probability, etc. Functional data analysis is a branch of statistics concerned with analysing data in the form of functions, [Ferraty and Vieu, (006)]. Definition.. A random variable X is called a functional variable if it takes values in a functional space E complete normed (or seminormed) space. Definition.. A functional dataset {X,..., X n} is the observation of n functional variables X,..., X n identically distributed as X. October, 04 6 / 9

7 Introduction Example of functional dataset in fda.usc Load the library fda.usc [Febrero-Bande and Oviedo de la Fuente, 0] An object called fdata as a list of the following components: data: typically a matrix of (n x m) dimension which contains a set of n curves discretized in m points or argvals. argvals: locations of the discretization points, by default: {t =,..., t m = m}. rangeval: rangeval of discretization points. names: list with an overall title, xlab, a title for the x axis and ylab, a title for the y axis. library(fda.usc) data(tecator) class(tecator$absorp.fdata) [] "fdata" names(tecator$absorp.fdata) [] "data" "argvals" "rangeval" "names" October, 04 7 / 9

8 Introduction Example of image data: Yoga dataset The dataset was obtained by capturing two actors transiting between yoga poses in front of a green screen. It has been shown recently that in many domains it can be useful to convert images into pseudo time series. Therefore we have converted the motion capture data into time series by a well known technique, see [Keogh et al., 0]. October, 04 8 / 9

It has been shown recently that in many domains it can be useful to convert images into pseudo time

9 Introduction Form image data to functional data The dataset was obtained by capturing two actors transiting between yoga poses in front of a green screen. It has been shown recently that in many domains it can be useful to convert images into pseudo time series. Therefore we have converted the motion capture data into time series by a well-known technique, see [Keogh et al., 0]. October, 04 9 / 9

10 Supervised classification Functional supervised classification Aim: How predict the class Y of a functional variable X Bayes rule: Given a sample X, the aim is to estimate the posterior probability of belonging to each group: The classification rule: p g(x ) = P(Y = g χ = X ) = E( Y =g χ = X ) Ŷ = arg maxˆp g(x ) The estimate of the posterior probability p g(x ) can be calculated using logistic regression or non parametric regression. The package allows the estimation of the groups in a training set of functional data by k-nearest Neighbor Classifier: classif.knn Kernel Classifier: classif.kernel Logistic Classifier (linear model): classif.glm Logistic Classifier (additive model): classif.gsam and classif.gkam Distance Classifier: classif.dist DD classifier: classif.dd October, 04 0 / 9

11 Classification via regression models Generalized Functional Linear Model The scalar response y is estimated by functional {X q(t)} Q q= and also non functional Z = { } J Z j covariates by: j= Q y i = g α + Z i β + X q i (t), β q(t) + ε i q= g() is the inverse link function and ε i are random errors with mean zero and finite variance σ. [Ramsay and Silverman, 005] uses fixed basis representation of X (t) and β(t): B spline, Fourier, Wavelets, create.basis. [Cardot et al., 999] uses so-called functional principal components regression (FPC),create.pc.basis. [Preda et al., 007] uses so-called functional partial least squares components regression (FPLS), create.pls.basis. October, 04 / 9

12 Classification via regression models Generalized Functional Linear Model ldata<-list("df"=tecator$y,"absorp.fdata"=tecator$absorp.fdata, "absorp.d"=fdata.deriv(tecator$absorp.fdata)) ldata$df$fat5<-factor(ifelse(tecator$y$fat<5,0,)) res.glm<- classif.glm( fat5 ~ absorp.d,data=ldata) res.glm -Call: classif.glm(formula = fat5 ~ absorp.d, data = ldata) -Probability of correct classification: res.glm$fit[[]] Call: glm(formula = pf) Coefficients: (Intercept) absorp.d.bspl4. absorp.d.bspl absorp.d.bspl4.3 absorp.d.bspl4.4 absorp.d.bspl Degrees of Freedom: 4 Total (i.e. Null); 09 Residual Null Deviance: 98 Residual Deviance: 8.5 AIC: 40.5 October, 04 / 9

13 Classification via regression models Generalized Functional Additive Model Generalized Functional Spectral Additive Linear Model (FGSAM), [Müller and Yao, 008], J ( y i = g α + f j Z j ) Q ( (t)) + s i q X q + ε i i j= q= where f ( ), s( ) are the smoothed functions. res.gsam<-classif.gsam(fat5~ s(absorp.d),data=ldata) res.gsam -Call: classif.gsam(formula = fat5 ~ s(absorp.d), data = ldata) -Probability of correct classification: res.gsam$fit[[]] Family: binomial Link function: logit Formula: [] "fat5~+s(absorp.d.bspl4.,k=-)+s(absorp.d.bspl4.,k=-)+s(absorp.d.bspl4.3,k=-)+s(absorp.d.bs Estimated degrees of freedom: total = 8.5 UBRE score: October, 04 3 / 9

14 Classification via regression models Generalized Functional Additive Model Generalized Functional Kernel Additive Linear Model (FGKAM), [Febrero-Bande and González-Manteiga, 03], Q ( (t)) y i = g α + K X q + ε i i q= where K( ) is the kernel estimator (extends the knn classifier). res.gkam<-classif.gkam(fat5 ~ absorp.d,data=ldata) res.gkam -Call: classif.gkam(formula = fat5 ~ absorp.d, data = ldata) -Probability of correct classification: res.gkam$fit[[]] Family: binomial Link function: logit alpha= -8.9 n= 5 **** **** **** **** **** **** h cor(f(x),eta) edf f(absorp.d) **** **** **** **** **** **** edf: Equivalent degrees of freedom AIC= 69.3 Deviance explained = 89.8 % R-sq.= 0.93 R-sq.(adj)= 0.89 October, 04 4 / 9

Classification by depth functions Classification by DD-classifier Make the group classification of a training dataset using DD-classifier estimation in the following steps. Step.

15 Classification by depth functions Classification by DD-classifier Make the group classification of a training dataset using DD-classifier estimation in the following steps. Step. The function computes the selected depth measure of the points in x (multivariate data or multivariate functional data) w.r.t. a subsample of each G level group. October, 04 5 / 9

Classification by depth functions DD plot Step. The function calculates the misclassification rate based on data depth computed in step () using the following classifiers: "MaxD": Maximum depth.

16 Classification by depth functions DD plot Step. The function calculates the misclassification rate based on data depth computed in step () using the following classifiers: "MaxD": Maximum depth. "DD","DD","DD3": Search the best separating polynomial of degree,, 3. DD-plot(HS,DD) DD-plot(HS,DD) depth depth depth depth DD-plot(HS,DD3) DD-plot(HS,MaxD) depth depth depth depth From left to right, top to bottom DD plot using DD, DD, DD3 and Maximum Depth classifiers to the DD-plot. The one-dimensional depth in all cases is the Tukey depth. October, 04 6 / 9

17 Classification by depth functions DD classifier for Multivariate Data "glm","gam": Generalized Linear (or Additive) Models. "lda","qda": Linear Discriminant (or Quadratic) Analysis. "knn","np": Non-parametric k-nearest Neighbour (or Kernel) classifier. DD-plot(HS,lda) DD-plot(HS,qda) depth depth depth depth DD-plot(HS,knn) DD-plot(HS,glm) depth depth depth depth DD-plot(HS,gam) DD-plot(HS,tree) depth depth depth depth From left to right, top to bottom DD plot using LDA, QDA, knn, GLM, GAM and tree classifiers to the DD plot. The one-dimensional depth in all cases is the Tukey depth. October, 04 7 / 9

Classification by depth functions DD classifier for functional data par(mfrow=c(,)) out=classif.dd(ldata$df$fat5,ldata$absorp.fdata,classif="gam") out=classif.dd(ldata$df$fat5,ldata$absorp.d,classif="gam") 0.

18 Classification by depth functions DD classifier for functional data par(mfrow=c(,)) out=classif.dd(ldata$df$fat5,ldata$absorp.fdata,classif="gam") out=classif.dd(ldata$df$fat5,ldata$absorp.d,classif="gam") DD plot(fm,gam) depth 0 depth DD plot(fm,gam) depth 0 depth 0 0 out$misclassification;out$misclassification [] [] 0.07 October, 04 8 / 9

19 Conclusions The functional discrimintation extends the logistic regression (GLM, GAM) using a basis representation of the functional data (B-spline, Fourier, wavelets, PC, PLS,...). The DD G classifier converts the functional data in a multivariate dataset whose columns are constructed using depths and the new classifiers are classical multivariate classifiers based on discrimination procedures (LDA, QDA) or regression ones (knn, NP, GLM, GAM). More classifiers could be considered here (SVM, neural networks,...). Several depth procedures can be taken into account at the same time in order to improve the classification or in order to diagnose whether a depth contains or not useful information for the classification process. The DD G classifier trick" is specially interesting in a functional or high dimensional framework because it changes the dimension of the classification problem from infinite or large dimension to G, where G depends only on the number of groups and the number of depths employed. The functions needed to perform these preseentation are freely available at CRAN in the fda.usc package ([Febrero-Bande and Oviedo de la Fuente, 0]) for versions higher than..0. October, 04 9 / 9

20 Cardot H, Ferraty F and Sarda P (999). Functional Linear Model. Statistics and Probability Letters, 45(), -. Cueata-Albertos, J.A., Febrero-Bande, M. and Oviedo de la Fuente, M. The DDG-classifier in the functional setting. Submitted. Cuevas, A., Febrero, M., and Fraiman, R. (007). Robust estimation and classification for functional data via projection based depth notions. Computational Statistics, (3): Fraiman, R. and Muniz, G. (00). Trimmed means for functional data. Test, 0(): Febrero-Bande, M. and Oviedo de la Fuente, M. (0). Statistical computing in functional data analysis: The R package fda.usc. Journal of Statistical Software, 5(4): 8. Febrero-Bande, M. and González-Manteiga, W. (03). Generalized additive models for functional data. TEST, ():78 9. Ferraty F and Vieu P (006). Nonparametric functional data analysis. Springer Series in Statistics, New York. Keogh, E., Zhu, Q., Hu, B., Hao. Y., Xi, X., Wei, L. and Ratanamahatana, C. A. (0) The UCR Time Series Classification/Clustering data/ Müller, H.G. and Stadtmüller, U. (005). October, 04 9 / 9

21 Generalized functional linear models. Ann Stat, 33, Müller, H.G. and Yao, F. (008), Functional additive models. Journal of the American Statistical Association 03, Preda C, Saporta G, Lévéder CL. (007). PLS classification of functional data. Comput. Stat, (), Ramsay, J. and Silverman, B. (005). Functional Data Analysis. Springer. Ripley, B. (996). Pattern Recognition and Neural Networks. Cambridge Uni. Press, Cambridge. Wood, S. N. (004). Stable and efficient multiple smoothing parameter estimation for generalized additive models. Journal of the American Statistical Association, 99(467): October, 04 9 / 9

k Nearest Neighbors Super simple idea! Instance-based learning as opposed to model-based (no pre-processing)

k Nearest Neighbors Super simple idea! Instance-based learning as opposed to model-based (no pre-processing) k Nearest Neighbors k Nearest Neighbors To classify an observation: Look at the labels of some number, say k, of neighboring observations. The observation is then classified based on its nearest neighbors