THE detailed spectral information of hyperspectral

Transcription

1 1358 IEEE GEOSCIENCE AND REMOTE SENSING LETTERS, VOL. 14, NO. 8, AUGUST 2017 Locality Sensitive Discriminant Analysis for Group Sparse Representation-Based Hyperspectral Imagery Classification Haoyang Yu, Student Member, IEEE, LianruGao,Member, IEEE, WeiLi,Member, IEEE, Qian Du, Senior Member, IEEE, and Bing Zhang, Senior Member, IEEE Abstract This letter proposes to integrate the locality sensitive discriminant analysis (LSDA) with the group sparse representation (GSR) for a hyperspectral imagery classification. The LSDA is to project the data set to a lower-dimensional subspace to preserve local manifold structure and discriminant information, while the GSR is to encode the projected testing set as a sparse linear combination of group-structured training samples for classification. The proposed approach, denoted as LSDA-GSR classifier (GSRC), is evaluated using two real hyperspectral data sets. Experimental results demonstrate that it can provide considerable improvement to the original counterparts, i.e., SRC and GSRC, with a relatively low computational cost. Index Terms Classification, group sparse representation (GSR), hyperspectral image, locality sensitive discriminant analysis (LSDA). I. INTRODUCTION THE detailed spectral information of hyperspectral imagery offers the capability of more accurate classification of materials. However, its analysis is challenging due to high correlation between adjacent bands and the limited number of training samples. In order to address these issues, several feature selection and feature extraction methods have been proposed to integrate with classifiers that are able to perform accurately with spectral or spectral spatial information [1] [3]. For instance, multitask joint sparse representation (SR) with stepwise Markov random field is one of the well-established frameworks which makes full Manuscript received March 17, 2017; revised May 5, 2017; accepted May 29, Date of publication June 26, 2017; date of current version July 20, This work was supported by the National Natural Science Foundation of China under Grant , Grant , and Grant (Corresponding author: Lianru Gao.) H. Yu and B. Zhang are with the Key Laboratory of Digital Earth Science, Institute of Remote Sensing and Digital Earth, Chinese Academy of Sciences, Beijing , China, and also with the College of Resources and Environment, University of Chinese Academy of Sciences, Beijing , China ( yuhy@radi.ac.cn; zb@radi.ac.cn). L. Gao is with the Key Laboratory of Digital Earth Science, Institute of Remote Sensing and Digital Earth, Chinese Academy of Sciences, Beijing , China ( gaolr@radi.ac.cn). W. Li is with the College of Information Science and Technology, Beijing University of Chemical Technology, Beijing , China ( liw@mail.buct.edu.cn). Q. Du is with the Department of Electrical and Computer Engineering, Mississippi State University, Mississippi State, MS USA ( du@ece.msstate.edu). Color versions of one or more of the figures in this letter are available online at Digital Object Identifier /LGRS use of the advantages of band selection to bring significant improvement for hyperspectral image classification [4] [6]. On the other hand, classic feature extraction techniques for dimensionality reduction like principle component analysis and linear discriminant analysis have widely been exploited for target detection and classification [7]. Both of them belong to spectral methods, which aim to estimate the global statistics and appear to be sensitive to the sample size. Since these methods focus on the Euclidean structure, they fail to discover the underlying structure if the original data reside in a manifold subspace of the ambient space. Recently, there have been several geometrically approaches to data analysis in highdimensional spaces. Examples include locally linear embedding, neighborhood preserving embedding (NPE), and locality preserving projection (LPP) [8]. Though these methods have been demonstrated to be effective in exploiting the geometrically structure, they may fail to discover the discriminant structure of the underlying manifold in the data. In this context, locality sensitive discriminant analysis (LSDA) has shown to be a powerful algorithm which can optimally preserve the local manifold structure, as well as discriminant information. The idea of applying LSDA to dimensionality reduction and classification relies on the construction of a nearest neighbor graph, which can be split into within-class graph and betweenclass graph to separately characterize the geometrical and discriminant structures of data manifold. Based on the graph Laplacian, a linear transformation matrix can be found to project the input data to a lower-dimensional subspace, where the margin between samples with different class labels is maximized at each local neighborhood [9]. In hyperspectral image classification, representation-based methods are of great interest due to no assumption of data density distribution. The principle behind such classification is based on the fact that a testing pixel can be represented by a sparse linear combination of labeled samples [10]. According to the constraints imposed to the weight coefficients, they can belong to an l 2 -norm regularized collaborative representation or an l 1 -norm regularized SR [11]. However, the SR-based classifier (SRC) may suffer from instability of sparse coefficients due to high dictionary coherency. Therefore, based on the inherent group-structured property, group SRC (GSRC) is designed to reconstruct the dictionary composed of several class-dependent subdictionaries such that a testing sample is represented by group atoms rather than individual ones [12] X 2017 IEEE. Personal use is permitted, but republication/redistribution requires IEEE permission. See for more information.

2 YU et al.: LSDA GSR-BASED HYPERSPECTRAL IMAGERY CLASSIFICATION 1359 With the homogeneity assumption, GSRC incorporates the contextual information by simultaneously representing the testing pixel and its neighborhood, and the class label is determined by the class with the minimum error [13]. Inspired by aforementioned work, this letter proposes a new approach for hyperspectral imagery classification which integrates LSDA with GSRC. The algorithm adopts LSDA to project the input data to a lower-dimensional subspace which preserves the local neighborhood information and the discriminant information. Then, the projected data can linearly be represented by group-structured training samples to classify the testing set by measuring the class approximation error. The proposed approach, abbreviated as LSDA-GSRC, takes advantage of both the local manifold structure and GSR for better characterization and classification for hyperspectral data set. II. PROPOSED CLASSIFICATION FRAMEWORK A. Locality Sensitive Discriminant Analysis Let X =x 1, x 2,...x n } be a hyperspectral image, where x i =[x i1, x i2,...,x id ] T denotes a spectral vector associated with an image pixel i S,andn and d represent the total number of pixels and spectral bands, respectively. The principle of a subspace-based projection is to find a transformation matrix A =a 1, a 2,...,a m } that maps X to Z =z 1, z 2,...z n } R m n (m << d) via Z = A T X. Under assumption that x i M (M denotes the underlying manifold), one can build agraphg by searching its t nearest neighbors (i.e., N(x i ) = x 1 i,...,xt i }) and putting an edge between x i and N(x i ). The local geometry of data manifold is then characterized by graph G with a weight matrix W which can be defined as 1, if x i N(x j ) or x j N(x i ) W ij = (1) 0, otherwise. In order to develop both geometrical and discriminant structures of data manifold, LSDA is adopted to construct withinclass graph G w and between-class graph G b by splitting N(x i ) into two subsets N w (x i ) and N b (x i ), which can be expressed as N w (x i ) = x j ( j) i l x i = l(xi ), 1 j t } N b (x i ) = x j ( j ) i l x i = l(xi ), 1 j t } (2) where l(x i ) is the class label of x i. Therefore, the weight matrices W w and W b of G w and G b can be defined as 1, if x i N w (x j ) or x j N w (x i ) W w,ij = W b,ij = 0, otherwise 1, if x i N b (x j ) or x j N b (x i ) 0, otherwise. To satisfy the criterion that connected points in G w are close enough and the ones in G b stay as distant as possible, the following two objective functions are defined for Z: min ij max ij (3) (z i z j ) 2 W w,ij (4) (z i z j ) 2 W b,ij. (5) Since Z = A T X, the objective function (4) can be reduced to (A T XD w X T A A T XW w X T A), whered w is a diagonal matrix with D w,ii = j W w,ij. Similarly, the objective function (5) can be reduced to A T XL b X T A, where L b = D b W b is the Laplacian matrix of G b,andd b is a diagonal matrix with D b,ii = j W b,ij. Considering the influence of D w,ii to x i, it is suggested to impose a constraint Z T D w Z = 1 to the objective function [14]. Then, the entire optimization problem can be represented as arg max A T X[αL b + (1 α)w w ]X T A A s.t. A T XD w X T A = 1 (6) where α [0, 1] is a suitable constant. The solution to (6) can be found by solving the following generalized eigenvalue problem: X[αL b + (1 α)w w ]X T A = λ XD w X T A. (7) Let, A be the solution to (7) with the decreasing eigenvalues and the result of LSDA can be obtained by Z = A T X,where A =a 1, a 2,...,a m } is a d m matrix. B. Group Sparse Representation-Based Classification In SRC, a testing pixel y R d is represented by a sparse linear combination of all the training samples under an l 1 -norm constraint as min 1 2 y Dβ λ β 1 (8) where D is the dictionary formed by the training set, β is the corresponding weight vector, and λ is the regularization parameter. The class label of y is determined by the minimum residual between y and its approximation class(y) = arg min c y Dδ c (β) 2 2 (9) where c 1, 2,..., K } is the class index, K is the total class number, and δ c (β) is the indicator operation which can zero out the atoms in β that do not belong to class c. With a further discussion of D, it is suggested that the inherent group-structured property can be exploited by reconstructing the dictionary with class subdictionaries [15]. Therefore, GSRC adopts D = (D 1, D 2,...,D K ) with each column of D i denoting a sample randomly selected from class i, such that the training samples belong to the same class are grouped together to form subdictionaries. Then, y can be represented by groups of β rather than individual atoms with activation of the coefficients for certain groups min 1 2 y Dβ λ g G ω g β g 2 (10) where g G 1, G 2,...,G K } is the K groups corresponding to D, β g refers to the coefficients of each group, and ω g is a parameter adopted to compensate for the different group sizes. (ω g = 1 if the groups are in the same size.) Based on the fact that pixels in a small neighborhood usually consist of the same or similar materials, it is recommended that the spatial correlation between y and its neighbors should

3 1360 IEEE GEOSCIENCE AND REMOTE SENSING LETTERS, VOL. 14, NO. 8, AUGUST 2017 TABLE I OVERALL ACCURACY AND CLASS-SPECIFIC ACCURACY FOR THE AVIRIS INDIAN PINES SCENE (120 SAMPLES PER CLASS) be considered into the framework [16]. In this context, let Y =y 1, y 2,...,y t } R d t be a testing set whose columns correspond to t pixels in a spatial neighborhood, and the above formulation can be extended to the collaborative group version min B 1 2 Y DB 2 F + λ ω g B g 2 (11) g G where B is a group sparse matrix and B g coefficients of each group. refers to the C. LSDA-GSR-Based Classification Considering that the main principle of SR-based classifiers is the representation of samples which is related to the training size, it is reasonable to implement a manifold subspace projection process for better characterization of features. Derived from the aforementioned algorithms, LSDA-GSRC for hyperspectral imagery is a two-step process: 1) local manifold subspace-based projection, where LSDA is adopted to project the training and testing sets to a lower-dimensional subspace and 2) representation-based classification, where the projected testing set is linearly represented by group-structured training samples via GSRC. Therefore, the final objective function of LSDA-GSRC can be represented as min B 1 2 L(Y) L(D)BL 2 F + λ ω g B L 2 g (12) g G where L(Y) and L(D) are projected by LSDA with Y and D, B L is a group sparse matrix, and Bg L refers to the coefficients of each group corresponding to L(D). Finally, the class label of the center testing pixel y in Y is determined by the minimum total residual error class(y) = arg min L(Y) L(D)δ c (B L ) 2 F (13) c where δ c (B L ) is the indicator operation which can zero out all the elements in B L that do not belong to class c. The pseudocode for the proposed LSDA-GSRC is summarized in Algorithm 1. III. EXPERIMENTAL RESULTS In this section, the proposed LSDA-GSRC is evaluated using two real hyperspectral data sets. The first one is collected by Airborne Visible/Infrared Imaging Spectrometer (AVIRIS) over the Indian Pines region. The scene contains pixels, with 220 spectral bands in the spectral range Fig. 1. (a) Lambda versus OA of LSDA-GSRC for different data sets. (b) Dimensionality m of the manifold subspace versus OA of LSDA-GSRC for different data sets. Algorithm 1 LSDA-GSRC Input: Available training data X =x i } n i=1 and a testing set in a spatial neighborhood Y =y j } t j=1. Step 1: Normalize X and Y to have unit l 2 -norm. Step 2: ObtainL(X) and L(Y) by LSDA using X and Y. Step 3: Obtain weight matrix B L in GSRC according to (12). Step 4: Compute the residual according to (13). Step 5: Identify the class label of the center pixel y in Y. Output: class (y). from 0.4 to 2.5 μm. To satisfy the sparsity requirement in the process of representation, eight mutually exclusive classes with a total of 8624 labeled samples are adopted from the ground reference data to avoid some classes with very few training samples. The other one is collected by the Reflective Optics System Imaging Spectrometer (ROSIS) over the University of Pavia in Italy in The ground-truth data contains nine classes. From the point of time efficiency, a size of patch with a total of 8981 labeled samples is selected from the original scene with 103 spectral bands after removing 12 bands of noise and water absorption. As for the parameter tuning, the spatial neighborhood size is generally adopted as t = 9. The regularized parameter λ and dimensionality m of the manifold subspace for the proposed approach are investigated. As illustrated in Fig. 1, we adopt λ = 1andm = 30 for GSRC and LSDA with the AVIRIS Indian Pines scene, while λ = 2andm = 10 with the ROSIS University of Pavia scene. In order to evaluate the proposed approach with regard to other related classic methods, experiments with support vector machine (SVM), SRC, locality preserving-based SRC (LPSRC), and GSRC are provided (the parameter settings

4 YU et al.: LSDA GSR-BASED HYPERSPECTRAL IMAGERY CLASSIFICATION Fig Classification maps for the AVIRIS Indian Pines scene (120 samples per class). The overall accuracies are provided in the parentheses. TABLE II OVERALL A CCURACY W ITH D IFFERENT N UMBER OF T RAINING S AMPLES ( PER C LASS ) FOR THE AVIRIS I NDIAN P INES S CENE Fig. 3. Classification maps for the ROSIS University of Pavia scene (100 samples per class). The overall accuracies are provided in the parentheses. TABLE III C OMPUTATIONAL C OST ( IN S ECONDS ) FOR THE AVIRIS I NDIAN P INES S CENE (120 S AMPLES PER C LASS ) follow those in the literature) [17]. In addition, we develop the integration of LPP and NPE with GSRC as comparisons to evaluate the proposed LSDA-GSRC. Overall accuracy (OA) and class-specific accuracy are provided with respect to the same number of randomly selected labeled samples per class (ωg = 1). In the first experiment using the AVIRIS Indian Pines scene, we randomly select 120 samples per class with a total size of 960 (which represents approximately 11% of the labeled samples), where the remaining 89% of the samples are used for validation. Table I reports the OA and class-specific accuracy of different methods. In this comparison, representation-based classifiers generally perform better than SVM. The improvement of GSRC toward SRC demonstrates the advantage of integration with spatial information. Moreover, local manifold subspace-based methods provide higher accuracies than the original counterparts, i.e., SRC and GSRC, which indicate that the proposed framework is robust and reliable. For illustrative purpose, Fig. 2 shows the corresponding classification maps. As reported, GSRC-based methods bring a significant improvement of details in the homogeneous region compared with SRC-based ones. Most importantly, LSDA-GSRC obtains

5 1362 IEEE GEOSCIENCE AND REMOTE SENSING LETTERS, VOL. 14, NO. 8, AUGUST 2017 TABLE IV OVERALL ACCURACY WITH DIFFERENT NUMBER OF TRAINING SAMPLES (PER CLASS) FOR THE ROSIS UNIVERSITY OF PAVIA SCENE TABLE V COMPUTATIONAL COST (IN SECONDS) FOR THE ROSIS UNIVERSITY OF PAVIA SCENE (100 SAMPLES PER CLASS) the best result with an overall accuracy of 92.69%, which is 11.98% and 4.04% higher than SRC and GSRC, respectively. The Indian Pines experiment also evaluates the proposed method with a varying size of training samples (from 30 to 120). Table II presents the overall classification accuracies provided by different methods, as a function of the number of training samples. As shown in Table II, local manifold subspace-based methods generally perform better than the original counterparts and the proposed LSDA-GSRC obtained the best results in all cases. Besides, the computational cost is also briefly reported for the compared SR-based methods in this letter. As shown in Table III, when a total of 960 labeled samples are used, LSDA-GSRC is obviously faster than SRC and GSRC, which demonstrates the efficiency of the proposed method. In the second experiment with the ROSIS University of Pavia scene, we evaluate the proposed method with different numbers of training samples. As reported in Table IV, the proposed LSDA-GSRC provides better results than the other tested methods in all cases. For instance, as illustrated in Fig. 3, when a total of 900 samples are used (approximately 10% of the labeled samples), LSDA-GSRC obtains the best result with an overall accuracy of 97.36%, which is 10.59% and 3.29% higher than SRC and GSRC, respectively. Besides, as shown in Table V, the local manifold subspace-based methods are generally faster than the original counterpart. Synthesizing the above results and analysis, it is convinced that the proposed LSDA-GSRC is effective and reliable. IV. CONCLUSION In this letter, a new local manifold subspace-based GSRC classifier, called LSDA-GSRC, was proposed for hyperspectral imagery. The main contributions include applying LSDA to project the input data to a local manifold subspace and representing the projected testing set by GSR for classification. The proposed method has been compared with the classic SRC, GSRC, and their integration with other subspace-based algorithms using two real hyperspectral data sets. Experimental results demonstrated that LSDA-GSRC outperformed the original counterparts and their variants, and provided better classification results with a relatively low computational cost. REFERENCES [1] S. Jia, Y. Xie, G. Tang, and J. Zhu, Spatial-spectral-combined sparse representation-based classification for hyperspectral imagery, Soft Comput., vol. 20, no. 12, pp , Dec [2] H. Yu, L. Gao, J. Li, S. S. Li, B. Zhang, and J. A. Benediktsson, Spectral-spatial hyperspectral image classification using subspacebased support vector machines and adaptive Markov random fields, Remote Sens., vol. 8, no. 4, p. 355, Apr [3] Z. Zhu, S. Jia, S. He, Y. Sun, Z. Ji, and L. Shen, Three-dimensional Gabor feature extraction for hyperspectral imagery classification using a memetic framework, Inf. Sci., vol. 298, pp , Mar [4] Y. Yuan, J. Lin, and Q. Wang, Hyperspectral image classification via multitask joint sparse representation and stepwise MRF optimization, IEEE Trans. Cybern., vol. 46, no. 12, pp , Dec [5] Q. Wang, J. Lin, and Y. Yuan, Salient band selection for hyperspectral image classification via manifold ranking, IEEE Trans. Neural Netw. Learn. Syst., vol. 27, no. 6, pp , Jun [6] Y. Yuan, J. Lin, and Q. Wang, Dual-clustering-based hyperspectral band selection by contextual analysis, IEEE Trans. Geosci. Remote Sens., vol. 54, no. 3, pp , Mar [7] W. Chen, J. Huang, J. Zou, and B. Fang, Wavelet-face based subspace LDA method to solve small sample size problem in face recognition, Int. J. Wavelets Multiresolut Inf. Process., vol. 7, no. 2, pp , Mar [Online]. Available: S [8] S. T. Roweis and L. K. Saul, Nonlinear dimensionality reduction by locally linear embedding, Science, vol. 290, no. 5500, pp , Dec [9] R. K. F. Chung, Spectral graph theory, in Regional Conference Series in Mathematics, vol. 92. Providence, RI, USA: AMS, [10] X. Zhang, C. Xu, M. Li, and X. Sun, Sparse and low-rank coupling image segmentation model via nonconvex regularization, Int. J. Pattern Recognit. Artif. Intell., vol. 29, no. 2, p , Mar [11] W. Li, Q. Du, and M. Xiong, Kernel collaborative representation with Tikhonov regularization for hyperspectral image classification, IEEE Geosci. Remote Sens. Lett., vol. 12, no. 1, pp , Jan [12] X. Sun, Q. Qu, N. M. Nasrabadi, and T. D. Tran, Structured priors for sparse-representation-based hyperspectral image classification, IEEE Geosci. Remote Sens. Lett., vol. 11, no. 7, pp , Dec [13] W. Li and Q. Du, A survey on representation-based classification and detection in hyperspectral remote sensing imagery, Pattern Recognit. Lett., vol. 83, no. 2, pp , Nov [14] D. Cai, X. He, K. Zhou, J. Han, and H. Bao, Locality sensitive discriminant analysis, in Proc. 20th Int. Joint Conf. Artif. Intell., Jan. 2007, pp [15] J. Huang and T. Zhang, The benefit of group sparsity, Ann. Statist., vol. 38, no. 4, pp , [16] Y. Chen, N. M. Nasrabadi, and T. D. Tran, Hyperspectral image classification using dictionary-based sparse representation, IEEE Trans. Geosci. Remote Sens., vol. 49, no. 10, pp , May [17] L. Gao, H. Yu, B. Zhang, and Q. Li, Locality-preserving sparse representation-based classification in hyperspectral imagery, J. Appl. Remote Sens., vol. 10, no. 4, p , Jun