Available online at ScienceDirect. Procedia Computer Science 93 (2016 )

Transcription

1 Available online at ScienceDirect Procedia Computer Science 93 (2016 ) th International Conference On Advances In Computing & Communications, ICACC 2016, 6-8 September 2016, Cochin, India Dimensionality Reduction using Band Selection Technique for ernel based Hyperspectral Image Classification Reshma.R a,, V.Sowmya a,.p.soman a a Centre for Computational Engineering and Networking (CEN), Amrita School of Engineering, Coimbatore, Amrita Vishwa Vidyapeetham, Amrita University, India Abstract Hyperspectral images have abundant of information stored in the various spectral bands ranging from visible to infrared region in the electromagnetic spectrum. High data volume of these images have to be reduced, preserving the original information, to ensure efficient processing. In this paper, dimensionality reduction is done on Indian Pines and Salinas-A datasets using inter band block correlation coefficient technique followed by Singular Value Decomposition (SVD) and QR decomposition. The dimensionally reduced images are classified using GURLS and LibSVM. Classification accuracies of the original image is compared to that of the dimensionally reduced image. The experimental analysis shows that, for 10% training sample the overall accuracy, average accuracy and kappa coefficient of the dimensionally reduced image (about 50% of the dimension is reduced) is i)83.52%, 77.18%, for Indian Pines and ii)99.53%, 99.40%, for Salinas-A dataset which is comparable to that of original image i)84.67%, 82.28%, for Indian Pines and ii)99.32%, 99.18%, for Salinas-A dataset. c 2016 The Authors. Published by Elsevier by Elsevier B.V. B.V. This is an open access article under the CC BY-NC-ND license ( Peer-review under responsibility of the Organizing Committee of ICACC Peer-review under responsibility of the Organizing Committee of ICACC 2016 eywords: Dimensionality reduction; hyperspectral image; classification; kernel methods; GURLS; LibSVM; accuracy; band selection. 1. Introduction Hyperspectral imaging have a remarkable development in the recent years with the advancement in hyperspectral sensors. High resolution and abundance of information in the hyperspectral images have attracted researchers from various fields like agriculture, environmental monitoring, mining, etc. The volume of data which is acquired at any time (t) is very high which pose a problem in data storage and processing. This reveals the importance of reducing the dimension of these images. Dimensionality reduction 1,2 is a process in which a higher dimensional data is represented using a small portion of the data ie, lower dimension, preserving its original features. Various techniques used for dimension reduction are Principal Component Analysis (PCA), Discriminant Analysis, Minimum noise fraction (MNF), etc 3. Inter band block correlation coefficient followed by Singular Value Decomposition(SVD) and QR decomposition is utilized in Corresponding author. Tel.: address: reshmaraju2992@gmail.com The Authors. Published by Elsevier B.V. This is an open access article under the CC BY-NC-ND license ( Peer-review under responsibility of the Organizing Committee of ICACC 2016 doi: /j.procs

2 R. Reshma et al. / Procedia Computer Science 93 ( 2016 ) the proposed work for reducing the dimension of the hyperspectral images 4. Hyperspectral classification techniques are applied to the dimensionally reduced HSI. Orthogonal Matching Pursuit (OMP), Regularized Least Square (RLS), Support Vector Machine (SVM), Subspace Pursuit (SP), Relevance Vector Machine (RVM), etc are the commonly used methods for classification. In the experiment we conducted, classification is performed using kernel based libraries 5,6,7 like Grand Unified Regularized Least Squares (GURLS) 8 and LibSVM 9. The classification accuracy of dimensionally reduced image and original image is compared using various accuracy assessment measures Dimensionality Reduction Hyperspectral images have redundant information due to its strong spectral correlation. So, it is useful to reduce the dimension of these images. Dimensionality reduction can be done based on selection or transformation techniques. In this paper, band selection based technique is used. Inter band block correlation coefficient method followed by Singular Value Decomposition (SVD) and QR decomposition is utilized for reducing the dimension of a hyperspectral image Inter Band Block Correlation Coefficient Inter Band Block Correlation Coefficient (IBBC) 4 is a preprocessing algorithm to remove the water absorption and noisy bands automatically. This technique reduces the computational complexity of band selection techniques. Consider a hyperspectral image of dimension m n l, where l is the number of bands. Each band is divided into blocks of size q q. Ifm, n is not divisible by q, the bands are zero padded with r = q mod(m, q) rows and c = q mod(n, q) columns. So the size of bands become M N, where M = m + r and N = n + c. Now, consider two consecutive bands and find the correlation between the block B i of j th band and B i of ( j + 1) th band. The equation for the correlation coefficient is given by ρ j (i) = (1) N p (b T b i i ) (e T b i )(e T b i ) ; [N p (b T i b i) (e T b i ) 2 ][N p ( b it b i ) (e T b i ) 2 ] 1 j l 1, 1 i N b where b i and b i are the vector form or 1D version of the blocks B i and B i respectively, e is the column vector of ones of size N p 1, N p = q 2 is the number of pixels in each block, N b = M q N q and ρ j R N b. The correlation matrix for all the blocks in consecutive bands is given by ρ = [ ] ρ 1,ρ 2,...,ρ (l 1). Standard deviation of the correlation matrix is found out and represented in vector form as σ = [σ 1,σ 2,...,σ l 1 ] where, σ j = 1 N b (ρ j μ j e) T (ρ j μ j e); (2) Here e is column vector of ones of size N b 1 and μ j = et ρ j N b is the mean of the vector ρ j. A threshold is determined such that all the elements above this threshold are eliminated. Threshold can be based on mean, standard deviation and 2-mean. In this paper, standard deviation of σ is taken as the threshold value Singular Value Decomposition Any matrix X of dimension a b can be decomposed into matrices U, Σ and V T. It is expressed as: X = UΣV T (3) where U is an a a orthogonal matrix with eigen vectors of XX T in its columns, Σ is a a b matrix with square root of eigen values of XX T or X T X as its diagonal elements and V is a b b orthogonal matrix with eigen vectors of X T X in its columns. In SVD based band selection, only the first k eigen values will be considered. ie, k most informative bands are selected and the rest are discarded.

3 398 R. Reshma et al. / Procedia Computer Science 93 ( 2016 ) QR Decomposition QR decomposition (also called as QR factorization) is a process of orthogonalization which decomposes a matrix into an orthogonal matrix Q and an upper triangular matrix R. Consider a a b matrix X with a b, QR decomposition can be written as: X = QR (4) where Q is a a a matrix and R is a a b matrix. Here the diagonal values of R are not arranged in order. To reorder this in descending order, a permutation matrix P is used such that: XP = QRP = Q R (5) 3. ernel Based Classification In this section, the two kernel based classification techniques used here, Support Vector Machine (SVM) 10 and Regularized Least Square(RLS) are briefly explained Regularized Least Square The main goal of Regularized Least Square (RLS) 11 is to minimize the L2 norm of error and there by calculating the weight matrix. This weight matrix is further utilized for predicting the labels of the testing samples. Consider a training set with training samples (x 1, x 2,...,x n ) and training labels (y 1, y 2,...,y n ) where x i R d, y i {1, 2,...,T} for i = 1, 2,...,n. Y is an n T output matrix with Y ij = 1ifi th training sample belongs to j th class and -1 otherwise. The optimization problem for a linear model can be formulated as: min W R d T { 1 n Y XW 2 F + λ W 2 F where X = [x 1, x 2,...x n ] T is a n d matrix. In the case of a non-linear model, the above formulation can be modified as: { } 1 min C R n T n Y C 2 F + λct C. (7) where is an n n matrix which includes the kernel functions ij = k(x i, x j ). Optimum value of C can be calculated and is used for prediction of class label. Grand Unified Regularized Least Squares (GURLS) is a software package used in various supervised learning problems. It utilizes RLS for its functioning and has a great importance in large scale multi class problems Support Vector Machine (SVM) SVM 6 is a supervised machine learning technique used for classification and regression. It was originally designed to serve binary classification. Consider a binary class linearly separable data with training set (x i, y i ) where x i R n is the training sample, y i ( 1, 1) is the training label and i = 1, 2,...,m. The problem formulation of SVM in matrix format for this case is: 1 min w,ξ 2 wt w (8) sub ject to : D(Xw ξe) e where D is a diagonal matrix of size m m with class labels as the diagonal elements, X is the m n data matrix, e is m 1 column vector of ones, w = [w 1, w 2,...w n ] T and ξ R. In case of non-linear data, a mapping function φ is utilized to map the data to higher dimension. LibSVM is a kernel based software library which utilizes multiclass SVM 12 for classification and regression. Usually for a multiclass problem, many binary classifiers are constructed. This can be achieved by one-against-one or one-against-all techniques. In LibSVM, one-against-one method is used to generate k(k 1) 2 binary classes for a k-class problem. } (6)

4 R. Reshma et al. / Procedia Computer Science 93 ( 2016 ) Fig. 1: Flow diagram of important processes in the proposed work. 4. Methodology The experiment is performed on standard Indian Pines and Salinas-A datasets. The image is captured using the 224 band sensor AVIRIS which operates in visible to infrared region in the electromagnetic spectrum. Indian Pines dataset consists of 16 classes spread across pixels of each band. Salinas-A is a subset of Salinas image (within the region ( , )) which comprises of 6 classes spread across pixels. The common experimental procedure is to manually remove the noisy and water absorption bands and using the remaining bands for classification. In this paper, Inter band block correlation coefficient method is used to automatically select and remove the noisy bands. A significant improvement in the computational complexity is achieved on applying the proposed method to the dataset before performing its singular value decomposition. The preprocessed data is then given for dimensionality reduction using SVD. The output matrix V of SVD is given as an input to the QR decomposition algorithm. ernel based classification is done using GURLS and LibSVM libraries on the original and dimensionally reduced data. The hyperspectral cube of size m n l is converted into a two dimensional data of size l mn. 10, 20, 30, 40,and 50% of pixel vectors from each class are chosen for training and all the pixel vectors in the data are taken for testing.before generating training and testing samples, the background pixels (class 0) are removed. Accuracy assessment measures like Class wise accuracy (CA), Overall accuracy (), appa coefficients () and Average accuracy () are used to validate the performance of classification with and without dimensionality reduction. These measures are calculated from the confusion matrix which is generated using original class label and predicted class label. The flow graph of these steps is shown in fig Experiments and Result Analysis The water absorption bands are removed manually from Indian Pines and Salinas-A datasets. This is achieved by discarding the bands [ ], [ ], 220 of Indian pines and bands [ ], [ ], 224 of Salinas-A dataset. The remaining bands are used for classification in GURLS and LibSVM packages. In GURLS, classification was done using LOO (leave one out) and HO (hold out) cross validations in linear, radial basis function (RBF) and randfeats kernels. In this library, the parameters are selected automatically. The classification in LibSVM was performed using linear, polynomial and RBF kernels with 5-fold cross validation. In this paper, the control parameters C and γ are chosen by trial and error method such that these parameters give the best cross validation accuracy. C and γ parameters were estimated to be 1000, 0.4 for Indian pines and 100, 0.5 for Salinas-A dataset. Inter band block correlation coefficient is performed on the hyperspectral image and the remaining 188 bands of Indian Pines and 183 bands of Salinas-A dataset are given for SVD and QR decomposition. After dimensionality reduction, only 100 bands are given for classification. C and γ parameters for dimensionally reduced datasets were estimated to be 10 6, 0.6 for Indian pines and 100,0.5 for Salinas-A. For 10% training sample, the overall accuracy for original and dimensionally reduced Indian pines dataset are 83.52% and 84.67% respectively. Similarly, for Salinas-A dataset the overall accuracies are 99.10% and %. From this experiment it is clear that even with 50% reduction in dimension, comparable accuracies are obtained by kernel based classification. Further to validate the analysis, the experiment was extended to 20, 30, 40 and 50% training samples. The classification accuracies for 10 and 40 % training samples of original and dimension reduced datasets are tabulated in Table1 and Table 2,3 respectively. Figure

5 400 R. Reshma et al. / Procedia Computer Science 93 ( 2016 ) Fig. 2: Ground Truth (a) Indian Pines (b) Salinas-A Table 1: Classification Accuracies of 10% and 40% Indian Pines and Salinas-A dataset for GURLS and LibSVM %of training data Accuracies GURLS ( RBF kernel) LibSVM (Polynomial kernel) Indian Pines Salinas-A Indian Pines Salinas-A Table 2: Classification Accuracies of dimensionally reduced IndianPines dataset for GURLS and LibSVM %of training data Accuracies GURLS LibSVM Linear RBF Randfeat Linear RBF Polynomial CV=loo CV=ho CV=loo CV=ho CV=loo CV=ho CV=5-fold CV=5-fold CV=5-fold shows the ground truth for Indian Pines and Salinas-A datasets. Classification map for 10% training samples of Salinas-A and 10, 30, 50% training samples of Indian pines dataset with dimensional reduction are as shown in fig.5 and fig.4 respectively. Comparison of the Indian pines and Salinas-A datasets with and without reducing dimension is shown in fig Conclusion In this paper, the dimension of Indian Pines and Salinas-A datasets are reduced effectively using band selection techniques. Dimensionality reduction reduces the computational complexity while preserving the critical information in the hyperspectral images. Experimental analysis shows that even with 50% reduction in dimension, the accuracies obtained from kernel based classification in dimensionally reduced image is comparable to that of the original image.

6 R. Reshma et al. / Procedia Computer Science 93 ( 2016 ) Table 3: Classification Accuracies of dimensionally reduced Salinas-A dataset for GURLS and LibSVM %of training data Accuracies GURLS LibSVM Linear RBF Randfeat Linear RBF Polynomial CV=loo CV=ho CV=loo CV=ho CV=loo CV=ho CV=5-fold CV=5-fold CV=5-fold Fig. 3: Comparison of Overall accuracy of the datasets with and without dimensionality reduction (a) Indian Pines (b) Salinas-A Fig. 4: Classification maps of dimensionally reduced Indian Pines data set for 10, 30, 50% training data with classifiers: GURLS - RBF kernel and LibSVM - polynomial kernel (a) GURLS-10% (b) LibSVM-10% (c) GURLS-30% (d) LibSVM-30% (e) GURLS-50% (f) LibSVM-50%

7 402 R. Reshma et al. / Procedia Computer Science 93 ( 2016 ) Fig. 5: Classification maps of dimensionally reduced Salinas-A data set for 10% training data with classifiers: GURLS a) linear-loo b) linear-ho c) rbf-loo d) rbf-ho e) randfeat-loo f) randfeat-ho and LibSVM g) linear h) rbf i) polynomial References 1. Harsanyi, J.C., Chang, C.I.. Hyperspectral image classification and dimensionality reduction: an orthogonal subspace projection approach. Geoscience and Remote Sensing, IEEE Transactions on 1994;32(4): Bruce, L.M., oger, C.H., Li, J.. Dimensionality reduction of hyperspectral data using discrete wavelet transform feature extraction. Geoscience and Remote Sensing, IEEE Transactions on 2002;40(10): Lodha, S.P., amlapur, S.. Dimensionality reduction techniques for hyperspectral images. International Journal of Application or Innovation in Engineering & Management (IJAIEM) 2014;3(10). 4. Bhushan, D.B., Sowmya, V., Manikandan, M.S., Soman,.. An effective pre-processing algorithm for detecting noisy spectral bands in hyperspectral imagery. In: Ocean Electronics (SYMPOL), 2011 International Symposium on. IEEE; 2011, p Schölkopf, B., Smola, A.J.. Learning with kernels: support vector machines, regularization, optimization, and beyond. MIT press; Soman,., Loganathan, R., Ajay, V.. Machine learning with SVM and other kernel methods. PHI Learning Pvt. Ltd.; Haridas, N., Sowmya, V., Soman,.. Gurls vs libsvm: Performance comparison of kernel methods for hyperspectral image classification. Indian Journal of Science and Technology 2015;8(24). 8. Tacchetti, A., Mallapragada, P.., Santoro, M., Rosasco, L.. Gurls: a least squares library for supervised learning. The Journal of Machine Learning Research 2013;14(1): Chang, C.C., Lin, C.J.. Libsvm: a library for support vector machines. ACM Transactions on Intelligent Systems and Technology (TIST) 2011;2(3): Gualtieri, J., Chettri, S.R., Cromp, R., Johnson, L.. Support vector machine classifiers as applied to aviris data. In: Proc. Eighth JPL Airborne Geoscience Workshop. 1999,. 11. Tacchetti, A., Mallapragada, P.S., Santoro, M., Rosasco, L.. Gurls: a toolbox for regularized least squares learning 2012;. 12. Hsu, C.W., Lin, C.J.. A comparison of methods for multiclass support vector machines. Neural Networks, IEEE Transactions on 2002; 13(2):