Hyperspectral Images Classification Using Energy Profiles of Spatial and Spectral Features

Transcription

1 journal homepage: Hyperspetral Images Classifiation Using Energy Profiles of Spatial and Spetral Features Hamid Reza Shahdoosti a a Hamedan University of ehnology, Department of Eletrial Engineering, Hamedan, , Iran, b arbiat Modares University, Department of Eletrial and Computer Engineering, ehran, , Iran ABSRAC 01 Elsevier Ltd. All rights reserved. his paper proposes a spatial feature extration method based on energy of the features for lassifiation of the hyperspetral data. A proposed orthogonal filter set extrats spatial features with maximum energy from the prinipal omponents and then, a profile is onstruted based on these features. he important harateristi of the proposed approah is that the filter set s oeffiients are extrated from statistial properties of data, thus they are more onsistent with the type and texture of the remotely sensed images ompared with those of other filters suh as Gabor. o assess the performane of the proposed feature extration method, the extrated features are fed into a support vetor mahine (SVM) lassifier. Experiments on the widely used hyperspetral images namely, Indian Pines, and Salinas data sets reveal that the proposed approah improves the lassifiation results in omparison with some reent spetral spatial lassifiation methods. Keywords: Classifiation, spatial features, filter set, spetral features, hyperspetral imagery. Corresponding author. el.: ; h.doosti@hut.a.ir

2 1. Introdution he availability of remotely sensed hyperspetral images aquired in the adjaent bands of the eletromagneti spetrum, makes it neessary to propose tehniques whih are able to interpret suh high-dimensional data in many various appliations. One of the most important appliations in the field of hyperspetral images is the lassifiation in whih land overs are distinguished from eah other. For this purpose, several tehniques have been developed to greatly benefit from the wealth of spetral and spatial information buried in the hyperspetral images. Various experiments have proven the sparseness of highdimensional data spaes suh that the data struture exists primarily in a subspae (Shahdoosti and Mirzapour, 017). As a onsequene, there is a need for feature extration methods by whih the dimensionality of the data an be redued to the right subspae without losing the essential information that allows for the separation of lasses. hese tehniques an be mainly lassified into two ategories (Shahdoosti and Javaheri 019): 1) Several methods use linear transforms to extrat the spetral or spatial information from hyperspetral data. Widely used linear feature extration methods in the spetral domain inlude prinipal omponent analysis (PCA) (Shahdoosti and Ghassemain, 01, 015, 016), independent omponent analysis (ICA) (Bayliss et al., 1997) and linear disriminant analysis (Bandos et al., 009), and those in the spatial domain inlude the Gabor filter bank (Mirzapour and Ghassemian, 015) and wavelets. ) Several tehniques exploit spetral or spatial features obtained through nonlinear transformations. Examples of these methods are morphologial analysis (Plaza et al., 005; Benediktsson et al., 005), kernel methods (Camps-Valls and Bruzzone, 005), and manifold regularization (Ma et al., 010). his paper fouses on using linear transformations for feature extration methods. Linear transformations are attrative for image proessing owing to the fat that they have a very low omputational burden while, at the same time they are suessful in extrating relevant information from the hyperspetral images. On the other hand, feature extration methods an be lassified into either supervised or unsupervised ones (Ghassemian and Landgrebe, 1988; Khayat et al. 008, Shahdoosti and Javaheri 018). For instane, PCA, Gabor filter bank, ICA and morphologial analysis are unsupervised, i.e. these methods do not use the training samples, available for the lassifiation proedure. But, LDA is an example of the supervised methods whose features are extrated by making use of the training samples. Spetral-based per-pixel lassifiation methods annot take into aount the spatial relationship between the pixels satisfatorily, and it has been widely aepted that the buried spatial information should be exploited as a omplementary feature soure for distinguishing the land overs. o this end, filter banks are usually used to extrat the spatial features of the hyperspetral data, to be used in the lassifiation proedure. A relevant riterion for determining the filter set s oeffiients is energy, suh that the filters maximizing the energy of features are the appropriate filters for whih we look. For this purpose, this paper proposes a diretional filter set whih extrats spatial information with maximum energy from the remotely sensed images. It will be reasoned that the proposed method has a low omputational burden too. he paper is strutured in five setions. Designing the filter set is presented in setion. In setion 3, the energy profile is onstruted inluding the spetral-spatial features of the hyperspetral data. Setion 4 presents the experimental results and disussions. Eventually, setion 5 is devoted to the onlusions.. Proposed filter set he aim is to design a two dimensional diretional filter f R : f1,1... f1, f =... f , f... f,1, (1) where denotes the number of rows and olumns of the filter, by whih the features of an image m R r l are extrated, where r and l are the number of rows and olumns in the image, respetively. By a matrix multipliation, one an desribe the onvolution operation (Jayaraman et al., 011): A = f* m = Matrix{ F C} () where the matrix A R r l is the extrated feature by the filter f whih should be used in the lassifiation proedure, * is the onvolution operation, F is the olumn-wise version of the filter 1 f, i.e., F= [ f1,1,..., f1,,..., f,1,..., f,] R and C R rl is a blok oeplitz matrix formed by m, suh that the matrix C is onstituted by the shifted and rearranged versions of the image m. In addition, the operation Matrix returns the result 1 of F C R rl into the matrix format. Fig. 1, shows the steps of desribing a two dimensional onvolution by the matrix analysis, aording to Ref. (Jayaraman et al., 011). A relevant riterion for determining F is energy, suh that the filter F maximizing the energy of features is the appropriate filter for whih we look. So, Var( F C ) should be maximized where Var denotes the variane. As it is lear, the maximum of Var( F C ) will not be ahieved for finite F, so a normalization onstraint should be added to the maximization problem. One an easily propose the onstraint F F = 1, that is, the sum of squares of elements of F equals 1. his onstraint doesn t impose the filter F to be a high-pass, low-pass or bandpass filter. So, any type of the filter an be obtained by this onstraint. he maximization problem is: 1 1 λ1 1 1 F J( F)=argmax{Var( F C)}- ( F F- 1) (3) 1 where J is the objetive funtion whih should be maximized. his eigenvalue problem has to be solved to find the filter F 1 i.e. the first filter extrating the first spatial feature. Differentiation of equation (3) with respet to F 1 gives: Ω F -λf = 0 (4) 1 1 he filter F 1 is omposed of the eigenvetor of Ω R orresponding to its first largest eigenvalue λ 1, where Ω is the ovariane matrix of C. he seond filter F, should maximize = Var( F C) F Ω F subjet to being unorrelated with F 1, or equivalently subjet to ov[ F C, F 1 C ] = 0. So: Cov[ F C, F1 C] = F Ω F1 = F1 Ω F =F λ1f1 = λ1f F1 = λ1f1 F = 0 Now, the optimization problem is: λ Ψ 1 F J( F )=argmax{ F Ω F }- ( F F -1) F F (6) where λ and Ψ are the Lagrange multipliers. Differentiation with respet to F and then multipliation of the result on the left by F 1 gives: (5)

3 representation in the mean square sense, this suggestion seems reasonable. But, it should be noted that with only one prinipal omponent, the hyperspetral data are redued from potentially several hundred data hannels into one dimensional data hannel. Moreover, although the first prinipal omponent represents most of the variation in the data, other prinipal omponents ontain some important information. hus, similar to Ref. (Benediktsson et al., 005), we make use of several different prinipal omponents to build an energy profile. In this paper, we make use of the prinipal omponents that aount for around 90% of the total variation in the hyperspetral data. For example, if two prinipal omponents ontain 90% of the variation, the energy profile would be a double profile where the first profile would be based on the first prinipal omponent and the seond profile would be based on the seond prinipal omponent (see Fig. ). For eah profile, the orresponding prinipal omponent is onsidered as an image to whih the proposed filter set is applied. By the proposed filter set, the features of the prinipal omponents are extrated. hese features onstruting the energy profile are fed into an arbitrary lassifier to obtain the lassifiation results. 4. Experimental results Fig. 1. Steps of a two dimensional onvolution by the matrix analysis (Jayaraman et al., 011). F Ω F λ F F Ψ F F = 0 (7) Beause the first two terms are zero and 1 1 F F = 1, one an easily onlude Ψ= 0. So the optimization problem of equation (6) an be simplified to: J( F )=argmax{ F Ω F }- λ ( F F -1) (8) F Similar to the solution of Eq. (3), λ is the seond largest eigenvalue of Ω, and F is the orresponding eigenvetor. It an be shown that F 3, F 4,, F n are the eigenvetors of orresponding to λ 3, λ 4,, λ n, the third to n th largest eigenvalues. his solution is similar to the PCA usually used for extration of the spetral features, but this time PCA is applied in the spatial domain. Note that the maximum number of orthogonal filters whih an be designed is (beause the dimension of Ω is ), and onsequently, the maximum number of features orresponding to eah pixel is equal to. Here several effiient and diretional filters were designed, by whih one an extrat the spatial features of hyperspetral images and feed these features to the lassifiers. he proposed method has a reasonable omputational omplexity. It is just needed to ompute a ovariane matrix ( Ω ) and ompute its eigen vetors to design the filter set. 3. Construting the energy profile In order to apply the filter set to hyperspetral data, omputing the prinipal omponents of the hyperspetral image is the first step. he most signifiant prinipal omponents, whih are the omponents with maximum energy, are used as base images for the energy profile, i.e., a profile based on more than one original image. When the proposed filter set is applied to hyperspetral data, a harateristi image needs to be extrated from the data. It was suggested to use the first prinipal omponent of the hyperspetral data for this purpose (Palmason et al., 003). Beause the prinipal omponent analysis is optimal for data Ω he goal of this setion is to evaluate the performane of the proposed energy profile for lassifiation of hyperspetral images. We apply the proposed method to the two well-known real hyperspetral data sets and lassify these images by the energy profile. As mentioned before, the lassifier used in our experiments is SVM (Huang et al., 015; Guo and Boukira, 015) with polynomial kernel of degree 3. he SVM lassifier is implemented using LIBSVM in whih the default kernel parameter values i.e. γ = 1/Number of features, and 0 = 0 are used (Chang and Lin, 001). A number of labeled samples of eah lass are randomly seleted to train the SVM lassifier. In order to study the effet of the training sample size on the proposed feature extration method, we have onsidered four different proportional shemes for the number of training samples: (a) 1%, (b) 5%, () 10% and (d) 1.5%. A point whih should be onsidered is that there are some lasses with small number of labeled samples in Indian Pines data. o ensure that we have enough training samples for the SVM lassifier, a minimum of 3 samples per lass is onsidered in shemes (a) and (b). In other words, we have n = max(3,0.01 N ) for sheme (a) and nl = max(3,0.05 Nl) for sheme (b), where N is the l number of labeled samples of the l th lass. Although the size of parameter is dependent on the spatial struture and texture of the remotely sensed image, this parameter should be large enough to guarantee apturing the spatial information of the image. he size of is seleted 35 in our experiments, beause l able 1. Sixteen ategories and orresponding number of labeled pixels in the Indian Pines data No Category pixels No Category pixels * Alfalfa 46 * Oats 0 * Corn-no till 148 * Soybeans-no till 97 * Corn-min till 830 * Soybeans-min till 455 * Corn 37 * Soybeans-lean 593 * Grass/pasture 483 * Wheat 05 * Grass/trees 730 * Woods 165 * Grass/pasture-mowed 8 * Bldg-Grass-ree-Drives 386 * Hay-windowed 478 * Stone-Steel-owers 93 l

4 Fig..Blok-diagram of the proposed energy profile. able. OA, AA, and Kappa statistis (κ ) obtained by the SVM lassifier on the Indian Pines data. Different ombinations of the extrated features are used. he symbol + denotes staking. Sheme (a) 1% raining set (randomly seleted) Sheme (b) 5% Sheme () 10% Sheme (d) 1.5% Case Input features to SVM OA AA κ OA AA κ OA AA κ OA AA κ * HS * HS+MP * HS+MP+Gab * HS+MP+Gab+SGl * HS+MP+Gab+SGl+Gl * PCA * PCA+MP * PCA+MP+Gab * PCA+MP+Gab+SGl * PCA+MP+Gab+SGl+Gl * LDA * LDA +MP * LDA +MP+Gab * LDA +MP+Gab+SGl * LDA+MP+Gab+SGl+Gl * Proposed the improvement is not aeptable for the filter set with bigger size. he first hyperspetral data set is Indian Pines one, aquired on 1 June 199 by the AVIRIS sensor, overing a.9 km.9 km portion of northwest ippeanoe County, Indiana, USA. wo-thirds of this sene is agriulture, and one-third of it, is forest or other natural perennial vegetation. his data inludes 0 bands with pixels and a spatial resolution of 0 m. wenty bands of this data set have been removed beause they over the water absorption spetrum band ( , , 0). he orreted data set inluding 00 bands has been used in our experiments. here are 16 different land-over lasses available in the original ground-truth map. able 1 lists the number of labeled pixels of eah lass. Due to the presene of mixed pixels in all available lasses and beause there are lasses with small number of labeled pixels, this data set onstitutes a hallenging lassifiation problem (Song et al., 014). Several ombinations of spetral and spatial features (16 ases) whih are very suessful aording to (Mirzapour and Ghassemian, 015) are fed into the SVM lassifier and the lassifiation results are given in table. Note that the results of all the tables (tables, 4 and 5) are reported after running 50 times (the Monte Carlo method) and averaging the values. he abbreviations in these tables are: OA is overall auray, AA is average auray, κ is kappa statistis, HS is hyperspetral image, MP is Morphologial Profile, Gab is Gabor features, Gl is GLCM (gray-level o-ourrene matrix) features and SGl is segmentation-based GLCM features. In addition, Fig. 3 shows the sample lassifiation map obtained by the proposed method.

5 able 3. Sixteen ategories and orresponding number of labeled pixels in the Indian Pines data No Category pixels No Category pixels * Brooli_green_weeds_1 009 * Soil_vinyard_develop 603 * Brooli_green_weeds_ 376 * Corn_senesed_weeds 378 * Fallow 1976 * Lettue_romaine_4wk 1068 * Fallow_rough_plow 1394 * Lettue_romaine_5wk 197 * Fallow_smooth 678 * Lettue_romaine_6wk 916 * Stubble 3959 * Lettue_romaine_7wk 1070 * Celery 3579 * Vinyard_untrained 768 * Grapes_untrained 1171 * Vinyard_vertial_trellis 1807 he seond data set is Salinas one, olleted by the AVIRIS sensor over Salinas Valley, California, USA. Eah spetral band has pixels, where the geometri resolution of eah pixel is 3.7 m. As with the Indian Pines sene, the 0 water absorption bands (108 11, , 4) are disarded, resulting in a orreted image ontaining 04 spetral bands. his data inludes 16 agriultural land overs with very similar spetral signatures (Plaza et al., 005). he number of labeled pixels of eah lass is listed in table 3. he mentioned indies are shown in table 4 aording to the orresponding shemes. Moreover, Fig. 4 shows the sample lassifiation map obtained by the proposed method. At the end of this setion, we provide a omparative assessment. he proposed method is ompared with some reently proposed spetral spatial feature extration methods. o do a reliable omparison, the lassifiation results for SVM-CK (Camps-Valls et al., 006) are obtained by using the publily available odes. But, the results for other methods are reported from the orresponding papers. hus, some results are absent. he results of this omparison is reported in table 5. As an be seen from this table, finding a unique feature extration method whih gives the best results for the different number of training samples and for the different data sets, is diffiult. In fat, it is not exaggerated if one say that there does not exist suh features (Kuo et al. 009). As an be seen from table 5, the proposed feature extration method is almost ertainly the best when there are enough training samples. In addition, the proposed method has a satisfatory performane when the number of training samples is low. 5. Conlusion A new spatial-spetral feature extration method for lassifiation of hyperspetral images was proposed. By maximizing the energy of the extrated features, and by able 4. OA, AA, and Kappa statistis (κ ) obtained by the SVM lassifier on the Salinas data. Different ombinations of the extrated features are used. raining set (randomly seleted) Sheme (a) 1% Sheme (b) 5% Sheme () 10% Sheme (d) 1.5% Case Input features to SVM OA AA κ OA AA κ OA AA κ OA AA κ * HS * HS+MP * HS+MP+Gab * HS+MP+Gab+SGl * HS+MP+Gab+SGl+Gl * PCA * PCA+MP * PCA+MP+Gab * PCA+MP+Gab+SGl * PCA+MP+Gab+SGl+Gl * LDA * LDA +MP * LDA +MP+Gab * LDA +MP+Gab+SGl * LDA+MP+Gab+SGl+Gl * Proposed Data set able 5. Overall auray of the proposed method ompared with some reently proposed lassifiation methods Number of raining samples Diverse AdaBoost SVM FODPSO + MSS (R = 5) + SVM EMAP (KPCA)+RF lassifier SUnSAL EMAP SVM CK Proposed 1% Indian Pines 5% % % % Salinas 5% % % Reported from Ref. (Ramzi et al., 013). Reported from Ref. (Ghamisi et al., 014). Reported from Ref. (Bernabé et al., 014). Reported from Ref. (Sonf et al., 014). Reported from Ref. (Camps-Valls et al., 006).

6 Fig. 3. he lassifiation result for the AVIRIS Indian Pines sene. a) Ground truth map. b) he lassifiation result of the proposed method (1.5% of the available labeled data is used). Fig. 3. he lassifiation result for the Salinas sene. a) Ground truth map. b) he lassifiation result of the proposed method (1.5% of the available labeled data is used). unorrelating the filters, oeffiients of the spatial filters are designed. o provide the energy profile inluding spetral-spatial features of hyperspetral data, the PCA is applied to the hyperspetral data at the first step. hen, the proposed filter set is applied to the prinipal omponents that aount for around 90% of the total variation in the hyperspetral data. o evaluate the performane of the proposed feature extration method, the SVM lassifier with the polynomial kernel of degree 3 was used to lassify the extrated features. wo real hyperspetral data sets namely, Indian Pines and Salinas data were used in our experiments. he experimental results have demonstrated that the proposed feature extration method an provide effiient features for the lassifiation appliation. Referenes a a b b [] Bayliss, J., Gualtieri, J. A., Cromp, R., Analysing hyperspetral data with independent omponent analysis. Pro. SPIE, vol. 340, pp [3] Benediktsson, J. A., Palmason, J. A., Sveinsson, J. R., 005. Classifiation of hyperspetral data from urban areas based on extended morphologial Profiles. IEEE rans. Geosi. Remote Sens., vol. 43, no. 3, pp [4] Bernabé, S., Marpu, P. R., Plaza, A., 014. Spetral spatial lassifiation of multispetral images using kernel feature spae representation. IEEE Geosi. Remote Sens. Lett., vol. 11, no. 1, pp [5] Camps-Valls, G., Bruzzone, L., 005..Kernel-based methods for hyperspetral image lassifiation. IEEE rans. Geosi. Remote Sens., vol. 43, no. 6, pp [6] Camps-Valls, G., Gomez-Chova, L., Muñoz-Marí, J., Vila-Franés, J., Calpe-Maravilla, J., 006. Composite kernels for hyperspetral image lassifiation. IEEE Geosi. Remote Sens. Lett., vol. 3, no. 1, pp [7] Chang, C.C., Lin, C.J., 001. LIBSVM: A Library for Support Vetor Mahines. [Online]. Available: [8] Ghamisi, P., Coueiro, M., Fauvel, M., Benediktsson, J. A., 014. Integration of segmentation tehniques for lassifiation of hyperspetral images. IEEE Geosi. Remote Sens. Lett., vol. 11, no. 1, pp [9] Ghassemian, H., Landgrebe, D.A., Objet-oriented feature extration method for image data ompation. IEEE Control Syst. Mag., vol. 8, no. 3, pp [10] Guo, L., Boukira, S., 015. Fast data seletion for SVM training using ensemble margin. Pattern Reognition Letters, vol. 51, pp [11] Huang, Q., Chang, S., Liu, C., Niu, B., ang M., Zhou, Z., 015. An evaluation of fake fingerprint databases utilizing SVM lassifiation. Pattern Reognition Letters, vol. 60, pp [1] Khayat, O., Shahdoosti, H.R. and Khosravi, M.H., 008, February. Image lassifiation using prinipal feature analysis. In Proeedings of the 7th WSEAS International Conferene on Artifiial intelligene, knowledge engineering and data bases (pp ). World Sientifi and Engineering Aademy and Soiety (WSEAS). [13] Khayat, O., Shahdoosti, H.R. and Motlagh, A.J., 008, February. An overview on model-based approahes in fae reognition. In International Conferene on Artifiial Intelligene, Knowledge Engineering and Databases (pp ). [14] Kuo, B.C,. Li, C.H., Yang, J.M., 009. Kernel nonparametri weighted feature extration for hyperspetral image lassifiation. IEEE rans. Geosi. Remote Sens., vol. 47, no. 4, pp [15] Li, J., Huang, X., Gamba, P., Biouas, J., Zhang, L., Benediksson, J., Plaza, A., 015. Multiple Feature Learning for Hyperspetral Image Classifiation. IEEE rans. Geosi. Remote Sensing, vol.53, no.3, pp [16] Ma, L., Crawford, M.M., ian, J., 010. Loal manifold learning-based k-nearest-neighbor for hyperspetral image lassifiation. IEEE rans. Geosi. Remote Sens., vol. 48, no. 11, pp [17] Mirzapour, F., Ghassemian, H., 015.Improving hyperspetral image lassifiation by ombining spetral, texture, and shape features. Int. J. Remote Sens., vol. 36, no. 4, pp [18] Palmason, J. A., Benediktsson, J. A., Arnason, K., 003. Morphologial transformations and feature extration for urban data with high spetral and spatial resolution. in Pro. IGARSS, vol. 1, oulouse, Frane, pp [19] Plaza, A., Martinez, P., Plaza, J., Perez, R., 005. Dimensionality redution and lassifiation of hyperspetral image data using sequenes of extended morphologial transformations. IEEE rans. Geosi. Remote Sens., vol. 43, no. 3, pp [0] Ramzi, P., Samadzadegan, F., Reinartz, P., 013. Classifiation of Hyperspetral Data Using an AdaBoost SVM ehnique Applied on Band Clusters. IEEE J. Sel. op. Appl. Earth Observ. Remote Sens., vol. 7, no. 6,

7 [] Shahdoosti, H.R. and Mirzapour, F., 017. Spetral spatial feature extration using orthogonal linear disriminant analysis for lassifiation of hyperspetral data. European Journal of Remote Sensing, 50(1), pp [3] Shahdoosti, H.R. and Javaheri, N., 018. A new hybrid feature extration method in a dyadi sheme for lassifiation of hyperspetral data. International Journal of Remote Sensing, 39(1), pp [4] Shahdoosti, H.R. and Javaheri, N., 018. A fast algorithm for feature extration of hyperspetral images using the first order statistis. Multimedia ools and Appliations, DOI: pp [5] Shahdoosti, H.R. and Javaheri, N., 018. A new kernel fuzzy based feature extration method using attration points. Multidimensional Systems and Signal Proessing, DOI: pp [6] Shahdoosti, H.R. and Javaheri, N., 017. Pansharpening of lustered MS and Pan images onsidering mixed pixels. IEEE Geosiene and Remote Sensing Letters, 14(6), pp [7] Shahdoosti, H.R. and Ghassemian, H., 016. Combining the spetral PCA and spatial PCA fusion methods by an optimal filter. Information Fusion, 7, pp [8] Shahdoosti, H.R. and Ghassemian, H., 01, May. Spatial PCA as a new method for image fusion. In Artifiial Intelligene and Signal Proessing (AISP), 01 16th CSI International Symposium on (pp ). IEEE. [9] Shahdoosti, H.R. and Ghassemian, H., 015. Multispetral and Panhromati Image Fusion by Combining Spetral PCA and Spatial PCA Methods. he Modares Journal of Eletrial Engineering, 11(3), pp [30] Song, B., Li, J., Dalla-Mura, M., Li, P., Plaza, A., Biouas-Dias, J. M., Benediktsson, J. A., Chanussot, J., 014. Remotely sensed image lassifiation using sparse representations of morphologial attribute profiles. IEEE rans. Geosi. Remote Sens., vol. 5, no. 8, pp