Hyperspectral Image Classification by Using Pixel Spatial Correlation

Transcription

1 Hyperspectral Image Classification by Using Pixel Spatial Correlation Yue Gao and Tat-Seng Chua School of Computing, National University of Singapore, Singapore Abstract. This paper introduces a hyperspectral image classification approach by using pixel spatial relationship. In hyperspectral images, the spatial relationship among pixels has been shown to be important in the exploration of pixel labels. To better employ the spatial information, we propose to estimate the correlation among pixels in a hypergraph structure. In the constructed hypergraph, each pixel is denoted by a vertex, and the hyperedge is constructed by using the spatial neighbors of each pixel. Semi-supervised learning on the constructed hypergraph is conducted for hyperspectral image classification. Experiments on two datasets are used to evaluate the performance of the proposed method. Comparisons with the state-of-the-art methods demonstrate that the proposed method can effectively investigate the spatial relationship among pixels and achieve better hyperspectral image classification results. Keywords: Hyperspectral image classification, spatial correlation, hypergraph learning. 1 Introduction A hyperspectral image is a spatially sampled image which is gathered from hundreds of contiguous narrow spectral bands (from the visible to the infrared bands) by hyperspectral sensors [3]. Recently, the hyperspectral image data is rapidly increasing collected by hyperspectral instruments such as NASA Airborne Visible Infra-Red Imaging Spectrometer and Reflective Optics System Imaging Spectrometer. Different from general images, hyperspectral image comes with high dimensional feature spaces and many of the cotents are not visible by humans. Hyperspectral imaging has attracted extensive research efforts [11]. One key research in Hyperspectral classification which aims to classify its pixels into different categories. By consider the fact that hyperspectral images contain hundreds of spectral bands and the human labelling is expensive, the main challenges in hyperspectral image classification lie in the need to deal with few training samples with high data dimensionality. Existing works on hyperspectral image classification mainly focus on either feature dimension reduction or semi-supervised classification. Traditional feature dimension reduction methods, such as Independent Component Analysis and Principal Component Analysis, have been investigated in previous works [17]. A kernel nonparametric weighted feature extraction [10] method has been proposed to extract hyperspectral image feature by using a kernel nonparametric method. For feature dimension reduction, S. Li et al. (Eds.): MMM 2013, Part I, LNCS 7732, pp , c Springer-Verlag Berlin Heidelberg 2013

2 142 Y. Gao and T.-S. Chua another approach is to perform band selection which aims to select a group of bands from the original high-dimensional feature space. In [9], the correlation between each two spectral bands is measured by mutual information, and the representative bands are selected by minimizing the distance between the selected bands and the estimated reference map. Then the representative bands are selected by using a clustering-based method [14] in which the bands with the largest similarity to other bands are chosen. For hyperspectral image classifiers, K-Nearest Neighbor classifier (KNN) and Support Vector Machine (SVM) have been employed [13]. A semi-supervised graph-based learning method [4] is introduced to represent the hyperspectral image by using a graph structure, and then a semi-supervised learning process on the graph is conducted for hyperspectral image classification. Gu et al. [8] introduced a representative multiple kernel learningapproach to automatically combine multiple kernels in the learning procedure. Manifold learning [18] has been investigated for hyperspectral image classification in combination with the KNN classifier. In [12], a manifold structure is constructed by the pixels and the local manifold learning is conducted in the manifold structure, and a weighted KNN classifier is employed for hyperspectral image classification. Classification of hyperspectral image with few training data has attracted wide research attention recently [2]. For instance in [15], sparse representation has been investigated in hyperspectral image classification to deal with the few labeled samples. Similar to traditional image corpus, hyperspectral images contain high spatial correlation among pixels. Nearby pixels in one hyperspectral image are captured from spatially close area, which are likely to share the same labels. This spatial information plays an important role in the understanding and classification of hyperspectral image categories. However, existing works mainly employ spatial information for noise removal or image smoothing. For instance, a spatial preprocessing approach is introduced in [16] to remove noise and smooth the image by enhancing spatial texture information with locally linear embedding in the feature space. However, none of the existing works considers spatial information embedding in terms of hyperspectral image representation, which is one of the main main challenges in the hyperspectral image classification task. In addition, how to better employ the spatial information of the pixels to improve the hyperspectral image classification performance still requires further investigation. In most of existing hyperspectral image classification works, each pixel is mainly described by the high dimensional feature, which leads to the curse of dimensionality. It is noted that the spatial information can be also used to explore the relationship among pixels. In this work, we propose to employ the pixel spatial correlation for hyperspectral image classification, in which a hypergraph structure is constructed to estimate the relationship among pixels. The use of hypergraph is to alleviate the curse of dimensionality problem as most computations are done at the local region, which has been widely investigated in image search [22,5], object classification [20,21], and 3D object retrieval [6] and recognition [7]. Hypergraph has also been employed in Hyperspectral image classification task [19], in which the constructed hypergraph aims to explore the feature-based pixels relationship. Different from [19], we focus on the pixel spatial correlation in this work. Figure 1 illustrates the flowchart of the proposed method. In this method, the relationship among pixels is formulated in a hypergraph structure, in which each vertex denotes one pixel in the hyperspectral image. To construct the hyperedges,

3 Hyperspectral Image Classification by Using Pixel Spatial Correlation 143 each pixel is connected to its spatial neighbor pixels, which generate one hyperedge for the hypergraph. By using the training data, we conduct semi-supervised learning in the constructed hypergraph for hyperspectral image classification. Experiments on two datasets, i.e., the Indian Pine and the Indian Pine Sub, are used to evaluate the performance of the proposed method. The advantages of the proposed hypergraph method are two-fold. First, it does not require the high dimensional feature reduction process. The relationship among pixels is constructed by using the spatial correlation. The distance in the feature space is only employed to estimate the weights for each pixel in one hyperedge as introduced in the next section. Second, the employed hypergraph structure is able to capture the complex relationship among different pixels, while leads to superior hyperspectral image classification results. Fig. 1. The flowchart of the proposed hyperspectral image classification method by using pixel spatial correlation The rest of the paper is organized as follows. Section 2 introduces the proposed hyperspectral image classification method by using the spatial information. Experimental results and comparison with the state-of-the-art methods on two datasets are provided in Section 3. Finally, we conclude the paper in Section 4. 2 Hyperspectral Image Classification by Using Pixel Spatial Correlation In this section, we introduce the proposed hyperspectral image classification method by using pixel spatial correlation as shown in Fig. 1. First, we introduce the hyperspectral hypergraph construction process by spatial correlation. Next, we describe the learning process on the constructed hypergraph. 2.1 Hypergraph Construction by Using Pixel Spatial Correlation In the proposed method, the relationship among pixels in the hyperspectral image is formulated in a hypergraph structure. In this part, we introduce the hypergraph

4 144 Y. Gao and T.-S. Chua (a) 4 neighbors (b) 8 neighbors (c) 12 neighbors (d) 20 neighbors (e) 24 neighbors Fig. 2. The illustration of the spatial-based hyperedge construction procedure construction procedure by using pixel spatial correlation. In the constructed hypergraph G = {V, E, W}, each vertex denotes one pixel in the hyperspectral image X={x 1,x 2,,x n }. Therefore, there are n vertices totally in G. In a hypergraph structure, each hyperedge connects multiple vertices. To construct the hyperedge, the spatial correlation of pixels are taken into consideration. In this process, each pixel is selected as the centroid and connected to its spatial neighbors, which generates one hyperedge. This hyperedge construction method is under the assumption that spatial connected pixels should have large possibility to have the same labels. As each pixel generates one hyperedge, there is a total of n hyperedges. Some spatial hyperedge construction examples are shown in Fig. 2, in which the numbers of spatial neighbors for the centroid are 4, 8, 12, 20, and 24 respectively. In Fig. 2, the red pixel is the centroid, and it connects its spatial neighbors (green pixels) in the constructed hyperedge. Under this formulation, the pixels which are closing in the spatial layout can be connected by hyperedges, and the connection relationship can be further extended through hyperedges. Let the selected number of spatial neighbors be K, and there are totally K+1 vertices in one hyperedge. Each hyperedge e Eisgiven a weight w (e) =1, which reveals that all hyperedges are with equal influence on the constructed hypergraph structure. Though each hyperedge plays an equal role in the whole hypergraph structure, the pixels connected by one hyperedge may be not close enough in the feature space. Therefore, these pixels may have different weights in the corresponding hyperedge. For a hyperedge e E, the entry of the incidence matrix H of the hypergraph G is generated by: { ( 1 ) if v = v c H (v, e) = exp d2 (v,v c) (1) 2σ otherwise 2 where v c is the centroid pixel, d (v, v c ) is the distance between one pixel v in E and v c,andσ is the mean distance among all pixels. Under this definition, the pixels in one hyperedge which are similar to the centroid pixel in the feature space can be strongly connected by the hyperedge, and other pixels are with weak connection by the hyperedge. By using the generated incidence matrix H, the vertex degree of a vertex v Vand the edge degree of a hyperedge e Eare generated by: d (v) = e E H (v, e) (2)

5 Hyperspectral Image Classification by Using Pixel Spatial Correlation 145 and d (e) = v V H (v, e) (3) In the above formulation, D v and D e denote the diagonal matrices of the vertex degrees and the hyperedge degrees respectively, and W denotes the diagonal matrix of the hyperedge weights, which is an identity matrix. 2.2 Learning on the Constructed Hypergraph With the constructed hypergraph structure, we conduct a semi-supervised learning for classification by using the training data, which follows the regularization framework proposed in [23] as follows: arg min {Ω(F)+λR emp(f)} (4) F In the above formulation, F =[f 1,f 2,,f C ] is the confidence score matrix for hyperspectral image classification, where f s (t) is the confidence score to categorize the s-th pixel into the t-th class, and C is the number of pixel categories. This formulation aims to minimize the empirical loss R emp under the constraint of the hypergraph regularizer Ω (F), and it guarantees that the pixels with strong spatial correlations have large possibilities to share the same labels. R emp is defined by: R emp = C f k y k 2, (5) k=1 where y k is an n 1 labeled training vector for the k-th class, and Y =[y 1,y 2,,y C ]. λ>0 is a tradeoff parameter, and Ω (F) is the hypergraph regularizer on the hyperspectral hypergraph structure, which is defined by Eq. (6), Ω (F) = 1 2 = C C k=1 e E u,v V = C k=1 e E u,v V F 2 u,k k=1 u V e E = C fk T k=1 w(e)h(u,e)h(v,e) δ(e) (I Θ) f w(e)h(u,e)h(v,e) δ(e) w(e)h(u,e) d(u) v V ( ) 2 F u,k F v,k d(u) d(v) ( ) F 2 u,k d(u) F u,k F v,k d(u)d(v) H(v,e) δ(e) e E u,v V F u,k H(u,e)w(e)H(v,e)F v,k d(u)d(v)δ(e) (6) where Θ = D 1 2 v HWD 1 e H T D 1 2 v. Here we let Δ = I Θ, andω (F) can be written as: Ω (F) = C fk T Δf k. (7) k=1

6 146 Y. Gao and T.-S. Chua Now the objective function can be rewritten as: { C } arg min fk T Δf k + λ C f k y k 2 F k=1 k=1 s.t. λ>1 (8) According to [23], it can be derived as: F = ( I + 1 λ Δ ) 1 Y (9) The pixel-category correlation can be obtained after the confidence score matrix F has been generated. With F, each pixel in the hyperspectral image can be classified to the category with the highest confidence score. 3 Experiments In this section, we first describe the testing datasets and then discuss the experimental results and the comparison with the state-of-the-art methods. 3.1 The Testing Datasets In our experiments, two datasets are employed to evaluate the performance of the proposed method. The first dataset is the Airborne Visible/Infrared Imaging Spectrometer (AVIRIS) image taken over NW Indiana s Indian Pine test site, which has been widely employed [1,4]. The Indian Pine dataset is with the resolution of pixels and has 220 spectral bands. 20 bands are removed due to the water absorption bands, and finally 200 out of the 220 bands are used in our experiment. There are originally 16 classes in total, ranging in size from 20 to 2455 pixels. Some small classes have been removed and only 9 classes are selected for evaluation. The details information about the selected classes is shown in Table 1. Table 1. Details of the Indian Pine Dataset Class #Pixels Class #ofpixels Class #ofpixels Soybeans-no till 972 Corn-no till 1428 Grass/pasture 483 Soybeans-min 2455 Corn-min 830 Grass/trees 730 Soybeans-clean till 593 Woods 1265 Hay-windrowed 478 Total 9134 We further select a subset scene of the Indian Pine dataset, consisting of the pixels [27 94] [31 116] for a size of dataset, denoted by Indian Pine Sub. In

7 Hyperspectral Image Classification by Using Pixel Spatial Correlation 147 Indian Pine Sub, there are 4 labeled classes in total. This dataset [4] aims to evaluate the hyperspectral image classification method when dealing with different classes with similar spectral signatures. 20 bands are removed due to the water absorption bands. The details about the Indian Pine Sub dataset are shown in Table 2. Table 2. Details of the Indian Pine Sub Dataset Class #ofpixels Class #ofpixels Soybeans-clean till 732 Corn-no till 1005 Soybeans-min 730 Grass/trees 1903 Total Compared Methods To evaluate the effectiveness of the proposed hyperspectral image classification approach, the following methods are employed for comparison. 1. Semi-Supervised Graph Based Method [4]. In semi-supervised graph based method, the hyperspectral image classification is formulated as a graph based semisupervised learning procedure. All pixels are denoted by the vertices in the graph structure, which is able to exploit the wealth of unlabeled samples by the graph learning procedure. For comparison, the Cross+Stacked kernel is chosen which shows the best results in [4]. This method is denoted by SSG+CS. 2. Representative Multiple Kernel Learning (RMKL) [8]. In RMKL, the multiple kernel-based learning method is employed for hyperspectral image classification. In this method, multiple kernels are selected and they are evaluated according to statistical significance and learned weights for better kernel combination, which is achieved by learning the linear combination of the basis kernels and minimizing the F-norm error. 3. Local Manifold Learning-Based k-nearest Neighbor (SML+KNN) [12]. SML+KNN combines the local manifold learning and the k-nearest neighbor classifier for hyperspectral image classification. In this method, all pixels are embedded in a manifold, and local manifold learning is conducted to estimate the relationship among pixels. Then, the weighted KNN classifier is employed for pixel classification. The Supervised Locally Linear Embedding (SLLE) method is used as the weighting methods due to its steady performance as introduced in [12]. 4. Hypergraph analysis with distance-based hyperedges for hyperspectral image classification (HGD) [19]. HGD is another hypergraph based hypespectral image classification method. In HGD, the hypergraph is constructed by using the neighborhood clustering method, where each pixel is connected to its several neighbor pixels in the feature space. 5. Hypergraph analysis with spatial hyperedges for hyperspectral image classification (HGS), i.e., the proposed method.

8 148 Y. Gao and T.-S. Chua (a) Overall Accuracy (b) Kappa Fig. 3. The classification results of compared methods in the Indian Pine dataset (a) Overall Accuracy (b) Kappa Fig. 4. The classification results of compared methods in the Indian Pine Sub dataset 3.3 Experimental Results In our experiments, the number of labeled training samples for each class varies from 10 to 100, i.e., {10, 20, 30, 50, 100}. To evaluate the hyperspectral image classification performance, the widely used overall accuracy (OA) and the Kappa statistic are employed [9] as the evaluation metrics. In the following experiments, K is set as 12, and λ=0.9. Experimental comparisons on the two testing datasets are shown in Fig. 3 and Fig. 4. In comparison with the state-of-the-art methods, the proposed method outperforms all compared methods in both of the two testing databases. Here we take the experimental results when 10 samples per class are selected as the training data as an example. In the Indian Pine dataset, the proposed method achieves a gain of 1.23%, 3.50%, 0.03%, and 34.38% in terms of the OA measure and a gain of 3.92%, 27.60%, 0.44%, and 30.52% in terms of the Kappa measure compared with SSG+CS, RMKL, SML-KNN and HGD. In the Indian Pine Sub dataset, the proposed method achieves a gain of 17.47%, 16.82%, 0.17%, and 16.48% in terms of the OA measure and a gain of 27.13%, 16.37%, 1.50%, and 14.01% in terms of the Kappa measure compared with SSG+CS, RMKL, SML- KNN and HGD. Experimental results show that the proposed method achieves the best image classification performance in most of cases in the two testing datasets, which indicates the effectiveness of the proposed method. Figure 5 and Figure 6 demonstrate the classification map of the proposed method in the testing datasets with different number of selected training samples per class.

9 Hyperspectral Image Classification by Using Pixel Spatial Correlation 149 (a) (b) (c) (d) (e) (f) Fig. 5. Classifications maps of the Indian Pine dataset. (a) Groundtruth map with 9 classes (b)-(f) Classifications maps with 10,20,30,50,and 100 labeled training samples for each class. (a) (b) (c) (d) (e) (f) Fig. 6. Classifications maps of the Indian Pine Sub dataset. (a) Groundtruth map with 9 classes (b)-(f) Classifications maps with 10,20,30,50,and 100 labeled training samples for each class. (a) OA in Indian Pine (b) Kappa in Indian Pine (c) OA in Indian Pine Sub (d) Kappa in Indian Pine Sub Fig. 7. Classification performance comparison with different K values by using 10 training sample per class in the Indian Pine dataset and the Indian Pine Sub dataset 3.4 On the Parameter K for Hyperedge Construction The parameter K determines the number of selected spatial neighbors for each pixel in the hyperedge construction procedure. A larger K value indicates that more pixels are connected by one hyperedge, and a smaller K value means that only a few pixels are linked in one hyperedge. To investigate the influence of different K selection on the hyperspectral image classification performance, we vary the parameter K as {4, 8, 12, 20, 24}. Figure 7 provides the OA and Kappa performance curves with respect to the variation of K in the two testing databases, where 10 samples per class are selected as the training data. Experimental results show that the results are stable with the variation of parameter K. WhenK is large (e.g., 20 or 24) or K is small (e.g., 4), the hyperspectral image classification performance is only a bit lower than that of K =8and K =12.These

10 150 Y. Gao and T.-S. Chua results indicate that the proposed method can achieve a steady performance with different settings of parameter K. 4 Conclusion In this paper, we propose a hyperspectral image classification method by using the spatial correlation of pixels. In the proposed method, the relationship among pixels in the hyperspectral image is formulated in a hypergraph structure. In the constructed hypergraph, each vertex denotes a pixel in the image, and the hyperedge is generated by using the spatial correlation among pixels. Semi-supervised learning on the hypergraph is conducted for hyperspectral image classification. This method employs the spatial information to explore the relationship among pixels, and the high dimensional feature is only used to further enhance the spatial-based correlation in the constructed hypergraph, which is able to avoid the curse of dimensionality. Experiments on the Indian Pine and the Indian Pine Sub datasets are performed, and comparisons with the state-of-the-art methods are provided to evaluate the effectiveness of the proposed method. Experimental results indicate that the proposed method can achieve better results in comparison with the state-of-the-art methods for hyperspectral image classification. Acknowledgements. This work was supported by NUS-Tsinghua Extreme Search (NExT) project under the grant number: R References 1. Bandos, T., Bruzzone, L., Camps-Valls, G.: Classification of hyperspectral images with regularized linear discriminant analysis. IEEE Transaction on Geoscience and Remote Sensing 47(3), (2009) 2. Berge, A., Solberg, A.: Structured gaussian components for hyperspectral image classification. IEEE Transaction on Geoscience and Remote Sensing 44(11), (2006) 3. Bilgin, G., Erturk, S., Yildirim, T.: Unsupervised classification of hyperspectral-image data using fuzzy approaches that spatially exploit membership relations. IEEE Geoscience and Remote Sensing Letters 5(4), (2008) 4. Camps-Valls, G., Marsheva, T.B., Zhou, D.: Semi-supervised graph-based hyperspectral image classification. IEEE Transaction on Geoscience and Remote Sensing 45(10), (2007) 5. Gao, Y., Wang, M., Luan, H., Shen, J., Yan, S., Tao, D.: Tag-based social image search with visual-text joint hypergraph learning. In: ACM Conference on Multimedia, pp (2011) 6. Gao, Y., Wang, M., Tao, D., Ji, R., Dai, Q.: 3D object retrieval and recognition with hypergraph analysis. IEEE Transactions on Image Processing 21(9), (2012) 7. Gao, Y., Wang, M., Zha, Z., Shen, J., Li, X., Wu, X.: Visual-textual joint relevance learning for tag-based social image search. IEEE Transactions on Image Processing (in press) 8. Gu, Y., Wang, C., You, D., Zhang, Y., Wang, S., Zhang, Y.: Representative multiple kernel learning for classification in hyperspectral imagery. IEEE Transaction on Geoscience and Remote Sensing 50(7), (2012)

11 Hyperspectral Image Classification by Using Pixel Spatial Correlation Guo, B., Damper, S.G.R., Nelson, J.: Band selection for hyperspectral image classification using mutual information. IEEE Geoscience and Remote Sensing Letters 3(4), (2006) 10. Kuo, B.-C., Li, C.-H., Yang, J.-M.: Kernel nonparametric weighted feature extraction for hyperspectral image classification. IEEE Transaction on Geoscience and Remote Sensing 47(4), (2009) 11. Landgrebe, D.: Hyperspectral image data analysis. IEEE Signal Process Magazine 19(1), (2002) 12. Ma, L., Crawford, M., Tian, J.: Local manifold learning-based -nearest-neighbor for hyperspectral image classification. IEEE Transaction on Geoscience and Remote Sensing 48(11), (2010) 13. Marconcini, M., Camps-Valls, G., Bruzzone, L.: A composite semisupervised svm for classification of hyperspectral images. IEEE Geoscience and Remote Sensing Letters 6(2), (2009) 14. Martinez-Uso, A., Pla, F., Sotoca, J.M., Garcia-Sevilla, P.: Clustering-based hyperspectral band selection using information measures. IEEE Transaction on Geoscience and Remote Sensing 45(12), (2007) 15. ul Haq, Q.S., Tao, L., Sun, F., Yang, S.: A fast and robust sparse approach for hyperspectral data classification using a few labeled samples. IEEE Transaction on Geoscience and Remote Sensing 50(6), (2012) 16. Velasco-Forero, S., Manian, V.: Improving hyperspectral image classification using spatial preprocessing. IEEE Geoscience and Remote Sensing Letters 6(2), (2009) 17. Wang, J., Chang, C.-I.: Independent component analysis-based dimensionality reduction with applications in hyperspectral image analysis. IEEE Transaction on Geoscience and Remote Sensing 44(6), (2006) 18. Wang, J., Zhang, Z., Zha, H.: Adaptive manifold learning. In: Proceedings of Advances in Neural Information Processing Systems (2004) 19. Wen, Y., Gao, Y., Liu, S., Cheng, Q., Ji, R.: Hyperspetral image classification with hypergraph modelling. In: Proceedings of International Conference on Internet Multimedia Computing and Service (2012) 20. Xia, S., Hancock, E.: Learning large scale class specific hyper graphs for object recognition. In: Proceedings of International Conference on Image and Graphics, pp (2008) 21. Yu, J., Tao, D., Wang, M.: Adaptive hypergraph learning and its application in image classification. IEEE Transactions on Image Processing 21(7), (2012) 22. Zass, R., Shashua, A.: Probabilistic graph and hypergraph matching. In: Proceedings of IEEE International Conference on Computer Vision and Pattern Recognition (2008) 23. Zhou, D., Huang, J., Schokopf, B.: Learning with hypergraphs: Clustering, classification, and embedding. In: NIPS (2007)