HYPERSPECTRAL images with hundreds of spectral

Transcription

1 IEEE TRANSACTIONS ON GEOSCIENCE AND REMOTE SENSING 1 Hyperspectral Image Classification With Imbalanced Data Based on Orthogonal Complement Subspace Projection Jiaojiao Li, Member, IEEE, QianDu, Fellow, IEEE, Yunsong Li, and Wei Li, Senior Member, IEEE Abstract Conventional classification algorithms have shown great success for balanced classes. In remote sensing applications, it is often the case that classes are imbalanced. This paper proposes a novel solution to solve the problem of imbalanced training samples in hyperspectral image classification. It consists of two parts: one is for large-size sample sets and the other is for small-size sets. Specifically, an algorithm based on the orthogonal complement subspace projection (OCSP) is proposed to select samples from large-size classes, and an algorithm also based on OCSP is proposed to create artificial samples for small-size ones. The impact on representation-based classifiers, i.e., sparse and collaborative representation classifiers and traditional classifiers (e.g., support vector machine), is investigated. Experimental results demonstrate that the proposed solution can outperform other existing solutions in the literature. Index Terms Collaborative representation, hyperspectral classification, imbalanced data, orthogonal subspace projection, sparse representation. I. INTRODUCTION HYPERSPECTRAL images with hundreds of spectral bands are employed in many applications, such as global environment monitoring, land cover detection, resource management, and medical diagnosis [1] [4]. Hyperspectral image classification is an important research topic that focuses on assigning class labels to pixels. Traditional classification algorithms, such as maximum likelihood classification [5], support vector machine (SVM) [6], artificial neural network [7], Manuscript received April 26, 2017; revised July 17, 2017, November 5, 2017, and January 28, 2018; accepted March 4, This work was supported in part by the National Nature Science Foundation of China under Grant , Grant , Grant , Grant , and Grant , in part by the 111 Project under Grant B08038, in part by the Fundamental Research Funds for the Central Universities under Grant JB170109, in part by the Natural Science Basic Research Plan in Shaanxi Province of China under Grant 2016JQ6023, and in part by the General Financial Grant from the China Postdoctoral Science Foundation under Grant 2017M (Corresponding author: Jiaojiao Li.) J. Li and Y. Li are with the State Key Laboratory of Integrated Service Networks, School of Telecommunications Engineering, Xidian University, Xi an , China ( jjli@xidian.edu.cn; ysli@mail.xidian.edu.cn). Q. Du is with the Department of Electronic and Computer Engineering, Mississippi State University, Starkville, MS USA ( du@ece.msstate.edu). W. Li is with the College of Information Science and Technology, Beijing University of Chemical Technology, Beijing , China ( liwei@ieee.org). Digital Object Identifier /TGRS and naive Bayes classifier [8], have achieved satisfactory accuracy when training samples of different classes have equal size. Many classification techniques can offer excellent performance with balanced distribution of training sets [9]. Recently, sparse representation-based classification (SRC), which does not need any prior knowledge about the density distribution of the data, has gained great interest. Sparse representation technology has been exploited in signal and texture classification since 2006 [10], [11]. SRC [12] [14] was developed for face recognition where classes are balanced with the same number of training samples per class. The essence of an SRC classifier is built on the concept that a pixel can be represented as a linear sparse combination of all the available labeled data, which can be achieved by imposing the l 0 -norm and well approximated by the l 1 -norm minimization. It does not include a training testing fashion. Instead, a testing pixel to be classified is sparsely represented by using all the training data, and its label is assigned according to the class whose labeled samples provide the smallest approximation error. In [15] and [16], SRC was applied to hyperspectral image classification. Similarly, l 2 -based collaborative representationbased classifier (CRC) was proposed, which may provide even better performance than SRC [17]. A diagonal weight matrix T can be added to adaptively adjust the regularization parameter, according to the similarity between the labeled samples and a testing pixel [18]. Detailed discussion can be found in [19]. The performance of SRC and CRC is also sensitive to the number of training samples per class used in the representation, because, in general, using more samples in the representation can yield smaller representation error. An imbalanced data problem is fundamental in remote sensing, because objects with various sizes present in an image scene and practical sample labeling is difficult. As inherent characteristic of remote sensing data, it is challenging to solve such a problem [20]. It is often the case that the numbers of training samples in each class are unequal, even with tremendous difference. By convention, in a sample-size-related imbalanced data set, the classes with small size are named minority classes, and the ones with large size are named majority classes [9]. The common situation in performance assessment is that the correct classification of large-size classes contributes more than that of small-size classes. Conventional IEEE. Personal use is permitted, but republication/redistribution requires IEEE permission. See for more information.

2 2 IEEE TRANSACTIONS ON GEOSCIENCE AND REMOTE SENSING hyperspectral classification algorithms usually concentrate on improving overall accuracy (OA) but ignoring class-specific accuracy [21] [24]. In reality, the correct classification of small-size classes often plays an important role, because they are usually foreground classes of interest. Nevertheless, the size of minority class is vastly outnumbered by that of majority class. Recent research has highlighted the uneven distribution data problem with great attention devoted to small-size sample set or category [24]. Under this situation, the lack of representative data of minority classes makes classification algorithms difficult to achieve satisfying performance. Different directions emerge for dealing with this imbalanced problem, such as preprocessing of imbalanced data [25] and modification of classifiers [26]. Typically, sampling methods can alter imbalanced data sets into balance distributions. Studies have exhibited that a balanced distribution can achieve better classification performance than an imbalanced distribution [27] [29]. The balance solution commonly is comprised of large-size solution and small-size solution. For large-size sets, methods, such as NearMiss1, NearMiss2, and NearMiss3, are proposed in [30], which are K-nearest neighbors undersampling algorithms for selecting some representative samples from majority sample sets. Due to the fact that features extracted from existing training samples are limited, repetitive or redundant training samples have great occurrence possibility in large-size sets. For small-size sets, algorithms, such as synthetic minority over-sampling technique (SMOTE) [31], are developed. For each minority sample, the SMOTE randomly selects one of the K-nearest neighbors, then multiplies the corresponding difference vector with a random value chosen from 0 to 1, and finally adds this vector to the sample to generate an artificial one. Let y 1 and y 2 be two training samples of the same class. Then, y s = y 1 + (y 2 y 1 ) δ is an artificial sample created via SMOTE, where δ [0, 1] is a random number. Obviously, the artificial samples are linear combinations of actual ones that may not convey new features in the existing samples. In this paper, we propose a novel solution to solve the sample-size-related imbalanced data problem more effectively. The new solution consists of two parts: one for large-size and the other for small-size. For large-size one, we use orthogonal complement subspace projection (OCSP) to select distinctive samples. For small-size one, a novel algorithm based on OCSP is developed to create nonlinear artificial samples that are not simply linear combinations of available samples, where new features embedded can be extracted. Here, we will focus on the problem of imbalanced samples in SRC and CRC. It will be demonstrated that using the OCSP-balanced sample sets, the performance of SRC and CRC can be improved. In addition, with the balanced solutions, the performance of SVM with radial bias function kernel and extreme learning machine (ELM) is also investigated. This paper is organized as follows. Section II briefly introduces SRC, CRC, and their modified versions with a diagonal weight matrix. Section III proposes our new solution for addressing the imbalanced data problem. Experimental results are presented and analyzed in Section IV. The conclusions are drawn in Section V. II. REPRESENTATION-BASED CLASSIFICATION A. Sparse Representation-Based Classification Let X R m n denote a data set of the training samples (m is the number of spectral bands and n is the number of training samples). The training samples in the ith class can be denoted as X i R m n i, where n = C i=1 n i (C represents the number of classes and n i denotes the number of samples in the ith class). Therefore, X =[X 1, X 2,...,X C ]. In SRC, an approximation of a testing pixel y R m 1 can be represented via a sparse linear combination of an available dictionary that consists of the training samples in X. Itisto find a vector α R n 1, which is a sparse coefficient vector. Thus, the objective function is α = arg min α y Xα λ α 1 (1) where λ is the regularization parameter. After α is calculated, we separate X into C class-specific subdictionaries and α into C class-specific subvectors, i.e., α =[α 1, α 2,...,α C ] and α i R n i 1. In SRC, the class label of the testing sample is determined depending on the class that minimizes the residual The label of y is assigned via [15] r i (y) = X i α i y 2. (2) label(y) = arg min {r l}. (3) l=1,...,c B. Collaborative Representation-Based Classification (CRC) SRC ignores the collaboration between samples. Considering the collaborative nature during classification, the CRC exploits l 2 -penalty in the style of Tikhonov regularization. Then, the objective function becomes α = arg min α y Xα λ α 2 2. (4) The value α can be obtained via [17] α = (X T X + λi) 1 X T y. (5) We can see that the l 2 -regularization term can overcome the ill-conditioning and ill-posedness in the inverse problem. Similarly, we partition X and α, and determine the class label of y with (2) and (3). C. SRC With a Diagonal Weight Matrix T In SRC with a diagonal weight matrix T (SRC-T), a distance-weighted diagonal matrix T l,y is imposed on the l 1 -norm penalty α l = arg min α y X lα λ T l,yα 1 (6) where T l,y is defined as T l,y = y x l, (7) 0 y x l,nl 2 computing the Euclidean distance between the testing pixel y to each of the samples in X l,wherex l,1 denotes the first sample in the lth class, x l,nl is the n l th samples in the lth class,

3 LI et al.: HYPERSPECTRAL IMAGE CLASSIFICATION WITH IMBALANCED DATA BASED ON OCSP 3 and n l is the number of samples in the lth class. Then, a strong penalty is imposed on samples dissimilar to y, preventing their participation in the representation of y. D. CRC With a Diagonal Weight Matrix T Similarly, in CRC with a diagonal weight matrix T (CRC-T), we calculate the per-class coefficients α l {1,...,C} through α l = arg min α y X lα λ T l,yα 2 2. (8) Each reconstructed y can be achieved via a closed form as [18] ỹ l = X l ( X T l X l + λt T l,y T l,y) 1X T l y. (9) By adding a Tikhonov regularization matrix T to discourage dissimilar samples in representation, CRC-T can significantly outperform CRC. Both CRC and CRC-T have closed-form solutions, so they are the most efficient classifiers [19]. As mentioned in Section I, the methods based on the SRC and the CRC are sensitive to the number of training samples used for representation in per class, due to the fact that more samples utilized in the representation can yield smaller representation error. CRC is more sensitive because all the samples participate in the representation, while SRC is better because it uses the sparsity constraint to select the smallest number of similar samples for representation. Fig. 1. Sample generated without considering gradient constraint. III. PROPOSED METHOD The proposed novel solution for the problem of imbalanced training samples contains two parts: sample generation for small-size classes and sample selection for large-size classes. A. OCSP for Small-Size Classes For small-size classes, previous methods often generate synthetic samples randomly, or linear combinations of existing samples, creating a bulk of irrelevant or redundant data instances. On the contrary, the proposed OCSP method is to produce relevant but irredundant samples. The algorithm includes two major steps: 1) generating synthetic samples with gradient constraint and 2) using OCSP to select the most reliable samples. The detailed process is described in the following. 1) Gradient-Constrained Random Sample Generation: Let S [s 1, s 2,...,s p ] be the sample set of a small-size class and p is the number of samples in S. We randomly generate artificial samples set R. In each band, the samples in S have the two extreme values: maximum s max and minimum s min. The values of the kth band in R are randomly selected within [smin k, sk max ], k {1,...,m}, wherem denotes the number of bands. However, only considering the rule that selecting values within the range of each band, the synthetic samples may deviate from the real samples. For example, in Fig. 1, the black solid line illustrates a real sample vector, and the black dotted line means a synthetic sample using the band range as the constraint. Obviously, the synthesized data may be quite different from the actual samples. Fig. 2. Artificial sample created using gradient constraint. To solve such a problem, the gradient constraint is imposed. Let s [s 1, s 2, s m ] be the mean sample vector of S. The gradient vector of s can be calculated as s [ s 1, s 2,, s k, s m 1 ] (10) where s k = s k+1 s k. Then, we can obtain the first derivative of the spectral signature, and the indicative representation can be formulated as: d [d 1, d 2,, d k, d m 1 ],whered k is 1 if s k > 0and 1otherwise. The indicative representation is utilized to ensure that the random instance has the same variation tendency as the true sample. Fig. 2 shows a synthetic sample in R after imposing the gradient constraint. The black solid line is the random data instance generated under the gradient constraint, which implies the same variation tendency. 2) Sample Selection via OCSP: The orthogonal complementary subspace PS is constructed as P S = I SS#,whereI is an identity matrix and S # = (S T S) 1 S T is the pseudoinverse of S. The samples generated in Section III-A1 may still deviate from the actual samples. To select the reliable samples from R, the projection to the orthogonal subspace of S is computed, i.e., PS R 2 2. Those samples yielding small projections will be selected. This means that finally generated samples do not deviate much from the original samples to further ensure the information that they carry is true to the class. Since the generated samples by OCSP are not simple linear combination

4 4 IEEE TRANSACTIONS ON GEOSCIENCE AND REMOTE SENSING of the original training samples, they can also provide new discriminative features. The detailed process is depicted in Algorithm 1. TABLE I NOTATIONS AND MEANINGS USED IN THE PROPOSED METHODS Algorithm 1 Pseudocode for Generating and Selecting Synthetic Samples First step: Initialization: Numge = 0; 1. Compute the sample mean s of sample set S and s [s 1, s 2, s m ]. 2. Calculate the gradient vector s of the sample mean s, s [ s 1, s 2,..., s k, s m 1 ], and s k = s k+1 s k. 3. The indicative representation is computed as: d [d 1, d 2,, d k, d m 1 ]. d k = 1 if s k > 0 and d k = 1otherwise. 4. Set k = Generate the random value g k from [smin k, sk max ] 6. Repeat: 7. Generate the random value g k+1 from [smin k+1 max ]. 8. Calculate the difference between g = g k+1 g k d k = 1if g k > 0, d k = 1otherwise. 9. Compare d k with d k. If equal, keep g k,setk k +1; else, discard g k.gotostep Until k = m (m denotes the number of bands) 11. A new sample is formulated as g = [g 1,...,g k,...,g m ], and added to R. Numge Numge + 1. Goto Step 4. Until Numge = f Ng is satisfied. (Ng denotes the number of samples to be created, and { f f > 1, f N+}) Second step: Compute: PS = I SS#,whereS # = ( S T S ) 1 S T. Compute: the projections P S R 2 2. Select the top Ng minimum projections and the corresponding new samples are the finally selected samples. B. OCSP for Large-Size Classes For a large-size training set H [h 1, h 2,...,h q ], q is the number of samples in H, we use OCSP to select distinctive training samples to represent within-class variations, when the number of training samples in a class needs to be reduced. Starting with an initial sample h f H and f {1,...,q}, the selected samples set is T = [h f ], and the rest of samples set can be represented as T =[h r H] {r = f } {1,...q}. Thus, an orthogonal subspace is constructed through PT = I T(TT T) 1 T T. Here, the OCSP is to find a next sample from T that yields the largest projection PT T 2 2,which contains the most distinctive feature among other within-class samples. Then, update T and T until the number of samples is the required. For the purposes of clarification, the notations in the proposed approaches are summarized in Table I. IV. EXPERIMENTS A series of experiments is designed to evaluate the performance of the proposed approaches for overcoming the effects of the imbalanced problem. The evaluation metrics are average accuracy (AA), OA, and kappa. Especially, AA is highlighted for the assessment of the proposed solution. Existing solutions, such as the SMOTE algorithm for small-size class and the NearMiss1, NearMiss2, and NearMiss3 algorithms for large-size class, are employed for comparison purposes. In this section, the proposed solution by exploiting OCSP for small-size class and OCSP for large-size class is denoted as OCSP + OCSP. Similarly, solutions using SMOTE for smallsize class and NearMiss-series methods for large-size class are named SMOTE + NearMiss1, SMOTE + NearMiss2, and SMOTE + NearMiss3, respectively. The SRC, SRC-T, CRC, and CRC-T are employed in these experiments. In addition, the SVM and ELM algorithms are also used to evaluate the proposed solution. The nonparametric McNemar s test based on the standardized normal test statistic is utilized to evaluate the statistical significance in the improvement of AA with different balance solutions. The McNemar s test statistic for two different solutions can be calculated as [32] z = ( f 12 f 21 ) / f 12 + f 21 (11) where f 12 denotes the number of samples misclassified via solution 2 but not solution 1 and f 21 means the number of samples misclassified through solution 1 but not solution 2. z is the absolute value of z. For 5% level of significance,

5 LI et al.: HYPERSPECTRAL IMAGE CLASSIFICATION WITH IMBALANCED DATA BASED ON OCSP 5 TABLE II LABELED SAMPLES IN THE INDIAN PINES DATA SET TABLE III SMALL-SIZE CLASSES WITH DIFFERENT BALANCE NUMBERS IN THE INDIAN PINES DATA SET Fig. 3. Synthetic samples generated via SMOTE and OCSP. Fig. 4. AA using different solutions under different balanced numbers. (a) SRC, (b) SRC-T, (c) CRC, and (d) CRC-T in the Indian Pines experiment. the z value is If a z value is greater than this quantity, the two balance solutions have significant discrepancy. If 1% level of significance is considered, then the z value is A. Indian Pines Experiment The scene of Indian Pines was collected by an Airborne Visible Infrared Imaging Spectrometer sensor in north-western Indiana. This image of size pixels includes

6 6 IEEE TRANSACTIONS ON GEOSCIENCE AND REMOTE SENSING TABLE IV INDIAN PINES CLASSIFICATION ACCURACY OF DIFFERENT SOLUTIONS WITH SRC TABLE V INDIAN PINES CLASSIFICATION ACCURACY OF DIFFERENT SOLUTIONS WITH SRC-T 220 spectral bands from 0.4- to 2.5-μm spectral ranges. After removing 20 water absorption bands, 200 bands are used in the experiment. The spatial resolution is 20 m/pixel. This image scene includes two-third agriculture and one-third forest; 16 classes are contained in this hyperspectral image, and their numbers of labeled samples are tabulated in Table II. We use 10% labeled data as the source training data set. Note that this is an imbalanced classification problem, as some

7 LI et al.: HYPERSPECTRAL IMAGE CLASSIFICATION WITH IMBALANCED DATA BASED ON OCSP 7 TABLE VI INDIAN PINES CLASSIFICATION ACCURACY OF DIFFERENT SOLUTIONS WITH CRC TABLE VII INDIAN PINES CLASSIFICATION ACCURACY OF DIFFERENT SOLUTIONS WITH CRC-T classes have hundreds of training samples but others have less than 10 samples. The bolded data are about minority classes. In particular, class oats has only two training samples. In this data set, the number of training data is 1043, and the average number of each class is calculated as 1043/16 =65. Thus, we balance the size of training samples in each class to 10, 20, 30, 40, or 50. With different balance numbers, the claimed small-size classes are also changed. The detailed labels of minority classes are tabulated in Table III. To analyze the OCSP solution for minority classes, we plot the synthetic samples generated via OCSP and SMOTE in Fig. 3. It can be seen that the OCSP-generated curve is

8 8 IEEE TRANSACTIONS ON GEOSCIENCE AND REMOTE SENSING TABLE VIII z VALUE [SIGNIFICANCE (1%)] IN MCNEMAR S TEST OF AA BETWEEN OCSP AND SMOTE IN THE INDIAN PINES EXPERIMENT TABLE IX z VALUES [SIGNIFICANCE (1%)] IN MCNEMAR S TEST OF AA BETWEEN OCSP+ OCSP AND OTHER SOLUTIONS IN THE INDIAN PINES EXPERIMENT TABLE X INDIAN PINES CLASSIFICATION ACCURACY OF DIFFERENT SOLUTIONS WITH SVM AND ELM more similar to the original ones, due to the fact that SMOTE is a simple linear solution to create new samples. Fig. 4 shows the AA values using different solutions when the number of training samples in each class is made to be 10, 20, 30, 40, and 50 (through either shrinking or expansion). Meanwhile, the vertical bars denote the confidence intervals on each balance number and imply the change range of AA. Fig. 4(a) represents the AA values achieved by SRC, while Fig. 4(b) (d) exhibits the AA values from SRC-T, CRC, and CRC-T, respectively. Obviously, the results from the OCSP + OCSP algorithm are better than other solutions in any scenarios. It is more suitable than SMOTE + NearMiss1, SMOTE + NearMiss2, and SMOTE + NearMiss3 to balance the training data for the Indian Pines data set. Especially, the results achieved via CRC-T with OCSP + OCSP have great superiority. Note that when the number of training samples per class becomes large, the confidence interval of the classification accuracy tends to be smaller, which implies that the accuracy becomes more stable. Tables IV VII show the accuracies based on the SRC, SRC-T, CRC, and CRC-T when using the balanced training set (i.e., 50 samples per class). We also tabulate the class accuracies without balance. The performances achieved by SRC-T and CRC-T are better than SRC and CRC due to the regularization matrix penalizing dissimilar samples in representation. According to Table IV, the small-size classes

9 LI et al.: HYPERSPECTRAL IMAGE CLASSIFICATION WITH IMBALANCED DATA BASED ON OCSP 9 TABLE XI LABELED SAMPLES IN THE UNIVERSITY OF PAVIA TABLE XII SMALL-SIZE CLASSES WITH DIFFERENT BALANCE NUMBERS IN THE PAVIA DATA SET Fig. 5. AA using different solutions under different balanced numbers. (a) SRC, (b) SRC-T, (c) CRC, and (d) CRC-T in the Pavia experiment. in this scenario are 1, 4, 7, 8, 9, 13, 15, and 16. For OCSP and SMOTE, the sizes of small-size sets are enlarged to 50, and the sizes of large-size sets are unchanged. All the values in boldface are the best performance achieved via balancing solutions that are better than or equal to the original performance without data balance. Compared with the results of the performance without balance, the AA is greatly improved; the OA and kappa values are very slightly improved due to the fact that most of the testing samples are from large-size classes that dominate the OA and kappa assessment. It can be seen that for minority classes, OCSP outperforms SMOTE. For OCSP + OCSP, SMOTE + NearMiss1, SMOTE + NearMiss2, and SMOTE + NearMiss3, large-size sets are reduced to have 50 training samples each (with OCSP, NearMiss1, NearMiss2, or NearMiss3) in addition to enlarge small-size classes (with OCSP or SMOTE). The total number of training samples after balance is = 800, less than 1043 without balance. The accuracy for large-size classes may be degraded, since only 50 training samples are used. However, comparing the proposed OCSP + OCSP with SMOTE + NearMiss1, SMOTE + NearMiss2, and SMOTE + NearMiss3, OCSP+OCSP provides higher classification accuracy in terms of AA, OA, and kappa under any circumstances, which means that for large-size classes, the OCSP offers better performance than NearMiss-type methods in distinctive sample selection. Tables IV VII also indicate that the performance improvement from balancing is more obvious in CRC and CRC-T, because collaborative representation is more sensitive to training sample size. After balancing, CRC-T offers the highest AA, and CRC-T with OCSP + OCSP using only 800 samples can offer similar accuracy as OCSP. To evaluate the statistical significance in performance improvement, Table VIII tabulates the average z values when the OCSP is compared against the SMOTE when enlarging small-size classes, and Table IX shows the average z values when OCSP + OCSP is compared against other methods, i.e., SMOTE + NearMiss1, SMOTE + NearMiss2, and SMOTE + NearMiss3, when balancing both small-size and

10 10 IEEE TRANSACTIONS ON GEOSCIENCE AND REMOTE SENSING TABLE XIII PAVIA CLASSIFICATION ACCURACY OF DIFFERENT SOLUTIONS WITH SRC TABLE XIV PAVIA CLASSIFICATION ACCURACY OF DIFFERENT SOLUTIONS WITH SRC-T large-size classes. The yes, here, indicates that the two methods in McNemar s test have significant performance discrepancy. Obviously, the proposed OCSP and OSCP + OCSP are statistically different from their counterparts with 1% (and 5%) significance level. Our proposed solutions are more reasonable and efficient, because: 1) no repeated or redundant samples are generated from samples in small-size classes and 2) distinctive samples are selected from samples in large-size classes. We also evaluate the classification performance of the proposed solution based on SVM and ELM algorithms. Table X shows the AA values obtained by SVM and ELM under different balancing situations. The values in boldface indicate the best performances achieved if they are larger than the one without balance. The proposed solution OCSP+OCSP obtains higher accuracy than other three existing solutions. With regard to the case that only enlarges the numbers of training samples in the minority classes, the OCSP is better than SMOTE. The results in Table X also imply that ELM offers nearly the same classification AA as SVM after balancing procedure. Note that, after balancing, the CRC-T in Table VII can outperform SVM and ELM. B. University of Pavia Experiment The second data set is about the University of Pavia, an urban scene acquired by the reflective optics system imaging spectrometer sensor, which generates 115 spectral bands over μm. The University of Pavia image has pixels, each having 103 bands after bad-band removal. The geometric resolution is 1.3 m. Nine groundtruth classes with the number of labeled samples are tabulated in Table XI. As shown in Table XI, 5% of labeled samples are utilized in this experiment. The imbalanced data problem also emerged in this data set. For instance, class Bitumen only has 1330 labeled samples. On the contrary, class Meadows

11 LI et al.: HYPERSPECTRAL IMAGE CLASSIFICATION WITH IMBALANCED DATA BASED ON OCSP 11 TABLE XV PAVIA CLASSIFICATION ACCURACY OF DIFFERENT SOLUTIONS WITH CRC TABLE XVI PAVIA CLASSIFICATION ACCURACY OF DIFFERENT SOLUTIONS WITH CRC-T TABLE XVII z VALUE [SIGNIFICANCE (1%)] IN MCNEMAR S TEST OF AA BETWEEN OCSP AND SMOTE IN THE PAVIA EXPERIMENT has samples, which is 14 times more than the size of class Shadows. In this data set, 5% of labeled data (i.e., 2140 samples) are selected to be the training samples, which contain nine classes. Thus, the average number of each class is 2140/9 =237. We select different balance numbers, i.e., 50, 100, 150, and 200. For different balance numbers, the corresponding small-size classes are also different in this data set, as listed in Table XII. In Fig. 5, the sizes of largesize classes are decreased to 50, 100, 150, or 200; meanwhile, the sizes of small-size sets are enlarged to the same size accordingly. Fig. 5 shows the AA values from the solutions, including OCSP + OCSP, SMOTE + NearMiss1, SMOTE + NearMiss2, and SMOTE + NearMiss3 algorithms. OCSP + OCSP solution can keep a relatively better performance than other algorithms, and its performance with a smaller number of total training samples is better than the original without balance in Tables XIII XVI. It is worth mentioning that the performance of CRC in Fig. 5(c) is degraded when the balance

12 12 IEEE TRANSACTIONS ON GEOSCIENCE AND REMOTE SENSING TABLE XVIII z VALUE [SIGNIFICANCE (1%)] IN MCNEMAR S TEST OF AA BETWEEN OCSP+ OCSP AND OTHER SOLUTIONS IN THE PAVIA EXPERIMENT TABLE XIX PAVIA CLASSIFICATION ACCURACY OF DIFFERENT SOLUTIONS WITH SVM AND ELM number becomes large. Greatly expanding small-size classes may increase the between-class similarity and obscure the decision boundary, which could be a problem to a weak classifier like CRC that lacks a strategy in controlling sample quality for representation. In order to further evaluate the proposed algorithms, Tables XIII XVI show the classification results from SRC, SRC-T, CRC, and CRC-T, respectively, when exploiting a balanced training set (200 samples per class). All the values in boldface indicate that the best performance obtained via different solutions is better or equal to the original performance without data balance. In order to assess OCSP and SMOTE methods, the sizes of minority classes are expanded to 200, and the sizes of majority classes are unchanged. Compared with the performance without data balance, the AA values are enhanced greatly, and the OA and kappa values are nearly the same. Obviously, for small-size classes, OCSP is more suitable than the SMOTE. It can be seen that the performance achieved via OCSP+OCSP with 1800 training samples is better than the original training samples (2140) without balance. AA achieved from OCSP +OCSP is still the best. OA and kappa values are also better than NearMiss-series solutions except CRC. SRC-T produces the best performance before balancing, and CRC-T outperforms after balancing. Therefore, with the closed-form solution and the balanced training data, CRC-T is the most appealing classifier. We also calculate the average z values when OCSP is compared with the SMOTE, as tabulated in Table XVII. The average z values when OCSP + OCSP is compared with other methods are shown in Table XVIII. From Tables XVII and XVIII, the AA improvements using OCSP and OSCP + OCSP are mostly significant with 1% significance level. To further demonstrate the superiority of the proposed solution, we also test SVM and ELM. According to the AA values in Table XIX, without balancing procedure, the SVM and the ELM can obtain the AA and with 2140 training samples, respectively. When the balance number is up to 100, the performance of all the solutions is improved. The proposed solution still outperforms, and its performance with = 900 training samples is 2% higher than that with 2140 training samples. Note that the performance of CRC-T with OCSP+OCSP in Table XVI is similar to SVM and much better than ELM. V. CONCLUSION The imbalanced data problem due to very few training samples in minority classes and many training samples for majority classes is investigated in this paper. We propose a novel approach for solving the sample-size-related imbalanced data problem in hyperspectral image classification. The concept of OCSP is proposed to use in this approach. Specifically, our approach expands a small-size training set through nonlinear

13 LI et al.: HYPERSPECTRAL IMAGE CLASSIFICATION WITH IMBALANCED DATA BASED ON OCSP 13 sample generation and also reduces a large-size training set as needed by selecting the most distinctive samples. Experimental results using several popular classifiers demonstrate that the novel solution can outperform other traditional solutions, more effectively improving classification accuracy. For algorithms senstive to training size, CRC-T with low computational cost is the remonmended classifier after sample balancing procedure. REFERENCES [1] G. Begelman, M. Zibulevsky, E. Rivlin, and T. Kolatt, Blind decomposition of transmission light microscopic hyperspectral cube using sparse representation, IEEE Trans. Med. Imag., vol. 28, no. 8, pp , Aug [2] K. Liu, H. Su, and X. Li, Estimating high-resolution urban surface temperature using a hyperspectral thermal mixing (HTM) approach, IEEE J. Sel. Topics Appl. Earth Observ. Remote Sens., vol. 9, no. 2, pp , Feb [3] J. A. Jablonski, T. J. Bihl, and K. W. Bauer, Principal component reconstruction error for hyperspectral anomaly detection, IEEE Geosci. Remote Sens. Lett., vol. 12, no. 8, pp , Aug [4] R. Pike, G. Lu, D. Wang, Z. G. Chen, and B. Fei, A minimum spanning forest-based method for noninvasive cancer detection with hyperspectral imaging, IEEE Trans. Biomed. Eng., vol. 63, no. 3, pp , Mar [5] J. A. Richards and X. Jia, Using suitable neighbors to augment the training set in hyperspectral maximum likelihood classification, IEEE Geosci. Remote Sens. Lett., vol. 5, no. 4, pp , Oct [6] X. Guo, X. Huang, L. Zhang, L. Zhang, A. Plaza, and J. A. Benediktsson, Support tensor machines for classification of hyperspectral remote sensing imagery, IEEE Trans. Geosci. Remote Sens., vol. 54, no. 6, pp , Jun [7] S. K. Meher, Knowledge-encoded granular neural networks for hyperspectral remote sensing image classification, IEEE J. Sel. Topics Appl. Earth Observ. Remote Sens., vol. 8, no. 6, pp , Jun [8] B. Yang, Y. Lei, and B. Yan, Distributed multi-human location algorithm using naive Bayes classifier for a binary pyroelectric infrared sensor tracking system, IEEE Sensors J., vol. 16, no. 1, pp , Jan [9] H. H. Maheta and V. K. Dabhi, Classification of imbalanced data sets using multi objective genetic programming, in Proc. Int. Comput. Commun. Informat. (ICCCI), Coimbatore, India, 2015, pp [10] K. Huang and S. Aviyente, Sparse representation for signal classification, in Proc. Adv. Neural Inf. Process. Syst., vol. 19, pp , Jan [11] K. Skretting and J. Husøy, Texture classification using sparse framebased representations, EURASIP J. Adv. Signal Process., vol. 2006, no. 1, pp. 1 11, Jan [12] A. Y. Yang, J. Wright, Y. Ma, and S. S. Sastry, Feature selection in face recognition: A sparse representation perspective, Dept. Elect. Eng. Comput. Sci., Univ. California, Berkeley, CA, USA, Tech. Rep. UCB/EECS , [13] J. Wright, A. Y. Yang, A. Ganesh, S. S. Sastry, and Y. Ma, Robust face recognition via sparse representation, IEEE Trans. Pattern Anal. Mach. Intell., vol. 31, no. 2, pp , Feb [14] L. Zhang et al., Kernel sparse representation-based classifier, IEEE Trans. Signal Process., vol. 60, no. 4, pp , Apr [15] Y. Chen, N. M. Nasrabadi, and T. D. Tran, Hyperspectral image classification using dictionary-based sparse representation, IEEE Trans. Geosci. Remote Sens., vol. 49, no. 10, pp , Oct [16] Y. Chen, N. M. Nasrabadi, and T. D. Tran, Hyperspectral image classification via kernel sparse representation, IEEE Trans. Geosci. Remote Sens., vol. 51, no. 1, pp , Jan [17] L. Zhang, M. Yang, and X. Feng, Sparse representation or collaborative representation: Which helps face recognition? in Proc. Int. Conf. Comput. Vis., Barcelona, Spain, 2011, pp [18] W. Li, E. W. Tramel, S. Prasad, and J. E. Fowler, Nearest regularized subspace for hyperspectral classification, IEEE Trans. Geosci. Remote Sens., vol. 52, no. 1, pp , Jan [19] W. Li and Q. Du, A survey on representation-based classification and detection in hyperspectral remote sensing imagery, Pattern Recognit. Lett., vol. 83, pp , Nov [20] H. He and E. A. Garcia, Learning from imbalanced data, IEEE Trans. Knowl. Data Eng., vol. 21, no. 9, pp , Sep [21] T. Sun, L. Jiao, J. Feng, F. Liu, and X. Zhang, Imbalanced hyperspectral image classification based on maximum margin, IEEE Geosci. Remote Sens. Lett., vol. 12, no. 3, pp , Mar [22] N. V. Chawla, N. Japkowicz, and A. Kotcz, Editorial: Special issue on learning from imbalanced data sets, ACM SIGKDD Explorations Newsl., vol. 6, no. 1, pp. 1 6, Jun [23] M. Lin, K. Tang, and X. Yao, Dynamic sampling approach to training neural networks for multiclass imbalance classification, IEEE Trans. Neural Netw. Learn. Syst., vol. 24, no. 4, pp , Apr [24] N. Japkowicz and S. Stephen, The class imbalance problem: A systematic study, Intell. Data Anal., vol. 6, no. 5, pp , Oct [25] L. Abdi and S. Hashemi, To combat multi-class imbalanced problems by means of over-sampling techniques, IEEE Trans. Knowl. Data Eng., vol. 28, no. 1, pp , Jan [26] K.-B. Lin, W. Weng, R. K. Lai, and P. Lu, Imbalance data classification algorithm based on SVM and clustering function, in Proc. 9th Int. Conf. Comput. Sci. Edu. (ICCSE), Vancouver, BC, Canada, 2014, pp [27] G. M. Weiss and F. Provost, The effect of class distribution on classifier learning: An empirical study, Dept. Comput. Sci., Rutgers Univ., New Brunswick, NJ, USA, Tech. Rep. ML-TR-44, [28] J. Laurikkala, Improving identification of difficult small classes by balancing class distribution, in Proc. Conf. AI Med. Eur., Artif. Intell. Med., 2001, pp [29] A. Estabrooks, T. H. Jo, and N. Japkowicz, A multiple resampling method for learning from imbalanced data sets, Comput. Intell., vol. 20, no. 1, pp , [30] J. Zhang and I. Mani, KNN approach to unbalanced data distributions: A case study involving information extraction, in Proc. ICML, Washington, DC, USA, 2003, pp [31] J. Mathew, M. Luo, C. K. Pang, and H. L. Chan, Kernel-based SMOTE for SVM classification of imbalanced datasets, in Proc. IEEE IECON 41st Annu. Conf. Ind. Electron. Soc., Yokohama, Japan, Nov. 2015, pp [32] G. M. Foody, Thematic map comparison: Evaluating the statistical significance of differences in classification accuracy, Photogramm. Eng. Remote Sens., vol. 70, no. 5, pp , May Jiaojiao Li (S 16 M 18) received the B.E. degree in computer science and technology, the M.S. degree in software engineering, and the Ph.D. degree in communication and information systems from Xidian University, Xi an, China, in 2009, 2012, and 2016, respectively. She was an Exchange Ph.D. Student with Mississippi State University, Starkville, MS, USA. She is currently a Post-Doctoral Researcher and a Teacher with Xidian University. Her research interests include hyperspectral remote sensing image analysis and processing, pattern recognition, and data compression. Qian Du (S 98 M 00 SM 05 F 18) received the Ph.D. degree in electrical engineering from the University of Maryland Baltimore County, Baltimore, MD, USA, in She is currently a Bobby Shackouls Professor with the Department of Electrical and Computer Engineering, Mississippi State University, Starkville, MS, USA. Her research interests include hyperspectral remote sensing image analysis and applications, pattern classification, data compression, and neural networks. Dr. Du is a fellow of the SPIE International Society for Optics and Photonics. She was a recipient of the 2010 Best Reviewer Award from the IEEE Geoscience and Remote Sensing Society (GRSS). She was a Co-Chair of the Data Fusion Technical Committee of the IEEE GRSS from 2009 to She was the Chair of the Remote Sensing and Mapping Technical Committee of the International Association for Pattern Recognition from 2010 to She was the General Chair of the fourth IEEE GRSS Workshop on Hyperspectral Image and Signal Processing: Evolution in Remote Sensing held at Shanghai, China, in She served as an Associate Editor for the IEEE JOURNAL OF SELECTED TOPICS IN APPLIED EARTH OBSERVATIONS AND REMOTE SENSING (JSTARS), the Journal of Applied Remote Sensing, and the IEEE SIGNAL PROCESSING LETTERS. Since 2016, she has been the Editor-in-Chief of the IEEE JSTARS.

14 14 IEEE TRANSACTIONS ON GEOSCIENCE AND REMOTE SENSING Yunsong Li received the M.S. degree in telecommunication and information systems and the Ph.D. degree in signal and information processing from Xidian University, Xi an, China, in 1999 and 2002, respectively. In 1999, he joined the School of Telecommunications Engineering, Xidian University, where he is currently a Professor and also the Director of the Image Coding and Processing Center, State Key Laboratory of Integrated Service Networks. His research interests include image and video processing, hyperspectral image processing, and high-performance computing. Wei Li (S 11 M 13 SM 16) received the B.E. degree in telecommunications engineering from Xidian University, Xi an, China, in 2007, the M.S. degree in information science and technology from Sun Yat-sen University, Guangzhou, China, in 2009, and the Ph.D. degree in electrical and computer engineering from Mississippi State University, Starkville, MS, USA, in Subsequently, he was a Post-Doctoral Researcher with the University of California at Davis, Davis, CA, USA, for one year. He is currently a Professor and the Vice Dean of the College of Information Science and Technology, Beijing University of Chemical Technology, Beijing, China. His research interests include hyperspectral image analysis, pattern recognition, and data compression. Dr. Li received the 2015 Best Reviewer Award from the IEEE Geoscience and Remote Sensing Society for his service for the IEEE JOURNAL OF SELECTED TOPICS IN APPLIED EARTH OBSERVATIONS AND REMOTE SENSING (JSTARS). He is an active Reviewer of the IEEE TRANS- ACTIONS ON GEOSCIENCE AND REMOTE SENSING, the IEEE GEOSCIENCE REMOTE SENSING LETTERS, and the IEEE JSTARS. He has served as a Guest Editor for the special issue of the Journal of Real-Time Image Processing, Remote Sensing, and the IEEE JSTARS. He is currently serving as an Associate Editor for the IEEE SIGNAL PROCESSING LETTERS.