1314 IEEE JOURNAL OF SELECTED TOPICS IN APPLIED EARTH OBSERVATIONS AND REMOTE SENSING, VOL. 7, NO. 4, APRIL 2014

Transcription

1 1314 IEEE JOURNAL OF SELECTED TOPICS IN APPLIED EARTH OBSERVATIONS AND REMOTE SENSING, VOL. 7, NO. 4, APRIL 2014 A Support Vector Conditional Random Fields Classifier With a Mahalanobis Distance Boundary Constraint for High Spatial Resolution Remote Sensing Imagery Yanfei Zhong, Member, IEEE, Xuemei Lin, and Liangpei Zhang, Senior Member, IEEE Abstract In this paper, a modified conditional random fields (CRFs) classifier, namely the support vector conditional random fields classifier with a Mahalanobis distance boundary constraint (SVRFMC), is proposed to perform the task of classification for high spatial resolution (HSR) remote sensing imagery. In SVRFMC, the CRFs model has the intrinsic ability of incorporating the contextual information in both the observation and labeling fields. Support vector machine (SVM) is set as the spectral term to get a more precise estimation of each pixel s probability of belonging to each possible class. To preserve the spatial details in the classification result, a Mahalanobis distance boundary constraint is considered as the spatial term to undertake appropriate spatial smoothing. By integrating SVM and a Mahalanobis distance boundary constraint, SVRFMC can not only avoid the explicit modeling of observed data, but can also undertake appropriate smoothing with the consideration of contextual information, thereby exhibiting more universality and validity in the application of HSR image classification, especially when the image has a complex land-cover class distribution and the training samples are limited. Three HSR images comprising QuickBird, IKONOS, and HYDICE imagery were utilized to evaluate the performance of the proposed algorithm in comparison to other image classification approaches: noncontextual multiclass SVM, a traditional object-oriented classifier (OOC), an object-oriented classification based on fractal net evolution approach (FNEA) segmentation (OO-FNEA), a simplified CRF model with boundary constraint (BC-CRF), and a recently proposed contextual classifier combining SVM and Markov random fields (Markovian support vector classifier). The experimental results demonstrate that the SVRFMC algorithm is superior to the other methods, providing a satisfactory classification result for HSR imagery, including both multispectral HSR imagery and hyperspectral HSR imagery, even with limited training samples, from both the visualization and quantitative evaluations. Index Terms Conditional random fields (CRFs), high spatial resolution (HSR) imagery, mahalanobis distance boundary constraint, support vector machine (SVM). Manuscript received June 13, 2013; revised September 27, 2013; accepted October 29, Date of publication January 28, 2014; date of current version April 18, This work was supported in part by the National Natural Science Foundation of China under Grant and in part by the Foundation for the Author of National Excellent Doctoral Dissertation of China (FANEDD) under Grant (Corresponding author: Y. Zhong.) The authors are with the State Key Laboratory of Information Engineering in Surveying, Mapping, and Remote Sensing, Wuhan University, Wuhan , China ( zhongyanfei@whu.edu.cn; lxm0349@qq.com; zlp62@ whu.edu.cn). Color versions of one or more of the figures in this paper are available online at Digital Object Identifier /JSTARS I. INTRODUCTION H IGH SPATIAL RESOLUTION (HSR) remote sensing imagery provides a precise representation of the Earth s surface with a great geometrical precision and a high level of thematic details. The thematic interpretation of HSR imagery is typically based on the classification results [1]. The conventional classification methods, such as the classical maximum likelihood classifier [2], neural networks [3], the decision tree [4], [5], and the recently developed support vector machine (SVM) classifier [6] [8], are based on the pixel level without any spatial information consideration. With the improvement in the spatial resolution, the traditional noncontextual pixel-based approaches are limited and will produce an inconsistent salt-and-pepper classification without clear thematic meanings. In fact, the image will have intrinsically strong correlations between the neighboring pixels. To utilize the spatial information to help reduce the labeling uncertainty [9], the object-oriented classification (OOC) technology has been proposed by incorporating segmentation methods, such as the fractal net evolution approach (FNEA) [10], the meanshift approach [11], and expectation maximization clustering [12]. Majority voting (MV) has also been applied to combine the spectral and spatial information, where the pixels are assigned to the most frequent class within each region in a segmentation map [12], [13]. Although OOCs can obtain more homogeneous results, they face a scale selection problem in that different scales will generate different segmentation results and ultimately lead to different classification accuracies. To utilize the spatial information and avoid a scale selection problem, contextual informationbased methods have been proposed, which are usually realized by a random field model. Markov random fields (MRFs) [14], as one of the traditional contextual information statistical models, have been widely used in the field of HSR remote sensing image processing [15], [16] [18]. However, with the MRF model, there is a conditional independence hypothesis on the observed data, so the contextual information is limited to the labeling field. The clear contextual correlations between the adjacent pixels in the observation field are indirectly captured by the MRF model. To simultaneously extend the direct contextual information in both the labeling field and the observation field, the conditional random fields (CRFs) model has been proposed. As an advanced contextual framework, CRF has been successfully applied in several image analysis fields, such as medical IEEE. Personal use is permitted, but republication/redistribution requires IEEE permission. See for more information.

2 ZHONG et al.: A SUPPORT VECTOR CONDITIONAL RANDOM FIELDS CLASSIFIER 1315 image processing [19], natural image segmentation [20], stereo vision [21], activity analysis [22], and urban area detection [23]. However, the traditional CRF model needs patches of training samples to train the CRF, which restricts the application of the CRF model in the field of remote sensing image processing. To solve this problem, Zhong and Wang developed a learning CRF (L-CRF) model for hyperspectral image classification [24], which can not only avoid the explicit modeling of likelihood in the hyperspectral imagery, but can also resolve the patches of training samples problem by a piecewise training strategy. However, an L-CRF only considers the homogenous properties during spatial smoothing and still needs pairwise training samples to train the pairwise term s parameters. To further simplify and improve the model, a simplified CRF incorporating a boundary constraint as the spatial term (BC-CRF) [25] was proposed, which not only considers the boundary information but also reduces the training burden for hyperspectral image classification. Otherwise, Zhong and Wang successively proposed sparse CRFs [26] and sparse higher-order potential CRFs [27] for the modeling and classification of hyperspectral imagery. The current CRF models have succeeded in performing the task of classification for hyperspectral remote sensing imagery, but we still need to redesign them for the application of HSR imagery classification, due to the intrinsic differences between the hyperspectral imagery and HSR imagery, including the spectral information and the spatial resolution. In this paper, a new CRFs model, namely a support vector conditional random fields classifier with a Mahalanobis distance boundary constraint (SVRFMC), is proposed to perform the task of classification for HSR remote sensing imagery. In SVRFMC, there are two key issues for HSR image classification: 1) defining the appropriate spectral terms under a complex distribution condition and 2) defining a suitable spatial term to undertake the appropriate smoothing, which can keep a balance between the spatial details and the classification accuracy. To take into consideration the two key challenges discussed above under the conventional CRF framework, the proposed SVRFMC adopts SVM as the spectral term and a Mahalanobis distance boundary constraint model as the spatial term, as follows. 1) SVM serves as the spectral term to undertake the task of modeling the relationship between the observed data and the corresponding label. As a discriminative model, SVM directly utilizes training samples to build the model under the structural risk minimization (SRM) principle, thereby avoiding the explicit modeling. Moreover, it can solve the linearly inseparable problem by integrating the kernel function, which often occurs in HSR image classification. Thus, with SVM as the spectral term, SVRFMC can lay a good foundation to undertake smoothing to achieve a higher classification accuracy. Furthermore, it can avoid explicit data distribution modeling. 2) A Mahalanobis distance-based boundary constraint is used to describe the spatial term of SVRFMC. The Mahalanobis distance as a similarity measure can find the underlying relationship between the two observed vectors under the local covariance structure of the observation [28]. It has been widely applied in fault detection [28], feature selection [29], and clustering [30]. With the Mahalanobis distance boundary constraint, SVRFMC can filter a misclassified result and can also simultaneously preserve an abundance of land-cover details to improve the classification accuracy. 3) Based on the framework of CRF, a global graphical model, SVRFMC, is used to describe the contextual relationship by integrating the spectral term and the spatial term. After the training process, the precise inference of the global joint likelihood is an NP-hard problem if there are more than two classes, because the global joint likelihood cannot be factorized into per-pixel decisions [31]. To solve this problem, in SVRFMC, a local approximate inference loopy belief propagation (LBP) is used to get satisfactory classification results. The proposed algorithm was tested and compared with the recently developed pixel-based algorithm, the OCC algorithm, and the previous contextual information-based CRF model, using three real HSR images acquired by QuickBird, IKONOS, and HYDICE. The experimental results demonstrate that the proposed method has a remarkable classification accuracy for both multispectral HSR imagery and hyperspectral HSR imagery, including visualization and the quantification evaluations. Furthermore, even under limited training samples, it robustly provides a satisfactory classification result. The remainder of this paper is organized as follows. the general CRF model is briefly introduced in Section II. Section III gives a detailed description of the proposed SVRFMC classifier for HSR remote sensing imagery. Section IV gives the experimental results and analysis. The sensitivity analysis of SVRFMC is discussed in Section V. Finally, the conclusion is given in Section VI. II. THE CRFS MODEL As a contextual model, the random fields model exploits spatial dependencies between the objects in a local or global framework, e.g., MRFs and CRFs. There have been many previous studies of random field models for the probabilistic modeling of local dependencies [32], [33]. Under the Bayesian framework, the joint probability can be transformed to consider the posterior, as follows: ; where represents the labeling field and represents the observation field. To the right side of the equal sign, we can have two special probabilistic frameworks to model the posterior: the generative MRF framework and the discriminative CRF framework [34]. The generative MRF framework models the joint probability of the observed data (such as the spectral vectors) and their corresponding labels [32], formulated as. is the probability mass function of the label vector. is the probability density function (pdf) of the global feature vector conditioned to the label vector. However, with MRF, there is a conditional independence assumption for the observed data to make the computation tractable. The observed data are modeled as independent, when given the class labels, leading to the contextual limitation that the contextual information is restricted to the labeling field.

3 1316 IEEE JOURNAL OF SELECTED TOPICS IN APPLIED EARTH OBSERVATIONS AND REMOTE SENSING, VOL. 7, NO. 4, APRIL 2014 Otherwise, it also needs to make an explicit data distribution to get the (pdf). Compared with the generative MRF requiring strict independence assumptions for computational tractability, CRF is an undirected graphical model that encodes a conditional probability distribution using arbitrary features. CRF is defined as follows. Let be an undirected model over sets of random variables and, which represent the labeled variables and observed features, respectively. If the random labeled variable obeys the Markov property and it is conditioned on the observed features, then forms a CRF. Let be the set of cliques in, and CRF models the conditional probability of a label sequence given the observed data sequence, as follows: where is a clique type, is a set of clique types, is the parameter of the model, is a normalization constant known as the partition function, and is the potential function for different types of cliques upon the observed data and their corresponding labels, including unary, pairwise potentials, and even high-order potentials. Generally, only unary and pairwise clique potentials are considered, and thus (2) can be written as where represents the vertex s location in, represents its adjacent vertices which can be connected to be an edge in the neighborhood system, and refers to the label of location in the labeling field. In addition, is the unary potential, which represents the single vertex s relationship between the observed data and its corresponding label; and is the pairwise potential, which models the relationship between the current vertex and its neighborhood in the consideration of the observation and labeled fields. It is worth noting that this CRF model with unary and pairwise potentials is not fully general, because it restricts the unary potential to depend only on the pixelwise observation, whereas it may generally depend on the whole observation field. Returning to (1) (3), we can see that the CRF framework accepts one well-defined model implicitly encoded in the definition of to make a direct explicit modeling of the data distribution, and can also incorporate the contextual information in a more flexible way. The CRF model has been demonstrated to be superior to the traditional hidden MRF model in the labeling of text sequences [35], and it has also been successfully applied in several image analysis fields [19] [27]. The key to the performance of CRF is the selection of the function model for each clique. Therefore, we adapt the CRF framework for the contextual classification of HSR imagery in this paper. The unary and pairwise potentials can be viewed as arbitrary local domain-special discriminative classifiers [36]. III. THE SVRFMC FOR HSR IMAGERY Considering the characteristics of HSR remote sensing imagery and the discriminative CRF framework, we propose a new CRF classifier, namely a SVRFMC to perform the task of supervised classification for HSR remote sensing imagery. To clearly describe the details of the proposed algorithm, the following notations and definitions are used. 1) Let be the input image as the observation field, where is a set of image pixel indices, is the number of the image pixels, and each site denotes a spectral vector, where is the number of spectral bands. 2) Let be the labeling field, in which the landcover classes are labeled, where is a set of image pixel indices, and, where is the number of classes. In the regular 2-D grid image, the SVRFMC model can be formulated as follows when only the unary and pairwise potentials are considered: where is the neighborhood of the site and in this paper a fourconnected neighborhood is considered. In SVRFMC, represents the spectral term, which builds the relationship between the label and its corresponding observed spectral vector. represents the spatial term, which constructs the interaction between the single site and its neighborhoods, with the consideration of the labeling field and the corresponding observation field. For an image, the global posterior probability is obtained from the probability for each pixel, which consists of the spectral term from the single spectral feature, and the spatial term from the relationship among the neighborhood (as shown in Fig. 1). In Section III-A, we introduce the newly designed spectral and spatial potentials to enable the model to be suitable for the classification of HSR remote sensing imagery. A. Spectral Term In SVRFMC, the spectral term builds the relationship between the single spectral feature vector and its corresponding label. The contradiction between the spectral information and the spatial information in HSR imagery makes the correct classification a challenging task because the observed spectral values belonging to one class show complex statistics, which prevents us from getting a more accurate classification. In the previous CRF model, the multinomial logistic regression (MLR), used as the spectral term for hyperspectral image classification [24], [26], [27], predicts the probabilities of classes on the basis of the input features/spectral information by ranking them according to their relative importance [37]. Furthermore, it needs sufficient training samples to get a generalized model. To redefine the spectral term, we apply a discriminative classifier SVM [38] which has exhibited excellent

4 ZHONG et al.: A SUPPORT VECTOR CONDITIONAL RANDOM FIELDS CLASSIFIER 1317 Fig. 1. Illustration of 2-D CRF. Fig. 2. (a) Spectral curves of water and shadow with four bands, (b) linearly inseparable phenomenon of water and shadow classes in the original space, and (c) through the kernel function, the linearly inseparable data set is projected into a higher-dimension space to be separable. performance in the field of classification of multispectral remote sensing images [7], [8], [39], [40]. The effectiveness of SVM is mainly derived from its generality. It requires few restrictive assumptions, so the observations do not need to be explicitly distributed. Otherwise, compared with MLR, the advantage of SVM is that it computes a linear discriminative function between the two classes in a nonlinearly transformed space by adopting the kernel function (as shown in Fig. 2), and it requires fewer training samples under the SRM principle. Hence, by integrating SVM as the spectral term, CRF can model the complex statistics in HSR imagery. In this paper, we apply a multiclass probability SVM model and the spatial potential in (4) can be represented as where represents the probability of belonging to a certain class under its observed spectral vector, is the Dirac delta function, is the multiclass probability SVM model, and is the number of classes. The multiclass probability SVM model estimates the multiclass probability by combining all the pairwise comparisons [41]. The objective function of the probability estimation is shown as follows: where class probability, and we use all. is the estimated pairwise to estimate B. Spatial Term If only considering the spectral information, even the most powerful classifier will get some discontinuous salt-and-pepper classification results. Therefore, the function of the spatial term is modeling the contextual information to help smooth and correct wrongly classified pixels to improve the accuracy of the classification. In general, HSR imagery has rich object details but poor observed data. Hence, modeling the contextual relationship under the poor observed data condition may not be so accurate. Fortunately, there is also a complex interaction among the neighborhood, including the spectral space and the spatial space.

5 1318 IEEE JOURNAL OF SELECTED TOPICS IN APPLIED EARTH OBSERVATIONS AND REMOTE SENSING, VOL. 7, NO. 4, APRIL 2014 A previous CRF model used the Potts model as the spatial term to capture the underlying spatial relationship for different class cases in both the label and observed data [24]. There has also been a boundary constraint contextual information model developed [25]. However, the current models generally do not consider the use of spectral contextual information in the observed data. In fact, in certain kinds of terrain, the spectral variation of a pixel may follow some underlying patterns in the neighboring region rather than being random. Consequently, we apply a new Mahalanobis distance boundary constraint to model the spatial term. The spatial term can then be written as where represents the scalar distance between the spectral vectors and and is the mean value of in the image.the kernel thought of this model is whether the labels agree with their feature vectors. If the pixel and pixel have similar feature vectors, then the value of is close to 1 and is close to 0. Therefore, the probability of them belonging to different labels is small. In contrast, if pixel and pixel have similar feature vectors, then the value of will be relatively increased. Such a spatial term considering the consistency between the observed feature vectors and their labels can pass on the correct information in the network. The most important factor is the similarity measurement. Here, we use the Mahalanobis distance as a similarity measure in the boundary constraint model. This is based on the probability function of a multivariate normal distribution [42], adaptively adjusting the measurement standard by considering the correlation in the original data. However, since the Mahalanobis distance will exaggerate a tiny change in the data variables, a modified Mahalanobis distance is proposed by integrating the Pearson product moment correlation coefficient [43], as follows. A similar spectral similarity measurement has been used for hyperspectral remote sensing image data [44] of, the more similar the two vectors. The effect of in is to measure the similarity more accurately, which restricts the influence of the tiny changes in the data variables. The covariance matrix enables the Mahalanobis distance generalization properties, which can build the correlation between any two uncertain vectors under the local covariance structure of the observation. Hence, the boundary constraint with the modified Mahalanobis distance can account for the spectral contextual information in the observed data. From the model shown in (7), we can see that is modulated by a variable weighting scheme, incorporating the spectral contextual information and the spatial contextual information; thus, the proposed SVRFMC can model the interaction between the labels under the pairwise discriminate of the observed data. C. Methodology Under the CRF framework, we incorporate the SVM and a Mahalanobis distance boundary constraint to perform the task of classification of HSR imagery, covering the shortcomings of the noncontextual pixel-based algorithms and the current contextual information models. After the modeling procedure, the complete implementation of the SVRFMC algorithm includes the following steps: 1) model training, 2) model initialization, and 3) model inference. The flowchart of SVRFMC is illustrated in Fig. 3. 1) The Training Process of SVRFMC: The training procedure (the model s parameter estimation) is one of the most important tasks for HSR image classification. First, based on the characteristics of the remote sensing image and the application purpose, the region of interest (ROI) or the samples for different classes need to be selected to represent the user s expected classes from the image. Then, with the selected training samples, let be the training samples, where is the corresponding label of the observed data, isacliquetype,and is a set of clique types. Under the modeled SVRFMC, for the computational tractability, a piecewise training strategy [45] [48] is applied for the SVRFMC parameter estimation, with the special divisions, where represents the kind of the clique type. In SVRFMC, is equal to 2, so only two types of cliques are considered as the unary and pairwise terms, which represent the spectral and spatial terms, respectively. Considering the spectral and spatial terms in the piecewise training framework, the objective function of (2) can be written as: where represents the Mahalanobis distance, is the Pearson product moment correlation coefficient, is the number of spectral bands, is the covariance matrix of the image, and and represent the mean values of the spectral vectors and, respectively. The smaller the value where is the potential function for the different types of cliques and represents the parameters in the model. Here, only the unary potential and pairwise potential are taken into account, and accordingly

6 ZHONG et al.: A SUPPORT VECTOR CONDITIONAL RANDOM FIELDS CLASSIFIER 1319 Fig. 3. Flowchart of the SVRFMC algorithm. the and represent the parameters in the unary potential and pairwise potential, respectively. According to (11), the SVRFMC model can independently train local undirected classifiers for each kind of clique in the piecewise training framework. Since the spatial term with the Mahalanobis distance-based boundary constraint does not contain parameters, there are no pairwise potential s parameters in the SVRFMC. Consequently, the whole model s parameters is equal to the unary potential s parameters, which are the parameters in the SVM model. The SVRFMC model parameter estimation can then be converted into a parameter optimization problem of the spectral term SVM, solving the quadratic problems. In this paper, we utilize the SMO-type decomposition method of Fan et al. [41]. For more details on parameter estimation, refer to [50], [51], and [52]. 2) The Initialization of the SVRFMC Contextual Information Structure: After accomplishing the training of SVM, the spectral term and spatial term should be combined in the SVRFMC framework [as shown in (4)] to obtain the trained SVRFMC for the HSR image classification. For each pixel in the HSR image, the spectral term is initialized by SVM, which gets the optimal parameters obtained from the training part, and the spatial term is initialized by the Mahalanobis distance boundary constraint. However, this approach may lead to an over-smoothing problem during inference in the combined model. To solve this problem, fixed powers are designed to compensate for the over-smoothing problem in the model. The model s posterior probability in (4) can be transformed as follows: where the fixed powers and are assigned to control the importance of the spectral and spatial terms, respectively. The fixed powers and use cross validation [53] to get their optimal value. For computational convenience, only needs to be adjusted, ranging from 0.1 to 1, and is fixed to be, accordingly. 3) The Inference of SVRFMC: With the initialization of the SVRFMC contextual information structure, the trained model can undertake inference to find the optimal labels for the test images. In this paper, the inference is carried out with an approximate local strategy LBP based on the maximum a posteriori (MAP) [24], in which the posterior probability equation (12) is written as

7 1320 IEEE JOURNAL OF SELECTED TOPICS IN APPLIED EARTH OBSERVATIONS AND REMOTE SENSING, VOL. 7, NO. 4, APRIL 2014 Fig. 4. Illustration of the LBP message passing. The LBP iteration is operated by passing the messages from all the parallel pixels around the four-connected image grid. represents the received message from the single corresponding spectral vector data, and is the edge message sent from pixel to pixel, which represents the influence of pixel on pixel. Here, the initialization of the message being passed is obtained from the SVRFMC model s spectral and spatial terms. The iteration then updates the edge message being passed in the network. When the message being passed in the network becomes stable, the trained network can be used to undertake inference of the unlabeled samples to obtain the final classification results. Fig. 4 illustrates the message related to pixel 5 in the network, where the blue nodes represent the labeling field and the yellow nodes represent the observation field. We can see that pixel 5 can receive a message from the corresponding observed data and the four-connected neighbor,,,, and pixel 5 can also affect its neighbor nodes by passing its message, e.g.,,. IV. EXPERIMENTS AND ANALYSIS A. Experimental Setup To evaluate the performance of the proposed SVRFMC, five previous HSR image classifiers are compared, including noncontextual pixel-based algorithms, object-oriented algorithms, and contextual classifiers. They are 1) noncontextual multiclass SVM [52], 2) a traditional OOC [54], 3) an OCC based on FNEA segmentation [10] (OO-FNEA) with an MV method [55], 4) a simplified BC-CRF [25], and 5) a recently proposed contextual classifier combining SVM and MRF, namely the Markovian support vector classifier (MSVC) [18]. The noncontextual multiclass SVM approach applies a radial basis function (RBF) as the kernel, and the cross-validation approach is used to determine the appropriate values of the parameters and [50]. The object-oriented methods are specifically designed for the classification of HSR remote sensing imagery. Thus, two common types of object-oriented methods are compared in this paper. One approach is extracting each mean spectral feature from each corresponding object, so that the pixels in a homogeneous object share the same spectral feature, then finally, a classifier is used (the SVM classifier is used in this paper) to undertake the classification [54]. The other approach is performing an MV [55] within the segments on the noncontextual multiclass SVM classification result. The FNEA segmentation, applied in experiments, is implemented by ecognition software. Since the scale parameter has a great effect on the segmentation result, the scale is set to range from 2 to 100 (scale 2, 5, 8, 10, 20, 30, 40, 50, 60, 70, 80, 90, 100). Otherwise, the image layer weights of the segmentation are all set to be 1, which means that the influence of each band is equal, and the shape and compactness parameters in the composition of the homogeneity criterion are set to 0.1 and 0. 5, respectively. The parameters of the MLR model in BC-CRF are optimized by the variable splitting and augmented Lagrangian (LORSAL) algorithm [56]. The smoothing parameters in BC-CRF and SVRFMC are obtained by cross validation [53]. The posterior probability of the related BC-CRF and SVRFMC are maximized by LBP, the iteration number of which is set to be 20 in the experiments. In particular, LBP in SVRFMC is initialized with the classification map obtained by the noncontextual multiclass SVM, and LBP in BC-CRF is initialized by the MLR classification result. In MSVC, the smoothing parameter is optimized by the Ho-Kashyap algorithm [57], and the energy function is minimized by ICM. The iteration of ICM is set to be 100. However, we need to explain that if the difference between the two iterations is smaller than the threshold, it will stop the iteration. Hence, generally, the ICM approach needs 5 15 iterations. To allow a fair comparison, ICM of MSVC is initialized with the aforementioned noncontextual multiclass SVM, the same as the SVRFMC s spectral term. All the models of SVM with RBF are optimally selected by the grid search method [52]. All six algorithms shown in Section IV are accomplished by a computer with an Intel Core 3.10 GHz and 8.00 GB of RAM. The noncontextual multiclass SVM is run in the Visual 6.0 platform, and for the two object-oriented algorithms, OOC and OO-FNEA, the segmentation is first undertaken by ecognition software, and the algorithms are then run in the Visual 6.0 platform. The other three algorithms, BC-CRF, MSVC, and SVRFMC, are all run in the MATLAB 2011b platform. Three different types of HSR images, acquired by QuickBird, IKONOS, and HYDICE, are used to test these classifiers. The overall accuracies (OAs) and Kappa statistics (Kappa) [58] are then used as measurements to carry out the quantitative evaluation. Lastly, we apply the McNemar s test [59] to measure the statistical significance of the difference between the two sets of classification results under the same training data set condition. If given two classifiers and, this test computes the following standardized normal statistics: where represents the number of pixels misclassified by, not by, and represents the number of pixels misclassified by, not by.if, this statistic can be treated as a chi-squared distribution. This test can measure whether the difference between the two classifier results is significant. Accepting the common 5% level of significance, then. If the McNemar s value is greater than, then the two algorithms results are significantly different. The computation times of the six algorithms are also provided. However, as the different approaches are implemented in the different languages and/or software platforms mentioned above, a computation time comparison of the six algorithms is not fair and cannot be used for an absolute comparison.

8 ZHONG et al.: A SUPPORT VECTOR CONDITIONAL RANDOM FIELDS CLASSIFIER 1321 Fig. 5. Fancun QuickBird image: (a) original RGB image (1, 2, 3) and (b) ground-truth image. B. Experiment 1: QuickBird HSR Image The first experiment is performed using a 2.4-m resolution QuickBird multispectral HSR image ( ) acquired in 2010 with four bands, from the Fancun area of Sanya, Hainan Province, China [60]. The observed image area is expected to consist of seven representative classes: water, tree, grass, bare soil, building, road, and shadow. Fig. 5 displays the experimental Fancun image and the corresponding ground-truth image. In this experiment, only 100 training samples are randomly selected for each class from the ground truth, and the other samples are used as test samples, as shown in Table I. In this experiment, the and parameters are set to be 256 and 0.5 by cross validation in SVM, respectively. OO-FNEA and SVRFMC are both developed from the SVM results. The scale of OO-FNEA is chosen to be 30. The fixed powers of the spectral term for BC-CRF and SVRFMC are set to be 0.3 and 0.2, respectively. The classification results using SVM, OOC, OO-FNEA, BC-CRF, MSVC, and the proposed SVRFMC are shown in Fig. 7(a) (f), respectively. With the consideration of the spatial information, OOC, OO-FNEA, BC-CRF, MSVC, and SVRFMC exhibit better visualization results, whereas the classification result obtained by noncontextual SVM has a lot of isolated classification noise. Furthermore, compared to OOC, OO- FNEA, and MSVC, the classification results of BC-CRF and SVRFMC show more complete and satisfactory results. To confirm the differences between these algorithms, two small image areas are zoomed and are shown in Fig. 6(g) and (h). In the zoomed Fig. 6(g), the noncontextual SVM result contains abundant land-cover information, as well as some isolated classification. OOC, OO-FNEA, and SVRFMC display smoother results, keeping good boundary information and also wiping out the salt-and-pepper-like classification. The MSVC s result effect is between the SVM and OOC, OO-FNEA, and SVRFMC results. In addition, BC-CRF shows the over-smoothing phenomenon, in that two footpaths existing in the original image have been ignored in the classification result. In the zoomed image in Fig. 6(h), the SVM contains salt-and-pepper-like isolated classification, and the other contextual classifiers results mostly keep a good shape of the objects. It is worth noting that there are some meaningless regions in one homogeneous area in OOC and TABLE I NUMBER OF TRAINING AND TEST SAMPLES OF THE SEVEN CLASSES IN THE FANCUN DATA SET MSVC; instead, OO-FNEA, BC-CRF, and SVRFMC display a more complete object visualization. In conclusion, compared with the object-oriented algorithms, SVRFMC can achieve a competitive classification visualization, keeping complete boundary information. Furthermore, compared to the recently developed contextual approaches, SVRFMC can perform appropriate smoothing and exhibit a satisfactory visualization. In order to carry out a quantitative evaluation, the OA and Kappa [48] of the different algorithms are calculated. The classification accuracies and computation times provided by the different methods are shown in Table II. It can be seen that the best OA is provided by SVRFMC with 95.43%. It is worth noting that the object-oriented classifiers (OOC and OO-FNEA) only have about 93% accuracy, although they have a similar visualization result to SVRFMC. The reason for this is that the result was very closely related to the segmentation s result, where the divided blocks/objects were regarded as belonging to the same class, and such a segmentation preprocessing procedure can successfully keep the boundary information, but the segmentation does not always match the actual object contours, which leads to the whole region being misclassified, thereby displaying an undesirable accuracy result. Hence, the scale selection is the key point of the object-oriented approach. The computation times of the different approaches are shown in the last column of Table II. The noncontextual multiclass SVM is run in the Visual 6.0 platform, and for the two object-oriented algorithms, OOC and OO-FNEA, the segmentation is first undertaken by

9 1322 IEEE JOURNAL OF SELECTED TOPICS IN APPLIED EARTH OBSERVATIONS AND REMOTE SENSING, VOL. 7, NO. 4, APRIL 2014 Fig. 6. Classification results for Experiment 1 the QuickBird data set: (a) SVM, (b) OOC, (c) OO-FNEA, (d) BC-CRF, (e) MSVC, (f) SVRFMC, (g) and (h) zoom images for all the classifiers results. Fig. 7. Wuhan IKONOS image: (a) original RGB image (1, 2, 3) and (b) ground-truth image.

10 ZHONG et al.: A SUPPORT VECTOR CONDITIONAL RANDOM FIELDS CLASSIFIER 1323 TABLE II CLASSIFICATION ACCURACIES OF THE DIFFERENT ALGORITHMS FOR THE FANCUN DATA SET TABLE III MC TEST FOR THE QUICKBIRD DATA SET ecognition software, and the algorithms are then run in the Visual 6.0 platform. It is worth noting that the computation time of OOC is about 10 times that of SVM. As OOC will get a new spectral image under each scale s segmentation, we set 10 scales in the experiments here, from 10 to 100. We then undertake classification on the new spectral images and choose the optimal classification accuracy as the final object-oriented method s result. Similarly, OO-FNEA uses voting in the segmentation regions based on the pixel classification result; hence, it is faster than OOC. The other three algorithms, BC-CRF, MSVC, and SVRFMC, are all run in the MATLAB 2011b platform. Compared to these algorithms under the experimental software conditions used in this paper, the computation time of the proposed SVRFMC is acceptable for the three experimental data sets. However, because the six approaches are not implemented in the same language or software, the computation time comparison is not fair, and it cannot be treated as an absolute comparison. In addition to the classification accuracy, Table III provides a pairwise comparison of the six algorithms using the McNemar s test. The value of the McNemar s test indicates whether the two algorithms have a significant difference. If the value is greater than, then it is regarded as being significantly different; furthermore, the greater the value, the greater the difference. In Table III, all the values are greater than the critical value of 3.84, which implies that all the classifiers have significantly different prediction rates. To summarize, balancing the visualization classification and the quantitative classification, SVRFMC can achieve a satisfactory classification by incorporating SVM and the Mahalanobis distance-based boundary constraint, in which SVM can provide a satisfactory unary foundation for the HSR images, and the Mahalanobis distance-based boundary constraint can undertake appropriate spatial smoothing based on the assumption of the covariance structure of the local observations. TABLE IV NUMBER OF TRAINING AND TEST SAMPLES OF THE SEVEN CLASSES IN THE WUHAN DATA SET C. Experiment 2: IKONOS Image The second data set was obtained by the IKONOS satellite over Wuhan University, Wuhan city, Hubei Province, China. The 4-m resolution Wuhan HSR image has, with four bands (blue, green, red, and near-infrared) [60]. Based on the local knowledge and the photo interpretation results, seven representative classes are considered: building, grass, water, shadow, bare soil, tree, and road. Fig. 7 displays the experimental Wuhan image and the corresponding ground-truth land-cover image. Table IV shows the number of training samples and test samples for each class. In this experiment, the and parameters in SVM are set to be 4096 and 0.5, respectively. The of BC-CRF and SVRFMC are set to be 0.3 and 0.2, respectively. Fig. 8(a) (f) illustrates the classification results using SVM, OOC, OO-FNEA, BC-CRF, MSVC, and SVRFMC, respectively. The smooth visualization results of the Wuhan image are similar to Experiment 1. However, the classifications of some regions are individually different, especially in the classification of the water and shadow classes. A possible reason for this is that the image s spatial resolution is 4 m,

11 1324 IEEE JOURNAL OF SELECTED TOPICS IN APPLIED EARTH OBSERVATIONS AND REMOTE SENSING, VOL. 7, NO. 4, APRIL 2014 Fig. 8. Classification results for Experiment 2 IKONOS data set: (a) SVM, (b) OOC, (c) OO-FNEA, (d) BC-CRF, (e) MSVC, and (f) SVRFMC. and it will contain more objects in one pixel, but the image has limited four-band information, so the divisibility between the different classes becomes lower, especially when distinguishing classes sharing the same spectral features, e.g., the water class and the shadow class. Therefore, some water regions are wrongly classified into shadow, and some buildings are classified into the shadow class, shown as the circles in Fig. 8. From the visualization comparison, BC-CRF exhibits the mostsmoothed result, while the results of OOC, OO-FNEA, and SVRFMC are competitive, displaying appropriate smoothing in both the homogeneous regions and heterogeneous regions. In OOC, there are large amounts of pixels wrongly classified into the shadow class. The reason for this is interpreted as above; otherwise, it is because the improper segmentation leads to the inaccurate extraction of the mean spectral feature from each region. Overall, OO-FNEA shows the best visualization with this data set. Next, a quantifiable evaluation is undertaken to assess the six classifiers. The classification accuracies and the computation times for the six classifiersaregivenin Table V. ThebestOAisprovided by OOC at 90.71%, and the SVRFMC ranks second at 0.38% lower. It is worth noting that the accuracy of the building and road classes are all about 60% 70%. This is because, under the four-band information, some classes have a poor description, with similar but limited spectral features in the image, which leads to the wrong classification. Experiment 2 demonstrates that the limited spectral information becomes a bottleneck in the multispectral HSR image classification. In this data set, OO-FNEA shows the best visualization, but the quantifiable evaluation is not as high as SVRFMC. We can ascribe this phenomenon to the segmentation. The human vision prefers structured shapes without isolated noise, and the segmentation preprocess can maintain good boundary information to exhibit a structured visualization. However, at a single scale, segmentation is not suitable for all the objects in the HSR image, since there are different scales of objects in the HSR image. This

12 ZHONG et al.: A SUPPORT VECTOR CONDITIONAL RANDOM FIELDS CLASSIFIER 1325 TABLE V CLASSIFICATION ACCURACIES OF THE DIFFERENT ALGORITHMS FOR THE WUHAN DATA SET TABLE VI MC TEST FOR THE IKONOS DATA SET optimal segmentation scale selection problem further leads to the wrong classification of the whole object. It is a remarkable fact that BC-CRF saccuracy is lowerthan SVMs. There aretwo main reasons for this: one is the poor spectral information, which makes correct classification a difficult task for MLR; the other reason is the limited training samples, which decrease MLR s generalization ability. Consequently, MLR provides wrong initialization of the spectral term in BC-CRF, leading to the final poor classification accuracy. It is worth recalling here that SVM is run in the Visual 6.0 platform, and for the two object-oriented algorithms, OOC and OO-FNEA, the segmentation is first undertaken by ecognition software, and the algorithms are then run in the Visual 6.0 platform. The other three methods, BC-CRF, MSVC, and SVRFMC, are all run in the MATLAB 2011b platform. Hence, the computation times of these six algorithms are just provided as an illustration of the computational efficiency under this paper s experimental configuration, and they cannot be treated as an absolute comparison. Table VI shows the McNemar s test between all the methods. From Table VI, we can see that SVRFMC is significantly different from the other classifiers, but the value of the McNemar s test between SVRFMC and MSVC, SVRFMC and OO-FNEA is relatively small, which indicates that the classifiers have similar classification decisions in the ground-truth region. In conclusion, compared with the other classifiers, SVRFMC can achieve appropriate smoothing of the heterogeneous regions when the image has complex components and can preserve more complete structural information, without a scale selection problem. D. Experiment 3: Washington DC HYDICE Image The third experiment is performed using a hyperspectral digital imagery collection experiment (HYDICE) airborne hyperspectral HSR image ( ) acquired over Washington DC, as shown in Fig. 9(a). The ground-truth image is shown in Fig. 9(b). The list of classes and their corresponding training and test samples are given in Table VII. In this experiment, the and parameters in SVM are set to be and , and the of BC-CRF and SVRFMC are set to be 0.3 and 0.6, respectively. Compared with the contextual classifiers, OOC, OO-FNEA, BC-CRF, MSVC, and SVRFMC, the result of the SVM approach contains some salt-and-pepper-like classification results. Among the contextual classifiers results, OOC, OO-FNEA, and MSVC exhibit similar classification visualization in the homogenous regions, which have incomplete boundary shapes and still have some isolated parts in the homogenous regions (shown as the circular building in the small square and the rectangular building in the big rectangle). Otherwise, it is notable that MSVC preserves more objects details in the results, such as the shape of trees. However, the over-smoothing phenomenon is exhibited in BC-CRF s result; for example, the rectangular building in the top-right of the image, shown in the black rectangle, has extended its region to the surrounding road. In comparison, the proposed SVRFMC shows the best visualization results, maintaining the best structured information of the objects, such as the boundary shape of the circular building and the rectangular building. To summarize, SVRFMC can appropriately discriminate the components at the region boundaries to keep the structured information of objects.

13 1326 IEEE JOURNAL OF SELECTED TOPICS IN APPLIED EARTH OBSERVATIONS AND REMOTE SENSING, VOL. 7, NO. 4, APRIL 2014 Fig. 9. Washington DC HYDICE image: (a) original RGB image (7, 101, 188) and (b) ground-truth image. TABLE VII NUMBER OF TRAINING AND TEST SAMPLES OF THE SEVEN CLASSES IN THE HYDICE DATA SET For a further quantitative evaluation, the classification accuracies of the six different methods with the HYDICE data set are summarized in Table VIII. The best classification accuracy is achieved by SVRFMC at 97.22%. MSVC s accuracy is higher than BC-CRF s; this may be due to the over-smoothing in BC- CRF and the more details in MSVC. It is worth noting that the classification accuracy of OOC is the worst result, although it does exhibit a satisfactory visualization. From Fig. 10, compared to the other results, OOC classifies some grass into tree, and some road is also classified into building. Therefore, the accuracy of roof and grass is low, which leads to the poor classification accuracy. The McNemar values between the pairwise algorithms in this data set are shown in Table IX. This demonstrates that SVRFMC is significantly different from the other classifiers, the values of which are all greater than As discussed in the experiment, in terms of visualization and quantitative evaluations, the proposed SVRFMC slightly outperforms the other five methods, which further demonstrates its validity in hyperspectral HSR image classification. V. SENSITIVITY ANALYSIS The SVRFMC has an important parameter, a fixed power on the spectral term, which evaluates the importance of the unary potential in SVRFMC. In this section, a sensitivity analysis of different values of the fixed power is carried out to evaluate the performance of SVRFMC. In addition, to further investigate the robust effectiveness of SVRFMC, a sensitivity analysis of the number of training samples for the six algorithms is undertaken using the three HSR images acquired by QuickBird, IKONOS, and HYDICE. A. Sensitivity in Relation to Parameter To study the SVRFMC sensitivity in relation to the model parameter [as shown in (18)], the fixed power in the spectral term is set from 0.1 to 1, with 0.1 as the interval. The relationship between the fixed power and the classification OA is shown in Fig. 11. In this study, we also display the BC-CRF sensitivity to the fixed power, because they are both derived from conventional CRF. As shown in Fig. 11(a) (c), with the increase of,theoaof SVRFMC and BC-CRF displays the decreasing trends for the HSR images. The increase in means increasing the spectral effect and decreasing the spatial effect; hence, as the effect of the spatial term decreases, the capability to smooth the isolated classification results becomes increasingly weak, leading to the decreasing trend in the classification accuracy. For the Quick- Bird and IKONOS images, the OA of SVRFMC ranges from 95.55% to 92.40% and 90.33% to 88.64%, compared with BC-CRF, which ranges from 94.21% to 89.79% and 87.71% to 83.31%, respectively. Compared to BC-CRF, the effect of the fixed power on SVRFMC is less in the HSR image. However, in the hyperspectral HSR image, both OAs increase to the highest value, and then decrease. From Fig. 12(c), we can see that the OA significantly improves with the decrease in,and it then begins to decrease when it reaches the highest OA. The former phenomenon may be because the over-smoothing problem occurred in the result due to the high weight of the spatial term. The latter phenomenon is because the capability of smoothing a misclassified pixel decreases with the increase in the fixed power. Hence, the selection of the fixed weight is important in the classification process, especially for hyperspectral HSR imagery. Finally, we conclude that the proposed SVRFMC model is a stable classifier for M-HSR imagery. Furthermore, with the appropriate fixed weight, the SVRFMC s OA can be greatly increased, compared to BC-CRF. B. Sensitivity in Relation to Training Set Size To study the SVRFMC sensitivity in relation to the training set size, the other parameters are kept the same as those in the

14 ZHONG et al.: A SUPPORT VECTOR CONDITIONAL RANDOM FIELDS CLASSIFIER 1327 TABLE VIII CLASSIFICATION ACCURACIES OF THE DIFFERENT ALGORITHMS FOR THE HYDICE DATA SET Fig. 10. Classification results for Experiment 3 Washington DC HYDICE data set: (a) SVM, (b) OOC, (c) OO-FNEA, (d) BC-CRF, (e) MSVC, and (f) SVRFMC. TABLE IX MC TEST FOR THE HYDICE DATA SET experiments. For all three data sets, six different numbers of training samples are selected for each class, 50, 100, 200, 300, 400, and 500, which are randomly selected for the training procedure, and the remaining samples are used to evaluate the classification accuracies. It should be noted that the classification of 100 training samples for each class has already been shown in Section IV. As shown in Fig. 12, as the number of training samples increases, the OAs obtained by the different methods display different trends. SVM, MSVC, and SVRFMC share similar

15 1328 IEEE JOURNAL OF SELECTED TOPICS IN APPLIED EARTH OBSERVATIONS AND REMOTE SENSING, VOL. 7, NO. 4, APRIL 2014 Fig. 11. Sensitivity of the spectral term fixed power: (a) Fancun QuickBird image, (b) Wuhan IKONOS image, and (c) Washington DC HYDICE image. Fig. 12. Sensitivity of the number of training samples: (a) Fancun QuickBird image, (b) Wuhan IKONOS image, and (c) Washington DC HYDICE image. trends, and the overall classification accuracy of SVRFMC is better than MSVC, and MSVC in turn is better than SVM. Otherwise, the BC-CRF s graph is similar to the three previously mentioned algorithms, except for the 100 training samples case for each class in the IKONOS data set, where the classification accuracies of BC-CRF and SVM are on the same level. This is because, under the limited training samples and poor spectral description in the IKONOS data set, it is difficult for the MLR model to get a robust and generalized model, leading to the poor final classification result of BC-CRF. However, BC-CRF s OA gradually increases with the increase in training samples, and especially in the 100 training samples for each class condition, it is significantly improved. The spectral term in BC-CRF, MLR, needs enough training samples to get a stable model, which is the initialization of BC-CRF. In addition, it is worth noting that the graph trends of the object-oriented approaches (OOC and OO- FNEA) are unstable, with the classification accuracy fluctuating. For example, OOC s classification accuracy is significantly improved under the 300, 400, and 500 training samples for each class in the IKONOS data set, whereas the accuracy of OO- FNEA is greatly decreased. The reason for this phenomenon is that the OCC results do not only depend on the segmentation results, but also depend on the classifier performance. If and only if both match, the classification can obtain the ideal result. Furthermore, the behavior of OOC in the HYDICE data set is very poor and is worse than the noncontextual SVM. This may be because the inappropriate segmentation led to the incorrect mean feature extraction, which resulted in the poor performance. Compared to these classifiers, SVRFMC displays the best results in all three data sets, even in the case of limited training samples. In conclusion, the SVRFMC model has a better robustness with both multispectral and hyperspectral HSR images. VI. CONCLUSION In this paper, an SVRFMC has been proposed to perform the task of classification for HSR imagery. To settle the contradiction between the classification accuracy and the land-cover details in HSR image classification, the proposed model redesigns the spectral term and spatial term in the CRF model. In SVRFMC, an SVM is set as the spectral term to cover the salt-and-pepper-like classification under the limited band information, and a modified Mahalanobis distance boundary constraint is used as the spatial term to preserve as much of the land-cover details as possible during spatial smoothing. With the combination of the spectral term and spatial term, SVRFMC can exhibit favorable classification, in both homogeneous and heterogeneous regions, even when the training samples are limited. In the proposed model, a piecewise training strategy is applied in the parameter training. Then, during the combination procedure of the model, the fixed powers, which are obtained by cross validation, are added to prevent the over-smoothing problem. With the trained terms and

16 ZHONG et al.: A SUPPORT VECTOR CONDITIONAL RANDOM FIELDS CLASSIFIER 1329 fixed powers, a global joint likelihood is undertaken; however, it cannot be factorized into per-pixel decisions. As a result, the exact inference is computationally intractable because it is an NP-hard problem. In this paper, a local approximate inference LBP is used to obtain satisfactory results. Three real data experiments using three types of HSR images from QuickBird, IKONOS, and HYDICE convincingly demonstrate the validity of the proposed model, compared with SVM, the oriented-object classifiers (OOC, OO-FNEA), the previous CRFs classifier (BC-CRF), and another kind of contextual classifier (MSVC). Among the classification results, SVRFMC obtains better thematic interpretation and classification results in both the homogeneous and heterogeneous regions, from both a visual and quantitative evaluation. Our future work will explore extended neighborhood and adaptive neighborhood for CRF approach modeling of the neighborhood relationship. Furthermore, an unsupervised or semi-supervised CRF algorithm for HSR imagery will also be in our future work to make the algorithm more intelligent. ACKNOWLEDGMENTS The authors want to thank the editor, associate editor, and the anonymous reviewers for their helpful comments and suggestions. They also would like to thank Beijing Panorama Space Technology Co., Ltd for providing the free QuickBird image and also the research group supervised by Prof. D. Landgrebe, Purdue University, West Lafayette, IN, USA, for providing the free downloads of the HYDICE image. REFERENCES [1] G. M. Foody, Status of land cover classification accuracy assessment, Remote Sens. Environ., vol. 80, no. 1, pp , Apr [2] J. D. Paola and R. A. Schowengerdt, A detailed comparison of backpropagation neural network and maximum-likelihood classifiers for urban land use classification, IEEE Trans. Geosci. Remote Sens., vol. 33, no. 4, pp , Jul [3] F. D. Frate, F. Pacifici, G. Schiavon, and C. Solimini, Use of neural networks for automatic classification from high-resolution images, IEEE Trans. Geosci. Remote Sens., vol. 45, no. 4, pp , Apr [4] S. Moustakidis, G. Mallinis, N. Koutsias, J. B. Theocharis, and V. Petridis, SVM-based fuzzy decision trees for classification of high spatial resolution remote sensing images, IEEE Trans. Geosci. Remote Sens., vol. 50, no. 1, pp , Jan [5] T. R. Tooke, N. C. Coops, N. R. Goodwin, and J. A. Voogt, Extraction urban vegetation characteristics using spectral mixture analysis and decision tree classifications, Remote Sens. Environ., vol. 113, no. 2, pp , Feb [6] J. Muñoz-Marí, F. Bovolo, L. Gómez-Chova, L. Bruzzone, and G. Camps- Valls, Semisupervised one-class support vector machines for classification of remote sensing data, IEEE Trans. Geosci. Remote Sens., vol. 48, no. 8, pp , Aug [7] F. Melgani and L. Bruzzone, Classification of hyperspectral remote sensing images with support vector machines, IEEE Trans. Geosci. Remote Sens., vol. 42, no. 8, pp , Aug [8] B. Waske and J. A. Benediktsson, Fusion of support vector machines for classification of multisensor data, IEEE Trans. Geosci. Remote Sens., vol. 45, no. 12, pp , Dec [9] G. Forzieri, G. Moser, E. R. Vivoni, F. Castelli, and F. Canovaro, Riparian vegetation mapping for hydraulic roughness estimation using very high resolution remote sensing data fusion, J. Hydraul. Eng., vol. 136, pp , [10] M. Baatz and A. Schape, Multiresolution segmentation An optimization approach for high quality multi-scale image segmentation, in Angewandte Geographische Informationsverarbeitung XII, J. Strobl, T. Blaschke, and G. Griesebner, Eds., Heidelberg, Germany: Wichmann-Verlag, 2000, pp [11] D. Comaniciu and P. Meer, Mean shift: A robust approach toward feature space analysis, IEEE Trans. Pattern Anal. Mach. Intell., vol. 24, no. 5, pp , May [12] Y. Tarabalka, J. A. Benedktsson, and J. Chanussot, Spectral spatial classification of hyperspectral imagery based on partitional clustering techniques, IEEE Trans. Geosci. Remote Sens., vol. 47, no. 8, pp , Aug [13] Y. Tarabalka, J. A. Benediktsson, J. Chanussot, and J. C. Tilton, Multiple spectral spatial classification approach for hyperspectral data, IEEE Trans. Geosci. Remote Sens., vol. 48, no. 11, pp , Nov [14] A. H. S. Solberg, T. Taxt, and A. K. Jain, A Markov random field model for classification of multisource satellite imagery, IEEE Trans. Geosci. Remote Sens., vol. 34, no. 1, pp , Jan [15] G. Moser, S. B. Serpico, and J. A. Benediktsson, Land-cover mapping by Markov modeling of spatial contextual information in very-high-resolution remote sensing images, in Proc. IEEE, Mar. 2013, vol. 101, no. 3, pp [16] Q. Jackson and D. Landgrebe, Adaptive Bayesian contextual classification based on Markov random fields, IEEE Trans. Geosci. Remote Sens., vol. 40, no. 11, pp , Nov [17] G. Trianni and P. Gamba, Boundary-adaptive MRF classification of optical very high resolution images, in Proc. IEEE Int. Geoscience Remote Sensing Symp., 2007, pp [18] G. Moser and S. B. Serpico, Combining support vector machines and Markov random fields in an integrated framework for contextual image classification, IEEE Trans. Geosci. Remote Sens., vol. 51, no. 5, pp , May [19] Y. Artan, D. L. Langer, M. A. Haider, T. H. van der Kwast, A. J. Evans, M. N. Wernick, and I. S. Yetik, Prostate cancer segmentation with multispectral MRI using cost-sensitive conditional random fields, in Proc. IEEE Int. Symp. Biomed. Imag., 2009, pp [20] X. He, R. S. Zemel, and M. A. Carreira-Perpinan, Multiscale conditional random fields for image labeling, in Proc. IEEE Computer Society Conf. Comput. Vis. Pattern Recognit., 2004, vol. 2, pp [21] D. Scharstein and C. Pal, Learning conditional random fields for stereo, in Proc. IEEE Conf. Comput. Vis. Pattern Recognit., Minneapolis, MN, USA, 2007, pp [22] C. Sminchisescu, A. Kanaujia, Z. Li, and D. Metaxas, Conditional models for contextual human motion recognition, Comput. Vis. Image Understand., vol. 104, pp , [23] P. Zhong and R. Wang, A multiple conditional random fields ensemble model for urban area detection in remote sensing optical images, IEEE Trans. Geosci. Remote Sens., vol. 45, no. 12, pp , Dec [24] P. Zhong and R. Wang, Learning conditional random fields for classification of hyperspectral images, IEEE Trans. Image Process., vol. 19, no. 7, pp , Jul [25] G. Y. Zhang and X. P. Jia, Simplified conditional random fields with class boundary constraint for spectral spatial based remote sensing image classification, IEEE Geosci. Remote Sens. Lett., vol. 9, no. 5, pp , Sep [26] P. Zhong and R. Wang, Learning sparse CRFs for feature selection and classification of hyperspectral imagery, IEEE Trans. Geosci. Remote Sens., vol. 46, no. 12, pp , Dec [27] P. Zhong and R. Wang, Modeling and classifying hyperspectral imagery by CRFs with sparse higher order potentials, IEEE Trans. Geosci. Remote Sens., vol. 49, no. 2, pp , Feb [28] G. Verdier and A. Ferreira, Adaptive Mahalanobis distance and k-nearest neighbor rule for fault detection in semiconductor manufacturing, IEEE Trans. Semicond. Manuf., vol. 24, no. 1, pp , Feb [29] D. Ververidis and C. Kotropoulos, Information loss of the Mahalanobis distance in high dimensions: Application to feature selection, IEEE Trans. Pattern Anal. Mach. Intell., vol. 31, no. 12, pp , Dec [30] H. Liu, J. Yih, D. Wu, and C. Chen, Fuzzy C-mean algorithm based on Mahalanobis distance and new separable criterion, in Proc. 6th Int. Conf. Machine Learning and Cybernetics, Aug. 2007, pp [31] S. Z. Li, Markov Random Field Modeling in Image Analysis, 3rd ed., New York, NY, USA: Springer-Verlag, [32] J. Modestino and J. Zhang, A Markov random field model-based approach to image interpretation, IEEE Trans. Pattern Anal. Mach. Intell., vol. 14, no. 6, pp , Jun

17 1330 IEEE JOURNAL OF SELECTED TOPICS IN APPLIED EARTH OBSERVATIONS AND REMOTE SENSING, VOL. 7, NO. 4, APRIL 2014 [33] J. Shotton, J. Winn, C. Rother, and A. Criminisi, Textonboost: Joint appearance, shape and context modeling for multi-class object recognition and segmentation, in Proc. Euro. Conf. Computer Vision, 2006, pp [34] P. Zhong and R. Wang, Using combination of statistical models and multilevel structural information for detecting urban areas from a single gray-level image, IEEE Trans. Geosci. Remote Sens., vol. 45, no. 5, pp , May [35] J. Lafferty, A. McCallum, and F. Pereira, Conditional random fields: Probabilistic models for segmenting and labeling sequence data, in Proc. Int. Conf., ML, 2001, pp [36] S. Kumar, Models for learning spatial interactions in natural images for context-based classification, Ph.D. thesis, Carnegie Mellon Univ., Pittsburgh, PA, USA, [37] Q. Cheng, P. K. Varshney, and M. K. Arora, Logistic regression for feature selection and soft classification of remote sensing data, IEEE Trans. Geosci. Remote Sens., vol. 3, no. 4, pp , Oct [38] V. N. Vapnik, Statistical Learning Theory, Hoboken, NJ, USA: Wiley, [39] L. Hermes, D. Frieauff, J. Puzicha, and J. M. Buhmann, Support vector machines for land usage classification in Landsat TM imagery, in Proc. IEEE Int. Geoscience Remote Sensing Symp., 1999, pp [40] C. Huang, L. S. Davis, and J. R. G. Townshend, An assessment of support vector machines for land cover classification, Int. J. Remote Sens., vol. 23, no. 4, pp , [41] R. E. Fan, P. H. Chen, and C. J. Lin, Working set selection using second order information for training SVM, J. Mach. Learn. Res., vol. 6, no. 6, pp , [42] N. Kato, M. Suzuki, S. Omachi, H. Aso, and Y. Nemoto, A handwritten character recognition system using directional element feature and asymmetric Mahalanobis distance, IEEE Trans. Pattern Anal. Mach. Intell., vol. 21, no. 3, pp , Mar [43] R. A. Fisher, On the probable error of a coefficient of correlation deduced from a small sample, Metron, vol. 1, pp. 3 32, [44] J. N. Sweet, The spectral similarity scale and its application to the classification of hyperspectral remote sensing data [C], in Proc. IEEE Workshop on Advances in Techniques for Analysis of Remotely Sensed Data, 2003, pp [45] C. Sutton and A. McCallum, Fast, piecewise training for discriminative finite-state and parsing models, Center for Intelligent Information Retrieval, Tech. Rep. IR-403, [46] C. H. Lee, R. Greiner, and M. Schmidt, Support vector random fields for spatial classification, Lect. Notes Comput. Sci., vol. 3721, pp , [47] C. Sutton and A. McCallum, Piecewise pseudolikelihood for efficient training of conditional random fields, in Proc. Int. Conf. Machine Learning, 2007, pp [48] H. Arora, N. Loefir, D. Forsyth, and N. Ahuja, Unsupervised segmentation of objects using efficient learning, in Proc. IEEE Conf. Computer Vision and Pattern Recognition, pp. 1 7, [49] J. Shawe-Taylor and N. Cristianini, Kernel Methods for Pattern Analysis, Cambridge, U.K.: Cambridge Univ. Press, [50] T. F. Wu, C. J. Lin, and R. C. Weng, Probability estimates for multi-class classification by pairwise coupling, J. Mach. Learn. Res., vol. 5, pp , [51] C. J. Lin and R. C. Weng, Simple probabilistic predictions for support vector regression, Dept. Comput. Sci., National Taiwan Univ., Tech. Rep., 2004 [Online]. Available: paperspsvrprob.pdf. [52] C. C. Chang and C. J. Lin, LIBSVM: A library for support vector machines, ACM T. Int. Syst. Technol. (TIST), vol. 2, no. 3, p. 27, [53] J. Shotton, J. Winn, C. Rother, and A. Criminisi, Textonboost: Joint appearance, shape and context modeling for multi-class object recognition and segmentation, in Proc. Eur. Conf. Computer Vision, 2006, pp [54] P. Wegner, The object-oriented classification paradigm, Publish in: Research Directions in Object-Oriented Programming, pp , [55] Y. Tarabalka and J. C. Tilton, Best merge region growing with integrated probabilistic classification for hyperspectral imagery, in Proc. IEEE Geosci. Remote Sens. Symp., Jul. 2011, pp [56] J. Li, J. M. Bioucas-Dias, and A. Plaza, Hyperspectral image segmentation using a new Bayesian approach with active learning, IEEE Trans. Geosci. Remote Sens., vol. 49, no. 10, pp , Oct [57] S. B. Serpico and G. Moser, Weight parameter optimization by the Ho Kashyap algorithm in MRF models for supervised image classification, IEEE Trans. Geosci. Remote Sens., vol. 44, no. 12, pp , Dec [58] J. A. Richards and X. P. Jia, Remote Sensing Digital Image Analysis: An Introduction, 3rd ed., Berlin, Germany: Springer-Verlag, [59] Q. McNemar, Note on the sampling error of the difference between correlated proportions or percentages, Psychometrika, vol. 12, no. 2, pp , Jun [60] Y. Zhong, B. Zhao, and L. Zhang, Multiagent object-based classifier for high spatial resolution imagery, IEEE Trans. Geosci. Remote Sens., vol. 52, no. 2, pp , Feb Yanfei Zhong (M 11) received the B.S. degree in information engineering and the Ph.D. degree in photogrammetry and remote sensing from Wuhan University, Wuhan, China, in 2002 and 2007, respectively. He has been with the State Key Laboratory of Information Engineering in Surveying, Mapping, and Remote Sensing, Wuhan University, Wuhan, China, since 2007, and is currently a Professor. His research interests include multi- and hyperspectral remote sensing image processing, artificial intelligence, and pattern recognition. Dr. Zhong has published more than 60 research papers, including more than 20 peer-reviewed articles in international journals such as IEEE TRANSACTIONS ON GEOSCIENCE AND REMOTE SENSING, IEEE TRANSACTIONS ON SYSTEMS, MAN, AND CYBERNETICS PART B, AND PATTERN RECOGNITION. He was the recipient of the National Excellent Doctoral Dissertation Award of China (2009) and New Century Excellent Talents in University of China (2009). He was a Referee of IEEE TRANSACTIONS ON SYSTEMS, MAN, AND CYBERNETICS PART B, IEEE JOURNAL OF SELECTED TOPICS IN APPLIED EARTH OBSERVATIONS AND REMOTE SENSING, IEEE GEOSCIENCE AND REMOTE SENSING LETTERS, AND PATTERN RECOGNITION. Xuemei Lin received the B.S. degree in surveying from Center South University, Changsha, China, in 2011 and the M.S. degree in surveying from Wuhan University, Wuhan, China, in Her research interests include high spatial resolution remote sensing image classification, change detection, and random field algorithms. Liangpei Zhang (M 06 SM 08) received the B.S. degree in physics from Hunan Normal University, Changsha, China, in 1982, the M.S. degree in optics from the Xi an Institute of Optics and Precision Mechanics of Chinese Academy of Sciences, Xi an, China, in 1988, and the Ph.D. degree in photogrammetry and remote sensing from Wuhan University, Wuhan, China, in He is currently with the State Key Laboratory of Information Engineering in Surveying, Mapping, and Remote Sensing, Wuhan University, Wuhan, China, as the head of the Remote Sensing Division. He is also a Chang-Jiang Scholar Chair Professor appointed by the Ministry of Education, China. He is currently the Principal Scientist for the China State Key Basic Research Project ( ) appointed by the Ministry of National Science and Technology of China to lead the remotesensingprogram in China. Heis an Executive Member(Board of Governor) of the China National Committee of International Geosphere Biosphere Programme. His research interests include hyperspectral remote sensing, high resolution remote sensing, image processing, and artificial intelligence. Dr. Zhang is a Fellow of the Institution of Electrical Engineers, an executive Member for the China Society of Image and Graphics, and others. He regularly serves as a Co-chair of the series SPIE Conferences on Multispectral Image Processing and Pattern Recognition, Conference on Asia Remote Sensing, and many other conferences. He edits several conference proceedings, issues, and the Geoinformatics Symposiums. He also serves as an Associate Editor of International Journal of Ambient Computing and Intelligence, International Journal of Image and Graphics, International Journal of Digital Multimedia Broadcasting, Journal of Geo-spatial Information Science, and the Journal of Remote Sensing. He has more than 260 research papers and is the holder of five patents. He is currently serving as an Associate Editor for the IEEE TRANSAC- TIONS ON GEOSCIENCE AND REMOTE SENSING.