Level Set Hyperspectral Segmentation:

Transcription

1 Level Set Hyperspectral Segmentation: Near-Optimal Speed Functions using Best Band Analysis and Scaled Spectral Angle Mapper John E. Ball, student member, IEEE, and L. M. Bruce, senior member, IEEE Department of Electrical and Computer Engineering, Mississippi State University, Mississippi State, MS, USA Abstract This paper presents a semi-automated supervised level set hyperspectral image segmentation algorithm. The proposed method uses near-optimal speed functions (which control the level set segmentation) that are composed of a spectral similarity term and a stopping term. The spectral similarity term is used to compare pixels to class training signatures and is based on an optimized best bands analysis (BBA) procedure developed previously by the authors [2]. The stopping term is created from a new BBA algorithm, which uses a modified version of the spectral angle mapper (SAM) called the scaled SAM (SSAM). The algorithm is validated with a HYDICE hyperspectral image of the Washington, D.C. Mall. The results of the proposed method are compared to previous results by the authors and show the efficacy of the new algorithm. The contributions of the paper include a nearly-optimal set of speed functions for hyperspectral level set analysis and an automated BBA algorithm based on the SSAM metric for creating the level set stopping term. Keywords- Best Bands Analysis, Classification, HYDICE, Hyperspectral, Level Set, Optimization, Remote Sensing, Spectral Angle Mapper, SAM, Segmentation, Supervised Classification. I. INTRODUCTION Hyperspectral image segmentation is an important problem in remote sensing. Accurate segmentation has diverse applications such as military target detection, precision agriculture, invasive species detection, etc. In a previous study, the authors demonstrated that level sets could be effectively used to segment hyperspectral remotely sensed images. Speed functions were created based on a simple spectral similarity metric and hand-coded class discriminator functions. The overall accuracy was about 86% [1]. In a follow-up to this study, the authors examined optimizing the spectral similarity speed function based on a BBA approach and achieved overall accuracies around 94 to 95% [2]. In this study, we examine the effects of using the optimized spectral similarity speed terms in conjunction with nearoptimal discriminator terms (stopping terms) which are created using BBA utilizing a modified SAM method. This method examines contiguous subsets of bands to determine the set with the maximum class separability. II. CURRENT METHODS A. Dimensionality Reduction (Best Bands Analysis) Hyperspectral data typically have highly correlated bands [3], and for many processing algorithms, dimensionality reduction is required. For example, Lin and Bruce utilized projection pursuits to perform BBA with receiver operating characteristics (ROC) area under the curve (A Z ) and Bhattacharyya distance metrics used for the band grouping and projection selection, and obtained accuracies > 95 % using sequential parametric projection pursuits [4]. Riedmann and Milton used HYMAP and CASI data and provided an algorithm for band selection, and obtained above 97% accuracy using 2 to 12 bands [5]. Venkataraman et al. utilized localized discriminant bases for band grouping, along with multiclassifiers, when utilizing hyperspectral data to discriminate between Cogongrass, an invasive species, and other grasses. They achieved classification accuracies of 80% 90% [6]. B. Image Segmentation using Traditional Methods Some of the more popular hyperspectral segmentation methods are based on statistical approaches, such as Bayesian probabilities, Markov processes, and maximum likelihood (ML) techniques. These traditional methods do not typically use level set theory. The ML classification algorithm is the most common method for remote sensing image segmentation. ML uses estimates of the class mean and covariance to decide the most likely class membership. The method is generally well understood and implemented in many commercial remote sensing software packages such as ERDAS Imagine, IDL/ENVI, and Multispec. A Bayesian approach was used by Jackson and Landgrebe, which employed an adaptive Bayesian contextual classification procedure in which the joint prior probabilities of the classes of each pixel and its spatial neighbors are modeled by a Markov random field [7]. Using the DC image [1], [8], they achieved 96 97% overall accuracy with classes roof, road, path, trees, grass and shadows [7]. Neher and Srivastava used Fisher s linear discriminant analysis (FLDA) and principal component analysis (PCA) in the DC mall image for classes grass, trees, roads, and buildings [9]. Using nonparametric density estimates and Markov chain to estimate site classifications, along with FLDA, they achieved an overall accuracy of 98.3%.

2 C. Image Segmentation using Level Set Methods For 2D image segmentation, the level set boundary is the zero level set of an implicit function φ : r 2 r. The level set methodology tracks the motion of the zero level set boundaries according to forces acting normally to the zero level set curves. The level set equation for front propagation with a 2D speed function, F(x,y), acting normal to the level set curve, is given by the partial differential equation 0 t φ + F φ =, (1) where φ ( x, y) is the gradient of φ and. is magnitude [10]. There has been extensive research into level set based image segmentation of grayscale and color RGB images, and this methodology is beginning to be used in multispectral and hyperspectral image analysis. For example, Keaton and Brokish used level sets to segment roads in pan sharpened IKONOS multispectral images [11], Dell'Acqua, Gamba and Prevedini extracted and tracked moving clouds in GOES and Meteosat IR satellite image sequences [12], and Harper and Reilly used level sets to segment faces in RGB video [13]. Lee, Snyder and Wang used active contours to segment real and synthetic RGB images. Their method used the level set methodology and used a multivariate mixture density model to analyze dissimilarities between regions in the image [14]. Ball and Bruce used level sets and created an optimized speed term based on BBA. Several methods were compared to optimize a matrix in the speed term, including an appropriate identity matrix, the inverse of the covariance matrix of the training data, the inverse of the between class covariance matrix from FLDA, and a matrix composed from the FLDA weight coefficients. The overall accuracy was about 86% in [1] and increased to 94 to 95% [2]. III. HYPERSPECTRAL LEVEL SET SEGMENTATION This section discusses the authors previous methods for level set hyperspectral image segmentation [1], [2]. Each endmember has a separate speed function, which is composed of a spectral similarity and a class discriminator term. The spectral similarity term acts as a region growing term and is used to create the initial segmentation seed points. The spectral similarity terms are used to identify the regions that are the most spectrally similar to a given endmember. In [2], the authors presented optimized speed functions based on BBA. Table I shows the matrices used with methods 1 5 (assuming that b bands were selected). The spectral similarity term equations are given as equations (5) and (6) in [2]. The class discriminator terms were hand-coded stopping terms which were created based on experimentation [1],[2]. The discriminator functions are given by eq. (5) (10) in [1]. IV. PROPOSED LEVEL SET METHODOLOGY Before processing the training data using the algorithms in Figures 2 and 3, each class was preprocessed by sorting the training signatures by the sum of the DN for all bands in ascending order. Then N=7 signatures were taken from each class in order to represent class variation. The signatures were chosen by taking the sorted signatures and selecting N of them. For example, if there are 5 training signatures and N=3, then the algorithm would use the 1 st, 3 rd and 5 th sorted signatures. In the proposed method, the spectral similarity term is the same as methods 1 5 in [2]. The class discriminator term, which was hard-coded in [1] and [2], is now based on a SAM metric. This metric was chosen because of the wide variability of the data typical in a hyperspectral image, and SAM is unaffected by changes in magnitude. The SAM metric is often used in hyperspectral image analysis [15]. The SAM metric is used to compare a signature to a reference signature for similarity: signatures that are very similar will tend to have low SAM values, while signatures from different classes tend to have larger SAM values [16]. The SAM value for a pixel with spectral signature s and a reference signature r is given by SAM (,) sr = cos (( s r) s r ), (2) in radians, where x is the vector norm and the dot denotes the inner product [16]. The proposed algorithm uses a modified form of SAM along with BBA to create the speed function stopping term. Unlike the previous approach listed above, the proposed discriminator functions are determined automatically for each class based on the training signatures. Figures 2 and 3 show the algorithms used to create the new discriminator terms. Each class c is compared to all of the other classes. The algorithm determines an appropriate threshold for the SAM metric such that most of the training signatures are less than the threshold, while the majority of the other classes SAM values would be greater than the threshold. We then rescaled the SAM values by dividing by this threshold, and any pixels with rescaled values < 1.0 were labeled with that training class c. The thresholds were set at K=501, TFP_THRESH = 0.98, and FPF_THRESH = 0.05, where TPF (FPF) stands for true positive fraction and false positive fraction, respectively. Also, only sets of contiguous bands of length 8, 12,..., 28 were considered, because this subset gave good A Z values (all > 0.98). Finally, small islands of pixels (for all classes except shadows) are replaced by the majority of their close neighbors. This is a common post-processing step for many thresholding operations. Also note that the bands used for the ML method are all of the similarity bands shown in Table III. V. DATA The hyperspectral data consists of a 191-band raw DN HYDICE hyperspectral image of the Washington D.C. Mall area in the U.S.A. This is the DC_1 image in [1] and the image used in [2]. For more information on the image, the interested reader may refer to [1], [2], [8], and [17]. The image is shown in false color in Figure 1(a). ERDAS Imagine was used to create a set of training and testing samples. Figures 1(b) and (c) show the image as a grayscale image overlaid with the training and testing data, respectively. Table I lists five methods for constructing the spectral similarity term. Table II lists 11 test cases used for segmentation. For instance, test case 1A uses

3 the spectral similarity term method 1 listed in Table I, and the previous (original) class discriminators from [1]. In contrast, test case 1B uses the same spectral similarity term but uses the proposed discriminator term. TABLE I. SPECTRAL SIMILARITY TERM Σ FROM [2]. Method Spectral Similarity Term Σ (matrix size) 1 Identity matrix (b x b) 2 Inverse of training data covariance matrix (b x b) 3 Inverse of FLDA between class matrix (b x b) 4 FLDA weight coefficients in diagonal entries (b x b) 5 Identity matrix using bands 62,53,36 (3 x 3) TABLE II. Similarity Method [1] / Discriminator TEST CASES. Similarity Method [1] / Discriminator 1A 1 / Previous 4A 4 / Previous 1B 1 / Proposed 4B 4 / Proposed 2A 2 / Previous 5A 5 / Previous 2B 2 / Proposed 5B 5 / Proposed 3A 3 / Previous 3B 3 / Proposed ML ML VI. RESULTS Table III shows the best bands selected for the similarity and discriminator terms for each class. Table IV shows the classification results in the form of user s, producer s, and overall accuracies. User s accuracies measure commission errors and are a representation of the actual probability that a pixel is classified actually represents what is on the ground. The producer s accuracy measures omission errors and represents how well the training set was classified. The overall accuracy is a measure of the classification accuracy of all of the classes [18]. The results from Table IV show very good classification for buildings except for ML. Grass was best segmented by proposed methods 2B, 3B, and 4B. Paths were segmented best by the proposed methods (1B 5B). Shadows were better segmented by the previous discriminator function. There was some confusion between shadows and trees, since some dark tree areas were marked as shadows. All methods were similar for segmenting trees and water. In Figure 1(c)-(i), notice that the small triangular grass areas in the B segmentations have many grass pixels mistaken for shadows. This area was used for training data. Using it for testing would have lowered the overall accuracies of the proposed methods. Figure 1 (j) and (k) show the SAM PMF (probability mass function) values (before scaling) for the class paths. Figure 1(k) shows the values of p, q, and τ from the algorithm in Figure 2 for the class paths. VII. CONCLUSIONS AND FUTURE WORK The best overall results were methods 1B, 3B, and 4B. In comparing the results to the methods of [7] and [9], the overall results are very similar or slightly better. Overall, the proposed methods 1B-5B had very similar results, indicating that the discriminator term is very important in level set segmentation and may be more significant than the similarity term, Σ. The hard-coded discriminator term for shadows was about 6% better than the new method. Finally, even though only seven training signatures were used for each class, the discriminator terms were very effective. It is well known that the ML classification methods require inversion of the covariance matrix, which can become illconditioned if the number of training samples is inadequate. Also, ML is subject to the Hughes phenomenon [19], which means as the data dimensionality increases, the classification accuracy will generally decrease. The SSAM based methods presented here use a small number of the training signatures. It would be worthwhile to examine comparisons of ML and the proposed algorithm with reduced sets of training data. Also, the method of choosing the SSAM reference signatures could be modified to choose the best subset of signatures, which would be an optimization problem. The approach used in [20] would be a good starting point. Finally, we would like to perform validation of the new procedure on more images, especially images created from other sensors. ACKNOWLEDGMENT The authors acknowledge the National Science Foundation for providing J. Ball s graduate research fellowship. REFERENCES [1] Ball, J. E. and Bruce, L. M., Level set segmentation of remotely sensed hyperspectral images. Proceedings of the 2005 IEEE International Geoscience and Remote Sensing Symposium, vol. 8, pp , July [2] Ball, J. E. and Bruce, L.M., Accuracy analysis of hyperspectral imagery classification using level sets. Proceedings of the 2006 ASPRS Annual Conference, 2006 (in press). [3] Lillesand, T.M., Kiefer, R.W., and Chipman, J.W., Remote Sensing and Image Interpretation, Wiley, [4] Lin, H.-D. and Bruce, L.M., Projection pursuits for dimensionality reduction of hyperspectral signals in target recognition applications. Proceedings of the 2004 IEEE International Geoscience and Remote Sensing Symposium, vol. 2, pp [5] Riedmann, M., and Milton, E.J., Supervised band selection for optimal use of data from airborne hyperspectral sensors. Proceedings of the 2003 IEEE International Geoscience and Remote Sensing Symposium, vol. 3, pp , [6] Venkataraman, S., Bruce, L.M., Cheriyadat, A., and Mathur, A., Hyperspectral dimensionality reduction via localized discriminant bases. Proceedings of the 2005 IEEE International Geoscience and Remote Sensing Symposium, vol. 2, pp , [7] Jackson, Q., and Landgrebe, D.A., Adaptive Bayesian contextual classification based on Markov random fields. IEEE Transactions on Geoscience and Remote Sensing, vol. 40, issue 11, pp , Nov [8] -, DC Mall image and band specifications for the HYDICE Washington D.C. Mall Image provided on the compact disk with Signal theory methods in multispectral remote sensing. available: ml. [9] Neher, R. and Srivastava, A. A Bayesian MRF framework for labeling terrain using hyperspectral imaging. IEEE Transactions on Geoscience and Remote Sensing, vol. 43, issue 6, pp , June [10] Sethian, J., Level set methods and fast marching methods, 2nd ed., Cambridge University Press, 1999.

4 [11] Keaton, T. and Brokish, J., A level set method for the extraction of roads from multispectral imagery. Proceedings of the Applied Imagery Pattern Recognition Workshop, pp , Oct [12] Dell'Acqua, F., Gamba, P. and Prevedini, P., Level-set based extraction and tracking of meteorological objects in satellite images. Proceedings of the Geoscience and Remote Sensing Symposium, Vol. 2, July 2000, pp [13] Harper, P, and Reilly, R.B., Color based video segmentation using level sets. Proceedings of the International Conference on Image Processing, vol. 3, pp , Sept [14] Lee, C.P., Snyder, W., and Wang, C., Supervised Multispectral Image Segmentation using Active Contours. Proceedings of the 2005 IEEE International Conference on Robotics and Automation, vol. 1, pp , [15] Schowengerdt, R.A., Remote Sensing: Models and Methods for Image Processing, 2nd ed., Academic Press, [16] Kruse, F.A, Lefkoff, A.B., Boardman, J.W., Heidebrecht, K.B., Shapiro, A.T., Barloon, P.J., and Goetz, A.F.H., The Spectral Image Processing System (SIPS) Interactive Visualization and Analysis of Imaging Spectrometer Data. Remote Sensing of Environment, Special issue on AVIRIS, v. 44, pp , May-June [17] Landgrebe, D.A., Signal theory methods in multispectral remote sensing, Wiley, [18] Congalton, R.G., Assessing the accuracy of remotely sensed data: principles and practices, CRC Press, [19] Duda, R.O., Hart, P. and Stork, D., Pattern Classification, 2nd ed., Wiley Interscience, N.Y., [20] Keshava, N., Best bands selection for detection in hyperspectral processing. Proceedings of the 2001 IEEE International Conference on Acoustics, Speech, and Signal Processing (ICASSP '01), vol. 5, pp , May [21] Fawcett, T., ROC Graphs: Notes and Practical Considerations for Data Mining Researchers. HP Laboratories Tech report HPL , Palo Alto, CA, USA. Available: pdf. TABLE III. BANDS SELECTED FROM SIMILARITY AND STOPPING TERM BBA Class Similarity Term Bands Discriminator Term Bands Class Similarity Term Bands Discriminator Term Bands Building 1, Shadow 56,113,114, Grass Trees 138,139,140, Paths 1, Water 76,77, (a) False Color Image (b) Training Pixels (c) ing Pixels (d) ML Segmentation (e) 1B Segmentation (f) 2B Segmentation (g) 3B Segmentation (h) 4B Segmentation (i) 5B Segmentation Legend for (b)-(i) (j) Paths SAM (k) Paths TPF, FPF, τ Figure 1. Washington D.C. Mall hyperspectral images. (a) Original image shown in false color using bands 60, 27 and 17 for red, green and blue, respectively. The Lincoln memorial is in the center of the image and the reflecting pool is below the memorial (b) Training pixel locations, coded by class. (c) ing pixel locations, coded by class. (d) ML Segmentation. (e)-(i) Segmentation results for methods 1B 5B. (j) SAM PMF (before scaling) for the class paths training data. Paths SAM values are green and all other classes are shown in red. The threshold τ is (k) Corresponding TPF and FPF for the unscaled SAM values from (j) for class paths training data. The values of p, q, and τ from the algorithm in Figure 2 are shown in dashed green, dashed red and solid blue, respectively.

5 For c C Set sall sbest () c sbest _ OTHER () c ALL Calculate the TPF and FPF for all points in Ω using methods in [21]. Set p (n) to the minimum (maximum) value of TPF (FPF) such that TPF TPF_THR (FPF FPF_THR) c p n τ c = p = and set Ω to the set of K linearly spaced points in min ( s ), max ( s ) If p < n, then set τ ( ) = ( + )/2, else set ( ) Figure 2. Scaled spectral angle mapper (SSAM) Threshold Selection Algorithm. TPF is true positive fraction and FPF in false positive fraction [21]. ALL Set AZ _ MAX() c = 0for c C, where C is the set of all classes. For m L, where L = {8,12,16, 20,24, 28} For each set S of contiguous bands of length L For c C Set c = C {} c. This is the set of classes that are not class c. For n N Set r to the SAM reference signature for class c. Set p (q) to the set of training samples for class c (all other classes) for band set S. Set sc = SAM ( p, r), sq = SAM ( q, r), and AZ = ROC( sc, sq). If AZ > AZ_ MAX() c, then set AZ _ MAX () c = AZ, SBEST ( c ) = S, sbest ( c) = sc and sbest _ OTHER () c = sq. Run the algorithm from Figure 2 for each class to determine the thresholds. For c C Set all pixels in D c, the discriminator image for class c, to zero. Set the SAM values for class c to their value divided by τ ( c). This is the Scaled SAM (SSAM) value. Set pixel (x,y) in D c to 1 if the SSAM value for that pixel is < 1.0. If the class is not shadows, then set all islands of pixels with area < 7 pixels to the majority of the vicinal pixels within a radius of two pixels of the border. Figure 3. Proposed discriminator term algorithm using the BBA with ROC A Z as the metric. TABLE IV. USER S, PRODUCER S AND OVERALL ACCURACIES Accuracies in %. (Class Legend: B = Buildings, G = Grass, P = Paths, S = Shadow, T = Trees, W = Water) User s Producer s Overall B G P S T W B G P S T W 1A B A B A B A B A B ML