Contextual High-Resolution Image Classification by Markovian Data Fusion, Adaptive Texture Extraction, and Multiscale Segmentation

Transcription

1 IGARSS-2011 Vancouver, Canada, July 24-29, 29, 2011 Contextual High-Resolution Image Classification by Markovian Data Fusion, Adaptive Texture Extraction, and Multiscale Segmentation Gabriele Moser Sebastiano B. Serpico

2 Outline 2 Introduction Contextual very high-resolution image classification The proposed method Key ideas and overview of the method Adaptive semivariogram extraction Region-based multiscale MRF Segmentation, estimation, and optimization Experimental results Data set and experimental set-up Results evaluation and comparisons Conclusion

3 Outline 3 Introduction Contextual high-resolution image classification The proposed method Key ideas and overview of the method Adaptive semivariogram extraction Region-based multiscale MRF Segmentation, estimation, and optimization Experimental results Data set and experimental set-up Results evaluation and comparisons Conclusion

4 Introduction 4 QuickBird, panchromatic, 1 m Very high-resolution (VHR) optical remotesensing images: Very interesting in land-use / land-cover mapping, especially in urban and built-up area analysis m resolution available thanks to current (e.g., IKONOS, QuickBird, WorldView-2, GeoEye- 1) and forthcoming (e.g., Pleiades) missions. Increased need to model spatial information due to limited spectral information (few spectral channels) A novel contextual classification method is proposed for HR optical images, based on: Adaptive texture extraction by semivariogram; Multiscale segmentation; Markov random fields for spatial information fusion.

5 Outline 5 Introduction Contextual high-resolution image classification The proposed method Key ideas and overview of the method Adaptive semivariogram extraction Region-based multiscale MRF Segmentation, estimation, and optimization Experimental results Data set and experimental set-up Results evaluation and comparisons Conclusion

6 The Proposed Approach 6 How to incorporate spatial information? Region-based approaches: usually effective for classes with geometrical structures (e.g., urban). Texture analysis: effective for natural and artificial textured classes, especially for images with few spectral channels; Texture analysis: often introduce artifacts at the object borders (due to moving-window processing). Key-ideas Integrating segmentation and texture information by incorporating semivariogram features into a previous multiscale region-based MRF model. Applying spatially adaptive texture extraction to prevent border artifacts.

7 Overview of the Proposed Method 7 Generate a preliminary classification map L 0 by applying a previous region-based MRF classifier [5] to the input image X. Extract a set F t of texture features by applying to X the proposed adaptive semivariogram method, based on the class borders in the current map L t. Stack together X and F t and generate a set S t of Q segmentation maps, each related to a different spatial scale, by applying a scale-dependent segmentation method to (X, F t ). Initialization phase Iterative phase t = t + 1 Generate the updated map L t + 1 by applying a previous regionbased MRF classifier [5] to the multiscale segmentation S t. yes convergence? no STOP

8 Outline 8 Introduction Contextual high-resolution image classification The proposed method Key ideas and overview of the method Adaptive semivariogram extraction Region-based multiscale MRF Segmentation, estimation, and optimization Experimental results Data set and experimental set-up Results evaluation and comparisons Conclusion

9 Adaptive Semivariogram Extraction 9 2 { 2 } 1 γi( h) = E ( xi x j ) i j = h ( h 0) 2 t t 2 δ( li, l j ) xi x j 2 t 1 j R ihw γˆ i( h w, L ) = t t 2 δ( li, l j ) j R ihw w Rihw = j : i j = h, i j < 1 2 i w w window Current map L t : colors denote class labels; yellow borders denote pixels used to estimate semivariogram Semivariogram Local 2 nd order statistics γ i (h) for a single-channel image. Multispectral extension by (possibly weighted) Euclidean distance. Usually estimated with a w w moving window. Proposed adaptive estimation Use, for each pixel i, the pixels that both belong to the related w w moving window and share the same label as i in the current map. 1-norm on the pixel grid for convenience.

10 Outline 10 Introduction Contextual high-resolution image classification The proposed method Key ideas and overview of the method Adaptive semivariogram extraction Region-based multiscale MRF Segmentation, estimation, and optimization Experimental results Data set and experimental set-up Results evaluation and comparisons Conclusion

11 Markov Random Fields 11 MRF model for the spatial context Representation of the statistical interactions between the pixel labels in an image by using only local relationships: ( lil j, ) = ( lil j, ) P j i P j i Labels in the neighborhood (here, 3 3) i MRF-based classification Minimization of a (posterior) energy function U( ), thanks to the Hammersley-Clifford theorem. Here: Q t q iq li 0 lil j i q= 1 i j t U( L S ) = α ln P( s ) α δ(, ) Pixelwise probability mass function (PMF) of the segment labels in the segmentation map at each scale and each iteration, conditioned to each class

12 Outline 12 Introduction Contextual high-resolution image classification The proposed method Key ideas and overview of the method Adaptive semivariogram extraction Region-based multiscale MRF Segmentation, estimation, and optimization Experimental results Data set and experimental set-up Results evaluation and comparisons Conclusion

13 Segmentation and PMF Estimation 13 Felzenszwalb & Huttenlocherm segmentation method Graph-based region-growing method depending on a scale parameter. Segmentation at different scales by varying the scale parameter. Class-conditional PMF estimation Extension of a previous method that computes relativefrequency estimate [5], based, at each t-th iteration, on a preliminary intermediate map M t obtained classifying (X, F t ). To generate M t from the HR stacked image (X, F t ), a nonparametric contextual method is desirable. Here, a recent (non-region-based) method that combines MRFs and support vector machines (SVMs) is used [9].

14 Parameter Estimation and Energy Minimization Weight parameters α in the MRF Extension of a recent method based on the Ho-Kashyap algorithm. 14 Energy minimization: iterated conditional mode (ICM) Initialized with the intermediate preliminary map M t. Converges to a local energy minimum. Usually good tradeoff between accuracy and processing time.

15 Outline 15 Introduction Contextual high-resolution image classification The proposed method Key ideas and overview of the method Adaptive semivariogram extraction Region-based multiscale MRF Segmentation, estimation, and optimization Experimental results Data set and experimental set-up Results evaluation and comparisons Conclusion

16 Data Set and Experimental Set-up 16 Data set Itaipu (Brazil/Paraguay), IKONOS, 3 channels, pixels Training map RGB false color Test map Set-up Q = 5 scales, 7 7 window (w = 7). Preliminary experiments suggested limited sensitivty of the accuracy to (w, Q) for 5 w 31 e 2 Q 5. SVM applied with Gaussian kernel. Kernel and regularization parameters in the SVM optimized by a recent method based on the numerical minimization of the span bound. urban herbaceous rangeland schrub and brush rangeland forest land barren land built-up (non-urban) water

17 Outline 17 Introduction Contextual high-resolution image classification The proposed method Key ideas and overview of the method Adaptive semivariogram extraction Region-based multiscale MRF Segmentation, estimation, and optimization Experimental results Data set and experimental set-up Results evaluation and comparisons Conclusion

18 Classification Accuracies 18 Very high test-set accuracies by the proposed method. Very similar test-set accuracies also by the previous method in [5] (multiscale segmentation and MRFs, no textures) and by an SVM applied to spectral and standard (non-adaptive) semivariogram features. Much lower test-set accuracies for an SVM applied only to the spectral channels (expected result: no spatial information used). But... test samples located only inside homogeneous areas and not at the class borders (usual in remote sensing).

19 Classification Maps: Previous Methods 19 RGB false color Method in [5] Relevant visual differences between the benchmark considered methods. Errors for herbaceous (textured class; e.g., white circle), but no border artifacts by the method in [5]. Correct classification of herbaceous, but irregular behavior at the class borders by SVM with standard semivariogram. SVM, spectral + semivariogram

20 Classification Maps: Proposed Method 20 Proposed method Method in [5] Correct classification of herbaceous no border artifacts by the proposed method. This suggests: effectiveness of the proposed adaptive semivariogram capability of the proposed classifier to fuse multiscale segmentation and texture SVM, spectral + semivariogram

21 Classification Maps: Further Comments 21 RGB false color Proposed method Visually noisy map by the SVM applied only to the spectral bands (as expected). Spatially regular result, but no appreciable oversmoothing by the proposed method. Time < 50 minutes for all considered methods on a 2.33-GHz, 4-GB RAM pc (usually acceptable time for land-cover mapping). SVM, only spectral

22 Outline 22 Introduction Contextual high-resolution image classification The proposed method Key ideas and overview of the method Adaptive semivariogram extraction Region-based multiscale MRF Segmentation, estimation, and optimization Experimental results Data set and experimental set-up Results evaluation and comparisons Conclusion

23 Conclusion 23 Novel MRF-based VHR image classifier combining the multiscale segmentation and texture to model spatial information. Very accurate results for both textured and geometricallystructured classes. No border artifacts, thanks to adaptive semivariogram. Improvement in class discrimination and/or border precision, compared to previous methods. Possible future generalizations Integrating edge information (e.g., line processes). Approaching global energy minimization (e.g., graph-cuts). Comparisons with other methods for VHR image classification Experiments with other VHR data sets.

24 References S. Li, Markov random field modeling in image analysis, Springer, X. Descombes and J. Zerubia, Marked point process in image analysis, IEEE Signal Processing Magazine, vol. 19, no. 5, pp , Q. Chen and P. Gong, Automatic variogram parameter extraction for textural classification of the panchromatic ikonos imagery, IEEE Trans. Geosci. Remote Sensing, vol. 42, no. 4, pp , M. De Martino, F. Causa, and S. B. Serpico, Classification of optical high-resolution images in urban environment using spectral and textural information, in Proc. of IGARSS-2003, Toulouse, France, 2003, vol. 1, pp G. Moser and S. B. Serpico, Classification of high-resolution images based on MRF fusion and multiscale segmentation, in Proc. of IGARSS-2008, Boston, USA, 2008, vol. II, pp A. H. S. Solberg, T. Taxt, and A. K. Jain, A Markov random field model for classification of multisource satellite imagery, IEEE Trans. Geosci. Remote Sensing, vol. 34, no. 1, pp , P. Li, T. Cheng, G. Moser, S. B. Serpico, and D. Ma, Multitemporal change detection by spectral and multivariate texture information, in Proc. of IGARSS-2007, Barcelona (Spain), July 2007, 2007, pp P. F. Felzenszwalb and D. Huttenlocherm, Efficient graph-based image segmentation, Int. J. Comp. Vis., vol. 59, pp , G. Moser and S. B. Serpico, Contextual remote-sensing image classification by support vector machines and markov random fields, in Proc. of IGARSS-2010, Honolulu (USA), July 2010, 2010, pp S. B. Serpico and G. Moser, Weight parameter optimization by the Ho-Kashyap algorithm in MRF models for supervised image classification, IEEE Trans. Geosci. Remote Sensing, vol. 44, pp , 2006.