Supervised classification from high resolution optical and SAR images for natural disaster management

Transcription

1 Imaging in Geospatial Applications September 23-26, 2013 Supervised classification from high resolution optical and SAR images for natural disaster management Presented by Josiane Zerubia¹ In collaboration with A. Voisin¹, V. Krylov¹, I. Hedhli¹ ², G. Moser² and S. B. Serpico² 1 2

2 Introduction 2 Objectives Supervised image classification General and sufficiently robust to different types of images. Key points Focus on (single-pol) radar (SAR) imagery extension to multi-sensor data (COSMO-SkyMed (CSK)/ GeoEye or CSK/Pleiades). General applications Global detection of urban areas, that are critical w.r.t. populations (risk management). Infrastructures mapping. Mapping the water before any disaster, or after a flooding. Land-cover or land-use maps.

3 HAITI : earthquake monitoring (2010) 3 Optical imagery SAR imagery Port au Prince (GeoEye) GeoEye Here considered as an additional information to SAR ( 0.5 m) Port au Prince (CSK) ASI All-weather conditions. SAR amplitude. SpotLight ( 1m), StripMap ( 2.5m), PingPong (10m). Challenge: Speckle noise.

4 Italy : piemonte flooding (2008) 4 Racconigi Lombriasco Cavallermaggiore Racconigi (CSK) StripMap ( 5m) ASI Lombriasco (CSK) StripMap ( 5m) ASI Cavallermaggiore (CSK) StripMap ( 5m) ASI

5 Supervised classifiers 5 Proposed methods 2 supervised contextual classifiers based on Markov random field (MRF) : Single scale MRF with textural features Hierarchical MRF integrating a prior update Contributions of A.Voisin (PhD, ), V. Krylov (Post-Doc, ) 2 supervised contextual classifiers : Shared learning: statistical modeling of the input images, by using adapted finite mixtures and d-variate copulas. Integration of the statistics in Markovian models: MRF with textural features, and hierarchical MRF integrating a prior update [2] A. Voisin, G. Moser, V. Krylov, S. B. Serpico, and J.Zerubia, Classification of very high resolution SAR images of urban areas by dictionary-based mixture models, copulas and Markov random fields using textural features, in [Proc. of SPIE Symposium on Remote Sensing], 78300O (2010). [3] A. Voisin, V. Krylov, G. Moser, S. B. Serpico, and J.Zerubia, Multichannel hierarchical image classification using multivariate copulas, in [Proc. of IS&T/SPIE Electronic Imaging], 82960K (2012).

6 Contents Joint PDF Single-scale Markovian model Hierarchical Markovian model Experimental results Conclusion and Perspectives

7 7 Copulas : Overview The measurement in SAR and optical bands are very different from each other How to address problem of SAR + optical PDF modeling? 1 2 First, estimate the marginal class-conditional statistics of each SAR/optical channel separately via distinct finite mixtures. then, model the joint PDF through copulas. [4] A copula is a kind of distribution function. Copulas are used to describe the dependence between random variables. (Ali-Mikhail- Haq) In order to maximize flexibility in the proposed method, a dictionary approach is adopted for copula modeling [4] Nelsen, R. B., [An introduction to copulas], Springer, New York, 2nd ed. (2006).

8 PDF estimates PDF estimates Joint PDF 8 Joint PDF : Overview For each class m, and at each resolution (multiresolution case): Build a joint PDF. Grey Levels Grey Levels Note : the two bands may come from multiple polarizations (e.g., CSK pingpong) and/or multiple sensors (e.g., CSK stripmap and GeoEye).

9 9 Marginal PDF modeling mi are the i-th component of the mixture that models the class m

10 10 Advantages of finite mixtures Unimodal density does not accurately model SAR amplitude statistics given their heterogeneity. Each component (of the sum) may reflect the contribution of the different materials.

11 11 Marginal PDF modeling: Optical image case * * Stochastic Expectation Maximization [5] [5] G. Celeux, D. Chauveau, and J. Diebolt, Stochastic versions of the EM algorithm: an experimental study in the mixture case, Journal of Statistical Computation and Simulation, 55(4), (1996).

12 12 Experimental validation PDF estimates Histogram Urban area modeling PDF estimates Port au Prince ( GeoEye) (0.5 m) Histogram Water modeling

13 Marginal PDF modeling: SAR image case [6] 13 (Maximum likelihood) Modified SEM algorithm - Settings Initialization: Kmax = 6. Stop criterion: Maximum number of iterations reached [6] Krylov, V., Moser, G., Serpico, S. B., and Zerubia, J., Supervised high resolution dual polarization SAR image classification by finite mixtures and copulas, IEEE J-STSP, 5(3), (2011).

14 14 Experimental validation PDF estimates Histogram Urban area modeling PDF estimates Port au Prince (CSK) ASI Histogram SpotLight ( 1m), StripMap ( 2.5m) PingPong (10m). Water modeling

15 Joint PDF Single-scale Markovian model Markov random fields Textural features Experimental results Hierarchical Markovian model Experimental results Conclusion and Perspectives

16 16 General presentation Classification of multi-band, single-resolution acquisitions into M classes. Contextual information via MRF. Use the Bayesian formulation: X, Y random variables and ω ϵ [1, M] m

17 17 Prior probabilities

18 18 Optimization Need to maximize the posterior probability to find the labels. Here: minimization of the energy function: Tools: Modified Metropolis dynamics. [8] Graph-cuts. [7] [7] Berthod, M., Kato, Z., Yu, S., and Zerubia, J., Bayesian image classification using Markov random fields, Image and Vision Computing 14(4), (1996). [8] Kolmogorov, V. and Zabih,R.,» Computing Visual Correspondence with Occlusions using Graph Cuts» IEEE International Conference on Computer Vision (ICCV), July 2001.

19 19 Textural features [9] Problem: often limited class discriminability in single-pol SAR amplitudes. Aim: Improve the classification accuracy by integrating some additional information: textural features. Urban area discrimination. Well-adapted textural feature: Haralick s Grey Level Co-occurrence Matrix (GLCM) variance. [10] Principle : Moving w w window, and estimation of the value of the central pixel by using its neighborhood (calculation of spatial second-order statistics). [9] Voisin, A., Moser, G., Krylov, V., Serpico, S. B., and Zerubia, J., Classification of very high resolution SAR images of urban areas by dictionary-based mixture models, copulas and Markov random fields using textural features, in [Proc. of SPIE Symposium on Remote Sensing], 78300O (2010). [10] R. M. Haralick, K. Shanmugam and I. Dinstein, Textural features for image classification, IEEE TRans. Syst., Man, Cybern. 3(6), (1973).

20 Joint PDF Single-scale Markovian model Hierarchical Markovian model Model presentation Transition probabilities Prior probability 4 5 Experimental results Conclusion and Perspectives

21 21 Considered DATA Joint classification of coregistered mono-/multi-band, multi-resolution and/or multi-sensor (SAR, optical) acquisitions into M classes Hierarchical graph: use multi-resolution data. Flexible enough to take into account different kinds of statistics (multisensor data).

22 22 Notations The novelty in this work is in keeping the multi-resolution aspect by integrating the multisensor data in an explicit hierarchical Markovian model, based on a quad-tree structure. Hierarchical model (quad-tree) Quad-tree notations

23 23 General presentation: Hierarchical method [11] Classification: Estimate the labels X at the finest resolution (here, level 0) given all the observations. Quad-tree structure: causality that allows to use a non-iterative algorithm. MPM (marginal posterior mode) penalizes the errors according to their number and the scale at which they occur. [11] Laferte, J.-M., Perez, P., and Heitz, F., Discrete Markov modeling and inference on the quad-tree, IEEE Trans. Image Process. 9(3), (2000).

24 24 Wavelet transforms are used in the proposed method to generate multiscale features. A hierarchical MRF-based approach is defined to fuse the extracted multiscale information and generate the output classification map.

25 COSMO-SkyMed Image (Port-au-Prince Haïti, ASI, 2009) and texture attribute Joint PDF 25 Initial marginal posterior mode (MPM) scheme p * ( n) ( n) ( n) ( n) ( n) ( n) ( n) ( n) m 1m 1 m 2m m( y xm ) p1 m( y1 xm ). p2m( y2 xm ) ( n) ( n) y1 y2 finite Mixture finite Mixture ² C F ( y x ), F ( y ( n) 2 x ( n) m ) Prior Bottom-up pass Top down pass Observations Quad-Tree p( y x ( N ) ( N ) ) N x p( x (N ) ) xˆ p( x r y) r arg max p( x r y)... N y... p( y (1) x (1) ) 1 x p( x (1) ) p( x (1) s, x (1) s y (1) p( x s y d (1) d ( s) (1) ( s) ) ) p( x (1) s x (1) ˆs y) 1 y p( y (0) x (0) ) 0 x p( x (0) ) p( x (0) s, x (0) s y (0) p( x s y s (0) s (0) ) ) p( x (0) s (0) xˆs y) 0 y

26 26 Global scheme - prior update [12] [12] A. Voisin, V. Krylov, G. Moser, S.B. Serpico and J. Zerubia, Classification of Very High Resolution SAR Images of Urban Areas Using Copulas and Texture in a Hierarchical Markov Random Field Model, IEEE Geoscience and Remote Sensing Letters, 10(1), (2013).

27 27 Optimization Need to maximize the posterior probability at the coarsest scale (top-down pass). Tool: modified Metropolis dynamics. [13] Posterior probability Expression of the partial posterior probability (bottom-up pass): Thus, we need to define the prior probabilities, the transition probabilities. The likelihood has already been defined (joint PDF at each level of the tree). [13] Kato, Z. Zerubia, J. and Berthod, M., Satellite image classification using a modified Metropolis dynamics, Acoustics, Speech, and Signal Processing, ICASSP-92., 1992 IEEE International Conference on (Volume:3 ).

28 28 Transition probabilities [14] For all sites s ϵ S and all scales n ϵ [0; R 1], R corresponding to the root With θ >1/M n The transition probabilities determine the hierarchical MRF since they represent the causality of the statistical interactions between the different levels of the tree. Prior probabilities Prior probabilities at the coarsest level: Updated. Prior probability at level n in [0; R 1]: [14] Bouman, C. and Shapiro, M., A multiscale random field model for Bayesian image segmentation, IEEE Trans. Image Process. 3(2), (1994).

29 Joint PDF Single-scale Markovian model Hierarchical Markovian model Experimental results HAITI : earthquake monitoring Italy : piemonte flooding 5 Conclusion and Perspectives

30 30 Experimental settings Set of classes M fixed by the user (5). Θ = 0.85 (transition probability). n For native single-resolution images, the multi-resolution acquisitions are obtained by wavelet transform (Daubechies and Haar have been tested) on R = 2 levels. Multi-sensor acquisition of Port-au-Prince (Haiti) 2 coregistered images of the quay of Port-au-Prince (Haiti): A single-polarized COSMO-SkyMed SAR image ( ASI,2010), HH polarization, StripMap acquisition mode (2.5 m pixel spacing), geocoded, single-look of pixels. A pan-sharpened (1-band) GeoEye acquisition ( GeoEye, 2010, 0.65 m pixel spacing) of pixels. [15] [15] I. Daubechies, Orthonormal bases of compactly supported wavelets, Communications on Pure and Applied Mathematics, 41(7), (1988).

31 31 HAITI : earthquake monitoring Classification results, 5 classes: water, urban, vegetation, sand, and containers. Optical image ( GeoEye, 2010) (0.65 m) K-means Hierarchical MRF-based classifier Single-scale MRF Experiments were conducted on an Intel Xeon quad-core(2.40 GHz and 12-MB cache) 18-GB-RAM 64-bit Linux system. SAR image (320x400) and optical image (1280x1600).

32 32 Experimental settings set of classes M fixed by the user (3). Θ = 0.85 (transition probability). n For native single-resolution images, the multi-resolution acquisitions are obtained by wavelet transform (Daubechies and Haar have been tested) on R = 2 levels. Mono-sensor acquisition of Cavallermaggiore (Italy) A single-polarized COSMO-SkyMed SAR image ( ASI,2010), HH polarization, StripMap acquisition mode geocoded, single-look of pixels.

33 33 Italy : piemonte flooding Classification results, 3 classes: water, urban, vegetation CSK image ( ASI, 2010) (5 m) Hierarchical MRF Laferté method Single-scale MRF Experiments were conducted on an Intel Xeon quad-core(2.40 GHz and 12-MB cache) 18-GB-RAM 64-bit Linux system.

34 Joint PDF Single-scale Markovian model Hierarchical Markovian model Experimental results Conclusion and Perspectives

35 35 Conclusion Urgent need of more data with ground truth on Haïti (both CSK and Pleiades) to validate the proposed methods. Satisfying classification results obtained by using these Markovian methods. (Smoothing effect of the MRF. ) Details provided by the hierarchical MRF. Selection of the best method according to the user. Perspectives Extension of the previous methods to : Change detection (2 dates) Satellite Image Time Series Analysis Extension of the copula dictionary

36 36 Acknowledgments We would like to thank : The French Defense Agency, CNES and INRIA for the partial financial support. The Italian Space Agency (ASI, Italy) for providing the COSMO-SkyMed images GeoEye Inc. and Google crisis response for providing the GeoEye images. Recent papers : Aurélie Voisin, Vladimir Krylov, Gabriele Moser, Sebastiano B. Serpico, Josiane Zerubia. Supervised Classification of Multi-sensor and Multi-resolution Remote Sensing Images with a Hierarchical Copula-based Approach. IEEE Transactions on Geoscience and Remote Sensing, IEEE, 2013 Aurélie Voisin, Vladimir Krylov, Gabriele Moser, Sebastiano B. Serpico, Josiane Zerubia. Classification of Very High Resolution SAR Images of Urban Areas Using Copulas and Texture in a Hierarchical Markov Random Field Model. IEEE Geoscience and Remote Sensing Letters, IEEE, 2013, 10 (1), pp

37 Thank you For your attention For more information : And previously

38 Stochastic Expectation Maximization algorithm

39 Dictionary-based SEM algorithm

40 Dictionary-based SEM algorithm The ML estimation process involved by the M-step turns out to be unfeasible for several above-mentioned SAR-specific PDFs Hence, we adopt in the M-step the method of log-cumulants (MoLC), which has been proved to be a feasible and effective estimation tool for all usual SAR parametric models. MoLC is based on the generalization of characteristic functions by using Mellin transforms instead of the usual Fourier transforms and expresses a parametric estimation problem as the solution of a set of (non-linear) equations relating the unknown parameters with logarithmic moments: m d E u H 1 i m s mi ln pi ( u i ) u du m ds u H 0 i m s 1 {ln } if 1 Var{ln } if 2 Mellin transform of the pdf of H i they can be estimated as sample moments

41 Copulas A copula is a bivariate cumulative distribution function (CDF) C whose marginals are uniform in [0, 1]. Thanks to Sklar theorem, for each pair (a, b) of (absolutely) continuous random variables with marginal PDFs f and g and CDFs F and G, there is a copula C such that the joint PDF is: 2 C( u, v) p( a, b) f ( a) g( b) c[ F( a), G( b)], where: c( u, v) uv Parametric copula estimation: for most copulas with one parameter a closed-form equation relates and Kendall s concordance-discordance measure: P{( a a )( b b ) 0} P{( a a )( b b ) 0} ( ) a sample estimate of can be obtained by rank statistics and an estimate of is derived by inverting = ( ).

42 Sklar s theorem Copulas : Sklar s theorem

43 Multivariate copulas dictionary In order to maximize flexibility in the proposed method, a dictionary approach is adopted for copula modeling, too: the copula related to each class-conditional joint PDF is drawn from a dictionary of five parametric copulas; parameter estimation (Kendall s tau) is separately performed for each class and each parametric copula in the dictionary; the optimal copula for each class is selected by a Pearson chisquare test of fitness.

44 Wavelet decomposition [16] A natural image processing model to generate a hierarchical decomposition is the wavelet transform The discrete 2-D wavelet transform is the decomposition of an image, generated by wavelet functions {Ψ LH, Ψ HL, Ψ HH } and by scaling functions Ψ LL The considered space includes the translation and the dilatation with parameters τ = {τ n } and a = a m, m, n Z We build a 2-D wavelet basis, for which the translation is linked to two parameters (horizontal and vertical descriptors, refered here as n and q). Thus, we can express the wavelet function basis as where B = {LH,HL,HH} and the scaling function basis as For a given scale J, the decomposition of the image for every couple (s, t) is : where the coefficients a n,q,j and d n,q,j are respectively the approximation and the detail coefficients. [16] S. G. Mallat. A theory for multiresolution signal decomposition: the wavelet representation. IEEE Trans. Patt. Anal. Mach. Intell., 11(7): , 1989.

45 Wavelet decomposition : Haar wavelet The Haar wavelet's mother wavelet function Ψ t can be described as Its scaling function φ(t) can be described as The Haar wavelet

46 Wavelet decomposition : Daubechies wavelet The Daubechies wavelets, are : A family of orthogonal wavelets defining a discrete wavelet transform. Characterized by a maximal number of vanishing moments for some given support. With each wavelet type of this class, there is a scaling function (called the father wavelet) which generates an orthogonal multiresolution analysis. The Daubechies wavelets are not defined in terms of the resulting scaling and wavelet functions; in fact, they are not possible to write down in closed form. (The graphs below are generated using the cascade algorithm) Daubechie 4 wavelet

47 Wavelet decomposition SAR image ( ASI) wavelet decomposition example using Daubechies wavelets.

48 Discussions about the ground truth Port au Prince (CSK) ASI

49 Discussions about the ground truth

50 Discussions about the ground truth

51 Confusion matrix for Cavallermaggiore CLASSIFICATION RESULTS Water Urban vegetation total Accuracy water % GROUND TRUTH Urban % vegetation % Overall accuracy 95.97%

52 K-nearest neighbors (K-NN) [16] Used to model the probability density functions. Integrated in a MRF model ( it is simply the single scale model, but instead of estimating Likelihood using finite mixture, it s estimated using KNN.) Supervised estimation of the probability of a given pixel by using a majority vote on the K-nearest (distance rule) known pixels. K estimated by cross validation. [16] G. Shakhnarovich, P. Indyk, T. Darrell, [Nearest-neighbor methods in learning and vision: theory and practice], MIT Press, (2005).