Advanced methods for the analysis of multispectral and multitemporal remote sensing images

Size: px

Start display at page:

Download "Advanced methods for the analysis of multispectral and multitemporal remote sensing images"

Blanche Preston
5 years ago
Views:

1 Remote Sensing Laboratory Dept. of Information Engineering and Computer Science University of Trento Via Sommarive, 9, I Povo, Trento, Italy Thesis dissertation on Advanced methods for the analysis of multispectral and multitemporal remote sensing images PhD (29 cycle): Massimo Zanetti Web: massimozanetti.altervista.org Web (RSLab): rslab-tech.disi.unitn.it Advisor: Lorenzo Bruzzone

2 Outline of the thesis 1. Introduction Motivations Contributions 2. Part I: Variational methods for image approximation 3. Part II: Statistical models for change detection in multispectral images 4. Conclusions and future activities 2

Motivations Earth Observation (EO) missions provide huge amount of data. Important missions provide open-access to multispectral (MS) imagery (e.g., Landsat-8, Sentinel-2).

3 Motivations Earth Observation (EO) missions provide huge amount of data. Important missions provide open-access to multispectral (MS) imagery (e.g., Landsat-8, Sentinel-2). Information needs to be extracted from images automatically and efficiently. Processing capability of modern machines is significantly increased w.r.t. the past. More chances for mathematical methods for image analysis to be applied on real images (not only remote sensing images). 3

4 Contributions Variational methods for image approximation. We propose efficient minimization techniques for variational functionals such as Mumford-Shah (MSh) and Blake-Zisserman (BZ). We extend their functional formulation to vector-valued inputs allowing the analysis of multi-band images and also curves in N-dimensional space. We develop a parallelizable domain-decomposition technique to address the minimization for large size images. Statistical models for change detection in multispectral images. We present a novel change detection thresholding approach based on the Rayleigh-Rice mixture. We introduce a novel compound multi-class model for the statistical description of the so-called difference image. We propose a statistical simplification approach for the estimation of class parameters based on a variational formulation. 4

5 Variational methods for image approximation

6 Variational methods: strengths and challenges Strengths Retrieve/extract useful information (smooth representations, edges and other geometrical features). Handle a large diversity of data: images at different geometrical resolutions, spectral bands, digital surface models (DSMs), curves. Challenges SoA techniques require high computational time. Vector-valued inputs require specific models. Image size is a bottleneck for standard algorithms. 6

7 Functional model Input Rectangular domain Ω R 2. Image g: Ω R + with g <. Unknowns A smooth function u: Ω R + Two indicator functions s, z: Ω [0,1]. Minimize (see [1]) F ε s, z, u = δ z 2 Hu 2 dx Ω + ξ ε (s 2 + o ε ) u 2 dx Ω + α β ε s 2 + s 1 2 dx Ω 4ε +β ε z 2 + z 1 2 dx 4ε Ω +μ u g 2 Ω dx [1] L. Ambrosio, L. Faina, and R. March. Variational approximation of a second order free discontinuity problem in computer vision, SIAM Journal on Mathematical Analysis, 32(6):1171{1197, This functional is a generalization of both the Mumford-Shah [2] and the Blake- Zisserman [3] models. [2] D. Mumford and J. Shah. Optimal approximations by piecewise smooth functions and associated variational problems. Communications on pure and applied mathematics, 42(5):577{685, [3] A. Blake and A. Zisserman. Visual reconstruction. MIT Press Series in Articial Intelligence. MIT Press, Cambridge, MA,

8 Functional model F ε s, z, u = δ z 2 Hu 2 dx Ω + ξ ε (s 2 + o ε ) u 2 dx Ω + α β ε s 2 + s 1 2 dx Ω 4ε +β ε z 2 + z 1 2 dx 4ε Ω +μ u g 2 Ω dx u is close to g and piecewise smooth s = 1 almost everywhere s = 0 for high norm u 2 s 2 avoids big oscill. of s z= 1 almost everywhere z= 0 for high norm Hu 2 z 2 avoids big oscill. of z g noisy grayscale image with 1 st and 2 nd order geometrical features u piecewise linear approximation of g s edge map (function discontinuity) z edge+crease map (gradient discontinuity) 8

9 Proposed algorithm for minimization Gauss-Seidel approach to minimization is inefficient s k+1 A s k s = b s z k+1 A z k z = b z u k+1 A u k u = b u We proposed an inexact approach based on block coordinate descent algorithm (BCDA). For any iteration k and for any v = s, z, u do: Compute search directions: d v k (shallow sol. of linear system); Compute optimal step-length: α k v = A v k v k b v dk v ; d v k T A v k d v k Update value: v k+1 = v k + α v k d v k ; k = k + 1. until a convergence condition is satisfied. M. Zanetti, V. Ruggiero, M. Miranda Jr., Numerical minimization of a second-order functional for image segmentation, Communications in Nonlinear Science and Numerical Simulation, Vol. 36, pp ,

1mt) 600 x 600 pixels publicly available at: http://sipi.usc.edu/database/databa se.php?

10 Experimental results: computational performance Datasets: airport aerial image 1024 x 1024 pixels barracks digital surface model (sp. res. 1mt) 600 x 600 pixels publicly available at: se.php?volume=misc#top. downloadable at pt/community/lidar/847/lidar/

11 Experimental results: computational performance Test: The proposed BCDA method is compared to GS in terms of computational performance. airport dataset method k inner iter. (d s, d z, d u ) time (s) GS Execution time vs energy value: airport dataset airport BCDA BCDAc GS barracks BCDA BCDAc Execution time vs energy value: barracks dataset 11

Experimental results: image restoration Task: Restoration of old painting degraded by the craquelure ageing effect. Dataset: The Girl with a pearl earring by Johannes Vermeer.

12 Experimental results: image restoration Task: Restoration of old painting degraded by the craquelure ageing effect. Dataset: The Girl with a pearl earring by Johannes Vermeer. Color image at 8-bits per band. Size: 600 x 600. M. Zanetti, L. Bruzzone, Piecewise linear approximation of vector-valued images and curves via 2nd-order variational model, IEEE Transactions on Image Processing, to appear,

Experimental results: image restoration Particulars of the red squared area Original Mumford-Shah Original image Approx. via 1 st order model (Mumford-Shah) Mixed Blake-Zisseramn Approx.

13 Experimental results: image restoration Particulars of the red squared area Original Mumford-Shah Original image Approx. via 1 st order model (Mumford-Shah) Mixed Blake-Zisseramn Approx. via mixed model Proposed approx. via 2 nd order model (Blake-Zisserman) The 1 st order penalization introduces over-segmentation. The pure 2 nd -order penalization returns a more natural solution. 13

Experimental results: curve approximation Digital surface model at spatial resolution 1m. The subject is a polygonal building (a barrack). Discrete curve representing the building shape.

14 Experimental results: curve approximation Digital surface model at spatial resolution 1m. The subject is a polygonal building (a barrack). Discrete curve representing the building shape. Obtained by proper processing of the DSM. Task: Recovering polygonal shapes from discrete points. Data: Discrete building boundaries extracted from DSMs. Objective: Emphasize the capability of the BZ model of providing discontinuous solutions. 14

15 Experimental results: curve approximation Smoothing cubic splines G u = 1 q u 2 dt T + q u g 2 dt T 1-D Blake-Zisserman model F u = u 2 dt T + λ u g 2 dt T + ν#(s u ) 15

16 Statistical methods for change detection in multispectral images

17 Motivations and challenges Motivations Many open-access databases of data (e.g., Landsat, Sentinel). Multitemporal multispectral images: study of the global change, environmental monitoring, etc. Challenges Automatic, robust/effective procedures needed to handle such amount of data. Mathematical models are needed to explain how to extract information. Standard approaches to change detection (effective on previous generation of images) do not show the same accuracy on last generation images. 17

18 Multispectral images Multispectral (MS) images are characterized by many spectral bands (from 5 to 13). Pixel value is a N-dimensional vector containing information about the spectral signature of the reflecting object. Multitemporal analysis of MS images allows to identify changes in the measured spectral signature. Satellite (sensor) Table: Relevant spaceborne multispectral sensors operating nowadays. Geometrical resolution (m) Swath (km) Spectral bands (intervals) Quant. (bits) Revisit time (days) Landsat 8 (OLI, TIRS) 15,30, (VNIR, SWIR, TIR, Coastal) SPOT (Pan), (Pan, VNIR) 12 1 to 3 Sentinel 2 (MSI) 10,20, (VNIR, SWIR, RedEdge)

19 Change Vector Analysis (CVA) We consider 2 spectral channels for each date (not a technical restriction). X 1 Vector difference D CVA change map X 2 Pixelwise representation of the polar decomposition in CVA channel 2 x 1 x 2 d ρ = d θ Magnitude image Direction image channel 1 19

20 Statistical models and binary decision The distribution of ρ = d is a mixture representing two classes p ρ = p ω n p ρ ω n + p ω c p(ρ ω c ) where: ω n unchanged pixels, and ω c changed pixels. To infer the class given a value ρ, we need to estimate distribution parameters. This can be done via Expectation-Maximization (EM) algorithm. Binary decision based on Bayes rule (thresholding). SoA statistical models: Standard approach [1] empirically assumes that classes are Gaussian. More advanced model [2] derives the distribution (under certain assumptions) as a Rayleigh-Rice mixture. [1] L. Bruzzone and D. F. Prieto. Automatic analysis of the dierence image for unsupervised change detection. IEEE Transactions on Geoscience and Remote Sensing, 38(3):1171{1182, [2] F. Bovolo and L. Bruzzone. A theoretical framework for unsupervised change detection based on change vector analysis in the polar domain. IEEE Transactions on Geoscience and Remote Sensing, 45(1):218{236,

21 Proposed EM algorithm We devised a parameter estimation method for the Rayleigh-Rice mixture through an iterative formulation of the EM algorithm. α k+1 = 1 N b 2 k+1 = x X D p ω n x, Ψ k x p ω n x, Ψ k x 2 2 p ω n x, Ψ k x Our approach is only based on independent parameters updating. The validity of the algorithm is very general. The method is fully unsupervised. σ 2 k+1 = ν k+1 = x p ω c x, Ψ k x I 1 I 0 p ω c x, Ψ k xν k σ k 2 xν k σ k 2 I 1 x p ω c x, Ψ k x 2 + ν k 2 2xν k 2 p ω c x, Ψ k x x I 0 xν k σ k 2 xν k σ k 2 x Zanetti, M. and Bovolo, F. and Bruzzone, L. Rayleigh-Rice mixture parameter estimation via EM algorithm for change detection in multispectral images, IEEE Transactions on Image Processing, 24 (12), pp ,

Experimental results: dataset Dataset A: Sensor: TM sensor Landsat-5.

Image size: 412 x 300. Change: lake enlargement channel 4 (Sept.

Landsat-8. Spatial resolution: 30 mt. Area: Lake Omodeo (Sardinia, Italy).

22 Experimental results: dataset Dataset A: Sensor: TM sensor Landsat-5. Spatial resolution: 30 mt. Area: Lake Mulargia (Sardinia, Italy). Image size: 412 x 300. Change: lake enlargement channel 4 (Sept. 1995) channel 4 (July 1996) reference map Dataset B: Sensor: OLI sensor - Landsat-8. Spatial resolution: 30 mt. Area: Lake Omodeo (Sardinia, Italy). Image size: 700 x 650. Change: fire. channel 4 (Sept. 1995) channel 4 (July 1996) reference map 22

23 Exprerimental results: fitting performance Dataset A Dataset B Mixture Χ P 2 KS proposed Rayleigh-Rice 0,0136 0,0362 standard Gaussian 0,0420 0,0836 Mixture Χ P 2 KS proposed Rayleigh-Rice 0,0215 0,0400 standard Gaussian 0,0500 0,

24 Experimental results: CD performance optimal (OE = 0,62%) Rayleigh-Rice (OE = 1,47%) Gaussian-mixture (OE = 3,22%) optimal (OE = 1,02%) Rayleigh-Rice (OE = 1,93%) Gaussian-mixture (OE = 2,99%) 24

25 Future activities

26 Future activities Automatic detection of geometric features from objects represented by unstructured 3D point clouds (raw LiDAR points, TomoSAR). Requires proper handling of 2 nd order differential operators via Finite Element Method. Provide mathematical quantification of the error in change detection methods based on the magnitude information. Low discriminability when many class members are assumed; Provide a lower bound in the selection of the number of bands for the calculation of the magnitude. Multiple change detection. It requires reduction of class-variance. This task can accomplished via variational methods. Preliminary results are very encouraging. 26

Unsupervised Change Detection in Remote-Sensing Images using Modified Self-Organizing Feature Map Neural Network

Unsupervised Change Detection in Remote-Sensing Images using Modified Self-Organizing Feature Map Neural Network Swarnajyoti Patra, Susmita Ghosh Department of Computer Science and Engineering Jadavpur