Analisi di immagini iperspettrali satellitari multitemporali: metodi ed applicazioni E-mail: bovolo@fbk.eu Web page: http://rsde.fbk.eu
Outline 1 Multitemporal image analysis 2 Multitemporal images pre-processing 3 Multitemporal information extraction 4 Applications 5 Conclusions 2
Multitemporal Image Analysis Multitemporal image analysis: aims at analyzing two or more remote sensing images acquired on the same geographical area at different times for identifying changes or other kinds of relevant temporal patterns occurred between the considered acquisition dates. Several multitemporal problems exist: Binary change detection. Multiclass change detection. Trend analysis in long time series. Binary change detection Tsunami damages Multiclass change detection Crop rotation Trend analysis in long time series Before Change After Change Before Change After Change True Color Composite NDVI 3
Multitemporal Analysis Block Scheme & Products Data Analysis Binary Change Detection X 1 Binary change map t 1 image Multiple Change Detection Multiple-change map X 2 t 2 image Multitemporal image pre-processing Radiometric corrections 1 Co-registration 1 Multitemporal Classification Land-cover transition map N N Map Updating Land-cover map @t n+1 Trend Analysis in Time Series X N Trend map t N image Change Detection in Time Series Seasonal/Annual Change Map F. Bovolo, L. Bruzzone, The Time Variable in Data Fusion: A Change Detection Perspective, IEEE Geoscience and Remote Sensing Magazine, Vol. 3, No. 3, pp. 8-26, 2015. 4
Multitemporal Images Co-registration In multitemporal image processing, poor co-registration is a source of errors and can be modeled as a noisy component. Registration noise statistical properties have been modeled by accounting for its multiscale behaviors in the difference image domain. 10-4 Multiscale CVA RN polar plot RN probability density function Multitemporal falsecolor composite of misaligned images Registration noise map T 0 Differential Level analysis 30 T RN F. Bovolo, L. Bruzzone, S. Marchesi, Analysis and Adaptive Estimation of the Registration Noise Distribution in Multitemporal VHR Images, IEEE Transactions on Geoscience and Remote Sensing, Vol. 47, No. 8, Part 1, pp. 2658-2671, 2009. 5
Multitemporal Images Co-registration Registration noise information can be used to improve multitemporal analysis by: Removing noisy pixels in post-processing [1]; Involving registration noise in the co-registration step [2]-[4]; Other more complex paradigms. Multitemporal chessboard images RMSE (pixels) STD (pixels) Before co-registration 16.87 2.13 SoA 3.70 2.21 Registration noise based co-registration 0.89 0.37 Before co-registration After co-registration [1] S. Marchesi, F. Bovolo, L. Bruzzone, A context-sensitive technique robust to registration noise for change detection in VHR multispectral images, IEEE Transactions on Image Processing, Vol. 19, No. 7, pp. 1877-1889, 2010. [2] Y. Han, F. Bovolo, L. Bruzzone, An Approach to Fine Co-Registration Between Very High Resolution Multispectral Images Based on Registration Noise Distribution, IEEE Transactions on Geoscience and Remote Sensing, Vol. 53, No. 12, pp. 6650 6662, 2015. [3] Y. Han, F. Bovolo, L. Bruzzone, Edge-Based Registration Noise Identification for VHR Multisensor Images, IEEE Geoscience and Remote Sensing Letters, Vol. 13, pp. 1231 1235, 2016. [4] Y. Han, F. Bovolo, L. Bruzzone, Segmentation-based Fine Registration of Very High Resolution Multitemporal Images, IEEE Trans. on Geoscience and Remote Sensing, in press 2017. 6
Hyperspectral Multitemporal Data Analysis: Challenges Multitemporal hyperspectral data are highly sensitive to detailed variations of the spectral signatures of land covers thus when multitemporal images are considered subtle changes/temporal variations can be detected. The definition of the concept of change in hyperspectral (HS) images is not clear in the literature and represents the first challenge. Only few approaches to multitemporal in hyperspectral images exist in the literature. State-of-the-art methods are mainly developed for multispectral images and do not explicitly handle the challenging issues that may arise due to the properties of hyperspectral images like: the high dimensionality; the presence of noisy channels and redundant information; the increase of computational cost; the increase of the possible number of changes; the high complex change representation and identification; the presence of changes at sub-pixel level (i.e., mixed pixels). 7
Hyperion data Spectral signature variation Spectral signature variation Hyperspectral Change Concept Low spectral resolution High spectral resolution R G B 823.65nm 721.90nm 620.15nm 2004 2007 Multitemporal false color composite 8
Spectral signature variation Hyperspectral Information Visualization & Management X 1 X 2 Pseudo binary change detection W c Pseudo binary change detection map W = {w n, W c, W u } Hierarchical spectral change analysis W u X D Change endmembers Uncertain pixel labeling Major changes w C1 wc 11 1 2 Change end-member Root W c wc 2 2 wc 2 1 1 w C2 wc 21 wc 2 1 2 wc w C3 Subtle changes T r w n Change detection map 0 1 2 10-4 F. Bovolo, S. Marchesi, L. Bruzzone, A Framework for Automatic and Unsupervised Detection of Multiple Changes in Multitemporal Images, IEEE Transactions on Geoscience and Remote Sensing, Vol. 50, No. 6, pp. 2196-2212, 2012. S. Liu, L. Bruzzone, F. Bovolo, P. Du, Hierarchical unsupervised change detection in multitemporal hyperspectral images, IEEE Transactions on Geoscience and Remote Sensing, Vol. 53, pp. 244-260, 2015. DOI 10.1109/TGRS.2014.2321277. S. Liu, L. Bruzzone, F. Bovolo, M. Zanetti, P. Du, Sequential Spectral Change Vector Analysis for Change Detection in Multitemporal Hyperspectral Images, IEEE Transactions on Geoscience and Remote Sensing, Vol. 53, pp. 4363 4378, 2015. 9
Hyperspectral Information Visualization & Management R G B 823.65nm 721.90nm 620.15nm 2004 2007 Multitemporal false color composite Proposed method Standard clustering 10
Reflectance Unmixing in Hyperspectral Multitemporal Images Change No-change Multiemporal information extraction can be designed as a X 1 X 2 X 1 X 2 multitemporal spectral unmixing problem at sub-pixel level. + - + - A pure multitemporal endmember (MT-EM) of either change or no change is a stacked feature vector composed by a pure spectral signature in both X 1 and X 2 components. X 1 X 2 No-change - - - - Change - + - + X S X X, X S 1 2 material class 1 +: pure spectrum - : mixed spectrum material class 2 material class 3 MT-spectral unmixing MT-EM Pool U Unmixing Model A U Final CD Map S. Liu, L. Bruzzone, F. Bovolo, P. Du, Unsupervised Multitemporal Spectral Unmixing for Detecting Multiple Changes in Hyperspectral Images, IEEE Transactions on Geoscience and Remote Sensing, Vol. 54, pp. 2733-2748, 2016. Change Analysis U n,u c Abundance Combination 11
Unmixing in Hyperspectral Multitemporal Images Abundance Maps 1 Change Class 2 Change Class 4 Extracted Endmembers 42 endmembers 0 16 change MT-EMs in U c 26 no-change MT-EMs in U n Change Class 6 No-change Class 12
Land-Cover Map Updating Low-cost definition of training set T 2 X 1 CD Driven Transfer Learning Initial Training set T 2 Training set T 1 Multitemporal Active Learning t 1 image W { w1, w2,..., w R } Final Training set T 2 Training set T 1 Cascade Classification X 2 t 2 image { 1, 2,..., N } Land-cover map at t 2 B. Demir, F. Bovolo, L. Bruzzone Updating Land-Cover Maps by Classification of Image Time Series: A Novel Change-Detection-Driven Transfer Learning Approach, IEEE Transactions on Geoscience and Remote Sensing, Vol.51, pp.300-312, 2013. 13
Applications Multitemporal analysis of hyperspectral images can be relevant in many applications, among the others those where small intensity changes and/or changes slow in time may strongly benefit: Forest analysis (e.g., forest disturbance, forest fires, forest classification, biomass analysis); Precision agriculture (e.g., crop mapping, crop rotation, crop stress analysis, fertilization); Water quality (e.g., chlorophyll monitoring, alga bloom); Vegetation (e.g., desertification, deforestation, vegetation stress); Etc. 14
Conclusions PRISMA will guarantee regular multitemporal hyperspectral images. Several activities have been developed in multitemporal image processing in multispectral images (satellite and airborne). Automatic supervised/unsupervised methods are available including: Multitemporal images preprocessing. Binary change detection. Multiclass change detection. Trend analysis in long time series. These methods can been adapted to the characteristics of PRISMA hyperspectral multitemporal images to generate new products. 15
16