Data Fusion Jeffrey S. Evans - Landscape Ecologist USDA Forest Service Rocky Mountain Research Station Forestry Sciences Lab - Moscow, Idaho Data Fusion Data Fusion is a formal framework in which are expressed means and tools for the alliance of data originating from different sources. It aims at obtaining information of grater quality; the exact definition of greater quality will depend upon the application. Wald (1999) Merging data from multiple sources to optimize data or create value added data Using multiple sources of data for a statistical estimate where the data interacts to enhance predictive power. Data Fusion Terms Merging/Combination Processesthat involve a mathematical operation that is performed on at least two sets of data Integration The concatenation of data from multiple sources Assimilation Inclusion of data from multiple sources into numerical models for the analysis of a system The extraction and use of the best pieces of information from each data source 1
IMAGE FUSION Description of Image Data Fusion Reasons for Image Fusion Visual interpretation and base maps. Delineation and object identification. Classification. Image segmentation. 2
Data Fusion in Remote Sensing Data fusion can be conducted at different image levels: 1. On a pixel-by-pixel basis 2. Derivation and combination of image features 3. Combination of pre-classified data derived from each separate image In remote sensing research, level 1 is normally used. A common example of data fusion in remote sensing is to fuse Multispectral data (e.g. TM, MODIS, etc) with high spatial resolution panchromatic imagery. The perfect result of data fusion is an image that is identical to the image that a Multispectral (MS) sensor with the spatial resolution of the panchromatic (PAN) image would produce. i.e. The aim is to increase the spatial details whilst preserving the spectral information. Image Fusion Methods Many different data fusion techniques exist, including: Adaptive Image Fusion Intensity -Hue-Saturation Transformation (ENVI) Principal Components Analysis/Substitution (ENVI) Brovey Transformation (ENVI) Gram-Schmidt Orthogonalization (ENVI) CN Spectral Sharpening (ENVI) Regression Variable Substitution Multiplicative Algorithms Wavelets Probabilistic/Statistical Techniques + Many Many More Intensity / Frequency Modulation Intensity modulation can be understood as a lowresolution image modulated by high resolution patterns. Effects the spectral / color distortion. Can degrade fidelity of original spectral information. ( λ) s = ( λlow)( γ ( γ ) χ high ) 3
Resample the low spatial-resolution MS image into the same number of pixels as the high spatial-resolution PAN image. ENVI: Transformation/Image Sharpening options: HSV Color Normalized (Brovey) PC Spectral Sharpening Gram-Schmidt Spectral Sharpening CN Spectral Sharpening PAN Image MS Image Spatial Resolution = 10m Spatial Resolution = 28m IHS (or HSV) Replaces the Intensity Bands of the MS IHS/HSV transformation with the PAN image and do an inverse transform to get a spatially enhanced MS image. 4
Principle Component Analysis Replaces the 1 st Principal component of the MS PCA transformation with the PAN image and do an inverse transform to get a spatially enhanced MS image. Brovey Transformation Normalizes the DN values of the MS image at the spatial resolution of the PAN image. Comparisons PAN Image MS Image IHS PCA Brovey 5
Adaptive Image Fusion Uses a filter based approach to smooth modulation. Assumes spectral response is gaussian distributed. Achieves high fidelity to original spectral information. Sharpens object edges Does not account for texture. Using a local neighborhood, identifies separate distributions between both images and uses a sigma filter to normalize variance. ( λ) s = ( γ γx) λ γδ + λx δ AIF 15m Multi-Spectral 30m PAN 15m Original ETM+7 Multi 30m Original ETM+7 Pan 15m AIF 15m PCA 15m 6
Transect Distribution of AIF Band 2 DN values vs. ETM+7 30m along 3000m transect Band 2 - AIF Digital Numbers 65 60 55 50 45 40 35 30 0 240 480 720 960 25 20 1200 1440 1680 1920 2160 2400 2640 2880 3120 3360 ETM AIF Transect Distribution of PCA Band 2 DN values vs. ETM+7 30m along 3000m transect Band 2 - PCA Digital Numbers 65 60 55 50 45 40 35 30 0 240 480 720 960 25 20 1200 1440 1680 1920 2160 2400 2640 2880 3120 3360 ETM PCA INTERGRATION 7
Principal Components Analysis Multivariate approach that transforms a set of potentially correlated variables into a smaller set of uncorrelated variables in order to: Reduce the dimensionality of a dataset while retaining as mush information as possible Identify Potential Underlying sources of variation Rotates the data such that the the maximum sources of variation are projected onto n number of axes. 1 st PC = the combination of variables that contain the greatest amount of variation 2 nd PC = the combination of variables with the next largest amount of variation Independent to the data in the 1 st PC (orthogonal to the 1 st. PCA) Conditional Prediction and Data Interaction Conditional Predictions conditioned by heuristic criteria (e.g. basal area by covertype) A-priori or model rules Integration Interaction of terms or variables in model that condition data (e.g.basal Area by canopy density) Term in model Conditional RandomForest CART Basal Area by Covertype Probability functions Data LiDAR Classified spectral 8
Data Interaction - Regression using Spectral ALI and LiDAR 9