DATA FUSION, DE-NOISING, AND FILTERING TO PRODUCE CLOUD-FREE TEMPORAL COMPOSITES USING PARALLEL TEMPORAL MAP ALGEBRA

Transcription

1 DATA FUSION, DE-NOISING, AND FILTERING TO PRODUCE CLOUD-FREE TEMPORAL COMPOSITES USING PARALLEL TEMPORAL MAP ALGEBRA Bijay Shrestha Dr. Charles O Hara Preeti Mali GeoResources Institute Mississippi State University

2 OUTLINE Introduction Map Algebra Temporal Map Algebra Vegetation Indices Parallel Temporal Map Algebra Results Conclusions

3 Motivation Undesired cloud cover in satellite images Need for cloud-free satellite observation Making use of high temporal resolution of satellite data using cross-platform fusion Moderate Resolution Imaging Spectroradiometer (MODIS) Aqua and MODIS Terra Rule based cross-platform fusion of MODIS Aqua and MODIS Terra Using Parallel Temporal Map Algebra

4 Map Algebra An approach to raster data handling which treats spatial data layers as variables which may be combined using mathematical operators. Source: Geographic Information Systems and Cartographic Modeling, Tomlin

5 Map Algebra (Contd.) Local Operation = Arithmetic, Relational, Bitwise, Boolean, Logical, Accumulative and Assignment Operators can be used for Map Algebra. 3 x 3 Focal Neighborhood

6 Temporal Map Algebra Temporal extension to conventional map algebra. Treats time series of imagery as three dimensional data set XY dimension represent Earth s surface Z dimension represents time Y Y X Image n Z Time Image 1 X

7 Temporal Map Algebra (Contd.) Temporal Map Algebra Local Function Temporal Map Algebra 3x3x3 Focal Neighborhood Row Time Column Source: Analyzing time series satellite imagery using temporal map algebra, J.Mennis and R. Viger

8 Vegetation Indices Dimensionless, radiometric measures that function as indicators green vegetation. Normalized Difference Vegetation Index (NDVI) NDVI = NIR NIR + Re Re d d Healthy, chlorophyll-based vegetation strongly reflects near-infrared wavelengths and reflects relatively weakly in the visible red. Range [-1, +1] NDVI is highly useful in vegetation studies.

9 Parallel Processing Decomposing a large process into small processes that can be solved simultaneously to provide faster execution time. Need for intensive computing to integrate and process large datasets. Many spatial programs are inherently parallel. Parallel processing can provide leap in performance.

10 Parallel Processing (Contd.) Requires large volume of satellite data Input Temporal Data Sets NDVI cube Image Quality cube Geo-location Angle cube

11 Performance Metrics Serial Execution Time (T S ) Parallel Execution Time (T P ) Speedup S= T S / T P Efficiency E = Speedup/p * p represents the number of processors used in parallel processing

12 Tools Used Matlab MatlabMPI LibTIFF

13 MODIS Datasets 6 MODIS datasets used MYD09GQK - 250m surface reflectance MODO9GQK data MYDMGGAD - 1 Km Geo-location angles MODMGGAD data MYD09GST - 1 Km surface reflectance MOD09GST quality data MYD - Aqua MOD - Terra

14 Methodology The following methodology was used Selection Collection Preprocessing Execution Analysis

15 Pre-processing Surface reflectance day 1 NDVI day 1 Surface reflectance day 2 Surface reflectance day N NDVI day 2 NDVI day N Aqua and Terra NDVI cubes Surf.Refl. Quality day 1 Surf. Refl. Quality day 2 Surf. Refl. Quality day N QMask day 1 QMask day 2 QMask day N Aqua and Terra Quality Mask cube Input to Fusion Algorithm Geolocation Angles day 1 view zenith angle day 1 Geolocation Angles day 1 Geolocation Angles day N view zenith angle day 2 view zenith angle day N Aqua and Terra View zenith angle cube

16 Fusion For day from 1 to 5 if angle is less than 48 AND mask is Land select pixel else get next pixel If no pixel is selected select pixel with highest NDVI

17 Example September Aqua MODIS Terra MODIS Fused MODIS

18 Parallel Processing (Contd.) Temporal Cubes Block Distribution & Processing 1 2 n Fused Product

19 Experimental Results (Contd.) The evaluation of parallel Temporal Map Algebra consists of following parts A measurement on the performance of serial processes The performance profiling of the parallel processes to fuse Aqua-Terra datasets

20 Experimental Results (Contd.) For the evaluation compositing was performed on 32-days MODIS data cubes using fusion algorithm The experiments were conducted on the EMPIRE cluster at HPC 2 at Mississippi State University. Initializing processes on different machines in cluster using MatlabMPI. Each Process Reads the cubes stored in Network File Server Performs computations Stores results in common storage Takes Readings Metrics are computed

21 Experimental Results (Contd.) Serial Run-Time 32 days Aqua-Terra Fusion Dimensions x 32 NDVI, Quality, Angular Temporal Cubes Input Size MB Output Size MB Serial Time hours

22 Experimental Results (Contd.) Parallel Run-times With Centralized Data Run-Times for Different Number of Processors Hours Number of Processors

23 Experimental Results (Contd.) Speedup and Efficiency SpeedUP Efficiency Speedup Efficiency Number Of Processors Number of Processors

24 Conclusion Further quantitative analysis needs to be performed to evaluate the fused Aqua-Terra product. The execution performances were analyzed using various metrics. The accuracy of the results, however, is affected by various factors such as network bandwidth, network traffic, processor speed, and network/cluster architecture. The use of Matlab and MatlabMPI as the programming environment has added some overhead to the total serial and parallel times for the computation. These overheads can be reduced by programming in C or C++ that have lower overhead. Temporal Map Algebra can be used to solve more complex spatio-temporal analysis problems on large datasets with many dimensions.

25 Proposed future Work Qualitative analysis of the cross-platform fused images Exploration of the use of TMA to solve newer spatiotemporal analysis problems with higher complexity with many dimensions. Achieve better performance by using software systems that have low overhead. Rapid prototyping/development capability

26 THANK YOU