WHAT IS REMOTE SENSING?

Transcription

1 How Does Artificial Intelligence Work with Remote Sensing Technologies for Multi-scale Environmental Change Detection? Prof. Ni-Bin Chang Director, Stormwater Management Academy University of Central Florida Orlando, FL, USA WHAT IS REMOTE SENSING? Remote sensing is defined as the acquisition, processing, and interpreting of images that are remotely obtained from sensors recording the interaction between electromagnetic energy and the Earth s surface Remote Sensing Platforms: Ground-based Airplane-based Satellite-based 2 1

2 SPECTRAL SIGNATURE OF SOIL, VEGETATION AND WATER, AND SPECTRAL BANDS OF LANDSAT 7 The amount of radiation that is emitted and reflected from an object is a function of wavelength Materials can be identified based upon their unique spectral signatures Source: Siegmund, Menz 2005 THE RESEARCH NICHES FOR ENVIRONMENTAL REMOTE SENSING Who kinds of limitation of remote sensing technology do we face at present?

3 WHAT IS LAKE EUTROPHICATION? Eutrophication is the excess loading of nutrients to a water body and the resulting effects of ecosystem health linked to the surplus quantities of nutrients. (a) Incipient stage (b) Fostering stage (b) Fostering stage (d) Bloom spreading stage Algal bloom event observed in Sept. 18-Sept. 22, 2008 (Chang et al., 2008). FEATURE EXTRACTION IN IMAGE CLASSIFICATION Analytical Method Semiempirical Empirical Methods Empirical Method Satellite Parameters Location Reference Artificial Neural Network Genetic Programming (GP) Radial Basis Function Neural (RBFN) Network Models Landsat TM SPOT Landsat TM Chlorophyll-a and suspended sediment Chlorophyll Chlorophyll-a and Suspended Matter Delaware Bay, U.S. Yeong-Her- Shan reservoir, China Beaver Lake, U.S. Keiner and Yan, (1998) Chen (2003) Panda et al., (2004) 3

4 FEATURE EXTRACTION METHODS FROM SATELLITE DATA s 2000s 2010s Regression ANN GP FUSION Chang, N. B., Imen, S., and Vannah, B. (2015): Remote sensing for monitoring surface water quality status and ecosystem state in relation to the nutrient cycle: a 40-year perspective. Critical Reviews of Environmental Science and Technology, 45(2), RECENT BREAKTHROUGHS IN RESEARCH Suspended Sediment Concentration: Landsat Chlorophyll-a: Hyperspectral remote sensing is more reliable for monitoring Chl-a concentration than multispectral remote sensing because it can consider the reflectance of the extremely narrow wavebands. Nutrients: Landsat and MODIS Total Organic Carbon: Landsat and MODIS Microsystin: MODIS and Landsat Source: Chang et al.,

5 IEEE Systems Journal Source Chang, N. B., Xuan, Z., and Yang, J. (2013): Exploring spatiotemporal patterns of nutrient concentrations in a coastal bay with MODIS images and machine learning models. Remote Sensing of Environment, 134,

6 FEATURE EXTRACTION USING GENETIC PROGRAMMING MYD09GA MYD09A1 MODIS Reprojection Tool Ground Truth Data ArcGIS Inverse Model (GP) Output Map Chang et al., 2013 GP model building; inverse modeling CATEGORIZATION OF MACHINE LEARNING ALGORITHMS Genetic Programming (GP) Supervised, unsupervised, and semi-supervised learning algorithm Support Vector Machine (SVM) Supervised learning algorithm toward unsupervised learning algorithm Artificial Neural Network (ANN) Supervised or unsupervised learning algorithm depending upon the training techniques employed 6

7 IN-SITU OBSERVATIONS AS GROUND-TRUTH Tampa Bay Water Atlas provides historical In situ data via Online Databases through USF. The data collections can be tracked back more than 40 years. 45 monitoring stations 581 data points for model calibration 159 data points for model validation RETRIEVAL OF SYNCHRONOUS SATELLITE DATA IN CLEAR DAYS TO AVOID CLOUD IMPACT Jan Feb Mar Apr May Jun Jul Aug Sep Oct Nov Dec , ,13 From the SeaDAS main menu, we selected several products related to our study These include the MODIS data with bandwidth between 405 and 683nm an two other products of information concerning colored dissolved organic material (CDOM) and chlorophyll-a (chlor-a). Chang et al.,

8 GENETIC PROGRAMMING MODEL CALIBRATION AND VALIDATION Overcome the current challenge of feature extraction Chang et al., 2013 BACKGROUND OF GP MODEL Genetic algorithms used in evolutionary computing were developed by John Holland in GP is a specialization of genetic algorithms that employ machine learning and data mining techniques to identify system behaviors based upon empirical data. GP mimics natural evolutionary pathways to optimize computer programs in order to converge toward a target solution. 8

9 GENERIC ALGORITHMS Crossover Choose a random point on the two parents Split parents at this crossover point Create children by exchanging tails P c typically in range (0.6, 0.9) Mutation: Alter each gene independently with a probability p m p m is called the mutation rate Typically between 1/pop_size and 1/ chromosome_length TREE-BASED GP MODEL a) General structure of a GP b) Crossover between programs c) Program mutation 9

10 ANIMATION: CROSSOVER ANIMATION: MUTATION 10

11 THE GP MODEL ANALYSIS FOR TP ESTIMATION TP (mg L -1 ) = (((X 2 + V 4 + V 0 ) / X 1 + V 2 ) 2 + V 0 ) / X f / X f X 1 = 1- X 2 -X 3 -X 4 X 2 = (X f - V 6 ) / (1 - X 3 -X 4 ) + V 0 X 3 = 2*(2X f) + V 1 + V 0 + 2V 2-2V 3 X 4 = 2V 12 *( f)-V 3 f: floating point register 11

12

13 Reference Chang N.B., Vannah B., Yang Y.J., 2014, Comparative sensor fusion between hyperspectral and multispectral satellite sensors for monitoring Microcystin distribution in Lake Erie, IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing, 7(6), pp URBANIZATION AND WATERSHED MANAGEMENT The toxins cause Toledo's water supply out of service on Aug AP Photo: Haraz N. Ghanbari 26 13

14 RETRIEVAL OF MICROCYSTIN FROM SATELLITE DATA Because Microcystis is a bacterium that uses photosynthesis for energy production, high concentrations of Microcystis can be linked with elevated chlorophyll-a levels. Phycocyanin is a pigment in all cyanobacteria that shares a positive correlation with Microcystin levels. Surface reflectance of phycocyanin, chlorophyll-a, and Microcystis are suitable indicators for the prediction of Microcystin levels in a lake. The surface reflectance curves for chlorophyll-a and phycocyanin in surface waters peak at 552, 650, 680, and 720 nm. FUSION OF TWO MULTISPECTRAL IMAGES FOR ENVIRONMENTAL APPLICATIONS STAR-FM = Spatial and Temporal Adaptive Reflectance Fusion Model Landsat High spatial resolution Low revisit rate MODIS High temporal resolution Low spatial resolution 14

15 INTEGRATED DATA FUSION AND MINING (IDFM) Objectives: generate daily high spatial resolution of synthetic image for intensive monitoring. Procedural Flow: 1. Data Acquisition 2. GIS Image Processing 3. Image Fusion (STARFM) 4. Train and Validate GP Model 5. Generate Microcystin Concentration Maps Created by inputting fused band data into the GP model 2 1 MERIS Reflectance Bands (300 m) Image Processing Steps: -Reproject to UTM 17N -Crop out Land MERIS Reflectance Bands (300 m) 3 Fused Surface Reflectance Bands (300 m) 4 MERIS and MODIS Data Acquisition Data Fusion Data Mining Microcystin Prediction Model MODIS Reflectance Bands (1000 m) Image Processing Steps: -Reproject to UTM 17N -Resample to 300 m -Crop out Land -Offset and scale data MODIS Reflectance Bands (300 m) Ground-Truthing Data 5 Microcystin Concentration Map (300 m) FUSION OF HYPERSPECTRAL AND MULTISPECTRAL IMAGES FOR ENVIRONMENTAL APPLICATIONS MODIS #1 MODIS #2 MODIS #3 Data Fusion Through STAR-FM Landsat #1 (F) Landsat #3 Input Data Streams: 3 MODIS 2 Landsat STARFM Output: 1 Fused Image Fused Image Goodness of Fit: R = R2 =

16 Integrating Hyperspectral and Multispectral Remote Sensing MERIS (MEdium Resolution Imaging Spectrometer) measures the reflectance of the Earth (surface and atmosphere) in the solar spectral range (390 to 1040 nm) and transmits 15 spectral bands back to the ground segment. Satellite Sensor Terra MODIS ENVISAT MERIS Product Selection Surface Reflectance (Level 2) Surface Reflectance (Level 2P) Spatial Resolution Temporal Resolution Bands Used 1000 m Daily m 1-3 Days 1-13 FUSION RESULTS OF BIO-OPTICAL MODELS Band centers with the highest performance for the traditional two-band models The band centers most used by the traditional two-band models fall in the range of nm and nm. This range corresponds to the spectral features observed in this figure. 16

17 STATISTICAL ANALYSIS The GP model using MERIS inputs had the best overall performance. All GP models exemplified higher explanatory power than the two band model. The fused model had the longest computational time, yet this model has a number of inherent advantages: 300 m spatial resolution for enhanced algal bloom delineation Daily revisit time, which is necessary for an early warning system Two Band MODIS GP MERIS GP Fused GP Observed Microcystin Mean (µg L ¹) Predicted Microcystin Mean (µg L ¹) Root Mean Square Error (µg L ¹) Ratio of St. Dev Mean Percent Error (%) Square of the Pearson Product Moment Correlation Coefficient (R2) Computational Time (seconds) < RESULTS OF PREDICTIVE CAPABILITY WITH 3 GENETIC PROGRAMMING MODELS MERIS GP Model 26 Data points Fused multispectral sensor 41 Data points Fused hyperspectral sensor 41 Data points More capable Predicting low concentration 17

18 RESULTS OF MICROCYSTIN CONCENTRATION MAPS (a) Hyperspectral sensor GP model (B) Multispectral sensor GP model The 30m spatial resolution of the multispectral image provides more detailed outline, while the coarser (300 m) hyperspectral resolution predicts microcystin concentration in locations that more closely align with harmful algal bloom (HAB) presence. Dark red dots denotes areas of high microcystin concentration that pose a health threat, while yellow spots indicate low to medium microcystin concentrations. RECENT BREAKTHROUGHS IN MY RESEARCH SIASS: Spectral Information Adaptation and Synthesis Scheme Reference SMIR: Smart Memory for Image Reconstruction Chang, Bai, K. N. X., B., Chang, Bai, K. N. X., B., and and Chen, Chen, C. C. F. F. (2015): (2015): Smart Spectral information information reconstruction adaptation via time-space-spectrum and synthesis scheme continuum for merging for cloud cross-mission removal in satellite consistent images, ocean IEEEcolor Journal reflectance of Selected observations Topics in Applied from MODIS Earth and Observations, VIIRS. IEEE 99, Transactions on Geoscience and Remote Sensing, 54(1), Bai, Chang, K. X., N. Chang, B., Bai, N. K. B., X., and and Chen, Chen, C. C. F. F. (2015): (2015): Spectral Smart information information adaptation reconstruction and synthesis via time-spacespectrum scheme for merging cross-mission continuum for consistent cloud removal ocean color in satellite reflectance images, observations IEEE Journal from MODIS of Selected and VIIRS. Topics in IEEE Applied Transactions Earth on Observations, Geoscience and 99, Remote Sensing, 54(1),

19 NEW APPROACHES Spectral Information Adaptation and Synthesis Scheme (SIASS) oprinciples: cross-mission sensors provide multiobservations at different crossing time omethod: merging cross-mission ocean color products could improve spatial and temporal coverage SMart Information Reconstruction (SMIR) o principles: memory effect of interrelationships between cloudy pixels and cloud free pixels o method: reconstruction from cloud free pixels Cross-mission Data Merging with Image Reconstruction and Mining (CDMIM) oprinciples: SIASS + SMIR + Feature Extraction o method: large scale complex systems SIASS: SPECTRAL INFORMATION ADAPTATION AND SYNTHESIS SCHEME Multiple Satellites Features Visible Infrared Imaging Radiometer Suite (VIIRS) Moderate-Resolution Imaging Spectroradiometer (MODIS)- Aqua Moderate-Resolution Imaging Spectroradiometer (MODIS)- Terra Courtesy of NASA 19

20 After merging, the spatial coverage has been greatly improved, but there are still some value missing pixels. chlor-a concentration retrieved from three ocean color bands Statistics of Terra-MODIS Ocean Color images with full coverage over Lake Nicaragua through Annual C5 C11 Year Num. of Images POC Min (%) Max Avg * C5: Composited climatologic images with the latest 5 years ( ) images, C11: same as C5 but with the past 11 years ( ) images. Figure Time series of the cloud coverage over Lake Nicaragua with 5 (a) and 11 (b) years composition. 20

21 ARTIFICIAL NEURAL NETWORKS Schematic comparison between a biological neuron and an artificial neuron. Biological Neurons Artificial Neurons Analogy between biological neurons and artificial neurons Feedforward Backpropagation Neural Networks after Winston, 1991; Rich and Knight, 1991 Courtesy of David Leverington THE ARCHITECTURE OF EXTREME LEARNING MACHINE Shen et al.,

22 Data Merging and Reconstruction Data Merging A: 18.6% A + V: 24.64% A + V +T: 24.92% Figure 1: Comparisons of (left) original MODIS-Aqua (A) ocean color reflectance image before and after fusing at 678 nm with (b) VIIRS-NPP (V) and (c) MODIS-Terra (T) on June 14, The percent coverage was given as the ratio between the number of pixels having valid data value and the total number of pixels covered in the lake, and a value of 100% means the fully coverage. Data Merging and Reconstruction Data Reconstruction A + V +T: 24.92% Reconstructed: 98.12% SMIR Figure 2: Comparisons of (left) the merged ocean color reflectance image and (right) the reconstructed one on June 14,

23 Water Quality Monitoring with the CDMIM Algorithm Total Nitrogen DOY: DOY: Figure 3: Predicted total nitrogen (TN) concentrations in Lake Nicaragua at June 17, 2014 (DOY: ) and March 16, 2015 (DOY: ) using data mining approaches. Water Quality Monitoring with the CDMIM Algorithm Total Phosphorous DOY: DOY: Figure 4: Predicted total phosphorous (TP) concentrations in Lake Nicaragua at June 17, 2014 (DOY: ) and March 16, 2015 (DOY: ) using data mining approaches. 23

24 SUMMARY 2012: IDFM provides the biases for advanced earth observations. 2014: SIASS removes the biases among cross-mission sensors prominently. 2014: SMIR can recall more invariant spectral features reserved in previous images. 2016: CDMIM can be applicable for uncovering many environmental change detection in almost all-weather conditions. 24