Quality assessment of RS data Remote Sensing (GRS-20306)
Quality assessment General definition for quality assessment (Wikipedia) includes evaluation, grading and measurement process to assess design, development, production, installation, servicing and documentation, includes the regulation of the quality of raw materials, assemblies, products and components Related to requirements of application or end-user source: http://www.velorution.biz/?cat=3
The remote sensing chain acquisition preprocessing image analysis variables/ products application Remote sensing processing chain
Quality assessment: overview Visual assessment Statistical approaches Histogram Scatterplot Signal-to-Noise Ratio (SNR) Standards for quality assessment Validation
Habitat status Eder Heide Acquisition October 7, 2007
Visual assessment Viewing complete image or individual pixel brightness values: Color composites (combine different bands) Gray scale images Extract per pixel spectral information Extract errors related to: Cloudiness and atmosphere Sensor characteristics and errors (smile effect) Saturation Sun angle Combination of human eye and brain is limited Laborious work
Visual assessment: case AHS Eder Heide Gray scale images for band 6: 550 nm (left) and band 31: 2000 nm (right) Question Which (quality) differences do you observe between band 5 and band 31 of the AHS image?
reflectance Visual assessment: case AHS Eder Heide 1 0.9 0.8 0.7 0.6 0.5 0.4 0.3 0.2 0.1 0 0.45 0.95 1.45 1.95 2.45 wavelength (um) R00-1 R00-2 R00-ref Comparison of reflectance for 2 flight lines and field measured reflectance Composite of flight line 1 and 2 Question How do you assess the quality of the composite image and why? Water vapor image
Statistical approaches Some statistical definitions: population, sample, sampling error Univariate statistics (per band): mean, median, mode, min, max, variance, standard deviation, range, skewness, kurtosis presentation: histogram Multivariate statistics (between bands): Correlation, covariance Presentation: feature space plot or scatterplot Measures of image quality Signal-to-Noise Ratio (SNR)
reflectance reflectance Summary univariate statistics Example: 1.2 1 AHS data Eder Heide 0.8 Mean 0.6 Stdev Min 0.4 Max Strip 1 0.2 0 0.4564 0.9564 1.4564 1.9564 2.4564 wavelength (μm) 1.2 1 0.8 0.6 0.4 Mean Stdev Min Max Strip 2 0.2 0 0.4564 0.9564 1.4564 1.9564 2.4564 wavelength (μm)
Histogram of single band Landsat TM
Common histogram types for RS data Skewness: measure of asymmetry Kurtosis: measure of sharpness of peak (compared to normal distribution) source: Jenssen 2004
Multivariate image statistics RS data: measurement of energy flux in more than one band covariance and correlation: determine how measurements for several bands are related PCA, feature selection, classification, accuracy assessment Scatterplot, scattergram or feature space plot: two-dimensional presentation of the brightness value for every pixel in the scene for two bands
Example of scatterplot for Landsat TM source: Schowengerdt 2006
Signal to Noise Ratio (SNR) Signal is informative part of a measurement Noise is corrupting signal and information content Signal to Noise Ratio (SNR): many definitions, be careful! SNR = stdev (signal) / stdev (noise) SNR often derived as image based (flat field approach): assumption signal of uniform dark region in image considered as noise
Example SNR over urban area SNR std = 1 SNR std = 2 SNR std = 5 SNR std = 10 source: Schowengerdt 2006
Image based SNR HyMap sensor SNR1 = July 28, 2004, Millingerwaard; SNR2 = Aug 2, 2004, Millingerwaard; SNR3 = Aug 2, 2004, Wageningen Question How do you assess the quality of the three HyMap flight lines and why?
Standards for quality assessment Quality assessment becomes integral part of recent launched RS sensors (good examples MODIS and MERIS): Organize and finance quality assessment (incl. validation measurements) evaluate and document the scientific quality of products with respect to their intended performance Quality parameters are stored as product metadata and as per-pixel information Automated and standardized processing of RS data Update with state-of-the-art processing algorithms (e.g., collection 5 for MODIS) Provide documentation, tools, updates via web portal
acquisition Pre-processing & product levels Level 0 Product raw data Laboratory Calibration (radiometric and spectral), Vicarious Validation System Correction & Radiometric Calibration Attitude Data, Position Data, DEM Radiative Transfer Model, Atmospheric Variables, Topographic Variables preprocessing Level 1 Product Geometric Correction Atmospheric Correction Level 2 Product at-sensor radiance data Level 2a Product Level 2b Product Atm. corrected data image analysis statistical or physical models & validation variables and application Level 3 Product thematic variables mapped on uniform space-time grid scales
Quality assessment for MODIS-LAI
Validation and quality assessment Definition: validation is the process of assessing by independent means the uncertainties of the data products derived from the system outputs. General Approaches: Direct Comparison with Independent Correlative Measurements Ground-based Networks Comprehensive Test Sites Field Campaigns Comparisons with Independent Satellite Retrievals Basic Stages: Pre-launch emphasis on algorithm development and characterization of uncertainties from parameterizations and algorithmic implementation Post-launch emphasis on algorithm refinement and data product assessments source: EOS validation program
Validation in progress Example of fieldwork activities: field spectroscopy, hemispherical camera, species composition Example of flux tower in the field (above) and distribution of RS based observation networks over the globe (left)
Validation: an example for LAI 2. sampling in 1 x 1 km area 3. comparison of MODIS based LAI with field LAI 1. study area: savanna source: Tian et al., 2002
Comparison of EO data products Comparison of CYCLOPES, ECOCLIMAP,GLOBCARBON, and MODIS LAI with spatialized ground measurements collected over various vegetation types : crop (red), grass (orange), Evergreen Needleleaf forest (light green), Evergreen Broadleaf Forest (dark green), and Mixed (ENF+DBF) Forest (magenta) (from Garrigues et al., 2007) source: http://postel.mediasfrance.org/en/projects/r&d/cyclopes/
Summary Quality assessment is becoming integral part of the remote sensing data analysis chain Data model approach allows systematic investigation of errors and uncertainties in RS data analysis chain Information on RS image and product quality can be extracted from: visual assessment statistical assessment Product documentation and metadata Validation
Web sources http://landweb.nascom.nasa.gov/cgi-bin/qa_www/newpage.cgi: MODIS Land Quality Assessment Site http://www.ncaveo.ac.uk/: Background on calibration and validation http://wgcv.ceos.org/wgcv/wgcv.htm: CEOS working group on calibration and validation http://eospso.gsfc.nasa.gov/ftp_docs/program_defs.pdf: EOS validation program
Recommended reading Schowengerdt, R.A. (2006). Remote sensing Models and Methods for Image Processing. Third edition. Elsevier. Chapter 4: Data models Jensen, J.R. (2004). Introduction Digital Image Processing. Prentice Hall. Chapter 4: Initial Statistics Extraction
Statistical formulas mean: variance: standard deviation: rangek k n i 1 k BV n ik max min n i1 var sk k k BV ik k n 1 k var 2 k mode: value that occurs most frequently in a distribution median: value midway in the frequency distribution skewness kurtosis k k 1 n n i 1 n i1 BV BV k s ik n k k s ik k 4 3 3