Uncertainties in ocean colour remote sensing

Transcription

1 NOWPAP / PICES / WESTPAC Joint Training Course on Remote Sensing Data Analysis Introduction and recent progress in ocean color remote sensing part I: Uncertainties in ocean colour remote sensing Roland Doerffer Helmholtz Zentrum Geesthacht Institute of Coastal Research roland.doerffer@hzg.de IOCCG International Ocean Colour research Coordination Group Helmholtz Zentrum Geesthacht (was GKSS) Geesthacht is close to Hamburg, North Germany Institute of Material Research Institute of Polymer Research Institute of Coastal Research Staff: ca. 75 furthermore on the campus: Brockmann Consult 1

2 Why this lecture? Coastal colour remote sensing is a difficult task, it can lead to failures and large uncertainties. Conditions for failures have to be identified. Uncertainties have to be quantified on a pixel by pixel bases. Certainties and Uncertainties: overview The variability of TOA radiance spectra Factors influencing the TOA radiance spectra The variability of water reflectance Uncertainty due to the bio-optical model Sensitivity, ambiguities and co-variances Strategies to determine out of scope conditions and uncertainties Comparison with in situ data Conclusions 2

3 Certainties and Uncertainties The variability of TOA radiance spectra Factors influencing the TOA radiance spectra The variability of water reflectance Uncertainty due to the bio-optical model Sensitivity, ambiguities and co-variances Strategies to determine out of scope conditions and uncertainties Conclusions Basic principles of Water Color RS sensor sun suspended particles phytoplankton pigment gelbstoff air molecules aerosols gases - atmospheric scattering and absorption - reflection and refraction - scattering and absorption by water and its constituents - bottom reflection 3

4 Phytoplankton Photos by Marion Rademaker Case 1 water algorithm based on reflectance ratio model R445 / R555 C =a[r(445) / R(555)] b Case 1 water: Morel / Antoine MERIS Case 1 water ATBD 4

5 The Biosphere as seen by SeaWiFS Organic matter plume Coccolithophorides Chatonella bloom suspended sediment source: NASA GSFC and ORBIMAGE Suspended Matter and Phytoplankton in Coastal Water 5

6 Radiance (Wm-2 sr-1 µm-1) 211/11/3 Spectral Variability top of atmosphere Pin Pin Pin Pin Pin Pin 4 The Information: radiance spectra at top of atmosphere TOA Radiance Spectra North Sea wavelength (nm) Pin 1 Pin 2 Pin 3 Pin 4 Pin 5 Pin 6 Aux data: Sun zenith angle Sun azimuth View zenith View azimuth Surface pressure Ozone Wind Expected ranges Knowledge about optical properties 6

7 Radiance (Wm-2 sr-1 µm-1) Pin 1 Pin 2 Pin 3 Pin 4 Pin 5 Pin 6 211/11/3 What determines the radiance spectrum at TOA TOA Radiance Spectra North Sea wavelength (nm) air molecules different aerosols thin clouds contrails Sky reflectance Sun glint foam floating material Suspended particles dissolved organic matter Vertical distribution chlorophyll different phytoplankton species Bottom reflection The dominant signal: Path radiance reflectance over coastal water Contrails Clouds haze 7

8 Rlw / Rltoa (%) Radiance reflectance (sr-1) Radiance reflectance (sr-1) Radiance reflectance (sr-1) Radiance reflectance (sr-1) 211/11/3 Uncertainties in atmospheric correction over coastal water MERIS NorthSea MERIS North Sea Rltoa, Rlpath, Rlw Rltoa, Rlpath, Rlw 7.E-2 7.E-2 6.E-2 6.E-2 5.E-2 5.E-2 4.E-2 3.E-2 Pin 1 Rltoa Pin 1 Rlpath Pin 1 Rlw 4.E-2 3.E-2 Pin 4 Rltoa Pin 4 Rlpath Pin 4 Rlw 2.E-2 2.E-2 1.E-2 1.E-2.E wavelength (nm).e Wavelength (nm) 6.E-2 MERIS North Sea Rltoa, Rlpath, Rlw 9.E-2 MERIS North Sea Rltoa, Rlpath, Rlw 5.E-2 8.E-2 4.E-2 3.E-2 Pin 3 Rltoa Pin 3 Rlpath Pin 3 Rlw 7.E-2 6.E-2 5.E-2 4.E-2 Pin 6 Tltoa Pin 6 Rlpath Pin 6 Rlw 2.E-2 3.E-2 1.E-2 2.E-2 1.E-2.E+.E wavelength (nm) wavelength (nm) Contriubtion by the water leaving radiance reflectance Rlw / Rltoa % MERIS North Sea Pin 1 Rlw/Rltoa Pin 2 Rlw/Rltoa Pin 3 Rlw/Rltoa Pin 4 Rlw/Rltoa Pin 5 Rlw/Rltoa Pin 6 Rlw/Rltoa wavelength (nm) 8

9 Uncertainties in atmospheric correction over coastal water Low reflectance in red or blue spectral range High reflectance in red and NIR part of the spectrum Extrapolation problem for standard AC procedures -> negative reflectances Thin or subpixel clouds and cloud shadows: cloud detection problem Certainties and Uncertainties The variability of TOA radiance spectra Factors influencing the TOA radiance spectra The variability of water reflectance Uncertainty due to the bio-optical model Sensitivity, ambiguities and co-variances Strategies to determine out of scope conditions and uncertainties Conclusions 9

10 RLw (sr-1) 211/11/3 Water leaving radiance reflectance spectra in coastal water 2.5E-2 4.5E-3 4.E-3 2.E-2 3.5E-3 1.5E-2 3.E-3 2.5E-3 1.E-2 Pin 6 2.E-3 Pin 5 1.5E-3 5.E-3 1.E-3 5.E-4.E E E-3 1.2E-2 4.5E-3 4.E-3 1.E-2 3.5E-3 8.E-3 3.E-3 2.5E-3 2.E-3 1.5E-3 Pin 1 6.E-3 4.E-3 Pin 2 1.E-3 5.E-4 2.E-3.E+.E E-3 2.E-3 1.8E-3 5.E-3 1.6E-3 4.E-3 1.4E-3 1.2E-3 3.E-3 2.E-3 Pin 3 1.E-3 8.E-4 6.E-4 Pin 4 1.E-3 4.E-4 2.E-4.E+.E Variability of water leaving reflectance spectra 2.5E-2 Water leaving radiance reflectance spectra MERIS North Sea E-2 1.5E-2 1.E-2 Pin 1 Pin 2 Pin 3 Pin 4 Pin 5 Pin 6 Uncertainties: Calibration of sensor Atmospheric correction 5.E-3.E wavelength (nm) 1

11 The inverse problem reflectance spectrum inverse model pure water phytoplankton suspended matter dissolved org. matter sun zenith view zenith azimuth diff forward model sun zenith view zenith azimuth diff Matrix inversion Inversion by optimization Inversion by neural network Success depends on: Bio-optical model ambiguities Inverse Modellierung using Optimization Procedures Start values Modell Parameters IOP / Konz Reflexion- Spektra Satellite Radiative transfer modell Reflexion- Spektra simulated Do spectra agree? yes Parameters are the IOPs / Konz. Change Parameters Determine Search direction Downhill in cost function no Test = (Rsim(i) Rsat(i)) 2 11

12 Simplified scheme of NN Algorithm Input layer: reflectances and angles R 1 Hidden layer Output layer: concentrations Chlorophyll R 2 R 3 W Susp. Matter Gelbstoff 3 y s( d w s( c v s( b u x ))) l l kl k jk j ij i k 1 j 1 i X 1 X o 2 s( bias w i x i ) incominglinks W i X i o X 3 X 4 Certainties and Uncertainties The variability of TOA radiance spectra Factors influencing the TOA radiance spectra The variability of water reflectance Uncertainty due to the bio-optical model Sensitivity, ambiguities and co-variances Strategies to determine out of scope conditions and uncertainties Conclusions 12

13 a (m -1 )) norm chlorophyll a [mg/l] 211/11/3 Uncertainties due to the bio-optical model Optically relevant variables in the water Optical components Proxy concentration with IOPs and variability variable Scattering material Absorbing material Phytoplankton pigments Scattering by all particles Absorption by organic material Absorption by phyto pigments Dry weight of suspended matter [ g m-3] Concentration of DOC / POC [mg m-3] Concentration of chlorophyll a [mg m-3] Uncertainties due to variability of optical properties Heincke187 a443 pigment n= wavelength (nm) Normalized absorption spectra of North Sea phytoplankton Summer period a(443) (m -1 )) Variability in the relationship between a_pig and chlorophyll concentration Conversions: Chl. a [mg m-3] = 21 * a_pig_442 ^1.4 13

14 b at 44 nm (m -1 )) bb4 44 (m -1 )) 211/11/3 TSM scattering, H H187 ac-9 scattering all ok stations 1 Heincke 187 all tsm [mg/l] AC tsm (mg/l) BB-4 Conversions: TSM [ g m-3] = 1.72 * b_tsm_442 Certainties and Uncertainties The variability of TOA radiance spectra Factors influencing the TOA radiance spectra The variability of water reflectance Uncertainty due to the bio-optical model Sensitivity, ambiguities and co-variances Strategies to determine out of scope conditions and uncertainties Conclusions 14

15 RLw [sr -1 ] Rsr (sr-1) Rsr (sr-1) 211/11/3 Sensitivity at different concentration ranges and spectral bands 9.E-3 8.E-3 7.E-3 Remote Sensing reflectance TSM 1 6.E-2 5.E-2 Remote Sensing reflectance TSM 1 6.E-3 4.E-2 5.E-3 4.E-3 3.E-3 Rsr 5_1_1 Rsr 1_1_1 3.E-2 2.E-2 Rsr 5_1_1 Rsr 1_1_1 2.E-3 1.E-3 1.E-2.E E wavelength (nm) wavelength (nm) Chl. 5/1 mg m-3 TSM 1 g m-3 ays(443).1 m-1 Chl. 5/1 mg m-3 TSM 1 g m-3 ays(443).1 m-1 Sensitivity at different concentration ranges and spectral bands RLw for MERIS bands 1 (412 nm), 6 (56 nm), 1 (78 nm) band 1 = 412 nm band 5 = 56 nm band 9 = 78 nm Sensitivity of the reflectance at a spectral band depends on the concentration To cover a large concentration range many bands from the blue to NIR range are necessary a_gelb_44:.2, a_part_44: SPM/25, pig: 2 mg m SPM [g m -3 ] 15

16 water depth m nn-derived µg/l 211/11/3 Signal depth at different spectral bands Multiband algorithms: the information for each band may come from a different water layer turbid coastel ocean Signal depth z9 z 9 =1/k coastal: TSM=5 mg/l Chlor.=5µg/l Gelb=a 38 =1m -1 open ocean: Chlor.=1µg/l -5 pure water wavelength µm Uncertainties due to ambiguities for different concentration mixtures 1 2 case 2 water chlorophyll retrieval with NN model chlorophyll µg/l All cases of turbid water 2 mg m-3 16

17 nn µg/l 211/11/3 Ambiguities Pigment in North Sea Water (gelb <.2 m-1, MSM < 5 mg/l) model µg/l Typical North Sea coastal water: ay_443: <.2 m-1, TSM < 5 mg /l Certainties and Uncertainties The variability of TOA radiance spectra Factors influencing the TOA radiance spectra The variability of water reflectance Uncertainty due to the bio-optical model Sensitivity, ambiguities and co-variances Strategies to determine out of scope conditions and uncertainties Conclusions 17

18 Strategies to determine uncertainties for coastal water 2 Problems (1) Reflectance spectrum is out of scope of our atmosphere or water model Concentrations Optical properties Other components present, not included in bio-optical model Vertical distribution and sub-pixel patchiness Surface effects (floating cyanobacteria) Bottom effects Contamination by subpixel clouds, contrails, small islands etc. (2) Reflectance spectrum is within scope, but: Uncertainty of a component varies with the concentration of the other components Transition between both problems 2 Requirements Procedure to detect / quantify out of scope conditions Procedure to quantify uncertainty on a pixel by pixel basis Strategies to determine uncertainties for coastal water: Solutions Out of scope detection Check of auxilary variabels, e.g. windspeed -> whitecaps Check of reflectance in particular bands: NIR reflectance for floating material Auto-associative neural network Combination of backward and forward neural network (standard MERIS processing) Convergence of optimization procedure on high deviation level Ambiguities in bio-optical / reflectance model Analysis of simulated data using the bio-optical model Uncertainties on a pixel by pixel basis Empirical from observations Variations in optimization procedure Determination using variations of simulated data set -> look up table, NN 18

19 Detection of out of scope conditions 2 Procedures have been developed Combination of an inverse and forward Neural Network Use of an autoassociative Neural Network Both produce a reflection spectrum, which is compared with the input spectrum Deviation between input and output spectrum can be computed as a chi2 A threshold can be used to trigger an out of scope warning flag Combination of inverse and forward NN R Input Inverse NN IOPs Forward NN R output Auto-associative NN R Input autoassociative NN R output Detection of out of scope conditions (MERIS processor) r g Reflectance spectrum r g Angles: Sun zenith View nadir Azimuth diff invnn r, r log of reflectances c log of concentrations g geometry information q quality indicator c IOPs or concentrations forwnn If RLw <.9, RLw =.9 ( ρ.3) r NN input NN output q c q c Reflectance spectrum Output of NN Comparator to trigger out of scope flag C: log(b_tot) log(a_ys+a_btsm) log(a_pig) All at 443 nm Out of scope flag flag PCD1_16,17 q true if sum (r(i) / r (i)) > 4. 19

20 Detection of out of scope conditions (MERIS processor) Top of atmosphere radiance spectra at normal and critical locations Detection of out of scope conditions (MERIS processor) Exeptional bloom, Indicated by high Chi_square value Chi_square is computed by comparing The input reflectance spectrum with the output of the forward NN 2

21 Detection of out of scope conditions using an aann Important to detect toa radiance specta which are not in the simulated training data set These are out of scope of the atmospheric correction algorithm Autoassociative neural network with a bottle neck layer Functions also as nonlinear PCA i.e. bottle neck number of neurons Provide estimate of Independent components Input layer Hidden 1 Bottleneck Hidden 3 output layer For the GAC training data Set of ~ 1Mio. Cases Bottleneck minimum was 4-5 Detection of out of scope conditions aann: example for L1 (TOA) data High SPM Sun glint Transect 21

22 Detection of out of scope conditions aann: example AutoNN test 12x5x12 Yellow Sea transect, MERIS band 7, nm AutoNN test 12x5x12 Yellow Sea transect, MERIS band 7, nm rltoa rltoa_ann difference ratio rel. deviation RLtoa longitude.1 18 AutoNN test Yellow Sea transect 12x5x12, MERIS band 7, nm longitude significant deviation in area with high SPM concentrations, but not in sun glint area rel. frequency Histogram of deviations shows 2 maxima, around 1 in sun glint.9 in high SPM area, which out of scope rel. radiance reflectance ratio to true Computation of uncertainties on pixel-by-pixel basis RLtosa RLath trans fornnatmo 3 par Compare RLtosa RLtosa RLtosa Select Bands Variables parameters constraints RLw fornnwater n_lam n_var Variables: 3 Atmo + 4 Water fixed: 3 angles, T, S Optimization Procedure LM constrained Min/max-> constraints RLW, kd uncertainties fornnwater Results: RLw IOPs uncertainties 22

23 deviation deviation 211/11/3 Searching for minimum: principle, 1D case Search for minimum: Deviation between measured and simulated spectrum Acceptance level Acceptance level Independent variable Independent variable Width can be estimated from the 2nd order derivative (Hessean matrix) Parameter sun zenith angle viewing nadir angle azimuth difference water temperature salinity Parameters Value Aerosol optical thickness at 55 nm.2 Angstrom coeffcient deg 32 deg 5 deg 15 deg 35 psu Wind Speed 3.7 m s-1 absorption coefficient of particulate matter at 443 nm absorption coefficient of yellow substance at 443 nm absorption coefficient of phytoplankton pigments at 443 nm scattering coefficient of particulate matter at 443 nm.1 m-1.6 m-1.2 m m-1 23

24 reflectance 211/11/3 Water and TOA spectrum Simulated test spectrum wavelength [nm] Comparison of TOA reflectances wavelength simulated Rtoa retrieved Rtoa Deviation %

25 Water reflectances wavelength Rw retrieved Rw true Deviation Deviation % E E E E E E E E E Results for independent variables AOT Angstrom wind a_part a_y a_pig true retrieved varaible AOT Angstrom wind a_part a_y a_pig b_part AOT E E E E-7 Angstrom 3.135E wind a_part a_y 3.32E a_pig b_part E Co-variance matrix aot ang wind apart agelb apig btot 6.324e e-3 4.6e e e e e-3 Stdev 25

26 Stdev of independent variables variable - 1 stdev mean +1 stedev TRUE AOT 2.E-1 2.1E-1 2.2E-1.2 angstrom 1.3E+ 1.35E+ 1.31E+ 1.3 wind 3.693E E E+ 3.7 a_part 5.161E E E-1.1 a_y 5.941E E E-1.6 a_pig 1.955E-1 2.5E+ 2.57E-1.2 b_part 2.9E+ 2.99E+ 2.18E+ 2.1 Uncertainties in water reflectances wavelength Rw Rw_min Rw_max difference uncertainty %

27 error % RLw [sr-1] 211/11/3 Simulated reflectance test simulated reflectance test error.1 stdev wavelength [nm] rlw_true rlw_err rlw_retr True spectrum simulated measured spectrum = true * random error Retrieved spectrum when LM has found solution Error of retrieved spectrum Error of retrieved RLw spectrum error rel to measured and to true wavelength [nm] error% error_true 27

28 Results and errors of retrieval Variable conc true conc retr. stdev of log_conc err. estimated % err true % chlorophyll [mg m-3] detritus [g m-3] gelbstoff a443 [m-1] min. SPM [g m-3] kdmin_true:.296 kdmin_ret:.289 error: -.33% kd49_true:.2636 kd49_ret:.269 error: -1.4% Determine uncertainties on a pixel by pixel basis r, r log of reflectances c log of concentrations g geometry information q quality indicator NN input NN output r adjust c r g g invnn c forwnn r q c q c iterate: Min! 28

29 Determine uncertainties on a pixel by pixel basis II 1 a(gelb+part) 5 conc_bp conc_pig chl conc_gelb tsm Certainties and Uncertainties The variability of TOA radiance spectra Factors influencing the TOA radiance spectra The variability of water reflectance Uncertainty due to the bio-optical model Sensitivity, ambiguities and co-variances Strategies to determine out of scope conditions and uncertainties Comparsion with in situ data Conclusions 29

30 Validation Determine the difference between remote sensing data and and reference data Accuracy: absolute difference Precision: scatter of differences Strategies: Comparison with other satellite data Comparison with simulated data Comparison with in situ data (match up, transects, statistics) Comparison with climatological data Problem: error / representativity of reference data Instruments used for validation 3

31 Uncertainties related to comparison with in situ data Error in method and handling, e.g. HPLC for chlorophyll determination Sample not representative for water volume of pixel Vertical distribution: water comes from a certain depth, e.g. 4 m for FerryBox Temporal difference between sample and satellite overpass Sub-pixel patchiness Scatter in bio-optical data, e.g. relationship between concentration and IOPs Bio-optical model: relationship between a_pig and chl_a (443 nm) 443 nm, log1 scale 31

32 Bio-optical model: relationship between a_pig and chl_f 443 nm, log1 scale, samples for chl_f Bio-optical model: relationship between a_pig and chl_a (665 nm) 665 nm, log1 scale 32

33 Relationship chl_f and backscattering coefficient 443 nm, 249 samples, log1 scale Test of NN based on measurements for chlorophyll Log1 scale, red: 1 by 1 line 33

34 Comparison of histograms: measured, NN computed NN for kd489 34

35 Histogram kd489 measured and NN derived Certainties and Uncertainties The variability of TOA radiance spectra Factors influencing the TOA radiance spectra The variability of water reflectance Uncertainty due to the bio-optical model Sensitivity, ambiguities and co-variances Strategies to determine out of scope conditions and uncertainties Conclusions 35

36 Summary and conclusions Uncertainty in coastal water products can be large due to the large number of factors in atmosphere and water, which determine the reflectance spectrum Conditions where algorithms (AC & water) fail Prerequisite for a successful retrieval are optical models of the atmosphere and the water, which meet the actual conditions Regional models might be necessary Reflectance spectra have to be tested if they are within the scope of these models Out of scope spectra have to be flagged, treated with special algorithms or excluded from further processing Limited sensitivity of reflectance spectrum and ambiguities lead to an uncertainty even for spectra, which are in scope Uncertainties have to be quantified on a pixel-by-pixel basis Validation in coastal waters by match up in situ samples can be difficult due to patchiness and fast changes Uncertainties in in situ match up data have to be quantified Validation should be complemented by statistical analysis of larger areas, transects and time series Acknowledgemets Supported by various projects ESA Case 2 Water Regional Algorithm ESA Glint correction ESA Water radiance ESA CoastColour ESA Dragon DLR DeMarine Neural Network Training software H. Schiller, GKSS MERIS data provided by ESA Implementation of C2R and Glint correction in BEAM: M., Brockmann-Consult 36