IOCS San Francisco 2015 Uncertainty algorithms for MERIS / OLCI case 2 water products

Transcription

1 IOCS San Francisco 2015 Uncertainty algorithms for MERIS / OLCI case 2 water products Roland Doerffer Brockmann Consult

2 The problem of optically complex water high variability of optical properties of water constituents more factors which determine top of atmosphere reflectances than can be inversely retrieved the model behind the retrieval algorithm can represent only a fraction of the actual conditions thus: the conditions and corresponding reflectance spectrum can be out of scope of the model and even when in scope ambiguities and masking effects can lead to significant uncertainties Requirement for MERIS / OLCI case 2 water algorithm: flags when NN inputs or outputs are at the limit of the training range co-algorithm to determine spectra, which are out of scope co-algorithm to determine uncertainties

3 OLCI and MERIS case 2 water algorithms MERIS (4th reprocessing) and OLCI standard products of case 2 water are based on 2 neural network algorithm systems atmosphere NN: Rtosa -> Rw (MERIS 12 / OLCI 14 spectral bands) water NN: Rw (9 / 11 bands) -> 5 IOPs (apig, ad, ag, bp, bw) the bio-optical model for water is based on parameters derived from the NOMAD data set (which has been extended to case 2 waters) concerning: IOPs, frequency distributions, co-variances. atmosphere model is based on aerosol optical properties derived from coastal stations of the AERONET large training data set of > 1 Mio. cases, simulation with a SOS atmosphere model (Zagolski, F., Santer R., Aznay O., 2007), and Hydrolight (Mobley, 1994) for water procedures for 3 types of checks: flag if inputs or outputs of the NNs are at the lower or upper limit of the traning data set flag if input spectrum is out of scope: test with auto-associative NN for atmosphere and a combination of an inverse and forward NN for water determination of IOP uncertainties of the output of the NN with respect to the training data set

4 Radiance (Wm-2 sr-1 µm-1) A manifold of factors determine the radiance spectrum at top of atmosphere TOA Radiance Spectra North Sea wavelength (nm) Pin 1 Pin 2 Pin 3 Pin 4 Pin 5 Pin 6 air molecules different aerosols thin clouds contrails Sky reflectance Sun glint foam floating material Suspended particles dissolved organic matter Vertical distribution chlorophyll Bottom reflection different phytoplankton species

5 Information content of TOA radiances: PCA and aann L1b TOA radiance spectra of transect L1b TOA Eigenvalues of radiance spectra radiance [W m-2 sr-1 µm-1] eigenvalue /sum of eigenvalues wavelength [nm] MERIS band order of eigenvalues Transect North Sea TOA radiance TOA radiance spectra along transect Principle Component Analysis: only 3 significant factors Auto-associative NN using a bottle-neck layer functions as a non-linear PCA For this transect 7 neurons are sufficient to represent the variability of TOA radiances Input layer Hidden 1 Bottleneck mean rms error Hidden 3 output layer aann MERIS L1b transect no. neurons

6 Detection of out of scope conditions 2 Procedures have been developed for MERIS / OLCI: Combination of an inverse and forward Neural Network Use of an autoassociative Neural Network Both produce a reflection spectrum, which is compared with the input spectrum Deviation between input and output spectrum can be computed as chi2 A threshold can be used to trigger an out of scope warning flag Combination of inverse and forward NN R Input Inverse NN IOPs Forward NN R output Auto-associative NN R Input autoassociative NN R output

7 Detection of out of scope conditions aann: example for L1 (TOA) data High SPM Sun glint Transect MERIS scene of the Yellow Sea

8 Detection of out of scope conditions aann: example AutoNN test 12x5x12 Yellow Sea transect, MERIS band 7, nm AutoNN test 12x5x12 Yellow Sea transect, MERIS band 7, nm rltoa rltoa_ann difference ratio rel. deviation RLtoa longitude longitude significant deviation in area with high SPM concentrations, but not in sun glint area rel. frequency AutoNN test Yellow Sea transect 12x5x12, MERIS band 7, nm Histogram of deviations shows 2 maxima, around 1 in sun glint 0.9 in high SPM area, which out of scope rel. radiance reflectance ratio to true

9 Uncertainties due to the bio-optical model Optically relevant variables in the water Optical components with IOPs and variability Proxy concentration variable / product Scattering material Scattering by standard particles (bp) Scattering by white particles (bw) Dry weight of suspended matter [ g m-3] Absorbing material Phytoplankton pigments Absorption by gelbstoff (ag) Absorption by detritus (ad) Absorption by phyto pigments (apig) adg_443 [m-1] Concentration of chlorophyll a [mg m-3]

10 Uncertainties due to variable relationship between a_pig and chlorophyll 443 nm, log scale, samples for chl_f, NOMAD data set

11 nn-derived µg/l nn µg/l Uncertainties due to ambiguities for different concentration mixtures 2 case 2 water chlorophyll retrieval with NN Pigment in North Sea Water (gelb < 0.2 m-1, MSM < 5 mg/l) mg m model chlorophyll µg/l All cases of turbid water model µg/l Typical North Sea coastal water: ay_443: < 0.2 m-1, TSM < 5 mg /l

12 Tests of NNs with different assumed uncertainties of reflectances Test of a_pig, no additional error Test of adg, no additional error Test of a_pig with an extra random error with a standard deviation of 3% Test of adg with an extra random error with a standard deviation of 3%

13 Determination of uncertainties This procedure determines uncertainties if the data are within the scope of the model due to ambiguities and masking effects test of simulated with the inverse neural network test with different assumed uncertainties of reflectances compute deviations between output of the NN and the corresponding data of the model are included in look-up table train a NN with the 5 IOPs as input and the deviations of the 5 IOPs as output: apig, ag, ad, bp, bw further NNs are trained with the 5 IOPs as input and the combined 3 products as output: absorption by phytoplankton pigments (apig / chlorophyll), absorption by yellow substance / gelbstoff and detritus (agd), scattering by standard and white particles (btsm / TSM)

14 Helgoland transect C27 Rtosa Rpath transect aann

15 Rw water reflectance 443 nm 560 nm 756 nm

16 Helgoland transect C27 atotal_443 TSM red dots: water samples, blue dots: MERIS data

17 Helgoland transect C27 apig_443 (blue dots) and uncertainties (green dots) MERIS data chlorophyll and uncertainties of water samples (red dots with bars) and MERIS data (blue dots with bars)

18 Summary and conclusion Problem In most case 2 waters we have more variables, which determine TOA radiances, than we can retrieve Importance of variables in water change with their concentrations and variables can mask each other and spectra might be ambiguous wrt. the IOPs or concentrations of water constituents The model behind a case 2 water algorithm is a simplification, reflectance spectra can occur, which are outside the scope of the model Requirement Thus, we have to check if a TOA or water reflectance spectrum is in scope of the model used for the algorithm and we have to determine the uncertainty of the retrieved IOPs due to ambiguities and masking effects Solution for MERIS / OLCI Co-algorithm to identify out of scope spectra by using an aann or combination of inverse and forward IOP-Rw NN uncertainty NN which is trained with the deviations between the IOPs of the simulated test data set and the IOPs as output of the inverse NN Open problem validation of the uncertainty NN with field data