Principal Component Inversion

Size: px

Start display at page:

Download "Principal Component Inversion"

Domenic Floyd
6 years ago
Views:

1 Prncpal Component Inverson Dr. A. Neumann, H. Krawczyk German Aerospace Centre DLR Remote Sensng Technology Insttute Marne Remote Sensng Prncpal Components - Propertes The Lnear Inverson Algorthm Optmsaton of the PCI: Segmentaton and Sem-logarthmc Approach Implementaton Scheme PCI and Atmospherc Correcton Examples Introducton The classcal way to develop ocean colour algorthms s to correlate expermental data sets,.e. n-stu analyses of concentratons wth remote sensng measurements emprcal algorthms, usually band ratos work good under case-1 condtons ths approach fals f we want to develop algorthms for spectral hgh resoluton data (large number of channels) and/or several ndependently but smultaneously varyng geo-physcal parameters multdmensonal, multvarate approach s necessary lookng for an algorthm senstve to the shape of the spectrum 1

Prncpal Components PCs are an effectve tool handle hgh dmensonal and correlated data, to analyse

PCs s problematc, there s no drect lnk to (geo-) physcal parameters some physcal understandng can

..) egenvectors and therewth the PCs depend on the covarance matrx, therefore the egenvector system

mathematcal tool to develop an algorthm for the physcal nterpretaton of multvarate, hgh dmensonal

2 Prncpal Components PCs are an effectve tool handle hgh dmensonal and correlated data, to analyse the nformaton content and dmensonalty of multvarate data sets however, a physcal nterpretaton of PCs s problematc, there s no drect lnk to (geo-) physcal parameters some physcal understandng can be ntroduced through the correspondng egenvectors (brghtness, greenness,...) egenvectors and therewth the PCs depend on the covarance matrx, therefore the egenvector system wll be dfferent for every scene or data set to be analysed the dea s to use the PCA as a mathematcal tool to develop an algorthm for the physcal nterpretaton of multvarate, hgh dmensonal data whch can be optmsed for dfferent water types 2 Prncpal Components - Example (I) 1. PC 2. PC Frst 4 PCs of MOS-IRS data over Scla 3. PC 4. PC 3

3 Prncpal Components - Example (II) Egenvalues and Egenvectors of the MOS-IRS Scla Scene 25 0,6 20 0,4 Egenvalue , ,2 1. EV 2. EV 3. EV Egenvalue No. -0,4-0,6 channel no. 4 The Remote Sensng Problem R( θ, φ) 1 Weghtng C matrx 1 R( θ, φ) 2 R( θ, φ) 3 R( θ, φ) 4... R( θ, φ) n G A Weghtng matrx 2 Weghtng matrx n S... Y! The nverse task s analytcally not solvable? Whch and how many parameters can be retreved? Whch and how many channels contrbute how much to each parameter {I} Knowns (nput) Transformaton/ Mappng {U} Unknowns (derved parameters)? How to construct an optmal algorthm 5

4 The Lnear Inverson Algorthm Basc deas developed and evaluated for MOS-IRS, operatonal mplementaton for MERIS goal: quanttatve retreval of the man components for case-2 remote sensng usng a lnear estmator: pˆ = k L + A where pˆ - estmate of the geophyscal parameter, e.g. chlorophyll, Gelbstoff and sedments k - weghtenng coeffcent n band for parameter L - measured radance/reflectance n band A - offset value for parameter - spectral band number, from 1 to N (1) 6 Step 1 - Forward Modellng The prncpal Component Inverson algorthm was developed usng smulated data sets contanng radance vectors and the correspondng geo-physcal parameters: {p } {L }, from 1 to N forward model the p are Chlorophyll concentraton C, Gelbstoff attenuaton a y (440nm), sedment scatterng b s (550nm) and aerosol-optcal thckness τ A (750), varyng ndependently and smultaneously bo-optcal model after Morel/Preur/Sathyendranath b R = V n the begnnng smplest atmospherc modellng after Gordon and Sturm usng Angstroem-law to descrbe the Aerosol B a Important remark: the used forward model s not a prncple ssue of the algorthm, any other forward model can be used! 7

5 Step 2 - Prncpal Component Analyss (PCA) The forward model generates a large set ( ) of vectors {C, a y, b s, τ A, L 1..L N } varablty ranges and, possbly correlaton between sngle parameters, are chosen correspondng to the area/season of nterest, specfc optcal propertes of water consttuents can be accounted for by the IOPs n the bo-optcal model Note: dependng of the goal other combnatons of parameters are possble, atmosphere may be absent when developng algorthms for BOA-measurements or atmosphercally corrected data, also reflectance values may be used a PCA as descrbed earler s appled to the radance values yeldng {PC k } and the correspondng egenvalues λ k, k=1..n 8 Step 3 - Instrnsc Dmensonalty The ntrnsc dmensonalty of the smulated radance data set now can be determned as: D = max(k) wth λ k >>1, D<N. thus, two groups of PCs are separated ones representng useful measurement nformaton (k 4 D) and ones contanng non-nterpretable varatons due to nose, quantsaton error etc. For the followng analyses only the frst group of PCs s used. Ths enhances the stablty of the nverson, suppresses nose and reduces the data set to the usable nformaton n the mathematcal sense 9

6 Step 4 - Reverse Correlaton (I) The PCs represent the dentcal nformaton as the radance data, but n a mathematcally defned, orthogonal coordnate system, except the small nose-lke porton of nformaton that s suppressed by reducng the number of used components to the "sgnfcant" ones therefore t must be possble to retreve the geo-physcal parameters from the PCs as well, as long as t s possble at all: D pˆ ~ Cm PCm (2) m = 1 where C m s the correlaton coeffcent between the th parameter and the mth prncpal component ths s already close to an nverse relatonshp of equ. (1)! 10 Step 4 - Reverse Correlaton (II) Composng now a data set contanng the PCs and the correspondng physcal values we can use a regresson to determne the coeffcents C m pˆ p = Cm PCm σ m λ m for MSE mn (3) where pˆ p - estmate of the parameter - mean value of the parameter σ - varance of the parameter C m - correlaton coeffcent MSE - mean square error so we can estmate the parameters from prncple components - but ths s not what we really want... 11

7 Step 5 - Reverse Transformaton to Radances Snce the PCs are computed from the radance vectors one can substtute them by the nverse transformaton and get the regresson formula n terms of radances: pˆ p σ D U D 1 ( L Ll ) U 2 ( L L ) = C1 + C +... L λ L λ = 1 2 = 1 (4) Step 6 - Determnaton of Coeffcents from (4) we can compute the desred coeffcents k and A for the estmator n equ. (1) by regresson the computed coeffcent set s stored n a look-up-table (LUT) and represents the nverson algorthm for the stuaton correspondng to the modellng: geometry, IOPs, atmosphere, varablty ranges and correlaton propertes etc. by repeatng the analyss for dfferent smulatons, the algorthm can be adapted to geometry, specfc regons, seasons, speces compostons etc. 12 PCI Bascs - Summary PCI s a fast and easy to mplement algorthm, all tme-consumng computatons can be performed off-lne PCs as an orthogonal representaton of the data allows the multvarate nverson, what would be hard to do wth radances due to hgh spectral correlaton (ll-posed problem of nvertng Cov{L / L }) the approach allows to account for measurement accuracy PCI s adaptable to wde varety of specfcs and parameters and allows a detaled analyss of nfluencng factors 13

8 Implementaton Modelng L( λ) = f(c, S, Y, τ,...) Prncpal Component Analyss Reverse Correlaton, Transformaton to L Modellng/Parametrsaton (off-lne) K n Look-up table of weghtng coeffcents Radances, Aux. Informaton pˆ = k L + A Maps of C, S, Y Parameter Retreval 14 PCI - Practcal Consderatons PCI s an optmal strategy for lnear relatonshps - but the relaton between radances and geo-physcal parameters s non-lnear, especally for wde ranges of varablty PCI technque gves an optmal estmate n the sense of MSE,.e. works most optmal for the mean value of the parameters, at the borders the retreval errors wll be larger therefore addtonal optmzaton was developed: dvdng the complete varablty range nto subranges for whch separate coeffcent tables are computed realze a sem-logarthmc expresson of the parameters 15

9 Segmentaton Procedure Investgatons on smulated data showed, that for case-2 waters the sedment scatterng s the domnatng factor n BOA radance data therefore a herarchcal segmentaton s appled: the entre range of b s s dvded n 5 subranges wthn each of these subranges C an a y vary over ther total range respectvely further optmsaton can be acheved by addtonal segmentaton of C durng retreval the results for all subranges and all pxels are computed. Vald n a subrange mage are only pxel values wthn the defned subrange, the others are masked out. Supermposng the resultng submages gves the fnal parameter map 16 Sem-logarthmc Parameters Because the orgnal relatonshp between parameters and radances s non-lnear, t was found by numercal tests that the algorthm performance ncreases f the PCI s appled to auxlary parameters defned by q = p + a ln(p ) wth a= 0.1 ths accounts for non-lnearty for small parameter values and ncreases the retreval accuracy sgnfcantly 17

10 Implementaton Scheme Data set entry n Map THEN Valdty test f S <= S <= S m n m a x and C <= C <= C m n m a x C; S; Y Lnear estmaton Input rs (1-8) out of scope Flag ELSE ELSE If S <=10 M a x ELSE If CM a x <=30 THEN THEN S = S m n m a x S = S + S m a x m a x C =0,C = C m n m a x C = C m n m a x C = C + C m a x m a x Coeffzent set from Mn to Max Mn = (C, S, 0) M n M n Max = (C, S, Y ) M a x M a x M a x Coeffzent set from Mn to Max Mn = (0, 0, 0) Max = ( C, S, Y ) M a x 18 PCI and Atmospherc Correcton the PCI algorthm was developed especally for case-2 waters (.e. turbd, hgh scatterng, sometmes shallow) under these condtons the usual atmospherc correcton schemes fal because the black water condton s nvald one possble soluton s to treat water and atmosphere n an ntegrated nverson procedure, accountng for these effects for PCI we therefore smulated TOA radances, ncludng the atmosphere (aerosol) as an addtonal parameter n the forward model (cp. Step 2) the result s a retreval algorthm for water consttuents, that s drectly appled to TOA radances and does not need an extra atmospherc correcton, the nverson automatcally accounts for the atmospherc nfluence 19

11 Summary Wth PCI nverson we have a very stable and fast algorthm for quanttatve retreval of water consttuents from spectral hgh resoluton remote sensng data t can easly be adapted to dfferent sensors, water types or geophyscal stuatons cp. to NN the tranng s much smpler and faster the nverson s very stable wth respect to nose TOA PCI provdes a fast retreval procedure wthout atmospherc correcton for both case-1 and case-2 waters. The accuracy for the nstu data avalable for MOS s ~30% the optmsed (.e. segmented sem-log) BOA PCI promses extreme good accuracy, but stll needs to be evaluated for expermental data 20 Examples - Scatterplot of Smulated and Retreved Parameters, BOA-PCI 21

-1-1 Examples - Relatve Retreval Error Hstograms, BOA-PCI 22 Examples - TOA-retreved Parameters MOS-IRS, 06.03.

12 -1-1 Examples - Relatve Retreval Error Hstograms, BOA-PCI 22 Examples - TOA-retreved Parameters MOS-IRS, , Black Sea µg/l b (550 nm) /m S (750 nm) /m τa Colour Composte Chlorophyll Sedments aer.opt. Thckness MOS-C 1.6 µm 23

13 Examples - TOA-retreved Parameters MOS-IRS, , German Bght 24

Feature Reduction and Selection

Feature Reduction and Selection Feature Reducton and Selecton Dr. Shuang LIANG School of Software Engneerng TongJ Unversty Fall, 2012 Today s Topcs Introducton Problems of Dmensonalty Feature Reducton Statstc methods Prncpal Components