3D object recognition with photon-counting integral imaging using independent component analysis

Transcription

1 3D object recognton wth photon-countng ntegral magng usng ndependent component analss Cuong Manh Do Department of Electrcal & Computer Engneerng, Unverst of Connectcut, 371 Farfeld Road U-2157, Storrs, Connectcut USA ABSRAC he author presents an overvew of 3D object recognton wth photon-countng ntegral magng usng Independent Component Analss (ICA). Hgh resoluton elemental mages of 3D objects are captured at dfferent angles to allow object recognton at dfferent orentatons usng snthetc aperture ntegral magng (SAII). Generated photon-countng elemental mages are used to reconstruct the 3D mages at dfferent dstances from the camera lens usng a mamum a posteror estmaton method. he kurtoss mamzaton-based algorthm s appled as a non-gaussan mamzaton method to etract the ndependent features from the tranng data set. Hgh dmensonal data s pre-processed usng Prncpal Component Analss (PCA) to reduce the number of dmensons. he author demonstrates how ths method can effectvel recognze 3D objects despte a small epected number of photons. hs ma be mportant for low lght applcatons n medcal or other settngs. 1. INRODUCION Integral magng s a three-dmensonal (3D) technque whch was frst proposed b Lppmann n 1908 and has been a subject of nterest recentl [1-5]. In snthetc aperture ntegral magng (SAII), a camera s translated horzontall and vertcall to capture two-dmensonal (2D) elemental mages of a 3D object whch can be supermposed to reconstruct the 3D mages of the objects. It has been shown that 3D objects can be recognzed wth a small number of photons [16]. Applcatons of photoncountng can be found n low-lght applcatons, such as nght vson, medcal magng, and radologcal magng. 3D reconstructon can be mplemented b mamum lkelhood or mamum a posteror estmaton usng a selected subset of the elemental mages [6-10]. he ndependent component analss technque (ICA) has been appled n 3D mage vsualzaton and recognton to etract statstcall ndependent features from a gven tranng data set [11-14]. A number of 3D object recognton applcatons usng ICA have been ntroduced [15-16]. In ths paper, ICA s appled to etract ndependent features from reconstructed 3D mages of a 3D scene. Large-dmensonal data are reduced effcentl usng Prncpal Component Analss. he paper s structured as follows: secton 2 ntroduces an overvew about snthetc aperture ntegral magng and the photon-countng model. An overvew of the FastICA algorthm s presented n secton 3. Secton 4 presents the applcaton of ICA n recognton of 3D objects usng the photon-countng ntegral magng sstem. Secton 5 shows the eperment results, and secton 6 ncludes the concluson. 2. SYNHEIC APERURE INEGRAL IMAGING AND PHOON-COUNING MODEL In snthetc aperture ntegral magng, a camera s mounted on a two-dmensonal optcal stage and s shfted from the left to rght, top to bottom to capture an mage of an object at each poston. Each elemental mage contans partal nformaton of the whole 3D scene from a partcular perspectve. he reconstructed 3D mage s the supermposton of the magnfed elemental mages, wth the magnfcaton factor as the rato between the dstance from the lens to the objects and the dstance from the lens to the magng plane [6,14]. hs s epressed as follows: V. 3 (p.1 of 7) / Color: No / Format: Letter / Date: :14:33 PM

2 I(, ) 1 KL K L = k = 1 l= 1 I ( + kδ, + lδ ), where I(,) s a pel of the reconstructed mage at the coordnate (,). K and L are the number of elemental mages n horzontal and vertcal drectons, respectvel. I kl denotes an elemental mage. Δ and Δ are the horzontal and vertcal offsets between elemental mages: δ * p Δ ( pels) =, (2) m δ * p Δ ( pels) =, (3) m where δ and δ are the offsets n mm n the horzontal and vertcal drectons. p and p are the number of horzontal and vertcal pels of the detector, respectvel. m and m are the detector arra szes n mm horzontall and vertcall, respectvel. he magnfcaton of elemental mages ma requre a lot of memor durng computatonal reconstructon. he ssue s resolved b supermposng non-magnfed elemental mages and dvdng the offsets b the magnfcaton rato. Photon lmted mages can be smulated from rradance mages usng the Posson dstrbuton. Assumng that the mean photon-count at each pel s proportonal to the rradance of the pel n a detector, we have the probablt of the photon event at a pel gven b [6-8,16]: kl,,,,,, 0,1,2,3, (4),! where I kl (, ) s the normalzed rradance at a pel (, ) such that, 1. N and N are the number of horzontal and vertcal pels, respectvel. N p s the epected number of photons n a smulated photon-countng elemental mage. C kl (, ) s the number of photons arrvng at a pel (, ) durng the eposure tme. 3D mages of the objects can be reconstructed usng the mamum lkelhood (MLE) or mamum a posteror (MAP) estmaton method. he calculaton of the posteror dstrbuton s based on the assumpton that the pror dstrbuton of the rradance s Gamma (α, β) where α, β are known numbers appromatel calculated from the sample mean and sample varance. he MLE and MAP estmators for the rradance s gven as follows [6,8]:, (5) (1), (6) 3. HE FAS FIXED-POIN ICA ALGORIHM BASED ON KUROSIS MAXIMIZAION Kurtoss s used n the non-gaussant mamzaton. Consder a data vector : =[ 1, 2 k ] n whch m statstcall ndependent components s=[s 1,s 2 s m ] can be etracted, where m k, each component of s a lnear combnaton of ndependent components n s usng the mng matr A: = As. (7) he ndependent components can be estmated usng the un-mng matr W: s = W. (8) We use kurtoss as a measure of nongaussant of a random varable, whch s defned as: [12]: { } 3( E{ }), kurt( ) = E (9) V. 3 (p.2 of 7) / Color: No / Format: Letter / Date: :14:33 PM

3 where E denotes the epectaton operator. Kurtoss of a Gaussan random varable s zero whle nongaussan random varables have non-zero kurtoss. he fast fed-pont algorthm, whch mamzes kurtoss of =w z s defned as follows [12]: 3 { z( w z) } 3w, w E (10) where denotes the transpose operaton, w s a row of the matr W, z s the whtened data vector obtaned b applng whtenng transform B such that [12]: 1/ 2 z = B, B = D E, (11) where D=dag(d 1 d n ) s the dagonal matr of the egenvalues of the covarance matr E( ). E=(e 1 e n ) s the egenvector matr, and s the data vector. 4. APPLICAION OF ICA IN RECOGNIION OF RECONSRUCED 3D OBJECS USING PHOON-COUNING INEGRAL IMAGING Each reconstructed 3D mage s converted nto a column vector for convenent computaton. he acqured reconstructed 3D mages are parttoned nto tranng and testng data sets. Prncpal Component Analss (PCA) s appled to the tranng data set to reduce the number of data dmensons, n whch egenvectors wth largest egenvalues are selected to form a PCA transform whch mamzes varance of projected data. he FastICA algorthm s then appled to PCA projected vectors to estmate an ICA transform whch s used n the testng stage. Snce the large number of dmensons ma lead to a computatonal memor ssue durng the calculaton of the covarance matr, a covarance matr C s used nstead of the covarance matr C [17]: 1 C = X X, (12) N where X s the zero-mean data matr X. ranng vectors are n columns of X where X R MR, R s the number of tranng vectors and M s the number of data dmensons. Performng egenvalue decomposton to covarance matr C, we have [17]: C = U λu, (13) where U s an egenvector matr whch contans egenvectors on ts columns, and λ s an egenvalue matr whch contans egenvalues on ts dagonal. So the egenvalues and egenvectors are calculated as [17]: λ = λ, u = Xu. Nλ (14) Matr C (R R) s much smaller than C (M M) when R<<M because mage dmensons are much larger than the number of tranng data samples. hus, the number of egenvectors are R<<M. he projecton along the drectons of the egenvectors wth the hghest egenvalues retans the hghest varablt of the orgnal data set. B choosng the m hghest value egenvectors, we can decrease the number of dmensons such that the prncpal component space retans most of the percentage of varance of the data set. Here the frst k egenvectors are chosen and normalzed to be a PCA transform W PCA (k R). he representaton PCAran of a tranng vector ran n the PCA doman s epressed as [15-16]: = W. (15) PCAran he representaton of ran n the ICA doman s as follows: PCA ran V. 3 (p.3 of 7) / Color: No / Format: Letter / Date: :14:33 PM

4 = W = W W, (16) ICAran ICA PCAran where W ICA s the estmated ICA transform. Smlarl, we obtan the ICA representaton for a testng vector as: ICA PCA ran = W W, (17) ICAtest where test s a testng vector whose column s formed b a reconstructed 3D mage. he cosne angle between a testng projected vector and the tranng projected vectors s evaluated to determne the object class [15-16]: ICAtest ICAtran c =, (18) j ICAtest ICAtran where ICAtest, j ICAtran are th and j th testng and tranng projected vectors, respectvel. o classf the test vector, the class n whch a tranng projected vector has the mamal cosne wth the testng projected vector s determned. ICA PCA test 5. EXPERIMEN RESULS hree to car objects appromatel 70 mm 30 mm 15 mm n sze are used n the eperment (See Fgure 1). Each car s placed on a rotatable stage at dstance of appromatel 270 mm from the camera lens. o capture elemental mages, the camera shfts ever 5 mm horzontall and vertcall on the 2D grd. he focal length s 40 mm. Each elemental mage has pels. he detector arra sze s mm. Fgure 1. hree car objects used n the eperment Each car object s rotated and captured at 6 dfferent angles of 0 0, 5 0, 10 0, 15 0, 20 0 and he two-dmensonal arra of elemental mages at each angle s 4 7 n sze. Fgure 2 llustrates the epermental set-up. Fgure 2. he snthetc ntegral magng set-up he elemental mages are cropped to the sze of pels. Photon-countng elemental mages are generated from the cropped elemental mages usng equaton (4). Fgure 3(a)-(c) shows the photon-countng elemental mages wth V. 3 (p.4 of 7) / Color: No / Format: Letter / Date: :14:33 PM

5 N p =1000 photons and Fgure 4(a)-(c) shows the photon-countng elemental mages wth N p =250 photons. he number of photons s ver small compared wth the sze of the photon-countng elemental mage, whch s appromatel 239,000 pels. Observng Fgure 4, we can see that the objects are much less recognzable compared wth Fgure 3. Fgure 3. Photon-countng elemental mages of 3 car to objects wth N p =1000 (a) class 1 (b) class 2 (c) class 3 Fgure 4. Photon-countng elemental mages of 3 car to objects wth N p =250 (a) class 1 (b) class 2 (c) class 3 Fgure 5. 3D reconstructed mages at dstance of 271 mm, N p =1000 (a) class 1 (b) class 2 (c) class 3 Fgure 6. 3D reconstructed mages at dstance of 271 mm, N p =250 (a) class 1 (b) class 2 (c) class 3 At ever rotaton angle, ffteen 3D mages are reconstructed from 28 cropped photon-countng elemental mages. hs s done usng the MAP estmaton method at dstances of 261 mm to 318 mm from the lens, whch corresponds to the pel offsets from 78 pels to 64 pels. he 3D mages of the three to cars at the angle of 25 0 are reconstructed at dstance V. 3 (p.5 of 7) / Color: No / Format: Letter / Date: :14:33 PM

6 of 271 mm. hese are shown n Fgure 5(a)-(c) and Fgure 6(a)-(c) for N p =1000 and N p =250 photons, respectvel. he reconstructed mages wth hgher N p are more recognzable than the one wth lower N p. Each reconstructed 3D mage s cropped to the sze of pels and transformed nto a column vector of n sze. hree reconstructed 3D mages are randoml selected out of 15 mages at each angle to form the tranng data set. he rest of reconstructed 3D mages are for testng. Snce there are N p =1000, N p =750, N p =500 and N p =250 photons, the number of tranng vectors s 216 whle that of testng vectors s 864. Prncpal component analss s appled to reduce the number of dmensons. he equatons (12)-(14) descrbed the calculatons. Usng ths method, the number of obtaned egenvectors of 216 s much smaller than We choose to keep 90% of the varance. Fgure 7. Classfcaton rate of 3 classes for dfferent Np (a) Np=250 (b) Np=500 (c) Np=750 (d) Np=1000 ICA algorthm s appled on the PCA projected tranng vectors to estmate an ICA transform. Each PCA tranng vector s then projected to the ICA doman usng the ICA transform. Ever reconstructed 3D mage from the test data set s transformed to ts ICA representaton followng the same procedure, and classfed to one of three classes usng a cosne angle classfer (See equaton (18)). We sum the number of msclassfed samples for each class and calculate the correct classfcaton rate. he total correct classfcaton rate s 100%, 99.24% and 97.47% for classes 1, 2 and 3. Fgure 7 shows a more detaled result for dfferent N p of 250, 500, 750 and 1000 photons. Wth N p =250 photons, the correct classfcaton rate for class 1 s as hgh as 100%, 99.49% for class 2 and 96.97% for class 3, respectvel. 6. CONCLUSION In ths paper, the SAII technque has been appled to capture elemental mages. Photon-countng mages are generated usng a Posson dstrbuton-based photon-countng model. he MAP estmaton method has been appled to reconstruct 3D mages from photon-countng elemental mages. We appl a kurtoss mamzaton-based algorthm as a nongaussan mamzaton method to etract the ndependent features from the tranng data set. Prncpal Component Analss (PCA) s used to reduce the number of dmensons and facltate the ICA processng. he cosne angle metrcs s used to calculate classfcaton performance. It has been shown that ths method can effectvel recognze 3D objects despte a ver small Np n a reconstructed 3D mage of appromatel 239,000 pels. ACKNOWLEDGEMENS he author would lke to thank Courtena Dunn-Lews for her assstance V. 3 (p.6 of 7) / Color: No / Format: Letter / Date: :14:33 PM

7 REFERENCES [1] Lppmann, G., La photographe ntegrale, C. R. Acad. Sc. 146, (1908). [2] Okano, F., Hoshno, H., Ara, J. and Yauma, I., Real tme pckup method for a three-dmensonal mage based on ntegral photograph, Appl. Opt., vol. 36, (1997). [3] Burckhardt, C. B., Optmum parameters and resoluton lmtaton of ntegral photograph, J. Opt. Soc. Amer., vol. 58, (1968). [4] Hong, S. and Javd, B., hree-dmensonal Vsualzaton of Partall Occluded Objects Usng Integral Imagng, J. Dspla echnol., vol 1, 354- (2005). [5] Schulen, R., Do, C. M., Javd, B., Dstorton-tolerant 3D recognton of underwater objects usng neural networks, he Optcal Socet of Amerca A, vol. 27, ssue 3, (2010) [6] avakol, B., Javd, B., and Watson, E.,"hree dmensonal vsualzaton b photon-countng computatonal Integral Imagng," Opt. Epress 16, (2008) [7] Jung, J., Cho, M., De, D. and Javd, B., "hree-dmensonal photon-countng ntegral magng usng Baesan estmaton," Opt. Lett. 35, (2010) [8] Do, C. M., hree-dmensonal photon-countng reconstructon of occluded objects b mamum a posteror estmaton, SPIE Optcs and Photoncs, accepted (2010). [9] Bar-Shalom, Y., Krubarajan,., L, X., [Estmaton wth Applcatons to rackng and Navgaton], John Wle & Sons, Inc. (2002). [10] Mukhopadha, N., [Probablt and statstcal nference], Marcel Dekker (2000). [11] Hvärnen, A., Hoer, P. O. and Oja, E., Sparse Code Shrnkage: Denosng b Nonlnear Mamum Lkelhood Estmaton, In Advances n Neural Informaton Processng Sstems 11 (NIPS1998), (1999) [12] Hvärnen, A., Karhunen, J. and Oja, E., [Independent Component Analss], John Wle & Sons, New York (2001). [13] Bartlett, M. S., Movellan, J. R. and Sejnowsk,. J., Face recognton b ndependent component analss, IEEE ransactons on Neural Networks 13, (2002) [14] Do, C. M. and Javd, B., 3D Integral Imagng Reconstructon of Occluded Objects Usng Independent Component Analss-Based K-Means Clusterng, IEEE Journal of Dspla echnolog, (2010). [15] Do, C. M., Martínez-Cuenca, R. and Javd, B., "hree-dmensonal object-dstorton-tolerant recognton for ntegral magng usng ndependent component analss," J. Opt. Soc. Am. A 26, (2009) [16] Do, C. M. and Javd, B., hree-dmensonal Object Recognton Wth Multvew Photon-Countng Sensng and Imagng, IEEE Photoncs Journal, vol 1, ssue 1, 9-20 (2009). [17] Murakam, H., and Kumar, B., Effcent Calculaton of Prmar Images from a Set of Images, IEEE rans. Pattern Analss and Machne Intellgence, vol. PAMI-4, No. 5 (1982) V. 3 (p.7 of 7) / Color: No / Format: Letter / Date: :14:33 PM