A Super-resolution Algorithm Based on SURF and POCS for 3D Bionics PTZ

Transcription

1 Sensors & Transducers 204 by IFSA Publshng, S. L. A Super-resoluton Algorthm Based on SURF and POCS for 3D Boncs PTZ Hengyu LI, Jqng CHEN, Shaorong XIE and Jun LUO School of Mechatroncs Engneerng and Automaton, Shangha Unversty, 49 Yanchang Road, Zhabe Dstrct, Shangha, , Chna Tel.: E-mal: lhengyu@shu.edu.cn Receved: 30 March 204 /Accepted: 30 Aprl 204 /Publshed: 3 May 204 Abstract: Image super-resoluton algorthm mproves mage resoluton n software. Before mage trackng, t needs to enhance mage resoluton to mprove the mage trackng accuracy of the bonc eye. The tradtonal super-resoluton reconstructon method can t satsfy the accuracy and real-tme system, so ths paper proposes super-resoluton mage reconstructon algorthm based on SURF (Speeded up Robust Features) and POCS (Projectons Onto Convex Sets). The algorthm apples SURF algorthm on mage regstraton and uses RANSAC (RANdom SAmple Consensus) algorthm to kck out fault feature to mprove the accuracy of mage regstraton. After moton estmaton, ths paper apples POCS algorthm to reconstruct a super-resoluton mage. Fnally, some experments are performed on the platform of a bonc eye, whch show that the algorthm can better mprove the mage resoluton and reach real-tme requrements to some extent, and provdng bass for the subsequent mage trackng. Copyrght 204 IFSA Publshng, S. L. Keywords: Super-resoluton Algorthm, SURF, POCS, Boncs PTZ, RANdom SAmple Consensus.. Introducton Vson technology gets more attenton n the study of robot technology. It s approprate to conduct mage trackng for robots because human eye can track fast movng objects. Image resoluton has a certan effect on the mage trackng accuracy. There are two common methods to mprove mage resoluton, whch s mprovng the resoluton of the camera or usng some super-resoluton algorthms. For better portablty, ths paper chooses the algorthm to mprove the mage resoluton. Image super-resoluton reconstructon s to reconstruct a hgh-resoluton mage by complementary nformaton of multple low-resoluton mages, the core dea of whch s to exchange tme bandwdth for a hghresoluton mage. Super-resoluton mage reconstructon algorthm can be dvded nto spatal method and frequency doman method []. Frequency doman method s ntutve but t s lmted n lnear space unchanged and t contans only a lmted spatal pror knowledge by usng the global translatonal moton degradaton model. Spatal doman s modelng of spatal doman factors whch affectng the low-resoluton mages results. Therefore the spatal doman method s closer to the actual applcaton. In prevous researches, the maxmum a posteror (MAP) method [2-3], the projecton on convex sets (POCS) method [4-5], the teratve back projecton method [6-7], and the wavelet-based method [8] are extensvely studed. These algorthms mprove resoluton to some extent, but there are stll some sub-pxel errors. 274 Artcle number P_2072

2 In recent years, SIFT (Scale Invarant Feature Transform) algorthm [9] s ntroduced to the super resoluton reconstructon. Seong Y. M. et al proposed a super-resoluton reconstructon method [0] of lowresoluton mages sequence, whch appled the SIFT algorthm n mage regstraton frstly and then get hgh-resoluton mages by the learnng superresoluton algorthm. Compared to the prevous methods, SIFT have some mprovement for superresoluton accuracy, but s tme-consumng. It cannot satsfy real-tme requrements. Snce the above algorthm can t satsfy the accuracy and real-tme requrements smultaneously. As the precson and speed of SURF algorthms s better than that of SIFT algorthm, ths paper proposes a super-resoluton algorthm based on SURF [-3] and POCS. Expermental results show that the algorthm can satsfy the accuracy and realtme requrements. The algorthm s shown n Fg.. Ths paper s organzed as follows: Part II descrbes the desgn of bonc eye moton platform; Part III descrbes the model of super-resoluton reconstructon; Part IV ntroduces the mage regstraton algorthm based SURF; Part V descrbes the prncple and process of POCS algorthm; part VI manly descrbes experments and experments show that the algorthm can satsfy the requrements of accuracy and real-tme for the system. Fg.. The process of the algorthm based on SURF and POCS. 2. The Desgn of 3D Boncs PTZ Currently the majorty of robots on the market are only equpped wth 2D boncs eye movement platform, whch s dfferent from the human eye movement. The human eye has 3 rotatonal degrees of freedom, so ths paper desgns a sphercal parallel mechansm accordng to human eye movement characterstcs and structure. The system has a spn degree of freedom that prevous robot eyeballs system don t have, thus mprovng the mage qualty by robot's stance or the change targets posture, reducng the dffculty and complexty of the human eye observng and computer mage processng. The sphercal parallel mechansm desgned n ths paper s shown as Fg. 2(a), whch s conssted of a top platform (eye), a bottom platform, 3 par branches whch conssted wth down lnks and up lnks. The motor fxed n bottom platform. Motor and down lnks, down lnks and up lnks, eyeballs and up lnks are connected by revolute par. 9 revolute axes ntersect at pont O, called the center of mechansm rotaton. Any pont on the lnks of the mechansm s constraned by the rotatonal center. Relatve movements between the lnks are rotatonal around the respectve axs whch penetrates the rotaton center, so the movng platform has 3 rotatonal degrees of freedom relatve to the statonary platform. The eye can realze 3 rotaton degrees of freedom relatve to the base by 3 motors. Fg. 2(b) s a 3D modelng of bonc platform desgned n ths paper and Fg. 2(c) s the object. (a) (b) (c) Fg. 2. Structure desgn of 3D bonc PZT. 275

3 3. The Model of Super-resoluton Reconstructon 3.. Degradaton Model Frstly ntroducton a common mage observaton model [5]. Observaton model s generated after a seres of deal mage degradaton. The factors affect mage degradaton ncludng atmospherc dsturbances, the dfference of the optcal system, blur generated by movement, nose and down samplng, etc. All of these factors make mage dstorton and blur. The degradaton processes shows as Fg. 3. Fg. 3. The processes of mage degradaton. If X s the deal mage, Y s sequence of lowresoluton mages obtaned by observaton, mage formng process can be expressed as: Y = MHFX + n =,2,..., k, () where s the number of frame, k s the lowresoluton mage frames, Y s the LNxColumn vectors, represents low-resoluton mages, X s the qlqnx Column vectors, represents hgh-resoluton mages, q s the amplfcaton coeffcent, M s the qlqnx qlqn down samplng matrx, n s the mage nose sgnal The Prncples of Super-resoluton Reconstructon For a lnear space nvarant magng system, the spatal doman magng process can be descrbed by the formula (2): That can be represented by the followng formula: f ( x) > 0, x X, (3) f ( x) = 0, x X where X means the sze of the target. (3) Nonlnear operaton: consderng the effect by the nose generated durng magng process, the general magng process can be expressed as follows: g ( x) = h( x)* f ( x) + n( x), (4) where n(x) represents the nose. The flow of mage super-resoluton reconstructon s shown as Fg. 4. The purpose of mage regstraton s gettng the offset between low-resoluton mages. Then project the mage to hgh-resoluton grd and t needs to conduct nterpolaton drectly or teratve nterpolaton to generate hgh-resoluton mages. gx ( ) = hx ( )* f( x), (2) where g(x) represents an mage; f(x) represents an objectve; h(x) represents pont spread functon; * denotes the convoluton operaton. Realze super-resoluton reconstructon s manly based on the theory as follow [4-20]: () Analytcal contnuaton theory: f a functon f(x) s the arspace bounded, then the spectrum F(u) s an analytc functon. For analytc functons, t wll be known anywhere f t confrm at a certan fnte nterval. (2) Informaton superposton theory: the actual mages have a non-negatve and boundedness constrants for common magng. That s the mnmum lght ntensty of the target or mage should be greater than 0, and t should have a certan sze. Fg. 4 The flow of mage super-resoluton reconstructon. 4. Image Regstraton Algorthm Based SURF The SURF feature ponts have nvarant sze and drecton, whch extract by Herbert Bay, Tnne Tuytelaars and Luc Van Gool. Compared to SURF and SIFT, the SURF s hgher accuracy and real-tme. The processes of mage regstraton algorthm show as Fg

4 RANSAC algorthm [6] to kck wrong matchng feature pont par. Fg. 5. The processes of mage regstraton algorthm. 4.. SURF Feature Pont Extracton SURF feature ponts are extracted referencng to the prncple of SURF algorthm [-3], the steps as follows: ) Convert a gray mage nto an ntegral mage; 2) Calculate nterest pont of the mage by Hessan matrx. Calculate one mage pont s local Maxmum value of Hessan matrx determnant at dfferent scales. Any pont n the mage X(x, y), the Hessan matrx n the scale σ s defned as: L H ( X, σ ) = L xx xy ( X, σ ) ( X, σ ) L L xy yy ( X, σ ) ( X, σ ), (5) where L xx (X, σ) represents convoluton of Gaussan second-order partal dervatves n drecton X; L xy (X, σ) and L yy (X, σ) have a smlar meanng. 3) Construct scale space. Establsh mage pyramd by adoptng ncreasng sze of flter box template. 4) SURF feature ponts extracton. Defne local maxmum value as a feature pont by comparng a flter pont wth 26 ponts whch ncludes 8 pxels n the same scale, 9 ponts adjacent to upper scale and the other adjacent to down scale. 5) Determne the characterstc vectors. Fnally 64-dmensonal feature vector s obtaned Feature Pont Matchng The paper uses the Eucldean dstance between two feature vectors as two feature ponts smlarty measure after SURF feature vectors are generated. N 2 = Σ, (6) = d( X, Y) ( x y ) where d(x, Y) represents the Eucldean dstance between the feature vectors; X=(x, x 2, x 3, x N ) and Y=(y, y 2, y 3, y N ) respectvely represent any pont of mages X and Y; represents the subcomponent of the vector descrpton; x and y respectvely represent the sub-component of the vector descrpton of mages X and Y; N means dmenson of the feature vector, whch s 64 n ths paper. Ths paper frstly takes a pont from the reference mage and then fnds the nearest and next nearest pont from another mage. If the rato between the square of nearest dstance and the next nearest dstance s less than 75 % [5], we regard the pont par as feature pont par. After that, ths paper apples 4.3. Calculatng Homographc Matrx The relatonshp between the two mages may be represented by a planar perspectve transformaton matrx. X = HX, (7) where X represents (x, y, ) T ; X j represents (x j, y j, ) T ; (x, y ) and (x j, y j ) are pars of matchng feature ponts; f matrx s a full rank of 3 3 matrx, t can be expressed as: 5. Equatons h h h H h h h 2 3 = h3 h32 h The Prncple of POCS Algorthm POCS s an teratve process [4-5]: for the nput mage ntal value, project t onto a convex set and terate untl t meets the requrement of the teraton. The soluton of the ntersecton s consdered to be the deal mage. The process of projecton teraton s shown as formula: n+ n f PP N N... Pf, n,2,..., N j = =, (8) where f s the deal hgh-resoluton mage; f refer to the ntal nput value of teraton (the ntal estmate of the hgh-resoluton mage); P k s the projecton operator of convex sets. In the POCS algorthm, low-resoluton mage can be represented by mathematcal model of hghresoluton mage. g ( mm, ) = fnnhmmnn (, ) (, ;, ) +η ( mm, ), l 2 2 l 2 2 l 2 ( nn, 2) (9) where g l (m, m 2 ) means the l low-resoluton mage; f(n, n 2 ) means hgh-resoluton mage; h l (m, m 2 ; n, n 2 ) means the functon of PSF; η(m, m 2 ) represents Gaussan nose. Furthermore, t should satsfy the convex set data consstency and gray value lmtaton constrants. Data consstency reflects the relatonshp between the pxels of hgh and low resoluton mages. The convex set s represented by (0): ( f ) { (,, ): (,, ) δ (,, ) } C = fmmk r mml mml, (0) mml, 2,

5 (,,) = (,,), f ( nnk,, ) hmmnnl (, ;,, ) () ( f ) r m m2 l g m m2 l mm, where r (f) (m, m 2, l) s the resdual error of ponts n the convex sets; δ 0 (m, m 2, l) s the statstcal propertes of Gaussan nose. In fact, gray value lmtaton can be expressed as ampltude boundedness, and ts mathematcal expresson s as follows: C A = { f ( x, y) α f ( x, y) β }, (2) where α=0; and β= The Flow of POCS Algorthm ) Conduct non-unform nterpolaton for the reference mage that s selected from low-resoluton sequence mages and the result of nterpolaton s consdered as ntal hgh-resoluton mage estmaton f (n, n 2, k). 2) Determne the PSF functon of the model, that s h l (m, m 2 ; n, n 2 ), whch s shown as formula (9). 3) Choose an mage from low-resoluton sequence mages g l (m, m 2, l). If ths mage s the last mage, the teraton stops. 4) Conduct mage correcton accordng to formula (9) for all pxels. 5) Convert to step 3. 6) Consder the obtaned estmated mage f (n, n 2, k).as the requred hgh-resoluton mage. 7) Constran the magntude by equaton (2). The algorthm flow chart s shown n Fg Expermental Results and Analyss The experment was taken on the bonc eye platform. The system can control camera s dsplacement and rotaton by controllng PTZ. The experment reconstructs super-resoluton based on four mages. In order to facltate better analyss on the effect of the algorthm, ths paper desgns 3 experments, whch are only rotaton, only translaton and both rotaton and translaton. Fg. 7(a), (b), (c), (d) are four orgnal lowresoluton mages that only translatons, among whch (a) s regarded as the reference mage. Fg. 7(e) explan the 4 fgures have enough feature pont pars, whch s the regstraton result of (a) and (b), (a) and (c), (a) and (d). Fg. 7 (f) s superresoluton mage obtaned by the algorthm proposed n ths paper. To better to llustrate the algorthm for the mprovng resoluton, the wngs n fgure (a) and (f) are enlarged to the same sze, whch s shown as red crcle n fgure (a) and (f). Compare the red crcle of the fgure (a) and (f), especally the red box and 2, red box porton n (f) s clear whle that n (a) obvous mosac. The experment shows that the algorthm can perform well for low-resoluton mages that only translaton. Fg. 6. The flow chart of POCS algorthm. 2 (a) (b) (c) (d) (e) (f) Fg. 7. The expermental result for low-resoluton mages that only translatons ( (a), (b), (c), (d) are orgnal mages, (e) the result of the matchng feature ponts, (f) the super-resoluton mage). 278

6 Fg. 8(a), (b), (c), (d) are four orgnal lowresoluton mages that only rotatons, among whch (a) s regarded as the reference mage. Fg. 8(e) that can explan the 4 fgures have enough feature pont pars s the regstraton result of (a) and (b), (a) and (c), (a) and (d). Fg. 8(f) s super-resoluton mage obtaned by the algorthm proposed n ths paper. To better to llustrate the algorthm for the mprovng resoluton, red crcle and 2 n fgure (a) are enlarged. Compare the bgger red crcle and 2 of the fgure (a) and that n fgure (f), the red crcle n (f) s clear whle that n (a) obvous mosac. The experment shows that the algorthm can perform well for low-resoluton mages that only rotaton. Fg. 9(a), (b), (c), (d) are four orgnal lowresoluton mages that translatons and rotatons, among whch (a) s regarded as the reference mage. Fg. 9(e) can explan the 4 fgures have enough feature pont pars and fgure (f) s super-resoluton mage. To llustrate the algorthm effect, red small box n fgure (a) and (f) are enlarged to red bg box, whch are n the same level. The red bg box n fgure (a) has a clear gap n texture secton whle that n (f) s clear. The experment shows that the algorthm can perform well for low-resoluton mages that translatons and rotatons. SURF s 62-dmensonal vector whle SIFT s 28-dmensonal vector, whch greatly reduces the dmenson of ts operaton n speed. The algorthm can handle about 20 mages n s by super-resoluton algorthm, and satsfy the real-tme requrements of the system. (a) (b) (c) (d) (e) (f) Fg. 8. The expermental result for low-resoluton mages that only rotaton ( (a), (b), (c), (d) are orgnal mages, (e) the result of the matchng feature ponts, (f) the super-resoluton mage). (a) (b) (c) (d) (e) (f) Fg. 9. The expermental result for low-resoluton mages that translatons and rotatons ( (a), (b), (c), (d) are orgnal mages, (e) the result of the matchng feature ponts, (f) the super-resoluton mage). 7. Conclusons To mprove the accuracy of real-tme trackng bonc eye, the system proposed a super-resoluton reconstructon algorthm based on SURF and POCS based on the platform of bonc eye movement. Ths paper apples SURF algorthm for moton estmaton that s mage regstraton, and then adopts RANSAC algorthm to kck out false SURF feature ponts. After gettng related moton estmaton, the paper apples super-resoluton reconstructon algorthm based on POCS. Fnally, ths paper performs experments on the platform of a bonc eye, whch shows that the algorthm can mprove mage resoluton well and reach real-tme requrements to some extent. Acknowledgements Ths project s supported by the Natonal Nature Scence Foundaton of Chna (No ), the outstandng academc leaders plan of Shangha (No.2XD402600), and Shangha Rsng-Star Trackng Program (2QH400900). 279

7 References []. Chunxa Wang, Revew of super-resoluton mage reconstructon technques, Computer Technology and Development, Vol. 2, Issue 5, 20, pp [2]. G. Chantas, N. Galatsanos and N. Woods, Superresoluton based on fast regstraton and maxmum a posteror reconstructon, IEEE Trans. Image Process., Vol. 6, Issue 7, 2007, pp [3]. S. Farsu, M. Robnson, M. Elad, and P. Mlanfar, Fast and robust multframe super-resoluton, IEEE Trans. Image Process., Vol. 3, Issue 0, 2004, pp [4]. A. Patt, M. Sezan and A. Murat Tekalp, Superresoluton vdeo reconstructon wth arbtrary samplng lattces and nonzero aperture tme, IEEE Trans. Image Process., Vol. 6, Issue 8, 997, pp [5]. C. Fan, J. Zhu, J. Gong, and C. Kuang, Pocs superresoluton sequence mage reconstructon based on mprovement approach of keren regstraton method, n Proceedngs of the 6 th Int. Conf. ISDA, 2006, pp [6]. M. Iran and S. Peleg, Moton analyss for mage enhancement: Resoluton, occluson, and transparency, Journal of Vsual Communcaton and Image Representaton, Vol. 4 Issue 4, 993, pp [7]. A. Tekalp, M. Ozkan and M. Sezan, Hgh-resoluton mage reconstructon from lower-resoluton mage sequences and space-varyng mage restoraton, n IEEE Internatonal Conference on Acoustcs, Speech, and Sgnal Processng, 992, pp [8]. H. J and C. Fermuller, Robust wavelet-based superresoluton reconstructon: Theory and algorthm, IEEE Transactons on Pattern Analyss and Machne Intellgence, Vol. 3, Issue 4, 2009, pp [9]. D. G. Lowe, Dstnctve mage features from scalenvarant key ponts, Internatonal Journal of Computer Vson, Vol. 60, 2004, pp [0]. Y. M. Seong, H. Park, A hgh-resoluton mage reconstructon method from low-resoluton mage sequence, n Proceedngs of the IEEE Internatonal Conference on Acoustcs, Speech, and Sgnal Processng, 992, pp []. H. Bay, T. Tuytelaars, and L. Van Gool, Surf: Speeded up robust features, Computer Vson ECCV, 2006, pp [2]. H. Bay, A. Ess, T. Tuytelaars, and L. Van Gool Speeded-up robust features (SURF). Computer Vson and Image Understandng, Vol. 0, 2008, pp [3]. E. Yong, Investgaton of mosacng technques for forward lookng sonar, Master's thess, Herot-Watt Unversty, 20. [4]. Yajng Zhang, Super-resoluton algorthm for the reconstructon of the sequence of mages, Master's thess, Northeastern Unversty, 20. [5]. B. Ztova and J. Flusser, Image regstraton methods: a survey, Image and Vson Computng, Vol. 2, 2003, pp [6]. M. A. Fschler and R. C. Bolles, Random sample consensus: a paradgm for model fttng wth applcatons to mage analyss and automated cartography, Communcatons of the ACM, Vol. 24, 98, pp [7]. X. Gao, K. Zhang, D. Tao and X. L, Jont learnng for sngle-mage super-resoluton va a coupled constrant, IEEE Transactons on Image Processng, Vol. 2, Issue 2, 202, pp [8]. X. Gao, K. Zhang, D. Tao and X. L, Image superresoluton wth sparse neghbor embeddng, IEEE Transactons on Image Processng, Vol. 2, Issue 7, 202, pp [9]. K. Zhang, X. Gao, D. Tao and X. L, Sngle mage super-resoluton wth non-local means and steerng kernel regresson, IEEE Transactons on Image Processng, Vol. 2, Issue, 202, pp [20]. C. Km, K. Cho and J. B. Ra, Example-Based Super- Resoluton va Structure Analyss of Patches, IEEE Sgnal Process. Lett., Vol. 20, Issue 4, 203, pp Copyrght, Internatonal Frequency Sensor Assocaton (IFSA) Publshng, S. L. All rghts reserved. ( 280