nverse-polar Ray Projecton for Recoverng Projectve Transformatons Yun Zhang The Center for Advanced Computer Studes Unversty of Lousana at Lafayette yxz646@lousana.edu Henry Chu The Center for Advanced Computer Studes Unversty of Lousana at Lafayette cce@cacs.lousana.edu Abstract A ray projecton n the nverse-polar space s proposed for recoverng a projectve transformaton between two segmented mages. The mages are converted from ther orgnal Cartesan space to the nverse-polar space. Then the two ray projectons one shft-nvarant and the other shft-senstve of the nverse-polar mages are computed to create two sets of data. Based on the obtaned projecton data a two-step strategy s employed to recover the projectve transformaton. n the frst step the shftnvarant data are used to recover the four affne parameters. n the second step the shft-senstve data are used to recover the two projectve parameters. The remanng two translaton-related parameters are recovered n e.g. an exhaustve search combned wth the two-step recovery strategy. The proposed approach has been tested successfully to recover a varety of projectve transformatons between real mages. 1. ntroducton The recovery of projectve transformatons from mages taken from dfferent vewponts s an essental task n computer vson. ethods such as the stratfed reconstructon rely on assumptons of the scene and features extracted from the mages. An alternatve approach s to extract the projectve transformaton parameters from the contour of maged regons (or objects). Based on the concept of ntegral geometry [9] the trace transform [4] a generalzaton of the Radon transform [ p. 55] has been proposed. t was successfully appled to recoverng smlarty or affne transformatons between segmented mages [4 5]. Practcal technques for evaluatng shft-nvarant or shftsenstve one-dmensonal (1-D) functons have also been developed [4 5 1]. We extend these technques n the present work for recoverng projectve transformatons. n ths paper a ray projecton n the nverse-polar space termed nverse-polar ray projecton s proposed to solve the problem of recoverng a projectve transformaton between two segmented mages. The proposed recoverng process makes use of nformaton about the shape and appearance of an maged object. n partcular three key contrbutons are made. Frst the nverse-polar transform whch was recently ntroduced for recoverng the two projectve parameters [11] s extended as a mathematcal tool for recoverng a complete projectve transformaton. Secondly a ray projecton a generalzaton of the fan-beam projecton [6 p. 9] s proposed as a workhorse to produce data that are used to determne the transformaton. Fnally a feasble two-step strategy for the recoverng s establshed. The proposed nverse-polar ray projecton has ts unque features. Ths can be clarfed by a comparson between the recently ntroduced 1-D mappngs [11] and the present approach. Frst n the prevous case the projecton parameters are derved drectly from mage lne matchng. Thus the recovery of the four affne parameters suffers from the perturbaton by the presence of the projectve ones. However n the present case the recovery of the affne dstorton can be completely solated from the projectve one. Thus the projectve dstorton has no effect at least geometrcally on the accuracy of the recovered affne parameters. Secondly the 1-D mappngs depend manly on ntensty values n a regon whereas the contour of the regon s used n the present approach. A further clarfcaton can be made by a comparson between the trace transform [5] and the present approach. Frst the trace transform can be used to recover an affne transformaton. n contrast the present approach can be used to recover a general projectve transformaton. Secondly the trace transform apples a parallel-beam projecton whle the present approach employs a ray projecton. Fnally the deas for evaluatng the 1-D functons are smlar n both approaches. However mathematcally the present approach where only one par of 1-D functons s employed s much smpler than the trace transform where multple trplets of functons are used. The rest of ths paper s structured as follows. n Secton the nverse-polar ray projecton for recoverng projectve transformatons between segmented mages s presented. Based on the nverse-polar ray projecton a 978-1-444-43-/8/$5. 8 EEE
two-step strategy s developed n Secton 3 for recoverng the eght projectve parameters. The expermental results are presented n Secton 4. Fnally the conclusons are presented n Secton 5 together wth some related ssues and the future work.. nverse-polar ray projecton.1. nverse-polar transform A two-dmensonal (-D) projectve transformaton T x x y n a source mage to ts from a pont T counterpart x x y n a target mage s: x a b gx y c d h y. (1) 1 e f 1 1 The projectve transformaton has eght ndependent parameters whch are four affne ones a b c andd two projectve ones e and f and two translaton-related ones g and h defned as the shft of the target projecton center relatve to ts source counterpart. uch work n moton estmaton has focused on fndng the translaton parameters. Our focus n ths work s on recoverng the affne and projectve parameters. An mage patch can be thought of as a collecton of radal lne segments from a pole n the patch so that each mage pont s a pont along a 1-D radal lne (or ray). A source pont on a ray wth radal angle and radal x rcos r sn. Smlarly the transformed pont can be represented by x r cos r sn T. Substtutng these polar expressons for x and x nto Eq. (1) and assumng the two patches have no relatve translaton the projectve transformaton n the nverse-polar space becomes: cos sn tan 1 c d a cos bsn () r r where 1 r 1 r 1 r and two auxlary parameters affnely related dstance r can be represented as T r and projectvely related aredefnedas: a cos bsn c cos d sn (3) ecos f sn. From the second expresson n Eq. () we see that after convertng a source mage and ts target counterpart from the Cartesan space and nto the nverse-polar space P and P the projectve transformaton of a radal lne becomes lnear. Smple relatonshps n terms of can be establshed between P and P and wth the help of the ray projecton descrbed n the next secton... Ray projecton A ray projecton of an maged object computes a 1-D functonal along each ray crossng the object from a pole. n general any 1-D (or radal) functon shft-nvarant or shft-senstve can ft the purpose of the ray projecton. n addton dependng on the shape of the object and the locaton of the pole a ray projecton can cover ether a partal angle range (<) formng a fan shape or the full angle range (=) formng a dsc shape. n that sense the proposed ray projecton can be consdered as a generalzaton of the fan-beam projecton [6 p. 9] where the radal functon s a lne ntegral and the angular span of projecton s less than. n the experments we llustrate ray projectons where the angle range of projecton s ether less than or equal to. n the present work a shft-nvarant functon and a shft-senstve functon are employed. The former s a lne ntegral as used n the Radon transform [ p. 55] the fan-beam projecton [6 p. 9] and the trace transform [5] whle the latter s a rato of two lne ntegrals as used n the trace transform [5]. For concseness the followng analyss only takes P as an example. However all the derved results should be applcable to P. Thus these two functons can be expressed as follows: Shft-nvarant radal functon: (4) r dr P P: Shft-senstve radal functon: Q: r P r dr P r dr. f we drectly compute the above functons we have to frst transform an mage nto the nverse-polar space. nstead we can evaluate them n the polar space as: r r dr (6) 3 r r dr r r dr 3 We can see that the ntegrands r r and r r become nfnte at the pole when (5). (7) s nonzero. Ths means that the pole of the ray projecton needs to be placed somewhere outsde the object. By applyng the ray projecton wth the above propertes to the nverse-polar mages we can solve for the projecton parameters..3. Recovery of projectve transformaton Based on the nverse-polar ray projecton a two-step approach s developed to recover projectve parameters. n
the frst step the affne parameters a b c and d are recovered whle n the second step the two projectve parameters e and f are recovered..3.1 Recovery of four affne parameters Before makng use of the shft-nvarant radal functonal propertes we frst establsh a relatonshp between two nverse-polar mages P and P.Fromthe second equaton n Eq. () ths relaton can be drectly wrtten as: P r P r P r. (8) Then by applyng n (9) where P r P r and n 1. n order to solve for a b c and d we proceed n a two-step normalzaton of the two angular functons and. A smlar procedure has been proposed n [5] for handlng the angular functons derved from the trace transform. The frst step normalzaton reduces the power of to one. As a byproduct and are normalzed. Thus by rasng both sdes of Eq. (9) to the power of 1 n and rearrangng we obtan: n n (1) where 1 n 1 n n and n. Ths states that n s the n scaled by the affne factor defned n Eq. (3). Besdes and are affnely related va the frst equaton n Eq. (). These two observatons llustrate that there exsts a affne transformaton A wth four elements a b c and d between two regons R and R delmted by n and n [5 11]. One way to determne A s through a normalzaton of R and R. Ths second normalzaton smplfes the relaton between R and R from an affnty to a smlarty [1 7]. The related procedure s to construct 1 1 transformatons and from the second moment matrces of R and of R and then to normalze R and R wth the derved transformatons. t s 1 1 worth notng that the matrx power and can be computed by usng an egenvalue method [3 p. 556]. After the normalzaton we obtan two normalzed regons delneated by the normalzed angular functons n and n. As an example n can be computed from n va a backward transform technque as follows: ac db a d sn c cos b cos sn 1 det ac 1 tan ac db n n (11) where a 1 d are the elements of. Smlarly n can be computed from n as 1 above wth 1 takng over the role of. As mentoned above there s a smlarty transformaton S between n and n whch can be expressed as follows: 1 1 S A sr cos sn (1) wth R sn cos where s s an sotopc scalng (assumng s wthout loss of generalty) and R s a rotaton by angle. To db determne S we need frst to determne mappng n and by 1-D n wth the normalzed cross-correlaton (NCC) as smlarty measure. Then we need to determne s as a rato of the medans of two overlappng segments n and n by whch has been detected wth a maxmal NCC. Usng Eq. (1) we can compute S and fnally recover the affne matrx A as: 1 1 A s R (13) 1 1 where and are normalzed. t s worth notng that the above dervaton demonstrates mathematcally that the affne part of the dstorton of an mage can be extracted from ts general projectve one n the nverse-polar space. Ths unque feature of the nverse-polar ray projecton s verfed expermentally later..3. Recovery of two projectve parameters After determnng the four affne parameters we can recover the two projectve parameters e and f by nvokng the shft-senstve radal functonal propertes. n partcular we apply Q to both sdes of Eq. (8) and obtan: n (14) where Q P r and Q P r.t s worth notng that n s always equal to 1 forshftsenstve functons [5]. Also snce 1 and we can rewrte the above equaton as:. (15)
After computng and compute va Eq. (3) and evaluatng va the nverse-polar ray projectons we can usng Eq. (15). From Eq. (3) we know that contans the two projectve parameters e and f. Thustosolvefore and f we need two constrants. Here we compute e and f as the medans of sequences of e and f. Each e and f n the sequences are derved usng a par of values of.e. are n the angle range 1 n where the two angles and.for to be 1 we need to compute t as an angle modulo the angular span 1. For smplcty the angle dfference s kept constant for all pars. n partcular two constrants from can be expressed as: e cos f sn (16) ecos f sn. Solvng these lnear equatons we obtan e and f as: e sn sn sn (17) f cos cos sn. n the above computaton gven one wehaveto choose another apart. For both and to be able to provde ndependent nformaton should be as large as possble. On the other hand to avod sn n the above computaton cannot be as large as. Thus the maxmal possble equals half way between and ts opposte. 3. Algorthm Based on the nverse-polar ray projecton descrbed above an algorthm for recoverng a projectve transformaton between two segmented mages s outlned below: (1) ntalzaton: Acqure two segmented mage patches a source mage and a target mage. Set the projecton center of the source mage patch outsde the object. Compute for the source mage patch wth the pole at the projecton center accordng to Eq. (6). by the two-step normalzaton Determne n accordng to Eqs. (1) and (11). 1 of s also determned. Compute for the source mage patch wth the pole at the projecton center accordng to Eq. (7). Set a search range for the projecton center of the target mage patch to for example 77 around a pre-estmated value. () terate the followng two steps for each pont n the search range of the target mage patch and update results: Step 1: Recoverng the four affne parameters: o Compute along each ray from the projecton center of the target mage patch accordng to Eq. (6). o Determne n by the two-step normalzaton of accordng to Eqs. (1) 1 and (11). s also determned. o Determne by 1-D mappng n and n andsetr accordng to Eq. (1). o Determne s as a rato of the medans of n and n. o Recover the affne transformaton A accordng to Eq. (13). Step : Recoverng the two projectve parameters: o Compute along each ray from the projecton center of the target mage patch accordng to Eq. (7). o Compute accordng to Eq. (15). o Compute the two projectve parameters e and f as medans of sequences e and f accordng to Eq. (17). Update results: Compute the -D smlarty (NCC) between the source mage patch and the target counterpart and update the records for the recovered projectve transformaton and the target projecton center f NCCshgher. 4. Expermental Results We expermentally verfy that the proposed nversepolar ray projecton can be used to recover a general projectve transformaton. Our test set conssts of: 78 78 key (Fg. 1) lock (Fg. 1) 64 48 stop sgn (Fg. 6) and 64 48 fsh (Fg. 6). Transformatons between mages are ether smlarty affne or projectve ones. We use global thresholdng to segment the object each of whch s less than 15 pxels n each dmenson. n two experments we use synthetc transformaton so that we can compare the recovered parameters wth the ground-truth. n these experments mage warpng s performed by the blnear nterpolaton. To show that the angle range of ray projectons can vary n the projecton centers are placed dfferently n source mages. For mage set key the
centers are nsde the hole so that the ray projecton covers the full angle range. For the other three mage sets the centers are outsde the objects. Thus the related ray projectons cover only a partal angle range. However for comparson each recovered transformaton s presented as f the source projecton center were at the center of the source mage. n all cases the ray projecton s performed n the polar space wth an angular samplng nterval 1 and a radal samplng nterval 1 pxel. Recovered transformatons are evaluated as follows. Frst a recovered transformaton s compared wth ts ground truth f avalable. Secondly a smlarty measure (NCC) s computed for each par of matchng mage patches. 4.1. Recovery of transformatons for zoom and rotaton Results from synthetc data. Two synthetc mage pars have been tested. A par s used to recover the smlarty Transformaton (1) lsted n Table 1; the transformaton s a.5 dgtal zoom and a rotaton of 6. The recovered transformaton s lsted n Table 1 as Transformaton (). We can see that the recovered values are close to ther ground truth counterparts. The transformed and reconstructed mages are shown n Fg.. The obtaned D smlarty measure s.963195 for Transformaton (). The par s used to check the projectve effect on the recovery of a smlarty transformaton lsted n Table as Transformaton (3). The recovered transformatons are shown n Table as Transformaton (4). The recovered values are n a very good agreement wth the ground truth. Ths gves an expermental support to the theoretc analyss that the affne dstorton can be separated from a general projectve one. The transformed and reconstructed mages are shown n Fg. 3. The obtaned smlarty measure s.94777 for Transformaton (4). Results from real data. A real mage par s tested wth actual perspectve changes captured n the mages. The recovered transformatons are lsted n Table 5 as Transformaton (9) and shown n Fg. 7. The obtaned D smlarty measure s.965766. 4.. Recovery of transformatons for vewpont changes Results from synthetc data. Two synthetc mage pars have been tested. A par s used to recover the affne transformaton (5) lsted n Table 3. Ths transformaton s actually extracted from other real mage pars [8]. The recovered transformaton s lsted n Table 3 as Transformaton (6). The transformed and reconstructed mages are shown n Fg. 4. The obtaned D smlarty measure s.96835. The other par s used to check agan the projectve effect on the recovery of an affne transformaton. Transformaton (7) n Table 4 adds two new projectve parameters. The recovered transformaton s lsted n Table 4 as Transformaton (8). The recovered values are agan n a good agreement wth the ground truth. Ths gves a second expermental support to the above analyss that the affne parameters can be determned n the presence of projectve dstorton. The mages are shown n Fg. 5. The smlarty measure s.95469. Results from real data. A real mage par s tested wth real perspectve changes. The recovered transformaton s lsted n Table 5 as Transformaton (1) and shown n Fg. 8. The obtaned D smlarty measure s.94638. Our drect expermental results (Table 5) nclude the detected projecton centers of the target mage patches. Ths s because the two translatonal parameters for a projectve transformaton can be determned by the shft of the target projecton center relatve to ts source counterpart. 5. Conclusons The above theoretc analyses and expermental results demonstrate that the proposed nverse-polar ray projecton can be used successfully to recover a general projectve transformaton between two segmented mages. n the case where only pure smlarty or affne transformatons exst between mages the detected values for e and f can n 4 general reach as hgh as an order of 1. Ths phenomenon becomes pronounced when a test object s small. We note that when an object s small t becomes dffcult to obtan a precse descrpton of ts boundary. Ths could be an mportant factor that affects the accuracy of detected values especally for the smaller projectve parameters. n our ongong work we change the search area for the target projecton center from a specfed regon to a zone surroundng the segmented object. Thus the recovery of a projectve transformaton can be carred out automatcally. We can also extend the present approach to extract projectve-nvarant features for an object. Acknowledgments Ths work was supported n part by the Lousana Governor s nformaton Technology ntatve. References [1] A. Baumberg Relable feature matchng across wdely separated vews n Proc. EEE Conf.
Computer Vson and Pattern Recognton Hlton Head sland SC pp. 774-781. [] N. R. Bracewell Two-Dmensonal magng. Prentce Hall 1995. [3] G. H. Golub and C. F. Van Loan atrx Computatons 3 rd ed. Johns Hopkns Unv. Press 1996. [4] A. Kadyrov and. Petrou The trace transform and ts applcatons EEE Trans. Pattern Analyss and achne ntellgence vol. 3 pp. 811-88 1. [5] A. Kadyrov and. Petrou Affne parameter estmaton from the trace transform EEE Trans. Pattern Analyss and achne ntellgence vol. 8 no. 1 pp. 1631-1645 6. [6] A.C. Kak and. Slaney Prncples of Computerzed Tomographc magng. EEE Press 1988. [7] T. Lndeberg and J. Gardng Shape-adapted smoothng n estmaton of 3-D shape cues from affne deformatons of local -D brghtness structure mage & Vson Computng vol. 15 pp. 415-434 1997. [8] K. kolajczyk T. Tuytelaars C. Schmd A. Zsserman J. atas F. Schaffaltzky T. Kadr and L. Van Gool A comparson of affne regon detectors nt. J. Comput. Vs. vol. 65 pp. 43-7 5. [9] A. B. J. Novkoff ntegral geometry as a tool n pattern percepton: Prncples of self organzaton n Trans. Unv. of llnos Symp. Self-Organzaton H. von Foester and G. Zopf Jr. eds. London Pergamon Press pp. 347-368 1961. [1]. Petrou and A. Kadyrov Affne nvarant features from the trace transform EEE Trans. Pattern Analyss and achne ntellgence vol. 6 pp. 3-44 4. [11] Y. Zhang and C. H. Chu One-dmensonal mappngs for recoverng large scale projectve transformatons n Proc. EEE nt. Conf. Acoustcs Speech Sgnal Processng Honolulu vol. 1 pp. 11-14 7. Table 1. Ground truth and recovered values for mage par wth synthetc zoom and rotaton. Transformaton atrx Center No..5 -.43313..43313.5... 1..6945 -.449556..48431.614. -.499.5 1..671 138.359 Fg.. mage key after Transformaton (1) (left) and reconstructed usng recovered Transformaton () (rght). Table. Ground truth and recovered values for mage par wth synthetc zoom rotaton and projecton. Transformaton atrx Center No. -.15 -.1656..1656 -.15..9 -.5 1. -.14589 -.1449..6551 -.1199..1876 -.359 1. 138.638.41 1 3 4 Fg. 1. Orgnal mages key (left) and lock (rght) usedn the synthetc transformaton experments. Fg. 3. mage key after Transformaton (3) (left) and reconstructed usng recovered Transformaton (4) (rght).
Table 3. Ground truth and recovered values for mage par wth synthetc vewpont change. Transformaton atrx Center No.43 -.67..44 1.1... 1..438 -.687868..438915 1.944..147 -.871 1. 11 11 11.896 99.783 5 6 Table 5. Recovered transformatons for real mage pars. Transformaton atrx Center No.17873 -.571547..576715.16475..319.389 1. 31.833 54.713 9.88663.618915. -.461713.966568. -.46 -.689 1. 3.347 9.946 1 Fg. 4. mage lock after Transformaton (5) (left) and reconstructed usng recovered Transformaton (6) (rght). Fg. 6. Orgnal mages stop sgn (left) and fsh (rght)used n the real transformaton experments. Table 4. Ground truth and recovered values for mage par wth synthetc vewpont change wth perspectve effect. Transformaton atrx Center No.66.68. -.15.97. -.3.95 1..63417.739764. -.1878 1.496. -.3158.1456 1. 11 11 11.443 11.31 7 8 Fg. 7. mage stop sgn after zoom and rotaton (left) andthe reconstructed mage usng recovered Transformaton (9) (rght). Fg. 5. mage lock after Transformaton (7) (left) and reconstructed usng recovered Transformaton (8) (rght). Fg. 8. mage fsh after transformaton due to vewpont change (left) and the reconstructed mage recovered Transformaton (1) (rght).