CALIBRATION OF A STRUCTURED LIGHT-BASED STEREO CATADIOPTRIC SENSOR *

Transcription

1 AIBRATION OF A STRUTURED IGHT-BASED STEREO ATADIOPTRI SENSOR * Rdu Orghidn, Joquim Slvi, El Mustph Mouddi omputer Vision nd Rootics Group, Institute of Informtics nd Applictions, Universitt de Giron, Avd. luis Sntlo, s/n Giron, tloni (Spin) entre de Rootique, Electrotechnique et d Automtique, Université de Picrdie Jules Verne, Rue du Moulin Neuf 8 Amiens, Frnce ABSTRAT tdioptric sensors re comintions of mirrors nd lenses mde in order to otin ide field of vie. In this pper e propose ne sensor tht hs omnidirectionl vieing ility nd it lso provides depth informtion out the nery surrounding. The sensor is sed on conventionl cmer coupled ith lser emitter nd to hyperolic mirrors. Mthemticl formultion nd precise specifictions of the intrinsic nd etrinsic prmeters of the sensor re discussed. Our pproch overcomes limittions of the eisting omnidirectionl sensors nd eventully leds to reduced costs of production. Keyords: omnidirectionl vie, hyperolic mirror, lser, structured light, stereo vision 1. INTRODUTION A lrge field of vie is n ttrctive gol in computer vision. Potentil pplictions re in the domins of surveillnce, teleconferencing, unknon environment eplortion, tourism, dvertising etc. Moreover, egomotion estimtion nd oject trcking cn lrgely enefit from enhncing the field of vie of imging systems. The idel omnidirectionl device is system hving the cpility to see simultneously 36º in ll directions. In this y pnormic imge of the environment is continuously processed. There re mny proposls for uilding omnidirectionl sensors: ) specil lenses such s fish-eye lenses; ) multiple imge cquisition y mens of rotting cmers or using structures of mny cmers ith complementry fields of vie; nd c) cmers ith especil mirrors (plnr, hyperolic, prolic, sphericl, dul conve nd conic) [3]. * A common constrint upon the omnidirectionl sensors modelling requires tht ll the imged rys pss through n unique point clled single vie-point (SVP). This is n importnt feture ecuse it permits the distortion-free * Work funded y the Spnish project IYT TAP reconstruction of the imge. onsequently, the omnidirectionl sensors cn e clssified depending on hether they hve SVP or not. Nyr nd Bker [4] eplored the clss of SVP ctdioptric sensors tht produce SVP. Omnidirectionl vie cn e chieved y mens of fisheye lenses ith the disdvntge tht such lenses introduce rdil distortion tht complictes the mthemticl model. The otined imges hve good resolution on the centrl region ut hve poor resolution on the mrginl region. Furthermore, fish-eye lenses do not meet the SVP constrint. High resolution pnormic vies cn e otined using multiple imges provided y rotting cmer. The min drck of this method is tht the pnormic imge is not formed ith single cpture. Moreover, omnidirectionl vie is formed from severl prtil imges, hich cn led to discontinuities in the resulting imge if the cmer is not precisely controlled involving loer roustness nd constrining the use of these sensors in root pplictions. mers ith complementry fields of vie overcome some prolems of the previous method ecuse there re not moving prts ut the sensor ecomes more epensive nd difficult to clirte since it involves severl cmers. The comintions of mirrors (ctoptrics) nd lenses (dioptrics) is clled ctdioptrics. A 36º horiontl field of vie is chieved t every cpture using single cmer looking into conve mirror. For instnce, Pegrd nd Mouddi [] uilt such sensor ith conic mirror hile Ym et l. [4] used hyperolic mirror. Both otined good results for root nvigtion nd loclition. Stereo ctdioptric sensors re specil structures of ctdioptrics used to otin depth from to omnidirectionl imges. Nene nd Nyr [1] studied severl stereo configurtions using the mirrors hich hve SVP. Also, Gluckmn et l. [9] designed ctdioptric stereo sensor sed on to prolic mirrors ith to cmers. Southell et l. [7] studied the ide of tringultion y mens of ctdioptric sensor ith iloed mirror. ter, Ollis [6] simulted vrious configurtions using to hyperolic mirrors ith one nd

2 to cmers. Depth mesurements ere otined y tringultion. The correspondence prolem s solved y indo mtching eteen oth imges y mens of correltion lgorithm considering the curvture of the mirrors. Fil nd Bsu [5] uilt such sensor using emedded sphericl mirror loes. Depth mesurements ere lso otined y mens of feture etrction nd tringultion. Besides, structured light is technique idely used to mesure 3D surfces y the projection of sequence of light ptterns [11]. The mesuring surfce is emphsised y the pprent deformtions of the imged pttern from the projected one. Such light ptterns re projected y mens of slide projectors, digitl projectors or lser light emitters hich forms different shpe of ptterns. Slides re difficult to use ith clirted systems ecuse of the deformtions cused y the functioning temperture of the projector lmp. Besides, digitl projectors hve focusing prolems hich limits the depth rnge of the mesuring re. Finlly, lser light offers ttrctive fetures such s monochromtic light hich permits the use of opticl filters for stright-forrd imge segmenttion. ser projectors re thin nd cn e plced inside compct devices. The eperiments conducted y Bldin nd Bsu [8] demonstrte tht right lights improves mtching, therefore, lser light seems good choice. Hoever, lser emitters cn only project simple types of ptterns such s points, lines nd grids. This pper presents ne ctdioptric pnormic sensor ith n emedded structured light projector. In section, severl possile configurtions re compred nd the mtemticl model of the most desirle one is nlyed. Section 3 dels ith the design of the optiml sie of the mirrors. The clirtion method is descried in section 4. Then, section 5 presents the simulted results of the proposed sensor. The rticle ends ith conclusions.. SENSOR DESIGN tdioptric devices cn e clssified relted to the SVP constrint. For instnce, sphericl shped mirrors do not meet such constrint nd the lck of perspective point mkes more complicted the imge reconstruction process. Nevertheless, sphericl mirrors re preferred for mnufcturility resons nd for specific properties such s the compct sie or the equl chrcteristics over imuth [5][7][8]. The second ctegory contins the mirrors tht meet the SVP constrint: the prol, the ellipse nd the hyperol. The prolic mirror orks in the sme y s the prolic ntenn. The incoming rys pss through the focl point nd re then reflected prllel to the rotting is of the prol. Therefore, in order to use the SVP property, the imge must e formed y cmer ith orthogrphic lenses. This is n epensive type of cmer nd increses the price of the ctdioptric device. Besides, the ellipsoid mirror hs smll prcticl use since the reflecting prt nd the cmer must e on the inner side of the surfce providing n ineffective field of vie. Finlly, the hyperolic mirror cn e oserved using common perspective cmer, therefore is considered n ttrctive choice. In ddition, stereo omnidirectionl using structured light cn e reched y mens of comintions of to hyperolic mirrors, cmer nd lser projector (figure 1). The mirrors re symmetricl out the is so the idimensionl grphicl representtion is lloed. The first configurtion (fig. 1.) is sed on to identicl mirrors locted long seline hich cn e fied depending on the ccurcy desired. The light pttern is projected onto.. c. d. e. Figure 1. Stereo omnidirectionl systems (the dotted lines ound the cmer nd the lser self-occlusion regions). one of the mirrors hile the other mirror grs the scene. The mirrors re clled lser mirror nd cmer mirror, respectively. Such setting, mkes invlule prt of the imge ecuse of the scene occlusion cused y the lser projector. This prolem cn e solved y plcing oth mirrors ck to ck long the rottion is (see fig 1.). Such system removes the occlusion prolem ut the lser hs to e projected on the peripherl sie of the mirror (in order to e seen y the cmer mirror) here mjor lens distortion is present leding to poorer resolution of the system. Plcing the to mirrors y single trnsltion long the verticl is (see fig 1.c), permits the lser to e projected nery the centre of the mirror here etter resolution is given ut the deth re of the cmer mirror is sted leding to lrger system. A nturl solution is to put the lser mirror inside the deth cone given y the cmer mirror (Fig 1.d) otining compct sensor, yet, ith the inconvenient tht the lser mirror occludes prt of the vision field of the cmer mirror. In order to void this prolem nd to enhnce the seline, the to mirrors cn e seprted (Fig 1.e). Tking into ccount the previous oservtions, ne ctdioptric sensor is proposed (see fig.). The sensor is composed of to hyperolic mirrors, lser projector nd cmer. The cmer nd the lser emitter re oth represented using the pinhole model since the

3 projector cn e seen s n inverted cmer. The lser emits circle onto the lser mirror hich is projected forrd 36º ll round the mesuring scene nd ckrds to the cmer mirror into the imge plne. Depth informtion is retrieved y nlying the D projection on the cmer of the points of the scene enlightened y the lser pttern. These points re isolted y n opticl filter nd further imge segmenttion techniques. The cmer mirror complies ith the SVP criteri, therefore the line on hich lys n oject point P cn e determined from its projective imge point Pi. The lser mirror enefits of the sme properties, tht is: the line long hich point of the pttern is projected cn lso e defined. Depth informtion is otined y crossing oth lines. Herefter, complete description of the sensor is detiled. Figure. Stereo ctdioptric sensor ith structured light projector. The ngle of vie (α) of the sensor is the ngle eteen F, P1 nd F,, here F (,y, ) is the focl point of the cmer mirror, P1 is the highest reflecting point of the mirror nd is the projection of point Pm on the cmer mirror. The ngle α nd the rnge of vie (Rov) determine the position of the highest (PM) nd loest (Pm) points of scene imged y the sensor. These points re reflected on the lser mirror on points P1 nd P, respectively. The focl points of the lser mirror (F) nd the cmer mirror (F) re ligned long the rottion is nd trnslted distnce dff. The hyperolic shpes of the cmer nd the lser mirrors re modeled y equtions (1) nd (), respectively. (( dff) + + ) + y 1 Solving eqution (1) for to solutions re found, hich re the equtions of the upper (3) nd loer (4) hyperols. e + + y + (3) 1 e + y + (4) here, e +. The prmeters of ech mirror re supposed to e linked y k/, here k is constnt vlue. The focl point of the cmer (F cmer ) is plced in the focus of the loer hyperol, t distnce dfe from F. The light em emitted y n infinitely smll point of the pttern hits the lser mirror in point (P) tht cn e clculted y solving the system formed y equtions () nd the eqution of the line defined y the light em. According to the SVP constrint, P is projected long the line F, P t n oject point PW hich is projected onto the cmer mirror t (, y, ), otining eqution (5) nd (6). PW(,y, ) is determined y tringultion since it elongs to oth lines F, P nd F,. ( + + ) y y y y + y () 1 (5) There re to solutions of the system ut only the point closer to the emitting light source must e considered. The coordintes of s function of PW re given y eqution (7). y 1, 1, 1, y P e (6) ( e + P ) (7) 1, 1, here, P + y +. Besides, the points F, P nd P re on the sme line. Since the coordintes of the points F,, F nd P re knon, the point PW is uniquely determined. ( + + ) + y 1 (1) 3. DESIGN OF THE OPTIMA MIRROR SIZE An importnt concern hen designing the sensor is its sie. The sensor is designed to e used in indoor

4 environments so it hs to e smll. Besides, imge processing requires good resolution hich is improved y using lrge mirrors. A compromise must e set up in order to otin optiml performnces. Figure 3. The reltion eteen R top, the sie of the sensor nd the intrinsic prmeters of the cmer In figure 3, P y is the imge point of the cmer mirror rim on the D, d is the distnce from the centre of the D to P y, f is the focl distnce of the cmer nd h is the verticl distnce from the focl point of the mirror to its order. Idelly, the cmer mirror is imged y the cmer s disc tngent to the D order, therefore, severl constrints hve to e tken into ccount during the design process of the sensor: onstrint 1. The point P1 must e on the line F, PM (fig 1.) so tht the cmer mirror is le to see the highest points of the required field of vie. onstrint. The points of the cmer mirror rim must e close to the line P y, P1 for n optimlly sied imge in the cmer. onstrint 3. From severl suitle hyperolic shpes for the cmer mirror the one ith the igger deth re is chosen since it ill provide lrger cone for plcing the structured light projector. A numeric lgorithm produces ll hyperolic shpes ithin given rnge for specific vlues of α, R top nd Rov. Every shpe is compred ith the idel one nd the shpes ith n error elo threshold re kept. According to third constrint, the desired one is selected. The reltion eteen r d nd FF is shon in figure 3. Vlues for severl shpes hve een clculted nd re presented in Tle 1. The most suitle one depends on the finl ppliction of the system. Figure 4. Vlues of r d nd dff for mirror shpes ith the prmeter vrying from up to 15. Tle 1. The prmeters for suitle sensors α R top H r top df AIBRATION In the folloing, the clirtion of the sensor is done in to steps considering first the clirtion of the pir cmer nd cmer mirror (see section 3.1) hich is then used for the clirtion of the pir lser nd lser mirror (see section 3.) The pir cmer nd cmer mirror. The pir formed y the cmer nd its coupled mirror is modelled y eqution (8). D M W Pi K K M F( KW P) (8) here: W P is knon oject point respect to the orld coordinte system, P i is the imge point corresponding to the projection of W P, M K W is the trnsformtion mtri tht reltes the orld coordinte system ith respect to the mirror coordinte system, K M is the trnsformtion mtri tht reltes the mirror coordinte system ith respect to the cmer coordinte system, D K is the projection mtri tht models the internl geometry of the cmer y set of intrinsic prmeters: α u u (9) D K α v v 1 here: α u f ku, α v f kv So, the projection of n oject point W P onto the mirror (otining the point ) is defined y non-liner function detiled in eqution (1). T y ( e P ) + (1) F( PW ) 1 P e Besides, the orld coordinte system might coincide ith the cmer coordinte system: M K W M K nd K M W K M inv( M K W ). Therefore, eqution (8) cn e ritten s: D W P K K F( inv( K ) P) (11) i M M nd, epnding the mtrices e otin eqution (1):

5 1 F (1) s i α u u 1 Fy s yi αv v 1 t F s Summriing eqution (1), it is ovious tht there re seven unknons: ) the four intrinsic prmeters of the cmer; ) the trnsltion vector corresponding to the trnsformtion mtri eteen the mirror nd the cmer coordinte system; nd c) the nd prmeters tht define the hyperolic shpe of the cmer mirror. The clirtion method otins the vlues of these seven unknons y mens of non-liner minimition of eqution (1) sed on evenerg-mrqurd nd set of knon oject points (clirting pttern). incoming nd outgoing lser rys. The sensor is plced in virtul room hose lls plne equtions re knon. The room is 3 meters nd.5 meters height. At clirtion time, the sensor s plced t 1 meter height from the floor. 3.. The pir lser nd lser mirror. The lser source emits opticl circle forming cone hich is determined y the lser pose nd the perture ngle of the lens. Tht cone is then projected onto the lser mirror tords the 3D scene spreding the light in set of opticl rys. Every opticl ry intersects ith knon clirting plnes otining set of 3D points W P hich re imged y the cmer system. Besides the perture ngle γ is knon nd given y mnufcturers. Then, the reltionship eteen the lser prmeters nd the set of 3D points is determined y eqution (13). tn( γ ) 4 ( e (( + W W ) ( y ) ( W ( e W + df dff (( ) ( ) W W + + W + yw ) ) ) ( + ( y ) y + ) yw ) ) (13) Figure 4. Virtul sensor used for clirtion The coordintes of the points W P re otined from their D projection on the imge plne nd the prmeters of the cmer system given y the clirting lgorithm eplined in section 3.1. Eqution (13) is composed y only four unknons prmeters: ) the mirror shpe prmeters, ; ) dff, the trnsltion eteen the orld coordinte system nd the focl point of the lser mirror nd c) df, the distnce eteen the lser source point nd the focl point of the nd the corresponding mirror. The solution of such non-liner system is lso otined y minimition lgorithm sed on evenerg-mrqurt. A set of itertions hs een provided to the clirting lgorithm considering Gussin noise from σ up to σ in increments of.1. In Figure 5 is presented the discrepncy eteen the 3D oject points (epressed in millimetres) given y the proper model (ith ero noise) ith respect to the model otined y clirtion in the presence of noise. Besides, Figure 6 shos the sme discrepncy ut ith respect to the imge points, represented in piels. 5. EXPERIMENTA RESUTS The hole model presented in section 3 hs een simulted in Mtl ith the im of nlysing the ccurcy of the clirting lgorithm in the presence of Gussin noise. The complete virtul sensor used is shon in figure 3. The reflecting shpes of the to mirrors cn e oserved. The dotted lines represent the Figure 5. Distnce eteen the pttern 3D points nd the reconstructed 3D points y mens of the model otined y clirtion in the presence of noise.

6 Figure 6. Distnce eteen the noiseless imge points nd the reconstructed D points using the model otined y clirtion in the presence of noise. 6. ONUSIONS Pssive stereo ctdioptric sensors re structures uilt in order to provide 3D informtion using pnormic imges. Depth is clculted y tringultion oserving the sme fetures from different points of vie. The min dvntge of such sensors is tht depth is clculted simultneously round field of vie of 36º hich mkes them suitle for rel-time pplictions. Hoever, stereo techniques hve mjor limittion relted to the high computtionl cost of the imge segmenttion stge nd the further mtching solving process. Introducing structured light projection, the oserved points re identified fster. Moreover, y using monochromtic lser projector, the pttern cn e isolted y mens of opticl filters. This pper proposes ne device tht permits the cquisition of three dimensionl informtion from unique imge voiding the correspondence prolem present in most stereo vision systems. Moreover, the use of monochromtic light implies rpid nd ccurte imge segmenttion reducing the computtionl requirements. A set of suitle lenses nd mirror shpes nd severl possile pnormic ctdioptrics sensors re presented nd, finlly, ne stereo ctdioptric sensor sed on lser structured light projection is introduced nd studied. This ork provides etensive informtion on the modelling nd using of such sensors nd represents compulsory step in the designing process. onference on Rootics nd Automtion, pge(s): vol.1, -8, Apr [5] M. Fil, A. Bsu, Feture etrction nd clirtion for stereo reconstruction using non-svp optics in pnormic stereo-vision sensor, Proceedings of the IEEE Workshop on Omni directionl Vision, Pge(s): 79-86,. [6] M. Ollis, H. Hermn nd S. Singh, Anlysis nd Design of Pnormic Stereo Vision Using Equi-Angulr Piel mers, tech. report MU-RI-TR-99-4, Rootics Institute, rnegie Mellon University, Jnury, [7] D. Southell, A. Bsu, M. Fil, M., J. Reyd, Pnormic stereo, Proceedings of the 13th Interntionl onference on Pttern Recognition, Volume: 1, Pge(s): [8] J. Bldin, A. Bsu, 3D Estimtion Using Pnormic Stereo, Proceedings of 15th Interntionl onference on Pttern Recognition, Pge(s): 97-1 vol.1,. [9] J. Gluckmn, S.K. Nyr nd Thores K.J., Rel-Time Omnidirectionl nd Pnormic Stereo, Proceedings of the 1998 DARPA Imge Understnding Workshop, Monterey, liforni, Novemer [1] S.A. Nene, S.K. Nyr, Stereo ith mirrors, Sith Interntionl onference on omputer Vision, Pge(s): , [11] J. Btlle, E. Mouddi nd J. Slvi. A Survey: Recent Progress in oded Structured ight s Technique to Solve the orrespondence Prolem. Pttern Recognition 31(7), pp , July REFERENES [1] S. Bker nd S.K. Nyr, A theory of ctdioptric imge formtion, Sith Interntionl onference on omputer Vision, Pge(s): 35-4, [] K. Ym, Y. Ygi, M. Ychid, Omnidirectionl imging ith hyperoloidl projection, Proceedings of the 1993 IEEE/RSJ Interntionl onference on Intelligent Roots nd Systems '93, Volume:, Pge(s): , 1993 [3] Y. Ygi, M.Ychid, Omnidirectionl Sensing nd Its Applictions, IEIE Trns. Inf. & SYST., Vol.E8-D, No.3, Mrch [4]. Pegrd, E.M. Mouddi, A moile root using pnormic vie,proceedings. of IEEE Interntionl