Projector-Camera Based System for Fast Object Modeling

Transcription

1 Projector-Camera Based System for Fast Object Modelng Guanghu Wang a,b, Zhany Hu a, Fuchao Wu a, Hung-at su b a. Natonal Laboratory of Pattern Recognton, Insttute of Automaton Chnese Academy of Scences, Bejng , P. R. Chna b. Department of Electronc Engneerng, he Chnese Unersty of Hong Kong Shatn, N.. Hong Kong ghwang@ee.cuhk.edu.hk, huzy@nlpr.a.ac.cn Abstract In ths paper, a practcal projector-camera based system for fast object modelng and reconstructon s reported. he system conssts of an off-the-shelf dgtal projector, a deo camera and a turntable. he system calbraton s mplemented accurately and automatcally by takng a sngle mage of an llumnated calbraton template. he multple color lght strpes emtted by the projector adopt only three colors and the color can be modfed easly accordng to the surface texture of the object. hs makes the system wdely applcable n many stuatons, and the assumpton about color neutralty, whch s a lmtaton to other smlar systems, can be largely relaxed. he regstraton and mergng algorthm s greatly smplfed by employng a turntable. he aldty and hgh accuracy of the method are demonstrated by extense real experments. 1. Introducton 3D reconstructon and modelng of objects s one of the man objectes of computer son and attracts a lot of attentons n recent years. Examples and applcatons of ths task nclude robot nagaton and obstacle recognton, augmented rtual realty, ndustral nspecton, reerse engneerng, medcal dagnoss and many more. Among arous modelng and reconstructon technques, the projector-camera based method (also called structured lght) s wdely utlzed because of ts smplcty and hgh accuracy. ypcally, a projector-camera system conssts of a camera and a projector. he projector projects an llumnated lght strpes or lght patterns on the object surface. he camera captures mages of the llumnated object, then the 3D nformaton of the sble surfaces can be recoered by usng the calbrated parameters of the system a trangulatons. Varous lght patterns are adopted by preous researchers, such as lght strps, lght grd/net, lght beams/dots, etc. he more complex the lght patterns are, the more dffcult the processng of the mages wll be. Most of the exstng systems adopt strp pattern, snce t s easy to recognze, sample and model [1][]. Many projector-camera based systems hae been proposed n the lterature [3][4][5][6]. Most commercal laser scanners use only a sngle strpe and sweep the strpe across the surface so as to obtan a full range mage. A long tme s usually requred to acqure mage data. In order to speed up the samplng speed, some researchers adopt multple strpes. Such a system must take specal care for the determnaton of strpe correspondence. here are three major ways n dong ths, and a good reew on the technques s gen by Batlle [1]. he frst way s by surface contnuty assumpton so that adjacent projected strpes are adjacent n the mage. hs assumpton holds true only f the obsered surface does not hae self-occluson or dsconnected components. he second way s dfferentatng the strpes by colors [3]. It may fal f the surface s textured rather than hang only neutral colors. he thrd way s to code the strpes by aryng ther llumnaton oer tme [4][5][6]. hs needs seeral frames to decode the lght and cannot be appled n the measurement of mong object. System calbraton s an mportant step for a projectorcamera system. he exstng calbraton methods for structured strpes can be broadly dded nto two categores. In the frst one, the system s calbrated n two stages: camera calbraton and projector calbraton. Usually, a standard calbraton scheme s adopted to estmate the projecton matrx and camera parameters. On the other hand, the projector can be calbrated n seeral ways. In [7], the lght strpe s modeled as a lght strpe plane and the calbraton s to recoer the parameters of the plane. McIor [8] and Valkenburg [5] model the projector as a reerse pnhole camera wth 1D mage that can be defned by a 4 matrx. Shen and Meng [] propose a method to calbrate all lght beams projected from the dgtal projector. In the second category, only a 4 3 mage to world conerson matrx s calbrated for

2 each strpe. It conerts mage ponts on a lght strpe nto ther correspondng 3D coordnates. o obtan the matrx, some world to mage pont correspondences should be establshed on each strpe. Howeer, t s n fact ery dffcult to precsely locate ponts on the lght strpe n the world system. Preous researchers tackle ths problem by employng some calbraton artfacts to generate the pont correspondences usng world lne to mage pont [9], world plane to mage pont [10], and world pont to mage pont correspondences [11]. Most of the aboe calbraton methods assume pnhole camera model, and a hgh precson mong mechansm s requred durng calbraton. hs paper manly focuses on deelopng a practcal and automatc system for fast object modelng. here are three man features of our system. Frst, the calbraton of the system s accurate and can be mplemented automatcally from a sngle llumnated mage. Second, the lght strpes are encoded by three dfferent colors that can be modfed flexbly accordng to the surface texture of the object. hs can, to some extent, oercome the lmtaton of other smlar systems, whch strongly depend on the color neutralty assumpton of the object. hrd, the regstraton and mergng algorthm s greatly smplfed by employng a turntable. he remanng parts of ths paper are organzed as follows. Secton ges a bref descrpton of the system confguraton and characterstc. Secton 3 presents a smple and robust method of system calbraton and object modelng. Some experment results and analyss are gen n secton 4, followed by a concluson at the end of ths paper. Fg.1. he operaton procedure s qute easy: Put the object to be modeled on the turntable, take an llumnated mage n addton to a pure texture mage, the sble surface can be reconstructed automatcally from the llumnated mage, whle the texture mage s used for texture mappng after reconstructon so as to form a more realstc object model. Rotate the turntable to another poston and repeat the same procedure untl eery surface s reconstructed, then the complete model s obtaned by mergng the reconstructed results together. he turntable here s optonal, neertheless, the regstraton problem,.e. transformng measurement from dfferent ews nto one common coordnate frame, can be greatly smplfed by employng the turntable. For example, f we choose the rotaton axs of the turntable as the Z-axs of the world coordnate system, the measurements at dfferent ewpont s smply related wth a rotaton matrx R (α ), where, α s the rotaton angle between two ews. cosα, snα, 0 R ( α) = snα,cosα, 0. 0, 0, 1. System descrpton urn table Lght plane Projector π Computer Object Camera Fg.1 Schematc fgure of the system confguraton he system conssts of an off-the-shelf dgtal projector, a deo camera and a turntable, as shown n Fg. Object llumnated by color lght strpes he reason for adoptng a deo projector s that t s wdely aalable, and can allow us to experment dfferent type of lght patterns. he lght pattern used n the system s multple color strpes, so the measurement on sble surface of the object can be taken from a sngle shot, ths makes the system fast and capable of modelng mong object. As shown n Fg., only three colors are used n the system, where, two colors project alternately, whle the thrd color (marked wth upward dagonals n Fg.) s only appled to the center strpe. hs specal strpe s taken as an ndex mark to sole the correspondence problem between dfferent strpes. Obously, ths s under the assumpton of space contnuty and hard to deal wth the scene wth seere self-occluson. Howeer, ths tradeoff makes the system

3 computatonally nexpense and easy to process automatcally, and n some cases, een the scene s dscontnuous (as the case n Fg.), the strpe correspondences can also be soled by ths specal strpe. Furthermore, the assumpton about color neutralty, whch s a lmtaton to other color structured lght system, can be relaxed, snce the three colors can be changed flexbly accordng to the texture property of the object to be modeled. We use the edge of each strpe to model the lght plane n practce, and the edge can be detected n sub-pxel accuracy [5,1]. System calbraton s another crucal step for the accuracy of measurement. he calbraton s done once for a fxed system. Howeer, when the parameters of some components, such as the focal length of projector or camera, are adjusted or the poston changes, the whole system needs to be re-calbrated. So t s mportant to fnd a smple, accurate and automatc calbraton method. In our system, calbraton s accomplshed automatcally by takng one shot of a calbraton template. he template s composed of two orthogonal planar surfaces, wth 18 crcular black fducal marks eenly dstrbuted on the two whte surfaces. he poston of each mark on the template s known wth hgh accuracy, whle the correspondng coordnates n the mage s estmated n sub-pxel accuracy by computng the centrod of each mark [8,13]. Fg.3 shows a schematc fgure of the template, where, only 18 fducal marks and one lght strpe edge are dsplayed. 3. Method of calbraton and modelng 3.1. System calbraton Under perspecte projecton, a 3D pont x = [ x, y, z] n space s projected to an mage pont m = [ u, ] a a 3 4 rank 3 projecton matrx P as: s m = P x (1) where, s s an unknown nonzero scalar; m and x are the homogeneous coordnate ectors of m and x n form of m = [ m,1], x = [ x,1]. he projecton matrx has 11 degrees of freedom snce t can only be defned meanngfully up to a scale factor, and each world to mage correspondence can ge rse to two ndependent lnear constrants on the projecton matrx as follows: A 1 p = 0 () x, y, z,1,0,0,0,0, ux, uy, uz, u where, A 1 =, 0,0,0,0, x, y, z,1, x, y, z, p [ p, p, p, p, p, p, p, p, p, p, p, p ], = p, s the (, j) -th element of matrx P. Gen N j ( N 6 ) pont correspondences, a N 1 matrx A s formed by stackng equaton () for each pont par. heoretcally, A s of rank 11, and t may be of full rank (wth rank 1) for non-perfect data wth nose. Howeer, p can be estmated by Sngular Value Decomposton (SVD) of matrx A. he soluton of p s the unt sngular ector correspondng to the smallest sngular alue. Snce the precson of projecton matrx s closely related to that of the whole system, we ntroduce a Maxmum Lkelhood Estmaton (MLE) algorthm to optmze the soluton. mn p d ( m, Px ), = 1,..., N (3) he problem can be soled a an terate method such as Gauss-Newton or Leenberg-Marquardt [14], whle the preous SVD estmaton s taken as the ntal alue of the teraton. L π h Fg.3 Schematc fgure for calbraton template Next, we wll present the method of lght strpe calbraton. Select the world coordnate system as shown n Fg.3, then the coordnates of the two orthogonal planes of the calbraton template are π = [0,1,0,0 ] and π = [0,0,1,0 ]. Suppose the -th lght strpe plane π h ntersects the plane π and π h at L and L respectely, ther correspondng mage can be extracted by edge detecton n sub-pxel precson. Let m ( j = 1,..., N ) be j the detected edge ponts of the mage of L, m k ( k = 1,..., N ) be that of L, then the correspondng space pont x j wth respect to m can be determned by the j ntersecton of the back-projected ray of m wth plane j π,.e. s m = j P xj (4) π x = 0 j It s easy to compute the coordnates of x j from equaton (4), and so does the coordnates of pont x k correspondng to m k. Snce ponts x and x are j k nonlnear and le on the same optcal plane π, so we hae: L π X Y Z

4 x j π = 0, j = 1,..., N, k = 1,..., N xk rewrte the equaton nto a lnear system as: A π = 0 (5) where, A s a matrx of dmenson ( N + N ) 4 formed by stackng all the homogeneous ectors of x and x j k. hus, the coordnates of plane π s computed by SVD decomposton, and a more fathful estmaton can be obtaned a some nonlnear terate algorthm by mnmzng the followng cost functon: f ( π ) = d ( x, π ) + d ( x, π ) (6) j j In the same way, we can calbrate all the lght strpe planes llumnated on the template. 3. Object modelng he computaton of the 3D model of the object s straghtforward once the system s calbrated. he algorthm can be summarzed as follows. ). Extract the edge ponts of each strpe from the mage of the llumnated object; ). Establsh the correspondences between the lght planes and the extracted edges by rtue of the specal strpe; ). 3D reconstructon by computng the ntersecton of the back-projected ray of each edge pont wth ts correspondng lght plane; ). Obtan the complete model by mergng the reconstructon results of each sde of the object. 4. est results he projector used n our system s Sony VPL-PX11, wth a resoluton of ; the dgtal camera s Nkon Coolpx 990 wth an USB lnk, whose resoluton s set to ; the turntable s just a manually operated one. he workng olume depends on the focusng capablty and the feld of camera s ew, usually, the object s placed at mm from the system. Durng the experment, totally 18 lght strpes are projected, among whch only 11 strpes are calbrated, snce other strpes fall outsde the calbraton template. After calbraton, we reconstruct the template accordng to the detected edges of each strpe, as shown n Fg.4, a total of 8046 ponts are reconstructed, where, ponts belong to the plane π of the template, and other ponts belong to π h. We also calculate the dstance of each reconstructed pont to the correspondng plane, the aerage dstance to π s mm, standard deaton s 0.153, whle the aerage dstance and k k 5 standard deaton to π h are mm and respectely. he results ndcate that the reconstructed model s of hgher accuracy. Fg.4 he reconstructed dense 3D ponts of the calbraton template under dfferent ewpont Fg.5 shows a reconstructon result of a olleyball, where, the upper left fgure s a texture mage taken by the camera, the upper rght one s the reconstructed 3D mesh, whle the lower two fgures are the reconstructed surface wth texture mappng. We ft the reconstructed ponts nto a ball equaton a a least squares technque as ( x 8.) + ( y 4.6) + ( z 10.5) = 104.1, ths result s ery close to the ground truth that the radus of the olleyball s 103.8mm. Note that only a sngle mage s used here, thus the equaton s estmated only from the ponts on the sble part of the surface, and the true alue s taken manually on the spot. Fg.5 he texture mage of a olleyball (upper left), reconstructed 3D mesh (upper rght) and the reconstructon result wth texture mappng (lower two)

5 Fg.6 hree pars of statue mages wth texture and llumnated strpes Fg.7 Vsble surface reconstructon correspondng to Fg.6 Fg.8 Complete model of plaster statue n VRML format (left two) and 3D ponts (rght)

6 We also use the system to model a plaster statue. Put the statue on the turntable and rotate the table to four dfferent postons, a texture mage and an mage llumnated by lght strpes are taken at each poston, so we hae four pars of mages altogether. Fg.6 shows three pars of the mages. Fg.7 ges the correspondng reconstructed 3D ponts and the results wth texture mappng. Fg.8 shows the complete statue model by mergng four sde results, a total of ponts are reconstructed for the complete model, but only one thrd of the ponts are shown n Fg.8. he reconstructed model s satsfactory and looks smlar to the orgnal one. Howeer, t s ery dffcult to obtan a representate ground truth for the model due to ts cured shape. Besdes, some small parts cannot be reconstructed by the system due to occluson. 5. Conclusons We reported a practcal modelng and reconstructon system based on color lght strpes. he system calbraton s carred out automatcally by takng a sngle mage of a calbraton template. he structured lght strpes hae only three colors and the color can be modfed easly accordng to the surface texture of the object. hs makes the method wdely applcable n many stuatons. he regstraton and mergng algorthm s greatly smplfed by employng a turntable. he aldty and hgh accuracy of the method are demonstrated by extense real experments. Generally speakng, the accuracy of the system depends on seeral factors: the resoluton and optcal qualty of the projector and camera, the baselne length between the projector and camera wth respect to the object, the reflecte propertes of object surface, etc. In practce, a tradeoff has to be made between the accuracy and complexty of data processng. he process of calbraton and reconstructon of our system s bascally automatc. hus t s ery easy to operate. Howeer, t s dffcult for the system to deal wth complcated object wth seere self-occlusons at present stage. Another thng to be noted s that lens dstorton s not consdered here snce the dstorton of the camera and projector used n our system s neglgble. Howeer, t must be taken nto account durng system calbraton f the lens dstorton does play a sgnfcant role n system error. Fnally, some of the test results and reconstructed 3D models can be downloaded from the author s homepage: Acknowledgements: he work s partally supported by the Natonal Natural Scence Foundaton of Chna under grant No , the Natonal Hgh echnology R&D Program under grant No. 00AA and the Hong Kong RGC Grant CUHK 4378/0E. References [1]. J. Batlle, E. Mouaddb and J. Sal, Recent progress n coded structured lght as a technque to sole the correspondence problem: a surey, Pattern Recognton, Vol.31(7), , (1998). [].. S. Shen and C. H. Menq, Dgtal projector calbraton for 3-D acte son systems, Journal of Manufacturng Scence and Engneerng, Vol.14(), , (00). [3]. C. Rocchn, P. Cgnon, C. Montan, P. Png and R. Scopgno, A low cost 3D scanner based on structured lght, Computer Graphcs forum. Vol.0(3), (001). [4]. S. Rusnkewcz, O. Hall-Holt, M. Leoy. Real-tme 3D model acquston, In: Proc. Sggraph 00, pp , (00). [5]. R. J. Valkenburg and A. M. McIor, Accurate 3D measurement usng a structured Lght System, Image and Vson Computng, Vol.16(), , (1998). [6]. J. Guehrng, Dense 3-D surface acquston by structured lght usng off-the-shelf components, In: Proceedngs of SPIE, Vol. 4309, pp. 0-31, (000). [7].. Stahs and F. Wahl, Fast and Versatle Range Data Acquston n a Robot Work Cell, In. Proc. IEEE Internatonal Conference on Intellgent Robots and Systems, Ralegh, N.C., pp , (199). [8]. A. M. McIor, Calbraton of a laser strpe profler, In: Proc. Second Internatonal Conference on 3-D Imagng and Modelng, Ottawa, Canada, pp. 9-98, (1999). [9]. C. H. Chen and A. C. Kak, Modelng and Calbraton of a Structured Lght Scanner for 3-D Robot Vson, In: Proc. IEEE Internatonal Conference on Robotcs and Automaton, Ralegh, NC, USA, pp , (1987). [10]. I. D. Red, Projecte Calbraton of a Laser-strpe Range Fnder, Image and Vson Computng, Vol. 14(9), , (1996). [11]. D. Q. Huynh, R. A. Owens and P. E. Hartmann, Calbratng a Structured Lght Strpe System: A Noel Approach, Internatonal Journal of Computer Vson, Vol. 33(1), 73-86, (1999). [1]. M. robna, Error model of a coded-lght range sensor, echncal Report BIWI-R164, EH Zurch, Swtzerland, (1995). [13]. M.A.G. Izquerdo, M.. Sanchez, A. Ibanez and L.G. Ullate, Sub-pxel measurement of 3D surfaces by laser scannng, Sensors and Actuators A: Physcal, Vol.76(1-3), 1-8, (1999). [14]. R. Hartley and A. Zsserman, Multple Vew Geometry n Computer Vson, Cambrdge Unersty Press, (000).