A multiview 3D modeling system based on stereo vision techniques

Transcription

1 Mchine Vision nd Applictions (2005) 16: Digitl Oject Identifier (DOI) /s Mchine Vision nd Applictions A multiview 3D modeling system sed on stereo vision techniques Soon-Yong Prk 1, Murli Suro 2 1 Computer Engineering Deprtment, Kyungpook Ntionl University, Degu, Kore 2 Electricl nd Computer Engineering Deprtment, Stte University of New York t Stony Brook, Stony Brook, NY , USA Received: 2 August 2003 / Accepted: 20 Septemer 2004 Pulished online: 25 Ferury 2005 c Springer-Verlg 2005 Astrct. This pper introduces stereo vision system to utomticlly generte 3D models of rel ojects. 3D model genertion is sed on the merging of multiview rnge imges otined from digitl stereo cmer. Stereo imges otined from the cmer re rectified, nd correltion-sed stereo mtching technique reconstructs rnge imges from them. A turntle stge is lso employed to otin multiple rnge imges of the ojects. To register rnge imges into common coordinte system utomticlly, we introduce nd clirte turntle coordinte system with respect to the cmer coordinte system. After the registrtion of multiview rnge imges, 3D model is reconstructed using volumetric integrtion technique. Error nlysis on turntle clirtion nd 3D model reconstruction shows the ccurcy of our 3D modeling system. Keywords: Stereo vision Multiview 3D modeling Turntle clirtion 1 Introduction Generting complete 3D model of n oject hs een topic of much interest in recent computer vision nd computer grphics reserch. Mny computer vision techniques hve een investigted to generte complete 3D models. There re two mjor pproches in this reserch.the first one is sed on merging multiview rnge imges into complete 3D model [4,7,23]. The second one is sed on processing photogrphic imges using volumetric reconstruction technique, such s voxel coloring nd shpe-from-silhouettes [5,21]. This pper presents computer vision system to utomticlly generte 3D computer models y merging multiview rnge imges of rel ojects. We employ stereo vision cmer nd turntle stge to develop n utomtic nd inexpensive vision system. Multiview 3D modeling hs een done y mny ctive or pssive rnging techniques. Lser rnge imging nd structured light techniques re the most common ctive techniques. These techniques project specil light ptterns onto the surfce of rel oject to mesure the depth to the surfce y simple tringultion technique [4,7]. Some common pproches Correspondence to: S.Y. Prk (e-mil: syprk@mil.knu.c.kr) of the structured light technique employ single line pttern [9], multiline pttern [17], color-coded pttern [24], nd spce-time coded pttern [19]. Advntges of using the ctive techniques re ccurcy nd speed of depth cquisition [24, 19]. However, ctive techniques re still more expensive thn pssive techniques. In contrst, however, reltively less reserch hs een done using pssive techniques, such s stereo imge nlysis. This is minly due to the inherent prolems (e.g., mismtching nd occlusion) of stereo mtching. Severl stereo mtching techniques hve een introduced; however, only few of them re empolyed for multiview 3D modeling [20]. Okutomi et l. [13] presented multiseline stereo mtching technique to reduce mtching miguity, nd their pproch is empolyed in mny multiview 3D modeling techniques [18]. Chen nd Medioni [3] used stereo cmer to otin rnge imges nd integrted them using volumetric method. In ech viewpoint, they used 3D voxel grid to find disprity surfce using dynmic progrmming technique. Rnder et l. [18] nd Vedulr et l. [22] lso used stereo vision techniques to crete 3D models of dynmic scene for virtul relity pplictions. They used considerle numer of video cmers nd multiple-seline stereo mtching technique proposed y [13]. In order to generte complete 3D models, we otin multiview rnge imges using stereo vision techniques. We use two inexpensive digitl still cmers to cpture stereo imges of n oject. The cmers re clirted y projective clirtion technique, nd stereo imges from them re rectified ccordingly. A rnge imge is then otined from pir of rectified stereo imges. Multiview rnge imges re otined y chnging the viewing direction to the oject. Different pproches to chnging viewing direction exist. Among them re moving oject on turntle with fixed sensor [4,5], moving sensor with fixed oject [1,12], nd other vritions [7,8]. One dvntge of using turntle is the ese of clirtion etween different views. We lso employ turntle stge to rotte the oject nd to otin multiple rnge imges. Multiple rnge imges re then registered nd integrted into single 3D model. In order to register rnge imges utomticlly, we define nd clirte turntle coordinte system (TCS) with respect to the cmer s coordinte system (CCS). To integrte multiple rnge imges into single mesh model, we use vol-

2 S.-Y. Prk, M. Suro: A multiview 3D modeling system sed on stereo vision techniques 149 umetric integrtion technique [10,14]. Error nlysis on rel ojects shows the ccurcy of our 3D model reconstruction. Section 2 presents the clirtion nd rectifiction of stereo imges nd comprison of 3D rnge reconstruction techniques. Section 3 presents the definition nd clirtion of the TCS with respect to the CCS. In Sect. 4, we present 3D modeling technique of merging multiview rnge imges nd error nlysis of our 3D modeling system. Finlly, we conclude the pper in Sect Rnge imge cquisition 2.1 Stereo clirtion nd rectifiction In this pper, we employ projective cmer model to clirte our stereo cmer. Clirtion of the projective cmer model cn e considered s n estimtion of projective trnsformtion mtrix from the world coordinte system (WCS) to the cmer s coordinte system (CCS). Let w =[xyz] T e the coordintes of 3D point W with respect to the WCS, p =[uv] T the coordintes of the projection of w to the retinl (CCD) plne of cmer, nd p =[u v ] T the coordintes of p in the picture plne (pixels). The mpping from 3D coordintes to 2D coordintes is the perspective projection, which is represented y liner trnsformtion in homogeneous coordintes. Let p =[uv1] T, p =[u v 1] T, nd w =[xyz1] T e the homogeneous coordintes of p, p, nd w, respectively. Then 3 4 perspective trnsformtion is given y mtrix M: p = K p = K M w, (1) where = mens equl up to scle fctor. The cmer is therefore modeled y trnsformtion mtrix K nd its perspective projection mtrix (PPM) M, which cn e decomposed into the product M = A[R t]. (2) The mtrices K nd A depend on the intrinsic prmeters only nd hve the following forms: K = k u k v 0, A = f u γu 0 0 f v v 0, (3) where, f u, f v re the focl lengths in the horizontl nd the verticl directions, k u, k v re the scling fctors from the retinl plne to the picture plne, (u 0,v 0 ) re the coordintes of the principl point in the retinl plne, nd γ is skew fctor. Stereo rectifiction determines trnsformtion of ech imge plne such tht pirs of conjugte epipolr lines ecome prllel to the horizontl imge xes. Using projection mtrices of the left nd the right cmers of the stereo vision system, we rectify stereo imges y using the rectifiction technique investigted y Fusiello et l. [6]. A trnsformtion mtrix T i tht rectifies homogeneous pixel p o in n originl imge plne to new pixel position p n is estimted s p n = T i p o. (4) The picture coordintes p n =[u,v, 1] T of the imge point p n re then otined y multiplying the trnsformtion mtrix K to the imge coordintes: p n = K p n. (5) However, when we sve rectified imge to 2D rry of picture frme, we need to consider the trnsltion of the principl point. Otherwise, we my lose some portion of the imge outside of the originl picture frme. This is ecuse of n offset etween the originl principl point (u o0,v o0 ) nd the new principl point (u n0,v n0 ), which is due to the rottion of the opticl xis of the cmer. In order to trnslte the rectified imge ck into the picture frme, we compute the new principl point (u n0,v n0 ) y dding the offset to the old principl point. The offset of the principl points cn e computed y mpping the origin of the retinl plne onto the new retinl plne: õ n = T i u o0 v o0, (6) 1 nd the new retinl coordintes re p n = K( p n õ n ). (7) We consider the offset only in the x direction ecuse rectifying the trnsformtion rottes the imge plne round the y xis. 2.2 Stereo system configurtion Our stereo cmer consists of two identicl digitl still cmers, which re Olympus C-3020 Zoom. The two cmers re instlled on verticl stereo mount. We fix the cmers on the mount with n ritrry toed-in ngle so tht the opticl xes of the cmers converge to out 600 mm from the cmer. Two digitl cmers re connected to personl computer, running on 1.8-GHz Intel Pentium, through two USB ports. Figure 1 shows picture of the stereo cmer nd turntle stge. Fig. 1. Stereo cmer system

3 150 S.-Y. Prk, M. Suro: A multiview 3D modeling system sed on stereo vision techniques YZ plne Z w Fig. 2. Checkerord pttern z f O w Y w x f X w y d x d XY plne We use checkerord pttern to clirte the stereo cmer. The pttern hs two plnes tht re prllel to the xy nd the yz plnes of the WCS. On ech plne re 48 control points tht compose set of 3D world coordinte points. Figure 2 shows digrm of the clirtion pttern. The specifictions of the pttern re s follows: Size of lck squre in x direction: x d =17.2mm. Size of lck squre in y direction: y d =17.18 mm. Offset to yz plne in x direction from origin: x f =6mm, which mens x = 6mmon the yz plne. Offset to edge of rightmost squre on xy plne in z direction: z f =11.5mm. 2.3 Stereo mtching From rectified stereo imge pir we cquire rnge imge y employing multiresolution stereo mtching technique using Gussin pyrmid [2]. A Gussin pyrmid for n imge I is sequence of copies of I, where ech successive copy hs hlf the resolution nd smple rte. The levels of Gussin pyrmid for given imge I re clculted s g k (i, j) = 2 2 m= 2 n= 2 w(m, n)g k 1 (2i + m, 2j + n), (8) g 0 (i, j) =I(i, j), where w(m, n) is 5 5 Gussin kernel. Becuse this kernel is seprle, we use 1D Gussin kernel w(m) whose length is 5. The weights of the Gussin kernel re w(0) = 0.4, w(1) = w( 1) = 0.25, w(2) = w( 2) = Three levels of Gussin pyrmid re used from level 0 to level 2. The level 0 imge corresponds to the originl imge, nd the level 2 imge corresponds to the smllest imge. The originl imge size is , nd the imge size t level 2 is We use vrile size of mtching lock for stereo mtching (15-2k) (15-2k) for the kth pyrmid level. An oject s silhouettes in the stereo imges re segmented y lue screen technique. A inry morphologicl closing nd opening opertion is used to remove noise in the imge segmenttion. The mtching lgorithm finds stereo correspondence only on the oject res in the left nd right imges. The oject s silhouettes re lso used lter for volume intersections in multiview integrtion process. SSD (sum of squred difference)-sed stereo mtching is done t ech level of the Gussin pyrmid from low resolution to high resolution. At the first level of the stereo mtching, the initil serch rnge of stereo disprity SR 0 t level 0 is set to [0,sr 0 ]. Then, t the lowest level of the pyrmid, where k =2, initil stereo disprity SR 2 ecomes [0,sr 0 /(2k)]. At successive levels of the pyrmid, the result of the stereo disprity t the lower levels decides the serch rnge of the corresponding level. If the disprity t the lower level is D i, then the serch rnge of the current level SR i is restricted to within [2 D i 2, 2 D i +2]so tht the stereo mtching lgorithm cn correct possile mismtches in the previous level. When there is pir of stereo imges, g k (l) nd g k (r) for left nd right imges, which re t level k of the pyrmid, SSD(i,j) t imge coordinte (i, j) is SSD(i, j) = m m k= m l= m { } g (l) k (i, j) g (r) k (i + k, j + l), (9) where 2m +1is the size of mtching lock. Figure 3 shows pir of rectified stereo imges of humn fce, nd Fig. 3 shows the result of 3D reconstruction. The depth to 3D point is mesured using the disprity etween projected points in stereo imges. The horizontl imge offsets õ n of the principl points re tken into ccount for depth mesure. In the left nd right retinl plnes, they re out (0.0742, 0) nd ( 1.299, 0) mm, respectively. In the next section, we compre two methods of depth computtion from stereo disprity. 2.4 Depth from tringultion After stereo rectifiction, we consider the new stereo configurtion s prllel stereo cmer. Therefore, we use simple Fig. 3. Stereo mtching results. Rectified left nd right stereo imges of humn fce. Texture-mpped rnge imge

4 S.-Y. Prk, M. Suro: A multiview 3D modeling system sed on stereo vision techniques 151 eqution for depth computtion. Let p l nd p r e the projections of 3D point w to the left nd right retinl plnes. If the disprity etween two imge points is d u in the x direction, the depth w z to 3D point from the origin of the cmer coordinte system (CCS) is w z = f B d u/k u +(u n1 u n2 ), (10) where B = c 1 c 2 is the length of the seline of the stereo cmer; in our system it is mm. u n1 nd u n2 re x coordintes of the new principl points in the left nd right imges, respectively. For the focl length f of the cmer we verge the clirtion results for oth left nd right focl lengths f u,f v, nd it turns out to e mm. 2.5 Depth of liner eqution The depth from two conjugte imge points is lso reconstructed y using Eq. 1. Given two conjugte points p 1 = [u 1,v 1, 1] T nd p 2 =[u 2,v 2, 1] T nd the two projection mtrices M n1 nd M n2, we cn write n overconstrined liner system: Aw = y, (11) where ( 1 u 1 3 ) T 14 + u 1 34 A = ( 2 v 1 3 ) T ( 1 u 2 3 ) T y = 24 + v u 2. (12) 34 ( 2 v 2 3 ) T 24 + v 2 34 Then w gives the position of the 3D point projected to the conjugte points. Column vectors i nd i re entry vectors of M n1 nd M n2, respectively. The 3D point w is represented with respect to the WCS. In this pper, however, we trnsform the world point to reference coordintes in order to represent it with respect to the CCS. Suppose we let the right cmer s coordinte system (RCCS) e the reference. Then we cn trnsform the point y simply using the externl clirtion prmeters [R t] of the right cmers. However, two trnsformtions cn e considered. One is to the old RCCS efore rectifiction, nd the other is to the new RCCS fter rectifiction. By tking into ccount multiview registrtion, which will e presented in the next section, we trnsform the point to the old RCCS y p r =[R o2 t] w, (13) where [R o2 t] is the old externl clirtion prmeters of the RCCS. Becuse the cmer needs to clirte the turntle for registrtion of multiview rnge imges, we represent ll rnge imges with respect to the old RCCS. 2.6 Comprison of reconstruction methods We compre the ccurcy of the two 3D reconstruction methods. To compre the results with the ground truth, we use nother checkerord pttern to compute the 3D positions of Fig. 4. A test checkerord pttern t 0 nd 45 Tle 1. Results of the width of the test pttern (ground truth is 100 mm) Method w 0 w 45 Liner eq. (mm) Tringultion (mm) ll control points. The pttern is plced on the tle nd rotted y 0 nd y 45. We tke pir of stereo imges t ech ngle, detect ll corners, nd compute the depth of the corners using the disprity etween their conjugtes. As shown in Fig. 4, we mesure the horizontl length etween two control points t the upper-left nd upper-right corners on the pttern. In order to minimize noise effects in the mesurement, we lso verge ll eight horizontl lengths etween the leftmost nd rightmost corners. The tringultion method using Eq. 10 shows smll reconstruction error. As shown in Tle 1, there is lso difference etween the two lengths mesured t 0 nd t 45. This difference cn cuse serious prolems if some multiview models re registered nd integrted into single 3D model. In fct, when we use this method to integrte multiple rnge imges, geometric distortion occurs on the 3D model. In contrst, using Eq. 11, we cn reconstruct more ccurtely the 3D model. Tle 1 shows tht the liner eqution method is more ccurte thn the tringultion method. 3 Turntle clirtion 3.1 Rottion xis clirtion As presented in n erlier section, we employ turntle to chnge the stereo cmer s viewing direction to n oject. To merge multiple rnge imges, we need to know the rigid trnsformtion of ech imge with respect to common coordinte system. Becuse we clirte the stereo cmer only once efore tking multiple rnge imges, ech rnge imge otined t different ngles hs n independent coordinte system. To register ll multiview rnge imges, we hve to know the rigid motions etween ll viewpoints. Suppose there re N viewing directions from V 0 to V N 1 nd the V 0 is the reference viewpoint. When there is 3D point p i i tht is otined nd represented y the ith view point,

5 152 S.-Y. Prk, M. Suro: A multiview 3D modeling system sed on stereo vision techniques we cn register it to new point p 0 i in the reference view s follows: p 0 i = T cs R i T 1 cs p i i, (14) where R i is the rottionl trnsformtion from V i to V 0, nd T cs is the trnsformtion from the TCS to the CCS s shown in Fig. 5, which is represented y T cs =[R cs t cs ]. (15) Let us define two independent coordinte systems in 3D spce, the WCS nd the TCS, whose origins re O w nd O s, respectively. Suppose we know the trnsformtion T cw, which is from the origin of the WCS O w to tht of the CCS O c.if we know nother trnsformtion T ws tht is from the origin of the TCS O s to tht of the WCS O w, then we cn derive the trnsformtion T cs = T cw T ws (16) =[R cw t cw ][R ws t ws ]. (17) Suppose there is the WCS in 3D spce with its origin t O w s shown in Fig. 6. In the figure, p 0 is the origin of the WCS (ut it is not necessry) nd p 0 is the sme point fter eing rotted y ngle θ long the Y s xis of the turntle. Given two 3D points nd the rottion xis Y s, we cn define plne Π s shown in the figure. Then we know the vector product (p 0 p 0 ) Y s =0. (18) In other words, p T 0x p 0x Y sx p 0y p 0y Y sy =0. (19) p 0z p 0z Y sz When we hve t lest three points in world coordintes, we cn solve n overdetermined liner eqution p 0x p 0x p 0y p 0y p 0z p 0z p 1x p 1x p 1y p 1y p 1z p Y sx 1z AY = Y sy =0... p Nx p Nx p Ny p Ny p Nz p Y sz Nz (20) X c O c Y c Z c T cw T cs Fig. 5. Geometry of the vision system θ Y w T ws X w O s Y s Z w Z s X s p' 0 O s θ -Y s Z w Z s t ws O w Fig. 6. Rottion xis clirtion with respect to the WCS using the SVD technique. When mtrix A is decomposed such tht A =(UDV T ), the solution of the eqution is column vector of V tht corresponds to the column of the lest eigenvlue in the D mtrix. We then normlize vector Y s to Ŷs. If the computed Y sy is negtive, then we chnge the direction of the xis so tht the xis is in the sme direction s the Y w xis of the WCS. To compute the X s nd Z s xes of the TCS, we pply the following computtions. Let us initilize the X s xis to (1.0,X sy, 1.0). Then X s Y s =0, X sy =( X sx Y sx X sz Y sz )/Y sy, ˆX s = X s / X s, nd Ẑs = ˆX s Ŷs. Finlly, the rottion mtrix from the turntle to the WCS is defined s ( ˆX s ) T T R ws = (Ŷs) T. (21) (Ẑs) T Let us now consider trnsltion from the origin of the TCS to the origin of the WCS. The origin of the TCS is defined s the intersection of the xis Y s nd the Π plne. If we trnsform two 3D points p 0 nd p 0 using the rottion in Eq. 21, the trnsformed points re on the xz plne of the TCS. Suppose the two points p 0 nd p 0 re trnsformed, with respect to the TCS, to new points p s0 nd p s0, respectively. Then the three points O s, p s0, nd p s0 re on the Π plne nd form n isosceles tringle. Therefore, using vector = p s0 p s0, where (p s0, p s0) =R T ws(p 0, p 0) nd the rottion ngle θ, we cn compute trnsltion vector t ws from p s0 to the origin O s. Let us consider the Π plne on which the origin is moved to p s0 nd the y component is zero. Then the center of rottion intersects with Π 3D point t i = [x, 0.0,z] T. Becuse the isosceles tringle is lso on the plne, the origin t i is one of the intersection points of the two circles c 1 nd c 2, s shown in Fig. 7. On the Π plne, the center of c 1 is X s p 0 Y w Xw Π

6 S.-Y. Prk, M. Suro: A multiview 3D modeling system sed on stereo vision techniques 153 Z s c 2 t i θ c 1 r X s Fig. 7. The rottion center is one of the intersections of two circles c 1 nd c 2 Fig. 8. Checkerord ptterns for turntle clirtion ()0 ;()45 t [0.0, 0.0, 0.0] T nd its dimeter is t i. Similrly, the center of c 2 is t [ x, 0.0, z ] T nd its dimeter is lso t i. Let r = t i nd = ; then we derive two circles equtions x 2 + z 2 = r 2, (22) (x x ) 2 +(z z ) 2 = r 2. Using the equtions we get z = 2 x + 2 z 2 x x. (23) 2 z From Eq. 22 we lso get r 2 = x 2 + (2 2 x x) 2 4 2, z 0=4 2 x 2 4 x 2 x +( 4 4r 2 2 z), (24) where r = /2 sin(θ/2). Therefore, the x coordinte of the two intersection points is the solution to the second-order inomil eqution s in Eq. 24. And the z coordinte is computed y Eq. 23. Given two intersection points, only one of them is the rel intersection point. If the intersection point is computed s t i =[x, 0.0,z] T on the Π plne, it should hve property such tht = t i, nd y > 0, ecuse we rotte point p 0 y positive ngle θ long the Y s xis of the turntle coordintes. Let us now derive the trnsformtion mtrix from the TCS to the CCS. Becuse we shift the origin O s to point p s0 to find point t i, the trnsltion from O w to O s ecomes (t i + p s0 ) with respect to the TCS nd R ws (t i + p s0 ) with respect to the WCS. Finlly, the trnsformtion from the turntle to the CCS is computed s T cs = T ws T cw where T ws =[R ws t ws ]=[R ws R ws (t i + p s0 )]. (25) To reduce the noise effect on computing t i, we verge the results of the vectors for some world points. 3.2 Turntle clirtion experiments To estimte the TCS, we use checkerord clirtion pttern s shown in Fig. 8. We plce the pttern on the turntle in such wy tht the xy plne of the pttern fces the cmer, leving the rottion xis ehind. Using the stereo cmer, we tke two pirs of stereo pictures t 0 nd t θ. Then we detect ll 48 corner points in ech picture. Using conjugte points in pir of stereo pictures, we compute the 3D position of the corner points with respect to the RCCS. Computing the trnsformtion from the WCS to the CCS is done s follows. As shown in Fig. 8, the trnsltion t cw is the vector from the cmer to the upper-left corner point p ul. The three xes of the WCS with respect to the cmer system re computed s ˆr wx = p ur p ul / p ur p ul, (26) ˆr wz = p ll p ul / p ll p ul, (27) ˆr wz = ˆr wz ˆr wz, (28) ˆr T T wx nd R cw =. (29) ˆr T wy ˆr T wz An exmple of the trnsformtion mtrix from the TCS to the CCS is computed s T cs = (30) We test our clirtion lgorithm t severl positions of the stereo cmer. Tle 2 shows the registrtion error etween two 3D control point sets, t 0 nd t 45, on the checkerord pttern. The trnsltion vector t cs shows the distnce from the CCS to the TCS. 4 3D model genertion nd error nlysis 4.1 Multiview registrtion nd integrtion Using the rnge imge cquisition nd clirtion techniques presented in erlier sections, we reconstruct 3D models of severl rel ojects. We otin multiple rnge imges of n oject from eight views of the oject. After otining rnge imges, we ring ll of them to common coordinte system

7 154 S.-Y. Prk, M. Suro: A multiview 3D modeling system sed on stereo vision techniques Tle 2. Registrtion error of turntle clirtion in mm t cs Men error Mx. error x y z using the turntle clirtion prmeters. Registered rnge imges re then refined gin y using the point-to-plne registrtion technique we introduced in [16]. After registrtion, rnge imges re integrted to otin 3D mesh model using volumetric modeling technique [10,14,15]. From multiview rnge imges of n oject we find the implicit surfce of the oject y computing the signed distnce of voxel to the surfce of the oject. The implicit surfce is then converted to 3D mesh model y the Mrching Cues lgorithm [10]. More detils of our multiview modeling techniques cn e found in [14,15] D model results Figure 9 shows 3D models of three rel ojects. The first column shows pictures of the ojects, the second column shows surfce representtions of the reconstructed models, nd the third column shows texture-mpped 3D models. If n oject hs little contrst on its surfce, we use slide projector to introduce rndom dot pttern to enhnce the performnce of stereo mtching. The oject in Fig. 9c is very complex nd difficult to reconstruct. It hs non-lmert surfces nd some concvities. We merge 16 multiview rnge imges in this cse. Texture-mpped 3D models show photorelistic reconstruction of the ojects. Tle 3 shows the processing time to generte 3D models of the ojects. The modeling time ctully depends on the resolution of the 3D grid of voxels in the Mrching Cues lgorithm is set. This tle shows only some exmples, where the voxel size of the ojects is 3 or 4 mm. Totl processing time is out 5 to 8 min depending on the complexity nd size of the ojects nd on the numer of views. In the duck oject, we use the slide projector to tke stereo pictures with rndom dot pttern. After tking multiview stereo pictures with norml illumintion, we tke nother set of stereo pictures gin with the rndom dot pttern. Imge cquisition for this oject tkes twice tht of the Mr. Pottohed oject. 4.3 Reconstruction error nlysis To nlyze the ccurcy of our modeling system, we reconstruct 3D models of two ground truth ojects. Two test ojects re shown in Figs. 10 nd 11. One is rectngulr prllelepiped, nd the other is cylinder. We reconstruct 3D models of them nd mesure dimensions of the models to compre c Fig. 9. Reconstruction results of rel ojects. Left to right: Picture of ojects, surfce models, texture-mpped models. Mr. Pottohed. Duck. c Indin couple Tle 3. 3D model genertion time (s) Oject Mr. Pottohed Duck Indin couple Voxel size (mm) No. of tringles No. of views Imge cquisition Rectifiction Stereo mtching Registrtion Integrtion Totl with those of the ground truth models. We choose these two ojects ecuse their dimensions re esily mesured. We use n ICP-sed registrtion technique to first register point clouds of the reconstructed 3D model to tht of the ground truth model. Then dimensionl errors re mesured etween ll points on the model nd their closest conjugtes on the ground truth. We itertively register the reconstructed 3D model to its ground truth until the registrtion error etween the 3D model nd the ground truth converge to constnt vlue. As n error metric, we mesure the verge distnce of the closest points etween the two models. Figure 12 shows the results of registering two ojects. We use 392 control points in the cues oject nd 104 points in cylinder. After the two 3D models the reconstructed model nd the ground truth model re registered, we mesure dimensionl errors on the reconstructed model with respect to the ground truth model. For the cues oject, the RMS errors

8 S.-Y. Prk, M. Suro: A multiview 3D modeling system sed on stereo vision techniques 155 W = 60 D = 60 H = 90 c Fig. 10. Cues oject for error nlysis. () Picture. () Dimension (mm). c Point clouds model R = mm Tle 4. RMS nd mximum errors of cues Dimension W (mm) D (mm) H (mm) V (mm 3 ) Size RMS error MAX error (%) error (RMS) Tle 5. RMS nd mximum errors of cylinder Dimension H (mm) R (mm) V (mm 3 ) Size RMS error MAX error (%) error (RMS) H = mm c Fig. 11. Cylinder oject for error nlysis. Picture. Dimension (mm). c Point clouds model RMS registrtion error (mm) RMS registrtion error (mm) Itertion Cues Cylinder Itertion Fig. 12. RMS registrtion error etween the ground truth nd the reconstructed 3D models. Cues. Cylinder in the W, H, nd D dimensions re mesured for ll vertices on corresponding plnes for exmple the top nd the ottom plnes for the H dimension with respect to the closest vertices on the ground truth. Similrly for the cylinder oject, errors in the R nd H dimensions re mesured using points on side surfces, nd top nd ottom surfces, respectively. Tle 4 shows RMS nd mximum errors in ll dimensions of the cues oject. We lso mesure the volume V of the reconstructed model using volume-mesuring technique descried in reference pper [11]. Tle 5 shows the results of the Cylinder oject. 5 Conclusions We hve introduced stereo vision system to utomticlly generte 3D computer models of rel ojects. The system consists of n inexpensive stereo cmer, turntle, nd personl computer. Clirtion of the stereo cmer nd the turntle stge is presented. We rectify stereo imges nd otin rnge imges of n oject from multiple viewpoints. Those rnge imges re then utomticlly registered to common coordinte system nd integrted into 3D mesh model. We hve introduced new turntle coo rdinte system nd simple nd ccurte clirtion technique. Reconstruction error nlysis shows the ccurcy of our 3D reconstruction. References 1. Allen P, Yng R (1998) Registering, integrting, nd uilding CAD models from rnge dt. In: IEEE interntionl conference on rootics nd utomtion, pp Burt P (1983) The Lplcin pyrmid s compct imge code. IEEE Trns Commun 31(4): Chen Q, Medioni G (1999) A volumetric stereo mtching method: pplictions to imge-sed modeling. In: Proceedings of the conference on computer vision nd pttern recognition 4. Curless B, Levoy M (1996) A volumetric method for uilding complex models from rnge imges. In: Proceedings of SIG- GRAPH, pp Fitzgion A, Cross G, Zissermn A (1998) Automtic 3D model construction for turn-tle sequences. In: Europen workshop SMILE 98. Lecture notes in computer science, vol Springer, Berlin Heidelerg New York, pp

9 156 S.-Y. Prk, M. Suro: A multiview 3D modeling system sed on stereo vision techniques 6. Fussiello A, Trucco E, Verri A (2000) A compct lgorithm for rectifiction of stereo pirs. Mch Vis Appl 12: Huer D (2001) Automtic 3D modeling using rnge imges otined from unknown viewpoints. In: Proceedings of the 3rd interntionl conference on 3D digitl imging nd modeling. IEEE Press, New York pp Lnder P (1998) A multi-cmer method for 3D digitiztion of dynmic, rel-world events. PhD disserttion, Crnegie Mellon University, Pittsurgh, PA 9. Levoy M et l (2000) The digitl Michelngelo project: 3D scnning of lrge sttues. In: Proceedings of SIGGRAPH, pp Lorensen W, Cline H (1987) Mrching Cues: high resolution 3D surfce construction lgorithm. Comput Grph 21(4): Mirtich B (1996) Fst nd ccurte computtion of polyhedrl mss properties. J Grph Tools 1(2): Niem W (1999) Automtic reconstruction of 3D ojects using moile cmer. Imge Vis Comput 17: Okutomi M, Knde T (1993) A multiple-seline stereo. IEEE Trns Pttern Anl Mch Intell 15(4): Prk S, Suro M (2002) Automtic 3D model reconstruction using voxel coding nd pose integrtion. In: Proceedings of the interntionl conference on imge processing, pp Prk S, Suro M (2003) Automtic 3D reconstruction sed on novel pose estimtion nd integrtion techniques. Imge Vis Comput 22(8): Prk S, Suro M (2003) An ccurte nd fst point-to-plne registrtion technique. Pttern Recog Lett 24 (16): Pulli K, Ai-Rched H, Duchmp T, Shpiro L, Stuetzle W (1998) Acquisition nd visuliztion of colored 3D ojects. In: IEEE interntionl conference on pttern recognition, pp Rnder P, Nrynn P, Knde T (1996) Recovery of dynmic scene structure from multiple imge sequences. In: Proceedings of the interntionl conference on multisensor fusion nd integrtion for intelligence systems, pp Rusinkiewicz S, Hll-Holt O, Levoy M (2002) Rel-time 3D model cquisition. In: Proceedings of SIGGRAPH Schrstein D, Szeliski R (2002) A txonomy nd evlution of dense two-frme stereo correspondence lgorithms. Int J Comput Vis 47(1 3): Seitz S, Dyer CR (1997) Photorelistic scene reconstruction y voxel coloring. In: Proceedings of the IEEE conference on computer vision nd pttern recognition, pp Vedul S, Rnder P, Sito H, Knde T (1998) Modeling, comining, nd rendering dynmic rel-world events from imge sequences. In: Proceedings of the 4th conference virtul systems nd multimedi, 1: Wheeler M, Sto Y, Ikenuchi K (1998) Consensus surfces for modeling 3D ojects from multiple rnge imges. In: IEEE conference on computer vision, pp Zhng L, Curless B, Seitz S (2002) Rpid shpe cquisition using color structured light nd multi-pss dynmic progrmming. In: 1st interntionl symposium on 3D dt processing, visuliztion, nd trnsmission, pp Soon-Yong Prk otined his B.S. nd M.S. in Electronics Engineering from Kyungpook Ntionl University, Tegu, Kore, in 1991 nd 1993, respectively. He otined his Ph.D in Electricl nd Computer Engineering from Stte University of New York (SUNY) t Stony Brook in From 1993 to 1999, he ws senior reserch stff in the Advnced Rootics L. t Kore Atomic Energy Reserch Institute. He is currently Postdoctorl Reserch Associte in the Deprtment of Electricl nd Computer Engineering t SUNY t Stony Brook. His reserch interests include 3D Sensing nd Modeling, 3D Pose Estimtion, nd Computer Grphics. Murli Suro otined B. Tech. in Electricl Engineering from the Indin Institute of Technology, Mdrs, nd M.S. nd Ph.D. in Computer Science from the University of Mrylnd, College Prk. He joined the fculty of the Deprtment of Electricl nd Computer Engineering, SUNY t Stony Brook, soon fter his Ph.D. He is the Founder nd Director of the Computer Vision Lortory in the deprtment. His reserch nd teching res include Computer Vision, Digitl Imge Processing, Softwre Engineering, Digitl Systems Design, nd We nd Internet Technology.