A Range Image Refinement Technique for Multi-view 3D Model Reconstruction

Similar documents
An efficient method to build panoramic image mosaics

New dynamic zoom calibration technique for a stereo-vision based multi-view 3D modeling system

Improvement of Spatial Resolution Using BlockMatching Based Motion Estimation and Frame. Integration

Range images. Range image registration. Examples of sampling patterns. Range images and range surfaces

Scan Conversion & Shading

Scan Conversion & Shading

A Fast Visual Tracking Algorithm Based on Circle Pixels Matching

Content Based Image Retrieval Using 2-D Discrete Wavelet with Texture Feature with Different Classifiers

A Binarization Algorithm specialized on Document Images and Photos

Computer Animation and Visualisation. Lecture 4. Rigging / Skinning

Structure from Motion

Subspace clustering. Clustering. Fundamental to all clustering techniques is the choice of distance measure between data points;

MULTISPECTRAL IMAGES CLASSIFICATION BASED ON KLT AND ATR AUTOMATIC TARGET RECOGNITION

An Image Fusion Approach Based on Segmentation Region

Parallelism for Nested Loops with Non-uniform and Flow Dependences

3D vector computer graphics

Computer Vision I. Xbox Kinnect: Rectification. The Fundamental matrix. Stereo III. CSE252A Lecture 16. Example: forward motion

A Probabilistic Method for Aligning and Merging Range Images with Anisotropic Error Distribution

Detection of an Object by using Principal Component Analysis

MOTION BLUR ESTIMATION AT CORNERS

PROJECTIVE RECONSTRUCTION OF BUILDING SHAPE FROM SILHOUETTE IMAGES ACQUIRED FROM UNCALIBRATED CAMERAS

R s s f. m y s. SPH3UW Unit 7.3 Spherical Concave Mirrors Page 1 of 12. Notes

A high precision collaborative vision measurement of gear chamfering profile

Articulated Motion Capture from Visual Hulls in High Dimensional Configuration Spaces

A Volumetric Approach for Interactive 3D Modeling

y and the total sum of

SLAM Summer School 2006 Practical 2: SLAM using Monocular Vision

High-Boost Mesh Filtering for 3-D Shape Enhancement

Multi-Resolution Geometric Fusion

A Comparison and Evaluation of Three Different Pose Estimation Algorithms In Detecting Low Texture Manufactured Objects

A Fast Content-Based Multimedia Retrieval Technique Using Compressed Data

3D Face Reconstruction With Local Feature Refinement. Abstract

Inverse-Polar Ray Projection for Recovering Projective Transformations

Vanishing Hull. Jinhui Hu, Suya You, Ulrich Neumann University of Southern California {jinhuihu,suyay,

Image Fusion With a Dental Panoramic X-ray Image and Face Image Acquired With a KINECT

Real-time Motion Capture System Using One Video Camera Based on Color and Edge Distribution

Face Recognition University at Buffalo CSE666 Lecture Slides Resources:

CS 534: Computer Vision Model Fitting

TN348: Openlab Module - Colocalization

Corner-Based Image Alignment using Pyramid Structure with Gradient Vector Similarity

Feature-based image registration using the shape context

3D Face Reconstruction With Local Feature Refinement

Skew Angle Estimation and Correction of Hand Written, Textual and Large areas of Non-Textual Document Images: A Novel Approach

Distance Calculation from Single Optical Image

Angle-Independent 3D Reconstruction. Ji Zhang Mireille Boutin Daniel Aliaga

3D Face Recognition Fusing Spherical Depth Map and Spherical Texture Map

Accessibility Analysis for the Automatic Contact and Non-contact Inspection on Coordinate Measuring Machines

GPU-based Parallel Construction of Compact Visual Hull Meshes

3D Modeling Using Multi-View Images. Jinjin Li. A Thesis Presented in Partial Fulfillment of the Requirements for the Degree Master of Science

Shape Representation Robust to the Sketching Order Using Distance Map and Direction Histogram

ScienceDirect. The Influence of Subpixel Corner Detection to Determine the Camera Displacement

Reducing Frame Rate for Object Tracking

Some Tutorial about the Project. Computer Graphics

Semarang, Indonesia. Sepuluh Nopember Institute of Technology, Surabaya, Indonesia

Multi-view 3D Position Estimation of Sports Players

MOTION PANORAMA CONSTRUCTION FROM STREAMING VIDEO FOR POWER- CONSTRAINED MOBILE MULTIMEDIA ENVIRONMENTS XUNYU PAN

Cluster Analysis of Electrical Behavior

Multi-stable Perception. Necker Cube

Feature Reduction and Selection

A Novel Accurate Algorithm to Ellipse Fitting for Iris Boundary Using Most Iris Edges. Mohammad Reza Mohammadi 1, Abolghasem Raie 2

Tsinghua University at TAC 2009: Summarizing Multi-documents by Information Distance

Positive Semi-definite Programming Localization in Wireless Sensor Networks

A Super-resolution Algorithm Based on SURF and POCS for 3D Bionics PTZ

A mathematical programming approach to the analysis, design and scheduling of offshore oilfields

Simplification of 3D Meshes

THE RENDERING OF BUILDING TEXTURE FROM LAND-BASED VIDEO

Accounting for the Use of Different Length Scale Factors in x, y and z Directions

METRIC ALIGNMENT OF LASER RANGE SCANS AND CALIBRATED IMAGES USING LINEAR STRUCTURES

Dynamic wetting property investigation of AFM tips in micro/nanoscale

Computer Generated Integral Imaging (II) System Using Depth-Camera

The Comparison of Calibration Method of Binocular Stereo Vision System Ke Zhang a *, Zhao Gao b

3D Rigid Facial Motion Estimation from Disparity Maps

PHOTOGRAMMETRIC ANALYSIS OF ASYNCHRONOUSLY ACQUIRED IMAGE SEQUENCES

Mathematics 256 a course in differential equations for engineering students

Vol. 5, No. 3 March 2014 ISSN Journal of Emerging Trends in Computing and Information Sciences CIS Journal. All rights reserved.

The Research of Ellipse Parameter Fitting Algorithm of Ultrasonic Imaging Logging in the Casing Hole

APPLICATION OF A COMPUTATIONALLY EFFICIENT GEOSTATISTICAL APPROACH TO CHARACTERIZING VARIABLY SPACED WATER-TABLE DATA

Lecture #15 Lecture Notes

Environmental Mapping by Trinocular Vision for Self-Localization Using Monocular Vision

Shape-adaptive DCT and Its Application in Region-based Image Coding

Invariant Shape Object Recognition Using B-Spline, Cardinal Spline, and Genetic Algorithm

A Gradient Difference based Technique for Video Text Detection

Simultaneous Object Pose and Velocity Computation Using a Single View from a Rolling Shutter Camera

A NEW APPROACH FOR SUBWAY TUNNEL DEFORMATION MONITORING: HIGH-RESOLUTION TERRESTRIAL LASER SCANNING

Type-2 Fuzzy Non-uniform Rational B-spline Model with Type-2 Fuzzy Data

UAV global pose estimation by matching forward-looking aerial images with satellite images

An Image Compression Algorithm based on Wavelet Transform and LZW

A Gradient Difference based Technique for Video Text Detection

Snakes-based approach for extraction of building roof contours from digital aerial images

Implementation of a Dynamic Image-Based Rendering System

Outline. Discriminative classifiers for image recognition. Where in the World? A nearest neighbor recognition example 4/14/2011. CS 376 Lecture 22 1

Adaptive Selection of Rendering Primitives for Point Clouds of Large Scale Environments

Model reconstruction and pose acquisition using extended Lowe s method

Radial Basis Functions

Recognizing Faces. Outline

Panorama Mosaic Optimization for Mobile Camera Systems

DIRECT SENSOR-ORIENTED CALIBRATION OF THE PROJECTOR IN CODED STRUCTURED LIGHT SYSTEM

Fusion of Data from Head-Mounted and Fixed Sensors 1

Analysis of Continuous Beams in General

Model-Based Bundle Adjustment to Face Modeling

Transcription:

A Range Image Refnement Technque for Mult-vew 3D Model Reconstructon Soon-Yong Park and Mural Subbarao Electrcal and Computer Engneerng State Unversty of New York at Stony Brook, USA E-mal: parksy@ece.sunysb.edu Abstract Ths paper presents a range mage refnement technque for generatng accurate 3D computer models of real objects. Range mages obtaned from a stereo-vson system typcally experence geometrc dstortons on reconstructed 3D surfaces due to the nherent stereo matchng problems such as occlusons or msmatchngs. Ths paper ntroduces a range mage refnement technque to correct such erroneous ranges by employng eppolar geometry of a multvew modelng system and the vsual hull of an object. After regsterng multple range mages nto a common coordnate system, we frst determne f a 3D pont n a range mage s erroneous, by measurng regstraton of the pont wth ts correspondences n other range mages. The correspondences are determned on 3D contours whch are nverseprojectons of eppolar lnes n other 2D slhouette mages. Then the range of the pont s refned onto the object s surface, f t s erroneous. We employ two technques to search the correspondences fast. In case that there s no correspondence for an erroneous pont, we refne the pont onto the vsual hull of the object. We show that refned range mages yeld better geometrc structures n reconstructed 3D models. 1. Introducton Generatng 3D computer models of real objects s of much nterest n Computer Vson and Computer Graphcs. One of the recent research nterests s generatng a complete and closed 3D model by mergng multple mages of an object. One common technque s employng a sngle or multple rangng sensors to acqure and merge mult-vew range mages [1, 4]. There s a varety of rangng technques to obtan range mages of a scene of nterest. In order to generate accurate 3D models, most nvestgatons employ actve rangng technques such as laser rangng, structured lght pattern, space-tme coded pattern, etc. In contrast, passve technques work only on naturally formed mages produced by reflected lght from the object. A common technque s Stereo Image Analyss (SIA). However, due to nherent stereo problems, relatvely fewer number of researchers have employed stereoscopc magng sensors to generate complete 3D models [3, 5, 14]. Range mages obtaned from a stereo rangng sensor typcally experence erroneous ponts on reconstructed 3D partal surfaces. In order to obtan accurate range mages usng SIA, some researchers project random dot patterns onto object s surfaces to enhance contrast on them [9]. Others employ mult-baselne or mult-resoluton [14] technque to reduce the number of stereo msmatchngs. Even usng these technques, obtanng accurate range mages s stll dffcult n some portons of the object s surfaces. One smple approach of removng erroneous ponts s employng a lnear low-pass flter or a non-lnear flter [4, 5, 12]. However, t s not easy for these approaches to remove some erroneous regons where the errors are domnant wthn a flterng mask. Another smple approach s obtanng a large number of range mages to average out the errors [4, 10, 13]. However, acqurng multple range mages usng SIA s a computatonally expensve task. Another approach s employng a range mage regstraton technque, such as Iteratve Closest Pont (ICP), to reduce noses n range mages [11]. Some researchers try to remove these errors usng the vsual hull of the object [7, 8]. However, ths technque can remove only those outsde the vsual hull but not nsde. In ths paper, we present a range mage refnement technque to reconstruct accurate 3D models of real objects. After regstraton of mult-vew range mages, we frst determne f a 3D pont n a range mage s erroneous, by measurng the regstraton of the pont wth ts conjugates n other range mages. The conjugate ponts are searched on 3D contours whch are nverse-projectons of eppolar lnes n other 2D slhouette mages. When the 3D pont s decded as erroneous or t s outsde the vsual hull of the object, we refne the range of the pont to the mean of the conjugates or

onto the surface of the vsual hull. In order to make searchng and refnement fast, we employ two technques, pontto-projecton search and orderng constrant. Expermental results on a ground truth object and real objects show that 3D models from refned range mages yeld better geometrc structures. 2. Range Error due to Stereo Msmatchng Range mages obtaned by a stereo camera could experence some erroneous ponts on ther surfaces due to stereo msmatchng. An example of erroneous range surface s shown n Fgure 1. Fgure 1 shows a pcture of an object and Fgure 1 shows a reconstructed mesh model of the front partal surface of the object, whch s seen from the top. As n the fgure, stereo msmatchng errors appear as sharp peaks on the partal shape. Shape dstortons due to stereo matchng errors Fgure 2. Error dstrbuton n 3D space. Top: A range mage wth erroneous surfaces. Bottom: The same mage wthout error. Most errors are dstrbuted along the Z c axs of the camera coordnate system 3. Refnement Methodology 3.1. Erroneous 3D ponts Z X Y Camera coordnate system Fgure 1. An example of stereo msmatchng An object A partal shape of the object shows some erroneous surfaces When we compute the depth of a 3D pont P, whch maps to an mage pont p, the 3D pont s on the ray of p. Suppose an object s placed consderably far from the camera, then the ray s almost parallel to the Z c axs (vewng drecton) of a camera coordnate system. Therefore, we can estmate that the coordnates of msmatchng errors, between a correct ponts set {P} and an erroneous ponts set {P }, dstrbute mostly on the Z c axs. An example of error dstrbuton between two partal surfaces n Fgure 2 s plotted n Fgure 2. In Fgure 2, the top range mage has some errors, and the bottom one has neglgble errors n contrast. As shown n Fgure 2, most errors are dstrbuted along the Z c axs, whch concdes wth the vewng vector of the range mage. As presented n the prevous secton, stereo msmatchng errors n a range mage occur mostly on such object s surfaces whose normal vectors are n a hgh angle wth respect to the vewng drecton of the mage. However, they may be seen better from another vewng drecton because the object s scanned from multple drectons. Therefore, t s possble to refne an erroneous 3D pont, f we know ts correspondences on other vew s range mages. Suppose there s a 3D pont P on an object s surface and two dfferent vews V and V j can see the pont. Then we can wrte two representatons of the pont as P and P j n the correspondng range mages R and R j, based on the vew pont of V. When the two ponts are regstered to a common coordnate system, ther coordnates should concde n an deal case. However, n a real case, they may not concde because of systematc or calbraton error of the vson system. If the two ponts do not concde n 3D space, a problem arses that how we can determne the correspondences between multple range mages. We assume that all correspondences occur on the ray of a 3D pont. Consderng R as a reference range mage and P as a reference pont, we fnd all ntersectons n the other range mages on the ray of the reference pont. Suppose there are two range mages R and R j as shown

n Fgure 3 and they are regstered to a common vew V. Assumng regstraton of the range mages s very accurate, we know that P s very close to another pont P j, whch s on the lne of sght of the pont P from V. If the orgn of the vew V s O, the correspondence of P on R j s on the ray = αp,α R. Therefore, we wrte P j = R j (1) and P j =0. (2) Therefore, the two matchng ponts P and P j n Fgure 3 are very close and there s only a small regstraton error ɛ between them such that ɛ = P P j 0. (3) et us consder another 3D pont P as shown n Fgure 3. Ths pont s consdered as an erroneous pont n the range mage R due to stereo msmatchng, whle ts conjugate P j on the R j s a correct surface pont. Then there should be sgnfcant regstraton error between two ponts as n the fgure, whch s larger than a threshold d T such that R j P j ɛ = P P j >d T. (4) R R j d T P j R In addton to the error measure, we also use the vsual hull of an object to decde f a 3D pont s erroneous. When the 3D pont s outsde the vsual hull of the object, we consder t s erroneous. Ths technque s the same as what the shape-from-slhouettes technque does. In summary, let VH(O) be the vsual hull of an object O, then the 3D pont P s also { true, f P erroneous / VH(O) not determned, f P VH(O). (6) We frst check f the pont s outsde the vsual hull. If the pont s outsde, we consder t s erroneous and refne t. In case that the pont s nsde the vsual hull, we use Equaton (5) to check f t s erroneous. 3.2. Correspondence on Eppolar Contour In order to refne a reference 3D pont n a reference range mage, we search all ts correspondences n the other range mages. Suppose agan there s a reference 3D pont P (or P )onr as shown n Fgure 4. The perspectve projecton of the 3D pont to the mage plane of V sa2d mage pont p. et us consder a problem to fnd the correspondences of P n other range mages, R j and R k n ths example. We employ the eppolar geometry of our system. If we know the fundamental matrx between V and V j,we can restrct searchng of ntersectons on the eppolar lne u j whch s u j = F j p, (7) where F j s the fundamental matrx between V and V j. O j P O O j P' O O k p k,e U k Object p k,s u k I k P U j Fgure 3. A typcal error on a partal shape due to stereo msmatchng. Two matchng 3D ponts are very close P s far from P j p p j,e u j p j,s If multple range mages are ntersected by the ray, t s possble to determne several correspondences n the range mages. Suppose there are K range mages R j for j =0,,K 1, whch are ntersected by. Then we determne f the pont P s O Fgure 4. Eppolar geometry of a mult-vew system I j O j { true, f P erroneous P k >d T, k, 0 k K 1 false, f P P k <d T, k, 0 k K 1. On the mage plane I j n the fgure, the eppolar lne u j (5) overlaps wth a slhouette of the object, whch s a bnary

representaton of the perspectve projecton of the object to V j. On the object s slhouette, we determne a startng pont p j,s and an endng pont p j,e on the eppolar lne. From the startng pont to the endng pont, we then nverse project all 2D mage ponts on the eppolar lne u j to correspondng 3D ponts from P j,s to P j,e on the surface of R j. All the 3D ponts on the surface then form a 3D contour {U j } Along the 3D contour {U j }, we search all ntersectons by the ray, whch satsfy Equaton (1) and (2). Then from these canddates we pck one of them as P j,c, the correspondence of the pont P. The ntersecton pont P j,c satsfes followng equatons. {U j } = {P P T j M 1 j {p j,s p j,e }} (8) P j,c P =0 and P j,c {U j }. (9), where M j s the perspectve transform matrx of the jth vew, and T j s the transformaton of coordnate system from the jth vew to the th vew. Because there could be several ntersectons on the 3D contour by the ray, we fnd all of them by checkng sgns of cross products of the vector of and other vectors to ponts on {U j }. More detals are n the next secton. 4. Range Image Refnement 4.1. Correspondence Search In order to search all ntersectons on the 3D contour P j,s P j,e,wefrst fnd all zero crossngs of the cross product P j P, where P j P j,s P j,e. Because the Y s axs of the turntable s coordnate system s almost parallel to the Y c axs of the camera coordnate system, we check sgn of the Y component of the product to search zero crossngs. If the sgn changes, we consder there happens a zero crossng. Fgure 5 shows an example of zero crossngs on a 3D contour. Of course there could be multple (N c ) ntersectons on the contour, for example n the fgure from P j,c,0 to P j,c,nc 1. However, from the pont of th vew pont, only one ntersecton s vsble, whch s the closest to P. In ths fgure therefore, we take P j,c,0 as the correspondence P j,c on jth vew s contour {U j }. To avod false correspondence, we test and remove some ntersectons whose normal vectors are n a hgh angle wth respect to ther orgnal vew ponts, whch are hdden from the reference vew pont, or whch are outsde the vsual hull of the object. O P j,s P P j,c,0 P j,h P j,c,1 U j P j,c,2 Oj P j,e P j,c,nc-1 Fgure 5. Intersectons on a 3D contour by a reference vector 4.2. Fast Searchng Technque In ths secton, we consder computaton tme of ntersecton searchng and ntroduce fast searchng technques. Suppose there are Q ponts on a 3D contour and computaton for ntersecton searchng takes O(Q) operatons. If there are K overlappng ranges from a reference pont, we need to search ntersectons on K contours, thus O(KQ) computatons. In order to refne a range mage, whose object area s equvalent to M X M Y mage sze, then we need to compute O(KQM X M Y ) operatons. To refne all N range mages, we fnally need O(NKQM X M Y ) operatons. These computatons are computatonally very expensve. In a typcal Pentum-4 1.8GHz computer, t takes more than 30 mnutes to refne 16 range mages wth 320 240 resoluton. In order to make the computaton tme fast, we employ two technques based on pont-to-projecton range searchng and orderng constrant. 4.2.1 Pont-to-Projecton Searchng Instead of searchng ntersectons on a 3D contour, we use a pont-to-projecton technque to solve the correspondence problem as shown n Fgure 6. To fnd the jthe vew s correspondence from a reference pont P, we project t to the mage plane of V j to a 2D pont p j, nversely project the 2D pont to jth range mage R j to get P j. In other words, P j = T j R j(m j T j P ), (10) where R j (p) s the jth vew s range of a 2D pont p. By searchng all nverse-projecton ponts P k, k = 0 K 1 n other range mages, we use Equaton (5) to determne f the reference pont s erroneous or not. Inverseprojecton s not exactly an ntersecton pont by the ray. However, f two range surfaces are very closely regstered, we can approxmate the pont as an ntersecton. Once the

R j Object P P j R P U j O j O Fgure 6. Pont-to-Projecton searchng I p (x,y ) p (x +1,y ) ; p' p j,s p j,s p j (x j,y j ) j,e p' j,e u j I j O reference pont s determned as erroneous, we then fnd the real ntersectons on the ray of usng the technque presented n Secton 3.2. Usng ths searchng technque, we refne all range mages n about 5 or 10 mnutes dependng on objects. 4.2.2 Orderng Constrant The next fast searchng technque s usng orderng constrant n a par of mult-vew mages. Once a reference pont P s determned as erroneous, we fnd ts conjugates on a 3D contour {U j }, whch s the nverse projecton of a 2D segment on u j. In Secton 3.2, we search the conjugates along u j startng from p j,s to p j,e. However, nstead of all ponts on the eppolar lne we use the orderng constrant between a par of mult-vew mages. et us consder an example n Fgure 7. As n the fgure, suppose a 2D reference pxel p (x,y ) on I matches wth ts conjugate p j (x j,y j ) on I j, where (x,y ) and (x j,y j ) are ther coordnates on I and I j respectvely. In other words, ther 3D conjugates P and P j match each other. Now consder another reference pont p (x +1,y ) on I, whch s the next pxel of p (x,y ). Then we know that ts matchng pxel on I j s always on the rght-sde of p j (x j,y j ) accordng to the orderng constrant. Therefore we search an ntersecton of the ray on a 3D contour along {U j }, whch s the nverse projecton of a lne segment p j,s p j,e rather than p j,sp j,e. Based on the constrant, the x coordnate of the startng pont could be the same wth that of p j (x j,y j ), that s x j. However, n order to fnd a zero crossng on the contour, we gve a margn n the left drecton. As a result, we set the range of ntersecton searchng along the x axs from x j δ to x j + δ +.By assumng there s no sudden depth change on the object s surface, we set δ to 2 and δ + to 10 pxels. 4.3. Refnement onto Vsual Hull Some erroneous range ponts are outsde the vsual hull (VH(O)) of an object. If a 3D pont s outsde of VH(O), Fgure 7. Reducng search range usng orderng constrant we have to refne t to a correct 3D pont on the object s surface. If there are enough matchng ponts n other range mages, we can refne t usng the presented technques. However, a problem arses f there s not enough or no matchng pont. In ths case, we decde to refne t onto the vsual hull of the object. As shown n Fgure 5, a vsual hull pont P j,h can be found by the ntersecton of two rays, and P j,s O j. Intersecton of two non-ntersectng rays can be solved usng a smple lnear equaton. Because there are multple vews, we acqure multple vsual hull ponts P k,h, 0 k K 1. Then the problem s now decdng whch one s on VH(O) of the object. et us consder an example n Fgure 8. On the ray from a vew orgn O, we acqure multple vsual hull ponts from P 0,h to P 3,h. To determne whch one s on VH(O), we smply project every pont to mage planes of all multple vews and check f t s on VH(O). In ths fgure, P 3,h s on the vsual hull. Then we move P to P 3,h. On a lne segment p j,s p j,e on an eppolar lne u j n Fgure 4, we get two vsual hull ponts usng two rays P j,s O j and P j,e O j. However, we take only one of them from P j,s O j, because the pont P can see only the startng pont P j,s. After acqurng all vsual hull ponts from K overlappng vews, we sort them accordng to dstance from the reference pont and check f they are nsde VH(O). 4.4. Refnement Steps Range mage refnement steps are as follows. For every 3D pont n a range mage, we frst determne f t s erroneous or not usng two methods. One s to determne f t s outsde of the vsual hull and the other s to determne f t s erroneous usng the pont-to-projecton searchng technque presented n Secton 4.2.1. If the pont s erroneous, we O j

Vsual Hull to R + N (K = N/2). We set the error threshold d T to 3 4 mm and the varance threshold σt 2 to 2 mm. O 0 Object 5.2. Error Analyss O 1 Cones of multple vews O 2 P 1,h O 3 Fgure 8. Refnement of P onto a vsual hull pont P 3,h search all ntersectons on surfaces of other range mages. If there are at least two ntersectons P j,c, 0 j<k 1, we change the coordnates of the pont to the mean P m of the all ntersectons f σp 2 <σt 2. Here P m and σp 2 are mean and varance of the ponts j P j,c and σ2 T s a threshold for the varance. When there are not enough ntersectons or σ p 2 >σ2 T, but f the reference pont s outsde vsual hull, we change the coordnates of P to P j,h as presented n Secton 4.3. In ths case P j,h should be also nsde the vsual hull of the object. 5. Expermental Results 5.1. Range Image Acquston P 0,h P O P 3,h We generate 3D models of a ground truth object to analyze and compare reconstructon errors of before and after refnement. Fgure 9 shows a pcture of a cylndrcal object for error analyss. Fgure 9 s the dmenson of the object. We acqure range mages of the object from 8 or 16 vew ponts and regster and ntegrate them to a 3D mesh model. Then we measure dmensonal and regstraton errors between the 3D model and a ground truth model. To generate the ground truth model, we reconstruct a 3D model of dense pont clouds. We synthesze the object s surface for every 1 mm 2 to produce about 50000 ponts to represent ts outer surface. R = 37.59 mm H = 105.7 mm Fgure 9. Cylnder object for error analyss Pcture of the object Dmenson of the object To generate mult-vew range mages, we use a stereobased range mage acquston system, whch conssts of a stereo camera, a turntable stage, and a personal computer. A range mage s obtaned from a par of stereo mages. In order to fnd correspondence n the mage par, we use a multresoluton stereo matchng technque [2, 8]. The stereo par s resampled to form a Gaussan mage pyramd and a smple SSD-based (Sum of Squared Dfference) correlaton technque s performed at each pyramd level. Mult-vew range mages are obtaned by rotatng the turntable wth a fxed rotaton angle accordng to number of vews. We acqure range mages of an object from 8 or 16 (N =8, 16) dfferent vew drectons. We use a Pentum-4 1.8GHz personal computer for our vson system. A range mage resoluton for each vew s 320 240 n row and column drectons. For every range mage R,werefne t pxel by pxel (or pont-by-pont). In order to refne an erroneous pont, we search ts correspondences n range mages from R N 4 The two 3D models, the reconstructed model and the ground truth model, are regstered nto a common coordnate system usng ICP regstraton technque. After regstraton, we measure mean and varance of the radus of the reconstructed model. Table 1 shows results of mean and varance of the radus of the object. We compare errors of two 3D models whch are generated by mergng orgnal range mages and refned range mages. We show the errors when the models are merged from 8 vews and 16 vews. To show the results wth dfferent sze of voxel, we reconstruct dfferent 3D models by changng voxel sze of 2 mm, 2.5 mm, and 3 mm. We also measure RMS and maxmum errors of closest dstance between a 3D pont on the reconstructed model and the ground truth model. Table 2 shows results of these errors. As shown n both tables, 3D models generated from refned range mages have better geometrc structures than those from orgnals.

Table 1. Mean (m) and varance (v) error of the radus of Cylnder object n mm Orgnal Range Refned Range N voxel 2.0 2.5 3.0 2.0 2.5 3.0 8 m 36.32 36.32 36.33 36.35 36.35 36.36 v 0.416 0.438 0.40 0.316 0.307 0.313 16 m 36.40 36.39 36.41 36.41 36.41 36.41 v 0.280 0.296 0.317 0.232 0.253 0.252 Comparson of two 3D models s shown n Fgure 12 and. In ths fgure, the sze of a voxel grd s 2.5 mm. The shaded surface models show the back surface of the object. The refned 3D model n Fgure 12 shows better surface structure than the orgnal n Fgure 12. Fgure 13 shows reconstructon results of another object Soccerball. Fgure 13 s the 3D model from orgnal range mages, and Fgure 13(c) s from refned mages. The fgure also shows that refned range mages yeld better surface structure. Table 2. RMS and MAX error of the radus of Cylnder object n mm Orgnal Range Refned Range N voxel 2.0 2.5 3.0 2.0 2.5 3.0 8 rms 1.33 1.33 1.32 1.27 1.27 1.26 max 3.56 3.60 3.37 3.58 3.14 3.32 16 rms 1.21 1.22 1.21 1.19 1.20 1.20 max 2.63 2.54 2.60 2.50 2.25 2.79 500 500 480 480 5.3. Real Objects The proposed technque s also tested on real objects. Fgure 10 shows an example of a sngle lne refnement of Potatohead object shown n Fgure 1. Fgure 10 shows the orgnal front vew (V 0 ) of the object and a 3D contour whch corresponds to a sngle horzontal lne on ts 2D plane. In Fgure 10, we plot all correspondng profles from V N N to V 4 + N on the XZ plane of the camera 4 coordnates. On the plot of V 0, there are some erroneous ponts as shown the contour n Fgure 10. A refned range mage and all profles for the same horzontal scan lnes are shown n Fgure 10 and (d). Some of orgnal and refned range mages of the object are shown n Fgure 11 and (c). Fgure 11 shows there are some artfacts n range mages due to stereo msmatchng. Some brght and dark portons on the object area represent shape dstortons on surface. The refned range mages n Fgure 11(c) show that ther surface structures are refned compared to ther orgnals. Processng tme of range refnement and number of range pxels on all 16 mages are shown n Table 3. Table 3. Processng tme of refnng range mages Object Potatohead Soccerball Number of vertces 264608 156662 Refnement tme (sec) 510.0 122.0 Z axs 460 440 vew12 vew13 420 vew14 vew15 400 vew0 vew1 380 vew2 vew3 vew4 360 0 10 20 30 40 50 60 70 80 90 X axs (c) Z axs 460 440 vew12 vew13 420 vew14 vew15 400 vew0 vew1 380 vew2 vew3 vew4 360 0 10 20 30 40 50 60 70 80 90 X axs Fgure 10. An example of range refnement. A horzontal scan lne s plotted on a orgnal partal shape. The same lne s plotted on a refned shape (c) Profles of all overlappng range contours before refnement (d) Profles of all overlappng range contours after refnement 6. Conclusons We present a range mage refnement technque to generate accurate 3D computer models. The proposed technque employs the eppolar geometry of our vson system and the vsual hull of an object to refne erroneous 3D ponts. After mult-vew range mages are regstered to a common coordnate system, we frst determne f a reference 3D pont n a range mage s erroneous by measurng regstraton error of the pont to ts correspondences n the other overlappng range mages. The correspondences are ntersectons by the ray of the reference pont. Then the range of an erroneous pont s refned onto the object s surface. Vsual hull of the object s also employed to decde f the pont s erroneous (d)

and to refne t onto the vsual hull. Expermental results show the proposed technque yelds better geometrc structures on reconstructed 3D computer models. References Fgure 11. Refnement results of Potatohead object. Some orgnal range mages Refned range mages Fgure 12. Reconstructon results of Potatohead object. A shaded surface on the orgnal 3D model The same surface on the refned 3D model (c) Fgure 13. Reconstructon results of Soccerball object. Pcture of the object A shaded surface on the orgnal 3D model (c) The same surface on the refned 3D model [1] F. Bernadn, I. M. Martn, and H. Rushmeer, Hgh-qualty Texture Reconstructon from Multple Scans, IEEE Transactons on Vsualzaton and Computer Graphcs, 7(4):318-332, 2001. [2] P.J. Burt, The aplacan Pyramd as a Compact Image Code, IEEE Transactons on Communcatons, 31(4):532 540,1983. [3] Q. Chen and G Medon, A Volumetrc Stereo Matchng Method: Applcaton to Image-Based Modelng, Proceedngs of Computer Vson and Pattern Recognton, 29 34, 1999. [4] B. Curless and M. evoy, A Volumetrc Method for Buldng Complex Models from Range Images, Proceedngs of SIG- GRAPH, 303 312, 1996. [5] R. Koch, Surface Segmentaton and Modelng of 3-D Polygonal Objects from Stereoscopc Image Pars, Proceedngs of Internatonal Conference on Pattern Recognton, Venna, 1996. [6] W. Matusk, C. Buehler, R. Raskar,. McMllan, and S. J. Gortler, Image-Based Vsual Hulls, SIGGRAPH, 2000. [7] W. Nem and R. Buschmann, Automatc Modellng of 3D Natural Objects from Multple Vews, In European Workshop on Combned Real and Synthetc Image Processng for Broadcast and Vdeo Producton, 23 24, 1994. [8] S. Y. Park and M. Subbaro, Automatc 3D Model Reconstructon usng Voxel Codng and Pose Integraton, Proceedngs of Internatonal Conference on Image Processng, 533 536, 2002. [9] S. Radm, Accurate Natural Surface Reconstructon from Polynocular Stereo, Proceedngs of NATO Advanced Research Workshop, 69 86, 2002. [10] S. Rusnkewcz, O. Hall-Holt, and M. evoy, Real-Tme 3D Model Acquston, Proceedngs of SIGGRAPH, 2002. [11] R. Sagawa, T. Osh, A. Nakazawa, R. Kurazume, and K. Ikeuch, Iteratve Refnement of Range Images wth Ansotropc Error Dstrbuton, Proceedngs of 2002 IEEE Internatonal Conference on Intellgent Robots and Systems, 79 85, 2002. [12] Y. Sheng and B. Chen, Measurng of a Three-Dmensonal Surface by Use of a Spatal Dstance Computaton, Appled Optcs, 42(11):1958 1972, 2003. [13] M. D. Wheeler, Y. Sato, and K. Ikenuch, Consensus Surfaces for Modelng 3D Objects from Multple Range Images, IEEE Conference on Computer Vson, 917 924, 1998. [14] R. Yang, G. Welch, and G. Bshop, Real-Tme Consensus- Based Scene Reconstructon usng Commodty Graphcs Hardware, Proceedngs of Pacfc Graphcs, 2002.