AUTOMATIC 3D SURFACE RECONSTRUCTION BY COMBINING STEREOVISION WITH THE SLIT-SCANNER APPROACH

Size: px

Start display at page:

Download "AUTOMATIC 3D SURFACE RECONSTRUCTION BY COMBINING STEREOVISION WITH THE SLIT-SCANNER APPROACH"

Herbert Bradford
6 years ago
Views:

1 AUTOMATIC 3D SURFACE RECONSTRUCTION BY COMBINING STEREOVISION WITH THE SLIT-SCANNER APPROACH A. Prokos 1, G. Karras 1, E. Petsa 2 1 Deartment of Surveying, National Technical University of Athens (NTUA), GR Athens, Greece 2 Deartment of Surveying, Technological Educational Institute of Athens (TEI-A), GR Athens, Greece anthro@central.ntua.gr, gkarras@central.ntua.gr, etsa@teiath.gr KEY WORDS: hotogrammetric scanning, surface reconstruction, triangulation, eiolar geometry, camera calibration ABSTRACT: In this aer, a 3D surface scanner is resented. Combining stereovision and slit-scanning, our system is comosed of two cameras and a hand-held laser lane. The camera air is calibrated using synchronized image airs of a coded chessboard; its imaged nodes are automatically identified, referred to the object oints and introduced into a self-calibrating bundle adjustment. For the scanning rocess, stereoscoic rofiles are continuously recorded as the 3D surface is swet by the laser line. After eiolar resamling of the synchronized image airs, search for oint corresondences is thus reduced to identifying intersections of image rows with the recorded laser rofiles. The maxima of Gaussian curves fitted to the gray-value data along the eiolar image rows rovide initial estimates for eak ositions, which are then refined using information from their neighbourhood. In our setu, 3D reconstruction by simle stereovision is strengthened by enforcing extra geometric constraints. First, the colanarity constraint is imosed on all 3D oints reconstructed from a single laser strie, and the coefficients of all laser lanes articiate as unknowns in the 3D reconstruction adjustment. Additionally, this also allows identifying mismatches since eiolar lines may have more than one eaks; the correct 3D oint is established according to a distance threshold from the laser lane. The solution is further reinforced by lacing the object in a corner formed by two background lanes (which are scanned along with the object), whose coefficients are also unknowns in the 3D reconstruction adjustment. The linear laser segments roduced on either side of the object have to satisfy the equation of both the corresonding lane and the laser lane. Image airs of the corner without the object (longer laser segments) are added to the dataset for a more accurate determination of lane equations. Results are resented and evaluated from this setu, whose tyical accuracy is estimated in the order of 0.2 mm in 3D deth estimation. 1. INTRODUCTION Recent years are witness to a growing demand for 3D surface models in several fields (e.g. cultural heritage documentation or industrial metrology). Ideally, the 3D models must be generated raidly and accurately by automatic techniques. As a resonse to this demand, a number of image-based scanners, both commercial and low-cost ones, have been reorted (Forest & Salvi, 2002; Blais, 2004). Of course, stereovision remains a standard aroach. Its main roblem is finding oint corresondences, in articular when dealing with surfaces of low texture. A way to overcome this roblem is relacing the second camera by devices which roject various atterns (e.g. structured light) but can be as simle as a laser lane. Most common among such triangulation-based range-finders are those using laser lanes (i.e. rojection of laser stries), also referred to a slit-scanners. Such systems tyically combine a camera and a rojected laser lane which intersects the object surface to highlight a rofile. The 3D oints of each rofile are found (without redundancy) as intersections of the laser lane and the rojection rays defined by the resective image oints of the rofile. Thanks to its simlicity, several low-cost systems of the slitscanner tye have been reorted. If the laser lane is moved by hand indeendently from the camera, its osition in sace must be calculated for each image. In Zagorchev & Goshtasby (2006) this is achieved through the intersection of the laser lane with a reference double-frame, whereas Winkelbach et al. (2006) use two external orthogonal lanes intersected by the laser lane. A scanner with simler comonents is that of Bouguet & Perona (1998) using one camera and the shadow roduced form a handheld moving rod. In a very interesting imlementation, Kawasaki & Furukawa (2007) use the mere fact that laser lines define colanar object rofiles (imlicit colanarity) to acquire dense 3D data, while also exloiting colanarity information from object lanes in the scene (exlicit colanarity). Projective results are ugraded to Euclidean via suitable constraints, e.g. orthogonal lanes (if necessary, the constraints are sulied by a device roducing two orthogonal laser lanes). Following Prokos et al. (2009), this aer resents a low-cost hotogrammetric range-finder which combines stereovision and the slit-scanner rincile. The two web cameras are automatically calibrated to rovide both their interior orientations and their true to scale relative orientation. A laser lane generator is used to code the scene and, hence, simlify the corresondence roblem to a eak detection question. Our setu is, essentially, similar to that of Davis & Chen (2001), yet the straightforward solution from stereovision is here enhanced by additional geometric constraints. The fact that all oints on a single laser rofile belong to the same lane (laser lane) is exloited to ose a colanarity constraint to these oints. Furthermore, the 3D object is laced in the corner formed by two unknown intersecting lanes; thus, the end-segments of the deicted laser rofiles (intersections s 1, s 2 with the background lanes 1, 2 in Fig. 1) are straight. This is exloited to introduce a further colanarity constraint as each such segment also belongs to the corresonding background lane. The equations of these lanes are estimated in the reconstruction algorithm (in which, along with the image airs for surface scanning, images of the lanes without the object may also articiate). It is to note that this (otional) second colanarity constraint is not always used, since the linear rofile end-segments may not be sufficiently long to rovide reliable data if larger image arts are occuied by the 3D object (this allows higher resolution in object sace). 505

2 2. SYSTEM DESCRIPTION The system consists of a air of web cameras in fixed relative osition and a hand-held laser strie generator. The 3D object is laced close to the intersection of two (unknown) background lanes. Thus, the hardware comonents of the system are: Two 640x480 colour web cameras, fixed in a constant relative osition throughout the rocess. The system is calibrated automatically as exlained below. In a tyical alication, the mean ixel size in object sace is ~0.8 mm. A green line laser with adjustable focus, allowing width of the laser line ~0.5 mm. A tyical black-and-white chessboard attern (with one red square to fix the object system) for calibration uroses. Two lanes 1, 2 forming a corner (otional). O left image q hand-held laser Figure 1. The system setu Q s 1 s 2 2 laser lane 1 right image As the 3D object surface is manually swet over with the laser strie, the cameras cature synchronized frames of the scene recording the object, the background lanes and the laser strie which codes the scene. If homologous oints q, q of the laser rofile are identified, the rojective rays thereby defined intersect at the 3D oint Q (Fig. 1). In addition, all 3D oints resulting from a single laser rofile are bound to be colanar (on the laser lane). Furthermore, all 3D oints reconstructed from laser rofiles on a background lane must simultaneously belong to yet another lane (the corresonding background lane). These two constraints introduce a significant redundancy in the adjustment, thereby allowing higher accuracy and reliability. q O x x o1 y y o1 c 1 = λ R 1 X X o1 Y Y o1 Z Z o1 is modified for the right camera to accommodate the matrix of relative rotations R 12 and the three base comonents: x x o2 y y o2 c 2 = λ R 12 R 1 X Y Z X o1 Y o1 + R T 1 Z o1 B x B y B z In all calibration adjustments erformed, the standard error was about 0.2 ixels. A tyical calibration outut is seen in Table 1 (k i, i are the coefficients of lens distortion). In Fig. 2 a tyical stereo air used for calibration is shown. Table 1. Calibration results (from 20 image airs) ο = 0.21ixel left camera right camera cx (ix) ± ± 0.25 cy (ix) ± ± 0.28 x o (ix) ± ± 0.73 y o (ix) ± ± 0.51 k 1 ( ) ± ± 0.02 k 2 ( 10 ) ± ± ( 10 ) ± ± ( 10 ) ± ± 0.19 relative orientation Bx (cm) ± 0.01 By (cm) 2.00 ± 0.00 Bz (cm) ± 0.02 ω ± 0.04 φ ± 0.05 κ ± System calibration 3. THE SCANNING PROCESS Our grou has resented an algorithm * which accets a number of images of simle lanar chessboard atterns to automatically estimate the interior orientation of the camera used (Douskos et al., 2008). The algorithm first extracts the chessboard nodes via a Harris oerator, orders them and finally determines the camera geometry elements by bundle adjustment. Since here the scaled relative orientation of the cameras is also required, inut to our modified calibration algorithm is synchronized image airs of a chessboard attern of known grid size and with one of its black squares changed to red. The latter is automatically detected, and thus the origin of the chessboard coordinate system can be fixed (see Prokos et al., 2009, for more details). Evidently, the collinearity equation used for the left camera * The source code in Matlab of the calibration toolbox FAUCCAL, with documentation, tis and test imagery, is available on the Internet at: htt:// 506 Figure 2. A stereo air used for the calibration. 3.2 Image acquisition and subtraction For scanning, stereo airs are continuously taken from each osition of the static camera system; each air records the instantaneous rofile of the 3D surface which is intersected by the laser lane as the latter is slowly moved manually over the surface. Dull surfaces may be scanned with normal illumination of the scene (which may also be sufficient for caturing object texture of good quality); shiny surfaces need to be scanned with no exterior light source. Either way, laser rofiles have to be isolated from the background, i.e. from all frames a reference image (generated here as the temoral median of a few images) has to be subtracted. If illumination is good, a further use of these background images is to suly all surface oints with their secific hoto-texture for the uroses of visualization; else, from each scanning osition an extra image air may be taken under suitable illumination simly for hoto-texturing.

3.3 Peak detection on eiolar images Using the calibration data all image airs are transformed to eiolar airs, whereby known systematic images errors (here lens distortion) are removed.

Several eak detection aroaches have been reorted (Fisher & Naidu, 1996).

3 3.3 Peak detection on eiolar images Using the calibration data all image airs are transformed to eiolar airs, whereby known systematic images errors (here lens distortion) are removed. Thus, the search for homologous oints on the laser rofile is confined on corresonding eiolar lines (image rows), i.e. eaks must be determined on each image row. Several eak detection aroaches have been reorted (Fisher & Naidu, 1996). Here, a Gaussian curve is adated directly to the intensity values of each row: f x = ae x b 2 2σ 2 + d First, a threshold is alied to each row, roviding an estimation of the osition of the eak (or eaks) on an eiolar line as well as eak width. Curves are fitted only to ositions which yielded widths below a limit, in order to exclude stretched stries due to laser lanes intersecting the 3D surface at a small angle. The subixel estimation of the eak osition is given by arameter b in the above equation. However, this eak estimation uses data only in the direction of the image row. Thus, in order to relax the strictness of 1D interolation, two additional Gaussian curves with a common b arameter in the image x-direction are simultaneously fitted, namely in the directions of the two main diagonals through the initial eak estimation. The data from the two diagonals contribute in the estimation with smaller weight. Consequently, the final eak osition remains on the eiolar line, but is influenced by gray values from the neighborhood of the initial estimation. In Fig. 3 one may see the effect of this rocedure. X = Bx 1 Y = By Z = Bc ( = x 1 x 2 ) The oints of the background lanes must be searated into two grous, each reresenting the resective lane. This is done for each image air by fitting two 3D lines using RANSAC. End result is two oint clouds, from which coefficients of the background lanes are estimated. 3.5 Reconstruction algorithm As regards object scanning, very good initial estimations of the 3D osition of all oints of a laser rofile are obtained from the arallax equations; from these the coefficients of the laser lane are estimated. Together with the coefficients of the background lanes, this allows sorting rofile oints in three grous, namely oints on the two background lanes and object oints. But the arallax equations yield 3D object oints without redundancy (i.e. without a means for estimating recision or for gross error detection). In our aroach the answer to this, as mentioned, is the introduction of extra geometric constraints. First, triangulation is strengthened by the additional constraint that all 3D oints reconstructed from a recorded laser strie are colanar. Thus, the coefficients of all laser lanes are involved as unknown arameters in the adjustment. A further constraint is enforced by means of the two background lanes (also intersected by the laser lane). Obviously, the end-arts of the laser rofiles on either side of the object are straight (Fig. 4). Therefore, their oints must simultaneously satisfy the equations of the corresonding laser lane as well as those of the corresonding background lane. Estimates for the coefficients of the two background lanes are known from scanning the corner beforehand. Figure 3. Profile along an eiolar line. Curve fitted only along the image row (red) and curve fitted together with curves along the main diagonals (green). Before extracting eak ositions, a 3x3 Gaussian filter removes image noise. This mild filter was generally sufficient, due to the good quality of the emloyed laser (for lasers of oorer quality used in revious exeriments a median filter had to be alied first). It is ointed out that an increase of the window size of the filters may give better recision in eak detection; however, the end result of the 3D oint cloud will robably be too smooth. For lines with multile eak encounters (e.g. close to occlusion borders or due to reflections) the eaks are stored searately and rocessed as exlained later. 3.4 Background lanes Prior to scanning the object, the background lanes are scanned. After eaks on eiolar lines have been identified for all oint airs as outlined above, their x 1, x 2 and y image coordinates are used in the simle arallax equations in order to reconstruct the 3D oints: Figure 4. A tyical stereo air used in the scanning rocess. Besides the object, the laser lane intersects the two background lanes roducing linear segments on either side of the object. In Prokos et al. (2009) each laser rofile was adjusted indeendently, i.e. the arallax equations were combined with the laser lane equation and soft constraints for the linear end-segments. Consequently, in each adjustment a total of 2 N + 3 unknowns were involved, namely the X and Z coordinates of all N oints of the laser rofile (Y-values are directly found afterwards from the final -values) and the laser lane coefficients. Here, on the contrary, the robust aroach of a unified 3D reconstruction adjustment has been adoted. This means that all laser rofiles recorded from a articular viewoint of the camera system are adjusted together, with individual oints forced to belong to their (unknown) laser lane and, if they are oints of end-segments, also constrained to lie on the corresonding (unknown) background lane. Thus, unknowns here are the X and Z coordinates of all oints of the n rofiles lus 3 n coefficients of the laser lanes lus 6 coefficients of the background lanes (which are the common unknowns). It is noted that, in order to have longer linear end-segments and also include observations close to the 507

intersection of the lanes, the images of the background lanes without the object are also included in the fitting adjustment.

occuy the largest ossible image art (higher resolution in object sace). In such a case, an overall adjustment is clearly ointless, i.e. each rofile is rocessed indeendently.

The images used to drae this 3D model with texture where not created with the temoral median aroach but taken searately, since the 3D oint cloud was acquired with the subject s eyes closed.

4 intersection of the lanes, the images of the background lanes without the object are also included in the fitting adjustment. The second constraint is otional, in the sense that the object might not be laced in a corner or, if laced, background lanes might be only marginally visible in the images to allow the object to occuy the largest ossible image art (higher resolution in object sace). In such a case, an overall adjustment is clearly ointless, i.e. each rofile is rocessed indeendently. hase which, at the moment, takes several minutes. Under these circumstances, the result (seen in Fig. 5, bottom) is satisfactory. The images used to drae this 3D model with texture where not created with the temoral median aroach but taken searately, since the 3D oint cloud was acquired with the subject s eyes closed. An extra ste is to back-roject all 3D oints onto the air of reference images in order to interolate sets of RGB values which comlement the 3D data to roduce a final XYZ RGB set. Finally, the results from the different scanning sessions (from the different viewoints of the camera systems) are co-registered in a single 3D surface model using ICP. 3.6 Multile eaks Eiolar lines which roduce more than one eak are stored searately and do not articiate in the solution, i.e. reconstructed are at first only oints resulting from eiolar lines with a single eak. After the adjustment, 3D oints are calculated for all ossible combinations of stored multile eaks on eiolar lines. The actual object oints among them are searated from the outliers by means of a distance threshold from the estimated laser lane (Prokos et al., 2009). This is a further exloitation of the fact that all oints of a rofile are colanar. 4. APPLICATION AND EVALUATION 4.1 Exected accuracy The recision of 3D coordinates is directly related to the error σ of the x-arallax (), which is the result of the uncertainty σ x in the x-direction of eak ositions estimated through Gaussian curve fitting (arameter b). The arallax error is roagated in 3D sace through the image scale and the base-to-distance ratio. For the setu of Table 1 (c = 950 ixel, B = 40 cm), an average imaging distance of 70 cm in scanning the test cylinder (see below) and σ x = 0.1 ixel for the uncertainty of eak estimation (i.e. σ = 0.15 ixel), the tyical exected recision in deth is estimated as σ z = 0.2 mm. 4.2 Evaluation of accuracy The validity of the above estimation was checked by scanning a white PVC lumbing tube with a nominal diameter of 125 mm. A cylinder was fitted to the 3670 XYZ values of the oint cloud from one scanning osition which reresented aroximately 2/5 of the erimeter. The standard error of the surface-fitting adjustment was 0.2mm (the same as in Prokos et al., 2009). 4.3 Practical alications Objects scanned with our system were a olyester souvenir statue of Venus (height ~15cm), a 1985 Australian dollar coin and the face of one of the authors. The first object was scanned with a 20 cm base; the 3D model is seen in Fig. 5 (to). The coin is a rather extreme case, since the system has not been designed for very small objects. A 10 cm base was used. Crucial was here the width of the laser strie: ixel size was less than 0.1mm, but the laser line could not be narrower than 0.5mm, i.e. 5 ixels. The end result, shown in Fig. 5 (middle), was noisy but the surface aears to be adequately catured. The last object, scanned with a 30 cm base from two viewoints, also reresents an extreme case since the erson should remain frozen during the scanning Figure 5. Images and final 3D models: small statue of Venus (to), coin (middle) and face of the rimary author (bottom). 508

5 5. CONCLUSION An imlementation of a low-cost 3D scanner has been reorted, based on the combination of the stereovision and the slit scanner rinciles, accomanied by the introduction of extra geometric constraints. Comared to revious work (Prokos et al., 2009), a main goal here was to imrove the overall reconstruction reliability. This has been achieved by the unified adjustment of all laser rofiles from each scanning osition. Some comutational roblems have to be solved if all laser rofiles from all scanning viewoints are to be adjusted in a single solution. Future tasks include establishing further means for detecting outliers within but also between oint clouds from different scanning ositions. REFERENCES Blais F., Review of 20 years of range sensor develoment. Journal of Electronic Imaging, 13(1), Bouguet J.-Y., Perona P., D hotograhy on your desk. Proc. IEEE Int. Conf. on Comuter Vision, Davis, J., Chen, X., A laser range scanner designed for minimum calibration comlexity. Proceedings of Third International Conference on 3-D Digital Imaging and Modeling Arch. Phot. Rem. Sens., 37(B5), Fisher R.B., Naidu D.K., A comarison of algorithms for subixel eak detection. Advances in Image Processing, Multimedia and Machine Vision. Sringer, Forest J., Salvi J., A review of laser scanning three-dimensional digitisers. IEEE/RSJ Int. Conf. on Intelligent Robots and Systems, 1, Kawasaki H., Furukawa R., Dense 3D reconstruction method using colanarities and metric constraints for line laser scanning. 6 th IEEE Int. Conf. on 3D Digital Imaging and Modeling (3DIM 07), Prokos A., Karras G., Grammatikooulos L., Design and evaluation of a hotogrammetric 3D surface scanner. Proc. 22 nd CIPA Symosium, October 11-15, Kyoto, Jaan. Winkelbach S., Molkenstruck S., Wahl F.M., Low-cost laser range scanner and fast surface registration aroach. Proc. DAGM 06, Lecture Notes in Comuter Science, 4174, Sringer, Zagorchev L., Goshtasby A.A., A aint-brush laser range scanner. Comuter Vision & Image Understanding, 101, Douskos V., Kaliserakis I., Karras G.E., Petsa E., Fully automatic camera calibration using regular lanar atterns. Int. 509

DESIGN AND EVALUATION OF A PHOTOGRAMMETRIC 3D SURFACE SCANNER

DESIGN AND EVALUATION OF A PHOTOGRAMMETRIC 3D SURFACE SCANNER A. Prokos 1, G. Karras 1, L. Grammatikopoulos 2 1 Department of Surveying, National Technical University of Athens (NTUA), GR-15780 Athens,