3D surface reconstruction of objects by using stereoscopic viewing Baki Koyuncu, Kurtuluş Küllü bkoyuncu@ankara.edu.tr kkullu@eng.ankara.edu.tr Computer Engineering Department, Ankara University, Ankara, Turkey Abstract 3D object scanning techniques were used to generate virtual images of objects. 2 cameras were employed to receive stereoscopic object surface information from an object on a rotating platform. The data received from both cameras were processed by using image processing techniques. Two point clouds, acquired from two cameras, were combined to obtain the 3D image of the object. Ball-Pivoting algorithm was used to reconstruct the object surface from these point clouds. Keywords: stereoscopic information, point clouds, Ball-Pivoting algorithm 1. Introduction 3D scanning is often used to create virtual 3-D models of real objects. There are various 3D scanning techniques each with its advantages and disadvantages. A common point for all these techniques is the resulting geometric information in the form of a set of 3D points for a scanned object. This set of points, representing the object surface geometry, is referred as point cloud. However, in most applications, object surfaces are modelled by polygon mesh models or Nonuniform rational B-spline (NURBS) models. Thus, a point cloud is often transformed to a surface model and this process is called surface reconstruction. Common techniques for producing polygon meshes from point clouds are Marching cubes, Marching triangles and Ball-Pivoting algorithm (BPA) all of which are based on the mathematical concept of Delaunay triangulation. In this study, BPA was implemented together with an additional preliminary statistical routine. This implementation was applied to a stereoscopic union of two point clouds acquired by two cameras for the same object. Finally these two point clouds generated the reconstructed 3D image of the test object. 2. Summary of Reconstruction Techniques Delaunay triangulation is the mathematical base for most of the reconstruction algorithms. Theoretically, a Delaunay Triangulation for point set in xy plane is such that no point in is inside the circumscribed circle (circumcircle) of any triangle in (See Figure 1). In 3D Delaunay Triangulation, the only difference was the consideration of circumscribed spheres (circumspheres) instead of circumcircles. Figure 1: A Delaunay triangulation in 2D plane with circumcircles Delaunay Triangulations, while constructing surface models, maximize the minimum angle among all angles of the triangles during triangulation. Hence, the generated triangle net eliminates the narrow angle triangles as much as possible. Marching cubes algorithm, which was developed by Lorensen and Cline [8] iterated through the 3D world by considering a cubic volume element (voxel) at every iteration. This algorithm determined the polygons which represented the part of the surface passing through a voxel cube. When symmetry and reflection was taken into account, there were 15 ISSN: 1792-4332 170 ISBN: 978-960-474-210-3
unique polygon possibilities inside a single cube (See Figure 2). The generated polygons were later combined to produce a complete object surface. Figure 2: Fifteen unique cube configurations Marching triangles and BPA methods overcame some of the limitations of marching cubes [2] [4]. They are both iterative in the sense that they grow a triangulation across an implicit surface based on the same local 3D Delaunay surface constraint [6]. This local surface constraint is explained in Section 3. 3. Ball-Pivoting Algorithm Given a partial surface mesh, a triangle,, can only be added to the mesh boundary at edge,, if no edge in the existing mesh is inside the sphere passing through the triangle vertices,,. touches a new point in point cloud. This new point together with the pivot edge forms a new triangle to be added to the mesh. Selection of a seed triangle consists of finding three vertices such that the ball touching them contains no other vertex point in point cloud. 4. Experiments An optical 3D scanner based on structured light was employed in this study [7]. Two digital cameras, simulating a pair of stereoscopic eyes, were integrated into the 3D scanner and each camera was calibrated with respect to their object distances (See Figure 4). Two cameras were placed on either side of a laser line source. The test object was stationary on a rotating platform in front of these cameras and the laser line source. Figure 3: M, T triangle and sphere representation BPA imposes the above constraint to ensure that the surface is locally a Delaunay surface. The application of BPA by using software techniques results in a final triangulated surface mesh, M, which is globally a Delaunay surface [3] [6]. BPA starts with a seed triangle and pivots a sphere (ball) of user defined radius around each edge of the current mesh boundary until the ball Figure 4: Layout of 3D scanner with two cameras, line laser and rotating platform Observation of a scanned object at different times or from different viewpoints produces independent multiple images. It is a common practice to align separate, partially overlapping images by using image transformation techniques into a single image. This process is referred as registration. Origin of the world coordinate-axes for 3D scanner was considered as the centre point of the rotating platform. Platform surface represented the xz plane and the yz plane was represented by the vertical plane which contained the laser light. Both cameras were calibrated with respect to these coordinates. They simultaneously captured images of the laser line on the object surface as the platform rotated at a constant speed for a full ISSN: 1792-4332 171 ISBN: 978-960-474-210-3
360 scan. See Figure 5 for image samples from 2 cameras. Figure 5: Image samples from camera #1 (left) and camera #2 (right) Ideally, two fully identical point clouds would be obtained from the scanning of the object since the conditions for both cameras were the same. Incident laser light was sometimes not seen by the camera. It was caused by the laser light being occluded by some of the object parts (See Figure 6). This was identified as occlusion problem and the final point cloud acquired from such an image contained holes (See Figure 7). Figure 6: Occlusion problem: there is an occluded section of incident laser line The problem was overcome by fitting the point clouds from two cameras on top of each other. The fitting, identified as registration, was simplified by the fact that both cameras produced point clouds with the same virtual world coordinates. Hence there was no image transformation involved due to coordinate differences. The small mismatches between the point clouds were the result of environmental effects in determining the 3D point locations during acquisition process. In order to reduce these mismatches, a preliminary statistical averaging routine was applied to the combined point clouds before using BPA. Figure 7: Part of point cloud acquired by camera #1 is missing due to occlusion (back half of the point cloud is clipped in this image for display purposes) 5. Averaging Routine Let be the set of points acquired from camera #1 and from camera #2. One should always keep in mind that numbers of 3D points in the two clouds were not equal ( ). Each 3D point, where represents the number of point cloud and represents the point number in the cloud P i, has a corresponding error vector x, y, z which represents the distance to the ideal 3D object image. Then each can be considered together with an error set whose elements are all for. When two point clouds are simply fitted on top of each other without any rotation or translation, a union was obtained where which needs to be considered together with (See Figure 8). A point in the new combined point cloud would ideally fall into two categories: (i) (ii) It may be obtained from 2 points one from each point cloud at the same location. It may be obtained from 1 point of a single point cloud. ISSN: 1792-4332 172 ISBN: 978-960-474-210-3
mean of minimum distances in the particular point cloud. Consider a small point cloud shown by Figure 9 and defined by: 0,0,0, 1,0,0, 1,1,0, 1,0,1, 1,1,1 Figure 9: Simple example 3D point cloud Figure 8: Point cloud P1 from camera #1 (top left) and point cloud P2 from camera #2 (top right) placed on top of each other to produce P (bottom) In practice, a point in (i) has several points in close proximity, whereas for a point in (ii) the closest points are those sampled from distinct surface locations. If a statistical averaging operation was applied to 3D regions in small enough not to include two points from the same cloud, one would expect a reduced identified as with a reduced resulting from the averaging effect. In this study, the primary concern was to determine the 3D region size where 3D region is defined as a cube with a side length identified as region size ( ). Various region sizes related to characteristics of the separate point clouds as well as their union were tested. Since a region would be small enough not to include two points from the same cloud, an indication of the distances between the points would be required. This indication was identified as average minimum distance shown with symbol. For each point cloud,a corresponding average minimum distance ( for, for, for, and for ) was calculated by the arithmetic To calculate the average minimum distance for, distance between the considered point and the point closest to it was calculated for each point in. Hence the minimum distances are: 1, 1, 1, 1, 2 Then, the average minimum distance would be 1.0828 1 1 1 1 2 5 In this study, average minimum distances and for the point clouds and were calculated similarly by using software techniques. Smaller of and was selected to determine the which was later used to calculate from. BPA would be applied on and with respect to various ball radii to reconstruct the object surface. In this context, the relationship between and values was analysed in two sets of experiments: (a) was determined by using and using equation. (b) was determined by using and equation. where was a constant arbitrarily changed in the experiments. ISSN: 1792-4332 173 ISBN: 978-960-474-210-3
6. Results The test object, a pencil holder (See Figure 10), was scanned during the experiments. The two point clouds, and were generated and presented in Figure 8. LATEST TRENDS on APPLIED MATHEMATICS, SIMULATION, MODELLING Figure 10: The test object scanned The number of 3D points in, and were 34447, 33593 and 68040 34447 33593 respectively. Corresponding average minimum distances for, and were then computed as: 4.62789 10 4.80166 10 4.49917x10 The for reduced point cloud was defined as: 2 min, 2 4.62789 10 9.25578 10 Finally, was calculated for different values in Table 1 in order to be used with BPA. 1 9.25578x10-2 4.49917x10-2 1.5 1.388367x10-1 6.748755x10-2 2 1.851156x10-1 8.99834x10-2 2.5 2.313945x10-1 1.1247925x10-1 3 2.776734x10-1 1.349751x10-1 3.5 3.239523x10-1 1.5747095x10-1 4 3.702312x10-1 1.799668x10-1 Table 1: Values of r for each test case The following Figures 11 and 12 show resulting surfaces for each experiment group respectively. It is seen in Figure 11 that a ball radius of 2.5 produces best results. Figure 11: Resulting surfaces for increasing c values in Table 1 from left to right (top to bottom) in experiment set (a) The results given in Figure 12 show very little reconstructed surface area. This is due to average minimum distance for which is a poor indication of point distances. It is caused by 3D points which are not reduced from the same surface location in and. Figure 12: Resulting surfaces for increasing c values in Table 1 from left to right (top to bottom) in experiment set (b) In conclusion, choosing the ball radius according to the average minimum distance in a point cloud is found to be a good approach for surface reconstruction. ISSN: 1792-4332 174 ISBN: 978-960-474-210-3
Bibliography [1] de Berg, M; van Kreveld, M; Overmars, M; Schwarzkopf, O. Computational Geometry: Algorithms and Applications. Springer- Verlag. ISBN: 978-3-540-77973-5. [2] Bernardini, F; Mittleman, J; Rushmeier, H; Silva, C; Taubin G. The Ball-Pivoting Algorithm for surface reconstruction. IEEE Transactions of Visualization & Graphics. 1999. [3] Fortune, S. Voronoi diagrams and Delaunay triangulations. In Computing in Euclidian Geometry, pages 193-230, 1992. [4] Fournier, M; Dischler, J.M; Bechmann, D. Marching Triangle polygonization for efficient surface reconstruction from its distance transform. Discrete Geometry for Computer Imagery, pages 168-179, Springer Berlin, 2009. [6] Hilton, A; Stoddart, A.J; Illingworth, J; Windeatt, T. Marching triangles: range image fusion for complex object modeling. Proc. of the International Conference on Image Processing, Vol. 1, September 1996. [7] Koyuncu, B; Kullu, K. Development of an optical 3D scanner based on structured light. Proc. of the 9 th WSEAS Int. Conf. on Artificial Intelligence, Knowledge Engineering and Data Bases (AIKED 10). Cambridge, UK, February 2010. [8] Lorensen, W.E; Cline, H.E. Marching Cubes: A high resolution 3D surface construction algorithm. Computer Graphics, Vol. 21 (4), July 1987. [9] Okabe, A; Boots, B; and Sugihara, K. Spatial Tessellations: Concepts and Applications of Voronoi Diagrams. New York: Wiley, 1992. [5] Guibas, L; Stolfi, J. Primitves for the manipulation of general subdivisions and the computation of Voronoi. ACM transactions on Graphics, Vol. 4 (2), April 1985. ISSN: 1792-4332 175 ISBN: 978-960-474-210-3