External Anatomical Shapes Reconstruction from Turntable Image Sequences using a Single off-the-shelf Camera

Transcription

1 External Anatomical Shapes Reconstruction from Turntable Image Sequences using a Single off-the-shelf Camera Teresa C. S. Azevedo INEGI Inst. de Eng. Mecânica e Gestão Industrial, LOME Lab. Óptica e Mecânica Experimental FEUP Faculdade de Engenharia da Universidade do Porto, PORTUGAL João Manuel R. S. Tavares & Mário A. P. Vaz INEGI, LOME DEMEGI Departamento de Engenharia Mecânica e Gestão Industrial, FEUP ABSTRACT: Three-dimensional (3D) human body reconstruction and modeling from multiple twodimensional (2D) images is a topic of great interest and research but still remains a difficult problem. Volumetric based methods are commonly used to solve this type of problem. With those methods, from a turntable image sequence of an object, a 3D model can be built. The final goal of the work presented in this paper is to reconstruct all-round 3D models of the human body, using a single off-the-shelf camera and volumetric techniques, with the best accuracy and photorealistic appearance. 1 HUMAN BODY 3D RECONSTRUCTION 3D imaging of physical objects has become a strong topic of research, mostly due to the increasing improvements in computational resources. Particularly, 3D reconstruction of the human body promise to open a very wide variety of applications: medicine, virtual reality, ergonomics applications, etc. Classical devices for external anatomical shape reconstruction are 3D scanners ((Levoy, Pulli et al., 2000), (Rocchini, Cignoni et al., 2001)). They are expensive but simple to use and fast. The technology involved can vary from laser light, structured light or time of flight, with an accuracy of millions of points ((Boehler, Heinz et al., 2001), (Remondino, 2004)). There are other techniques called image-based. 3D human body models are built from images, using approaches such as anthropometric statistics ((Barrón and Kakadiaris, 2000), (Mori and Malik, 2006)), multi-view geometry ((Remondino, 2004)) or occluding contours (Boyer and Berger, 1997). Shape from silhouettes ((Laurentini, 1994), (Srivastava and Ahuja, 1990)) is one of the most used volumetric methods. It combines silhouette images of an object with calibration information to set the visual rays in scene space for all silhouette points, which define a generalized cone within the object must be placed. The intersection of these cones defines a volume of scene space (visual hull) in which the object is guaranteed to be (Figure 1). Accuracy depends on the number of views used, the positions of each viewpoint, the calibration quality and the object s complexity. 1.1 Volumetric methods Traditional methods, like stereo matching or disparity maps, fail to capture shapes with complicated topology, like the human body because of its smoothness (Zeng, Lhuillier et al., 2005). Volumetric or voxel-based approaches have been quite popular for some time ((Seitz and Dyer, 1997), (Slabaugh, Culbertson et al., 2001)). They assume that there is a bounded volume in which the object of interest is placed. Figure 1. Visual hull from three viewpoints. Much refined voxel models can be obtained using space carving ((Kutulatos and Steiz, 1998), (Sainz, Bagherzadeh et al., 2002)). The reconstruction is initialized with a bounding box of voxels containing the true 3D object. The 3D shape of the object is constructed by removing (carving) voxels that are not photo-consistent with the reference views (Figure 2).

2 From a reconstructed volumetric model, a polygonal surface approximation can be constructed using, for example, the Marching Cubes algorithm ((Heckbert and Garland, 1997), (Lorensen and Cline, 1987)). Figure 2. Non photo-consistency lemma: if p isn t photoconsistent with some camera(s), it is non consistent with the entire set of cameras. 2 OUR METHODOLOGY In this paper, we present a volumetric method - Space Carving - for object 3D reconstruction. First step is camera calibration, which means finding the map between the 3D world and the 2D image space (Figure 3). We based our approach in Zhang s algorithm (Zhang, 2000). Several views of a chessboard pattern were used, acquired with a single off-the-shelf camera. Taken as inputs the correspondences between 2D image points and 3D scene points, Zhang s algorithm outputs the intrinsic camera matrix (focal length and principal point; skew γ is admited as zero), distortion coefficients and extrinsic parameters. For each calibration image, keeping the camera untouched, the object to reconstruct is placed on top of the pattern and another image is taken. So, for each turntable position we have a pattern image (for calibration) and an object image (for 3D reconstruction). To avoid all calibration images from having the pattern on the same plane, some extra images of the pattern in different orientations were taken. After calibration is done, for each object image of the turntable sequence, background/foreground segmentation is performed using some basic image processing tools. After the segmentation process, the object s model is built using a Generalized Voxel Coloring (GVC) technique (Loper, 2002). It allows arbitrary camera placement and generally provides better results than some other color-consistency algorithms (Slabaugh, Culbertson et al., 1999). Specifics of the algorithm are on Table 1. Finally, the volumetric model is polygonized (smoothed) using the Marching Cubes algorithm. Table 1. Pseudo-code for GVC algorithm. initialize surface voxel list (SVL) for every voxel V carved(v) = false loop { visibilitychanged = false compute item buffers by rendering voxels on SVL for every voxel V in SVL { compute vis(v) if (consist(vis(v)) = false) { visibilitychanged = true carved(v) = true remove V from SVL for all voxels N that are adjacent to V if (carved(n) = false and N is not in SVL) add N to SVL if (visibilitychanged = false) { save voxel space quit 3 EXPERIMENTAL RESULTS Figure 3. Camera s intrinsic and extrinsic parameters, mapping 3D world point coordinates into 2D image point coordinates. Our method was tested on two objects, very different on size and shape complexity: an human upper-torso (Figure 4) and an hand model (Figure 5). 3.1 Calibration results The camera to be calibrated was a NIKON D200 CCD camera. The chessboard plane contained 7x9 squares, so there where 48 inner corners, each square measuring 30x30mm. On Table 2 and Table 3 are the intrinsic results for the torso and hand model, respectively. Regarding extrinsic calibration results, the standard deviation of the reprojection error (in pixel), in both x and y directions, was [0.3584;0.3648] for the upper-torso object and [0.5513; ] for the hand object. It can be seen that, although the reprojection error is slightly higher for the hand model, the intrinsic parameters are much more accurate.

3 Figure 5. Hand object image sample. Three top pairs: object and pattern pair of turntable sequence; last pair: extra images used in calibration. Table 2. Intrinsic camera calibration results for torso model. Intrinsic camera matrix Distortion coefficients f x ± k ± f y ± k ± c x ± p ± c y ± p ± Table 3. Intrinsic camera calibration results for hand model. Intrinsic camera matrix Distortion coefficients f x ± k ± f y ± k ± c x ± p ± c y ± p ± Figure 4. Torso object image sample. Three top pairs: object and pattern pair of turntable sequence; last pair: extra images used in calibration. 3.2 Segmentation results Background/foreground segmentation means to binarize an image, extracting the object of interest (foreground) from the rest of the image (background). Using some image processing tools, like Sobel operator for edge detection, and different morphology functions in combination, as flood-fill operation to fill separated regions, it was possible to obtain reasonably good silhouettes for both objects turntable image sequence (Figure 6 and Figure 7). 3.3 Carving results Some snapshots of the 3D models obtained for both objects can be seen in Figure 8 and Figure 9.

4 Figure 6. Torso image and respective silhouette. Figure 9. 3D model of the hand model: left, original image; middle, voxel model; right, polygonized model. Although not very good for any of the objects, three-dimensional modelling had better results for the upper-torso object, because its shape has no abrupt changes comparing to the hand object and also because the higher distance between camera and object minimizes the effects of calibration errors. Figure 7. Hand image and respective silhouette. 4 CONCLUSIONS AND FUTURE WORK We can easily conclude that to have a complete and accurate 3D model of an object is very difficult. It gets even harder when the objects to modelize have strong shape variations, like the fingers in our hand model. Also, good calibration results are essential to get exact information about the object placement in 3D space. Further work will concentrate on improving the calibration method, specially on dividing into intrinsic and extrinsic calibration. Some chessboard pattern images will be used for intrinsic parameters estimation and the rotation and translation of the object will be estimated directly by the object images. Other improvements will be on Space Carving, mainly on the study of the influency of voxel coloring (since our objects have smooth colors). 5 ACKNOWLEDGMENTS This work was partially done in the scope of the project Segmentation, Tracking and Motion Analysis of Deformable (2D/3D) Objects using Physical Principles, with reference POSC/EEA- SRI/55386/2004, financially supported by FCT Fundação para a Ciência e a Tecnologia from Portugal. Figure 8. 3D model of the upper-torso model: left, original image; middle, voxel model; right, polygonized model.

5 REFERENCES C. Barrón, I. A. Kakadiaris, Estimating Anthropometry and Pose from a Single Image, IEEE Computer Society Conference on Computer Vision and Pattern Recognition Proceedings, New York: , W. Boehler, G. Heinz, A. Marbs, The potential of non-contact close range laser scanners for cultural heritage recording, XVIII International Symposium of CIPA Proceedings, Working Group VI, Potsdam, Germany, E. Boyer, M.-O. Berger, 3D Surface Reconstruction Using Occluding Contours, International Journal of Computer Vision, 22 (3): , P. S. Heckbert, M. Garland, Survey of Polygonal Simplification Algorithms, Multiresolution Surface Modeling Course, Siggraph Course Notes, ACM Press, New York (25), K. N. Kutulatos, S. M. Steiz, A Theory of Shape by Space Carving, Technical Report TR692, Computer Science Department, University of Rochester, New York, USA, A. Laurentini, The visual hull concept for silhouettebased image understanding, IEEE Transactions on Pattern Analysis and Machine Intelligence, 16 (2): , M. Levoy, K. Pulli, B. Curless, et al., The Digital Michelangelo Project: 3D scanning of large statues, Siggraph 2000, Computer Graphics Proceedings, ACM Press, ACM SIGGRAPH, Addison Wesley: , M. Loper, Archimedes - A Generalized Voxel Coloring Implementation, s_docs/html/main.html, W. E. Lorensen, H. E. Cline, Marching cubes: A high resolution 3D surface construction algorithm, International Conference on Computer Graphics and Interactive Techniques Proceedings, ACM Press, New York, USA, 21 (4): , G. Mori, J. Malik, Recovering 3D Human Body Configurations Using Shape Contexts, IEEE Transactions on Pattern Analysis and Machine Intelligence, 28 (7): , F. Remondino, 3-D Reconstruction of Static Human Body Shape from Image Sequence, Computer Vision and Image Understanding, 93 (1):65-85, C. Rocchini, P. Cignoni, C. Montani, et al., A low cost 3D scanner based on structured light, EURO- GRAPHICS Proceedings, Interlaken, Switzerland, 20 (3): , M. Sainz, N. Bagherzadeh, A. Susin, Carving 3D Models from Uncalibrated Views, Proceedings of the 5th IASTED International Conference Computer Graphics and Imaging, Hawaii, USA: , S. N. Seitz, C. R. Dyer, Photorealistic Scene Reconstruction by Voxel Coloring, Proceedings of the Computer Vision and Pattern Recognition Conference, San Juan, Puerto Rico: , G. Slabaugh, W. B. Culbertson, T. Malzbender, et al., A survey of methods for volumetric scene reconstruction from photographs, International Workshop on Volume Graphics Proceedings, New York, USA:21-22, G. G. Slabaugh, W. B. Culbertson, T. Malzbender, Generalized Voxel Coloring, Workshop on Vision Algorithms Proceedings, Corfu, Greece: , S. K. Srivastava, N. Ahuja, Octree generation from object silhouettes in perspective views, Computer Vision, Graphics and Image Processing, 49:68-84, G. Zeng, M. Lhuillier, L. Quan, Recent Methods for Reconstructing Surfaces from Multiple Images, Lecture Notes in Computer Science, Springer Verlag, 3519: , Z. Zhang, A Flexible New Technique for Camera Calibration, IEEE Transactions on Pattern Analysis and Machine Intelligence, 22 (11): , 2000.