Illustrations of 3D Projective Shape Reconstructions from Digital Camera Images

Transcription

1 Illustrations of 3D Projective Shape Reconstructions from Digital Camera Images Daniel E. Osborne Vic Patrangenaru Chayanne Burey Abstract In the past twenty years, digital images have increasingly becoming more accessible due to the technological advancements of computers, cellular phones, and digital cameras. Stereo cameras historically were used to simulate human binocular vision. A stereo camera is a type of camera with two or more lenses with a separate image sensor or film frame for each lens. This feature gave stereo cameras the ability to capture three-dimensional images. Human vision and pinhole camera image acquisition are based on a central projection principle. Moreover, two or more images of the same planar scene, in pixel coordinates, acquired by an ideal pinhole digital camera differ by a projective transformation of R 2. In this work, we illustrate some advantages of using two or more images of the same 3D scene for retrieval of the spatial information. This includes capturing the visual shape and texture of the object in the same 3D scene of images. All images were acquired using a basic digital camera. We conclude by outlying an estimation methodology in 3D Projective Shape Analysis problem for estimating the mean projective shapes of k-ads in general position in 3D. Key Words: 1. Introduction Surfaces are commonly found in nature as boundaries of ever-changing 3D scenes. Often time surfaces of interest are studied based on their images in face and object recognition. Collecting 3D surface data used to be a challenging task though, given that the output of a digital camera image, the widest spread type of imaging, is recorded on a flat surface, thus distorting most of the parts of the scene pictured, even in absence of occlusions. The first results in collecting 3D data on the cheap, from digital camera images were due to Longuet- Higgins [9], Faugeras [4], Hartley, Gupta and Chang [7] and others. These results, pointing to the idea that one may reconstruct a 3D scene, up to a 3D projective transformation, led Patrangenaru et. al. (see [17]) to the conclusion that all we see are 3D projective shapes. The 3D reconstruction methodology is summarized in a Computer Vision eight point algorithm given by Hartley and Zisserman ( see [8], [10] ). The problem of the 3D reconstruction of a configuration of points in 3D from two ideal uncalibrated camera images with unknown camera parameters, is equivalent to the following: given two camera images in RP1 2, RP 2 2 of unknown relative position and internal camera parameters and two matching sets of labeled points {p a,1,..., p a,k } RPa 2, a = 1, 2, find all the sets of points in the 3D space p 1,..., p k in such that there exist two positions of the planes RP1 2, RP 2 2 and internal parameters of the two cameras c a, a = 1, 2 with the property that the c a -image of p j is p a,j, a = 1, 2, j = 1,..., k. Department of Mathematics, Florida Agriculture and Mechanical University, Tallahassee, FL Department of Statistics, Florida State University, Tallahassee, FL Department of Mathematics & FAMU-FSU College of Engineering, Florida A&M University, Tallahassee, FL 32307

2 In absence of registration errors, in the uncalibrated case, the solution to the reconstruction problem unique up to a projective transformation (see Faugeras[4] and Hartley et. al. [7]). DEFINITION 1.1. Two sets of labeled points {p a,1,..., p a,k } RP 3 a, a = 1, 2, have the same projective shape if there is a projective transformation β : RP 3 RP 3, such that β(p 1,j ) = p 1,j, j = 1,... k. The reconstruction algorithm was therefore reformulated as follows by Sughatadasa [19] and by Patrangenaru et al. [18]: THEOREM 1.1. In absence of occlusions, any two 3D reconstructed configurations R, R obtained from a pair of 2D matched configurations in uncalibrated cameras images of a 3D configuration C, have the same projective shape. Note that the solution of the reconstruction problem, from a pair of 2D images depends on a landmark correspondence. So the key step in the reconstruction is finding the matched point configurations in the images. This in turn can be achieved easier if we use colored images, since the desired matching can be achieved if the two (or more) images are taken under the same illumination conditions, and then unlabeled points are matched via an RGB - correlation algorithm. Once the configurations are matched, one may attach at each reconstructed point in the resulting point cloud, the average RGB vectors at the corresponding points in the original images of the given scene. Such examples of 3D reconstructions with textures are given in Section Examples of 3D surface data collection In this section we give four examples of multiple images of 3D scenes and, separately, 3D reconstruction of the surfaces of these scenes, including textures. Figure 1: Images of art object by ceramic sculptor James Tanner. Figure 2: 3D reconstruction of art object by James Tanner. Figure 3: Images of porcelain tigers figurine. 2.1 Projective frames and projective shapes A mechanism for quantifying the projective shape data retrieved from images is given briefly in this section. Computations are left for a follow-up undergraduate project. A projective frame in RP 3 is an ordered 5-tuple of points π = (p 1,..., p 5 ), any 4 of which are in general position. The standard projective frame π 0 is the projective frame associated

3 Figure 4: 3D reconstruction of porcelain tigers figurine. Figure 5: Images of basket with solid balls. with the standard vector basis e = (e 1,..., e 4 ), of R 4, in this case π 0 = (p 0,1,..., p 0,4 ), where p 0,j = [e j ], j = 1,..., 4, and p 0,5 = [e e 4 ]. Note that since the action of a projective transformation is uniquely determined by its action on a projective frame (see Mardia and Patrangenaru [12]), given a point p RP m, its projective coordinates p π w.r.t. a projective frame π = (p 1,..., p 5 ) are defined as the image of p under the projective transformation that takes π to π 0. The projective coordinates of a point on the projective space w.r.t. a projective frame, as well as their equations, in terms of the coordinates of the projective frame are given in Patrangenaru [16], Mardia and Patrangenaru [12], Munk et. al.[14], or Patrangenaru et. al.[18]. Such projective coordinates are instrumental in the projective shape analysis of the scenes pictured. Acknowledgements V. Patrangenaru thanks and acknowledges FSU for travel support, and Vlad Patrangenaru for his help with searching for the reconstruction sofware Agisoft. The authors are grateful to the organizers of the 2017 JSM program. D. Osborne and V. Patrangenaru applaud Ms. Chayanne Burey efforts on this project as an Undergraduate Research Assistant. References [1] R. N. Bhattacharya, L. Ellingson, X. Liu and V. Patrangenaru and M. Crane (2012). Extrinsic Analysis on Manifolds is Computationally Faster than Intrinsic Analysis, with Applications to Quality Control by Machine Vision. Applied Stochastic Models in Business and Industry. 28, [2] M. Buibas, M. Crane, L. Ellingson and V. Patrangenaru (2012). A Projective Frame Based Shape Analysis of a Rigid Scene from Noncalibrated Digital Camera Imaging Outputs. In JSM Proceedings, 2011, Miami, FL. Institute of Mathematical Statistics, pp [3] M. Crane; V. Patrangenaru. (2011). Random Change on a Lie Group and Mean Glaucomatous Projective Shape Change Detection From Stereo Pair Image. Journal of Multivariate Analysis. 102, [4] Faugeras O. D. (1992) What can be seen in three dimensions with an uncalibrated stereo rig? In Proc. European Conference on Computer Vision, LNCS 588 pp [5] Ma, Y., Soatto, S., Kosecka, J. and Sastry, S.S. (2006). An Invitation to 3-D Vision, Springer, New York.

4 Figure 6: 3D reconstruction of basket with solid balls. Figure 7: Images of basket with flowers. [6] Goodall,C. and Mardia, K.V. (1999). Projective shape analysis, J. Graphical & Computational Statist., 8, [7] Hartley, R. I. ; Gupta R. ; and Chang T. (1992). Stereo from uncalibrated cameras, in Proc. IEEE Conference on Computer Vision and Pattern Recognition, pp [8] Hartley, R.I. and Zisserman A. (2004). Multiple view Geometry in computer vision,; 2 edition Cambridge University Press. [9] Longuet-Higgins C. (1981). A computer algorithm for reconstructing a scene from two projections, Nature, bf 293, [10] Ma, Y., Soatto, S., Kosecka, J. and Sastry, S.S. (2006). An Invitation to 3-D Vision, Springer, New York. [11] Mardia, K. V.; Goodall, Colin; Walder, Alistair. (1996). Distributions of projective invariants and model-based machine vision. Adv. in Appl. Probab.28, [12] K. V. Mardia and V. Patrangenaru (2005). Directions and Projective Shapes. Annals of Statistics 33, [13] K.V.Mardia, V. Patrangenaru, G. Derado and V. P. Patrangenaru (2003). Reconstruction of Planar Scenes from Multiple Views Using Affine and Projective Shape. In Proceedings of the 2003 Workshop on Statistical Signal Processing [14] Munk, A.; Paige, R.; Pang, J. ; Patrangenaru, V. and Ruymgaart, F. H.(2008). The One and Multisample Problem for Functional Data with Applications to Projective Shape Analysis. J. of Multivariate Anal.. 99, [15] Patrangenaru, V. (1999) Moving projective frames and spatial scene identification, in Proceedings in Spatial-Temporal Modeling and Applications, Edited by K. V. Mardia, R. G. Aykroyd and I.L. Dryden, Leeds University Press, p [16] Patrangenaru, V. (2001). New Large Sample and Bootstrap Methods on Shape Spaces in High Level Analysis of Natural Images. Communications in Statistics Theory and Methods [17] V. Patrangenaru, M. A. Crane, X. Liu, X. Descombes, G. Derado, W. Liu, V. Balan, V. P. Patrangenaru, H. W. Thompson (2012). Methodology for 3D Scene Reconstruction from Digital Camera Images. Proceedings of the International Conference of Differential Geometry and Dynamical Systems (DGDS-2011) October 6-9, 2011, Bucharest, Romania - BSG Proceedings 19,

5 Figure 8: 3D reconstruction of basket with flowers. [18] Patrangenaru, V; Liu, X. and Sugathadasa, S. (2010). Nonparametric 3D Projective Shape Estimation from Pairs of 2D Images - I, In Memory of W.P. Dayawansa. Journal of Multivariate Analysis. 101, [19] Sughatadasa, S. M. (2006) Affine and Projective Shape Analysis with Applications, Ph.D. Dissertation, Texas Tech Univesity.