Structure From Motion for Scenes Without Features

Size: px
Start display at page:

Download "Structure From Motion for Scenes Without Features"

Transcription

1 Structure From Motion for Scenes Without Features Anthony J. Yez Stefano Soatto Electrical and Computer Engineering, Georgia Institute of Technology, Atlanta - GA 3033 ayez@ece.gatech.edu Computer Science Department, University of California, Los Angeles - CA 90095, soatto@ucla.edu Figure 1: Images of a scene with smooth surfaces and constant isotropic radiance. Although pictorially simple, these scenes challenge most SFM algorithms because of the lack of photometrically distinct features. Abstract We describe an algorithm for reconstructing the 3D shape of the scene and the relative pose of a number of cameras from a collection of images under the assumption that the scene does not contain photometrically distinct features. We work under the explicit assumption that the scene is made of a number of smooth surfaces that radiate constant energy isotropically in all directions, and setup a regionbased cost functional that we minimize using local gradient flow techniques. 1 Introduction We address the problem of estimating the shape of a scene and the relative pose of a collection of cameras from a number of images. This is one of the classical problems of computer vision, known as structure from motion (SFM), and extensive literature exists, for which we refer the reader to Work supported by NSF IIS /CCR , ONR N , and AFOSR F the textbook [4]. Most of the existing work 1 in SFM concentrates on the case where the scene contains a number of photometrically distinct features, that can be associated to geometric primitives, such as points or lines, in the scene. When this is the case, feature correspondence across images can be established in a number of ways (again, see [4] for details), at which point the problem becomes purely geometric, and the framework of epipolar geometry captures the essential relationships among corresponding points in the images and their relation to the three-dimensional structure of the scene. In this paper, we concentrate on scenes that do not fit in this general scheme, in the sense of not having any photometrically distinct point features, such as the scene in figure 1. We operate under the explicit assumption that the scene is composed of smooth surfaces that support a constant radiance function, which projects onto the image to yield a piecewise smooth irradiance. These scenes challenge the most common algorithms for recovering SFM. We assume that the internal parameters of the cameras are known, and we seek to infer the shape of the scene, represented by a description of the surface of each object in some Euclidean reference frame, as well as the relative pose of the cameras. Since we cannot rely on individual point correspondences, we set up a cost functional that aims at matching regions, and integrate the irradiance over the entire domain of each image. We then develop gradient-based algorithms to estimate both the shape of the scene and the relative pose of the cameras. 1.1 Relation to previous work In [5], a method is proposed to solve the multi-frame shape reconstruction of a smooth shape with constant radiance as the joint region segmentation of a collection of calibrated images. The reconstruction was remarkably robust with respect to image noise and deviation from the assumed 1 There are exceptions, which we discuss in section 1.1. Techniques that address this type of scenes by checking the photoconsistency of each voxel are available, although most require precise knowledge of the position of the cameras (algorithms that exploit the occluding boundaries to estimate both shape and camera pose are also available; see section 1.1 for more details). 1

2 piecewise constant radiance assumption, but also particularly sensitive to (extrinsic) calibration errors, thereby requiring precise positioning of the cameras. We extend their results to allow the position and orientation of each camera to be unknown, and therefore part of the inference process. Therefore, we estimate simultaneously surface shape, constant radiance, and camera motion. Our work is also closely related to the variational approach to stereo reconstruction, championed by Faugeras and Keriven [3]. They also assume the camera positions to be known, and therefore our work can be interpreted as a special case of of [3], extended to allow arbitrary camera pose. It should be note that our approach, like [3, 5, 16], is based on gradient descent algorithms, and therefore we always assume that the rough positioning of the cameras is available to be used as initial conditions to the algorithms. Therefore, our algorithm can be interpreted as a refinement step of the results of any multi-frame stereo calibration or structure from motion algorithm. However, the initialization needs not be precise. In section 4 we will show results obtained by initialing the camera positions and orientation by hand. Similarly, this work relates to shape carving techniques [5], since the reconstruction is achieved by evolving a volume in a way that is consistent with the image data. We explicitly enforce smoothness constraints, and therefore our approach does not work for arbitrary objects. However, for objects that satisfy the assumptions, our approach exhibits significant robustness to measurement noise. Furthermore, to the best of our knowledge, motion estimation has not been addressed within the context of shape carving, where the cameras are assumed to be calibrated. Since we assume constant or smooth radiance, most of the shape information concentrates at the occluding boundaries, and therefore our work relates to the literature on shape from silhouettes []. That work has indeed been extended to allow inference of camera motion as well as scene shape [1], although that was done within the framework of epipolar geometry. We estimate motion directly using a gradient procedure, and therefore do not require establishing correspondence between (real or virtual) points. Nevertheless, it should be noticed that the conditions for unique reconstruction are, of course, the same, and therefore we are subject to the same constraints as in silhouette-based methods. For instance, a unique camera motion cannot be recovered in the presence of symmetries of the object. Nevertheless, the shape can still be recovered, albeit relative to an unknown reference frame. For the computational methods it uses, this paper is also related to a wealth of contributions in the field of regionbased segmentation, starting from Mumford and Shah s pioneering work [6], and including [11, 4, 14, 15, 18, 0, 1,, 3, 13]. Our numerical implementation is based on Osher and Sethian s level set methods [8]. 1. Outline and contributions of this paper In Section, we will review the model proposed in [5] for joint image segmentation and shape reconstruction for a calibrated stereo rig. In section 3 we will extend this model to estimate motion parameters. Although this extension is conceptually straightforward, in practice its implementation is entirely non trivial; we report the calculations in section 3, which constitutes the original contribution of this paper, and discuss the implications in section 3.1. The resulting algorithms are tested on real and synthetic image sequences in section 4. Reconstruction for calibrated cameras (review) In [5], a model for joint image segmentation and shape reconstruction has been proposed. It is assumed that the scene is composed of a number of smooth, closed surfaces supporting smooth Lambertian radiance functions (or dense textures with spatially smooth statistics) and the background, which occupies the rest of the image. Under these assumptions, a subset of brightness (or texture) discontinuities correspond to occluding boundaries. These assumptions make the image segmentation problem well-posed, although not general..1 Notation Let S to be a smooth surface in R 3 with local coordinates (u, v) R. Let da be its Euclidean area element, i.e. da = S u S v ; X = [X, Y, ] T the coordinates of a generic point on S. We measure n images, I i, i = 1,,..., n and are given the internal calibration parameters, so that the camera is modeled as an ideal perspective projection: π i : R 3 Ω i ; X x i, where x i = [x i, y i ] T = [X i / i, Y i / i ] T, Ω i R is the domain of the image I i, with area element dω i. We will use X i = [X i, Y i, i ] T to represent X in the camera coordinates with respect to the i-th camera. X and X i are related by a rigid body transformation, described by an element of the Euclidean group g i SE(3), represented in coordinate by a rotation matrix R i SO(3) and a translation vector T i R 3, so that 3 X i = g i X = R i X + T i. We describe the background B as a sphere with angular coordinates Θ = (θ, η) R that may be related in a one-to-one manner with the coordinates x i of each image domain Ω i through the mapping Θ i. We assume that the background supports a radiance function h : Θ R + and the surface supports another radiance function f : S R +. We 3 The reader will pardon an abuse of notation, since we mix the motion g i and its representation (R i, T i ); this is done for convenience of notation.

3 define the region Ω i = π i (S) Ω i and denote its complement by Ω c i. Although the perspective projection π i is not one-to-one (and therefore not invertible), the operation of back-projecting a point x i from Ω i onto the surface S can be defined by tracing the ray starting from the i-th camera center and passing through x i, and defining the first intersection point as the back-projection of x i onto S. Therefore, with an abuse of notation we denote this back-projection by i : Ω i S; x i X. In order to infer the shape of a surface S, one can impose a cost on the discrepancy between the projection of a model surface and the actual measurements. Such a cost, E, depends upon the surface S as well as upon the radiance of the surface f and of the background h, as well as the motion g i (through the projection π i ): E = E(f, h, S, g 1,..., g n ). For simplicity, we indicate by g the collection of camera motions g 1,..., g n. One can then adjust the shape of the model surface and radiances to match the measured images. Since the unknowns (surface S and radiances f, h) live in an infinite-dimensional space, we need to impose regularization. Therefore, the cost functional is a weighted average of three terms: E(f, h, S) = E data (f, h, S) + αe geom (S) + βe smooth (f, h, S) where α, β R +. The data fidelity term E data (f, h, S, g) quantifies the discrepancy between measured images and the images predicted by the model. For simplicity, we compute it in the sense of L on the image domain by E data = n ( i=1 Ω i f(π 1 i (x i )) I i (x i ) ) dxi + n ( i=1 Ω c h(θi (x i )) I i (x i ) ) dxi. E smooth (f, h, S) and i E geom (S) measure the smoothness of the radiance functions and the surface respectively. They are given by E geom = S da = area(s), and E smooth = S Sf da + B h dθ, where S denotes the intrinsic gradient on the manifold S. (The exact definition and details on its computation can be found in [10]).. Computation of the gradient flow (review) The data fidelity term may be measured To facilitate the computation of the variation with respect to S, we express these integrals over the surface S. This can be done using the characteristic functions χ i (X) = 1 if X visible from the i-th camera and χ i (X) = 0 otherwise. The data term E data is therefore given by: = nx i=1 nx i=1 Ω i ρ i (x i) dx i + Ω i ρ i (x i) dx i + Ωi f( i (x i )) I i (x i ) ρ i (x i ) dx i χ i (X)( ρ i (X) ρ i (π i(x)))σ i (X, N) da S where ρ i (X) = f(x) I i (π i (X)) and ρ i (x i ) = h(θ i (x i )) I i (x i ). We use the fact that Ω i is the projection of S in the i-th image and that the area measure dx i of the image is related to the area measure da of the surface by dx i = (X i N i )/i 3 da, where N is the inward unit normal to S, and N i is N with respect to the coordinates of the i-th camera. σ i (X, N) is a shorthand notation for (X i N i )/i 3. In the simpler case where both radiance functions f and h are constant, the overall cost functional can be simplified to: n E constant = α da + ρ i (x i ) dx i S i=1 Ω i n + χ i (X)( ρ i (X) ρ i (π i (X)))σ i (X, N)dA. i=1 S This simplification relates to the approach of Chan and Vese [4] who considered a piece-wise constant version of the Mumford-Shah functional for -D images in the level set framework [17, 19]. The gradient flow of the cost functional E has been derived in [5]. The flow corresponding to the data fidelity term is given by ds dt = 1 (f h) (I f)+(i h) ( χ S)+χ(I f)( f S) N z 3 (1) Notice that this flow depends only upon the image values, not the image gradient, which makes it more robust to image noise when compared to other variational approaches to stereo (i.e. less prone to become trapped in local minima). The gradient flow corresponding to the smoothness term, also derived in [5], is given by ds dt = ( II( S f N) S f H ) N () where the second fundamental form of S f N is computed as II( S f N) = f ug f u f v f + f v e EG F and the coefficients e, f, g of the second fundamental form are given by e = N, S uu, f = N, S uv, and g = N, S vv and the coefficients of the first fundamental form are E =S u S u, F =S u S v, G=S v S v. The term χ S must be defined in the distributional sense because the characteristic function χ is discontinuous. It can be shown that χ S = κ u S δ(s N) (3) where κ u denotes the normal curvature of S in the u- direction (the direction S where S N = 0). The overall flow can be computed by summing the flow corresponding 3

4 to each component of the cost functional. For the case of constant radiance, for instance, one gets ds dt = 1 z 3 (f h)[ (I f) + (I h) ] ( χ S)N = κ u S z 3 (f h) [ (I f) + (I h) ] δ(s N)N. 3 Evolving the motion parameters We now consider the same energy functional E as a function of the motion parameters g i SE(3). We will use the exponential parameterization of SE(3) via the twist coordinates ξ i R 6. The parameterization is established by a map from R 6 to the Lie algebra se(3) via ξ i ξ i, which is exponentiated to lead the motion g i = exp( ξ i ) SE(3). The reader can consult [7] for more details on twists and exponential coordinates for rigid motions. What matters, however, is that we can represent locally g with a six-parameter vector ξ. We will denote the local parameterization via g i = g i (ξ) where ξ = (ξ i1,..., ξ i6 ) for each camera image I i. Notice that the only term in our energy functional E which depends upon ξ i is the corresponding fidelity term in E data (due to the dependence of i and Θ i on ξ i ): E data,i (S, f, h, ξ i ) is therefore given by ( f(π 1 i ( x)) I i ( x) ) ( dxi + h(θi ( x)) I i ( x) ) dxi Ω i Ω c i (4) If we let c i = Ω i denote the boundary of Ω i then we may express the partial derivative of E with respect to one of E the calibration parameters ξ ij. is given by the sum of three terms: a boundary term, a foreground term and a background term, given respectively by ci f( c i i ( x)) I i ( x) h(θi ( x)) I i ( x) Ω i f( i ( x)) I i ( x) S f i ( x), Ω c i h(θ i ( x)) I i ( x) B h Θ i ( x),, n i d s i ( x) Θ i ( x) In the boundary term, d s denotes the arc-length measure of c i, and n i denotes its outward unit normal. In the foreground term, S denotes the natural gradient operator on the surface S, while in the background term, B denotes the gradient operator with respect to the angular coordinates of the background B. It is convenient to express the contour integral around c i ( s) in the image plane as a contour integral around C i (s) on the surface S instead, (where π i (C i )= c i and where s is the arc-length parameter of C i ). They are related by c i, n i d s = π i (C i ), s Jπ i(c i ) ds, dx i dx i [ ] 0 1 where J = and, therefore, the expression 1 0 above is given by * 0 z 1 x i y i i 3 s, 4 z i 0 x i y i x i 0 D = 1 xi E 3 s, xi x i ds = 1 D 3 = xi xi E 3, N i ds 3 5 xi + ds D xi, x i xi s E ds since x i and x i s are perpendicular tangent vectors to S. Thus, the boundary term written as an integral on the surface S (along the occluding contour C i ) has the following form. C i ( (f ) ( ) ) xi gi Ii h Ii 3, N i ds (5) The first step in rewriting the foreground/background integrals is to re-express the derivative of the back-projected 3D point x = i ( x, g i ) with respect to the calibration parameter ξ ij in terms of the derivative of the forward projection π i (x, g i ) = π(g i (x, g i )), since π i has an analytic form while i does not. We begin( by fixing a) D image point x and note that x = π i x( x, gi ), g i where x( x, g i ) = i ( x, g i ) = g 1 ( ) i ( x), g i and thus differentiation with respect to ξ ij yields: 0 = π i x π i x, g i = + π i x ξ ij = 1 0 x i gi x x i gi 0 z i y i x 0 z i y i ξ ij 0 x i gi x = 0 z i y i x 0 x i 0 z i y i gi (6) Notice, though, that (6) does not uniquely specify x/ but merely gives a necessary condition. We must supplement (6) with the additional constraint that x/ must be orthogonal to the unit normal N of S at the point x in order to obtain a unique solution. x N = 0 (7) Now, combining equations (6) and (7), we have 3 z i 0 x i 4 0 z i y i 5 g i x = N ix N iy N x iz x = 4 z i 0 x i 0 z i y i g i gi 1 I x i N i gi (8) x x i N i The second step proceeds in the same manner as outlined earlier in rewriting the data fidelity terms in E data by noting 4

5 that the measure in the image domain dx i and the area measure on the surface da are related by dx i = σ(x i, N i ) da where σ(x i, N i ) = (x i N i )/ 3. = Ω i f I i S f i ( x), i ( Ω i ) f I i S f(x), i ( x) dx i x xi N i 3 da The integrand above can be written more explicitly as f I i z i 3 S f(x), gi 1 (x x i N i ) g i gi N ξ i x i ij A similar derivation can be followed for the background term. The calculations above yield the gradient of the cost functional E with respect to the local coordinates of the motion parameters ξ, E ξ. This is transformed into a vector in the tangent space to the motion parameters g via the lifting ( E to the Lie algebra, ξ ) se(3). The evolution of the motion parameters is finally given by The final expression for the flow with respect to the local coordinates of the motion parameters is given by ( dg dt = ) E gdt (9) ξ To complete the algorithm, one or more steps of the flow (9) are alternated to one or more steps of the flow (1), until the value of the cost functional reaches steady state (it is easy to prove that, with an appropriate choice of step-size, every step lowers the value of the cost functional). 3.1 Uniqueness, or lack thereof Note that the flow converging to steady-state guarantees that the shape and motion parameters converge somewhere, but in general it does not guarantee that they converge to the correct shape or relative pose of the camera. For instance, consider the case of a sphere, imaged by a number of cameras distributed around a circumference centered at the center of the sphere. The image of the scene in each camera is identical, and therefore there is no way to tell where the cameras are. Nevertheless, one can conclude from the images that the scene is a sphere (assuming the cameras are in general position), and minimize the discrepancy of the model image (the projection of the estimated sphere) from the measured images. More in general, Euclidean symmetries in shape will generate ambiguities in the estimates of relative pose. One can have continuous symmetries (such as in the example of the sphere) or discrete symmetries (such as in the case of a homogeneous cube). Nevertheless, if one is interested in the shape of the scene, regardless of the positioning of the camera, the alternation of the flow (9) and (1) will indeed provide an estimate of shape that simultaneously explains each given image. total pixel error iteration Figure : Value of total squared reprojection error (E data in our cost functional) initially as only the shape is evolved while fixing the camera poses given by an external calibration procedure (solid line, 500 steps) and subsequently as the camera poses are also evolved (dotted line, 150 steps). Units are the sum over all pixels in each image of the squared intensity differences between the pixel and model intensities (units are 10 9 ). In general, a full-fledged analysis of the uniqueness of the minimizers of the functional we describe is well beyond the scope of this paper. However, some conclusions may be drawn from the analysis of SFM for the case of point features, for which we refer the reader to [4]. Naturally, since the algorithm we propose is a gradient flow, convergence is only guaranteed locally, since the algorithm can get trapped in local minima. However, in every experiment we have performed, some of which are reported in the next section, we have seldom experienced convergence to local minima despite coarse initialization. 4 Experiments In figure 1 we show a few images of a test scene meant to challenge the assumptions common to most SFM algorithms. Our scheme is design to work under these assumptions. In figure 3 we show the evolution of the estimate of shape if the pose of the cameras is taken to be the result of an external calibration procedure, and the significant improvement that follows when the camera pose is allowed to vary and is part of the inference process. This improvement is quantified in figure. In figure 4 we show the reprojection error, i.e. the best estimate of shape projected onto the image according to the best estimate of the camera pose, for when the camera parameters are fixed (top) or allowed to vary (middle). To emphasize the improvement that follows the evolution of the motion parameters, we also show the reprojection error when the true shape, but wrong parameters, are used. Finally, to emphasize the importance of incorporating 5

6 Figure 3: Evolution of the estimate of shape when the camera pose is fixed with an external calibration procedure (top, after 0, 50, 100, 300 and 500 steps); evolution of estimate of shape joined with the estimate of the motion parameters (middle, after 30, 60, 90, 110 and 150 steps); final estimate from several viewpoints (bottom). Figure 4: Reprojection error when the camera pose is fixed with an external calibration procedure (top), and when camera pose is estimated along with scene shape (middle). Reprojection error for the correct shape if the camera parameters were fixed (bottom). 6

7 the segmentation procedure behind our model as during the process of shape reconstruction and pose estimation rather than as a first step, we show in figure 4 the results of constructing the visual hull directly from the segmented images. Such a procedure, which relates much more directly to space carving and shape-from-silhouettes than our approach may seem quite tempting since the images are individually easy to segment. However, as can be seen in the figure, the final reconstruction obtained using this serial method of first segment then reconstruct suffers terribly in the presence of calibration errors. To highlight this point we also show the visual hull reconstruction using the final updated camera poses obtained after the evolution illustrated in the previous figures. Reconstruction via the visual hull 5 Conclusions We have presented an algorithm to estimate the shape of a scene composed of smooth surfaces with constant radiance as well as the relative pose of a collection of cameras. We define a cost functional that penalizes the discrepancy between the measured images and the projection of the estimated model onto the image, as well as regularing terms to enforce the smoothness assumptions. We define a gradient flow procedure that is guaranteed to minimize (locally) the cost functional. As the experiments show, our algorithm is very robust to image noise and to the initialization of the scene shape. It does require initialization of the relative pose of the cameras, although a manually inputed guess is usually sufficient. It performs well under the assumptions it is designed for. It does not work when the scene has nonsmooth radiance, a condition that allows other algorithms for SFM to work well. References [1] K. Astrom, R. Cipolla, and P. J. Giblin. Motion from the frontier of curved surfaces. pages 69 75, [] R. Cipolla and A. Blake. Surface shape from the deformation of apparent contours. Int. J. of Computer Vision, 9 (), 199. [3] O. D. Faugeras and R. Keriven. Variational principles, surface evolution pdes, level set methods and the stereo problem. INRIA Technical report, 301:1 37, [4] R. Hartley and A. isserman. Multiple view geometry in computer vision. Cambridge University Press, 000. [5] K. Kutulakos and S. Seitz. A theory of shape by space carving. In Proc. of the Intl. Conf. on Comp. Vision, [6] D. Mumford and J. Shah. Optimal approximations by piecewise smooth functions and associated variational problems. Comm. on Pure and Applied Mathematics, 4: , [7] R. M. Murray,. Li, and S. S. Sastry. A Mathematical Introduction to Robotic Manipulation. CRC Press, [8] S. Osher and J. Sethian. Fronts propagating with curvaturedependent speed: algorithms based on hamilton-jacobi equations. J. of Comp. Physics, 79:1 49, Reconstructions according to (poor) initial pose estimates. Reconstructions according to final evolved pose estimates Reconstruction via stereoscopic segmentation Figure 5: Direct shape reconstructions using the visual hull of the (pre)segmented images are shown on top both for the initial incorrect pose estimates (left) and for the final improved pose estimates recovered by our algorithm (right). Our shape reconstructions in both cases are shown below. Note that while the photo hull reconstruction on the top-right gives a decent shape, such a reconstruction was only possible by using the final pose estimates yielded simultaneously with the bottom-right shape by our algorithm. Also note that the initial shape (bottom-left) recovered by our algoritm before allowing it to evolve the pose estimates in order to obtain the final shape (bottom-right) still gives at least a somewhat recognizable coarse scale representation of the horse despite the significant calibration errors whereas the corresponding direct reconstruction of the visual hull (top-left) is not at all recognizable. 7

8 [9] R. Tsai. A versatile camera calibration technique for high accuracy 3d machine vision metrology using off-the-shelf tv cameras and lenses. IEEE J. Robotics Automat., RA-3(4):33 344, [10] M. Bertalmio, L. Cheng, S. Osher and G. Sapiro. Variational Problems and Partial Differential Equations on Implicit Surfaces. Journal of Computational Physics, pp , vol 174, No., Dec 001. [11] V. Caselles, R. Kimmel, G. Sapiro. Geodesic Active Contours. Int. J. Computer Vision, vol., no. 1, pp 61 79, Feb [1] A. Chakraborty and J. Duncan. Game-Theoretic Integration for Image Segmentation. IEEE Trans. Pattern Anal. Machine Intell., vol. 1, no. 1, pp. 1 30, Jan [13] A. Chakraborty, L. Staib, and J. Duncan. Deformable Boundary Finding in Medical Images by Integrating Gradient and Region Information. IEEE Trans. Medical Imaging, vol. 15, no. 6, pp , Dec [14] L. Cohen. On Active Contour Models and Balloons. CVGIP: Image Understanding, vol. 53, pp , [15] M. Kass, A. Witkin, and D. Terzopoulos, Snakes: Active Contour Models. Int. Journal of Computer Vision, vol. 1, pp , [16] R. Kimmel. 3D shape reconstruction from autostereograms and stereo. Journal of Visual Communication and Image Representation, 13:34-333, March 00. [17] S. Osher and C.-W. Shu. High-order Essentially Nonoscillatory Schemes for Hamilton-Jacobi equations. SIAM J. Numer. Anal., 8(4):907 9, [18] R. Ronfard, Region-Based Strategies for Active Contour Models. Int. J. Computer Vision, vol. 13, no., pp. 9 51, [19] Panagiotis E. Souganidis, Approximation Schemes for Viscosity Solutions of Hamilton-Jacobi Equations. J. Differential Equations, vol. 59, no. 1, pp 1 43, [0] H. Tek and B. Kimia. Image Segmentation by Reaction Diffusion Bubbles. Proc. Int. Conf. Computer Vision, pp , [1] D. Terzopoulos and A. Witkin. Constraints on Deformable Models: Recovering Shape and Non-rigid Motion. Artificial Intelligence, vol. 36, pp , [] S. hu, T. Lee, A. Yuille. Region Competition: Unifying snakes, Region Growing, and Bayes/MDL for Multiband Image Segmentation. Proc. Int. Conf. Computer Vision, pp , [3] S. hu and A. Yuille. Region Competition: Unifying snakes, Region Growing, and Bayes/MDL for Multiband Image Segmentation. IEEE Transactions on Pattern Analysis and Machine Intelligence, vol. 18, no. 9, pp , Sep [4] T. Chan and L. Vese. Active contours without edges IEEE Transactions on Image Processing, vol. 10, no., pp , Feb [5] A. Yez and S. Soatto. Stereoscopic segmentation. In Proc. of the Intl. Conf. on Computer Vision, pages 59 66,

Stereoscopic Segmentation

Stereoscopic Segmentation Stereoscopic Segmentation Anthony J. Yezzi Georgia Institute of Technology Electrical and Computer Engineering 777 Atlantic Dr. N.W. Atlanta GA 30332 ayezzi@ece.gatech.edu Stefano Soatto UCLA Computer

More information

Variational Multiframe Stereo in the Presence of Specular Reflections

Variational Multiframe Stereo in the Presence of Specular Reflections Variational Multiframe Stereo in the Presence of Specular Reflections Hailin Jin Anthony J. Yezzi Stefano Soatto Electrical Engineering, Washington University, Saint Louis, MO 6330 Electrical and Computer

More information

Active Geodesics: Region-based Active Contour Segmentation with a Global Edge-based Constraint

Active Geodesics: Region-based Active Contour Segmentation with a Global Edge-based Constraint Active Geodesics: Region-based Active Contour Segmentation with a Global Edge-based Constraint Vikram Appia Anthony Yezzi Georgia Institute of Technology, Atlanta, GA, USA. Abstract We present an active

More information

Automated Segmentation Using a Fast Implementation of the Chan-Vese Models

Automated Segmentation Using a Fast Implementation of the Chan-Vese Models Automated Segmentation Using a Fast Implementation of the Chan-Vese Models Huan Xu, and Xiao-Feng Wang,,3 Intelligent Computation Lab, Hefei Institute of Intelligent Machines, Chinese Academy of Science,

More information

Space-time Isosurface Evolution for Temporally Coherent 3D Reconstruction

Space-time Isosurface Evolution for Temporally Coherent 3D Reconstruction Space-time Isosurface Evolution for Temporally Coherent 3D Reconstruction Bastian Goldluecke Marcus Magnor Max-Planck-Institut für Informatik Graphics-Optics-Vision Saarbrücken, Germany {bg,magnor}@mpii.de

More information

Multiple View Geometry

Multiple View Geometry Multiple View Geometry CS 6320, Spring 2013 Guest Lecture Marcel Prastawa adapted from Pollefeys, Shah, and Zisserman Single view computer vision Projective actions of cameras Camera callibration Photometric

More information

Local or Global Minima: Flexible Dual-Front Active Contours

Local or Global Minima: Flexible Dual-Front Active Contours Local or Global Minima: Flexible Dual-Front Active Contours Hua Li 1,2 and Anthony Yezzi 1 1 School of ECE, Georgia Institute of Technology, Atlanta, GA, USA 2 Dept. of Elect. & Info. Eng., Huazhong Univ.

More information

Multiple Motion and Occlusion Segmentation with a Multiphase Level Set Method

Multiple Motion and Occlusion Segmentation with a Multiphase Level Set Method Multiple Motion and Occlusion Segmentation with a Multiphase Level Set Method Yonggang Shi, Janusz Konrad, W. Clem Karl Department of Electrical and Computer Engineering Boston University, Boston, MA 02215

More information

Stereo and Epipolar geometry

Stereo and Epipolar geometry Previously Image Primitives (feature points, lines, contours) Today: Stereo and Epipolar geometry How to match primitives between two (multiple) views) Goals: 3D reconstruction, recognition Jana Kosecka

More information

A Survey of Light Source Detection Methods

A Survey of Light Source Detection Methods A Survey of Light Source Detection Methods Nathan Funk University of Alberta Mini-Project for CMPUT 603 November 30, 2003 Abstract This paper provides an overview of the most prominent techniques for light

More information

Active Contour-Based Visual Tracking by Integrating Colors, Shapes, and Motions Using Level Sets

Active Contour-Based Visual Tracking by Integrating Colors, Shapes, and Motions Using Level Sets Active Contour-Based Visual Tracking by Integrating Colors, Shapes, and Motions Using Level Sets Suvarna D. Chendke ME Computer Student Department of Computer Engineering JSCOE PUNE Pune, India chendkesuvarna@gmail.com

More information

Silhouette-based Multiple-View Camera Calibration

Silhouette-based Multiple-View Camera Calibration Silhouette-based Multiple-View Camera Calibration Prashant Ramanathan, Eckehard Steinbach, and Bernd Girod Information Systems Laboratory, Electrical Engineering Department, Stanford University Stanford,

More information

Affine Surface Reconstruction By Purposive Viewpoint Control

Affine Surface Reconstruction By Purposive Viewpoint Control Affine Surface Reconstruction By Purposive Viewpoint Control Kiriakos N. Kutulakos kyros@cs.rochester.edu Department of Computer Sciences University of Rochester Rochester, NY 14627-0226 USA Abstract We

More information

Extract Object Boundaries in Noisy Images using Level Set. Literature Survey

Extract Object Boundaries in Noisy Images using Level Set. Literature Survey Extract Object Boundaries in Noisy Images using Level Set by: Quming Zhou Literature Survey Submitted to Professor Brian Evans EE381K Multidimensional Digital Signal Processing March 15, 003 Abstract Finding

More information

A Toolbox of Level Set Methods

A Toolbox of Level Set Methods A Toolbox of Level Set Methods Ian Mitchell Department of Computer Science University of British Columbia http://www.cs.ubc.ca/~mitchell mitchell@cs.ubc.ca research supported by the Natural Science and

More information

calibrated coordinates Linear transformation pixel coordinates

calibrated coordinates Linear transformation pixel coordinates 1 calibrated coordinates Linear transformation pixel coordinates 2 Calibration with a rig Uncalibrated epipolar geometry Ambiguities in image formation Stratified reconstruction Autocalibration with partial

More information

A Statistical Consistency Check for the Space Carving Algorithm.

A Statistical Consistency Check for the Space Carving Algorithm. A Statistical Consistency Check for the Space Carving Algorithm. A. Broadhurst and R. Cipolla Dept. of Engineering, Univ. of Cambridge, Cambridge, CB2 1PZ aeb29 cipolla @eng.cam.ac.uk Abstract This paper

More information

3D Computer Vision. Dense 3D Reconstruction II. Prof. Didier Stricker. Christiano Gava

3D Computer Vision. Dense 3D Reconstruction II. Prof. Didier Stricker. Christiano Gava 3D Computer Vision Dense 3D Reconstruction II Prof. Didier Stricker Christiano Gava Kaiserlautern University http://ags.cs.uni-kl.de/ DFKI Deutsches Forschungszentrum für Künstliche Intelligenz http://av.dfki.de

More information

An Active Contour Model without Edges

An Active Contour Model without Edges An Active Contour Model without Edges Tony Chan and Luminita Vese Department of Mathematics, University of California, Los Angeles, 520 Portola Plaza, Los Angeles, CA 90095-1555 chan,lvese@math.ucla.edu

More information

Multi-View 3D-Reconstruction

Multi-View 3D-Reconstruction Multi-View 3D-Reconstruction Cedric Cagniart Computer Aided Medical Procedures (CAMP) Technische Universität München, Germany 1 Problem Statement Given several calibrated views of an object... can we automatically

More information

Multiview Reconstruction

Multiview Reconstruction Multiview Reconstruction Why More Than 2 Views? Baseline Too short low accuracy Too long matching becomes hard Why More Than 2 Views? Ambiguity with 2 views Camera 1 Camera 2 Camera 3 Trinocular Stereo

More information

Lecture 16: Computer Vision

Lecture 16: Computer Vision CS4442/9542b: Artificial Intelligence II Prof. Olga Veksler Lecture 16: Computer Vision Motion Slides are from Steve Seitz (UW), David Jacobs (UMD) Outline Motion Estimation Motion Field Optical Flow Field

More information

Some books on linear algebra

Some books on linear algebra Some books on linear algebra Finite Dimensional Vector Spaces, Paul R. Halmos, 1947 Linear Algebra, Serge Lang, 2004 Linear Algebra and its Applications, Gilbert Strang, 1988 Matrix Computation, Gene H.

More information

Dense 3D Reconstruction. Christiano Gava

Dense 3D Reconstruction. Christiano Gava Dense 3D Reconstruction Christiano Gava christiano.gava@dfki.de Outline Previous lecture: structure and motion II Structure and motion loop Triangulation Today: dense 3D reconstruction The matching problem

More information

Stereo Vision. MAN-522 Computer Vision

Stereo Vision. MAN-522 Computer Vision Stereo Vision MAN-522 Computer Vision What is the goal of stereo vision? The recovery of the 3D structure of a scene using two or more images of the 3D scene, each acquired from a different viewpoint in

More information

Model Based Perspective Inversion

Model Based Perspective Inversion Model Based Perspective Inversion A. D. Worrall, K. D. Baker & G. D. Sullivan Intelligent Systems Group, Department of Computer Science, University of Reading, RG6 2AX, UK. Anthony.Worrall@reading.ac.uk

More information

Volumetric stereo with silhouette and feature constraints

Volumetric stereo with silhouette and feature constraints Volumetric stereo with silhouette and feature constraints Jonathan Starck, Gregor Miller and Adrian Hilton Centre for Vision, Speech and Signal Processing, University of Surrey, Guildford, GU2 7XH, UK.

More information

Method of Background Subtraction for Medical Image Segmentation

Method of Background Subtraction for Medical Image Segmentation Method of Background Subtraction for Medical Image Segmentation Seongjai Kim Department of Mathematics and Statistics, Mississippi State University Mississippi State, MS 39762, USA and Hyeona Lim Department

More information

Segmentation. Namrata Vaswani,

Segmentation. Namrata Vaswani, Segmentation Namrata Vaswani, namrata@iastate.edu Read Sections 5.1,5.2,5.3 of [1] Edge detection and filtering : Canny edge detection algorithm to get a contour of the object boundary Hough transform:

More information

Level Set Evolution without Reinitilization

Level Set Evolution without Reinitilization Level Set Evolution without Reinitilization Outline Parametric active contour (snake) models. Concepts of Level set method and geometric active contours. A level set formulation without reinitialization.

More information

Project Updates Short lecture Volumetric Modeling +2 papers

Project Updates Short lecture Volumetric Modeling +2 papers Volumetric Modeling Schedule (tentative) Feb 20 Feb 27 Mar 5 Introduction Lecture: Geometry, Camera Model, Calibration Lecture: Features, Tracking/Matching Mar 12 Mar 19 Mar 26 Apr 2 Apr 9 Apr 16 Apr 23

More information

College of Engineering, Trivandrum.

College of Engineering, Trivandrum. Analysis of CT Liver Images Using Level Sets with Bayesian Analysis-A Hybrid Approach Sajith A.G 1, Dr. Hariharan.S 2 1 Research Scholar, 2 Professor, Department of Electrical&Electronics Engineering College

More information

Notes 9: Optical Flow

Notes 9: Optical Flow Course 049064: Variational Methods in Image Processing Notes 9: Optical Flow Guy Gilboa 1 Basic Model 1.1 Background Optical flow is a fundamental problem in computer vision. The general goal is to find

More information

Geodesic Active Regions for Tracking I.N.R.I.A Sophia Antipolis CEDEX, France.

Geodesic Active Regions for Tracking I.N.R.I.A Sophia Antipolis CEDEX, France. Geodesic Active Regions for Tracking Nikos Paragios? Rachid Deriche I.N.R.I.A BP. 93, 24 Route des Lucioles 692 Sophia Antipolis CEDEX, France e-mail: fnparagio,derg@sophia.inria.fr Abstract. In this paper

More information

Segmentation and Tracking of Partial Planar Templates

Segmentation and Tracking of Partial Planar Templates Segmentation and Tracking of Partial Planar Templates Abdelsalam Masoud William Hoff Colorado School of Mines Colorado School of Mines Golden, CO 800 Golden, CO 800 amasoud@mines.edu whoff@mines.edu Abstract

More information

Perception and Action using Multilinear Forms

Perception and Action using Multilinear Forms Perception and Action using Multilinear Forms Anders Heyden, Gunnar Sparr, Kalle Åström Dept of Mathematics, Lund University Box 118, S-221 00 Lund, Sweden email: {heyden,gunnar,kalle}@maths.lth.se Abstract

More information

Structure from Motion. Introduction to Computer Vision CSE 152 Lecture 10

Structure from Motion. Introduction to Computer Vision CSE 152 Lecture 10 Structure from Motion CSE 152 Lecture 10 Announcements Homework 3 is due May 9, 11:59 PM Reading: Chapter 8: Structure from Motion Optional: Multiple View Geometry in Computer Vision, 2nd edition, Hartley

More information

Epipolar geometry contd.

Epipolar geometry contd. Epipolar geometry contd. Estimating F 8-point algorithm The fundamental matrix F is defined by x' T Fx = 0 for any pair of matches x and x in two images. Let x=(u,v,1) T and x =(u,v,1) T, each match gives

More information

A Variational Approach to Problems in Calibration of Multiple Cameras

A Variational Approach to Problems in Calibration of Multiple Cameras A Variational Approach to Problems in Calibration of Multiple Cameras Gozde Unal Siemens Corporate Research Princeton, NJ 08540 gozde@scr.siemens.com Anthony Yezzi Georgia Institute of Technology Atlanta,

More information

Multi-view stereo. Many slides adapted from S. Seitz

Multi-view stereo. Many slides adapted from S. Seitz Multi-view stereo Many slides adapted from S. Seitz Beyond two-view stereo The third eye can be used for verification Multiple-baseline stereo Pick a reference image, and slide the corresponding window

More information

Volumetric Scene Reconstruction from Multiple Views

Volumetric Scene Reconstruction from Multiple Views Volumetric Scene Reconstruction from Multiple Views Chuck Dyer University of Wisconsin dyer@cs cs.wisc.edu www.cs cs.wisc.edu/~dyer Image-Based Scene Reconstruction Goal Automatic construction of photo-realistic

More information

Computer Vision Lecture 17

Computer Vision Lecture 17 Computer Vision Lecture 17 Epipolar Geometry & Stereo Basics 13.01.2015 Bastian Leibe RWTH Aachen http://www.vision.rwth-aachen.de leibe@vision.rwth-aachen.de Announcements Seminar in the summer semester

More information

Computer Vision Lecture 17

Computer Vision Lecture 17 Announcements Computer Vision Lecture 17 Epipolar Geometry & Stereo Basics Seminar in the summer semester Current Topics in Computer Vision and Machine Learning Block seminar, presentations in 1 st week

More information

Level lines based disocclusion

Level lines based disocclusion Level lines based disocclusion Simon Masnou Jean-Michel Morel CEREMADE CMLA Université Paris-IX Dauphine Ecole Normale Supérieure de Cachan 75775 Paris Cedex 16, France 94235 Cachan Cedex, France Abstract

More information

Snakes reparameterization for noisy images segmentation and targets tracking

Snakes reparameterization for noisy images segmentation and targets tracking Snakes reparameterization for noisy images segmentation and targets tracking Idrissi Sidi Yassine, Samir Belfkih. Lycée Tawfik Elhakim Zawiya de Noaceur, route de Marrakech, Casablanca, maroc. Laboratoire

More information

Prof. Fanny Ficuciello Robotics for Bioengineering Visual Servoing

Prof. Fanny Ficuciello Robotics for Bioengineering Visual Servoing Visual servoing vision allows a robotic system to obtain geometrical and qualitative information on the surrounding environment high level control motion planning (look-and-move visual grasping) low level

More information

Recap: Features and filters. Recap: Grouping & fitting. Now: Multiple views 10/29/2008. Epipolar geometry & stereo vision. Why multiple views?

Recap: Features and filters. Recap: Grouping & fitting. Now: Multiple views 10/29/2008. Epipolar geometry & stereo vision. Why multiple views? Recap: Features and filters Epipolar geometry & stereo vision Tuesday, Oct 21 Kristen Grauman UT-Austin Transforming and describing images; textures, colors, edges Recap: Grouping & fitting Now: Multiple

More information

Segmentation in Noisy Medical Images Using PCA Model Based Particle Filtering

Segmentation in Noisy Medical Images Using PCA Model Based Particle Filtering Segmentation in Noisy Medical Images Using PCA Model Based Particle Filtering Wei Qu a, Xiaolei Huang b, and Yuanyuan Jia c a Siemens Medical Solutions USA Inc., AX Division, Hoffman Estates, IL 60192;

More information

Variational Methods II

Variational Methods II Mathematical Foundations of Computer Graphics and Vision Variational Methods II Luca Ballan Institute of Visual Computing Last Lecture If we have a topological vector space with an inner product and functionals

More information

Camera Calibration Using Line Correspondences

Camera Calibration Using Line Correspondences Camera Calibration Using Line Correspondences Richard I. Hartley G.E. CRD, Schenectady, NY, 12301. Ph: (518)-387-7333 Fax: (518)-387-6845 Email : hartley@crd.ge.com Abstract In this paper, a method of

More information

Level-set MCMC Curve Sampling and Geometric Conditional Simulation

Level-set MCMC Curve Sampling and Geometric Conditional Simulation Level-set MCMC Curve Sampling and Geometric Conditional Simulation Ayres Fan John W. Fisher III Alan S. Willsky February 16, 2007 Outline 1. Overview 2. Curve evolution 3. Markov chain Monte Carlo 4. Curve

More information

Structure from Motion and Multi- view Geometry. Last lecture

Structure from Motion and Multi- view Geometry. Last lecture Structure from Motion and Multi- view Geometry Topics in Image-Based Modeling and Rendering CSE291 J00 Lecture 5 Last lecture S. J. Gortler, R. Grzeszczuk, R. Szeliski,M. F. Cohen The Lumigraph, SIGGRAPH,

More information

Visual Recognition: Image Formation

Visual Recognition: Image Formation Visual Recognition: Image Formation Raquel Urtasun TTI Chicago Jan 5, 2012 Raquel Urtasun (TTI-C) Visual Recognition Jan 5, 2012 1 / 61 Today s lecture... Fundamentals of image formation You should know

More information

Kernel Density Estimation and Intrinsic Alignment for Knowledge-driven Segmentation: Teaching Level Sets to Walk

Kernel Density Estimation and Intrinsic Alignment for Knowledge-driven Segmentation: Teaching Level Sets to Walk C. Rasmussen et al. (Eds.), Pattern Recognition, Tübingen, Sept. 2004. LNCS Vol. 3175, pp. 36 44. c Springer Kernel Density Estimation and Intrinsic Alignment for Knowledge-driven Segmentation: Teaching

More information

Precise Omnidirectional Camera Calibration

Precise Omnidirectional Camera Calibration Precise Omnidirectional Camera Calibration Dennis Strelow, Jeffrey Mishler, David Koes, and Sanjiv Singh Carnegie Mellon University {dstrelow, jmishler, dkoes, ssingh}@cs.cmu.edu Abstract Recent omnidirectional

More information

Passive 3D Photography

Passive 3D Photography SIGGRAPH 99 Course on 3D Photography Passive 3D Photography Steve Seitz Carnegie Mellon University http:// ://www.cs.cmu.edu/~seitz Talk Outline. Visual Cues 2. Classical Vision Algorithms 3. State of

More information

Lecture 16: Computer Vision

Lecture 16: Computer Vision CS442/542b: Artificial ntelligence Prof. Olga Veksler Lecture 16: Computer Vision Motion Slides are from Steve Seitz (UW), David Jacobs (UMD) Outline Motion Estimation Motion Field Optical Flow Field Methods

More information

3D Shape Recovery of Smooth Surfaces: Dropping the Fixed Viewpoint Assumption

3D Shape Recovery of Smooth Surfaces: Dropping the Fixed Viewpoint Assumption IEEE TRANSACTIONS ON PATTERN ANALYSIS AND MACHINE INTELLIGENCE, VOL., NO., 1 3D Shape Recovery of Smooth Surfaces: Dropping the Fixed Viewpoint Assumption Yael Moses Member, IEEE and Ilan Shimshoni Member,

More information

Dynamic Shape Tracking via Region Matching

Dynamic Shape Tracking via Region Matching Dynamic Shape Tracking via Region Matching Ganesh Sundaramoorthi Asst. Professor of EE and AMCS KAUST (Joint work with Yanchao Yang) The Problem: Shape Tracking Given: exact object segmentation in frame1

More information

3D Dynamic Scene Reconstruction from Multi-View Image Sequences

3D Dynamic Scene Reconstruction from Multi-View Image Sequences 3D Dynamic Scene Reconstruction from Multi-View Image Sequences PhD Confirmation Report Carlos Leung Supervisor : A/Prof Brian Lovell (University Of Queensland) Dr. Changming Sun (CSIRO Mathematical and

More information

COMPARISON OF PHOTOCONSISTENCY MEASURES USED IN VOXEL COLORING

COMPARISON OF PHOTOCONSISTENCY MEASURES USED IN VOXEL COLORING COMPARISON OF PHOTOCONSISTENCY MEASURES USED IN VOXEL COLORING Oğuz Özün a, Ulaş Yılmaz b, Volkan Atalay a a Department of Computer Engineering, Middle East Technical University, Turkey oguz, volkan@ceng.metu.edu.tr

More information

Isophote-Based Interpolation

Isophote-Based Interpolation Isophote-Based Interpolation Bryan S. Morse and Duane Schwartzwald Department of Computer Science, Brigham Young University 3361 TMCB, Provo, UT 84602 {morse,duane}@cs.byu.edu Abstract Standard methods

More information

Week 2: Two-View Geometry. Padua Summer 08 Frank Dellaert

Week 2: Two-View Geometry. Padua Summer 08 Frank Dellaert Week 2: Two-View Geometry Padua Summer 08 Frank Dellaert Mosaicking Outline 2D Transformation Hierarchy RANSAC Triangulation of 3D Points Cameras Triangulation via SVD Automatic Correspondence Essential

More information

Region Based Image Segmentation using a Modified Mumford-Shah Algorithm

Region Based Image Segmentation using a Modified Mumford-Shah Algorithm Region Based Image Segmentation using a Modified Mumford-Shah Algorithm Jung-ha An and Yunmei Chen 2 Institute for Mathematics and its Applications (IMA), University of Minnesota, USA, 2 Department of

More information

Last update: May 4, Vision. CMSC 421: Chapter 24. CMSC 421: Chapter 24 1

Last update: May 4, Vision. CMSC 421: Chapter 24. CMSC 421: Chapter 24 1 Last update: May 4, 200 Vision CMSC 42: Chapter 24 CMSC 42: Chapter 24 Outline Perception generally Image formation Early vision 2D D Object recognition CMSC 42: Chapter 24 2 Perception generally Stimulus

More information

arxiv: v1 [cs.cv] 28 Sep 2018

arxiv: v1 [cs.cv] 28 Sep 2018 Camera Pose Estimation from Sequence of Calibrated Images arxiv:1809.11066v1 [cs.cv] 28 Sep 2018 Jacek Komorowski 1 and Przemyslaw Rokita 2 1 Maria Curie-Sklodowska University, Institute of Computer Science,

More information

NIH Public Access Author Manuscript IEEE Trans Pattern Anal Mach Intell. Author manuscript; available in PMC 2013 May 16.

NIH Public Access Author Manuscript IEEE Trans Pattern Anal Mach Intell. Author manuscript; available in PMC 2013 May 16. NIH Public Access Author Manuscript Published in final edited form as: IEEE Trans Pattern Anal Mach Intell. 2011 June ; 33(6): 1098 1115. doi:10.1109/tpami.2010.162. A Nonrigid Kernel-Based Framework for

More information

Feature Transfer and Matching in Disparate Stereo Views through the use of Plane Homographies

Feature Transfer and Matching in Disparate Stereo Views through the use of Plane Homographies Feature Transfer and Matching in Disparate Stereo Views through the use of Plane Homographies M. Lourakis, S. Tzurbakis, A. Argyros, S. Orphanoudakis Computer Vision and Robotics Lab (CVRL) Institute of

More information

Joint Priors for Variational Shape and Appearance Modeling

Joint Priors for Variational Shape and Appearance Modeling Joint Priors for Variational Shape and Appearance Modeling Jeremy D. Jackson and Anthony J. Yezzi Georgia Institute of Technology School of Electrical and Computer Engineering 777 Atlantic Av. Atlanta,

More information

STATISTICS AND ANALYSIS OF SHAPE

STATISTICS AND ANALYSIS OF SHAPE Control and Cybernetics vol. 36 (2007) No. 2 Book review: STATISTICS AND ANALYSIS OF SHAPE by H. Krim, A. Yezzi, Jr., eds. There are numerous definitions of a notion of shape of an object. These definitions

More information

Massachusetts Institute of Technology Department of Computer Science and Electrical Engineering 6.801/6.866 Machine Vision QUIZ II

Massachusetts Institute of Technology Department of Computer Science and Electrical Engineering 6.801/6.866 Machine Vision QUIZ II Massachusetts Institute of Technology Department of Computer Science and Electrical Engineering 6.801/6.866 Machine Vision QUIZ II Handed out: 001 Nov. 30th Due on: 001 Dec. 10th Problem 1: (a (b Interior

More information

Dense 3D Reconstruction. Christiano Gava

Dense 3D Reconstruction. Christiano Gava Dense 3D Reconstruction Christiano Gava christiano.gava@dfki.de Outline Previous lecture: structure and motion II Structure and motion loop Triangulation Wide baseline matching (SIFT) Today: dense 3D reconstruction

More information

CS231A Course Notes 4: Stereo Systems and Structure from Motion

CS231A Course Notes 4: Stereo Systems and Structure from Motion CS231A Course Notes 4: Stereo Systems and Structure from Motion Kenji Hata and Silvio Savarese 1 Introduction In the previous notes, we covered how adding additional viewpoints of a scene can greatly enhance

More information

Visualization 2D-to-3D Photo Rendering for 3D Displays

Visualization 2D-to-3D Photo Rendering for 3D Displays Visualization 2D-to-3D Photo Rendering for 3D Displays Sumit K Chauhan 1, Divyesh R Bajpai 2, Vatsal H Shah 3 1 Information Technology, Birla Vishvakarma mahavidhyalaya,sumitskc51@gmail.com 2 Information

More information

University of Southern California, 1590 the Alameda #200 Los Angeles, CA San Jose, CA Abstract

University of Southern California, 1590 the Alameda #200 Los Angeles, CA San Jose, CA Abstract Mirror Symmetry 2-View Stereo Geometry Alexandre R.J. François +, Gérard G. Medioni + and Roman Waupotitsch * + Institute for Robotics and Intelligent Systems * Geometrix Inc. University of Southern California,

More information

Some books on linear algebra

Some books on linear algebra Some books on linear algebra Finite Dimensional Vector Spaces, Paul R. Halmos, 1947 Linear Algebra, Serge Lang, 2004 Linear Algebra and its Applications, Gilbert Strang, 1988 Matrix Computation, Gene H.

More information

Global Illumination and the Rendering Equation

Global Illumination and the Rendering Equation CS294-13: Special Topics Lecture #3 Advanced Computer Graphics University of California, Berkeley Handout Date??? Global Illumination and the Rendering Equation Lecture #3: Wednesday, 9 September 2009

More information

Dynamic 3D Shape From Multi-viewpoint Images Using Deformable Mesh Model

Dynamic 3D Shape From Multi-viewpoint Images Using Deformable Mesh Model Dynamic 3D Shape From Multi-viewpoint Images Using Deformable Mesh Model Shohei Nobuhara Takashi Matsuyama Graduate School of Informatics, Kyoto University Sakyo, Kyoto, 606-8501, Japan {nob, tm}@vision.kuee.kyoto-u.ac.jp

More information

Outdoor Scene Reconstruction from Multiple Image Sequences Captured by a Hand-held Video Camera

Outdoor Scene Reconstruction from Multiple Image Sequences Captured by a Hand-held Video Camera Outdoor Scene Reconstruction from Multiple Image Sequences Captured by a Hand-held Video Camera Tomokazu Sato, Masayuki Kanbara and Naokazu Yokoya Graduate School of Information Science, Nara Institute

More information

Computer Vision I - Algorithms and Applications: Multi-View 3D reconstruction

Computer Vision I - Algorithms and Applications: Multi-View 3D reconstruction Computer Vision I - Algorithms and Applications: Multi-View 3D reconstruction Carsten Rother 09/12/2013 Computer Vision I: Multi-View 3D reconstruction Roadmap this lecture Computer Vision I: Multi-View

More information

1 (5 max) 2 (10 max) 3 (20 max) 4 (30 max) 5 (10 max) 6 (15 extra max) total (75 max + 15 extra)

1 (5 max) 2 (10 max) 3 (20 max) 4 (30 max) 5 (10 max) 6 (15 extra max) total (75 max + 15 extra) Mierm Exam CS223b Stanford CS223b Computer Vision, Winter 2004 Feb. 18, 2004 Full Name: Email: This exam has 7 pages. Make sure your exam is not missing any sheets, and write your name on every page. The

More information

STRUCTURE AND MOTION ESTIMATION FROM DYNAMIC SILHOUETTES UNDER PERSPECTIVE PROJECTION *

STRUCTURE AND MOTION ESTIMATION FROM DYNAMIC SILHOUETTES UNDER PERSPECTIVE PROJECTION * STRUCTURE AND MOTION ESTIMATION FROM DYNAMIC SILHOUETTES UNDER PERSPECTIVE PROJECTION * Tanuja Joshi Narendra Ahuja Jean Ponce Beckman Institute, University of Illinois, Urbana, Illinois 61801 Abstract:

More information

Photometric Stereo. Lighting and Photometric Stereo. Computer Vision I. Last lecture in a nutshell BRDF. CSE252A Lecture 7

Photometric Stereo. Lighting and Photometric Stereo. Computer Vision I. Last lecture in a nutshell BRDF. CSE252A Lecture 7 Lighting and Photometric Stereo Photometric Stereo HW will be on web later today CSE5A Lecture 7 Radiometry of thin lenses δa Last lecture in a nutshell δa δa'cosα δacos β δω = = ( z' / cosα ) ( z / cosα

More information

General Principles of 3D Image Analysis

General Principles of 3D Image Analysis General Principles of 3D Image Analysis high-level interpretations objects scene elements Extraction of 3D information from an image (sequence) is important for - vision in general (= scene reconstruction)

More information

A Survey of Image Segmentation Based On Multi Region Level Set Method

A Survey of Image Segmentation Based On Multi Region Level Set Method A Survey of Image Segmentation Based On Multi Region Level Set Method Suraj.R 1, Sudhakar.K 2 1 P.G Student, Computer Science and Engineering, Hindusthan College Of Engineering and Technology, Tamilnadu,

More information

Global Minimization of the Active Contour Model with TV-Inpainting and Two-Phase Denoising

Global Minimization of the Active Contour Model with TV-Inpainting and Two-Phase Denoising Global Minimization of the Active Contour Model with TV-Inpainting and Two-Phase Denoising Shingyu Leung and Stanley Osher Department of Mathematics, UCLA, Los Angeles, CA 90095, USA {syleung, sjo}@math.ucla.edu

More information

3D Reconstruction through Segmentation of Multi-View Image Sequences

3D Reconstruction through Segmentation of Multi-View Image Sequences 3D Reconstruction through Segmentation of Multi-View Image Sequences Carlos Leung and Brian C. Lovell Intelligent Real-Time Imaging and Sensing Group School of Information Technology and Electrical Engineering

More information

CHAPTER 3. Single-view Geometry. 1. Consequences of Projection

CHAPTER 3. Single-view Geometry. 1. Consequences of Projection CHAPTER 3 Single-view Geometry When we open an eye or take a photograph, we see only a flattened, two-dimensional projection of the physical underlying scene. The consequences are numerous and startling.

More information

Using Subspace Constraints to Improve Feature Tracking Presented by Bryan Poling. Based on work by Bryan Poling, Gilad Lerman, and Arthur Szlam

Using Subspace Constraints to Improve Feature Tracking Presented by Bryan Poling. Based on work by Bryan Poling, Gilad Lerman, and Arthur Szlam Presented by Based on work by, Gilad Lerman, and Arthur Szlam What is Tracking? Broad Definition Tracking, or Object tracking, is a general term for following some thing through multiple frames of a video

More information

Shading and Recognition OR The first Mrs Rochester. D.A. Forsyth, UIUC

Shading and Recognition OR The first Mrs Rochester. D.A. Forsyth, UIUC Shading and Recognition OR The first Mrs Rochester D.A. Forsyth, UIUC Structure Argument: History why shading why shading analysis died reasons for hope Classical SFS+Critiques Primitives Reconstructions

More information

Occluded Facial Expression Tracking

Occluded Facial Expression Tracking Occluded Facial Expression Tracking Hugo Mercier 1, Julien Peyras 2, and Patrice Dalle 1 1 Institut de Recherche en Informatique de Toulouse 118, route de Narbonne, F-31062 Toulouse Cedex 9 2 Dipartimento

More information

Spacetime-coherent Geometry Reconstruction from Multiple Video Streams

Spacetime-coherent Geometry Reconstruction from Multiple Video Streams Spacetime-coherent Geometry Reconstruction from Multiple Video Streams Marcus Magnor and Bastian Goldlücke MPI Informatik Saarbrücken, Germany magnor@mpi-sb.mpg.de Abstract By reconstructing time-varying

More information

Submitted by Wesley Snyder, Ph.D. Department of Electrical and Computer Engineering. North Carolina State University. February 29 th, 2004.

Submitted by Wesley Snyder, Ph.D. Department of Electrical and Computer Engineering. North Carolina State University. February 29 th, 2004. Segmentation using Multispectral Adaptive Contours Final Report To U.S. Army Research Office On contract #DAAD-19-03-1-037 Submitted by Wesley Snyder, Ph.D. Department of Electrical and Computer Engineering

More information

Hand-Eye Calibration from Image Derivatives

Hand-Eye Calibration from Image Derivatives Hand-Eye Calibration from Image Derivatives Abstract In this paper it is shown how to perform hand-eye calibration using only the normal flow field and knowledge about the motion of the hand. The proposed

More information

Image Segmentation II Advanced Approaches

Image Segmentation II Advanced Approaches Image Segmentation II Advanced Approaches Jorge Jara W. 1,2 1 Department of Computer Science DCC, U. of Chile 2 SCIAN-Lab, BNI Outline 1. Segmentation I Digital image processing Segmentation basics 2.

More information

Snakes, Active Contours, and Segmentation Introduction and Classical Active Contours Active Contours Without Edges

Snakes, Active Contours, and Segmentation Introduction and Classical Active Contours Active Contours Without Edges Level Sets & Snakes Snakes, Active Contours, and Segmentation Introduction and Classical Active Contours Active Contours Without Edges Scale Space and PDE methods in image analysis and processing - Arjan

More information

Occlusion-Based Accurate Silhouettes from Video Streams

Occlusion-Based Accurate Silhouettes from Video Streams Occlusion-Based Accurate Silhouettes from Video Streams Pedro M.Q. Aguiar, António R. Miranda, and Nuno de Castro Institute for Systems and Robotics / Instituto Superior Técnico Lisboa, Portugal aguiar@isr.ist.utl.pt

More information

The Level Set Method. Lecture Notes, MIT J / 2.097J / 6.339J Numerical Methods for Partial Differential Equations

The Level Set Method. Lecture Notes, MIT J / 2.097J / 6.339J Numerical Methods for Partial Differential Equations The Level Set Method Lecture Notes, MIT 16.920J / 2.097J / 6.339J Numerical Methods for Partial Differential Equations Per-Olof Persson persson@mit.edu March 7, 2005 1 Evolving Curves and Surfaces Evolving

More information

Rectification and Distortion Correction

Rectification and Distortion Correction Rectification and Distortion Correction Hagen Spies March 12, 2003 Computer Vision Laboratory Department of Electrical Engineering Linköping University, Sweden Contents Distortion Correction Rectification

More information

Implicit Active Contours Driven by Local Binary Fitting Energy

Implicit Active Contours Driven by Local Binary Fitting Energy Implicit Active Contours Driven by Local Binary Fitting Energy Chunming Li 1, Chiu-Yen Kao 2, John C. Gore 1, and Zhaohua Ding 1 1 Institute of Imaging Science 2 Department of Mathematics Vanderbilt University

More information