SINGLE VIEW GEOMETRY AND SOME APPLICATIONS

Size: px

Start display at page:

Download "SINGLE VIEW GEOMETRY AND SOME APPLICATIONS"

Lester May
5 years ago
Views:

1 SINGLE VIEW GEOMERY AND SOME APPLICAIONS

2 hank you for the slides. hey come mostly from the following sources. Marc Pollefeys U. on North Carolina Daniel DeMenthon U. of Maryland Alexei Efros CMU

Action of projectie camera on planes [ ] [ ] p p p 0 p p p p PX x 4 4 3 Y X Y X he most general transformation that can occur

3 Action of projectie camera on planes [ ] [ ] p p p 0 p p p p PX x Y X Y X he most general transformation that can occur between a scene plane and an image plane under perspectie imaging is a plane projectie transformation (affine camera-affine transformation)

4 Action of projectie camera on lines forward projection ( μ) P(A + μb) PA + μpb a μb X + back-projection Π P l Π X l PX

5 he importance of the camera center P KR[I C],P' K' R'[I C ~ ] P' K' R' ( )- KR P x' P' X K' R' x' Hx with H ( )- KR PX K' R' ( KR) - x K' R' ( KR) -

6 Moing the image plane (ooming) x x' K[I 0]X K'[I 0]X K' ( )- K x k)x ( ) ~ - ki ( 0 H K' K 0 k f f ki ( k)x ~ k)x ~ 0 ki ( 0 A K' K ka x~ 0 ki 0 K 0 0 ' x~ 0

7 Camera rotation x x' K[I 0]X K[R 0]X KRK - x - H KRK conjugate rotation { e iθ μ,μ,μe iθ }

8 Moing the camera center motion parallax epipolar line

9 Synthetic iew (i) Compute the homography that warps some a rectangle to the correct aspect ratio (ii) warp the image

10 Planar homography mosaicing

11 Projectie (reduced) notation 4 3 ) (0,0,0,,X (0,0,,0),X (0,,0,0),X (,0,0,0) X 4 3 ) (,, x, (0,0,) x, (0,,0) x, (,0,0) x d c d b d a P d c b a ),,, ( C

12 Object Pose with D Image Plane What happens if we don t know object s angle? 4

13 More Points Limited number of object poses ( or ) Head lights and one taillight ransparent car 5

14 Correspondence Problem When we know correspondences (i.e. matchings), pose is easier to find When we know the pose, correspondences are easier to find. But we need to find both at the same time Below, we first assume we know correspondences and describe how to sole the pose gien n corresponding points in image and object Perspectie n-point Problem hen we explore what to do when we don t know correspondences 6

15 w M i H N i P i K M 0 u Z 0 G C f k O m p i i m 0 y j i x Figure : Perspectie projection (m i ) and scaled orthographic projection (p i ) for an object point M i and a reference point M 0. 4

16 9 Iteratie Pose Calculation S S S y x Z Y X w u 3 r r r X r r r 3 y x w u i i i i Z Y X w ),,.( + r 3 X r r y x u [ ] y x u r r X y x Z Y X Z Y X Z Y X Z Y X u u u u r r u u u u y x - M r r Non coplanar points needed (otherwise matrix M is singular). At least 4 points.

17 0 Iteratie Pose Calculation Compute model matrix M and its inerse Assume Compute u i w i x i, i w i y i Compute Compute, x, y, r, r, then r 3 r x r Compute Go back to step and iterate until conergence i i i i Z Y X w ),,.( + r u u u u y x - M r r ),, ( i i i i w Z Y X. r 3

18 Iteratie Pose Calculation. Find object pose under scaled orthographic projection. Project object points on lines of sight 3. Find scaled orthographic projection images of those points 4. Loop using those images in step r 3

19 POSI for a Cube Left: Actual perspectie image for cube with known model op: Eolution of perspectie image during iteration Bottom: Eolution of scaled orthographic projection

20 4-3--? 4 - point perspectie solution Unique solution for 6 pose parameters Computational complexity of n 4 m point perspectie solution Generally two solutions per triangle pair, but sometimes four. Reduced complexity of n 3 m 3

21 3 Points Each correspondence between scene point and image point determines equations Since there are 6 degrees of freedom in the pose problems, the correspondences between 3 scene points in a known configuration and 3 image points should proide enough equations for computing the pose of the 3 scene points the pose of a triangle of known dimension is defined from a single image of the triangle But nonlinear method, to 4 solutions 4

22 riangle Pose Problem here are two basic approaches Analytically soling for unknown pose parameters Soling a 4th degree equation in one pose parameter, and then using the 4 solutions to the equation to sole for remaining pose parameters problem: errors in estimating location of image features can lead to either large pose errors or failure to sole the 4th degree equation Approximate numerical algorithms find solutions when exact methods fail due to image measurement error more computation 5

23 Numerical Method for riangle Pose A α B C' A' B' γ δ C Center of Projection β If distance R c to C is known, then possible locations of A (and B) can be computed they lie on the intersections of the line of sight through A' and the sphere of radius AC centered at C Once A and B are located, their distance can be computed and compared against the actual distance 6 AB

24 Numerical Method for riangle Pose H A A' α C' γ δ B' C β B Not practical to search on R c since it is unbounded Instead, search on one angular pose parameter, α. R c AC cos α sin δ R a R c cos δ ± AC sin α R b R c cos γ ± [(BC -(RC sin γ) ] his results in four possible lengths for side AB Keep poses with the right AB length 7

25 Choosing Points on Objects Gien a 3-D object, how do we decide which points from its surface to choose for its model? Choose points that will gie rise to detectable features in images For polyhedra, the images of its ertices will be points in the images where two or more long lines meet hese can be detected by edge detection methods Points on the interiors of regions, or along straight lines are not 8easily identified in images.

26 Choosing the Points Example: why not choose the midpoints of the edges of a polyhedra as features midpoints of projections of line segments are not the projections of the midpoints of line segments if the entire line segment in the image is not identified, then we introduce error in locating midpoint 0

27 Reducing the Combinatorics of Pose Estimation Reducing the number of matches Consider only quadruples of image features that Are connected by edges Are close to one another But not too close or the ineitable errors in estimating the position of an image ertex will lead to large errors in pose estimation Generally, try to group the image features into sets that are probably from a single object, and then only construct quadruples from within a single group 4

28 RANSAC RANdom SAmple Consensus Randomly select a set of 3 points in the image and a select a set of 3 points in the model Compute triangle pose and pose of model Project model at computed pose onto image Determine the set of projected model points that are within a distance threshold t of image points, called the consensus set After N trials, select pose with largest consensus set 6

29 e j» ertex (fixed point) A7. Planar homologies 69 \ \. fixed \ \. lines \ \ ^^_ ^ a axis (line of fixed points) Fig. A7.. A planar homology. A planar homology is a plane projectie transformation which has a line a of fixed points, called the axis, and a distinct fixed point, not on the line, called the centre or ertex of the homology. here is a pencil of fixed lines through the ertex. Algebraically, two of the eigenalues of the transformation matrix are equal, and the fixed line corresponds to the D inariant space of the matrix (here the repeated eigenalues are A and A3 J. Consider two images obtained by a camera rotating about its centre (as in figure.5- (p36)b); then as shown in section 8.4.(p04), the images are related by a conjugate rotation. In this case the complex eigenalues determine the angle (6) through which the camera rotates, and the eigenector corresponding to the real eigenalue is the anishing point of the rotation axis. Note that 6 - a metric inariant - can be measured directly from the projectie transformation. A7. Planar homologies A plane projectie transformation H is a planar homology if it has a line of fixed points (called the axis), together with a fixed point (called the ertex) not on the line, see figure A7.. Algebraically, the matrix has two equal and one distinct eigenalues and the eigenspace corresponding to the equal eigenalues is two-dimensional. he axis is the line through the two eigenectors (i.e. points) spanning this eigenspace. he ertex corresponds to the other eigenector. he ratio of the distinct eigenalue to the repeated one is a characteristic inariant \i of the homology (i.e. the eigenalues are, up to a common scale factor, {,,}). Properties of a planar homology include: Lines joining corresponding points intersect at the ertex, and corresponding lines (i.e. lines through two pairs of corresponding points) intersect on the axis. his is an example of Desargues' heorem. See figure A7.a. he cross ratios defined by the ertex, a pair of corresponding points, and the intersection of the line joining these points with the line of fixed points, are the same for all points related by the homology. See figure A7.b. For cures related by a planar homology, corresponding tangents (the limit of neighbouring points defining corresponding lines) intersect on the axis. he ertex ( dof), axis ( dof) and inariant ( dof) are sufficient to define the homology completely. A planar homology thus has 5 degrees of freedom.

30 630 Appendix 7 Some Special Plane Projectie ransformations ertex % Fig. A7.. Homology transformation, (a) Under the transformation points on the axis are mapped to themseles; each point off the axis lies on a fixed line through intersecting a and is mapped to another point on the line. Consequently, corresponding point pairs x <-> x' and the ertex of the homology are collinear. Corresponding lines - i.e. lines through pairs of corresponding points - intersect on the axis: for example, the lines (xi,x) and (xi,x ). (b) he cross ratio defined by the ertex, the corresponding points x, x' and the intersection of their join with the axis i, is the characteristic inariant of the homology, and is the same for all corresponding points. For example, the cross ratios of the four points {,Xj,xi,ii} and {,x,x,i} are equal since they are perspectiely related by lines concurrent at P. It follows that the cross ratio is the same for all points related by the homology. 3 matched points are sufficient to compute a planar homology. he 6 degrees of freedom of the point matches oer-constrain the 5 degrees of freedom of the homology. A planar homology arises naturally in an image of two planes related by a perspectiity of 3-space (i.e. lines joining corresponding points on the two planes are concurrent). An example is the transformation between the image of a planar object and the image of its shadow on a plane. In this case the axis is the imaged intersection of the two planes, and the ertex is the image of the light source, see figure.5(p36)c. Parametriation. he projectie transformation representing the homology can be parametried directly in terms of the 3-ectors representing the axis a and ertex, and the characteristic ratio [i, as H! + (- ) a a where I is the identity. It is straightforward to erify that the inerse transformation is gien by he eigenectors are F \ a I JJL I a {ei,e a^,e 3 a^-}

$A7.3 Elations 63 with corresponding eigenalues {Ai i,a,a 3 } where af- are two ectors that span the space orthogonal to the 3-ector a, i.e. a a i L 0 and a a] x a^.$

31 A7.3 Elations 63 with corresponding eigenalues {Ai i,a,a 3 } where af- are two ectors that span the space orthogonal to the 3-ector a, i.e. a a i L 0 and a a] x a^. If the axis or the ertex is at infinity then the homology is an affine transformation. Algebraically, if a (0. 0,), then the axis is at infinity; or if (i,, 0), then the ertex is at infinity; and in both cases the transformation matrix H has last row (0,0,). Planar harmonic homology. A specialiation of a planar homology is the case that the cross ratio is harmonic ( ). his planar homology is called a planar harmonic homology and has 4 degrees of freedom since the inariant is known. he transformation matrix H obeys H I, i.e. the transformation is a square root of the identity, which is called an inolution (also a collineation of period ). he eigenalues are, up to a common scale factor, {,,}. wo pairs of point correspondences determine H. In a perspectie image of a plane object with a bilateral symmetry, corresponding points in the image are related by a planar harmonic homology. he axis of the homology is the image of the symmetry axis. Algebraically, H is a conjugate reflection where the conjugating element is a plane projectie transformation. In an affine image (generated by an affine camera) the resulting transformation is a skewed symmetry, and the conjugating element is a plane affine transformation. For a skewed symmetry the ertex is at infinity, and the lines joining corresponding points are parallel. he harmonic homology can be parametried as a Again, if the axis or ertex is at infinity then the transformation is affine. A7.3 Elations An elation has a line of fixed points (the axis), and a pencil of fixed lines intersecting in a point (the ertex) on the axis. It may be thought of as the limit of a homology where the ertex is on the line of fixed points. Algebraically, the matrix has three equal eigenalues, but the eigenspace is -dimensional. It may be parametried as H I + xa with a 0 (A7.) where a is the axis, and the ertex. he eigenalues are all unity. he inariant space of H is spanned by a^.a^. his is a line (pencil) of fixed points (which includes since a 0). he inariant space of H is spanned by ectors f, ^ orthogonal to. his is a pencil of fixed lines, a^- + 3^-, for which l 0, i.e. all lines of the pencil are concurrent at the point. An elation has 4 degrees of freedom: one less than a homology due to the constraint a 0. It is defined by the axis a ( dof), the ertex on a ( dof) and the parameter H ( dof). It can be determined from point correspondences.

32 How can we model this scene?. Find world coordinates (X,Y,Z) for a few points. Connect the points with planes to model geometry exture map the planes Finding world coordinates (X,Y,Z). Define the ground plane (Z0). Compute points (X,Y,0) on that plane 3. Compute the heights Z of all other points

33 Perspectie cues Comparing heights Vanishing Point 6

34 Measuring height Camera height Computing anishing points (from lines) q q p p Intersect p q with p q Least squares ersion Better to use more than two lines and compute the closest point of intersection See notes by Bob Collins for one good way of doing this: 7

35 Criminisi 99 Vanishing line Vertical anishing point (at infinity) Vanishing point Vanishing point Measuring height without a ruler C Z ground plane Compute Z from image measurements Need more than anishing points to do this 8

36 Measuring height r x anishing line (horion) t 0 H t R H y b 0 b Measuring height r anishing line (horion) t 0 x t 0 y m 0 t b b 0 What if the point on the ground plane b 0 is not known? Here the guy is standing on the box Use one side of the box to help find b 0 as shown aboe b 9

37 he cross ratio A Projectie Inariant Something that does not change under projectie transformations (including perspectie projection) P P P 3 P P P P P P P P P he cross-ratio of 4 collinear points Can permute the point ordering 4! 4 different orders (but only 6 distinct alues) his is the fundamental inariant of projectie geometry i i i i Z Y X P P P P P P P P P Z r t b t b r r b t Z Z image cross ratio Measuring height B (bottom of object) (top of object) R (reference point) ground plane H C B R R B scene cross ratio Z Y X P y x p scene points represented as image points as R H R H R

38 What if is not infinity? 0

39 Measuring height r x anishing line (horion) t 0 H t R H y b 0 t b r b Z Z r t image cross ratio H R b Measuring heights of people Here we go! 85.3 cm reference

40 Forensic Science: measuring heights of suspects Vanishing line Reference height Reference height Assessing geometric accuracy Are the heights of the groups of people consistent with each other? Flagellation, Piero della Francesca Estimated relatie heights 3

points Specify 3D and D positions of 4 points on reference plane Compute homography H Specify a reference height Compute

41 Assessing geometric accuracy he Marriage of the Virgin, Raphael Estimated relatie heights Criminisi et al., ICCV 99 Complete approach Load in an image Click on lines parallel to X axis repeat for Y, Z axes Compute anishing points Specify 3D and D positions of 4 points on reference plane Compute homography H Specify a reference height Compute 3D positions of seeral points Create a 3D model from these points Extract texture maps Cut out objects Fill in holes Output a VRML model 4

42 Interactie silhouette cut-out Occlusion filling Geometric filling by exploiting: symmetries repeated regular patterns exture synthesis repeated stochastic patterns 5

More Single View Geometry

More Single View Geometry with a lot of slides stolen from Steve Seitz Cyclops Odilon Redon 1904 15-463: Computational Photography Alexei Efros, CMU, Fall 2008 Quiz: which is 1,2,3-point perspective Image