TD2 : Stereoscopy and Tracking: solutions

Transcription

1 TD2 : Stereoscopy and Tracking: solutions Preliminary: λ = P 0 with and λ > 0. If camera undergoes the rigid transform: (R,T), then with, so that is the intrinsic parameter matrix. C(Cx,Cy,Cz) is the point such that P = 0 (R T) = 0 or So that : M2R Spécialité interaction JM Vézien 1

2 Exercice 1 : Homography and rectification a) Compute the 2d transforms (D C1 XZ and D C2 XZ) of the XZ plane, that «straigthen» the Z axis of both cameras, making them parallel to the Z axis of the world, but leaving the centers unchanged. D C1 XZ = and D C2 XZ = b) The Y coordinate is unchanged. We compute the displacement that brings the tilted camera C1 back to vertical (rotation around Y, -α), then shift camera back to origin (+T on X). Inverse for camera 2:, c) M2R Spécialité interaction JM Vézien 2

3 m 1 is the projection of M on parallel camera C 1. From preliminary: λ 1 = K(Id T) or Eq.(1) with = m1 is the projection of M on camera C1. Again: λ 1 = K(R T) or : Eq. (2) with From Eq (1) and (2), we see that: so that: (H 2 = K R K -1 ) In practice, the planar homographies H have 9 unknowns but only 8 d.o.f. (determined up to a scale λ). A 2D point matching (m,m ) will provide 2 equations, so that 4 point correspondences are enough to provide M2R Spécialité interaction JM Vézien 3

4 rectification homography. For example, one can image a rectangle, and precisely locate the four corners in each image. In our case, there are only 3 unknowns, f,t and α ( 2 points is enough!) Exercice 2: Stereoscopy : a practical case: Let us now consider the two parallel cameras obtain in exercice 1. a) From the projection equations it immediately follows that: and so that Or: b) Let us now consider the inverse, reconstruction process. M2R Spécialité interaction JM Vézien 4

5 i.) m 1 and m 2 have the same y 1 =y 2 =y. Same drawing applies, the two lines are in the same plane (O 1 m 1 O 2 m 2 ). Again, line of sights are: and (6 equations, only 5 unknowns). Developing, one finds: (Δx = x 1 -x 2 ). ii.) m 1 and m 2 have different y 1 and y 2. The two lines do not intersect anymore, but come close to one another in 3D space. Same equations apply: and This time there are 8 unknowns and only 6 equations! Actually, because the cameras are parallel, we will again have: So there are only really 6 unknowns but 4 usable equations in X 1,Y 1,X 2,Y 2,Z 1,Z 2. The reconstructed 3D segment M 1 M 2 can be any length without another explicit constraint. Let us compute (M 2 -M 1 ) 2 = (M 2 -M 1 ). (M 2 -M 1 ). Its derivative w.r.t. and should be zero, which will provide the missing equations. Let us call L = (M 2 -M 1 ). L 2 minimum inplies that and. Geometrical interpretation if that the segment L is orthogonal to its derivative. Let us write the equations now: and Producing the set of orthogonality equations: M2R Spécialité interaction JM Vézien 5

6 Matrix has form, so inverse is, giving and Optimal Z is defined as midpoint of segment L. One can verify that if y1=y2=y, segment vanishes. Exercice 3: Monocular Reconstruction Let us now consider a single camera C, which we will (for convenience) position at the origin of the world. Two rigidly bound points P 1 and P 2 form a segment S in 3D space, of length L. a) Let us call P 0 the midpoint of segment P 1 P 2. P 0 has depth Z 0 = ½(Z 1 +Z 2 ), and X P0 = 0. The equation has 3 parameters: Then and. Projected segment has length: b) Now midpoint has abscissa X 0, a fourth parameter: and M2R Spécialité interaction JM Vézien 6

7 c) General case: There are 5 D.o.F, with length L being fixed. We express the problem with 6 params + 1 constraint. and so that with one constraint: An alternative is to use P1 + polar coordinates for P2 relative to P1 d) The lines of sight emanating from p1 and p2 are: and. The 2 unknowns and are free parameters (in monocular view). Length L of segment P 1 P 2 is a constraint, so there is one free parameter. Here is the situation in the case of a segment in the XZ plane (simpler to draw): M2R Spécialité interaction JM Vézien 7

8 There can be two solutions, as in any length-contrained problem, lying on a circle. As P1 slides on the line of sight, potential P2 reconstructions are obtained by intersecting circle of radius In the case illustrated, the second solution is invalid (P2 in front of its projection). The limit of extent of P1 is the grazing case: Note that the situation is not symmetric, unless of course the segment P1P2 is symmetric relative to the camera Z axis. So one should build two sets of circles to get the complete locus for P1 and P2. e) Now if we have three points, we get three unknowns, and. The calibration target has three additional constraints: P1P2 has length L, so does P1P3, and. So this time it should be possible to get an exact and unique solution for P1, P2 and P3. Denote: We express Denoting, we get M2R Spécialité interaction JM Vézien 8

9 The solution in, and is non-linear and tricky to derive. And of course, as we are observing an external object with a camera attached to the world, this is an outside-in approach for tracking. Exercice 4: point detection in practice To perform the reconstruction of Exercice 3, we need to detect 2D points in images. Points per se do not exist in images. Rather, homogeneous zones can be detected by a variety of algorithms (SIFT, Harris detector, etc.). Let us suppose that such an algorithm is available and capable of perfect detection. Point P 1,P 2, P 3 of the right-angle object are in fact three small spheres coated with a special paint that makes detection perfect (meaning: all visible points of the sphere surface are imaged and perfectly detected). a) A circle, defined as the projection of the tangent points of sphere S. The representative is obviously the center of the circle, which, for symmetry, coincides with the projection of the sphere center. Computation: Sphere equation: Points of interest (silhouette): the optical ray tangent to the sphere (grazing) is orthogonal to the radius to the center of sphere : Combining the two gives: and ) The silhouette is a circle getting smaller as the sphere gets nearer. Projection equations are : x = Xf/Z and y = Yf/Z, so one derives: M2R Spécialité interaction JM Vézien 9

10 = r 2 The projection is a circle of radius r, this time getting bigger as the sphere approaches the camera! b) The projection of a sphere on a plane is an ellipse (never noticed that? Just make the experiment!). Let us shift the sphere center by offet Xs. The previous equations become: Developing and substracting the second equation from the first, one obtains: This is the equation of a vertical plane, making an angle α with the Z axis: = D, Distance D is the distance of the silhouette to the center of projection. Symmetry imposes it is a circle, of radius R 1. Let us denote S1 the center of the circle, and S2 and S3 the intersection of the silhouette circle with the plane XZ (see figure). M2R Spécialité interaction JM Vézien 10

11 Simple pythagora theorem says that angle, so that:. Again 0S 1 S 2 is a square = R 1 Finally the coordinates of S 1, S 2, S 3 are:, Because S 2 and S 3 do NOT have the same Z coordinate, when computing their projection, we no longer have same distance to projection of center S: c) The natural choice for representative of projection is the center of the ellipse. But strangely enough, it is not lying exactly where the projection of the sphere is. In other words, the center of gravity of a shape is not preserved by projection. To see this, one has simply to see that the center of segment s2s3 is NOT the projection s1 of sphere center S. One may notice that if then the distance D >> R, R 1 so that the projection is nearly orthographic (linear), with all points at the same depth relative to the camera plane. In this case, the two projections coincide. This is typically the case for marker detection, where the markers are small and far away (R/Zs < 0.1). M2R Spécialité interaction JM Vézien 11