Epipolar Constraint. Epipolar Lines. Epipolar Geometry. Another look (with math).

Transcription

1 Epipolar Constraint Epipolar Lines Potential 3d points Red point - fied => Blue point lies on a line There are 3 degrees of freedom in the position of a point in space; there are four DOF for image points in two planes Where does that fourth DOF go? Each point in one image corresponds to a line of possibilities in the other. Epipolar Line Epipolar Geometr => Red point lies on a line baseline Blue point - fied An epipole is the image point of the other camera s center. All epipolar lines meet at the epipoles. Epipoles lie on the cameras baseline. Is there other structure available among epipolar lines? We have two images, with a point in one and the epipolar line in the other. Lets take awa the image plane, and just leave the image centers. a translation vector defining where one camera is relative to the other.

2 a translation vector defining where one camera is relative to the other. A point on one image lies on ra in space with direction Which ras from, the second camera center might intersect ra p? Those ras lie in the plane defined b the ra in space and the second camera center. normal of the plane is perpindicular to both p and t. All lines in the plane are perpindicular to the normal to normal to the plane. Math fact: a b is a vector perpindicular to a and b. Math fact. a T b = 0 if a is perpindicular to b

3 Putting it all together. Lets put the images back in. Fact: Cameras seperated b translation t Ra from one camera center in direction p Ra from second camera center q P is relative to some coordinate sstem. All three vectors in the same plane: q is relative to some coordinate sstem, but that camera ma have rotated. So the q in the first coordinate sstem is some rotation times the q measured in the second coordinate sstem Put images even more back in. (,) K transforms piel coordinate locations into 3d directions from the camera center. Is called the calibration matri. F is the fundamental matri.

4 F is the fundamental matri. 33. Defined up to scale. One equation per corresponding point. How is it different from the homograph equation we were solving for? So what? Correspondence: Epipolar constraint. Potential 3d points Red point - fied => Blue point lies on a line Epipolar Line limits area of search for correspondence. Eamples Eamples

5 Eamples Eamples Geometricall, wh do all epipolar lines intersect? Rectification Rectification P f l p l Z r p r T + l Z f r = T Z Disparit: T Z = f l r d = l r O l O r b Then given Z, we can compute X b is the stereo baseline and Y. d measures the difference in retinal position between corresponding g points (Camps)

6 Using these constraints we can use matching for stereo Correspondence: Photometric constraint For each epipolar line For each piel in the left image compare with ever piel on same epipolar line in right image pick piel with minimum match cost This will never work, so: Same world point has same intensit in both images. Lambertian fronto-parallel Issues: Noise Specularit Foreshortening Improvement: match windows Comparing Windows: =? f g Window size (Camps) Most popular For each window, match to closest window on epipolar line in other image. Effect of window size W = 3 W = 20 Better results with adaptive window T. Kanade and M. Okutomi, A Stereo Matching Algorithm with an Adaptive Window: Theor and Eperiment,, Proc. International Conference on Robotics and Automation, D. Scharstein and R. Szeliski. Stereo matching with nonlinear diffusion. International Journal of Computer Vision, 28(2): , Jul 1998 Stereo results Data from Universit of Tsukuba Results with window correlation Scene Ground truth Window-based matching (best window size) Ground truth

7 Results with better method Stereo Results Trinocular stereo sstem available from Point Gra Research for $5K (circa 97) State of the art method Bokov et al., Fast Approimate Energ Minimization via Graph Cuts, International Conference on Computer Vision, September Ground truth Results Results Is there hope? Order! Once the epipolar geometr is known, we can increase constraints on potential correspondences further... Order! Once the epipolar geometr is known, we can increase constraints on potential correspondences further... corresponding points along corresponding epipolar lines corresponding points along corresponding epipolar lines 3d points giving rise to those images

8 Out-of-order points indicate seeing around an object, and generall doesn t occur... Order! Out-of-order points indicate seeing around an object, and generall doesn t occur... Order! corresponding points along corresponding epipolar lines corresponding points along corresponding epipolar lines Can ou dnamic programming! 3d points giving rise to those images Human abilities Human abilities