Stereo. CS 510 May 2 nd, 2014

Transcription

1 Stereo CS 510 May 2 nd, 2014

2 Where are we? We are done! (essentiay) We covered image matching Correation & Correation Fiters Fourier Anaysis PCA We covered feature-based matching Bag of Features approach Incuding: Feature Extraction Feature Description Custering Not a bad introduction to vision

3 But we skipped a few topics Like 3D vision 3D sensors (e.g. Kinect) Stereo (i.e. mutipe overapping cameras) Structure from Motion Like Video Object tracking Motion segmentation Activity recognition

4 Let s fix one omission: Stereo he abiity to infer 3D structure and distance from two or more overapping images taken simutaneousy from different viewpoints Are these stereo images? Describe the viewpoints

5 Scenarios Most common: perpendicuar optica axes Aso common: converging optica axes (e.g. eyes) More common than you might think: arbitrary axes

6 wo SubProbems: Image Matching (correspondence) identifying which points in image #1 match which points in image #2 note: not a points in image #1 match anything in image #2. Why not? Note: not a matching points can be found. Reconstruction Given point matches, determine their 3D position Requires trianguation (impicit or expicit)

7 Image Matching Find common scene points in two images Occusion Incompete overap of visua fieds Potentiay strong perspective effects Genera Methods: Correation based Cross-correate every pixe in eft image to right image Epipoar geometry can constrain this search Feature based Extract points, edges, ines, etc., and match them across image

8 Reconstruction as rianguation Assume that the positions and baseines of the cameras are known: P = t [ x, y, f ] [ ] P =!" b x, b y, b z # $ + t r x r, y r, f r f f r Sove for t s, compute coordinate of point. Q: Isn t this overconstrained? baseine

9 Epipoar Geometry For any point in image #1, there is a ine of points in image #2 such that its match (if one exists) must ie on that ine. here is a pane defined by the two foca points and the 2D point in image #1. he 3D point must ie in this pane. Aso, the matching point in image #2 must ie in this pane.

10 Epipoar (cont.) Since the intersection of two panes is a ine, there is a ine in image #2 on which the matching point must ie. his is caed the epipoar ine. If you know the vrp and prp of both cameras, you can compute the epipoar ine for any point in image #1. If axes are parae and B z =0, then the epipoar ines are scan ines. he Essentia Matrix (E) aows you to compute epipoar geometry without knowing the camera parameters a priori

11 Getting Forma about Stereo Do not panic about the next N sides; my goa is just to expose you to terms & concepts in case you go to a vision conference Epipoar Line P P P r p Epipoar Pane p r O e e r O r

12 Basic Equations P r = R P ( ) 1:Reation between 3D views of point P P 2:Norma to epipoar pane ( P ) ( P ) = 0 3:Panarity constraint ( R P ) r ( P ) = 0 4:Rewrite of #3, using #1

13 A Cever Equation P = SP You can rewrite a cross product as dot product, so!!! " # $ $ $ % & = x y x z y z S where

14 More Equations ( R P ) r SP = 0 5: Substitute dot for cross in #4 P RSP R P EP R = 0 = 0 6: Appy transpose equivaency 7: Let RS = E E is caed the Essentia Matrix. It is rank 2 (because of S), and shows a inear reationship between the projections of points in two images

15 Or in 2D. P Z f p = 8: Definition of perspective p f Z P = 9: same = 0!! " # $ $ % &!! " # $ $ % & R r r r p f Z E p f Z 10: rewrite of #7, with #8 = 0 R Ep p 11: drop non-zero constants

16 Back to Epipoar So E is a inear reation between p and p r u r =Ep, where u r is the ine of points in R that might match point p If you know E For every image point p : cacuate the ine u r ony cross-correate aong that ine E can be cacuated from 8 image correspondences Why 8? (How many DOF? How many constraints per correspondence?)

17 Stereo Practicum he arger the baseine, the more the perspective distortion he harder it is to match points he smaer the baseine, the smaer the ange between P and P r, the higher the reconstruction error. Errors aways highest in Z