Epipolar Geometry CSE P576. Dr. Matthew Brown

Transcription

1 Epipolar Geometry CSE P576 Dr. Matthew Brown

2 Epipolar Geometry Epipolar Lines, Plane Constraint Fundamental Matrix, Linear solution + RANSAC Applications: Structure from Motion, Stereo [ Szeliski 11] 2

3 2-view Geometry How do we transfer points between 2 views? (u 1,v 1 ) (u 2,v 2 )? 3

4 X<latexit sha1_base64="nietpjtpxuzhajhprtgjay03bgk=">aaab7hicbvbns8naej3ur1q/qh69lbbbu0leqmeif48vtftoq9lsj+3szsbsboqs+hu8efdeqz/im//gbzudtj4yelw3w8y8mbvcg9f9dkobm1vbo+xdyt7+wefr9fikrznmmfrzihlvdalgwsx6hhub3vqhjuobnxbyn/c7t6g0t+sjmayyxhqkecqznvby+2feuonqza27c5b14hwkbgvag+pxf5iwlezpmkba9zw3nufolefm4kzszzsmle3ochuwshqjdvlfstnyyzuhirjlsxqyuh9p5dtwehqhtjomzqxxvbn4n9flthqt5fymmuhjlouitbctkpnnzmgvmiomllcmul2vsdfvlbmbt8wg4k2+ve7av3xprxsp17xmbrfhgc7ghc7bgwy04r5a4amdds/wcm+odf6cd+dj2vpyiplt+apn8wc4fi5i</latexit> <latexit sha1_base64="nietpjtpxuzhajhprtgjay03bgk=">aaab7hicbvbns8naej3ur1q/qh69lbbbu0leqmeif48vtftoq9lsj+3szsbsboqs+hu8efdeqz/im//gbzudtj4yelw3w8y8mbvcg9f9dkobm1vbo+xdyt7+wefr9fikrznmmfrzihlvdalgwsx6hhub3vqhjuobnxbyn/c7t6g0t+sjmayyxhqkecqznvby+2feuonqza27c5b14hwkbgvag+pxf5iwlezpmkba9zw3nufolefm4kzszzsmle3ochuwshqjdvlfstnyyzuhirjlsxqyuh9p5dtwehqhtjomzqxxvbn4n9flthqt5fymmuhjlouitbctkpnnzmgvmiomllcmul2vsdfvlbmbt8wg4k2+ve7av3xprxsp17xmbrfhgc7ghc7bgwy04r5a4amdds/wcm+odf6cd+dj2vpyiplt+apn8wc4fi5i</latexit> <latexit sha1_base64="nietpjtpxuzhajhprtgjay03bgk=">aaab7hicbvbns8naej3ur1q/qh69lbbbu0leqmeif48vtftoq9lsj+3szsbsboqs+hu8efdeqz/im//gbzudtj4yelw3w8y8mbvcg9f9dkobm1vbo+xdyt7+wefr9fikrznmmfrzihlvdalgwsx6hhub3vqhjuobnxbyn/c7t6g0t+sjmayyxhqkecqznvby+2feuonqza27c5b14hwkbgvag+pxf5iwlezpmkba9zw3nufolefm4kzszzsmle3ochuwshqjdvlfstnyyzuhirjlsxqyuh9p5dtwehqhtjomzqxxvbn4n9flthqt5fymmuhjlouitbctkpnnzmgvmiomllcmul2vsdfvlbmbt8wg4k2+ve7av3xprxsp17xmbrfhgc7ghc7bgwy04r5a4amdds/wcm+odf6cd+dj2vpyiplt+apn8wc4fi5i</latexit> <latexit sha1_base64="nietpjtpxuzhajhprtgjay03bgk=">aaab7hicbvbns8naej3ur1q/qh69lbbbu0leqmeif48vtftoq9lsj+3szsbsboqs+hu8efdeqz/im//gbzudtj4yelw3w8y8mbvcg9f9dkobm1vbo+xdyt7+wefr9fikrznmmfrzihlvdalgwsx6hhub3vqhjuobnxbyn/c7t6g0t+sjmayyxhqkecqznvby+2feuonqza27c5b14hwkbgvag+pxf5iwlezpmkba9zw3nufolefm4kzszzsmle3ochuwshqjdvlfstnyyzuhirjlsxqyuh9p5dtwehqhtjomzqxxvbn4n9flthqt5fymmuhjlouitbctkpnnzmgvmiomllcmul2vsdfvlbmbt8wg4k2+ve7av3xprxsp17xmbrfhgc7ghc7bgwy04r5a4amdds/wcm+odf6cd+dj2vpyiplt+apn8wc4fi5i</latexit> 2-view Geometry How do we transfer points between 2 views? (planar case) (u 1,v 1 ) (u 2,v 2 )? Planar case: one-to-one mapping via plane 4

5 2-view Geometry How do we transfer points between 2 views? (non-planar) X? X? X? (u 1,v 1 ) (u 2,v 2 )? Non-planar case: depends on the depth of the 3D point 5

6 Epipolar Line How do we transfer points between 2 views? (non-planar) X? X? X? (u 1,v 1 ) (u 2,v 2 )? A point in image 1 gives a line in image 2 6

7 Epipolar Lines [ R. Cipolla ] 7

8 Epipolar Lines [ R. Cipolla ] 8

9 The Epipolar Constraint For rays to intersect at a point (X), the two rays and the camera translation must lie in the same plane

10 Epipolar Geometry Example: 2-view matching in 3D 10

11 Raw SIFT matches Epipolar Geometry 11

12 Epipolar lines Epipolar Geometry? Can use RANSAC to find inliers with small distance from epipolar line 12

13 Consistent matches Epipolar Geometry 13

14 <latexit sha1_base64="3myafgfbz85tqxrlykm907s7qug=">aaacs3ichzfnt9wweiadul7sqhc4crg6kuppfsducefc7avhkngfabmkjjnzlbwnsh3eksof5nhb/02dsbflqepihr2az0yznklkwbxx/t+ou/zhfwnza9v7+gln9/ngb3+ii0oxglncfoomoroelza23ai4krxqpbfwndz/bpn1ayjnc3llfixmcjqxpoomghukb09elmccyzrjqvh8sfgqmoaj/nb5tlwizpocv5ozucakszmtchornlhuoqbhqy+chou9cnsuhs1/mj3gxbyvor862jnnx4ohp/i7w28fwyohwtpfppgdpqwrcpcgcar1lpilmdvugc4e2p6vhpkyezqhqzws5qbndbfzbn+1krrnhbjpgtxf/62oaa71ik9spp3xtq+ynvgem1ymo53vxjavacmeg2wvwkba7qfxyhuwixzwuka4nrwzo6oom/bmnl0cwf3ywzejrsqfkcvj4fmp5tq20ch6gr4hgk7qofqfltaymcd3jk7s3lrf3ambuolzqussaw7qk3pzv55szeg=</latexit> <latexit sha1_base64="3myafgfbz85tqxrlykm907s7qug=">aaacs3ichzfnt9wweiadul7sqhc4crg6kuppfsducefc7avhkngfabmkjjnzlbwnsh3eksof5nhb/02dsbflqepihr2az0yznklkwbxx/t+ou/zhfwnza9v7+gln9/ngb3+ii0oxglncfoomoroelza23ai4krxqpbfwndz/bpn1ayjnc3llfixmcjqxpoomghukb09elmccyzrjqvh8sfgqmoaj/nb5tlwizpocv5ozucakszmtchornlhuoqbhqy+chou9cnsuhs1/mj3gxbyvor862jnnx4ohp/i7w28fwyohwtpfppgdpqwrcpcgcar1lpilmdvugc4e2p6vhpkyezqhqzws5qbndbfzbn+1krrnhbjpgtxf/62oaa71ik9spp3xtq+ynvgem1ymo53vxjavacmeg2wvwkba7qfxyhuwixzwuka4nrwzo6oom/bmnl0cwf3ywzejrsqfkcvj4fmp5tq20ch6gr4hgk7qofqfltaymcd3jk7s3lrf3ambuolzqussaw7qk3pzv55szeg=</latexit> <latexit sha1_base64="3myafgfbz85tqxrlykm907s7qug=">aaacs3ichzfnt9wweiadul7sqhc4crg6kuppfsducefc7avhkngfabmkjjnzlbwnsh3eksof5nhb/02dsbflqepihr2az0yznklkwbxx/t+ou/zhfwnza9v7+gln9/ngb3+ii0oxglncfoomoroelza23ai4krxqpbfwndz/bpn1ayjnc3llfixmcjqxpoomghukb09elmccyzrjqvh8sfgqmoaj/nb5tlwizpocv5ozucakszmtchornlhuoqbhqy+chou9cnsuhs1/mj3gxbyvor862jnnx4ohp/i7w28fwyohwtpfppgdpqwrcpcgcar1lpilmdvugc4e2p6vhpkyezqhqzws5qbndbfzbn+1krrnhbjpgtxf/62oaa71ik9spp3xtq+ynvgem1ymo53vxjavacmeg2wvwkba7qfxyhuwixzwuka4nrwzo6oom/bmnl0cwf3ywzejrsqfkcvj4fmp5tq20ch6gr4hgk7qofqfltaymcd3jk7s3lrf3ambuolzqussaw7qk3pzv55szeg=</latexit> <latexit sha1_base64="3myafgfbz85tqxrlykm907s7qug=">aaacs3ichzfnt9wweiadul7sqhc4crg6kuppfsducefc7avhkngfabmkjjnzlbwnsh3eksof5nhb/02dsbflqepihr2az0yznklkwbxx/t+ou/zhfwnza9v7+gln9/ngb3+ii0oxglncfoomoroelza23ai4krxqpbfwndz/bpn1ayjnc3llfixmcjqxpoomghukb09elmccyzrjqvh8sfgqmoaj/nb5tlwizpocv5ozucakszmtchornlhuoqbhqy+chou9cnsuhs1/mj3gxbyvor862jnnx4ohp/i7w28fwyohwtpfppgdpqwrcpcgcar1lpilmdvugc4e2p6vhpkyezqhqzws5qbndbfzbn+1krrnhbjpgtxf/62oaa71ik9spp3xtq+ynvgem1ymo53vxjavacmeg2wvwkba7qfxyhuwixzwuka4nrwzo6oom/bmnl0cwf3ywzejrsqfkcvj4fmp5tq20ch6gr4hgk7qofqfltaymcd3jk7s3lrf3ambuolzqussaw7qk3pzv55szeg=</latexit> Computing F Single correspondence gives us one equation u1 v 1 1 f 11 f 12 f 13 4f 21 f 22 f f 31 f 32 f 33 x 1 y =0 Multiply out u 1 x 1 f 11 + u 1 y 1 f 12 + u 1 f 13 + v 1 x 1 f 21 + v 1 y 1 f 22 +v 1 f 23 + x 1 f 31 + y 1 f 32 + f 33 =0 14

15 Computing F Rearrange for unknowns, add points by stacking rows [ u 1 x 1 u 1 y 1 u 1 v 1 x 1 v 1 y 1 v 1 x 1 y 1 1] [ u 2 x 2 u 2 y 2 u 2 v 2 x 2 v 2 y 2 v 2 x 2 y 2 1] [ u 3 x 3 u 3 y 3 u 3 v 3 x 3 v 3 y 3 v 3 x 3 y 3 1] [ u 4 x 4 u 4 y 4 u 4 v 4 x 4 v 4 y 4 v 4 x 4 y 4 1] [ u 5 x 5 u 5 y 5 u 5 v 5 x 5 v 5 y 5 v 5 x 5 y 5 1] [ u 6 x 6 u 6 y 6 u 6 v 6 x 6 v 6 y 6 v 6 x 6 y 6 1] [ u 7 x 7 u 7 y 7 u 7 v 7 x 7 v 7 y 7 v 7 x 7 y 7 1] [ u 8 x 8 u 8 y 8 u 8 v 8 x 8 v 8 y 8 v 8 x 8 y 8 1] f 11 f 12 f 13 f 21 f 22 f 23 f 31 f 32 f = This is a linear system of the form Af = 0 can be solved using Singular Value Decomposition (SVD) 15

16 Pairwise Matching Find feature matches and use RANSAC to find inliers to F 16

17 Structure from Motion Given an (unordered) set of input images, compute camera positions and 3D structure of the scene 17

18 Sparse Reconstruction 18

19 Dense Reconstruction Use multiview stereo to compute dense depth (search along epipolar lines for matches) 19

20 Stereo Matching Stereo cameras are typically arranged by horizontal translation so epipolar lines (thus point matches) are horizontal (a) (b) (c) Stereo algorithms attempt to match along scanlines to find depth [ D. Scharstein ] 20

21 Disparity and Depth [ D. Scharstein ] 21

22 Stereo Images Image pair with depth dependent disparity 22

23 3D TV Most common use field sequential (shutter) glasses Transmit alternate left/right image at 120Hz Glasses free 3D systems send different images directly to each eye 23

24 Recap: 2-view Geometry Planar geometry: one to one mapping of points u = Hx viewing a plane, rotation full 3D 24

25 Recap: 2-view Geometry Epipolar geometry: point to line mapping u T Fx =0 moving camera, 3D scene 25

26 Recap: RANSAC Robust estimation of H/F and consistent matches Raw feature matches RANSAC for F 26

27 Next Lecture Multiview geometry + optimization 27