EECS 442 Computer vision. Multiple view geometry Affine structure from Motion

Transcription

1 EECS 442 Computer visio Multiple view geometry Affie structure from Motio - Affie structure from motio problem - Algebraic methods - Factorizatio methods Readig: [HZ] Chapters: 6,4,8 [FP] Chapter: 2 Some slides of this lectures are courtesy of prof. J. Poce, prof FF Li, prof S. Lazebik & prof. M. Hebert

2 Applicatios Courtesy of Oford Visual Geometry Group

3 Structure from motio problem X j j M m M mj 2j M 2 Give m images of fied 3D poits ij = M i X j, i =,, m, j =,,

4 Structure from motio problem X j j M m M mj 2j M 2 From the m correspodeces ij, estimate: m projectio matrices M i 3D poits X j motio structure

5 Affie structure from motio (simpler problem) Image World Image From the m correspodeces ij, estimate: m projectio matrices M i (affie cameras) 3D poits X j

6 Fiite cameras p q r O Q R K R M TX P

7 Questio: R T??

8 Fiite cameras p q r R Q P O X T K R T R K M 3 3 Caoical perspective projectio matri Affie homography (i 3D) Affie Homography (i 2D) y s K o y o

9 Projective & Affie cameras X T K R T R K M y s K o y o Projective case Affie case

10 Weak perspective projectio Whe the relative scee depth is small compared to its distace from the camera ' m y' my Scalig fuctio of the distace (magificatio)

11 Orthographic (affie) projectio Whe the camera is at a (roughly costat) distace from the scee ' y' y Distace from ceter of projectio to image plae is ifiite

12 Trasformatio i 2D Affiities: y H y t A y' ' a

13 Projective & Affie cameras X T K R y α s α o y o K T R K M T R K M y s K o y o Projective case Affie case Parallel projectio matri (poits at ifiity are mapped as poits at ifiity) Magificatio (scalig term)

14 Affie cameras X T K R y K T R K M b A 4affie] [4 3affie] 3 [ b a a a b a a a M X b AX Euc M b b Z Y X a a a a a a y [Homogeeous] [o-homogeeous image coordiates] b A M M Euc ; P M Euc

15 Affie cameras p P p M = camera matri To recap: from ow o we defie M as the camera matri for the affie case p u v AP b M P ; M A b

16 The Affie Structure-from-Motio Problem Give m images of fied poits P j (=X i ) we ca write N of cameras N of poits Problem: estimate the m 24 matrices M i ad the positios P j from the m correspodeces p ij. How may equatios ad how may ukow? 2m equatios i 8m+3 ukows Two approaches: - Algebraic approach (affie epipolar geometry; estimate F; cameras; poits) - Factorizatio method

17 Algebraic aalysis (2-view case) - Derive the fudametal matri F A for the affie case - Compute F A - Use F A to estimate projectio matrices - Use projectio matrices to estimate 3D poits

18 . Derivig the fudametal matri F A p P v p u Homogeeous system Dim=? 44

19 Derivig the fudametal matri F A where The Affie Fudametal Matri! Are the epipolar lies parallel or covergig?

20

21 Affie Epipolar Geometry

22 Estimatig F A From correspodeces, we obtai a liear system o the ukow alpha, beta, etc Measuremets: u, u, v, v v u v u v u v u f Computed by least square ad by eforcig f = SVD

23 Estimatig projectio matrices from F A p P p

24 Affie ambiguity Affie p M P M Q - Q P A A

25 Estimatig projectio matrices from F A p P p

26 Estimatig projectio matrices from F A Choose Q such that ~ M ' Where a,b,c,d ca be epressed as fuctio of the parameters of F A See HZ page 348

27 A factorizatio method Tomasi & Kaade algorithm C. Tomasi ad T. Kaade. Shape ad motio from image streams uder orthography: A factorizatio method. IJCV, 9(2):37-54, November 992. Ceterig the data Factorizatio

28 Ceterig: subtract the cetroid of the image poits j i k k j i k i k i i j i k ik ij ij A X X X A b A X b A X A factorizatio method - Ceterig the data X k ik i ^

29 Ceterig: subtract the cetroid of the image poits j i k k j i k i k i i j i k ik ij ij A X X X A b A X b A X A factorizatio method - Ceterig the data

30 Ceterig: subtract the cetroid of the image poits k k j i k i k i i j i k ik ij ij X X A b A X b A X j i ij X A A factorizatio method - Ceterig the data Assume that the origi of the world coordiate system is at the cetroid of the 3D poits After ceterig, each ormalized poit ij is related to the 3D poit X i by

31 A factorizatio method - Ceterig the data X A ij i X j

32 Let s create a 2m data (measuremet) matri: m m m D cameras (2 m ) poits ( ) A factorizatio method - factorizatio

33 Let s create a 2m data (measuremet) matri: m m m m X X X A A A D cameras (2 m 3) poits (3 ) The measuremet matri D = M S has rak 3 (it s a product of a 2m3 matri ad 3 matri) A factorizatio method - factorizatio (2 m ) M S

34 Factorizig the measuremet matri Source: M. Hebert

35 Factorizig the measuremet matri Sigular value decompositio of D: Source: M. Hebert

36 Factorizig the measuremet matri Sigular value decompositio of D: Sice rak (D)=3, there are oly 3 o-zero sigular values Source: M. Hebert

37 Factorizig the measuremet matri Obtaiig a factorizatio from SVD: S = structure M = Motio (cameras) What is the issue here? D has rak>3 because of - measuremet oise - affie approimatio

38 Factorizig the measuremet matri Obtaiig a factorizatio from SVD: M = motio S = structure D D

39 Affie ambiguity The decompositio is ot uique. We get the same D by usig ay 3 3 matri C ad applyig the trasformatios M MC, S C - S We ca eforce some Euclidea costraits to resolve this ambiguity (more o et lecture!)

40 Algorithm summary. Give: m images ad features ij 2. For each image i, ceter the feature coordiates 3. Costruct a 2m measuremet matri D: Colum j cotais the projectio of poit j i all views Row i cotais oe coordiate of the projectios of all the poits i image i 4. Factorize D: Compute SVD: D = U W V T Create U 3 by takig the first 3 colums of U Create V 3 by takig the first 3 colums of V Create W 3 by takig the upper left 3 3 block of W 5. Create the motio ad shape matrices: M = M = U 3 ad S = W 3 V 3 T (or U 3 W 3 ½ ad S = W 3 ½ V 3 T ) 6. Elimiate affie ambiguity

41 Recostructio results C. Tomasi ad T. Kaade. Shape ad motio from image streams uder orthography: A factorizatio method. IJCV, 9(2):37-54, November 992.

42 Net lecture Multiple view geometry Perspective structure from Motio