Basic principles - Geometry. Marc Pollefeys

Transcription

1 Basic principles - Geometr Marc Pollees

2 Basic principles - Geometr Projective geometr Projective, Aine, Homograph Pinhole camera model and triangulation Epipolar geometr Essential and Fundamental Matri (8-point algorithm) Absolute pose problem Linear pose, P3P Robust estimation RANSAC Model selection H vs. E vs. F (GRIC, QDEGSAC, etc.)

3 D points and lines Homogeneous representations a + b + c = ( )(,, ) T = ( ) T ( ) T a,b,c ~ λ a,b,c," λ ¹ a,b,c (,,) ~ µ (,,)," µ ¹ Join o points and lines = l l' l = ' l T = l l l l T T

4 Ideal points and the line at ininit Intersection o parallel lines l = l l' = ( a, b, c) T and l' = ( a, b, c' ) T ( b, -a,) T Ideal points (,, ) T Line at ininit ( ) T l =,, l l' l l' P = R Èl Note that in P there is no distinction between ideal points and others

5 Dualit o points and lines in P Homogeneous representation identical Equations smmetric l T = = l l' l = '

6 D projective transormations Deinition: A projectivit is an invertible mapping h rom P to itsel such that three points,, 3 lie on the same line i and onl i h( ),h( ),h( 3 ) do. Theorem: A mapping h:p P is a projectivit i and onl i there eist a non-singular 33 matri H such that or an point in P reprented b a vector it is true that h()=h Deinition: Projective transormation æ ' ö éh = ' h è ' ø 3 ëh h h h h h h 3 3 ùæ ûè ö ø or ' = H DOF projectivit=collineation=projective transormation=homograph

7 Transormation o D points and lines For a point transormation ' = Transormation or lines l'= H -T H l Notice: l # = l # H '( H = l #

8 Hierarch o D transormations Projective 8do Aine 6do Similarit 4do Euclidean 3do éh h ë h éa a ë ésr sr ë ér r ë 3 h h h a a r r 3 sr sr h3ù h 3 h 33û t ù t û t ù t û t ù t û transormed squares invariants Concurrenc, collinearit, order o contact (intersection, tangenc, inlection, etc.), cross ratio Parallellism, ratio o areas, ratio o lengths on parallel lines (e.g midpoints), linear combinations o vectors (centroids). The line at ininit l Ratios o lengths, angles. The circular points I,J lengths, areas.

9 Homograph application : Planar rectiication (notice uncertaint or transormed point depends on which points used or computation o homograph)

10 Homograph application : Panoramic images pure rotation

11 3D points, lines and planes Representation o points and planes AX + BY + CZ + D = P T X = Representation o lines Join o two points Intersection o two planes Dualit: points and planes, lines and lines λx + µx = λ P + µ P =

12 Projective transormations Projective transormations X = HX Aine transormations X = é ë A X Similarit transormations X b ù û plane at ininit ied = éσr tù X ë û absolute conic ied

13 Camera model Relation between piels and ras in space (i.e. between D image and 3D world)?

14 Pinhole camera

15 The pinhole camera z= e object point (,) e z e o image plane ocal point camera P=(X,Y,Z) Uniorm triangles gives:

16 Projection matri Include coordinate transormation and camera intrinsic parameters [ ] û ù ë é - û ù ë é = û ù ë é γ λ Z Y X t R R s T T [ ]X t R R K T T - = λ λ = PX

17 Pinhole camera model T T Z Y Z X Z Y X ) /, / ( ),, (! ø ö è æ û ù ë é = ø ö è æ ø ö è æ Z Y X Z Y X Z Y X! linear projection in homogeneous coordinates!

18 Pinhole camera model ø ö è æ û ù ë é = ø ö è æ Z Y X Z Y X ø ö è æ û ù ë é û ù ë é = ø ö è æ Z Y X Z Y X = PX [ ] I,), diag( P =

19 Principal point oset T T p Z Y p Z X Z Y X ) + /, + / ( ),, (! principal point T p p ), ( ø ö è æ û ù ë é = ø ö è æ + + ø ö è æ Z Y X p p Z Zp Y Zp X Z Y X!

20 Principal point oset ø ö è æ û ù ë é = ø ö è æ + + Z Y X p p Z Zp Y Zp X [ ] cam X = K I û ù ë é = p p K calibration matri

21 Camera rotation and translation = R( X ~ -C ~ ) X ~ cam - RC ~ æ X ö ~ ér ù Y ér - RCù X cam = = X ë û Z ë û [ ] è ø = KRK I [ I -C X ~ cam ]X = PX P = K[ R t] t = -RC ~

22 CCD camera û ù ë é = p p K a a û ù ë é û ù ë é = p p m m K

23 General projective camera û ù ë é = p p s K a a û ù ë é = p p K a a [ ] C ~ P = KR I non-singular do (5+3+3) [ ] t P = K R intrinsic camera parameters etrinsic camera parameters

24 Action o projective camera on points and lines projection o point = PX orward projection o line ( ) = P(A+µB) = PA +µpb= a + µb X µ back-projection o line Π = P T l P T X = l T PX ( l T = ; = PX)

25 Radial distortion Due to spherical lenses (cheap) Model: R R é (, ) ( K( ) K( )...) ù = ë û straight lines are not straight anmore

26 Linear camera pose estimation X ( X X Given D and 3D points, compute the camera pose λ = PX λ = P ( P P X P X= P ( X P X= P X P? X 3 X 3 X 3 X 3 ( P ( 3 P 3 P X 5 X 6 X 7

27 Three points perspective pose p3p (Haralick et al., IJCV94) All techniques ield 4 th order polnomial Haralick et al. recommends using Finsterwalder s technique as it ields the best results numericall 84 93

28 3D rom images L C m M? L m C l Triangulation - calibration - correspondences

29 Triangulation L C X L Triangulation - calibration - correspondences C

30 Triangulation Backprojection C L X L C Triangulation Maimum Likelihood Triangulation Iterative least-squares

31 Optimal 3D point in epipolar plane Given an epipolar plane, ind best 3D point or (m,m ) m l m l m l m m m l Select closest points (m,m ) on epipolar lines Obtain 3D point through eact triangulation Guarantees minimal reprojection error (given this epipolar plane)

32 Non-iterative optimal solution Reconstruct matches in projective rame b minimizingthe reprojection error ( ) m, PM D( m P M) D +, 3DOF Non-iterative method Determine the epipolar plane or reconstruction ( ( )) m, l a + D( m l ( a) ) (Hartle and Sturm, CVIU 97) D, (polnomial o degree 6) l (a) m m l (a) DOF Reconstruct optimal point rom selected epipolar plane Note: onl works or two views

33 The epipoles The epipole is the projection o the ocal point o one camera in another image. e C e C Image Image P P A - b = [ A ] b [ ] PC = A b = = [ A ] b ë û - é- A [ ] b ù - P C = A b = b - A A b é- ë ù û

34 3D rom images L C X L l C

35 Epipolar geometr l = e C P M L P = P = l Pl L l l T l e + T P F[e T P ] = = Fundamental matri (33 rank matri) e C l T l = l + = P P l

36 F-matri computation 8 point For ever match (m,m ): é [ [ ] T = ] = m Fm = ë ùéù û ë û é ë ù û

37 = û ù ë é û ù ë é n n n n n n n n n n n n!!!!!!!!! Stack data rows Þ Linear sstem o equations F-matri computation 8 point ~ ~ ~ ~ ~ ~ ~ ~! Orders o magnitude dierence Between column o data matri least-squares ields poor results Need to normalized (Hartle PAMI 97)

38 F-matri computation 7 point Linear equations ield parameter amil o solutions F +lf s 3 F 7pts F F F obtain or 3 solutions 3 det( F + lf ) = a3l + al + al + a = (cubic equation) 38

39 Essential matri For calibrated camera and normalized image coordinates the undamental matri becomes the essential matri E = R t ; The essential matri allows to directl derive R and t (up to scale)

40 E-matri computation 5 point (Nister CVPR 3) Linear equations ield 3 parameter amil o solutions E +l E +l 3 E 3 +l 4 E 4 9 cubic constraint on E Eventuall ields degree polnomial to solve Can deal with planar data Requires calibration 4

41 E-matri computation 5 point (Nister, CVPR3) Linear equations or 5 points " $ $ $ $ $ $ # $!!!!!!!!!!!! 3! 3 3! 3 3! 3! 3 3! 3 3! 3 3! 4 4! 4 4! 4! 4 4! 4 4! 4 4 4! 5 5! 5 5! 5! 5 5! 5 5! Linear solution space E = X + Y + zz + ww " $ $ % $ ' $ ' $ ' $ ' $ ' $ ' $ &' $ $ $ $ # E E E 3 E E E 3 E 3 E 3 E 33 % ' ' ' ' ' ' ' = ' ' ' ' ' ' & scale does not matter, choose w = Non-linear constraints det( E) = EE T E tr( EE T )E = cubic polnomials

42 E-matri computation 5 point (Nister, CVPR3) Perorm Gauss-Jordan elimination on polnomials [n]represents polnomial o degree n in z -z -z -z

43 Minimal relative pose with know vertical Fraundorer, Tanskanen and Pollees, ECCV -g 43 Vertical direction can oten be estimated inertial sensor vanishing point 5 linear unknowns linear 5 point algorithm 3 unknowns quartic 3 point algorithm

44 robust estimation (RANSAC) To avoid outliers, generate hpothesis with as ew data as possible Keep doing until succesul e.g. line itting

45 Robust F-matri computation (RANSAC) Step. Etract eatures Step. Compute a set o potential matches Step 3. do Step 3. select minimal sample (i.e. 7 matches) Step 3. compute solution(s) or F Step 3.3 determine inliers (veri hpothesis) until G(#inliers,#samples)<95% Step 4. Compute F based on all inliers Step 5. Look or additional matches Step 6. Reine F based on all correct matches } (generate hpothesis) G = - ( - ( ) #inliers 7 # samples ) #matches #inliers 9% 8% 7% 6% 5% #samples Some recent work: QDEGSAC (Frahm and Pollees CVPR6), ARRSAC (Raguram et al.eccv8), RANSAC with uncertaint (Raguram et al ICCV 9), USAC (Raguram et al TPAMI)

46 F-matri non-linear reinement Minimize geometric instead o algebraic error ( ( ) ( ) ) T m, Fm D m, F m minå D + «min å T m Fm F F T m m Fm m F Meaningull error in image space ( ( ) ( ) ) D m, Fmˆ D m, mˆ minå + F, mˆ mˆ m m Fmˆ Maimum Likelihood Estimastion (given Gaussian noise error model) (Initialize with linear algorithm; Use e.g. Levenberg-Marquardt) 46

47 Finding more matches or F-matri restrict search range to neighborhood o epipolar line (±.5 piels) rela disparit restriction (along epipolar line) 47

48 Computed F-matri & matches geometric relations between two views is ull described b recovered 33 matri F 48

49 Computing F or quasi-planar scenes QDEGSAC (Frahm & Pollees CVPR6) 337 matches on plane, o plane #inliers %inclusion o out-o-plane inliers data rank 7% success or RANSAC % or QDEGSAC

50 Ke-rame selection Select ke-rame when F ields a better model than H Use Robust Geometric Inormation Criterion (Torr 98) bad it penalt model compleit Given view i as a ke-rame, pick view j as net ke-rame or irst view where GRIC(F ij )>GRIC(H ij ) (or a ew views later) H-GRIC F-GRIC 5 (Pollees et al. )

51 Dealing with dominant planar scenes USaM ails when common eatures are all in a plane Solution: part Model selection to detect problem (Pollees et al., ECCV ) 5

52 Dealing with dominant planar scenes USaM ails when common eatures are all in a plane Solution: part Dela ambiguous computations until ater sel-calibration (couple sel-calibration over all 3D parts) (Pollees et al., ECCV ) 5

53 Questions?