MAPI Computer Vision

Transcription

1 MAPI Computer Vision Multiple View Geometry In tis module we intend to present several tecniques in te domain of te 3D vision Manuel Joao University of Mino Dep Industrial Electronics

2 - Applications - Camera Models - Projective transformations - Vanising points and vanising lines - Calibration Metods Intrinsic Camera Parameters Etrinsic Parameters - Geometry of Multiple Views 2 Camera Geometry Epipolar Geometry Fundamental matri F Stereo Vision 3 Camera Geometry Epipolar Geometry - Active Vision - Background subtraction and optical flow

3 Te material used is based on te following books: Te first is especially dedicated to tese specific issues, and te second presents a wide range of computer vision tecniques

4 Multiple View Geometry - applications - Automation Vision inspection systems In several computer vision applications we ave to transform te image coordinates to world coordinates, even in applications using only 2D information.

5 Robotics Multiple View Geometry - applications - Manipulator - PUMA Mobile robots Robotics is anoter field were te transformation from image coordinates to world coordinates is also mandatory. To perform grasping and tracking te robots must know te position of te objects, visualized by te vision system, in teir own coordinates. So a set of transformation between frames must be performed.

6 Surveillance Multiple View Geometry - applications - Human motion capture Measurement Evaluate te posture Traffic applications

7 Multiple View Geometry Computer Grapics - applications - Planar panoramic Mosaicing 3D reconstruction Also require information in world coordinates acquired troug visual information ttp://potosynt.net/default.asp

8 Multiple View Geometry - applications - Augmented Reality ARToolKit

9 Pinole cameras One of te first step to determine te relation between te image coordinates and te world coordinates is te definition of a camera model. Abstract camera model Bo wit a small ole in it Note inverted image

10 Equation of projection Tis equation presents te relation between te piels coordinates and te world coordinates based on te pinole model

11 Some Qualitative Properties Forward projection Can project curves by projecting teir points Wen project surfaces or solids, some points may not be visible. Straigt lines project to straigt lines, circles project to ellipses Angles are not preserved, in general Back projection Given an image point, only an orientation for corresponding 3-D point can be determined Given an image line, corresponding line must be on a specific plane

12 Some Qualitative Properties Projections of parallel lines are not parallel Tey meet in a common point called te vanising point Te vanising point depends on te direction of te set of parallel lines Sets of parallel lines on te same plane lead to collinear vanising points. Te line is called te orizon for tat plane

13 Coordinate Systems Previous transformation matri requires object coordinates to be epressed in te camera coordinate system (wit origin at lens center) Tis, in general, is not very convenient Object coordinate system Aligned wit some components of te object te tree sides of a rectangular solid World coordinate system Cosen for global convenience, e.g. lines forming corner of a room, or eart coordinates (latitude, longitude, eigt) Coordinate transformations define relations between different coordinate systems

14 Projective transformations 2D Definition A planar projective transformation is a linear transformation on omogeneous 3-vectors represented by a non-singular 33 matri = H =

15 Projective transformations

16 Projective transformations A projectivity is an invertible mapping from points in IP 2 (tat is omogeneous 3-vectors) to points in IP 2 tat maps lines to lines. Projectivities form a group since te inverse of a projectivity is also a projectivity A projectivity is also called a collineation a projective transformation or a omograpy A mapping : IP 2 IP 2 is a projectivity if and only if tere eists a non-singular 33 matri H suc tat for any point in IP 2 represented by a vector it is true tat () = H.

17 Projective transformations Removing te projective distortion from a perspective image of a plane Sape is distorted under te perspective imaging Parallel lines on a scene plane are not parallel in te image but instead converge to a finite point

18 Projective transformations Removing te projective distortion from a perspective image of a plane Tis can be written in inomogeneous form = y y = = y y y = =

19 Projective transformations Removing te projective distortion from a perspective image of a plane Eac point correspondence generates two equations for te elements of H. Tese equations are linear in te elements of H Four points correspondence lead to eigt suc linear equations in te entries of H Sufficient to solve H

20 Projective transformations Removing te projective distortion from a perspective image of a plane

21 EXE Projective transformations Homograpic relation between images Applications eamples 4 points Syntetic views b) Fronto-parallel view of te corridor floor generated from a) using te four corners of te floor tile to compute te omograpy c) Fronto-parallel view of te corridor wall generated from a) using te four corners of te door frame to compute te omograpy

22 Projective transformations Removing te projective distortion from a perspective image of a plane Te computation of te rectifying transformation H in tis way does not require knowledge of any of te camera's parameters or te pose of te plane If eactly four correspondences are given, ten an eact solution for te matri H is possible. Tis is te minimal solution. Points are measured ineactly ("noise"), if more tan four suc correspondences are given, ten tese correspondences may not be fully compatible wit any projective transformation, determining te "best" transformation given te data. Tis will generally be done by finding te transformation H tat minimizes some cost function. robust estimation algoritms» RANSAC RANdom SAmple Consensus

23 Projective transformations RANSAC - RANdom SAmple Consensus Objective Compute te 2D omograpy between two images. Algoritm (i) Interest points: Compute interest points in eac image. (ii) Putative correspondences: Compute a set of interest point matces based on proimity and similarity of teir intensity neigbourood. (iii) RANSAC robust estimation: Repeat for N samples (a) Select a random sample of 4 correspondences and compute te omograpy H. (b) Calculate te distance d for eac putative correspondence. (c) Compute te number of inliers consistent wit H by te number of correspondences for wic d < t = sqrt(5.99)σ piels. Coose te H wit te largest number of inliers. In te case of ties coose te solution tat as te lowest standard deviation of inliers. (iv) Optimal estimation: re-estimate H from all correspondences classified as inliers, by minimizing te ML (Maimum Likeliood) cost function (v) Guided matcing: Furter interest point correspondences are now determined using te estimated H to define a searc region about te transferred point position. Te last two steps can be iterated until te number of correspondences is stable.

24 Projective transformations RANSAC Application domain Scenes sould be ligtly tetured Te searc window proimity constraint places an upper limit on te image motion of corners (te disparity) between views Smaller searc window means tat fewer corner matces must be evaluated Robust estimation Moderate immunity Independent motion Canges in sadows Partial occlusions,

25 Projective transformations Homograpic relation between images Applications eamples Planar panoramic mosaicing Tree images acquired by a rotating camera may be registered to te frame of te middle one by projectively warping te outer images to align wit te middle one.» Given 4 points» More points - ransac

26 Projective transformations Homograpic relation between images Applications eamples Planar panoramic mosaicing Eigt images acquired by rotating a camcorder about its centre

27 Projective transformations Homograpic relation between images Applications eamples Planar panoramic mosaicing Algoritm: (i) Coose one image of te set as a reference. (ii) Compute te omograpy H wic maps one of te oter images of te set to tis reference image. (iii) Projectively warp te image wit tis omograpy, and augment te reference image wit te non-overlapping part of te warped image. (iv) Repeat te last two steps for te remaining images of te set.

28 Vanising points and vanising lines One of te distinguising features of perspective projection is tat te image of an object tat stretces off to infinity can ave finite etent. An infinite scene line is imaged as a line terminating in a vanising point. Similarly, parallel world lines, suc as railway lines, are imaged as converging lines, and teir image intersection is te vanising point for te direction of te railway.

29 Vanising points and EXE vanising lines Vanising point estimation involves fitting a line (sown tin ere) troug v to eac measured line (sown tick ere). maimum likeliood (ML) estimate E. Houg transform Te maimum likeliood (ML) estimate of v is te point wic minimizes te sum of squared ortogonal distances between te fitted lines and te measured lines' end points

30 Vanising points and vanising lines Vanising lines Parallel planes in 3-space intersect π in a common line, and te image of tis line is te vanising line of te plane. It is clear tat a vanising line depends only on te orientation of te scene plane; it does not depend on its position. Since lines parallel to a plane intersect te plane at π, it is easily seen tat te vanising point of a line parallel to a plane lies on te vanising line of te plane.

31 Vanising points and Vanising lines vanising lines

32 Vanising points and EXE Vanising lines vanising lines

33 Vanising points and vanising lines Affine 3D measurements and reconstruction Given: te vanising line of te ground plane l te vertical vanising point v, ten te relative lengt of vertical line segments can be measured provided teir end point lies on te ground plane.

34 Vanising points and vanising lines Affine 3D measurements and reconstruction world geometry image geometry

35 Vanising points and vanising lines Affine 3D measurements and reconstruction