http://www.cs.unc.edu/~marc/ Multiple View Geometry in Computer Vision 2011.09.
2
3
4
5
Visual 3D Modeling from Images 6
AutoStitch http://cs.bath.ac.uk/brown/autostitch/autostitch.html 7
Image Composite Editor http://research.microsoft.com/en-us/um/ redmond/groups/ivm/ice/ 8
Deep Zoom & HD View http://research.microsoft.com/enus/um/redmond/groups/ivm/hdview/ 9
Photosynth http://photosynth.net/ 10
PhotoCity http://photocitygame.com/ 11
Projective Transformations
Homogeneous coordinates Homogeneous representation of lines ax by c 0 a,b,c T ka a,b,c T ~ ka,b,c T ( ) x ( kb) y kc 0, k 0 equivalence class of vectors, any vector is representative Set of all equivalence classes in R 3 (0,0,0) T forms P 2 Homogeneous representation of points x xy,,1 on l a,b,c T if and only if ax by c 0 T x,y, 1 a,b,c x,y, 1 l 0 T T x, y,1 ~ kx, y,1, k 0 The point x lies on the line l if and only if x T l=l T x=0 Homogeneous coordinates x, y, z T Inhomogeneous coordinates x, y T but only 2DOF 13
Points from lines and vice-versa Intersections of lines The intersection of two lines l and l' is x l l' Line joining two points The line through two points x and x' is l x x' Example y 1 x 1 14
Ideal points and the line at infinity Intersections of parallel lines l a, b, c T and l' a, b, c' T l l' b, a,0 T Example b, a a,b tangent vector normal direction x 1 x 2 Ideal points x, 2,0 T 1 x 0,0,1 Line at infinity T l P 2 R 2 l Note that in P 2 there is no distinction between ideal points and others 15
A model for the projective plane exactly one line through two points exactly one point at intersection of two lines 16
Duality x l x T l 0 l T x 0 x l l' l x x' Duality principle: To any theorem of 2-dimensional projective geometry there corresponds a dual theorem, which may be derived by interchanging the role of points and lines in the original theorem 17
Projective 2D Geometry 18
Projective transformations Definition: A projectivity is an invertible mapping h from P 2 to itself such that three points x 1,x 2,x 3 lie on the same line if and only if h(x 1 ),h(x 2 ),h(x 3 ) do. Theorem: A mapping h:p 2 P 2 is a projectivity if and only if there exist a non-singular 3x3 matrix H such that for any point in P 2 reprented by a vector x it is true that h(x)=hx Definition: Projective transformation x' x' x' 1 2 3 h h h 11 21 31 h h h 12 22 32 h h h 13 23 33 x x x 1 2 3 or x' H x 8DOF projectivity=collineation=projective transformation=homography 19
Mapping between planes central projection may be expressed by x =Hx (application of theorem) 20
Removing projective distortion select four points in a plane with know coordinates x' 1 h11x h12y h13 x' 2 h21x h22y h x' y' x' h x h y h x' h x h y h x' y' 3 31 32 h31x h32y h33 h11x h12y h13 h31x h32y h33 h21x h22y h23 33 3 31 32 (linear in h ij ) (2 constraints/point, 8DOF 4 points needed) 23 33 21
22
More Examples 23
Transformation for lines For a point transformation x' H x Transformation for lines l' H -T l 24
Isometries x' y' 1 cos sin 0 sin cos 0 tx x t y y 1 1 1 orientation preserving: orientation reversing: 1 1 x' H E x R T 0 t x 1 R T R I 3DOF (1 rotation, 2 translation) special cases: pure rotation, pure translation Invariants: length, angle, area 25
Similarities 26 1 1 0 0 cos sin sin cos 1 ' ' y x t s s t s s y x y x x 0 x x' 1 t T R H s S I R R T also know as equi-form (shape preserving) metric structure = structure up to similarity (in literature) 4DOF (1 scale, 1 rotation, 2 translation) Invariants: ratios of length, angle, ratios of areas, parallel lines
Affine transformations 27 1 1 0 0 1 ' ' 22 21 12 11 y x t a a t a a y x y x x 0 x x' 1 t T A H A non-isotropic scaling! (2DOF: scale ratio and orientation) 6DOF (2 scale, 2 rotation, 2 translation) Invariants: parallel lines, ratios of parallel lengths, ratios of areas DR R R A 2 1 0 0 D
Projective transformations 28 x v x x' v P T t A H Action non-homogeneous over the plane 8DOF (2 scale, 2 rotation, 2 translation, 2 line at infinity) Invariants: cross-ratio of four points on a line (ratio of ratio) T 2 1, v v v v v s P A S T T T T v t v 0 1 0 0 1 0 t A I K R H H H H T RK tv A s K 1 det K upper-triangular, decomposition unique (if chosen s>0)
Overview Transformations Projective 8dof Affine 6dof Similarity 4dof Euclidean 3dof h h h a a 0 sr sr 0 r11 r21 0 11 21 31 11 21 11 21 h h h a a r r 12 22 32 12 22 0 sr sr 12 22 0 12 0 22 h13 h 23 h 33 tx t y 1 tx t y 1 tx t y 1 Concurrency, collinearity, order of contact (intersection, tangency, inflection, etc.), cross ratio Parallellism, ratio of areas, ratio of lengths on parallel lines (e.g midpoints), linear combinations of vectors (centroids). The line at infinity l Ratios of lengths, angles. The circular points I,J lengths, areas. 29
Line at infinity 30 2 2 1 1 2 1 2 1 0 v x v x v x x x x v A A T t 0 0 0 2 1 2 1 x x x x v A A T t Line at infinity becomes finite, allows to observe vanishing points, horizon, Line at infinity stays at infinity
The line at infinity 31 v 1 v 2 l 1 l 2 l 4 l 3 l v 1 v 2 l 2 1 1 l l v 4 3 2 l l v The line at infinity l is a fixed line under a projective transformation H if and only if H is an affinity l 1 0 0 1 t 0 l l A A H T T A
Affine properties from images projection rectification H PA 1 0 l1 0 1 l 2 0 0 H l 3 A T l l l, l 0 l 1 2 3 3 32
Affine Rectification v 1 v 2 l l 1 l 3 l 2 l 4 v v l l1 2 l3 4 v 1 v 2 1 l 2 l 33
Projective 3D geometry Projective 15dof A T v t v Intersection and tangency Affine 12dof A T 0 t 1 Parallellism of planes, Volume ratios, centroids, The plane at infinity π Similarity 7dof s R T 0 t 1 The absolute conic Ω Euclidean 6dof R 0 T t 1 Volume 34
Camera Calibration
Pinhole Camera Model 36 T T Z fy Z fx Z Y X ) /, / ( ),, ( 1 0 1 0 0 1 Z Y X f f Z fy fx Z Y X 1 0 1 0 1 0 1 1 Z Y X f f Z fy fx
Pinhole Camera Model x K I 0X cam K f f px p y 1 Principal point offset Calibration Matrix 37
Internal Camera Parameters 38
Camera rotation and translation X ~ R X~ -C ~ cam RC ~ X R Y X cam 0 1 Z 1 R 0 RC X 1 x K I 0X cam x KR I C ~ X x PX P K R t t RC ~ 39
Camera Parameter Matrix P 40
CCD Cameras 41
Correcting Radial Distortion of Cameras 42
43
Calibration Process 44
x c ( x c )(1 k r k r k r )cos 2 4 6 u x d x 1 d 2 d 3 d ( y c )(1 k r k r k r )sin 2 4 6 d y 1 d 2 d 3 d y c ( x c )(1 k r k r k r )sin 2 4 6 u y d x 1 d 2 d 3 d ( y c )(1 k r k r k r )cos 2 4 6 d y 1 d 2 d 3 d x y p p m x m y m m x 0 u 1 u 2 6 u m7yu 1 m x m y m m x 3 u 4 u 5 6 u m7yu 1 45
OpenCV http://opencv.willowgarage.com/documentation/camera_c alibration_and_3d_reconstruction.html 46
Camera Calibration Toolbox http://www.vision.caltech.edu/bouguetj/calib_doc/ 47
Camera Calibration 48
Calibrating a Stereo System 49
Robust Multi-camera Calibration http://graphics.stanford.edu/~vaibhav/projects/calibcs205/cs205.html 50
Multi-Camera Self Calibration http://cmp.felk.cvut.cz/~esvoboda/ 51
Multi-Camera Self Calibration 52
ARToolKit Camera Calibration 53
Epipolar Geometry and 3D Reconstruction.
Introduction Computer vision is concerned with the theory behind artificial systems that extract information from images. The image data can take many forms, such as video sequences, views from multiple cameras. Computer vision is, in some ways, the inverse of computer graphics. While computer graphics produces image data from 3D models, computer vision often produces 3D models from image data. Today one of the major problems in Computer vision is correspondence search. 55
Epipolar Geometry http://www.ai.sri.com/~luong/research/meta3dviewer/epipolargeo.html C,C,x,x and X are coplanar 56
Epipolar Geometry epipole 57
Epipolar Geometry All points on p project on l and l 58
Epipolar Geometry Family of planes p and lines l and l Intersection in e and e 59
Epipolar Geometry epipoles e,e = intersection of baseline with image plane = projection of projection center in other image = vanishing point of camera motion direction an epipolar plane = plane containing baseline (1-D family) an epipolar line = intersection of epipolar plane with image (always come in corresponding pairs) 60
The Fundamental Matrix F x' H π x l' e' x' e' H x Fx π H : projectivity=collineation=projective transformation=homography 61
Cross Products 62
Epipolar Lines 63
Three Questions (i) Correspondence geometry: Given an image point x in the first view, how does this constrain the position of the corresponding point x in the second image? (ii) Camera geometry (motion): Given a set of corresponding image points {x i x i }, i=1,,n, what are the cameras P and P for the two views? (iii) Scene geometry (structure): Given corresponding image points x i x i and cameras P, P, what is the position of (their pre-image) X in space? 64
Parameter estimation 2D homography Given a set of (x i,x i ), compute H (x i =Hx i ) 3D to 2D camera projection Given a set of (X i,x i ), compute P (x i =PX i ) Fundamental matrix Given a set of (x i,x i ), compute F (x i T Fx i =0) 65
The fundamental matrix F Algebraic Derivation X λ F e' P P' P x λc l' e' x' e' H x Fx l' P'C P'P x π P x P P I X λ (note: doesn t work for C=C F=0) 66
The fundamental matrix F Correspondence Condition The fundamental matrix satisfies the condition that for any pair of corresponding points x x in the two images x' T Fx 0 67
The Fundamental Matrix Song http://danielwedge.com/fmatrix/ 68
The Fundamental Matrix http://www.cs.unc.edu/~blloyd/comp290-089/fmatrix/ 69
Epipolar geometry: basic equation 70 0 Fx x' T separate known from unknown 0 ' ' ' ' ' ' 33 32 31 23 22 21 13 12 11 f yf xf f y yf y xf y f x yf x xf x 0,,,,,,,,,1, ',, ', ' ',, ', ' T 33 32 31 23 22 21 13 12 11 f f f f f f f f f y x y y y x y x y x x x (data) (unknowns) Af 0 0 f 1 ' ' ' ' ' ' 1 ' ' ' ' ' ' 1 1 1 1 1 1 1 1 1 1 1 1 n n n n n n n n n n n n y x y y y x y x y x x x y x y y y x y x y x x x
the NOT normalized 8-point algorithm 71 0 1 1 1 33 32 31 23 22 21 13 12 11 2 2 2 2 2 2 2 2 2 2 2 2 1 1 1 1 1 1 1 1 1 1 1 1 f f f f f f f f f y x y y y y x x x y x x y x y y y y x x x y x x y x y y y y x x x y x x n n n n n n n n n n n n ~10000 ~10000 ~10000 ~10000 ~100 ~100 1 ~100 ~100! Orders of magnitude difference Between column of data matrix least-squares yields poor results
the normalized 8-point algorithm Transform image to ~[-1,1]x[-1,1] (0,500) (700,500) 2 700 0 2 500 1 1 1 (-1,1) (0,0) (1,1) (0,0) (700,0) (-1,-1) (1,-1) Least squares yields good results (Hartley, PAMI 97) 72
The fundamental matrix F F is the unique 3x3 rank 2 matrix that satisfies x T Fx=0 for all x x (i) Transpose: if F is fundamental matrix for (P,P ), then F T is fundamental matrix for (P,P) (ii) Epipolar lines: l =Fx & l=f T x (iii) Epipoles: on all epipolar lines, thus e T Fx=0, x e T F=0, similarly Fe=0 (iv) F has 7 d.o.f., i.e. 3x3-1(homogeneous)-1(rank2) (v) F is a correlation, projective mapping from a point x to a line l =Fx (not a proper correlation, i.e. not invertible) 73
Fundamental matrix for pure translation 74
Fundamental matrix for pure translation 75
Fundamental matrix for pure translation e' e' F H 1 H K RK Example: 0 e' 1,0,0 T F 0 0 x' T Fx 0 y y' 0 0 1 0-1 0 x x' PX K[I 0]X -1 P'X K[I t] K x Z ( X,Y,Z ) T -1 K x/z x' x Kt/Z motion starts at x and moves towards e, faster depending on Z pure translation: F only 2 d.o.f., x T [e] x x=0 auto-epipolar 76
General Motion x' x' T T x' e' Hx 0 e' xˆ 0 K'RK -1 x K't/Z 77
Projective transformation and invariance Derivation based purely on projective concepts xˆ x Hx, xˆ' H'x' Fˆ PX H' -1 PH H X Pˆ Xˆ -1 P'H H X Pˆ' Xˆ -T FH -1 F invariant to transformations of projective 3-space x' P'X P, P' F F P,P' unique not unique Canonical form P [I 0] P' [M m] F m M F e' P' P 78
Projective ambiguity of cameras given F 79 previous slide: at least projective ambiguity this slide: not more! Show that if F is same for (P,P ) and ~ (P,P ), ~ there exists a projective transformation H so that P=HP and P =HP P [I 0] P' [A a] F a A ~ a lemma: ~ a A ~ ka A ~ k 1 A av ~ ~ P [I 0] P' [A ~ ~ a] T af a a A 0 ~ af rank 2 T a A ~ a a ka ~ - A 0 ka ~ - A av A ~ 1 H k I 0 1 T k v k 1 P'H [A a] k I 1 k v T 0 k [ k ~ ~ 1 ~ T A -av ka] P' ~ a ka (22-15=7, ok)
Canonical cameras given F F matrix corresponds to P,P iff P T FP is skew-symmetric X T P' T FPX 0, X F matrix, S skew-symmetric matrix P [I 0] [SF e' ] T Possible choice: P [I 0] P' [SF e' ] T T F[I 0] F S F T e' F P' [[e'] F (fund.matrix=f) e' ] T 0 F S 0 0 T F 0 0 Canonical representation: P [I 0] P' [[e'] F e' v T λe'] 80
The Essential matrix ~ Fundamental matrix for calibrated cameras (remove K) t T E R R[R t] xˆ ' T E Exˆ K' T 0 FK xˆ 5 d.o.f. (3 for R; 2 for t up to scale) K -1 x; xˆ' K -1 x' E is essential matrix if and only if two singularvalues are equal (and third=0) T E Udiag(1,1,0)V 81
3D Reconstruction 82
Epipolar Geometry Underlying structure in set of matches for rigid scenes l T 1 l 2 T m F m1 2 0 Fundamental matrix (3x3 rank 2 matrix) C1 m1 l m1 1 e 1 e 2 P m2 l2 m2 C 2 M L2 L1 Canonical representation: P [I 0] P' [[e'] F e' v T λe'] 1. Computable from corresponding points 2. Simplifies matching 3. Allows to detect wrong matches 4. Related to calibration 83
3D reconstruction of cameras and structure Reconstruction Problem: given x i x i, compute P,P and X i xi PX i x i PX i for all i without additional informastion possible up to projective ambiguity 84
Outline of 3D Reconstruction (i) (ii) (iii) Compute F from correspondences Compute camera matrices P,P from F Compute 3D point for each pair of corresponding points computation of F use x i Fx i =0 equations, linear in coeff. F 8 points (linear), 7 points (non-linear), 8+ (least-squares) computation of camera matrices T use P [I 0] P' [[e'] F e' v λe' ] triangulation compute intersection of two backprojected rays 85
Reconstruction ambiguity: similarity x i PX i -1 PH H X S S i PH -1 S K[R t] R' 0 T - R' λ T t' K[RR' T RR' T t' λt] 86
Reconstruction ambiguity: projective x i PX i -1 PH H X P P i 87
Terminology x i x i Original scene X i Projective, affine, similarity reconstruction = reconstruction that is identical to original up to projective, affine, similarity transformation Literature: Metric and Euclidean reconstruction = similarity reconstruction 88
The projective reconstruction theorem If a set of point correspondences in two views determine the fundamental matrix uniquely, then the scene and cameras may be reconstructed from these correspondences alone, and any two such reconstructions from these correspondences are projectively equivalent x i x i P P P -1 2 1H P,P ', P,P ', 1 1 X1i P theorem from last class 2 2 X2i X except :Fx xf 0-1 2 P 1H 2 HX1-1 HX1 i P1 H HX1 i P1 X1 i xi P2 2i 2 X along same ray of P 2, idem for P 2 two possibilities: X 2i =HX 1i, or points along baseline key result : allows reconstruction from pair of uncalibrated images i i 89
90
Stratified reconstruction (i) (ii) (iii) Projective reconstruction Affine reconstruction Metric reconstruction 91
Projective to affine Remember 2-D case 92
Projective to affine P,P', π T H - A, B, C, D T 0,0,0, 1 T X i π I 0 H π 0,0,0,1 T (if D 0) theorem says up to projective transformation, but projective with fixed p is affine transformation can be sufficient depending on application, e.g. mid-point, centroid, parallellism 93
Translational motion points at infinity are fixed for a pure translation reconstruction of x i x i is on p F [e] [e' ] P [I 0] P [I e'] 94
Scene constraints Parallel lines parallel lines intersect at infinity reconstruction of corresponding vanishing point yields point on plane at infinity 3 sets of parallel lines allow to uniquely determine p remark: in presence of noise determining the intersection of parallel lines is a delicate problem remark: obtaining vanishing point in one image can be sufficient 95
Scene constraints 96
Scene constraints Distance ratios on a line known distance ratio along a line allow to determine point at infinity (same as 2D case) 97
The infinity homography P [M m] P' [M' m' ] -1 H M' M unchanged under affine transformations P [M m] A a 0 1 H M' AA -1 M -1 [MA Ma m] X X ~,0 T affine reconstruction P [I 0] P [H e] x MX ~ x' M' X ~ 98
One of the cameras is affine According to the definition, the principal plane of an affine camera is at infinity to obtain affine recontruction, compute H that maps third row of P to (0,0,0,1) T and apply to cameras and reconstruction e.g. if P=[I 0], swap 3 rd and 4 th row, i.e. H 1 0 0 0 0 1 0 0 0 0 0 1 0 0 1 0 99
Affine to metric identify absolute conic transform so that : X Y Z 0, on π then projective transformation relating original and reconstruction is a similarity transformation 2 in practice, find image of W image w back-projects to cone that intersects p in W 2 2 * note that image is independent of particular reconstruction * 100
Affine to metric given P [M m] ω possible transformation from affine to metric is H A 0-1 0 1 AA T T M ωm 1 (cholesky factorisation) proof: P ω M * M -1 ω PH M -1 M M -1 M -T [MA m] T M AA MAA T T M T * I 0 0 0 101
Orthogonality vanishing points corresponding to orthogonal directions T v ωv2 1 0 vanishing line and vanishing point corresponding to plane and normal direction l ωv 102
Correspondence and RANSAC Algorithm.
Correspondence Search 104
Feature Matching To match the points in one image to the points in the other image by exhaustive search(to match one point in one image to all the points in the other image) is a difficult and long process so some constraints are applied. As geometric constraint to minimize the search area for correspondence. The geometric constraints is provided by the epipolar geometry 105
Harris Detector: Mathematics Change of intensity for the shift [x,y]: E( x, y) w( u, v)[ I( x u, y v) I( x, y)] uv, 2 Window function Shifted intensity Intensity Window function w(u,v) = or 1 in window, 0 outside Gaussian 106
E x y I x y I x x y y 2 (, ) [ (, ) (, )] u x,v y w x I( x, y) I( x, y) [ I x( x, y) I y( x, y)] w y x [ I x( x, y) I y( x, y)] w y 2 2 ( I x( x, y)) I x( x, y) I y( x, y) w w x y x 2 I x( x, y) I y( x, y) ( I y( x, y)) y w w x x yc( x, y) y 2 107
Harris Detector: Mathematics For small shifts [u,v] we have a bilinear approximation: u E( u, v) u, v M v where M is a 22 matrix computed from image derivatives: M 2 I x I xi y w( x, y) 2 xy, I xi y I y 108
( max ) -1/2 ( min) -1/2 Harris Detector: Mathematics Intensity change in shifting window: eigenvalue analysis u E( u, v) u, v M v 1, 2 eigenvalues of M If we try every possible orientation n, the max. change in intensity is 2 Ellipse E(u,v) = const 109
Harris Detector: Mathematics Classification of image points using eigenvalues of M: 2 Edge 2 >> 1 Corner 1 and 2 are large, 1 ~ 2 ; E increases in all directions 1 and 2 are small; E is almost constant in all directions Flat region Edge 1 >> 2 1 110
Harris Detector: Mathematics Measure of corner response: trace 2 R det M k M det M trace M 1 2 1 2 (k empirical constant, k = 0.04-0.06) 111
Harris Detector: Mathematics R depends only on eigenvalues of M R is large for a corner R is negative with large magnitude for an edge R is small for a flat region 2 Edge R < 0 Corner R > 0 Flat R small Edge R < 0 1 112
Harris Detector The Algorithm: Find points with large corner response function R Take the points of local maxima of R 113
Harris Detector : Workflow 114
Compute corner response R Harris Detector : Workflow 115
Find points with large corner response: R>threshold Harris Detector : Workflow 116
Take only the points of local maxima of R Harris Detector : Workflow 117
Harris Detector : Workflow 118
Correlation for Correspondence Search i 0,0 j 7,7 Left image: 1. From the left feature point image we select one feature point. 2. Draw the window(n*n) around, with feature point in the center. 3. Calculate the normalized window using the formula s 1 N N i1 j1 w 1 ( i, j)* w 1 ( i, j) N is the size of window w 1( nor) w ( i, j) 1 ( i, j) s 119 1
Correlation Algorithm Select one feature point from the first image. Draw the window across it(7*7). Normalize the window using the given formula. s w ( i, j)* w ( i, j) N is the size of window 1 1 1 i1 j1 w 1( nor) N N ( i, j) w1 ( i, j) s 1 Find the feature in the right image, that are to be considered in the first image,(this should be done by some distance thresholding) After finding the feature point the window of same dimension should be selected in the second image. The normalized correlstion measure should be calculated using the formula N N s w ( i, j)* w ( i, j) 2 1 2 i1 j1 cormat N N i1 j1 s 2 w ( i, j)* w ( i, j) 2 2 120
RANSAC 121
RANSAC Select sample of m points at random 122
RANSAC Select sample of m points at random Calculate model parameters that fit the data in the sample 123
RANSAC Select sample of m points at random Calculate model parameters that fit the data in the sample Calculate error function for each data point 124
RANSAC Select sample of m points at random Calculate model parameters that fit the data in the sample Calculate error function for each data point Select data that support current hypothesis 125
RANSAC Select sample of m points at random Calculate model parameters that fit the data in the sample Calculate error function for each data point Select data that support current hypothesis Repeat sampling 126
RANSAC Select sample of m points at random Calculate model parameters that fit the data in the sample Calculate error function for each data point Select data that support current hypothesis Repeat sampling 127
RANSAC ALL-INLIER SAMPLE RANSAC time complexity k number of samples drawn N number of data points t M time to compute a single model m S average number of models per sample 128
Feature-space outliner rejection Can we now compute H from the blue points? No! Still too many outliers What can we do?
Matching features What do we do about the bad matches?
RAndom SAmple Consensus Select one match, count inliers
RAndom SAmple Consensus Select one match, count inliers
Least squares fit Find average translation vector
RANSAC for estimating homography RANSAC loop: 1. Select four feature pairs (at random) 2. Compute homography H (exact) 3. Compute inliers where SSD(p i, H p i) < ε 4. Keep largest set of inliers 5. Re-compute least-squares H estimate on all of the inliers
RANSAC for Fundamental Matrix Step 1. Extract features Step 2. Compute a set of potential matches Step 3. do Step 3.1 select minimal sample (i.e. 7 matches) Step 3.2 compute solution(s) for F (verify hypothesis) Step 3.3 determine inliers until (#inliers,#samples)<95% Step 4. Compute F based on all inliers Step 5. Look for additional matches Step 6. Refine F based on all correct matches (generate hypothesis)
RANSAC 136
RANSAC 137
Example: Mosaicking with homographies www.cs.cmu.edu/~dellaert/mosaicking 138
Recognizing Panoramas 139
Recognizing Panoramas 140
THANK YOU!! 141
9/28 10/5 10/12 10/19 Liu Chang photosynth create account Take pictures Construct photosynth Markus Photomodeller Select one small object Make 3d model data Yang Yun PhotoCity Select one small object Make 3d model data Wang Ping PhotoCity Presentation file How to install & use Cygwin, bunder, Matlab CC Liu Pengxin Zhu Zi Jian 142 Wang Yuo Remove Perspective distortion Load image Choose feature points SIFT & ASIFT OpenCV CC PhotoTourism ARToolkit Camera Calibration PhotoTourism Matlab CC OpenCV CC Search others photosynth works Select one point in DSU, take pictures Projector Price? Matlab CC Lab Image Data SIFT Fundamental Matrix Remove PT OpenCV CC PT PhotoTourism ARToolkit Camera Calibration Fundamental Matrix M C C led lamp
Topics in Image Processing
144 2014-04-08
145
146
147 2014-04-08
148 2014-04-08
illisis 149
LYYN 150
151 2014-04-08
faceapi http://www.seeingmachines.com/product/faceapi/ 152
Total Immersion - D Fusion Pro http://www.t-immersion.com/products/dfusionsuite/dfusion-pro 153
D Fusion Pro - Markless Tracking 154
155
Monitoring System 156
Scenarios Clear Vision - Denoising Motion Deblurring SuperResolution Panoramic View Background Modeling 157
Around View Monitor 158
Projective Transformations
Projective 2D Geometry 160
Projective transformations Definition: A projectivity is an invertible mapping h from P 2 to itself such that three points x 1,x 2,x 3 lie on the same line if and only if h(x 1 ),h(x 2 ),h(x 3 ) do. Theorem: A mapping h:p 2 P 2 is a projectivity if and only if there exist a non-singular 3x3 matrix H such that for any point in P 2 reprented by a vector x it is true that h(x)=hx Definition: Projective transformation x' x' x' 1 2 3 h h h 11 21 31 h h h 12 22 32 h h h 13 23 33 x x x 1 2 3 or x' H x 8DOF projectivity=collineation=projective transformation=homography 161
Mapping between planes central projection may be expressed by x =Hx (application of theorem) 162
Removing projective distortion select four points in a plane with know coordinates x' 1 h11x h12y h13 x' 2 h21x h22y h x' y' x' h x h y h x' h x h y h x' y' 3 31 32 h31x h32y h33 h11x h12y h13 h31x h32y h33 h21x h22y h23 33 3 31 32 (linear in h ij ) (2 constraints/point, 8DOF 4 points needed) 23 33 163