2D Image Transforms 16-385 Computer Vision (Kris Kitani) Carnegie Mellon University
Extract features from an image what do we do next?
Feature matching (object recognition, 3D reconstruction, augmented reality, image stitching) How do you compute the transformation?
Given a set of matched feature points {x i, x 0 i} set of point correspondences point in one image point in the other image and a transformation x 0 = f(x; p) transformation function parameters Find the best estimate of p
What kind of transformation functions are there? x 0 = f(x; p)
2D Transformations
translation rotation aspect affine perspective cylindrical
2D Planar Transformations
2D Planar Transformations Scale Each component multiplied by a scalar Uniform scaling - same scalar for each component
2D Planar Transformations Scale Scale x 0 = ax y 0 = by Each component multiplied by a scalar Uniform scaling - same scalar for each component
2D Planar Transformations Scale apple x 0 y 0 = Scale apple a 0 0 b apple x y scaling matrix S Each component multiplied by a scalar Uniform scaling - same scalar for each component
2D Planar Transformations Scaling Shear
2D Planar Transformations Shear Scaling Shear x 0 = x + a y y 0 = b x + y
2D Planar Transformations Scaling Shear Shear apple apple x 0 1 a y 0 = b 1 apple x y
2D Planar Transformations x = apple x y
2D Planar Transformations x 0 = apple x 0 y 0 x = apple x y
2D Planar Transformations x 0 = apple x 0 y 0 Rotation x 0 = x cos y sin y 0 = x sin + y cos x = apple x y
(x, y ) Polar coordinates x = r cos (φ) y = r sin (φ) x = r cos (φ + θ) y = r sin (φ + θ) θ φ (x, y) Trig Identity x = r cos(φ) cos(θ) r sin(φ) sin(θ) y = r sin(φ) cos(θ) + r cos(φ) sin(θ) Substitute x = x cos(θ) - y sin(θ) y = x sin(θ) + y cos(θ)
2D Planar Transformations apple x 0 x 0 = y 0 apple x 0 y 0 = Rotation apple cos sin sin cos apple x y x = apple x y
2D linear transformation (can be written in matrix form) x 0 = f(x; p) apple x 0 y 0 = M apple x y parameters p point x
Scale Flip across y M = apple sx 0 0 s y M = apple 1 0 0 1 Rotate M = apple cos sin sin cos Flip across origin apple 1 0 M = 0 1 Shear Identity M = apple 1 sx s y 1 M = apple 1 0 0 1
How do you represent translation with a 2 x 2 matrix? x 0 = x + t x y 0 = y + t x y apple?? M =??
How do you represent translation with a 2 x 2 matrix? x 0 = x + t x y 0 = y + t x not possible
Q: How can we represent translation in matrix form? x 0 = x + t x y 0 = y + t y 25
Homogeneous Coordinates
apple x y add a one here 2 3 x =) 4 y 5 inhomogenous coordinates 1 homogenous coordinates Represent 2D point with a 3D vector
Q: How can we represent translation in matrix form? x 0 = x + t x y 0 = y + t y A: append 3rd element and append 3rd column & row apple x y 2 3 2 3 x ) 4 y 1 5 M = 4 1 0 t x 0 1 t y 0 0 1 5
Homogeneous Coordinates t x =2 t y =1
A 2D point in an image can be represented as a 3D vector x = apple x y () X = 2 4 x 1 x 2 3 5 x 3 where x = x 1 x 3 y = x 2 x 3 Why?
Think of a point on the image plane in 3D y image plane X P z x z =1 X is a projection of a point P on the image plane You can think of a conversion to homogenous coordinates as a conversion of a point to a ray
Conversion: 2D point homogeneous point append 1 as 3rd coordinate apple x y ) 2 4 x y 1 3 5 homogeneous point 2D point divide by 3rd coordinate 2 4 x y w 3 5 ) apple x/w y/w Special Properties Scale invariant Point at infinity Undefined x y w > = x y w > x y 0 0 0 0
Basic 2D transformations as 3x3 matrices Translate Scale Rotate Shear
Matrix Composition Transformations can be combined by matrix multiplication p = T(t x,t y ) R(Θ) S(s x,s y ) p Does the order of multiplication matter?
2D transformations
Affine Transformation Affine transformations are combinations of Linear transformations, and Translations Properties of affine transformations: Origin does not necessarily map to origin Lines map to lines Parallel lines remain parallel Ratios are preserved Closed under composition (affine times affine is affine) Will the last coordinate w ever change?
Coming soon Projective Transform Projective transformations are combos of Affine transformations, and Projective warps Properties of projective transformations: Origin does not necessarily map to origin Lines map to lines Parallel lines do not necessarily remain parallel Ratios are not preserved Closed under composition Models change of basis Projective matrix is defined up to a scale (8 DOF)