http://grail.cs.washington.edu/projects/rotoscoping/ Image Warping, mesh, and triangulation CSE399b, Spring 7 Computer Vision
Man of the slides from A. Efros.
Parametric (global) warping Eamples of parametric warps: translation rotation aspect affine perspective clindrical
Parametric (global) warping Transformation T is a coordinate-changing machine: p = T(p) Linear transformation: T as a matri: p = M*p T p = (,) p = (,) = M = d c b a
All 2D Linear Transformations are matri Linear transformations are combinations of Scale, Rotation, Shear, and Mirror = a c b d Think we are taking weighted average of - coordinate Transforme d -ais Transforme d -ais
Most general 2D Linear transform: Projective Transformation
Mapping From Plane to Plane
Projective transformation Facts: ) parallel lines intersect, 2) circle becomes ellipses, 3) straight line is still straight
2D image transformations These transformations are a nested set of groups Closed under composition and inverse is a member
Homogeneous Coordinates Homogeneous coordinates represent coordinates in 2 dimensions with a 3-vector homogeneou scoords ( A point:
Homogeneous -> Real Coordinates divide the third number out: (,, w) represents a point at location (/w, /w) (,, ) represents a point at infinit (in direction,) (,, ) is not allowed 2 (2,,) or (4,2,2) or (6,3,3) Convenient coordinate sstem to represent man useful transformations 2
Basic 2D Transformations Basic 2D transformations as 33 matrices ( = cos sin sin cos = t t = sh sh Translate Rotate Shear = s s Scale
Special case: Similarit Transformation Similarit
Special case: Affine Transformations Affine transformations are combinations of Linear transformations, and Translations = w f e d c b a w Note: all point of infinit is mapped to infinit Therefore all parallel lines remain parallel
Special Case: Affine Transformation
Full Projective Transformations Projective transformations Affine transformations, and Projective warps = w i h g f e d c b a w
Special case: Projective transformation
Matri Composition Transformations can be combined b matri multiplication ( ) * * +, - - -. - = w s s t t w cos sin sin cos p = T(t,t ) R() S(s,s ) p
Image warping T(,) f(,) g(,) Given a coordinate transform (, ) = h(,) and a source image f(,), how do we compute a transformed image g(, ) = f(t(,))?
Forward warping T(,) f(,) g(,) Send each piel f(,) to its corresponding location (, ) = T(,) in the second image Q: what if piel lands between two piels?
Forward warping T(,) f(,) g(,) Send each piel f(,) to its corresponding location (, ) = T(,) in the second image Q: what if piel lands between two piels? A: distribute color among neighboring piels (, ) Known as splatting
Inverse warping T - (,) f(,) g(,) Get each piel g(, ) from its corresponding location (,) = T - (, ) in the first image Q: what if piel comes from between two piels?
Inverse warping T - (,) f(,) g(,) Get each piel g(, ) from its corresponding location (,) = T - (, ) in the first image Q: what if piel comes from between two piels? A: Interpolate color value from neighbors nearest neighbor, bilinear, Gaussian, bicubic
Bilinear interpolation Sampling at f(,):
Recovering Transformations? T(,) f(,) g(,) What if we know f and g and want to recover the transform T? willing to let user provide correspondences How man do we need?
Translation: correspondences? How man correspondences needed for translation? How man Degrees of Freedom? What is the transformation matri? T(,)? = p p p p M
Euclidian: correspondences?? T(,) How man correspondences needed for translation+rotation? How man DOF?
Affine: correspondences?? T(,) How man correspondences needed for affine? How man DOF?
Projective: correspondences?? T(,) How man correspondences needed for projective? How man DOF?
= How man independent para? Can we alwas set h33 =?
Eample: warping triangles Given two triangles: ABC and A B C in 2D (2 numbers) Need to find transform T to transfer all piels from one to the other. What kind of transformation is T? How can we compute the transformation matri: T(,)? A B C A C B Source Destination = f e d c b a
HINT: warping triangles (,) A B Source T (,) (,) Inverse change of basis change of basis C A T 2 B Destination Don t forget to move the origin too C
Image morphing: Lecture notes borrowed from A. Efros, T. Cootes
Morphing = Object Averaging The aim is to find an average between two objects Not an average of two images of objects but an image of the average object How can we make a smooth transition in time? Do a weighted average over time t How do we know what the average object looks like? We haven t a clue But we can often fake something reasonable Usuall required user/artist input
Averaging Points What s the average of P and Q? v = Q - P P Q Linear Interpolation (Affine Combination): New point ap + bq, defined onl when a+b = So ap+bq = ap+(-a)q P +.5v = P +.5(Q P) =.5P +.5 Q P and Q can be anthing: P +.5v = P +.5(Q P) = -.5P +.5 Q (etrapolation) points on a plane (2D) or in space (3D) Colors in RGB or HSV (3D) Whole images (m-b-n D) etc.
Idea : Cross-Dissolve Interpolate whole images: Image halfwa = (-t)*image + t*image 2 This is called cross-dissolve in film industr But what is the images are not aligned?
Idea 2: Align, then cross-disolve Align first, then cross-dissolve Alignment using global warp picture still valid
Dog Averaging What to do? Cross-dissolve doesn t work Global alignment doesn t work Cannot be done with a global transformation (e.g. affine) An ideas? Feature matching Nose to nose, tail to tail, etc. This is a local (non-parametric) warp
Idea 3: Local warp, then cross-dissolve Morphing procedure: for ever t,. Find the average shape (the mean dog ) local warping 2. Find the average color Cross-dissolve the warped images
Local (non-parametric) Image Warping Need to specif a more detailed warp function Global warps were functions of a few (2,4,8) parameters Non-parametric warps u(,) and v(,) can be defined independentl for ever single location, Once we know vector field u,v we can easil warp each piel (use backward warping with interpolation)
Image Warping non-parametric Move control points to specif a spline warp Spline produces a smooth vector field
Warp specification - dense How can we specif the warp? Specif corresponding spline control points interpolate to a complete warping function But we want to specif onl a few points, not a grid
Warp specification - sparse How can we specif the warp? Specif corresponding points interpolate to a complete warping function How do we do it? How do we go from feature points to piels?
Triangular Mesh. Input correspondences at ke feature points 2. Define a triangular mesh over the points Same mesh in both images Now we have triangle-to-triangle correspondences 3. Warp each triangle separatel from source to destination How do we warp a triangle? 3 points = affine warp Just like teture mapping
Triangulations A triangulation of set of points in the plane is a partition of the conve hull to triangles whose vertices are the points, and do not contain other points. There are an eponential number of triangulations of a point set.