CS 674: Intro to Computer Vision Affine and Projective Transformations Prof. Adriana Kovaska Universit of Pittsburg October 3, 26
Alignment problem We previousl discussed ow to matc features across images, of te same or different objects Now let s focus on te case of two images of te same object (e.g. i and i ) and eamine it in more detail Wat transformation relates i and i? In alignment, we will fit te parameters of some transformation according to a set of matcing feature pairs ( correspondences ). i T i Adapted from Kristen Grauman and Derek Hoiem
Two questions T Alignment: Given two images, wat is te transformation between tem? T Warping: Given a source image and a transformation, wat does te transformed output look like? Kristen Grauman
Motivation for feature-based alignment: Image mosaics Kristen Grauman Image from ttp://grapics.cs.cmu.edu/courses/5-463/2_fall/
Wat are te correspondences? Min SSD = matc? Compare content in local patces, find best matces. Simplest approac: Scan i wit template formed from a point in i, and compute Euclidean distance (a.k.a. SSD) or normalized crosscorrelation between list of piel intensities in te patc. Kristen Grauman
Wat are te transformations? Eamples of transformations: translation rotation aspect affine perspective Alosa Efros
Parametric (global) warping T p = (,) p = (, ) Transformation T is a coordinate-canging macine: p = T(p) Wat does it mean tat T is global? It is te same for an point p It can be described b just a few numbers (parameters) Let s represent T as a matri: p = Mp Alosa Efros M
Scaling Scaling a coordinate means multipling eac of its components b a scalar Uniform scaling means tis scalar is te same for all components: 2 Alosa Efros
Scaling Non-uniform scaling: different scalars per component: X 2, Y.5 Alosa Efros
Scaling Scaling operation: Or, in matri form: b a q p n m scaling matri S Alosa Efros b a b a q p n m
2D Linear Transformations a c b d Onl linear 2D transformations can be represented wit a 22 matri. Linear transformations are combinations of Scale, Rotation, Sear, and Mirror Alosa Efros
2D Rotate around (,)? cos sin sin cos cos sin sin cos 2D Sear? s s s s 2D Scaling? s s s s Modified from Alosa Efros Fig. from ttps://www.siggrap.org/education/materials/hpergrap/modeling/mod_tran/2dsear.tm Wat transforms can we write wit a 22 matri?
Wat transforms can we write wit a 22 matri? 2D Mirror about Y ais? 2D Mirror over (,)? Alosa Efros 2D Translation? t t CAN T DO!
Homogeneous coordinates To convert to omogeneous coordinates: omogeneous image coordinates Converting from omogeneous coordinates Kristen Grauman
t t Translation t t t t t = 2 t = Homogeneous Coordinates Alosa Efros
2D Affine Transformations Affine transformations are combinations of Linear transformations, and Translations Maps lines to lines, parallel lines remain parallel w f e d c b a w Adapted from Alosa Efros
Fitting an affine transformation Assuming we know te correspondences, ow do we get te transformation? ), ( i i ), ( i i 2 4 3 2 t t m m m m i i i i i i i i i i t t m m m m 2 4 3 2 Alosa Efros
Fitting an affine transformation How man matces (correspondence pairs) do we need to solve for te transformation parameters? Once we ave solved for te parameters, ow do we compute given? Were do te matces come from? i i i i i i t t m m m m 2 4 3 2 ), ( new new Modified from Kristen Grauman ), ( new new
Projective Transformations Projective transformations: Affine transformations, and Projective warps Parallel lines do not necessaril remain parallel w i g f e d c b a w Kristen Grauman
Projection: world coord image coord Optical. center f z.. P z. p Camera center Modified from Derek Hoiem and Kristen Grauman Scene point Image coordinates
Image mosaics: Goals... image from S. Seitz Obtain a wider angle view b combining multiple images. Kristen Grauman
Image mosaics: Camera setup Two images wit camera rotation/zoom but no translation Adapted from Derek Hoiem Camera Center
Mosaic: Man 2D views of same 3D object Steve Seitz mosaic plane Te mosaic as a natural interpretation in 3D Te images are reprojected onto a common plane Te mosaic is formed on tis plane Mosaic is a sntetic wide-angle camera
Image reprojection Basic question How to relate two images from te same camera center? ow to map a piel from PP to PP2 Answer Cast a ra troug eac piel in PP Draw te piel were tat ra intersects PP2 PP2 Observation: Rater tan tinking of tis as a 3D reprojection, tink of it as a 2D image warp from one image to anoter. PP Alosa Efros
Projective transforms A projective transform is a mapping between an two PPs wit te same center of projection rectangle sould map to arbitrar quadrilateral parallel lines aren t but preserves straigt lines Also called Homograp PP2 w w w p H p PP Adapted from Alosa Efros
How to stitc togeter a panorama (a.k.a. Basic Procedure mosaic)? Take a sequence of images from te same position Rotate te camera about its optical center Compute te omograp (transformation) between second image and first Transform te second image to overlap wit te first Blend te two togeter to create a mosaic (If tere are more images, repeat) Modified from Steve Seitz
Computing te omograp,, 2, 2, 2 2,, n n n n To compute te omograp given pairs of corresponding points in te images, we need to set up an equation were te parameters of H are te unknowns Kristen Grauman
Computing te omograp p = Hp w a b c w d e f w g i Can set scale factor i=. So, tere are 8 unknowns. Set up a sstem of linear equations: A = b were vector of unknowns = [a,b,c,d,e,f,g,] T Need at least 8 eqs, but te more te better Solve for. If overconstrained, solve using least-squares: min A b 2 Kristen Grauman
Computing te omograp Assume we ave four matced points: How do we compute omograp H? p w w w 9 8 7 6 5 4 3 2 H 9 8 7 6 5 4 3 2 Derek Hoiem p =Hp Appl SVD: UDV T = A [U, S, V] = svd(a); = V smallest (column of V corr. to smallest singular value) A
How to stitc togeter a panorama (a.k.a. Basic Procedure mosaic)? Take a sequence of images from te same position Rotate te camera about its optical center Compute te omograp (transformation) between second image and first Transform te second image to overlap wit te first Blend te two togeter to create a mosaic (If tere are more images, repeat) Modified from Steve Seitz
w w w H p p w w w w,,, To appl a given omograp H Compute p = Hp (regular matri multipl) Convert p from omogeneous to image coordinates Modified from Kristen Grauman Transforming te second image Image canvas Image 2
Transforming te second image Image 2 Image canvas H(,) f(,) g(, ) Forward warping: Send eac piel f(,) to its corresponding location (, ) = H(,) in te rigt image Modified from Alosa Efros
Transforming te second image H(,) f(,) g(, ) Forward warping: Send eac piel f(,) to its corresponding location (, ) = H(,) in te rigt image Q: wat if piel lands between two piels? A: distribute color among neigboring piels (, ) Alosa Efros
Transforming te second image Image 2 Image canvas H - (,) f(,) g(, ) Inverse warping: Get eac piel g(, ) from its corresponding location (,) = H - (, ) in te left image Modified from Alosa Efros
Transforming te second image H - (,) f(,) g(, ) Alosa Efros Inverse warping: Get eac piel g(, ) from its corresponding location (,) = H - (, ) in te left image Q: wat if piel comes from between two piels? A: interpolate color value from neigbors >> elp interp2
Homograp eample: Image rectification p p To unwarp (rectif) an image solve for omograp H given p and p : p =Hp Derek Hoiem
Summar Write 2d transformations as matri-vector multiplication (including translation wen we use omogeneous coordinates) Projection equations epress ow world points mapped to 2d image. Fitting transformations: solve for unknown parameters given corresponding points from two views linear, affine, projective (omograp) Mosaics: uses omograp and image warping to merge views taken from same center of projection Perform image warping (forward, inverse) Adapted from Kristen Grauman
Te net 5 idden slides give some more detail about ow image formation works, i.e. ow 3D objects are mapped to 2D images. Please review on our own time if ou re interested.