Affine and Projective Transformations

Transcription

1 CS 674: Intro to Computer Vision Affine and Projective Transformations Prof. Adriana Kovaska Universit of Pittsburg October 3, 26

2 Alignment problem We previousl discussed ow to matc features across images, of te same or different objects Now let s focus on te case of two images of te same object (e.g. i and i ) and eamine it in more detail Wat transformation relates i and i? In alignment, we will fit te parameters of some transformation according to a set of matcing feature pairs ( correspondences ). i T i Adapted from Kristen Grauman and Derek Hoiem

3 Two questions T Alignment: Given two images, wat is te transformation between tem? T Warping: Given a source image and a transformation, wat does te transformed output look like? Kristen Grauman

4 Motivation for feature-based alignment: Image mosaics Kristen Grauman Image from ttp://grapics.cs.cmu.edu/courses/5-463/2_fall/

5 Wat are te correspondences? Min SSD = matc? Compare content in local patces, find best matces. Simplest approac: Scan i wit template formed from a point in i, and compute Euclidean distance (a.k.a. SSD) or normalized crosscorrelation between list of piel intensities in te patc. Kristen Grauman

6 Wat are te transformations? Eamples of transformations: translation rotation aspect affine perspective Alosa Efros

7 Parametric (global) warping T p = (,) p = (, ) Transformation T is a coordinate-canging macine: p = T(p) Wat does it mean tat T is global? It is te same for an point p It can be described b just a few numbers (parameters) Let s represent T as a matri: p = Mp Alosa Efros M

8 Scaling Scaling a coordinate means multipling eac of its components b a scalar Uniform scaling means tis scalar is te same for all components: 2 Alosa Efros

9 Scaling Non-uniform scaling: different scalars per component: X 2, Y.5 Alosa Efros

10 Scaling Scaling operation: Or, in matri form: b a q p n m scaling matri S Alosa Efros b a b a q p n m

11 2D Linear Transformations a c b d Onl linear 2D transformations can be represented wit a 22 matri. Linear transformations are combinations of Scale, Rotation, Sear, and Mirror Alosa Efros

12 2D Rotate around (,)? cos sin sin cos cos sin sin cos 2D Sear? s s s s 2D Scaling? s s s s Modified from Alosa Efros Fig. from ttps:// Wat transforms can we write wit a 22 matri?

13 Wat transforms can we write wit a 22 matri? 2D Mirror about Y ais? 2D Mirror over (,)? Alosa Efros 2D Translation? t t CAN T DO!

14 Homogeneous coordinates To convert to omogeneous coordinates: omogeneous image coordinates Converting from omogeneous coordinates Kristen Grauman

15 t t Translation t t t t t = 2 t = Homogeneous Coordinates Alosa Efros

16 2D Affine Transformations Affine transformations are combinations of Linear transformations, and Translations Maps lines to lines, parallel lines remain parallel w f e d c b a w Adapted from Alosa Efros

17 Fitting an affine transformation Assuming we know te correspondences, ow do we get te transformation? ), ( i i ), ( i i t t m m m m i i i i i i i i i i t t m m m m Alosa Efros

18 Fitting an affine transformation How man matces (correspondence pairs) do we need to solve for te transformation parameters? Once we ave solved for te parameters, ow do we compute given? Were do te matces come from? i i i i i i t t m m m m ), ( new new Modified from Kristen Grauman ), ( new new

19 Projective Transformations Projective transformations: Affine transformations, and Projective warps Parallel lines do not necessaril remain parallel w i g f e d c b a w Kristen Grauman

20 Projection: world coord image coord Optical. center f z.. P z. p Camera center Modified from Derek Hoiem and Kristen Grauman Scene point Image coordinates

21 Image mosaics: Goals... image from S. Seitz Obtain a wider angle view b combining multiple images. Kristen Grauman

22 Image mosaics: Camera setup Two images wit camera rotation/zoom but no translation Adapted from Derek Hoiem Camera Center

23 Mosaic: Man 2D views of same 3D object Steve Seitz mosaic plane Te mosaic as a natural interpretation in 3D Te images are reprojected onto a common plane Te mosaic is formed on tis plane Mosaic is a sntetic wide-angle camera

24 Image reprojection Basic question How to relate two images from te same camera center? ow to map a piel from PP to PP2 Answer Cast a ra troug eac piel in PP Draw te piel were tat ra intersects PP2 PP2 Observation: Rater tan tinking of tis as a 3D reprojection, tink of it as a 2D image warp from one image to anoter. PP Alosa Efros

25 Projective transforms A projective transform is a mapping between an two PPs wit te same center of projection rectangle sould map to arbitrar quadrilateral parallel lines aren t but preserves straigt lines Also called Homograp PP2 w w w p H p PP Adapted from Alosa Efros

26 How to stitc togeter a panorama (a.k.a. Basic Procedure mosaic)? Take a sequence of images from te same position Rotate te camera about its optical center Compute te omograp (transformation) between second image and first Transform te second image to overlap wit te first Blend te two togeter to create a mosaic (If tere are more images, repeat) Modified from Steve Seitz

27 Computing te omograp,, 2, 2, 2 2,, n n n n To compute te omograp given pairs of corresponding points in te images, we need to set up an equation were te parameters of H are te unknowns Kristen Grauman

28 Computing te omograp p = Hp w a b c w d e f w g i Can set scale factor i=. So, tere are 8 unknowns. Set up a sstem of linear equations: A = b were vector of unknowns = [a,b,c,d,e,f,g,] T Need at least 8 eqs, but te more te better Solve for. If overconstrained, solve using least-squares: min A b 2 Kristen Grauman

29 Computing te omograp Assume we ave four matced points: How do we compute omograp H? p w w w H Derek Hoiem p =Hp Appl SVD: UDV T = A [U, S, V] = svd(a); = V smallest (column of V corr. to smallest singular value) A

30 How to stitc togeter a panorama (a.k.a. Basic Procedure mosaic)? Take a sequence of images from te same position Rotate te camera about its optical center Compute te omograp (transformation) between second image and first Transform te second image to overlap wit te first Blend te two togeter to create a mosaic (If tere are more images, repeat) Modified from Steve Seitz

31 w w w H p p w w w w,,, To appl a given omograp H Compute p = Hp (regular matri multipl) Convert p from omogeneous to image coordinates Modified from Kristen Grauman Transforming te second image Image canvas Image 2

32 Transforming te second image Image 2 Image canvas H(,) f(,) g(, ) Forward warping: Send eac piel f(,) to its corresponding location (, ) = H(,) in te rigt image Modified from Alosa Efros

33 Transforming te second image H(,) f(,) g(, ) Forward warping: Send eac piel f(,) to its corresponding location (, ) = H(,) in te rigt image Q: wat if piel lands between two piels? A: distribute color among neigboring piels (, ) Alosa Efros

34 Transforming te second image Image 2 Image canvas H - (,) f(,) g(, ) Inverse warping: Get eac piel g(, ) from its corresponding location (,) = H - (, ) in te left image Modified from Alosa Efros

35 Transforming te second image H - (,) f(,) g(, ) Alosa Efros Inverse warping: Get eac piel g(, ) from its corresponding location (,) = H - (, ) in te left image Q: wat if piel comes from between two piels? A: interpolate color value from neigbors >> elp interp2

36 Homograp eample: Image rectification p p To unwarp (rectif) an image solve for omograp H given p and p : p =Hp Derek Hoiem

37 Summar Write 2d transformations as matri-vector multiplication (including translation wen we use omogeneous coordinates) Projection equations epress ow world points mapped to 2d image. Fitting transformations: solve for unknown parameters given corresponding points from two views linear, affine, projective (omograp) Mosaics: uses omograp and image warping to merge views taken from same center of projection Perform image warping (forward, inverse) Adapted from Kristen Grauman

38 Te net 5 idden slides give some more detail about ow image formation works, i.e. ow 3D objects are mapped to 2D images. Please review on our own time if ou re interested.