Image Warping. Computational Photography Derek Hoiem, University of Illinois 09/28/17. Photo by Sean Carroll

Transcription

1 Image Warping 9/28/7 Man slides from Alosha Efros + Steve Seitz Computational Photograph Derek Hoiem, Universit of Illinois Photo b Sean Carroll

2 Reminder: Proj 2 due monda Much more difficult than project get started asap if not alread Must compute SSD cost for ever piel (slow but not horribl slow using filtering method; see tips at end of project page) Learn how to debug visual algorithms: imshow, plot, dbstop if error, keboard and break points are our friends Debugging suggestion: For quilt_simple, first set upper-left patch to be upper-left patch in source and iterativel find minimum cost patch and overla --- should reproduce original source image, at least for part of the output

3 Review from last class: Gradient Domain Editing General concept: Solve for piels of new image that satisf constraints on the gradient and the intensit Constraints can be from one image (for filtering) or more (for blending)

4 Project 3: Reconstruction from Gradients. Preserve - gradients 2. Preserve intensit of one piel Source piels: s Variable piels: v. minimize (v(+,)-v(,) - (s(+,)-s(,))^2 2. minimize (v(,+)-v(,) - (s(,+)-s(,))^2 3. minimize (v(,)-s(,))^2

5 Project 3 (etra): NPR Preserve gradients on edges e.g., get cann edges with edge(im, cann ) Reduce gradients not on edges Preserve original intensit Perez et al. 23

6 Colorization using optimization Solve for uv channels (in Luv space) such that similar intensities have similar colors Minimize squared color difference, weighted b intensit similarit Solve with sparse linear sstem of equations

7 See Perez et al. 23 for man eamples Gradient-domain editing Man image processing applications can be thought of as tring to manipulate gradients or intensities: Contrast enhancement Denoising Poisson blending HDR to RGB Color to Gra Recoloring Teture transfer

8 Gradient-domain processing Salienc-based Sharpening

9 Gradient-domain processing Non-photorealistic rendering

10 Project 3: Gradient Domain Editing General concept: Solve for piels of new image that satisf constraints on the gradient and the intensit Constraints can be from one image (for filtering) or more (for blending)

11 Project 3: Reconstruction from Gradients. Preserve - gradients 2. Preserve intensit of one piel Source piels: s Variable piels: v. minimize (v(+,)-v(,) - (s(+,)-s(,))^2 2. minimize (v(,+)-v(,) - (s(,+)-s(,))^2 3. minimize (v(,)-s(,))^2

12 Project 3 (etra): NPR Preserve gradients on edges e.g., get cann edges with edge(im, cann ) Reduce gradients not on edges Preserve original intensit Perez et al. 23

13 Take-home questions ) I am tring to blend this bear into this pool. What problems will I have if I use: a) Alpha compositing with feathering b) Laplacian pramid blending c) Poisson editing? Lap. Pramid Poisson Editing

14 Take-home questions 2) How would ou make a sharpening filter using gradient domain processing? What are the constraints on the gradients and the intensities?

15 Net two classes: warping and morphing Toda Global coordinate transformations Image alignment Tuesda Interpolation and teture mapping Meshes and triangulation Shape morphing

16 Photo stitching: projective alignment Find corresponding points in two images Solve for transformation that aligns the images

17 Capturing light fields Estimate light via projection from spherical surface onto image

18 Morphing Blend from one object to other with a series of local transformations

19 Image Transformations image filtering: change range of image g() = T(f()) f T f image warping: change domain of image g() = f(t()) f f T

20 Image Transformations image filtering: change range of image g() = T(f()) f T g image warping: change domain of image f g() = f(t()) T g

21 Parametric (global) warping T p = (,) p = (, ) Transformation T is a coordinate-changing machine: p = T(p) What does it mean that T is global? Is the same for an point p can be described b just a few numbers (parameters) For linear transformations, we can represent T as a matri p = Mp M

22 Parametric (global) warping Eamples of parametric warps: translation rotation aspect affine perspective clindrical

23 Scaling Scaling a coordinate means multipling each of its components b a scalar Uniform scaling means this scalar is the same for all components: 2

24 Scaling Non-uniform scaling: different scalars per component: X 2, Y.5

25 Scaling Scaling operation: Or, in matri form: b a b a scaling matri S What is the transformation from (, ) to (, )?

26 2-D Rotation (, ) (, ) = cos() - sin() = sin() + cos()

27 2-D Rotation f (, ) (, ) Polar coordinates = r cos (f) = r sin (f) = r cos (f + ) = r sin (f + ) Trig Identit = r cos(f) cos() r sin(f) sin() = r sin(f) cos() + r cos(f) sin() Substitute = cos() - sin() = sin() + cos()

28 2-D Rotation This is eas to capture in matri form: cos sin sin cos Even though sin() and cos() are nonlinear functions of, is a linear combination of and is a linear combination of and What is the inverse transformation? Rotation b For rotation matrices R T R R

29 22 Matrices What tpes of transformations can be represented with a 22 matri? 2D Identit? 2D Scale around (,)? s s * * s s

30 22 Matrices What tpes of transformations can be represented with a 22 matri? 2D Rotate around (,)? * cos * sin * sin * cos cos sin sin cos 2D Shear? k k * * k k

31 22 Matrices What tpes of transformations can be represented with a 22 matri? 2D Mirror about Y ais? 2D Mirror over (,)?

32 22 Matrices What tpes of transformations can be represented with a 22 matri? 2D Translation? t t NO!

33 All 2D Linear Transformations Linear transformations are combinations of Scale, Rotation, Shear, and Mirror Properties of linear transformations: Origin maps to origin Lines map to lines Parallel lines remain parallel Ratios are preserved Closed under composition a c b d a c be d g f i h k j l

34 Homogeneous Coordinates Q: How can we represent translation in matri form? t t

35 Homogeneous Coordinates Homogeneous coordinates represent coordinates in 2 dimensions with a 3-vector homogeneous coords

36 Homogeneous Coordinates 2D Points Homogeneous Coordinates Append to ever 2D point: ( ) ( ) Homogeneous coordinates 2D Points Divide b third coordinate ( w) (/w /w) Special properties Scale invariant: ( w) = k * ( w) (,, ) represents a point at infinit (,, ) is not allowed 2 Scale Invariance (2,,) or (4,2,2) or (6,3,3) 2

37 Homogeneous Coordinates Q: How can we represent translation in matri form? A: Using the rightmost column: t t Translation t t

38 Translation Eample t t t t t = 2 t =

39 Basic 2D transformations as 33 matrices cos sin sin cos t t Translate Rotate Shear s s Scale

40 Matri Composition Transformations can be combined b matri multiplication w s s t t w cos sin sin cos p = T(t,t ) R() S(s,s ) p Does the order of multiplication matter?

41 Affine Transformations f e d c b a Affine transformations are combinations of Linear transformations, and Translations Properties of affine transformations: Origin does not necessaril map to origin Lines map to lines Parallel lines remain parallel Ratios are preserved Closed under composition

42 Projective Transformations Projective transformations are combos of Affine transformations, and Projective warps Properties of projective transformations: Origin does not necessaril map to origin Lines map to lines w Parallel lines do not necessaril remain parallel Ratios are not preserved Closed under composition Models change of basis Projective matri is defined up to a scale (8 DOF) a d g b e h c f i w

43 2D image transformations These transformations are a nested set of groups Closed under composition and inverse is a member

44 Recovering Transformations? T(,) f(,) g(, ) What if we know f and g and want to recover the transform T? willing to let user provide correspondences How man do we need?

45 Translation: # correspondences? How man Degrees of Freedom? How man correspondences needed for translation? What is the transformation matri? T(,)? p p p p M

46 Euclidean: # correspondences?? T(,) How man DOF? How man correspondences needed for translation+rotation?

47 Affine: # correspondences?? T(,) How man DOF? How man correspondences needed for affine?

48 Affine transformation estimation Math Matlab demo

49 Projective: # correspondences?? T(,) How man DOF? How man correspondences needed for projective?

50 Take-home Question ) Suppose we have two triangles: ABC and A B C. What transformation will map A to A, B to B, and C to C? How can we get the parameters? B B? A Source T(,) C A Destination C

51 Take-home Question 2) Show that distance ratios along a line are preserved under 2d linear transformations. o o (2,2) p=(,) o (3,3) a c b d p2 p3 p p p2 p3 p p Hint: Write down 2 in terms of and 3, given that the three points are co-linear

52 Net class: teture mapping and morphing