Prof. Feng Liu. Winter /05/2019

Transcription

1 Prof. Feng Liu Winter /05/2019

2 Last Time Image alignment 2

3 Toda Image warping The slides for this topic are used from Prof. Yung-Yu Chuang, which use materials from Dr. Richard Szeliski, Prof. Steve Seitz, Prof. Tom Funkhouser and Prof. Aleei Efros 3

4 Image warping image filtering: change range of image g() = h(f()) f h g h()= image warping: change domain of image g() = f(h()) f h g h()=2

5 Image warping image filtering: change range of image g() = h(f()) f h g h()= image warping: change domain of image g() = f(h()) f h g h([,])=[,2]

6 Parametric (global) warping Eamples of parametric warps: translation rotation aspect affine perspective clindrical

7 Parametric (global) warping p = (,) Transformation T is a coordinate-changing machine: p = T(p) What does it mean that T is global? Is the same for an point p can be described b just a few numbers (parameters) Represent T as a matri: p = M*p T p = (, ) M

8 Scaling Scaling a coordinate means multipling each of its components b a scalar Uniform scaling means this scalar is the same for all components: 2 f g 2 2

9 Non-uniform scaling: different scalars per component: Scaling 2, g f

10 Scaling Scaling operation: Or, in matri form: b a b a 0 0 scaling matri S What s inverse of S?

11 2-D Rotation This is eas to capture in matri form: cosq sinq sin q cos q Even though sin(q) and cos(q) are nonlinear to q, is a linear combination of and is a linear combination of and What is the inverse transformation? Rotation b q R For rotation matrices, 1 T R R

12 22 Matrices What tpes of transformations can be represented with a 22 matri? 2D Identit? D Scale around (0,0)? s s * * s s 0 0

13 22 Matrices What tpes of transformations can be represented with a 22 matri? 2D Rotate around (0,0)? * cos * sin * sin * cos q q q q q q q q cos sin sin cos 2D Shear? sh sh * * sh sh 1 1

14 22 Matrices What tpes of transformations can be represented with a 22 matri? 2D Mirror about Y ais? D Mirror over (0,0)?

15 All 2D Linear Transformations Linear transformations are combinations of Scale, Rotation, Shear, and Mirror Properties of linear transformations: Origin maps to origin Lines map to lines Parallel lines remain parallel Ratios are preserved Closed under composition a c b d

16 22 Matrices What tpes of transformations can not be represented with a 22 matri? 2D Translation? t t NO! Onl linear 2D transformations can be represented with a 22 matri

17 Translation Eample of translation t t t t t = 2 t = 1 Homogeneous Coordinates

18 Affine Transformations Affine transformations are combinations of Linear transformations, and Translations Properties of affine transformations: Origin does not necessaril map to origin Lines map to lines Parallel lines remain parallel Ratios are preserved Closed under composition w f e d c b a w 1 0 0

19 Projective Transformations Projective transformations Affine transformations, and Projective warps Properties of projective transformations: Origin does not necessaril map to origin Lines map to lines Parallel lines do not necessaril remain parallel Ratios are not preserved Closed under composition w i h g f e d c b a w

20 Image warping Given a coordinate transform = T() and a source image I(), how do we compute a transformed image I ( ) = I (T())? T() I() I ( )

21 Forward warping Send each piel I() to its corresponding location = T() in I ( ) T() I() I ( )

22 Forward warping fwarp(i, I, T) { for (=0; <I.height; ++) for (=0; <I.width; ++) { (, )=T(,); I (, )=I(,); } } I I T

23 Forward warping Send each piel I() to its corresponding location = T() in I ( ) What if piel lands between two piels? Will be there holes? Answer: add contribution to several piels, normalize later (splatting) h() f() g( )

24 Forward warping fwarp(i, I, T) { for (=0; <I.height; ++) for (=0; <I.width; ++) { (, )=T(,); Splatting(I,,,I(,),kernel); } } I I T

25 Inverse warping Get each piel I ( ) from its corresponding location = T -1 ( ) in I() T -1 ( ) I() I ( )

26 Inverse warping iwarp(i, I, T) { for (=0; <I.height; ++) for (=0; <I.width; ++) { (,)=T -1 (, ); I (, )=I(,); } } I T -1 I

27 Inverse warping Get each piel I ( ) from its corresponding location = T -1 ( ) in I() What if piel comes from between two piels? Answer: resample color value from interpolated (prefiltered) source image f() g( )

28 Inverse warping iwarp(i, I, T) { for (=0; <I.height; ++) for (=0; <I.width; ++) { (,)=T -1 (, ); I (, )=Reconstruct(I,,,kernel); } } I T -1 I

29 Reconstruction Computed weighted sum of piel neighborhood; output is weighted average of input, where weights are normalized values of filter kernel k q p d width p i k( q k( q i i ) color=0; weights=0; for all q s dist < width d = dist(p, q); w = kernel(d); color += w*q.color; weights += w; p.color = color/weights; i ) q i

30 Reconstruction (interpolation) Possible reconstruction filters (kernels): nearest neighbor bilinear bicubic sinc (optimal reconstruction)

31 Bilinear interpolation (triangle filter) A simple method for resampling images

32 Non-parametric image warping Specif a more detailed warp function Splines, meshes, optical flow (per-piel motion)

33 Non-parametric image warping Mappings implied b correspondences Inverse warping? P

34 Non-parametric image warping P w A A w B B w C C P w A A w B B w C C Barcentric coordinate P P

35 Barcentric coordinates P t t t A 1 1 t2 t3 PA2 A A 2 t 2 1 A A 3 A 3 2 t 3 A 3

36 Non-parametric image warping P w A A w B B w C C P w A A w B B w C C Barcentric coordinate

37 Image Morphing

38 Image morphing The goal is to snthesize a fluid transformation from one image to another. Cross dissolving is a common transition between cuts, but it is not good for morphing because of the ghosting effects. image #1 image #2 dissolving

39 Image morphing Wh ghosting? Morphing = warping + cross-dissolving shape (geometric) color (photometric)

40 Image morphing image #1 cross-dissolving image #2 warp morphing warp

41 Image morphing create a morphing sequence: for each time t 1. Create an intermediate warping field (b interpolation) 2. Warp both images towards it 3. Cross-dissolve the colors in the newl warped images t=0 t=0.33 t=1

42 An ideal eample t=0 middle face (t=0.5) t=1

43 Warp specification (mesh warping) How can we specif the warp? 1. Specif corresponding spline control points interpolate to a complete warping function eas to implement, but less epressive

44 Warp specification How can we specif the warp 2. Specif corresponding points interpolate to a complete warping function

45 Solution: convert to mesh warping 1. Define a triangular mesh over the points Same mesh in both images! Now we have triangle-to-triangle correspondences 2. Warp each triangle separatel from source to destination How do we warp a triangle? 3 points = affine warp! Just like teture mapping

46 Warp specification (field warping) How can we specif the warp? 3. Specif corresponding vectors interpolate to a complete warping function The Beier & Neel Algorithm

47 Beier&Neel (SIGGRAPH 1992) Single line-pair PQ to P Q :

48 Algorithm (single line-pair) For each X in the destination image: 1. Find the corresponding u,v 2. Find X in the source image for that u,v 3. destinationimage(x) = sourceimage(x ) Eamples: Affine transformation

49 Multiple Lines D i X i X i length = length of the line segment, dist = distance to line segment The influence of a, p, b. The same as the average of X i

50 Full Algorithm

51 Resulting warp

52 Comparison to mesh morphing Pros: more epressive Cons: speed and control

53 Warp interpolation How do we create an intermediate warp at time t? linear interpolation for line end-points But, a line rotating 180 degrees will become 0 length in the middle One solution is to interpolate line mid-point and orientation angle t=0 t=1

54 Animation

55 Animated sequences Specif keframes and interpolate the lines for the inbetween frames Require a lot of tweaking

56 Results Images from [Beier and Neel 1992]

57 Multi-source morphing

58 References Thaddeus Beier, Shawn Neel, Feature-Based Image Metamorphosis, SIGGRAPH 1992, pp Detlef Ruprecht, Heinrich Muller, Image Warping with Scattered Data Interpolation, IEEE Computer Graphics and Applications, March 1995, pp Seung-Yong Lee, Kung-Yong Chwa, Sung Yong Shin, Image Metamorphosis Using Snakes and Free-Form Deformations, SIGGRAPH Seungong Lee, Wolberg, G., Sung Yong Shin, Polmorph: morphing among multiple images, IEEE Computer Graphics and Applications, Vol. 18, No. 1, 1998, pp Peinsheng Gao, Thomas Sederberg, A work minimization approach to image morphing, The Visual Computer, 1998, pp George Wolberg, Image morphing: a surve, The Visual Computer, 1998, pp