Image Warping. Srikumar Ramalingam School of Computing University of Utah. [Slides borrowed from Ross Whitaker] 1

Transcription

1 Image Warping Srikumar Ramalingam School of Computing University of Utah [Slides borrowed from Ross Whitaker] 1

2 Geom Trans: Distortion From Optics Barrel Distortion Pincushion Distortion Straight lines bulge out as in a barrel Corners of points form elongated points as in a cushion 2

3 Geom Trans: Distortion From Optics Barrel Distortion Pincushion Distortion 3

4 Geom. Trans.: Brain Template/Atlas 4

5 Warping Application: Lens Distortion Radial transformation lenses are generally circularly symmetric - Optical center is known 5

6 Warping application Lens r = xҧ i = xҧ xҧ i Distortion x, ҧ x ҧ optical centers xҧ i (1 + k 1 r 2 + k 2 r 4 + k 3 r 6 + ) xҧ r xҧ i x ҧ xҧ i Not all warps are radially symmetric how do we handle more general warps? 6

7 Specifying Warps Another Strategy Let the # DOFs (unknown parameters to compute) in the warp equal to the # of control points (i.e., point correspondences) Interpolate with some grid-based interpolation E.g. binlinear, splines xҧ i xҧ i 7

8 Landmarks Not On Grid Landmark positions driven by application Interpolate transformation at unorganized correspondences Scattered data interpolation How do we do scattered data interpolation? Idea: use kernels! Radial basis functions Radially symmetric functions of distance to landmark 8

9 ҧ Gaussian Examples of RBFs x - center of distortion, or control point ε constant, xҧ i point near the control point φ i Inverse quadratic φ i x ҧ = Polyharmonic spline x ҧ = e ε 1 x ҧ xҧ i ε xҧ xҧ 2 i xҧ 2 xҧ N xҧ x 1 ҧ 9

10 RBFs Formulation Represent function f as weighted sum of basis functions Sum of radial basis functions We are given the vectors x, ҧ ҧ x 1,, xҧ N. Basis functions centered at positions of data Our goal is to compute the N unknown parameters k 1,, k N. We are given the function values f for several points. 10

11 ҧ ҧ ҧ RBFs Formulation We are given several point correspondences between two images as shown by { x 1 ҧ, x 1 ҧ,, x N, xҧ N }. Need interpolation for vector-valued function, T: x = T x y = T y x = x = N i=1 N i=1 k x i φ i ( x) ҧ k y i φ i ( x) ҧ 11

12 ҧ ҧ Solve for parameters (k s) With Landmarks as Constraints We have a total of 2N unknowns x = T x y = T y x = x = i=1 N Rewriting it in the matrix form: N i=1 k x i φ i ( x) ҧ k y i φ i ( x) ҧ 12

13 Issue: RBFs Do Not Easily Model Linear Trends Example : Assume that we are given function values for several 1D points { x 1, f 1,, x N, f N }. f(x) f 3 f 2 f 1 x x 1 x 2 x 3 13

14 RBFs Formulation with additional Linear Terms Represent f as weighted sum of basis functions and linear part Linear part of transformation Sum of radial basis functions Basis functions centered at positions of data Original N RBF terms 3 linear terms per coordinate 14

15 RBFs Formulation with additional Linear Terms Represent f as weighted sum of basis functions and linear part Linear part of transformation Sum of radial basis functions Basis functions centered at positions of data Need interpolation for vector-valued function, T: We have N point correspondences and 2N+6 unknowns. How do we solve for unknown parameters k s and p s? 15

16 RBFs Solution Strategy Find the k s and p s so that f() fits at data points The k s can have no linear trend (force it into the p s) Constraints -> linear system Correspondences must match Keep linear part separate from deformation N N N N k x i x i = 0 k x i y i = 0 k y i x i = 0 k y i y i = 0 i=1 i=1 i=1 i=1 16

17 RBFs Linear System We have 2N + 6 equations from N point correspondences N equations N equations N N N N 6 equations k i x x i = 0 k x i y i = 0 k y i x i = 0 k y i y i = 0 i=1 i=1 i=1 i=1 Rewriting them in matrix form: 17

18 RBFs Linear System Ax = b Solution for unknowns (2N k s and 6 p s): x = A 1 b 18

19 RBF Warp Example We are given 6 point correspondences to compute the warp parameters: 19

20 ҧ What Kernel Should We Use Gaussian Variance is free parameter controls smoothness of warp φ i x ҧ = e x ҧ 2σ 2 x i 2 s 2.5 s 2.0 s 1.5 From: Arad et al

21 RBFs Aligning faces From: Arad et al

22 RBFs Aligning Faces Mona Lisa Target Venus Source Venus Warped 22

23 RBFs Special Case: Thin Plate Splines A special class of kernels xҧ 2 xҧ N xҧ x 1 ҧ xҧ xҧ 1 23

24 Thin-plate spline 24

25 RBFs Special Case: Thin Plate Splines Minimizes the distortion function (bending energy). The name thin plate spline refers to a physical analogy involving the bending of a thin sheet of metal. 25

26 RBFs Special Case: Thin Plate Splines You can think of these functions as f(x,y) that tries to minimize the bending energy: 26

27 RBFs Special Case: Thin Plate Splines There are no free parameters that need manual tuning. It has closed-form solutions for both warping and parameter estimation. There is a physical explanation for its energy function. Produces smooth surfaces. Bookstein,

28 Monalisa Target Aligning faces Venus Source Venus Warped Venus warped + Monalisa 28

29 Image Morphing Source image Intermediate morphed image Destination image Morphing refers to a seamless transition from one image to another. This is called cross-dissolve in film industry. 29

30 Application: Image Morphing Combine shape and intensity with time parameter t = [0,1] Just blending with amounts t produces fade Use control points with interpolation in t Use {c1, c(t)} landmarks to define T1 that warps I1 to I, and {c2,c(t)} landmarks to define T2 that warps I2 to I. 30

31 Image Morphing Create from blend of two warped images I 1 I t I 2 T 1 T 2 Source image Intermediate morphed image Destination image 31

32 Application: Image Templates/Atlases Build image templates that capture statistics of class of images Accounts for shape and intensity Mean and variability Purpose Establish common coordinate system (for comparisons) Understand how a particular case compares to the general population 32

33 Templates Formulation N landmarks over M different subjects/samples Images Correspondences Mean of correspondences (template) Transformations from subjects to mean c i Ƹ = T j ( c i ҧ j ) 33

34 Cars 34

35 Car Landmarks and Warp 35

36 Car Landmarks and Warp 36

37 Car Mean 37

38 Cars 38

39 Cats 39

40 Brains 40

41 Brain Template 41

42 Automatic Registration of Images/Objects Within a (limited) class of transformations find the one that best matches the image E.g. find rotation, translation to optimize the correlation (or squared difference) between two images How to solve: Gradient descent Derivatives of energy with respect to transformation Includes derivatives of image Can get stuck in local minima Search E.g. brute force (all angles and translations), or be smart 42

43 Rigid Registration of Objects/Point Sets Optimize squared distance to nearest point in other set S 0 S 1 Algorithm (iterative closest point, Besl 1992) 1. For every point in S 0 find nearest point on S 1 2. Find best rigid alignment of these correspondences Closed form solution, Horn Repeat until converged 43

44 Corresponding Point Set Alignment Let M be a model point set and let S be a scene point set. Assume closest points correspond Objective function : argmin R,T N (S i RM i T i ) i=1 R rotation, T - translation

45 Aligning 2D Data Converges if starting position is close enough