CSE 554 Lecture 7: Deformation II

Transcription

1 CSE 554 Lecture 7: Deformation II Fall 2011 CSE554 Deformation II Slide 1 Review Rigid-body alignment Non-rigid deformation Intrinsic methods: deforming the boundary points An optimization problem Minimize shape distortion Maximize fit Example: Laplacian-based deformation Before After Source Target CSE554 Deformation II Slide 2 1

2 Extrinsic Deformation Computing deformation of each point in the plane or volume Not just points on the boundary curve or surface Credits: Adams and Nistri, BMC Evolutionary Biology (2010) CSE554 Deformation II Slide 3 Extrinsic Deformation Applications Registering contents between images and volumes Interactive spatial deformation CSE554 Deformation II Slide 4 2

3 Techniques Thin-plate spline deformation Free form deformation Cage-based deformation CSE554 Deformation II Slide 5 Thin-Plate Spline Given corresponding source and target points Computes a spatial deformation function for every point in the 2D plane or 3D volume Credits: Sprengel et al, EMBS (1996) CSE554 Deformation II Slide 6 3

4 Thin-Plate Spline A minimization problem Minimizing distances between source and target points Minimizing distortion of the space (as if bending a thin sheet of metal) There is a closed-form solution Solving a linear system of equations CSE554 Deformation II Slide 7 Thin-Plate Spline Input p i q i Source points: p 1,,p n Target points: q 1,,q n Output p A deformation function f[p] for any point p f[p] CSE554 Deformation II Slide 8 4

5 Thin-Plate Spline Minimization formulation E E f E d E f : fitting term Measures how close is the deformed source to the target E d : distortion term Measures how much the space is warped l: weight Controls how much non-rigid warping is allowed CSE554 Deformation II Slide 9 Thin-Plate Spline Fitting term Minimizing sum of squared distances between deformed source points and target points n E f f p i q i 2 i 1 CSE554 Deformation II Slide 10 5

6 Thin-Plate Spline Distortion term Minimizing a physical bending energy on a metal sheet (2D): E d 2 f x f x y 2 2 f y x y The energy is zero when the deformation is affine Translation, rotation, scaling, shearing CSE554 Deformation II Slide 11 Thin-Plate Spline Finding the minimizer for E E f E d Uniquely exists, and has a closed form: n f p M p p p i v i i 1 where r r 2 Log r M: an affine transformation matrix v i : deformation vectors Both M and v i are determined by p i,q i,l r CSE554 Deformation II Slide 12 6

7 Thin-Plate Spline Result At higher l, the deformation is closer to an affine transformation 0 Credits: Sprengel et al, EMBS (1996) CSE554 Deformation II Slide 13 Thin-Plate Spline Application: image registration Manual or automatic feature pair detection Source Target Deformed source Credits: Rohr et al, TMI (2001) CSE554 Deformation II Slide 14 7

8 Thin-Plate Spline Advantages Smooth deformations, with physical analogy Closed-form solution Few free parameters (no tuning is required) Disadvantages Solving the equations still takes time (hence cannot perform Interactive deformation) CSE554 Deformation II Slide 15 Free Form Deformation Uses a control lattice that embeds the shape Deforming the lattice points warps the embedded shape Credits: Sederberg and Parry, SIGGRAPH (1986) CSE554 Deformation II Slide 16 8

9 Free Form Deformation Warping the space by blending the deformation at the control points Each deformed point is a weighted sum of deformed lattice points CSE554 Deformation II Slide 17 Free Form Deformation Input Source lattice points: p 1,,p n Target lattice points: q 1,,q n Output A deformation function f[p] for any point p in the lattice grid: p f[p] n f p w i p q i i 1 p i q i w i [p]: determined by relative position of p with respect to p i CSE554 Deformation II Slide 18 9

10 Free Form Deformation Desirable properties of the weights w i [p] Greater when p is closer to p i So that the influence of each control point is local Smoothly varies with location of p So that the deformation is smooth p f[p] n 1 w i p i 1 So that f[p] = S w i [p] q i is an affine combination of q i p i q i n p w i p p i i 1 So that f[p]=p if the lattice stays unchanged CSE554 Deformation II Slide 19 Free Form Deformation Finding weights (2D) Let the lattice points be p i,j for i=0,,k and j=0,,l Compute p s relative location in the grid (s,t) p 0,2 p 1,2 p 2,2 p 3,2 Let (x min,x max ), (y min,y max ) be the range of grid t p p 0,1 p 1,1 p 2,1 p 3,1 s p x x min x max x min t p y y min y max y min p 0,0 p 1,0 s p 2,0 p 3,0 CSE554 Deformation II Slide 20 10

11 Free Form Deformation Finding weights (2D) Let the lattice points be p i,j for i=0,,k and j=0,,l Compute p s relative location in the grid (s,t) p 0,2 p 1,2 p 2,2 p 3,2 The weight w i,j for lattice point p i,j is: t p u i,j p i,j B i k s B j l t p 0,1 p 1,1 p 2,1 p 3,1 w i,j p k i 0 u i,j p l j 0 u i,j p p 0,0 p 1,0 s p 2,0 p 3,0 l i,j : importance of p i,j B: Bernstein basis function: B i k s k i si 1 s k i CSE554 Deformation II Slide 21 Free Form Deformation Finding weights (2D) Weight distribution for one control point (max at that control point): w 1,1 p p 3,2 p 3,1 p 0,2 p 1,1 p 0,1 p 2,0 p 3,0 p 1,0 p 0,0 CSE554 Deformation II Slide 22 11

12 Free Form Deformation A deformation example CSE554 Deformation II Slide 23 Free Form Deformation Registration Embed the source in a lattice Compute new lattice positions over the target Fitting term Distance between deformed source shape and the target shape Agreement of the deformed image content and the target image Distortion term Thin-plate spline energy Non-folding constraint Local rigidity constraint (to prevent stretching) CSE554 Deformation II Slide 24 12

13 Free Form Deformation Registration example Deformed w/o rigidity Source Deformed with rigidity Target Credits: Loeckx et al, MICCAI (2004) CSE554 Deformation II Slide 25 Free Form Deformation Advantages Smooth deformations Easy to implement (no equation solving) Efficient and localized controls for interactive editing Can be coupled with different fitting or energy objectives Disadvantages Too many lattice points in 3D The lattice structure is not suitable for organic, non-cubical shapes CSE554 Deformation II Slide 26 13

14 Cage-based Deformation Use a control mesh ( cage ) to embed the shape Deforming the cage vertices warps the embedded shape Credits: Ju, Schaefer, and Warren, SIGGRAPH (2005) CSE554 Deformation II Slide 27 Cage-based Deformation Warping the space by blending the deformation at the cage vertices p p i n f p w i p q i i 1 w i [p]: determined by relative position of p with respect to p i f[p] q i CSE554 Deformation II Slide 28 14

15 Cage-based Deformation Finding weights (2D) Problem: given a closed polygon (cage) with vertices p i and an interior point p, find weights w i [p] such that: 1) 2) n 1 w i p i 1 n p w i p p i i 1 p p i 3) They vary smoothly with p CSE554 Deformation II Slide 29 Cage-based Deformation Finding weights (2D) A simple case: the cage is a triangle The weights are unique, and are the barycentric coordinates of p p 1 w 1 p Area p,p 2,p 3 Area p1,p 2,p 3 p p 2 p 3 CSE554 Deformation II Slide 30 15

16 Cage-based Deformation Finding weights (2D) The harder case: the cage is an arbitrary (possibly concave) polygon The weights are not unique A good choice: Mean Value Coordinates (MVC) Can be extended to 3D p i-1 u i p Tan i 2 Tan i 1 2 p i p u i p w i p n i 1 u i p p α i α i+1 p i p i+1 CSE554 Deformation II Slide 31 Cage-based Deformation Finding weights (2D) Weight distribution of one cage vertex in MVC: w i p p i CSE554 Deformation II Slide 32 16

17 Cage-based Deformation Application: character animation Using improved weights: Harmonic Coordinates CSE554 Deformation II Slide 33 Cage-based Deformation Registration Embed source in a cage Compute new locations of cage vertices over the target Minimizing some fitting and energy objectives Not seen in literature yet future work! CSE554 Deformation II Slide 34 17

18 Cage-based Deformation Advantages over free form deformations Much smaller number of control points Flexible structure suitable for organic shapes Disadvantages Setting up the cage can be time-consuming The cage needs to be a closed shape Not as flexible as point handles CSE554 Deformation II Slide 35 Further Readings Thin-plate spline deformation Principal warps: thin-plate splines and the decomposition of deformations, by Bookstein (1989) Landmark-Based Elastic Registration Using Approximating Thin-Plate Splines, by Rohr et al. (2001) Free form deformation Free-Form Deformation of Solid Geometric Models, by Sederberg and Parry (1986) Extended Free-Form Deformation: A sculpturing Tool for 3D Geometric Modeling, by Coquillart (1990) Cage-based deformation Mean value coordinates for closed triangular meshes, by Ju et al. (2005) Harmonic coordinates for character animation, by Joshi et al. (2007) Green coordinates, by Lipman et al. (2008) CSE554 Deformation II Slide 36 18