Image Morphing Using Deformation Techniques

Transcription

1 Image Morphing Uing Deformation Technique Seung-Yong Lee, Kyung-Yong Chwa, Jame Hahn, and Sung Yong Shin Department of Computer Science Korea Advanced Intitute of Science and Technology Kuong-dong Yuong-gu Taejon, , Korea Department of EE & CS The George Wahington Univerity nd Street NW, Wahington, DC 20052, USA SUMMARY Thi paper preent a new image morphing method uing a two-dimenional deformation technique which provide an intuitive model for a warp The deformation technique derive a C 1 -continuou and one-to-one warp from a et of point pair overlaid on two image The reulting inbetween image preciely reflect the correpondence of feature pecified by an animator We alo control the tranition behavior in a metamorphoi equence by taking another deformable urface model, which i impler and thu more efficient than the deformation technique for a warp The propoed method eparate tranition control from feature interpolation and i eaier to ue than the previou technique The multigrid relaxation method i employed to olve a linear ytem in deriving a warp or tranition rate Thi method make our image morphing technique fat enough for an interactive environment Keyword: Image morphing, Deformation technique, Energy minimization method, Variational principle, Multigrid relaxation method 0

2 1 Introduction Image morphing deal with the metamorphoi of an image to another image The metamorphoi generate a equence of inbetween image in which an image gradually change into another image over time Image morphing technique have been widely ued in creating pecial effect for televiion commercial, muic video uch a Michael Jackon Black or White[1], and movie uch a Willow and Indiana Jone and the Lat Cruade[2] The problem of image morphing i baically how an inbetween image i effectively generated from two given image[1] When two face image are given, for example, a middle image may look like a third face reembling the given face An inbetween image can be derived from two image by properly interpolating the poition of correponding feature and their hape and color A feature of an image i it characterizing part uch a the profile of a face and eye and uually identified by a boundary curve at which color change abruptly A warp i a two-dimenional geometric tranformation and generate a ditorted image when it i applied to an image When two image are given, an image morphing method firt etablihe the feature correpondence between them The correpondence i then ued to compute warp that ditort the image to align the poition of feature and their hape A cro-diolve of color at each correponding pair of pixel in the ditorted image finally give an inbetween image The mot difficult part of image morphing i to derive warp which ditort image to align their feature The feature on an image are uually pecified by an animator with a et of point or line egment overlaid on the image A warp i then computed from the correpondence between the feature on two image Therefore, an image morphing technique mut be convenient in pecifying feature and how a predictable ditortion which reflect the feature correpondence In meh warping[2], feature are pecified by a nonuniform control meh, and a warp i computed by a pline interpolation Nihita et al[3] alo ued a nonuniform control meh to pecify feature and computed a warp uing a two-dimenional free-form deformation and Bézier clipping Field morphing[1] pecifie feature with a et of line egment and compute a warp by taking the weighted average of the influence of line egment Meh warping and the method of Nihita et al how good ditortion behavior but have a drawback in pecifying feature A control meh i alway required while the feature on an image can have an arbitrary tructure Field morphing give an eay-to-ue and expreive method in pecifying feature However, it uffer from unexpected ditortion referred to a ghot, which prevent an animator from realizing precie warp a hown in Section 7 The time for computing a warp i proportional to the number of line egment Thi i diadvantageou when a complicated feature et mut be pecified Thee drawback can be overcome by a phyically-baed approach which provide an intuitive model for a warp Conider an image printed on a heet of rubber When elected point on the heet are moved, the heet deformation thu obtained make the image appear ditorted The ditorted image conform to the diplacement of each elected point and how a proper ditortion over the entire image If a et of point pair pecifie the feature correpondence between two image, we can derive a neceary warp from the heet deformation which move each feature point to it correpondent There have been a number of reult[4, 5, 6] in flexible object modeling that give concrete theory and technique for upporting thi approach Thi paper take the rubber heet model and preent a new two-dimenional deformation technique for deriving warp The technique efficiently generate C 1 -continuou and one-to-one deformation from poitional contraint Thi approach doe not retrict a feature et to have any tructure uch a a meh, allowing more freedom in deigning a warp The reulting warp how natural ditortion which preciely reflect the 1

3 feature correpondence between image Another intereting but not yet fully invetigated problem of image morphing i the control of tranition behavior in a metamorphoi equence In generating an inbetween image, the rate of tranition i uually applied uniformly over all point on the image Thi reult in an animation in which the entire image change ynchronouly to another image If we control the tranition rate on different part of an inbetween image independently, a more intereting animation can be obtained Meh warping[2] aign a tranition curve for each point of the meh, and thee curve determine the tranition rate when the poition of feature are interpolated When complicated mehe are ued to pecify the feature, it i tediou to aign a proper tranition curve to every meh point Nihita et al[3] mentioned that the peed of tranition can be pecified by a Bézier function defined on the meh However, the detail of the method were not provided except only one example Thi paper ue a deformable urface model to control the tranition behavior, by aigning the tranition curve for elected point on an image Thee point are not necearily the ame a thoe ued for pecifying feature The tranition rate on an inbetween image are derived from the curve by contructing a deformable urface Thi approach eparate tranition control from feature interpolation and thu i much eaier to ue than the previou technique Section 2 explain the tep for generating an inbetween image and define the problem to be olved for completing the tep The following two ection concentrate on deformation technique to give the olution of the problem Section 5 introduce the multigrid relaxation method ued for olving a linear ytem in deriving a warp and tranition rate Section 6 preent the extenion of the baic technique employed for the new image morphing method Section 7 compare the preented method to the previou one in warp generation and give metamorphoi example Section 8 ummarize the contribution of thi paper 2 Problem in Image Morphing 21 Application of a warp to an image An image I can be repreented by a function from a bounded two-dimenional region to a color pace A warp W i a function from to, which pecifie a new poition for each point on I When W i applied to I, each pixel on I i copied onto the ditorted image I 0 at the poition determined by W The four-corner mapping paradigm[2] conider each pixel on I a a quare and tranform it into a quadrilateral on I 0 The quadrilateral often traddle everal pixel on I 0 or lie in the interior of one pixel A partial contribution i handled by caling the intenity of the pixel on I in proportion to the fractional part of the pixel on I 0 Thi technique generate a ditorted image without hole and properly reolve the collaped pixel To implement the four-corner mapping, we hould evaluate a warp function W at each corner of the pixel on an image I Hence, when the domain of W i dicretized for a numerical olution, the ize of a grid i choen a the reolution of I Once W ha been computed on the grid, the four-corner mapping can be performed by the blending hardware of a SGI machine[7] in a hort time 22 Inbetween image generation When two image I 0 and I 1 are given, the image morphing problem i to generate a equence of inbetween image I(t) uch that I(0) = I 0 and I(1) = I 1 We aume that time t varie from 0 to 1 when the ource image I 0 continuouly change to the detination image I 1 2

4 Let W 0 be the warp function which pecifie the correponding point on I 1 to each point on I 0 When it i applied to I 0, W 0 ha to ditort I 0 to match I 1 in the poition of feature and their hape Let W 1 be the warp function from I 1 to I 0 The requirement for W 1 i to map the feature on I 1 to the feature on I 0 when it ditort I 1 To generate an inbetween image I(t), we derive two warp function W 0 (t) and W 1 (t) from W 0 and W 1 by linear interpolation in time t I 0 and I 1 are then ditorted by W 0 (t) and W 1 (t), reulting in intermediate image I 0 (t) and I 1 (t), repectively The correponding feature on I 0 and I 1 have the ame poition and hape on I 0 (t) and I 1 (t) Finally, I(t) i obtained by cro-diolving the color between I 0 (t) and I 1 (t) That i, W 0 (t) = (1? t) R + t W 0 (1) W 1 (t) = t R + (1? t) W 1 (2) I 0 (t) = W 0 (t) I 0 (3) I 1 (t) = W 1 (t) I 1 (4) I(t) = (1? t) I 0 (t) + t I 1 (t); (5) where R denote the identity warp function, and W I denote the application of a warp W to an image I In the above procedure, time play the role of tranition rate which determine the relative influence of the ource and detination image on an inbetween image A tranition rate i a value between 0 and 1 With a tranition rate near zero, an inbetween image look more imilar to the ource image Tranition rate near one imply that inbetween image hould be much like the detination image With the formulae (1), (2), and (5), the ame tranition rate t i applied to all point on the inbetween image I(t) Therefore, the characteritic of the ource and detination image are reflected in the ame ratio all over an inbetween image The rate of tranition can be made different from point to point to derive a more intereting inbetween image We introduce a tranition function to facilitate the control of tranition behavior in generating an inbetween image A tranition function T pecifie the rate of tranition for each point on an image over time Let T 0 be a tranition function defined on the ource image I 0 In generating I(t), T 0 (t) determine how fat each point on I 0 move to the correponding point on the detination image I 1 T 0 (t) alo determine how much the color of each point on I 0 i reflected on the correponding point on I(t) Let T 1 be the tranition function defined on the detination image I 1, which pecifie the ame tranition behavior with T 0 T 1 can be derived from T 0 with the correpondence of point between I 1 and I 0 To each point on I 1, T 1 (t) hould aign the tranition rate which T 0 (t) give to the correponding point on I 0 To control the movement of each point on an inbetween image I(t), we replace time t in formulae (1) and (2) with T 0 (t) and T 1 (t), repectively For the color tranformation, however, time t in formula (5) cannot be imply replaced by T 0 (t) and T 1 (t) It i becaue the tranition function T 0 and T 1 are not defined on the ditorted image I 0 (t) and I 1 (t) but the given image I 0 and I 1, repectively Hence, we rearrange formulae (3), (4), and (5) o that T 0 (t) and T 1 (t) are ued to attenuate the color intenitie of I 0 and I 1 before applying warp function That i, W 0 (t) = (1? T 0 (t)) R + T 0 (t) W 0 W 1 (t) = T 1 (t) R + (1? T 1 (t)) W 1 I 0 (t) = W 0 (t) ((1? T 0 (t)) I 0 ) I 1 (t) = W 1 (t) (T 1 (t) I 1 ) 3

5 I(t) = I 0 (t) + I 1 (t): The tranformation of poition and color can be independently handled by pecifying different tranition function 23 Problem To complete the above procedure for image morphing, the following two problem need further invetigating: how to get the warp function W 0 and W 1, and how to get the tranition function T 0 and T 1 3 Warp Function Generation Thi ection preent a deformation technique for deriving warp function and explain how to obtain the warp function W 0 and W 1 with the technique 31 The deformation model Deformation technique baed on variational principle have been widely ued in computer graphic to model flexible object in three dimenion[4, 5, 8, 9] In thee technique, the requirement for a deformation uch a moothne are repreented by energy functional, and the deired hape of an object i derived by minimizing the um of energy functional The energy minimization problem are then tranformed to partial differential equation, which are uually olved by numerical method A warp can be conidered a a deformation of a rectangular heet in two-dimenional pace Previou deformation technique cannot be directly applicable to obtain warp becaue the deformation of rectangular heet are confined on two dimenion In thi paper, we preent a new two-dimenional deformation technique which efficiently generate a C 1 -continuou and one-to-one warp with variational principle Let be a rectangular thin plate and p = (u; v) a point on If every point on the plate i placed on the xy-plane, a hape of the plate can be repreented by a vector-valued function, w(p) = (x(p); y(p)) The function w pecifie the poition of each point p on the plate, lying in the xy-plane The natural undeformed hape of the plate i a rectangle on the xy-plane and repreented by the identity function, r(p) = p Suppoe that the elected point on the plate are required to move to the given poition on the xy-plane The contraint can be forced by minimizing the poition energy, E P (w) = X k kw(p k )? q k k 2 ; where q k i the new poition pecified for a point p k on The parameter control the tightne of the poitional contraint The pline energy of a function w, E S (w) = Z Z 2 w w 4

6 integrate the curvature variation of w over the domain Among the function atifying the poitional contraint, a mooth function w can be obtained by minimizing the pline energy The reulting function w ha continuou firt partial In addition to C 1 -continuity, one-to-one correpondence of a function w can be obtained by minimizing the Jacobian energy, where Z Z E J (w) = (J? 1) 2 @u : The function w i one-to-one on if the Jacobian J i not zero in the interior of and if w i one-to-one on the boundary of [11] Minimizing the Jacobian energy fulfill the firt condition becaue it trie to make J one at each point on On the boundary of, w will be made one-to-one by the boundary condition ued for the numerical olution in Section 32 For a hape w of the plate, the Jacobian J determine the infiniteimal area at a point on [12, 13] It i eay to ee that the Jacobian J i one at every point on when the plate i in it undeformed hape r Hence, the Jacobian energy integrate the area variation of the hape w from the undeformed hape r over the plate The parameter control the reitance of the plate to area variation from the undeformed hape Conequently, the deired function w can be derived by minimizing the energy functional, ED(w) = 1 2 (E S(w) + E J (w) + E P (w)): If a function w minimize the energy functional ED(w), the firt variational derivative of ED(w) mut vanih all over the domain [14] The condition can be repreented by the vector expreion, ED w = 1 ES E 2 w + J E w + P = 0; (6) w where! E S w = w 4 E J w @v!? J ; E P w = 2 (w(p k)? q k ) : Here, w? denote the vector (?y; x) which i perpendicular to the vector w = (x; y) The poition force E P =w appear only at a point p k on for which it poition q k i pecified The partial differential equation given in Equation (6) i called the Euler-Lagrange equation Unfortunately, it i in general very difficult to obtain an analytic olution for the Euler-Lagrange equation Thi ugget a numerical method applied to a dicrete verion of the equation 5

7 32 Numerical olution We dicretize the domain to an M N regular grid and repreent the function w by it value at the node on the grid The poitional contraint are converted to the contraint on the value of the nodal variable The tandard finite difference approximation[15] tranform the differential equation given in Equation (6) into a ytem of equation which conit of MN unknown vector and MN vector equation If the nodal variable compriing the function w are collected into an M N dimenional vector, the ytem can be written in a matrix form, Aw + j(w) + (I 0 w? q) = 0: (7) A i an MN MN matrix which contain the coefficient of the nodal variable reulting from the pline force E S =w j(w) i an MN dimenional vector which approximate the Jacobian force E J =w on the nodal variable I 0 i an MN MN diagonal matrix in which an element i one only if the poitional contraint i aigned to the correponding nodal variable The M N dimenional vector q contain the poitional contraint on the nodal variable We ue the boundary = 2 w=@u 2 2 w=@v 2 = 0 in deriving the matrix A and the vector j(w) To olve Equation (7), we rewrite the equation a a = Aw + j(w) + (I 0 w? q): (8) An initial ditribution w relaxe to an equilibrium olution a t! 1 At the equilibrium, all time derivative vanih and hence w i the olution of Equation (7) When differencing Equation (8) with repect to time, we evaluate the right-hand ide at time t rather than time t? 1, which reult in the implicit Euler cheme For computing the y- and x-component of j(w), x- and y-component of w are aumed contant during a time tep, repectively The aumption make the nonlinear term j(w) linear with repect to w The reulting equation are?(x t? x t?1 ) = Ax t + B(y t?1 )x t + (I 0 x t? x q ) (9)?(y t? y t?1 ) = Ay t + B(x t?1 )y t + (I 0 y t? y q ): (10) x t and y t are the x- and y-component vector of the function w at time t B(y t?1 ) and B(x t?1 ) are MN MN matrice which contain the coefficient of x t and y t in the linear approximation of j(w), repectively x q and y q denote the poitional contraint on x and y The parameter control the tep ize in time Equation (9) and (10) can be arranged in the form, (A + B(y t?1 ) + I + I 0 )x t = Ix t?1 + x q (11) (A + B(x t?1 ) + I + I 0 )y t = Iy t?1 + y q ; (12) in which x t and y t can be calculated from x t?1 and y t?1 The multigrid relaxation method in Section 5 efficiently olve Equation (11) and (12) by exploiting the bandedne of the matrice on the left-hand ide Thi method for olving Equation (8) take the implicit Euler cheme for the pline and poition force and the emi-implicit Euler cheme for the Jacobian force Hence, the olution of the equation can be found very robutly and rapidly with a big time tep The initial hape x 0 and y 0 for Equation (11) and (12) i obtained from an approximate equilibrium olution computed on a hierarchy of coare grid With the initial hape, the equilibrium olution can be derived by olving Equation (11) and (12) in everal time 6

8 Figure 1 how a deformation example in which the grid ize i In the figure, black pot repreent the poition of elected point to which poitional contraint are aigned It take 16 econd to derive the deformation on a SGI Crimon When the ize of the grid i , the computation time increae to 267 econd The value of parameter,, and are 100, , and 00001, repectively Thi example verifie that the propoed method generate a deired deformation very effectively (a) The undeformed hape (b) A deformation of the plate Figure 1: A deformation example 33 Generation of warp function W 0 and W 1 When two image are given, an animator pecifie a et of point pair on the image which repreent the correpondence of feature Let P be a et of point pair (p i ; q i ), where p i and q i are point on the ource and detination image, I 0 and I 1, repectively The warp function W 0 ha to ditort the image I 0 o that each point p i matche the correponding point q i in their poition The requirement for W 1 i to map each point q i to the correponding point p i when ditorting the image I 1 toward I 0 Then, the warp function are reduced to deformation of a rectangular plate which place the pecified point at the given poition There are everal method for deriving a warp function from the poitional contraint aigned to the point on an image In the method, the x- and y-component of a warp function are derived by contructing mooth urface which interpolate cattered point The warp generation in thi approach wa extenively urveyed in [2, 16] In addition, Booktein ued the thin-plate urface model and derived a olution by decompoing a urface into a linear part and independent nonlinear deformation of progreively maller geometric cale[17] Two imilar method were independently propoed which employ the multigrid relaxation method to compute numerical olution of the thin-plate urface[18, 19] However, any of thee method doe not guarantee that the reulting warp function have the one-to-one property The deformation model in Section 31 generate C 1 -continuou and one-to-one warp function from the poitional contraint When a warp function i applied to an image, the one-to-one property guarantee that the ditorted image doe not fold back upon itelf In generating a warp function, the grid ize i choen 7

9 a the reolution of the given image A large value i uually ued for the parameter o that the reulting warp function exactly move feature point to their correpondent For the parameter, a mall value i ufficient to provide an one-to-one warp function 4 Tranition Function Generation Thi ection give the deformable urface model for a tranition function and the way for deriving the tranition function T 0 and T 1 with the urface model 41 The urface model At a given time, the tranition rate of point on an image can be pecified by a real-valued function defined on a rectangular region If we conider a function value a the height from the region, the function can be repreented by a urface deformed only in the vertical direction The deformation technique given in Section 3 i inappropriate for deriving the deformable urface becaue the technique generate a deformation in two dimenion Hence, we reduce the deformation model decribed in Section 31 to the thin plate urface model[14], which i impler and enable a more efficient numerical method The thin plate urface model ha been ued in computer viion to olve the viual urface recontruction problem[10, 20, 21] Let be a rectangular thin plate on the uv-plane and p = (u; v) a point on If the plate i allowed to be deformed only in the direction perpendicular to the uv-plane, a hape of the plate can be repreented by a function, f (p) The function f pecifie a real value for each point on the plate Suppoe that the function f hould have the given value at elected point on the plate A mooth function f which atifie the contraint can be derived by minimizing the energy functional, ES(f ) = 1 2 Z Z 2 2! ! dudv + X k (f (p k )? t k ) 2 1 A : Here, t k i the value pecified for a point p k on If a function f minimize the energy functional ES(f ), the firt variational derivative of ES(f ) mut vanih all over the domain [14] The condition can be repreented by the expreion, ES (f (p k)? t k ) = 0: (13) f The lat term containing appear only at a point p k on for which a value t k i pecified 42 Numerical olution We dicretize the domain to an M N regular grid and repreent a function f by it value at the node on the grid The tandard finite difference approximation tranform Equation (13) into a ytem of linear equation which contain M N unknown and M N equation If the nodal variable compriing the function f are collected into the M N dimenional vector f, the ytem may be written in a matrix form, (A + I 0 )f = t; (14) 8

10 where t i an M N dimenional vector which contain the contraint on the value of f Equation (14) can be efficiently olved by the multigrid relaxation method given in Section 5 Figure 2 how a urface example in which the grid ize i 6464 In the figure, black pot repreent the interpolated value It take 04 econd on a SGI Crimon to generate the urface by the multigrid relaxation method When the ize of the grid i , it computation time i 50 econd Thi how that the multigrid relaxation method i efficient enough for interactive ue Figure 2: A urface example 43 Generation of tranition function T 0 and T 1 To control the tranition behavior in a metamorphoi, an animator elect a et of point on an image and pecify a tranition curve for each point The point et i not necearily the ame a the point et ued for deriving warp function A tranition curve give the tranition behavior of a point over time a hown in Figure 3 Tranition rate Time Figure 3: A tranition curve Let P be a et of point on the ource image I 0 for which tranition curve are pecified Let C(p k ) be the tranition curve for a point p k in P For a given time t, the tranition function T 0 (t) hould have the tranition rate C(p k ; t) at each point p k in P With the et of C(p k ; t) a the contraint on value, uch a tranition function can be derived by the urface model decribed in Section 41 The reulting tranition function i C 1 -continuou and properly propagate the pecified tranition curve all over the image I 0 9

11 The tranition function T 1 i pecified on the detination image I 1 and hould give the tranition behavior which i the ame with T 0 If a point p on I 0 correpond to a point q on I 1, the tranition rate T 1 (q; t) hould be the ame with T 0 (p; t) for each time t Hence, the tranition function T 1 (t) can be derived by ampling T 0 (t) with the warp function W 1, that i, T 1 (q; t) = T 0 (W 1 (q); t) When a equence of inbetween image i generated with tranition function, a urface hould be contructed for determining the function T 0 (t) at each time t In thi cae, the olution for a urface i ued for the initial olution of the urface at the next time tep Becaue the urface change moothly with time, thi approach provide a good initial olution and reduce the computation time 5 Multigrid Relaxation Method Equation (11), (12), and (14) are linear ytem in the form, Af = b; where A i an MN MN matrix, f i an MN dimenional unknown vector, and b i an MN dimenional vector Due to the local nature of a finite difference dicretization, A ha the nice computational propertie uch a parene and bandedne Many type of algorithm have been developed for olving a pare linear ytem Relaxation algorithm uch a Jacobi, Gau-Seidel, or ucceive-overrelaxation method exploit the parene and bandedne of the matrix A to efficiently olve the ytem of equation[15] In a relaxation, the value of each node i updated with a local computation to atify the equation for that node The iteration of relaxation generate a equence of approximate olution which converge aymptotically to the exact olution A major drawback of a relaxation cheme i that it converge lowly in general The multigrid approach wa developed to overcome the drawback and ha been actively reearched by the numerical analyi community[22, 23] Terzopoulo firt applied the multigrid approach to derive a thin-plate urface for olving the viual urface recontruction problem in computer viion[21] The multigrid approach applie the idea of neted iteration and coare grid correction to a hierarchy of grid[22] The neted iteration i the way to improve a relaxation cheme with a good initial gue To obtain an improved initial gue, the neted iteration perform preliminary iteration on a coare grid and then ue the reulting approximation a an initial gue on a fine grid Relaxation on a coarer grid are le expenive ince there are fewer unknown to be updated The coare grid correction accelerate the peed of convergence baed on an analyi of the error reduction behavior The analyi how that the high frequency component of an error are hort-lived while it low frequency component perit through many iteration The important point i that a low frequency on a fine grid may turn into a high frequency on a coare grid When a relaxation begin to tall, ignalling the predominance of low frequency error, the coare grid correction move the relaxation to a coarer grid, on which thoe error appear more ocillatory, and thu the relaxation will be more effective If g i an approximation to the exact olution f, then the error e = f? g atifie the reidual equation, Ae = r = b? Ag: Once we get the error e, the exact olution f can be immediately derived by f = g + e The following recurive algorithm incorporate the idea of coare grid correction with the reidual equation by relaxing the error on a hierarchy of grid In the algorithm, the upercript h denote the inter-node pacing of a grid The 10

12 matrix A h on a grid h i the approximation of the matrix A on the finet grid The vector f h, g h, and b h comprie the correponding nodal variable on the grid h I 2h h denote the decimation of a vector from a finer grid h to a coarer grid 2h while I h 2h i the interpolation in the oppoite direction 1 and 2 are the parameter for controlling the number of relaxation V-Cycle Algorithm g h MV h (g h ; b h ) 1 Relax 1 time on A h f h = b h with a given initial gue g h 2 If h i the coaret grid, then go to 4 Ele b 2h I 2h h (bh? A h g h ) g 2h 0 g 2h MV 2h (g 2h ; b 2h ) 3 Correct g h g h + I h 2h g2h 4 Relax 2 time on A h f h = b h with initial gue g h The algorithm telecope down to the coaret grid and then walk it way back to the finet grid Figure 4(a) how the chedule for the grid in the order in which they are viited Becaue of the pattern of thi diagram, thi algorithm i called the V-Cycle 8h h 2h 4h 2h 4h 8h h on 2h V-Cycle V-Cycle on h V-Cycle on 4h (a) (b) Figure 4: Relaxation chedule for (a) V-Cycle (b) Full Multigrid V-Cycle, all on four level The idea of neted iteration can enhance the V-Cycle algorithm by uing a hierarchy of grid to provide an improved initial gue on the finet grid The following recurive algorithm how the final multigrid relaxation cheme which incorporate the idea of neted iteration and coare grid correction on a hierarchy of grid Full Multigrid V-Cycle Algorithm g h F MV h (g h ; b h ) 1 If h i the coaret grid, then go to 3 Ele b 2h I 2h h (bh? A h g h ) g 2h 0 11

13 g 2h F MV 2h (g 2h ; b 2h ) 2 Correct g h g h + I2h h g2h 3 g h MV h (g h ; b h ) The Full Multigrid V-Cycle algorithm tart at the finet grid Figure 4(b) how the cheduling of relaxation on grid Each V-Cycle i preceded by a maller V-Cycle deigned to provide a better initial gue Scheme for relaxation and interpolation are required to implement the Full Multigrid V-Cycle algorithm The Gau-Seidel method i alway twice uperior to Jacobi method in the peed of convergence The ucceive-overrelaxation method how untable approximation in the firt few iteration though it i fater than the other Becaue we take mall number of relaxation in the V-Cycle algorithm, the Gau- Seidel method i choen a the relaxation cheme When a vector v i interpolated from a coarer grid 2h to a finer grid h, we ue the Catmull-Rom pline interpolation[24] which guarantee C 1 -continuity The decimation of vector v from h to 2h i done by weighted averaging the value of neighborhood node That i, v 2h i;j = 1 16 [vh 2i?1;2j?1 + vh 2i?1;2j+1 + vh 2i+1;2j?1 + vh 2i+1;2j+1 + 2(v h 2i;2j?1 + vh 2i;2j+1 + vh 2i?1;2j + vh 2i+1;2j ) + 4vh 2i;2j ]; (15) where v i;j denote the value of the vector v at the node (i; j) When the relaxation i performed on a coare grid 2h, the matrix A 2h hould be approximated from the matrix A h on the fine grid h A row in the matrix A h contain the coefficient of an equation which i olved for a nodal variable on h in a relaxation For a node on 2h, we derive the row in the matrix A 2h by taking the weighted average of the related row in the matrix A h Each coefficient for a node on 2h i computed by applying formula (15) to the correponding coefficient for it neighborhood node on h In the general multigrid approach, the convergence rate and error analyi are performed to determine the number of relaxation on a grid[23] The relaxation on a grid move to a coarer grid whenever the convergence rate low down and end a oon a the etimated error i le than a given bound It ha been theoretically proven that the multigrid approach require O(M N ) operation to reduce the error to the truncation error level[22] We implemented the Full Multigrid V-Cycle algorithm in which the parameter 1 and 2 control the number of relaxation The computational effort can be calculated in term of the work unit W which i defined a the amount of computation required for one relaxation on the finet grid The neceary computational effort i le than 7=2( )W [22] Becaue mall 1 and 2 are uually ufficient for deriving a atifactory olution, thi i a great enhancement compared to the conventional relaxation cheme 6 Extenion 61 Fat approximation of a warp function In thi paper, a warp function w i derived by numerically olving Equation (7) When the parameter i zero, Equation (7) reduce to the linear ytem, (A + I 0 )w = q: (16) Equation (16) can be rapidly olved by deriving the x- and y-component of w with the multigrid relaxation method given in Section 5 When image are not heavily ditorted, Equation (16) can be ued for generating 12

14 mooth warp function which exactly reflect the feature correpondence[18] However, the warp function may not be one-to-one epecially when they contain large local ditortion a in Figure 1 62 A warp function with a fixed boundary When a warp function i applied to an image, there may be hole near a boundary of the ditorted image Thee hole are generated if the boundary of the original image i mapped to the inide of the ditorted image The problem may be overcome by eparating background from object However, if an object i attached to a boundary, the boundary need to be frozen over the metamorphoi A frozen boundary can be obtained by adjuting the boundary condition in computing a warp function For the horizontal boundarie, we make the y-component of a warp function w equal to that of the undeformed hape r Then, the point on the boundarie can move only in the horizontal direction The vertical boundarie can be handled imilarly 63 General feature control primitive The multigrid relaxation method i ued to olve a linear ytem in generating a warp function The number of poitional contraint hardly affect the computation time required in the multigrid relaxation method Hence, the total computation time remain nearly contant regardle of the number of feature point Thi trong merit make it poible to eaily extend the feature control primitive to include line egment and curve When a pair of line egment are pecified to etablih the correpondence of feature between image, a dicretization of the line egment generate a et of correponding point pair When curve are ued to control feature correpondence, the Catmull-Rom pline curve[25] are adopted to interpolate the point pecified by an animator By properly dicretizing the parameter pace and computing the point on the curve, we get a et of correponding point lying on the matching curve Thee generalized primitive can alo be ued for controlling tranition function 64 Procedural tranition function The preented image morphing technique alway generate an inbetween image on which the feature point pair on two image match their poition regardle of tranition function Therefore, procedural tranition function can be ued to generate variou intereting inbetween image For example, let the tranition function T 0 be defined by T 0 (u; v; t) = ( 2t(1? u=u max ); if 0 t 1; 2 1 1? 2(1? t)u=u max ; if < t 1: 2 T 0 generate a equence of inbetween image in which the ource image gradually change to the detination image from left to right The correponding tranition function T 1 i derived by ampling T 0 with the warp function W 1 13

15 7 Experimental Reult 71 Comparion with field morphing Meh warping[2] provide a fat and intuitive technique for deriving warp and can be eaily upported by hardware The method of Nihita et al[3] can produce variou type of warp which are mooth up to the deired degree However, thee meh-baed method have a drawback in pecifying the feature on an image The diadvantage of meh warping are fully detailed in [1], mot of which are alo applied to the method of Nihita et al Field morphing[1] i comparable to the warp generation technique in thi paper in that the feature can be effectively pecified by line egment In thi ection, example of warp are provided which how the advantage of the propoed technique over field morphing To generate a warp with the propoed technique, the point on line egment are ampled a mentioned in Section 63 Figure 5(a) i the original image in which a letter F lie on a meh We overlay line egment on the image and move them to obtain ditorted image Figure 5(b) i generated when the warp i computed by the field morphing technique In the image, the lower bar in F doe not hrink in the amount pecified by the movement of line egment The right end of the upper bar how a ditortion while the line egment on it i fixed Thee abnormal ditortion reult from that the effect of two or more line egment are blended by imple weighted averaging In Figure 5(c), the warp i computed by the technique in thi paper Figure 5(c) exactly reflect the movement of line egment and how proper ditortion over the entire image In Figure 5(d) and 5(e), the letter F in Figure 5(a) i ditorted to obtain the letter T Field morphing generate the image in Figure 5(d), which doe not how the deired ditortion In the image, the influence of line egment crumble each other, and the diplacement of any line egment i not properly reflected In contrat, the propoed technique give the exact ditortion a hown in Figure 5(e) Figure 5(f) and 5(g) how the image obtained when the image in Figure 5(a) i ditorted to the letter P An image in Figure 5 i of ize and it take 191 econd for the propoed technique to generate a ditorted image on a SGI Crimon The field morphing technique require 5195 econd to generate a ditorted image When the number of line egment get larger, the computation time increae in field morphing while the time remain nearly contant in the propoed technique 72 Metamorphoi example Figure 6 how a metamorphoi example Figure 6(a) i a face image of the firt author of thi paper, and Figure 6(b) i an image of a cat Figure 6(c) and 6(d) how the feature pecified on the image Figure 6(e) i the middle image in which the ame tranition rate i applied to all part Figure 6(f) i an inbetween image in which tranition rate are different from part to part The eye, noe, and mouth of the image look more like the human face than the remaining part Figure 6(g) and 6(h) are example of applying procedural tranition function explained in Section 64 T 0 (u; v; t) = u=u max and T 0 (u; v; t) = (in(4u=u max )+1)=2 are ued for Figure 6(g) and 6(h), repectively In Figure 7, an inconitency of feature between image i overcome by controlling the tranition behavior Figure 7(a) and 7(b) how face image which are coniderably different near the ear We pecify the correpondence of feature in Figure 7(c) and 7(d) Without tranition control, the ear and hair are jumbled up in the middle image a hown in Figure 7(e) To obtain a better inbetween image, we elect the part near the ear in Figure 7(f) and aign the tranition curve in Figure 7(i) The tranition curve in Figure 7(j) i pecified for the line egment in the middle of Figure 7(f) Figure 7(g) how the inbetween image at time 047 where the tranition rate are computed from the pecified tranition curve In the image, the part near 14

16 (a) (b) (d) (f) (c) (e) (g) Figure 5: Comparion of warp: (a) i the original image; (b), (d), and (f) are from field morphing; (c), (e), and (g) are from the propoed technique the ear reemble the firt image Figure 7(h) i the inbetween image at time 053 on which the econd image dominate the part near the ear In Figure 8, two different facial expreion of a peron are interpolated to obtain a facial animation The econd image in the upper and lower row are the ource and detination image, repectively The firt image in the row how the pecified feature correpondence The other are generated inbetween image They demontrate that the feature are nicely controlled by the propoed technique For example, the motion of the mouth look natural in the inbetween image 73 Performance We ue a worktation SGI Crimon(R4400) to generate the example in thi paper The reolution of an image in Figure 6 i It take 192 econd to derive a warp on a grid Hence, about 384 econd are taken to generate the image in Figure 6(e) When the tranition rate are different from part to part in an inbetween image, a deformable urface hould be contructed to compute tranition function It take 37 econd to generate the urface on a grid Then, about 421 econd are taken to obtain the image in Figure 6(f) Except the evaluation of procedural tranition function, the computation required for the image in Figure 6(g) and 6(h) i the ame a that for the image in Figure 6(e) An image in Figure 7 i , and it take 181 econd and 37 econd to derive a warp and urface, repectively Each image in Figure 8 i , and it take 172 econd to generate a warp Once the warp are derived, an inbetween image can be obtained in le than one econd All metamorphoi example in thi ection are directly derived from the given image An inbetween image i generated without a making proce which extract object from the background No additional manipulation are taken to enhance the generated inbetween image To prevent hole near a boundary of a 15

17 (a) (b) (c) (d) (e) (f) (g) (h) Figure 6: A metamorphoi example: from a peron to a cat 16

18 (a) (b) (c) (d) (e) (f) Tranition rate 1 (g) Tranition rate 1 (h) (i) Time 0 (j) 1 Time Figure 7: Tranition control for overcoming an inconitency between feature 17

19 Figure 8: A facial animation ditorted image, a warp function i computed by freezing the boundary a decribed in Section 62 General primitive of line and curve mentioned in Section 63 enable an animator to efficiently pecify the feature correpondence between two image In deriving an inbetween image, the poition of feature are repeatedly adjuted until the deired metamorphoi i obtained Becaue the warp computed in thi paper preciely reflect the pecified feature correpondence, the iterative proce can be uccefully completed in a few trial Each metamorphoi example in thi ection wa generated from the given image in le than one hour 8 Concluion Thi paper preent a new approach to image morphing in deriving a warp and controlling tranition behavior We develop a two dimenional deformation technique to generate a warp from a et of feature point pair overlaid on two image The reulting warp i C 1 -continuou and one-to-one and preciely reflect the feature correpondence between the image Becaue any tructure uch a a meh i not neceary for feature pecification, an animator enjoy freedom in deigning a metamorphoi The freedom together with good warp make it poible to obtain a deired inbetween image very effectively We eparate the tranition behavior control from the feature interpolation in generating a metamorphoi equence The eparation reult in a method which i much eaier to ue and more effective than the previou technique A tranition cenario i realized by pecifying the tranition curve for elected point on an image In addition, more intereting tranition behavior can be derived by procedural tranition function The multigrid relaxation method i taken to olve a linear ytem in deriving a warp or tranition rate Thi method how a great enhancement in computation time compared to the conventional relaxation cheme With the table numerical method for a differential equation and the multigrid relaxation method, the preented image morphing technique i fat enough for an interactive environment 18

20 The mot tediou part of image morphing i to etablih the correpondence of feature between image by an animator Technique of computer viion may be employed to automate thi tak An edge detection algorithm can provide important feature on image, and an image analyi technique may be ued to find the correpondence between detected feature One of the mot challenging problem in image morphing i to develop an efficient method for pecifying feature and their correpondence, epecially in the morphing between two image equence Reference [1] T Beier and S Neely Feature-baed image metamorphoi Computer Graphic, 26(2):35 42, 1992 [2] G Wolberg Digital Image Warping IEEE Computer Society Pre, 1990 [3] T Nihita, T Fujii, and E Nakamae Metamorphoi uing Bézier clipping In Proceeding of the Firt Pacific Conference on Computer Graphic and Application, page , Seoul, Korea, 1993 World Scientific Publihing Co [4] D Terzopoulo, J Platt, A Barr, and K Fleicher Elatically deformable model Computer Graphic, 21(4): , 1987 [5] D Terzopoulo and K Fleicher Modeling inelatic deformation: Vicoelaticity, platicity, fracture Computer Graphic, 22(4): , 1988 [6] J C Platt and A H Barr Contraint method for flexible model Computer Graphic, 22(4): , 1988 [7] Silicon Graphic Inc Graphic Library Programming Guide [8] G Celniker and D Goard Deformable curve and urface finite-element for free-form hape deign Computer Graphic, 25(4): , 1991 [9] W Welch and A Witkin Variational urface modeling Computer Graphic, 26(2): , 1992 [10] D Terzopoulo Regularization of invere viual problem involving dicontinuitie IEEE Tranaction on Pattern Analyi and Machine Intelligence, PAMI-8(4): , 1986 [11] G Meiter and C Olech Locally one-to-one mapping and a claical theorem on chlicht function Duke Mathematical Journal, 30:63 80, 1963 [12] R C Buck Advanced Calculu McGraw-Hill, third edition, 1978 [13] M P do Carmo Differential Geometry of Curve and Surface Academic Pre, econd edition, 1990 [14] I Gelfand and S Fomin Calculu of Variation Prentice-Hall, 1963 [15] W H Pre, S A Teukolky, W T Vetterling, and B P Flannery Numerical Recipe in C Cambridge Univerity Pre, econd edition, 1992 [16] D Ruprecht and H Müller Image warping with cattered data interpolation method Reearch Report 443, Fachbereich Informatik der Univerität Dortmund, Dortmund, Germany,

21 [17] F L Booktein Principal warp: Thin-plate pline and the decompoition of deformation IEEE Tranaction on Pattern Analyi and Machine Intelligence, 11(6): , 1989 [18] S Y Lee, K Y Chwa, J Hahn, and S Y Shin Image morphing uing deformable urface In Proceeding of Computer Animation 94, page 31 39, Geneva, Switzerland, 1994 IEEE Computer Society Pre [19] P Litwinowicz and L William Animating image with drawing In SIGGRAPH 94 Conference Proceeding, page ACM Pre, 1994 [20] W Grimon An implementation of a computational theory of viual urface interpolation Computer Viion, Graphic, and Image Proceing, 22:39 69, 1983 [21] D Terzopoulo Multilevel computational procee for viual urface recontruction Computer Viion, Graphic, and Image Proceing, 24:52 96, 1983 [22] W L Brigg A Multigrid Tutorial SIAM, Lancater Pre, Lancater, PA, 1987 [23] A Brandt Multi-level adaptive olution to boundary-value problem Mathematic of Computation, 31(138): , 1977 [24] D H Kochanek and R H Bartel Interpolating pline with local tenion, continuity, and bia control Computer Graphic, 18(3):33 41, 1984 [25] G Farin Curve and Surface for Computer Aided Geometric Deign Academic Pre, econd edition,