Structure Preserving Manipulation and Interpolation for Multi-element 2D Shapes

Transcription

1 DOI: /j x Pacfc Graphcs 2012 C. Bregler, P. Sander, and M. Wmmer (Guest Edtors) Volume 31 (2012), Number 7 Structure Preservng Manpulaton and Interpolaton for Mult-element 2D Shapes Wenwu Yang 1,2 Jeqng Feng 1 Xun Wang 2 1 State Key Laboratory of CAD&CG, Zhejang Unversty 2 Zhejang Gongshang Unversty Abstract Ths paper presents a method that generates natural and ntutve deformatons va drect manpulaton and smooth nterpolaton for mult-element 2D shapes. Observng that the structural relatonshps between dfferent parts of a mult-element 2D shape are mportant for capturng ts feature semantcs, we ntroduce a smple structure called a feature frame to represent such relatonshps. A constraned optmzaton s solved for shape manpulaton to fnd optmal deformed shapes under user-specfed handle constrants. Based on the feature frame, local feature preservaton and structural relatonshp mantenance are drectly encoded nto the objectve functon. Beyond deformng a gven mult-element 2D shape nto a new one at each key frame, our method can automatcally generate a sequence of natural ntermedate deformatons by nterpolatng the shapes between the key frames. The method s computatonally effcent, allowng real-tme manpulaton and nterpolaton, as well as generatng natural and vsually plausble results. Categores and Subject Descrptors (accordng to ACM CCS): I.3.3 [Computer Graphcs]: Geometry Shape Deformaton 1. Introducton 2D shape deformaton s useful n many applcatons, such as real-tme lve performance and enrchng graphcal user nterfaces. Ths technque can also be commonly found n commercal software for vdeo edtng or 2D vector-based anmaton, such as Adobe After Effects, Adobe Flash, or Toon Boom Studo. A general representaton of a 2D shape s the usage of a smple polygon to represent the boundary of the shape, whle the nteror of the shape s descrbed usng the nteror texture of the polygon. However, n real-world nputs arsng from practcal demands, dfferent meanngful parts of a shape are not only dsjoned but also often overlapped,.e., occluson occurs (see the left column of Fg. 1(a)). For such a shape, a smple polygon s nsuffcent to fully capture ts semantc boundares, such that the dfferent parts of the shape may fal to move separately durng deformaton (see Fg. 1(c)). Mult-element 2D shapes: In practcal applcatons such as vector-based cartoon anmaton, artsts create a complex Correspondence: jqfeng@cad.zju.edu.cn cartoon character typcally by drawng dfferent parts of the character n dfferent layers that are then combned together. Smlarly, we propose a mult-element 2D shape representaton whch can correctly descrbe the semantc boundares of the shapes lke that n Fg. 1(a). To create a mult-element 2D shape, the user needs to draw a closed or open curve to represent the boundary of each meanngful part of the shape (see the rght column of Fg. 1(a)), where each curve s called a sub-shape. Wth the user-specfed layer levels [SSJ 10], all of the sub-shapes are then layered together to form the mult-element 2D shape. Recently, a varety of 2D shape deformaton algorthms that model the rgdty of the shapes [IMH05, WXW 06] have been ntroduced. These methods allow the user to drectly manpulate a shape through a clck-and-drag nterface and are able to generate ntutve and physcally plausble deformatons for varous 2D shapes [IMH05]. Whle these methods work well for the 2D shapes whose boundary can be represented by a smple closed polygon, they cannot be appled to the mult-element 2D shapes straghtforwardly. For example, n the method of [IMH05], a smple closed polygon s used to descrbe the shape boundary so that a trangulated mesh can be generated nsde the boundary: the deformaton Computer Graphcs Forum c 2012 The Eurographcs Assocaton and Blackwell Publshng Ltd. Publshed by Blackwell Publshng, 9600 Garsngton Road, Oxford OX4 2DQ, UK and 350 Man Street, Malden, MA 02148, USA.

2 2250 W. Yang & J. Feng & X. Wang / Manpulaton and Interpolaton for Mult-element 2D Shapes (a) (b) where the dfferent shape parts are allowed to deform separately and the structural relatonshps between them are well preserved. Ths method s motvated by the observaton that the structural relatonshps between dfferent parts of a mult-element 2D shape are mportant for capturng ts feature semantcs. The key dea s to ntroduce a smple structure whch we call the feature frame to represent these structural relatonshps. The feature frame allows us to preserve the feature semantcs of the nput mult-element 2D shape durng subsequent manpulaton and nterpolaton (see Fg. 1(b)). Addtonally, the feature frame s closely related to the shape geometry and naturally reflects the local characterstcs of the shape, thus makng t convenent to retan local shape features. Furthermore, the feature frame s easy to construct and s applcable to mult-element 2D shapes of arbtrary subjects. (c) Fgure 1: Manpulaton of a cartoon boy. (a): nput object (left) and the sub-shapes (rght); (b): our manpulaton approach; (c): the method of Igarash et al. [IMH05], where the nput object s enclosed by a smple polygon and the nteror of the polygon s trangulated (rght); (d): the approach of [YF09a], where the semantc features on the face of the boy and around hs neck are dstorted. s frst conducted on the trangle mesh, and then the orgnal shape s mapped from the orgnal mesh to the deformed mesh. As shown n Fg. 1(c), snce the trangulaton corresponds to a connected manfold mesh, the sub-shapes of a mult-element 2D shape that are spatally overlapped but semantcally separated wll be ted together, possbly leadng to unnatural deformaton results. Furthermore, the deformaton flexblty of the mult-element 2D shape would be reduced, snce a sngle trangulated mesh could not reveal the separately movng parts of the shape. One possble approach to address the above problems would be to connect all of the sub-shapes as a whole usng lne segments, where vertces connected by a lne segment are consdered as neghbors, and generate a trangulated mesh for each sub-shape, and then conduct the deformaton on the trangulaton of each sub-shape separately. A smlar work had been done n [YF09a]. However, ths approach does not account for the structural relatonshps between dfferent sub-shapes, whch would napproprately dstort semantc features n underlyng subjects (see Fg. 1(d)). In ths paper, we present a method that generates natural and ntutve deformatons for mult-element 2D shapes, (d) To manpulate a mult-element 2D shape, the user places handles on the shape and moves the handles around (see Fg. 1(b)). Beyond deformng the nput mult-element 2D shape nto a new one at each key frame, the method can nterpolate the shapes between each two successve key frames to generate a sequence of ntermedate deformatons. In combnaton, the method of manpulaton and nterpolaton for multelement 2D shapes enables the user to create vsually pleasng anmatons from sngle nput mult-element 2D shapes of any subjects. The man contrbutons of ths paper are: At frst, we ntroduce a smple structure called the feature frame for encodng the structural relatonshps between parts of a mult-element 2D shape. Based on the feature frame, we then develop a unfed framework for the manpulaton and nterpolaton of mult-element 2D shapes. 2. Related work Our work s closely related to the technques of 2D shape manpulaton and 2D shape nterpolaton: we cover relevant references below. Shape manpulaton. Many deformaton technques have been developed for shape manpulaton. Skeletondrven technque uses a pre-defned skeleton to manpulate the nput shape [LCF00, YN03]. It assocates each pont of the shape wth one or more coordnate frames, each defned by a skeleton bone, and then changes the overall shape relatve to the skeleton that s manpulated by the user. However, bndng a shape to a skeleton, ether manually or automatcally, s stll a challengng task, especally for the shapes that lack an obvous jonted structure. In FFD, a space deformaton s defned va a smple control object for shape manpulaton [GB08]. The user manpulates the control object and then the system nterpolates the deformaton of the control object to the entre space. However, the space warp s

3 W. Yang & J. Feng & X. Wang / Manpulaton and Interpolaton for Mult-element 2D Shapes 2251 generally defned n a pure mathematcal manner (e.g., under the crtera of smoothness and contnuty), oblvous to the geometrc or structural characterstcs of the nput shape. Thus, the feature semantcs of the shape that s deformed by the space warp could be easly destroyed. In recent years, a varety of 2D shape deformaton algorthms that model the rgdty of the shapes have been ntroduced [IMH05, WXW 06, SDC09]. These methods generate natural deformaton results by mnmzng local shape dstortons. (These approaches can be regarded as varants of detal-preservng mesh deformaton technques [YZX 04, SCOL 04, BS08].) As mentoned n the ntroducton, these methods, however, are not sutable for the mult-element 2D shapes. Hornung et al. [HDK07] ntroduce an augmented verson of the as-rgd-as-possble shape manpulaton technque to the context of deformng mult-element 2D shapes. However, ther method needs the adjacent subshapes to share common boundary vertces n order to convey the mpresson of a connected object, thus s unsutable for the shapes wth dsjoned or overlapped parts. Yang et al. [YF09a] unleash ths vertex-sharng lmtaton by usng lne segments to connect dsjoned sub-shapes, and conduct the deformaton of each sub-shape separately. Our work on mult-element 2D shape manpulaton s also free of ths lmtaton. Moreover, our method takes nto account the structural relatonshps between parts of the shape, whch s crucal n preservng the semantc features n the underlyng subjects (see Fg. 1(b) and (d)). Further related to our work, Gal et al. [GSMCO09] and Zheng et al. [ZFCO 11] show that when edtng manmade 3D objects, hgh-level structures (e.g., symmetry, parallelsm) can be effectvely mantaned usng an analyzeand-edt approach. Xu et al. [XWY 09] preserve the nature between components of 3D mechancal objects usng a jont-aware deformaton framework. However, nether of these methods demonstrates how to extend to the case of mult-element 2D shapes for preservng the structural relatonshps between parts. Ho et al. [HKT10] and Zhou et al. [ZXTD10] represent spatal relatonshps between object parts usng edges that connect vertces between close components. However, the vertex edges may connect the parts that are spatally close but semantcally separated, and how to correct such napproprate part connecton wth a lght work has not been mentoned n ther papers. In ths work, we defne a smple and effectve structure called the feature frame to represent the structural relatonshps between parts of a mult-element 2D shape. The feature frame allows the user to modfy alternatve semantc relatons wth a very small amount of assstance. Our work s smlar to the method of Sumner et al. [SSP07], where a deformaton graph s used to buld the local transformatons of the nput objects. The key dfference s that our feature frame focuses on the structural relatonshps between parts, whle such relatonshps have not been dealt wth n the deformaton graph. Shape nterpolaton. Shape nterpolaton nvolves the creaton of a smooth transton from a source shape to a target shape [WNS 10]. Because smple lnear nterpolaton of correspondng vertces tends to generate ntermedate shapes wth shrnkages, Sederberg et al. [SGWM93] propose to nterpolate the ntrnsc propertes (e.g., edge length, vertex angle) of the nput shapes. To preserve local volumes, the nteror of the shapes s further taken nto account by applyng the blend to the compatble trangulatons of the nput objects [ACOL00,BIBA09]. However, these methods are prmarly desgned for 2D shapes that are represented by a sngle polygon and are not sutable for the nterpolaton of mult-element 2D shapes, snce they do not account for the structure relatonshps between shape parts. Addtonally, n the nteror-consderng methods, the automatc creaton of compatble trangulatons s nvolved, and though possble for the shapes that are represented by a smple polygon, t s stll a challengng work for mult-element 2D shapes where all sub-shapes should be taken nto account. Motvated by the nterpolaton technques that are based on mult-resoluton representatons [HBCC07], Yang et al. [YF09b] ntroduce a herarchcal nterpolaton approach. The approach decomposes the nput shapes nto a par of coarse polygons and several pars of correspondng features, and then nterpolates the coarse polygons and feature pars usng the nteror-consderng method [ACOL00] and ntrnsc method [SGWM93], respectvely. Ths approach acheves pleasng results: not only the overall shapes of the nput objects are blended naturally, but also the detals of the nput objects are well preserved. Our work on shape nterpolaton s smlar to ths method, but extends to the context of nterpolatng mult-element 2D shapes, where preservng the feature semantcs of the nput objects s mportant n obtanng natural results. 3. Feature Frame Ths secton presents the detals of feature frame constructon and establshes the assocaton between the nput multelement 2D shape and ts feature frame Feature frame constructon Intutvely, the vsual appearance or structure of a shape s manly determned by ts salent features. Therefore, t s reasonable to delegate the structural relatonshps between parts of a mult-element 2D shape to ts salent features. Then, the feature frame that represents these structural relatonshps s constructed through the followng steps: Feature extracton. To extract the salent features, we dentfy feature ponts those that attract more attenton n human percepton than others. For a mult-element 2D shape, t s reasonable to assgn hgh vsual salence to the cusp, nflecton, and curvature extrema ponts of ts subshapes, as well as the ntersecton ponts between the subc 2012 The Author(s)

4 2252 W. Yang & J. Feng & X. Wang / Manpulaton and Interpolaton for Mult-element 2D Shapes shapes and the end ponts of each of open sub-shapes. In our mplementaton, a smple and effcent algorthm of Chetverkov and Szabo [CS99] s used to detect the cusp, nflecton, and curvature extrema ponts of the sub-shapes. These feature ponts then segment each sub-shape nto several salent features, each of whch corresponds to a curve segment that s delmted by two successve feature ponts. An example of the feature ponts and salent features s demonstrated n the rght sub-fgure of Fg. 2(a). (a) (b) Sub-frame generaton. Intutvely, the structural relatonshps between salent features are determned by ther spatal postons and orentatons. For each sub-shape, we sequentally connect ts feature ponts to form a polylne or a polygon, as llustrated by the thn lne segments n Fgure 2(d). We call the polylne or polygon as a sub-frame snce t seems lke a scaffoldng of the correspondng sub-shape. Obvously, each edge of the sub-frame corresponds to a salent feature and thus s called a feature lne for clarty. It s clear that each feature lne descrbes the poston, sze, and orentaton of the assocated feature n general. Hence, for each sub-shape a sub-frame can be used to encode the structural relatonshps between ts salent features. Sub-frame sttchng. Durng deformaton, sub-shapes of the nput mult-element 2D shape cannot move ndependently, but have to be sttched together to convey the mpresson of a connected body. Due to the one-to-one correspondence between the salent features and the feature lnes, the sttchng can be establshed by connectng sub-frames together. Moreover, to mantan the deformaton flexblty of a mult-element 2D shape, ndvdual sub-shapes should be allowed to move separately. Hence, sub-frames are expected to be connected together, but n a loose way. We can model the problem as graph connecton. The graph contans one node N per sub-frame SF, wth an edge (, j) between each par of nodes N and N j. The edge length of (, j) s defned as the dstance between the sub-frames SF and SF j : d(sf,sf j ) = mn v,l v j,m where v,l and v j,m are the vertces of SF and SF j respectvely. The problem s then to decde when a par of nodes are to be connected n the graph so that the graph s connected wth mnmum edges. The Eucldean Mnmum Spannng Tree (EMST) solves the problem and tends to connect subframes that are near each other. For each edge n the EMST, we put a lne segment to connect the vertces of the correspondng sub-frames that defne the edge length. For clarfy, we call ths lne segment as a sewng lne, to dstngush from the feature lnes of subframes. An example s shown n the left sub-fgure of Fg. 2(c), where the output ncludes only one sewng lne, whch s labeled as SL 1 (the sewng lnes SL 2 and SL 3 are mposed by the user later, as mentoned below). Obvously, (c) Fgure 2: Feature frame constructon. (a): nput shape and ts feature ponts (rght: crcular dots); (b): deformaton result wth only sewng lne SL 1 ; (c): the automatcally generated sewng lne SL 1 and the manually specfed sewng lnes SL 2 and SL 3 (left), as well as deformaton result wth the sewng lne SL 1 and the addtonal sewng lnes SL 2 and SL 3 (rght); (d): The fnal feature frame. through the sewng lne SL 1, all parts of the butterfly (left and rght antenna and body) are sttched together. On the other hand, each sewng lne s also used to ndcate that the structural relatonshps between the parts, whch are connected by the sewng lne, should be preserved durng deformaton. However, the EMST over the sub-frames may not be suffcently dense n edges to serve the purpose. Moreover, whle the sewng lnes derved from the EMST s edges tend to connect neghborng parts, they may jon together the parts that are spatally close but semantcally separated. We let the user manually modfy the sewng lnes for desred deformaton effects. The modfcaton process s smple and ntutve, where the user only needs to add, delete or move the sewng lnes between shape parts. If the local vsual appearance or structure between two parts of the shape are desred to be preserved durng deformaton (e.g., the left antenna and the left upper body or the rght antenna and the rght upper body n Fg. 2(c)), a sewng lne (e.g., SL 2 or SL 3 n Fg. 2(c)) s then put by the user between the nearly mddle ponts of the two parts. The effect s demonstrated by comparng the deformaton results wthout and wth the addtonal sewng lnes, as shown n Fg. 2(b) and the rght sub-fgure of Fg. 2(c), respectvely. In our mplementaton, for each end vertex of the new sewng lne, f ts dstance to an exstng feature pont s less than a threshold, the two ponts wll be merged; otherwse t wll be nserted as a new feature pont on the nvolved sub-shape and the correspondng sub-frame s updated accordngly. We use a dstance threshold of a quarter (d)

5 W. Yang & J. Feng & X. Wang / Manpulaton and Interpolaton for Mult-element 2D Shapes 2253 of the average edge length of the nput shape for the examples n the paper. Smlarly, the user can unte the shape parts by deletng redundant sewng lnes between them. Dscusson: In our experments, the modfcaton of sewng lnes seems ntutve to the user, snce only necessary connectons between the parts, whch typcally correspond to the semantc features of the shape, are nvolved. Moreover, the automatcally generated sewng lnes from the EMST over the sub-frames wll connect the neghborng parts. Hence, a small amount of user nteracton s suffcent n general. As shown n Table 1, for all examples shown, the number of sewng lnes s wthn 20, where only several of them (2-6) are specfed by the user and the nteracton tme s less than 2 mnutes. It bears notng that the sewng lnes can be adjusted on the fly durng the shape manpulaton, whch provdes the user ntutve and flexble control of the deformaton results. Fnally, the feature frame, whch represents the structural relatonshps between parts of a mult-element 2D shape, s formed by the feature and sewng lnes as well as ther vertces, as llustrated n Fg. 2(d). Notaton. Throughout, we denote 2-vectors by boldface lower case letters, whle we denote 2 2 matrces by boldface upper case letters. Furthermore, we denote a feature frame vertex usng an ndex, e.g.,. Smlarly, we also denote a feature or sewng lne usng an ndex, e.g., j. Specfcally, v and v denote the ntal and deformed postons of each feature frame vertex, [1...q], respectvely. An example of feature frame vertces s shown n Fg. 2(d),.e., the crcular dots. v j (t) and v j (t), t [0,1], denote the ntal and deformed postons of each pont of the feature or sewng lne j, respectvely. An example of feature and sewng lnes s shown n Fg. 2(d),.e., the thn and thck lne segments, respectvely Parametrc representaton The feature frame can be consdered a smplfed verson of a mult-element 2D shape. To facltate the preservaton of local features and structural relatonshps durng subsequent manpulaton and nterpolaton, we further take the feature frame as a proxy of the shape by recordng the shape vertces n parametrc coordnates. Gven a feature lne j (v 1 j,v 2 j), a local orthogonal coordnate system can be defned as: (v 1 j,v 1 jv 2 j,v 1 jv 2 j ), where v 1 jv 2 j = v 2 j v 1 j and s a 2D operator such that: (x,y) = ( y,x). Then, for the salent feature assocated wth the feature lne j, each of ts vertces s parameterzed usng two local coordnates, wth respect to the local orthogonal coordnate system. Ths approach s smlar to the skeleton coordnate systems [LCF00, YN03]. However, our approach has two man advantages: Whle bndng a shape and skeleton bones s a nontrval work, the assocaton between a shape and feature lnes of the feature frame s explct. Furthermore, a skeleton s more approprate to the shapes that have an obvous artculated structure; however, the feature frame s general and applcable to arbtrary shapes, snce t s manly determned by the shape geometry, but not the shape s jonted structure. Wth these confguratons, we then develop a unfed framework for the manpulaton and nterpolaton of multelement 2D shapes. 4. Shape manpulaton As the user manpulates the nput mult-element 2D shape, a constraned optmzaton s solved to fnd optmal deformed shapes. The user-specfed handles serve as postonal constrants for the optmzaton problem, n whch the deformed postons of the feature frame vertces comprse the unknown varables. Addtonally, the objectve functon contans two other constrants,.e., rgdty constrants for preservng local shape features, and couplng constrants for mantanng the structural relatonshps between parts of the shape. Wth the new confguraton of the feature frame, the deformed shape s determned straghtforwardly usng the recorded parametrc coordnates Deformaton constrants Rgdty constrants. It s a well-establshed goal for any nteractve shape manpulaton tool to preserve local features when a broad change n shape s made. Achevng that usng the feature frame s very straghtforward. Snce the shape vertces are recorded n parametrc coordnates, the feature lnes wll drve the local shape deformatons. Thus, by confnng the feature lnes to undergo rgd transformatons, the local features of the shape wll be well preserved (Fg. 3). We mplement such rgdty constrants by addng an energy term E rg, whch sums on each feature lne the squared dstances between the actual transformed poston of each of ts ponts and an optmal rgd transformaton appled to the pont: 1 E rg = v j (t) v c j ( v j (t) v c j) R j 2 dt (1) j 0 where R j s a normalzed orthogonal matrx correspondng to the optmal rotaton of the ponts of the feature lne j, whle v c j and v c j are the ntal and deformed rotaton center, correspondng to the mddle pont of the feature lne j. Couplng constrants. As descrbed n the secton ntroducton, t s mportant to preserve the structural relatonshps between shape parts, so as to mantan the feature semantcs of the underlyng subject. To satsfy ths requrement, we couple neghborng feature lnes together and let

6 2254 W. Yang & J. Feng & X. Wang / Manpulaton and Interpolaton for Mult-element 2D Shapes them deform as a rgd body, whch wll preserve the relatve postons and orentatons between them, resultng n the mantenance of the structural relatonshps of our nterest (Fg. 3). In the couplng constrants, we let the ponts of the neghborng feature lnes transform as a rgd unt: E cou = 1 v j (t) v c ( v j (t) v c ) ˆR 2 dt (2) j N() 0 where N() s the ndces set of the neghborng feature and sewng lnes that are ncdent to the feature frame vertex ; ˆR s the optmal rotaton of the set N(), whle v c and v c are the ntal and deformed rotaton centers, correspondng to the feature frame vertex. Postonal constrants. To satsfy the user-specfed postonal constrants, a postonal energy term s defned as follows. Gven a handle k, ts source and target postons are c k and c k. We fnd the shape pont closest to the handle and assume (v,v j ) be the feature lne assocated wth the shape feature where the closest shape pont les. Wth respect to the local orthogonal coordnate system (v,v v j,v v j ), whch s derved from the feature lne (v,v j ), the source poston c k has a unque expresson: c k = v + uv v j + vv v j where (u,v) s the local coordnates of c k n the coordnate system. Then the penalty term, whch preserves the poston of the handle k relatve to ts nearest shape feature, s: (3) Pos(c k ) = v + uv v j + vv v j ck 2 (4) Fnally, the energy term for all handles s: E pos = k Pos(c k ) Optmzaton To obtan the deformed postons v, [1...q], of the feature frame vertces, we solve the followng optmzaton problem: arg mn w re rg + w ce cou + w pe pos (5) v 1,...v q We use the weghts w r = 1, w c = 1, and w p = 100 for the examples shown n the paper. It bears notng that whle the ndvdual rgdty and couplng constrants here demonstrate the dfferent deformaton effects ntutvely (Fg. 3), they can be combned as one n the mplementaton, snce the couplng constrants mplctly mpose the local rgdty. Eq. (5) s a nonlnear optmzaton problem, whch contans 2q unknown varables,.e., {v = (x,y )} q =1. Smlar to [SA07], ths nonlnear optmzaton s solved usng an alternatng mnmzaton approach. In the optmzaton, the objectve functon of Eq. (5) s treated as a functonal dependng on both {v }, and {R j } of Eq. (1), as well as { ˆR } of Eq. (2). The optmzaton s then performed n an alternatng manner: fxng {v } to compute the optmal {R j } and { ˆR }; usng {R j } and { ˆR } to fnd Fgure 3: A smple deformaton case shows the effect of the three constrant terms of the objectve functon. Top left: nput shape and ts feature ponts (crcular dots). For clear demonstraton the feature ponts are manually specfed so that the shape s segmented nto three ntutve parts,.e., three rectangles. As llustrated n the top rght, each feature lne (.e., a thck lne segment) and ts assocated shape part (.e., a rectangle) are drawn n the same color. Wthout the rgdty term, unnatural dstortons of local features can occur (bottom left). The couplng term preserves the structural relatonshps between shape parts (bottom rght). the best {v }, and repeatng untl convergence. In our experments, the optmzaton wll converge after about 12 teratons. In each teraton, each optmal R j s computed ndependently, snce R j s only related to the energy E rg of Eq. (1) and each term n the sum of Eq. (1) nvolves only one rotaton R j. Thus, we seek an R j that mnmzes the correspondng term n Eq. (1). In 2D case, the soluton to R j has an analytc form [SMW06]. Smlarly, the analytc soluton for each optmal ˆR can also be obtaned. In order to obtan {v } from the gven {R j } and { ˆR }, we should compute the gradents of E rg, E cou and E pos wth respect to each v. Whle the gradent computaton s smple for E pos, t seems complcated for E rg and E cou, snce they both nvolve the ntegral calculatons. Thanks to the carefully desgned energy terms E rg and E cou, ther partal dervatves wth respect to v are both analytcally ntegrable (see Appendx A). Fnally, we obtan a sparse lnear system for {v }. The system matrx s constant at all teratons; thus, the full factorzaton can be reused [Dav04], resultng n a very effcent solver. As shown n Table 1, the optmzaton of Eq. (5) requres only about 5 ms for the presented examples. 5. Shape nterpolaton One can make 2D anmatons by deformng the multelement 2D shapes va drect manpulaton frame by frame.

7 W. Yang & J. Feng & X. Wang / Manpulaton and Interpolaton for Mult-element 2D Shapes 2255 Fgure 4: Interpolaton of feature frames and trangle sets of the butterfly shape n Fg. 2(a). (left, rght): a source and target feature frames and the correspondng trangle sets, where each trangle s rendered wth alpha blendng usng the same color as the feature frame vertex from whch t s derved. (mddle): an ntermedate trangle set and feature frame. However, ths s a tedous process and we reduce the manual effort by usng the shape nterpolaton technque. To create a 2D anmaton, the user creates key deformatons of the nput shape va drect manpulaton at only several keyframes, and then the system automatcally generates smooth ntermedate deformatons by nterpolatng the shapes between each two consecutve keyframes. Our goal s to nterpolate shapes n a shape-preservng way, so as to obtan vsually pleasng anmaton. Let S 1 and S 2 be two deformed mult-element shapes, and F 1, F 2 the assocated feature frames. Durng nterpolaton, the key s to mantan the structural relatonshps encoded n the feature frames F 1 and F 2, so that the feature semantcs of the nput shapes are preserved. Wth ths goal n mnd, we propose the followng algorthm to nterpolate the shapes S 1 and S 2. The algorthm ncludes two steps. Step 1: Structural relatonshps transferrng. In ths step, we construct two trangle sets T 1 and T 2 from the feature frames F 1 and F 2, respectvely. The process of trangle set constructon s on the bass of feature frame vertex. Gven a feature frame, for each of ts vertces, let r be the total number of the feature and sewng lnes that are ncdent to the vertex: f r s equal to 2, one trangle s created; f r s greater than 2, r trangles are created. An example s shown n Fg. 4. It s evdent that one-to-one correspondence exsts between the trangle of sets T 1 and T 2, snce F 1 and F 2 have dentcal connectvty. In ths smple way, the structural relatonshps between parts of a shape that are encoded by the relatve postons and orentatons between the neghborng feature lnes of the correspondng feature frame are explctly transferred nto a set of trangles. Step 2: Structure preservng nterpolaton. Ths step performs an as-rgd-as-possble nterpolaton between the compatble trangle sets T 1 and T 2, as shown n Fg. 4 (The concept of as-rgd-as-possble transformatons was tself frst ntroduced by Alexa [ACOL00], thus we refer the readers to [ACOL00] for detals). Due to the nature of as-rgd-aspossble shape nterpolaton n mnmzng redundant dstortons, the transferred structural relatonshps of S 1 and S 2 n the trangle sets T 1 and T 2 are preserved durng nterpolaton. (a) (b) (c) Fgure 5: Interpolaton of two deformed mult-element 2D shapes of the cartoon boy n Fg. 1(a). (a): nput shapes; (b): our nterpolaton approach; (c): lnear blendng method. Based on the vertex assocaton between the trangle set and the feature frame, t s straghtforward to obtan the ntermedate feature frames from the ntermedate trangle sets (Fg. 4). Furthermore, by applyng the recorded parametrc coordnates of shape vertces to the ntermedate feature frames, the ntermedate shapes are fnally determned (Fg. 5(b)). 6. Results We have mplemented the proposed method of manpulaton and nterpolaton for mult-element 2D shapes on a PC wth 2.4GHz Intel Core 2 Duo CPU and 2GB RAM. Our mplementaton s sngle threaded and uses only one core. Lve manpulaton and nterpolaton examples are demonstrated n the accompanyng vdeo, and key results are hghlghted n ths secton. Fg. S. VER F. LINES F. VER PRE SOL 1(b) (13, 5) (b) (13, 6) (b) (12, 5) (b) (4, 2) Table 1: Statstcs and performance data for shape manpulaton. Tmng s measured n mllseconds. Shape manpulaton. Table 1 shows tme statstcs of manpulaton for the mult-element 2D shapes n ths paper. In the table, S. VER denotes the number of shape vertces, F. LINES the number of the feature and sewng lnes of the feature frame (brackets: number of the total sewng lnes and number of the user-specfed sewng lnes, respectvely), F. VER the number of the feature frame vertces, PRE

8 2256 W. Yang & J. Feng & X. Wang / Manpulaton and Interpolaton for Mult-element 2D Shapes (a) (b) (c) Fgure 6: Manpulaton of a cartoon candle. (a): nput shape; (b): our manpulaton approach; (c): the method of [YF09a], where the semantc feature on the left arm of the candle s dstorted. runtme of pre-factorzaton, and SOL runtme of all teratons. Moreover, the tme for determnng deformed shapes from deformed feature frames s less than 1 ms for all examples, and thus s neglected. Obvously, the shape deformaton can be performed completely nteractvely. Fg. 1 and 6 demonstrate that our manpulaton approach for mult-element 2D shapes s more approprate to complex nputs than prevous rgdty-modelng methods [IMH05, WXW 06] and the stck fgure manpulaton approach [YF09a]. Our method preserves both local shape features and the structural relatonshps between shape parts, thus obtanng vsually plausble deformaton results. In these fgures, the pleasng deformaton results are obtaned by manpulatng only several handles, whch can be challengng wth the prevous shape manpulaton technques. Fg. 8 demonstrates that our manpulaton approach retans more deformaton flexblty of the mult-element 2D shapes than the prevous rgdty-modelng methods [IMH05, WXW 06, SDC09]. Fg. 8(c) shows the deformaton result of a grl shape usng the method of Igarash et al. [IMH05]. Snce the grl shape s entrely embedded n a trangulaton, all of ts parts s tghtly coupled. Consequently, lftng the grl s rght leg wll pull her body accordngly, leadng to a unnatural pose. Furthermore, the grl s har and left arm are nseparable durng manpulaton, because the smple polygonal outlne cannot fully capture the semantc boundares of the overlapped parts. However, n our approach, dfferent parts of the nput shape are loosely connected and thus are allowed to move separately, fnally leadng to natural deformaton results, as shown n Fg. 8(b). Fg. 9 further demonstrates the deformaton flexblty of our manpulaton approach. It bears notng that the concept of feature frame s general. It s possble to construct a feature frame usng arbtrary ponts of the shape nstead of ts feature ponts. Ths s demonstrated wth a sprng-lke curve (Fg. 7), where we have replaced the feature ponts of the shape wth ts sample ponts at unform ntervals of arc-length for feature frame Fgure 7: Squashng of a sprng-lke curve. constructon. In ths case, our deformaton framework s smlar to the as-rgd-as-possble curve edtng [IMH05]. By usng dfferent sample ntervals, e.g., half, two tmes, and eght tmes the average edge length of the curve, respectvely, feature frames wth three dfferent hgh-to-low resolutons are constructed, and then nterestng soft-to-stff stffness behavors are effectvely mmcked (see the lower row of Fg. 7). Shape nterpolaton. Snce both the vertex number and the number of the feature and sewng lnes of feature frame are small, as shown n Table 1, the trangle set for feature frame wll have a small number of vertces and trangles. Fnally, our nterpolaton approach s very effcent: t s able to generate over 100 ntermedate shapes per second for the examples shown. Fg. 5 demonstrates that our shape nterpolaton approach generates natural results. As shown n Fg. 5(b), the local shapes of the nput objects and ther feature semantcs are effectvely preserved. Fg. 5 also demonstrates that our nterpolaton approach yelds ntutve rotatons of the source shape towards the target shape, thus effectvely avod the shrnkage effects, whch are common n naïve lnear blendng methods (Fg. 5(c)). More examples are shown n the accompany vdeo. 7. Concluson We have presented a method that generates natural and ntutve deformatons va drect manpulaton and smooth nc 2012 The Author(s)

9 W. Yang & J. Feng & X. Wang / Manpulaton and Interpolaton for Mult-element 2D Shapes 2257 (a) (b) (c) Fgure 8: Manpulaton of a fgure. (a): nput shape; (b): our manpulaton approach; (c): the method of Igarash et al. [IMH05]. (a) (b) (c) Fgure 9: Manpulaton of a sut. (a): nput shape; (b): our manpulaton approach; (c): the manpulaton method of Igarash et al. [IMH05], where the two front peces of the sut are ted together and the body of the sut and ts left sleeve are nseparable, leadng to unnatural deformaton result. terpolaton for mult-element 2D shapes. The mult-element 2D shape representaton can correctly descrbe the semantc boundares of arbtrary shapes even when dfferent parts of the shape are overlapped. We ntroduce a smple structure called the feature frame to represent the structural relatonshps between parts of the shape. Based on the feature frame, a unfed framework s developed for the manpulaton and nterpolaton of mult-element 2D shapes. Our method s computatonally effcent, allowng real-tme manpulaton and nterpolaton. There are stll mprovements needed. Whle t s a smple task to draw sub-shapes for most objects, as shown n our examples, there are some objects whch have so much detaled decoraton that t wll be tedous for the user to draw the large number of sub-shapes for them. One possble soluton s the usage of nteror texture to descrbe the subtle detals. Furthermore, we ntend to nvestgate the possblty of automatcally extractng mult-element 2D shapes from mages [ZCZ 09]. In the extreme cases where a handle s dragged too far from the shape, the structural relatonshps between neghborng parts mght be destroyed to some extent, so as to satsfy the user-specfed handle constrants. In these extreme cases, an unnatural jaggedness between neghborng local shape features mght occur. We tred to smooth out the jaggedness by blendng adjacent local transformatons, but the result was not very satsfactory. Hsu and Lee [HL94] propose a skeleton-drven deformaton model based on an deal materal wth non-localzed deformaton, whch can effectvely elmnate the jaggedness-lke effects, such as wrnkles or fold-backs. We would lke to experment wth ths by fndng a way to ntegrate ther deformaton model nto our deformaton framework for shape manpulaton n Secton 4. Furthermore, we hope to apply the method to multcomponent 3D models. The man problem seems to be the constructon of the feature frame -lke structure for the models. The ntellgent wres [GSMCO09] or componentwse controllers [ZFCO 11] mght be a good startng pont for ths purpose. Acknowledgements We would lke to thank the anonymous revewers for ther helpful comments. Ths research was partally funded by the NSFC (No , ), the Foundaton of Zhejang Educatonal Commttee (No. Y ), the Program for New Century Excellent Talents n Unversty (NCET ), and the 973 program of Chna (No. 2009CB320801). References [ACOL00] ALEXA M., COHEN-OR D., LEVIN D.: As-rgd-aspossble shape nterpolaton. In SIGGRAPH 00 (2000), pp [BIBA09] BAXTER III W. V., BARLA P., ANJYO K.-I.: Comc 2012 The Author(s)

10 2258 W. Yang & J. Feng & X. Wang / Manpulaton and Interpolaton for Mult-element 2D Shapes patble embeddng for 2d shape anmaton. TVCG 15, 5 (2009), [BS08] BOTSCH M., SORKINE O.: On lnear varatonal surface deformaton methods. TVCG 14, 1 (2008), [CS99] CHETVERIKOV D., SZABO Z.: A smple and effcent algorthm for detecton of hgh curvature ponts n planar curves. In Proceedngs of 23rd Workshop of the Austran Pattern Recognton Groups (1999), pp [Dav04] DAVIS T. A.: Algorthm 832: UMFPACK V4.3 an unsymmetrc-pattern multfrontal method. ACM Transactons on Mathematcal Software 30, 2 (2004), [GB08] GAIN J., BECHMANN D.: A survey of spatal deformaton from a user-centered perspectve. TOG 27, 4 (2008), [GSMCO09] GAL R., SORKINE O., MITRA N. J., COHEN-OR D.: wres: an analyze-and-edt approach to shape manpulaton. TOG 28, 3 (2009), [HBCC07] HAHMANN S., BONNEAU G.-P., CARAMIAUX B., CORNILLAC M.: Multresoluton morphng for planar curves. Computng 79, 2 (2007), [HDK07] HORNUNG A., DEKKERS E., KOBBELT L.: Character anmaton from 2d pctures and 3d moton data. TOG 26, 1 (2007), 1. [HKT10] HO E. S. L., KOMURA T., TAI C.-L.: Spatal relatonshp preservng character moton adaptaton. TOG 29, 4 (2010), 1 8. [HL94] HSU S. C., LEE I. H. H.: Drawng and anmaton usng skeletal strokes. In SIGGRAPH 94 (1994), pp [IMH05] IGARASHI T., MOSCOVICH T., HUGHES J. F.: Asrgd-as-possble shape manpulaton. TOG 24, 3 (2005), [LCF00] LEWIS J. P., CORDNER M., FONG N.: Pose space deformaton: a unfed approach to shape nterpolaton and skeleton-drven deformaton. In SIGGRAPH 00 (2000), pp [SA07] SORKINE O., ALEXA M.: As-rgd-as-possble surface modelng. In SGP 07 (2007), pp [SCOL 04] SORKINE O., COHEN-OR D., LIPMAN Y., ALEXA M., RÖSSL C., SEIDEL H.-P.: Laplacan surface edtng. In SGP 04 (2004), pp [SDC09] SÝKORA D., DINGLIANA J., COLLINS S.: As-rgd-aspossble mage regstraton for hand-drawn cartoon anmatons. NPAR 09, pp [SGWM93] SEDERBERG T. W., GAO P., WANG G., MU H.: 2-D shape blendng: an ntrnsc soluton to the vertex path problem. In SIGGRAPH 93 (1993), pp [SMW06] SCHAEFER S., MCPHAIL T., WARREN J.: Image deformaton usng movng least squares. TOG 25, 3 (2006), [SSJ 10] SÝKORA D., SEDLACEK D., JINCHAO S., DINGLIANA J., COLLINS S.: Addng depth to cartoons usng sparse depth (n)equaltes. Computer Graphcs Forum 29, 2 (2010), [SSP07] SUMNER R. W., SCHMID J., PAULY M.: Embedded deformaton for shape manpulaton. TOG 26 (2007). [WNS 10] WHITED B., NORIS G., SIMMONS M., SUMNER R. W., GROSS M., ROSSIGNAC J.: Betweent: An nteractve tool for tght nbetweenng. Computer Graphcs Forum 29 (2010), [WXW 06] WENG Y., XU W., WU Y., ZHOU K., GUO B.: 2d shape deformaton usng nonlnear least squares optmzaton. The Vsual Computer 22, 9 (2006), [XWY 09] XU W., WANG J., YIN K., ZHOU K., VAN DE PANNE M., CHEN F., GUO B.: Jont-aware manpulaton of deformable models. TOG 28 (2009), 35:1 35:9. [YF09a] YANG W., FENG J.: 2d shape manpulaton va topology-aware rgd grd. Computer Anmaton and Vrtual Worlds 20, 2 3 (2009), [YF09b] YANG W., FENG J.: 2d shape morphng va automatc feature matchng and herarchcal nterpolaton. Computers and Graphcs 33 (2009), [YN03] YAMANE K., NAKAMURA Y.: Natural moton anmaton through constranng and deconstranng at wll. TVCG 9, 3 (2003), [YZX 04] YU Y., ZHOU K., XU D., SHI X., BAO H., GUO B., SHUM H.-Y.: Mesh edtng wth posson-based gradent feld manpulaton. TOG 23, 3 (2004), [ZCZ 09] ZHANG S.-H., CHEN T., ZHANG Y.-F., HU S.-M., MARTIN R. R.: Vectorzng cartoon anmatons. TVCG 15 (2009), [ZFCO 11] ZHENG Y., FU H., COHEN-OR D., AU O. K.-C., TAI C.-L.: Component-wse controllers for structure-preservng shape manpulaton. In Computer Graphcs Forum (2011), vol. 30, pp [ZXTD10] ZHOU K., XU W., TONG Y., DESBRUN M.: Deformaton transfer to mult-component objects. Computer Graphcs Forum 29 (2010), Appendx A Here we provde the analytcal expressons of the partal dervatves of energy terms E rg of Eq. (1) and E cou of Eq. (2) wth respect to v k (the deformed poston of any feature frame vertex k). We take the computaton of the dervatve of E cou/ v k as example. Let (v 1 j,v 2 j) and (v 1 j,v 2 j) be the ntal and deformed postons of the two end vertces of each feature or sewng lne j n the set N() of Eq. (2), we have: { v j (t) = (1 t)v 1 j +tv 2 j, v c = αv 1 j + βv 2 j v j (t) = (1 t)v 1 j +tv 2 j, v c = αv 1 j + βv 2 (6) j Snce every feature or sewng lne j n the set N() s ncdent to the feature frame vertex, whch corresponds to the rotaton center n Eq. (2), t s evdent that (α = 0,β = 1) or (α = 1,β = 0). By substtutng Eq. (6) nto Eq. (2), we have: E cou 1 ( ) [ 2 v k = v j (t) v c [ v j (t) v c ] v j (t) v c ] ˆR dt j N() 0 v k = j N() j N() j N() j N() [ ( 1 [ ( 1 [ ( 1 [ ( 1 3 α+α2 )v 1 j +(1 6 α+β 2 6 α+β 2 + αβ)v 2 ] v 1 j j + v k + αβ)v 1 j +(1 3 β+β2 )v 2 ] v 2 j j v k 3 α+α2 )v 1 j ˆR +( 1 6 α+β 2 6 α+β 2 + αβ)v 2 j ˆR ] v 1 j v k + αβ)v 1 j ˆR +( 1 3 β+β2 )v 2 j ˆR ] v 2 j v k