As-Rigid-As-Possible Shape Manipulation

Transcription

1 As-Rgd-As-Possble Shape Manpulaton akeo Igarash 1, 3 omer Moscovch John F. Hughes 1 he Unversty of okyo Brown Unversty 3 PRESO, JS Abstract We present an nteractve system that lets a user move and deform a two-dmensonal shape wthout manually establshng a skeleton or freeform deformaton (FFD) doman beforehand. he shape s represented by a trangle mesh and the user moves several vertces of the mesh as constraned handles. he system then computes the postons of the remanng free vertces by mnmzng the dstorton of each trangle. Whle physcally based smulaton or teratve refnement can also be used for ths purpose, they tend to be slow. We present a two-step closed-form algorthm that acheves real-tme nteracton. he frst step fnds an approprate rotaton for each trangle and the second step adjusts ts scale. he key dea s to use quadratc error metrcs so that each mnmzaton problem becomes a system of lnear equatons. After solvng the smultaneous equatons at the begnnng of nteracton, we can quckly fnd the postons of free vertces durng nteractve manpulaton. Our approach successfully conveys a sense of rgdty of the shape, whch s dffcult n space-warp approaches. Wth a multple-pont nput devce, even begnners can easly move, rotate, and deform shapes at wll. CR Categores: I.3.6 [Computer Graphcs]: Methodology and echnques Interacton echnques; I.3.3 [Computer Graphcs]: Pcture/Image Generaton Dsplay algorthms; I.3.5 [Computer Graphcs]: Computatonal Geometry and Object Modelng Geometrc algorthms. Keywords: Shape Manpulaton, Deformaton, Image Edtng, Mesh Edtng, Anmaton, Interacton 1 Introducton Wth a D mage or drawng at hand, a user mght want to manpulate t move, rotate, stretch, and bend t. he prmary applcaton we have n mnd s an edtng tool for drawng or mage-edtng systems, but our nteractve shape manpulaton technque s also useful n varous applcatons such as real-tme lve performance [Ngo et al. 0] and enrchng graphcal user nterfaces [Bruce and Calder 1995]. One popular approach for shape manpulaton s to use a predefned skeleton. he user manpulates the skeleton confguraton and the system adjusts the overall shape relatve to the skeleton. However, defnng a skeleton structure for a shape s not a trval task [Lews et al. 0] and s not effectve for objects, such as jelles, that lack an obvous jonted structure. Another popular method s free-form deformaton (FFD) [MacCracken and Joy 1996] n whch the user explctly dvdes the space nto several domans and manpulates each doman by movng control ponts defnng t. But settng FFD domans s tedous and the user must laborously manpulate many control vertces. Copyrght 5 by the Assocaton for Computng Machnery, Inc. Permsson to make dgtal or hard copes of part or all of ths work for personal or classroom use s granted wthout fee provded that copes are not made or dstrbuted for commercal advantage and that copes bear ths notce and the full ctaton on the frst page. Copyrghts for components of ths work owned by others than ACM must be honored. Abstractng wth credt s permtted. o copy otherwse, to republsh, to post on servers, or to redstrbute to lsts, requres pror specfc permsson and/or a fee. Request permssons from Permssons Dept, ACM Inc., fax +1 (1) or e-mal permssons@acm.org. 5 ACM /05/ $5. hs paper presents an nteractve system that allows the user to manpulate a shape wthout usng a skeleton or FFD. he user chooses several ponts nsde of the shape as handles and moves each handle to a desred poston. he system then moves, rotates, and deforms the overall shape to match the gven handle postons whle mnmzng dstorton. By takng the nteror of the shape nto account, our approach can model ts rgdty, makng the result much closer to the behavor of real-world objects than n space-warp approaches as n [Barrett and Cheney ; Llamas et al. 3]. We use a two-step closed-form algorthm for fndng the shape confguraton that mnmzes dstorton. he typcal approach s to use a physcally based smulaton or nonlnear optmzatons [Sheffer and Kraevoy 4], but these technques are too slow for nteractve manpulaton. A key aspect of our approach s the desgn of a quadratc error metrc so that the mnmzaton problem s formulated as a set of smultaneous lnear equatons. he system solves the smultaneous equatons at the begnnng, and can therefore quckly fnd a soluton durng nteracton. Ideally we would lke a sngle quadratc error functon that handles all propertes of a shape, but no such functon exsts (see Appendx A). We therefore splt the problem nto a rotaton part and a scale part. hs dvdes the problem nto two least-squares mnmzaton problems that we can solve sequentally. hs method can be seen as a varant of the method proposed by Sorkne et al. [4]. Our technque can be useful n standard draggng operatons wth a mouse, but t s partcularly nterestng when usng a multplepont nput devce such as a SmartSkn touchpad [Rekmoto ] (Fgure 1). Wth such a devce, one can nteractvely move, rotate, and deform an entre shape as f manpulatng a real object usng both hands. hs s dffcult wth exstng shape deformaton tools because most allow only local modfcaton whle the overall poston and orentaton of the shape s fxed. Fgure 1: Shape manpulaton usng a SmartSkn touchpad. he user can nteractvely move, rotate, and deform the shape usng both hands as f manpulatng a real object. 1134

2 Related Work Shape manpulaton technques fall roughly nto two categores. One s to deform the space n whch the target shape s embedded; the other s to deform the shape whle takng ts structure nto account. Deformaton usng skeletons, FFD, and other space-warp approaches belong to the frst category. Wth skeletons, each pont n the shape s assocated wth a coordnate frame defned by a bone [Lews et al. 0]. In FFD, each pont s assocated wth a closed regon n a FFD grd [MacCracken and Joy 1996]. Other space warp technques deform the global space [Mllron et al. ]. Beer and Neely used space deformaton for morphng [199]. wster deforms the global space accordng to two 6-DOF controls [Llamas et al. 3], and Barrett and Cheney [] used space-warp deformaton for dgtal mage edtng. Brooksten [1989] used thn-plate splnes to fnd space deformaton that s defned by several control ponts. A drawback of these approaches s that they model the rgdty of the ambent space, rather than that of the shape tself, and thus the resultng deformaton dffers greatly from the behavor of real objects. he second category ncludes physcally based methods, the most popular of whch are mass-sprng models [Gbson and Mrtch 1997]. hese are very easy to mplement, but ther behavor s too elastc for many applcatons and s slow to converge. In addton, careful tunng of varous parameters s requred to make them really work. More physcally accurate smulaton s possble wth fnte-element methods [Celnker and Gossard 1991], but these are very complcated and expensve to solve, makng them napproprate for nteractve manpulaton of smple drawngs. he ArtDefo system [James and Pa 1999] acheved physcally accurate, nteractve shape deformaton usng a boundary-element method, but t s lmted to very small deformatons such as pokng the surface and s not applcable to large deformatons lke bendng an arm. he work presented here belongs to the second category. Our goal s to ntroduce nternal model rgdty nto shape manpulaton. However, nstead of usng physcally based models, we use smple geometrc approach smlar to a technque used n [Alexa et al. 0]. hey obtan an as-rgd-as-possble nterpolaton between shapes by computng a rgd transformaton for each trangle element geometrcally and sttchng them together. Smlarly, we acheve as-rgd-as-possble manpulaton by geometrcally mnmzng the dstorton assocated wth each trangle n a mesh. Sheffer and Kraevoy [4] ntroduce a smlar deformaton tool, but use an teratve computaton that s too expensve for nteractve manpulatons, especally when the control vertces move quckly. he algorthm we use can be seen as a varant of the Laplacan surface-edtng method proposed by Sorkne et al. [4]. hey acheved fast detal-preservng deformaton by usng rotaton- and scale-nvarant Laplacan coordnates. hey also proposed scalng the Laplacans of the deformed shape back to ther orgnal scale and re-solvng. Smlarly, we add a scale-preservng effect to the ntal deformaton process. We show that ths scale-preservaton effect makes possble more dynamc manpulaton than s seen n ther paper s examples, where the user fxes most of the shape and moves only a specfc regon of nterest. 3 Overvew We start wth an overvew of the system to establsh a context for the core algorthms descrbed n the next secton. he user frst mports a D shape represented ether by vector graphcs or a btmap mage nto the system. he only requrement s that the boundary of the shape can be represented as a smple closed polygon. For btmap mages, we currently manually remove backgrounds and apply automatc slhouette tracng usng the marchng squares algorthm. he system then generates a trangulated mesh nsde the boundary. Varous trangulaton methods are avalable [Shewchuk 1996], but better manpulaton results are acheved by usng near-equlateral trangles of smlar szes across the regon. We use a partclebased algorthm to obtan such a mesh [Markosan 1999]. Startng wth a standard constraned Delaunay trangulaton, the system teratvely refnes the mesh by adjustng vertex postons and mesh connectvty. o work at nteractve rates, t s mportant that the mesh not be too large. Our current mplementaton generates meshes wth 1-3 vertces wthn a few seconds. he resultng trangulaton s regstered as the rest shape, and the system performs a pre-computaton (whch we call regstraton ) to accelerate the computaton durng the nteracton. (Fgure a). Orgnal drawng or mage rangle Mesh Handles Result a) rangulaton and regstraton b) Complaton c) Manpulaton Fgure : Overvew of the system. he system frst trangulates the orgnal shape, and performs some pre-computaton. he user adds handles. Movng the handles results n a fast deformaton. he user manpulates the shape by ndcatng handles on the shape and then nteractvely movng the handles (Fgure b,c). he user clcks on the shape to place handles and drags the handles to move them. We currently let the user place handles only at exstng mesh vertces. Ideally, the system would allow the user to put handles at arbtrary locatons and modfy the mesh structure to nclude the handle. We plan to ncorporate such a re-meshng mechansm nto the system n the future. Our system also supports multple-pont nput devces. We currently use SmartSkn [Rekmoto ], whch can track multple fngers touchng ts surface. By projectng the drawng onto the SmartSkn we brng the user's fngers nto drect correspondence wth the constrant ponts (Fgure 1). hs lets the user grasp and manpulate the drawng as f manpulatng a realworld object. We are also testng a Wacom tablet wth two orentaton-senstve mce. In ths case, each mouse s assocated wth a couple of vertces whch are moved and rotated by the mouse (Fgure 16). he nterface has some smlarty to wster [Llamas et al. 3], but the operaton usng our system s more lke manpulaton of physcal objects whle ther system s desgned for model constructon. Shape manpulaton s frst appled to the trangle mesh; the system then maps the orgnal drawng or mage from the orgnal mesh to the deformed mesh. When manpulatng vector graphcs, 1135

3 we use the barycentrc coordnates of each vertex wthn the correspondng trangle of the mesh. When manpulatng a btmap mage, we smply use standard lnear texture mappng. he system performs addtonal pre-computatons when new handles are added or removed (Fgure b). We call ths process complaton because ths process actually prepares a functon that takes the handle confguraton as nput and returns the resultng shape as output. Durng nteracton, the system repeatedly sends the updated handle confguraton to ths functon. 4. Algorthm he nput to the algorthm s the set of all xy-coordnates of the constraned mesh vertces (Fgure 3a) and the output s the xycoordnates of the remanng free vertces that mnmze the dstorton assocated wth all trangles n the resultng mesh (Fgure 3d). he central challenge s to fnd approprate defnton for the dstorton of an ndvdual trangle. Our strategy s to desgn an error metrc that s quadratc n ts free varables so the system can solve the mnmzaton problem as a smple matrx computaton. Ideally we would lke a sngle quadratc error functon that approprately represents overall dstorton. We have examned varous possbltes, but fnally concluded that t s mpossble to desgn such a functon (see Appendx A). Our soluton s to splt the problem nto a rotaton part and a scale part so that each part s handled by an ndependent quadratc error functon. Wth ths decomposton, we can obtan the fnal result by sequentally solvng two least-squares problems. Gven the coordnates of the constraned vertces, the frst step generates an ntermedate result by mnmzng an error metrc that prevents shearng and non-unform stretchng but permts rotaton and unform scalng (Fgure 3a). he second step takes ths result and adjusts the scale of each trangle. hs second step s further decomposed nto two sequental processes. he system frst fts each orgnal trangle to the correspondng ntermedate trangles wthout changng scale (Fgure 3b), and then computes the fnal result by mnmzng an error metrc that represents the dfference between the trangle and the resultng trangles (Fgure 3c). he followng subsectons descrbe each step n detal. Step one generates an ntermedate result by mnmzng an error functon that allows rotaton and unform scalng. he nput s the xy-coordnates of the constraned vertces and the output s the xycoordnates of the remanng free vertces. Note that ths algorthm does not use the prevous result as an ntal confguraton, as do physcally-based smulaton or relaxaton methods. Instead, we provde a closed-form soluton for the problem. hs step corresponds to the D case n Laplacan edtng [Sorkne et al. 4]. Our formulaton s slghtly dfferent n that we use a trangle mesh rather than the boundary and we assgn quadratc error functonals to each ndvdual trangle rather than each vertex. We beleve that our formulaton s slghtly easer to mplement but ther formulaton can certanly be used nstead of ours n ths step. he error functon for a deformed trangle {v 0, v 1, v } s defned as follows (Fgure 4). For the correspondng trangle n the rest shape {v 0, v 1, v }, the system frst computes relatve coordnates {x 01, y 01 } of v n the local coordnate frame defned by v 0 and v 1 (R 90 denotes rotaton counterclockwse by 90 degrees): v = v0 + x01v0v1 + y01r90v0v (1) 1 Gven v 0, v 1, x 01, and y 01, the system can compute the desred locaton for v. 0 1 v desred = v0 + x01v0 v1 + y01r90v0 v1 where R90 = () 1 0 he error assocated wth v s then represented as desred { } = v v E v (3) We can defne v desred 0 and v desred 1 smlarly, and we defne the error assocated wth the trangle as y 01 90deg desred E = v v (4) { v0, v1, v} v 0 x 01 v 1 Orgnal rangle v = 1,,3 v 0 y 01 90deg error for v x 01 v 1 arget trangle v v desred Fgure 4: Error metrc used n step one. v desred s obtaned by fttng the orgnal trangle to the target trangle by translaton, rotaton, and scalng so that v 0 and v 1 match v 0 and v 1. a b c Fgure 3: Overvew of the algorthm. For the gven handle confguraton, the system frst generates an ntermedate result by mnmzng conformal (.e., scale-ndependent) dstorton (a). he system then fts trangles from the rest shape to correspondng trangles n the ntermedate result (b). he system generates a fnal result (c) by mnmzng the dfference between the trangles and the correspondng trangles. 4.1 Step one: scale-free constructon he error for the entre mesh s smply the sum of errors for all trangles n the mesh. Snce the error metrc s quadratc n v =(v 0x, v 0y,, v nx, v ny ), we can express t n matrx form: E = v Gv (5) 1{ v } he mnmzaton problem s solved by settng the partal dervatves of the functon E 1{v } wth respect to the free varables u = (u 0x, u 0y,, u mx, u my ) n v to zero. By reorderng v to put the free varables frst we can wrte v =(u q ) where q represents the constraned vertces. hs gves us E 1 E 1 = (G u We rewrte ths as G u = v Gv = q G 10 + G )u + (G 01 G G G u q 10 )q = 0 (6) (7) 1136

4 G u + Bq = 0 (8) Note that G and B are fxed and only q changes durng manpulaton. herefore, we can obtan u by smple matrx multplcaton by pre-computng G -1 B at the begnnng. G s a m m sparse, symmetrc matrx wth approxmately 1 entres per column, because of the near-equlateral structure of the mesh. Computng the soluton n step one, as shown n Fgure 3a, s very fast; t requres only one matrx multplcaton durng nteracton. Step one generates reasonable results as long as the dstances between handles are close to ther dstances n the rest shape, as shown n [Sorkne et al. 4]. For example, one can successfully translate or rotate the shape usng ths step alone. However, snce the error functon does not capture changes n scale, the shape nflates as the handles move away from each other and shrnks as they approach each other. We fx ths problem n step two. 4. Step two: scale adjustment hs step takes the ntermedate result from step one (the xycoordnates of all vertces) as nput and returns the fnal result (updated xy-coordnates of the free vertces) by adjustng the scale of the trangles n the mesh (Fgure 3b, c) Fttng the orgnal trangle to the ntermedate trangle he system frst fts each trangle n the rest shape to the correspondng trangle n the ntermedate result, allowng rotaton and translaton but not shearng or scalng (Fgure 3b). here are a couple of methods for ths sort of fttng; we use the followng method n our current mplementaton. Gven a trangle {v 0, v 1, v } n the ntermedate result and correspondng trangle n the rest shape {v 0, v 1, v }, the frst problem s to fnd a new trangle {v 0, v 1, v } that s congruent to {v 0, v 1, v } and mnmzes the followng functonal (Fgure 5): E = v v (9) f { v0, v1, v } = 1,,3 Snce t s dffcult to obtan such a result drectly, we approxmate t by frst mnmzng the error allowng unform scalng and then adjustng the scale afterwards. v the partal dervatves of E f over the four free varables to zero. he result s an easly-solved 4 4 lnear system. In matrx form, E f = Fw + C = 0 (11) w F s fxed for a gven mesh and C s defned by the result of step one. herefore, we compute F and nvert t durng regstraton. By solvng ths equaton, we obtan a newly trangle {v 0, v 1, v } that s smlar to the orgnal trangle {v 0, v 1, v }. We make t congruent smply by scalng the trangle by the factor of v 0 -v 1 / v 0 -v 1. We apply ths fttng operaton to all trangles n the mesh. Note that each vertex of the orgnal mesh appears n several trangles and hence corresponds to multple vertces n the fttng trangles (gray trangles n Fgure 3b). Reconclng these dstnct locatons s the sole remanng task. 4.. Generatng the fnal result usng the trangles he system now computes the fnal xy-coordnates of the free vertces for gven xy-coordnates of the constraned vertces by mnmzng the dfference between the resultng trangle n the mesh and the trangle (Fgure d). Note that we use only the trangles here and no longer need the ntermedate mesh. hs process s very smlar to the assembly process n [Alexa et al. 0][Sumner and Popovc 4][Yu et al. 4]. We agan begn the explanaton wth the sngle trangle {v 0, v 1, v } (Fgure 6). Gven the correspondng trangle {v 0, v 1, v }, we defne a quadratc error functon by = v { 0 1, } v v v v j (, j) {(0,1),(1,),(,0)} j E v v (1) Note that we assocate an error wth each edge, not each vertex. hat s, we use the rotaton of the trangle and gnore ts poston. he translaton s solved for as a sde effect only. he error s clearly mnmzed when the trangles {v 0, v 1, v } and {v 0, v 1, v } are dentcal. But snce the vertex v 0, for nstance, may le n several trangles, the optmal poston for v 0 wll be some average of the postons desred by each trangle n whch t appears. v 0 v v 0 v v 0 v 1 Orgnal rangle v v 0 v 0 v v 1 v 1 Intermedate trangle and trangle Fgure 5: Fttng the orgnal trangle to the ntermedate trangle by translaton and rotaton. Usng the coordnates x 01 and y 01 defned n Secton 4.1, we can express v usng v 0 and v 1 : v = v0 + x01v0 v1 + y01r90v0 v (10) 1 so the fttng functonal becomes a functon of just the coordnates of v 0 and v 1, a quadratc n the four free varables of w=(v 0x, v 0y, v 1x, v 1y ). We can mnmze E f by settng trangle v 1 v 1 target trangle Fgure 6: Error metrc used n step two. hs metrc measures the dfference between the edge vectors of the trangle and those of the target trangle. he error for the entre mesh can be represented n a matrx form: E v = v Hv + fv + c (13) { } Note that H s defned by the connectvty of the orgnal mesh and s ndependent of the trangles, whle f and c are determned by the trangles and thus change durng nteracton. We mnmze E by settng the partal dervatves of E over free vertces u to zero. By reorderng v, we can wrte v = (u q ). hs gves us 1137

5 H H u 01 u u E = v Hv + fv + c = q H 10 H 11 q q E = (H u We rewrte ths as + H )u + (H 01 + H 10 ( f f ) c )q + f 0 = 0 (14) (15) H u + Dq + f0 = 0 (16) H and D are fxed but q and f 0 change durng manpulaton. We therefore pre-compute LU factorzaton of H at the begnnng and solve the equaton usng t durng nteracton. Actually, the x and y components are mutually ndependent n H, so we can perform the above computaton for each component separately. For each component, H s an m m sparse, symmetrc matrx wth approxmately 6 entres for each column. 4.3 Algorthm summary Our algorthm can be summarzed as follows. 1 Regstraton (when a new rest shape s defned) 1-1 Construct matrces G and H usng the vertex coordnates n the rest shape. 1- Construct F and nvert t for each trangle. Complaton (when handles are added or removed) -1 Construct G and B from G and compute G -1 B. - Construct H and D from H and construct LU factorzaton of H. 3 Durng manpulaton (when handles are moved) 3-1 Obtan ntermedate coordnates for the free vertces as -G -1 Bq where q represents the coordnates of handles. 3- Construct C for each trangle usng the ntermedate vertex coordnates. Multplyng F -1 and C and adjust ts scale to obtan each trangle. 3-3 Construct f 0 usng the trangles and obtan the fnal result by solvng H u+dq+f 0 =0 usng pre-computed LU factorzaton. 5. Extensons overlappng regons s stll an open problem. We currently use a statcally predefned order as a startng pont. We have also mplemented a very smple mechansm to acheve frame-to-frame coherence, but t s stll n prelmnary form. rest shape wthout depth adjustment wth depth adjustment Fgure 7: Colluson detecton and depth adjustment. Wthout approprate depth assgnment, one can see nterpenetraton (center). We detect overlappng regons and adjust depth on the fly. Fgure 8: Lmtaton of statc depth assgnment. 5. Weghts for controllng rgdty Because our algorthm mnmzes a smple error functonal, addng dfferent weghts to dfferent parts of the mesh s straghtforward. We do ths to control the local rgdty of the shape. We provde a pantng nterface n whch the user can make certan parts stffer than others. We currently use a weght of 1 for the panted trangles and 1 for the others. hs s useful n preventng mportant features such as a head from beng dstorted (Fgure 9). It s possble to enforce perfect rgdty by reducng free varables n the mnmzaton, thus obtanng smlar results, but we found that weghtng s more flexble, and produces more pleasng results under extreme dstortons.. hs secton dscusses varous adjustments necessary to make the system work n practce, as well as other enhancements. 5.1 Collson detecton and depth adjustment We must be careful when dfferent parts of the shape overlap. If we assgn depths napproprately, the overlappng parts can nterpenetrate (Fgure 7 mddle). One problem s that one cannot assgn statc consstent depth values that work for all possble deformatons. Fgure 8 shows a smple case. Suppose we had contnuous depth values across the shape. Gven three ponts a, b, and c, we can assume that a s depth < b s depth < c s depth. If we brng vertex b between a and c by deformaton, there must exst an edge where one vertex s deeper than b and the other s shallower. hs produces the undesrable artfact n Fgure 8. Our approach s to dynamcally adjust depth durng nteracton (Fgure 7 rght). We contnuously montor the mesh for selfntersecton and assgn approprate depth values to the overlappng parts. hs process s smlar to the hdden-lne removal algorthm n [Hornung 1984]. We frst locate boundary ntersectons and then search for overlappng regons startng from those boundary ntersectons. Determnng the depth order of the rest shape wthout weghts wth weghts Fgure 9: Adjustng rgdty wth weghts. By addng extra weghts to mportant regons, one can prevent undesrable dstorton. 5.3 Anmatons Our shape manpulaton technque s useful n makng D anmatons, for example by settng the character shape at each key frame. hs s especally helpful when one wants to anmate drawngs wth detaled surface decoraton, snce drawng them manually s tedous. It also makes t practcal to make anmatons by deformng photographed objects. Another approach s to use t for performance-drven anmaton. By manpulatng multple control vertces smultaneously, one can make nterestng anmatons by recordng lve performances. We are currently testng a technque smlar to Ngo et al. s system 1138

6 [0] and facal anmaton example n [Lews et al. 0]. he user frst sets a pose by manpulatng control vertces and assocates the control vertex confguraton wth a specfc pont on the canvas. After settng several such key poses, the user can drect multple control vertces smultaneously by smply movng a control cursor. he system computes an approprate control vertex confguraton by nterpolatng nearby key poses usng a radal bass functon (Fgure 10). Flash, but t allows only local changes and does not let the user change the nfluence regon. Fgure 1: Curve edtng wth a peelng nterface. As the user pulls the curve further away, the nfluence regon grows (left to rght). he user can also explctly specfy the regon beforehand (bottom left). 6. Results Fgure 10: Desgnng performance-drven anmaton usng spatal keyframng. he users specfy a set of key poses (yellow mark) by manpulatng handles. hey can then manpulate the entre body by draggng a control cursor (red mark). he system blends nearby poses usng a radal bass functon. he resultng moton s very smooth and lvely because the user s own hand movement appears n the resultng anmaton. In addton, t s much easer and faster than tradtonal temporal keyframng usng exstng shape deformaton technques. 5.4 As-rgd-as-possble curve edtng We have appled our technque to varous drawngs and mages. Fgure 13 shows the manpulaton of arms and legs by controllng the end ponts. hs s smlar to the pn-and-drag nterface for artculated characters [Yamane and Nakamura 3], but our system works wth no explct skeleton structures. Fgure 14 shows manpulaton by controllng the nternal ponts. Note that the arms and legs are dsplaced approprately due to the rgdty of the body. By contrast, a mass-sprng model can take some tme to propagate the effect to the entre body. Fgure 15 shows manpulaton of a shape that lacks artculated structure; note that the shape s stretched and squashed approprately. Fgure 1 and 16 show applcatons of our technque to mages. he natural deformaton effects gve the feel of manpulatng real 3D objects. We have also appled our two-step algorthm to curve edtng. For curve edtng, we take a polylne nstead of a trangle mesh as nput. We frst apply D Laplacan curve edtng [Sorkne et al. 4] and then apply our scale adjustment procedure to the result. Fgure 11 shows an example operaton. Wthout scale adjustment, the curve grows and shrnks freely. Wth scale adjustment, the curve behaves as f t s rgd. Fgure 13: Manpulaton of a shape by controllng the end ponts. wthout scale adjustment wth scale adjustment Fgure 11: Curve edtng wth and wthout scale-adjustment procedure. Wthout scale adjustment, the stretched regon grows and the squashed regon shrnks. Fgure 14: Manpulaton of a shape by controllng the nternal ponts. he user moves the handle at the center horzontally and the entre body s deformed approprately. o let the user adjust the nfluence regon dynamcally durng nteracton, we ntroduce a peelng nterface n whch the nfluence regon grows as the user drags the curve father away (Fgure 1). hs frees the user from specfyng an nfluence regon beforehand and makes the nteracton very ntutve. As an opton, we also allow the user to explctly specfy the nfluence regon by puttng vrtual pushpns along the curve before draggng. A smlar grab-and-pull curve edtng tool s used n Macromeda Fgure 15: Stretch and squash of a non-artculated shape. 1139

7 apply for curve edtng, where the densty ncreases only lnearly. he error n a dense regon s estmated as smaller, thus makng the regon softer. he smplest soluton s to approxmate the curve by an evenly spaced polylne and apply our algorthm to the resampled curve. rest shape squashed Fgure 16: Manpulaton of an mage usng two rotaton senstve mce. Fgure 1 shows a snapshot of operaton usng a SmartSkn multpont touchpad [Rekmoto ]. est users found t easy to bend and stretch the shapes, and enjoyed expermentng wth the shapes and the movements they could produce. Snce the deformaton s updated n real tme and s easy to control, several fngers can be used to steer dfferent parts of the shape and perform smple anmatons. wo users can work together to create more complex moton. able 1 summarzes the performance of the current mplementaton. hs data s for a system runnng Java 1.4 on a Wndows XP notebook PC (Pentum III 1GHz processor and 756 MB of memory). We use a natve sparse LU solver [Davs 3] for matrx computatons. We obtaned the data by runnng the correspondng routnes 1 tmes, so these are very rough estmates severely affected by garbage collecton and CPU cache. he result ndcates that step two s the bottleneck durng manpulaton. In our experence, one can obtan qute nce results wth a very coarse mesh wth fewer than 1 vertces (Fgure 17 left). All examples n ths paper are obtaned wth smlar mesh szes. In ths range, the system shows completely real-tme performance and the user experences no delay. One can obtan smoother results by usng a fner mesh but the nteracton eventually becomes choppy. he delay becomes obvous at a vertex count of around 3 on our notebook PC (Fgure 17 rght). able 1: Sample runnng tmes (mllseconds) for the meshes n Fgure 17 # of Regstraton Complaton Update vertces Step1 Step Step1 Step Step1 Step Fgure 17: Example trangulatons. he number of vertces s 85, 156, 98 respectvely and three handles are attached to each. We have expermentally examned the effect of uneven trangulaton on our algorthm and found t to be farly robust aganst rregularly spaced mesh. Fgure 18 shows an example. he dense and sparse regons are evenly squashed and stretched, and smlar behavor s observed for bendng. hs can be explaned as follows. If trangle sze s reduced by a factor of 1/n, the dstorton assocated wth each trangle decreases by a factor of 1/n. At the same tme, the densty of trangles becomes n tmes hgher. As a result, trangle sze does not affect the total cost. hs does not stretched Fgure 18: he effect of uneven trangulaton. Our algorthm s not strongly affected by the mesh densty. 7. Lmtatons and Future Work he two-step algorthm ntroduced here s merely a practcal approxmaton to acheve nteractve performance. It works surprsngly well n most cases as shown n our examples, but n some cases ts lmtatons are revealed. Fgure 19 shows an example. When the control handle s moved one mght expect the result shown on the rght, but our algorthm returns the result shown n the mddle due to ts nherent lnear nature. he free vertces only move parallel to the lne connectng the constraned vertces. o handle these cases, a more accurate dstorton metrc smlar to the one n [Sheffer and Kraevoy 4] s probably necessary. rest shape current results physcally plausble results Fgure 19: A lmtaton of our algorthm. If the user stretches a shape (left), the current algorthm returns the result n the mddle; the results on the rght would be more desrable. As s dscussed n [Alexa et al. 0], a lnear mappng from an orgnal trangle to a deformed trangle gnorng the translaton factor can be represented by a affne transformaton matrx A. Usng sngular value decomposton (SVD), the matrx A can be represented as a combnaton of a rotaton part R γ, a shearng part s h, and a scalng part s x, s y [Shoemake and Duff 199]: sx sh A = R = = α DRβ ( Rα Rβ )( Rβ DRβ ) R (17) γ sh s y Gven ths formulaton, one can obtan as-rgd-as-possble mappng by mnmzng s h, s x -1, and s y -1. We would lke to experment wth ths by fndng a way to mnmze these errors drectly. Volume preservaton s another feature that the current formulaton cannot acheve [Angelds et al. 4]. Wth volume preservaton, an object s squashed vertcally when t s stretched horzontally. In the decomposton above, volume preservaton s smulated by mnmzng s x s y -1. We tred to mplement ths effect by adjustng the target trangle n the step, but the result was not very satsfactory. Our experence suggests that t s necessary to mplement volume transfer between trangle elements to obtan a 1140

8 globally convncng result. We plan to explore ths path n the future. We want very much to extend the technque to 3D shapes. he ablty to freely move, rotate, and deform a 3D object s very attractve n varous applcatons such as object manpulaton n vrtual envronments. Unfortunately, the extenson s not straghtforward. We have expermented wth varous formulatons but we have yet to fnd quadratc error functons equvalent to those we use n D, and t seems that we must take a very dfferent approach. We plan to contnue our exploraton. Acknowledgements We would lke to thank Jun Rekmoto and Kentaro Fukuch for provdng SmartSkn. We also thank Danel Keefe and UrumaDelv for ther lovely drawngs. Fnally we thank the user nterface research group at the Unversty of okyo and the Brown Unversty graphcs group for helpful dscussons. References ALEXA, M., COHEN-OR, D., and LEVIN, D. 0. As-Rgd-As-Possble Shape Interpolaton. In Proceedngs of ACM SIGGRAPH 0, ACM Press / ACM SIGGRAPH, Computer Graphcs Proceedngs, Annual Conference Seres, ACM, ANGELIDIS, A., CANI, M., WYVILL, G., and KING, S. 4. Swrlng- Sweepers: Constant-Volume Modelng. Pacfc Graphcs 4, BARRE, W. A., and CHENEY, A. S.. Object-based Image Edtng. ACM ransactons on Graphcs, 1, 3, Beer,. and Neely, S. 199 Feature-based mage metamorphoss, In Computer Graphcs (Proceedngs of SIGGRAPH 9), 6,, Booksten, F. L Prncpal Warps: hn-plate Splnes and the Decomposton of Deformatons, IEEE ransactons on Pattern Analyss and Machne Intellgence, 11, 6, BRUCE, H.., and CALDER, P Anmatng Drect Manpulaton Interfaces. In Proceedngs of UIS 95, 3-1. CELNIKER, G., and GOSSARD, D Deformable Curve and Surface Fnte Elements for Free-form Shape Desgn. In Computer Graphcs (Proceedngs of ACM SIGGRAPH 91), 5, 4, ACM, DAVIS,. A. 3. Umfpack Verson 4.1 User Gude. echncal report R-03-8, Unversty of Florda. GIBSON, S. F., and MIRICH, B A Survey of Deformable Models n Computer Graphcs. echncal report R-97-19, Mtsubsh Electrc Research Laboratores. HORNUNG, C A Method for Solvng the Vsblty Problem. IEEE Computer Graphcs and Applcatons, 4, 7, JAMES, D. L., and PAI, D. K ArtDefo Accurate Real me Deformable Objects, In Proceedngs of SIGGRAPH 1999, LEWIS, J. P., CORDNER, M., and FONG, N. 0. Pose Space Deformatons: A Unfed Approach to Shape Interpolaton and Skeleton-drven Deformaton. In SIGGRAPH 0 Conference Proceedngs, LLAMAS, I., KIM, B., GARGUS, J., ROSSIGNAC, J., and SHAW, C. D. 3. wster: A Space-Warp Operator for the wo-handed Edtng of 3D Shapes. In Proceedngs of SIGGRAPH 3, ACM Press/ACM SIGGRAPH, Ed., Computer Graphcs Proceedngs, Annual Conference Seres, ACM, MACCRACKEN, R., and JOY, K.I Free-form Deformatons wth Lattces of Arbtrary opology. In Proceedngs of ACM SIGGRAPH 1996, ACM Press / ACM SIGGRAPH, Ed., Computer Graphcs Proceedngs, Annual Conference Seres, ACM, MARKOSIAN, L., COHEN, J.M., CRULLI,., and HUGHES, J.F Skn: a Constructve Approach to Modelng Free-form Shapes. In Proceedngs of ACM SIGGRAPH 1999, ACM Press/ACM SIGGRAPH, Los Angeles, Ed., Computer Graphcs Proceedngs, Annual Conference Seres, ACM, MILLIRON,., JENSEN, R., BARZEL, R. and FINKELSEIN, A.. A Framework for Geometrc Warps and Deformatons. ACM ransactons on Graphcs, 1, 1, NGO,., CURELL, D., DANA, J., DONALD, B., LOEB, L. and ZHU, S. 0. Accessble Anmaton and Customzable Graphcs va Smplcal Confguraton Modelng. In Proceedngs of SIGGRAPH 0, ACM Press/ACM SIGGRAPH, Ed., Computer Graphcs Proceedngs, Annual Conference Seres, ACM, REKIMOO, J.. SmartSkn: An Infrastructure for Freehand Manpulatons on Interactve Surfaces. In Proceedngs of CHI 0, SHEWCHUK, J.R rangle: Engneerng a D Qualty Mesh Generator and Delaunay rangulator. In Proceedngs of Frst Workshop on Appled Computatonal Geometry, SHEFFER, A., and KRAEVOY, V. 4. Pyramd Coordnates for Morphng and Deformaton. In Proceedngs of 3D Data Processng, Vsualzaton, and ransmsson, nd Internatonal Symposum on (3DPV'04), Shoemake K. and Duff, Matrx Anmaton and Polar Decomposton. Proceedngs of Graphcs Interface 9, SORKINE, O., COHEN-OR, D., LIPMAN, Y., ALEXA, M., ROSSL, C., and SEIDEL, H. 4. Laplacan Surface Edtng. In Proceedngs of the Eurographcs/ACM SIGGRAPH Symposum on Geometry Processng, SUMNER, R. W., and POPOVIC, J. 4. Deformaton ransfer for rangle Meshes, ACM ransacton on Graphcs, 3, 3, YAMANE, K., and NAKAMURA, Y. 3. Natural Moton Anmaton hrough Constranng and Deconstranng at Wll. IEEE ransacton on Vsualzaton and Computer Graphcs, 9, 3, YU, Y., ZHOU, K., XU, D., SHI, X., BAO, H., GUO, B. and SHUM, H. Y. 4. Mesh Edtng wth Posson-Based Gradent Feld Manpulaton. ACM ransactons on Graphcs, 3, 3, A Proof of nonexstence of quadratc error metrc We clamed that there s no quadratc functon of the locatons of the deformed mesh vertces that measures the dstorton from the orgnal mesh, n the sense that t s mnmzed exactly when the deformed mesh s orented congruent to the orgnal. he proof s by contradcton: Suppose there s such a measure, and apply t to a one-trangle mesh wth an orgnal trangle ((0,0), (1,0), (0,1)) and a deformed trangle ((0,0), (x,y), (u,v)). hs gves a functon (x,y,u,v) that s quadratc n the varables x,y,u,v. By subtractng the constant term, we can get a new quadratc wth no constant term, but wth exactly the same mnma. Because rotaton through 180 degrees s an orentaton preservng congruence, we know that (x,y,u,v)= (-x,-y,-u,-v) for all (x,y,u,v). hs means that can be wrtten wth no lnear terms. We already removed the constant term, so s pure quadratc (.e., all terms have total exponent two). he functon must be postve semdefnte (.e., (x,y,u,v) >= 0 for all (x,y,u,v)), because f (x 0,y 0,u 0,v 0 ) = K < 0, then (tx 0,ty 0,tu 0,tv 0 ) = t K < 0; as we ncrease t, the values of become arbtrarly negatve; hence has no mnmum value, whch contradcts the hypothess. However, for such a functon, the value at (0,0,0,0) s zero, and ths s certanly the global mnmum. But (0,0,0,0) represents a degenerate trangle that s not congruent to our chosen one. hs s a contradcton. 1141