And Image Processing

Transcription

1 Constraned Parameterzaton Wth Applcatons To Graphcs And Image Processng Thess by HONGCHUAN YU n partal fulfllment of the requrements of Bournemouth Unversty for the degree of Doctor of Phlosophy Natonal Centre for Computer Anmaton Bournemouth Unversty, Poole, UK March 212

2 COPYRIGHT STATEMENT Ths copy of the thess has been suppled on condton that anyone who consults t s understood to recognze that ts copyrght rests wth ts author and due acknowledgement must always be made of the use of any materal contaned n, or derved from, ths thess. 212 Hongchuan Yu All Rghts Reserved

3 ACKNOWLEDGEMENTS There are many people whose gudance and support made my research possble. Frst, I thank my supervsor, Prof Jan Jun Zhang, for hs comprehensveness and generosty. Wth hs knd gudance, I have worked n a relaxed and motvatng academc envronment whch has shaped me as a researcher and as a scentst. For these experences I am forever grateful. I am also deeply ndebted to my second supervsor, Dr Hammad Nat-Charf, whose encouragement and enthusasm not only elevated ths work but have also profoundly affected me personally and phlosophcally. I have been extremely fortunate to work wth many exceptonally talented and gfted researchers over the years. Dr Lhua You, Dr. Xaosong Yang, Dr Jan Chang, and Dr Rchard Southern have been a pleasure to work wth and have enrched my experences both n research and personally. Fnally, and most mportantly, I thank my famly, my wfe, my lovely daughter, my parents, my brother and sster. Your love and encouragement kept me sane and grounded n ths wonderful chapter of my lfe.

4 ABSTRACT Surface parameterzaton s to establsh a transformaton that maps the ponts on a surface to a specfed parametrc doman. It has been wdely appled to computer graphcs and mage processng felds. The challengng ssue s that the usual postonal constrants always result n trangle flppng n parameterzatons (also called foldovers). Addtonally, dstorton s nevtable n parameterzatons. Thus the rgd constrant s always taken nto account. In general, the constrants are applcaton-dependent. Ths thess thus focuses on the varous constrants depended on applcatons and nvestgates the foldover-free constraned parameterzaton approaches ndvdually. Such constrants usually nclude, smple postonal constrants, tradeoff of postonal constrants and rgd constrant, and rgd constrant. From the perspectve of applcatons, we am at the foldover-free parameterzaton methods wth postonal constrants, the as-rgd-as-possble parameterzaton wth postonal constrants, and the well-shaped well-spaced preprocessng procedure for low-dstorton parameterzatons n ths thess. The frst contrbuton of ths thess s the development of a RBF-based reparameterzaton algorthm for the applcaton of the foldover-free constraned texture mappng. The basc dea s to splt the usual parameterzaton procedure nto two steps, 2D parameterzaton wth the constrants of convex boundares and 2D reparameterzaton wth the nteror postonal constrants. Moreover, we further extend the 2D re-parameterzaton approach wth the nteror postonal constrants to hgh dmensonal datasets, such as, volume data and polyhedrons. v

5 The second contrbuton s the development of a vector feld based deformaton algorthm for 2D mesh deformaton and mage warpng. Many presented deformaton approaches are used to employ the bass functons (ncludng our proposed RBF-based re-parameterzaton algorthm here). The man problem s that such algorthms have nfnte support, that s, any local deformaton always leads to small changes over the whole doman. Our presented vector feld based algorthm can effectvely carry on the local deformaton whle reducng dstorton as much as possble. The thrd contrbuton s the development of a pre-processng for surface parameterzaton. Except the developable surfaces, the current parameterzaton approaches nevtably ncur large dstorton. To reduce dstorton, we proposed a preprocessng procedure n ths thess, ncludng mesh partton and mesh smoothng. As a result, the resultng meshes are parttoned nto a set of small patches wth rectangle-lke boundares. Moreover, they are well-shaped and well-spaced. Ths pre-processng procedure can evdently mprove the qualty of meshes for lowdstorton parameterzatons. v

6 CONTENTS COPYRIGHT STATEMENT..... ACKNOWLEDGEMENT..... ABSTRACT... v LIST OF FIGURES. x LIST OF TABLES....x CHAPTER 1 INTRODUCTION MOTIVATION CHALLENGES IN SURFACE PARAMETERIZATION DISSERATION CONTRIBUTIONS SIMPLE POSITIONAL CONSTRAINTS TRADEOFF OF POSITIONAL CONSTRAINTS AND ROGID CONSTRAINT WELL-SHAPED AND WELL-SPACED MESH THESIS OVERVIEW.. 8 CHAPTER 2 LITERATURE REVIEW TEXTURE MAP TEXTURE SYNTHESIS IMAGE WARPING/DEFORMATION v

7 2.4 MESH EDITING MESH COMPLETION AND COMPRESSION...22 CHAPTER 3 RBF-BASED RE-PARAMETERIZATION METHOD FOR CONSTRAINED TEXTURE MAPPING PROBLEM FORMULATION BASIC IDEA AND MOTIVATION ITERATIVE RBF-BASED REPARAMETERIZATION PROCEDURE ITERATIVE MECHANISM FOLDOVER-FREE CONDITION ITERATIVE STEP-LENGTH ESTIMATION TRIANGLE SUBDIVISION EXPERIMENTS AND DISCUSSIONS FOLDOVER COMPARISON OF EXPERIMENTAL RESULTS COMPLEXITY ANALYSIS LIMITATIONS CHAPTER SUMMARIZATION..45 CHAPTER 4 EXTENSION OF RBF-BASED REPARAMETERIZATION METHOD TO HIGH DIMENSIONAL DATASETS FOLDOVERS IN HIGH DIMENSIONAL DATASETS..46 v

8 4.2 ALGORITHM OVERVIEW FOLDOVER-FREE ITERATIVE MECHANISM FOLDOVER-FREE CONDITION ITERATIVE FRAMEWORK EXPERIMENTS AND DISCUSSIONS EXPERIMENTS DISCUSSIONS CHAPTER SUMMARIZATION..63 CHAPTER 5 TOPOLOGY PRESERVED SHAPE DEFORMATION PROBLEM FORMULATION PROPOSED METHOD OUTLINE OF OUR PROPOSED METHOD POSITIVE JACOBIAN CONSTRAINTS CONSISTENCY OF DISPLACEMENT FIELDS APPLICATION: IMAGE/VIDEO MAGNIFIER EXPERIMENTS AND ANALYSIS CHAPTER SUMMARIZATION...83 CHAPTER 6 WELL-SHAPE AND WELL-SPACED MESH PROBLEM FORMULATION METHOD OVERVIEW.88 v

9 6.3 PARTITION BY NORMALIZED CUTS SMOOTHING AND RESAMPLING DIFFERENTIAL SURFACE REPRESENTATION SMOOTHING STEP RESAMPLING STEP IMPLEMENTATION AND ANALYSIS CHAPTER SUMMARIZATION.114 CHAPTER 7 CONCLUSIONS SUMMARY FUTURE WORK.12 BIBLIOGRAPHY..122 x

10 LIST OF FIGURES Fgure 2-1. Illustraton of swappng edge...14 Fgure 2-2. Illustraton of texture synthess..15 Fgure 3-1. Illustraton of the foldover results usng four recent parameterzaton methods, and consderng nternal constrants...38 Fgure 3-2. Illustraton of the condton of Eq Fgure 3-3. Illustraton of convergence n an extreme case..43 Fgure 3-4. Illustraton of the teratve results of the proposed method...38 Fgure 3-5. Smoothness comparson Fgure 3-6. Illustraton of texture mappng wth two nteror boundares...41 Fgure 4-1. Illustraton of the tetrahedral foldover on a 3D polyhedral tube..48 Fgure 4-2. Illustraton of the condton of Eq.4.4 n 2D scenaros Fgure 4-3. Illustraton of the condton of Eq.4.4 n 3D scenaros..56 Fgure 4-4. Illustraton of convergence of an extreme case..57 Fgure 4-5. Illustraton of the foldovers on 3D volume data. 59 Fgure 4-6. Illustraton of muscle deformaton..6 Fgure 4-7. Illustraton of the teratve results of our method..6 Fgure 4-8. Illustraton of texture mappng..61 Fgure 4-9. Dstorton from surface reparameterzaton 63 x

11 Fgure 5-1. Deformaton on quad and trangle meshes.65 Fgure 5-2. Illustraton of foldovers by usng recent approaches Fgure 5-3. Illustraton of foldovers by usng the local/global schemes...67 Fgure 5-4. The teratve results of examples n Fg.5-2 and Fg.5-3 by our approach.8 Fgure 5-5. Illustraton of texture mappng...8 Fgure 5-6. Comparson of smoothness.81 Fgure 5-7. Illustraton of vdeo magnfer 83 Fgure 6-1. Schematc dagram of the effect of skn slde over an underlyng structure..86 Fgure 6-2. The partton result of a hand skn surface.95 Fgure 6-3. Comparson of mesh smoothng..12 Fgure 6-4. Illustraton of reorentng δ-coordnates..13 Fgure 6-5. Character transferrng..19 Fgure 6-6. Experments of skn sldng. 11 Fgure 6-7. Illustraton of bendng seahorse tal.112 Fgure 6-8. Illustraton of transferrng the characters to a twsted tube surface.112 x

12 LIST OF TABLES Table 3-1. Dstorton metrcs of texture mappng examples n Fg Table 3-2. Statstcs of texture mappng examples n Fg Table 6-1. Runnng tme..114 x

13 CHAPTER 1 INTRODUCTION 1.1 MOTIVATION Ths thess addresses a classc topc of dfferental geometry, surface parameterzaton, wth applcatons to the texture mappng, mesh sknnng and mage warpng. Surface parameterzaton can be vewed as a bjectve mappng between a surface and a sutable parameter doman. In general, the parameter doman tself s a surface and parameterzaton means computng a both one-to-one and onto mappng n between the parameter doman and the embedded surface. Typcally, a sphere s projected onto a plane doman usng a system of grdlne,.e. longtude and lattude. In general, a surface s represented or approxmated by a trangular mesh and hence the problem of computng such a mappng s referred to as mesh parameterzaton. The mappng between the meshes and the parameter domans are pecewse lnear. 1

14 Mesh parameterzatons between surface meshes and a varety of parameter domans have many applcatons n computer graphcs, geometry processng and mage processng, such as mage regstraton, scattered data fttng and geometry modelng. In recent works, many methods have been presented for satsfyng dverse parameter domans and mantanng dfferent parameterzaton propertes. However, the challengng ssues from dfferental geometry of surface theory and numercal computaton are becomng desperate for beng solved, for example, hard postonal constrants, foldover or trangle flps, sometrc (namely zero dstorton), and conformal maps. The man motvaton of ths thess s to look for the mathematc essence underlyng the dstorton challenge, e.g. trangle foldovers. Ths s because the foldover s not only reported to appear n many applcatons of computer graphcs, e.g. texture mappng [Kraevoy et al. 23, Ecksten et al. 21, Levy 21, and Sander et al. 21], remeshng [Dong et al. 26, Gu et al. 22, and Guskov et al. 2, and Lee et al. 2], mesh edtng [Alexa et al. 2, James et al. 25, Lews et al. 2, and Sorkne et al. 24], but also s reported as dfferental ssue n mage regstraton applcatons [Dong et al. 29, Wang et al. 28, Wang et al. 21, and Schaefer et al. 26] as well. Partcularly, due to the quckly developng 3D scannng technology and the resultng demand for effcent scattered data fttng, parameterzaton or further compresson methods of ncreasngly complex trangulatons have been the actve research ssues n recent years. Usually, such applcatons request some specfed postonal constrants or area and shapepreservng propertes, whch nevtably leads to the trangle foldover n parameter domans. Ths thess focuses on the specfed constrant condtons of 2

15 parameterzatons, snce the constrants are always applcaton-dependent and appear n applcatons of both computer graphcs and mage processng. 1.2 CHALLENGES IN SURFACE PARAMETERIZATION Surface parameterzatons almost always cause dstorton n ether angles or areas. Seekng an allowable mappng (.e. to acheve some dstorton tolerance) s an actve topc of dfferental geometry. If the mappng s length-preservng, we call such a mappng as an sometry. Ths s an deal mappng, snce t preserves just about everythng we request, such as areas, angles and lengths. If we only ask for anglepreservaton, the mappng s called as conformal mappng. If the mappng s only area-preservng, we call t as equareal mappng. To fnd a well-behaved mappng, the constrants of area-preservaton and angle-preservaton are usually taken nto account together. Furthermore, snce all the parameterzaton methods are used to approxmate the underlyng smooth surface n a dscrete settng (e.g. a trangular mesh), we may encounter a serous dstorton,.e. some trangles flp over or foldover. Addtonally, n varous mage and graphcs applcatons, parameterzatons are usually requested to satsfy some specfed postonal constrants. Ths further aggravates mesh dstorton,.e. foldovers spread over the whole doman. The followng outlnes the challengng ssues n dfferent constraned surface parameterzaton applcatons. (1) Parameterzaton wth the fxed boundares. Ths knd of constraned parameterzaton methods s categorzed nto the dscrete harmonc maps. In general, harmonc maps are guaranteed to be one-to-one for convex regons [Tutte 1963]. 3

16 However, the convexty constrant may yeld bg dstortons near the boundary. The challengng ssue from such applcatons s how to choose a sutable boundary mappng such that the constraned parameterzaton ncurs the mnmum dstorton. Ths mples that we choose the sutable boundares not only on the parameter doman, but also on the orgnal surface. (2) Parameterzaton wthout the fxed postonal constrants. Ths knd of constraned parameterzaton methods s categorzed nto the dscrete conformal mappngs. Usually, they maxmze the conformalty (.e. angle preservaton) of the pecewse lnear mappng wthout the fxed postonal constrants. Instead, the shapes of the parameter domans are determned by the methods themselves. The challengng ssue from such applcatons s that a number of constrants have to be taken nto account to guarantee the valdty of the soluton. In general, solvng a large nonlnear system s not smple task. The above mentoned parameterzaton applcatons ndeed pursue the sometrc mappng. Thus, the constrant of the mnmum dstorton can be referred to rgd constrant. To dstngush from the rgd constrant, we wll refer to the nternal constrant ponts as the postonal constrants n ths thess. (3) Parameterzaton wth the nternal postonal constrants. Unlke the rgd constrant, these postonal constrants are used to establsh maps wth feature correspondence between objects. The challengng ssue s two-fold, 1) parameterzaton can accommodate the specfed postonal constrants; 2) parameterzaton ncurs dstortons as small as possble. 1.3 DISSERTATION CONTRIBUTIONS 4

17 Ths thess s concerned wth the surface constraned re-parameterzaton, and further nvestgates the parameterzatons (1) and (3) mentoned n Secton 1.2. Because the constrants are applcaton-dependent, ths thess ams at the followng three knds of constrants n terms of applcatons, that s, postonal constrants, tradeoff of postonal constrants and rgd constrant, and rgd constrant. The encountered challenges nclude, the trangle foldovers, the sutable boundary mappng and mnmum dstorton. Our contrbutons are shown as follows SIMPLE POSITIONAL CONSTRAINTS We focus on the applcaton of foldover-free texture mappng wth the specfed postonal constrants. Texture mappng needs to embed a 3D surface nto a 2D doman wth some postonal constrants. Usually, t s concerned wth the postonal constrants nstead of the caused dstortons. The specfed postonal constrants always result n large deformaton n such scenaros. We present the foldover-free reparameterzaton method based on radal bass functons (RBF). The major contrbutons are as follows: To the best of our knowledge, the proposed method s the frst RBF-based approach that ensures user-specfed constrants are satsfed and that foldovers are avoded. An explct mathematcal condton guarantees that no mesh foldover s generated durng the RBF-based reparameterzaton. Ths s called the foldover-free condton; The RBF-based method s a mesh-free approach. Thus, generatng smooth 5

18 texture mappng s possble wthout an extra computatonally expensve smoothng optmzaton, as requred n [Kraevoy et al. 23 and Lee et al. 28]; To the best of our knowledge, the proposed method presents the frst mplementaton of satsfyng postonal constrants wthout predefned fxed boundares. Furthermore, the proposed method can handle models wth nteror boundares wthout addtonal treatment, such as cuttng the model nto several peces, as requred n [Lee et al. 28]. The related work has been publshed n, Yu, Hongchuan, Lee, TongYee, et al., 212, RBF-Based Reparameterzaton Method for Constraned Texture Mappng, IEEE Trans. on Vsualzaton and Computer Graphcs, Vol.18, Issue 7. Yu, Hongchuan, Zhang, Jan J. and Lee, Tong-Yee, Foldover-free Hgh Dmensonal Deformaton, The Vsual Computer Journal, (to appear) TRADEOFF OF POSITIONAL CONSTRAINTS AND RIGID CONSTRAINT Ths ndeed nvolves two knds of constrants, rgd constrant and postonal constrants. The former refers to the dstorton caused by deformaton as small as possble. Image warpng and regstraton usually expect reduce dstorton caused by deformaton as much as possble. For example, the bones should reman rgd n mage regstraton rather than any bendng. The latter refers to the usual postonal constrants. We focus on the applcaton of 2D shape and mage deformaton wth such constrants. A segment or patch of an mage has been selected. The user deforms ths segment/patch by movng a set of constrant ponts wthn the segment. We hope to compute a natural deformaton of ths segment to algn the specfed postons of the constrant ponts. Smultaneously, we expect that such deformaton 6

19 causes a farly mnor dstorton as well, that s, deformaton s close to sometrc. The challengng ssue s to seek a tradeoff between the postonal constrants and as-rgdas possble deformaton. We presented a topology preserved shape deformaton algorthm accordngly. The major contrbutons nclude, Consstency of deformaton felds. Unlke the prevous approaches, we do not drectly solve 2D shape and mage deformaton as a global optmal problem wth constrants, but frst convert the deformaton nto vector felds takng nto account the specfed postonal constrants. Then, the deformaton feld s further reconstructed from a feasble subspace mantanng vector feld consstency. Ths can effectvely overcome the foldover challenge. As-Rgd-As-Possble Deformaton. We ncorporate wth the consderaton of as As-Rgd-As-Possble deformaton n our mplementaton. Ths can effectvely reduce dstorton caused by deformaton. The related work has been publshed n, Yu, Hongchuan and Zhang, Jan J., 212, Topology preserved shape deformaton, The Vsual Computer Journal, Vol.28, No.6-8, pp WELL-SHAPED AND WELL-SPACED MESH Ths essentally requests a sutable boundary mappng to reduce dstorton caused by parameterzatons. The emphass s the rgd constrant nstead of the nternal postonal constrants. The rgd constrant s usually requested by varous mesh edtng systems. Consder such a map between two surfaces that allows transferrng the detals from one model to another, or nterpolatng the shape and appearance of 7

20 the two models. In ths applcaton, overlappng the two 2D parameterzatons for matchng s the foundaton. Unfortunately, the current 2D parameterzaton approaches nevtably ncur large dstorton except the developable surfaces. Recent works [Gu et al. 22, Guskov et al. 2, and Lee et al. 2] ndcate that the small mesh wth rectangle, trangle or crcle-lke and smooth boundares s easy to flatten. Moreover, the well-spaced and well-shaped meshes dstrbute samples more unformly over the surface and hence better capture surface features. In other words, the meshes need to be pre-processed for low-dstorton parameterzatons. Followng ths ssue, we develop two pre-processng procedures for that we would lke n our applcaton of skn sldng. The man contrbutons of our work nclude, Parttonng a whole mesh nto a set of patches. Moreover, the boundares of the patches are becomng smooth by a further refnement procedure; Mesh smoothng. We present a new approach to smooth meshes whle preservng ther ntrnsc features. The resultng meshes reman well-shaped and well-spaced. The related work has been submtted to, Yu, Hongchuan, Yang, Xaosong and Zhang, Jan J., Laplacan Edtng For Skn Sldng, submtted to Computer Graphcs Forum Journal. 1.4 THESIS OVERVIEW The rest of ths thess s organzed n the followng sx chapters: Chapter 2 gves a detaled lterature revew of the related works. 8

21 Chapter 3 addresses the ssue of the postonal constrants n surface parameterzaton. In ths chapter, ths foldover problem s addressed by developng the radal bass functon (RBF)-based re-parameterzaton. Gven ntal 2D embeddng of a 3D surface, the proposed method can re-parameterze 2D embeddng nto a foldover-free 2D mesh, satsfyng a set of user-specfed constrant ponts. In addton, ths approach s mesh-free. Therefore, generatng smooth texture mappng results s possble wthout extra smoothng optmzaton. Chapter 4 further addresses the generalzaton of the proposed RBF-based reparameterzaton approach to hgh dmensonal datasets rather than 2D parametrc doman. We further apply the generalzed approach to 3D bran volume dataset deformaton and 3D polyhedral deformaton. Chapter 5 stll addresses the postonal constrants n 2D parameterzaton. Unlke Chapter 3 and 4, there s an addtonal constrant to be taken nto account here, that s, rgd constrant. The man problem n the proposed RBF-based reparameterzaton method and ts extenson n Chapter 3 and 4 s that they have nfnte support. Ths mples that any local deformaton could result n small dstortons over the whole mage doman. Our 2D shape and mage deformaton applcaton requests the resultng dstorton as small as possble. Thus, ths s ndeed to seek a tradeoff of the postonal constrants and as-rgd-as-possble deformaton. Chapter 6 addresses the pre-processng for 2D low-dstorton parameterzatons, that s, mesh partton and smoothng. Ths s because well-spaced and well-shaped meshes suffer small dstorton n 2D parameterzaton. The applcaton s the skn sldng that smulates the skn movng over underlyng layers of fat, muscle and bone. Skn sldng, as a secondary anmaton technque, brngs about extra realsm to character anmaton. The physcally based skn sldng 9

22 approaches usually suffer from nherent computatonal complcatons. To avod ths numercal challenge, some nterpolaton technques are appled to the mplementaton of skn sldng. However, there are many defcences n practce, e.g. mssng out features of skn surface and smoothness ssue etc., whch greatly reduce a realstc appearance. Essentally, skn sldng assumes that the shape can be preserved and the features of skn surface can be transferred to the target mesh. We focus on these two aspects and further reformulate the mplementaton of skn sldng based on the graph Laplacan framework, whch helps our proposed algorthm to mplement the mesh partton, shape and feature preservaton. The elements worth mentonng nclude the mesh partton and mesh smoothng. The former s a new applcaton of the actve research ssue: mesh segmentaton. The novelty of our method s to automatcally extract the sknnng regons and further smooth the partton boundares. The latter s a new method for mesh smoothng. Chapter 7 summarzes our research work presented n ths thess, and further gves out our future works. 1

23 CHAPTER 2 LITERATURE REVIEW Surface parameterzaton was frst ntroduced nto computer graphcs as a method of texture mappng [Benns et al. 1991], whch has been becomng a vtal tool for many graphcs applcatons n the last decade. Parameterzaton not only refers to mappng a surface nto a plane doman, but t s also requested to map the surface nto some 3D smplex, such as spheres. Floater et al. [Floater and Hormann 25] and Sheffer et al. [Sheffer et al. 26] have gven the detaled surveys on ths topc respectvely. From the vewpont of users, ths chapter surveys the applcatons whch beneft from surface parameterzaton. 2.1 TEXTURE MAPS 11

24 Texture map s usually expected to enrch the appearance of a model n a statc mage. The challengng problem s computng texture coordnates to satsfy userspecfed correspondence between the 3D model and texture mage. However, ths has not been gven much consderaton n lterature. Few studes have been conducted on meetng soft constrants [Levy and Mallet 1998, Cabral et al. 29] (.e., to satsfy the postonal constrants approxmately). Levy [Levy 21] and Desbrun et al. [Desbrun et al. 22] proposed a least-squares system and Lagrange multplers as solutons, respectvely. The basc dea can be smply descrbed as follows, m T T E T ) = ( X ) j T ( U j ) + + dudv 2 2 j= 1 u v Ω 2 ( λ, (2.1) 2 where T s a functon representng a surface, X j denotes a set of m constrant ponts that are passed through by the surface T(U), and these constrant ponts are assocated wth the parameter-space ponts U j. Mnmzng the above functonal yelds a sutable mappng T from the parameter space to the surface. The frst term represents the squared devaton at the constrant ponts and the second term enforces the smoothness of the soluton. The resultng soluton may be a compromse between the accuracy of the fttng and the smoothness of the soluton. The gven constrants are therefore called as soft constrants. However, these two methods fal to guarantee a bjectve embeddng. Zhang et al. [Zhang et al. 25] focused on a specal case (.e., deformng a patch by stretchng ts boundary). Occurrence of foldovers when nternal postonal constrants are added n the orgnal patch, and whether these can converge to the expected postons, were not clearly stated. In contrast, hard constrants were studed n [Kraevoy et al. 23 and Ecksten et al. 21] because a perfect texture algnment s essental at certan delcate areas of a 12

25 mesh. Ecksten et al. [Ecksten et al. 21] proposed a constraned smplfcaton to algn constrants, addng Stener vertces to avod foldovers. Theoretcally, although the above method can handle large sets of constrants, t s extremely complcated and not very robust [Kraevoy et al. 23]. In addton, only smple examples were shown n [Ecksten et al. 21]. Thus, whether the above method can handle more complcated constrants s not clear. Kraevoy et al. [Kraevoy et al. 23] and Lee et al. [Lee et al. 28] performed embeddng by addng a fxed rectangular vrtual boundary, after whch the Delaunay method was appled to trangulate the regon between true and vrtual boundares. After algnng user-specfed hard constrants, the embeddng s usually hghly dstorted. Therefore, a post-smoothng procedure s requred to reduce the dstorton, addng to computaton costs. To avod trangle foldovers, swappng edge s appled here. Fgure 2-1 llustrates the dea of swappng edge [Lee et al. 28]. When the vertex v moves to v α as shown n Fg 2-1a, the shaded trangles wll be degenerated. Frstly, we can swap the edge vq 4 wth q to 5 the trangle vq 3 q4. Then, we swap the edge vq 3 wth q q 2 5 to remove the trangle 3 q vq q. After that, swappng the edge 2 3 vq 7 wth q 6q8 to remove the trangle vq q, 7 8 all of the potental folodover trangles are removed. Kraevoy et al. [Kraevoy et al. 23] faled to completely remove foldovers because the consstent neghborng orderng was not consdered n fndng matchng trangulatons. Fujmura et al. [Fujmura and Makarov 1998] presented an magewarpng method. To satsfy postonal constrants, the Delaunay trangulaton and edge swaps were repeatedly used n ther work to avod foldovers. However, edge swaps can damage the geometrc surface when used to texture map a 3D mesh, as dscussed n [Ecksten et al. 21]. 13

26 Fgure 2-1. Illustraton of swappng edge. Tang et al. [Tang et al. 23] and Lee et al. [Lee and Huang 23] proposed an RBF-based parameterzaton method. However, nether method mentoned the foldover challenge. Tddeman et al. [Tddeman et al. 21] appled the condton of postve Jacoban determnant to remove foldovers n ther mage warpng applcaton. Ths condton s well known n dfferental geometry to ensure one-toone mappng [Mesters and Olech 1963]. The method starts from an ntal dense mappng that s lkely to contan foldovers. Foldovers are then removed by teratvely scalng the gven mappng. However, dense mappng s dffcult to establsh beforehand. Moreover, the prmary defcency of ths method s that the convergence cannot be guaranteed. In a gven dscrete settng, scalng a gven dense mappng usually results n teratve step-length towards zero quckly, as admtted by the authors. In a few extreme cases, the method cannot satsfy the specfed postonal constrants. 2.2 TEXTURE SYNTHESIS 14

27 Texture synthess refers to creatng texture over an arbtrary surface mesh usng a gven 2D texture element, whch s usually n the form of an mage or a patch. For smplcty, we can synthesze the texture drectly on a 2D parameterzaton n terms of the gven exemplar texture nstead of overlappng wth a whole texture mage. To dstngush from sold textures, we use the term of surface texture to descrbe such geometry-nfluenced textures here. The type of texture can be qute vared, e.g. the natural examples of surface texture ncludng the pattern of bark on a tree, spots and strpes of horses, fshes and brds etc., and the patterns of flowers and trees on a hllsde. Fgure 2-2. Illustraton of texture synthess. The current technques for texture synthess on surfaces can be roughly categorzed nto two groups. The frst group [Gorla et al. 21, Tong et al. 22, Zhang et al. 23, and Ashkhmn 21] s based on per-pxel nonparametrc samplng. [Heeger et al. 1995] utlzes the mage pyramd strategy to capture the statstcal propertes of the exemplar mage at dfferent levels of resoluton. 15

28 Intalzng the syntheszed texture by random nose, each level of the syntheszed pyramd wll converge to the appearance of the correspondng exemplar mage pyramd by usng Hstogram matchng operator teratvely. However, ths approach usually works well on stochastc textures whle s not sutable for the structured exemplar texture. The relatonshp between pxel color and ts spatal neghborhood s another research focus of ths group. In [Efros et al. 1999], each pxel of the exemplar texture s frst modeled by ts neghborhood kernel. Then, each pxel n the target mage s substtuted one by one wth the one that has the most smlar neghbor n the exemplar texture. [We and Levoy 2] further extends ths method n the followng aspects, mplementng mult-resoluton by usng Gaussan pyramds, and modfyng the orgnal neghborhood matchng to the order-dependent matchng. The basc dea s llustrated n Fgure 2-2. Pxels are generated n scanlne order. Thus, each pxel mantans ther ndvdual L-shaped neghborhood. The value of a pxel s determned by choosng the best pxel n the nput mage. Best pxel s the one whose L-shaped neghborhood most closely resembles the neghborhood of the pxel currently beng syntheszed n the output mage plane. The dsadvantage of ths group s that per-pxel samplng s susceptble too much. Ths s because the used L 2 -norm s poor to perceptual smlarty. As a result, t s dffcult for the algorthms n ths group to mantan texture patterns wth certan types [Zhang et al. 23, Ashkhmn 21]. The second group refers to the patch based methods [Efros and Freeman21, Praun et al. 2]. Roughly speakng, ths knd of algorthms s namely to copy and paste mage patches repeatedly to fll the target mage. [Efros and Freeman 21] makes use of the overlap regon between adjacent patches to approprately qult them, so that these patches all ft together. The patch s selected from a set of canddates 16

29 accordng to an overlap error mnmzaton. To speed up ths algorthm, [Kwatra et al. 23] solves the error mnmzaton problem usng a graph-cut algorthm. Moreover, [Kwatra et al. 25] presents a global optmzaton framework to synthesze a new texture. It essentally mnmzes an energy functon that consders all the pxels together. The energy functon measures the smlarty to the exemplar texture. To hde patch seams and enhance the smoothness across the seams, ths knd of algorthms also performs graphcut on surfaces. Addtonally, users always nteractvely select the exemplar texture for the dfferent regons of a surface separately n practce,.e. cuttng a patch for texture synthess and then pastng t back. The potental challenges nclude, surface partton, mesh remeshng and seamless embeddng. Ths essentally requres a pre-processng on the nput meshes before the followng parameterzaton procedure. 2.3 IMAGE WARPING/DEFORMATION We frst menton the medcal magng, snce the use of medcal magng has been rapdly ncreasng n recent years. Image regstraton s ndeed one of the kernel technques n medcal magng. Medcal mage data usually refers to computed tomography (CT), ultrasound (US), magnetc resonance magng (MRI), postron emsson tomography (PET) and sngle photon emsson computed tomography (SPECT). For vsualzaton and pathology analyss purposes, such mage datasets are usually requred to algn to a specfed template or vce versa,.e. to establsh a bjectve mappng between mages. Such mappng s called as mage regstraton. The deal regstraton method should nclude local as well as global deformaton, that 17

30 s, fully affne and elastc deformaton. The usual constrants are the postonal constrants (also called as landmarks). Roughly speakng, mage regstraton can be classfed nto two categores, PDEs based approaches and parametrc model based ones. In the frst category, mage regstraton s usually found by solvng a nonlnear PDE system. For example, bran tssues are regarded as a lnear elastc or vscous flud that s subject to a constraned deformaton [Karacal and Davatzkos 24, Haber and Moderstzk 27, Beg et al. 25]. The basc dea s namely to fnd a smooth transformaton T by mnmzng the followng functonal, d E T ) = D( I, I ( T ( X ))) + α S( T ), X R, d 2 or 3, (2.2) ( 1 = where I and I 1 are the gven mages and α s a regularzaton parameter and compromses between smlarty and regularty. The functonal D measures the dstance between I and I 1 n terms of the sum of squares dfference,.e., 2, I1( T )) I I1( ) D( I = T. (2.3) The regularzaton term s usually expected to yeld a smooth and unque transformaton T. It can be mplemented by the elastc regularzaton as below, j S( T ) = β T + β T, (2.4) j where β 1 and β 2 are the so-called Lame-constants. Moreover, for the large deformaton scenaros, t s hard to prevent foldovers based on the above-mentoned functonal. To deal wth ths challenge, Jacoban constrant s requested n many algorthms, that s, ( I + T ) <, for all X < C( T ) = det d, (2.5) 18

31 where I d denotes an dentty Matrx. As a result, the functonal s rewrtten as, ( I, I ( T )) S( T ) + γ C( ) 1 E( T ) = D α T. (2.6) The second category s to model the regstraton by a parametrc functon wth a set of the undetermned parameters. Usually the mappng s known contnuously and the resoluton of the mappng can be controlled ndependently of the mage resoluton. [Kybc and Unser 23] ntroduced B-splnes to model the mage regstraton. As the usual PDEs based regstraton, they added a penalzaton term as the regularzaton term. [Sorzano et al. 25] further evaluated the effcency of [Kybc and Unser 23] and showed that B-splnes were a good alternatve compared to other parameterzaton, such as wavelet or Fourer representaton. In [Rohlfng et al. 23], the penalzaton term s defned as the absolute value of the log of the Jacoban of the deformaton or the square of the second dervatve for each voxel. They used fnte dfferences to compute the gradent of the Jacoban term. Due to computatonal complexty, [Musse et al. 21] suggested a block nonlnear Gauss-Sedel algorthm to mnmze the energy between the mages wth the constrant that the Jacoban was contnuously postve. We hope to pont out that the constrants on the Jacoban are also the foldover-free constrant descrbed n ths thess. Then, we summarze the some recent mage warpng applcatons as well. Image reszng s used to stretch-and-squeeze mage so as to ft dfferent dsplay devces. The salent mage regons should reman unchanged or have a mnmal dstorton. Vdeo retargetng further extends the mage reszng technques to a vdeo dsplay. Lke the other mage warpng or morphng applcatons [Fujmura and Makarov 1998, Tddeman et al. 21, Weber et al. 29], the challenges nclude pxel overlap, 19

32 jagged and blurred edges. Many current nterpolaton technques, e.g. blnear and bcubc nterpolaton, can effectvely amend the ssue of edge smoothness. However, to the best of my knowledge, pxel overlap (.e. non-bjectve warpng) s stll an unsolved problem. I therefore focus on ths challenge n ths thess. Addtonally, mage or vdeo frame are usually vewed as the quadrangle meshes that are ntally a grd of axs-algned squares. Pxel overlap can be vewed as trangle/quad flppng. It s called the trangle/mesh foldover. Compared to trangle meshes, t s hard to formulate ths challenge by usng quadrangle meshes. 2.4 MESH EDITING Shape deformaton or nterpolaton s an mportant research topc n Computer graphcs. A number of current shape deformaton approaches focus on the As-Rgd- As-Possble technques [Igarash et al. 25] that computes a natural shape deformaton. Ths dea s further extended to 3D mesh deformaton [Alexa et al. 2] and 3D mesh parameterzaton [Sorkne and Alexa 27, Lu et al. 28]. It s nterestng that they both adopted an alternatng least squares scheme (.e. known as local/global algorthm) to approxmate the As-Rgd-As-Possble deformaton. In the local step, to measure the rgdty of a deformed mesh, we have the followng functonal, n ( p p ) R ( p p ) E( S ) = w w, (2.7) j = 1 j N ( ) j j 2 where w and the mesh S and w j are some fxed cell and edge weghts, S denotes a deformaton of R denotes a rotaton of some cell. For one cell, the optmal rotaton 2

33 R brng about rgd deformaton nstead of any non-rgd ones. Wthn the neghborhood of some vertex, we have the followng lnear system by mnmzng the above functonal, w w. (2.8) j ( p p j ) = ( R + R j )( p p j ) j j N ( ) j N ( ) 2 In the global step, applyng the dscrete Laplace-Beltram here yelds the followng lnear system, L p = b, (2.9) where vector p contans the unknown coordnates of the vertces, and the constrant vertces can be concatenated to the parameter matrx L and vector b. Ths local/global scheme can be appled to 2D meshes as well [Karn et al. 29]. Other approaches, such as [Weng et al. 25], cast deformaton as an energy mnmzaton problem. Detal-preservng and postonal constrants are added nto the energy functonal as constrant terms. The dstnct advantage of the former s to easly add the constrants to each trangle by solvng a local optmzaton problem. However, regardless of global optmzaton or local/global schemes, foldovers of the underlyng mesh have not been overcome yet. [Karn et al. 29] suggested to explctly restrctng the local step to avod t, whle [Wang et al. 28] proposed to prevent t n a heurstc manner. For 3D scenaros, foldovers usually take place at sknny trangles. Numercal nstablty caused by sknny trangles s essentally from trangle reflecton. Moreover, mesh edtng benefts from the local parameterzaton between pars of mesh patches as well. Usually, one can locally parameterze the regons of nterest (ROI) on the two models n a 2D doman and then overlap the 2D parameterzatons. 21

34 The cut-and-paste transfer presented n [Bermann et al. 22] can effectvely transfer the detals between models usng local parameterzaton. [Sorkne et al. 24, Levy 23] also used the local parameterzaton for mesh composton n a smlar manner. They frst overlapped the 2D parameterzatons of ROIs to yeld a mappng and then made use of t for extractng and smoothly blendng shape from the two models. In such applcatons, there are no any postonal constrants n 2D parameterzatons except the fxed boundares. However, for more precse controls, t s natural to add some nternal postonal constrants. Ths wll lead to the challengng ssue of trangle foldovers agan. 2.5 MESH COMPLETION AND COMPRESSION Trangulaton on range data usually result n a mesh contanng holes and multple components. [Levy 23] extracted the hole boundares usng 2D parameterzaton and trangulated those. To a set of patches, we have to buld a model by regstraton. In many cases, there exsts pror knowledge on the overall shape of the scanned model. For example, for human scan, a generc human shape model s readly avalable. [Allen et al. 23, Anguelov et al. 25] respectvely used ths pror model to facltate completon of scans. They calculated a mappng between the scan and the pror human model. Moreover, [Kraevoy and Sheffer 25] presented a general and robust template based approach for completon of any type of scans. They fxed the boundary of a group of base mesh faces, updated the barycentrc coordnates n the nteror, and then possbly re-assgned some vertces to dfferent faces nsde the group. 22

35 Whle keepng the mesh completon, mesh compresson s used to compactly store geometry models [Allez and Gotsman 23]. Compresson rate s nversely proportonal to the data entropy. When meshes are regular n both topology and geometry, the compresson rates are hgher. Topologcal regularty usually refers to meshes where almost all vertces have the same degree. Geometry regularty refers to meshes where almost all the trangles are smlar to each other n terms of shape and sze, and vertces are close to the centrod of ther neghbors [Gu et al. 22, Hoppe and Praun 25]. Such meshes can be obtaned by parameterzng the orgnal mesh and remeshng them [Guskov et al. 2, Khodakovsky et al. 23]. 23

36 CHAPTER 3 RBF-BASED RE-PARAMETERIZATION METHOD FOR CONSTRAINED TEXTURE MAPPING 3.1 PROBLEM FORMULATION BASIC IDEA AND MOTIVATION Ths chapter addresses the surface re-parameterzaton wth the postonal constrants. The challengng s to overcome the trangle foldovers n mesh parameterzatons. The overvew of the proposed algorthm s as follows. An nput 3D surface s frst embedded nto a 2D convex doman wth harmonc mappng [Guo et al. 25]. A mathematcal foldover-free condton (see Secton 3.2) s derved, and ncorporated nto an RBF-based reparameterzaton algorthm. The algorthm then teratvely algns user-specfed postonal constrants. The man dea s to frst estmate the teratve step length (.e., scalng factor) subject to the foldover-free 24

37 condton, and then to successvely approxmate the desred postons through RBFbased deformaton. In short, RBF s used to teratvely deform the 2D mesh to algn user-specfed constrants. Wth the foldover-free condton at each teratve step, the deformaton s prevented from beng over-aggressve (.e., to nduce foldovers). Due to more mathematc computatons, we frst descrbe the proposed RBF-based reparameterzaton method brefly, and then separately address the each step n detals n the followng sectons ITERATIVE RBF-BASED REPARAMETERIZATION PROCEDURE 2 For a gven 2D mesh embeddng S of R, a transformaton T s a one-to-one 2 mappng of ponts X S onto another 2D parametrc doman U Ω of R, wth * * arbtrary m constrant pont pars ( X U ) : T T X = ( xy, ) S UX ( ) = ( ux ( ), vx ( )) Ω T : * * subject to U( X ) = U, = 1,..., m. (3.1) The reparameterzaton algorthm s developed based on the RBF scheme. RBF ensures a smooth fnal parameterzaton due to ts numerous excellent propertes, such as beng mesh-free and C 2 contnuty. Moreover, the most mportant advantage s the sutablty of RBF for mplementaton n a successve approxmaton. Ths can smoothly deform S to algn user-specfed constrants, as demonstrated later. The RBF-based method s renforced wth the proposed foldover-free condton to approprately control the dsplacement of X S at each teraton. The dsplacement of each pont coordnate s computed wth the RBF scheme to mplement successve approxmaton: 25

38 m λ (3.2) U= PX ( ) + φ( X C) T where the coeffcent λ = ( λ, λ u v) s a vector, C ( c, c x y) ponts, U ( u, v) T T = denotes the constrant =, and P(X) s an affne transformaton PX ( ) a a a X. Although varous radal bass functons exst, thn plate = b1 b + 2 b 3 2 splne (.e., φ ( r) = r ln r) s adopted for ts smplcty. The deformed U s obtaned by updatng U(X) = X + ΔU. For the next teraton, let X U(X). The followng pseudocode gves an overvew of the reparameterzaton algorthm. The algorthm s executed teratvely; the superscrpt j stands for the teraton ndex. Let the ntal 2D mesh embeddng be S (), a set of user-specfed constrant pont () (*) pars be ( ) C, C, = 1,..., m, on S and U. * C denotes the fnal constrant ponts whereas C s the updated pont per teraton. ( j ) Loop: (1) Estmate the scalng factor δ by Eq.3.14 (see Secton 3.2.2) based on the ( j) confguraton of the current m constrant ponts C ; ( j+ 1) * ( j) (2) Compute the current m constrant pont dsplacements by C = ( C C ) δ, ( 1) ( 1) such that j + j j + C = C + C ; (3) If δ < δthreshold, apply trangle subdvson (see Secton 3.3) then go to Step (1); otherwse, (4) Compute the dsplacements of the ponts on S ( j) by Eq.3.7 (see Secton ( j 1) 3.2.2) based on all m updated C + ( ) ( 1) and updatng j j S S + ; ( j ) * Repeat untl C = C. In ths procedure, C * denotes the desred postons. Intally, f C * s appled to drectly deform S n Equaton 3.2, the result s usually too aggressve and foldovers may occur. Therefore, n Step (1), a conservatve scalng factor δ needs to be computed and used to ensure that ( j 1) C + s not over-aggressve. Note that the teratve RBF-based reparameterzaton procedure can defntely change mesh S to a 26

39 foldover-free state. However, the fnal postons may not algn exactly wth the constrants * C n some extreme scenaros. Ths mples that the method s only able to handle soft constrants. To allevate ths problem (.e., to approxmate hard constrants as much as possble), the mesh n Step (3) s subdvded by addng extra Stener vertces. For more detals, see Secton ITERATIVE MECHANISM FOLDOVER-FREE CONDITION From a mathematcal perspectve, a foldover-free parameterzaton yelds a one-to-one mappng between correspondng surfaces (or meshes) and parametrc domans. In the present work, the ntal 2D embeddng of a 3D surface s gven n advance. Focus s gven on deformng ths ntal embeddng wth a set of nternal constrant pont pars. Ths requres that the mappng T s globally unvalent or globally one-to-one (.e., the topology or the relatonshp between any par of vertces n the mesh should reman unchanged before and after parameterzaton). Mathematcally, ths means the determnant of the Jacoban matrx must always be postve [Mesters and Olech 1963]: det( U ) >. (3.3) Accordng to the Gerschgorn crcle theorem [26], a suffcent condton of satsfyng Eq.3.3 can be descrbed as follows: 27

40 u > x v > y u y v x. (3.4) Ths s usually called as the foldover-free condton. The geometrc meanng of Eq.3.4 s smply that the two vectors u ( xy, ), v ( xy, ) are lnearly ndependent of each other; thus, ther ncluded angle s less than π. The former s easy to understand. The latter mples that the rght-hand rule n vector calculus s satsfed over the entre doman. Holdng det( U) < at any pont would result n lefthandedness nstead of rght-handedness. Ths change would cause mesh foldover ITERATIVE STEP-LENGTH ESTIMATION Our reparameterzaton algorthm employs an teratve framework and the dsplacements of vertces are estmated consderng the condton of Eq.3.4. Equaton 3.2 must be rewrtten to mplement the procedure, such that the dsplacements of some ponts lnearly depend on the constraned ponts. Ths mples that deformaton the mesh s acheved by adjustng the dsplacements of the constraned ponts n an teratve manner. A further expectaton s that foldovers wll be avoded by controllng the dsplacement of the constraned ponts n each teraton. The RBF coeffcents ( λ, λ, ab, ) of Eq.3.2 are frst computed, where u v a = ( a1, a2, a3) T and b = b1 b2 b3 (,, ) T. For a gven set of constraned ponts and ther dsplacements, ths can be acheved by solvng the followng lnear system: 28

41 λu xc λv yc K = and K =, (3.5) a b where ϕ P K = T P, ϕj ϕ( C Cj ) =, and P contans the constraned ponts coordnates (.e., c, c y, 1 ) and the vectors x, y hold the dsplacements of the x 1 m T 1 m constrant ponts as c = ( cx,..., cx ), c = ( cy,..., cy ) expressed as follows: c c x y. Ths can be T λu λv 1 xc yc = K. (3.6) a b (For a detaled RBF computaton, refer to [Carr et al. 21].) Substtutng ( λ, λ, ab, ) nto Eq.3.2, the new expresson s as follows: u v 1 xc u = M( X) K 1 yc v = M( X) K ( φ 1 φ m ) M( X) = ( X C ),..., ( X C ), x, y,1. (3.7) Note that Equaton 3.7 descrbes a lnear system of the dsplacement of any X (.e. u, v) and of the constrant ponts (.e. x, y ). Whether or not the resultng mesh satsfes the condton of Eq.3.4 should depend on the confguraton of the 1 current constrant ponts [.e., M(X) and K c c ], rather than ther dsplacements, x c, y. Moreover, durng teratons, M(X) and c 1 K are unfxed and depend on the 1 confguraton of the current constrant ponts. Hence, M(X) and K are gven focus. The dervatves of ( uv, ) ( xy, ) are computed as follows: 29

42 u 1 xc = 1 + xk x M u 1 xc = M yk y v 1 yc = M xk x v 1 yc = 1 + M yk y (3.8) where M (or M ) denotes the partal dervatves of M(X). x y Substtutng the above dervatves nto Eq.3.4 yelds 1 xc 1 xc 1 + MxK yk > M. (3.9) 1 yc 1 yc 1 + MyK > xk M In general, the dsplacement vectors of the constrant ponts ( cx, cy) T can be obtaned by the dfferences of the current constrant ponts coordnates and ther ndvdual targets coordnates. To satsfy the above nequaltes, we may lmt the length of each dsplacement vector by scalng the vectors x, y as follows, c c 1 xc 1 xc 1 + MxK δ yk δ > M, (3.1) 1 yc 1 yc 1 + MyK δ > xk δ M where, δ denotes a scalng factor and δ>. The regons defned by the above nequaltes can be further descrbed as follows: 3

43 1 xc 1 xc Ω 1( δ) = (1 + αβ, ) : α MxK δ, β yk δ,1 α β M + > 1 yc 1 yc Ω 2( δ) = ( α,1 + β) : α MxK δ, β yk δ,1+ β > α M (3.11) where (α,β) denotes a pont n Ω. Fgure 3-2 llustrates the regons Ω δ ), Ω ( ). The 1( 2 δ scalng factor δ s not a constant, and depends on the dsplacements of the constrant ponts. Thus, the dashed lne s used to hghlght these undetermned boundares. β Ω1 l 3 l 5 l 2 l 1 1 l Ω2 α -1 Fgure 3-2. Illustraton of the condton of Eq.3.4. The dashed lnes denote the undetermned boundares The Eq.3.4 condton mples that vectors u ( xy, ), v ( xy, ) should be lnearly ndependent of each other. Fgure 3-2 ntutvely llustrates ths concept by the fve lnes: l 1, l 2, l 3, l 4, l 5. For example, α and β should be above the lne l 3 or under the ne l 4 and above the lne l 5, so that the lnear ndependence can be guaranteed. Ths can be acheved by the scalng factor n Eq Moreover, for smplcty, assume that the vectors x, y have the same dstrbuton. The regons Ω δ ), Ω ( ) c c 1( 2 δ 31

44 would then have the same sze n terms of Eq Ths wll lead to the overlap of straght lnes l 1 and l 2 and form a new dvdng lne, whch s α+β= n Fg.3-2. Lne α+β= guarantees the ncluded angle s less than π. Consequently, the condton of Eq.3.4 can be re-expressed as xc 2 1 xc 2 1 MxK δ yk δ + M 2. (3.12) yc 2 1 yc 2 1 MxK δ + yk δ M 2 Notce that the possble values of u ( xy, )(or v ( xy, )) are assumed to be evenly dstrbuted around the center of (1, ) and (, 1). Ths s because of varous possble confguratons of the constrant ponts (e.g. M, M, ). Hence, crcles K 1 x y are employed to estmate the domans of u ( xy, ) and v ( xy, ). To satsfy the above nequaltes, let 1 xc 1 xc 1 MxK yk δ + M 2. (3.13) 1 yc 1 yc 1 MxK + yk δ M 2 Therefore, the scalng factor s estmated as, δ = mn δ( X ), (3.14) X S where, 32

45 1 1 δ ( X ) = mn, 1 xc 1 yc 2( Mx + My) K 2( x + y) K M M for all vertces X of S. Note that the vectors x, y are the dfferences of the current constrant ponts postons and ther ndvdual tragets postons. Scalng factor δ depends on the confguraton of the current constrant ponts. For any constrant pont C, ts dsplacement needs to be scaled by δ n an teratve way so as to approxmate ts target C *. The confguraton of all the current constrant ponts may be defned as the current state of the mesh. When the constrant ponts postons are updated, the state s changed accordngly as well. Thus, δ can further be vewed as the teratve step length of updatng state of the mesh. c c Our teratve scheme for constraned texture mappng has been outlned. The teratve step length s adaptvely estmated by the current constrant pont confguraton. Before proceedng further, the teratve step length δ of Eq.3.14 s taken as an estmate of the lower bound for our purpose of foldover-free reparameterzaton. The estmate of Eq.3.14 s suffcent [.e., there may be an teratve step length δ beyond the estmate of Eq.3.14 to yeld a foldover-free soluton]. Note that scalng the dsplacements of the constrant ponts only elmnates all probable foldover cases n order to guarantee that the mesh topology s contnually preserved. The goal of Eq.3.14 s to guarantee that the doman s completely foldover-free. Thus, Eq.3.14 only provdes an estmate of the lower bound. An nevtable ssue s the convergence of the proposed teratve process. To answer ths ssue, let us frst gve out a proposton, 33

46 Proposton: The algorthm proposed n secton 3.1 converges to the desred postons. Before proceedng our proof, we need to defne some notatons. Our algorthm proposed n Secton 3.3 s denoted as U = P(X), where P denotes the mappng from the mesh to the parameterc doman. Followng the Banach fxed pont theorem [Krk and Khams 21], we rewrte t as, X = P(X) U + X, and denote t as X = A(X), where A s vewed as an operator. We expect that the constraned ponts X can converge to the desred postons U * n the fnal deformed mesh. Thus, for the constraned ponts (note that X refers to the constraned ponts n ths appendx), our algorthm s expressed as * X PX ( ) U X = + and once converged, t s expected that U = AU ( ). * * Proof: Let Ω be a complete metrc space, e.g. Eucldean space. Defne the sequence { X }, for X1 X = AX ( ), X = AX ( ),... Ths means that the constraned ponts Ω, let n our Algorthm mght start from any ntal locatons. Frst consder XY, Ω. Let α < 1 and d(.,.) denotes the metrc functon, e.g. Eucldean dstance. Heren let dxy (, ) max ( X U *, Y U * ) =. Our algorthm yelds, = + = ( ) + * X PX ( ) U X * Y PY U Y. Due to radal shrnk toward * U, we have d( A( X), A( Y)) αd( X, Y). Then consder the sequence of { X }. We have, 34

47 d( X, X ) = d( A( X ), A( X )) αd( X, X ) d( X, X ) = d( A( X ), A( X )) αd( X, X ) α d( X, X ) d( X, X ) d( A( X ), A( X )) d( X, X ) d( X, X )... n 2 n 1 n n 2 n 1 α n 2 n 1 α = 1 2 For any natural number k, we can further yeld, k d( X, X ) d( X, X ) n n+ k n+ 1 n+ = n 1 n n+ k 2 ( α α... α ) d( X1, X2) n α d( X1, X2), ( n ) 1 α * * * Thus, let n, we have X U,.e. U AU n = ( ). Furthermore, consder the unqueness of our algorthm. Suppose that another U ˆ * * * exsts and Uˆ U. Then we have, < 垐 = α 垐 <, * * * * * * * * du (, U) d( AU ( ), AU ( )) du (, U) du (, U) whch s contradcton. Proof ends. Remark A number of exstng approaches [Kraevoy et al. 23, Lee et al. 28] have also been used to acheve a foldover-free soluton by addng Stener vertces and usng edge-swap operatons [Lee et al. 28]. These are unlke our proposed method, whch utlzes successve approxmaton. Compared to the prevous approaches, our proposed method can generate a smooth soluton wthout the need for postprocessng. In addton, because of the contnuty of the RBF functon, t leads to smaller dstorton durng reparameterzaton. These advantages over other methods are further llustrated n the experment secton. 35

48 3.3 TRIANGLE SUBDIVISION In general, the proposed RBF-based reparameterzaton can effectvely generate a contnuous deformaton to match postonal constrants exactly. However, for extreme scenaros wth large deformaton, Eq.3.2, together wth the estmate of Eq.3.14, may not always converge the mesh to the most deal poston. Lookng at Fg.3-3 for example, two constrant ponts are to be swapped whle the other two ponts are fxed. Wthout trangle subdvson, although the scheme of Eqs.3.2 and 3.14 ensure that the mesh wll converge to a foldover-free state (see Fg.3-3b), the poston s not deal. Ths s a defcency of our proposed scheme n Eqs.3.2 and New vertces should be added by subdvdng the trangles to crcumvent ths ssue. Ths step s smlar to that presented n [Kraevoy et al. 23, Lee et al. 28], n whch extra Stener vertces are added. The basc dea of the subdvson strategy n the present study s to frst determne the potental foldng vertces, and then to dentfy the edges that the vertces wll most lkely cross. Thus, the trangles sharng these edges can be subdvded by addng new vertces around the potental foldng vertces. The underlyng dea s very smple: to approxmate the contnuous mplct functon (.e. RBF) by local upsamplng. More samplng ponts provde more freedom and the hgher the probablty that foldovers could be avoded. The teratve step lengths δ(x) s estmated wth Eq.3.14 for all vertces, to determne the potental foldng vertces when ther δ(x)s are below an emprcally selected threshold d threshold. The approach s summarzed as follows. Assume N selected foldng vertces: Determnng the Most Probable Edges 36

49 DO = 1, N, (1) Extract the 1-rng of the selected vertex v, then compute the probable locaton v of v by Eq.3.7 (Secton 4) ** usng 2 3 tmes the threshold δ threshold (.e., multplyng the vectors xc and y c wth 2 δthreshold or 3δ threshold ; (2) Determne the 1-rng edge of v that ntersects wth lne of vv. Ths edge s called the most probable edge for v ; (3) Bsect the selected edge. The mdpont s then added to the mesh as a new vertex. END DO **Equaton 3.7 s another expresson of Eq.3.2 because Eq.3.7 offers a lnear expresson about the dsplacement of the current constrant ponts. 3.4 EXPERIMENTS AND DISCUSSIONS In ths secton, the proposed method s appled to a number of examples to evaluate ts valdty, effcency, and robustness. For smplcty, the 2D meshes are normalzed n [,1] [,1] doman, and the texture mages are smlarly normalzed, regardless of the aspect rato. Based on ths normalzaton, the threshold used n the algorthm of Secton 3.3 can be preset wthout further tunng FOLDOVER Fgure 3-1 shows the results produced usng several establshed methods [Kraevoy et al. 23, Tang et al. 23, Sorkne and Cohen-Or 24]. As shown n the fgure, the methods are ncapable of completely crcumventng foldovers durng the reparameterzaton process. The frst experment n the present study s to test the proposed method on the same head model n shown n Fgure 3-1a for comparson purposed. The ntal 2D mesh (.e., embeddng or parameterzaton) obtaned by conventonal harmonc mappng s shown n Fg.3-4a. Fgure 3-4 shows the results 37

50 wth dfferent teratons of the RBF-based reparameterzaton. In Fg.3-4a, red stars mark the constrants that need to move to the ponts crcled n whte. No foldover trangles occur durng the teratons; thus, the nternal constrants are satsfed. Note that durng the teratve reparameterzaton process, the boundary of the 2D parameterzed mesh does not have to be fxed on the ntal predefned convex doman. Readers are referred to the accompanyng vdeo (at Ths dstnct advantage offers more freedom than the other methods to reduce mesh dstorton. b. a. 3D mesh c. d. e. f. Fgure 3-1. Illustraton of the foldover results usng four recent parameterzaton methods, and consderng nternal constrants (Note: the mesh detals can be seen more clearly by zoomng n on the document): a) 3D mesh; b) least squares meshes [Sorkne and Cohen-Or 24]; c) RBF-based embeddng [Tang et al. 23]; d) harmonc mappng [Guo et al. 25, Floater and Hormann 25]; e) Delauney trangulaton-based mappng [Kraevoy et al. 23] (the red lnes mark the boundares of the trangle patches wthn whch there s no foldover; however, foldover trangles can be observed around the red lnes.); and f) nset showng the detals of dstorton around the red lne a. b. c. Fgure 3-4. Illustraton of the teratve results of the proposed method: a) ntal mesh wth constrant pont pars; b) 3 teratons; and c) 5 teratons 38

51 3.4.2 COMPARISON OF EXPERIMENTAL RESULTS The technques proposed n Refs. [Kraevoy et al. 23, Lee et al. 28] represent state-of-the-art methods n texture mappng subject to hard constrants. [Lee et al. 28] expermentally showed that ther method s able to handle challengng examples and generate satsfactory results. Therefore, n ths chapter, the proposed algorthm s compared wth the work of [Lee et al. 28], tested on the same set of models. A further test was performed wth a chessboard texture for smoothness comparson. A vsual comparson shows that the smoothness of usng the proposed approach s much better than that n [Lee et al. 28] (Fg.3-5, fourth column). In partcular, the areas of the constrant ponts are smoother wth the proposed method. Ths s because the prevous method [Lee et al. 28] cannot ensure smoothness of deformaton n such areas. As a result, no further postprocessng for smoothng s necessary n the mplementaton of the proposed algorthm. In [Kraevoy et al. 23, Lee et al. 28], ths requred postprocessng tme usually takes much longer than that of the feature matchng process, and becomes the bottleneck of the entre algorthm. However, ther results are not very satsfactory wthout such postprocessng (Fg.3-5, second column). Moreover, to quanttatvely study the dstorton of reparameterzaton, the stretch metrcs defned n [Sander et al. 21] are used. The L-2 norm s used to measure the overall stretch of the parameterzaton, whereas the L-Inf measures the greatest stretch. Good parameterzaton s expected to have very small L-2 and L-Inf. These two metrcs are used to measure dstorton of all the examples n Fg.3-5 (see Table 39

52 3-1). The proposed approach performs sgnfcantly better n most cases than that of [Lee et al. 28], even wth ther smoothng process. Furthermore, the proposed approach s capable of handlng specal models that have more than one border. Fgure 3-6 shows an example of texture mappng the photograph of an orangutan onto a 3D human head model wth three boundares. Ths fgure shows that the proposed method produces a very smooth parameterzaton whle keepng the two nteror boundares (.e., the eyes). Applyng prevous methods [Lee et al. 28] to ths example would usually requre extra treatment, such as cuttng t nto several peces to ensure each pece has no nteror boundares. The proposed approach s essentally a mesh-free method and does not need new any addtonal treatment. 4

53 Fgure 3-5. Smoothness comparson. The 1 st column shows the texture usng the proposed approach. The 2 nd column shows the results wthout the post-processng procedure [Lee et al. 28]. The 3 rd column shows the results wth the post-processng procedure [Lee et al. 28]. The 4 th column shows the results usng our proposed approach. The 5 th column shows the fnal texture mappng results usng the proposed approach. Fgure 3-6. Illustraton of texture mappng wth two nteror boundares. 41

54 3.4.3 COMPLEXITY ANALYSIS The core advantage of the proposed RBF-based reparameterzaton s that the 3 RBF coeffcents are updated at every teraton. The man computaton cost, O(2 M ), s to determne the nverse of a real symmetrc matrx, where M s the number of the 3 constrant ponts. The tme complexty can be estmated as O(2 KM ), where K denotes the number of teratons. Furthermore, consderng the trangle subdvson procedure, computng the estmated teratve step lengths usng Eq.3.14 at every teraton s necessary. Ths wll cost O(N) each tme, where N denotes the number of vertces on the mesh. The tme for trangle subdvson s nearly fxed for each selected foldng vertex. At each teraton, the runnng tme of the trangle subdvson depends on the number of selected foldng vertces m, whch s generally much fewer 3 than N. Therefore, the total tme cost s OK ( (2 M + N+ m)). The majorty of the tme spent s on the computaton of matrx nverse when there are a number of constraned ponts. The tme spent for trangle subdvson s not an ssue. All the experments were conducted wth MatLab on an Intel Pentum GHz PC wth 1 Gb of RAM. Table 3-2 shows the runnng tme of all the examples n Fg.3-5 usng the proposed approach, whch usually converges around 5 8 teratons. Each example usually takes only several seconds to compute wth the proposed method because post-processng for mesh smoothng s unnecessary. In [Lee et al. 28], the post-processng takes more than 1 mnute to obtan the result LIMITATIONS 42

55 The proposed method deals wth soft constrants. In Secton 3.5, a subdvson approach s proposed to ncrease the chances of exactly matchng the desred postonal constrants. However, ths smple approach has ts lmtatons. For example n Fg.3-3a, two constraned ponts can move close to each other, but not reach the desred postons, even when the trangle subdvson strategy s appled. The dsplacement vectors can be rotated to crcumvent ths ssue. The dstance between any two constraned ponts s taken, and the dsplacement vectors of the two selected constraned ponts are rotated 9 clockwse. Fgure 3-3c shows the ntermedate result of rotatng the dsplacement vectors. Fgures 3d 3e show the convergence result wth trangle subdvson and dsplacement rotaton, where the desred deformaton s acheved wthout trangle foldover. Thus, the above ssue s successfully addressed. However, rotatng the dsplacement vectors mght fal f too many constraned ponts crowd together. Fortunately, such extreme cases are rarely seen n texture mappng applcatons. Another lmtaton s that the convergence of the proposed teratve algorthm has not been proven. Although ths s not an emprcal ssue, deally, mathematcal comprehensve proof should stll be gven. Ths wll be studed n future work. a. b. c. d. e. Fgure 3-3. Illustraton of convergence n an extreme case: a) the ntal confguraton of constrant ponts; b) the result wthout trangle subdvson; c) the ntermedate result of rotatng dsplacement vectors; d) the fnal result usng trangle subdvson and dsplacement rotaton; and e) the zoomed-n mage correspondng to the selected regon. Table 3-1. Dstorton metrcs of texture mappng examples n Fg

56 Examples Row 1 Row 2 Row 3 Row 4 Row 5 Row 6 Row 7 Row 8 L L-Inf L-2 [Lee] L-Inf [Lee] We hghlght the cases that the performance of our proposed approach s worse than that of [Lee et al. 28] by shadng. Table 3-2. Statstcs of texture mappng examples n Fg.3-5. Examples Row 1 Row 2 Row 3 Row 4 Row 5 Row 6 Row 7 Row 8 #Vertces #Trangles #Features #Added Vertces #Added Vertces 183/38 77/7 128/1 94/34 127/8 122/25 383/21 69/2 [Lee] Tme (s) Row 6 shows the numbers of the added ponts before and (/) after mesh optmzaton [Lee et al. 28]. Table 3-2 shows the results of the trangle subdvson of all the examples n Fg.3-5. In Row 5, the number of the added vertces n the proposed approach s greater than that n [Lee et al. 28]. Ths mples that the smple subdvson approach adds multple redundant vertces. Lookng at Fg.3-6, the number of added vertces may depend on the level of smoothness and dstorton because addtonal vertces are necessary for smoothness and low dstorton. Nevertheless, the newly added vertces only ncrease the vertex number N on the mesh, rather than the constrant vertex number M or the selected foldng vertex number m. Therefore, ths does not result n a vsble ncrease of total runnng tme. Further extenson of the trangle foldover ssue wll ncur another research focus: global self-ntersecton, that the boundary ntersects tself. Ths stll remans challengng. We thnk, Jacoban constrant of Eq.3.3 cannot suffcently prevent the 44

57 global self-ntersecton. 3.5 CHAPTER SUMMARIZATION Ths Chapter addresses the ssue of the surface parameterzaton wth the usual postonal constrants. The motvaton s from the classc texture mappng applcatons. Thus, ths chapter focuses on the postonal constrants and the caused trangle foldovers rather than the caused dstortons. To ths end, we frst presented the foldover-free condton, and further proposed the RBF-based reparameterzaton approach ncorporatng wth ths constrant condton. However, n some extreme scenaros, the proposed method cannot converge to the desred postons. To handle such challenges, we have to ntroduce the trangle subdvson technques nto the mplementaton of the proposed RBF-based reparameterzaton approach. Our method s appled to the applcatons of 2D mesh re-parameterzaton. It can actually be extended nto 3D or hgher dmensonal datasets, e.g. volume data and polyhedrons. Ths wll be the task of the next chapter. 45

58 CHAPTER 4 EXTENSION OF RBF-BASED RE- PARAMETERIZATION METHOD TO HIGH DIMENSIONAL DATASETS 4.1 FOLDOVERS IN HIGH DIMENSIONAL DATASETS Ths chapter wll extend the RBF-based reparameterzaton approach presented n Chapter 3 to hgh dmensonal datasets. In computer graphcs, hgh dmensonal warpng approaches have been wdely appled to scattered data nterpolaton, feature-based metamorphoss for 2D mages and 3D volume data [Leros et al. 1995], texture mappng, moton synthess [Kovar and Glecher 24], free form deformatons (FFDs) [Sederberg and Parry 1986] and many more. For vsualzaton purposes, we perform t on surface parameterzaton, polyhedral deformaton and 3D volume data regstraton. 46

59 In surface parameterzaton, a 3D surface mght frstly be mapped onto a convex 2D doman wthout foldovers, and then the resultng parameterzaton s further deformed to match the nternal postonal constrants. Foldovers wthn a mesh can be observed when the nternal constrant ponts are taken nto account. Fgure 3-1 of Chapter 3 shows some examples of parameterzaton, whch satsfy the gven postonal constrants. Trangle foldovers can be observed clearly. Voxel-based volume deformaton has been wdely used n mage segmentaton and regstraton, e.g. CT and MRI datasets. Foldovers wthn the volume deformaton usually hde n the volume dataset nstead of ts surface. For mages or volume dataset warpng, the connecton relatonshp of the pxels or voxels s pre-defned by ther grd structures. Due to dense pxels or voxels, t s easy to observe that multple pxels or voxels are mapped nto the same place, whch s called foldover [Fujmura and Makarov 1998]. Usually, foldovers result n unexpected unsmooth features n the deformed mages or volume datasets, such as rregular edges, spkes and jumps. To see t clearly, we performed 3D FFDs on the precentral gyrus of a segmented MRI bran volume dataset as shown n Fgure 4-6a to 6c. We use the Marchng Cubes algorthm to extract the sosurface. One can observe the tetrahedral foldng. For 3D meshes, although the dstrbuton of the vertces s sparser than that of the voxels n 3D volume dataset, foldovers would stll take place durng a large deformaton. Fgure 4-1 shows a smple example of blowng up a segment of tube. The nternal and external surfaces of the tube are represented n a polyhedral form. When the deformaton of the nternal surface s too large, the nternal surface ntersects wth the external one. Our experence from mplementng the sx exstng methods suggests that none of them are able to robustly overcome the challenge of mesh or tetrahedral foldover, 47

60 whch s an unsolved ssue n hgh dmensonal deformable felds as well. In the followng we present our soluton to ths ssue. We wll frst gve out an overvew of our foldover-free deformaton method, whch ncorporates radal bass functons n our teratve deformaton mechansm. Then, we wll further gve out the estmate of the teratve step length for 2D mesh and hgh dmensonal dataset deformaton. Fgure 4-1. The llustraton of the tetrahedral foldover on a 3D polyhedral tube. The red denotes the vertces of the nternal surface, whle the blue denotes those of the external surface. For llustraton, we only show the vertces here. 4.2 ALGORITHM OVERVIEW Consderng a gven dataset S n whch maps the ponts X * * m constrant pont pars ( X U ) n R, a transformaton T s a one-to-one mappng n S nto another desred doman U Ω of R wth arbtrary,.e. T X S U( X ) Ω *. (4.1) subject tou( X ) = U, = 1,..., m : * Our algorthm s developed based on radal bass functons. Ths s due to many well behaved propertes of the RBF scheme, e.g. mesh-free and C 2 contnuous. The most mportant s that the RBF scheme s sutable for mplementaton n an teratve manner, as wll be seen later. 48

61 Unlke the prevous applcatons of RBFs [Guo et al. 25, Tang et al. 23], n our algorthm, the RBF scheme s used to compute the dsplacement of the pont s coordnates, m u = P ( X) + λφ j ( X Cj ), = 1,..., n (4.2) j where the dsplacement s represented as a vector of = ( ), RBF coeffcents denote λ j, Cj,..., T U u u 1 n n R denotes the constrant ponts, and ( ) P X s a affne n transformaton,.e. P( X) = a + a x, n denotes the dmensonalty of the datasets. k k k Although there are varous forms of radal bass functons, we adopt the thn plate splne as φ here for smplcty. The deformed U s obtaned by updatng = +. For next teraton, let X U( X) U( X) X U. Our deformaton algorthm s summarzed as below. The proposed algorthm s executed teratvely, and the superscrpt j stands for the teraton ndex. The teratve mechansm wll be descrbed n detal n the followng secton. (1) Input: Intal dataset S () and a set of user-specfed constrant pont pars () * ( ) C, C, = 1,..., m; (2) Loop: estmate the scalng factor δ by Eq.4.15 (see secton 4.3.2) based on the ( j) * confguraton of the current m constrant pont pars ( C, C ), = 1,..., m ; ( j+ 1) * ( j) (3) Computng the current constrant ponts dsplacements by C = δ ( C C ), ( 1) ( 1) such that j + j j + C = C + C ; (4) Computng the dsplacements of ponts on S ( j) by Eq.4.6 (see secton 4.3.2) and ( ) ( 1) updatng j j S S + ; ( n) * (5) End Loop untl C = C. 4.3 FOLDOVER-FREE ITERATIVE MECHANISM 49

62 From a mathematcal pont of vew, a foldover-free deformaton gves a one-toone mappng between the orgnal surfaces (or datasets) and ther target domans. We wll frst gve out the foldover-free condton, and then formulate our teratve mechansm n a general form FOLDOVER-FREE CONDITION The goal of our work s to develop a foldover-free deformaton approach wth a set of the postonal constrants. Ths requres that the mappng T s globally unvalent or globally one-to-one, that s, the topology or the connecton relatonshp between any par of vertces n the datasets should keep unchanged before and after the deformaton. Mathematcally t means the determnant of the Jacoban matrx must be postve everywhere, det( U ) >. (4.3) Accordng to the Gerschgorn crcle theorem, a suffcent condton of satsfyng Eq.4.3 s expressed as, u x > u n, (4.4) j= 1, j xj where = 1,,n. The geometrc meanng of Eq.4.4 s that the vectors u ( x,..., x ) are lnearly ndependent of each other. For 2D scenaros, ths 1 n mples that the ncluded angle of the vectors s less than π. For 3D scenaros, such three vectors should not stay wthn a plane. Usually, Eq.4.4 s called as the foldoverfree condton. 5

63 4.3.2 ITERATIVE FRAMEWORK Our deformaton algorthm employs an teratve framework and the dsplacements of vertces are determned consderng the condton of Eq.4.4, whch elmnates foldovers f satsfed. We rewrte Equaton 4.2 here, such that the dsplacements lnearly depend on the constrant ponts. To ths end, the RBF coeffcents ( λ, a ), = 1,..., n can be computed by the followng lnear system, where 1 m T n T (,..., ), a ( a,..., a ) λ = λ λ =, λ 1 c K. a = (4.5) Heren, there are m gven constrant ponts. The dsplacements of such constrant ponts denotes as ( 1 m c x,..., x ) T =. K s a symmetrc matrx flled wth the radal bass functons ϕ = ϕ( C C ), k, j = 1,..., m and the constrant ponts kj k j coordnates. (For a detaled RBF representaton, refer to [Carr et al. 21]) Substtutng ( λ, a ) nto Eq.4.2, we can obtan a new expresson of Eq.4.2 as follows, =1,,n, 1 c u = M( X) K M( X) = ( X C ),..., ( X C ), x,..., x,1 ( ϕ 1 ϕ m 1 n ) (4.6) Note that Equaton 4.6 descrbes a lnear system of solvng the dsplacement of any X (.e. { u }) by usng those of the constrant ponts, (.e. { c }). The kernel s 51

64 M( X) K 1 that descrbes the current confguraton of the constrant ponts. The dsplacements { c } gude to the convergence drecton. The dervatves of ( u,..., u ) ( x,..., x ) are computed as follows, 1 n 1 n 1 c 1+ Mx K j j u = = x j c,, 1,..., 1 Mx K j j= n j, (4.7) where M x j denotes the partal dervatves of M(X). Moreover, substtutng the above dervatves nto Eq.4.4 yelds, n 1 c 1 c 1+ MxK M x K j > j= 1, j, =1,,n. (4.8) In general, the dsplacements of the constrant ponts { c } can be obtaned by the dfferences of the current constrant ponts coordnates and ther ndvdual targets coordnates. To satsfy the above nequaltes, we may lmt the length of each dsplacement vector by scalng the vectors { c } as follows, n 1 c 1 c 1+ MxK δ M x K δ j > j= 1, j, (4.9) where δ denotes a scalng factor and δ>. The regons defned by the above nequaltes can be further descrbed as follows, =1,,n, 1 c ( r1,...,1 + r,..., rn) : rj Mx K, 1,..., j δ j = n Ω ( δ ) =, (4.1) n 1+ r > r j j= 1, j 52

65 where ( r1,..., r n) denotes a pont n Ω. The scalng factor δ s not a constant, and depends on the dsplacements of the constrant ponts. For smplcty, assume that the vectors { c } have the same dstrbuton. The regons Ω ( δ ) would then have the same sze n terms of Eq.4.1. In a n-dmenson Eucldean space, the condton of Eq.4.4 can be descrbed as, 2 n 1 c 2 1 Mx K δ j < j= 1 n, (4.11) for the regon Ω ( δ ) and =1,,n. To further llustrate the geometrc meanng of Eq.4.1, we consder the 2D and 3D scenaros here. When n=2, Eq.4.1 s expressed as, c 1 1 c 1 1 Ω 1( δ) = (1 + r1, r2) : r1 MxK, 2 y,1 1 2 δ r M K r r δ + >. (4.12) c 1 2 c 1 2 Ω 2( δ) = ( r1,1 + r2) : r1 MxK δ, r2 yk δ,1+ r2 > r1 M Fgure 4-2 llustrates the regons of Ω δ ), Ω ( ). The dashed lne s used to hghlght these undetermned boundares. 1( 2 δ The condton of Eq.4.4 mples that vectors u ( xy, ), v ( xy, ) should be lnearly ndependent of each other. Fgure 4-2 ntutvely llustrates ths concept by the fve lnes: l 1, l 2, l 3, l 4, l 5. For example, r 1 and r 2 should be above the lne l 3 or under the lne l 4, and above the lne l 5, so that the lnear ndependence can be guaranteed. Ths can be acheved by the scalng factor n Eq.4.1. Moreover, for smplcty, assume that the vectors c1, c2 have the same dstrbuton. The regons Ω δ ), Ω ( ) would then have the same sze n terms of Eq.4.1. Ths wll lead to the 1( 2 δ 53

66 overlap of straght lnes l 1 and l 2 and form a new dvdng lne, whch s r 1 + r 2 = n Fg.4-2. Lne r 1 + r 2 = guarantees the ncluded angle s less than π. Consequently, the condton of Eq.4.4 can be re-expressed as, c1 2 1 c MK x MK y δ + δ < 2. (4.13) c2 2 1 c MK x δ + MK y δ < 2 Moreover, when n=3, Eq.4.1 s expressed as, 1 c1 (1 + rr 1, 2, r3) : r1 MK x, δ c 1 1 Ω 1( δ) = r2 M yk δ,, c 1 1 r3 MK z δ,1 r1 r2 r + > c2 ( r1,1 + r2, r3) : r1 MK x, δ c 1 2 Ω 2( δ) = r2 M yk δ,, c 1 2 r3 MK z δ,1 r2 r1 r + > + 3 c 1 3 ( rr 1, 2,1 + r3) : r1 MK x, δ 1 c 3 Ω 3( δ) = r2 M yk δ,. 1 c 3 r3 MK z δ,1 r3 r2 r + >

67 Fgure 4-3 shows the regons of Ω1( δ), Ω2( δ), Ω 3( δ). Takng the condton of Eq.4.4 nto account, we hope to pont out that the dvdng plane of r 1 + r 2 + r 3 = can guarantee the determnant of Jacobans s greater than Zero. Smlar to 2D scenaros, the condton of Eq.4.4 s expressed as, c1 2 1 c1 2 1 c1 2 1 MxK δ + M yk z δ + M K δ < c 1 2 c c MxK δ M yk δ MzK δ + + <. (4.14) c 1 3 c c MK x δ + MK y δ + MK z δ < 3 Furthermore, the scalng factor s estmated as, δ = mn δ( X ) (4.15) X S where, δ ( X) mn n n 1 1 c = Mx K = 1 j n j= 1 Note that the vectors c, = 1,..., n are the dfferences of the current constrant ponts postons and ther ndvdual targets postons. Scalng factor δ depends on the confguraton of the current constrant ponts. For any constrant pont C, ts dsplacement needs to be scaled by δ n an teratve way so as to approxmate ts target * C. The confguraton of all the current constrant ponts may be defned as the current state of the dataset. When the constrant ponts postons are updated, the state s changed accordngly as well. Thus, δ can further be vewed as the teratve step length of updatng the state of the dataset. 55

68 r 1 Ω1 l 3 l 5 l 2 l 1 1 l 4 Ω2-1 1 r 2-1 Fgure 4-2. Illustraton of the condton of Eq.4.4 n 2D scenaros. The dashed lnes denote the undetermned boundares. Fgure 4-3. The llustraton of the condton of Eq.4.4 n 3D scenaros. So far we have outlned the proposed teratve deformaton mechansm and gven out an estmate of the scalng factor. The dsplacements of the constrant ponts are adaptvely changed n terms of the estmated scalng factor δ of Eq However, a crtcal problem arses, that s, because the proposed teratve mechansm s namely 56

69 * () to dvde C = C C nto a few segments n terms of the scalng factor, how does t guarantee that the dataset could be deformed to the desred deformaton n such a successve approxmaton manner? Before proceedng further, we frstly assume that there exsts a foldover-free soluton for deformaton based on the gven constrant pont pars. Ths s because for some specal constrant confguratons, the fnal deformed dataset may not always converge to the most deal postons, although t can produce a vald useable deformaton. Lookng at Fgure 4-4 for example, there are two constrant ponts to be moved n opposte drectons n a mesh. Occasonally ther movng paths may ntersect each other, e.g. movng from a red pont to the whte pont. Our approach can only converge at an ntermedate state. Obvously, there s no foldover-free soluton for the fnal desred mesh deformaton here. Then, we also hope to pont out that Equaton 4.15 s a suffcent condton, whch means that there mght be some soluton that does not satsfy Equaton 4.15 but stll s foldover-free. Note that the presented foldover free condton of Eq.4.15 only elmnates all probable cases of the foldover (see the dvdng lnes n Fg.4-2 and 4-3), otherwse cannot guarantee the mesh topology-preservng contnually. The usual case s that the warp aganst the Equaton 4.15 s foldover-free at some ponts whle causes foldovers at the other ponts. The goal of Equaton 4.15 s to elmnate all potental foldover cases over the doman. Fgure 4-4. Illustraton of convergence of an extreme case. 57

70 In general, to estmate the scalng factor δ, t s requred to test all the ponts X wthn a dataset. However, our observaton suggests that the constrant ponts always hold larger dsplacements than others at each teraton. Thus, t s suffcent to estmate the δ only by testng the constrant ponts nstead of all the ponts. The core of our RBF-based deformaton approach s to update the RBF coeffcents at each teraton. The man cost s therefore to compute the nverse of a 3 real symmetrc matrx, whch costs O nm ( ), where m s the number of the constrant ponts and n denotes the dmensonalty of the dataset. The tme complexty can be 3 estmated as O( knm ), where k denotes the teraton number. In practce, the number of constrant ponts s always far smaller than that of the ponts wthn a dataset. And our algorthm usually converges wth 3-6 teratons. Although t s costly to nvert a matrx, because the dmenson of the matrces to be nverted s small, n practce the cost s neglgble. 4.4 EXPERIMENTS AND DISCUSSIONS In ths secton, we perform the above-developed method on three knds of datasets, 3D human bran volume data, 3D polyhedron and 2D mesh EXPERIMENTS 58

71 a b c Fgure 4-5. Illustraton of the foldovers on 3D volume data. Applyng 3D FFD to the gyrus of bran volume data, and the resultng deformaton n a) the whole deformed bran gyrus; b) the local detal of foldovers; c) the sosurface wth mesh correspondng to b). For comparson, the results by usng our approach are shown n d)-f). d e f 3D Volume Data The frst 3 mages of Fgure 4-5 show the deformed precentral gyrus of a human bran MRI volume data usng the 3D FFD approach. For comparson, the last 3 mages of Fgure 4-5 show the results of our approach. We utlzed the Marchng Cubes to extract the sosurface for renderng purposes. Ths can hghlght the foldovers occurred n the 3D volume data by comparng the meshes shown n Fgure 4-5c and 5f. One can note the foldng of trangles,.e. some edges nsert n the faces nstead of connectng the edges or vertces of trangles n Fg.4-5c. 59

72 (a) (b) (c) Fgure 4-6. Illustraton of muscle deformaton. (a) shows the surface meshes at the bndng pose; (b) shows the collson occurs between the bone and muscle; (c) shows the foldover-free result by usng our method,.e. collson s solved. 3D Mesh Fgure 4-6 shows another useful applcaton of our method for muscle deformaton. In ths skeleton-muscle-skn three layers system, the bone s movement follows rgd transformatons, whle the skn and muscle are deformed by usng the Maya muscle package. When a character moves ts arm, sometmes the collson between these three structures s nevtable. So, we converted the three surface models nto a tetrahedral volume mesh. The collson between the surface meshes becomes a foldover problem n the polyhedral mesh. One can note that the low bone goes n the muscle n Fg.4-6b. a. Intal b. Iteraton 3 c. Iteraton 5 Fgure 4-7. Illustraton of the teratve results of our method. 6

73 Fgure 4-8. Illustraton of texture mappng. 2D Mesh Fgure 3-1 shows the results created wth four recently publshed methods, whch, as can be seen from the Fgure 3-1b to 1f, are not able to avod foldovers completely durng the re-parametersaton process. In order to make a comparson wth them, we frst tested the above-developed method on the same head model as shown n Fgure 3-1a. An ntal 2D mesh s generated by harmonc mappng n advance, and the constrant pont pars are shown n Fgure 4-7a. The constrant ponts marked wth the red stars need to move to the ndvdual targets wth whte crcles. The results of each teraton wth our RBF-based re-parametersaton are shown n Fgure 4-7b and 7c. The results confrm that by ncorporatng our teratve deformaton mechansm, no foldng of trangles appears durng the deformaton. As an applcaton of surface parameterzaton, we then appled our approach to texture mappng. The texture of an orangutang s photo s mapped onto the 3D human head model as shown n Fgure 4-8. The zoomed-n mage shows that our method produces a very smooth parameterzaton. Ths example also demonstrates that our 61

74 method s very fast. It converges n only fve teratons, even when a large number of constrants are nvolved. Ths s very encouragng as hgh qualty texture mappng s essental n anmaton producton DISCUSSIONS There s already exstng lterature on dstorton measurement for 2D meshes [Sander et al. 21, Knupp 21]. [Sander et al. 21] proposed some stretch metrcs, whereby the L-2 norm s used to measure the overall stretch of the parameterzaton, whle the L-Inf s used to measure the greatest stretch. A good parameterzaton s expected to have very small L-2 and L-Inf. Although a full treatment of dstorton measurement s beyond the scope of ths paper, t s nformatve to use these two metrcs (.e. L-2 and L-Inf) to measure the dstorton from our texture mappng experment n Fg.4-9 (#vertces:1672, #constrants:51, #teraton:5, L-2:1.19, L- Inf:4.447, tme:3.17s). We also used an extreme example smlar to Fgure 12 n [Lee et al. 28] for comparson purposes. A square mesh s deformed by our foldoverfree re-parameterzaton approach wth some specfed constrant ponts. To hghlght the ssue of dstorton, n Fgure 4-9a only one constrant pont s specfed to move to a new locaton (whte crcle). The orentaton of the other two constrant ponts s unchanged. It can be seen that our approach does not result n large deformaton. Compared wth the result (Fgure 12e) from [Lee et al. 28], n terms of deformaton, ours s more desrable. 62

75 a.ntal mesh b.result from our method Fgure 4-9. Dstorton from surface re-parameterzaton. c. result from [LYY8] As can be seen from the results, although our RBF-based deformaton approach does specfcally am for mnmal dstorton, the teratve deformaton mechansm guarantees the deformaton felds to be dffeomorphc. Ths s n favour of dstorton reducton. The experments were undertaken usng Matlab on an Intel Pentum 4 3.2GHz PC wth 1 Gbyte of RAM. Although the code s far from optmsed, because our method has a low computaton complexty as dscussed earler, t s very fast for 2D meshes and 3D polyhedrons. 4.5 CHAPTER SUMMARIZATION Ths chapter extends the RBF-based reparameterzaton method proposed n Chapter 3 to hgh dmensonal datasets. The man contrbuton s to gve the generalzaton of the RBF-based reparameterzaton wth the postonal constrants. However, Chapter 3 and 4 focuses on the postonal constrants rather than the caused dstorton (.e. rgd constrant). Next chapter wll take the caused dstorton nto account as well. 63

76 CHAPTER 5 TOPOLOGY PRESERVED SHAPE DEFORMATION 5.1 PROBLEM FORMULATION Topology preservaton and low dstorton are always concerned n many mage and shape deformaton applcatons. Indeed, the challengng ssue s to seek a tradeoff between the topology preservaton and the dstorton n terms of varous applcatons. In ths chapter, we wll address these two ssues. Regardng mage warpng, there exst two knds of meshes for mage dscretzaton,.e. trangle and quadrangle meshes. In our paper, we prefer trangle mesh. Ths s due to the followng observaton. Consder a quad cell of a quad mesh as shown n Fg.5-1a. If we allow a large rregular deformaton, the quad mesh can cause many numercal dffcultes. For example as shown n Fg.5-1b, pont A moves a large dstance across the edge BC to A. Ths movement causes a twst of the quad 64

77 ABCD, and ths twst cannot be detected by computng the area of ths quad. The quad area s obtaned by the dfference between the areas of the trangles DCB and A BB. Let pont A be close to pont B. The quad area ncreases whle the twst happens. If the quad cell s splt nto two trangles, the twst makes the area of trangle A BB become negatve. Ths s called trangle flppng or foldover. There s no foldover f the sgn of the trangle area keeps unchanged. Therefore, dscretzaton on a trangulaton wll effectvely help detect and further prevent foldovers and sngular Jacobans durng deformaton. A B A B A B D C D C a) a quad cell b) twst Fgure 5-1. Deformaton on quad and trangle meshes. Moreover, we further llustrate that the trangle foldovers cannot be avoded by usng the current proposed As-Rgd-As-Possble approaches [Igarash 25, Karn 29] n Fg.5-2. We frst performed these algorthms on an example mm of [Igarash 25]. To demonstrate trangle foldovers, we set the constrant ponts (red stars) n an extreme confguraton n Fg.5-2. Fgure 5-2a and 5-2b are from [Igarash 25] wth and wthout scale-adjust respectvely. It appears the foldover problem was not consdered n [Igarash 25]. Sorkne et al. n [Sorkne and Alexa 27] proposed to apply a local/global scheme to the As-Rgd-As-Possble deformaton. The shape deformaton wth postonal constrants was frst carred out. Then, the egenvalues of Jacoban matrx were restrcted n the local step for foldover avodance. The global step was used to reconcle these trangles modfed n the local 65

78 step. The usual procedure of the global step s to solve a sparse Posson-type system wth specfed postonal constrants. However, there exst two questons as follows, (1) Is t true that modfyng the egenvalues of Jacoban matrx n the local step could overcome foldovers? (2) Is t possble that the global step may ntroduce foldovers agan? To answer these two questons, we desgned two approaches for the local step and two for the global step respectvely based on the local/global scheme. The local step ncludes the method of egenvalue modfcaton and that of affne decomposton. The former s proposed n [Karn 29]. The latter wll be addressed n the followng secton. The global step ncludes the methods of solvng a sparse Posson-type system wth and wthout the specfed postonal constrants. Fgure 5-2c and 5-2d show the combnaton of the egenvalue modfcaton (n the local step) plus the global step wth and wthout postonal constrants respectvely. As can be seen, foldovers occur due to the postonal constrants used n the global step (Fg.5-2d). Addtonally, due to lack of scale-adjust, the deformed parts appear too large or too small. Ths means the egenvalue modfcaton n the local step causes a large scalng dstorton. To get around ths defcency, we try to utlze the affne decomposton technque here. The basc dea s to decompose a local trangle affne transform nto rotaton and scale-shear transformatons, and then further constran the rotaton and scale-sheer transformatons respectvely for a natural shape deformaton. Fgure 5-2e and 5-2f show the combnaton of the affne decomposton (n the local step) plus the global step wth and wthout postonal constrants respectvely. It can be observed that foldovers enter through the postonal constrants n the global step 66

79 agan (Fg.5-2f). Both Fgure 5-2d and 5-2f show that foldovers are caused by the postonal constrants and enter through the global step agan. It can also be noted that there s no foldover found n Fg.5-2c and 5-2e. Foldovers do not arse va the global step f the postonal constrants are not nvolved. However, ths s not enough to conclude that the local step can overcome foldovers, whch are stll seen n some cases. To llustrate ths, we performed the above two local/global algorthms that do not nvolve the postonal constrants n global step on a square trangle mesh as shown n Fg.5-3. b c d a Fgure 5-2. Illustraton of foldovers by usng recent approaches. Red stars ndcate the specfed constrant ponts. Foldovers are hghlghted by crcles. a) and b) are from [Igarash et al. 25]; c) egenvalue modfcaton n the local step plus the global step wthout constrants; d) egenvalue modfcaton n the local step plus the global step wthout constrants; e) moton decomposton n the local step plus the global step wthout constrants; f) moton decomposton n the local step plus the global step wth constrants. e f a. egenvalue modfcaton n the local step b. moton decomposton n the local step Fgure 5-3. Illustraton of foldovers by usng the local/global schemes. The red stars ndcate the specfed constrant ponts. Fgure 5-2 and 5-3 have shown that the ssue of foldover cannot be resolved by exstng approaches. An mportant observaton s that the prevous approaches cannot enforce consstency under the composton of all local trangle deformatons, 67

80 (.e. the feld that s formed by the dsplacement of the vertces over a mesh should be smoothed). From a mathematcal pont of vew, the term consstency accounts for the propertes of vector felds,.e. ntegrablty or smoothness. For some large deformaton scenaros, Posson-type systems n the global step s not able to average out the nconsstency between the local deformatons. The nconsstency usually leads to numercal nstablty of Posson systems, such as foldovers. Ths paper ams to tackle ths challenge and further formulate a generalzed soluton to the foldoverfree 2D mesh deformaton. 5.2 PROPOSED METHOD To deform a contnuous surface s to determne a dsplacement feld d between the surfaces before and after deformaton. The deformaton transformaton φ s modelled as, φ(x) = X + d(x) or φ 1 ( X) = X d( X), wth X n doman Ω. The deformaton gradent refers to the dervatves of φ called the Jacoban matrx, φ. Topology preservaton n a dsplacement fled s to preserve the connectvty between the neghbourng structures,.e. the resultng mappng s bjectve. If foldover appears n the dsplacement, t ndcates the resultng mappng s not one-toone and s not nvertble. An ndcator of determnng whether there s foldover s the determnant J ( X ) of the Jacoban matrx φ over the whole doman Ω, whch can be computed by, d J ( X) = det( φ) = det( I + d( X)), (5.1) d 68

81 where I denotes an dentty matrx. The Jacoban matrx φ encodes the local stretchng, shearng and rotatng of the dsplacements d(x). If the determnant J d at some X s negatve, ths mples that the one-to-one mappng has been broken. To prevent foldovers from occurrng, we need to restrct φ wth the constrants of postve J d durng the deformaton OUTLINE OF OUR PROPOSED METHOD Regardng 2D dscrete settngs, let X and Y denote the vertces n the meshes before and after deformaton respectvely, and the deformaton transforms defned by Y = φ(x). For 2D mesh deformaton, due to pecewse affne transformatons, the deformaton of each trangle can be smply defned by the correspondng edges nstead of the vertces, Y = φ(x) = AX, (5.2) where, A s a 2 2 matrx, X = ( X X, X X ) and Y ( Y Y, Y Y ) =. Heren, A s namely the Jacoban matrx that s constant wthn a trangle. Followng Eq.5.1, t s requested to restrct the deformaton φ wth the pecewse postve Jacobans det(a) to avod foldovers. Fgure 3 further shows that even f the Jacoban of every trangle det(a) s postve, the dsplacement vectors between the neghbours may stll be nconsstent and can further result n foldovers. Thus, the encounterng dffcultes nclude, (1) how to add the constrant of postve Jacoban det(a) for every trangle; 69

82 (2) how to guarantee the desred accord or consstency over the dsplacement feld. Our basc dea s to restrct Jacobans for every trangle and further construct a consstent dsplacement feld by the feasble subspace method that wll be addressed later. Our proposed method also adopts the local/global scheme due to ts smple mplementaton. Essentally, ths scheme s an alternatng least squares teratve framework, and n partcular, t has been proved to yeld an optmal soluton n a least squares sense [Karn 29]. For clarty, our algorthm s outlned as follows. Foldover-free Constraned Deformaton Algorthm () Intalze: X = X ; Local Step: 1) Compute the affne for each trangle wth the specfed postonal constrants n ( 1) terms of the current X ; 2) Add the postve Jacoban constrant to the affne for each trangle respectvely; Global Step: 1) Construct a feasble subspace wth the dsplacement feld consstency; 2) Project the nconsstent dsplacement feld onto the feasble subspace; * 3) Reconstruct the deformed X from the resultng dsplacement feld; Convergence: If convergence s reached up to some tolerance, then end; otherwse, go to Local Step. The above presented algorthm s actually a generalzed framework. The local step can adopt varous Jacoban constrants n terms of dfferent applcatons. For example, Secton 3.2 wll ntroduce two methods of Jacoban constrants. In the global step, besdes the nterpolaton approaches based on Posson-type system that wll be addressed n Secton 3.3, one can adopt other nterpolaton technques for deformaton, e.g. radal bass functons. Heren, we hope to pont out that constructng a consstent dsplacement feld n advance s ndependent of the 7

83 followng nterpolaton methods and s a crucal step to obtan a stable soluton at ths stage. Moreover, n the followng sectons we wll also address the other ssue, that s, the generaton of super-resoluton mage patches, whch wll be appled to the development of an mage and vdeo magnfer POSITIVE JACOBIAN CONSTRAINTS Intally, the undeformed mesh s frst deformed wth all postonal constrants by usng the approach of [Igarash et al. 25], harmonc map [Eck et al. 1995] or others. There mght be foldovers on the resultng mesh. Then, we can determne the affne transform for every trangle accordngly. In the local/global scheme, the target mesh s updated teratvely untl convergence s reached. In the local step, we focus on the affne transform of each trangle. Usually, Jacoban constrants are carred out at that tme. In terms of Eq.5.2, the Jacoban matrx φ of the trangle T s denoted by matrx A,.e. A = φ. In order to add the Jacoban constrant, we can smply prescrbe the thresholds ( ε mn, ε max ) to bound the Jacoban det(a), and then modfy matrx A by solvng the followng nequalty, εmn Jα εmax, (5.3) Jα = det( I + ( A I) α) where α 1. It can be noted that the Jacoban of the trangle T can be bounded n the desred nterval by adjustng α. However, ths processng usually results n some unexpected dstorton. 71

84 Moreover, for the purpose of the As-Rgd-As-Possble deformaton, matrx A can be further decomposed nto a rotaton component and a scale-shear one by polar decomposton as suggested n [Alexa et al. 2], A T = RVDV, (5.4) where D s a dagonal matrx contanng the sngular values of A. These two sngular values ndcate the amount of the trangle stretchness. Applyng logarthm to the rotaton matrx R yelds the rotaton angle,.e. log(r). One thus can prescrbe the thresholds to the rotaton angle and sngular values for the As-Rgd-As-Possble deformaton. Nevertheless, the modfed Jacoban matrces A must guarantee the Jacobans det( A ) postve everywhere CONSISTENCY OF DISPLACEMENT FIELDS Followng the local step, we obtan a set of modfed deformaton Jacoban matrces { A }, =1 m, correspondng to the ndvdual trangles of the mesh, where m denotes the number of trangles. The global step wll make the deformaton Jacoban matrces consstent and reconstruct the deformed mesh by Posson solver. Takng Cartesan system nto account, t can be noted that for trangle T, the two row vectors of A contrbute the dsplacements to the Cartesan components of the affne ϕ along x and y axes by, 72

85 ( ) ( (2,1), (2,2)) f( X) A(1,1), A(1, 2) X ϕ( X ) = =. (5.5) g( X) A A X To deform the mesh, the two Cartesan components f and g of deformaton φ need to be reconstructed separately from ther ndvdual correspondng gradent felds,.e. ( A (1,1), A (1, 2) ) T and ( A A ) (2,1), (2,2) T. For smplcty, consder the deformaton of a scalar feld f(x) assocated wth a T T gradent feld that s denoted as vx ( ) = ( X, X ) ( pq, ) n x y 2 R. The gven gradent felds v(x) are usually nconsstent because of the nternal postonal constrants and modfcatons of the local step. Smlar to the tradtonal PCA technque, our basc dea s to construct a subspace of gradent felds wth gradent vector consstency, and then project the orgnal feld v(x) onto t to form a feasble gradent feld, whch wll be used to reconstruct the desred deformed scalar felds f * ( X ). The challenge s how to buld up a feasble subspace of gradents from the gven v(x). Snce the unknown f * ( X ) s a scalar feld, the consstent gradent feld v(x) should be a potental vector feld and satsfy the ntegrablty condton,.e. y x curl( v) = p q =, over the doman [Arfken and Weber 1995]. Due to the abovementoned constrants, the gven v(x) usually cannot satsfy the curl-free condton. To get round ths ssue, we subtract the curl components from the gven v(x). The problem can then be further formulated that for a gven vector feld v(x), our goal s to fnd a vector feld v ( X ) whose curl captures the curl part of the orgnal v(x). c Ths happens to be the soluton of the followng mnmzaton problem, 2 vc v dv. (5.6) Ω mn ( ) vc 73

86 In a dscrete settng, solvng the above mnmzaton problem requests to overcome the rregular connectvty of meshes or polyhedral. Recent work on overcomng ths dffculty s to approxmate the smooth felds wth dscrete felds and redefne the dscrete dfferental operators for trangulated 2D manfolds [Desbrun et al. 22, Tong et al. 23]. A dscrete vector feld on a mesh s defned as a pecewse-lnear one wth a constant vector wthn each trangle. The vector feld v c can be expressed n an affne fashon wthn a neghbourhood of vertex X,.e. v = φ ( X) v, wth φ beng the pecewse-lnear bass functon valued 1 at vertex c c X and at all other vertces and v c beng the value of v c at vertex X. For a gven dscrete vector feld v(x) on a mesh, ts curl at X s defned as, ( curl v)( X ) = ( φ v ) T, (5.7) k k k Tk N() where, N() s the set of trangles sharng the vertex wthn 1-rng range, T k denotes the trangle k and T k s area of T k, φk s the gradent vector of φ wthn T k, and v k denotes the vector of v(x) nsde the T k. Accordng to the defnton of Eq.5.7, for each vertex X, one can yeld the followng sparse and lnear system for the unknown curl part vector feld v c by a smple sum over the 1-rng neghbourhood of the underlyng mesh [Tong et al. 23], φ ( v ) T = φ v T, (5.8) k c k k k k k Tk N() Tk N() where, = 1 n, wth n beng the vertex number. Subtractng the resultng vector felds vc( X ) from the orgnal v(x) yelds a consstent gradent feld,.e. 74

87 v( X) = vx ( ) v( X), that can further be appled to the reconstructon of the g c unknown f * ( X ). Remark As far as Eq.5.7 and 5.8 s concerned, we prefer to subtract the curl feld v ( X ) from the non-ntegrable fled v(x) rather than to estmate the dvergence feld v ( X ) drectly. Ths s due to the fact that the curl part of a non-ntegrable feld contans nformaton about the dvergence of the underlyng ntegrable feld, that s, the dvergence part contans ncomplete nformaton about the dvergence of the underlyng ntegrable feld. We further explan t by the followng example. Suppose due to nose or unexpected dsturbance, a non-ntegrable feld v(x) n 2 R s equal to the true gradent feld v ( X ) rotated by an angle θ. The dvergence and curl of the feld v(x) are computed respectvely as, ( ) θ ( ) ( v ) cosθ ( v ) θ Dv( v) = v cos + v snθ =. Curl( v) = sn c g It can be noted that the dvergence part of the feld v(x) s only the projecton of the v(x) onto the dvergence dmenson. Partal nformaton of the feld v ( X ) hdes n the curl part of the feld v(x). Thus, we hope to keep the magntude of the estmated dvergence feld vg ( X ) as that of the v ( ) X, such that, 2 2 Dv( vg ) Dv( v) Curl( v) v = + =. To ths end, we need to frstly estmate the curl feld vc ( X ) from the gven nonntegrable feld v(x), and then subtract t from the v(x). Because, 75

88 Curl( v) = Curl( v) + Curl( vc), = Curl( v ) c where, Curl( v ) =, one can estmate the curl feld vc( X ) accordngly. Then v ( ) ( ) ( ) X = vx vc X. Ths can be vewed as an explanaton of Eq.5.8. So far, we have been able to apply the scheme of Eq.5.7 and 5.8 respectvely to the Cartesan components f and g of Eq.5.5 for ther ndvdual consstent gradent felds that form feasble Jacoban matrces A. However, t can be noted that there s a large dfference between the resultng { } A and the orgnal { A }. It s natural to seek a set of feasble Jacoban matrces as close to the orgnal { A } as possble. Here, we need to approxmate the orgnal { A } based on the resultng { } approxmate the two gradent felds of Ths can be fulflled as follows. A,.e., to A, ( A(1,1), A (1, 2) ) T and ( ) A(2,1), A (2,2) T ). Consder an ntegrable vector feld v(x) and ts counterpart v ( X ) wth the vector consstency. We seek a feasble soluton as close to the orgnal v(x) as possble. The gven v ( X ) s utlzed to construct a feasble subspace of gradent g felds wth the vector consstency. The optmal approxmaton s then obtaned by the projecton of v(x) onto the feasble subspace. g In general, each sample of vg ( X ) can be extended as a vector n a 2ndmensonal Hlbert space, where n denotes the sample number. Such n vectors can further span a subspace whch has dmensonalty not greater than n. The orthonormal bass vectors { u } can be constructed by applyng the Gram-Schmdt orthonormalzaton approach to the sample set of v ( X ). Suppose that the gven g 76

89 feld v(x) belongs to the resultng subspace spanned by the resultng orthogonal bass { u }. We can express t as, k = 1 T ( ) v= uvu. T If the vector feld v(x) does not belong to ths subspace, they can be approxmated by, k = 1 T ( ) T vˆ = uvu. (5.9) The geometry meanng says that the projecton vx ˆ( ) of v(x) onto the subspace spanned by the bass vectors { u } s the closest to v(x) n ths subspace. Indeed, Eq.5.9 gves out a mnmum norm soluton n the specfed gradent subspace. Applyng Eqs to the Cartesan components f and g yelds the desred deformaton φ. Moreover, to reconstruct the vertces * X of the deformed mesh, we defne a quadratc error functonal assocated wth each edge of a trangle by, ( ( ), ( )) T 2 * j j j, (, j) {(1,2),(2,3),(1,3)} E = X f X g X where the edge of XX j denotes as X j = X j X. Instead of a sngle trangle, we solve ths quadratc functonal on a trangulaton T { T(,, )} = by mnmzng, mn { X X } 1,..., m E, (5.1) T where m denotes the vertex number. The soluton to ths mnmzaton problem can be obtaned by the lnear system of normal equatons of Eq.5.1. Note that we do not add any postonal constrants nto ths lnear system at ths stage n order to mantan the consstency of dsplacement feld. The specfed constrants wll be approxmated gradually n an teratve manner. Although we have not been able to provde a strct 77

90 proof of reachng the desred postonal constrants exactly by our proposed approach, our experment results have been very postve. In the followng we wll apply the above 2D mesh deformaton technque to the applcaton of an mage and vdeo magnfer for amplfyng any specfed ROIs wthn a vew feld. The amplfed ROI patch s usually expected to preserve ts orgnal aspects wth lttle dstorton. Moreover, the outsde of the ROIs should be kept as close to sometrc as possble. Ths can be fulflled because of the affne decomposton utlzed n the local step. 5.3 APPLICATION: IMAGE/VIDEO MAGNIFIER As an mage and vdeo magnfer, n addton to zoomng n a specfed ROI for hgh resoluton, we set all vertces of the ROI patch boundary as postonal constrants to preserve the relatonshp between the ROI and the surroundng mage. We frst generate a super-resoluton mage patch correspondng to the specfed ROI and then paste t onto the amplfed ROI wthn the low resoluton mage/vdeo. Ths produces the amplfed ROI patch wthout any dstorton. The XSW s SR algorthm [Xong et al. 21] s used to generate the desred SR mage patches. The XSW s SR algorthm ncludes two steps. The frst one s the PDE-based ansotropc regularzaton that needs balance artefact removal and prmtve preservaton. The other one s the learnng-based par matchng. Unlke other SR approaches, the tranng dataset conssts of a set of co-occurrng patch pars extracted 78

91 from example mages at two dfferent resoluton levels. The fnal SR effect manly reles upon the par matchng accuracy. Dfferent from the sngle-mage scenaro, the nterframe nteracton and spatotemporal coherency are taken nto account n vdeo applcaton. Fortunately, the PDE regularzaton strength used n XSW s SR algorthm can well adapt to the varable degradaton between the successve frames caused by fast moton or scene swtch. Moreover, prmtves n the nterpolated frames also need to be enhanced. Ths can be acheved by smply thresholdng the par matchng tolerance n the XSW s SR algorthm, effectvely reducng the compresson artefacts. Ths s well sutable for varous compressed vdeos. Before embeddng a super-resoluton patch nto a low resoluton mage/vdeo, the specfed ROI wthn the orgnal vew feld s frstly amplfed by our foldoverfree constraned deformaton technque presented n secton 3.1. The orgnal low resoluton mage s then mapped onto the deformed vew feld. Embeddng the SR patch nto the specfed ROI wthn the deformed vew feld s mplemented by the Posson clonng method [Pérez et al. 23]. Although embeddng a super-resoluton patch nto a low resoluton mage/vdeo always results n dstnct transton of boundares, the Posson system can effectvely remove them. The soluton s obtaned by solvng the followng Laplace equaton wth boundary condtons, * ( ) f =, over Ω, wth f = f g, (5.11) Ω Ω where, g and * f denote the super-resoluton and low resoluton mage ntenstes respectvely. f wthn Ω s a membrane nterpolaton of the dfference * ( f g) between the super-resoluton and low resoluton mages along the boundary Ω. 79

92 5.4 EXPERIMENTS AND ANALYSIS We frst perform our proposed deformaton algorthm on the example mm n Fg.5-2 and the rectangular trangle mesh n Fg.5-3. Eq.5.4 s employed n the local step. The ntermedate and fnal teratve results are shown n Fg.5-4. a. b. Fgure 5-4. The teratve results of examples n Fg.5-2 and Fg.5-3 by our approach. Red stars ndcate the postonal constrants. a. b. c. d. Fgure 5-5. Illustraton of texture mappng. a) orgnal 3D model; b) matchng 2D parameterzaton of 3D model wth texture mage; c,d) mappng fsh mage to 3D model. a. b. c. e. f. g. 8

93 Fgure 5-6. Comparson of smoothness. a and b) chessboard mappng results before and after smoothng usng [Lee et al. 28]; c) chessboard mappng result usng our method; e and f) texture mappng results before and after smoothng usng [Kraevoy et al. 23]; g) texture result usng our method. Fgures of 6e and 6f are from [Kraevoy et al. 23]. Comparng Fg.5-4a wth Fg.5-2a and 5-2b, t can be noted Eq.5.4 n the local step can effectvely solve the elbow-collapse problem shown n Fg.5-2a and 5-2b. Moreover, the ntermedate nterpolated shapes show lttle dstorton. Our approach does not need any post-processng for reducng dstorton, such as the scale-adjust presented n [Igarash et al. 25]. Thus, t s useful n producng 2D anmaton. Fgure 5-4b shows that the amplfed regon can keep ts orgnal aspect wthout dstorton durng mesh deformaton. Our approach satsfes the specfed postonal constrants teratvely gven any tolerance. Image warpng or mesh deformaton s also the bass for texture mappng. Readers are referred to [Lee et al. 28] for a state of the art revew on texture mappng. One usually frst deforms the parameterzaton of the 3D model to match the gven texture mage wth some specfed postonal constrants, and then maps the texture mage to the 3D model accordng to the resultng (/deformed) reparameterzaton. Any foldovers arsen wll lead to unpleasng artefacts n the fnal textured model. In Fgure 5-5, we perform our foldover-free constraned deformaton algorthm on the parameterzaton of the snake head model to match the mage of a tropcal fsh. Postonal constrants are placed on the eye, nose and mouth. The mapped texture s well ft, whch can be seen from the smoothly spread strps on the snake head model. Exstng texture mappng methods employ a smoothng process for reducng dstorton after algnng the postonal constrants. Because of the feasble subspace wth the vector consstency, our method can yeld a smooth re-parameterzaton wthout any post-processng. To hghlght ths advantage, n Fg.5-6, we compare our 81

94 deformaton method wth two state-of-the-art methods publshed n [Lee et al. 28] and [Kraevoy et al. 23]. Fgure 5-6a, 6b and 6c show the chessboard texture mappng results by usng the algorthm of [Lee et al. 28] and our method. Fgure 5-6d, 6e and 6f show the results by [Kraevoy et al. 23] and ours. Among the three methods, ours produces the most smooth results. The experments are undertaken usng Matlab on an Intel Pentum 4 3.2GHz PC wth 1 Gbyte of RAM. The runnng tme of our method s 1.91 sec. In the mplementaton of our mage and vdeo magnfer, we paste the superresoluton mage patches onto the amplfed ROIs wthn a deformed vew feld to avod any potental dstorton. The ROI s a square regon. We performed our proposed magnfer presented n secton 3.4 on a vdeo as shown n Fg.5-7. The upscalng factor s 4 for SR patches. The SR algorthm s able to upscale an mage patch szed under 5 5 around 1 second on average. f14 f78 f97 12 f 155 f 16 f 82

95 18 f 27 f 227 f Fgure 5-7. Illustraton of vdeo magnfer, total 9 frames shown here. The vdeo resoluton s of 48x27. For comparson, we put the orgnal frames (upper) and ther amplfed versons (lower) together. The amplfed ROIs are embedded by the super-resoluton patches. 5.5 CHAPTER SUMMARIZATION Ths chapter addresses the 2D shape deformaton wth the postonal constrants and rgd constrant nto consderaton. Unlke the prevous chapters, the rgd constrant s taken nto account besde the usual postonal constrants here. Ths s ndeed to seek a tradeoff of the postonal constrants and mnmum dstorton n terms of varous applcatons. To hghlght ths ssue, we further develop an mage/vdeo magnfer that s requested to both satsfy the postonal constrants and cause as-rgd-as-possble deformaton. The next chapter wll focuses on the rgd constrant rather than the postonal constrants based on the mesh edtng applcatons. 83

96 CHAPTER 6 WELL-SHAPED AND WELL-SPACED MESH 6.1 PROBLEM FORMULATION Ths chapter addresses an applcaton based on the surface parameterzaton wth constrant of mnmal dstorton. Unlke the prevous chapters, ths chapter focuses on the rgd constrant nstead of the postonal constrants. The challenge s to generate a qualty mesh after deformaton so as to preserve the detals wth the mnmal dstorton. Inspred by the applcaton of the skn sldng smulaton, the goal s to develop a pre-processng procedure for low-dstorton parameterzaton. For clarty, we address ths applcaton n detals as below. In human skn smulaton, the ntally created model normally has a very hgh qualty mesh confguraton, n order to present delcate detals and beautful textures. However along wth the movement of the underlyng skeleton, large dstortons are 84

97 frequently observed on the skn surface. Ths greatly reduces the realstc appearance. From an anatomcal perspectve, the skn surface, as an elastc layer coverng multple anatomy structures, should preserve ts own tenson durng deformaton rather than over-stretches and compresson that only happen n some local areas (e.g. jont). In practce, due to large-scale deformaton and lack of precse control, the appearance of many sknny trangles s unavodable [Yang et al. 29, Zhang et al. 211]. Ths further results n many vsble artfacts or dstortons on the surfaces. For realstc purposes, skn sldng s presented to deal wth ths challengng ssue. Fgure 6-1 llustrates the ntutonal geometrc meanng of skn sldng. Roughly speakng, skn sldng s a varant of remeshng for removng sknny trangles, whch s resamplng the 3D coordnates of vertces whle preservng the mesh connecton relatonshp. Some efforts have been made to mplement ths real lfe effect nto character anmaton. For example, n the move Hellboy [Stnson and Thurot 24], the system employed a sprng network, where all the vertces of the skn mesh are ndvdually connected to the muscles or the bones usng sprng forces. Ths allows skn surface sldes above the underlyng structures nstead of fxng soldly to certan ponts on bones or muscles. However n ths system, snce the elastcty of the skn mesh tself was not consdered, some dstortons can stll be seen on the skn surface. One possble soluton s to replace the edges of the skn mesh wth sprngs as well. Ths would ncur a drastc ncrease n the computaton burden. Moreover, tunng so many sprng parameters would also be a bg challenge to anmators. To avod such a complcated control system, some nterpolaton technques were appled to smulate the elastc property of the skn surface nstead of the sprng network. The mesh before deformaton s regarded as a reasonable skn mesh, whle after deformaton t s always largely dstorted. Hence, skn sldng s appled to the mesh 85

98 after deformaton for removng dstorton. To mprove the realty, t s further proposed to, Keepng the geometry and topology attrbutes of skn mesh as close as possble to the mesh before deformaton, whch s assumed to have the desred features of the skn surface. The geometry attrbutes of a mesh refer to the relatve postons between vertces, whle the topology attrbutes refer to the connecton relatonshp of vertces. It s desred to transfer the attrbutes of skn mesh onto the mesh after deformaton; Increasng smoothness. Intutvely, t s always desred to generate a smooth skn surface ; Keepng the skn mesh as close as possble to the shape after deformaton, whch s assumed to be the desred shape created by anmators. Usually, anmators pay attenton to the global shape of the mesh after deformaton rather than the local sknny trangles, snce the shape results from underlyng anatomcal structures. It s expected that the skn surface at least overlays multple layers of the underlyng anatomcal structures wthout any unexpected bulge or shrnkage. a. b. c. Fgure 6-1. Schematc dagram of the effect of skn slde over an underlyng structure. (a) The three blue ponts denote ponts on the skn surface over the underlyng anatomcal structure. (b) Incorrect sldng, the blue ponts shft along wth the underlyng structure, and the skn s stretched on one sde and compressed on the other sde. (c) Correct sldng, the blue ponts reman statc thus makng the skn appear to slde over the nternal structure. (from [Yang et al. 29]) 86

99 To ths end, [Yang et al. 29] appled a lnear nterpolaton method to skn sldng smulaton based on 2D parameterzatons. Because of lnear nterpolaton, the computaton s very smple. However, the defcences are dstnct. The man challenge s that there s no guarantee of preservng the shape and feature of the orgnal surface. There s a tradeoff between the computatonal complexty and realsm on ths ssue. It s nevtable that lnear nterpolaton ncurs a bg loss of realsm. These ssues wll be further addressed later. In pursut of a realstc appearance, we focus on two aspects of skn sldng n terms of the above-mentoned goals, that s, parttonng the meshes and preservng the detals of the mesh before deformaton. Ths s due to the followng observaton. Skn sldng s always performed on a set of dstorted patches nstead of the whole mesh doman. Hence, removng dstorton from a skn mesh mplctly demands two procedures, one s to partton meshes nto a set of patches (e.g. dstorted and nondstorted patches) for further resamplng, and the other s to preserve the orgnal features or detals of the skn mesh. We further reformulate the skn sldng mplementaton by ntroducng the graph Laplacan framework [Sorkne 26]. The man contrbutons nclude, parttonng a whole mesh nto a set of patches, transferrng and preservng features and shape wthout any unexpected bulge and shrnkage, whch wll be addressed later. Moreover, our proposed method can be ntegrated nto the developed anmaton ppelne [Yang et al. 29] wthout any nterference wth the tradtonal methods of sknnng, gvng the anmator the freedom to control and desgn actvely durng the sknnng phase. 87

100 6.2 METHOD OVERVIEW We frst recap the skn sldng method presented n [Yang et al. 29] brefly, and then pont out some exstng challenges. After that, our proposed algorthm wll be outlned. In [Yang et al. 29], skn sldng method s usually appled to a specfed regon of nterest (ROI) nstead of the whole mesh doman. The nput s a par of patches M and M d before and after deformaton. The correspondence between them s establshed by embeddng M and M d nto a common 2D doman. In terms of the resultng embeddng maps U and U d, one can fnd out the facets of U d where the vertces of U fall n, that s, the correspondence between the facets of U d and vertces of U. Moreover, the vertces of M d are updated by the vertces nterpolated n the facets of M d accordngly. The resultng patch s fnally embedded back nto the deformed mesh doman. It s worth notng that these two patches M and M d before and after deformaton are matched by overlappng the 2D embeddngs U and U d. Intutvely, ths s to buld up a samplng mesh over the patch M d. As a result, the resamplng s fulflled by a lnear nterpolaton accordngly. A smple assumpton arses here, that s, the vertces of M d movng to ther updated locatons can be mrrored n the embeddngs U d and U, just lke the effect of the vertces movng across the surface. Ths ndeed rsks a bg dstorton. All of the exstng 2D embeddng approaches nevtably ntroduce large dstorton, such as the cotangent weghted Laplacan coordnates [Desbrun et al. 1999], bar-net embeddng [Yang et al. 29] and Mean Value 88

101 Coordnates [Floater 1997]. The resultng samplng mesh based on 2D embeddngs mght lead to the worst case that sparse samplng s performed on a hgh-frequency regon whle dense samplng on a low-frequency regon. To deal wth ths challenge, the skn sldng method proposed n [Yang et al. 29] was performed on small patches that were manually cropped n advance. Ths s because a small patch s easly flattened wth lttle dstorton. Moreover, skn deformaton always appears at some local regons rather than the whole surface. Desrably, skn sldng methods should be performed on the sknnng regons nstead of the whole surface. The challengng ssues rased here are how to cut the surface nto a set of patches and how to remove the vsble seams on the reconstructed surface. Our frst contrbuton n ths chapter s to present an approach for automatcally detectng and segmentng sknnng regons of a skn surface. patch Another pont worth notng s that nterpolaton takes place on the facets of the M d rather than others. Ths guarantees that the nterpolated mesh comples wth the deformed patch M d. However, the defcency s also clear, that s, there s no guarantee of transferrng the detals of M to the target skn mesh. Moreover, because of lnear nterpolaton, the notceable drawback s that f two adjacent patches are resampled ndvdually and then embedded back to the orgnal deformed mesh doman, there are vsble seams across the patch boundares on the reconstructed surface. Addtonally, there s the vsble shrnkage of the skn mesh, snce the over-stretch of the local trangles msses out some promnent parts. Our second contrbuton s to overcome these challenges n ths chapter. Compared to the prevous approaches, our proposed algorthm reformulates the skn sldng mplementaton based on the graph Laplacan framework. The dstnct advantage s that ths framework covers two mportant applcatons, spectral 89

102 clusterng [Sh and Malk 2] and dfferental coordnates [Sorkne 26]. The former has been appled to machne learnng and computer vson felds. Recent work further demonstrates that t can be appled to mesh parameterzaton [Mullen et al. 28], segmentaton [Lu and Zhang 27] and compresson [Karn and Gotsman 2] as well. The latter can be vewed as a combnatoral analog of Laplace- Beltram operator defned on a manfold. It can effectvely approxmate the dfferental propertes of a smooth surface and further results n the desred detalpreservng manpulatons. Our skn sldng algorthm s outlned as follows. Graph Laplacan based skn sldng algorthm (1) Parttonng a gven par of meshes by Normalzed cut algorthm; (2) Smoothng specfed patch pars; (3) Resamplng by Laplacan coordnates; (4) Embeddng the resampled patches back to the orgnal mesh doman for the reconstructed surface. The proposed algorthm can provde a global, shape and detal-preservng soluton. In the followng sectons, we wll further address the mplementaton of each step. 6.3 PARTITION BY NORMALIZED CUTS To deform a skn surface, the followng can be appled over the whole mesh doman: stretchng, compresson and non-deformaton. In terms of the gven mesh par before and after deformaton, the goal of the surface partton step s to dentfy 9

103 these three types of Regons Of Interest (ROIs). It s always expected that the parttoned patch boundares can mantan more natural shapes wth genus-, such as quadrangle, trangle and crcle-lke boundares, n order to reduce dstorton n the followng step. Ths s because the followng step wll embed the cropped patches nto a common 2D parametrc doman for further resamplng. Our algorthm utlzes one of the spectral clusterng approaches,.e. normalzed cut algorthm [Sh and Malk 2], to partton the gven meshes. Although t has been appled n mesh segmentaton [Lu and Zhang 27], we further extend t to smooth the resultng cluster boundares by usng the mn-cut/max-flow algorthm [Boykov and Kolmogorov 24] here. The startng pont s the graph Laplacan matrx, whch s applcaton-dependent. In the partton step, we construct the graph Laplacan as follows. Let M and M d be the meshes before and after deformaton wth the defnton as M = {V,E,F},.e. vertex set denotes V = { v = 1,..., n}, edge set denotes E = { e = ( v, v ) = 1,..., m} and facets set denotes 1 2 F = { f = ( v, v, v ) = 1,..., h} Our goal s to partton the mesh nto a set of patches n terms of the three categores, stretch, compresson and non-deformaton. Consder a dual graph G={F,E} wth the facets as nodes. For each par of the correspondng trangles f, f from M and () ( d ) M d, we can separately add the 4 th vertex to form a par of correspondng tetrahedrons as used n [Sumner and Popovć 24] and then compute the affne transformaton A between them. Further consder the followng dentty, det( A ) = Vol Vol, d 91

104 where Vol denotes the volume of the f. The volume rato has the three possble values,.e. greater than 1, equal to 1 and less than 1, correspondng to the three deformatons of the f respectvely,.e. stretch, non-deformaton and compresson. We employ the volume rato as the attrbute of the f here. Moreover, for a par of the adjacent f and f j, we defne the dstance between them as dst( f, f ) = det( A ) det( A ). The edge connectng the f and f j n the dual graph j j G s further weghted as, w j dst( f, f j) = exp 2 σ 2, (6.1) otherwse w j = for the nonadjacent facets. The kernel wdth σ s the standard devaton of parwse dstance between facets. It does not obvously nfluence the partton results, as long as t s not too small. In terms of such weghted adjacency matrx, we can further construct the graph Laplacan of G as, L = D W, (6.2) where D s a dagonal matrx wth the degrees, d n = w on the dagonal. = 1 j Moreover, we apply the normalzed cuts to the graph Laplacan of Eq.6.2 for partton purposes. Heren, the resultng graph Laplacan L s very sparse, large and symmetrc. To explot these propertes of our egnsystem, Lanczos algorthm [Golub and Van 1996] s thus employed. Note that each egenvector x may take on h real values, and the elements of x ndvdually correspond to the facets of the mesh. Consequently, h facets can be 92

105 mapped nto the subspace (also called egenspace) spanned by the t egenvectors assocated wth the frst t smallest egnevalues. Ths mappng s regarded as a t-d embeddng of the vertex set. Accordngly, the normalzed cut algorthm teratvely approxmates the desred soluton by thresholdng the resultng t-d embeddng. Although the weghted adjacency matrx W s employed n Eq.6.2, the t egenvectors can smoothly dstrbute over the egenspace rather than the dstnct gaps nbetween them. At each teraton, we have to choose a splttng pont to bpartton the gven facet set nto two parts n our mplementaton. To search for such splttng pont, the normalzed cut employs the followng metrc, k Ncut( P1,..., Pk ) = cut( P, Q) = w P, j Q assoc( P) = d P = 1 j cut( P, P) assoc( P), (6.3) where P s the complement of the set P and k denotes the number of means (Refer to [Sh and Malk 2] for detals). The termnatng crteron here s defned by measurng the cluster dvergence n the egenspace. To ths end, we frst construct the covarance matrces for the ndvdual clusters n the current egenspace, and then carry out the SVD decomposton on them to obtan ther ndvdual ratos between the mnmum and maxmum sngular values. The greater the cluster dvergence, the smaller the rato wll be. In our experments, we found that smple thresholdng on the rato can effectvely allevate the over-segmentaton. We set that threshold value at.25 n our experments. In addton, post-processng s stll needed to deal wth the resultng fragments. However, applyng the Normalzed cuts algorthm to the graph Laplacan L yelds a coarse partton, snce the cluster boundares tend to be jagged. These rregular 93

106 boundares usually ncur large dstorton n the subsequent 2D parameterzaton procedure. Further refnement s thus necessary to dentfy accurate and sound boundares. To ths end, we employ the mn-cut/max-flow algorthm [Boykov and Kolmogorov 24] here, that s, to fnd a cut that has the mnmum cost among all cuts based on a graph. Snce our goal s to refne the boundares of the resultng clusters, we can defne a narrow band envelopng the boundares and perform the mn-cut/max-flow algorthm on t nstead of the whole mesh doman. In our mplementaton, the neghborhood of the boundares typcally covers the 4 rngs of the vertces on the boundares. We thus construct the desred dual graph n terms of the selected narrow band. In the context of the graph cut based mnmzaton technques, the mn-cut/max-flow algorthm essentally solves the shortest path problem on a weghted graph. The edge weghts need to nvolve the cue of the edge length. Accordngly, the edge weght between the facets f and f j s re-defned as, dst( f, f j) w j = ej exp 2 σ 2, (6.4) where e j denotes the edge shared by the facets f and f j n the mesh M. As a result, ths refnement makes the boundares smoother. One can compare the effects of the partton procedure before and after refnement processng n the followng experment. Our partton procedure s summarzed as follows, Partton Procedure (1) Gven a par of meshes, set up a weghted adjacency matrx W and degree dagonal matrx D; (2) Solve the generalzed egensystem Lx = λdx for egenvectors x wth the smallest egenvalues λ; 94

107 (3) Use the t egenvectors wth the frst t smallest egenvalues to bpartton the meshes, e.g. k-means clusterng algorthm; (4) Recursvely re-partton the segmented parts f necessary; (5) Employ mn-cut/max-flow algorthm to refne the partton boundares. a. b. c. Fgure 6-2. The partton result of a hand skn surface. (a) shows the partton results of the fngers before and after deformaton wthout refnement; (b) plots the egenvectors correspondng the 2 nd smallest egenvalue to the 7 th smallest one; (c) shows the refnement results. Fgure 6-2 shows the partton results of a hand model that contans 7997 vertces and trangular facets. In the orgnal normalzed cut algorthm, only the egenvector wth the 2 nd smallest egenvalue s used, whereas the next few smallest egenvectors ndeed contan useful partton cues as well. Ths can be observed n the plots of the 2 nd, 3 rd and 4 th smallest egenvectors n Fg.6-2c. The other egenvectors (.e. 5 th, 6 th and 7 th smallest egenvectors n Fg.6-2c) almost have the same dstrbuton. Obvously, t s enough to span the egenspace by usng the 2 nd, 3 rd and 4 th egenvectors here. Addtonally, t can be noted that the sknnng regons (.e. 95

108 stretch and compresson ROIs) appear around jonts whle others belong to nondeformaton regons. The stretch (compresson) ROIs mantan quadrangle-lke boundares n Fg.6-2c. In skn surface deformaton, we pay attenton to stretch ROIs rather than others. The compresson ROIs usually come along wth self-ntersecton. From the perspectve of an anmator, the outward appearance of a deformed model receves more attenton than nsde. 6.4 SMOOTHING AND RESAMPLING Our basc dea conssts of: (1) frstly generatng the base of the orgnal mesh, (2) then extractng the detals accordngly, (3) further transferrng them to a specfed surface. For clarty, we frst address the Laplacan Coordnates brefly and then present our smoothng and resamplng steps separately DIFFERENTIAL SURFACE REPRESENTATION The graph Laplacan has long been vewed as a combnatoral verson of the Laplace-Beltram operator and appled to geometry processng. Unlke the spectral clusterng, t works drectly on a mesh rather than a dual graph of the mesh. For completeness, we frst address how to represent a surface by changng the weghts of the graph Laplacan matrx Eq.6.2,.e. Laplacan Coordnates, and then return to the smoothng and resamplng procedures. Consder a trangle mesh M = {V,E,F}. The 96

109 graph Laplacan of Eq.6.2 s vewed as a dscretzaton of the contnuous Laplace- Beltram operator as follows, 1 L I D W =, (6.5) where I denotes an dentty matrx. Usually, we smply assume that the mesh M s a pecewse-lnear approxmaton of a smooth surface. The Laplacan operator at vertex v s wrtten as, ( ) Lv ( ) = w v v, (6.6) j j j N() where N() denotes the 1-rng neghborhood of the vertex v, and w j s the weght of edge e j subject to j N() w j = 1. The basc geometrc meanng of Laplacan operator s that the relatve locaton of vertces s encoded n δ-coordnates (or called Laplacan coordnates) as follows, δ = Lv ( ). (6.7) Intutvely, the detals of the mesh M are preserved n δ-coordnates. To perform surface modelng by usng Laplacan coordnates, one can fx the absolute poston of several vertces as Drchlet boundary condtons and further solve a sparse lnear system, Lv = δ, derved from Eqs.6.5 and 6.7 for restorng a global soluton v of Cartesan coordnates of the vertces SMOOTHING STEP 97

110 The cropped patches need to be smoothed. The smoothed verson of a patch s vewed as ts base so that the detals of the patch can be represented by the offsets. An effcent smoothng approach s the curvature flow smoothng algorthm proposed n [Desbrun et al. 1999]. The dstnct advantage s speed. But there s also an obvous defcency that the curvature flow may shrnk the volume of a mesh substantally. Moreover, for rregular connectvty meshes, there s a lack of suffcent control over global behavour durng smoothng, such as large naccuraces for rregular meshes. Ths mples that the sharp features wll be smoothed out frstly by the curvature flow. In our sknnng applcatons, large deformaton always yelds many sknny trangles, and hence the smoothng step s expected to generate a well spaced and well shaped mesh. For all the above-mentoned purposes we would lke, we present a smoothng method based on the tangent plan of a surface n ths chapter. From a dfferental geometry perspectve, Laplacan coordnates of Eq.6.5 can be vewed as the dscrete verson of the mean curvature normal of a surface by usng cotangent weghts nstead of unform ones [Meyer et al. 23]. The curvature flow drves the surface shrnk or expanson along the normal drecton of the surface. In essence, the curvature flow reduces curvatures and ends up removng them everywhere. Inspred by the ansotropc dffuson technques n mage processng, our basc dea s to smooth the surface along some tangent drecton of the surface rather than the normal ones here. Ths can preserve the underlyng structures of the surface as much as possble durng smoothng. Our method explots the tangent planes and restrcts the updated vertces wthn the tangent planes. 98

111 Consder the vertex v of a gven mesh M. The normal of the tangent plane at v T, j N() can be determned by covarance matrx as follows. Let C = ( vj v)( vj v) where N() denotes the 1-rng neghborhood of v. Takng SVD on the real and symmetrc matrx C yelds three orthogonal bases of the local frame at v. The egenvector assocated wth the mnmum sngular value s namely the normal the tangent plane at v. N of Once the normal N at v avalable, the tangent plane s determned accordngly. Vertex v wll be updated wthn t. The neghbours n N() are projected onto the tangent plane. Our goal here s to determne a proper locaton for updatng v on the tangent plane n terms of these projectons. It s clear that v wll move wthn the tangent plane nstead of the normal drecton of the mesh. Choosng a proper locaton for updatng v should tend to allevate the ssue of sknny trangles and mprove the trangle qualty as much as possble. To ths end, we weght the projectons v ˆ j of the neghbors vj, j N () of the v by the areas of the adjacent trangles nstead of the centrod of the projectons v ˆ j. Ths s llustrated n Fg.6-3a. For each projecton v ˆ j, there exst two adjacent trangles ( v垐 j 1vv j ) and 垐 ( vv j j+ 1v) sharng the edge ˆ j vv on the tangent plane. We weght the v ˆ j by, w j = Aj 1 + Aj 2 A, (6.8) j j N() 99

112 where A j denotes the area of the jth adjacent trangle. As a result, the v s updated wthn the tangent plane by, ( ). (6.9) v垐 = v + λ wj vj v j N() where λ denotes the teratve step length. Moreover, for the entre mesh M, Eq.6.9 yelds a hghly nonlnear system. To deal wth such nonlnear optmzaton problem, we utlze a Gauss-Sedel style teraton to smooth the mesh wth a seres of local optmzaton steps nstead of solvng ths problem globally. The proposed smoothng procedure s summarzed as follows. Tangent Plane Based Smoothng Method Input: the current mesh M; Repeat (1) Choose a vertex v and compute ts normal vector neghborhood; (2) Project all the neghbors onto the tangent plane of v ; N by the 1-rng (3) Compute the weghts and update the v by Eq.6.9 wthn the tangent plane; Untl numercal convergence. Fgure 6-3c to 3d show the smoothng results acheved usng our tangent plane based smoothng method and the curvature flow algorthm respectvely. It s obvous that the proposed smoothng method can effectvely preserve the sharp features and areas wth large curvatures compared to the curvature flow algorthm. Moreover, t can also be observed that our method can allevate the ssue of sknny trangles and effectvely mprove the mesh qualty. However, the proposed tangent plane based smoothng method essentally mnmzes the curvature of a surface n a subspace of 1

113 the tangent space. Much teraton may stll degrade the order of accuracy. In addton, the stablty crteron requres the teratve step length of Eq.6.9 λ<1. If ths crteron s not satsfed, rpples appear and further result n serous oscllatons of growng magntude over the entre surface. In our mplementaton, let λ<.25. Before proceedng further, we assume that the partton step has cut off a par of ROI patches respectvely from the skn meshes before and after deformaton, and the smoothng step has further produced the smoothed versons of these patches. We assume that these two patches share the same connectvty. Hence the correspondence s readly gven. To defne the mappng between these two patches, we apply the 2D embeddng to them as descrbed n [Yang et al. 29]. Snce the cropped patches usually mantan the quadrangle-lke boundares from the partton procedure, we dentfy correspondng boundary vertces and fx them at the same boundares of a unt square doman n our mplementaton. However, we also have to pont out a specal case,.e. surface twst. Although t seldom appears n skn deformatons, we stll encounter ths challenge, for example, turnng the head. It can be observed that the stretched and compressed trangles mx together n the regon of neck. Thus, such sknnng regon s vewed as an entre ROI patch nstead of as stretch and compresson ROI patches ndvdually. Addtonally, to speed up the correspondence between the 2D embeddngs, we apply the approxmate nearest neghbour (ANN) search [Merkwrth et al. 29] to the 2D embeddngs here. Remark We hope to pont out that the dstnct advantages of the partton and smoothng steps, that s, smooth partton boundares and well-shaped and well-sapced meshes. In the followng 2D parameterzaton, smooth boundares can reduce dstorton 11

114 compared to the jagged boundares. Well-shaped and well-sapced meshes can further yeld a plausble correspondence by overlappng the two parameterzatons. j-1 î a. j j+1 b. c. d. Fgure 6-3. Comparson of mesh smoothng. a) The dashed lnes show the projectons of 1-rng neghbourhood on the tangent plane at the th vertex; b) shows the orgnal bent tube model; c) shows the smoothng result produced by our proposed smoothng method; d) shows the smoothng result produced by the curvature flow algorthm RESAMPLING STEP Our goal s to resample the coordnates of the vertces based on the abovementoned mappng between the cropped ROI patches. To ths end, Eq.6.7 s employed for extractng and transferrng the detals of the skn surface. However, the challengng ssue s that δ-coordnates are usually not nvarant to rotaton and scalng. Indeed, these dfferental coordnates are senstve to lnear transformatons [Sorkne 26]. Roughly speakng, Laplacan coordnates are encoded n a global coordnate system whle deformatons are always local and non-rgd. For example, f a surface were bent, preservng the δ-coordnates n ther orgnal orentaton wth respect to the global coordnate system would lead to the dstorted orentaton of the local detals. Thus, δ-coordnates need to be properly reorented for deformaton purposes as shown n Fg.6-4a. Ths seems to be a typcal chcken and egg 12

115 problem. To reorent these δ-coordnates, we need to know the geometry of the unknown deformed model and vce versa. We attempt to solve ths challenge through two matched ROI patches. v δ b. c. a. Fgure 6-4. Illustraton of reorentng δ-coordnates. a) Durng deformaton, δ-coordnate s reorented wthn 1-rng neghborhood of v ; b) shows the orgnal M d ; c) shows the shrnkage of overlappng M d and M d. Reorentng δ-coordnates It s natural to apply an approprate local transformaton to each vertex for the rotaton-nvarant representaton of the detals smlar to [Sorkne et al. 24]. Nevertheless, the man dfference here s that n our skn sldng applcaton, the local transformaton for the ndvdual vertces can be obtaned explctly through the two matched patches rather than an mplct optmzaton procedure as used n [Sorkne et al. 24]. For clarty, we consder the rotaton and scalng of the local transformaton separately here. Let M and M d be the smoothed versons of the gven ROI patches M and respectvely that have been well matched n advance. The local detals of meshes are encoded nto the δ-coordnates by Eq.6.7,.e. δ = Lv ( ), = 1,..., n. In terms of the mappng between the smoothed versons M and M d, a vertex n M can be M d mapped to some arbtrary pont nsde a trangular facet n M d. Ths s ndeed a resamplng pont n M d that we can determne by usng the Barycentrc coordnates, 13

116 that s, computng the Barycnetrc coordnates n 2D embeddngs and applyng them to 3D surface accordngly. We can therefore set up a resamplng mesh over the entre M d and further replace M d wth ths resamplng mesh (that stll denotes t as M d thereafter). Moreover, for each par of correspondng vertces between M and M d, we can defne ther ndvdual affne transformatons n terms of the 1-rng neghborhood. Usually, the polar decomposton s employed here for shear and rotaton decomposton [Alexa et al. 2]. However, ths decomposton may yeld a reflecton matrx nstead of rotaton. To overcome ths drawback, we develop a rotaton-scale-shear decomposton as below,.e. to decompose the affne A of the th par of the correspondng vertces nto the shear matrces A = ShScR. Sh, scalng Sc and rotaton R Wthout loss of generalty, let A be a gven affne transformaton of sze 3-by-3. We wll decompose t nto an orthogonal matrx R and an upper trangle matrx U, T and hence denote the row vectors of R and A as R = ( r, r, r ) and A= ( a, a, a ) respectvely. In terms of the Gram-Schmdt orthonormalzaton, we have, T 1 r3 = a3 a 3 a2, r3 1 r2 = r3 + a2, (6.1) a 2 a 2 a1, r3 a1, r2 1 r1 = r3 r2 + a1 a 1 a 1 a 1 where, denotes the nner product, and the row vectors of A are orthonormalzed by, 14

117 a 3 = a3 a2 = a2 a2, r3 r3. a1 = a1 a1, r3 r3 a1, r2 r2 We can yeld the orthogonal matrx R accordngly. Moreover, we further construct the upper trangle matrx U as follows, a a, r a, r = 2 2, 3 U a a r a 3. (6.11) It can be observed that the dagonal tems are postve here. U s therefore postve defnte. We defne the scalng transform as a dagonal matrx,.e. Sc a 1 = a 2, (6.12) a 3 and the shear transform as an upper trangle matrx,.e. 1 a, r a, r = 1 2, 3 Sh a r 1 As a result, factorng A yelds, A = Sh Sc R. End of the proof.. (6.13) We refer to t as the rotaton-scale-shear decomposton. Compared to the polar decomposton, the proposed rotaton-scale-shear decomposton can explctly gve out the rotaton, scalng and shear components of a gven affne transformaton. Note that multplyng the shear wth the scalng yelds an upper trangle matrx that s postve defnte. For the resultng R, there s no guarantee that R s defntely a rotaton. Takng the Egn-Value Decomposton on R, t can be noted that the θ θ egenvalues are of e, e, + 1/ 1 respectvely. Obvously, when the egenvalues 15

118 nvolve -1, R s a reflecton matrx. Hence, we can utlze a trck to deal wth ths ssue here. If R s a reflecton matrx, let B 1 = 1 A 1 (assume that the 3 rd egenvalue s of -1 here). The resultng B s postve defnte. Applyng our proposed rotaton-scale-shear decomposton to B updates the shear Sc and rotaton R. Then, we can further yeld, Sh, scalng 1 A = 1 ShScR. (6.14) 1 Ths guarantees that the updated R s a rotaton matrx rather than others. The scalng matrx mght be negatve defnte. In general, the scalng Sc s of a dagonal matrx and the shear Sh satsfes det( Sh ) = 1. Thus, the shear Sh can be omtted here. Moreover, to compare the nfluence of rotaton and scalng, we frst consder the rotaton R here,.e. to apply the resultng R to the δ-coordnates. Applyng Eq.6.7 to the smoothed and unsmoothed versons of the patch M respectvely yelds ther ndvdual δ-coordnates, δ and δ. Moreover, the detals of M can be obtaned by ther dfference,.e. ξ = δ δ. To transfer the detals to the smoothed verson of the patch M d, we can update the δ-coordnates of M d by, δ = δ + Rξ, (6.15) ( d) ( d) where ( d ) δ s the δ-coordnates of the smoothed verson d M. Obvously, the ( ) reorented δ d can be computed explctly and ndvdually here. 16

119 We can further solve the resamplng vertces v n M d by mnmzng the followng error functon, n ( d) 2 c 2 δ j j = 1 j N( c), (6.16) EV ( ) = Lv ( ) + v v where V denotes the set of the unknown vertces v, c v denote the gven constraned vertces, and N(c) denotes the set of the constraned vertces. The constraned vertces refer to the specfed boundary of the patch M d here. The resultng patch denotes as M = { V, EF, }, that preserves the orgnal topology. d Fttng the Laplacan coordnates of the unknown geometry V to the gven δ- coordnates yelds an over-determned and sparse lnear system. As a result, we can obtan a global, detal-preservng and non-teratve soluton V. However, snce Eq.6.15 just takes nto account rotaton rather than scalng, t can be observed that the Laplacan coordnate framework leads to some geometrc dstorton, such as shrnkage as shown n Fg.6-4c. Shape and Feature Preservaton Our goal s to transfer the desred detals to the surface of another specfed model wthout any notceable artfacts. Shape and feature preservaton depends on the smoothed verson M d and the detals ξ of M. Assume that M d s smooth enough here. Approprately reorented and scaled detals ξ wll not lead to unexpected dstortons. In some scenaros, mproper scalng may lead to vsble artfacts. Our basc dea s to take scalng nto account n the transformaton Eq.6.15 between the smoothed versons M and M d as well as rotaton for shape and feature 17

120 preservaton. Smlar to rotaton, the scalng has been obtaned by our rotaton-scaleshear decomposton. Accordngly, the δ-coordnates of revsed verson of Eq.6.15 as follows, M d can be updated by the δ = δ + Sc Rξ. (6.17) ( d) ( d) In general, compared to rotaton, the nfluence of the scalng seems farly mperceptble. Ths s because Eq.6.16 gves out a soluton of the resamplng vertces V n a least square sense, whch can allevate the errors from the scale dsagreement. However, the least squared soluton s not always vald n some scenaros. To emphasze ths ssue, we delberately twst a tube and transfer the protuberant characters from on another tube to t as show n Fg.6-5. Snce the tube s a regular geometry model, t s easy to extract the character detals by smply subtractng the estmated base of the tube n Fg.6-5a. Ths helps us to exclude the effects from the prevous steps, partcularly the smoothng step whch cannot yeld a smooth base just lke the tube wthout protuberant characters. Moreover, t can be observed that the topologes of the models n Fg.6-5a and 5b are dfferent. We thus specfy some postonal constrants (e.g. red lnes) manually and further deform the 2D embeddngs to match them. Ths guarantees that the characters can be properly mapped to the desred postons. For comparson, we employ Eq.6.15 and 6.17 to Eq.6.16 respectvely to llustrate the effects of the detal transferrng wth and wthout scalng constrants n Fg.6-5c and 5d. Note that the twsted tube n Fg.6-5b contans both rotaton and shrnkage. Due to a lack of scalng constrant, t can be observed that the resultng mesh n Fg.6-5c has vsble artfacts. Ths s due to the fact that the volume change from Fg.6-5a to Fg.6-5b (.e. mddle part of tube) s too large to overcome the scale dsagreement of the characters and the twsted tube 18

121 effectvely n the least squares sense. When further nvestgatng the local mesh, t can be observed that there are many self-ntersecton trangles n Fg.6-5c. a. b. c. d. Fgure 6-5. Transfer the characters on the tube n (a) to the twsted tube surface n (b), and show the results n (c) by usng Eq.6.15 and (d) by usng Eq.6.17 respectvely. The red lnes n (a) and (b) ndcate the postonal constrants usng for matchng 2D embeddngs. 6.5 IMPLEMENTATION AND ANALYSIS In our proposed skn sldng algorthm, the partton step needs to solve a generalzed egensystem wth a large, sparse and symmetrc matrx by usng Lanczos algorthm. Fgure 6-2 ndcates that we just need to compute the frst t egenvectors of the graph Laplacan nstead of all egenvectors. We thus utlzed a freely avalable code--egfp() at [Money and Ye 25] here, whch mplements an nverse free precondtoned Krylov subspace projecton method. The cropped sknnng patches (.e. stretch regons n our experments) can usually mantan quadrangle-lke boundares. a. 19

122 b. c. d. e. Fgure 6-6. (a) shows the arm muscle bulge. The frst 3 mages demonstrate the mesh change durng arm bendng. We hghlght the results by our method n blue whle the result by [Yang et al. 29] n red. The last 2 mages show the dfference of the character locatons due to skn sldng by our method (the 4 th mage) and the lnear nterpolaton [Yang et al. 29] (the 5 th mage) respectvely. (b) shows the skn sldng effect of clenchng the fst. The red mesh s maded by our method whle the blue one by [Yang et al. 29]. (c) shows the skn sldng effect at the elbow durng arm bendng. The red mesh s maded by our method whle the blue one by [Yang et al. 29]. (d) shows the skn sldng effect of frownng expresson. The blue mesh s made by our method whle the red one by [Yang et al. 29]. (e) shows the skn sldng effect of elephant runnng. The blue mesh s made by our method whle the red one by [Yang et al. 29]. Snce the cropped compresson and stretch ROIs are adjacent to each other here, we set them as a whole sknnng patch for further processng. In some rare scenaros, t s possble to encounter some large sknnng patches. The small patches are easer to flatten nto such a 2D rectangular parametrc doman 11

123 wth low dstorton. We thus have to cut them nto a set of adjacent small patches manually. However, n the resamplng step, such adjacent patches are not processed ndvdually but are stacked together nto Eq.6.16 for a quas global soluton. Moreover, t s possble to stack all of the adjacent and nonadjacent patches together nto Eq.6.16 for a global soluton. However, due to the nverse of the large Laplacan matrx, we stll processed the nonadjacent patches ndvdually n our mplementaton. In the followng experments, we frst performed our proposed skn sldng algorthm on a set of human sknnng regons n Fg.6-6. To demonstrate the effcacy of our proposed method n comparson to the exstng method [Yang et al. 29], we overlapped the resultng meshes by usng our algorthm and [Yang et al. 29] respectvely together. In the above experments, the skn surfaces are too smooth. To demonstrate the feature preservaton and transferrng, we performed our proposed graph Laplacan based skn sldng algorthm on the tal of a seahorse model. The seahorse tal was bent up here. Because of the large stretch and bend, the bumpy surface of the tal became both very flat and very smooth, that s, t lost many concave convex detals (see Fg.6-7b). Our goal s to recover the detals of the tal mesh as much as possble rather than ts volume before deformaton. The skn sldng result s shown n Fg.6-7c. For the comparson purposes, we also show the result usng lnear nterpolaton method [Yang et al. 29] n Fg.6-7d. It can be observed that the result acheved usng the lnear nterpolaton method n Fg.6-7d s smoother than that acheved usng our proposed skn sldng algorthm n Fg.6-7c. Ths mples that the lnear nterpolaton method cannot transfer detals to the surface of the target model. Moreover, one can observe that the seahorse tal after bendng n Fg.6-7b s very 111

124 close to the nterpolaton result n Fg.6-7d. Snce the tal surface after bendng s too smooth n Fg.6-7b, performng any nterpolaton technque on t wthout any extra cues of the surface wll not brng about any new detals or features. Ths further justfes that the lnear nterpolaton methods cannot transfer the features to the target surface. Further Extenson a. c. d. b. Fgure 6-7. The orgnal seahorse models before and after bendng ts tal are shown n (a) and (b) respectvely. Snce the tal s bent up and stretched, the surface s too smooth. The result produced by our proposed skn sldng algorthm s shown n (c). For comparson, the result acheved usng the lnear nterpolaton method n [Yang et al. 29] s shown n (d). Our method s performed on the tal part nstead of the whole seahorse mesh. Thus, the entre tal was frst cropped manually as the nput of the partton step, contanng vertces 3153 and trangles Then the partton step yelded two patches, stretch and compresson ROIs. Snce they were adjacent, we vewed them as an entre sknnng patch contanng vertces 765 and trangles a. b. c. Fgure 6-8. Illustraton of transferrng the characters to a twsted tube surface. The source and target models are shown n (a) and (b) respectvely. (c) shows the result of our proposed method. Snce the partton step only determnes the sknnng regon rather than stretchng and compressng regons correspondngly, we performed our method on a sknnng part of tube (.e. hghlghted by red rectangle) nstead of the whole tube. The cropped mesh contans the vertces 3317 and trangles