Temporal Texture Synthess by Patch-Based Samplng and orphng Interpolaton Chao-Hung La and Junn-Ln Wu Department of Computer Scence and Engneerng, Natonal Chung Hsng Unversty Tachung 40, Tawan, ROC. 886-4-840497-91 {phd9415, lwu}@cs.nchu.edu.tw ABSTRACT We present a new algorthm for syntheszng temporal textures, whch s smple and requres only a statc texture mage as nput to produce a contnuous varyng stream of realstc mages. We frst ntroduce the bass sequence generaton procedure n whch the chosen patches from the nput texture mage are sttched on the output frame va blended alpha mattes formed by mnmum error cuts. All the frames n the generated bass sequence are torodal. We then employ the probabltes of smlarty and transton lnks to generate an nexhaustble sequence wth quasperodc qualty. To ensure the smoothness of frame-to-frame transton, we nterpolate natural metamorphoss n each transton lnk usng the effcent automatc morphng technque whch solves the problem of morphng between chaotc textures wth obscure features. In addton, the proposed method allows users to smply and promptly control the moton parameters and nteractvely render the landscape anmatons. We combne the proposed approach wth the color transfer technque to augment the vsual gratfcaton. Several examples ncludng scenes for cloud, fre, and water are presented to demonstrate that the proposed algorthm s smple, effcent, and controllable for syntheszng temporal textures. Categores and Subect Descrptors I.3.8 [Computer Graphcs]: Applcatons General Terms Algorthms Keywords mnmum error cut, temporal texture synthess, morphng nterpolaton, color transfer 1. INTRODUCTION Renderng realstc scenes s one of the maor goals n computer graphcs. Texture mages are wdely used to synthesze the scenery for ths purpose. However, many applcatons requre dynamc qualty to augment the vsual realty, such as computer game, computer-aded desgn, and computer anmaton. For example, a number of nature phenomena, such as a wavng lake, a flowng rver, a burnng fre, or flyng clouds, should be descrbed as moton effects. Statc texture mages are ncompetent for portrayng the tme-varyng panoramas. In consequence, the promsng materals used for constructng vvd scenes wth dynamc qualty are temporal textures. Temporal textures are motons wth ndetermnate extent both n space and tme [0]. They can be used to represent dverse dynamc natural phenomena such as fre, smoke, and water. any researches have been devoted to producng temporal textures, and they can be broadly categorzed nto three classes. The frst class ncludes the technques of mathematcal smulaton [15, 19, 1]. These technques are able to produce photorealstc or nonphotorealstc scenes. However, they have to determne partcular formulas for specfc textures, whch are dffcult to develop. Therefore, an algorthm that s able to generate varety of temporal textures would be preferred. The second class ncludes the frame-based methods whch use a raw vdeo clp as nput sample to synthesze a new vdeo sequence [1, 3, 17]. They splt the orgnal vdeo nto components, whch are clps or frames, and reorganze the order of those components wth smooth vsual realsm by some transton probabltes. These technques can produce temporally nfnte output vdeos, but whch are not spatally expansble. The thrd class ncludes the technques based on D texture synthess methods, whch are extended to 3D temporal texture synthess approaches by replacng the D enttes wth ther 3D counterparts [6, 8, 0]. They analyze the sample vdeo and extract ts structure whch s then used to synthesze a new smlar lookng vdeo. However, the user s requred to provde the source vdeo, whch contans not only one but a number of vdeo frames. Furthermore, t takes tme to analyze the sample vdeo and buld the structure. Recently, La and Wu [10] presented a temporal texture synthess algorthm whch requres only a statc texture mage as nput to synthesze temporal textures wth nexhaustble and sem-regular qualtes. They used a chessboard fllng approach to synthesze a bass frame sequence. Then ther system rendered temporal textures va the leads of the weghted probabltes matrx. And the smooth qualty s mproved by usng the cross-fadng technque. Ther user nterface allows users to nteractvely render landscape anmatons usng the syntheszed temporal textures. In ths paper, we ntroduce an effcent and controllable algorthm, whch can entrely replace the method presented n the earler work [10]. The proposed method has followng contrbutons:
Frst, n the stage of generatng bass sequence, we use the approach of blended mnmum error cuts to handle the overlappng regons between adacent texture patches. Wth lttle ncreasng the computatonal cost, we mprove the seamless qualty n each syntheszed frame of the bass sequence. Second, we replace the probabltes matrx wth explct transton lnks whch are more effcent and space-savng. Thrd, we replace the cross-fadng wth morphng nterpolaton technque and mprove the transton qualty. In addton, our morphng technque s able to solve the problem of fndng correspondng features between chaotc textures n whch the features are obscure. It s also capable of been used to morph between mages wth smlar or moved around features. Fnally, we nsert the color matchng procedure nto the nteractve renderng method. Ths color matchng step s mportant when the user wants the syntheszed temporal textures to have the color characterstcs of the target scene to augment the vsual harmony. The rest of ths paper s organzed as follows. In secton, we revew the related work brefly. In secton 3, the proposed algorthm s presented. Experments are conducted n secton 4 to verfy the effectveness of the proposed method. Fnally, conclusons and future work are ncluded n secton 5.. RELATED WORK any researches for syntheszng temporal textures have been proposed recently. We and Levoy [0] modeled the nput vdeo as 3D neghborhoods. They drectly copy those pxels whose neghborhoods are matched to the output textures pxel by pxel and slce by slce. Although accelerated by vector quantzaton, ther pxel-wse algorthm s stll tme-consumng. Schödl et al. [17] proposed a termnology called vdeo texture. They decompose the nput vdeo nto frames or clps and recombne them by the asssts of the transton probabltes. They use the cross-fadng or vew morphng technque to varnsh the vsual dscontnutes f there are no graceful transtons n the vdeo. Vdeo texture technque can produce temporally nfnte but not spatally expansble output vdeos. Joseph et al. [6] analyzed the whole nput sgnals and construct herarchcal multresoluton analyss trees, called RA trees. They transform the newly statstcally merged RA trees back nto sgnals, yeldng output textures. They extend the dea from D textures to 3D tmevaryng textures. Ths technque s unable to create nfntely long sequences. Kwatra et al. [8] used graph cut to synthesze D and 3D textures. In syntheszng temporal textures, the nput vdeos are concatenated wth optma seams, whch are rregular 3D cut surfaces. Although accelerated by FFT-based technque, the searchng for the cut surface s stll tme consumng. Bhat et al. [1] captured the motons of textured partcles n the nput vdeo along user-specfed flow lnes. They nteractvely synthesze seamless vdeo of arbtrary length by enforcng temporal contnuty along a second set of user-specfed flow lnes. However, the motons of the syntheszed vdeo are the same as the captured one. Recently, there are other ways to generate a vdeo from a sngle mage. Kwatra et al. [9] extended ther texture optmzaton algorthm to synthesze the texture sequence that moves accordng to a gven flow feld. However, ther flow-guded synthess method requres large number of RA to synthesze each frame, and ts speed s slow. Chuang et al. [] anmated the nput mage usng the spectral method, whch employees a physcally based spectrum flter and nverse Fourer transform to create the stochastc moton texture. Ther method can apply a wave feld to a calm water surface of no rpples, but not reproduce the wavng motons of whch the surface already has. 3. THE PROPOSED ALGORITH Gven only a statc texture mage as nput, our system can create plausble texture moves whch can be further used to produce composte landscape anmatons. The algorthm descrbed n ths paper s a fully mproved verson from the prevous one [10]. Please refer to the orgnal work for more detals. Our algorthm for syntheszng temporal textures and nteractvely renderng landscape anmatons ncludes the followng man steps: 1). Bass sequence generaton, ). Transton lnks buldng, 3). Automatc morphng nterpolaton, 4). oton parameters controllng, and 5). Interactve renderng. The basc concept of our algorthm s that gven a D texture mage as nput, we annex the temporal component to the syntheszng process to produce 3D temporal textures. At frst, a fast and effcent bass sequence generaton procedure s used to synthesze a plausble sequence of mages. Whle syntheszng each frame n the bass sequence, we sttch the chosen patches va the blended mnmum error cuts. At the next stage, we measure the smlarty of n-between frames and buld frame-to-frame transton lnks for reorganzng the order of the frames wth smooth vsual realsm. In order to ensure the smoothness of frame-to-frame transtons, the proposed automatc morphng technque s used to nterpolate smooth metamorphoss between each par of frames connected by the transton lnks. In addton, our user nterface allows users to smply control the moton parameters and nteractvely render composte landscape anmatons. Furthermore, we combne the nteractve renderng method wth color transfer technque [16] to acheve color matchng effects. F -1 temporally matched F canddates search spatally matched nput texture Fgure 1. Spatal and temporal constrants. The canddates n the set of spatally matched patches are then extracted by the smlarty wth the patch at the same poston n the prevous frame. 3.1 Bass Sequence Generaton We use the method upgraded from the bass sequence synthess procedure ntroduced by La and Wu [10] to synthesze an output bass sequence. Fgure 1 llustrates the process of syntheszng each frame n the sequence. When searchng for matched patches n the syntheszng process, we also consder both the spatal and temporal constrants. What s mproved from the earler procedure s how the overlappng regons between adacent texture patches are handled.
The earler work used feather blendng technque whch s the same as [11]. Another common method s usng mnmum cost cuts [4, 8]. Nealen and Alexa [14] even proposed an approach for re-syntheszng nvald pxels. Although they use the acceleratng technque based on k-coherence search, the re-syntheszng approach s stll tme-consumng. We ponder both the smoothness qualty and computatonal cost and, therefore, use the method of blended mnmum error cuts unfyng the works of [4] and [11]. We fnd the mnmum error cuts n the overlappng boundares around the sttched patch. These four cut paths are naturally connected and form a closed loop whch s used to make an alpha matte, as shown n Fgure (left). We use a 3 3 Gaussan flter to blend the alpha matte, as shown n Fgure (rght). Fgure. Closed mnmum error cuts and blended alpha matte Then, the patch area s rendered as follows: P ( = ( O( + (1 ( ) B(, (1) where B s the chosen patch from the nput texture, O s the current output texture, and s the Gaussan blended alpha matte. ( x, merely ndcates the same pxel poston. The method of blended mnmum error cuts s effcent for the guarantee of seamless qualty and fast wth ncreasng lttle computatonal cost. 3. Transton Lnks Buldng In order to produce a contnuous nfntely varyng stream of vdeo mages from the fnte bass sequence, we have to fnd smooth transtons for replayng the sequence from one part to another. La and Wu [10] ntroduced a loopng rule whch s usng a weghted probabltes matrx. However, after we prune the matrx and keep the good transtons, the matrx becomes sparse. The key pont s what the good transtons are. We want to have the best transtons whch create the mnmal vsble dscontnuty. But, f only the best transtons are chosen, the dsplay sequences are prone to appear monotonc repeats. Consequently, prunng the probabltes matrx and avodng monotonc repeats are both needed. We use a set of explct lnks whch ndcate the transtons from one frame to another, along wth an assocated local probablty. At the same tme, t s mportant to fnd a crteron of smlarty between frames for mantanng the smoothness qualty of the syntheszed sequences. L dstance s smple and commonly used [17]. However, the human vsual system s most senstve to edges, corners, and other hgh-level features n textures []. In vew of ths, frame feature s more mportant for computng the measure of smlarty between the par of frames. However, t s hard to extract the mplct features for chaotc nature textures, lke water, smoke, and fre. Before computng the measure of smlarty between the par of frames, we use the mplementaton of the Laplacan flter [5] to sharpen the obscure features. And then, we compute the summed squared dfferences of colors. So that, the frame features gve more contrbutons to the measure of the L dstance. We call ths dstance sharpened L dstance. We use a set of explct lnks from one frame to another, along wth an assocated probablty. At frst the matrx of dstance, n whch each element D of the matrx descrbes the sharpened L dstance from frame F, we fnd the frames F to F and + r < n and r F, are computed. For each frame F, wth mnmal k D and D, k 0 < k <, and n s the number where of frames n the basc sequence. To avod trval loop, we set the constant r > 1. Each frame has the best forward and backward transtons, except for the frst and last r frames whch only have forward or backward transtons. And then, we turn all forward transtons to backward, and remove all duplcate transton pars. Fnally, we buld a forward lnk for each frame to the next frame n the orgnal order except for the last frame. The reachable complexon s guaranteed that each frame have the way to another frame F, where F may, by walkng through certan of transton lnks. All the probabltes assocated wth each transton lnk are ndependent to each other. We locally set the probablty as: D, + 1 P =, () D + D, + 1 where P s the probablty assocated wth the transton lnk from frame F to F. Each frame may have several backward transton lnks and must have only one forward lnk to the next frame n the orgnal order except for the last frame whch only has backward lnks. Our renderng system walks through the transton lnk each step accordng wth the probablty assocated to the lnk. Ths onte-carlo process makes the syntheszed sequences have quas-perodc qualty. 3.3 Automatc orphng Interpolaton By means of the assocated probabltes, we can fnd smooth transtons from one frame to another when playng the sequence to provde contnuous nfntely varyng stream of vdeo mages. However, as we have mentoned above, the sequence s syntheszed from a sngle texture mage. Sometmes, there stll appear notceable abrupt changes when playng the sequence. We have to ensure the smoothness of frame-to-frame transton to gve the guarantee of the smoothness qualty of the syntheszed temporal texture. La and Wu [10] used cross-fadng technque to reduce the vsble dscontnutes. Cross-fadng s smple and fast, whch s orgnally used to produce morphng effects [17]. By usng ths technque, the notceable dscontnutes are blended away and the smoothness s mantaned. However, cross-fadng brngs a certan
level of blur n the dsplay sequences. Another approach to smooth the transtons and, meanwhle, avod the blurrness s morphng whch dstorts and fades one mage nto another through a seamless transton. But, morphng does not work well for chaotc motons because t s hard to fnd correspondng features [8]. Ths s the key pont we have broken through. We present an automatc morphng technque whch s able to solve the problem of fndng correspondng features between chaotc textures n whch the features are obscure. Some varants of ths technque have been presented before. Kang et al. [7] presented an automated mult-resoluton lattce deformaton technque to perform warp trackng. Ths technque practces well at smple warp, but they do not address the warp between stochastc textures. Lu et al. [1] presented a texture metamorphass technque that s able to resolve the warp between textures wth stochastc patterns. Usng ths technque, the user needs to specfy a pattern n the source and target textures to establsh the local feature correspondence. atusk et al. [13] proposed a morphable nterpolaton for textures. They buld a morphable model that facltates the nterpolaton of textures usng a warp deformaton. Ths technque s sutable for textures wth explct features. Our morphng nterpolaton approach s based on [7] and [13]. However, our method s specalzed from the prevous works and s able to acheve the morphng between chaotc textures wth ndstnct features, such as cloud, fre, and water. Furthermore, there s no need for the user to specfy the mappng ponts. In order to guarantee the smoothness qualty of the syntheszed temporal texture, we nterpolate smooth morphng frames nto each lnked transton from one frame to another. Reference to [7] and [13], we compute coarse-to-fne mappng functon based on regularly trangular mesh. And barycentrc formulaton and blnear nterpolaton [18] are used to compute the pxel values n the warped mesh. The basc dea s as follows. Consderng a transton from frame F to another frame F, we downsample both frames to multresoluton pyramds. Startng at the lowest resoluton, we overlap regular trangulaton mesh onto the source frame F. For each vertex of the mesh, we search for the mappng poston whch s the best matched n the target frame F. For each hgher resoluton, we magnfy the mesh by a factor of two and subdvde the mesh by splttng each trangle nto four. The mappng and mesh ncreasng processes contnue untl the orgnal scale s reached. Then frame F and F are warped to each other by computng the affne mappng and both are blended together to produce the ntermedate morphng versons. 3.3.1 esh appng Functon When computng the mappng functon, there are two constrants to search for the best matched poston of each vertex n the mesh. The frst, the vertces are not allowed to move past ther one-rng range. The second, we only search for the optmal poston at whch the cost s mnmal n the allowed range. Smlar to the prevous works [7, 13], there are two components contrbuted to the matchng cost. One s the pxels dfference, the other s the measurement of deformaton. Kang et al. [7] used the metrc whch contaned the color dfference and the amount of deformaton experenced by each grd edge. atusk et al. [13] used the L -error of the per-pxel scalar feature strength and the measure of warpng each trangle of ts one-rng neghborhood. We consder of both the features and localty of the textures n whch the features are obscure. Before the frames are downsampled, the Laplacan flter s used to sharpen the obscure features. When we search for the mappng poston of each mesh verte ts neghborhood s compared aganst all possble neghborhoods n the allowed range from the target frame. In our examples, ths neghborhood s a 3 3 or 5 5 block centered by the vertex. Whle the deformaton penalty s measured, we smply use the amount of moton of the verte snce we have constraned the movement not to pass ts one-rng range. For elaboraton, we defne the symbols and functons used n our method and summarze the algorthm n Fgure 3. Let F and F denote the source and the target frame, and I and I are ther Laplacan sharpened versons respectvely. G denotes the L- th level pyramd. V ( L)( r, denotes the vertex at the r-th row and c-th column n the trangulaton and the poston s ( x, n G. We want to compute the L-th level mappng functon, makng: I, I Laplacan flter F, F G ( L), G ( L) pyramds for each level of I, I, L > 0 For each level L, startng at the lowest resoluton For each vertex v n V N(v) neghborhood of v Search for mnmum cost at each poston n one-rng range, that : cost weghted average( pxels dfference, deformaton measure ) ( L) V ( L)( r, V ( L)( r, x, y ) For each vertex v n V ( L), L > 0 V ( L 1)(r,x,y ) V ( L)( r, x, y ) Dvde each grd nto four (0) Fgure 3. The approach of computng mesh mappng functon V ( L) ( L)( r, V ( L)( r, x', y' ) 1 ( L), (3) where 1 ( L ) denotes the nverse mappng, that s, 1 ( L) = ( L). So, we map a vertex whose poston s ( x, n the mesh over G to the poston ( x, y ) n G by, and vce versa. Fgure 4 shows a mappng
result, where = (0), whch s the mappng functon at the orgnal scale. F F V 1 Fgure 4. The mappng functon of frame V F and 3.3. orphng Interpolaton Barycentrc formulaton and blnear nterpolaton technque [18] are used to produce the nterpolated morphng versons between frame F and F. We defne a functon W ˆ ( Iˆ, ˆ,αˆ ) as returnng an mage whch s the warped verson of the mage Î by usng the mappng functon ˆ and the warpng parameter αˆ. The nterpolated morphng formula s shown n (4). F ( F,, t α ) + ( 1 t α ) Wˆ ( F,, t α ) t α W ˆ 1, (4) 1 where α = 1 /( T + 1), T s the number of ntermedate morphng frames between F and F. For each transton lnk descrbed at secton 3., there are T morphng frames nterpolated. Fgure 5. The morphng nterpolaton of two frames. The mages are lsted row-wsely. The frst mage s the source frame, and the last one s the target frame. Fgure 5 shows the morphng nterpolaton of two frames lnked by a transton. Our morphng technque creates smooth metamorphoss between each lnked couple. And the warpng step s performed wthout user-specfed control ponts. Now, consder a transton from frame F to F or to F, when t occurs, our + 1 system play the ntermedate morphng nterpolaton tll the target frame, and next transton s determned from now. Our system s able to walk through every transton lnks smoothly. RS value 3.5 1.5 1 0.5 0 9-10 8-9 7-8 6-7 5-6 4-5 3-4 -3 1-0-1 frame number Fgure 6. The seres of RS values We use the formula of root mean square (RS) to calculate the measure of the varyng quantty. The less s the RS value, the more smlar are the two frames. The RS value of the source frame and the target one from Fgure 5 s about 11.73. Fgure 6 depcts the seres of RS values between each par of adacent frames shown n Fgure 5, n whch frame 0 denotes the source frame and frame 10 denotes the target one. As shown n Fgure 6, after nterpolatng the seres of morphng mages, the average RS value of every two adacent frames s reduced to about.19 and the varyng range s about ± 0. 5. The smallness and statonary varyng of RS values verfy that the proposed morphng technque s able to nterpolate natural metamorphoss between the lnked frames.
temporal texture to have the color characterstcs of the target scene to augment the vsual harmony. Fgure 11 depcts ths llustraton. The color transfer technque s used to transfer the color mood n the orgnal nput texture mage to the masked area n the landscape photo. The transferred texture mage s used as nput to synthesze the temporal texture whch s then sttched on the landscape photo. Consequently, the rendered landscape anmaton wll have no obtrusve color dstrbutons. Fgure 7. The metamorphoss of a man s face n dfferent phases The automatc morphng technque s also capable of beng used to morph between mages wth smlar or moved around features, lke two vews of the same person at dfferent phases. For clearly, we depct the metamorphoss that a man s face transforms between two photographs, as shown n Fgure 7. The resultng anmaton shows that the portons above neck change naturally. But there below neck appear ghostng n ntermedate frames because the features of the necks of the shrts are very dfferent to each other. The frames syntheszed by the bass sequence generaton technque are so smlar that our automatc morphng technque works exactly. In addton, the problem of morphng between chaotc textures wth ndstnct features s also addressed. 3.4 oton Parameters Controllng The key pont of allowng users to easly control the moton parameters, ncludng movng drecton and ambulatng speed, of the syntheszed temporal textures s that all the frames n the sequence are torodal. La and Wu [10] used ths tleable characterstc to augment the vsual gratfcaton. We also preserve ths faclty, snce all the output textures n the bass sequence and all the morphng nsertons n the transton lnks are torodal as before. The movng drecton can be specfed by ust a mouse clck and the ambulatng dstance per step, even pxels or subpxels, can be specfed va a dalog. In practce, we use barycentrc technque [18] to perform the acton of movng the dsplay frame torodally. 3.5 Interactve Renderng The applcablty of our temporal texture synthess algorthm s easly augmented by allowng users to nteractvely create artfcal landscape anmatons. Some varants of ths practce have been consdered before [1,, 8, 10]. We stll upgrade ths applcaton from the work presented by La and Wu [10]. For comparson, we brefly ntroduce the basc steps lsted n [10]: 1). Image mask desgn, ). Warp grds desgn, 3). Renderng. Frst, mage mask s desgned for separatng the masked area n the landscape photo from the background. Second, the user specfes the warp grds to warp the syntheszed temporal texture to cover the masked area. Fnally, the warped temporal texture s sttched on the landscape photo va the mage mask. In ths nteractvely renderng stage, we upgrade the approach proposed above by annexng the color transfer technque [16]. Ths mprovement s mportant when the user wants the sttched 4. EXPERIENTAL RESULTS AND DISCUSSION We appled our algorthm to several textures, such as cloud, fre, and water. We mplement our algorthm va crosoft Vsual C++ 6.0 on a machne wth Intel Celeron G and 56 B RA. Our algorthm requres only a statc texture mage as nput, and the process of bass sequence generaton follows to create a bass sequence, n whch all frames are then connected by transton lnks accordng to the probabltes of smlarty. The algorthm nterpolates smooth metamorphoss n each transton couple usng the proposed automatc morphng technque. The system walks through the transton lnk determned by the probabltes of smlarty. Whle renderng each experenced frame, the system shfts the frame accordng to the user-specfed drecton and ambulatng speed. Fnally, the realstc temporal texture s syntheszed by our algorthm. Fgures 8, 9, and 10 show some of our results, n whch the left column shows the nput textures and the rght column lsts the syntheszed temporal textures. Table 1 summarzes the processng tme for those results. The runnng tme for calculatng transton lnks s around 1 second for all the examples. cloud 1 cloud Fgure 8. Results for cloud
The number of frames n the bass sequence s 10 for each result. For cloud 1, the resolutons of the nput texture and the output frame are 18 18 and 88 88 respectvely. There are 15 transton lnks bult. For each lnk, we nterpolate 9 morphng frames n t. For cloud, the resolutons of the nput texture and the output frame are 160 160 and 80 80 respectvely. There are 17 transton lnks bult. For each lnk, we also nterpolate 9 morphng frames n t. When renderng both examples, we move 0.8 pxels n each step toward 135 o. The rendered sequences for both cloud examples look lke the true cloud formaton. For fre 1, the resolutons of the nput texture and the output frame are 18 18 and 88 88 respectvely. There are 15 transton lnks bult. For each lnk, we nterpolate 3 morphng frames n t. Because the moton speed of fre s faster than the speed of cloud, so we use fewer ntermedate morphng frames. When renderng, we move 1 pxel n each step toward 45 o. For fre, the resoluton of the nput texture and the output frame are 19 19 and 56 56 respectvely. There are 15 transton lnks bult. For each lnk, we also nterpolate 3 morphng frames n t. When renderng, we move 1.5 pxels n each step toward 90 o. The rendered sequences for both fre examples look lke the burnng fre. water 1 water Fgure 10. Results for water fre 1 fre Fgure 9. Results for fre For water 1, the resolutons of the nput texture and the output frame are 19 19 and 56 56 respectvely. There are 15 transton lnks bult. For each lnk, we nterpolate 3 morphng frames n t. For water, the resolutons of the nput texture and the output frame are 160 160 and 56 56 respectvely. There are 16 transton lnks bult. For each lnk, we nterpolate 5 morphng frames n t. When renderng, we move 1 pxel n each step toward 0 o. The rendered sequences for both water examples look lke the flowng water. In all of these examples, the proposed temporal texture synthess algorthm s successful to produce the texture sequences whch resemble true vdeos. The output resoluton and the length of the rendered sequence are at the user s wll. In addton, the moton parameters can be specfed by the user at run tme. It s effectve, fast, and convenent that our algorthm requres only a texture mage as nput to synthesze temporal textures wth spatally and temporally expansble qualtes, and the moton parameters are controllable. Fgure 11 shows two results for composte landscape anmatons. In ths fgure, the top row shows the orgnal nput texture (left) and the color-transferred one (rght), the second row shows the orgnal landscape photo, the thrd row shows the rendered landscape anmaton by usng the orgnal texture mage as nput, and the bottom row shows the rendered landscape anmaton by usng the color-transferred texture mage as nput. The scene shown at the thrd row looks dscord, but the one at the bottom row looks harmony and lke a rver flowng n a snow scene.
Table 1. The processng tme of the proposed method n seconds Input texture Bass sequence generaton orphng nterpolaton cloud 1 1 14 cloud 1 146 fre 1 16 64 fre 3 41 water 1 34 41 water 39 73 5. CONCLUSIONS AND FUTURE WORK Temporal textures are nterestng and mportant for a wde varety of anmatons n computer graphcs. In ths paper we have presented a fast and effcent algorthm for temporal texture synthess. The proposed method s smple and t requres only a statc texture mage as nput to produce plausble mage sequence wth nexhaustble and quas-perodc qualtes. In addton, the user nterface allows users to smply and promptly control the moton parameters and nteractvely render composte landscape anmatons. Several scenes for cloud, fre and water are presented n ths paper. Expermental results confrm that our temporal texture synthess algorthm s smple, effcent, and controllable. We have successfully syntheszed realstc temporal textures and acheve the extensblty and controllablty. However, for some examples, lke wavng ocean, the shape of the waves would be broken when nterpolatng morphng frames. Ths s due to the mss mappng of the mesh vertces. In future work, we are nterested n the physcal flow felds to help make the mesh mappng functon complete. Fgure 11. Results for nteractve renderng. The red arrow ndcates the movng drecton. The orgnal landscape photo s borrowed from http://www.pcture-newsletter.com.
6. REFERENCES [1] Bhat, K. S., Setz, S.., Hodgns, J. K., and Khosla, P. K. Flow-based Vdeo Synthess and Edtng. AC Trans. on Graphcs, 3, 3 (Aug. 004), 360-363. [] Chuang, Y. Y., Goldman, D. B., Zheng, K. C., Curless, B., Salesn, D. H., and Szelsk, R. Anmatng Pctures wth Stochastc oton Textures. AC Trans. on Graphcs, 4, 3 (Jul. 005), 853-860. [3] Doretto, G., Chuso, A., Soatto, S., and Wu, Y. Dynamc Textures. Internal Journal of Computer Vson, 51, (Feb. 003), 91-109. [4] Efros, A. A., and Freeman, W. T. Image Qultng for Texture Synthess and Transfer. In Proceedngs of SIGGRAPH 001. Los Angeles, CA, 001, 341-346. [5] Gonzalez, R. C., and Woods, R. E. Dgtal Image Processng. Prentce-Hall, Inc., New Jersey, 00. [6] Joseph, Z. B., Yanv, R. E., Lschnsk, D., and Werman,. Texture xng and Texture ove Synthess Usng Statstcal Learnng. IEEE TVCG, 7, (Jun. 001), 10-135. [7] Kang, T., Gemmell, J., and Toyama, K. A Warp-Based Feature Tracker. crosoft Research Techncal Report, SR-TR-99-80, Redmond, Washngton, 1999. [8] Kwatra, V., Schödl, A., Essa, I., Turk, G., and Bobck, A. Graphcut Textures: Image and Vdeo Synthess Usng Graph Cuts. AC Trans. on Graphcs,, 3 (Jul. 003), 77-86. [9] Kwatra, V., Essa, I., Bobck, A., and Kwatra, N. Texture Optmzaton for Example-Based Synthess. AC Trans. on Graphcs, 4, 3 (Jul. 005), 795-80. [10] La, C. H., and Wu, J. L. A Novel Algorthm for Syntheszng Drectonal Temporal Texture Anmaton. In Proceedngs of nd IASTED nternatonal conference on Advances n computer scence and technology. Puerto Vallarta, exco, 006, 180-185. [11] Lang, L., Lu, C., Xu, Y. Q., Guo, B., and Shum, H. Y. Real-Tme Texture Synthess by Patch-Based Samplng. AC Trans on Graphcs, 0, 3 (Jul. 001): 17-150. [1] Lu, Z., Lu, C., Shum, H. Y., and Yu, Y. Pattern-based Texture etamorphoss. In Proceedngs of the 10 th Pacfc Conference on Computer Graphcs and Applcatons. Beng, 00, 184-193. [13] atusk, W., Zwcker,., and Durand, F. Texture Desgn Usng A Smplcal Complex of orphable Textures. AC Trans. on Graphcs, 4, 3 (Jul. 005), 787-794. [14] Nealen, A., and Alexa,. Fast and Hgh Qualty Overlap Repar for Patch-Based Texture Synthess. In Proceedngs of CGI 004. Crete, Greece, 004, 58-585. [15] Perln, K. Improvng Nose. AC Trans. on Graphcs, 1, 3 (Jul. 00), 681-68. [16] Renhard, E., Ashkhmn,., Gooch, B., and Shrley, P. Color transfer between mages. IEEE CG&A, 1, 5 (Oct. 001), 34-41. [17] Schödl, A., Szelsk, R., Salesn, D. H., and Essa, I. Vdeo Textures. In Proceedngs of SIGGRAPH 000. New Orleans, LA, 000, 489-498. [18] Shrley, P. Realstc Ray Tracng. A. K. Peters, Ltd. Natck, A, USA, 000. [19] Tessendorf, J. Smulatng ocean water. In Smulatng Nature: Realstc and Interactve Technques, AC SIGGRAPH 001 course notes No. 47, 001. [0] We, L. Y., and Levoy,. Fast Texture Synthess Usng Tree-Structured Vector Quantzaton. In Proceedngs of SIGGRAPH 000. New Orleans, LA, 000, 479-488. [1] We, X., L, W., ueller, K., and Kaufman, A. Smulatng Fre wth Texture Splats. In Proceedngs of IEEE Vsualzaton 00. Boston, A, 00, 7-37. [] Wu, Q., and Yu, Y. Feature atchng and Deformaton for Texture synthess. AC Trans. on Graphcs, 3, 3 (Aug. 004), 36-365.