Efficient Region-Based Pencil Drawing

Transcription

1 Effcent Regon-Based Pencl Drawng Shuo Sun Tanjn, Chna, School of Scence Tanjn Polytechnc Unversty Dongwe Huang Tanjn, Chna, School of Scence Tanjn Polytechnc Unversty Abstract Ths paper proposes an extenson to the exstng automatc pencl drawng generaton technque based on Lne Integral Convoluton (LIC. The orgnal LIC pencl flter utlzes mage segmentaton and texture drecton detecton technques for defnng outlnes and stroke drectons, and the qualty of a resultng mage depends largely on the result of the whte noses and the texture drectons. It may fal to generate a reasonable result when the whte noses and the texture drectons are not consstent wth the texture structure of the nput mage. To solve ths problem, we propose n ths paper to mprove the exsted LIC-based method. Frst, a more accurate and rapd graphbased mage segmentaton method s ntroduced to dvde the mage nto dfferent regons. Second, we present a new regon-based way to produce whte noses and texture drectons. We also demonstrate the enhanced LIC pencl drawng s closer to the real artstc style. KeyWord pencl drawng, lne ntegral convoluton, mage segmentaton, Non-photo-realstc renderng 1. Introducton Recently, Non-Photo-Realstc renderng (NPR has become one of the most mportant research topcs of computer graphcs. A number of technques have been developed to smulate tradtonal artstc meda and styles, such as pen and nk llustraton[wn94a, Wn96a, Sal94a, Sal97a], graphte and colored pencl drawng[tak99a, Sou99a, Sou99b], mpressonst styles[lt97a], pantngs of varous materals ncludng ol[her98a], water color[cur97a] and so on. The exstng researches on panterly mage generaton manly take two dfferent approaches. The frst Permsson to make dgtal or hard copes of all or part of ths work for personal or classroom use s granted wthout fee provded that copes are not made or dstrbuted for proft or commercal advantage and that copes bear ths notce and the full ctaton on the frst page. To copy otherwse, or republsh, to post on servers or to redstrbute to lsts, requres pror specfc permsson and/or a fee. Copyrght UNION Agency Scence Press, Plzen, Czech Republc. approach s to provde physcal smulaton to the materals and sklls, and has been manly combned wth nteractve pantng systems or 3D non-photorealstc renderng systems for generatng realstc panterly mages. The second approach s the panterly flterng, whch nvolves takng an mage and applyng some knd of mage processng or flterng technques to convert t nto an mage of a panterly look. Whle many excellent panterly flterng technques have been developed for generatng brushstroke based pantngs [Goo01a], relatve few publcatons can be found on convertng a source mage nto lne stroke based drawngs. In case of drawng, geometrc nformaton such as the outlne of regons, the drecton and shape of strokes becomes more crtcal, whle t s usually dffcult to extract such nformaton from D raster mages automatcally. Instead of modelng lne strokes geometrcally, Mao etc [Mao01a] have developed a pencl drawng flter usng Lne Integral Convoluton (LIC, a texture based flow vsualzaton technque [Cab93a]. The technque utlzed the smlarty between the appearance of LIC mages and pencl strokes, and succeeded n generatng lne stroke lke mages wth pxel-by-pxel mage flterng. It employs mage segmentaton and texture analyss technque to automatcally detect the outlnes and decde the stroke Full Papers 79 ISBN

2 orentatons for the mages. Then for each regon, f t contans drectonal textures, the texture drectons are used as the stroke drectons, otherwse a randomly chosen stroke drecton s assgned. It may fal to generate a reasonable result when the whte noses and the texture drectons are not consstent wth the texture structure of the nput mage. To solve ths problem, we propose n ths paper to mprove the exsted LIC-based method. Frst, a more accurate and rapd graph-based mage segmentaton method s ntroduced to dvde the mage nto dfferent regons. Second, we present a new regon-based way to produce whte noses and texture drectons. We realze automatc generaton pencl drawng from D mages. There are several advantages of our method comparng to the exsted. Frst, accordng to the characterstcs of the pencl drawngs produced by the artsts, dfferent areas have dfferent texture expressons. General speakng, some areas mght have very complcated textures and others mght be fant n textures, and that we are desre to acheve ths knds of artstc effects. Through controllng the noses of the dfferent areas, we can not only express the shade areas of the texture but also hold some subtle area and necessary blank area n the mage. The result s more smlar to the artstc style. The remander of ths paper s organzed as the follows: Secton gves a short survey on related work. Secton 3.1 ntroduces the orgnal LIC-Based pencl flter method. Secton 3. descrbes the algorthm graph-based mage segmentaton and Secton ntroduces our regon-based pencl flter method. Secton 4 concludes the paper and gves some examples.. Related Works Pencl drawng has been an mportant topc snce the begnnng of panterly mage generaton research hstory. In an early D pantng system called Pencl Sketch [Ver89a], a mouse based vrtual tablet s provded for allowng users to nteractvely specfy a set of parameters, such as the hardness of pencl, the pressure appled to a pencl, and the orentatons of strokes. Recently, Sousa and Buchanan developed several pencl drawng renderng technques based on an observaton model of pencl drawngs [Sou99a, Sou99b]. They bult the models of pencl, paper and how lead pencls nteract wth drawng paper through a careful nvestgaton of the real pencl drawngs usng scannng electron mcroscope. When the parameters of those models and the strokes are specfed, a D mage can be converted nto a pencl drawng. 3D polygon models can be automatcally rendered nto pencl drawngs by referrng to the tone value lookup table for the parameter values of the models. Takag and Fujshro proposed to model the paper mcrostructure and color pgment dstrbuton as 3D volume data and use volume ray-tracng for renderng color pencl drawngs [Tak99a]. Other exstng panterly mage generaton technques closely related wth our work are probably those successful works on pen and-nk llustratons [Wn94a, Wn96a, Sal94a, Sal97a]. In ther nteractve systems, pen-and-nk llustratons can be generated ether from 3D models or D mages by usng a set of pre-stored stroke textures. The largest dfference between our technque and all these exstng technques s that our technque can generate a pencl drawng from a source mage n a completely automatc way whle all these exstng technques rely, to certan extent, on user nterventons, for specfyng the attrbutes and drectons of strokes. Several commercal packages provde some flters for creatng pencl drawng effects. For example, Jasc Pant Shop Pro software supports a black pencl flter. However, to obtan a satsfactory result wth those flters, a user usually needs to combne the effects of many other flters and explore the best generaton process expermentally through tral and error for many tmes. 3. LIC Pencl Flter 3.1. LIC Algorthm Lne Integral Convoluton (LIC s a texture based vector feld vsualzaton technque [Cab93a]. As shown n Fgure 1, t takes a D vector feld and a whte nose mage as the nput, and generates an mage whch has been smeared out n the drecton of the vector feld through the convoluton of the whte nose and the low-pass flter kernels defned on the local streamlne of the vector fled. Fgure 1 Lne Integral Convoluton (LIC (a Input vector feld; (b Input whte nose; (c Output result The mages n fgure show the basc algorthm of the LIC. The nputs are the vector felds and whte noses. Full Papers 80 ISBN

3 Fg.3: Comparson of a real pencl drawng (a and an LIC texture (b. Fgure the basc algorthm of the LIC P s the output pxel, ρ ( τ s stream lne ( L τ L,L s the half length of the stream lne T ( ρ( τ lne, ( τ n P s ( ρ( 0 s the nose texture value n the stream K s a convoluton kernel. So the pxel value T : T ( ρ( 0 = L L k ( τ T ( ρ( τ L L k ( τ dτ dτ The dscrete form descrbes as follows: p s the dscrete pont n the stream lne; W s the contrbuton of p to P, namely the area that K( τ cover between p- 1 wth p. T ( P = N T = 0 N = 0 ( p W W The dea of usng LIC for pencl drawng generaton was nspred by the vsual smlarty of LIC mages and pencl drawngs. As an LIC mage s obtaned by low-pass flterng a whte nose along the local streamlnes of a vector feld, we can observe the traces of streamlnes along whch ntensty vares randomly. Such traces have a smlar appearance of pencl strokes where the varance of ntensty s caused by the nteracton of lead materal and the roughness of paper surface. Fgure 3(a s a dgtzed sample of a typcal mtatve tone used n real pencl drawngs. Fgure 3(b, presents the very smlar features as the tone mage by LIC processng. 3.. The exsted LIC pencl drawng method In general, for producng a pencls drawng from a D source mage, several steps are done. Frst we generate a whte nose from the source mage, then the orgnal mage s dvded nto dfferent regon and the boundary s extracted. Next we generate the vector feld representng the orentaton of strokes, and produce pencl drawng by applyng LIC to the whte nose and the vector feld. Fgure 4 depcts the algorthm of orgnal LIC pencl flter. It converts a D source mage nto a pencl drawng n the followng seven steps: 1. Generate a whte nose (Fgure 4(b from the source mage (Fgure 4(a.. Segment the nput mage (Fgure 4(a nto dfferent regons (Fgure 4(c. 3. Extract regon boundary (Fgure 4(d. 4. Generate the vector feld (Fgure 4(e representng the orentaton of strokes. 5. Generate stroke mage (Fgure 4(f by applyng LIC to the whte nose (Fgure 4(b and the vector feld (Fgure 4(e. 6. Add the boundary (Fgure 4(d to obtan the drawng wth outlnes (Fgure 4(g. 7. Composte the resultng mage (Fgure 4(g wth the paper sample (Fgure 4(h to obtan the fnshed pencl drawng (Fgure 4(. Full Papers 81 ISBN

4 pxel n the mage, the edge set E s constructed by connectng pars of pxels that are neghbors n an 8- connected sense (any other local neghborhood could be used. A weght s assocated wth each edge based on some property of the pxels that t connects, such as ther mage ntenstes. Neghbor edges are clustered nto a forest, and each tree n the forest s related to a mnmum spannng tree (MST. Fnally each MST s a sub-area. To judge whether two trees can be merged nto one tree, a predcate s defned. The predcate expresson s showed as follow. True: f( Df( C1, C > MnTC ( 1, C D( C1, C = False: otherwse Fgure4. The exsted LIC algorthm (the mage comes from [Mao01a] 4. Enhanced Regon-based LIC Pencl Flter 4.1. Graph-Based Image Segmentaton A well used technque n pencl drawng for conveyng the 3D shapes of objects and spatal relatonshp among dfferent objects n a scene s to emphasze the boundary between two dfferent regons by drawng outlnes or changng the appearance of strokes n the two regons. To create such effect, we propose to dvde the nput mage nto dfferent regons usng exstng mage segmentaton technque. In our current mplementaton, a Graph-Based mage segmentaton technque [Fel04a] s used for the regon extracton. Contrastng to the method n [Mao01a], ths method can dramatcally promote the performance of our pencl flter. As shown n fgure 5, the left mage s segmented nto many areas whch represent the respectve color dstrbutons for each regon. Fgure 5. Graph-Based mage segmentaton Graph-based mage segmentaton technques generally represent the problem n terms of a graph G = (V;E where each node v V corresponds to a D( C 1, C means the mergng predcate of the areas of C 1 and C, Df (C 1, C means the dfference between the area C1 and the area C. MnT (C 1, C s the mnmum nternal dfference. The advantage of the method s that the accuracy of the mage segmentaton can be adjusted by users and some detals of the certan regons can be gnored. Ths wll mprove on the effect of the pencl drawng. In addton, ths results n a graph wth O(n edges for n mage pxels, and an overall runnng tme of the segmentaton method of O(n log n tme. 4. Regon-based Nose Producton The whte nose mage s generated n a way that the probablty a whte value s set for a pxel s proportonal to the ntensty level of the correspondng pxel n the nput mage. The gray-scale tone of a resultng pencl drawng s manly decded by the whte nose mage. To match the tone between the nput mage and the resultng pencl drawng, we use the tone of the nput mage to gude the dstrbuton of nose. An mportant characterstc of the pencl drawng s ts ablty to preserve detal n low-varablty mage regons whle gnorng detal n hgh-varablty regons. The nput mage s then dvded nto many small regons whch have correspondng meanngs. The method mentoned n [Mao01a] dealt wth the nose accordng to unform crteron. The result noses would be falure to dstngush the mportant elements and the unmportant elements, when the range of the ntensty of the mage s small. Our regon-based method solves the queston through dynamc adjustng the threshold value of dfferent areas. It s very mportant to pencl drawng. We smply ntroduce the algorthm. Full Papers 8 ISBN

5 Because the processng of LIC s global, so some local regons wth exquste texture wll be destroyed. In addton to, as a knd of sketch drawng, pencl drawng should keep some blank areas from the pont of aesthetcs. So we need to segment the orgnal mage nto dfferent regon through above graph-based method, then we deal wth the each area through gvng certan threshold value. Commonly we select the average gray ntensty of the certan regons as the threshold values. Our method mproves the producng way of whte nose. We deal wth the orgnal gray mage accordng to dfferent gray ntensty range n the respectve regons. So the contrasts between lght and shade n the results are more emnent and more smlar to the artstc style. The formulaton of whte nose s showed as follows. Let I nput be the ntensty of a pxel n the nput mage, and P s a floatng-pont number generated wth a pseudo-random functon; R s the average ntensty n the th regons, and t s selected as the threshold to control the dstrbuton of the nose. Then the ntensty I nose of the correspondng pxel n the nose mage s decded n the followng way: max1:( f( P> T1 I nose1 = ( f( Input R mn1:( else Inose= ( P [0,1] max:( f( P> T Inose = ( f( Input> R mn:( else Here max1 and max are the maxmum gray values of the output pxels, whch usually are 55; and mn1 and mn are the mnmum gray values of the output pxels, whch usually are R. However, we can adapt these values to ft for the whole tone of the pencl drawng. I nput I nput T1 = k,1 ( 1, T = k, ( I nput I nput k,1 = λ1 (1, k, = λ ( λ, λ [0,1] 1 Accordng to our method, λ 1 and λ are two experental values. Our ntal experment result suggests that a default value of 0.7 and 0.3 produce a vsually acceptable result for most scenes. We also allow users to nteractvely adjust the values of λ 1 and λ. Through above method, the nose of each area can be controlled to match the features of the pencl drawng to a great extent. As shown n fgure 6, the pcture (a show the nose produced by old way, and the pcture (b s produced by our method. Obvously, the result nose dealt wth by our method can reflect the nature of the pencl drawng. Some areas need to be emphaszed correspondng to more dense noses, and some areas need to be fade out correspondng to sparse noses. (a (b Fgure 6 The exsted nose algorthm (a, and Regon-Based nose method (b. 4.3.Stroke Orentaton When usng mtatve tone, the drecton of strokes s an mportant factor contrbutng to the mpresson of a pencl drawng. The stroke drecton s also mportant for conveyng the shapes and textures of objects. If a texture presents a drectonal feature, then strokes should be orented n a drecton matchng that of the orgnal texture. For example, when we pant human har, we would have our pencl follow the drecton of har strands. To produce such effect, Mao etc [Mao01a] mplemented a Fourer texture analyss technque for extractng the local texture drectons. The basc prncple of the technque s that f a texture presents a specfc drecton, then after transform the texture nto frequency doman, the power spectrum should have large values n the drecton orthogonal to the drecton of the texture. They deal wth each pxel Full Papers 83 ISBN

6 n the nput mage and compute respectve texture drecton by the analyss of the FFT. The method mentoned above s very tmeconsumng, and they don t consder the local structure features of the mage. Usually, some regons have smlar drectons of the strokes, and these areas can be dealt wth the unform drecton accordng to artstc style. On the other hand, the drectons of the strokes vary dramatcally n some regons, so we keep the results of the respectve drectons, and then the detals features n the area should be represented. As shown n fgure 7, the drecton of har strands s mportant n (a and the detals of the cloud should be depcted n (b. D m ( V, j V j= 0 = The fnal feld vector of the each pxel s determned through the followng crterons: FV, j V, j : f ( D / m > T = V : ( else Here T s a gven threshold. Experments suggest that a default value of 0.7 produce a vsually acceptable result for most scenes. (a (a Fgure 7 the drecton of har strands (a, and the detals of the cloud (b. Here we propose a method to produce the drecton of the stroke accordng to the features n the regons. Frst, the orentaton vectors of the pxels are calculated by sobel operator. Then a crteron s presented to decde the drecton of the stroke. Let V, j = 0,1,,... m be the feld vector of j ( the jth pxel n the th area. It s computed by sobel operator. Here s the numbers of the pxels n the m j jth area. Then the mean of the feld vectors n each area s calculated. At the same tme, the CV (Coeffcent of Varaton s computed to reflect the orentaton s dstrbuton. V = ( m j= 0 V, j / m (b (c Fgure 8. The contrasts of the stroke drectons Fgure 8 (a s orgnal mage, and (b s the result n [Mao01a]. The pcture (c s our result. Obvously, Full Papers 84 ISBN

7 our result not only reflects the detals n some areas but also embody the style of the pencl drawng. 5. Experment results We have mplemented all the technques descrbed above and bult an automatc pencl drawng generatng system on Wndows envronment. Bascally, after the nput mage s specfed, a pencl drawng style mage wll be generated automatcally wth a set of default value and other nformaton derved by the system from the nput mage. Users are also allowed to nteractvely specfy some parameters, such as the parameters for controllng the regon tone of the output mage, the threshold used for graph-based mage segmentaton and the length of convoluton. Comparng to the exstng method n [Mao01a], the new algorthm s much faster than the exstng one and more smlar wth the artstc style of the pencl drawng. The exstng LIC pencl flter took about 60 seconds to generate an mage of sze 350*80 on a Pentum IV PC. Our method only requre about 5 seconds. For an mage of sze 104*768, our method takes about 0 second, and the more mportant feature contrastng to the method n [Mao01a] s that our method dynamcally analyzes the texture structure of the regons. So t s effectve and lfelke. Several results of the pencl drawng as shown n fgure 9. Fgure 9. Several results of the pencl drawng 6. Concluson We propose n ths paper to mprove the exsted LIC-based method. Frst, a more accurate and rapd graph-based mage segmentaton method s ntroduced to dvde the mage nto dfferent regons. Second, we present a new regon-based way to produce whte noses and texture drectons. The new algorthm s much faster than the exstng one and more smlar wth the artstc style of the pencl drawng. We beleve the purpose of NPR systems s not to replace artsts, but rather to provde a tool for users wth no tranng n a partcular medum. Potental applcaton felds of our technque nclude producng posters from photo graphs, processng vdeos nto pencl drawng style anmatons, obtanng the preparatory sketches for creatng pantngs of other styles, and so on. Full Papers 85 ISBN

8 The major future research drectons nclude the realzaton of abstracton and focusng, establshment of the curved strokes, extenson to colored pencl drawng and applcaton to non-photorealstc renderng of 3D models and scenes. 7. Acknowledgements The work reported n ths artcle s supported by the Natonal Natural Scence Foundaton of Chna (No , No References Pacfc Graphcs'99 conference proceedngs, page 50-58, [Tam84] H. Tamura Edted, Introducton to Computer Image Processng (In Japanese, Soken Shuppan Publsher Inc. 1984, ISBN [Ver89a] A. H. Vermeulen and P. P. Tanner, Pencl sketch A Pencl-Based Pant System, Graphcs Interface '8.,1989 [Wn94a] G. Wnkerbach, D. H. Salesn, Computer- Generated Pen-and-Ink Illustraton, SIGGRAPH94 conference proceedngs, pages , [Wn96a] G. Wnkenbach and D. H, Salesn, Renderng Parametrc Surfaces n Pen and Ink, SIGGRAPH96 conference proceedngs, pages , [Cab93a] B. Cabral and C. Leedom, Imagng Vector Feld Usng Lne Integral Convoluton, SIGGRAPH93 conference Proceedng, pages 63-70, [Cur97a] C. J. Curts, S. E. Anderson, J. E. Sems, Kurt W. Flescher, and Davd H. Salesn, Computer-Generated Watercolor, SIGGRAPH 97 conference proceedngs, pages , [Fel04a] P.F. Felzenszwalb, D.P. Huttenlocher Effcent Graph-Based Image Degmentaton Internatonal Journal of Computer Vson[J], 004 [Goo01a] Gooch, Non-Photorealstc Renderng, AK Peters Ltd, 001. [Her98a] A. Hertzmann, Panterly Renderng wth Curved Brush Strokes of Multple Szes, SIGGRAPH 98 conference proceedngs, pages , [Lt97a] P. Ltwnowcz, Processng Images and Vdeo for An Impressonst Effect, SIGGRAPH97 conference proceedngs, pp , [Mao01a] X. Mao, Y. Nagasaka and A. Imamya", Automatc Generaton of Pencl Drawng from D Images Usng Lne Integral Convoluton", Proceedngs of the Seventh Internatonal Conference on Computer Aded Desgn and Computer Graphcs CAD/GRAPHICS001, PP , 001. conference proceedngs, pages , [Sal94a] M.P. Salsbury, S. E. Anderson, R. Barzel, and D. H. Salesn, Interactve Pen And Ink Illustraton, SIGGRAPH 94 conference proceedngs, pages , [Sal97a] M. P. Salsbury, M. T. Wong, J. F. Hughes, and D. H. Salesn, Orentable Textures for Image-Based Penand-Ink Illustraton, SIGGRAPH 97 Conference Proceedngs, pages , [Sou99a] M.C.Sousa and J.W.Buchanan, Observatonal Model of Blenders and Erasers n Computer-Generated Pencl Renderng, Graphcs Interface '99 conference proceedngs, pages , [Sou99b] M.C.Sousa and J.W.Buchanan, Computer- Generated Graphte Pencl Renderng of 3D Polygonal Models,EUROGRAPHICS '99 conference proceedngs, pages , [Tak99a] S. Takag, I. Fujshro and M. Nakajma, Volumetrc modelng of colored pencl drawng, Full Papers 86 ISBN