Appl. Math. Inf. Sc. 6-3S, 769-775 (2012) 769 Realstc and Detal Renderng of llage rtual Scene Based on Pxel Offset Chunjang Zhao, Huaru Wu and Ronghua Gao Natonal Engneerng Research Center for Informaton Technology n Agrculture, Bejng 100097, Chna Key Laboratory for Informaton Technologes n Agrculture, Mnstry of Agrculture, Bejng 100097, Chna Emal: haocj@nercta.org.cn Receved: 21 Feb. 2012; Revsed 16 May 2012; Accepted 03 Jul. 2012 ceved: May 8, 2012; Revsed May 10, 2012; Accepted May 20, 2012 Abstract: Many complex detal objects n the vllage vrtual scene are mportant to the photo-realstc renderng technology. Bump mappng, normal mappng and parallax mappng have a capablty to represent the realstc renderng by dsturbng normal vector or shftng the texture coordnates along the vew drecton. However, an ncorrect offset wll lead to the sldng phenomenon n the detal renderng. In ths paper, a correspondng relaton between the offset and the current vewpont s proposed by establshng mathematcs functon of the detal mage pxel coordnates and the heght values. Furthermore, the new pxel offset s calculated by usng GPU technology. Wth GPU technology, sldng phenomenon does not appear when the vewpont changes. Thus, the detal objects have the realstc effect n vllage vrtual scene. The expermental results demonstrate that ths proposed method has acheved better performance n representng the realstc and detal renderng than tradtonal methods of parallax mappng and texture coordnate offset correcton wth GPU technology. Keywords: Realstc, bump mappng, parallax mappng, pxel offset, vrtual scene 1 Introducton It s an mportant research branch n vrtual realty to make the vrtual objects photo-realstc. There are two man approaches to render detals of a scene: geometry-based renderng and mage-based renderng. The former can generate detals mages from dfferent vewponts wth 3-D data sets, such as polygon meshes and ther surface propertes. However a complex scene may nvolve large data sets that can be prohbtve to process n real-tme. The latter renders detals from dfferent vewponts by nterpolatng between photographc mages, because renderng tme does not depend on the scene s complexty, so t s easy to generate photorealstc output. However ths approach tends the vewpont and scene structure. Further, ts calculatons tend to be slow because of the lack of graphc hardware acceleraton. Texture mappng technology[1-2] can descrbe object detals and reduce the complexty of the vrtual scene when the object structure s complex. However, texture mappng technology just consders surface color, thus t can descrbe varous decoratve pattern desgns only on a smooth surface, but can not present rough detal nformaton brought about by bumpy mcro geometry shape. When the vewpont s close to the observaton object, the Appled Mathematcs & Informaton Scences An Internatonal Journal @ 2012 NSP Natural Scences Publshng Cor. demand for the model accuracy s hgher, and the object detal nformaton descrbed by texture mappng technology cannot satsfy user s demand of vsual experence. Of partcular note, an mage mapped onto object surfaces does not have coarseness detal effect wthn a surface. Ths problem s magnfed when nspectng a texturemapped surface wth stereoscopc vson, because lack of bnocular dsparty between each mapped polygon s a depth cue that the surface t represents s flat. To solve ths problem, the technology of bump mappng s proposed. It can smulate surface bumpy detal nformaton wthout any overload of ncreasng extra polygons, and make the vrtual scene wth more realstc. But t lacks the moton parallax effect-an apparent effect of relatve movement of the surface detal due to a vewpont. Parallax mappng[3] s a technology that can reale the 3D renderng on dynamc vewpont of mage detals, and t can reale 3D parallax effect of mage surface detals through the texture coordnate offset. However, ncorrect texture coordnate offset wll lead to a sldng effect n the result, then reducton n realsm. In order to cover ths shortage of the parallax mappng technology, Donnelly[4] descrbes a ray tracer that uses 3D textures and a sphere tracng data structure, whch
770 Chunjang Zhao, et al: Realstc Detal Renderng of llages rtual Scene. avods the applcatons of parallax mappng, and reales the 3D parallax effect of mage surface detals. But ts effcency s not hgh due to the ray tracng calculaton. Tatarchuk[5], Polcarpo[6] and some others descrbe an mproved method based on tradtonal normal vector. It avods the sldng effect of parallax mappng, but the change of parallax s not so bg and the 3D effect s not obvous. Along wth the development of computer hardware, 3D renderng of mage detals under dynamc vewpont develops further. Steep parallax mappng technology[7]can calculate out the selfprojecton of mage detal nformaton n the Pxel Shader and enhance the 3D dsplay effect of the mage detals wthout the use of geometrc data. Nevertheless, Steep parallax mappng technology costs much. It needs to use 20 or more layers to present the hgh qualty dsplay effect. Ths wll waste a lot of tme. It s more mportant that t takes up a lot of executon tme of Pxel Shader, and ths knd of Shader s 20 tmes slower than Normal Mappng. Instead of tradtonal vertex transformaton and lghtng equaton to correct the devaton of texture coordnates[8], the custom program can use parameters to adjust and fx offset se, and draw t as near as possble to the correct value quckly, n order to allevate texture sldng effect. Many objects wth detal nformaton n the vllages vrtual scene, such as art relef of custom buldng, scraggy wall, and so on. In order to solve the detals renderng under the premse of not takng up too much tme of system, a mappng functon of mage pxels coordnates and heght values s defned n ths paper, whch can get a correct vrtual pxel offset related to vewpont through the pxel to pxel shft for detals mage. The expermental results demonstrate that the detal renderng method based on pxels offset does not ncrease algorthm complexty, and when the vewpont draws near to the object surface gradually, surface detals of object can present a 3D effect wth no sldng phenomenon. 2 Method of Detal Renderng 2.1 Bump mappng Bump mappng as a technque that makes a surface appear rough or wrnkled. Ths affect s acheved by perturbng the normal used n the llumnaton computaton. Hence, t only affects the shadng, not the underlyng geometry. The bump map can contan heght values that wll affect the normal of each pxel. However, t can as an alternatve contan normal that have been computed from such a bump map and ths map s then called a normal map[9]. Especally movng frame bump mappng[10]s closely related to shadng snce t ncludes vector nterpolaton, whch s the essence of Phong shadng. Bump mappng s also closely related to texture mappng[11]. It wll also suffer from smlar alasng problems when bumps are magnfed or mnfed. Blnn used an approxmaton for the perturbed normal, whch depends on the surface normal and the heght nformaton n the bump map. The perturbed normal s calculated as ' Fu ( n Pv ) Fv ( n Pu ) n n n Where n s the surface normal, P u and Pv are the partal dervatves of the surface n the u and v drectons respectvely. Fu and Fv are the gradents of the bump map. Bump mappng combnes per-fragment lghtng wth surface normal perturbatons suppled by an mage, n order to smulate detal nformaton on object surfaces. Ths effect s acheved wthout requrng excessve geometrc tessellaton of the surface. Wth bump mappng, the detaled surface features that nfluence an object s lt appearance can be captured n an mage nstead of actually ncreasng the object s geometrc complexty. A bump-mapped scene has more geometry and surface rchness than s actually present. 2.2 Dsplacement mappng Although standard bump mappng offers a relatvely nexpensve way to add surface detal, there are several downsdes to ths technque. Bump mappng approaches lack the ablty to represent vew-dependent unevenness of detaled surfaces, and therefore fal to represent moton parallax the apparent dsplacement of the object due to vewpont change, and today s some graphcs hardware can compute t n real-tme[12]. In recent years, new approaches for smulatng dsplacement on surfaces have been ntroduced. Dsplacement mappng was frst mentoned by Cook[13] as a technque for addng surface detal to objects n a smlar manner to texture mappng. It s
Chunjang Zhao, et al: Realstc Detal Renderng of llages rtual Scene. 771 a powerful technque for addng detal to three dmensonal objects and scenes. Although t can modfy the slhouette and reale parallax effects, t addresses the ssues above by actually modfyng the underlyng surface geometry. So t needs many polygons to generate a transformaton through heght map data. It s a knd of modelng technque that s unsutable for real-tme renderng. To apply a detal mage as a dsplacement map on a subdvson surface, the geometrc elements are mapped the mage doman. There are several mappng technques performng such a task, the smplest planar mappng technque s adopted, whch s a functon defned as follows. F : R R 3 2 F( v ) N p N s a 33matrx project the vertex p to the mage plane. The typcal choces of N are as follows. 1 0 0 1 0 0 0 0 0 Nxy 0 1 0, N 0 0 0 x, N y 0 1 0 0 0 0 0 0 1 0 0 1 Strctly speakng the functon F does produce a 3 vector n R. The mage plane concdent to one of the three prncple plane ( xy, x, y ), n whch the projected vertces resde. The projecton of vertces n R 2 wll be normale to e(1,1) square doman, snce detal mage s algned wth the square doman. The desgner s free to choose n whch prncple plane the mage should resde. The regon of nterest s also nteractvely selected by the desgner. Therefore, approprate mage pxel coordnates can be nteractvely assgned to each trangle.the generaton of the subdvson surface s acheved by teratvely applyng a set of refnement rules, each teraton new vertces are nserted to the surface. 2.3 Parallax mappng Parallax mappng s mplemented by dsplacng the pxel coordnates at a pont on the rendered polygon by a functon of the vew angle n tangent space and the value of the heght map at that pont. At steeper vew-angles, the pxel coordnates are dsplaced more, gvng the lluson of depth due to parallax effects as the vew changes. Wthout changng the geometry property of the object surface, the mage 3D lluson of object surface can be drawn by the translaton of the pxel coordnates along vew drecton[14].now, assumng that a model wth geometrcal detals s on the upper part of the object. Its secton can be descrbed wth a curve. The observaton vector s, f the object surface s observed along the observaton vector = (,, ), the pxel coordnates ( uv, ) would be x y seen on the detal surface, whle the pont correspondng to the practcal geometrc detal model s ( u, v ). To get the correct pxel coordnates, t s needed to shft the coordnates of each pxel of the mage a certan dstance along vew drecton n order to reale parallax effect when the vewpont shfts on the object surface. That s just shftng pont ( uv, ) to ( u, v ) along the vew drecton. (, ) (, ) u v u v offset ( uv, ) s the actual poston of the vewpont, and ( u, v ), the corrected pxel coordnate of the actual pxel coordnate, s correspondng to the geometrc detal model of the vewpont, and offset s the offset of pxel coordnate. Retreve some mages wth the corrected coordnates, and the mage wll show the 3D parallax effect when the sght moves relatve to the uneven surface. 3 Detal Renderng Based on Pxel Offset 3.1 Pxel offset calculaton To calculate the pxel offset offset of pont P on the current vewpont, the frst step s to normale the eye vector. Heght value of the mage pxel coordnate ( uv, ) can be read out from the nput heght map, and then assume that the heght value of pont ( uv, ) s set as h. As the offset along the vew drecton, so offset drecton s determned by the sght drecton, and the se of the offset s determned by the heght value correspondng to ( uv, ), so the pxel offset of pont P s defned as. offset h Where s the projecton of the eye vector on to the axs n on the cuttng plane space n,, t b, namely the n -component of, and vector s the composte vector of the t -component and the b - component of the eye vector, as shown n Fg.3.1.
772 Chunjang Zhao, et al: Realstc Detal Renderng of llages rtual Scene. Fgure. 3.1. Eye vector composton And then get the new mage pxel coordnate u v as: (, ) ( u, v ) ( u, v) h Pxel offset calculaton above s processed under the premse of assumng that pont ( u, v ) and pont ( uv, ) have the same heght value, but ths case s not usually seen n realty. When the object surface heght values are not completely approxmately equal, mage pxel coordnate offset from the calculaton wll have a lttle dfference wth the actual stuaton, whch wll lead to a dstorton of the detals of the rendered object. Amng at the defcency of parallax mappng, some mprovement method are proposed: as to the local computer common prncple, when the dstance between two ponts on the surface of an object were close, the heght values of the two n the heght map would not be too dfferent wth each other, so the heghts can be consdered approxmately equal f lmted n a certan dstance range. Therefore, for the calculaton of the mage pxel coordnate offset, f offset s lmted n a certan range, the heght values of the ponts n ths range would be approxmately equal. To calculate the approprate lmts, the frst step s to halve the value of offset, namely. 1 offset h 2 Although the drawng effect has been mproved wth ths method, t s stll unsatsfactory. Moreover, multplyng 1 also ncreases a certan amount of 2 calculaton. Then the expresson was smplfed as follows: offset h x, y However, when the vewng angle s sharp, the offset dstance s small, whch means that the heghts of pont ( u, v ) and pont ( uv, ) are very close. When the vewng angle becomes very large, the pxel offset wll tend to nfnty, whch wll lead to that the offset s bgger than the actual offset value, then the fnal detal renderng results wll not be correct. 3.2 Pxel offset translaton Incorrect pxel offset wll lead to that the 3D effect obtaned dose not meet the actual geometrc detal model, and could even cause sldng phenomenon. In order to get the correct pxel coordnate offset n dfferent vewng angles, as shown n Fg.3.2, the method of pxel offset translaton s proposed. Fgure.3.2 Pxel offset translaton In the graph s the eye vector, and ( u, v) s the actual mage pxel coordnate, and N s the normal vector of the smooth surface mappng the detaled sense, and ( u d, v d ) s a new pxel coordnates after translaton. When the surface heghts of the actual geometrc detal model are not equal, n Fg.3.2 the dstance between normal vector N and the ntersecton pont of the eye vector and the geometrc detal model s the correct offset. 3.2.1 Mappng between pxel coordnate and heght value To get a true offset translaton, mappng relatonshp should be establshed frst, whch s between pxel coordnates and the heght values. Assume that the ses of the mage map and the heght map are M N, both stored n chan tables. The data structure s as Fg.3.3.
Chunjang Zhao, et al: Realstc Detal Renderng of llages rtual Scene. 773 eye vector on the plane and the axs. The expresson s as follows. Fgure.3.3 Data structure of mappng Where, P0, P1 PM N 2, PM N 1 s the coordnates chan table, P ( u, v ), 0,1, M N 1 and H0, H1 HM N 2, HM N 1 s the heght chan table,where H h 0,1, M N 1,and the mappng functon s f, whch defnes the mappng relatonshp of the pxel coordnate P and the heght value H, f ( u, v ) h 0,1, M N 1 Each pxel coordnate s correspondng to a heght value, as shown n Fg.3.4, whch shows the mappng functon of coordnates and the heght values, and new pxel coordnates P ( u, v ), 0,1, M N 1 are correspondng to the heght values ( u0, v0) ( u1, v1) ( u, v ) 2 2 ( u, v ) M N 2 M N 2 ( u, v ) M N 1 M N 1 H h 0,1, M N 1.... Pxel coordnates lnked lst f ( u, v ) h 0 0 0... f ( u, v ) h 0,1, M N 1... h 0 h 1 h 2 hm N 1 hm N 2 Heght lnked lst Fgure3.4. Pxel coordnates and heght lnked lst 3.2.2 Translaton value calculaton Before calculatng translaton, f mappng functon of the pxel coordnates and the heght values should be defned frst. Now assumng that the eye vector s and ts angle wth the object surface s, as shown n Fg.3.5, and the cotangent value of s equal to the rato of ector and ector, whch are the component vectors of the Fgure.3.5 Translaton value calculaton cot On the current vewpont, the rato of the correct coordnate offset and the correspondng heght value after the offset s offset translaton cot H d Thus, the relatonshp between the new pxel coordnate offset and the current vewpont coordnates s offset translaton H Therefore, the current pxel coordnates T moves along the eye vector pxel to pxel, and accordng to the functon f, the heght H 1 of the new pxel coordnate P 1 can be obtaned. Move the coordnate pxel to pxel, and then the rato of the number of the pxels of the coordnate offset and the heght of the mage coordnates can be obtaned, When the equaton d H s true, the ordnate wll no longer move, and then d offset translaton s the correct pxel offset of detal renderng. 4 Expermental Results In order to verfy the realstc result wth ths detal renderng method based on pxel offset, n ths paper, some flower relef of vllages buldng and wall of cvl house are renderng uses CG language, on an ordnary PC, transfers data from..
774 Chunjang Zhao, et al: Realstc Detal Renderng of llages rtual Scene. Pxel Shader to Fragment Shader, and threedmensonal realstc renderng of detal objects n the vllages are realed. Expermental results as Fg. 4.1. (a) Flower relef renderng based on pxel offset (b) Wall renderng based on pxel offset Fgure4.1Renderng results based on pxel offset Can be seen from Fg.4.1, there wll be a knd of realstc effect created as the vewpont moves. And some detals wll show dfferent effects as the mage pxel coordnate offsets are dffere. For the complexty of the algorthm, compare the tme costs of dfferent methods renderng brck surface detal mage and wrought detal mage n dfferent resoluton of 128 128, 256 256, 512 512 and 1024 1024, as shown n Fg.4.2. (a) Brck surface renderng tme (b) Wrought renderng tme Fgure.4.2 Renderng tme contrastaton of dfferent methods Dvson n the detal renderng s omtted n paper[14], and t costs the shortest tme n the same resoluton. Yet, to ensure the correctness of the mage pxel coordnate offset, a dot product method[15] s used, so t costs the longest tme. Ths paper concludes a method, whch conssts of a dvson and a pxel translaton, to obtan the correct mage coordnate offset and the realstc detal object, by shftng the mage coordnates pxel to pxel. The shftng won t ncrease the algorthm complexty, so the algorthm complexty only relates to the dvson. Therefore, the calculaton of the pxel coordnates by per-pxel shftng won t ncrease the whole complexty of algorthm. Meanwhle, the object surface detals are formed by some tny ups and downs changes, and the step of the per-pxel shftng s short, so the mproved method of parallax mappng wth pxel to pxel shft for mage coordnates does not decrease the effcency of mage detal drawng.[16]. 6 Concluson A lot of detal objects n the vrtual vllage renderng, bump mappng, normal mappng and parallax mappng can produce 3D effect,but offset wll cause detal nformaton sldng dstortons. When the vewpont moves to make the angle wth plane vary small, the stuaton wll be very sharp, and the effect wll be unacceptable. Therefore calculaton pxel coordnate offset s a very mportant problem. In ths paper, a one-one mappng functon of mage coordnate and heght s defned, and a detal object renderng method based on pxel offset s proposed. Movng the pxel coordnates to the eye vector n pxels, when the dstance and the current vewpont coordnates meet
Chunjang Zhao, et al: Realstc Detal Renderng of llages rtual Scene. 775 the gven equaton, the rght coordnate offset s obtaned. Ths method can get the true coordnate offset, and correct the dstorton deformaton phenomenon, meantme, pxel shftng won t ncrease the algorthm complexty. Therefore, t enhances the realstc effect of detal object renderng wth no ncrease n algorthm complexty. Acknowledgements Ths work was supported by the by the Natonal Natural Scence Foundaton of Chna (60871042, 61102126), Bejng Muncpal Natural Scence Foundaton (54122034), Natonal Technology R&D Program(2010ZX01045-001-004,2011BAD21B02). References [1] H. X. Ren, Y. Y. Yn, Y. C. Jn, Realstc renderng of large-scale ocean wave scene. Journal of Computeraded Desgn & Computer Graphcs, 20, 1617-1622 (2008). [2] G.Francos, S.Pattanak, K.Bouatouch, et al. Subsurface texture mappng. Computer graphcs and applcatons, 28, 34-42(2008). [3] T. Kaneko, T. Takahe, M. Inam, et al. Detaled shape representaton wth parallax mappng. Proceedngs of the 11th Internatonal Conference on Artfcal Realty and Telexstence, Tokyo, Japan, 205-208(2001). [4] P. Matt, F.Randma. GPU Gems 2: Programmng technques for hgh performance graphcs and generalpurpose computaton. Boston: Addson-Wesley, 123-136,(2005). [5] N.Tatarchuk. Dynamc Parallax occluson mappng wth approxmate soft shadows. Computer Graphcs Proceedngs, Annual Conference Seres, ACM SIGGRAPH Symposum on Interactve 3D Graphcs and Games, Redwood, 63-69 (2006). [6] F.Polcarpo, M.M.Olvera,J.L.D.Comba.Real-Tme relef mappng on arbtrary polygonal surfaces. Computer Graphcs Proceedngs, Annual Conference Seres, ACM SIGGRAPH Symposum on Interactve 3D Graphcs and Games, Washngton, 155-162 (2005). [7] M.Morgan, M.Max. Steep parallax mappng. Computer Graphcs Proceedngs, Annual Conference Seres, ACM SIGGRAPH Symposum on Interactve 3D Graphcs and Games, Washngton, Poster (2005). [8] M.J.Klgard. A Practcal and Robust Bump-mappng Technque for Todays GPUs Game Developers Conference, Advanced OpenGL Game Development. (2000). [9] M. Peercy, A. Arey, B.Cabral. Effcent Bump Mappng Hardware. Proceedngs of SIGGRAPH, 303-306 (1997). [10] P. S. Heckbert. Survey of Texture Mappng. IEEE Computer Graphcs and Applcatons, 56-67 (1986). [11] M. Bóo et al. Hardware Support for Adaptve Subdvson Surface Renderng. Proceedngs of Sggraph/Eurographcs Workshop on Graphcs Hardware, ACM Press, 33-40 (2001). [12] A. Schllng. Towards Real-Tme Photorealstc Renderng: Challenges and Solutons. Proceedngs of Sggraph/Eurographcs Workshop on Graphcs Hardware, ACM Press, 7-15 (1997). [13] R. L. Cook. Shade Trees. Proceedngs of Sggraph, ACM Press, 223-231 (1984). [14] T. Welsh. Parallax mappng wth offset lmtng: a perpxel approxmaton of uneven surfaces. Techncal report, Infuscate Corporaton, (2004). [15] R. H. Gao, B. C. Yn, D. H. Kong, et al. An mproved method of parallax mappng. Proceedngs of IEEE 8th Internatonal Conference on Computer and Informaton Technology, Sydney, 7,30-34 (2008). ------------------------------------------------------------- Chunjang Zhao s a professor at Natonal Engneerng Research Center for Informaton Technology n Agrculture. He receved the Ph.D. degree n agronomy from Chna Agrcultural Unversty, Bejng, Chna, n 1991. He s currently a Senor Scentst wth and the Drector of the Natonal Engneerng Research Center for Informaton Technology n Agrculture, Bejng. He s also a Member of the Scence and Technology Commsson, Mnstry of Agrculture, Chna. Hs research nterests nclude precson farmng, ntellgent nformaton technology n agrculture, and crop decson support system.dr.zhao receved the Excellent Scentst Award by Mnstry of Scence and Technology of Chna n 2001. Huaru Wu s a professor at Natonal Engneerng Research Center for Informaton Technology n Agrculture. He s nterested n studyng Artfcal Intellgence. In recent years, he has partcpated n 18 natonal and provncal key scentfc research projects, and publshed over 20 academc papers. He got the frst pre of Bejng Scence and Technology n 2005, and thrd pre of agrcultural technology promoton n 2003. Unversty. Ronghua Gao s an Assstant Researcher at Natonal Engneerng Research Center for Informaton Technology n Agrculture. She s nterested n developng smple and practcal algorthms for computer graphcs. She has a PhD n computer scence from the Bejng Technology