Rail-Track Viewer An Image-Based Virtual Walkthrough System

Transcription

1 Eghth Eurographcs Workshop on rtual Envronments (00) S. Müller, W. Stürzlnger (Edtors) Ral-Track ewer An Image-Based rtual Walkthrough System Lnng Yang, Roger Crawfs Department of Computer and Informaton Scence The Oho State Unversty Abstract Complex renderngs of synthetc scenes or vrtual envronments, once deemed mpossble for consumer renderng, are becomng avalable as tools for young artsts. These renderngs, due to ther hghqualty mage synthess, can take mnutes to hours to render. Nowadays, as the computng power has ncreased dramatcally, the sze and complexty of the datasets generated by the super-computer can be overwhelmng. It s almost mpossble for the vsualzaton technques to acheve nteractve frame rates. Our work focuses on usng Image-Based Renderng (IBR) technques to manage and explore large and complex datasets and vrtual scenes on a remote dsplay across the world-wde-web. The key dea for ths research s to pre-process the datasets and render key vewponts on pre-selected paths nsde the dataset. We present new technques to reconstruct approxmatons to any vew along the path, whch allows the user to roam around nsde the datasets wth nteractve frame rates. We have mplemented the ppelne for generatng the sampled key vewponts and reconstructng panoramcbased IBR models. Our mplementaton ncludes an effcent two-phase cachng and pre-fetchng scheme. The system has been successfully tested on several datasets and satsfyng results have been obtaned. Analyss of errors s also presented. Keywords: Image-based Renderng, rtual Walkthrough, rtual Envronment, sblty, Cachng. Introducton Complex renderngs of synthetc scenes or vrtual envronment, due to ther hgh-qualty mage synthess, can take mnutes to hours to render. Ray-tracng or global llumnaton usng a tool such as PORAY [0] that can render hgh-qualty mages however s very tme consumng. Nowadays, as the computng power ncreases dramatcally, the sze and complexty of the datasets generated can be overwhelmng. Buldng and renderng the geometry of these large datasets are also tme consumng. An nteractve vrtual walkthrough of these large and complex scenes s almost mpossble on a low to md-end system usng tradtonal renderng technques. Our goal s to determne a soluton for allowng the user to examne and walkthrough the scene from an nternal vantage pont on a relatvely hgh-resoluton dsplay. To acheve ths goal, we decded to apply Image-Based Renderng (IBR) technques as a post-processng tool for any tradtonal renderer. {yangl, crawfs}@cs.oho-state.edu 395 Dreese Lab, 05 Nel Ave., Columbus, OH 430, USA IBR s a new research area n both the computer graphcs and vsualzaton communty, and offers advantages over the tradtonal renderng technques. It does not requre buldng or modelng complcated geometrc models. It can utlze real lfe mages and llumnaton for photo-realstc renderng. Fnally, IBR requres a fxed or lmted amount of work, regardless of the vew or data context. However, ths amount of work s fxed to the desred output. As a result, many IBR technques [9] [0] [] [] focus on accurate renderng of relatvely lowresoluton mageres. Here we explore technques for large dsplays havng a resoluton from kxk to 8Kx3K as n our new vdeo wall. Our work can be vewed as an extenson to QuckTme R [3] or other panoramc representatons [9]. Panoramc magery allows one to spn around at ther current vewng poston, but does not allow the user to move forward or backward. We developed a system to allow movement along a lnear path n threedmensons. At any poston on ths curve, the user can nteract wth the scene as n a panoramc vewer. We termed ths type of vewng, a raltrack vew, n that the user can move forward and backward along the track, whle vewng the outsde scenery n any drecton. Darsa, et al [6] nvestgated technques to ncorporate The Eurographcs Assocaton 00

2 Yang / Crawfs: Ral-Track ewer An Image-Based rtual Walkthrough System nformaton about the depth of the mage nto the panoramc scene. Depth allows for proper reprojecton of the pxels from dfferent vewponts and provdes a sense of moton parallax to gve a true three-dmensonal sense to the magery. For effcent renderng, we apply a mesh smplfcaton method to smplfy the depth mage from every pxel to a more manageable geometrc quad-mesh. An ntellgent cachng and pre-fetchng scheme s employed to further mprove the renderng speed. We also present and analyze the sources of the errors n our system. The paper s organzed as follows: Frst we dscuss relevant background and prevous work n the IBR area. We then present an overvew followed by mplementaton detals of our system. Next we dscuss the pre-processng and data organzaton scheme for effcent renderng. A two-phase cachng and pre-fetchng technque s also presented n ths secton. We wll then dscuss the sources and a quanttatve measurement of the errors. Fnally we gve some test results and conclude wth future work.. Related Work Recently, a lot of effort has been put nto Image-Based Renderng systems. Image-based renderng has the property of a bounded computaton accordng to the nput and output mage sze. Ths s advantageous over tradtonal polygonal based renderng where complex scenes or models can have ndvdual polygons smaller than a pxel and therefore constructng and renderng these can be unbounded complcated.. Plenoptc Functon IBR systems are based on the Plenoptc functon. The 7D plenoptc [] functon s defned as the ntensty of lght rays passng through the camera center at every locaton defned by x, y, z, at every possble vewng angle θ and φ, for every wavelength λ and at any tme t. µ = Plenoptc( θ, φ, λ, x, y, z, t) () Even wth faster CPU s and more memory, ths functon overwhelms modern archtecture, makng t mpractcal for nteractve applcatons. In order to make practcal use of Image based renderng concepts, ths model has to be smplfed. McMllan [] et al smplfed the model to a 5D plenoptc functon (equaton ) by fxng tme and breakng up the wavelength as RGB components. µ λ = µ ( θ, φ, ) (), t Note that s a vector representaton of x, y, z. McMllan s mage warpng system s based on ths 5D plenoptc functon. If the user movement s restrcted to le outsde of a box, the model can be further smplfed to a 4D functon such as Lumgraph [9] or Lghtfeld [0]. A 3D representaton of the plenoptc functon s constructed n Concentrc Mosacs [8] as they restrct the user s movement to le wthn a D crcle. QuckTme R systems [3] [9] reduce the functon to a D one by lettng the user stand at a fxed pont and look around. Ths research focuses on allowng the user to move on a preselected path and look around. Hence t s modelng a 3D slce through the plenoptc functon. Havng a contnuous plenoptc functon, we need to sample and dscretze t. Equaton 3 s a dscretzed verson of the 5D plenoptc functon. µ = µ ( θ, φ, ) (3) λ, t Obvously we can sample the vewng drecton and vews. IBR systems can vary n terms of how and when to choose vew samples. Some systems choose to sample the vewponts outsde of the scene [4] [7] whle others put ther sample vewponts nsde the scene, on a path [] [6], nsde a crcle [7], or at a fxed pont [3]. Systems may choose the reference vews a pror, pre-renderng the sampled plenoptc functon. Whle other systems [6] choose vew samples on the fly and reconstruct mages of new vews from these untl the mage qualty degrades to where new reference vews are needed.. Depth Functon For opaque scenes, the depth functon ξ s of value for reconstructng the novel vews (.e. vews not located at the sample ponts). Equaton 4 descrbes the dstance to the closest object n the scene. Samplng and dscretzng ths functon gves us ξ. Equaton 3 and 5 provde a framework for IBR representaton of a scene. ξ = ξ ( θ, φ, ) (4) ξ = ξ ( θ, φ, ) (5) IBR systems can also dffer n how they represent ths depth functon (geometrc nformaton). QuckTme R [3] and sphercal panoramc systems [9] do not store any geometrc nformaton. The mages for new vew-ponts are reconstructed usng mplct geometrc relatonshps. Ths s adequate for scenes far away from the user at where the user s only allowed to look around or zoom. The Eurographcs Assocaton 00

3 Yang / Crawfs: Ral-Track ewer An Image-Based rtual Walkthrough System.3 Occluson and Ds-occluson 3D warpng systems [6] [] [] store depth nformaton per pxel. Users can move away from the pre-selected vewponts and the depth and color nformaton s used to construct novel vewponts. The occluson and dsoccluson problems are not addressed by the 3D warpng system, because t only has one layer of the depth nformaton assocated wth each pxel. As the user moves farther away, holes wll appear and nformaton for more than one reference vewpont s needed to fll these holes. Layered Depth Images (LDI) [7] [] store several layers of depth and color nformaton for each pxel. When renderng from an LDI, the user can move farther away and expose surfaces that were not vsble n the frst layer. The prevously occluded nformaton can be rendered usng nformaton from later layers. The model s neffcent when we have complcated scenes or when the output mage has extremely hgh resolutons. Other systems partton the object space nto several slabs along the vewng drecton [8] [3]. The data wthn a slab s projected towards the mage plane and then texture mapped onto a quad orented parallel wth the mage plane and perturbed by the z- values of ts correspondng slab. When reconstructed for a novel vew, ths can approxmate an mage warp. Most of the IBR systems concentrate on accurate renderng of relatve low-res mageres. Systems based on 3D warpng [][][][7] use per-pxel operaton and do not utlze much hardware acceleraton. They are not sutable for nteractve walkthroughs on very hghresoluton dsplays. Lumgraph [9] and Lghtfeld Renderngs [0] need very dense samplngs, whch s both tme consumng and space neffcent. ew dependent texture mappng (DTM) [7] s a typcal example utlzng texture hardware to accelerate renderng. However, for one reference vewpont, the complete vewng drecton s not adequately sampled and therefore can not allow head rotaton durng the fly-through. Cohen-Or et al [5] also looked nto ways to pre-compute the geometry and textures on a path and stream the results across the network. They use projectve texture mappng for close-by vews. Agan, ther work lacks the ablty to allow the user to change the vewng drecton durng the walkthrough because of the ncomplete samplng. Ther method focuses on how to compress the resultng textures and effcently stream them down the network. The closest mplementaton to ours s Darsa et al s [6], n whch they used cubcal envronmental maps wth smplfed trangular meshes. Three blendng methods were explored for smooth walkthroughs between close-by vewponts. Our system dffers from thers n that we used cylndrcal models nstead of cubcal ones. Cylndrcal models have the advantages of representng texture nformaton homogeneously wthout dscontnutes at the edges lke the cubcal maps. We have also adopted a multlayer approach comprsed of several depth meshes to better solve the occluson and dsoccluson problems. We have also nvestgated an ntellgent two-phase cachng and pre-fetchng scheme to mprove the performance of our system. Darsa s system reports a frame rate of 3.53 FPS on an Infnte Realty Engne usng poston weghted blendng wth a 56x56 wndow, whle our system can acheve around 5 FPS usng the same blendng method wth kxk resoluton on a low-cost Sun Blade 000 workstaton wth an Elte 3D graphcs board. Fgure : two possble tracks nto the Nature dataset we rendered usng Povray. The black arrowed curves represent the tracks and red dots represent the preselected vewponts. The Eurographcs Assocaton 00

4 Yang / Crawfs: Ral-Track ewer An Image-Based rtual Walkthrough System Pre renderer and Pre processor PORAY TK RADIANCE AS... Server sde Cachng Pre fetchng Network Clent sde Cachng Pre fetchng IBR Database Reconstructon Interpolaton Usng Java 3D Clent Dsplay Fgure. shows the system dagram. Frst datasets are pre-rendered and pre-processed usng approprate renderng engnes, for example, PORAY, TK, Radance and so on and stored on a server-sde database. The server keeps a cache and retreves from the database and stores nto the cache accordng to the clent demand then transmt the data across the network to the clent. The clent keeps ts own cache and can reconstruct the results usng our layeredpanoramc IBR model. 3. Overvew The goal of ths research s to nteractvely manage complex, tme-consumng renderngs of large datasets. We concentrated our efforts to allow users to roam nteractvely nsde a scene, explorng nterestng features wth the ablty to look around at the same tme. We acheve ths by restrctng the users movement on a pre-selected path and allow them to look around at any pont n both the vertcal and horzontal drectons. Fgure depcts two possble tracks nto the socalled Nature dataset we rendered usng Povray []. The black arrowed curves represent the tracks and red dots represent the pre-selected vewponts. The user s allowed to move back and forth on ths track and change hs vewng drectons freely. Although some software packages allow dscrete jumps from one vewpont to another, ths dsturbng teleportaton removes the user s focus from smooth walkthrough to one of tryng to fgure out where they are and regan ther orentaton or bearng. Fgure descrbes the dea of our IBR system. Frst the datasets are pre-rendered and pre-processed usng approprate renderng engnes, for example, PORAY, TK, Radance and so on and the resultng mageres and geometry nformaton are stored on a server-sde database. The server keeps a cache and retreves the reference vews from the database and stores them nto the cache whenever the clent requests. The cached data s then transmtted across the network to the clent. The clent receves the nformaton and stores t nto ts own cache. Novel vews can now be reconstructed usng our layered-panoramc IBR model, whch wll be descrbed later. 4. sblty Polyhedrons Movng from one vewpont n a complex scene to another wth the freedom of lookng around s a challengng problem. Let s assume that we want to move from to n a complex scene. Consder the followng defntons. Defnton : The vsblty polyhedron of a vewpont s the locus of ponts vsble from the vewpont. The Eurographcs Assocaton 00

5 Yang / Crawfs: Ral-Track ewer An Image-Based rtual Walkthrough System Defnton : A polyhedron P s sad to be star-shaped f there exsts a pont z not external to P such that for all ponts p of P the lne segment zp les entrely wthn P. Defnton 3: The locus of the ponts z s havng the above property s called the kernel of P. Theorem : The vsblty polyhedron of the vewpont s a star-shaped polyhedron and has a kernel that at least contans the vewpont. These defntons are extensons from those on polygons [4]. Let s assume that and le nsde each other s kernels. The vsblty polyhedrons of the two vewponts are therefore dentcal. Any ponts 3 n between the lne segment connectng the two ponts are also nsde that kernel due to defnton. Therefore, the vsblty polyhedron of 3 s the same as or s. We can use the combnaton of these two to represent 3 s as descrbed n equaton 6. P = P t P = P = P (6) 3 The assumpton of two close-by vewponts lyng nsde each other s kernels does not always hold. We can approxmate 3 s vsblty polyhedron from and s vsblty polyhedrons. If and are close enough ths approxmaton s qute reasonable. The samplng rate can be ether user defned, chosen by the database desgner or at fxed ntervals. It can also be controlled by the maxmum mean squared errors allowed for the reconstructed scenes along the track, whch wll be one of the future works. We approxmate the vsblty polyhedron for each vewpont to make the system more effcent, and texture-map t wth the ntal renderng. We then use equaton (6) to combne the close-by reference vews as the user walks between them. In the Next secton we wll address detals of how we mplemented our system. 5. Implementaton Detals In practce, we use a pre-renderer to generate the mageres and the assocated depth nformaton. The depth values are obtaned ether from the z-buffer or the frst ntersecton ponts wth the vewng rays. In ether case, we connect the depth values to form a polyhedron ZP, whch s a dscretzed representaton of the vsblty polyhedron P. We need to smplfy ZP snce renderng each pxel wth ts own depth value s too tme-consumng for hgh-resoluton mageres. Therefore, we reduce the geometrc complexty by down-samplng the depth buffer nto quad-meshes. We call ths smplfed mesh QZP. The reason we chose to use a quad-mesh as our smplfed geometry s because of ts smplcty and resultng effcency ssues for our cachng and pre-fetchng scheme. The vertces of each tle of the quad-mesh have ther actual depth values. Color and opacty are represented by texture maps. We assume a blnear functon for reconstructng the depth values of the nteror ponts of the tles, whch s not always vald. Therefore, ths mesh smplfcaton scheme ntroduces errors, whch wll be talked about n more detals n a later secton. In order to allow for occluded nformaton at a vew to be retaned [7], we dvde the vewng frustum nto several depth ranges, by settng the near and far clppng planes. We call each range a slab QZP. A bnary opacty mask s assgned to enable per-pxel occluson. By compostng the slabs, we acheve the complete scene for one vewpont. dscretze ( P) = ZP QZP = QZP + QZP + QZP (7) Ths wll allow the user to move away from the pre-selected vewponts whle revealng the prevously occluded nformaton usng addtonal slabs. Fgure 3 Shows depth meshes for one vewpont. Fgure 3(a) s the mesh vewed from the orgnal vewpont that provdes a seamless vew. Fgure 3(b) shows a sde vew of the mesh for llustraton purpose. Note the ndvdual slab meshes that comprse the vsblty polyhedron for ths vewpont. We now have the smplfed vsblty polyhedrons, constructed usng the slab meshes for both and wth color as texture maps. How do we smoothly move from one pont to another? To walk between two vewponts we smply combne the two slab-mesh sets. To make the transton smooth, we nterpolate the slab mage sets of two nearby reference vews. The correct way to do the nterpolaton s to use an over operaton on correspondng slabs of the two vews and then compute the result of slab over the result of slab and so on. Ths can be mplemented usng a specal p-buffer, whch s normally not avalable for consumer-level graphcs board. Instead, we compute slab for over slab for and then compute ths over slab of and so on. The nterpolaton requres a weght w based on the dfference (normalzed to be between 0 and ) between the new vew and one of the reference vews. The correspondng reference mage sets have ther opactes modulated by the correspondng weght 3 The Eurographcs Assocaton 00

6 Yang / Crawfs: Ral-Track ewer An Image-Based rtual Walkthrough System factor, and then projected n an nterleaved fashon n depth sort-order from the new vewpont. The equaton for the resultng color after slab mage sets are weghted s: C = C * α * w + ( α * w)* C *( w) * α (8) Here C s the fnal computed color at the new vewpont, C and C are colors of slab mages from the two neghborng reference vews, and α and α are alpha values from opacty maps of the slab mages. Results show that when usng lnear nterpolaton the computed mage looks very close to a correct renderng when the new vewpont s close to a reference vew. However, when the new vew s located at the mddle between reference vews, the computed mage s darker and transparent lookng. The reason for ths s due to the fact that when the new vewpont s near the mddle of the two references, C s less than the orgnal C or C colors, and thus appears more transparent. For example, assumng C equals C and both pxels have α s of, and our weght w equals 0.5. We have a resultng C = 0.5* C + 0.5* C *0.5 = 0.75* C (9) An alternatve nterpolaton s to use a nonlnear weghtng scheme. We defne a non-lnear blendng equaton as: C = C * α * w + ( α * w)* C * (0) ( pow( w, b))* α ) Here b s a constant. Let s assume the same example as before wth b equalng. We have a resultng color of C = 0.5* C + 0.5* C *0.75 = 0.875* C () By usng ths equaton, the dark and transparent problem s allevated. In practce we choose a value of 3 for b 6. Preprocessng We could use the whole rendered mage as a texture map to map to the smplfed depth mesh. However, there are areas of the mage that contan no nformaton. Storng and renderng those s a waste of resource. Therefore, we break the resultng mage nto tles that are correspondng to the depth mesh. The sze of the mage tles and the quad-mesh affects the mage loadng tme, storage, performance and the errors. These wll be addressed n a later secton. Those mage tles contanng no nformaton are detected and labeled as empty tles. Thus we treat each mage tle as a texture map for the correspondng quad. The problem wth renderng each mage tle as a texture map s that the system has lmts on texture bndng unts. Texture-map thrashng results when too many small mage tles are produced, and nteractvty s lost. To allevate ths problem we group ndvdual mage tles nto bgger texture unts for renderng. Consder a full mage of sze W by H, we dvde the mage nto equal szed small tles w x h so that each row has W / w tles and each column has H / h tles. To remove empty tles and merge the remanng nto a larger texture map, we squeeze each column, removng the empty tles, and lnkng the resultng columns of tles nto a one-dmensonal tle array and use ths as the texture unt. A smple ndexng scheme s mplemented to quckly calculate the poston of the columns needed for our cachng and pre-fetchng engne. The IBR renderng engne determnes the closest two reference vewponts on the path. At any gven tme, only the nformaton of at most two reference vewponts are needed to reconstruct novel vews. Therefore, t s reasonable to have only two reference vews n memory at a gven tme. When the users move to a new path segment, requrng a new reference vew, they wll encounter severe latency whle the needed data for reconstructon s read nto memory. Instead of loadng just two sampled vews, the system loads several vews nto memory and stores them nto a cache. The prefetchng engne, whch s mplemented as a separate thread, contnually fetches texture maps as the user moves along the track and stores them nto the cache. The cache s organzed usng a Least Recently Used (LRU) rule. The maxmum number of vews allowed n cache s determned by memory sze, dsc latency and the vewsamplng rate. Our frst experment treats the whole panorama as a cachng unt. It allevated, but dd not elmnated, the latency problem. When the user moved quckly along the track, notceable latency stll occurred snce the dsk readng could not keep up wth the demand. The Eurographcs Assocaton 00

7 Yang / Crawfs: Ral-Track ewer An Image-Based rtual Walkthrough System Fgure 3: Shows depth meshes for one vewpont. (a) The mesh vewed from the orgnal vewpont. (b) The mesh vewed from a vewpont off of the track for llustraton purpose. We can see the slab meshes whch comprse the vsblty polyhedron for that vewpont. Segment Segment (a) (b) (c) Fgure 4 (a) defnes a path wth through 6. A track segment s defned between the two closest-by vewponts. The current novel vewpont s on segment number whch s between and. (b) The entre panorama for and are loaded nto the cache, whle only parts of the panoramas (tles) are loaded n for 3, 4 and 5. And less nformaton s actually loaded for 5 over 3. (c) The user moves nto segment, the novel vew les between and 3, we use another thread to load n the remanng nformaton for 3 and pre-fetch partal nformaton for 6 whch currently has no nformaton n the cache. The Eurographcs Assocaton 00

8 Yang / Crawfs: Ral-Track ewer An Image-Based rtual Walkthrough System Keepng the whole panorama n the memory allows the user to look around n horzontal and vertcal drectons nteractvely, but t requres too much memory and dsk I/O. By examnng the mnmal nformaton needed to reconstruct a novel vew, we can reduce these demands and ncrease our pre-fetchng length. We consder two scenaros. The frst case s when the user heads straght down the track and keeps the vew plane normal constant all the tme. In ths case, only the nformaton wthn the user s vewng drecton s needed. However, for fast movement, more than the two closest reference vews are requred. The second case s when the user only looks around and never moves on the track. Ths requres only the nformaton of the whole panoramc of the current two closest reference vews. Usually, the user can both move along the track and stop to look around. Ths means that a far amount but not all of the panoramc s needed for more than the two closest reference vews. Therefore, we have the choce of prefetchng more of the panoramc of current vews or more of the next vews along the track. Ths s determned by the user whether movng along the track fast or lookng around nteractvely s more mportant. A compromse was acheved usng an adaptve two-phase pre-fetchng scheme one along the track and the other along the vewng drecton. We adaptvely reduce the nformaton we pre-fetch as the pre-fetched vews are farther away from the current vewpont. Therefore for the closest two sampled vews we load n most of the panorama. As we pre-fetch nformaton of the farther away vew samples we pre-fetch less and less of the nformaton nto the cache. Ths s llustrated n Fgure 4. Fgure 4 (a) defnes a path wth through 6 reference vews. A track segment s defned between the two close-by vewponts. We have labeled these segments through 5. The current novel vewpont s on segment number whch s between and. 3, 4 and 5 are farther and farther away from the current vew. Fgure 4 (b) shows that the whole panoramas for and are loaded nto the cache, whle only parts of the panorama (tles) are loaded n for 3, 4 and 5. When we move to segment number, the novel vew les between and 3, we use another thread to fetch the remanng nformaton for 3 and prefetch partal nformaton for 6 whch currently has no nformaton n the cache. Ths s shown n Fgure 4 (c). 7. Error analyss 7. Depth-Mesh Errors We consder two sources of errors. The frst source of errors comes from down-samplng the depth-meshes. Reconstructon usng standard graphcs cards utlzes a b-lnear functon to determne the depth values n the nteror. Ths assumpton, of course, s not always vald. Therefore, some errors are ntroduced when we reconstruct the reference vews and novel vews. We can certanly reduce ths knd of error by usng a fner quad-mesh (smaller tle sze). However, ths ncreases the renderng tme, mage and geometry loadng tme and storage requrements. Table shows the Mean Squared Error (MSE), the mage and geometry loadng tme, the renderng tme (FPS) and the storage requrement usng varyng tle szes for the PORAY rendered Nature dataset. These are also plotted graphcally n Fgure 5. From the Fgure we can see that the MSE decreases almost lnearly when we decrease the tle sze. However, the loadng tme ncreases qute dramatcally wth smaller tle szes. The renderng speed decreases wth decreased tle sze. As tle sze ncreases, the storage frst decreases, hts a low value and then ncreases. The decrease at frst s due to the fact that we need to store more depth values when we have more tles wth smaller sze. The later ncrease s because when the tles are too bg, there are fewer empty tles to remove. Ths s also the reason why the renderng speed levels off when the sze ncreases to a certan value we have more empty space to rasterze. After observatons, we found that 6x6 or 3x3 tle szes are good canddates for our applcaton. Loadng Tme(s) FPS MSE Storage (MB) 64x x x x x x Table : shows the change of loadng tme, renderng speed, Mean Squared Error and Storage of one typcal vewpont wth dfferent tle sze. Ths s for the Nature dataset. MSE s based on the maxmum number of 56. The Eurographcs Assocaton 00

9 Yang / Crawfs: Ral-Track ewer An Image-Based rtual Walkthrough System Loadng Tme, FPS, MSE, Storage vs. Tle Sze x 4x4 8x8 6x6 3x3 64x64 Tle Sze Loadng Tme (s) FPS MSE Storage (MB) Fgure 5: Shows the Mean Square Error (MSE), the loadng tme, Frame Per Second (FPS) and the storage requrement wth dfferent tle szes for a typcal sampled vewpont of the PORAY rendered Nature dataset. MSE sblty Errors t vews 3 vews 5 vews Fgure 6: Mean Square Error of nterpolated vews aganst actual rendered vews. Interpolated vews have a tle sze of 3x3 7. sblty Errors Another source of errors are the vsblty errors that come from reconstructng the novel vew by cross-dssolvng the two close-by sampled vews. As descrbed prevously, f the vsblty polyhedrons of the two sampled vews concde wth each other perfectly, no vsblty errors wll occur. Fgure 6 shows the MSE of the resultng mages for nterpolated vews aganst Povray rendered vews at correspondng postons along our path. Three dfferent samplng rates are examned and a tle sze of 3x3 s used. We can see that the peak errors occur somewhere close to the mddle of the two sampled vewponts for all three cases. Ths s expected because the vsblty errors are most serous n the mddle of the two samples. The curve whch uses sampled vewponts has the coarsest samplng rate and therefore has the hghest mean square error. The Peak s about 7 out of 56, whch s approxmately 6.7%. The curve whch uses 5 vews has the fnest samplng rate and peaks out wth an MSE of about 4-5. The mnmum MSE occurs at the sampled vewponts, whch comes entrely from the down-samplng errors. Fgures 8 and 9 (see color secton) show the nterpolated and actually rendered mages usng the same vewpont, Fgure 0 (see color secton) shows the dfference mage. Here we see the majorty of the errors occur on slluoettes where the depth values change substantally. 8. Results and Dscussons Our ral-track vewer was mplemented n Java/Java3D. We ran our IBR framework on datasets. The frst one s a vrtual scene rendered usng Povray []. The scene s farly complcated contanng trees, bushes, rocks and brds. It requres almost 30 mnutes to render one frame usng Povray on our dual Pentum II 500MHZ, 5MB SGI vsual workstaton. One path was chosen for ths scene wth 0 sampled vewponts along the track. Three panoramc layers wth a resoluton of 04x4096 per layer were pre-computed for each vew sample. The total sze for the mage database after preprocessng was 0MB. The geometry and magery was broken up nto 3x3 quads. The pre-renderng takes about 40 hours on the vsual workstaton. Our second dataset s the LOX post dataset whch sualzaton Toolkt (TK) [] provdes. Ths dataset smulates the flow of lqud oxygen across a flat plate wth a cylndrcal post perpendcular to the flow. It s a complex scentfc dataset whch contans both scalar and vector felds n the data. A renderng was chosen wth the post, a slce plane and several stream-polygons. One path was preselected gong nto the stream-polygons regon wth 3 vew samples. Four panoramc layers wth a resoluton of 5x048 per layer were pre-computed for each vew sample. The reference mage database was pre-rendered usng TK and the total sze for the mage database s 38MB. The geometry and magery was broken up nto 6x6 quads. Ths database requred 9. hours to pre-render. For the Povray dataset, our IBR vewer acheves 0-5 frames per second wth a kxk mage resoluton on the Sun Blade 000 The Eurographcs Assocaton 00

10 Yang / Crawfs: Ral-Track ewer An Image-Based rtual Walkthrough System workstaton wth dual Ultra-Sparc III 750MHZ, GB of memory and Elte 3D graphcs card. For the LOX dataset, we see 5-0 frames per second wth a 5x5 resoluton on the same platform. Fgure 7 (see color secton) shows one rendered vew for the nature dataset wth the quad-meshes represented by whte squares and color nformaton as texture maps. Fgures 8, and (see color secton) show some resultng mages from our IBR system. 9. Conclusons and Future Work Ths paper presents our framework for allowng the user to examne and walkthrough the scene from an nternal vantage pont on a relatvely hgh-resoluton dsplay nteractvely. We acheved ths goal by lmtng the user movement on a track and utlzng IBR technques. Intellgent data organzaton, cachng and pre-fetchng make our system more effcent. We prove by results that our system s sutable for explorng complex vrtual envronments and large datasets wth reasonably small errors. Trangular meshes would decrease the down-samplng error [6]. We are lookng at ways of usng trangular meshes to replace fxed sze quad-meshes. Even wth the empty tle removal scheme, our IBR database stll tends to be very large, especally wth larger resoluton and fner samplng. Future work entals lookng at approprate compresson technques to reduce the data storage requrements. 0. Acknowledgements We would lke to thank the Department of Energy ASCI Program for generous support for ths project. Addtonal support was provded through an NSF Career Award (#98760). Equpment was provded by the ASCI project and by NSF grants (#98839) and (998605). References [] E. H. Adelson, J. R. Bergen, The plenoptc Functon and the Elements of Early son, Computatonal Models of sual Processng, Chapter, Edted by Mchael Landy and J. Anthony Movshon, The MIT Press, Cambrdge, Massachusetts, 99 [] Chun-Fa Chang, Gary Bshop, Anselmo Lastra, LDI Tree: A Herarchcal Representaton for Image-Based Renderng, Proc. SIGGRAPH 99, pp. 9-98, 999 [3] S. E. Chen, QuckTme R An Image- Based Approach to rtual Envronment Navgaton, Proc. SIGGRAPH 95, pp. 9-38, 995 [4] J-J Cho and Y. Shn, Effcent Image-Based Renderng of olume Data, Proc. Pacfc Graphcs 98, pp , 998 [5] D. Cohen-Or, Y. Mann and S. Fleshman, Deep Compresson for Streamng Texture Intensve Anmatons, Proc SIGGRAPH 99, pp [6] L. Darsa, B. Costa, and A. arshney, Navgatng statc envronments usng magespace smplfcaton and morphng, 997 Symposum on Interactve 3D Graphcs, pp. 5-34, 997 [7] P. Debevee, Y. Yu and G. Borshukov, Effcent ew-dependent Image-Based Renderng wth Projectve Texture Mappng, In 9 th Eurographcs Renderng Workshop, enna, Austra, June 998 [8] X. Decoret, G. Schaufler, F. Sllon, J. Dorsey, Multlayered mposters for accelerated renderng, Proc. Eurographcs 99, pp , 999 [9] S. Gortler, R. Grzeszczuk, R. Szelsk, and M. Cohen, The Lumgraph, Proc SIGGRAPH 96, pp , 996 [0] M. Levoy and P. Hanrahan, Lght Feld Renderng, Proc. SIGGRAPH 96, 996. [] W. Mark, L. McMllan, and G. Bshop, Post-Renderng 3D Warpng, 997 Symposum on Interactve 3D Graphcs, pp. 7-6, 997 [] L. McMllan and G. Bshop, Plenoptc Modelng: An Image-Based Renderng System, Proc. SIGGRAPH 95, pp , 995 [3] K. Mueller, N. Shareef, J. Huang, and R. Crawfs, IBR-Asssted olume Renderng, LBHT IEEE sualzaton 99, pp. 5-8, 999 [4] F. P. Preparata, M. I. Shamos, Computatonal Geometry, An Introducton, Chapter, Sprnger-erlag New York Inc, 985 [5] B. Oh, M. Chen, J. Dorsey and F. Durand, Image Based Modelng and Photo Edtng, Proc. SIGGRAPH 0, pp , 00 [6] H. Qu, M. Wan, J. Qn and A. Kaufman, Image Based Renderng Wth Stable Frame Rates, Proc. IEEE s 000, pp 5-58, 000 [7] J. Shade, S. Gortler, L-We He, and R. Szelsk, Layered Depth Images, Proc. SIGGRAPH 98, pp 3-4, 998 [8] Heung-Yeung Shum, L-We He, Renderng Wth Concentrc Mosacs, Proc. SIGGRAPH 99, pp , 999 [9] R. Szelsk and H. Y. Shum, Creatng Full ew Panoramc mage Mosacs and Texture- Mapped Models, Proc. SIGGRAPH 97, pp 5-58, 997 [0] [] [] The Eurographcs Assocaton 00

11 Yang / Crawfs: Ral-Track ewer An Image-Based rtual Walkthrough System Fgure 7: shows one rendered vew wth quad-meshes represented by whte squares and color nformaton as texture maps. Ths fgure also shows our empty tle removal scheme. Any tles wth no useful nformaton won t be reconstructed or rendered. Fgure 8: The nterpolated mage for one vewpont whch s n the mddle of the two reference vewponts Fgure 9: The actual rendered mage for the vewpont at the same poston as n Fgure 8 Fgure 0: The dfference mage between Fgure 8 and Fgure 9 Fgure : The nterpolated mage for one vewpont whch s outsde of the stream-polygon regon of the LOX dataset Fgure : The nterpolated mage for one vewpont whch s nsde of the stream-polygon regon of the LOX dataset The Eurographcs Assocaton 00