A NEW APPROACH FOR 3D MODELS TRANSMISSION

Transcription

1 A NEW APPROACH FOR 3D MODELS TRANSMISSION A. Guarnieri a, F. Piroi a, M. Ponin a, A. Veore a a CIRGEO, Inerdep. Research Cener of Carography, Phoogrammery, Remoe Sensing and GIS Universiy of Padova, Agripolis Legnaro (PD) - Ialy anonio.veore@unipd.i ; cirgeo@unipd.i KEYWORDS: 3D modeling, Inerne, daa compression, documenaion, daa rerievial, VR ABSTRACT The visualizaion and ineracive navigaion of 3D models, like virual visis of museums, churches or even whole ciies (cyberciy models), are gaining a more and more increasing use and imporance wih ime. This has been accomplished in las years hank o he augmened power of modern CPUs, graphic hardware devices and hank o a wider access o he web. Even in he survey field he ineres owards 3D models has grown, given he advanages offered by ground-based laser scanners in erms of survey ime and he prey good qualiy of obained 3D models. Indeed, several ess and applicaions have been performed so far all around world wih such insrumens: 3D daa have been colleced from objecs of differen shape and size, ranging from culural heriage o land infrasrucures, accuracy has been invesigaed and corresponding 3D models have been creaed using appropriaed sofware. Beside issues relaed o he accuracy and he qualiy of resuling models, anoher ineresing opic should be considered: he remoe use of hese 3D models, i. e. how hey can be opimally ransmied over he web. To his aim we have developed a VRML spli-browser, i.e. a 3D model visualizaion shared sysem based on wo differen server side and clien-side applicaions. The former execues he rendering of he scene and send i o he clien, which is locaed on he remoe PC of he user. In urn, he clien provides he user wih a graphical inerface o explore he 3D model ineracively. The sysem is based on image compression and projecive ransformaions, which allow o share he 3D models over he web wih limied hroughpu o he clien. In his paper, we will presen he frame predicion algorihm, based on a space displacemen of he viewpoin and he way adoped o apply i in he remoe visualizaion sysem. 1. INTRODUCTION Over las imes, he 3D modeling of real objecs, of whaever size and complexiy, is gaining a more and more ineres and imporance by he scienific communiy. For insance, he availabiliy of close-range and long-range ground-based laser scanners, which are able o collec several housands of poins in shor ime, have increased he ineres owards 3D models as a useful and alernaive mean for analysis and sudy in he field of medicine, indusry, culural heriage, engineering, archiecure, ec. Tess and applicaions have been exensively performed all around he world in order o assess he accuracy and he qualiy of hose 3D models. Beside such opics, anoher ineresing issue should be considered: he remoe use of hese 3D models, i. e. how hey can be opimally ransmied over he web. To his aim we have developed a VRML spli-browser, i.e. a 3D model visualizaion shared sysem based on wo differen server side and clien-side applicaions. The main advanage of proposed soluion relies on he fac ha he server sends o he clien a periodic image sequence, composed by a reference image and a se of error images, insead of he whole VRML model. Basically, he server execues he rendering of he scene and sends i o he clien, which is locaed on he remoe user s PC. In urn, he clien provides he user wih a graphical inerface o explore he 3D model ineracively. Thus, using suied image compression echniques and projecive ransformaions, 3D models can be shared over he web wih limied hroughpu o he clien. Though he developed echnique looks like he algorihms implemened in he MPEG compression, in his case he keyadvanage relies on he a-priori knowledge of he image ransformaion parameers, as hey are deermined direcly by he reques of viewpoin change made by he user hrough he clien GUI. In his paper, we will presen he frame predicion algorihm, based on a space displacemen of he viewpoin and he way adoped o apply i in he remoe visualizaion sysem. The paper is srucured as follows. In he secion 2 a shor overview of he VRML language is provided, highlighing he disadvanages ha show up when large 3D daa ses are employed. In secion 3 we describe he framework of our VRML spli-browser, we developed in order o improve he remoe ransmission of 3D daa. Then, secion 4 focuses on he mahemaics which he implemened frame compression algroihm is based on, while in secion 5 he resuls of he ess performed using simple geomeric shapes are presened. Finally, secion 6 repors he conclusions. 2. THE VRML LANGUAGE Technically speaking, VRML is neiher virual realiy nor a modeling language; a is core, VRML is simply a 3D inerchange forma. I defines mos of he commonly used semanics found in oday's 3D applicaions such as hierarchical ransformaions, ligh sources, viewpoins, geomery, animaion, fog, maerial properies, and exure

2 mapping. One of he primary goals in designing VRML was o ensure ha i a leas succeeded as an effecive 3D file inerchange forma. Moreover, VRML can be considered as a 3D analog o HTML. This means ha VRML serves as a simple, muliplaform language for publishing 3D Web pages. This is moivaed by he fac ha some informaion is bes experienced hree dimensionally, such as games, engineering and scienific visualizaions, educaional experiences, and archiecure. Typically hese ypes of projecs require inensive ineracion, animaion, and user paricipaion and exploraion beyond wha is capable wih a page-, ex-, or image-based forma (i.e., HTML). VRML provides he echnology ha inegraes hree dimensions, wo dimensions, ex, and mulimedia ino a coheren model. When hese media ypes are combined wih scriping languages and Inerne capabiliies, an enirely new genre of ineracive applicaions are possible. In 1997 a communiy of programmers, engineers, graphers founded he VAG (VRML Archiecure Group) in order o exend he capabiliies of he previous VRML version (1.0), which feauured several properies of he Invenor File Forma, a graphic sandard developed by Silicon Graphics. The overall goal for he new VRML vs. 2.0 was fairly modes: o allow objecs in he world o move and o allow he user o inerac wih he objecs in he world, allowing he creaion of more ineresing user experiences han hose creaed wih VRML 1.0. The vs. 2.0 feaured following properies: 1) Composabiliy, i should be relaively easy o ake files creaed by various people or ools and compose hem ogeher o creae a new documen. This is anoher propery ha VRML shares wih HTML: I is easy o cu and pase ex from several HTML documens using eiher a generic ex edior or a specialized ediing ool, jus as i is easy o cu and pase objecs beween VRML worlds. 2) Scalabiliy, is a consrain on he VRML 2.0 design. VRML is designed o scale in hree ways. Firs, i should be heoreically possible for a VRML browser o handle a world disribued across he Inerne ha conains millions or billions of objecs. Second, VRML should work well when used wih boh very powerful and very inexpensive machines, allowing he VRML browser o rade off image or simulaion qualiy for improved performance and o scale well wih increased hardware performance. And hird, VRML worlds should scale wih nework performance, from he 14.4K modems ha are common oday o muligigabi connecions ha migh become common in he fuure. 3) Exensibiliy, he language should allow he developer o add funcionaliies, like new geomeric primiives, for insance. The VRML forma allows, herefore, o creae 3D environmens, ensuring basic properies like full web access o he model and plaform independence: only a suiable viewer is needed, capable o inerpre he VRML files. Such ineracive echnology can be profiably employed in differen applicaion fields, like for example he virual visi of a museum or even of a whole ciy (cyberciy). E- Commerce is anoher ineresing opic: a consumer could virually explore, in all is pars, he 3D model of he objec o be purchased, before o move o he shop. More recenly, he availabiliy of 3D models of culural heriage elemens and land infrasrucures (like buidlings, churches, bridges, road asses, ec.) conribued o increase he use of VRML as a mean o share such models hrough he web. However, he ineres of he scienific communiy (medicine, engineering, archiecure, ec) owards 3D models has revealed in he same ime some limis of he VRML as inerchange file forma when large daa ses, acquired ypically by close-range or long-range ground-based laser scanners, are used. Major disadvanages are he following: 1) File size. I is no uncommon o manage VRML files of hundreds of MByes; 2) Inpu peripherals. Ineracive user commands are acivaed hrough mouse and/or keyboard, which ofen are no he opimal peripherals o navigae in he 3D environmen; 3) Compuing resources. A VRML viewer needs ofen high compuaional performance in order o manage and render a complex scene, composed by housands of poins or riangles. Obviously, a more profiable way o share complex and large 3D models requires he developmen of soluions aimed o overcome above menioned issues. 3. THE VRML SPLIT-BROWSER Beside he VRML language, he srucure of he browser/viewer has o be considered, as well. Basically, i is a sofware applicaion which is able o display a 3D scene, whose conen is described in a VRML file. Moreover, a specific GUI (Graphical User Inerface) is provided o he user along wih a suied se of commands, allowing o navigae he 3D environmen. A VRML viewer works according o a sraified srucure, whose main componens can be summarized as follows (fig. 1): 1) Parser. I checks he grammaical and sinacical conen of he file. If any error is encounered, he file is discarded; 2) World loader, which deals wih he loading of all elemens, like exures, sounds and scrips, required o display correcly he virual world conen; 3) World manager, whose ask enails wih he loading in he sysem memory of he represenaion of he virual scene. Furhermore some basic componens of he scene have o be racked, like objec posiions, posiion and orienaion of he eye-poin, posiion and properies of ligh sources, objec maerials and he viewing volume. The world manager can be considered as he analog of he graphical engine used for he 3D models processing; 4) World renderer, which works ogeher wih he world manager o correcly display on he user s monior he conen of he VRML file. The renderer checks he scene coninuously, if any change is deeced (displacemen of he user s eye-poin) he new image o be displayed is compued as fas as possible. Of course his ask requires high compuaional effor from he user s hardware, which is no always he case. Figure 1: Concepual framework of a VRML viewer

3 In order o solve for he issues described in previous secion, a VRML spli-browser has been developed. Is working is based on a compuer nework model, where a cenral processor (he server) provides differen kind of services o a number of hoss (cliens). In pracice, his kind of viewer is obained by applicaion of a clien-server model o a generic VRML viewer. As shown in figure 2, he proposed model clearly disinguishes beween server-side and clien-side asks. The server manages he whole virual environmen compuing new scene views, processing user s commands and execuing scrips or animaions. From he clien-side, an ineracive GUI is provided o he user by which he virual views sen form he server are displayed. Moreover, he user can inerac wih he server by means of a number of navigaion commands available on he clien s GUI. Afer an iniial seup phase, needed o negoiae he parameers of he communicaion proocol (e.g. maximum hroughpu acceped by he clien), a bidirecional communicaion loop begins beween server and clien. In pracice, he user acivaes a command in he clien s GUI, hrough mouse and/or keyboard. Such command is ransmied o he server as even, i.e. as a reques of scene change. Then he server inrepres he even and, if possible, sends back o he clien he new view, which will displayed on he user s GUI. The described loop is repeaed according o he user ineracion. Operaive Sysem on which he server-side and he clien-side applicaions should run. 5) The copyrigh on he 3D model is keeped, as he user will never receive on his PC he whole VRML model, bu only a se of images corresponding o he acual scene displayed on he monior screen; 6) The user s GUI is managed by he clien only, independenly from he server. In his way a higher effeciveness of he whole sysem is ensured, since he server-side of he viewer doesn need o dedicae compuing resources for he user s GUI managemen. In order o allow he clien o navigae he 3D model as fluen as possible, regardless he size and complexiy of he model and clien s compuing capabiliies, above menioned feaures of our proposed spli-browser are sill no sufficien. An image compression echnique has o be implemened o ge an opimized clien/server communicaion proocol. To his aim wo differen compression algorihms have been invesigaed, he LZ77 and he JPEG, comparing he resuls of heir applicaions o our spli-browser ransmiing simple geomeric 3D shapes. However obained resuls revealed a limied compression capabiliy (higher for he LZ77 respec wih he JPEG) for our goals. This can be explained considering ha hose algorihms are general purpose : hey can be applied o whaever kind of file, regardless is conen. Therefore we developed an ad hoc compression echnique which would exploi he knowledge abou he conex where i should be applied. The descripion of his echnique will be he opic of he following secion. 4. THE FRAME COMPRESSION ALGORITHM The alernaive procedure we developed in order o furher reduce he hroughpu beween server and clien is based on he applicaion of projecive ransformaions. The underlying idea is ha subsequen views, processed by he server and displayed by he clien o he user, are joined each oher hrough such kind of ransformaions. Since he knowledge abou he geomery of he virual environmen (e.g. eye-poin posiion, viewing direcion, ec) is needed, projecive ransformaions don allow o preserve he same level of flexibiliy and versailiy as LZ77 and JPEG image compression algorihms. Therefore our mehod can be applied o specific cases only, where geomery relaed images are handled. As shown in figure 3, he server/clien communi-caion procedure works as follows: Figure2: server/clien-based VRML viewer archiecure This clien-server approach shows following advanages: 1) The amoun of daa sen o he clien is grealy reduced respec wih he ransmission of he whole 3D model in VRML forma. Indeed he server sends o he clien an image sequence only: adoping an image compression echnique, he hroughpu could be furher reduced; 2) The clien doesn need high compuaional capabiliies, as he world managemen sep is fully performed on he server; 3) Since server and clien are independen each oher, he LOD (Level of Deail) of each scene can be ineracively changed by he clien ; 4) The sysem feaures a cerain degree of porabiliy, as he source codes can be modified accordingly o he Server-side operaions 1) Given he user inpu (scene change) a ime n, he server processes he reques generaing he corresponding new view of he 3D model (view B). 2) As he geomerical parameers describing he 3D scene a ime n-1 and n are known, he server can deermine he projecive ransformaion, mapping view A o view B. Compued parameers are hen sen o he clien. 3) The view A, describing he 3D scene a ime n-1, is applied he previous compued projecive ransformaion. Consequenly, a prediced view is generaed, which should represen a good approximaion ov view B. he predicion is generaed by ransforming he reference frame (view A) on he basis of he displacemen of he user poin of view (known from he seleced command of he GUI) and he disance from he image pixel projecion plane (known from he conen of he Z buffer).

4 4) Then, a pixel by pixel difference beween view B and he prediced view is calculaed by he server. A so called error-image is obained, which is compressed wih LZ77 algorihm and sen o he clien. Clien-side operaions 1) The clien receives he projecive ransformaion parameers and he error-image, which is decompressed, rerieving he original image. 2) The projecive ransformaion is applied o he view A, generaing a so called prediced view. 3) Then, he error-image is added pixel by pixel o he prediced view, in order o creae he requesed view B, which is hen displayed by he clien in he user s GUI. Typically, each projecive ransformaion is associaed a marix P R 3x3, called projecive marix, allowing a more compac noaion for eq. (1): A M b a11 a12 b1 P = L L L = a a b (2) c M d c1 c2 d In he following, he procedure adoped o compue he projecive ransformaion parameers will be described using a simple polygon as arge objec in he 3D space. This assumpion doesn limi he effeciveness of our procedure, since i is well known ha each 3D objec can be described in erms of a number of inerconneced polygons (ypically riangles). Therefore, in principle, i would be sufficien o apply he algorihm o each composing polygon. According o figure 4, he aim of our procedure deals wih he compuaion of he posiion of poin T 2, by applicaion of a projecive ransformaion o poin T 1 (idenified by P 3 ). Figure 3: The VRML Spli-browser srucure The decripion presened above shows ha he compuaional load on he clien is grealy reduced, respec wih he classical approach (ransmission of he whole VRML model). Indeed, he clien is required only o apply he projecive ransformaion o he curren view and o decompress he error-image as fas as possible. Nowadays, requesed compuaional capabiliies o perform such asks are commonly available on mos home deskop PCs. In he following subsecion, some mahemaics will be provided abou he developed frame compression algorihm. 4.1 The projecive ransformaion A projecive ransformaion can be considered as a subse of he more general group of coordinae ransformaions, which maps a given inpu 2D image poin x = [x 1, x 2 ] ino a new image poin y = [y 1, y 2 ]. Adoping a marix noaion, his mapping fucnion can be defined as follows: where Ax + b y = (1) c x + d a11 a12 A = ; b = [ b1 b2 ] ; c = [ c1 c2 ] ; d R a21 a22 Figure 4: Geomeric model for he projecive ransformaion If projecive ransformaions P 1 and P 2, mapping T on T 1 and T 2 respecively, are known, following relaionship can be esablished: T2 2( 2 1, inv T1 = P T) = P ( P ( )) (3) In pracice, he projecive ransformaion which maps he view-plane α 1 (par of polygon β displayed on he user s GUI a ime n-1) on α 2 (view of polygon β a ime n) can be defined in erms of a marix produc: 1 P = P P (4) 3 where P i denoes he projecive marix associaed o he i-h projecive ransformaion. As shown by eq (4), in order o deermine P 3, we need o compue P 1 and P 2 before. To his aim, we consider firsly he parameric equaions of α and β planes (see figure 5): α x " = A " + d β x' = A ' + c (5) where A, B R 3x2 ; c, d R 3 ;, R 2 and x, x R 3. Then we add wo consrains: a) vecors generaing he planes are orhogonal (eq. 6); b) vecors c and d, defining he disance 2 1

5 of α and β planes from he origin of he absolue reference frame, are orhogonal o heir corresponding planes (eq. 7). B B = A A = I (6) c B = d A = 0 (7) Afer some subsiuions and aking ino accoun eq. 7, we ge d ( d E) * = d B' + ( d c d E) (13) Using eq. 10 and 13, he following fundamenal relaionship is obained, which represens he projecive ransformaion we searched for: F ' + g = h ' + q " (14) Indeed, he 2D vecor conains he coordinaes of a generic poin x in a β plane fixed reference frame, while he vecor, obained from by eq. 14, conains he 2D coordinaes of poin x on a α plane fixed reference frame. Figure 5: Geomeric model (parial view) Now le us suppose o know B and c (i.e. he β plane), he posiion in he 3D space of he eye-poin E, he user s viewing direcion v, he uni vecor up of he eye-poin and m, he disance from he eye-poin o α plane. Given such parameers, we can compue he α plane, i.e. parameers A and d of eq. 5, according o he consrains repored in eq 6,7. These consrains mean ha he wo vecors generaing he α plane should be orhonormal and coinciden wih he columns of marix A, and vecor d is orhogonal o hem. Assuming he view-plane (α) is orhogonal o uni vecor v, i can be easily demonsraed ha marix A becomes 5. TEST AND RESULTS In order o evaluae he performance of he compression algorihm, a server/clien daa ransmission wih he developed spli-browser has been carried ou. To his end we employed simple geomeric shapes (polygons) of differen colors, as shown in figure 6. Here view A represens he scene observed by he user a ime n-1, B is he exac view as compued by he server, o be displayed a ime n, according o user inpu, while he prediced view is he one obained by applicaion of he projecive ransformaion o view A on he clien. upx dxx A = [ up dx] = up y dx (8) y upz dxz where dx = up v. Similarly, vecor d can be easily obained according o following relaionship: d = ( < E, v > m) v (9) where he symbol <,> denoes he scalar produc. A his sep we have all elemens needed o deermine he projecive ransformaion P, which maps a generic poin x of β plane on poin x, lying on he α plane. Le be s he bundle of sraigh lines passing hrough he eye-poin E, 3 s s = k + E ; k R, R (10) Wih some sraighforward algebra, we ge he equaion of he sraigh line of s, which passes hrough poin x : r = k + E ; wih k = x' E (11) Thus, in order o compue he value * assumed by parameer, when he sraigh line r inersecs he α plane on poin x, we consider following equaliy: k * + E = x" = A" + d (12) Figure 6: Views A (lef) Prediced views (middle) Views B (righ) Following crierions were defined o compare he resuls each oher: 1) Average bi/pixel number of compressed view B. This parameer defines he number used on average o code each pixel ov compressed view B. As we deal wih 24 bi RGB images, he resuling value should be lower han 24 o pass he es. 2) Average bi/pixel number of compressed error-image. I is similar o he previous parameer, bu in his case he hreshold value for he pass/fail es should be lower han he previous one. 3) Compression raio. I defines he raio beween he size (in byes) of compressed view B and he sum of he size

6 of compressed error-image and he byes needed o code he projecive ransformaion parameers. pixels are obained as difference of he color values of adjacen pixels on he source image. Table 1: Resuls of he frame compression algorihm Polygon Compr.sed Compr.sed Bi/ Bi/ Raio Error-image (byes) View B (byes) pixel error pixel view B Green Red Yellow I should be noed ha he compression raio is always greaer han 1, wha confirms he effeciveness of implemened compression procedure. Indeed, his means ha sending o he clien boh he compressed error-image and he parameers of he projecive ransformaion, generaes a hroughpu lower han he ransmission of he compressed view only. As shown in able 1, he gain is ranging beween 25% and 28% in erms of byes employed. 6. CONCLUSIONS In his paper an innovaive concep of VRML browser has been proposed. I is aimed mainly o allow he display of complex and large 3D models, as he ones obained by ground-based laser scanners, reducing he compuaional load on he user-side. To his end, we developed a VRML splibrowser, which is concepually based on a server/clien paradigm, bu a he same ime i keeps all he advanages of he VRML inerchange file forma as suiable mean o share 3D objecs along he web. The main idea on he ground of developed archiecure is he spliing of he asks beween server and clien. The former performs all he operaions requiring he mos compuaional effor, reducing he working load of he laer. This has been accomplished using perspecive ransformaions and image compression algorihms, like LZ77. In our sysem he server sends o he clien only a se o parameers, defining he perspecive ransformaion beween o 3D scenes, and a compressed error-image, insead of he whole VRML 3D model. Then he clien uses such elemens o reconsruc he view of he scene according o he inpu command acivaed by he user on he clien s GUI. The proposed work is sill in progress, as improvemens can be sill applied, in order o furher opimize he performance of our spli-browser. We are invesigaing he use of he Bounding Box, view-images caching and he adopion of a linear predicion scheme in he LZ77 algorihm. The former would allow o compress only he porion of he whole 3D model ha is acually enclosed by he bounding box, saving in his way he number of byes used for he image coding. The second soluion deals wih he creaion of a memory buffer on he clien side, where all he views sen by he srever are sequenially sored. In his way, if he eye-poin (user s viewing direcion) reurns back on a previous visied posiion, he clien needs o resore he corresponding view from he buffer: no daa have o be sen from server o he clien. A furher server/clien hroughpu reducion could be achieved by means of he linear predicion: he pixel color of an image can be prediced on he baiss of he color of he neighbours. Accordingly, he server would no compress he original image bu raher he so called residual-image, whose ACKNOWLEDGEMENTS This work was developed wih he projec Applicaion in he survey, sore and managemen of environmenal and culural resources of GNSS/INS posiioning and saellie, aerial, erresrial phoographic and laser scanning daa, ransmied by DARC, GSM/GLOBAL STAR, INTERNET mehods, parly financed by MURST (Ialian Minisry of Universiy and Research) in 2002 as projec of relevan Naional ineres. Naional coordinaor: Giorgio Manzoni, head of he Research uni Anonio Veore. REFERENCES Carey R., Bell G, The Annoaed VRML 97 Reference Foley J., Dam A. V., Feiner S., Hughes J., Compuer Graphics: Principles and Pracice. Addison-Wesley, 2 nd ediion. Gleicher M., Projecive Regisraion wih Difference Decomposiion. URL: hp// Papers/Gleicher/California/rack-Final.pdf Harley R., Zissermann A., Muliple View Geomery. CVPR, June. Mason W, Open-GL Programming Guide. Addison- Wesley, 2 nd ediion. Mian G.A., Rinaldo R., Elaborazione e Trasmissione delle Immagini. Ed. Libreria Progeo, Padova. Nelson M., The Daa Compression Book. M&T Books. Ziv J., Lempel A., A Universal Algorihm for Sequenial Daa Compression. IEEE Transacions on Informaion Theory, n. 23, pp