Avocado: A Distributed Virtual Reality Framework

Transcription

1 Avodo: A Distriuted Virtul Relity Frmework Henrik Trmerend GMD - Germn Ntionl Reserh Center for Informtion Tehnology henrik.trmerend@gmd.de Astrt We present Avodo, our ojet-oriented frmework for the development of distriuted, ertive VE pplitions. Dt distriution is hieved y trnsprent replition of shred sene grph mong the prtiipting proesses of distriuted pplition. A sophistited group ommunition system is used to gurntee stte onsisteny even in the presene of lte joining nd leving proesses. We lso desrie how the fmilir dtflow grph found in modern stnd-lone 3D-pplition toolkits extends niely to the distriuted se. 1. Introdution Mny toolkits for the development of stnd-lone VE pplitions exist tody. They provide the progrmmer with high-level erfe to represent omplex geometry in sene grph nd to render tht sene grph. The progrmmer is shielded from the detils of deling with low-level grphis nd system APIs, nd n onentrte on the development of the pplition itself. Severl ttempts to offer similr toolkits for distriuted VE pplition development hve een mde in reent yers (eg. RB2 [18], DIVE [4], MR [16], dvs [7], WTK [14, 15]). They provide vrious degrees of support for network sed ommunition etween the distriuted proesses tht form n pplition. However, using these filities often requires signifint effort from the progrmmer. Commonly, only prts of the pplition stte, suh s trnsformtion mtries whih desrie ojet positions, re shred etween the distriuted proesses. Sometimes, expliit speifition of ommunition endpos for shred ojet ttriutes is neessry. The resulting dtse duplition prolem, ensuring tht ll proesses work on onsistent opies of the shred dtse, is left s n exerise to the pplition progrmmer. This n e tedious nd error prone tsk espeilly in the presene of lte joins, where dditionl proesses join n lredy running distriuted pplition. Avodo provides progrmmers with the onept of shred sene-grph, essile from ll proesses forming distriuted pplition. Eh proess owns lol opy of the sene grph nd the ontined stte informtion, whih is kept synhronized. Our ojet-oriented frmework llows the retion of pplition speifi lsses, whih inherit these distriution properties. Furthermore, the shred sene-grph is ugmented with distriuted dtflow grph. This provides the sme evlution hrteristis in distriuted pplitions s in stnd-lone pplitions, nd effetively supports the development of distriuted ertive pplitions. In ontrst to similr systems like Repo-3D [11] we fous on high-end, rel-time, virtul environments like CAVEs [5] nd Workenhes [10, 9], therefore the development is sed on SGI Performer [13]. With Avodo, we provide frmework tht omines the fmilir progrmming model of existing stnd-lone toolkits with uilt-in support for dt distriution tht is lmost trnsprent to the pplition developer. We strt with the desription of our si system design, nd show how this n e extended in stright forwrd wy to the domin of distriuted pplition development. 2. Bsi Conepts Avodo is progrmming frmework for uilding distriuted, ertive VE pplitions. It uses the C++ progrmming lnguge to define two tegories of ojet lsses. Nodes provide n ojet-oriented sene-grph API whih llows the representtion nd rendering of omplex geometry. Sensors provide Avodo with its erfe to the rel world nd they re used to import externl devie dt o n pplition. All Avodo ojets re fieldontiners, representing ojet stte informtion s olletion of fields. They support generi streming erfe, whih llows ojets nd their stte informtion to e written to strem, nd the susequent reonstrution of the ojet from tht strem. This erfe is one of the si uilding loks used for the implementtion of ojet distriution. Avodo uses onnetions etween fields to uild

2 dtflow grph orthogonl to the sene grph, whih is used to speify ehvior nd llows for ertive pplitions. We hose to se the Avodo frmework on SGI Performer to hieve the mximum possile performne for our pplitions. Advned rendering tsks like ulling, level-of-detil swithing nd ommunition with the grphis hrdwre re ll hndled y Performer. Whenever the underlying hrdwre llows, Performer utilizes multiple proessors nd multiple grphis pipelines. In ddition to the C++ API, Avodo fetures omplete lnguge inding to the erpreted lnguge Sheme [1], [6]. Sheme is generl purpose progrmming lnguge desended from Algol nd Lisp. It is high-level lnguge, supporting opertions on strutured dt suh s strings, lists nd vetors. All high-level Avodo ojets n e reted nd mnipulted from Sheme. The Avodo sripting erfe suggests two trked pproh to pplition development. Complex nd performne ritil funtionlity is implemented in C++ y sulssing nd extending existing Avodo lsses. The pplition itself is then just olletion of Sheme sripts whih instntite the desired Avodo ojets, ll methods on them, set their field vlues nd define reltionships etween them. Sripts n e written nd tested nd deugged while the pplition is running. This leds to very short turnround times in the development yle nd provides very powerful environment for rpid pplition prototyping. The reminder of this setion gives rief rodution to the si onepts of Avodo nd lys the foundtion for setion 3, whih desries how these onepts re extended to the domin of distriuted pplitions Fields nd Fieldontiners The effiient implementtion of generi streming erfe for heterogeneous ojets requires dditionl met informtion out ojet ttriutes nd their types, nd wy to ess those ttriutes without knowing the ext type of the ontining ojet. The C++ progrmming lnguge does not tret lsses s first-lss ojets, so this met informtion is not esily ville on lnguge level. Performer, for exmple, uses memer funtion API to ess the stte ttriutes of n ojet. A symmetri pir of getter nd setter funtions exists for eh ttriute. Setting one ttriute my hnge nother ttriute of tht ojet s side effet. However, no strt informtion out the numer of ttriutes, their type nd their vlue n e otined from n ojet vi the Performer API. To eliminte this defiieny we dopted n pproh tken y the Inventor Toolkit [17] nd lso found in the VRML speifition [3]. Avodo uses fields s ontiners for ojet stte ttriutes. Fields enpsulte si dt templte<lss T> lss vsinglefield : puli vfield { puli: void setvlue(onst T& vlue); onst T& getvlue() onst; templte<lss T> ostrem& opertor<<(ostrem& strem, onst vsinglefield<t>& field); templte<lss T> istrem& opertor>>(istrem& strem, vsinglefield<t>& field); Figure 1. Prt of the vsinglefield erfe. vsinglefield is templte lss nd is used to instntite single vlue fields on si dt types. lss vfieldcontiner { puli: getnumfields(); vfieldptrve& getfields(); proteted: virtul void notify(vfield& field); virtul void evlute(); ostrem& opertor<<(ostrem& strem, vlink<vfieldcontiner>& f); istrem& opertor>>(istrem& strem, vlink<vfieldcontiner>& f); Figure 2. The vfieldcontiner enpsultes the stte informtion of n ojet nd represents it s olletion of fields. types nd provide generi streming erfe. They re implemented s puli lss memers nd re thus inherited y derived lsses. They re diretly essile y lient lsses nd re Avodo s premier erfe to ojet stte ttriutes. Fields n e one of two different types. Single-fields ontin one si dt type vlue, while multifields ontin vetor of n ritrry numer of vlues. Avodo mkes extensive use of the Stndrd Templte Lirry (STL). Multifield vlues re implemented s STL vetors of the field type.fields re implemented using C++ templtes, whih llow prmeteriztion of the vfield<> lss with the required dt type. As n exmple for the field lss API, prt of the templte lss definition for the single field is shown in figure 1. Aess to field vlue is provided y the getvlue() nd setvlue() methods. For eh field lss pir of strem opertors exist, llowing seriliztion of the field vlue o strem, nd the reonstrution of the field vlue from strem. Avodo ojets re fieldontiners, whih represent ojet stte s olletion of fields. A fieldontiner n e queried for the numer of ontined fields nd their referenes. Relevnt prts of the fieldontiner erfe re

3 lss pfgroup { setnme(onst hr *nme); onst hr* getnme(void); ddchild(pfnode *hild); insertchild( index, pfnode *hild); replechild(pfnode *old, pfnode *new); removechild(pfnode* hild); serhchild(pfnode* hild); pfnode* getchild( index); getnumchildren(void); Figure 3. The Performer pfgroup erfe uses memer funtions to query nd lter the ojet stte. shown in figure 2. This, together with the generi streming erfe of fields, llows us to provide streming funtionlity t the fieldontiner level without knowing the ext type of the underlying ojet. This extends to ll lsses further derived from vfieldcontiner. Now, y sulssing from the onrete Performer senegrph API lsses nd using multiple inheritne to mix in the vfieldcontiner lss, we n provide Performer ojets with the field oriented Avodo API. In other words, the Avodo sene grph API inherits the struture nd funtionlity of the Performer sene-grph API to uild onrete lsses on top of its strt fieldontiner erfe. Adptor fields re used to ridge from the Performer method sed API to the Avodo field oriented API. They will forwrd getvlue() nd setvlue() requests to the pproprite getter nd setter funtions of the Performer API. This ensures tht Performer relted stte informtion is orretly updted ording to field vlue hnges, nd possile side effets re properly evluted. As n exmple we show the sulssing proedure for pfgroup node. pfgroup hs prtilly inherited erfe s shown in figure 3. An vgroup lss is derived from pfgroup nd vfieldcontiner nd equipped with suitle olletion of dptor fields (Figure 4). The Nme field implements its getvlue() nd setvlue() methods in terms of lls to pfgroup::getnme() nd pfgroup::setnme(). The Children field shows the dption of multifield to whole rnge of Performer ess memer funtions used to mnge the hildren of group node. The return type of Children.getVlue() is STL vetor of vlink<vnode>. The templte lss vlink<> defines smrt poer to Avodo ojets whih hides referene ounting detils from the progrmmer nd gretly redues the diffiulty of writing exeption-sve ode. Avodo s fieldontiner erfe provides solid foundtion for the susequent, generi implementtion of dvned fetures like dtflow omputing, sripting nd dislss vgroup : puli pfgroup, puli vfieldcontiner { puli: vmultiadptorfield<vlink<vgroup> > Prents; vsingleadptorfield<string> Nme; vmultiadptorfield<vlink<vnode> > Children; Figure 4. Avodo ojets mp the memer funtion erfe for ojet stte ess to field sed erfe. triution. These pilities re inherited y ny pplition speifi extensions dded y the progrmmer. Beuse Avodo nodes re sulssed from Performer nodes, oth node types n e freely omined to uild the sene grph Fieldonnetions Muh like the Inventor Toolkit, Avodo uses the onept of fieldonnetions. Fields of omptile type n e onneted suh tht whenever the vlue of the soure field hnges, it is immeditely forwrded to the destintion field. Using fieldonnetions, dtflow grph n e onstruted whih is oneptully orthogonl to the sene grph. Avodo utilizes this mehnism to speify dditionl reltionships etween nodes, whih nnot e expressed in terms of the stndrd sene grph. This filittes implementtion of ertive ehvior nd the import of rel world dt o the sene grph. The evlution of the dtflow grph is performed one per rendering frme. Fieldonnetions forwrd vlue hnges immeditely, so tht there is no propgtion dely for sded onnetion pths in the grph. Loops re deteted nd properly resolved. A fieldontiner n implement side effets on field hnges y overloding the notify() nd evlute() memer funtions. Whenever field is set to new vlue, the notify() method is lled on the fieldontiner with referene to the hnged field s n rgument. The fieldontiner n do whtever is neessry to hieve the desired effet, inluding the mnipultion of other fields. After ll notifitions for ll fields on ll fieldontiners hve een mde for one frme, the evlute() method is lled on eh fieldontiner whih hd t lest one of its fields notified. This llows the fieldontiner to perform tions whih depend on more thn one updted field vlue for eh frme Nodes Fieldontiner dptions exist for ll Performer node lsses (pfgroup, pfdcs, pflod,...) nd most of the Performer ojet lsses (pfgeostte, pfmteril, pftexture,...), whih together represent the Performer

4 lss vtrkersensor : puli vdeviesensor { puli: vsinglefield<string> Sttion; // inherited vsinglefield<vmtrix> Trnsform; vsinglefield<ool> Button; Figure 5. The Avodo sensor lsses mp dt vlues from externl devies to fields. Sensors represent Avodo s erfe o the rel world. They re derived from the vfieldcontiner lss, ut not from ny Performer node, nd thus nnot e inserted o the sene grph. Sensors enpsulte the ode neessry to ess input devies of vrious kinds. The dt generted y devie is mpped to the fields of the sensor (Figure 5). Whenever devie genertes new dt vlues, the fields of the orresponding sensor re updted ordingly. Fieldonnetions from sensor fields to node fields in the sene grph re used to inorporte devie dt o n pplition. The vtrkersensor lss, for exmple, provides n erfe to six-degree-of-freedom spe-trker like the Polhemus Fstrk or the Asension Flok of Birds. Avodo utilizes devie demon proess, whih hndles the diret ertion with the devies vi seril line or network onnetions. The demon updtes the devie dt vlues o shred memory segment, where the vdeviesensor lsses n ess them. A sttion nme is used to identify the desired devie dt in the shred memory segment, nd every vdeviesensor lss speifies this identifier in it s Sttion field. After onneting to the devie demon, the vtrkersensor lss provides the urrent position nd orienttion informtion from the seleted trking devie represented s mtrix in its Trnsform field. By onneting the Mtrix field of n vdcs node to the Trnsform field, the sutree rooted y the vdcs node will move ording to the trker movement reported from the seleted sttion. Mny vsensor lsses hve een defined to support gret vriety of input devies, suh s mouse, keyord, trkers nd gloves. sene-grph API. By onvention the Avodo nodes reple the Performer pf prefix with the v prefix. The ility to mix Avodo nodes with Performer nodes to onstrut the sene grph n e onveniently used to define new nodes with dditionl funtionlity. The vfile node, for exmple, is derived from the dption node vdcs. It inherits the erfe of vdcs, whih silly onsist of Children nd Mtrix field. The vfile node dds URL field of type string. With n overloded notify() method, vfile rets to hnges of the URL field y retrieving geometry from the speified URL nd dding tht geometry to its list of hildren. Thus, vfile imports the geometry o the sene grph, nd n e seen s n opque hndle to it. Susequent hnges to the URL field reple the old with the newly speified geometry Sensors 3. Distriution The min gol driving the design of Avodo with respet to distriution ws to mke the development of distriuted pplitions s simple s the development of stndlone pplitions. The pplition progrmmer should e llowed to onentrte on the pplition itself, while Avodo provided the neessry infrstruture to mke tht pplition distriuted. To hieve this degree of network trnspreny, the onept of distriution is n egrl prt of Avodo s ojet model. In the reminder of this setion we will first desrie how the si Avodo progrmming onstruts, fields, fieldontiners, fieldonnetions, the sene grph nd dtflow evlution extend to distriuted Avodo ojets, nd then elorte on the detils of the underlying network lyer infrstruture The Distriuted Shred Memory Model The high ndwidth requirements during ojet ess for rendering neessittes tht eh ojet exists in the lol memory of the rendering proess. Thus, distriution support in Avodo is sed on ojet replition nd the distriuted shred memory model. Shred memory denotes lol memory re whih n e simultneously essed y more thn one proess on single mhine. Every proess tthed to shred memory segment will see ll hnges tht other tthed proesses pply to the segment. The onept of distriuted shred memory (DSM) extends this to proesses distriuted over network. Eh proess holds lol opy of the DSM segment, whih is kept synhronized with respet to hnges other proesses my pply to their lol opy. Lolly, the proesses only del with their memory segment nd my not even need to know tht it is distriuted one. Ojets whih would e lloted from suh DSM segment would e visile to ll prtiipting proesses. Avodo supports distriuted ojets y oneptully llowing ojets to e lloted from DSM segment shred y set of network distriuted proesses. Avodo proesses n tth themselves to one or more distriution groups (Figure 6). Ojets n e instntited

5 Proess A Distriuted Ojet Copies Distriution Group e d Proess B e d Lol Ojets Figure 6. Avodo ojets n either e reted lol to proess, or they n e migrted to distriution group, nd thus e replited over ll tthed proesses. either lolly s lol ojets, or in one of the tthed distriution groups s distriuted ojets. A lol ojet exists only in the ddress spe of the reting proess, while distriuted ojet will trnsprently instntite opies in the ddress spe of eh proess tthed to the distriution group. Susequent hnges to n ojet s stte will e forwrded to ll distriuted opies of the ojet. Distriuted ojets n e migrted etween different distriution groups, ut n only elong to one group t time Sene Grph As desried erlier, Avodo ojets re fieldontiners tht enpsulte the ojet stte in set of fields. The streming erfe of field nd fieldontiner lsses llows very elegnt implementtion of the distriuted ojet semntis. Distriuted ojet retion in Avodo is two stge proess. First, lol ojet is reted, whih is then, in seond step, migrted to the desired distriution group. The migrtion involves the nnounement of new ojet to the distriution group nd the dissemintion of the urrent ojet stte to ll group memers. The streming erfe of the fieldontiner is used to serilize the ojet stte o network dt uffer, whih is then sent to ll group memers. The group memers will reverse this proess nd rete opy of the ojet from the serilized stte informtion. The newly reted distriuted ojet will then exist s lol opy in eh of the prtiipting proesses. In Avodo, ojet stte is essile through ojet field vlues. Whenever field vlue on distriuted ojet is lolly hnged, the new vlue is stremed to network uffer nd sent to ll memers of the distriution group to keep the distriuted opies of tht ojet synhronized. As stted in setion 2.1, the prent hild reltionship etween ojets in the sene grph is represented y multifield of referene vlues on the prent node. Beuse ll field vlues, inluding vlink<> types, re stremle, the distriution sheme desried ove will not only distriute nd synhronize singulr ojets, ut lso prent hild reltionships etween them. It will replite omplete sene grph to ll proesses in distriution group. Being le to distriute the entire sene grph with ll prent hild reltionships etween the ojets, is key step towrds simplified development of distriuted pplitions. However, it is not suffiient. Consider the se of distriution group with two memer proesses nd. Both proesses hve lredy reted severl distriuted ojets in tht group. Now third proess joins the group. From now on, ll three proesses will e notified of future ojet retions nd mnipultions, ut proess will not know of the ojets tht nd hd reted efore it joined. Avodo overomes this prolem y performing n tomi stte trnsfer to every joining memer. When new proess joins n lredy populted distriution group, one of the older group memers tkes the responsiility to trnsfer the urrent stte of the distriution group to the new memer. This involves sending ll ojets, urrently distriuted in the group, with ll their field vlues to the newomer. After the stte trnsfer, the new memer will hve the proper set of ojet opies for this distriution group. To prevent onsisteny prolems, the stte trnsfer is performed s one tomi tion y suspending ll other ommunition during the proess. The ility to replite the entire sene grph, pired with the stte trnsfer to joining memers, effetively solves the dtse duplition prolem. New memers n join n existing distriution group t ny time nd will immeditely reeive their lol opy of the sene grph onstruted so fr in the distriution group. Furthermore, the pplition progrmmers do not need to onern themselves with distriution detils. They n tke the sene grph for grnted on per-proess level, nd n onentrte on the semntis of their distriuted pplitions. The Avodo sene grph n e ugmented y dtflow grph. Next, we will show how the notifition sheme used on the dtflow grph nturlly extends to the distriuted se Notifition First we hve to refine the notifition sheme to distinguish etween lol side-effets nd distriuted side-effets with respet to the semntis of the notifition funtions notify() nd evlute(). Consider the following onfigurtion (Figure 7). Three proesses, nd hve joined distriution group nd smll sene grph hs een onstruted. Now, proess hnges the vlue of field on node ( shll denote

6 Distriution Group Proess A Proess B Proess C ) Distriution Group Proess A Proess B Proess C notify() {.setvlue(); } notify_lse() evlute() evlute_lse() notify_lse() evlute_lse() notify_lse() evlute_lse() f3 f3 f3 Figure 7. The notifition protool is different for the originting fieldontiner nd the distriuted opies. the distriuted opy of node living in proess ). Beuse the node is distriuted, the new vlue is ommunited to field on the opies nd of the node. As desried in setion 2.2 the notify() nd evlute() methods will e eventully lled on node llowing the implementtion of side effets to the hnge in vlue. This my involve the modifition of other field vlues from within the notify() nd evlute() funtions. Lets ssume the notify() funtion on node will set new vlue to field on node s result of hnge to field. If we now lled notify() on eh opy of s result of the distriuted field hnge to, eh invotion would pply the hnge to field on node. This hnge would in turn e distriuted to ll opies. Thus field would e updted extly times on eh opy of node, where is the numer of distriuted opies involved, in this se three. This is lerly not desirle. If we tried to ttk this prolem y neither lling notify() nor evlute on the opies of the modified fieldontiner, we would not e le to implement the vfile node, rodued in setion 2.3, s distriuted node. The vfile node lods speified geometry s side-effet of hnges to it s Url field. If one proess hnges the Url field on distriuted vfile node, we wnt this to hppen on ll distriuted opies of the node, so tht ll opies represent the sme geometry. To resolve this prolem we split eh notifition funtion o two prts, one for lol side-effets nd one for distriuted side-effets. The distriuted side-effet versions notify() nd evlute() re only lled on one distriuted opy of the node. These funtions my perform modifitions to other fields without destroy- Fieldonnetion Distriuted Field Updte ) Overly A,B,C f3 Figure 8. The distriuted dtflow grph () exhiits the sme ehvior s stnd-lone version with identil onnetions () would. ing the distriuted notifition sheme. On the other hnd, the lol side-effet versions notify lse() nd evlute lse() will e invoked on ll distriuted opies of the node. They n e used to implement sideeffets whih hve to e exeuted unonditionlly, regrdless of whether field hs een hnged lolly, or s result of distriuted hnge, s in the se of the vfield node. The only restrition is tht the lol side-effet versions must not modify other field vlues Fieldonnetions Now tht we hve working notifition sheme for field hnges on distriuted nodes, we n investigte how fieldonnetions, rodued in setion 2.2, will ehve in distriuted environment. Fieldonnetions re not distriuted. They only exist in the proess whih onneted the fields involved. Consider the exmple given in figure 8. Three proesses, nd re tthed to distriution group nd shre smll sene grph of four nodes. Proess hs onneted field on node to field on node, while proess hs on-

7 neted field on node to field on node. Now we will tre the hin of notifitions tht will e initited when proess writes new vlue to field on node. Due to the fieldonnetion, field on node will immeditely reeive the new vlue. Then, ll distriuted opies of node will reeive the new vlue for field. Beuse of the fieldonnetion in proess, the field on node is notified immeditely of the hnge, nd will reeive the new vlue. This, in turn, will trigger n updte of on ll distriuted opies of node. If we now overly the three sene grphs in figure 8 to otin figure 8, nd look t the resulting dtflow grph, we immeditely see tht the ehvior of our distriuted dtflow grph extly mthes the ehvior we would expet from stnd-lone grph with identil onnetions. If field is hnged, the new vlue propgtes vi field to field. From the pplition progrmmer s po of view, uilding ertive distriuted pplitions is now just s strightforwrd s uilding stnd-lone pplitions. 4. The Detils: Relile Group Communition In this setion we desrie the underlying network infrstruture tht hs een used to implement distriution support for Avodo. Avodo uses the Ensemle [8] system from Cornell University whih is reimplementtion of Horus [12] nd s suh implements the proess group model [2] to provide relile multist ommunition etween distriuted proesses. Prt of Ensemle is the Mestro toolkit, olletion of C++ lsses tht provide higher-level group ommunition strtions to the progrmmer. Severl properties of Mestro enle us to provide the neessry infrstruture for onsistent dissemintion of pplition stte in Avodo Totl Messge Ordering Like other distriuted 3D systems, for exmple Repo-3D [11], whih provide onsistent shred pplition stte etween group memers, we rely on totl messge ordering to gurntee onsisteny in the presene of inrementl stte updtes. Mestro llows the delivery of messges to e totlly ordered, suh tht every group memer will reeive ll messges sent to the group in extly the sme order. Although this option rodues dditionl network lteny, it is onvenient nd powerful wy to gurntee onsisteny Group Memership Servie Mestro mnges the proesses whih ommunite in one group s list of group memers, lled view. Whenever new memer joins the group, or n old memer leves the group, the view is updted ordingly. View hnges re nnouned to ll memers, so tht every memer lwys hs n up-to-dte list of ll other memers in the group. Messges to the group re sent view tomi, suh tht messges sent in one view re gurnteed to e delivered to ll memers in tht sme view, nd only to those memers. New memers joining group will not see ny messges from old memers until the new view is instlled nd is ville to ll memers. Tking o ount the totl ordering properties of the ommunition, the replited pplition stte is gurnteed to e onsistent immeditely fter new view hs een instlled. At tht po ll old memers hve reeived extly the sme messges in extly the sme order, while new memers hve reeived no messges t ll. Now we n suspend ll norml messging etween memers nd initite speil stte trnsfer phse Atomi Stte Trnsfer Immeditely fter joining group, new memers hve no knowledge out the history of the group, ie. they do not possess opy of the shred pplition stte. Beuse we know tht ll old memers shre onsistent stte t this po, we n initite n tomi stte trnsfer from one of the old memers to the new memer. During the stte trnsfer ll other ommunition in the group is suspended. After the trnsfer the new memer hs extly the sme stte informtion s the old memers, nd we n resume norml opertion. This enles us to dd new memers to group t ny time without destroying onsisteny. The pprent drwk of this pproh is the suspension of norml ommunition during the stte trnsfer. If the pplition stte is rther lrge nd the trnsfer tkes onsiderle mount of time to omplete, this will e notiele to the user, s the pplition freezes with respet to updtes from the network. 5. Conlusions nd Future Work We hve presented Avodo, n pplition frmework for distriuted ertive VE pplitions. By providing trnsprently distriuted shred sene grph, Avodo shields most of the distriution speifi spets from the progrmmer. A distriuted dtflow grph effetively supports the development of distriuted ertive pplitions. We hve shown how the sophistited group om-

8 munition infrstruture ville tody n e used to gurntee onsisteny of the shred pplition stte in distriuted VE pplitions, even while llowing lte joins of group memers. There re severl res where Avodo n e enhned. During the tomi stte trnsfer to new group memers, ll other network ommunition is urrently suspended. An inrementl stte trnsfer protool tht would disseminte stte informtion to new memers prllel to ongoing ommunition in the group, would mke the proess of joining group more semless. However, gurnteeing onsisteny in the presene of suh trnsfer is diffiult prolem. Currently very strong delivery properties re required from the underlying network infrstruture to gurntee onsisteny of the distriuted pplitions stte. This is due to the inrementl updte of shred stte elements. One the initil stte trnsfer to memer is performed, no more solute stte informtion is sent, nd stte onsisteny relies on totl messge ordering. However, ertin rpidly hnging stte ttriutes, like positionl informtion for ojets for exmple, ould e trnsmitted s solute stte updtes nd would thus require muh weker delivery properties. This would help to redue the overll network lteny for distriuted ertion. Currently we re developing prototype for distriuted visuliztion pplition for the petroleum industry. This projet fouses on exploring different ertion nd nvigtion tehniques in distriuted virtul environment. The hllenge is the hndling of highly omplex senes inluding multiple levels of detil nd high resolution volume dt. This provides serious test se for our frmework nd will llow us to ssess the potentil nd limits of our pproh. Referenes [1] H. Aelson, N. I. Adms, D. Brtley, G. Brooks, W. Clinger, R. K. Dyvig, D. P. Friedmn, R. Hlsted, C. Hnson, C. T. Hynes, E. Kohleker, D. Oxley, K. M. Pitmn, J. Rees, G. J. Rozs, G. J. Sussmn, nd M. Wnd.!" # Report on the Algorithmi Lnguge Sheme. Tehnil Memo AIM-848, Msshusetts Institute of Tehnology, Artifiil Intelligene Lortory, Nov [2] K. P. Birmn. The Proess Group Approh to Relile Distriuted Computing. Communitions of the ACM Computer Siene Tehnil Report, (revised), Cornell University, Jn 1992, 36(12):36 53, [3] R. Crey nd G. Bell. The VRML 2.0 Annotted Referene Mnul. Addison-Wesley, Reding, MA, USA, Jn Inludes CD-ROM. [4] C. Crlsson nd O. Hgsnd. DIVE A Pltform for Multi-user Virtul Environments. Computers nd Grphis, 17(6): , Nov. De [5] C. Cruz-Neir, D. J. Sndin, nd T. A. DeFnti. Surround- Sreen Projetion-Bsed Virtul Relity: The Design nd Implementtion of the CAVE. In J. T. Kjiy, editor, Computer Grphis (SIGGRAPH 93 Proeedings), volume 27, pges , Aug [6] R. K. Dyvig. The Sheme Progrmming Lnguge: ANSI Sheme. P T R Prentie-Hll, Englewood Cliffs, NJ 07632, USA, seond edition, [7] C. Grimsdle. dvs - Distriuted Virtul Environment System. In Proeedings of Computer Grphis 1991 Conferene, [8] M. Hyden. The Ensemle System. Tehnil Report TR , Cornell University, Jn [9] W. Krüger, C.-A. Bohn, B. Fröhlih, H. Shüth, W. Struss, nd G. Weshe. The Responsive Workenh: A Virtul Work Environment. IEEE Computer, pges 42 48, July [10] W. Krüger nd B. Fröhlih. The Responsive Workenh. IEEE Computer Grphis nd Applitions, pges 12 15, My [11] B. MIntyre nd S. Feiner. A Distriuted 3D Grphis Lirry. In Proeedings of SIGGRAPH 98 (Orlndo, Florid, July 19 24, 1998). ACM SIGGRAPH, July [12] K. P. B. Roert vn Renesse nd S. Mffeis. Horus, flexile Group Communition System. Communitions of the ACM, Apr [13] J. Rohlf nd J. Helmn. IRIS Performer: A High Performne Multiproessing Toolkit for Rel Time 3D Grphis. In A. Glssner, editor, Proeedings of SIGGRAPH 94 (Orlndo, Florid, July 24 29, 1994), Computer Grphis Proeedings, Annul Conferene Series, pges ACM SIGGRAPH, ACM Press, July ISBN [14] Sense8 Corportion. World2World Relese1 Tehnil Overview, [15] Sense8 Corportion. WorldTooKit Tehnil Overview, [16] C. Shw, M. Green, J. Ling, nd Y. Sun. Deoupled Simultion in Virtul Relity with the MR Toolkit. ACM Trnstions on Informtion Systems, 11(3): , [17] P. S. Struss. IRIS Inventor, A 3D Grphis Toolkit. In A. Pepke, editor, Proeedings of the 8th Annul Conferene on Ojet-Oriented Progrmming Systems, Lnguges nd Applitions, pges , Wshington, DC, USA, Sept.26 Ot ACM Press. [18] VPL Reserh In. Relity Built for Two: RB2 Opertion Mnul, Jn