SHARED VIRTUAL WORLD TECHNOLOGIES

Transcription

1 SHARED VIRTUAL WORLD TECHNOLOGIES BY Xiuping Hu SUBMITTED N PARTML FULFILLMENT OF THE REQUatEMENTS FOR TRE DEGREE OF MASTER OF SCIENCE,4T DALHOUSIE UNIVERSITY HALIFLY, NOVA SCOTL4 M.4FtCH 12, Copyright by Xiuping Hu, 1998

2 Acquisitians and Bibliographie SeMces Acquisitions et semces bibliographiques The author has granted a nonexclusive licence allowing the National Libraxy of Canada to reproduce, loan, distriiute or sel1 copies of this thesis in microfoq paper or electronic fomts. The author retains ownership of the copyright in this thesis. Neither the thesis nor substantial extracts fiom it may be printed or othecwise reproduced without the author's permission. L'auteur a accordé une licence non exclusive permettant à la Bibliothèque nationale du Canada de reproduire, prêter, distribuer ou vendre des copies de cette thèse sous la forme de microfiche/film. de reproduction sur papier ou sur format électronique. L'auteur conserve la propriété du droit d'auteur qui protège cette thése. Ni la thèse ni des extraits substantiels de ceileci ne doivent être imprimés ou autrement reproduits sans son autorisation.

3 Contents List of Tables List of Figures Abstract Acknowledgement viii x xi 1 Introduction VRML and Multiuser Virtud Worlds Networking issues Related work ObjectivesoftheThesis Outline of the Thesis Introduction to VRML VRML file structure D graphics presentations Multimedia presentations in VRML Mapping texture (Image, Movie) Sound(Audio, Video) Animation in VRML Animation execution mode

4 LoQMOO II. Multiuser networking applications LogiMOO and LogiMOO II LogiMOO II System Architecture The classes of LogiMOO II The Manager The Multi-threaded Server The Multithreaded Client Synchronization Coordination with Linda blackboards Integrating LogiMOO II with virtual worlds D ynamically creating server side world scene file Storing avatar's messages in Blackboard Fetching avatar's messages from Blackboard Deleting avatar's messages from Blackboard Distributing the avatar's behavior Multiuser Virtual Worlds The VRML?.O/Java Interface Java applet establishes links to a VRML world How a VRML world sends events to an applet How an applet sends user events to VRML Shared Virtud World Extensions to VRML Defining a New Node type LLAvatax" Using the new node type "Avatar" Text-based User Interface Towards natural language control for Avatars in VRML The avatar Control Command Parser Improving Performance... 68

5 5.5.1 Reducing file sizes Increasing Rendering Speed Conclusion and future work A A proof of concept implementation Bibliography

6 List of Tables 5.1 The avatar control commands vii

7 List of Figures 2.1 VRML Scene Graph A plain roof A textured roof The Animation execution model [30] VRML representation of key-frame animation Avatar Walk Synchronization LogiMOO on the Web LogiMOO II System Architecture Basic Socket Operation Basic Linda operations Linda Term: Data Representation Shared Virtual World Avatar Control Commands Process Flow Chat... 68

8 To My parents' Spirit in heaven.

9 Abstract VRML(Virtua1 Reality Modeling Language) is a standard for 3D objects on the Internet. VRML 2.0 adds its 3D authoring ability, animation, interactivity and multimedia capability. An important extension of VRML is to support multiuser virtual worlds on the Internet, with applications from Virtual Gaming [18] to Virtual teaching [23]; from commercial applications ( oews/press_releases/press-i04.html) to science fiction story telling. Most existing multiuser VRML applications use an Intemal Interface, in which Java classes are called from inside a script-node. Our approach is based on a more flexible and more general Extemal Authoring Interface (EAI). We have implemented a prototype VRML virtual world which is integrated with the LogiMOO Java-based MOO's networking components, to enhance it with a more realistic form of virtual presence where users are represented by synchronized 3D avatars. The primary objective of the thesis is to present how VRML can be combined with Java [31] [33] to communicate with an extemal application; how VRML 2.0 can be further extended to support multiuser shared interactions and group work, and how to integrate the VRML world with Java components by adding and rnodifying methods of LogiMOO II; and proposing a scalable client-server solution for the rnultiuser virtual world. This paper discusses the technical details of our approach, the principles used in extending VFML to support the multiuser functionality as well as a prototype command language allonring the text-based multiuser system to be integrated with naturd language understanding and speech recognition in future work.

10 Acknowledgement This thesis was written under the supervision of Professors, Dr. Paul Tarau of University of Moncton, and Dr. blichael Shepherd of Dalhousie University. The author would like to sincerely thank Dr.Pau1 Tarau and Dr. Michael Shepherd for their excellent guidance and invaluable advice during the whole process of preparing this thesis. They are always the source of kindness, encouragement, knowledge and support. The author would like to thank Dr. Dennis Riordan for his careful reading and valuable suggestions and comrnents. Many thanks go to the faculties of the Department of Math., Stat., & CS. of Dalhousie University and the Computer Science Department of Moncton University. Sincere thanks also go to my friends, especially Dr. Yan-Nong Huang for his sincere encouragement and help, Mrs. Margaret Ryan for her carefully and patiently reading the thesis so that numerous mistakes in English were fortunately corrected and the writing was much colored, and Ms. Hoang-Anh Nguyen and Mr. Marc Hebert for their nice cooperation and valuable help in VRML and LogiMOO dunng the project developrnent. The author especially thanks her husband, Minqi, and son, Longfei, for their understanding, love, support, and encouragement. During the whole process of rny study, they have been always with me, with my heart, no matter where 1 was and where 1 am! 1 deeply love them, forever!

11 Chapter 1 Introduction VRiML, Virtual Reality Modeling Language, is an open, platform-independent file format for describing interactive 3D objects and worlds on the Intemet. VRML was introduced at the Second International World Wide Web Conference in the Fa11 of 1994 [19]. Its first version, VRML 1.0, is a static, platform-independent, hierarchical graphics modeling language. As a result of VRML 1.0's success, VRML 2.0, started in the Surnmer of 1995 and halized in August of 1996, is capable of representing static and animated dynamic 3D and multimedia objects with hyper-links to other media such as text, sounds, movies, and images [22]. The External -4uthoring Interface, developed by SGI [25] is a set of Java classes which allow users to use an euternal program to manipulate the VRML scene and communicate between a VRML scene and a Java applet. MUDs and MOOs (Multi User Domains - Object Oriented) are used for virtual presence in chat, game playing and education tools. Virtual Worlds can be seen as MO07s direct descendent with 3D visualization and provide a strong unifymg metaphor for various fonns of net-walk, net-chat, and Internet-based virtual presence in general [39]. LogiMOO, developed by Dr. Tarau and his research groups [39][34], is a BinProlog-based Virtud World running under Netscape. 1 have contributed to the development of LogiMOO II, a Java based Linda implementation which is based on

12 a minimal set of logic progamming components. Multiuser networked virtual worlds have been a major area of interest for years [28] [27]. Since multiuser virtual worlds are intended to have characteristics of the real world such as interaction, animation, and multimedia presentations, VRML is particularly suitable for implementing these kinds of applications (171. Our initial motivation was to create a shared virtual world using VRML on the Internet in which the users can chat, see, and interact with each other; users communicate by using human languages in a naturd and intuitive way. When this project is finished, LogiMOO II, as a Virtual World MO0 application on the Intemet, will be the integration of Prolog, Java, VRML, Natural Laquage Parser and Speech Recognition. My contributions to the project include: 1. Researching existing VRML Browsers and authoring tools to determine a suitable Browser and platform. 2. Using VRML 2.0 and EAI to create a Virtual World. 3. Using VRML 2.0% prototype to extend VRML 2.0's multiuser hinctionaiities. 4. Programming a Java Parser to parse avatar control commands. 5. Designing and implementing a user interface. 6. Integrating the VRML world with LogiMOO II by programming and modi&- ing methods of LogiMOO II to allow the system to recognize the VRML user interface commands and process the avatar's status information. Many capabilities of VRML permit building a large scale virtual world on the Internet [16] [7]. First, it is open. No Iicense is needed for using the language, and the language itself leaves many hooks that make it easy for VRML to connect to other high level languages such as c/c++, Java, ActiveX and Visual Basic, etc.. Secondly, VRML is totally platform independent (as long as VRML forms a common standard,

13 it Nill run on any platform). Thirdly, VRML is extensible. With VRML 2.0, the authors can define new node types. This characteristic provides VRML unlimited extensibility which couid be ody bounded by authors' imagination, creativity and programming skills. Lastly, VRML 2.0's interactivity, animation, and multimedia characteristics bring 'We" into the Web. Based on the above considerations, we decided to use VRML as our authoring language to build our virtual world and use LogiMOO II for the network components. The only thing that was rnissing was the glue that would connect VRML and the external application. This connection is very important for practical applications. For example, some companies already have 2D MO0 systems, and would like to e-xtend them to 3D virtual World while some other companies may prefer to buy generic client-semer software and only pay attention to their otvn virtual world applications. Successfully merging the two parts will have an important practical meaning. The critical requirements for this "glue7' include: 1. It rnust be an external programming language which can generate a VRML scenes and communicate to the scenes. 2. It must have a nice interface with Java because LogiMOO is implemented in Java. 3. tt must be platform independent. The optional requirements are: 1. It is object-oriented. 2. It cornes from reliable companies or organizations. 3. There are nice online discussion groups. 4. There are nice tutorials available on the Web or published papers. Several candidates have been evaluated, such as Sony's Java API [33], Liquid Reality's VRML 2.0 toolkit [15] and SG17s(Silicon Graphics Inc.) External Authoring

14 Interface, etc.. Sony's Java API is a very good tool to write a multiuser application, and Sony's Community Place is based on this APL But this Java.4PI is referred to as an internal interface,which means that the Java programs mn interndy to the browser, and are cailed from script nodes. So this intemal interface is not suitable to be integrated with an existing external application. The Liquid Reality developed by DimensionX, is another nice VRML 2.0 toolkit to create a simple multiuser virtual world. This toolkit provides a lot of useful Java methods for driving VFWL from Java, but it is also an internal interface. SGITs EAI is an Extemal Authoring Interface and consists of many Java classes. With its Java classes, the EAI provides a two-way communication between a Java applet and a VRML browser. As an extenid programming language, the EAI can generate a VRML scene from a Java applet and transfer the events between the Java applet and the VRML browser. It meets dl the critical and optional requirements mentioned above. So we selected the EAI as Our glue to integrate the VRML world with the LogiMOO II network application. The main challenge to create a multiuser virtual world is "sharing". In the same viewpoint, VVYSIWIS(What You See IS What 1 See) is aimed for. The main issues include how to automatically generate an avatar for a user; how to synchronize between a Java applet and a VRML browser as well as avatars' behaviors in the VREvIL browser; how to Save avatars' information on the server side; and how to broadcast the avatars' behaviors to al1 the users, etc.. We have developed a multiuser virtual world system prototype with up to 10 avatars. It can be scaled to a larger scale virtual reality system if some network problems are resolved [35] [IO]. In this thesis, we will discuss our approaches to resolving the above issues. First, ive introduce VRML 2.0's important characteristics for creating a vistual world; secondly, we present our extension to VRML 2.0 for supporting multiuser functionalities. That is, add a new node type "Avatar" and take advantage of the automatic viewpoint binding characteristic to dynamically generate avatars; thirdly, we discuss LogiMOO

15 II's network classes, some extensions of LogiMOO II to support the virtual world and some integration issues; lady, we introduce our Command Language Interface and future work about Naturai Language interface and use of Speech Recognition. 1.1 VRML and Multiuser Virtual Worlds -4 multiuser virtual world means that ail users' browsers must be kept informed about the current status of the world, and dl instances of the world must be kept consistent. VRML is a standard for 3D data on the Web. The scene descriptions (world files) are distnbuted over the Intemet by HTTP servers, using the standard HTTP protocol, and displayed on the individual host by VRML browsers such as SGI's Cosrno Player, Netscape's Live3D7 and Internet Explorer etc.. To extend VRML 2.0 for multiuser functionality, the system needs to add at least a new node type, *Avatar, so that the system can dynamically generate avatars for users. 1.2 Networking issues "JavaScnpt has no networking capabilities, and is therefore inappropriate as a script- ing language for applications t hat require networking." [20] [p For creating multiuser VRML applications, we must use an evternal programming language to implement networking abilities. Due to Java's platfonn independent feature and high degree of WWW integration, Java is the best candidate as a networking language so far. Before applying Java as an extemal application language to create a shared VRML world, many issues have to be taken care of. First of dl, Java has to have a good interface with VRML, so that they can easily communicate with each other; secondly, the semer needs to have the ability to maintain the world scene. When the scene changes, it will be broadcast to al1 of the browsers connected to the scene, so that the scene will aiways be consistent; thirdly, as all the browsers in the system might send

16 changes at same time, the synchronization has to be guaranteed by the system; the last is the performance issue. If every motion of the avatars is sent to the server, that means a huge amount of the data will have to be transferred between the client and the server, the system performance d l be down dramatically. So we have to balance the scene's consistency and the system performance. 1.3 Related work Virtual Worlds technologies pioneered by [5, 36, 44, 211' are becoming a part of 'standard setting' applications like Netscape Communicator or Internet Explorer. Applications of Virtual Worlds and MO0 technology usually combined with VRML navigation are spreading quite fast. Recently, World 1nc.k Active World is available for accepting immigrants. As long as yor are registered as an immigrant with your payment, your properties will be protected by virtual world law! Microsoft has also released its V-Chat, a multiuser virtual Chat. Among the few virtual worlds, the most impressive sites are: Sony's Community Place, http : //vs. sony. CO. jp/vs-e/vstop. html. Sony7s Cyber Passage is the first VRML 2.0-cornpliant Browser. We can ex- plore the world by using either keyboard or mouse. CyberPassage 2.0 has multiuser-world-browser capability, but it uses SEVRML(a Sony Enhanced VRML) rather than VRML 2.0 to implement the multiuser world. Community Place, a new version of the Cyber Passage, is based on the VRML 2.0 standard. In addition to VRML 3D models, the browser uses most VRML's multimedia characteristics to show sound, image, movie in the scene, and uses Java Script language to animate the world and interact with the world. This browser supports the multiuser access by using society server Community Place 'The cybergate is replaced by CommUILity Platfom; The cyberpassage is replaced by Community Place; alphaward is replaced by Worlds Chat Gold, the Moondo is no longer supported.

17 Bureau [37]. This browser is a ctclick-basedn application and only supposes one way communication between VRML world and Java. SGI's Cosmo Player, http ://m. sgi. corn/. SGI, the most influentid VRML player, has introduced Cosmo Player that works as a plug-in for Netscape Navigator and Microsoft Explorer for multiple platforms. Cosmo Player's extemal package and Netscape's LiveConnection to- gether provide an EAI (Extemal Authoring Interface), which provides a two way communication between W IL worlds and Java applets. One of the CosmoPlayer's EAI applications is Transaction Information System's financial visudization software ( t isny. corn/). In this application, Java, VRML, and the E.41 were used as a development platform. Liquid Realiw [I 11. Liquid Reality is a VRML toolkit from DimensionX. It consists of a set of Java class libraries that give users VRML 2.0 functionalities to create cornpliant viewers, tools, and solutions. In addition, the toolkit is extensible by using Java, which allows us to create custom nodes to meet our needs. One of the Liquid Reality applications is -4lligator Descartes's interfacing Java and VRML demo (151. This demo uses Liquid Reality 1.0bll and VRML 2.0 to create a simple client/server communication. The server performs some basic ta&, such as handling users' login and logoff; broadcasting any movements made by any connected users; passing basic messages among users (chat-room style) and the ability to see which users are currently connected. The client is a stand-alone Java applet. It c m open and close a connection to the server, receive information and messages from the server, navigate the world and interact with O t her users. This demo uses a cube as an avatar, and events are generated only from the browser instead of rom both the browser and the applet.

18 a Worlds 1nc.k WorldChat, http : //un. worlds.net. The WorldChat demonstrates a multiuser virtual environment linking events via the htemet. It was created by using Virtual Reality Modelùig Laquage+ (VRML+), a proprietary extension of VRML. In this world, the users cm follow the Internet to the Worlds Chat Space Station - a visually rich 3D virtual environment, and chat with other red people from d around the planet. These other people can be seen in cyberspace as "avatars" - the visual embodiments that these people have chosen for themselves. 0 Black Sun's CyberGate, http: //vv3. blacksun. com/index. html. CyberGate is Black Sun's custom browser with multi-user support. It includes CyberHub(renarned as Cornmunity Se~er now), a server, and a CyberKit, a toolkit for building virtual worlds [8]. The CybertIub is a multi-user server that dows chat, communication and movement of avatars, al1 on the top of VRML worlds. The Community Semer is dinerent from other multiuser virtual worlds. It allows developers to build rnulti-user capabilities into new and ex- ist ing applications [18], as the Company claimed t hat "Worlds make movies, we make cameras and lighting". MUD/MOO environments fa11 into two general categories. Eniironments such as "CyberGate" by BlackSun [5] and "Community Place" by Sony [36] belong to the category of point and click graphical environments. These completely avoid the need for NL processing as the only text involved is that for chatting with other avatars, and al1 movement and actions are completed with mouse point and click actions. The second category, which environments such as MediaMOO [2] and the Avatar MUD [l] fa11 into, includes teut-based systems. In these systems, dl movements and actions are controlled by Natural Language in text processing andior speech recognition.

19 1.4 Objectives of the Thesis The emergence of VRML makes it possible for people to author 3D data on the Internet. People dernonstrate an unprecedented ent husiasm to the new technology. In only three years from the first VRML proposa1 to the current VRML 2.0, many virtual world applications have been developed and some of them are available on the Internet. MUDs and MOOs are used for Mrtual presences in chato game playing and education tools. Virtud Worlds can be seen as MOO's direct descendents with 3D visualization. Some MO0 applications use VRML to provide more attractive contents and scenes. hhtltiuser Virtual worlds, as tools of virtual chat, virtual education, virtual gaming, and virtual shopping, etc., show good prospects. The research work presented in this thesis has been motivated by the above observations and the high demand for new Internet technologies. The primary objective of the thesis is to present a multiuser 3D virtud world by extending VRML 2.0 and integrating a VRML virtual world with LogiMOO II. For this purpose, the author ha. presented how VRML 2.0 can be applied to author a 3D virtual world; how to use VRML 2.0's animation and multimedia presentation to provide a virtual reality world; how VRML can be combined and communicated with an extemal Java program; how to extend VRML to support multiuser, shared interactions, and group work; and how to integrate the virtual world with an existing MO0 application. And the author has also described the scalable client-server solution for the multiuser virtual world based on LogiMOO semer's blackboard and broadcast ing. It has been shown that a Java based Linda coordination language using associative search can be used to synchronize multiuser shared virtual worlds. The author has used the EAI in an innovative way, together with networking to make it possible for multiple users to see each other's avatars in a 3D virtual world, and also prote typed a command interface ready for natural language based interaction with VRML worlds, and hally integrated the work with other Java based LogiMOO II modules supporting online chat and execution of remote commands.

20 -4nother objective is to create a text-based multiuser system which will be integrated with Natural Language Understanding and Speech Recognition connected to the VRML world. The implementation does not include the NL understanding itself, which is part of LogiMOO II. 1.5 Outline of the Thesis Chapter 2 introduces VRML 2.0's basic concepts and its animation model. The topics include: VEtPvIL file stmcture, VRML 3D Graphics presentation, Multimedia presentation in VRML, and Animation in VRML 2.0. Some examples of code from the project are presented. Chapter 3 analyzes LogiMOO II system's technical issues and progamming. The topics include Interface, Manager, Connecter, Semer, SeMce, Servant, and Client. Chapter 4 discusses integration of LogiMOO II and virtual worlds. Chapter 5 presents the Multiuser Virtual World's design and implementation issues. VRML 2.0/Java Interface is introduced by using example code. Some extensions to VRML 2.0 for multiuser application are also discussed. A Multiuser Virtual World prototype is presented; finally, some performance issues are discussed. Conclusions and future work are presented in Chapter 6.

21 Chapter 2 Introduction to VRML 2.0 The World Wide Web is baçed on networking protocols, such as HTTP, and a number of formats that describe how text, images, sound, video and audio are represented across the different types of computers connected to the Internet. Since the Intemet came out, the most important format has been HTML which allows text and graphics to be displayed together by using a twedimensiond page metaphor. Since the Spring of 1995 when VRML [22] was introduced at the Second International World Wide Web Conference, it has been experiencing explosive growth, and now is considered a core Web technology. VRML is an open, platform-independent file format for descnbing interactive 3D objects and worlds. VRML is designed to be used on the Intemet, Intranet, and local client systems. VRML is also intended to be a universal interchange format for integrated 3D graphics and multimedia. VRML may be used in a variety of application areas such as engineering and scientific visudization, multimedia presentations, entertainment and educationd software, web pages, and shared virtual worlds, etc.. This project uses VRML 2.0 as an authoring tool to create a 3D virtual world. VRML 2.O provides rich functiondities to present a dynamic scene wit h multimedia and animations. Understanding VRPIIL 2.0 makes it easy to understand the rest of the chapters. This chapter will briefly introduce this new technology and some important characteristics used in this project.

22 2.1 VRML file structure VRML is a scene description language. Though VRML is a kind of computer lan- page, it is not a fully functional programming language. VRML files are not compiled but parsed and interpreted by VRML browsers (such as Netscape, CosmoPlayer and Explorer etc.). VRML is basically an open file format and has many hooks within it to interface with other programming languages through script nodes or EdI, such as Java, C++, SmallTalk, etc.. No License is needed for using this language. VRML files are ASCII text files. Like an HTML file, after a VRML file is loaded, the result can be shown on the Browser immediately. Unlike an HTML file, a VRML file has no tags and can describe 3D scenes and animations. A VRML file consists of four main components: the header, the prototypes, the scene graph, and the event routing. 1. The file header A VRML browser has to be able to recognize VRML files in order to parse them. This is accomplished with a simple file header string at the beginning of each VmIL file. # VRML V2.0 utf8 The identifier utf8 represents the character set using the UTF-8 encoding of the ISO standard. 2, Prototypes VRML provides a set of node types that covers basic needs of the world authors. VRML 2.0 adds an important feature "prototyping". A prototype allows a set of VRML node types to be defined in terms of other pre-defined nodes. We cal1 these node types user node type. The new feature not only provides an object modeling mechanism that allows an object to be defined once and re-used later, but also provides the world authors a powemil tool to create their own node types.

23 A prototype is defined in a VRML file by using the keyword PROTO. The syntax is as foilows: PROTO prototype-name [ event In event In-t ype event -name eventûut eventht-type eventout-name exposedfield exposecifield-type exposecifield-name field f ield-type f ield-name 1 > The body of the PROTO node definition The PROTO declaration consists of three parts: the narne of the PROTO; the parameters of the PROTO; and the body of the PROTO definition. The name is any legal VRML variable name; the parameters include eventh, eventout, exposedfield and field. The eventin, eventout and exposedfield can accept and/or send events. The field is a global variable for the body of the PROTO; the body of the PROTO is any legal VRML file. After defining a PROTO, we can instantiate it as any normal VRML node in the same file; or use EXTERNPROTO to include an extemal PROTO file in a world file..4nd also a prototype can accept eventins and/or send eventouts through parameters. In this way, we can create VRML libraries and share Our work in multiple projects. 3. The Scene Graph The scene graph, a VRML file's main body, contains nodes which describe objects and their properties. The scene graph can contain any combination of the following nodes: any number of prototypes; any number of children nodes; and any number of ROUTE statements. VRML 2.0 uses a tree-based node

24 traversal strategy for ordering its scene gaph. Any VRML scene has a root node, the root node has chiidren nodes, and each child node, in turn, can have its onm children nodes, and so on. In this way, A well-organized hierarchical stmcture is formed. (Fig 2.1) O Any type of Nodcs: Tr~ufm Node or Iasmceof prototype or R0UTEst;urmcnt a Inline nodt a S d node eu.. Figure 2.1: VRML Scene Graph 4. Event routing Events are actions that can be performed upon or generated by some nodes in a VRML world. Some VRML nodes generate events in response to environmentai changes or users' interactions. -4 ROUTE is a sentence that logically connects the events. Events are classified into two types: eventin and eventout. Once an eventout is generated, it will be sent to the ROUTE destination, eventin. In general, the sensor nodes and SCRIPT nodes are nodes that generate events. To create very cornplex behaviors, such as a networked multiuser environment, it is necessary to create an interface between the virtual world and an eutemal application. The SCRIPT node has been dehed to provide an interface between

25 W1L and some programming languages. The ided event model will process al1 events instantaneously in the order they are generated D graphics presentations The modeling of an object's shape and geometry is a large and diverse field. Some approaches attempt to model the exact geornetry of real world objects. Other methods seek to create simplified representations. VRML systems sacrifice details and exactness for simplicity for the sake of the rendering speed. VRML provides primitive objects, such as cubes, cones, spheres, and cylinders. These objects can be slightly deforrned by scale, rotation and translation to provide more interesting ob jects. Using these objects dramatically reduces rendering time compared with using polygons. VRML supports 3D PolyLine and PolyPoint. A PolyPoint is simply a collection of points in space. -4 Polyline is a set of vectors that fom a continuous line. We can construct very compelling shapes by using polygons. Generally, these points are made by modeling tools. For some very complicated scenes, these files will be very huge for rendering. -4n alternative way to the above polygons is the Extrusion node. The Extrusion node specifies geornetric shapes based on a two dimensional cross section extruded along a third dimensional spine. The cross section can be scaled and rotated at each spine point to produce a wide variety of shapes. For the same scene, compared with using IndexLineset, Extrusion reduces the file size dramatically. 2.3 Multimedia presentations in VRML Mapping texture (Image,Movie) Texture mapping in VRML is basically the same as texture mapping in al1 other areas of 3D graphics. The Texture map is a two-dimensional image that extends from O to

26 1 in both horizontal and vertical directions. The te?cture can be an IrnageTexture, PkelTexture, or WovieTexture. An ImageTexture is a node's Appearance field that defines a texture map. When texturing a shape the texture is applied by default to each of the faces of the shape. For example, the roof of the tower in our scene is textured by a jpg file: Transf orm ( Shape { appearance Appearance { texture DEF roofirnage ImageTexture C 3 url " jpgs/roof. jpg" texturetransform TextureTransform ( 3 > scaïe 1 30 > )#End Roof geometry DEF Roof IndexedFaceSet ( The -4ppearance node is used to define some attributes of a shape, such as the color, Iight, image source, etc.. The ImageTexture node specifies the image data in its url field. The TextureTransforrn node aliows us to translate, rotate and scde the texture coordinates before they are applied to a shape. Here we use the scale field to speciq the texture scale of (1,30) on the object. This means that the texture coordinates (0,O) to ( 14 are multiplied by 1 in horizontal direction and 30 in vertical direction, which results in coordinates rom (0,O) to (1,30). The texture is thus

27 repeated 30 times in the vertical direction. The geometry node defines an object to be texture mapped. Figure 2.2 and the figure 2.3 show a roof before and afier being texture mapped, respectively. Figure 2.2: A plain roof The pixeltexture node dehes a 2D image-based map in its uimage" fieid. The MovieTexture node defiries an animated movie texture map. Besides the fields defined in the imagetexture node, some other fields that reflect the video characteristics are given, such as the speed at which to play the movie, the start and stop time, audio node played with the movie, etc Sound(Audi0, Video) VRML 2.0 allows us to implement spatialized 3D audio in a world. A spatial audio is a sound that has been proceçsed to give audience a sense of the location where the virtual sound come from in a virtual iistening space. There are two kinds of sound nodes: sound node and sound source node. The

28 Figure 2.3: A textured roof sound node specifies the sound spatial parameters, such as the sound location and the range in which the sound cm be heard. The çound source node defines the sound file to be played, it can be AuclioClip node(audio file) or MovieTexture node(video file) in which a sound file can be defined in its url field. A sound node example is the Sound node in our "girl.wrp' file, the audio file plays when the girl is dancing: DEF dancesound Sound ( location maxback 500 maxfront 500 minback 500 minfront 500 source DEF DanceSouadSource AudioClip { loop url FALSE '8casschor. vav"

29 1 1 spatialized FALSE The location field tells the location of the sound emitter in the local coordinate system. The maxback, maxfront, minback, minront define the range of the sound tieard from the sound source location. The minfront and minback define a smd elliptical area where the sound cm be heard at its full volume; while the mdront and madack define a bigger elliptical area. -4n outer area is formed between the outlines of the bigger area and the small area. In the outer area, the sound's volume fades as the distance from the location inmeases. Out of the outer area, there is no sound to be heard. We define rnidack = minfront and maxback = rnaxfront to produce direction-less sounds that fade with distance from the location. The url field of the Sound node specifies a.wav file that is one of the audio file formats that the VRML can recognize. In the spatialized field, a FALSE means that the sound is like arnbient background sound. 2.4 Animation in VRML 2.0 VRML 2.0 is al1 about animation and interactivity [29]. VRML97 includes a variety of animation objects called Interpolators as well as object-controlling Nodes such as Sensors, Routes and Scripts. Using all of them allows users to author the virtual world with full interactivity and animation Animation execution model VRML 2.0 provides the possibility of combining keyframe animation and event-driven animation. A user's input generates an event that controls the scene's animation which is achieved by some kinds of Sensors, fields and ROUTES. VRML 2.0 execution model is described in figure 2.4:

30 Figure 2.4: The Animation execution model [30] Three basic factors of the execution model are Events, Routes, and Triggers. Any browser has an interna1 clock. This clock wiil start when the scene is loaded. At each tick of the clock, al1 sources of events will be checked. When a trigger is fired, an event is generated, and then the browser will catch the event and check dl ROUTE sentences to see if the event is wired to any other nodes. If so, the browser sends this event value to the correspondhg field of the destination node in ROUTE staternent. This process continues until the scene is replaced. 1. Event Events are data messages sent to or received from other nodes to change some object's statu within the scene. An eventin field of a node means that the node can receive events; an eventout field of a node means that the node can generate events; an exposedfield declaration implies that a node is able to receive events (eventh), and generate events (eventout). When an event occurs, the node which generated the events outputs a value

31 or array depending on the event type, the value or array is routed to the node which receives the event. In tum, the receiving node alters the value of the field specified by the event. So the receiving node's status is changed. 2. ROUTE The connection between a node generating an event and a node receiving the event is called a ROUTE. -4 node that generates events c m be routed to a node that receives the events by using the following syntax: ROUTE NodeName.eventOut-changed TO NodeName.setaventIn-Iame With this rnechanism, fields can be linked to enable eventouts of some nodes to change evenths of some other nodes. 3. Trigger A trigger is a mechanism that generates eventouts when some tire conditions are satisfied. Animations can be started by a variety of eventouts generated by triggers. Based on the eventout types, triggers are divided into three categories: Native trigger, Geometric sensor trigger, and Script Node trigger. (a) Native trigger Triggers that generate SFTime eventouts are called the native triggers. The Native triggers include: TouchSensor Collision ProximitySensor A TouchSensor defines an object. When the object is clicked, a SFTime eventout WU be generated. Collision node is a grouping node that specifies the coilision detection properties for its children and t heir descendants. When a collision occurs between a user's avatar and a node's geometry, a SFTime (collidetime) eventout will be generated. A ProximitySensor dehes a region ( generdy a box for simplicity). When a viewer entas or exits the region,

32 a SFTime event Out will be generated. Vis ibilit ysensor A VisibilitySensor defines a box-shaped area, then the sen- sor will detect whether the region is visible or not. When the box enters into or exits from the viewer's viewpoint, a SFTime eventout (entertime or exittime) wiu be ge- nerated. The SFTime eventouts generated by above native trigger can be routed to starttime fields of TimeSensors, so that the TimeSensors will be activated. (b) Geometric sensor trigger A geometric sensor is a special class of sensor that not only tracks the motion of a user's pointing device but also moves the objects within the same group as the Sensor. There are three types of geometric Sensors: PlaneSensor CylinderSensor SphereSensor A PlaneSensor lets users move objects in a virtud XY plane. A CylinderSensor maps the movement to the surface of a concept ual cylinder. A SphereSensor maps the movement to the surface of a conceptuai sphere. (c) ScriptNode trigger [3].4 ScriptNode trigger is a logical trigger which is defined in a Script node, and can generate a variety of eventouts, such as SFBool, SFTime, SFFloat, SFInt, etc.. a Script node can be used to control simple ScriptNode triggers to get desired behaviors depending on some programming logics. 4. KEYFRAME animation in VRML -4 general scenario for video animation is a keyframe animation. In the VRML world, we can describe a keyf'rame animation with Interpolators, sensors and

33 routes, etc. The scheme of a keyframe animation can be seen in Fig TimeSensor starts when it is triggered by either a native trigger or a ScnptNode tngger, or when the scene is loaded. As long as a TimeSensor starts, it will produce a set of fraction outputs. While these fraction outputs will be fanned out to the keys and keyframes stored in the Interpolator, and then the keyfiames are routed to the fields of the objects by some ROUTE statements. With the changes of the keyframes, an animation is performed. Key KeyVdue Native Trigger f nterpolator Animated Object Figure 2.5: VRML repres entation of key-fram e animation The following is a simple keyframe animation example - a shaking fan. DEF Fun Transfom { #define a Fun object translation scale rotation 1 O O 0.7 children C

34 DEF Timer TimeSensor { cycleinterval 0.2 loop TRUE 3 DEF Rot OrientatioaInterpolator { key C0.0, 0.11, 0.22, 0.33, 0.44, 0-55, 0.66, 0.77, 0.88, 11 keyvalue C 1 O O 0, 1 O O 0.28, 1 O O -0.28, 1 O O 0.28, 1 O O -0.28, 1 O O 0.28, 1 O O -0.28, 1 O O 0.28, 1 O O -0.28, 1OOOJ > ROUTE Timer. f raction-changed TO Rot. set-f raction ROUTE Rot-value-changed TO Fun.set,rotation The TimeSensor is initidized with a Ioop set to TRUE, the starttime and stoptime default to O. In this case, the TimeSensor wi11 start to generate events when the scene is loaded and will not stop until the scene is repiaced. Within the cycleinterval, e.g. 0.2 seconds, the tirnesensor generates an average of 10 eventouts, e-g. one eventout each 0.02 seconds. Whenever the TimeSensor reaches its cycletime (cycletime = (start Time + cycleinterval*loop~times)), it will repeat the above process. The tirnesensor is as follows: TimeSensor C exposecifield SFTime cycleintervaï 1 exposecifield SFBool enabled TRUE exposeclfield SFBool loop TRUE exposecifield SFTime starttime O exposecifield SFTime stoptime O event Out SFTime cycletime O

35 eventht SFFloat f ract ion-changed eventht SFBool isactive event Out SFTime the The cycleinterva1 defines the TirneSensor's active cycle in seconds. At the beginning of each cycle, an eventout cycletime is generated and usually is used for synchronization. During a cycle, an eventout the wiil be generated every second in the "time" field, we can use this eventout as an interna1 dock of the scene. The field fraction-changed contains a value between O and 1 and this value corresponds to the time proportion of the cycle that has elapsed. In this example, the cycleint erval is 0.2 seconds, if 0.1 second passes, the fraction-changed value will hold 0.5. The sentence "ROUTE Timer. fraction-changed TO Rot.set fraction" will route this fraction-changed value to the corresponding field of the key, 0.44, while the corresponding value-changed in Rot will be 1 O O The second Route sentence "ROUTE Rot.value-changed TO AnimerEventail.setrotation" will assign the value (1 O O 0.28) to AnimerEventaii1s rotation field. In this process, as the rotation of the AnMerEventail is cyclically changed, the animation is generated. 5. Building logic into the VRML world When native triggers are not expressive enough, using a scnpt node can help to build decision logic into the VRML world. A script node receives incoming events and passes the event values to the function specified in the "ud' field. After processing the values, a script node can output an outgoing eventout to some other nodes. In Javascript, each eventh has the same name as its associated function in the scnpt node. When an eventin arrives, the browser calls the function and passes

36 the event 's value and t ime stamp to the function. The t ime stamp indicates the tirne when an event happens. For example, we have an eventin SFBool "walk" in our avatar's PROTO, and in the script, nre have a function "walk": function walk(value, time) < enablewalk = value ; This function has the same name as the eventin. When a "walk" event turns TRUE, the function sets the enablewalk eventout and some other values, so it results in a TirneSensor set to 'cactive" and then the start of a walk animation. 1. Maintainhg animation status When an avatar is represented in the scene, it is in some states (stopping, walking, etc.). With a script node, we could set some logic to keep track of the avatar's states, so that we can make decisions on how to control the avatar's actions depending on the user's input. In our scene, we maintain four avatar states: stopping, walking, walkloop, and nodding. field SFInt 32 state O field SFInt32 stopping O field SFInt32 walkîng 1 field SFInt32 walk,loop 2 field SFInt32 nodding 3 When an event that changes an avatar's state is received, the program first checks if the avatar is already in the state. If so, the program will do nothing, othenvise, it will send an eventout to set the new state and activate a new animation. For example, we start wdking by the following function:

37 function walk(value, time) < if (state!= walking) ( state = valking; stepsout=steps ; enableiialk = TRUE; In each waiking cycle, we need to make some decisions about the avatar's walking path depending on its state: if (state = valking) C steps += 1; stepsout = steps; 1 else if (state == walk-loop 1 C > //set,path In this way, an avatar's behavior can be intellectualized. 2. Walking synchronization To simulate a human's walking, the waving of an avatar's arms and legs and the moving of the avatar body's position have to be kept in synchronization. Due to the CPU speed and complicated scene interpretations, it is very hard to keep synchronization by only using OrientationInterpolator and PositionInterpolator. Script nodes can provide more capabilities and flexibility to keep the synchronization, especially in the case of communicating with an exterual program. In Our scene, we use both interpolators and scipt nodes to achieve the synchronization. Here is the Walking synchronization process:

38 First, we define a ScriptNode as a walking's trigger and a thesensor as the waking's synchronization clock. When the "enablewalk" in the ScriptNode is set to TRUE, the tirnesensor starts to output an eventout (SFTime) in each 0.25 seconds. Walk animation DEF Walk-The-Sensor TimeSensor { cycleinterva1 1 enabled FALSE loop TRUE 1 The four OrientationInterpolators are defined as waiking engines to control the arms and legs animation. Take one OrientationInterpolator as an example: DEF Lef tlegrotat ioninterp DrientationInterpolator C key [ 0, 0.33,0.66, 1 1 keyvalue [ IO O 0.03, 1 O O -0.4, 1 O O 0.03,1 O O The ROUTE statements will wire the tirnesensor, O~entationInterpolator, 'the arms and legs nodes together. Each tirnesensor's output nrill trigger a new "key" value, the corresponding "keyvaiue" will be sent to the setxotation fields of the arms' or legs' node. ROUTE AvatarControl.enableWalk TO Walk,Time,Sensor.set,enabled ROUTE Walk,Time,Sensor. f ract ion-changed TO LeftLegRotion1nterp.set-fraction ROUTE LeftLegRotationInterp.value,changed TO left,leg.set,rotation

39 To synchronize the arms and legs waving and position moving, we take ad- vantage of the tirnesensor's cycletime field. The field will generate a SFTime eventout whenever the Interpolator loops to its start point. We use another ROUTE sentence to wire the cycletime time with a script node's function walkcycleend(). ROUTE Walk-Tirne-Sensor. cy cletime TO Avat arcontrol. ualkcy cleend So in each start point of the TirneSensor's cycle, the script oode's walkcy- cleend() function is called. we set the avatar to go one step forward in the script function: function walkcycleend(value, the).c... S.. // change avatarpos value to forvard one step. 3 avatarposout = avatarpos ; Note that the avatarposout is an eventout field. After it gets a new value, it will be routed to the Body node by the following ROUTE sentence. ROUTE AvatarControl.avatarPos0ut TO Body.set,translation After executing this ROUTE staternent, the Body's translation field is set to the new value of the avatarposout. The execution logic is shown in figure 2.6.

40 From Java Applet \ wak = me EventIn IKIY / Object eventout Orientation Interpolator F p 1 ArmsandLegs Animations 1 Object Figure 2.6: Avatar Walk Synchronization

41 Chapter 3 LogiMOO II - Multiuser networking applications In this chapter, the LogiMOO II system design architecture and some programming details related to the networking realization will be presented. LogiMOO II is a Javabased Linda implement ation based on a minimal of logic programming components. Linda is a high level paradigm for a expressing pardel computation that provides an abstraction of communication, coordination and task creation through use of a globally accessible object level store. Linda is not a language, but consists of a number of parallel operators that are used to augment an existing sequential language. The Linda operators, the in, rd and out, provide access to the globally accessible object store called a tuple space. LogiMOO II uses Blackboard as the tuple space and provides programs that expresses the Linda operations in terms of methods in a Java class. Our multiuser virtual world is developed on the base of LogiMOO and LogiMOO II. Using Linda implementation in a multiuser virtual world provides a persistent robust, synchronized and fast communication among shared users on the Internet. To understand the overall system implementation, it is necessary to review LogiMOO and LogiMOO II projects ht, and then further to discuss related issues about how to implernent a shared virtual world and how to integrate LogiMoo, virtual world and Linda implernentations.

42 3.1 LogiMOO and LogiMOO II This section is a review of LogiM00 [14, 42, 4011, which is a BinProlog-based Virtual World mnning under Netscape for distributed group work over the Lnternet and usercrafted virtual places, virtual objects and agents. Netscape is used to implement CGIbased BinProlog remote toplevel interacting with a remote LogiMOO server (Fig. 3.1). Objects in LogiMOO are represented as hyper-links (URLs) towards their owners' home pages where their 'native' representations actually reside in various formats (HTML, VRhlL, GIF, JPEG etc.). Embedding in Netscape allows advanced VRML or HTML kame-based navigation and multimedia support, while LogiMOO handles virtuai presence and acts as a very high-level universal object broker. Figure 3.1: LogiMOO on the Web LogiMOO II is a Java based Linda implementation, based on a minimal set of logic programming components (associative search). Associative search consists in trying to unify a pattern, let's Say a(x,x) with al1 the objects on the blackboard. If there are following items in blackboard a(1,2),a(2,2),a(173),a(3,3), only a(2,2) and a(3,3) lelectronically adable and remotely executable with Netscape fiom URL http : //clement. inf O moncton. ~a/~tarau/logimoo-

43 would quali& to be returned as a result of the search. For simplicity, "in" operations always retum the first matching object, i.e. a(2,2) in this case. To retrieve dl, without removing them, we can use dl(a(x,x),xs) returning Xs=[a(2,2),a(3,3)]. It dlows us to commuaicate bidirectionally with the existing LogiMOO fiamework and dlows the creation of combined Java/Prolog mobile-agent programs. Ln particular, the more resource consuming CGIs are now replaced by Java applets. LogiMOO II's Java applet keeps track of the current users on the server, it can play the role of a chat line and also communicate with the user's avatar and the VRML environment. Whereas LogiMOO shows objects as hyper-links, LogiMOO II is using VmfL to display most of the world's objects. This technology gives users a more concrete understanding of the world surrounding them. With the LogiMOO natural language interface [41, 32, 121, speech recognition could be used to control the objects of the VRML environment in the future. 3.2 LogiMOO II System Architecture Figure 3.2 shows the overall system architecture of LogiMoo II. This architecture is based on the replicated architecture of application sharing [43]. Each client runs a copy of the application, the client's input is sent to the server that then distributes the input to al1 other clients in the system. There is a class Manager which monitors the system running, manages Connector threads, and processes incoming Linda commands. The class Server, class Service, and class Connector cooperate to process server side communication tasks; the class Servant and class Client execute the client side tasks. The Blackboard is used as a global tuple space which stores al1 system information and avatars' status data for consistency.

44 Servant Clicni Appkt Figure 3.2: LogiMOO II System Architecture 3.3 The classes of LogiMOO II LogiMOO II is a Java version MO0 application. As shown in 3.2, the main networking classes in LogiMOO II include Manager, Server, Service, Connector, Client and Servant. These classes cooperatively work together to execute a high performance multi-t hread multiuser application The Manager The class Manager is a stand alone program which sits on the top Level of the system and manages the overall running of the system. It launches Connector threads and controls their starts and terminations. It acts as a rnultipurpose processor, plays a manager, a client, and a server roles. As a Manager, it schedules the Connector threads and manages the Clients and the Servers that are connected to the system. 1. Schedule a Connector thread

45 The class Comector is the super class of Client, Server, SeMce, etc.. The class Manager will schedule all threads for Comector class as well as ail its children classes. public static Thread makethread(comector connector,string name) { Thread this,thread=thread. currentthread0 ; Thread connector,thread=new Thread((Ru~able)connector,name); if (IO.peer==null) connecter-thread. setdaemon(true) ; connector.thread=connector,thread; return connecter-thread; public static void ninconnectorthread(connector connector,string name) < Thread connector,thread= makethread(connector,name) ; > connecter-thread. start 0 ; S. Start a Server When a Semer(port) command is entered on the system console, the Manager class first registers the Server as a Connector thread, and then starts the thread. -4fter the server is started, the Server will deal with al1 connections from clients. for(;;) String op=((fun)t).name() ; else if(op.equals("semer")) C > > thisport=(int) ((Int) arg).val; RegisteredServer s=new RegisteredServer(thisPort); Connector.serverRegister = true; s.runo;

46 3. Create a chat connection. A chat connection can be created from the Clients or the Manager, so that the manager can talk to any user in the system at will. if (op. equals ("talk") { talk(arg, that Adàr, thatport ) ; > static void talk(tem who, String thatad&, int thatport ) C 1 // loop forever to accept the chat message for(;;) { 3 IO *print(">") ; // The system comm;ind prompt sign String s=io.readln(); if (s==null) break; // read a command from the console // exit if the command is nul1 // accept new command if the string is empty if (s.length()==o) continue; //Construct the Term message from the chat string Term Mes=new Fun("mesn, who, (new Const (s)).nametolist 0 ) ; //Create a Client Client c=new Client (thataddr, thatport,mes. tostring0 ) ; // stop output into output area IO. shoaoutput=f aise ; // schedule a thread for the client to send the chat message //to the Server.. startconnector(c) ; // resume the output and ready for accepting the new command. IO. ~ho~output-t~e ; As a client, the Manager talks to its server with "out", 'cin" and "ail" commands which are implemented independently. For example, if a query "all(x)" is entered

47 on the system console, the Manager will accept the command and then create a Comector thread to send this query to the server. The server will actually execute the query, and then send the mwer to the Manager by displaying it on the system console. // Create the client to send the query to the server. Client c=new Client (thatad&, thatport,query); startconnector(c); //create a thread to run the client. -4s a server, the Manager understands "out", "all", "cin", "rd", and "in" corn- rnands coming rom clients and uses multiple threads to coordinate them. if ("in". equals (op)) C val=(term)bboard. idkey,val) ; The Multi-t hreaded Server The tasks of our multi-threaded server are to create communication channels (Socket and streams), accept multi-t hread connections fiom clients, and broadcast everyt hing it receives to other clients. Server-side networking looks quite simple. A Server picks a port number and listens on the port for connections; clients communicate with the Semer through Sockets. Multithreading and broadcast increase the complexities of developing servers. For effectively realizing the multithreading, a Server class and a Service class are developed to cooperately work. -4fter the class Semer creates a SenierSocket on a local port, the class Service will start to listen on the port and accept connections from clients. Broadcast is achieved by employing multithreading and Linda implementation.

48 Sockets -4 socket is one end of a two-way communication between a client and a server running on the network. There are two kinds of the sockets in Java.net package: the Socket and the Server- Socket. The Socket class is used to establish a streams-based communication channel with a remote host and implement the client side of connection. The basic steps to create and run a client Socket are: 1. Create a socket. 2. Open an input stream and output stream to the socket. 3. Read from and mite to the socket. The ServerSocket class is a rnechanism by which a server can listen to either local or remote clients to connect them on public ports and implement the server side of the connection. The Serue~socket class only creates a socket for each client connection. The server then handles these connections by extracting an input stream and an output stream and communicating with the clients through the stream interfaces. The basic steps to create and run a SeruerSocket are: 1. Create a server socket. 2. Accept a client socket. 3. Loop to open an input stream and an output strearn to each connected client socket. 4. Read from and mite to each client socket. The basic Socket communication is shown in figure 3.3.

49 Clients Sava Socket cv-1 i rami Slrmm -'\ Hosc eve Figure 3.3: Basic Socket Operation The Class Server The class Semer is an subciâss of a Connector class. -4s a central Server, it communicates with the Blackboard for accessing the server-side global file, creates the ServerSocket on a public port, and manages Seruice threads which d l actually handle the connections between the semer and clients. import java. net. * ; import java.io.*; class Server extends Connector implements Ru~able { ServerSocket server,socket=null; private int port ; Server (int port) C bboard=new PrologBlackBoard(); this.port-port ;

50 This constmctor creates an instance of the PrologBlackBoard which serves as a central database to store the semer-side file for persistance, then it picks a public port number to which clients will connect. Then the Semer will listen on the port by creating a ServerSocket: private final boolean listen(int port) { > boolean ok=false; try C server,socket=new okrtrue ; > catch (IOExcept ion e) {ok=f alse ; 3 return ok; ServerSocket (port) ; // Create a ServerSocket If the ServerSocket is created successfully, the Semer will instance a Service class for each connection from the clients, and this Service class will be scheduled to mn as Connector thread by Manager class and actually handle the connections betnreen the Semer and the Client. Service s=new Service(this) ; Manager. runconnectorthread(s, "service,"+i) ; The Class Service The class Sentisce, a subclass of the Connector class, is responsible for handling connections between clients and a server. When a Client sends a "connnection" request to a Server, a Service thread will be created. This thread will create a client's Socket and set up the input and output streams attached to the Socket.

51 boolean start-service () { boolean ok=false ; if (server. f inished) return ok; 3 trjr ( client~socket=server.server,socket.accept(); //listen on the socket. open-streams (); //open input and output Stream. ok=true ; 3 catch (IOException e) {) return ok; Openstrearns(), a method of its super class Connecter, creates I/O streams by which the messages can be transferred between clients and the server. Then, it will cal1 the super class's answer() method to do the actual read and wnte on the streams. protected void open,streams() throws IOException < f rom=new DataInputStream(new Buf f eredinputstream( //Input Stream client-socket. getinputstream0 )); to=nev DataOutputStream(new Buff eredoutputstread //output Stream client,socket.getoutputstream~))); The Multit hreaded Client The Client Interface is shown in Figure 5.1. One input field is used as a VRML command and chat input. One big output area displays al1 the clients' chat and avatarsy actions. The right input areas will accept the port number and server name that the user will connect to. When filling the right fields and pushing the "connect" button, the user wili be connected to the port and the server given in the right input areas by creating a client thread.

52 After connecting to the semer, the user can communicate with the system by typing a text into the input field, and clicking the return key to send the text to the server. If the text is a chat message, the server will broadcast the message to the clients in the system, then the text will be displayed on each client's output window; if the text is a VRML command, it will be put on the Blackboard, and then broadcast to al1 the clients and al1 the VRML scenes of clients will be changed. These processes are fulfilled by classes of Client, Servant, and Connector. The Class Client -4 Socket is a Java representation of a TCP network connection. Using the Socket, a client can establish a strearn-based communication charnel with a remote host. In order to communicate with a remote Semer, the client must first create a Socket to the remote host where the Server is located, and also speci@ the host name and port number which the Server is actively listening to. private boolean create-socket () { boolean ok=f alse ; for(int i=o; i<connector.max,retry; i++) { t v ( client,socket=nev Socket (host,port) ; // create the client socket return ok; 3 After establishing a Socket, same as a Server, a client also need input and output streams in order to conmninicate with the remote server. The same open-stream0 method as one the Server called vil1 set up these streams. private boolean connect 0 C

53 boolean ok=false ; for(int i=o; iccomector.max,retry; i++) { trg < open-streams O ; ok=true ; > return ok; After successfully creating the 110 channel, the client can communicate with the rernote semer over the communication channel. The Ciass Servant Due to Java's security bugs, Java applet can not modib a Server thread. The system uses a Servant class to communicate with the Server when a client starts to connect to the system. The "Servant" has two main purposes: 1. Register Once a user enters the system, its 5enmnt" registers its client to the Server. The Server mil1 give the ''Servant" a unique ServantId that is the identification of the user and will be put in the Blackboard. -4t the same time, the user name d l be saved in the semer's blackboard for future use. When this user quits from the systern, the ServantId and user name will be removed from the system's Blackboard. boolean registero < // A add-servant command is sent to the server by the client. Term Cmd=new Fun("add,servant",new Int (incservingo 1) ; String s=cmd.tostring();

54 //create a client that will send the comand to the server. Client c=new Client (gethost 0,getPort 0, S. this) ; c.run0 ; // sent the commnnd to the semer. Term R=getReply (1 ; // get the ServantId from the semer. myapplet. servantid = R; // sent the ServantId to the applet // Check that al1 information is present if (!(R instanceof Nonvar) ) return f alse ; if (((NonvarIR).name(). equals ("no")) return f alse ; set Id (RI; //this vil1 put a user logged list on the blackboard if (myapplet! = null) C s = "add-user ("+myapplet. chatpanel. getusername O+") cl.run(); > Client cl = nev Client(getHost0,getPort(),s, this); //send the user name to the server. " ; 2. Communicate with the Server. Afker the Servant successfuiiy registers itse If to the Server, it will sit abot-e the input output channels, loop forever to send the Client's query to the server through the Connector class and get answers back to the Client. public void runo C trsr ( //loops forever until the work() return false for(;;) C > if(!worko) { break;)

55 > > catch(exception e) { e. printstacktrace () protected boolean work() ( Tenn T-1; if (begino) // Connect successfully ( T=interact (query); ) // Sent the query to the server else ( return false;) //the comection is fail disconnect0; //After sending the query to the server or //failhg to connect, disco~ect from the server if (T==null) { return f alse ;) Term Id=getServerId(T) ; Tem Goal=getCoal (T); Term Answer=reactToTerm(getGoal(T));// get answer from the semer if(answer.tostring().equals("done")) return fdse; return titie; Synchronization Multithreading allows us to wnte efficient and robust serves that can process separate client connections in separate threads. In a single threaded program, the execution begins at entry point of a program, follows along a single path, and terminates at the end of the program. This scenario is useless for mu1 tiuser applications.

56 Multithreading means that there can be multiple flows or threads of an execution in the same program at the same tirne. Each thread conceptually nuis in parallel with the other threads within a program. This type of programming is ideal for multiuser situations. The server can process clients' connections simultaneously. If a message from a client needs to be broadcast, this message will be sent to the Server first, and then the Server will send the message to dl the clients connected to it in different threads, so that all the messages are sent in pardel, and they wï11 reach al1 the clients at almost the same time. Threads introduce the need for synchronization. Synchronization is a mechanism which makes the relative execution of the threads be controlled to prevent some undesirable resuits. For example, if a reading thread and a writing thread are accessing the same memory area at the same time, the result that the reading thread gets might be unexpected. We c d this area a critical area and the code that accesses the area a synchronized block. In Java, "synchronized statement marks the codes or an object as a synchronized block. -4ny thread that accesses the block has to get the system semaphor before it enters the block, and return the systern semaphor before it exits the block. The characteristic that the system has only one semaphor guarantees that only one thread can access the synchronized block at the same time. In our program, the Blackboard is a critical axea. Any methods that access this area are marked synchronized blocks. A lock mechanism has been used. When a thread wants to read from the Blackboard, it locks the blackboard first and then reads. The other threads that want to write to the Blackboard have to wait until the reading thread finishes and unlocks the blackboard. No specific deadlock prevention or resohtion is attached to the blackboard, as this is the programmer's responsability. However, as LL~~t'7 operations never suspend, the only way to create deadlocks is by having 2 processes waiting for each other for "in" transactions. A timeout mechanism has been used to break such deadlocks.

57 //lock private final synchronized void bblock() { tq C //waiting for notification from the thread that is in the area. ) catch (InterniptedErception e) {> > //Uriiock private final synchronized void bbunlock0 C 3 bus* alse ; notifyo; //notify the thread that is waiting for entering the area. //mite into Blaksoard public Term cin(string k,term pattern) < bblock0; /Dock the critical area before mite bbunlock(); //dock the area before return return T; 3 //read from Blakboard public Term out(string k,term pattern) { bblocko; // lock the area before fetching something from blackbo~d bbunlock(); //dock the area after finishing the fetch thread.resume(); // uake up the sleeping thread return yes ; 3

58 3.3.5 Coordination with Linda blackboards LogiMOO is built upon BinPrologYs Java based Linda tuple space (blackboard) [39]. The blackboard's basic characteristics are (i) it is persistent; (ii) the data are manip ulated associatively (Le., based on their content, rather than on their address); (iii) all accesses are automatically synchronized. These are precisely the requirements for a M00: the status of the MO0 should be stored somewhere, and it should not be volatile; if we look for a particular object in a SIOO, we el1 always refer to it Nith a name, never with its address location; a MO0 is a multiuser, so it should be synchronized at any tirne. The basic operations on the blackboard are: run-server (Port) out (X) in(x) rd(x) all(x,xs) remotenm(goa1) local-out (X) localzd(x) stopsemer Starts a blackboard server listening on a port. Puts X on the server. Waits until it can take a. object matching X fkom the server. Waits untii it can take an object matching X fkom the server, but not destroy the X. Reads the list Xs matching X currently on the server. S tarts a thread executing Goal. Puts private information X on the local default blackboard. Checks whether an object matching X is on the local blackboard. Stops the server. The only blocking operation is in(x). It executes as a consumer, any items matching with X will be fetched out, and the items will no longer exist in the blackboard. The non-blocking operation out (a(1) ) executes as a producer, after executing the out(a(l)), an a(1) is put in the blackboard. Any in(a(1)) opteration will be blocked until the a(1) is available, and blocking rd(a(1) ) is easily emulated in tems of

59 in(a(1)) and out (a(1)). Non-blocking rd(a(1) is emulated with ail (X). (see also Figure 3.4). Figure 3.4: Basic Linda operations

60 Chapter 4 Integrating LogiMOO II with virt ual worlds LogiMOO II is integated witith VRML virtual worlds to provide multiuser virtual world. For a multiuser virtual world, two standards are critical: synchronization and consistency. The world scene should have only one copy in the system and stored in the server side. We use Blackboard as a world scene's container, and it will guarantee the operations synchronization and the scene's consistency. The integration has been done by modifj4ng classes of LogiMOO II and adding some new classes so that the system can communicate among browsers, applets, comrnands' parsers and server. The integration is the key to the multiuser virtual world presented in t his thesis. Many issues were solved during the integration, such as blackboard operations and storing a semer-side world file, extension to the prolog parser to deal with VRML execution, and many programming details to merge two systems together. This chapter will introduce the main integration and modification.

61 4.1 Dynarnically creating server side world scene file The server side world scene file includes avatar's position, state information and is dynamically generated and changed. When the server starts, the file is empty. The background scene is in the client side. When a user logs in, the server side scene file will be loaded and then an avatar d l be loaded into the scene to represent the user and then avatar Ml1 be also added into this world file. When a user logs out, the avatar's information will be deleted rom the file. The file will be saved in the blackboard, some Linda operators will be used to maintain the systern's consistency Storing avatar's messages in Blackboard. The Blackboard is an ideal engine to store avatars' status information. LogiMOO II server is a Term server. Al1 data are presented in Term for coordination with Prolog. The Linda blackboard's [9] [13] data representation is shown in figure 4.1. To store an avatar's status data (such as position, rotation, etc.) into the server's blackboard, the data must first be converted to Term fonn as following: public void put-avatar0 Term T = new Fun("avatarOut", neu Fun ("avatar", nev Int (p. current -avatar-num) ), new Fun("posII, new Real ((double) avatar-translat ion [avatar-num] [O] 1, new Real ((double) avatar-translation [avatar-num] Cl] ), new Real ((double) avatar-translation [avatar-nd [2] 1 ),

62 f > Term f Var \ Figure 4.1: Linda Term: Data Representation Then the Term T will be sent to the semer by creating and running a Client: String s = T.toString0 ; Client c = nev ~lient(a~~let~ost,~o~nector.~~~~~t-~~~~,s c.mo; Fetching avatar's messages from Blackboard. To fetch an avatar's data from the blackboard, we have to give the data's pattern and key to a Term variable, then the Term variable will be sent to the server with an operand "avatadn". When parsing an incoming message that starts with an "avatarin", the server will search the blackboard to get the data and then send the data back to the user. public Term get-avat ar (int key-num) C Var num = new Vax (1 ;

63 //The Var x, y, z ail1 hold the avatar's translation data- Var x = new Var(); Var y = new Var(); Var z = new Var(); //The key and pattern of the data to be fetched Term Ti= new Fun ( "avatarin", 1; new Fun("avatartl, new Int (key-num) ), new Fun( "avatar", num), nev ~un("pos",x,y,z), //Send the key and pattern to the server String s = Tl. tostring0 ; Client c = new Client(appletHost,Co~ector.DEFAULT,PORT,s,~.SCENE); c.run0; //get the data of the avatar Term R=c.getReply(); return R; // The data got from the server Deleting avatar's messages fkom Blackboard. When a user logs out from the system, his or her avatar will be removed from server side scene file. public void remove-avatar (int avatar-id) //The key and pattern of the data to be fetched

64 Term Tl= neu Fun( "avatarremove", neu Fun("avatarU, new 1nt (avatar-id) 1, --..*. 1; //Send the key and pattern to the server String s = Tl-toString0; Client c = new C ~~~~~(~~~~~~HOS~,COIL~~~~~~.DEFAULT~PORT, c.runo; 4.2 Distributing the avatar's behavior When a server receives a user command to move an avatar, the server broadcasts the cornmand to al1 the clients in the system depending on the userid in the blackboard. protected void fornard-to-servants(tem Mes) < Enurneration e=bboard.to~numeration~or("serpant,id"); while (e. hasmoreelement s ( ) ) C Fun ServantId= (Fun) e. nextelement ; Term ServerId=ServantId. args [O] ; String key="query"; Term Query=new Fudkey, ServerId, Mes) ; bboard.out(key,query);

65 Chapter 5 Mult iuser Virtual Worlds The multiuser virtual world was developed under Cosmo Player 1.0 and its Java EAI. Instead of using an interna1 authoring interface, using an external authoring interface allows us to integrate the virtual world with an existing external application and provide a more flexible way to communicate with a Java applet and implement a NL interface. This chapter introduces the EAI and how to use it to create a virtual world. -4 method to extend VWlL 2.0 to support multiuser virtual world will be suggested and a prototype to use the extension will also be presented. A text-based user interface and a command set will be introduced. This interface will be extended towards Natural Language interface and Speech Recognition in future work. -4t the end, some performance issues will be discussed. 5.1 The VRML 2.0/Java Interface Chris Manin's proposal for the VRML 2.0 Extemal Authorhg Interface(E-41) provides Zway communication between a VRML world and an external applications [25] The EAI includes the following packages and classes:

66 With the EN, we can establish a link between a Java applet and a VFUvlL world, so that the applet can send user commands and events to the VRML world, and the VFWL world can also send events to the Java applet. At any tirne, the appiet can trace the avatar's status in the world Java applet establishes links to a VRML world There is a Browser object in the VRML EAI package. This object provides Java applets an ability to access the node interface. In a Java applet, we first get the reference to the browser and then create links to the nodes in the scene through the browser. Once we get the link from the Java applet, we can access the nodes' events just like in the VRML scene. For instance, we have the VRML file: DEF ROOT Croup > DEF AVA Avatar0 We can build the links to the "Browser", "ROOT", and "-4W" nodes by using the following Java sentences, and create some links to the evenths: brovser = Browser.getBrowser(this); try ( root = brovser.getnode("r0ot"); avatar = browser.getnode("avau) ; addchildren = (EventInMFNode) root.geteventin("addchildrea"); removechildren = (EventInMFNode) root.geteventin("remo~echi1&en");

67 > catch (1 nvalidnodeexception e) { > System.out.printla("PROBLEMS! : " + e) ; Later, if we want to add children nodes to the root node, we can use the addchil- dren variable as following: addchildren. set Value (node-name) ; and if we want to remove a child node, we can cal1 the removechildren variable as following: removechildren. setvalue (node-name) ; How a VRML world sends events to an applet..a VRML world can send user events and avatar motion information to a Java applet through eventout or exposedfield fields in the scene: The EAI provides an EventOutObsenrer class. This class and the eventout object provide a protocol where EventOutputObserver objects watch the status changes and user events in any number of observerable eventout objects. To receive notifications fiom a VRML scene, an applet class or other class must subclass an EventOutObserver class to become an EventOutObserver object. In Our system, we define a NodeLoader class that implements EventOutObse~er.

68 public class NodeLoader extends Object implements EventOutObserwerC In this class, we create some references to the exposedfields in the VRML file. So the avatar+raoslation-changed variable defined in the applet class holds Avatar- Position's values, the avatar~otation-changed variable defined in applet class holds AvatarRotation7s values, and so on. parent.avatar,translation,changed = (EventOutSFVec3f) parent. avatar. getevent0ut (" AvatarPosit ion") ; parent. avatar-rotat ion-changed = (EventOutSFRotat ion) parent. avatar. geteventout (" AvatarRotationH ); parent.avatar,vieupoint,changed = (EventOutSFVec3f) parent. avatar. getevent0ut ("this-vievpoint") ; Then, the advise() method of the EventOut class will set some eventout objects to be observable eventout objects. In followingcode, for instance, the avatar-translation-changed, avatarrotation-changed, and avatar-viewpoint-changed are set to observable eventout objects. The advise() method passes the EventOutObserver object as its first param- eter to the callback method. Any changes of the event Out it refered to will be notified by the callback method. parent. avatar-translat i~n~changed. advise ( parent.avatar,rotation-changed.advise( parent. avatar,vieupoint,changed. advise ( this, (0bject)nev Integer(avatar-pos)) ; this, (Ob j ect)nev Integer (avatar-rot) ) ; this, (Ob ject )neu Integer (avatar-vied ) ;

69 Whenever an event is generated for that observable eventout, the callback method will be executed and passes the eventouts and their timestamp to relative methods for further processing. For instance, if an avatar's position changes, the variable avatarfranslationrhanged gets a set of new values, and these values will be notified by the callback method. public void callback(event0ut event, double the, Object userdata) //reset avatar's new position 3 parent.resetavatar() ; In our callbacko method, we cal1 the resetawtar() method of the Interactor class. This method accepts eventouts happened in the VRML scene and assigns the new eventout values into local variables of the Java applet for further processing. void reset Avatar (int avatar-num) < current,avatar,translation Cavatararndi avat ar-translat ion-changed [avatar-num]. getvalue ( ) ; current,avatar,rotation [avatar-nd = avatar-rotation-changed [avatar-num]. getvalue 0 ; current,avatar-vieirpoint [avatar-num] = avat ar-viewpoint -changed [avatar-num]. getvalue ( ) ; So, al1 user events and avatar movements cm be sent to the Java applet. This process creates a very convenient communication channel between VRML and Java How an applet sends user events to VRML. By defining eventins in a VRML file and pointers to the eventins in a Java applet, we can also create communication channels from a Java applet to a VRML scene.

70 In Our new node type Avatar, some eventins have been defined. Let's take one eventh for instance: We can define a reference to the eventin in the Java applet: parent. walk = (Event InSFBool) parent. avatar. getevent In(llwalk") ; When we receive the command "walk" From a user, we can send the event to the VRPUIL file t hrough the following statement : Zn the VRML file, when the eventin "walk" is received, it is routed to the script node to enable TirneSensor to animate the walking action. 5.2 A Shared Virtual World X shared virtual world allows users to share the virtual space. In Our virtual world, s shown in Figure 5.1, when a user enters our world, a representation of the user, an avatar, will be added into the scene that other users are exploring. Shared virtual world means that all users have to see the same scene no matter where the user is or when the user cornes in. When some users are watching the girl dancing, other avatars are walking around. These activities must occur sirnultaneously across al1 instances of the world. To do so, the avatars' status and positions in the scene must be kept somewhere so that when a new user logs in, he/she can have the same scene as the other users have. An avatar's motion has to be broadcast to al1 the users in the scene, and kept as much synchronization as possible, so that the users in the scene can see and interact each other just like in the real world. Figure 5.1 shows a virtual world in which three avatars are wandering in the world and a girl is dancing with Taichi movements.

71 Figure 5.1: Shared Virtual World 5.3 Extensions to VRML 2.0 VRML 2.0 does not support multiuser [33]. There are two main issues about the multiuser hinctionality. One is the low-level mechanism about how a shared scene works. This relates to many topics, such as the networking, the shared behavior, the multiuser interaction, and the collision, etc.. Another issue is the actual semantics of the shared entities in the scene, and it relates to the actual scene's consistency, universal avatar [2011, etc.. Considering the second issue, our approach is to create a new node type "Avatar" [6] which will be referenced as a standard node. Ideally, this node type should be defined in a VRML Node class and an EAI class for general use. For simplicity, we define this node type in an interna1 file or an extemal file as a prototype (We have trkd both ways). In the VRML file, we reference this node just as any other node types defined in VRML 2.0 and EN. 'a standard avatar definition that enables a user to create an avatar and then use that avatar in a wide range of multiuser systems.

72 5.3.1 Defining a New Node type "Avatary' The purpose of defining a new node type "Avatar" is to provide a standard setting for avatars. In a multiuser environment, each instance of the world needs to know the user's avatar that may corne rom the same world or fiom an outside world. Without a standard setting, it is very hard to communicate among different avatars. The "Avatar" has some eventin and eventout fields in its PROTO, with these events, avat ar7s behaviors can be defined standardly. To simulate human behaviors, we define 'kalk" (similady we can defme "Run" ), "nad" (express agreement), "stop" and "tum", and provide some exposedfields such as "Leg", "arm", "head", 'posi- tion", "rotation", "Shirt Colof, "Pant Color", and "Cap Color", etc.. With these events, we can dynarnically generate avatars with different characteristics. Avatar C The foiiowing is the avatar's definition: exposecifield SFVec3f AvatarScale O O O exposedfield SFVec3f AvatarPosition exposecifield SFRotation AvatarRotation O exposecifield SFVec3f AvatarCenter O O -3 exposecifield SFVec3f CapColor exposedfield SFVecSf ShirtColor O O 1 exposecifield SFVec3f PantColor exposecifield SFRotation left-leg-rotation 1 O O O exposecifield SFRotation right-leg-rotation 1 O O O exposedfield SFRotation left-am-rotation 1 O O O exposecifield SFRotation right--,rotation exposecifield SFVec3f this-vieupoint O O -10 exposedfield SFString user-name " " exposecifield SFInt32 stepsout O 1 O O O

73 eventh SFBool walk-loop eventin SFBool nod eventb SFBool twpleft eventin SFBool stop > Using the new node type "Avatar" To control an avatar's behavior and give it some degree of intelligence, we need some events to communkate with an extemal program. These events have to be pre-defined in an external program. Some multiuser virtual world applications use other approaches to solve this problern, such as the CyberPassage 2.0 which uses SE VRML(a Sony Enhanced VRML) rather than VRML 2.0 to irnplement the multiuser world [4]. First, we define 10 "avatars" in our world file with the new node type. Each avatar's translation is given at an absolute coordinates as shown in above ".4vatar3 prototype, the position is out of the system viewpoint, so an avatar d l not be loaded untii its position is set to within the viewpoint. DEF AVAi Avatar<) DEF AVAS Avatar0 DEF AVA3 Avatar0 DEF AVA4 Avatar0 DEF AVAS Avatar0 DEF AVA6 Avatar0 DEF AVA7 Avatar0 DEF AVA8 Avatar0 DEF AVA9 Avatar0 DEF AVAiO Avatar() When a user logs in, an available avatar that represents the user is allocated. The avatar's characteristics such as shirt color, pant color, and body height, etc. will be

74 dynamically generated by the applet, so that each user has a dif5erent avatar. public void login(string login-name, int this-avatar-nm) /** Load an avatar into the scene for the user, at this the, the avatar are not be loaded because it is out of the viewpoint. */ avatarloader [t his,avat ar-nd = new NodeLoader (this, NodeLoader. AVATAR, this-avatar-num) ; /** Calculate the avatar's random position and put in val array */ /** update the scene, set the avatar's set,avatar,translation [this,avatar,nd position to the nev position. setvalue (val) ; /** Calculate the avatar's random shirt color and put in val array */ a.. m.. /** update the scene. set the avatar's shirt color to new color */ > set,shirt,color [this,avatar,nw]. setvalue (shirt-val) ; /** set the avatar's status */ current,avatar,status [this-avatar-numl = STOP ; /** put this avatar in server's blackboard if (this-avat ar-num == my-avatar-num) { 3 f.put-avatar(this,avatar-num) ; The avatar's colors and positions are generated by Java's Randorno method, then the positions are scaled within the current viewpoint and a visible avatar will be Ioaded into the scene.

75 When a user logs out, a reverse process WU be executed. At first, the avatar will be scaled to (0,0,0), then set the position back to (1000,1000,1000), and 6nally the avatar's data dl be removed fiom the biackboard. 5.4 Text-based User Interface There are two general interface categories in MUD/MOO environments: one is a point and click graphical environment, the other is a text-based system. The point and click systems avoid the need for NL processes as al1 user commands that express avatars' actions and emotions are done by mouse and keyboard. The text-based interface will accept Natural Language commands to control avatars' actions and emotions and will provide more possibilities and flexibility to express abstract concepts Towards natural language control for Avatars in VRML Natural Language is one of maay interface styles that enable users to communicate with computers in English or other human languages (241- This interface style is especially suitable to Virtual Reality systems [38]. -4 Natural Language interface has many advantages: it is easy for users to learn and remember, because its structure and vocabulary are already familiar to users; it is more flexible for fulfilling a task because there are several ways to do the same jobs; in networked collaborative applications, it is more powerful for the participants to cooperatively work together without taking care of the computer language syntax. The virtual world is a condensation of the real world. It is very hard to interact with it by only using the limited hnctions of a mouse or keyboard or applet buttons. A natural language understanding interface, even if it is limited to a small set of concepts, enables users to interact with the virtual world in a natural and intuitive way. It is especially useful in controliing things that do not have a physical presence in the environment, such as object scale, display characteristics, and viewpoint. It also provides a powemil method to access the knowledge that underlies the virtual

76 environment by allowing users to ask questions about the system. In a networked environment, natural language commands are directly sent to the server, and the natural language procesor on the semer-side parses the command and then broadcasts the VRML commands to the clients. It dramaticdly simplifies the client's interface design and can be used to add intelligent experts into the server in the futire The avatar Control Command Parser A Natural Language Processor and interface are under development. -4 simple avatar control command parser has been developed as a separate object which will be integrated with the NL processor in the near future. The parser uses the Java's String- Tokenizer class to parse incorning commands, tokenize the string, and call the corresponding methods in the Interactor class to control the avatars in the scene. 1. Parser Processing Flow Chart There are two parsers in the system, a system command parser and an avatar control command parser. The system command parser is in the server side, it parses al1 the commands the server receives. The avatar control command parser is in the Client side. When a user inputs an English text $om an input field, the message will be sent to the server. If a chat message cornes, the server directly broadcasts the message to al1 the clients in the system; while if the message is an avatar control command, the server will call the ParserInput class that will parse the command, as shonm in Figure The avatar control Commands Al1 avatar control commands should start with "vrml" to distinguish from chat messages. The avatar control commands the system can recognize are classified in Table 5.1:

77 1 VRML COMMAND 1 L I 1 VRML Cornmand 1 Description 1 L vnnl avatar walk vrml avatar stop vrml avatar nod vrml avatar go fonvard 10 set avatar walk set avatar stop walk set avatar nod (agree) set avatar go forward 10 steps (any integer) vrml avatar go backward 1C set avatar go backward 10 steps vrml avatar go left 10 set avatar go left 10 steps m l avatar go right 10 set avatar go right 10 steps vrml avatar turn left set avatar turn left vrml avatar turn right 1 set avatar turn right vrml avatar turn back ( set avatar turn back vrml avatar turn 45 1 set avatar tum 45 degree(any integer) vrml girl dance set the girl in the scene starts dance vrml girl stop 1 set the girl in the scene stops dance 1 vml make cube 1 make a cube(or cone or sphere or cylinder) in the scene's random place 1 vrml color red vrrnl color r g b vrml active 1 vrml remove 1 change the current O bject to red(green, blue) change current object to the color of r g b active the nurnber 1 object to be current object or use mouse to click rernove the number 1 object Table 5.1: The avatar control cornrnands