Visionair. TNA - Collaborative remote visualization ENSAM URJC. Miguel Ramos García. Universidad Rey Juan Carlos, Móstoles (Spain)

Transcription

1 Visionair TNA - Collaborative remote visualization ENSAM URJC Miguel Ramos García Universidad Rey Juan Carlos, Móstoles (Spain)

2 Index Introduction Framework, the first contact Information flow diagram Analysing the framework for collaborative viewer applications Searching a better Transport Protocol Stream Control Transmission Protocol (SCTP) User Datagram Protocol (UDP) Real-Time Transport Protocol (RTP) Integrating RTP into the framework JRTPlib The encoder and decoder protocol Defining the features of Collaborative Viewer Implementing the features of Collaborative Viewer Allowing users to navigate through the scene Annotating data or elements in the visualization environment Reproducing and saving previous sessions at the local machine Audio and video call Miguel Ramos García 2

3 Introduction The present document describes the stage performed by Miguel Ramos García at the ENSAM placed in Chalon Sur Saône (France). The main objective of this stage was to carry out the first steps of an application which allows the collaborative visualization of data sets by teams located at two remote sites, located specifically in Chalon Sur Saône (France) and Mostoles (Spain). The final goal of this work is to study and solve all difficulties which complicate or prevent this kind of applications. The following sections contents: Review of the previous application framework developed by ENSAM and URJC. Performance Analysis of the existing application framework and the required features for this kind of applications The necessary modifications and their degree of development Miguel Ramos García 3

4 1 Framework, the first contact. The main goal of the present stage was to study the existing application framework, since it was the basis over which the rest of the software would be based. This framework had been previously developed by Julien Ryard and José Miguel Espadero, before this stage took place. The existing framework consisted of three MPI threads: the Render, the Device and the Collaborative. For interaction with the scene, this last one used Virtual-Reality Peripheral Network. The following paragraphs describe each of the three processes: Render process: It is in charge of the rendering process. It receives information from the Device process, being in charge of all of the rendering operations. It does not send any kind of data to any another process. Device process: This is the central process of the application, working as an intermediary between all the other processes. It receives the information from the remote CAVE through the Collaborative process, using MPI message. This information should include all actions that the partner takes in his CAVE, but at this moment it includes only the position and attitude of the partner s avatar. In addition, the information from Virtual-Reality Peripheral Network is sent to this process. On the other hand, this process sends all the aforementioned information to update data for the Render process, using p2p messages. Additionally, the Device process sends all interactions that are done in the local CAVE to Collaborative. Collaborative : This process manages the interconnection between the two CAVES, sending and receiving all the interaction done in each CAVE to the other partner. At this moment, it is done using TCP. 1.1 Information flow diagram Collaborative TCP Collaborative Cave 1 VRPN MPI Device MPI Device VRPN Cave 2 P2P P2P Render Render Fig. 1. Overall framework structure Miguel Ramos García 4

5 2 Analysing the framework for collaborative viewer applications After studying the framework in detail, the features and performance of the framework were analysed from the point of view of the required characteristics for real time applications such as collaborative data analysis tasks. Framework features and performance Real Time Applications requirements (At least) Render FPS Over Speed of messages between CAVES 2 per minute Similar to Render FPS Messages data between CAVES Avatar position and rotation All interaction between scene and avatar Fig. 2. Performance of the existing application framework versus required performance As figure 2 shows, the main difficulty for reaching the requirements for real time applications is the communication between both CAVES. This communication is focused on two phases, one phase for communication between Collaborative processes and the other, between the Collaborative process and Device process. The second phase is carried out inside each machine, so it is much faster than the first one. In summary, it is necessary to improve the speed of messages between the local and remote CAVES There are two possibilities for solving these problems, based on two aspects: One of this is the bandwidth, in relation to message size. Presently, this is not the main problem, although it could become a bottleneck in the future. Latency on package shipment. In general it is widely admitted that connections over TCP are slower than other approaches because in order to ensure the reception and integrity of messages, it is necessary to add a lot of information to each packet. In addition, it is possible that a node can be busy with pending messages, delaying overall communication performance. Thereby, it is required that change TCP by other approach that can avoid the described problems. Miguel Ramos García 5

6 3 Searching a better Transport Protocol The third issue covered during the stage was analysing other transport protocols in order to select a better one, better suited for Real Time Applications. 3.1 Stream Control Transmission Protocol (SCTP) This protocol attempts to merge all the advantages from TCP and UDP. However, it provides new restrictions. The first one is the requirement to use Static Public IPs, although this, for institutions like ENSAM and URJC, it is not an important restriction. Usually, those kinds of institutions already use Static Public IPs for their servers and other machines Another drawback of SCTP is that sometimes there are problems for detecting machines connected through a local network. Generally, CAVES include several computers interconnected, which can result in problems when this method is used Fig. 3 TCP versus SCTP (picture from User Datagram Protocol (UDP) This protocol attempts to correct all the disadvantages from TCP. Nevertheless, it introduces several problems. One of the most important is a higher probability of data lost. Other disadvantage is the absence of data flow control, so it possible to send several times the same information. Because of that, this is not the best option for our goal. Miguel Ramos García 6

7 Fig. 4. TCP and STPC versus UDP (picture from Real-Time Transport Protocol (RTP) This protocol was designed for applications that need to transmit real time data. In addition, it allows the connection among several machines at the same time, allowing a higher speed among systems, only limited by network bandwidth. RTP does not ensure data integrity, since it is possible to have some data being lost, although it is limited to small figures such as a 4% of transmitted data. In any case, for our application, having some data being occasionally lost is not a fatal error as long as the percentage of lost data is small enough, since it only represents an incorrectly rendered frame. Last, message encoding and decoding has to be programmed for every application. This is not really a disadvantage, facilitating the control of message size, using compression methods to encode data. Consequently, RTP has been selected as the protocol used for transmitting messages between CAVES. Miguel Ramos García 7

8 4 Integrating RTP into the framework. After RTP was selected as the transmission protocol, work went on with its integration into the framework, which was done first through the selection of a library that would ensure working with this protocol. 4.1 JRTPlib After several days searching a library to use RTP, JRTPlib was found. JRTPlib is an objectoriented RTP library written in C++. It was first developed for the thesis of Jori Liesenborgs at the School for Knowledge Technology (or 'School voor Kennistechnologie' in Dutch), a cooperation between the Hasselt University and the Maastricht University. The library offers support for the Real-time Transport Protocol (RTP), defined in RFC It makes it very easy to send and receive data and the RTCP (RTP Control Protocol) functions are handled entirely internally. This permits concentrate a great part of the effort on the application and not on the communication. 4.2 The encoder and decoder protocol In order to make the least change as possible, the framework architecture is respected and only is added things that are required by JRTPlib. The most important thing is the encoder decoder protocol that must be design in this case. Really, the only way to send data is using a predefined size vector of unsigned int (8 bits). This is the reason for encoder and decoder protocol. 5 Defining the features of Collaborative Viewer In addition to modifying the application framework, an essential part of the stage was the specification of the required features that Collaborative viewer should have. After several tests and meetings with Luis Pastor, Jean-Rémy Chardonnet, Julien Ryard and José Miguel Espadero; the following set of features were selected: It should allow users to navigate through the scene, letting each partner view the actions, point of view and data visualized by the remote partners It should allow saving and reproducing data analysis sessions at the local machine, even without the connection of a remote partner. Actually, this would allow sequential collaborative data analysis, where a user analyses data at a specific time, and additional users can reproduce the analysis session performed by the first user, introducing new insight into the data set, environment or models being analysed. Miguel Ramos García 8

9 It should permit marking elements along the visualized environment as well as introducing annotations as text, audio and video for highlighting and enhancing important detail. It should connect in Real Time all partners with audio and video call for facilitating discussions 6 Implementing the features of Collaborative Viewer Last, during the stage a number of application features were implemented. The following paragraphs list them, together with their degree of development. 6.1 Allowing users to navigate through the scene. The framework already had this feature implemented, so it did not require any modification. 6.2 Annotating data or elements in the visualization environment. This feature is connected with the features of reproduce and save previous sessions. At the moment of leave Chalon, the application can make mark and save text in current session but cannot save this to reproduce later. 6.3 Reproducing and saving previous sessions at the local machine. This feature is connected with the features of reproduce and save previous sessions. At the moment of departure from Chalon, the application allows entering and saving text in the current session, but cannot save this to have it reproduced later on. 6.4 Audio and video call. At the moment of leaving Chalon, it is being performed through applications such as Skype, in order to test these features. It will also require new. However, since audio and video communication can be obtained through other applications, this feature is not essential. Miguel Ramos García 9