A Scalable Object-Oriented Client/Server Architecture for Interactive Multimedia Applications over Internet

Transcription

1 P A Scalable Object-Oriented Client/Server Architecture for Interactive Multimedia Applications over Internet A Scalable Object-Oriented Client/Server Architecture for Interactive Multimedia Applications over Internet 2 Md. Mahbubur Rahman, Dentcho N. BatanovP P, Susumu Horiguchi School of Information Science, JAIST Asahidai 1-1, Tatsunokuchi, Ishikawa , Japan TUTUTUTUTUmahbubcse@yahoo.comUT, TUTUTUTUTUhori@jaist.ac.jpUT 2 PAsian Institute of Technology, Thailand Abstract Multimedia communication can be defied as the field pertaining to the formation, storage, retrieval, dissemination and usages of multiple media such as images, texts, graphics, animation, video and audio. The recent explosive growths of the Internet and multimedia research have raised the interest in distributed multimedia systems over Internet. In this paper, we propose an architecture for capturing, distributing and displaying multimedia content through existing Internet and Intranet infrastructures. An object-oriented architecture for the building of customized multimedia clients and servers facilitates software maintenances and updates as application requirements evolve and code reuse. The architecture is consisted of client/server model. In our prototype implementation Java s multithreading capabilities were extensively used to obtain a highly flexible and extendible system that is dynamically configurable at runtime. Key Words Object-oriented, Client/server, Unified Modeling Language, TCP/IP, UDP, RTP. 1. Introduction The Internet and Multimedia are technologies, which have been around us for a several years, and both are expanding in terms of use and technological advancement. The two technologies are quickly converging because the infrastructure often is already in place and adopting existing datagram networks and Internet connections will save expensive software development. On the other hand, this approach seems more economical and easier to manage then separate datagram and real-time networks [1]. Recent advances in compression technology for video and audio has also prompted the introduction of multimedia integrated to Internet. But the problem is that voice and video are time sensitive and they have to be sent in a stream. Also the current packet technologies like TCP/IP can t guarantee that the traffic will get where it needs to go without being jumbled and jerky. To cope with this problem, the Integrated Services working group in the IETF (Internet Engineering Task Force) developed an enhanced Internet service model that includes best-effort service and real-time service. The Resource Reservation protocol (RSVP) together with Realtime International Journal of The Computer, the Internet and Management Vol. 13.No.3 (September-December, 2005) pp

2 Md. Mahbubur Rahman, Dentcho N. Batanov, Susumu Horiguchi Transport protocol (RTP), Realtime Control protocol, Realtime Streamming protocol provides a working foundation for IP networks to carry voice and video in addition to the data traffic. In past years several research groups have worked on architectures for distributing digital multimedia content through the Internet and Intranet Infrastructures. In particular researchers working on the technologies related to the MBone and to adaptive audiovisual streaming protocols. However the existing MBone and unicast solutions do not provide the degree of control, flexibility and the possibilities for QoS management that the next generation applications require. Augmented QoS mechanism equipped applications were also proposed. A distributed real-time interactive multimedia application requires a design that emphasizes flexibility, modularity, expandability and efficiency. The object-oriented paradigm offers proven methods for the design and implementation of complex applications [2]. In this paper we present an Objectoriented model of real-time interactive multimedia application over Internet. Our system consists of client/server architecture where client side handles for local data encoding for transmission and decoding of received remote data for presentation. On the other hand, the server handles the session control and data forwarding. The software is completely written in Java incorporating Real-time Transport. Therefore, platform independence has been achieved by using Java Virtual Machine. Java's multithreading capabilities have been used to design a highly extendible and flexible system that can be dynamically configured at runtime. The overall functionality of the system will be explained using UML class diagram, block diagram and state diagram describing protocol exchange between client and server. 2. System Architecture The proposed object-oriented client/server architecture for real-time interactive multimedia applications over Internet consists of two components client and server. Multimedia clients run on user workstations, receiving data from remote clients and displaying it to user. This involves decrypting, decoding and rendering of multimedia data. We assumed that like a video conferencing tool, the client component also plays the role of service provider. They locally capture, encode and send data out at the same time they receive, decode and display it. Communication between clients is achieved by server component. It is responsible for spreading the data through the network in an efficient way. The server can be configured by server interface. 2.1 Client Architecture The client functions are effected by an Application Controller, a ClientInterface, a Data, a Sess-ion and a Connection objects. The ApplicationController serves as the controller of the client functions. All major events, such as the event of reception of data from remote user or local user commands are reported to the ApplicationController. The Application- Controller then invokes the corresponding methods of the Data, the - or the ClientInterface to inform them of the event. The ClientInterface handles the interaction with the user. Therefore, it provides GUI objects for configuring and monitoring architecture components and triggers the notification of major application events to the user. The 2

3 A Scalable Object-Oriented Client/Server Architecture for Interactive Multimedia Applications over Internet ApplicationController Data ClientInterface Connection Application Management BufferInput DataEncoder BufferOutput DataDecoder Data and control flow DataPort ControlPort Data Flow Control Flow Figure 1: Client architecture Application Controller ClientInterface Data Connection DataBuffer Data Transformer Configuration Transporter Translator DataPort ControlPort Buffer Input Buffer Output DataEncoder DataDecoder Encoder Decoder Send Output Receive Output local Decoder Remote Decoder Figure 2: Class diagram for client International Journal of The Computer, the Internet and Management Vol. 13.No.3 (September-December, 2005) pp

4 Md. Mahbubur Rahman, Dentcho N. Batanov, Susumu Horiguchi Data manages the internal and outgoing dataflow. We assume that any kind of multimedia data can be handled and Data object is able to dynamically change its configuration according to the nature of the data. The is responsible for managing the session. It manages the operation of session initiation, session configuration and session description. Connection does data and contorl connection management using DataPort and ControlPort objects. The data handling in client is co-operatively done by the BufferInput, DataEncoder, Data-Decoder, BufferOutput, and DataPort objects. Therefore in Figure 1 we show the two levels of classes. The upper lavel is responsible for management specific tasks whereas the lower level classes are responsible for performing actual operations. 2.2 Dataflow in Client Dataflow in client is illustrated by a sequence diagram in Figure 3. BuffreInput object is responsible for capturing data. Its capture() method performs the capturing data online. After capturing, it invokes the encodedata() method of LocalEncoder object to encode the data yet captured. After encoding, to enable realtime transmission, object encode further the captured data using its protoencode() method. Then data is sent by Dataport object s send() method. On the other hand incoming data is received by Dataport object s receive() method. Then data is decoded by and Remote-Decoder objects. object s protodecode() method and RemoteDecoder objects decodedata() method performs the necessary decoding operations. After that data is played by presentdata() method of BufferOutput object. Control and session information travels through, Connection, and ControlPort objects. The methods responsible are sendsession(), Decodesession(),send(), decodesession(), receivesession() and storesesson(). For local data presentation the Localencoder and LocalDecoder objects just bypass the data to BufferOutput object. Buffer Input Local Encoder LocalDecoder Data Port Remote Decoder Buffer Output 1: encodedata 2: decodedata 3: pickdata 4: presentdata 5: protoencode 6: send 7: protodecode 8: decodedata 9: pickdata 10: presentdata Figure 3: Sequence diagram for dataflow in Client 4

5 A Scalable Object-Oriented Client/Server Architecture for Interactive Multimedia Applications over Internet 2.3 Server Architecture The management of the server functions is effected by an ApplicationController, a ServerInterface, a Data, a -, and a Connection objects. The ApplicationController serves as the controller of the server functions. All major events, such as the event of reception of data from user is reported to the Appli cationcontroller. The ApplicationController then invokes the corresponding methods of - the Data, the or the ServerInterface to inform them of the event. ApplicationController ServerInterface Data Connection Application management DataForwarder Data and control flow DataPort ControlPort Data Flow Control Flow Figure 4: Server architecture Application Controller ClientInterface Data Connection Data Forwarder Configuration Transporter Translator DataPort ControlPort Encoder Decoder Figure 5: Class diagram for server International Journal of The Computer, the Internet and Management Vol. 13.No.3 (September-December, 2005) pp

6 Md. Mahbubur Rahman, Dentcho N. Batanov, Susumu Horiguchi The Server-Interface handles the interaction with the user. Therefore, it provides GUI objects for configuring and monitoring architecture components and triggers the notification of major application events to the user. The Data manages the internal and intra-applications dataflow. It uses DataForwarder object to handle data forwarding. controls the whole session of a dialogue between clients. It receives information about sessions initiated by remote clients, stores the session information and exports information about status of the sessions to the clients. The uses session description and announcement protocols for these purposes. It stores the session description in a session directory. Start Waiting for new connection If new request comes calls Connection Stop Figure 6: Server states Start GUI event 2.4 Data flow in server The DataPort, and Control-Port are the objects involved in dataflow in server. The receive() method of DataPort object is responsible for getting data from network. Data is then goes through and Dataforwarder objecs. Dataforwarder objects forward() method forward the data to other destination. only bypass it to Data- Forwarder object. But for session data, s protoencode() method performs encoding the session data. Data- Forwarder after getting the data, decides its destination and sends again to the natwork through and DataPort object. 3. Client/server Communication A predefined protocol through which client and server interact with each other is an important issue in Client/server system. Here we describe protocol we used in our implementation. No Server side protocol Request Connection to Server Connected SuperUser ONLINE message Connect Server through a DataPort Call DialogueExchanger Stop Figure 7: Client states Response comes Yes GUI event The server should be always listening to new request from users. If a user requests to connect with another remote user the server first searches its directory if the remote user is online or not. We call the user as first user and call the remote user as second user. It also registers the user in its directory. If the second user is online then it informs the second user that first user wants to talk with 6

7 A Scalable Object-Oriented Client/Server Architecture for Interactive Multimedia Applications over Internet him. If the second user agrees then the server informs the first user and opens a new connection to listen to from first user. Also it does the same for the second user. When they both connect to server through a data port the server handles data forwarding between the users. If the second user is not online it tells the first user to wait or quit. to send data to more than one client is not supported by the TCP protocol. Payload Type Sequence Number Time Stamp SenderSourRCe (SSRC) Identifier Figure 8: RTP header Client side protocol When the user wants to talk with a remote user first it connects to server to a predefined port. Also it specifies the address of the remote user. The server maintains a directory of online users who have registered to it. The user can choose a partner from the directory of server also. After requesting to the server the user waits for response to receive from server. The server informs the presence of remote users availability for conversation. If available then the user opens a new data connection to server and continues conversation. The server manages the forwarding of data. Client Application Java Virtual Machine RTP TCP UDP Server Application Java Virtual Machine Client Application Java Virtual Machine RTP TCP UDP 4. Data Distribution TCP UDP As the core Internet protocols like TCP/IP and UDP/IP do not provide any realtime functionality, therefore they are not able to fulfill the requirements needed for transmitting multimedia data. TCP is a connection-oriented protocol that is reliable including flow control and supports bytestream in full duplex mode. But it is not a good basis to transport real-time multimedia data over a network. Multimedia demands a constant bit rate even if sometimes a packet gets lost and that is exactly what TCP cannot guarantee. In multimedia applications multicast methods are often used to serve a couple of clients at the same time. So the connection oriented ability of TCP is not very helpful, allows it only a point-to-point connection on application level between a server and one client. The multicast method Figure 9: Client/server on Internet Figure 10. Audio Processing in Client International Journal of The Computer, the Internet and Management Vol. 13.No.3 (September-December, 2005) pp

8 Md. Mahbubur Rahman, Dentcho N. Batanov, Susumu Horiguchi On the other hand, UDP is a connectionless protocol that tries best effort delivery with no flow-control and included message support. But UDP is not reliable and that is one of the key advantages for real-time transmissions. It is a best effort delivery protocol. That is exactly what is needed for multimedia data. UDP also supports multicast methods. In the manner of broadcasting a video to more than one client this is exactly what is needed. Multicast simply multiplies the data on a network segment to a group of clients. It would be necessary to set up sessions with each client and to multiply the data on application level. So it would generate a multiple of streams on a single network segment. But there are not only good things on UDP. This protocol does not support flow-control and that is actually very important for real-time applications. So UDP is just a good base that goes in the right direction but it lacks a couple of key functions. The Integrated Services working group in the IETF (Internet Engineering Task Force) developed an enhanced Internet service model called Integrated Services that includes best-effort service and real-time service. The real-time service will enable IP networks to provide quality of service to multimedia applications. Resource Reservation (RSVP), together with Real-time Transport (RTP), Real- Time Control (RTCP), Real-Time Streaming (RTSP), provides a working foundation for real-time services. Integrated Services allows applications to configure and manage a single infrastructure for multimedia applications and traditional applications. It is a comprehensive approach to provide applications with the type of service they need and in the quality they choose. The Real-time Transport (RTP) provides support for the transport of real-time data such as video and audio streams. The services provided by RTP include time reconstruction, loss detection, security and content identification. RTP is designed to work in conjunction with the auxiliary control protocol RTCP to get feedback on quality of data transmission and information about participants in the ongoing session. As RTP itself does not provide any mechanism to ensure timely delivery it needs support from lower layers that actually have control over resources in switches and routers. RTP depends on RSVP to reserve resources and to provide the requested quality of service. Therefore, RTP/RTCP provides functionality and control mechanisms necessary for carrying real-time content. In our application we used RTP and UDP as vehicle for transmitting data among clients (Figure 9). 5. System Implementation Issues The prototype has been implemented [5] as a Java application. Since, Java was designed for programming on the Internet, most of its characteristics are developed in a way that it can create programs that can be executed on different machines and operating systems. Sun claims that Java is "a simple, object-oriented, distributed, interpreted, robust, secure, architecture neutral, portable, highperformance, multithreaded, and dynamic language". Much of Java's current popularity has come about because of Javacapable WWW browsers and their support for applets. Unfortunately we could not use applet due to the fact that we had to use native methods to access audio device. We have chosen windows platform because of the availability of resources. In our application we used two clients namely user 8

9 A Scalable Object-Oriented Client/Server Architecture for Interactive Multimedia Applications over Internet Figure 11: Two users are communicating (User and Super User). Upper portion of the interface is for receiving and lower portion of the interface is for sending text. and super User (Figure 7). Clients were installed in windows platform where server application was installed on a Unix platform. We have developed our application deployable on the internet environment. The problem is that pure TCP and UDP are not an appropriate vehicle for delivery of audio or other time based data as we have discussed in section 4. As our application uses audio, therefore, we used RTP along with UDP. The benefit of RTP packets is that it contains timing information and a sequence number that allow the receivers to reconstruct the timing produced by the source. 6. Conclusions We have proposed an object-oriented architecture for the multiple clients communicating through a server in a TCP/IP network. As an application, a real-time interactive client/server system has been implemented. Text and audio have been used as communication media. The software system is written multithreaded and offers a great flexibility in opening and closing connections and adding new modules because of a consequent use of objectoriented design methods. For audio exchange the system showed poor performance due to straight-forward transmission and reception of audio data. In future we hope to use some state of the art encoding techniques for audio data. Video data transmission can also be included in future applications. References [1] June S. Park., Multimedia over IP nets, [2] Booch, G., Object-Oriented Analysis and Design.The Benjamin/ Cummings Pub-lishing Company, [3] Braden, R., et al., Integrated Services in the Internet Architecture: An Overview. Network Working Group, Request for Comments: 1633, [4] Husemann, D., Multimedia Data Streams In Distributed Object-Oriented Operating Systems, PhD thesis, IMMD IV, University of Erlangen- Nürnberg, Germany, [5] Rahman, M. M., Development of Dialogue Based Object Oriented Client/ server Applications On the Internet: An Online Interviewing System, AIT Masters Thesis CS , [6] Berson, Alex., Client-server architecture. McGraw-Hill, [7] Douglas, C., Internetworking with TCP/IP - Principles, protocols, and architecture. Prentice Hall, International Journal of The Computer, the Internet and Management Vol. 13.No.3 (September-December, 2005) pp

10 Md. Mahbubur Rahman, Dentcho N. Batanov, Susumu Horiguchi [8] England, P., Allen, R., Underwood, R., RAVE: Real-time services for the Web, Computer Networks and ISDN Systems 28: , TUTUTUTUTUww5conf.inria.fr/fich_html/papers/UT, [9] Greenberg, S., Roseman, M., GroupWeb: A Groupware Web Browser, ACM CSCW'96 Conference on Computer Supported Cooperative Work, Boston, USA, [10] Evans, E., Rogers, D., Using Java applets and CORBA for multiuser distributed applications, IEEE Internet Computing. 1(3): 43-45, 1997 [11] Steinmetz, R., et al., Multimedia: Computing, Communications and Applications. Prentice Hall, Upper saddle River, N.J., 1995 [12] Seffah, A., An Object- Oriented framework for modeling and designing Intelligent Training Systems, Software -Practice and Experience, 27(10): ,