Heidelberg Transport System[6], which copes with the constraints of system environments, provides media scaling to keep the consistency of the present

Transcription

1 MSDL-II: A Scenario Language with QoS Scheme in a Distributed Multimedia System Kazutoshi Fujikawa fujikawa@is.aist-nara.ac.jp Graduate School of Information Science, Nara Institute of Science and Technology Ikoma, Nara Japan Toshio Matsuura matsuura@media.osaka-cu.ac.jp Faculty of Human Life Science, Osaka City University Osaka, Osaka 558 Japan Takahiro Shiroshima Shinji Shimojo Hideo Miyahara fsirosima,shimojo,miyaharag@ics.es.osaka-u.ac.jp Department of Information and Computer Sciences, Faculty of Engineering Science, Osaka University Toyonaka, Osaka 560 Japan Abstract Recently, QoS control mechanisms have been proposed for distributed multimedia systems. However, in such systems QoS control is done by the system or by its users, it is desired that appropriate QoS control is specied in the scenario which represents contents of the presentation. It is not suciently discussed how to deal with such specications. In this paper, we propose a new scenario description language for a multimedia presentation, called "MSDL-II", which can specify QoS control scheme along with each scene. Keywords Multimedia, Scenario description, QoS parameters, QoS control 1 Introduction Recently, high speed network and multimedia workstations/personal computers bring us a possibility to realize distributed multimedia applications such as a video-on-demand system or a desktop video conferencing. WWW(World Wide Web)[1] and Gopher[9] are examples of distributed multimedia applications in wide area network. Such kind of applications can become sophisticated to a distributed multimedia information services where a client retrieves multimedia data from dierent servers on the y and plays back at client's site in an integrated fashion[2, 3] when connecting with high speed wide area networks. Thus, the basis for a distributed multimedia presentation system have been provided. Since current multimedia applications run on various system environment such as networks, server/client machines and operating systems, an important scene of a video material may be skipped or an important portion of a lecture may not be heard due to instant network overload or lack of hardware capacity. There can be the case that the original intention of an author who make the presentation of multimedia information, may be lost. Consequently a user can not get the presentation of multimedia information as expected. However, even if computer/network resources such as available network bandwidth or CPU capacity are not enough, an author wishes to keep his intention of the presentation as possible. A notion called QoS(Quality of Service) was proposed mainly on network protocols to designate user's requirement for the quality when transferring data. Some network protocols which can guarantee or try to guarantee QoS such as ATM[10], ST-II[15] and HeiTP[6] are proposed and implemented.

2 Heidelberg Transport System[6], which copes with the constraints of system environments, provides media scaling to keep the consistency of the presentation by degrading the quality of multimedia information when the requested quality can not be archived in the transport system. In media scaling, in order to estimate the QoS of a presentation, the minimum required QoS parameters should be specied. QtPlay[13], which is a video-on-demand system, guarantees the quality of audio/visual information by reserving the necessary network bandwidth and coordinating the scheduling of the operating system. In QtPlay or nv as a video teleconferencing tool, a user adjusts QoS parameters when displaying multimedia information. We call such systems user control types. In a multimedia presentation, the required QoS parameters may dier from each scene. However, it is dicult for a user to specify the necessary QoS parameters for each scene of a presentation in user control types. Moreover, a user can not specify the appropriate QoS parameters which reect the author's intention. MODE[2], HeiTS[6] and AMF[16] propose QoS control mechanisms. In these systems, the QoS parameters is automatically specied by the system. Or, once a user species the QoS parameters, he can not change the QoS parameters dynamically according to each scene of a presentation. We call such systems system control types. System control types can not cope with the request that the QoS parameters should be specied appropriately according to each scene of a presentation. In order to preserve the author's intention in a multimedia presentation, it is reasonable that an author species QoS parameters in a scenario which represents the multimedia presentation. Therefore, it is not necessary for a user to specify QoS parameters while seeing a multimedia presentation. In this paper, we propose a new scenario description language called \MSDL-II", suitable for a multimedia presentation, in which an author can specify QoS parameters. Moreover, in order to prove the eectiveness of MSDL-II, we have implemented a distributed multimedia presentation system \Symphony"[8] which can interpret MSDL-II. In the next section, we discuss the eciency and the problem specifying QoS parameter in scenario description language. In section 3, we dene MSDL-II based on the proposed scenario model. In section 4, we mention Symphony and we conclude in section 5. 2 QoS Control by Scenario Description In this section, we consider how QoS parameters should be specied for a multimedia presentation, and describe the eectiveness of our proposed QoS control type that QoS parameters are specied in a scenario description which represent a multimedia presentation. Moreover, we discuss the key issues when the QoS scheme is specied in a scenario description. 2.1 Requirement of QoS Specication First, we discuss the requirement of QoS control for a multimedia presentation. Some types of network protocols like ATM, ST-II or HeiTP can guarantee or try to guarantee QoS parameters such as delay, jitter, and bandwidth designated by a user. Another ones, like TCP/IP, which can not guarantee QoS provide only best-eort service. Even if we can specify QoS parameters in network protocol level, it is dicult for a user to designate these parameters directly. Therefore, a denition of QoS for application perspective are required. Especially, when considering QoS in application level, QoS for multiple media such as video, audio, or animation should be integrated and synchronization of streams for these media is also considered. Accordingly, we should dene application level QoS parameters. The elements representing the quality of multimedia information are classied into intra-media QoS parameters and inter-media QoS parameters. As intra-media QoS parameters, we consider the following four parameters: Latency represents an allowable period which the presentation of an information is delayed. Temporal crops represent an error rate of information in temporal scale. In case of motion video or audio, displaying of some portion are skipped or postponed due to the system condition. Therefore, this parameter limits a rate of improper portion of information.

3 Spatial crops represent a rate of skipped information in spatial scale. Spatial scales are resolution, color scale, display size, or display area for motion video and sampling rate or sampling bit number for digital audio. This parameter limits the degradation of spatial resolution. Continuity represents smoothness of degrading information. And, as inter-media QoS parameters, we consider the two parameters as follows: Latency represents an allowable delay which the presentation of a scene begins with. SyncLag represents an allowable time lag between start times of multiple information. SyncType represents the manner how to keep the synchronization among multiple information when the synchronization is disordered. Each multimedia information to be displayed has its own requirements related to QoS control, which is how these QoS parameters should be preserved, or adjusted when the desired QoS parameters can not be achieved. Even if the same information is displayed, the QoS control requirements may vary from one condition to another. Moreover, it is considered that the medium of an information to be displayed should be changed to another one, if the requested QoS parameters can not be satised. For instance, a motion video is changed to an animation which represents the same information, or an audio is changed to a text which has the same meaning. We call such situation media substitution. Based on the above discussion, when specifying QoS parameters in a multimedia presentation, the following schemes are required: The minimum requirements of intra-media/inter-media QoS parameters. The manner how to degrade the quality of a presentation or substitute media, if the desired QoS parameters can not be preserved. 2.2 The Eectiveness of QoS Control by Scenario Description In this paper, we propose a new QoS control model suitable for a multimedia presentation, which is called scenario control type. In a scenario based on this type, an author can specify QoS schemes according to the requirements of each scene in a multimedia presentation. In the scenario control type, since the application controls QoS based on a scenario description, it is not necessary for users to specify QoS parameters over and over along with the progress of the presentation while seeing the presentation of the scenario. 2.3 The Issues of QoS Control Specied in Scenario When QoS schemes are specied in a scenario, there exist the following problems: Complication of description QoS schemes should be specied appropriately for each scene along with the progress of a scenario. Therefore, in scenario two types of descriptions should be needed: one represents the progress of presentation, and the other does the QoS specication. A scenario description is required that an author can easily understand the progress of the presentation. However, when QoS schemes are specied in a scenario, the scenario descriptions for the progress of presentation may be mixed up with that for the QoS specication. In this case, the scenario description becomes very complicated. Consequently, the understandability of the progress of a presentation is reduced.

4 Dependency on system environments It may be considered that a method to degrade QoS parameters is described procedurally in a scenario, when the requested QoS parameters can not be achieved. In such a case, it is necessary for an author to grasp all system environments, in order to describe an appropriate method, because the load of CPU/network or the existence of special devices such as video board or digital sound processor dier from each environment. However, the description which depends on system environments causes a scenario to be more complicated and more expanded. Handling multiple media QoS schemes for multiple media or a scene should be specied in an integrated fashion including synchronization among them as well as QoS schemes of a single medium. It is important to handle the QoS specication for multiple media easily. 3 A Scenario Description Language \MSDL-II" Considering the problems as described in the previous section, we propose a new scenario description language \MSDL-II 1 " suitable for a multimedia presentation. In this section, we describe a design overview of MSDL-II and show its example. 3.1 Design Overview Avoiding complication of description caused by the mixture of the descriptions of the presentation schemes and those of the QoS schemes, in MSDL-II a scenario consists of the object denition part where objects employed in a presentation are dened and the scenario progression part where presentation schemes are described. QoS schemes are specied in the object denition part by dening an information as an object including QoS specication. In the scenario progression part, an author describes only a presentation scheme, that is, the layout and the displaying order of each object. In MSDL-II an author can specify some candidates which have dierent QoS parameters for each object, in order that a presentation system can select an appropriate one according to the system environment. Thus, an author need not specify procedural method for selecting the appropriate one form the candidates. Moreover, in order to specify QoS schemes to multiple media or a scene, MSDL-II can dene a scene including several media as an object. To realize these design basis, MSDL-II provides three types of objects as follows: primitive objects Media objects, such as string, image, video and audio, can be specied as primitive. QoS parameters can be specied as their attributes. alternative objects For each object identier, several candidates, which are primitive, can be specied as an alternative object. composite objects Another scenario can be specied as an object. Primitive objects have the following attributes: Location species object's location. Its value is URL description 2. TemporalCrop 1 Multimedia Scenario Description Language II 2 URL is determined in RFC1738, and used as an identier in WWW or Gopher.

5 SpatialCrop TemporalCrop and SpatialCrop represent allowable rates against original quality of an information. That is, the range of their values is from 0(%) to 100(%). If the value is 0, it means that the quality should not be degraded at all. Latency The value is specied in seconds. Continuity represents the interval between non-skipped portions in an information, when degrading the information by skipping some portions. The value is specied in seconds. SyncLag represents an allowable time lag among synchronized several objects. The value is specied in seconds. SyncType represents the manner to adjust QoS. Its value is one of Realtime, Motion, and Sound. When Realtime is specied for a video object, the system does its best to keep the real-time property even if some frames may be skipped. When Motion is specied, all the information are displayed without loss of any portion even if the real-time property is ignored. When Sound is specied for an audio object, the system does its best for a user to be able to listen the audio clearly by pausing it in silent periods[7]. When displaying several object simultaneously, every information does not necessarily has the same priority. Therefore, in MSDL-II an author can specify the priority among objects. 3.2 A Sample Description of MSDL-II Now we show a sample description of MSDL-II(see Figure1). We consider an interactive TV news program as an example. In the object denition part from the third line to the twenty-third line, six objects are dened. On the third and forth lines there are two primitive objects named \menu1" and \menu2" which are headlines. And on the fth line to the twenty-second line there exist four alternative objects named \video1", \video2", \narration1" and \narration2". Video1 is a motion video compressed in MPEG format. In this example, QoS control scheme for an object is expressed by specifying two candidates which have dierent QoS parameters for the same MPEG information. This object is specied that if the QoS parameters of the rst candidate can not be preserved, the second candidate will be displayed instead of the rst one. Narration1 has an audio information as the rst candidate and a subtitle by an animation, which represents the same meaning as the audio information, as the second candidate. Video2 and narration2 are dened in similar way to video1 and narration1 respectively. On the twenty-forth line, the priority of narration1 is specied higher than that of vide1. Also, the priority of video2 is specied higher than that of narration2. The scenario progression part which species the spatial and temporal arrangement of objects is on the twenty-fth line to the forty-ninth line. There are three presentation schemes main, course1 and course2. When the scenario begins, rst of all main will be executed. In this example, rst the two headlines menu1 and menu2 are put on the screen. These headlines become sensitive to mouse click in order that a user can select the headlines. Until a user selects one of the headlines, they move between right side and left side of the screen. When menu1 is selected, the presentation of course1 will begin. Also, when menu2 is selected, that of course2 will begin. During the presentation of course1, if the system environment provides sucient network bandwidth and CPU power and special device such as a video board or a digital sound processor, this scenario is realized in the highest quality video and audio. Otherwise, the quality of motion video is degraded and audio information is substituted by an animation. 4 Design of A Presentation System for MSDL-II We have designed and implemented a presentation system called Symphony which interprets descriptions of MSDL-II.

6 1 Title: "TV news", 2 Object: %% object definition 3 { "menu1" String(Value: "Politics"), 4 "menu2" String(Value: "Sports"), 5 "video1" Alternative( 6 Candidate: Mpeg(Location: "cmtp://.../v2.mpg", 7 TemporalCrop: 50, 8 SpatialCrop: 0, 9 SyncType: Realtime), 10 Candidate: Mpeg(Location: " 11 TemporalCrop: 80, 12 SpatialCrop: 20, 13 SyncType: Realtime)), 14 "narration1" Alternative( 15 Candidate: Audio(Location: "cmtp://.../v2.au", 16 TemporalCrop: 40, 17 Latency: 100, 18 SyncType: Sound), 19 Candidate: Composite(Location: " 20 SyncType: Realtime)) 21 "video2" Alternative(...), 22 "narration2" Alternative(...), 23 }, 24 Preference: narration1 > video1, narration2 > video2, 25 Behavior: %% scenario progression 26 main { 27 [nemu1 <-put(at: 200@100)]; 28 [menu2 <-put(at: 500@200)]; 29 [menu1 <-sensitive(action: course1)]; 30 [menu2 <-sensitive(action: course2)]; 31 [while TRUE 32 [menu1 <- move(to: 500@100, Duration: 5) 33 menu2 <- move(to: 200@200, Duration: 5)]; 34 [menu1 <- move(to: 200@100, Duration: 5) 35 menu2 <- move(to: 500@200, Duration: 5)]; 36 endwhile]; 37 }, 38 course1 { 39 [video1 <-put(at: 300@300)]; 40 [narration1 <-put(at: 400@400)]; 41 [video1<-play() narration1<-play() ]; 42 [return]; 43 }, 44 course2 { 45 [video2 <-put(at: 300@300)]; 46 [narration2 <-put()]; 47 [video2<-play() narration2<-play() ]; 48 [return]; 49 } Figure 1: An example of MSDL-II

7 4.1 Architecture for QoS Guarantee In the environment where Symphony is implemented, there are a client where a user can see a multimedia presentation and several servers which provide the client with multimedia data. The client executes actual multimedia presentation with servers transmitting multimedia data. In order to guarantee QoS request in such distributed multimedia applications, it is required to manage resources suitable for transmitting and playing back multimedia data, such as network bandwidth or memory size. In this case, there are three problems as follows: 1. There are several QoS types which are provided by network protocols or operating systems. 2. There is a gap between the QoS provided by network protocols or operating systems and the QoS requested by applications. It is necessary to ll the gap. 3. It should be done to balance QoS requests among client, servers and network. In [11], three types of QoS guarantee are classied in the network protocol, compulsory, threshold, and best eort. In compulsory type, the negotiated resources are absolutely guaranteed. This type corresponds to CBR(Constant Bit Rate) in ATM terminology. In threshold type, the bounded resources are guaranteed with a probability. This type corresponds to VBR(Variable Bit Rate) in ATM. In best eort type, Any resource is not guaranteed. Resources are provided as much as possible. TCP/IP is a best eort network protocol and UNIX is a best eort operating system. Recently, some network protocols which have QoS guarantee facility are proposed[10, 15, 17]. Also, some operating systems provide with deadline scheduling and resource reservation facility[14, 5]. Therefore, in order to satisfy the QoS request of applications, it is necessary to cope with the above QoS guarantee facilities in network protocols and operating systems. The QoS parameters provided by network protocols are bandwidth, jitter, or delay, and those provided by operating systems are memory size or CPU time slice. However, the QoS parameters requested by applications are frame rate or Q factor in MPEG video. In order to satisfy the requested QoS, it should be possible to estimate the QoS parameters of network protocols and operating systems such as bandwidth or memory size from those of applications such as frame rate or Q factor. While it is easy to do so in a simple application handling only single medium even if dealing with motion video, it becomes dicult in a complex application displaying multiple media like a multimedia presentation. Although network protocols and operating systems have the QoS guarantee facility, it is necessary to balance the QoS requirements among a client, servers, and networks. For instance, it is considered that network can't provide with enough bandwidth even if a server provides with CPU resources enough to transmit a motion video. Consequently, it is dicult to satisfy the requested QoS in such case. Therefore, a QoS management mechanism among a client, a server and a network is required. [4] proposes an integrated architecture to achieve the requested QoS in a distributed multimedia application, because the resource management is spread out into each component of multimedia system such as an application, a network and an operating system. QoS broker[12] balances the QoS requirements among a client, a server and a network in a distributed system. However, since only very simple applications are considered such as transferring a few streams of audio and video in these systems, the behavior of applications is assumed to be known a priori such as processing rate of displaying video at certain frame size and rate. Therefore, it is dicult to deal with a application such as a multimedia presentation in which multiple media are related with each other complicatedly. In addition, only compulsory type of network protocols is considered. Since we are trying to handle complicated applications like a multimedia presentation, we can't know behavior of applications perfectly. Therefore, we estimate the necessary QoS from the history information of previous executions or a scenario of a multimedia presentation[8]. If the actually required QoS exceeds the estimate one during execution, media scaling will be performed. Adjusting the QoS parameters such a way as above, it is possible to cope with threshold type systems and best eort type systems in addition to compulsory type systems. 4.2 Overview of Symphony

8 User Local Area Network Interaction Scenario X Server Devices Presentation Manager Scenario Manager Multimedia Data QoS Manager Wide Area Network Media Server Media Server Media Server DB DB DB Figure 2: The system architecture of Symphony

9 The structure of Symphony is given in Figure 2. Symphony consists of media servers, presentation manager, scenario manager, and QoS manager. Media servers have media scaling facilities which can adjust the qualities of temporal and spatial resolutions. Therefore, they can transmit continuous media data such as motion video or audio according to the system environments. The scenario manager interprets the description of MSDL-II and instructs the presentation manager to play back the multimedia information. The QoS manager balances the QoS resources among a client, servers, and a network. The presentation manager has the responsibility of displaying the multimedia information which is transmitted from media servers. It consists of an execution control part and media components corresponding to each medium. A user on a client species a scenario to see a multimedia presentation. Before the presentation begins, the scenario manager interprets the specied scenario and creates a application QoS request table which describes the requested QoS of application level along with the progress of the presentation. The application QoS request table is divided into subtables corresponding to each scene and the subtables are sent to the presentation manager. At the same time, the scenario manager estimates the necessary QoS for transmitting multimedia information from media servers. The presentation manager estimates the required resources of networks and operating systems from the QoS requested by each media component, and creates a system QoS request table which is sent to the scenario manager. the QoS manager receives the system QoS request table from the scenario manager, and adjusts the QoS parameters among a client, servers, and a network. Then admission control will be performed and the necessary resources will be reserved, in case that the system is compulsory type or threshold type. In case of best eort type, the available QoS is inferred from monitoring the current resource utilization. Then admission control will be done based on the inferred QoS. Such admission control and QoS adjustment are performed in the QoS manager which is an enhanced QoS broker. If the requested QoS described in the system QoS request table can't be achieved, the QoS manager notify the scenario manager of that. The scenario manager decides the next best policy based on the description of the scenario, and creates a new system QoS request table. When the QoS parameters are xed, the presentation manager begins a presentation. While the presentation is executed, each media component monitors the QoS of information. If the QoS can't be achieved as requested, media scaling will be performed based on the description of the scenario. Now, we have implemented the prototype system of Symphony on SPARCstation 10 whose operating system is Solaris 2.3. The prototype system is implemented as an application Motif. As such environment, currently it is implemented only for best eort type. 5 Conclusion In this paper, we proposed a new scenario description language MSDL-II which can specify the QoS scheme in addition to the presentation scheme, and showed an example of MSDL-II. Also we have designed and implemented a multimedia presentation system \Symphony" for MSDL-II on the best eort type system. By using MSDL-II. an author can specify the presentation scheme and the QoS scheme without losing the understandability of a scenario. MSDL-II provides the facility that a user can see a presentation according to the system environment without adjusting QoS parameters by the user. Currently we have nished the design of MSDL-II, and now enhanced Symphony to cope with the compulsory type system and the threshold type system. References [1] T. Berners-Lee, \HyperText Transfer Protocol { HTTP/1.0," Internet Draft, Dec [2] G. Blakowski, J. Hubel, and U. Langrehr, \Tool Support for the Synchronization and Presentation of Distributed Multimedia," Computer Communications, Vol.15, No.12, pp.611{618, Dec [3] D.C.A. Bulterman, \Specication and Support of Adaptable Networked Multimedia," ACM/Springer Verlag Multimedia Systems Journal, Vol.1, No.2, pp.68-76, Sep [4] A. Campbell, G. Coulson, and D. Hutchison, \A Quality of Service Architecture," ACM Computer Communication Review, Vol.24, No.2, pp.6{27, April 1994.

10 [5] G. Coulson, A. Campbell, P. Robin, G. Blair, M. Papathomas and D. Hutchison, \The Design of a QoS Controlled ATM based Communications System in Chorus," Internal Report No.MPG-94-05, Department of Computing, Lancaster University, March [6] L. Delgrossi, C. Halstrick, D. Hehmann, R.G. Herrtwich, O. Krone, J. Sandvoss, and C. Vogt, \Media Scaling for Audiovisual Communication with the Heidelberg Transport System," Proceedings of ACM Multimedia '93, Anaheim CA, USA, pp.99{104, Aug [7] K. Fujikawa, S. Shimojo, T. Matsuura, S. Nishio, and H. Miyahara, \The Synchronization Mechanisms of Multimedia Information in the Distributed Hypermedia System Harmony," Proceedings of the 1st International Conference on Multi-Media Modeling, pp , Singapore, Nov [8] K. Fujikawa, S. Shimojo, H. Shimamura, Y. Teranishi, T. Matsuura, and H. Miyahara, \Application Level QoS Modeling for A Distributed Multimedia System," Proceedings of 1995 Pacic Workshop on Distributed Multimedia Systems, pp.44{51, Honolulu HI, USA, March [9] M. McCahill, \The Internet Gopher Protocol: A Distributed Server Information System," Connexious { The Interoperability Report, Vol.6, No.7, July [10] D.E. McDysan and D.L. Spohn, \ATM Theory and Application," McGraw-Hill, Inc, [11] I. Miloucheva, \Quality of Service Research for Distributed Multimedia Applications," Proceedings of 1995 Pacic Workshop on Distributed Multimedia Systems, pp.148{155, Honolulu HI, USA, March [12] K. Nahrstedt, and J.M. Smith, \The QOS Broker," IEEE Multimedia, Vol.2, No.1, pp.53{67, [13] T. Nakajima, and H. Tezuka, \A Continuous Media Application supporting Dynamic QOS Control on Real-Time Mach," Proceedengs of ACM Multimedia '94, San Francisco CA, USA, pp.289{297, Aug [14] H. Tokuda, T. Nakajima, and P. Rao, \Real-Time Mach: Towards a Predictable Real-Time System," Proceedings of USENIX Mach Workshop, Oct [15] C. Topolcic, \Experimental Internet Stream Protocol, Version 2(ST-II)," RFC1190, Oct [16] G. von Rossum, J. Jansen, S. Mullender, and D.C.A. Bulterman. \CMIFed: A Presentation Environment for Portable Hypermedia Documents," Proceedings of ACM Multimedia '93, Anaheim CA, USA, pp.183{ 188, Aug [17] L. Zhang, S. Deering, D. Estrin, S. Shenker, and D. Zappala, \RSVP: A New Resource ReSerVation Protocol," IEEE Network, Vol.7, No.5, pp.8{18, Sep