A NEW METHODOLOGY FOR IMPLEMENTING INTERACTIVE OPERATIONS OF MPEG-2 COMPRESSED' VIDEO

Transcription

1 A NEW METHODOLOGY FOR IMPLEMENTING INTERACTIVE OPERATIONS OF MPEG-2 COMPRESSED' VIDEO K. E. Psannis M. G. Hadjinicolaou Dept of Electronic & Computer Engineering Brunel University Uxbridge, Middlesex UB8 3PH, UK Kostas. uk Marious. uk Abstract We present an efjicient approach for supporting interactive operations such as Fast Forward (FF) and Fast Rewind (FR) in MPEG-based Interactive Video On Demand (IVOD) system. This approach is based on transmitting additional data of the same movie from the server to the Digital Storage Device (DSD) at a client station. The special additional data consists of Elementary Stream (ES) which has all I frames. The "interactive" data is generated from the storage at the server every n-th frame from the original MPEG-2 compressed video. During the appropriate procedure the server can pe@orm the FF/FR data in a Group of Pictures (GoPs) structure in an independent fashion. Fast Forward (FF) and Fast Rewind (FR) operations are supported using much or little extra network bandwidth than that is already allocated for normal playback. Assuming that the decoder consumes data at the constant rate of normal playback and the data transferred over the network with the average bit rate, then the number of supported speedups we can achieve depends on the Group of Pictures Lengths (N) and the recording ratio during the normal play. Keyword: Speedups, Playback Rates, Consumption Speed, MPEG-2, Fast Forward, Fast Rewind, Interactive Video On Demand (IVOD). I. INTRODUCTION Interactive television is a technology for delivering, on demand, television programs and multimedia applications to households and businesses through a broadcasting infrastructure. This scheme is applicable not only to "on-demand services'' such as videoon-demand, but also to many kinds of information delivery, such as public-information services [I]. In a Video On Demand (VOD) system multimedia streams are stored on a storage server (the video server) and played out to the user station upon request. Interactive Video On Demand (IVOD) is an extension of Video On Demand in which additional functionalities are implemented. These functionalities pose new requirements and challenges on the system implementation. Possible interactive functions include play, /01$ IEEE 21

2 stop, resume, pause, jump forward, jump backward, fast forward (FF), fast rewind (FR), Slow Down, Reverse, Slow Reverse [2]. Other interactive features include the ability to avoid or select advertisements, to investigate additional details and to browse, select and purchase goods. The difficulty of supporting interactivity in an NOD system varies from one interactive function to another. A stop, jump, or pause followed by resume are relatively easy to support, as they do not require more bandwidth than what is required for normal playback. On the other hand, Fast Forward (FF) and Fast Rewind (FR) involve displaying frames at several times the normal rate. Transporting and decoding frames at multiple times the normal frame rate is prohibitively expensive and is infeasible with today s hardware decoders. Fast Rewind (FR) is even more difficult to support in compression schemes that involve motion-interpolated frames, such as B frames in the MPEG scheme [3], [4]. In the case of MPEG, all the reference frames in a Group of Pictures (GoPs) must be decoded before B frames of that GoPs can be played back in reverse order. Several approaches have been proposed to support interactivity in a VOD system. Interactive functions can be supported by dropping parts of the original MPEG-2 video stream [5], [6]. Typically, dropping is performed after compression and aims to reduce the transport and decoding requirements without causing significant degradation in video quality. Alternatively interactive functions can also be supplorted using separate copies of the movie that are encoded at lower quality of the normal playback copy [7], [8]. Other conventional schemes that support interactive functions either display frames at rate much higher than the normal playback, for example 90 Fps [9] or involves downloading the video data in a player device (not real time playout- downloading can be done prior to viewing) located at the customer premises so that the customer can view without further intervention form the network [lo]. In this work, we introduce an efficient approach for supporting Fast Forward (FF) and Fast Rewind (FR) in an Interactive Video On Demand (NOD) system. Our approach is based on transferring additional data from the server to the digital storage at the client premises. The additional data is being generated at the server every n-th frame from the original movie, which is compressed using MPEG-2 video coding. Assuming that the decoder can consume data at the constant rate of the normal playback. In this case the available speeclups we can achieve depend on the Group of Pictures (GoPs) Length (N) and the recording ratio during the normal play. We consider that the server sends the data to the client based on the average bit rate of an MPEG-2 video file. The average bit rate of each video file can easily be determined by using any MPEG statistical program. The remainder of this paper is organized as follows. In Section 11 we describe the MPEG data structure. Section I11 provides description of the proposed methodology. In section IV we compare the additional storage at the client with the storage of the server and other extra storage device used by other method. Section V analyzes the switching operations and the supported speedups. Finally Section VI concludes the paper and points to open research. 11. MPEG STRUCTURE The structure of MPEG stream imposes several constraints on the video data storage and playout. MPEG video stream consists of intra frames I, predictive frames P, and interpolated frames B. I frames are coded such that they are independent of any other frame in the sequence. On the other hand, P frames are coded using motion estimation 22

3 and have a dependency on the preceding I or P frame. Similarly B frames depend on two "anchor" frames: the preceding I/P frame and the following I/P frame. In figure 1 we can see the inter-frame dependencies in a sequence of MPEG frames. Figure 1 MPEG Stream Two parameters, N and M, describe the succession of I, P and B frames k N is the distance between two successive I frames, defining a "Group of Pictures" (GoPs). k M is the distance between consecutive I or P frames. Usually set to 3. Note that, N must be a multiple of M. (N = a xm ).M =1 means no B frames in the sequence. M =O implies only I frames PROPOSED METHODOLOGY The block diagram of the proposed model is depicted in Figure 2.In our proposed work we create in real time an alternative data stream from the storage of the server. In this case we consider that the server can support the creation of two data streams of the same storage. The additional data is originated from the compressed MPEG-2 video and transferred into the digital device over the same channel if the bandwidth permits or over a communication channel that is different from the one used for normal playback The FFFR data consists of an Elementary Stream (ES), which has all I-frames. We refer to the additional data as the FF/FR data and the one used for normal playback as the MPEG- 2 data. Fast Forward /Fast Rewind data 'i'i+lbi+4s'i+60',+9~i+l*~ Digital Storage Device (DSD) Assume that the characteristic of an MPEG-2 video file is that shown in Table I 23

4 ~.. Frames I-frame size P-frame size B-frame size Max (bytes) Average (bytes) Table 11: GoPs Lengths (N), frames ratio and recording ratio Table I1 (a), (b), (c) provides a typical GoPs Lengths(N), frames ratio and recording ratio. The bandwidth for normal play is given by Re cording ratio x AverageFrane(1PB)Size x 8 bits byte (1) / where, 1 1 AverageFrame(IPB)Size=Iav""ee + PmcrOs, x (--- ) + BIlvrrrrsr x (1--) 1 (B ytedframe) N M N M According to table I and Table II (a), (b), (c), (d) we get from (10) (a> Re cording ratio x AverageFrane(IPB)Size x8bi%yte = 1.99Mbps (b) Recording ratio xaveragefrane(ipb)size xsbifsbyte = 1.6Mbps (C) Re cording ratio x AverageFrane(IPB)Size x8 = 1.8Mbps (d) Re cording ratio x AverageFrane(IPB)Size x8bifsbyte = 2.2Mbps In figure 3 we can see that the server can vary the additional bit rate by varying the frame rate. For example if the server transmits additional data 6I/sec it will achieve 0.99Mbps and the total bit rate will be 2.98Mbps(a), 2.59Mbps (b), 2.79Mbps ~(c), 3.19Mbps (d) m, 12 r - 10 r s 4 / / /, Bit rate (Mbps).I.. _. IfpslBit rate IfpslTotal Bit rate lfpsnotal Bit rate Ifps-Total Bit rate Ifos-Total Bit rate I Figure: 3: Increase of the bit rate as a function of the number of I frames per second. 24

5 IV. COMPARISON THE ADDIITONAL STORAGE We can compute the additional storage at the client end Wclient as a function of the storage at the server Wse,er, or the extra storage Wclien,(P+I) required by P + I conversion method [lo] as follows. where, = f ('average 9 'average ) Q, = f ('average 9 'average 9 'average 9 M VJ = f ('average 9 'average 7 M ) According to table I, table 11, (2) and (3) we get 5= l.llxa, Wclienr (4) Table 111: Computation of - WSemer W server 1 W client W client(p-11) client Wclient and Wclient(P+i) Wclienl for a =3,4,5,6 a =3 a =4 a =5 a =6 3,76 4,87 5,98 7,09 4,62 6, , a Figure4: - Wserver or Wclient Wc[ient( P+i) Wclient as a function of CY 25

6 / Wcljent and Wcljen,(,D+,, / Wclienr Figure 4 shows that as a increases the ratio W, increases linearly. By reducing the variable@ we reduce the ratio of W,,,,,/ Wcljenr. For a =5 we have W, / Wclient =5,95. This indicates that the capacity of our extra storage is {Equation(4)) less than the storage of the server depending on the Group xa of Pictures Length (N) ( N = ax M ). In the same graph of Figure 4 we can see that our extra storage is less than the additional storage based on P+I conversion method [lo]. For example for a=4 our extra storage is 1/6 times less approximately than the additional storage required by P +I method. From (4) and (5) we get YliP t(p4) 1.54xa - W,,,, xa (6) For a=5 we have from (6) that the additional storage required tly P+I conversion method Wcljent(P+,) is 1.28 times higher than the storage of the seiver W,,,,,. This is because P-I method converts first the P frames into I frames and then stores all the frames of each GoPs. V. SWITCHING OPERATIONS AND SUPPORTED SPEEDUPS Interactive operations such as FF/FR are implemented both at the server and the Digital Storage Device (DSD). Switching from one version to another is performed online, in response to a client request. To maintain the GoPs periodicity, switching from a normal to an interactive version must take place at an I frame. Moreover to enable correct decoding of all the P and B frames of the normal version, this I frame must be common to both versions. When the FF/FR request arrives at the server, the server continues to send frames from the normal version up to and excluding the first P frame that follows a common I frame. From that point and on, the Digital Storage Device (DSD) switches to the interactive FFRX mode. Generally there is no disruption in the playback during the transition from normal to E/FR mode and vice versa. The swiitching operation is transparent to the decoder. The number of supported speedups we can achieve using our methodology depends on the consumption rate of the decoder. Supporting variable playback rates is analogous to allowing slow motion at the client erid. Slow motion can be performed at the client end by a software decoder. Current MPEG hardware decoders do not allow variable playback rates. Assume that the decoder can consume data at constant rate of normal playback, then the number of the supported speedups depends on the Group of Picture Lengths (N) and the recording ratio during the normal play. If the decoder consumes data at higher or lower rate than the one specified it would result in slight hiccups at the client end. This phenomenon will occur in any system where the server s or the DSD s production rate differs from the consumption rate of the decoder. Either the decoder will eventually starve or overrun its buffers. It is certainly conceivable that different end users have different hardware decoder cards or even software decoders, 26

7 each with different consumption rates We consider that the server sends data to the decoder based on the average bit rate of MPEG-2 video (table I). Table IV: Computation of the supported speedups for different GoPs Lengths, frames ratio and recording ratio (table I1 (a), (b), (c), (d)) GoP (N) Frames ratio (FPS) Table I1 (a)9(b),(c),(d) (a) (b) (c) (dl Consumption Consumption Differences rate of I Frames rate of in Speedups Normal Play Rate Interactive consumption (Mbyteshec) Mode rate (Mb yteslsec) % 0, , , , , , , , For example for Group of Pictures (GoPs) Length N=15 and recording ratio 3Ofps (table 11 (a) we can achieve a speedup of 6. VI. CONLUSIONS In this paper we presented an approach for supporting interactive operations in an IVOD system. Fast Forward (FF) and Fast Rewind (FR) operations are supported by generating additional data of each movie. Using I frames only for the interactive mode we can support both Fast Forward (FF) and Fast Rewind (FR). The I frame at the beginning of a GoPs serves as a basic entry point to facilitate random seek or channel switching and also provides coding robustness to source coding and transmission errors. In addition I frames are decoded such that they are independent of any other frame in the sequence. Another advantage comes using our methodology is that we don't use an extra storage at the server. Moreover our extra storage 'at the client station is less than the storage of the server and the additional storage required by P+I conversion method.the number of supported speedups we can achieve using our methodology depends on the Group of Pictures (GoPs) Length (N) and recording ratio during the normal playback In an IVOD system clients should be given the flexibility to choose from a set of available services that offer different levels of interactivity. Billing would then be done based on the quality and flexibility associated with the selected service. Our future work includes analysing the limitations on the interactive functions (visual quality, maximum duration of the interactive mode) supports other interactive functions (pause, jump forward, jump backward, Slow Down, Reverse, Slow Reverse) and proposes a method for transcoding from Variable Bit Rate (VBR) Elementary Stream (ES) to Constant Bit Rate (CBR) Elementary Stream. One other point for open research is to design and integrate the Digital Storage Device to the next generation of Integrated Receivers Decoders (IRD). 27

8 ACKNOWLEDGMENTS The authors wish to thank all who.reviewed this paper, especially Dr A Krikelis the Chief Technology Officer of Aspex Technology for his helpful comments. REFERENCES T.Sanuki, Y. Asakawa, Design of a video-server complex for interactive television, IBM Journal of Research & Development, 1998 Miranda KO, Irene Koo, An Overview of Interactive Video On Demand System, Computer Science Department of University of British Columbia, 1996 D LeGall. MPEG: A Video Compression Stahdard for Multimedia Applications CO. of the ACM, Vol. 34, No 4, April 199 1, pp International Standard Information Technology - Generic Coding of Moving Pictures and Associated Audio: Information Video. Banu Ozden, Alexandros Biliris, Rajeen Rastogi, Avi Silberschatz. A Low-Cost Storage Server for Movie on Demand Databases. In: Roc. of the 20th VLDB Conference, Santiago, Chile, 1994.pp Michael Vemick, Chitra Venkatramani, Tzi-cher Chinueh. Adventures in Building the Stony Brook Video Server. In: Proc. ACM Multimedia, November 1996 Boston, MA, pp Marwan Krunz, George Apostolopoulos. Efficient Support for interactive scanning operations in MPEG-based video on video on demanti. Multimedia Systems, vo1.8, no. 1, Jan. 2000, pp Prashant J.Shenoy, Harrick M.Vin. Efficient Support For Scari Operations In Video Servers. In: Proc. ACM Multimedia Conference, November ACM Press, San Francisco, CA, pp Jayanta K Dey-Sircar, James D.Salehi, James F Kurose, Don Towsley. Providing VCR Capabilities in Large-scale Video Servers In: Proc. AChl Multimedia Conference, Oct 1994, San Francisco, CA, pp M-S.Chen, D.D. Kandlur. Downloading and Stream Conversion: Supporting interactive Playout of Video in a Client Station. In: Proc IEEE Multimedia Conference (1995).pp