Implementing Interactive Operations of MPEG-2 Compressed Video

Transcription

1 Implementing Interactive Operations of MPEG-2 Compressed Video Kostas Psannis Marios Hadjinicolaou Department of Electronic & Computer Engineering Brunel University Uxbridge, Middlesex, UB8 3PH United Kingdom Abstract: - A new method for supporting Fast Forward and Fast Rewind operations of MPEG-2 compressed video is presented. Transcoding of MPEG intra-coded frames can be used to support operations. The effectiveness of our approach in terms of reducing the per- (I) frame allocated bandwidth of the alternative stream is evaluated through extensive simulations. Moreover the visual quality during the mode is analyzed. Key-Words: - MPEG- Fast Forward/Rewind operations--vcr control operations-scanning operations Introduction Digital Video Cassette Recording (VCR) functionality enables quick and user-friendly browsing of multimedia content, and thus is highly desirable in streaming video applications. It is generally believed that video-on-demand services which support VCR-like functions will be one of the most demanding residential services to be provided in emerging high-speed networks. Users are no longer restricted to being passive watchers. They are allowed to choose the program contents, to decide the viewing schedule, and to interact with the programs with such operations as stop, resume, pause, jump forward, jump backward, fast forward (FF), fast rewind (FR), Slow Down, Reverse and Slow Reverse. Thus, the video server must be capable of accommodating numerous concurrent user requests to watch and to interact with different parts of the video stream. The difficulty of supporting interactivity in a Video on Demand (VoD) system varies from one function to another. A stop, jump, or pause followed by resume are relatively easy to support, as they do not require more bandwidth than what is required for normal playback. On the other hand, Fast Forward (FF) and Fast Rewind (FR) involve displaying frames at several times the normal rate. Transporting and decoding frames at multiple times the normal frame rate is prohibitively expensive and is infeasible with today s hardware decoders. In the case of MPEG, all the reference frames in a Group of Pictures (GoPs) must be decoded before B frames of that GoPs can be played back in reverse order []. Fast Rewind (FR) is even more difficult to support in compression schemes that involve motioninterpolated frames, such as B frames in the MPEG scheme. Several approaches have been proposed to support interactivity in a VOD system. Interactive functions can be supported by dropping parts of the original MPEG-2 video stream [2], [3]. Typically, dropping is performed after compression and aims to reduce the transport and decoding requirements without causing significant degradation in video quality. Alternatively functions can also be supported using separate copies of the movie that are encoded at lower quality of the normal playback copy [4], [5]. Other conventional schemes that support functions either display frames at rate much higher than the normal playback, for example 9 fps [6] or involves downloading the video data in a player device (not real time playoutdownloading can be done prior to viewing) located at the customer premises so that the customer can view without further intervention form the network [7]. Moreover functions can be supported by transmitting additional data of the same movie at the server to the Digital Storage Device (DSD) at the customer premises [8]. In this work, we introduce an efficient approach for supporting Fast Forward (FF) and Fast Rewind (FR) in an Interactive Video on Demand (IVoD) system. Our method uses dual bit streams at the server to support operations. Based on the dual-bit-stream structure, we propose a frame selection scheme at the server to minimize the required network bandwidth and the decoder complexity. Multimedia Systems & Parallel Computing Research Group, Department of Electronic & Computer Engineering, Brunel University, Uxbridge, Middlesex UB8 3PH, United Kingdom

2 The rest of the paper is organized as follows. In Section 2 we discuss the MPEG-2 structure. In Section 3 we describe the generation of the bitstream. Section 4 presents the rate control schemes for the bitstream. Section 5 analyzes the network bandwidth allocation for the normal playback and the mode. Section 6 outlines the number of supported speedups. In section 7 the visual quality is analyzed through extensive simulations. Finally, conclusions and some points for open research are given in Section 8. 2 MPEG-2 Structure An MPEG-2 sequence is typically partitioned into small intervals called GoPs (Group of Pictures), which in turn are categorized by I (intracoded or intrapicture), P (predicted), and B (bi-directional predicted). The number of bits per GoP is distributed such that allocation for an I-picture is more than that for P-picture This is because a P-picture uses motion estimation (ME) technique to estimate its content; as a result, a motion-compensated frame difference (MCFD) with lower entropy than the original source is encoded. B-pictures use the smallest number of bits because their (ME) techniques are more intensive than those for P-pictures Let TF be the Total (T) number of Frames (F) in the normal version. The number of I-P-B frames can be defined as follows TF I = () P = TF ( ) (2) M B = TF ( ) (3) M is the distance between two successive I frames, defining a group of pictures (GoP). M is the distance between consecutive I or P frames. ote that M = means no B frames in the sequence. M = implies I frames only. 3 Generate the Interactive Bit-stream This approach is based on abstracting the I- frames of the original stream in order to create an alternative stream. The first step of generating the alternative stream is to abstract video data from MPEG system stream expect audio data and system stream data. For reducing the load on the network and the decoder module we only abstract I- frames. Abstracting only the I frames of the primary stream to generate the secondary stream results in less frame rate of the secondary stream. The frame rate of the secondary stream can be artificially increased by including so called P (M) marionette frames. These P (M) frames are represented by fixed pattern []. Effectively this results in repeating the previous I-frame in the decoder. The P (M) marionette frames are originated by transcoding (decode/re-encode) only the intra-coded frames of the original stream. The re-encoding process is performed using different encoding parameters ( int eractive = 5, M int eractive ) =. This enhances the visual output of mode since the P (M)-frame repeats the previous I-frame. For Fast Rewind operations we abstract the I-frames in the reverse order of the MPEG-2 compressed video. By selecting only the I-frames from the original stream, the resulting bit rate is too big. It is necessary to perform the alternatively stream in GoP structure in an independent fashion. This GoP should perform the proper bit rate and frame rate in order to confirm with the goal that there is neither additional bit rate during the mode nor an increased in the complexity of the decoder module respectively. 4 Rate Control Schemes To generate scan versions, which conform to the goal no additional bit rate during the operations the transcoded I-frames in the scan version must be rate-controlled. A common approach to control the size of an MPEG frame is to vary the quantization factor on a per-frame basis. This result in variable video quality during the Fast Forward/Rewind mode (however the quality is still constant during the normal version). The amount of quantization may be varied. This is the mechanism that provides constant quality rate control. The quantized coefficients QF( u, are computed from the DCT coefficients, the quantization scale MQUAT, and a quantization_matrix W ( u,, according to the following equation QF( u, 6 F( u, = (4) MQUAT W ( u, The normalized qunatization factor is

3 MQUAT W ( u, w ( u, = (5) 6 The quantization step makes many of the values in the coefficient matrix zero, and it makes the rest smaller. The result is a significant reduction in the number of coded bits with no visual apparent differences between the decoded output and the original source data. The quantization factor may be varied in two ways Varying the quantization scale ( MQUAT ) Varying the quantization matrix ( W ( u, ) For W ( u, = for all the u, v we have loss-less encoding. To bound the size of I transcoded -frames of the mode, the encoder uses two predefined (higher-lower) thresholds. An I-frame is re-encoded such that its size is between Threshold lower higher I transcoded Threshold (6) After an I-frame of a scan version has been reencoded as an I-frame the encoding algorithm checks whether the size of the compressed frame is between the two pre-defined thresholds. If it is not, then the quantization factor for the corresponding frame is increased/decreased and the frame is reencoded. Two different schemes can be used to initialize the quantization factor when an I-frame is to be reencoded. In the first scheme, when an I-frame is to be re-encoded for the first time the encoder starts with a fixed quantization factor. The main problem with this method is the quantization value might be kept unnecessarily high resulting in low quality during the operations. Moreover the encoder in this scheme uses one predefined higher threshold ( I transcoded Threshold ) In the second scheme, the encoder tries to track the nominal quantization value, which was used in encoding the same type of frame in the normal version. In the first encoding attempt, the encoder starts with the nominal quantization value that was used to encode the same I-frame of the normal version. After the first encoding attempt, if the resulting frame size is between the two pre-defined thresholds (6), the encoder proceeds to the next frame. Otherwise, the quantization factor {quantization matrix ( W ( u, } varies and the same frame is re-encoded. The quantization matrix can be modified by maintaining the same value at the neardc coefficients but with different slope towards the higher frequency coefficients. This procedure is repeated until the size of the compressed frame is between the two predefined thresholds [6]. The advantage of this scheme is that it tries to produce operations with the same constant quality of normal playback, but when it is not possible it minimizes the fluctuation in video quality during the FF/FR mode. On the other hand, since re-encoding is done online the encoding time may be in more important than the video quality. 5 etwork Bandwidth Allocation There are many different schemes to encode the MPEG video, depending on the desirable server/network/client complexity requirements. Since the network bandwidth usually is the highest concern, the video during the normal version is coded with all I-P-B frames in order to achieve high compression ratios for the transport over the network with minimum bandwidth resources. In addition it is desirable to support operations with minimum requirement in the network bandwidth. 5. ormal Playback Assume that the MPEG-2 file has a playback rate of 3 frames per second. However, there are no hardware or software MPEG decoders that can run faster than 3 frames per second. For our experiment we used an MPEG-2 encoder developed by the Software Simulation Group (SSG) [9]. We encoded 8 frames of motor race clip. The Group of Pictures (GoPs) format was =5 and M=3. Therefore we had a larger I or P frame, followed by two smaller B-frames. Figure shows the size of I, P and B-frames. Size (Bytes) Moto-Race trace (=5,M=3) I-frames Size P-frames Size Averg B-frames Frame umber Figure : Frames sizes for motor race clip According to Figure the bandwidth during

4 the normal playback is given by 3fps Average(IPB)Size 8bits =. 98MBps byte I Average(IPB)Size= 5.2 Interactive mode + P x ( ) + B M x ( ) M The total number of I-P(M) frames in the mode can be computed as follows 3fps Average(IP) Average(IP) interactiv e I Size = Size 8bits =.53MBps byte + P 2 nd Scheme { w ( u, = variable} 9 I Itranscoded I The bandwidth is given by x ( M i ) i TF = I Interactive I, P( M ) ineractive (7) 3fps Constant (IP) Size 8bits =.23MBps byte I is the number of I-frames (). (, M ) are the new reencoding parameters. According to () and (7) for TF=8, (=5,M=3), = 5, M ) we get ( interactiv e = Interactive TF I, P( M ) = 6 The bit rate allocation during the mode depends on the proposed encoding scheme. Figure 2 depicts the bit rate allocation for the proposed schemes. 6 Supported Speedup It is useful to derive a closed-form formula to show the number of the supported speedups of the proposed method. The speedups can be computed as follows RecordingRatioIneractiveMode Speedup = int eractive RecordingRatioormalPlay (8) Size (bytes) st Scheme Motor Race trace Frame umber st-scheme 2nd-Scheme Figure 2: Frame sizes of bitstream { w ( u, = fixed } higher I Threshold = I transcoded The bandwidth is given by To minimize the complexity at the decoder module the frame rate during the mode is the same as the one during the normal playback. If the decoder consumes data at higher or lower rate than the one specified it would result in slight hiccups at the client end. This phenomenon will occur in any system where the server s production rate differs from the consumption rate of the decoder. Either the decoder will eventually starve or overrun its buffers. It is certainly conceivable that different end users have different hardware decoder cards or even software decoders, each with different consumption rates Thus from (8) we get Speedup = (9) int eractive Figure 3 depicts the number of supported speedups

5 for various GoPs Lengths of normal play (). SpeedUps(sec) SpeedUps =5 = Figure 3: Relative decrease in the speedup as a function of int eractive PSR value for the 6- frames is 43.3 db for the first scheme and 47.6 db for the second scheme, i.e., the quality is better under the second scheme. The absolute values of PSR do not convey the advantage of the second re-encoding scheme. For this purpose, we compute the PSR values for the 6 frames when re-encoding is done without any constraints and uses these values as a reference. For each frame, we compute the difference between its reference PSR value and the PSR value resulting from each of the two-rate control schemes. These differences are plotted in figure 5 of a segment of the mode. In the same figure a large values indicates a large deviation from the reference PSR, and thus lower quality. Clearly the second scheme achieves better visual quality than the first one, but at expense of more re-encoding attempt. 7 Analyze the Visual Quality There are two types of criteria that can be used for the evaluation of video quality; subjective and objective. It is difficult to do subjective rating because it is not mathematically repeatable. For this reason we measure the visual quality of the mode using the Peak Signal-to-oise ratio (PSR). The two approaches can be constrained with respect to video quality using the (PSR). We use the PSR of the Y-component of a decoded frame. The PSR is obtained by comparing the original raw frame with its decoded version with encoding being done using the proposed schemes. PSR (db) Motor Race Trace Frame Index st-scheme 2nd-Scheme Figure 4: PSR for encoded frames in the mode Figure 4 depicts the PSR values for motor race movie. Both schemes achieve acceptable visual quality because the PSR is sufficiently large. The PSR difference (db) Motor Race trace Frame Index st-scheme 2nd-Scheme Figure 5: Difference in PSR between constrained and unconstrained encoding. 8 Conclusions In this paper, we presented an approach for supporting scanning operations such as Fast Forward and Fast Rewind in a Video On Demand System. Interactive operations are supported by transcoding the intra-coded frames of the original MPEG-2 compressed video in order to originate the bit-stream. During the transcoding process the stream can be performed in a Group of Pictures (GoPs) structure in an independent fashion. The frame rate of the secondary stream can be artificially increased by including so called P (M) marionette frames. Effectively this results in repeating the previous I- frame in the decoder. Simulation results show that the visual quality of the secondary bit stream is in

6 acceptable quality. On the other hand, there is a trade off between the real time transcoding and the visual quality during the mode. For network applications, real-time transcoding is a must. So the computation is a major concern. Besides the constraint of real-time operation, retaining the video quality is another important issue. In the second encoding approach the visual quality during the fast scanning mode varies smoothly around the nominal quality at the expense of an increase in the number of encoding attempts. Future research will focus on applications of the proposed concept to MPEG-4 stream. One other point for open research is to support other functions such as slow down, slow reverse, jump forward and jump backward. San Francisco, CA, pp [7] M-S.Chen, D.D. Kandlur. Downloading and Stream Conversion: Supporting Interactive Playout of Video in a Client Station. In: Proc IEEE Multimedia Conference (995). [8] Kostas Psannis, Marios Hadjinicolaou Transmitting Additional Data of MPEG-2 Compressed Video to Support Interactive Operations. In: Proc. IEEE ISIMP, Hong Kong, May 2 pp 38-3 [9] MPEG Software Simulation Group, MPEG-2 Encoder/ Decoder Version.2 ftp://ftp.mpeg.org/pub/mpeg/mssg/ ACKOLEGDMET The authors wish to thank all who reviewed this paper, especially Dr A Krikelis the Chief Technology Officer of Aspex Technology for his helpful comments. References: [] International Standard Information Technology- Genetic Coding of Moving Pictures And Associated Audio: Video [2] Banu Ozden, Alexandros Biliris, Rajeen Rastogi, Avi Silberschatz. A Low-Cost storage Server for Movie on Demand Databases. In: Proc. of the 2 th VLD Conference, Santiago, Chile, 994.pp [3] Michael Vernick, Chitra Venkatramani, Tzi-cher Chinueh. Adventures in Building the Stony Brook Video Server. In: Proc. ACM Multimedia, ovember 996 Boston, MA pp [4] Marwan Krunz, George Apostolopoulos. Efficient Support for scanning operations in MPEG-based video on video on demand. Multimedia Systems, vol.8, no., Jan. 2, pp.2-36 [5] Prashant J.Shenoy, Harrick M.Vin. Efficient Support For Scan Operations In Video Servers. In: Proc. ACM Multimedia Conference, ovember 995. ACM Press, San Francisco, CA, pp 3-4 [6] Jayanta K Dey-Sircar, James D.Salehi, James F Kurose, Don Towsley. Providing VCR Capabilities in Large-Scale Video Servers In: Proc. ACM Multimedia Conference, Oct 994,