PACKET-HEADER-BASED-QUALITY-ESTIMATION MODEL FOR MOBILE AUDIOVISUAL MEDIA STREAMING RECOMMENDATION ITU-T P

Transcription

1 Proceedings of Seventh International Workshop on Video Processing and Quality Metrics for Consumer Electronics January 30-February 1, 2013, Scottsdale, Arizona PACKET-HEADER-BASED-QUALITY-ESTIMATION MODEL FOR MOBILE AUDIOVISUAL MEDIA STREAMING RECOMMENDATION ITU-T P Kazuhisa Yamagishi NTT Network Technology Laboratories, NTT Corporation, Tokyo , Japan Gao Shan Media Technology Laboratory, Huawei Technologies Co., Ltd., Shenzhen, P.R.China ABSTRACT The International Telecommunication Union Telecommunication Standardization Sector ITU-T) Study Group 12 standardized a parametric non-intrusive sessment of audiovisual media streaming quality lower resolution application area Recommendation ITU-T P The P model estimates audio, video, and audiovisual quality for mobile audiovisual media streaming using packet headers. The computational power of the model is very light because it is a no-reference i.e., non-intrusive) model that operates by analyzing packet header information. Therefore, the model can be applicable to in-service quality monitoring by incorporating it into a client terminal e.g., mobile phone). 1. INTRODUCTION Internet protocol television IPTV) is widely used. Since the quality of IPTV is generally affected by a processing chain composed of audio and video compression, a transmission scheme i.e., user datagram protocol UDP) or transmission control protocol TCP)) and transmission behavior i.e., packet loss or delay), and client behavior i.e., packet loss concealment PLC) or buffering), quality planning and monitoring are important in terms of providing a high-quality IPTV service. Quality planning can be carried out by using the relationship between subjective quality and quality parameters such the bit rate. For example, Recommendation ITU-T G.107 E-model) [1], which is formulated by relationship subjective quality and quality parameters is used a planning tool of speech service and Recommendation ITU-T G.1070 [2] is used a planning tool of videophone service. It is often the ce that real-time monitoring is necessary. Therefore, developing a quality estimation model is desirable. The entire content delivery chain can be divided into multiple domains: content provider, service provider, network provider, and end-user [3]. Therefore, it is ideal that quality monitoring be operated at the head-end in a network, and at a receiver such a mobile phone and set-top box STB). Quality monitoring at the head-end is important because quality degradations at this point affects the quality of all end-users. Unforeseen and problematic quality degradations due to coding artifacts need to be detected by head-end monitoring. Since an uncompressed reference source can be used in the head-end, full-reference media-layer models are suitable. For example, monitoring can be carried out using Recommendation ITU-T J.341, J.247, or J.144, which can be used for estimating video quality bed on the uncompressed reference video and coded video signals. In the network and receiver, computational power is an issue for incorporating a model into a probe or receiver. Since there are many streams in a network, a technology that takes media signals input is not realistic in terms of computational power. In addition, a technology in the receiver that requires complexity is not suitable because the receiver is used for receiving the stream then the stream is decoded. Therefore, a technology that does not require much computational power is desirable. Another issue is that uncompressed reference source cannot be used in the network and receiver. Therefore, no-reference model is necessary. The International Telecommunication Union Telecommunication Standardization Sector Study Group 12 ITU-T SG12) had studied the parametric non-intrusive sessment of audiovisual media streaming quality provisional code: P.NAMS) [4] lower resolution LR) [5] and higher resolution HR) [6] application are that take packet headers input. The P.NAMS-LR P ) model can be applied to quarter common intermediate format QCIF, pixels), quarter video graphics array QVGA, pixels), or half VGA HVGA, pixels) video service and can estimate quality due to coding, packet loss, and/or rebuffering artifacts. In contrt, the P.NAMS-HR P ) model can be applied to standard definition SD, pixels) and high definition HD, or pixels) video service and can estimate qual- 52 VPQM2013

2 ity due to coding artifacts and/or packet loss. However, the model cannot be used for estimating the quality affected by rebuffering. Finally, these two new ITU-T Recommendations have been consented at the ITU-T SG12, Working Party 2 meeting in September, The P model w standardized by ITU-T bed on individual proposals from NTT Corporation and Huawei Technologies Co., Ltd. We introduce the P i.e., P.NAMS-LR) model, which can be used for estimating the quality of mobile audiovisual media streaming service. The remainder of this paper is structured follows. The P model is explained in Section 2. The validation results of the model are explained in Section 3. Finally, we summarize and suggest possible directions for future studies in Section P MODEL The P model can be used for estimating the quality of UDP-bed lower resolution video streaming i.e., QCIF, QVGA, and HVGA), shown in Table 1. Quality degradations the model can evaluate are follows: 1) compression artifacts are introduced due to the encoding process, 2) packet loss artifacts are introduced due to packet loss, but artifacts depend on a PLC scheme, 2-1) slicing artifacts are introduced when packet losses are concealed using the PLC scheme of the receiver trying to repair erroneous video slices or frames, 2-2) freezing artifacts are introduced when the PLC scheme of the receiver replaces the erroneous frames either due to packet loss or error propagation) with the previous error free frame until a decoded picture without errors h been received this type of artifact is also called freezing with skipping), and 3) rebuffering artifacts coming from rebuffering events at the receiver, which could be a result of a stream arriving late this type of artifact is also called freezing without skipping). Note that audio quality estimation model cannot estimate the quality affected by rebuffering. The block diagram of the P model is shown in Fig. 1. The model takes packet headers and side information e.g., encoder and decoder codec), PLC, and rebuffering behavior) input. The streams may be encrypted or unencrypted. The parameter-extraction modules P-E) extract audio, video, and rebuffering-related parameters and output them to parameter calculation modules. Parameter calculation modules derive parameters that are used by quality estimation modules. Finally, quality estimation modules output audio, video, and audiovisual mean opinion scores MOSs). The model details are described in the following sections. Table 1. Application are, test factors, and coding technologies [4] P Lower Resolution LR) Applications the models are intended for In-service monitoring of audiovisual, video, and audio UDP-bed streaming Performance and quality sessment of live networks including codecs) including effect due to encoding bitrate and transmission errors Test factors the models have been validated for Encoding compression) degradation of audio and video with variety of bitrates Video: kbps Audio: kbps Packet loss degradation of audio and video both random and bursty packet loss patterns) Rebuffering degradation audio-only re-buffering not validated) Video content of different spatio-temporal complexity Different video keyframe and frame-rates Frame rates: 5 30 fps GOP lengths 1 / keyframe rate): 2 10 sec Different video resolutions: HVGA, QVGA, QCIF Different decoder-side packet loss concealment strategies freezing with skipping, one slice per RTP packet/frame) Coding technologies models have been trained on Video: MPEG4 Part 2, H.264 MPEG4 Part 10) Audio: AMR-NB/WB+, AAC-LC, HE-AACv1/v Audio quality estimation module In the estimation of audio quality, coding distortion and packet-loss degradation must be taken into account. In the estimation of coding distortion, the bit rate is a key factor. In the estimation of packet-loss degradation, lost audio frame length is a key factor. Therefore, the bit rate and lost audio frame length are used in the estimation of audio quality. The parameter extraction module for audio P-E-A) extracts the RTP timestamp, sequence number, and payload. Bed on the audio RTP timestamp and clock rate, P-E-A calculates the meurement time for audio A MT ). Bed on the audio RTP sequence number, P-E-A extracts the packetloss length per packet-loss event A P LLk) and the audio packet-loss-event frequency A P LEF ), shown in Fig. 2. Bed on the audio RTP payload, P-E-A counts the number of received audio RTP packets and the audio payload per RTP packet A receivedbytes i ). To compensate for the lost audio payload due to packet loss, P-E-A estimates the audio payload per RTP packet A lostbytes j ) for the lost packet in bytes using the average of the current and previous received audio RTP payloads i.e., A receivedbytes i and A receivedbytes i 1 ). 53

3 Packet header information P-E-V Parameter calculation module for video Video quality estimation module VideoMOS RTP header UDP header P-E-R Audiovisual quality estimation module Audiovisual MOS IP header P-E-A Parameter calculation module for audio Audio quality estimation module Audio MOS Side information about codec, client behavior etc. Available to all modules Key: P-E-V Defined Interface Parameter extraction module for video P-E-R P-E-A Parameter extraction module for rebuffering Parameter extraction module for audio Fig. 1. Block diagram of P model [5] To take into account the ce in which one audio RTP packet contains several audio frames, the parameter calculation module for audio P-C-A) estimates the lost audio frame length per audio RTP packet A LF LpP ) using audio RTP timestamp and clock rate. At the same time, P-C-A calculates the number of audio packets per RTP timestamp A NP pt S). Then, the lost audio frame length A LF L) in milliseconds is calculated using A P LEF, A LF LpP, average audio burst packet loss length A ABP LL), A NP pt S, and audio frame length audiof ramelength) follows; A LF L = A P LEF maxaudiof ramelength, K), 1) A ABP LL + A NP pt S 1 K = A LF LpP. A NP pt S 2) The audio bit rate A BR) in kbps is calculated per RTP packet audio payload A Bytes i ) and the estimated amount of lost audio data A lostbytes j ) A BR = A MT 3) A RP J A Bytes i + A lostbytes j ), i=1 j=1 where A RP represents the total received number of RTP audio packets and J represents the total number of lost RTP audio packets. Finally, the audio quality estimation module estimates audio quality A MOS) follows: A MOS = 1 + A MOSC 1) MA, 4) A MOSC = a1 1 + a1 ), 1 + A BR/a2) a3 5) MA = 10 A LF L 1 a4) exp a5 A MT ) +a4 exp 10 A LF L ), 6) a6 A MT where A MOSC is audio quality due to compression, and the coefficient values a1 to a6) can be found in Ref. [5] Video quality estimation module In the estimation of video quality, coding distortions, packetloss degradations, and rebuffering degradations must be taken into account. In the estimation of coding distortion, the bit rate and frame rate are key factors. Taking into account video content complexity is also important. This characteristic is taken into account by the average number of bytes per I-frame. In the estimation of packet-loss degradation, lost video frames and the number of packet-loss events are key factors. In the estimation of rebuffering degradation, the number of rebuffering events, rebuffering length, and interval between rebuffering events are key factors. Therefore, these parameters are used in the estimation of video quality. The parameter extraction module for video P-E-V) extracts the video RTP timestamp, sequence number, market bit, and payload. 54

4 Ce 1: A_PLL1 = 4, A_PLEF = 1 Ce 2: A_PLL1 = 1, A_PLL2 = 1, A_PLL3 = 2, A_PLEF = 3 Ce 3: A_PLL1 = 1, A_PLL2 = 3, A_PLEF = 2 Ce 4: A_PLL1 = 2, A_PLL2 = 1, A_PLL3 = 1, A_PLEF = 3 Received packet Lost packet Fig. 2. Examples of calculating A P LLk and A P LEF The parameter calculation module for video P-C-V) calculates video packet-loss length V lostp ackets) bed on the video RTP sequence number and lost bytes for lost video RTP packets V lostbytes) using the same method that of P-E-A. The number of lost video frames V lostf rames) between two consecutive received video RTP packets i.e., current and previous video RTP packets) is calculated bed on the video RTP timestamp and clock rate, and video frame rate. The marker bit and video RTP timestamp are used to identify the video frame boundary between video RTP packets. P-C-V estimates the video frame type i.e., I- or P-frame) bed on the number of bytes per video frame. P-C-V calculates the average number of bytes per I- frame V ABIF ) bed on the number of I-frames and total bytes of all I-frames. In addition, the impairment rate due to the packet loss and spatial error propagation per video frame V IRpF ) is calculated, shown in Fig. 3. Quality parameters for compression artifacts are calculated follows. The meurement time for video, in seconds, is calculated bed on the total number of video frames and video frame rate videof ramerate). The video bit rate V BR) is calculated bed on the total bytes and the meurement time V MT ) V BR = V T V F i=1 V T BpF i, 7) V MT where V T V F represents the total number of video frames and V T BpF i represents the number of bytes per video frame. Since the spatial quality per video frame depends on the video bit rate and video frame rate, the normalized video bit rate V NBR) is calculated V NBR = V BR min30, videof ramerate). 8) Then, the video content complexity factor V CCF ) is calculated bed on the video bit rate and average number of bytes per I-frame V CF F = min V BR, 1.0). 9) V ABIF Quality parameters for the packet loss are calculated follows. The average impairment rate of the video frame V AIRF ) is the sum of the impairment rate per video frame V IRpF ) divided by the number of damaged video frames V NDF ) that includes those due to temporal error propagation. The impairment rate of video stream V IR) is the number of damaged video frames V NDF ) divided by the total number of video frames V T NF ). The counting method of the video packet-loss event frequency V P LEF ) is different for the PLC scheme. For the PLC to generate slicing artifacts, the video packet-loss event frequency V P LEF ) is equal to the number of video frames actually damaged by packet loss i.e., video frames damaged by temporal error propagation are not counted). For the PLC to generate freezing artifacts, the video packet-loss event frequency V P LEF ) is equal to the number of damaged group of pictures GoPs). 55

5 P-frame P-frame P-frame P-frame I-frame V _IRpF = 0.5 V _IRpF = 1.0 V _IRpF = 1.0 Received packet Lost packet Damaged packet due to spatial or temporal error propagation Fig. 3. Examples of calculating V IRpF Although the parameter extraction module for rebuffering P-E-R) is not a part of P-E-V and P-C-V, we explain it because the quality parameters for rebuffering is used for video quality estimation. Quality parameters for rebuffering calculated by the parameter extraction module for rebuffering P-E-R) are follows. The number of rebuffering events N RE) is counted. The average rebuffering length ARL) is the average value of the total length of the rebuffering event. The multiple rebuffering events effect factor MREEF ) is the average value of all the rebuffering intervals between two consecutive rebuffering events, and it is calculated when NRE > 1. The video quality estimation module estimates video quality V M OS) follows. Video quality due to compression V MOSC) is calculated IF videof ramerate 24 V MOSC = 5 V DC, 10) ELSE IF videof ramerate < 24 V MOSC = 5 V DC) F, 11) F = 1 + v1 V CF F v2 V CF F 1000 log videof ramerate ). Video distortion quality due to compression V DC) is calculated V DC = 4/1 + G), 12) G = V NBR. v3 V CF F + v4) v5 V CF F +v6) Video quality due to packet loss V MOSP ) is calculated V MOSP = V MOSC V DP. 13) Video distortion quality due to packet-loss V DP ) is calculated IF videop LC = SLICING V DP = H I V MOSC 1) 1 + H I, 14) H = V AIRF V IR v7 V CF F + v8 )v9, I = V P LEF v10 V CF F + v11 )v12. ElSE IF videop LC = F REEZING V DP = J K V MOSC 1) 1 + J K, 15) J = V IR v7 V CF F + v8 )v9, K = V P LEF v10 V CF F + v11 )v12. Video quality due to rebuffering V M OSR) is calculated V MOSR = V ideo Quality V DR, 16) where V ideo Quality = V MOSP when packet loss occurs, but V ideo Quality = V MOSC when packet loss does not occur. Video distortion quality due to rebuffering V DR) is calculated V DR = V ideo Quality 1) M = NRE/v13) v14, N = ARL/v15) v16, O = MREEF/v17) v18. M N O 1 + M N O, The coefficient values v1 to v18) can be found in [5] Audiovisual quality estimation module 17) Audiovisual quality is calculated bed on audio and video quality. Audiovisual quality due to compression AV M OSC) is calculated AV MOSC = av1 V MOSC + av2 A MOSC +av3 V MOSC A MOSC + av4. 18) Audiovisual quality due to packet loss AV MOSP ) is calculated AV MOSP = AV MOSC AV DP. 19) 56

6 Audiovisual distortion quality due to packet loss AV DP ) is calculated AV DF V = AV DF A = AV DF = V MOSC V ideo Quality, V MOSC 20) A MOSC Audio Quality, A MOSC 21) av5 AV DF V + av6 AV DF A 1 + av5 AV DF V + av6 AV DF A, 22) AV DP = AV MOSC 1) AV DF, 23) where V ideo Quality = V MOSP and Audio Quality = A MOS when packet loss occurs, but V ideo Quality = V MOSC and Audio Quality = A MOSC when packet loss does not occur. Audiovisual quality due to rebuffering AV M OSR) is calculated AV M OSR = Audiovisual Quality AV DR, AV DR = Audiovisual Quality 1) S, S = P Q R 1 + P Q R, P = NRE/av7) av8, Q = ARL/av9) av10, R = MREEF/av11) av12, 24) where Audiovisual Quality = AV M OSP when packet loss occurs, but Audiovisual Quality = AV M OSC when packet loss does not occur. Coefficient values av1 to av12) can be found in Ref. [5]. 3. VALIDATION RESULTS The audio, video, and audiovisual quality estimation modules were validated using a variety of test factors, shown in Table 1. The root mean square error RMSE) and Pearson s correlation PC) [7] are listed in Table 2. From these statistical values, it can be said that the P model reaches a satisfactory level in terms of practical use of the in-service quality monitoring. Table 2. Performance of P model [4] RMSE PC Samples Audiovisual samples Video samples Audio samples 4. CONCLUSION We introduced Recommendation ITU-T P , which can be applied for estimating the quality of mobile audiovisual media streaming service. The P model can evaluate audio, video, and audiovisual quality due to coding, packet loss, and rebuffering artifacts. However, the model cannot estimate audio quality due to rebuffering artifacts. The estimated quality with the model correlates with subjective quality, so it can be applied to in-service quality monitoring. Further studies are needed to extend the model. Recently, a progressive download video service h been widely used due to advances in tablet computers and smartphones. Since these devices support higher resolution such a HD video format, the model needs to be extended for higher resolution. In addition, not RTP-bed but TCP-bed video is used in the progressive download video service. Therefore, the model must extract parameters from TCP headers. 5. REFERENCES [1] Recommendation ITU-T G.107, The E-model: a computational model for use in transmission planning, December, [2] Recommendation ITU-T G.1070, Opinion model for video-telephony applications, July, [3] Recommendation ITU-T G.1081, Performance monitoring points for IPTV, October, [4] Recommendation ITU-T P.1201, Parametric nonintrusive sessment of audiovisual media streaming quality, October, [5] Recommendation ITU-T P , Parametric nonintrusive sessment of audiovisual media streaming quality lower resolution application area, October, [6] Recommendation ITU-T P , Parametric nonintrusive sessment of audiovisual media streaming quality higher resolution application area, October, [7] Recommendation ITU-T P.1401, Methods, metrics and procedures for statistical evaluation, qualification and comparison of objective quality prediction models, July,