The impact of GOP patter ad packet loss o the video quality of H.264/AVC compressio stadard MIROSLAV UHRINA, JAROSLAV FRNDA, LUKÁŠ ŠEVČÍK, MARTIN VACULÍK Departmet of Telecommuicatios ad Multimedia Uiversity of Zilia Uiverzita 1, 010 26 Zilia SLOVAKIA miroslav.uhria@fel.uiza.sk, jaroslav.frda@vsb.cz, lukas.sevcik@vsb.cz, marti.vaculik@fel.uiza.sk Abstract: - This article focuses o the relatioship betwee the GOP patter of the H.264/AVC compressio stadard ad the packet loss ad its impact o the video quality. This testig is the secod part of the research of a global video quality factor which will rate the impact of the compressio ad the trasmissio lik o the video quality. I the first part of this paper a short characteristic of the H.264/AVC compressio stadard is writte. The secod part deals with the objective video quality assessmet. I the last part the measuremets ad experimetal results are described. Key-Words: - H264/AVC, GOP, packet loss, objective assessmet, SSIM 1 Itroductio I the last years the multimedia domai has rapidly icreased, which meas the broadcastig, trasmissio ad receivig the video, audio ad other data i oe stream the multimedia stream. Because of the progress, the measurig of the video quality as oe part of the multimedia techology has become very importat role. The video quality is affected by: the resolutio of the scaig part of the camera, the processig of the televisio sigal i the studio, the compressio techology, the trasmissio lik imperfectio. The compressio techology ad the trasmissio lik imperfectio are two mai factors that ifluece the video quality 2 H.264/AVC MPEG-4 Part 10 (H.264/AVC) is oe of the most used compressio stadards i recet years. It is desiged for a wide rage of applicatios, ragig from video for mobile phoes through web applicatios to TV broadcastig (HDTV). The key advatages of this stadard are: up to 50% bit rate savig, high quality video, error resiliece, etwork friedliess. Some of the feature ehacemets i this compressio stadard over the earlier codecs are: DCT algorithm works at 4x4 pixels istead of 8x8 but also supports 8x8, DCT is layered usig Hadamard trasforms, colour samplig supported at 4:2:2 ad 4:4:4., up to 12 bits per pixel are possible, motio compesatio blocks are variable sizes, arithmetic variable-legth codig, built-i de-blockig filter ad hitig mechaism, rate-distortio optimizer, weighted bi-directioal predictio, redudat pictures, ISBN: 978-960-474-379-7 150
flexible macroblock orderig, direct mode for B-frames, multiple referece frames, sub-pixel motio compesatio. H.264/AVC defies a set of three profiles (Baselie, Mai, Exteded), each supportig a particular set of codig fuctios ad each specifyig what is required of a ecoder or decoder that complies with the profile. The Baselie profile supports itra ad iter-codig (usig I-slices ad P- slices) ad etropy codig with cotext-adaptive variable-legth codes (CAVLC). The Mai profile icludes support for iterlaced video, iter-codig usig B-slices, iter codig usig weighted predictio ad etropy codig usig cotext-based arithmetic codig (CABAC). The Exteded profile does ot support iterlaced video or CABAC but adds modes to eable efficiet switchig betwee coded bitstreams (SP- ad SI-slices) ad improved error resiliece (Data Partitioig). H.264/AVC also defies levels for the coded bitsream, a set of costraits imposed o values of the sytax elemets i H.264/AVC bistream [1] - [5]. 3 Group of pictures Very importat factor that also iflueces the video quality is the frame type. There are three defied types of frames: o I (itra) frames are coded without referece to other frames (without ay motio-compesated predictio), i a very similar maer to JPEG, which meas that they cotai all the iformatio ecessary for their recostructio by the decoder; for this reaso, they are the essetial etry poit for access to a video sequece. A I frame is used as a referece for further predicted frames (P ad B). The compressio rate of I frames is relatively low. o P (predicted) frames are iter-coded usig motio-compesated predictio from a referece frame (the P frame or I frame precedig the curret P frame). Hece a P frame is predicted usig forward predictio ad a P frame may itself be used as a referece for further predicted frames (P ad B frames). The compressio rate of P frames is sigificatly higher tha for I frames. o B frames are iter-coded usig motiocompesated predictio from two referece frames, the P ad/or I frames before ad after the curret B frame. Two motio vectors are geerated for each macroblock i a B frame: oe poitig to a matchig area i the previous referece picture (a forward vector) ad oe poitig to a matchig area i the future referece picture (a backward vector). A motio-compesated predictio macroblock ca be formed i three ways: forward predictio usig the forward vector, backwards predictio usig the backward vector or bidirectioal predictio (where the predictio referece is formed by averagig the forward ad backward predictio refereces). Typically, a ecoder chooses the predictio mode (forward, backward or bidirectioal) that gives the lowest eergy i the differece macroblock. B frames offer the highest compressio rate. All these differet frame types are the combied i a specific repeatig order the Group of Pictures (GOP). The first frame of the GOP is always the I frame. A typical GOP patter is IBBPBBPBBPBBI, where each letter represets viewig order ad type of the frame. The size (or iterval betwee each repeatig I frame) of the typical GOP is 12. The GOP patter ca be described by two letters - N ad M. N represets distace betwee each I frame, i.e. GOP legth. M idicates distace betwee each P frame [6], [7]. 4 Objective quality assessmet The video quality evaluatio ca be divided ito objective ad subjective assessmet. The subjective assessmet cosists of the use of huma observers who score the video quality. It is the most reliable way how to determie the video quality ad ca be regarded as the best oe but it is time cosumig method ad huma resources are eeded. Because of this fact, the objective methods are mostly used. They cosist of the use of computatioal methods called metrics which produce values that score the video quality. They measure the physical characteristics of a video sigal such as the sigal amplitude, timig, sigal to oise ratio. They are ISBN: 978-960-474-379-7 151
repeatable. The well-kow ad mostly used objective metrics are Peak Sigal-to-Noise Ratio (PSNR), Video Quality Metric (VQM) ad Structural Similarity Idex (SSIM). 4.1 PSNR The PSNR i decibels is defied as: m 2 PSNR = 10log [db] (1) MSE where m is the maximum value that pixel ca take (e.g. 255 for 8-bit image) ad MSE (Mea Squared Error) is the mea of the squared differeces betwee the gray-level values of pixels i two pictures or sequeces I ad I ~ : 1 MSE = TXY t x y ~ [ I( t, x, y) I ( t, x, y) ] for pictures of size X x Y ad T frames. 2 (2) Techically, MSE measures image differece, whereas PSNR measures image fidelity. The biggest advatage of the PSNR metric is the easy ad fast computig [1]. 4.2 SSIM The SSIM metric measures three compoets the lumiace similarity, the cotrast similarity ad the structural similarity ad combies them ito oe fial value, which determies the quality of the test sequece (figure 1). This method differs from the methods described before, from which all are error based, by usig the structural distortio measuremet istead of the error oe. It is due to the huma visio system that is highly specialized i extractig structural iformatio from the viewig field ad it is ot specialized i extractig the errors. Owig to this factor, SSIM metric achieves good correlatio to subjective impressio [8]. sigal x sigal y lumiace measuremet lumiace measuremet - + - + cotrast measuremet cotrast measuremet Fig.1 The block diagram of SSIM metric. The results are i iterval [0,1], where 0 is for the worst ad 1 for the best quality. 4.3 VQM The VQM metric computes the visibility of artefacts expressed i the DCT domai. Figure 2 shows the block diagram of this metric, which ca be divided ito 9 steps. referece sequece test error metric + + crop poolig Fig.2 The block diagram of VQM metric. colour trasform cotrast maskig lumiace compariso cotrast compariso structure compariso combiatio The iput of the metric is a pair of colour image sequeces the referece oe ad the test oe. Both sequeces are cropped, the coverted from the iput colour space to the YOZ colour space, the trasformed to blocked DCT ad afterwards coverted to uits of local cotrast. I the ext step the iput sequeces are subjected to temporal filterig, which implemets the temporal part of the cotrast sesitivity fuctio. The DCT coefficiets, expressed i a local cotrast form, are the coverted to just-oticeable-differeces (jds) by dividig by their respective spatial thresholds. This implemets the spatial part of the cotrast sesitivity fuctio. I the ext step, after the coversio to jds, the two sequeces are subtracted to produce a differece sequece. I the followig step the cotrast maskig operatio to the differece sequece is performed. Fially the masked differeces are weighted ad pooled over all dimesios to yield summary measures of visual error [9]. The output value of the VQM metric idicates the amout of distortio of the sequece for o impairmet the value equal to zero ad for risig level of impairmet the output value rises, too. DCT _ local cotrast time filter SCSF ISBN: 978-960-474-379-7 152
5 Measuremets 5.1 Source videos I our experimets two test sequeces were used oe with the dyamic scee (the Football sequece figure 3) ad oe with the slow motio (called the Trai sequece figure 4). Fig.3 The Football sequece. The GOP legth was set to 12 ad two B frames stored (N=12, M=3) called GOP 12 BF 2. The GOP legth was set to 24 ad two B frames stored (N=24, M=3) called GOP 24 BF 2. The GOP legth was set to 48 ad two B frames stored (N=48, M=3) called GOP 48 BF 2. The GOP legth was set to 12 ad six B frames stored (N=12, M=7) called GOP 12 BF 6. The GOP legth was set to 12 ad two B frames stored (N=12, M=11) called GOP 12 BF 10. Each frame of the sequece was ecoded as I frame called Oly I. The parameters of the ecoded sequeces were set to Mai Profile, Level 3. Fig.4 The Trai sequece. Both sequeces were i the resolutio 720x576 with 25fps (frames per secod). The legth of these sequeces were 220 frames, i.e. 8,8 secods. They were dowloaded from [10] i the ucompressed format (*.yuv) ad used as the referece sequeces. 5.2 Ecodig Afterwards these sequeces were ecoded to the H.264/AVC compressio stadard with differet GOP legths ad umber of B frames. It was doe usig x264 tool [11]. The bitrate of all ecoded sequeces was set to 3 Mbps. May types of GOP patters were used: The GOP legth was set to 12 ad two B frames stored (N=6, M=3) called GOP 6 BF 2. 5.3 Trasmissio over a etwork Subsequetly the test sequeces from oe to aother PC over a etwork were trasmitted. For trasmissio the Evalvid tool versio 2.7 was used [12]. Evalvid represets a complete framework ad tool-set for evaluatio of the quality of a video trasmitted over a real or simulated commuicatio etwork. Besides measurig QoS parameters of the uderlyig etwork, like loss rates, delays, ad jitter, it also supports a subjective video quality evaluatio of the received video based o the frameby-frame PSNR calculatio. The tool-set has a modular costructio, makig it possible to exchage both the etwork ad the codec. I the figure 5 the structure of the Evalvid framework is show. ISBN: 978-960-474-379-7 153
Fig.5 The structure of the Evalvid framework. The NetDisturb tool as a etwork emulator that geerated packet loss was used. NetDisturb is a IP etwork emulator software that ca geerate impairmets over IP etworks (IPv4 & IPv6) such as: latecy, delay, jitter, badwidth limitatio, loss, duplicatio ad modificatio of the packets. packets) ad more. NetDisturb allows disturbig flows over IP etworks helpig to study the behavior of applicatios, devices or services i a "disturbed" etwork eviromet. NetDisturb is iserted betwee two Etheret segmets actig as a bridge ad operates bi-directioal packet trasfer o Etheret, Fast Etheret ad Gigabit etwork iterface cards. I our experimets the packet loss was geerated i the rage from 0,2% to 3% with the step 0,2%. The trasmissio over etwork cosisted of these steps: 1. Before the trasmissio it was ecessary to hit the sequece. It was doe usig the MP4Box tool versio 0.4.6 [13]. This hit track told the server how to packet the data for the etwork (e.g. the MTU size). 2. The the video trace file by streamig a hited video over a etwork was geerated. It was doe by the mp4trace tool i the Evalvid toolset. This examied every frame of the video ad produced a log file cotaiig statistics such a frame umber, frame type, size i kilobytes, packets required to trasmit ad trasmissio time i relatio to the overall duratio of the video sequece. 3. Simultaeously the seder (o the streamig PC) ad receiver (o the receiver PC) dump files were geerated usig WiDump tool versio 3.9.5 [14]. These files cotaied packet data such as a trasmissio time, uique packet id ad packet size. 4. Fially by mergig all files (the seder dump, receiver dump, video trace files ad origially ecoded hit file) the corrupted video file that reflected the packet loss of the etwork was created. This was doe usig the etmp4 tool from the Evalvid toolset. Lost frames were filled with 0, so the umber of set ad received frames was the same. The etmp4 tool also geerated a loss text file, where the percetages of all types of loss frames were give. 5.4 Decodig Subsequetly the trasmitted sequeces were decoded back to *.yuv format usig the same tool x264 [11]. 5.5 Assessmet Fially these trasmitted sequeces were compared to ucompressed sequeces. For the video quality evaluatio the MSU Measurig Tool versio 2.7.3 was used [15]. The sequeces (files) were assessed usig SSIM metric. The whole process of the video quality measurig of both sequeces is show i the figure 6. Fig.6 The process of measurig the video quality For all packet loss rates ad GOP patters as well as both sequeces the measuremets were doe te times. For the results the mea values usig formula 1 [16] were calculated ad plotted to the graphs. ISBN: 978-960-474-379-7 154
1 x = x i i= 1 (1) The figures from 7 ad 8 show the measuremets results of the relatioship betwee video quality (SSIM) ad packet loss rate of MPEG-4 H.264/AVC compressio stadard for various GOP patters for "Football" ad "Trai test sequeces. The figure 7 shows the measuremets results of the "Football" test sequece ad the figure 8 shows the measuremets results of the "Trai" test sequece. more frames are lost tha i sequeces with shorter GOP - by oe I frame loss the whole GOP is lost, i.e. by oe I frame lost i "GOP12-BF2" patter sequece 12 frames are lost whereas i "GOP48- BF2" patter 48 frames are lost. This results i bigger picture degradatio. Cotet impact of the sequece o the video quality From previous measuremets the graphs that show the cotet impact of sequece could be doe. Fig. 7. The relatioship betwee video quality (SSIM) ad packet loss rate of MPEG-4 H.264/AVC compressio stadard for various GOP patters for "Football" test sequece. Fig. 9. The compariso of relatioships betwee video quality (SSIM) ad packet loss rate of MPEG-4 H.264/AVC compressio stadard with "GOP12-BF2" patter for both test sequeces. Fig. 10. The compariso of relatioships betwee video quality (SSIM) ad packet loss rate of MPEG-4 H.264/AVC compressio stadard with "Oly I" patter for both test sequeces. Fig. 8. The relatioship betwee video quality (SSIM) ad packet loss rate of MPEG-4 H.264/AVC compressio stadard for various GOP patters for "Trai" test sequece. Accordig to the graphs, the video quality is iflueced by the GOP legth - the packet loss has bigger effect o video quality i sequeces with loger GOP loger GOP, lower video quality. It is due to the fact, that i sequeces with loger GOP Accordig to the graphs, the cotet of sequece has impact o the video quality - the packet loss degrades video quality i slow sequeces more tha i sequeces with dyamic scee. It is due to the fact that P ad B frames (which cotai oly motio-compesated differece iformatio) i slow sequeces carry less iformatio tha i sequeces with dyamic scee. After bitrate restrictio (e.g. 3 Mbps) it results i fact that I ISBN: 978-960-474-379-7 155
frames (which cotai all ecessary iformatio) i slow sequeces carry more iformatio tha i sequeces with dyamic scee, i.e. are trasmitted i more packets (figure 11 ad 12). That leads to higher probability of I frame lost which results i degradatio ad worse video quality. This fact cofirms figures 13 ad 14 that show the relatioship betwee type I frame loss rate ad packet loss rate of differet bitrates i both tested sequeces. Fig. 14. The relatioship betwee the type I frame loss rate ad packet loss rate of MPEG-4 H.264/AVC compressio stadard with bitrate 3Mbps for various GOP patters for "Trai" test sequece. Fig. 11. The frame order of MPEG-4 H.264/AVC compressio stadard with bitrate 3Mbps with "GOP12-BF2"patter for "Trai" test sequece. Colum height represets the frame size, the colour represets the frame type (red - I frame, blue - P frame, gree - B frame). Fig. 11. The frame order of MPEG-4 H.264/AVC compressio stadard with bitrate 3Mbps with "GOP12-BF2"patter for "Football" test sequece. Colum height represets the frame size, the colour represets the frame type (red - I frame, blue - P frame, gree - B frame). This fact ca be also expressed mathematically by calculatig the probability of type I frame loss for various sequeces. The computatio is doe usig the formula: [( p) ] P = 1 1, (2) where p expresses the probability of packet loss (etered as a parameter for NetDisturb tool) expresses the average umber of packets eeded to carry a type I frame. The example of computig the probability of I frame loss for "GOP12-BF2" patter with 3Mbps bitrate for both test sequece by probability of packet loss 0,012 is: P Football = 1 p P Trai = 1 p [( 1 ) ] = 1 ( 1 0,012) [( 1 ) ] = 1 ( 1 0,012) [ ] 28 = 0, 26683 [ ] 73 = 0, 58576 6 Coclusio Fig. 13. The relatioship betwee the type I frame loss rate ad packet loss rate of MPEG-4 H.264/AVC compressio stadard with bitrate 3Mbps for various GOP patters for "Football" test sequece. I this article the relatioship betwee the GOP patter of the H.264/AVC compressio stadard ad the packet loss ad its impact o the video quality was tested. First the impact of GOP legth o the video was examied, secod the cotet impact of the sequece o the video quality was tested. This testig is the secod part of the research of a global video quality factor which will rate ISBN: 978-960-474-379-7 156
the impact of the compressio ad the trasmissio lik o the video quality. Ackowledgmet Cetre of excellece for systems ad services of itelliget trasport II., ITMS 26220120050 supported by the Research & Developmet Operatioal Programme fuded by the ERDF. "Podporujeme výskumé aktivity a Slovesku/Projekt je spolufiacovaý zo zdrojov EÚ" Refereces: [1] S. Wikler, Digital Video Quality: Visio Models ad Metrics, Joh Wiley ad Sos Ltd., 2005, 175 pages, ISBN 0-470-02404-6. [2] C. Wootto, A Practical Guide to Video ad Audio Compressio, Elsevier Ic., 2005, 787 pages, ISBN 0-240-80630-1. [3] E.G. Richardso, The H.264 Advaced Video Compressio Stadard, 2d ed., Joh Wiley ad Sos Ltd., 2003, 316 pages, ISBN 978-0-470-51692-8. [4] E.G. Richardso, H.264 ad MPEG-4 Video Compressio, Joh Wiley ad Sos Ltd., 2003, 281 pages, ISBN 0-470-84837-5. [5] Scietific Atlata, MPEG-4 Part 10 AVC (H.264) Video Ecodig, Scietific-Atlata, Jue 2005, 19 pages, Part Number 7007887 Rev B. [6] H. Beoit, Digital Televisio, 3rd ed., Focal Press, 2008, 289 pages, ISBN 978-0-240-52081-0. [7] E.G. Richardso, Video Codig Desig, Joh Wiley ad Sos Ltd., 2002, 299 pages, ISBN 0-470-84783-2. [8] H.R. Wu, K.R. Rao, Digital Video Image Quality ad Perceptual Codig, Taylor ad Fracis Group LLC, 2006, 594 pages, ISBN 0-8247-2777-0. [9] H.M. Loke, P.E. Og, W. Li, Z. Lu, S. Yao, Compariso of video quality metrics o multimedia videos. Image Processig IEEE, 2006, p. 457-460, ISSN 1522-4880. [10] Video quality test sequeces. Available: http://media.xiph.org/vqeg/testseqeces/th umbnails/. [11] Ecoder x264. Available: http://www.videola.org/developers/x264.ht ml. [12] Evalvid Framework Tool. Available: http://www.tk.tuberli.de/research/evalvid/. [13] MP4Box tool. Available: http://www.videohelp.com/tools/mp4box. [14] WiDump tool. Available: http://www.wipcap.org/widump/. [15] MSU Measuremet tool. Available: http://compressio.ru/video/quality_measur e/video_measuremet_tool_e.html. [16] Arithmetic mea: http://mathworld.wolfram.com/arithmeticm ea.html ISBN: 978-960-474-379-7 157