EE5359 Spring 2014 1 EE5359 MULTIMEDIA PROCESSING Spring 2014 Project Proposal Intra Prediction Efficiency and Performance Comparison of HEVC and VP9 Under guidance of DR K R RAO DEPARTMENT OF ELECTRICAL ENGINEERING UNIVERSITY OF TEXAS AT ARLINGTON Presented by Shruti Sukumaran UT ID 1000870264 shruti.sukumaran@mavs.uta.edu 2/23/2014
EE5359 Spring 2014 2 List of Abbreviations AVC CABAC CTB CTU HEVC ISO ITU-T JCT-VC JM MC MI MPEG MSE MV NAL PSNR SSIM TU TM VCEG Advanced Video Coding Context Adaptive Binary Arithmetic Coding Coding Tree Blocks Coding Tree Units High Efficiency Video Coding International Organization for Standardization International Telecommunication Union -Telecommunication Standardization Sector Joint collaborative team on video coding Joint model software Motion Compensation Mode Info Moving Picture Experts Group Mean square error Motion Vector Network Abstraction Layer Peak signal to noise ratio Structural similarity index Transformation Units True Motion Video Coding Experts Group
EE5359 Spring 2014 3 Abstract The rapid growth and insatiable demand for consumption of video over the internet continues to stress the limits of the available bandwidth [9]. The emerging of a more efficient next generation video coding standard is of a high demand at present. To address this, ISO/IECMPEG/ITU-T VCEG developed a next generation video coding standard called HEVC (High Efficiency Video Coding) starting from current generation codec H.264/AVC [8]. Likewise Google embarked on a project in the late 2011 to develop next generation open source video codec VP9 starting from its predecessor VP8 as the baseline [8].They seem to be two main contenders for the position of the next state-ofthe-art video compression standard. The announced aim of HEVC is to achieve twice more efficient compression compared to H.264/AVC, at the same visual quality and VP9 was developed to get half the bit-rate of VP8 with royalty-free video codec [10]. Intraprediction is one of the main features which determines the compression efficiency of the entire codec. This proposal is a detailed comparison of intra-prediction modes in HEVC and VP9. Experiments will be carried out using JCT-VC HM [16] [17] and WebM VP9 [19] encoders. The analysis will focus on the intra frame coding and different image performance metrics like MSE, PSNR, bit rate, SSIM [22][23][24] [25]and hence video quality will be evaluated for high resolution videos. Overview of HEVC HEVC is a new standard for video compression that has the potential to deliver better performance than the earlier standards such as H.264/AVC. Source video, consisting of a sequence of video frames, is encoded or compressed by an HEVC video encoder (Figure1) to create a compressed video bit stream. The compressed bit stream is stored or transmitted. A video decoder decompresses the bit stream to create a sequence of decoded frames. HEVC decoder block diagram is shown in the Figure 2. HEVC has the same basic structure as previous standards such as MPEG-2 Video and H.264/AVC [26]. HEVC is a video coding standard that can enable better compression, at the cost of potentially increased processing power [2].
EE5359 Spring 2014 4 Figure 1: HEVC encoder block diagram [11] Figure 2 HEVC decoder block diagram [11]
EE5359 Spring 2014 5 HEVC intra frame coding Like H.264/AVC, HEVC uses block-based intra prediction to take advantage of spatial correlation within a picture. HEVC follows the basic idea of H.264/AVC intra prediction but makes it far more flexible. For example, H.264 could grab a 16x16 macroblock of pixels and perform nine intra-prediction directions - that allowed the pixels to be rebuilt within each block (Figure 4) HEVC can grab 64x64 superblocks and perform 35 infraprediction directions to rebuild them. The 33 luma intra prediction modes for HEVC are as shown in Figure 3. HEVC also includes a planar intra prediction mode, which is useful for predicting smooth picture regions. In planar mode, the prediction is generated from the average of two linear interpolations (horizontal and vertical). Figure 3: Available prediction directions in the unified intra prediction in HM 1.0 [11]
EE5359 Spring 2014 6 Figure 4: H.264 Intra-prediction modes [20] Furthermore, intra prediction can be implemented at different block sizes, ranging from 4X4 to 64X64 (whatever size the PU has) (Figure 5). Figure 5: Subdivision of a CTB into CBs [and transform block (TBs)].Solid lines indicate CB boundaries and dotted lines indicate TB boundaries (a) CTB with its partitioning. (b) Corresponding quad tree [3]. VP9 Intra-prediction VP9 is similar on the surface. It can also take 64x64 superblocks, but unlike H.265 these do not need to be square so it can sample 64x32 or 4x8 blocks for greater efficiency. On the flip side it only has 10 prediction modes to rebuild them. Cynics argue VP9 [19] changes H.265 just enough for it to avoid copyright infringement. Needless to say both standards require more computational power than H.264 and VP8 for all this rebuilding [8]. The encoder block diagram for VP6 is shown in Figure 6.
EE5359 Spring 2014 7 Figure 6: VP6 Encoder Block Diagram [21] Figure 7 : VP9 Intra prediction [13]
EE5359 Spring 2014 8 At block-size 4x4, VP9 supports ten intra prediction modes; DC, Vertical, Horizontal, TM (True Motion), Horizontal Up, Left Diagonal, Vertical Right, Vertical Left, Right Diagonal, and Horizontal Down(the same set defined by VP8). For blocks from 8x8 to 64x64 there is also support for ten intra modes; DC, Vertical, Horizontal, TM, and six angular predictors corresponding, approximately, to angles of 27, 45, 63, 117, 135, and 153 degrees. Furthermore, there is additionally the option of applying a low-pass filter to the prediction that can be signaled in the bitstream [15]. Figure 8: VP9 Superblock splitting example with solid lines for block split: a) with prediction splitting depicted as dotted lines; b) with transform splitting depicted as dotted lines [10] Figure 9: VP9 intraprediction block [10] Figure 10: VP9 Angular Intra Prediction modes [10]
EE5359 Spring 2014 9 Experiments Analysis will be performed on the various image quality measurements such as:- Objective quality measures-mse,psnr [23][24] Subjective quality measures- SSIM [25] BD-PSNR [25] BD-Bit Rate [25] Computational Complexity MSE and PSNR[23][24] MSE and PSNR for a NxM pixel image are defined as where o is the original image and r is the reconstructed image. M and N are the width and height of an image and =L is the maximum pixel value in the NxM pixel image. Video Test Sequences [9] Sequence Resolution Freame Rate(fps) BlowingBubbles 416x240 50
EE5359 Spring 2014 10 Basketball Drive 1,920 X 1,080 50 PeopleOnStreet 2,560 X 1,600 30 Conclusions The aim of this project is to experiment and present a thorough study, implementation and exhaustive comparison of intra-frame only of the contending video coding standards HEVC and VP9 based on different performance metrics like MSE, PSNR, SSIM etc. for different video coding sequences [9]. Based on the values of these performance metrics, conclusions will be drawn as to which video coding standard is best suited for high definition video compression.
EE5359 Spring 2014 11 REFERENCES [1]G.Sullivan, J.Ohm, W.Han and T.Wiegand, Standardized Extensions of the High Efficiency Video Coding (HEVC) Standard, Selected Topics in Signal Processing, IEEE Journal of Volume: 7, Issue: 6, Pages: 1001-1016, 2013. [2]Iain Richardson/Vcodex.com HEVC An Introduction to High Efficiency Video Coding 2013 [3]M. Goldman High Efficiency Video Coding (HEVC) Next Generation Compression technology SMPTE Vol. 121 No. 5 pages: 27-33, July-August 2012 [4]M.Shafique and J.Henkel Low Power Design of the Next-Generation High Efficiency Video Coding IEEE Automation Conference (ASP-DAC), 19th Asia and South Pacific, Pages: 274 281, 2014. [5]D.Grois and D.Marpe Performance Comparison of H.265/MPEG-HEVC, VP9, and H.264/MPEG- AVC Encoders IEEE Picture Coding Symposium (PCS) Pages: 394 397, 2013. [6] Access website http://forum.doom9.org/ How HEVC/H.265 works, technical details & diagrams [7] Access website http://forum.doom9.org How VP9 works, technical details & diagram [8] D.Mukherji et al, The Latest open-source video codec VP9- An overview and Preliminary results IEEE Picture Coding Symposium (PCS) Pages: 390 393, 2013. [9] J.Bankoski et al, Towards a Next Generation Open source Video Codec SPIE Vol. 8666 Page 2, 2013. [10]P.Sharabayko et al, Intra Compression Effciency in VP9 and HEVC Applied Mathematical Sciences, Vol.7 pp no.137, 6803 6824, 2013. [11] K.R.Rao, D. Kim and J.J. Hwang, Video coding standards: AVS China, H.264/MPEG-4 Part10, HEVC, VP6, DIRAC and VC-1", Springer, 2014. [12]Access the website http://www-ee.uta.edu/dip/courses/ee5359/ for Thesis Proposal on Multiplexing/Demultiplexing of main profile of HEVC/H.265 video stream with AAC Audio bit stream, and achieving lip synchronization Mridula Warrier, 2014. [13] Access website http://www.webmproject.org/vp9/ PPT on WebM and the new Open Video Codec S. Bultje, M.Frost. [14] Access the website http://www-ee.uta.edu/dip/courses/ee5359/ Project on Implementation and performance analysis of H.264 intra frame, A.G.Solanki, 2012 [15]A.Grange, Internet draft from Network Working Group A VP9 Bitstream Overview August 2013. [16]HEVC JCT-VC HM reference software https://hevc.hhi.fraunhofer.de/svn/svn_tmucsoftware/tags/hm-1.0. HM-1.0
EE5359 Spring 2014 12 [17] F.Bossen, D.Flynn and K.Suhring (July 2011), HEVC reference software manual http://phenix.intevry.fr/jct/doc_end_user/documents/6_torino/wg11/jctvc-f634-v2.zip [18]JCT-VC documents are publicly available at: http://ftp3.itu.ch/av-arch/jctvc-site, http://phenix.itsudparis.eu/jct/. [19] WEBM-VP9 http://www.webmproject.org/code/, http://git.chromium.org/webm/libvpx.git. [20] I.Richardson, H.264 and MPEG-4 video compression: video coding for next generation multimedia, Wiley 2 nd edition, Aug. 2010. [21]Access the website http://www-ee.uta.edu/dip/courses/ee5359/ for Thesis on Complexity Reduction for VP6 to H.264 Transcoder using motion vector reuse Jay R Padia, May 2010. [22]PSNR http://en.wikipedia.org/wiki/peak_signal-to-noise_ratio. [23]MSE http://en.wikipedia.org/wiki/mean_squared_error [24]SSIM http://en.wikipedia.org/wiki/structural_similarity [25]X.Li et al, Rate-complexity-distortion evaluation for hybrid video coding IEEE ICME Pages: 685 690, 2010. [26] H.Schwarz et al Overview of the Scalable Video Coding Extension of the H.264/AVC Standard Volume: 17, Issue: 9, Pages: 1103 1120, 2007. [27]Computational Complexity http://en.wikipedia.org/wiki/computational_complexity_theory.