Department of Electrical Engineering

Transcription

1 Department of Electrical Engineering Multimedia Processing Spring 2011 IMPLEMENTATION OF H.264/AVC, AVS China Part 7 and Dirac VIDEO CODING STANDARDS INSTRUCTOR Dr. K R. Rao Term Project Sharan K Chandrashekar ( ) sharan.chandrashekar@mavs.uta.edu

2 ACKNOWLEDGEMENT I would like to acknowledge the continuous guidance and support of Dr. K.R. Rao (Electrical Engineering Department at the University of Texas at Arlington), throughout the various stages of this project. His sound technical knowledge of the subject and professional acumen has been an outstanding source of inspiration in pursuing this course. I sincerely appreciate the help and advice given by Dr.Rao for the successful completion of this project. I would also like to thank Mr. Touseef Khan, senior graduate student at the University, for helping me understand the key concepts underlying the project. Finally I would like to thank my fellow students for their valuable inputs which helped greatly in improvising the project at various stages.

3 Introduction Effective digital representation of image and video has been a subject of overwhelming research over the past 20 years [1]. The need for an international image and video coding standards arose with the increased commercial interest in video communications. New applications in the field of communication, multimedia and broadcasting became possible based on digital video coding technology. Introduction to the video coding and decoding process Figure 1: General video coding and decoding process [7] Firstly, the sender might choose to pre-process the video using format conversion or enhancement techniques. The encoder encodes the video representing it as a bit stream. The video is decoded following which, an optional post-processing step such as format conversion, error concealment or video enhancement may be performed before the receiver receives it. - Video coding standards H.264/AVC, Dirac and AVS China are the latest standards adopted by ITU-T/ISO-IEC, BBC and China standards organization respectively [2].

4 H.264/AVC [2]: H.264/AVC, an open licensed standard, was developed as a result of the collaboration between the ISO/IEC Moving Picture Experts Group and the ITU-T Video Coding Experts Group. It is the most efficient video compression technique available today. It provides an increase in the coding efficiency up to 50% over a wide range of bit rates and resolutions compared to its predecessors. Some of its major applications include video broadcasting, Video on demand, MMS over various platforms like DSL, Ethernet, LAN, wireless and mobile networks etc. H.264 Encoder:

5 H.264 Decoder: H.264 Standard Profiles: Figure 4: H.264 decoder block diagram [7] -

6 Three profiles were defined for the first version of H.264/MPEG4-AVC namely Baseline, Extended and Main profiles. The Baseline profile was targeted at applications in which a minimum of computational complexity and a maximum of error robustness [8]. The Main profile was aimed at applications that require a maximum of coding efficiency, with somewhat less emphasis on error robustness [8]. The Extended profile was designed to provide a compromise between the Baseline and Main profile capabilities with an additional focus on the specific needs of video streaming applications, and further added robustness to errors and packet losses [8]. Dirac [10]: Dirac is a video compression system developed by the British Broadcasting Corporation (BBC) utilizing motion compensation and wavelet transforms. It is a motion-compensated video codec like the Moving Picture Experts Group (MPEG) standard [10], i.e. the image motion is tracked and the motion information is used to track a later frame. Dirac video codec applications span from mobile and internet to ultra HDTV and film and video production. Wavelet Transforms: The 2D discreet wavelet transforms provide Dirac with the flexibility to operate at a range of resolutions. Applied to two-dimensional images, wavelet filters are normally applied in both vertical and horizontal directions to each image component to produce four so called sub bands termed Low-Low (LL), Low-High (LH), High-Low (HL) and High-High (HH) [24]. In the case of two dimensions, only the LL band is iteratively decomposed to obtain the required data. The number of samples in each resulting sub band is as implied by figure 7. The critical sampling ensures that after each decomposition the resulting bands all have one quarter of the samples of the input signal.

7 Dirac encoder: - Figure 9: Dirac encoder block diagram [10]

8 Dirac Decoder: AVS China [6]: Figure 10: Dirac decoder block diagram [2] The AVS video codec was developed by the Audio Video Coding Standard Working Group of China. The ten parts of the AVS China family is as shown in table 1. Table 1: Ten Parts of the AVS China Standard Family [2]

9 AVS China comprises of four different profiles namely Jizhun, Jiben, Shenzan and Jiaqiang of which the Jiben profile (basic profile) is defined in AVS Part 7 for mobile applications. Key applications Jizhun profile Jiben profile Shenzhan profile Jiaqiang profile Television broadcasting, HDTV, etc. Mobility applications, etc. Video surveillance, etc. Multimedia entertainment, etc. Table 2: Applications of various profiles of AVS China [5] AVS China Encoder:

10 AVS China Decoder: CIF and QCIF Formats: Figure 12: AVS China decoder block diagram [17] Common Intermediate Format (CIF) and Quadrature Common Intermediate Format (QCIF) determine the resolution of the frame. The resolution of CIF is 352x288 and the resolution of QCIF is 1/4 of CIF, which is 176x144 [26]. Consider the YCbCr family of color spaces where Y represents the luminance, Cb represents the blue-difference chroma component and Cr represents the red-difference chroma component [25]. For QCIF and CIF, the luminance Y is equal to the resolution. If sampling resolution 4:2:0 is used, the for CIF, the Cb and Cr are 176 x 144 lines and for QCIF, the Cb and Cr are 88 x 72 lines.

11 Some Important Parameters: Structural Similarity Metric (SSIM) [22]: This index is a method for measuring the similarity between two frames. It is a full reference metric, or in other words, the measuring of image quality is done using an initial uncompressed or distortion-free frame as reference. Mean Squared Error (MSE) [22]: The MSE is computed by averaging the squared intensity differences of the distorted and reference image/frame pixels. Two distorted images with the same MSE may have very different types of errors, some of which are much more visible than others. Peak Signal-to-Noise Ratio (PSNR) [22]: The PSNR is most commonly used as a measure of quality of reconstruction of compression codecs. The signal in this case is the original data, and the noise is the error introduced by compression.

12 Implementation of the JM 17.2 [14] Software for the H.264/AVC video codec YUV File 1: coastguard_cif.yuv Specifications: CIF sequence: coastguard_cif.yuv Total number of frames: 300 Height: 352; Width: 288 Total number of frames used: 100 Original file size: kb Frame rate: 30 frames/second

13 QP = 0 QP = 30 QP = 50 YUV File 1: coastguard_cif.yuv JM 17.2 Software

14 MSE PSNR (db) EE 5359 SPRING 2011 YUV File 1: coastguard_cif.yuv 70 PSNR (db) vs x MSE vs x 10 4 JM 17.2 Software

15 YUV File 2: coastguard_qcif.yuv Specifications: QCIF sequence: coastguard_qcif.yuv Total number of frames: 300 Height: 176; Width: 144 Total number of frames used: 100 Original file size: 3713 kb Frame rate: 30 frames/second

16 QP = 0 QP = 30 QP = 50 YUV File 2: coastguard_qcif.yuv JM 17.2 Software

17 MSE PSNR (db) EE 5359 SPRING 2011 YUV File 2: coastguard_qcif.yuv 80 PSNR (db) vs MSE vs JM 17.2 Software

18 YUV File 3: container_cif.yuv Specifications: CIF sequence: container_cif.yuv Total number of frames: 300 Height: 352; Width: 288 Total number of frames used: 100 Original file size: kb Frame rate: 30 frames/second

19 QP = 0 QP = 30 QP = 50 YUV File 3: container_cif.yuv JM 17.2 Software

20 MSE PSNR (db) EE 5359 SPRING 2011 YUV File 3: container_cif.yuv 70 PSNR (db) vs MSE vs JM 17.2 Software

21 YUV File 4: container_qcif.yuv Specifications: QCIF sequence: container_qcif.yuv Total number of frames: 300 Height: 176; Width: 144 Total number of frames used: 100 Original file size: 3713 kb Frame rate: 30 frames/second

22 QP = 0 QP = 30 QP = 50 YUV File 4: container_qcif.yuv JM 17.2 Software

23 MSE PSNR (db) EE 5359 SPRING 2011 YUV File 4: container_qcif.yuv 70 PSNR (db) vs MSE vs JM 17.2 Software

24 Implementation of the Dirac Software [15] for the Dirac video codec YUV File 1: coastguard_cif.yuv Specifications: CIF sequence: coastguard_cif.yuv Total number of frames: 300 Height: 352; Width: 288 Total number of frames used: 100 Original file size: kb Frame rate: 12.5 frames/second

25 QF = 0 QF = 8 QF = 12 YUV File 1: coastguard_cif.yuv Dirac Software

26 MSE PSNR (db) EE 5359 SPRING 2011 YUV File 1: coastguard_cif.yuv 50 PSNR (db) vs MSE vs Dirac Software

27 YUV File 2: coastguard_qcif.yuv Specifications: QCIF sequence: coastguard_qcif.yuv Total number of frames: 300 Height: 176; Width: 144 Total number of frames used: 100 Original file size: 3713 kb Frame rate: 12.5 frames/second

28 QF = 0 QF = 8 QF = 12 YUV File 2: coastguard_qcif.yuv Dirac Software

29 MSE PSNR (db) EE 5359 SPRING 2011 YUV File 2: coastguard_qcif.yuv 50 PSNR (db) vs MSE vs Dirac Software

30 YUV File 3: container_cif.yuv Specifications: CIF sequence: container_cif.yuv Total number of frames: 300 Height: 352; Width: 288 Total number of frames used: 100 Original file size: kb Frame rate: 12.5 frames/second

31 QF = 0 QF = 8 QF = 12 YUV File 3: container_cif.yuv Dirac Software

32 MSE PSNR (db) EE 5359 SPRING 2011 YUV File 3: container_cif.yuv 50 PSNR (db) vs MSE vs Dirac Software

33 YUV File 4: container_qcif.yuv Specifications: QCIF sequence: container_qcif.yuv Total number of frames: 300 Height: 176; Width: 144 Total number of frames used: 100 Original file size: 3713 kb Frame rate: 12.5 frames/second

34 QF = 0 QF = 8 QF = 12 YUV File 4: container_qcif.yuv Dirac Software

35 MSE PSNR (db) EE 5359 SPRING 2011 YUV File 4: container_qcif.yuv 50 PSNR (db) vs MSE vs Dirac Software

36 Implementation of the AVS-M Software [27] for the AVS China video codec YUV File 1: coastguard_cif.yuv Specifications: CIF sequence: coastguard_cif.yuv Total number of frames: 300 Height: 352; Width: 288 Total number of frames used: 100 Original file size: kb Frame rate: 30 frames/second

37 QP = 0 QP = 30 QP = 50 YUV File 1: coastguard_cif.yuv AVS-M Software

38 MSE PSNR (db) EE 5359 SPRING 2011 YUV File 1: coastguard_cif.yuv 60 PSNR (db) vs x MSE vs x 10 4 Dirac Software

39 YUV File 2: coastguard_qcif.yuv Specifications: QCIF sequence: coastguard_qcif.yuv Total number of frames: 300 Height: 176; Width: 144 Total number of frames used: 100 Original file size: 3713 kb Frame rate: 30 frames/second

40 QP = 0 QP = 30 QP = 50 YUV File 2: coastguard_qcif.yuv Dirac Software

41 MSE PSNR (db) EE 5359 SPRING 2011 YUV File 2: coastguard_qcif.yuv 60 PSNR (db) vs MSE vs Dirac Software

42 YUV File 3: container_cif.yuv Specifications: CIF sequence: container_cif.yuv Total number of frames: 300 Height: 352; Width: 288 Total number of frames used: 100 Original file size: kb Frame rate: 30 frames/second

43 QP = 0 QP = 30 QP = 50 YUV File 3: container_cif.yuv Dirac Software

44 MSE PSNR (db) EE 5359 SPRING 2011 YUV File 3: container_cif.yuv 60 PSNR (db) vs x MSE vs x 10 4 Dirac Software

45 YUV File 4: container_qcif.yuv Specifications: QCIF sequence: container_qcif.yuv Total number of frames: 300 Height: 176; Width: 144 Total number of frames used: 100 Original file size: 3713 kb Frame rate: 30 frames/second

46 QP = 0 QP = 30 QP = 50 YUV File 4: container_qcif.yuv Dirac Software

47 MSE PSNR (db) EE 5359 SPRING 2011 YUV File 4: container_qcif.yuv 60 PSNR (db) vs MSE vs Dirac Software

48 MSE PSNR(dB) EE 5359 SPRING 2011 Comparison of the three codecs YUV File 1: coastguard_cif.yuv 70 PSNR (db) vs H Dirac AVS China x MSE vs. H.264 Dirac AVS China x 10 4

49 PSNR (db) SSIM EE 5359 SPRING SSIM vs H.264 Dirac AVS China x 10 4 YUV File 2: container_qcif.yuv 70 PSNR (db) vs H Dirac AVS China

50 SSIM MSE EE 5359 SPRING MSE vs. H.264 Dirac AVS China SSIM vs H.264 Dirac AVS China

51 MSU Video Quality Measurement Tool [21]

52 CONCLUSIONS The H.264/AVC has an edge over the AVS China and Dirac video codecs in terms of performance providing higher compression ratios and quality. The plots suggest that the PSNR and SSIM increase with increasing bit rates and the error decreases. Faster processors can be used to tackle competitive video file sizes. FUTURE WORK So far, all the work done has been for the baseline profiles of the three codes. There is plenty of scope to explore the mainline and high profiles. Other video formats such as SD and HD can also be used for simulation, provided ample time and faster processors. Also, it could be interesting to compare other video codecs (successor versions of the H.264) and analyze their performance.

53 Abbreviations and Acronyms: AVC: Advanced Video Coding AVS: Audio Video Standard CIF: Common Intermediate Format HDTV: High-Definition Television IEC: International Electrotechnical Commission ISO: International Organization for Standardization ITU-T: International Telecommunication Union - Telecommunication Standardization sector MSE: Mean Square Error PSNR: Peak Signal to Noise ratio QCIF: Quarter Common Intermediate Format SMPTE: Society of Motion Picture and Television Engineers SSIM: Structural Similarity Metric References [1] T. Sikora, Digital Video Coding Standards and Their Role in Video Communications, Signal Processing for Multimedia, J.S. Byrnes (Ed.), IOS press, pp , [2] K. R. Rao, and D. N. Kim, Current Video Coding Standards: H.264/AVC, Dirac, AVS China and VC-1, IEEE 42nd Southeastern symposium on system theory (SSST), March , pp. 1-8, March [3] K. Onthriar, K. K. Loo and Z. Xue, Performance comparison of emerging Dirac video codec with H.264/AVC, IEEE International Conf. on Digital Telecommunications, ICDT 2006, vol. 6, Page: 22, Issue: 29-31, Aug [4] X-F Wang, and D-B Zhao, Performance comparison of AVS and H.264/AVC video coding standards, J. Comput. Sci. & Technol., vol. 21, No. 3, pp , May [5] L. Yu, S. Chen, and J. Wang, Overview of AVS video coding standards, Signal Processing: Image Communication, vol. 24, pp , April [6] L. Fan et al, Overview of AVS Video Standard, IEEE International conference on multimedia and expo (ICME), vol.1, pp , June [7] J. Ostermann et al, Video coding with H.264/AVC: Tools, Performance, and Complexity, IEEE Circuits and Systems magazine, vol. 4, Issue 1, pp. 7 28, Aug [8] D. Marpe, T. Wiegand, and G.J. Sullivan, The H.264/MPEG4 Advanced Video Coding Standard and its Applications, IEEE Communications magazine, vol. 44, Issue: 8, pp , August [9] I. E. Richardson, The H.264 advanced video compression standard, Wiley, 2 nd edition, [10] T. Borer and T. Davies, Dirac video compression using open technology, BBC EBU Technical Review, July 2005.

54 [11] T. Borer, Dirac coding: Tutorial & Implementation, EBU Networked Media Exchange seminar, EBU, Geneva, June [12] Dirac Specification, Version 2.2.3, Available: [13] BBC Research on Dirac: [14] H.264/AVC Software source reference: [15] Dirac video download source reference: [16] YUV video sequences source: [17] L. Fan, Mobile Multimedia Broadcasting Standards, Springer, [18] H. Malvar et al, Low-complexity transform and quantization in H.264/AVC, IEEE Transactions on Circuits and Systems for Video Technology, vol. 13, pp , July [19] J. Ribas-Corbera et al, A generalized hypothetical reference decoder for H.264/AVC, IEEE Transactions on Circuits and Systems, vol. 13, no. 7, pp , July [20]Compression for High-Quality, High Bandwidth Video, Intel Software Network: [21] MSU video quality measurement tool: [22] Z. Wang, et al, Image quality assessment: From error visibility to structural similarity, IEEE Transactions on Image Processing, vol.13, no.4, pp , April [23] A. Ravi, and K.R. Rao, Performance analysis and comparison of the Dirac video codec with H.264/MPEG-4 part 10 AVC, International Journal of Wavelets, Multiresolution and Information Processing (accepted), January Available: [24] Dirac developer support documentation: [25] YCbCr Article: [26] S.Kwon, A. Tamhankar and K.R. Rao, Overview of H.264 / MPEG-4 Part 10, Journal of Visual Communication and Image Representation, vol. 17, pp , April [27] AVS China Software Download Tool (FTP Client):