PERFORMANCE ANALYSIS OF AVS-M AND ITS APPLICATION IN MOBILE ENVIRONMENT Under the guidance of Dr. K R. Rao FINAL REPORT By Vidur Vajani (1000679332) vidur.vajani@mavs.uta.edu
Introduction AVS stands for the Audio Video coding Standard Workgroup of China, who develops audio/video coding standards as well as system and digital right management standards AVS-M is the AVS video coding standard targeting for mobile multimedia applications AVS M is a Jiben Profile (7 th part) out of the 10 different part of AVS.
History Figure 1 : History of audio video coding standards [4]
Parts of AVS standard Table 1: AVS- China Parts[6]
Profile Comparison[10] Table 2 : Comparing different profiles of AVS China [10]
AVS China Profiles[6] Table 3 : Profiles and their applications [6]
Data Formats[7]: Progressive scan: This format is directly compatible with all content that originates in film. AVS also codes progressive content at higher frame rates and significantly less complex than coding of interlaced data. Interlaced scan: AVS also provides coding tools for interlaced scan format
Layered Structure Figure 2: AVS layered data structure[7]
Sequence [ 7] Figure 3 : Video sequence example [7] The Sequence layer provides an entry point into the coded video. Sequence headers should be placed in the bitstream to support user access appropriately for the given distribution medium
Picture/Frame[1], [6], [7]: The Picture layer provides the coded representation of a video frame. It comprises a header with mandatory and optional parameters and optionally with user data. Three types of Picture are defined by AVS: Intra Pictures (I-pictures) Predicted Pictures (P-pictures) Interpolated Pictures (B-Pictures) Figure 4 :Current picture predicted from previous P pictures [7]
Slices [7] The Slice structure provides the lowest-layer mechanism for resynchronizing the bitstream in case of transmission error. Figure 5 : Slice Layer example [7]
Macroblock:[7] A Macroblock includes the luminance and chrominance component pixels Macroblock in 4:2:0 format Macroblock in 4:2:2 format Figuer 6 : Macroblock format [8]
Blocks The block is the smallest coded unit and contains the transform coefficient data for the prediction errors. In the case of Intra-coded blocks, Intra prediction is performed from neighboring Blocks.
AVS-M Encoder[10] Figure 7: AVS-M encoder [10]
AVS Tools [19]: High level tools similar to H.264/AVC: Network abstraction layer (NAL) unit structure Parameter sets. Instantaneous decoding refresh (IDR) picture Gradual decoding refresh (GDR) or gradual random access Flexible slice coding Reference picture numbering Non-reference P picture Constrained intra prediction Loop filter disabling at slice boundaries Byte-stream format.
AVS Tools [19]: High Level Tools / Features Different Than H.264/AVC Picture ordering and timing Random access point indicator Picture header Signaling of scalable points Reference picture management Hypothetical reference decoder (HRD)
Network Abstraction Layer (NAL) [7] In AVS M Video compression, a compressed video bitstream is made up of Access units (AUs). AU contains information for decoding a picture. AU consists a no. of NAL units, some of them are optional. A NAL unit can be a sequence parameter set(sps), a picture parameter set(pps), a picture header, or a slice_layer_rbsp (raw byte sequence payload) which consists of a slice_header followed by slice data.17
Intra Prediction[16] Two types of intra prediction modes are adopted in AVS-P7. 1. Intra_4x4 2. Direct Intra Prediction (DIP).
Intra_4x4 Fig. 8 Intra_4x4 Prediction including current block and its surrounding coded pixels for prediction [16] Fig. 9 Eight Directional Prediction Modes in AVS-P7 [16]
Intra_4x4 Fig. 10 Nine Intra_4x4 prediction Modes in AVS-P7 [16]
3 Direct Intra Prediction[16] direct intra prediction mainly contains 5 steps. Step 1: All 16 4 4 blocks in a MB use their MPMs to do Intra_4 4 prediction and calculate RDCost(DIP) of this MB. Step 2: Mode search of Intra_4 4, find the best intra prediction mode of each block, and calculate RDCost(Intra_4x4). Step 3: Compare RDCost(DIP) and RDCost(Intra_4x4). If RDCost(DIP) is less than RDCost(Intra_4x4), DIP flag equals to 1 then go to step 4, else DIP lag equals to 0 go to step 5. Step 4: Encode the MB using DIP and finish encoding of this MB. Step 5: Encode the MB using ordinary Intra_ 4 4 and finish encoding of this MB
Interframe Prediction[1], [12] The positions of the integer, half and quarter pixel samples are shown in the figure 8. Capital letters indicate integer sample positions, while small letters indicate half and quarter sample positions. Figure 11 : The position of integer, half and quarter pixel samples [1]
Interframe Prediction[1], [12] If the half_pixel_mv_flag is equal to 1, the precision of the motion vector is up to ½ pixel, otherwise the precision of motion vector is up to ¼ pixel. When half_pixel_mv_flag is not present in the bitstream, it shall be inferred to be 11. The interpolated values at half sample positions can be obtained using 8 tap filter F1 = (-1, 4,- 12,41,41,-12,4,-1) and 4 tap filter F2 = (-1, 5,5, - 1).
Deblocking Filter [3] AVS M makes use of a simplified deblocking filter, wherein boundary strength is decided at MB level. Filtering is applied to the boundaries of luma and chroma blocks except for the boundaries of the pictures or slice. Figure 12 luma and chroma block edges [2]
When deblocking filter is applied For both inter and intra mode filter edge sample level filtering decision is made. If the following three conditions hold good then the filtering process is applied otherwise the filtering process is bypassed [6]. p0-q0 < α (IndexA) p1-p0 < β (IndexB) q1-q0 < β (IndexB) where α and β can be calculated by IndexA and IndexB. p1, p0, q1 and q0 are samples across every sample-level boundary. Figure 13 : Horizontal or vertical Edge of 4 4 Block [1]
AVS-M Decoder[12] Figure 14 : AVS-M decoder [12]
Inverse Transform and Quantization: Only integer 4 X 4 block-based DCT is adopted in AVS-M to simplify the hardware design and decrease the processing complexity of residuals. Figure 15 Inverse DCT matrix of AVS-M [12]
In-loop deblocking [12]: Deblocking is performed across the edges of 4x4 block on the nearest two pixels. The filter mode is determined by the macroblock type and the QP of current macroblock, i.e. if the macroblock is INTRA coded, the filter is INTRA mode; and if the macroblock is not SKIP or the QP is not less than the pre-defined threshold, the filter is INTER mode.
Error concealment tools in AVS-M decoder [17] Many techniques have been proposed to deal with the transmission error problem. Generally, all these techniques can be categorized into three kinds: forward error concealment, backward error concealment interactive error concealment.
Error concealment tools in AVS-M decoder [17] Forward error concealment refers to techniques in which the encoder plays the primary role. which partitions video data into more than one layer with different priority. The layer with higher priority is delivered with a higher degree of error protection. Better quality can be achieved when more layers are received at decoder side.
Error concealment tools in AVS-M decoder [17] Backward error concealment refers to the concealment or estimation of lost information due to transmission errors in which the decoder fulfills the error concealment task. The decoder and encoder interactive techniques achieve the best reconstruction quality, but are more difficult to implement. Generally speaking, a feedback channel is needed from decoder to encoder and low time delay should be guaranteed.
Flow chart for encoder [15] GenerateParamterSets() Figure 16: Flow chart for main() [15]
CIF and QCIF formats Most common test sequences are Common Intermediate Format (CIF) and Quadrature Common Intermediate Format (QCIF). Their formats are shown in figure 3 and 4 respectively. Fig 17: Common Intermediate Format (CIF) 4:2:0 chroma sampling [24] Fig 18: Quadrature Common Intermediate Format (QCIF) 4:2:0 chroma sampling [24]
Simulation Results for qcif sequence #1 Input Sequence: miss-america_qcif.yuv Total No: of frames: 30 frames. Original file size : 1114Kb Width: 176. Height: 144. Frame rate: 30 fps.
QCIF Sequence #1 miss-america-qcif (4:2:0)[22] Original File QP = 10 QP = 50 QP = 63 Fig 19: Video quality at various QP values for miss_america_qcif
Results for miss-america_qcif Sequence QP COMPRESSED Compression Bit Rate [Kbps] Y-PSNR[dB] Y-SSIM FILE SIZE [kb] Ratio 0 330 3.3757:1 2698.85 54.2773 0.9972 5 201 5.5422:1 1641.18 51.3345 0.9946 10 111 10.036:1 902.83 49.1926 0.9916 20 23 48.434:1 179.76 44.9110 0.9842 30 8 139.25:1 56.80 40.7322 0.9716 40 4 278.50:1 30.66 36.0983 0.9429 50 2.80 397.85:1 22.21 30.7096 0.8869 55 2.55 436.86:1 20.17 28.0461 0.8455 60 2.40 464.16:1 18.98 25.1375 0.7999 63 2.33 478.11:1 18.42 22.0501 0.7829 Table 4: Compressed file size, compression ratio, bit rate, PSNR and SSIM at various QP for miss-america_qcif sequence
Plots for miss-america_qcif Fig 20: PSNR Vs Bit Rate Fig 21 : SSIM Vs Bit Rate
Simulation Results for Qcif sequence #2 Input Sequence: mother-daughter_qcif.yuv Total No: of frames: 30 frames. Original file size : 1139Kb Width: 176. Height: 144. Frame rate: 30 fps.
Screenshot of mother-daughter_qcif (4:2:0)[22] Original file QP = 10 QP = 50 QP = 63 Fig 22: Video quality at various QP values for mother_daughter_qcif
Results for mother-daughter_qcif Sequence QP COMPRESSED Compression Bit Rate [Kbps] Y-PSNR Y-SSIM FILE SIZE [kb] Ratio [db] 0 237 4.8059:1 1937.45 53.97741 0.9981 5 127 8.9685:1 1037.33 51.1885 0.9964 10 63 18.0794:1 514.99 48.8490 0.9945 20 19 59.9474:1 152.79 43.4243 0.9856 30 8 142.3750:1 60.56 38.4661 0.9617 40 4 284.7500:1 31.99 33.7337 0.9030 50 2.79 408.2437:1 22.15 29.4328 0.8023 55 2.56 444.9219:1 20.29 26.0557 0.6981 60 2.38 478.5714:1 18.83 23.0817 0.6371 63 2.16 527.3148:1 18.53 22.3413 0.6221 Table 5: Compressed file size, compression ratio, bit rate, PSNR and SSIM at various QP for mother-daughter_qcif sequence
Plots for mother-daughter_cif Fig 23 :PSNR Vs Bit Rate Fig 24 : SSIM Vs Bit Rate
Simulation Results for CIF Sequenct #1 Input Sequence: stefan_cif.yuv Total No: of frames: 15 frames. Original file size : 2227.5 Kb Width: 352 Height: 288. Frame rate: 30 fps.
Screenshot of stefan_cif (4:2:0)[22] Original file QP = 10 Fig 25: Video quality at various QP values for stefan_cif
Screenshot of stefan_cif (4:2:0)[22] QP = 50 QP = 63 Fig 26: Video quality at various QP values for stefan_cif
Results for stefan_cif Sequence QP COMPRESSED Compression Bit Rate [Kbps] Y-PSNR[dB] Y-SSIM FILE SIZE [kb] Ratio 0 1082 2.0587:1 17722.04 53.7192 0.9987 5 810 2.7500:1 13257.49 50.5553 0.9973 10 588 3.7883:1 9616.75 48.0813 0.9953 20 270 8.2500:1 4419.03 41.8208 0.9884 30 107 20.8178:1 1749.46 36.0297 0.9742 40 41 54.3293:1 655.40 30.7737 0.9403 50 19 117.2368:1 309.20 25.9537 0.8419 55 15 148.5000:1 233.75 23.6506 0.7556 60 11 202.5000:1 177.33 20.7062 0.5875 63 10 222.7500:1 151.39 19.0242 0.4688 Table 6: Compressed file size, compression ratio, bit rate, PSNR and SSIM at various QP for stefan_cif sequence
Plots for stefan_cif Fig 27 : PSNR Vs Bit Rate Fig 28 : SSIM Vs Bit Rate
Simulation Results for CIF Sequence #2 Input Sequence: silent_cif.yuv Total No of frames: 15 frames. Original file size : 2227.5Kb Width: 352. Height: 288. Frame rate: 30 fps.
Screenshot of silent_cif (4:2:0)[22] Original File QP = 10 Fig 29: Video quality at various QP values for silent_cif
Screenshot of silent_cif (4:2:0)[22] QP = 50 QP = 63 Fig 30: Video quality at various QP values for silent_cif
Results for silent_cif Sequence QP COMPRESSED Compression Bit Rate [Kbps] Y-PSNR[dB] y-ssim FILE SIZE [kb] Ratio 0 592 3.7627:1 9694.90 53.5225 0.9982 5 357 6.2395:1 5836.70 50.6134 0.9965 10 199 11.1935:1 3244.26 47.7497 0.9934 20 66 33.7500:1 1076.57 41.8517 0.9769 30 28 79.5536:1 445.79 36.5718 0.9329 40 13 171.3462:1 206.23 32.0900 0.8498 50 8 278.4375:1 121.74 28.1807 0.7315 55 7 318.2143:1 101.13 25.9689 0.6612 60 5.54 402.0758:1 89.21 23.8874 0.6125 63 5.25 424.2857:1 84.36 22.1109 0.5366 Table 7: Compressed file size, compression ratio, bit rate, PSNR and SSIM at various QP for silent_cif sequence
Plots for silent_cif Fig 31: PSNR Vs Bit Rate Fig 32: SSIM vs Bit Rate
Software used for Quality Measurement[20] Fig 33: Screenshot of MSU Video Quality Measurement Tool software
AVS-M applications AVS-M has lower complexity, and is suitable for mobilemultimedia applications. Major applications of AVS-M are given below. Record and local playback on mobile devices Multimedia Message Service (MMS) Streaming and broadcasting Real-time video conversation Video on demand
Conclusions: AVS part 7 targets low complexity and low picture resolution mobility applications. The AVS encoder and decoder are implemented using AVS M software. Tests are carried out on various QCIF and CIF sequences. The bit rate, PSNR and SSIM values are tabulated. The performance of AVS-china was analyzed by varying the quantization parameter (QP). The PSNR and bit rate and SSIM were calculated. It can be observed that at higher QP the performance is best but decoded file is size is also large. As QP decreases quality of video and size of video decreases.
References: [1] J.Ostermann et al, Video coding with H.264/AVC: Tools, Performance, and Complexity, IEEE Circuits and Systems Magazine, vol. 4, Issue:1, pp. 7 28, Aug. 2004. [2]B. Tang, Y. Chen and W. Ji AVS Encoder Performance and Complexity Analysis Based on Mobile Video Communication, DOI 10.1109/CMC.2009.171, 2009 International Conference on Communications and Mobile Computing [3]L. Fan, S. Ma, F. Wu, Overview of AVS Video Standard, IEEE International Conference on Multimedia and Expo (ICME), pp 423-427, 2004 [4 ] AVS working group official website, http://www.avs.org.cn [5] H. Tiejun AVS Technology, IPR and Applications, available at, www.avs.org.cn/reference/avs 进展 (20101112.ppt [6]S. Devaraju and K.R. Rao, A Study on AVS-M Video Standard, M.S. Thesis, Electrical Engineering Department, University of Texas at Arlington, Arlington, TX, 2009. [7] W. Gao et al, AVS the Chinese next-generation video coding standard, National Association of Broadcasters, Las Vegas, 2004. [8] K. R. Rao and D.N. Kim, Current Video Coding Standards: H.264/AVC, Dirac, AVS China and VC-1, IEEE 42nd Southeastern symposium on system theory (SSST), pp. 1-8,March 2010. [9] W.Gao and T.Huang AVS Standard -Status and Future Plan, Workshop on Multimedia New Technologies and Application, Shenzhen, China, Oct. 2007. [10] L.Fan, Mobile Multimedia Broadcasting Standards, ISBN: 978-0-387-78263-8, Springer US, 2009. [11]W. Gao, K.N. Ngan and L. Yu, Special issue on AVS and its applications: Guest editorial, Signal Process: Image Commun, vol. 24, Issue 4, pp. 245-344, April 2009. [12] Z. Ma et al, Complexity analysis OF AVS-M jiben profile decoder, Proceedings of 2005 International Symposium on Intelligent Signal Processing and Communication Systems, December 13-16, 2005 Hong Kong
References(contd.) [13] M. Liu and Z. Wei, A fast mode decision algorithm for intra prediction in AVS-M video coding Vol. 1, ICWAPR apos;07, Issue, 2-4, pp.326 331, Nov. 2007. [14] Y. Cheng et al, Analysis and application of error concealment tools in AVS-M decoder, Journal of Zhejiang University Science A, vol. 7, pp. 54-58, Jan 2006. [15] S.Hu, X.Zhang and Z.Yang, Efficient Implementation of Interpolation for AVS, Congress on Image and Signal Processing, 2008. Vol 3, pp133 138, 27-30 May 2008 [16] R. Schafer and T. Sikora, Digital video coding standards and their role in video communications, Proc. of the IEEE, vol. 83, pp. 907-924, June 1995. [17] B. Lei et al, Optimization and Implementation of AVS-M Decoder on ARM, DOI 10.1109/ICIG.2007.166, IEEE 2007 [18] L. Yu, AVS Project and AVS-Video Techniques, http://www-ee.uta.edu/dip/courses/ee5351/ispacsavs.pdf, Dec.13, 2005 ISPACS 2005 [19]Y. Wang AVS_M: From standards to Applications, Journal of Computer Science and Technology - Special section on China AVS standard Vol.21. No.3 pp. 332-344, May 2006. [20] MSU video quality measuement tool, http://www.softrecipe.com/download/msu_video_quality_measurement_tool.html [21] MATLAB software, http://www.mathworks.com/products/matlab/tryit.html [22] Test video sequences http://trace.eas.asu.edu/yuv/ [23] Z. Wang et al, "Image quality assessment: From error visibility to structural similarity," IEEE Trans. Image Processing, vol. 13, no. 4, pp. 600-612, Apr. 2004 [24] S.Kwon, A. Tamhankar and K.R. Rao, Overview of H.264 / MPEG-4 Part 10, J. Visual Communication and Image Representation, vol. 17, pp.186-216, April 2006.
Thank you.