STUDY AND PERFORMANCE COMPARISON OF HEVC AND H.264 VIDEO ENCODERS

FINAL REPORT ON STUDY AND PERFORMANCE COMPARISON OF HEVC AND H.264 VIDEO ENCODERS A PROJECT UNDER THE GUIDANCE OF DR. K. R. RAO COURSE: EE5359 - MULTIMEDIA PROCESSING, SPRING 2014 SUBMISSION DATE: 1 st MAY 2014 SUBMITTED BY UMA SAGAR MADHUGIRI DAYANANDA UT ARLINGTON ID: 1001002964 EMAIL ID: uma.sagar@mavs.uta.edu DEPARTMENT OF ELECTRICAL ENGINEERING UNIVERSITY OF TEXAS, ARLINGTON Page 1

Table of Contents 1. Scope of Project... 4 2. Fundamental Concepts in Video Coding... 4 2.1 Color Spaces... 4 3. High Efficiency Video Coding... 5 3.1 Introduction... 5 3.2 Encoder and Decoder in HEVC... 5 3.3 Features of HEVC... 7 3.3.1 Partitioning... 7 3.3.2 Prediction... 8 3.3.3 Transform and Quantization... 8 3.3.4 Entropy coding... 8 4. H.264/Advanced Video Coding... 9 4.1 Introduction... 9 4.2 Encoder and Decoder in H.264... 9 4.3 Features of H.264... 11 4.3.1 Prediction... 11 4.3.2 Transform and Quantization... 11 5. Comparison Metrics... 12 5.1 Peak Signal to Noise Ratio... 12 5.2 Bjontegaard Delta Bitrate (BD-BR) and Bjontegaard Delta PSNR (BD-PSNR)... 12 5.3 Implementation Complexity... 12 6. Profiles used for Comparison... 13 6.1 Main profile in HEVC... 13 6.2 High Profile in H.264... 13 7. Test Sequences... 13 8. Comparison Methodology... 14 8.1 Configuration of HM 13.0... 14 8.2 Configuration of JM 18.6... 14 8.3 Testing Platform... 15 9. Test Configurations... 16 9.1 Intra-only configuration... 16 9.2 Random Access Configuration... 16 10. Results... 17 10.1 All-Intra Configuration... 17 10.2 Random Access Configuration... 23 11. Further Work... 28 12. References... 29 Page 2

List of Acronyms and Abbreviations AVC: Advanced Video Coding. BD-BR: Bjontegaard Delta Bitrate. BD-PSNR: Bjontegaard Delta Peak Signal to Noise Ratio. CABAC: Context Adaptive Binary Arithmetic Coding. CTB: Coding Tree Block. CTU: Coding Tree Unit. CU: Coding Unit. DBF: De-blocking Filter. DCT: Discrete Cosine Transform. HEVC: High Efficiency Video Coding. HM: HEVC Test Model. IEC: International Electro-technical Commission. ISO: International Organization for Standardization. ITU-T: International Telecommunication Union- Telecommunication Standardization Sector. JCT: Joint Collaborative Team. JCT-VC: Joint Collaborative Team on Video Coding. JM: H.264 Test Model. JPEG: Joint Photographic Experts Group. MC: Motion Compensation. ME: Motion Estimation. MPEG: Moving Picture Experts Group. MSE: Mean Square Error. PB: Prediction Block. PSNR: Peak Signal to Noise Ratio. QP: Quantization Parameter SAO: Sample Adaptive Offset. SSIM: Structural Similarity Index. TB: Transform Block. TU: Transform Unit. VCEG: Video Coding Experts Group. Page 3

1. Scope of Project This project aims at studying the state-of-the-art High Efficiency Video Coding (HEVC) [12] and H.264/Advanced Video Coding (AVC) [13] video codecs and gaining an understanding of various techniques in video encoding such as prediction, transform, quantization and coding. A performance comparison of these video encoders based on various metrics such as computational time, PSNR, BD- Bitrate and BD-PSNR [11] [15] in all-intra and random access modes are carried out. The HM 13.0 [24] and JM 18.6 [25] test models for HEVC and H.264 respectively are used for this purpose. 2. Fundamental Concepts in Video Coding 2.1 Color Spaces The common color spaces for digital image and video representation are: RGB Color space Each pixel is represented by three numbers indicating the relative proportions of red, green and blue colors Color space Y is the luminance component, a monochrome version of color Y C r C b image. Y is a weighted average of R, G and B: Y k R k G k B, where k are the weighting factors. r g b The color information is represented as color differences or chrominance components, where each chrominance component is difference between R, G or B and the luminance Y. As the human visual system is less sensitive to color than the luminance component, Y C r C b has advantages over RGB space. The amount of data required to represent the chrominance component reduces without impairing the visual quality [1]. The popular patterns of sub-sampling [1] are: 4:4:4 The three components, Y : luminance samples there are 4 C r C r : and 4 C b C b have the same resolution, which is for every 4 samples. 4:2:2 For every 4 luminance samples in the horizontal direction, there are 2 samples. This representation is used for high quality video color reproduction. 4:2:0 C r and C b each have half the horizontal and vertical resolution ofy used in applications such as video conferencing, digital television and DVD storage. C r and 2 C b. This is popularly Figure 1: 4:2:0 sub-sampling pattern [1] Page 4

Figure 2: 4:2:2 and 4:4:4 sub-sampling patterns [1] 3. High Efficiency Video Coding 3.1 Introduction High Efficiency Video Coding (HEVC) [12] is an international standard for video compression developed by a working group of ISO/IEC MPEG (Moving Picture Experts Group) and ITU-T VCEG (Video Coding Experts Group). The main goal of HEVC standard is to significantly improve compression performance compared to existing standards (such as H.264/Advanced Video Coding [13]) in the range of 50% bit rate reduction at similar visual quality [5]. HEVC is designed to address existing applications of H.264/MPEG-4 AVC and to focus on two key issues: increased video resolution and increased use of parallel processing architectures [5]. It primarily targets consumer applications as pixel formats are limited to 4:2:0 8-bit and 4:2:0 10-bit. The next revision of the standard, planned for 2014, will enable new use-cases with the support of additional pixel formats such as 4:2:2 and 4:4:4 and bit depth higher than 10-bit [20], embedded bitstream scalability and 3D video [33]. 3.2 Encoder and Decoder in HEVC Source video, consisting of a sequence of video frames, is encoded or compressed by a video encoder to create a compressed video bit stream. The compressed bit stream is stored or transmitted. A video decoder decompresses the bit stream to create a sequence of decoded frames [17]. The video encoder performs the following steps: Partitioning each picture into multiple units Predicting each unit using inter or intra prediction, and subtracting the prediction from the unit Transforming and quantizing the residual (the difference between the original picture unit and the prediction) Entropy encoding transform output, prediction information, mode information and headers The video decoder performs the following steps: Entropy decoding and extracting the elements of the coded sequence Rescaling and inverting the transform stage Predicting each unit and adding the prediction to the output of the inverse transform Reconstructing a decoded video image Figure 3 represents the block diagram of HEVC CODEC [17]: Page 5

Figure 3: Block Diagram of HEVC CODEC [17] Figure 4[9] and 5[31] represent the detailed block diagrams of HEVC encoder and decoder respectively: Figure 4: Block Diagram of HEVC Encoder [9] Page 6

3.3 Features of HEVC Figure 5: Block Diagram of HEVC Decoder [31] 3.3.1 Partitioning HEVC supports highly flexible partitioning of a video sequence. Each frame of the sequence is split up into rectangular or square regions (units or blocks), each of which is predicted from previously coded data [17]. After prediction, any residual information is transformed and entropy encoded. Each coded video frame, or picture, is partitioned into tiles and/or slices, which are further partitioned into coding tree units (CTUs). The CTU is the basic unit of coding, analogous to the macro-block in earlier standards [17], and can be up to 64x64 pixels in size. A coding tree unit can be subdivided into square regions known as coding units (CUs) using a quadtree structure. Each CU is predicted using inter or intra prediction and transformed using one or more transform units. Figure 6: Picture, Slice, Coding Tree Unit (CTU), Coding Unit (CU) [17] Page 7

3.3.2 Prediction Frames of video are coded using intra or inter prediction: Intra prediction: Each PU is predicted from neighbouring image data in the same picture, using DC prediction (an average value for the PU), planar prediction (fitting a plane surface to the PU) or directional prediction (extrapolating from neighbouring data). Inter prediction: Each PU is predicted from image data in one or two reference pictures (before or after the current picture in display order), using motion compensated prediction. Figure 7: Modes and angular intra prediction directions in HEVC [5] 3.3.3 Transform and Quantization Any residual data remaining after prediction, is transformed using a block transform based on the Discrete Cosine Transform (DCT) or Discrete Sine Transform (DST). One or more block transforms of size 32x32, 16x16, 8x8 and 4x4 are applied to residual data in each CU. Then the transformed data is quantized. Figure 8: CTU showing range of transform (TU) sizes [17] 3.3.4 Entropy coding A coded HEVC bit stream consists of quantized transform coefficients, prediction information such as prediction modes and motion vectors, partitioning information and other header data. All of these elements are encoded using Context Adaptive Binary Arithmetic Coding (CABAC). Page 8

4. H.264/Advanced Video Coding 4.1 Introduction H.264/Advanced Video Coding (AVC) [13] is video coding standard of the ITU-T Video Coding Experts Group and the ISO/IEC Moving Picture Experts Group. The main goals of the H.264/AVC standard were to enhance compression performance and provision of a network-friendly video representation addressing conversational (video telephony) and non-conversational (storage, broadcast, or streaming) applications [8]. 4.2 Encoder and Decoder in H.264 An H.264 video encoder carries out prediction, transform and encoding processes (Figure 9) to produce a compressed H.264 bit stream. An H.264 video decoder carries out complementary processes of decoding, inverse transform and reconstruction to produce a decoded video sequence [18]. Figure 9: Block Diagram of H.264 CODEC [18] Figures 10 and 11[10] represent the detailed block diagrams of H.264 encoder and decoder respectively: Page 9

Figure 10: Block Diagram of H.264 Encoder [10] Figure 11: Block Diagram of H.264 Decoder [10] Page 10

4.3 Features of H.264 4.3.1 Prediction The encoder processes a frame of video in units of a macro-block (16x16 displayed pixels) [18]. It forms a prediction of the macro-block based on previously-coded data, either from the current frame (intra prediction) or from other frames that have already been coded and transmitted (inter prediction). The encoder subtracts the prediction from the current macro-block to form a residual. Intra prediction uses 16x16 and 4x4 block sizes to predict the macro-block from surrounding, previously coded pixels within the same frame (Figure 12). Figure 12: Intra prediction in H.264 [18] Inter prediction uses a range of block sizes (from 16x16 down to 4x4) to predict pixels in the current frame from similar regions in previously coded frames (Figure 11). Figure 13: Inter prediction in H.264 [18] Finding a suitable inter prediction is often described as motion estimation. Subtracting an inter prediction from the current macro-block is motion compensation. 4.3.2 Transform and Quantization A block of residual samples is transformed using a 4x4 or 8x8 integer transform, an approximate form of the Discrete Cosine Transform (DCT) [34]. The transform outputs a set of coefficients, each of which is a weighting value for a standard basis pattern. When combined, the weighted basis patterns recreate the block of residual samples. The output of the transform, a block of transform coefficients, is quantized, i.e. each coefficient is divided by an integer value. Quantization reduces the precision of the transform coefficients according to a quantization parameter (QP). Page 11

5. Comparison Metrics 5.1 Peak Signal to Noise Ratio Peak signal-to-noise ratio (PSNR) [22] [26] is an expression for the ratio between the maximum possible value (power) of a signal and the power of distorting noise that affects the quality of its representation. Because many signals have a very wide dynamic range (ratio between the largest and smallest possible values of a changeable quantity), the PSNR is usually expressed in terms of the logarithmic decibel scale. PSNR is most commonly used to measure the quality of reconstruction of lossy compression codecs. The signal in this case is the original data, and the noise is the error introduced by compression. When comparing compression codecs, PSNR is an approximation to human perception of reconstruction quality. Although a higher PSNR generally indicates that the reconstruction is of higher quality, in some cases it may not. One has to be extremely careful with the range of validity of this metric; it is only conclusively valid when it is used to compare results from the same codec (or codec type) and same content. PSNR is defined via the mean squared error (MSE). Given a noise-free m x n monochrome image I and its noisy approximation K, MSE is defined as: The PSNR is defined as: m 1 n 1 1 MSE I i, j K i, j mn PSNR i 0 j 0 MAX MSE 2 I 10 log10 MAX I 20 log10 Here, MAX I is the maximum possible pixel value of the image. MSE 2 MAX MSE 20 log 10 log 10 I 10 For test sequences in 4:2:0 color format, PSNR is computed as a weighted average of luminance ( PSNR ) and chrominance ( PSNR, PSNR ) components [14] as given below: Y U V 6 PSNR PSNR PSNR PSNR 8 Y U V 5.2 Bjontegaard Delta Bitrate (BD-BR) and Bjontegaard Delta PSNR (BD-PSNR) To objectively evaluate the coding efficiency of video codecs, Bjontegaard Delta PSNR (BD-PSNR) was proposed. Based on the rate-distortion (R-D) curve fitting, BD-PSNR provides a good evaluation of the R-D performance [11] [15]. BD metrics allow to compute the average gain in PSNR or the average per cent saving in bitrate between two rate-distortion curves. However, BD-PSNR has a critical drawback: It does not take the coding complexity into account [11]. 5.3 Implementation Complexity The computational time of HEVC and AVC encoders will be compared and this serves as an indication of implementation complexity. Page 12

6. Profiles used for Comparison The HM 13.0 main profile [5] and JM 18.6 high profile [8] will be used for comparison in this project. 6.1 Main profile in HEVC This profile allows for a bit depth of 8-bits per sample with 4:2:0 chroma sampling, which is the most common type of video used with consumer devices. 6.2 High Profile in H.264 This profile allows for a bit depth of 8-bits per sample with 4:2:0 chroma sampling. It is used for broadcast and disc storage applications, particularly for high-definition television applications (for example, this is the profile adopted by the Blu-ray Disc storage format and the DVB HDTV broadcast service). 7. Test Sequences The following test sequences [23] of various resolutions are used for study of HEVC and H.264 CODECs: Test Sequence Resolution Frame rate (fps) RaceHorses_416x240_30.yuv 416 x 240 30 BasketballDrill_832x480_50.yuv 832 x 480 50 KristenAndSara_1280x720_60.yuv 1280 x 720 60 ParkScene_1920x1080_24.yuv 1920 x 1080 24 RaceHorses_416x240_30.yuv BasketballDrill_832x480_50.yuv KristenAndSara_1280x720_60.yuv ParkScene_1920x1080_24.yuv Figure 14: Test Sequences [23] Page 13

8. Comparison Methodology Test sequences in.yuv format are encoded using HM 13.0 [24] and JM 18.6 [25]. Different resolutions of test sequences are used. The main profile in HM 13.0 and high profile in JM 18.6 are used for comparison. The following sections give brief description of configuration settings used in the encoding tools. 8.1 Configuration of HM 13.0 Main all-intra mode settings IntraPeriod : 1 # Period of I-Frame ( -1 = only first) GOPSize : 1 # GOP Size (number of B slice = GOPSize-1) QP : 22 # Quantization parameter(0-51) (22, 27, 32 or 37 is used at a time) Main random access mode settings IntraPeriod : 64 # Period of I-Frame ( -1 = only first) GOPSize : 8 # GOP Size (number of B slice = GOPSize-1) QP : 22 # Quantization parameter(0-51) (22, 27, 32 or 37 is used at a time) FastSearch : 1 # 0:Full search 1:TZ search SearchRange : 64 # (0: Search range is a Full frame) Command line parameters for using HM 13.0 encoder: TAppEncoder [-h] [-c config.cfg] [--parameter=value] Options: -h Prints parameter usage -c Defines configuration file to use. Multiple configuration files may be used with repeated c options. --parameter=value Assigns value to a given parameter. Sample command line parameters for HM 13.0 encoder: C:\ Documents\HM_13_HEVC\bin\vc10\Win32\Release>TAppEncoder.exe -c encoder_intra_main.cfg -wdt 832 -hgt 480 -fr 50 -f 10 -i BasketballDrill_832x480_50.yuv 8.2 Configuration of JM 18.6 High all-intra mode settings FramesToBeEncoded : 20 # Number of frames to be coded FrameRate : 50.0 # Frame Rate per second (0.1-100.0) ProfileIDC : 100 # Profile IDC (66=baseline, 77=main, 88=extended; FREXT Profiles: 100=High, 110=High 10, 122=High 4:2:2, 244=High 4:4:4, 44=CAVLC 4:4:4 Intra, 118=Multiview High Profile, 128=Stereo High Profile) IntraProfile : 1 # Activate Intra Profile for FRExt (0: false, 1: true) LevelIDC : 40 # Level IDC (e.g. 20 = level 2.0) Page 14

IntraPeriod : 1 # Period of I-pictures (0=only first) IDRPeriod : 1 # Period of IDR pictures (0=only first) QPISlice : 22 # Quant. param for I Slices (0-51) (22, 27, 32 or 37 is used at a time) High random access mode settings FramesToBeEncoded : 20 # Number of frames to be coded FrameRate : 50.0 # Frame Rate per second (0.1-100.0) ProfileIDC : 100 # Profile IDC (66=baseline, 77=main, 88=extended; FREXT Profiles: 100=High, 110=High 10, 122=High 4:2:2, 244=High 4:4:4, 44=CAVLC 4:4:4 Intra, 118=Multiview High Profile, 128=Stereo High Profile) IntraProfile : 0 # Activate Intra Profile for FRExt (0: false, 1: true) LevelIDC : 52 # Level IDC (e.g. 20 = level 2.0) QPISlice, QPPSlice, QPBSlice : 22 # Quant. param for I, P and B Slices (0-51) IntraPeriod : 0 # Period of I-pictures (0=only first) IDRPeriod : 0 # Period of IDR pictures (0=only first) SearchMode : 3 # Motion estimation mode (Enhanced Predictive Zonal Search (EPZS) Command line parameters for using JM 18.6 encoder: lencod [-h] [-d defenc.cfg] {[-f curenc1.cfg]...[-f curencn.cfg]} {[-p EncParam1=EncValue1]...[-p EncParamM=EncValueM]} Options: -h Prints parameter usage. -d Use <defenc.cfg> as default file for parameter initializations. If not used then file defaults to encoder.cfg in local directory. -f Read <curencm.cfg> for resetting selected encoder parameters. Multiple files could be used that set different parameters. -p Set parameter <EncParamM> to <EncValueM>. The entry for <EncParamM> is case insensitive. Sample command line parameters for JM 18.6 encoder: C:\ H.264\01_JM 18.6\JM Software\JM 18.6\JM\bin>lencod.exe f encoder_high.cfg -p InputFile="C:\Users\uma\Desktop\BasketballDrill_832x480_50.yuv" p SourceWidth=832 -p SourceHeight=480 8.3 Testing Platform Processor Intel Core(TM) i3 CPU M370, 2.40 GHz Number of cores 2 Memory 4 GB Operating System 64 bit Windows 7 Home Basic Service Pack 1 Page 15

9. Test Configurations 9.1 Intra-only configuration In the test case for Intra-only coding, each picture in a video sequence is encoded as IDR picture [39]. No temporal reference pictures are used. It is not allowed to change QP during a sequence within a picture. Figure 15 gives graphical presentation of Intra-only configuration. The number associated with each picture represents encoding order. 0 1 2 3 4 5 6 7 8 QPI QPI IDR Picture time Figure 15: Graphical presentation of Intra-only configuration [39] 9.2 Random Access Configuration For the random-access test condition, hierarchical B structure is used for coding [39]. Figure 16 shows graphical presentation of Random-access configuration. The number associated with each picture represents encoding order. Intra picture shall be inserted cyclically per about one second. The first intra picture of a video sequence shall be encoded as IDR picture and the other intra pictures shall be encoded as non-idr intra pictures. The pictures located between successive intra pictures in display order shall be encoded as B-pictures. The GPB picture shall be used as the lowest temporal layer that can refer to I or GPB picture for inter prediction. The second and third temporal layers consists of referenced B pictures, and the highest temporal layer contains non-referenced B picture only. QP of each inter coded picture shall be derived by adding offset to QP of Intra coded picture depending on temporal layer. Reference picture list combination is used for management and entropy coding of reference picture index. 5 6 7 8 Non-referenced B Picture 3 Referenced B 4 Picture QPB L4 =QPI+4 2 0 1 GPB(Generalized P and B) Picture QPB L3 =QPI+3 QPB L2 =QPI+2 QPI IDR or Intra Picture Referenced B Picture QPB L1 =QPI+1 time Figure 16: Graphical presentation of Random Access configuration [39] Page 16

10. Results 10.1 All-Intra Configuration Simulation results of all-intra configuration for the test sequences are tabulated in Tables 1-4: RaceHorses_416x240_30.yuv, Number of frames encoded = 20 HEVC, Main All-intra profile H.264, High All-intra profile QP PSNR in db Bit rate in PSNR in db Bit rate in 22 42.5720 5057.16 38.942 42.5096 6100.91 40.350 27 38.6146 3152.10 31.973 38.425 3846.35 34.613 32 34.9992 1814.89 26.551 34.7295 2238.22 30.114 37 31.9718 979.584 23.452 31.913 1291.26 26.404 Table 1: Results for RaceHorses_416x240_30.yuv sequence in All-Intra Configuration BasketballDrill_832x480_50.yuv, Number of frames encoded = 20 HEVC, Main All-intra profile H.264, High All-intra profile QP PSNR in db Bit rate PSNR in db Bit rate in 22 42.3716 20407.88 137.080 42.2071 27612.22 141.971 27 39.0915 11014.04 121.178 38.6897 15152.94 120.309 32 36.2986 5847.02 98.149 35.9167 8225.54 103.683 37 33.9144 3200.72 84.450 33.6168 4732.34 93.305 Table 2: Results for BasketballDrill_832x480_50.yuv sequence in All-Intra Configuration KristenAndSara_1280x720_60.yuv, Number of frames encoded = 20 HEVC, Main All-intra profile H.264, High All-intra profile QP PSNR in db Bit rate in PSNR in db Bit rate in 22 45.3544 21763.1280 232.462 45.2825 34380.02 45.760 27 43.1139 13024.7280 218.169 42.8397 21592.03 40.806 32 40.6635 7883.6640 188.127 40.2345 13614.43 37.768 37 38.1338 4769.9760 203.367 37.7711 8983.22 35.900 Table 3: Results for KristenAndSara_1280x720_60.yuv sequence in All-Intra Configuration ParkScene_1920x1080_24.yuv, Number of frames encoded = 20 HEVC, Main All-intra profile H.264, High All-intra profile QP PSNR in db Bit rate in PSNR in db Bit rate in 22 42.1277 49000.6272 672.179 42.0305 60931.83 128.042 27 39.3244 26468.7648 541.421 39.0987 33353.53 106.798 32 36.7479 13760.0640 468.735 36.4413 17731.82 94.365 37 34.4159 6776.3232 419.828 34.2227 9554.87 88.086 Table 4: Results for ParkScene_1920x1080_24.yuv sequence in All-Intra Configuration Page 17

Figures 17-20 represent the R-D plots for various test sequences in all-intra mode: 44 R-D plot for RaceHorses Test Sequence 42 40 PSNR(dB) 38 36 34 H.264 HEVC 32 30 0 1000 2000 3000 4000 5000 6000 7000 Bitrate(Kbit/sec) Figure 17: R-D plot for RaceHorses_416x240_30.yuv sequence 43 R-D plot for BasketBallDrill Test Sequence 42 41 40 39 PSNR(dB) 38 37 36 H.264 HEVC 35 34 33 0 0.5 1 1.5 Bitrate(Kbit/sec) 2 2.5 3 x 10 4 Figure 18: R-D plot for BasketballDrill_832x480_50.yuv sequence Page 18

46 R-D plot for KristenAndSara Test Sequence 45 44 43 PSNR(dB) 42 41 40 39 H.264 HEVC 38 37 0 0.5 1 1.5 2 2.5 3 3.5 Bitrate(Kbit/sec) x 10 4 Figure 19: R-D plot for KristenAndSara_1280x720_60.yuv sequence 43 R-D plot for Park Scene Test Sequence 42 41 40 H.264 HEVC PSNR(dB) 39 38 37 36 35 34 0 1 2 3 4 5 6 7 Bitrate(Kbit/sec) x 10 4 Figure 20: R-D plot for ParkScene_1920x1080_24.yuv sequence From Figures 17-20, it can be inferred that for a given QP, HEVC has higher PSNR and lower bitrate when compared to H.264 in all-intra mode and hence HEVC is better than H.264 in all-intra mode. Page 19

BD-PSNR in db BD-Birate % 0-5 -10-15 -20-25 -30-35 -40-45 -50 RaceHorses_416x240 _30.yuv % BD-Bitrate of HEVC and H.264 BasketballDrill_832x4 80_50.yuv KristenAndSara_1280 x720_60.yuv Figure 21: Comparison of HEVC and H.264 based on BD-Bitrate ParkScene_1920x108 0_24.yuv BD-Bitrate (%) -21.7502-34.3692-47.058-28.1203 Test Sequences Figure 21 indicates that the main profile in HEVC has an average BD-bitrate savings of 32.82% over high profile in H.264 in all-intra mode. BD-PSNR Comparison of HEVC and H.264 3.5 3 2.5 2 1.5 1 0.5 0 1.509 RaceHorses_416x240_3 0.yuv 1.9314 BasketballDrill_832x480 _50.yuv 3.2036 KristenAndSara_1280x7 20_60.yuv 1.2485 ParkScene_1920x1080_ 24.yuv BD-PSNR (db) 1.509 1.9314 3.2036 1.2485 Test Sequences Figure 22: BD-PSNR Comparison Figure 22 indicates that main profile in HEVC has an average increase in BD-PSNR of 1.973 db over high profile in H.264 in all-intra mode. Page 20

in Seconds Comparison of between HEVC and H.264 for Race Horses Test Sequence 45 40 35 30 25 20 15 10 5 0 22 27 32 37 Quantization Parameter for HEVC for H.264 Figure 23: Comparison of encoding time in all-intra configuration for RaceHorses_416x240_30.yuv sequence 160 140 120 100 80 60 40 20 Comparison of between HEVC and H.264 for Basket Ball Drill Test Sequence 0 22 27 32 37 Quantization Parameter for HEVC for H.264 Figure 24: Comparison of encoding time in all-intra configuration for BasketballDrill_832x480_50.yuv sequence From Figures 23 and 24, it can be seen that the encoding time for HEVC encoder is slightly less or equal to H.264 encoder in all-intra mode. Page 21

Encoding Time Encoding Time in Seconds Comparison of between HEVC and H.264 for KristenAndSara Test Sequence 250 200 150 100 50 0 22 27 32 37 Quantization Parameter for HEVC for H.264 Figure 25: Comparison of encoding time in all-intra configuration for KristenAndSara_1280x720_60.yuv sequence Comparison of between HEVC and H.264 for Park Scene Test Sequence 800 700 600 500 400 300 200 100 0 22 27 32 37 Quantization Parameter for HEVC for H.264 Figure 26: Comparison of encoding time in all-intra configuration for ParkScene_1920x1080_24.yuv sequence From Figures 25 and 26, it can be seen that the encoding time for HEVC is significantly greater than H.264 for higher resolution sequences and indicates that HEVC encoder is more complex than H.264 encoder in all-intra mode. Page 22

10.2 Random Access Configuration Simulation results of random access configuration for the test sequences are tabulated in Tables 5-7: RaceHorses_416x240_30.yuv, Number of frames encoded = 20 HEVC, Main Profile, Random Access H.264, High Profile, Random Access QP PSNR in db Bit rate in PSNR in db Bit rate in 22 39.7490 1505.02 168.766 40.2073 2049.46 114.218 27 36.1523 769.05 142.296 36.4668 1063.16 108.427 32 33.0258 392.10 121.389 33.7828 628.01 107.418 37 30.4964 203.61 114.431 30.7851 317.75 103.331 Table 5: Results for RaceHorses_416x240_30.yuv sequence in Random Access Configuration BasketballDrill_832x480_50.yuv, Number of frames encoded = 20 HEVC, Main Profile, Random Access H.264, High Profile, Random Access QP PSNR in db Bit rate PSNR in db Bit rate in 22 41.4220 3479.84 483.657 41.8238 5909.52 373.645 27 38.4933 1698.98 404.454 38.6733 2881.68 363.388 32 35.7846 841.90 352.771 35.8981 1448.78 341.996 37 33.5674 447.10 323.220 33.3300 742.12 320.481 Table 6: Results for BasketballDrill_832x480_50.yuv sequence in Random Access Configuration KristenAndSara_1280x720_60.yuv, Number of frames encoded = 20 HEVC, Main Profile, Random Access H.264, High Profile, Random Access QP PSNR in db Bit rate in PSNR in db Bit rate in 22 44.4383 2676.0000 703.941 44.2062 3740.76 459.124 27 42.6707 1227.7680 629.886 42.2390 1727.02 431.474 32 40.6380 666.7920 600.960 39.9612 922.97 408.706 37 38.3784 381.4800 573.958 37.5176 536.57 404.473 Table 7: Results for KristenAndSara_1280x720_60.yuv sequence in Random Access Configuration ParkScene_1920x1080_24.yuv, Number of frames encoded = 20 HEVC, Main Profile, Random Access H.264, High Profile, Random Access QP PSNR in db Bit rate in PSNR in db Bit rate in 22 40.7442 8410.77 2217.322 41.1195 28723.88 1442.802 27 38.4090 3671.82 1842.406 38.2227 10391.96 1293.122 32 36.1320 1698.69 1649.413 36.3136 5724.94 1241.135 37 34.0683 792.30 1535.159 33.9822 2732.38 1741.023 Table 8: Results for ParkScene_1920x1080_24.yuv sequence in Random Access Configuration Page 23

Figures 27-30 represent the R-D plots for various test sequences in random access mode: 42 R-D plot for Race Horses Test Sequence 40 38 PSNR(dB) 36 34 H.264 HEVC 32 30 200 400 600 800 1000 1200 1400 1600 1800 2000 2200 Bitrate(Kbit/sec) Figure 27: R-D plot for RaceHorses_416x240_30.yuv sequence 42 R-D plot for BasketBallDrill Test Sequence 41 40 39 PSNR(dB) 38 37 H.264 HEVC 36 35 34 33 0 1000 2000 3000 4000 5000 6000 Bitrate(Kbit/sec) Figure 28: R-D plot for BasketballDrill_832x480_50.yuv sequence Page 24

45 R-D plot for KristenAndSara Test Sequence 44 43 42 PSNR(dB) 41 40 39 H.264 HEVC 38 37 0 500 1000 1500 2000 2500 3000 3500 4000 Bitrate(Kbit/sec) Figure 29: R-D plot for KristenAndSara_1280x720_60.yuv sequence 42 R-D plot for Park Scene Test Sequence 41 40 39 PSNR(dB) 38 37 36 H.264 HEVC 35 34 33 0 0.5 1 1.5 Bitrate(Kbit/sec) 2 2.5 3 x 10 4 Figure 30: R-D plot for ParkScene_1920x1080_24.yuv sequence From Figures 27-30, it can be inferred that for a given QP, HEVC has higher PSNR and lower bitrate when compared to H.264 in random access mode and hence HEVC is better than H.264 in random access mode. Page 25

BD-PSNR in db %BD-Bitrate % BD-Bitrate of HEVC and H.264 0-10 -20-30 -40-50 -60-70 -80 RaceHorses_416x240 _30.yuv BasketballDrill_832x 480_50.yuv KristenAndSara_128 0x720_60.yuv ParkScene_1920x108 0_24.yuv BD-Bitrate(%) -25.0485-37.876-40.5843-66.8356 Test Sequences Figure 31: Comparison of HEVC and H.264 based on BD-Bitrate From Figure 31, it can be seen that main profile in HEVC has an average BD-bitrate savings of 42.58% over high profile in H.264 in random access mode. 3.5 3 BD-PSNR of HEVC and H.264 2.5 2 1.5 1 0.5 0 RaceHorses_416x240 _30.yuv BasketballDrill_832x 480_50.yuv KristenAndSara_128 0x720_60.yuv ParkScene_1920x108 0_24.yuv BD-PSNR (db) 1.3663 1.9692 1.671 3.2428 Test Sequences Figure 32: Comparison of HEVC and H.264 based on BD-PSNR From Figure 32, it can be seen that main profile in HEVC has an average increase in BD-PSNR of 2.062 db over high profile in H.264 in random access mode. Page 26

Comparison of of HEVC and H.264 for Race Horses Test sequence 180 160 140 120 100 80 60 40 20 0 22 27 32 37 Quantization Parameter for HEVC for H.264 Figure 33: Comparison of encoding time in random access configuration for RaceHorses_416x240_30.yuv sequence Comparison of of HEVC and H.264 for BasketBallDrill Test sequence 600 500 400 300 200 100 0 22 27 32 37 Quantization Parameter for HEVC for H.264 Figure 34: Comparison of encoding time in random access configuration for BasketballDrill_832x480_50.yuv sequence Page 27

Comparison of of HEVC and H.264 for KristenAndSara test sequence 800 600 400 200 0 22 27 32 37 Quantization Parameter for HEVC for H.264 Figure 35: Comparison of encoding time in random access configuration for KristenAndSara_1280x720_60.yuv sequence Comparison of of HEVC and H.264 for ParkScene test sequence 2500 2000 1500 1000 500 0 22 27 32 37 Quantization Parameter for HEVC for H.264 Figure 36: Comparison of encoding time in random access configuration for ParkScene_1920x1080_24.yuv sequence From Figures 33-36, it can be seen that the encoding time for HEVC is significantly greater than H.264 for higher resolution sequences and indicates that HEVC encoder is more complex than H.264 encoder in random access mode. 11. Further Work Exploration of topics such as using parallel processing architectures with HEVC, transcoding between HEVC and H.264 video codecs can be done. Implementation for calculation of SSIM in HM13.0 or later can be explored. Page 28

12. References [1] I.E.G. Richardson, Video Codec Design: Developing Image and Video Compression Systems, Wiley, 2002. [2] K.R. Rao, D.N. Kim and J.J. Hwang, Video Coding Standards: AVS China, H.264/MPEG-4 Part 10, HEVC, VP6, DIRAC and VC-1, Springer, 2014. [3] I.E.G. Richardson, H.264 and MPEG-4 Video Compression, Hoboken, NJ, Wiley, 2003. [4] I.E.G. Richardson, The H.264 advanced video compression standard, 2 nd Edition, Hoboken, NJ, Wiley, 2010. [5] G. J. Sullivan et al, Overview of the High Efficiency Video Coding (HEVC) Standard, IEEE Transactions on Circuits and Systems for Video Technology, Vol. 22, No. 12, pp. 1649-1668, Dec. 2012. [6] F. Bossen et al, HEVC Complexity and Implementation Analysis, IEEE Transactions on Circuits and Systems for Video Technology, Vol. 22, No. 12, pp. 1685-1696, Dec. 2012. [7] J. R. Ohm et al, Comparison of the Coding Efficiency of Video Coding Standards Including High Efficiency Video Coding (HEVC), IEEE Transactions on Circuits and Systems for Video Technology, Vol. 22, No. 12, pp. 1669-1684, Dec. 2012. [8] T. Wiegand et al, Overview of the H.264/AVC Video Coding Standard, IEEE Transactions on Circuits and Systems for Video Technology, Vol. 13, No. 7, pp. 560-576, Jul. 2003. [9] D. Marpe et al, The H.264/MPEG4 advanced video coding standard and its applications, IEEE Communications Magazine, Vol. 44, pp. 134-143, Aug. 2006. [10] A. Puri et al, Video coding using the H.264/MPEG-4 AVC compression standard, Signal Processing: Image Communication, vol. 19, pp. 793-849, Oct. 2004. [11] X. Li et al, Rate-complexity-distortion evaluation for hybrid video coding, IEEE International Conference on Multimedia and Expo (ICME), pp. 685-690, Jul. 2010. [12] B. Bross et al, High Efficiency Video Coding (HEVC) Text Specification Draft 10, Document JCTVC- L1003, ITU-T/ISO/IEC Joint Collaborative Team on Video Coding (JCT-VC), Mar. 2013 available on http://phenix.it-sudparis.eu/jct/doc_end_user/current_document.php?id=7243 [13] JVT Draft ITU-T recommendation and final draft international standard of joint video specification (ITU-T Rec. H.264-ISO/IEC 14496-10 AVC), March 2003, JVT-G050 available on http://ip.hhi.de/imagecom_g1/assets/pdfs/jvt-g050.pdf [14] J. Vanne et al, Comparative Rate-Distortion-Complexity Analysis of HEVC and AVC Video Codecs, IEEE Transactions on Circuits and Systems for Video Technology, Vol. 22, No. 12, pp. 1885-1898, Dec. 2012. [15] G. Bjontegaard, Calculation of Average PSNR Differences between RD Curves, document VCEG- M33, ITU-T SG 16/Q 6, Austin, TX, Apr. 2001. Page 29

[16] D. Grois et al, Performance Comparison of H.265/ MPEG-HEVC, VP9, and H.264/ MPEG-AVC Encoders, available on: http://iphome.hhi.de/marpe/download/performance_hevc_vp9_x264_pcs_2013_preprint.pdf [17] HEVC tutorial by I.E.G. Richardson: http://www.vcodex.com/h265.html [18] H.264 tutorial by I.E.G. Richardson: https://www.vcodex.com/h264.html [19] HEVC white paper-ittiam Systems: http://www.ittiam.com/downloads/en/documentation.aspx [20] HEVC white paper-ateme: http://www.ateme.com/an-introduction-to-uhdtv-and-hevc [21] HEVC white paper-elemental Technologies: http://www.elementaltechnologies.com/lp/hevch265-demystified-white-paper [22] White paper on PSNR-NI: http://www.ni.com/white-paper/13306/en/ [23] Test Sequences: ftp://ftp.tnt.uni-hannover.de/testsequences/ [24] Access to HM 13.0 Reference Software: http://hevc.hhi.fraunhofer.de/ [25] Access to JM 18.6 Reference Software: http://iphome.hhi.de/suehring/tml/ [26] Website on PSNR: http://en.wikipedia.org/wiki/peak_signal-to-noise_ratio [27] Website on SSIM: http://en.wikipedia.org/wiki/structural_similarity [28] Access the website http://www-ee.uta.edu/dip/courses/ee5359/ and refer to the project by S. Kulkarni on Transcoding from H.264/AVC to High Efficiency Video Coding (HEVC), University of Texas, Arlington, Spring 2013. [29] Access the website http://www-ee.uta.edu/dip/courses/ee5359/ and refer to the project by H. B. Jain on Comparative and performance analysis of HEVC and H.264 intra frame coding and JPEG 2000, University of Texas, Arlington, Spring 2013. [30] Access the website http://www-ee.uta.edu/dip/courses/ee5359/ and refer to the thesis by S. Vasudevan on Fast intra prediction and fast residual quad-tree encoding implementation in HEVC, University of Texas, Arlington, 2013. [31] C. Fogg, Suggested figures for the HEVC specification, ITU-T / ISO-IEC Document: JCTVC J0292r1, July 2012. [32] Z. Wang et al, Image Quality Assessment: From Error Visibility to Structural Similarity, IEEE Transactions on Image Processing, Vol. 13, No. 4, pp. 600-612, Apr. 2004. [33] G. Sullivan et al, Standardized Extensions of High Efficiency Video Coding (HEVC), IEEE Journal of selected topics in Signal Processing, Vol. 7, No. 6, pp. 1001-1016, Dec. 2013. [34] N. Ahmed, T. Natarajan and K.R. Rao, Discrete Cosine Transform, IEEE Transactions on Computers, Vol. C-23, pp. 90-93, Jan. 1974. [35] Special Issue on emerging research and standards in next generation video coding, IEEE Transactions on Circuits and Systems for Video Technology (CSVT), Vol.22, pp. 1646-1909, Dec. 2012. [36] Special Issue on emerging research and standards in next generation video coding, IEEE Transactions on Circuits and Systems for Video Technology (CSVT), Vol.23, pp. 2009-2142, Dec. 2013. Page 30

[37] Introduction to the Issue on Video Coding: HEVC and Beyond, IEEE Journal of Selected Topics in Signal Processing, Vol.7, pp.931-1151, Dec. 2013. [38] S. Vasudevan and K.R. Rao, Combination method of fast HEVC Encoding, IEEE ECTI-CON-2014, Korat, Thailand, May 2014. [39] HM Encoder Description: http://mpeg.chiariglione.org/standards/mpeg-h/high-efficiency-videocoding/high-efficiency-video-coding-hevc-test-model-9-hm-9 [40] H.264/AVC Software Reference Manual: http://iphome.hhi.de/suehring/tml/jm%20reference%20software%20manual%20(jvt-ae010).pdf Page 31