High Efficiency Video Coding: The Next Gen Codec Matthew Goldman Senior Vice President TV Compression Technology Ericsson
High Efficiency Video Coding Compression Bitrate Targets Bitrate MPEG-2 VIDEO 1994 50% bitrate saving target AVC 2003 50% bitrate saving target HEVC 2013 1995 2005 2015 Ericsson AB 2013 2013-04-03 Page 2
HEVC A new standardized compression algorithm An evolution of AVC (H.264 MPEG-4 Part 10) HEVC standardization A Joint Collaborative Team on Video Coding (JCT-VC) of MPEG & VCEG Aim: To deliver same picture quality for half the bitrate of AVC Up to 10x more computational complexity to encode and 2x-3x to decode Standards nomenclature: ISO/IEC 23008-2 MPEG-H Part 2 and ITU-T Rec. H.265 Key dates January 2013 Final Draft International Standard (Version 1) Main Profile, Main 10 Profile, and Main Still Profile approved VCEG consented; Final text available March 1 MPEG to issue ballot for ratification January 2014 FDAM Range extensions (Contribution applications) & Multi-view extensions July 2014 FDIS Scalable HEVC (SHVC) Ericsson AB 2013 2013-04-03 Page 3
HEVC Encoder Headers ENCODER CONTROL Source Ref. Buffer Intra ALF Recon Prediction SAO T -1 DF Q -1 ENTROPY CODER Bitstream ME/MC Mode Dec T Q Residuals Ericsson AB 2013 2013-04-03 Page 4
High Level Tool Comparison AVC 16X16 block size Various Inter partitions down to 4x4 9 intra modes HEVC 64x64 block size Hierarchical quad-tree partitioning down to 8x8 + 4x4 Transform Units 35 intra modes 8x8 and 4x4 transform sizes 32x32, 16x16, 8x8, 4x4 transform sizes Ericsson AB 2013 2013-04-03 Page 5
Coding Tree Coding Tree is collection of Coding Units (CU) CU size 64x64 to 8x8 CUs can have independent coding modes Further partitioning using Prediction Units (Motion Vectors) Independent Transform Tree partitioning from 32x32 to 4x4 Ericsson AB 2013 2013-04-03 Page 6
CUs: Prediction & Transform Units Source: JCTVC-A124 Separation of prediction and transform structures allows more flexible and efficient coding of video under various conditions and resolutions Ericsson AB 2013 2013-04-03 Page 7
HEVC Tools Intra Prediction AVC DC + 8 directional modes HEVC DC + Planar + 33 directional modes Ericsson AB 2013 2013-04-03 Page 8
HEVC Tools In-Loop Filters Deblocking Filter Similar to AVC deblocking filter but does not filter 4x4 block edges Sample Adaptive Offset (SAO) Filter Calculates edge and band offsets signaled to decoder Offsets added to reconstructed pixels SAO is not restricted to block boundaries No In-loop filters Deblocking filter + SAO filter Ericsson AB 2013 2013-04-03 Page 9
Tool Comparison: AVC HP vs. HEVC MP AVC High Profile 16x16 Macroblock Partitions 16x16 to 4x4 square + non-square (inter) 8x8 and 4x4 transforms Intra prediction (8 directions + DC) Inter prediction luma 6-tap + 2-tap, to ¼ pel Inter prediction chroma bi-linear interpolation Motion vector prediction In-loop deblocking filter CABAC or CAVLC HEVC Main Profile Coding Unit quadtree structure, 64x64 down to 8x8 Prediction Units, 64x64 to 8x8 square + non-square (inter) + asymmetric (inter) Transform Units, 32x32 to 4x4, 4x4 skip Intra prediction (33 directions + DC + planar) Inter prediction luma 8-tap, to ¼ pel Inter prediction chroma 4-tap. to 1/8 pel Advanced motion vector prediction (spatial + temporal) In-loop deblocking filter & Sample Adaptive Offset (SAO) filter CABAC using parallel operations Ericsson AB 2013 2013-04-03 Page 10 CABAC = Context Adaptive Binary Arithmetic Coding CAVLC = Context Adaptive Variable Length Coding
HEVC Potential For DTH, HEVC will give 40-50% saving over best AVC encoder today HD: AVC 6-9 Mbps HEVC 3-4.5 Mbps SD: AVC 1.5-2.5 Mbps HEVC 0.8-1.5 Mbps UHD 4K: AVC 16-24 Mbps (estimate) HEVC 8-12 Mbps Fits in existing channel bandwidth of currently deployed HD! HEVC practical availability Driven by availability of receive devices Software-based implementations available now; address a subset of applications First production silicon available during 2013 first practical systemization 1H14 Implementation issues (impacts practical ability to deploy) Migration of deployed (legacy) vs. greenfield Intellectual property licensing needs to be settled MPEG LA UHDTV (4K) uncertainty: Main 10 Profile? Higher frame rate? Better interlaced content support? Contribution market Professional profiles ( RExt, e.g., 4:2:2) planned Jan. 2014 Ericsson AB 2013 2013-04-03 Page 11
Industry Drivers - Efficiency Mobile TV Expensive bandwidth Increasing demand Multi-screen More HD More screens xdsl reach More subscribers DSNG More HD Expensive bandwidth UHDTV High bit-rate need Terrestrial Broadcast Spectrum efficiency Satellite Distribution Spectrum efficiency Ericsson AB 2013 2013-04-03 Page 12
HEVC Version 1 Contains 3 Profiles Main Profile supports general consumer-grade video (4:2:0 8-bit) Main 10 Profile targets large screen consumer applications that require higher resolution Main Profile + bit-depth up to 10 bits UHDTV and large screens Not for professional applications (contribution, mastering) Still Picture Profile targets specialty applications Strict sub-set of Main Profile One IDR picture Some practical concerns still remain Ericsson AB 2013 2013-04-03 Page 13
Interlaced Tools Disagreements De-interlace prior to encoding Shown to give great efficiency results Doesn t fit all usage scenarios DTA, embedded transcoding Billions of pre-encoded MPEG-2 Video & AVC source content PAFF/MBAFF (AVC) vs. SAFF/Field Only Coding/Frame Only Coding (HEVC) What about PAFF without MBAFF? Chroma bleeding in 4:2:0 (no chroma motion vector offset when predicting top-to-bottom / bottom-to-top field as AVC) WG11 has asked for more evidence before proceeding further Ericsson AB 2013 2013-04-03 Page 14
Ericsson AB 2013 2013-04-03 Page 15 Resolution revolution
What is UHDTV(4K)? 4x spatial resolution of HDTV 3840 x 2160 YCrCb (4:2:2 10bit) SDTV HDTV 4KTV Film OCN Ericsson AB 2013 2013-04-03 Page 16
Ericsson AB 2013 2013-04-03 Page 17 UHDTVs Now Available
Visual Perception - Resolution 0.01 1.3x to 2.6x diagonal 84 TV 9 to 18' for 84 diagonal Ericsson AB 2013 2013-04-03 Page 18
UHDTV Key Consumer Value High-quality movies Popular sports events Ericsson AB 2013 2013-04-03 Page 19
UHDTV Short Term Challenges HDMI 1.4 does not support 4Kp60 4 x 3G-SDI Production mixers Graphics Live cameras Set-top boxes for DTH Ericsson AB 2013 2013-04-03 Page 20
On What Format Will Industry Settle For 4K UHDTV? 4K HEVC could require up to 80x more horsepower vs. HD AVC SDTV HDTV 4KTV P30 8b 4:2:0 4KTV P60+ 10b 4:2:2 Film OCN Ericsson AB 2013 2013-04-03 Page 21
Driving UHDTV Content & Events Movies 2014 TV dramas Live sports Ericsson AB 2013 2013-04-03 Page 22
Contribution: 4:2:2 vs. 4:2:0 Chroma samples 4:2:0 production flow 4:2:0 4:2:2 4:2:2 production flow Chroma rate = ¼ Luma rate Saves 50% bandwidth Chroma rate = ½ Luma rate Saves 33% bandwidth Keying with 4:2:0 Source: http://www.dv.com/dv/magazine/2006/november/dv0611.hdvout.fig4.jpg Ericsson AB 2013 2013-04-03 Page 23
10-Bit Quantization 8 bit 10 bit 8 bit = 0 to 255 10 bit = 0 to 1023 Ericsson AB 2013 2013-04-03 Page 24
HEVC Range Extensions HEVC range extensions (RExt) will be an Annex to HEVC v1 4:2:2/4:4:4/RGB chroma formats Higher bit-depths, including 10/12/14 Mixed Chroma formats (4:2:0 & 4:4:4) mixed video (4:2:0) and computer graphics (4:4:4) Lossless Coding As of January 2013 meeting cycle Use square transform and square intra prediction Intra prediction angle adjustment for 4:2:2 Software to support both square and non-square transforms Timeline DAM July 2013 FDAM January 2014 But, schedule may slip out due to efficiency concerns Ericsson AB 2013 2013-04-03 Page 25
Comparison of AVC-I and HEVC-I vs. SStP RGB 4:4:4 YCbCr 4:2:2 Sequence name Target BD-Rates of AVC vs SStP BD-Rates of HEVC vs SStP BD-Rates of HEVC vs AVC Y/G SNR Y/G Cb/B Cr/R Y/G Cb/B Cr/R Y/G Cb/B Cr/R Kimono 56.72-12.57-12.73-12.60-14.44-14.66-14.43-2.14-2.21-2.09 EBULupoCandlelight 59.03-15.42-15.48-15.41-15.96-15.88-15.90-0.63-0.48-0.57 EBURainFruits 58.85-13.96-14.00-13.72-17.06-17.15-16.24-3.61-3.67-2.93 VenueVu 63.78 QP < 1 required QP < 1 required QP < 1 required DucksAndLegs 60.93 20.51 7.90 20.49 3.82-7.61-1.63-13.64-14.37-18.36 OldTownCross 52.48-12.84-12.94-12.91-15.18-14.37-14.35-2.69-1.65-1.66 ParkScene 54.74-12.13-12.28-12.17-14.80-14.60-14.35-3.03-2.65-2.49 Average sequence 56.71-10.68-11.31-10.65-14.81-15.48-15.08-4.63-4.70-4.96 Kimono 54.86-17.29-14.02-14.14-18.64-18.78-18.66-1.63-5.54-5.26 EBUHorse 50.14-17.13-12.90-10.94-21.81-18.47-16.47-5.65-6.40-6.21 EBUGraphics 51.4-18.84-15.34-15.37-25.13-22.23-22.20-7.75-8.15-8.07 EBUWaterRocksClose 50.96-17.08-12.94-12.46-21.35-18.10-17.77-5.16-5.92-6.07 EBUKidsSoccer 48.78-18.74-13.87-14.01-22.34-17.61-17.71-4.43-4.34-4.30 Seeking 50.34-17.11-13.15-13.25-20.99-18.13-18.19-4.68-5.73-5.70 Average sequence 50.73-17.87-13.92-13.44-22.34-19.38-18.90-5.43-6.35-6.31 Only 5% improvement over AVC! Ericsson AB 2013 2013-04-03 Page 26
Possible Profiles and Conflicting Goals Possible profiles Main-tier (lower bitrates) Simple (few changes) vs. vs. High-tier ([very] high [intra] Complex (new tools) bitrates) Mixed Content Consumer lower bitrates, few changes over v1 Professional high bitrates, new tools for performance gains, divergence from v1 Mixed content more applicable to consumer applications (e.g., gaming) but requires new tools (divergence from v1) Is consumer profile a subset of Professional or can there be a divergence at a tool level ( i.e., mutually exclusive transform designs)? Ericsson AB 2013 2013-04-03 Page 27
4K UHD Contribution using AVC UHD/4K display 4K source 4x 3G-SDI or fiber 4x AVC encoders synchronized 4x AVC decoders phase-lock synchronization Ericsson AB 2013 2013-04-03 Page 28
Scaleable HEVC (SHVC) Annex F Annex G Add new reference pictures for Scaleable and Multi-view Extensions Motion compensation on up-sampled decoded base layer pictures Annex H New CU level coding tool for Inter layer prediction Ericsson AB 2013 2013-04-03 Page 29