MPEG-2 And Scalability Support Nimrod Peleg Update: July.2004
MPEG-2 Target...Generic coding method of moving pictures and associated sound for...digital storage, TV broadcasting and communication... Dedicated for high quality services e.g. HDTV, Networked data services etc. Eveloped from CCIR-601, CCIR-648, JPEG, H.261 and MPEG-1 former standards
MPEG-2 Btstream structure Sequence Sequence Sequence Seq. Hdr. Seq. Ext. Ext.&User GOP GOP GOP Header Ext.&User Picture Data Picture Data Pic. Hdr. Pic. Coding Ext. Ext.&User Slice Slice Slice Header Macroblock Macroblock Macroblock Hdr. Block Block Block
Scalability Modes SPATIAL SCALABILITY TEMPORAL SCALABILITY SNR SCALABILITY DATA PARTITIONING
Spatial Scalability Support for different picture resolutions in a single video stream Base layer is coded at a low spatial resolution. Each enhancement layer gives a progressively higher spatial resolution Example: support for standard TV and HDTV
Temporal Scalability Support for different temporal resolutions (frame rate) in a single video stream Base layer is encoded at a low frame rate. Enhancement layers provide higher frame rates Base layer Enhancement layer
SNR Scalability Allows handling of at least two different video qualities Base layer includes coarsely quantized DCT coefficients. Each enhancement layer further quantizes the base layer quantization error Base layer data VLD Q -1 IDCT MC Video out Enh. layer data Refinement of DCT coeffs VLD Q -1 IDCT MC Video out
Data Partitioning The base layer contains the most critical components, such as header information, motion vectors and (optionally) low-frequency DCT coefficients The enhancement layer contains all remaining coded data (usually less critical to successful decoding)
Data Partitioning (Cont d) The bitsteam is split into 2 layers: partition0, partition1 The Priority Breakpoint (in sequence header) indicates which syntax elements are placed in partition0 which is the base or high priority partition
Data Partitioning (Cont d) The remainder of the bitstream is placed in partition1 which is the low priority partition Seq.,GOP,picture and slice headers are duplicated from partition0 to partition1 VBV refers to the sum of the 2 partitions Partition0 contains sequence scalable extension
Data Partitioning (Cont d) No Data Partitioning Quant Scale DC coeff DCT coeff1 DCT coeff2 DCT coeff3 EOB DC coeff DCT coeff1 EOB Partitions for Priority Break Point 64: Quant Scale DC coeff DCT coeff1 DC coeff DCT coeff1 Partition0 DCT coeff2 DCT coeff3 EOB EOB Partition1
Data Partitioning Possible Breakpoints: After slice header (Cont d) After macroblock address increment (macroblock header) Before coded block pattern (after Motion Vector) After any number of DCT coefficients (excluding one)
SNR Scalability Refinement of the DCT coeffs. encoded in base layer by the enhancement layer coeffs. Base layer contains coarser quantization then enhancement layer Only non intra quant. matrices are used in the enhancement layer Enhancement layer contains coded refinement DCT coeffs. and a small overhead
SNR Scalability (Cont d) DCT coeffs. in base layer are added to DCT coeffs. in enhancement layer The combined layer decoding process is identical to decoding of a non-scalable bitstream Different rate control for the 2 layers
SNR Scalability Scheme Lower Layer Code VLC Inverse Scan Inverse Quant Enhancement Layer Code + VLC Inverse Scan Inverse Quant Inverse DCT Motion Compensation Decoded Samples Frame Storage Memory
Profiles and Levels Each profile defines a set of capabilities, e.g. interlaced video support, B pictures, etc. Each level defines spatial and temporal resolutions Examples: - Main profile at main level (mp@ml) is used for standard TV - Main profile at high level (mp@hl) is used for HDTV
Profiles and Levels (cont d)
Bit Rate Control (BRC) Objectives Smart bit allocation: picture level & MB level Prevent buffer overflow Bit rate control Video frames Encoder Output buffer Channel Variable rate Constant rate
BRC Test Model 5 Frame level target bit allocation Frame type, remaining bits in the GOP, previous picture complexity
BRC Test Model 5 (cont d) MB level buffer monitoring - Choose quantizer step size to meet the target frame rate MB level adaptive quantization Current frame MB activity map (logarithmic scale)