Multimedia Standards SS 2017 Lecture 5 Prof. Dr.-Ing. Karlheinz Brandenburg Karlheinz.Brandenburg@tu-ilmenau.de Contact: Dipl.-Inf. Thomas Köllmer thomas.koellmer@tu-ilmenau.de 1
Organisational issues Preliminary list of lectures updated version is on the website Tuesday, 17:00, K-Hs1 Thursday, 13:00, K-Hs2 CW* 14 Introduction CW15 Standardization I Standardization II CW16 CW17 CW18 Video Coding I CW19 Video Coding II Video Coding III CW20 CW21 MPEG Systems CW22 Metadata Standards CW23 Psychoacoustic Fundamentals MPEG Audio CW24 Speech Coding CW25 CW26 CW27 MPEG Systems CW28 System Standards I System Standards II Regular Date Alternate Date * ISO 8601 Representation of dates and times ch. 2.2.10 : calendar week number: ordinal number which identifies a calendar week within its calendar year according to the rule that the first calendar week of a year is that one which includes the first Thursday of t hat year and that the last calendar week of a calendar year is the week immediately preceding the first calendar week of the next calendar year 2
Video Coding Standards H.261 MPEG 1 Video MPEG 2 Video 3
ITU-T Recommendation H.261 Standardization: 1990 Field of application: Videotelephony and Videoconferencing via ISDN Only coding of the Video Signal, audio and signaling in further ITU-T recommendations (Overview in Rec. H.200) Video conference systems based on IP networks: Rec. H.261, G.711, and G.721 Also called: p x 64 (p x 64 kbit/s channels) p=1 30, 64 kbit/s 1.92 Mbit/s Realtime compression/decompression in hardware CIF or QCIF 4
H.261 CIF and QCIF No predefinition of a particular video system (625 lines PAL/SECAM or 525 lines NTSC) Own video format Common Intermediate Format (CIF): 288 lines, 352 pixels each Frame rate: 30000/1001=29,97 frames/sec Optional for the decoder Quarter-CIF (QCIF) Smaller resolution: 144 lines, 176 pixels each Mandatory for the decoder No statement concerning the kind of conversion of analog or digital video signals in CIF / QCIF within the standard 5
H.261 CIF and QCIF 6
H.261 - Macroblock Sampling: YCbCr 4:2:2 (similar to CCIR-601) Macroblocks Six 8x8 pixel blocks Quantized with 8 bit according to CCIR-601 Four blocks for Y (luminance matrix) One block for C r (r-chrominance matrix) One block for C b (b-chrominance matrix) Block group Matrix of 11x3 macroblocks Start code in the header of each GOB Resynchronization tags 7
H.261 - Macroblock 8
H.261 - Structure of the Coder Encoder: Source coder Video multiplex coder Transmission components transmission buffer transmission coder Decoder: Receiving decoder Receiving buffer Video multiplex decoder Source decoder 9
H.261 - Structure of the Coder 10
H.261 Source Coder 11
H.261 Source Coder Coding control: Intraframe Interframe Segmentation to blocks of 8x8 pixels Marking if a block is transmitted or not Transmitted blocks are transformed Resulting coefficients are quantized and Variable length coded No criteria for the identification of the blocks to be transmitted Choice may be varied dynamically as part of the coding control strategy 12
H.261 Source Coder IWS software codec Intraframe mode: transmission of all blocks Interframe mode: only transmission of blocks where the difference exceed a threshold Only the requirements for the decoding are described Source coder has to operate in a way, that the image to be transmitted can be reconstructed within defined error limits 13
H.261 Source Coder Prediction Loop filter 14 Motion Compensation (MC) is optional in the encoder Decoder will accept one vector per macroblock Both horizontal and vertical components of the motion vectors have inter values not exceeding +/- 15 (used for all luminance blocks) The motion vector for both color difference blocks is derived by halving the component values of the macroblock vector Separable into one vertical and one horizontal non-recursive filter Coefficients ¼, ½, ¼ At block edges (one of the tabs would fall outside the block): Coefficients 0,1,0 Filter is switched on/off for all six blocks in a macroblock according to the macroblock type
H.261 Source Coder DCT-Transformation Separable 2D-DCT for calculating the coefficients of the blocks (8 x 8 pixels) The arithmetic procedures for computing the transforms are not defined The inverse transform is given including the error tolerance DCT coefficients are clipped to a range -2048 2047 and quantized 15
H.261 Source Coder Quantization Intraframe mode Quantization of the DC coefficients linear quantized with 8 bit Quantization of the AC coefficients with a variable step size in the range 2..64 Interframe mode: Quantization of all coefficients with a variable step size in the range of 2..64 Quantized values (level) in the range -128..127 Level are transmitted using a code with variable bit length and zero suppression The formulas and coding tables necessary for reconstruction are given in H.261 16
H.261 Video Multiplex Coder The video multiplex is arranged in a hierarchical structure with four layers: 17 1. Picture 2. Group of Blocks (GOB) 3. Macroblock (MB) 4. Block Picture layer consists of Picture start code (PSC), 20 bit Picture number Information about the picture status (Type Information, PTYPE) GOB Data Type information, PTYPE, contains flags for: Split screen, document camera, Freeze picture release, source format (QCIF or CIF)
H.261 Video Multiplex Coder 18
H.261 Video Multiplex Coder 19
H.261 Video Multiplex Coder Each picture is devided into groups of blocks (GOBs) One GOB = 1/12 of a CIF = 1/3 of a QCIF Each GOB consists of: 16 bit GOB start code (GBSC) Group number Quantizer information Macroblock data 20
H.261 Video Multiplex Coder 21
H.261 Video Multiplex Coder A GOB consists of 33 Macroblocks Macroblocks are arranged line-by-line in a GOB A macroblock contains Coefficients of luminance and chrominance blocks Information about the block type Where required a block-own quantization step size Motion vector Bit vector (Information about which blocks are transmitted) End of block code (EBO) Macroblocks are not transmitted when they contain no information for that part of the picture 22
H.261 Video Multiplex Coder 23
H.261 Video Multiplex Coder 24
H.261 Video Multiplex Coder 25
H.261 Video Multiplex Coder 26
H.261 Video Multiplex Coder Transmission of the bit stream in groups of 492 bits Stuffing blocks, in case that data to be transmitted isnot available (ignored by the receiver) Structure of a data block: Synchronization bit Data 18 bit BCH checksum (forward error correction) Usage of the checksum is optional for the decoder Frame rate of the input signal: Approx. 30 frames/sec 15, 10, or 7.5 frames/sec by ignoring frames Selection of the frame rate by the user 27
H.261 Video Multiplex Coder Loop: Degree of filling of the buffer for adjusting the data rate (quality of the picture) Adjustment of the data rate by: Adjusting the threshold that controls the transmission of difference blocks in the interframe mode Adjusting the Quantization step size: huge quantization step size lower picture quality entropy coding lower data volume per block Transmission of the macroblock in the intra or inter frame mode (not defined in the standard) Change of a scene intraframe mode (parts of the picture are completely retransmitted) 28
H.261 Coding Control Accumulation of transmission or reconstruction errors Transmission of an intraframe block once in a while for resynchronization Loss of synchronization with the sender: Receiver can request a new intraframe image (fast update request) Receiver freezes the image and waits for a new intraframe Sender sets freeze picture release flag in the image header After reception of an intraframe the receiver resumes operation 29
H.261 Overview 30
H.261 Error Correction Frame 31
H.261 Forward Error Correction Error correcting code Transmitted bit stream contains a BCH (511,493) forward error correction code Use of this by the decoder is optional Generator polynomial g(x) = (x 9 + x 4 + 1) (x 9 + x 6 + x 4 + x 3 +1) Example: For the input data of 01111... 11 (493 bits) the resulting correction parity bits are 011011010100011011 (18 bits) 32
MPEG-1 Video ISO/IEC IS 11172-2 Coding of Moving Pictures and Associated Audio for Digital Storage Media at up to about 1.5 Mbit/s Part 2 Video Unanimously approved in Nov. 1992 Annex A, B, and C are normative Annex D and E are informative Field of application: Video on CD, DAT and Hard Disc 33
Coding process 34
Restrictive Parameter Limits 35
Temporal Workflow of the Coding Intra Frame Coding: The whole picture is coded Inter Frame Coding: The picture is coded comprising the previous and/or following pictures Intra Frames are necessary for Random Access Inter Frames allow higher compression rates Block by block motion compensation to reduce the temporal redundancy Macroblock size: 16 pel x 16 lines Difference signal is DCT transformed and quantized 36
Temporal Workflow of the Coding Three different kinds of frames: I-Frame: Intra-coded pictures P-Frames: Predictive coded pictures B-Frames: bidirectional predictive coded pictures Additionally: D-Frames: I-Frames in low quality for fast forward / backward search 37
Temporal Workflow of the Coding 38
Block Diagram of the Encoder 39
Block Diagram of the Encoder 40
Block Diagram of the Encoder 41
Requirements on the Input Signal Digital input signal Pictures coded as (Y, C r, C b ) 2:1 down sampling of C r and C b Coding order of luminance and chrominance: 0 1 2 3 4 5 Y C b C r 42
Block Diagram of the Decoder 43
Block Diagram of the Decoder 44
Syntax definition for the coded bit stream 45
Group of Pictures (GOP) In the encoder the order of the transmitted pictures is swapped 46
8x8 Quantizer 47
Buffer Size and Delay 48
Specification for Zig-Zag Scan 49
Specification for Variable Length Coding 50
Coding of the Motion Vectors 51
MPEG-2 Video / H.262 ISO / IEC IS 13818 Part 2 ITU-T H.262 Generic coding of motion pictures and associated audio information Part 2: Video Approved in November 1994 Based on MPEG-1 Video principally better quality with lower data rates higher complexity more processing power needed Field of application: studio to home electronics Basic for digital television and DVD typical data rates: 1 8 Mbit/s 52
MPEG-2 Video: Profiles and Levels supports a high range of resolutions, data rates, qualities, etc. divided to profiles and leves: profile@level profiles: selection of tools levels: allowed parameters 4:2:2 profile integral part since 1996 53
MPEG-2 Video: Levels (selection) 54
Next Lectures KW 19 KW 21 Thursday 11.05., 13:00 K-HS2 Tuesday 23.05., 17:00 K-HS1 55