Professor Laurence S. Dooley School of Computing and Communications Milton Keynes, UK
How many bits required? 2.4Mbytes 84Kbytes 9.8Kbytes 50Kbytes
Data Information Data and information are NOT the same! Data is the way information is conveyed Compression reduces the amount of data to represent a given quantity of information while preserving as much info as possible
Why compress? Uncompressed images require high data storage and transmission bandwidth 512 x 512 grayscale image: 262KB 512 x 512 colour image: 786KB Bit-rate for uncompressed digital cinema data >1Gbps!!
Compression Approaches Lossless Information preserved Low compression ratios (2 3) Lossy Does not preserve information Much higher compression ratios (40+) Trade-off: Quality vs Compression Ratio
Spatial Redundancy Take advantage of the similarity between most neighbouring pixels SR compression ratios up to 30:1
Temporal Redundancy For video we exploit the similarity between successive frames frame #N #N+1 #N+2 TR compression up to 200:1 for HQ video
Lossy Compression Loss of information unacceptable in a text or program file For A/V information, loss can be acceptable as we does not always perceive small changes Basis of: Joint Photographic Expert Group (JPEG) Motion Picture Expert Group (MPEG)
Lossy Compression NOT inferior to lossless techniques 10:1 compression without perceptible distortion. VGA example Sub-sample pixels to reduce resolution Exploit HVS for higher compression.
JPEG (.jpg) 1. Lossy compression based on the Discrete Cosine Transform (DCT) 2. Exploits HVS small changes in hue perceived less than small changes in brightness 3. Degree of loss varies with desired quality 4. Small file sizes archiving and thumbnails
JPEG (.jpg) 5. High compression leads blocky appearance 48Kbytes 3Kbytes
JPEG 3 Stages Discrete Cosine Transform (DCT) Quantisation Compression (Coding)
JPEG Stage 1 Divide image in 8 x 8 pixel macroblocks (MB) MB MB MB
The DCT Reversible (lossless) Fourier Transform Transforms 8 x 8 pixel MB into 64 spatial frequency coefficients. dc coefficient is average gray value of all 64 pixels ac coefficients reflect pixel changes Small changes in neighbouring pixels = lowvalued ac coefficients
Observation Most energy in a typical image or frame is concentrated at low ac coefficients Original image/ frame DCT t x
increasing vertical frequencies DCT basis functions dc coefficient increasing horizontal frequencies ac coefficients MB size N=4 Take an JPEG image and zoom-in to see these block structures
DCT example Consider the following picture DCT Only 4 non-zero coefficients (CR=4) so to reconstruct the image use the 4 corresponding DCT basis functions
DCT example
JPEG Stage 2 Many ac coefficients =0 or 0 The rest are quantised using a Q-Table Q-Table reflects HVS in weighting each ac coefficient Fine quantising for low frequencies fewer bits for high frequencies (not so visible)
JPEG Stage 3 Quantised values read in zigzag pattern Puts ac coefficients in ascending horizontal and vertical frequency order in bitstream dc term coded separately good savings as they contain large % of image energy Many values = 0 so apply Huffman Coding
JPEG Input Image D C T 8x8 MB Q u a n t i s e r ac zigzag dc difference RLC Huffman Coding Q-Table See DCT examples
JPEG Performance 8 bits for luminance Y 16bpp 16 bits for U & V (sampled at Y rate) 2 bpp Quality CR 2 Identical 8:1 1.5 Excellent 11:1 0.75 Very Good 21:1 0.5 Good 32:1 0.25 OK 64:1
100% BMP
2:1
25:1
50:1
JPEG Summary Lossy image compression standard Very good performance >0.25bpp Best for colour/gray-scale images Visual artefacts at higher compression JPEG2000 superior at 0.25bpp BUT still not yet widely accepted!!
The 3 rd Dimension video from the Latin I See
What is Motion? 1 16 32 48 64 80 96 112 128 144 160 176 192 208 224 Motion describes the dynamic content in a video Previous position 240 1 16 32 48 64 80 96 112 128 144 160 176 192 208 224 240 256 272 288 304 320 336 352 Current Current position position 240 1 16 32 48 64 80 96 112 128 144 160 176 192 208 224 240 256 272 288 304 320 336 352 Camera (Global) motion depends on the pan & zoom of the camera Object (Local) motion depends upon only an object s velocity and projection angle 1 16 32 48 64 80 96 112 128 144 160 176 192 208 224
Temporal Redundancy Difference between successive frames often small Frame #1 Frame #2 Aim: remove inter-frame redundancy
Temporal Difference Frame # 69 Frame # 70 Frame Differential Frame difference shows presence of both object and camera motion
Motion Picture Expert Group 1 6 10...... 10 frame GIF sequence showing a cricketer playing a defensive stroke While the ball, bat and man move, the rest stays the same. M-JPEG does NOT exploit inter-frame redundancy
Motion Picture Expert Group Uses motion vectors (MV) for inter-frame compression Predict motion between frames to obtain MV Then only encode what changes Different frame types: I, P & B JPEG still used for intra-frame compression
Motion Vectors Let frame #1 be encoded using JPEG What is the best way to encode frame #2? Send another JPEG frame not very efficient Use Motion Prediction motion vectors frame #1 frame #2
Finding a MV Example Foreman MB Occlusion Motion Vector (MV) Frame i-1 Reference frame (available) matching MB Frame i Current frame (to be encoded)
Motion Prediction P-frame I-frame MV defines how to move from a position in the I frame to a new position in the P frame. MV has horizontal & vertical components in the range -64 +63
Motion Prediction Changes between frames are not always simple displacement of pixels Displacement and rotation I-frame P-frame Use prediction error compensation
Motion Compensation Add MV to each pixel (x, y) co-ods I-frame MV + forward prediction + +/- error to the relevant pixel colour value Prediction error compensation Final P-frame
Frame #1
Frame #2
macroblocks 16 x 16 pixel Frame #2 MB
Select 4 example MB
Best match in Frame #1
MB Predictions Block in Frame #2 Best Match in Frame #1 Difference error Motion Prediction Error = 0
Four MV
Difference error between two frames (many bits) Lower error with motion P & C (fewer bits) Motion P&C example Frame #N-1 note tree position Frame #N MV superimposed global motion visible
H.264: The Leader Advanced Video Coding (AVC) Standard without H.264 with H.264
H.264 features Same quality 50% less MPEG-2 bandwidth Uses an integer transform not the DCT Supports low-bit rate streaming over IP networks Scalability: different spatial, temporal and amplitude (SNR) resolutions in one encoded bitstream Audio uses MPEG-4 AAC
H.264+: The Future Part of MPEG-4/AVC Part 10 Standard for Blu-ray Discs Streaming MM YouTube, itunes Adobe Flash Player TV direct broadcast satellite & cable services Multi-View Video Coding (MVC) Bitstreams representing more than one view Stereoscopic and 3DTV Free viewpoint TV MVC an extension of H.264/AVC Blue-ray 3D standards
http://xgmt.open.ac.uk/laurence_dooley