06/12/2017. Image compression. Image compression. Image compression. Image compression. Coding redundancy: image 1 has four gray levels

Transcription

1 Theoretical size of a file representing a 5k x 4k colour photograph: 5000 x 4000 x 3 = 60 MB 1 min of UHD tv movie: 3840 x 2160 x 3 x 24 x 60 = 36 GB 1. Exploit coding redundancy 2. Exploit spatial and temporal redundancy 3. Eliminate irrelevant or imperceptible information Size can be reduced by a factor Coding redundancy: image 1 has four gray levels Bit rate: Fixed-length code: L avg = S k l(r k ) p r (r k ) = 8 bpp 4-levels fixed-length code: L avg = 2 bpp Variable-length code: L avg = 2x x x x0.03 = 1.81 bpp Spatial redundancy: image 2 has the histogram Irrelevant or imperceptible information: image 3 could be represented by 1 byte only (average gray level) Or, it can undergo histogram equalization: Its rows are randomly ordered (vertical correlation = 0) Its columns are identical (horizontal correlation = 1) A run-length code can be used, formed by a sequence of pairs (graylevel, length) The compression ratio is (256 x 256 x 8) / ( ( ) x 8) = 128 I.e. bit rate: 8 bpp 8/128 = bpp Its further coding becomes application-dependent Note that if one decides to discard the "noise" and/or the "structure", the compression becomes irreversible

2 Standard and popular image and video formats

3 Huffman coding (1952) [exploits coding redundancy] For a given source, it yields the smallest number of code symbols per source symbols (among scalar coders; vector coders can do better) Example: (assignment of 0 or 1 at each level is irrelevant) L fixed-length = 3 bpp L avg = 1x x x x x( ) = 2.2 bpp while the entropy of a zero-memory source having this pdf is: H = -S k p r (r k ) log 2 p r (r k ) = 2.14 bpp Note: code strings are uniquely-decodable. E.g.: Note: The code dictionary has to be saved/sent too, or has to be fixed More sophisticated (but sometimes proprietary) coding tools exist, such as Arithmetic coding Moreover, coders can be made adaptive, via context-dependent estimation of the pdf: a codeword is assigned basing on the probability of the symbol, the values of symbols in a neighbourhood of the coded symbol Lempel-Ziv-Welch coding (1984) Choose codebook size; e.g. 512 codewords, 9 bits [exploits spatial redundancy too] Initialize by placing all symbols in the codebook; e.g. [ ] Start scanning; each input sequence that is not in the codebook is placed in the first unused position; e.g., "67,70" 256 From now on, every "67,70" pair found in the source will be coded as 256, i.e. with a 9-bit codeword instead of two 8-bit codewords Example: Image data are

4 C B When scanning ends, nine codewords have been added to the codebook, which now contains = 265 codewords Several repeated sequences are identified The original 128-bit image is represented with 90 bits (ten 9-bit codes) Note: no need for knowledge of source distribution A D Design options: 1. Choose a balanced codebook size too small few sequences are identified; too large long codewords worsen the compression 2. Decide how to handle codebook overflow Flush when full Flush when compression ratio becomes poor Track and replace least used codebook entries Run-length coding (around 1950) [exploits spatial redundancy too] encoded mode: each homogeneous string of pixels is represented by two 8-bit words: (length, value) absolute mode: first byte = 0; second byte =... : 0 end of line 1 end of image 2 new position: hor and vert offset given in next pair no. of following pixels that are represented by one-byte graylevel or color index If few runs of identical pixels exist, can expand data Especially useful for binary images (or for bit-plane coding) the value of the run can be represented by specifying the value of the first run of the row, or by assuming that such first run is white (and may have length zero) the run lengths can be in turn coded using variable-length coding based on their statistics (e.g. Huffman) CCITT (now ITU-T) codes for bi-level images (1988 and later) Developed for fax equipment, then used for storing TIFF and PDF docs Standard: A4 200x100 dpi 1728 bits/line, 1188 lines/page One-dimensional coding: Modified Huffman (MH) Two-dimensional coding: Modified READ (Relative Element Address Designate), (MR). Info from previous line used to encode present line) Group 3 standard (ITU-T: T.4): MH or MR, robust to tx errors, 1- or 2-D Group 4 standard (ITU-T: T.6): MR, more efficient, 2-D only 1-D coding: a line (1728 px) is represented as a sequence of alternating b/w runs First run of line is always white (possibly of length zero) EoL symbol terminates line (to recover from synchro errors) Each run is coded with an MH code Codes have been chosen basing on the probabilities of the runs in typical handwritten documents

5 2-D coding: Modified READ (MR) is a variant of READ (Relative Element Address Designate) Many images have a high degree of vertical coherence between consecutive lines 2-D coding: G. Monfardini, Univ. of Siena Changing elements in the current line are coded wrt a nearby (±3 places) change of the same color in the previous (reference) line vertical mode Changing elements in the current line without a correspondent in the reference line switch to horizontal mode (1D) On the opposite, if the reference line has a run with no counterpart in the current line special pass code Note 1: transmission errors tend to propagate to the following lines a reference line is coded in 1-D every k lines (k=2,4) Note 2: Compression ratios of T.4 and T.6 coders strongly depend on the data, can vary from a few units to several tens " " Block transform coding Lossless Perceptually lossless Difference Goal: pack most of the block information into the smallest number of transform coefficients Quantizer block Intrinsically lossy coding Risk of blocking artifact The encoding steps can be tuned to the local image content adaptive transform coding

6 In principle, any unitary (A N -1 = AN *T ) (orthogonal if real) transform can be used: DFT, DCT, WHT, KLT... Cosine and Walsh-Hadamard basis functions for n=4 Best transform? Lena: reconstructed image and reconstruction error 8x8 blocks, largest 50% of the coefficients in each block are used (this is adaptive coding) DFT (rmse=2.32) WHT (1.78) DCT (1.13) Periodicity artifact of DFT vs. DCT and WHT Best block size? Lena: reconstruction error vs. n, using 25% of the coefficients - Note: when n=2 only the DC component is kept in all cases - Note: DFT performance is monotonous due to the strong (decreasing) contribution of the repetition artifact computational complexity Best way to select the coefficients? Reconstruction using Largest magnitude subset of 12.5% of the coefficients Image detail: DCT, n = 2, 4, 8, 512 Maximum variance subset of 12.5% of the coefficients (ideal in the informationtheoretic sense)

7 Note: the positions of the selected coefficients must be stored/sent too Or, a standardized path can be followed for the selection, such as zigzag ordering Quantizer Coefficient selection until now was on/off Can be refined using scaling + rounding T ( u, v) Encoding: Tˆ( u, v) round Z( u, v) ~ Decoding: T ( u, v) Tˆ( u, v) Z( u, v) Typical quantization matrix Z(u,v) for coefficients in the range [-128:127] : JPEG The quantization matrix itself can be scaled to change its effects. Compression ratios below are: 12, 19, 30; 49, 85, 182 JPEG baseline coding (lossy; a lossless version exists) 8 x 8 image block shifted by -128 (to enable using signed integer format [-128,127] for all data)

8 : JPEG : JPEG JPEG baseline coding JPEG baseline coding DCT The difference between the DC term and the one of the previous block is calculated and Huffman-coded. For AC terms, a special Huffman coding allowing for runs of zeros is used Quantized DCT (-26-prev) Note: spaces are inserted here only for readability zig-zag ordered JPEG baseline coding When decoding, the DCT block becomes (compare to original one) : JPEG : JPEG JPEG baseline coding reconstructed image, error, detail 25:1 and results in some reconstruction error 52:1

9 : JPEG JPEG baseline coding of COLOR images Wavelet coding (lossy) separate coding of RGB channels (poor choice, correlated data) RGB YCbCr conversion with possible downsampling of the chroma components Quant=1; 4:4:4 Quant=1; 4:2:0 Quant=4; 4:2:0 No need to operate on blocks, since wavelet transform is inherently local no blocking artifacts Images can however be split into few tiles, to permit independent coding/decoding of image portions Wavelet coding (lossy) Haar Daubechies JPEG 2000 (lossy) 25:1 52:1 Computational complexity (ampl.<1.5) 75:1 2+2=4 Multiply+Add per coeff. 8+8=16 Multiply+Add per coeff. 105:1

10 Predictive coding (lossless) Simplest 2-D predictor 1 e n = f n f n f ˆ ( x, y) round[ f ( x, y 1)] f n = e n + f n 1-D signals; easily extended to 2-D or 3-D data, provided a causality description (ordering) is defined Optimal predictors can be derived basing on the correlation matrix of the signal Note: prediction error has 1 more bit than original data. However... Error distribution is approx. Laplacian Achievable compression ratio is approx. the entropy ratio

11 : JPEG LS : JPEG LS : JPEG LS

12 : JPEG LS : JPEG LS : JPEG LS Predictive coding (lossy) for adaptive symbol-by-symbol Golomb encoding contexts: 1. [< -21] 2. [-21,-8] 3. [-7, -4] 4. [-3, -1] 5. [0] 6. [1,3] 7. [4,7] 8. [8,21] 9. [>21] e n = f n f n f n = e n + f n (context [0]) Decoder is the same as in the lossless case, apart from rounding Predictor loop in the encoder avoids error buildup at the decoder For both encoder and decoder, f ( n) e ( n) fˆ( n)

13 Simplest case: Delta modulation (1 bit per pixel) Simplest case: Delta modulation fˆ ( n) f ( n 1); e ( n) (according to the sign of the error) More about JPEG Simplest 3-D predictor 1 f ˆ ( x, y, t) round[ f ( x, y, t 1)] Jpeg z13_jpeg_itu-t.81.pdf Jpeg 2000 z13_jpeg_2000_ebrahimi01.pdf Jpeg LS z13_jpeg_itu-t.87.pdf Jpeg XT z13_jpeg_xt.pdf Jpeg Pleno z13_jpeg_pleno_ebrahimi16.pdf Note smaller error range, smaller entropy than 2-D predictor

14 Motion-compensated prediction block matching: select macroblock size (e.g. 4x4, 16x16) (luminance+subsampled chrominances) select max. displacement (e.g. ±8, ±64) search (exhaustive, tree,...) for minimum mean absolute error determine and encode the motion vector Group of Pictures (GoP) I-frame (Intra) P-frame (Predictive) B-frame (Bidirectional) Motioncompensated prediction Sub-pixel motion compensation Motion vectors may point to out-of-grid positions Note "unpredictable" left strip of newly visible Earth surface High computational demands, but only at the encoder As usual, standards give prescriptions only for the decoder

15 Predictive coding in video compression standards A motion-compensated video encoder More about video coding z13_videocodingstandards.pdf Digital cinema