Course INF581 Multimedia Coding and Applications Introduction and JPEG Ifi, UiO Norsk Regnesentral Vårsemester 28 Wolfgang Leister This part of the course...... is held at Ifi, UiO... (Wolfgang Leister) parts of this lecture were developed by Peter Oel, and Clemens Knoerzer
The story so far... Data compression information theory run length encoding Huffman coding Ziv-Lempel(-Welch) algorithm Arithmetic coding JPEG Joint Photographic Expert Group Developed 1991 1993 CCITT, ISO 1918 Lossless coding (Comp. 2:1) Coding with loss (1:1-4:1) Parameters control image quality Not limited to certain image types
JPEG Discrete Cosine Transformation (DCT) Huffman- or Arithmetic Coding Modes: Lossless Coding Sequential Coding Progressive Coding Hierarchical Coding Not a file format!!! JFIFF Lossless Coding Using Prediction C - 1 A A X B (Vhs. Diff.)(Vhs. Diff.)(Vhs. Diff)... Nr. Prediction 2 B 3 C 4 A+B-C 5 A+(B-C)/2 6 B+(A-C)/2 7 (A+B)/2
Lossy Coding Subdivision in 8x8 Blocks Transformation in frequency space Quantising Coding (Huffman or Arithmetic Coding) FDCT Quantiser Encoder 1111.. Why Use a Transform? Why use frequency domain? better statistic distribution many low frequency parts few high frequent parts quantising better achievable Humans see high frequencies only for high contrast values
Fourier Transform Fourier transform: Inverse FT: Discrete FT: Inverse DFT: See: http://en.wikipedia.org/wiki/discrete_fourier_transform Why DCT? Why not Fourier Transform? 8x8 Blocks FT: ringing at block edges
Why DCT? Why not Fourier Transform? 8x8 Blocks FT: ringing at block edges Why DCT? Why not Fourier Transform? 8x8 Blocks FT: ringing at block edges Mirroring produces smooth function Sinus coefficients disappear
DCT cos, sin functions cos used only in DCT f(t) = cos( u t) sampling of function DCT - basis functions
DCT - example The (I)DCT Formula f x =c u Pixels pixels f x,y =c u c 1 cos g u,x Coefficients v c 1 c 2 cos g u,x cos g v,y
DCT 1D - 2D Inverse DCT (decoding) pixels f x,y = 1 4[ u= 7 7 v= coefficients C u C v F u,v cos 2x 1 u 16 cos 2y 1 v 16 ] with C u,c v ={ 1 2 } for u,v= 1 else
(Forward)DCT unsigned signed FDCT: F u,v = 1 4 C u C v [ x= 7 7 y= f x,y cos 2x 1 u 16 cos 2y 1 v 16 ] with C u,c v ={ 1 } for u,v= 2 1 else FDCT Quantiser Encoder 1111.. 139 144 149 153 155 155 155 155 144 151 153 156 159 156 156 156 15 155 16 163 158 156 156 156 159 161 162 16 16 159 159 159 159 16 161 162 162 155 155 155 161 161 161 161 16 157 157 157 162 162 161 163 162 157 157 157 162 162 161 161 163 158 158 158 (Forward)DCT unsigned signed 11 16 21 25 27 27 27 27 16 23 25 28 31 28 28 28 22 27 32 35 3 28 28 28 31 33 34 32 32 31 31 31 31 32 33 34 34 27 27 27 33 33 33 33 32 29 29 29 34 34 33 35 34 29 29 29 34 34 33 33 35 3 3 3 FDCT Quantiser Encoder 1111.. FDCT 235.6-1. -12.1-5.2 2.1-1.7-2.7 1.3-22.6-17.5-6.2-3.2-2.9 -.1.4-1.2-1.9-9.3-1.6 1.5.2 -.9 -.6 -.1-7.1-1.9.2 1.5.9 -.1..3 -.6 -.8 1.5 1.6 -.1 -.7.6 1.3 1.8 -.2 1.6 -.3 -.8 1.5 1. -1. -1.3 -.4 -.3-1.5 -.5 1.7 1.1 -.8-2.6 1.6-3.8-1.8 1.9 1.2 -.6 -.4
An example Image example from wikipedia.org % read the image RGB = imread('phalaenopsis_(aka).jpg'); % convert pixels to the [ 1] range RGB = im2double(rgb); % convert to grayscale I = rgb2gray(rgb); % evaluate DFT (using log scale) F = log(abs(fft2(i))); % evaluate DCT (using log scale) C = log(abs(dct2(i))); % normalize C: immin = min(c(:)); immax = max(c(:)); C = (C - immin)/(immax - immin); % normalize F using the same scale as C F = (F - immin)/(immax - immin); F(find(F>immax)) = 1; F(find(F<immin)) = ; % evaluate the square of each term % (to make dark darker and bright brighter) F = F.*F; C = C.*C; % save outputs imwrite(i,'flower_original.png'); imwrite(f,'flower_fft.png'); imwrite(c,'flower_dct.png'); Quantising Q F(u,v) F (u,v) =Integer Q(u, v) FDCT Quantiser Encoder 1111..
235.6-1. -12.1-5.2 2.1-1.7-2.7 1.3-22.6-17.5-6.2-3.2-2.9 -.1.4-1.2-1.9-9.3-1.6 1.5.2 -.9 -.6 -.1-7.1-1.9.2 1.5.9 -.1..3 -.6 -.8 1.5 1.6 -.1 -.7.6 1.3 1.8 -.2 1.6 -.3 -.8 1.5 1. -1. -1.3 -.4 -.3-1.5 -.5 1.7 1.1 -.8-2.6 1.6-3.8-1.8 1.9 1.2 -.6 -.4 Quantising 15-1 -2-1 -1-1 16 11 1 16 24 4 51 61 12 12 14 19 26 58 6 55 14 13 16 24 4 57 69 56 14 17 22 29 51 87 8 62 18 22 37 56 68 19 13 77 24 35 55 64 81 14 113 92 49 64 78 87 13 121 12 11 72 92 95 98 112 1 13 99 FDCT Quantiser Encoder 1111.. Quantiser 1 DC-Coefficient Coding 63 AC-Coefficientes DC FDCT Quantiser Encoder 1111.. 15-1 -2-1 -1-1 AC
DC-Coding DC -1 DC -1-2 -1 DC -1-2 -1-1 -1 DC -1-2 -1-1 -1-2 -1-1 -1-1 -1 FDCT Quantiser Encoder 1111.. ΔDC i =DC i -DC i-1 ΔDC i =DC i -DC i-1 DC-Coding ΔDC i =(Length, Value) Length=Huffman-Coded Value=(Sign, abs Value) FDCT Quantiser Encoder 1111..
1 2 3 4 5 6 7 8 9 1 11 DC-Coding Length Value Examples: -1, 1-3,-2, 2,3-7..-4, 4..7-15..-8, 8..15-31..-16, 16..31-63..-32, 32..63-127..-64, 64..127-255..-128, 128..255-511..-256, 256..511-123..-512, FDCT 512..123 Quantiser Encoder 1111.. -247..-124, 124..247-3 : (2).1.1-1 : (1).1. : ().. 5 : (3)..1 67 : (7)..11 1 2 3 4 5 6 7 8 9 1 11 DC-Coding Length Code Examples: -3 : (2).1.1 : 11.1.1-1 : (1).1. : 1.1. : ().. :.. 5 : (3)..1 : 1..1 67 : (7)..11: 1111..11 1 11 1 11 11 111 1111 11111 111111 1111111 FDCT Quantiser Encoder 1111.. 11111111
Zig-Zag Serialising Zero-Run Length Huffman-Coding AC-Coding FDCT Quantiser Encoder 1111.. Zig-Zag Serialising DC -1-2 -1-1 -1 FDCT Quantiser Encoder 1111..
AC-Coding,-2,-1,-1,-1,,,-1,,,,,... (1x),-2,(x),-1,(x),-1,(x),-1, (2x),-1,<EOB> (1),-2,(),-1,(),-1,(),-1,(2),-1,<EOB> FDCT Quantiser Encoder 1111.. 15-1 -2-1 -1-1 AC-Coding,-2,-1,-1,-1,,,-1,,,,,... (1x),-2,(x),-1,(x),-1,(x),-1, (2x),-1,<EOB> (1),-2,(),-1,(),-1,(),-1,(2),-1,<EOB> (1),((2).1.),(),((1).1),(),((1).1),(), ((1).1),(2),((1).1),<EOB> FDCT Quantiser Encoder 1111.. 15-1 -2-1 -1-1
AC-Coding,-2,-1,-1,-1,,,-1,,,,,... (1x),-2,(x),-1,(x),-1,(x),-1, (2x),-1,<EOB> (1),-2,(),-1,(),-1,(),-1,(2),-1,<EOB> (1),((2).1.),(),((1).1),(),((1).1),(), ((1).1),(2),((1).1),<EOB> (1/2).1.,(/1).1,(/1).1,(/1).1,(2/1).1,<EOB> FDCT Quantiser Encoder 1111.. 15-1 -2-1 -1-1 AC-Coding <EOB> 11 1/1 11 /1 1/2 1111 /2 1... /3 1 2/1 111 /4 111 2/2 111111,-2,-1,-1,-1,,,-1,,,,,... (1x),-2,(x),-1,(x),-1,(x),-1, /5 111... (2x),-1,<EOB> /6 1111 15/1... (1),-2,(),-1,(),-1,(),-1,(2),-1,<EOB> /7 11111... (1),((2).1.),(),((1).1),(),((1).1),(),... 15/1... ((1).1),(2),((1).1),<EOB> (1/2).1.,(/1).1,(/1).1,(/1).1,(2/1).1,<EOB> 1111 1 1 1 1 111 1 11 63 AC-Coefficients 26 Bit /1... <ZR16> 111111111
DeCoding Decompression Rescale by DeQuantising Inverse DCT 1111.. Decoder DeQuantiser IDCT Compression Results n:1 Quality 3-2 usable - good 2-1 good - very good 1-5 excellent 5-4 not distinguishable from original
Progressive Mode Transfer coefficients partially in several runs. Two Possibilities: Spectral Transfer Approximated Transfer DCT Koeff. Blocks. 1... 62 63 7 6... 1 Bits. Progressive Mode Spectral Transfer DC AC 1 2 3 4 5 61 62 63
DCT Koeff. Blocks. 1... 62 63 7 6... 1 Bits. Progressive Mode AC DC 7 6 5 4 Approximated Transfer 1 2... 62 63 3... Hierarchical Mode Code image with low resolution first Code higher resolution as difference to previous lower resolution Image is represented in several resolutions Unnecessry data are not transferred
Images with several channels JPEG uses several (Colour)-Channels (e.g., YC b C r ) Channels can have different resolution Resolution factor as integer number JPEG method uses each channel separately! JFIFF JPEG defines algorithm only. JPEG is not a file format JPEG is colour-blind Parameters and tables are pre-defined Based on JPEG mode of TIFF 6. Consists of segments which are defined by markers (like TIFF)
Is JPEG good enough? Visible blocks for high compression rates For compression rates > 4:1 JPEG does not work well. low frequencies are not taken into account. Wavelet Coding Good quality up to 6:1 Linear degradation for higher compression rates No visible blocks JPEG 2 Standard Fractal Coding Comparison
JPEG: Literature Pennebaker,Mitchell: JPEG, Still Image Data Compression Standard, Van Nostrand Reinhold (1993) Wavelet-Coding: Daubechies: Ten Lectures on Wavelets, Society for Industrial and Applied Mathematics The End of this Lecture