Multimedia Signals and Systems Still Image Compression - JPEG

Transcription

1 Multimedia Signals and Systems Still Image Compression - JPEG Kunio Takaya Electrical and Computer Engineering University of Saskatchewan January 27, 2008 ** Go to full-screen mode now by hitting CTRL-L 1

2 Contents 1 Information Entropy 4 2 Huffman Coding 8 3 JPEG Image Compression 18 4 Application of DCT to 8 8 bolcks 27 5 Coding the reduced size DC image 41 6 Encoding DCT (AC) coefficients 46 7 Assignment JPEG 55 2

3 R µ ν 1 2 R δµ ν = 8πG c 4 T µ ν Here T µ ν is tensor of energy momentum. black red green yellow blue magenta cyan 3

4 1 Information Entropy The average amount of information is defined by the information entropy measured in bits. E = L 1 i=0 L 1 1 p i log 2 = p i i=0 p i log 2 p i The probability for the pixel value i to occur p i can be determined from the histogram of a picture by p i = h(i)/(n M). If a difference image, capable of reconstructing the original image (loss less), is produced by I (i, j) = I(i, j) 1 fi(i 1, j) + I(i, j 1)g 2 4

5 for 0 i N 1 and 0 j M 1, the information entropy for I is smaller than that of the image I. As long as the first row and the first column of the original image I are retained, the original image I is restored by I(i, j) = I (i, j) 1 fi(i 1, j) + I(i, j 1)g 2 The reconstruction must be done in the sequence of raster scanning by using the first row and column. Another simpler difference image define by I (i, j) = I(i, j) I(i 1, j) is usded in DPCM for the reduced size DC component image resulting from the 2D DCT. In this case, the first column must be retained for lossless reconstruction. 5

6 6

7 Information Entropy of the image I: Information Entropy of the image I :

8 2 Huffman Coding Huffman coding is an entropy encoding algorithm used for lossless data compression. The term refers to the use of a variable-length code table for encoding a source symbol (such as a character in a file) where the variable-length code table has been derived in a particular way based on the estimated probability of occurrence for each possible value of the source symbol. It was developed by David A. Huffman while he was a Ph.D. student at MIT, and published in

9 Huffman coding is based on the frequency of occurance of a data item (pixel in images). The principle is to use a lower number of bits to encode the data that occurs more frequently. Codes are stored in a Code Book which may be constructed for each image or a set of images. In all cases the code book plus encoded data must be transmitted to enable decoding. The Huffman algorithm is a bottom-up approach. Consider a case of having five symbols (A, B, C, D, E). 9

10 Symbol Count Probability Entropy A B C D E Total (average) From this table, the theoretical total information is = bits. If we use a fixed length code of 3 bits, the total would be 4 39 = 156 bits. The procedure of the Huffman coding is as follows: 10

11 1. From the table, pick two nodes (symbols) having the lowest frequencies or probabilities. Assign 1 to the one with the lowest, then 0 to the second lowest. Or, simply assign 1 to one of the two, and 0 to the other. Create a parent node of these two symbols combined, DE for this case. Symbol Count Probability Code A B C D E

12 2. Update the table with the newly created parent node, then repeat the previous step. Symbol Count Probability Code A B C DE (D), -1 (E) 12

13 3. Repeat until the table has only one node left. Symbol Count Probability Code A BC (B), 01 (C) DE (D), 11 (E) 13

14 4. Repeat until the table has only one node left. Symbol Count Probability Code A (BC)(DE) (B), 101 (C) 110 (D), 111 (E) 14

15 15

16 Symbol Count Probability Entropy Code Subtotal A B C D E Total (avg) 87 Compare this with the theoretical total information of = bits. 16

17 problem: When the probabilities of occurrence for alphabets A to H are given, verify the following Huffman codes by drawing the code tree. Symbol Probability Code A B C D E F G H

18 3 JPEG Image Compression A joint ISO/CCITT committee known as JPEG (Joint Photographic Experts Group) has established the international compression standard for continuoustone still images, both grayscale and color, early in 1990 s. JPEG now supports four modes of operation, sequential encoding, progressive encoding, lossless encoding, hierarchical encoding. The most fundamental sequential encoding that encodes a picture in a single left-to-right, top-to-bottom scan, is discussed here. This is a lossy compression. 18

19 The encoder consists of 3 major components, (1) Forward DCT (Discrete Cosine Transform, (2) Quantizer based on the quantization table, and (3) Entropy Encoder that employs Huffman coding and Run-Length coding. These are applied to each of three components in the YUV (YCbCr) color space, sequentially. The paper, Gregory K. Wallace, The JPEG Still Picture Compression Standard, is a good reference available at 19

20 1. Forward 8 8 DCT An image is divided into a stream of 8 8 blocks of gray scale image samples. The image is scanned left-to-right, top-to-bottom. Source image samples grouped in 8 8 blocks are shifted from unsigned integers [0, 2 p 1] to signed integers [ 2 p 1, 2 p 1 1]. Each block of 8 8 pixels is then transformed by the forward DCT into the spectral domain. The forward DCT (FDCT) is given by F (u, v) = 1 4 C(u)C(v) 7 x=0 y=0 C(u), C(v) = 7 f(x, y) cos (2x + 1)uπ for u, v = 0 1 otherwise cos (2y + 1)vπ 16 20

21 clear all; close all; I=imread( lenna-y.jpg ); imshow(i); hold on; [x0,y0]=ginput(1); x0=fix(x0); y0=fix(y0); x=x0; y=y0; x=[x-1,x+8,x+8,x-1,x-1] y=[y-1,y-1,y+8,y+8,y-1] plot(x,y, -r ); I88=double(I(x0:x0+7,y0:y0+7))-128 DCT88=fix(dct2(I88)) % FUJIFILM - FinePix F40fd () A=[ ] DQ=fix(DCT88./A) 21

22 An area of DCT block set in the eye. Numerical values are sampled from this area. 22

23 A selected 8 8 image block: I88 =

24 The result of forward DCT: DCT88 =

25 A quantization matrix (FUJIFILM - FinePix F40fd): A =

26 The DCT matrix after quantizxation: DQ =

27 4 Application of DCT to 8 8 bolcks DCT, Quantization, DC-component The detailed procedure of JPEG encoding takes the following steps: 1. Subdivide a given picture into blocks of a 8 8 pixel area. Then, process the blcoks sequentially from left-to-right then top-to-bottom. 2. Convert pixel values of a block from the unsigned integer [0, 2 N 1] to signed integer [ 2 N 1, +2 N 1 1]. 3. Apply 2D DCT to the block. For a 8 bit gray scale image [0, 255], the values of 2D DCT transform are in the range of 2048 I(i, j) 2048, 12 bits. The double summation makes the possible largest value be 255 (8 bits) times 64, 8 8, (6 bits). The constant 1 4 divides the result by 4 or (2 bits). Thus, 27

28 the DCT values are 12 bits. 4. Divide each of the 8 8 DCT elements by the corresponding value in the quantization table. 5. Repeating these steps until all blocks are processed. The encoding part of JPEG compression without Huffman and Run-length coding looks like the following: I=imread( lenna-y.jpg ); imshow(i); Img=double(I); % FUJIFILM - FinePix F40fd () A=[ ] 28

29 [row,col]=size(i); DCTimg=zeros(row,col); blrow=row/8; blcol=col/8; DCcomp=zeros(bLrow,bLcol); % encoding JPEG for ii=1:blrow for jj=1:blcol r0=8*(ii-1)+1; r7=r0+7; c0=8*(jj-1)+1; c7=c0+7; I88=double(I(r0:r7,c0:c7))-128; DCT88=fix(dct2(I88)); DQ88=fix(DCT88./A); DCTimg(r0:r7,c0:c7)=DQ88; DCcomp(ii,jj)=DQ88(1,1); end end figure; imshow((dctimg+128)/256); figure; imshow((dccomp+128)/256); Application of 2D DCT produces an image of DCT coefficients, and an image of DC component of all blocks, which is the reduced size image of the original, reduced by 8 horizontally, and 8 vertically. 29

30 The original picture of Lena, and its quantized DCT image. 30

31 The image made of only the DC components of 2D DCT. The size is one eighth ( 1 8 ), horizontally and vertically. 31

32 Inverse DCT to reconstruct a compressed JPEG image In order to reconstruct the image from the quantized DCT coefficients, actually the image of quantized DCT shown above, the process of encoding with the 2D DCT was entirely reversed as shown in the following MATLAB codes. The 2D DCT was replaced by the 2D inverse DCT (idct2). 32

33 % decoding JPEG RCNimg=zeros(row,col); for ii=1:blrow for jj=1:blcol r0=8*(ii-1)+1; r7=r0+7; c0=8*(jj-1)+1; c7=c0+7; DQ88=DCTimg(r0:r7,c0:c7); DCT88=DQ88.*A; idct88=idct2(dct88)+128; RCNimg(r0:r7,c0:c7)=iDCT88; end end figure; imshow(rcnimg/256); 33

34 The original picture of Lena, and its reconstructed image with the inverse DCT and dequantization. 34

35 PSNR and Entropy Values Image reconstruction from the DCT image, quantized DCT coefficients to be exact, was successful in appearance. The peak signal to noise ratio PSNR was measured for the reconstructed image referenced to the original image. The value obtained was 37.8 db. The peak value of 255 was used. The mean squared error MSE was 10.78, giving an average error in magnitude be around 3.28, compared with the maximum pixel value of 255. The information entropy was calculated for the original and the reconstructed. They are very close, vs The entropy of the DCT image was only , which means that this image can be compressed down to about 1 bit per pixel. 35

36 MSE = PSNR = Entropy_Img = Entropy_Rcn = Entropy_dct = Finished 36

37 The histogram of the original picture of Lena, and the histogram of the DCT image. 37

38 MATLAB program to test JPEG encoding and decoding clear all; close all; I=imread( lenna-y.jpg ); imshow(i); Img=double(I); % FUJIFILM - FinePix F40fd () A=[ ] [row,col]=size(i); DCTimg=zeros(row,col); blrow=row/8; blcol=col/8; DCcomp=zeros(bLrow,bLcol); % encoding JPEG for ii=1:blrow 38

39 for jj=1:blcol r0=8*(ii-1)+1; r7=r0+7; c0=8*(jj-1)+1; c7=c0+7; I88=double(I(r0:r7,c0:c7))-128; DCT88=fix(dct2(I88)); DQ88=fix(DCT88./A); DCTimg(r0:r7,c0:c7)=DQ88; DCcomp(ii,jj)=DQ88(1,1); end end figure; imshow((dctimg+128)/256); figure; imshow((dccomp+128)/256); % decoding JPEG RCNimg=zeros(row,col); for ii=1:blrow for jj=1:blcol r0=8*(ii-1)+1; r7=r0+7; c0=8*(jj-1)+1; c7=c0+7; DQ88=DCTimg(r0:r7,c0:c7); DCT88=DQ88.*A; idct88=idct2(dct88)+128; RCNimg(r0:r7,c0:c7)=iDCT88; end end 39

40 figure; imshow(rcnimg/256); % Calculate PSNR SSE=0; for i=1:row for j=1:col SSE=SSE+(Img(i,j)-RCNimg(i,j))^2; end end MSE=SSE/(row*col) PSNR=10*log10(255^2/MSE) Entropy_Img=entropy(Img/256) Entropy_Rcn=entropy(RCNimg/256) Entropy_dct=entropy((DCTimg+128)/256) figure; subplot(121); imhist(img/256); subplot(122); imhist((dctimg+128)/256); disp( Finished ); 40

41 5 Coding the reduced size DC image DPCM Losless Coding F (u, v) = 1 4 C(u)C(v) 7 x=0 y=0 C(u), C(v) = 7 f(x, y) cos (2x + 1)uπ for u, v = 0 1 otherwise cos (2y + 1)vπ 16 In the DCT image F (u, v), F (0, 0) is DC component of an image block, located at the upper left corner of a 8 8 matrix. An image of the reduced size, down to 1/8 both horizontally and vertically is constructed from the DC components of all blocks. In JPEG, this reduced size image is coded with the lossless DPCM coding. The DC components that take a value F (u, v) 1023 because of 41

42 C(0) = 1/ 2, is subjected to the difference operation (DPCM), where I(i, j) is the reduced size DC image. D(i, j) = I(i, j) I(i 1, j) Possible values of D(i, j), D(i, j) 2047 are grouped into bins defined by [ 2 n 1, 2 n 1] and [2 n 1, 2 n 1] for each block of n = 1, 11. Then, these bins are Huffman coded for the group number. Elements in each block are coded with an additional bit length of n, which is the code group number. This coding scheme is dipicted in the following figure. 42

43 1 1(010) 2 0(00) 2 (011) Group code 3 3(100) 4(101) -7, -6, -5, -4 4, 5, 6, 7 Difference code 8, 9, 10,11,12,13,14,15 (000,001,010,011) (100,101,110,111)

44 Gr. Difference of DC values Group code Added bits , ,-2,2, , , , , , , , , ,

45 Examples: difference=-5, Group code (100) + Added code (011) = (100011) difference=63, Group code (1110) + Added code (111111) = ( ) difference=1, Group code (101) + Added code (1) = (1011) difference=0, Group code (00) + no Added code 45

46 6 Encoding DCT (AC) coefficients Zigzag scanning DCT coefficients of a block other than the DC components are 46

47 scanned in a zigzag fashion as shown in the figure above. The zigzag scanning moves from lower frequencies to higher frequencies. As the DCT tends to concentrate AC coefficients (components) in the upper left area, the zigzag scan encounters more zeros as it goes to higher frequencies. Therefore, JPEG uses Huffmann coding for non-zero DCT coefficients, and Run-length coding to encode the length of repeated zeros (of the DCT coefficinets) in order to encode the DCT s AC components. Nonzero AC DCT coefficients Nonzero AC DCT coefficients, namely valid coefficients for Huffman coding, are grouped into 10 groups. the same number of bits as the group number are appended to the Huffman code for a group code. 47

48 Gr. Difference of DC values bits appended , ,-2,2, , , , , , , , ,

49 Huffman coding for Run-length, Group Number combined 49

50 Gr. Group No. Huffman Code 0 EOB

51 Now consider to encode the following AC DCT coefficients. DQ =

52 Symbol Group Huffman Code bits Appended Run 0 value Run 0 value Run 0 value Run 0 value Run 1 value EOB The final code sequence is ( )( )( )( )( )(1010) 52

53 Chroma Subsampling Color images are transformed from RGB to YUV in JPEG. Luma component Y, and two chroma components U (Cb) and V (Cr) are independently quantized then entropy coded. However, chroma has less amount of informatio compared with luma. In JPEG, chroma subsampling can be specified. Chroma subsampling notation is shown in the figure below. Typically, one of 4:4:4, 4:2:2, or 4:2:0 is used. 53

54 4:4:4 4:2:2 4:2:0 x x x x x - x - x - x - x x x x x - x x x x x x - x - x - x - x x x x x - x

55 7 Assignment JPEG Free, portable C code for JPEG compression is available from the Independent JPEG Group. Source code, documentation, and test files are included. Version 6b is available from ftp.uu.net:/graphics/jpeg/jpegsrc.v6b.tar.gz. If you are on a PC you may prefer ZIP archive format jpegsr6b.zip, which you can find at This assignment is to compile this JPEG source code on either PC or Linux platform then study jpeg files produced by this software with various control parameters. This free portable C code was tested with djgpp on Windows XP. Djgpp is a complete 32-bit C/C++ development system for Intel (and higher) PCs running DOS. If you are not familiar with Linux (Unix based PC operating system), you are advised to use 55

56 djgpp. Note that this source code jpegsr6b.zip provides Makefile and config.h for djgpp, but not specifically for Linux. The coding part of the compiled program cjpeg.exe has a command line switch -quality N which scales the quantization tables to adjust image qaulity. Quality is 0 (worst) to 100 (best); default is 75. For an image of your choice in ppm, bmp, gif, run cjpeg.exe with different setting of -quality N to obtain an output image of that specified quality. Calculate first the entropy of the input image to compare the degree of compression achieved for varied -quality N. Try N=100, N=75, N=50, N=25 and N=10. Measure the achieved entropy by dividing the total number of bits of the jpeg file by the image size. Also calculate MSE and PSNR of each image generated. Discuss how -quality N affects compression, and image quality. The wizard switches are intended for experimentation with 56

57 JPEG. These switches are documented in the file wizard.doc. One of the wizard switches is -qtables file. You can specify the quantization tables given in the specified text file to use it instead of default file. Quantization tables used in digital cameras are found at Try the Quantization Table for FUJIFILM - FinePix F40fd () to encode your image. Discuss how this quantization table affects compression, and image quality. By default, cjpeg uses 2:1 horizontal and vertical downsampling when compressing YCbCr data. Other chroma subsampling can be experimented by -sample HxV[,...]. Try 4:4:4, 4:2:2 and 4:2:0. Discuss how this switch for setting subsampling factor affects compression, and image quality. 57