Lecture 6 Introduction to JPEG compression

Transcription

1 INF5442/INF9442 Image Sensor Circuits and Systems Lecture 6 Introduction to JPEG compression 11-October-2017

2 Course Project schedule Task/milestone Start Finish Decide topic and high level requirements 13.sep 20.sep Create project plan (tasks, milestones, schedule) 20.sep 27.sep MS1 - project plan approved by Johannes 27.sep 27.sep Study literature on the topic 27.sep 04.okt Design/simulation 04.okt 25.okt Write up prelim report (incl references, design, results) 25.okt 01.nov MS2 - submit prelim report to Johannes 01.nov 01.nov Design/simulation 01.nov 15.nov Write up final report (incl references, design, results) 15.nov 22.nov MS3 - submit final report to Johannes & 5min presentation 22.nov 22.nov Grading (PASS/FAIL) by Johannes & Soman 22.nov 29.nov Exam 18.des INF5442/9442 2

3 Key takeaways from previous lecture Low-pass filter: average accross neighboring pixels, h f(x,y) High-pass filter (unsharp): g(x, y) = {f(x,y) α (h f(x,y))} / (1 α) 3 AWB methods: grey world, brightest point, color gamut

4 Why JPEG compression? Reduce output data bandwidth Enable higher video frame rate (when output data bandwidth is the bottleneck) Reduce I/O area and power consumption Save image storage space on host side

5 Basic concepts used in JPEG Human visual system more sensitive to luma levels than chroma levels Sub-sample chroma data Small changes between neighboring pixels Transform into frequency domain which enables efficient coding schemes Human visual system more sensitive to low frequency than high frequency content Remove HF contents that HVS cannot see

6 Conceptual diagram av JPEG encoder Chroma sub sampling Cr Cb Y Compressed image

7 Step-1: RGB => YCbCr Y = R G B Cb = (B- Y) Cr = 0.713(R - Y)

8 Step-2: Sub-sample chroma information 4:2:2 sampling 4:2:0 sampling Chosen sampling scheme written in JPEG header file to enable receiver CPU to correctly decompress the image

9 Step-3: divide up picture in macro blocks of 8x8 pixels Studies have shown that 8x8 is optimal block size High correlation (redundancy) inside pixel cluster Relatively inexpensive HW implementation (only 8 rows of memory required to process 8x8 matrix)

10 VGA example VGA picture: 640x480 pixels 4:2:0 coding 4800 (80x60) luma macro blocks (Y) 2x1200 (40x30) chroma macro blocks (Cb, Cr)

11 Step-4: DCT transformation Analogous to FFT. Signal transformed to frequency domain. Each macro block described as weighted sum of 64 frequencies (i.e. jpeg-defined reference pictures) Input (X ij ): 8x8 matrix (original picture) Output (Y ij ): 8x8 matrix (coefficients for each frequency ) DCT transformasjon : Y ij A X ij A T

12 Conceptual diagram of DCT transform JPEG defined 8x8 DCT patterns

13 DCT example X ij AX ij A T Y ij HF content close to zero

14 Step-5: Quantization (lossy) Divide by quantisation matrix, Q, and round off decimals Level of lossyness defined by Q which is defined in JPEG header Coefficients at high frequencies can be rounded to zero without visible image artifacts Luma: Z ij Y Round Q ij ij Chroma:

15 Example Original picture (X ij ) After DCT (Y ij ) After quantization (Z ij ) X ij Y ij Y ij = Z ij x Q ij Reversing the process

16 Step-6: Coding of DCT coefficients after quantization DC values (0,0 position) coded with Differential PCM (DPCM) AC values (all other positions) coded with Run Length Coding (RLC) DPCM coding of DC-values DC i-1 DC i DCi DCi DCi

17 Run Level Coding (RLC) of AC values Efficient coding method of all zeros in the matrix Diagonal (ZigZag) scanning of the matrix Count zeroes (A) until non-zero value (B) Transmit (A,B), and start counting zeros again RLC example: (0, 32), (0, 6), (0, -1), (0, -1), (1,-1), (3, -1), (2, 1), (EOB) EOB=End of block

18 Step-7: Entropy encoding Lossless Frequent codes represented with few bits Rare codes represented with many bits Also called Variable Length Coding (VLC) Available VLC algorithms Huffman coding (most common) Adaptive Huffman coding Shannon-Fano Algorithm Arithmetic coding Jpeg header contains look-up table for selected VLC algorithm

19 Huffman coding of DC and AC coefficients Huffman coding of DC coefficients Huffman coding of AC coefficients

20 Some JPEG variants Progressive JPEG: picture stored as series of scans that make the picture gradually sharper Lossless JPEG: no quantization JPEG 2000: wavelet transformation instead of DCT

21 Other compression standards MPEG H.264 Wavelet (e.g. JPEG-2000) fractal vector quantization

22 JPEG references W.B. Pennebaker and J.L. Mitchell, JPEG Still Image Data Compression Standard, International Thomson Publishing, Chapman & Hall, NY, 1992 Digital Image Processing by R.C. Gonzalez and R.E. Woods

23 Key takeaways JPEG often embedded in CIS with on-chip color processing (aka SOC imagers) to reduce output datarate Image sectioned into 8x8 blocks Color compressed by chroma sub-sampling DCT transforms image into frequency domain Quantization matrix defines level of compression Data is transmitted using efficient coding schemes (DPCM/RLC/Huffman) for minimum redundancy