INF5442/INF9442 Image Sensor Circuits and Systems Lecture 6 Introduction to JPEG compression 11-October-2017
Course Project schedule Task/milestone Start Finish Decide topic and high level requirements 13.sep 20.sep Create project plan (tasks, milestones, schedule) 20.sep 27.sep MS1 - project plan approved by Johannes 27.sep 27.sep Study literature on the topic 27.sep 04.okt Design/simulation 04.okt 25.okt Write up prelim report (incl references, design, results) 25.okt 01.nov MS2 - submit prelim report to Johannes 01.nov 01.nov Design/simulation 01.nov 15.nov Write up final report (incl references, design, results) 15.nov 22.nov MS3 - submit final report to Johannes & 5min presentation 22.nov 22.nov Grading (PASS/FAIL) by Johannes & Soman 22.nov 29.nov Exam 18.des INF5442/9442 2
Key takeaways from previous lecture Low-pass filter: average accross neighboring pixels, h f(x,y) High-pass filter (unsharp): g(x, y) = {f(x,y) α (h f(x,y))} / (1 α) 3 AWB methods: grey world, brightest point, color gamut 11.10.2017 3
Why JPEG compression? Reduce output data bandwidth Enable higher video frame rate (when output data bandwidth is the bottleneck) Reduce I/O area and power consumption Save image storage space on host side 11.10.2017 4
Basic concepts used in JPEG Human visual system more sensitive to luma levels than chroma levels Sub-sample chroma data Small changes between neighboring pixels Transform into frequency domain which enables efficient coding schemes Human visual system more sensitive to low frequency than high frequency content Remove HF contents that HVS cannot see 11.10.2017 5
Conceptual diagram av JPEG encoder Chroma sub sampling Cr Cb Y Compressed image 11.10.2017 6
Step-1: RGB => YCbCr Y = 0.299 R + 0.587 G + 0.11B Cb = 0.564 (B- Y) Cr = 0.713(R - Y) 11.10.2017 7
Step-2: Sub-sample chroma information 4:2:2 sampling 4:2:0 sampling Chosen sampling scheme written in JPEG header file to enable receiver CPU to correctly decompress the image 11.10.2017 8
Step-3: divide up picture in macro blocks of 8x8 pixels Studies have shown that 8x8 is optimal block size High correlation (redundancy) inside pixel cluster Relatively inexpensive HW implementation (only 8 rows of memory required to process 8x8 matrix) 11.10.2017 9
VGA example VGA picture: 640x480 pixels 4:2:0 coding 4800 (80x60) luma macro blocks (Y) 2x1200 (40x30) chroma macro blocks (Cb, Cr) 11.10.2017 10
Step-4: DCT transformation Analogous to FFT. Signal transformed to frequency domain. Each macro block described as weighted sum of 64 frequencies (i.e. jpeg-defined reference pictures) Input (X ij ): 8x8 matrix (original picture) Output (Y ij ): 8x8 matrix (coefficients for each frequency ) DCT transformasjon : Y ij A X ij A T 11.10.2017 11
Conceptual diagram of DCT transform JPEG defined 8x8 DCT patterns 11.10.2017 12
DCT example X ij AX ij A T Y ij HF content close to zero 11.10.2017 13
Step-5: Quantization (lossy) Divide by quantisation matrix, Q, and round off decimals Level of lossyness defined by Q which is defined in JPEG header Coefficients at high frequencies can be rounded to zero without visible image artifacts Luma: Z ij Y Round Q ij ij Chroma: 11.10.2017 14
Example Original picture (X ij ) After DCT (Y ij ) After quantization (Z ij ) X ij Y ij Y ij = Z ij x Q ij Reversing the process 11.10.2017 15
Step-6: Coding of DCT coefficients after quantization DC values (0,0 position) coded with Differential PCM (DPCM) AC values (all other positions) coded with Run Length Coding (RLC) DPCM coding of DC-values DC i-1 DC i DCi DCi DCi 1...... 11.10.2017 16
Run Level Coding (RLC) of AC values Efficient coding method of all zeros in the matrix Diagonal (ZigZag) scanning of the matrix Count zeroes (A) until non-zero value (B) Transmit (A,B), and start counting zeros again RLC example: (0, 32), (0, 6), (0, -1), (0, -1), (1,-1), (3, -1), (2, 1), (EOB) EOB=End of block 11.10.2017 17
Step-7: Entropy encoding Lossless Frequent codes represented with few bits Rare codes represented with many bits Also called Variable Length Coding (VLC) Available VLC algorithms Huffman coding (most common) Adaptive Huffman coding Shannon-Fano Algorithm Arithmetic coding Jpeg header contains look-up table for selected VLC algorithm 11.10.2017 18
Huffman coding of DC and AC coefficients Huffman coding of DC coefficients Huffman coding of AC coefficients 11.10.2017 19
Some JPEG variants Progressive JPEG: picture stored as series of scans that make the picture gradually sharper Lossless JPEG: no quantization JPEG 2000: wavelet transformation instead of DCT 11.10.2017 20
Other compression standards MPEG H.264 Wavelet (e.g. JPEG-2000) fractal vector quantization 11.10.2017 21
JPEG references W.B. Pennebaker and J.L. Mitchell, JPEG Still Image Data Compression Standard, International Thomson Publishing, Chapman & Hall, NY, 1992 Digital Image Processing by R.C. Gonzalez and R.E. Woods http://www.wikipedia.org 11.10.2017 22
Key takeaways JPEG often embedded in CIS with on-chip color processing (aka SOC imagers) to reduce output datarate Image sectioned into 8x8 blocks Color compressed by chroma sub-sampling DCT transforms image into frequency domain Quantization matrix defines level of compression Data is transmitted using efficient coding schemes (DPCM/RLC/Huffman) for minimum redundancy 11.10.2017 23