ΝΤUA. Τεχνολογία Πολυμέσων

Transcription

1 ΝΤUA Τεχνολογία Πολυμέσων

2 3. Διάλεξη 3: Transform Coding

3 Rate Distortion Theory D may be the Mean Square Error or some human perceived measure of distortion

4 Types of Lossy Compression VBR Variable Bit Rate CBR Constant Bit Rate we Discuss Later

5 Κωδικοποίηση μετασχηματισμού (transform coding) Στη κωδικοποίηση μετασχηματισμού, το σήμα υφίσταται ένα μαθηματικό μετασχηματισμό από το αρχικό πεδίο του χρόνου ή του χώρου σε ένα αφηρημένο πεδίο το οποίο είναι πιο κατάλληλο για συμπίεση. x(t) --> g(λ) Αυτή η διαδικασία είναι αντιστρεπτή, δηλαδή υπάρχει ο αντίστροφος μετασχηματισμός που θα επαναφέρει το σήμα στην αρχική του μορφή.

6 Πχ Μετασχηματισμός Fourier Έστω σήμα x(t), τότε ο μετασχηματισμός Fourier X(f) ορίζεται: j2 ft X ( f ) x( t) e dt Ο αντίστροφος μετασχηματισμός Fourier ορίζεται: 1 2 F X ( f ) x( t) X ( f ) e j ft df A function f is resolved into Fourier series: a linear combination of sines and cosines (in blue)

7 Μετασχηματισμός Fourier (2) - Εδω ο μετασχηματισμός μας νοιάζει για άλλον λόγο: θα μεταδώσουμε τους Fourier συντελεστες αντι για το σημα (αγνοώντας τους μικρούς). - Στην πραγματικότητα θα χρησιμοποιήσουμε έναν άλλον μετασχηματισμό, τον discrete cosine transform (DCT) και όχι τον Fourier, γιατί ο DCT δουλεύει καλύτερα (σε εικόνες που περιέχουν ευθείες γραμμές κλπ) Πεδίο χρόνου ή χώρου Πεδίο συχνοτήτων Τ Οι συντελεστές αυτοί αγνοούνται t Οι σημαντικότεροι συντελεστές για πολλούς τύπους Πληροφορίας συγκεντρώνονται στις χαμηλές συχνότητες f

8 JPEG compression algorithm JPEG "JPEG" stands for Joint Photographic Experts Group, the name of the committee that created the JPEG standard and also other still picture coding standards The JPEG standard specifies how an image is compressed into a stream of bytes and decompressed back into an image, but not the file format used to contain that stream. The Exif and JFIF standards define the commonly used file formats for interchange of JPEG-compressed images. The JPEG compression algorithm is at its best on photographs and paintings of realistic scenes with smooth variations of tone and color. JPEG may not be as well suited for line drawings and other textual or iconic graphics, where there are sharp contrasts between adjacent pixels (such images may be better saved in a lossless graphics format such as TIFF, GIF, PNG, or a raw image format) JPEG uses a lossy form of compression based on the discrete cosine transform (DCT).

9 The Discrete Cosine Transform (DCT) DCT is a Fourier-related transform similar to the discrete Fourier transform (DFT) It transforms a signal or image from the spatial domain to the frequency domain A discrete cosine transform (DCT) expresses a finite sequence of data points in terms of a sum of cosine functions oscillating at different frequencies. The use of cosine rather than sine functions is critical for compression, since it turns out (as described below) that fewer cosine functions are needed to approximate a typical signal Most high frequency coefficient are near zero and can be ignored

10 The Discrete Cosine Transform (DCT) The discrete cosine transform (DCT) helps separate the image into parts (or spectral sub-bands) of differing importance (with respect to the image's visual quality). The general equation for a 1D (N data items) DCT is defined by the following equation: where The general equation for a 2D (N by M image) DCT is defined as: Φυσικά υπάρχει και ο αντίστοιχος μετασχηματισμός για να πάρουμε πίσω το σήμα

11 Digitalization 2D (από την εισαγωγή) An analog image a(x,y) in a 2D continuous space The 2D continuous image a(x,y) is divided into N rows and M columns. The intersection of a row and a column is termed a pixel. A digital image a[m,n] described in a 2D discrete space is derived through digitization Pixel

12 Digitalization 2D (από την εισαγωγή) In grayscale (monochrome) images, the value assigned to every pixel is the average brightness in the pixel rounded to the nearest integer value. The process of representing the amplitude (brightness) of the 2D signal at a given coordinate as an integer value with L different gray levels is usually referred to as amplitude quantization or simply quantization.

13 The Discrete Cosine Transform (DCT) The basic operation of the DCT is as follows: The input image is N by M; f(i,j) is the intensity of the pixel in row i and column j; F(u,v) is the DCT coefficient in row u and column v of the DCT matrix. For most images, much of the signal energy lies at low frequencies; these appear in the upper left corner of the DCT. Compression is achieved since the lower right values represent higher frequencies, and are often small enough to be neglected with little visible distortion. The DCT input is an 8 by 8 array of integers. This array contains each pixel's gray scale level; 8 bit pixels have levels from 0 to 255. The output array of DCT contains a set of coefficients (between and 2048). It is computationally easier to regard the DCT as a set of basis functions which given a known input array size (8 x 8) can be precomputed and stored. This involves simply computing values for a convolution mask (8 x8 window) with the coefficients that gets applied to the basis functions (cosines) The values are simply calculated from the DCT formula.

14 The Discrete Cosine Transform (DCT) The basis functions of the DCT talk about the spatial correlation (exploited by JPEG) and the artifacts than can be expected in the reconstructions. The 9 lowest-frequency DCT basis functions are Increasing the number of cosines-basis functions we increase the number of shapes we can produce

15 The Discrete Cosine Transform (DCT)

16 The Discrete Cosine Transform (DCT) The 64 (8 x 8) DCT basis functions, used in JPEG Cosine waves Frequency is increasing in the x- and y-axis Combined to make any 8x8 images, weighted by the coefficient The coefficient show the contribution of each of the basis functions The 8x8 images is a linear combination of these basis images

17 The Discrete Cosine Transform (DCT)

18 The Discrete Cosine Transform (DCT)

19 The Discrete Cosine Transform (DCT) F(0,0) is called the DC coefficient and the rest are called the AC coefficient Because there is a strong correlations between the DC coefficients of adjacent 8x8 blocks, the (quantized) DC coefficient is encoded separately from the AC components.

20 Approximation by DCT basis

21 Encoder Fundamentals of JPEG Encoder DCT Quantizer Entropy coder Decoder Compressed image data IDCT Dequantizer Entropy decoder Decoder

22 Encoder Image block Fundamentals of JPEG Encoder DCT Quantizer Entropy coder Decoder Compressed image data IDCT Dequantizer Entropy decoder Decoder

23 Encoder Image block Fundamentals of JPEG Transform Coefficients Encoder DCT Quantizer Entropy coder Decoder Compressed image data IDCT Dequantizer Entropy decoder Decoder

24 Encoder Image block Fundamentals of JPEG Transform Coefficients Encoder 2D->1D DCT Quantizer Entropy coder Decoder Compressed image data IDCT Dequantizer Entropy decoder Decoder

25 Encoder Image block Fundamentals of JPEG Transform Coefficients Encoder 2D->1D DCT Quantizer Entropy coder Bitstream Decoder Compressed image data IDCT Dequantizer Entropy decoder Decoder

26 Encoder Image block Fundamentals of JPEG Transform Coefficients Encoder 2D->1D DCT Quantizer Entropy coder Bitstream Decoder Compressed image data IDCT Dequantizer Entropy decoder Bitstream Decoder

27 Encoder Image block Fundamentals of JPEG Transform Coefficients Encoder 2D->1D DCT Quantizer Entropy coder Bitstream Decoder Compressed image data IDCT Dequantizer Entropy decoder Bitstream Decoder 1D->2D

28 Encoder Image block Fundamentals of JPEG Transform Coefficients Encoder 2D->1D DCT Quantizer Entropy coder Bitstream Decoder Compressed image data IDCT Dequantizer Entropy decoder Bitstream Decoder Reconstructed Transform Coefficients 1D->2D

29 Encoder Image block Fundamentals of JPEG Transform Coefficients Encoder 2D->1D DCT Quantizer Entropy coder Bitstream Decoder Compressed image data IDCT Dequantizer Entropy decoder Bitstream Reconstructed Image block Decoder Reconstructed Transform Coefficients 1D->2D

30 Fundamentals of JPEG JPEG works on 8 8 blocks Extract 8 8 block of pixels Convert to DCT domain Quantize each coefficient Different stepsize for each coefficient, based on sensitivity of human visual system Order coefficients in zig-zag order The quantized DC coefficient is encoded as the difference from the DC term of the previous block (differential pulse code modulation - DPCM) The AC components go through the Run Length Encoding ( stored as a single data value and count) Last data are encoded through entropy coding process (Huffman or Arithmetic coding)

31 Quantization of DCT coefficients

32 Default quantization table in JPEG A common quantization table (for luminance component) is

33 Example: quantized indices

34 Example: quantized coefficients

35 Effect of QP Nearly all software implementations of JPEG permit user control over the compression-ratio (as well as other optional parameters), allowing the user to trade off picture-quality for smaller file size.

36 Zig-Zag Scanning Purpose of the Zig-zag Scan: to group low frequency coefficients in top of vector maps 8 x 8 to a 1 x 64 vector prepares data for source coding

37 Zig-zag ordering Zig-Zag Scanning

38 Zig-Zag Scanning

39 Transform Coding Note: RLE uses skip and value, where skip is the number of zeros and value is the next non-zero component.

40 Example: reconstructed image

41 JPEG Modes Sequential Mode In "Motion JPEG", Sequential JPEG is applied to each image in a video. Progressive Mode Display low quality image (like preview) and successively improve. Good for low bandwidth connections Spectral selection Send lower frequency coefficients first Successive approximation Send lower precision first, and subsequently refine This also relates to image rendering on the receiver Lossless Mode A special case of the JPEG where indeed there is no loss. It does not use DCT-based method Hierarchical Mode Good for viewing high resolution image on low resolution display

42 JPEG Modes Sequential Mode In "Motion JPEG", Sequential JPEG is applied to each image in a video. Progressive Mode Display low quality image (like preview) and successively improve. Good for low bandwidth connections Spectral selection Send lower frequency coefficients first Successive approximation Send lower precision first, and subsequently refine This also relates to image rendering on the receiver Lossless Mode A special case of the JPEG where indeed there is no loss. It does not use DCT-based method Hierarchical Mode Good for viewing high resolution image on low resolution display

43 Example of JPEG Coding(Encoder)

44 Example of JPEG Coding(Encoder) Transform coding(dct)

45 Example of JPEG Coding(Encoder) Transform coding(dct)

46 Example of JPEG Coding(Encoder) Transform coding(dct) Quantization

47 Example of JPEG Coding(Encoder) Transform coding(dct) Quantization

48 Example of JPEG Coding(Encoder) Transform coding(dct) Quantization Zigzag Scan

49 Example of JPEG Coding(Encoder) Transform coding(dct) Quantization Zigzag Scan

50 Example of JPEG Coding(Encoder) Transform coding(dct) Quantization Zigzag Scan Entropy Coding (bit stream)

51 Example of JPEG Coding(Encoder) Transform coding(dct) Quantization Zigzag Scan Entropy Coding (bit stream) Inverse Entropy Coding (bit stream)

52 Example of JPEG Coding(Encoder) Transform coding(dct) Quantization Zigzag Scan Inverse Zigzag Scan Entropy Coding (bit stream) Inverse Entropy Coding (bit stream)

53 Example of JPEG Coding(Encoder) Transform coding(dct) Quantization Zigzag Scan Inverse Quantization Inverse Zigzag Scan Entropy Coding (bit stream) Inverse Entropy Coding (bit stream)

54 Example of JPEG Coding(Encoder) Transform coding(dct) Quantization Zigzag Scan Inverse Transform coding(dct) Inverse Quantization Inverse Zigzag Scan Entropy Coding (bit stream) Inverse Entropy Coding (bit stream)

55 Example of JPEG Coding(Encoder) Transform coding (DCT) DCT

56 Example of JPEG Coding(Encoder) Quantization -415/16 =

57 Example of JPEG Coding (Encoder) Quantization

58 Example of Zig Zag Scanning 2D->1D Zigzag Scan Entropy Coding (bit stream) EOB