5.1 Introduction. Shri Mata Vaishno Devi University,(SMVDU), 2009

Transcription

1 Chapter 5 Multiple Transform in Image compression

2 Summary Uncompressed multimedia data requires considerable storage capacity and transmission bandwidth. A common characteristic of most images is that the neighboring pixels are correlated and therefore contains redundant information. Image compression research aims at reducing the number of bits needed to represent an image by removing the spatial and spectral redundancies as much as possible. The DCT based Image coding method Known as JPEG has become the industry standard. The DCT based encoder can be thought of as essentially compression of a stream of 8*8 blocks of image samples. The method utilizes fixed size blocks with no or little consideration for background and foreground statistics and spectral composition. The performance of various transforms for signal processing has been extensively studied. Suboptimum orthogonal transform with predetermined basic functions such as DFT, DCT, and WH have fast implementation and has its own features, which makes them more suitable for specific class of signals. The optimal compression can be attained using varied transforms and variable size blocks. An algorithm using variable image segmentation and mixed transforms is presented to capitalize on the narrow band and broadband signal components for image compression. A mixture of transforms can exploit the different spectral bands in an image to achieve higher compression as compared to single transforms. However, the optimal distribution of the signal into transform specific spectral sub bands is computationally intensive and requiring multiple switching between time and transform domains. The algorithm tries to achieve this segregation by decomposing the image into variable size blocks using a quad tree. The quad tree is formed with 4 4, 8 8 and blocks. It is pruned by fusing smaller blocks according to the smoothness in different region of the image. The blocks in the pruned quad tree are preprocessed by removing the DC coefficient of DCT from each pixel of the block followed by DCT-WHT transform to capitalize on the residual image.

3 5.1 Introduction Image and video compression is a dynamic application area in image processing. Improvement of compression technologies for image and video continues to play an important role for success of multimedia communication and applications. Direct transmission of these video images without any compression through today s communication channels in real-time is an intricate proposition. Interestingly, both the still and video images have significant amount of visually redundant information in their canonical depiction. The redundancy lies in the fact that the neighboring pixels in a smooth homogeneous region of a natural image have very little variation in their values which are not noticeable by a human observer. Similarly, the consecutive frames in a slow moving video sequence are quite similar and have redundancy embedded in them temporally. Image and video compression techniques, essentially reduce such visual redundancies in data representation in order to represent the image frames with significantly smaller number of bits and hence reduces the requirements for storage and effective communication bandwidth. The recent growth of data intensive multimedia based applications have not only sustained the need for more efficient ways to encode signals and images but have made compression of such signals central to storage and communication technology For video compression and still image compression various techniques and standard are available[166]. A typical technique for video compression starts by optimally encoding the first frame using a still image compression method. It then encodes each successive frame by identifying the differences between the frame and its predecessor, and encoding these differences. If a frame is very different from its predecessor, it should be coded independently, of any other frame. In the video compression, a frame that is coded using its predecessor is called inter frame (or just inter), while a frame that is coded independently is called intra frame (or just intra). An intra frame is labeled I, and an inter frame is labeled P (for predictive). Another type of frame that is encoded based on both past and future frames is labeled B (for bidirectional). Therefore end up with a succession of compressed frames of the three types I, P and B. An I frame is decoded independently of any other frame. A P frame is decoded using the preceding I or P frames. A B frame is decoded using the preceding and following I or P frames. 238

4 Figure 5.1 Coding order Figure 5.2: Decoding order 239

5 Figure 5.1 shows a sequence of such frames in the order in which they are generated by the encoder (and input by the decoder). Figure 5.2 shows the same sequence in the order in which the frames are output by the decoder and displayed. For overall optimal video compression, I-Frame is to be compressed individually. The techniques available for still images compressions are usually utilized. 5.2 Methods of Image compression There exist two classes of compression techniques for still images: i) Lossless Lossy compression ii) Predictive and Transform coding In lossless compression schemes, the reconstructed compressed image is numerically identical to the original image. This scheme is most suited for medical image compression. Whereas in lossless compression, achieves modest amount of compression. Reconstructed image suffers degradation relative to original. It discards redundant information to achieve higher compression with no or minimum visible loss. In predictive coding, information already sent or accessible is used to predict future values and the difference is coded. It is simple to implement and adapted to local image characteristics, as processing is done in the image in spatial domain. DPCM is an example of predictive coding. In Transform coding, it first transforms the image from spatial domain representation to a different type of representation using well known transform and then codes the transformed values after quantization. This provides greater data compression as compared to predictive coding but at the higher computational expense. In most of the application, other than medical data the compression technique utilized is lossy using transform coding technique. Lossy signal compression has been achieved by processing the signal in spatial domain or transform domain [167][168][169][170][171]. In transform domain an invertible transform maps the signal to a set of coefficients. It typically converts the statistically correlated data to uncorrelated data coefficients. The dominant transform coefficients are retained and the remaining ones are discarded. The dominant coefficient are quantized and encoded for transmission or storage. This is termed as constrained representation, where an input signal is represented with a 240

6 limited set of transform coefficients. Choice of transform depends on its energy compactness capability to represent compact output data in fewer numbers of coefficients. The performance of the various transforms has been extensively compared and studied in literature [172]. Image transforms are designed to have two properties: i) To reduce image redundancy by reducing the sizes of most pixels ii) To identify the less important parts of the image by isolating the various frequencies of the image. Thus transform is a dominant tool for squeezing out the redundancy in data. The Karhunen-Loeve Transform (KLT) utilizes the Eigen-vectors of the covariance matrix as its basis functions and therefore provides the optimum set of basic functions [169]. KLT is data dependent and computationally intensive. It is not recommended for signal representations which are generally non- stationary. Various suboptimum orthogonal transforms with predetermined basis functions such as DFT, Discrete Cosine Transform (DCT), Haar and Walsh Hadamard (WH) transform are generally used for fast implementation and near optimal signal compression. Each of the transforms has their own explicit features, which makes them more proficient and functional for an explicit class of signals. Like for instance DFT utilizes complex exponentials as it fundamental functions. It can give compact representation to spectrally narrowband signals. WH and Haar utilizes the square and rectangular waveforms as basic signals which make them appropriate for spectrally wideband signals. It is shown that, while the DCT approximate the KLT in terms of information compression capability for first order Markov process with exponential correlation, the WH and Haar have least computational complexity. 5.3 DCT in Image Compression The DCT represents an image as a sum of sinusoids of varying magnitudes and frequencies. The DCT [169] has the property that, for a typical image, most of the visually significant information about the image is concentrated in just a few coefficients of the DCT. 241

7 For this reason, the DCT is often used in image compression applications. The DCT is closely related to the discrete Fourier transform. It is a separable linear transformation; that is, the two-dimensional transform is equivalent to a onedimensional DCT performed along a single dimension followed by a one-dimensional DCT in the other dimension. The definition of the two-dimensional DCT for an input image A of size M N and output image B is defined as follows eq. (5.1). cos 2 1 /2 cos 2 1 /2 (5.1) Where, 0 p M-1 0-1, 0, 1 1, 0, 1 1 Where, M and N are the number of row and column, of A, respectively. If you apply the DCT to real data, the result is also real. The DCT tends to concentrate information, making it useful for image compression applications. 5.4 Walsh Hadamard Transform (WHT) Walsh Hadamard Transform (WHT) [169] has low compression efficiency. It is however, speedy, since it can be computed and implemented with just additions, subtractions, and an occasional right shift. Given an N N block of pixels P xy (where N must be a power of 2, N=2 n) its two dimensional WHT and Inverse WHT are defined by eq. (5.3) and eq. (5.5) respectively:,,,, (5.2) = 1 (5.3) H u, v h x,y,u,v (5.4) 242

8 = H u, v 1 (5.5) Where H (u, v) are the results of the transform (i.e. WHT coefficients) the quantity bi (u) is bit i of the binary representation of the integer u and p i (u)is defined in terms of the eq.(5.6). ` P o (u) =b n-1 (u), P 1 (u) =b n-1 (u) +b n-2 (u) P 2 (u) =b n-2 (u) +b n-3 (u) P n-1 (u) =b 1 (u) +b 0 (u) (5.6) The quantities g(x, y, u, v) and h(x, y, u, v) are kernels of WHT. These matrices are identical. They consist of +1 and -1 and they are multiplied by the factor 1/N. 5.5 JPEG Encoder JPEG is lossy DCT compression method for still image compression. It corresponds to the ISO/IEC international standard for digital compression and coding of continuous tone (multi-level) still images [169]. Figure 5.3 shows the block diagram for JPEG encoder. The main steps are: i) Transform RGB to YIQ or YUV and subsample color ii) Perform DCT on 8x8 image blocks. iii) Apply quantization iv) Perform zigzag ordering and Run length encoding v) Perform entropy coding. The choice of small blocks size in JPEG is a compromise as a larger block size would have made precision at low frequency better but the 8 8 size has made the computation fast. The block has lead to effect of isolating each block from its neighboring context. At high compression ratio these block artifacts are generally visible. The image as a whole, which is divided in blocks of fixed size irrespective to its statistic. This is its main hitch. It is recommended that for static, uniform background large block size should be utilized and for object foreground small block size is to be used. 243

9 The compression ratio can be high if variable block size based on block static is chosen. JPEG supports sequential, progressive, hierarchical and lossless mode. JPEG2000 [166] is new method which provides a better rate distortion trade off and improved subjective image quality. It works in two modes DCT and wavelet based. In this region of interest coding is possible which can be coded with better quality than the rest. Here variable segmentation is available. Other image compression techniques are discussed in [173]. Other intuitive methods like sub-sampling, quantization and sub band transforms are also used for compact representation of signal. Figure 5.3: Block diagram For JPEG Encoder 244

10 5.6 Advancement in Image Compression Most of the compression techniques utilize single transform. To have optimal representation/compression of complex signal like image, which comprises of different spectral component multiple transform should be used. It is possible that certain spectral features of a signal can be compactly represented by a particular transform; while some other features are represented by another transform. Thus it would be beneficial to appropriately resolve the signal into sub signals such that each constituent is efficiently represented by the dominant component in a particular transform domain. The entire signal then is represented by superimposing subsets of basic function from different domain. It is also to be noted that these subset in general be non-orthogonal with respect to each other. This will help in optimal representation and improved overall SNR (Signal to-noise ratio). In brief, image compression is a vital issue in digital image processing and finds extensive applications in many fields. This is the fundamental operation performed frequently by any multimedia and digital photography technique to capture an image. To improve the conventional techniques of image compressions using the DCT have already been reported and sufficient literatures are available on this. The JPEG is a lossy compression scheme, which employs the DCT as a tool and used mainly in digital cameras for compression of images. In the recent past the demand for low power, low complexity and low bit budget image compression is growing. As a result, various research workers are actively engaged to develop efficient methods of image compression using most recent digital signal processing techniques[173] [178]. The objective is to achieve an efficient and reasonable compression ratio, as well as better quality of reproduction of image with moderate bit budget (entropy) and computational cost. Keeping these objectives in mind the research work in the present thesis has been undertaken. 245

11 5.7 Quad Tree Based Multiple Transforms Image Compression Technique Compression is achieved by reducing the redundancies in a signal by decorrelating its components. A transform endeavors to reduce the redundancies in an image by mapping the set of correlated coefficients to a set of uncorrelated coefficients. The amount of compression and quality of the compressed image is a function of the energy concentration ability of the transform and the number and values of dominant transform coefficients retained and the coefficients discarded by the quantization process [174]. However, images contain both narrowband and wideband components and a single transform with fixed basis functions cannot optimally represent an image. A mixture of transforms each representing a sub-signal, can represent the total signal more efficiently as compared to a single transform [175]. Each transform can optimally represent a set of spectral features of a sub-signal. These hybrid or mixed transform approaches use subsets of orthogonal or non-orthogonal basis functions chosen from multiple transforms for efficient signal representation. For this, different mixed base representations based on DCT, DWT, DHT have been proposed [176] [177] [178], wherein each transform needs to extract its appropriate dominant components from the signal in succession for efficient signal representation. This transform based segregation involves multiple switching between the spatial and transforms domains. This process takes a large number of iterations for optimal distribution of spectral components [176]. An alternate hybrid coder proposed in [178] takes the advantage of short and long range correlations present in an image through image segmentation and a block-based hybrid transform. The hybrid transform is based on DCT and fractal transform and performs optimal bit allocation in the rate distortion sense. However, the determination of the constrained shortest path in the quad tree based trellis is an NP complete problem. Different Methods [175] based on above techniques exhibit superior performance as compared to single transform techniques like JPEG, but at the cost of high computational complexity. It is suggested that, an image is partitioned on the basis the smoothness in a portion of the image. The segmented image is then coded using multiple transform based hybrid 246

12 transformations for exploiting the narrowband and broadband sub-signals in different blocks. 5.8 Proposed Technique The proposed technique dwells on the idea of exploiting the short term correlations of an image by partitioning an image into variable segments. The narrowband and broadband components are segregated in each segment and independently coded. An image is characterized by regions of smoothness and sudden variations, which can be utilized for delineating different spectral components. Narrowband signal components of an image reside in smooth regions with gradual variations while sudden variations portray broadband components. In the proposed technique, these smooth regions are grouped as a single partition to exploit the short range correlations present as narrowband signals. In a segment, the invariant component is extracted and the remaining broadband component is coded WHT. This technique is divided in two stages. In the first stage, an image is segmented into variable size blocks using the variation in pixel values and represented as a quad tree. In the subsequent coding stage, in each image segment S, the DC coefficient of the DCT which contains the average value of a segment is calculated. The difference segment ( Sdiff ) which is essentially the broadband portion of S, is generated by the difference of each pixel value of S and the DC coefficient. This dynamic content is represented by WHT. WHT is a unitary and orthogonal transform [169] [178] composed by rectangular waveforms with values +1 and 1 as the basis functions is chosen because of its ability to compact the broadband spectral content. The WHT coefficients are rounded off to the nearest integers and the entropy coded. 5.9 Image Segmentation A full quad tree is constructed with blocks size of 4 4, 8 8 and Instead of decomposing larger blocks, smaller blocks in the initial full quad tree are fused to form a pruned quad tree. Pruning is done to exploit the extent of short term correlations in an image by determining the sudden variations in pixel values within a group of blocks. The smooth segments are identified and fused into larger blocks and 247

13 the portions dominated by change are retained as smaller blocks. A gradient based method (i.e.) Prune-Gradient is employed to differentiate the smooth parts from portions dominated by change Prunes-Gradient The correlations between the adjacent pixels in an image are preserved when the 2D image is converted into 1D Hilbert scan sequence. This Hilbert Curve (HC) preserves the order of the blocks by locating similar blocks with natural gradation spatially contiguous. It also preserves the coherence of the 2D image. Thus, if the curve is smooth for a portion of an image, then, the same will hold also in the corresponding HC. Moreover, the local monotonic behavior in spatially contiguous pixels implies that change in pixel values on the curve of a partition is small. Thus, when the HC for a group of blocks is smooth and slow varying they are fused into larger blocks Fusion of Blocks The quad tree has square block sizes of three dimensions 4 4, 8 8 and Four 4 4 blocks can be fused to form 8 8 block and four 8 8 blocks can be joined to form a block. The fusion of the blocks involves the following steps and is done in a single scan of the HC. The 2D image is converted is converted into 1D sequence through Hilbert scanning. For the current block with size 4 4, HC is viewed as a 1-D segment with 16 entries. The fusion of the sub-segments is a recursive process. It starts at one end of the segment and at each fusion step four 2 k sub-segments are fused into one 2 2k sub-segment if the four sub-segments form a homogenous segment. This continues till the end of the Hilbert segment is reached. A group of four 2 m sub-segments is called homogenous if the corresponding HC is monotonic. The HC can be considered to be slow varying and smooth if the gray level of the each pixel as predicted by the HC in the sub-segment is in the vicinity of the its real gray level. The problem thus reduces to the estimation of the pixel values. Suppose, the indices of the two points of the sub-segments are i 1 and i 2 and the mean of corresponding pixel values are p i1 and p i2, respectively. A straight line is drawn 248

14 using the two mean values p i1 and p i2. Using linear interpolation, the estimated gray level of a pixel value with an index i, i 1 i i 2 is p i = p p p ( i ) i2 i1 i1 + i1 i2 i1 (5.7) If substantial number of pixels have values in the proximity of the estimated values, the blocks in sub-segments are homogenous and fused into one. The tolerance range ε and percentage of pixel within the tolerance range ρ are content dependent and should be adapted to yield minimum bit allocation Simulation To evaluate the performance of the proposed scheme, five different test images with different characteristics were considered. i) Lena ii) Mother-daughter (1st frame) iii) Peppers iv) Mandrill v) Barb These standard images (image 1, 2, 3, 4, 5) are chosen for their different spectral characteristics and details. Images like Lena and Mother-Daughter (1st frame) are largely smooth while others like the Peppers exhibit large spatial changes. To compare the result of proposed technique standard images are also compressed using the default DCT based JPEG with a quality factor of 90% Results and Conclusion Each image was taken and segmented into blocks of , 8 8 and 4 4 by pruning the quad tree by fusing the blocks. Figure 5.5 shows the original mother daughter image with its pruned quad tree segmentation in Figure.5.6. The quad tree was pruned using the prune gradient method. In the full quad tree, the HC corresponding to first four contiguous 4 4 blocks was considered. The gradient line was constructed using the mean pixel values of the first and the last 4 4 blocks on 249

15 the chosen HC segment. The intensity distribution of the pixels for a typical block is as shown in Figure.5.4 where the tolerance range was taken as ±10 of the estimated pixel value and 75% of the total number of pixels were within the tolerance band. It can be seen from Figure.5.6 that the smooth background was fused into blocks while portions with large variations were not fused and retained as smaller blocks. Table -1 and Table-2 depict the PSNR and entropy (in bits per pixel) of the compressed image. The PSNR of all the images compressed with the proposed technique showed an improvement of 1.7 to 5 db. The gain increased for images with larger changes and the Mandrill image yielding a maximum gain of around 5 db. This was expected as portions with variations require large number of DCT coefficients to yield a high PSNR. In such images, WHT was able concentrate the broadband signals with large variation content. When the background was removed from the segmented image, what remained were the variations. Since there existed little correlation between these variations in the residual blocks, a large number of DCT coefficients were required for coding those changes while WHT could give a compact representation. WHT was able to concentrate the changes around a single value so that the resulting skewed distribution could be effectively encoded with the entropy encoding schemes. It was observed that WHT concentrated the random residues around a specific value and they lay in a small range. Moreover, IWHT coefficients of the rounded off coefficients were very close to the original values. Figure.5.7 shows the reconstructed image of mother daughter image (PSNR db). Figure.5.20 gives the histogram of the residual error calculated by taking the difference between the compressed image and the original image for mother daughter frame. The histogram shows the error to be concentrate at zero and is almost uncorrelated. This indicates that almost all the redundancy has been removed from the image. The error can be separately encoded for lossless image compression. Figure 5.8 and Figure 5.9 shows the original and quad tree representation for Peppers image. Figure 5.10 shows the reconstruction of the peppers image using proposed method. Figure 5.11, 5.12 and 5.13 shows the respective results for mandrill image. Figure 5.14, 5.15 and 5.16 shows the respective results for Barb image. Figure 5.17, 5.18 and 5.19 shows the respective results for Lena image. 250

16 The entropy (bits per pixel) were comparable for the both the present scheme and JPEG compression. The entropy of the proposed technique was slightly higher than JPEG for most the images except Mandrill. This indicates that coding efficiency of both the schemes is comparable. However, the PSNR value of the images compressed by the proposed method is better than DCT based JPEG compression. The experimental results confirm the hypothesis that the variable image segmentation is allows the exploitation of short range correlation present in an image together with effective coding of the broadband contents. The hybrid technique is able to enhance the PSNR for comparable compression ratios. Table 5.1 PSNR Comparison PSNR Image Name Proposed JPEG Mother-Daughter Lena Peppers Mandrill Barb Table 5.2 Entropy Comparison Entropy (bpp) Image Name Proposed JPEG Mother-Daughter Lena Peppers Mandrill Barb

17 Intensity of Pixels Threshold Figure 5.4 Mo-Do Seq: Distribution of Pixels as function of Intensity 252

18 Figure 5.5: original mother daughter frame 1 Figure 5.6: Quad Tree mother daughter frame 2 253

19 Figure 5.7: Reconstructed frame mother daughter Figure 5.8: original frame Peppers 254

20 Figure 5.9: Quad tree Peppers Figure 5. 10: Reconstructed frame Peppers 255

21 Figure 5.11: original Frame Mandrill Figure 5.12: Quad tree Mandrill 256

22 Figure 5.13: Reconstructed frame Mandrill. Figure 5. 14: original Frame Barb 257

23 Figure 5.15: Quad tree Barb Figure 5.16: Reconstructed frame Barb 258

24 Figure 5.17: Original frame Lena Figure 5.18: Quad tree Lena 259

25 Figure 5.19: Reconstructed frame Figure 5.20 Distribution of the Residual Error 260