CSE237A: Final Project Mid-Report Image Enhancement for portable platforms Rohit Sunkam Ramanujam (rsunkamr@ucsd.edu) Soha Dalal (sdalal@ucsd.edu) Project Goal The goal of this project is to incorporate an image enhancement block into a standard JPEG decoder for a portable platform. The enhancement unit will primarily perform two tasks improving the contrast of the image and removing any artifacts and noise introduced as a result of contrast enhancement. The contrast enhancement technique used manipulates the DCT coefficients of a JPEG image to improve its quality. It is carried out in the decompression stage so that the compressibility of the image is not affected. Since DCT and its inverse transform IDCT are performed on 8 x 8 blocks of pixels, any image processing algorithm implemented in the transform domain produces flaws known as blocking artifacts. These artifacts are a direct result of the independent processing of blocks which does not take into account the existing correlations among pixels in adjacent blocks. So the enhancement unit also tries to minimize these blocking artifacts introduced as a result of processing in the transform domain. Project Background Motivation Many techniques of contrast enhancement have been listed in literature but most of them manipulate the spatial domain pixel values. Spatial domain techniques include point to point mapping of pixel values between the original and the enhanced images (contrast stretching), mask processing, histogram equalization etc. Since most of the spatial domain coefficients of an image have non-zero values the total number of computations to be performed is directly related to the image size. However, in portable devices energy consumption is a key issue and reducing the number of computations at the algorithmic level will result in substantial energy savings. Many images are compressed using the JPEG standard which uses Discrete Cosine Transform (DCT) for image compression. Manipulating data in the DCT domain is an energy efficient image enhancement technique since a large number of DCT coefficients are 0. Background work and State of Art techniques Enhancement methods for contrast improvement range from simple contrast stretch techniques to filtering and image transforms. The most common technique includes global histogram equalization and histogram specification which try to make the histogram of an image uniform over all intensity levels or follow a specific pattern. A number of variants of the above have come up recently which include dynamic histogram specification [1] and minimum mean brightness error bi-histogram equalization [2]. Other common spatial domain techniques are adaptive unsharp masking [3] and edge detection [4]. Recent transform domain techniques for contrast enhancement include homomorphic filtering in the wavelet domain [5] and enhancement based on a contrast
measure defined in the DCT domain [6]. The algorithm proposed in [6] was implemented in this project since it has a wide application domain (JPEG and MPEG use DCT) and it is energy efficient. Many advanced post-filtering techniques exist to remove blocking artifacts in JPEG images [7]. A selective enhancement technique is used in this project to counter the side effects of contrast enhancement using minimum resources. Challenges and Limitations For contrast enhancement on embedded platforms like Xscale, the major challenge is to keep the number of computations low to save energy and minimize computation time. This calls for transform domain techniques which require some amount of post processing to remove blocking artifacts. The challenge is to achieve maximum quality improvement with minimum processing. Project Description As mentioned earlier, the enhancement unit is added to the JPEG decoder just before the IDCT transformation. The block diagram of the modified decoder is shown below Coded Image data Huffman/Ari -thmetic Decoding Image Enhancement Unit IDCT and Inverse quantization Enhanced Image Quantization Table Fig.1. Block Diagram of the modified JPEG decoder The image enhancement block enhances the contrast and decreases the blocking artifacts of a DCT transformed image. Hence, it is applicable to any DCT-based image compression standard, such as JPEG, MPEG 2, and H. 261. This algorithm exploits the fact that contrast measure is a ratio of high-frequency content and low-frequency content in the bands of the DCT matrix. The invocation of the enhancement block will be user controlled. The JPEG decoder has 2 IDCT modes fast and slow. The enhancement unit was incorporated only into the slow IDCT function so that very fast decoding without enhancement is still possible. The high level algorithmic strategy is as follows :- The coefficients of the DCT block are arranged left to right and top to bottom in the order of increasing spatial frequencies in the horizontal and the vertical spatial dimensions respectively. The 8x8 matrix is divided into 15 frequency bands as shown below. Each band is characterized by a band number n where - n = row index + column index A higher value of n implies a higher frequency band. The algorithm involves increasing the ratio of the average amplitude of the coefficients of band n to the average amplitude value of all the coefficients in
bands having number less than n by a constant factor. Intuitively it involves enhancing the high frequency components which correspond to higher contrast in the spatial domain. Blocking artifacts are removed by applying the above strategy only to regions with intensity variations and leaving the smooth regions unchanged. Results Parameters that were varied while enhancing the images: Degree of Contrast (λ) Average D.C level of the image (I) The threshold for selecting the regions of the image to apply the algorithm (for minimizing blocking artifacts) All these parameters can be controlled by the user depending on his/her needs and on the quality of the original image. Original image Enhanced Image Enhanced Image λ = 2 λ = 2.7 Fig.2. Enhancement performed on grayscale images Contrast enhanced Contrast enhanced and reduced blocking Fig.3. Minimizing blocking artifacts through selective application of the algorithm
By selectively applying the algorithm only to regions with intensity variations, blocking artifacts in the smooth regions of the image is reduced to a great extent as seen above. Further Improvisation The algorithm keeps the average intensity level over all the pixels a constant. Hence it may not work well for poorly illuminated images where increasing the average intensity is necessary for better perception. Increasing the average intensity also reduces blocking artifacts. Hence, in our enhancement unit the average intensity level of a block can be controlled by the user if needed. Original image Enhanced Image Enhancement on Color Images Fig.4. Effect of increasing average intensity level JPEG color images are stored in the YUV format. The algorithm was applied only to the luminance blocks (Y) of the image leaving the chrominance blocks (U, V) unchanged. A 4:1:1 sampling scheme was assumed where the chrominance components are downsampled by a factor of 2 in both the X and Y directions. This knowledge helped us in detecting the chrominance blocks at a much lower level in the decoder software and made the task of modifying the software easier. Original Image Enhanced Image Enhanced Imaged λ = 2 λ = 2.7 Fig.5. Contrast Enhancement on Color images (Average intensity increased by 20%)
Comparison of the computational efficiency with Histogram Equalization Type of Computation Additions Multiplications Divisions Histogram Equalization nm (forming histogram) nm (mapping intensities) nm (mapping intensities) DCT based Enhancement nm Number of non-0 DCT coefficients 15/64 (nm) Table.1. Number of Computations for an n x m Grayscale image The number of non 0 DCT coefficients is image dependent. For a typical image this may be around 15% which will result in 85% reduction in the number of multiplications. The computational efficiency is even higher for color images since the DCT based technique is only implemented on the luminance blocks of the image (assuming histogram equalization works on RGB planes). Hence such an enhancement unit is ideal for embedded platforms where energy is an important criterion for choosing an algorithm. Summary and Future Work The contrast enhancement algorithm with artifact removal scheme works well for both grayscale and color images. Its merits include few computations, high speed, degree of enhancement and average intensity are user controlled and can be changed to suit the image and the application area. Its demerits include introduction of blocking artifacts (since it works in the transform domain) which can be minimized to an extent but cannot be completely eliminated. So the main challenge for the future is to come up with energy efficient post processing techniques to further boost the performance of the DCT based algorithm. References [1] Chi-Chia Sun, Shanq-Jang Ruan, Member, IEEE, Mon-Chau Shie, Member, IEEE, and Tun-Wen Pai, Member, IEEE, Dynamic Contrast enhancement based Histogram Specification, Vol.51, November 2005. [2] S. D. Chen and A. R. Ramli, Minimum mean brightness error bihistogram equalization in contrast enhancement, IEEE Trans. Consumer Electron., vol. 49, no. 4, pp. 1310 1319, Nov. 2003. [3] A. Polesel, G. Ramponi, and V. J. Mathews, Image enhancement via adaptive unsharp masking, IEEE Trans. Image Processing, vol. 9, pp. 505 510, Mar. 2000. [4] A. Beghdadi and A. L. Negrate, Contrast enhancement technique based on local detection of edges, Comput. Vis. Graph. Image Process., vol.46, pp. 162 274, 1989. [5] Yoon, J.H. Ro, Y.M., Enhancement of the Contrast in Mammographic Images Using the Homomorphic Filter Method IEICE Transactions on Information and Systems, Vol.85-D, No.1. (2002) 291 297. [6] Jinshan Tang, Eli Peli, and Scott Acton,Image, Enhancement Using a Contrast Measure in the Compressed Domain, IEEE Signal Processing letters, VOL. 10.10,October 2003 [7] Ying Luo and Rabab K. Ward, Removing the Blocking Artifacts of Block-Based DCT Compressed Images, IEEE transactions on Image Processing, Vol. 12, No. 7, July 2003.