Image Interpolation using Collaborative Filtering 1,2 Qiang Guo, 1,2,3 Caiming Zhang *1 School of Computer Science and Technology, Shandong Economic University, Jinan, 250014, China, qguo2010@gmail.com 2 Shandong Provincial Key Laboratory of Digital Media Technology, Jinan, 250014, China 3 School of Computer Science and Technology, Shandong University, Jinan, 250101, China doi:10.4156/jdcta.vol5. issue4.2 Abstract An iterative method based on the block matching collaborative filtering is proposed for image interpolation. Unlike the conventional interpolation methods, the proposed method first utilizes the wavelet-based linear interpolation method to generate an initial estimate of the original image. Then the observation constraint provided by the given low-resolution image is enforced on the estimate to produce a combined estimate of the high-resolution image. In order to reduce the distortion of the combined estimate, the block matching collaborative filtering is exploited. Repeating the observation constraint and filtering process until the resulting image satisfying the stop criterion. Experimental results show that the proposed method obtains better performance than some existing interpolation methods. 1. Introduction Keywords: Image Interpolation, Wavelet Transform, Collaborative Filtering Image interpolation is the process of reconstructing a high-resolution (HR) image from its lowresolution (LR) capture. It has a wide range of applications in computer vision, high definition television (HDTV) displays, medical imaging, remote sensing and video surveillance, etc. Many interpolation methods already have been developed in the literature, but they suffer from one or more artifacts. Linear interpolation methods, such as bilinear, bicubic and cubic spline, deal with aliasing, blurring and ringing effects because they do not utilize any information relevant to edges and texture patterns in the original image [1]. Nonlinear interpolation methods incorporate more adaptive image models and priori knowledge which often improve linear interpolators. The edge-directed image interpolation takes advantage of the geometric priori information of image structures by performing the interpolation along edge directions [2]. While this method generates superior results, the direction of edge is difficult to identify, and its computational complexity is prohibitive due to the large window size used to estimate local covariance. Recently, wavelet-based interpolation methods, which enhance image resolution by estimating preserved high frequency information from the LR image, have been proposed by performing the interpolation in the wavelet domain [3-6]. Following this scheme, literatures [7,8] utilize the correlation between the inter-scale coefficients to estimate the unknown coefficients in the detailed scales. Mueller et al. proposed an iterative image interpolation method based on the wavelet and contourlet transforms, which improve the visual quality of resulting images by enforcing a sparsity constraint on the transform coefficients. These methods assume that the LR image is a lowpass output of the wavelet transform applied to the HR image. However, the aforementioned assumption is not tenable because the LR image also contains high frequency information. In practice, therefore, waveletbased interpolation methods are inefficient due to leading the resulting images to be oversmoothed. In this paper, we propose an image interpolation method based on the wavelet transform and block matching collaborative filtering. Firstly, the bicubic method is used to interpolate low frequency wavelet coefficients of the LR image, and lead to an initial estimate of the HR image. Secondly, the observation constraint provided by the given LR image is enforced on the estimate to generate a combined estimate of the HR image. Thirdly, the block matching collaborative filtering is used to - 12 -
reduce the distortion of the combined estimate. Repeating the second and third steps until the resulting image satisfying the stop criterion. The rest of this paper is organized as follows. Section 2 presents the proposed image interpolation method. In section 3, we report the experimental results and draw the conclusion in section 4. 2. Proposed interpolation method In this paper, we consider the following observation model for LR image y = Dx (1) where the observed LR image y is a downsampled version of the HR image x, and D represents the non-invertible downsampling operator. If the frequency support of x is included in [-π/2, π/2] 2 then linear interpolations can yield a perfect reconstruction of x. However, when the frequency support of x exceeds [-π/2, π/2] 2, linear interpolations produce Gibbs oscillations, blur and zigzag patterns along contours [9,10]. Therefore, we can use different interpolation methods for low frequency component and high frequency component of x, respectively. In this section, the bicubic interpolation method is used to interpolate the low frequency component which is separated by using the wavelet transform. Based on the low frequency image interpolated and the observation constraint, the high frequency component is implicitly interpolated by the block matching collaborative filtering. 2.1. Wavelet-based initial image interpolation Due to the advantage of linear interpolators for the low frequency component, we use the wavelet transform to separate the low frequency component of the LR image y, then the low frequency component is interpolated by the bicubic interpolation method. In general, one decomposition level of the wavelet transform W is sufficient to separate the low frequency information of y. The aforementioned process can be formulated by x W U T W y (2) 1 0 ( ( ( ( )))) where W represents the wavelet transform, U is the bicubic interpolator, T is a thresholding process which only preserves the low frequency wavelet coefficients, i.e., the approximation subband in the wavelet domain, and x 0 is an approximate estimation of the HR image x which only includes the low frequency component of x and has the same size with x. This process is illustrated in Figure 1(a). 2.2. Observation constraint In order to increase the high frequency component of x 0, we need to use the priori information y=dx again. A simple way to use this observation constraint is to fuse the LR image y and the estimate x 0 of the HR image. For simplification of expressions, let D denote a 0-1 downsampling matrix. Therefore, the process of fusing y and x 0 can be formulated as follows x y x.* D x.* D x.* D (3) 1 0 0 where.* denotes the element-wise multiplication operation. Figure 1(b) illustrates the observation constraint. - 13 -
2.3. Collaborative filtering The above described interpolation method is not perfect, which can introduce distortion in the interpolation process. In this paper, we consider the distortion as noise. Therefore, we use the collaborative filtering, which is proposed by Dabov et al. in [11], to reduce the distortion. The collaborative filtering exploits an enhanced image sparse representation in transform domain and nonlocal similarity between image blocks. It turns out to be a much more effective filter than other noise reduction methods based on image sparse representation [12,13]. The collaborative filtering mainly consists of the following steps (for more detail see [11]). Grouping by block matching. Use the block matching approach to find the blocks that are similar to the currently processed one and then stack them together in 3-D array. Figure 1. The block diagrams of process steps. (a) the wavelet-based interpolation, (b) the observation constraint model Collaborative filtering by thresholding transform coefficients. Apply a 3-D transform (2-D biorthogonal wavelet transform/2-d discrete cosine transform + 1-D Haar transform) on the formed group, reduce the noise by hard-thresholding or Wiener-filtering of the transform coefficients, produce estimates of all grouped blocks by applying the inverse 3-D transform on the filtered coefficients, and return the estimates of the blocks to their original positions. Global estimate by aggregation. Produce the final estimate of the noise-free image by aggregating all of the obtained block-wise estimates using a weighted average. - 14 -
2.4. Iterative image interpolation The proposed iterative image interpolation method can be summarized as follows. a) Use the estimate x 0 of the low frequency component (2) as the initial estimate of the HR image x. b) Enforce the observation constraint to increase the high frequency information by the equation (3). Let x k represent the estimate at the kth step. The new estimate x k+1 can be obtained by x k 1 y xk.* D x.* D xk.* D (4) c) Improve the quality of image interpolation by using the collaborative filtering to reduce the distortion introduced by the interpolation process. d) Repeat the steps b) and c) until the resulting image satisfying the stop criteria. Figure 2. The zoomed comparison of the Barbara image. (a) original image, (b) interpolated image by the wavelet-based method (23.17dB), (c) interpolated image by the bicubic method (22.79dB), (d) interpolated image by the proposed method (25.17dB). 3. Experimental results In this section, we performed several sets of experiments to validate the effectiveness of the proposed iterative interpolation method. For the experiments we use six 512 512 test images including Barbara, Lena, Man, Mandrill, Peppers, and Goldhill. In order to show the performance of interpolation methods, we first downsampled each test image by a factor of two to obtain the LR image y, and then interpolated y back to the original size. In the experiments, we use the symlet wavelet with length 8 for the wavelet transform and predetermine the maximum iteration number of the proposed method to be 30. - 15 -
We compare the results generated by our interpolation method with the bicubic interpolation and the simple wavelet-based interpolation. The performance of these interpolation methods is evaluated by visual quality, peak signal-to-noise ratio (PSNR) and structural similarity (SSIM) index [14], respectively. Visual comparisons for Barbara and Lena are shown in Figure 2 and Figure 3. It can be observed that the interpolated images of the proposed method exhibit less distortion than the other two interpolation methods. For quantitative comparison, the PSNR and SSIM values of interpolated results for test images are listed in Table 1 and Table 2, respectively. It is clearly shown that the proposed method substantially outperforms the wavelet-based interpolation and the bicubic interpolation. Figure 3. The zoomed comparison of the Lena image. (a) original image, (b) interpolated image by the wavelet-based method (28.39dB), (c) interpolated image by the bicubic method (30.22dB), (d) interpolated image by the proposed method (33.67dB). Table 1. Performance comparison with interpolation methods in term of PSNR Wavelet method Bicubic method Proposed method Barbara Lena Man Mandrill Peppers Goldhill 23.17 28.39 26.28 21.06 27.19 26.87 22.79 30.22 27.85 21.87 28.89 28.38 25.17 33.67 30.24 22.88 31.35 30.00 Table 2. Performance comparison with interpolation methods in term of SSIM Wavelet method Bicubic method Proposed method Barbara Lena Man Mandrill Peppers Goldhill 0.6274 0.7705 0.6907 0.4619 0.7360 0.6496 0.7209 0.8426 0.8038 0.6508 0.7853 0.7709 0.8016 0.8855 0.8565 0.7149 0.8459 0.8121-16 -
4. Conclusions This paper presents an iterative image interpolation method which uses the wavelet-based interpolation and the observation constraint to produce a combined estimate of the HR image and exploits the block matching collaborative filtering to reduce the distortion introduced by the interpolation process. Experimental results demonstrate that the proposed method outperforms the wavelet-based method and the bicubic interpolation method not only in the sense of visual quality but also of PSNR and SSIM. 5. Acknowledgment The authors would like to thank K. Dabov et al. for making their collaborative filtering implementation codes available to us for our experiments. This work is partially supported by the National Natural Science Foundation of China under Grant No. 11078013 and Projects of International Cooperation and Exchanges NSFC under Grant No. 61020106001. 6. References [1] T. Lehmann, C. Gönner and K. Spitzer, Survey: interpolations methods in medical image processing, IEEE Trans. on Medical Imaging, vol.18, no.11, pp.1049-1075, 1999. [2] X. Li and M.T. Orchard, New edge-directed interpolation, IEEE Trans. on Image Processing, vol.10, no.10, pp.1521-1527, 2001. [3] W. K. Carey, D. B. Chang and S. S. Hermami, Regularity-preserving image interpolation, IEEE Trans. on Image Processing, vol.8, no.9, pp.1293-1297, 1999. [4] G.K. Hassana, B. Zou, and S. Msami, Optimal color image enhancement using wavelet and K- means clustering, International Journal of Digital Content Technology and its Applications, vol. 5, no. 1, pp.112-122, 2011. [5] S.A. Ali, S. Vathsal and K.L. Kishore, CT image denoising technique using GA aided windowbased multiwavelet transformation and thresholding with the incorporation of an effective quality enhancement method, International Journal of Digital Content Technology and its Applications, vol.4, no.4, pp.75-87, 2010. [6] J. Li, H. Fan and D. Yuan, Fractal digital image inpainting, International Journal of Digital Content Technology and its Applications, vol.4, no.4, pp.140-150, 2010. [7] A. Temizel and T. Vlachos, Wavelet domain image resolution enhancement, IEE Proc.-Vision, Image, and Signal Processing, vol.153, no.1, pp.25-30, 2006. [8] Y. Piao, I. Shin and H.W. Park, Image resolution enhancement using inter-subband correlation in wavelet domain, In Proceedings. of the IEEE International Conference Image Processing, pp.445-448, 2007. [9] S. Mallat and G. Yu, Super-resolution with sparse mixing estimators, IEEE Trans. on Image Processing, vol.19, no.11, pp.2889-2900, 2010. [10] S. Mallat and G. Yu, Structured pursuits for geometric super-resolution, In Proceedings of the IEEE International Conference Image Processing, pp.1477-1480, 2009. [11] K. Dabov, A. Foi, V. Katkovnik, and K. Egiazarian, Image denoising by sparse 3-D transformdomain collaborative filtering, IEEE Trans. on Image Processing, vol.16, no.8, pp.2080-2094, 2007. [12] Q. Guo and S. Yu, Shearlet-based image denoising using a local multivariate prior model, ICIC Express Letters, vol.3, no.3, pp.751-756, 2009. [13] Z. Zhang, Y. Ohara, H. Toda, T. Miyake, and T. Imamura, De-noising method by combining adaptive line enhancer and complex discrete wavelet transform, ICIC Express Letters, vol.1, no.2, pp.145-151, 2007. [14] Z. Wang, A.C. Bovik, H.R. Sheikh, and E.P. Simoncelli, Image quality assessment: from error visibility to structural similarity, IEEE Trans. Image Processing, vol.13, no.4, pp.600-612, 2004. - 17 -