OBJECTIVE IMAGE QUALITY ASSESSMENT WITH SINGULAR VALUE DECOMPOSITION. Manish Narwaria and Weisi Lin

Transcription

1 OBJECTIVE IMAGE UALITY ASSESSMENT WITH SINGULAR VALUE DECOMPOSITION Manish Narwaria and Weisi Lin School of Computer Engineering, Nanyang Technological University, Singapore, {mani008, ABSTRACT In this paper, we develop a new image quality assessment scheme with the use of Singular Value Decomposition (SVD). In a previous work [], only the singular values out of SVD were used for quality assessment. We show that singular vectors are even more important than singular values for quality assessment. Hence, we use both singular vectors and values for comprehensive image quality evaluation, and demonstrate that the quality prediction is significantly improved. Extensive experimental results are reported as evidence of the efficiency of the proposed scheme, in comparison with the relevant existing methods, for three open and independent datasets with totally 2647 distorted images.. INTRODUCTION Auto-assessing the quality of digital image/video is one of the challenging problems in image and video processing and many practical situations, such as process evaluation, optimization (like for video encoder) and monitoring (e.g., in transmission and manufacturing sites). In addition, how to evaluate picture quality plays a central role in shaping most (if not all) visual processing algorithms and systems, as well as their implementation. With regards to visual quality assessment, the findings from psychovisual research provide some useful insights. It is well known that human visual system (HVS) is sensitive to spatial frequency and structures i.e. the HVS is highly adapted to extract structural information from visual scenes [2]. Therefore, during the recent years there has been a growing interest to take image structure into account for picture quality evaluation [2, 6, 7], because structural properties play a big role in the human perception and image recognition. A well cited perceptual metric based upon structure has been proposed by Wang and Bovik: firstly as the universal image quality index (UI) [7] and then its improved form known as the structural similarity index () [2]. In this paper, we use the method of Singular Value Decomposition to develop an image quality metric which is based on the loss of image structure and texture. Extensive experimental results on 3 open and independent databases are reported as evidence of the efficiency of the proposed scheme. 2. IMAGE ANALYSIS WITH SINGULAR VALUE DECOMPOSITION Singular value decomposition of image, A (of size r x c), can be written as A = U σ V T... () where U, V and σ represent the left singular vector matrix, the right singular vector matrix, and the diagonal matrix of singular values: U = [u u 2 u r ] V = [v v 2 v c ] σ=diag (σ, σ 2 σ t ) where u i and v k are column vectors while σ i is a singular value (i=,2,,r and k=,2, c); and σ > σ 2> > σ t with t= min(r,c). In the existing metric [], the change in σ has been used to evaluate perceptual quality. This is reasonable since singular values have been shown to be reasonably effective for image texture classification [2]. However, the metric ignores the singular vectors U, V which better represent image structure. The matrix UV T can be termed as the ensemble of the basis images and be used as a structural or geometrical representation of the image. Figure shows the geometrical framework denoted by U and V matrices. It is known from the perturbation analysis [3, 4] that U and V are sensitive to perturbations and can therefore be used to estimate the changes in the image (and thus, evaluate its quality). As can be seen from the images in Figure, adding distortion leads to quality degradation and the same is captured effectively in the distorted geometry denoted by U and V. For instance, in Fig. (c), Gaussian blurring of the original image damages the structure (especially the edges

2 are damaged in this case). The corresponding loss of the structure is well captured by the perturbed (blurred) singular vectors as shown in Fig. (c). Other types of distortions (perturbations) also affect the image structure/texture in different ways. The changes in U and V account for the structural degradations and thus provide an effective basis for assessing visual quality. Furthermore, Ref [5] demonstrates that the use of U and V improves the performance of an SVD based face recognition system. This is because U and V contain more information about image structure and geometry as compared to σ. In addition, we have conducted experiments and found that compared to changes in σ, the changes in U and V have more effect on the perceptual visual quality. An example is demonstrated on an image from the LIVE [8] dataset in Figure 2. We constructed images by perturbed U and V combined with unperturbed (i.e. original) σ, and vice-versa. We can see that images with perturbed U and V are perceptually more degraded than images with perturbed σ. We can note that images with unperturbed U, unperturbed V and perturbed σ always have higher PSNR value than the ones with perturbed U, perturbed V and unperturbed σ. This indicates that perturbation of U and V has a larger effect on the visual quality than the perturbation of σ. Therefore, the changes in U and V must be considered to assess the visual quality. U, V and σ contain complete information about the image and quality degradation is reflected by their changes. This forms the basis of image noise filtering methods [6, 7] which are based on the idea of eliminating the changes in U, V and σ to restore perceptual quality. This indicates that the changes in U, V and σ provide meaningful information about the visual quality. In [5] singular vectors were shown to be effective for image quality assessment. In this paper, we combine the changes in U, V and σ since this will provide a more comprehensive and effective premise for visual quality assessment in comparison to using singular vectors (see [5]) or singular values alone (see []). 3. THE PROPOSED VISUAL UALITY METRIC We decompose original image A (of size r x c) using () and the perturbed image A (p) (also of size r x c) as A (p) = U (p) σ (p) V (p) T We calculate α j = u j. u j (p) (2) β j = v j. v j (p)..(3) where α j (j= to t) represents the dot product between the unperturbed and the perturbed j th left (u j and u j (p) ) singular vectors and β j denotes that for the right singular vectors (v j and v j (p) ). Let γ j = α j + β j (4) The numerical measure for change in U and V is expressed as q s = {( t j= γ j m )} /m (5) where m (>) is a control parameter. A larger m puts more emphasis on large γ j value. We have used m = 2 in the experiments in this study. Since the magnitude of any singular vector is always unity whether perturbed or not (as aforementioned), measuring the angular deviation between the perturbed and the unperturbed singular vectors is an intuitively satisfying way of characterizing structural distortions. We can see that q s approaches 0 for large perturbations and equals 2 x (min(r, c)) /m for perfectly matched images. Thus if r = c = 52 and m = 2 the maximum value of q s would be 2* 52= Since the dynamic range for q s is large (for example [0, ] for the aforementioned case) in general, we use a logarithmic scale to reduce the dynamic range of q s and define S = ln (+q s ) (6) where the inclusion of the constant of avoids the infinite value when q s approaches 0. It may be noted that q s and s represent the same quantity though on different scale. For the measuring the change in singular values, we use the existing metric [], in which the difference between singular values of the reference and distorted image blocks is calculated: L = B j= D j B D where for each j th block we calculate mid (7) where U (p), V (p), σ (p) denote the left, right singular vectors and singular value matrix respectively for A (p). Our aim is to measure the deviations introduced in the singular vectors due to distortions. D j = b i= ( { σ σ i } p) 2 i

3 Original image Original U, V (a) JPEG image (b) Noisy image (c) Blurred image (a) JPEG U, V (b) Noisy U, V (c) Blurred U, V Figure Structure denoted by U, V in images where b defines the block size (in an image with size of r x c), B = r/b x c/b, and D mid represents the midpoint of the sorted D j s. For the experimental results reported in this paper, we have used the commonly used block size of 8x8 i.e. b=8. We now combine S and L to obtain the overall quality metric. It is found that a linear combination yields satisfactory performance as demonstrated by the experimental results. Moreover, combining S and L linearly keeps computational requirements to a minimum. The overall quality metric is therefore defined as = S - µ L (8) where µ is a user defined parameter which is chosen empirically (in this paper we have used µ = 5) and the negative sign is to accommodate the opposite trend of change of S and L., as defined by (8), can be considered as a composite measure of image quality. 4. EXPERIMENTAL RESULTS We have evaluated the performance of the proposed metric on 3 open and independent datasets with a total of 2647

4 (a) PSNR= 7.29 (a) PSNR= 9.74 (a2) PSNR= (b) PSNR= 2.42 (b) PSNR= 2.80 (b2) PSNR= Figure 2. Images (a), (b) are with WGN, JPEG distortions respectively, images (a), (b) are with perturbed U, V and original σ, images (a2), (b2) are with original U, V and perturbed σ. distorted images with varied distortion types have been evaluated as way to demonstrate the effectiveness and generality of the proposed scheme. A brief description of the datasets is given below. The LIVE [8] database contains 779 distorted images with five distortions type: JPEG, JPEG2000, white noise, Gaussian blur and Fastfading. Subjective evaluations were converted to difference scores (between the test and the reference) and then Z-scores and then scaled and shifted to the full range ( to 00) to compute the Difference Mean Opinion Scores (DMOS) for each image. The Toyama subjective database [9] contains 82 images of 768x52 pixels. Out of all, 4 were original images (24 bit/pixel RGB) in each group. The rest of the images were JPEG and JPEG2000 coded images (i.e. 84 compressed images for each type of distortion). Six quality scales and six compression ratios were respectively selected for the JPEG and JPEG2000 encoders. The TID dataset [0] is the largest and the most comprehensive dataset available for testing image quality metrics. It consists of 25 original reference images which have been processed by 7 different types of distortions which are listed in Table I. There are 4 distortion levels and thus it consists of a total of 700 (25 x 7 x 4) distorted images. The performance of is compared with [2], [3], [] and [4]. A nonlinear mapping between the objective model outputs and the subjective quality ratings was also employed []: we fitted objective scores to subjective scores via a fourparameter cubic polynomial a x 3 +a 2 x 2 +a 3 x+a 4 where a, a 2, a 3 and a 4 are determined by using the subjective scores and the objective outputs. The Pearson correlation coefficients, Spearman correlation coefficients and Root Mean Square Error (RMSE) are shown in Figure 3. We can see that generally performs better than the other metrics. To assess the statistical significance of each metric s performance relative to the other metrics, an F-test was

5 performed on the prediction residuals between the objective predictions (after non-linear mapping) and the subjective scores. Obviously smaller the residuals, the better the metric is. The test is based on an assumption of Gaussianity of the residual differences. Let σ 2 X and σ 2 denote the variances of the residuals from metrics X and respectively, and then the F-statistic with respect to metric is given by F= σ 2 X / σ 2. When F> Fcritical metric X has significantly larger residuals than metric at a given confidence level. Likewise when F < /F critical then metric has significantly larger residuals than metric X at that confidence level. F critical is computed based on the number of residuals and the confidence level. For the experimental results reported in this paper, we have used a 99% confidence level for the calculation of F critical values. The values of F > F critical indicated by boldfaced letters in Table II denote that has significantly smaller residuals than the corresponding metric and so performs statistically better than that metric. The F statistics for each metric s residuals tested against s residuals are reported in Table II. Values of F shown in boldface signify that with 99% confidence the metric has significantly larger residuals than. Since the method [] uses only σ, it is worth pointing out that is statistically better than with all databases; this demonstrates the effectiveness of incorporating U and V. We can also see that there is big margin for F to be compared with /F critical (even for the three cases in Table II, in which F<F critical ). The experimental results and the related statistical analysis therefore confirm that the use of U and V along with σ improves the quality prediction performance significantly. 5. CONCLUSIONS In this paper, we have proposed a scheme for effective image quality assessment with SVD. The major contribution is the use and justification of singular vectors together with singular values to gauge the changes in images due to distortions and hence assess their perceptual quality, better in line with subjective viewing groundtruth. A major advantage of the proposed scheme is that the proposed metric is general and effective in assessing visual quality of images with diverse distortion types as demonstrated through the experimental evaluation of 3 independent and comprehensive databases. The proposed scheme is found to perform better than the relevant existing metrics as shown by the experimental results and the related statistical analysis. Table I Distortion types in TID dataset No. Type of distortion Additive Gaussian noise 2 Different additive noise in color components 3 Spatially correlated noise 4 Masked noise 5 High frequency noise 6 Impulse noise 7 uantization noise 8 Gaussian blur 9 Image denoising 0 JPEG compression JPEG2000 compression 2 JPEG transmission errors 3 JPEG2000 transmission errors 4 Non eccentricity pattern noise 5 Local block-wise distortions of different intensity 6 Mean shift (intensity shift) 7 Contrast change Table II F statistics of different metrics with respect to Dataset/ LIVE Toyama TID Metric ( L ) F critical / F critical REFERENCES [] A.M. Eskicioglu, A. Gusev, and A. Shnayderman, An SVD-Based Gray-Scale Image uality Measure for Local and Global Assessment, IEEE Trans. Image Processing, vol. 5, no. 2, pp , [2] A. T. Targhi and A. Shademan, Clustering of singular value decomposition of image data with applications to texture classification, in Proc. SPIE Visual Communications and Image Processing, 2003, vol. 550, pp , Lugano, Switzerland, July [3] G. W. Stewart, Stochastic Perturbation Theory, SIAM Review, Vol. 32, No.4 pp , 990. [4] J. Liu, X. Liu, X. Ma, First Order Perturbation Analysis of Singular Vectors in Singular Value Decomposition IEEE Trans. on Signal Processing, vol. 56, no. 7, pp , July 2008.

6 LIVE Toyama TID 0.5 LIVE Toyama TID (a) (b) Toyama TID LIVE (c) (d) Figure 3 (a) Comparison of Pearson correlation coefficients, (b) Spearman correlation coefficients (c) Comparison of Root Mean Square Error for Toyama and TID datasets (d) Comparison of Root Mean Square Error for LIVE dataset [5] Yuan Tian, Tieniu Tan, Yunhong Wang, Yuchun Fang Do singular values contain adequate information for face recognition?, Pattern Recognition 36 (2003),pp [6] Z. Devˇci c, S. Lonˇcari c, SVD block processing for nonlinear image noise filtering, J. of Computing and Information Technology, 7(3) (999) [7] W. i, A. Morimoto, R. Ashino and R. Vaillancourt, Image denoising using spline and block singular value decomposition, Scientific Proceedings of Riga Technical University, 2 (2004), pp [8] H. R. Sheikh, Z. Wang, A. C. Bovik, and L. K. Cormack. Image and Video uality Assessment Research at LIVE. [Online] Available: [9] Y. Horita, Y. Kawayoke, and Z. M. Parvez Sazzad, Image quality evaluation database, [0] Ponomarenko N., Carli M., Lukin V., Egiazarian K., Astola J., Battisti F. Color Image Database for Evaluation of Image uality Metrics, Proc. of Intern. Workshop on Multimedia Signal Processing, Australia, pp , Oct [] VEG, Final Report from the Video uality Experts Group on the Validation of Objective Models of Video uality Assessment, Phase II August 2003 [Online]. Available: [2] Z. Wang, A. Bovik, H. Sheikh, and E. Simoncelli, Image quality assessment: From error visibility to structural similarity, IEEE Tran.s Image Process, vol. 3, no. 4, pp , Apr [3] H. R. Sheikh, A. C. Bovik, and G. de Veciana, An information fidelity criterion for image quality assessment using natural scene statistics, IEEE Trans. Image Process., vol. 4, no. 2, pp , Dec [4] Damon M. Chandler and Sheila S. Hemami, : A Wavelet-Based Visual Signal-to-Noise Ratio for Natural Images IEEE Trans. Image Processing, vol. 6, no. 9, pp , [5] Manish Narwaria and Weisi Lin, Scalable Image uality Assessment based on Structural Vectors in Proc. of IEEE International Workshop on Multimedia Signal Processing (MMSP 09), October 5-7, 2009, Rio de Janeiro, Brazil. [6] Z. Wang and A. C. Bovik. Modern Image uality Assessment. Morgan & Claypool Publishers, [7] Z. Wang and A. Bovik, A universal image quality index, IEEE Signal Processing Letters, Vol. 9, No. 3, pp. 8-84, March 2002.