A reduced reference image quality metric based on feature fusion and neural networks

Transcription

1 Univerity of Wollongong Reearch Online Faculty of Engineering and Information Science - Paper: Part A Faculty of Engineering and Information Science 2011 A reduced reference image quality metric baed on feature fuion and neural network Aladine Chetouani Univerite Pari Azeddine Beghdadi Univerite Pari Mohamed Deriche King Fahd Univerity Abdeelam Bouzerdoum Univerity of Wollongong, bouzer@uow.edu.au Publication Detail Chetouani, A., Beghdadi, A., Deriche, M. & Bouzerdoum, A. (2011). A reduced reference image quality metric baed on feature fuion and neural network. 19th European Signal Proceing Conference, EUSIPCO 2011 (pp ). Reearch Online i the open acce intitutional repoitory for the Univerity of Wollongong. For further information contact the UOW Library: reearch-pub@uow.edu.au

2 A reduced reference image quality metric baed on feature fuion and neural network Abtract A Global Reduced Reference Image Quality Metric (IQM) baed on feature fuion uing neural network i propoed. The main idea i the introduction of a Reduced Reference degradation-dependent IQM (RRIQM/ D) acro a et of common ditortion. The firt tage conit of extracting a et of feature from the waveletbaed edge map. Such feature are then ued to identify the type of degradation uing Linear Dicriminant Analyi (LDA). The econd tage conit of fuing the extracted feature into a ingle meaure uing Artificial Neural Network (ANN). The reult i a degradation- dependent IQM meaure called the RRIQM/ D. The performance of the propoed method i evaluated uing the TID 2008 databae and compared to ome exiting IQM. The experimental reult obtained uing the propoed method demontrate an improved performance even when compared to ome Full Reference IQM. Keyword neural, fuion, feature, network, metric, reference, quality, reduced, image Dicipline Engineering Science and Technology Studie Publication Detail Chetouani, A., Beghdadi, A., Deriche, M. & Bouzerdoum, A. (2011). A reduced reference image quality metric baed on feature fuion and neural network. 19th European Signal Proceing Conference, EUSIPCO 2011 (pp ). Thi conference paper i available at Reearch Online:

3 19th European Signal Proceing Conference (EUSIPCO 2011) Barcelona, Spain, Augut 29 - September 2, 2011 A REDUCED REFERENCE IMAGE QUALITY METRIC BASED ON FEATURE FUSION AND NEURAL NETWORKS Aladine Chetouani 1, Azeddine Beghdadi 1, Mohamed Deriche 2, and Abdeelam Bouzerdoum 3 1 L2TI, Intitut Galilée, Univerité Pari13 99, av. J-B Clément, 93430, Villetaneue, France 2 EE. Dept., King Fahd Univerity P.O. BOX 1427, KFUPM, Dhahran, 31261, Saudi Arabia 3 School of Electrical, Computer & Telecommunication Engineering, Univerity of Wollongong, Wollongong, NSW 2522, Autralia aladine.chetouani@univ-pari13.fr, azeddine.beghdadi@univ-pari13.fr, mderiche@kfupm.edu.a, a.bouzerdoum@uow.edu.au ABSTRACT A Global Reduced Reference Image Quality Metric (IQM) baed on feature fuion uing neural network i propoed. The main idea i the introduction of a Reduced Reference degradation-dependent IQM (RRIQM/D) acro a et of common ditortion. The firt tage conit of extracting a et of feature from the wavelet-baed edge map. Such feature are then ued to identify the type of degradation uing Linear Dicriminant Analyi (LDA). The econd tage conit of fuing the extracted feature into a ingle meaure uing Artificial Neural Network (ANN). The reult i a degradation-dependent IQM meaure called the RRIQM/D. The performance of the propoed method i evaluated uing the TID 2008 databae and compared to ome exiting IQM. The experimental reult obtained uing the propoed method demontrate an improved performance even when compared to ome Full Reference IQM. 1. INTRODUCTION In practical application, image undergo different type of proceing including acquiition, tranmiion or compreion, which often generate ome annoying ditortion. The mot annoying impairment are blocking, ringing, and blur artifact. Blocking manifet a artificial horizontal and vertical dicontinuitie in ome block-baed compreion method. Blur i alo a common artifact which affect the detail of the image due to everal phenomena uch a defocuing or filtering. Ringing i another annoying degradation which i eentially due to quantization, and i generally defined a noie around edge point or in contrated tranition. To quantify the viual impact of thee annoying degradation, a number of ubjective and objective meaure have been propoed [1]. Subjective evaluation i regarded a the mot reliable approach for aeing image quality. Unfortunately, ubjective method are complex, time conuming and impractical for real-time application. Undertanding and applying the knowledge of human viual perception i recognized a the mot promiing approach for developing objective method conitent with human judgement. Three categorie of image quality aement are commonly ued: Full-Reference (FR), No-Reference (NR) and Reduced Reference (RR) method. Full Reference method need both the original image and it degraded verion. Mot of the exiting Image Quality Metric (IQM), uch a PSNR, SSIM [2], SNRWAV [3], VIF [4] and o on, belong to thi family. However, in real application the original image i not alway available. Hence, NR method are mot appropriate a they require only the degraded image. During the lat decade, a number of NR metric have been propoed [5]-[8]. However, NR-IQM are limited by the type of the degradation contained in the image. Indeed, NR-IQM are developed for particular artifact, hence limiting their ue to certain application only. Reduced Reference approache preent a good compromie between FR and NR approache. Only ome feature, uch a edge or ome viual decriptor, are extracted from the original and the degraded image. From the tructural information conveyed by thee decriptor, a metric i then derived and ued a an IQM. There are relatively few ucceful RR-IQM that have been dicued in the literature, including the popular RRIQA [9]-[10]. In thi paper, we focu our work on RR-IQM approache. In particular, we propoe a new metric baed on a multicale feature fuion cheme uing an Artificial Neural Network (ANN) model. The main idea developed here i to compute an RR-IQM which i degradation-dependent (RR-IQM/D). The type of the degradation contained in the image i initially detected uing imple Linear Dicriminant Analyi, (LDA) baed on the extracted feature from the edge map of the image and it ditorted verion. The remainder of the paper i organized a follow. Section 2 dicue the feature extraction and ditortion claification tage, and preent the propoed neural network fuion method. The experimental reult are preented in Section 3, followed by concluding remark and ome perpective in Section 4. EURASIP, ISSN

4 2. THE PROPOSED APPROACH The flowchart of the propoed method i preented in Fig.1. For a given type of degradation, a number of feature are extracted from the original image and it degraded verion. Thee feature are then combined uing a neural network cheme in order to etimate a local image quality index, depending on the degradation type. Thi degradationdependent IQM i called RR-IQM/D. The type of the ditortion contained in the degraded image i determined through an initial claification tep. In what follow, the feature extraction proce, the fuion and claification tep are dicued in more detail. Sender ide Image Wavelet Decompoition (3 level) Edge Map Computation (for each level) Extracted Feature Figure 2 The feature extraction tage. Original Image Tranmiion Channel Degraded Image Feature Extraction Ancillary Channel For a given image, a imple wavelet decompoition i performed. Here, the number of decompoition level i fixed to 3 (additional layer did not improve the reult). Then, an edge map i derived at each decompoition level k [11] uing: ( ) ( ) ( ) EMap( k) = CH k + CV k + CD k (1) where CH, CV and CD denote the horizontal, vertical, and diagonal detail, repectively. Feature Extraction Degradation type identification Fig. 3 illutrate the wavelet decompoition and the aociated edge map for an image containing a multitude of patial frequency component. Note that the approximation coefficient are not conidered in the edge map, ee Eq. (1). CH RR-IQM/D CV CD Receiver ide (a) EMap (b) Figure 1 Flowchart of the propoed RR-IQM/D. 2.1 Feature Extraction Stage The feature are extracted from the edge map image derived from the wavelet domain a illutrated in Fig.2. (c) Figure 3 a) Tet image, b) Wavelet decompoition, c) Edge map (EMAP for k=1) Once the different edge map are obtained, a et of key feature i derived from it. Here, we ue the mean and the tandard deviation a primary decriptor. Six feature are e- 590

5 lected from each image: 2 feature per decompoition level. Overall, 12 feature are extracted, 6 from the original image at the tranmitter ide, and 6 from the degraded image at the receiver end. For a given pair of image (original and degraded image), the extracted feature are concatenated in a ingle feature vector a follow:. Input μ EMAP (1) μ (1) EM AP d μ EM AP (2) μ EM AP (2) d μ EM AP (3) μ EM AP (3) = d σ EMAP (1) σ EMAP (1) d σ EMAP (2) σ EMAP (2) d σ EMAP (3) σ EMAP (3) d (2) μ ( ) where EMAP k μ ( ) and EMAP k d denote the mean of the k th edge map of the original and the degraded image, σ EMAP ( k ) σ ( ) repectively, and EMAP k and d repreent the tandard deviation of k th edge map of the original image and it degraded verion, repectively. with x N c c 1 c c 1 n = σ ω nu xu u = 0 σ. (3) 2V e 1 ( V ) = (4) 2V e + 1 where N c denote the number of neuron in Layer c, ω nu i c 1 xu the weight of the neuron n, i the output of the neuron u in Layer c-1, and (.) i the activation function. For a given ditortion, the ANN model conit of 12 input and 1 output. The input correpond to the extracted feature and the output i the predicted ubjective core (Mean Opinion Score (MOS)). The number of hidden layer i fixed here to 1. The input and output value are firt caled to the range [-1, +1]. For the output, value -1 and +1 denoting wort and bet quality repectively. The back propagation algorithm i ued to train the MLP. The different parameter of the MLP model are diplayed in Table 1. Note that for each type of degradation, we deign a different ANN model. TABLE 1. ANN model for each conidered ditortion c 2.2 A Reduced Reference Image Quality Metric For a given ditortion, the RR-IQM/D i obtained by combining the extracted feature uing an Artificial Neural Network (ANN). Here, a imple Multi Layer Perceptron (MLP) i ued (ee Fig. 4). Input Hidden layer Output Activation function Learning tep 12 (i.e. number of extracted feature). 1 (the number with). 1 (i.e. MOS). Sigmoid Back propagation with cro validation. Input Layer c-1 Layer c c 1 ω nj + Layer c+1 Output 2.3 Identifying the Degradation Type Before uing the propoed IQM dicued above, we need to identify the type of degradation affecting the tet image. Different claifier can be ued. In thi tudy, a imple Linear Dicriminant Analyi (LDA) claifier i ued. The extracted feature above are ued a input pattern in the training tage. After projecting the pattern over the LDA orthogonal bai, a imple minimum ditance claifier i ued for ditortion identification. 3. RESULTS AND DISCUSSION The output value given by: Figure 4 Baic tructure of the MLP ANN. c x n of a given neuron n at Layer c i To evaluate the performance of the new propoed image quality index, we ued the popular Tampere Image Databae (TID 2008) [14]. Thi databae ue 25 reference image and contain 17 type of degradation with 100 image per ditortion (with their aociated MOS). Here, we focu only on ome common ditortion, namely Gauian noie (GN), 591

6 Gauian Blur (GB), JPEG (i.e. blocking) and JPEG2000 (i.e. Ringing) artifact a hown in Table 2. All image from the TID 2008 databae are ued through a cro-validation approach. Here, at each fold, 80% of the image databae i ued for the training tep and 20% for the tet tep (5-fold cro validation). All image are then ued to tet the efficiency of the propoed method. Table 2. Degradation conidered in thi work Degradation Type 1 Additive Gauian noie (WN) 2 Gauian blur (GB) 3 JPEG compreion (JPEG) 4 JPEG2000 compreion (JP2K) To evaluate the performance of the propoed approach, we compared our reult to thoe of RRIQA (Reduced Reference Image Quality Aement) metric propoed in [9]. Thi method i baed on a tatitical model in the wavelet domain. The hitogram of ome elected wavelet ub-band are firt modelled (Gauian model). The Kullback-Leiber ditance i then ued to evaluate the imilarity index between the original and the degraded image. The propoed method i alo compared to 2 full reference meaure, namely the SSIM [2] (tructural-baed) and the VIF [4] (mutual information-baed). Thee metric are the mot ued and are available in [15]. Table 3. TID 2008 databae: Pearon and Spearman correlation for each conidered ditortion, Gauian Noie (GN), Gauian Blur (GB), JPEG and JPEG2000. IQM Pearon Correlation GN GB JPEG JPEG2000 SSIM [2] VIF [4] RRIQA [9] RR-IQM/D Spearman Correlation SSIM [2] VIF [4] RRIQA [9] RR-IQM/D Table 3 preent the Pearon and Spearman correlation coefficient obtained for four different IQM, including the propoed method RR-IQM/D. The propoed IQM index clearly outperform the reduced reference RRIQA [9] in all conidered degradation, except for blur where both method give imilar reult. Even though the propoed method doe not exploit the full reference data, it outperform ome traditional FR-IQM. Indeed, better reult are obtained for Gauian blur, JPEG and JPEG2000 degradation compared to thoe obtained with the full reference metric SSIM and VIF. Once the efficiency of the propoed IQM cheme i validated, the claification tep i then aeed in term of claification accuracy. The overall claification ytem i compoed of 4 clae (i.e. 4 degradation type) and 12 input (ee equation 2). Table 4 how the confuion matrix obtained for each conidered degradation. The mean percentage i equal to 93.5% with ome confuion between clae, particularly between GB and JPPEG2000 ditortion. Thi i eentially due to the fact that blur appear alo in JP2K compreed image. True Ditortion Predicted Ditortion WN GB JPEG JP2K WN GB JPEG JP2K CONCLUSION In thi tudy, a new approach for Reduced Reference Image Quality evaluation i introduced. We how that the multicale feature extraction tage baed on the wavelet decompoition lead to robut viual decriptor of the image tructural information. The fuion of thi viual information through a neural network proce offer an efficient image quality metric which can uccefully be ued for real-time application uch a image quality monitoring. In the future, we will analyze the impact of wrong ditortion type claification. Alo, the propoed method will be extended to other artifact uch a color ditortion. REFERENCES [1] H.R. Wu and K.R. Rao, Digital Video Image Quality and Perceptual Coding, CRC Pre, [2] Z. Wang, E.P. Simoncelli and A.C. Bovik, Multi-cale tructural imilarity for image quality aement, IEEE Ailomar Conference on Signal, Sytem and Computer, 2003, Vol. 2, pp [3] A. Beghdadi and B. Pequet-Popecu, A New Image Ditortion Meaure Baed Wavelet Decompoition, 2003, IEEE International Sympoium on Signal Proceing and It Application, 2003, pp

7 [4] H.R. Sheikh and A.C. Bovik, Image information and viual quality, IEEE Tranaction on Image Proceing, 2006, Vol.15, no.2, pp [5] Z. Wang, H.R. Sheikh and A.C. Bovik, No-reference perceptual quality aement of JPEG compreed image, IEEE International Conference on Image Proceing, 2002, pp [6] F. Crête, Etimer, meurer et corriger le artefact de compreion pour la téléviion, Univerité Joeph Fourier, [7] A. Chetouani, G. Motafaoui and A. Beghdadi, A New Free Reference Image Quality Index Baed on Perceptual Blur Etimation, IEEE Pacific-Rim Conference on Multimedia, 2009, pp [8] A. Chetouani, A. Beghdadi and M. Deriche, A new free reference image quality index for blur etimation in the frequency domain, IEEE International Sympoium on Signal Proceing and Information Technology, 2009, pp [9] Z. Wang and E.P. Simoncelli, Reduced-reference image quality aement uing a wavelet-domain natural image tatitic model, Human Viion and Electronic Imaging X, Proc. SPIE, 2005, Vol. 5666, pp [10] M. Carnec, P. Le Callet and B. Dominique, Objective quality aement of color image baed on a generic perceptual reduced reference, Signal Proceing: Image Communication, 2008, Vol. 23, Iue 4, pp [11] H. Tong, M. Li, H. Zhang and C. Zhang, Blur detection for digital image uing wavelet tranform, IEEE International Conference on Multimedia and Expoition, 2004, Vol. 1, pp [12] A. Chetouani, A. Beghdadi and M. Deriche, Statitical Modeling of Image Degradation Baed on Quality Metric, International Conference on Pattern Recognition, 2010, pp [13] N. Peter, J. Belhumeur, P. Hepanha and D. Kriegman, Eigenface v. Fiherface: Recognition Uing Cla Specific Linear Projection, IEEE Tranactionon Pattern Analyi and Machine Intelligence, 1997, pp [14] N. Ponomarenko, M. Carli, V. Lukin, K. Egiazarian, J. Atola and F. Battiti, Color Image Databae for Evaluation of Image Quality Metric, International Workhop on Multimedia Signal Proceing, 2008, pp [15] 593