Evaluating the Visual Quality of Watermarked Images

Transcription

1 Evaluating the Visual Quality of Watermared Images Alesandr Shnayderman and Ahmet M Esicioglu Department of Computer and Information Science CUNY Broolyn College, 2900 Bedford Avenue, Broolyn, NY ABSTRACT A recent image quality measure, M-SVD, can express the quality of distorted images either numerically or graphically Based on the Singular Value Decomposition (SVD), it consistently measures the distortion across different distortion types and within a given distortion type at different distortion levels The SVD decomposes every real matrix into a product of three matrices A = USV T, where U and V are orthogonal matrices, U T U = I, V T V = I and S = diag(s 1, s 2, ) The diagonal entries of S are called the singular values of A, the columns of U are called the left singular vectors of A, and the columns of V are called the right singular vectors of A M-SVD, as a graphical measure, computes the distance between the singular values of the original image bloc and the singular values of the distorted image bloc, where n x n is the bloc size If the image size is x, we have (/n) x (/n) blocs The set of distances, when displayed in a graph, represents a distortion map The numerical measure is derived from the graphical measure It computes the global error expressed as a single numerical value In this paper, we will extend the SVD-based image quality measure to evaluate the visual quality of watermared images using several watermaring schemes Keywords: Singular Value Decomposition, image quality, watermaring, Discrete Wavelet Transform, Discrete Cosine Transform, Discrete Fourier Transform, Quantization Index Modulation, Lapped Orthogonal Transform 1 INTRODUCTION Measurement of image quality is a challenging problem in many image processing fields ranging from lossy compression to printing The quality measures in the literature can be classified into two groups: Subjective and objective Subjective evaluation is cumbersome as the human observers can be influenced by several critical factors such as the environmental conditions, motivation, and mood The most common objective evaluation tool, the Mean Square Error (MSE), is very unreliable, resulting in poor correlation with the human visual system (HVS) [1] In spite of their complicated algorithms, the more recent HVS-based objective measures do not appear to be superior to the simple pixel-based measures lie the MSE, Pea Signal-to-Noise Ratio (PSNR), or Root Mean Squared Error (RMSE) It is argued that an ideal image quality measure should be able to describe [2]: (1) amount of distortion, (2) type of distortion, and (3) distribution of error Undoubtedly, there is a need for an objective measure that provides more information than a single numerical value Only a few multi-dimensional measures exist in the relevant literature today [2] Image quality measures can be classified using a number of criteria such as the type of domain (pixel or transform), the type of distortion predicted (noise, blur, etc), and the type of information needed to assess the quality (original image, distorted image, etc) Table 1 gives a classification based on these three criteria, and includes representative examples of recently published papers Measures that require both the original image and the distorted image are called fullreference or non-blind methods, measures that do not require the original image are called no-reference or blind methods, and measures that require both the distorted image and partial information about the original image are called reduced-reference methods Corresponding author s address: esicioglu@scibroolyncunyedu 1

2 We have recently developed a new measure, M-SVD, that can express the quality of distorted images either numerically or graphically [3,4,5] Based on the Singular Value Decomposition (SVD), it consistently measures the distortion across different distortion types and within a given distortion type at different distortion levels Comparison with the state-ofthe-art metrics Q and MSSIM shows that the performance of M-SVD is superior Every real matrix A can be decomposed into a product of 3 matrices A = USV T, where U and V are orthogonal matrices, U T U = I, V T V = I and S = diag(s 1, s 2, ) The diagonal entries of S are called the singular values of A, the columns of U are called the left singular vectors of A, and the columns of V are called the right singular vectors of A This decomposition is nown as the Singular Value Decomposition of A It is one of the most useful tools of linear algebra with several applications to multimedia, including image compression and watermaring Table 1 Classification of image quality measures Publication\Criterion Domain type Type of distortion predicted Type of information needed Van der Ween, Nachtegael and Kerre, 2003 [6] Beghdadi and Pesquet- Popescu, 2003 [7] Pixel Salt and pepper noise, enlightening and darening Full-reference Discrete Wavelet Gaussian noise, grid Full-reference Transform pattern, JPEG compression Bovi and Liu, 2001[8] Discrete Cosine Transform JPEG compression No-reference Wang, Bovi and Evans, Discrete Fourier Transform JPEG compression No-reference 2000 [9] Wang, and Bovi, 2002 Pixel JPEG compression No-reference [10] Marziliano, Dufaux, Winler and Ebrahimi, 2002 [11] Pixel Gaussian blur, JPEG 2000 compression No-reference Ong, Lin, Lu, Yang, Yao, Pan, Jiang and Moschetti, 2003 [12] Meesters and Martens, 2002 [] Carnec, Le Callet and Barba, 2003 [14] Pixel Gaussian blur, JPEG 2000 compression No-reference Pixel JPEG compression No-reference Pixel JPEG compression, JPEG 2000 compression Reduced-reference M-SVD is a bivariate measure that computes the distance between the singular values of the original image bloc and the singular values of the distorted image bloc: D i = [ ˆ 1 ( 2 Sqrt n i = s i s ) ], i where s i are the singular values of the original bloc, ŝ i are the singular values of the distorted bloc, and n x n is the bloc size If the image size is x, we have (/n) x (/n) blocs The set of distances, when displayed in a graph, represents a distortion map The numerical measure is derived from the graphical measure It computes the global error expressed as a single numerical value, depending on the distortion type: where M-SVD = D mid represents the mid point of the sorted Di s ( / n) x( / n) i = 1 D D i mid, ( / n) x( / n) 2

3 In this paper, we will extend the SVD-based image quality measure to evaluate the visual quality of watermared images We have developed three watermaring algorithms: A non-blind scheme using DWT and SVD [15] A non-blind scheme using DWT [16] A semi-blind scheme using DFT [17] We also have access to the benchmaring suite [18] in the School of Electrical and Computer Engineering at Purdue University On this web site, there are several watermaring schemes including: A non-blind scheme using DCT [19] A semi-blind scheme using DWT [20] A blind scheme based on the Quantization Index Modulation Algorithm (QIM) [21] A non-blind scheme using the Lapped Orthogonal Transform (LOT) based adaptive algorithm [22] We will use M-SVD to compare the visual quality of watermared images obtained by the above algorithms The embedding and detection algorithms for the watermaring schemes are as follows: Reference 15: Watermar embedding 1 Using DWT, decompose the cover image A into 4 subbands: LL, HL, LH, and HH 2 Apply SVD to each subband image: T A = U Σ V, = 1,2,3,4, where denotes LL, HL, LH, and HH bands, a a a and λ i, i=1,,n are the singular values of Σ a 3 T Apply SVD to the visual watermar: W U V wi, i = 1,,n are the singular values of Σ W W W W 4 Modify the singular values of the cover image in each subband with the singular values of the visual watermar: λ = λ + α λ, i = 1,,n, and = 1,2,3,4 i i wi T 5 Obtain the 4 sets of modified DWT coefficients: A = U Σ V, = 1,2,3,4 a a a 6 Apply the inverse DWT using the 4 sets of modified DWT coefficients to produce the watermared cover image A Watermar extraction 1 Using DWT, decompose the watermared (and possibly attaced) cover image A into 4 subbands: LL, HL, LH, and HH 2 Apply SVD to each subband image: T A = U Σ V, = 1,2,3,4, where denotes the attaced LL, HL, LH, a a a and HH bands 3 Extract the singular values from each subband: ( λwi = λi λi ) / α,, i = 1,,n, and = 1,2,3,4 4 Construct the four visual watermars using the singular vectors: W = U T V W Σ W W 3

4 Reference 16: Watermar embedding (first level decomposition) 1 Using two-dimensional separable dyadic DWT, obtain the first level decomposition of the cover image I 2 Modify the DWT coefficients V in the LL, HL, LH, and HH bands: Vw, = V + α W, i,j=1,,n, and =1,2,3,4 3 Apply inverse DWT to obtain the watermared cover image I w Watermar extraction (first level decomposition) 1 Using two-dimensional separable dyadic DWT, obtain the first level decomposition of the watermared (and possibly attaced) cover image I w 2 Extract the binary visual watermar from the LL, HL, LH, and HH bands: W = ( Vw, V ) / α, i,j=1,,n, and =1,2,3,4 3 If W > 0 5, then W = 1else W = 0 Watermar embedding (second level decomposition) 1 Using two-dimensional separable dyadic DWT, obtain the second level decomposition of the cover image I 2 Modify the DWT coefficients V in the LL2, HL2, LH2, and HH2 bands: Vw, = V + α W, i,j=1,,n, and =1,2,3,4 3 Apply inverse DWT to obtain the watermared cover image I w Watermar extraction (second level decomposition) 1 Using two-dimensional separable dyadic DWT, obtain the second level decomposition of the watermared (and possibly attaced) cover image I w 2 Extract the binary visual watermar from the LL2, HL2, LH2, and HH2 bands: W = ( Vw, V ) / α, i,j=1,,n, and =1,2,3,4 3 If W > 0 5, then W = 1else W = 0 Reference 17: Watermar embedding 1 Convert the NxN RGB cover image to YUV 2 Compute the DFT of the luminance layer 3 Move the origin to the center 4 Obtain the magnitudes of DFT coefficients 5 Divide the NxN matrix of magnitudes into four (N/2)x(N/2) matrices M ul, M ur, M ll, M lr ul: upper left, ur: upper right, ll: lower left, lr: lower right 6 Define three frequency bands: low, middle, and high 7 Embed a visual binary watermar in these three bands by determining the embedding locations 4

5 8 In each band, repeat the following steps for all pairs of magnitudes: a Get a pair of magnitudes a and b: a from matrix M ul, and the corresponding magnitude b from matrix M ur b Compute the mean m = (a+b)/2, and choose the value of the parameter p c Embedding bit 1: If a < m-(p/2m) then do not modify a and b else a=m-(p/2m) and b=m+(p/2m) d Embedding bit 0: If a > m+(p/2m) then do not modify a and b else a=m+(p/2m) and b=m-(p/2m) 9 Copy the modified magnitudes in matrix M ul to matrix M lr 10 Copy the modified magnitudes in matrix M ur to matrix M ll 11 Obtain the DFT coefficients of the entire image using the modified magnitudes 12 Compute the inverse DFT to obtain the luminance layer of the watermared cover image Convert the YUV watermared cover image to RGB Watermar extraction: 1 Convert the NxN watermared (and possibly attaced) RGB cover image to YUV 2 Compute the DFT of the luminance layer 3 Move the origin to the center 4 Obtain the magnitudes of DFT coefficients 5 Divide the NxN matrix of magnitudes into four (N/2)x(N/2) matrices M ul, M ur, M ll, M lr 6 Use the three frequency bands and the embedding locations defined in the embedding process: low, middle, and high 7 For each pair of magnitudes in each band, if a > b then bit=0 else bit=1 Reference 19: Watermar embedding 1 Compute the DCT of the NxN gray scale cover image I 2 Embed a sequence of real values X = x 1, x 2,, x n, according to N(0,1), into the n largest magnitude DCT coefficients, excluding the DC component: ' v = v (1 + α x ), i 1,, n i i i = 3 Compute the inverse DCT to obtain the watermared cover image I Watermar detection 1 Compute the DCT of the watermared (and possibly attaced) cover image I 2 Extract the watermar 3 Evaluate the similarity of 4 If sim ( X, i ( i i i = n X : x = v v ) / α v, i 1,, X and X using sim ( X, X ) = X ) > T, a given threshold, the watermar exists X X ( X X 1/ 2 ) 5

6 Reference 20: Watermar embedding 1 Compute the DWT of the NxN gray scale cover image I 2 Exclude the low pass DWT coefficients 3 Embed the watermar into the DWT coefficients t i > T 1 : t i = t i + α t i x i, where i runs over all DWT coefficients > T 1 4 Replace T = {t i } with T = {t i } in the DWT domain 5 Compute the inverse DWT to obtain the watermared cover image I Watermar detection 1 Compute the DWT of the watermared (and possibly attaced) cover image I 2 Exclude the low pass DWT coefficients 3 Select all the DWT coefficients higher than T 2 4 Compute the sum z = 1 i t i, where i runs over all DWT coefficients > T 2, y i represents either the real M i= 1 watermar or a fae watermar, t i represents the watermared and possibly attaced DCT coefficients 5 α Choose a predefined threshold T z = 2 t i M i= 1 6 If z exceeds T z, the conclusion is the watermar is present Reference 21: Watermar embedding The encoder embeds the watermar into the host signal: S(x;m) = q (x + d(m)) d(m), where x is the host signal, q (x) is a uniform quantizer with quantization interval, d(m) is the dither signal, m=0,1 is the message bit The dither signal d(0) and d(1) are constructed with the constraint d(1) = d(0) + /2 if d[,0] < 0, d(1) = d(0) - /2 if d[,0] >= 0 This constraint ensures that the two dithered signals S(x;0) and S(x;1) have the maximum distance from each other d(0) is the output from a uniform pseudo random number generator with range [- /2, /2] Watermar detection The decoder decodes the watermar message m from the received composite signal S(x;m) without having access to the original host signal x The message m that should be transmitted is the index for the quantizer used for quantizing the host-signal vector While retrieving the hidden information, one evaluates a distance metric to all quantizers The index of the quantizer with the smallest distance contributes to the message m 6

7 Reference 22: Watermar embedding 1 The Lapped Orthogonal Transform (LOT) divides the NxN cover image I into overlapping 16x16 blocs, and maps each bloc into an 8x8 bloc in the frequency domain 2 Introduce a perceptual analysis module to extract from each bloc a feature nown as the Texture Masing Energy (TME) The blocs are classified into one of the following four categories according to the TME: texture, finetexture, edge, and flat region The edge blocs are further categorized according to the direction of the edges The watermar embedding energy in each bloc is adjusted accordingly to adapt to the sensitivities of the HVS 3 The quantization matrix for the th bloc is Q L ()=Q S xm bloc ()xm edge (), where x denotes the element-wise multiplication Q S is the standard quantization matrix used in JPEG, and M bloc (), M edge () are used to adjust the quantization steps in Q S for the th bloc 4 In each bloc, the 5 most visually important AC coefficients bear the watermar: ' X ( in,, jn, ) = X ( in,, jn, ) + αql ( in,, jn, ) w( n), where X is the LOT coefficient of the original image, X is the corresponding watermared LOT coefficient, w(n) is the watermar element, and (i n,, j n, ) denotes the specified position to bear the watermar in the th bloc 5 Compute the inverse LOT to obtain the watermared cover image I Watermar detection 1 Extract the watermar using the LOT coefficients of the watermared (and possibly attaced) cover image I and the LOT coefficients of the original cover image I: ˆ ' w( n) = ( X ( in,, jn, ) X ( in,, jn, )) /( αql ( in,, jn, )) 1 Compare the extracted watermar Wˆ and the original watermar W using the formula sim( W, Wˆ ) = W Wˆ W Wˆ 2 If the value of sim ( W, W ˆ ) is larger then a specified threshold, the watermar is present 2 EXPERIMENTS The original 512x512 images used with the watermaring algorithms are shown in Figure 1 Peppers (luminance layer) Goldhill (gray scale) Lena (gray scale) Figure 1 Original cover images The luminance layer of each full color watermared image, the watermared gray scale images, the corresponding distortion maps, and the values of the numerical measure are given in Figure 2 7

8 Watermared image Distortion map and global error Reference 15 M-SVD = 9684 S S Reference 16 (first level) M-SVD = Reference 16 (second level) M-SVD = 33 S 10 5 S Reference 17 M-SVD = 1298 Figure 2 Watermared images, distortion maps, and numerical measure values 8

9 Watermared image Distortion map and global error Reference 19 M-SVD = S Reference 20 M-SVD = 15 S Reference 21 M-SVD = 0795 S S Reference 22 M-SVD = 0385 Figure 2 Continued 9

10 For the Quantization Index Modulation (QIM) algorithm [21], the text embedded in the color image Peppers is: Broolyn College The City University of New Yor 2900 Bedford Avenue Broolyn, NY CONCLUSIONS A recent measure, M-SVD, can express the quality of distorted images either numerically or graphically Based on the Singular Value Decomposition (SVD), it consistently measures the distortion both across different distortion types and within a given distortion type at different distortion levels The SVD decomposes every real matrix into a product of 3 matrices A = USV T, where U and V are orthogonal matrices, U T U = I, V T V = I and S = diag(s 1, s 2, ) The diagonal entries of S are called the singular values of A, the columns of U are called the left singular vectors of A, and the columns of V are called the right singular vectors of A M-SVD, as a graphical measure, computes the distance between the singular values of the original image bloc and the singular values of the distorted image bloc, where n x n is the bloc size If the image size is x, we have (/n) x (/n) blocs The set of distances, when displayed in a graph, represents a distortion map The numerical measure is derived from the graphical measure It computes the global error expressed as a single numerical value In this paper, we extended the SVD-based image quality measure to evaluate the visual quality of watermared images using seven algorithms Even if the hacer nows the algorithm for M-SVD, she will not be able to understand the algorithms for [17], [19], and [22] as these algorithms produce distortion maps that appear to be random For [15], [16], [20], and [21], the graphical measure indicates what has been embedded For the DWT-SVD domain algorithm [15], the contours of Lena are apparent For the DWT domain algorithm [16], the embedded binary logo ( BC ) is definitely visible For the DWT domain algorithm [20], which embeds the watermar in LH, HL, and HH bands, the edges of the peppers can be noticed because these three bands correspond to higher frequency areas For QIM [21], if the embedded text is longer, the graphical measure adds more columns in the distortion map The numerical measure computes a global error with values that range from 0385 to 16152, depending on what is embedded in the cover images The smallest and largest values are computed for [22] and [19], respectively, probably because of the following reasons: In [22], an HVS model is used, and only 5 most visually important AC coefficients are modified in each 8x8 bloc in the LOT domain In [19], no HVS model is used, and the highest 1000 AC coefficients are modified in the DCT domain In future wor, we will apply M-SVD to video sequences such as aiyo, flowergarden, and tennis ACKNOWLEDGEMENT This wor was supported in part by a grant from the Air Force Research Lab (AFRL) under Award FA

11 REFERENCES [1] A M Esicioglu and P S Fisher, Image quality measures and their performance, IEEE Transactions on Communications, Vol 43, pp , December 1995 [2] A M Esicioglu, Quality measurement for monochrome compressed images in the past years, Proceedings of IEEE International Conference on Acoustics, Speech, and Signal Processing, Vol 4, pp , Istanbul, Turey, June 5-9, 2000 [3] A Shnayderman, A Gusev, and A M Esicioglu, A multidimensional image quality measure using singular value decomposition, Proceedings of SPIE Image Quality and System Performance, Vol 5294, pp 82-92, San Jose, CA, January 19-20, 2004 [4] A Shnayderman, A Gusev, and A M Esicioglu, An SVD-based gray-scale image quality measure for local and global assessment, IEEE Transactions on Image Processing, 2006 [5] A Shnayderman and A M Esicioglu, Assessment of full-color image quality with singular value decomposition, IS&T/SPIE s 17th Symposium on Electronic Imaging, Image Quality and System Performance II Conference, San Jose, CA, January 16 20, 2005 [6] D Van der Ween, M Nachtegael and E E Kerre, A new similarity measure for image processing, Journal of Computational Methods in Sciences and Engineering, Vol 3, No 2, pp , 2003 [7] A Beghdadi and B Pesquet-Popescu, A new image distortion measure based on wavelet decomposition, 7th International Symposium on Signal Processing and Its Applications, Paris, France, July 1-4, 2003 [8] A C Bovi and S Liu, DCT-domain blind measurement of blocing artifacts in DCT-coded images, Proceedings of International Conference on Acoustics, Speech, and Signal Processing, Salt Lae City, UT, May 7-11, 2001 [9] Z Wang, A C Bovi and B L Evans, Blind measurement of blocing artifacts in images, Proceedings of IEEE 2000 International Conferencing on Image Processing, Vancouver, BC, Canada, September 10-, 2000 [10] Z Wang, H R Sheih and A C Bovi, No-reference perceptual quality assessment of JPEG compressed images, Proceedings of IEEE 2002 International Conferencing on Image Processing, Rochester, NY, September 22-, 2002 [11] P Marziliano, F Dufaux, S Winler and T Ebrahimi, A no-reference perceptual blur metric, IEEE 2002 International Conference on Image Processing, Rochester, NY, September 22-, 2002 [12] E-P Ong, W Lin, Lu, Z Yang, S Yao, F Pan, L Jiang and F Moschetti, A no-reference quality metric for measuring image blur, 7th International Symposium on Signal Processing and Its Applications, Paris, France, July 1-4, 2003 [] L Meesters and J-B Martens, A single-ended blociness measure for JPEG-coded images, Signal Processing, Vol 82, pp , 2002 [14] M Carnec, P Le Callet and D Barba, An image quality assessment method based on perception of structural information, 2003 International Conference on Image Processing, Barcelona, Spain, September 14-17, 2003 [15] E Ganic and A M Esicioglu, Robust DWT-SVD domain image watermaring: embedding data in all frequencies, ACM Multimedia and Security Worshop 2004, Magdeburg, Germany, September 20-21,

12 [16] P Tao and A M Esicioglu, A robust multiple watermaring scheme in the DWT domain, Optics East 2004 Symposium, Internet Multimedia Management Systems V Conference, Philadelphia, PA, October -28, 2004 [17] J Kusy and AM Esicioglu, An effective blind watermaring scheme for color images through comparison of DFT coefficients, Optics East 2005 Symposium, Internet Multimedia Management Systems VI Conference, Boston, MA, October 23-26, 2005 [18] H C Kim, H Ogunley, O Guitart and E J Delp, The Watermar Evaluation Testbed, Proceedings of SPIE Electronic Imaging, Security, Steganography, and Watermaring of Multimedia Contents VI, San Jose, CA, January 19 22, 2004 [19] I J Cox, J Kilian, T Leighton, and T Shamoon, Secure spread spectrum watermaring for multimedia, IEEE Transactions on Image Processing, Vol 6, No 12, pp , 1997 [20] R Dugad, K Rataonda, and N Ahuja, A new wavelet-based scheme for watermaring images, Proceedings of the International Conference on Image Processing, October 4-7, 1998,Vol 2, pp [21] B Chen and G W Wornell, Quantization index modulation methods for digital watermaring and information embedding of multimedia, Journal of VLSI Signal Processing, Vol 27, pp 7-33, 2001 [22] Y Liu, B Ni, X Feng, and E J Delp, LOT-based adaptive image watermaring, SPIE International Conference on Security, Steganography, and Watermaring of Multimedia Contents VI, January 18-22, 2004, San Jose, CA 12