Sparse Matrices in Image Compression

Transcription

1 Chapter 5 Sparse Matrices in Image Compression 5. INTRODUCTION With the increase in the use of digital image and multimedia data in day to day life, image compression techniques have become a major area of research. The main objective of this is to reduce the space occupied and to enable easy storage and transmission. Images are essentially stored in the computer in a digital form. In general, the space complexity of an image is represented as O (mn) where the m refers to the height and n the width of the image. Spatial and frequency domain are the two different methodologies of image compression. Frequency domain compression achieves better compression compared to spatial domain. Spatial domain compressions are nowadays popular in ubiquitous computing as it requires less computational complexity. Image data in spatial domain can be compressed without sacrificing the pixel value of the image. This form of image compression is known as lossless image compression. Applications where loss of pixel value doesn t affect the image quality is known as lossy image compression techniques. Image compression is applied to a single image to set of similar images. Single image compression can be achieved via lossless and lossy techniques. But similar image compression which is prevalent in medical and satellite images adapts only the lossless techniques. In this chapter, we adapt the sparse representation of the lossy image compression techniques such as SVD and rectangular segmentation method for single image and lossless set redundancy multi-level centroid method for similar image compression techniques. Singular Value Decomposition (SVD), a concept of linear algebra has been used by many researchers like (Gunta et al (203), Lahabar and Narayanan (2009), Rajamanickam (2009), Soliman et al (2007) for reducing the data. It is used to find the best approximation of original data values with the help of fewer dimensions. In this approach, an image represented in a matrix A is decomposed into three components T U, S and V such that the relation A U S V is satisfied. Here we used SVD technique for image compression along with the CSR sparse matrix storage format. Rectangular segmentation method proposed by Chen et al (202) overcomes the drawback of quarter tree segmentation technique by considering the blocks of any 7

2 size. To further reduce the storage space of the image they have adapted the COO format. Here we analyse the effect of using various formats in rectangular segmentation technique. Karadimitriou (996) in his thesis presented several Set Redundancy Compression (SRC) techniques that are used to compress a set of similar images. El- Sonbaty et al (2003) in their paper proposed an improved SRC technique called the multi-level centroid technique, which leads to the representation of images as sparse matrices. Here we made an attempt to use the sparse storage formats in this technique. It is common to have images represented in a matrix form having huge number of zeroes compared to non-zero values. An elegant method of storing such matrices via sparse matrix storage formats is an active research area. In this method, only the non-zero values of the matrix is stored and thereby reducing the total storage desirable for the original matrix. This also helps in decreasing the amount of data to be transmitted for communication across computers. In this chapter, section 5.2 describes the SVD technique and its application in image compression with sparse storage format, section 5.3 deals with rectangular segmentation technique with various sparse matrix storage formats, section 5.4 explores the use of sparse matrices in multi-level centroid method, Section 5.6 presents the results of SVD, rectangular segmentation and multi-level centroid method with sparse storage format for image compression, Section 5.7 provides the conclusion SINGULAR VALUE DECOMPOSITION (SVD) SVD is a method of advanced linear algebra. It is based on the filling the maximum energy of a signal into a smaller number of coefficients. It is an effective way to split a matrix into linearly autonomous elements where each component has its own influence in terms of energy. SVD plays a vital role in many application areas such as efficient medical imaging, topographical analysis, image processing, watermarking schemes and latent semantic analysis. In image compression, SVD offers its advantage in the form of factorizing the image matrix A into three components U, S and V. Here U represents the orthogonal matrix of rows; S denotes the diagonal matrix of the singular values of A and V is called the orthogonal matrix of columns. These 72

3 elements of the matrix satisfy the relation algorithm defined in GNU scientific Library (GSL). T A U S V. Here we used the SVD 5.2. SVD ALGORITHM IN GSL Generally there are various algorithms existing in the literature to compute the SVD of a matrix. All popular scientific packages such as LAPACK and GSL use Golub Reinsch algorithm given by Golub and Reinsch in (970).It is an efficient, popular and stable technique for calculating SVD of an arbitrary matrix. It is also well suited for MIMD and SIMD architecture. Running time of SVD using this algorithm is O (mn 2 ). The algorithm is specified in 5. is used for SVD to compress the images and to parallelize the operation using OpenMP multithreading programming language. Algorithm 5.: Golub Reinsch algorithm Input: Given Matrix A of m x n order Output: The components of SVD like U, S and V.. Bidiagonalization of A to produce B B = Q T AP [Here, A is the original matrix considered, Q and P are the considered unitary householder matrices and B is the diagonal matrix considered] 2. Diagonalization of B to produce S S = X T BY [B is the matrix obtained from the step, S is the original diagonal matrix; X and Y are the considered orthogonal matrices.] 3. Compute orthogonal matrix U and V U = QX V T = (PY) T 4. Compute SVD of A A = UxSxV T 73

4 5.2.2 SVD EXAMPLE A given matrix A of size m x n is represented in SVD with the following three matrix components. U: a matrix with the dimension of m x m, S: a diagonal matrix of dimension m x n, V: a matrix with the dimension of n x n and V T is the transpose of the matrix V. It is represented in equation 5.. A U S V (5.) T m n m m m n n n To realise the process of SVD let us consider an example of a 2 x 3 matrix. A = [ 3 3 ] To calculate the eigen values and eigen vectors for the calculation of U and V, A T is calculated. 3 A T = [ 3 ] The first step in SVD includes calculating A x A T and A T x A, which is represented as: A x A T = [ ] x [ 3 ] = [ ] and 3 A T x A= [ 3 ] x [ ] = [ 0 0 2] Further, we solve for the Eigen vectors of A such that A x A T λi = 0 (5.2) Using equation 5.2, the Eigen values are calculated as λ = 0 and λ2 = 2. In the next step we determine the values of the matrix U as follows: columns of U are determined by solving the Eigen vectors of A x A T. This should satisfy the relation, [A x A T - λi] [x] = 0. Using the Eigen values, we get the Eigen vectors as X 74

5 and X 2. To get the columns of U, we convert the matrix [ ] into orthogonal matrix by following Gram-Schmidt orthonormalization process proposed by Meyer (2000) to determine the Eigen vectors of u and u2. Therefore, orthogonal matrix U mxn is represented in equation 5.3. U = [ ] (5.3) Similarly to determine the columns of V we first determine the Eigen vectors using the relation [A T x A- λi][x] = 0. Further we calculate the Eigen vectors v, v2 to 2 determine the columns of V in a similar manner and have V=[ 2 2 ]. By 0 5 applying Gram-Schmidt orthonormalization process we determine the orthogonal matrix of V nxn as shown in equation 5.4. V = [ ] (5.4) By finding the Eigen vectors of A x A T and A T x A, we can easily find the diagonal matrix S. The singular values of A are defined as the square root of the roots the Eigen vector. It is given in equation 5.5. S= [ ] (5.5) Now the matrix A has been factorized into the three matrices U, S and V and it can be proved that the relation T A U S V holds good. [ 3 3 ] = [ ] x [ ] x VT = [ ] 75

6 SVD computation includes sequence of vector operations, matrix to matrix and matrix to vector multiplication as shown in Rangarajan (204) and Ye (2005). This feature enables SVD computation a good candidate for parallelization SVD IN IMAGE COMPRESSION The objective of image compression is to denote an image with reduced amount of data than the image is normally composed of and the capability to reconstruct the image from its reduced form. This reduces the storage requirement of an image and decreases the amount of data used to transmit the image across computers as specified by Andrews Patterson (976), Golub and Van Loan (202) and Yang and Lu (2004). Here, we have used SVD as lossy image compression method compared to other methods because it is basic, simple and works almost for all kind of matrix and it is well suited for image compression and to apply parallelism using OpenMP. Normally, images are stored in the form of matrices in the computer memory. The amount of memory required to store an image depends on the type of image and the dimension of the image. A grayscale image occupies mxn space where m and n denotes the height and the width of the image. A colored image occupies a space of mxnx3, for representing the colors red, green and blue. An illustration of both these schemes has been shown in Figure 5.. Figure 5. Image representation in color and grayscale From the properties of SVD, the image matrix A can be represented in the form of its SVD components as specified in equation 5.6. A U S V U S V U S V (5.6) T T T n n n In equation 5.6, it is worth mentioning that the value of S > S 2 > S 3 >...>S n. The above relation implies that, the contribution of the first few components of the sum yields highest approximation compared to the last component. Thus, by considering the first r members of the above summation, we can get a considerable image approximation of A. This property of SVD is used for image compression as given in 76

7 Ding and Ye (2005) and Yang and Lu (2004). The relation for the compression of an image considering the first r singular values is shown in the equation 5.7 A U S V U S V U S V (5.7) T T T r r r r Here A r represents the first r singular values of the singular matrix S.The space overhead in storing the matrix of size m x n can be minimized by only storing the matrices U mxr,v nxr and the singular vector S r and to reconstruct the image as : A U S V. Thus, it leads to a decrease in the amount of memory required T r m r r nxr to store the image and the space complexity of the image is given as A r( m n ). The compression ratio, the mean square error and peak signal to noise ratio is defined as specified in equations 5.8, 5.9 and 5.0. r Compression ratio C r m n r( m n ) (5.8) Mean square error (5.9) 2 ( MSE) ( Oij Ri j ) / ( m n) where O represents the original image and R represents the reconstructed image of dimension m x n Peak signal to noise ratio 2 ( PSNR) 0log(255 / MSE) (PSNR) (5.0) IMPLEMENTATION OF SVD The advent of multi-core processors provided the new opportunity for parallel processing and to improve the processing capabilities. As the traditional ways of programming are inherently based on single core processors, they are not capable to tap the high computational power offered by multi core processors, even if available. Here we have used OpenMP parallel programming model to improve the speedup of SVD technique. This chapter also explores the use of sparse matrix storage format for storing the matrices of SVD technique. With OpenMP model.5x speedup is achieved and SVD shows better compression ratio using sparse matrix storage format OPENMP PROGRAMMING MODEL OpenMP is an Application Programming Interface (API) used as a portable programming model in shared memory architecture. OpenMP works as a fork join 77

8 model and distribute the work among the available processors. We have preferred this programming model compared to the other model because of its unified code for both sequential and parallel implementation. It is also simple to use compared to Message Passing Interface (MPI) and it provides an easy way of embedding compiler directives to the C/C++ source code. Here, we have used OpenMP in Visual Studio environment to parallelize SVD image compression technique. OpenMP for SVD compression is, used because of high computation power required for large image size. OpenMP performs thread-level parallelism to reduce the execution time of an application. By analysing the code we have to identify the parallelizable and non-parallelizable part. Based on this analysis, various parallelizable directives are used to parallelize the code SPARSE STORAGE FORMAT SVD algorithm of lossy image compression technique comprises of matrix-matrix multiplication and matrix-vector multiplication. The singular matrix used in SVD computation consists of many zeros. These unnecessary zeros in these matrices are removed by representing them via efficient sparse matrix storage format. There are various techniques of sparse matrix storage format such as Coordinate format, Compressed Row Storage (CSR), ELLPACK, Compressed Column storage CCS, row grouped CSR, Quad tree, Quad CSR, Blocked compressed Row storage, Minimal QCSR format and unified SELL-C σ as discussed in chapter 2. Among these, CSR is the most efficient and general purpose format for all types of sparse matrices. We have used CSR format in which the matrix is stored in the form of three vectors asshown in Figure 5.2 M = [ ] Ptr = [ ] Column = [ ] Data = [ ] Figure 5.2 CSR format of an image with sparse structure 78

9 The size of these three matrices is purely dependent on the distribution of the non-zero elements in the matrix, and it remains the same irrespective of the number of zeros padded. The ptr array facilitates fast multiplication, and the code remains unchanged for Sparse Matrix Vector multiplication SpMV IMPLEMENTATION DETAILS Figure 5.3 shows the steps used in SVD based Image compression. The input RGB image is converted to its corresponding gray scale version. This conversion is performed using Matlab function rgb2gray ( ) and the size of the image is now m x n. This gray scale image is written to a file for further processing. The binary file and the rank value form the input to the SVD computation. SVD computation is done by distributing the work among the cores in the multi-core architecture with OpenMP programming model. Here, CSR format of sparse matrix is used to represent the final Input RGB image RGB to Grayscale conversion in Mat lab Input Gray scale image SVD using OpenMP Recreation of image using compressed files Generation of compressed image files Figure 5.3 Flow diagram for SVD based image compression output and to achieve better compression compared to the original file. Reconstruction of the image with different rank values are carried out in Matlab by using imshow () function RECTANGULAR SEGMENTATION Rectangular segmentation technique proposed by Chen et al (202) overwhelms the disadvantage of the quarter tree segmentation technique given by Xing-heng (2008). Quarter tree decomposition technique groups the similar blocks of the image in the powers of 2. This result in more number of blocks and the compression ratio is less. It also results in more number of non-zero values. This drawback can be overcome by using rectangular segmentation methodology where the block sizes need not to be in 79

10 powers of 2. This in turn reduces the number of blocks and the number of non-zero values in the matrix. This is illustrated in Figure Figure 5.4a Quarter tree decomposition b Rectangular segmentation Figure 5.4a shows the quarter tree segmentation technique and the number of blocks obtained is 25. If each block is represented with its non-zero value in the left corner of the block and all other pixels in that block are replaced by zero then each block will have only one non-zero value constituting of 25 non-zero values for the given example. Figure 5.4b shows the rectangular segmentation of an image pixel where the number of blocks obtained is and the non-zero values are also. This clearly shows the reduction of non-zero values in rectangular segmentation method. As the resultant image consists of more number of zero values, we tried to use the various sparse matrix storage formats to reduce the storage space of the matrix. The basic architectural diagram of rectangular segmentation technique with sparse matrix storage format is shown in Figure 5.5. The input image represented in a matrix is processed with rectangular segmentation algorithm. It is a lossy image compression technique where the loss of information doesn t affect the visual impact of the image. Figure 5.5 Process flow of rectangular segmentation with sparse storage format 80

11 5.3. RECTANGULAR SEGMENTATION ALGORITHM The given input image is converted into a matrix using Matlab software. Rectangular segmentation as specified in algorithm 5.2 is applied to the matrix values. The matrix after rectangular segmentation shows sparse model such that more number of zero elements compared to non-zero values. To reduce the storage space of the sparse matrix, general storage formats such as COO, CSR, QCSR and Minquad as discussed in chapter 2 are applied to reduce the space complexity of the algorithm. Figures 5.6(a) and 5.6(b) show the example matrix and the matrix using rectangular segmentation algorithm for an image size of 8 x 8 pixel values. Algorithm 5.2. Rectangular Segmentation Algorithm Inputs: Image (I) to be compressed, Threshold value (t), Number of rows in input image (r), Number of Columns in input image (c) Output: Compressed Image (rect). Begin 2. Initialize rt, cf, pc and pr to While rt < r 4. x = r 5. p = I[r][cf] 6. if (p==0)then 7. cf++ 8. end if 9. if(cf==r)then 0. cf=0. end if 2. r++ 3. continue 4. c=cf 5. while( abs(p-im[r][c]) <= t) 6. for each item j from c to r 7. if( abs(p - I[r][j]) <= t && c<x)then 8. I[r][j]=0 9. pc=c 20. c++ 2. else 22. x = c 23. end if 24. pr = r 25. r++ 8

12 26. end for 27. break 28. c = cf 29. rect [pr][pc] = p 30. cf = 0 3. r = c = End (a) (b) Figure 5.6 (a) Original Matrix (b) Rectangular Segmented matrix The segmented image is then subjected to sparse matrix storage formats and it is represented as follows:. COO format: Row= [ ] Column = [ ] Data= [ ] 2. CSR format: Column = [ ] Data= [ ] Ptr= [ ] 3. QCSR format:

13 MinQuad format: MULTI-LEVEL CENTROID TECHNIQUE It is a lossless compression technique used to store similar images of medical and satellite applications. It works with the principle of set redundancy technique. This algorithm is an improved version of basic centroid algorithm. The mathematical model of the centroid method is given in equations 5. and 5.2. Here, the estimated image is achieved by calculating the median of all pixel values of particular position in all similar images and a set of difference images are obtained by subtracting the pixel value with the median value calculated. F m x m (5.) i, j i, j i, j i, j D x F (5.2) i, j i, j i, j The multi-level centroid technique is obtained by performing the centroid technique recursively on the difference images as shown in mathematical model specified in the equation 5.3 and 5.4. C M D M (5.3) i, j, l l, i i, j l FD D C (5.4) i, j, l i, j i, j, l 83

14 Where C i+, j, l is the predicted value of position i+ in image j at level one, M l, i+ is the pixel value of position i+ in the median image at level one, D i, j is the value of position i in the input image j, and FD i+, j, l is the final difference value between the predicted and input value at position i+ in image j at level one. The difference images obtained in the last level of the implementation are sparse matrices. Hence the various sparse matrix storage formats that were discussed in chapter 2 can be used to further increase the compression ratio of the images. Figure 5.7 represents the multilevel centroid technique with sparse matrix storage formats. Figure 5.8 shows a sample image with its corresponding difference image and median images. Figure 5.7 Process flow of multi-level centroid method with sparse matrix 84

15 Original image : Difference image : Median image : Median image 2: Figure 5.8 Matrices when using Multi-level centroid technique The difference images obtained in the last level of the multi-level centroid technique are sparse matrices. Hence, sparse matrix storage formats discussed in chapter 2 can be used to upgrade the compression ratio of the images. The difference image obtained can be represented in various sparse matrix storage formats as follows:. COO format: Data= [ - ] Column= [ ] Row= [ ] 2. CSR format: Data= [ - ] Column= [ ] Ptr= [ ] 3. QCSR format:

16 Min Quad format: EXPERIMENTAL RESULTS Experimental analyses were performed to show the importance of sparse matrix storage format in single and similar image compression techniques SVD We have considered three different colored images and converted to the gray scale images using Matlab. SVD is performed in visual studio environment by taking the grayscale image input. The performance of SVD varies based on the type of image used as shown in Figure 5.9. Three images that have been considered are Fruits (320x240) Human Face (550x500) and Room (60x423). Figure 5.9a, 5.9b and 5.9c shows the reconstruction of images with distinct rank values. The accuracy of the SVD compression is represented in PSNR and the MSE and it is shown in Figure 5.0 and 5.. Speedup shows the performance gain achieved through thread-level parallelism. Tables 5., 5.2 and 5.3 shows execution time of sequential and parallel version of SVD for various image sizes. PSNR has been computed and higher value of PSNR directs a better reconstruction of the image. Figures 5.2, 5.3 and 5.4 shows the size of the image obtained using SVD with sparse matrix CSR format for different rank values. Figure 5.9a Reconstruction of the image Fruits. Figure 5.9b Reconstruction of Human Face. 86

17 Figure 5.9c Reconstruction of Room at different Ranks. Figure 5.0 Variation of PSNR with Rank. Figure 5. Variation of MSE with Rank. 87

18 Table 5. Performance details of SVD on image Fruits Rank Compression Ratio Speed Up Ex Time OpenMP(sec) Ex Time Serial Reconstruction Time ms Table 5.2 Performance details of SVD on Room Rank Compression Ratio Speed Up Ex Time OpenMP(sec) Ex Time Serial Reconstruction Time ms Table 5.3 Performance details of SVD on the Human face Rank Compression Ratio Speed Up Ex Time OpenMP (sec) ExTime Serial (sec) Reconstruction Time milli sec

19 Size in Kb Size in Kb Size in Kb rank Size of original image in Kb Size of compressed image (sparse matrix) in Kb Figure 5.2 Variation of compression with different rank values for fruit image Size of original image in Kb rank Size of compressed image (sparse matrix) in Kb Figure 5.3 Variation of compression with different rank values for human face image Size of original image in Kb rank Size of compressed image (sparse matrix) in Kb Figure 5.4 Variation of compression with different rank values for room image. 89

20 5.5.2 RECTANGULAR SEGMENTATION Initially we applied rectangular segmentation technique to the images of size 8 x 8 and compressed further with sparse matrix storage formats. Table 5.4 shows PSNR value and the space complexity obtained for two different threshold values two and five with different storage formats. Table 5.4 Space complexity for various storage formats and PSNR value Images RSSMS with COO RSSMS with CSR RSSMS with QCSR RSSMS with MinQuad PSNR Q=2 Q=5 Q=2 Q=5 Q=2 Q=5 Q=2 Q=5 Q=2 Q=5 Boat Lena Goldhill Table 5.4 shows the compression ratio achieved using Min quad format which is high compared to other formats. But, using this format it is not possible to reconstruct the image since it stores only the structure of the non-zero values. The next best case is obtained by using CSR format. The PSNR values of the reconstructed images with threshold 2 are high compared to 5. This indicates that distortion increases with increase in threshold value. Next, we considered various images of size 256 x 256 and performed the rectangular segmentation technique with storage formats. Table 5.5 shows the space complexities obtained with sparse matrix storage formats with the threshold assumed to be 2. From Table 5.5, it is clear that next to the Min Quad format, the COO format gives the better space complexity. This is due to the increase in number of non-zero values in the sparse matrix as the size of the image increases. This when stored using CSR format with increasing multiple times in the ptr array causes a space overhead. Rectangular segmentation technique with COO format gives better results for larger images compared to the other format since it stores only the row column indices and data values. Figure 5.4 shows the compression ratio obtained for Lena image with rectangular segmentation technique and sparse storage formats. 90

21 compression ratio Table 5.5 Space complexities (in bytes) for image size of 256 x 256 with rectangular segmentation technique Images Original RSSMS RSSMS with COO RSSMS with CSR RSSMS with QCSR RSSMS with MinQuad Barbara Boat Goldhill Lena Peppers Zelda Compression Ratio Compression Ratio sparse storage formats Figure 5.5 Compression ratio of Lena image with rectangular segmentation technique and storage formats MULTI-LEVEL CENTROID TECHNIQUE Next for experimental analysis of multi-level centroid technique, set of similar images of Brain CT scan images are considered and analysed the effect of using sparse matrix storage formats. Figure 5.5 depicts the compression ratio obtained using multi-level centroid technique with sparse matrix storage format. The optimal number of levels considered in paper given by El-Sonbaty et al (2003) is 2 to increase the compression ratio. Figure 5.5 depicts the usage of these sparse matrix storage formats is not suitable for this technique. On analysis, we found that COO format which gave better compression with rectangular segmentation technique fail to do so in multi-level technique. This is because, the non-zero values in the difference image requires (3 x 4 x 8) 96 bits of storage space with COO format which is greater compared to the 9

22 compression ratio normal multi-level centroid technique. Though Minimal Quad tree gives better compression with multi-level centroid technique, as mentioned earlier, it will not be possible to reconstruct the image using this technique. Hence multi-level centroid technique is not suitable with sparse matrix storage format. Conventional compression techniques like Huffman coding and arithmetic coding yields better compression ratio with multi-level centroid technique Set Set 2 Set Multi-level COO CSR QCSR MinQuad sparse storage formats Figure 5.6 Compression ratio of multi-level centroid technique with sparse matrix storage formats 5.6 CONCLUSION Experimental results show that the use of sparse matrix plays a significant role in SVD and rectangular segmentation method and fails in multi-level centroid method. The performance of SVD differs depending on image type being compressed. SVD performs better for human face compared to the other two images by showing high PSNR value. The use of CSR storage format of sparse matrix shows better compression ratio for gray scale image. This can be extended to RGB matrices and can be used to compress colored image. Here, we have used OpenMP parallel programming model to improve the execution time of SVD technique. The use of OpenMP has increased the execution speed of SVD technique and results in faster compression time of images. OpenMp constructs when applied to loops with less number of iteration yields lesser performance due to the overheads involved in thread management. Rectangular segmentation technique of single image lossy image compression method shows better compression with CSR format for small sized 92

23 images. For larger images, COO format gives better results due to the disadvantage of ptr array in CSR format. Use of sparse matrix storage format in multi-level centroid method of compressing similar images doesn t give positive results. Hence to improve this technique, Huffman and Arithmetic coding must be used. 93