Parameter Optimization Of Fast Curvature Based Interpolation Using Genetic Algorithm

Transcription

1 Parameter Optimization Of Fast Curvature Based Interpolation Using Genetic Algorithm Muhammad Haris 1, Kazuhito Sawase 1, Takuya Sawada 2, Kazushi Kamijima 2, M. Rahmat Widyanto 3, Hajime Nobuhara 1 1 University of Tsukuba, CMU Laboratory mharis, sawase, nobuhara@cmu.iit.tsukuba.ac.jp 2 Hokkaido Denshi Kiki sawada, kamijima@hdks.co.jp 3 University of Indonesia widyanto@cs.ui.ac.id [Received 20/05/2012; accepted 09/07/2012] Abstract: An improvement of fast curvature based interpolation in image super resolution is proposed by using genetic algorithm optimization. The fast curvature based image super resolution requires low computation time, and it is suitable for real time video superresolution. This paper furthermore optimizes the parameters of FCBI based on genetic algorithm. Through the android device implementation, we confirm the quality of obtained super-resolution images by the proposed method is better than those of the existing methods without sacrificing the computational complexity. Keywords: super resolution, fast curvature based interpolation, parameter optimization, genetic algorithm, android. 1 INTRODUCTION Interpolation technique has been known as one of the fastest way to upscale the image. It uses one to one image transformations, which means that one low-resolution image transforms into one image super-resolution. Several methods have been implemented to improve the quality without sacrificing the computational complexity [1]. The problem is quite complex because no hints on the real high resolution are available. Many approaches such as using data dictionary [2] and machine learning have been conducted [3]. In [4], the interpolated pixel is classified to use adaptive filters. This method has a good result but the problem is that the huge variety of natural edge and scales makes a general purpose use of the techniques quite difficult, though it can be efficiently applied to particular tasks (i.e. search of patterns such as faces, trees, etc.). In [5], machine learning is used as a parameter of edge model to patch the low-resolution image and it uses local example. In the reality, interpolation is hard to use statistical data. The transformation better rely only on the low resolution image (self similarities) [2]-[6]. The process could be non linear and hard to determine using sample. By this situation, it s only left smoothness and edge continuity as the parameter to generate higher resolution image. Therefore, the simplest algorithms used linear function to interpolate new pixel values. The classic bilinear, bicubic, lanczos and other methods widely applied in image viewers and image-processing tools have been obtained also using these methods. These methods are computationally efficient, even if the images obtained do not appear natural due to excessive blurring and jagged artifacts. Nonlinear algorithms also have been performed to overcome this limitation. They usually exploit local features like edges (often called edge-adaptive) that also by NEDI, which has been proposed by [7]. The NEDI technique provides good results by adapting locally at each interpolating surface and assuming local regularity in the curvature. New edge direction interpolation also has been proven as a simple and effective technique for one image interpolation [8]. Requiring low complexity, FCBI is the suitable method to be applied in mobile device. We have tried to implement the algorithm and proposed optimized parameter in the Table PC device, and it showed promising method with low computation time, even with the table PC hardware specification. Widely explanation will be written in the next chapter. From Gartner survey [9], smartphone population on Q is already 472 million which 50.9% of it is android device. The needs of SR application in Android is increasing since the same application is available for ios. But most of SR mobile application that already in market is using multiple SR, which is required many inputs and high complexity. This shows us that new application innovation, more over in Android device, is needed. This method is also could be considered as one of the disaster management methodology. During disaster, the network capability is limited. This method could help the disaster management process to transfer medical image via smartphone. A low resolution image/video is also have low byte size, which can be transferred with limited network. It could be quite helpful during the disaster recovery management. After the image were transfer, the post process is SR step which transform the image into high resolution. Therefore, implementation of the proposed method was also applied for dental X-Ray image. 2 FAST CURVATURE BASED INTERPOLATION (FCBI) FCBI [8] is one of the state of the art in edge direction image super resolution. It has proven that it achieved good interpolation result and also required low computation time. It gets the new interpolation pixel by averaging the two opposite neighbors with lower difference. The difference is computed from second order derivatives in the two diagonal directions, it is used to identify the edge curvature. The interpolated pixel will be computed from the two opposite neighbors, which the 1

2 derivative is lower. The original image is formulated where M as the height and N as the width, the total pixels will be M N. Let X = {x m : m = 0, 1, 2,..., M 1} and Y = {y n : n = 0, 1, 2,..., N 1} be finite sets which determines number of pixels. The original image can be define as a function f : X Y I where I = {0, 1, 2,..., 255} is the value of each pixel. The general description between original image and Super Resolution (SR) image could also be formulated. The size of SR image (M N ) applied this condition where M > M and N > N. Let X is {x m : m = 0, 1, 2,..., M 1} and Y = {y n : n = 0, 1, 2,..., N 1} be finite sets which determines number of pixels form SR image. Ĩ 1 (x p, y q ) = f(x m 1, y n+1 ) + f(x m, y n ) + f(x m+1, y n 1 ) 3f(x m, y n+1 ) 3f(x m+1, y n ) + f(x m, y n+2 ) + f(x m+1, y n+1 ) + f(x m+2, y n ) Ĩ 2 (x p, y q ) = f(x m, y n 1 ) + f(x m+1, y n ) + f(x m+2, y n+1 ) 3f(x m, y n ) 3f(x m+1, y n+1 ) + f(x m 1, y n ) + f(x m, y n+1 ) + f(x m+1, y n+2 ) (1) (2) Then using this rule we assign the point Ĩ(x p, y q ) by compute the average of two neighbor in the direction which the derivative is lower. f(x m,y n)+f(x m+1,y n+1) 2 if Ĩ1(x p, y q ) < Ĩ2(x p, y q ) f(x m+1,y n)+f(x m,y n+1) 2 ; otherwise. Fig. 1: FCBI Interpolation Step From this equation, the FCBI formulation could be derived. Size of FCBI image is powered by order 2 from the original image. Let P as the height and Q as the width, the total pixels will be P Q where P = ((M 2 zk ) (2 zk 1)) and Q = ((N 2 zk ) (2 zk 1)) ; zk is the zoom factor of the interpolation. Technically, the FCBI image I(x p, y q ) copies all the original image pixels to the sub-region of the SR image and then computes the interpolated pixels from the original pixels (fig.1). The first interpolated pixels will be the odd pixels since the FCBI image is achieved using multiplied the original image by order 2 and the even pixels is copied from the original image. The first interpolated pixels could be defined as Ĩ(x p, y q ) where X = {x p : p = 1, 3, 5,..., P 1} and Y = {y q : q = 1, 3, 5,..., Q 1}. The FCBI procedure is usually performed in two steps: in the first one, pixels indexed by two odd values are computed as a weighted average of the four diagonal neighbors (pixels of the original image); in the second the remaining holes are filled with the same rule, as a weighted average of the 4 nearest neighbors (in horizontal and vertical directions). ICBI is also introduced to improve the natural interpolated images. By using iterative procedure, ICBI introduces a way to replace the diagonal derivatives in the energy terms with horizontal and vertical ones and iteratively modifying only the values of the newly added pixels. Although it can achieve better result, but the computation time highly increase. The FCBI interpolation is the edge-directed interpolation algorithm that approximately duplicating the original pixels (x m, y n ) into enlarged grid (x p, y q ) and then filling the interpolation pixel that satisfy the rule below. The first algorithm is to gain the edge direction of the interpolation pixels frame. Using neighbor pixels of the interpolated pixel, it is computing the second derivatives of the 8-neighbor pixel to get the edge direction. We can get the relation between (p, q) and (m, n). If p and q is available then we can compute m = ((p 1)/2) and n = ((q 1)/2). Floor function is define as x = max{r Z r x} where x is real numbers and {Z, r} are set of integer (positive, negative, and zero). This step is performed for both vertical and horizontal directions. Then it will be stop after all pixels have been calculated. This step is illustrated in (Fig.1). This rule is avoided when the first order derivatives of intensity are larger than a fixed. In this case, it interpolated in the direction with lower gray level difference. Image, that is obtained by this method, does not present the jagged artifacts. Based on experiments, this method also achieved good result and low complexity. Comparing with bilinear and bicubic, FCBI could achieved better computational time. Even bicubic has smaller RMSE value, but by using subjective test the image, FCBI is better than bicubic. 3 IMPROVEMENT OF FCBI FCBI has simple calculations which impact to the low computational complexity and also have good result based on RMSE test and human visual score. A fix variable in the edge detection matrix is one of the limitation of FCBI. This conditions make the new interpolated value cannot describe the edge continuity well. It will contain edge smoothness and lose the edge from the original image. This is the reason, [GIA10] also proposes the iterative edge detection technique, ICBI. The ICBI is using iterative correction of the new interpolated pixels. This method is good but it will highly increase the computation time. Besides, FCBI will be more suitable for mobile device implementation, since it has low computation time but still have a good image quality. 3.1 FCBI Parameter Optimization Simple experiment was conducted as preliminary research. To optimize the FCBI, simple iteration algorithm has been made to achieve minimum error. From equation (1) and (2), matrix [1 1 1; ; 111] is used to do edge detection step which is similar to compute the second derivatives. This matrix has the same mechanism with Laplacian Filter, the matrix shown the weight of each 8-neighbors for the interpolated pixel. In the experiment, we used 3 variables (x, y, z) and it formulated as matrix as in (Fig.2). Using this variables, we expected to gain the local optimize parameter as preliminary research. This method showed better result that the original FCBI, but still have low computational time. 2

3 in the image domain (fig. 3). Furthermore, the high frequency image still have higher error which could be assumed that this parameter is only effective for low frequency image. Fig. 2: Edge Detection Matrix Parameter (x, y, z) (X r, Y r, Z r ) X r, Y r = { 10, 9, 8,..., 0, 1, 2,..., 10} Z r = { 20, 19, 18,..., 0, 1, 2,..., 20} 4X r + 2Y r + 2Z r = 0 (3) By using parameter matrix (Fig.2) and constraint in equation 3, we could get below equation as edge detection. From this set of integer, total solution spaces is X r Y r Z r = Fig. 3: FFT Analysis Ĩ 1 (x p, y q ) = x f(x m 1, y n+1 ) + y f(x m, y n ) + x f(x m+1, y n 1 ) + z f(x m, y n+1 ) + z f(x m+1, y n ) + x f(x m, y n+2 ) + y f(x m+1, y n+1 ) + x f(x m+2, y n ) Ĩ 2 (x p, y q ) = x f(x m, y n 1 ) + y f(x m+1, y n ) + x f(x m+2, y n+1 ) + z f(x m, y n ) + z f(x m+1, y n+1 ) + x f(x m 1, y n ) + y f(x m, y n+1 ) + x f(x m+1, y n+2 ) From this experiment, we produced a local optimum parameter. The optimized parameter is shown in below (Table I). It was using Lena image as an input. TABLE I: SAMPLE RESULT FOR LENA IMAGE x y z PNSR The optimum parameter using iterative process that has highest PNSR value is found. The highest optimal variable is (X r, Y r, Z r ) = (1, 10, 12). This experiment is categorized as local optimum experiment since the solution space is not global, but it adequate to achieve better result than FCBI in 50 testing images (table II). Table 2 shows that averagely this parameter achieve better result than original FCBI, but there are some cases that it is worst. From this background, we could assumed that the relation between FCBI parameter and the image are not linear. New method to find global optimum parameter is proposed. TABLE II: FCBI LOCAL OPTIMIZATION PNSR Method 2x 4x FCBI ICBI FCBI V For the lower error, most of Fourier domain is present in the center of the image, which corresponds to low frequency data (4) (5) 3.2 FCBI GA Optimization From the previous chapter, we may assumed that we got local optimized parameter for the edge detection as preliminary result. In this chapter, we tried to propose new method using optimization algorithm to get better quality of FCBI image. Using Genetic Algorithm (GA), we tried to define the optimum parameter to some certain images. Genetic algorithm employed a search technique with simple operator, thus simplifying the computation complexity of the search, as well as abstracting the search for a solution from the specific details of the problem [10]. We assumed that there are non linear relation between FCBI parameter and the image characteristic. The relationship between high resolution and low resolution patterns could be represented by example of natural image. By using the training set which presented many edge appearance, we tried to get edge profile to improve FCBI image quality. To avoid premature convergence problem, a stage of learning procedure was adopted. Based on [11], GA could be used to find optimal parameter for image super resolution. The standard GA [12] was employed and it s main characteristics are as follows. Codification: a chromosome is a floating point vector p within range [-10,10] of dimension 3, where each element of the vector represent an adjustable in FCBI Matrix edge detection (Fig.2). Fitness function: each chromosome is evaluated by fit(p) = 1 MSE(p) where MSE(p) is the mean squared error of the network codified by p, defined as MSE(p) = 1 N N (O n F CBI(D n, y 1, y 2, y 3 )) i=1 where O n is the original image and D n is the downsample image. Selection operator: chromosomes are selected for the next generation using the roulette wheel operator, which assigns selection probabilities proportional to the fitness of the individual. 3

4 Crossover operator: uniform crossover, where elements of 2 parent chromosomes are exchanged with certain probability. Mutation operator: gaussian mutation, where element p i of a chromosome selected for the mutation is selected using gaussian random variable within range [-10,10]. In this experiment, the code was implemented, and evaluated based on Matlab. After training step is done, the testing experiment scheme is shown in the fig. 5. There are 3 steps include: Downsampling, Upsampling, and Error Evaluation. 4 EXPERIMENTAL RESULTS To confirm the efficiency of the proposed learning scheme, the experiments were conducted. There were 2 types of experiments: objective test and subjective test for medical image. We conducted both objective and subjective experiment to compare between quantitatively image quality and human perception of the image quality. For experimental needs, the testing images contained various patterns, and natural objects. We used images dataset from USC SIPI Image Databases ( The image criteria of the experiment are: Grayscale images (Intensity range 8 bit) Original Images (256 x 256 pixels) Experimental 80 images (50 training images and 30 testing images) 4.1 Objective Test Fig. 4: Training Image The objective test compares images that is obtained by using downsampling from the original images and then do the enlargement with different method and various scale (Fig.4). Grayscale image is used since the RGB image is not really relevant with the test. We used RMSE and computational time as the evaluation factor. In this experiment, we compared 6 method: Nearest Neighbor, Bi-Linear, Bi-Cubic, FCBI, ICBI, and new FCBI (optimized parameter). The proposed GA algorithm was used in the training step with the following GA parameter (table III): TABLE III: GENETIC ALGORITHM PARAMETER Parameter Value Population Size 150 Max Generations Crossover Probability 0.8 Mutation Probability 0.01 Obtained Result ( , , ) Fig. 5: Objective Test Scheme The comparison algorithms are chosen based on the single image interpolation technique. We compared the Nearest Neighbor, Bilinear, Bicubic, FCBI, ICBI, and Optimized FCBI. Any other method could be tested with the same experiment. All of the images and scripts are available in my website ( mharis/newfcbi.html). Root mean square error (RMSE) is one of many ways to quantify the difference between values of source and target image. RMSE measures the average square of errors. By finding the minimum RMSE of the images, the optimized results can be achieved. From the fig. 5, we could defined the RMSE evaluation as below. Let O i is the matrix of original image where n is total of original images and F (x, y, z, Di) is the FCBI function with 4 input parameter which is 3 matrix parameter (x, y, z)and D i is the downsampling image. RMSE = 1 n = 1 n n (0 i SR i ) 2 i=1 n (0 i F (x, y, z, D i )) 2 i=1 After the training is done, an optimize parameter is achieved. Testing step is established to evaluate the training process. The image result (fig. 6) shows that new FCBI almost has the same quality with ICBI and better that original FCBI and Bicubic. There are some quality improvements that proven by evaluation function (RMSE). The RMSE equation is below (eq. 6): From table III, the RMSE value of New FCBI is always lower than the original FCBI which means it has better quality. From table IV below, the result of CPU computation time is shown. From the result, it is proven that New FCBI parameter doesn t sacrifice the computational time. Obviously, there are much room for acceleration, such as parallel processing or GPGPU, to achieve real time video enlargement. (6) 4

5 Fig. 6: Result with 4x zoom factor TABLE IV: AVERAGE RMSE RESULT Method 2x 4x 8x NN Bi-Linear Bicubic FCBI ICBI New FCBI transfer specially during disaster recovery where as network is limited. Dental X-Ray images are used as testing images. Below are sample of the images: TABLE V: AVERAGE COMPUTATION TIME (CPUTIME) 4.2 Subjective Test Method 2x 4x 8x NN Bi-Linear Bicubic FCBI ICBI New FCBI It is also known that optimizing method in order to achieve minimum RMSE is not necessary best thing to achieve good images. RMSE is not always corresponding to visually perceived quality especially for the medical image. Therefore, subjective test for X-Ray image is also conducted (fig. 7). Different with objective test, the subjective test is using people as the human subject. The goal is how to achieve smallest zoom factor without eliminated the information of image. With small size image, we can also obtain fast network data Fig. 7: Dental X-Ray Image In this experiment, we used 8x and 16x zoom factor to see how it can be applied in medical image. From image below (fig. 8), we can see both ICBI and New FCBI have almost the same quality. However, both have quite different computational time. From the subjective perspective, we still could see the image characteristic clearly up to 8x zoom factor. Even with 16x zoom factor, we still could see the difference. 4.3 Android Tablet PC Implementation The proposed method is also implemented in table PC device (fig. 9). The software runs well on Galaxy Tab 10.1 using Android Gingerbread V3.2. We have implemented various single SR method: Nearest Neighbor, Bilinear, Bicubic, and FCBI. 5

6 Fig. 8: Dental X-Ray SR with 8x zoom factor In this tablet PC, it is shown that FCBI has lower computation time and better quality than Bilinear and Bicubic. implementation in medical image also indicates promising evidence, it can reached 16x zoom factor which could highly reduced size of images. Fig. 9: Android Implementation Table IV below shows the comparison between each method in the Android Table PC. TABLE VI: COMPUTATION TIME (MS) ON ANDROID TABLET PC Image Size NN Bilinear Bicubic New FCBI Android 600x Lena 512x Kid 320x Compared with other device, currently there are no SR application that available in Android Market. In ios, there were some applications that were developed, but used multiple SR, which required high complexity and many inputs. So, the development of single SR in the android device is really promising, since the android device population is still increasing. 5 CONCLUSION The optimized parameter of FCBI is proposed. It is proven for both objective and subjective tests that it has improvement in both quality and computation time. We suggest that the algorithm can be applied to video coding based on the nature of low computational complexity and good image quality. The REFERENCES [1] Sung Cheol Park, Min Kyu Park, and Kang Moon Gi. Image reconstruction: A technical overview. IEEE Signal Processing Magazine, [2] Chih Yuan Yang, Jia Bin Huang, and Ming Hsuan Yang. ACM on Computer Vision, pages [3] Lyndsey C Pickup, Stephan J. Roberts, and Andrew Zisserman. Optimizing and learning for super-resolution. Machine Vision, [4] Russell Hardie. A fast image super-resolution algorithm using an adaptive wiener filter. IEEE Trans. on Image Processing, 16: , [5] Gilad Freedman and Fattal Raanan. Image and video upscaling from local self-examples. ACM Transaction on Graphics, 30, [6] Shuyuan Yang, Zhizhou Liu, Min Wang, Fenghua Sun, and Licheng Jiao. Multitask dictionary learning and sparse representation based single-image super-resolution reconstruction. Neurocomputing, 74: , [7] Xin Li and Michael T. Orchard. New edge-directed interpolation. IEEE Trans. on Image Processing, 10: , [8] Andrea Giachetti and Nicola Asuni. IEEE Transaction on Image Processing, pages [9] Mark Brownlow. Smartphone statistics and market share. wireless-mobile/smartphone-statistics.htm, [Online; accessed 4-May-2012]. [10] E. David Goldberg. Genetic Algorithms in Search, Optimization, and Machine Learning. Addison-Wesley Professional, [11] Barry Ahrens. Genetic algorithm optimization on super resolution parameters. Genetic and Evolutionary Computation, 1:58 65, [12] Michalewicz Z. Genetic Algorithms + Data Structures = Evolution Program. Springler 3rd ed.,