Image Deconvolution. Mathematics of Imaging. HW3 Jihwan Kim Abstract This homework is to implement image deconvolution methods, especially focused on a ExpectationMaximization(EM) algorithm. Most of this homework is based on Bertero and Boccacci's paper [1] after getting basic understanding of EM [3]. I will also show how image denoising methods can be used with the image deconvolution methods to generate a better result. Discrete Fourier Transform is also well known approach to recover blurred image, but found some limitation with noised image. For all deblurred images shown here, we assume we know the original blurring filter and use the exact same filter to deblurr the images. Introduction Most captured digital image are degraded by blurring and noise and people in the image processing area put much effort to remove the blurring and noise from the true image. Although it is often uncertain what the true image is, we hope to get the best image, which is most likely to produce the detected image. In Statistics, the Maximum Likelihood approach is mostly used. If the detected image g, the PSF (Point Spread Function) h, and the background b are known, the conditional probability density function P g f is a function of f, which we want to get. The paper [1] define the followings. Definition 1 - For a given detected image g the likelihood function is the function of the object f Lg f =P g f defined by: Definition 2 - A ML estimate of the object f is any object f ML which maximize the likelihood function: f ML =arg max f Lg f In all practical application, we define log-likelihood function. J g f = log Lg f terms depending only on g Since this - log function has a minimum point, the non-negative ML estimates are solutions of the problem: f ML =arg minf 0 J g f The Poisson Case. In the Poisson case, the log likelihood function has the following expression based on the reference [1] g n J g f = g n ln Af b n g n Af b n n An iterative method for the computation of the ML solution was proposed by several authors and known as Expectation-Maximization (EM) is as follows:
i) f 0 0 ii) f k 1 =f k T A g Af k b (eq. 1) The Gaussian Case In the Gaussian case, the log likelihood function is given by: 2 2 J g f = g Af b 2= Af g s 2, where g s=g b Although this equation distinguish gs from g, my implementation treats them same because my background 'b' value is 0. Iterative Space Reconstruction Algorithm (ISRA) is: g k 1 = f k AT T s k (eq. 2) i) f 0 0 ii) f A Af Adding Regularization Term From the homework 2, I implemented four different denoising algorithms: H1 regularizations using a Conjugate Gradient and using a Gradient Descendant, Total Variation method, and TV Hybrid Gradient. During this homework 3, I adopted a gradient descent and a TV method to eliminate noise more. From the Poisson case and the Gaussian case, f k 1 is calculated on each iteration. Now, the following equation is used before running the next iteration. Gradient Descendant f k 1 =f k 1 f k 1 f k 1 g (eq. 3) Total variation f k 1 2 f k 1 g (eq. 4) k 1 f For more detail about the above equations, my previous homework or [2] can be referenced. f k 1 =f k 1 dt f k 1 div Fast Fourier Transforms Value of the Fourier Transform of the Least Square solution is F = a inverse FT of G. It means a deblurred image is H G. In matlab, the following equations also eliminate some noises. H [im, in] = size(g); Hf = fft2(h, im, in); f = ifft2((abs(hf) > Threshold).* fft2(g)./hf); (eq. 5) G. For an H image with some noise, a relatively larger threshold value is necessary to eliminate low frequency filter value because low frequency filer value will create huge amplification of the noise. For a blurred image without noise, use a very small threshold value to keep the value of
Testing Results 1. Without Noise Testing filter 1: fspecial('gaussian', 7, 5); Testing filter 2: fspecial('gaussian', 30, 15); Original Image Testing Image 1 Testing Image 2 Using a two different Gaussian filters, the original image is blurred. Larger filter generates wider blurred edges of the square images. From Poisson Test 1 Test 2
Result after 200 After 200/100 After 20/35 iteration Because of convergence pattern, more iteration doesn't always generate better result. Although there is some wiggling intensity on the white square area, edges of the square is well deblurred.
L2 norm 200/100 L2 norm between true image and deblurred image on each iteration shows that the test 1 case converges but the test 2 case doesn't. As a result, test case 1 generates a better image on more, but the test 2 case shows a better result with 35~ 40. From Gaussian Logic Test 1 Result after 200 After 200 Test 2
Similar to the Poisson case, Gaussian equation also recovered blurred image well. Edges get much sharper, but there is some intensity fluctuation on the white square area. L2 norm 200/100 Both cases converge, but the convergence rate is much slower with larger filter size, which causes more blurred Image. 2. With Noise Poisson Case With the testing image, two different filter size is used to blur the image. Then, some Poisson noise is generated to the blurred images. The below results shows the deblurred images and the intensity distribution at the middle (100th) row of the image. For each test case, I also have three different deblurring logic. The first one is the simple deblurring algorithm, and the second and the third one uses a regularization term from a gradient descendant and from a total variation as shown on the second page of this report. For simplicity, I plot the intensity values at 100th row for the three logic and they are shown on the 3rd row of the below results. Test 1 filter: A = fspecial('gaussian', 7, 5); Ib = conv2(double(i), A, 'same'); Ib = uint8(ib); Ibn = imnoise(ib, 'poisson'); Test 2 filter: A = fspecial('gaussian', 30, 15); Ib = conv2(double(i), A, 'same'); Ib = uint8(ib); Ibn = imnoise(ib, 'poisson'); Test 1: Smaller Filter Test 2: Larger Filter
Input Image With Noise Result after 18/40 Result with a TV Regularization. Although there are some remaining noise, much of noise and deblurring were removed. Especially, the boundary edges are well recovered. After 18/40 at 100th row After 18th After 40th Without any regularization term, the middle square area (between 50 ~ 150 on the x axis) has much intensity fluctuation. (True value should be an intensity of 250) Added regularization terms smoothed the fluctuation and also made edge a little sharper.
L2 norm 100 This graph shows a value of L2 norm between a true image and calculated image on each iteration. Without any regularization term, we can see that the process doesn't get converged. With the TV regularization term, the convergence is not also guaranteed. So, the number of iteration needs to be well controlled during the deblurring process. Gaussian Case Test 1 filter: A = fspecial('gaussian', 7, 5); Ib = conv2(double(i), A, 'same'); Ib = uint8(ib); Ibn = imnoise(ib, 'gaussian', 0, 0.01); Test 2 filter: fspecial('gaussian', 30, 15); Ib = conv2(double(i), A, 'same'); Ib = uint8(ib); Ibn = imnoise(ib, 'gaussian', 0, 0.01); Test 1 Result after 23 Test 2
With GD Reg. With GD Reg After 3, 15, 23/ 20, 100, 100 L2 norm 100 Fourier Transforms: Simple square image used for my experiment shows fairly good result when the image is only blurred. When there is some (Gaussian) noise, the recovered image still shows periodic intensity fluctuation. Shown testing image has a simple shape structure. More nature image with much intensity change and several objects on one image was much harder to recover from the noise using the Fourier Transform. Test 1 without Noise Test 1 with Gaussian noise
Input Result From the (eq. 5), a threshold value 0.000000001 is used for the blurred image and 0.2 for the blurredgaussian noise image. By having a threshold value 0.2, all FFT values less than the threshold value become 0. Intensity of the result at 100th row. Result of an noisy image after the FT still has much wiggling intensity values. Conclusion During this homework, I have tried to understand deconvolution methods focused on EM algorithm. Every method shown here works well with blurred image, but the improvement is degraded as an image has some noise. With an noisy-blurred image, the EM works better than other methods.
Disappointing part is that I assume the blurring filter is known. In real world, many trial-error with different filters and parameters will be necessary because we would not know how an image is blurred. For me, it was also a good chance to understand some of Fourier Transform. Reference [1] M. Bertero and P. Boccacci. Image Deconvolution. www.disi.unige.it/person/berterom/paperspdf/asi.pdf [2] Laurent Younes. Mathematical Image Analysis Lecture Note. Johns Hopkins University. [3] Sean Borman. The Expectation Maximization Algorithm - A Short Tutorial. 2006