Luleå University of Technology Matthew Thurley and Johan Carlson Last revision: Oct 27, 2011 Industrial Image Analysis E0005E Product Development Phase 6 The 2D Fourier transform & image filtering Contents 1 Background and Introduction 2 2 Assessment 2 3 Test the Fourier Transform 3 4 Image filtering 4 4.1 Task 1........................................ 4 4.2 Task 2........................................ 5 4.3 Task 3........................................ 7 4.4 Task 4........................................ 7 5 Image reconstruction 7 5.1 Tasks........................................ 7 6 Reporting Assessment 8 1
1 Background and Introduction Read all of the instructions in this document before starting. This week you experiment with image filtering based in the frequency domain and use the 2D fourier transform. This will give you enhanced knowledge of image filtering as well as providing you with the necessary background to help you identify tools in development phase 8 using a technique called fourier descriptors. There are no optional parts to this weeks development phase. Remember that you need to write a clear and reasoned report outlining how and what you have done as outlined in development phase 1. 2 Assessment Each Development Phase will be graded fail (U), pass (3), good (4), or very good (5) based on the following criteria; As this weeks development phase does not contain any optional components a grade of 5 will be determined based on the quality of your explanations and answers to questions. 1. How well does the report answer all the questions outlined in the development plan. Does the student provide a thorough and complete answer? 2. How clearly written is the report and how well reasoned is the explanation? How easy would it be for a technically qualified person to understand what you have done and why, and then follow the development phase instructions and your report and repeat the results? 3. When you are asked to explain or discuss something, explain why your result is this way in relation to the theory from the course material (necessary in order to achieve a grade of 4 or 5) 2
3 Test the Fourier Transform Here is an artificial example of decomposing an image into its frequency components using the fourier transform. In this example we make it easier to think about by first constructing the image from sinusoids and then using the fast fourier transform to find the frequencies from which we constructed the image. (Credit to Anders Landström for this example). Optionally you may want to close all existing figures and clear memory with these commands close all force clear clc Next we create two images using a meshgrid() and linear space function, and then display these two images. %% Create two coordinate grids (equivalent to two images) [X Y] = meshgrid(linspace(0,2*pi,512),linspace(0,2*pi,512)); %% Display the grids and wait for the user to press a key imtool(x,[]); imtool(y,[]); Now we fill in these images with intensity values created from simple sinusoids, and then add the images together to create a master image called M that contains sinusoids in the X and Y axes, that we will perform a fourier transform on. %% Create sinusoids with values in the interval [0 5] and display them: % sin(2*x): this will have 2 periods along the X axis Z1 = (sin(2*x)+1)./4; imtool(z1); % sin(4*y): This will have 4 periods along the Y axis Z2 = (sin(4*y)+1)./4; imtool(z2); %% Add the sinusoids together to create our Image M = (Z1 + Z2); imtool(m); Now we perform the fourier transform and show the resultant image. 3
%% Fourier transform: m = size(m); F = fft2(m,m(1),m(2)); %% Shift the data and look at the content. Fs = fftshift(f); imtool((abs(fs)),[]); The first thing you will notice is that the resultant image, Fs is almost entirely black with a tiny white spot in the middle. If you zoom in on this spot you discover that it is actually 5 non black pixels. The white pixel is at the origin, and the grey pixels are at (-2,0) (2,0) (0,4) (0,-4) and they indicate that there are sinusoids in the image with period 2 in the x axis direction, and sinusoids with period 4 in the y axis direction. So if we want to remove the sinusoids along the x axis, we can set the grey pixels at (-2,0) and (2,0) to 0. The pixel coordinates for these values are actually (255,257) and (259,257). So lets see what happens when we set them to zero and then do the inverse fourier transform. %% Remove appropriate frequencies in order to bring back the Y-sinusoids only % (Tip: Look at the peaks in abs(fs) and where they are in relation to % the origin (the middle of the shifted image) % Set two values in the image to zero at the same time Fs(257,[255 259]) = 0; % Reduce the intensity at the origin because the x sinusoids add to this % value as well Fs(257,257) = Fs(257,257)/2; %% Undo the shifting: F2 = ifftshift(fs); %% and go back to the spatial domain: MF = ifft2(f2,m(1),m(2)); MF = MF(1:size(M,1),1:size(M,2)); %% Display the result: imtool(mf); As we desire, the image MF contains only the y axis sinusoids So what we see is that the fourier transform represents the sinusoidal components of an image, and if we edit the fourier transform, or filter it in someway, and then invert the fft, we can remove parts of the original image. 4
4 Image filtering 4.1 Task 1 Load and view the image caesar.tif into variable Im. Take the 2D Fourier transform of the image Im (see help fft2) and store the result in the variable Ti. Study the magnitude of the result with (see MATLAB help for abs and fftshift). Note: when displaying images in this phase you will need to rescale the intensity range. You may need to be aware of what the actual min and max values in the image are in order to understand some of your results. There is also another function imagesc that combines image and a version of rescale. image(rescale(fftshift(abs(ti)))); You probably can not say much by looking at the figure because most of the image is too dark to see the variation. The reason is that most of the figure s energy is confined to the lowest frequencies. To see some of this information, type plot(fftshift(abs(ti(129,:)))); What does Ti(129,:) mean? We re extracting something from the spectrum of the image, but what? A common way of displaying the spectrum is in a logarithmic scale as it lets us more easily view the range of information in the fourier transform. Try this: imagesc(fftshift(log(abs(ti)+1))); The logarithm will compress the dynamic range of intensities, and preserve the zeros (because of the +1 in the expression above). The Fourier transform is complex number valued. Hence, it can be written as: F(k,l) = F(k,l) e j F(k,l), where F(k,l) is the magnitude and F(k,l) is the argument (or the phase) of F(k,l), respectively. Now, let s remove the phase information. Use help ifft2 and help real to see what these function do. NoPhase = real(ifft2(abs(ti))); imagesc(nophase); Study the result! Now, let s keep the phase information while removing the magnitude, i.e. NoMag = real(ifft2(ti./abs(ti))); imagesc(nomag); Reporting Which of phase and magnitude is the most important regarding edge information and why? When designing filters, what is most important: preserving the phase or the magnitude? 5
Think about what sort of filters and give some examples. 4.2 Task 2 Load the file h1.mat and study it using load h1; colormap(hsv); mesh(h1); h1 is an isotropic (which means it is rotational symmetric) FIR filter. Now look at the transformed filter with H1 = fft2(h1,256,256); % Zero-padd h1 to size 256x256 mesh(fftshift(abs(h1(1:2:256, 1:2:256)))); What are the frequency characteristics of this filter? It might be clearer if you write the following to plot a part of the filter; f = -pi:(pi/128):(pi-pi/128); plot(f, fftshift(abs(h1(128,:)))); Now, let s perform the filtering in the spatial domain and study the result Ci = conv2(im,h1, same ); colormap(gray(256)); imagesc(ci); Then, perform the filtering in the frequency domain and study the result [x,y] = meshgrid(0:255); Cm = exp(1j*2*pi*7*(x+y)/256); Fi = real(ifft2(ti.*h1.*cm)); imagesc(fi); Note: The Cm variable is a technicality in terms of a translation in the Fourier domain, necessary if we later want to compare the images. Spend a couple of minutes thinking about what it actually means. Hint: F{x(t τ)} = X(f)e j2πfτ. Compare the two resulting images (filtering in the spatial domain and in the frequency domain) and study the difference abs(ci-fi). Ci is the result of a linear convolution in the spatial domain, whilefi is the result of a circular product in the frequency domain. Hence, the boundary conditions differ for the two cases (see DIP, section 4.6.6). Perform a linear convolution in the Fourier domain, by first zero-padding the signals. Use the new padded image size (256+14) (256+14), i.e. we add a frame of size size(h1)-1 to the images. See help fft2 on how to get a padded image when performing a transform, and recalculate the necessary transformed images. Call the result after the inverse transform Pi. You will also need the translation operator Cm as below in the inverse transform 6
[x,y] = meshgrid(0:255+14); Cm =exp(1j*2*pi*7*(x+y)/(256+14)); Now, extract the relevant part of the image Pi, everything except the padding, using Li = Pi(1:256,1:256) and study the difference abs(ci-li). Reporting: Supply difference images abs(ci-fi) and abs(ci-li) and explain the results with reference back to the material presented in the course. 4.3 Task 3 Load the.mat-file h2 (which is another FIR filter). Repeat all the operations in from task 2 using h2 instead of h1. Reporting: Supply difference images abs(ci-fi) and abs(ci-li) and explain the results with reference back to the material presented in the course. 4.4 Task 4 Implement a linear convolution of Im with the LoG operator from development phase 3. Reporting: Supply your (well commented) MATLAB code and the resultant image. 5 Image reconstruction 5.1 Tasks I recently visited a sporting event and took some pictures with my camera. Things were going rather fast and unfortunately I used a much too long exposure time. This resulted in a blurred image. Load the image blur and study it (it is of size 512 512). After thinking hard about the situation I came up with a discretized model of the blurring process. The blurred image B is related to the original (ideal) image I as B(i,j) = N a k I(i,j k), k=1 i.e. B is a weighted sum of N translated copies of I. I estimated the number N = 256. The coefficients a k is a model of the declining flash light, given by a k = 256 k N l=1 l. The plan is to reconstruct the original image I from the blurred image B in the Fourier domain by first determining an expression of the 2D discrete Fourier transform Fb of the blurred image. If the transform Fb can be written as Fb = Fi H, where Fi is the 2D Fourier transform of the ideal image I, then it might be possible to reconstruct I by inverse transforming the expression F b/h. Your job is, of course, to test this hypothesis. 7
First, find an expression for the blur operator H, then create H as a MATLAB matrix. Then create Fb and finally, reconstruct I with the command I = real(ifft2(fb./h)); Smooth the result by convolving it with h1 from the previous assignment and remove the border ( 10 pixels) from the resulting image. Reporting: Supply your (well commented) MATLAB code and the de-blurred image. Explain the results. 6 Reporting Assessment Write a short evaluation of how well your report satisfies the following criteria. How well does your report answer all the questions outlined in the development plan. Do you provide a thorough and complete answer? How clearly written is your report and how well reasoned the discussion. How easy would it be for a technically qualified person to follow your report to repeat the results? Are there any problems with your solution? The purpose of this assessment question is for you to evaluate your own work like you would have to in a real work situation before handing it over to your technical supervisor. 8