The Wavelet Transform & Motion Estimation. 1 The EESERVER Resources (at EEE, Trinity College)

Transcription

1 IMAGE PROCESSING 1 The Wavelet Transform & Motion Estimation Dr. David Corrigan & Prof. Anil Kokaram & Dr. Gary Baugh Electronic and Electrical Engineering Dept. corrigad@tcd.ie anil.kokaram@tcd.ie baughg@tcd.ie sigmedia The aims of this laboratory are as follows 1. To compare the suitability of different wavelet filters for compression using rate Distortion Curves 2. To implement a Block Matching Algorithm in Matlab 3. To investigate the effects of changing BM parameters 4. To explain where BM works in the image and where it does not THIS LABORATORY SESSION COUNTS FOR 60 MARKS 1 The EESERVER Resources (at EEE, Trinity College) Test images can be found in y:/image stills y:/sequences. Raw video for restoration can be found in y:/sequences/dirty. All data without.bmp;.jpg;.mpg etc extensions are stored in RAW BINARY FORMAT. Grey scale images are 8 bits per pixel (each pixel is stored as Unsigned Chars). See the read me files in each directory to work out how big the images are. Ask a demonstrator where the data is located in case you have a different set up. You will need to remember what you did in the last laboratory session to do this lab. IN THIS LABORATORY HANDOUT SPECIFIC INSTRUCTIONS ARE INDI- CATED WITH THIS SYMBOL :. You might want to use the Matlab command plot for plotting graphs. 2 The Wavelet Transform The Matlab file wavelet demo.m contains code that calculates the 4-level 2D Discrete Wavelet Transform (DWT) of lenna using the haar wavelet filters. The wavelet coefficients are the quantised with a quantisation step size of 15 and the inverse transform is applied to estimate the reconstructed image.

2 IMAGE PROCESSING 2 Edit the code to calculate the mean absolute error between the original and reconstructed images and wite down your answer below. 2 Marks Figure 2 generates an image containing the wavelet coefficients. Adjust the code to change the number of levels to 2. Draw a diagram that indicates the filter action on each band (eg. HiHi, LoHi etc.) and also states the level of each band. 3 Marks

3 IMAGE PROCESSING Comparing Wavelets for Compression A rate distortion curve is a parametric plot of the Mean Absolute Error (MAE) against the DWT entropy and is obtained by measuring these quantities for different values of the quantisation step size. Using 4 levels of the DWT, generate a rate distortion curve for the haar wavelet on the lenna image using the following quantisation step sizes. Draw the graph you obtain below. Q step = [ ]. 5 Marks

4 IMAGE PROCESSING 4 Matlab allows you to implement the wavelet transform using many other types of wavelet filters (apart from the haar wavelet). The waveinfo function gives a list of the wavelet filters that matlab implements. It is also possible to implement any other wavelet filter that you might want to. Adjust your code to estimate rate distortion curves for both the db4 and the bior2.2 wavelets using the same quantisation step sizes as before. Plot each curve on the same axis. The plot should also contain the curve for the haar wavelet you plotted earlier. Make sure to clearly label your plot. 7 Marks

5 IMAGE PROCESSING 5 Use the curve to select the wavelet filters most useful for compression and justify your answer. 3 Marks

6 IMAGE PROCESSING 6 3 The Block Matching Algorithm The Matlab file bm lab template.m contains the skeleton of a Block Matching algorthm with just the error calculation missing. It uses a full search strategy. The matrix error is supposed to hold the MAE corresponding to each vector tested for a particular block. Edit the file to calculate the MAE for each vector in the place indicated. Write the lines of matlab code (that you added) in the box below. Edit the file to display the motion compensated error for the whole frame (stored in dfd) as Figure Marks The data that you will use initially is in qonly.360x288.y. It is a sequence of frames from a Bond movie stored as raw data. Each pixel is 8bit, and each frame is stored one after the other sequentially in a binary file. Each frame is stored in the same way as a raw image would be. 3.1 Performance It is typical to evaluate the performance of a motion estimator by plotting the average motion compensated error in some way versus the frame number. The motion compensated frame difference at each pixel site is already stored for you in dfd, noting that the edges are ignored. Each entry in that matrix therefore shows e(h, k) = I n (h, k) I n 1 (h+vn,n 1(h, x k), k+vn,n 1(h, y k)) at each site (h, k). Here the motion vector mapping position (h, k) in frame n into the previous frame n 1 is (vn,n 1(h, x k), vn,n 1(h, y k)), where the horizontal component of motion is vn,n 1(h, x k) and vn,n 1(h, y k) is the vertical component. The MAE over the whole frame between frames n

7 IMAGE PROCESSING 7 and n 1 is then E n,n 1 = 1 HK h=h 1 h=0 k=k 1 k=0 e(h, k) (1) Edit the Matlab file to calculate this error for the first 30 frames in qonly.360x288.y. Plot a graph of E n,n 1 Vs frame below. 5 Marks

8 IMAGE PROCESSING 8 Increase the search with w used in the algorithm. Superimpose the new graph on the plot above. Comment below on the difference in performance, mention speed and quality of motion estimation. Explain the differences. 5 Marks

9 IMAGE PROCESSING 9 4 Where does it work? By considering your results point out the regions in the images where Block Matching works well and where it does not. Explain your findings. 5 Marks

10 IMAGE PROCESSING 10 5 Final Investigation Try changing block sizes and motion threshold to improve the MAE plots for qonly.360x288.y. List all the combinations you try and rank them in in terms of their MAE performance and computation times. How does changing the block size and motion threshold affect the resulting MAE and motion field? 10 Marks