Motion Compensated Processing Dr Anil Kokaram Electronic and Electrical Engineering Dept.

Transcription

1 IMAGE PROCESSING 1 Motion Compensated Processing Dr Anil Kokaram Electronic and Electrical Engineering Dept. anil.kokaram@tcd.ie sigmedia The aims of this laboratory are as follows 1. To use motion vectors in prediction and processing of images 2. To observe how motion compensated processing differs from non-motion compensated processing 3. To evaluate the difference between IPPP and IBBP schemes for prediction THIS LABORATORY SESSION COUNTS FOR 85 MARKS 1 The EESERVER Resources (at EEE, Trinity College) Test images can be found in y:/image stills y:/sequences. Raw video for restoration can be found in y:/sequences/dirty. All data without.bmp;.jpg;.mpg etc extensions are stored in RAW BINARY FORMAT. Grey scale images are 8 bits per pixel (each pixel is stored as Unsigned Chars). See the read me files in each directory to work out how big the images are. Ask a demonstrator where the data is located in case you have a different set up. You will need to remember what you did in the last laboratory session to do this lab. IN THIS LABORATORY HANDOUT SPECIFIC INSTRUCTIONS ARE INDI- CATED WITH THIS SYMBOL :. YOu might want to use the Matlab command plot for plotting graphs. 2 Motion compensation In the last laboratory you had implemented a Block Matching process in Matlab. For the sequence qonly.360x288.y edit your motion estimation m-file to calculate the motion vectors for frame 1 to 0, and create the motion compensated frame 1 by applying the motion vectors between frames 1 and 0. This is easy to say but suprisingly tricky to do the first time you do it. Here are some tips. 1. Create an image called PredCurrent which is the same size and type as the current image but is filled with zeros. Remember about using zeros(m,n) in Matlab? 2. When you work out the best motion vector for a particular block with top left hand corner (j, i), say, store the best matching block in the past frame as a block PredBlock.

2 IMAGE PROCESSING 2 3. Place PredBlock into the corresponding location in PredCurrent. It should go into the location with top left hand corner starting at (j, i). When you re done with all the blocks in an image, then PredCurrent should be the motion compensated version of the previous frame. Hence the difference between the current frame and this prediction frame is the motion compensated difference between each frame. Write below a table of the mean absolute error of this motion compensated frame difference for frames 1 to 10 in qonly... By displaying both the motion compensated frame difference and the motion compensated frame, comment on where the motion estimation is allowing a good prediction and where it is not.

3 IMAGE PROCESSING 3 3 B Vs P In MPEG2 there are three types of image frames I (Intra), B (Bidirectional) and P (Predicted) frames. B frames are created by predicting the current image using the previous and next I or P frame as appropriate for motion compensation. What you have just done in the previous section is generated motion compensated errors for P frames always based on the previous frame recieved in the sequence. Now edit your m-file to create B frames using the previous and next frame in the sequence. This means for each block you have to do motion estimation twice, once from the Current frame into the Past frame and again for the current frame into the Next frame. Here are some more hints. 1. Add a line to load up the next frame into Matlab as well as the current and previous. 2. Create another array to store vectors for the Forward direction as well as the backward 3. Copy your block error code into the lines below so that there are two error loops. Change the second one so that it compares the current with the next frame and generates Forward motion vectors as well as forward prediction and error pictures. 4. At each pixel site choose the prediction which gives the smaller of the two DFDs and make up the corresponding bi-directionally predicted frame as well as bi-directionally predicted error.

4 IMAGE PROCESSING 4 Write below a table of the mean absolute error of this Bi-directional motion compensated frame difference for frames 1 to 10 in qonly Marks By displaying both the motion compensated frame difference and the motion compensated frame, compare and contrast prediction with P frames and B frames.

5 IMAGE PROCESSING 5 With what you have just learned, comment on the coding efficiency in bits/pel of P frames, B frames and I frames in MPEG2. 5 Marks 4 Noise reduction The sequence qonly.360x288.v100.y is a noisy image sequence generated from qonly.360x288.y which is the clean orginal. It is to be processed with two kinds of video noise reducers. Type A is non-motion compensated while Type B is motion compensated. Make a copy of your motion estimation files above and rename the copy nmcdnoiser.m. Edit that file to load in the corresponding frame from the clean and corrupted files and calculate the mean absolute error between the two frames for each frame 1 to 30. Plot this error in a graph below showing frame on the x axis and MAE on the y axis.

6 IMAGE PROCESSING 6 5 Marks Edit that file to remove the calculation of the motion vectors and instead implement a simple non-motion compensated noise reducer with output frame În defined as follows. Î n = 0.2În G n (1) where G n is the noisy input image sequence. Plot the MAE between the de-noised sequence În and the orginal clean sequence superimposed on the graph above, for frames 1 to 30. Is the MAE better or worse than before? Why? Make a copy of your m file that you used to generate P frames. Edit that file to perform motion compensated noise reduction as follows where În 1(v) is the motion compensated previous output frame. Î n = 0.2În 1(v) + 0.8G n (2) Plot the MAE between the de-noised sequence În and the orginal clean sequence superimposed on the graph above, for frames 1 to 30. Compare and contrast the motion compensated and non-motion compensated methods with respect to picture quality and MAE.

7 IMAGE PROCESSING 7 20 Marks