Simulation and Analysis of Interpolation Techniques for Image and Video Transcoding

Transcription

1 Multimedia Communication CMPE-584 Simulation and Analysis of Interpolation Techniques for Image and Video Transcoding Mid Report Asmar Azar Khan

2 Objective: The objective of the project is to analyze different Interpolation techniques used for Image scaling ad resizing. The issues faced and their handling. Transcoding is transferring one scheme of data encoding to some other technique on the fly. This is required when multimedia data is shared over network by different types of nodes, where every node has its own spatial resolution and decoding scheme. Transcoding is decoding the original bit-stream and encode that according to the node decoding schemes so that there could be various types of receivers available for sharing. This project only focuses the Image scaling part of the transcoding. I have simulated different interpolation schemes and compared them on the basis of their spatial quantitative and frequency domain analysis. The performance metric and simulation results will be followed later. Image Scaling: Image scaling is the process of changing the size of a digital image. Scaling is a non-trivial process that involves a trade-off between speed, smoothness and sharpness. When increasing the size of images, the fact that digital images are made of pixels becomes particularly evident. Digital Image Processing is an active area of research these days because of digital representation of signals. The Image scaling has been adopted by the researchers as per the requirements.image scaling can involve either subsampling (reducing or shrinking an image) or zooming (enlarging an image). Generally though, the zoom operation is used more rarely than the shrink operation. Different techniques which have been used for Image and video scaling are as follows Nearest Neighbor Interpolation Bilinear Interpolation Bicubic Interpolation Quadratic Interpolation Lagrange Interpolation Spline Interpolation These are the techniques which are commonly used in image scaling techniques. As it has been discussed in earlier chapter therefore the need of the image scaling is not discussed here again. However these all techniques are different in implementation, design and time complexity. CMPE-584 1

3 Ideal Interpolation: Any signal is best described in by its Fourier spectrum. The image also follows the same when we need to define the image and interpolate it the desired phenomenon is to have a scenario when no frequency component is dropped. This generates the requirement of understanding the Nyquist theorem. The theorem suggests that if we want to sample a continuous signal in digital domain we should sample at a rate greater than twice the largest frequency component present there. This will avoid aliasing effects. The Fourier transform can be described by the following expression and the time complexity is N x N. Therefore for image scaling purpose we do not want such complex operations and avoid Fourier transforming prior to source coding. The kernels which are used in the all the techniques of interpolations are mostly implemented in spatial domain. Coming back to ideal interpolation point, we say that it is ideal to have a band pass filter for interpolation which passes all the frequency components in the required band and stop rest all. i.e. it should be a rectangular step function in frequency domain filter. This equates to a sinc function defined as follows in the time domain. h(x) = sin(pi*x)/(pi*x) fig1: (a) Spatial response of an ideal kernel, (b) Frequency response of the same Courtesy: Survey: Interpolation methods in medical image processing Thomas M. Lehman IEEE TRANSACTIONS ON MEDICAL IMAGING, VOL. 18, NO. 11, NOVEMBER 1999 CMPE-584 2

4 This is the response of any ideal filter which can be used as interpolation kernel, but to generate this we need to have an infinite time response of the image which is impossible so many researchers have come up with the approximation of this response and those are the interpolators which are commonly used in image and video scaling domain, some of them are; Nearest Neighbor Interpolation: Nearest neighbor interpolation is a simple method of multivariate interpolation in 1 or more dimensions. The nearest neighbor algorithm simply selects the value of the nearest point, and does not consider the values of other neighboring points at all. The algorithm is very simple to implement, and is commonly used in real-time 3D Image and video applications. The function is easiest to implement. This requires only 1 next value of the required pixel and the impulse response of the filter reuqired is ggiven by the following function. Here we assign the value of the nearest pixel to the required pixel or interpolated pixel. This impulse response when convoluted with the image will generate the nearest neighborhood interpolation. Any filter s impulse response can best be anlysed in the frequency domain, so by taking the fourier transform of the above mentioned filter will generate the sinc function. Bilinear Interpolation: Linear interpolation is a method of curve fitting using linear polynomials. It is heavily employed in mathematics, and numerous applications including computer graphics. We know the coordinates (x 0, y 0 ) and (x 1, y 1 ). We want to pick points on this line with a given x in the interval [x 0, x 1 ]. Bilinear interpolation is an extension of linear interpolation for interpolating functions of two variables on a regular grid. The key idea is to perform linear interpolation first in one direction, and then in the other direction. Bicubic Interpolation: Bicubic interpolation is a robust technique compared to with others interpolation techniques and their results are contained within the best ones. Bicubic interpolation requires a neighborhood of 16 pixels, 16 floating-point operations, and there is similarity with window processing. Therefore techniques of spatial and temporal parallelism can be used for their implementation, making the bicubic interpolation technique candidate to be implemented using FPGA technology. Bicubic interpolation does no suffer the step-like boundary problem of nearestneighbourhood interpolation, and copes with linear interpolation blurring as well. Bicubic interpolation is often used in raster displays that enable zooming with respect to an CMPE-584 3

5 arbitrary point. If nearest-neighbourhood methods were used, areas of the same brightness would increase. Bicubic interpolation preserves fine details in the image very well. Quadratic Interpolation: One of the easy kernals to implement for sinc interpolation is quadratic. Their advantage is easy determination and uniform approximation of continuous functions at finite intervals. In the previous sections, constant and linear polynomials have been discussed. Quadratic functions have been disregarded largely because they have been thought to introduce phase distortions. In fact, if the polynomials span -1 to 2, asymmetric kernels with nonlinear phases are produced. where Ai, Bi and Ci are real numbers. To form a kernel useful for interpolation, additional restrictions must be imposed. The polynomials should fit exactly at the kernel s starting and ending points as well as at their contacting points. Lagrange Interpolation: Lagrange interpolation is a way to pass a polynomial of degree N-1 through N points. Lagrange polynomials are the interpolating polynomials that equal zero in all given points, save one. The Lagrange form of the interpolation polynomial shows the linear character of polynomial interpolation and the uniqueness of the interpolation polynomial. Therefore, it is preferred in proofs and theoretical arguments. Lagrange and other interpolation at equally spaced points, yield a polynomial oscillating above and below the true function. Lagrange interpolation is often used in digital signal processing of audio for the implementation of fractional delay FIR filters. Spline Interpolation: Spline is a special function defined by polynomials. In interpolating sense Spline is often preferred to others because it yields similar results, even when using low degree polynomials, while avoiding Runge s phenomenon for higher degrees. Spline is preferred because of their ease and accuracy of evaluation, and their capacity to approximate complex shapes through curve fitting and interactive curve design. Using polynomial interpolation, the polynomial of degree n which interpolates the data set is uniquely defined by the data points. The spline of degree n which interpolates the same data set is not uniquely defined, and we have to fill in n-1 additional degrees of freedom to construct a unique spline interpolant. CMPE-584 4

6 Simulation: MATLAB is used for the comparative analysis of all these interpolation schemes. By now I have implemented nearest, linear, cubic and spline interpolation techniques and tested. Any image when resample is filtered by a low pass FIR filter. This is done to avoid aliasing. This aliasing occurs because of sampling the data at a rate lower than twice the largest frequency component of the data. So a low pass filter will remove the image high frequency components and for this purpose the filter used was a 7 tap filter which was suggested by MPEG-1 coding scheme. This filter is used for conversion of CCIR to SIF resolution conversion. [ ]*1/ Test Iamge: fig 2: Frequency response of the linear phase FIR low pass filter Test Image was taken as cameraman image. The image was initially 512 x 512 resoultion. For analysis purposes this was reduced to 256 x 256 by dropping the samples. Then the reduced image was enlarged by using different interpolaiton techniques. And the results are shown in following figures. Interpolation SNR (dbs) Linear Nearest Cubic Spline Table 1 : SNR calculation for different Interpolation techniques CMPE-584 5

7 Performance Analysis : The performance analysis in spatial and fourier domain is next in the project. Final report will comprsie all the results and performance analysis of the tachniques. References: Thomas M. Lehmann Survey: Interpolation Methods in Medical Image Processing IEEE transactions on medical imaging, vol. 18, no. 11, November 1999 George Wolberg Monotonic Cubic Spline Interpolation Robert G. Keys Cubic convolution interpolation for digital image Processing IEEE transactions on acoustics, speech, and signal processing, vol. Assp-29, no. 6, December Anthony Vetro Video Transcoding architectures and techniques an overview YongQing Liang and Yap- Peng Tan A new content-based hybrid video Transcoding method 2001 Tsung-Ching Lin, Trieu-Kien Truong DCT-Based Image Codec Embedded Cubic Spline Interpolation with Optimal Quantization Leonid P. Yaroslavsky DFT and DCT Based Discrete Sinc-interpolation Methods for Direct Fourier Tomographic Reconstruction YongTaek Hong KyungHo Leet, Jingsang Kim, and Won-Kyung Cho High Speed Architecture for MPEG-2 and H.264 Video Transcoding Masaru Sugano, Yasuyuki Nakajima, Hiromasa Yanagihara, and Akio Yoneyama An efficient Transcoding from MPEG-2 to MPEG CMPE-584 6

8 fig 3: Test Image cameraman Nearest Interpolation twice scaling fig 4: Enlarged image 2 times using Nearest Interpolation CMPE-584 7

9 Linear Interpolation twice scaling fig 5: Enlarged image 2 times using Linear Interpolation Cubic Interpolation twice scaling fig 6: Enlarged image 2 times using Cubic Interpolation CMPE-584 8

10 Spline Interpolation twice scaling fig 7: Enlarged image 2 times using Spline Interpolation CMPE-584 9