Research Article Video Superresolution Reconstruction Using Iterative Back Projection with Critical-Point Filters Based Image Matching

Transcription

1 Advances in Multimedia Volume 2015, Article ID , 10 pages Research Article Video Superresolution Reconstruction Using Iterative Back Projection with Critical-Point Filters Based Image Matching Yixiong Zhang, Mingliang Tao, Kewei Yang, and Zhenmiao Deng Department of Communication Engineering, Xiamen University, Xiamen, Fujian , China Correspondence should be addressed to Yixiong Zhang; Received 27 December 2014; Accepted 23 February 2015 Academic Editor: Constantine Kotropoulos Copyright 2015 Yixiong Zhang et al. This is an open access article distributed under the Creative Commons Attribution License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited. To improve the spatial resolution of reconstructed images/videos, this paper proposes a Superresolution (SR) reconstruction algorithm based on iterative back projection. In the proposed algorithm, image matching using critical-point filters (CPF) is employed to improve the accuracy of image registration. First, a sliding window is used to segment the video sequence. CPF based image matching is then performed between frames in the window to obtain pixel-level motion fields. Finally, high-resolution (HR) frames are reconstructed based on the motion fields using iterative back projection (IBP) algorithm. The CPF based registration algorithm can adapt to various types of motions in real video scenes. Experimental results demonstrate that, compared to optical flow based image matching with IBP algorithm, subjective quality improvement and an average PSNR score of 0.53 db improvement are obtained by the proposed algorithm, when applied to video sequence. 1. Introduction Since high-resolution (HR) images/videos are important in many applications, such as astronomy, military monitor, medical diagnosis, and remote sensing, superresolution (SR) reconstruction has a great significance in practice [1]. The concept of superresolution (SR) reconstruction refers to reconstructing a high-resolution (HR) image from one or more low-resolution (LR) images. The purpose of superresolution (SR) reconstruction is using digital image processing algorithm to enhance the spatial resolution by transcending the limiting factors of optical imaging system [2, 3]. The innate character of superresolution (SR) reconstruction is using complementary content of multiple images to extend high frequency component. Most superresolution reconstruction methods contain four steps: registration, map, interpolation, noise, and blur removal. Registration refers to estimating motion vectors between two different video frames or images. Then the motion vectors are used to map the pixels of the input lowresolution frames to a common high-resolution reference frame. Interpolation is used to obtain the pixel value of the superresolution grid, by utilizing the mapped pixels. Finally, noise and blur removal is applied to eliminate the optical sensor blur [4]. A variety of iterative superresolution reconstruction algorithms have been proposed. These algorithms can be divided into two types: frequency domain method and spatial domain method.thefrequencydomainapproachwasfirstproposed by Tsai and Huang [5]. They formulated a series of equations which relate high-resolution frames to low-resolution frames, making use of the shift property of Fourier transform. But their imaging model does not consider motion blur and additive noise and is only restricted to global translational motion. Interpolation based method, iterative back projection (IBP) method [1, 6, 7], projection onto convex set (POCS) method [3, 8], and maximum a posteriori (MAP) method [9]arefour main spatial domain algorithms for superresolution reconstruction. Interpolation based method is the simplest spatial domain algorithm, which uses multiple registered images to generate the HR image, based on some interpolation approach, such as nearest-neighbor interpolation, bilinear interpolation, and cubic spline interpolation [10]. The other three spatial domain SR algorithms (IBP, POCS, and MAP) are all based on iterative reconstruction and have better results than interpolation based method. Another class of

2 2 Advances in Multimedia superresolution reconstruction algorithm is example-based superresolution algorithm. Freeman et al. [11] presented the idea of example-based superresolution reconstruction algorithm. A state-of-the-art algorithm via structure analysis ofpatcheswasproposedbykimetal.in[12]. The key to the spatial domain algorithms is accurate image registration. Traditional registration approaches estimate the global translational or rotational motion. But if more than one type of motion coexists in the scene such as natural video, global motion estimation does not work. In video superresolution, image registration algorithms such as block based matching [13] and optical flow based matching [14, 15] are often used. But both of these two algorithms have some drawbacks and are not appropriate in the superresolution reconstruction. Motion vectors cannot be accurately obtained because of block based matching criterion, and the performance of optical flow based matching may be seriously degraded due to changes of brightness. In this paper, image matching using multiresolutional critical-point filters (CPF-IM) [16 18] is proposed to be applied in the process of superresolution reconstruction. CPF-IM is suitable for representing both the global and local motions. Moreover, the influence of brightness change is small in CPF-IM. In our experiments, pixel-level motion fields are first obtained by CPF-IM. Then the iterative back projection (IBP) algorithm is employed to reconstruct the high-resolution image based on the motion fields. Experimental result shows that the IBP algorithm with CPF-IM has better performance than bilinear interpolation algorithm, bicubic interpolation algorithm, and optical flow based image matching with IBP algorithm. The rest of this paper is organized as follows. Section 2 briefly introduces the CPF based image matching algorithm. Section 3 describes the proposed superresolution algorithm using iterative back projection (IBP) algorithm with CPF- IM. Section 4 discusses the experimental results. Finally, Section5 concludes this paper. 2. Image Matching Using Multiresolutional Critical-Point Filters Multiresolutional critical-point filters (CPF) [16 18] provide a means of matching two images in pixel-level accurately. Suppose there are two images, a source and a destination. A set of multiresolution subimages are constructed for both images. Then, mappings from the source to the destination subimages are performed at each level from the coarsest resolution to the finest resolution. The mapping is computed pixelbypixelconstrainedbytheinheritedandthebijectivity conditions. The mapping with the minimum energy will be selected as final correspondence. Supposing the width and the height of the original image size are W and H, respectively,n denotes the hierarchy level of the finest resolution. A multiresolution hierarchy of size W/2 (n m) H/2 (n m) images can be computed. There are four subimages to be calculated, which are formed by extraction of the minimum, the maximum, and the saddle points, respectively, at each level of the hierarchy. Let p (m,s) denote the pixel at (i, j) in the subimage, where m is the level of hierarchy and s is the type of subimage. The pixels of subimages in the hierarchy are recursively calculated from the pixels of its higher level subimages in the hierarchy as follows: p (m,0) p (m,1) p (m,2) p (m,3) = min (min (p (m1,0) (2i,2j),p (m1,0) (2i,2j1) ), min (p (m1,0) (2i1,2j),p(m1,0) (2i1,2j1) )), = max (min (p (m1,1) (2i,2j),p (m1,1) (2i,2j1) ), min (p (m1,1) (2i1,2j),p(m1,1) (2i1,2j1) )), = min (max (p (m1,2) (2i,2j),p (m1,2) (2i,2j1) ), max (p (m1,2) (2i1,2j),p(m1,2) (2i1,2j1) )), = max (max (p (m1,3) (2i,2j),p (m1,3) (2i,2j1) ), max (p (m1,3) (2i1,2j),p(m1,3) (2i1,2j1) )), where p (n,0) =p (n,1) =p (n,2) =p (n,3) =p,whicharethe pixels of the original image. Once the multiresolution hierarchy is constructed, a top down method is utilized to map pixels from the source image to the destination image. The number of candidate mappings at each level is constrained by the mapping at its upper level. Apixelp at level m ofthesourceimageissearchingforits corresponding pixel q in the destination image. Suppose the 4nearestpixelsofthepixelp are a, b, c,andd. Their parents (A, B, C, D) aremappedtoa, B, C, D at level m 1.For each of the parents, one child pixel is selected. The four children pixels define an inherited quadrilateral a b c d,inside whichwesearchthepixelq with a minimum mapping energy. Let p (m,s) (i, j) bethepixeltomapinsourceimageat location (i, j) and q (m,s) (k, l) the pixel to test in destination image at location (k, l) = f (m,s) (i, j). The mapping energy E (m,s) (i, j) consists of G (m,s) (i, j) and H (m,s) (i, j), defined as E (m,s) =λg (m,s) (1) H (m,s), (2) where λ 0 is a real number. G (m,s) (i, j) denotes the intensity differences between the source image pixel and its corresponding pixel in the destination image. Consider G (m,s) (i, j) = I(p(m,s) (i, j)) I (q (m,s) (k, l)) 2, (3) where the function I( ) denotes the intensity of image pixel. And H (m,s) (i, j) is the cost related to the locations of pixel. Consider H (m,s) =ηh (m,s) 0 H(m,s) 1, (4)

3 Advances in Multimedia 3 where η 0is a real number. Consider H (m,s) 1 = i H (m,s) 0 = (i, j) f(m,s) (i, j) 2, j i =i 1 j =j 1 ( (f(m,s) (i, j) (i, j)) (f (m,s) (i,j ) (i,j 2 )) ) (4) 1. (5) H 0 is determined by the differences between (i, j) and f(i, j) to prevent a pixel being mapped to a pixel too far away. H 1 is determined by the distance between the displacement of p (m,s) (i, j) and the displacement of its neighbors. This energy is used to smooth the mapping. In the mapping procedure, the energy of the candidate pixels satisfying the above conditions will be computed and compared. Then the pixel with the minimum energy will be determined as the final corresponding pixel. The advantage of integrating CPF based image matching into video image superresolution is that pixel-level motion fields can be accurately obtained. Shifts usually vary across small regions between video frames; therefore, block based image matching is not accurate enough. Moreover, optical flow based image matching is seriously affected by brightness change. In summary, CPF based image matching can overcome the defects of traditional image matching algorithms and is applicable to global motion and local motion model. In the process of image matching, one of the low-resolution images is selected as the reference image. CPF based image matching is performed between the reference image and the other low-resolution images to obtain motion fields. 3. Integrate CPF Based Image Matching into Video Image Superresolution This section introduces how to integrate CPF-IM into video image superresolution. A new framework of iterative back projection algorithm is proposed to reconstruct the highresolution video image Video Superresolution Reconstruction Model. The acquisition of low-resolution images is shown in Figure 1. The relation between the kth observation image g k and the original high-resolution image f can be expressed as g k =V k U k W k fn k, (6) where W k represents the affine operation matrix for g k, U k denotes the blur operation caused by point spread function (PSF), and V k is the downsampling matrix. The point spread function (PSF) and the downsampling operator used to be the same in the process of image acquisition in an imaging system. Thus, the blur operator U k and the downsampling operator V k canbeassumedtobe f Affine transform Low-resolution video frame PSF blur Downsampling W k U k V k Figure 1: Low-resolution image acquisition model. g 1 g 2 g 3 g 4 g 5 g 6 Sliding window move forward f 1 f 2 n k g k High-resolution video frame Figure 2: Video superresolution reconstruction model. constants in a sequence of images acquisition. So the imaging model (6) canbetransformedto g k =UVW k fn k. (7) The purpose of superresolution is to obtain the optimal estimation of the high-resolution image f from a series of low-resolution images g k. This is the inverse process of imaging as shown in (7). The key to superresolution reconstruction algorithms is accurate image registration, which is a difficult problem, since motion between video frames is complex and accurate motion field is difficult to obtain. Most traditional superresolution reconstruction algorithms are proposed for image superresolution rather than video superresolution, as motion field between images is much easier to estimate. The algorithm proposed in this paper is suitable for video image superresolution, and the experimental results will verify its effectiveness. For a video sequence, the motion vectors between two adjacent frames are relatively small. So we select the successive frames as the low-resolution images. A high-resolution frame can be reconstructed from the adjacent frames through (7). A video superresolution reconstruction model based on sliding window [19] is depicted in Figure 2. As shown in Figure 2, in video superresolution reconstruction, the sliding window is moved forward to produce successive high-resolution frames in the output sequence. In order to ensure the accuracy of registration, the middle frame of the sliding window is selected as the reference frame. Image matching is performed between the reference frame and other frames in the sliding window to obtain the motion fields. Then the high-resolution frames are reconstructed basedonthesemotionfields.ifweassumethatthetotal length of video is N and P is the length of sliding window, (NP 1) high-resolution frames can be reconstructed from the low-resolution video sequence Original Iterative Back Projection Algorithm. Iterative back projection [1, 6, 7] starts with an initial estimation of high-resolution image. The initial high-resolution image is generated by interpolating an input low-resolution image.

4 4 Advances in Multimedia Image acquisition model Observation low- resolution image Blur Affine transform g il g n Downsampling il The previous image fh n Upsampling Deblur Reverse affine transform G i P G= G i i=1 Back projection The iterative image fh n1 Figure 3: Flow of the original IBP algorithm. Then, a series of simulated low-resolution images are generated from the reconstructed superresolution image by the image acquisition model. If the simulated low-resolution images are consistent with the corresponding input lowresolution images, the reconstructed superresolution image is identical with the original high-resolution image. Otherwise, the differences between the simulated low-resolution images and the input low-resolution images will be projected to the reconstructed superresolution image. This process is repeated iteratively until the differences between the simulated lowresolution images and the input low-resolution images are small enough. Iterative back projection algorithm can be formulated as f n1 H i=1 =fn H λ P H BP i (g n il g il), g n il =VUW kf n H, (8) where n represents the iteration number, f n1 H is the reconstructed superresolution image in the (n 1)th iteration, and f n H is the reconstructed superresolution image in the nth iteration. λ is a constant value controlling the convergence speed, which is set as 0.1 for every iterative process. The length of sliding window is defined as P, which also represents the number of input low-resolution images. H BP i is the ith back projection operation, which is the same projection operator. g n il denotes the simulated low-resolution image defined by the image acquisition model, and g il denotes the input lowresolution image. W k represents the affine operation matrix for g n il, U denotes the blur operation caused by point spread function (PSF), and V is the downsampling matrix. The flow of the original iterative back projection algorithm is shown in Figure 3, and deblur block denotes the inverse matrix of U Proposed Iterative Back Projection Algorithm Combined with CPF Based Image Matching. In traditional image superresolution, global motion estimation is usually adopted as image matching algorithm, and global motion parameters are obtained between low-resolution images. In this case, the motion parameters between high-resolution images can be directly obtained by transforming the motion parameters between low-resolution images. And thus the affine transformation is easily implemented at high resolution. In this paper, pixel-level motion fields are obtained by CPF based image matching. And the affine transformation should be performed pixel by pixel, which is difficult to implement. Moreover, the affine transformation between the input lowresolution images is already included in the process of CPF matching. Therefore, the affine transformation in the image acquisitionmodelcanbereplacedbythatincpfmatching, and a new framework of iterative back projection algorithm combined with CPF-IM (IBP-CPF-IM) is proposed. The proposed IBP-CPF-IM algorithm starts with taking an input low-resolution image as the reference image. Then, CPF-IM is performed between the reference image and the other nonreference input low-resolution images to obtain pixel-level motion fields. And a series of simulated reference images x il are produced from these nonreference low-resolution images using the motion fields. Each input low-resolution image corresponds to a simulated reference image. Next, an initial reconstructed superresolution image f 0 H is generated by interpolating the reference low-resolution image. Generally, better initial estimation directly leads to better quality of the final reconstructed image. In addition, high quality of the initial estimation image can speed up the convergence of the iteration algorithm. Finally, simulated low-resolution image x n il is generated through downsampling the reconstructed superresolution image. If the simulated low-resolution image x n il is consistent with the simulated reference images x il, the reconstructed superresolution image is identical with the original high-resolution image. Otherwise, the differences between the simulated low-resolution image and the simulated reference images will be projected to the reconstructed superresolution image. This process is repeated iteratively until the differences between the simulated lowresolution image and the simulated reference images are

5 Advances in Multimedia 5 Simulated low-resolution image acquisition Simulated reference frames x il x n il Downsampling Blur The previous image yh n CPF-IM Upsampling Deblur G i P G= G i i=1 Back projection Observation lowresolution image g il The iterative image yh n1 Figure 4: Flow of the proposed IBP with CPF-IM. small enough. The proposed IBP-CPF-IM algorithm can be formulated as f n1 H i=1 =fn H λ P x il = CPF (g il ), x n il =VUfn H, H BP i (x n il x il), where x il represents the simulated reference image, x n il is the simulated low-resolution image through downsampling the superresolution image, and CPF( ) represents the operation which transforms input low-resolution image to simulated reference image. In the original IBP algorithm, affine transformation is implemented between high-resolution images using global motion parameters. However, the affine transformation in the proposed algorithm is replaced by CPF matching between low-resolution images. And the convergence of the proposed algorithm is going to speed up. The relative error used to control when to stop the iterative process is defined as follows: fn1 H error = fn H, (10) fn H where represents L 2 norm. Iteration stops when error is smallerthanathreshold(setas10 4 in our experiments) or the iteration number reaches the maximum value which is set as 100. The processes of IBP-CPF-IM algorithm for video superresolution are presented as follows. Step 1. Movetheslidingwindowandtakefivevideoframes as the input low-resolution frames. In general, when the distance between the middle frame and the matching frame becomes larger, their differences also become larger. In this (9) case, the accuracy of CPF matching will decrease, and subtle quality improvement of the generated frame can be obtained. Nevertheless, the computational complexity will increase greatly. To get a tradeoff between computational complexity and image quality, the length of sliding window P is set as five empirically. Step 2. The middle frame of the sliding window is taken as the reference frame. CPF based image matching is conducted between the reference frame and the other frames in the sliding window. Four motion fields are obtained through this process. And four simulated reference frames are generated using the motion fields. Step 3. The reference frame is interpolated to obtain the initial reconstructed superresolution frame. Step 4. The simulated low-resolution frame is obtained through downsampling the reconstructed superresolution frame. Step 5. Ifthesimulatedlow-resolutionframeisnotthesame as the simulated reference frame, differences between them are projected to the reconstructed superresolution frame and we return to Step 4. If the differences between the simulated low-resolution frame and the simulated reference frame are small enough, the reconstructed superresolution frame is the desired frame. The flow of proposed iterative back projection algorithm is shown in Figure 4, and deblur block denotes the inverse matrix of U. 4. Experiment Results 4.1. Experiments on Static Image. To evaluate the performance of the proposed algorithm, standard images are tested. The size of the input high-resolution image is

6 6 Advances in Multimedia (a) Original image (b) Bilinear interpolated image (c) Bicubic interpolated image (d) IBP-FR image (e) IBP-OL2004 (f) IBP-OL2010 (g) IBP-CPF-IM Figure 5: Comparison of Peppers. The low-resolution images are obtained by the image acquisition model as shown in Figure 1. The high-resolution image is first performed by affine transform which contains various unique vertical shifts, horizontal shifts, and rotations. Then, these affined images are blurred through low pass filter and downsampled by a factor of 2 to generate low-resolution images. The number of input simulated low-resolution images is 4. In this part, the performance of the proposed algorithm is compared with bilinear interpolation, bicubic interpolation, IBP with frequency domain registration (IBP- FR) [20], and IBP with optical flow based image matching (IBP-OL) algorithms (optical flow based image matching algorithms used in the experiments are proposed in [14]and [15], resp.). CPF based image matching is applied to obtain motion fields between images. A stable result about each test image in the experiments is obtained when iterative number is around 15. Figures 5(a) and 6(a) show the original high-resolution images of Peppers and Lena. One of the generated lowresolution images is selected as the reference image about each test image. Figures 5(b) and 6(b) are the interpolated images from the reference image using bilinear algorithm, while Figures 5(c) and 6(c) are the interpolated images using bicubic algorithm. Superresolution images reconstructed using iterative back projection algorithm based on frequency domain registration [20]are shown in Figures 5(d) and 6(d). Figures 5(e), 6(e), 5(f), and6(f) are images reconstructed using iterative back projection algorithm with optical flow based image matching [14, 15]. IBP-OL2004 represents optical flow based image matching algorithm proposed in [14].

7 Advances in Multimedia 7 (a) Original image (b) Bilinear interpolated image (c) Bicubic interpolated image (d) IBP-FR image (e) IBP-OL 2004 (f) IBP-OL2010 (g) CPF-IM Figure 6: Comparison of Lena. IBP-OL2004 denotes the used optical flow based image matching algorithm proposed in [15]. And the images reconstructed by the proposed algorithm are shown in Figures 5(g) and 6(g). Ringing artifacts of Figures 5(g) and 6(g) are effectively reduced compared with Figures 5(d) and 6(d). Moreover, the edges of Figures 5(g) and 6(g) are handled better than those in Figures 5(e), 5(f), 6(e),and6(f). In order to evaluate the quality of the reconstructed imageobjectively,psnriscalculatedbetweentheoriginal high-resolution image and the reconstructed superresolution image. Table 1 gives the performance of the proposed algorithm and the compared algorithms. It is apparent that the proposed algorithm achieves an improvement of objective quality Experiments on Video. To verify the effect of the proposed algorithm on video image superresolution reconstruction, a series of standard video sequences are tested. The tested videos are in format of YUV. Proposed algorithm is applied to each component of video. In order to calculate PSNR performance of reconstructed video, low-resolution videos are obtained through downsampling high-resolution videos. And the PSNR performance of the experiment results is calculated based on Y component. For each sequence, only

8 8 Advances in Multimedia Table 1: PSNR of experimental image shown in Figures 3 and 4 and the other experimental images. Bilinear Bicubic IBP-FR IBP-OL2004 IBP-OL2010 IBP-CPF-IM Barbara db db db db db db Lena db db db db db db Peppers db db db db db db (a) (b) (c) (d) (e) (f) (g) (h) (i) (j) (k) (l) Figure 7: Comparison of hall: (a) original image, (b) the enlarged portion of (a), (c) bilinear interpolated image, (d) the enlarged portion of (c), (e) bicubic interpolated image, (f) the enlarged portion of (e), (g) IBP-OL2004, (h) the enlarged portion of (g), (i) IBP-OL2010, (j) the enlarged portion of (i), (k) IBP-CPF-IM, and (l) the enlarged portion of (k). thefirst100framesareusedandthesizeofthevideoimageis In this experiment, the scheme of sliding window (Figure 2) is adopted. The pixel-level motion fields between the reference frame and the other frames are estimated by CPF-IM. Taking the scheme of sliding window into consideration, the length of the output reconstructed frames is 96. The proposed algorithm is compared with bilinear interpolated algorithm, bicubic interpolated algorithm, and IBP with optical flow based image matching algorithm (optical flow based image matching algorithms used in the experiment are proposed in[14] and[15], resp.). Thus the motion mode which frequency domain registration algorithm requests is only global translation or rotation. iterative back projection algorithm based on frequency domain registration is not suitable for video superresolution. For bilinear and bicubic interpolation, the output reconstructed frame is acquired by interpolating the reference frame. For each frame in the video, a stable result is obtained before iterative number reaches the maximumvalue.thepsnrperformancesofthesealgorithms are listed in Table 2. Compared to bicubic interpolation, IBP- CPF-IM achieves a linear average PSNR gain of 1.27 db. These demonstrate that the amount of reconstruction information which we get from more than one frame is much more than that from only one frame, and the objective quality of the reconstructed frame by superresolution algorithm is generally higher than the reconstructed frame by interpolation. When compared with the IBP-OL2004 and IBP- OL2010 algorithms, the PSNR gain of IBP-CPF-IM is about 0.52 db and 0.60 db. The reason is that optical flow is sensitive to brightness change, which frequently occurs on video sequence. And the motion vectors cannot be accurately obtained by using optical flow based image matching. Figures7 and 8 show examples of the experimental results about hall and container sequences. The original images are

9 Advances in Multimedia 9 Table 2: PSNR of experimental video frames shown in Figures 5 and 6 and the other experimental video frames. Bilinear Bicubic IBP-OL2004 IBP-OL2010 IBP-CPF-IM Coastguard db db db db db Container db db db db db Hall db db db db db Foreman db db db db db Mother-daughter db db db db db Tempete db db db db db Average db db db db db (a) (b) (c) (d) (e) (f) (g) (h) (i) (j) (k) (l) Figure 8: Comparison of container: (a) original image, (b) the enlarged portion of (a), (c) bilinear interpolated image, (d) the enlarged portion of (c), (e) bicubic interpolated image, (f) the enlarged portion of (e), (g) IBP-OL2004, (h) the enlarged portion of (g), (i) IBP-OL2010, (j) the enlarged portion of (i), (k) IBP-CPF-IM, and (l) the enlarged portion of (k). shown in (a). The reconstructed images by bilinear, bicubic IBP-OL2004, IBP-OL2010, and the proposed IBP-CPF-IM are shown in (c), (e), (g), (i), and (k), respectively. In order to show details of the reconstructed images, a portion of these reconstructedimagesisenlargedwithascalefactorof1:4, asshownin(d),(f),(h),(j),and(l),respectively.compared to bicubic interpolation, the enlarged portion of Figure 7(l) reconstructed by IBP-CPF-IM is clearer. When there is a large motion or the change is abrupt between consecutive frames, the ghosting effect is produced by IBP-OL as shown in Figure 7(h), while IBP-CPF-IM can reduce this defect, as shown in Figure 7(j). 5. Conclusion The performance of superresolution reconstruction algorithm depends largely on the accuracy of image registration. Traditional registration approaches for superresolution reconstruction algorithm usually demand that the motion between images is global translation or rotation. For video frames, this does not hold. Moreover, block based matching and optical flow based matching algorithms have their own limitations. To improve the accuracy of image registration, this paper introduces multiresolutional critical-point filters based image matching algorithm (CPF-IM) to image/video

10 10 Advances in Multimedia superresolution. The elastic motion model in CPF-IM can adapt to various nontranslational motions, such as zoom and rotation. The combination of IBP and CPF-IM, which is called IBP-CPF-IM, is applied to image and video superresolution. Experimental results show that the proposed approach achieves better performance in objective and subjective qualityforbothimageandvideoframes,whencompared with bilinear and bicubic interpolation algorithms and IBP- FR and IBP-OL algorithm. CPF-IM can also be integrated into other superresolution reconstruction algorithms, such as projections onto convex sets (POCS) algorithm and maximum a posteriori (MAP) algorithm. Conflict of Interests There is no conflict of interests regarding the paper. Acknowledgments This project is supported by the National Natural Science Foundation of China (no ), China Postdoctoral Science Foundation (no. 2012M511433), and China Postdoctoral International Exchange Programme. References [1]B.Wan,L.Meng,D.Ming,H.Qi,Y.Hu,andK.D.K.Luk, Video image super-resolution restoration based on iterative back-projection algorithm, in Proceedings of the IEEE International Conference on Computational Intelligence for Measurement Systems and Applications (CIMSA 09), pp.46 49,May [2] B. Narayanan, R. C. Hardie, K. E. Barner, and M. Shao, A computationally efficient super-resolution algorithm for video processing using partition filters, IEEE Transactions on Circuits and Systems for Video Technology, vol.17,no.5,pp , [3] J.Yu,C.Xiao,andK.Su, Amethodofgibbsartifactreduction for POCS super-resolution image reconstruction, in Proceedings of the 8th International Conference on Signal Processing (ICSP 06), vol. 2, IEEE, Beijing, China, November [4] A.J.Patti,M.I.Sezan,andA.M.Tekalp, Superresolutionvideo reconstruction with arbitrary sampling lattices and nonzero aperture time, IEEE Transactions on Image Processing, vol. 6, no. 8, pp , [5]R.Y.TsaiandT.S.Huang, Multiframeimagerestoration and registration, in Advances in Computer Vision and Image Processing, T.S.Huang,Ed.,vol.1,pp ,JAIPress, Greenwich, Conn, USA, [6] H.Song,X.He,W.Chen,andY.Sun, Animprovediterative back-projection algorithm for video super-resolution reconstruction, in Proceedings of the Symposium on Photonics and Optoelectronic (SOPO '10), pp. 1 4, Chengdu, China, June [7] M. Irani and S. Peleg, Improving resolution by image registration, CVGIP: Graphical Models and Image Processing, vol.53, no. 3, pp , [8] H. Xi, C. Xiao, and C. Bian, Edge halo reduction for projections onto convex sets super resolution image reconstruction, in Proceedings of the 14th International Conference on Digital Image Computing Techniques and Applications (DICTA 12), pp.1 7, December [9] J. Chen, J. Nunez-Yanez, and A. Achim, Video super-resolution using generalized Gaussian Markov random fields, IEEE Signal Processing Letters,vol.19,no.2,pp.63 66,2012. [10] H. S. Hou and H. C. Andrews, Cubic splines for image interpolation and digital filtering, IEEE Trans Acoust Speech Signal Process,vol.26,no.6,pp ,1978. [11] W. T. Freeman, T. R. Jones, and E. C. Pasztor, Example-based super-resolution, IEEE Computer Graphics and Applications, vol.22,no.2,pp.56 65,2002. [12] C. Kim, K. Choi, and J. B. Ra, Example-based super-resolution via structure analysis of patches, IEEE Signal Processing Letters, vol. 20, no. 4, pp , [13] F. Dufaux and F. Moscheni, Motion estimation techniques for digital TV: a review and a new contribution, Proceedings of the IEEE,vol.83,no.6,pp ,1995. [14] A. Mitiche and A.-R. Mansouri, On convergence of the Horn and Schunck optical-flow estimation method, IEEE Transactions on Image Processing,vol.13,no.6,pp ,2004. [15] D. Sun, S. Roth, and M. J. Black, Secrets of optical flow estimation and their principles, in Proceedings of the IEEE International Conference on Computer Vision and Pattern Recognition (CVPR 10), pp ,SanFrancisco,Calif,USA,June [16] Y. Shinagawa and T. L. Kunii, Unconstrained automatic image matching using multiresolutional critical-point filters, IEEE Transactions on Pattern Analysis and Machine Intelligence, vol. 20, no. 9, pp , [17] K. Habuka and Y. Shinagawa, Image interpolation using enhanced multiresolution critical-point filters, International Journal of Computer Vision,vol.58,no.1,pp.19 35,2004. [18] Y. Zhang, W. Wang, and L. Zheng, Image interpolation using multiresolutional critical point filters with unconstrained boundary extension, in Proceedings of the 3rd International Congress on Image and Signal Processing (CISP 10), pp , October [19] F.-Q. Qin, X.-H. He, W.-L. Chen, X.-M. Yang, and W. Wu, Video superresolution reconstruction based on subpixel registration and iterative back projection, Journal of Electronic Imaging,vol.18,no.1,ArticleID013007,2009. [20] P. Vandewalle, S. Süsstrunk, and M. Vetterll, A frequency domain approach to registration of aliased images with application to super-resolution, EURASIP Journal on Advances in Signal Processing,vol.2006,ArticleID071459,2006.

11 Rotating Machinery Engineering Journal of The Scientific World Journal Distributed Sensor Networks Journal of Sensors Journal of Control Science and Engineering Advances in Civil Engineering Submit your manuscripts at Journal of Journal of Electrical and Computer Engineering Robotics VLSI Design Advances in OptoElectronics Navigation and Observation Chemical Engineering Active and Passive Electronic Components Antennas and Propagation Aerospace Engineering Modelling & Simulation in Engineering Shock and Vibration Advances in Acoustics and Vibration