Variational Motion Compensated Deinterlacing
|
|
- Gabriella Greer
- 5 years ago
- Views:
Transcription
1 Variational Motion Compensated Deinterlacing Sune Keller, François Lauze and Mads Nielsen The IT University of Copenhagen The Image Group Rued Langgaardsvej 7, 2300 Copenhagen S, Denmark Abstract. In this paper we present a novel motion compensated deinterlacing scheme based on a joint variational framework of motion estimation and creation of new pixel values. We propose a specific energy formulation for that purpose, optimized by solving corresponding Euler- Lagrange equations. The difficult issue of calculating motion on interlaced image sequences is addressed and the presented algorithm proves to solve what we call The Interlacing Problem: depicting highly detailed regions in motion without artifacts to the human visual system (HVS). 1 Introduction and Background Since the birth of television in the 1930 s interlacing have been the dominant scanning format and even today it is used in HDTV in Japan, USA and Europe. In interlaced scan only half the horizontal lines of each frame are recorded and displayed as illustrated in Fig. 1 alternating between the even and odd numbered lines and the recorded partial frames are called even and odd fields. Most modern displays are progressive, i.e. display full frames and thus conversion from interlaced to progressive scan, deinterlacing, is needed to create every other line of the sequence (white lines in Fig. 1). Deinterlacing can be done very simple using methods like line averaging with reasonably acceptable output quality, but interlacing artifacts are generally not removed. Two types of interlacing artifacts, serration and line crawl, are shown in Fig. 1, while a third type, line flicker appears when a detail is only seen in one horizontal line and thus only displayed at half the intended screen refresh rate. For further details on interlacing artifacts see [1] and [6]. The output of motion adaptive deinterlacers like the ones described in [6] and state of the art chips sets such as the DCDi R by Faroudja has almost no interlacing artifacts left. Motion adaptive deinterlacers are however unable to solve what we like to call The Interlacing Problem: the problem of correctly handling vertical high frequency image content in motion, illustrated in Fig. 2. A way to handle the problem may be to smooth the region in order to avoid any interlacing artifacts, but it will also blur out visually important details. The spatially missing information is often present across several frames and if the motion is known, it could be propagated along the motion trajectories over time
2 Fig. 1. Left: The interlacing (width, height, time)-volume. Interlacing artifacts shown by a merge of a time t field with its neighboring time t+1 field: In the middle serration due to horizontal motion and to the right it is line crawl due to vertical motion Fig. 2. The Interlacing Problem. Still 1&2: Zoom in on two consecutive progressive frames of the sequence Truck of a truck driving towards the camera. Still 3&4: the same two frames interlaced: knowledge of the correct spatial structure is almost gone to new pixel positions, and this is the principle of motion compensated deinterlacing (MCDI). Fig. 2 clearly illustrates how difficult it would be to to recreate the correct structure using only spatial information. The video corresponding to these frames shows that the perforated structure of interlaced image volumes in combination with the highly detailed content, makes motion computation a non-trivial task. Motion compensated deinterlacing (MCDI) use temporally coherent information from a motion field / optical flow estimate (we will use the two terms interchangeably) as input to an interpolation of the missing lines with maximized output quality. It must also be robust to inaccuarete motion estimation and switch to pure spatial interpolation, for instance. This is not a simple task. For instance, Biswas and Nguyen propose in [2] a motion compensated algorithm which produce results that may still contain serration artifacts. Bellers and de Haan has done extensive work on developing and testing motion compensated deinterlacing and a selection of 11 algorithms are presented and tested in [1]. They show that the presented motion compensated deinterlacers are better than simple deinterlacers, although motion adaptive methods are left out of the test. A major problem with motion estimation in interlaced sequences is that standard motion estimation algorithms are developed for progressive sequences and thus not directly applicable.typically a simple deinterlacer is used to create an initial progressive sequence [1]. This influences the accuracy of the resulting obtained flow due to the interlacing artifacts introduced to the sequence. The
3 problem of obtaining an accurate flow will be most severe in the regions troubled by The Interlacing Problem and where an accurate flow is needed the most. Deinterlacing can be seen as a form of Image Sequence Inpainting and in this paper we propose a new motion compensated scheme based on the inpainting variational formulation of Lauze and Nielsen [8] which we briefly discuss in Section 2. Due to the very nature of interlaced sequences, a straightforward application of this algorithm is problematic. In Section 3 we propose instead a three steps algorithm for motion estimation and deinterlacing. We present our test results in Section 4 and conclude in Section 5. 2 Motion Compensated Inpainting In [8], Lauze and Nielsen proposed a generic Bayesian formulation for image sequence inpainting and motion recovery. The observed damaged image sequence is denoted u 0 and we define it to be on a spatiotemporal domain denoted with D being the locus of the missing data in, that is D. v is the motion field of the de-noised and restored image sequence u, the desired result of the inpainting process. They show that under mild assumptions, the conditional probability p(u, v u 0, D) can be factorized as p(u, v u 0, D) p(u 0 u, D) p(u s ) }{{}}{{} P 0 p(u t u s, v) }{{} P 1 P 2 p(v) }{{} P 3. (1) where u s and u t are the spatial and temporal distribution of intensities respectively. P 0 is the image sequence likelihood, P 1 the spatial prior on image sequences, P 3 the prior on motion fields and P 2 acts as both a likelihood for the motion field and a temporal prior on image sequences. The Maximum a posteriori (MAP) is then sought for in order to reconstruct the image sequence and recover the motion field. Using the Bayesian to Variational rationale of Mumford ([9]), the problem is transformed into this continuous minimization problem E(u, v) = E 0 (u, u 0 ) + E 1 (u s ) + E 2 (u s, u t, v) + E 3 (v). (2) Using Calculus of Variations, a minimizer for the above energy is characterized by the coupled system of equations, E(u, v)/ u = 0 and E(u, v)/ v = 0. Selecting the distributions to use for each of the E i -terms in (2) is a tradeoff between tractability and modelling accuracy. Discussions on the matter can be found in [7] and in [8], the above energy (2) is instantiated as E(u, v) = λ 0 (u u 0 ) 2 dx + λ 1 ψ( u 2 )dx + λ 2 ( u ψ( V ) 2 + γ u V \D 2)dx + λ 3 ( ψ( v1 2 ) + ψ( v 2 2 ) ) dx (3) where is the spatial gradient operator, λ i and γ are some constants and v 1 and v 2 are the x- and y-components of the flow field, i.e. v = (v 1, v 2 ) T and
4 V = (v T, 1) T. The two / V terms are the V -directional derivatives of u and u respectively 1. Instead of using the function which is non differentiable at the origin, we replace it by the approximation ψ(s 2 ) = s 2 + ε 2, where ε is a small positive constant. The spatial gradient term is thus a variation of the term introduced by Rudin et al. in [10], while the flow part is a slight rewriting of the energy proposed by Brox et al. in [3]. The resulting system of Euler-Lagrange equations are solved within a multiresolution framework, interleaving motion recovery and motion compensated equations, starting with a basic inpainting at coarsest scale, assuming null motion. For further details see [7]. By taking as missing data locus D the set of missing lines in an interlaced sequence, this would provide a deinterlacing algorithm. However, each hole in the sequence is only one line high and thus does not scale down and a multiresolution inpainting cannot be used. In the next section we propose an algorithm for deinterlacing based on inpainting methodology presented in this section. 3 Deinterlacing Algorithm As mentioned in section 1, some care must be taken in motion calculations on the interlaced input. We do it by computing two separate flows respectively for even and odd fields, then remoulding the flow to cover the full progressive output domain and finally computation of the intensities the actual deinterlacing. 3.1 Motion Estimation Steps The motion estimation part that we propose is a two steps process. In order to be as close as possible to the known data, we proceed the following way. We extract the odd field sequence, i.e. the sequence of the known lines of odd fields, and compress it vertically by removing the unknown lines. In the same way, we produce the even field sequence. 1. Compute the flow v o for the odd field sequence and the flow v e for the even field sequence. 2. Fusion v o and v e in order to produce a candidate flow field for the full progressive sequence. The flow recovery algorithm that we have used is the already mentioned one of [3]. Generally variational methods produce the best and most accurate flow field of any methods available today, as the survey in [4] shows. We have two forward flow fields, one for the odd original fields and one for the even original fields, both containing flows from fields at time t to time t + 2. To get flows for all frames in the output we have to fuse these two flows together: we start by a simple interleaving (and find initial values in the new lines by line averaging), we scale the resulting flow from the compressed fields used in the motion estimation and halve the motion vectors to be from time t to time t f(x)/ V = lim ɛ 0 (f(x + ɛv ) f(x)) /ɛ = f(x) V
5 and not to time t+2 (assuming linear flow from t to t+2). Finally we smooth the new flow field to get it consistent, correct errors due to the assumptions made in the two first steps and improve the estimates in the new lines. This all ends up being a fuse-scale-smoothing process as expressed by this energy minimization v = Arg min v (v v o/e ) 2 dx + λ ψ( v 2 )dx (4) \D where composing v o/e is the fusion-scaling part and the full minimization is the smoothing. As before ψ(s 2 ) = s 2 + ε 2. After deriving an Euler-Lagrange equation of (4), discretization is done using standard methods; The first term is treated similar to the E 0 in section 3.2 and the second term is the same as the smoothing term of the flow algorithm (see [3]) and thus treated similarly. 3.2 Motion Compensated Deinterlacing Step The actual deinterlacing is done by minimizing the following energy E(u) = λ 0 (u u 0 ) 2 dx + λ 1 ψ 1 ( u 2 )dx + λ 2 ψ 2 ( u v + u t 2 )dx \D λ 0 looks superfluous, but serves an important purpose; set to zero it makes the scheme an implementation of the energy without de-noising. most deinterlacers do not de-noise and leave all the original intensities of u 0 untouched. λ 0 allows to test our method in this setting as well. As argued in [1] de-noising can help remove some temporal flickering noise. We agree on this, but stress the need to carefully control the de-noising to avoid over-smoothing. The Euler-Lagrange equation corresponding to the energy formulation in (5) is E u = λ 0χ(u u 0 ) λ 1 div 2 ( ) u ψ 1 ( u 2 ) ( ψ λ 2 div 2 ( u v + u t 2 ) 3 ( u v + u t )V u v + u t ) (5) = 0 (6) where λ 0 is two times λ 0 in (5) and χ is the characteristic function, which takes on the value 1 in \D and 0 elsewhere. The discretization of the gradient descent equation corresponding to (6) is detailed in [7] and references therein. 4 Experiments It is often discussed whether to use objective or subjective evaluation of the performance of deinterlacers, e.g. in [1] and [6]. We have chosen subjective as the human visual system will be the final judge and also it seems to be the choice in industry, e.g. at Bang & Olufsen. We have used three experts and two laymen to judge the quality in an A/B comparisons with Faroudja s DCDi R
6 Table 1. Optimal settings for our three step motion compensated deinterlacing scheme fixed point inner convergence weights iterations iterations threshold Motion estimation (3) γ = 100, λ 3 = 70 Fuse-scale-smoothing (4) λ = 1 Deinterlacing (6) λ 0 = 1, λ 1 = 0.1 λ 2 = 1, 5, 10 (combined motion and spatial edge adaptive deinterlacing) in the DVP-1010 Video Processor, which is considered state of the art in the world of home cinema. In our algorithm there are 17 free parameters and after initial testing we chose a set of standard settings and tested at least two alternative values to each parameter besides the values tested initially. This of course far from covers all possibilities, but the amount of output data created in such a test and the time needed to evaluate it would require resources beyond our reach. The optimal settings found can be seen in Table 1. Two of the 17 parameters are not in the table: For the motion estimation step we use levels with a scale factor of 1.04 in the multiresolution pyramid on PAL resolution material ( ). For the weight λ 2 in the deinterlacing step three choices are given. Our tests showed λ 2 = 5 to be the best choice. Generally our three step algorithm has proven quite robust to changes in parameter settings. The flow calculations can give very detailed (or controlled poorly, over-segmented) flows, but they are some degree smoothed out in the fuse-scale-smoothing and thus working on improved motion estimation on interlaced material for deinterlacing will most likely prove worthwhile. On seven different test sequences with The Interlacing Problem details in motion our algorithm clearly outperformed the Faroudja DCDi R according to all five test persons. Each of the three test sequences having a progressive original was judged almost indistinguishable from their original. On an eighth sequence with very complex motion (hand held camera in a close-up of fingers typing on a keyboard) the motion was not recovered correctly and some serration were observed by one person in the test panel. In Fig. 3 some examples of the high quality outputs are shown. Due to copy protection we are unable to produce stills or record video of the output from the Faroudja processor. On the Copenhagen Pan Faroudja has some flicker on the buildings and also produces quite a lot of serration on the roof windows of the castle, where our deinterlacer produces a perfect result. On BBC3 our deinterlacer recovers the details and the motion is smooth, while Faroudja has staggering motion and the details are hard to see. On Truck, which was used to illustrate The Interlacing Problem in Fig. 2, we are able to (re)create the correct structure of the grill while Faroudja fails at this and creates some dark diagonal lines instead. The test sequence Credits (not shown) has rolling titles, a type of content on which most deinterlacers including the Faroudja in test perform horribly, but the output of our algorithm
7 Fig. 3. Three examples of the high quality output of our Motion Compensated Deinterlacer. Top: Copenhagen Pan ( the motion is a right-to-left pan), notice the details on the house in the bottom middle and the roof window of the castle. Left: Truck, compare to original in Fig. 2. Right: In this still from BBC3 (rotational motion on an axis in the lower left corner) a few artifacts are seen in the still but not during playback is indistinguishable from the progressive original. The general performance of the Faroudja is typical for motion adaptive deinterlacers: It is on the same level as the variational motion adaptive deinterlacer in [6], which we compared it to. In the test of motion compensated deinterlacers in [1] there is to few subjective results given in [1] to really compare to our scheme. Given the relatively small improvement in objectively evaluated performance compared to simple line averaging in [1], we believe to outperform all the schemes in [1] based on the subjective performance of our deinterlacer presented above. 5 Conclusion and Future Work As the previous section revealed we have a very good deinterlacer, but it is also very computationally heavy. We have shown that a technique so far used
8 for inpainting (see [8]) can be redeveloped to do deinterlacing and further on our Total Variation Motion Compensated Deinterlacing outperforms Faroudja s state of the art deinterlacer. We produce high quality outputs when the input flow is correct and precise, and generally solves The Interlacing Problem, although improvements can be done. In terms of running time, we are far from realtime, but to get closer to the desired realtime version we plan to replace the SOR solver with multigrid solvers shown to run realtime for variational flow computations in [5]. All though the multiresolution scheme used in [8] cannot be used for deinterlacing the idea to simultaneous update flow and intensities could be used to improve the flow obtained and the final output quality of our deinterlacer. Other options are further tweaking of the parameters of the existing algorithm, the use of other models than total variation. References 1. E.B. Bellers and G. de Haan. De-interlacing. A Key Technology for Scan Rate Conversion. Elsevier Sciences Publishers, Amsterdam, M. Biswas and T.Q Nguyen. A novel de-interlacing technique based on phase plane correlation motion estimation. International Symposium on Circuits and Systems, ISCAS, 2: , T. Brox, A. Bruhn, N. Papenberg, and J. Weickert. High accuracy optical flow estimation based on a theory for warping. In T. Pajdla and J. Matas, editors, Proceedings of the 8th European Conference on Computer Vision, volume 4, pages 25 36, Prague, Czech Republic, Springer Verlag. 4. A. Bruhn, J.Weickert, and C.Schnörr. Lucas/kanade meets horn/schunck: Combining local and global optic flow methods. International Journal of Computer Vision, 61(3): , A. Bruhn, J. Weickert, C. Feddern, T. Kohlberger, and C. Schnörr. Variational optic flow computation in real-time. IEEE Trans. Image Proc., 14(5): , S. Keller, F. Lauze, and M. Nielsen. A total variation motion adaptive deinterlacing scheme. In R. Kimmel, N. Sochen, and J. Weickert, editors, Procedings of the 5th Scale-Space Conf., volume 3459 of LNCS, pages , Berlin, Springer. 7. F. Lauze. Computational Methods For Motion Recovery, Motion Compensated Inpainting and Applications. PhD thesis, IT University of Copenhagen, F. Lauze and M. Nielsen. A variational algorithm for motion compensated inpainting. In S. Barman A. Hoppe and T. Ellis, editors, British Machine Vision Conference, volume 2, pages BMVA, D. Mumford. Bayesian rationale for the variational formulation. In Geometry- Driven Diffusion In Computer Vision, pages Kluwer Academic Publishers, Computational Imaging And Vision. 10. L. Rudin, S. Osher, and E. Fatemi. Nonlinear total variation based noise removal algorithms. Physica D, 60: , 1992.
A Total Variation Motion Adaptive Deinterlacing Scheme
A Total Variation Motion Adaptive Deinterlacing Scheme Sune Keller, François Lauze, and Mads Nielsen The IT University of Copenhagen Rued Langgaardsvej 7, 2300 Kbh. S Denmark {sunebio,francois,malte}@itu.dk
More informationMotion Compensated Video Super Resolution
Motion Compensated Video Super Resolution Sune Høgild Keller 1, François Lauze 2 and Mads Nielsen 1,2 1 The Image Group, Department of Computer Science, University of Copenhagen, Universitetsparken 1,
More informationA Variational Algorithm For Motion Compensated Inpainting
A Variational Algorithm For Motion Compensated Inpainting François Lauze, Mads Nielsen The IT University of Copenhagen Rued Langgaards Vej 7 300 Copenhagen S, enmark francois,malte@itu.dk Abstract A novel
More informationNotes 9: Optical Flow
Course 049064: Variational Methods in Image Processing Notes 9: Optical Flow Guy Gilboa 1 Basic Model 1.1 Background Optical flow is a fundamental problem in computer vision. The general goal is to find
More informationTechnion - Computer Science Department - Tehnical Report CIS
Over-Parameterized Variational Optical Flow Tal Nir Alfred M. Bruckstein Ron Kimmel {taln, freddy, ron}@cs.technion.ac.il Department of Computer Science Technion Israel Institute of Technology Technion
More informationOptical Flow Estimation with CUDA. Mikhail Smirnov
Optical Flow Estimation with CUDA Mikhail Smirnov msmirnov@nvidia.com Document Change History Version Date Responsible Reason for Change Mikhail Smirnov Initial release Abstract Optical flow is the apparent
More informationMarcel Worring Intelligent Sensory Information Systems
Marcel Worring worring@science.uva.nl Intelligent Sensory Information Systems University of Amsterdam Information and Communication Technology archives of documentaries, film, or training material, video
More informationSURVEY OF LOCAL AND GLOBAL OPTICAL FLOW WITH COARSE TO FINE METHOD
SURVEY OF LOCAL AND GLOBAL OPTICAL FLOW WITH COARSE TO FINE METHOD M.E-II, Department of Computer Engineering, PICT, Pune ABSTRACT: Optical flow as an image processing technique finds its applications
More informationSUPER-resolution (SR) is a thoroughly investigated subject
1 Video Super-Resolution using Simultaneous Motion and Intensity Calculations Sune Høgild Keller*, François Lauze and Mads Nielsen Abstract In this paper, we propose an energy-based algorithm for motion-compensated
More informationMotion Estimation. There are three main types (or applications) of motion estimation:
Members: D91922016 朱威達 R93922010 林聖凱 R93922044 謝俊瑋 Motion Estimation There are three main types (or applications) of motion estimation: Parametric motion (image alignment) The main idea of parametric motion
More informationIMA Preprint Series # 1979
INPAINTING THE COLORS By Guillermo Sapiro IMA Preprint Series # 1979 ( May 2004 ) INSTITUTE FOR MATHEMATICS AND ITS APPLICATIONS UNIVERSITY OF MINNESOTA 514 Vincent Hall 206 Church Street S.E. Minneapolis,
More informationImage denoising using TV-Stokes equation with an orientation-matching minimization
Image denoising using TV-Stokes equation with an orientation-matching minimization Xue-Cheng Tai 1,2, Sofia Borok 1, and Jooyoung Hahn 1 1 Division of Mathematical Sciences, School of Physical Mathematical
More informationAn Improved 3DRS Algorithm for Video De-interlacing
An Improved 3DRS Algorithm for Video De-interlacing Songnan Li 1, Jianguo Du 1, Debin Zhao 1,2, Qian Huang 2,3, Wen Gao 2,3 1 Department of Computer Science and Engineering, Harbin Institute of Technology,
More informationMatching. Compare region of image to region of image. Today, simplest kind of matching. Intensities similar.
Matching Compare region of image to region of image. We talked about this for stereo. Important for motion. Epipolar constraint unknown. But motion small. Recognition Find object in image. Recognize object.
More information3D Computer Vision. Dense 3D Reconstruction II. Prof. Didier Stricker. Christiano Gava
3D Computer Vision Dense 3D Reconstruction II Prof. Didier Stricker Christiano Gava Kaiserlautern University http://ags.cs.uni-kl.de/ DFKI Deutsches Forschungszentrum für Künstliche Intelligenz http://av.dfki.de
More informationMotion Estimation (II) Ce Liu Microsoft Research New England
Motion Estimation (II) Ce Liu celiu@microsoft.com Microsoft Research New England Last time Motion perception Motion representation Parametric motion: Lucas-Kanade T I x du dv = I x I T x I y I x T I y
More informationSuper-Resolution. Many slides from Miki Elad Technion Yosi Rubner RTC and more
Super-Resolution Many slides from Mii Elad Technion Yosi Rubner RTC and more 1 Example - Video 53 images, ratio 1:4 2 Example Surveillance 40 images ratio 1:4 3 Example Enhance Mosaics 4 5 Super-Resolution
More informationCS 565 Computer Vision. Nazar Khan PUCIT Lectures 15 and 16: Optic Flow
CS 565 Computer Vision Nazar Khan PUCIT Lectures 15 and 16: Optic Flow Introduction Basic Problem given: image sequence f(x, y, z), where (x, y) specifies the location and z denotes time wanted: displacement
More informationGlobal Minimization of the Active Contour Model with TV-Inpainting and Two-Phase Denoising
Global Minimization of the Active Contour Model with TV-Inpainting and Two-Phase Denoising Shingyu Leung and Stanley Osher Department of Mathematics, UCLA, Los Angeles, CA 90095, USA {syleung, sjo}@math.ucla.edu
More informationHigh Accuracy Optical Flow Estimation Based on a Theory for Warping
High Accuracy Optical Flow Estimation Based on a Theory for Warping Thomas Brox, Andrés Bruhn, Nils Papenberg, and Joachim Weickert Mathematical Image Analysis Group Faculty of Mathematics and Computer
More informationMotion. 1 Introduction. 2 Optical Flow. Sohaib A Khan. 2.1 Brightness Constancy Equation
Motion Sohaib A Khan 1 Introduction So far, we have dealing with single images of a static scene taken by a fixed camera. Here we will deal with sequence of images taken at different time intervals. Motion
More informationImage Smoothing and Segmentation by Graph Regularization
Image Smoothing and Segmentation by Graph Regularization Sébastien Bougleux 1 and Abderrahim Elmoataz 1 GREYC CNRS UMR 6072, Université de Caen Basse-Normandie ENSICAEN 6 BD du Maréchal Juin, 14050 Caen
More informationA Novel Image Super-resolution Reconstruction Algorithm based on Modified Sparse Representation
, pp.162-167 http://dx.doi.org/10.14257/astl.2016.138.33 A Novel Image Super-resolution Reconstruction Algorithm based on Modified Sparse Representation Liqiang Hu, Chaofeng He Shijiazhuang Tiedao University,
More informationGRID WARPING IN TOTAL VARIATION IMAGE ENHANCEMENT METHODS. Andrey Nasonov, and Andrey Krylov
GRID WARPING IN TOTAL VARIATION IMAGE ENHANCEMENT METHODS Andrey Nasonov, and Andrey Krylov Lomonosov Moscow State University, Moscow, Department of Computational Mathematics and Cybernetics, e-mail: nasonov@cs.msu.ru,
More informationImproving Motion Estimation Using Image-Driven Functions and Hybrid Scheme
Improving Motion Estimation Using Image-Driven Functions and Hybrid Scheme Duc Dung Nguyen and Jae Wook Jeon Department of Electrical and Computer Engineering, Sungkyunkwan University, Korea nddunga3@skku.edu,
More informationMultiple Motion and Occlusion Segmentation with a Multiphase Level Set Method
Multiple Motion and Occlusion Segmentation with a Multiphase Level Set Method Yonggang Shi, Janusz Konrad, W. Clem Karl Department of Electrical and Computer Engineering Boston University, Boston, MA 02215
More informationModule 7 VIDEO CODING AND MOTION ESTIMATION
Module 7 VIDEO CODING AND MOTION ESTIMATION Lesson 20 Basic Building Blocks & Temporal Redundancy Instructional Objectives At the end of this lesson, the students should be able to: 1. Name at least five
More informationAn Edge-Based Approach to Motion Detection*
An Edge-Based Approach to Motion Detection* Angel D. Sappa and Fadi Dornaika Computer Vison Center Edifici O Campus UAB 08193 Barcelona, Spain {sappa, dornaika}@cvc.uab.es Abstract. This paper presents
More informationRedundancy and Correlation: Temporal
Redundancy and Correlation: Temporal Mother and Daughter CIF 352 x 288 Frame 60 Frame 61 Time Copyright 2007 by Lina J. Karam 1 Motion Estimation and Compensation Video is a sequence of frames (images)
More informationPerformance Evaluation Metrics and Statistics for Positional Tracker Evaluation
Performance Evaluation Metrics and Statistics for Positional Tracker Evaluation Chris J. Needham and Roger D. Boyle School of Computing, The University of Leeds, Leeds, LS2 9JT, UK {chrisn,roger}@comp.leeds.ac.uk
More informationMoving Object Segmentation Method Based on Motion Information Classification by X-means and Spatial Region Segmentation
IJCSNS International Journal of Computer Science and Network Security, VOL.13 No.11, November 2013 1 Moving Object Segmentation Method Based on Motion Information Classification by X-means and Spatial
More informationAccurate 3D Face and Body Modeling from a Single Fixed Kinect
Accurate 3D Face and Body Modeling from a Single Fixed Kinect Ruizhe Wang*, Matthias Hernandez*, Jongmoo Choi, Gérard Medioni Computer Vision Lab, IRIS University of Southern California Abstract In this
More informationBCC Optical Stabilizer Filter
BCC Optical Stabilizer Filter The Optical Stabilizer filter allows you to stabilize shaky video footage. The Optical Stabilizer uses optical flow technology to analyze a specified region and then adjusts
More informationOptical Flow. Adriana Bocoi and Elena Pelican. 1. Introduction
Proceedings of the Fifth Workshop on Mathematical Modelling of Environmental and Life Sciences Problems Constanţa, Romania, September, 200, pp. 5 5 Optical Flow Adriana Bocoi and Elena Pelican This paper
More informationTotal Variation Denoising with Overlapping Group Sparsity
1 Total Variation Denoising with Overlapping Group Sparsity Ivan W. Selesnick and Po-Yu Chen Polytechnic Institute of New York University Brooklyn, New York selesi@poly.edu 2 Abstract This paper describes
More informationFactorization with Missing and Noisy Data
Factorization with Missing and Noisy Data Carme Julià, Angel Sappa, Felipe Lumbreras, Joan Serrat, and Antonio López Computer Vision Center and Computer Science Department, Universitat Autònoma de Barcelona,
More informationMotion Estimation with Adaptive Regularization and Neighborhood Dependent Constraint
0 Digital Image Computing: Techniques and Applications Motion Estimation with Adaptive Regularization and Neighborhood Dependent Constraint Muhammad Wasim Nawaz, Abdesselam Bouzerdoum, Son Lam Phung ICT
More informationAutomated Segmentation Using a Fast Implementation of the Chan-Vese Models
Automated Segmentation Using a Fast Implementation of the Chan-Vese Models Huan Xu, and Xiao-Feng Wang,,3 Intelligent Computation Lab, Hefei Institute of Intelligent Machines, Chinese Academy of Science,
More informationComparison of Vessel Segmentations Using STAPLE
Comparison of Vessel Segmentations Using STAPLE Julien Jomier, Vincent LeDigarcher, and Stephen R. Aylward Computer-Aided Diagnosis and Display Lab, The University of North Carolina at Chapel Hill, Department
More informationFast Image Registration via Joint Gradient Maximization: Application to Multi-Modal Data
MITSUBISHI ELECTRIC RESEARCH LABORATORIES http://www.merl.com Fast Image Registration via Joint Gradient Maximization: Application to Multi-Modal Data Xue Mei, Fatih Porikli TR-19 September Abstract We
More informationA new approach to reference point location in fingerprint recognition
A new approach to reference point location in fingerprint recognition Piotr Porwik a) and Lukasz Wieclaw b) Institute of Informatics, Silesian University 41 200 Sosnowiec ul. Bedzinska 39, Poland a) porwik@us.edu.pl
More informationCOMPUTER VISION > OPTICAL FLOW UTRECHT UNIVERSITY RONALD POPPE
COMPUTER VISION 2017-2018 > OPTICAL FLOW UTRECHT UNIVERSITY RONALD POPPE OUTLINE Optical flow Lucas-Kanade Horn-Schunck Applications of optical flow Optical flow tracking Histograms of oriented flow Assignment
More informationTracking-Learning-Detection
IEEE TRANSACTIONS ON PATTERN ANALYSIS AND MACHINE INTELLIGENCE, VOL. 6, NO., JANUARY 200 Tracking-Learning-Detection Zdenek Kalal, Krystian Mikolajczyk, and Jiri Matas, Abstract Long-term tracking is the
More informationCPSC 425: Computer Vision
CPSC 425: Computer Vision Image Credit: https://docs.adaptive-vision.com/4.7/studio/machine_vision_guide/templatematching.html Lecture 9: Template Matching (cont.) and Scaled Representations ( unless otherwise
More informationCHAPTER 3 DISPARITY AND DEPTH MAP COMPUTATION
CHAPTER 3 DISPARITY AND DEPTH MAP COMPUTATION In this chapter we will discuss the process of disparity computation. It plays an important role in our caricature system because all 3D coordinates of nodes
More informationOptical Flow Estimation
Optical Flow Estimation Goal: Introduction to image motion and 2D optical flow estimation. Motivation: Motion is a rich source of information about the world: segmentation surface structure from parallax
More informationDenoising an Image by Denoising its Components in a Moving Frame
Denoising an Image by Denoising its Components in a Moving Frame Gabriela Ghimpețeanu 1, Thomas Batard 1, Marcelo Bertalmío 1, and Stacey Levine 2 1 Universitat Pompeu Fabra, Spain 2 Duquesne University,
More informationFace Tracking. Synonyms. Definition. Main Body Text. Amit K. Roy-Chowdhury and Yilei Xu. Facial Motion Estimation
Face Tracking Amit K. Roy-Chowdhury and Yilei Xu Department of Electrical Engineering, University of California, Riverside, CA 92521, USA {amitrc,yxu}@ee.ucr.edu Synonyms Facial Motion Estimation Definition
More informationA Duality Based Algorithm for TV-L 1 -Optical-Flow Image Registration
A Duality Based Algorithm for TV-L 1 -Optical-Flow Image Registration Thomas Pock 1, Martin Urschler 1, Christopher Zach 2, Reinhard Beichel 3, and Horst Bischof 1 1 Institute for Computer Graphics & Vision,
More informationLocal Image Registration: An Adaptive Filtering Framework
Local Image Registration: An Adaptive Filtering Framework Gulcin Caner a,a.murattekalp a,b, Gaurav Sharma a and Wendi Heinzelman a a Electrical and Computer Engineering Dept.,University of Rochester, Rochester,
More informationCalibrating an Overhead Video Camera
Calibrating an Overhead Video Camera Raul Rojas Freie Universität Berlin, Takustraße 9, 495 Berlin, Germany http://www.fu-fighters.de Abstract. In this section we discuss how to calibrate an overhead video
More informationIMA Preprint Series # 1979
INPAINTING THE COLORS By Guillermo Sapiro IMA Preprint Series # 1979 ( May 2004 ) INSTITUTE FOR MATHEMATICS AND ITS APPLICATIONS UNIVERSITY OF MINNESOTA 514 Vincent Hall 206 Church Street S.E. Minneapolis,
More informationRange Imaging Through Triangulation. Range Imaging Through Triangulation. Range Imaging Through Triangulation. Range Imaging Through Triangulation
Obviously, this is a very slow process and not suitable for dynamic scenes. To speed things up, we can use a laser that projects a vertical line of light onto the scene. This laser rotates around its vertical
More informationMotion Estimation using Block Overlap Minimization
Motion Estimation using Block Overlap Minimization Michael Santoro, Ghassan AlRegib, Yucel Altunbasak School of Electrical and Computer Engineering, Georgia Institute of Technology Atlanta, GA 30332 USA
More informationPart II: Modeling Aspects
Yosemite test sequence Illumination changes Motion discontinuities Variational Optical Flow Estimation Part II: Modeling Aspects Discontinuity Di ti it preserving i smoothness th tterms Robust data terms
More informationComparison of Vessel Segmentations using STAPLE
Comparison of Vessel Segmentations using STAPLE Julien Jomier, Vincent LeDigarcher, and Stephen R. Aylward Computer-Aided Diagnosis and Display Lab The University of North Carolina at Chapel Hill, Department
More informationCoarse-to-fine image registration
Today we will look at a few important topics in scale space in computer vision, in particular, coarseto-fine approaches, and the SIFT feature descriptor. I will present only the main ideas here to give
More informationIsophote-Based Interpolation
Isophote-Based Interpolation Bryan S. Morse and Duane Schwartzwald Department of Computer Science, Brigham Young University 3361 TMCB, Provo, UT 84602 {morse,duane}@cs.byu.edu Abstract Standard methods
More informationVideo Alignment. Literature Survey. Spring 2005 Prof. Brian Evans Multidimensional Digital Signal Processing Project The University of Texas at Austin
Literature Survey Spring 2005 Prof. Brian Evans Multidimensional Digital Signal Processing Project The University of Texas at Austin Omer Shakil Abstract This literature survey compares various methods
More informationCorrespondence and Stereopsis. Original notes by W. Correa. Figures from [Forsyth & Ponce] and [Trucco & Verri]
Correspondence and Stereopsis Original notes by W. Correa. Figures from [Forsyth & Ponce] and [Trucco & Verri] Introduction Disparity: Informally: difference between two pictures Allows us to gain a strong
More informationBayesian Methods in Vision: MAP Estimation, MRFs, Optimization
Bayesian Methods in Vision: MAP Estimation, MRFs, Optimization CS 650: Computer Vision Bryan S. Morse Optimization Approaches to Vision / Image Processing Recurring theme: Cast vision problem as an optimization
More informationDetecting Printed and Handwritten Partial Copies of Line Drawings Embedded in Complex Backgrounds
9 1th International Conference on Document Analysis and Recognition Detecting Printed and Handwritten Partial Copies of Line Drawings Embedded in Complex Backgrounds Weihan Sun, Koichi Kise Graduate School
More informationImage Inpainting by Hyperbolic Selection of Pixels for Two Dimensional Bicubic Interpolations
Image Inpainting by Hyperbolic Selection of Pixels for Two Dimensional Bicubic Interpolations Mehran Motmaen motmaen73@gmail.com Majid Mohrekesh mmohrekesh@yahoo.com Mojtaba Akbari mojtaba.akbari@ec.iut.ac.ir
More informationVariational Optic Flow Computation: From Continuous Models to Algorithms
Variational Optic Flow Computation: From Continuous Models to Algorithms Joachim Weickert, Andrés Bruhn, Nils Papenberg, and Thomas Brox Mathematical Image Analysis Group Faculty of Mathematics and Computer
More informationNonlinear Multiresolution Image Blending
Nonlinear Multiresolution Image Blending Mark Grundland, Rahul Vohra, Gareth P. Williams and Neil A. Dodgson Computer Laboratory, University of Cambridge, United Kingdom October, 26 Abstract. We study
More informationMotion and Tracking. Andrea Torsello DAIS Università Ca Foscari via Torino 155, Mestre (VE)
Motion and Tracking Andrea Torsello DAIS Università Ca Foscari via Torino 155, 30172 Mestre (VE) Motion Segmentation Segment the video into multiple coherently moving objects Motion and Perceptual Organization
More informationCOMPARATIVE STUDY OF DIFFERENT APPROACHES FOR EFFICIENT RECTIFICATION UNDER GENERAL MOTION
COMPARATIVE STUDY OF DIFFERENT APPROACHES FOR EFFICIENT RECTIFICATION UNDER GENERAL MOTION Mr.V.SRINIVASA RAO 1 Prof.A.SATYA KALYAN 2 DEPARTMENT OF COMPUTER SCIENCE AND ENGINEERING PRASAD V POTLURI SIDDHARTHA
More informationProceedings of the 6th Int. Conf. on Computer Analysis of Images and Patterns. Direct Obstacle Detection and Motion. from Spatio-Temporal Derivatives
Proceedings of the 6th Int. Conf. on Computer Analysis of Images and Patterns CAIP'95, pp. 874-879, Prague, Czech Republic, Sep 1995 Direct Obstacle Detection and Motion from Spatio-Temporal Derivatives
More informationAn Approach for Reduction of Rain Streaks from a Single Image
An Approach for Reduction of Rain Streaks from a Single Image Vijayakumar Majjagi 1, Netravati U M 2 1 4 th Semester, M. Tech, Digital Electronics, Department of Electronics and Communication G M Institute
More informationA Bottom Up Algebraic Approach to Motion Segmentation
A Bottom Up Algebraic Approach to Motion Segmentation Dheeraj Singaraju and RenéVidal Center for Imaging Science, Johns Hopkins University, 301 Clark Hall, 3400 N. Charles St., Baltimore, MD, 21218, USA
More informationProduction of Video Images by Computer Controlled Cameras and Its Application to TV Conference System
Proc. of IEEE Conference on Computer Vision and Pattern Recognition, vol.2, II-131 II-137, Dec. 2001. Production of Video Images by Computer Controlled Cameras and Its Application to TV Conference System
More informationEE795: Computer Vision and Intelligent Systems
EE795: Computer Vision and Intelligent Systems Spring 2012 TTh 17:30-18:45 FDH 204 Lecture 14 130307 http://www.ee.unlv.edu/~b1morris/ecg795/ 2 Outline Review Stereo Dense Motion Estimation Translational
More informationImage Restoration under Significant Additive Noise
1 Image Restoration under Significant Additive Noise Wenyi Zhao 1 and Art Pope Sarnoff Corporation 01 Washington Road, Princeton, NJ 08540, USA email: { wzhao, apope }@ieee.org Tel: 408-53-178 Abstract
More informationComputer Vision I - Filtering and Feature detection
Computer Vision I - Filtering and Feature detection Carsten Rother 30/10/2015 Computer Vision I: Basics of Image Processing Roadmap: Basics of Digital Image Processing Computer Vision I: Basics of Image
More informationCHAPTER 9 INPAINTING USING SPARSE REPRESENTATION AND INVERSE DCT
CHAPTER 9 INPAINTING USING SPARSE REPRESENTATION AND INVERSE DCT 9.1 Introduction In the previous chapters the inpainting was considered as an iterative algorithm. PDE based method uses iterations to converge
More informationMultiple Combined Constraints for Optical Flow Estimation
Multiple Combined Constraints for Optical Flow Estimation Ahmed Fahad and Tim Morris School of Computer Science, The University of Manchester Manchester, M3 9PL, UK ahmed.fahad@postgrad.manchester.ac.uk,
More informationLecture 2 September 3
EE 381V: Large Scale Optimization Fall 2012 Lecture 2 September 3 Lecturer: Caramanis & Sanghavi Scribe: Hongbo Si, Qiaoyang Ye 2.1 Overview of the last Lecture The focus of the last lecture was to give
More informationFast and Effective Interpolation Using Median Filter
Fast and Effective Interpolation Using Median Filter Jian Zhang 1, *, Siwei Ma 2, Yongbing Zhang 1, and Debin Zhao 1 1 Department of Computer Science, Harbin Institute of Technology, Harbin 150001, P.R.
More informationOver-Parameterized Variational Optical Flow
DOI 10.1007/s11263-007-0051-2 Over-Parameterized Variational Optical Flow Tal Nir Alfred M. Bruckstein Ron Kimmel Received: 4 August 2006 / Accepted: 5 March 2007 Springer Science+Business Media, LLC 2007
More informationNon-Rigid Image Registration III
Non-Rigid Image Registration III CS6240 Multimedia Analysis Leow Wee Kheng Department of Computer Science School of Computing National University of Singapore Leow Wee Kheng (CS6240) Non-Rigid Image Registration
More informationReview and Implementation of DWT based Scalable Video Coding with Scalable Motion Coding.
Project Title: Review and Implementation of DWT based Scalable Video Coding with Scalable Motion Coding. Midterm Report CS 584 Multimedia Communications Submitted by: Syed Jawwad Bukhari 2004-03-0028 About
More informationMulti-View Image Coding in 3-D Space Based on 3-D Reconstruction
Multi-View Image Coding in 3-D Space Based on 3-D Reconstruction Yongying Gao and Hayder Radha Department of Electrical and Computer Engineering, Michigan State University, East Lansing, MI 48823 email:
More informationFeature Tracking and Optical Flow
Feature Tracking and Optical Flow Prof. D. Stricker Doz. G. Bleser Many slides adapted from James Hays, Derek Hoeim, Lana Lazebnik, Silvio Saverse, who 1 in turn adapted slides from Steve Seitz, Rick Szeliski,
More informationChapter 3 Image Registration. Chapter 3 Image Registration
Chapter 3 Image Registration Distributed Algorithms for Introduction (1) Definition: Image Registration Input: 2 images of the same scene but taken from different perspectives Goal: Identify transformation
More informationBiomedical Image Analysis. Point, Edge and Line Detection
Biomedical Image Analysis Point, Edge and Line Detection Contents: Point and line detection Advanced edge detection: Canny Local/regional edge processing Global processing: Hough transform BMIA 15 V. Roth
More informationOcclusion-Based Accurate Silhouettes from Video Streams
Occlusion-Based Accurate Silhouettes from Video Streams Pedro M.Q. Aguiar, António R. Miranda, and Nuno de Castro Institute for Systems and Robotics / Instituto Superior Técnico Lisboa, Portugal aguiar@isr.ist.utl.pt
More informationStep-by-Step Model Buidling
Step-by-Step Model Buidling Review Feature selection Feature selection Feature correspondence Camera Calibration Euclidean Reconstruction Landing Augmented Reality Vision Based Control Sparse Structure
More informationPublic Library, Stereoscopic Looking Room, Chicago, by Phillips, 1923
Public Library, Stereoscopic Looking Room, Chicago, by Phillips, 1923 Teesta suspension bridge-darjeeling, India Mark Twain at Pool Table", no date, UCR Museum of Photography Woman getting eye exam during
More informationMotion Estimation for Video Coding Standards
Motion Estimation for Video Coding Standards Prof. Ja-Ling Wu Department of Computer Science and Information Engineering National Taiwan University Introduction of Motion Estimation The goal of video compression
More informationKeeping features in the camera s field of view: a visual servoing strategy
Keeping features in the camera s field of view: a visual servoing strategy G. Chesi, K. Hashimoto,D.Prattichizzo,A.Vicino Department of Information Engineering, University of Siena Via Roma 6, 3 Siena,
More informationImage Inpainting Using Sparsity of the Transform Domain
Image Inpainting Using Sparsity of the Transform Domain H. Hosseini*, N.B. Marvasti, Student Member, IEEE, F. Marvasti, Senior Member, IEEE Advanced Communication Research Institute (ACRI) Department of
More informationLocally Weighted Least Squares Regression for Image Denoising, Reconstruction and Up-sampling
Locally Weighted Least Squares Regression for Image Denoising, Reconstruction and Up-sampling Moritz Baecher May 15, 29 1 Introduction Edge-preserving smoothing and super-resolution are classic and important
More informationMOTION ESTIMATION WITH THE REDUNDANT WAVELET TRANSFORM.*
MOTION ESTIMATION WITH THE REDUNDANT WAVELET TRANSFORM.* R. DeVore A. Petukhov R. Sharpley Department of Mathematics University of South Carolina Columbia, SC 29208 Abstract We present a fast method for
More informationFundamentals of Stereo Vision Michael Bleyer LVA Stereo Vision
Fundamentals of Stereo Vision Michael Bleyer LVA Stereo Vision What Happened Last Time? Human 3D perception (3D cinema) Computational stereo Intuitive explanation of what is meant by disparity Stereo matching
More informationAn edge-preserving multilevel method for deblurring, denoising, and segmentation
An edge-preserving multilevel method for deblurring, denoising, and segmentation S. Morigi 1, L. Reichel 2, and F. Sgallari 1 1 Dept. of Mathematics-CIRAM, University of Bologna, Bologna, Italy {morigi,sgallari}@dm.unibo.it
More informationAutomatic Logo Detection and Removal
Automatic Logo Detection and Removal Miriam Cha, Pooya Khorrami and Matthew Wagner Electrical and Computer Engineering Carnegie Mellon University Pittsburgh, PA 15213 {mcha,pkhorrami,mwagner}@ece.cmu.edu
More informationComparison between Motion Analysis and Stereo
MOTION ESTIMATION The slides are from several sources through James Hays (Brown); Silvio Savarese (U. of Michigan); Octavia Camps (Northeastern); including their own slides. Comparison between Motion Analysis
More informationFast 3D Mean Shift Filter for CT Images
Fast 3D Mean Shift Filter for CT Images Gustavo Fernández Domínguez, Horst Bischof, and Reinhard Beichel Institute for Computer Graphics and Vision, Graz University of Technology Inffeldgasse 16/2, A-8010,
More informationImage Segmentation for Image Object Extraction
Image Segmentation for Image Object Extraction Rohit Kamble, Keshav Kaul # Computer Department, Vishwakarma Institute of Information Technology, Pune kamble.rohit@hotmail.com, kaul.keshav@gmail.com ABSTRACT
More informationLevel lines based disocclusion
Level lines based disocclusion Simon Masnou Jean-Michel Morel CEREMADE CMLA Université Paris-IX Dauphine Ecole Normale Supérieure de Cachan 75775 Paris Cedex 16, France 94235 Cachan Cedex, France Abstract
More information