Optical Flow for Large Motion Using Gradient Technique

Transcription

1 SERBIAN JOURNAL OF ELECTRICAL ENGINEERING Vol. 3, No. 1, June 2006, Optical Flow fo Lage Motion Using Gadient Technique Md. Moshaof Hossain Sake 1, Kamal Bechkoum 2, K.K. Islam 1 Abstact: In this pape, we pesent the gadient-based optical flow method that estimates the two-dimensional velocity of object motion. a multi-esolution smoothing opeation poposes in this pape as a pe-pocessing step fo ovecoming the difficulty of lage motion estimation by gadient-based optical flow techniques. the effectiveness of the poposed method has confimed by applying image sequence of lage motion expeimental esults with an image sequence show a qualitative impovement. Keywods: Motion estimation, lage motion, optic flow. 1 Intoduction The analysis of image sequence is one of the most actively studied aeas of compute vision and image pocessing. Motion estimation is a key poblem in the analysis of image sequence. Usually, this is accomplished by detemining the motion by which an object moves fom one fame to the next. The motion field is the 2D vecto field, which is pespective pojection on the image plane of the 3D velocity field in the scene. Fom the infomation available fom an image sequence (e.g.; the spatial and tempoal vaiation of the bightness patten), it is only possible to deive an estimate of the motion field, which is called optical flow o image flow [1,2,3,4,6]. In ecent eseach, much attention has been paid to moe pecise estimation techniques of optical flow, which ae computationally expensive. Most of these techniques use moe than two fames to estimate the flow and cannot achieve eal-time pefomance. Anothe disadvantage of the published methods is that they ae wok well fo small optical flow. Howeve, the computational efficiency and the popety to estimate lage displacement accuately ae impotant peconditions fo optical flow techniques in lage motion, navigation and so foth. Optical flow methods [1,9] have the advantages that they can estimate velocity field with details local vaiations using two o thee fames. Howeve, 1 Depatment of EEE, Islamic Univesity of Technology, Gazipu-1704, Bangladesh; MHSake@iut-dhaka.edu 2 School of Computing & Infomation Technology, Univesity of Wolvehampton, WV1 1EQ, UK; K.Bechkoum@wlv.ac.uk 103

2 M.H. Sake, K. Bechkoum, K.K. Islam they behave pooly in the estimation of lage motion due to Taylo's appoximation of the gadient constaint equation that is validity only in the small vicinity of motion. Fo a bette undestanding of the dependencies between image, flow vectos, flow egions ae shown in Fig. 1. image flow vectos, which fom flow fields flow egions with constant optical flow Fig. 1 Relationship between image, flow egions, flow vectos and field. Definition 1: Lage Motion in image sequences involves the subtaction of successive fames, and/o it affects to enlage the flow egion size whee the pesentation of pixels can be changed. Logical aspects on behalf of lage motion can be pesented as : estimating lage camea motions fo omni-diectional vision including combinations of lage displacements, estimating the lage displacement on the pediction of global flow field paametes and so foth [4,5,8]. The est of the pape is oganized as follows: Fistly, we give a bief intoduction of motion estimation of objects. Secondly, we discuss the teatment of lage motion estimation, multi-esolution smoothing opeation technique and the eo estimation due to motion of the object. Thidly, we analyze the poposed method by using some image sequences and we show some expeimental esults. Finally, we conclude the pape. 2 Related Woks A numbe of eseaches [4-9] have efeed to lage motion. Thee ae seveal eseach pojects in the field of motion of object in image sequences. In this pape, we pesent a gadient-based technique that pefoms the motion estimation of an object with lage motion in the image sequence, usually human motion in monocula image sequences. We intoduce a pe-pocessing multi-esolution smoothing opeation in which image velocity is educed due to a decease in image aea and impove the oveall flow estimation. We estimate 104

3 Optical Flow fo Lage Motion Using Gadient Technique the eo fo evaluating the esults of oiginal image sequence and smoothed sequence. 3 Lage Object-Motion Estimation In this section, we biefly discuss the spatio-tempoal deivative technique fo motion estimation and dawback of gadient-based technique. Conside p( xt, ) is the usual, continuous, space time bightness function of a point x( xy, ) in the image plane at time t. If the bightness emains pactically constant along a motion tajectoy, we have: dp( x, t)/ dt = 0, whee x vaies by t accoding to motion tajectoy. This is the total deivative expession and denotes the ate of change of intensity along the motion tajectoy. Based on constant bightness assumption, the bightness of the object point in the images sampled at times t and t +δ t is the same. That means, p( xt, ) = px ( +δ xt, +δt). Expanding by Taylo's seies and omitting highe ode tems, we have the following gadient constaint equation o Optical Flow Constaint (OFC) equation [2, 3]: pxt (, ) pxt (, ) pxt (, ) vx( x, t) + vy( x, t) + vz( x, t) = 0, (1) x y z whee vx ( x, t) = dx/ dt and vy ( x, t) = dy/ dt denotes the component of image velocity vecto in tem of the continuous image coodinates, and the patial spatial deivatives of the image bightness ae the components of the spatial gadient p. Equation (1) can be witten as the image bightness constancy equation: ( p) T v+ p t = 0, (2) whee p t denotes patial diffeentiation with espect to time. Equation (2) povides one linea equation fo two unknown components of velocity vecto; hence, futhe constaints ae necessay to solve the value v. Global o local optimization techniques can be used to ovecome this poblem. In global optimization [2], the velocity field is detemined by using an eo function based on the gadient constaint and a global smoothing tem. On the othe hand, least squae solution of the gadient constaint equation uses the local optimization [3,5,7] without assigning any smoothness The simplest constaint is to assume that the motion is the same on a small spatial neighbohood: the optical flow can estimate within local egion as vecto, v minimizes the paamete Ψ defines as follows: 105

4 M.H. Sake, K. Bechkoum, K.K. Islam Ψ= p v + p dxdt, (3) ( xt, ) Γ (( ) T ) 2 t whee, Γ is the local egion o neighbohood of the obsevation point. The velocity is estimated by minimizing the paamete Ψ. Diffeentiating with espect to v and, and then setting it to zeo, we have the following set of equations: x whee v y p ij is given by vp x xx + vp y xy + ptx = 0, (4) vp x xy + vp y yy + pty = 0, (5) p p ( x, t) p ( x, t) dxdt, = ij i j (,: i j x,) t Fom the equation (4) and (5), the component of image velocity ( vx x, t ) and (, ) vy x t at any position ( x, y) in Γ is detemined. The velocity obtained using above equations is: p p v ( x, t) = x p p v ( x, t) = y p p yt xy xt yy 2 pxx pyy pxy p p xt xy yt xx 2 pxx pyy pxy The magnitude and phase angle coespond in Γ is v φ= actan y. vx, (6). 7) 2 2 V( x, t) = vx + vy In the above motion estimation, the OFC equation deives by taking the Taylo expansion of the image constancy equation dp( x, t) / dt = 0 up to fist ode tems. The highe-ode tems neglect unde the assumption that the time step between consecutive fames is abitay small, which is most often violated in pactice. In eality, time step is usually fixed. Theefoe, the lack of highe ode tems becomes main souces of eo in the data constaint. The eliability of the image fom constaint equation depends on the magnitude of the highe ode deivatives of the image bightness function, p( xt, ). If the image bightness function in the neighbohood of a point is well appoximated by a linea function, i.e. its highe ode deivatives ae small, then the flow constaint is said to be vey eliable at this point. On the contay, the image flow constaint is vey uneliable at the locations with significant highe-ode deivatives, which is mainly in the neighbohood of bightness discontinuities. 106 and

5 Optical Flow fo Lage Motion Using Gadient Technique 4 Desciption of the System Thee ae seveal possibilities to estimate lage motions of single flow egion with sufficient accuacy. The possible ways ae as follows: (i) Inceasing the size of flow egions, (ii) Pedicting the global flow field paametes, (iii) Computing the flow at multiple esolutions. 4.1 Inceasing Flow Region Size By inceasing the size of flow egions, lage motion can estimate. Unfotunately, the egion cannot be enlaged too much, because in gadientbased motion estimation, we assume that optical flow is constant fo a small egion o vey small aeas. 4.2 Pediction of Global Flow Field Paametes Fo a given flow field it is possible to estimate paametes descibing the movement of the camea (ego-motion). It allows the motion estimation by pedicting of next quaduple of paametes [5]. 4.3 Multi-esolution Smoothing The optical flow can estimate at diffeent levels of an image hieachy, stating at top level. Fo each level of hieachy, image ows and columns ae down sampled by a facto of two and the measuable motions double. The multi-esolution smoothing opeation with a window size w= m n is shown in Fig. 2 in which the image aea and the velocity ae educed by the 2 factos of m and m espectively. Due to educed image aea, the motion educes in smoothing image. Theefoe, we get smoothing estimated motion. Fig. 2 Multi-esolution smoothing opeation with a window size of w = 2pixels 2pixels Theefoe, the multi esolution smoothing opeation has some advantages stated below: (i) Noise educes due to aveaging filte effect, (ii) Image aea educes as well as motion doubles, so, computation complexity deceases, (iii) Lage motion educes due to multi-esolution smoothing, theefoe lage motion estimation is possible by smoothing. 107

6 M.H. Sake, K. Bechkoum, K.K. Islam 4.4 Eo Estimation It is mentioned that in the neighbohood, the numbe of gadient constaint equations is lage than the unknown vaiables. Theefoe, an ove detemined system of equations is fomed in the neighbohood. If an ove detemined system of equations is used to estimate motion, the measuement eo in the gadients and in compatibilities among the constaint equations due to diffeential motion will be eflected in the esidual of the solution [3, 8]. The esidual vecto can be estimated by: = ( p) T v+ pt. (8) Now, in ode to estimate the eo ( ε ) at each pixel, we have nomalized the esidual by the spatial gadient of bightness at each pixel. The eo specifies the degee at which each pixel satisfies the image bightness constancy equation and it is given by the following equation: T ( p) v + pt ε=. (9) T ( p) When the constaint equation is completely satisfied ove a local egion, then ε becomes zeo. The eo will be inceases if ε incease. 4.5 Compute envionment We use WINDOWS envionment in the compute system fo expeimental implementation of the human motion detection whee AVS / EXPRESS and MATLAB ae used fo this wok. 5 Expeiments and Discussions To estimate the lage motion of object, fist multi-esolution smoothing opeation ove the image sequences is used. The gadient-based local optimization method is applied as discussed in the pevious section with Γ= 4pixels 4pixels 2fames. The gadients, p x, p y, and p t ae detemined consideing a fames spatio-tempoal neighbouhood as depicts by [2,9]. 108

7 Optical Flow fo Lage Motion Using Gadient Technique Fig. 3 Oiginal image fames and smoothed image fame in the hall monito sequence. We have applied ou algoithm to hall monito image sequences and mob image sequences with fast walking. These image sequences contain the moving objects as humans with lage motion and complex backgound shown in Figue 3 and figue 4. The hall monito image sequence size is 352pixels 240 pixels with 300 fames, and the mob image sequence size is 480pixels 640 pixels with 60 fames. In the pe-pocessing steps of multi-esolution smoothing, the pixels within the window aveages. The educed image fame of the fist sequence is shown along with Fig. 3 in the ight side. Fig. 4 Oiginal image fame and smoothed image fame in the mob sequence of fast walking [top] oiginal image, motion image, and eo image, [bottom] smoothed image, motion image and eo image. Fig. 5 Eo images of the oiginal and smoothed image fame 34 and 4 of the image sequence. 109

8 M.H. Sake, K. Bechkoum, K.K. Islam Fig. 4 shows the second fame of mob sequence image with fast walking. The top thee images show the oiginal image, motion image and eo image espectively. In the same fashion, the bottom thee images ae smoothed image, coesponding motion image and eo image afte multi-esolution and applying optical flow. The motion and as well as eo is calculated by using 2 pixels 2 pixels in consecutive 2 fames without any ovelapping in spatial dimension of the image. Fig. 6 Motion in a typical position of the oiginal an smoothed image sequence of hall monito sequence. Fig. 7 Estimated eo in the image sequence befoe and afte smoothing opeation. 110

9 Optical Flow fo Lage Motion Using Gadient Technique Fig. 8 Nomalized motion of each fame in the hall monito image sequence [50 fames ae shown]. Fig. 9 Nomalized eo of each fame in the hall monito image sequence. Fom Fig. 4, it is clea that the eo deceases due to multi esolution smoothing. Fig. 5 shows the eo image caused by motion in 34 and 45 fames of hall monito sequence. We estimated the optical flow on the intensity and calculated the eo in each pixel. We have shown the motion as well as the eo in a typical position of the image sequence ove seveal fames in Fig. 6 and Fig. 7. In figues oiginal and smooth epesent the data fo oiginal image sequence and sequence afte multi-esolution smoothing teatment espectively. Fom figues, it is found that estimation of velocity impoves afte multi esolution smoothing opeation. The eo is educed and the poblem of gadient-based method is hence impoved. 111

10 M.H. Sake, K. Bechkoum, K.K. Islam Fig. 10 Aveage motion of each fame in the mob image sequence [40 fames ae shown]. Fig. 11 Aveage eo of each fame in the mob image sequence of fast walking. Fig. 8 and 9 show the nomalized motion and nomalized eo ove seveal image fames in the hall monito sequence. We have shown a limited numbe of fames in the figues. The eo ove the whole image sequence is educed in the smoothed sequence of the image. In the same way, the velocity as well as the eo in the fast walking of the mob sequence is shown in Figs. 10 and 11. The velocity educes in Fig. 10 is due to educed size of image aea fo the smooth image which ovecomes the difficulty of lage motion estimation in gadientbased technique. The eo educes significantly with the smoothing opeation. 6 Conclusion In this pape, we have shown that gadient-based optical flow estimates can be impoved by including the multi-esolution smoothing opeation pe- 112

11 Optical Flow fo Lage Motion Using Gadient Technique pocessing technique. This gadient technique ovecomes the difficulty of estimation lage motion. The eo estimation evaluates the pefomance of the technique and show the impovement. The algoithm is easy, noise is educed, and the oveall computational cost is inexpensive. 7 Refeences [1] J.L. Baon, D.J. Fleet, S.S. Beauchemin: Pefomance of optical flow technique, Intenational Jounal of Compute Vision, Vol. 12, No. 1, pp.43-77, [2] B.K.P. Hon, B.G. Schunck: Detemining Optical Flow, Atificial Intelligence 17, pp , [3] J.K. Keaney, W. B. Thompson, D.L. Boley: Optical flow estimation: An Eo Analysis of Gadient-Based Methods with Local Optimization, IEEE Tansactions on Patten Analysis and Machine Intelligence, Vol. 9, No , Mach [4] J.W. Lee, S. You, U. Neumann: Lage motion estimation fo omnidiectional vision, Poceedings of the IEEE wokshop on omnidiectional vision, pp , June [5] T. Radtke, H. Salzwedel: Estimation of optical flow fo lage displacements, Poceedings of the Intenational Confeence on Signal and Image Pocessing, Kawaii, USA, pp , 13 August [6] M. Geoge, T. Tjahjadi: Multi-esolution optical flow estimation using adaptive shifting, Poceedings of 1999 Intenational Confeence on Image Pocessing, Vol. 3, pp , Oct [7] B-G Kim, J-I Shim, D-J Pak: Fast image segmentation based on multi-esolution analysis and wavelets, Patten Recognition Lettes 24, pp , [8] A.M. Wink, J.B.T.M. Roedink: Denoising functional MR images: a compaison of wavelet denoising and Gaussian smoothing, IEEE Tansactions on Medical Imaging, Vol. 23, Issue 3, pp , Mach [9] Q. Wang; Q. Wang; D.D. Feng, R. Zhao, Z. Chi: A fast 2D entopic thesholding method by wavelet decomposition, Intenational Confeence on Image Pocessing. 2002, Vol. 3, pp. III- 265-III-268, June