Second Order Variational Optic Flow Estimation

Transcription

1 Second Order Variational Optic Flow Estimation L. Alvarez, C.A. Castaño, M. García, K. Krissian, L. Mazorra, A. Salgado, and J. Sánchez Departamento de Informática y Sistemas, Universidad de Las Palmas de Gran Canaria, Camps de Tafira s/n, Las Palmas de Gran Canaria, Spain Tel.: , Fax: {lalvarez,ccastano,mgarcia,krissian,lmazorra,asalgado, jsanchez}@dis.lpgc.es Abstract. In this paper we present a variational approach to accrately estimate the motion vector field in a image seqence introdcing a second order Taylor expansion of the flow in the energy fnction to be minimized. This featre allows s to simltaneosly obtain, in addition, an estimation of the partial derivatives of the motion vector field. The performance of or approach is illstrated with the estimation of the displacement vector field on the well known Yosemite seqence and compared to other techniqes from the state of the art. 1 Introdction Optic flow estimation is a problem that has focsed the attention of many researchers in the domain of image processing, probably de to the hge amont of applications in which the estimated displacement between consective frames is an important sorce of information. In this sense, many different approaches have been presented in the literatre, starting from classical methods like the ones proposed by Horn and Schnck [1] or by Lcas and Kanade [2], trying either to overcome some limitations of the existing methods or to exploit some apriori information in order to improve the accracy on the estimation. For instance, different reglarization terms have been proposed in variational approaches like in [3,4], pyramidal decompositions have also been proposed in order to detect large displacements [5,6] or spatio-temporal reglarization constraints have also been taken into accont [7,8]. In this paper, we propose a variational formlation to accrately estimate the motion vector field in a image seqence introdcing a second order Taylor expansion of the flow in the energy fnction to be minimized. This featre allows s to simltaneosly obtain, in addition, an estimation of the partial derivatives of the motion vector field. This idea is qite interesting in the field of flid dynamics, since partial derivatives is an important sorce of information in this field as they appear in the Navier-Stokes eqations and other intersting parameter sch as divergence, vorticity, strain rate tensor and dissipation rate. Moreover, since a reglarity constraint is also imposed on the second order flow parameters, the estimated flow will better preserve the continity behavior assmed in flid dynamics. R. Moreno-Díaz et al. (Eds.): EUROCAST 2007, LNCS 4739, pp , c Springer-Verlag Berlin Heidelberg 2007

2 Second Order Variational Optic Flow Estimation 647 The paper is organized as follows. In section 2, we describe the details of or variational approach and the way we have adapted the energy fnction to be minimized in order to directly estimate significative flid parameters sch as vorticity and strain rate tensor components. In section 3, we present nmerical experiments on synthetic and real data and we analyze the behavior of the proposed method in comparison with other standard techniqes described in the literatre. Finally, in section 4, we present the main conclsions of the paper. 2 A Variational Approach for Second Order Motion Estimation To estimate the optic flow of a given seqence we propose a variational approach based on the minimization of an energy fnction E(ũ) defined for each point x 0 with the special fact that it depends not only on the displacement vector components =(, v) T, bt also on their partial derivatives ũ =(, v, x, y,v x,v y ) T, as shown in the following eqation: E(ũ) =E(, v, x, y,v x,v y )= ( ( ) 2 Ω(x K 0) σ(x x 0 ) I 1 (x) I 2 (x +(, v) T x + y (x x v x v 0 ))) dx (1) y where I 1 (x) is the first image of the seqence and I 2 (x) is the following frame, where the time increment from frame to frame is assmed to be normalized (Δt =1),K σ (x x 0 ) is a Gassian kernel with standard deviation σ which weighs the pixels in the domain Ω(x 0 ) centered at point x 0. Hence, the goal is to find the components of vector ũ, that is, the displacement and its partial derivatives sch that minimize the error between I 1 (x) andi 2 (x) displaced by the nknown flow. At a first glance, the dependence on the partial derivatives ( x, y,v x,v y ) might seem a redndant operation since they can be compted from the obtained motion vector field. However, it can be shown that nmerical differentiation of the estimated motion vector field yields to inaccrate reslts mainly de to ndesired nmerical error amplification [9]. 2.1 Energy Minimization In order to be able to obtain the six components vector of nknowns ũ = (, v, x, y,v x,v y ) T that minimize Eq. 1 we formlate the soltion at step n +1 as a fnction of the soltion at step n and the six components vector of residals h =(h,h v,h x,h y,h vx,h vy ) T compted at each step, as shows the following expression: ũ n+1 = ũ n + h. (2) Then, introdcing Eq. 2 in Eq. 1 we are able to obtain an iterative operator towards the local minimm of the energy fnction similar to a gradient descent

3 648 L. Alvarez et al. algorithm [10]. Bt first, the following approximations are still necessary to simplify the term that depends on I 2 (x) in order to obtain or iterative operator: ( ) I 2 (x +( n+1,v n+1 ) T n+1 x + n+1 y vx n+1 vy n+1 (x x 0 )) = ( ) ( ) I 2 (x +( n,v n ) T n + x n y vx n vy n (x x 0 )+(h,h v ) T hx h + y (x x h vx h 0 )). vy To simplify notation, let s define I2 n as: ( )( ) I2 n (x) =I 2(x +( n,v n ) T n + x n y x x0 vx n ) (3) vn y y y 0 where the vector x x 0 has been decomposed into their components (x x 0,y y 0 ) T. The partial derivatives are denoted as: ( )( ) I2,x n (x) = I 2 x (x +(n,v n ) T n + x n y x x0 vx n ). (4) vn y y y 0 ( )( ) I2,y n (x) = I 2 y (x +(n,v n ) T n + x n y x x0 vx n ). (5) vn y y y 0 Then, sing the new notation, the soltion at step n + 1 can be approximated by its Taylor expansion over ũ n to provide: ( ( )( )) T ( ) I2 n+1 (x) I2 n (x)+ (h,h v ) T hx h + y x x0 I n 2,x (x) h vx h vy y y 0 I2,y(x) n. (6) Finally, by sing vectorial notation I 2 (x) n+1 can be expressed as: where I n+1 2 (x) I n 2 (x)+în 2 (x)t h (7) Î n 2(x) =(I2,x n (x),in 2,y (x), (x x 0)I2,x n (x), (y y 0)I2,x n (x), (x x 0 )I2,y n (x), (y y 0)I2,y n (x))t. (8) Hence, introdcing Eq. 7 in Eq. 1 we obtain the approximation for the energy fnction shown in Eq. 9, where, in addition, we introdce a reglarization term in the energy fnction weighted by the parameter α. The role of this reglarization term is to provide at every point a smooth vector field, by means of an additional constraint on the norm of the vector h, which is forced to be as small as possible. In this sense, the parameter α determines the importance of this additional constraint on the vector field. ( Ẽ( h)= K σ (x x 0 ) I 1 (x) I2 n (x) Î n 2(x) T h ) 2 dx+α h 2 dx. (9)

4 Second Order Variational Optic Flow Estimation 649 This formlation allows s to easily obtain an analytical expression to compte the local minimm of the energy as a fnction of h: Ẽ( h) = 2 ( h) Ω(x K 0) σ(x x 0 ) I 1 (x) I2 n(x) În 2 (x)t În 2 (x)dx + which is eqivalent to: 2α hdx = 0 (10) K σ (x x 0 )(I 1 (x) I n 2 (x)) În 2 (x)dx = ( K σ (x x 0 )(Î n 2(x) Î n 2(x) T )+αi) hdx. (11) This system of eqations can be expressed with the standard matrix notation Ax = b, where the vector of nknowns in this case is the vector h, while the system matrix and the independent term can be compted as: ( ) A = K σ (x x 0 )(Î n 2(x) Î n 2(x) T )+αi dx. (12) b = K σ (x x 0 )(I 1 (x) I2 n (x)) În 2 (x)dx. (13) The soltion of the system of eqations is given by h = A 1 b,soweonlyhave to invert the 6 6matrixA or se any other algorithm to solve the system of eqations. We observe that if α>0, matrix A is positive definite and therefore the system of eqations is well-posed (it has a niqe soltion). It means that the parameter α avoids instabilities in the soltion of the linear system of eqations. 2.2 Nmerical Implementation Next, we describe the main steps we followed to derive an efficient algorithm of the method proposed in section 2, inclding the nmerical considerations involved on the comptation of integrals and spatial derivatives in the proposed method. As it can be seen in Algorithm 1, or variational approach starts from an initial estimation of the motion vector field 0 which can be obtained sing any other optic flow estimation method. Then, we start the iterative procedre towards the local minimm of the energy fnction given in Eq. 1. At each iteration we check the convergence of or algorithm in order to discard the soltions that do not provide the optimal response. This algorithm is then applied for every point in the image (or a grid at a given scale) to obtain the desired first and second order flow parameters estimation, sing information within a neighborhood of that point determined by the σ, parameter of the Gassian kernel. The initialization of the energy at the first step reqires the comptation of the image derivatives. In or implementation, we have sed central finite differences becase it is a good compromise between an easy implementation, low time of comptation and low error propagation [9].

5 650 L. Alvarez et al. Algorithm 1. Implementation of the second order flow estimation method Initialization of the parameters: σ (window size), α, ũ 0 = 0, E(ũ 0 ), N iter, Failres=0,MaxFailres Initial Estimation of the motion field 0 =(, v) with any estimation flow techniqe for n =0toN iter do Update I n 2 (x) andî n 2 sing ũ n Comptation of the energy E(ũ n ), A and b Solve system of eqations h = A 1 b if E(ũ n ) E(ũ n 1 ) then ũ optimized = ũ n + h {The method converges towards the soltion} Update ũ n+1 Failres=0 else if E(ũ n ) E(ũ n 1 )andfailres MaxFailres then Update ũ n+1 Reject this soltion as ũ optimized {The method diverges from the soltion} Failres++ else Exit end if end for 3 Reslts In order to show the performance of the proposed approach, we present here the nmerical experiments we have performed to evalate the accracy of the method, working on both synthetic and real data. It is interesting to remark that the initial estimation of the flow can be obtained by any desired method, which mst be, at least, a rogh approximation of the nderlying motion vector field so that the algorithm is able to converge. In or work, we se the method described in [11] to achieve sch initialization. 3.1 Experiment 1: Yosemite Seqence In or first experiment we se the well-known Yosemite seqence to evalate the performance of or approach. From a qalitative point of view, it is interesting to notice that the vector field obtained with or approach is sally smoother than the initial estimation. This is more clearly seen on Fig. 1, where on the left side we represent the anglar error between the grondtrth and the initial estimation, while on the right we show the anglar error for or response. As it can be seen, this error featre is lower and more reglar with or method. From a qantitative point of view, table 1 shows some statistics on the eclidean error (Eq. 14) and the anglar error (Eq. 15) for that seqence according to the ideas in [12]. The error was compted removing the clods in order to be able to provide a meaningfl error measre, following the recommendation in [13]. As it

6 Second Order Variational Optic Flow Estimation 651 Fig. 1. On the left, anglar error between the grondtrth and the initial estimation of the vector field. On the right, the vector field obtained with or method. Table 1. Qantitative error measres obtained for the initial estimation and for or method Anglar Error Std. Dev. Eclidean Error Std. Dev. Initialization Or Method can be seen in the referred table, the error rate is lower once we se or method to process the initial vector field. RMSV D = 1 N N i ref i. (14) i=1 ψe = 180 Nπ N arccos( i ref i ). (15) i=1 3.2 Experiment 2: Satellite Seqence Finally, we present the reslts we obtained with or approach sing real satellite image seqence provided by the Laboratoire de Meteorologie Dynamiqe (LMD) from Paris (France). This data tracks the hrricane Vince that became the first known tropical cyclone to reach the Iberian Peninsla between October 8th and 11th, The main interest in processing this kind of data is to estimate the clod motion between consective frames. In Fig. 2 we compare the vector field obtained with or approach (represented in yellow, light arrows) with that sed to initialize or method (represented in red, dark arrows). As it can be seen, the vector field obtained is more reglar than the initial estimated flow.

7 652 L. Alvarez et al. Fig. 2. Details of the motion vector field in the satellite seqence. In red (dark arrows), we represent the method sed to initialize or algorithm and in yellow (light arrows) we display the vector field obtained with or method. 4 Conclsions In this paper, we have presented a new variational method to estimate the motion vector field in a image seqence. In this sense, or variational method improves the qality of the motion vector field sed as initial estimation introdcing a reglarity constraint on the first and second order moments of the motion vector field to be estimated. Or qantitative reslts on synthetic data show the encoraging reslts we obtain sing or method as a post-processing step for the reglarization process. The performance of or method is also tested with real satellite image seqences, where the vector fields we obtain also present a reglar behavior. Acknowledgements This work has been fnded by the Eropean Commission nder the Specific Targeted Research Project FLUID (contract no. FP ). References 1. Horn, B., Schnck, B.: Determining optical flow. MIT Aritificial Intelligence Laboratory (1980) 2. Lcas, B., Kanade, T.: An iterative image registration techniqe with an application to stereo vision. In: Proc. Seventh International Joint Conference on Artificial Intelligence, Vancover (1981) Nagel, H.H., Enkelmann, W.: An investigation of smoothness constraints for the estimation of displacement vector fields from image seqences. IEEE Trans. Pat. Anal. andmach. Intell. 8 (1986) Abert, G., Deriche, D., Kornprobst, P.: Compting optical flow via variational techniqes. SIAM Jornal on Applied Mathematics 60(1) (1999) Mémin, E., Pérez, P.: Dense estimation and object-based segmentation of the optical flow with robst techniqes. IEEE Transactions on Image Processing 7(5) (1998)

8 Second Order Variational Optic Flow Estimation Alvarez, L., Weickert, J., Sánchez, J.: Reliable estimation of dense optical flow fields with large displacements. International Jornal of Compter Vision 39(1) (2000) Weickert, J., Schnorr, C.: A theoretical framework for convex reglarizers in pdebased comptation of image motion. Draft, Departament of Mathematics and Compter Science (2000) 8. Weickert, J., Papenberg, N., Brhn, A., Brox, T.: High accracy optical flow estimation based on a theory for warping. Volme 4. (2004) Raffel, M., Willert, C., Kompenhans, J.: Particle Image Velocimetry. A Practical Gide. Springer Verlag (1998) 10. Fletcher, R., Reeves, C.: Fnction minimization by conjgate gradients. Compter Jornal 7 (1964) Alvarez, L., Castaño, C., García, M., Krissian, K., Mazorra, L., Salgado, A., Sánchez, J.: Symmetric optical flow. In: Proceeding of EroCast, Las Palmas, Spain (2007) 12. Barron, J., Fleet, D., Beachemin, S.: Performance of optical flow techniqes. IJCV 12(1) (1994) Black, M.: (2006)