NON-RIGID OBJECT TRACKING: A PREDICTIVE VECTORIAL MODEL APPROACH

Size: px

Start display at page:

Download "NON-RIGID OBJECT TRACKING: A PREDICTIVE VECTORIAL MODEL APPROACH"

Shanon Burns
5 years ago
Views:

1 NON-RIGID OBJECT TRACKING: A PREDICTIVE VECTORIAL MODEL APPROACH V. Atienza; J.M. Valiente and G. Andreu Departaento de Ingeniería de Sisteas, Coputadores y Autoática Universidad Politécnica de Valencia. Caino de Vera, Valencia. Spain. Fax: (34) E-ail: vatienza@disca.upv.es Abstract This paper deals with the developent of coputer vision techniques for tracking the position of rigid and non-rigid objects for real-tie applications. The tracking approach is based on the application of discrete techniques relying on the correspondences between several feature points fro frae to frae. A vectorial object odel is introduced to exploit sooth otion constraints and support deforable otion. The tracking algorith has been ipleented using a PC coputer equiped with specific iage-processing hardware. To test the algorith with real iages a traffic-supervision application has been ipleented. In this experient the algorith succesfully tracks iages in which vehicle deforations are observed due to changes of perspective as result of otion. 1. Introduction The objective of otion tracking is to deterine the position and other otion paraeters of a obile object fro a video sequence. Tracking algoriths differs ainly in: (i) the iage analisis techniques used for otion estiation; (ii) the dynaic odel used to describe the evolution of the otion in tie. There are two general approaches to deal with otion analysis for visual tracking: (a) differential ethods, which use the relationships between the spatial and teporal gradients of iage intensity to copute optic flow [1][2]. (b) discrete techniques, based on finding the correspondence between feature points of iages taken at different oents [3][4]. Our tracking approach is based on the application of discrete techniques relying on the correspondences between several feature or interesting points fro frae to frae. This ethod is valid either for sall or great object displaceents and is efficiently supported by the iageprocessing hardware used. Thus, real-tie ipleentations can be achieved if tracking is applied to a liited nuber of points. In otion tracking a dynaic odel describing the evolution of the otion in tie is used. The dynaic odel akes use of the fact that physical objects generally exhibit teporally sooth otion to obtain ore robust results than estiation fro two views ethods. These otion odels can be roughly classified in rigid and deforable odels [5]. The use of deforable 2-D odels has great interest in areas like agricultural and food industry autoation. Soe iportant applications can be considered: the handling of non-rigid products and the quality control and inspection of agricultural products and food-packages. A previous work [6] dealt with the application of the algorith to autoated planting operations where physical deforation of the tracked seedlings was observed. But apparent deforations in the 2-D iage plane ay also be observed due to variation of the rigid object orientation with respect to the caera. In this work, the general deforable otion tracking algorith presented in [6] is applied to a rigid otion scene in which this kind of apparent deforation is given. The algorith has been developed in such a way that it could be ipleented using a PC-based digital iage processing hardware. This constraint has strongly conditioned the techniques to be applied and the algorith coplexity. Particularly, the use of binary teplates has been forced by the best hardware perforance in processing binary iages. Moreover, the axiu nuber of feature points to be considered has been liited. 2. Tracking algorith The proposed tracking algorith is coposed of the following four basic steps: 1. Feature point selection. 2. Object odel definition. 3. Model atching. 4. Trajectory prediction and odel update.

Figure 1.- Selection of interest points. In this exaple, the ste-leaves union points of the plant are taken and the corresponding binary teplates are shown. 2.1. Feature point selection In this first step a set of feature points belonging to the object to be tracked ust be selected.

2 Figure 1.- Selection of interest points. In this exaple, the ste-leaves union points of the plant are taken and the corresponding binary teplates are shown Feature point selection In this first step a set of feature points belonging to the object to be tracked ust be selected. The suitability of any tracking algorith depends strongly on the selected points. In the literature [7] we can find that the ost appropriate points ust have four properties: distinctness, uniqueness, invariance and stability. Due to its natural coplexity we leave the autoatic selection of these points for further developent. At the present we use a anual procedure to ark feature points using a cursor over the static iage. Lets F 0 = f(x,y,t 0 ) be a n* gray level frae acquired at t 0. Fro F 0 a subset of l points IP = {p 1 (x 1,y 1 ), p 2 (x 2,y 2 ),...,p l (x l,y l )} F 0, naed feature points, are selected for each object on the iage, where x i, y j represents respectively the row and colun indexes Object Model Definition With the subset of interest points IP = {p 1 (x 1,y 1 ), p 2 (x 2,y 2 ),...,p l (x l,y l )} a second step is carried out to obtain an iconic representation of each one. This representation uses a sall area or window W i with p x q pixels and centered on each point. Then a Sobel operator is applied to W i (x,y) providing an edge iage window EW i (x,y). Finally, a binary representation is obtained by thresholding the edge iage EW i (x,y), where the edges or boundaries of the objects are labeled 1, and all other points are labeled 0. A ean value U i is used as thresholding level: U i p q EW i xy (, ) x y p* q = ( xy, ) Wi (ec.2.1) The advantage of using teplates T i is that this representation captures the object shape characteristics and exhibits certain iunity to changes of illuination and contrast in the iage. The figure 1 shows an exaple where a subset of four feature points are selected fro a plant. The corresponding teplates are also shown. Using these feature points, a vectorial object odel is defined. The ter vectorial refers to the use of a vector set where each vector represents the relative point position with respect to a reference point. This odel captures and odels the global object shape in an instant. Moreover, it can also be used to predict the object deforations through a prediction technique discussed below. The Vectorial Object Model is defined as follows: Given the interest point subset IP = {p 1 (x 1,y 1 ), p 2 (x 2,y 2 ),...,p l (x l,y l )} F 0, one of the is randoly chosen as the Base Point. Without lack of generality, we choose the first point selected p 1 as base point. Then, the position vectors fro p 1 to the rest (of points) are coputed, getting the l-1 vector set { r 12, r 13, r 14,... r 1l } relative to the base point. The object odel can be represented as the 2-tuple 1 = [ p 1, { r 12, r 13, r 14,..., r 1l }] Vectorial Object Model 2.3 Model atching Once the feature points have been selected and the corresponding teplates have been defined, the atching process is carried out in each successive frae F 2, F 3,... This process ust find the best pose of teplates T i in each new iage. In order to atch T i against F i (i.e., a piece of F i having the sae size of T i ) one should adopt soe easure of atch. Our approach uses a su of absolute differences easure. Obviously, before coputing the atch is necessary to apply the Sobel filter and thresholding the overall iage. Let F i be any gray level frae acquired in t i, and let EF i the corresponding binary edge iage, the su of absolute difference between T i and EF i at position p i (x i,y i ) is defined as: Teplate-point correspondence easure: p q S( p ) = EF ( x + k, y + l) T ( kl, ) i i i i i k= 0 l= 0 (ec.2.2)

3 as the weighted su of teplate-point correspondence on p i plus the teplate-point correspondences of points located at vectors extree. Multiple Match Measure MMM ( p, ) = K S( p ) + K S( p + r ) i i i 1 i 2 i ij j i l (ec. 2.3.) where r ij = r 1j - r 1i. The K 1 and K 2 factors have been introduced in order to give different iportance to the resulting atch obtained for p i of the reaining points. The experients results have shown that K 1 = l-1 and K 2 =1 are the best values. Figure 2. In the area of interest A 1 the teplatepoint correspondence process takes a axiu in the point p 1. Nevertheless applying the vectorial odel 1 = [ p 1, {r 1,2, r 1,3 }] on p 1 the other teplates produce low atches, as result the Multiple Match Measure is low. High teplate-point correspondences are obtained when the odel is applied either on p 2. or p 3. Supposing that the axiu of ultiple atch easureent is obtained on p 2 it s decided that the safest point in A 1 is p 1, rejecting p 1. Searching of correspondences is perfored through a sequential scan over EF i. The best atch is obtained in the position where S(x,y) takes a iniu. However, finding correspondences iplies a high coputational cost and it depends highly on the iage size to be exained. Moreover, if we will have to add the cost to generate EF i, the necessary global tie would be excessive and unacceptable for our ai. In order to realize the tracking in real tie it should be necessary to reduce the area where the atching process to apply. The forer ay be accoplished by defining a sall area of interest (AOI). It consist on a piece of iage that in the first step is centered around each original point p i. In successive fraes this AOI will be centered on a likely location deterined by eans of the vectorial odel prediction technique. Based on this teplate atching easure a Multiple Match process is defined. This process tries to axiize a consistence easureent between the point correspondences obtained on each frae and the expected object deforation included in the predicted vectorial object odel. For that, a Multiple Match Measure (MMM) of a vectorial odel located on a point, is defined as follows: Given a point p i and a vectorial object odel i, sitting the relative vectors of odel fro p i, the MMM is defined Then a Multiple Match Process is perfored as follows: With each new frae acquired, the actual and predicted vectorial object odels ( 1 and 1 ) are coputed. Next, an AOI i around of predicted points { p 1, p 1,..., p l } is posed. For each AOI the atching process (ec.2.2) is applied in order to obtain the point of axiu correspondence p i. Lets { p1, p2, p3,..., pl } be the axiu correspondence point set. The Multiple Match process consist of taking out the easureent MMM i between each p i and the predicted vectorial object odel i, and then selecting the point in which MMM i (p i, i ) takes a iniu. Aplying the vectorial odel fro this point, we have the best pose of odel Trajectory prediction and odel update As stated before, a trajectory prediction and soothing echanis is required to cope with the tracking of the oving point set. This additional process is intended to: a) Use the predicted positions to perfor a restricted iage processing by defining reduced areas of likely localization for each tracked point. b) Copensate for the iage acquisition and processing delays. c) Produce a trajectory soothing in addition to the prediction capability and reducing the noise due to isleading correspondence estiations. To accoplish a set of α-β Kalan filters [9] has been introduced. Kalan filters have been widely studied in control and estiation theory. The α-β filter is constant-coefficient Kalan filter which is intended to odel and predict target otion based on assuptions of constant velocity between sapling intervals.

4 Figure 3. Fraes #1, #11, #21 and #31 of a traffic scene. In the first frae squares ark the feature points selected (four per vehicle). Subsequent pictures show the traffic evolution and the trajectories described by the tracked points.

5 The vectorial odel update is perfored as follows. For each feature point i, if the axiu teplate atch location is within a axiu radi R fro location deterined by the best pose of odel, then the corresponding predicted odel vector r 1i is updated aking use of this atch location. In other case, the axiu atch point is ignored and the best pose location is directly used for vector prediction. With the sae strategy, absolute position p 1 of the base point is predicted obtaining the predicted vectorial object odel: 1 = [ p 1, { r 12, r 13, r 14,..., r 1l }] Predicted Vectorial Object Model One of the proble that akes the correspondence proble quite difficult, using teplates, is the transforation experiented by objects fro frae to frae. In general, little distortions or size changes due to perspective are well supported. But there are strong lack of correspondences when the points experient an iportant orientation change in the iage plane. Deforable teplates or recursively adaptable teplates [8] are soe of the experiented solutions. In order to iprove the algorith against this kind of probles a process to estiate points rotations has been ipleented. This ethod assues that when an object rotates, two or ore points belonging to it experient the sae rotation. This assuption is only valid for rigid objects but is also applicable to sooth deforations. To estiate point rotations, the vectorial object odel obtained at first tie was stored. This odel is naed Reference Model. Once the new predicted odel has been coputed on each frae, the angles between the vectors of reference and predicted odels are coputed (ϕ 1, ϕ 2, ϕ 3,..., ϕ l ). Probably the points will suffer different rotations due to deforations. Nevertheless the experients have shown that using the ode instead of individual angles better results ight be obtained. 3. Traffic scene tracking experient The application of visual tracking techniques to traffic supervision tasks allows perforing ore sophisticated studies than those possible with classical sensors. Besides to classical global easures, like vehicle counting, density, flow or speed paraeters, other coplex studies can be ade: e.g. origin and destination of vehicles in intersections or roundabouts, ore frequent paths, overtakings in conflictive sections, etc. The algorith has been developed for achieving real tie operation using a coercial iage processing syste Matrox Iage. This hardware is capable to perfor the atching process, using binary teplates up to 17x17 pixels, for the overall iage spending about 40 s. The binary edge extraction has a siilar cost. Figure 3 shows the superiposition of three tracking experients perfored with three vehicles tracing different trajectories. A set of four 17x17 teplates featuring each of its corners has been defined. Note the changes in size observed in the vehicles at the botto. The change of a vehicle orientation can also be observed along the sequence. 4. Conclusions An algorith for visual tracking of non-rigid objects in 2-D has been presented. It has been successfully proved with real iages for a traffic supervision application where shape deforation is observed due to change of vehicle position and orientation with respect to the caera. Fine tracking of individual vehicles characterized by four 17 x 17 teplates with a sapling rate of 12.5 fraes per second (80 s. period) has been achieved. The caera positioning gets good requisites of background unifority (road surface) and sooth shape distortion. The resulting algorith has been ipleented on the available coputer vision syste. To perfor tracking of any points in real tie has been necessary to liit these costly processes to a reduced area of the iage. To carry out the atching on these areas of interest has not been a severe constraint to the generality of the algorith. With a high frae rate and supposing sooth object otion the likelihood that the points lie inside the AOI is very high. This approach also has the advantage to relax the requireent of point uniqueness. This is, the existence of two siilar points don t produce abiguity whenever they are separated enough to not be included in the sae window. We conclude that the algorith gets good results when observed deforations are due to changes of perspective as result of object otion. References [1] P. K. Allen, A.Ticenko, B. Yoshii, and P. Michelan, Trajectory Filtering and Prediction for Autoated Tracking and Grasping of a Moving Object, Proc. IEEE Int. Conf. on Robotics and Autoation, pp , [2] A. Verri and T. Poggio, Motion Field and Optical Flow: Qualitative Properties, IEEE Trans. Patt. Anal. Machine Intell., Vol. 11, nº 5, pp , [3] S. T. Barnard and W. B. Thoson, Disparity Analysis of Iages, IEEE Trans. Patt. Anal. Machine Intell., Vol. 2, nº 4, pp , July [4] J. J. Weng, 3-D Motion Analysis fro Iage Sequences Using Point Correspondences, Handbook of Pattern Recognition and Coputer Vision, pp , 1993.

6 [5] A. Murat Tekalp, Digital Video Processing, Prentice Hall, [6] V. Atienza, J.M. Valiente, G. Andreu and P. Albertos, Non-Rigid Object Visual Tracking in 2- D,: Proc. of the Fourth Worshop in Agriculture and the Food-Industry (IARP), Toulouse, october-1995, pp [7] R. M. Haralick and L. G. Shapiro, Coputer and Robotic Vision, Addison-Wesley Publishing Copany, [8] J. Aat, J. Aranda, A. Casals, Tracking Capabilities in Coputer Vision for Industrial Applications, Optical Engineering, Vol. 32, pp , 1993 [9] A. J. Wavering and R. Luia, Predictive Visual Tracking, SPIE Intelligent Robots and Coputer Vision XII, Vol. 2056, pp , 1993.

IMAGE MOSAICKING FOR ESTIMATING THE MOTION OF AN UNDERWATER VEHICLE. Rafael García, Xevi Cufí and Lluís Pacheco

IMAGE MOSAICKING FOR ESTIMATING THE MOTION OF AN UNDERWATER VEHICLE Rafael García, Xevi Cufí and Lluís Pacheco Coputer Vision and Robotics Group Institute of Inforatics and Applications, University of