Tracking by Cluster Analysis of Feature Points and Multiple Particle Filters 1

Transcription

1 Tracng by Cluster Analyss of Feature Ponts and Multple Partcle Flters 1 We Du, Justus Pater Unversty of Lège, Department of Electrcal Engneerng and Computer Scence, Insttut Montefore, B28, Sart Tlman Campus, B-4000 Lège, Belgum wedu@montefore.ulg.ac.be, ustus.pater@ulg.ac.be Abstract. A movng target produces a coherent cluster of feature ponts n the mage plane. Ths motvates our novel method of tracng multple targets by cluster analyss of feature ponts and multple partcle flters. Frst, feature ponts are detected by a Harrs corner detector and traced by a Lucas-Kanade tracer. Clusters of movng targets are then ntalzed by groupng spatally co-located ponts wth smlar moton usng the EM algorthm. Due to the non-gaussan dstrbuton of the ponts n a cluster and the mult-modalty resultng from multple targets, multple partcle flters are appled to trac all the clusters smultaneously: one partcle flter s started for one cluster. The proposed method s well suted for the typcal vdeo survellance confguraton where the cameras are stll and targets of nterest appear relatvely small n the mage. We demonstrate the effectveness of our method on dfferent PETS datasets. 1 Introducton Tracng of movng targets s an elementary tas n many computer vson applcatons such as vdeo survellance, sports analyss, human computer nteracton, etc. Many dfferent types of features have been used for tracng ncludng ponts, edges, color, and templates. In ths paper, we explore pont features as they are ubqutous and can be easly detected by e.g. the popular Harrs corner detector [1]. Most prevous wor on pont tracng focused on reconstructng ndvdual pont traectores as long as possble. For nstance, the Kanade-Lucas-Tomas (KLT) algorthm [2] matches ponts by mnmzng the sum of squared ntensty dfferences. As mnmzaton s senstve to local extrema, KLT fals easly n case of occlusons and target deformaton. In Arnaud et al [3], a stochastc flterng framewor that blends a dynamc pror model and measurements provded by a matchng technque was ntroduced and proved capable of dealng wth abrupt moton changes and partal occlusons. In Shafque et al [4], optmal matchng was adopted to explot smlarty nformaton of feature ponts n multple frames so that tracng s done by means of -frame pont correspondence usng graph theory. However, the ey problem remans: when a target s occluded or deforms, feature ponts 1 Ths wor has been sponsored by the Régon Wallonne under DGTRE/WIST contract 031/5439.

2 become less stable - corners dsappear durng occluson or turn to edges durng deformaton - mang tracng or matchng ndvdual ponts dffcult. In ths paper, a novel method that attacs the nstablty problem wth a dfferent methodology s presented. The ultmate goal for most tracers s to detect and trac movng targets, and the tracng of ponts s the means to acheve ths goal. By observng that a movng target produces a coherent cluster of feature ponts n the mage plane, tracng s converted to cluster analyss of feature ponts. Frst, feature ponts are detected by a Harrs corner detector and traced by a KLT tracer. Clusters of movng targets are then ntalzed by groupng spatally co-located ponts wth smlar moton usng the EM algorthm [5]. Due to the non-gaussan dstrbuton of the ponts n a cluster and the mult-modalty resultng from multple targets, multple partcle flters [6] are appled to trac the clusters n the followng sequences. Therefore, nstead of tracng ndvdual ponts, we capture the stochastc propertes of the clusters of feature ponts durng tracng so that mssng or unstable feature ponts don t affect the tracng results very much. Our method s well suted for the typcal vdeo survellance confguraton where the cameras are stll and targets of nterest appear relatvely small n the mage, thus feature ponts on them show strong coherence n space and moton. We demonstrate the effectveness of our method on dfferent PETS datasets [7]. The dea of tracng by cluster analyss was ntroduced by Pece [8] and borrowed nto ths wor. Our contrbutons are, frst, to apply t to ponts nstead of regons, thus avodng bacground modelng whch s senstve to llumnaton changes; second, to tae moton coherence nto account when computng measurements of clusters, whch mproves the robustness of cluster analyss; thrd, to ntegrate cluster analyss n the framewor of partcle flterng, whch stablzes the estmaton of the cluster parameters sgnfcantly. Secton 2 descrbes the overvew of our method and states the problem. Automatc ntalzaton by EM based cluster analyss s gven n Secton 3. Secton 4 ntroduces multple target tracng usng multple partcle flters. Results on sequences from PETS 2001 are llustrated n Secton 5. 2 Overvew The motvaton of ths wor s to develop a multtarget tracer for vdeo survellance applcatons. By detectng Harrs corners and applyng KLT n each frame, all the feature ponts wth ther assocated veloctes n the sequence are obtaned, as shown n Fgure 1. Ponts on movng targets exhbt large dsplacements, whereas ponts on the statc bacground are characterzed by very lttle moton. An ntutve soluton of tracng targets va feature ponts s to cluster coherent ponts usng the EM algorthm [9]. However, the problems of usng EM drectly are, frst, the number of ponts n a cluster vares from target to target and over tme, dependng on the sze and appearance of the target. Sometmes few ponts n a cluster are detected due to the lac of texture nformaton. Then, the spatal dstrbuton of ponts n a cluster s not well represented by a Gaussan model; a fnte unform dstrbuton s more approprate. In contrast, the moton dstrbuton of a cluster s well approxmated by a Gaussan.

3 Fg. 1. Result of the Harrs corner detecton and the KLT tracng. In the left panel, pont dstrbutons of clusters are shown n the mage plane. All the corners n the sequence are dsplayed n the spato-temporal space n the rght panel. After removng bacground ponts, the structure of the traectores of movng targets can be clearly seen. We apply multple partcle flters to solve these problems, as partcle flters are well nown for ther ablty to handle clutter and non-gaussanty [10]. The man dea behnd t s smple: Snce feature ponts n a cluster are too sparse to model ts dstrbuton, a set of partcles are sampled n a cluster. Each partcle s evaluated accordng to some dstance functon so that t receves a weght reflectng the lelhood that the partcle orgnates from the cluster. The cluster parameters are then updated from the weghted partcles. Based on a pror moton model, the cluster dstrbuton s propagated n the sequence so that the target s traced. Multple partcle flters are appled to trac multple targets smultaneously. New flters are started when a large number of feature ponts exst that are not assocated wth any exstng flters. Ther parameters are ntalzed by clusterng ponts usng the EM algorthm. Exstng flters are termnated when the total weghts of ther partcles drop below a threshold. Ths happens n case of occlusons and targets leavng the scene. 2.1 Problem Statement A feature pont x s represented by ts mage coordnates u and ts velocty s. A cluster of a target O s represented by a set of coherent feature ponts { o v x = ( u, s ), = 1... n }, and s parameterzed by a Gaussan ( o, Σ, v, Σ ), where o v o s the spatal center, Σ s the spatal covarance, v s the average velocty, and Σ s the velocty covarance. The spatal and moton dstrbutons of the ponts n a cluster are assumed ndependent. Therefore, the problem of tracng s stated as: gven the parameters of clusters n the prevous frame, detect how many clusters are present n the current frame and assgn each feature pont to a cluster. In the followng sectons, we show how t s solved by ntalzng wth EM based cluster analyss and tracng wth multple partcle flters.

4 3 EM based Cluster Analyss Automatc ntalzaton s crucal to the success of a vdeo survellance system. Targets should be located when they frst appear. An EM based cluster analyss algorthm s appled when a large number of feature ponts exst that are not assocated wth any exstng clusters. Note that new targets may not only occur at the borders but anywhere wthn the mage. Decdng the number of clusters n the data s usually the hardest problem n cluster analyss. A votng technque was devsed to solve ths problem. Intutvely, each pont spreads a weght to ts neghbors based on the dstance between them. After votng, each pont computes ts weght by collectng all the votes receved. Ponts near the center of a cluster tend to have a larger weght. Ths method s ncdentally the frst phase ( sparse votng ) of tensor votng [11]. By loong for local maxma, the number of new clusters and ther centers are detected. Usng these results for ntalzaton, an EM algorthm s appled to estmate the cluster parameters. The probablty that a feature pont orgnates from a cluster can be estmated from ts locaton and the velocty, defned as f ) exp( dst( x, O ))), where the dstance between a pont and a cluster s ( o u o Σ 0 u o dst ( x, O ) =. v s v 0 Σ s v T Accordng to Bayes theorem, the posteror probablty that pont s generated by one of the clusters s p ( ) = w w f ( ) f ( ) 1, where w s the pror probablty of cluster defned as the fracton of mage pxels generated from cluster. Ponts are assocated wth the cluster that maxmzes the posteror probablty. Once all the ponts are assgned, the parameters of each cluster are re-estmated by summng the evdence over all ts ponts. Ths s terated untl EM converges to a local maxmum of the lelhood of the observed data. A phase of K-Means clusterng s nserted to obtan a better ntalzaton so that the EM algorthm converges wth fewer teratons. In fact, n cases where the targets are well separated, EM does not change the output of K-Means at all. Results are shown n Fgure 2. (1) 4 Multple Partcle Flters Multple partcle flters are a smplfed mplementaton of the mxture partcle flter whch s capable of mantanng the mult-modalty of the posteror dstrbuton and of tracng multple targets smultaneously [6, 12, 13]. Wth a smlar dea, we model each cluster wth an ndvdual partcle flter, start a flter when a cluster s detected and termnate t when the cluster dsappears.

5 4.1 Intalzaton of a partcle flter Gven the ntal parameters of a cluster obtaned from the cluster analyss step, a partcle flter s started. Two sets of partcles are sampled n each flter: one from the ntal dstrbuton of the cluster and the other around each feature pont n the cluster, shown n Fgure 3. Fg. 2. Results of ntalzaton of clusters n the frst frame. Feature ponts detected by the Harrs corner detector and traced by KLT are grouped nto clusters representng targets. Fg. 3. Results of ntalzaton of multple partcle flters. The green dots are sampled partcles.

6 Let u, s ) x = be the m-th partcle n the -th flter (correspondng to the -th m, t ( m, t m, t cluster) at tme t, so where x O, u ε u x m, 0 = sample( O ) or x m 0 = x + ε s ε and,, ε s are random varables modelng respectvely the changes n space and moton. The reason of samplng 2 sets of partcles s because of the non-gaussanty of the feature ponts n a cluster. In ths way, the partcles are scattered n the cluster and the dstrbuton s fully and well approxmated. In all experments, 100 partcles are sampled around a feature pont, and the number of partcles sampled from the cluster dstrbuton s proportonal to the sze of the cluster. (2) 4.2 Tracng by Multple Partcle Flters A partcle n flter s propagated n the sequence based on the constant velocty assumpton, 1 1 x m, t + 1 = x 0 1 m, t ε u + ε s and weghted by a functon of the dstances between the partcle and the feature ponts around t, defned as where the dstance s m 1 = dst( xm, 1, x, w exp( 1)),,, (4) (3) 1 0 m, 1, x, 1) = m, 1, 1, + 1, 1 0 m t t s T u ( x x ) ( x x ) dst ( + x. (5) u and s are set to balance the nfluence of the dstance n space and n velocty. The parameters of cluster are then estmated from the weghted partcles. The computed parameters are not good enough because of the possble target deformaton and the unstable feature pont detecton. For nstance, the way that the weght s computed n Equaton 4 tends to attract partcles to the closest feature pont. As a result, when new feature ponts appear n a frame, there may be few partcles of large weghts near them (especally when these new feature ponts are near the border of the cluster) so that ther contrbutons to the estmaton of the cluster parameters are unfortunately gnored.

7 To solve ths problem, a one-step clusterng s nserted to assgn all the feature ponts to one of the clusters usng ther current parameters based on the dstance defned by Equaton 1. New partcles are sampled around each feature pont. The new sampled partcles plus all the exstng partcles n a cluster are then reweghted by a functon that averages the prevously computed weght and the dstances between partcles and ther clusters, defned as w = α exp( dst( xm, 1, x, 1)) + (1 α)exp( dst( xm, 1, O, t 1)). (6) m, 1 + The frst term of the above equaton descrbes the smlarty measurement of the partcle wth ts neghborng feature ponts, whle the second term penalzes how coherent the partcle s wth the cluster. Fnally, the parameters of the cluster are refned from ts reweghted partcle set. The fnal step of a partcle flter s to resample partcles based on ther weghts so that partcles wth small weghts are lely to be dscarded and those wth large weghts are duplcated. Note that a fxed number of partcles n a flter are resampled durng tracng. In summary, the tracer conssts of the followng steps: (1) Predcton: partcles are propagated usng Equaton 3. (2) Weghtng: ther mportance weghts are computed usng Equaton 4. (3) Clusterng: assgn feature ponts n the current frame to a cluster; new partcles are sampled around each feature pont. (4) Reweghtng: partcles are reweghted usng Equaton 6, and the parameters of the clusters are refned. (5) Resamplng: resample partcles usng the Monte Carlo Samplng technque. These steps are terated to propagate the dstrbutons of the clusters n the sequence. At the Clusterng step, f a large number of feature ponts exst that are not assocated wth any exstng flters, a new partcle flter wll be started and ntalzed by the EM based cluster analyss, as s stated n Secton 3. At the Weghtng step, f the total weght of all the partcles n a flter drops below a threshold, the flter wll be termnated. Ths happens when the target s occluded or leaves the scene. 5 Results The proposed method s evaluated on dfferent sequences from PETS2001. Fgure 4 shows the result of tracng a subsequence of 300 frames n the sequence of Camera 1 of Dataset 1. Note that two crossng targets n the sequence are traced separately durng the occluson, shown n the rght panel of Fgure 4, snce they exhbt dfferent moton. Four challengng subsequences from the nosy sequence of Camera 1 of Dataset 3 are selected to evaluate the robustness of the method, as s demonstrated n Fgure 5. They contan substantal and rapd llumnaton changes, shadows, severe occlusons and groups of people enterng and leavng. The algorthm proves robust to substantal changes n llumnaton snce the Harrs corner detector s relatvely nsenstve to lghtng changes. As shadows move along wth the targets that cast them, they are traced as a part of the targets and ntroduce only small tter n the traectores. The algorthm has problems mantanng a stable number of clusters n case of severe occlusons, because shadows connect dstnct clusters and people move from one cluster to another. We are currently

8 studyng complementary methods for tracng ndvdual targets usng model-based approaches. Fgure 6 and 7 llustrate the results of the comparson of our method wth drect KLT tracng and our prevous bacground-subtracton method [14]. The frst comparson shows that KLT tracer fals durng target deformaton and occlusons, because when corners turn to edges, the tracs of ponts slde along edges, and when occluded, ponts drft from one target to another; meanwhle, our method s able to capture the stochastc propertes of targets and s not affected by unstable feature ponts. The second comparson shows that bacground models are dffcult to mantan n the presence of rapd lghtng changes and fal n such stuatons (consult Pater et al [14] for more detals), whereas our method s less senstve and contnues to trac. The only problem s that shadows show up or dsappear when llumnaton changes rapdly, whch affects the parameters of clusters. Nevertheless, a practcal drawbac of our method s that tracs of targets tend to be lost f they move slowly or possess lttle texture. Another drawbac s that the method s only capable of dealng wth partal occlusons. In case of complete occluson, new targets are detected and are not lned to ther correspondences before occluson due to the lac of other nformaton such as the appearance of the targets. However, an advantage of our method s that the errors wll not be propagated n the sequence so that nteractve rentalzaton s unnecessary. 6 Conclusons and Future Wor Ths paper presents a novel method of tracng movng targets va feature ponts. The method s sutable for the vdeo survellance confguraton where the cameras are stll and targets are relatvely small n the mage so that feature ponts on a target form coherent spato-temporal clusters. The EM algorthm and multple partcle flters are appled to cluster feature ponts and to trac all the targets smultaneously. As demonstrated, the method s robust and capable of dealng wth partal occlusons, shadows and llumnaton changes. We are currently focusng on tracng n dffcult stuatons such as severe occlusons. Complementary methods for tracng ndvdual targets over long sequences are beng developed usng model-based approaches and probablstc data assocaton. An extenson of the current wor to movng cameras s also ongong and wll broaden ts applcaton to e.g. sports analyss. References 1. Harrs, C., Stephens, M.: A Combned Corner and Edge Detector, Fourth Alvey Vson Conference, pp , Lucas, D.B., Kanade, T.: An Iteratve Image Regstraton Technque wth an Applcaton to Stereo Vson, Internatonal Jont Conference on Artfcal Intellgence, pp , 1981.

9 Fg. 4. Results of tracng. All the partcles n the sequence are dsplayed n the spato-temporal space n the mddle panel. Fg. 5. Results of tracng four nosy sequences to evaluate the robustness of our method. Note that only long traectores wth large certantes are dsplayed. 3. Arnaud, E., Memn, E., Cernusch-Fras, B.: A Robust Stochastc Flter for Pont Tracng n Image Sequences, Asan Conference on Computer Vson, Korea, Shafque, K., Shah, M.: A Nonteratve Greedy Algorthm for Multframe Pont Correspondence, IEEE Transactons on PAMI, Vol. 27, No. 1, pp , January Dempster, A.P., Lard, N.M., Rubn, D.B.: Maxmum Lelhood from Incomplete Data va the EM Algorthm, Journal of the Royal Statstcal Socety, Vol. 39, No. 1, pp. 1-38, Loongbll, A., Leb, D., Stavens, D., Thrun, S.: Learnng Actvty-based Ground Models from a Movng Helcopter Platform, ICRA, Span, Ferryman, J.: PETS 2001 Datasets, 8. Pece, A.E.C.: Generatve-Model-Based Tracng by Cluster Analyss of Image Dfferences, Robotcs and Autonomous Systems, Vol. 49, No. 3, pp , 2002.

10 Fg. 6. Comparson of the method wth drect KLT tracng. 3 KLT tracs are dsplayed n the mddle panel. Note that the red one drfts from the group of people to the vehcle durng occluson (the frst row of the rght panel), and the blue one umps from one leg of a pedestran to another durng deformaton (the second row of the rght panel). As our method captures the stochastc propertes of the clusters of feature ponts, these unstable feature ponts don t affect the tracng results very much, shown n the left panel. Fg. 7. Comparson of the method wth our prevous wor whch ntegrates bacground subtracton and moton hstory detecton n the framewor of a Kalman flter [14]. As expected, the trac of the pedestran s lost durng the rapd llumnaton changes, shown n the rght panel, whereas our new method succeeds n tracng n such stuatons, shown n the left panel. In the mddle panel, two submages of the same pedestran at dfferent tme are dsplayed to show how the parameters of the cluster are affected by the shadow. 9. Du, W., Pater, J., Verly, J.: Tracng by Perceptually Groupng Feature Ponts nto Clusters, submtted. 10. Doucet, A., Fretas, N., Godor, N., eds: Sequental Monte Carlo Methods n Practce, Sprnger Verlag, Medon, G., Tang, C.K.: Inference of Integrated Surface, Curve and Juncton Descrptons from Sparse 3-D Data, IEEE Transactons on PAMI, Vol. 20, No. 11, pp , Vermaa, J., Doucet, A., Perez, P.: Mantanng Mult-Modalty through Mxture Tracng, Internatonal Conference on Computer Vson, Nce, France, Ouma, K., Taleghan, A., Fretas, N.D., Lttle, J.J., Lowe, D.G.: A Boosted Partcle Flter: Multtarget Detecton and Tracng, ECCV, Vol. 1, pp , Pater, J., Crowley, J.: Mult-Modal Tracng of Interactng Targets Usng Gaussan Approxmatons. Proceedngs of the Second IEEE Internatonal Worshop on Performance Evaluaton of Tracng and Survellance, Hawa, USA, 2001.