Effective Tracking of the Players and Ball in Indoor Soccer Games in the Presence of Occlusion

Transcription

1 Effective Tracking of the Players and Ball in Indoor Soccer Gaes in the Presence of Occlusion Soudeh Kasiri-Bidhendi and Reza Safabakhsh Airkabir Univerisity of Technology, Tehran, Iran {kasiri, Abstract Object tracking is one of the ajor subjects in achine vision and plays a ain role in detection of ajor events in indoor soccer atches. In this paper, a novel approach for tracking the ball and players is proposed. In this ethod, the ground lines are segented and eliinated using a fast and effective ethod. Then, the reaining non-field pixels are considered and labeled as players and the ball. A fast level set contour is used to track players and the ball. The proposed ethod can track players and the ball in presence of occlusion. Experients show that the proposed ethod is robust to occlusion and different field colors. 1. Introduction Tracking oving objects is one of the ost iportant fields in achine vision. Segenting and tracking indoor soccer players is a critical step since it is the first step of analyzing the oveent of the players. By tracking the players and extracting their oveent, it is possible to understand the tactic of the gae, and with this understanding, it is possible to autoatically extract the player s oveent, speed, strength, and ability in specific situations to instruct other players. Being able to track players quickly is iportant because the result of the tracking can be used in further analyses. Tracking soccer players using broadcast TV iages is a challenging proble because of players' quick oveents, occlusions, and caera oveents. Even though several systes have been suggested for tracking players and analyzing their oveents, there are still challenges in tracking players and detecting iportant events based on visual features. Research on segenting and tracking soccer players is growing in recent years. Figueroa[1] perfored the tracking through a graph representation in which the nodes correspond to the blobs obtained by iage segentation and the edges which represent the distance between the nodes. The edges are weighted using the blob inforation and trajectory in the iage sequence. Pers[2] suggested a tracking syste for indoor gaes based on cobination of otion detection, teplate atching, and color tracking. Chaven[3] segented and tracked players by background subtraction and a neural network-based digit recognition algorith. Needha [4] suggested a fraework for ulti-object tracking based on the Condensation approach and the Kalan filter. Utsui [5] detected and tracked players based on their color rarity and local edge properties. Choi [6] proposed a ethod in which players are tracked by teplate atching and Kalan filtering and their position is estiated based on the transforation between the input iage and the field odel. Deardena[7] used particle filtering to track players. Sato [8] suggested a fraework to track ultiple players using the teporal spatio-velocity (TSV) transfor. Zhu [9] used support vector achines and particle filters to track players in broadcasted sport videos. Beetz [10] suggested a fraework based on MHT [11] to deterine the coordinates and trajectories of football players in TV broadcasts. Ekin [12] proposed a ethod based on the KLT [13] ethod to track players, and finally Lefevre [14] used active contours to track soccer players. Most of the previous efforts which used iage sequences captured fro TV broadcasts segented the players and ball fro the non-field pixels of the iages. For this purpose, the opening orphological operator is usually used to eliinate noise and ground lines. However, using this operator to ground lines has the following drawbacks: 1. Specifying the size of the structuring eleent of operator depends on the specific iage. 2. The opening operator oits not only the ground lines, but also players feet which are very iportant in deterining the position of players in the ground odel. 3. It ay divide a player to several segents. 4. Soe of the players only have their legs in the ground field iage because they are next to the

2 ground borders. These players are eliinated by the operator and so are lost. 5. The cross-section of the circle and the central line of the ground ay not be eliinated by the opening operator, and soeties ay be istakenly considered as a player. 6. It ay eliinate the ball or ay cut apart the players feet aking sall sections which look siilar to the ball. Kalan and particle filters have usually been used for tracking players [4,6,7,9,15]. But tracking with these filters depends on specific paraeters and cannot handle topological changes [16]. Since active contours provide an independent fraework of paraeters and topological changes, they are suitable for tracking nonrigid objects. In this paper a novel approach based on a fast level set contour is proposed to track the players and ball in indoor soccer gaes. The ethod can handle occlusion effectively. The rest of this paper is organized as follows. In section 2, the general overview of the proposed syste is illustrated. In section 3, segentation of the ground lines, players, and ball is presented. Section 4 explains a fast level set algorith for tracking players and the ball. Experiental results are given in section 5. The conclusions are suarized in section General fraework overview The general fraework of the proposed ethod is given in Fig. 1. For each video frae, the playfield is detected first. Then the lines inside the extracted field are segented and the regions inside the extracted field are considered as player candidates. For each player regions, if it is identified as a newly appeared player, a tracker is assigned. The necessary steps are described in the ore details in the following sections. 3. Segentation of lines, players and ball 3.1 Field Extraction Extracting the region of the ground field is the ost iportant stage for tracking players since all iportant objects in the iage such as players, referee, ball, and the lines lie on the ground. In long shot iages, the field occupies the largest area in the iage. As a result, the peak value of the color histogra is generally considered as the field color and the iage is segented using a threshold. However, this threshold depends on the percentage of the field area pixels in the iage which varies in different iage sequences. For solving this proble, the distinctive colors of the iage are extracted and clustered into 10 clusters. Then color pixels are apped into these 10 clusters. The largest cluster specifies the pixels of the ground field. The ground color is odeled by Gaussian distribution. Figure 1. Players and ball tracking fraework To extract the entire field region, a binary iage of the field and non-field pixels should be created. For this purpose, we use the Mahalanobis distance between pixels of the frae and the ground color distribution. Fig. 1 shows typical result of applying the proposed algorith to different indoor soccer iages. By calculating the convex hull of the largest region, the field ask is extracted. Knowing the field regions in the iage allows sipler tracking by reoving clutter fro the crowd regions. The shape of the field region can also give inforation about the location of the field on which the caera is focused. 3.2 Line Segentation Ground lines are iportant parts of the field and are usually used to find the iage-to-odel transforation. To find the position of players and the ball in the field odel, the transforation between the iage and field odel ust be known. Ground lines usually collide with the players and ball and ake the segentation of players and ball difficult. If lines are segented accurately, segenting and tracking players and ball becoe easier.

3 Rule1: Connected coponents whose areas are not sall are selected as players. Rule2: Connected coponents whose areas and roundness are sall and are close to the border are selected as players. (a) (b) (c) (d) Figure 2. Ground field detection result. (a-c) saple iages fro different indoor soccer videos. (b-d) Corresponding ground field detection result (a) (b) Ground lines are white and thin. These features are coon in all iage sequences and they can be used in segentation. The proposed steps for segenting the ground lines are as follows: 1- White pixels of each frae are extracted and the Canny edge operator is applied to the frae. 2- By applying siple horizontal and vertical asks to the edges of the gray level iage, deterine the directed horizontal and vertical edges. 3- White pixels between positive and negative edges which have a distance less than 10 pixels are deterined and pixel values of negative and positive edges are replaced with distances between these edges. Other edge pixels are oitted (Fig. 3) 4- Adjacent edge pixels which are different at ost 2 are connected. The nuber of pixels each edges coponent has deterines its weight. 5- The edge coponents which have low weights are oitted. The regions between the reaining edge coponents are obtained and arked as ground lines (Fig. 3). 3.3 Player Segentation By oitting ground lines, the reaining connected coponents in the iage are players and the ball. To eliinate noise, soothing players and the ball regions, a edian filter is used. Fig. 4 shows the binary non-field iage without lines. The area and roundness (equation 1) of the connected coponents are used to detect the players. The following rules are used to specify players. Fig. 5a shows the result of segenting players. 4π Area Roundness = 2 (1) perieter (c) (d) Figure 3. (a) Directed horizontal edge. (b) Directed verticals edge. (c) Connected edges. (d) Segented ground lines 3.4 Ball Segentation By eliinating players and ground lines, other connected coponents are exained to detect the ball. For this purpose, area, roundness and the color of the connected coponent is used. Connected coponent which have a sall area, high roundness, and the difference between its ean color and white color is low is selected as the ball (Fig. 5). 4. Tracking Players and Ball Since level set contours are powerful techniques in handling topological changes such as the erging and splitting of object regions, they are ostly used for tracking ultiple objects. A fast level set contour which is proposed by Shi [17] is used for tracking players and the ball. In this fast algorith, the evolution of the curve is realized by siple operations such as switching eleents between two linked lists; therefore there will not be any need to solve any partial differential equations. Since the curve is still represented iplicitly, topological changes can be handled autoatically. Figure 4. Non-field pixels without lines

4 (a) (b) Figure 5. Result of segenting. (a) Players. (b) the ball In this level contour, a siple tracking strategy is used as follows. For each frae, the tracking results fro the last frae are used as the initial curves, and then each curve evolves to locate the object boundaries in the current frae. The boundaries of M object regions are denoted as C 1, C2,, CM. For the representation of ultiple object regions, two functions are used: one region indication function ψ and one level set functionφ. The region indication function is defined as follows: Ψ ( x) =, if x Ω ( = 0,1,, M ) (2) The level set function φ is negative inside all the object regions and positive in the background. For each object region Ω ( = 0,1,, M ) two lists of neighboring pixels L in and follows: L out L in L out can be defined as { x φ( x) > 0 and y N 4 ( x) such that φ( ) < 0} (3) { x φ( x) < 0 and y N4 ( x) suchthat φ( ) > 0} (4) = y = y where N ( ) 4 x is a 4-connected discrete Neighborhood of a pixel x ψwith x itself reoved. The interior pixels are those pixels inside the object regions but not contained in any L in, and exterior pixels are those pixels in the background but not in any L out. The level set function is defined as: 3 if x is an exterior pixel 1 if x L for 1 M φ ( x) = out (5) 1 if x L in for 1 M 3 if x is an interior pixel To switch pixels between L in and L out, two basic procedures are defined. The procedure switch_in() for a pixel x L out ( 0 < M ) ove the curve outward one pixel at x by switching it fro L out to L in and adding all its neighboring exterior pixels to L out. Forally, this procedure is defined as follows: switch_in( x) for x L out ( 0 < M ) : Step 1: Delete x fro L out and add it to L in. Set φ ( x) = 1 andψ ( x ) =. Step 2: y N 4( x ) satisfying φ ( x) = 3, add y to L out, and set φ(y) = 1. Siilarly, the switch_out() procedure that oves the curve inward one pixel at x L in ( 0 < M ) is defined as follows: switch_out( x) for x L in ( 0 < M ) : Step 1: Delete x fro φ ( x) = 1 andψ ( x) = 0. in L and add it to L out. Set Step 2: y N 4( x ) satisfying φ ( y) = 3, add y to in L, and set φ( y) = 1 and ψ ( x ) =. To track the object boundary, binary iage of the field and non-field pixels is coputed and stored in the array F d. First, the list is scanned and a switch_in() procedure is applied at a pixel if F d = 1. After this scan, soe of the pixels in Lin becoe interior pixels and they are deleted. Then, the list L in is scanned and a switch_out() procedure is applied for a pixel with F d = 0. Siilarly, exterior pixels in L out are deleted after this scan. At the end of this iteration, a stopping condition will be checked. If it is satisfied, the evolution will stop; otherwise, this iterative process will continue. Stopping condition is defined as follows: Stopping Condition: The curve evolution algorith stops if either of the following conditions is satisfied: The value of F d at each neighboring pixel satisfies: F ( x ) == 0 x ; F d L out d ( x ) == 1 x L in A pre-specified axiu nuber of iterations is reached. The details of this algorith for tracking M objects are listed as follows: Step 1: Initialize the arrayφ,ψ, F d, L in and L out. Step 2(cycle): For i=1: N do Copute the d a F for pixels in out < L out and L in For each pixel x L ( 0 M ), switch_in(x) if ( x ) > 0 F d

5 in < For each pixel x L ( 0 M ), if y N 4( x ), φ ( y) < 0, delete x fro Lin, and set φ ( x) = 3 in < For each pixel x L ( 0 M ), switch_out(x) if ( x) < 0 F d out < For each pixel x L ( 0 M ), if y N 4( x ), φ ( y) > 0, delete x fro L, and set φ ( x) = 3 out Check the stopping condition. 5. Results The perforance of proposed the fraework has been tested on test clips which are extracted fro 2 videos. The iage sequences were obtained fro international indoor soccer atches by different sources. Since, the videos have different forat and resolution. They were converted to AVI forat and the resolution of This algorith is ipleented in MATLAB on Windows XP with 2.4 GHz processor. The coputation period for tracking players depend on any factors, such as the nuber of players, occlusion, iage size, etc. On the average, the tie required to process each frae was about 11s. This is far fro real tie, but the current code is not optiized and contains large aounts of output visualization code. Fig. 6 illustrates the perforance of the proposed ethod on clips. The proposed ethod is able to detect all players in the ground field. We can see that our approach is able to detect all the players in the scene including the situation of player coing in (Fig.6 c-d) and out of frae (Fig.6 b-c). It tracks occluded players by one contour and the contour was divided into two contours when they ove away fro one another (Fig.6 d-f). The players were tracked successfully as long as they were detected. However, the contour of a player is segented in soe fraes. This is because of fast otion of the players, which akes soe regions of the player fade, specially their legs (Fig.6 h, ). It not only fails to eliinate soe parts of lines next to players and considers the as players region (Fig. 6 h, n), but also fails to track the ball in soe scenes where it has been shot (Fig. 6 h). 6. Conclusions and Future Work The ain objective of this paper is to track the players and ball in indoor soccer videos accurately. A novel approach is presented for segenting and tracking the players and the ball, which is robust in occlusion handling and fast caera oveents. Accurate segentation of ground lines allows us to find iage-to-odel transforation accurately, which is essential for finding position of players and the ball on the field odel. We wish to deterine the players position accurately for odeling the players behavior in future works. 7. References [1] P.J. Figueroa and N.J. Leite, Tracking soccer players aiing their kineatical otion analysis, Coputer Vision and Iage Understanding, vol. 101, pp , [2] J. Pers and S. Kovacic, Coputer vision syste for tracking players in sports gaes, First Int l Workshop on Iage and Signal Processing and Analysis, pp , [3] A. Chavan and S. Sasi, Vision-based real-tie speed tracking, IEEE International Workshop, Signal Processing, pp , [4] C. J. Needha and R. D. Boyle, Tracking ultiple sports players through Tracking ultiple sport players through occlusion, congestion and scale, British Machine Vision Conference, [5] O. Utsui, K. Miura, I. IDE, S. Sakari, and H. Tanaka, An object detection ethod for describing soccer gaes fro video, Proc. of IEEE Intentional Conf. Multiedia & Expo (ICME), pp , [6] S. Choi, Y. Seo, H. Ki, and K. S. Hong, Where are the ball and players? Soccer gae analysis with color based tracking and iage osaick, Proc. ICIAP Springer, [7] A. Deardena, Y. Deirisa, and O. Graub, Tracking football player oveent fro a single oving caera using particle filters, Proceeding of CVMP, pp , [8] K. Sato and J.K. Aggarwal, Tracking soccer players using broadcast TV iages, Proc. IEEE Conf. Advanced Video and Signal Based Surveillance,, pp , [9] G. Zhu, C. Xu, Q. Huang, and W. Gao, Autoatic ultiplayer detection and tracking in broadcast sports video using support vector achine and particle filter, Proc. IEEE International Conf. Multiedia and Expo, pp , [10] M. Beetz, S. Gedikli, J. Bandouch, Visually tracking football gaes based on TV broadcasts, Joint Conference on Artificial Intelligence (IJCAI), [11] D. B. Reid, An algorith for tracking ultiple targets, IEEE Transactions on Autoatic Control, pp , [12] A. Ekin, A.M. Tekalp, and R. Mehrotra, Autoatic extraction of low-level object otion descriptors, IEEE Conf. on Iage Processing, pp , [13] C. Toasi and T. Kanade, Detection and tracking of point features, Tech. Rep [14] S. Lef`evre, C. Fluck, B. Maillard, and N. Vincent, A fast snake-based ethod to track football players, Proc. IAPR International Workshop on Machine Vision Applications, pp , [15] J. Kang, I. Cohen, and G. Medioni, Tracking objects fro ultiple stationary and oving caeras, Proc. Of IEEE Conf. Intelligent Dist. Surveillance Systes, [16] Y. Rathi, N. Vaswani, A. Tannenbau, and A. Yezzi, Tracking deforing objects using particle filtering for geoetric active contours, IEEE Trans. on Pattern Analysis and Machine Intelligence, vol. 29, no. 8, [17] Y. Shi and W.C. Karl Real-tie tracking using level sets, IEEE Conf. on Coputer Vision and Pattern Recognition, pp , 2005.

6 (a) 10 th frae (b) 20 th frae (c) 30 th frae (d) 43 trd frae (e) 55 th frae (f) 64 th frae (g) 82 nd frae (h) 110 th frae (i) 117 th frae (j) 1 st frae (k) 22 th frae (l) 40 th frae () 65 th frae (n) 85 th frae (p) 85 th frae Figure 6. Final contour of applying the proposed tracking algorith to two saple sequences: (a) to (i) for sequence 1. (j) to (p) for sequence 2.