Visualization of Dominant Region in Team Games and Its Application to Teamwork Analysis

Transcription

1 Visualization of Dominant Region in Team Games and Its Application to Teamwork Analysis Tsuyoshi Taki and Jun-ichi Hasegawa School of Computer and Cognitive Sciences, Chukyo University 101 Tokodachi, Kaizu-cho, Toyota-shi, Aichi , Japan Phone: , Facsimile: Abstract In this paper, we present a method for visualization of an invisible feature in human group motion. This feature called dominant region is a kind of dynamic sphere of influence. A dominant region is defined as a region in where the person can arrive earlier than any other persons and can be formulated by replacing the distance function in the Voronoi region with a time function. As an application of the dominant region, a motion analysis system of team ball games was developed. In this system, the dominant region is used for quantitative evaluation of basic teamwork. From the experiments using actual soccer game scenes, it was shown that inferior or superior areas for each player and each team in the game can be observed visually and that some basic factor for teamwork can be evaluated quantitatively by using the dominant region. These evaluation results almost corresponded with those by some professionals. 1. Introduction Many techniques for extraction of human s motion features from motion pictures have been developed and used in various fields, such as sports, medicine, welfare and security [1],[2]. Some of extracted motion features can be visualized on the original pictures again. Such a value-added picture makes it possible for us to reduce much labor for watching of the scene and to promote understandings of the meaning. If moving trajectory or spatial relation between some peoples is visualized on the pictures as an added value, then it will become helpful tool for a surveillance system or a sports commentary which detects a person with something characteristic movements, or understands a cooperative movement constructed by several people from motion pictures. It is very important to evaluate how each individual s motion is linked to the group motion. Especially, in game analysis for team sports, group motion analysis based on each individual s movement is indispensable. Sometimes, the sphere of influence of each player or each team (the space where the player or the team can be superior to others) is expected as one of the objective and basic features. So, in this paper, we visualize and analyze the sphere of influence of each player or team, to evaluate group behavior in team ball games or to support a sports commentary [3],[4]. The feature proposed here is a sphere of influence of each individual in group behavior which is derived from spatial relation of each individual s position obtained directly from motion pictures and the player s physical ability. The sphere of influence is formulated by extending the Voronoi region [5], and the concrete calculation method using an approximate model of human s physical ability is presented. Then, it is shown that quantitative evaluation of teamwork based on movement of each individual for keeping enough space for ball passing is possible, with visualization of the sphere of influence. 2. Group Motion in Team Sports We selected a ball game in sports as a concrete subject in group behavior, because the area where each individual can move is limited, the aim of a movement is clear, and a pattern of group movements is comparatively easy to understand. The ball game considered here has two groups (teams) which are composed of the same number of individual (players), and they compete with each other to get a score by moving a ball to the opponent s goal. There are two basic skills used by players in this ball game; one is to move a ball by himself and the other is to send and receive a ball between the same team. It is necessary for moving a ball up to the opponent s goal to perform these basic plays continuously. Though there are some cases where forcible play or passing with a risk is

2 needed in an actual game, it is in general important, in various attacks or in the recovery of a mistake to perform each play stably. The sphere of influence of each player or each team ( the space where the player or the team can be superior to others) is expected as one of the objectives and basic features in order to have a sense of stability. Cooperative movement between players, in order to extend and preserve the sphere of influence, can be regarded as a most basic group motion (teamwork) in team ball games. Also, cooperative movement in order to remove the sphere of influence of an attacking team by a defending team can be regarded as one of teamwork. So, the movement obtained as a result of each player or team trying to keep their own sphere of influence can be regarded as the competitive movement between teams. 3. Modeling and Calculation of Dynamic Sphere of Influence The space around each individual as stated above can be considered as a kind of sphere of influence. Generally, given a set of points in a space, spatial territory of each point can be expressed by the Voronoi region [5]. We formulate this kind of region associated with each individual in group motion by extending the Voronoi region. 3.1 Voronoi Region Let P = { p 1, p 2,..., p n } be a set of n-points, I n = { 1, 2,..., n } be a set of natural numbers and R 2 be a set of real numbers, then the Voronoi region of p k P is defined as follows: V(p k ) = { x R 2 d(x,p k ) d(x,p m ) for m k, m I n } (1) where p k indicates both the label and the location vector of the point, and d(x,y) means the Euclidean distance from y to x. The set of Voronoi regions associated with members of P is called the Voronoi diagram, written by W(P)= V(p k ). (2) Then, let P = { p 1, p 2,..., p n } be a set of n-individuals at a certain time t. The Voronoi region for the individual p k is defined as: V(p k ) = { x R 2 d(x,p k ) d(x,p m ) for m k, m I n } (3) This can be regarded as a kind of sphere of influence for each individual at the moment t, but it is no more than a static sphere of influence based on the distance. 3.2 Dominant Region In an actual group, each individual moves with changing direction and speed according to their own physical ability. It is better for practical reasons to define the sphere of influence based on the shortest time rather than the distance in a dynamic environments like this. The sphere of influence of the individual p k P is defined again as follows: D(p k ) = { x R 2 t s (x,p k ) t s (x,p m ) for m k, m I n } (4) where t s (x,p k ) called shortest time is the time necessary for the individual p k to move from his current position to the point indicated by x, on condition that the individual move with all his might. That is, D(p k ) is a region where the individual p k can arrive earlier than any other individual when starting at t. This region called the dominant region of the individual p k at the moment t. Though the dominant region is defined only by replacing the distance d with shortest time t s, it is greatly different from the Voronoi at a point where the dominant region can express a sphere of influence based on each individual s movement and physical ability. In the same way, it is also different from what is called the weighted Voronoi region replacing the distance d with a linear function of d. 3.3 Calculation Model of Shortest Time To calculate the shortest time of a individual to each point x; the current position, the current moving vector and the acceleration vector of the individual are needed. The current position and the current moving vector can be estimated from motion images, while the acceleration vector should be predetermined by the individual s own will and physical ability. Then the accelerating ability is modeled here as acceleration patterns based on the physical ability of an average individual. An acceleration pattern consists of all possible acceleration vectors. Our system uses an acceleration model as shown in Fig. 1. The model indicated by Fig. 1 (a) is a simple acceleration model which has the same acceleration ability in every direction, unrelated to the moving speed of the individual. While in the model indicated by Fig. 1 (b), the vector pattern varies with the moving vector of the individual. This means that a human can accelerate in every direction with the same strength from standing still or moving at a very low speed, but as the moving speed becomes higher, it becomes harder to accelerate in the direction of movement. If the acceleration model, which reflects the physical

3 ability of each individual is based on the model as this is constructed, then the shortest time from the current position of a individual to each point can be approximately calculated using the model. For calculation of the precise accelerating ability, a more complicated model is needed, but the above model is a first approximation. 3.4 Practical Calculation (1) Approximation of Shortest Time Calculation of shortest time based on the model shown in Fig. 1 (b) is complicated and costs much. So, we have adopted the model indicated by Fig. 1 (a). Supposing that once the acceleration vector in order for p k to move up to a certain point is given at the initial time, the vector is fixed while the computation, the locus of p k becomes accelerated motion. Therefore, the shortest time t s for p k to move up to the point x is given by the following equation: 1 = min + + = 0, 2 ( t ) 2 t ( v ) a( pk x), t 0 ( 0 ) ( t0 ) ( t0 ) t s t at v t pk x a A where p k (t0) and v (t0) show the location vector and the moving vector of p k at the time t 0 respectively, and A shows the set of the acceleration vector in Fig. 1 (a). That is, the shortest time (5) t s is obtained as a positive real radical by solving the equation with regard to t as the direction of the vector a is changed. Because the area of action is restricted in a sports field such as a soccer field,, the average error between the approximate value and true value of the shortest time is not so large. The practical procedure for calculation of the shortest time is as follows: (a) First, a digital image is prepared for each individual at one moment in order to restore the shortest time pattern. (b) Second, compute x based on Eg. (5), while changing the value of the accelerating direction and the time t. (c) Then the pixel value on the image indicated by x is set to t as the shortest time. On the occasion that some computational results are obtained for a pixel, the value is replaced by the minimum one. (d) The pixel whose value is not yet set is interpolated using some set values (for example, the values of A, B and C are used in the Fig. 2). Finally, a digital pattern where each pixel is constituted of shortest time is obtained for each individual. We call this pattern the shortest time pattern (STP) of the individual. The calculation of the shortest time pattern as stated above is performed for all of the individuals. (2) Calculation of Dominant Diagram The dominant diagram is represented in the form of a labeled pattern. Each label indicates the individual s ID D time contour (a) C A B t=t1 t=t2 (b) : moving vector : acceleration vector Fig.1 Examples of acceleration model. : initial moving vector : starting point Fig.2 A computational model of shortest time pattern.

4 whose value is obtained by a minimum operation for shortest time patterns of all individuals. That is, let the shortest time pattern of p k be T k ={t ij (k) } and the labeled image for dominant diagram be E={e ij }, then * ( *) ( ) = k = min, 1 k n (6) e t t k ij ij ij k k The region which consists of the same label is the dominant region of the individual corresponding to the label. Also, the pattern which consists of the minimum value at each point for the shortest time pattern of all of the individuals is called the merged shortest time pattern. Fig. 3 shows dominant diagram for two individuals for various strengths of acceleration and of a moving vector. In this figure, the upper-left shows that both strength of acceleration and of a moving vector are small, and as the figure goes down, the strength of acceleration increases, while as the figure goes right, that of moving vector increases. Fig. 4 shows another examples of dominant diagram for two individuals. As seen in these figures, borders of the dominant regions are not always linear, and the dominant region of a individual is not always a single connected component. For example, when a individual runs to another individual, his dominant regions are formed not only around the individual but also behind another individual. The behavior of dominant region corresponds with behavior which can be observed in actual game scenes, and the features like this are never represented by a simple Voronoi region. However, in the special case where the acceleration models of all individuals are the same and moving vectors of all individuals are zero, dominant region corresponds with the Voronoi region. 4. Application to Soccer Game Analysis In actual group ball games, space management, ball passing and cooperative movement can be considered as significant factors in teamwork. We suppose here that good teamwork include cooperative movements of players with the purpose of making ball passing easier and getting more space to do it stably. Then, such group motions by players can be evaluated by using the shortest time pattern and the dominant region proposed in the previous chapter. In this chapter, our soccer game analysis system developed for verifying its possibility is shown with experimental results. low initial velocity high low acceleration high (a) (b) (c) (d) (e) :individual moving vector :individual s position Fig.3 Example of dominant diagram for some different initial velocity and acceleration. :individual s position :moving vector :dominant region Fig.4 Examples of dominant diagram for the movements of two individuals.

5 4.1 System Outline The system consists of two parts; motion analysis and teamwork evaluation. In the motion analysis part, a sequence of a soccer game taken by several cameras is digitized, and static objects such as lines or goal posts on the soccer ground and moving objects such as players or the ball are extracted and tracked respectively from each camera scene. Then players positions extracted from all camera scenes are transformed and merged into a soccer field space. In the teamwork evaluation part, cooperative movement and ball passing by player, are quantitatively evaluated using the results of motion analysis. The final results are visualized as 2D and 3D animation. 4.2 Motion Analysis Part (1) Camera Placement The telecasting image is not useful for analysis of the technical features in teamwork or the ability of each player on a team, because most scenes are intermittent and are focused on a player with the ball. In our system, several cameras are placed along the touch-line on the field as shown in Fig. 5 in order to cover the entire soccer field. Also, for every camera, its optical axis is fixed to be perpendicular to the touch-line. (2) Extraction of lines and players Extraction of static objects such as lines on the soccer field is comparatively easy because each camera s focus is fixed. Thresholding and line detecting by the Hough transformation are used. For tracking of player(s), a relatively simple method based on template matching is used here. The procedure is as follows. At first, a body part of each player is located manually in the first frame of the scene as an initial template for the player and then each player is tracked by correlationbased template matching. Each template is renewed automatically so as to cope with the changes of posture frame by frame. Also, occlusions are detected by using the distance between the center points of the templates of two players. If once an occlusion occurs, the previous template of the occluded player is stored and kept during the occlusion, and then the occluded player is tracked again with the stored template when occlusion is dissociated (when the distance between two templates is greater than a thresholding value). In the case where the above tracking procedure fails by occlusion or other factors, manual correction is applied, because we focus on teamwork evaluation rather than on motion tracking in this paper. (3) Transforming and Merging onto Soccer Field Space Finally it is necessary that the results obtained from each camera are merged into a result on the soccer field space, because the soccer filed is being filmed by several cameras. Then, players positions obtained from each camera scene Fig.5 Camera placement and examples of images taken from each camera

6 are transformed onto the soccer field space, and the same player is identified and merged, if the player is taken by a different camera simultaneously. Footing points of all players on the image space are transformed into their positions on the soccer field space using the intersection points of lines on the field. In practical calculation, simple ratio and cross ratio are used. The cross ratio for four points on a straight line is invariant to the perspective view transformation [6]. Then, if there is a player on the overlapped area where adjoining cameras have been filming, the points of the same player are merged based on the nearness of distance between those points. 4.3 Teamwork Evaluation Part In this section, basic teamwork such as cooperative movement and ball passing using dominant region and shortest time pattern as stated above, are evaluated quantitatively. (1) Evaluation of space making Increasing or preserving space in both attacking and defending are most important and basic teamwork. So, for quantifying the space, the area of team dominant region and its time variation are used as a criteria. It is possible for flexible evaluation to give the degree of significance as a weight to each point on the dominant region based on the positional relation such as the distance between the point and a goal point. (2) Evaluation of ball passing First of all, successful or unsuccessful ball passing is most important in evaluation of ball passing. Evaluation of passing is performed at the moment when the pass is done. Suppose that a pass is considered successful if the first player who can receive the passing the ball belongs to the same team. Not only the actual pass but also the virtual pass can be evaluated here. It is necessary in pass evaluation to compare the shortest time of all players with that of the ball at each point. The shortest time of the ball is the same formally as that of player. But the motion of the ball projected on the field space is considered to be approximately straight, and the shortest time is assigned only to points on the straight line. The ball can be received at the first point where the shortest time of the ball becomes greater than that of a player. Then, if the point is dominated by the player belonging to the same team, the pass can be evaluated as good. (3) Evaluation of pressure works Putting psychological pressure on the opposite team is also a basic and effective tactics in both attacking and defending. We consider here only pressure directly caused by spatial factors, for example, distances from players, density of players, sphere of influence, distance from the goal and distance from the ball. This kind of pressure is called spatial pressure. The spatial pressure SP for each player is defined by SP = m P + ( 1 m) P ( m 1 ) (7) D B where PD is the pressure concerned with distribution of dominant regions around the player, PB is that caused by the distance between the ball and the player, and m is a weight. Here, PD and PB are calculated by Tr PD = 1 S (8) and P B = r 1 d (9) D respectively, Sr means the area of a circle around the player, Tr means the area of team dominant region of the player within the circle, d is the distance between the ball and the player, and D means the maximum length of d in the field. 4.4 Experimental Results In this section, the system as stated above is applied to actual soccer game scenes taken by four conventional cameras. The cameras are set on the upper row of a stadium, as the views cover a half of the soccer field. Digitized image has width=640[pixel], height=480[pixel] and the number of gray level=256, and its one pixel is equivalent to from 25[mm] to 60[mm] on the soccer field. It has enough image resolution, because the radius of ball which is the smallest object for tracking in this system is 220[mm]. In tracking players, stable results were obtained except in some cases where there was occlusion. However, there were many failures in case where the players were completely occluded by other players. Also, tracking results become unstable when a player s posture is changed severely or when a player is crossing a white line on the soccer field. In these cases, manual correction was applied. Fig. 6 shows an example of dominant diagram and each player s position. The dominant region of each player is distinguished by different gray tone. So, as another visualization method, dominant region of each player was mapped on the original picture in order to exhibit as it is. Fig. 7. shows an example of mapping of boundary lines of each player s dominant region. Fig. 8 and Fig. 9 show an example of a sequence of mapping each player s dominant region and each team s dominant region, respectively. In the Fig. 9, dark gray regions show the team dominant region of the attacking team. From these results, the time variation of the attacking team s dominant region can be dynamically observed. Fig. 10 shows examples of animation reproduced from the tracking result, with dominant region of each player. In

7 Fig.6 An examples of dominant diagram and each player s position Fig.7 Mapping of boundary lines of individuals dominant regions Fig.8 Examples of mapping of individuals dominant regions on the original images Fig.9 Examples of mapping of team dominant regions on the original images Fig.10 Examples of animation with team dominant region

8 the animation, we can observe the scenes from arbitrary view point, but each individual s motion such as jumping or kicking is not represented, because available data is only the sequence of players position. Fig. 11 shows the change of the area of the team s dominant region of attacking and the defending team s respectively, and the horizontal axis means the frame number and the vertical axis means the area (the amount of the pixel dominated by the team). In the figure, the area of attacking team s dominant region increases in spite of on the opponent s side and then excels that of another team finally. That is, as for cooperative movement, it can be evaluated that the teamwork of the attacking team was superior to that of the defending team, in the scene where the attacking team actually made a goal. It was very remarkable for the change of the dominant region in this example. Fig. 12 shows an example of pressure evaluation results by computer and human for a defensive player. In this ratio of team dominant region 100 (%) defending team frame number attacking team Fig.11 Time variation of team dominant ratio for the attacking team. pressure human r= 5[m] r=10[m] r=15[m] frame number Fig.12 A result of pressure evaluation for a defensive player A in the defending team by team dominant region and by human. A B (a)merged shortest time pattern (b)dominant diagram (a) Merged shortest time pattern (b) Dominant diagram, : initial moving vector : player s position : initial moving vector, : player s position Fig. 13 An example of pass evaluation in a simulated game scene. Fig. 14 An example of pass evaluation in an actual game scene.

9 case, r=5, 10, 15 [m], m=0.7 are used. The results by computer and human are almost similar up to about 300 frame, but in the latter part of this sequence there are some disparities in evaluation. The main reason is that this player missed a kick for clearing the ball just at the front of the goal of his own team. This means that there is a psychological factor which is very hard to evaluate by spatial features. However, we think that the evaluation method proposed here can be used for teamwork analysis except evaluation of such as psychological factor. Fig. 13 shows a pass evaluation result for a simulated game scene. Fig. 13 (a) is the result of merging the shortest time patterns of the player A, B and the ball, and (b) is the dominant diagram. In (b), the arrow indicates the initial moving vector, the white line segment indicates the dominant region of the ball and the white cross indicates the point where the pass is received first. The result shows that player B receives the passed ball first because the edge of the ball s dominant region ( a white cross ) is included in the player s dominant region. Fig. 14 shows the result for an actual ballpassing scene in a soccer game. The pass was evaluated to be a success and it really succeeded in the actual game. 5. Conclusion In this paper, we defined and visualized a dynamic sphere of influence called dominant region for analysis of human group motion. Also as an application of the dominant region, we developed a motion analysis system of team ball games. In the system, the dominant region was used for quantitative evaluation of basic teamwork such as cooperative movement, ball passing and pressure working. From the experiments using actual soccer game scenes, it was shown that inferior or superior areas for a team could be visualized by the team. Also, goodness of teamwork in a game was quantitatively evaluated by using time variation of the area of the team dominant region. The experimental scale is small yet, but the evaluation results almost corresponded with those by some professionals. In the paper, the dominant region was used for teamwork evaluation of soccer games, but it is applicable to other similar ball games such as hockey, water polo, American football and so on. Furthermore, it is possible to apply not only to sports field but also other fields, for example, security, marketing and so on. Acknowledgments The authors thank Prof. H. Taki, Faculty of Physical Education, Chukyo University, Dr. O. Miyagi, Department of Physical Education, National Defense Academy of Japan and Mr. N. Yamashita, Nagoya Grampus Eight Inc. for their helpful suggestions from sports scientific view point, and colleagues of the Hasegawa s laboratory for much discussion. This work was supported in part by the grant-in-aid for Private University High-Tech Research Center and the grantin-aid for Scientific Research from the Ministry of Education, Japan. References [1] A.F. Bobick: Video Annotation: Computers Watching Video, Proc. 2nd Asian Conf. on Computer Vision, pp.i19-i23, [2] R. Hosie, S. Venkatesh and G. West: Classifying and Detecting Group Behaviour from Visual Surveillance Data, Proc. 14th Int. Conf. on Pattern Recognition Volume I, pp , Aug [3] T. Taki, J. Hasegawa and T. Fukumura, Development of Motion Analysis System for Quantitative Evaluation of Teamwork in Soccer Games, Proc. IEEE Int. Conf. on Image Processing (ICIP-96), pp , [4] T. Taki and J. Hasegawa, Dominant region: A basic feature for group motion analysis and its application to teamwork evaluation in soccer games, Proc. IS&T/SPIE Conf. on Videometrics VI, 3641, pp , [5] A. Okabe, B. Boots and K. Sugihara, Spatial Tessellations Concepts and Applications of Voronoi Diagrams, John Wiley & Sons, New York, [6] J. Blinn, Jim Blinn's Corner -The Cross Ratio-, IEEE Computer Graphics and Applications, pp.78-80, Nov./Dec