Soccer Server and Researches on Multi-Agent Systems NODA, Itsuki and MATSUBARA, Hitoshi ETL 1-1-4 Umezono, Tsukuba, Ibaraki 305, Japan fnoda,matsubarg@etl.go.jp Abstract We have developed Soccer Server, a simulator of soccer, which provides a common test-bench to evaluate multi-agent systems and cooperative algorithms. This server enables a team of players written any kinds of programming systems to play a match with another team written another kind of systems. Using this server, we can compare the eciency of cooperative actions of various multi-agent systems from a couple of viewpoints. 1 Introduction Soccer, the association football, is a typical team-game, in which each player is required to play cooperatively. Moreover, soccer is a real-time game in which situation changes dynamically. Because of these features, soccer can be considered as a standard problem of multi-agent systems and cooperative algorithms. The World Cup Robot Soccer Initiative (RoboCup) is an attempt to foster AI and intelligent robotics research by a standard problem where wide range of technologies can be integrated and examined. In the history of AI, standard problems have been the driving force for AI research. Chess is one of the typical standard problems in AI and a lot of search algorithms has been investigated by computer Chess researches. Chess is a two-person, complete information game, whereas Soccer is a multi-person, incomplete information game. We think the Soccer Server oers AI research good test-bench for multi-agent systems. We have developed Soccer Server, a simulator of soccer, to provide a common test-bench to evaluate various multi-agent systems and cooperative algorithms. Using the server, a team of players written any kind of programming systems can play a match with another team written another kind of systems. 2 Soccer and Multi-agent System From the standpoint of multi-agent systems, soccer, which is just one of typical team sports, make a good example of problems in the real world that is moderately abstracted. Multi-agent systems provide us with research subjects such as cooperation protocol by distributed control and eective communication, while having advantages as follows: eciency of cooperation adaptation robustness real-time-ness Soccer has the following characteristics: Robustness is more important than elaboration. A team should take a fail-safe strategies, because playplans and strategies may fail by accidents. Adaptability is required for dynamic change of plans according to the operations of the opposing team. Team play is advantageous. A match is uninterrupted. Each player is required to plan in real-time, because it is dicult to make a whole plan before plays. As precise communication by language is not expected, eectiveness must be provided by combining simple signals with the situation. These characteristics show that soccer is an appropriate example for evaluation of multi-agent systems. Various examples have been used to evaluate performance of multi-agent systems, but such examples are simple and small. Soccer is complex and large, so it is suitable to evaluate them in the large scale. Moreover, because soccer has a long history and is familiar to many people [Asada et al., 1995; Sahota, 1993; Sahota, 1994; Stone and Veloso, 1995], there are many techniques and strategies that may be also useful for multi-agent systems as a test-bench. 3 Soccer Server 3.1 Overview Soccer Server enables a soccer match of two teams of player-programs written by dierent programing systems. A match using Soccer Server is carried out by
a client-server system: Soccer Server provides a virtual soccer eld (Figure 1) and simulates movements of players and a ball. s are brains of players and control their actions via a computer network. Soccer Server consists of 3 modules: a eld simulator module, a referee module and a message-board module (Figure 3). The eld simulator module calculates movements and collisions objects on the eld. The referee module controls a game according to rules. The message-board module manages communication among clients. A client connects with the server by a UDP/IP socket. Using the socket, the client sends commands to control an assigned player and receives information from player's sensors. Basically, each client can control only one player 1, so that the server connects with 22 clients at most 2. All communication between the server and each client is done by using ASCII strings. Therefore clients can be written by any kinds of programing systems on any architectures that have facilities of UDP/IP and string manipulations. Moreover, players in a team can be written by various systems, so that it is possible to make a picked team like all-star selections. 3.2 Simulation of Objects' Movements The soccer eld and all objects on it are 2-dimensional. This means that only horizontal movements are taken into account and vertical movements are ignored. The size of the eld is 105m 2 68m 3.The width of goals is 14.64m, that is the doubled size of ordinary rules, because 2-dimensional simulation makes it dicult to get goals. Players and a ball are treated as circles like Figure 2. Their movements are simulated stepwise one by one in a simple way: Velocity of an object is added to its position, while the velocity decays in a certain rate and increases according to its acceleration given by commands from clients. The collision is also simplied. If an object overlaps another object, that is, two objects collide with each other after a movement, the object is moved back until it does not overlap other objects. Then its velocity is multiplied by 00:1. There also exist various landmarks, that is ags, lines and goals, on the eld, which are visible but do not move. They are used for clients to determine their players' position. s can not know absolute positions of their own players, but know only the relative position to other objects. So clients need to determine their positions from the relative position of these xed objects. 1 Technically, it is easy to cheat the server. Therefore this is a gentleman's agreement. 2 In order to test various kind of systems, we may permit a client controls multi-players when each control module of a player is separated logically from each other in the client. 3 The unit is meaningless, because parameters used in the simulation do not be decided based on the physical parameters, but be tuned in order to make matches be meaningful as evaluation of multi-agent systems. In order to reect features of a real world, the server introduces various types of uncertainty to the simulation. The rst uncertainty is noise added to movements of objects. Noise whose size is decided according to the speed of the object is added to its movement. The second uncertainty is fuzzy-ness of parameters of commands. The server added random numbers to parameters of commands send from clients. The third uncertainty is unreliability of execution of commands. The server executes only one command for each player in a simulation cycle, but does not inform clients which command is executed. The fourth uncertainty is inaccuracy of sensing. A client can know sensor information about relative positions of other objects, in which farther and farther an object is rougher and rougher its relative position is. 3.3 Protocol All communication between the server and each client is done using by ASCII strings. The protocol of the communication consists of control commands and sensor informations: Control commands are messages sent from a client to the server to control actions of the client's player. Sensor informations are messages sent from the server to a client to inform situation of the eld from the viewpoint of the player. Description of these messages is relatively abstracted. For example, control commands consist of `turn', `dash', `kick', and `say' with a couple of parameters. This abstracted description makes users focus topics concerned with both real-world problems and multi-agent systems. If we use more physical level description, for example, power value to drive motors, users' attentions shift only to controlling actions of a player, so that features of multi-agent systems like team-play may be meaningless. If the description is more abstract-level, for example, tactical commands like `passball-to' and `cut-shoot-course', a match becomes a logical game in which situation is not so vary. In this case, the feature of real-world problem in a match becomes meaningless. Using middle-level protocols, a match becomes a test-bench that has a multi-agent factor and also variation of the real world. The server also provides a method for clients to communicates with each other. A client can inform a message to other clients using `say' command. The message is broadcast all clients immediately as an auditory information. This communication is unreliable because of unreliability of command execution described in Section 3.2. The length of messages is also restricted. Therefore, a client should communicate with other clients eectively. This means that it is dicult to control all clients in a team by a captain client. The details of these protocols are shown in Table 1 and Table 2 3.4 Rules, Referee and Coach The server controls a match according to only 3 rules: When the ball is in a goal, the server announces the goal to all clients, moves the ball to the center mark, and restarts the match. When the ball is out of the eld,
Table 1: Control Command (turn Moment) Change the direction of the player according to Moment. Moment should be 0180 180. Actual change of the direction is reduced when the player is moving fast. (dash Power) Increase the velocity of the player toward its direction according to Power. Power should be 030 100. (kick Power Direction) Kick the ball by Power to the Direction if the ball is near enough. Power should be 030 100, and Direction should be 0180 180. (say Message) Broadcast Message to all players. Message is informed immediately to clients using a (hear...) format described below. (see Time ObjInfo ObjInfo...) Table 2: Sensor Information Inform visual information. Time indicates the internal time(step-cycles of simulation). ObjInfo is information about a visible object, whose format is: (ObjName Distance Direction) ObjName ::= (player Teamname UNum) j (goal Side) j (ball) j (flag [l c r] [t b]) j (line [l c r t b]) As the distance to a player increases, more information about the player is lost. Actually, UNum is lost in the case of farther than a certain distance, and Teamname is lost in the case of very far. This message is sent 2 times per second. (The frequency may be changed.) (hear Time Direction Message) Inform auditory information. This message is sent immediately when a client sends (say Message) command. Direction is the direction of the sender. Time indicates the current time. Judgements of the referee is also informed using this form. In this case, Direction is `referee'.
the server moves the ball to a suitable restart point and keep defending players to be 9.15 meters apart from the ball. When the rst and second halves end, the server announce `time-up' and suspend the match. Rules about usage of hands is ignored because players have no hands. Therefore, there are no goalies. The o-side rule is also ignored because it will make a match low-scored. `Obstruction' and `pushing' are judged by a human referee. These fouls concerned with player's intention that is dicult to judge automatically. The human referee can take these fouls by clicking a player in the eld window by a mouse. Then the server moves the ball to the player's position, and restarts a match by a free-kick of the opponent team. When the server is invoked with a -coach option, the server can connect with a special client (called a coach client) who can control a match instead of the referee module in the server. The coach client can move all objects, start and stop simulation, and announce a message to all clients. This facility is useful for tuning and debugging client programs, which require testing behaviors of clients many times in a certain situation. For example, experiments described below were done using this. 3.5 Implementation and Access Point Soccer Server is implemented using g++ and X-window system with Athena widget-set. Currently, we support SunOS 4.1.x, Solaris 2, DEC OSF/1, NEWS OS and Linux. we are planning to support other OSs and architectures. Sources of Soccer Server are available by FTP: ftp://ci.etl.go.jp/pub/soccer/server/sserver-* Manuals and sample clients are also available from the following homepage: http://ci.etl.go.jp/~noda/soccer/server.html 4 Research Issues Soccer Server is developed in order to provide a common test-bench to evaluate AI systems. As discussed in Section 2, soccer includes various research issues of AI. In this section, we will discuss such issues from various points of view. 4.1 Real World Problem Unlike Chess, soccer includes various uncertainty of real world. For example, when a player kick a ball to a direction, the ball sometimes goes to a dierent directions. Moreover, the ball may happen to changes its direction because of irregular bound. In the case of sensing environment, human has limited view so that he/she can not recognize whole things in the eld. In a match carried out by Soccer Server, each client also must deal with various kind of uncertainty in controlling its player eectively. In order to make simulated environment similar to one of the real world, we introduced random factors in players' action and restricted its performance in sensing. Therefore, designers of clients must consider about how to process such low-reliable information. For example, the designers may deal with the following problems. Unreliability in Action Control commands sent by clients are not always executed. The commands may be lost in the network or may not be accepted by the server because of timing of sending the message 4. Moreover, executions of the commands are not accurate. For example, the server adds noise to movements of objects. Therefore the ball may roll on toward an unexpected direction when a client sends a kick command. Movements of players are also unreliable. `(turn 90)' does not mean that the player turns right. It means that the player changes direction in a certain angle that is decided by the Power (= 90), his velocity and noise factor. Therefore clients should have mechanisms to conrm eects of commands sent by itself. Limited Sensor Information Each client can get limited visual information because of narrow view (only 90 degrees). Therefore, in order to get complete information of the eld, the client rotates its player slowly 5 using the turn command and integrates visual informations sent from the server. In this case, uncertainty of actions becomes a problem. Because turn commands are not executed exactly, the client can not integrate visual informations before and after a turn command simply. Moreover, information about objects in sight is lost when distances to the objects are long. For example, the client gets the team-name and the uniform number of a player in sight if the player is near enough. If the player is distant the client can not get the uniform number, and if the player is very far, the client cannot get even the team-name. Therefore, in order to know such information about players that are very far, the client must infers it from a past history of information sent from the server. Realtime Processing Unlike in Chess, the situation is changing dynamically in a match of soccer. In Soccer Server, simulation of objects' movements and communication with clients are executed independently. Therefore, if a client take a long time for deep planning, the situation will change so that the planning will not be eective. On the other hand, if clients control players reactively without planning, the team will not be able to perform team-plays. Therefore, clients must 4 In the current version, the server accepts only the last message on the message cue of each socket at the timing when the server checks the socket. 5 If the client rotates its player quickly, it can not get complete information because interval of sending visual information is set relatively long.
Figure 1: Window Image of Soccer Server Soccer Server Message Board Referee Field Simulator Network (UDP/IP) X window Network (UDP/IP) Figure 2: Close-up of Players and a Ball Figure 3: Overview of Soccer Server
Figure 5: One Side Cut be designed to use both of reactive control and deep planning eectively. 4.2 Cooperation in Multi-Agent Environment Because soccer is a multi-agents game, team-plays, that is cooperations with team-mates, are important factors to decide the ability of teams. For example, the following team-plays become elementary problems. Pass A pass is the most basic team-play. In order to pass a ball among players, the players must have a consensus about pass-courses. Even in the case of pass between two players it is not easy to reach a consensus. For example, consider a situation like Figure 4-(a). In this case, we can consider two passplans: `one-two-pass'(figure 4-(b)) and `throughpass'(figure 4-(c)). In order to break the defense, players of the oense must decide which of the two pass-plans they adopt. One-Side Cut `One-side cut' is a basic combination play in defense. In this play, a defender nearest to an oense keeping a ball chases the oense from left or right side, while other defenders cover another side (Figure 5). This play involves the same problem as the `pass', that is, how to reach a consensus on choosing a side to cut. Coaching As mentioned above, each client can get limited information about the eld. Communicating and sharing such informations among clients is a solution to overcome the problem. In human soccer, such communication are called as `coaching'. The coaching in the soccer server is not so simple. The server provides a way of communication, the say command and the hear information, but its capacity is restricted 6. Therefore the communication 6 The server restricts the frequency of acceptance of say should be eective. 4.3 Learning Learning under dynamic multi-agent environments is also an important research topic in soccer. Conventional machine learning methods are developed for single learner in a static environment. Therefore, it is dicult to apply them to dynamic multi-agent environment like soccer. Especially, representation of time will become an important topic of the research. Moreover, co-learning is also an important research topic. In a multi-agent environment, result of learning of a learner aects learning of another learner. Therefore, process of learning is not straightforward. This is a quite new topic of machine learning. 5 Conclusion In this paper, we have reported the Soccer Server and the research issue of AI using soccer. Soccer Server provides a virtual eld in which players controlled by client programs play a soccer match. The server is a useful tool of researches on multi-agent systems. We hope by using the server on RoboCup the several new techniques for multi-agent systems will be investigated. Acknowledgements The authors thank to Hiroaki Kitano (Sony CSL), Minoru Asada (Osaka Univ.), Yasuo Kuniyoshi (ETL), Peter Stone (CMU), Milind Tambe (USC) and Eiichi Osawa (Sony CSL) for their valuable suggestions to Soccer Server. The authors also thank to users of Soccer Server who send various comments and reports about bugs and implementation. References [Asada et al., 1995] Minoru Asada, Shoichi Noda, and Koh Hosoda. Non-physical intervention in robot learning based on lfe method. In Proc. of Machine Learning Conference Workshop on Learning from Examples vs. Programming by Demonstration, 1995. [Sahota, 1993] Mickael K. Sahota. Real-time intelligent behaviour in dynamic environments: Soccer-playing robots. matster thesis, Department of Computer Science, The University of British Columbia, Aug. 1993. [Sahota, 1994] M. K. Sahota. Reactive delivation: an architecture for real-time intelligent control in dynamic environments. In Proc. of AAAI-94, pages 1303{1308, 1994. [Stone and Veloso, 1995] P. Stone and M. Veloso. Learning to pass or shoot: collaborate or do it yourself, 1995. unpublished manuscript. commands.
(b) (c) (a) Figure 4: Pass