Re-routing Agents in an Abstract Traffic Scenario

Transcription

1 Re-routing Agents in an Abstract Traffic Scenario Ana L.C. Bazzan and Franziska Klügl 1 Instituto do Informatica, UFRGS Porto Alegre, Brazil bazzan@inf.ufrgs.br 2 Dep. of Artificial Intelligence, University of Würzburg Würzburg, Germany kluegl@informatik.uni-wuerzburg.de Abstract. Human drivers may perform replanning when facing traffic jams or when informed that there are expected delays on their planned routes. In this paper, we address the effects of drivers re-routing, an issue that has been ignored so far. We tackle re-routing scenarios, also considering traffic lights that are adaptive, in order to test whether such a form of co-adaptation may result in interferences or positive cumulative effects. An abstract route choice scenario is used which resembles many features of real world networks. Results of our experiments show that re-routing indeed pays off from a global perspective as the overall load of the network is balanced. Besides, re-routing is useful to compensate an eventual lack of adaptivity regarding traffic management. 1 Introduction In traffic, human drivers especially when equipped with a navigation system do re-route when they are informed that a jam is on their planned route. In simple two-route scenarios it makes little sense to study re-routing. These scenarios are only adequate to study chaotic traffic patterns with drivers changing their routes in periodic ways, like in [3,12]. Scenarios in which agents can change their routes on the fly are just beginning to be investigated. It is unclear what happens when drivers can adapt to traffic patterns in complex traffic networks and, at the same time, there is a traffic authority controlling the traffic lights, trying to cope with congestions at local levels or at the level of the network as a whole. New questions are just arising such as: What happens if impatient drivers do change route when they are sitting too long in red lights? How can a traffic authority react to this behavior? In this paper we discuss some issues related to on the fly re-routing, aiming at providing some answers to those questions. To this aim we use a simulation scenario already addressed [2,4]. The goal in these previous works was to investigate what happens when different actors adapt, each having its own goal. For instance, the objective of local traffic control is obviously to find a control G. Zaverucha and A. Loureiro da Costa (Eds.): SBIA 2008, LNAI 5249, pp , c Springer-Verlag Berlin Heidelberg 2008

2 64 A.L.C. Bazzan and F. Klügl scheme that minimizes queues. On the other hand, drivers normally try to minimize their individual travel time. Finally, from the point of view of the whole system, the goal is to ensure reasonable travel times for all users, which can be highly conflicting with some individual utilities. This is a well-known issue: Tumer and Wolpert [11] have shown that there is no general approach to deal with this complex question of collectives. As mentioned, we now return to that scenario adding a new feature regarding drivers, namely that they can replan their routes on the fly. In the next section we review these and related issues. In Section 3 we describe the approach and the scenario. Results are shown and discussed in Section 4, while Section 5 presents the concluding remarks. 2 Agent-Based Traffic Simulation and Route Choice Route choice simulation is an essential step in traffic simulation as it aims at assigning travel demand to particular links. The input consists of a matrix that expresses the estimated demand (how many persons/vehicles want to travel from an origin to a destination); of a network for which relevant information of nodes and links are given such as maximum speed and capacity; and of a function that relates density and speed/travel time. The output is an assignment of each traveler to a route. Thus, route choice is a traveler-focused view to the assignment step of the traditional four-step process of traffic simulation [8]. Traditionally, route choice simulations have been done as discrete choice models based on stochastic user equilibrium concepts. On the other hand, in agent-based traffic simulation there are basically two goals behind route choice simulation models: integration of simple route choice heuristics into some larger context, and the analysis of self-organization effects of route choice with traffic information. In large scale traffic simulations agents are capable of more than just solving one phase of the overall travel simulation problem: they plan their activities, select the locations and departure times, and connect activities by selecting routes to the respective destinations. A prominent agent-based example is MATSim [1]. Here route choice is tackled using some shortest path algorithm. In [6] a software architecture that may support driver adaptivity is described. Agent-based route choice simulation has been intensively applied to research concerning the effects of intelligent traveler information systems. What happens to the overall load distribution, if a certain share of informed drivers adapt? What kind of information is best? These are frequent questions addressed by agent-based approaches. In these approaches, route choice simulation has been based on game-theory, e.g. minority games. Examples for such research line, which includes adaptive and learning agents, can be found in [3] for the Braess paradox, in [7] for an abstract two-route scenario, or in [5] where a complex neural net-based agent model for route choice is presented regarding a three route scenario. In [9], a simple network for fuzzy-rule based routing (including qualitative decisions) is used.

3 Re-routing Agents in an Abstract Traffic Scenario 65 One problem with these approaches is that their application in networks with more than two routes between two locations is not trivial. The first problem is that, to model the route choice simulation as an agent choice problem, an option set consisting of reasonable alternatives has to be generated. The problem of generation of routes for route choice models is well known in traditional discrete choice approaches. Different solutions have been suggested. Using the n shortest paths is often a pragmatic solution, yet it yields routes that differ only marginally. Additionally, all approaches, including all agent-based ones, consider one route as one complete option to choose. On-the-fly re-routing during the simulation has hardly been a topic for research. We suspect this happens for two reasons. The first is that the set of routes corresponds to the set of strategies or actions that an agent may select for e.g. game-theoretic analysis. Agents learn to optimize the selection of an action or a route but are not able to modify the set of known routes as this would mean creating a new strategy on the fly. The second reason is of practical nature. Travel demand generation, route choice, and driving simulation have been tackled as subsequent steps for which different software packages are used. Often, there is no technical possibility to allow drivers to do re-routing from arbitrary positions in the network using different software packages. Even sophisticated agent architectures like that proposed in [10] lack the technical basis for re-routing during the actual traffic simulation of driving. Therefore the study of the effects of drivers re-routing is still an open question. 3 Approach and Scenario In order to address a non trivial network, we use a grid with 36 nodes, as depicted in Figure 1. All links are one-way and drivers can turn to two directions in each crossing. Although it is apparently simple, from the point of view of route choice and equilibrium computation, it is a complex one as the number of possible routes between two locations is high. In contrast to simple two-route scenarios, it is possible to set arbitrary origins (O) and destinations (D) in this grid. For every driver agent, its origin and destination are randomly selected according to probabilities given for the links: to A B C D E F Fig. 1. 6x6 grid showing the main destination (E4E5), the three main origins (B5B4, E1D1, C2B2), and the main street (darker line)

4 66 A.L.C. Bazzan and F. Klügl render the scenario more realistic, neither the distribution of O-D combinations, nor the capacity of links is homogeneous. On average, 60% of the road users have the same destination, namely the link labeled as E4E5 which can be thought as something like a main business area. Other links have, each, 0.7% probability of being a destination. Origins are nearly equally distributed in the grid, with three exceptions (three main residential areas ): links B5B4, E1D1, and C2B2 have, approximately, probabilities 3, 4, and 5% of being an origin respectively. The remaining links have each a probability of 1.5%. Regarding capacity, all links can hold up to 15 vehicles, except those located in the so called main street. These can hold up to 45 (one can think it has more lanes). This main street is formed by the links between nodes B3 to E3, E4, and E5. The control is performed via decentralized traffic lights. These are located in each node. Each of the traffic lights has a signal plan which, by default, divides the overall cycle time in the experiments 40 time steps 50-50% between the two phases. One phase corresponds to assigning green to one direction, either north/south or east/west. The actions of the traffic lights are to run the default plan or to prioritize one phase. Hence these particular strategies are fixed (always run the default signal plan) and greedy (to allow more green time for the direction with higher current occupancy). Regarding the demand, the main actor is the simulated driver. In the experiments we have used 700 driver agents because this corresponds to a high occupancy of the network (72%) and proved a difficult case in a previous study [2]. This simple scenario goes far beyond simple two-route or binary choice scenario; we deal with route choice in a network with a dozens of possible routes. Additionally, the scenario captures properties of real-world scenarios, like interdependence of routes with shared links and heterogeneous capacities and demand throughout the complete network. Every driver is assigned to a randomly selected origin-destination pair. The initial route is generated using a shortest path algorithm (from his origin to his destination), considering free flow. This means that the only route it knows in the beginning is the shortest path assuming no congestion. After generating this initial route, the driver moves through the network. Every link is modeled as a FIFO queue. The occupancy of a link is the load divided by the capacity. It is used as cost or weight for some alternatives to generate the shortest path (see below). The lower the occupancy the lower the cost so that in a weighted shortest path generation, paths with lower occupancy are preferred. In every time step drivers behind the two first agents in the queue are able to perceive the traffic situation regarding the link that they are suppose to take next. If the occupation of this next link is higher than a threshold τ, the agent may trigger a replanning step. Replanning involves the generation of a new route on the fly. This is done using shortest path algorithms with different flavors, from the link where the drivers wishes to replan to his destination. These flavors are summarized below: UC ( uniform cost ): Every link between the current position and the destination has the same cost, except that the next link in the agent s plan is

5 Re-routing Agents in an Abstract Traffic Scenario 67 weighted badly to force it to avoid this link (after all, they should replan, not use the same old plan); however, if no other path is possible, the next link will be taken. OCENL (occupancy-based, except next link): This second alternative differs from the first only by the costs that are used for computing the shortest path. Here, the current occupancy is used as cost for all links except for the next one which has a very high value so that it will be avoided if possible. This corresponds to a situation where agents have complete information about the current network status. OCAL (occupancy-based for all links): In this third alternative the current occupancy of the next link is used as cost. 4 Experiments and Results We have conducted several experiments using the alternative ways described in the previous section to re-generate routes on the fly. With these experiments the following questions can be addressed: In which context adaptation should be triggered? When should the driver re-evaluate its decision? What information is useful for finding the new route? How this form of driver agent adaption interferes with other agents adaptation (e.g. with intelligent traffic lights)? In the following, we will discuss the cases in which traffic lights do not adapt (Sections 4.1 and 4.2), as well as cases where lights do adapt (Section 4.3). We also notice that these simulations run in a time frame of minutes. 4.1 Re-routing with Uniform Costs In this experiment we have varied the re-planning threshold τ. If the perceived occupancy on the next link is higher than this threshold, re-routing is triggered. The procedure for computing the remaining route is the same as for computing 500 Mean traveltime of all drivers depending on threshold tau with uniform cost re-routing 400 traveltime ,5 0,6 0,7 0,8 0,9 1 tau Fig. 2. Mean travel time over all drivers, varying the threshold for re-routing (τ); comparison of UC situation (circles) and no re-routing (dash)

6 68 A.L.C. Bazzan and F. Klügl Travel times of four randomly selected drivers 500 Traveltime drv0 drv105 drv109 drv Iteration Fig. 3. Travel time for four randomly selected drivers. In rounds that are marked with small circles the driver re-planned (driver drv128 re-plans in every round). the initial route, namely shortest path assuming the same cost for every link. Figure 2 depicts the mean travel times over all drivers (mean of 20 runs with standard deviation between runs). In this figure, one can clearly identify that re-planning (even a quite basic one) is advantageous for the overall system. The distribution of load is improved when agents can avoid links with high occupancies. Interestingly, τ plays almost no role, except when τ = 1 because in this case drivers re-plan only if the next link is completely occupied, which is of course an extreme situation. The fact that we observe no significant change in travel times with changing τ can be explained by the fact that occupancies are generally high due to the high number of agents in the network. A threshold lower or equal than 0.5 seems to be not as good as the higher ones as it triggers re-planning too often. Having a deeper look into the dynamics of this basic simulation, it turns out that this overall improvement does not come at expense of only a few individuals. In an example simulation run with τ =0.8, in every iteration around half of the agents (371 ± 12) re-plan at least once. For the individual agents that replan, this may cause a quite severe individual cost as depicted in Figure 3 for four randomly selected agents. We marked in this graph every data point where the particular agent did some re-planning, except for the agent drv128 that did replan in every round. The question arises whether using other forms of information for computing the deviation route produce better results. 4.2 Re-routing Based on Occupancy Instead of using arbitrary but uniform costs, drivers in this experiment use the current occupancy of the links in the network to compute the remaining route. To this aim, they have information about the occupancy of each link. Figure 4 depicts the results for both, the OCENL and the OCAL cases (mean of 20 runs depicted with standard deviation between runs). These results show that allowing the drivers to access the current occupancies to compute the shortest path to their destination produces overall better mean

7 Re-routing Agents in an Abstract Traffic Scenario 69 Mean traveltime of all drivers depending on threshold tau with occupancy-based re-routing traveltime ,5 0,6 0,7 0,8 0,9 1 tau Fig. 4. Mean travel time when agents calculate the remaining of the route based on occupancy of links; comparison between three situations: no re-routing (dashes), OCENL (triangles), and OCAL (squares) travel time than without online route adaptation. However, compared to the results observed in the UC situation, there is hardly any improvement, except for large values of τ. In the OCAL the mean travel time is worse than OCENL for all values of τ. Taking standard deviations into consideration, there is even no significant difference from the corresponding situation where drivers cannot re-plan if they encounter a jam during their travel. This is due to the fact that the tradeoff between taking a deviation and the original route seems to be not high enough. Reasons for this might be that we use occupancy as link weight instead of some travel time forecast for computing the route from the current position. Nevertheless, the next steps in our research agenda concern more intelligent forms of planning and re-planning, as discussed in Section 5. Because there seems to be no clear difference between UC and OCENL, we left re-routing techniques based on individually experienced travel times to future work as we expect no significant improvement in comparison to both techniques that we have examined here. 4.3 Re-routing Drivers in Adaptive Context Adaptive and learning traffic light agents are able to improve the overall traffic flow. The main objective of this paper is to test whether adapting drivers can provide more improvement or if there are hindering interferences. To this aim, we have performed simulation experiments similar to the ones shown above, this time with adaptive traffic lights. As mentioned above, we selected a reactive form of adaptation called greedy. The traffic light immediately increases the green time share for directions with higher occupancy. In our previous work, we showed in the same artificial scenario that we are using here, that this form of adaptivity produces at least similar results than other more elaborate forms

8 70 A.L.C. Bazzan and F. Klügl Table 1. Mean travel time (mean and standard deviation of 20 runs) for greedy traffic lights and drivers re-routing using UC and OCAL τ UC OCAL ± ± ± ± ± ± ± ± ± ± 52 of learning (e.g. Q-learning or learning automata). As greedy traffic lights use the same information for immediate adaptation as the drivers described here for re-routing, we expect co-adaption effects to play a significant role. The results (in terms of travel times) are given in Table 1 and in Figure 5. In Table 1, traffic lights use a greedy strategy and drivers use either the uniform costs (UC) or occupancy based re-routing (OCAL). In Figure 5, we compare the mean travel times for traffic lights with fixed and greedy strategies, while drivers re-route using the OCAL strategy. For sake of comparison, we notice that when drivers do not replan (i.e. always use their shortest path route computed at the beginning) and traffic lights act greedily, the overall mean travel time is 260 ± 45; thus it is higher than when drivers do re-plan indicating a positive co-adaption effect. Analyzing Table 1, one can see that the lowest travel time is different for UC and for OCAL. Whereas the travel time under UC is best around τ =0.7, OCAL rerouting is best when τ =0.9. Both are very similar for situations when τ is low. For higher thresholds, OCAL becomes better (although not significantly). The reason might be that the OCAL re-routing uses the current information on occupancy. However when one link has high occupancy, it is very likely that nearby links also Mean traveltime of all drivers and fixed and greedy traffic lights 400 traveltime ,5 0,6 0,7 0,8 0,9 1 tau Fig. 5. Mean Travel time (mean of 20 runs with standard deviation) depending on the re-routing threshold τ, for fixed and greedy adaptive traffic lights

9 Re-routing Agents in an Abstract Traffic Scenario 71 have high occupancy thus the driver has really not so much room to replan, but can use this room more intelligently when informed about it. Finally, we can state that driver agents that are able to adapt their route on the fly as a reaction to traffic conditions may at least compensate an eventual lack of adaptivity by the traffic lights. If the drivers use more up-to-date information for generating their remaining routes, the situation improves slightly more. However, our experiments show that in the current driver re-routing model, the actual value of the threshold τ does not matter much except for the extreme case of τ = 1. Therefore, a further elaboration of the agent decision making model, turning it more realistic, may be helpful. This may show additional positive effects of adaptation in traffic scenarios thus permitting to really evaluate the effects of adaptive and learning traffic lights in realistic scenarios. 5 Conclusions and Future Work In this paper we have analyzed the effects of drivers computing new routes on the fly. In most previous simulation scenarios route adaptation was only allowed before and after the actual driving. However, en-route modifications cannot be ignored in a realistic simulation of intelligent human decision making in traffic. In an artificial scenario that resembles all potentially problematic features of realworld scenarios, we have tested the effects of re-routing with results indicating that the overall system performance depends on different facets of the agents rerouting: on the criteria drivers use to decide whether to deviate from the originally planned route, and on the procedure they use to compute a new route. We have shown that re-routing may compensate less efficient traffic management and still increase the overall performance if traffic lights adapt to the traffic situation. We are far from modeling intelligent human decision making about routing or even way-finding in traffic situations. However, this would be necessary for actually being able to evaluate re-routing effects in traffic networks. Our next steps will therefore comprise more intelligent ways regarding drivers perceiving that there is jam ahead and deciding whether to trigger a re-routing procedure. A simple extension is the integration of the status of other perceivable links into the decision. Additionally we want to analyze the effects of an experience-based computation of the alternative route: a driver memorizes travel times that it has already experienced and uses this information to predict cost on the links that will potentially be contained in its deviation route. Also, we want to depart from the assumption that the complete network is known. Thus an important step in our research program is to use mental maps, thus moving from abstract game-theory based models to real-world application. Acknowledgments We thank CAPES and DAAD for their support to the bilateral project Large Scale Agent-based Traffic Simulation for Predicting Traffic Conditions. Ana Bazzan is/was partially supported by CNPq and Alexander von Humboldt Stiftung.

10 72 A.L.C. Bazzan and F. Klügl References 1. Balmer, M., Cetin, N., Nagel, K., Raney, B.: Towards truly agent-based traffic and mobility simulations. In: Jennings, N., Sierra, C., Sonenberg, L., Tambe, M. (eds.) Proceedings of the 3rd International Joint Conference on Autonomous Agents and Multi Agent Systems, AAMAS, New York, USA, July 2004, vol. 1, pp IEEE Computer Society, New York (2004) 2. Bazzan, A.L.C., de Oliveira, D., Klügl, F., Nagel, K.: Adapt or not to adapt consequences of adapting driver and traffic light agents. In: Tuyls, K., Nowe, A., Guessoum, Z., Kudenko, D. (eds.) ALAMAS 2005, ALAMAS 2006, and ALAMAS LNCS (LNAI), vol. 4865, pp Springer, Heidelberg (2008) 3. Bazzan, A.L.C., Klügl, F.: Case studies on the Braess paradox: simulating route recommendation and learning in abstract and microscopic models. Transportation Research C 13(4), (2005) 4. Bazzan, A.L.C., Klügl, F., Nagel, K.: Adaptation in games with many co-evolving agents. In: Neves, J., Santos, M.F., Machado, J.M. (eds.) EPIA LNCS (LNAI), vol. 4874, pp Springer, Heidelberg (2007) 5. Dia, H., Panwai, S.: Modelling drivers compliance and route choice behaviour in response to travel information. Special issue on Modelling and Control of Intelligent Transportation Systems, Journal of Nonlinear Dynamics 49(4), (2007) 6. Illenberger, J., Flötteröd, G., Nagel, K.: Enhancing matsim with capabilities of within-day re-planning. Technical Report 07 09, VSP Working Paper, TU Berlin, Verkehrssystemplanung und Verkehrstelematik (2007) 7. Klügl, F., Bazzan, A.L.C.: Route decision behaviour in a commuting scenario. Journal of Artificial Societies and Social Simulation 7(1) (2004) 8. Ortúzar, J., Willumsen, L.G.: Modelling Transport, 3rd edn. John Wiley & Sons, Chichester (2001) 9. Peeta, S., Yu, J.W.: A hybrid model for driver route choice incorporating en-route attributes and real-time information effects. Networks and Spatial Economics 5, (2005) 10. Rossetti, R., Liu, R.: A dynamic network simulation model based on multi-agent systems. In: Applications of Agent Technology in Traffic and Transportation, pp Birkhäser (2005) 11. Tumer,K.,Wolpert,D.:Asurveyofcollectives.In:Tumer,K.,Wolpert,D.(eds.) Collectives and the Design of Complex Systems, pp Springer, Heidelberg (2004) 12. Wahle, J., Bazzan, A.L.C., Klügl, F., Schreckenberg, M.: Decision dynamics in a traffic scenario. Physica A 287(3 4), (2000)