PARALLEL AND DISTRIBUTED PLATFORM FOR PLUG-AND-PLAY AGENT-BASED SIMULATIONS. Wentong CAI

Transcription

1 PARALLEL AND DISTRIBUTED PLATFORM FOR PLUG-AND-PLAY AGENT-BASED SIMULATIONS Wentong CAI Parallel & Distributed Computing Centre School of Computer Engineering Nanyang Technological University Singapore

2 Outline Agent-based Modelling and Simulation (ABMS) Computer Simulation Complex Adaptive System & Agent-based Modelling and Simulation Our Work on Crowd Simulation & Traffic Simulation Calibration of ABMS Motivation Evolution-based Approach Cloud-based Execution Speed-up and Scale-up ABMS Top-down vs. bottom-up decomposition Cluster-based partitioning (crowd simulation) Adaptive graph partitioning (traffic simulation) Federated simulation Challenges 2

3 Computer Simulation Simulation imitates the operations (behavior) of a real-world system or process over time, usually via computer programs, to evaluate and improve system performance. Simulation involves the generation of an artificial history of a system and the observation of that artificial history to draw inferences concerning the operating properties of the real system. analyze systems before they are built analyze operational systems create virtual environments for training, entertainment 3

4 Complex Adaptive Systems Complex Adaptive Systems are characterized by apparently complex behaviours that emerge as a result of often nonlinear spatial-temporal interactions among a large number of component systems at different levels of organization. decentralized decision making local-global interaction, selforganization, emergence heterogeneity 4

5 Agent-based Modelling & Simulation An Intelligent Agent is a computer system capable of flexible autonomous actions in some environment [Woodbridge]. reactive: deal with changes pro-active: goal directed behavior social: interact with other agents An Agent-based Model is an abstraction of a real or physical system described in terms of multiple intelligent agents. 5

6 Key Elements of ABM Autonomous decision-making How they can make decision vary Local interactions Behaviour Modelling No central authority exists Neighbourhoods (or environments) ABMS consist of a space, framework or environment in which interactions take place and a number of agents whose behaviour is defined in this space is defined by a basic set of rules and by characteristic parameters. [Reynolds] 6

7 ABMS Toolkits NetLogo Toolkits that support implementation and execution of agent-based models tools for environment modelling (e.g., spatial indexing) simulation engine (e.g., scheduler) supporting libraries (e.g., random number generator) MASON RePast 7

8 Crowd Modelling and Simulation Usefulness: planning, training, and operational decision making Importance: riot control, disaster management, emergency evacuation, and rescue operations 8

9 Navigation & Path Planning 9

10 Goal Oriented Agents 10

11 Large-scale Nanoscopic Traffic Simulation TUM CREATE Centre for Electromobility, Singapore 12

12 Outline Agent-based Modelling and Simulation (ABMS) Computer Simulation Complex Adaptive System & Agent-based Modelling and Simulation Our Work on Crowd Simulation & Traffic Simulation Calibration of ABMS Motivation Evolution-based Approach Cloud-based Execution Speed-up and Scale-up ABMS Top-down vs. bottom-up decomposition Cluster-based partitioning (crowd simulation) Adaptive graph partitioning (traffic simulation) Federated simulation Challenges 13

13 Motivation & Objective Parameter settings are very important for crowd simulation e.g., social force model Given a desired crowd scenario, finding the proper settings to reproduce the scenario is difficult and tedious Large search space Noise The objective of the study is to develop an evolutionary algorithm to automate the proper parameter configuration process Challenge similarity measure 14

14 Density-based Matching Scheme for Fitness Evaluation Z Axis * Z Axis Z Axis X Axis Y Axis X Axis Y Axis X Axis Y Axis weight matrix Reproduced behavior desired behavior T grid_ num 0 1 i i i i f w 15

15 Evolving Model Parameters initial simulation models simulation models Core Engine Module model execution Evolution Computing Module evaluate and generate new simulation models Evolve simulation model parameters to match desired crowd scenario multiple runs for each multiple simulation runs for model each simulation model 16

16 Cloud-based Execution initial simulation models Core Engine Module model execution multiple runs for each simulation model simulation models IaaS/PaaS multiple runs for each simulation model Evolution Computing Module evaluate and generate new simulation models multiple runs for each simulation model 18

17 Outline Agent-based Modelling and Simulation (ABMS) Computer Simulation Complex Adaptive System & Agent-based Modelling and Simulation Our Work on Crowd Simulation & Traffic Simulation Calibration of ABMS Motivation Evolution-based Approach Cloud-based Execution Speed-up and Scale-up ABMS Top-down vs. bottom-up decomposition Cluster-based partitioning (crowd simulation) Adaptive graph partitioning (traffic simulation) Federated simulation Challenges 19

18 Top-down vs. Bottom-up middleware to support federated simulation 20

19 Cluster Based Partitioning Partitioning is not a unique problem to crowds or agent simulation. Load balancing (distributing computational load among computers) is a long studied problem in many research fields. The work we do is obviously also applicable to Distributed Virtual Environments and MMORPGs. We replicate the virtual environment on each CPU and then divide the agents among them. Our approach is to use clustering techniques from data mining in order to identify groups of agents and then assign these groups to compute nodes. Use a k-means approach to group agents close together Grid-based clustering 21

20 Cluster Based Partitioning: K-Means K-means clusters objects into k clusters on some attributes which form a vector space. In our case, the agents positions are chosen as the attribute. Agents which are close to one another need to know about one another so should be in same cluster. Floyd s algorithm is used to minimize J J j 1 i 1 p i c j where: c j is the center of each cluster and p i is the position of an agent Each cluster is mapped to a compute node. Agents will be transferred between nodes if they move to another cluster. K-means executes at predefined interval periodically. k n 2 22

21 Cluster Based Partitioning: K-Means Random_normal: agents randomly walk in the square Random_emergency: agents evacuate via the exit individually Group_normal: agents are formed into groups and walk randomly in the square Group_emergency: agents are formed into groups and evacuate via exit Static Fast Adaptive Load Balancing K-means 23

22 Cluster Based Partitioning: K-Means K-means works well for some cases To maintain the clusters one must migrate agents between machines. This involves sending the agent s state across the network So in some cases the time spent moving agents between machines will negate any gain achieved from reducing communication! Consider the crossing example Can maintain clustering and migrate agents. Perhaps a better option, just ignore ideal clustering while the two groups cross. Pay communication expense during crossing. In addition to location, use agents goals as another metric of similarity -> Grid-based Partitioning 24

23 Grid Based We define a uniform grid mesh over the environment space, each area of the grid is a cell. Each cell contains some number of agents. We can control execution time of algorithm with grid resolution. A set of statistics is collected for each cell For case study: Average Goal, Goal Variance The clustering algorithm then groups cells into clusters based on the difference of these statistics. Note that cells with high goal variance (many agents with different goals) are never clustered. These may be added to clusters later. Bounding box (shared among nodes along with statistics of clusters) is defined around cluster for determining when communication is necessary. Agents outside cluster are simulated as individuals on a specially dedicated compute node. 25

24 Grid Based We now have some collection of clusters each of which contains agents which are nearby each other and have similar statistics (goal). Next, we evenly assign clusters and individuals to partitions. The number of clusters C is determined by the algorithm. The number of partitions P is defined by number of machines we have. Clusters may need to be split if necessary. We actually have a parallel implementation of this algorithm. Clusters are formed (centrally) once at startup. Each partition then manages its own clusters and collaboratively performs the clustering algorithm with other partitions. Load-balancing is performed by splitting a large cluster into smaller ones transferring clusters between partitions. 26

25 Performance Sparse Groups Close Groups 27

26 Performance 28

27 Partitioning of Traffic Simulation P3 P2 P4 P1 29

28 Federated Simulation CO2 Emission Simulation Weather Simulation data logging & Control HLA/RTI Traffic Simulation Crowd Simulation Epidemic Simulation Simulation of Urban Dynamic 30

29 Service-oriented HLA/RTI Dynamic deployment of service components & decentralized services -> flexibility and scalability Client Federate Client Federate Shared services and communication overheads -> adaptive algorithms Client Federate 31

30 Vision Plug & Play Simulation RTI RTI RTI federate Model Factory federate Model Factory RTI RTI Discovery of Models Discovery of Resources Management of Simulation Execution 32

31 Outline Agent-based Modelling and Simulation (ABMS) Computer Simulation Complex Adaptive System & Agent-based Modelling and Simulation Our Work on Crowd Simulation & Traffic Simulation Calibration of ABMS Motivation Evolution-based Approach Cloud-based Execution Speed-up and Scale-up ABMS Top-down vs. bottom-up decomposition Cluster-based partitioning (crowd simulation) Adaptive graph partitioning (traffic simulation) Federated simulation Challenges 33

32 Challenges Verification & Validation Lack of data, predict past vs. predict the future Relying on SMEs (past experience)? Real-time Model Adaptation How to make sense from data to adapt agent behaviour? How to do this in real-time? Interoperability and composability Different types of simulations Different level of abstraction Scalability Ways to support execution of large models How to meet real-time requirement when the model is scaled up? 34

33 Questions? 35