Coordination and Control in Multi-Agent Systems

Transcription

1 Coordination and Control in Multi-Agent Systems Prof. dr. Ming Cao Institute of Engineering and Technology University of Groningen The Netherlands

2 Outline What are agents and multi-agent systems? Mobile robots and their formations Rigidity graph theory Localization and formation control Dealing with inconsistent measurements Coordination through strategic interactions To cooperate or defect in group tasks Evolutionary games on networks Towards optimal control of evolutionary games

3 Motivating Example: Ocean Sampling Oceanographic study in a 20km-by-40km region northwest of Monterey Bay, California, USA, August 2006 Princeton Glider Coordinated Control System (GCCS) 3

4 Field test

5 Princeton GCCS Ocean Models MBARI Server Web Interface/ ASAP Virtual Control Room Fully Automated Closed Loop 5

6 Control of multi-agent systems Enabled by key technological breakthrough Sensing Actuation Communication Computation Driven by growing wide engineering applications Mobile autonomous sensor networks Smart energy systems Autonomous robotic teams Emerging trend in the theoretical study of complex networks One effective way of testing the understanding of the underlying mechanism of complex behavior of social or biological systems is to effectively control them Introducing extremists in opinion forming in social networks Introducing robotic members to fish schools or bird flocks

7 Outline What are agents and multi-agent systems? Mobile robots and their formations Rigidity graph theory Localization and formation control Dealing with inconsistent measurements Coordination through strategic interactions To cooperate or defect in group tasks Evolutionary games on networks Towards optimal control of evolutionary games

8 Formation Maintenance with Distance Constraints The problem: Maintaining a formation in 2D Autonomous agent : Point in the plane

9 Formation Maintenance with Distance Constraints The problem: Maintaining a formation in 2D Autonomous agent : Point in the plane Maintenance links

10 Formation Maintenance with Distance Constraints The problem: Maintaining a formation in 2D Autonomous agent : Point in the plane Maintenance links when each link is maintained by both agents (an undirected graph), it is necessary for the formation to be rigid (Eren et al, 2002)

11 Four Bar Chain Triangulated Structure

12 Framework Universal (ball) Joint V: vertices Rigid Rod (bar) E: edges Framework: F= ( G, p) p n : V R embedding G = ( V, E) graph

13 Rigid Formations A formation is a collection of agents (point agents) in two or three dimensional space F= ( V, E, p( t)) A formation is rigid if the distance between each pair of agents does not change over time if its distance constraints are satisfied 2 p ( t) p ( t)) ( p ( t) p ( t)) = c ( i j i j ij In a rigid formation, normally only some distances are explicitly maintained, with the rest being consequentially maintained. NOT rigid Rigid In 2D Rigid In 3D

14 Global Rigid Formations A formation is globally rigid if it uniquely models the given set of distances that are to be explicitly maintained during a continuous move. 2D Rigid, but not globally rigid: Discontinuous Flip Ambiguity Rigid, but not globally rigid: Discontinuous Flex Ambiguity Globally Rigid Globally Rigid

15 History Euler s Conjecture (1766) A closed spacial figure allows no changes as long as it is not ripped apart Cauchy s Theorem (1813) strictly convex polyhedra with congruent corresponding faces are congruent.

16 History (cont d) Maxwell s Constraint Counting (1864) Counting the constraints and determine that the internal and external degree of freedom to be equal -> combinatorial rigidity H. Lamb s Theorem (1928) A necessary but not sufficient condition for a graph to be just rigid in 2D is that it has at least 2 V -3 edges (where V is number of vertices).

17 More graph rigidity results L. Henneberg 1911 Gives two basic operations for constructing 2D rigid graph. G. Laman 1970 Gives necessary and sufficient condition for a graph to be rigid in 2D. Tay & Whiteley 1985 Grow rigid graphs in various ways in both 2D and 3D. Connelly, Jackson & Jordan, Gortler etc. since 2003 Advances on globally rigid graphs

18 Other applications Molecular Structure (Biology, Physics, Chemistry) Tensegrities (Structural) Point Matching (Computer Vision)

19 Control Engineer s Intuition How many edges are necessary for a graph to be rigid? = How many distance constraints are necessary to limit a formation to only congruent displacement? 2D Total degrees of freedom: 2n

20 How many edges are necessary for a graph to be rigid? 2D Each edge can remove a single degree of freedom Rotations and translations will always be possible, so at least 2n-3 edges are necessary for a graph to be rigid. (Lamb 1928)

21 Are 2n-3 edges sufficient? 2D n = 3, 2n-3 = 3 n = 4, 2n-3 = 5 n = 5, 2n-3 = 7 yes yes NO Need at least 2n-3 well-distributed edges.

22 Further intuition If a subgraph has more edges than necessary, some edges are redundant. Non-redundant edges are independent. Each independent edge removes a degree of freedom. Therefore, 2n-3 independent edges guarantee rigidity in R 2. (Well known Laman s Theorem, 1970)

23 Sensor network localization

24 Formation Maintenance with Distance Constraints The problem: Maintaining a formation in 2D Autonomous agent : Point in the plane Maintenance links when each link is maintained by both agents (an undirected graph), it is necessary for the formation to be rigid (Eren et al, 2002)

25 Formation Maintenance with Distance Constraints The problem: Maintaining a formation in 2D Autonomous agent : Point in the plane Maintenance links when each link is maintained by both one agents (an undirected graph) graph), it is necessary for the formation to be rigid persistent (Eren(Hendrickx et al, 2002) et al, 2005)

26 Formation Maintenance with Distance Constraints The potential problem of controlling directed formation containing cycles (Baillieul et al 2003)

27 Formation Maintenance with Distance Constraints Chasing tails? The potential problem of controlling directed formation containing cycles (Baillieul et al 2003) A directed triangle is the simplest asymmetric formation which is both rigid and contains a cycle Cao et al showed in 2007 the almost global exponential convergence

28 d d 1 Gradient control: Exponential convergence 1 d 2 However, what if there are measurement errors? d 3 3

29 In 2D, the robots move collectively in formation following a closed orbit that is determined by a single sinusoidal signal; In 3D, the orbit becomes helical that is determined by a single sinusoidal signal and a constant drift.

30 Estimator-based formation control What kind of measurement errors can we handle? Small random noise as guaranteed by the gradient control law A constant signal + a finite number of sinusoidal signals with known frequencies

31 Estimator-based formation control What kind of measurement errors can we handle? Small random noise as guaranteed by the gradient control law A constant signal + a finite number of sinusoidal signals with known frequencies How to handle? Step 1: pick estimating agents, one per distance constraint Step 2: each estimating agent follow estimator-based control What are the improved performances? Exponential convergence Eliminating measurement errors and steady-state undesirable movement

32 Internal model principle P : C : W : ½ _e = 2R(z)R(z) T e 2R(z)S 1 (z) T (¹ ^¹ ) y = e ½ _» = M» + B (y + ¹ ^¹ ) ^¹ = B T» ½ _w = M w ¹ = B T w ;

33 Choosing estimating agents

34 Simulation results

35 Outline What are agents and multi-agent systems? Mobile robots and their formations Rigidity graph theory Localization and formation control Dealing with inconsistent measurements Coordination through strategic interactions To cooperate or defect in group tasks Evolutionary games on networks Towards optimal control of evolutionary games

36 Forming circle formations for autonomous robots

37 Forming circle formations for autonomous robots

38 Forming circle formations for autonomous robots x 2 x 1 x 3 x 4 x 5

39 Forming circle formations for autonomous robots x 2 _x 1 = u w 1 = (x 2 + x 5 )=2 u = w 1 x 1 x 3 x 4 x 5

40 Forming circle formations for autonomous robots x 2 _x 1 = 1 2 (x 2 + x 5 ) x 1 x 3 x 4 x 5

41 Forming circle formations for autonomous robots _x 1 = 1 2 (x 2 + x 5 ) x 1 _x 2 = 1 2 (x 3 + x 1 ) x 2 _x 3 = 1 2 (x 4 + x 2 ) x 3 _x 4 = 1 2 (x 5 + x 3 ) x 4 Linear time-invariant dynamical system Equilibrium: x 1 = 1 2 (x 2 + x 5) _x 5 = 1 2 (x 1 + x 4 ) x 5

42 Forming circle formations for autonomous robots

43 Simulation results Is cooperation really advantageous? Assigning the roles of leaders and followers

44 Simulation results Assigning the roles of leaders and followers To defect: move only when u is positive Can agents learn to adapt?

45 Simulation results Assigning the roles of leaders and followers Evolve into the scenario that all agents choose to defect.

46 Genetic and cultural evolution of cooperative behavior Collaborating with theoretical biologists to Develop and analyze models for genetics, development, physiology, behavior and ecological interactions from an evolutionary perspective Focus on mechanistic models, both concerning phenotypic architecture and fitness (e.g. self-organization) Developing control theory and tools for evolutionary behavior in networks

47 The paradox of cooperation Cooperation is ubiquitous in nature and, in particular, in human society; but it is often difficult to explain: Cooperation is often costly for the individual, while benefits are distributed over a collective Free-rider problem Individuals differ with regard to costs and benefits, inducing them to prefer different options Coordination problem

48 Free-rider problem: the Prisoner s dilemma cooperate defect cooperate b-c -c defect b 0 b > c >0 Mutual cooperation more profitable than mutual defection But: under all circumstances defect is the best strategy Defect is the only evolutionary stable strategy

49 At group level: Public good game Everybody profits from public good, whether contributing or not Individually, it pays to profit without contributing Erosion of the public good

50 Proposed solutions for the dilema Group selection Kin selection Reciprocity ( tit-for-tat ) Partner choice / partner fidelity Cooperation networks Punishment / policing Institutions (humans) rich theory limited predictive power

51 There is more to cooperation than the prisoner s dilemma The Snowdrift game: cooperate defect cooperate b 1 2 c b c defect b 0 b > c >0 mixed-strategy ESS: p ˆC b c = 1 b c 2

52 There is more to cooperation than the prisoner s dilemma The Stag hunt game: cooperate hunt alone cooperate B 0 hunt alone b b B > b > 0 two ESS: coordination game

53 Trendy topics in this context 1. Individual differences ( personalities ) 2. Self-organized division of labour 3. Responsive strategies 4. Behavioural architecture 5. Cultural group selection 6. Social learning strategies In my lab: Learning in multi-agent systems using evolutionary game theory Decision making in complex networks using cognitive models

54 Evolutionary game theory: History and motivation John Maynard Smith was interested in why so many animals engage in ritualized fighting ( The logic of animal conflict, Nature, 1973) Game theory, which had been used primarily to study the outcomes of decisions made by competing, rational economic agents, offered a potential framework for answering these questions.

55 Description Evolutionary game theory (EGT) refers to the study of large populations of interacting agents, and how various behaviors and traits might evolve. Differences from classical game theory Players = sub-populations, employing a common strategy Strategies = behaviors/traits, encoded in genes Payoffs = fitness, which determines reproductive potential Key concept The fitness of an individual must be evaluated in the context of the population in which it lives and interacts

56 Evolutionary games (EG) on networks Classical EGT often assumes a well mixed population Networks model populations with structured interactions

57 Applications of EGs on networks Biological networks Social networks

58 Why control EGs on Networks Potential reasons: To catalyze the resolution of social dilemmas To drive a network towards a desired strategy state Examples: Environmental preservation Crowd-sourcing Marketing Robotic swarms

59 NEG Framework: Strategy state

60 NEG Framework: Games and payoffs

61 Strategy update rules Strategy update is analogous to reproduction in classical EGT More successful strategies are adopted more often Types of update rules: Deterministic / stochastic Generally based on payoff differences between neighbors

62 NEG Framework: Strategy update Imitation rule: if a neighboring node using a different strategy has a higher total payoff than all neighbors (including me) using my strategy, switch; otherwise keep my strategy.

63 Example: Prisoner's dilemma

64 Example: Prisoner's dilemma

65 Problem statement

66 Results: Ring network

67 Results: Ring network

68 Results: Torus network

69 Results: Algorithm for arbitrary trees

70 Tree algorithm

71 Tree algorithm

72 Tree algorithm

73 Tree algorithm

74 Tree algorithm

75 Tree algorithm

76 Tree algorithm

77 Tree algorithm

78 Results: Uniform branching trees

79 Results: Uniform branching trees

80 Results: Uniform branching trees

81 Results: General networks

82 Graph partitioning approach: Example

83 Graph partitioning approach: Example

84 Graph partitioning approach: Example

85 Graph partitioning approach: Example

86 Graph partitioning approach: Example

87 Graph partitioning approach: Example

88 Simulations

89 Outline What are agents and multi-agent systems? Mobile robots and their formations Rigidity graph theory Localization and formation control Dealing with inconsistent measurements Coordination through strategic interactions To cooperate or defect in group tasks Evolutionary games on networks Towards optimal control of evolutionary games

90 Concluding remarks Challenges: Local information Evolution over time Models are nonlinear or descriptive Cyber-physical interactions Future works Adaptation and evolution Stochastic stability Robustness analysis Safety ---- The end ----