Distributed Constraint Optimization

Transcription

1 Distributed Constraint Optimization Gauthier Picard MINES Saint-Étienne LaHC UMR CNRS 5516 Some contents taken from OPTMAS 2011 and OPTMAS-DCR 2014 Tutorials UMR CNRS 5516 SAINT-ETIENNE

2 Contents Introduction Constraint Optimization Problems DCOP Framework Application Domains Complete Algorithms for DCOP Asynchronous Distributed Optimisation (ADOPT) Dynamic Programming Optimization Protocol (DPOP) Approximate Algorithms for DCOP Distributed Stochastic Search Algorithm (DSA) Maximum Gain Message (MGM-1) Synthesis Panorama Gauthier Picard Distributed Constraint Optimization 2

3 Constraint Optimization Problems Sometimes satisfaction is not possible Overconstrained problem Solution is not binary Switch from satisfaction to optimization Minimizing the number of violated constraints Minimizing the cost of violated constraints Maximizing the overall utility of the system... Gauthier Picard Distributed Constraint Optimization 3

4 DCOP Framework Motivations In dynamic and complex environments not all constraints can be satisfied completely Satisfaction! Optimisation (combinatorial) I ex: minimizing the number of unchecked constraints, minimizing the sum of the costs of violated constraints, etc. Definition (DCOP) A DCOP is a DCSP ha, X, D, C, i with a cost function f ij : D i D j 7! N [1for each pair x i,x j an objective function F : D 7! N [1evaluating an assignment A with f ij(d i,d j) for each pair x i,x j Gauthier Picard Distributed Constraint Optimization 4

5 DCOP Framework (cont.) Links VALUE messages Neighbors x1 xi xj f(xi,xj) Parent/Child x1 x2 VIEW Messages x1 x2 x2 x3 x3 x3 x4 x4 x4 (a) (b) (c) Objective Function X F (A) = f ij(d i,d j) where x i d i and x i d i in A x i,x j 2X In figure (a): F ({(x 1, 0), (x 2, 0), (x 3, 0), (x 4, 0)}) =4 F ({(x 1, 1), (x 2, 1), (x 3, 1), (x 4, 1)}) =0 Gauthier Picard Distributed Constraint Optimization 5

6 DCOP Algorithms DCOP Algorithms Complete Algorithms Incomplete Algorithms Search Algorithms Inference Algorithms Search Algorithms Inference Algorithms Sampling Algorithms e.g., SBB, ADOPT, AFB e.g., DPOP, Action- GDL e.g., MGM, DBA, DSA e.g., maxsum e.g., DUCT, D-Gibbs Gauthier Picard Distributed Constraint Optimization 6

7 Introduction Complete DCOP Approximate DCOP Synthesis References Application Domains Gauthier Picard Distributed Constraint Optimization 7

8 Contents Introduction Complete Algorithms for DCOP Asynchronous Distributed Optimisation (ADOPT) Dynamic Programming Optimization Protocol (DPOP) Approximate Algorithms for DCOP Synthesis Gauthier Picard Distributed Constraint Optimization 8

9 Asynchronous Distributed Optimisation (ADOPT) [MODI et al., 2005] ADOPT: DFS tree (pseudotree) ADOPT assumes that agents are arranged in a DFS tree: constraint graph! rooted graph (select a node as root) some links form a tree / others are backedges two constrained nodes must be in the same path to the root by tree links (same branch) Every graph admits a DFS tree: DFS graph traversal x 1 x 2 x 3 x 4 Gauthier Picard Distributed Constraint Optimization 9

10 ADOPT Features Asynchronous algorithm Each time an agent receives a message: I Processes it (the agent may take a new value) I Sends VALUE messages to its children and pseudochildren I Sends a COST message to its parent Context: set of (variable value) pairs (as ABT agent view) of ancestor agents (in the same branch) Current context: I Updated by each VALUE message I If current context is not compatible with some child context, the later is initialized (also the child bounds) Gauthier Picard Distributed Constraint Optimization 10

11 ADOPT Procedures Initialize (1) threshold 0; CurrentContext {}; (2) forall d Di,xl Children do (3) lb(d,xl) 0; t(d,xl ) 0; (4) ub(d,xl) Inf ; context(d,xl) {}; enddo; (5) di d that minimizes LB(d); (6) backtrack; when received (THRESHOLD, t, context) (7) if context compatible with CurrentContext: (8) threshold t; (9) maintainthresholdinvariant; (10) backtrack; endif; when received (TERMINATE, context) (11) record TERMINATE received from parent; (12) CurrentContext context; (13) backtrack; procedure backtrack (38) if threshold == UB: (39) di d that minimizes UB(d); (40) else if LB(di)>threshold: (41) di d that minimizes LB(d);endif; (42) SEND (VALUE,(xi, di )) (43) to each lower priority neighbor; (44) maintainallocationinvariant; (45) if threshold == UB: (46) if TERMINATE received from parent (47) or xi is root: (48) SEND (TERMINATE, (49) CurrentContext {(xi,di)}) (50) to each child; (51) Terminate execution; endif;endif; (52) SEND (COST, xi, CurrentContext, LB, UB) to parent; when received (VALUE,(xj,dj)) (14) if TERMINATE not received from parent: (15) add (xj,dj )tocurrentcontext; (16) forall d Di,xl Children do (17) if context(d, xl) incompatible with CurrentContext: (18) lb(d,xl) 0; t(d,xl ) 0; (19) ub(d,xl) Inf ; context(d,xl) {}; endif; enddo; (20) maintainthresholdinvariant; (21) backtrack; endif; when received (COST, xk, context, lb, ub) (22) d value of xi in context; (23) remove (xi,d)fromcontext; (24) if TERMINATE not received from parent: (25) forall (xj,dj ) context and xj is not my neighbor do (26) add (xj,dj )tocurrentcontext;enddo; (27) forall d Di,xl Children do (28) if context(d,xl) incompatible with CurrentContext: (29) lb(d,xl) 0; t(d,xl) 0; (30) ub(d,xl) Inf ; context(d,xl) {};endif;enddo;endif; (31) context compatible with CurrentContext: if (32) lb(d,xk) lb; (33) ub(d, xk) ub; (34) context(d,xk) context; (35) maintainchildthresholdinvariant; (36) maintainthresholdinvariant; endif; (37) backtrack; Algorithm 1: ADOPT Procedures Gauthier Picard Distributed Constraint Optimization 11

12 ADOPT Messages value(parent! children [ pseudochildren, a): parent informs descendants that it has taken value a cost(child! parent, lowerbound, upperbound, context): child informs parent of the best cost of its assignement; attached context to detect obsolescence threshold(parent! child, t): minimum cost of solution in child is at least t termination(parent! children): sent when LB = UB Gauthier Picard Distributed Constraint Optimization 12

13 ADOPT Data Structures 1. Current context (agent view): values of higher priority constrained agents x i x j... a c Bounds (for each value, child) x j a b c d I lower bounds lb(x k ) I upper bounds ub(x k ) I thresholds th(x k ) I contexts context(x k ) Stored contextes must be active: context 2 currentcontext If a context becomes no active, it is removed (lb 0,th 0,ub 1) Gauthier Picard Distributed Constraint Optimization 13

14 ADOPT Bounds x j a b c Gauthier Picard Distributed Constraint Optimization 14

15 ADOPT Bounds (value) =cost with higher agents (b) = X c ij(a, b) i2curctx x j a b c Gauthier Picard Distributed Constraint Optimization 14

16 ADOPT Bounds (value) =cost with higher agents (b) = X c ij(a, b) i2curctx OPT(x j,ctx)=min (d)+ d2d X j OPT(x k,ctx[ x k 2children (x j,d)) a b x j c Gauthier Picard Distributed Constraint Optimization 14

17 ADOPT Bounds (value) =cost with higher agents (b) = X c ij(a, b) i2curctx OPT(x j,ctx)=min (d)+ d2d X j OPT(x k,ctx[ x k 2children (x j,d)) a b x j c lb 1 ub 1 lb 2 ub 2 lb 3 ub Gauthier Picard Distributed Constraint Optimization 14

18 ADOPT Bounds (value) =cost with higher agents (b) = X c ij(a, b) i2curctx OPT(x j,ctx)=min (d)+ d2d X j OPT(x k,ctx[ x k 2children (x j,d)) a b x j c lb 1 ub 1 lb 2 ub 2 lb 3 ub [lb k,ub k ]=cost of lower agents Gauthier Picard Distributed Constraint Optimization 14

19 ADOPT Bounds (value) =cost with higher agents (b) = X c ij(a, b) i2curctx x j OPT(x j,ctx)=min (d)+ d2d X j OPT(x k,ctx[ x k 2children (x j,d)) LB(b) = (b)+ X lb(b, x k ) x k 2children a b c LB =min b2d j LB(b) lb 1 ub 1 lb 2 ub 2 lb 3 ub [lb k,ub k ]=cost of lower agents UB(b) = (b)+ X x k 2children UB =min b2d j UB(b) ub(b, x k ) Gauthier Picard Distributed Constraint Optimization 14

20 ADOPT Value Assignment An ADOPT agent takes the value with minimum LB Eager behavior: I Agents may constantly change value I Generates many context changes Threshold: I lower bound of the cost that children have from previous search I parent distributes threshold among children I incorrect distribution does not cause problems: the child with minor allocation would send a COST to the parent later, and the parent will rebalance the threshold distribution Gauthier Picard Distributed Constraint Optimization 15

21 ADOPT Properties For any x i, LB apple OPT(x l,ctx) apple UB For any x i, its threshold reaches UB For any x i, its final threshold is equal to OPT(x l,ctx)! ADOPT terminates with the optimal solution Gauthier Picard Distributed Constraint Optimization 16

22 ADOPT Example 4 variables (4 agents) x 1, x 2, x 3 and x 4 with D = {a, b} 4 binary identical cost functions x i x j cost Constraint graph x 1 x 2 x 3 x 4 Gauthier Picard Distributed Constraint Optimization 17

23 ADOPT Example (cont.) x 1 = a x 2 = a x 3 = a x 4 = a Gauthier Picard Distributed Constraint Optimization 18

24 ADOPT Example (cont.) x 1 = a x 1 = a [1, 1,x 1 = a] x 2 = a [2, 2,x 1 = x 2 = a] x 2 = a [1, 1,x 2 = a] x 3 = a x 4 = a x 3 = a x 4 = a Gauthier Picard Distributed Constraint Optimization 18

25 ADOPT Example (cont.) x 1 = a x 1 = a [1, 1,x 1 = a] x 1 = b x 2 = a [2, 2,x 1 = x 2 = a] x 2 = a [1, 1,x 2 = a] x 2 = a x 3 = a x 4 = a x 3 = a x 4 = a x 3 = a x 4 = a Gauthier Picard Distributed Constraint Optimization 18

26 ADOPT Example (cont.) x 1 = a x 1 = a [1, 1,x 1 = a] x 1 = b x 2 = a [2, 2,x 1 = x 2 = a] x 2 = a [1, 1,x 2 = a] x 2 = a x 3 = a x 4 = a x 3 = a x 4 = a x 3 = a x 4 = a x 1 = b [0, 1,x 1 = b] [2, 2,x 1 = b, x 2 = a] x 2 = b x 3 = b x 4 = a Gauthier Picard Distributed Constraint Optimization 18

27 ADOPT Example (cont.) x 1 = a x 1 = a [1, 1,x 1 = a] x 1 = b x 2 = a [2, 2,x 1 = x 2 = a] x 2 = a [1, 1,x 2 = a] x 2 = a x 3 = a x 4 = a x 3 = a x 4 = a x 3 = a x 4 = a x 1 = b x 1 = b [0, 1,x 1 = b] [0, 3,x 1 = b] [2, 2,x 1 = b, x 2 = a] x 2 = b [0, 0,x 1 = x 2 = b b, x 2 = b] [0, 0,x 2 = b] x 3 = b x 4 = a x 3 = b x 4 = b Gauthier Picard Distributed Constraint Optimization 18

28 ADOPT Example (cont.) x 1 = a x 1 = a [1, 1,x 1 = a] x 1 = b x 2 = a [2, 2,x 1 = x 2 = a] x 2 = a [1, 1,x 2 = a] x 2 = a x 3 = a x 4 = a x 3 = a x 4 = a x 3 = a x 4 = a x 1 = b x 1 = b x 1 = b [0, 1,x 1 = b] [0, 3,x 1 = b] [0, 0,x 1 = b] [2, 2,x 1 = b, x 2 = a] x 2 = b [0, 0,x 1 = x 2 = b b, x 2 = b] [0, 0,x 2 = b] x 2 = b x 3 = b x 4 = a x 3 = b x 4 = b x 3 = b x 4 = b Gauthier Picard Distributed Constraint Optimization 18

29 Dynamic Programming Optimization Protocol (DPOP) [PETCU and FALTINGS, 2005] 3-phase distributed algorithm PHASES MESSAGES 1. DFS Tree construction token passing 2. Utility phase: from leaves to root 3. Value phase: from root to leaves util (child! parent, constraint table [-child]) value (parent! children [ pseudochildren, parent value) Gauthier Picard Distributed Constraint Optimization 19

30 DFS Tree Phase Distributed DFS graph traversal: token, ID, neighbors(x) 1. X owns the token: adds its own ID and sends it in turn to each of its neighbors, which become children 2. Y receives the token from X: it marks X as visited. First time Y receives the token then parent(y )=X. Other IDs in token which are also neighbors(y ) are pseudoparent. If Y receives token from neighbor W to which it was never sent, W is pseudochild. 3. When all neighbors(x) visited, X removes its ID from token and sends it to parent(x). A node is selected as root, which starts When all neighbors of root are visited, the DFS traversal ends Gauthier Picard Distributed Constraint Optimization 20

31 DFS Tree Phase: Example root x 1 x 4 [x 1] x 1 parent of x 2 x 2 x 3 Gauthier Picard Distributed Constraint Optimization 21

32 DFS Tree Phase: Example root x 1 x 4 [x 1] x 1 parent of x 2 x 2 x 3 x 1 x 4 x 2 parent of x 3 x 1 pseudoparent of x 3 x 2 x 3 [x 1,x 2] Gauthier Picard Distributed Constraint Optimization 21

33 DFS Tree Phase: Example root x 1 x 4 [x 1] x 1 parent of x 2 x 2 x 3 x 1 x 4 x 2 parent of x 3 x 1 pseudoparent of x 3 x 2 x 3 [x 1,x 2] x 1 [x 1,x 2,x 3] x 4 x 3 parent of x 4 x 3 pseudoparent of x 1 x 2 x 3 Gauthier Picard Distributed Constraint Optimization 21

34 DFS Tree Phase: Example root x 1 x 4 [x 1] x 1 parent of x 2 x 2 x 3 x 1 x 1 x 4 x 2 parent of x 3 x 1 pseudoparent of x 3 x 2 x 2 x 3 [x 1,x 2] x 3 x 4 x 1 [x 1,x 2,x 3] x 4 x 3 parent of x 4 x 3 pseudoparent of x 1 x 2 x 3 Gauthier Picard Distributed Constraint Optimization 21

35 Util Phase Agent X: receives from each child Y i a cost function: C(Y i) combines (adds, joins) all these cost functions with the cost functions with parent(x) and pseudoparents(x) projects X out of the resulting cost function, and sends it to parent(x) From the leaves to the root Gauthier Picard Distributed Constraint Optimization 22

36 Util Phase: Example X Gauthier Picard Distributed Constraint Optimization 23

37 Util Phase: Example X T parent children X X Y X Z Gauthier Picard Distributed Constraint Optimization 23

38 Util Phase: Example X Y X T parent children X add X Z X Y Z T a a a a 3 a a a b 4 a a b a 4 a a b b 5 a b a a 4 a b a b 5 a b b a 5 a b b b 6 b a a a 6 b a a b 4 b a b a 4 b a b b 2 b b a a 4 b b b b b b All value combinations Costs are the sum of applicable costs Gauthier Picard Distributed Constraint Optimization 23

39 Util Phase: Example X Y X T parent children X add X Z X Y Z T a a a a 3 a a a b 4 a a b a 4 a a b b 5 a b a a 4 a b a b 5 a b b a 5 a b b b 6 b a a a 6 b a a b 4 b a b a 4 b a b b 2 b b a a 4 b b b b b b All value combinations Costs are the sum of applicable costs Project out X X Y Z T a a a a 3 a a a b 4 a a b a 4 a a b b 5 a b a a 4 a b a b 5 a b b a 5 a b b b 6 b a a a 6 b a a b 4 b a b a 4 b a b b 2 b b a a 4 b b b b b b Remove X Remove duplicates Keep the min cost Gauthier Picard Distributed Constraint Optimization 23

40 Value Phase 1. The root finds the value that minimizes the received cost function in the util phase, and informs its descendants (children [ pseudochildren) 2. Each agent waits to receive the value of its parent / pseudoparents 3. Keeping fixed the value of parent/pseudoparents, finds the value that minimizes the received cost function in the Util phase 4. Informs of this value to its children/pseudochildren This process starts at the root and ends at the leaves Gauthier Picard Distributed Constraint Optimization 24

41 DTREE : DPOP for DCOPs without backedges X Y Z W Gauthier Picard Distributed Constraint Optimization 25

42 DTREE : DPOP for DCOPs without backedges X X Y Y Z Y Z Y W W Gauthier Picard Distributed Constraint Optimization 25

43 DTREE : DPOP for DCOPs without backedges X X Y Y Y a b 1 0 Z Y Z Y W W Gauthier Picard Distributed Constraint Optimization 25

44 DTREE : DPOP for DCOPs without backedges X X Y Y Y a b 1 0 Y a b 1 0 Z Y Z Y W W Gauthier Picard Distributed Constraint Optimization 25

45 DTREE : DPOP for DCOPs without backedges X X Y X a b 2 0 Y Y a b 1 0 Y a b 1 0 Z Y Z Y W W Gauthier Picard Distributed Constraint Optimization 25

46 DTREE : DPOP for DCOPs without backedges X X Y X a b 2 0 X b Y Y a b 1 0 Y a b 1 0 Z Y Z Y W W Gauthier Picard Distributed Constraint Optimization 25

47 DTREE : DPOP for DCOPs without backedges X X Y X a b 2 0 X b Y Y a b 1 0 Y b Y a b 1 0 Z Y Z Y W W Gauthier Picard Distributed Constraint Optimization 25

48 DTREE : DPOP for DCOPs without backedges X X Y X a b 2 0 X b Y Y a b 1 0 Y b Y a b 1 0 Z b Z Y Z Y W W W b Gauthier Picard Distributed Constraint Optimization 25

49 DTREE : DPOP for DCOPs without backedges X X Y X a b 2 0 X b Optimal solution: Y a b 1 0 Y Y b Y a b 1 0 linear number of messages message size: linear Z b Z Y Z Y W W W b Gauthier Picard Distributed Constraint Optimization 25

50 DPOP for any DCOP X Y Z W Gauthier Picard Distributed Constraint Optimization 26

51 DPOP for any DCOP X Z X X Y Y Z Y Z Y W W Gauthier Picard Distributed Constraint Optimization 26

52 DPOP for any DCOP X Z X X Y Y a b X a 2 2 b 2 0 Y Z Y Z Y W W Gauthier Picard Distributed Constraint Optimization 26

53 DPOP for any DCOP X Z X X Y Y a b X a 2 2 b 2 0 Y Y a b 1 0 Z Y Z Y W W Gauthier Picard Distributed Constraint Optimization 26

54 DPOP for any DCOP X Z X X Y X a b 2 0 Y a b X a 2 2 b 2 0 Y Y a b 1 0 Z Y Z Y W W Gauthier Picard Distributed Constraint Optimization 26

55 DPOP for any DCOP X Z X X Y X a b 2 0 X b X b Y a b X a 2 2 b 2 0 Y Y a b 1 0 Z Y Z Y W W Gauthier Picard Distributed Constraint Optimization 26

56 DPOP for any DCOP X Z X X Y X a b 2 0 X b X b Y Y a b X a 2 2 b 2 0 Y b Y a b 1 0 Z Y Z Y W W Gauthier Picard Distributed Constraint Optimization 26

57 DPOP for any DCOP X Z X X Y X a b 2 0 X b X b Y Y a b X a 2 2 b 2 0 Y b Y a b 1 0 Z b Z Y Z Y W W W b Gauthier Picard Distributed Constraint Optimization 26

58 DPOP for any DCOP X Z X X Y X Y a b X a 2 2 b 2 0 X a b 2 0 b Y Y b X b Y a b 1 0 Optimal solution: linear number of messages message size: exponential Z b Z Y Z Y W W W b Gauthier Picard Distributed Constraint Optimization 26

59 Contents Introduction Complete Algorithms for DCOP Approximate Algorithms for DCOP Distributed Stochastic Search Algorithm (DSA) Maximum Gain Message (MGM-1) Synthesis Gauthier Picard Distributed Constraint Optimization 27

60 Approximate Algorithms for DCOPs Complete algorithms e.g. ADOPT [MODI et al., 2005] and DPOP [PETCU and FALTINGS, 2005] 3 complete 7 slow Aproximate algorithms exist (fast, but sub-optimal in many case) Search algorithms I DBA [YOKOO, 2001], DSA [ZHANG et al., 2005], MGM [MAHESWARAN et al., 2004] Inference algorithms I Max-sum [FARINELLI et al., 2008] Gauthier Picard Distributed Constraint Optimization 28

61 Why Approximate Algorithms Motivations I Often optimality in practical applications is not achievable I Fast good enough solutions are all we can have Example Graph coloring I Medium size problem (about 20 nodes, three colors per node) I Number of states to visit for optimal solution in the worst case 3 20 =3M states Key problem I Provides guarantees on solution quality Gauthier Picard Distributed Constraint Optimization 29

62 Exemplar Application: Surveillance Event Detection I Vehicles passing on a road Energy Constraints I Sense/Sleep modes I Recharge when sleeping Coordination I Activity can be detected by single sensor I Roads have different traffic loads Aim I Focus on road with more traffic load Gauthier Picard Distributed Constraint Optimization 30

63 Centralized Local Greedy approaches Greedy local search I I I Start from random solution Do local changes if global solution improves Local: change the value of a subset of variables, usually one Gauthier Picard Distributed Constraint Optimization 31

64 Centralized Local Greedy approaches Greedy local search I I I Start from random solution Do local changes if global solution improves Local: change the value of a subset of variables, usually one Gauthier Picard Distributed Constraint Optimization 31

65 Centralized Local Greedy approaches Greedy local search I I I Start from random solution Do local changes if global solution improves Local: change the value of a subset of variables, usually one Gauthier Picard Distributed Constraint Optimization 31

66 Centralized Local Greedy approaches Greedy local search I I I Start from random solution Do local changes if global solution improves Local: change the value of a subset of variables, usually one Gauthier Picard Distributed Constraint Optimization 31

67 Centralized Local Greedy approaches Greedy local search I I I Start from random solution Do local changes if global solution improves Local: change the value of a subset of variables, usually one Gauthier Picard Distributed Constraint Optimization 31

68 Centralized Local Greedy approaches Problems I I Local minima Standard solutions: Random Walk, Simulated Annealing Gauthier Picard Distributed Constraint Optimization 32

69 Centralized Local Greedy approaches Problems I I Local minima Standard solutions: Random Walk, Simulated Annealing Gauthier Picard Distributed Constraint Optimization 32

70 Centralized Local Greedy approaches Problems I I Local minima Standard solutions: Random Walk, Simulated Annealing Gauthier Picard Distributed Constraint Optimization 32

71 Centralized Local Greedy approaches Problems I I Local minima Standard solutions: Random Walk, Simulated Annealing Gauthier Picard Distributed Constraint Optimization 32

72 Distributed Local Greedy approaches Local knowledge Parallel execution I I A greedy local move might be harmful/useless Need coordination Gauthier Picard Distributed Constraint Optimization 33

73 Distributed Local Greedy approaches Local knowledge Parallel execution I I A greedy local move might be harmful/useless Need coordination Gauthier Picard Distributed Constraint Optimization 33

74 Distributed Local Greedy approaches Local knowledge Parallel execution I I A greedy local move might be harmful/useless Need coordination Gauthier Picard Distributed Constraint Optimization 33

75 Distributed Local Greedy approaches Local knowledge Parallel execution I I A greedy local move might be harmful/useless Need coordination Gauthier Picard Distributed Constraint Optimization 33

76 Distributed Stochastic Search Algorithm (DSA) [ZHANG et al., 2005] Greedy local search with activation probability to mitigate issues with parallel executions DSA-1: change value of one variable at time Initialize agents with a random assignment and communicate values to neighbors Each agent: I Generates a random number and execute only if rnd less than activation probability I When executing changes value maximizing local gain I Communicate possible variable change to neighbors Gauthier Picard Distributed Constraint Optimization 34

77 DSA-1: Execution Example Gauthier Picard Distributed Constraint Optimization 35

78 DSA-1: Execution Example rnd? > 1 4 rnd? > 1 4 rnd? > 1 4 rnd? > Gauthier Picard Distributed Constraint Optimization 35

79 DSA-1: Execution Example Gauthier Picard Distributed Constraint Optimization 35

80 DSA-1: Execution Example Gauthier Picard Distributed Constraint Optimization 35

81 DSA-1: Execution Example Gauthier Picard Distributed Constraint Optimization 35

82 DSA-1: Discussion Extremely cheap (computation/communication) Good performance in various domains I e.g. target tracking [FITZPATRICK and MEERTENS, 2003; ZHANG et al., 2003] I Shows an anytime property (not guaranteed) I Benchmarking technique for coordination Problems I Activation probablity must be tuned [ZHANG et al., 2003] I No general rule, hard to characterise results across domains Gauthier Picard Distributed Constraint Optimization 36

83 Maximum Gain Message (MGM-1) [MAHESWARAN et al., 2004] Coordinate to decide who is going to move I Compute and exchange possible gains I Agent with maximum (positive) gain executes Analysis I Empirically, similar to DSA I More communication (but still linear) I No Threshold to set I Guaranteed to be monotonic (Anytime behavior) Gauthier Picard Distributed Constraint Optimization 37

84 MGM-1: Example 1 1 Gauthier Picard Distributed Constraint Optimization 38

85 MGM-1: Example Gauthier Picard Distributed Constraint Optimization 38

86 MGM-1: Example G = 2 Gauthier Picard Distributed Constraint Optimization 38

87 MGM-1: Example G = 2 Gauthier Picard Distributed Constraint Optimization 38

88 MGM-1: Example G = 2 G =0 Gauthier Picard Distributed Constraint Optimization 38

89 MGM-1: Example G = 2 G =0 Gauthier Picard Distributed Constraint Optimization 38

90 MGM-1: Example G = G =0 1 1 G =2 Gauthier Picard Distributed Constraint Optimization 38

91 MGM-1: Example G = G =0 1 1 G =2 Gauthier Picard Distributed Constraint Optimization 38

92 MGM-1: Example G = G =0 1 1 G =2 1 1 G =0 Gauthier Picard Distributed Constraint Optimization 38

93 MGM-1: Example G = G =0 1 1 G =2 1 1 G =0 Gauthier Picard Distributed Constraint Optimization 38

94 MGM-1: Example G = G =0 1 1 G =2 1 1 G =0 Gauthier Picard Distributed Constraint Optimization 38

95 MGM-1: Example 1 1 Gauthier Picard Distributed Constraint Optimization 38

96 To sum up on local greedy approaches Exchange local values for variables I Similar to search based methods (e.g. ADOPT) Consider only local information when maximizing I Values of neighbors Anytime behaviors Could result in very bad solutions Gauthier Picard Distributed Constraint Optimization 39

97 GDL-based approaches Generalized Distributive Law [AJI and MCELIECE, 2000] I Unifying framework for inference in Graphical models I Builds on basic mathematical properties of semi-rings I Widely used in Info theory, Statistical physics, Probabilistic models Max-sum I DCOP settings: maximise social welfare Gauthier Picard Distributed Constraint Optimization 40

98 Max-Sum Agents iteratively computes local functions that depend only on the variable they control m 1!2 x 1 x 2 m 2!1 m 4!1 m 1!4 m 3!2 m 2!3 x 4 m 4!3 m 3!4 x 3 Gauthier Picard Distributed Constraint Optimization 41

99 Max-Sum Agents iteratively computes local functions that depend only on the variable they control m 1!2 x 1 x 2 m 4!1 x 4 x 3 Gauthier Picard Distributed Constraint Optimization 41

100 Max-Sum Agents iteratively computes local functions that depend only on the variable they control m 1!2 x 1 x 2 m 4!1 x 4 x 3 m 1!2(x 2)=max x1 (F 12(x 1,x 2)+m 4!1(x 1)) Gauthier Picard Distributed Constraint Optimization 41

101 Max-Sum Agents iteratively computes local functions that depend only on the variable they control m 1!2 x 1 x 2 m 4!1 x 4 x 3 Shared constraint m 1!2(x 2)=max x1 (F 12(x 1,x 2)+m 4!1(x 1)) Gauthier Picard Distributed Constraint Optimization 41

102 Max-Sum Agents iteratively computes local functions that depend only on the variable they control m 1!2 x 1 x 2 m 4!1 x 4 x 3 Shared constraint m 1!2(x 2)=max x1 (F 12(x 1,x 2)+m 4!1(x 1)) All incoming messages except x 2 Gauthier Picard Distributed Constraint Optimization 41

103 Max-Sum Agents iteratively computes local functions that depend only on the variable they control z 1 (x 1 ) x 1 x 2 x 4 x 3 Shared constraint m 1!2(x 2)=max x1 (F 12(x 1,x 2)+m 4!1(x 1)) All incoming messages except x 2 Gauthier Picard Distributed Constraint Optimization 41

104 Max-Sum Agents iteratively computes local functions that depend only on the variable they control z 1 (x 1 ) x 1 x 2 x 4 x 3 Shared constraint m 1!2(x 2)=max x1 (F 12(x 1,x 2)+m 4!1(x 1)) z 1(x 1)=m 4!1(x 1)+m 2!1(x 1) All incoming messages except x 2 Gauthier Picard Distributed Constraint Optimization 41

105 Max-Sum Agents iteratively computes local functions that depend only on the variable they control z 1 (x 1 ) x 1 x 2 m 2!1 m 4!1 x 4 x 3 Shared constraint m 1!2(x 2)=max x1 (F 12(x 1,x 2)+m 4!1(x 1)) z 1(x 1)=m 4!1(x 1)+m 2!1(x 1) All incoming messages except x 2 Gauthier Picard Distributed Constraint Optimization 41

106 Max-Sum Agents iteratively computes local functions that depend only on the variable they control z 1 (x 1 ) x 1 x 2 m 2!1 m 4!1 x 4 x 3 Shared constraint m 1!2(x 2)=max x1 (F 12(x 1,x 2)+m 4!1(x 1)) z 1(x 1)=m 4!1(x 1)+m 2!1(x 1) All incoming messages All incoming messages except x 2 Gauthier Picard Distributed Constraint Optimization 41

107 Max-Sum Agents iteratively computes local functions that depend only on the variable they control z 1 (x 1 ) Choose argmax x 1 x 2 m 2!1 m 4!1 x 4 x 3 Shared constraint m 1!2(x 2)=max x1 (F 12(x 1,x 2)+m 4!1(x 1)) z 1(x 1)=m 4!1(x 1)+m 2!1(x 1) All incoming messages All incoming messages except x 2 Gauthier Picard Distributed Constraint Optimization 41

108 Max-Sum on acyclic graphs Max-sum Optimal on acyclic graphs I Different branches are independent I Each agent can build a correct estimation of its contribution to the global problem (z functions) Message equations very similar to Util messages in DPOP I Sum messages from children and shared constraint I Maximize out agent variable I GDL generalizes DPOP [VINYALS et al., 2011] x 2 x 1 x 3 x 4 m 1!2(x 2)=max x1 (F 12(x 1,x 2)+m 4!1(x 1)) Gauthier Picard Distributed Constraint Optimization 42

109 Max-sum Performance Good performance on loopy networks [FARINELLI et al., 2008] I When it converges very good results I Interesting results when only one cycle [WEISS, 2000] I We could remove cycle but pay an exponential price (see DPOP) Gauthier Picard Distributed Constraint Optimization 43

110 Max-Sum for low power devices Low overhead I Msgs number/size Asynchronous computation I Agents take decisions whenever new messages arrive Robust to message loss Gauthier Picard Distributed Constraint Optimization 44

111 Contents Introduction Complete Algorithms for DCOP Approximate Algorithms for DCOP Synthesis Panorama Gauthier Picard Distributed Constraint Optimization 45

112 Panorama Algorithm Type Memory Messages Remarks ADOPT COP Polynomial Exponential Complete DPOP COP Exponential Linear Complete DSA COP Linear? Not complete MGM COP Linear? Not complete Max-Sum COP Exponential Linear on acyclic Complete on trees Table: DCOP algorithms Gauthier Picard Distributed Constraint Optimization 46

113 References AJI, S.M. and R.J. MCELIECE (2000). The generalized distributive law. In: Information Theory, IEEE Transactions on 46.2, pp ISSN: DOI: / FARINELLI, A., A. ROGERS, A. PETCU, and N. R. JENNINGS (2008). Decentralised Coordination of Low-power Embedded Devices Using the Max-sum Algorithm. In: Proceedings of the 7th International Joint Conference on Autonomous Agents and Multiagent Systems - Volume 2. AAMAS 08. International Foundation for Autonomous Agents and Multiagent Systems, pp ISBN: FITZPATRICK, Stephen and Lambert MEERTENS (2003). Distributed Coordination through Anarchic Optimization. In: Distributed Sensor Networks: A Multiagent Perspective. Ed. by Victor LESSER, Charles L. ORTIZ, and Milind TAMBE. Boston, MA: Springer US, pp ISBN: MAHESWARAN, R.T., J.P. PEARCE, and M. TAMBE (2004). Distributed Algorithms for DCOP: A Graphical-Game-Based Approach. In: Proceedings of the 17th International Conference on Parallel and Distributed Computing Systems (PDCS), San Francisco, CA, pp MODI, P. J., W. SHEN, M. TAMBE, and M. YOKOO (2005). ADOPT: Asynchronous Distributed Constraint Optimization with Quality Guarantees. In: Artificial Intelligence 161.2, pp PETCU, Adrian and Boi FALTINGS (2005). A scalable method for multiagent constraint optimization. In: IJCAI International Joint Conference on Artificial Intelligence, pp ISBN: VINYALS, Meritxell, Juan A. RODRÍGUEZ-AGUILAR, and Jesus CERQUIDES (2011). Constructing a unifying theory of dynamic programming DCOP algorithms via the generalized distributive law. In: Autonomous Agents and Multi-Agent Systems 3.22, pp ISSN: DOI: 1.1 7/s Gauthier Picard Distributed Constraint Optimization 47

114 References (cont.) WEISS, Yair (2000). Correctness of Local Probability Propagation in Graphical Models with Loops. In: Neural Comput. 12.1, pp ISSN: DOI: / URL: YOKOO, M. (2001). Distributed Constraint Satisfaction: Foundations of Cooperation in Multi-Agent Systems. Springer. ZHANG, W., G. WANG, Z. XING, and L. WITTENBURG (2005). Distributed stochastic search and distributed breakout: properties, comparison and applications to constraint optimization problems in sensor networks.. In: Journal of Artificial Intelligence Research (JAIR) , pp ZHANG, Weixiong, Guandong WANG, Zhao XING, and Lars WITTENBURG (2003). A Comparative Study of Distributed Constraint Algorithms. In: Distributed Sensor Networks: A Multiagent Perspective. Ed. by Victor LESSER, Charles L. ORTIZ, and Milind TAMBE. Boston, MA: Springer US, pp Gauthier Picard Distributed Constraint Optimization 48