Constraint Games. Huawei Technologies Ltd University of Caen-Normandie. Arnaud Lallouet joint work with Anthony Palmieri GDR IA-RO

Transcription

1 Constraint Games Huawei Technologies Ltd University of Caen-Normandie GDR IA-RO Arnaud Lallouet joint work with Anthony Palmieri

2 Huawei Technologies Co., Ltd. 2 Contents 1 Huawei French Research Center 2 Game theory 3 Constraint Games 4 Solving methods 5 Benchmarks 6 Conclusion and perspectives

3 Huawei Technologies Co., Ltd. 3 Contents 1 Huawei French Research Center 2 Game theory 3 Constraint Games 4 Solving methods 5 Benchmarks 6 Conclusion and perspectives

4 Huawei Technologies Co., Ltd. 4 Huawei Technologies Ltd Huawei s vision: Building a better connected world

5 Artificial Intelligence in Huawei Huawei Technologies Co., Ltd. 5

6 Artificial Intelligence in Huawei Huawei Technologies Co., Ltd. 6

7 French Research Center Huawei Technologies Co., Ltd. 7

8 Huawei Technologies Co., Ltd. 8 Artificial Intelligence in Huawei Research done in FRC Smartphone (see new Kirin 970: CPU+GPU+NPU) Cloud services Network applications, SDN, content delivery Internal use: recommendation, big data analytics 5G for connected and autonomous car and IOT An integration strategy Mostly integration of software Contributions to open source AI in Transition from Hardware to Software

9 Huawei Technologies Co., Ltd. 9 Contents 1 Huawei French Research Center 2 Game theory 3 Constraint Games 4 Solving methods 5 Benchmarks 6 Conclusion and perspectives

10 Huawei Technologies Co., Ltd. 1 Strategic games: the setting Set of Players P Each player i performs Actions A i and wants to maximize an Utility depending on other players actions Each player i has its own utility function (u i ), or preference Strategic form, also called multimatrix model In this talk: finite games only (also simultaneous, perfect information, selfish players)

11 Huawei Technologies Co., Ltd. 1 Solution concept Deviation We say that a player has a beneficial deviation if he can increase its utility by changing its own strategy How can we tell when a player is satisfied? Find a point where each player chooses the best strategy for him/herself... and for which no player can improve his/her utility by changing to another action: Pure Nash Equilibrium

12 A generic algorithm to solve games Solve function solve(s): tuple for s ΠA do if nash(s) then return s end if end for return not found Deviation function deviation(s, i): boolean for v A i, v s i do if u i (v.s i ) > u i (s) then return true end if end for return false Nash function nash(s): boolean for i P do if deviation(s, i) then return false end if end for return true Analysis Inefficient but still the baseline algorithm Implemented in the Gambit solver along with IESDS [McKelvey and al, 2010] Huawei Technologies Co., Ltd. 1

13 Game representation Matrix in Normal Form games grow exponentially with the number of players Exploiting Structure: Graphical Games[KLS01] dependencies between player are provided with a graph to reduce the matrix size Exploiting Semantics: Action-graph Games [XKNB07] Symmetry + anonymity Exploiting Semantics: Boolean Games [HvdHMW01] Represents utilities with SAT problems Like SAT for satisfiability, Boolean Games are a fundamental tool to study interaction between agents Huawei Technologies Co., Ltd. 1

14 Huawei Technologies Co., Ltd. 1 Contents 1 Huawei French Research Center 2 Game theory 3 Constraint Games 4 Solving methods 5 Benchmarks 6 Conclusion and perspectives

15 Constraint Games The idea Use CSP to express utilities Constraint Satisfaction Game A Constraint Satisfaction Game (CSG) is a 4-uple CG = (P, V, D, G) where P is a set of players V is a set of variables, Player i controls V i V D = (D x ) x V defines a (finite) domain for each variable G = (G i ) i P is a family of CSP Preferences CSPs provide a compact and natural formalism to express satisfaction for a player: G i is called Goal of Player i Goals express preferences and an equilibrium may hold if a player is not satisfied (and cannot be) Huawei Technologies Co., Ltd. 1

16 Huawei Technologies Co., Ltd. 1 COG and hard constraints Hard constraints CSG/COG can be enhanced with a set of hard constraints (HC) to forbid invalid equilibria a strategy profile which does not satisfy HC cannot be an equilibrium and should not be checked for deviations impossible to represent in the matrix model (even by giving a dummy value) Constraint Optimization Games Constraint Programming provides an easy way to express optimization: add min(x ) or max(x ) to the goal of each player Allows to represent in a natural way many useful games (see examples after)

17 Huawei Technologies Co., Ltd. 1 Example: Collaboration game Defintion two drivers meet on a narrow dirty road. Both have to swerve in order to avoid a head-on collision Right Left Left Right A simple model, is to choose an action, the utility of a player corresponds to the number of player choosing the same action

18 Collaboration game as Constraint Game p u b l i c c l a s s C o l l a b o r a t i o n G a m e extends AbstractGameModel { p r i v a t e f i n a l i n t n b S t r a t ; p u b l i c C o l l a b o r a t i o n G a m e ( i n t n b P l a y e r s, i n t n b S t r a t ) { s u p e r ( n b P l a y e r s ) ; t h i s. nbstrat = nbstrat ; O v e r r i d e p u b l i c v o i d b u i l d M o d e l ( Model m) { IntVar Choice [ ] = m. intvararray ( " Choice ", players. length 1, 0, nbstrat 1 ) ; IntVar Objectives [ ] = m. intvararray ( " objective ", players. length 1, 1, players. length ) ; f o r ( i n t i = 0 ; i < p l a y e r s. l e n g t h 1; i ++) { IntVar Evalue = m. intvar (m. generatename ( "COUNT_ " ), 0, players. length ) ; m. count ( Evalue, Choice, O b j e c t i v e s [ i ] ). p o s t ( ) ; ; } } f o r ( i n t i = 1 ; i < p l a y e r s. length ; i++) { p l a y e r s [ i ]. own ( Choice [ i 1 ] ) ; p l a y e r s [ i ]. s e t O b j e c t i v e ( R e s o l u t i o n P o l i c y.maximize, O b j e c t i v e s [ i 1 ] ) ; } } p u b l i c s t a t i c v o i d main ( S t r i n g [ ] a r g s ) { CongaSolver cg = new CongaSolver (new CollaborationGame ( 5, 5 ) ) ; cg. s e t C o n s t r a i n t B u i l d e r ( C o n s t r a i n t F a c t o r y.bound CONSTRAINT ) ; Algorithm a=a l g o r i t h m F a c t o r y.complete SEARCH. b u i l d ( cg ) ; a l g o. e x e c u t e ( ) ; } Huawei Technologies Co., Ltd. 1

19 ConGa: a solver for Constraint Games Fully integrated in CP solver Complete tree structured search Based on Choco solver V4 [PFL17] Nash constraint: global constraint to express player s preferences Different filtering power for Nash Constraints [NL14, PL17] Easy to extend Many available and reusable algorithms Price of anarchy Price of stability Pure Nash equilibria enumeration Pareto Nash equilibria Iterated Best Response search Huawei Technologies Co., Ltd. 1

20 Huawei Technologies Co., Ltd. 2 Contents 1 Huawei French Research Center 2 Game theory 3 Constraint Games 4 Solving methods 5 Benchmarks 6 Conclusion and perspectives

21 ConGa 1.0: A complete algorithm [Nguyen & Lallouet 2014] A result by [Gottlob and al, 2005] Nash Constraint N i for Player i encodes deviations, i.e. tuples t = (s i, s i ) such that s i is a best response to s i (not unique) Theorem: i P N i = PNE In Conga 1.0, we compute incrementally the N i First Tree-search algorithm The idea is to traverse all tuples of the search space using a complete ordering of players and values Record each player s undominated strategies Pruning when a tuple has already been proved subject to deviation (complete detection) Pruning when a tuple is NBR at last level Constraint solver is used to compute hard constraints and deviations Huawei Technologies Co., Ltd. 2

22 Huawei Technologies Co., Ltd. 2 ConGa 2.0: A new complete algorithm [PL17] Again using Nash constraints [Gottlob and al, 2005] In Conga 2.0, we filter the N i up to a certain level of consistency Tree-search algorithm Like in Conga 1.0, complete traversal of the search space Each player has an implicit Nash constraint to encapsulate preferences Nash constraint is implemented as a global constraint Arc-consistency is costly compute an approximation Better integration in the solver, better filtering... but no hard constraints

23 Nash constraint filtering: arc-consistency Arc-consistency for N i Nash constraint is of the same arity as the problem Arc-consistency consists in removing from D i values which do not belong to any best response Theorem Deciding if a is a NBR is Π P 2 -complete Proof Intuitively, check that for all assignments of the other players, there exists a better response than a Reduction from a QCSP to a 2-players 0-sum CSG Too complex to be implemented Huawei Technologies Co., Ltd. 2

24 Nash constraint filtering: approximations Approximation #1: filtering only objective values Decision variable values will be filtered from the expression of the objective function If a non best response state has same objective value than a best response state, its values will not be filtered Approximation #2: filtering only objective bounds Other inconsistent values are not filtered Computation of lower bound Mm(S) = max s i A i min s i S i s opti Approximation #3: use AC instead of traversal of S i to evaluate min s i S i s opti Complete search over A i (not S i because deviations are global for a player!) Branch and Bound on lower bound of the objective: operator BB i (S) Huawei Technologies Co., Ltd. 2

25 Huawei Technologies Co., Ltd. 2 Nash constraint filtering in practice Implementation of BB i Launch a new solver instance at choice point of the main search tree Start enumeration of all strategies for Player i to check deviations and remove NBR Theorem Main search tree BB i (S) is a propagator for N i Proof Check that BB is correct, contracting, monotonic and singleton complete Deviation search tree

26 Huawei Technologies Co., Ltd. 2 Special handling of Graphical Games Nash constraints in Graphical Games Use the dependency graph to restrict the arity of the Nash constraints Less wake-up conditions, only useful propagations Dynamic dependencies for free Nash checking for a player when its objective is instanciated Can appear in the middle of a search tree Entails the Nash constraint

27 Huawei Technologies Co., Ltd. 2 Contents 1 Huawei French Research Center 2 Game theory 3 Constraint Games 4 Solving methods 5 Benchmarks 6 Conclusion and perspectives

28 Benchmark suite Gamut: a library of game generators The games (only n-players) (#players,#strategies) MEG: Minimum Effort Game (payoff am be where M is the min effort. Coordination game) TD: Traveller s Dilemma DG: Dispersion Game (anti-coordination game, models load balancing) CG: Collaboration Game (the highest payoffs when all player choose the same action) AR: Arm Race (x and y are arm levels. Payoff is C(x) + B(x y). Symmetric) EFBG: El Farol Bar Game (If less than 60% of the population go to the bar, they ll all have a better time than if they stayed at home) CB: Colonel Blotto Game (distribution of resources along battlefields) (#players,#battlefields,#troops) Huawei Technologies Co., Ltd. 2

29 Huawei Technologies Co., Ltd. 2 The solvers Comparison of different solving techniques enum: this solver performs a complete enumeration of all the search space. This solver is similar to Gambit except that it uses the Constraint Game representation Settings Conga 1.0: this solver uses tables to record best responses and pruning at the last level Conga 2.0: this solver uses filtering on Nash constraints All times are given in seconds Time out is set to 600s

30 Experimental results Comparison of the different algorithms on Gamut games Game Param. #PNE enum Conga 1.0 Conga MEG TD DG CG AR EFBG CB Huawei Technologies Co., Ltd. 3

31 Experimental results Comparison of the different algorithms on graphical games Topology is C (circle, #players), T (tree, #players, #highest degree) and B (complete bipartite, #players) Game Topology #PNE enum Conga 1.0 Conga 2.0 C C T T PGG B B B C C C T TGC T B B B C C C RG T T B B B Huawei Technologies Co., Ltd. 3

32 Huawei Technologies Co., Ltd. 3 Contents 1 Huawei French Research Center 2 Game theory 3 Constraint Games 4 Solving methods 5 Benchmarks 6 Conclusion and perspectives

33 Huawei Technologies Co., Ltd. 3 Conclusion and perspectives Complete and generic solver Full integration in Choco: global constraints, heuristics, etc. Improvement by orders of magnitude for Pure Nash Equilibrium enumeration Symmetries: generalization of symmetry properties to Constraint games Heuristics (for first, and all solutions) Application to SDN

34 Huawei Technologies Co., Ltd. 3 Thank you for your attention Questions?

35 Bibliography I Paul Harrenstein, Wiebe van der Hoek, John-Jules Ch. Meyer, and Cees Witteveen. Boolean Games. In Johan van Benthem, editor, TARK. Morgan Kaufmann, Michael J. Kearns, Michael L. Littman, and Satinder P. Singh. Graphical models for game theory. In Jack S. Breese and Daphne Koller, editors, UAI, pages Morgan Kaufmann, Thi-Van-Anh Nguyen and Arnaud Lallouet. A complete solver for constraint games. In Barry O Sullivan, editor, Principles and Practice of Constraint Programming - 20th International Conference, CP Springer, Charles Prud homme, Jean-Guillaume Fages, and Xavier Lorca. Choco Documentation. TASC, INRIA Rennes, LINA CNRS UMR 6241, COSLING S.A.S., Huawei Technologies Co., Ltd. 3

36 Bibliography II Anthony Palmieri and Arnaud Lallouet. Constraint games revisited. In IJCAI, Albert Xin, Jiang Kevin, Leyton-brown Navin, and AR Bhat. Action-graph games