Parity games and the automaticity of. M. K. Lehtinen Pembroke College

Transcription

1 Parity games and the automaticity of modal µ M. K. Lehtinen Pembroke College Supervised by Prof. A. Dawar A dissertation submitted to the University of Cambridge in partial fulfilment of the requirements for the degree of Master of Philosophy in Advanced Computer Science University of Cambridge Computer Laboratory William Gates Building 15 JJ Thomson Avenue Cambridge CB3 0FD United Kingdom mkl25@cl.cam.ac.uk June 13, 2012

2

3 Declaration I M. K. Lehtinen of Pembroke College, being a candidate for the M.Phil in Advanced Computer Science, hereby declare that this report and the work described in it are my own work, unaided except as may be specified below, and that the report does not contain material that has already been used to any substantial extent for a comparable purpose. Total word count: Signed: Date: Acknowledgements I am grateful for Prof. A. Dawar s supervision. His time, support and guidance were invaluable for the success of this project. This dissertation is copyright c 2012 M. K. Lehtinen All trademarks used in this dissertation are hereby acknowledged.

4

5 Abstract Parity games are an important class of games for model checking. In particular, solving parity games is equivalent to the model checking problem for the modal µ-calculus. The exact complexity of solving parity games in unknown, although it is unlikely to be NP-complete. The first contribution of this project is to survey the various approaches to solving parity games. Automaticity is a measure of structural complexity of properties of finite structures. The automaticity of any property expressible in modal µ is known to be at most polynomial. The complexity of Jurdziński s progress measure algorithm for parity games is related to the automaticity of the winning regions of parity games. This project conjectures that this automaticity is exponential in the number of priorities of the parity game and therefore progress measures are unlikely to yield a polynomial time algorithm for solving parity games. The main contribution is a construction on parity games which shows that a lower bound on a simply stated combinatorial problem is sufficient to establishing a lower bound on the automaticity of modal µ. The best previously known lower bound to the automaticity of modal µ is linear. This project shows that the automaticity of winning regions of parity games of size n with d priorities is Ω(dn log n), improving the known lower bound on the automaticity of modal µ.

6

7 Contents 1 Introduction 1 2 Background: From parity games to modal µ Parity games Modal µ calculus Logics and games First order logic and reachability games Parity games and modal µ Defining the winning regions of parity games Algorithms for parity games A recursive algorithm Overview Complexity Strategy improvement Overview Variants, improvements and complexity Progress measures Overview The algorithm Complexity Variations Family of difficult games for Jurdziński s small progress measure algorithm A lower bound for the automaticity of modal µ Labelling systems The automaticity of modal µ A lower bound for the automaticity of φ d i

8 4.3.1 Acyclic graphs as parity games A combinatorial problem A new lower bound for the automaticity of φ d Discussion 47 ii

9 List of Figures 2.1 A parity game and the players respective winning regions Recursive algorithm from [1] Progress measure lifting algorithm from [2] The odd level for H 4,3. Even vertices are depicted as circles, odd vertices are depicted as squares. The numbers in the vertices are their priorities The first even layer (i = 1) Game graph for H 4, A brief genealogy of algorithms for solving parity games Poorly coloured graphs The graph G(A, A ) iii

10 iv

11 Chapter 1 Introduction Establishing the correctness of a system is a difficult but important task. There are two inherent challenges involved in verification: one is to model properties of real systems; the other is to check whether those properties hold. The difficulty of formally expressing a property can be related to the complexity of verifying it. The field of descriptive complexity studies this relationship. This project focuses on the relation between game-theoretic models and the modal µ calculus. In general, games can model interaction between agents. These might be a program and its environment, distributed systems, or virtually any set of individuals acting independently on a common state. We consider games played on finite graphs on which one or several players move a token from vertex to adjacent vertex according to their respective strategies. A winning condition on the sequence of moves decides the winner. For instance, in order to win a reachability game between Player and Opponent, the Player has to force the token to a winning state. Deciding who has a winning strategy in a game can determine whether a property holds in the system which the game models. Model checking any property expressible as a first order logic formula can be formalised in game-theoretic terms: for any formula and model, there is a reachability game played by Verifier and Falsifiers such that Verifier has a winning strategy if and only if the formula is satisfied by the model. The 1

12 reducibility of model checking to solving games makes game theory a popular topic in verification. This project focuses on parity games and the modal µ calculus. The vertices of a parity game graph are labeled with priorities. The winning player is determined by the parity of the highest-priority vertex which is visited infinitely often in a play. Modal µ is a propositional modal logic with minimal and maximal fix-point operators. Many properties which can not be expressed in first order logic such as fairness, termination and infinite occurences are expressible in modal µ. Describing a property as a sentence of µ and evaluating it on a model of a system can prove whether the system behaves correctly. This is a rich mathematical formalism which is well suited for describing transition systems. Unsurprisingly, the related model checking problem receives a lot attention. Evaluating a formula of modal µ on a finite model can be reduced in polynomial time to deciding the winner of a parity game [3]. The number of priorities in the parity game depends on the number of alternations between least and greatest fix-point operators in the µ-calculus formula. Solving parity games with a fixed highest priority or, equivalently, evaluating formulae of modal µ with fixed alternation depth, can be done in polynomial time. The more alternations are allowed, the more expressive that fragment of modal µ is: the alternation hierarchy is said to be strict [4]. Bounded priority parity games are therefore not sufficient for the general model checking problem for µ-calculus. This motivates the search for a polynomial time algorithm for parity games with unbounded priorities. Chapter 2 introduces parity games, the modal µ calculus and the relationship between them. The complexity of solving parity games and model checking µ-calculus is an interesting one: it is unlikely to be NP-complete yet whether there is a polynomial time algorithm is still an open question. The range of proposed algorithms is varied and Chapter 3 surveys the three main approaches. The first approach is the idea of a divide-and-conquer algorithm, due to McNaughton [5]. It solves the problem recursively for sub-games and is appealing in its simplicity; however it is not competitive in its complexity which is exponential 2

13 in the number of priorities d. The second approach, strategy improvement algorithms, uses a valuation of strategies. The algorithm searches for local changes in the current strategy which would improve its valuation in order to converge to an optimal strategy. Jurdziński s discrete strategy improvement algorithm [6], like the recursive algorithms, is exponential. However, the exponent is smaller, d/2. Unlike other such attempts, it does not suffer from an exponential worst-case space complexity. Until recently, strategy improvement algorithms seemed to be the best hope for a polynomial algorithm. However, since Friedmann [7] proved that such algorithms have exponential worst-case behaviour, all bets are off. The third approach consists of using progress measures, which are functions quantifying how much progress a computation has made. For an enumerative model checking algorithm for example, the proportion of the state-space which the algorithm has checked could be a progress measure. Although Jurdziński uses them to devise an algorithm for solving parity games, the idea of progress measures is deeper and relate to the evaluation of logical formulae. Immerman [8] used a similar measure, stage functions, to prove that the first order fixed point alternation hierarchy collapses at its first level: every fixed point formula can be expressed by a a single least fixed point. Progress measures are related to both the algorithmic complexity of model checking and the expressivity of logics. Section 4.2 relates Jurdziński s algorithm for solving parity games with the structural complexity of modal µ. The measure of structural complexity of a property is its automaticity. Automaticity first described the complexity of languages on strings but Dawar and Kreutzer [9] generalised the idea to finite structures. The authors introduce labelling systems as a way of measuring the complexity of properties of finite structures and proving differences in the expressivity of different logics. Using this construction, they showed that any property expressible in modal µ has at most polynomial automaticity and that there are properties with no less than linear automaticity. However they conjecture a stronger result: for any polynomial p there would be a property of µ-calculus with automaticity Ω(p). Chapter 4 describes the main contribution of this project: how to use 3

14 parity games to establish a lower bound to the automaticity of modal µ. We prove that labelling system for parity games have to satisfy a simple set of constraints. This reduces the question of automaticity of modal µ to a purely combinatorial problem, which we state is Section Finally, we establish a new lower bound to the automaticity of modal µ. Section shows that the property true in winning regions of parity games of maximal priority d has automaticity Ω(dn log n). This improves the previously known linear lowr bound to the automaticity of modal µ. 4

15 Chapter 2 Background: From parity games to modal µ 2.1 Parity games A parity game is an infinite two-player game played on a finite game graph of which the vertices are partitioned between the two players and annotated with priorities. The Even Player and her opponent, the Odd Player, move a token along the edges of the graph. A play consists of the infinite sequence of vertices visited by the token. The winner is decided by the parity of the highest priority which the token visits infinitely often: the Even Player wins if the largest priority visited infinitely often is even; otherwise the Opponent wins. This section gives a formal definition of parity games, strategies and what it means to decide the winner of a parity game. Definition 1. A game graph G = (V, V even, V odd, E, p) is a directed graph (V, E) of which the vertices are partitioned into V even and V odd and annotated with a priority by the function p : V N. Every vertex is required to have at least one outgoing edge. Without loss of generality, we consider priority functions p : V {0, 1,..., d} 5

16 of highest priority d, where d is even. If V is the set of vertices of a game graph G and W is a set of vertices in V, we write G W for the induced subgame on the vertices of W : (W, V even W, V odd W, {(u, v) (u, v) E u, v W }, p : W d) where p (v) = p(v) for all v W. For G W to be a well formed game, every vertex in W must have at least one successor in W. Write G W for the induced game G (V W ). It is well formed if no vertex in V W has all its successors in W. Definition 2. A play on a game graph G starting from a vertex v 0 V is an infinite path v 0 v 1... such that i.(v i, v i+1 ) E. For σ {even, odd}, a play is winning for Player σ if the highest priority which the play visits infinitely often is of parity σ. If σ is even then write σ for odd and vice versa. Definition 3. A strategy τ for Player σ maps a finite prefix of a play v 1...v n where v n V σ to a successor vertex u such that (v n, u) E. A play v 0 v 1... conforms to a strategy τ for Player σ if whenever v i V σ it is the case that v i+1 = τ(v 0 v 1...v i ). Given two strategies, one for Player σ one for Player σ, they uniquely define a play from each vertex of the game graph. Definition 4. A strategy for Player σ is winning if for any opposing strategy the resulting play is winning for σ. The winning region, win σ (G) V, is the set of vertices from where Player σ has a winning strategy. Definition 5. A set of vertices D V is a σ-dominion if Player σ has a winning strategy which does not leave D from every vertex of D. The Player σ s winning region, win σ (G) is a σ-dominion. Definition 6. A strategy τ for Player σ is memoryless if for u, v V σ we have τ(u 0 u 1...u) = τ(v 0 v 1...v) whenever u = v. Then τ : V σ V is a successor function for vertices in V σ : the player s move depends only on the current position of the game. Theorem 1 (Positional determinacy [10]). Parity games are positionally determined: win odd (G). every vertex of a parity game G is either in win even (G) or 6

17 0 2 3 Winning region for the Opponent Winning region for the Player Player (even) Opponent (odd) Figure 2.1: A parity game and the players respective winning regions If a vertex v is in win σ (G) and τ is the winning strategy, then let τ (v) map to τ(v), the τ-successor for the game prefix of length one starting at v. τ is also a winning strategy. More generally, memoryless strategies are sufficient in parity games. From now, all strategies will be memoryless. To solve a parity game locally is to determine who has a winning strategy from a starting vertex v. To solve a parity globally is to solve it at every starting vertex v. 7

18 2.2 Modal µ calculus Modal µ calculus is a modal logic with fixpoint operators. It is evaluated on finite transition systems which can model process behaviour. The modal operators and allow universal and existential reasoning about successor states. A sentence x is true at a vertex if it has a successor at which x holds. The least fixpoint operator µ provides a mean to reason about transitive closure and express global system properties such as reachability. For instance, p x is true for a state if either p holds or there is a successor at which x holds. The µ operator can bind x to express the closure over the edge relation of the transition system: µx.p x is true for the least set of vertices X for which either p holds or they have a successor in X. This is the set of vertices from which there is a path to a vertex where p holds. The sentence νx.p x on the other hand is true for the greatest such set X. This is the set of vertices from which there are no finite paths on which p does not hold. In general, µ-sentences correspond to liveness properties (eventually something happens) while ν-sentences express safety (something always happens). Similarly, until and while can be expressed using µ and ν respectively: µ expresses finiteness and ν infinite looping. Alternating greatest and least fixpoint operators, µ and ν, increases the expressibility of sentences of modal µ to properties such as infinite repetition and fairness. However, the cost is comprehensibility: sentences with more than two fixpoint operators are notoriously difficult to make sense of. This section defines formally the syntax and semantics of modal µ. Syntax and semantics of modal µ A modal µ formula is defined inductively over a set of propositional variables AP and a set of variables X. Let p denote a propositional variable in AP and x a variable in X. Well formed formulae of µ-calculus are defined as follows: φ ::= p p x φ φ φ φ φ φ µx.φ νx.φ 8

19 Note that we restrict the syntax to sentences in negation normal form where negations only appear at propositional variables. This will simplify the reduction to parity games and does not reduce the expressivity of the logic. Indeed every modal µ formula without this restriction is equivalent to a formula in negation normal form. A sentence of modal µ is evaluated on transition system, represented as a finite Kripke structure M = (S, R, L) with starting state s S where S is a set of states, the universe, R : S S is the binary edge-relation and L assigns a set of propositional variables to each state. The semantics of modal µ are defined inductively on the structure of the formula. The sentence p holds at a state s if p L(s); p holds at s whenever φ does not hold at s; φ ψ holds at s if either φ or ψ holds at s; φ ψ holds at s if both φ and ψ hold at s; φ holds at s if there is a successor state s such that (s, s ) R and φ holds at s ; φ holds at s if φ holds at every successor s of s, (s, s ) R; µx.φ holds at s if s is in the least set X for which φ[x/x] = X where φ[x/x] denotes φ with every instance of x substituted by X; νx.φ holds at s if s is in the greatest set X for which φ[x/x] = X. We write K, s φ whenever the formula φ holds at the state s of a finite structure K. To evaluate a fixpoint formula φ = µx.ψ, we define the operator ψ n induction: by ψ 0 = ψ[ /x]; ψ n+1 = ψ[ψ n /x]. 9

20 Setting φ = ψ i, i N gives the semantics of the µ formula φ. Greatest fixpoint formulae φ = νx.ψ are defined analogously: ψ 0 = ψ[ /x] where is true at all states; ψ n+1 = ψ[ψ n /x]. Setting φ = ψ i, i N gives the semantics of the µ formula φ. The number of alternations between least and greatest fixpoint operators is an indication of the complexity of a modal µ formula. The alternation depth formalises this measure, taking into account only truly dependent operators. Definition 7. A formula φ is said to have alternation depth 0 and be in Σ 0 and Π 0 if it contains no fixpoint operators. Σ n+1 is given by: Σ n Π n Σ n+1 ; If φ, ψ Σ n+1, then φ ψ, φ ψ, ψ, ψ Σ n+1 ; If φ Σ n+1 then µx.φ Σ n+1 ; If φ(x), ψ Σ n+1 then φ(ψ) Σ n+1 provided that no free variable in ψ is bounded by a fixpoint variable in φ. Define Π n+1 analogously, using ν for µ and Π n for Σ n. The alternation depth of a modal µ formula φ the least n for which φ Σ n Π n. For every fixpoint variable x in ψ, bounded by a fixpoint operator λ, write alt(x) for the alternation depth of λx.φ, the subformula of ψ which binds x. If we consider modal µ with bounded alternation depth, the more alternations are allowed, the more expressive the logic. That is to say, for any d, there are properties which require more than d alternations to be expressible in modal µ. The alternation hierarchy of modal µ calculus is said to be strict [4]. 10

21 2.3 Logics and games Deciding of the validity of a logical statement can be thought of as a two player game between the Verifier and the Falsifier. The Verifier attempts to prove that the statement is true and the Falsifier tries to prove that it is false. Who has a winning strategy depends on the validity of the statement. We formalise this as a model checking game where the players move on a game graph of which the vertices are sub-formulae of the original statement. To evaluate a statement in some logic, it is sufficient to decide who has a winning strategy in the corresponding game. This section first exemplifies the relation between logics and games by showing how a first order formula is equivalent to a reachability game. The same idea is then extended to modal µ calculus and parity games First order logic and reachability games A first order sentence φ in negation normal form can be translated into a reachability game. The initial node is the full formula φ and other nodes are deribed from sub-formulae with values from the domain assigned to every free variable. The atomic formulae and negated atomic formulae which evaluate to true are winning for the Verifier while those which evaluate to false are winning for the Falsifier. The game is a reachability game: Verifier must try to force the game to a true vertex, otherwise Falsifier wins. The verifier decides the move from existentially quantified and disjunctive vertices, while the Falsifier decides the move from universally quantified and conjunctive vertices. There is an edge from a disjunction to each sub-formula and an edge from an existentially quantifies vertex to every immediate sub-formulae where the existentially quantified variable take a value from the domain. If a disjunction or an existentially quantified formula holds, the Verifier can make a move to the sub-formula which witnesses this. Universally quantified and conjunctive vertices belong to the Falsifier. A conjunction has an outgoing 11

22 edge to both sub-formulae and a universally quantified vertex has an outgoing edge to every immediate sub-formula where the bound variable has taken a value from the domain. If the formula does not hold, the Falsifier can take a move to the sub-formula which provides a counter example. Conversely, we can attempt to describe the winning region of a game by a statement φ in some logic, such that only the vertices in the Verifier s winning region satisfy φ in the structure given by the game graph. However, the expressivity of first order logic is limited to local properties. Reachability, along with connectivity and other global properties is not first-order definable. Defining the winning regions of the reachability game therefore require a more expressive logic than first order logic Parity games and modal µ The model checking problem for modal µ is to decide for a formula ψ of modal µ and a finite structure K with starting state s whether K, s φ. This problem is reducible in polynomial time to the problem of solving a parity game. This section shows how, given ψ and K, we can obtain a parity game G with initial vertex v for which the Even Player has a winning strategy from v if and only if ψ holds for K at s. Every vertex of G corresponds to a pair S sub(ψ): a state of the Kripke structure and a sub-formula of ψ. The parity and outgoing edges of a vertex depend on the sub-formula φ at each node v = (s, φ). If φ is an atomic proposition or the negation of one, then there is an edge (v, v) and if φ L(s) then v belongs to the even player and its parity is even. Otherwise, it belongs to the Odd Player and its priority is odd; if φ is a conjunction φ 0 φ 1 then there is an edge to (s, φ 0 ) and an edge to (s, φ 1 ). The vertex belongs to the Odd Player and its priority is 0; if φ is a disjunction φ 0 φ 1 then these is an edge to (s, φ 0 ) and an edge 12

23 to (s, φ 1 ). The vertex belongs to the Even Player and its priority is 0; if φ is a modal formula ψ then there is an edge to every vertex (s, ψ) such that (s, s ) is in R. If no such successor state exist, there is an edge to v. The vertex belongs to the Odd Player and has priority 0; if φ is a modal formula φ then there is an edge to every vertex (s, φ ) such that (s, s ) is in R and if no such state exists then there is an edge to the vertex itself. The vertex belongs to the Even Player and its priority is 1; if φ is a greatest fixed point formula νx.ψ(x) then there is an edge to (s, x). The vertex has priority 0; if φ is a least fixed point formula µx.ψ(x) then there is an edge to (s, x) and the vertex has priority 0; if φ is a variable x bound by some fixpoint operator in a sub-formula λx.ψ where λ is µ or ν, there is an edge to the vertex (s, ψ). The vertex v has priority alt(x). Only the vertices which are reachable from the initial vertex are kept in the game graph. In this construction fixpoint variable receive their alternation depth as their priority. Variables bounded by ν have even priority while priorities bounded by µ have an odd priority. Inner variables have smaller priorities than outer variables and every loop in which a fixpoint variable appears, the highest priority is the priority of a fixpoint variable. As a result, a play can either end up looping at a single node of priority 0 or 1 or it can end up in a loop where the highest priority is a that of a fixpoint variable. Then, whether that variable is bounded by ν or µ decides the winner of the play. The number of priorities in the game depends on the alternation depth of the formula. This is significant since the complexity of known algorithms for parity games is exponential but the exponent depends on the number of priorities. For games with bounded priority, these are polynomial algorithms. 13

24 However, since the alternation hierarchy of modal µ is strict, this is not sufficient for the general case Defining the winning regions of parity games Thus far we have established that the model checking problem for a sentence of modal µ with alternation depth d on a finite structure with starting state s can be reduced in polynomial time to deciding the winner of a parity game of maximum priority d. The structure satisfies the formula if and only if the even player has a winning strategy from the initial vertex. Conversely, we can define the winning region of a parity game G of bounded maximal priority d with a modal µ formula φ d such that G, v = φ D if and only if the Even Player has a winning strategy from v in G. We describe parity games as a finite structure G = (V, V 0, V 1, E, P 0,..., P d ) where P q is the set of nodes of priority q. If d is the maximal priority in the game G the sentence φ d is given by: νx 0.µX 1...νX d. i d(v 0 P i X i ) (V 1 P i X i ) For a parity game G, φ d is true for vertices in the winning region of the Even Player. The correctness of this result can be shown by reducing G φ d to solving a parity game G. We can simplify G by removing moves which are clearly not in the winning strategy of either player and collapse non-branching paths. This does not affect the winning region of G which collapses to G. Unlike first order logic, modal µ is expressive enough to define the winning region of parity games of fixed maximal priority. 14

25 Chapter 3 Algorithms for parity games The following sections outline three distinct approaches to solving parity games: recursive algorithms, strategy improvement algorithms and progress measure algorithms. A representative instance exemplifies each family of algorithm, followed by a brief discussion of variations thereupon. The complexity of progress measure algorithms is closely related to the structural complexity of parity games so Jurdziński s progress measure algorithm is given particular attention. In particular, Section details a family of parity games which yields exponential running time for this algorithm. Although the different approaches are presented separately, the currently best performing algorithms combine several of the three. The brief genealogy of parity game algorithms in Figure gives an overview of the relations between different algorithms. 15

26 3.1 A recursive algorithm Overview Zielonka s inductive algorithm for parity games [11] is based on McNaughton s algorithm [5] for omega regular games, a more general class of games. The algorithm requires computing the set of vertices from where a player can force a play into the set of vertices of highest priority and then solving the game for the remaining portion of the game. Definition 8. A set of vertices B V is said to be σ-closed if player σ can always choose to stay in B and σ can not escape from it. v B V σ. u B.(v, u) E v B V σ. u V.(v, u) E u B Since parity games are positionally determined, the set win σ (G) is σ-closed. Definition 9. Given a set of vertices A V, the σ-attractor of A is the set of vertices from which Player σ can force a play into A at least once in zero or a finite number of steps. We write attr σ (A, G) for the σ-attractor of A in a game G and attr σ (A) whenever G is clear from context. The σ-attractor of a set can be computed in linear time to the number of edges in the game. The set itself is trivially in the attractor. Each iteration adds every vertex from V σ which has a successor in the set computed so far and every vertex from V σ which has all its successors in the set computed so far. Lemma 1. Given a set of vertices A V, the set V attr σ (A) is σ-closed. If Player σ can escape from a vertex v into attr σ (A) then v is in attr σ (A). That is to say, every vertex in both V attr σ (A) and V σ has all its successors in V attr σ (A); every vertex in both V attr σ (A) and V σ does not have all its successors in attr σ (A). Since all vertices of V are required to have at least one successor, all vertices of V attr σ (A) have at least one successor which is not in attr σ (A). 16

27 Lemma 2. Given a parity game G, if W win σ (G) and W = attr σ (W ), then win σ (G) = win σ (G W )) W and win σ (G) = win σ (G W ) This follows from the observation that W is in win σ (G). Lemma 3. Let d be the highest priority in a parity game G, let A be the set of vertices of priority d and σ be the parity of d. Let G = G attr σ (A). Then win σ (G ) win σ (G). Furthermore, if win σ (G ) = then win σ (G) = V. From Lemma 1 Player σ can not force the game out of the vertices of G. A winning strategy for Player σ in G is therefore also winning in G. Furthermore, if Player σ can not win in G, Player σ has a winning strategy for G, τ, from every vertex of G. Then Player σ has a winning strategy in G: follow τ from the vertices of G and otherwise force the game into A. Then, if the play exits G an infinite number of times then the play visits A an infinite number of time and d is the maximal priority visited infinitely often. Otherwise, the game eventually stays in either attr σ (A) or in G. In either case, Player σ wins by either visiting A infinitely often or by following τ. Figure shows an algorithm which recursively solves a parity game and returns the winning regions of σ and σ. The base case consists of there being a single colour, 0. In this case V is a winning region for Player even. The algorithm returns (V, ). Otherwise, the algorithm takes n to be the highest colour in the game and σ its parity. It initialise W σ to be and recursively finds the winning regions (W 0, W 1) of the induced subgame G attr σ (A). If Player σ wins for all the positions in the induced game, the algorithm terminates, returning all vertices except those in W σ as W σ. Otherwise, it computes the σ-attractor of the winning region W σ and repeats the recursive step on the subgame G attr σ (W σ). 17

28 Figure 3.1: Recursive algorithm from [1] 18

29 3.1.2 Complexity The complexity of this algorithm is bounded by an exponential O(n d ). Furthermore, Firedmann [1] showed that there is a family of parity games of linear size for which this algorithm has an exponential running time. Note that while this algorithm computes the attractor of the maximal priority vertex set, any dominion can be used instead without affecting the correctness of the algorithm. The larger the dominion, the fewer recursions the algorithm will perform. The algorithm can therefore be optimised to take larger steps by searching in O(n l ) dominions of size l. The optimal value for l is 2n (rounded above) which reduces the running time of the overall algorithm to n O( n). This is the idea behind Schewe s [12] big step algorithm. 3.2 Strategy improvement The idea of strategy improvement algorithms is to define a valuation of strategies which either witnesses that a strategy is winning or can be used to improve it. Then, by evaluating a strategy and comparing it to strategies which differ on a single vertex, the algorithm can use local changes to converge towards an optimal strategy. A specific strategy improvement algorithm specifies an ordering of strategies and a policy to choose which improvement to implement. The ordering must be easily computable and the improvement policy must be well-behaved in the sense that after sufficient iterations, no further improvements are possible and the strategy is optimal. This section sketches one such algorithm, due to Vöge and Jurdziński [13] Overview We define an order of preference on plays which relies on three characteristics: the highest priority p appearing infinitely often in a game, the set of priorities 19

30 greater than p which occur in the play and the index of the first vertex of priority p which is not followed by vertices of greater priority. The order indicates which priorities are more favorable for the Even Player. It extends to plays on which it indicates which plays are better for the Even Player. To evaluate strategies, assume that the Odd Player is playing optimally: the counter strategy the Odd Player uses yields the worst play which confroms to the Even Player s strategy. The Even Player s strategies are ordered according to the valuation of this play. Definition 10. For priorities i and j, define i j if either: (i) i is odd and j is even, or (ii) i and j are even and i j, or (iii) i and j are odd and j i. This ordering extends to plays. Let π and π be two plays and the values p and p are the highest priorities appearing in π and π infinitely often; Q and Q are their respective sets of priorities higher than p and p. Let e and e be the indices at which the vertices of priority p and p which are visited infinitely often first appear. All following priorities are lesser or equal to p and p in their respective plays. Definition 11. π π if either : (i) p p, or (ii) p = p and max(p P ) is odd and in P or even and in P, or (iii)p = p, P = P and either p is odd and e < e, or p is even and e > e. Let τ and τ be strategies. Let π τ (v) be the worst play from v which conforms to τ. That is to say, π τ (v) is the play from v where one player uses strategy τ and the opponent uses the optimal strategy on the induced one-player game. The order extends to strategies. Definition 12. τ τ if and only if π τ (v) π τ (v) The strategy improvement algorithm starts with an arbitrary strategy. For each vertex, it finds the worst play for σ which is consistent with Player σ s strategy and uses it as the valuation of that vertex. If σ wins the play, the vertex is in the winning region of Player σ. If the strategy is not winning 20

31 from a vertex but there is a better successor vertex than the current one, the strategy function is changed in order to improve the strategy. While the strategy can be locally improved at some vertex, the algorithm does so, eventually reaching a stable strategy which is also optimal. That is to say, the final strategy for Player σ is winning from all vertices of win σ (G). This algorithm depends on an efficient computation of the opponent s optimal strategy once the strategy of Player σ is fixed. This is a much easier task of solving the one player parity game induced by fixing one strategy. An optimal counter-strategy exists and can be computed in O(nmd) according to Björklund [14]. For the algorithm to be sound the ordering of strategies must have the following properties. Local optima must be global. That is to say, if a strategy can not be improved by changing the successor at some vertex, the strategy must be globally optimal. Then, the strategy at termination can not be improved. A globally optimal strategy must be winning at all vertices of σ s winning region. For the valuation to be useful, it must be consistent in subgames: the optimal counter-strategy in the subgame induced by fixing a strategy must also be optimal in the original game Variants, improvements and complexity Strategy improvement algorithms have been considered to be the most likely family of algorithms to eventually yield a polynomial running time. In fact, until recently, it wasn t clear whether there was already a polynomial strategy improvement algorithm which was simply missing a proof of the upper bound. Unsurprisingly, significant efforts have gone into tightening the upper bound on the running time of strategy improvement algorithms. The running time of Vöge and Jurdziński s [13] algorithm above can be improved by reducing the maximum the number of iterations. Björklund, Sandberg, and Vorobyov [14] use a more precise evaluation measure on strategies, guarantees fewer iterations and faster convergence to an optimal strategy. 21

32 Instead of evaluating a play based on the winning priority, the set of higher priorities and the length of the prefix of a play, Björklund et al. also records the number of vertices of each priority. This guarantees that the longest sequence of improving strategies is exponential in the number of priorities and not in the number of vertices. Furthermore, the authors integrate a randomized scheme for linear programming due to Kalai [15] and Ludwig [16] to limit the number of improvement steps. Schewe [17] presents a strategy improvement algorithm which takes into account the global effect of a local improvement, thus making it optimal in a precise sense. Vöge and Jurdziński s algorithm considers the current valuations of successor nodes when evaluating it, which may depend on the current choice. Schewe s scheme takes into account cross-effects of simultaneous strategy modifications. Friedmann [18] proposes a local strategy improvement algorithm which considers the slightly weaker question of deciding the winner and optimal strategy for a single vertex. This algorithm is local in the sense that it explores the game graph starting from the input vertex, expanding it lazily and terminates as soon as it discovers a winning strategy for one of the players. This task is not significantly easier since any algorithm which solves the game locally at one vertex also solves it at every vertex reachable by the Opponent given a winning stragtegy of the Player. However, it is closer to the model checking poblem for modal µ calculus: evaluating a formula of modal µ on a model only requires solving a parity game locally. However, Friedmann [7] proved an exponential lower bound to strategy improvement algorithms by constructing a family of games displaying worst case behaviour. This answers the long standing question of the complexity of strategy improvement algorithms which were considered a candidate for a polynomial time algorithm for parity games. 22

33 3.3 Progress measures Overview The progress measure algorithm constructs a labelling which assigns either or a tuple to every vertex. The tuple counts the vertices of odd priority higher than the priority of the current vertex until a vertex of high even priority. If the tuple is finite, then this witnesses the existence of a winning strategy for the Even player. The labelling is called a progress measure. This sections gives a brief overview of the algorithm before a formal definition and proof of correctness. The algorithm initially assigns a tuple (0,..., 0) of length d to every vertex. 2 Each component corresponds to two priorities, one even and one odd. When comparing the tuple assigned to a vertex and its successor, only the most significant components are compared, up to the component corresponding to the priority of the vertex. There are four distinct type of vertices: even vertices of even parity, odd vertices of even parity, odd vertices of odd parity and even vertices of odd parity. A vertex is in conflict in each of the four cases: an even vertex of even parity has a progress measure strictly smaller than all of its neighbours; an even vertex of odd parity has a progress measure equal or smaller than all of its neighbours; an odd vertex of even parity has a progress measure strictly smaller than some of its neighbours; an odd vertex of odd parity has a progress measure equal or smaller than some of its neighbours. Lifting the progress measure of a conflicted vertex resolves a conflict by increasing the tuple to the least tuple which is not in conflict: the component corresponding to the priority of the vertex increases and the less significant 23

34 components are 0. If any vertex is lifted more than n times, the Odd Player has a winning strategy from that vertex. It is then assigned. When no conflicts arise the algorithm terminates with a game parity progress measure. Then, the Even Player has a winning strategy from every vertex which is not assigned The algorithm This section assumes, without loss of generality, that the highest priority d is even. A progress measure is a labeling of vertices which witnesses the existence of a winning strategy for the Even Player. Construct a function ξ : V N d 2 between vertices of the game graph and a tuple of values, one for each odd parity, from d component functions 2 ξ d 1,..., ξ 3, ξ 1 : V N such that ξ(v) = (ξ d 1,..., ξ 3, ξ 1 ) for all v in V. Let ξ [q] (v) stand for the truncated tuple (ξ d 1,..., ξ q ) if q is odd and (ξ d 1,..., ξ q+1 ) if q is even. Definition 13. ξ : V N is a parity progress measure if for every successor w of every vertex v : ξ [p(v)] (v) ξ [p(v)] (w) if p(v) is even; ξ [p(v)] (v) > ξ [p(v)] (w) if p(v) is odd. where is the lexicographical ordering of tuples. That is to say a node s progress measure has to be at least as large as that of its largest successor, and larger if the node s priority is odd. In an infinite play, every occurrence of an odd priority q must decrease the q component or a more significant component of the progress measure. The existence of a parity progress measure guarantees that on any infinite play starting from a vertex v, the number of occurrences of an odd priority q before a priority higher than q is finite. Hence the maximum priority which occurs infinitely often must be even. 24

35 Definition 14. A game parity progress measure is a function ξ : V N such that for every vertex v in V : if v V even, then ξ [p(v)] (v) min (v,w) E ξ [p(v)] (w) if p(v) is even; ξ [p(v)] (v) > min (v,w) E ξ [p(v)] (w) if p(v) is odd. otherwise, ξ [p(v)] (v) max (v,w) E ξ [p(v)] (w) if p(v) is even ξ [p(v)] (v) > max (v,w) E ξ [p(v)] (w) if p(v) is odd. That is to say, if the even player s strategy is to minimise ξ and the odd player s strategy is to maximise ξ, then ξ is a progress measure if in the plays v 0 v 1... conforming to these two strategies, the series ξ(v 0 ), ξ(v 1 )... is decreases as described. Then, the even player s strategy which minimises ξ is winning: the maximal priority occurring infinitely often is even. This construction witnesses a winning strategy for the Even Player from every vertex. A more flexible measure extends the co-domain of ξ with a top element to indicate that the Odd Player has a winning strategy from a vertex. Then, rom every vertex, either ξ(v) = or the conditions of Definition 14 hold. Then, write dom(ξ) for the vertices v in V for which ξ(v). If ξ : V N d 2 { } is a game parity progress measure, then dom(ξ) win even (G). To obtain small parity progress measures on an even-dominion D, each ξ q is bounded by the number of nodes of parity q in D. Write M D { } for the restricted co-domain. Lemma 4. There is a game progress measure ξ : V M V { } such that dom(ξ) is the winning set of Player even. Proof: We construct a game parity progress measure by induction on the size of the game. If a parity game has a single vertex, the theorem holds trivially. For the induction step, consider a game of vertex count larger than one. If odd wins from all vertices, ξ(v) = is a game progress measure. Otherwise let W = win even (G). If W V, the induction hypothesis is that there is a game progress measure ξ : W M W { } for G W. Assigning 25

36 to every vertex in V W gives a game progress measure for G. Consider the case where W = V and the highest priority d is even. Let D be the set of verticed of priority d. By induction hypothesis, there is a progress measure ξ : V (attr σ (D)) M V attrσ(d) for the induced game G attr σ (D). There is also a progress measure ξ : attr σ (D) M attrσ(d) for the induced game G attr σ (D). The function ξ (ξ + (n d 1,..., n 1 )) : V M V where n q is the number of vertices of priority q in attr σ (D) is a progress measure for G. Finally, consider the case where W = V and the highest priority d is odd. Let D be the set off vertices of priority d and let A = attr σ (D). The induction hypothesis provides a progress measure ξ : A M A for the induced game G A and a progress measure ξ : V A M V A for the induced game G A. The function ξ (ξ + (n d,..., n 1 )) where n q is the number of vertices of priority q in V A is a small progress measure for G. Then define P rog, a function which gives a least progress measure for a tuple given a neighbouring measure. Definition 15. Let g : V M G { } and (v, w) E for a game G with vertices V and edges E. Then P rog(g, v, w) is the least tuple m in M G such that m p(v) g(w) if p(v) is even and m > p(v) g(w) otherwise, or m = g(w) =. In other words, it is the least tuple which satisfies the parity progress measure conditions locally. Definition 16. The Lift operator is defined as follows: Lift(g, v)(u) = g(u) if u v; for a vertex v V odd let Lift(g, v)(v) = max{g(v), max (v,w) E P rog(g, v, w)}; for a vertex v V even let Lift(g, v)(v) = max{g(v), min (v,w) E P rog(g, v, w)}. Progress measures are ordered by such that g f if g(v) f(v) for all vertices v. Lift(, v) is monotone with respect to for every vertex v V. The algorithm computes the progress measure as a least fixed point Lift. 26

37 Figure 3.2: Progress measure lifting algorithm from [2] Complexity The algorithm increases the value of ξ for v from (0,..., 0) to the least tuple which satisfies the requirements to be a parity progress measure until it is a game parity progress measure. The complexity of the algorithm depends on the number of iterations it performs. The small co-domain is of size O( n 0.5d ). At every iteration, the algorithm can consider all O(m) successors 0.5d in the game graph. The comparison of tuples can be done in O(d) time. The overall algorithm is O(dm( n 0.5d )) = O(n 0.5d+O(1) ). 0.5d Variations The progress measure algorithm can be adapted for computing small dominions of fixed size (or establishing the lack thereof) by limiting the co-domain to tuples which elements are bounded so that the sum of these bounds does not exceed a fixed value. Schewe [12] uses this method instead of brute force to improve the recursive algorithm: by efficiently generating dominions of optimal size, the recursive algorithm needs fewer iterations Family of difficult games for Jurdziński s small progress measure algorithm Jurdziński s progress measure algorithm [2] has a complexity bound of O(dm( n d ) d 2 ). For any n and d, d n, there is a family of games of size O(n) and priorities no larger than d which forces the algorithm to go through n d 2 d lifts for n d vertices. 27

38 For every b and l in N, construct a game H l,b of 2bl+l vertices with maximum priority 2l 1. This game has one odd level and l 1 even levels such that the odd player wins from the odd level only. That is to say, the progress measure for the vertices of the odd level is. The construction forces the lift operator to proceed in small steps: the progress measure can only be incremented to the immediate successor in the tuple-order. The odd level consists of b+1 even vertices e 0 1,..., e 0 b+1 of priority 0 and b odd vertices o 1 1,..., o 1 b vertices of priority 1. These are connected by undirected edges (e 0 1, o 1 1), (o 1 1, e 0 2),..., (o 1 b, e0 b+1 ) so that even and odd parities alternate. Figure 3.3: The odd level for H 4,3. Even vertices are depicted as circles, odd vertices are depicted as squares. The numbers in the vertices are their priorities. Every even level i (for 1 i l 1) consists of b even vertices e 2i 1,..., e 2i b of priority 2i and b + 1 odd vertices o 2i 1,..., o 2i b+1 also of priority 2i. Similarly to the vertices of the odd level, these vertices are connected by undirected edges (o 2i 1, e 2i 1 ), (e 2i 1, o 2i 2 ),..., (e 2i b, o2i b+1 ) alternating even and odd vertices. Furthermore, there are b additional vertices a1 2i+1,..., a 2i+1 b of in and outdegree one. Two directed edges (o 2i j, a 2i+1 j ) and (a 2i+1 j, e 2i j ) connect every vertex a 2i+1 j to the even level i. Since there is a single outgoing edge from each of these vertices, their parity does not matter. In the depicted example (Figure 3.3.5), they are given to the even player. Figure 3.4: The first even layer (i = 1). 28

39 Every even level i is connected to the odd level by b undirected edges (e 2i j, o 1 j) between the even vertices of the even level and odd vertices of the odd level. Figure 3.5: Game graph for H 4,3. In this game, the Odd Player wins from the odd level by forcing the game to stay in the odd level. Since the even player can not escape to another level and the maximal parity is 1, the Odd Player wins. The even player wins from the even levels by forcing in a finite number of turns the play into a loop of even priority, here at the right of the game graph. Let g be the progress measure which the algorithm maintains. For every vertex w connected to o 1 i the edge is undirected. Therefore, according to the definition of P rog, it is the case that P rog(g, w, o 1 i ) [p(w)] g(o 1 i ) [p(w)] and P rog(g, o 1 i, w) [1] > g(w) [1]. Then at every iteration P rog(g, w, o 1 i ) [p(w)] = g(o 1 i ) [p(w)] and all less significant elements are 0. Since the priority of o 1 i is 1, the Lift operator increments g(o 1 i ) to the immediate successor only. 29