Algebraic Dynamic Programming. ADP Theory II: Algebraic Dynamic Programming and Bellman s Principle

Transcription

1 Algebraic Dynamic Programming Session 4 ADP Theory II: Algebraic Dynamic Programming and Bellman s Principle Robert Giegerich (Lecture) Stefan Janssen (Exercises) Faculty of Technology Sommer robert@techfak.uni-bielefeld.de

2 Another thought on symbolic evaluation Think of a calculation that returns some result say a search for a minimmal answer from some search space yields the number Several operations have led to this result, starting from the input data, which were (say) 42 3, 4, 5, 6, 7, 8 Let s assume we executed this calculation symbolically we would obtain (say) 3 + min(4 10, 5 + min( , 8 9)) Written as a term/tree in a appropriate signature this is plus(3, min(times 10 (4), plus(5, min(plus(6, times 4 (7)), times(8, 9)))))

3 Distribution of choice over evaluation Since min is monotone with respect to + and, we can move it to be the last operation, obtaining min[ , , ] Here we evaluate all candidates separately this means increased effort. The initial formula finds the minimum without making the computations shown in green here.

4 Combinatorial explosion Typically we face the following situation: the size of the formula with min applied to sub-formulas is polynomial in the size of the input the expanded list of all candidates with min applied once at the top has exponential size. The early choice implies that only one (or a few), optimal result(s) for a sub-problem must be stored in a table for later consideration. Dropped candidiates are not evaluated further.

5 Combinatorial explosion Typically we face the following situation: the size of the formula with min applied to sub-formulas is polynomial in the size of the input the expanded list of all candidates with min applied once at the top has exponential size. The early choice implies that only one (or a few), optimal result(s) for a sub-problem must be stored in a table for later consideration. Dropped candidiates are not evaluated further. Consequence: If we cannot distribute the choice function towards the inside of the candidates, we cannot solve the problem efficiently.

6 Bellman s Principle Let X, X i, Y denote lists over the value domain of a Σ-algebra A. A satisfies Bellman s Principle if for all operators f in Σ, h[f (x 1,..., x n ) x 1 X 1,..., x n X n ] = h[f (x 1,..., x n ) x 1 h(x 1 ),..., x n h(x n )] For all lists X and Y, h([]) = [] and h(x ++Y ) = h(h(x )++h(y ))

7 Remarks on formulation of Bellman s Principle The functions f may also take arguments from A which come from the input, as in match(a, x, b) = x + w(a, b). In that case, X 1 = [a] and X 3 = [b], where a and b are characters from the input strings and not from lists of subproblem solutions. The second equation implies for Y = [ ] idempotence of h h(h(x ) = h(x )

8 The most common special case Most often we have f (x) = x + c, or f (x, y) = x + y, or f (x) = some other function on numbers h = min or h = max In this case, Bellman s Principle specializes to monotonicity of f : x y f (..., x,...) f (..., y,...)

9 The most common special case Most often we have f (x) = x + c, or f (x, y) = x + y, or f (x) = some other function on numbers h = min or h = max In this case, Bellman s Principle specializes to monotonicity of f : x y f (..., x,...) f (..., y,...) For h = min k or h = max k, Bellman s Principle specializes to strict monotonicity of f x < y f (..., x,...) < f (..., y,...) with respect to each argument which is of the answer type.

10 Historical formulation of Bellman s Principle Bellman 1964: An optimal solution for a given problem is always composed solely from optimal solutions of it s subproblems. This is less general than the definition of the previous slide: it speaks of a single solution (rather than lists of solutions) it speaks of optimal and thinks of minimization or maximization Our objective function h may also compute other types of information. The historical formulation is also more general (vague...?), as nothing is said about how evaluation works, how candidate solutions are obtained, etc.

11 A caveat about Bellman s Principle There is a subtlety often overlooked: If the evaluation algebra is monotone, but not strictly montone, we may have functions f (x) = x + 1 g(x) = 5 h([]) = [], h(xs) = [max(xs)] and candidates g(f (x)) with (say) x [1, 2]. Now f (2) is optimal and f (1) is not. However, both g(f (2)) = 5 = g(f (1)) are optimal!

12 A caveat about Bellman s Principle (2) This situation implies: 1 If we maximize over the candidates f (x), i.e. keep f (2) and drop f (1), we loose the optimal solution g(f (1)). 2 Bellman s formulation must be adjusted to state that there is an optimal solution composed solely from optimal sub-solutions, but this does no longer hold for any optimal solution. 3 If we maximize over f (x), the overall solution returned will be one which is composed solely of optimal sub-solutions. 4 If we optimize for the k-best solutions, the answer will be incomplete (and hence wrong).

13 Who satisfies Bellman s Priciple? Positive examples are min and max with respect to + and on positive numbers min(k), max(k) just as well distributes over + Negative examples are: min, max when f is not monotone second best in terms of min, max Note that the algebra in this example violates our formalization of Bellman s Principle.

14 Algebraic dynamic programming conceptual idea We solve our optimization problem conceptually in three phases: compute a list of trees that represent all the candidates, evaluate them all to a list of values, apply the objective function to this list. In the implementation, these phases will be amalgamated, as the objective function is applied to intermediate results. But for algorithm design, we do not want worry about this issue of efficiency. We only need to make sure that the amalgamation is mathematically correct.

15 Algebraic dynamic programming definition An ADP algorithm is specified by an evaluation signature Σ over an alphabet A a tree grammar over Σ a concrete evaluation algebra E with an objective function h satisfying Bellman s Principle of Optimality. Bellman s Principle ensures that the objective function distributes over evaluation.

16 Solution of an ADP problem Given: yield grammar G, evaluation algebra E with objective function h, and input sequence x Definition (Solving an ADP problem) Compute G(E, x) = h [E(t) t L(G), y(t) = x] in polynomial time and space (where y computes the yield of a candidate tree).

17 Yield parsing: Constructing the candidate space from the input The yield of a tree is its string of leaf symbols Yield language of tree grammar: L y (G) = {y(t) t L(G)} Note: Yield languages are context-free (string) languages, whereas tree languages are regular! A tree grammar used to parse (yield) strings is called yield grammar.

18 Implementation of an ADP algorithm Implement a generic yield-parser based on CYK algorithm via templates in C++ class methods in Java combinators in Haskell or use our native ADP C Compiler... More on this in one of the next sessions.

19 Fun for theoreticians yield parsing paradox optimal table design (NP-complete) semantic ambiguity (undecidable) scope of ADP (yield languages beyond context free)

20 First fun for programmers Exploit modularization for development & testing opt: optimization algebra k-opt: for near-optimals enum: for program introspection pretty: for output, ambiguity checking count: for combinatorics and testing prob: for statistical significance, E-values

21 Preview next session Goto online examples using ADP pages at but don t forget to come back to this point. Next session we will introduce Bellman s GAP a domain specific language (DSL) for ADP.