General Purpose Methods for Combinatorial Optimization

Transcription

1 General Purpose Methods for Combinatorial Optimization 0/7/00 Maximum Contiguous Sum Σ = 87 Given:... N Z, at least one i > 0 ind i, j such that j k k = i is maximal 0/7/00 0/7/00

2 MCS: lgorithm L U MaxSoar := 0 for L := to N do sum := 0 for U := L to N do sum := sum + x[u] sum of x[ L... U ] MaxSoar := max (MaxSoar, sum) aster: one nested for-loop less 0/7/00 3 nalysis of lgorithm MaxSoar := 0 for L := to N do sum := 0 for U := L to N do sum := sum + x[u] sum of x[ L... U ] MaxSoar := max (MaxSoar, sum) Constant time per inner loop iteration Count number of times inner loop is executed N N O() = O(N ) L= U = L 0/7/00 4 0/7/00

3 Challenge: lgorithm 3 Can you develop a linear time algorithm? Solution next lecture this Σ = 87 0/7/00 lgorithm 3: Linear Time (solution) MaxEndingHere MaxSoar Empty vector gives maximum sum == 0 Scan the array just once Keep track of max sum in first I elements (-> MaxSoar) nd of maximum sum ending in position I (-> MaxEndingHere) MaxSoar := 0 MaxEndingHere := 0 for I := to N do MaxEndingHere := max (0, MaxEndingHere + x[i]) MaxSoar := max (MaxSoar, MaxEndingHere) 0/7/00 6 0/7/00 3

4 Comparison algorithm 3 computation time (µsec) computation time as a function of N 3.4 N 3 3 N 33 N sec 30 ms 3.3 ms 0 3 hour 3 sec 33 ms days min 0.33 sec 0 08 years. days 3.3 sec centuries months 33 sec 0/7/00 7 Traveling Salesman Problem No polynomial algorithm known Exact solution only by exhaustive search ½((N-)!) possible routes Stirling: lnn! nlogn n Large n: n! nlogn e = log ( n n n n e ) = n = o( e ) 0/7/00 8 0/7/00 4

5 Result for 000 cities Clearly sub-optimal 0/7/00 9 Simulated nnealing important much applied method for various combinatorial optimalization problems. Idea (thermo dynamics, static mechanics) Growing of crystals by means of nnealing. start off with high temperature. cool down slowly Theoretical Model matter strives for state with lowest energy movement of atoms small random displacements E < 0 accept displacement E > 0 accept with probability exp(- E/k B T) 0/7/00 0 0/7/00

6 Simulated nnealing () state of atoms configuration displacement move energy cost function perfect crystal optimal solution lgorithm T := T 0 X := start configuration while (not satisfied stop criterium) { for (a few times) { X new := new configuration if (accept (cost (X new ) cost (X), T)) X := X new } T := update (T) } accept ( cost,t) { if ( cost < 0) return TRUE else { if (exp(- cost /ct) > random (0,)) return TRUE else return LSE } } 0/7/00 Traveling Salesman by means of Simulated nnealing Randomly select two cities, interchange order of visiting: B C D C B D relatively long calculations be careful with detailed parameters of algorithm 0/7/00 0/7/00 6

7 Shortest Path Given: Weighted, directed graph G=(V,E). Single-source shortest path ind shortest path form source vertex s V to each v V. Single-destination shortest path equivalent by reducing (reverse direction of edges) 3. Single-pair shortest path route-planning no known algorithm being asymptotically better then 4. ll-pairs shortest path 0/7/00 3 Dijkstra s Shortest Path lgorithm v 6 v 3 v v 3 v 4 v 6 Q: What is the shortest path from v to v? : v v 4 v v 6 v of length 4. ind from v the shortest path to its neighbors. Say u is the closest to v 3. See if any of the routes from v to the neighbors of u becomes shorter if passing through u 4. Continue with step, until reaching v (target) 0/7/00 4 0/7/00 7

8 Dijkstra s Shortest Path lgorithm v 6 v 3 v v 3 v 4 v v 4 v v 6 v v 6 0/7/00 Dijkstra s Shortest Path lgorithm v 6 v 3 v v 3 v 4 Shortest distance to these vertices is known v 6. ind from v the shortest path to its neighbors. Say u is the closest to v 3. See if any of the routes from v to the neighbors of u becomes shorter if passing through u 4. Continue with step, until reaching v (target) This is vertex u. Shortest distance to these vertices has just become known 0/7/00 6 0/7/00 8

9 def Dijkstra (G, start, end = None, animate=0): # G[v] is a dict of distances keyed by vertices adjacent to v # G[v][u] is the distance from v to u if u is adjacent to v T = set () D = dict () Dijkstra in PYTHON language # set of vertices for which min distance is known. # dictionary of final distances, # with vertex as key and distance as value V = G.keys () # set which is initialized to all vertices. T.add (start) V.remove (start) D[start] = 0 for v in V: D[v] = sys.maxint for v in G[start]: D [v] = G[start][v] # initialize all distances # loop over all vertices reachable from start while end not in T and len (V) > 0: if animate: print "T", T, "\n", "D", D u = closest (D, V) T.add (u) V.remove (u) for v in G[u]: if D [v] > G[u][v] + D[u]: D [v] = G[u][v] + D[u] # find closest vertex to start, among those in V return D 0/7/00 7 Optimization Public Transport: It brings you from a place where you are not, to a place where you don t want to be, at a time that you do not prefer ED: It provides a solution to a problem that you don t want to solve Why? Because most ED problems are just too hard to be solved exactly Many optimization problems are NP-hard Therefore, problems are usually replaced by simpler problems that are solved instead. 0/7/00 8 0/7/00 9

10 Typical ED Optimization Setting Chip optimization problem Standard problem Transform problem into similar, known, solvable problem in mathematics world Mathematics / computer science discipline of science / engineering Standard lgorithm Problem (approximately) solved bend back into real world Sub-optimal chip 0/7/00 9 Continuous vs Discrete Optimization Continuous Optimization: Solution space forms a continuous domain Example: adjust transistor sizes in a network to optimize the speed of the network, under the assumption of continuously variable transistor dimensions Discrete Optimization: the number of solutions can be counted, from a discrete set (might be infinite) Example: traveling salesman in graph there are a finite number of distinct tours visiting all vertices Or: adjust transistor sizes in a network to optimize the speed of the network, under the assumption that transistor dimensions should be integral multiples of a step size. 0/7/00 0 0/7/00 0

11 Discrete Optimization Optimization algorithms typically go through a sequence of steps Set of choices at each step (Result of) each step is called a configuration bove we see 6 configurations with different cost, 4 choices in config of cost 6. 0/7/00 Discrete Optimization () 6 3 Π p X p = X = {x k } c(x k ) G = (X, E) step (move) E 4 given optimization problem instance of Π algorithm for Π set of all possible (legal) configurations (solutions) of p cost of configuration x k a directed graph, the configuration graph the transformation (by the algorithm) of configuration x i in x j set of all steps that algorithm considers 0/7/00 0/7/00

12 3 4 local minimum global minimum greedy algorithm Discrete Optimization (3) x i is a local minimum in case no neighboring configurations exist with lower cost desired configuration with absolute lowest cost algorithm that only takes steps that result in a lower cost convex configuration graph in case every local minimum also global minimum (and strongly connected) greedy algorithm will give exact solution. (Not necessarily in optimal time.) Uphill move 6 What is the global minimum? Is this example convex? If not: What is/are local minima? Can you change the graph to become convex? step of algorithm that (temporarily) increases cost aimed at escaping local minima 0/7/00 3 Standard Optimization Techniques Backtracking, branch and bound (.) Dynamic programming (.3) Linear programming (LP), integer LP (ILP) (.4) Local Search (.) Simulated nnealing (.6) Tabu search (.7) Genetic algorithms (.8) Boolean Satisfiability (ST) Neural networks Simulated evolution Matching, max flow, shortest path, Non-linear programming: Lagrange relaxation, Levenberg- Marquardt, 0/7/00 4 0/7/00

13 f 3 E D 8 B C 3 E D 8 B Branch and Bound C Example: TSP in graph The blue partial solution leads to a kill f B E f 3 C D B C D E f 4 f f 6 f 7 D C D E C B C D E D E D C E D B B C C B D C x x x x E E D E C B C B D B x x x x C B D C E D x x D E B C x x E B /7/00 f 3 E 4 7 D Bound-step using MST 9 B 3 8 C E 7 D 3 E D f f 3 f 4 f f 6 B E C D D C D E C E C E D B 3+8 x x X MST can be solved in polynomial time! + f /7/00 6 0/7/00 3

14 Linear Programming (LP) Maximize z = x + x + 3x 3 - ½x 4 Such that x i > 0 for all i x + x x 7x 4 0 x x 3 + x 4 ½ x + x + x 3 + x 4 = 9 (objective function) (primary constraints) (additional constraints) Example with independent variables and only inequalities easible region x Lines (hyperplanes) of constant z x Optimum is in cornerpoint LP solver searches for optimum corner point 0/7/00 7 LP Solvers Ellipsoid algorithm runs in polynomial time In practice, Simplex algorithm is often faster, but it has exponential worst-case time complexity Simplex algorithm is clever way of enumerating boundary points, it steps from point to point along the outside of the feasible region Based on principles from linear algebra Many on-line sources (e.g. see wikipedia -> linear programming) lso see the book Numerical Recipes in {C/C++/ortran} Press et al. On-line C version of the book (pdf) via See lp_solve originally by Michel Berkelaar Now at TUD/EWI/CS 0/7/00 8 0/7/00 4

15 Integer Linear Programming (ILP) Linear Programming with the x i restricted to integer numbers Surprise: ILP is NP-hard (LP has polynomial time complexity) Does not work: solving an equivalent LP and rounding to nearest integer; solution may not be optimal, or even feasible. Yet other variant is 0- LP, x i {0, } 0/7/00 9 Genetic lgorithms irst parent: (f (k) ) irst child: (f (k ) ) Second parent: (g (k) ) Second child: (g (k ) ) Inspired on biology: survival of the fittest Work with a population P (k), set of feasible solutions Each f P (k) is encoded as a chromosome, e.g. a bit-string Cross-over operations between members of P (k) to derive P (k+) Prefer parents with lower cost for mating Many variations possible (mutation, cloning, ) 0/7/ /7/00

16 Dynamic Programming ibonacci numbers: 0,,,, 3,, 8, 3,, 34, function Each fib(n) number starting from if n 3rd = 0 is or sum n = of return previous n two else Write return algorithm fib(n ) + fib(n ) var f := map(0 0, ) function fib(n) if n not in keys(f) f[n] := fib(n ) + fib(n ) return f[n] Recursive algorithm is a lot of work: e.g. fib(), fib(3) computed more then once Dynamic programming: Overlapping subproblems Optimal substructure Memoization 0/7/00 3 ibonacci Sequence Using Dynamic Programming in Python f = {0: 0, : } def dynfib (n): if n not in f: f[n] = dynfib (n-) + dynfib (n-) return f[n] def dynfib (n): if n == 0 or n == : return n prevprev = 0 prev = for i in range (, n+): fib = prev + prevprev prevprev = prev prev = fib return fib Recursive version Time and space O(n) Runs into max recursion depth problems Iterative (bottom-up) version Time O(n), space O() Runs easily for n = /7/00 3 0/7/00 6

17 Dynamic Programming Elegant approach Two variants: bottom-up and top-down Several applications in ED E.g. in channel routing, later lecture. E.g. technology mapping, Dirk-Jan Jongeneel, Optimized boolean equations Library of std cells Gate netlist Timing constraints [Jongeneel and Otten, Integration, 000] 0/7/00 33 Library of Std Cells [Jongeneel and Otten, Integration, 000] 0/7/ /7/00 7

18 Matching of Library Patterns Cost at all primary inputs Cost of library element = + #inputs (for sake of example) Cost at node after partial matching [Jongeneel and Otten, Integration, 000] 0/7/00 3 Partial Matching of Library Patterns with Optimal Cost Covering by contracting saved matches at every node, from output to inputs [Jongeneel and Otten, Integration, 000] 0/7/ /7/00 8

19 Techn. Map. Result [Jongeneel and Otten, Integration, 000] 0/7/00 37 Summary Graph algorithms and optimization very common in ED Discrete vs continuous optimization Chip optimization problem Standard problem Transform problem into similar, known, solvable problem in mathematics world Mathematics / computer science discipline of science / engineering Standard lgorithm Problem (approximately) solved bend back into real world Sub-optimal chip Many different algorithms, with different properties inding right one depends on transformation from and back into real world 0/7/ /7/00 9

20 Summary Many different optimization algorithmic techniques Backtracking, branch and bound (.) Dynamic programming (.3) Linear programming (LP), integer LP (ILP) (.4) Local Search (.) Simulated nnealing (.6) Tabu search (.7) Genetic algorithms (.8) Boolean Satisfiability (ST) Neural networks Simulated evolution Matching, max flow, shortest path, Non-linear programming: Lagrange relaxation, Levenberg- Marquardt, Many applications of optimization in ED (e.g. techn. mapping) symptotic complexity important for effectivity of algorithm Implementation can also count (e.g. recursive vs iterative implementation) 0/7/ /7/00 0