Lecture 14: General Problem-Solving Methods

Lecture 14: General Problem-Solving Methods

Problem Solving Methods Greedy Method Divide-and-Conquer Backtracking Dynamic Programming Branch-and-Bound

Greedy Method The greedy method consists of an iteration of the following computations: selection procedure-a selection procedure is created to choose the next item to add to a list of locally optimalsolutions to sub problems feasibility check-a test is made to see if the current set of choices satisfies some locally optimal criterion. solution check -when a complete set is obtained it is checked to verify that it constitutes a solution for the original problem. Aquestionthatshouldcometomindis:Whatismeantbylocallyoptimal? This term refers to the amount of work necessary to determine a solution or to measure the level to which a criterion is met. If the computation leading to an optimal solution or the evaluation of a criterion does not involve an exhaustive search then that activity may be considered local.

Divide-and-Conquer As its name implies this method involves dividing a problem into smaller problems that can be more easily solved. While the specifics can vary from one application to another, divide-and-conquer always includes the following three steps in some form: Divide-Typically this steps involves splitting one problem into two problems of approximately 1/2 the size of the original problem. Conquer-The divide step is repeated (usually recursively) until individual problem sizes are small enough to be solved (conquered) directly. Recombine-The solution to the original problem is obtained by combining all the solutions to the sub-problems. Divide and Conquer is not applicable to every problem class. Even when D&C works it may not produce an efficient solution.

Backtracking Backtracking is used to solve problems in which a feasible solution is needed rather than an optimal one, such as the solution to a maze or the arrangement of squares in the 15-puzzle. Typically, the solution to a backtracking problem is a sequence of items(or objects) chosen from a set of alternatives that satisfy some criterion. A backtracking algorithm is a scheme for solving a series of sub-problems each of which may have multiple possible solutions and where the solution chosen for one sub-problem can affect the possible solutions of later sub-problems. To solve the overall problem, we find a solution to the first sub-problem and then attempt to recursively solve the other sub-problems based on this first solution. If wecannot,orwewantallpossiblesolutions,webacktrackandtrythenextpossible solution to the first sub-problem and so on. Backtracking terminates when there are no more solutions to the first sub-problem orasolutiontotheoverallproblemisfound. http://dictionary.die.net/backtracking

Dynamic Programming In mathematics and computer science, dynamic programming is a method of solving complex problems by breaking them down into simpler steps. It is applicable to problems that exhibit the properties of overlapping sub problems and optimal substructure. Overlapping sub problems means that the space of sub problems must be small, that is, any recursive algorithm solving the problem will solve the same sub problems over and over, rather than generating new sub problems. Dynamic programming takes account of this fact and solves each sub problem only once. Optimal substructure means that the solution to a given optimization problem can be obtained by the combination of optimal solutions to its sub problems. Consequently, the first step towards devising a dynamic programming solution is to check whether the problem exhibits an optimal substructure.

Branch-and-Bound Graph traversal refers to the problem of visiting all the nodes in a graph in a particular manner. Graph traversal can be used as a problem-solving method when a problem state can be represented as a graph and its solution can be represented as a path in the graph. When the graph is a tree it can represent the problem space for a wide variety of combinatorial problems. In these cases the solution is usually at one of the leaf-nodes of the tree or is the path to a particular leaf-node. To make graph traversal efficient we need to avoid searching paths that can be determined will not result in good (or optimal) solutions. Techniques such a branch-and-bound can be used to reduce the number of operations required to search the graph or tree problem space by eliminating infeasible or unpromising branches.

Coin Changing An optimal solution to the coin changing problem is the minimum number of coins whose total value equals a specified amount. For example what is the minimum number of coins (current U.S. mint) needed to total 83 cents. 83-25 = 58 58-25 = 33 33-25 = 8 8-5 = 3 3-1 = 2 2-1 = 1 1-1 = 0 A Greedy Algorithm for Coin Changing 1. Set remval=initial_value 2. Choose largest coin that is less than remval. 3. Add coin to set of coins and set remval:=revamal-coin_value 4. repeat Steps 2 and 3 until remval = 0;

Coin Changing - Another Example The greedy method works for the U.S. minted coin set, but will it work for any coin set? Consider the coin set below and a change value of 82 cents.. 82-26 = 56 56-26 = 30 30-26 = 4 4-1 = 3 3-1 = 2 2-1 = 1 1-1 = 0 By the coin-changing algorithm we obtain 3 26 cent pieces and 4 pennies, but we can use 2 26 cent pieces and 3 dimes to total 82 cents. Hence the greedy approach does not work for this coin set.

Quicksort Example i j 1 2 3 4 5 6 7 8 9 10 11 12 13 14 2 1 6 9 8 5 7 4 6 3 5 4 7 6 0 5 4 2 6 5 8 9 7 4 6 3 5 4 7 6 0 5 6 3 6 5 4 9 7 8 6 3 5 4 7 6 0 5 8 4 6 5 4 3 7 8 6 9 5 4 7 6 0 5 9 5 6 5 4 3 5 8 6 9 7 4 7 6 0 5 10 6 6 5 4 3 5 4 6 9 7 8 7 6 0 5 13 7 6 5 4 3 5 4 0 9 7 8 7 6 6 5 14 8 6 5 4 3 5 4 0 5 7 8 7 6 6 9 8 5 5 4 3 5 4 0 6 7 8 7 6 6 6 pivot value items being swapped new sublists for next pass of quicksort

N-Queens Problem A classic backtracking algorithm is the solution to the N-Queens problem. In this problem you are to place queens (chess pieces) on an NxN chessboard in such a way that no two queens are directly attacking one another. That is no two queens share the same row, column or diagonal on the board. Backtracking Approach -Version 1: Until all queens are placed, choose the first available location and put the next queen in this position. If queens remain to be placed and no space is left, backtrack (by removing the last queens placed and placing it in the next available position).

N-Queens Problem Backtracking Approach - Version 2: A more efficient backtracking approach is to note that each queen must be in its own column and row. This reduces the search from (N 2 )! to N!.

The Binomial Coefficient (a+b) 0 = 1 (a+b) 1 = a+b (a+b) 2 = a 2 +2ab+b 2 The binomial theorem gives a closed-form expression for the coefficient of any term in the expansion of a binomial raised to the nth power. (a+b) 3 = a 3 +3a 2 b+3ab 2 +b 3 (a+b) 4 = a 4 +4a 3 b+6a 2 b 2 +4ab 3 +b 4 ( a + b) n n! n = k= 0 k! ( n k )! a k b n k The binomial coefficient is also the number of combinations of n items taken k at a time, sometimes called n-choose-k. 1 1 1 1 2 1 1 3 3 1 1 4 6 4 1 1 5 10 10 5 1 n 1 n 1 n + = k 1 k k 1 for 0 < k k = 0 or < n k = n

Traveling Salesperson with Branch-and-Bound In the most general case the distances between each pair of cities is a positive value with dist(a,b) dist(b,a). In the matrix representation, the main diagonal values are omitted (i.e. dist(a,a) 0). A B C D E F G H A B A - 5 7 6 4 10 8 9 H C B C 8-14 9 3 4 6 2 7 9-11 10 9 5 7 D 16 6 8-5 7 7 9 G D E F 1 3 2 5-8 6 7 12 8 5 3 2-10 13 F E G H 9 5 7 9 6 3-4 3 9 6 8 5 7 9 -

The live-node list is ordered by bounding value and then by level of completion. That is, if two nodes have the same bounding value we place the one that is closer to completion ahead of the other node. This is because the bounding value of the of the nodeclosertocompletionismorelikelytobeclosertotheactualtourlength.

An Example Branch-and-Bound TSP The initial tour is A-F-E-D-B-C-A with a total length of 32. The sum of the minimum costs of leaving every node is 29 so we know that a minimal tour cannot have length less than 29.

The node shown in red were expanded from an active node (F 3 ) with a possible tour length of 29. However the completed tours turned out to be significantly larger.

Expanding D 3 we find an actual tour that is shorter than our initial tour. We do not yet know if this is a minimal tour but we can remove nodes from our live-node list that have a bounding value of >= 31. Now we expand D 2 to find that it leads to tours oflength>=33.

Expansion of E 2 eliminates this path as a candidate for a minimal tour.

When the live-node list is empty we are finished. In this example a minmal tour is A-C-E-D-B-F-Awithalengthof31wehavefoundatourthatisshorterthantheoriginal candidate tour.