CSC 8301 Design and Analysis of Algorithms: Exhaustive Search Professor Henry Carter Fall 2016
Recap Brute force is the use of iterative checking or solving a problem by its definition The straightforward approach Pros: Easy to understand Quick to build Cons: Usually inefficient Provides little insight into the problem 2
Combinatorics & Optimization Mathematical problems dealing with combinations, permutations, or optimizing parameters General problems seeking specific solutions The brute force solution: generate all solutions, select the best 3
Who s combining and optimizing? Many real-world problems reduce to combinatorial versions (e.g., routing, scheduling, logistical planning) Mathematically interesting! Combinatorics problems even show up on technical interviews (Two of) today s problems represent a special class of problems: NP-Hard NP-Hard problems have no known polynomial-time solution Arguably the greatest open question in computing 4
Common orders of growth 1 constant log n logarithmic n linear n logn n-log-n (linearithmic) n 2 quadratic n 3 cubic 2 n exponential n! factorial
Traveling Salesman Problem A salesman needs to visit n cities without repeating any. What is the cheapest route? In mathematical terms: given a weighted graph of n vertices, find a cycle that travels through every vertex exactly once 6
Example: TSP Tour Cost a b c d a 2+3+7+5 = 17 a b d c a 2+4+7+8 = 21 a c b d a 8+3+4+5 = 20 a c d b a 8+7+4+2 = 21 a d b c a 5+4+3+8 = 20 a d c b a 5+7+3+2 = 17 a 2 b 8 5 3 4 c 7 d 7
Knapsack Problem A thief is choosing from a set of n objects, where each object has a value v i and a weight w i. With a knapsack of capacity c, find the most valuable subset of objects In mathematical terms: Given a set of n objects with value v i and weight w i and a maximum capacity c, find the subset of objects with the highest total value v and a total weight w < c 8
Example: Knapsack Example: Knapsack capacity W=16 item weight value 1. 2 $20 2. 5 $30 3. 10 $50 4. 5 $10 9
Example: Knapsack Subset Total weight Total value {1} 2 $20 {2} 5 $30 {3} 10 $50 {4} 5 $10 {1,2} 7 $50 {1,3} 12 $70 {1,4} 7 $30 {2,3} 15 $80 {2,4} 10 $40 {3,4} 15 $60 {1,2,3} 17 not feasible {1,2,4} 12 $60 {1,3,4} 17 not feasible {2,3,4} 20 not feasible {1,2,3,4} 22 not feasible 10
Example: Knapsack 11
Assignment Problem There are n people who need to be assigned to n jobs. The ith person has a cost C i,j for working the jth job. Find a job assignment that minimizes the total cost. In mathematical terms: Given a matrix of costs C, select one entry in each row such that there is only one selection in each column and the sum of the costs is minimized 12
Example: Assignment 9 2 7 8 C = 6 4 3 7 5 8 1 8 7 6 9 4 Assignment (col.#s) Total Cost 1, 2, 3, 4 9+4+1+4=18 1, 2, 4, 3 9+4+8+9=30 1, 3, 2, 4 9+3+8+4=24 1, 3, 4, 2 9+3+8+6=26 1, 4, 2, 3 9+7+8+9=33 1, 4, 3, 2 9+7+1+6=23 etc. 13
Example: Assignment 14
Good news, everyone! Unlike the first two problems, the assignment problem is in P (a polynomial time algorithm is known) Hungarian mathematicians developed a O(n 4 ) algorithm Further optimizations have been developed since 15
8 Queens Problem How many different ways can you arrange 8 queens without endangering any of them? 16
8 Queens Problem 92 unique solutions How many different positions are there so that: No two queens are on the same square? No two queens are on the same row? No two queens are on the same row OR in the same column? 17
Exhaustive (Graph) Search Graphs are a very common construction in computer science Specially structured graphs called trees are a common way to store data How do we search graph vertices in an ordered fashion? 18
Mathematical Graphs A set of vertices and edges that connect them Trees are graphs with no cycles (loops) Common representation: Adjacency matrix Adjacency list 19
Depth-First Search Searches vertices by moving as far as possible from the starting node Produces a set of trees containing the searched vertices called a Depth-First Search Forest Implemented with a stack (last in first out) data structure (or a recursive call stack) 20
DFS Algorithm 21
DFS: Example Graph 22
DFS: Analysis List: Matrix: 23
Practice f c b e d a g 24
Who cares? Mathematical: graph connectivity, acyclicity, etc. Foundation for iterative deepening tree search Often applied to AI search spaces (e.g., determining which move to make in chess, solving a maze) File Systems arranged as trees can be traversed using DFS 25
Next Time... Levitin Chapter 3.5, 4.1-4.2 Remember, you need to read it BEFORE you come to class! Homework 3.4: 1, 4 3.5: 1, 2, 3 26