Introduction to Algorithms: Brute-Force Algorithms

Transcription

1 Introduction to Algorithms: Brute-Force Algorithms

2 Introduction to Algorithms Brute Force Powering a Number Selection Sort Exhaustive Search 0/1 Knapsack Problem Assignment Problem CS Analysis of Algorithms 2

3 Brute-Force Algorithm Design Straightforward, usually based on problem definition. Rarely the most efficient but can be applied to wide range of problems. For some elementary problems, almost as good as most efficient. May not be worth cost. May work just as well on smaller data sets. Used to measure other algorithms against. CS Analysis of Algorithms 3

4 Calculating Powers of a Number Problem: Compute a n, where n N. Naive algorithm: Θ(n). a n = n * n * n * * n a times CS Analysis of Algorithms 4

5 Selection Sort Given a list of n orderable items, rearrange them in nondecreasing order. 1. Scan entire list to find smallest item. 2. Exchange it with first item. First element now in its final, sorted position. 3. Scan remaining n 1 items, starting with second element, and find smallest item. 4. Exchange it with second item. Second element now in its final, sorted position. 5. Repeat for a total of n 1 times. CS Analysis of Algorithms 5

6 Selection Sort - Example Selection sort on the list: 89, 45, 68, 90, 29, 34, 17. Each line corresponds to an iteration of the algorithm. The values in bold are the smallest item for that iterations. Elements to the left of the vertical bar are in their final positions CS Analysis of Algorithms 6

7 Selection Sort - Analysis Selection sort is implemented using nested for loops: Outer loop: iterates from 0 to n 2 don t have to visit element because already in final sorted position Inner loop: finds smallest value remaining the list. Even though gets smaller each iteration, still on order of n. Therefore: T(n) = Θ(n 2 ) CS Analysis of Algorithms 7

8 Sequential search In a list No better than linear time We can do better if not a list Worst case O(n) CS Analysis of Algorithms 8

9 Exhaustive Search Brute-force approach to combinatorial problems (i.e. permutations, combinations, subsets of a given set). 1. Generate all elements of problem domain (all possible solutions). 2. Select those that satisfy the constraints of the problem. 3. Chose one or more that are most desirable (i.e. optimize the objective function). CS Analysis of Algorithms 9

10 0/1 Knapsack Problem Analysis The most costly operation is generating all of the subsets of n items. Since there are 2 n subsets, Brute-force Approach to Knapsack problem: Ω(2 n ). Not feasible for any but the smallest values of n. CS Analysis of Algorithms 10

11 0-1 knapsack Algorithm public boolean knapsack (double [] v, double V, double [] w, double W, } if(i<v.length){ }else{ } T(n)=2T(n-1) +1 T(n-1)<=C*2^{n-1} T(n)<=C*2*2^(n-1)+1 T(n)<=C*2*2^(n-1)+1<=C*2^n double counterv, double counterw, int i){ boolean b1=knapsack(v,v,w,w, counterv,counterw, i+1); boolean b2=knapsack(v,v,w,w,counterv+v[i],counterw+w[i], i+1); return b1 b2; return (counterv>= V) && (counterw<= W)

12 0-1 knapsack Algorithm public boolean knapsack (double [] v, double V, double [] w, double W, } if(i<v.length){ }else{ } T(n)=2T(n-1) +1 T(n-1)<=C*2^{n-1} T(n)<=C*2*2^(n-1)+1 T(n)<=C*2*2^(n-1)+1<=C*2^n double counterv, double counterw, int i){ boolean b1=knapsack(v,v,w,w, counterv,counterw, i+1); boolean b2=knapsack(v,v,w,w,counterv+v[i],counterw+w[i], i+1); return b1 b2; return (counterv>= V) && (counterw<= W) T(n)=2T(n-1) +1 T(n-1)<=C*(n-1)2^{n-1} T(n)<=C*2*2^(n-1)*(n-1)+1 T(n)<=C*2*2^(n-1)*(n- 1)+1<=C*n*2^n

13 0/1 Knapsack Problem Given n items of known weights w 1, w 2,, w n and values v 1, v 2,, v n, and a knapsack of capacity W, find most valuable subset of items that will fit. 1. Generate all subsets of n items. 2. Calculate the weights of each and eliminate all the infeasible solutions. 3. Find the subset with the maximum value. CS Analysis of Algorithms 13

14 Assignment Problem There are n jobs that need to be completed, and n people that need to be assigned to a job, one person per job. The cost for the i th person to perform job j is known C[i, j]. Find the assignment with the lowest cost. 1. Generate all permutations of n people assigned to n jobs. 2. Calculate the cost of each permutation/solution. 3. Find the solution with the minimum value. CS Analysis of Algorithms 14

15 Assignment Problem Analysis The most costly operation is generating all of the permutations. Since there are n! permutations, Brute-force Approach to Assignment problem: Θ(n!). Not feasible for any but the smallest values of n. CS Analysis of Algorithms 15

16 String Matching CS Analysis of Algorithms 16

17 String Matching CS Analysis of Algorithms 17

18 String Matching CS Analysis of Algorithms 18

19 Complexity m(n-m+1) O(nm) CS Analysis of Algorithms 19

20 Travelling salesman problem CS Analysis of Algorithms 20

21 Travelling salesman problem Path={(A,B),(B,C),(C,D),(D,A)} CS Analysis of Algorithms 21

22 Travelling salesman problem Path={(A,B),(B,D),(D,A),(A,B), (B,A)} Path={(A,B),(B,C),(C,D),(D,A)} CS Analysis of Algorithms 22

23 Hamiltonian Path Path={(A,B),(B,D),(D,A),(A,B), (B,A)} Path={(A,B),(B,C),(C,D),(D,A)} CS Analysis of Algorithms 23

24 Hamiltonian Path A Hamiltonian path or traceable path is a path that visits each vertex exactly once Path={(A,B),(B,D),(D,A),(A,B), (B,A)} Path={(A,B),(B,C),(C,D),(D,A)} CS Analysis of Algorithms 24

25 Hamiltonian Path A Hamiltonian path or traceable path is a path that visits each vertex exactly once Path={(A,B),(B,D),(D,A),(A,B), (B,A)} Path={(A,B),(B,C),(C,D),(D,A)} CS Analysis of Algorithms 25

26 Hamiltonian Path A Hamiltonian path or traceable path is a path that visits each vertex exactly once Path={(A,B),(B,D),(D,A),(A,B), (B,A)} Path * ={(A,B),(B,C),(C,D),(D,A)} Optimal CS Analysis of Algorithms 26

27 Graph Coloring CS Analysis of Algorithms 27

28 Binary counter When increment up to n, change is at most 1+floor(log n) bits - number of bits in the binary representation of n. Thus, the cost of counting up to n, a sequence of n increments, is O(n log n). Counter value A[k] A[k-1] A[2] A[1] A[0] Cost k k CS Introduction to Algorithms 28

29 Binary counter P INCREMENT.A/ 1 i D 0 2 while i<a:length and AŒi == 1 3 AŒi D 0 4 i D i C 1 5 if i<a:length 6 AŒi D 1 Figure 17.2 shows what happens to a b CS Analysis of Algorithms 29

30 NP-Hard NP-Hard Optimized approach: n Cut n Branch and Bound CS Analysis of Algorithms 30

31 Conclusion The Brute Force approach: Straightforward approach to solving problem, usually based directly on problem description. Wide applicability and simple. But in general, has subpar performance. CS Analysis of Algorithms 31