CPE702 Algorithm Analysis and Design Week 7 Algorithm Design Patterns

Transcription

1 CPE702 Algorithm Analysis and Design Week 7 Algorithm Design Patterns Pruet Boonma pruet@eng.cmu.ac.th Department of Computer Engineering Faculty of Engineering, Chiang Mai University Based on Slides by M.T. Goodrich and R. Tamassia

2 2 In this week Algorithm Design Patterns Brute-force Search Greedy method Divide-and-conquer

3 3 Brute-force Search AKA exhaustive search or generate and test. Brute-force method is a trivial method to enumerate all possible candidates for the solution. When all solutions are tested, the best one is picked. It is useful when the problem size is limited because the cost is usually grow (very) fast. Known as combinatorial explosion. E.g., if the problem has one solution variable, it will be O(z). But if it has two, it will be O(z n ) and so on; where z is the number of possible value of each variable.

4 4 The Greedy Method The greedy method is a general algorithm design paradigm, built on the following elements: configurations: different choices, collections, or values to find objective function: a score assigned to configurations, which we want to either maximize or minimize It works best when applied to problems with the greedy-choice property: a globally-optimal solution can always be found by a series of local improvements from a starting configuration.

5 5 Text Compression Given a string X, efficiently encode X into a smaller string Y Saves memory and/or bandwidth A good approach: Huffman encoding Compute frequency f(c) for each character c. Encode high-frequency characters with short code words No code word is a prefix for another code Use an optimal encoding tree to determine the code words

6 6 Encoding Tree Example A code is a mapping of each character of an alphabet to a binary code-word A prefix code is a binary code such that no code-word is the prefix of another code-word An encoding tree represents a prefix code Each external node stores a character The code word of a character is given by the path from the root to the external node storing the character (0 for a left child and 1 for a right child) a b c d e a d e b c

7 7 Encoding Tree Optimization Given a text string X, we want to find a prefix code for the characters of X that yields a small encoding for X Frequent characters should have long code-words Rare characters should have short code-words Example X = abracadabra T 1 encodes X into 29 bits T 2 encodes X into 24 bits T 1 T 2 c d b a b r a r c d

8 8 Huffman s Algorithm Given a string X, Huffman s algorithm construct a prefix code the minimizes the size of the encoding of X It runs in time O(n + d log d), where n is the size of X and d is the number of distinct characters of X A heap-based priority queue is used as an auxiliary structure Algorithm HuffmanEncoding(X) Input string X of size n Output optimal encoding trie for X C distinctcharacters(x) computefrequencies(c, X) Q new empty heap for all c C T new single-node tree storing c Q.insert(getFrequency(c), T) while Q.size() > 1 f 1 Q.minKey() T 1 Q.removeMin() f 2 Q.minKey() T 2 Q.removeMin() T join(t 1, T 2 ) Q.insert(f 1 + f 2, T) return Q.removeMin()

9 9 Example X = abracadabra Frequencies a a b c d r c d b r 6 a b c d r a c d b r a b c d r a c d b r

10 Extended Huffman Tree Example 10

11 11 The Knapsack Problem Given: A set S of n items, with each item i having b i - a positive benefit w i - a positive weight Goal: Choose items with maximum total benefit but with weight at most W. If we are allowed to take fractional amounts, then this is the fractional knapsack problem. In this case, we let x i denote the amount we take of item i Objective: maximize i S b ( x / w ) i i i Constraint: i S x i W

12 12 Example Given: A set S of n items, with each item i having b i - a positive benefit w i - a positive weight Goal: Choose items with maximum total benefit but with weight at most W. knapsack Items: Weight: Benefit: Value: 3 ($ per ml) ml 8 ml 2 ml 6 ml 1 ml $12 $32 $40 $30 $ ml Solution: 1 ml of 5 2 ml of 3 6 ml of 4 1 ml of 2

13 13 The Knapsack Algorithm Greedy choice: Keep taking item with highest value (benefit to weight ratio) Since i S Run time: O(n log n). Why? Correctness: Suppose there is a better solution there is an item i with higher value than a chosen item j, but x i <w i, x j >0 and v i <v j If we substitute some i with j, we get a better solution How much of i: min{w i -x i, x j } Thus, there is no better solution than the greedy one b ( x / w ) = ( b / w ) x i i i i S i i i Algorithm fractionalknapsack(s, W) Input: set S of items w/ benefit b i and weight w i ; max. weight W Output: amount x i of each item i to maximize benefit w/ weight at most W for each item i in S x i 0 v i b i / w i {value} w 0 {total weight} while w < W remove item i w/ highest v i x i min{w i, W - w} w w + min{w i, W - w}

14 14 Task Scheduling Given: a set T of n tasks, each having: A start time, s i A finish time, f i (where s i < f i ) Goal: Perform all the tasks using a minimum number of machines. Machine 3 Machine 2 Machine

15 15 Task Scheduling Algorithm Greedy choice: consider tasks by their start time and use as few machines as possible with this order. Run time: O(n log n). Why? Correctness: Suppose there is a better schedule. We can use k-1 machines The algorithm uses k Let i be first task scheduled on machine k Machine i must conflict with k-1 other tasks But that means there is no non-conflicting schedule using k-1 machines Algorithm taskschedule(t) Input: set T of tasks w/ start time s i and finish time f i Output: non-conflicting schedule with minimum number of machines m 0 {no. of machines} while T is not empty remove task i w/ smallest s i if there s a machine j for i then schedule i on machine j else m m + 1 schedule i on machine m

16 16 Example Given: a set T of n tasks, each having: A start time, s i A finish time, f i (where s i < f i ) [1,4], [1,3], [2,5], [3,7], [4,7], [6,9], [7,8] (ordered by start) Goal: Perform all tasks on min. number of machines Machine 3 Machine 2 Machine

17 17 Divide-and-Conquer Divide-and conquer is a general algorithm design paradigm: Divide: divide the input data S in two or more disjoint subsets S 1, S 2, Recur: solve the subproblems recursively Conquer: combine the solutions for S 1, S 2,, into a solution for S The base case for the recursion are subproblems of constant size Analysis can be done using recurrence equations

18 18 Merge-Sort Merge-sort on an input sequence S with n elements consists of three steps: Divide: partition S into two sequences S 1 and S 2 of about n/2 elements each Recur: recursively sort S 1 and S 2 Algorithm mergesort(s, C) Input sequence S with n elements, comparator C Output sequence S sorted according to C if S.size() > 1 (S 1, S 2 ) partition(s, n/2) mergesort(s 1, C) mergesort(s 2, C) S merge(s 1, S 2 ) Conquer: merge S 1 and S 2 into a unique sorted sequence

19 19 Merging Two Sorted Sequences The conquer step of merge-sort consists of merging two sorted sequences A and B into a sorted sequence S containing the union of the elements of A and B Merging two sorted sequences, each with n/2 elements and implemented by means of a doubly linked list, takes O(n) time Algorithm merge(a, B) Input sequences A and B with n/2 elements each Output sorted sequence of A B S empty sequence while A.isEmpty() B.isEmpty() if A.first().element() < B.first().element() S.insertLast(A.remove(A.first())) else S.insertLast(B.remove(B.first())) while A.isEmpty() S.insertLast(A.remove(A.first())) while B.isEmpty() S.insertLast(B.remove(B.first())) return S

20 20 Merge-Sort Tree An execution of merge-sort is depicted by a binary tree each node represents a recursive call of merge-sort and stores unsorted sequence before the execution and its partition sorted sequence at the end of the execution the root is the initial call the leaves are calls on subsequences of size 0 or

21 21 Execution Example Partition

22 22 Execution Example (cont.) Recursive call, partition

23 23 Execution Example (cont.) Recursive call, partition

24 24 Execution Example (cont.) Recursive call, base case

25 25 Execution Example (cont.) Recursive call, base case

26 26 Execution Example (cont.) Merge

27 27 Execution Example (cont.) Recursive call,, base case, merge

28 28 Execution Example (cont.) Merge

29 29 Execution Example (cont.) Recursive call,, merge, merge

30 30 Execution Example (cont.) Merge

31 31 Analysis of Merge-Sort The height h of the merge-sort tree is O(log n) at each recursive call we divide in half the sequence, The overall amount or work done at the nodes of depth i is O(n) we partition and merge 2 i sequences of size n/2 i we make 2 i+1 recursive calls Thus, the total running time of merge-sort is O(n log n) depth #seqs size 0 1 n 1 2 n/2 i 2 i n/2 i

32 32 Recurrence Equation Analysis The conquer step of merge-sort consists of merging two sorted sequences, each with n/2 elements and implemented by means of a doubly linked list, takes at most bn steps, for some constant b. Likewise, the basis case (n < 2) will take at b most steps. Therefore, if we let T(n) denote the running time of merge-sort: T ( n) = 2T ( n / b 2) + bn if if n < n 2 2 We can therefore analyze the running time of merge-sort by finding a closed form solution to the above equation. That is, a solution that has T(n) only on the left-hand side.

33 33 Iterative Substitution In the iterative substitution, or plug-and-chug, technique, we iteratively apply the recurrence equation to itself and see if we can find a pattern: T ( n) = 2T ( n / 2) + bn = = = = = = 2(2T ( n / T ( n / 2 T ( n / 2 T ( n / 2 i i 2 T ( n / 2 )) + b( n / 2)) + bn ) + 2bn ) + 3bn ) + 4bn ) + ibn Note that base, T(n)=b, case occurs when 2 i =n. That is, i = log n. So, T ( n) = bn + bn logn Thus, T(n) is O(n log n)

34 34 The Recursion Tree Draw the recursion tree for the recurrence relation and look for a pattern: depth T s size 0 1 n T ( n) = b 2T ( n / 2) + bn if if n n < 2 2 time bn 1 2 n/2 i 2 i n/2 i bn bn Total time = bn + bn log n (last level plus all previous levels)