Greedy algorithms 2 4/5/12. Knapsack problems: Greedy or not? Compression algorithms. Data compression. David Kauchak cs302 Spring 2012

Transcription

1 Knapsack problems: Greedy or not? Greedy algorithms 2 avid Kauchak cs02 Spring 12 l 0-1 Knapsack thief robbing a store finds n items worth v 1, v 2,.., v n dollars and weight w 1, w 2,, w n pounds, where v i and w i are integers. The thief can carry at most W pounds in the knapsack. Which items should the thief take if he wants to maximize value. l Fractional knapsack problem Same as above, but the thief happens to be at the bulk section of the store and can carry fractional portions of the items. For example, the thief could take % of item i for a weight of 0.2w i and a value of 0.2v i. ata compression ompression algorithms l Given a file containing some data of a fixed alphabet Σ (e.g.,,, ), we would like to pick a binary character code that minimizes the number of bits required to represent the data. minimize the size of the encoded file

2 Simplifying assumption: frequency only ssume that we only have character frequency information for a file Fixed length code Use ceil(log 2 Σ ) bits for each character = = = = = Fixed length code Use ceil(log 2 Σ ) bits for each character Fixed length code Use ceil(log 2 Σ ) bits for each character = 00 = 01 = 10 = 11 2 x + 2 x + 2 x + 2 x = = 00 = 01 = 10 = 11 2 x + 2 x + 2 x + 2 x = 260 bits How many bits to encode the file? 260 bits an we do better? 2

3 Variable length code ecoding a file What about: = 0 = 01 = 10 = 1 1 x + 2 x + 2 x + 1 x = = 0 = 01 = 10 = bits How many bits to encode the file? What characters does this sequence represent? ecoding a file Variable length code = 0 = 01 = 10 = or? What about: = 0 = 100 = 101 = 11 1 x + x + x + 2 x = What characters does this sequence represent? 21 bits (18% reduction) How many bits to encode the file?

4 Prefix codes prefix code is a set of codes where no codeword is a prefix of some other codeword Prefix tree We can encode a prefix code using a full binary tree where each child represents an encoding of a symbol = 0 = 01 = 10 = 1 = 0 = 100 = 101 = 11 = 0 = 100 = 101 = 11 ecoding using a prefix tree l To decode, we traverse the graph until a leaf node is ecoding using a prefix tree = 0 = 100 = 101 = 11 4

5 ecoding using a prefix tree ecoding using a prefix tree ecoding using a prefix tree ecoding using a prefix tree 5

6 ecoding using a prefix tree ecoding using a prefix tree etermining the cost of a file etermining the cost of a file cost( T) n = = f i i depth( i 1 ) 6

7 etermining the cost of a file etermining the cost of a file What if we label the internal nodes with the sum of the children? 2 ost is equal to the sum of the internal nodes and the leaf nodes 2 etermining the cost of a file greedy algorithm? s we move down the tree, one bit gets read for every nonroot node times we see a 0 by itself Given file frequencies, can we come up with a prefixfree encoding (i.e. build a prefix tree) that minimizes the number of bits? 60 times we see a prefix that starts with a 1 of those, times we see an additional 1 the remaining 2 times we see an additional of these, times we see a last 1 and times a last 0 7

8 Heap Heap merging with this node will incur an additional cost of 2 2 Heap Heap 60 8

9 Is it correct? l The algorithm selects the symbols with the two smallest frequencies first (call them f 1 and f 2 ) 60 2 Heap 10 Is it correct? l The algorithm selects the symbols with the two smallest frequencies first (call them f 1 and f 2 ) l onsider a tree that did not do this: Is it correct? l The algorithm selects the symbols with the two smallest frequencies first (call them f 1 and f 2 ) l onsider a tree that did not do this: cost( T) n = = f i i depth( i 1 ) f 1 f 1 f i - frequencies don t change - cost will decrease since f 1 < f i f i f 2 f i f 2 f 1 f 2 contradiction 9

10 Runtime? Non-optimal greedy algorithms 1 call to MakeHeap 2(n-1) calls ExtractMin n-1 calls Insert O(n log n) l ll the greedy algorithms we ve looked at today give the optimal answer l Some of the most common greedy algorithms generate good, but non-optimal solutions l set cover l clustering l hill-climbing l relaxation 10