Course schedule. COMP9319 Web Data Compression and Search. Huffman coding. Original readings

Transcription

1 Course schedule COMP9319 Web Data Compression and Search Lecture 2: daptive Huffman, WT Data compression Search Data compression + Search Web data compression + Search Optional topics 1 3 Original readings Huffman coding Login to your cse account: cd ~cs9319/papers Original readings of each lecture will be placed there. S Freq Huffman a b 6 10 c d e k a b c d e

2 Huffman not optimal H = log log / log L = (100000*1 + )/ Problems of Huffman coding eed statistics & static: e.g., single pass over the data just to collect stat & stat unchanged during encoding To decode, the stat table need to be transmitted. Table size can be significant for small msg. => daptive compression e.g., adaptive huffman 5 7 Problems of Huffman coding daptive compression Huffman codes have an integral # of bits. E.g., log (3) = while Huffman may need 2 bits oticeable non-optimality when prob of a symbol is high. => rithmetic coding Encoder Initialize the model Repeat for each input char ( Encode char Update the model ) Decoder Initialize the model Repeat for each input char ( Decode char Update the model ) Make sure both sides have the same Initialize & update model algorithms. 6 8

3 daptive Huffman Coding (dummy) daptive Huffman (lgorithm outline) Encoder Reset the stat Repeat for each input char ( Encode char Update the stat Rebuild huffman tree ) Decoder Reset the stat Repeat for each input char ( Decode char Update the stat Rebuild huffman tree ) 1. If current symbol is YT, add two child nodes to YT node. One will be a new YT node the other is a leaf node for our symbol. Increase weight for the new leaf node and the old YT and go to step 4. If not, go to symbol's leaf node. 2. If this node does not have the highest number in a block, swap it with the node having the highest number 3. Increase weight for current node 4. If this is not the root node go to parent node then go to step 2. If this is the root, end daptive Huffman Coding (dummy) Encoder Reset the stat Repeat for each input char ( Encode char Update the stat Rebuild huffman tree ) Decoder Reset the stat Repeat for each input char ( Decode char Update the stat Rebuild huffman tree ) The update procedure from Introduction to Data Compression by by Sayood Khalid This works but too slow lso, Wikipedia provides a good summary, example and explanation (i.e., en.wikipedia.org/wiki/ daptive_huffman_coding) 10 12

4 daptive Huffman More example abbbbba: abbbbba: : W=8 256: W=18 e 255: W=10 252: W=4 253: W=4 a 248: W=2 b 249: W=2 c 250: W=2 d 251: W=2 13 a: b: Modified from Wikipedia 15 More example More example 256: W=17 256: W=19 254: W=7 e 255: W=10 254: W=9 e 255: W=10 252: W=3 253: W=4 252: W=4 253: W=5 More aaaa. coming a 248: W=1 b 249: W=2 c 250: W=2 d 251: W=2 d 248: W=2 b 249: W=2 c 250: W=2 a 251: W=

5 More example More example 256: W= : W=20 254: W=10 e 255: W=10 254: W=10 +1 e 255: W=10 252: W=4 253: W=6 a 252: W=5 253: W=6 251: W=4 d 248: W=2 b 249: W=2 c 250: W=2 a 251: W=4 c 250: W= d 248: W=2 b 249: W=2 More example More example 256: W=20 256: W=21 252: W=4 254: W=10 253: W=6 e 255: W=10 e 254: W=10 a 252: W=5 255: W=11 253: W=6 251: W=4 d 248: W=2 b 249: W=2 c 250: W=2 a 251: W=4 c 250: W= d 248: W=2 b 249: W=2

6 daptive Huffman Question: daptive Huffman vs Static Huffman Vitter s experiments Include overheads such as symbol tables / leaf node code etc. 95 SCII chars + <end-of-line> Exclude overheads such as symbol tables / leaf node code etc From Vitter s paper. You know where it is. Compared with Static Huffman Dynamic and can offer better compression (cf. Vitter s experiments next) i.e., the tree can be smaller (hence shorter the code) before the whole bitstream is received. Works when prior stat is unavailable Saves symbol table overhead (cf. Vitter s expt next) More experiments 22 24

7 ext WT WT: urrows Wheeler Transform It is a transform, not a compression; but it usually helps compression (esp. text compression). Input: #S simple example 25 Excerpted from Wikipedia 27 Excerpted from Wikipedia Recall from Lecture 1 s RLE and WT example rabcabcababaabacabcabcabcababaa$ aabbbbccacccrcbaaaaaaaaaabbbbba$ aab4ccac3rcba10b5a$ ll rotations #S S# S# S# S# S# S# S#

8 29 Sort the rows #S S# S# S# S# S# S# S# Exercise: you can try the example rabcabcababaabacabcabcabcababaa$ aabbbbccacccrcbaaaaaaaaaabbbbba$ Output #S S# S# S# S# S# S# S# Input: S # 32 ow the inverse

9 First add dd again S # S# # S Then sort Then sort # S # S S# 34 36

10 Then add Then add S# S # S# S# S# # S S# Then sort Then sort # S# S S# # S S# S# S# 38 40

11 Then add S# S S# # S# S# Then add S# S# S# # S S# S# Then sort # S# S# S S# S# Then sort # S S# S# S# S# S# 42 44

12 Then add S# S S# S# # S# S# S# Then add S# S# S# S# #S S# S# S# Then sort # S# S# S# S S# S# S# Then sort (?) #S S# S# S# S# S# S# S# 46 48

13 Implementation Do we need to represent the table in the encoder? o, a single pointer for each row is needed. InverseWT(S) function inversewt (string s) create empty table repeat length(s) times insert s as a column of table before first column of the table // first insert creates first column sort rows of the table alphabetically return (row that ends with the 'EOF' character) WT(S) function WT (string s) create a table, rows are all possible rotations of s sort rows alphabetically return (last column of the table) Move to Front (MTF) Reduce entropy based on local frequency correlation Usually used for WT before an entropyencoding step uthor and detail: Original paper at cs9319/papers

14 Example: abaabacad Other ways to reverse WT Symbol Code List a 0 abcde.. b 1 bacde.. a 1 abcde.. a 0 abcde.. b 1 bacde.. a 1 abcde.. c 2 cabde.. a 1 acbde.. d 3 dacbe.. 53 To transform a general file, the list has 256 SCII symbols. Consider L=WT(S) is composed of the symbols V 0 V -1, the transformed string may be parsed to obtain: The number of symbols in the substring V 0 V i-1 that are identical to V i. For each unique symbol, V i, in L, the number of symbols that are lexicographically less than that symbol. 55 WT compressor vs ZIP Example ZIP (i.e., LZW based) WT+RLE+MTF+C Symbol # LessThan From 56

15 ???????????? Symbol # LessThan Symbol # LessThan ?????????? Symbol # LessThan Symbol # LessThan

16 ???? Symbol # LessThan Symbol # LessThan ?? Symbol # LessThan Symbol # LessThan

17 Symbol # LessThan Occ / Rank C 67 n illustration First Last 66 68

18

19 73 Dynamic WT? Instead of reconstructing WT, local reordering from the original WT. Details: Salson M, Lecroq T, Léonard M and Mouchard L (2009). " Four-Stage lgorithm for Updating a urrows Wheeler Transform". Theoretical Computer Science 410 (43):