Data Compression Algorithms

Transcription

1 Data Compression Algorithms Adaptive Huffman coding Robert G. Gallager Massachusetts Institute of Technology Donald Ervin Knuth Stanford University NSWI072-6

2 Static adaptive methods (Statistical) data compression modeling coding Static model model model source encoder decoder 2

3 Static adaptive methods Adaptive model model source encoder model update model decoder model update 3

4 Adaptive Huffman code Algorithm AdaptiveHuffman: Encoding Inicialize Huffman tree while not EOF do read symbol encode symbol update tree Algorithm AdaptiveHuffman: Decoding Inicialize Huffman tree while not EOF do decode symbol write symbol update tree 4

5 Adaptive Huffman code Brute Force Huffman tree reconstruction after each frequency change after reading next k symbols after change of order of symbols sorted by frequencies 5

6 Characterization of Huffman trees Huffman tree binary tree with nonnegative vertex weights Two vertices of a binary tree with the same parent are called siblings Binary tree with nonnegative vertex weights has a sibling property if parent: weight(parent) = Σ weight(child) each vertex except root has a sibling vertices can be listed in order of non-increasing weight with each vertex adjacent to its sibling 6

7 Characterization of Huffman trees Theorem (Faller 1973) Binary tree with nonnegative vertex weights is a Huffman tree iff it has the sibling property. FGK algorithm Faller(1973), Gallagher(1978), Knuth(1985) 7

8 15 7 e:8 3 4 a:1 b:2 c:2 d:2 8

9 e: a:1 b:2 c:2 d:

10 e: a:1 b:2 c:2 d: read symbol a 10

11 e: a:2 b:2 c:2 d:

12 e: a:2 b:2 c:2 d: read symbol a 12

13 e: a:3 b:2 c:2 d:

14 e: d:2 b:2 c:2 a:

15 e: d:2 b:2 c:2 a:

16 e: d:2 b:2 c:2 a:

17 1 16 e: d:2 b:2 c:2 a:

18 1 17 e: d:2 b:2 c:2 a:

19 Zero-frequency problem How to encode a novel symbol? 1 st solution: Initialize Huffman tree with all symbols of the source alphabet, each with weight one. 2 nd solution: Initialize Huffman tree with a special symbol esc. encode the 1 st occurrence of symbol s as a Huffman code of esc followed by s insert a new leaf representing s into the Huffman tree 19

20 Zero-frequency problem Initialization esc:0 s is a novel symbol which does not occur in the tree esc:0 0 esc:0 s:0 update the tree 20

21 Updating the Huffman tree s is the current input symbol if s does not occur in the tree then esc:0 i 0 i esc:0 s:0 i+2 i+1 v else v:= leaf representing s 21

22 Updating the Huffman tree if v is a sibling of esc then exchange v with the leaf of the least order among the vertices of the same weight as v v.weight++; v := parent(v) while v root do exchange v with the vertex of the least order among the vertices of the same weight as v (subtrees are exchanged with their roots) v.weight++; v := parent(v) the last symbol may be stored directly into esc vertex 22

23 FGK: Encoding Initialize Huffman tree T repeat read s until EOF if 1st occurrence of s then write(code(esc)) write(s) else write(code(s)) update tree T with s 23

24 FGK: Decoding Initialize Huffman tree T vertex := root of T repeat while vertex is not a leaf do read bit if bit=0 then vertex:=vertex.left-child else vertex:=vertex.right-child if vertex.symbol=esc then read s else s := vertex.symbol write s to output update tree T with s until EOF 24

25 Vitter s algorithm J.S.Vitter (1987) one exchange only while updating the tree FGK l/2 exchanges l = current codeword length Vitter minimizes i l i a max i l i l i = i-th codeword length l A - average codeword length for algorithm A l V l H +1 l FGK l H + O(1) 25

26 Implementation notes Overflow problem longest codeword length file length (root.weight) Solution: multiply all weights by a coefficient r < 1 Disadvantage: tree reconstruction may be necessary r = 1/

27 Tree reconstruction ? Tree reconstruction is necessary!

28 Statistics aging each frequency /= 2 older frequencies have lower weight that new ones better compression ratio each frequency *= (1- x) in each step, x 0 current_symbol_frequency += (1+ x) t at time t Example: we need (1+x) d = 2 x ln(1+x) for x 0 x (ln 2)/d for x 0 Choice of d: stable probability distribution frequent fluctuations d = 1000, 1+x = integral arithmetic scaling 1 st step: increment by nd step: increment by 10000(1+x) = rd step: increment by 10000(1+x) 2 = when reaching the upper bound, each frequency /= 2 28