16.3 The Huffman code problem

Transcription

1 6.3 The Human code problem Human codes are widely used and very eective techniques or data compression. The principle behind this technique is to design a binary character code or each character, using a variable number o bits to represent each character, so as to reduce the total length o a document. Example:,-character document using {a,b,c,d,e,}. Using (ixed-length) 3 bits per character, the document needs 3, bits. Using a variable-length per character, the document can be stored in 224, bits. COMP36/6466: Lecture 7 28

2 6.3 The Human code problem A string abaabe then can be encoded as, using the ixed-length encoding, using the variable-length encoding COMP36/6466: Lecture

3 6.3 The Human code problem Another example: In the MP3 audio compression scheme, a sound signal is encoded in three steps. It is digitized by sampling at regular intervals, yielding a sequence o real numbers s,s 2,...,s T. For instance, at a rate o 44, samples per second, a 5-minute symphony would correspond to T = , 3 million measurements. Each real-valued sample s i is quantized: approximated by a nearby number rom a inite set Γ. This set is careully chosen to exploit human perceptual limitations, with the intention that the approximating sequence is indistinguishable rom s,s 2,...,s T by the human ear. The resulting string o length T is encoded in binary, using Human encoding. COMP36/6466: Lecture

4 6.3 The Human code problem (continued) To allow easy, unambiguous decoding, we require the code to be a preix code: no code is a preix o another. A danger with having codewords o dierent lengths is that the resulting encoding may not be uniquely decipherable. For example, i the codewords are {,,,}, the decoding o string like is ambiguous. You can interpret it as -, or. We avoid this problem by insisting on the preix-ree property: no codeword can be a preix o another codeword, where codeword is a string o digits to represent a character in the encoding. COMP36/6466: Lecture

5 6.3 The Human code problem (continued) Preix codes are easily representable as binary trees. COMP36/6466: Lecture

6 6.3 The Human code problem (continued) Given a character set C and the requency (c) o each character c C in a document A, the problem is to ind a preix code that minimizes the total length o the document. A tree representing such an optimal code is called a Human tree T. We aim to minimize where B(T,A) = c C (c)d T (c), d T (c) is the depth o character c in tree T (numbers o bits used to encode character c), or the length o the codeword o character c, where the depth o the tree root is. COMP36/6466: Lecture

7 B(T,A) is the total number o bits or document A based on the coding by the Human tree T. The Human code problem is to develop a tree T to minimize B(T,A).

8 6.3 The Human code problem (continued) Let T be a Human tree o a character set C or a document A. Observations: There are no nodes with only a single child. Why? Rearranging characters at the same level o the tree T does not change the value B(T, A). Why? Swapping a character at a greater depth with another less-requent character at a lower depth reduces B(T,A). We assume that the two least-requent characters x and y are siblings at the greatest depth. We have a greedy strategy or the Human code problem: Replacing x, y and their parent by a lea representing a new character xy with requency ( xy ) = (x) + (y) gives a Human tree or C { xy } \ {x,y}. COMP36/6466: Lecture

9 6.3 The Human code problem (continued) a x y b x a y b x y,, T T max{ }< min{ a b } COMP36/6466: Lecture

10 6.3 The Human code problem (continued) COMP36/6466: Lecture

11 6.3 The Human code problem (continued) For eicient implementation o a Human tree, we can use a priority queue data structure - a minimum heap. It has a set o weighted items as its key value, with ast (such as O(logn)) execution o the operations Extract MIN: ind and delete the element with the least weight, and Insert: insert an element with a key to the priority queue. Human(C) n C ; 2 Q C; /* The priority queue */ 3 or i to n 4 do z allocate node(); 5 le t[z] = x Extract MIN(Q); 6 right[z] = y Extract MIN(Q); 7 [z] [x] + [y]; 8 Insert(Q, z); 9 return Extract MIN(Q). COMP36/6466: Lecture 7 28

12 6.3 Correctness o Human s algorithm Assuming that a > x and b > y in tree T, we have B(T ) B(T ) = c d T (c) c d T (c) c C c C = x d T (x) + a d T (a) x d T (x) a d T (a) = x d T (x) + a d T (a) x d T (a) a d T (x) = a (d T (a) d T (x)) x (d T (a) d T (x)) = ( a x ) (d T (a) d T (x)) > as a > x by the assumption and d T (a) > d T (x). () We can show that B(T ) B(T ) > similarly, B(T ) B(T ) >. Thus, B(T ) > B(T ) > B(T ). The process continues until no such swapping are possible. COMP36/6466: Lecture 7 28

13 6.3 The Human code problem (continued) Exercise: What is the running time o the greedy algorithm or the construction o a Human tree? assuming that there are n characters in a document. COMP36/6466: Lecture