16.3 The Huffman code problem

Transcription

1 6.3 The Human code problem Human codes are widely used and very eective techniques or data compression. The principle behind this technique is to design a binary character code or each character, using a variable number o bits to represent each character, so as to reduce the total length o the document. Example:,-character document using {a,b,c,d,e,}. Using (ixed-length) 3 bits per character, the document needs 3, bits. Using a variable-length per character, the document can be stored in 224, bits. COMP36/6466: Lecture 7 27

2 6.3 The Human code problem A string abaabe then can be encoded as, using the ixed-length encoding, using the variable-length encoding COMP36/6466: Lecture

3 6.3 The Human code problem Another example: In the MP3 audio compression scheme, a sound signal is encoded in three steps. It is digitized by sampling at regular intervals, yielding a sequence o real numbers s,s 2,...,s T. For instance, at a rate o 44, samples per second, a 5-minute symphony would correspond to T = , 3 million measurements. Each real-valued sample s i is quantized: approximated by a nearby number rom a inite set Γ. This set is careully chosen to exploit human perceptual limitations, with the intention that the approximating sequence is indistinguishable rom s,s 2,...,s T by the human ear. The resulting string o length T is encoded in binary, using Human encoding. COMP36/6466: Lecture

4 6.3 The Human code problem (continued) To allow easy, unambiguous decoding, we require the code to be a preix code: no code is a preix o another. A danger with having codewords o dierent lengths is that the resulting encoding may not be uniquely decipherable. For example, i the codewords are {,,,}, the decoding o string like is ambiguous. You can interpret it as -, or. We thus avoid this problem by insisting on the preix-ree property: no codeword can be a preix o another codeword, where codeword is a string o digits to represent a character in the encoding. COMP36/6466: Lecture

5 6.3 The Human code problem (continued) Preix codes are easily representable as binary trees. COMP36/6466: Lecture

6 6.3 The Human code problem (continued) Given a character set C and the requency (c) o each character c C in the document A, the problem is to ind a preix code that minimizes the total length o the document. The tree representing such an optimal code is called a Human tree T. We aim to minimize where B(T,A) = c C (c)d T (c), d T (c) is the depth o character c in the tree T (numbers o bits used to encode character c), or the length o the codeword o character c. B(T,A) is the total number o bits or document A based on the Human tree T. The problem is to design a tree T to minimize B(T,A). COMP36/6466: Lecture

7 6.3 The Human code problem (continued) Observations: Let T be a Human tree o a character set C. There are no nodes with only a single child. Why? Rearranging characters within the same level o the tree T does not change B(T,A). Why? Swapping a character at a higher level with another less-requent character reduces B(T,A), where the level o the tree root is the lowest level. We thus can assume that the two least-requent characters x and y are siblings at the greatest depth. We have a greedy strategy or the Human coding problem: Replacing x, y and their parent by a lea representing a new character xy with requency ( xy ) = (x) + (y) gives a Human tree or C { xy } \ {x,y}. COMP36/6466: Lecture

8 6.3 The Human code problem (continued) a x y b x a y b x y,, T T max{ }< min{ a b } COMP36/6466: Lecture

9 6.3 The Human code problem (continued) COMP36/6466: Lecture

10 6.3 The Human code problem (continued) For eicient implementation o a Human tree, we can use a priority queue data structure - a minimum heap. It has a set o weighted items as its key value, with ast (such as O(logn)) execution o the operations Extract MIN: ind and delete the element with the least weight, and Insert: insert an element with a key to the priority queue. Human(C) n C ; 2 Q C; /* The priority queue */ 3 or i to n 4 do z allocate node(); 5 le t[z] = x Extract MIN(Q); 6 right[z] = y Extract MIN(Q); 7 [z] [x] + [y]; 8 Insert(Q, z); 9 return Extract MIN(Q). COMP36/6466: Lecture 7 27

11 6.3 The Human code problem (continued) Exercise: What is the running time o the greedy algorithm or the construction o a Human tree? assuming that there are n characters in a document. COMP36/6466: Lecture 7 27