COMP 423 lecture 11 Jan. 28, 2008

Transcription

1 COMP 423 lecture 11 Jn. 28, 2008 Up to now, we hve looked t how some symols in n lphet occur more frequently thn others nd how we cn sve its y using code such tht the codewords for more frequently occuring symols re shorter thn the codewords for less frequently occuring symols. The methods we hve seen still ignore severl common types of proilistic structure tht is found in rel sequences, however, nmely interctions etween the occurences of different symols in n lphet. In English, q is lmost lwys followed y u for exmple. (The word Irq is one exception, since in tht cse q is followed y lnk.) The next few methods we exmine will ddress these interctions. Lempel-Ziv methods Lempel-Ziv methods re nmed fter two reserchers, Lempel nd Ziv, who invented them nd who provided the first proofs on their performnce. Suppose we hve sequence of n symols drwn from some lphet {A 1, A 2,...,A N }. Let X j stnd for the j th symol in the sequence. (Lter we will think of X j s rndom vrile. For now, just tret it s vrile which hs prticulr vlue, nmely some symol in the lphet. ) LZ version 1 (LZ1) Here s the sic Lempel-Ziv ide which I ll cll LZ1. Suppose we hve lredy encoded the first j 1 symols in the sequence (X 1, X 2,..., X j 1 ) for some j, using some s yet unspecified scheme. Now we wnt to encode the rest of the sequence, (X j,...,x n ). We do so y finding the lrgest susequence of (X j,...,x n ) tht egins t index j nd tht mtches susequence tht egins prior to position j. As long s this lrgest mtching susequence hs length greter thn or equl to one, then we don t need to code it, since we hve seen it lredy! Insted we cn just write down how mny symols long is the mtch, nd where the mtching susequence egn. There re two cses: 1. The symol X j did not occur in (X 1, X 2,..., X j 1 ). In this cse, we encode tht the length is 0, nd we send code of tht symol. (We need to hve code for ech of the symols of the lphet.) 2. The symol X j did occur in (X 1, X 2,...,X j 1 ). In this cse, we encode the length of the mtching susequence (> 0) nd the offset. The offset is the numer of symols ck in the sequence where this mtching susequence egins i.e. If the susequence egins t symol, X j m, then the offset is m. The length is just the length of the longest mtching susequence. Note tht we could hve done it slightly differently, coding the offset first (with 0 offset mening tht there ws no previous mtching sustring) nd then encoding either the length (offset > 0) or the new symol (offset = 0). A mtching susequence is clled phrse. The ide of Lempel-Ziv methods is to prtition the sequence into phrses nd to encode ech phrse. In cse 1 ove, the phrse hs only one symol, nmely the new symol tht hs not occured efore. In cse 2, the phrse hs 1 or more symols. To implement the ove coding method, we need to hve three codes: C 1 : code for the length of est mtch, defined on {0, 1, 2, 3,... } C 2 : code for the individul symols, defined on {A 1, A 2,... } 1

2 COMP 423 lecture 11 Jn. 28, 2008 C 3 : code for the offsets, defined on {1, 2, 3,... } Notice tht the code for the lengths is defined on 0 s well s the positive integers, wheres the code for offsets is defined only on the positive integers. Exmple 1: meet me t the thetre length symol offset 0 m 0 e t 0 lnk h r 1 4 The sequence is encoded: C 1 (0)C 2 (m)c 1 (0)C 2 (e)c 1 (1)C 3 (1)C 1 (0)C 2 (t)c 1 (0)C 2 (lnk)c 1 (2)C 3 (5)C 1 (1)C 3 (3)C 1 (0)C 2 ()... The decoder cn decode this sequence of its, provided it knows the codes C 1, C 2, C 3, nd provided it knows tht ech phrse will e encoded s pir: either (length,symol) or (length,offset), depending on whether length is zero or greter thn zero. Exmple 2:... The second exmple is more sutle: length symol offset The sutlety here is tht the est mtching susequence (strting t X j ) must itself strt in X 1...X j 1, ut it needn t e restricted to X 1...X j 1. (This might seem like contrived exmple. However, in cses where you hve lots of lnk chrcters in file, this phenomenon comes up quite lot.) 2

3 COMP 423 lecture 11 Jn. 28, 2008 Let s mke some generl oservtions out this lgorithm. Ech symol in the lphet is coded t most once (using code C 2 ) so we don t need to worry much out efficiency for this code. The lengths tend to e smll numers. While it is common to hve words or sequences of words repeted in n English text (e.g. in the morning ), it is rre tht these repeted sequences re more thn sy 50 chrcters long. This would e considered d style. The offsets tend to e lrge. For English text, words might e repeted throughout the text, nd they my even come in clusters, ut phrses re typiclly seprted y hundreds or even thousnds of positions. Thus, we don t wnt to use the sme code for offsets nd we do for lengths of mtch! Lempel-Ziv version 2 (LZ2): sliding window One of the issues with LZ1 ove is tht it might use too mny its to encode the offset. Since the est mtching phrse cn egin nywhere in (X 1, X 2,...,X j 1 ), the offset cn e ny numer from 1 to j. One simple method to void the prolem, which I ll cll LZ2, is to restrict the serch to finite size window. The window is of size n w symols, where n w is power of 2, typiclly 2 15 or so. The lgorithm serches ckwrds only over the window (X j nw,...,x j 1 ) rther thn ll the wy ck to the eginning of the sequence. Of course, if j < n w then it only serches ck to the eginning of the sequence. This version of the lgorithm is clled sliding window ecuse the window of n w symols is lwys the n w symols tht come efore the next symol(s) to e coded. Agin need to encode offsets, lengths, nd symols. Suppose tht n w nd N re oth powers of 2. We could use: length of mtch, L: use Golom or Elis code defined in n erlier lecture to encode L + 1. (It is possile tht L = 0.) This uses shorter codewords for shorter length mtches, since shorter mtch lengths tend to e more likely in prctice. offset: use log n w its (fixed length code) symols: use log N its (fixed length code) Lempel-Ziv method 3 (LZ3) The next LZ method we consider, which I will refer to s LZ3, tkes different pproch. Rther thn serching for the longest mtch to ny susequence strting in X 1,...,X j 1, LZ3 insted serches for the longest mtch to ny phrse in X 1,...,X j 1. Once it finds this mtch, the next phrse (strting t X j ) is the longest mtching phrse just found, conctented with the next (unmtched) symol. LZ3 does not encode the offset; insted it encodes the phrse numer. LZ3 uilds tree of phrses (see elow). Note tht the numer of phrses in X 1,...,X j 1 tends to e much less thn the numer j 1 of symols tht hve ppered. Thus, to encode the est mtching phrse typiclly requires fr fewer its thn to encode the ritrry offset s in LZ1. 3

4 COMP 423 lecture 11 Jn. 28, 2008 Exmple Let s consider n exmple. Tke the sequence. We prse it s follows:,,,,,,,,. As the encoder prses the sequence, it uilds the tree shown on the left elow. Note tht the finl phrse termintes t n interior node in the tree. (It could e root node, or non-root internl node.) How do we del with this cse? One simple wy would e to encode the current numer of phrses prsed plus 1. (Cll this phrse ct + 1.) The decoder then knows tht the encoder hs reched the finl phrse. Once this fct hs een encoded, the encoder could give the internl node t which this finl phrse termintes. This could e nywhere from node 0 to node phrse ct ENCODER TREE DECODER TREE For the ove exmple, the encoder would send the following codewords. Let C e the code for numers nd let C e the code for symols. C(0) C () C(0) C () C(2) C () C(3) C () C(3) C () C(5) C () C(6) C () C(1) C () C(9) C(2) Encoding lgorithm (LZ3) phrse ct = 0; root = empty; while not endoffile phrse ct = phrse ct + 1; trverse tree to find the longest mtching prefix phrse; 4

5 COMP 423 lecture 11 Jn. 28, 2008 if end of file encode (phrse ct, index of mtching prefix phrse) // if no mtch, then longest mtching phrse is root i.e. phrse 0) else encode (index of longest mtching prefix phrse, next non-mtching symol); insert child (phrse ct, next symol) elow lef of longest mtching prefix; endif end while LZ1 nd LZ2 vs. LZ3: numer of phrses Let s riefly compre the prsings of LZ1 nd LZ2 vs. LZ3 in terms of the numer of phrses. Exmple:... LZ1/2 would prse the sequence s,..., wheres LZ3 would prse it s,,,,,,.... Thus, LZ2 hs fewer phrses for this exmple. You might suspect tht LZ2 lwys gives fewer phrses thn LZ3 since LZ2 cn mtch to ny previous sequence. However, this intuition turns out to e flse, s the following exmple shows. Exmple: meet me t the thetre LZ2: m,e,e,t,,me,,,t,t,h,e,the,t,r,e (16 phrses) LZ3: m,e,et,,me,,t, t,h,e,th,e,tr,e (14 phrses) Wht s going on here? Although LZ3 cn only mtch to previously seen phrses, when it does mtch it dds symol the length of the est mtch. In this sense, the phrses of LZ3 cn e longer thn those of LZ2. LZ3 decoder Insted of constructing tree, the LZ3 decoder constructs tle. The tle consists of set of pirs: (index to longest mtching prefix phrse, next symol). These re the pirs tht the encoder hd encoded. Note tht the entries in the tle define inry tree similr to the one used y the encoder except tht now the rnches point upwrds from child to prent, rther thn downwrd from prent to child. (See DECODER TREE on previous pge.) Here is wht the tle would look like for the ove exmple where the * indictes tht different coding method is used for the finl phrse in the sequence. 5

6 COMP 423 lecture 11 Jn. 28, 2008 index index of prent/prefix lst symol 0 null Mke sure tht you understnd how the decoder cn use the tle to reconstruct the originl sequence. In prticulr, note tht the prefixes re reconstructed ckwrds, since the decoder trverses the tree/tle from lef to root. 6