Error-Resilient LZW Data Compression

Transcription

1 Error-Resilient LZW t Compression Yonghui Wu Stefno Lonrdi ept. Computer Science & Engineering University of Cliforni Riverside, CA {yonghui stelo}@cs.ucr.edu Wojciech Szpnkowski eprtment of Computer Sciences Purdue University West Lfyette, IN 490 sp@cs.purdue.edu Abstrct 1 Introduction As mny prctitiones know, compressed strems re very sensitive the trnsmission errors. Even single error cn hve devstting effects nd compromise ll the dt downstrem [5]. In fct, the non-resilience of dptive dt compression hs been prcticl drwbck of its use in mny pplictions. Joint source-chnnel coding [4, 1] hs emerged s possible solution to this problem. Usully, joint source-chnnel coding trdes source bits for chnnel bits or vice vers, nd more thn often requires some djustments in source coding nd chnnel coding prts. In this pper, we del with very populr compression scheme, nmely the so-clled Lempel- Ziv-Welch (LZW). LZW is lossless dt compression lgorithm developed by T. Welch in 1984 [8] s n improved version of the LZ8 dictionry coding lgorithm developed by A. Lempel nd J. Ziv [9]. The method becme modertely widely-used when the progrm compress becme more or less the stndrd compression utility in Unix systems circ by the yer of In 198, the lgorithm ws dopted s prt of the IF imge formt, nd hs since been very widely used. LZW is used in the V.42bis modem stndrd [] nd cn lso optionlly be used in TIFF imges nd PF formtted documents. Here focus on the problem of dding error-resiliency to the LZW compression scheme. Compred to our previous work on LZ- [1], extrcting from LZW/LZ-8 the extr redundncy bits needed to store the error correcting prity bits turned out to be significntly trickier. The constrins tht we hd in the design of the new scheme were minly two. First, we wnted to mintin the bckwrd comptibility with the originl LZW, tht is, we wnted file compressed with LZW nd ugmented for error-resiliency to be decodble by the originl LZW. Bckwrd-comptibility is very desirble property becuse it mkes it possible to deploy the new scheme grdully over the existing one, without disrupting service. Second, we wnted the compression rtio not to be ffected too much by the embedding of the extr prity bits for the detection nd correction of errors. We were ble to chieve both objectives by relxing (i.e., shortening) few selected LZW phrses in the compressed strem, so tht the pttern of shorter phrses would encode the extr 1

2 C C C bits. Since we re not shortening too mny phrses the compression is not ffected much (see Experimentl results). In fct, ccording to our experimentl results, sometimes the sum of the 2 Bsic concepts Let be text of length over finite lphbet, nd let be its corresponding LZWcompressed strem. We write to indicte the symbol of the text. We use shorthnd for #, where!, with the convention tht. Substrings in the form $ % correspond to the prefixes of, nd substrings in the form & ( to the suffixes of. iven two strings ) nd, ),+- is the string obtined by conctenting ) with. 0/ :::<; We use. to denote the LZW dictionry used during the encoding nd decoding processes. By construction of the LZW dictionry we hve tht 1= > for some nd B. 1 The length C C of phrse 1E is the number of symbols it contins, i.e., C B$+H. For completeness of presenttion we briefly review the originl LZW scheme [8, 9]. The lgorithm prses the text online left to right into phrses, where ech phrse is the longest mtching phrse seen previously plus one extr symbol. Ech new phrse is dded to the dictionry. of phrses, which is first initilized to include every single symbol in the input lphbet. The index of the longest mtching phrse is dded to the output s new codeword, wheres the extr symbol, i.e., the lst symbol of the current phrse, becomes the first symbol of the next phrse. To decode the compressed text I, the decoder first fills the dictionry. with ll the symbols in the input lphbet. It then reds the codewords one by one from the compressed text. Every time the decoder reds new codeword, it looks up the dictionry for the corresponding phrse. The string identified by the codeword is dded to the output, while new phrse, which is constructed by ppending the first symbol of the current codeword to the previous codeword, is dded to the dictionry.. 3 Extrcting bits from the LZW strem ue to the greediness inherent to the LZW lgorithm, t ny point of the encoding process there is lwys only one wy of producing the next phrse, nd hence, every phrse in the dictionry. is unique. The greediness prevents us from embedding directly extr bits into the compressed dt strem. A simple solution to this problem is to relx the greediness on some of the phrses (i.e., by mking them little shorter) in such wy to encode the extr bits. Relxing the length of too mny phrses will, however, degrde the compression performnce considerbly. Cre must be tken to select the set of phrses to shorten tht will llow the necessry extr spce to store the bits for error detection nd correction. Also, we must ensure tht the new compressed strem cn be decompressed by the originl lgorithm. Note tht by relxing the greediness some entries in the dictionry. will hve multiple codewords ssocited with them. A somewht similr pproch ws tken in [6], where the uthor nlyze scheme where the entries in the dictionry to hve multiplicity up to constnt J. Figure 1 show n exmple. On the left, the text =b is compressed with the greedy LZW lgorithm. The corresponding compressed dt strem is K =0231. By the end of the encoding process, the dictionry contins five phrses, nmely 13L =, 13 =b, 1E =, 1E =, 2

3 Phrses:,,, b Output: 0, 2, 3, 1 Phrses:,,,, b Output: 0, 2, 2, 0, 1 b b 0 b b 4 greedy-lzw lgorithm relxed-lzw lgorithm Figure 1: reedy-lzw prsing vs. relxed-lzw prsing on the input =b nd 1 =b. Phrses 1EL nd 13 re from the input lphbet, wheres 1F, 1E, nd 1, re constructed greedily s the text is prsed from left to right. If insted of generting the next phrse in greedy mnner ll the time, we reduce the length of 1 by one, we will get the result s shown in the right prt of the figure. This time, the dictionry contins 6 phrses nd the output, 02201, is one codeword longer thn tht in the greedy cse. The two phrses, 1 nd 1, re identicl, which is s expected. The dictionry. nd. re both shown s tries in the figure. The crucil question is whether this modifiction in the encoder will produce compressed file tht cn still be decompressed by the originl LZW lgorithm. The nswer to this question depends on the dt structure used to represent the dictionry. Although in the literture, the dictionry is lwys represented s trie, ll the implementtion we checked use fixed-size hsh tble (typiclly with 4096 entries). Thus, the decoder is not ffected by the multiplicities in the dictionry entries. All it does is to refer to the existing phrses by their indices, whether there re multiple such phrses or not, nd conctentes two strings to produce the next phrse. Becuse of this, our relxed-lzw scheme is bckwrd-comptible with the greedy-lzw. We verified the bckwrd comptibility of our scheme on vrious softwre tht support the LZW scheme. For exmple, for IF imges, we tested S Pint, S Internet Explorer nd ozill Firefox. We lso tested Unix Compress nd Winzip. All these ltter softwre were ble to decompress relxed- LZW strem. We expect PF nd TIFF files to be cpble of hndling our scheme s well. Once the bckwrd comptibility hve been ssessed, we need to tke the compression performnce into considertion. A detiled description on the relxed-lzw scheme is in order. We cll the LZW phrses tht will become shorter becuse of the relxtion, non-greedy phrses. Let denote the messge over the binry lphbet tht is going to be embedded into the compressed text. The cpcity of the embedding chnnel depends on two integer prmeters nd. The integer specifies the number of bits of the messge tht re embedded in the intervl between two consecutive non-greedy phrses, wheres the prmeter specifies the number of bits tht is embedded in the length of ech non-greedy phrse. For exmple, when we... The messge to be embedded is first logiclly divided into consecutive blocks of + bits. Every block is further divided into two sub-blocks of nd bits ech. The contents of the 3

4 ESSAE EBEIN LZW ENCOER get next bits from 3. if then 4. while (hve not finished encoding yet) 5.! #%$&() next phrse s ccording to the stndrd LZW lgorithm 6. if +#%$,&-,./%0+ then. 1! if56 then get next bits from 11. if :8;< then 8; %$&() reduce the length of + #%$,&- by 8 symbols 13. get next bits from 14. if4 then 15. 2=+ #%$,&- 16. return Figure 2: The sketch of the encoder cpble of embedding messge while compressing into LZW strem. nd re two integer prmeters of the lgorithm (see text) two sub-blocks re interpreted s positive integers, nd for clrity purposes, we denote them by 4E 4H B nd > respectively. Note tht 1 B@?ACB H(F nd >? AB 0 H-F. The messge is embedded into one block time while compressing ccording to LZW, s follows. We initilize counter to B, nd generte new phrses greedily s per the greedy LZW lgorithm. Every time new phrse is generted, its length is compred to the vlue 0 (note tht 0 is the mximum vlue for >, see footnote). If the length of the phrse is greter we increment the counter by 1. As soon s the counter equls the vlue B we relx the length of tht phrse by > symbols. Then, the counter is reset to 0, nd new cycle begins (see Figure 2) The decoding process is rther strightforwrd. As codewords re red from, phrses re being reconstructed. For ech phrse the decoder determines whether the phrse is greedy or nongreedy. If the phrse is non-greedy phrse, block of messge is recovered ccording to the rules we followed to embed it. At the sme time, the originl text is lso rebuilt s per the generic LZW lgorithm. Note tht in order to determine whether or not phrse is non-greedy, we need to look hed severl phrses. This cn be done by employing some sort of look-hed buffer. A sketch of the decoder is illustrted in Figure 3. As finl remrk, we wnt to note tht in the strtegy described bove we hve not exploited the multiplicities in the dictionry to embed even more bits of the messge. When the relxed-lzw lgorithm looks for longest prefix of text to be compressed tht is contined the dictionry., there might be multiple such longest phrses in the dictionry, due to the fct tht we hve reduced the lengths of some of the previous phrses. If tht is the cse, we cn embed free of chrge 9N nother I:J5K,L where is the multiplicity of the longest phrse. A similr ide is exploited in [1, 2] to embed extr bits in LZ- strems. 1OQP@R nd S PTR re treted s specil cse, by mpping them to U(V nd UHW, respectively. 4

5 C ESSAE EBEIN LZW ECOER 1. E( 2. while (hve not finished decoding yet) 3.! #%$&(3 decode the next phrse s ccording to the stndrd LZW decoder %$&(3 5. Fill E 6.! #%$&( encoder the text in E s ccording to the stndrd LZW encoder. if +#%$,&--6,./ ! if +#%$,&-3 +6%$&( & 2 +#%$,&--6!#%$&(3 & return ) Figure 3: The sketch of the decoder cpble of recovering the embedded messge. nd re two integer prmeters of the lgorithm (see text) from LZW strem 4 Selection of prmeters nd Embedding extr informtion into the LZW dt strem is clerly not free of chrge. With dditionl bits embedded, the size of the new LZW strem is usully lrger thn tht of the originl one, s is illustrted by the exmple in Figure 1. However, by choosing the two prmeters nd judiciously, the size will not increse by too much. In fct, we will show in the Section tht when is 5 or higher, the size of the new LZW file is typiclly less thn the sum of the sizes of the originl file nd the messge file. The compression tend to degrde s decreses nd increses, but t the sme time the number of bits embedded in the messge will increse. In the error resilient ppliction it is crucil to determine the best trdeoff. Once the file is compressed with the originl greedy-lzw, the strem is broken into blocks (see Section 6) nd the totl number of prity bits will be computed. In this section, we discuss how to choose nd so tht we cn estimte the number of bits tht cn be embedded in I. The im is to crete enough extr bits for the length of the messge to be embedded, i.e., ll the prity bits of the error-correcting code, but not much more so tht the compression will suffer. In our nlysis, we ssume tht during the embedding of the messge, the lengths of the phrses re lwys greter thn 0, which of course is not true t the beginning of the file. However, s long s the text is long enough nd s new phrses re being generted nd inserted into the dictionry, the ssumption is likely to be stisfied. For simplicity, we ssume the messge to be embedded is generted by n i.i.d. model with 0 nd 1 hving equl probbilities. Then, on verge we will get non-greedy phrse every + phrses. On verge, the length of the non-greedy phrse & will be reduced by 0 + symbols. For simplicity in our exposition, we set + nd 0 +. Let ssume tht C nd C C. Let us cll the portion of tht is encoded by greedy phrses, nd cll the portion of encoded by non-greedy phrses. Intuitively, if we zip nd t the points where is broken into phrses, we get bck. Let the sizes of nd be nd respectively. Clerly, +. 5

6 ) 4 + L / > The set of unique phrses in the dictionry is determined by the greedy phrses (in ). The number of the unique phrses,, is then pproximtely equl to, where is the entropy of. The verge length of the greedy phrses, >, will be roughly. The number of blocks of messge,, tht is embedded in the text will be, nd hence we hve + +. Let > be the verge length of the non greedy phrses. Then we hve >, nd >. The number of non-greedy phrses in the dictionry is equl to the number of blocks of messges embedded. The totl number of phrses,, generted by our lgorithm is the sum of tht of greedy phrses nd of the non-greedy phrses, which is equl to +. To sum up, we hve the following equlities: > > + J K,L& J K,L + From the bove set of equtions, cn be determined given nd, so will be, nd. The cpcity of our messge-embedding chnnel cn be represented s the rtio of over. The performnce of our lgorithm, in terms of compression rtio, cn be determined by compring! the totl number of phrses,, tht is generted by our lgorithm to the number of phrses, #%$&!, tht would be produced if we were to compress the conctention of the text nd the messge + simply by the generic LZW lgorithm. 5 Asymptotic nlysis of the redundncy In this section we re concerned with the nlysis of the redundncy of the new scheme when the input is lrge. In prticulr we re interested in compring the redundncy of the relxed-lzw scheme with the originl LZW/LZ-8. Here, we follow the nottion from [3]. On verge our relxed-lzw lgorithm decreses the number of mnipulted symbols by ( where ( is the yet unknown (verge) number of phrses when string of length is compressed. Let, s in [3], define ( ( J5K,L+(=, (= ( H+.- J K,L+( 0 (1) where,

7 B L L 3 L (, ) B nd where J5K,L + J5KL J K,L, nd J K,L 9: B is the @ J is the Euler constnt, The function 4 ( is fluctuting function with men zero nd smll mplitude for J5KL J K,L rtionl (e.g., the mplitude of 4 ( is smller thn %B for the unbised cse, where ), nd J 4 ( B otherwise. efine now ( s ( Then, s in [3], the verge number of phrses of our lgorithm is A the code length of our lgorithm is? -AJ K,L ( Thus, the reltive verge redundncy ( + becomes? -AJ K,L ( + J5K,L? + (F + (F F ( E (2). Observe tht Compring it to the regulr LZW/LZ 8, the redundncy of relxed-lzw is incresed by J5K,L. 6 Error resilient LZW Experimentl Results In order to vlidte our theoreticl studies nd test the correctness of our scheme, we implemented our lgorithm nd tested it on IF files, which, s is well-known, is compressed by LZW lgorithm. We downloded from Internet severl stndrd pictures for digitl imge processing, nd tested our lgorithm with them. Tble 1 shows our experimentl results with nd being set to nd B respectively. The essge (bytes) column indictes how much rndom informtion we cn embed into the originl IF file. Column Cmprssn Loss is defined to be the size of the new IF file minus the sum of the size of the originl IF file plus the size of the rndom messge. If this number is negtive, it mens tht overll we re gining more compression thn the stndrd LZW lgorithm. As we cn see from the tble, irplne.gif nd couple.gif re better thn the rest of the IF files in terms of compression loss, nd bboon.gif is the worst mong ll of them. This is becuse the compression rtio for irplne.gif nd couple.gif (5. nd 4.88 respectively) is much higher thn tht for bboon.gif (2.49), nd reducing the lengths of phrses will not hve s drmtic negtive impct on the file size for IF files with high compression rtios thn for those with low compression rtios. Column chnnel cpcity is defined to be the rtio of the size of the rndom messge over the size of the originl IF (3)

8 Originl IF file Width Height BPP Size (bytes) Averge LZW Phrse Size (bytes) essge embedded IF file Averge ssg LZW Cmprssn Phrse Loss (bytes) Chnnel Cpcity irplne.gif bboon.gif couple.gif girl.gif len.gif peppers.gif Tble 1: Compring the size of some IF files compressed with the stndrd LZW versus the messge embedding LZW scheme : 3 file. We notice tht the vrition of chnnel cpcity mong the files re quite smll, except bboon.gif, which gin we believe is due to its reltively poor compression rtio. Tble 2 shows our experimentl results with vrious prmeter settings. Clerly, the chnnel cpcity decreses s the prmeter increses. However, the reltionship between the chnnel cpcity nd the prmeter is not s cler. To our observtions, when the prmeter is smll, sy less thn 4, incresing the prmeter from 0 to 1 usully increses the chnnel cpcity s well. However, going beyond 1 neither helps chnnel cpcity nor does it improve the imge compression rtio. References [1] LONARI, S., AN SZPANKOWSKI, W. Joint source-chnnel LZ coding. In IEEE t Compression Conference, CC (Snowbird, Uth, rch 2003), J. A. Storer nd. Cohn, Eds., IEEE Computer Society TCC, pp [2] LONARI, S., SZPANKOWSKI, W., AN WAR,. Error resilient LZ nd its nlysis. In IEEE Interntionl Symposium on Informtion Theory (ISIT 04) (Chicgo, IL, June 2004), p. 56. [3] LOUCHAR,., AN SZPANKOWSKI, W. On the verge redundncy rte of the Lempel-Ziv code. IEEE Trns. Inf. Theory 43 (199), 2 8. [4] SAYOO, K., OTU, H., AN EIR, N. Joint source/chnnel coding for vrible length codes. IEEE Trns. Inf. Theory 48 (2000), [5] STORER, J. A., AN REIF, J. H. Error-resilient optiml dt compression. SIA Journl on Computing 26, 4 (199), [6] SZPANKOSWKI, W., AN KNESSL, C. A note on the symptotic behvior of the height in -tries for lrge. Electronic J. of Combintorics (2000), R39. [] THOBORSON, C. The V.42bis stndrd for dt-compressing modems. IEEE icro 12, 5 (Oct. 1992),

9 ssg (bytes) Averge LZW Phrse Cmprssn Loss Chnnel Cpcity ssg (bytes) Averge LZW Phrse Cmprssn Loss Chnnel Cpcity irplne.gif bboon.gif couple.gif girl.gif len.gif peppers.gif irplne.gif bboon.gif couple.gif girl.gif len.gif peppers.gif irplne.gif bboon.gif couple.gif girl.gif len.gif peppers.gif irplne.gif bboon.gif couple.gif girl.gif len.gif peppers.gif irplne.gif bboon.gif couple.gif girl.gif len.gif peppers.gif irplne.gif bboon.gif couple.gif girl.gif len.gif peppers.gif irplne.gif bboon.gif couple.gif girl.gif len.gif peppers.gif Tble 2: Experimentl results for severl choices of the prmeters 9

10 [8] WELCH, T. A. A technique for high-performnce dt compression. IEEE Computer 1, 6 (June 1984), [9] ZIV, J., AN LEPEL, A. Compression of individul sequences vi vrible-rte coding. IEEE Trns. Inf. Theory 24, 5 (Sept. 198),