Lempel-Ziv-Welch Compression

Transcription

1 Lempel-Ziv-Welch Compression Brad Karp UCL Computer Science CS th February 2018 (lecture notes derived from material from Hari Balakrishnan, Katrina LaCurts, and Terry Welch) 1

2 The Compression Problem Network links and storage devices ever-increasing in capacity But data sets we manipulate grow (at least) commensurately What to do when a data set won t fit on your disk, or transfer quickly enough on your (e.g.) DSL line? Compress the data: transform into an informationpreserving, yet more compact (in bits) representation Transformation should be reversible (compression, decompression) Lossy compression: get back something close to original data (e.g., JPEG image compression, MPEG-4 video compression, MP3 or AAC audio compression) Lossless compression: get back exactly the original data, bit-for-bit 2

3 Lempel-Ziv-Welch Compression Proposed by Terry Welch in 1984 (MIT graduate; researcher at systems powerhouse Digital Equipment Corporation (DEC)) Enhances earlier algorithm due to Lempel and Ziv Central ideas: No foreknowledge of frequency of patterns in input needed (compressor learns patterns as it compresses) No explicit transmission of dictionary mapping compressed patterns to decompressed patterns (decompressor derives dictionary as it decompresses; wicked cool!) Requires choice of input symbol width (in bits) Typically 8 bits for compressing arbitrary file data; CW3 puts a twist on this, and requires you to use 2-bit symbols Requires choice of initial dictionary size (in entries, which determines initial width of quantum of compressed output, in bits) 3

4 LZW: Concepts and Definitions Central data structure: table of codes ( string table in [Welch]) Compressor starts with table of size 2 N, where N is parameter chosen by implementor; [Welch] uses N=12 Input divided into symbols; in [Welch] and this lecture, input symbols are 8 bits wide (bytes); 2 bits wide in CW3! Output consists of codes; fixed width (12 bits) in [Welch]; variable width in CW3, initialized to 3 bits wide! 4

5 LZW: Concepts and Definitions Central data structure: table of codes ( string table in [Welch]) Compressor starts with table of size 2 N, where N is parameter chosen by implementor; [Welch] uses N=12 Input divided into symbols; in [Welch] and this lecture, input symbols are 8 bits wide (bytes); 2 bits wide in CW3! Output consists of codes; fixed width (12 bits) in [Welch]; variable width in CW3, initialized to 3 bits wide! index string initialized to all 256 byte values populated with strings from input during compression 5

6 LZW Compressor Pseudocode (where + denotes concatenation) 6

7 LZW Example: Compress Carnegie Mellon abbbabbbab (integer code output) S: c: abbbabbbab 7

8 LZW Example: Compress Carnegie Mellon abbbabbbab (integer code output) S: a c: abbbabbbab 8

9 LZW Example: Compress Carnegie Mellon abbbabbbab (integer code output) S: a c: b S+c in table? No. Emit index[s]. Add S+c to table. abbbabbbab 9

10 LZW Example: Compress Carnegie Mellon abbbabbbab (integer code output) S: a c: b 97 S+c in table? No. Emit index[s]. Add S+c to table. S ß c abbbabbbab 10

11 LZW Example: Compress Carnegie Mellon abbbabbbab (integer code output) S: b c: 97 S+c in table? No. Emit index[s]. Add S+c to table. S ß c abbbabbbab 11

12 LZW Example: Compress Carnegie Mellon abbbabbbab (integer code output) 258 S: b c: b 97,98 S+c in table? No. Emit index[s]. Add S+c to table. abbbabbbab 12

13 LZW Example: Compress Carnegie Mellon abbbabbbab (integer code output) 258 S: b c: b 97,98 S+c in table? No. Emit index[s]. Add S+c to table. S ß c abbbabbbab 13

14 LZW Example: Compress Carnegie Mellon abbbabbbab (integer code output) 258 S: b c: 97,98 S+c in table? No. Emit index[s]. Add S+c to table. S ß c abbbabbbab 14

15 LZW Example: Compress Carnegie Mellon abbbabbbab (integer code output) 258 S: b c: b 97,98 S+c in table? Yes. S ß S+c abbbabbbab 15

16 LZW Example: Compress Carnegie Mellon abbbabbbab (integer code output) 258 S: bb c: 97,98 S+c in table? Yes. S ß S+c abbbabbbab 16

17 LZW Example: Compress Carnegie Mellon abbbabbbab (integer code output) 258 S: bb c: a 97,98 S+c in table? No. Emit index[s]. Add S+c to table. abbbabbbab 17

18 LZW Example: Compress abbbabbbab (integer code output) abbbabbbab 258 bba S: bb c: a 97,98,257 S+c in table? No. Emit index[s]. Add S+c to table. S ß c 18

19 LZW Example: Compress abbbabbbab (integer code output) abbbabbbab 258 bba S: a c: 97,98,257 S+c in table? No. Emit index[s]. Add S+c to table. S ß c 19

20 LZW Example: Compress abbbabbbab (integer code output) abbbabbbab 258 bba S: a c: b 97,98,257 S+c in table? Yes. S ß S+c 20

21 LZW Example: Compress abbbabbbab (integer code output) abbbabbbab 258 bba S: ab c: 97,98,257 S+c in table? Yes. S ß S+c 21

22 LZW Example: Compress abbbabbbab (integer code output) abbbabbbab 258 bba S: ab c: b 97,98,257 S+c in table? No. Emit index[s]. Add S+c to table. 22

23 LZW Example: Compress abbbabbbab (integer code output) abbbabbbab 258 bba abb S: ab c: b 97,98,257,256 S+c in table? No. Emit index[s]. Add S+c to table. S ß c 23

24 LZW Example: Compress abbbabbbab (integer code output) abbbabbbab 258 bba abb S: b c: 97,98,257,256 S+c in table? No. Emit index[s]. Add S+c to table. S ß c 24

25 LZW Example: Compress abbbabbbab (integer code output) abbbabbbab 258 bba abb S: b c: b 97,98,257,256 S+c in table? Yes. S ß S+c 25

26 LZW Example: Compress abbbabbbab (integer code output) abbbabbbab 258 bba abb S: bb c: 97,98,257,256 S+c in table? Yes. S ß S+c 26

27 LZW Example: Compress abbbabbbab (integer code output) abbbabbbab 258 bba abb S: bb c: a 97,98,257,256 S+c in table? Yes. S ß S+c 27

28 LZW Example: Compress abbbabbbab (integer code output) abbbabbbab 258 bba abb S: bba c: 97,98,257,256 S+c in table? Yes. S ß S+c 28

29 LZW Example: Compress abbbabbbab (integer code output) abbbabbbab 258 bba abb S: bba c: b 97,98,257,256 S+c in table? No. Emit index[s]. Add S+c to table. 29

30 LZW Example: Compress abbbabbbab (integer code output) abbbabbbab 258 bba abb bbab S: bba c: b 97,98,257,256,258 S+c in table? No. Emit index[s]. Add S+c to table. S ß c 30

31 LZW Example: Compress abbbabbbab (integer code output) abbbabbbab 258 bba abb bbab S: b c: 97,98,257,256,258 End of input. Emit index[s]. 31

32 LZW Example: Compress abbbabbbab (integer code output) abbbabbbab 258 bba abb bbab S: b c: 97,98,257,256,258,98 End of input. Emit index[s]. 32

33 What If the Table Fills? Two overall possibilities: Grow size of table and continue filling in new entries Note that width of code emitted as compressed data will now be variable: depends on how many entries currently in table If add entry to table whose index won t fit in current code width, must increment code width and begin emitting codes of this greater width (for all codes in table) We require you use this approach in CW3 (details in handout) Reinitialize table contents and start over Potential benefit that retrains table contents based on current region of input (if distribution of symbols changes as input goes on) Typical implementation: compressor outputs special CLEAR value to alert decompressor to reset its table (Don t take this approach in CW3; will fail tests!) 33

34 LZW Decompressor Pseudocode output STRING 34

35 LZW Example: Decompress (97, 98, 257, 256, 258, 98) code: S: E: 97,98,257,256,258,98 35

36 LZW Example: Decompress (97, 98, 257, 256, 258, 98) code: 97 S: E: Initialize S to table[code]. 97,98,257,256,258,98 36

37 LZW Example: Decompress (97, 98, 257, 256, 258, 98) code: 97 S: a E: Initialize S to table[code]. Emit S. 97,98,257,256,258,98 37

38 LZW Example: Decompress (97, 98, 257, 256, 258, 98) 97,98,257,256,258, code: 97 S: a E: a Initialize S to table[code]. Emit S. 38

39 LZW Example: Decompress (97, 98, 257, 256, 258, 98) 97,98,257,256,258, code: 98 S: a E: a E ß table[code] Emit E. 39

40 LZW Example: Decompress (97, 98, 257, 256, 258, 98) 97,98,257,256,258, code: 98 S: a E: b ab E ß table[code] Emit E. 40

41 LZW Example: Decompress (97, 98, 257, 256, 258, 98) code: 98 S: a E: b E ß table[code] Emit E. Add S+E[0] to table. 97,98,257,256,258,98 ab 41

42 LZW Example: Decompress (97, 98, 257, 256, 258, 98) code: 98 S: b E: E ß table[code] Emit E. Add S+E[0] to table. S ß E 97,98,257,256,258,98 ab 42

43 LZW Example: Decompress (97, 98, 257, 256, 258, 98) 97,98,257,256,258, code: 257 S: b E: ab 43

44 LZW Example: Decompress (97, 98, 257, 256, 258, 98) 97,98,257,256,258, code: 257 S: b E: ab E ß S+S[0] Emit E. 44

45 LZW Example: Decompress (97, 98, 257, 256, 258, 98) 97,98,257,256,258, code: 257 S: b E: bb abbb E ß S+S[0] Emit E. 45

46 LZW Example: Decompress (97, 98, 257, 256, 258, 98) 258 code: 257 S: b E: bb E ß S+S[0] Emit E. Add S+E[0] to table. 97,98,257,256,258,98 abbb 46

47 LZW Example: Decompress (97, 98, 257, 256, 258, 98) 258 code: 257 S: bb E: E ß S+S[0] Emit E. Add S+E[0] to table. S ß E 97,98,257,256,258,98 abbb 47

48 LZW Example: Decompress (97, 98, 257, 256, 258, 98) 97,98,257,256,258, code: 256 S: bb E: abbb 48

49 LZW Example: Decompress (97, 98, 257, 256, 258, 98) 97,98,257,256,258, code: 256 S: bb E: abbb E ß table[code] Emit E. 49

50 LZW Example: Decompress (97, 98, 257, 256, 258, 98) 97,98,257,256,258, code: 256 S: bb E: ab abbbab E ß table[code] Emit E. 50

51 LZW Example: Decompress (97, 98, 257, 256, 258, 98) 97,98,257,256,258, bba code: 256 S: bb E: ab abbbab E ß table[code] Emit E. Add S+E[0] to table. 51

52 LZW Example: Decompress (97, 98, 257, 256, 258, 98) 97,98,257,256,258, bba code: 256 S: ab E: abbbab E ß table[code] Emit E. Add S+E[0] to table. S ß E 52

53 LZW Example: Decompress (97, 98, 257, 256, 258, 98) 97,98,257,256,258, bba code: 258 S: ab E: abbbab 53

54 LZW Example: Decompress (97, 98, 257, 256, 258, 98) 97,98,257,256,258, bba code: 258 S: ab E: bba abbbabbba E ß table[code] Emit E. 54

55 LZW Example: Decompress (97, 98, 257, 256, 258, 98) 258 bba abb code: 258 S: ab E: bba E ß table[code] Emit E. Add S+E[0] to table. S ß E 97,98,257,256,258,98 abbbabbba 55

56 LZW Example: Decompress (97, 98, 257, 256, 258, 98) 258 bba abb code: 258 S: bba E: E ß table[code] Emit E. Add S+E[0] to table. S ß E 97,98,257,256,258,98 abbbabbba 56

57 LZW Example: Decompress (97, 98, 257, 256, 258, 98) 97,98,257,256,258, bba code: 98 S: bba E: abbbabbba abb 57

58 LZW Example: Decompress (97, 98, 257, 256, 258, 98) 97,98,257,256,258, bba code: 98 S: bba E: b abbbabbbab abb E ß table[code] Emit E. 58

59 LZW Example: Decompress (97, 98, 257, 256, 258, 98) 258 bba abb bbab code: 98 S: b E: E ß table[code] Emit E. Add S+E[0] to table. S ß E 97,98,257,256,258,98 abbbabbbab 59

60 LZW Corner Case: xsxsx (red if clause in Decompressor) Consider compressor with input xsx, where x is some symbol and S is some string (including null string) Suppose that xs is already in the compressor and decompressor s tables, but xsx new to compressor Compressor outputs code for xs, enters xsx in its table, sets S ß x Decompressor outputs xs, sets S ß xs Carnegie Mellon Now suppose further that next input symbols to compressor are Sxy Compressor constructs xsxy; not in table; outputs code for xsx and adds entry for xsxy Decompressor doesn t yet have xsx in table, though! It does, however, have xs. In this corner case, only possibility is that at compressor, newest table entry must have identical first and last symbols So compressor adds xsx to its table before decoding code for xsx! 60