0ШШШ J&> ELSEVIER. Lossless. (ompression Handbook KHALID SAYOOD EDITOR ^ 1 R H F A»V. -ашштюшшг

Transcription

1 - 0ШШШ J&> ELSEVIER Lossless (ompression Handbook EDITOR KHALID SAYOOD 1 ш 1 Щ: i щ v w ^ 1 W W R H F :: мяш' ж щвшш Яш' ЁИИк ШШШШШ A»V -ашштюшшг 1 % ;! ' >

2 Contents List of Contributors Preface xvii xix Part I: Theory Chapter 1: Information Theory behind Source Coding 3 Frans M. J. Willems and TjallingJ. Tjalkens 1.1 Introduction Definition of Entropy Properties of Entropy Entropy as an Information Measure Joint Entropy and Conditional Entropy Properties of Joint Entropy and Conditional Entropy Interpretation of Conditional Entropy Sequences and Information Sources Sequences Information Sources Memoryless Sources Binary Sources Discrete Stationary Sources The Entropy Rate Variable-Length Codes for Memoryless Sources A Source Coding System, Variable-Length Codes for Source Symbols Unique DecodabiUty, Prefix Codes Kraft's Inequality for Prefix Codes and Its Counterpart Redundancy, Entropy, and Bounds Variable-Length Codes for Blocks of Symbols Variable-Length Codes for Sources with Memory Block Codes Again The Elias Algorithm Representation of Sequences by Intervals Competitive Optimality 25 vii

3 viii CONTENTS 1.5 Fixed-Length Codes for Memoryless Sources, the AEP The Fixed-Length Source Coding Problem Some Probabilities An Example Demonstrating the Asymptotic Equipartition Property The Idea behind Fixed-Length Source Coding Rate and Error Probability A Hamming Ball An Optimal Balance between R and P f The Fixed-Length Coding Theorem Converse and Conclusion Chapter 2: Complexity Measures 35 Stephen R. Tate 2.1 Introduction An Aside on Computability Concerns with Shannon Information Theory Strings versus Sources Complex Non-random Sequences Structured Random Strings Kolmogorov Complexity Basic Definitions Incompressibility Prefix-free Encoding Computational Issues of Kolmogorov Complexity Resource-Bounded Kolmogorov Complexity Lower-Bounding Kolmogorov Complexity Relation to Shannon Information Theory Approach 1: An InfiniteSequence of Sources Approach 2: Conditional Complexities Discussion Historical Notes Further Reading Part II: Compression Techniques Chapter 3: Universal Codes 55 Peter Fenwick 3.1 Compact Integer Representations Characteristics of Universal Codes Polynomial Representations Unary Codes Levenstein and Elias Gamma Codes Elias Omega and Even-Rodeh Codes Rice Codes Golomb Codes Start-Step-Stop Codes 64

4 CONTENTS ix 3.10 Fibonacci Codes Zeckendorf Representation Fraenkel and Klein Codes Higher-Order Fibonacci Representations Apostolico and Fraenkel Codes A New Order-3 Fibonacci Code Ternary Comma Codes Summation Codes Goldbach Gi Codes Additive Codes Wheeler 1/2 Code and Run-Lengths The Wheeler 1/2 Code Using the Wheeler 1/2 Code Comparison of Representations Final Remarks Chapter 4: Huffman Coding 79 Steven Pigeon 4.1 Introduction Huffman Codes Shannon-Fano Coding Building Huffman Codes N-ary Huffman Codes Canonical Huffman Coding Performance of Huffman Codes Variations on a Theme Modified Huffman Codes Huffman Prefixed Codes Extended Huffman Codes Length-Constrained Huffman Codes Adaptive Huffman Coding Brute Force Adaptive Huffman The Faller, Gallager, and Knuth (FGK) Algorithm Vitter's Algorithm: Algorithm Л Other Adaptive Huffman Coding Algorithms An Observation on Adaptive Algorithms Efficient Implementations Memory-Efficient Algorithms Speed-Efficient Algorithms Conclusion and Further Reading Chapter 5: Arithmetic Coding 101 Amir Said 5.1 Introduction Basic Principles Notation 103

5 X CONTENTS Code Values Arithmetic Coding Optimality of Arithmetic Coding Arithmetic Coding Properties Implementation Coding with Fixed-Precision Arithmetic Adaptive Coding Complexity Analysis Further Reading Chapter 6: Dictionary-Based Data Compression: An Algorithmic Perspective 153 S. Cenk Sahinalp and Nasir M. Rajpoot 6.1 Introduction Dictionary Construction: Static versus Dynamic Static Dictionary Methods Parsing Issues Semidynamic and Dynamic Dictionary Methods Extensions of Dictionary Methods for Compressing Biomolecular Sequences The Biocompress Program The GenCompress Program Data Structures in Dictionary Compression Tries and Compact Tries Suffix Trees Trie-Reverse Trie Pairs Karp-Rabin Fingerprints Benchmark Programs and Standards The gzip Program The compress Program The GIF Image Compression Standard Modem Compression Standards: v. 42bis and v Chapter 7: Burrows-Wheeler Compression 169 Peter Fenwick 7.1 Introduction The Burrows-Wheeler Algorithm The Burrows-Wheeler Transform The Burrows-Wheeler Forward Transformation The Burrows-Wheeler Reverse Transformation Illustration of the Transformations Algorithms for the Reverse Transformation Basic Implementations The Burrows-Wheeler Transform or Permutation Move-To-Front Recoding Statistical Coding Relation to Other Compression Algorithms Improvements to Burrows-Wheeler Compression 180

6 CONTENTS xi 7.7 Preprocessing The Permutation Suffix Trees Move-To-Front Move-To-Front Variants Statistical Compressor Eliminating Move-To-Front Using the Burrows-Wheeler Transform in File Synchronization Final Comments Recent Developments Chapter 8: Symbol-Ranking and ACB Compression 195 Peter Fen wick 8.1 Introduction Symbol-Ranking Compression Shannon Coder History of Symbol-Ranking Compressors An Example of a Symbol-Ranking Compressor A Fast Symbol-Ranking Compressor Buynovsky's ACB Compressor Part 111: Applications Chapter 9: Lossless Image Compression 207 K. P. Subbalakshmi 9.1 Introduction Preliminaries Spatial Prediction Hierarchical Prediction Error Modeling Scanning Techniques Prediction for Lossless Image Compression Switched Predictors Combined Predictors Hierarchical Lossless Image Coding Conclusions Chapter 10: Text Compression 227 Amar Mukherjee and Fauzia Awan 10.1 Introduction Information Theory Background Classification of Lossless Compression Algorithms Statistical Methods 229

7 xii CONTENTS Dictionary Methods Transform-Based Methods: The Burrows-Wheeler Transform (BWT) Comparison of Performance of Compression Algorithms Transform-Based Methods: Star (*) Transform and Length-Index Preserving Transform Star (*) Transformation Length-Index Preserving Transform (LIPT) Experimental Results Timing Performance Measurements Three New Transforms ILPT, NTT, and LIT Conclusions Chapter 11: Compression of Telemetry 247 Sheila Horan 11.1 What is Telemetry? Issues Involved in Compression of Telemetry Why Use Compression on Telemetry Structure of the Data Size Requirements Existing Telemetry Compression Future of Telemetry Compression Chapter 12: Lossless Compression of Audio Data 255 Robert С Mäher 12.1 Introduction Background Expectations Terminology Principles of Lossless Data Compression Basic Redundancy Removal Amplitude Range and Segmentation Multiple-Channel Redundancy Prediction Entropy Coding Practical System Design Issues Numerical Implementation and Portability Segmentation and Resynchronization Variable Bit Rate: Peak versus Average Rate Speed and Complexity Examples of Lossless Audio Data Compression Software Systems Shorten Meridian Lossless Packing (MLP) Sonic Foundry Perfect Clarity Audio (PCA) Conclusion

8 CONTENTS xiii Chapter 13: Algorithms for Delta Compression and Remote File Synchronization Torsten Suel and Nasir Memon Introduction Problem Definition Content of This Chapter Delta Compression Applications Fundamentals LZ77-Based Delta Compressors Some Experimental Results Space-Constrained Delta Compression Choosing Reference Files Remote File Synchronization Applications The rsync Algorithm Some Experimental Results for rsync Theoretical Results Results for Particular Distance Measures Estimating File Distances Reconciling Database Records and File Systems Conclusions and Open Problems Chapter 14: Compression of Unicode Files Peter Fenwick Introduction Unicode Character Codings Big-endian versus Little-endian UTF-8 Coding Compression of Unicode Finite-Context Statistical Compressors Unbounded-Context Statistical Compressors LZ-77 Compressors Test Compressors The Unicode File Test Suite Comparisons UTF-8 Compression Conclusions Part IV: Standards Chapter 15: JPEG-LS Lossless and Near Lossless Image Compression Michael W. Hoffman 15.1 Lossless Image Compression and JPEG-LS 15.2 JPEG-LS

9 xiv CONTENTS Overview of JPEG-LS JPEG-LS Encoding JPEG-LS Decoding Summary Chapter 1 6: Pen-Shu Yeh The CCSDS Lossless Data Compression Recommendation for Space Applications 311 Introduction The e-rice Algorithm The Adaptive Entropy Coder Fundamental Sequence Encoding The Split-Sample Option Low-Entropy Options No Compression Code Selection Preprocessor Predictor Reference Sample Prediction Error Mapper Coded Data Format Decoding Testing Implementation Issues and Applications Additional Information Chapter 17: Lossless Bilevel Image Compression 327 Michael W. Hoffman Bilevel Image Compression JBIG JBIG Summary Overview of JBIG Encoding/Decoding JBIG Encoding Data Structure and Formatting JBIG Decoding Overview of JBIG2 JBIG2 Decoding Procedures Decoding Control and Data Structures Chapter 18: JPEG2000: Highly Scalable Image Compression 351 AH Bilgin and Michael W. Marcellin 18.1 Introduction JPEG2000 Features Compressed Domain Image Processing/Editing 353

10 CONTENTS xv Progression The JPEG2000 Algorithm Performance Tiles and Component Transforms The Wavelet Transform Quantization Bit-Plane Coding Packets and Layers JPEG2000 Codestream Chapter 19: PNG Lossless Image Compression 371 Greg Roelofs Historical Background Design Decisions Compression Engine zlib Format zlib Library Filters Practical Compression Tips Compression Tests and Comparisons MNG Further Reading Chapter 20: Facsimile Compression 391 Khalid Sayood 20.1 A Brief History The Compression Algorithms Modified Huffman Modified READ Context-Based Arithmetic Coding Run-Length Color Encoding The Standards ITU-T Group 3 (T.4) Group 4 (T.6) JBIG and JBIG2(T.82 and T.88) MRC T Other Standards Further Reading Part V: Hardware Chapter 21: Hardware Implementation of Data Compression 405 Sanjukta Bhanja and N. Ranganathan 21.1 Introduction Text Compression Hardware 407

11 xvi CONTENTS Tree-Based Encoder Example Lempel-Ziv Encoder Example Image Compression Hardware DCT Hardware Wavelet Architectures JPEG Hardware Video Compression Hardware Some Detailed Examples Commercial Video and Audio Products Index 447