Coding Trees in the Deflate format
|
|
|
- Percival Scott
- 5 years ago
- Views:
Transcription
1 Coding Trees in the Deflate format Christoph-Simon Senjak Lehr- und Forschungseinheit für Theoretische Informatik Institut für Informatik Ludwig-Maximilians-Universität München Oettingenstr.67, München PUMA/AlgoSyn Workshop 26. März 2013 Christoph-Simon Senjak (LMU München) Deflate Trees / 7
2 Outline Deflate is probably the most widespread compression format We use it as case study for low-level verification Deflates way to store encoding trees (huffman trees) is a lot more complicated than expected Christoph-Simon Senjak (LMU München) Deflate Trees / 7
3 Deflate - Basics Specified in RFC (RFC 1952 specifies the GZIP file format, and RFC 1950 specifies the ZLIB file format. Both are often confused with Deflate). Christoph-Simon Senjak (LMU München) Deflate Trees / 7
4 Deflate - Basics Specified in RFC (RFC 1952 specifies the GZIP file format, and RFC 1950 specifies the ZLIB file format. Both are often confused with Deflate). Can make use of Huffman-Codings and (extended) Run-length encoding. Christoph-Simon Senjak (LMU München) Deflate Trees / 7
5 Deflate - Basics Specified in RFC (RFC 1952 specifies the GZIP file format, and RFC 1950 specifies the ZLIB file format. Both are often confused with Deflate). Can make use of Huffman-Codings and (extended) Run-length encoding. Does not require any specific compression algorithm, even though some are recommended. Christoph-Simon Senjak (LMU München) Deflate Trees / 7
6 Deflate - Basics Specified in RFC (RFC 1952 specifies the GZIP file format, and RFC 1950 specifies the ZLIB file format. Both are often confused with Deflate). Can make use of Huffman-Codings and (extended) Run-length encoding. Does not require any specific compression algorithm, even though some are recommended. Widely used: HTTP, ZIP/RAR, PNG, SSH $ apt cache rdepends z l i b 1 g wc l 1418 Christoph-Simon Senjak (LMU München) Deflate Trees / 7
7 Deflate - Overview An informal illustration of the format: Deflate ::= ( 0 Block)* 1 Block (0 1)* Block ::= 00 UncompressedBlock 01 DynamicallyCompBl 10 StaticallyCompBl UncompressedBlock ::= length ~length bytes StaticallyCompBl ::= CompBl(standard coding) DynamicallyCompBl ::= header coding CompBl(coding) CompBl(c) ::= [^256]* 256 Christoph-Simon Senjak (LMU München) Deflate Trees / 7
8 Deflate - Overview An informal illustration of the format: Deflate ::= ( 0 Block)* 1 Block (0 1)* Block ::= 00 UncompressedBlock 01 DynamicallyCompBl 10 StaticallyCompBl UncompressedBlock ::= length ~length bytes StaticallyCompBl ::= CompBl(standard coding) DynamicallyCompBl ::= header coding CompBl(coding) CompBl(c) ::= [^256]* 256 We are interested in the coding part. Christoph-Simon Senjak (LMU München) Deflate Trees / 7
9 Deflate Codings 1. Must be prefix-free, that is, no code prefixes another code, so they form a tree. Christoph-Simon Senjak (LMU München) Deflate Trees / 7
10 Deflate Codings 1. Must be prefix-free, that is, no code prefixes another code, so they form a tree. 2. The shorter codes must lexicographically precede longer codes. (RFC 1951, 3.2.2) Christoph-Simon Senjak (LMU München) Deflate Trees / 7
11 Deflate Codings 1. Must be prefix-free, that is, no code prefixes another code, so they form a tree. 2. The shorter codes must lexicographically precede longer codes. (RFC 1951, 3.2.2) 3. All codes of a given bit length have lexicographically consecutive values, in the same order as the symbols they represent. (RFC 1951, 3.2.2) Christoph-Simon Senjak (LMU München) Deflate Trees / 7
12 Deflate Codings 1. Must be prefix-free, that is, no code prefixes another code, so they form a tree. 2. The shorter codes must lexicographically precede longer codes. (RFC 1951, 3.2.2) 3. All codes of a given bit length have lexicographically consecutive values, in the same order as the symbols they represent. (RFC 1951, 3.2.2) 4. If there is a lexicographically smaller code c of the same length for some code D(a) in the coding, then it is prefixed by some image of the coding D: a,c.c D(a) len c = len D(a) b.d(b) [ ] D(b) c Christoph-Simon Senjak (LMU München) Deflate Trees / 7
13 Deflate Codings 1. Must be prefix-free, that is, no code prefixes another code, so they form a tree. 2. The shorter codes must lexicographically precede longer codes. (RFC 1951, 3.2.2) 3. All codes of a given bit length have lexicographically consecutive values, in the same order as the symbols they represent. (RFC 1951, 3.2.2) 4. If there is a lexicographically smaller code c of the same length for some code D(a) in the coding, then it is prefixed by some image of the coding D: a,c.c D(a) len c = len D(a) b.d(b) [ ] D(b) c { y for cx = i :: y Lemma: If c is a deflate coding, then so are c i x = [ ] otherwise for i {0, 1} Christoph-Simon Senjak (LMU München) Deflate Trees / 7
14 Deflate Codings 1. Must be prefix-free, that is, no code prefixes another code, so they form a tree. 2. The shorter codes must lexicographically precede longer codes. (RFC 1951, 3.2.2) 3. All codes of a given bit length have lexicographically consecutive values, in the same order as the symbols they represent. (RFC 1951, 3.2.2) 4. If there is a lexicographically smaller code c of the same length for some code D(a) in the coding, then it is prefixed by some image of the coding D: a,c.c D(a) len c = len D(a) b.d(b) [ ] D(b) c Lemma: If c is a deflate coding, then so are c i x = { y for cx = i :: y [ ] otherwise for i {0, 1} Coding trees as internal representation possible, but too expensive for storage. Christoph-Simon Senjak (LMU München) Deflate Trees / 7
15 Deflate Codings 1. Must be prefix-free, that is, no code prefixes another code, so they form a tree. 2. The shorter codes must lexicographically precede longer codes. (RFC 1951, 3.2.2) 3. All codes of a given bit length have lexicographically consecutive values, in the same order as the symbols they represent. (RFC 1951, 3.2.2) 4. If there is a lexicographically smaller code c of the same length for some code D(a) in the coding, then it is prefixed by some image of the coding D: a,c.c D(a) len c = len D(a) b.d(b) [ ] D(b) c Lemma: If c is a deflate coding, then so are c i x = { y for cx = i :: y [ ] otherwise for i {0, 1} Coding trees as internal representation possible, but too expensive for storage. 4. is complicated but crucial to find a simpler representation. Christoph-Simon Senjak (LMU München) Deflate Trees / 7
16 Deflate Coding Sequences A deflate sequence is a sequence (a i ) i of code-lengts or 0 that satisfies Kraft s inequality 2 a i 1. i,c i 0 Christoph-Simon Senjak (LMU München) Deflate Trees / 7
17 Deflate Coding Sequences A deflate sequence is a sequence (a i ) i of code-lengts or 0 that satisfies Kraft s inequality 2 a i 1. i,c i 0 Main Theorem: The type of lists that satisfy Kraft s inequality is isomorphic to the type of Deflate codings. Christoph-Simon Senjak (LMU München) Deflate Trees / 7
18 Deflate Coding Sequences A deflate sequence is a sequence (a i ) i of code-lengts or 0 that satisfies Kraft s inequality 2 a i 1. i,c i 0 Main Theorem: The type of lists that satisfy Kraft s inequality is isomorphic to the type of Deflate codings. Saving a sequence of lengths is not expensive, and that is the way it is done in Deflate, and is sufficient, as this theorem shows. Christoph-Simon Senjak (LMU München) Deflate Trees / 7
19 Deflate Coding Sequences A deflate sequence is a sequence (a i ) i of code-lengts or 0 that satisfies Kraft s inequality 2 a i 1. i,c i 0 Main Theorem: The type of lists that satisfy Kraft s inequality is isomorphic to the type of Deflate codings. Saving a sequence of lengths is not expensive, and that is the way it is done in Deflate, and is sufficient, as this theorem shows. We made a detailled informal proof and now start to formalize it in Coq. Christoph-Simon Senjak (LMU München) Deflate Trees / 7
20 Conclusion Christoph-Simon Senjak (LMU München) Deflate Trees / 7
21 Conclusion The algorithm as such can be implemented very efficiently. Christoph-Simon Senjak (LMU München) Deflate Trees / 7
22 Conclusion The algorithm as such can be implemented very efficiently. Verification is, however, very complicated. Christoph-Simon Senjak (LMU München) Deflate Trees / 7
23 Conclusion The algorithm as such can be implemented very efficiently. Verification is, however, very complicated. Even more useful to have a verified reference implementation. Christoph-Simon Senjak (LMU München) Deflate Trees / 7
Coding Trees in the Deflate format
Coding Trees in the Deflate format Christoph-Simon Senjak Lehr- und Forschungseinheit für Theoretische Informatik Institut für Informatik Ludwig-Maximilians-Universität München Oettingenstr.67, 80538 München
A Case Study On Practical Usability Of Dependently Typed Languages - Deflate
A Case Study On Practical Usability Of Dependently Typed Languages - Deflate Christoph-Simon Senjak Lehr- und Forschungseinheit für Theoretische Informatik Institut für Informatik Ludwig-Maximilians-Universität
Optimization of a verified Deflate implementation
Optimization of a verified Deflate implementation Christoph-Simon Senjak Lehr- und Forschungseinheit für Theoretische Informatik Institut für Informatik Ludwig-Maximilians-Universität München Oettingenstr.67,
Haskell Beats C using Generalized Stream Fusion
Haskell Beats C using Generalized Stream Fusion Christoph-Simon Senjak, Christian Neukirchen Lehr- und Forschungseinheit für Theoretische Informatik Institut für Informatik Ludwig-Maximilians-Universität
Dependent Types and Irrelevance
Dependent Types and Irrelevance Christoph-Simon Senjak Technische Universität München Institut für Informatik Boltzmannstraße 3 85748 Garching PUMA Workshop September 2012 Dependent Types Dependent Types
Huffman Coding. Version of October 13, Version of October 13, 2014 Huffman Coding 1 / 27
Huffman Coding Version of October 13, 2014 Version of October 13, 2014 Huffman Coding 1 / 27 Outline Outline Coding and Decoding The optimal source coding problem Huffman coding: A greedy algorithm Correctness
ENSC Multimedia Communications Engineering Topic 4: Huffman Coding 2
ENSC 424 - Multimedia Communications Engineering Topic 4: Huffman Coding 2 Jie Liang Engineering Science Simon Fraser University JieL@sfu.ca J. Liang: SFU ENSC 424 1 Outline Canonical Huffman code Huffman
Data Compression Techniques
Data Compression Techniques Part 1: Entropy Coding Lecture 1: Introduction and Huffman Coding Juha Kärkkäinen 31.10.2017 1 / 21 Introduction Data compression deals with encoding information in as few bits
Category: Informational May DEFLATE Compressed Data Format Specification version 1.3
Network Working Group P. Deutsch Request for Comments: 1951 Aladdin Enterprises Category: Informational May 1996 DEFLATE Compressed Data Format Specification version 1.3 Status of This Memo This memo provides
Compressing Data. Konstantin Tretyakov
Compressing Data Konstantin Tretyakov (kt@ut.ee) MTAT.03.238 Advanced April 26, 2012 Claude Elwood Shannon (1916-2001) C. E. Shannon. A mathematical theory of communication. 1948 C. E. Shannon. The mathematical
15 July, Huffman Trees. Heaps
1 Huffman Trees The Huffman Code: Huffman algorithm uses a binary tree to compress data. It is called the Huffman code, after David Huffman who discovered d it in 1952. Data compression is important in
COMPSCI 650 Applied Information Theory Feb 2, Lecture 5. Recall the example of Huffman Coding on a binary string from last class:
COMPSCI 650 Applied Information Theory Feb, 016 Lecture 5 Instructor: Arya Mazumdar Scribe: Larkin Flodin, John Lalor 1 Huffman Coding 1.1 Last Class s Example Recall the example of Huffman Coding on a
Brotli Compression Algorithm outline of a specification
Brotli Compression Algorithm outline of a specification Overview Structure of backward reference commands Encoding of commands Encoding of distances Encoding of Huffman codes Block splitting Context modeling
File Fragment Encoding Classification: An Empirical Approach
DIGITAL FORENSIC RESEARCH CONFERENCE File Fragment Encoding Classification: An Empirical Approach By Vassil Roussev and Candice Quates Presented At The Digital Forensic Research Conference DFRWS 2013 USA
Epilog: Further Topics
Ludwig-Maximilians-Universität München Institut für Informatik Lehr- und Forschungseinheit für Datenbanksysteme Knowledge Discovery in Databases SS 2016 Epilog: Further Topics Lecture: Prof. Dr. Thomas
RMNet function calls. Parameters: Usage: Micro Focus. RM/COBOL Development System - RMNET
RMNet function calls All the calls except NetGetError, NetCleanup, and NetFree return a which, when nonzero, the string Free the errorpointer with NetFree. HttpPost This function initiates an HTTP POST
MCS-375: Algorithms: Analysis and Design Handout #G2 San Skulrattanakulchai Gustavus Adolphus College Oct 21, Huffman Codes
MCS-375: Algorithms: Analysis and Design Handout #G2 San Skulrattanakulchai Gustavus Adolphus College Oct 21, 2016 Huffman Codes CLRS: Ch 16.3 Ziv-Lempel is the most popular compression algorithm today.
Information Theory and Communication
Information Theory and Communication Shannon-Fano-Elias Code and Arithmetic Codes Ritwik Banerjee rbanerjee@cs.stonybrook.edu c Ritwik Banerjee Information Theory and Communication 1/12 Roadmap Examples
A Framework for Supporting the Workflow for Archaeo-related Sciences: Managing, Synchronizing and Analyzing Data
Ludwig-Maximilians-Universität München Institut für Informatik Lehrstuhl für Datenbanksysteme und Data Mining Presentation: Daniel Kaltenthaler A Framework for Supporting the Workflow for Archaeo-related
Digital Communication Prof. Bikash Kumar Dey Department of Electrical Engineering Indian Institute of Technology, Bombay
Digital Communication Prof. Bikash Kumar Dey Department of Electrical Engineering Indian Institute of Technology, Bombay Lecture - 26 Source Coding (Part 1) Hello everyone, we will start a new module today
Data compression with Huffman and LZW
Data compression with Huffman and LZW André R. Brodtkorb, Andre.Brodtkorb@sintef.no Outline Data storage and compression Huffman: how it works and where it's used LZW: how it works and where it's used
Knowledge Discovery in Databases II Summer Term 2017
Ludwig Maximilians Universität München Institut für Informatik Lehr und Forschungseinheit für Datenbanksysteme Knowledge Discovery in Databases II Summer Term 2017 Lecture 3: Sequence Data Lectures : Prof.
Horn Formulae. CS124 Course Notes 8 Spring 2018
CS124 Course Notes 8 Spring 2018 In today s lecture we will be looking a bit more closely at the Greedy approach to designing algorithms. As we will see, sometimes it works, and sometimes even when it
Efficient Enumeration Algorithms for Constraint Satisfaction Problems
Efficient Enumeration Algorithms for Constraint Satisfaction Problems Henning and Ilka Schnoor Institut für Theoretische Informatik Leibniz Universität Hannover 2.10.2006 Efficient Enumeration Algorithms
Lecture Notes on CASE-Tools: Together
Lecture Notes on CASE-Tools: Together Software Engeneering Christoph Vilsmeier Technische Universität München Institut für Informatik 2 (based on slides from Günter Teubner) Friday, 10 th Nov. 2000 Christoph
Information Theory and Coding Prof. S. N. Merchant Department of Electrical Engineering Indian Institute of Technology, Bombay
Information Theory and Coding Prof. S. N. Merchant Department of Electrical Engineering Indian Institute of Technology, Bombay Lecture - 11 Coding Strategies and Introduction to Huffman Coding The Fundamental
ENSC Multimedia Communications Engineering Huffman Coding (1)
ENSC 424 - Multimedia Communications Engineering Huffman Coding () Jie Liang Engineering Science Simon Fraser University JieL@sfu.ca J. Liang: SFU ENSC 424 Outline Entropy Coding Prefix code Kraft-McMillan
precise rules that govern communication between two parties TCP/IP: the basic Internet protocols IP: Internet protocol (bottom level)
Protocols precise rules that govern communication between two parties TCP/IP: the basic Internet protocols IP: Internet protocol (bottom level) all packets shipped from network to network as IP packets
Chapter 9. Greedy Technique. Copyright 2007 Pearson Addison-Wesley. All rights reserved.
Chapter 9 Greedy Technique Copyright 2007 Pearson Addison-Wesley. All rights reserved. Greedy Technique Constructs a solution to an optimization problem piece by piece through a sequence of choices that
Data Compression - Seminar 4
Data Compression - Seminar 4 October 29, 2013 Problem 1 (Uniquely decodable and instantaneous codes) Let L = p i l 100 i be the expected value of the 100th power of the word lengths associated with an
Vorlesung Advanced Topics in HCI (Mensch-Maschine-Interaktion 2)
Vorlesung Advanced Topics in HCI (Mensch-Maschine-Interaktion 2) Ludwig-Maximilians-Universität München LFE Medieninformatik Heinrich Hußmann & Albrecht Schmidt WS2003/2004 http://www.medien.informatik.uni-muenchen.de/
Header Compression for TLV-based Packets
Header Compression for TLV-based Packets Marc Mosko Nov 5, 2015 Copyright Palo Alto Research Center, 2015 1 Header Compression in TLV World Compress all the signaling Fixed Header T and L fields V fields
Abusing JSONP with Rosetta Flash
Abusing JSONP with Rosetta Flash Michele Spagnuolo http://miki.it @mikispag CVE-2014-4671 The attack scenario 1 The attacker controls the first bytes of the output of a JSONP API endpoint by specifying
Authenticated Encryption in TLS
Authenticated Encryption in TLS Same modelling & verification approach concrete security: each lossy step documented by a game and a reduction (or an assumption) on paper Standardized complications - multiple
GEARMOTORS AF MOTORS FOR INVERTER C-1
C-1 C-2 C-3 C-4 R R C-5 C-6 C-7 1.47 14.7 584Hz 438Hz 318Hz 270Hz 234Hz 206Hz 167Hz 140Hz 121Hz 100Hz 81.4Hz 68.6Hz 59.4Hz 49.2Hz 40.2Hz 33.7Hz 28.9Hz 24.5Hz 21.2Hz 17.9Hz 15.2Hz 12.8Hz 3.11 4.15 5.70
Knowledge Discovery in Databases
Ludwig-Maximilians-Universität München Institut für Informatik Lehr- und Forschungseinheit für Datenbanksysteme Lecture notes Knowledge Discovery in Databases Summer Semester 2012 Lecture 8: Clustering
Hyper Text Transfer Protocol Compression
Hyper Text Transfer Protocol Compression Dr.Khalaf Khatatneh, Professor Dr. Ahmed Al-Jaber, and Asma a M. Khtoom Abstract This paper investigates HTTP post request compression approach. The most common
Compression Device Drivers. Release
Compression Device Drivers Release 18.08.0 August 09, 2018 CONTENTS 1 Compression Device Supported Functionality Matrices 1 1.1 Supported Feature Flags............................... 1 2 ISA-L Compression
Compiler Construction I
TECHNISCHE UNIVERSITÄT MÜNCHEN FAKULTÄT FÜR INFORMATIK Compiler Construction I Dr. Michael Petter, Dr. Axel Simon SoSe 2014 1 / 59 Organizing Master or Bachelor in the 6th Semester with 5 ECTS Prerequisites
Avro Specification
Table of contents 1 Introduction...2 2 Schema Declaration... 2 2.1 Primitive Types... 2 2.2 Complex Types...2 2.3 Names... 5 3 Data Serialization...6 3.1 Encodings... 6 3.2 Binary Encoding...6 3.3 JSON
Category: Informational December 1998
Network Working Group R. Pereira Request for Comments: 2394 TimeStep Corporation Category: Informational December 1998 Status of this Memo IP Payload Compression Using DEFLATE This memo provides information
Huffman Codes (data compression)
Huffman Codes (data compression) Data compression is an important technique for saving storage Given a file, We can consider it as a string of characters We want to find a compressed file The compressed
HTTP/2: What You Need to Know. Robert
HTTP/2: What You Need to Know Robert Boedigheimer @boedie About Me Web developer since 1995 Pluralsight Author 3 rd Degree Black Belt, Tae Kwon Do ASP.NET MVP boedie@outlook.com @boedie weblogs.asp.net/boedie
Variable-length Splittable Codes with Multiple Delimiters
Variable-length Splittable Codes with Multiple Delimiters Anatoly V. Anisimov, Member, IEEE and Igor O. Zavadskyi 1 arxiv:1508.01360v1 [cs.it] 6 Aug 2015 Abstract Variable-length splittable codes are derived
From Types to Sets in Isabelle/HOL
From Types to Sets in Isabelle/HOL Extented Abstract Ondřej Kunčar 1 and Andrei Popescu 1,2 1 Fakultät für Informatik, Technische Universität München, Germany 2 Institute of Mathematics Simion Stoilow
Requirements Engineering
Practical Course: Web Development Requirements Engineering Winter Semester 2016/17 Juliane Franze Ludwig-Maximilians-Universität München Practical Course Web Development WS 16/17-01 - 1 Today s Agenda
Parallel DEFLATE Decoding using GPGPU COMP Single Semester Project
Parallel DEFLATE Decoding using GPGPU COMP4560 - Single Semester Project Matthew Whittaker May 29, 2015 1 Contents 1 Why parallel DEFLATE decoding? 3 2 The project s goal 3 3 Background topics 3 3.1 Huffman
Algorithmen II. Peter Sanders, Christian Schulz, Simon Gog. Übungen: Michael Axtmann. Institut für Theoretische Informatik, Algorithmik II.
Schulz, Gog, Sanders: Algorithmen II - 31. Januar 2017 Algorithmen II Peter Sanders, Christian Schulz, Simon Gog Übungen: Michael Axtmann Institut für Theoretische Informatik, Algorithmik II Web: http://algo2.iti.kit.edu/algorithmenii_ws16.php
Configuring Traffic Policies
CHAPTER 11 Date: 4/23/09 Cisco Application Networking Manager helps you configure class maps and policy maps to provide a global level of classification for filtering traffic received by or passing through
A Dynamic Labeling Scheme using Vectors
A Dynamic Labeling Scheme using Vectors Liang Xu, Zhifeng Bao, Tok Wang Ling School of Computing, National University of Singapore {xuliang, baozhife, lingtw}@comp.nus.edu.sg Abstract. The labeling problem
Master Course Computer Networks IN2097
Chair for Network Architectures and Services Prof. Carle Department for Computer Science TU München Chair for Network Architectures and Services Prof. Carle Department for Computer Science TU München Master
Compact Encoding of the Web Graph Exploiting Various Power Laws
Compact Encoding of the Web Graph Exploiting Various Power Laws Statistical Reason Behind Link Database Yasuhito Asano, Tsuyoshi Ito 2, Hiroshi Imai 2, Masashi Toyoda 3, and Masaru Kitsuregawa 3 Department
16.Greedy algorithms
16.Greedy algorithms 16.1 An activity-selection problem Suppose we have a set S = {a 1, a 2,..., a n } of n proposed activities that with to use a resource. Each activity a i has a start time s i and a
Range Searching II: Windowing Queries
Computational Geometry Lecture : Windowing Queries INSTITUT FÜR THEORETISCHE INFORMATIK FAKULTÄT FÜR INFORMATIK Tamara Mchedlidze Chih-Hung Liu 23.11.2015 1 Object types in range queries y0 y x x0 Setting
Compression Device Drivers. Release rc3
Compression Device Drivers Release 19.02.0-rc3 January 20, 2019 CONTENTS 1 Compression Device Supported Functionality Matrices 1 1.1 Supported Feature Flags............................... 1 2 ISA-L Compression
Analysis of the effects of removing redundant header information in persistent HTTP connections
Analysis of the effects of removing redundant header information in persistent HTTP connections Timothy Bower, Daniel Andresen, David Bacon Department of Computing and Information Sciences 234 Nichols
Parameterized Complexity of Independence and Domination on Geometric Graphs
Parameterized Complexity of Independence and Domination on Geometric Graphs Dániel Marx Institut für Informatik, Humboldt-Universität zu Berlin, Unter den Linden 6, 10099 Berlin, Germany. dmarx@informatik.hu-berlin.de
6. Finding Efficient Compressions; Huffman and Hu-Tucker
6. Finding Efficient Compressions; Huffman and Hu-Tucker We now address the question: how do we find a code that uses the frequency information about k length patterns efficiently to shorten our message?
4.8 Huffman Codes. These lecture slides are supplied by Mathijs de Weerd
4.8 Huffman Codes These lecture slides are supplied by Mathijs de Weerd Data Compression Q. Given a text that uses 32 symbols (26 different letters, space, and some punctuation characters), how can we
Chapter 5: Outlier Detection
Ludwig-Maximilians-Universität München Institut für Informatik Lehr- und Forschungseinheit für Datenbanksysteme Knowledge Discovery in Databases SS 2016 Chapter 5: Outlier Detection Lecture: Prof. Dr.
Avro Specification
Table of contents 1 Introduction...2 2 Schema Declaration... 2 2.1 Primitive Types... 2 2.2 Complex Types...2 2.3 Names... 5 2.4 Aliases... 6 3 Data Serialization...6 3.1 Encodings... 7 3.2 Binary Encoding...7
SIGNAL COMPRESSION Lecture Lempel-Ziv Coding
SIGNAL COMPRESSION Lecture 5 11.9.2007 Lempel-Ziv Coding Dictionary methods Ziv-Lempel 77 The gzip variant of Ziv-Lempel 77 Ziv-Lempel 78 The LZW variant of Ziv-Lempel 78 Asymptotic optimality of Ziv-Lempel
ECE 7650 Scalable and Secure Internet Services and Architecture ---- A Systems Perspective
ECE 7650 Scalable and Secure Internet Services and Architecture ---- A Systems Perspective Part II: Data Center Software Architecture: Topic 3: Programming Models RCFile: A Fast and Space-efficient Data
Master Course Computer Networks IN2097
Chair for Network Architectures and Services Prof. Carle Department for Computer Science TU München Chair for Network Architectures and Services Prof. Carle Department for Computer Science TU München Master
A New Compression Method Strictly for English Textual Data
A New Compression Method Strictly for English Textual Data Sabina Priyadarshini Department of Computer Science and Engineering Birla Institute of Technology Abstract - Data compression is a requirement
Master Course Computer Networks IN2097
Chair for Network Architectures and Services Prof. Carle Department for Computer Science TU München Master Course Computer Networks IN2097 Prof. Dr.-Ing. Georg Carle Christian Grothoff, Ph.D. Chair for
BD separates ABC into two parts ( 1 and 2 ),then the measure
M 1312 section 3.5 1 Inequalities in a Triangle Definition: Let a and b be real numbers a > b if and only if there is a positive number p for which a = b + p Example 1: 7 > 2 and 5 is a positive number
Lecture 7b: HTTP. Feb. 24, Internet and Intranet Protocols and Applications
Internet and Intranet Protocols and Applications Lecture 7b: HTTP Feb. 24, 2004 Arthur Goldberg Computer Science Department New York University artg@cs.nyu.edu WWW - HTTP/1.1 Web s application layer protocol
zlib Update Jim Johnston, TPF Development Lab March 23, 2015 TPFUG Dallas, TX
zlib Update Jim Johnston, TPF Development Lab March 23, 215 TPFUG Dallas, TX Presentation Summary I. zlib General Information II. III. IV. z/tpf Specific zlib Information zlib Performance zlib Compression
1 Problem Description
Tree Isomorphism: An Exercise in Functional Programming Jayadev Misra 9/10/01 1 Problem Description The problem is to decide if two unordered trees are the same. More precisely, define a binary relation
Digital Communication Prof. Bikash Kumar Dey Department of Electrical Engineering Indian Institute of Technology, Bombay
Digital Communication Prof. Bikash Kumar Dey Department of Electrical Engineering Indian Institute of Technology, Bombay Lecture - 29 Source Coding (Part-4) We have already had 3 classes on source coding
The pitfalls of protocol design
2014 IEEE Security and Privacy Workshops The pitfalls of protocol design Attempting to write a formally verified PDF parser Andreas Bogk Principal Security Architect HERE Berlin, Germany andreas.bogk@here.com
CSE 417: Algorithms and Computational Complexity
CSE : Algorithms and Computational Complexity More Graph Algorithms Autumn 00 Paul Beame Given: a directed acyclic graph (DAG) G=(V,E) Output: numbering of the vertices of G with distinct numbers from
Semantic Web and Natural Language Processing
Semantic Web and Natural Language Processing Wiltrud Kessler Institut für Maschinelle Sprachverarbeitung Universität Stuttgart Semantic Web Winter 2014/2015 This work is licensed under a Creative Commons
Data Compression for Bitmap Indexes. Y. Chen
Data Compression for Bitmap Indexes Y. Chen Abstract Compression Ratio (CR) and Logical Operation Time (LOT) are two major measures of the efficiency of bitmap indexing. Previous works by [5, 9, 10, 11]
Kapitel 4: Clustering
Ludwig-Maximilians-Universität München Institut für Informatik Lehr- und Forschungseinheit für Datenbanksysteme Knowledge Discovery in Databases WiSe 2017/18 Kapitel 4: Clustering Vorlesung: Prof. Dr.
About Texts and Links. Quick tour of basic design guidelines (1) Vorlesung Advanced Topics in HCI (Mensch-Maschine-Interaktion 2)
Vorlesung Advanced Topics in HCI (Mensch-Maschine-Interaktion 2) Ludwig-Maximilians-Universität München LFE Medieninformatik Albrecht Schmidt & Andreas Butz SS2005 http://www.medien.ifi.lmu.de/ Quick tour
Refinement Using µ-charts: The Compaq Grand Slam Cup Case Study Revisited
Refinement Using µ-charts: The Compaq Grand Slam Cup Case udy Revisited Hubert Baumeister Institut für Informatik Universität München Oettingenstr. 67 80538 München, Germany Christoph Maier FAST e.v. Arabellastr.
Binary Encoded Attribute-Pairing Technique for Database Compression
Binary Encoded Attribute-Pairing Technique for Database Compression Akanksha Baid and Swetha Krishnan Computer Sciences Department University of Wisconsin, Madison baid,swetha@cs.wisc.edu Abstract Data
Huffman Coding Implementation on Gzip Deflate Algorithm and its Effect on Website Performance
Huffman Coding Implementation on Gzip Deflate Algorithm and its Effect on Website Performance I Putu Gede Wirasuta - 13517015 Program Studi Teknik Informatika Sekolah Teknik Elektro dan Informatika Institut
Chapter 4: Application Protocols 4.1: Layer : Internet Phonebook : DNS 4.3: The WWW and s
Chapter 4: Application Protocols 4.1: Layer 5-7 4.2: Internet Phonebook : DNS 4.3: The WWW and E-Mails OSI Reference Model Application Layer Presentation Layer Session Layer Application Protocols Chapter
Data Structure and Algorithm Midterm Reference Solution TA
Data Structure and Algorithm Midterm Reference Solution TA email: dsa1@csie.ntu.edu.tw Problem 1. To prove log 2 n! = Θ(n log n), it suffices to show N N, c 1, c 2 > 0 such that c 1 n ln n ln n! c 2 n
Lempel-Ziv compression: how and why?
Lempel-Ziv compression: how and why? Algorithms on Strings Paweł Gawrychowski July 9, 2013 s July 9, 2013 2/18 Outline Lempel-Ziv compression Computing the factorization Using the factorization July 9,
This paper proposes: Mining Frequent Patterns without Candidate Generation
Mining Frequent Patterns without Candidate Generation a paper by Jiawei Han, Jian Pei and Yiwen Yin School of Computing Science Simon Fraser University Presented by Maria Cutumisu Department of Computing
Chapter 2: The Game Core. (part 2)
Ludwig Maximilians Universität München Institut für Informatik Lehr- und Forschungseinheit für Datenbanksysteme Lecture Notes for Managing and Mining Multiplayer Online Games for the Summer Semester 2017
BELOW, we consider decoding algorithms for Reed Muller
4880 IEEE TRANSACTIONS ON INFORMATION THEORY, VOL. 52, NO. 11, NOVEMBER 2006 Error Exponents for Recursive Decoding of Reed Muller Codes on a Binary-Symmetric Channel Marat Burnashev and Ilya Dumer, Senior
Abusing JSONP with. Michele - CVE , CVE Pwnie Awards 2014 Nominated
Abusing JSONP with Michele Spagnuolo @mikispag - https://miki.it Pwnie Awards 2014 Nominated CVE-2014-4671, CVE-2014-5333 Rosetta Flash FWSÏx DADË
Knowledge Discovery in Databases
Ludwig-Maximilians-Universität München Institut für Informatik Lehr- und Forschungseinheit für Datenbanksysteme Lecture notes Knowledge Discovery in Databases Summer Semester 2012 Lecture 3: Frequent Itemsets
THE RELATIVE EFFICIENCY OF DATA COMPRESSION BY LZW AND LZSS
THE RELATIVE EFFICIENCY OF DATA COMPRESSION BY LZW AND LZSS Yair Wiseman 1* * 1 Computer Science Department, Bar-Ilan University, Ramat-Gan 52900, Israel Email: wiseman@cs.huji.ac.il, http://www.cs.biu.ac.il/~wiseman
Network Working Group. Category: Informational August 1996
Network Working Group J. Woods Request for Comments: 1979 Proteon, Inc. Category: Informational August 1996 Status of This Memo PPP Deflate Protocol This memo provides information for the Internet community.
Vorlesung Kommunikationsnetze
Picture 15 13 Vorlesung Kommunikationsnetze Prof. Dr. H. P. Großmann mit B. Wiegel sowie A. Schmeiser und M. Rabel Sommersemester 2009 Institut für Organisation und Management von Informationssystemen
Lecture: Analysis of Algorithms (CS )
Lecture: Analysis of Algorithms (CS483-001) Amarda Shehu Spring 2017 1 The Fractional Knapsack Problem Huffman Coding 2 Sample Problems to Illustrate The Fractional Knapsack Problem Variable-length (Huffman)
Text Compression through Huffman Coding. Terminology
Text Compression through Huffman Coding Huffman codes represent a very effective technique for compressing data; they usually produce savings between 20% 90% Preliminary example We are given a 100,000-character
Guided Research: Intelligent Contextual Task Support for Mails
Guided Research: Intelligent Contextual Task Support for Mails Simon Bönisch, 28.05.2018, Kick-off Presentation Chair of Software Engineering for Business Information Systems (sebis) Faculty of Informatics
HTTP Inspection Engine
HTTP Inspection Engine Last Updated: October 16, 2011 The HTTP Inspection Engine feature allows users to configure their Cisco IOS Firewall to detect and prohibit HTTP connections--such as tunneling over
Master Course Computer Networks IN2097
Chair for Network Architectures and Services Prof. Carle Department for Computer Science TU München Master Course Computer Networks IN2097 Prof. Dr.-Ing. Georg Carle Christian Grothoff, Ph.D. Chair for
Uniquely Decodable. Code 1 Code 2 A 00 0 B 1 1 C D 11 11
Uniquely Detectable Code Uniquely Decodable A code is not uniquely decodable if two symbols have the same codeword, i.e., if C(S i ) = C(S j ) for any i j or the combination of two codewords gives a third
Volume 2, Issue 9, September 2014 ISSN
Fingerprint Verification of the Digital Images by Using the Discrete Cosine Transformation, Run length Encoding, Fourier transformation and Correlation. Palvee Sharma 1, Dr. Rajeev Mahajan 2 1M.Tech Student
DRAFT. Computer Science 1001.py. Instructor: Benny Chor. Teaching Assistant (and Python Guru): Rani Hod
Computer Science 1001.py Lecture 21 : Introduction to Text Compression Instructor: Benny Chor Teaching Assistant (and Python Guru): Rani Hod School of Computer Science Tel-Aviv University Fall Semester,
6.338 Final Paper: Parallel Huffman Encoding and Move to Front Encoding in Julia
6.338 Final Paper: Parallel Huffman Encoding and Move to Front Encoding in Julia Gil Goldshlager December 2015 1 Introduction 1.1 Background The Burrows-Wheeler transform (BWT) is a string transform used