Implementation of Robust Compression Technique using LZ77 Algorithm on Tensilica s Xtensa Processor
|
|
- Duane Edwards
- 5 years ago
- Views:
Transcription
1 2016 International Conference on Information Technology Implementation of Robust Compression Technique using LZ77 Algorithm on Tensilica s Xtensa Processor Vasanthi D R and Anusha R M.Tech (VLSI Design and Embedded Systems) CMR Institute of Technology Bangalore -37, Karnataka, India vasanthidr11@gmail.com & anusha005r@gmail.com Abstract SSD Technology is used to read/write data to the external device with high speed and low power. A lossless LZ77 data compression technique is used here in order to retain the information content by removing data redundancy and also employs a text window in congestion with look-ahead buffer to serve as a dictionary. In the proposed technique the unmatched string present in the dictionary are encoded as a length and offset. Encoding and decoding process in data compression enables SSD s, so that the data written is less in turn yields the endurance of SSD performance. Eventually using novel method, pointers are effectively selected and encoded for the longest matched string to obtain greater compression. For the unmatched character compression ratio is 3.12times which is better than existing algorithm. Data compression improves the performance of SSD, hence this reduces the amount of data transfer to flash memory. The speed of secure data compression increases with a creation of preferred search function for a HEX sample data size 32Kbyte. Even though if extra time is acquired, asymmetric compression decompresses the data quickly resulting in good compression. Keywords SSD; LZ77; Compression Ratio; Tensilica s Xtensa Processor; Xtensa SIMD Processor; FLIX Instruction; Vinay B K Asst. Professor, ECE Dept. CMR Institute of Technology Bangalore -37, Karnataka, India vinay.9cool@gmail.com LZ77 type decoders expand the compressed file by expanding "copy items" in the compressed file into larger strings enabling "duplicate string elimination" and only the first copy of a string is stored in the compressed file. Many decoder algorithms combine LZ77 with other compression algorithm and they extract the "length" and the "distance" from the huge variety of decoder structure [3]. Hence the Compression can reduce the size of data in order to save the space and transmission time. Due to this it increases the effective storage capacity of devices and an added advantage is that data compression reduces write path wherein instead of writing complete blocks only a few bytes of data are actually modified. II. LEMPEL ZIV (LZ77) Data compression technique was first described by Lempel Ziv in Encoder examines input sequence by sliding of last n bytes of data that has been processed, and for the subsequent bytes it searches for the longest match. I. INTRODUCTION HDD s read access time is lesser than SSD in terms of milliseconds which is an advantage for SSD. Rather than using disk and read/write heads SSD uses flash memory, hence information is retained even at times of power off. Due to all this reason current technology uses SSD s more in place of HDD s. SSD s can be made smaller, uses less power and doesnot make noise, hence its reliable and last longer on a single battery charge than HDD s. In SSD, LZ77 data compression is a key feature having high data throughput rates and less power consumption. Compression eliminates the redundancy of data in a reversible manner and also increases entropy by reducing the size of date [1]. Lempel & Ziv algorithm is categorized into lossless and lossy compression. In lossless compression, files are compressed and decompressed by producing the same file out, whereas in lossy compression data cannot reproduce the exact file instead it returns to approximate file [2].The dictionaries used in compression technique does not match with the programming structure that holds the data table, and also it does not have external dictionary causing problem while decompressing another data. Therefore the compression algorithm tends to implement the programming object libraries files, so that the algorithm can easily be reusable and updated. Fig. 1: Encoding using the LZ77 approach. A sliding window is categorized into search buffer and look-ahead buffer as shown in fig. 1. In search buffer a portion of recently encoded sequence is formed whereas in look-ahead buffer next portion of sequence is formed. Practically size of buffer is five times larger than look-ahead buffer. Distance of pointer from look-ahead buffer is called offset [4] and number of consecutive symbols in search buffer which matches the consecutive symbol in look-ahead buffer is called length of the match. Encoder searches the search buffer for longest match, once the longest match is found encoder encodes it with triple (o,l,c). If the size of buffer is S,size of window is W and size of source alphabet A, then number of bits to be encoded in triple using fixed length code is given in equation (1). (1) /16 $ IEEE DOI /ICIT
2 For example, if the minimum match length is of 3 characters and even if there is no current match, a sequence of 3 new bytes is typically the minimum searched buffer. If the match is not less than or equal to 3 character is found, then the 1 st byte of the string is output of literal byte. Then the window s start and end are adjusted by one, and the next input byte is appended to the end of 2 remaining bytes, hence a new search commences. If a 3 character match is found, new bytes are one-at-atime is added to the end of a match string and searches are made, to determine the longer string that has atleast a similar match character in the window. determined, a compression technique like lossless comp data includes LZ77, LZ78, LZW, LZSS etc is applied to the data [6]. Here LZ77 lossless compression is used where data can be compressed in better n simpler approach. Then the compressed data file is accessed by the user and user checks if the compressed data is correct. If data is correct then user ends the program and the compressed file is deleted, if the data is not correct then the sliding window slides to the next position n search continues until the match is found. To retrieve the original data obtained compressed file is decompressed. LZ77 has three different possibilities for its coding process. a) The next character to be encoded in the window has no match in the sequence [5]. b) If the character present in search buffer is same as the character in look-ahead buffer, then the length of the match is found. c) The matched string must be extended inside the look-ahead buffer. Compression ratio can be defined as ratio between size of compressed file and size of source file. (In simple it is ratio of size after compression to size before compression). If compression ratio is more, than speed increases thereby data sending and retrieving process is faster between host and the flash memory. Hence compression ratio can be increased by making kernel to denote pointers of match substring and its offset of longest match string to be encoded effectively. III. METHODOLOGY Compression is an encoding process that takes less space e.g. to reduce load on memory, disk, I/O etc. Lossless decoder reproduces message exactly whereas lossy decoder reproduces message approximately. Fig. 3: Flow chart of LZ77 compression. Fig. 2: Compression and decompression model. The data sent to the compressor block is compressed for a different models and the compressed data is again an input to the decompressor block, wherein an original data is retrieved back as shown in fig. 2. A. Flowchart of LZ77 The file size is determined for the data or text taken for compression and decompression. Once the file size is Compression can be done using two methods, that is fixed length codes and variable length codes. 1) Fixed Length Codes A fixed length code is a code in which fixed number of source data is encoded into a fixed number of output data. It can be carried out using three methodologies. a) Short bytes: Storage unit in short bytes is of 5 bits. If alphabet is <=32 symbols, then 5 bits per symbols are used. 149
3 If alphabet is >32 symbols and <= 60 then either of the case can be used: Use 1-30 for most frequent symbols ( Base case ). Use 1-30 for least frequent symbols ( shift case ). Use 0 and 31 to shift back and forth. (e.g. type writer). This type of compression works well when shifts do not occur often. One of the optimization methods that can be used in this algorithm is either, temporary shift and shift lock or multiple cases. b) Bigrams / Digrams: Storage unit in bigrams is 8bits which contains from 0 to 255. From the first of 1 to 87 bits characters like blank, uppercase, lowercase, digits and 25 special characters are used. Rest of the bits from 88 to 255 is used for bigrams, which is the combination of master plus combining. Master consists of 8bits they are blank, A, E, I, O, N, T, U and combination consists of 21 bits they are blank, plus everything but J, K, Q, X, Y, Z. Therefore total codes consist of: 88+8*21=88+168=256. By this bigrams are simple, fast and requires less memory, maximum compression is of 50%. c) n-grams: Storage unit in n-grams is 8bits. This is similar to that of bigrams, but extended to cover sequences of 2 or more characters. The goal of n-gram is to encode each length of unit that is greater than one which occurs with high probability. This is common for two and three symbol words and can capture longer phrases and names. All the above three methods are simple and very effective when their assumptions are correct. 2) Variable Length Codes Variable length code is a code which maps the source data to a variable number of bits. Variable length codes can allow source to be compressed and decompressed with zero error by technique of lossless data compression and thereby data can be read back. Continue the process until the tree contains all nodes and symbols. Frequent symbols are usually nearer to the root, hence it is a short code and less frequent symbols are deeper giving them longer codes. b) Lempel Ziv: Lempel Ziv is based on the adaptive dictionary approach to variable length coding. In order to build the dictionary, encounter the text that has been already used [10]. A good dictionary can be built if the text follows zipf s file. Some of the variant used in the compression are LZ77, Gzip, LZ78, LZW and UNIX. These variants differ from, The way how dictionary is built. Representation of pointers and Pointers limitation on which it s referred. IV. PROPOSED METHOD The previously existed method was a set of sequence is considered comprising a sliding window that contains both search and look ahead buffer. Search pointer points or slides from backwards, either of the two criteria is obtained that is match found and match not found. Fig.4 shows the encoding and decoding process. So if match is not found in search buffer which is present in look-ahead buffer then a bit is shifted to one position and the sequence will be considered until the length of the match is found. Thereby a triple is obtained containing offset, length of the match and code word for both encoding and decoding process. a) Huffman codes: Huffman coding is a method for compressing data with variable length code [8]. This method has a set of variable length code words with the shortest average length that is assigned to the data by using the frequencies of occurrences. Process includes gathering of probabilities for symbols like character, words or a mix. Later build a tree as follows Find at least two frequency symbols or nodes and join it with parent node. Label the least probable as 0 and other branch as 1. Fig. 4: LZ77 compression and decompression. In the proposed method, a set of data is considered in a sliding window where the size of search buffer is larger than the look-ahead buffer as shown in fig
4 b) The implementation of LZ77 is more efficient on Xtensa using Tensilica instructions. c) The encoder must keep track of some amount of data such as the last 2KB, 4KB, 32KB. d) The performance of LZ77 in SSD increases, thereby the speed of read/write cycle can be improved up to 8byte per cycles. Fig. 5: LZ77 Proposed Method Block Diagram. Now the search pointer position is at the left of the search buffer and the search pointer starts its search until it finds a character which is present at the start of look-ahead buffer, if a match is found, then the encoding process is stopped, thereby length of the match is found. Copy the characters to the end, so that left out end character will be represented as a code word. Therefore a triple (o,l,c) is obtained, where o is offset, l is length of the match, c is code word that is encoded and the same triple can be decoded. Here the offset, length of the match and code word should be matched or same for both encoding and decoding process. If the length of the string in search buffer is larger, then the data is stored in memory, so that the search buffer works faster thereby search speed increases. Text or data is taken from host interface and is transmitted to the write path during compression. Therefore stored data is made to uncompressed so as to retrieve back the original data which is sent to the host interface. Finally the data is verified, whether the data sent and retrieved back is same. The total throughput of a compression size will be of a single stream, where it is capable for implementing in giga bits per second. Latency of compression ratio increases from the start to the end of compression and decompression and it is based on Huffman code. Compression ratio reduces the size of original file that is, if the final result is 1/3 the size of the starting compression file. V. CHALLENGES OF PROPOSED LZ77 IN SSD During the implementation of SSD the challenges faced by proposed LZ77 data compression is, a) Data compression benefits in reducing write amplification, i.e, even though only a few bytes of data are changed yet it needs the whole blocks to be written. Fig. 6: Chart Showing Compression ratio for the Proposed LZ77 Family in SSD. In the proposed LZ77 compression technique, the compression ratio achieved is 3.12 times better than any other lossless compression technique as shown in fig. 6. The optimization of LZ77 data compression has been improved by 550% on Xtensa processor. The output of compression ratio in LZ77 which is of literal bytes, occupies 9bits and a matched string upto 258bytes occupies 24bits. Worst case result for lz77 is 1/8=12% growth and a best case is of 24/258*8=98.8% which helps in reduction of size. VI. IMPLEMENTATION RESULTS The results of proposed LZ77 were evaluated on Tensilica s Xtensa Processor for each block (compression and decompression) and obtained different results and performance. Combination of proposed LZ77 yields a better performance and compression ratio for its read path and write path. The total compression ratio obtained for unmatched character is 3.14 times, which was better than any other existing algorithms and compression ratio for matched character is double than that of unmatched character. The implementation of SSD in lz77 will improve the performance of read/write speed, thereby capacity and reliability also increases. The total code size obtained for proposed LZ77 in profile cycles is bytes. The cycles obtained for proposed LZ77 are, for 12 cycles it is 1 execute with 5 unconditional instruction fetch along with 6 unconditional load and for 22 cycles it is 1 execute with 16 unconditional instruction fetch along with 5 unconditional load. Therefore the data was securely compressed which was robust to many of the compression technique. 151
5 The Xtensa C compiler additionally supports Tensilica s FLIX (Flexible Length Instruction Extension) instruction to confess from 4-byte to 16-byte VLIW (very long instruction word) instruction. The level of performance and efficiency increases 10 to 100x for lower energy consumption. Multi-processor subsystem shows each cascade for easy load assessment and re-partitioning guidance. The call graph view enables to view the entire application s caller hierarchy and callee function. High performance code from c source can offer compiler to operate Xtensa SIMD processor and FLIX instruction to obtain hybrid sample and cycle accurate results. Fig. 7: Final Build and Results. The Robust LZ77 data compression code debug was implemented on Tensilica s Xtensa Processor is based on the GNU compiler which is highly customized for targeting the compact of 16/24-bit Xtensa ISA as shown in fig. 8. Fig. 9: Multi-core profiling (Hybrid sampled). In multi-core profiling, fig. 10 shows the cycle accurate results, which was obtained by user-defined write and read path size for a different sizes like 4kb, 8kb, 16kb, 32kb, so that host interface can write the data through write path during compression and the data can be read through read path during uncompression. Hybrid sampled results are obtained without user-defined sizes as shown in fig. 9. Profiling information is used to optimize application code to further reduce branch delays and improves in-lining. Fig. 8: LZ77 Output Fig. 10: Multi-core profiling (Cycle accurate). Fig. 8: LZ77 output. 152
6 [2] Adrian Traian, Murgan and Radii Radescu. "A Comparison of Algorithms for Lossless Data Compression Using the Lempel-Ziv- Welch Type Methods". IEEE, pp , [3] Senthil Shanmugasundaram, Robert Lourdusamy A Comparative Study Of Text Compression Algorithms International Journal of Wisdom Based Computing, Vol. 1 (3), December [4] Khalid Sayood, Introduction to Data Compression, 2nd Edition, San Francisco, CA, Morgan Kaufmann, [5] C. Fraser. An instruction for direct interpretation of LZ77-compressed programs. Technical report MSR-TR , 9/2002. Fig. 11: Pipeline view of instruction stalls and latency issues (Cycle Accurate). VII. CONCLUSION LZ77 algorithm explores mechanism for compressing data over SSDs. The Robust LZ77 data compression technique is simple and effective approach to compress the data and uses redundant nature of data to provide compression ratio. A 12% increase in LZ77 code improves the performance for 8bytes per cycle at a time. Due to the wide memory interface available at the local memories processing can be extended for 16, 32 or 64 bytes at a time. The approximate average speed of LZ77 in solid state drive controller (SSD) read/write cycles is 180Mb/s. Compression ratio wil be more, when speed increases thereby data sending and retrieving process is faster between host and the flash memory in SSD s. References [1] David Salomon. Data Compression: The Complete Reference, 4th Edition. (With contributions by Giovanni Motta and David Bryant). Published by Springer, Dec [6] Sungjin Lee, Jihoon Park, Kermin Fleming, Arvind Improving Performance and Lifetime of Solid-State Drives Using Hardware- Accelerated Compression, IEEE Transactions on consumer electronics year:2011,volume :57,Issue:4, pp: [7] Hu Yuanfu; Wu Xunsen Signal processing, The methods of improving the compression algorithms, 3 rd International conference on year :1996,Volume 1Pages : Vol.1,DOI: /ICSIGP [8] Huffman D.A., A method for the construction of minimumredundancy codes, Proceedings of the Institute of Radio Engineers, 40 (9), pp , September [9] Simple Hashing LZ77 sliding Dictionary Compression Program, PROG1.C, by Rich Geldreih, Jr October, [10] Gipfeli High Speed Compreeion Algorithm Rastislav Lenhardt, and Jyrki Alakuijala, University of Oxford, united Kingdom. 153
Comparative Study of Dictionary based Compression Algorithms on Text Data
88 Comparative Study of Dictionary based Compression Algorithms on Text Data Amit Jain Kamaljit I. Lakhtaria Sir Padampat Singhania University, Udaipur (Raj.) 323601 India Abstract: With increasing amount
More informationSo, what is data compression, and why do we need it?
In the last decade we have been witnessing a revolution in the way we communicate 2 The major contributors in this revolution are: Internet; The explosive development of mobile communications; and The
More informationA Comparative Study Of Text Compression Algorithms
International Journal of Wisdom Based Computing, Vol. 1 (3), December 2011 68 A Comparative Study Of Text Compression Algorithms Senthil Shanmugasundaram Department of Computer Science, Vidyasagar College
More informationEntropy Coding. - to shorten the average code length by assigning shorter codes to more probable symbols => Morse-, Huffman-, Arithmetic Code
Entropy Coding } different probabilities for the appearing of single symbols are used - to shorten the average code length by assigning shorter codes to more probable symbols => Morse-, Huffman-, Arithmetic
More informationAbdullah-Al Mamun. CSE 5095 Yufeng Wu Spring 2013
Abdullah-Al Mamun CSE 5095 Yufeng Wu Spring 2013 Introduction Data compression is the art of reducing the number of bits needed to store or transmit data Compression is closely related to decompression
More informationA Compression Technique Based On Optimality Of LZW Code (OLZW)
2012 Third International Conference on Computer and Communication Technology A Compression Technique Based On Optimality Of LZW (OLZW) Utpal Nandi Dept. of Comp. Sc. & Engg. Academy Of Technology Hooghly-712121,West
More informationFPGA based Data Compression using Dictionary based LZW Algorithm
FPGA based Data Compression using Dictionary based LZW Algorithm Samish Kamble PG Student, E & TC Department, D.Y. Patil College of Engineering, Kolhapur, India Prof. S B Patil Asso.Professor, E & TC Department,
More informationA Research Paper on Lossless Data Compression Techniques
IJIRST International Journal for Innovative Research in Science & Technology Volume 4 Issue 1 June 2017 ISSN (online): 2349-6010 A Research Paper on Lossless Data Compression Techniques Prof. Dipti Mathpal
More informationEE-575 INFORMATION THEORY - SEM 092
EE-575 INFORMATION THEORY - SEM 092 Project Report on Lempel Ziv compression technique. Department of Electrical Engineering Prepared By: Mohammed Akber Ali Student ID # g200806120. ------------------------------------------------------------------------------------------------------------------------------------------
More informationS 1. Evaluation of Fast-LZ Compressors for Compacting High-Bandwidth but Redundant Streams from FPGA Data Sources
Evaluation of Fast-LZ Compressors for Compacting High-Bandwidth but Redundant Streams from FPGA Data Sources Author: Supervisor: Luhao Liu Dr. -Ing. Thomas B. Preußer Dr. -Ing. Steffen Köhler 09.10.2014
More informationHARDWARE IMPLEMENTATION OF LOSSLESS LZMA DATA COMPRESSION ALGORITHM
HARDWARE IMPLEMENTATION OF LOSSLESS LZMA DATA COMPRESSION ALGORITHM Parekar P. M. 1, Thakare S. S. 2 1,2 Department of Electronics and Telecommunication Engineering, Amravati University Government College
More informationKeywords Data compression, Lossless data compression technique, Huffman Coding, Arithmetic coding etc.
Volume 6, Issue 2, February 2016 ISSN: 2277 128X International Journal of Advanced Research in Computer Science and Software Engineering Research Paper Available online at: www.ijarcsse.com A Comparative
More informationAn Advanced Text Encryption & Compression System Based on ASCII Values & Arithmetic Encoding to Improve Data Security
Available Online at www.ijcsmc.com International Journal of Computer Science and Mobile Computing A Monthly Journal of Computer Science and Information Technology IJCSMC, Vol. 3, Issue. 10, October 2014,
More informationITCT Lecture 8.2: Dictionary Codes and Lempel-Ziv Coding
ITCT Lecture 8.2: Dictionary Codes and Lempel-Ziv Coding Huffman codes require us to have a fairly reasonable idea of how source symbol probabilities are distributed. There are a number of applications
More informationSimple variant of coding with a variable number of symbols and fixlength codewords.
Dictionary coding Simple variant of coding with a variable number of symbols and fixlength codewords. Create a dictionary containing 2 b different symbol sequences and code them with codewords of length
More informationStudy of LZ77 and LZ78 Data Compression Techniques
Study of LZ77 and LZ78 Data Compression Techniques Suman M. Choudhary, Anjali S. Patel, Sonal J. Parmar Abstract Data Compression is defined as the science and art of the representation of information
More informationDesign and Implementation of FPGA- based Systolic Array for LZ Data Compression
Design and Implementation of FPGA- based Systolic Array for LZ Data Compression Mohamed A. Abd El ghany Electronics Dept. German University in Cairo Cairo, Egypt E-mail: mohamed.abdel-ghany@guc.edu.eg
More informationBasic Compression Library
Basic Compression Library Manual API version 1.2 July 22, 2006 c 2003-2006 Marcus Geelnard Summary This document describes the algorithms used in the Basic Compression Library, and how to use the library
More informationDEFLATE COMPRESSION ALGORITHM
DEFLATE COMPRESSION ALGORITHM Savan Oswal 1, Anjali Singh 2, Kirthi Kumari 3 B.E Student, Department of Information Technology, KJ'S Trinity College Of Engineering and Research, Pune, India 1,2.3 Abstract
More informationCS 493: Algorithms for Massive Data Sets Dictionary-based compression February 14, 2002 Scribe: Tony Wirth LZ77
CS 493: Algorithms for Massive Data Sets February 14, 2002 Dictionary-based compression Scribe: Tony Wirth This lecture will explore two adaptive dictionary compression schemes: LZ77 and LZ78. We use the
More informationMultimedia Networking ECE 599
Multimedia Networking ECE 599 Prof. Thinh Nguyen School of Electrical Engineering and Computer Science Based on B. Lee s lecture notes. 1 Outline Compression basics Entropy and information theory basics
More informationNoise Reduction in Data Communication Using Compression Technique
Digital Technologies, 2016, Vol. 2, No. 1, 9-13 Available online at http://pubs.sciepub.com/dt/2/1/2 Science and Education Publishing DOI:10.12691/dt-2-1-2 Noise Reduction in Data Communication Using Compression
More informationA New Compression Method Strictly for English Textual Data
A New Compression Method Strictly for English Textual Data Sabina Priyadarshini Department of Computer Science and Engineering Birla Institute of Technology Abstract - Data compression is a requirement
More informationA Comprehensive Review of Data Compression Techniques
Volume-6, Issue-2, March-April 2016 International Journal of Engineering and Management Research Page Number: 684-688 A Comprehensive Review of Data Compression Techniques Palwinder Singh 1, Amarbir Singh
More informationLossless Compression Algorithms
Multimedia Data Compression Part I Chapter 7 Lossless Compression Algorithms 1 Chapter 7 Lossless Compression Algorithms 1. Introduction 2. Basics of Information Theory 3. Lossless Compression Algorithms
More informationA Novel Image Compression Technique using Simple Arithmetic Addition
Proc. of Int. Conf. on Recent Trends in Information, Telecommunication and Computing, ITC A Novel Image Compression Technique using Simple Arithmetic Addition Nadeem Akhtar, Gufran Siddiqui and Salman
More informationParallelizing Inline Data Reduction Operations for Primary Storage Systems
Parallelizing Inline Data Reduction Operations for Primary Storage Systems Jeonghyeon Ma ( ) and Chanik Park Department of Computer Science and Engineering, POSTECH, Pohang, South Korea {doitnow0415,cipark}@postech.ac.kr
More informationSIGNAL COMPRESSION Lecture Lempel-Ziv Coding
SIGNAL COMPRESSION Lecture 5 11.9.2007 Lempel-Ziv Coding Dictionary methods Ziv-Lempel 77 The gzip variant of Ziv-Lempel 77 Ziv-Lempel 78 The LZW variant of Ziv-Lempel 78 Asymptotic optimality of Ziv-Lempel
More informationDictionary Based Compression for Images
Dictionary Based Compression for Images Bruno Carpentieri Abstract Lempel-Ziv methods were original introduced to compress one-dimensional data (text, object codes, etc.) but recently they have been successfully
More informationINTERNATIONAL JOURNAL OF PURE AND APPLIED RESEARCH IN ENGINEERING AND TECHNOLOGY
Rashmi Gadbail,, 2013; Volume 1(8): 783-791 INTERNATIONAL JOURNAL OF PURE AND APPLIED RESEARCH IN ENGINEERING AND TECHNOLOGY A PATH FOR HORIZING YOUR INNOVATIVE WORK EFFECTIVE XML DATABASE COMPRESSION
More informationTHE RELATIVE EFFICIENCY OF DATA COMPRESSION BY LZW AND LZSS
THE RELATIVE EFFICIENCY OF DATA COMPRESSION BY LZW AND LZSS Yair Wiseman 1* * 1 Computer Science Department, Bar-Ilan University, Ramat-Gan 52900, Israel Email: wiseman@cs.huji.ac.il, http://www.cs.biu.ac.il/~wiseman
More informationInformation Technology Department, PCCOE-Pimpri Chinchwad, College of Engineering, Pune, Maharashtra, India 2
Volume 5, Issue 5, May 2015 ISSN: 2277 128X International Journal of Advanced Research in Computer Science and Software Engineering Research Paper Available online at: www.ijarcsse.com Adaptive Huffman
More informationError Resilient LZ 77 Data Compression
Error Resilient LZ 77 Data Compression Stefano Lonardi Wojciech Szpankowski Mark Daniel Ward Presentation by Peter Macko Motivation Lempel-Ziv 77 lacks any form of error correction Introducing a single
More informationEngineering Mathematics II Lecture 16 Compression
010.141 Engineering Mathematics II Lecture 16 Compression Bob McKay School of Computer Science and Engineering College of Engineering Seoul National University 1 Lossless Compression Outline Huffman &
More informationCategory: Informational May DEFLATE Compressed Data Format Specification version 1.3
Network Working Group P. Deutsch Request for Comments: 1951 Aladdin Enterprises Category: Informational May 1996 DEFLATE Compressed Data Format Specification version 1.3 Status of This Memo This memo provides
More informationTEXT COMPRESSION ALGORITHMS - A COMPARATIVE STUDY
S SENTHIL AND L ROBERT: TEXT COMPRESSION ALGORITHMS A COMPARATIVE STUDY DOI: 10.21917/ijct.2011.0062 TEXT COMPRESSION ALGORITHMS - A COMPARATIVE STUDY S. Senthil 1 and L. Robert 2 1 Department of Computer
More informationDepartment of electronics and telecommunication, J.D.I.E.T.Yavatmal, India 2
IJESRT INTERNATIONAL JOURNAL OF ENGINEERING SCIENCES & RESEARCH TECHNOLOGY LOSSLESS METHOD OF IMAGE COMPRESSION USING HUFFMAN CODING TECHNIQUES Trupti S Bobade *, Anushri S. sastikar 1 Department of electronics
More informationAn Asymmetric, Semi-adaptive Text Compression Algorithm
An Asymmetric, Semi-adaptive Text Compression Algorithm Harry Plantinga Department of Computer Science University of Pittsburgh Pittsburgh, PA 15260 planting@cs.pitt.edu Abstract A new heuristic for text
More informationWIRE/WIRELESS SENSOR NETWORKS USING K-RLE ALGORITHM FOR A LOW POWER DATA COMPRESSION
WIRE/WIRELESS SENSOR NETWORKS USING K-RLE ALGORITHM FOR A LOW POWER DATA COMPRESSION V.KRISHNAN1, MR. R.TRINADH 2 1 M. Tech Student, 2 M. Tech., Assistant Professor, Dept. Of E.C.E, SIR C.R. Reddy college
More informationModeling Delta Encoding of Compressed Files
Modeling Delta Encoding of Compressed Files EXTENDED ABSTRACT S.T. Klein, T.C. Serebro, and D. Shapira 1 Dept of CS Bar Ilan University Ramat Gan, Israel tomi@cs.biu.ac.il 2 Dept of CS Bar Ilan University
More informationIMAGE COMPRESSION TECHNIQUES
IMAGE COMPRESSION TECHNIQUES A.VASANTHAKUMARI, M.Sc., M.Phil., ASSISTANT PROFESSOR OF COMPUTER SCIENCE, JOSEPH ARTS AND SCIENCE COLLEGE, TIRUNAVALUR, VILLUPURAM (DT), TAMIL NADU, INDIA ABSTRACT A picture
More informationA High-Performance FPGA-Based Implementation of the LZSS Compression Algorithm
2012 IEEE 2012 26th IEEE International 26th International Parallel Parallel and Distributed and Distributed Processing Processing Symposium Symposium Workshops Workshops & PhD Forum A High-Performance
More informationEncoding. A thesis submitted to the Graduate School of University of Cincinnati in
Lossless Data Compression for Security Purposes Using Huffman Encoding A thesis submitted to the Graduate School of University of Cincinnati in a partial fulfillment of requirements for the degree of Master
More informationData Compression Techniques
Data Compression Techniques Part 2: Text Compression Lecture 6: Dictionary Compression Juha Kärkkäinen 15.11.2017 1 / 17 Dictionary Compression The compression techniques we have seen so far replace individual
More informationDesign and Implementation of Lossless Data Compression Coprocessor using FPGA
Design and Implementation of Lossless Data Compression Coprocessor using FPGA Udaya Kumar H PG Student(VLSI Design and Embedded Systems) SIET, Tumkur Karnataka, India Madhu B C Assistant Prof., Dept. of
More informationData Compression. Media Signal Processing, Presentation 2. Presented By: Jahanzeb Farooq Michael Osadebey
Data Compression Media Signal Processing, Presentation 2 Presented By: Jahanzeb Farooq Michael Osadebey What is Data Compression? Definition -Reducing the amount of data required to represent a source
More informationAnalysis of Parallelization Effects on Textual Data Compression
Analysis of Parallelization Effects on Textual Data GORAN MARTINOVIC, CASLAV LIVADA, DRAGO ZAGAR Faculty of Electrical Engineering Josip Juraj Strossmayer University of Osijek Kneza Trpimira 2b, 31000
More informationMODELING DELTA ENCODING OF COMPRESSED FILES. and. and
International Journal of Foundations of Computer Science c World Scientific Publishing Company MODELING DELTA ENCODING OF COMPRESSED FILES SHMUEL T. KLEIN Department of Computer Science, Bar-Ilan University
More informationJournal of Computer Engineering and Technology (IJCET), ISSN (Print), International Journal of Computer Engineering
Journal of Computer Engineering and Technology (IJCET), ISSN 0976 6367(Print), International Journal of Computer Engineering and Technology (IJCET), ISSN 0976 6367(Print) ISSN 0976 6375(Online) Volume
More informationA Comparative Study of Entropy Encoding Techniques for Lossless Text Data Compression
A Comparative Study of Entropy Encoding Techniques for Lossless Text Data Compression P. RATNA TEJASWI 1 P. DEEPTHI 2 V.PALLAVI 3 D. GOLDIE VAL DIVYA 4 Abstract: Data compression is the art of reducing
More informationAn On-line Variable Length Binary. Institute for Systems Research and. Institute for Advanced Computer Studies. University of Maryland
An On-line Variable Length inary Encoding Tinku Acharya Joseph F. Ja Ja Institute for Systems Research and Institute for Advanced Computer Studies University of Maryland College Park, MD 242 facharya,
More informationCh. 2: Compression Basics Multimedia Systems
Ch. 2: Compression Basics Multimedia Systems Prof. Ben Lee School of Electrical Engineering and Computer Science Oregon State University Outline Why compression? Classification Entropy and Information
More informationImage Compression for Mobile Devices using Prediction and Direct Coding Approach
Image Compression for Mobile Devices using Prediction and Direct Coding Approach Joshua Rajah Devadason M.E. scholar, CIT Coimbatore, India Mr. T. Ramraj Assistant Professor, CIT Coimbatore, India Abstract
More informationLOSSLESS DATA COMPRESSION AND DECOMPRESSION ALGORITHM AND ITS HARDWARE ARCHITECTURE
LOSSLESS DATA COMPRESSION AND DECOMPRESSION ALGORITHM AND ITS HARDWARE ARCHITECTURE V V V SAGAR 1 1JTO MPLS NOC BSNL BANGALORE ---------------------------------------------------------------------***----------------------------------------------------------------------
More informationA Method for Virtual Extension of LZW Compression Dictionary
A Method for Virtual Extension of Compression Dictionary István Finta, Lóránt Farkas, Sándor Szénási and Szabolcs Sergyán Technology and Innovation, Nokia Networks, Köztelek utca 6, Budapest, Hungary Email:
More informationEE67I Multimedia Communication Systems Lecture 4
EE67I Multimedia Communication Systems Lecture 4 Lossless Compression Basics of Information Theory Compression is either lossless, in which no information is lost, or lossy in which information is lost.
More informationCode Compression for RISC Processors with Variable Length Instruction Encoding
Code Compression for RISC Processors with Variable Length Instruction Encoding S. S. Gupta, D. Das, S.K. Panda, R. Kumar and P. P. Chakrabarty Department of Computer Science & Engineering Indian Institute
More informationGipfeli - High Speed Compression Algorithm
Gipfeli - High Speed Compression Algorithm Rastislav Lenhardt I, II and Jyrki Alakuijala II I University of Oxford United Kingdom rastislav.lenhardt@cs.ox.ac.uk II Google Switzerland GmbH jyrki@google.com
More informationData Compression. An overview of Compression. Multimedia Systems and Applications. Binary Image Compression. Binary Image Compression
An overview of Compression Multimedia Systems and Applications Data Compression Compression becomes necessary in multimedia because it requires large amounts of storage space and bandwidth Types of Compression
More informationSTUDY OF VARIOUS DATA COMPRESSION TOOLS
STUDY OF VARIOUS DATA COMPRESSION TOOLS Divya Singh [1], Vimal Bibhu [2], Abhishek Anand [3], Kamalesh Maity [4],Bhaskar Joshi [5] Senior Lecturer, Department of Computer Science and Engineering, AMITY
More informationFundamentals of Multimedia. Lecture 5 Lossless Data Compression Variable Length Coding
Fundamentals of Multimedia Lecture 5 Lossless Data Compression Variable Length Coding Mahmoud El-Gayyar elgayyar@ci.suez.edu.eg Mahmoud El-Gayyar / Fundamentals of Multimedia 1 Data Compression Compression
More informationHighly Secure Invertible Data Embedding Scheme Using Histogram Shifting Method
www.ijecs.in International Journal Of Engineering And Computer Science ISSN:2319-7242 Volume 3 Issue 8 August, 2014 Page No. 7932-7937 Highly Secure Invertible Data Embedding Scheme Using Histogram Shifting
More informationComparison of Text Data Compression Using Run Length Encoding, Arithmetic Encoding, Punctured Elias Code and Goldbach Code
Comparison of Text Data Compression Using Run Length Encoding, Arithmetic Encoding, Punctured Elias Code and Goldbach Code Kenang Eko Prasetyo 1, Tito Waluyo Purboyo 2, Randy Erfa Saputra 3 Computer Engineering,
More informationThe Effect of Non-Greedy Parsing in Ziv-Lempel Compression Methods
The Effect of Non-Greedy Parsing in Ziv-Lempel Compression Methods R. Nigel Horspool Dept. of Computer Science, University of Victoria P. O. Box 3055, Victoria, B.C., Canada V8W 3P6 E-mail address: nigelh@csr.uvic.ca
More informationCompression and Decompression of Virtual Disk Using Deduplication
Compression and Decompression of Virtual Disk Using Deduplication Bharati Ainapure 1, Siddhant Agarwal 2, Rukmi Patel 3, Ankita Shingvi 4, Abhishek Somani 5 1 Professor, Department of Computer Engineering,
More informationVolume 2, Issue 9, September 2014 ISSN
Fingerprint Verification of the Digital Images by Using the Discrete Cosine Transformation, Run length Encoding, Fourier transformation and Correlation. Palvee Sharma 1, Dr. Rajeev Mahajan 2 1M.Tech Student
More informationOptimized Compression and Decompression Software
2015 IJSRSET Volume 1 Issue 3 Print ISSN : 2395-1990 Online ISSN : 2394-4099 Themed Section: Engineering and Technology Optimized Compression and Decompression Software Mohd Shafaat Hussain, Manoj Yadav
More informationROOT I/O compression algorithms. Oksana Shadura, Brian Bockelman University of Nebraska-Lincoln
ROOT I/O compression algorithms Oksana Shadura, Brian Bockelman University of Nebraska-Lincoln Introduction Compression Algorithms 2 Compression algorithms Los Reduces size by permanently eliminating certain
More informationOPTIMIZATION OF LZW (LEMPEL-ZIV-WELCH) ALGORITHM TO REDUCE TIME COMPLEXITY FOR DICTIONARY CREATION IN ENCODING AND DECODING
Asian Journal Of Computer Science And Information Technology 2: 5 (2012) 114 118. Contents lists available at www.innovativejournal.in Asian Journal of Computer Science and Information Technology Journal
More informationInternational Journal of Advanced Research in Computer Science and Software Engineering
ISSN: 2277 128X International Journal of Advanced Research in Computer Science and Software Engineering Research Paper Available online at: Enhanced LZW (Lempel-Ziv-Welch) Algorithm by Binary Search with
More informationMedical Image Compression using DCT and DWT Techniques
Medical Image Compression using DCT and DWT Techniques Gullanar M. Hadi College of Engineering-Software Engineering Dept. Salahaddin University-Erbil, Iraq gullanarm@yahoo.com ABSTRACT In this paper we
More informationCIS 121 Data Structures and Algorithms with Java Spring 2018
CIS 121 Data Structures and Algorithms with Java Spring 2018 Homework 6 Compression Due: Monday, March 12, 11:59pm online 2 Required Problems (45 points), Qualitative Questions (10 points), and Style and
More informationLempel-Ziv-Welch (LZW) Compression Algorithm
Lempel-Ziv-Welch (LZW) Compression lgorithm Introduction to the LZW lgorithm Example 1: Encoding using LZW Example 2: Decoding using LZW LZW: Concluding Notes Introduction to LZW s mentioned earlier, static
More informationData compression with Huffman and LZW
Data compression with Huffman and LZW André R. Brodtkorb, Andre.Brodtkorb@sintef.no Outline Data storage and compression Huffman: how it works and where it's used LZW: how it works and where it's used
More informationAn Effective Approach to Improve Storage Efficiency Using Variable bit Representation
Volume 114 No. 12 2017, 145-154 ISSN: 1311-8080 (printed version); ISSN: 1314-3395 (on-line version) url: http://www.ijpam.eu ijpam.eu An Effective Approach to Improve Storage Efficiency Using Variable
More informationA New Technique of Lossless Image Compression using PPM-Tree
A New Technique of Lossless Image Compression PP-Tree Shams ahmood Imam, S.. Rezaul Hoque, ohammad Kabir Hossain, William Perrizo Department of Computer Science and Engineering, North South University,
More informationLossless compression II
Lossless II D 44 R 52 B 81 C 84 D 86 R 82 A 85 A 87 A 83 R 88 A 8A B 89 A 8B Symbol Probability Range a 0.2 [0.0, 0.2) e 0.3 [0.2, 0.5) i 0.1 [0.5, 0.6) o 0.2 [0.6, 0.8) u 0.1 [0.8, 0.9)! 0.1 [0.9, 1.0)
More informationDesign of a Pipelined 32 Bit MIPS Processor with Floating Point Unit
Design of a Pipelined 32 Bit MIPS Processor with Floating Point Unit P Ajith Kumar 1, M Vijaya Lakshmi 2 P.G. Student, Department of Electronics and Communication Engineering, St.Martin s Engineering College,
More informationECE 499/599 Data Compression & Information Theory. Thinh Nguyen Oregon State University
ECE 499/599 Data Compression & Information Theory Thinh Nguyen Oregon State University Adminstrivia Office Hours TTh: 2-3 PM Kelley Engineering Center 3115 Class homepage http://www.eecs.orst.edu/~thinhq/teaching/ece499/spring06/spring06.html
More informationV.2 Index Compression
V.2 Index Compression Heap s law (empirically observed and postulated): Size of the vocabulary (distinct terms) in a corpus E[ distinct terms in corpus] n with total number of term occurrences n, and constants,
More informationCh. 2: Compression Basics Multimedia Systems
Ch. 2: Compression Basics Multimedia Systems Prof. Thinh Nguyen (Based on Prof. Ben Lee s Slides) Oregon State University School of Electrical Engineering and Computer Science Outline Why compression?
More informationIndexing. CS6200: Information Retrieval. Index Construction. Slides by: Jesse Anderton
Indexing Index Construction CS6200: Information Retrieval Slides by: Jesse Anderton Motivation: Scale Corpus Terms Docs Entries A term incidence matrix with V terms and D documents has O(V x D) entries.
More informationGzip Compression Using Altera OpenCL. Mohamed Abdelfattah (University of Toronto) Andrei Hagiescu Deshanand Singh
Gzip Compression Using Altera OpenCL Mohamed Abdelfattah (University of Toronto) Andrei Hagiescu Deshanand Singh Gzip Widely-used lossless compression program Gzip = LZ77 + Huffman Big data needs fast
More informationImage Compression Algorithm and JPEG Standard
International Journal of Scientific and Research Publications, Volume 7, Issue 12, December 2017 150 Image Compression Algorithm and JPEG Standard Suman Kunwar sumn2u@gmail.com Summary. The interest in
More informationImproving the Performance of Spatial Reusability Aware Routing in Multi-Hop Wireless Networks
IOSR Journal of Engineering (IOSRJEN) ISSN (e): 2250-3021, ISSN (p): 2278-8719 Vol. 08, Issue 6 (June. 2018), V (III) PP 48-53 www.iosrjen.org Improving the Performance of Spatial Reusability Aware Routing
More informationData Compression Techniques for Big Data
Data Compression Techniques for Big Data 1 Ms.Poonam Bonde, 2 Mr. Sachin Barahate 1 P.G Student, 2 Assistent Professor in I.T. Department 1 Student of YTGOIFOE, Mumbai, India 2 Padmabhushan Vasantdada
More informationComparative data compression techniques and multi-compression results
IOP Conference Series: Materials Science and Engineering OPEN ACCESS Comparative data compression techniques and multi-compression results To cite this article: M R Hasan et al 2013 IOP Conf. Ser.: Mater.
More informationOn Data Latency and Compression
On Data Latency and Compression Joseph M. Steim, Edelvays N. Spassov, Kinemetrics, Inc. Abstract Because of interest in the capability of digital seismic data systems to provide low-latency data for Early
More information15 Data Compression 2014/9/21. Objectives After studying this chapter, the student should be able to: 15-1 LOSSLESS COMPRESSION
15 Data Compression Data compression implies sending or storing a smaller number of bits. Although many methods are used for this purpose, in general these methods can be divided into two broad categories:
More informationIMAGE COMPRESSION. Image Compression. Why? Reducing transportation times Reducing file size. A two way event - compression and decompression
IMAGE COMPRESSION Image Compression Why? Reducing transportation times Reducing file size A two way event - compression and decompression 1 Compression categories Compression = Image coding Still-image
More informationNetwork Working Group Request for Comments: December 1998
Network Working Group Request for Comments: 2395 Category: Informational R. Friend R. Monsour Hi/fn, Inc. December 1998 IP Payload Compression Using LZS Status of this Memo This memo provides information
More informationData Compression Scheme of Dynamic Huffman Code for Different Languages
2011 International Conference on Information and Network Technology IPCSIT vol.4 (2011) (2011) IACSIT Press, Singapore Data Compression Scheme of Dynamic Huffman Code for Different Languages Shivani Pathak
More informationCompression. storage medium/ communications network. For the purpose of this lecture, we observe the following constraints:
CS231 Algorithms Handout # 31 Prof. Lyn Turbak November 20, 2001 Wellesley College Compression The Big Picture We want to be able to store and retrieve data, as well as communicate it with others. In general,
More informationA SIMPLE DATA COMPRESSION ALGORITHM FOR ANOMALY DETECTION IN WIRELESS SENSOR NETWORKS
Volume 117 No. 19 2017, 403-410 ISSN: 1311-8080 (printed version); ISSN: 1314-3395 (on-line version) url: http://www.ijpam.eu ijpam.eu A SIMPLE DATA COMPRESSION ALGORITHM FOR ANOMALY DETECTION IN WIRELESS
More informationOverview. Last Lecture. This Lecture. Next Lecture. Data Transmission. Data Compression Source: Lecture notes
Overview Last Lecture Data Transmission This Lecture Data Compression Source: Lecture notes Next Lecture Data Integrity 1 Source : Sections 10.1, 10.3 Lecture 4 Data Compression 1 Data Compression Decreases
More informationText Data Compression and Decompression Using Modified Deflate Algorithm
Text Data Compression and Decompression Using Modified Deflate Algorithm R. Karthik, V. Ramesh, M. Siva B.E. Department of Computer Science and Engineering, SBM COLLEGE OF ENGINEERING AND TECHNOLOGY, Dindigul-624005.
More informationFILE SYSTEMS. CS124 Operating Systems Winter , Lecture 23
FILE SYSTEMS CS124 Operating Systems Winter 2015-2016, Lecture 23 2 Persistent Storage All programs require some form of persistent storage that lasts beyond the lifetime of an individual process Most
More informationLZW Compression. Ramana Kumar Kundella. Indiana State University December 13, 2014
LZW Compression Ramana Kumar Kundella Indiana State University rkundella@sycamores.indstate.edu December 13, 2014 Abstract LZW is one of the well-known lossless compression methods. Since it has several
More informationDictionary techniques
Dictionary techniques The final concept that we will mention in this chapter is about dictionary techniques. Many modern compression algorithms rely on the modified versions of various dictionary techniques.
More informationA NOVEL APPROACH FOR A HIGH PERFORMANCE LOSSLESS CACHE COMPRESSION ALGORITHM
A NOVEL APPROACH FOR A HIGH PERFORMANCE LOSSLESS CACHE COMPRESSION ALGORITHM K. Janaki 1, K. Indhumathi 2, P. Vijayakumar 3 and K. Ashok Kumar 4 1 Department of Electronics and Communication Engineering,
More information