Arithmetic Coding. Arithmetic Coding

Transcription

1 Contents Image Compression Lecture 3 Arithmetic Code Introduction to & Decoding Algorithm Generating a Binary Code for Huffman codes have to be an integral number of bits long, while the entropy value of a symbol is almost always a fraction number, theoretical possible compressed message cannot be achieved. For example, if a statistical method assign 90% probability to a given character, the optimal code size would be 0.5 bits. Arithmetic coding bypasses the idea of replacing an input symbol with a specific code. It replaces a stream of input symbols with a single floating-point output number. Arithmetic coding is especially useful when dealing with sources with small alphabets, such as binary sources, and alphabets with highly skewed probabilities. 3 4 Arithmetic code Alphabet extension (blocking symbols) can lead to coding efficiency How about treating entire sequence as one symbol! Not practical with Huffman coding Arithmetic coding allows you to do precisely this Basic idea - map data sequences to sub-intervals in [0,) with lengths equal to probability of corresponding sequence. ) Huffman coder: H R H + bit/pel 2) Block coder: H n R n H n + /n bit/pel 3) Arithmetic code: H R H + bit/message (!) Arithmetic code: Algorithm () 0) Start by defining the current interval as [0,). ) REPEAT for each symbol s in the input stream a) Divide the current interval [L, H) into subintervals whose sizes are proportional p to the symbols's probabilities. b) Select the subinterval [L, H) for the symbol s and define it as the new current interval 2) When the entire input stream has been processed, the output should be any number V that uniquely identify the current interval [L, H).

2 Arithmetic code: Algorithm (2) 0.70 Arithmetic code:algorithm (3) Probabilities: p, p 2,, p N. Cumulants: C =0; C 2 =C +p =p ; C 3 =C 2 +p 2 =p +p 2 ; etc. C N =p +p 2 + +p N- ; C N+ =; 0) Current interval [L, H) = [0.0,.0): ) REPEAT for each symbol s i in the input stream: H L + (H L)*C(s i+ ), L L + (H L)*C(s i ); 2) UNTIL the entire input stream has been processed. The output code V is any number that uniquely identify the current interval [L, H). Example : Statistics Message: 'SWISS_MISS' Char Freq Prob [C(s i ), C(s i+ )) S 5 5/0=0.5 [0.5,.0) W /0=0. [0.4, 0.5) I 2 2/0=0.2 [0.2, 0.4) M /0=0. [0., 0.2) _ /0=0. [0.0, 0.) Example : Encoding S 0.5 [0.5,.0) W 0. [0.4, 0.5) I 0.2 [0.2, 0.4) M 0. [0., 0.2) 0. [0.0, 0.) Example () The final value, named a tag, will uniquely encode the message SWISS_MISS. Any value between and can be a tag for the encoded message, and can be uniquely decoded. Encoding algorithm for arithmetic coding. Low=00;high=0; 0.0 =.0 ; while not EOF do range = high - low ; read(c) ; high = low + range high_range(c) ; low = low + range low_range(c) ; enddo output(low); 2

3 Decoding is the inverse process. Since falls between 0.5 and.0, the first character must be S. Removing the effect of S from by first subtracting the low value of S, 0.5, giving Then divided by the width of the range of S, 0.5. This gives a value of Then calculate where that lands, which is in the range of the next letter, W. The process repeats until 0 or the known length of the message is reached. 3 4 Example : Decoding V [ , ) S 0.5 [0.5,.0) W 0. [0.4, 0.5) I [0.2, [02 0.4) 04) M 0. [0., 0.2) 0. [0.0, 0.) Decoding algorithm r = input_code repeat search c such that r falls in its range ; output(c) ; r = r - low_range(c) ; r = r/(high_range(c) - low_range(c)); until r equal 0 6 Example : Compression? V [ , ) How many bits do we need to encode a number V in the final interval [L, H)? m=4 bits: 6=2 4 intervals of size =/6. The number of bits m to represent a value in the interval of size : m= -Log 2 ( ) bits. Example : Compression () V [L, H) = [ , ) n Interval size (range) r: r = p i i= r=0.5*0.*0.2*0.5*0.5*0.*0.*0.2*0.5*0.5= The number of bits to represent a value in the interval [L, H)=[L, L+r) of size r: log2 n = r = log2 p i = Entropy i= m= -log 2 r = -log 2 ( ) = 9.6 =20 bits m

4 Example : Compression (2) Entropy =.96 bits/char Arithmetic coder: a) Codeword V: L V < H V = ( ) 0 = ( ) bits < < b) Codelength m: m= -log 2 (r) = -log 2 ( ) = 9.6 =20 bits c) Bitrate: R=20 bits/0 chars = 2.0 bits/char Huffman coder: ( )/0=2.2 bits/char Example (2) Symbol probability Range [0.00, ) [0.80, 0.82) [0.82,.00) Suppose that we want to encode the message Example (2) Example (2) Encoding: New character Low value High value T x ( 32) = = Example (2) Decoding: r c low high range ( )/0.8= ( ) / 0.8= ( )/0.02= In summary, the encoding process is simply one of narrowing the range of possible numbers with every new symbol. The new range is proportional to the predefined probability attached to that symbol. Decoding is the inverse procedure, in which the range is expanded in proportion to the probability of each symbol as it is extracted

5 Conclusions JPEG, MPEG-/2 uses Huffman and arithmetic coding preprocessed by DPCM JPEG-LS JPEG2000, MPEG-4 uses arithmetic coding only 2009/7/23 資料壓縮 Unit 4 Arithmetic Coding 25