IMPROVED JPEG DECOMPRESSION OF DOCUMENT IMAGES BASED ON IMAGE SEGMENTATION. Tak-Shing Wong, Charles A. Bouman, and Ilya Pollak

Transcription

1 IMPROVED DECOMPRESSION OF DOCUMENT IMAGES BASED ON IMAGE SEGMENTATION Tak-Shing Wong, Charle A. Bouman, and Ilya Pollak School of Electrical and Computer Engineering Purdue Univerity ABSTRACT We propoe a decompreion algorithm for document baed on egmentation. Our egmentation algorithm claifie each block of the into one of three clae: text, background, or picture. We develop a different decoding trategy for each cla and apply thi trategy to every block in thi cla. Our experiment demontrate that thi approach can improve the quality of decompreion. 1. INTRODUCTION The algorithm i till one of the mot prevalent compreion algorithm today. Further, a large number of have been compreed with the algorithm and archived in variou document and databae. Thee conideration motivate reearch on improving the quality of decompreion. We how that, for the cla of document, the quality of the decompreed can be improved ignificantly by applying different decompreion trategie over region with different characteritic. We propoe partitioning a document into text, background, and picture block, and uing different decompreion algorithm for each cla. Section 2 i a review of the baeline tandard. Section 3 i an overview of our algorithm. Section 4 decribe the egmentation algorithm we propoe for claifying each block. Section 5 introduce the three decompreion trategie ued for the three different clae. 2. REVIEW OF THE BASELINE STANDARD With compreion [1], an achromatic i partitioned into 8 8 block. Each block then undergoe the forward dicrete coine tranform (FDCT), quantization, and entropy encoding (Fig. 1). Let X be a column vector of the 64 pixel of the 8 8 block. The FDCT i an orthogonal tranform given by Y = BX, where B i a matrix whoe column are pairwie orthogonal. The firt element Y,0 of Y i called the DC coefficient of the block; the other element are the AC coefficient. The next tage ue a quantizationhmatrix, to perform element-wie uniform quantization Ỹ,i = Y,i i i, where [ ] i the integer-round operator. Finally the quantized DCT coefficient are packed into the bit-tream by entropy encoding. For a color, thi operation equence i applied to each color channel independently. Uually the i firt converted to the YC b C r color pace before compreion. Baeline decompreion revere the operation of compreion. The compreed i firt entropy decoded. Since entropy encoding i a lole operation, the quantized DCT Thi work wa upported in part by a grant from the Xerox Foundation. input X compreed compreed FDCT Entropy Decoding Y Ỹ uantization De quantization Ỹ Entropy Encoding Baeline Compreion Ỹ IDCT Baeline Decompreion Fig. 1. Baeline compreion and decompreion Image block egmentation Background block decoding Text/graphic block decoding Picture block decoding Combine reult Fig. 2. Overview of the cheme compreed Decompreed X decompreed coefficient Ỹ are retrieved perfectly. De-quantization then etimate the original true DCT coefficient (Y,i) byỹ,i = iỹ,i. Finally, invere dicrete coine tranform (IDCT) produce the decoded block a X = B t Ỹ. 3. OVERVIEW OF THE PROPOSED ALGORITHM Fig. 2 how a block diagram of our cheme. The input i firt egmented uing the 8 8 block of the into three clae: (i) text block, (ii) background block, and (iii) picture block. Thee three clae repreent region with very different characteritic. Accordingly, different technique are employed for decoding block of each cla. For a color, the decoding tage i applied to each color channel independently. Text block cover the region of text and graphic which uually contain many harp edge. The pixel in text block are typically either the background or the foreground. Thee block uffer mot everely from the ringing artifact of becaue they have a high level of high frequency content. Background block include the background of the document and mooth region of natural. They have almot no high frequencie. The low variation of the intenity in thee region make them very uceptible to the blocking artifact of X/07/$ IEEE 488 SSP 2007

2 NO Feature vector of all block block Compute AC energy E E <T AC? Compute 2 D feature vector [D,1,D,2] SMAP egmentation YES Text/picture block Picture cluter Text block S txt Picture block S pic Feature vector of text/picture block K mean clutering GMM fit for picture block Background block S bkg Fig. 3. Block-baed Segmentation Text cluter GMM fit for text block The remaining picture block conit of non-mooth region of natural. They uffer from both the ringing and blocking artifact. However, a noted in [2], ignificant high-frequency content in highly textured block make the artifact le noticeable. 4. BLOCK-BASED SEGMENTATION To exploit the unique characteritic of each cla of block, a blockbaed unupervied egmentation algorithm (Fig. 3) i firt applied to the input to partition the et of block S into the three clae: S bkg for background block; S txt for text block; S pic for picture block. Firt, background block are differentiated from the other block by computing the AC energy in each block and threholding. Next, a 2-D feature vector i calculated for each block uing the luminance channel. The feature vector of text and picture block are clutered into two group by k-mean clutering [3]. To make ue of patial regularity, each group of feature vector i fit with a Gauian mixture model (GMM) whoe order i determined by the minimum decription length criterion [4]. Finally, the two model for text and picture block are ued to egment the feature vector by the SMAP egmentation algorithm [5]. The reult of the SMAP algorithm i combined with the background block detected in the AC energy threholding tage to produce the final egmentation map Background Block Detection For an achromatic, the AC energy of the block i defined a E = P 63 i=1 Ỹ,i 2 where Ỹ,i i the etimate of the i-th DCT coefficient of the block, produced by decompreion. E i compared with a mall poitive threhold T AC. IfE <T AC, the block i claified a a background block. For a color, the AC energy i calculated for each of the three color channel of the block. The maximum i defined a E and compared with T AC Feature Vector The claification of the non-background block intotext and picture block i baed on a 2-D feature vector computed from the luminance channel. It wa reported that the code length of text block after entropy encoding tend to be longer than non-text block due to the higher level of high frequency content in thee block [6],[7]. Thu the firt feature i baed on the encoding length and computed a [7]! 63 f(ỹ,0 Ỹ 1,0)+ X f(ỹ,i) D,1 = 1 (1) 64 i=1 j log2 ( x )+4 if x > 1 where f(x) = 0 otherwie The econd feature meaure how cloe a block i to being a twocolor block. For each block, we perform a two-color projection a follow with the luminance channel decoded by conventional. The luminance value from a window centered at the block are firt clutered into two group by k-mean clutering, with mean denoted by θ,1 and θ,2. The two-color projection i formed by clipping each luminance of each pixel to the mean of the cluter to which the luminance value belong. Thi two-color projection i the bet repreentation of the block with two color in the MMSE ene. The l 2 ditance between the luminance of the block and it two-color projection i then a meaure of how cloely the block reemble a two-color block. We normalize thi projection error by the quare of the difference of the two etimated mean, θ,1 θ,2 2, o that a high contrat block ha a higher chance to be claified a a text block. The econd feature i then calculated a 1 X63 D,2 = θ,1 θ X,i X,i 2 (2),2 2 where X,i i the etimate of the i-th pixel of the block, produced by decompreion, and X,i i the value of the i-th pixel of the two-color projection. If θ,1 = θ,2, wedefine D,2 = Text and Picture Block Claification In the lat tage, the feature vector of non-background block are clutered into two group by k-mean clutering. For the norm ued in k-mean clutering, the contribution of the two feature are differently weighted. Specifically, for a vector D t = [D 1,D 2], the norm i calculated by D = p D1 2 + γd2 2. In our experiment, we et γ =15. The reulting cluter with a higher mean of D,1 and a lower mean of D,2 repreent the cla of text block, and the other cluter repreent the picture block. Each cluter i fit with a Gauian mixture model. The two model are then employed within the SMAP egmentation algorithm [5] in order to claify each non-background block a either a text block or a picture block. 5. IMAGE BLOCK DECODING decoding can be poed a an invere problem. Becaue quantization i a many-to-one tranform, many poible block X can produce the ame de-quantized DCT coefficient Ỹ at the decoder. The non-uniquene of the olution of recontructing the original block X baed on the de-quantized DCT coefficient Ỹ make thi problem ill-poed. We regularize the problem by auming a prior model for the original block. We then decode each block by computing the maximum a poteriori (MAP) etimate. 489

3 Let T ( ) be the quantization tranform o that T (Y )=Ỹ. The invere B 1 T 1 (Ỹ) ={x R64 : T (Bx) =Ỹ} i the et of block which encode to Ỹ. The forward model i: ( pỹ X (Ỹ x) = 1 if x B 1 T 1 (Ỹ) (3) 0 otherwie If the original block X follow the prior ditribution p X (x), the MAP etimate of X i ˆX =argmax{p X Ỹ(x Ỹ)} x R 64 (4) = argmin x B 1 T 1 (Ỹ) { log p X (x)} The contraint x B 1 T 1 (Ỹ), however, i difficult to enforce. A the DCT i an orthogonal tranform, intead of ˆX,we will be olving for the MAP etimate of the DCT of ˆX a follow: Ŷ = argmin y T 1 (Ỹ) { log p X (B 1 y)} (5) It hould be noted that the contraint now become the impler pointwie contraint y i Ỹ,i i/2 for i = The ignificance of egmentation manifet itelf here in that it allow u to apply a different prior model for each cla of block. The prior model choen for each cla capture the unique characteritic of the cla and conequently produce a decoded block that i of higher quality Text Block Decoding We oberve that pixel in a text block typically concentrate around two value of the foreground and the background color, and chooe a prior model reflecting thi. Let θ,1 and θ,2 be thee two value of the text block, θ,1 <θ,2. The bimodal ditribution of the pixel i modeled a 63X p θ (x 1 )= z exp{ 1 min( x 2σ 2,i θ,1 2, x,i θ,2 2 )} (6) where θ =[θ,1,θ,2] are the parameter of the ditribution, z i a normalization contant, and σ 2 i related to the variance of each cluter. The MAP etimate of the pixel X in each block then depend on the unknown parameter vector θ. We ue the maximum likelihood etimate of θ intead of it true value, to yield the following MAP etimate of X : ˆX = argmin x B 1 T 1 n 1 X63 o min( x,i θ,1 2, x,i θ,2 2 ) θ 2σ 2 (Ỹ) min (7) Intuitively, we eek a et of value for an block x B 1 T 1 (Ỹ) and two color θ,1 and θ,2 o that each pixel value i a cloe a poible to one of the two color. While thi decoding cheme produce reaonably good reult, we have found that the etimation of the two color i not accurate for ome text block. Thi happen when, for example, the majority of the pixel are in the foreground, leaving very few background pixel for a good etimate of the background color. To improve the accuracy, we oberve that two neighboring text block generally have imilar foreground and background color. Alo, a background block next to a text block i uually a continuation of the background of the text. Baed on thee obervation, we apply patial regularization to the etimation of the two color of each text block. Specifically, we let ρ( ) be a monotone function, and penalize ρ( θ r,1 θ,1 )+ρ( θ r,2 θ,2 ) (8) for every pair {r, } of neighbor text block. (We ue neighborhood of ize eight.) We ue the convention that θ,1 <θ,2 for all block, o that our penalty enforce imilarity between the lighter color of two neighbor block, and between the darker color of two neighbor block. The pecific function ρ that we ue in our experiment i given by ρ(x) =min(x 2,T 2 ) where the threhold i T = 40. Thi threhold i impoed on the growth of the penalty function in order to account for the poibility that two neighboring text region may come from different piece of text and therefore may have very different background and/or foreground color. In thi ituation, we would like to avoid impoing an exceive penalty. In addition, for every pair of neighboring block {r, } uch that i a text block and r i a background block, we aume that it i very likely that the background color of the two block are imilar. We etimate the color of the background block r from it etimated DC coefficient a μ r = Ỹr,0/8, and penalize the difference between thi color and the one of the two color of the text block that i cloer to μ r,byuing the following penalty: min(ρ(μ r θ,1),ρ(μ r θ,2)). (9) Combining the two new penalty term of Eq. (8,9) with the cot function of Eq. (7) and recating the cot in the DCT tranform domain, we obtain the following global cot function: 63X C = X min( (B t y ) i θ,1 2, (B t y ) i θ,2 2 ) +w X ˆρ(θr,1 θ,1)+ρ(θ r,2 θ,2) r, +w X r, min(ρ(μ r θ,1),ρ(μ r θ,2)) (10) where w>0i a weight factor, (B t y ) i i the i-th element of the intenity vector x = B t y for the block,and the firt ummation i performed over all text block ; the econd ummation i performed over all the pair of neighboring text block and r; the third ummation i performed over all the pair of neighboring block and r uch that i a text block and r i a background block. Thi cot mut be minimized with repect to all the DCT coefficient y of all the text block and all the parameter θ of all the text block. Thi minimization problem i non-convex and large, and for thee reaon i difficult to olve. To compute an approximate olution, we ue the iterative coordinate decent (ICD) algorithm. The ICD algorithm allow u to ue a local cot function for which it i eaier to contruct the optimization update. Our experiment how that the reulting two-color decoding trategy decode the text block at high quality. The ICD algorithm ucceively optimize the cot function with repect to one variable at a time. In our cheme, one full iteration of the ICD algorithm conit of two phae. In the firt phae, called block update, the cot function i optimized with repect to the DCT coefficient of all the text block. In the econd phae, called parameter update, the cot function i optimized with repect to the 490

4 parameter θ of all the text block. The DCT coefficient y of all the text block are initialized with the de-quantized coefficient Ỹ. The parameter θ of a text block i initialized with the mean of the two cluter reulting from k-mean clutering of X. The ICD iteration i repeated until the change in the cot function between two ucceive iteration i maller than a predetermined threhold. During the block update tage, the cot function i optimized with repect to one DCT coefficient at a time. The update of the k-th DCT coefficient of the text block i guided by y,k. Depending on the ign of y,k, y,k i adjuted until y,k =0or y,k reache either one of the end point of the contraint interval [Ỹ,k i/2, Ỹ,k + i/2]. During the parameter update tage, the cot function i optimized with repect to one parameter at a time. Similar to the block update tage, θ,1 i adjuted according to the ign of θ,1 =0. θ,1 until 5.2. Background Block and Picture Block Decoding Background block include both the background of the document and mooth region in natural. For thee block, mot of the information i captured by the DC coefficient. To mooth out the dicontinuitie acro block boundarie, a continuou Gauian Markov random field (GMRF) [8], [9] i choen a the prior model for the DC coefficient of the background block. AGMRFipecified completely by the neighborhood ytem and the prediction filter. In our application, we chooe a 8-point neighborhood ytem, and a patial invariant prediction filter h with coefficient given by h,r =1/6if r and are vertical or horizontal neighbor block, and h,r =1/12 if r and are diagonal neighbor block. The MAP etimate of the background block i again computed uing the ICD algorithm. For picture block, we ue the conventional decoding. Tranaction on Circuit and Sytem for Video Technology, vol. 9, no. 3, pp , Apr [3] J. Mcueen, Some method for claification and analyi of multivariate obervation, Proceeding of the Fifth Berkeley Sympoium on Mathematical Statitic and Probability, pp , [4] C. A. Bouman, Cluter: An unupervied algorithm for modeling Gauian mixture, Available from bouman, April [5] Charle A. Bouman and Michael Shapiro, A multicale random field model for Bayeian egmentation, IEEE Tranaction on Image Proceing, vol. 3, no. 2, pp , Mar [6] Ricardo L. de ueiroz, Proceing -compreed and document, IEEE Tranaction on Image Proceing, vol. 7, no. 12, pp , Dec [7] K. Kontantinide and D. Tretter, A variable quantization method for compound document, IEEE Tranaction on Image Proceing, vol. 9, no. 7, pp , Jul [8] Julian Beag, On the tatitical analyi of dirty picture, Journal of the Royal Statitical Society, vol. 48, no. 3, pp , [9] Julian Beag, Spatial interaction and the tatitical analyi of lattice ytem, Journal of the Royal Statitical Society, vol. 36, no. 2, pp , RESULTS AND CONCLUSION Fig. 5(a) how an converted from PDF at 300dpi. Fig. 4(b) and 6(b) how enlarged verion of the two rectangular region marked in Fig. 5(a). The reult of applying the block-baed egmentation algorithm of Section 4 to the in Fig. 5(a) i hown in Fig. 5(b). The original of Fig. 5(a) wa compreed by baeline with default quantization matrix multiplied by 3, and decompreed by baeline. The ame compreed wa alo decompreed by our propoed cheme. Fig. 6 how the reult of both baeline and our cheme for a mall text region. Ringing artifact are effectively eliminated by our propoed algorithm. Note that our formulation allow the foreground and background color to be locally adaptive, and there i no obviou color hift in the decoded. Fig. 4 how the reult for a mall region containing both background block and picture block. In the lowly varying blue ky, contouring and blocking artifact are ignificantly reduced by our algorithm. (a) Segmentation (b) Original 7. REFERENCES [1] Gregory K. Wallace, The till picture compreion tandard, Commun. ACM, vol. 34, no. 4, pp , [2] Thoma Meier, King N. Ngan, and Gregory Crebbin, Reduction of blocking artifact in and video coding, IEEE (c) Baeline (d) Propoed algorithm Fig. 4. Background block and picture block decoding. 491

5 (a) Segmentation (a) Original (b) Original (c) Baeline (b) Block-baed egmentation Fig. 5. Block-baed egmentation. (a) Original. (b) Segmentation. White: Background block; Red: Text block; Blue: Picture block. (d) Text block decoding Fig. 6. Text block decoding reult. 492