Segmentation Based Recovery of Arbitrarily Warped Document Images

Transcription

1 Segmentation Based Recovey of Abitaiy Waped Document Images B. Gatos, I. Patikakis and K. Ntiogiannis Computationa Inteigence Laboatoy, Institute of Infomatics and Teecommunications, Nationa Cente fo Scientific Reseach Demokitos, GR Agia Paaskevi, Athens, Geece Abstact Non-inea waping appeas in document images when captued by a digita camea o a scanne, especiay in the case that these documents ae digitized bounded voumes. Abitaiy waped documents may have sevea sope changes aong the text ines as we as aong the wods of the same text ine. In this pape, a nove segmentation based technique fo efficient estoation of abitaiy waped document images is pesented. The poposed technique ecoves the documents eying upon (i) text ines and wods detection using a nove segmentation technique appopiate fo waped documents, (ii) a fist daft binay image de-waping based on wod otation and tansation accoding to uppe and owe wod baseines, and (iii) a ecovey of the oigina waped image guided by the daft binay image de-waping esut. Expeimenta esuts on sevea abitaiy waped documents pove the effectiveness of the poposed technique. 1. Intoduction Document image acquisition by a digita camea o a fatbed scanne often esuts into sevea image distotions. Non-inea waping is a majo distotion that occus especiay when the scanned documents ae bounded voumes (see Fig. 1a). Waping not ony diminishes document s eadabiity but aso educes the accuacy of an OCR appication. Sevea techniques have been poposed fo coecting the document image waping that can be cassified in two main categoies: (i) 2D image pocessing techniques ([1], [2], [3], [4], [5]) and (ii) techniques on 3D document shape econstuction ([6], [7], [8]). Ou wok is eated to the fist categoy of techniques since the second categoy equies image captue with specia camea setup as we as document suface epesentation by using a 3D shape mode. Appoaches of the fist categoy have been epoted by sevea authos. In [1], a defomabe system to staighten cuved text image is pesented. Restoation is accompished by using an active contou netwok based on an anaytica mode with cubic B-spines which have been poved moe accuate than Bezie cuves. A mode fitting technique has aso been poposed using cubic spines to define the waping mode of the document image [2]. Fo moe accuate de-waping, a vetica division of a document image into some patia document images is aso suggested. Anothe mode fitting technique [3] divides the document image into shaded and non-shaded egion and then uses poynomia egession to mode the waped text ines with quadatic efeence cuves. In [4], the textue of a document image is cacuated so as to infe the document stuctue distotion. A mesh of the waped image is buit using a non-inea cuve fo each text ine. The cuves ae fitted to text ines by tacking the chaacte boxes on the text ines. The eoneousy fitted cuves ae detected and excuded by a post pocessing based on sevea heuistics. The appoach of [5] eies on a pioi ayout infomation and is based on a ine-by-ine de-waping of the obseved pape suface. Each ette in the input image is encosed within a quadiatea ce, which is then mapped to a ectange of coect size and position in the esut image. In ode to ecove abitaiy waped gay scae document images, we popose a nove technique that is based on (i) text ines and wods detection using a nove segmentation technique appopiate fo waped documents, (ii) a fist daft binay image de-waping based on wod otation and tansation accoding to uppe and owe wod baseines, and (iii) a ecovey of the oigina waped image guided by the daft binay image de-waping esut. The emaining of this pape is stuctued as foows: In Section 2, we detai the poposed appoach. Ou expeimenta esuts ae descibed in Section 3, whie in Section 4, concusions ae dawn.

2 2. The poposed appoach In ou appoach, we addess the pobem of estoing abitaiy waped gay scae document images using sevea distinct steps expained in the foowing sections. As a fist step, we poceed to document image binaization of the gay scae image I g using agoithm in [9], thus poducing the binay image I b Text ine and wod detection In this step, a nove efficient text ine and wod detection technique fo waped documents is intoduced. Fist, a wods ae detected using a pope image smoothing. Then, hoizontay neighboing wods ae consecutivey inked in ode to define text ines. Fo the sake of caity, we povide the compete stepwise pocess in the foowing: Step 1: Appy connected component abeing [10]. Step 2: Cacuate the histogam with the heights of a detected connected components. The maximum vaue of the histogam coesponds to the aveage chaacte height H. Step 3: Remove noise and non-text components with height > 3H o < H/4 o width < H/4. Step 4: Appy hoizonta smoothing (RLSA [11]) with theshod H foowed by a connected component abeing in ode to detect wods. Step 6: Detect the fist connected component in a topdown scanning. Set this component as wod W with bounding box coodinates ( x 1, x 2, y 1, y 2 ) and assign this to the fist text ine L. Step 7: Find wod W with bounding box coodinates ( x 1, x 2, y 1, y 2 ) that neighbos at a sma distance in the ight side of wod W. This is impemented as foows: fom a the connected components which satisfy the condition [ y1, y2] [ y1, y2], we seect the one with the smae distance D = x 1 - x2 ony if 0<D<5H. Step 8: Repeat step 7 fo the new wod W unti thee is no new wod that can be found in the ight diection. Step 9: Repeat steps 7 and 8 fo the eft side of W. Step 10: Labe a wods found in a eft to ight scanning ode and assign them to the fist text ine. Futhemoe, a fag is enabed to indicate that these wods wi not paticipate to futhe cacuations. Step 11: Repeat steps 6 to 10 fo the emaining text ines unti a the wods ae assigned to text ines. At the end of this pocedue, evey text ine is consideed as L i whie evey wod j that beongs to ine L i as W ij. An exampe of the text ine and wod detection pocedue is demonstated in Fig. 1. A fina task of this phase invoves: (i) meging the fist two wods (fom the eft) of evey text ine if the width of the fist wod is ess than 5H and (ii) ignoing wods with width ess than 2H. These conditions ae necessay since shot wods cannot ead to a safe wod sope detection which is necessay to the next step. On the othe hand, it is impotant to have accuate sope detection fo the fist wod fom the eft side which guides the aignment of the entie text ine. (c) Figue 1. Exampe of text ine and wod detection: oigina image; esut afte hoizonta smoothing fo wod detection (step 4) and (c) detected text ines afte consecutivey extacting ight and eft neighboing wods to the fist wod detected (indicated with a fied box) afte top-down scanning Wod owe and uppe baseine estimation This step concens the detection of the owe and uppe baseines which deimit the main body of the wods. Stating fom the smoothed wod image, we foow the methodoogy given in [12] which is used fo owe baseine detection. Accoding to this appoach, a inea egession is appied on the set of points that ae the owest back pixes fo each image coumn. In ou appoach, we aso use a simia pocedue to cacuate the uppe baseine. Afte this pocedue, uppe baseine of wod W ij (see Fig. 2) is defined as: y = aij x + b ij (1) Simiay, owe baseine of wod W ij (see Fig. 2) is defined as: y = a x + (2) ij b ij

3 Figue 2. Exampe of uppe and owe baseine estimation Daft de-waped binay image estimation In this step, a detected wods ae otated and tansated in ode to obtain a fist daft estimation of the binay de-waped image. The sope of each wod is deived fom the coesponding baseine sopes. Uppe and owe baseine sopes u θij and θ ij of wod W ij ae denoted as: u θ ij = actan( a ij ), θ ij = actan( aij ) (3) Since the smae absoute sope is usuay the most epesentative, the eftmost wod s sope can be defined as: u u θi0, if θi0 < θi0 θ i0 = (4) θi0, othewise whie the sope of a othe wods is the one with the neaest vaue to the sope of the pevious wod: u u θij, if θij < θij θ ij = (5) θij, othewise whee j>0. An exampe of detecting the wods sope is given in Fig. 3b. The otation of the wod W ij (x,y) is cacuated as foows: min y = ( x xij )*sin( θ ij ) + y *cos( θij ) (6) x = x min whee W ij ( x, y ) is the otated wod and x ij is the eft side of the bounding box of the wod W ij. An exampe of coecting the skew of the wods is given in Fig. 3c. Afte wod otation, a the wods of evey text ine, except fom the eftmost, must be veticay tansated in ode to estoe hoizonta aignment. The otation and tansation of the wod W ij (x,y) is done as foows: s y = y + dij s (7) x = x s s s whee W ij ( x, y ) is the otated and tansated wod and y is denoted at eq. 6. (c) (d) Figue 3. Exampe of daft binay image de-waping: oigina image; wod sope detection based on uppe and owe baseines; (c) wod skew coection and (d) fina de-waped document image afte wod aignment. d ij coesponds to the vetica wod tansation and is given by the foowing fomua: u u u yi0 yij, if θij < θij d = (8) ij yi0 yij, othewise whee: u min yij = ( aijxij + bij ) *cos( θij ) (9) and min yij = ( aijxij + bij )*cos( θij) (10) The eason of having two atenatives of tansation is that each wod may be otated eithe by its owe baseine o uppe baseine sope. Hence, it has to be tansated so that its owe o uppe baseine is aigned with the owe o uppe baseine of the eftmost wod of the text ine. An exampe of the daft binay image de-waping steps is given in Fig. 3. Duing this step, we stoe the tansfomation factos (T xy, Θ xy, X xy ) fo each pixe that is otated and tansated accoding to eq.7. These factos wi be used at the next step of ou agoithm fo the fina image ecovey (see section 2.4). The distinct steps we foow in ode to constuct the binay de-waped image, ae as foows: Step 1: Initiaization of dewaped binay image: Ib _ dew( x, y) = 0, x x ], y [1, y ] (11) [1, max max Step 2: Initiaization of tansfomation factos: T Θ = X = NULL, x x ], y [1, y ] (12) xy = xy xy [1, max max

4 Step 3: Dewaped binay image cacuation: 3. Expeimenta esuts ( x, y ) : Wij ( x, y ) = 1 I b _ dew ( x, y ) = 1, (13) whee y = ( x xijmin ) * sin( θij ) + y * cos(θij ) + d ij AND Txy = dij AND Θ xy = θij AND X xy = xijmin whee xijmin is the eft side of the bounding box of the wod Wij, θ ij is the sope of the wod Wij cacuated fom the uppe and owe baseine sopes (see eq. 5) and dij is defined in eq. 8. The poposed agoithm was tested using sevea abitaiy waped documents. We mainy focused on histoica abitaiy waped documents as we as on documents with majo distotions. Some epesentative esuts ae shown in Fig As we obseved, the expeimenta esuts indicate the effectiveness of the poposed technique. Some pobems that appea in ou expeiments ae mainy due to eoneous wod baseine detection Fina ecovey of the waped image In this step, we poceed to a compete estoation of the oigina gay scae waped image guided by the daft binay de-waping esut of the pevious stage. Since the tansfomation factos fo evey pixe in the fina binay de-waped image have been aeady stoed, the evese pocedue is appied on the gay scae pixes in ode to etieve the fina gay scae dewaped image. Fo a pixes that tansfomation factos have not been aocated, the tansfomation factos of the neaest pixe ae used. The foowing steps ae foowed: Step 1: Initiaization of dewaped gay scae image: (14) I g _ dew ( x, y ) = 0 fo x [1, x max ], y [1, y max ] Step 2: Repace NULL vaues of the tansfomation factos with the coesponding vaues of the neaest pixes: if I b _ dew ( x, y ) = 0 Txy = T x y AND Θ xy = Θ x y AND X xy = X x y (15) whee x, y : ag min ( x x + 2 * y y ) x, y I b _ dew ( x, y ) =1 The minimum distance used in ode to define the neaest pixe is biased to the x-axis in ode to achieve a pefeence to the pixes beonging to the same text ine. Step 3: Fina dewaped gay scae image cacuation: I g _ dew ( x, y ) = I g ( x, y ), (16) y - Txy - ( x X xy )sin(-θ xy ) whee y = cos(θ xy ) Figue 4. Recovey of a histoica abitaiy waped document: oigina image and de-waped image.

5 waped documents indicate the effectiveness of the poposed technique. Afte obseving some pobems that occued, we focus ou futue wok to deveop a moe efficient wod baseine detection agoithm. Acknowedgements This eseach is caied out within the famewok of the Geek Ministy of Reseach funded R&D poject POLYTIMO [13] which aims to pocess and povide access to the content of vauabe histoica books and handwitten manuscipts. Refeences Figue 5. Recovey of a document with majo distotions: oigina image and de-waped image. 4. Concusions and futue wok In this pape we pesent a nove segmentation based technique fo efficient estoation of abitaiy waped document images. Ou appoach is based on (i) text ines and wods detection using a nove segmentation technique appopiate fo waped documents, (ii) a fist daft binay image de-waping based on wod otation and tansation accoding to uppe and owe wod baseines, and (iii) a ecovey of the oigina waped image guided by the daft binay image de-waping esut. The expeimenta esuts on sevea abitaiy [1] O. Lavaie, X. Moines, F. Angea & P. Bayou, Active Contous Netwok to Staighten Distoted Text Lines, Poc. Int. Conf. Image Pocessing, 2001, pp [2] H. Ezaki, S. Uchida, A. Asano & H. Sakoe, Dewaping of document image by goba optimization, Poc. ICDAR 00, 2005, pp [3] Z. Zhang & C. L. Tan, Coecting document image waping based on egession of cuved text ines, Poc. ICDAR 03, 2003, pp [4] C. Wu & G. Agam, Document image de-waping fo text/gaphics ecognition, SSPR&SPR 2002, LNCS 2396, 2002, pp [5] A. Uges, C.H. Lampet & T.M. Beue, Document image dewaping using obust estimation of cued text ines, Poc. ICDAR 05, 2005, pp [6] C.L. Tan, L. Zhang, Z. Zhang & T. Xia, Restoing Waped Document Images though 3D Shape Modeing, IEEE Tans. Patten Anaysis and Machine Inteigence 28(2), 2006, pp [7] M.S. Bown & W.B. Seaes, Image Restoation of Abitaiy Waped Documents, IEEE Tans. Patt. Anaysis and Machine Inteigence, 26(10), 2004, pp [8] H. Cao, X. Ding & C. Liu, Rectifying the Bound Document Image Captued by the Camea: A Mode Based Appoach, Poc. Int. Conf. Comp. Vision, 2003, pp [9] B. Gatos, I. Patikakis & S.J. Peantonis, Adaptive Degaded Document Image Binaization, Patten Recognition, 39, 2006, pp [10] L. Shapio & G. Stockman. Compute Vision (Pentice Ha, 2001) [11] F.M. Wah, K.Y Wong & R.G. Casey, Bock Segmentation and Text Extaction in Mixed Text/Image Documents, Compute Gaphics and Image Pocessing, 20, 2006, pp [12] U.V. Mati & H. Bunke, Using a statistica anguage mode to impove the pefomance of an HMM-based cusive handwiting ecognition system, Int. Jouna of Patten Recognition and Atifica Inteigence, 15(1), 2001, pp [13] POLYTIMO_poject,