A Methodology for Document Image Dewarping Techniques Performance Evaluation

Transcription

1 Internatonal Conference on Document Analss and Recognton A Meodolog for Document Image Dewarpng Technques Performance Evaluaton N. Stamatopoulos, B. Gatos, I. Pratkaks Computatonal Intellgence Laborator, Insttute of Informatcs and Telecommuncatons, Natonal Center for Scentfc Research Demokrtos, GR-53 0 Aens, Greece {nstam, bgat, pratka}@t.demokrtos.gr Abstract One of e maor challenges n camera document analss s to deal w e page curl and perspectve dstortons. In spte of e prevalence of dewarpng technques, no standard for er performance evaluaton meod ests w most of e evaluaton done to concentrate n vsual pleasng mpressons. Ths paper presents an obectve evaluaton meodolog for document mage dewarpng technques. Frst, manuall selected sets of ponts of e ntal warped mage are matched w e correspondng ponts of e dewarpng result usng e Scale Invarant Feature Transform (SIFT). Each set corresponds to a representatve tet lne of e mage. Then, based on cubc polnomal curves at ft to e selected tet lnes, a comprehensve measure whch reflects e entre performance of a dewarpng technque n a concse quanttatve manner s calculated. Eperments applng e proposed performance evaluaton meodolog on two state of e art dewarpng technques as well as a commercal package are presented.. Introducton All modern OCR sstems are based on e assumpton at e tet lnes n a document are straght and horzontal whle s descrpton does not actuall hold. Document mage acquston b a dgtal camera often results nto several mage dstortons. Non-lnear warpng s a maor dstorton at occurs especall when e scanned documents are bounded volumes. Tet n such cases s strongl dstorted and nfluences e performance of furer processng. Over e last decade, man dfferent approaches have been proposed for document mage dewarpng []. These approaches can be classfed nto two broad categores based on () 3-D document shape reconstructon [2] and () 2-D document mage processng [3-7]. Approaches of e frst categor requre specalzed hardware or pror metrc knowledge, so e lmt e fleblt of capturng document w camera. On e oer hand, approaches n e second categor, whch have caught more attenton recentl, use onl 2-D nformaton from camera document mages. In spte of e prevalence of dewarpng technques, no standard performance evaluaton meodolog ests w most of e evaluaton done to concentrate n vsual pleasng mpressons [3-5]. As a result, e performance of ese meods s based on perceptual, subectve and qualtatve human vson evaluaton; hence obectve evaluatons or quanttatve comparsons among e dfferent dewarpng technques can not be obtaned. Furermore, e use of OCR as a means for ndrect evaluaton s wdel used n e evaluaton of dewarpng technques [6-8]. However, n man cases, such as n handwrtten or hstorcal documents, OCR sn t alwas avalable or t doesn t produce satsfactor results. In s paper, a novel meodolog for performance evaluaton of document mage dewarpng technques s presented, at avods e dependence on an OCR engne or human nterference. It s based on a pont-topont matchng procedure usng e Scale Invarant Features Transform (SIFT) [9] as well as e use of cubc polnomal curves for e calculaton of a comprehensve measure whch reflects e entre performance of a dewarpng technque n a concse quanttatve manner. The remander of e paper s organzed as follows. In Secton 2, e proposed evaluaton meodolog s detaled. Eperments applng s performance evaluaton meodolog on two state of e art dewarpng technques as well as a commercal package are presented n Secton 3. Fnall, conclusons are drawn n Secton /09 $ IEEE DOI 0.09/ICDAR

2 2. Evaluaton Meodolog The performance evaluaton of a dewarpng technque consders at e epected result should be consttuted onl from horzontal straght tet lnes, wout sufferng from an dstortons due to perspectve and page warpng. The proposed evaluaton meodolog s descrbed n e flowchart of Fg.. Frst, we manuall mark specfc ponts on e warped mage whch correspond to N representatve tet lnes of e orgnal mage. Then, usng SIFT transform, e marked ponts of e warped mage are matched to e correspondng ponts of e dewarped mage. Fnall, we estmate e cubc polnomal curves whch ft to ese ponts and based on e estmated curves we proceed to e etracton of e dewarpng evaluaton measure ( ). The dstnct stages of e proposed meodolog are analzed n e followng of s secton. whch must be corrected b a dewarpng technque. Fg. 2 depcts an eample w ree selected tet lnes ( N = 3 ). Tet Lne # Tet Lne #2 Tet Lne #3 Fgure 2. An eample of a marked warped mage w ree selected tet lnes (N=3); Enlarged mage porton of Pont-to-Pont Matchng Fgure. Flowchart of e proposed evaluaton meodolog. 2.. Manual markng on e warped mage Ths s e onl stage of e whole evaluaton process at requres e user s nterventon. However, s stage should be done onl once for each warped mage. Then, we can evaluate dfferent dewarpng results usng e same marked mage; as a result, we wll have a far comparson among e dfferent dewarpng technques. The user marks N set of ponts on e orgnal warped mage where each set corresponds to a representatve tet lne of e mage. The user should mark e ponts n e mddle of e man bod of e words. Also, e selected tet lnes should not be too small, should not be ttles or subttles so at e wll be representatve of e mage. Snce document deformaton due to perspectve and page warpng s generall not unform, we select N approprate tet lnes of e document w representatve deformaton At s stage, usng e SIFT transform [9] e manuall marked ponts of e orgnal warped mage are matched w e correspondng ponts of e dewarped mage. SIFT transform dentfes ke ponts b lookng n scale space for mama or mnma at e dfference-of-gaussan mage. Each pont s used to generate a feature vector at descrbes e local mage regon. The resultng feature vectors are called SIFT kes. In our meodolog, we etract e SIFT kes from e warped and e correspondng dewarped mage and en we dentf matchng ke ponts between em. In s wa, w e help of matchng ke ponts, e manuall marked ponts of e warped mage are matched w e correspondng ponts of e dewarped mage. For each marked pont M( m, m ) n e warped mage we fnd e two nearest ke ponts K( k, k) and K( 2 k2, k2 ) usng Eucldean dstance. Then, usng e matchng ke ponts K( k, k) and K(, ), respectvel, we defne e 2 k2 k2 957

3 correspondng pont M ( m, m ) n e dewarped mage w lnear nterpolaton as follows (see Fg 3): m = m a + b m = m a + b where a,b,a and b are denoted as follows: a a ( ) /( ) f = oerwse k2 k k2 k k k2 k k () (2) b = a (3) ( )/( ) f = oerwse k2 k k2 k k k2 k k (4) b = a (5) Fgure 3. Pont-to-Pont matchng usng SIFT transform and lnear nterpolaton. Fgure 4 depcts an eample of pont-to-pont matchng. As t can be observed, e marked ponts of e warped mage are matched w e correspondng ponts of e dewarped mage w great success. In Secton 3, we present epermental results on performance evaluaton of pont-to-pont matchng usng SIFT transform Evaluaton Ths s e fnal stage of e proposed evaluaton meodolog n whch we proceed to e computaton of e dewarpng evaluaton measure ( ) at reflects e entre performance of a dewarpng technque n a concse quanttatve manner. Usng e selected N set of ponts of e warped mage (each set corresponds to ponts at belong to a representatve tet lne of e mage) and e matchng ponts of e dewarped mage, we appromate each tet lne of e document w cubc polnomal curves. The ntegral of each cubc polnomal curve, over an nterval delmted b e curve endponts, ndcates e performance of e dewarpng technque, as e epected result should be consttuted onl from horzontal straght tet lnes. Let (, ) represent e marked ponts of e tet lne n e warped mage (see Fg. 2), where mn m a, 0 < L and N. Smlarl, (, ) represent e matchng ponts of e correspondng tet lne n e dewarped mage, where mn ma. Also, let LettH represent e average character heght of e dewarped mage whch s calculated as n [0]. We calculate e average value X of all and ever pont (, ) s ecluded f X > LettH n order to elmnate possble errors of e pont-to-pont matchng and erefore we wll have a better estmaton of e cubc polnomal curve. A Least Square Estmaton (LSE) meod s used to fnd e cubc polnomal curves at ft all ponts of bo mages. After s process e cubc polnomal curves at ft e tet lnes are defned as follows: and = a + a + a + a (6) = a + a + a + a (7) Fgure 4. An eample of pont-to-pont matchng; manuall selected ponts n warped mage; correspondng ponts n dewarped mage usng SIFT transform. In s wa, based on e estmated cubc polnomal curves, we defne e measure whch reflects e performance of e dewarpng technque w respect to e tet lne as follows: Ar Ar, f < = Ar Ar (8) 0, oerwse where 958

4 and Ar Ar ma 3 2 a 3 + a 2 + a (9) mn = ma mn ma 3 2 a 3 + a 2 + a mn (0) = ma mn As t can be observed 0. measure s equal to one when e tet lne n e dewarped mage s a horzontal straght tet lne at s e epected optmal result. It shows at e dewarpng technque produces e best result. On e oer hand, measure s equal to zero when e dewarped mage s e same or worse an e orgnal warped mage. For an overall quanttatve measure at consders e complete document mage we defne e as e mean value of all measures: N = = 00% N () where 0% 00%. The hgher e value of e, e better s e performance of e dewarpng technque. 3 2 = a + a + a mn 3 2 ma mn 3 2 = a + a + a 3 2 mn mn ma ma ma Fgure 5. The ntegrals of cubc polnomal curves of e warped mage and ts correspondng dewarped mage. 3. Epermental Results Pont-to-pont matchng s a crucal stage of e proposed evaluaton meodolog. For s reason we evaluate s stage usng 0 warped mages where we have manuall marked e correct matchng ponts on er dewarped mages. Then, we appl e two frst steps of our meodolog as descrbed n Sectons 2. and 2.2. Because small varatons don t affect e proposed evaluaton meodolog we consder a match as correct f e Eucldean dstance between e matchng pont and correct marked pont s smaller an a reshold. Table llustrates e percentage of e marked ponts at match correct f Eucldean dstance s 0 (perfect match),, 2, 3, and 4 respectvel. The results of e evaluaton demonstrate e success of pont-to-pont matchng usng SIFT transform. Table. Evaluaton of pont-to-pont matchng Eucldean Dstance Correct Match (%) Threshold 0 79,4 87,8 2 95,6 3 97,0 4 97,4 The proposed evaluaton meodolog for document mage dewarpng technques was eamned on a set of 20 warped mages usng two state of e art dewarpng meods [3,6] as well as e commercal package BookRestorer []. The frst meod [3] uses a novel segmentaton technque approprate for warped documents and en all words are pose normalzed guded b e lower and upper word baselnes. The second meod [6] uses a two-step approach. At e frst stage a coarse dewarpng s accomplshed w e help of a transformaton model. At second step fne rectfcaton s obtaned on e word level. In our eperments we used two cases of nteracton b markng ponts n ree tet lnes ( N = 3 ) and s tet lnes ( N = 6 ) w representatve deformaton nstances. Table 2 llustrates e average of all dewarpng technques. For comparatve reasons we have also ncluded e dewarpng results after applng onl e coarse dewarpng step of e meod [6]. It s wor mentonng at e parameter N does not nfluence e comparatve performance. Accordng to ese results meod [6] had better results an meod [3], whch s verfed also b work [6] where OCR accurac was used. Moreover, as t was epected, meod [6] had better performance an applng onl e coarse dewarpng step. Fgure 6 shows a representatve result. 959

5 Table 2. Comparatve results usng e proposed evaluaton meodolog Dewarpng Technque N = 3 N = 6 Usng BookRestorer [] 72% 73% Usng dewarpng meod [3] 78% 79% Usng dewarpng meod [6] applng onl e coarse dewarpng step Usng dewarpng meod [6] applng coarse and fne dewarpng steps 4. Conclusons 86% 86% 90% 89% Ths paper proposes an obectve meodolog for performance evaluaton of document mage dewarpng technques. Manuall selected sets of ponts of e warped mage are matched w e correspondng ponts of e dewarpng result usng e SIFT transform. Each set corresponds to a representatve tet lne of e mage. Then, based on cubc polnomal curves at ft to e selected tet lnes we etract a comprehensve measure whch reflects e performance of a dewarpng technque n a concse quanttatve manner. The evaluaton results of e proposed meodolog are verfed also b work [6] where OCR accurac was used. 5. Acknowledgement The research leadng to ese results has receved fundng from e European Communts Seven Framework Programme under grant agreement n (proect IMPACT). 6. References [] J. Lang, D. Doermann, and H. L. Camera-based analss of tet and documents: a surve Internatonal Journal on Document Analss and Recognton, 7(2-3), 2005, pp [2] C.L. Tan, L. Zhang, Z. Zhang and T. Xa, Restorng Warped Document Images rough 3D Shape Modelng, IEEE Trans. on Pattern Analss and Machne Intellgence, 28(2), 2006, pp [3] B. Gatos, I. Pratkaks and I. Ntroganns, Segmentaton Based Recover of Arbtrarl Warped Document Images Internatonal Conference on Document Analss and Recognton, Curtba, Brazl, 2007, pp [4] H. Ezak, S. Uchda, A. Asano & H. Sakoe, Dewarpng of document mage b global optmzaton Internatonal Conference on Document Analss and Recognton, Seoul, Korea, 2005, pp [5] Y. C. Tso and M. S. Brown, Geometrc and shadng correcton for mages of prnted materals A unfed approach usng boundar Conference on Computer Vson and Pattern Recognton, 2004, pp [6] N. Stamatopoulos, B. Gatos, I. Pratkaks and S. J. Perantons, A Two-Step Dewarpng of Camera Document Images, 8 Internatonal Workshop on Document Analss Sstems, Nara, Japan, 2008, pp [7] J. Lang, D. DeMenon, and D. Doermann, Flattenng curved documents n mages Conference on Computer Vson and Pattern Recognton, San Dego, USA, 2005, pp [8] F. Shafat, T. M. Breuel, Document Image Dewarpng Contest, In 2nd Int. Workshop on Camera-Based Document Analss and Recognton, Brazl, 2007, pp [9] D. G. Lowe, Dstnctve mage features from scalenvarant keponts Internatonal Journal of Computer Vson, 60, 2 (2004), pp [0] B. Gatos, T. Kondars, K. Ntzos, I. Pratkaks and S. J. Perantons, A Segmentaton-free Approach for Keword Search n Hstorcal Tpewrtten Documents, Internatonal Conference on Document Analss and Recognton, Seoul, Korea, 2005, pp [] BookRestorer: (c) (d) (e) Fgure 6. Recover of a warped mage: orgnal warped mage; usng BookRestorer; (c) usng meod [3]; (d) usng meod [6] applng onl e coarse dewarpng step; (e) usng meod [6] applng coarse and fne dewarpng steps. 960