ICDAR2007 Page Segmentation Competition

Transcription

1 ICDAR2007 Page Segmentaton Competton A. Antonacopoulos 1, B. Gatos 2 and D. Brdson 1 1 Pattern Recognton and Image Analyss (PRImA) Research Lab School of Computng, Scence and Engneerng, Unversty of Salford, Manchester, M5 4WT, Unted Kngdom 2 Computatonal Intellgence Laboratory, Insttute of Informatcs and Telecommuncatons, Natonal Center for Scentfc Research Demokrtos, GR Aga Paraskev, Athens, Greece bgat@t.demokrtos.gr Abstract Ths paper contnues the authors attempt to address the need for obectve comparatve evaluaton of layout analyss methods n realstc crcumstances. It descrbes the Page Segmentaton Competton (modus operand, dataset and evaluaton crtera) held n the context of ICDAR2007 and presents the results of the evaluaton of three canddate methods. The man obectve of the competton was to compare the performance of such methods usng scanned documents from commonlyoccurrng publcatons. The results ndcate that although methods contnue to mature, there s stll a consderable need to develop robust methods that deal wth everyday documents. 1 Introducton Layout analyss methods page segmentaton n partcular contnue to be reported n the lterature on a frequent bass, despte ths beng one of the most researched sub-felds of Document Image Analyss. It s not dffcult to see that the reason for ths s that the problem s far from beng solved. Successful methods have certanly been reported but, frequently, those are devsed wth a specfc applcaton n mnd and are fnetuned to the test mage dataset used by ther authors. The varety of documents encountered n real-lfe stuatons s far wder than the target applcatons of most methods. There s no doubt that, for a gven applcaton or for a generc selecton of real-lfe documents, t would be desrable to obtan an obectve evaluaton of the performance of dfferent layout analyss methods. However, such a drect comparson between algorthms s not straghtforward as t requres both the creaton of sutable ground truth (a relatvely laborous and precse task) as well as the defnton of a set of obectve evaluaton crtera (and a method to analyse them). Ths competton focuses on the evaluaton of page segmentaton and regon classfcaton subsystems. To the best of the authors knowledge, ths s only the thrd nstance of an nternatonal generc layout analyss competton (the prevous two beng the ICDAR2003 and ICDAR2005 Page Segmentaton Compettons [1 2]). It should be mentoned that a relatvely close prevous nstance, focusng on a specfc applcaton doman, was the Frst Internatonal Newspaper Page Segmentaton Contest [3] held by the authors n the context of ICDAR2001. Pror to that, an evaluaton of page segmentaton (as part of OCR systems) was performed at UNLV [4], based on the results of OCR. That approach, however, cannot not be strctly consdered to evaluate layout analyss methods snce the OCR-based evaluaton does not gve suffcent nformaton on the performance of page segmentaton and regon classfcaton and s only applcable to regons of text (or text-only documents). The motvaton for ths competton was the evaluaton of page segmentaton and regon classfcaton methods n realstc crcumstances. By realstc t s meant that the partcpatng methods are appled to scanned documents from a varety of sources, occurrng n real lfe. Ths s n contrast to the maorty of exstng datasets and reports of method results usng mostly structured documents (e.g., techncal artcles). The competton s descrbed next. In Secton 3, an overvew of the dataset and the ground-truthng process s gven. The performance evaluaton method and metrcs are descrbed n Secton 4, whle each of the partcpatng methods s summarsed n Secton 5. Fnally, the results of the competton are presented and the paper s concluded n Sectons 6 and 7, respectvely. 2 The competton The obectve of the competton was to evaluate layout analyss (page segmentaton and regon classfcaton) methods usng scanned documents from commonly-occurrng publcatons. In addton to the comparatve assessment, another obectve was to obtan a broad look at the performance of dfferent classes of methods (e.g., connected component analyss, morphologcal processng, analyss of background etc. as submtted for evaluaton) n dentfyng dfferent types of regons n a varety of documents.

2 sgnfcantly updated dataset. Ths dataset, whch wll shortly be released by the PRImA research lab, contans rcher ground truth (n a correspondngly updated XML format) that provdes a very wde range of nformaton on regon attrbutes (physcal and logcal). Although the dataset contans nstances of an exhaustve lst of document types, the competton subset focuses (for meanngful evaluaton purposes) on the most heavly used (n terms of nformaton content and need to analyse) types of documents, such as magazne pages and techncal artcles. It should be noted that, as the competton s on page segmentaton, the mages n the dataset have been processed to remove skew and other artefacts that would affect pre-processng and therefore mplctly also evaluate the pre-processng capabltes of the canddate methods. Fgure 1. Sample page mages from the tranng dataset. The competton ran n an off-lne mode. The authors of canddate methods regstered ther nterest n the competton and downloaded the tranng dataset (document mages and assocated ground truth). One week before the competton closng date, regstered authors of canddate methods were able to download the document mages of the evaluaton dataset. At the closng date, the organsers receved the results of the canddate methods, submtted by ther authors n a pre-defned format. The organsers then evaluated the submtted results. It should be noted that the off-lne mode s based on trust that the results submtted by the methods authors are genune. Ths trust s even more necessary f the evaluaton system s publcly avalable. In ths case, the evaluaton system was not made avalable (only the prncples were publcsed) and above all, the organsers have fath n the authors scentfc ntegrty. 3 The dataset It should be noted that there has been scarce avalablty of ground truth for the evaluaton of methods analysng complex layouts (e.g., havng non-rectangular regons). Such a dataset was created for the ICDAR2003 and ICDAR2005 compettons [1 2]. However, the current competton was based on a subset of a Fgure 2. Sample page mage from the tranng dataset showng the supermposed descrpton of regon contours. A balance had to be acheved between logstcs (a manageable number of document mages) and tractablty for current methods. The decson was, therefore, made to focus on a cross secton of 32 page mages, comprsng 47% techncal artcles (not necessarly wth Manhattan layouts) and 53% magazne pages. It should be noted that also for reasons of tractablty, the competton mages were b-level (n the general dataset the orgnal mages are n colour). A sample of page mages gven as part of the tranng dataset can be seen n Fg. 1. The ground truth of each page mage s an XML fle (defned as part of the general dataset) that contans mage and layout-specfc nformaton as well as the descrpton

3 of the regons n terms of sothetc (havng only horzontal and vertcal edges) polygons. The ground truth for the competton was produced usng a sem-automated tool developed by the authors. An XML vewer was developed for examnng the mages and the correspondng ground-truth XML, and was dstrbuted to the competton partcpants. Another sample page mage wth the correspondng descrpton of regons supermposed as sothetc polygons can be seen n Fg. 2. The types of regons defned for the competton (smplfed from the total number of dfferent types n the general dataset) are: () text, () graphcs, () lne art, (v) separator graphcal lne segments between regons, and (v) nose. 4 Performance evaluaton The performance evaluaton method used s based on countng the number of matches between the enttes detected by the algorthm and the enttes n the ground truth [5 7]. We use a global MatchScore table for all enttes whose values are calculated accordng to the ntersecton of the ON pxel sets of the result and the ground truth (a smlar technque s used n [8]). Let I be the set of all the ON mage ponts, G the set of all ponts nsde the ground truth regon, R the set of all ponts nsde the result regon, g the entty of ground truth, r the entty of result, Τ(s) a functon that counts the elements of set s. Table MatchScore(,) represents the matchng results of the ground truth regon and the result regon. Based on a pxel-based approach [5], and usng a global MatchScore table for all enttes, we can defne that: T( G R I ) MatchScore (, ) =, where a = T( (G R ) I ) { 1, f g = r 0, otherwse a (1) If N s the count of ground-truth elements belongng to entty, M s the count of result elements belongng to entty, and w 1, w 2, w 3, w 4, w 5, w 6 are pre-determned weghts, we can calculate the detecton rate and recognton accuracy for entty as follows: one2one g_one2many DetectRate + g_many2one = w1 + w 2 w 3 N N N (2) one2one d_one2many d_many2one RecognAccu racy w 4 + w 5 + w 6 M M M = (3) where the enttes one2one, g_one2many, g_many2one, d_one2many and d_many2one are calculated from MatchScore table (1) followng the steps of [5] for every entty. A performance metrc for detectng each entty can be extracted f we combne the values of the entty s detecton rate and recognton accuracy. We can defne the followng Entty Detecton Metrc (EDM ): 2DetectRate RecognAccuracy EDM + = (4) DetectRate RecognAccuracy A global performance metrc for detectng all enttes can be extracted f we combne all values of detecton rate and recognton accuracy. If I s the total number of enttes and N s the count of ground-truth elements belongng to entty, then by usng the weghted average for all EDM values we can defne the followng Segmentaton Metrc (SM): SM = N I I EDM 5 Partcpatng methods (5) Bref descrptons of the methods whose results were submtted to the competton are gven next. Each account has been provded by the method s authors and edted (summarsed) by the competton organsers. 5.1 The Tsnghua methods D. Wen and X. Dng, of Tsnghua Unversty (State Key Laboratory of Intellgent Technology and Systems), n Beng, Chna submtted two methods they developed as part of ther effort to buld a mult-language page segmentaton method. Both methods are mproved versons of the methods submtted to the ICDAR2005 competton [2]. Both methods are based on the same kernel, whch s called the Text Lne Extracton (TLE) module. The TLE s desgned to solve the (common to both approaches) problem of extractng text lnes n varous types of document, whether magaznes or newspapers, wth regular or rregular layouts, Englsh or Chnese (or any other language). It s a bottom-up aggregatng method, whch starts from connected components and merges them ncrementally to obtan herarchcal layout structures. The frst step of TLE s Canddate Lne Mergng, where connected components are merged accordng to ther 4-drecton Nearest Neghbour Connectng Strength [9] Then n the second step, Text Lne Fttng, canddate lne segments are further merged nto ntegrated text lnes by comprehensve consultaton of three factors: background separators, sngle lne consstency and neghbourng lnes consstency. That s, each par of neghbourng canddate lnes s merged when: 1) there s no background column separator between them; 2) the merged lne has good consstency n N

4 character szes, algnments and spacng; 3) at least one of ther common neghbourng lnes n the vertcal drecton suggests them to be merged. It s based on the results from TLE that dfferent regons are formed. In ths subsequent step, the frst Tsnghua method (TH1) s dfferent from the second (TH2) wth respect to the regon shape t supports. TH1 only supports rectangular regons. That s, each regon s only represented by ts boundng rectangle. For the nonrectangular (sothetc) textual regons, t tends to splt them nto several rectangular sub-regons. As for rregular graphcs and mage regons, t wll output ther boundng boxes only, even f they may overlap wth other regons. On the other hand, TH2 can support rregular regons. It takes the results from TH1 n terms of foreground nformaton and uses a background analyss method to trace the contours of textual regons [10]. Neghbourng textual regons are glued and output as sothetc polygonal regons. However, for the graphcs and mage regons, the process s stll nherted from TH1 so they are stll output as boundng boxes. 5.2 The BESUS method Ths method BESUS stands for Bengal Engneerng and Scence Unversty, Shbpur (Inda) was submtted by S.P. Chowdhury, S. Mandal and A.K. Das (of that unversty) n assocaton wth B. Chanda of the Indan Statstcal Insttute (ISI) n Calcutta. Smlarly to the earler versons of the method submtted by the authors to the ICDAR2003 and ICDAR2005 compettons [1 2], ths s a system constructed usng a number of morphology-based modules [11]. The segmentaton procedure s applcable to both Manhattan and non- Manhattan layouts and t can detect text n any orentaton. The segmentaton s carred out through the followng phases: 1. Pre-processng. Skew correcton s performed (not necessary n the competton dataset). The nformaton zone s also found out of the whole document by omttng boundary nose. 2. Graphcs segmentaton. A pseudo-greyscale mage s frst created (the method works n greyscale whereas the test mages were b-level) usng a low-pass adaptve flter based on the sze of obects and on the frequency of ther occurrence. Morphologcal open and close operatons are then used to generate a unque feature known as OCF matrx [12] whch s examned to estmate and remove the graphcs regons from the mage. 3. Lne art segmentaton. At ths stage the page mages contan manly lne art and text. The dea s to remove lne art regons usng the fact that they do not exhbt regular band structures as text lnes do. An extended mask regon s computed on all components to form groups and the smlarty of the components s examned. Lne art regons exhbt dfferent characterstcs to text and are dentfed and removed from the mage [13]. 4. Text segmentaton. Text mostly remans n the mage at ths pont, exhbtng a regular structure of textlnes and gaps between them. A vertcal wndow of sze 2(Text ht +Gap ht ) s created adaptvely based on the statstcal estmaton of the heght of the text band (Text ht ) and the lne gap (Gap ht ) n between two text lnes. Usng ths wndow a rough estmaton of text lnes s obtaned. Further refnement s acheved through the use of addtonal features such as pen wdth [14]. 6 Results The performance of the 3 segmentaton algorthms (BESUS, TH1 and TH2) was evaluated usng equatons (1) (5) for all 32 test mages wth parameters w 1 = 1, w 2 = 0.75, w 3 = 0.75, w 4 = 1, w 5 = 0.75 and w 6 = These parameters are set to gve maxmum score to one-to-one matches and rather generous scores to other (partal) matches. Evaluaton results for all types of enttes are shown n Fg. 3 where the EDM values averaged over all mages are depcted ( nose regons are omtted as ther number was not sgnfcant enough). Fg. 4 presents the Segmentaton Metrc (SM) values for all segmentaton algorthms averaged over all mages. The BESUS method has a slght overall advantage over TH2 and TH1 wth SM results of 55.75%, 55.46% and 51.75% respectvely. In more detal, concernng text regon segmentaton, the BESUS method acheved the hghest averaged EDM rate value (68.29%) whle TH1 and TH2 acheved an averaged EDM rate value of 53.82% and 58.56%, respectvely. For graphcs, TH1 acheved the hghest averaged EDM rate value (17.32%). For lne-art enttes, the BESUS method acheved the hghest averaged EDM rate value (14.52%) whle for separator detecton, TH1 and TH2 both acheved the hghest averaged EDM rate value (64.38%). Both Tsnghua methods acheved zero EDM rate values for lne-art segmentaton. 7 Conclusons The motvaton for the ICDAR2007 Page Segmentaton Competton was to evaluate exstng approaches for page segmentaton and regon classfcaton usng a realstc dataset and an obectve performance analyss system. The mage dataset used comprsed both scanned techncal artcles and (mostly) magazne pages. The performance evaluaton method used s based on countng the number of matches between the enttes detected by the algorthm

5 and the enttes n the ground truth. The competton ran n an off-lne mode and evaluated the performance of three segmentaton algorthms. The evaluaton results show that the BESUS method has an overall advantage (and gves better results for text and lne-art). TH1 and TH2 performed better at segmentng separator regons, whle the TH1 method performed best on graphcs regons Text Graphcs Lne-Art Separator BESUS method TH1 method TH2 method Fgure 3. Evaluaton results for all enttes (EDM values averaged over all mages) BESUS method TH1 TH2 Fgure 4. Averaged Segmentaton Metrc (SM) values. Acknowledgement The authors gratefully acknowledge the support of Google n creatng the dataset used n ths competton. References [1] A. Antonacopoulos, B. Gatos and D. Karatzas, ICDAR2003 Page Segmentaton Competton, Proceedngs of the 7 th Internatonal Conference on Document Analyss and Recognton (ICDAR2003), Ednburgh, UK, August 2003, pp [2] A. Antonacopoulos, B. Gatos and D. Brdson, ICDAR2005 Page Segmentaton Competton, Proceedngs of the 8 th Internatonal Conference on Document Analyss and Recognton (ICDAR2005), Seoul, South Korea, August 2005, pp [3] B. Gatos, S.L. Mantzars and A. Antonacopoulos, Frst Internatonal Newspaper Segmentaton Contest, Proceedngs of the 6 th Internatonal Conference on Document Analyss and Recognton (ICDAR2001), Seattle, USA, September 2001, pp [4] J. Kana, S.V. Rce, T.A. Nartker and G. Nagy, Automated Evaluaton of OCR Zonng, IEEE Transactons on Pattern Recognton and Machne Intellgence, Vol. 17, No. 1, January, 1995, pp [5] I. Phllps and A. Chhabra, "Emprcal Performance Evaluaton of Graphcs Recognton Systems," IEEE Transacton of Pattern Analyss and Machne Intellgence, Vol. 21, No. 9, pp , September [6] A. Chhabra and I. Phllps, "The Second Internatonal Graphcs Recognton Contest - Raster to Vector Converson: A Report," n Graphcs Recognton: Algorthms and Systems, Lecture Notes n Computer Scence, volume 1389, pp , Sprnger, [7] I. Phllps, J. Lang, A. Chhabra and R. Haralck, "A Performance Evaluaton Protocol for Graphcs Recognton Systems" n Graphcs Recognton: Algorthms and Systems, Lecture Notes n Computer Scence, volume 1389, pp , Sprnger, [8] B.A. Yankoglu, and L Vncent, "Pnk Panther: a complete envronment for ground-truthng and benchmarkng document page segmentaton", Pattern Recognton, volume 31, number 9, pp , [9] M. Chen, X. Dng, et al. Analyss, Understandng and Representaton of Chnese newspaper wth complex layout. Proceedngs of 7 th IEEE Internatonal Conference on Image Processng, Sept. 2000, Vancouver, BC, Canada, IEEE. [10] A. Antonacopoulos, Page Segmentaton Usng the Descrpton of the Background Computer Vson and Image Understandng, vol. 70, no. 3, 1998, pp [11] A.K. Das and B. Chanda, Segmentaton of Text and Graphcs n Document Image: A Morphologcal Approach, Proceedngs of the Internatonal Conference on Computatonal Lngustcs, Speech and Document Processng (ICCLSDP 98), Calcutta, Inda, December 1998, pp. A50 A56. [12] A. K. Das and B. Chanda, Extracton of half-tones from document mages: A morphologcal approach, Proceedngs of the Internatonal Conference on Advances n Computng, Calcut, Inda, Aprl 6 8, 1998, pp [13] S. P. Chowdhury, S. Mandal, A. K. Das, and B. Chandha, An effcent method for graphcs segmentaton from document mages, Proceedngs of the 6 th Internatonal Conference on Advances n Pattern Recognton, Kolkata, Inda, Jan. 2-4, 2007, pp [14] S. P. Chowdhury, S. Mandal, A. K. Das, and B. Chandha, Segmentaton of text and graphcs from document mages, Proceedngs of the 9 th Internatonal Conference on Document Analyss and Recognton (ICDAR 2007), Sept, 2007, Curtba, Brazl.