A review of branch and band algorithms for geometric and statistical layout analysis

Transcription

1 A revew of branch and band algorthms for geometrc and statstcal layout analyss Thomas M. Breuel To cte ths verson: Thomas M. Breuel. A revew of branch and band algorthms for geometrc and statstcal layout analyss. Jun 2004, <sc_ > HAL Id: sc_ Submtted on 7 Dec 2004 HAL s a mult-dscplnary open access archve for the depost and dssemnaton of scentfc research documents, whether they are publshed or not. The documents may come from teachng and research nsttutons n France or abroad, or from publc or prvate research centers. L archve ouverte plurdscplnare HAL, est destnée au dépôt et à la dffuson de documents scentfques de nveau recherche, publés ou non, émanant des établssements d ensegnement et de recherche franças ou étrangers, des laboratores publcs ou prvés.

2 A Revew of Branch-and-Bound Algorthms for Geometrc and Statstcal Layout Analyss Thomas M. Breuel Unversty of Kaserslautern and DFKI Résumé : Many dfferent approaches to the geometrc and statstcal analyss of document layouts have been proposed n the lterature. The development of practcal branchand-bound algorthms for solvng geometrc matchng problems under nose and uncertanty has enabled the formulaton of new classes of geometrc layout analyss methods based on globally optmal maxmum lkelhood nterpretatons for well-defned models of the spatal statstcs of document mages. I revew ths approach to geometrc layout analyss usng text lne fndng and column fndng n the presence of nose and uncertanty as examples and compare the approach wth selected other statstcal and geometrc layout analyss methods. Mots-clés : document layout analyss, geometrc matchng, text lne fndng, branch-and-bound algorthms, global optmzaton 1 Introducton In addton to ther purely textual content, rendered documents contan a wealth of nformaton n the geometrc arrangement of the text and fgures on the page the page layout. Examples of propertes encoded n the page layout are nformaton about whch text corresponds to the ttle, author, page number, and abstract of a document, the order n whch the body text s to be read (the readng order), and major logcal dvsons n the body text. Recoverng ths nformaton s the problem of document layout analyss. Document layout nformaton has a varety of uses. It s a key step n the converson of scanned documents nto machne readable form ; that s, what we typcally thnk of optcal character recognton (OCR) actually comprses both layout analyss and recognton of ndvdual characters. In fact, even the recognton of characters n OCR depends on correct document layout analyss, snce the nterpretaton of certan characters s affected by ther poston relatve to the text lne and snce statstcal language models depend on the correct readng order of the text. Furthermore, the user of an OCR system usually expects to obtan not just a vector graphcs fle wth thousands of characters placed at specfc locaton n the mage, but nstead an edtable and structured text fle that contans text n ts correct readng order and correctly dentfes actual lne and paragraph breaks. But whle OCR s perhaps the most mportant use of document layout analyss, t s not the only one. Document databases need to extract nformaton that permts ndexng and retreval of documents. Ths nformaton can often only be derved from the layout of the text (as opposed to the textual content or even font propertes). For example, ttles and authors of scentfc papers tend to be prnted at the top of the frst page, centered, wth the ttle mmedately precedng the author and separated from the rest of the text by whtespace, propertes that are recoverable by document layout analyss. Another applcaton of document layout analyss s magebased reformattng and reflow of documents, a technque that allows the dsplay of scanned documents on small-screen devces wthout OCR errors and whle preservng the appearance of the orgnal document [BRE 02b]. 2 Layout Prmtves The actual layout of a document s the result of the applcaton of complex, nteractng rules about where to place text on the page. Some of those rules are consequences of propertes of the human vsual system and attempts to make text more readable (e.g., keepng lne lengths below a certan number of characters per lne), others are the results of physcal constrants (e.g., page sze), constrants mposed by tradtonal type settng equpment (e.g., the use of straght and parallel text lnes), conventon (e.g., where page numbers and ttles go), as well as stylstc and artstc consderatons. Whle layouts can become enormously complex, almost all layouts tend to be composed of a number of recurrng prmtves. The most mportant of these are text lnes, text columns, sectons, and paragraphs. Furthermore, these prmtves have a number of common geometrc relatonshps between them, defned by ther relatve sze, spacng, algnment, and justfcaton. We call the extracton of these prmtves physcal document layout analyss. We refer to the extracton of hgher-level propertes of a document (lke ttles, authors, page numbers, etc.) as logcal document layout analyss. Logcal document layout analyss generally makes use of physcal document layout analyss to acheve ts goals. Ths work deals prmarly wth physcal document layout analyss, that s, the relable extracton of prmtves lke text lnes and text columns, and the geometrc relatonshp between those prmtves. 3 Prevous Methods A large number of physcal document layout analyss technques have been proposed n the lterature ([CAT 98] provdes a good overvew). In order to perform ther functon, layout analyss technques make assumptons (explctly or

3 mplctly) about the geometrc propertes of the layouts that they are appled to. Commonly made assumptons, by dfferent systems, are that text lnes are parallel to each other, that text on the page s composed of characters of approxmately the same font sze, that paragraph boundares are straght and perpendcular to text lnes, that paragraph boundares are rectangular, that larger logcal dvsons correspond to greater physcal spacng of text, that paragraphs of text contan only lmted amounts of whtespace, and many more. Such assumptons are approxmate expressons of the physcal, stylstc, and perceptual constrants on layout analyss. In practce, none of the assumptons made by current layout analyss systems result n human-lke performance on real documents ; that s, for each of those assumptons, there wll be some classes of documents that volate those assumptons, yet are easly nterpretable by a human reader. A complete taxonomy of the assumptons made by dfferent document layout analyss methods goes beyond the scope of ths paper, but we wll try to examne mportant dfferences as they arse. Whle the models revewed n ths paper does not permt the nterpretaton of all documents that a human reader could easly nterpret, t does relax some commonly made assumptons, whch makes t applcable to a wder class of document layouts. Frst, many prevous physcal layout analyss methods explctly or mplctly assume that text lnes on a page are (mostly) parallel to each other. For example, projecton methods and Fourer transform methods for skew estmaton assume that most of the text on the page s at a sngle orentaton and then attempt to correct page rotaton ( skew ) and transform all text lnes nto lnes that are orented at multples of 90 o relatve to the page boundares ; text lnes are then dentfed n a second step n ths corrected mage (see [HAR 94, BAI 87, SRI 89, BAI 87, OKU 99] for further detals). Ths assumpton of a global page rotaton of an otherwse rectlnear layout means that such methods are not applcable to some mportant classes of documents, such as those captured wth dgtal cameras and hence subject to perspectve dstortons. Some prevous methods for fndng text lnes do not requre rectlnear layouts, but they have had to sacrfce precse geometrc models of text lnes (n terms of baselnes, descenders, and ascenders) for smpler methods usng proxmty of connected components [O G 93, KIS 98] text lne fnder descrbed here does not make pror assumptons about text lne parallelsm, but stll models the geometry of each text lne precsely. Another assumpton commonly made by other layout analyss systems s that of a physcally represented layout herarchy. That s, layout components lke sectons, paragraphs, and text lnes form a herarchy based on ncluson : sectons are composed of paragraphs, and paragraphs are composed of text lnes. It seems plausble to take a top-down or bottomup approach to recoverng these layout components. That s, such methods frst dentfy sectons, then paragraphs, and fnally text lnes. Examples of such approaches are approaches lke X-Y cuts [NAG 84] that successvely splt the page nto smaller and smaller unts top-down, and approaches that use proxmty of connected components [O G 93, KIS 98] or mathematcal morphology [CHE 95] group elements on the page nto successvely hgher level groupng n a bottom up manner. However, n practce, the logcal layout herarchy does not appear to be very well represented n smple geometrc propertes of the physcal layout ; hgher-level layout propertes do not, n general correspond to larger amounts of whtespace or other large-scale features. Instead, logcal layout components lke paragraphs and sectons are often represented only through subtle means lke ndentaton or changes n font style (e.g., bold secton headngs), whle physcal layout structures lke text columns or page breaks that do not form part of the logcal layout herarchy often have much smpler geometrc representatons on the page. The layout analyss technques descrbed here are not organzed around a logcal layout herarchy ; nstead, they concentrate on dentfyng the vsually most salent propertes of the page layout ts columnar structure and ts text lnes. Logcal layout propertes lke paragraphs and sectons can then be recovered from that nformaton. 4 Statstcal and Geometrc Models 4.1 Text Lnes Probably both for typesettng and for perceptual reasons, text n many wrtng systems s set along straght lnes. For Latn scrpts, whch we focus on n ths work, each character rests ether on a baselne or on a lne of descenders (Fgure 1). The top of each character reaches ether up to the x-heght, or to the lne of ascenders. Note that the locaton of each character on the baselne or lne of descenders s only approxmate typographc conventons resultng from perceptual phenomena dctate that n well-desgned fonts, the actual locaton of ndvdual characters dffers slghtly from ther precse locaton relatve to the baselne. As s commonly done, we approxmate the shape of each character by ts (axs-algned) boundng box and use the bottom center of the boundng box as a reference pont [BAI 87]. If there s no rotaton present at all, ths reference pont wll le on the baselne or lne of descenders f the character does ; for small page rotatons, the poston of the reference pont wll dffer from the actual locaton of the character relatve to the baselne, but for the range of page rotatons encountered n practce, the dfference s slght. The amount of dfference depends, however, on the character shape, and t wll be larger for characters whose boundng box s defned by straght, off-center lnes (e.g., H ) than for characters lke T or O. Devatons from the baselne, errors resultng from usng the boundng box approxmaton, mage nose, and scanner quantzaton all result n some nose n the locaton of reference ponts relatve to the baselne or lne of descenders. We approxmate ths nose by a Gaussan dstrbuton, but bound the maxmum allowable devaton ; that s, beyond a certan devaton from a gven baselne, t s more lkely that the character n queston belongs to a dfferent baselne or background nose rather than belongng to the same baselne wth a large devaton. The text lne model shown n Fgure 1 s parameterzed by two lne parameters for the baselne (we use angle θ and dstance from orgn, r), plus three parameters for the sze of the

4 FIG. 1 The text lne model used by the algorthm. descenders, the x-heght, and the ascenders. However, every character rests ether on the baselne or lne of descenders. Therefore, n order to dentfy all characters contrbutng to a text lne, t s suffcent to model the baselne and the lne of descenders, a three-parameter problem. 1 Ignorng the lne of descenders for a moment, the lkelhood wth whch a character s found at some pont p gven a set of lne parameters (θ, r) s then (up to normalzaton factors) : P (p θ, r) max(β, G σ (d(p, l θ,r ))) (1) Here, l θ,r s the baselne, d(p, l θ,r ) s the dstance of p from the baselne, G σ (x) s a Gaussan dstrbuton wth zero mean and standard devaton σ, and β s a background probablty. Ths model s analogous to the model for geometrc matchng descrbed by [III 97], where the reader can fnd a more extensve justfcaton. Takng nto account the lne of descenders, we arrve at a mxture dstrbuton P (p θ, r, d d ) max(β, (1 λ) G σ (d(p, l θ,r )) + λ G σ (d(p, l θ,r dd ))) (2) = max(β, (1 λ) G σ (d(p, l θ,r )), λ G σ (d(p, l θ,r dd ))) (3) Here, d d s the dstance of the lne of descenders from the baselne, and λ s the frequency wth whch descenders occur. We have assumed that the ranges for whch the dstrbutons around the baselne and around the lne of descenders are greater than the background dstrbuton are well separated, whch has allowed us to replace the sum n Equaton 2 wth the max n Equaton 3. Wth the above statstcal model, we can now formulate text lne fndng as that of fndng a maxmum lkelhood soluton for our statstcal model : ˆl θ,r,dd = arg max P (p θ, r, d d ) (4) For the purpose of fndng maxmum lkelhood solutons, t s easer to take the logarthm of ths equaton (the logarthm 1 Modelng the lne of ascenders and the x-heght can be used to reject characters from a match that accdentally fall on the baselne or the lne of descenders. That can mprove robustness for very nosy documents or documents wth unusual layouts. s a monotonc functon) : ˆl θ,r,dd = arg max = arg max log P (p θ, r, d d ) (5) max(log β, log(1 λ) + log G σ (d(p, l θ,r )), log λ + log G σ (d(p, l θ,r ))) (6) = arg max max(0, log(1 λ) + log G σ (d(p, l θ,r )) log β, log λ + log G σ (d(p, l θ,r )) log β) (7) By choosng parameters approprately and wrtng x = d(p, l θ,r ) ths reduces to a partcularly smple form : ˆl θ,r,dd = arg max max(0, c 1 c 2 x 2, c 3 c 4 x 2 ) (8) Here, the parameters c 1,..., c 4 depend on β, σ, and λ. Ths s the optmzaton that we need to perform n order to fnd the maxmum lkelhood soluton to the text lne fndng problem under our statstcal model. It s closely related to robust least square fttng n robust statstcs [HUB 81]. Optmzng these knds of functons s dffcult wth tradtonal methods, such as gradent descent, because they have many local mnma and large flat regons. However, below, we wll see how we can fnd globally optmal solutons to such optmzaton problems. As stated above, the approach assumes that there s only a sngle text lne on the page plus background nose. However, for each reference pont that contrbutes to the total lkelhood for a partcular choce of lne parameters, we not only get a contrbuton to the total lkelhood, but also an ndcaton of whether t was consdered part of the text lne or whether t contrbuted as a background pont (β). Ths means that once we have found a maxmum lkelhood soluton, we can dvde the reference ponts that contrbuted to t nto those that contrbuted as background ponts and those that dd not. The reference ponts that dd not contrbute as background ponts are then counted as beng part of the lne, whle the background ponts can be re-examned for the next-best maxmum lkelhood soluton 2 2 A formal justfcaton of ths approach goes beyond the scope of ths

5 FIG. 2 A smple column model. The red rectangles are boundng boxes for connected components, the green dots are the reference ponts used for each boundng box, and the blue lne s a robust lnear ft to the reference ponts. 4.2 Column Boundares At frst sght, the problem of fndng text columns s qute smlar to the problem of fndng text lnes. That s, text s typeset n columns so that the leftmost character n each column algns on a straght lne wth the other leftmost characters on each text lne that s part of the column. We can therefore pck the bottom left corner of each character as an algnment pont and perform a lne matchng operaton smlar to that used for text lne fndng. The lkelhood model and optmzaton problems are also analogous to that for text lne fndng, except that modelng of baselnes s not requred. Ths approach s llustrated n Fgure 2. Such an approach has many of the advantages of the text lne fndng algorthm mentoned above : t can fnd column boundares even f they are not parallel to each other (e.g., n perspectvely dstorted document mages), and t does not rely on any global propertes of the page. An example of the applcaton of a combned text lne and column fndng method can be found n Fgure 3, where t s appled to the problem of removng perspectve dstortons from mages captured wth a hand-held camera. However, whle useful n some nstances, unlke the text lne model, such a column model s not satsfactory for complex documents because of the statstcs of real documents. One smple way of understandng that dfference s the followng observaton. If we pck out an arbtrary connected component correspondng to a character, t s wth hgh probablty part of some text lne. In contrast, only a small fracton of all characters are actually part of a column boundary (namely, only those characters at the begnnng of a text lne). Therefore, there s almost no possblty for false postves durng text lne fndng, snce there are very few connected components not part of text lnes, and those components are unlkely to be algned lnearly, whle there s a sgnfcant probablty of false postves durng text lne fndng, namely when characters that are otherwse not part of any column boundary accdentally algn lnearly. In order to recognze text columns relably, we need to take advantage of addtonal nformaton that helps us dstngush paper. actual text columns from accdental algnments of characters. One property that suggests tself s the use of whtespace. In fact, many algorthms for geometrc layout analyss rely only on whtespace for detectng column boundares. Examples are the use of X-Y segmentatons [NAG 84], and whtespace covers [BAI 94, BAI 90]. However, n our experence, whtespace covers by themselves, wthout the use of addtonal nformaton, are not a relable method for column detecton ether. But combnng the computaton of whtespace covers wth the computaton of columnar algnments turns out to result n a method that emprcally recognzes column boundares wth lower error rates than ether whtespace-only or algnment-only methods alone. There are several ways of formalzng the combnaton of whtespace covers wth measurng the algnment of connected components as a column boundary. Frst, we can start wth the maxmum whtespace rectangle formulaton as descrbed by Bard et al. [BAI 94, BAI 90] and add a requrement that a mnmal number of connected components be present around the outsde of that rectangle. Alternatvely, we can start wth a column fnder based on lnear algnment of reference ponts, as n Fgure 2 and add a requrement that sgnfcant amounts of whtespace be present next to the column boundary. For hstorcal reasons maxmal rectangle whtespace covers had already proven to be farly relable at detectng column boundares and snce we developed smple and effcent algorthms for computng them we adopted the frst approach. Ths approach and nsght then leads to a computatonal geometry problem for fndng column separators. In the case of axs-algned document layouts, we can state ths problem formally as follows (also llustrated n Fgure 5. Defnton 1 Axs-Algned Maxmum Whtespace Rectangle wth Halo Problem. The nput to the algorthm s a rectangular outer bound B, a collecton of axs-algned rectangular obstacles R, a dstance ɛ, and a threshold count h. The output of the algorthm s a rectangle M wth maxmal area satsfyng (1) M B, (2) for all R, area(m R ) = 0, and (3) the number of rectangular obstacles R wthn a dstance

6 FIG. 3 Applcaton of the text lne and column fnders to the problem of removng perspectve dstortons of documents captured wth hand-held cameras. The applcaton s enabled by the fact that the text lne and column fnders do not make any assumptons about parallelsm among text lnes or columns. FIG. 5 Maxmal whtespace rectangles wth halo. See the text for detals. of ɛ of M s greater than h, #{R : dst(r, M) ɛ} h. Ths defnton can be generalzed easly to non-axs algned rectangles, although (as we wll see) the resultng algorthm dffers sgnfcantly. Furthermore, n practce, t s useful to lmt the aspect rato of the maxmal rectangles found by the algorthm. Note that our reasonng for statng ths approach to column fndng has a rather dfferent motvaton from our approach to text lne fndng. In the case of text lne fndng, we started wth a clearly defned geometrc model of text lnes (characters restng ether on the baselne or lne of descenders), chose an error model, and derved the maxmum lkelhood soluton. In the case of column fndng, whle we were motvated by a smlar model algnment of characters along a lne we ultmately chose an algorthm based on maxmal whtespace rectangles because such algorthms have been found to work well n prevous work [BAI 94, BAI 90] ; the constrant of lnear algnment of the connected components correspondng to the frst character of each text lne adjacent to the column separator comes n through the halo requrement. Ths requrement of havng a halo of connected components under a gven error bounds corresponds to a bounded unform error nose model, as opposed to the Gaussan error model used wth the text lne fnder. 5 Branch-and-Bound Algorthms In the prevous secton, we have dscussed the knds of statstcal and geometrc models that correspond to text lnes and column boundares. However, we have so far left open the queston of how to fnd optmal solutons under those models ; wthout practcal algorthms, such models would smply not be very useful. In fact, the use of the models descrbed n the prevous secton has only been enabled through the development of a new class of practcal optmzaton algorthms. These algorthms, descrbed below, combne deas from branch-and-bound geometrc matchng algorthms n computer vson [BRE 92] and nterval arthmetc optmzaton [HAN 80]. Furthermore, they ncorporate an mportant optmzaton that we refer to as matchlsts [BRE 92]. The basc dea behnd these algorthms s to formulate the search for an optmal soluton as a search over a multd-

7 FIG. 4 A column model that takes nto account whtespace. The red rectangles are boundng boxes for connected components, the green dots are the reference ponts used for each boundng box, and the blue lne s a robust lnear ft to the reference ponts. The blue area to the left of the blue lne s the columnar whtespace. mensonal parameter space. In the case of fndng maxmum lkelhood text lne solutons, the parameter space s threedmensonal : t conssts of a range of angles [θ, θ], a range of dstances from the orgn [r, r], and a range of dstances for the lne of descenders from the baselne [d d, d d ]. In the case of fndng non-axs algned rectangles wth halo, the parameter space s four dmensonal 3 and conssts of a range of rectangle centers [x, x] and [y, y], a range of orentatons [θ, θ], and an aspect rato [r, r]. In both cases, the objectve functons (Equaton 8 and the area functon subject to the constrants n Defnton 1) are ll-suted to the usual gradent-based optmzaton methods because they contan large flat regons and many local mnma. In order to fnd good solutons to these problems, we need global optmzaton technques. The combnaton of branch-and-bound methods together wth nterval arthmetc has proven to be a powerful tool for these knds of problems [HAN 80, BRE 03b]. The dea s that we subdvde the parameter space nto hyper-rectangular subregons and compute upper and lower bounds on the value of the objectve functon over each subregon. Ths can be accomplshed usng nterval arthmetc. Interval arthmetc allows us to take an objectve functon n algebrac notaton and replace the arthmetc operatons occurrng n that objectve functon wth ther nterval equvalents. (Ths can be accomplshed automatcally n languages lke C++ through overloadng.) The resultng nterval objectve functon then computes upper and lower bounds on the value of the objectve functon over each hyper-rectangle n parameter space. Those bounds are not guaranteed to be tght, but they are guaranteed to be convergent [JAU 01] nformally, as the hyper-rectangle n parameter space becomes smaller, the bounds are guaranteed to approxmate the true value of the objectve functon closer and closer. In order to make ths approach practcal for these applca- 3 Such rectangles are actually descrbed by fve parameters, but the requrement that the rectangles are maxmal reduces the parameter space that needs to be searched to four dmensons, gven a partcular set of obstacles. tons, we need to apply an mportant optmzaton. Observe that the objectve functon for text lne fndng, gven by Equaton 8 s a sum of ndvdual contrbutons. Furthermore, we have rewrtten that objectve functon so that the contrbuton of background features to the total sum s zero. The nterval equvalent of ths objectve functon wll be evaluated repeatedly n the nner loop of the search algorthm. But t can be shown easly that f a contrbuton from some feature pont p s zero for a hyper-rectangle n parameter space, t wll reman zero for any subrectangle of that hyper-rectangle. All ponts makng zero contrbutons therefore need not be consdered durng further evaluatons of the objectve functon for subrectangles n parameter space. We keep track of the ponts makng non-zero contrbutons on a smple data structure, the matchlst. As we consder subrectangles of a hyper-rectangle n parameter space, ponts are removed from the matchlst and need not be evaluated any further n the computaton of the objectve functon for such subrectangles. Usng these nsghts, we can now wrte down pseudo-code for the global optmzaton algorthms for these geometrc problems (for brevty, we use the term regon nstead of hyper-rectangle n parameter space ) ; ths s shown n Fgure 6. Actual code for ths problem s, n fact, very close to def globally_optmze(regon,ponts): q = objectve_functon(regon,ponts) queue.enqueue(q,regon,ponts) whle not queue.s_empty(): (q,regon,ponts) = queue.dequeue_max() f accurate_enough(regon): return regon subregons = splt(regon) for subregon n subregons: subponts = [lst of pont n ponts f pont contrbutes to objectve_functon] subq = objectve_functon(subregon,subponts) queue.enqueue(subq,subregon,subponts) FIG. 6 Pseudo-code for fndng globally optmal solutons to the maxmum lkelhood problems descrbed n the text. the pseudo-code shown n Fgure 6. A queston that remans about these algorthms s how ef-

8 fcent they are. Commonly used methods for analyzng the performance of geometrc and numercal algorthms determne asymptotc complextes n terms of elementary arthmetc operatons. Such analyses are both dffcult and not very useful for these classes of algorthms. They are dffcult because the complexty of these algorthms depends on the desred accuracy of the computed results. Increasng the desred accuracy, the asymptotc complexty of the algorthm can be shown to be lnear n the number of solutons and the nput, but n practce, the algorthms are usually used wth a fxed, fnte accuracy dctated by the problem requrements. Furthermore, the problem nstances themselves are lmted n complexty : snce they are derved from scanned documents and there are physcal and perceptual constrants on the sze and complexty of such documents, asymptotc complexty s of less nterest than actual complexty on the range of problem complextes found n real-world applcatons. Havng sad that, we fnd average runnng tmes of approxmately 1 second for the text lne fnder (fndng all text lnes n a document mage) and less than 0.25 seconds for the whte space cover on modern PC hardware on the documents n the Unversty of Washngton Database 3 (UW3), wth approxmately lnear scalng of runnng tmes n terms of the number of connected components over the range of document complextes found n the UW3 database. 6 Dscusson Ths paper has revewed well-defned models of the spatal statstcs of document layouts and algorthms for fndng maxmum lkelhood solutons under such models. These approaches to layout analyss dffers from prevous methods for fndng geometrc layout prmtves n a number of mportant ways. Frst, they are guaranteed to fnd globally optmal solutons under the specfc geometrc and statstcal model chosen. That makes ther results reproducble and determnstc ; f the algorthm fals to come up wth a correct soluton, we know that ths must be a problem wth the statstcal model tself, no smply a falure of the algorthm to fnd the optmal soluton to the model. Emprcally, ths smplfes debuggng and parameter estmaton for layout analyss systems based on these methods. Second, the algorthms are expressed n terms of welldefned models of spatal statstcs and errors ; the dstrbutonal parameters (error bounds, frequency of background features) can be estmated from sample mages and have ntutve nterpretatons. At the same tme, the models also ncorporate strong pror knowledge about the geometry of document mages (e.g., the fact that text lnes are usually straght). Ths s n contrast to other well-founded statstcal models of page layout proposed recently [LIA 99, LIA 01] ; those models also take advantage of statstcal constrants, but they rely on more general statstcs of the relatve spatal dstrbutons of layout elements, makng both the parameter estmaton and the nference problem consderably harder. It s also n contrast wth projecton-based methods for layout analyss [BAI 87, NAG 84], whose crtera for determnng the locaton of text lnes or paragraphs are generally not drectly related to spatal statstcs of the locaton of characters on the page. Thrd, the methods presented here can operate locally ; that s, ndvdual text lnes and column separators can be found ndependently of each other. Ths allows us to apply the methods both to novel document types (e.g., documents wth text at varous dfferent orentatons) and to standard document types captured n novel ways (e.g., document capture wth hand-held cameras, resultng n perspectve dstorton). In contrast, many commonly used methods for geometrc layout analyss rely on global layout propertes. For example, methods lke projecton-based text lne fndng and layout analyss generally assume rectlnear layouts and the ablty to fnd a sngle, global page rotaton that wll deskew the page. There are some methods for geometrc layout analyss that do not make such global assumptons, such as the Vorono-based technque of [KIS 98] or closely related methods based on lnkng nearby connected components. However, whle they are able, n prncple, to operate on pages that have been perspectvely dstorted or contan text at varous dfferent orentatons, those methods rely on much weaker geometrc models of text lnes than the methods presented n ths paper. That suggests that they wll be more susceptble to false postve errors n the detecton of text lnes ; however, ths remans to be demonstrated on actual datasets. In ths short revew, we have focused on motvatng partcular models of spatal statstcs for layout analyss and outlned an approach to nference based on branch-and-bound methods that makes usng such models feasble for real document analyss problems. For detaled performance data n specfc applcatons, the reader s referred to the lterature. For data on the accuracy of text lne fndng and deskewng usng these methods, [BRE 02c] contans an evaluaton on the UW2 database. For data on the relablty wth whch the method fnds column separators, the reader s referred to [BRE 02a]. Results on non-axs algned column separator dentfcaton can be found n [BRE 03a]. More complete document analyss systems takng advantage of the novel capabltes of these methods have also been descrbed, ncludng a system for mage-based reflowng of document mages [BRE 02b]. However, the reader should keep n mnd that these methods can also be used as a drop-n replacement for commonly used exstng methods n document analyss, often yeldng some mprovement n performance, relablty, or functonalty over exstng methods. For example, the text lne fnder descrbed above can be used n place of projecton-based text lne fnders for relable skew estmaton and text lne dentfcaton. And the maxmal empty rectangle methods outlned above can be used as a drop-n replacement for the whtespace cover approach descrbed by [BAI 94, BAI 90]. Future work on these algorthms dvdes nto fundamental algorthmc mprovements on the one hand, and the exploraton of addtonal applcaton areas on the other. On the algorthmc sde, one open problem s the applcaton of these optmzaton methods to hgher-dmensonal parameter spaces. Currently parameter spaces beyond four or fve dmensons lead to mpractcally long computaton tmes. However, our lab s currently examnng methods for speedng up search that may allow us to extend that lmt for

9 many knds of objectve functons. We are also examnng other approaches to copng wth problems such as matchng smoothly curved text lnes (as they occur, for example, on photographc mages of curved book pages). On the applcaton sde, one of the most mportant areas s the use of these methods wth non-latn scrpts, languages, and page layouts. Furthermore, a detaled performance evaluaton and comparson between other recently proposed layout analyss technques [LIA 99, LIA 01, KIS 98] and these methods remans to be done. In order to facltate more wdespread adopton of these methods, we are makng avalable sample mplementatons for research purposes ; please contact the author for more nformaton. Références [BAI 87] BAIRD H. S., The Skew Angle of Prnted Documents, Proc., 1987 Conf. of the Socety of Photographc Scentsts and Engneers, Rochester, New York, [BAI 90] BAIRD H. S., JONES S. E., FORTUNE S. J., Image Segmentaton by Shape-Drected Covers, Proceedngs of the Tenth Internatonal Conference on Pattern Recognton, Atlantc Cty, New Jersey, 1990, pp [BAI 94] BAIRD H. S., Background Structure n Document Images, H. Bunke, P. S. P. Wang, & H. S. Bard (Eds.), Document Image Analyss, World Scentfc, Sngapore, 1994, pp [BRE 92] BREUEL T. M., Fast Recognton usng Adaptve Subdvsons of Transformaton Space, Proceedngs IEEE Conf. on Computer Vson and Pattern Recognton, 1992, pp [BRE 02a] BREUEL T. M., Two Algorthms for Geometrc Layout Analyss, Proceedngs of the Workshop on Document Analyss Systems, Prnceton, NJ, USA, [BRE 02b] BREUEL T. M., JANSSEN W. C., POPAT K., BAIRD H. S., Paper-to-PDA, Proceedngs of the Internatonal Conference on Pattern Recognton (ICPR 02), Quebec Cty, Quebec, Canada, [BRE 02c] BREUEL T., Robust Least Square Baselne Fndng usng a Branch and Bound Algorthm, Proceedngs of the SPIE - The Internatonal Socety for Optcal Engneerng, 2002, page (n press). [BRE 03a] BREUEL T. M., An Algorthm for Fndng Maxmal Whtespace Rectangles at Arbtrary Orentatons, Internatonal Conference on Document Analyss and Recognton, [BRE 03b] BREUEL T. M., On the Use of Interval Arthmetc n Geometrc Branch-and-Bound Algorthms, Pattern Recognton Letters,, [CAT 98] CATTONI R., COIANIZ T., MESSELODI S., MO- DENA C. M., Geometrc Layout Analyss Technques for Document Image Understandng : a Revew, rapport, 1998, IRST, Trento, Italy. [CHE 95] CHEN S., Document Layout Analyss Usng Recursve Morphologcal Transforms, PhD thess, Ph.D. thess, Unv. of Washngton, [HAN 80] HANSEN E., Global optmzaton usng nterval analyss the mult-dmensonal case, Numersche Mathematk, vol. 34, 1980, pp [HAR 94] HARALICK R. M., Document Image Understandng : Geometrc and Logcal Layout, CVPR94 : IEEE Computer Socety Conference on Computer Vson and Pattern Recognton, 1994, pp [HUB 81] HUBER P., Robust Statstcs, Sons, New York, John Wley and [III 97] III W. W., Statstcal approaches to feature-based object recognton., Internatonal Journal of Computer Vson, vol. 21, n o 1/2, 1997, pp [JAU 01] JAULIN L., KIEFFER M., DIDRIT O., WALTER E., Appled Interval Analyss, Sprnger Verlag, Berln, [KIS 98] KISE K., SATO A., IWATA M., Segmentaton of page mages usng the area Vorono Dagram, Computer Vson and Image Understandng, vol. 70, n o 3, 1998, pp [LIA 99] LIANG J., PHILLIPS I., HARALICK R., A Unfed Methodology for Document Structure Analyss, Workshop on Document Layout Interpretaton and ts Applcatons (DLIA), [LIA 01] LIANG J., PHILIPS I. T., HARALICK R. M., An Optmzaton Methodology for Document Structure Extracton on Latn Character Documents, Pattern Analyss and Machne Intellgence,, 2001, pp [NAG 84] NAGY G., SETH S., Herarchcal Representaton Of Optcally Scanned Documents, 7ICPR, vol. 84, 1984, pp [O G 93] O GORMAN L., The Document Spectrum for Page Layout Analyss, IEEE Transactons on Pattern Analyss and Machne Intellgence, vol. 15, n o 11, 1993, pp [OKU 99] OKUN O., PIETIKAINEN M., SAUVOLA J., Robust document skew detecton based on lne extracton, Proc. of the 11th Scandnavan Conference on Image Analyss (SCIA 99), June 7-11, Kangerlussuaq, Greenland, 1999, pp [SRI 89] SRIHARI S., GOVINDARAJU V., Analyss of textual mages usng the Hough transform, Machne Vson and Applcatons, vol. 2, 1989.