Text Line Segmentation Based on Morphology and Histogram Projection

Transcription

1 th Iteratioal Coferece o Documet Aalsis ad Recogitio Tet Lie Segmetatio Based o Morpholog ad Histogram Projectio Rodolfo P. dos Satos, Gabriela S. Clemete, Tsag Ig Re ad George D.C. Calvalcati Ceter of Iformatics, Federal Uiversit of Perambuco Recife, PE, Brazil - {rps2,gsc2,tir,gdcc}@ci.ufpe.br Abstract Tet etractio is a importat phase i documet recogitio sstems. I order to segmet tet from a page documet it is ecessar to detect all the possible mauscript tet regios. I this article we propose a efficiet algorithm to segmet hadwritte tet lies. The tet lie algorithm uses a morphological operator to obtai the features of the images. Followig, a sequece of histogram projectio ad recover is proposed to obtai the lie segmeted regio of the tet. First, a Y histogram projectio is performed which results i the tet lies positios. To divide the lies i differet regios a threshold is applied. After that, aother threshold is used to elimiate false lies. These procedures, however, cause some loss o the tet lie area. So, a recover method is proposed to miimize this effect. I order to detect the etreme positios of the tet i the horizotal directio, a X histogram projectio is applied. The, as i the Y directio, aother threshold is used to elimiate false words. Fiall, i order to optimize the area of the mauscript tet lie, a tet selectio is carried out. Eperimetal results usig the IAM-database showed that this ew approach is robust, fast ad produces ver good score rates. 1. Itroductio Tet lie etractio or segmetatio is a importat problem that does ot have a uiversal accepted solutio i the cotet of automatic hadwritte documet recogitio sstems [1]. Tet characteristics ca var i fot, size, shape, stle, orietatio, aligmet, teture, color, cotrast ad backgroud iformatio. These variatios tur the process of word detectio comple ad difficult [2]. I the case of hadwritte mauscripts, differetl from machie prited, the compleit of the problem eve icreases. Sice hadwritte tet ca var greatl depedig o the user skill, dispositio ad eve cultural backgroud. Here, we preset a method to segmet tet lies based o morpholog ad histogram projectio. Morphological operatios are used to produce a biar image. This procedure was first proposed b Wu et. al. [3], as a iitial step i the process of tet lie etractio from video images cotaiig tet iformatio. I their applicatio, a ot precise bo cotaiig the regio of the tet is used as output of the sstem to idetif machie prited tet i differet video cotets. We have adapted ad improved this idea for hadwritte tet lie segmetatio problem. A importat fact i relatio to image aalsis based o cotrast is that this characteristic is robust i relatio to chages i illumiatio ad it is ivariat to differet image trasformatios such as scalig, traslatio ad skewig. Oce the page documet has bee preprocessed, a techique based o projectio profiles is applied. Projectio profiles are commol used for prited documet segmetatio ad ca also be adapted to hadwritte documets [1]. I the work of Mariai ad esi [4], the projectio curves are used to segmet music sheets i order to etract the basic smbols ad their positios. The segmetatio approach proposed is divided i 3 levels ad utilizes projectio profiles alog the Y ad X aes alteratel. Mamatha ad Rothfeder [5] used projectio profiles i the horizotal directio to segmet words of historical hadwritte documets durig the lie segmetatio stage. I this work, a projectio profile i the horizotal directio is iitiall applied to obtai the tet lies positios. Some improvemets were ecessar i this procedure to correctl idetif the lie segmets, so a recover process is also developed. A similar process is used to obtai the word borders of a lie usig projectio profiles i the vertical directio. We refer to projectio profile as histogram projectio. Eperimets are performed o hadwritte documets radoml selected from the IAM-database [6], showig that the proposed techique produces ecouragig results. A /09 $ IEEE DOI /ICDAR

2 aalsis based o a performace evaluatio method described i [7] is carried out. This work is orgaized as follows. I Sectio 2, the proposed method ad all its stages are described. I Sectio 3, the eperimetal results are aalzed ad discussed. Fiall, i Sectio 4, some discussio ad cocludig remarks are preseted. 2. Proposed Method I this sectio we propose a ew method to automaticall idetif ad segmet the tet lie regios of a hadwritte documet. The sstem cosists of eight stages, as show i Figure 1 ad described below. The feature etractio or biarizatio step is applied to the iput image, resultig i Figure 2b. The, a Y histogram projectio (Fig. 2c) is obtaied to detect the possible lies. Due to some oises, a tet lie separatio (Fig. 2d) is ecessar. Oce the false lies are foud, the must be ecluded. After that, the lie regio recover step (Fig. 2e) is performed i order to recover some losses itroduced b the precedig step. A X histogram projectio that is applied to each lie detected (Fig. 2f) takes out possible false words, mail at the lateral edges of the page. Fiall, we obtai the tet lies regio (Fig. 2g). (c) (d) Iput Image Feature Etractio Y Histogram Projectio Tet Lie Separatio False Lie Eclusio Tet Selectio False Word Eclusio X Histogram Projectio Lie Regio Recover Figure 1. Flowchart of the proposed sstem, showig the eight stages of the procedure. (e) (f) 2.1. Biarizatio Tet i hadwritte documet image, most of the times, have high cotrast i relatio to the backgroud of the images. The widths ad heights of the tet lies are relativel uiform ad have a horizotal directio. The feature etractio used here is the method proposed b Wu et. al. [3], which is based o morphological operators. The origial idea of this techique was to etract high cotrast regios from a video image, where there is a lot of iformatio i the backgroud. The authors mai objective was to be able to localize tet regio from video image i a real outdoors eviromet. Figure 3 shows the flowchart of this operatio alog with the morphological operatios used to obtai the desired results. (g) Figure 2. Itermediate stages: Iput image, Feature etractio step, (c) Y histogram projectio with itersectio betwee letters highlighted, (d)tet lie separatio with a false lie highlighted, (e) Lie regio recover, (f) X histogram projectio with a false word highlighted ad (g)tet selectio. 652

3 Figure 3. Flowchart of feature etractio stage. I equatios 1-5, I(, deotes the gra level value of the piel located at positio (, ad S m, is the structural elemet of size m where m ad are odd values lager tha zero. Figure 4 shows two kids of structural elemets used i mathematical morpholog operatios. S 1, Iput Image Average (1) Closig (2) Opeig (3) Diferece (4) Closig Biarizatio (5) Labelig Feature Etractio m/2 /2 1 ES m ( I(, ) I( + i, + j) Sm, ( i, j) (1) m i m/2 i /2 I (, S m, ( I (, S m, ) ΘS m, I (, Sm, ( I(, ΘSm, ) Sm, D( I1, I 2 ) I1(, I 2 (, 255, if ( I (, > T; T ( I(, ) 0, otherwise S 3,7 Figure 4. Structure elemets masks used b the morphological operators. To obtai the biarizatio resultig output, first a smoothig operatio is used b applig a average filter that have a widow size of 3 piels, which is defied as a structure elemet S 3,3 i mathematical morpholog cotet. The et operatios are closig ad opeig usig a structural elemet S 1,20. The (2) (3) (4) (5) differece obtaied from subtractig both images is the result of the followig step. The, aother closig operatio is performed with a structural elemet S 5,5 to tur the border of the resultig image more compact ad closer. Afterwards, a threshold procedure is applied followed b a labelig process to etract the tet segmets. I the threshold procedure a parameter T is defied damicall accordig to the backgroud of the image. This parameter is resposible to determie the limit value of the biarizatio operatio. A eample of the result of this process is show i Figure 2b Y Histogram Projectio Oce the feature etractio of the images is performed, the Y histogram projectio of the whole image is obtaied. The idea is to use a simple ad fast method to correctl distiguish possible lie segmets i the hadwritte tet. I Figure 2c it is clear that each tet lie correspods to a peak i the histogram. The histogram represets the added piels for each value. So the empt spaces betwee the peaks represet possible regios betwee differet tet lies Tet Lie Separatio Oce all the potetial lies are detected, a procedure to appl a threshold is performed to obtai a possible lie separatio i the tet. This threshold is damicall calculated ad it is proportioal to the average legth of the lies i the tet (Y histogram values). This process applied to the histogram aims to remove the regios i the histogram that are ot referred to the lies i the tet, or the elimiatio of oises that cofuses with the tet lies. The choice of the parameter to be used as threshold is itrisic related to the iformatio i the tet, so that the algorithm utilized the miimum possible of heuristic techiques to determie the lie separatio poits. Actuall, this stage tries to idetif the locatio of each tet lie. The separatio of the possible tet lies regios usig the histogram shows a difficult due to the upper ad lower regios of some letters as show i Figure 5. Figure 5: Regios that provoke false lies. 653

4 The red marked areas i Figure 2c shows a detailed view of these regios i the histogram. To separate the tet lies, we eclude part of the histogram based o the average Y histogram values. However, b doig so, part of the letters is lost sice the are located i the regio of the eclusio. To solve this problem a recover method is also proposed. As a result of the stage described, we obtaied the ew Y histogram projectio, show i Figure 2d. ote that the peaks are clearl separated b empt regios compared to Figure 2c False Lie Eclusio This procedure tries to eclude possible oises close to the tet lies regios. Oce the possible tet lie regios are separated b removig a offset from the histogram, we determie the average height of these regios to eclude false lies that might be detected. I Figure 2d we ca observe this effect, a small peak i the histogram show i red ellipse. If this regio poses eough height it ca be cofused with a tet lie segmet b the algorithm. The height of a lie is obtaied b takig the limit values of the correspodig regio i the Y histogram ad calculatig the differece betwee them. The equatio bellow provides the average height of the lies foud i a page: iitial p fial, (6) where iitial is the positio where the tet regio begis, fial is the positio where the tet regio eds ad p is the umber of regios foud i the page. The lies with height below a pre-determied threshold are removed. The value of this threshold is proportioal to the average height of the tet lies i the whole image Lie Regio Recover This procedure determies the average poit betwee the regios foud. The idea is to fid the maimum area that each lie might be iscribed, b determiig the superior ad iferior coordiates i the ais. Figure 2e shows the limits of these regios after the eclusio threshold is applied. The dashed lies are the limits betwee two adjacet lie regios. I this wa, the ecluded regios are recovered. ote that the limit lies establish the maimum ad miimum coordiates for each tet lie X Histogram Projectio A histogram was also projected i the X directio for each lie segmeted. This procedure determies the word positios i the tet lie. The result of this stage is crucial to detect oise (or false words) i the image False Word Eclusio The X projectio is also used to remove regio oise, i the same wa that it is described i sectio 2.4. Figure 2f shows this effect, the red regio shows a false word regio that should be ecluded. I this wa we determie the etreme poits of each setece or the word border poits (right ad left). Besides that, for the precise determiatio of the maimum border poits, four times the maimum word width amog the words ecluded is added. This value was obtaied through several eperimets Tet Selectio Oce idetified the coordiates that delimits the possible tet regio, a algorithm is applied to fid out the smallest regio bo that iscribes that tet regio. The procedure is similar to the algorithm used i [3]. It cosists i optimizig the dimesios of the rectagle that ecompasses the tet lie. This regio is defied as the fial segmeted tet area ad it should ot cross over parts of the word or have larger dimesios tha the lie. Figure 2g shows a eample of this procedure ad the fial result of the segmetatio process. 3. Eperimetal Results The eperimets of the proposed method are doe usig 150 images (total umber of tet lies 1353) radoml chose from the IAM hadwritte database. This public available database [6] provides images with 300 dpi, ad 256 gra levels i a tiff graphic format. The database also provides the iformatio about the segmeted lie regios, so we compared our results with the oe provided b the database. Figure 6 shows two eamples of the tet lie segmeted regios obtaied b the proposed method. ote that i Figure 6a all the segmets were correctl idetified. However, i Figure 6b we show a case where the method still fails because lie 3 is a false lie. However this kid of problems occurred i less the 1,2 % of the total processed tet lies. 654

5 false _ alarms FalseAlarmRate M Usig a set of groud-truth with 1353 lies i our eperimets, the false alarm rate is equal to 15/ This meas that ol 15 false lies are detected. 4. Coclusios ad Discussios Figure 6. Eemple of two image documets ad the segmeted regio obtaied b the algorithm. To evaluate the performace of the procedure we compared each segmeted lie positio, that is the, of the leftmost positio, the height (h) ad legth (w) of the bo, to the oes iformed b the database. We obtaied agreemet i piel positio level with a ver small deviatio. For eample i Figure 6 lies 3 to 7 all the piels positio coicides. Ad for lies 0 to 2, a disagreemet of less the 5 piels was obtaied for all variables,, w ad h. The performace measuremet of the algorithm was evaluated accordig to the method described i [7]. Table 1 shows the detectio rate ad the missed detectio rate, usig the formula provide b equatios below: oe2oe g _ oe2ma DetectioRate w1 + w2 + w3 misses MissedDete ctiorate g _ ma2oe Table 1. Results (Total lies 1353) Acceptace Threshold Detectio Rate Missed Detectio Rate The false alarm rate was obtaied b: I coclusio, we preseted a ovel algorithm for tet lie segmetatio. The procedure proved to be fast ad ver accurate usig the well-kow IAM database. The results, eve though we used a small part of all the images cotaied i the database, showed to be ver ecouragig. Tet lie segmetatio is just the first phase for the solutio of a automatic hadwritte documet recogitio sstem. The et step of the problem is to segmet each word of the tet lie ad the each letter of the word. We believe that this procedure ca be achieved b usig the same kid of ideas used here i idetifig the tet lie. We are curretl ivestigatig this solutio. Refereces [1] L. Likforma-Sulem, A. Zahour, B. Tacoet, Tet lie segmetatio of historical documets: a surve, Iteratioal Joural o Documet Aalsis ad Recogitio, 2007, pp [2] K. Juga, K.I. Kimb, A.K. Jai, Tet iformatio etractio i images ad video: a surve, Patter Recogitio, 2004, pp [3] J.C. Wu, J.W. Hsieh, Y.S. Che, Morpholog-based tet lie etractio, Machie Visio ad Applicatios, 2008, pp [4] S. Mariai, P. esi, Projectio Based Segmetatio of Musical Sheets, Documet Aalsis ad Recogitio, ICDAR 1999, pp [5] R. Mamatha, J.L., Rothfeder, A scale space approach for automaticall segmetig words from historical hadwritte documets, IEEE Tras. Patter Aal. Mach. Itell., 2005, pp [6] U.V. Marti, H. Buke, The IAM-database: a Eglish setece database for offlie hadwritig recogitio, Iteratioal Joural o Documet Aalsis ad Recogitio, 2002, pp [7] B. Gatos, A. Atoacopoulos,. Stamatopoulos, Hadwritig Segmetatio Cotest, Documet Aalsis ad Recogitio, ICDAR 2007, pp