Handwriting Stroke Extraction Using a New XYTC Transform

Transcription

1 Hadwritig Stroke Etractio Usig a New XYTC Trasform Gilles F. Houle 1, Kateria Bliova 1 ad M. Shridhar 1 Computer Scieces Corporatio Uiversity Michiga-Dearbor Abstract: The fudametal represetatio of hadwritig is a quasi-cotiguous set of strokes. I offlie settigs hadwritig stroke etractio is complicated by adjacet ad overlappig strokes. This paper describes a recostructio of the stroke iformatio usig a ewly developed XYTC trasform. This represetatio is very appropriate for hadwritig feature etractio, stroke segmetatio, uderlie removal, cross-out detectio, ad word recogitio. Applicatios related to fiacial documet processig are described. 1. Itroductio This paper presets a ew approach to stroke etractio for applicatios such as frame removal i forms, uderlie removal i legal amout images, stroke features for character recogitio, ad cursive word segmetatio. There has bee a umber of papers published [1-4] for specific applicatios such as detectio ad elimiatio of cross-outs, elimiatio of uderlies i word images, lie detectio for forms processig, etc. The Freema chai code is a popular represetatio for coected compoets i a biary image. Statistical measure of this iformatio over a small eighborhood ca be used as discrimiatig features. For istace histogram of the eight directios (or folded four orietatios i sub-regios of a character has bee show to yield accurate character recogitio [5]. If this iformatio is supplemeted by taget ad curvature iformatio (t, c at every boudary poit, the we ca recostruct the image aroud a local poit (, y usig the (t, c estimates. I order to get a accurate estimate of the taget vector at a poit alog the cotour some itegratio i the viciity of each cotour poit is required. For hadwritig, the variatio of the taget leads to curvature, which is appropriate to represet strokes. I the case of hadwritig we ca begi recostructio of the stroke by joiig cotour poits o each side of a stroke if their respective curvature ad taget values are opposite. Aother method of etractig the stroke is by creatig a skeleto to represet the ceter of the stroke [6,7]. I this paper we propose to covert the Freema chai code to a XYTC represetatio. I the XYTC represetatio, ay poit o the image boudary is represeted by its (, y coordiates as well as the value of the taget ad curvature (t, c at that poit. The mai advatage of the XYTC trasform is the local ad global cotet at each cotour poit. Based o the XYTC trasform, a mathematical model is created to predict the trajectory of a stroke. I off-lie settigs the case of close ad overlappig strokes ca oly be hypothesized. Lack of pe pressure ad bad image quality will result i broke strokes. A miimum curvature chage is the basis of stroke predictio ad etractio. Sectio gives the details o how to create the XYTC trasform from a Freema chai code. From this represetatio Sectio 3 eplais how we defie XYTCcliques, which are the basic elemets to costruct meshes that effectively represet partial strokes. From the meshes we ca predict ad recostruct the complete strokes. Sectio 4 describes applicatios, which show the value of the proposed XYTC-based approach. Selected applicatios i the US ad Europea fiacial documet processig are discussed. Fially coclusios ad future work ca be foud i sectio 5.. XYTC Trasform Assume that the iitial represetatio of a image I is a Freema chai code with directio vectors varyig from 0 (0 degree, to 7 ( degrees. For each C coected compoet we have a startig poit ad the correspodig chai code that represets the cotour with a sequece of (, y poits. We defie I { C }, for 0, 1,... N-1, where N is the umber of coected compoets ad the set of coordiates deoted by C {(, y }, for m 0.. M -1, where M is the umber of data poits to completely eclose the coected compoet (figure 1. The cotour smoothess depeds o the piel resolutio, which is typically betwee 00 to 300 dpi. To Proceedigs of the Sith Iteratioal Coferece o Documet Aalysis ad Recogitio (ICDAR /01 $ IEEE 1

2 smooth the cotour we ca represet it by quadratic parametric fuctios: Cˆ {( ˆ, yˆ } {( ~ ( t, ~ y ( t}, [1] For m 0.. M -1, ad ~ ( t a b t c t, ad ~ y ( t a b t c t. y y From this represetatio we compute the taget vector at each poit by: Cˆ ( ( ~ m T ( m y ( t iy ~ ( t iy ( ( y Here i is the imagiary umber ad the dot otatio is used to represet derivative operator. We ca the compute the curvature vector at each poit by: κ T ( iy ( ( y ( y y ( y i ( ( y 4 If we take the orm of the curvature vector we fially get: κ ( y y 3 ( ( y I summary, oe ca efficietly trasform a Freema chai code ito a sequece of poits, each of which is represeted by the quad vector (, y, t, c, where (, y are the coordiates of a poit alog the cotour as defied by the chai code, ad (t, c are the estimated taget, ad curvature ( κ iformatio, respectively. This represetatio cotais local ad global iformatio. I other words, give a (, y, t, c vector, we ca etrapolate aroud a sigle poit, usig the taget ad curvature iformatio. 3. Creatio of Strokes Give a biary image of a hadwritig sample, the XYTC represetatio is used to progressively recostruct the strokes. The first step is to form a clique; a associatio with aliged ad parallel eighbors alog the cotour. A clique is a graph i which each ode is a XYTC poit, ad the edges coect to o more tha 5 eighbors as defied i figure. The clique is very similar to what was proposed by Suzuki ad Mori [7] i defiig the regular regios suitable for skeletoizatio. The cliques are the combied to form a mesh based o miimum curvature chages. The meshes represet partial strokes, from which iterpolatio allows to coect meshes ito strokes. 3.1 Clique Formatio Fudametally, a quasi-cotiuous movemet of the pe creates hadwritig. From a (, y, t, c represetatio we ca detect the o-ambiguous stroke segmets (i.e., o-overlappig strokes by fidig the parallel poits alog the cotour ad regroupig them ito partial strokes. We defie a clique (figure as a set of 6 poits (aliged ad parallel poits. χ k { α, α, α1 k, β, β, β1 k } where β {(, y } α, ij ij, ad α is parallel to β. At each poit a clique is created if it satisfies the followig coditios: All α are parallel to β (tagets ad curvature [3] are opposite The stroke thickess for all three poits { α, β } are comparable Noe of the poits are ear a ed poit. Noe of the poits are ear a juctio like a Y. [4] Figure 3 shows some eamples of clique-based images obtaied by paitig black lies betwee odes of the cliques foud alog the cotour. It should be clear from this image that cliques truly idetify uambiguous stroke segmets. Stroke itersectios appear white or dotted idicatig lack of cotiuity. Although or 4 adjacet poits may be cosidered [7], a 6 poit-clique reduces the ifluece of sigular poits (stroke eds or crossigs, which ca cause error i stroke recostructio. 3. Mesh Formatio Regroupig adjacet cliques forms a mesh. A mesh has a head ad a tail, which ca be ear either a ed poit or a juctio where strokes overlap. Near a juctio, a mesh cotais rich iformatio to allow etrapolatio to predict the stroke path assumig miimum curvature chages. Liear regressio of the XYTC poits is used to eted the mesh o both eds. We compute the ceter (, y of each clique the fit a fuctio of the type: F( t af bft cft where t0 is the predicted value ad t-1,-,-3, are the Proceedigs of the Sith Iteratioal Coferece o Documet Aalysis ad Recogitio (ICDAR /01 $ IEEE

3 previous ceter poits. We progressively grow a head or tail of a mesh as log as the predicted et poit is a black piel (i.e., part of a coected compoet. Figure 4 shows some meshes (the wider stem of the stroke ad the etrapolated stroke ceter (thi lie. The overlappig head ad tail of meshes with similar curvature are the joied to form more complete strokes as show i figure 4. I figure 4 a stroke is represeted by the same color. I the case of hadwritig samples, the strokes are ordered from left to right takig slat ito accout. This allows us to regroup icomplete strokes to form characters i a segmet-based recogitio scheme. 4. Applicatios Oe of the first applicatios is i hadlig double 00 s typically ecoutered i the cets portio of a courtesy amout o checks. As see i figure 5, stroke etractio provides a atural way to segmet hadwritig to assist recogitio. The gray color idicates overlaps of strokes. The horizotal lie is detected as oe log stroke ad removed to help legal lie recogitio. Aother eample is to detect ad remove a frame aroud a courtesy amout. 5. Coclusios ad Future Work This paper described a framework that allows for the recostructio of hadwritig strokes startig from a XYTC trasform. The XYTC poits are regrouped ito cliques, which are the merged to form meshes or partial strokes. The ceter of the cliques alog the mesh cotais XYTC iformatio, which ca be used to etrapolate the directio of a stroke at a juctio. The ed result is a sequece of costat curvature segmet that represets hadwritig, ad lie-based objects like boes. Other tha character segmetatio, other advatages of the stroke etractio based o XYTC are skew-idepedet ad o-straight lies detectio as foud o fiacial documets. Applicatios such as uderlie removal i legal lie recogitio, or bo removal i courtesy amout have bee demostrated. The approach described is curretly beig tested i cursive word segmetatio/recogitio. 6. Refereces 1. S. H. Kim, S. H. Jeog, H. K. Kwag, Lie Removal Ad Character Restoratio Usig Bag Represetatio of Form Images, Iteratioal Workshop Frotiers I Hadwritig Recogitio (IWFHR VII Sep Amsterdam, The Netherlads, pp. 43-5, T. Artieres, J-M. Marchad, P. Galliari, Stroke Level Modelig of O Lie Hadwritig Through Multi-Modal Segmetal Models, Iteratioal Workshop Frotiers I Hadwritig Recogitio (IWFHR VII Sep Amsterdam, The Netherlads, pp , S. Liag, M. Ahmadi ad M. Shridhar, Segmetatio of Hadwritte Iterferece Marks Usig Multiple Directioal Stroke Plaes ad Reformalized Morphological Approach, IEEE Tras. Image Processig, Vol. 6, August 1997, pp G.F. Houle, Hadwritte Word Segmetatio. Fudametals i Hadwritig Recogitio, Spriger-Verlag Berli Heidelbert 1994, pp F. Kimura ad M. Shridhar, Hadwritte Numeral Recogitio Based o Multiple Algorithms, Patter Recogitio, Vol. 4, No. 10, pp , T.Y. Zhag, ad C.Y. Sue, A Fast Parallel Algorithm for Thiig Digital Patters, ACM, vol. 7, No.3, 1997, pp T. Suzuki ad S. Mori, Structural Descriptio of Lie Images by The Cross Sectio Sequece Graph, Iteratioal Joural of Patter Recogitio ad Artificial Itelligece, vol. 7, o. 5, 1993, pp Proceedigs of the Sith Iteratioal Coferece o Documet Aalysis ad Recogitio (ICDAR /01 $ IEEE 3

4 C (, y Ĉ :smooth cotour dc d dy dc d dy Figure 1. Cotour defiitio α α α 1 k α 1k S:Stroke β β β 1 k Figure. Clique Defiitio a O rigial im age b Etracted clique-based im ages Figure 3. Cliques eam ples Proceedigs of the Sith Iteratioal Coferece o Documet Aalysis ad Recogitio (ICDAR /01 $ IEEE 4

5 a Origial image b Etracted strokes Figure 4. Etrapolatio-Stroke Recostructio a Origial image Courtesy amout $65- a Legal amout b Etracted strokes b Etracted strokes c Etracted mesh c Removed frame d Removed lie Figure 5. Eamples of amouts processig Proceedigs of the Sith Iteratioal Coferece o Documet Aalysis ad Recogitio (ICDAR /01 $ IEEE 5