Perspective rectification for mobile phone camera-based documents using a hybrid approach to vanishing point detection

Transcription

1 Perspectve rectcaton or moble phone camera-based documents usng a hbrd approach to vanshng pont detecton Xu-Cheng Yn Jun Sun Satosh ao Fujtsu R&D Center Co. td, Bejng, Chna {uchengn; sunjun; nao}@cn.ujtsu.com Yusau Fuj Katsuhto Fujmoto Fujtsu aboratores td, Kawasa, Japan {uj.usau; ujmoto.at}@jp.ujtsu.com Abstract Documents captured b a moble phone camera oten have perspectve dstortons. In ths paper, a ast and robust method or rectng such perspectve documents s presented. Prevous methods or perspectve document correcton are usuall based on vanshng pont detecton. However, most o these methods are ether slow or unstable. Our proposed method s ast and robust wth the ollowng eatures: (1) robust detecton o tet baselnes and character tlt orentatons b heurstc rules and statstcal analss; () quc detecton o vanshng pont canddates b clusterng and votng; and (3) precse and ecent detecton o the nal vanshng ponts usng a hbrd approach, whch combnes the results rom clusterng and projecton analss. Our method s evaluated wth more than 400 mages ncludng paper documents, sgnboards and posters. The mage acceptance rate s more than 98% wth an average speed o about 100ms. 1. Introducton Recentl, camera-based document analss becomes a hot research eld [6][10]. Wth wdespread usage o the cheap dgtal camera bult-n the moble phone (MobleCam n abbrevaton thereater) n people s dal le, the demand or smple, nstantaneous capture o document mages emerges. Derent rom the tradtonal scanned mage, lots o the MobleCam-based document mages have perspectve dstortons (see Fg. 6(a)(b)). Consequentl, rectng MobleCam-based perspectve document mages becomes an mportant ssue. As a general computer vson problem, most perspectve correcton methods rel on vanshng pont detecton. And these methods nvolve etractng multple lnes and ther ntersectons, or usng teture and requenc nowledge [4][11]. In document analss, there are also varous wors on correcton o perspectve documents captured b general dgtal cameras [][3][7][8][9][1][13]. Man o these methods use document boundares and tet lnes to vote and detect vanshng ponts. And other methods tae advantage o normaton o tet lnes and character tlt orentatons. We dvded the methods o vanshng pont detecton nto two sub groups: drect approaches and ndrect approaches. The drect approaches drectl analze and calculate on mage pels, such as projecton analss rom a perspectve vew or horzontal vanshng pont detecton []. These approaches have rather good precsons. But the search space n such approaches s an nnte D space, and a ull or partal search o the space s computatonall epensve, even mpossble. The ndrect approaches convert the orgnal space nto a clue space, and search the vanshng pont n that new small space. Most o the ndrect approaches nvolve etractng multple straght or llusor lnes 1 and votng vanshng ponts b model ttng. To calculate the horzontal vanshng pont, some methods eplctl t a lne bundle n the lnear clue eature space [8][9]. Some researchers use the spacng between adjacent horzontal lnes o tet to vote the vertcal vanshng pont [][8]. u et al. use character stroe boundares and tp ponts to correct perspectve dstortons based on multple uzz sets and morphologcal operators [7].ang et al. use the equal tet lne spacng propert to calculate and vote vanshng ponts, and the suggest ther method can be appled on moble devces [1][13]. These ndrect approaches are tme ecent. However, the model ttng n these methods are senstve. Moreover, n MobleCam-based document analss, the captured mages are alwas n low resoluton wth blurrng, and the captured tet contans oten a partal porton o the whole document. In concluson, there are two man challenges or rectng MobleCam-based perspectve documents. Frst, the rectng engne should be hghl precse wth ast 1 The straght and llusor lnes used n ths paper are smlar to the hard and llusor lnes respectvel termed n [9]. 37

2 speed. The above methods can t cover the two ssues well at the same tme. Second, a MobleCam-based mage s usuall a partal porton o a whole document wth ew document boundares. But man tradtonal methods manl rel on document boundares or correcton. Thereore, the tradtonal drect and ndrect approaches have lmtatons or practcal stuatons. To solve the above problems amng at a practcal MobleCam applcaton, we ocus on a ast and robust method or rectng general perspectve documents. Frst, we propose a hbrd approach or robust real-tme vanshng pont detecton b ntegratng the drect and ndrect approaches ecentl. As or the second challenge, we utlze horzontal tet baselnes and character tlt orentatons as llusor lnes to vote and compute vanshng ponts. Snce the character stroes are used n lne detecton, paper boundares are not necessar. Our rectng method s descrbed n Fg. 1, where preprocessng ncludes grascale mage converson, thresholdng, and edge calculaton, et al. The straght lnes, horzontal tet baselnes, and character vertcal stroes all are etracted b connected component analss, heurstc rules and statstcal analss. Then, the hbrd approach clusters and detects the horzontal and vertcal vanshng ponts respectvel. Fg. 1. The lowchart o our method or perspectve rectcaton. The man contrbutons o ths paper are as ollows. The rst contrbuton s a hbrd approach to vanshng pont detecton, whch combnes a greed ndrect method wth a hgh-qualt pel-based drect method. The ast ndrect method gves a set o canddates, whch are rened aterwards usng a drect approach. And the decson s made b lnearl combnng these two methods. The second contrbuton s the clusterng learnng or vanshng pont canddates n the ndrect method, whch ast selects potental vanshng pont canddates n the ntersecton pont dstrbuton rom all straght and llusor lnes. The thrd contrbuton s a robust and ast perspectve dstorton mage recter, whch s a worng sstem or applcatons o moble phone camera-based document mages. The remander o ths paper s organzed as ollows. Secton ntroduces the basc prncple o our method. Secton 3 eplans how to etract the lnes and the stroes n detal. In Secton 4, we show the strateg o vanshng pont detecton. Secton 5 s the eperments and result analss. Fnall we conclude the paper n Secton 6.. Basc prncple or vanshng pont detecton The vanshng pont (horzontal or vertcal) n a D space can be descrbed as (v, v ), where v and v are the X and Y coordnates respectvel n the D Eucldean space, {(,, }, (1) where (0,0) s the center pont o the mage. In general, the vanshng pont does not le wthn the mage tsel. Generall speang, vanshng pont detecton s to nd a proper pont accordng to an optmzaton process n the mage plane. That s to sa, ( v, v ) arg ma, where ( s the prot uncton or the optmzaton. For the drect approaches or vanshng pont detecton, the search space s (equaton (1)). Obvousl, search n such a space s computatonall epensve. In ths paper, we propose a novel and hbrd approach or vanshng pont detecton. Our approach rst votes and clusters lne ntersectons nto vanshng pont canddates. Ths process belongs to an ndrect approach. Then projecton analss rom perspectve vews on these canddates s perormed, whch s a drect approach. The vanshng pont s obtaned b an optmzaton o a uncton based on the prevous two steps. The uncton can be epressed as ollowng: g G( ndrect(, drect( ), () where ndrect ( and drect ( are the prot unctons or the ndrect and drect approaches respectvel. For vanshng pont detecton, rst, we locate all straght and llusor lnes. Then calculate all ntersectons or ever lne par. The set o the resultng ntersecton ponts s Set( Pt) {( 1, 1),( ),...,(, )}, PT PT where PT s the number o ntersectons. These ponts are parttoned nto several groups b a clusterng algorthm. Thereore, we get a new space S, S { S S, 1,..., }. The center pont o each cluster s regarded as a tpcal representaton o ts sub regon, whch s C { c c S, 1,..., }. (3) 38

3 Rather than searchng on the whole space n, we search on the representatve pont set n C or speed up. Snce the pont set n C s representatve enough, the searched mamum n C s a good appromaton to the global mamum. Consequentl, the nal resultng vanshng pont s gven b ( v, v ) argma g(. C where C s dened n Equaton (3). ow, we perorm a drect approach (e.g., projecton analss rom a perspectve vew) on the new search space. Compared C n equaton (3) wth n equaton (1), the search space o our hbrd approach s just composed b several ponts, whch s much smaller than that o the drect approaches. Hence, t s tme ecent. The above dea s used n horzontal and vertcal vanshng pont detecton. I the number o detected lnes s enough, sucent lne ntersectons wll be generated. And the true vanshng pont wll be embedded n these ntersectons wth a hgh probablt. Derent rom the conventonal methods, our method nds the lne canddates not onl b paper boundares lnes but also b tet baselnes and the nature-born long character stroes n the tet content. Thereore, the robustness o our method s mproved. 3. ne and stroe detecton and selecton When a document s lned up n a horzontal drecton, we call t a horzontal document. Each tet row has a clue horzontal drecton. Our method uses document boundares and tet rows to detect the horzontal vanshng pont. However, n the vertcal drecton o a horzontal document, there wll be no tet columns or vertcal clues. Smlar wth the method descrbed n [7], we etract the vertcal character stroes as llusor vertcal lnes to detect a stable vertcal vanshng pont. In Secton 3.1, straght lne detectn ncludes document boundares and other straght lnes. And tet baselne detecton s descrbed n Secton 3.. Character tlt orentatons are detected n Secton Straght lne detecton ne detecton s solved wth well-nown conventonal technques. A lot o related wor or such geometrc problems s proposed, such as RASAC-based methods, east-square-based methods, and Hough transorm-based methods [14]. In ths paper, n order to perorm n a more ecent wa, our lne detecton algorthms are based on edge normaton, connected component analss, heurstc rules, and statstcal analss. Frst, the nput mage s down-sampled. Then the edge s etracted b Cann edge detector [1]. Connected component analss s used to nd long connected components, whch are merged n the horzontal or vertcal drecton accordng to shape and sze normaton. The merged connected components are regarded as lne canddates. Gven a connected component C, ts correspondng lne, C, s tted b the east-square algorthm. The dstance rom one pont ( n C to the tted lne s a b c DIS. a b And we can get 1 en( C ) len _ thres, C n _ thres _ lne ( ) C 0 otherwse, where ( DIS,, ), and PC ( lne lne ) 1 P (, ) (, C p thres lne I C 0 otherwse, C I C C. In the above equaton, en(c ) s the length o C, and, ) s a Gaussan dstrbuton or C wth mean and standard devaton. And lne and have been lne determned epermentall rom derent mages. I (C ) s equal to 1, then C wll be a straght lne. Horzontal and vertcal lne detecton and selecton can be perormed b the above steps respectvel. 3.. Horzontal tet lne smearng and detecton Ths process s based on a bnarzed mage. We use a Bloc-Otsu algorthm or mage bnarzaton. Ater connected component analss on the bnar mage, character canddates are selected b component sze and shape analss. Then, the are merged nto horzontal tet lnes b a smearng algorthm derved rom [5]. Fnall, the horzontal drecton o each smeared tet lne s computed. The above procedure sometmes wll produce smeared blocs that nclude more than one horzontal tet lnes because o perspectve dstortons. Thereore, we use a robust lne drecton detecton method whch s descrbed n ollowng. Frst, we estmate the shape and sze o each smeared tet lnes. Through vertcal projecton, we can obtan the upper and lower contour ponts o each smeared tet lne respectvel. The upper contour ponts are {( 1, 1 ),( ),...,(, )}, where s the wdth o the smeared tet lne. The lower contour ponts are {( 1, 1 ),( ),...,(, )}. The average dstance between each upper contour pont and ts correspondng lower contour pont, contour_thres, s In ths paper, the equaton o lnes s descrbed as a+b+c=0. 39

4 then calculated. I the dstance o one contour pont s less than contour_thres, then t s dscarded. The reserved contour ponts are Set( ) {( 1, 1 ),( ),...,( M, M )} And Set( ) {( 1, 1 ),( ),...,( M, M )}, where M s the number o the reserved contour ponts. And the mddle ponts o the above contour ponts are Set( C) {( 1, ( 1 1 ) / ), (, ( ) / ),..., (, ( M M M ) / )}. Three lnes are tted b the east-square algorthm accordng to the above upper, lower and mddle contour ponts respectvel: the upper baselne, the lower baselne, and the center baselne. We select a smeared lne as a real horzontal lne when cross _ angle(, ) angle _ thres And ave _ heght(, ) heght _ thres, where and represent the upper and lower baselnes respectvel, cross_angle s the cross angle between two lnes, and ave_heght s the average heght between the upper and lower baselnes. Both angle_thres and heght_thres are thresholds. And the horzontal drecton o one tet lne s the drecton o the center baselne Character vertcal stroe etracton In man stuatons, vertcal clues are scarce. When an mage s a partal porton o a whole document, there ma be ew or even no straght vertcal lnes. But character tlt orentatons can be regarded as clue drectons and the character vertcal stroes can be used as vertcal clues. However, these vertcal clues are not stable. Though vertcal stroe detecton s solved wth several conventonal technques [7], our method s rather ecent or MobleCam-based document analss. Derent rom the multple uzz sets used n [7], our method etracts a stable vertcal stroe set b heurstc rules and statstcal analss. (a) (b) (c) Fg.. Vertcal character stroe detecton: (a) captured mage, (b) edge mage, (c) vertcal stroes detected. The character vertcal stroe etracton s also based on the edge mage o the document. Fg. (a) s the orgnal perspectve document, and Fg. (b) shows an eample edge mage. Ater connected components analss on the vertcal edge mage, some long-stroe-le connected components are selected. These selected connected components nclude vertcal, horzontal, and slant drecton. On the assumpton that the perspectve angle n the vertcal drecton s less than 45 degree, we select vertcal stroe canddates b a smple rule: the heght o a connected component s much longer than ts wdth, t s a vertcal stroe canddate. In Chnese, Japanese and Englsh characters and tets, there are man curve stroes, such as, we remove these curve canddates b detectng the straghtness o the stroe whch s smlar to the one descrbed n Secton 3.1. Gven a connected component C, ts tted lne C, and the dstance rom one pont ( n C to C s DIS (, there s 1 n _ thres _ stroe C ( C ) 0 otherwse, where, ( DIS,, ), and PC ( stroe stroe ) 1 P p _ thres _ stroe (, C I C 0 otherwse, C I C C. In the above equaton,, ) s a Gaussan dstrbuton or C wth mean and standard devaton. And and stroe are also determned epermentall. stroe Because there are some nose and curves, we use the above steps to measure straghtness o detected lnes. That s, one lne s straght enough, t can be taen as a real lne. I (C ) s equal to 1, then C wll be a vertcal stroe. In order to detect straght vertcal stroes, p_thres_stroe taes a hgh value (e.g., near to 1), and n_thres_stroe s near to the number o pels n ths component. The resultng vertcal stroes o one document (Fg. (a)) are shown n Fg. (c). Snce n Chnese, Japanese and Englsh tets, most slant stroes are curve stroes, ater the above processng, the real vertcal stroes are obtaned. Consequentl, the vertcal vanshng pont detecton wll be ver robust. 4. Vanshng pont detecton b a hbrd approach We use a hbrd approach or vanshng pont detecton. Ater the lne ntersectons are calculated b lne pars, the ntersecton ponts are parttoned b clusterng algorthm, and tpcal ponts are selected as relable vanshng pont canddates. Ths process can be vewed as an ndrect approach. et, a drect approach s perormed b projecton analss rom perspectve vews onto these pont canddates. Fnall, results o both approaches are lnearl combned. The optmal canddate s selected as the nal vanshng pont. 40

5 4.1. Clusterng or vanshng pont detecton Wthout loss o generalt, we descrbe the clusterng based method or locatng the horzontal vanshng pont. All horzontal lnes (ncludng straght lnes and smeared lnes detected n Secton 3) are Set( ne) {( a1, b1, c1),..., ( a, b, c )}, where s the number o all horzontal lnes. As we now, two lnes wll produce an ntersecton pont. As a result, there are P ( 1) / ntersectons whch are possble canddates o the horzontal vanshng pont. These ntersectons are Set( Pt) {( 1, 1),...,(, )}. P P Ater checng the dstrbuton o lne ntersectons, we dscover that these ntersectons are located n the D space wth one or more groups wth hgh denst. It s natural to partton these ponts nto groups b clusterng. A sample o the dstrbuton o ntersectons or a horzontal vanshng pont s descrbed n Fg. 3. (a) (b) (c) Fg. 3. Intersecton dstrbuton or a horzontal vanshng pont: (a) captured mage, (b) horzontal lnes, (c) pont dstrbuton. Our clusterng space s D Eucldean space, and the smlart measure o two ponts s the Eucldean dstance, d(, j ) ( j ) ( j ), where (, ) s the eature vector (a pont n the space). The -means clusterng algorthm s a rather robust clusterng algorthm, whch s also proved rom our ntal eperences. The number o clusters n -means s speced b the ollowng smple rule: cluster ma( ln( P ), 10). 4.. Vanshng pont detecton and selecton For horzontal vanshng pont detecton, the rst tas s to locate several vanshng pont canddates based on clusterng ntroduced n Secton 4.1. Ater clusterng, we wll obtan cluster clusters. Each cluster s composed b some ntersecton ponts. The number o ponts n each cluster s Set( um) { 1,,..., }. cluster And the seres o centers o all clusters s c c c X C { 1,,..., }, cluster where the center o each cluster s the average o all the ntersecton ponts n ths cluster. In our method, these centers are canddates o the horzontal vanshng pont. And each canddate has a weght rom clusterng. The weght s gven b w / c cluster 1. Each o these weghts can be regarded as the prot uncton or the ndrect approach. that s c, ) w (, ). ndrect ( In order to get a more stable vanshng pont, we use a drect approach to rene vanshng pont canddates n the above search space. As shown n [], or a perspectvel sewed target, the bns represent angular slces projected rom H(, and mappng rom an mage pel p or a bn B ( s as ollows: 1 ( H, H p) (4) ( p, H ) where ( H, H p) s the angle between pel p and the center o the mage, relatve to the vanshng pont H(, and s the sze o the angular range wthn the document s contaned, agan relatve to H(. s obtaned rom T, T ) ( R where T and T R are the two ponts on the boundng crcle whose tangents pass through H(. These are shown n Fg. 4. For each cluster center, the above projecton analss s perormed, and the dervatve-squared-sum o the projecton proles rom a perspectve vew s calculated, drect ( c ) B 1 j1 ( B j1 B ). where B( s a projecton prole wth a vanshng pont canddate H(, and B s the number o projecton bns. Ths s the prot uncton or the drect approach. For a computatonal convenence, the used prot s changed to a coecent b drect ( c ) drect ( c ). ( c ) 1 drect Then accordng to equaton (), we combne these two prots n a lnear wa, g ) ndrect ) drect ). where 1. In our eperments, The resultng horzontal vanshng pont s gven b ( v, v ) arg ma g(, ). (, ) X C The last step s to conrm the resultng vanshng pont. Our rejecton strateg s that the dervatve-squared-sum o the true vanshng pont wll be larger than values o other ponts. The dervatve-squared-sum o the resultng vanshng pont s ( v, v ), whch s calculated b drect j (5) 41

6 equaton (5). The unchanged horzontal vanshng pont s (,0). And the dervatve-squared-sum o t s drect (,0). I the ollowng condton s satsed, then the nal horzontal vanshng pont s v, v ) : drect ( v, v ) (1 ) drect ( (,0), where 0 1, and n our method, Otherwse, we tae a rejecton strateg, and the nal vanshng pont wll be (,0). (a) mage, relatve to the vanshng pont V(, and s the sze o the angular range wthn the th tet row s contaned. Then, the prot uncton o the optmzaton becomes drect K 1 1 ( I ), where I = 1 or 0, and K s the number o tet rows. I B =0, then I =1; otherwse, I =0. Consequentl, a canddate wth a mamum number o whte columns o all rows rom perspectve vews s the vertcal vanshng pont. 5. Eperments The rectcaton transorm o our sstem s perormed as ollows. Gven the horzontal and vertcal vanshng ponts, we can recover documents wth ronto-parallel vews o the perspectvel sew document. For perspectve transormaton on a D plane, the transormaton relaton s d m11 m1 m13 u d m1 m m3 u 1 m m3 where ( u, u ) s the rected (undstorted) mage coordnate, and ( d, d ) s the orgnal (dstorted) mage coordnate. (b) Fg. 4. Analss o projecton proles rom a perspectve vew: (a) generatng projecton proles rom H(, (b) the wnnng projecton proles rom the vanshng pont H A and a poor eample rom H B (derved rom []). The wa or the vertcal vanshng pont detecton and selecton s smlar to the horzontal vanshng pont. In vertcal vanshng pont detecton, we use a projecton analss method whch s slghtl derent rom the one used n []. In character segmentaton, vertcal whte-space oten serves to separate derent characters. In a smlar wa, gven a vertcal vanshng pont canddate V(, the bns represent angular slces projecton rom V( o each tet row. Smlar wth equaton (4), mappng rom an mage pel p n the th tet row or the bn B s as ollows: 1 ( V, V p) ( p, V, ), where s the number o bns on the th tet row. And ( V, V p) s the angle between p and the center o the Fg. 5. The perspectve transorm relaton. Gven the horzontal and vertcal vanshng ponts on the mage plane, we can calculate a conve quadrangle on the mage plane whch s accordng to a rectangle n the rected mage plane. A versatle method or detectng such a conve quadrangle s descrbed n Fg. 5. In Fg. 5, the ellpse s the crcum-ellpse o the mage rectangle. The aspect rato o the result mage s decded as ollows. The average length o the top and bottom sdes o the quadrangle s the wdth o the result mage, and the average length o the let and rght sdes o the quadrangle s the heght o the result mage. 4

7 The eperment database s 418 test samples captured b several moble phone cameras. These mages are n RGB color ormat wth a resoluton. More than 90% o the mages have perspectve dstortons, and other mages have wea perspectve dstortons. Some samples are shown n Fg. 6. Gven a resultng vanshng pont, VP, the relatve dstance rom the ground truth VP t s calculated. I VP VP (6) VP t t T VP, then VP s regarded as a correct vanshng pont. In our sstem, the ground truth vanshng ponts are calculated rom the manuall mared horzontal and vertcal lnes. When the derence n Equaton 6 s less than the threshold (T VP =1/0), then there s no seemngl perspectve dstorton. It s also a concluson rom our eperments. We dvde our rected mages nto ve groups: (1) HIT, successul or perspectve rectcaton n both horzontal and vertcal drectons; () HHIT, successul n the horzontal drecton; (3) VHIT, successul n the vertcal drecton; (4) REJ, the rected mage s the same as the orgnal mage; (5) ERR, represents rectng wth wrong perspectve angles. We compared our method (M1) to other our methods: M doesn t use character vertcal stroes or vertcal vanshng pont detecton; M3 uses the ndrect approach based on clusterng onl to detect vanshng ponts; M4 uses model ttng n [9] nstead o clusterng; and M5 uses a sequental correcton stle (horzontal drecton correcton then vertcal drecton correcton) to compare the speed wth our method. The accurac results are descrbed n Fg. 7. And some rected mages wth ront-parallel vews o our method are shown n Fg. 6. And the resultng mage s the nner rectangle area o the detected perspectve quadrangle. As shown n Fg. 6, test samples nclude man derent tpes. There are even some street sgns and non-document mages. The racton o these non-document mages s about 0%. The REJ rate o our method s 6.3%, whch s manl caused b too large dstortons. For a moble phone wth some proper nteractve GIs, users ma accept the results o HIT, HHIT, VHIT, and REJ because the resultng mage rom these has a much better qualt than (or a same qualt as) the orgnall captured mage. In ths wa, the acceptance rate o our method s 98.33%. Compared wth M, our method (M1) mproves the HIT groups b 11.48%. Ths shows that character vertcal stroes are ver useul to detect the vertcal vanshng pont or documents wthout vertcal boundares. Compared wth M3, our method mproves 6.94% or the HIT accurac. Our hbrd approach s more adaptve and robust or vanshng pont detecton. Compared wth M4, our method mproves the HIT accurac b.39% and decreases the processng tme b 1ms, whch shows that our clusterng strateg s robust and ast compared to the tradtonal model ttng. Our method has a smlar perormance wth M5, though M5 uses a sequental stle wth partal rectcaton. (a) (a ) (b ) Fg. 6. Samples and the rected mages b our method ( are the correspondng rected mages): (a) general documents, and (b) sgnboards and posters. The processng speed s shown n Table 1, where Tme represents the average processng tme or each mage wthout ncludng the tme or the grascale mage converson and the nal perspectve transormaton. Eperments are run on a DE PC wth 3GHz CP, G Memor on a Wndows XP OS. Table 1. Results o the average processng tme. M1 M M3 M4 M5 Tme (ms) As shown n Table 1, the average processng tme o our method s largel less than M5, and the reduced tme s 13ms. The addtonal tme o M5 s spent n partall rectng. We also test the drect approach b a herarchcal search or horzontal vanshng pont detecton n [], whch s more tme consumng. For one mage n test samples, the detecton tme s more than one second. In concluson, the acceptance rate o our method s more than 98%, whle the error rate s less than %. And the processng tme s onl about 100ms. Wth serous or unstable dstortons, we tae the rejecton strateg, whch ma be more acceptable or a moble user. Furthermore, wth llusor lnes derved rom smeared tet baselnes and character tlt orentatons, our method can rect perspectve documents wthout ull boundares (see Fg. 6(a)). All these show that our rectcaton method s ast and robust. (b) 43

8 Fg. 7. Accurac (%) o each resultng group. 6. Conclusons Perspectve rectcaton o MobleCam based documents aces several challenges, such as speed, robustness, non-boundar documents, etc. In ths paper, we present a ast and robust method to deal wth these problems. In our method, the hbrd approach or vanshng pont detecton combnes drect and ndrect approaches wth hgh precson and ast speed. Furthermore, our method robustl detects tet baselnes and character tlt orentatons b heurstc rules and statstcal analss, whch s eectve or documents wthout ull boundares. The eperments on derent document mages captured b moble phone cameras show that our method has a good perormance wth an average speed o about 100ms on a regular PC. Reerences [1] J.F. Cann, A computatonal approach to edge detecton, IEEE Trans. on PAMI, vol. 8, no. 6, pp , [] P. Clar, and M. Mrmehd, Rectng perspectve vews o tet n 3D scenes usng vanshng ponts, Pattern Recognton, vol. 36, no. 11, pp , 003. [3] C.R. Dance, Perspectve estmaton or document mages, Proceedngs o SPIE Conerence on Document Recognton and Retreval IX, pp , 00. [4] W.. Hwang, C.-S. u, and P.-C. Chung, Shape rom teture: estmaton o planar surace orentaton through the rdge suraces o contnuous Wavelet transorm, IEEE Trans. on Image Processng, vol. 7, no. 5, pp , [5] D.X. e, G.R. Thoma, and H. Wechsler, Automated borders detecton and adaptve segmentaton or bnar document mages, Proceedngs o Internatonal Conerence on Pattern Recognton, vol. 3, pp , [6] J. ang, D. Doermann, and H.P., Camera-based analss o tet and documents: a surve, Internatonal Journal on Document Analss and Recognton, vol. 7, no. -3, pp , 005. [7] S.J. u, B.M. Chen, and C.C. Ko, Perspectve rectcaton o document mages usng uzz set and morphologcal operatons, Image and Vson Computng, vol. 3, no. 5, pp , 005. [8] C. Monner, V. Ablavs, S. Holden, and M. Snorrason, Sequental correcton o perspectve warp n camera-based documents, Proceedngs o IEEE Conerence on Document Analss and Recognton, vol. 1, pp , 005. [9] M. Plu, Etracton o llusor lnear clues n perspectvel sewed documents, Proceedngs o IEEE CVPR, pp , 001. [10] S. Pollard, and M. Plu, Buldng cameras or capturng documents, Internatonal Journal on Document Analss and Recognton, vol. 7, no. -3, pp , 005. [11] J. Shuelt, Perormance evaluaton and analss o vanshng pont detecton technques, IEEE Trans. on PAMI, vol. 1, no. 3, pp. 8-88, [1] J. ang, D. DeMenthon, and D. Doerman, Flattenng curved documents n mages, Proceedngs o IEEE CVPR, pp , 005. [13] J. ang, D. DeMenthon, and D. Doerman, Camera-based document mage mosacng, Proceedngs o Internatonal Conerence on Pattern Recognton,pp , 006. [14] K.R. Castleman, Dgtal Image Processng, Prentce Hall,