Trainable Context Model for Multiscale Segmentation

Transcription

1 Trainable Context Model for Multicale Segmentation Hui Cheng and Charle A. Bouman School of Electrical and Computer Engineering Purdue Univerity Wet Lafayette, IN {hui, ecn.purdue.edu Abtract Mot previou approache to Bayeian egmentation have ued imple prior model, uch a Markov random field (MRF), to enforce regularity in the egmentation. While thee method improve claification accuracy, they are t well uited to modeling complex contextual tructure. In thi paper, we propoe a context model for multicale egmentation which can capture very complex behavior on both local and global cale. Our method work by uing binary claification tree to model the tranition probabilitie between egmentation at adjacent cale. The claification tree can be efficiently trained to model eential apect of contextual behavior. In addition, the data model in our approach i vel in the ene that it can incorporate the correlation among the wavelet feature vector acro cale. We apply our method to the problem of document egmentation to illutrate it uefulne. 1 Introduction It i well kwn that accurate prior modeling can ubtantially improve egmentation accuracy by incorporating prior information regarding context. To thi end, previou approache to tatitical egmentation have generally relied on imple prior model, uch a Markov random field (MRF), that encourage the formation of large uniformly claified region. However, in many application prior information regarding context can be very complex, with both local and global apect. For example, when egmenting a cene, we may kw that ky may t be urrounded by ground or that car mut have mooth boundarie while tree have irregular boundarie. In thi paper, we introduce a multicale Bayeian context model which can capture complex apect of both local and global contextual behavior. The Thi work wa upported by the Xerox Corporation. Appeared in ICIP 98, vol. 1, pp , Chicago, IL, October 4-7, method i baed on the ue of tree claifier [1, 2] to model the tranition probabilitie between adjacent cale in the multicale tructure. Thi multicale tructure i imilar to previouly propoed egmentation model [3, 4, 5, 6], with the egmentation at each cale forming a Markov chain in cale. However, the tree baed claifier allow for much more complex tranition rule, while uing only a moderate number of parameter. Moreover, the tree baed claifier i computationally efficient to both train and apply. In addition, the multicale data model in thi approach can capture both the textural information at all cale and the correlation among feature vector acro cale. Thi i different from mot of other approache which aume independence among image feature. In our data model, the textural information at variou cale i captured through a hidden Markov model [7], and the dependence of feature between adjacent cale i extracted uing linear prediction. We apply thi algorithm to the problem of document egmentation becaue it repreent a excellent example of an application in which complex contextual dependencie can be very important [8]. The model i trained uing training image and their ground truth egmentation in a one-pa coare-to-fine proce. 2 Multicale Segmentation Model Fig. 1 how the tructure of our multicale egmentation model [6]. The pyramid to the left of Fig. 1 i the multicale context model. Each point in the pyramid contain a value x (n) which i a cla label at cale n and poition in the 2-D lattice S (n). Each label x (n) i aumed to depend on ome neighborhood of point at the coarer cale, x (n+1). Here, dete a neighborhood of point in the lattice S (n+1) at cale n+1. Therefore, the egmentation x (n) form a Markov chain in cale. The pyramid to the right of Fig. 1 i the multicale date model which contain the feature vector

2 X (0) X (1) X (2) Y (2) Y (1) Y (0) parent Figure 1: The tructure ued for our multicale egmentation model. The left pyramid model the contextual behavior, while the right pyramid model the data feature extracted uing a Haar wavelet tranform. extracted from the gray cale image uing a imple Haar bai wavelet tranform. Each feature, y (n),i a vector of three number correponding to the three wavelet coefficient at that location. The feature vector y (n) i aumed conditionally independent given the cla label x (n) and feature vector y (n+1) d at the coarer cale, where d i referred to a the parent of. We alo aume that y (n) can be predicted by α (n) x y (n+1) d + β x (n), and the prediction error ỹ (n) are conditionally independent given x (n) ỹ (n) = y (n) [ α (n) x y (n+1) d, ] + β x (n) where α (n) x, β x (n) are the prediction coefficient for cla x (n) at cale n. Our data model i different from other approache in that it model t only the textural information at each cale, but alo capture the correlation among the wavelet feature vector acro cale. To compute the egmentation, we ue the equential MAP (SMAP) egmentation approach of [3]. In the SMAP approach, claification proceed equentially from coare to fine cale, by computing the MAP egmentation given Y and the previou coarer cale egmentation ˆx (n+1). Formally, the SMAP egmentation i given by ˆx (n) = arg max 0 k M 1 +logp x (n) { log py x (n)(y k) (1) x (n+1) (k ˆx (n+1) ) } where M i the number of clae. Importantly, the term log p (n) y x (y k) may be efficiently computed in cloed form uing a imple hidden Markov model define for a quadtree [6]. Notice that (1) conit of two term, the firt i a data term while the econd enforce a priori kwledge regarding context, and i olely determined by the tranition probability p (n) x (k ˆx (n+1) x (n+1) ). coare neighbor children Figure 2: Thi diagram illutrate the tranition probabilitie that interpolate a pixel at the previou coarer reolution into four pixel at the current reolution uinga5 5 coare cale neighborhood. 3 Trainable Context Model For our context model, we aume that each label x (n) only depend on the label of a neighborhood of pixel at the previou coarer reolution x (n+1).intuitively, thi i a model for interpolating a pixel (n+1) into four children pixel (n) i (i =1, 2, 3, 4) at the next finer reolution. Fig. 2 how the cae of interpolating four children pixel uing a 5 5 neighborhood at the coarer cale. Since there are four ditinct children for each coare neighborhood, we will ue four ditinct tranition probability function, p (n) i ( ) (i =1, 2, 3, 4) at each reolution n. Define an indicator function I(), where I() = i, if i the i-th child of it parent. Alo, we ue the variable c and f to dete x (n) and x (n+1) repectively. Then, the tranition probabilitie are given by p x (n) x (n+1) (c f) = p (n) I() (c f). (2) Unfortunately, p (n) i (c f) may be very difficult to etimate, if the coare cale neighborhood i large. For example, a 5 5 neighborhood uing M =2reultin 2 25 poible value of f, which make direct parameter etimation impoible. In order to model each of the tranition probabilitie p (n) i (c f), we ue a cla probability tree (CPT) [1] a hown in Fig. 3. The CPT repreent a equence of deciion or tet that mut be made in order to compute the ditribution of c given f. The input to the tree i f, an encoded verion of the feature vector f. At each interior de, a plitting rule i ued to determine which of the two child de hould be taken. In our cae, the plitting rule i computed by comparing A t f µt to 0, where A t i a pre-computed matrix and µ t i a pre-computed calar. In thi way, f goe down

3 ~ A 1 f µ 1 ye ~ ~ A 2f µ 2 A f µ 3 3 X (0) X (1) X (2) ˆX (2) ˆX (1) ˆX (0) ye ye Ground Truth SMAP Etimate p (c f) p (c f) p (c f) ye ~ A 4f µ 4 p (c f) p (c f) 4 5 Figure 3: The cla probability tree ued to repreent tranition probabilitie between adjacent cale. At each interior de, a linear tet i ued to plit the de. The probability ma function p (n) i (c f) iapproximated eparately at each leaf de. the tree until it reache a leaf de. Each leaf de t i aigned a probability ma function ˆp t (c). When f reache a leaf de t, ˆp t (c)iuedaanapproximation to the true tranition probability p (n) i (c f). To contruct a CPT, we ue the recurive tree contruction algorithm (RTCA) of Gelfand, Ravihankar, and Delp [2] together with a leat quare baed plitting rule. The RTCA partition the training ample et into two halve and alternate the role of each half. Initially, a tree i grown uing the firt half and pruned on the econd half. Then the pruned tree i re-grown uing the econd half and pruned uing the firt half. Thi proce i repeated until the tree converge. The ample et for contructing a CPT are generated from a databae of training image which i produced by canning typical document. Scanned image are then manually egmented into deired component which form the ground-truth egmentation at each reolution. The training of the CPT i done in a coare-tofine procedure which i illutrated in Fig. 4. The method work by etimating the tranition probabilitie p (n) i (c f) from the ground truth egmentation x (n) and the coarer cale SMAP egmentation ˆx (n+1). Importantly, ˆx (n+1) doe t depend on the tranition probabilitie p (n) i (c f). Thi can be een from (1), the equation for computing the SMAP egmentation. Thi i a crucial fact ince it allow ˆx (n+1) to be computed before p (n) i (c f) i etimated. Once p (n) i (c f) ieti- decimation parameter etimation SMAP etimation Figure 4: Parameter etimation of the context model: (1) Compute the egmentation at the coaret reolution, ˆx (2). (2) Etimate the tranition probabilitie p (1) i (c f) uing the SMAP egmentation ˆx (2) and the decimated ground truth egmentation x (1).(3) Compute ˆx (1) uing p (1) i (c f). (4) Etimate p (0) i (c f) uing ˆx (1) and x (0). Thi procedure i then repeated for all cale. mated, it i then ued to compute ˆx (n), allowing the etimation of p (n 1) i (c f). Thi proce i then recurively repeated until the tranition parameter at all cale are etimated. Thi training procedure i very computationally efficient and yield accurate egmentation reult. 4 Simulation Fig. 5 how an example of egmenting a document image that i t contained in the training data et. The document i egmented into background, text and image clae. The model wa firt trained uing 20 canned document image together with hand egmented ground truth. Fig. 5(a) i the original image, Fig. 5(b) how the reult of egmentation uing the propoed egmentation algorithm, referred a the trainable SMAP (TSMAP) algorithm, with a 5 5 coare cale neighborhood, Fig. 5(c) how the egmentation uing TSMAP with a 1 1 coare cale neighborhood, and Fig. 5(d) how the egmentation uing only the finet reolution feature combined with the MRF a the context model. Notice that the larger neighborhood dramatically improve the quality and accuracy of egmentation. It i intereting to te that image region are enforced to be uniform, while text region are allowed to be mall with fine detail. Even ingle text line, revere text and page mark are labeled correctly. Our algorithm alo correctly claifie image region from different background, uch a paper background, halftone background and black background. Among the egmentation reult hown in Fig. 5, the egmentation computed uing the MRF

4 (a) (b) (c) (d) Figure 5: (a) Original image. (b) Segmentation reult uing TSMAP with a 5 5 neighborhood. (c) Segmentation reult uing TSMAP with a 1 1 neighborhood. (d) Segmentation uing MRF. Red, green and blue repreent text, image and background, repectively. i the pooret. There are many mall background region between text line that are miclaified a image region, and the boundarie of image region are quite irregular. The miclaification indicate that the MRF fail to capture eugh contextual information. Fig. 6 how the egmentation reult of two other image outide the training et uing TSMAP with a 5 5 coare neighborhood. In Fig. 7, we how the effect of the number of training image on the quality of the reulting egmentation. The TSMAP algorithm with a 5 5 coare cale neighborhood i trained on three training et which conit of 20, 10 and 5 training image, repectively. The reulting egmentation are hown in Fig. 7(b)- (d). Notice that the egmentation degrade gracefully a the number of training image decreae. There i little difference between the egmentation computed uing the TSMAP trained on 20 image and the egmentation baed on 10 training image. But when the number of training image i too mall, uch a 5, the egmentation reult (ee Fig. 7 (d)) begin to degrade. The experiment how that the TSMAP algorithm i t exceively enitive to the ize of the training et. Reference [1]L.Breiman,J.H.Friendman,R.A.Olhen,and C. J. Stone, Claification and Regreion Tree. Belmont, CA: Wadworth International Group, [2] S. B. Gelfand, C. S. Ravihankar, and E. J. Delp, An iterative growing and pruning algorithm for claification tree deign, IEEE Tran. on Pattern Analyi and Machine Intelligence, vol. 13,. 2, pp , February [3] C. A. Bouman and M. Shapiro, A multicale random field model for Bayeian image egmentation, IEEE Tran. on Image Proceing, vol. 3,. 2, pp , March [4] J. M. Laferte, F. Heitz, P. Perez, and E. Fabre, Hierarchical tatitical model for the fuion of multireolution image date, Proc. Int l Conf. on Computer Viion, June 1995, Cambridge, MA, pp [5] M. L. Comer and E. J. Delp, Multireolution image egmentation, Proc. of IEEE Int l Conf. on Acout., Speech and Sig. Proc., May 1995, Detroit, Michigan, pp [6] H. Cheng, C. A. Bouman, and J. P. Allebach, Multicale document egmentation, Proc. of IS&T 50th Annual Conf., May 1997, Cambridge, MA, pp [7] M. S. Croue, R. D. Nowak, and R. G. Baraniuk, Wavelet-baed tatitical ignal proceing uing hidden markov model, IEEE Tran. on Signal Proceing, vol. 46,. 4, pp , April [8] R. M. Haralick, Document image undertanding: Geometric and logical layout, Proc. of IEEE Computer Soc. Conf. on Computer Viion and

5 (a) (b) (c) (d) Figure 6: (a) Original image. (b) Segmentation reult uing TSMAP with a 5 5 neighborhood. (c) Original image. (d) Segmentation reult uing TSMAP with a 5 5 neighborhood. Red, green and blue repreent text, image and background, repectively. Pattern Recognition, vol. 8, April 1994, Seattle, WA, pp

6 (a) (b) (c) (d) Figure 7: (a) Original image. (b) TSMAP egmentation when trained on 20 image. (c) TSMAP egmentation when trained on 10 image. (d) TSMAP egmentation when trained on 5 image. Red, green and blue repreent text, image and background, repectively.