MosaicShape: Stochastic Region Grouping with Shape Prior

Transcription

1 Boston University Coputer Science Technica Report No , Feb To appear in Proc. CVPR, MosaicShape: Stochastic Region Grouping with Shape Prior Jingbin Wang Erdan Gu Margrit Bete Coputer Science Departent, Boston University, MA, Coputer and Inforation Science Departent, University of Pennsyvania, PA, Abstract A nove ethod that cobines shape-based object recognition and iage segentation is proposed for shape retrieva fro iages. Given a shape prior represented in a uti-scae curvature for, the proposed ethod identifies the target objects in iages by grouping oversegented iage regions. The probe is foruated in a unified probabiistic fraewor and soved by a stochastic Marov Chain Monte Caro (MCMC) echanis. By this eans, object segentation and recognition are accopished siutaneousy. Within each saping ove during the siuation process, probabiistic region grouping operations are infuenced by both the iage inforation and the shape siiarity constraint. The atter constraint is easured by a partia shape atching process. A generaized parae agorith [1], cobined with a arge saping jup and other ipeentation iproveents, greaty speeds up the overa stochastic process. The proposed ethod supports the segentation and recognition of utipe occuded objects in iages. Experienta resuts are provided for both synthetic and rea iages. 1. Introduction Successfu segentation or retrieva of objects of interest fro iages is of crucia iportance for a wide range of appications. A successfu ethod shoud provide a good approxiation to the optia segentation soution under various situations. However, the appearance of an object in rea iages can be affected by any factors, such as different coor/texture distributions of the object appearance or various iuination conditions. It becoes particuary chaenging when the object is present in the foreground with other objects as cutter. One soution strategy is to incorporate high-eve prior nowedge, such as a shape prior, into an existing iage segentation ethod. Contour-based segentation ethods, e.g., active contours [6] or eve set ethods [10], expicity or ipicity defor an active contour to capture the boundary of the ob- (a) (b) Figure 1: (a): Contour-based segentation resut by the traditiona active contour ethod [6], where the initia contour was shown as the in of bue nots and the fina contour was represented as the in of red nots. (b): Region-based segentation resut by data-driven MCMC [18]. (c): Segentation resut by the proposed ethod. ject. In previous wors by Leventon et a. [8] and Chen et a. [3], earned shape priors were introduced to constrain the 2D (3D) contour (surface) deforations so that target objects with theses predefined boundary shapes coud be extracted fro the cuttered bacground. However, the origina iitations inherent in contour-based ethods, such as the we-nown initiaization and oca inia probes (Fig. 1a), sti reained. Copared to the contour-based ethods, recent regionbased ethods [18, 13, 7] have the foowing advantages. First, region-based ethods are botto up and data driven. They do not require an initiaization step in genera, and can approxiate the gobay optia soution in any cases. Second, different types of high-eve prior nowedge, such as coor/texture odes [18, 13], boundary continuity hypotheses [18, 13], or perceptua easureents [7], can be incorporated into the botto-up segentation process. However, since this prior nowedge typicay cannot capture the arge variabiities of an object s appearance or shape that occur in practice, appication of the above region-based ethods was ainy restricted to generate perceptua groupings or visuay peasing segentation resuts (Fig. 1b). This paper proposes a nove ethod to incorporate prior nowedge of shape into a botto-up region-based segentation process for segenting and recognizing objects of the interest in iages (Fig. 1c). Our ain contributions are suarized beow: (c)

2 Given prior nowedge of shape, the proposed ethod identifies target objects in iages by grouping oversegented iage regions via a stochastic Marov Chain Monte Caro (MCMC) echanis 1. By this eans, segentation and recognition of utipe occuded object are accopished siutaneousy. During the stochastic siuation process, probabiistic region grouping operations are infuenced by both the iage inforation and the shape siiarity constraint. A great speedup of the segentation process is gained by carefuy adapting a parae agorith, caed Swendsen-Wang Cut (SWC) agorith [1], to the current probe and providing new ipeentation iproveents. (a) (b) The wor ost reevant to the current ethod was proposed by Scaroff et a. [15]. They used a deforabe shape tepate as a shape constraint for grouping iage regions. Their ethod was deterinistic and assued the correct initia condition was guaranteed in ost cases. Moreover, object segentation via deforabe shape tepates [21, 5] ay not capture the arge variabiity of the shape within the given cass. As a resut, arge occusions cannot be handed [15]. Recent wor by Tu et a. [17] pushed in the direction of accopishing object segentation and recognition siutaneousy within a unified fraewor [18]. Another idea of cobining the top-down and botto-up segentation was deonstrated recenty by Borenstein et a. [2] for segenting the foreground objects fro iages. These systes required the feature based prior odes to be carefuy constructed through a tie-consuing earning process. Therefore, they were of iited use for retrieving objects of arge variabiity in appearance or of arbitrary shapes fro iages. Moreover, these ethods did not hande occuded objects expicity. 2. Probe Definition and Method Overview Given an input coor iage (Fig. 2a), its oversegented iage (Fig. 2b) can be obtained by soe existing ethod, for instance, a ean shift ethod [4]. However, because an object of interest ay be partitioned into utipe atoic regions, such an oversegented iage cannot directy provide eaningfu object interpretations. When a shape prior is introduced (Fig. 2c), a eaningfu segentation can be achieved (Fig. 2d), where the objects of the interest in the iage are identified and recognized. The segentation probe to be soved is, given a shape prior represented in soe for, how to group the atoic regions in the iage such that the objects siiar to the shape 1 The atheatica bacground for MCMC theory was given in [20]. (c) Figure 2: (a): Input coor iage. (b): Oversegented iage. (c): Shape prior. (d): Recognized objects of interest, incuding either copete or partia shape inforation. prior can be correcty identified. Finding the gobay optia soution for such a probe is NP-hard [15]. On one hand, a deterinistic ethod [15] does not guarantee an optia soution in genera. For instance, because an iage region ay ony provide a partia hypothesis for the shape siiarity easureent, a best atched partia region boundary does not necessariy ipy that it wi be atched with the shape prior in the fina optia soution. On the other hand, an exhaustive top-down shape atching process coud be extreey sow because the shape of the objects in the iage can be transation, rotation, and scae invariant. Therefore, we appy a stochastic echanis as a coproise between the above two strategies in the proposed ethod. Section 3 describes how a shape prior is represented and used for easuring the shape siiarity between an iage region and the prior shape by perforing a partia shape atching. In Section 4, a stochastic MCMC echanis is foruated as the proposed soution for the given segentation probe. A Swendsen-Wang Cut agorith is appied to carry out the stochastic echanis and speed up the overa coputationa process(section 4.1), within which, the probabiistic region grouping operations are carefuy designed by taing both the iage and shape constraints into account (Section 4.2). During the siuating process, a arge saping jup for fast convergence can be obtained by a sipe shape registration (Section 4.3). Detais about syste ipeentation are given in Section 5. (d) 2

3 3. Shape Prior and Muti-Scae Curvature Representation Defining a good representation of a shape is a chaenging probe in itsef [9, 19]. For the probe at hand, the representation of the shape prior ust be transation, rotation, and scae invariant. In the proposed ethod, we therefore use boundary curvatures to define the shape of an object. Given the 2D cosed contour of an object, paraeterized by the arc ength paraeter as {(x(u),y(u)) u [0, 1]}, a soothed version of boundary curvature can be cacuated [11] by convoving the curve with different sizes of Gaussian ernes to yied (x(u, σ),y(u, σ)), where σ is the width of the erne, and then coputing κ(u, σ) = x (u, σ)y (u, σ) x (u, σ)y (u, σ). (x 2 (u, σ)+y 2 (u, σ)) 3/2 (1) To achieve a scae invariant representation, Ref. [11] noraized the curvature vaues by the ength of the contour, which was however probeatic for handing the partia shape atching probe described beow. The proposed ethod therefore instead precoputes a set of boundary curvatures for the object present in different scaes. In particuar, a shape prior S is defined as S = {C i,σ j,i [1,...,],j [1,...,n]}, (2) which consists of sequences of curvature vaues C i,σ j aong the boundary of the shape prior, soothed by a Gaussian erne of width σ j in the ith eve of scae. The tota nuber of such curvature sequences are n, for different eves of scae and n different sizes of Gaussian ernes Partia Shape Matching Probe The boundaries of oversegented iage regions typicay ony partiay atch the contour of a given shape prior (Fig. 2b). In order to appy the inforation on prior shape to group the oversegented iage regions, partia shape siiarity between the iage regions and the given shape prior needs to be easured. To our nowedge, finding a genera soution for identifying the atches of different parts of the shapes is sti an unsoved probe [19, 12]. In the probe at hand, we assue the partiay atched portions of the boundary of the sae object are connected and ony sa gaps are aowed between the. Given an input object V (e.g., an iage region) and a shape prior S, two boundary curvature sequences C V for V and C for S can be coputed by Eq. 1. A revised 1D correation process is ipeented (Agorith 1) to identify the ongest subsequence that atches in both C and C. The ength of this subsequence is noraized by the boundary ength of the input object. The second iteration, repeating 2 (C V ) ties, aows the atching to start fro any position on the region Agorith 1 : PARTIALSHAPECORRELATION(Curvatures C V of Input Object, Curvatures C of Prior Shape, Curvature Siiarity Threshod T, Aowed Gap Size E) Maxhits =0 for i =1to (C ) do Hits = Unhits =0 for j =1to 2 (C V ) do if j>(c V ) then = j (C V ) if (i + j) >= (C ) then n = i + j (C ) if C() C (n) <T then Hits++ ese Unhits++ if Unhits / Hits >E then if Hits > Maxhits then Maxhits = Hits UnHits = Hits =0 Norhits = Maxhits /(C V ) Return Norhits boundary due to its cycic representation. Threshod E contros the gap size aowed in the fina atched sequence, e.g.,, E [0.1, 0.15] in the current ipeentation. To easure the shape siiarity M(V,S) between V and S, Agorith 1 requires to be perfored for a size and soothness eves M(V,S)=ax{d i,j, =psc(c V,σi,C j,σ ), i, j, } (3) where psc() is the partia shape correation function coputed by Agorith 1, and C V,σi is the soothed boundary curvature for iage region V. We assue the ength of the region boundary that incudes the partiay atched shape is aways shorter than the ength of the boundary of the prior shape in its atched scae. Therefore, the shape siiarity resut ony needs to be coputed between C V,σi and a subset of curvatures in S. 4. Stochastic Region Grouping with Shape Prior In this section, we foow the atheatica fraewor proposed by Tu et a. [18, 1] and derive a Bayesian foruation for grouping iage regions with the introduced shape prior. We use a region adjacency graph, which contains a vertex for each atoic region of the oversegented iage and an edge e ij between vertices v i and v j in the graph if the regions represented by v i and v j are adjacent in the iage, i.e., share a boundary (Fig. 3(a) (c)). During the segentation process, the atoic regions ay be abeed as different region groups via a series of dynaic region grouping (graph partition) operations [1] (Fig. 3(d) (f)). The objective of the current segentation tas is to correcty group the atoic regions so that the region groups satisfying the given iage and shape constraints can be found and identified as the objects of interest. Given a region adjacency graph, an iage segentation is represented by: W =(n, (V 1,θ I1 ), (V 2,θ I2 ),...,(V n,θ In )) (4) 3

4 previous, a posterior probabiity for the segentation W is: V V V (a) (b) (c) V V V V V V V V V State V State (d) Spit Merge (e) Birth (f) Death Figure 3: Region Adjacency Graph and Region Grouping Operations: (a) An oversegented iage of eaves occuding each other. (b) Iage with region adjacency graph and graph vertices paced on the centroid of each atoic region. (c) A region grouping resut where the vertices beonging to the two eaves are ared in bue and red, respectivey. (d)-(f) Three types of region grouping operations are defined for the segentation state transition fro ϕ to ϕ, where vertices with the sae abe are connected by turned-on edges (thicer edges) and ared as the sae region group. During the state transition, a subgraph V is chosen in V, then it coud be erged into one of its neighboring groups ((d) for V V and (f) for V = V ), or becoe a new region group before the erging operation are defined as Cut(V,V V ) and ared with crosses in the idde coun. (e). A set of region edges between V and V as a rando variabe, whose different assignents correspond to different segentation states during the region grouping process. For instance, V 1,...,V n are n region groups or subgraphs in soe state, and each V i ay incude a nuber of atoic regions, such as V i = {v i 1,...,vi n } and V i V j =. Paraeter θ Ii suarizes a predefined iage ode for each iage region and can be earned in advance. If we assue V 1,...,V n are utuay independent, given an observed iage I and a shape prior S defined as. p(w I,S) p(i S, W )p(s W )p(w ) (5) n n [ p(i Vi V i,s)][ p(s V i )]p(w ) (6) i=1 i=1 n n [ p(i Vi θ Ii,S)][ p(s C Vi )]p(w ) (7) i=1 i=1 where I Vi represents iage patch associated with V i, and C Vi stores the boundary curvatures of the region V i. Furtherore, p(i Vi θ Ii,S) exp( D(I Vi,θ Ii )) (8) p(s C Vi ) exp ( (1 M(V i,s))) (9) p(w ) exp ( c 1 n c 2 Σ n i V i τ c 3 Σ n i C Vi ), (10) respectivey. Here, the discriinative ode D(I Vi,θ Ii ) easures the copatibiity of the observed iage data I Vi and the predefined appearance odes θ Ii for the target objects. It can contain one of two types of inforation: a coor ode defined as a Gaussian G(µ, σ) with specified ean µ and variance σ or a texture ode as an n-bin histogra H(n, h 1,...,h n ) on different intensity eves (h 1,...,h n ). Both odes can be earned in advance depending on the appearance of the object of interest. Siiarity M(V i,s) is defined in Eq. 3 and easures the shape siiarity between the current region V i and the shape prior S. As suggested by the previous statistica study [18], the nuber n of region groups, the size V i of each region group and the boundary soothness C Vi of each region are taen into account for defining the prior probabiity p(w ) in Eq. 10, where C Vi represents the suation of curvature agnitudes aong the region boundary. Intuitivey, a segentation is iey to incude a sa nuber of arge regions with sooth boundaries. The soution for the above segentation probe is approxiated by siuating the posterior probabiity p(w I,S) via a Marov chain. It can be reaized by a Metropois-Hastings echanis [20]. For our probe, saping the segentation states of W corresponds to a series of region grouping operations. Given the soution space Ω={ϕ ϕ is a possibe state of W } of the segentation probe, we define ϕ, ϕ Ω to be the two configurations of W that respectivey correspond to the segentation resuts before and after a region grouping operation (Fig. 3). The probabiity q(ϕ ϕ )=p(ϕ ϕ, I, S) indicates how iey it is for the current state ϕ to transfer to the next state ϕ. When a proposed ove fro ϕ to ϕ is accepted by α(ϕ ϕ )=in(1, q(ϕ ϕ) q(ϕ ϕ ) p(ϕ I,S) ), (11) p(ϕ I,S) the Metropois-Hastings ethod guarantees that the above Marov chain wi converge to p(w I,S) as its stationary 4

5 distribution. Therefore, given the definition of p(w I,S), there is a arge chance a good segentation can be achieved after any saping iterations Swendsen-Wang Cut Agorith One ajor disadvantage for ost MCMC ethods is that, during each saping iteration, ony a sa oca saping ove is aowed and the adjacent states are generay siiar. Therefore, for convergence, a ong siuation process is usuay required. The recenty deveoped Swendsen- Wang Cut (SWC) ethod [1] generaized a we accepted stochastic parae agorith [16] and was appied to sove a graph partition probe. This ethod aows a arge saping ove between very different graph configurations, thus providing fast siuation and optiization. For the current probe, we appy the SWC-2 agorith (journa preprint of [1]) cobined with other odifications to sape the different segentation configurations and perfor the region grouping operations, so that an idea segentation resut can be achieved efficienty. The ain steps of the new agorith MOSAICSHAPE are suarized beow, where the odified parts are accentuated in bod and wi be described further. Agorith 2 : MOSAICSHAPE (Iage I, Prior Shape S, Iage Mode θ I, Other Segentation Paraeters P ) Return vaue: List of Retrieved Objects; 1. Generate over-segentation resuts for I. 2. Copute boundary curvature for each atoic region. 3. Copute band probabiities b ij between a pairs of adjacent atoic regions. // Sape p(w I,S) by proposed ove ϕ ϕ (Fig.3) 4. For the current segentation ϕ, 4.1. Randoy choose an unared atoic region v and record its parent region group as V Turn on the edge e ij with the band probabiity b ij for a pairs of adjacent atoic regions inside the group V Foow the turned-on edges connected with the chosen v, find the connected region coponent inside V, and record it as V. 5. Merge V with its adjacent region group V with probabiity q( V,ϕ,I,S) and record this new state as ϕ. 6. Chec if region group V is siiar enough to S, or, if a arge saping jup is aowed, record the atched region group into a List of Retrieved Objects and ar its atoic regions. 7. Accept the new state ϕ = ϕ with the probabiity with the probabiity α(ϕ ϕ ). 8. Repeat fro Step 2 unti the convergence criterion is et or the expected nuber of the objects is retrieved. To appy the MOSAICSHAPE agorith, three probabiities, naey, b ij (or b e ), q( V,ϕ,I,S) and α(ϕ ϕ ), ust be defined in advance. The ain concusion fro previous wor [1] was that when: q(ϕ ϕ) q(ϕ ϕ ) = e Cut(V,V V ) (1 b e) e Cut(V,V V ) (1 b e) q( V,ϕ,I,S) q( V,ϕ,I,S) (12) was defined for α(ϕ ϕ ) (Eq. 11), the agorithic process was ergodic, aperiodic, and had p(w I,S) as its stationary distribution, where Cut(V,V V )={e ij v i V,v j (V V )} was defined as the set of region edges between region group V and V V (Fig. 3). As described next, once b ij (or b e ) and q( V,ϕ,I,S) are defined, the acceptance probabiity α(ϕ ϕ ) can be coputed directy given Eq. 12 and the posterior probabiity p(w I,S) defined in Eq Revised Band Probabiity and Proposed Move A band probabiity b ij = p(e ij = on v i,v j,i,s) is introduced for the edge e ij between two adjacent atoic regions v i,v j, and it deterines how iey a pair of adjacent regions shoud be grouped together. Intuitivey, b ij shoud be arge if the iage inforation incuded in two regions is copatibe, or if the shape of the erged region is ore siiar to the shape prior than either v i or v j. As a resut, we define b ij p(i v i,v j ) p(s v i,v j ) e MI(Iv i,iv j ) e MS(vi,vj,S) (13) + ɛ where M S (v i,v j,s) M(v i j,s) η = ax(m(v i,s),m(v j,s)) + M(v i j,s), with η =in{1, c (v i j,s)/ ax{ c (v i,s), c (v j,s)}}, (14) and M I (I vi,i vj ) (H i () ()) 2 (),() = H i()+h j (). (15) 2 As iustrated in Fig. 4, v i j is a new region introduced by erging v i and v j ; c (v i j,s) represents the ength of the ongest atched subsequence between the curvature sequence of v i j and those sequences stored in S. Onone hand, the shape siiarity defined by M(v i j,s) in Agorith 1 is noraized by the boundary ength of v i j so that a arge region with the coparabe ength of the atched boundary is penaized. On the other hand, the scae factor η pays the roe of encouraging the existence of a arge region. For instance, whie M(v i j,s) is saer than ax(m(v i,s),m(v j,s)), η coud be greater than 1 when v i j was atched with the shape prior in a arge scae. 5

6 v j v i e i j v i v j v i+j S M( v i, S) M( v j, S) M( v iuj, S). shape prior. A pair (t, θ) of transation and rotation paraeters can be cacuated by the east-squares ethod for registering P S with P V. Based on the obtained (t, θ), the prior shape S in the atched scae can be transfored onto the iage and noted as S T. A set of atoic regions inside this transfored shape prior S T is recorded as: {v i v i S T, for a i}, (18) ( v iuj, S) Figure 4: Shape-Based Band Probabiity: The ongest partiay atched boundaries of the iage regions are shown in bod. M I (I vi,i vj ) easures the iage copatibiity between the region ces v i and v j. In the current ipeentation, we copute n-bin coor histogras H i (n) and H j (n) for each of v i and v j, respectivey. Many dissiiarity easureents can be appied to copute the distance between two histogras [14]. In the current ipeentation, M I (I vi,i vj,i) is coputed as a χ 2 statistics distance. As shown in Fig. 3, the neighboring region groups for the chosen V can be represented by {V 1,V 2,...,V V,...,V n, } (16) and indexed fro 1 to n +1, where V V represents the reaining region group after V was spit fro V, and is an epty set. The probabiity q( V,ϕ,I,S) defines how iey it is that V is erged with a region group V aong a candidate groups, which is siiary defined as Eq. 13: q( V,ϕ,I,S)=p(V is erged with V ) p(i V,V ) p(s V,V ) (17) and noraized by Σ n+1 i=1 q(i V,ϕ,I,S). Because the band probabiity (Eq. 13) and the proposed ove (Eq. 17) characterize the doinant properties odeed by the posterior probabiity (Eq. 5) we, they aow the proposed ove ϕ ϕ to be accepted with a high probabiity such that the designed Marov chain wi quicy converge to the expected soution Large Saping Jup by Shape Registration The shape siiarity easureent coputed in Section 3.1 coud provide an additiona correspondence constraint between the atched iage region and the shape prior. A we-estabished correspondence constraint is very usefu for perforing the idea shape registration in genera, which ay aow the siuation process to reaize a arge saping jup. Given an observed iage region V and the shape prior S, the sets of the correspondence points on their atched boundaries can be recorded as P V and P S, which respectivey beong to the region boundary and the where the operation is sipy ipeented by judging if the centroid of v i is inside S T. A arge region group V Σ is then generated by erging a atoic regions in the above set (Eq. 18). When the size of V Σ is coparabe to the shape prior in the atched scae, we can further copute the reevant posterior probabiities (Eqs. 8 and 9) for V Σ. V Σ is recorded as the recognized object when the obtained posterior probabiity is arger than a given threshod, for instance 0.8, depending on what degree of occusion is aowed for the objects to be retrieved in the current ethod. However, the obtained segentation state via a arge saping jup does not correspond to the states noray reached by the designed Marov chain. Such a saping jup is not reversibe in genera and coud cause the resut be biased fro its optia soution. Therefore, in the current ipeentation, we ebedded this operation within each proposed ove and checed if any atched object coud be recognized, whie continuing with the Metropois-Hasting saping in its usua way. 5. Syste Ipeentation The ain steps of MOSAICSHAPE agorith were ipeented as foows. For an given input iage I, a shape prior S, a earned iage ode θ I and other segentation paraeters P, the syste first appied a ean shift ethod [4] to generate the oversegented iages (Step 1), in which a bandwidth vaue and the iniu region size were chosen so that the nuber of generated oversegented regions was ept oderatey sa. The boundary points of each region were identified and ined into a 2D cosed contour. The curvature vaues of each contour were coputed by Eq. 1, where a Fast Fourier Transforation (FFT) and inverse FFT were appied to speed up the convoution operations (Step 2). Based on these curvatures, the band probabiities (Eq. 13) between adjacent atoic regions were coputed via the PARTIALSHAPECORRELATION agorith (Step 3). A saping ove fro the current state ϕ to the next ϕ was ipeented by a Metropois-Hastings echanis (Steps 4 7). During this process, the atoic regions ying within a previousy we atched region group were chosen as v with a very sa probabiity (Step 4.1). Within every proposed ove, the recent region group and a arge saping jup were checed, and the atched object was recorded (Step 6). The evauation of probabiities de- 6

7 fined in Eqs. 5, 13 and 17 was ipeented by a reguarization fraewor, where a sa weight (0.2) was associated with the reevant iage probabiity, and a arge weight (0.6) was assigned for the shape reevant probabiity. Moreover, to speed up the overa siuation process, for soe operations requiring an expensive coputation, such as curvature cacuation, partia shape atching, or coor histogra construction, these resuts were ony coputed once for a newy generated region group, and they were then stored into a sorted ined ist (or hash tabe) indexed by the abes of the inner atoic regions of the current region group. Afterwards, when the sae region group was visited, these resuts coud be accessed efficienty by a search with a tie copexity O(og(n)). et. In soe situations, difficuties inherent in the origina partia shape atching probe ay be responsibe for situations when the current ethod fais (Fig. 9)2(c). (a) P(W I, S) (b) (c) 6. Experients The proposed ethod was tested on both synthetic and rea iages. In the synthetic experients, an irreguar star shape was first created as the shape prior S (Fig. 5a) and stored in a uti-scae curvature for, where the scae range was fro 0.75 to 1.25 copared to the ean size, and the widths for the chosen Gaussian ernes were 1, 2 and 4, respectivey. Synthetic iages were then created by randoy pacing severa osaic star objects onto rea iages that incuded a ot of cutter in their bacgrounds. Fig. 5 shows the segentation resuts for detecting the star objects with a copete shape in different rotations and scaes. Fig. 6 shows the resuts of detecting the objects with ony partiay atched shapes. The desired partia objects were retrieved by specifying an acceptabe shape siiarity threshod in this experient 0.7. To test the current ethod on rea iages, an eipse shape prior was first earned fro a set of 12 eaves objects (Fig. 7). We then used soe faen eaves as the objects to be detected in the rea iages. Different coor distributions were observed on the surfaces of these eaves, which yieded a nuber of sa atoic regions in the oversegented iages. To cover the shape variabiity aong the training sapes, during the arge saping jup process, the registration paraeters (t, θ) were first coputed between the iage regions and the earned ean shape. These paraeters were then appied for registering each shape sape in the training set onto the iage. Soe experienta resuts for detecting the objects with copete or partiay atched shapes are shown in Figs. 8 and 9. The overa segentation process too s for synthetic and s for rea iages. As can be seen in ost experients, our ethod provided satisfactory resuts for retrieving the shape fro the iages, where the stochastic siuation process usuay started as a sow anneaing process and recognized the target objects by aing a arge saping jup once a good partia atching criterion was (d) (e) iterations Figure 5: (a) Input iage. (b) Oversegented iage with region adjacency graph. (c) Cose-up view for soe oca region. The respective band probabiities for edges 1 4 are 0.24, 0.68, 0.67 and 0.39 (Eq. 14). (d) Segentation resut (green). (e) Posterior energy during the siuation. In red, saping states at which the target objects were identified. (f) Segentation resut by DDM- CMC ethod [18]. P(W I, S) (a) (d) (b) iterations Figure 6: (a) Input iage. (b) Oversegented iage. Object in arge red circe has partia shape atching siiarity (c) Our resut. Atoic regions in sa red circe were incuded in a atched object due to a arge saping jup. (d) Posterior energy and two saping states at which two objects with a partia shape siiarity arger than 0.7 were found. (e) Segentation resut by DDMCMC ethod [18]. 7. Discussion and Concusion The paper proposed a nove fraewor for object segentation and recognition. The ain contribution was to integrate the decoposed shape constraints into a bottoup iage segentation process by a partia shape atching (e) (f) (c) 7

8 (1a) (1b) (1c) Figure 7: The shape sapes of a eave object. (2a) (2b) (2c) (2d) (1a) (1b) (1c) Figure 9: 1(a)-2(a): Input iages; 1(b)-2(b): Oversegented iages; 1(c)-2(c): Our resuts. The atching abiguity inherent in the partia shape atch probe coud directy affect the process of recognizing the object of interest. As shown in 2(d), the occuded eave with a partiay atched boundary (ared in red) coud ead into an unexpected shape registration resut (ared in bue). (2a) (2b) (2c) (3a) (3b) (3c) Figure 8: 1st coun: Input iages; 2nd coun: Our resuts; 3rd coun: Resuts by DDMCMC [18]. process. As a resut, the segentation and recognition of utipe occuded objects can be achieved siutaneousy. The current ethod ay be iproved in the foowing aspects. First, a better shape siiarity easureent ight hep produce ore accurate resuts. For instance, a dynaic tie warping ethod [12] ay be appied to support partia shape atching for objects with distorted shapes. Second, the variabiity within the cass of the shape prior ay be odeed by principe coponent anaysis, however, additiona chaenges ay be encountered in appying partia shape atching. Finay, integrating coprehensive iage odes [18, 13] into the current syste ay hep capture additiona variabiities of the object appearance and thus obtain iproved segentation resuts. References [1] A. Barbu and S. Zhu. Graph partition by Swendsen-Wang cuts. In Proceedings of the Ninth IEEE Internationa Conference on Coputer Vision (ICCV 03), pages , Nice, France, Oct A preprint of journa version can be found at abarbu/. [2] E. Borenstein, E. Sharon, and S. Uan. Cobining topdown and botto-up segentation. In Proceedings of IEEE Worshop on Perceptua Organization in Coputer Vision, Washington DC, USA, [3] Y. Chen, H. Tagare, and etc. Using prior shapes in geoetric active contours in a variationa fraewor. Int J Coput Vis, 50: , [4] D. Coaniciu and P. Meer. Mean shift: A robust approach toward feature space anaysis. IEEE Transactions on Pattern Anaysis and Machine Inteigence, 24(5): , [5] A. K. Jain, Z. Yu, and S. Lashanan. Object atching using deforabe tepates. IEEE Transactions on Pattern Anaysis and Machine Inteigence, 18(3): , [6] M. Kass, A. Witin, and D. Terzopouos. Snaes: Active contour odes. Int J Coput Vis, 1: , [7] J. Kaufhod and A. Hoogs. Learning to segent iages using region-based perceptua features. In CVPR, pages , Washington DC, USA, [8] M. E. Leventon, W. E. L. Grison, and O. Faugeras. Statistica shape infuence in geodesic active contours. In Proceeding of IEEE Conference on Coputer Vision and Pattern Recognition, Voue I, pages , Hiton Head, SC, USA, June [9] S. Loncaric. A survey of shape anaysis techniques. Pattern Recognition, 31(8): , [10] R. Maadi, J. Sethian, and B. Veuri. Shape odeing with front propagation: A eve set approach. IEEE Transactions on Pattern Anaysis and Machine Inteigence, 17: , [11] F. Mohtarian. Sihouette-based isoated object recognition through curvature scae space. IEEE Transactions on Pattern Anaysis and Machine Inteigence, 17(5): , [12] E. Petrais, A. Diparos, and E. Mios. Matching and retrieva of distorted and occuded shapes using dynaic pro- 8

9 graing. IEEE Transactions on Pattern Anaysis and Machine Inteigence, 24(11): , [13] X. Ren and J. Mai. Learning a cassification ode for segentation. In Proceedings of the Ninth IEEE Internationa Conference on Coputer Vision (ICCV 03), pages 10 17, Nice, France, Oct [14] Y. Rubner, C. Toasi, and L. Guibas. The earth over s distance as a etric for iage retrieva. Int J Coput Vis, 40:99 121, [15] S. Scaroff and L. Liu. Deforabe shape detection and description via ode-based region grouping. IEEE Transactions on Pattern Anaysis and Machine Inteigence, 23(5): , [16] R. Swendsen and J. Wang. Nonuniversa critica dynaics in Monte Caro siuations. Phys Rev Lett, 58(2):86 88, [17] Z. W. Tu, X. R. Chen, A. L. Yuie, and S. C. Zhu. Iage parsing: unifying segentation, detection, and recognition. In Proceedings of the Ninth IEEE Internationa Conference on Coputer Vision (ICCV 03), pages 18 25, Nice, France, Oct [18] Z. W. Tu and S. C. Zhu. Iage segentation by data-driven Marov Chain Monte Caro. IEEE Transactions on Pattern Anaysis and Machine Inteigence, 23(5): , [19] R. C. Vetap. Shape atching: Siiarity easures and agoriths. In Proceedings of the Internationa Conference on Shape Modeing and Appications, pages , Genova, Itay, [20] G. Winer. Iage Anaysis, Rando Fieds and Dynaic Monte Caro Methods. Springer Verag, [21] A. Yuie, D. Cohen, and P. Hainan. Feature extraction fro faces using deforabe tepates. Int J Coput Vis, 8(2):99 111,