A Graphcal Model Framework for Couplng MRFs and Deformable Models Ru Huang, Vladmr Pavlovc, and Dmtrs N. Metaxas Dvson of Computer and Informaton Scences, Rutgers Unversty {ruhuang, vladmr, dnm}@cs.rutgers.edu Abstract Ths paper proposes a new framework for mage segmentaton based on the ntegraton of MRFs and deformable models usng graphcal models. We frst construct a graphcal model to represent the relatonshp of the observed mage pxels, the true regon labels and the underlyng object contour. We then formulate the problem of mage segmentaton as the one of jont regoncontour nference and learnng n the graphcal model. The graphcal model representaton allows us to use an approxmate structured varatonal nference technque to solve ths otherwse ntractable jont nference problem. Usng ths technque, the MAP soluton to the orgnal model s obtaned by fndng the MAP solutons of two smpler models, an extended MRF model and a probablstc deformable model, teratvely and ncrementally. In the extended MRF model, the true regon labels are estmated usng the BP algorthm n a band area around the estmated contour from the probablstc deformable model, and the result n turn gudes the probablstc deformable model to an mproved estmaton of the contour. Expermental results show that our new hybrd method outperforms both the MRF-based and the deformable model-based methods. 1. Introducton Image segmentaton s one of the most mportant and dffcult prelmnary processes for hgh-level computer vson and pattern recognton problems. The man goal of mage segmentaton s to dvde an mage nto ts consttuent parts that have a strong correlaton wth objects or areas of the real world depcted by the mage. Regon-based and edge-based segmentatons are the two major classes of segmentaton methods. Though one can label regons accordng to edges or detect edges from regons, these two knds of methods are naturally dfferent and have respectve advantages and dsadvantages. Regon-based methods assgn mage pxels to a regon accordng to some mage property (e.g., regon homogenety). These methods work well n nosy mages, where edges are usually dffcult to detect whle the regon homogenety s preserved. The dsadvantages of regon-based methods are that they may generate rough edges and holes nsde the objects, and they do not take account of object shape. On the other hand, edge-based methods generate boundares of the segmented objects. A pror knowledge of object shape and topology can be easly ncorporated to constran the segmentaton result. Whle ths often leads to suffcently smooth boundares, the oversmoothng may be excessve. Because edge-based methods rely on edge detectng operators, they are senstve to mage nose and need to be ntalzed close to the actual regon boundares. A hybrd segmentaton method that combnes regonbased and edge-based methods may mprove the segmentaton results over the two methods alone. In ths paper we propose a hybrd segmentaton framework to combne the Markov Random Feld (MRF)-based and the deformable model-based segmentaton methods. To tghtly couple the two models, we construct a graphcal model to represent the relatonshp of the observed mage pxels, the true regon labels and the underlyng object contour. Exact nference n the graphcal model s ntractable because of the large state spaces and the couplngs of model varables. To tackle ths problem we use a varatonal nference method to seemngly decouple the graphcal model nto two smpler models: one extended MRF model and one probablstc deformable model. Then we obtan the MAP soluton n the orgnal model by solvng the MAP problems of the two smpler models teratvely and ncrementally. In the extended MRF model, the true regon labels are estmated usng the Belef Propagaton (BP) algorthm n a band area around the estmated contour from the probablstc deformable model, and the result n turn gudes the probablstc deformable model to an mproved estmaton of the contour. The rest of ths paper s organzed as follows: secton revews the prevous work; secton 3 ntroduces a new ntegrated model and ts decoupled approxmaton usng the varatonal nference method; detaled nferences on the decoupled models are descrbed n secton 4; secton 5 shows the expermental results; and secton 6 summarzes the paper and future work. Proceedngs of the 004 IEEE Computer Socety Conference on Computer Vson and Pattern Recognton (CVPR 04) 1063-6919/04 $0.00 004 IEEE
. Prevous work Most segmentaton methods are ether regon-based or edge-based. Among regon-based methods, besdes the classcal regon growng method [1], the MRF-based methods have been extensvely used n dfferent areas [][3][4]. Because the exact MAP nference n MRF models s computatonally nfeasble, varous technques for approxmatng the MAP estmaton have been proposed, such as Markov Chan Monte Carlo (MCMC) [5], terated condtonal modes (ICM) [6], maxmzer of posteror margnals (MPM) [7], etc. Reference [8] presents a comparatve analyss of some of these methods. In edge-based methods, snce Kass et al. ntroduced Snakes [9], deformable models have attracted much attenton. Varants of deformable models have been proposed to address dfferent problems. For nstance, Balloons [10] and Gradent Vector Flow (GVF) Snakes [11] ntroduces dfferent external forces, and topologcally adaptable Snakes [1] allow changes n the model s topology. See [13] for a revew of deformable models and [1] for other edge-based methods and some basc edge detectng operators. Hybrd approaches [14][15][16] attempt to combne regon-based and edge-based segmentatons to allevate defcences of the ndvdual methods. There are dfferent choces of the combnaton. For nstance, [16] proposes a way of ntegratng MRFs and deformable models. MRFs are used to ntally estmate the boundary of objects n nosy mages. Balloons are then ftted to the estmated boundary. The result of the fttng s n turn used to update the MRF parameters. Fnal segmentaton s acheved by teratvely ntegratng these processes. Whle ths hybrd method attempted to take advantage of both MRFs and deformable models, the model couplng was loose. Ths may cause falure of deformable models f the ntal estmaton of the boundary by MRF s not closed, and t may also yeld oversmoothed boundares. 3. Our method We propose a new framework to combne MRFs and deformable models. The goal of our segmentaton method s to fnd one specfc regon wth a smooth and closed boundary. A seed pont s manually specfed and the regon contanng t s then segmented automatcally. Thus, wthout sgnfcant loss of modelng generalty, we smplfy the MRF model and avod possble problems caused by segmentng multple regons smultaneously. In ths secton, we frst brefly revew MRFs and deformable models, defne the notaton, and then ntroduce our hybrd framework. 3.1. MRF-based segmentaton MRF models are often used for mage segmentaton, because of ther ablty to capture the context of an mage (.e., dependences among neghborng mage pxels) and deal wth the nose. A typcal MRF model for mage segmentaton, as shown n Fgure 1, s a graph wth two knds of nodes: hdden nodes (crcles n Fgure 1, representng regon labels) and observable nodes (squares n Fgure 1, representng mage pxels). Edges n the graph depct relatonshps among the nodes. Fgure 1. MRF model Let n be the number of the hdden/observable states (.e., the number of pxels n the mage). A confguraton of the hdden layer s: x = ( x1,..., xn), x L, = 1,..., n (1) where L s a set of regon labels, such as L = {nsde, outsde}. Smlarly, a confguraton of the observable layer s: y = ( y1,..., yn), y D, = 1,..., n () where D s a set of pxel values, e.g., gray values 0-55. The relatonshp between the hdden states and the observable states (also known as local evdence) can be represented as the compatblty functon: φ ( x, y) = P( y x) (3) Smlarly, the relatonshp between the neghborng hdden states can be represented as the second compatblty functon: ψ ( x, xj) = P( x, xj) (4) Now the segmentaton problem can be vewed as a problem of estmatng the MAP soluton of the MRF model: xmap = arg max P( x y ) (5) x where P( x y) P( y x) P( x ) φ( x, y ) ψ( x, x ) (6) j (, j) As mentoned prevously, the exact MAP nference n MRF models s computatonally nfeasble, and varous technques have been used for approxmatng the MAP estmaton. In our method, we use the BP algorthm. The Proceedngs of the 004 IEEE Computer Socety Conference on Computer Vson and Pattern Recognton (CVPR 04) 1063-6919/04 $0.00 004 IEEE
estmaton of the MRF parameters (.e., the parameters n the compatblty functons) s another related problem, often solved usng the EM algorthm [3]. However, n the presence of multple regons n the mage, the automatc determnaton of the number of regons and the ntal guess of the parameters could be dffcult. More mportantly, lke other regon-based methods, MRFs do not take account of object shape and may generate rough edges and even holes nsde the objects. 3.. Deformable model-based segmentaton Many deformable model-based methods have also been used n segmentaton. A deformable model s usually a parameterzed geometrc prmtve, whose deformaton s determned by geometry, knematcs, dynamcs and other constrants (e.g., materal propertes, etc.) [17]. Snakes [9], a specal case of deformable models, are a parametrc contour: Ω= [0,1] R, s c( s) = ( x( s), y( s)), where s s the parametrc doman and x and y are the coordnate functons. The energy of the contour: E() c = Ent() c + Eext() c c c (7) = ω1() s + ω() s + F(()) c s ds Ω s s where ω () s and 1 ω () s control the "elastcty" and "rgdty" of the contour, and F s the potental assocated to the external forces. The fnal shape of the contour corresponds to the mnmum of ths energy. To mnmze the above energy term, one can use the dscretzed frst order Lagrangan dynamcs equaton: d + Kd = f (8) where d s dscretzed verson of c, K s the stffness matrx calculated from ω 1 () s and ω () s, and f s the generalzed force vector. Image gradent forces are usually used to attract a deformable model to edges. However, when far from the true boundary, the model often gets attracted to spurous mage edges. Balloon forces have been ntroduced to solve ths problem [10]. Namely, the deformable model s consdered a balloon, whch s nflated by an addtonal force and stopped by strong edges. The ntal contour need no longer be close to the true boundary. Mathematcally, a force along the normal drecton to the curve at contour node c(s) wth some approprate ampltude k s added to the orgnal forces. f' = f +kn ( s) (9) Deformable models can also be vewed n a probablstc framework [13]. The nternal energy E nt (c) leads to a Gbbs pror dstrbuton of the form: 1 P( c) = exp( Ent ( c )) (10) Z whle the external energy E ext (c) can be converted to a sensor model wth condtonal probablty: 1 P( I c) = exp( Eext ( c )) (11) Ze where I denotes the mage, and E ext (c) s a functon of the mage I. The deformable models can now be ftted by solvng the MAP problem: cmap = arg max P( c I ) (1) c where P( c I) P( c) P( I c ) (13) One lmtaton of the deformable model-based method s ts senstvty to mage nose, a common drawback of edge-based methods. Ths may result n the deformable model beng "stuck" n a local energy mnmum of a nosy mage. 3.3. Integrated model As shown n equaton (5) and (1), both the MRFbased and the deformable model-based segmentatons can be vewed as the MAP estmaton problems. In prevous work [16], these two models were loosely coupled. Our new framework uses the graphcal model theory to tghtly couple the two models. Ths s acheved, as depcted n Fgure, by addng a new hdden state to the tradtonal MRF model to represent the underlyng contour. Fgure. Integrated model In the new model, the segmentaton problem can also be vewed as a jont MAP estmaton problem: (, cx) MAP = argmax P(, cx y ) (14) cx, where P(, cx y) P( y x) P( x c) P() c (15) Proceedngs of the 004 IEEE Computer Socety Conference on Computer Vson and Pattern Recognton (CVPR 04) 1063-6919/04 $0.00 004 IEEE
To defne the jont dstrbuton of the ntegrated model, we model the mage lkelhood term P( y x ) as: P( y x ) = P( y x ) = φ( x, y ) (16) dentcal to the tradtonal MRF model. The second term P( x c ), modelng the dstrbuton of the regon labels condtoned on the contour, s defned as: P( x c) = P( x, x ) P( x c) = (, j) (, j) j ψ ( x, x ) P( x c) j (17) where we ncorporated a shape pror c to constran the regon labels x, n addton to the orgnal Gbbs dstrbuton. Snce we only segment one specfc regon at one tme, we need only consder the pxels near the contour, and label them ether nsde or outsde the contour. Now we can model the dependency between the contour c and the regon labels x usng the softmax functon: 1 Px ( = nsde c) = (18) 1 + exp( dst (, c)) P( x = outsde c) = 1 P( x = nsde c ) (19) nduced by the sgned Chamfer dstance of pxel from the contour c: dst(, c) = sgn(, c)mn loc() c ( s) (0) s Ω where sgn(, c ) = 1 f pxel s nsde contour c; sgn(, c ) = 1 when t s outsde, and loc() denotes the spatal coordnates of pxel. Lastly, the pror term P() c, as n equaton (10), can be represented as a Gbbs dstrbuton when the shape pror s gven by a parametrc contour c. Despte the compact graphcal representaton of the ntegrated model, the exact nference n the model s computatonally ntractable. One reason for ths s the large state space sze and the complex dependency structure ntroduced by the Gbbs dstrbuton of the pror P() c. The second reason s the exstence of loops n the graphcal model, whch preclude polynomal-tme nference. To deal wth these problems we propose an approxmate, yet tractable, soluton based on structured varatonal nference. 3.4. Approxmate nference usng structured varatonal nference Structured varatonal nference technques [18][19] consder parameterzed dstrbuton whch s n some sense close to the desred posteror dstrbuton, but s easer to compute. Namely, for a gven mage y Y, a dstrbuton Q(, cx y, θ ) wth an addtonal set of varatonal parameters θ s defned such that the Kullback Lebler dvergence between Q(, cx y, θ ) and P(, cx y ) s mnmzed wth respect to θ : * P(, cx y) θ = arg mn Q( cx, y, θ) log (1) θ cx, Q (, cx y, θ ) The dependency structure of Q s chosen such that t closely resembles the dependency structure of the orgnal dstrbuton P. However, unlke P the dependency structure of Q must allow a computatonally effcent nference. In our case we defne Q by decouplng the MRF model and the deformable model components of the orgnal ntegrated model n Fgure. The orgnal dstrbuton s factorzed nto two ndependent dstrbutons: an extended MRF model Q M wth varatonal parameter a and a deformable model Q D wth varatonal parameter b (Fgure 3). The extended MRF model means we have an addtonal layer to the tradtonal MRF model to deal wth the shape pror. Fgure 3. Decoupled models Because Q M and Q D are ndependent, Q(, cx yab,, ) = QM( x ya, ) QD( c b ) () Accordng to the extended MRF model, we have: QM( x y, a) QM( y x) QM( x a ) (3) Q ( y x ) = P( y x ) = φ( x, y ) (4) (, j) M Q ( x a) = P( x, x ) P( x a ) = M j (, j) ψ ( x, x ) P( x a ) j (5) Hence, Q ( x y, a ) φ( x, y ) ψ( x, x ) P( x a ) (6) M j (, j) The deformable model yelds: QD( c b) QD( b c) QD( c ) (7) QD( b c) = P( b c ) (8) leadng to Q ( c b) P( b c) Q ( c ) (9) D D Proceedngs of the 004 IEEE Computer Socety Conference on Computer Vson and Pattern Recognton (CVPR 04) 1063-6919/04 $0.00 004 IEEE
The optmal values of the varatonal parameters θ = ( ab, ) are obtaned by mnmzng the KL-dvergence. It can be shown, usng e.g., [0], that the optmal * * * parameters θ = ( a, b ) should satsfy the followng equatons: * * log Px ( a ) = QD( c b )log Px ( c ) (30) c * * log Pb ( ) = QM( x, )log Px ( ) x L c y a c (31) To obtan the term QM( x ya, ) we use the nference n the extended MRF wth soft nputs a contrbutng accordng to Px ( a ). On the other hand, to compute Q ( ) D c b one would have to fnd the dstrbuton of all contours c gven the "label mage" energy landscape log P( b c ), accordng to equaton (11). Snce the state space of c s too large, we smply use wnner-take-all strategy and approxmate QD ( c b ) as a delta functon: 1 f c = arg max QD ( c b) c QD '( c b ) = (3) 0 else and equaton (30) can be smplfed to: * Px ( a) = Px ( c ) (33) Equatons (31) and (33) together wth the nference solutons form a set of fxed-pont equatons. Soluton of ths fxed-pont set yelds a tractable approxmaton to the ntractable orgnal posteror. 3.5. Algorthm descrpton Intalze contour c; whle (error > maxerror) { 1. Calculate a band area B around c. Perform remanng steps nsde B;. Calculate Px ( a ) based on equaton (33) usng c; 3. Estmate the MRF-MAP soluton QM( x y, a) based on equaton (6) usng Px ( a ) ; 4. Calculate log Pb ( c ) based on equaton (31) usng QM( x y, a ); 5. Ft a deformable model wth balloon forces based on equaton (9) usng log Pb ( c ) ; } The varatonal nference algorthm for the hybrd MRF-DM model s summarzed as above. Steps and 4 follow drectly from equatons (33) and (31). The detals of steps 1, 3 and 5 are dscussed n next secton. 4. Implementaton ssues 4.1. Solve MRF-MAP wth EM and BP Step 3 of our algorthm solves the MAP problem n the extended MRF Model. The EM algorthm s used to estmate both the MAP soluton of regon labels x and the parameters of the model. Partcularly, n E-step, the MAP soluton of regon labels x s estmated based on current parameters. Unlke most of the prevous work mentoned n secton, we solve ths MRF-MAP estmaton problem usng the BP algorthm. BP s an nference method proposed by Pearl [1] to effcently estmate Bayesan belefs n the network by the way of teratvely passng messages between neghbors. It s an exact nference method n the network wthout loops. Even n the network wth loops, the method often leads to good approxmate and tractable solutons []. There are two varants of the BP algorthm: sum-product and max-product. The sumproduct message passng rule can be wrtten as: m ( x ) = Ψ ( x, x ) Φ ( x ) m ( x ) (34) j j j j k x k ℵ()\ j The max-product has analogous formula, wth the margnalzaton replaced by the maxmum operator. At convergence: x = arg max Φ ( x ) m ( x ) (35) MAP j x j ℵ() Accordng our extended MRF model the compatblty functons are: Φ ( x) = P( y x) P( x a) = φ( x, y) P( x a) (36) Ψ j( x, xj) = P( x, xj) = ψ ( x, xj) (37) We agan note the dfference from a tradtonal MRF model, due to the ncorporated shape pror. Px ( a ) s calculated n step of the algorthm. φ ( x, y) and ψ ( x, xj) can be calculated usng current MRF parameters. In ths model we assume the mage pxels are corrupted by whte Gaussan nose: 1 ( y µ ) x φ( x, y) = exp (38) πσ σ x x On the other hand, 1 δ ( x xj) ψ ( x, xj) = exp (39) Z σ where δ ( x) = 1 f x = 0; δ ( x) = 0 f x 0, σ controls the smlarty of neghborng hdden states, and Z s a normalzaton constant. As shown n step 1, n our algorthm belef propagaton s restrcted to a sngle band of model varables around the current contour estmates. A reason for ths s that, n practce, we only need to care about the statstcs of pxels near the boundary. More mportantly, the banded nference sgnfcantly speeds up the whole algorthm. Although convergence of the banded algorthm s not Proceedngs of the 004 IEEE Computer Socety Conference on Computer Vson and Pattern Recognton (CVPR 04) 1063-6919/04 $0.00 004 IEEE
guaranteed, n our experments, the BP algorthm does converge, usually n only one or two teratons. In M-step, the MRF parameters are updated based on the MAP soluton of regon labels x usng followng equatons: QM( x = l y, a) y µ l = (40) Q ( x = l y, a ) σ = l M Q ( x = l y, a )( y µ ) M l where l { nsde, outsde}. Q ( x = l y, a ) M 4.. Probablstc deformable model (41) In step 5, as mentoned n secton 3.4, we use the negatve log term, log P( b c ), as the external energy n the deformable model. In that case, the mage force s smply (log P( b c )). Wth the addtonal balloon forces, ths leads to the dscretzed frst order Lagrangan dynamcs equaton: d + Kd= (log P( b c)) + kn ( s) (4) We note that ths formulaton s dfferent from that of [16] where the deformable model s ftted to a bnary mage obtaned from the MAP confguraton of x. That s, the label of each pxel s fxed,.e., P( b c) = max P( x c ) (43) x whle n our method, we use a probablstc measurement of label of each pxel as specfed n equaton (31). Fnally, followng the defnton n equatons (18)~(0), we note that the gradent of the couplng energy at pxel, (log P( b c )), can be shown to be: 5. Experments log P( b c) log P( b c) = c loc() (44) Our algorthm was mplemented n MATLAB and all the experments were tested on a 1.5GHz P4 Computer. Most of the experments took less than one mnute on the mages of sze 18 18. 5.1. Synthetc mages The ntal study of propertes and utlty of our method was conducted on a set of synthetc mages. The mages were syntheszed n a way smlar to [8]. In [8], the 64 64 perfect mages contan only gray levels representng the "object" (gray level s 160) and the "background" (gray level s 100) respectvely. In our experments, we made the background more complcated by ntroducng a gray level gradent. The gray levels of the background are ncreasng from 100 to 160, along the normal drecton of the object contour (Fgure 4a). (a) (b) (c) (d) (e) (f) (g) (h) Fgure 4. Experments on synthetc mages Fgure 4b shows the result of a tradtonal MRF-based method. The object s segmented correctly, however some regons n the background are msclassfed. On the other hand, the deformable model fals because of the leakng from the hgh-curvature part of the object contour, where the gradent n the normal drecton s too weak (Fgure 4c). Our hybrd method, shown n Fgure 4d, results n a sgnfcantly mproved segmentaton. We next generated a test mage (Fgure 4e) by addng Gaussan nose wth mean 0 and standard devaton 60 to Fgure 4a. The result of the MRF-based method on the nosy mage (Fgure 4f) s somewhat smlar to that n Fgure 4b, whch shows the MRF can deal wth mage nose to some extent. But sgnfcant msclassfcaton occurred because of the complcated background and nose levels. The deformable model ether stcks to spurous edges caused by mage nose or leaks (Fgure 4g) because of the weakness of the true edges. Unlke the two ndependent methods, our hybrd algorthm, depcted n Fgure 4h, correctly dentfes the object boundares despte the excessve mage nose. For vsualzaton purposes we supermpose the contour on the orgnal mage (Fgure 4a) to show the qualty of the result n Fgures 4g and 4h. 5.. Real mages Experments wth synthetc mages n the prevous secton outlned some of the benefts of our hybrd method. The real world mages usually have sgnfcant, often non-whte nose and contan multple regons and objects, renderng the segmentaton task a great deal more dffcult. In ths secton we show results of applyng our method to real medcal mages on whch we can hardly get satsfyng results wth ether the MRF-based or the deformable model-based methods alone. Proceedngs of the 004 IEEE Computer Socety Conference on Computer Vson and Pattern Recognton (CVPR 04) 1063-6919/04 $0.00 004 IEEE
In the followng comparsons, we manually specfed the nsde/outsde regons to get an ntal guess of the parameters for the MRF-only method. For the deformable model method, we started the balloon model at several dfferent ntal postons and use the best results for the comparson. On the other hand, our hybrd method s sgnfcantly less senstve to the ntalzaton of the parameters and the ntal seed poston. Fgure 5a shows a D MR mage of the left ventrcle of the human heart. Fgure 5b s the result of the MRFbased method. Whle t s promsng, the result stll exhbts rough edges and holes. Fgure 5c depcts the result of the deformable model-based method. Although we carefully chose the magntude of the balloon forces, parts of the contour begn to leak others stck to spurous edges. Our hybrd method, started from the ntal contour shown n Fgure 5e, generated better result (Fgure 5d). One of the ntermedate teratons s shown n Fgure 5f. The correspondng external energy n the band area s depcted n Fgure 5g (gray values are proportonal to the magntude of the energy), showng a more useful profle than the tradtonal edge energy ( Gσ * I) shown n Fgure 5h. (a) (b) (c) (d) Fgure 7. Experments on medcal mages (3) (a) (b) (c) (d) (a) (b) Fgure 8. Experments on medcal mages (4) (e) (f) (g) (h) Fgure 5. Experments on medcal mages (1) Fgure 6a s an ultrasound mage. The MRF gets rough edges and holes n the objects (Fgure 6b) whle the deformable model cannot escape a local mnmum (Fgure 6c). Our hybrd method elmnates the rough edges and holes caused by the MRF whle outlnng the regon more accurately than the deformable model. (a) (b) (c) (d) Fgure 6. Experments on medcal mages () Fnally, Fgures 7a and 8a are both examples of dffcult mages wth complcated global propertes, requrng the MRF-based method to automatcally determne the number of regons and the ntal values of the parameters. Fgure 7b s obtaned by manually ntalzng the MRF model. Our method avods ths problem by creatng and updatng an MRF model locally and ncrementally. The mages are also dffcult for deformable models because the boundares of the objects to be segmented are ether hgh-curvature (Fgure 7a) or low-gradent (Fgure 8a). Fgure 7c exemplfes the oversmoothed deformable models. Our method s results, shown n Fgures 7d and 8b, do not suffer from ether of the problems. 6. Conclusons and future work We proposed a new framework to combne the MRFbased and the deformable model-based segmentaton methods. The framework was developed under the Proceedngs of the 004 IEEE Computer Socety Conference on Computer Vson and Pattern Recognton (CVPR 04) 1063-6919/04 $0.00 004 IEEE
auspces of the graphcal model theory allowng us to employ a well-founded set of statstcal estmaton and learnng technques. In partcular, we employed an approxmate, computatonally effcent soluton to otherwse ntractable nference of regon boundares. We showed the advantages and utlty of our hybrd method on a number of synthetc and real-world mages. Our current method lacks adaptve parameter selecton for both the deformable and the couplng model, an ssue we plan to address n the future. Smlarly, homogenety of the MRF s a lmtng factor that can be addressed usng spatally-dependent models. Another nterestng ssue s how to apply other approxmate nference algorthms, n partcular belef propagaton, to the whole model nstead of the extended MRF alone. Ths may requre a dfferent representaton of the probablstc contour, possbly smlar to [3] or [4]. Fnally, the proposed framework can be extended to 3D segmentaton as well as trackng problems requrng a shape pror, easly representable n our formalsm. Reference [1] M. Sonka, V. Hlavac, and R. Boyle, Image Processng, Analyss and Machne Vson, Second Edton, PWS Publshng, 1998. [] K. Held, E.R. Kops, B.J. Krause, W.M. Wells III, R. Kkns, and H.-W. Müller-Gärtner, "Markov Random Feld Segmentaton of Bran MR Images", IEEE Transacton on Medcal Imagng, 16(6), 1997. [3] Y. Zhang, M. Brady, and S. Smth, "Segmentaton of Bran MR Images Through a Hdden Markov Random Feld Model and the Expectaton-Maxmzaton Algorthm", IEEE Transacton on Medcal Imagng, 0(1), 001. [4] A.K. Jan and S.G. Nadabar, "MRF Model-Based Segmentaton of Range Images", Proceedngs of ICCV, 1990. [5] S. Geman and D. Geman, "Stochastc Relaxaton, Gbbs Dstrbutons and the Bayesan Restoraton of Images", IEEE Transacton on Pattern Analyss and Machne Intellgence, 6(6), 1984. [6] J. E. Besag, "On the Statstcal Analyss of Drty Pctures", Journal of the Royal Statstcal Socety B, 48(3), 1986. [7] J. Marroqun, S. Mtter, and T. Poggo, "Probablstc Soluton of Ill-posed Problems n Computatonal Vson", Journal of Amercan Statstcal Assocaton, 8(397), 1987. [8] R.C. Dubes, A.K. Jan, S.G. Nadabar, and C.C. Chen, "MRF Model-Based Algorthm for Image Segmentaton", Proceedngs of ICPR, 1990. [9] M. Kass, A. Wtkn, and D. Terzopoulos, "Snakes: Actve contour models", Internatonal Journal of Computer Vson, 1(4), 1987. [10] L.D. Cohen, "On Actve Contour Models and Balloons", Computer Vson, Graphcs, and Image Processng: Image Understandng, 53(), 1991. [11] C. Xu and J.L. Prnce, "Gradent Vector Flow: A New External Force for Snakes'', Proceedngs of CVPR, 1997. [1] T. McInerney and D. Terzopoulos, "Topologcally Adaptable Snakes", Proceedngs of ICCV, 1995. [13] T. McInerney and Demetr Terzopoulos, "Deformable Models n Medcal Image Analyss: A Survey", Medcal Image Analyss, 1(), 1996. [14] R. Ronfard, "Regon-based Strateges for Actve Contour Models", Internatonal Journal of Computer Vson, 13(), 1994. [15] T.N. Jones and D.N. Metaxas, "Image Segmentaton Based on the Integraton of Pxel Affnty and Deformable Models", Proceedngs of CVPR, 1998. [16] T. Chen and D.N. Metaxas, "Image Segmentaton Based on the Integraton of Markov Random Felds and Deformable Models", Proceedngs of MICCAI, 000. [17] D.N. Metaxas, Physcs-based Deformable Models: Applcatons to Computer Vson, Graphcs and Medcal Imagng, Kluwer Academc Press, 1997. [18] M.I. Jordan, Z. Ghahraman, T. Jaakkola, and L.K. Saul, "An Introducton to Varatonal Methods for Graphcal Models", Machne Learnng, 37(), 1999. [19] V. Pavlovc, B.J. Frey, and T.S. Huang, "Varatonal Learnng n Mxed-State Dynamc Graphcal Models", Proceedngs of UAI, 1999. [0] Z. Ghahraman, "On Structured Varatonal Approxmatons", Techncal Report CRG-TR-97-1, 1997. [1] J. Pearl, Probablstc Reasonng n Intellgent Systems: Networks of Plausble Inference, Morgan Kaufmann Publshers, 1988. [] Y. Wess, "Belef Propagaton and Revson n Networks wth Loops", Techncal Report MIT A.I. Memo 1616, 1998. [3] Y. Chen, Y. Ru, and T. S. Huang, "JPDAF Based HMM for Real-Tme Contour Trackng", Proceedngs of CVPR, 001. [4] P. Pérez, A. Blake, and M. Gangnet, "JetStream: Probablstc Contour Extracton wth Partcles", Proceedngs of ICCV, 001. Proceedngs of the 004 IEEE Computer Socety Conference on Computer Vson and Pattern Recognton (CVPR 04) 1063-6919/04 $0.00 004 IEEE