3D SMAP Algorithm. April 11, 2012

Transcription

1 3D SMAP Algorithm April 11, 2012 Baed on the original SMAP paper [1]. Thi report extend the tructure of MSRF into 3D. The prior ditribution i modified to atify the MRF property. In addition, an iterative trategy i propoed to refine the egmentation. 1 Goal Segment 3D image volume into region of ditinct tatitical behavior. 2 Baic Model Y i oberved 3D image volume containing ditinct texture. X i unoberved field containing the cla of each pixel. S i 3D lattice of point. i a member of S. The lower cae letter x, y denote the correponding determinitic realization. 3 Approach Outline 1. Auming the behavior of each oberved pixel in Y i dependent on a correponding unoberved label pixel in X. 1

2 2. The dependence i pecified through p y x (y x), which i modeled by Gauian mixture ditribution (GMM) in our experiment. The parameter of GMM i computed uing cluter algorithm decribed by [2]. 3. Prior knowledge about the region will be pecified by the prior ditribution p(x), which i modeled by multicale random field (MSRF). 4. Solve following optimization problem which minimize the average cot of an erroneou egmentation. ˆx = arg min x E[C(X, x) Y = y] (1) 5. Once the egmentation i obtained, we can re-etimate the GMM parameter for each cla uing only a ubet of the original data, whoe member have the ame egmentation label. Therefore, we can iteratively refine the egmentation reult until the termination condition i atified. 4 Multicale Random Field Multicale Random Field (MSRF) i compoed of a erie of 3D random field at varying cale or reolution. A demontration picture i hown by Fig 1. At each cale, n, the egmentation or labeling i denoted by the random field X, and the et of lattice point i denoted by S. In particular, X (0) i aumed to be the finet cale random field with each point correponding to a ingle image pixel. Each label at the next coarer cale X (1) then correpond to a group of eight point in the original image. Therefore, the number of point in S (1) i 1/8 the number of point in S (0). Aume the equence of random field from coare to fine cale form a Markov chain. Then P (X = x X (l) = x (l), l > n) (2) =P (X = x X (n+1) = x (n+1) ) =P x x (n+1)(x x (n+1) ) Define the cot function a follow: C SMAP (X, x) = 1 L n 1 C n (X, x) (3) n=0

3 Figure 1: 3D random field with three different cale. Each cube repreent a pixel in the 3D image volume where L i the coaret cale in X and C n (X, x) i the penalty function aociate with labeling error at the n th cale. Suppoe K th cale i the unique cale uch that X (K) x (K), and X (i) = x (i) for all i > K. Define the function C n a 1, if n K C n (X, x) = 0, if n > K Uing (3) and (4) We can compute the total cot C SMAP (X, x) = 2 K. Intuitively, C SMAP (X, x) can be interpretated a the width of the larget grouping of miclaified pixel. We can determine the etimator which minimize the propoed cot function by evaluating (1). ˆx = arg min E[C SMAP (X, x) Y = y] (5) x L = arg min 2 n {1 P (X (i) = x (i), i n Y = y)} x = arg max x n=0 L 2 n {P (X (i) = x (i), i n Y = y)} n=0 Uing recurive approach, [1] ha proved the following olution for (6) ˆx = arg max x {log p x x (n+1),y(x ˆx (n+1), y) + ɛ(x )} (6) where ɛ i a econd order term which vanihe to 0 a number of pixel N increae. i.e. lim N ɛ(x ) = 0. (4)

4 Ignoring ɛ when N i large reult in the following recurive equation: ˆx (L) = arg max x (L) log p x (L) y(x (L) y) (7) ˆx = arg max x {log p x x (n+1),y(x ˆx (n+1), y)} (8) Appying the Baye rule, the Markov propertie of X, and auming that X (L) i uniformly ditributed, the SMAP recurion may be rewritten in a form which i more eaily computed. ˆx (L) = arg max x (L) log p y x (L)(y x (L) ) (9) ˆx = arg max x {log p y x (y x ) + log p x x (n+1)(x ˆx (n+1) )} (10) Now we introduce 3D SMAP algorithm to etimate the olution to (9) and (10). 5 3D SMAP algorithm Thi algorithm pecify the conditional denity p y x (y x ) together with the coare to fine cale tranition denitie p x x (n+1)(x ˆx (n+1) ). A hybrid model, which incorporate 3D quadtree and 3D graph i ued to decribe the tructure of MSRF. The algorithm etimate the parameter of MSRF during the egmentation proce. The computation i baed on following aumption and propertie of Markov Random Field (MRF). Firt, the oberved pixel are conditionaly independent given their cla label. Then the conditional denity function for the image ha the form. p y x (0)(y x (0) ) = p y x (0) S (0) (y x(0) ) (11) where p y x ( k) i the conditional denity function for an individual pixel given the cla (0) label k. We ue a multivariate Gauian mixture denity in experiment, but other ditribution can be applied a well. Second, the labeling in X will be conditionally independent, given the labeling in X (n+1).

5 Furthermore, each labeling X will only be dependent on a local neighborhood of labeling at the next coarer cale. That i where p x x (n+1) x (n+1). p x x (n+1)(x ˆx (n+1) ) = p x S i the probability denity for x x (n+1) (x ˆx (n+1) ) (12) given it neighbor at the coarer cale The choice of neighborhood i important, which affect the tructure of multicale pyramid a well a the pecification of probability denity function decribed earlier in thi ection. 1) Compute p y x (y x ) with 3D quadtree The tructure of the quadtree i hown in Fig 2. Each point in the pyramid i only dependent on a ingle point at the coarer cale. According to Hammerley-Clifford theorem, the probability denity function can be characterized by Gibb ditribution. We ue the following function to model the probability that X ha cla m, given that it parent i of cla k. Figure 2: 3D random field with two ucceive cale. Each cube repreent a pixel in the 3D image volume. The red pixel in cale n = 0 have the ame parent in coarer level n = 1, which i alo in red. p x x (n+1) (m k) = 1 Z exp( θ 0 δ m k ) (13) where δ m k = 1 if m k, otherwie δ m k = 0. Z = M m=1 exp{ θ 0 δ m k } i called partition function, where M i the number of poible clae. θ 0 > 0 i the regularization parameter

6 of cale n. Large θ 0 reult higher probability that the label remain the ame from cale n + 1 to n. An important property of the quadtree tructure i that the conditional ditribution of Y given X, ha a product form that may be computed uing a imple fine-to-coare recurion. Let y be the et of leave of the 3D quadtree that are on the branch tarting at S. [1] prove that the conditional denity for Y given X ha the product form Furthermore, the denity function p y x p y (n+1) x (n+1) p y x (y x ) = p y (y x x ) (14) S (y (n+1) k) = r d 1 () m=1 may be computed uing the following recurion, M p y r x r (y r m)p x where d 1 () i a et of point whoe firt ucceive parent i. r x (n+1) r (m k) (15) Conider the dynamic range, we take the log likelihood function defined a follow. l (k) log p y x (y k) (16) Subtitute the tranition ditribution of (13) into (15) and converting to log likelihood function yield the new recurion l (0) (k) = log p y (0) l (n+1) (k) = r d 1 () x (0)(y(0) k) (17) M 1 log{ Z exp(l r (m) θ 0 δ m k )} m=1 Uing (17), we can rewrite the firt term of right-hand-ide of (10) a log p y x (y x ) = l S 2) Compute p x x (n+1)(x ˆx (n+1) ) with 3D graph (x ) (18) A diadvantage of the 3D quadtree model i that pixel that are patially adjacent may not have common neighbor at the next coarer cale. In order to compenate the weakne of dicontinuity of boundary, we introduce 3D graph to model the tranition probability

7 p x x (n+1)(x ˆx (n+1) ). The main idea i to increae the number of coare cale neighbor for each pixel. According to the decription in 1), point ha only one coare cale neighbor, which i the parent node in the 3D quad tree. Now we expand the number of coare cale neighbor from one to four a hown in Fig. 3. In order to expre the poition of the neighbor, we explicitly denote a pixel at cale D a = (t, i, j), where t, i, j are the frame, row and column indice. The four neigbor at cale D + 1 may then be computed uing the function odd(i) = 1 if i i odd and -1 if i i even and the greatet maller integer function. (a) (b) Figure 3: 3D random field with two ucceive cale. Each cube repreent a pixel in the 3D image volume. a) how 32 pixel in level D, b) how 4 pixel in coarer level D + 1 correponding to a). In particular, pixel in level D ha four coarer level neighbor: 0, 1, 2, 3. 0 = ( t/2, i/2, j/2 ) 1 = ( t/2, i/2, j/2 ) + (odd(t), 0, 0) 2 = ( t/2, i/2, j/2 ) + (0, odd(i), 0) 3 = ( t/2, i/2, j/2 ) + (0, 0, odd(j))

8 The tranition function which we chooe for thi pyramid graph i a natural extenion of the tranition function ued for the quadtree baed model. In order to ditinguih (13) from the following formula, we ue p intead. p x x (n+1) =P (X = 1 Z exp( (m m r ), r d() (19) = m X (n+1) r = m r ) r d() β r δ m mr ) Similar to (13), Z = M m=1 exp{ r d() β r δ m mr }, d() = { 0, 1, 2, 3 } i a et of point contain all the neighbor of in the next coarer level. β r can be viewed a a weight parameter aociated with the neighbor point r in cale n + 1. The larger the β r, the larger the probability that the label of will be the ame a r. Beide, we can make the tranition moother by adding neighbor point in the ame cale. Define d 0 () = { 0,..., 5} be a et containing ix neighbor point in the ame cale D a. See Fig. 4 for a viual demontration. The coordinate of are explicitly given by Then (19) can be rewritten a p x 0 = (t, i, j 1), 1 = (t, i, j + 1) 2 = (t, i 1, j), 3 = (t, i + 1, j) 4 = (t 1, i, j), 5 = (t + 1, i, j) x (n+1),x =P (X = 1 Z exp( (m m r, m q ), r d(), q d 0 () (20) = m X (n+1) r r d() = m r, X q = m q ) β r δ m mr q d 0 () β q δ m mq ) where Z = M m=1 exp{ r d() β r δ m mr q d 0 () β q δ m mq }. For implicity, we can ue identical weight for point in the ame cale, i.e. θ 1, q, β q θ 2. r, β r Uing thi hybrid pyramid tructure of MSRF, the conditional likelihood of (8) and (10) can

9 Figure 4: 3D random field of one cale. Each cube repreent a pixel in the 3D image volume. i a pixel in level D with ix neighbor in the ame level: 0, 1, 2, 3, 4, 5. be expreed in the form log p y,x x (n+1)(y, x ˆx (n+1) ) = l S (x ) + log p x x (n+1),x which reult in the following formula for the SMAP etimate of X : (x ˆx (n+1), ˆx ) (21) ˆx (L) ˆx = arg max 1 k M l(l) (k) (22) = arg max 1 k M {l = arg max 1 k M {l (k) + log p x (k) log Z θ x (n+1),x 1 r d() (k ˆx (n+1), ˆx δ m mr θ 2 )} q d 0 () δ m mq } 6 Etimation of Tranition Probability Parameter We introduced three parameter θ = [θ 0, θ 1, θ 2 ] to determine the tranition probability denity function baed on two different model of pyramid tructure. To be pecific, θ 0 are required for fine-to-coare in computing the log likelihood function (17). And the θ 1, θ 2 are required for coare-to-fine operation in the SMAP egmentation (22). The parameter are aigned with ome initial value and can be updated in the proce of egmentation by

10 olving a joint optimization problem pecified by (17) and (22). Thi problem can be olved uing ICM [3]. 7 Etimation of GMM parameter The tatitical behavior of each egment i modeled by a Gauian mixture ditribution (GMM) with parameter η k = [π k, µ k, R k ], k = 1,..., M, where M i the poible number of clae. Let π k R N, µ k R N, R k R N N, where N i the number of Gauian in the mixture denity. And N 1 i=0 π k i = 1, where π ki i the i th entry of π k. Then the probability denity function of one pixel given it label k can be written explicitly a p y x,η(y k, η k ) = N 1 i=0 For impler cae when N = 1, (23) can be reduced to p y x,η(y k, η k ) = π { ki (2π) R N/2 k i 1/2 exp 1 } 2 (y µ ki ) T R 1 k i (y µ ki ) 1 { (2π) R k 1/2 exp 1 } 1/2 2 (y µ k ) T R 1 k (y µ k ) (23) (24) (23) and (24) are ued to compute the log likelihood function defined in (16). We tart the egmentation with ome initially value of η k. After applying the 3D SMAP algorithm to the entire data et Y, we obtain the label of pixel X. Now we can divide the Y into M dijoint ubet. For each ubet, we can ue [2] to etimate new η k, with which we can apply the 3D SMAP algorithm again to refine the egmentation. Thi proce i repeated until the change of GMM parameter between two ucceive iteration i le than certain amount of value. For intance, we can top the iteration when the following criteria i atified. M µ k µ k ɛ 3 (25) k=1 where i the infinity norm and ɛ 3 = 1. The complete SMAP egmentation algorithm with parameter etimation i ummarized below.

11 1. Set the initial parameter value for all n, θ 0 = 5 1 n, θ 1 = 4, and θ 2 = 2, n = 0,..., L. 2. Compute the likelihood function uing (17). 3. Compute ˆx (L) (k) uing (22). 4. For cale n = L 1 to n = 0 (a) Set I = deired number of iteration i. Compute ˆx uing (22). ii. Update ˆθ. iii. Reduce I by one. (b) Proceed if I i negative or no further change happen, otherwie go back to (a) 5. Re-etimate GMM parameter 6. Repeat tep 2 through 5 until (25) i atified. 8 Experiment Reult Data et: NiCrAl 1 kev Simulation, ource from BlueQuartz Software. Data Decription: The data et conit of 257 lice of 513x513 pixel TIFF image. Each image repreent a 15 nm thick lice, and the pixel dimenion are alo 15x15 nm. Microcope parameter i 1 kev. Experiment condition: The algorithm i implemented in C. The experiment for thi particular dataet i performed on a PC with 2.0GHz clock peed, 1.5GB RAM. The running time of one iteration i econd including tep 1 to 4 decribed above, importing and exporting image. Segmentation reult of the 201 t to 205 th image are provided a follow. The three different reult are obtained by applying 2D SMAP, 3D SMAP without GMM parameter re-etimation and 3D SMAP with one iteration of GMM parameter re-etimation.

12 (a) (b) (c) (d) Figure 5: 201(a)Orignial image; (b)smap; (c)3d SMAP; (d)3d SMAP uing re- etimated GMM parameter with number of iteration equal one

13 (a) (b) (c) (d) Figure 6: 202(a)Orignial image; (b)smap; (c)3d SMAP; (d)3d SMAP uing re- etimated GMM parameter with number of iteration equal one

14 (a) (b) (c) (d) Figure 7: 203(a)Orignial image; (b)smap; (c)3d SMAP; (d)3d SMAP uing re- etimated GMM parameter with number of iteration equal one

15 (a) (b) (c) (d) Figure 8: 204(a)Orignial image; (b)smap; (c)3d SMAP; (d)3d SMAP uing re- etimated GMM parameter with number of iteration equal one

16 (a) (b) (c) (d) Figure 9: 205(a)Orignial image; (b)smap; (c)3d SMAP; (d)3d SMAP uing re- etimated GMM parameter with number of iteration equal one

17 9 reference [1] C. A. Bouman and M. Shapiro, A multicale random eld model for Bayeian image egmentation, IEEE Tran. Image Proceing 3: , [2] C. A. Bouman, M. Shapiro. G. W. Cook, C. B. Atkin, H. Cheng, Cluter: An unupervied algorithm for modeling gauian mixture, In cluter [3] C. A. Bouman, B. Liu, Multiple Reolution Segmentation of Textured Image, IEEE Tran. Pattern Analyi and Machine Intelligence, vol. 13, no. 2, pp , Feb