A CONDITIONAL RANDOM FIELD MODEL FOR TRACKING IN DENSELY PACKED CELL STRUCTURES. Anirban Chakraborty, Amit Roy-Chowdhury

Transcription

1 A CONDITIONAL RANDOM FIELD MODEL FOR TRACKING IN DENSELY PACKED CELL STRUCTURES Anirban Chakrabory, Ami Roy-Chowdhury Deparmen of Elecrical Engineering, Universiy of California, Riverside, USA ABSTRACT Auomaed racking of plan and animal cells in ime lapse live-imaging daases of developing mulicellular issues is required for quaniaive, high hroughpu analysis of cell division, migraion and cell growh. In his paper, we presen a novel cell racking mehod ha explois he igh spaial opology of neighboring cells in a mulicellular field as conexual informaion and combines i wih physical feaures of individual cells for generaing reliable cell lineages. The 2D image slices of mulicellular issues are modeled as CRFs and spaio-emporal cell o cell correspondences are obained by performing inference on his CRF using loopy belief propagaion. We presen resuls on a 3D+) confocal image sack of Arabidopsis shoo merisem and show ha he mehod can handle many visual analysis challenges associaed wih such cell racking problems, viz. poor feaure qualiy of individual cells, low SNR in pars of images, variable number of cells across slices and cell division deecion. Index Terms Cell racking, Condiional Random Field, Spaial conex, Live cell imaging. 1. INTRODUCTION In developmenal biology, he causal relaionship beween cell growh paerns and gene expression dynamics has been one of he major opics of ineres. A proper quaniaive analysis of he cell growh and division paerns in boh he plan and he animal issues has remained mosly elusive so far. Towards his goal, wih he advancemens in microscopy and oher imaging echniques, ime lapse videos are being colleced o quanify he behavior of hundreds of cells in a issue over muliple days. For high-hroughpu analysis of hese large volumes of image daa, developmen of fully auomaed image analysis pipelines are becoming necessiies, hereby giving rise o many new auomaed visual analysis challenges. Auomaed cell racking wih cell division deecion is one of he major componens of all such pipelines such as [1]). The compuaional challenges relaed o a robus design of cell racker come from muliple sources such as variable number of cells in he field of view FoV), deformaion of Corresponding auhor, amirc@ee.ucr.edu cell shapes, complex opologies of cell clusers, low SNR in he images, ec. In his paper, we presen an auomaed visual racker for cells ighly packed in developing mulilayer issues. This calls for developing sraegies for emporal associaions of he cells. Moreover, since a every ime poin of observaion a cell could be imaged across muliple spaial planes, he racking mehod mus be capable of finding correspondences in he spaial direcion as well. Beyond hese, he racker has o be able o deec cell divisions, deec new cells as he deeper layers of he issues are imaged, differeniae beween cells in a close neighborhood sharing similar physical feaures and generae correc maches in presence of low SNR. There has been some work on auomaed racking and segmenaion of cells in ime-lapse images, for boh plans and animals. One of he well-known approaches for segmening and racking cells is based on evoluion of acive conours [2, 3, 4, 5, 6]. Bu his mehod is no suiable for racking where all he cells are in close conac wih each oher and share very similar physical feaures, nor is here any repored resul on spaial correspondence. The Sofassign mehod uses he informaion on poin locaion o simulaneously solve boh he problem of global correspondence as well as he problem of affine ransformaion beween wo ime insans ieraively [7, 8]. However, hese mehods are more suiable for aligning global feaures han finding correspondences beween nonuniformly growing individual cells. Alhough [8] presens a sample resul on a shoo merisem wihou validaing agains ground ruh, i is no enough o evaluae he accuracy of his mehod on a ypical 4D confocal daa. Besides he aforemenioned approaches, racking based on associaion beween deecions such as [9, 10, 11] has shown good performance on ime-lapse images. However, hese mehods perform well when he feaure qualiy or he underlying moion model is reliable. We are looking a a more challenging problem, where he feaures exraced from each cell may no be reliable or discriminaing enough such as cell shape/area which is ofen sereoypical even in a small cluser). In [12, 13], a spaio-emporal racking algorihm for Arabidopsis SAM was proposed, where relaive posiional infor- This work was parially suppored by NSF gran IIS

2 Waershed segmenaion Compuaion of node and edge poenials and inference using loopy BP Cell division deecion No Mach Z Z Raw spaio-emporal image sack Formaion of condiional random field on pairs of images along z and ) Spaio-emporal cellular correspondences Fig. 1. Proposed cell racking framework - differen sequenial componens in he proposed mehod. maion of neighboring cells was used o generae unique feaures for each cell. However, his mehod employs an ieraive search sraegy by growing correspondence from a seed cell pair which ends o accumulae error and can hrow he racker off for cells spaially disan from he seed. In his work, we propose o solve he spaio-emporal racking problem as a graph inference problem. To rack cells beween wo image slices consecuive in ime or space, we build a graph on one of he images wih individual cells as he nodes and neighboring nodes sharing an undireced edge beween hem. We furher define a Condiional Random Field CRF) on he graph, he probable saes of each node being he candidae cell correspondences from he nex image. A disance defined on he physical feaures exraced from a cell and ha of each of is candidae maches is used o consiue he node poenial. The spaial conex is modeled on each of he edges based on he relaive locaion of he cell and is neighbors by uilizing he igh spaial opology of he cell clusers. We obain he correspondences by maximizing he marginal disribuion compued a each node cell). The approximae marginals are obained by a Loopy Belief Propagaion scheme. The overall racking pipeline is shown in Fig SPATIO-TEMPORAL CELL TRACKING METHOD 2.1. Pre-processing: Segmenaion and Regisraion The inpu o our cell racking sysem is a 3D+) image sack, which is a collecion of 2D image slices along he deph of a issue and along muliple ime poins of observaion. A 2D segmenaion echnique such as Waershed as in [14]) is employed o segmen ou individual 2D cell cross secions on each of he image slices. The 3D sacks are furher regisered emporally using a local graph based regisraion scheme [15] Graph Formaion on 2D Segmenaions Le us define he problem o be o find correspondences beween he cells in wo confocal image slices I G and I M. The Waershed segmenaion of I G and I M produces wo ses of cell segmens Ω G and Ω M respecively and O = Ω G Ω M. For emporal racking, we firs deec if some cells form I G have divided ino pairs of cells in I M following he mehod described in Sec. 2.4 and remove he paren cells ha have undergone division from Ω G and he divided children from Ω M. The graph and he candidae saes of each node of he graph are hereafer formed using he remaining subses of cells V G and V M conaining N G and N M cells respecively, i.e. he remaining cells v 1 G, v2 G, vn G G V G Ω G and vm 1, v2 M, vn M M V M Ω M. The graph is buil on I G and he se of nodes V G is same as he se of segmened cells sans he cells undergoing division. Any wo nodes vg i and v j G will have an edge beween hem if vi G and vj G are spaial neighbors Fig. 1). For ighly packed cluser of cells, vg i and v j G are neighbors if hey share a common boundary or if hey are wihin close spaial proximiy for non-compacly arranged cellular essellaions Deerminaion of Candidae Saes For Every Node Each node in he graph, corresponding o a cell slice vg i represens a random variable x i ha can ake a label s i k from he se SG i which is he se of K closes segmens in he slice I M around he poin c i G, he cenroid of vi G on I G. Now, we add an addiional label s i 0 o he candidae se SG i ha represens he case where he cell slice vg i is no imaged in he slice I M. Thus, he complee se of candidae saes becomes SG i = {si 0, s i 1, s i K }.

3 2.4. Cell Division Deecion To deec cell divisions before forming he graph g G in emporal racking, we firs compue he candidae ses CG i in I M for a segmened cell slice ωg i Ω G following similar mehod as in Sec Nex we form all possible pairs of he candidae { cells from CG i ha share a boundary as in } Di G = cd i p, cd i q) s.. cd i p nborcd i q) and cd i p, cd i q CG i. Now, if he cell ωg i has divided ino wo children cells cdi p and cd i q, hen ideally he shape of ωg i should be very similar o he combined shape of cd i p and cd i q, aken ogeher i.e. o he shape of cd i p cd i q)) and each of cd i p and cd i q would be approximaely half he size of ωg i. Moivaed by his physical propery associaed wih cell division, we compue a se of disances as dωg i, Di G ) = { 1 1 MHDbωG [ i ), bcdi p cd i ] q)) areacdi p ) areaω + 1 G i ) 2 areacdi q ) areaωg i ) for all cd i p, cd i q) DG i }, where b is he se of boundary poins on a shape recompued wih respec o is cenroid and MHD is he modified Hausdorff disance. If min dωg i, Di G ) 1, hen i is inferred ha he cell ωg i has divided ino he cell pair in DG i for which his minimum is obained. The values of he parameers 1 and 2 are learned from a small raining se Condiional Random Field Modeling Le he se of random variables associaed wih V G be X = {x 1, x 2, x NG }, which are o be esimaed given he observaion I M. These random variables correspond o he sae of each node in he graph and he suppor for each of hese variables is he candidae se as discussed in Sec These variables are modeled as a Condiional Random Field CRF) and he node and edge poenial funcions associaed wih his CRF are compued via he following echniques Compuaion of Observaion/Node Poenial: The node poenial is defined on every node of he graph, which is he likelihood on he label aken by a node belonging o V G, given he observaion O. This poenial is compued independenly for each node based on is shape similariies wih each of is candidaes. For measuring similariies beween cell shapes, we generae a shape hisogram descripor for each of he cells, which is very similar o one of he mehods described in [16]. Le he shape hisogram associaed wih he cell slice v i G be hi G and ha wih he candidae slice s i j be hj M as si j V M ). We compued he K-L divergence KLD) beween h i G and hj M which gives us a disance measure beween hese wo cell slices and suppose i is represened as d i v i G, si j). The corresponding node poenial for each node is φ i xi = s i j; O ) = exp d i v i G, s i j)/λ ) j = 1, 2 K 1) φ i xi = s i 0; O ) { = 1 max φ i xi = s i j; O ) }, j = 1, K j 2) 2.7. Compuaion of Spaial Conex / Edge Poenial: This poenial funcion is defined on edges connecing pairs of neighboring nodes and is represenaive of he condiional disribuion P x j x i, O). The compuaion of he poenial funcion depends on he fac ha if wo neighboring cells vg i and v j G are racked o wo cell slices vp M and vq M, hen he relaive posiion of v j G wih respec o vi G should be very similar o ha of v q M and vp M. As a resul, if vi G is racked o vp M hen he probabiliy ha v j G corresponds o vq M ges boosed if c j G ci G cq M cp M, where ci G, cj G, cp M, cq M be he cenroids of vg i, vj G, vp M, vq M respecively. Clearly, he addiional evidences for maching wo cell slices in I G and I M come in he form of local neighborhood srucure based conexual informaion. Thus, he conexual ransiion poenials beween any wo nodes vg i and vj G aking non-zero saes can be expressed as a funcion of he shif beween he relaive posiions of hose nodes ψ i,j xi = s i p, x j = s j q; O ) { } = exp γ c j G ci G) c sj q M csi p M ) 2 p, q = 1, 2, K, where s i p S i G, sj q S j G. Now, he ransiion poenials mus also incorporae he case where one of he cells is no racked and is neighboring cell is mached o one of he cells in he nex slice or no mached o any cell. This poenial can be aken as uniform over he suppor of he neighboring cell. ψ i,j xi = s i 0, x j = s j q; O ) 1 = q = 0, 1, K. K + 1 4) Finally, when vg i has a mach in he nex spaial or emporal image slice I M, bu is neighbor v j G does no, hen he corresponding edge poenial enries become ) ψ i,j x i = s i p, x j = s j 0 ; O { = 1 max ψ i,j xi = s i p, x j = s j q; O ) }, q = 1, 2, K q 5) for p Inference: Loopy Belief Propagaion The nex sep is o do he inference on he CRF, which involves he compuaion of he marginal probabiliy disribuions for he saes x i of each node v i G V G, given he observaions O. For compuaion of he marginals a each node, we 3)

4 Accuracy 0.9 Our cell racker Local graph based racker [12] 0.75 Shape based baseline racker Spaial Tracking Temporal Tracking Cell Division Deecion Fig. 3. Quaniaive comparison of racking accuracies obained by proposed mehod, mehod in [12] and he baseline racker on he enire daase. Table 1. Tracking Resul Summary Fig. 2. Resuls showing combined spaio-emporal racking on Arabidopsis SAM daase bes seen in color). choose o use a very popular ieraive message-passing algorihm known as Loopy Belief Propagaion LBP) based on he Sum-Produc algorihm [17]. If LBP converges a ieraion L, he esimaed marginals a each node would be P L) xi ; O) and he MAP esimae for he mos likely sae is compued as x i = argxi max P L) xi ; O). This opimum sae corresponds o eiher he no-mach case or a specific cell in IM. 3. TRACKING RESULTS AND ANALYSIS We have esed our proposed cell racking mehod on a 4D confocal sack of Arabidopsis shoo apical merisem SAM) ha showcases all he challenges associaed wih any spaioemporal cell racking problem in a ighly packed mulilayer issue. The 3D srucure of he issue is imaged using singlephoon confocal laser scanning microscope, hereby generaing a series of serial opical image slices of cells only he cellwalls are visible). Observaions are aken every hree hours o yield a 3D+) confocal sack. We perform spaial cell racking across he deph of he 3D confocal image sacks and combine hem wih emporal racking of he same cells across ime. In Fig. 2, we sample hree consecuive spaial slices from confocal sacks a 4 differen ime poins a 12h, 15h, 18h and 21s hours of observaion) and he racking resul for hem are shown. The 2D slices coming from he same 3D cell are correcly racked for all he cells across 4 differen ime insans and are marked wih he same color. The children cells afer division are marked by red dos. I can be observed ha slices of new cells appear as we go deeper ino he issue and as expeced, hey are no mached o any cell from he slice above. The complee racking resul on he daase 12 ime poins and 7 slices a each ime poin) is summarized quaniaively in Table 1. TP corresponds o he cases where wo cell Spaial Temporal Division TP 86% 83% 31/33 FP 0.25% 0% 0 TN 12.13% 14.66% - FN 1.62% 2.34% 2/33 slices are correcly mached eiher in space or ime. When cell slices from wo differen cells are incorrecly mached ogeher, i falls under FP. When he racker fails o pick up a correc correspondence, i is represened by FN and is opposie case is abulaed under TN. In boh spaial and emporal racking, he accuracy is more han 97%. A full quaniaive comparison of racking accuracies obained by proposed mehod, mehod in [12] and he baseline racker on he enire daase is presened in Fig. 3. We designed he baseline racker on he same local cell shape feaures as used o compue he node poenials in Sec. 2.6 and he racker associaes cell slices across images using Hungarian algorihm. I can be observed from Fig. 3 ha our proposed mehod subsanially ouperforms boh [12] and he baseline racker, especially in spaial racking. We have also compared he resuls obained by he proposed mehod and he mehod in [12] in deecing cell divisions and observed ha he proposed mehod marginally ouperforms [12]. 4. CONCLUSION We have presened a mehod for auomaically racking individual cells in closely packed developing mulilayer issues. We observed ha cells in a close cluser in he issue can have very similar image feaures and hence we leveraged upon he local spaial geomeric srucure and opology of he relaive posiions of he neighboring cells o robusly rack growing cells in he issue in presence of imaging noise. Fuure work would include he inegraion of his spaio-emporal racking mehod wih oher image analysis componens such as a cell resoluion 3D reconsrucion mehod [18] o design a complee 4D image analysis pipeline.

5 5. REFERENCES [1] Romain Fernandez, Pradeep Das, Vincen Mirabe, Eric Moscardi, Jan Traas, Jean-Luc Verdeil, Gregoire Malandain, and Chrisophe Godin, Imaging plan growh in 4d: robus issue reconsrucion and lineaging a cell resoluion, Naure Mehods, vol. 7, no. 7, pp , [2] O. Dzyubachyk, W.A. Van Cappellen, J. Essers, W.J. Niessen, and E. Meijering, Advanced level-se-based cell racking in ime-lapse fluorescence microscopy, Medical Imaging, IEEE Transacions on, vol. 29, no. 3, pp , [3] Kang Li and Takeo Kanade, Cell populaion racking and lineage consrucion using muliple-model dynamics filers and spaioemporal opimizaion, in Proceedings of he 2nd Inernaional Workshop on Microscopic Image Analysis wih Applicaions in Biology, [4] Kang Li, Mei Chen, Takeo Kanade, Eric Miller, Lee Weiss, and Phil Campbell, Cell populaion racking and lineage consrucion wih spaioemporal conex, Medical Image Analysis, vol. 12, no. 5, pp , [5] Dirk R. Padfield, Jens Rischer, Nick Thomas, and Badrinah Roysam, Spaio-emporal cell cycle phase analysis using level ses and fas marching mehods., Medical Image Analysis, vol. 13, no. 1, pp , [6] A. Dufour, V. Shinin, S. Tajbakhsh, N. Guillen-Aghion, J. C. Olivo-Marin, and C. Zimmer, Segmening and racking fluorescen cells in dynamic 3-D microscopy wih coupled acive surfaces, IEEE Transacions on Image Processing, vol. 14, no. 9, pp , [7] Haili Chui and Anand Rangarajan, A new algorihm for non-rigid poin maching, in in CVPR, 2000, pp [8] Vicoria Gor, Michael Elowiz, Tigran Bacarian, and Eric Mjolsness, Tracking cell signals in fluorescen images, IEEE Compuer Sociey Conference on Compuer Vision and Paern Recogniion Workshops, vol. 0, pp. 142, [9] Nezamoddin N. Kachouie, Paul Fieguh, John Ramunas, and Eric Jervis, Probabilisic model-based cell racking, Inernaional Journal of Biomedical Imaging, Aerospace and Elecronic Sysems, vol. 37, no. 1, pp. 2 21, [11] Ryoma Bise, Zhaozheng Yin, and Takeo Kanade, Reliable cell racking by global daa associaion, in IEEE Inernaional Symposium on Biomedical Imaging: From Nano o Macro, 2011, pp [12] Min Liu, Ram Kishor Yadav, Ami Roy-Chowdhury, and G. Venugopala Reddy, Auomaed racking of sem cell lineages of arabidopsis shoo apex using local graph maching, Plan journal, Oxford, UK, vol. 62, pp , [13] Min Liu, Anirban Chakrabory, Damanpree Singh, Ram Kishor Yadav, Gopi Meenakshisundaram, G Venugopala Reddy, and Ami Roy-Chowdhury, Adapive cell segmenaion and racking for volumeric confocal microscopy images of a developing plan merisem, Molecular Plan, vol. 4, no. 5, pp , [14] Kaya Mkrchyan, Damanpree Singh, Min Liu, G. Venugopala Reddy, Ami K. Roy Chowdhury, and M. Gopi, Efficien cell segmenaion and racking of developing plan merisem, in IEEE Inernaional Conference on Image Processing, 2011, pp [15] Kaya Mkrchyan, Anirban Chakrabory, and Ami K. Roy-Chowdhury, Auomaed regisraion of live imaging sacks of arabidopsis, in Biomedical Imaging ISBI), 2013 IEEE 10h Inernaional Symposium on, 2013, pp [16] Mihael Ankers, Gabi Kasenmüller, Hans-Peer Kriegel, and Thomas Seidl, 3d shape hisograms for similariy search and classificaion in spaial daabases, in Proceedings of he 6h Inernaional Symposium on Advances in Spaial Daabases, 1999, pp [17] Frank R. Kschischang, Brendan J. Frey, and Hans- Andrea Loeliger, Facor graphs and he sum-produc algorihm, IEEE Transacions on Informaion Theory, vol. 47, pp , [18] Anirban Chakrabory, Mariano M. Perales, G. Venugopala Reddy, and Ami K. Roy-Chowdhury, Adapive geomeric essellaion for 3d reconsrucion of anisoropically developing cells in mulilayer issues from sparse volumeric microscopy images, PLoS ONE, vol. 8, no. 8, pp. e67202, [10] Thiagalingam Kirubarajan, Yaakov Bar-Shalom, and Krishna R. Paipai, Muliassignmen for racking a large number of overlapping objecs, IEEE Trans. on