Tracking the Left Ventricle through Collaborative Trackers and Sparse Shape Model

Transcription

1 Tracking the Left Ventricle through Collaborative Trackers and Sparse Shape Model Yan Zhou IMPAC Medical Systems, Elekta Inc., Maryland Heights, MO, USA Abstract. Tracking the left ventricle plays an important role in cardiac diagnosis and surgeries. One challenge is the significant cardiac wall motion modulated by blood flow, breathing and respiratory movements. We propose a framework to combine appearance-based collaborative trackers with a sparse shape model. On one hand, a coarse contour is generated based on a collaborative network of trackers at every time step. On the other hand, a sparse shape model is used to refine the coarse contour on the fly, and eventually obtain accurate contours of the left ventricle in real-time. The approach is validated on 14 MRI sequences from two types of acquisition protocols, with comparisons to the state-of-the-art approaches. Experimental results show that our approach is able to provide a robust, accurate and fast solution to track the left ventricle of the beating heart. 1 Introduction Of the four chambers in the human heart, the left ventricle (LV) plays an important role in that it receives oxygenated blood from the left atrium via the mitral valve, and pumps it into the aorta via the aortic valve. Delineation of the left ventricle from MRI sequences, especially the inner boundary of the left endocardium, is crucial in quantifying ejection fraction, which measures the percentage volume of blood transmitted out of LV in a given cardiac cycle. In addition, real-time MRI emerges as a feasible modality for guiding interventions and surgeries on the beating heart[1]. Path planning tasks for intra-cardiac procedures require fast and accurate endocardium segmentation throughout the entire cardiac cycle. Quantification of the cardiac motion also helps physicians to evaluate the health of the myocardial muscles. All of the above tasks require an automatic, robust, accurate and fast algorithm to extract boundaries of the LV from MRI sequences. Accurate contour tracking of the LV is non-trivial because of the significant cardiac wall motion modulated by blood flow, breathing and respiratory motion [2]. In particular, blood that flows into the ventricles in high speed produces sudden higher intensity areas in the acquired images, making appearance-based trackers highly unreliable. The non-linear deformations of the LV caused by breathing are disastrous for shape models to delineate the cardiac wall boundary, due to large shape variation. Additionally, chambers move up and down because of respiratory motion, thus the periodical cardiac motion becomes irregular. All

2 the mentioned difficulties pose great challenges for shape-based or appearancebased tracking approaches. Shape-based tracking methods often incorporate either statistical shape models or deformable models. Adaptive Shape Model ASM [3]is a typical statistical shape model. Firstly, all the training shapes are aligned using similarity transformation. Then their statistical pattern is captured by principal component analysis (PCA). Given an incoming image, the model is able to constrain the shape deformation to accommodate variability. However, the learned statistical pattern averages the training images so that some important details are missing. Occasionally, the resulted shapes are over-smoothed. Deformable models are curves or surfaces that deform under the influence of internal smoothness and external image forces to delineate the object boundary [4 7]. Performance of such methods relies on good initializations. If the initialization is poorly done [6] or there is no overlap between the initialization with the target region [7], the methods would fail. The computational cost can be expensive as well. Appearance-based collaborative tracking methods [8][9][10] utilize appearance of local image patches and their context to ensure stability. Comparing with deformable models, using local image patches is computationally light weighted, and quite flexible when applying to different organs. For example, a cardiac tracker can be directly used as a lung tracker without specifically learning a lung model. However, these tracking methods aim at predicting linear transformation of the whole region instead of computing non-linear deformations of the object boundary. Therefore, it still requires a shape model to refine the contour. We propose a framework to integrate appearance-based collaborative tracking with sparse shape model. On one hand, a collaborative trackers network provides a linear interpolated contour at each time step [8]. Each tracker in the network is a particle filter tracker featuring the appearance of the tracking region as the template. The coordination of the trackers in the network is modelled by Bayesian network. The estimated states from trackers controls the contour interpolation. On the other hand, a sparse shape model serves as shape prior to refine the coarse contour, and iteratively obtains a smoothed contour. Our approach has the following advantages: 1) The tracking performance of the collaborative trackers network not only depends on the local appearance of individual trackers, but also depends on its neighbouring trackers, namely its context. Thus the tracking is robust. 2) Unlike ASM which smooths the trained contours, sparse shape model is superior in preserving shape details even under large shape variation. 3) The approach is computationally light weighted. It takes 0.32 second on average to process one image on a 2.4GHz desktop. 2 Methodology The goal of the approach is to accurately estimate the contour at each time step. For this purpose, we first use a collaborative tracking network to track the LV (Figure 1 (a)). An initialized contour is also provided for the first frame. For the subsequent frames, an interpolated contour is automatically generated by

3 Collaborative Trackers Contour Interpolation Edge Detection Sparse Shape Refinement (a) (b) (c) (d) Fig. 1. Algorithmic framework, which includes the tracking network and shape prior modules. linearly interpolating the initialized contour according to the motion parameters (vertical and horizontal translations, and rotation)of each tracker (Figure 1 (b)). The interpolated contour is then driven to the boundary by an edge detection algorithm (Figure 1 (c)). Finally a sparse shape model is applied to refine the edge detection results ((Figure 1 (d))). 2.1 Collaborative Tracking Collaborative tracking network utilizes a network of simple trackers to track a region. When some trackers in the network fail due to abrupt motion or appearance change, other trackers that have been less affected produce good performance, and are able to re-initialize the failing trackers. The interaction behaviour of the neighbouring trackers is modelled by Bayesian Network [8]. The network structure of trackers not only models the local appearance information, but also models the context of each tracker through its neighbours. The collaborative trackers network is composed of n particle filter trackers, n = 4 for the case in Figure 1. At time t, for each individual tracker, we measure its tracking performance by computing its surviving probability in Equation 1. p(s t i, ˆΘ t i B t 1 i, Zi t ) p(b t 1 i ) k N(ˆθ t i ˆθ t k, σ 2 ) k p(ˆθ t k z t k) (1) ˆΘ i t = {ˆθ i t, ˆθ j t,..., ˆθ m} t and Zi t = {zt i, zt j,..., zt m} are the estimated states and observations of tracker iand its adjacent trackers. Si t represents the event that tracker i survives. B t 1 i is the Bayesian network at time t 1, whose probability p(b t 1 i ) is known at time t. N(ˆθ i t ˆθ k t, σ2 ) is the probability density of ˆθ i t on the Normal distribution centred at ˆθ k t with variance σ2. Detailed explanations can be found in [8]. 2.2 Contour Interpolation and Edge Detection We initialize a contour of the LV inside the tracking region at the first time step. At every time step, each tracker in the network provides the state estimation of

4 its local patch. We then use it to linearly interpolate the part of contour inside this tracker (Fig. 1 (b)). To move the interpolated contour to the edge, we search on the normal direction of each point of the interpolated contour, and found the pixel that has the largest gradient. The resulted edge looks very noisy (Fig. 1 (c)), since there is no smoothing mechanism involved in the edge detection step. That s why we need to further refine the shape. 2.3 Sparse Shape Refinement A popular approach of shape prior modelling is based on Principal Component Analysis (PCA). The shape database is represented as a combination of its mean shape and major variations. This representation is not robust to outliers, and the shape details cannot be preserved if they are not statistically significant. In our framework, we use sparse representation based shape prior [11][12], which can alleviate these above-mentioned problems. Instead of learning a generative shape model, this shape prior is incorporated on-the-fly through the sparse shape composition. Specifically, a sparse set of shapes in the shape repository is selected and composed together to refine an input shape (i.e., the intermediate result). Then this composed shape is used to replace the input one. The a-priori information is thus implicitly incorporated on-the-fly. This model leverage two sparsity observations of the input shape instance: 1) the input shape can be approximately represented by a sparse linear combination of shapes in the shape repository; 2) parts of the input shape may contain gross errors but such errors are sparse. Mathematically, this model is formulated as a sparse learning problem: arg min x,e,β T (v, β) Dx e 2 2, s.t. x 0 < k 1, e 0 < k 2, (2) where T (v, β) is a global transformation operator with parameter β, which aligns the input shape v to the common canonical space of D. x R K denotes the coefficient/weights of linear combination. The L0-norm of x ensures that the number of non-zero elements in x is less than a sparsity number (i.e., k 1 or k 2 ). In other words, only a sparse set of shape instances can be used to approximate the input shape, which prevents the over-fitting to errors from missing/misleading appearance cues. e R DN is a vector that models the large residual errors. The sparsity constraint is imposed on e to incorporate the observation that gross errors might exist but are occasional. Using L1 norm relaxation, it can be solved by an efficient expectation-maximization (EM) type of framework. Compared to traditional statistical shape models, e.g., active shape model, this method is able to remove gross errors from local appearance cues and preserve shape details even if they are not statistically significant. 3 Experiments Tracking the left ventricle is very useful in many medical applications such as ejection fraction computation and interventional robotics surgeries. For example,

5 in MRI-guided robotic interventions [1], the extracted contours would guide the robotic needle to go through the valve without touching the cardiac wall. So the segmented contours should be highly accurate. In addition, the computing should be fast to aid for surgeries on-the-fly. In our experiments, the performance is evaluated in terms of accuracy and computation time. Experimental data was acquired by two types of MRI acquisition protocols: (a) CINE MRI sequences where patients hold the breath during MRI scanning; This type of MRI sequences contains relatively few slices (12-25 frames) since the patient is not able to hold the breath for long time. (b) Real-time MRI sequences where patients have natural breathing during MRI scanning. This type of MRI sequences contains more slices ( frames) and the cardiac motion pattern is more complicated because of the additional respiratory motion. We performed experiments on 14 MRI sequences from 7 patients, with slices (frames) for each sequence and 1728 slices (frames) in total. From all the images, we randomly selected 150 images for training, and tested on the rest of images. We compare the following state-of-art methods: Interpolated contour approach (IC) [8]: The contour is computed by interpolating the initialized contour according to the current state of the deformation mesh. Edge detection approach (ED): This approach simply computes the edge along the normal direction of the interpolated contour. Active shape model [3] (ASM): Applying ASM on the intermediate tracking results provided by the collaborative tracking network. Sparse shape model (SSM): Our proposed approach. Fig.2 shows the results from a long real-time sequence. (a) shows the result from the interpolated contour. The contour was linearly deformed according to each tracker, thus can t accurately describe the non-linear LV deformation. Most part of the resulted contour lies off the edge. (b) shows the result from edge detection, which is very noisy. Because it only detects local minimums. (c) shows the result from our approach. Using ASM provides similar visual results as using sparse shape model in this case. So we omit it in the figure. Fig. 3 shows the quantitative comparisons of results from using 1) interpolated contour, 2) edge-based detection, 3) ASM-type of shape prior, and 4) sparse shape model. The results were generated from a sequence of 150 images. The ground-truth contours were annotated by an expert. We report the sensitivity, the specificity, and the dice similarity coefficient (DSC). The sensitivity shows the accuracy of the segmented foreground, while the specificity shows background. DSC represents both and is a more comprehensive measurement. As shown in this figure, the interpolated contour method has the lowest DSC. The reason is that the linear interpolation method ignores the true boundary location. Edge detection can slightly improve the result, since each point on the contour tries to move toward the image boundary without being misled by others. However, the resulting boundary is not continues as there is no shape constraint. ASMtype of shape prior can significantly improve the accuracy, since it constrains the deformed shape by following the patterns of training data. The sparse shape

6 (a) (b) (c) Fig. 2. Comparative results on a real-time testing sequence with 750 frames. (a) Results from interpolated contour approach. (b) Results from edge detection. (c) Results from the proposed method. The rows corresponds to frame index 239, 251, 409. Sensitivity Specificity DSC Fig. 3. Boxplots of sensitivity, specificity and DSC of all testing data. In each subplot, y-axis is the performance of sensitivity, specificity or DSC. x-axis aligns 4 methods 1) IC 2) ED 3) ASM 4) SSM.

7 model achieves comparable performance, and is even slightly better. The reason is that sparse shape model is able to handle erroneous edge detection, and is able to preserve shape details. In general, the sparse shape model achieves the best accuracy in this application. We evaluate the computing time of the proposed approach by averaging the processing time of all testing cases. It takes 0.32 second to process one image on a 2.4GHz desktop, while using ASM takes 0.17 second, which is a little faster. Both are promising for real-time applications. The reason is that particle filter trackers in the tracking network are computationally light weighted. And the produced contours are in good quality to serve as initializations for the shape refinement tasks. 4 Conclusion Integrating the tracking network and the sparse shape model is more robust to tissue motion, and can handle complex organ boundaries than purely using the tracking algorithm. Our model converges quickly and robustly towards the boundary because of the appearance-based collaborative tracking results and the constraint of the sparse shape model. The size of the training set can be small (150 training images in our case), because the tracking algorithm already enforces both spatial and temporal consistency. Another advantage is that the sparse shape model preserves detail structures of the boundaries, when comparing with traditional ASM. References 1. Yeniaras, E., Navkar, N.V., Sonmez, A.E., Shah, D.J., Deng, Z., Tsekos, N.V.: MR-based real time path planning for cardiac operations with transapical access. MICCAI (2011) Zhou, Y., Peng, Z., Zhou, X.S.: Automatic view classification for cardiac MRI. The IEEE International Symposium on Biomedical Imaging (2012) 3. Cootes, T.F., Taylor, C.J., Cooper, D.H., Graham, J.: Active shape models: Their training and application. CVIU 61 (1995) Kohlberger, T., Cremers, D., Rousson, M., Ramaraj, R., Funka-Lea, G.: 4D shape priors for a level set segmentation of the left myocardium in SPECT sequences. MICCAI (2006) McEachen, J.I., Duncan, J.: Shape-based tracking of left ventricular wall motion. TMI 16(3) (1997) C.Li, C.Xu, C.Gui, M.D.Fox: Level set evolution without re-initialization: A new variational formulation. 1 (2005) Thirion, J.: Image matching as a diffusion process: an analogy with maxwell s demons. Medical Image Analysis 2(3) (1998) Zhou, Y., Yeniaras, E., Tsiamyrtzis, P., Tsekos, N., Pavlidis, I.: Collaborative tracking for MRI-guided robotic intervention on the beating heart. MICCAI (2010) Yang, M., Wu, Y., Hua, G.: Context-aware visual tracking. IEEE Trans. on Pattern Analysis and Machine Intelligence 31(7) (2009)

8 10. Zhou, Y., Zhang, S., Tsekos, N., Pavlidis, I., Metaxas, D.: Left endocardium tracking via collaborative trackers and shape prior. The IEEE International Symposium on Biomedical Imaging (2012) 11. Zhang, S., Zhan, Y., Dewan, M., Huang, J., Metaxas, D.N., Zhou, X.: Sparse shape composition: A new framework for shape prior modeling. IEEE Conference on Computer Vision and Pattern Recognition (2011) 12. Zhang, S., Zhan, Y., Dewan, M., Huang, J., Metaxas, D., Zhou, X.: Towards robust and effective shape modeling: Sparse shape composition. Medical Image Analysis 16(1) (2012)