A Bilinear Model for Temporally Coherent Respiratory Motion

Transcription

1 A Bilinear Model for Temporally Coherent Respiratory Motion Frank Preiswerk (B) and Philippe C. Cattin Medical Image Analysis Center, University of Basel, Basel, Switzerland Abstract. We propose a bilinear model of respiratory organ motion. The advantages of classical statistical shape modelling are combined with a preconditioned trajectory basis for separately modelling the shape and motion components of the data. The separation of a linear basis into bilinear form leads to a more compact representation of the underlying physical process and the resulting model respects the temporal regularity within the training data, which is an important property for modelling quasi-periodic data. Bilinear modelling is combined with a Bayesian reconstruction algorithm for sparse data under observation noise. By applying the model to liver motion data, we show that our bilinear formulation of respiratory motion is significantly more parsimonious and can even outperform linear PCA-based models. Keywords: Respiratory motion Bilinear model Liver motion 1 Introduction Modelling respiratory organ motion is an active field of research. Having an accurate model of respiratory motion is desirable for many practical applications including motion segmentation, registration, tracking and reconstruction as well as tumour tracking in a clinical scenario. A comprehensive overview of the field is available in [1]. Such models can serve different purposes, two of which are worth mentioning here. First, respiratory motion models are used to estimate the organ position for a given point in time, e.g. based on sparse measurements. Second, they are used to predict the future position based on current and/or past values. Respiratory motion is approximately repetitive, thus it is reasonable to take the repetitive nature into account when designing a model. While respiratory motion is very similar from cycle-to-cycle, there are variations that must be taken into account. On the other hand, we wish the model not to be able to describe arbitrary motion patterns. This trade-off between generalisation and specificity is typically encountered in model selection. Recently, statistical motion models which are learned from data, typically by employing Principal Component Analysis (PCA) techniques, have been proposed in various studies [2 7]. In [4], a kernel-pca is learned to model the relationship between fiducial movement and lung surface motion. In [7] PCA, is computed c Springer International Publishing Switzerland 2014 H. Yoshida et al. (Eds.): ABDI 2014, LNCS 8676, pp , DOI: /

2 222 F. Preiswerk and P.C. Cattin on dense motion fields extracted from 4DMRI of the liver through non-rigid registration. In [2], the coefficients of a PCA model are further restricted by a single governing parameter as prior which models the distribution of the observed data s PCA coefficients. Although all mentioned approaches provide a compact, low-dimensional parameterisation of valid shapes within a breathing cycle, they do not take into account the temporal regularity of respiratory motion. In [8], a bilinear factorisation of the model coefficients of each sample is proposed to compute a statistical model for reconstruction of the heart at discrete points in the cardiac cycle, inspired by the work on separating style and content [9]. Recently, a bilinear spatiotemporal model was proposed for facial animation [10]. This model factorises the basis of a dataset into separate shape and trajectory bases, as compared to the bilinear factorisation of the model coefficients in [8, 9]. In this paper, we propose a general approach to model respiratory organ motion based on a bilinear model. We augment the bilinear model with a Bayesian algorithm for reconstruction from sparse and noisy data, leading to a variety of interesting possible applications. 2 Bilinear Motion Model Let the vector x = (x 1,y 1,...,x n,y n ) T R p be an instance of a 2d shape (w.l.o.g for higher dimensions) and the matrix X =[x 1, x 2,...,x t ], (1) a sequence of t shapes. In a linear model of shape, each instance is represented by a linear combination of base shapes x j = i b i c i,j (2) and the complete sequence can thus be represented as X = B s C s, (3) where B s R p ks consists of k s basis vectors b i R p that span the model space and C s R ks t is a matrix of t coefficient vectors that define linear combinations of shape basis vectors. Conversely, a linear model of motion can be built where the rows (not the columns) of X are modeled as a linear combination of motion trajectories, x i = c i,j b j, (4) j and the data matrix is represented as X = C m B T m, (5) where B m =[b 1, b 2,...,b km ] R t km and C m R p km are k m motion basis and p coefficient vectors, respectively [10]. In practice, the dimensionality of a linear model is often reduced using some type of dimensionality reduction approach,

3 A Bilinear Model for Temporally Coherent Respiratory Motion 223 e.g. principal component analysis (PCA). However, in a linear representation, the temporal regularity of trajectories is ignored. As an illustrative example, consider an ordered sequence of shape vectors [x 1, x 2,...,x t ]. Any permutation [x p(1), x p(2),...,x p(t) ] of the columns in X only results in a permutation of the coefficients in C s but does not affect the shape basis B s. The same also applies forshufflingrowsofx, affecting only rows in C m. In this sense, a linear model is an overparameterisation because it fails to exploit the underlying spatiotemporal structure. For modelling respiratory motion, we would like to respect temporal regularities in the data. A similar problem has been studied in [10], where it is shown that a factorisation X = B s CB T m (6) exists that links together shape and motion bases in a single model. In this bilinear model, the matrix C R ks km defines weights for the outer products of the i-th shape vector b i s and the j-th trajectory vector b j m. Intuitively, it describes how the points of the shape modes b i s vary over time. Figure 1 shows how the coefficients c ij in C are related to the outer product of shape and trajectory basis vectors b i s and b j m. p k s b i s ks k m t k m c ij b j m B s : shape basis C : coefficient matrix B T m : trajectory basis Fig. 1. Illustration of bilinear multiplication. The coefficients c ij define a weighting of each outer product between shape basis vectors b i s and trajectory basis vectors b j m. In general, both bases and the coefficient matrix must be estimated from data using tensor decomposition methods. However, in [10] the problem is considerably simplified by using a conditioned motion basis. It has been shown that the discrete cosine transform (DCT) basis converges to the optimal PCA basis if the data is generated from a stationary first-order Markov process [11]. Moreover, for respiratory motion, we are interested in low-frequency cyclic patterns, which makes the DCT basis a suitable choice for compression. The use of an analytical basis like DCT is also useful for adapting the bilinear model to varying cycle lengths. Since the shape and motion bases factor out separately and as a result, only B m depends on t, the motion basis can be adjusted to appropriate size by changing the length of the basis vectors b j m while leaving the shape basis unchanged. This is not possible with a linear model because shape and motion information are intermingled.

4 224 F. Preiswerk and P.C. Cattin Given a conditioned motion basis B m and a series of respiratory cycle data X 1,...,X N, we can solve for a shape basis that minimises the squared reconstruction error. It is given by the row-space computed through singular value decomposition (SVD) of the matrix Π =[ˆX T 1, ˆX T 2,..., ˆX T N ] T, (7) where ˆX i = X i B m B + m denotes the reconstruction of X from its trajectory projection and + denotes the Moore-Penrose pseudoinverse. This way, the shape basis B s can be computed directly from the data in closed form. It defines a space where variations along the individual dimensions are uncorrelated. In order to simplify the calculations in the next section, we assume that coefficients are scaled to unit variance in each of the two subspaces, i.e. C = Σs 1 B T s XB m Σm 1. 3 Reconstruction from Sparse Data Applications for the bilinear motion model are manifold. A natural application is to use it as a prior for motion segmentation and registration. Here, we focus on another scenario, which is the reconstruction from only a small set of observations, e.g. when measurements from implanted electromagnetic beacons or 2d projections are given and the goal is to reconstruct the entire shape over time. In order to estimate entire breathing cycles from partial observations, we extend the scheme presented in [12] to the bilinear case. Our observation model is as follows, R = L s XL T m, (8) where L s : R p R lp,l p <pacts on the columns (i.e. features) and L m : R t R lt,l t <tacts on the rows (i.e. time frames) of X. Although these matrices can be any linear transformation, in our context L s typically deletes rows (features) over the entire sequence and L m deletes all features in a particular frame. In general, both mappings are not injective, thus there is no unique solution of Eq. 8. The least-squares solution is found by optimising the error function E(C) = Q s CQ + m R F, (9) where Q s = L s Σ s B s, Q m = L m Σ m B T m and F denotes the Frobenius norm. This cost function is minimal for Ĉ = Q + s RQ m. (10) However, the result in Eq. 10 only applies for noiseless measurements. Assuming that measurements are subject to uncorrelated Gaussian noise of variance σn 2, the likelihood of observing R is given by p(r C m )=ν exp( 1 2σN 2 Q s C m R F ), (11)

5 A Bilinear Model for Temporally Coherent Respiratory Motion 225 where we have defined C m = CQ + m and with normalisation constant ν = ((2π) l 2 σn ) 1. Using Bayes rule to compute the posterior probability and maximising it leads to the following minimisation problem, E(C m )= Q s C m R F + η C m F. (12) The regularisation factor η = σn 2 allows to find a trade-off between matching quality and prior probability. It can be shown [12] that the global optimum is obtained via SVD of Q s = U s W s Vs T according to w s,i Ĉ η = V s diag( ws,i 2 + η )UT s R, (13) where w s,i are the diagonal elements of W s. Note that all matrices in Eq. 13 except R are constant and can thus be precomputed. 3.1 Model Size and Compression Any bilinear model can be rewritten in linear form by computing the Kronecker product, B = B m B s R p t, (14) i.e. taking all possible products between the elements of the shape and trajectory bases. Note that this result does not imply that linear and bilinear models are equivalent because the opposite operation, i.e. obtaining a bilinear factorisation from a linear basis, is not always possible. Retaining only the k s basis vectors in B s with highest corresponding singular values leads to optimal dimensionality reduction in a least-squares sense. Since respiratory motion can be encoded primarily by lower-frequency components, the model can be further compressed by only retaining the k m lowest-frequency basis vectors in B m. For example, setting k s = k t = 5 in a model of p = 100 coordinates and 10 time steps leads to a shape basis of size and a motion basis of size 10 5, totalling in 550 values. The corresponding linear model according to Eq. 14 is of size or more than 45 times the number of entries. Formally, the size of the bilinear model grows linearly in the number of model points O(p k s + t k m ) while the size of the corresponding linear model grows according to O(p t k s k m ), which reflects the fact that the bilinear model exploits the structure of the underlying data more efficiently. 4 Experiments We applied the bilinear model to a sequence of sagittal 2d MRI images of the liver of a male subject under free breathing. A total of 120 respiratory cycles was acquired and split into a training set of 40 cycles and a test set of n =80cycles. The images were pre-processed as follows. First, they were non-rigidly registered using NiftyReg [13], an open-source medical image registration suite, by deforming a reference exhalation image to all other images. Due to the discontinuities

6 226 F. Preiswerk and P.C. Cattin MAE [mm] Bilinear model of cycles Linear model of cycles Linear model of states MAE [mm] Bilinear model of cycles Linear model of cycles Linear model of states η (a) Uncompressed models η (b) Compressed models Fig. 2. Mean absolute error (MAE) of reconstruction from full testset over different values of the regularisation factor η. In (a), the complete bases were used. In (b), k s = 5 shape and k m = 5 motion basis vectors were used in the bilinear case and 25 basis vectors were used for the linear models. between the sliding organ and the abdominal wall, the liver was manually masked in the reference image. Additionally, the data was normalised wrt. the number of frames in each cycle, resulting in exactly t = 10 states per cycle. The displacement fields were then equidistandly sampled at n = 85 locations per frame and a bilinear model was built from these vertices as described above. 4.1 Model Compression To compare the proposed bilinear model of respiratory cycles, we computed a linear PCA model of cycles (where each of the n test cycles is concatenated into a column vector of size p t) as well as a linear PCA model of states (where the samples consist of t n column vectors of length p). For the PCA models we use the linear version of the same reconstruction algorithm described above. We reconstructed the full training set without additional noise to see how well the models can describe unseen data. As can be seen in Fig. 2(a), the uncompressed bilinear model has high generalisation capabilities, especially if no regularisation is applied (see η = 0). When the models are compressed, the bilinear model still outperforms the linear model of cycles for all values of η (Fig. 2(a)). It is not surprising that the linear model of states performs well in both experiments because it fits every observed state individually rather than fitting entire cycles. 4.2 Missing Values In order to elaborate on the performance of the models in a setting where data is missing and additionally subject to measurement noise, we performed reconstructions on various percentages of randomly removed data. Figure 3(a) shows the error plot for randomly removing an increasing number of model vertices

7 A Bilinear Model for Temporally Coherent Respiratory Motion Bilinear model of cycles Linear model of cycles Linear model of states 2 Bilinear model of cycles Linear model of cycles MAE [mm] 2 MAE [mm] Removed model points [%] (a) Removed frames [%] (b) Fig. 3. Mean absolute error (MAE) of reconstruction with Gaussian random noise of σ N =1.0 and fixed regularisation factor η =1.0. (a) Model vertices randomly removed over the entire test sequence. (b) Complete time frames randomly removed (linear model of states is not able to reconstruct missing frames). over the entire test sequence. The bilinear model performs better or equal to its linear counterpart, while the linear model of states clearly performs worse in most cases. Lastly, we randomly removed an increasing number of entire frames from the sequence. Figure 3(b) shows the results. Here, the linear model shows a superior error curve that increases exponentially, while the error of the bilinear model increases linearly. This experiment does not apply for the linear model of states because it cannot produce meaningful reconstructions in case of missing frames. 5 Conclusion We presented a novel model for modelling respiratory motion that respects the temporal regularity of the underlying data. Shape and motion information is separated into individual bases and a coefficient matrix defines weights of the basis vectors outer products. This separation leads to significantly more compact models compared to conventional linear models. Furthermore, the model is more flexible because the motion basis can be independently exchanged while leaving the remaining components of the model unchanged in order to generate cycle instances of arbitrary length, although we have restricted our experiments to cycles of the same length for simplicity. We described a Bayesian algorithm for reconstruction from noisy and sparse data in the bilinear case and we could show that the model performs better in most experiments compared to its less compact linear counterparts. All components of the reconstruction algorithm can be precomputed, therefore the algorithm is computationally efficient. The model can potentially be used in registration algorithms as well as for motion compensation in tumour therapy.

8 228 F. Preiswerk and P.C. Cattin References 1. McClelland, J., Hawkes, D.J., Schaeffter, T., King, A.P.: Respiratory motion models: a review. Med. Image Anal. 17, (2013) 2. King, A.P., Blackall, J.M., Penney, G.P., Hawkes, D.J.: Tracking liver motion using 3-D ultrasound and a surface based statistical shape model. In: Proceedings of IEEE Workshop on Mathematical Methods in Biomedical Image Analysis, pp (2001) 3. Zhang, T., Lu, W., Olivera, G.H., Keller, H., et al.: Breathing-synchronized delivery: a potential four-dimensional tomotherapy treatment technique. Int. J. Radiat. Oncol. Biol. Phys. 68, (2007) 4. He, T., Xue, Z., Xie, W., Wong, S.T.C.: Online 4-D CT estimation for patientspecific respiratory motion based on real-time breathing signals. In: Jiang, T., Navab, N., Pluim, J.P.W., Viergever, M.A. (eds.) MICCAI 2010, Part III. LNCS, vol. 6363, pp Springer, Heidelberg (2010) 5. Li, R., Lewis, J., Jia, X., Zhao, T., et al.: PCA-based lung motion model. Technical report (2010) 6. Schneider, M.: Model-based respiratory motion compensation for image-guided cardiac interventions. In: Informatiktage. LNI, vol. S-9, pp GI (2010) 7. Preiswerk, F., Arnold, P., Fasel, B., Cattin, P.C.: A bayesian framework for estimating respiratory liver motion from sparse measurements. In: Yoshida, H., Sakas, G., Linguraru, M.G. (eds.) Abdominal Imaging. LNCS, vol. 7029, pp Springer, Heidelberg (2012) 8. Hoogendoorn, C., Sukno, F.M., Ordás, S., Frangi, A.F.: Bilinear models for spatiotemporal point distribution analysis. Int. J. Comput. Vis. 85, (2009) 9. Tenenbaum, J.B., Freeman, W.T.: Separating style and content with bilinear models. Neural Comput. 12, (2000) 10. Akhter, I., Simon, T., Khan, S., Matthews, I., Sheikh, Y.: Bilinear spatiotemporal basis models. ACM Trans. Graph. 31, 17:1 17:12 (2012) 11. Rao, K.R., Yip, P., Rao, K.R.: Discrete Cosine Transform: Algorithms, Advantages, Applications, vol Academic Press, Boston (1990) 12. Blanz, V., Vetter, T.: Reconstructing the complete 3D shape of faces from partial information. Informationstechnik und Technische Informatik 44, (2002) 13. Modat, M., Ridgway, G.R., Taylor, Z.A., Lehmann, M., et al.: Fast free-form deformation using graphics processing units. Comput. Meth. Prog. Biomed. 98, (2010)