Segmentation of Volume Images Using Trilinear Interpolation of 2D Slices

Transcription

1 Segmentation of Volume Images Using Trilinear Interpolation of 2D Slices Pouyan TaghipourBibalan,Mehdi Mohammad Amini Australian National University Abstract Accurate segmentation of biomedical images can contribute to improved diagnosis, surgical planning and prognosis. A review of biomedical image segmentation methods using deformable models is presented. A 2D deformable model is applied to axial slices from a 3D MRI volume image of the human brain to determine the extent of the Occipital lobes. Trilinear interpolation is used to obtain slices inbetween or intersecting the available 2D slices. Using this technique the computational load is reduced compared to a direct 3D segmentation technique but it is shown that the applicability of this technique is bound to setting constraints between the plains. The difficulties of biomedical image segmentation are discussed and suggestions for addressing some of the issues are offered. 1 Introduction Recent advances in medical imaging allows the anatomic structure of internal organs to be viewed and gross pathology of organs and diseases to be recognized without the need to open the body. In addition to structural information, dynamic and functional information on biochemical and patho-physiologic processes may be imaged, all with unprecedented precision. For example fmri intensity measures the amount of oxyhemoglobin in the blood that has been converted to deoxyhemoglobin as a result of oxygen consumption by nearby cells. In 3D medical imaging applications conversion of the raw MRI data into an image useful to a radiologist involves the following processing: back projection to form 2D slices, noise reduction and image registration to align multiple 2D slices. The MRI data used in this project has already had this processing applied. We investigate further processing to establish the extent of objects of interest within these volume images. This is known as the image segmentation problem. 1.1 Image Segmentation The aim of image segmentation is to partition the image to meaningful regions, usually by establishing the boundaries of objects. It is an early step in many image processing applications. Difficulties arise in the face of image noise and inhomogeneities in the sensors. The nature of the object being segmented may introduce further difficulties, such as when the boundary is indistinct or not captured by the same image features over its entirety. Fully automated approaches are not generally applicable in the face of these problems and interaction with a human operator is usually required. Some systems allow the operator to add additional forces (e.g. a spring force) during deformation, to guide the model towards the desired boundary. The large body of medical image processing litterature offers a wide range of techniques applicable in different applications, with no single 1

2 technique being generally applicable. In this work we concentrate on segmentation using deformable models. 1.2 Deformable Models The goal of methods based on deformable models is to move an initial curve (in the 2D case) or surface (in the 3D case) towards the desired boundary, thus providing a model for the boundary. The boundary may be provided by edge detection methods based on image intensity or any other set of features that detect the desired boundary (e.g. texture, colour or other multi-spectral information such as provided by multiple types of MRI experiment on the same subject). A series of points lying on the curve/surface may be used to represent the current model during deformation. These are called parametric models. Parametric models vary in what forces are applied to the points to make the curve/surface move towards the boundary. Alternatively the current model may be implicitly represented by a level set, e.g. all the points which lie on a contour where some function has a constant value. These are called geometric models. In each iteration the level set is moved closer to the boundary under the influence of a speed function, without the need for the contour to be explicitly evaluated until sufficient convergence has been achieved. Geometric models vary in the nature of the speed function used. Parametric and geometric deformable models share common fundamentals and [17] discusses their inter-relationship. Simple segmentation techniques such as edge detection and thresholding are not generally sufficient for segmenting soft tissues, due to the difficulties mentioned in the introduction. Deformable models, also known as snakes, active contours or surfaces and balloons, can successfully deal with noise and indistinct edges. Figure 1: Active Contour from [10] 2 Parametric Models An active contour is defined by an ordered collection of points in the image plane (see figure 1). V = {v 1,... v n }; v i = (x i, y i ); i = 1... n (1) The points in the contour iteratively approach the boundary of an object. In this section the approaching process is demonstrated through the solution of an energy minimization problem. For each point in the neighborhood of V i, an energy term E i is computed, E i = αe int (v i ) + βe ext (v i ) (2) where E int (v i ) is an internal energy function dependent on the shape of the contour and E ext (v i ) is an external energy function dependent on the image properties, such as the gradient near the point v i. α and β are constants providing relative weighting of the energy terms. The value at the center of each matrix corresponds to the contour energy at point v i. If the energy functions are chosen correctly, the contour V approaches and stops at the object boundary. In this section an energy minimizing formulation for deriving parametric deformable models is presented. The solution to this equation 2

3 satisfies a minimum principle. Next finite differences is utilized to derive a numerical implementation. 2.1 Mathematical Representation The objective here is to iteratively find the curve X(s) = [x(s), y(s)], s [0, 1] that minimizes the energy functional E. It has been shown that the curve that minimizes E must satisfy the following Euler-Lagrange equation [9] [5] s (α X s ) 2 s 2 (β 2 X ) P (X) = 0 (3) s2 This equation has the same form as a force balance equation F int (X) + F pot (X) = 0 and can be interpreted as F int being responsible for stretching and bending and F pot pulling the contour toward the desired object boundary. Introduction of a new variable t helps in solving this equation, so that we can deal with X(s) as a function of time. The equation is thus presented in the following form. γ X t = s (α X s ) 2 s 2 (β 2 X ) P (X) s2 (4) The coefficient γ is introduced to make the units on the left side consistent with the right side. By utilizing dynamic force formulation, it can be shown that equation 4 can be changed to equation?? regardless of the type of the external energy; it can be a combination of forces either being potential forces or non potential forces. γ X t = F int(x) + F ext (X) (5) In the next section some forms of external forces are presented. 2.2 External Forces Gaussian Potential Force The Gaussian potential force is of the form P (x, y) = ω e [G σ (x, y) I(x, y)] 2 (6) where ω e is a positive weighting parameter, G σ (x, y) is a two dimensional Gaussian function with standard deviation σ and is the 2D image convolution operator. A larger σ implies a larger attraction range at the cost of losing the ability to track small features Pressure Force - Balloons The Gaussian potential force has only a short range. Adding a pressure force allows the curve/surface to move over areas of low gradient and pass through weak edges [5]. The pressure force F p is given by F p (X) = ω p N(X) (7) where N(X) is the inward unit normal of the model at the point X and ω p is a weighting parameter. The value of ω p determines the strength of the pressure force. It is selected in such a way that the pressure force is slightly smaller than the Gaussian potential force at significant edges but large enough to pass through weak edges. The pressure force keeps inflating or deflating the model until stopped by the Gaussian potential force. A disadvantage is that pressure forces may cause the curve/surface to cross itself and form loops [15] Distance Potential Force The distance potential force is designed to move a point towards the nearest edge, extending the attraction range. More details on this type of force can be found in [4] Gradient Vector Flow The Gradient Vector Flow (GVF) force is a diffusion of the gradient vectors [19] [20]. It has a wide attraction range and it deforms well in the presence of boundary concavities, unlike distance potential forces. In this method, first an edge map f(x, y) is derived from the image I(x, y) such that it is larger near the image edges. f(x, y) = E ext (x, y) (8) 3

4 wheree ext (x, y) can be P (x, y), the gaussian potential forece, or other external forces mentioned in literature. The gradient vector flow is defined to be the vector field v(x, y) = (u(x, y), v(x, y)), such that it minimizes the energy functional ε = µ(u 2 x+u 2 y+vx+v 2 y)+ f 2 2 v f 2 dxdy (9) It can be observed that when the field f is small the energy is dominated by the partial derivatives of the vector field, yielding a smooth field and when the field is large the second term dominates the integrand and is minimized by setting v = f. µ is a regularization parameter governing the tradeoff between the first term and the second term. µ is set according to the noise in the image. The more the noise in the image, the higher the value of µ. The GVF is found by solving the Euler equations µ 2 u (u f x )(f 2 x + f 2 y ) = 0 (10) µ 2 v (v f y )(f 2 x + f 2 y ) = 0 (11) where 2 is the Laplacian operator. The above equations are solved by treating u and v as functions of time and solving u t (x, y, t) = µ 2 u(x, y, t) (u(x, y, t) f x (x, y)).(f x (x, y) 2 + f y (x, y) 2 ) (12) approximating the derivatives in 5 with finite differences and converting to the vector notation [21] X n X n 1 = AX n + F ext (X n 1 ) (14) τ where τ = t γ and Xn, X n 1 and F ext (X n 1 ) are m 2 matrices, and A is an m m pentadiagonal banded matrix with m being the number of sample points. This is the formulation from which the MAT- LAB simulations are derived. By solving for X n the complete set of points which present the deformed curve X(s) in each iteration can be found. By choosing different types of F ext it is possible to overcome the issues concerned with the attraction range and boundary concavities. 2.4 Limitations of Parametric Models Although parametric models are successful in many applications, they have some serious limitations: the model must be explicitly evaluated at each iteration; as the model deforms points must be regularly redistributed over the curve/surface and more points are required as the model complexity increases. For complex 3D surfaces the computation can become prohibitive; v t (x, y, t) = µ 2 v(x, y, t) (v(x, y, t) f y (x, y)).(f x (x, y) 2 + f y (x, y) 2 ) (13) These equations are also known as generalized diffusion equations. The steady state solution as t of the above equations is the solution to the Euler equation. 2.3 Numerical Implementation Several numerical techniques have been reported in the literature, such as finite differencs [1], dynamic programming [1], greedy algorithm [18] and finite elements [18] [3] [16]. By as the model approaches the boundary the topology may change, e.g. a closed curve may pinch and split into two separate closed curves or merge back together. A parametric model can only handle such situations with computationally expensive reparameterization. 3 Geometric Models (Free Form) Geometric deformable models [11] address the limitations of parametric models described 4

5 4 Simulation Results 4.1 Tri-linear Interpolation within Volume Images Figure 2: Cuboid Plane Intersection from [8] above, in particular they naturally handle changes in topology and do not require redistribution of points along the contour. In this method the evolving model is represented implicitly as a level set, that is the contour for which some function has a constant value. The model is only explicitly evaluated after convergence to the boundary. We are not goint to go into details of geometric models since our simulations were based on parametric ones. An excellent introduction is available in [13]. InterpDemo computes the image on a plane of any orientation through a volume image. The world frame is taken to be that of the volume image. The plane is defined by z = 0 in the O frame, which is defined by a homogeneous transformation T of the world frame. The method described in [8] is used to compute the (3 to 6) vertices of the polygon of intersection of the plane with the volume image. The (x, y, z) coordinates of the vertices are computed in the world frame then transformed into (x, y, z ) coordinates in the O frame. The range of the x, y values of the vertices is computed in the O frame. A mesh of coordinates is constructed in the O frame over this range, with z = 0 fixed. This mesh is transformed into the world frame and the Matlab function interp3 is used to evaluate the intensity at each of these points by tri-linear interpolation. Figure 3 shows the InterpDemo controls used to generate the image shown in figure 4. Red circles in the image indicate the vertices of the polygon of intersection of the plane with the volume image. The parameters are: (θ x, θ x, θ x ) = (0, , 0) (x 0, y 0, z 0 ) = ( , , ) 3.1 Limitations of Geometric Models Because the topology is fixed in parametric models, they may be more appropriate in applications where the topology is known [21] [12] [2]. In cases where the volume image is too large to fit in main memory [7], out-of-core techniques may be applied [6] [14]. Figure 3: InterpDemo Settings 5

6 4.2.3 External Force Figure 4: InterpDemo Interpolated Image 4.2 Segmentation of the Occipital lobes Overview Three axial slices through the Occipital lobes are segmented using the 2D parametric snake segmentation implementation provided in [22]. The initial contour for each slice is manually entered by the user. Interpolated slices midway between these slices are computed and also segmented. The resulting five curves delimiting part of the surface of the Occipital lobes are displayed in a 3D Matlab plot, allowing the user to explore the surface Image Preprocessing A parametric snake converges to a boundary under the action of an external force. In this case we applied a Gaussian blur with a standard deviation of 3 pixels to remove high frequencies and noise and used Canny edge detection with high and low thresholds set to 0.12 and 0.05 to define the boundaries. We experimented with a range of blur sizes and edge detection methods and threshold values before settling on these parameters. Clearly these steps are crucial to the result as they define the goal for subsequent processing. The resulting edge map was again blurred with a Gaussian (size 3 as before) and the Gradient Vector Flow (GVF) [20] external force field was computed from the result. Figure 5 shows an image slice, the corresponding edge map with the initial user defined contour and a small portion of the GVF field enlarged. We have also obtained similar results using the traditional gradient external force field. Computation of the field was much faster than with GVF, but it provided slower convergence to the boundary. From exploration of the volume image, it appears that the interface surface between lobes in the brain are often not clearly delimited by discontinuities in image intensity. Perhaps features other than intensity, such as texture, might detect these surfaces better. This is an area we have not further explored. If the snake exhibits a tendency to get caught on short discontinuous edges, the addition of an internal pressure force can overcome the external force towards these short edges, causing expansion to continue towards a more significant edge. We used a small pressure force, but obtained a similar results without it Segmentation Results Figure 6 shows the result of 2D snake segmentation of the five axial slices through the Occipital lobes. The segmentation is performed on the slices obtained from the interpolation of the available slices. Stacking up these segmented plains as in Figure 6, we obtain a 3D segmented representation. It was found that unless the different slices are perfectly alligned this technique does not give satisfactory results. 5 Conclusions Image preprocessing is a major part of medical image segmentation, with a large body of literature devoted to it and a range of methods required depending on the application. Image intensity may not be the best choice of feature 6

7 for detecting the interface surface between different lobes of the brain. Applying segmentation to each plane independently, without constraints between the planes, results in curves that are not smooth across the planes. Unsurprisingly, 3D techniques are required to generate good 3D surfaces. References Figure 5: Image, Edge map and GVF [1] A. A. Amini, T. E. Weymouth, and R. C. Jain. Using dynamic programming for solving variational problems in vision. IEEE Transactions. Pattern Analysis and Machine Inteligence, 12(9): , [2] D. L. Pham C. Xu and J. L. Prince. Finding the brain cortex using fuzzy segmentation, isosurfaces, and deformable surface models. Proc. Information Processing in Medical Imaging (IPMI97), page , [3] V. Caselles, F. Catt, T. Coll, and F. Dibos. A geometric model for active contours. Numerische Mathematik. [4] L. D. Cohen and I. Cohen. Finiteelement methods for active contour models and balloons for 2-d and 3-d images. IEEE Trans. Pattern. Analysis Machine Inteligence, 15(11): , [5] Cohen L D. On active contour models and balloons. CVGIP:Image Understanding, 53(2): , [6] R. Farias and C Silva. Out-of-core rendering of large unstructured grids. IEEE Comput. Graph. Appl, 21(4):4250, Figure 6: Occipital lobe segmentation [7] Leandro S. Marturelli Gilson A. Giraldi, Paulo S. Rodrigues and Rodrigo L. S. Silva. Improving the initialization, convergence, and memory utilization for deformable models. In Handbook of Biomedical Image Analysis, Segmentation Models, pages , New York, NY, USA, Kluwer Academic / Plenum. 7

8 [8] Gernot Hoffmann. Cube plane intersection, hoffmann/cubeplane pdf. [9] Terzopoulos D. Kass M, Witkin A. Snakes: Active contour models. International Journal of Computer Vision, 1(4): , [10] Blair Mackiewich. Intracranial boundary detection and radio frequency correction in magnetic resonance images. [20] C. Xu and J. L. Prince. Snakes, shapes, and gradient vector flow. IEEE Trans. Imag. Proc., 7(3):359369, stella/papers/blairthesis/main/node29.html. [21] Chenyang Xu, Dzung L. Pham, and [11] R. Malladi, J. Sethian, and B. Vemuri. Jerry L. Prince. Image segmentation using Shape modelling with front propagation. deformable models. pages The IEEE Trans. Pattern Anal. Machine Intell, 17: , John Hopkins University, [12] G. Sapiro P. C. Teo and B. A. Wandell. Creating connected representations of cortical gray matter for functional mri visualization. IEEE Trans. Med. Imag, 16:852863, [13] J. A. Sethian. Level set methods: An act of violence. American Scientist, sethian/2006/publications/popular/popular.html. [14] Jimenez W. Giraldi G. A. Silva R. Strauss, E. and A. F. Oliveira. A semi-automatic surface reconstruction framework based on t-surfaces and isosurface extraction methods. International Symposium on Computer Graphics, Image Processing and Vision (SIBGRAPI2002), [15] H. Tek and B. B. Kimia. Volumetric segmentation of medical images by threedimensional bubbles. Comp. Vis. Imag. Under., 65:246258, [16] D. Terzopoulos and D. Metaxas. Dynamic 3d models with local and global deformations:deformable superquadrics. IEEE Trans. Patt. Anal. Mach. Intell., 13: , [17] R. Kimmel V. Caselles and G. Sapiro. Geodesic active contours. Intl J. Comp. Vis, 22:6179, [18] D. J. Williams and M. Shah. A fast algorithm for active contours and curvature estimation. CVGIP:Image Understanding, 55(1):14 26, [19] C. Xu and J. L. Prince. Generalized gradient vector flow external forces for active contours. Signal Processing, 71(2):131139, [22] Chenyang Xu and Jerry L. Prince. Matlab implementation of gradient vector flow with a 2d parametric model, 8