CHAPTER 1 AN INTRODUCTION TO COMPUTER VISION IN MEDICAL IMAGING

Transcription

1 CHAPTER 1 AN INTRODUCTION TO COMPUTER VISION IN MEDICAL IMAGING Chi Hau Chen University of Massachusetts Dartmouth cchen@umassd.edu There has been much progress in computer vision and pattern recognition in the last two decades, and there has also much progress in recent years in medical imaging technology. Although images in digital form can easily be processed by basic image processing techniques, effective use of computer vision can provide much useful information for diagnosis and treatment. It has been a challenge to use computer vision in medical imaging because of complexity in dealing with medical images. In this chapter a brief introduction of the subject is presented by addressing the issues involved and then focusing on the active contour model for medical imaging. 1. Introduction There has been enormous progress in medical imaging techniques and modalities in the last decade or so. For example ultrasound has found its use in many areas previously using x-ray or other techniques. Accompanied with the progress is the greatly increased use of computer vision techniques in medical imaging. In computer vision (see e.g. [1][2]) we talk about the low-level processing which involves basic image processing operations like noise filtering, contrast enhancement and image sharpening, the mid-level processing which involves image segmentation and pattern recognition as well as 3D reconstruction and the high-level processing which involves making sense of an ensemble of recognized objects and performing the cognitive functions at the far end of the processing sequence. Medical imaging refers to the techniques and processes used to create images of the human body for clinical purposes, or procedures seeking to reveal, diagnose or examine disease or studying normal anatomy and physiology [3]. Medical imaging evolved from the discovery of x-rays to the newest magnetic resonance image (MRI). The most commonly used techniques these days are x-ray, computer tomography (CT), ultrasound, MRI and positron emission tomography (PET). The emphasis of medical imaging is to help doctors 1

2 2 C. H. Chen or other trained personal to provide better diagnosis and treatment and thus the low level and mid-level computer vision is particularly important in the medical area. It is evident that medical imaging has significant impact on medicine and computer vision making use of enormous computing power has enormous impact on medical imaging. The following sections will briefly discuss the nature of some of these medical images and the technology behind them. The remaining chapters of the book cover important aspects of computer vision in medical imaging written by leading experts in the field. 2. Some Medical Imaging Methods 2.1. X-ray X-ray is the first and oldest medical technique available to doctors for the visualization of the body without surgery. X-rays were first discovered by Wilhelm Rontgen in They penetrate most biological tissues with little attenuation and thus provide a comparatively simple means to produce shadow or projection, images of human body. However X-rays have ionizing effects on the body and hence should not be repeatedly used. The X-ray imaging system involves having a film or screen containing a radiation-sensitive material exposed to the x-rays transmitted through the region of the body. The developed film or excited phosphorous screen exhibits a geometric pattern produced by the structures in the beam path [4]. However X-ray imaging is limited as the signal can be reduced due to the scattering of a large percentage of radiation from the body and much detail is lost in the radiographic process with the superposition of 3D structural information onto a 2D surface. Fig. 1 is the X-ray image of the bones. X-ray system has now been greatly improved. Its use for digital mammogram is particularly important (see e.g. Fig. 2, based on our 1996 data base [14]). Fig. 1. X-ray image of a skull with a fracture in the right parietal bone.

3 An Introduction to Computer Vision in Medical Imaging 3 Fig. 2. Mammograms of a patient. Fig. 3. MRI image of the human head Magnetic Resonance Image (MRI) Magnetic Resonance Image was developed in the early 1970s and has become versatile and clinically useful diagnostic imaging modality [4,5]. MRI is a noninvasive imaging technology that does not use ionizing radiation and provides much more contrast between different soft tissues of the body than computed tomography (CT). It is based on perturbing magnetic fields with radio waves. In MRI, hydrogen nuclei (protons) are imaged due to their strong magnetic moment and prevalence in the soft tissue of the body. The magnetic field is produced by passing an electric current through wire coils in most MRI units. Other coils, located in the machine and in some cases, placed around the part of the body being imaged, send and receive radio waves, producing signals that are detected by the coils. A computer then processes the signals and generates a series of images each of which shows a thin slice of the body. The images can then be studied from different angles by the interpreting physician. Fig. 3 shows the MRI image of the human head. MRI studies brain or body anatomy. More recently, functional MRI has been particularly useful to study brain physiologic function Intravascular Ultrasound (IVUS) Intravascular Ultrasound (or IVUS) allows us to see the coronary artery from the inside-out. This unique picture, Fig. 4, generated in real time, yields information that is not possible with routine imaging methods or even non-invasive Multislice CT scans. A growing number of cardiologists think that new information yielded by IVUS can make a significant difference in how patient is treated and can provide more accurate information which will reduce complications and incidence of heart diseases. Intravascular ultrasound (IVUS) is a catheter-based

4 4 C. H. Chen Fig. 4. Typical IVUS image. technique which provides high-resolution images allowing precise tomographic assessment of lumen area. IVUS uses high-frequency sound waves called ultrasound that can provide a moving picture of your heart. These pictures come from inside the heart rather than through the chest wall. The sound waves are sent with a device called a transducer. The transducer is attached to the end of a catheter, which is threaded through an artery and into your heart. The sound waves bounce off of the walls of the artery and return to the transducer as echoes. The echoes are converted into images on a television monitor to produce a picture of your coronary arteries and other vessels in your body. In the above IVUS image, the lumen is typically a dark echo-free area adjacent to the imaging catheter and the coronary artery vessel wall mainly appears as three layers: intima, media, and adventitia. As the two inner layers are of principal concern in clinical research, segmentation of IVUS images is a must to isolate the intima-media and lumen boundaries which provides important information about the degree of vessel obstruction as well as the shape and size of plaques. IVUS is only one of many uses of ultrasound in medicine. Actually ultrasonic method is highly versatile, inexpensive and effective in many medical diagnosis uses. Details of the above methods and other medical imaging modalities are well presented in Ref Roles of Computer Vision, Image Processing and Pattern Recognition There has been a long history of computers in medicine. The more advanced the medical instrumentation, the more it relies on the computer capability. For medical imaging, computer vision, image processing and pattern recognition techniques are particularly important to provide the required information in diagnosis and treatment. The progress in these techniques is reflected in the sophisticated software tools some of which are commercially available, and

5 An Introduction to Computer Vision in Medical Imaging 5 others may still be in research and development stage. The software making use of computing power is very useful to deal with the enormous amount of data in medical imaging. For medical images, the first thing we need to consider is to use image processing techniques (see e.g. [2][6]). They can include image enhancement in spatial and frequency domains, restoration of object from distorted or convoluted images, color image processing, wavelet and mutiresolution image processing, image compression, morphological image processing and image segmentation. Both image enhancement and segmentation can be considered as low level computer vision. For mid-level computer vision, computers can organize the knowledge (information) acquired in the low-level vision to make some useful decisions. For high-level vision, computers must further provide some thinking capability like humans do. The above stated division of computer vision tasks may not be precise, but for medical imaging, the low level vision is most important as the results are used for human expert (physicians) to interpret. Thus interactive and 3-D visualization capabilities can be significant. Pattern recognition is closely linked to the image processing and computer vision especially for image segmentation. Some basic tasks in pattern recognition (see e.g. [7][8]) are feature extraction, pattern description (be it statistical, syntactic, structural or else), learning, and decision making process. A major objective in pattern recognition is to make correct decisions or classification with the help of the other three tasks. A good classification is much needed for image segmentation. Though artificial neural networks and support vector machines have made classifier easier, they still cannot be fully relied on in practical recognition problems. There have been tremendous progress in image processing, pattern recognition and computer vision in the last decades (see e.g. [9-13]) with many applications including the medical area. For images in medical imaging, it is necessary to use the multiple information sources, like intensity, texture, shape, and contextual information within an image and between images. In fact information that captures the dynamics of the medical image patterns can be the key to the success of extracting desired diagnosis information from the images. With greatly improved medical sensors/devices and the use of multiple sensors, it may be necessary to fuse the data from several sources to aid in the diagnosis and treatment. A note on the performance measure is needed here. The simple percentage correct recognition rate can be highly inadequate. For medical imaging, the ROC (receiver operating characteristics) curve is most popular. It is a plot of recognition rate versus the false alarm rate. Other measures include the sensitivity (true positive rate) versus specificity (true negative rate). The accuracy

6 6 C. H. Chen is probably a major challenge for computer vision/pattern recognition in medical imaging. While there was success in early developments like computer chest X-ray screening for black lungs, the high accuracy desired for medical imaging, such as automated detection and segmentation, and extraction of area or volume information for regions of interest, is still not reached to the author s knowledge. This means that automated method cannot replace the manual operations. With constant progress in computer vision and pattern recognition, however, the opportunities are really unlimited to reach our goals in medical imaging. Among the large number of signal/image processing techniques have been examined for medical imaging use, the multiresoution image processing stands out to be useful most of the time at it captures the intensity and texture information well and for single image the contextual information also (see e.g. [14]). Effort is much needed to make use of information from a sequence of images to achieve improved segmentation. While other important topics in medical imaging are covered by experts as presented in this book, the rest of this chapter will be focused on the active contours problem for intravascular ultrasound images. 4. Active Contours The technique of active contours has become quite popular for variety of applications, particularly image segmentation and motion tracking, during the last decade. The active contour model has an advantage of being less sensitive to blurred edges and also avoiding broken contour lines compared to other methods like thresholding and edge-based methods. Active contour models are based on deforming an initial contour C towards the boundary of the object to be detected, through minimizing a functional designed such that its minimum is obtained at the boundary of the object. Energy minimization involves two components, the smoothness of the curve and one for pulling the curve closer to the boundary. The active contour consists of a set of control points connected by straight lines as shown below in Fig. 5. The active contour is defined by the number of control points as well as sequence of each other. However fitting active contours to shapes is an interactive process. The user must suggest an initial contour as shown in Fig. 4 below, which is quite close to the intended shape. Then we have to create an attractor image so that the contour will be attracted to features in the image extracted by internal energy. There are two main approaches in active contours based on the mathematical implementation: snakes and level sets. The classical snake model was first introduced by Kass et al. [15], while the level sets approach was first proposed by Osher and Sethian [16]. Both are discussed further in the following.

7 An Introduction to Computer Vision in Medical Imaging 7 Fig. 5. Basic form of active contour Snakes A snake is an energy minimizing spline guided by external constraint forces and influenced by edges, localizing them accurately. It is also the first model of active contour proposed by Kass et al. Let us define a contour parameterized by arc s as C (s) { (x(s), y(s)) : 0 s L } : R where L denotes the length of the contour C, and denotes the entire domain of an image I(x, y). Define an energy function, E (C) = E int + E ext (1) where E int and E ext respectively denote the internal energy and external energy functions. The internal energy function determines the regularity i.e. smooth shape, of the contour. The internal energy is a quadratic functional which is given by: L 2 2 E int α C( s) β C ( s) ds (2) 0 where α controls the tension of the contour and β controls the rigidity of the contour. The external energy on the other hand, determines the criteria of contour evolution depending on the image I (x, y) which is defined as: E ext 0 L E img (C(s)) ds (3) where E img (x, y) denotes a scalar function defined on the image plane, so the local minimum of E img attracts the snakes to edges.

8 8 C. H. Chen A very common example of the edge attraction function is the function of image gradient given by: E img (x, y) = 1/ G * I (x, y) : R (4) where G denotes a Gaussian smoothing filter with the standard deviation and is a suitably chose constant. Now in order to solve the problem of snakes we have to find the contour which minimizes the total energy term E with the given set of weights, and. Also, a set of snake points residing on the image plane are defined in the initial stage and then the next positions of those snake points are determined by local minimum E. The connected form of these particular snake points is called the contour. Fig. 6 shows an example of classic snakes in its initial stage. Fig. 7 shows the final stage. Fig. 6. An example of classic snake in its initial stage. Fig. 7. An example of classic snake in its final stage.

9 An Introduction to Computer Vision in Medical Imaging 9 The snake points eventually stop on the boundary of the object. The classic snakes provide an accurate location of the edges only if the initial contour is given sufficiently near the edges as they make use of only the local information along the contour. However, we have a difficult problem estimating a proper position of initial contours without prior knowledge. Also, classic snakes cannot detect more than one boundary simultaneously as the snakes maintain same topology during the evolution stage. The snakes cannot split to multiple boundaries or merge from multiple initial contours. The next model which is the Level set theory has given a solution to this particular problem Level set methods Since the level set method was first proposed by Osher and Sethian, it has become more and more popular theoretical numerical framework within fluid mechanics, graphics, image processing and computer vision. The level set method is basically used for tracking moving fronts by considering the front as the zero level set of an embedding function, called the level set function. It is a widely used tool for image segmentation. Many features can be considered simultaneously such as edges, region statistics, shape and kind of multidimensional data depending upon how propagation speed of the front is defined. Fig. 8 shows the use of shape prior for segmentation of a partly occluded object. It shows the evolution of the active contour (solid line) with the aid of a prior knowledge of its shape (the dotted line). The main purpose of the level set methods is to consider the moving interface as a set of zero values of an embedding function. The level set method can be applied to any kind of problem where an interface is moving with a speed F defined on every point. Fig. 8. Examples of the power of the level sets as a segmentation tool.

10 10 C. H. Chen The level set equation is briefly describes as follows. Let us assume that F is known. Consider an interface that can be a curve, surface, hyper-surface in N, as a boundary between two regions, one inside the boundary and another one outside the boundary. The interface represented as a curve in 2 is embedded in a level set function and is constituted of all points where the value of the level set function is zero (t) = {x(t) (x(t), t) = 0}. In other words, a level set function is constructed around the interface that constitutes the zero value of the embedding function. All other points of the level set function have the value of the distance from the point to the closest point on the boundary. The distance is positive if this point is situated on the outside of the boundary area and negative if it is situated on the inside. The level set function will change with time according to the speed F and the interface is always constituted by the points where the level set function equals zero. The interface,, embedded by the level set function,, can be expressed as, (t) := {x(t) (x(t), t) = 0} (5) where t being the time and x(t) are the points of, < 0 for points lying inside the surface and 0 for points lying outside the surface. Computing the time derivative of the level set (x(t), t) by using the chain rule gives: /t + (x(t), t). dx/dt = 0 (6) where is the gradient operator. We introduce F as the speed in the outward normal direction such that F = (dx/dt). n where n = /. If the initial level set function is known, (x, t = 0), the level set equation becomes /t + F = 0 (7) This partial differential equation (PDE) will propagate the boundary towards the optimal solution Geodesic active contours After Kass et al. [15] introduced the concept of snakes; extensive research was done on snakes or active contour models for boundary detection. These active contours are examples of general technique of matching deformable model to image data by means of energy minimization. The energy function is basically

11 An Introduction to Computer Vision in Medical Imaging 11 composed of two components, one controls the smoothness of the curve and another attracts the curve towards the boundary. This particular energy model is not capable of handling changes in the topology of the evolving curve when direct implementations are performed. The Geodesic active contour [18] includes a new component in the curve velocity based on image information that improves the active contour model. The new velocity component accurately tracks boundaries with high variation in their gradient including small gaps. It also allows simultaneous detection of interior and exterior boundaries in several objects without special contour tracking procedures Region-based active contours Chan and Vese [9] have presented a method for segmenting images without edge detection by using a weak formulation of the Mumford-Shah functional. In this method, they separate the image into two regions, one inside the level set function < 0 and the other region on the outside of the boundary 0. Each region is represented by mean intensities over the interior and exterior of the curve. The method extracts the objects by minimizing an energy function using region properties instead of edge properties. Mathematically the energy to be minimized is given by E = Ω (I u) 2 da + Ω (I v) 2 da (8) where and represent the interior and exterior of the curve, and u and v represent the mean image intensities over and respectively. This energy is very robust to noise and curve placement as it looks at integrals of image data rather than image derivatives and although the curve moves locally, it inspects global image statistics rather than just looking along the curve. There are many advantages of region-based approaches than edgebased techniques. However techniques that attempt to model regions using global statistics are not ideal for segmenting heterogeneous objects. As a result regionbased active contour may lead to erroneous segmentation Hybrid evolution method The method [19] is to combine both the local based Geodesic active contour and region based active contour method to derive the advantages of both. A hybrid energy is defined and minimized iteratively and a new class of active contour energies should be considered which utilizes local information and also incorporates the benefits of region-based techniques. This hybrid energy is

12 12 C. H. Chen minimized iteratively. Mathematically we begin with the geodesic active contour where the energy of the curve E is given by the following function where C represents the evolving curve and I represent the image data E f( I) ds (9) C( s) where f is a any positive, decreasing function of the image data and its values from the metric over which the minimum length geodesic will be found as the curve is deformed. The hybrid energy defined below begins with the geodesic energy given in Eq. (9), 2 2 xω l x Ω l (10) C(s) E [ (I(x,s) u (s)) (I (x,s) v (s)) ] ds where and represent the region on the interior and exterior of the curve respectively and the f we chose in Eq. (9) makes use of image data over local regions thus making it similar to the region-based flows. We should select f to be the smallest so that our key assumptions are set. s in the above equation specifies every point along it as the contour integral is evaluated. Also the u l (s) and v l (s) are the arithmetic means of points in local neighborhoods around the point C(s). The characteristic function defines these neighborhoods and the position of the curve. The function evaluates as 1 in a local neighborhood defined by small radius and 0 elsewhere. The contour divides the region selected by function into interior local points and exterior local points. It is noted that there have been much further work on the active contours by increasing the speed, and efficiency and avoiding false contours, etc. The following section deals with IVUS image segmentation using active contour or its variation IVUS image segmentation The raw data of our IVUS images were provided by Brigham Women s Hospital in Boston, MA. By using algorithms developed by Prakash Manandhar, a Ph.D. student in University of Massachusetts at Dartmouth the image data base was used for the segmentation study. The hybrid active contour method was compared to the edge based active contour method using gradient vector flow and region based active contour method. The hybrid model has shown better results compared to the other models. The experiments are conducted on two IVUS images and both of these images are compared to the other active contour models. Both images were segmented correctly using the presented hybrid

13 An Introduction to Computer Vision in Medical Imaging 13 method compared to other active contour models. The contour extracted is for lumens. One original IVUS image we used is shown in Fig. 9. The lumen was detected correctly after 800 iterations, even though there is presence of guide wire (Fig. 10). Fig. 9. Fig. 10.

14 14 C. H. Chen The algorithms developed by P. Manandhar, which employ different approach based on active contours are presented in Chapter 20 of this volume. Another approach we have examined and which is a departure from active contours is to use both the texture and intensity information along with radial basis function (RBF) [20] to derive the closed contours for both lumen boundary and outer (Media-Advantita) boundary of the artery. The method uses composite operator [21] which depends both on texture and intensity. Lumen contour can be traced based on finest texture and intensity specifics. Once we have the contour of lumen we can obtain the media-adventitia border by finding the coarse-most texture located outside the lumen border. Once we have this information of the two contour initializations we can use low pass filtering/2d Radial basis functions to obtain smooth 2D contours. The method is being used for automated segmentation of a large database from the Brigham and Women Hospital. 5. Concluding Remarks The discussion on IVUS image segmentation methods and results show only a very small part of many computer vision in medical imaging activities. Clearly many problem areas are evident, just based on the IVUS image segmentation study. 1. The manual segmentation which presumably is more accurate is very time consuming. 2. Automated algorithms still have a long way to go achieve the desired high accuracy. 3. The 3-D segmentation results are still much needed to be useful for physicians, though some effort has been made in this direction [22]. Even for 2-D segmentation, the use of time-dependent images can help (see [23]) for the IVUS images. As a concluding remark, computer vision making use of powerful computer resources is extremely useful to deal with a large amount of data for on-line or off-line operations. The IVUS image segmentation example as shown above clearly shows one of important potential and opportunities of computer vision in medical imaging. There are very many other issues such as involving shape, 3D construction (see e.g. [24-26]) and volume measurement, etc. are well within the computer vision domain that present challenges. Finally it must be noted that with constantly improved sensors and acquisition of very high resolution images, the need for computer vision in medical imaging is expected to increase greatly with time.

15 Acknowledgment An Introduction to Computer Vision in Medical Imaging 15 I thank my former graduate students S. Murphy, P. Manandhar, L. Potdar, R. Chittineni, and A. Vaishampayan as well as current graduate student A. R. Gangidi for their constructive participation of the IVUS image segmentation study project. References [1] L. Shapiro and G. Stockman, Computer Vision, Prentice-Hall, [2] R.C. Gonzalez and R.C. Woods, Digital Image Processing, 3 rd edition, [3] [4] R.A. Robb, Biomedical Imaging, Visualization and Analysis, Wiley, [5] A.P. Dhawanm, Medical Image Analysis, IEEE Press-Wiley, [6] U. Qidwai and C.H. Chen, Digital Image Processing, an algorithmic approach with MATLAB, CRC Press, [7] R. Duda, P. Hart, and D.G. Stork, Pattern Classification, second edition, Wiley, [8] S. Theodoridis and K. Koutroumbas, Pattern Recognition, 4 th edition, Academic Press, [9] C.H. Chen, L.F. Pau, and P.S.P. Wang, editors, Handbook of Pattern Recognition, vol. 1, World Scientific Publishing, [10] C.H. Chen, L.F. Pau, and P.S.P. Wang, editors, Handbook of Pattern Recognition, vol. 2, World Scientific Publishing, [11] C.H. Chen and P.S.P. Wang, editors, Handbook of Pattern Recognition, vol. 3, World Scientific Publishing, [12] C.H. Chen, editor, Handbook of Pattern Recognition, vol. 4, World Scientific Publishing, [13] C.H. Chen, editor, Emerging Topics in Computer Vision, World Scientific Publishing, [14] C.H. Chen and G.G. Lee, On digital mammogram segmentation and microcalcification detection using multiresolution wavelet analysis, Graphical Models and Image Processing, vol. 59, no. 5, September 1997, pp [15] M. Kass, A. Witkin, and D. Terzopoulos, Snakes: Active contour models, International Journal of Computer Vision, vol. 1, pp , [16] J.A. Sethian, Curvature and the evolution of Fronts, Comm. in Math. Phys., 101, pp , [17] V. Caselles, R. Kimmel, and G. Sapiro, Geodesic active contours, International Journal of Computer Vision, 22(1), pp , [18] Tony F. Chan and Luminita A. Vese, Active contours without edges, IEEE Trans. on Image Processing, vol. 10, no. 2, pp , [19] C.H. Chen, L. Potdar, and R. Chittineni, Two novel ACM (active contour model) methods for intravascular ultrasound image segmentation, Review of Quantitative NDE, vol. 29A, American Institute of Physics, [20] G. Giannoglou, Y. Chatzizisis, V. Koutkias, I. Kompatsiaris, M. Papadogiorgaki, V. Mezaris, E. Parissi, P. Diamantopoulos, M.G. Strintzis, N. Maglaveras, G. Parcharidis, and G. Louridas, A novel active contour model for fully automated segmentation of intravascular ultrasound Images: in-vivo validation in human coronary arteries, Computers in Biology and Medicine, September 2007; 37:

16 16 C. H. Chen [21] A.R. Gangidi, P. Manandhar, and C.H. Chen, Fast and accurate automated IVUS contour detection and 3D visualization, presented at the 18 th Annual Sigma Xi Research Exhibition at UMass Dartmouth, April 30, [22] M. Roy Cardinal, Segmentation of IVUS images, Ph.D. thesis, University of Montreal, [23] T. Löfstedt, O. Ahnlund, M. Peolsson, and J. Trygg, Dynamic ultrasound imaging A multivariate approach for the analysis and comparison of time-dependent musculoskeletal movements, BMC Medical Imaging, 2012 Sep 27; 12(1):29. [24] A.C. Kak and M. Slaney, Principles of Computerized Tomographic Imaging, SIAM (Society of Industrial and Applied Mathematics) Press, [25] A.C. Kak, Computerized tomography with X-ray, emission, and ultrasound sources, Prof. of the IEEE, vol. 67, no. 9, pp , Sept [26] Y. Sun, I. Liu, and J.K. Grady, Reconstruction of 3D tree-like structures from three mutually orthogonal projections, IEEE Trans. on Pattern Analysis and Machine Intelligence, vol. 16, pp , 1994.