Indian Journal of Science and Technology, Vol 8(25), DOI: 10.17485/ijst/2015/v8i25/80492, October 2015 ISSN (Print) : 0974-6846 ISSN (Online) : 0974-5645 A Study of Medical Image Analysis System Kim Tae-Eun * Department of Multimedia, Namseoul University, South Korea; tekim5@empas.com Abstract This study aims to develop an image analysis system by using medical imaging and stereo cameras. The entire system is organically integrated by developing modules that perform image display functions such as image processing, image segmentation, three-dimensional modeling, etc., modules related to stereo camera system, and modules that match data and detect changed parts. For integrated software of the integrated system, with the development of modules which perform image display functions as image processing, image segmentation, three-dimensional modeling, etc., and of modules related to stereo camera system, data matching and change detection modules, the entire system is organically integrated. This study proposes the probe location extraction method by using of optical mode means. The users can easily execute tasks with the system by using a GUI that was arranged to perform functions of the system. The technologies developed in this study, such as the stereo matching technique, three-dimensional modeling, and image segmentation, could be used as the most basic technologies for producing three-dimensional simulators, etc. Keywords: Medical Imaging, Probe, Segmentation, Stereo Camera 1. Introduction Image diagnosis technology has made bigger advancements in the recent times, and it is being developed under the current trend into three-dimensional image technology, which enables thorough examination of the inner body without dissecting it. In order to raise the accuracy of surgical procedures when performing a precise operation such as brain surgery, the information the doctor wants, that is whether the cancerous region found in the two-dimensional medical image corresponds to which part in the actual third dimension, and the area or volume of the part to be removed, should be known in advance prior to the operation. Thus, by means of segmenting the medical image taken in advance through the automatic/semiautomatic technique for each tissue of interest, and by applying three-dimensional modeling for each tissue, it is possible to calculate the location or area, volume of each tissue. Also, once the actual surgery has begun, in order to monitor the subcutaneous surgical procedure, some form of image from the location of the needle is necessary. In order to obtain this, the patient receiving surgery is adjusted to the pre-made threedimensional model, after which a probe is placed in the surgical site of the surgical patient. The image of the probe is obtained with two cameras, and then the threedimensional location is extracted through the stereo matching technique. By means of examining in advance the surgical site or the surrounding tissue by using the three-dimensional location extracted in such a way, a prearranged plan for a surgery can be made, or it can be used as a support system in either a simulated surgery or in an operating room. The three-dimensional data pre-made from medical images and the actual patient are matched, and the practitioner can check in real time whether the region in the patient touched by the pointer corresponds to certain parts in the pre-made three-dimensional data; hence, the practitioner or the person planning a surgical operation can perform a more precise surgery while monitoring * Author for correspondence
A Study of Medical Image Analysis System the surgical site with their own eyes. Also, by providing information on the regions that can be overlooked by the doctor through the semiautomatic/automatic detection of the changed part in the perioperative image, a more precise image diagnosis is made possible. Such a system can be used for educational purposes in performing a simulated surgery. using medical image systems as MRI or CT through the DICOM (Digital Imaging and Communications in Medicine) protocol, and composes a volumetric image with these data. 2 2. Research Content 2.1 The Coordinate System Conversion of the Image System The system developed in this study carries out the acquisition of medical image and camera calibration in the initial stage. Here, camera calibration means initializing the system in order to use indicates system initialization for using the stereo camera system. When camera calibration is performed, the coordinate system is set up in the stereo camera system, and the threedimensional coordinate values can be calculated through stereo matching. For the next stage, which is for matching coordinate systems, the matching points of coordinate systems are is extracted within the medical image. These matching points of coordinate systems are used for matching coordinate systems together with the matching points of coordinate systems extracted while an operation is in progress. While the operation is in progress, the calculation of the patient and the probe location, and medical image display are mainly performed using the stereo image system. First, matching points of coordinate systems are extracted using the stereo image system, and then the three-dimensional coordinates of these matching points are calculated with the stereo matching method. Coordinate systems are matched by using the threedimensional coordinates calculated in the above method and by using the coordinates of the matching points of coordinate systems extracted from the medical image in the surgery preparation process. Once coordinate systems are matched, the coordinate system transformation matrix between the medical image system and the stereo image system is calculated, and so transformation between two coordinate systems can be easily performed. 2.2 Medical Image Display 2.2.1 Medical Image Display Procedure Medical image display receives the images acquired Vol 8 (25) October 2015 www.indjst.org Figure 1. Example of a tomographic image and the threedimensional image of an experimental phantom. In this study, medical images are transmitted using DICOM protocol, and displays them as tomographic images or a three-dimensional reconstructed images to the user. For the image display, the volumetric imaging task is performed in the image transmitted by the DICOM protocol; also, for three-dimensional imaging, the division of interest tissue or the division of object/background is performed. In the case of twodimensional tomographic imaging, displaying is done using the volumetric image without performing other additional tasks in the volumetric image. Figure 1 shows the two-dimensional tomographic image and the threedimensional reconstructed image of the MR image of an experimental phantom. 2.2.2 Medical Volumetric Image Medical volumetric image is an image which is structurally made so that two-dimensional tomographic images or three-dimensional images can be easily restored from the acquired original image. For volumetric images, the starting point and direction are determined by the configuration at the time of acquiring the image, and the three-dimensional coordinate system is defined to express the volumetric image. Volumetric image is an image basic to all processes of medical image display. When volumetric image is scanned using tomography in the direction of x-y, y-z, z-x planes, the two-dimensional tomographic image of coronal view, sagittal view, and axial view are composed for each. In order to restore volumetric image Indian Journal of Science and Technology
Kim Tae-Eun to a third dimension, three-dimensional elements for each image must be identified. Also, the elements regarding the pixel information of the image must be extracted for three-dimensional imaging. Such properties must have the same value for each image file. 2.3 Medical Image Processing In this research and development, emphasis on image processing was placed mainly on allowing the user to easily identify an image. In most cases, the contrast of an image is low because there are wide variations in the brightness value of an object acquired according to the characteristics of an object, and also because the brightness resolution used by an acquisition machine is high. While the general image data use 8 bits to express one pixel, medical image data express pixel by using 12 bits or 16 bits. Data expressible with 8 bits hold 256 values, ranging from 0 to 255; by comparison, 12 bits can express 4096 bytes of data, and 16 bits can express 65536 bytes of data. However, since the representation method of image brightness of computer is restricted to 8 bits, image processing is inevitable in order to display medical image with the brightness resolution of more than 8 bits,. In the case of displaying an image acquired with a 16-bit brightness resolution at 8-bit brightness resolution, there would not be a problem if the data value of the acquired image existed between 0 and 255, but there would be a problem if the data value of the acquired image was more than 255. In order to express an image while maintaining brightness differences or contrast that exist between the internal tissues of an object, the method used is that of expressing by expanding or reducing the differences of the image brightness values. For 16-bit images, the data value of medical image exists between 0 and 65535, and this must be expressed by mapping the value between 0 and 255. When x is considered the original value of medical image data, and y is considered the value after processing, if the user sets min, level, max values between 0 and 65535, (provided, min < level < max) an image can be expressed with the converted brightness value by using a conversion formula as shown below. y = 0, where, x< min or x> max x- min level- min y = x128, where, x< level x- level max y = x128+ 128, where, x> level Equation 1. Brighteness Conversion Formula Figure 2. Leveling. Figure 2 depicts the mapping characteristics expressed using min, level, max values. The adjustment of brightness value is possible through randomly setting min, level, max values by the user. Here the maximum expression value M has the value between 128 and 256 according to min, level values. 2.4 Object/Background Division using a Histogram Separation of object and background can be easily executed using a histogram, and for this task, automation is possible. Figure 3(a) and Figure 3(c) show a tomographic image of a volumetric image and a histogram of pixel values of a volumetric image. As can be seen in this histogram, the image pixel values can be classified largely into two groups. Considering the point that the image signal values exist in the object, it can be known that the upper group in the histogram is defined with the pixel values of the object. The process of dividing object/background is as shown in Figure 3. First, the task of creating a histogram of volumetric image is performed. The histogram is calculated with the brightness value of volumetric image pixels. Second, by analyzing the histogram obtained in such a way, the most suitable binary-coded value is calculated. In this analysis and in the calculation process of binary-coded values, Otsu s method 1 is applied. This method calculates the binarycoded value which minimizes intraclass variance by using a histogram. Third, volumetric image is binarized by using the binary-coded value obtained with Otsu s method. For the binary-coded volumetric image, object is expressed as 255 or 65535, and background is expressed as 0. Vol 8 (25) October 2015 www.indjst.org Indian Journal of Science and Technology 3
A Study of Medical Image Analysis System Fourth, object and background are differentiated through the process of labeling. If this division information is used in imaging, three-dimensional surface imaging of object is possible. Figure 3(b) shows the object area with separated background through such a process. Since in Otsu s method, the binary-coded value is automatically calculated, for this separation of background, automatic performance is possible. 2.5 Three-Dimensional Modeling There is the volume rendering method different from the one above which takes two-dimensional image array as one volume, and directly projects it on the surface. This method is a concept which expanded on surface rendering. Also, one pixel of two-dimensional image becomes one voxel of three-dimensional volume, thereby rendering this voxel just as the vertex of each surface is rendered in surface rendering. Each voxel rendered in such a way, through the process of modeling, viewing and projection transform, is projected on the final image surface. For such a method, there are methods such as Ray-casting, splatting, etc. also, there is the shear-warp factorization method 2 which developed the splatting method. Both 3D reconstruction and volume rendering have their merits and demerits. (a) (b) (c) Figure 3. Separation of Object and Background. (a) Tomographic Image. (b) Background Separation. (c) Object Histogram. Figure 4. Marching Cubes. In this study, by providing 3D reconstruction using marching cubes and volume rendering using shear-warp factorization, each other s strengths and weaknesses were complemented. Also, by providing UI (User Interface) that is convenient for the user, the three-dimensional structure was made so that it could be easily understood. For example, by showing a three-dimensional image together with a two-dimensional image slice, they can be compared to each other; also, the structure of each tissue was made to be grasped more clearly. Real-time rendering allows easy observation of a three-dimensional model through the mouse. 4 Vol 8 (25) October 2015 www.indjst.org Indian Journal of Science and Technology
Kim Tae-Eun 2.6 Data Matching 2.6.1 The Coordinate System In this study, there are two coordinate systems used for obtaining data, and for expressing the coordinates of the relevant data. Namely, the camera coordinate system and the MRI coordinate system. If four or more control points are used, it is possible to calculate transformation matrix between these two coordinate systems also, after calculating this transformation matrix, it is possible to express an arbitrary point, obtained in one coordinate system, in another coordinate system. If such characteristics are used, three-dimensional location of the probe obtained from a stereo camera image can be expressed in MRI image. The camera coordinate system is a coordinate system which takes a picture of an object with a stereo camera, and which is defined to obtain three-dimensional coordinates. The starting point and the coordinate axis can be randomly set also, in this study, the upper left of chessboard pattern of camera calibration is used as the starting point of the coordinate system, and chessboard pattern is used as x-y plane of the coordinate system. The three-dimensional coordinate value obtained through stereo matching is the value expressed by this coordinate system. Compared to this, MRI coordinate system is a coordinate system which is defined to acquire MRI. Three planes of coronal plane, sagittal plane and axial plane are defined as coordinate systems forming x-y, y-z, z-x planes respectively. For matching between these two coordinate system, by using four or more control points found in stereo camera image and MRI, transformation matrix between coordinate systems can be calculated using the pseudo inverse method. 2.7 The Stereo Camera System Image acquired through the camera appears as twodimensional since the three-dimensional world is projected through a lens. By acquiring such a twodimensional image with two left and right cameras, it is possible to restore three-dimensional information. With this method, the three-dimensional location and direction of the right camera can be obtained, and it is possible when each corresponding point formed in the left and right cameras can be found. For this, the methods of camera calibration and pattern recognition were used in this study, and the three-dimensional coordinates of particular points were obtained by using the stereo matching technique. As for the methods for camera calibration, various methods have been researched so far, and in this study, the method proposed by Heikkila 3-5 was used. Camera calibration implies calculating extrinsic parameters such as rotation and translation of camera, and intrinsic parameters, such as the focal distance of camera, distortion of lens, etc., with a particular point as the reference point in three-dimensional space. 3. Experiments and Results 3.1 Experiment Environment The composition of the system developed in this study is largely divided into two parts: the medical image system and the integrated system. The medical image system Figure 5. Integrated Development Environment. Vol 8 (25) October 2015 www.indjst.org Indian Journal of Science and Technology 5
A Study of Medical Image Analysis System acquires the anatomy of a patient; and the integrated system performs the integrated system functions, such as the acquisition request for medical images, control of the stereo camera system, and the medical image display. In particular, the interaction between the doctor and the image display system is made possible by using probes. The integrated system performs a function of integrating data acquired from the medical image system and stereo camera. In the integrated system, the integrated software, which performs the aforementioned functions, is developed and installed. The integrated software displays images acquired by the medical image system and incoming stereo camera images in real time, and also effectively displays medical images by calculating the location of patient and probe existing within stereo camera image. In addition, it performs many functions such as requesting for the acquisition of medical images, saving acquired medical images, dividing tissues within a medical image, creation and display of a threedimensional model, etc. Integrated software is a software mounted on an integrated system, and has GUI as shown in Figure 5. It is used for showing tomographic image of acquired medical image, three-dimensional reconstructed image, or stereo camera image. The buttons and various controls on the right allow easy manipulation of various functions the software provides. 4. Conclusion In this study, the image analysis system was proposed. The entire system is composed of the medical image system for acquiring medical images, the camera system for tracking the location of the patient and probe, and the system which performs tasks as image display, etc. The entire system operates in mutually coordinated union of modules performing image display functions such as image processing, image segmentation, threedimensional modeling, etc., the modules related to the stereo camera system for tracking the location of probe and patient, and the modules of data matching and changed area detection. Camera calibration must be performed for initialization of the stereo camera system, and for this task, a tool with chessboard pattern was used. There are advantages of easy feature points extraction for performing calibration, and of accurately defining image coordinates of the feature points in sub-pixel units. For the image segmentation process, which is a preprocessing stage of three-dimensional image reconstruction, convenience was maximized for the user by allowing semi-automatic segmentation using GUI for segmentation between tissues and automatic segmentation using a histogram for segmentation between object and background. This study was developed as a computer medical image analysis system. It is expected that the research methods and results will be valuably used for providing information on changes in tissues before and after the operation, and information on three-dimensional positions between tissues for experts and students who study anatomical structure. 5. Acknowledgment Funding for this paper was provided by Namseoul University. 6. References 1. Otsu N. A threshold selection method from gray-level histograms. IEEE Transactions on Systems, Man, and Cybernetics. 1979; 9(1):62 6. 2. Lorensen WE, Cline HE. Marching cubes: A high resolution 3D surface construction algorithm. Proccedings of SIGGRAPH 87 on Computer Graphics. 1987; 21(4):163 9. 3. Heikkila J. Geometric camera calibration using circular control points. IEEE Transactions on Pattern Analysis and Machine Intelligence. 2000; 22(10):1066 77. 4. Press WH, Teukolsky SA, Vetterling WT, Flannery BP. Numerical recipes in C - the art of scientific computing. 2nd ed. Cambridge University Press; 1992. 5. Pluim JPW, Maintz JBA, Viergever MA. Interpolation artefacts in mutual information-based image registration. Computer Vision and Image Understanding. 2000; 77(1):211 32. 6 Vol 8 (25) October 2015 www.indjst.org Indian Journal of Science and Technology