Thermographic Image Analysis Method in Detection of Canine Bone Cancer (Osteosarcoma)

Transcription

1 2012 5th International Congress on Image and Signal Processing (CISP 2012) Thermographic Image Analysis Method in Detection of Canine Bone Cancer (Osteosarcoma) Maryamsadat Amini, Peng Liu and Scott E Umbaugh Computer Vision and Image Processing Laboratory Electrical and Computer Engineering Southern Illinois University Edwardsville (SIUE) Edwardsville, IL , USA Dominic J. Marino and Catherine A. Loughin Long Island Veterinary Specialists (LIVS) 163 South Service Road Plainview, NY, USA Abstract Bone cancer is a pathologic condition which may occur for both humans and canines. This tumor develops quickly from within the bone tissue and become painful as it grows outward. A metastasized bone tumor may be cured by amputation, otherwise it will be fatal. The current diagnostic imaging methods for bone cancer are X-rays, Computed Tomography scan (CT scan), and Magnetic Resonance Imaging (MRI). The disadvantages of these methods include not enough detail in X-ray images, high dose of radiation from CT scans, and high expense of the timeconsuming MRI method. In most of the bone cancer cases, when this tumor is detected by these imaging methods, it has already metastasized. The study is to investigate whether it is possible to detect canine bone cancer by thermography imaging. This alternative imaging method will decrease diagnostic time, expenses and prevent radiation exposure. The best classification success rate obtained in this study is 80.77%. Keywords-bone cancer; osteosarcoma; thermographic image analysis ; feature extraction; pattern classification; CVIPtools I. INTRODUCTION Thermography is a noninvasive diagnostic imaging technique that represents the cutaneous temperature distribution in the form of a color image. Various conditions of disease or injury can influence local dermal microcirculation and temperature. The correlation between temperature recordings and the presence of a disease or injury is the clinical basis for thermography. Clinically, thermography can be used as a diagnostic tool to improve interpretation of physical examination results [1]. Canine bone cancer (or osteosarcoma; osteo = bone, sarcoma = cancer) is a common type of cancer that grows fast and may be fatal. Larger breeds are more prone to bone cancer. Bone cancer most frequently occurs in middle-aged, older dogs, and usually in the limbs, which is called appendicular osteosarcoma. Tumorous bone is not as strong as normal bone and tends to break with slight injury. These types of breaks, which never heal, are called pathologic fractures [2]. Unfortunately, in most of the bone cancer cases, when the tumor is detected by current diagnostic methods such as X- rays, MRI, and biopsy, the malignancy has already metastasized to the adjacent tissues and organs. The metastasized bone tumor may be cured by amputation, otherwise it is fatal. This has encouraged the researchers to look for alternative diagnostic imaging systems such as thermography. The investigation here is to determine if the thermography imaging technique enables the detection of the abnormality even before its visual detection by providing physiological information, which ultimately accelerates the treatment procedure. The investigation includes three main steps: Image segmentation and mask creation Feature selection and extraction Data normalization and pattern classification II. MATERIALS A. Thermographic Images A thermographic image (thermogram) is a color image in which each color represents a specific temperature. The thermograms used in this study use an 18-shade color map. The color map describes warmer temperatures as white and red and cooler temperatures as blue and black. The thermographic images were taken from 22 dogs with bone cancer confirmed by biopsy, in the same room with temperature being controlled at 21 C. The camera was located approximately 1.5 to 4.6 m from the dogs, depending on the region of interest (ROI). The method for imaging includes full left and right lateral limb views of both forelimbs and hind limbs, and cranial and caudal views [1]. A total number of 197 thermal images including cancer and non-cancer classes with four different limb regions are used; full-limb, shoulder/hip, elbow/knee, and wrist. The number of images in each region, considering both cancer and non-cancer classes, are 26 for full-limb region, 35 for shoulder/hip region, 84 for elbow/knee region, and 52 for wrist region. Examples of the thermographic images of the four limb regions of a healthy dog (non-cancer) and a dog diagnosed with cancer (cancer) are shown in Fig. 1 and in Fig. 2, respectively /10/$ IEEE 599

2 Figure 1. (Left to right) full-limb, hip, knee, and wrist regions of non-cancer images including their corresponding ROIs Figure 2. (Left to right) full-limb, shoulder, elbow, and wrist regions of cancer images including their corresponding ROI In the most of the cases, only one of the limbs of each dog is affected by bone cancer. Therefore, the limb opposite the cancerous one is considered as a non-cancer limb. According to this fact, approximately half of the total thermal images are categorized as non-cancer images. B. Masks In order to focus only on the tumor and its metastasized areas of each image, a total number of 197 masks are created (one mask per each image) manually based on the ROI determined by the veterinarian specialists. The masks are created by using the Computer Vision and Image Processing tools (CVIPtools) software [3]. Fig. 3 shows an original image, its corresponding mask, and the mask performance in the ROI extraction. C. Programs and Softwares CVIPtools is used in the manual mask creation for the image segmentation step. This software allows for the manual processing of one image at a time and produces an instant result [3]. Four color normalization methods are implemented on the original images. The four color normalization methods are Luminance, Norm-Grey, Norm-RGB, and Norm-RGB-Lum. The methods are utilized to remap colors corresponding to the different temperatures in the thermographic images. Although each color represents a specific temperature with in each image, the color to temperature mapping for an image is affected by the camera setting at the time of the image capture. Therefore one color may represent several different temperatures across the images which is caused by camera recalibration between each image capture [4]. The Luminance method generates a gray level value for each color component based on the linear equation in which each color band is weighted as follows: GreyLevel = 0.3 Red Green Blue (1) In the Norm-Gray method, temperatures from the minimum to the maximum are mapped to the gray levels from 0 to 255. The gray level of each pixel in the normalized image represents the corresponding pixel temperature in the original image. Norm-RGB is similar to the Norm-Grey, but temperatures are mapped to a continuous version of the original color palette instead of gray levels from 0 to 255. Norm-RGB-Lum method applies Luminance color normalization method to the Norm-RGB result [4]. The results of the four color normalization methods and their corresponding original image are shown in Fig. 4. The Computer Vision and Image Processing-Feature Extraction and Pattern Classification (CVIP-FEPC) software provides both feature extraction and pattern classification in a single run [5]. This software implements each pattern classification method on all the combinations of the features selected by the user. The pattern classification results are reported by a total correctnesss classification rate and also the classification rate for each class, by which the sensitivity and specificity of each set of experiment can be calculated. By using the software, the best feature set for each experiment can be extracted according to the best classification success rate. Figure 3. Original image and its corresponding mask extraction illustrating the ROI Figure 4. (Left to right) Luminance, Norm-Grey, Norm-RGB, Norm-RGB- and the original Lum color normalized images image Long Island Veterinary Specialists, Veterinary Thermographic Image Analysis: Canine Bone Cancer, 2012 Southern Illinois University Edwardsville, School of Engineering, Electrical and Computer Engineering Department 600

3 The Partek Discovery Suite is data analysis software used to apply six different pattern classification methods [6]. In this study, the best feature sets selected by the best CVIP-FEPC classification success rates are analyzed by these other pattern classification methods. III. METHODS The research study follows three principal steps to determine the diagnostic ability of thermographic images for canine bone cancer detection. The procedure is: Image segmentation and mask creation Feature selection and extraction Data normalization and pattern classification The first step provides only the ROI of the thermal images by mask creation to avoid storage and analysis of unnecessary information. All of the image masks are created manually. In the second step, several features such as histogram and texture features, which are used for analysis and classification, are extracted. In the last step, the extracted information are normalized and classified into two classes of cancer and noncancer. Also, the classification correctness metrics (or the success rates) are evaluated for each of the implemented pattern classification method. The number of sample images that are classified correctly represents the success rate, in other words, the success rates indicate whether the thermal images correlate with biopsy results. The procedure is applied on either the original or one of the four types of color normalized images which makes a total of five different experiment sets. Since each of the experimental sets is performed on each of the four limb regions and the total images, a total number of 25 different experimental sets are analyzed based on the different limb regions and color normalization methods. A. Mask Creation At first, the manual masks are created based on the ROIs determined for each image by the veterinarian specialist in the Long Island Veterinary Specialists (LIVS). The masks are created by using the original images. However, they can be applied to extract the ROI for both of the original and color normalized images. Extracting the ROI decreases the complexity (time and memory) of the image processing procedure. B. Feature Selection and Extraction In this step, the feature types to be extracted for each experiment set are defined. Each experiment set includes as many single experiments as the number of all combinations of selected feature types. For experiment sets using original images, 10 feature types are extracted including spectral, four histogram, and five texture feature types. The histogram feature types include histogram standard deviation, skew, energy, and entropy. The texture features are texture energy, inertia, correlation, inverse difference, and entropy. The total number of 10 feature types produces 1024 experiments implemented on all combinations of the selected features. After color normalization, we can rely on each color to be a specific temperature in all the images, so the histogram mean feature also is added to the mentioned 10 feature types for experiment sets using color normalized images. Totally, each experiment set implemented using color normalized images includes 2048 different experiments to analyze each combination of the 11 feature types. The mentioned number for each feature represents only the number of different types of histogram, texture, and spectral features. However, in the real experiments with the CVIP- FEPC, the total number of 43 features is extracted for original images and 46 features for the color normalized images. Since each histogram feature is extracted for three bands of red, green, and blue (RGB), the four types of histogram features would be in total 12 features. In case of using color normalized images, the total number of 15 histogram features are extracted. Also, for each type of texture feature (five types), the range and average of the four directions are calculated which results in total 10 texture features. Moreover, the spectral feature is measured for three rings and three sectors. The spectral features of each sector and each ring are extracted for three color bands (RGB) which makes 18 spectral features. The number of 18 spectral features also should be added to three features of the spectral DC value for three color bands, so in total there are 21 spectral features. C. Data Normalization and Pattern Classification All the extracted feature values are normalized using the data normalization methods including standard normal density normalization and softmax scaling with r = 1 [7]. Feature extraction, data normalization, and pattern classification are implemented by CVIP-FEPC, simultaneously. In addition to CVIP-FEPC, Partek Discovery Suite is also used for applying more pattern classification methods. In both of the CVIP-FEPC and the Partek software, full leave-one-out is the selected testing method for all the performed experiments. The pattern classification method of K-Nearest-Neighbor (KNN) with K = 4 is applied in the CVIP-FEPC. The CVIP- FEPC experiments are performed on all the combinations of the (10 or 11) feature types and two mentioned data normalization methods, which make the experiment sets comprise 2048 or 4096 single experiments. The experimental sets not only give us the pattern classification results but also provide the significant feature types by the ranking of the classification success rates including their corresponding feature sets. The feature sets related to the best classification success rates are selected to insert to the Partek software for other pattern classification methods analysis. Therefore, we implement six pattern classification methods provided by the Partek software only on the feature sets including significant feature types. By using the Partek software, six different pattern classification methods are applied including linear discriminant analysis with equal prior probability, linear discriminant analysis with proportional prior probability, nearest centroid with equal prior probability, KNN with K = 1, 3, and 5. Overall, we obtain the classification success rates of the seven different pattern classification methods on the same data 601

4 (best feature sets selected by the best results of the KNN with K = 4). Based on the success rates comparison of the seven classification methods, the KNN with K = 1 (Nearest- Neighbor) performance is better than other methods. Accordingly, we implemented the Nearest-Neighbor pattern classification method again, but this time, this classification method is applied on all the combinations of the (10 or 11) feature types by CVIP-FEPC. By ranking the success rates with their corresponding feature sets, we determined the best feature sets obtained by the best Nearest-Neighbor classification results. IV. RESULTS Totally, 200 different sets of experiments are implemented by the CVIP-FEPC and Partek software, from which 50 experiment sets are executed by CVIP-FEPC, and the Partek software is applied for 150 experiment sets. From CVIP-FEPC experiments, 25 experiment sets are performed by KNN with K = 4 classification method for five limb region conditions (four limb regions and total limb regions), and five color normalization conditions (four color normalization methods and non-color normalization). The best success rates produced by KNN with K = 4 is illustrated in Figure X. As shown in Figure X, among all the 25 CVIP-FEPC experiment sets using KNN with K = 4, the full-limb region with Norm-RGB-Lum color normalized images produces the best classification success rate of 76.92% with a sensitivity value of 84.62% and a specificity value of 69.23%. A total number of 150 experiments are executed with the Partek software, including all combinations of the five limb region conditions, five color normalization conditions, and six classification methods. The best correct classification rates obtained by the Partek software are shown in Figure Y in Appendix A. In the experiments, the best correct classification rate, 80.77%, is obtained by the full-limb region images which are color normalized by Norm-RGB-Lum and classified by the Nearest-Neighbor. Among all the six classification methods, Nearest-Neighbor and KNN with K = 3 and K = 5 produced the highest classification rates (shown in Figure Y), from which Nearest-Neighbor performance was better for more limb regions. This encourages us to perform Nearest-Neighbor classification method by CVIP-FEPC to determine the best feature sets from all the combinations of the (10 or 11) selected feature types. A total number of 25 experiment sets are performed by CVIP-FEPC to determine the most significant feature types based on the Nearest-Neighbor best classification success rates. The experiment sets includes all combinations of the five limb region conditions and five color normalization conditions. The best success rates for each limb region are shown in Figure Z in Appendix A. The highest classification success rate is 80.77% (shown in Figure Z) with a sensitivity value of 92.31% and specificity value of 69.23%. The best success rate is achieved by Norm-RGB-Lum color normalized images of the full-limb region. According to the best success rates of the KNN with K = 4 and their corresponding feature sets, histogram standard deviation, histogram skew, and texture correlation appear frequently in the feature sets. Also, based on the best success rates of the Nearest-Neighbor classification method, histogram standard deviation, histogram skew, and texture inverse difference are the most three significant features. V. SUMMARY The purpose of this research study is to determine the capability of thermographic imaging for canine bone cancer detection. The currently used diagnostic imaging methods provide significant disadvantages. Although, in all the experiments performed in this study, we apply heterogeneous data including canine with different breeds and genders, the color normalized images with Norm-RGB-Lum of the full-limb region produced the best classification success rate of 80.77%. The number of thermal images in the specific experiment is 26 including 13 cancer and 13 non-cancer thermograms. Three features are extracted in the experiment which are texture inverse difference, histogram skew, and histogram entropy with using standard normal density data normalization method. The best success rate is obtained by Nearest-Neighbor method. Given more thermographic images it may be possible to perform separate experiments for each canine breed which may increase the success rates. VI. FUTURE SCOPE In this study, we performed the analysis for different canine breeds in a single set of experiments, due to the small amount of data available. We may be able to increase the success rates by implementing the experiments for each breed separately. REFERENCES [1] C. A. Loughin, DVM, and D. J. Marino, DVM, Evaluation of thermographic imaging of the limbs of healthy dogs, American veterinary Medical Association (AVMA), vol. 68, No. 10, pp , October [2] Vetinfo. The dog bone cancer. Retrieved 15 May [3] Computer Vision and Image processing Tools (CVIPtools). Retrieved [4] S. E Umbaugh, P. Solt, Veterinary thermographic image analysis. Data and temperature normalization. SIUE CVIP Laboratory report number , January 23, 2008, unpublished. [5] CVIP-FEPC. Retrieved [6] Partek Discovery Suite. Retrieved [7] S. E Umbaugh, Digital image processing and analysis: human and computer applications with CVIPtools, second edition, The CRC Press, Boca Raton, FL,

5 Appendix A CVIP-FEPC Best Classification Success Rates KNN with K = % 74.28% 73.08% % 65.48% CVIP-FEPC Best Classification Success Rates Nearest-Neighbor % 76.92% % 69.05% % Figure X. CVIP-FEPC best classification success rates for KNN with K = 4 method based on each limb region Figure Z. CVIP-FEPC best classification success rates for Nearest-Neighbor method based on each limb region Partek Best Classification Success Rates % 80.77% 71.40% 65.48% 77.57% Figure Y. Partek best classification success rates with their corresponding classification methods based on each limb region 603