Rupture Disease. by Jiyuan Fu, Bachelor of Science

Transcription

1 1 Using Thermographic Image Analysis in Detection of Canine Anterior Cruciate Ligament Rupture Disease by Jiyuan Fu, Bachelor of Science A Thesis Submitted in Partial Fulfillment of the Requirements for the Degree of Master of Science in the field of Electrical and Computer Engineering Advisory Committee: Scott E Umbaugh,Chair Brad Noble Robert LeAnder Graduate School Southern Illinois University Edwardsville December, 2014

2 2 ABSTRACT 3 USING THERMOGRAPHIC IMAGE ANALYSIS IN DETECTION OF CANINE ANTERIOR CRUCIATE LIGAMENT RUPTURE DISEASE by JIYUAN FU Chairperson: Professor Scott E Umbaugh Introduction: Anterior cruciate ligament (ACL) rupture is a common trauma which frequently happens in overweight dogs. Veterinarians use MRI (Magnetic resonance imaging) as the standard method to diagnose this disease. However MRI is expensive and timeconsuming. Therefore, it is necessary to find an alternative diagnostic method. In this research, thermographic images are utilized as a prescreening tools for the detection of ACL rupture disease. Additionally, a quantitative comparison is made of new feature vectors based on Gabor filters with different frequencies and orientations. Objectives: The main purpose of the research study is to investigate whether thermographic imaging can be used effectively in canine ruptured anterior cruciate ligament (ACL) disease detection. And to determine whether using the Gabor Filter with different frequencies and orientations for the new feature extraction can improve the result. Methods: The mask made manually will be used for focusing on the region of interest (ROI).For the canine anterior cruciate ligament ruptures investigation, four color normalization methods are implemented on each category based on three different views: anterior, lateral, posterior. Histogram, texture and spectral features are extracted by CVIP- ii

3 FEPC. After these twelve filters being convolved with the thermographic image, the new feature vectors are used for pattern classification. Results: When using first/second order histogram features, the best classification rate of anterior view is 83.93% which is produced by the NormGrey images. The best classification rate of lateral view is 83.93% which is from NormGrey and NormRGB images. The best classification rate of posterior view is 82.14% which is from NormRGB-lum images. Using the Gabor filter based features for the anterior, lateral and posterior view images, the best classification success rate 87.50%, 83.93%, 85.71% was achieved respectively. Comparing to results using the first/second order histogram features, the best classification rates using Gabor filter increased from 0.00% to 3.57%. Conclusion: It is possible to detect the canine Anterior Cruciate ligament (ACL) ruptures with thermographic images. Also images with Gabor filter processing which involve frequency and orientation information provide a small improvement in the best success rate. iii

4 4 ACKNOWLEDGEMENTS First and foremost I would like to express my great appreciation to my advisor, Dr. Scott Umbaugh, for all his generous and professional support and guidance during my master s study. I am very grateful for what he has done for me. I am lucky and also proud of being a student of his. Also,I am very thankful to my advisory committee members, Dr. Brad Noble and Dr. Robert LeAnder, for their help and patience. Besides, I want to provide my true gratitude to my fellow group members, Samrat Subedi, Ravneet Kaur, Krishna Regmi, Heema Poudel and Hari Bhogala for their great help. Also I would like to thank Dr. Dominic J. Marino and Dr. Catherine A. Loughin from Long Island Veterinary Specialists [LIVS] for providing funding and help for this research. Last but not the least I would like to thank my whole family for their love and encouragement. iv

5 TABLE OF CONTENTS ABSTRACT... ii ACKNOWLEDGEMENTS... iv LIST OF FIGURES... vii LIST OF TABLES... i Chapter INTRODUCTION Objectives of the Thesis Outline of the Thesis LITERATURE REVIEW Background The Infrared Thermographic Image and Image Processing Technique The Infrared thermographic technique Image processing technique Gabor Filter MATERIALS AND TOOLS The Thermographic Images Border Masks Software Tools CVIPtools v5.5d CVIP-FEPC (Feature Extraction and Pattern Classification) Matlab v2013a Color normalization software Microsoft Excel METHODS AND EXPERIMENTS Border Masks Color Normalization Feature Extraction Histogram features Spectral features Texture features Gabor filter v

6 4.4 Pattern Classification Data normalizaiton Distance and similarity measures Classification algorithm Success measure and evaluation RESULTS AND ANALYSIS ACL Disease Detection by Using CVIP-FEPC Anterior view Lateral view Posterior view Summary Using Gabor filter in Thermographic Image for ACL Detection Anterior view Lateral view Posterior view Summary SUMMARY AND CONCLUSION FUTURE SCOPE REFERENCES APPENDICES I. Combined Color Normalization Results vi

7 LIST OF FIGURES Figure Page 3.1 Thermographic Images of a Canine with Anterior View Thermographic Images of a Canine with Lateral View Thermographic Images of a Canine with Posterior View Region of Interest to Detect ACL Disease and Original Image Manual Masks Created from Anterior, Lateral and Posterior View An Original Image and Four Corresponding Color Normalization Images Four Different Wavelengths of Gabor Filter in Spatial Domain Four Different Wavelengths of Gabor Filter in Frequency Domain Three Different Orientation of Gabor Filter in Spatial Domain Three Different Orientation of Gabor Filter in Frequency Domain Three Different Aspect Ratios of Gabor Filter Grey Image Convolved With Gabor Filter without Remapping Grey Image Convolved Gabor Filter Remapping in Spatial Domain Grey Image Convolved Gabor Filter Remapping In Frequency Domain The Best Success Result with Different Color Normalization Methods The Statistic of Features with Best Result of Fifteen Experiments Best Success Rate by Involving Gabor Filter with Different Parameters vii

8 LIST OF TABLES Table Page 3.1 The Number of Thermographic Images in Each Category Results for Original and Color Normalized Images of Anterior View Group Detail for Original and Color Normalized Images Of Anterior View Group Results for Original and Color Normalized Images of Lateral View Group Detail for Original and Color Normalized Images of Lateral View Group Results for Original and Color Normalized Images of Posterior View Group Detail for Original and Color Normalized Images of Posterior View Group a Results Involved Gabor Filter with 3.5 Wavelength of Anterior View Group b Results Involved Gabor Filter with 4 Wavelength of Anterior View Group c Results Involved Gabor Filter with 4.5 Wavelength of Anterior View Group a Results Involved Gabor Filter with 3.5 Wavelength of Lateral View Group b Results Involved Gabor Filter with 4 Wavelength of Lateral View Group c Results Involved Gabor Filter with 4.5 Wavelength of Lateral View Group a Results Involved Gabor Filter with 3.5 Wavelength of Posterior View Group b Results Involved Gabor Filter with 4 Wavelength of Posterior View Group c Results Involved Gabor Filter with 4.5 Wavelength of Posterior View Group. 54 i

9 5 CHAPTER 1 6 INTRODUCTION Ligaments are injury prone for both humans and animals. When the anterior cruciate ligament (ACL) ruptures, the ligament may become malpositioned. Also, sports might cause second-time injury to the ACL. Each year, the incidence rate of ACL rupture disease appears to be increasing and therefore exploring an efficient method to detect ACL rupture is necessary. For current methodology, ultrasonography, x-ray, computed tomography (CT) and magnetic resonance imaging (MRI) diagnostic methods are commonly utilized for detection of the diseases. Comparing with others, the MRI is the optimal method. But, it is costly and time consuming. Additionally, it is quite difficult for an injured dog to remain in the same position for the long time required for the MRI and the radiation exposure can be harmful. To reduce the overall cost and save time, the goal of this research study is to investigate the efficacy of the thermographic imaging diagnostic method for canine ruptured cruciate ligament detection. This research utilizes of computer imaging processing and pattern recognition techniques in detection of canine ACL rapture disease. Those thermographic images taken under the infrared camera from Long Island Veterinary Specialists (LIVS) are divided into normal, or healthy and abnormal, or injured categories. According to different camera views, these thermographic images are also divided into contralateral anterior, lateral and posterior views.

10 2 In general, images contain a huge amount of data. Image processing and computer vision techniques are used to extract pertinent features to minimize the redundant information. Feature extraction is used convert the image data into a feature space. The feature space is established with different combinations of the features, which are automatically extracted using CVIP-FEPC. Finally, these features vectors are utilized for pattern classification. Also, to improve the possibility to achieve the goal, the Gabor filters are utilized in this research to compare with the first/second order histogram feature measurements The Gabor filter is a linear filter which is appropriate for texture segmentation and discrimination, and is used here to generate new features. 1.1 Objectives of the Thesis The main purpose of the research study is to investigate whether thermographic imaging can be used effectively in canine ruptured anterior cruciate ligament (ACL) disease detection. The objectives of this research are shown as follows: Determine the efficacy of using thermographic images to classify canine ACL rupture disease Implement Gabor filter with different frequencies and orientations. Classify the images by using first/second order histogram feature measurement and new feature measurements which involve the Gabor filter. Determine the best combination of features for first/second order histogram feature measurement and best orientation and frequency for Gabor filter feature measurement.

11 3 Make a comparison of result between standard texture features based on histograms to Gabor filter features. Analyze and compare results for different camera views Analyze and compare results for different color normalization methods

12 4 1.2 Outline of the Thesis As follows: Chapter 1 concisely introduce what in this research. Chapter 2 presents background and literature review of this study, the technique of infrared imaging and the Gabor filter. Chapter 3 provides a brief introduction of the materials and software used in this research. Chapter 4 presents the specific processing for experiments. Four main processes are involved: border mask creation, color normalization, feature extraction, pattern classification. Also, the Gabor filter implementation is discussed. Chapter 5 discusses and analysis the results of first/second order histogram feature measurement and Gabor filter feature measurement. Chapter 6 summarizes the results and conclusion of experiments. Chapter 7 shows the future outlook of this study.

13 5 7 CHAPTER 2 8 LITERATURE REVIEW Diagnostic imaging is a process for acquiring visual representations of the human or animals body for medical analysis and research. It seeks to reveal internal structures hidden by the skin and bones [Nick; 2014]. Therefore the diagnostic method is an extremely important part of diagnosing the diseases and determining treatment of diseases. The aim of this paper is to compare different feature extraction methods while diagnosing diseases with thermographic images. The object of this chapter is to provide some relevant arguments and background knowledge. This section contains three categories:1) Background, 2) Infrared thermographic image and image processing technique, 3) Gabor Filter technique. 2.1 Background Anterior cruciate ligament (ACL) rupture is a common trauma for canine. The anterior cruciate ligament controls rotational movement and prevents forward movement of the tibia in relation to the femur [Hyalay; 2012].When bruised, the ligament may become malpositioned or ruptured injuries can occur anytime even within normal activity levels. It happens often in overweight canines, because obesity provides more pressure to the ligament [Foster; 2010]. A variety of diagnostic methods have been widely used in detection of the disease.typically it is required to scan through the body of patients. This technique can be helpful for the physician to determine the position of the injury. Diversified diagnostic methods can be used for different cases.

14 6 Currently,diagnostic methods may vary depending on different diseases. Ultrasonography, x-ray, computed tomography (CT) and magnetic resonance imaging (MRI) diagnostic methods are typical methods for disease detection. Most of methods are painful and time consuming [Freddie;2008]. Additionally, some methods are invasive which may be harmful to patients. Among these methods, the MRI imaging method is considered as the most optimal method. Magnetic resonance imaging (MRI) can detect the internal organs by scanning the body with magnetic waves. MRI is excellent for physical examination. Comparing with CT and other diagnostic methods, MRI offers high-level resolution and is noninvasive [Naranje; 2008][Huysse,;2008] It is required that patients or animals be keep motionless under the magnetic scanning in this case. Meanwhile, it is very difficult for dogs to maintain the same posture for a long time and it may be harmful to them to stand long in strong radiation. Also, MRI is an expensive and complex diagnostic method. While MRI has its benefits, it is potentially beneficial to find a new alternative diagnostic method. To reduce the cost and save time, the infrared thermographic image technique is used for this research for canine ACL injury disease detection. 2.2 The Infrared Thermographic Image and Image Processing Technique Digital Infrared Thermal Imaging (DITI) is a noninvasive diagnostic method which has been widely used in the medical field. In this section, the background of infrared thermographic technique and image processing technique are discussed The infrared thermographic technique The infrared thermography technique (IRT) makes it possible to observe thermal information by transferring heat data which is emitted from objects into a visible

15 7 thermographic image. All objects with temperatures above absolute zero emit heat, which makes the information collectable by thermography equipment [Azmat; 2005]. Infrared thermography transfers light radiation into the IR region, therefore accurately measures the intensity of thermal radiation with different wavelengths [Barr; 1965]. Warmblooded creatures become visible against the environment with a thermal imaging camera because warm objects and cooler backgrounds offer different temperature distributions. As a consequence, the infrared thermography technique is of great use and importance for medical science and military applications [Lisowska-Lis; 2011]. In an attempt to detect the pathological condition of canines with ruptured anterior cruciate ligaments, thermographic images taken with the infrared camera by the Long Island Veterinary Specialists (LIVS) will be classified into normal and abnormal categories. All dogs in the normal category have had a physical exam and diagnostic x-rays to confirm that there are no orthopedic issues currently or in the past. All dogs in the abnormal category have had an examination, x-rays, and corresponding surgery confirming that the cruciate ligament of the lame leg is torn and the opposite leg is considered normal [LIVS; 2013]. To obtain more information for the distribution of temperature, these thermographic images are taken with three different views: anterior, lateral, posterior view. Anterior, lateral, posterior views are referring to taking images from front, side and back of the animal respectively Image processing technique In this research, the thermographic images from LIVS will be classified by using computer vision and image processing techniques. These images need to be analyzed by simplifying the raw image data into higher level information which involves a dimensional

16 8 reduction. The process of transforming the row image data to the set of features is known as feature extraction. After the features are extracted from all images, a combination of pattern classification methods selected by the user is implemented. To explore more features for detection, the Gabor filter is used. 2.3 Gabor Filter The Gabor filter, is a linear filter which has been utilized effectively for edge detection. It is characterized by frequency and orientation. Because of the attribution, it has been regarded to be suitable for texture segmentation and discrimination [Kruizinga ;1999]. In the spatial domain, a 2D Gabor filter is the result of multiplication of a 2D Gaussian function and an exponential function, which can be represented as follow: g(λ, θ, φ, σ, γ) = exp ( x(θ)2 +γ 2 y(θ) 2 )exp(i(2π x(θ) + φ)) 2σ 2 λ Details of the Gabor filter will be explained in Chapter 4. A set of Gabor filters with several frequencies and orientations may be helpful for extracting useful features from images. Therefore Gabor filters have been widely used in pattern classification applications. Many applications have used the Gabor filter effectively, such as face recognition and fingerprint recognition. According to previous research, the Gabor features are extracted from human faces, then slow feature analysis (SFL) is applied which was effectively used for face recognition [Gao et al; 2008]. Also, another application extracted the local Gabor features including eyes, nose, ears and lib with eight different angles and five different frequencies. Then they mark common points and calculate the distance between them. Finally, these distances are

17 9 compared with database. If match occurs, the images are successfully recognized [Muhammad et al; 2011]. There are some application that use Gabor filter for fingerprint recognition. The fingerprint images were separated into sets of 32*32 small subimages. They used Gabor filter-based features vectors which are directly extracted from grey-level subimages, as new feature vectors to a nearest neighbor and k-nearest neighbor (K=2,3) classifier to compare with a fingerprint database. The result shows an improvement by using the convolved Gabor filter [Lee et al; 1999]. Both face and fingerprint recognition can be improved by convolving the Gabor filter at different scales. Therefore, the Gabor filter can be utilized as a texture mask for enhancing the orientation and frequency data which is beneficial for pattern classification. Gabor features can be used directly as input to a classification or segmentation operator or they can be firstly transformed into new features which are which are then used as such an input [Grigorescu; 2002]. In this research, the Gabor filters are implemented with three different frequencies and four different orientations. After these twelve filters are applied to the thermographic images, the Gabor feature vectors are used for pattern classification. The Gabor feature vectors are obtained by extracting the standard first and second order histogram/texture features from the Gabor images themselves One goal of the research study is reached to investigate whether the Gabor filter can improve the efficiency of anterior cruciate ligament (ACL) disease detection.

18 10 9 CHAPTER 3 10 MATERIALS AND TOOLS The main purpose of the research study is to investigate whether thermographic imaging can be used effectively in canine ruptured anterior cruciate ligament (ACL) disease detection. The materials utilized in this research include thermographic images from the Long Island Veterinary Specialists and border masks created in the Computer Vision and Image Processing (CVIP) research lab at SIUE. There are five main programs involved: CVIPtools, CVIP-FEPC, Matlab, Color Normalization software, and Microsoft Excel. 3.1 The Thermographic Images In this research, a digital infrared thermal imaging (DITI) system from Meditherm Med2000 IRIS is used. It is provided by the Long Island Veterinary Specialists (LIVS). The med2000 incorporates all of the necessary criteria required for successful clinical digital infrared thermal imaging (DITI). It offers accurate measurement and has the ability to statistically analyze the thermograms at a later date which is very important in clinical work [Meditherm; 2012]. The med2000 has two parts, the IR camera and a standard PC or laptop computer, making the system very portable.with a high-resolution display, the system can measure temperatures ranging from 10 C - 55 C to an accuracy of 0.01 C. Focus adjustment covers small areas down to 75 x 75mm [Meditherm; 2012]. Thermograms produced by the med2000 are stored as TIFF RGB images with 319 columns by 238 rows, 8-bits per pixel per color band. The images used in this research are

19 11 supplied by Long Island Veterinary Specialists. A total of 18 colors are used in these images [LIVS;2012]. In this research study, as many as twenty-eight dogs with two different groups are used: 1) fourteen normal canines 2) fourteen abnormal canines. In the normal group, both sides of the canines are normal. In the abnormal group, canines have ACL issue with only one limb. A total of 168 thermographic images are separated into three groups based on these different views: 1) anterior view, 2) lateral view, 3) posterior view. Each group has 56 images. All images can also be categorized into two classes: 1) limbs with ruptured anterior cruciate ligament (ACL) disease, 2) limbs without disease. The number of images in each category is shown in Table 3.1. Examples of the thermographic images of a healthy canine (normal) and of a canine diagnosed with ACL disease (abnormal) with different views are shown in Figure 3.1, Figure 3.2 and in Figure 3.3 respectively

20 12 Table 3.1 The Number of Thermographic Images in Each Category Pathology The Number of Images Anterior Lateral Posterior Healthy ACL rupture (a) Normal thermal image (b) Abnormal thermal image Figure 3.1 Thermographic Images of a Canine with Anterior View (a) Normal thermal image (b) Abnormal thermal image Figure 3.2 Thermographic Images of a Canine with Lateral View

21 13 (a) Normal thermal image (b) Abnormal thermal image Figure 3.3 Thermographic Images of a Canine with Posterior View 3.2 Border Masks An image may be considered to contain many sub-images. To maintain high accuracy and eliminate the noise and error, extracting the region of interest (ROI) is helpful for focusing on the area under consideration. Border masks are used to extract the ROI with white being the object and black being the background. The ACL disease area is considered the region of interest which is determined by experts from the Long Island Veterinary Specialists (LIVS). Manually border masks creation is implemented in CVIPtools, with Utilities->Create->Border mask. 3.3 Software Tools In this research, the main processing can be categorized in two areas: feature extraction and pattern classification. There are six primary programs utilized for achieving the goal: CVIPtools, CVIP-FEPC, Matlab, Color Normalization software, Partek and Microsoft Excel.

22 CVIPtools v5.5d CVIPtools is a Windows-based software which was developed by the Computer Vision and Image Processing (CVIP) Laboratory in the Department of Electrical and Computer Engineering of Southern Illinois University Edwardsville (SIUE). CVIPtools is created to facilitate the development of both human and computer vision applications. This software provides an environment for the user to implement different functions and instantly get the results [CVIPtools; 2012]. CVIPtools version 5.5d is the newest version. In this research, CVIPtools is utilized for manual mask creation. In addition, it is also efficient for comparison of two different images CVIP-FEPC (Feature Extraction and Pattern Classification) CVIP-FEPC [CVIP-FEPC; 2010] allows users to perform feature extraction and pattern classification experiments with one program running. This software will automatically extract all combinations of features based on user selection. It will then perform each pattern classification combination on these features. Finally, the application produces the success rate for the different feature combinations which allows the user to easily find the optimal feature and classification combinations. In addition, sensitivity and specificity metrics are provided in the result which indicates the accuracy of our prediction of disease and absence of disease Matlab v2013a Matlab is a high-level language and interactive environment for numerical computation, visualization, and programming. Matlab provides functions through which the user can analyze data, develop algorithms, and create models and applications [Matlab; 2013].In this research, Matlab is used for implementing the code of the Gabor filter.

23 Color normalization software The color normalization algorithm normalizes the original thermographic image into four different color space based on different temperature distribution. These four spaces refer to as luminance (lum), normalized grey (normgrey), normalized RGB (normrgb), and normalized RGB luminance (normrgb-lum) [Umbaugh, Solt; Jan 2008]. These temperature data is provided by the Long Island Veterinary Specialists Microsoft Excel Microsoft Excel is a spreadsheet application for data collection and calculation. In this research, Excel is utilized for result collection and data sorting.

24 16 11 CHAPTER 4 12 METHODS AND EXPERIMENTS In this study, there are a total of four steps for ACL disease diagnosis, including: Border Mask creation Color Normalization Feature Extraction Pattern Classification 4.1 Border Masks To eliminate unnecessary image data and focus on the disease area, the border mask operation was used to create the border mask. The border mask image is a binary image. Figure 4.1 shows a ROI with their corresponding original image as provided by LIVS. Figure 4.2 shows images of anterior, lateral and posterior views with their created manually corresponding masks, and the results after application of the mask to extract the ROI. The masked image is the image after And operation between original image and mask.

25 17 (a) region of interest(roi) area (b) original image Figure 4.1 Region of Interest to Detect ACL Disease and Original Image (a) anterior view original image (b) anterior view mask image (c)anterior view masked image

26 18 (d) lateral view original image (e) lateral view mask image (f) lateral view masked image (g) posterior view original image (h) posterior view mask image

27 19 (i) posterior view original image Figure 4.2 Manual Masks Created from Anterior, Lateral and Posterior View. 4.2 Color Normalization The distribution of color data in the image depends on illumination, the thermographic data, and the settings of the image acquisition system. In this study, each of the images uses the same color palette consisting of 18 colors. Every color represents a specific temperature value. Since the camera may be recalibrated between each image capture, within a set of images one color may map to several different temperatures. For this study each class, normal and abnormal, was color normalized separately (note that combined color normalization results are in Appendix I). In order to remap the temperature to a common (normalized) temperature scale, four different color normalization methods are used including: Luminance, Norm-Gray, Norm-RGB, Norm-RGB-Lum [Umbaugh, Solt, 2008]. Luminance: Each pixel in the normalized image generates the gray value level follow by the formula: GrayLevel 0.3*Red 0.6* Green 0.1* Blue [C.A. Loughin et al, Oct 2007]

28 20 Which Red, Green, Blue is the value with three bands of RGB color model. Norm-Gray: Each of the images with same color palette consists 18 colors. Temperatures base on this18 colors from minimum to maximum are remapped from 0 to 255. Norm-RGB: Norm-RGB method is similar with Norm-Gray while instead of temperatures being remapped to gray level from 0 to 255. They are mapped to a continuous version of the original color palette. Norm-RGB-lum: Norm-RGB-lum method produces the gray-level image by perform Luminance normalization after Norm-RGB normalization [Umbaugh, Solt, 2008]. Figures 4.3a through 4.3e show an original image and four corresponding color normalization images.

29 21 Figure 5a Original Image Figure 5b Luminance Image Figure 5c NormRGB Image Figure 5d NormGrey Image Figure 5e NormRGB-lum Image Figure 4.3 An Original Image and Four Corresponding Color Normalization Images

30 Feature Extraction Feature extraction is important to simplify the raw image data into higher level, meaningful information. After color normalization of the original image, the feature extraction operation is performed. To eliminate redundancy in a huge amount of data, such as an image the input data will be transformed into a reduced representation as a feature vector. That is to transform the input image into a set of features. After that, feature analysis involves examining the features extracted from the images and determining how they can be used to solve the imaging problem under consideration [Umbaugh;2011] Feature extraction starts with feature selection. The selected features will be the major factor that determines the complexity and success of the analysis and pattern classification process [Umbaugh;2011]. In this research, histogram features, spectral features, texture features are used. Either original images or color normalized images can be used to extract these features by using CVIP-FEPC. While using the original image, four types of histogram features including histogram standard deviation, skew, energy, and entropy, five types of texture features include texture energy, inertia, correlation, inverse difference, and entropy with a texture distance(pixel) of 6, spectral feature with three rings and three sectors measurement are utilized. The histogram mean feature is the additional feature while using color normalized images for feature extraction because the histogram mean feature reflect the average temperature data. Gabor filters are bandpass filters which are used for feature extraction and feature analysis. A set of Gabor filters with different frequencies and orientations are helpful for feature extraction from images. A comparison between original feature vectors and the Gabor feature vectors has been discussed in this study.

31 Histogram features Histogram provides account of pixels versus the gray-level distribution for the image or sub-image. The histogram of an image is the frequency gray-level distribution with number of pixels for each value [Umbaugh;2011]. Four types of histogram features for the original images or five types of features for the color normalized images are selected with three bands of red, green, blue (RGB) in this research include histogram mean, standard deviation, skew, energy, and entropy. The mean is the average value which represents the brightness of the image. Mean I( r, c) r c M where M is the number of pixel in image or subimage. mean value. A bright image will have a high mean value whereas dark image will have a low The standard deviation which is known as the square root of the variance describes the contrast of image. L 1 g 0 2 ( g g) P( g) and g P( g) N( g) M The skew measures the asymmetry about the mean in the gray-level distribution. L Skewness ( g g) P( g) 3 g g 0 and N( g) P( g) M

32 24 The skew will be positive if the tail of the histogram spreads to the right, and negative if the tail of the histogram spreads to the left. The energy measure shows how the gray levels are distributed. L 1 Energy P( g) and g 0 2 P( g) N( g) M The entropy is a measure for counting how many bits need to code the image data. L 1 Entropy P( g)log P( g and g 0 2 ) P( g) N( g) M A complex image has higher entropy than a simple image. Entropy measure tends to vary inversely with the energy measure [Umbaugh; 2011] Spectral features Spectral features, or frequency/sequency-domain based features are special features and the primary metric is power. The power spectrum is calculated by the magnitude of the spectral components squared: Power = T (u, v) 2 The generic T(u,v) could be used in any transforms which typically use the Fourier transform. The spectral features are measured by calculating the power in various spectral regions, and these regions could be rings, sectors, or boxes. The sector measurement could find power of specific orientation regardless the frequency, while the ring measurement

33 25 could find the power of specific orientation whatever the orientation. In this study, three rings and three sectors are used Texture features Texture is a visual pattern attribute. It is a property of areas, and consisting of subpatterns which are related to the pixel distribution in a region [André; 2010].Texture features reflect properties including: smoothness, coarseness, roughness and regular patterns. One method for measuring texture feature is to use the second-order histogram of the gray levels. Texture features involved with second order histograms are used for purposes of texture classification or segmentation. The second order histogram methods are also referred to as gray-level co-occurrence matrix or gray-level dependency matrix methods which use a second order histogram could count based on pairs of pixels and corresponding gray levels. These are two parameters important for these features: distance and angle. Distance is the distance between the pairs of pixels which are utilized by the second order statistics. The angle is the angle between the pairs of pixels. Usually, there are four different angles used: vertical, horizontal, left diagonal and right diagonal directions [Umbaugh; 2011]. Five types of texture features are used in this study including energy, inertia, correlation, inverse difference and entropy. The energy measures the smoothness by counting the distribution among the gray levels. The inertia shows the contrast while the correlation provides value of the similarities between pixels. The inverse difference measures the

34 26 homogeneity of the texture and entropy which inverse the energy being able to measure the information content [Umbaugh; 2011] Gabor filter The Gabor filter is a linear filter which is excellent for edge detection and has both frequency-selective and orientation-selective properties. Therefore the Gabor filter is particularly appropriate for texture discrimination, texture analysis and feature classification. Gabor feature vectors could be used as a classification or segmentation operator for new feature vector performance. In this study, the features extracted from Gabor filter were used for the new feature vectors that were used as input for the pattern classification. The Gabor filter can be viewed as a sinusoidal wave of frequency and orientation, convolved by a Gaussian envelope. A 2-D Gabor filter acts as a local band-pass filter with specific frequency and orientation. Mathematically, in spatial domain, a 2D Gabor filter is the result of multiplication of a 2D Gaussian function and an exponential function, which can be represented as follow: g(λ, θ, φ, σ, γ) = exp ( x(θ)2 +γ 2 y(θ) 2 )exp(i(2π x(θ) + φ)) 2σ 2 λ According Euler s formula e iθ = cos θ + i.sin θ, the complex form is expressed as a real number plus an imaginary number.therefore, the complex Gabor filter could be separated with real and imaginary parts: Real: g(λ, θ, φ, σ, γ) = exp ( x(θ)2 +γ 2 y(θ) 2 ) cos(2π x(θ) + φ) 2σ 2 λ Imaginary: g(λ, θ, φ, σ, γ) = exp ( x(θ)2 +γ 2 y(θ) 2 ) sin(2π x(θ) + φ) 2σ 2 λ Where x(θ)= xcos θ + y sin θ, y(θ)= -xsin θ+ycos θ

35 27 In this study, only the real part was considered. For this equation, λ is wavelength of the sinusoidal factor. Therefore, 1 λ is the frequency of factor cos(2π x(θ) λ + φ). Figure 4.4 shows Gabor filter with four different wavelengths (λ) in spatial domain and Figure 4.5 shows Gabor filter with four different wavelengths (λ) in frequency domain. λ = 2 λ = 5 λ = 10 λ = 20 Figure 4.4 Four Different Wavelengths of Gabor Filter in Spatial Domain λ = 2 λ = 5 λ = 10 λ = 20 Figure 4.5 Four Different Wavelengths of Gabor Filter in Frequency Domain θ is the orientation of the normal to the parallel stripes of a Gabor function. Figure 4.6 shows Gabor filter with three different orientations (θ) in spatial domain. 4.7 shows Gabor filter with three different orientations (θ) in frequency domain.

36 28 θ = 0 θ = π 4 θ = π 2 Figure 4.6 Three Different Orientation of Gabor Filter in Spatial Domain Figure 4.7 Three Different Orientation of Gabor Filter in Frequency Domain φ is the offset of phase. σ represents the standard deviation of the Gaussian factor of the Gabor equation. γ is aspect ratio which reflect the ellipticity of Gabor equation which default value is 0.5. Figure 4.8 shows Gabor filter with three different aspect ratios (γ).

37 29 γ = 0.3 γ = 0.5 γ = 1 Figure 4.8 Three Different Aspect Ratios of Gabor Filter b is the half-response spatial frequency bandwidth (in octaves) of a Gabor filter is related to the ratio σ / λ [Kruizinga ;1999].. σ λ b=log π+ ln σ λ π ln 2 2 σ λ = 1 π ln 2. 2b b 1 Two two-dimensional Gabor filter with same standard deviation but different ratio provide different frequency and bandwidths. In Gabor function, the standard deviation of the Gaussian factor σ could be specified through bandwidth rather than specified directly. The default bandwidth is 1. Hence, σ = 0.56 λ. To investigate how Gabor filters perform for pattern classification via texture discrimination in thermographic images, code was implemented with Matlab. In Matlab, the Gabor filter was performed following by the real part formula which mentioned in this section. Four different equidistant orientations [θ = k ( π ), k = 0,1,2,3] and three different 4

38 30 wavelength scales base on the texture of thermographic image are utilized. The rest of factors are used default value. Therefore, there are totally twelve different Gabor filters. The next step is to convert the original thermographic images from color scale to grey scale to eliminate the error caused by the color shifting. The next step is convolving the grey scale images with different Gabor filters. The next step is to extract texture features from the Gabor filtered image to be used for pattern classification. The Gabor filter function is related to the negative exponential form. And the kernel pixel values in the Gabor filter are very small values between -1 and 1.Thus, after being convolved with the grey image, the output image will have small values and may look like a black image. Therefore, the output image needs to be remapped back to [0,255]. Figure 4.9 shows grey image convolved with Gabor filter without remapping. Figure 4.10 shows grey image convolved with Gabor filter with remapping in spatial domain. Figure 4.11 shows grey image convolved with Gabor filter with remapping in frequency domain. Figure 4.9 Grey Image Convolved With Gabor Filter without Remapping

39 31 Figure 4.10 Grey Image Convolved Gabor Filter Remapping in Spatial Domain Figure 4.11 Grey Image Convolved Gabor Filter Remapping In Frequency Domain The new output images are input to CVIP-FEPC to extract features followed by pattern classification in CVIP-FEPC.

40 Pattern Classification Pattern classification uses the features to classify image objects which typically is the final step in the process. After different features have been extracted from raw images, feature analysis processing is necessary. Different features are selected by the variable selection methods which constitute feature vectors. The set of feature vectors need to be analyzed and prepared for developing the classification algorithm. The user needs to find the optimal combination of features which provide the best results. In the CVIP-FEPC, there are three steps for pattern classification including: data normalization, application of distance and/or similarity metrics, and the classification algorithm itself Data normalizaiton Data normalization is adopted to avoid biasing distance and similarity measures due to the varying range on different vector components [Umbaugh; 2011]. There are several data normalization methods in CVIP-FEPC including: range-normalize, unit vector normalization, standard normal density normalization, min-max normalization, softmax scaling method. Standard normal density normalization and softmax scaling methods are used in this study. These two data normalization method provide better result comparing with others. Standard normal density is a kind of statistical-based method which normalize the feature vector by subtracting the mean value and dividing by the standard deviation [Umbaugh; 2010] for each feature. The process is as follows: F j = {F 1, F 2,, F k }, F j is feature space which contain different k feature vectors. Every feature vector involves n features.

41 33 f 1j f F j = 2j [ f nj ] for j = 1, 2,, k Means: m i = 1 k f k j=1 ij for i = 1, 2,, n Standard deviation: σ i = 1 k (f k j=1 ij m i ) 2 for i = 1, 2,, n Every feature component normalizes by subtracting the mean and then divides by the standard deviation. f ijsnd = f ij m i σ i The new feature distribution on each vector after normalization is called standard normal density (SND). Softmax scaling is a nonlinear method which is desired if the data distribution is skewed, that is not evenly distributed about the mean. Softmax will normalize the spread of data by moving the mean and rescaling the data range from 0 to 1 [Umbaugh; 2010]. There are two steps for softmax scaling method: STEP1 y = f ij m i rσ i where m i is the mean value, f ij is the feature vectors, σ i is the standard deviation, r is the factor which is defined by the user. Step 1 is similar with SND while r is the new factor which is defined by the user. The factor determines the range values of the feature f ij. STEP2 f ijsmc = 1 1+e y for all i, j

42 34 f ijsmc is the feature after normalized. If y is small enough, this process is almost linear and the feature data is rescaled exponentially Distance and similarity measures After feature extraction and normalization, comparison of two feature vectors is necessary to perform the pattern classification. The idea is basically to find the difference or similarity between two feature vectors. The difference can be measured by the distance measure in the feature space. The smaller distance between two feature vectors, the greater similarity and the less difference. There are several distance measure and similarity measure method In CVIP-FEPC. In this study, the Euclidean distance measure has been utilized which frequently used for distance comparison in optimization problems. Euclidean distance need to be measured by the square root of the sum of the squared of the differences between vector components [CVIPtools;2012].The process is showed as follow: A and B are different feature vectors which both contain n feature components. a 1 a 2 b 1 b A = [ ] B = [ 2 ] a n b n The Euclidean distance is: D E (A, B) = n i=1 (a i b i ) 2 = (a 1 b 1 ) 2 + (a 2 b 2 ) (a n b n ) 2

43 Classification algorithm One method to develop a classification algorithm, requires that the feature data are separated into a training set and a test set. The training set consist of a set of training examples which is utilized for classification algorithm development and test set is used for testing the classification algorithm. To provide an unbiased result, both training set and test set should represent all types of classes in the application domain. Theoretically, maximizing size of the training set will provide the best while maximizing the size of the test set will provide maximum confidence that the test results will be a valid predictor for future results[umbaugh;2011]. Leave-one-out cross validation method is used in this study which is a special case of training and test sets algorithm. With leave-one-out cross validation method, only one sample is left for the test set and rest of samples are marked as training set. This is done for each sample in the entire set. In CVIP-FEPC, there are three classification algorithms available with leave-one-out method including: Nearest Neighbor, K-Nearest Neighbor, and Nearest Centroid. In this study, nearest neighbor and K-nearest neighbor methods were applied. For this research there are only fourteen images for the abnormal set, and the Nearest Centroid method is only appropriate with large data sets. Nearest neighbor is the simplest classification algorithm which classifies the unknown as the closet sample in the training set by using distance measure or similarity measure. Therefore, it is not robust enough. K-nearest neighbor method could consider top k nearest neighbors to the query. K=5 was used in this research. For the previous experiments, K=5 could perform better result comparing with 3 and 7. [Umbaugh;2011]

44 Success measure and evaluation The success rate measures classification accuracy. Sensitivity and Specificity are two statistical measures of success evaluation, often used in medical studies, which have the following definitions: True Positive (TP): sick person classified correctly. False Positive (FP): healthy person classified as sick. True Negative (TN): healthy person classified correctly. False Negative (FN): sick person classified as healthy. The relationship with True Positive (TP), False Positive (FP), True Negative (TN), False Negative (FN) could be showed as: Condition positive Condition negative Test positive True Positive (TP) False Positive (FP) Test negative False Negative (FN) True Negative (TN) Sensitivity and Specificity are defined as follow: Sensitivity = number of True Positives number of True positives+ number of False Positives Specificity = number of True Negatives number of True Negatives+ number of False Positives Sensitivity indicates how accurate of identifying a disease as prediction and Specificity indicates how accurate prediction of absence of the disease is [Umbaugh;2011].

45 37 13 CHAPTER 5 14 RESULTS AND ANALYSIS The goal of the research is to determine the accurate of thermographic image analysis in detection of canine ACL disease. The result of the research is separated into two sections. In the first section, the result is obtained by using the features extracted from CVIP-FEPC which involve histogram features, texture features and spectral feature for pattern classification. In the second section, the result is utilizing the new feature vector from images which were convolved by the Gabor filter for pattern classification. 5.1 ACL Disease Detection by Using CVIP-FEPC Based on different views of the canines, the 168 images from 28 canines have been divided into three groups: 1) anterior view, 2) lateral view, 3) posterior view. Every group includes 56 images with 14 abnormal images and 42 normal images. The images are categorized into two classes: Normal and Abnormal. All three groups of images are processed with performed color normalization operations. Different features extraction and pattern classification methods are performed in CVIP-FPEC Anterior view In this section, the result of the anterior view is discussed and analyzed. The thermographic images are taken from in front of the canine for this view. As mentioned in the Chapter 4, there are totally five different experiments including one experiment using original images and four experiments using the color-normalized images. The original images use ten different features with four histogram features, five texture features and

46 38 spectral features. To perform all the combinations of the feature vectors, totally =1023 combined feature sets have been formed. Color normalized images which contain the additional feature of the histogram mean, for a total of eleven features with five histogram features, five texture features and spectral feature. Hence there are total of =2047 combined feature sets for the color normalized images. These feature vectors extracted with CVIP-FPEC are data normalized with standard normal density normalization method and softmax scaling normalization method. In this study, the Euclidean distance measure is used as the distance measure. Nearest neighbor and K-nearest neighbor where K=5 are utilized as classification methods in these experiments and leave one out is used as the testing method. The best result for original and four different color normalization experiments of anterior view are shown in Table 5.1.

47 39 Table 5.1 Results for Original and Color Normalized Images of Anterior View Group Color Normalization Method Camera View Classification Success Rate Original Anterior 78.57% Lum Anterior 78.57% NormGrey Anterior 83.93%* NormRGB Anterior 78.57% NormRGB-Lum Anterior 82.14% The classification success rate is the numbers of objects correctly matching the predicted categories for normal or abnormal class. As the result shown in Table 5.1, the classification success rate of images which involved the color normalization methods always obtain the same or better result than the original images. Among these methods, the best classification success rate is from NormGrey images with 83.93%. More details of features combinations and classification method for anterior group are shown in Table 5.2.