Comparison of observer reliability of threedimensional cephalometric landmark identification on subject images from Galileos and i-cat CBCT.

Transcription

1 University of Iowa Iowa Research Online Theses and Dissertations 2011 Comparison of observer reliability of threedimensional cephalometric landmark identification on subject images from Galileos and i-cat CBCT. Rujuta Amol Katkar University of Iowa Copyright 2011 Rujuta Amol Katkar This thesis is available at Iowa Research Online: Recommended Citation Katkar, Rujuta Amol. "Comparison of observer reliability of three-dimensional cephalometric landmark identification on subject images from Galileos and i-cat CBCT.." MS (Master of Science) thesis, University of Iowa, Follow this and additional works at: Part of the Other Dentistry Commons

2 COMPARISON OF OBSERVER RELIABILITY OF THREE-DIMENSIONAL CEPHALOMETRIC LANDMARK IDENTIFICATION ON SUBJECT IMAGES FROM GALILEOS AND I-CAT CBCT by Rujuta Amol Katkar A thesis submitted in partial fulfillment of the requirements for the Master of Science degree in Stomatology in the Graduate College of The University of Iowa December 2011 Thesis Supervisor: Professor Axel Ruprecht

3 Copyright by RUJUTA AMOL KATKAR 2011 All Rights Reserved

4 Graduate College The University of Iowa Iowa City, Iowa CERTIFICATE OF APPROVAL MASTER'S THESIS This is to certify that the Master's thesis of Rujuta Amol Katkar has been approved by the Examining Committee for the thesis requirement for the Master of Science degree in Stomatology at the December 2011 graduation. Thesis Committee: Axel Ruprecht, Thesis Supervisor Lina Moreno Ubribe Veeratrishul Allareddy Michael Finkelstein Deborah Dawson

5 To my wonderful daughter, Alisha To my loving husband, Dr. Amol Katkar for his constant support and guidance in my career To my dear parents, for their love, support and encouragement throughout my life ii

6 ACKNOWLEDGMENTS My heartfelt gratitude goes to Dr. Axel Ruprecht who served as my thesis supervisor and also guided me throughout my academic program. He and the other members of my thesis committee, Dr. Lina Moreno, Dr. Veeratrishul Allareddy, Dr. Michael Finkelstein, and Dr. Deborah Dawson guided me through the dissertation process through their valuable advice. I thank them all. I extend my appreciation to Ms. Colleen Kummet, Division of Biostatistics and Research Design, for her valuable input in generating the statistical analysis for this research. I would like to thank my past colleagues Dr. Rumpa Ganguly and Dr. Nidhi Handoo and my coresidents Gayle, Ali, Sindhura, Krishna, Chelsia and Mahdieh for their encouragement, love and support. My sincere thanks to Dr. John Hellstein, Dr. Steve Vincent and Dr. Sherry Timmons for their valuable guidance in my residency program. I am deeply indebted to my parents for their unending love and support throughout my academic career. I cannot thank my husband, Amol, enough for his love, co-operation, understanding and support during my years of training. iii

7 TABLE OF CONTENTS LIST OF TABLES...v LIST OF FIGURES. vii CHAPTER I INTRODUCTION...1 Digital imaging...1 Computed tomography...2 Cone beam computed tomography...3 Aim...7 Hypotheses...7 CHAPTER II MATERIALS AND METHODS...9 Landmark definitions...11 Overview and methods...17 CHAPTER III RESULTS...20 CHAPTER IV DISCUSSION D Cephalometry...59 CHAPTER V CONCLUSION...66 BIBLIOGRAPHY...68 iv

8 LIST OF TABLES Table 1. Summary of Intra-Observer Reliability for Observer 1: Parametric Intraclass Correlations (ICCs) Based Upon Duplicate Measurements of 10 Subjects...30 Table 2. Summary of Intra-Observer Reliability for Observer 2: Parametric Intraclass Correlations (ICCs) Based Upon Duplicate Measurements of 10 Subjects...31 Table 3. Summary of Rothery Nonparametric Intra-observer Reliability for Observer Table 4. Summary of Rothery Nonparametric Intra-observer Reliability for Observer Table 5. Observer 1 X Coordinate Comparison of Intraclass Correlations for 2 machines (n=10 per machine)...34 Table 6. Observer 1 Y Coordinate Comparison of Intraclass Correlations for 2 machines (n=10 per machine)...35 Table 7. Observer 1 Z Coordinate Comparison of Intraclass Correlations for 2 machines (n=10 per machine)...36 Table 8. Observer 2 X Coordinate Comparison of Intraclass Correlations for 2 machines (n=10 per machine)...37 Table 9. Observer 2 Y Coordinate Comparison of ICCs for 2 machines (n=10 per machine)...38 Table 10. Observer 2 Z Coordinate Comparison of ICCs for 2 machines (n=10 per machine)...39 Table 11. Summary of Euclidean Distances between 1 st and 2 nd Measures of Observer Table 12. Summary of Euclidean Distances between 1 st and 2 nd Measures of Observer Table 13. Results of Linear Mixed Model Predicting Euclidean Distance between Dual Landmarks with Fixed Machine and Random Subject Effect. (Observer 1)...42 Table 14. Results of Linear Mixed Model Predicting Euclidean Distance between Dual Landmarks with Fixed Machine and Random Subject Effect. (Observer 2)...43 Table 15. Summary of Inter-Observer Reliability for Observer 1 and Observer 2 (First Measures)...44 Table 16. Summary of Inter-Observer Reliability for Observer 1 and Observer 2 (Second Measures)...45 v

9 Table 17. X Coordinate Comparison of Intraclass Correlations measuring Inter- Observer Reliability (First Measures)...46 Table 18. Y Coordinate Comparison of Intraclass Correlations measuring Inter- Observer Reliability (First Measures)...47 Table 19. Z Coordinate Comparison of Intraclass Correlations measuring Inter- Observer Reliability (First Measures)...48 Table 20. X Coordinate Comparison of Intraclass Correlations measuring Inter- Observer Reliability (Second Measures)...49 Table 21. Y Coordinate Comparison of Intraclass Correlations measuring Inter- Observer Reliability (Second Measures)...50 Table 22. Z Coordinate Comparison of Intraclass Correlations measuring Inter- Observer Reliability (Second Measures)...51 Table 23. Summary of Euclidean Distances between First Measures of Observers 1and Table 24. Summary of Euclidean Distances between Second Measures of Observers 1and Table 25. Results of Linear Mixed Model Predicting Euclidean Distance between Dual Landmarks with Fixed Machine and Random Subject Effect. (First Measures)...54 Table 26. Results of Linear Mixed Model Predicting Euclidean Distance between Dual Landmarks with Fixed Machine and Random Subject Effect. (Second Measures)...55 Table 27. Results of Linear Mixed Model Predicting Euclidean Distance between Dual Landmarks with Fixed Machine and Rater, and Random Subject within Machine Effect...56 Table 28. Summary of All Results for Galileos and i-cat CBCT Study*...57 vi

10 LIST OF FIGURES Figure 1. Laptop computer used for landmark localization...13 Figure 2. Dolphin imaging screen...14 Figure 3. Location of Orbitale landmark on the MPR slices and 3D surface rendered image...15 Figure 4. Location of the Upper molar landmark on MPR slices and 3D surface rendered image...16 vii

11 1 CHAPTER I INTRODUCTION Cephalometric analysis is an integral part of orthodontic diagnosis and treatment planning. The traditional method involves making a lateral cephalometric radiograph and defining skeletal landmarks on the acquired two-dimensional image. The correct identification of craniofacial landmarks on a lateral cephalometric radiograph lays the foundation for cephalometric analysis; however, the process is prone to significant variation. This is partly due to projection errors, variation in observer interpretation and radiographic technique. 1-5 As certain anatomical structures are more identifiable than others, a systematic pattern of error prevails in landmark identification, with observers exhibiting higher precision with respect to some points over others. 6-8 In particular, sella, nasion and pogonion tend to be less subject to error, whereas landmarks like A point, porion and condylion tend to exhibit greater variation in identification. 2,3,9 Furthermore, dental landmarks tend to exhibit greater variability than skeletal anatomic points. 10 Due to the variability in the location of certain landmarks, cephalometric references based on landmarks that are not reliably identified are not recommended. 3,5,9 Inasmuch as the variability of porion and orbitale (Frankfort Horizontal plane) is significant, a more identifiable reference, sella to nasion is recommended for cephalometric analysis. Although some studies have shown that the precision of landmark identification does not alter the results of the cephalometric diagnosis greatly, 2,11 the assessment of growth and treatment outcomes does require proper identification and utilization of landmarks that are more easily identified. 6,12 Digital imaging Traditionally, cephalometric analysis has been done on analog lateral cephalometric radiographs on which the landmarks were identified and traced by hand.

12 2 This process is often time consuming and prone to considerable examiner error. 6,13 Today, lateral cephalometric images can be acquired digitally or digitized through the use of a flatbed scanner and transparency adaptor. Digital radiography uses an x-ray source that is identical to that found in conventional analog radiography, but replaces the film with an imaging screen (storage phosphor) (computed radiography) or an electronic sensor such as a charge-coupled device (CCD) (digital radiography). These receptors usually require less radiation and are processed more quickly than conventional radiographic films. The digital radiographs acquired by such electronic sensors are instantly viewable on a computer. Furthermore, use of digital radiographs eliminates errors associated with scanning conventional radiographs. The quality of the digital image that is presented on a computer monitor is directly dependent on the number of dots per inch (dpi). This number can be changed by changing the scanner settings. As long as standard scanner settings are used, adequate image detail can be obtained on the computer monitor for precise landmark identification. 14,15 The digital lateral cephalometric radiograph offers reliability similar to that of an analog cephalometric radiograph with respect to landmark identification in a significantly more efficient manner. 6,13,16,17 Aside from the additional benefits of rapid accessibility, transferability and postprocessing of the images, digital lateral cephalometric radiography is prone to the same projection errors and offers comparable diagnostic value to that of analog radiographs. 11,18-20 Computed tomography Since the invention of the first computed tomography (CT) scanner by Godfrey Hounsfield in Britain in 1972, CT scanners have improved in efficiency and sophistication, experiencing widespread use in clinical applications in the medical, dental and other scientific fields. Modern CT imaging uses helical or spiral CT machines, which use a narrow, fan-shaped x-ray beam, generated by a high output, rotating anode. The x-

13 3 ray source is housed in a gantry and rotates around the patient as the patient moves through the gantry on a table. The images are recorded in multiple axial slices that are stacked to produce the final data volume. The data are then reconstructed, using appropriate algorithms, into a three-dimensional dataset, from which any desired perspective or multiplanar slice can be created by appropriate software CT images offer high visual contrast and acuity of soft and hard tissues in multiple planes. Several studies have reported on the dimensional accuracy of various CT systems for mandibular height and width, as well as for size and location of the mandibular canal. CT images provide high accuracy of measurements with no significant difference between the measurement of actual landmarks and that of CT images Statistical analysis has shown conventional CT 3D cephalometry of dry skulls to be highly reliable and accurate. 25,30-32 Despite the contribution of CT in the medical field, adoption of the CT machines in dentistry has been hampered by the relatively large radiation doses, high cost, and space requirements associated with these machines. Cone beam computed tomography To address the concerns of conventional CT in dentistry, cone beam computed tomography (CBCT) NewTom 9000 was introduced in 1997 in Italy. 33 CBCT was initially developed for angiography, but more recent medical applications have included radiotherapy guidance and mammography. 34 In the past decade CBCT technology has evolved to allow 3D visualization of the oral and maxillofacial complex at a much smaller radiation dose than that of conventional CT. 35 CBCT uses a cone-shaped x-ray beam generated from a low output anode source and an array of either solid-state flat panel or amorphous silicon detectors. Like the conventional CT, the x-ray source rotates around the head; however due to the cone shape of the x-ray beam every degree of rotation captures a larger area of the skull. A

14 4 single rotation of between 180 and 360 degrees is all that is required for image acquisition. During the rotation, many exposures are made at fixed intervals, providing single projection images known as basis, frame or raw images 34. The complete series of basis images is referred to as the projection data. Most CBCT machines use a pulsed source of radiation, thus the exposure time is considerably shorter than the acquisition time. The radiation exposure may be decreased to 1/15 th of the dose delivered by conventional CT and comparable to that of complete mouth periapical exposure However, there are many factors affecting the radiation exposure, including the type of machine, field of view, and resolution at which the patient is scanned. CBCT produces images with submillimeter isotropic voxel resolution ranging from as high as mm to as low as 0.4 mm. Once the DICOM dataset is transferred to compliant software, it can be postprocessed to obtain multiplanar reconstructed images (axial, coronal and sagittal) with a level of spatial resolution accurate enough for measurement in maxillofacial applications. 34 Orthogonal, panoramic and lateral cephalometric views can also be generated from the same image dataset. CBCT allows 3D visualization of the oral and maxillofacial complex. This imaging modality eliminates the shortcomings of 2D imaging, produces a smaller radiation dose than that of conventional CT and enables clinicians to make more accurate treatment planning decisions, which should lead to more successful surgical outcomes. 40,41 The information obtained from CBCT can be used for evaluation of osseous or dental pathoses, the temporomandibular joint complexes, anatomic variations and trauma, as well as for endodontic and orthodontic treatment planning. CBCT is particularly helpful in presurgical planning for dental implant placement by localizing the anatomical structures to be avoided during surgery. It helps to measure the quantity and the quality of the bone available for the placement of implants. 35 CBCT provides submillimeter pixel resolution of projection images leading to high spatial resolution of the image. 37 CBCT is primarily used for investigating bone. Although CBCT is able to

15 5 depict the associated soft tissue in the region imaged, because of high amount of scatter radiation, which contributes to noise, it is not able to distinguish among different types of soft tissues. CBCT was developed as an alternative to conventional CT to shorten the time of image acquisition of the entire FOV (Field of View) with a comparatively less expensive radiation detector. The main disadvantage, especially with larger FOVs, is a limitation in image quality related to noise and contrast resolution because of detection of large amounts of scattered radiation. 34 Several studies have proved the accuracy of linear measurement on CBCT images obtained on dry skulls Some cadaver studies have also shown the linear measurements to be accurate on CBCT images. 48,33 Inherent errors and limitations exist in CBCT. These include background scan noise, image artifacts and the field of view. Background noise can be related to the cone beam projection geometry which results in irradiation of a large volume of tissue, resulting in large amount of scattered radiation which does not reflect the actual attenuation of an object along the path of an x-ray beam. Artifacts in CBCT data can result from beam hardening and attenuation from objects such as metallic restorations and even dense cortical bone. A large field of view leads to increased radiation exposure and decreased spatial resolution whereas a limited field of view leads to partial volume artifacts and may not cover the entire region of interest. Generally, the smaller the FOV, the less the scatter, the better the image resolution and lower the radiation exposure. 21,49,50 CBCT use in orthodontics has been mainly limited to adjunctive procedures including assessment of severe facial asymmetries, identification of impacted or ectopic teeth and presurgical evaluations. Recently, there has been a surge of interest in the use of CBCT as a substitute for conventional panoramic and cephalometric images for orthodontic treatment planning. 21,49,51-54

16 6 To this end, several studies have established the precision and accuracy of linear measurements of different CBCT systems using the reconstructed 2D tomographic slices and lateral cephalometric images. The diagnostic reliability is found to be comparable to or even better than that of conventional lateral cephalographs. 42,55-62 However, the development of 3D landmark-based cephalometric analysis requires definition of 3D landmarks on complex curving structures or redefining landmarks that are the result of superimposition of points in different planes, neither of which is a trivial problem. There is a lack of literature about suitable operational definitions for the landmarks in the 3 planes of space (coronal, sagittal, and axial). Practical considerations like identification errors, coupled with the need for biologic relevance and a balanced representation of components of the craniofacial form, limit the number and nature of landmarks available for analysis. For these reasons, the development of 3D landmarkbased cephalometric analysis demands suitable operational definitions of the landmark location in each of the 3 planes of space, and reproducibility of landmark identification necessary to take full advantage of the 3D diagnostic power offered by CBCT imaging. In a study published in 2005, the precision of identification of a new skeletal landmark ELSA was assessed in ten adolescent patients using a NewTom QR-DVT The results demonstrated a high level of repeatability for locating this 3D skeletal landmark (kappa = 0.998). 63 Pinsky and coworkers demonstrated high reliability, Intraclass correlation (ICC) = 0.96, among five examiners when measuring linear distances on images of dry human mandibles scanned on an i-cat scanner. 64 The overall interobserver and intra-observer reliability in identifying certain landmarks in vivo with 3D CBCT imaging is found to be improved when compared with digital 2D lateral cephalograms. 65 In one study, human skulls were used to compare the accuracy of linear measurements using the NewTom 3G CBCT in ideal and rotated positions. The resulting datasets were used to create 3D surface-rendered images, 2D tomographic slices, and 2D lateral and postero-anterior projections. The 3D-rendered models were found to be the

17 7 most accurate and convenient to use and were least affected by the patient s head position in the scanner. 57 The overall intra- and interobserver reliability was found to be excellent in a study evaluating the observer reliability in 3D landmark identification using CBCT. The ICC was > 0.9 for 86% for intra-observer assessments and 66% for interobserver assessments in this study. It was concluded that 3 dimensional landmark identification using CBCT can offer consistent and reproducible data if a protocol for operator training and calibration is followed, particularly for landmarks not easily specified in all 3 planes of space. 58 In the review of current literature, there was no study comparing the observer reliability for identification of 3D cephalometric landmarks on CBCT images acquired from two different machines. Aim The aim of this study is to assess the observer reliability for anatomical landmark identification on subject images from Galileos and i-cat CBCT. Hypotheses 1. There are no differences in the anatomical landmark coordinates identified by the two investigators using the images from Galileos CBCT scans with Dolphin 11 software for 3D cephalometric analysis. 2. There are no differences between first and second landmark coordinates identified by the single investigator for 3D cephalometric analysis using the images from Galileos CBCT scans with Dolphin 11 software. 3. There are no differences in the anatomical landmark coordinates identified by the two investigators using the images from i-cat CBCT scans with Dolphin 11 software for 3D cephalometric analysis.

18 8 4. There are no differences between first and second landmark coordinates identified by a single investigator for 3D cephalometric analysis using the images from i- CAT CBCT scans with Dolphin 11 software. 5. There is no difference in the observer agreement between the landmark coordinate identifications from i-cat and Galileos CBCT scans with Dolphin 11 software for 3D cephalometric analysis.

19 9 CHAPTER II MATERIALS AND METHODS CBCT images of 20 subjects in the age group of years were selected from the records- 10 each from Galileos (Sirona Dental Systems Inc., Bensheim, Germany) and Next Generation i-cat (Imaging Sciences International, Hatfield, PA) datasets. Images from both the machines had been acquired and reconstructed with isotropic voxels of size 0.3mm 0.3mm 0.3mm. The field of view (FOV) was 15cm (H) 15cm (D) for Galileos and 17cm (H) 23cm (D) for i-cat. The images were selected on the basis of the following inclusion and exclusion criteria: Inclusion criteria: Acceptable quality images Sufficient FOV to include temporomandibular joints, sella turcica and the Exclusion criteria: tip of chin Motion artifacts Large number of metallic attenuation artifacts Prior orthognathic surgery or trauma to head and neck region causing gross changes in the osseous structures Missing permanent incisors or first molars Unerupted or supernumerary teeth overlapping the incisor apices The study was approved by the University of Iowa Institutional Review Board. A research database was created in Axium on the collegiate server. The images were anonymized and imported into Dolphin-3D version 11 (Dolphin Imaging and Management Systems, Chatsworth, California) for viewing.

20 10 Two observers, one resident (observer 1) and one faculty member with 40 years of experience (observer 2) in oral and maxillofacial radiology, were involved in this research. The landmarks were defined and the way to localize them using the two dimensional planes of space axial, sagittal and coronal- was decided by discussion among the two observers and an orthodontist who is a member of the research committee. Not all three planes were required to locate all the landmarks. Inter-observer calibration on the landmark locations was obtained prior to initiating the study. Training on the use of the 3D imaging software and identification of cephalometric landmarks was provided by the orthodontist on the research committee. The observers were allowed to use any of the software s image enhancing features to better visualize structures of interest. The scanned volumes were oriented using the 3D hard tissue surface representation such that the midsagittal plane was vertical, the transorbital plane was horizontal and the Frankfort horizontal plane was horizontal. The orientation was saved and was kept the same, so that every image of each subject had the same reference planes. This allowed constant references to which x-, y- and z- coordinates were derived for each landmark identified by an observer. The landmarks were localized using the axial, sagittal and coronal sections as needed. The landmarks were then digitized, i.e. were assigned x-, y- and z- coordinates and were exported into a Microsoft Excel ( 2007 Microsoft Corporation) worksheet by the Dolphin software. All the digitized landmark identification sessions took place in a dimly lit room without interruption for as long as each observer needed to finish. A notebook computer- HP Pavilion dv 7 (Hewlett-Packard Company, California) with an NVIDIA GeForce GT 230M graphics card was used. The computer had a 32 bit color monitor. A screen resolution of pixels was used for display. Each observer localized each landmark on each subject twice, with a gap of at least 7 days between the first and the second measurements.

21 11 Landmark definitions 1. Upper molar (U6) - most inferior point of the mesiobuccal cusp of the upper right 1 st molar. When going medially on the sagittal, followed by going posteriorly on the coronal. 2. Lower molar (L6) - most superior point of the mesiobuccal cusp of the lower right 1 st molar. When going medially on the sagittal, followed by going posteriorly on the coronal. 3. Mx1 Tip (U1T) - midpoint of the incisal edge of the upper right central incisor. When going inferiorly on the Axial, followed by going posteriorly on the Coronal, followed by going medially on the Sagittal. 4. Mx1 Root (U1R) - root apex of the upper right central incisor. When going inferiorly on the Axial, followed by going posteriorly on the Coronal, followed by going medially on the Sagittal. 5. Md1 Tip (L1T) - midpoint of the incisal edge of the lower right central incisor. When going superiorly on the Axial, followed by going posteriorly on the Coronal, followed by going medially on the Sagittal. 6. Md1 Root (L1R) - root apex of the lower right central incisor. When going superiorly on the Axial, followed by going posteriorly on the Coronal, followed by going medially on the Sagittal. 7. A point- point of maximum concavity in the midline of the alveolar process of the maxilla. When going posteriorly on the Coronal, followed by going medially on the Sagittal, followed by going inferiorly on the Axial. 8. ANS- most anterior midpoint of the nasal spine of the maxilla. When going posteriorly on the Coronal, followed by going medially on the Sagittal, followed by going inferiorly on the Axial. 9. PNS- most posterior midpoint of the nasal spine of the maxilla. When going anteriorly on the Coronal, followed by going medially on the Sagittal, followed by going inferiorly on the Axial.

22 Nasion- point of maximum concavity in the midline in the region of the frontonasal suture. When going posteriorly on the Coronal, followed by going medially on the Sagittal, followed by going inferiorly on the Axial. 11. B Point- point of maximum concavity in the midline of the alveolar process of the mandible. When going posteriorly on the Coronal, followed by going medially on the Sagittal, followed by going inferiorly on the Axial. 12. Menton- most inferior midpoint of the chin on the outline of the mandibular symphysis. When going posteriorly on the Coronal, followed by going medially on the Sagittal, followed by going inferiorly on the Axial. 13. Pogonion- most anterior midpoint of the chin on the outline within one centimeter from the inferior border. When going posteriorly on the Coronal, followed by going medially on the Sagittal, followed by going inferiorly on the Axial. 14. Gonial angle- most postero-inferior point on the curvature of the right angle of the mandible in the sagittal slice where the posterior border of the ramus is first continuous, and the most lateral point in the coronal plane that cuts through the first position of the point. Sagittal Coronal 15. Condylion- most posterosuperior point of the right mandibular condyle in the sagittal plane, where the sagittal plane corresponds to the most posterior point on the coronal plane. Coronal Sagittal 16. Orbitale- most inferior point of right infraorbital rim midway on the cortex of the bone on the sagittal plane, which corresponds to the first axial plane showing complete rim when going inferiorly. Axial Sagittal Coronal 17. Sella turcica (Landmark 1) - geometric center of the pituitary fossa. When going inferiorly on the Axial, followed by going medially on the Sagittal. 18. Sigmoid notch (Landmark 2) - most inferior point along the superior border of right ramus of the mandible, which corresponds to the first axial plane that shows complete border when going inferiorly, followed by going medially on the Sagittal.

23 Figure 1. Laptop computer used for landmark localization 13

24 Figure 2. Dolphin imaging screen 14

25 Figure 3. Location of Orbitale landmark on the MPR slices and 3D surface rendered image 15

26 Figure 4. Location of the Upper molar landmark on MPR slices and 3D surface rendered image 16

27 17 Overview and methods The main goal of this study is to compare the observer reliability for anatomical landmark identification on 3D cephalometric subject images from Galileos and i-cat cone beam computerized tomography (CBCT). Twenty subjects were involved in this study, 10 subjects for the Galileos CBCT and a different set of 10 subjects for the i-cat CBCT. Three dimensional coordinates were recorded for 18 landmarks twice by two different observers. Intra-observer Reliability A one-way ANOVA model was created for each coordinate of each landmark separately for each machine (108 models in all) with groups defined by participant ID in each case. The residuals from each model were examined for normality using the Shapiro-Wilk test. In the cases where the normality assumption was validated, the ICC described by Shrout and Fleiss (1979) was used to assess intra-observer landmark reliability. A method to compare independent correlation coefficients utilizing the Fisher z transformation (Fisher, 1970; Zar, 1998) was performed to compare the ICCs from the two machines for each landmark. For the variables in which the residuals from the ANOVA model were not normally distributed, a non-parametric rank-based ICC by Rothery (1979), referred to as RICC, was used to assess intra-observer agreement. The appropriate ICC was computed for each of the coordinates measured, i.e., for the X, Y, and Z planes for each landmark. The parametric ICC reflects the proportion of variance attributable to between subject differences. The non-parametric version is based on the probability of certain types of concordances among the observations, and is calculated based on ranked data. In both instances, the null hypothesis associated with significance testing is that the relevant coefficient is equal to zero. The ICC ranges from 0 to 1, with 1

28 18 indicating perfect agreement. A commonly adopted minimum acceptable ICC is 0.80 (Shrout and Fleiss, 1979), with 0.90 and above generally considered excellent agreement. The Euclidean distance, the square root of the sum of squared differences between the two selected landmark positions, was calculated for each observer and descriptive statistics are included in the results. These distances for each landmark were tested for differences in the machines using a Wilcoxon Rank Sum test. To further assess differences between machines, a linear mixed model (LMM) with Euclidean distance between the dual measures as the dependent variable was used including a fixed machine effect and a random subject nested within fixed machine effect. LMM residuals were examined for validation of assumptions including normality; and data transformations, including square root and natural logarithm, of the dependent variable were explored and implemented where required. Inter-Observer Reliability Similar to the comparison of intra-observer reliability, a one-way ANOVA model was created for each coordinate of each landmark separately for the first measures and second measures (108 models in all) with groups defined by participant ID in each case. The residuals from each model were examined for normality using the Shapiro-Wilk test. In the cases where the normality assumption was validated, the ICC described by Shrout and Fleiss (1979) was used to assess inter-observer landmark reliability. The Rothery ICC (RICC) was used in the event that assumptions of the one-way ANOVA model were not met. Note that the nonparametric Rothery method, when compared to the one-way ANOVA, gives a relative asymptotic efficiency of 15 2 or approximately However, simulations have shown that asymptotic relative efficiency tended to increase with the estimate of intraclass correlation, and was generally higher than the asymptotic value of 88% (Rothery, 1979). These findings imply that the Rothery has good

29 19 performance relative to the parametric ICC, with power to detect significant agreement that is only slightly less than that of the ICC, when normality assumptions are met. Also, a comparison of independent correlation coefficients utilizing the Fisher z transformation (Fisher, 1970; Zar, 1998) was performed to compare the intraclass correlations from the first measures of two machines and separately for the second measures for each landmark. To assess inter-observer reliability between machines, a LMM with Euclidean distance between the two first measures (one by each observer) as the dependent variable was used including a fixed machine effect and a random subject nested within fixed machine effect. LMM residuals were examined for validation of assumptions including normality; and data transformations, including square root and natural logarithm, of the dependent variable were explored and implemented where required. The LMM process was repeated for the second measures. The Euclidean distance was calculated for the first measures and separately for the separate measures. A descriptive summary of the Euclidean distances was compiled and the distances from the two machines for each landmark were compared using a Wilcoxon Rank Sum test. Unified Model A LMM was used to model the Euclidean distances for each landmark with a fixed effect for observer, a fixed effect for machine, and a random subject within machine effect. LMM residuals were examined for validation of assumptions including normality; and data transformations, including square root and natural logarithm, of the dependent variable were explored and implemented where required. Analysis was done using SAS Enterprise Guide 4.2 and R statistical software, and a significance level of 0.05 was specified.

30 20 CHAPTER III RESULTS Intra-Observer Agreement Ten subjects were imaged using the Galileos CBCT machine and 10 using the i- CAT machine and dual three dimensional coordinates recorded for each of 18 landmarks in three dimensions. Table 1 shows the within-coordinate estimates of ICC to assess intra-observer reliability of the dual coordinates recorded by observer 1. Included in the table are footnotes containing information on the significance probabilities of the ICC and of the test of normality of model residuals indicating the conformance to model assumptions. The mean ICC is included as is routine in the literature on this topic. All ICCs in each landmark and coordinate were greater than 0.80, and most greater than 0.90, indicating excellent reliability throughout. The ICC values below 0.85 are highlighted in yellow on the table. The results for observer 1 in Table 1 show that the parametric intraclass correlation estimate with the lowest estimates parametric ICC estimates occur in the X coordinate of Nasion (Galileos ICC=0.8277; and i-cat ICC=0.8485) followed by the X coordinate of the anterior nasal spine (ANS) (Galileos ICC=0.8618; and i-cat ICC=0.9012) and Md1 Tip (Galileos ICC=0.9983; i-cat ICC=0.8944). All other ICC estimates were higher than 0.93 (p<0.0055) for the Galileos CBCT and higher than 0.91 (p<0.0034) for i-cat indicating excellent intra-observer agreement for images taken from both CBCT machines for observer 1. The same subjects were landmarked twice by observer 2. Table 2 shows the within-coordinate estimates of parametric ICC to assess intra-observer reliability of the dual coordinates recorded by observer 2. The ICC values below 0.85 are highlighted in yellow on the table. The landmarks with the lowest reliability were the X coordinates of Nasion (Galileos ICC=0.9255; i-cat ICC=0.3199), ANS (Galileos ICC=0.6574; i-cat

31 21 ICC=0.9739), Mx1 Root (Galileos ICC=0.7231; i-cat ICC=0.9526), Md1 Tip (Galileos ICC=0.9854; i-cat ICC=0.6062), and Md1 Root (Galileos ICC=0.9841; i-cat ICC=0.8300). For the X coordinate of the ANS landmark on the Galileos machine the agreement was not significant (p=0.0552). The same was true for Nasion X coordinate (p=0.2777) and the Md1 Tip landmark (p=0.0814) on the i-cat CBCT machine. For the Y coordinate of Orbitale the ICC was for Galileos, but for i-cat. All other ICC estimates of intra-observer reliability for observer 2 were >0.92 (p<0.0289) for the Galileos machine and ICC > 0.89 (p<0.0053) on the i-cat machine. Tables 3 (observer 1) and 4 (observer 2) shows the nonparametric estimates for all coordinates and all landmarks for the purpose of comparison between the two CBCT machines. The ICC values below 0.85 are highlighted in yellow in the table. It should also be noted that a number of Rothery ICC estimates had values of 1.0. In this rank-based procedure, a value of 1.0 indicates that the dual coordinates (in the specific dimension) had contiguous ranks for all subjects. It is not an indication of perfect agreement, i.e. of no distance between dual measures, simply an indication that the variation in the ranks of the coordinate values is completely explained when the data are grouped by subject. The intra-observer agreement using the nonparametric measure was excellent overall, and in only two instances (ANS ICC=0.7444, and Nasion ICC=0.7500) for observer 2 was the ICC <0.80. It is worth noting that in a few landmarks and coordinates the ICC estimates were much higher for the nonparametric measure. This is most likely a reflection of one or two subjects with a large difference between landmark replications. Because it is based on ranks, the nonparametric method is more robust to outliers in the dataset and will generally give a higher estimate of agreement in these cases.

32 22 Comparison of Intraclass Correlation Coefficients The parametric ICC estimates on the Galileos CBCT machine were compared to the parametric ICC estimates on the i-cat machine for each landmark and each coordinate. The method of comparing correlation coefficients due to Fisher was used (Fisher, Zar). For observer 1 there were significant differences in the reliability of Md1 Root R (Galileos: vs. i-cat: ; p<0.0001), Md1 Tip R (Galileos: vs. i-cat: ; p<0.0001), and Mx1 Tip R (Galileos: vs. i-cat: ; p=0.0050) in the X coordinate, with Galileos being more reliable for Md1 Root R and Md1 Tip R and i-cat more reliable for the landmark Mx1 Tip R. (Table 5) Significant differences are highlighted in yellow on Tables In the Y dimension, there were significant differences in Md1 Tip R (Galileos: vs. i-cat: ; p=.0018), Me (Galileos: vs. i-cat: ; p=0.0015), Nasion (Galileos: vs. i-cat: ; p=0.0466), and Orbitale (Galileos: vs. i-cat: ; p=0.0010) for observer 1 with Galileos being more reliable for Md1 Tip R, Me, and Orbitale, but i-cat more reliable for Nasion. (Table 6) There were significant differences in the Z coordinate of the landmarks Condylion (Galileos: vs. i-cat: ; p=0.0439), Gonial Angle (Galileos: vs. i- CAT: ; p=0.0313), and Mx1 Tip R (Galileos: vs. i-cat: ; p=0.0418), and Nasion (Galileos: vs. i-cat: ; p=0.0007) with Galileos being more reliable for Condylion, Gonial Angle, and Mx1 Tip R, but i-cat more reliable for Nasion. (Table 7) For observer 2 there were significant differences in the reliability of ANS (Galileos: vs. i-cat: ; p=0.0046), B Point (Galileos: vs. i-cat: ; p=0.0036), Gonial Angle (Galileos: vs. i-cat: ; p=0.0457), Md1 Root R (Galileos: vs. i-cat: ; p=0.0116), Md1 Tip R (Galileos: vs. i-cat: ; p=0.0003), and Nasion (Galileos: vs. i-cat: ;

33 23 p=0.0076), in the X coordinate, with Galileos being more reliable for all landmarks in which there were differences except ANS where i-cat was the more reliable. (Table 8) In the Y dimension, there were significant differences in Landmark 2 (Sigmoid notch) (Galileos: vs. i-cat: ; p=0.0275), Md1 Root R (Galileos: vs. i-cat: ; p=0.0428), Md1 Tip R (Galileos: vs. i-cat: ; p=0.0342), and Orbitale (Galileos: vs. i-cat: ; p=0.0057), for observer 2 with Galileos being more reliable for all these landmarks compared to the i-cat machine. (Table 9) There were significant differences in the Z coordinate of the landmarks Condylion (Galileos: vs. i-cat: ; p=0.0324), Lower Molar (Galileos: vs. i- CAT: ; p=0.0011), and Md1 Root R (Galileos: vs. i-cat: ; p=0.0289), with Galileos being more reliable for the Condylion and Md1 Root R landmarks and i-cat more reliable for the Lower Molar landmark. (Table 10) Intra-Observer Euclidean Distance Analysis The Euclidean distance between first and second landmark coordinates were calculated between the two points in three-dimensional space. The Euclidean distance between two points is defined to be the square root of the sum of squared within coordinate differences, and is a measure of clinical accuracy. Descriptive statistics were compiled for the Euclidean distance between the first and second landmark selections. (Tables 11 and 12, significant differences highlighted in yellow) For observer 1, the landmarks with the greatest median Euclidean distances were Gonial Angle (Galileos: 0.96mm, i-cat: 1.45mm), Pog (Galileos: 0.70mm, i-cat: 1.01mm), Condylion (Galileos: 0.93mm, i-cat: 0.88mm), and Nasion (Galileos: 0.85mm, i-cat: 0.73mm). (Table 11) The distances for each landmark were tested for significant differences using the Wilcoxon Rank Sum test. For observer 1, the landmarks Landmark 1 (Sella) (p=0.0046),

34 24 Md1 Tip R (p=0.0312), and Upper Molar R (p=0.0126) were significantly different, with the distances significantly smaller on the Galileos machine than the i-cat machine for the Md1 Tip R landmark. The dual selections of the Sella and Upper Molar were significantly closer on the i-cat compared to the Galileos machine. (Table 11) For observer 2, the landmarks with the greatest median Euclidean distances were PNS (Galileos: 0.83mm, i-cat: 1.13mm), Gonial Angle (Galileos: 2.01mm, i-cat: 1.11mm), Pogonion (Galileos: 1.23mm, i-cat: 0.98mm), Orbitale (Galileos: 1.35mm, i- CAT: 0.53mm), A (Galileos: 1.34mm, i-cat: 0.69mm), ANS (Galileos: 1.24mm, i- CAT: 0.76), Mx1 Root R (Galileos: 1.25mm, i-cat: 0.91mm), and Me (Galileos: 1.24mm, i-cat: 0.81mm). (Table 12) There were significant differences, using the Wilcoxon Rank Sum test, in the Euclidean distances between the two landmark selections for Sella (p=0.0140) and Me (p=0.0140) for observer 2. In both cases, the i-cat machine had significantly smaller distances between landmark selections than did the Galileos machine. (Table 12) Intra-Observer Euclidean Distance Linear Mixed Models A LMM was used to model the Euclidean distances for each landmark and each observer with a fixed machine effect and a random subject within machine effect. For observer 1 ratings, the models for Sella (p=0.0010), Md1 Tip R (p=0.0242), and Upper Molar R (p=0.0053) had significant machine effects. The initial model for the Md1 Tip R landmark did not satisfy the assumption of the normality of residuals and the Euclidean distances were natural log transformed in the final model reported here. (Table 13) Significant machine differences are highlighted in yellow. For observer 2ratings, there was a significant machine effect in the models for Gonial Angle (p=0.0023), Sella (p=0.0116), and Me (p=0.0153). Note that the final model for the landmark Me involved a natural log transformation of the Euclidean distance. (Table 14) Significant machine differences are highlighted in yellow.

35 25 It is worth noting that the results of the LMM validated all significant machine differences in Euclidean distance measures detected with the Wilcoxon Rank Sum test. However, in addition to these, a significant machine difference (p=0.0023) was also found for Gonial Angle R for observer 2. The variance of this landmark was much greater than that of the other landmarks (Table 12) particularly on the i-cat CBCT. Whereas the Wilcoxon test is excellent at detecting differences in location, the model may be more sensitive to the magnitude of variance that exists in the data. Inter-Observer Agreement The inter-observer agreement was determined using ICC as measure of agreement for the first measures and also for the second measures. Results are given in Tables 15 and 16 with ICC < 0.85 highlighted in yellow. Table 15 shows the parametric intraclass correlation estimate for the first measures with the lowest estimates parametric ICC estimates for the Galileos machine occurring in the X coordinate of ANS (Galileos ICC=0.6153, p=0.1081) followed by the Y coordinate of the Nasion (Galileos ICC=0.6954, p=0.0243) and the X coordinate of Nasion (Galileos ICC= , p=0.0086). All other ICC estimates for the Galileos machine were higher than 0.86 (p<0.0042). For the i-cat machine the lowest ICC were for the X coordinate of Nasion (i-cat ICC=0.2824, p=0.3718) and Md1 Root R (i-cat ICC=0.7581, p=0.0289) with all other ICC >0.90. Similar to the reliability of the first measure, the X coordinates of Nasion were among the lowest ICC estimates (Galileos ICC=0.7909, p=0.0178; i-cat ICC=0.7867, p=0.0204) along with the Nasion Y coordinate for the Galileos machine (Galileos ICC=0.6707, p=0.0329). However for the second measures there was a lower ICC estimate for the Md1 Tip R on the i-cat machine (i-cat ICC=0.4945, p=0.1994) and also for the Mx1 Root R ICC estimate for the Galileos machine (Galileos ICC=0.8323, p=0.0093).

36 26 Comparison of Intraclass Correlation Coefficients: Inter- Observer The ICCs for inter-observer reliability of the first measures were compared. For the X coordinate, the landmarks Gonial Angle (Galileos: vs. i-cat: , p=0.0408), Md1 Root R (Galileos: vs. i-cat: , p<0.0001), Mx1 Tip R (Galileos: vs. i-cat: , ), Orbitale (Galileos: vs. i-cat: , ), and Pog (Galileos: vs. i-cat: , ) differed significantly with respect to the reliability for each machine. The i-cat machine was more reliable on the Gonial Angle, Mx1 Tip R, and Orbitale landmarks, while the Galileos machine was more reliable for the Md1 Root R and Pog landmarks. (Table 17) In the Y coordinate of the first measures, the landmarks Md1 Root R (Galileos: vs. i-cat: , ), Md1 Tip R (Galileos: vs. i-cat: , ), and Orbitale (Galileos: vs. i-cat: , ) differed significantly between the Galileos and i-cat machines. The Galileos machine was more reliable for all of the landmarks in which the ICC differed significantly in the Y coordinate. (Table 18) In the Z coordinate, the ICCs for the landmarks Gonial Angle (Galileos: vs. i-cat: , ), Mx1 Tip R (Galileos: vs. i-cat: , ), and Nasion (Galileos: vs. i-cat: , ) differed significantly when the first measures were used. The Galileos machine was more reliable in the Gonial Angle and Mx1 Tip R landmarks, but the i-cat machine was more reliable for the Nasion Z coordinate measure. (Table 19) The ICCs for the second measures were compared and the results given in Tables They were similar to the first measure results in the fact that Gonial Angle and Md1 Root R were both significantly different in the X coordinate reliability with Galileos being the more reliable machine for both of these landmarks. However other landmarks were found to differ when the second measures were used (X Coordinate: A, Md1 Tip R;

37 27 Y Coordinate: Me; Z Coordinate: A, Condylion) as opposed to the first measure. See Tables for details. Inter-Observer Euclidean Distance Analysis The Euclidean distance between observer 1 and observer 2 first measure of each of the landmark coordinates were calculated between the two points in three-dimensional space. Descriptive statistics were compiled for the Euclidean distance between observer 1 and observer 2 first measure of each landmark selections. (Tables 23 and 24) For the first measures, the landmarks with the greatest median Euclidean distances were Nasion (Galileos: 2.94mm, i-cat: 0.96mm), Gonial Angle (Galileos: 2.81mm, i-cat: 4.89mm), Orbitale (Galileos: 1.49mm, i-cat: 0.93mm), and B Point (Galileos: 1.23mm, i-cat: 0.99mm). (Table 23) The distances for each landmark were tested for significant differences using the Wilcoxon Rank Sum test. For the first measures, the landmarks Sella (p=0.0140), and Mx1 Root R (p=0.0140were significantly different, with the distances significantly smaller on the Galileos machine than the i-cat machine for the Mx1 Root R landmark, however the two measures of Sella were significantly closer on the i-cat compared to the Galileos machine. (Table 23) For the second measures, the landmarks with the greatest median Euclidean distances were Gonial Angle (Galileos: 5.54mm, i-cat: 4.79mm), Nasion (Galileos: 2.13mm, i-cat: 0.85mm), ANS (Galileos: 1.07mm, i-cat: 1.39mm), and Orbitale (Galileos: 1.42mm, i-cat: 0.80mm). (Table 24) There were significant differences, using the Wilcoxon Rank Sum test, in the Euclidean distances between the two second landmark selections for Nasion (p=0.0073) and Orbitale (p=0.0073). In both cases, the i-cat machine had significantly smaller distances between landmark selections than did the Galileos machine. (Table 24)