Measurement of distances between anatomical structures using a translating stage with mounted endoscope

Transcription

1 Measurement of distances between anatomical structures using a translating stage with mounted endoscope Lueder A. Kahrs *a, Gregoire S. Blachon a, Ramya Balachandran a, J. Michael Fitzpatrick b, Robert F. Labadie a a Department of Otolaryngology, Vanderbilt University Medical Center, st Avenue South, MCE 10420, South Tower, Nashville, TN USA b Department of Electrical Engineering and Computer Science, Vanderbilt University, th Avenue South, 363 Jacobs Hall, Nashville, TN USA ABSTRACT During endoscopic procedures it is often desirable to determine the distance between anatomical features. One such clinical application is percutaneous cochlear implantation (CI), which is a minimally invasive approach to the cochlea via a single, straight drill path and can be achieved accurately using bone-implanted markers and customized microstereotactic frame. During clinical studies to validate CI, traditional open-field cochlear implant surgery was performed and prior to completion of the surgery, a customized microstereotactic frame designed to achieve the desired CI trajectory was attached to the bone-implanted markers. To determine whether this trajectory would have safely achieved the target, a sham drill bit is passed through the frame to ensure that the drill bit would reach the cochlea without damaging vital structures. Because of limited access within the facial recess, the distances from the bit to anatomical features could not be measured with calipers. We hypothesized that an endoscope mounted on a sliding stage that translates only along the trajectory, would provide sufficient triangulation to accurately measure these distances. In this paper, the design, fabrication, and testing of such a system is described. The endoscope is mounted so that its optical axis is approximately aligned with the trajectory. Several images are acquired as the stage is moved, and threedimensional reconstruction of selected points allows determination of distances. This concept also has applicability in a large variety of rigid endoscopic interventions including bronchoscopy, laparoscopy, and sinus endoscopy. Keywords: triangulation, 3D reconstruction, feasibility study, endoscope measurement 1. DESCRITION OF UROSE Thousands of rigid endoscopic procedures are performed annually. While the intent of many of these procedures is to assess a potentially pathological condition (e.g. assessment of the airway for tumor growth), accurate measurement of critical distances is lacking. Recently, we have recognized the need for accurate measurement of anatomical features during surgical interventions. Specifically, we need to accurately predict the path of a proposed trajectory defined by a customized microstereotactic frame for a new surgical procedure called the percutaneous cochlear implantation (CI) 1-4 in which image-guided, minimally-invasive access of the cochlea is desired. During CI, fiducial markers are implanted in the temporal bone following which a computed tomography (CT) scan is acquired. ertinent anatomy is identified via non-rigid registration of the CT to an atlas 5, and a suitable drilling trajectory is selected 6. A customized microstereotactic frame is designed and constructed to mount on the fiducial markers and constrain the drill along the planned trajectory. Validation studies of the CI technique are underway in which a traditional, open-field approach to the cochlea is performed (mastoidectomy with facial recess) after which the customized microstereotactic frame is affixed and the trajectory achieved by the frame is verified 7,8. The planned drilling trajectory is chosen such that it targets the cochlea without damaging critical structures such as the facial nerve, which, if damaged, causes ipsilateral facial paralysis, and the chorda tympani, which, if damaged, causes ipsilateral taste insensitivity. During the validation, we sought to determine how close the surgical trajectory that is achieved by the customized microstereotactic frame comes to the vital anatomy the facial nerve and the chorda tympani. These measurements were initially done by passing sham drill bits (shaped like a drill bit but with no cutting surfaces) of varied sizes through the customized microstereotactic frame. Due * l.a.kahrs@vanderbilt.edu; phone ; fax ;

2 to limited access within the facial recess (region bounded by the facial nerve and the chorda tympani), the actual distances from the bit to anatomical features could not be measured directly with calipers. Instead, we estimated these distances post-operatively using photographs acquired during the validation via an endoscope while the drill bit was positioned at the facial recess (Figure 1). The drill bit in the picture was used to provide a scale, but the accuracy of this approach was limited by the projective geometry of the three-dimension (3D) field to the two-dimension (2D) photograph. To overcome this problem, herein we propose a method that allows intra-operative evaluation of distances for ensuring safe margins from the drill bit to critical structures using an endoscope translation stage (ETS), which, when combined with triangulation, allows accurate, repeatable, and real-time distance measurements during CI validation studies. Figure 1. CI validation using sham drill bit. A picture is captured using an endoscope in the operating room during the validation with the sham drill bit at the facial recess. The points on the facial nerve (plus sign) and the chorda tympani (asterisk) that are closest to the drill are manually marked in the picture. Using the drill bit as a reference for scale, the distances between the bit and the critical structures are then computed. 2. MEASUREMENT RINCILE Triangulation and 3D reconstruction from different camera views is well-known from the literature 9,10. A point in 3D is observed from a minimum of two known camera poses, resulting in camera images representing the projection of the 3D point. In this paper, the measurement principle is based on motion of a single camera observing a stable scene. In our scenario, an endoscope is utilized on a linear stage, and we have a special case that involves translation between camera poses that is roughly coaxial with the optical axis. For simplification of the following explanation, it is assumed that the optical axis (OA) is the same as the axis of the endoscope movement and both axes include the point E, which is the intersection of the trajectory of the stage and the image plane (equivalent to the epipole in two-camera stereo vision and the focus of expansion in optic flow). Variation between the camera poses can be described by changing only one transformation parameter and using different translational positions z i for capturing images. Figure 2 illustrates three simulated endoscopic images from different coaxial poses z i visualizing a 3D point on a structure S. Inside the particular images the corresponding 2D points (z i ) with their pixel coordinates (u,v) belong to the same 3D point. As shown in the far-right subfigure of Figure 2, all points (e.g. (z i )) observed from different camera poses appear to move radially along lines originating at E, and a point closer to E the will radiate less when is z i varied. All 2D points (z i ) can be mapped into 3D rays, R i, (shown in Figure 3) that pass through the optical center of the camera (, view point) with an angular difference i from the OA. The intersection of two or more rays (R 1, R 2,, R i ) from different camera poses leads to reconstruction of a point I, as our measured estimate of the above mentioned 3D point. The subscript I signifies that it is a point of intersection and differentiates it from the ground truth point. Figure 3a-c illustrates the measurement principle by showing a cross section, including the optical axis and point I. The angle i shown in Figure 3a corresponds to the distance (in the sense of the pixel coordinates) between E and (z i ) in Figure 2.

3 v u (z 1 ) (z 2 ) (z 3 ) S E Figure 2. Simulated endoscopic views from different coaxial distances z i (left panel, z 1 ; center-left, z 2 ; center-right, z 3; and far-right, overlay of all three) of a representative structure S and corresponding 2D points (z i ). The epipole E is the only point which does not move in the image while z i changes. Corresponding points move along radial lines away from the E as z i changes with points that are closer to E moving less. (a) OA (b) (c) (d) z 2 z 1 z z z 2 z 3 z 1 R i i R 2 I R 3 I C I23 I13 I12 R 1 R 1 R 2 Figure 3. 2D illustration of triangulation principles for a coaxial moving camera (endoscope). (a) A ray R i passes the optical center ( ) and has an angle i from the optical axis (OA). (b) A second ray will intersect at I under certain conditions, e.g. both rays are in a common plane or the value z 1 -z 2 and angular difference 1-2 are in a certain range. (c) Examples of three rays that intersect at one point I. (d) Inaccuracies, e.g. in segmentation or determination of relative coaxial camera poses can lead to multiple intersection points, where a mean, calculated intersection point C needs to be determined. Real measurements following this principle usually lead to rays that do not perfectly intersect because of pixelation, image noise, non-precise feature segmentation, inaccurate knowledge of the relative camera movements, etc. Figure 3d shows an example with multiple intersection points Iij and the need for the determination of a mean, calculated intersection point C. In a real 3D scenario the rays are usually skew lines, and as a result potential errors must be tackled. Two options are described in the following: One possibility is the determination of the shortest line segment L that joins the skew lines (see Figure 4). The midpoint of this segment is the calculated intersection point C, but it is usually not identical with the ground truth () and result in a back projection error. Another possibility is the search for a minimized length of L (best case: intersecting rays, length is zero) by varying the pixel coordinates of (z i ). The distance of the variation is called d i. This is illustrated in Figure 5: The variation of (z i ) is symbolized by single-headed arrows and examples are given for (z 1 ) and d 1. The best variation of the measurement

4 ((z 1 ), grey dot) is towards the (unknown) ground truth (black dot), which would achieve a small back projection error. According to the literature 9,10, the back projection error is minimized, while L is kept approximately zero and the d i are minimized in the least-squares sense as well., z1 (b) (a) (z 1 ) OA R 1 C L, z3 R 3 (z 3 ) Figure 4. (a) 3D visualization of the measurement and triangulation principles for pure coaxial movements between two camera poses (z 1 and z 3, compare Figure 2). The image planes are mirrored around the optical centers for a compact depiction. The thicker rays merge at the ideal (ground truth) intersection point, while the thinner rays illustrate real measurements, which usually leads to skew lines. (b) The line segment L (black line) connects skew lines at their nearest distance and the point C symbolizes the midpoint of L. The distance (white doubleheaded arrow) is the back projection error in this case. (a) (z 1 ) (b) (c) (d) L C I d 1 Figure 5. (a) A measured point ((z 1 ), grey dot) will usually be unequal to the ideal feature point (black dot) and leads to skew lines during 3D reconstruction. Modifying the pixel coordinates of the endoscopic measurement (arrow) changes the rays and allow the search for a minimized length of the line segment L (best case: intersecting rays at I, see subfigures (b) and (c)). (d) Minimizing the distances d i in the least-squares sense (d 1 shown as example) at the same time while minimizing L (boundary condition) in all endoscopic camera views reduces the back projection error to a minimum.

5 3. SYSTEM ROTOTYE We designed the ETS for distance measurement by triangulation. For easy integration with our ongoing CI and for accurate measurement, the ETS was built as a linear guide system similar to a linear drill guide designed for CI 4. Figure 6 shows the realized prototype. However, other designs are possible for other endoscopic applications (e.g. laparoscopy). The endoscope is mounted on the slider at the endoscope s cubic portion, in which the light adapter connects to the main portion of the endoscope. We use a GYRUS ACMI explorent (diameter at tip: 1.7 mm, straight view) endoscope, although many other models could be used as well. The endoscope is attached via a C-mount endoscope coupler (focal length = 25 mm) to a USB camera (BigCatch EM-310C, 3.2Mpixel) and approximately 0.5 Mpixel (diameter ~800 pixels) are visible. Metal rods of specific lengths were used as distance holders (spacers) between the slide stop and the slider, to guarantee accurate determination of the relative coaxial camera movement (z i values). As a further enhancement to the ETS, we are able to include a potentiometer, which allows an electronic determination of the z i value. Software to integrate these hardware components was written to provide online measurements based on selecting corresponding points in images and a method to determine the ray intersections. Figure 6. rototype of ETS. (a) Adaptor to mounts the ETS to the customized microstereotactic frame, (b) main portion (bracket) of the ETS, (c) location for distance holders, (d) slider with clamp for cubic portion of an endoscope (e), (f) rails for sliding, (g) oblong hole for access of the linear potentiometer and (h) endoscope coupler. The inset (i) shows three spacers with length of 2.5, 10, and 20 mm. 4. EVALUATION We designed and built a tool to measure distances between points of interest, for our application from the centerline of the planned trajectory to vital structures, during CI validation studies. We have recently conducted our first patient trials in order to optimize the intra-operative workflow for this new device (Fig 7a). All parts were sterilizable. Distance holders (inset of Figure 6) were used to allow capturing images of the target, the facial recess, at different depths along the trajectory achieved by the customized microstereotactic frame. For more flexibility, we designed the distance holders such that they rest between the camera and holder (location c in Figure 6) so that the entire ETS can rotate when coupled to the microstereotactic frame. A long term solution is electronic measurement of the camera position along the rails using an optical encoder. Figure 7b shows the usage of the ETS during a cadevaric temporal bone study. The upper part shows two endoscopic images with superimposed labeling (black dashed lines and X s) from different measurement positions. Selecting corresponding points of interest in both images allows 3D reconstruction and determination of the distance between those points. Two corresponding points are shown as X s in the upper part and as spheres with (x,y,z)-value on the black line in the lower part of Figure 7b. The two spheres inside the transparent cylinder are the two on the OA.

6 (a) (b) (-1.24,0.77,10.87) 1.99mm (-1.09,-1.11,11.5) Figure 7. (a) Usage of the ETS in the operating room. (b) Triangulation example on a human cadaver temporal bone. Images are captured from two different coaxial endoscope poses. ositions of corresponding points in both images can be triangulated and distances between them can be determined. 5. DISCUSSION, CONCLUSION AND OUTLOOK Endoscopic procedures are often used to visualize the anatomy and determine pathological condition. The measurement of distances between anatomical structures during endoscopic evaluation is useful in several applications including validation of CI surgery. While others have reported on similar approaches 11-13, to the best of our knowledge this is the first report using a slide mechanism and triangulation to achieve the purpose of distance measurement using endoscopic images. By analyzing the literature it can be stated that the most common approaches to measurement with endoscopes include the usage of electromagnetic 11 or optical tracking 12 to determine pose changes of the endoscope. An advantage of our ETS in comparison to those tracking methods is that no additional navigation system is needed (cost reduction). A further idea of endoscopic measurements uses laser projections onto the organ surface and analyzes the pattern distribution by means of triangulation 13. After calculating how far the organ surface is away, virtual rulers are superimposed to measure the lateral dimensions on continuous surfaces. These rulers may work only piecewise, and it is unclear whether the algorithms work correctly on non-flat surfaces such as those inside the temporal bone. We show in this work a method to use the endoscope to measure distances between anatomical structures via an accurate sliding mechanism for an endoscope (ETS) and an appropriate triangulation methodology. The rigid connection of the ETS with the skull by the customized microstereotactic frame and the precise sliding mechanism allow accurate measurements. Automatic 3D reconstructions of outlines of anatomy (such as those illustrated as black lines in Figure 7b) are achievable. Ongoing work includes determination of, and comparison with, additional rays, e.g. the CI trajectory or the movement axis (epipole), and absolute calibration so that, in addition to the distances (and without an additional registration step), positions can be measured in the patient coordinate system using the rigid relationship that exists between the endoscope, the microstereotactic frame, and the patient CT.

7 ACKNOWLEDGMENT This work was supported by Award Number R01DC and R01DC from the National Institute on Deafness and Other Communication Disorders. The content is solely the responsibility of the authors and does not necessarily represent the official views of the National Institute on Deafness and Other Communication Disorders or the National Institutes of Health. REFERENCES [1] Labadie, R. F., Chodhury,., Cetinkaya, E., Balachandran, R., Haynes, D. S., Fenlon, M. R., Jusczyzck, A. S. and Fitzpatrick, J. M., "Minimally invasive, image-guided, facial-recess approach to the middle ear: demonstration of the concept of percutaneous cochlear access in vitro," Otol. Neurotol. 26(4), (2005). [2] Warren, F. M., Balachandran, R., Fitzpatrick, J.M. and Labadie, R. F., "ercutaneous cochlear access using bonemounted, customized drill guides: demonstration of concept in vitro," Otol. Neurotol. 28(3), (2007). [3] Labadie, R. F., Mitchell, J., Balachandran, R., and Fitzpatrick, J. M., "Customized, rapid-production microstereotactic table for surgical targeting: description of concept and in vitro validation," Int. J. Comput. Assist. Radiol. Surg. 4(3), (2009). [4] Balachandran, R., Mitchell, J. E., Blachon, G., Noble, J. H., Dawant, B. M., Fitzpatrick, J. M. and Labadie, R.F., "ercutaneous cochlear implant drilling via customized frames: an in vitro study," Otolaryngol. Head Neck Surg. 142(3), (2010). [5] Noble, J. H., Dawant, B. M., Warren, F. M. and Labadie, R.F., "Automatic identification and 3D rendering of temporal bone anatomy," Otol. Neurotol. 30(4), (2009). [6] Noble, J. H., Majdani, O., Labadie R. F., Dawant, B. and Fitzpatrick, J. M., "Automatic determination of optimal linear drilling trajectories for cochlear access accounting for drill-positioning error," Int. J. Med. Robot. 6(3), (2010). [7] Labadie, R. F., Noble, J. H., Dawant, B. M., Balachandran, R., Majdani, O., Fitzpatrick, J. M., Clinical validation of percutaneous cochlear implant surgery: initial report, Laryngoscope 118(6), (2008). [8] Labadie, R. F., Balachandran, R., Mitchell, J. E., Noble, J. H., Majdani, O., Haynes, D. S., Bennett, M. L., Dawant, B. M. and Fitzpatrick, J. M., "Clinical validation study of percutaneous cochlear access using patient-customized microstereotactic frames, " Otol. Neurotol. 31(1), (2010). [9] Kanatani, K., Sugaya, Y. and Niitsuma, H., "Triangulation from two views revisited: Hartley-Sturm vs. optimal correction, " roc. 19th British Machine Vision Conf., (2008). [10] Hartley, R. I. and Sturm,., "Triangulation," Computer Vision and Image Understanding 68(2), (1997) [11] Koishi, T., Sasaki, M., Nakaguchi, T., Tsumura, N. and Miyake, Y., "Endoscopy System for Length Measurement by Manual ointing with an Electromagnetic Tracking Sensor," Optical Review 17(2), (2010). [12] Wang, H., Mirota, D., Hager, G. and Ishii, M., "Anatomical reconstruction from endoscopic images: toward quantitative endoscopy," Am. J. Rhinol. 22(1), (2008). [13] Nakatani, H., Abe, K., Miyakawa, A. and Terakawa, S., "Three-dimensional measurement endoscope system with virtual rulers," J. Biomed. Opt. 12(5), (2007).