Video Registration Virtual Reality for Non-linkage Stereotactic Surgery P.L. Gleason, Ron Kikinis, David Altobelli, William Wells, Eben Alexander III, Peter McL. Black, Ferenc Jolesz Surgical Planning Laboratory, Brigham and Women s Hospital, Divisions of Neurosurgery and Magnetic Resonance Imaging, Departments of Surgery and Radiology, Harvard Medical School, Boston, Mass., USA Key Words Virtual reality, Stereotactic surgery, non-linkage, Brain tumor, Imaging, three-dimensional Abstract We have combined three-dimensional (3D) computer reconstructed neuroimages with a novel video registration technique for virtual reality based, image guided surgery of the brain and spine. This technique allows the surgeon to localize cerebral and spinal lesions by superimposing a 3D reconstructed MR or CT scan on a live video image of the patient. Once the patient s scan has been segmented into the relevant components (e.g., tumor, ede ma, ventricles, arteries, brain and skin), the surgeon studies the 3D anatomy to determine the optimal surgical approach. The proposed intraoperative surgeon s perspective is dis played in the operating room at the time of surgery using a portable workstation. The patient is then brought to the operating room and positioned according to the planned approach. A video camera is trained on the patient from the proposed intraoperative surgeon s perspective. A video mixer merges the images from the video camera and the 3D computer recon struction. This video mixer can vary the output intensity of the two input images between 100% of either and 50% of both. This visually superimposes the two images, not unlike a photographic double exposure. The patient s position and the 3D reconstruction are then adjusted until the images on the video mixer s output monitor are identical in terms of scale, position and rotation. This superimposition is facilitated by aligning various surface land marks such as the external auditory canal, lateral canthus, and nasion. In some cases, such as with spinal tumors, capsules placed on the skin prior to scanning serve as fiducials. After alignment of the video and computer skin images, the computer image of the skin is selectively deleted leaving the 3D image of the underlying brain or spine superimposed on the video image of the patient s skin. The borders of the tumor and important cortical sulci or spinal anatomy may then be outlined on the patient s skin using indelible markers. These markings allow the surgeon to plan an adequate opening with minimal exposure of adjacent structures. So far we have used this technique to localize seven brain tumors and one intra dural, extramedullary spinal tumor. In addition, the same technique was used to guide the repositioning of the bones in a reconstructive craniofacial surgery. In each case we found excellent correlation between the operative findings and the predicted anatomy. No patients suffered any permanent neurologic deficit. In the past the task of correlating the pre and intraoperative imaging studies with the
view of the operative field during surgery has been left to the surgeon s imagination. Planning and executing surgeries with three-dimensional (3D) computer reconstructions of diagnostic images has made the task of localization easier and more precise. To incorporate these technological advances in central nervous system tumor surgery we combined the use of such 3D computer reconstructed images with a video patient registration technique to facilitate the planning and guidance of brain and spinal tumor surgery (fig. 1). Image Processing The process begins with the production of 3D renderings of MR and CT images which can be displayed and manipulated in real time on computer workstations (Sun Microsystems, Mountain View, Calif.). To facilitate the reconstruction, optimized scanning techniques are used to gather the image data. Maximal contrast between tissues of interest simplifies the segmentation process, and con sequently the choice of pulse sequences is critical. We have found that 1.5 mm post gadolinium coronal SPGR images of the head are best for providing contrast between the skin, brain, tumor, arteries and cerebrospinal fluid. In the spine we have used 4 mm spiral CT slices after the administration of intravenous contrast. After image acquisition, the data is transferred from the CT or MR scanner console over a computer network to a SUN workstation in our Surgical Planning Laboratory. The first step in image processing is noise reduction to reduce artifacts and bring out real borders. An anisotropic diffusion filter accomplishes this without blurring important morphologic details [1], Next the medically relevant components contained in the images must be identified. This task has been approached using a supervised tissue classification followed by a statistical multivariate analysis (parzen windows algorithm) to calculate the tissue classifiers [2], A connectivity algorithm is then used to define various subsets of each tissue class, such as intraventricular or subarachnoid cerebrospinal fluid [3], After segmentation, the computer processed image must be translated into a readily appreciated form in order to plan the surgical approach. In our lab, the segmented image is generated in a virtual environment via a dividing cube surface rendering algorithm [4].
Fig. 1. Video registration of 3D reconstruction with patient intraoperatively. This 48 year old white female presented with a recurrent right frontal glioblastoma multiforme and underwent resection with intraoperative guidance using a 3D computer reconstruction of her MRI. A merger of a video image of the patient in the operating room with the 3D reconstruction of the patient s scalp, brain and tumor (the latter is shown in green) is shown. The image registration is based on visual surface matching of the two images. Surgical Planning and Video Registration The surgical planning process begins with a careful study by the surgeon of the 3D anatomy rotated on the computer screen. Based on that study, the operative approach and precise surgical trajectory are selected. The patient is then brought to the surgical planning laboratory or operating room and placed in the planned operative position. A video camera is trained on the patient from the proposed intraoperative surgeon s perspective. The 3D reconstruction of the patient s anatomy is simultaneously displayed on a computer from the same visual perspective. The high resolution NTSC format computer screen image is then converted to standard lower resolution video format using a scan converter (CVS-980 NTSC scan converter, YEM. Okada, Japan). The images from the video camera and the computer scan converter are then combined using a video mixer (Panasonic WJ-AVE5). This video mixer allows one to vary the intensity of the two input images so that 100% of either or 50% of each or any ratio in between can be
displayed. This permits the two images to be superimposed, much like a double exposure in photography. Care must be taken so that neither the patient nor the video camera move once the image registration process is undertaken. This can be facilitated by performing the registration in the operating room after induction of anesthesia or application of the Mayfield head holder in the case of a craniotomy. The selection and maintenance of the same perspective is important because of the parallax effect. If the operation were carried from a different angle the projected borders of the tumor on the skin would be misplaced for that operative approach, though nonetheless valid for the angle at which the registration was performed. The patient s position and the 3D rendering are then adjusted until the two images as displayed on the video mixer s output monitor are identical in terms of scale, position and rotation. This precise positioning can be accomplished by aligning various surface landmarks such as the external auditory canal, lateral canthus, and nasion. In some cases, such as occipital or spinal lesions, capsules placed on the skin prior to scanning to serve as fiducials and facilitate the align ment process. Once the video and 3D computer images of the patient s skin have been aligned, the computer image of the skin is selectively removed leaving the 3D image of the underlying cranial or spinal contents superimposed on the video image of the patient s skin. The surgeon then outlines the borders of the tumor and important cerebral, sulcal or spinal anatomy on the patient s skin using inde lible markers. These markings allow the surgeon to plan an adequate opening with minimal exposure of adjacent structures. In addition to preoperative localization of lesions the above described technique can be applied to intraoperative surgical guidance. By continuing to display the video image of the patient after the drapes are placed and the incision made the operative exposure can be visually correlated with the 3D reconstruction. This modification of the technique can be used for localization of subcortical tumors using sulci as registration landmarks, thereby aiding the surgeon in the selection of a corticectomy. Intraoperative application of this video registration technique could also theoretically facilitate the definition of a tumor s margins, although the limitations of the preoperatively obtained images must be considered in the light of operative maneuvers such as retraction or resection. Outlook for the Future This technique for non-linkage stereotactic surgery offers several advantages over conventionally displayed 3D neuroimages for surgical planning. The technique not only offers the ability to register anatomic space with image space, but also affords one the opportunity to navigate anatomic space intraoperatively. Despite its limitations, this video registration technique offers a simple way to integrate 3D anatomic reconstructions produced with any system into the surgical planning and navigation processes. To further refine this technique in the future, automated registration techniques need to be developed, particularly for those parts of the body with few prominent registration landmarks. One possible approach for this problem is the implantation of fiducials, similar to the above mentioned placement of capsules prior to scanning, but providing greater stability and reproducibility [5]. Another alternative for automated image registration is the use of a topographical laser surface scanner with subsequent computerized matching to the patient s skin with the 3D model s surface. In addition the limitations of preoperatively obtained imaging data must be overcome, either by developing
plastic deformability char acteristics in the reconstructions or utilizing intraoperative imaging modalities to update the 3D models. References 1. Gerig G, Kubler O, Kikinis R, Jolesz FA: Nonlinear anisotropic filtering of MRI data. IEEE Trans Med Imaging 1992;11/2:221-232. 2. Cline HE, Lorensen WE, Kikinis R, Jolesz F: Three-dimensional segmentation of MR images of the head using probability and connectivity. J Comput Assist Tomogr 1990;14:1037-1045. 3. Cline HE, Dumoulin CL, Lorensen WE, Hart HR, Ludke S: 3D reconstruction of the brain from magnetic resonance images using a connectivity algorithm. Magn Reson Imaging 1987;5:345-352. 4. Cline HE, Lorensen WE, Ludke S, Crawford CR, Teeter BC: Two algorithms for the three dimen sional reconstruction of tomograms. Med Phys 1988;15:320-327. 5. Maciunas RJ, Fitzpatrick JM, Galloway RJ, Allen GS: Neurosurgical navigation using implantable fiducial markers. Program of the 43rd Annual Meeting of the Congress of Neurological Surgeons, Park Ridge, 111., 1993, p 87. P. Langham Gleason, MD, Brigham and Women s Hospital, Department of Neurosurgery, 25 Francis Street, Boston, MA 02115 (USA)