3D Ultrasound Reconstruction By The 3 Cons: Michael Golden Khayriyyah Munir Omid Nasser Bigdeli

Transcription

1 3D Ultrasound Reconstruction By The 3 Cons: Michael Golden Khayriyyah Munir Omid Nasser Bigdeli Client Contact: Dr. Joseph McIsaac Hartford Hospital 80 Seymour St. PO Box 5037 Hartford, CT (860)

2 Executive Summary This project is intended to provide physicians with a low cost method to image various regions of a patient s body in three dimensions during surgery. Our client is an anesthesiologist who currently uses ultrasound technology to image the brachial plexus in two dimensions to accurately administer anesthesia prior to surgery. Using the images produced by the ultrasound and the exact spatial location of the ultrasound probe, calculated using complex algorithms, the 2D images can be reconstructed into a 3D image in real-time. This removes the necessity for the physician to imagine what this region in the neck looks like for each individual, and can rely on the 3D image produced using our project. We intend to complete this project with a budget of $1000 and have the information freely available to hospitals nationwide, enabling many physicians to take advantage of this useful tool.

3 1 Introduction 1.1 Background This project is designed to assist anesthesiologists in imaging the brachial plexus region in the neck when administering anesthesia. The client is Dr. Joseph McIsaac, an anesthesiologist at Hartford Hospital. He currently uses an ultrasound probe to image in two dimensions the region of the brachial plexus. It would be more informative if this image was a three dimensional image eliminating the need for the doctor to mentally recreate the patient s nerve. This design would be freely available to the public, allowing physicians nationwide to make use of this tool. 1.2 Purpose of the Project The main purpose of this project is to provide a low cost, unpatented tool for reconstructing 2D ultrasound images into 3D images which can be seen during the administration of anesthesia and throughout the procedure. Anesthesiologists currently have to imagine what the patient looks like subcutaneously (a 3D image), using the 2D ultrasound as a stepping stone. There is currently a product marketed by BrainLab which provides 3D images in real time, but its costs greatly exceeds the budget of many hospitals. This project will achieve the same goal as the product that already exists using two inexpensive web cameras and LabVIEW to acquire and process the image. This LabVIEW program and design of our tool will be made available to the public, allowing physicians nationwide to make use of this tool in a very cost effective manner. 1.3 Previous Work Done by Others Summer Interns In the summer of 2010, two high school juniors from the Avon robotics team worked as interns in Dr. McIsaac s lab and started this project. They did a lot of preliminary research establishing industry contacts, and writing a LabVIEW program which accomplishes many of the necessary tasks individually, but they have not integrated the various aspects together. It is expected that we will use what they have accomplished as a foundation for our project, and tie together what they have already produced. We have purchased the same cameras they used to ensure proper image acquisition with the existing LabVIEW program. The web cameras will image three balls placed on top of the ultrasound probe to recognize its spatial location. The model Taylor and Justin produced will be very similar to what we use, as this is what currently works well with what already exists in industry BrainLAB BrainLAB, a Munich, Germany company which specializes in technology for neurosurgery currently markets VectorVision. In April 1997, BrainLAB received 510(k) clearance from the Food and Drug Administration to market this product. VectorVision uses three reflective balls arranged in an equilateral triangle, and is imaged by two cameras. This is the platform for image-guided surgery, especially neurosurgical and orthopedic procedures. The system does not have wires and can be integrated with any instruments currently used in the

4 operating procedure. Using VectorVision, surgeons can follow the movements of their instruments on the computer screen in real-time during surgical procedures. The basis of BrainLAB s product is the starting point for our project, but we hope to achieve the same goal at a significantly smaller fraction of the cost.

5 2 Project Description 2.1 Objective This project seeks to provide an inexpensive way for physicians to produce an image of any region on a patient s body in three dimensions using existing ultrasound technology which produces two dimensional images. The entire process needs to be completed quickly enough so that the 3D image can be produced while the physician is still imaging, with as little lag time as possible. This will be accomplished using three distinct components, which when integrated together will be fully functional. The three components are image acquisition, comparison of the two web camera images, and 3D reconstruction of 2D images Image Acquisition Two low cost web cameras will be set up at a fixed distance from each other, with the ultrasound probe at their focal point. On top of the ultrasound probe will be three different colored balls, all the same size and an equal distance apart, in a triangular formation. The size, color, and distance of the balls will be dependent on the ability of the cameras to differentiate them from one another, as well as the capability of LabVIEW to recognize the balls as separate objects. It is most desirable to have the balls as small as possible and as close to each other as possible to decrease the amount of crowding of tools that the physician is using. This tracking pyramid will be fixed to the ultrasound probe, so after the location of these balls is calibrated, by recognizing where these balls are, the location of the ultrasound itself will be known Comparison of Images The location of the web cameras with respect to each other is fixed and known. Because they will both be imaging the same thing, we can triangulate the data produced from each camera to determine the exact spatial location of the ultrasound probe. This will enable the physician to be certain that what is produced as the 3D image is actually what is imaged by the ultrasound probe Reconstruction The ultrasound technology produces its images in the form of a DICOM (Digital Imaging and Communications in Medicine) movie. This movie will be disassembled into multiple 2D images, coordinating the acquisition rate of the cameras with the speed of the DICOM. These single 2D images will then be reassembled into a 3D image using existing software, such as MatLab. Once a 3D image exists, it is possible to take slices at any angle to better understand the specifics of the imaged region for each patient.

6 2.2 Methods The methods of this project are broken down further into different categories. The parts of the image acquisition are the setup of the cameras and the creation of the tracking pyramid. The comparison of the two images is very involved, using stereo triangulation and determining the value of K, an experimental parameter. Finally, the reconstruction of the two images will use the actual images of the ultrasound Setup of System We must first create a design that will allow us to guarantee that the two cameras have the exact same optical axes, meaning that the two lenses are pointing in the exact same direction, and that each camera s optical axis is parallel to the ground. Currently, the two cameras are will be connected via 80/20 Inc. s T-slotted aluminum framing, as shown in Figure Figure T-Slotted Aluminum Framing. The cameras will be able to slide left and right along this bar, which is useful during our testing phase. Ideally, only one axis will be mobile at a time, minimizing error. The movement of the cameras in only one plane allows for parameters to be set during testing and allows for proper calibration of the cameras. The Logitech Webcam Pro 9000s used in this project look like the one in Figure The support of the cameras has been removed, exposing several options to properly secure the camera to the support bar. It is only after the cameras are fixed to this bar can several other key steps towards completion be taken.

7 2.2.2 Tracking Pyramid Figure Logitech Pro 9000 Webcam. While the setup of the cameras is being completed, the device attaching to the ultrasound probe can be configured. This device has been named the tracking pyramid for simplicity. This device will be able to track the movement of the ultrasound probe in space while it moves along the patient. Since we want to be able to determine a plane in 3D space (the slice of the ultrasound image), we need to determine the position of at least three non-collinear points in space, because a minimum of three, non-collinear points is needed to define a plane. We must also make sure that LabVIEW will be able to locate the position of these three points using the image recognition software regardless of their orientation in 3D space. An object that has the same general outline regardless of which angle you take a picture of it is a sphere. Therefore, we have decided to use three spheres to determine the position and plane of the ultrasound probe. The orientation of the three spheres, with respect to each other and with respect to the ultrasound probe, is important. Several major factors must be accounted for when determining this. The first concept we must consider is that the spheres should be connected to the probe in such a way that when the doctor moves the probe along the neck of the patient and at different angles, the three spheres do not hinder the doctor s desired movement of the probe. For example, if the tracking pyramid is placed in front of the probe instead of behind it, the spheres will hit the patient s neck and limit the angular movement of the probe. Also, the spheres must come out far enough behind the probe so that the neck of the patient will never hinder the view of the spheres from the cameras. Another factor that must be considered is that the spheres are oriented with respect to the probe in such a manner that the 3D position and plane of the ultrasound probe can be calculated using vector analysis. One orientation that addresses the previosously stated issues is to use an equilateral triangle. An image of our proposed orientation is shown below in Figure , where points A, B and D are the three spheres of the tracking pyramid. The centroid (C) of triangle ABD lies on the same line as the probe. Distance d is the length from the centroid to the tip of the ultrasound probe.

8 2.2.3 Stereo Triangulation Figure Preliminary Design of Tracking Pyramid. The first task is to determine a method to locate the position of an object in 3D space. With the suggestion of Taylor and Justin, the method known as stereo triangulation is going to be used. The concept behind stereo triangulation is that if one knows the distance and orientation of two cameras with respect to each other, then one can calculate the position of an object in 3D space by analyzing the difference in position of the object in the pictures taken by the two different cameras. The simplest form of stereo triangulation occurs when the camera lens have the same optical axes and are at a certain distance, b, apart, as can be seen from Figure When looking at this image, the XZ plane is parallel to the ground, and the Y-axis corresponds to an altitude or height. The optical axes point in the direction that the lens are looking, and in this case the optical axes of both cameras are pointing towards and parallel to the positive Z-axis. X 1 and X 2 correspond to the x-distances from the center vertical axis of the two images.

9 Figure Stereo Triangulation Assuming the reference origin in this 3D space is the center of the left camera lens, noted as L in Figure , then we can calculate the X, Y and Z positions of point P with the following equations: ( ) ( ) ( ) ( ) ( ) In these equations, f is the focal length of the two cameras. The focal length of the Logitech Webcam Pro 9000, through online research, is 3.7 mm. As mentioned earlier, b is the distance between the two cameras. A suitable length between the two cameras has not yet been determined, but is predicted to be between 2 and 4 feet, depending on factors mentioned later Determining the Value of K K is a constant that we added into these equations to account for the arbitrary scale that LabVIEW uses to determine the X and Y distances of the object in the two different images. To determine the value of this constant K, the two cameras will be aligned as shown in Figure An object will be placed at a certain distance with a measured Z value. Using LabVIEW, the images will be analyzed and values for X 1, X 2 and Y 2 will be obtained.

10 Therefore, we will be able to solve for the value of K because we will know all the other values in the above equation for Z Reconstruction of 3D Image In order to reconstruct the 3D image from the series of 2D images, each image will have to be aligned to one another. Several calculations are necessary to follow the probe in 3D space and then put the images together to make a 3D image. The tracking pyramid gives triangle ABD and centroid C to follow. In addition, the vector from the centroid to the probe ( ) is used in reconstruction Triangle ABD and Centroid C Once the two cameras in the proper alignment and determine the value for K in our stereo triangulation equations, the X, Y, and Z coordinates of points A, B, and D, which are the centers of the three spheres, can be determined. Therefore, the three-sphere system can be tracked in 3D space. Point C is the centroid of triangle ABD. Placing the center of the probe scanner, P, along the line perpendicular to the plane defined by ABD at the centroid allows for maximum angular revolution of the probe about the neck and maximum rotation about the line CP without the spheres hitting the neck. The combination of the placement of P along the line perpendicular to ABD at the centroid and the fact that the three spheres are aligned such that ABD is an equilateral triangle allows for unbiased movement of the probe about the neck. Once we calculate the coordinates of points A, B, and D using LabVIEW and our stereo triangulation equations, our next step is to calculate the centroid of the triangle ABD using the following equations: ( ) ( ) ( ) Vector The next step is to determine the vector A vector has two components: direction and magnitude. Because is perpendicular to the plane defined by and, the direction of is defined by either ( ) or ( ) By further inspection of Figure , one can see that the direction of is in fact ( ). In order to calculate this cross product, one must first calculate vectors and. The following equations can be used to calculate and ( ) ( )

11 ( ) ( ) Therefore, ( ) can be defined by the following equation: ( ) ( ) [( )( ) ( )( )] [( )( ) ( )( )] ( ) [( ) ( ) ( )( )] For simplicity of future equations, the X, Y and Z components of ( referred to as L X, L Y and L Z, respectively, such that ) are [( )( ) ( )( )] [( )( ) ( )( )] ( ) [( ) ( ) ( )( )] and ( ) ( ) Now that the direction of has been calculated, is ready to be defined because we already know that it s magnitude is d, as can be seen from Figure Vector can be defined as the unit vector in the direction of ( ) multiplied by the magnitude of, which is d. The unit vector of a vector can be calculated by dividing each component of a vector by the square root of the sum of the square of the components, as one can see below: ( ) Therefore, we can define with the following equation: ( )

12 Center of Probe Scanner (P) Figure illustrates that. Therefore, P is defined with the following equation where all the values on the right side are originally known or have been previously calculated. ( ) ( ) Plane of 2D Images So far the calculations for the 3D position of the center of the ultrasound probe are complete. The next step is to calculate and find the 2D image associated with each 3D position. A plane can be defined by two non-collinear vectors. The two vectors that will define the plane of the 2D ultrasound image, according to Figure , are and Vector is unknown, but vector (a parallel vector) will be used instead because it is already known. Therefore, we can define the plane of the 2D ultrasound image using two known vectors, and. However, we must know the algebraic equation of the 2D ultrasound plane so that we can input it into a program for 3D reconstruction. To define a plane algebraically, one must know a normal vector to the plane and the coordinates of at least one point on the plane. If * + ( ) is the normal vector to a plane, and ( ) is a point on the plane, then the plane can be defined by the following equation: such that. The normal vector by the following equation: Point P is assigned to the point ( ) and will be used as the beginning of the normal vector. Therefore, we can define the 2D ultrasound plane using the above equation for a plane such that a, b, and c are the and values of, respectively, and Square of 2D Image ( ) So far the location of the center of the 2D ultrasound image has been determined, along with the plane of that image. In reality, this plane does not reach to infinity and is assumed to be a square. All the images will have a certain length and width is determined by the specific probe that will be used, which is undetermined at this point. However, once the dimensions of the images are known, the four endpoints of the 2D image can be calculated. Figure shows a representation of the theoretical 2D rectangular image with respect to the previously defined points and vectors.

13 Figure D Ultrasound Image. From this figure, the follow equations are generated to acquire the coordinates of m, n, o and q (the four endpoints of the 2D rectangular image). ( ) ( ( ) ( ( ) ( ( ) ( ) ( ) ) ( ) ) ( ) ) ( )

14 3 Budget The budget for this project is $1000, courtesy of the School of Engineering at the University of Connecticut. The intent is to complete this project under. Although this project will not be patentable, due to patents on similar products, the intent is to release this information to the public for all those who are interested. If we complete this project under budget, then other hospitals will be able to use 2D ultrasound to reconstruct a 3D image for under $1000. This project will require purchasing two inexpensive webcams. These webcams will be able to follow the movement of the ultrasound probe by watching the sphere and rod system designed to mount to the probe. The cameras must be mounted and supported. The parts for this project that must be purchased are the two individual webcams, the support bar for the cameras, as well as the sphere and rod setup for the ultrasound probe. The ultrasound probe used in this project will be one that the hospital already owns. 3.1 Webcams The two webcams used in this project will mirror the ones Dr. McIsaac has at Hartford Hospital. They are Logitech Pro 9000 webcams found on Amazon.com for $60.00 each /20 Support Bar The support bar will be made using the 15 Series slotted aluminum framing designed by 80/20 Inc. This system has slotted aluminum bars that allow for easy construction and manipulation of the system once designed. The necessary components from 80/20 Inc. are the aluminum bar itself (20 section for $130.00), two angle supports (2 at $6.00 each), a clamp block for free degree of angle (2 at $15.00 each), and hardware to connect the two together (20 sets at $0.60 each). The sum total for the components needed from 80/20 Inc. is roughly $ before tax and shipping. 3.3 Spheres and Rods In order to determine what size spheres and rods will be used, an assortment of Styrofoam balls will be used along with an assortment of wooden Popsicle sticks. A craft store such as A.C. Moore will have these items for less than $40.00.

15 4 Conclusion In conclusion, this project will efficiently image the brachial plexus for anesthesia in a cost effective manner. Performing this task with such a low budget is not only challenging, but unique. It costs hospitals over $100,000 to do what this project will accomplish for less than $1000. By using the image acquisition software of LabVIEW, two low cost webcams will follow the movement of the ultrasound probe. This ultrasound probe will have a tracking pyramid attached a three sphere system designed to be watched and detected by the cameras. This tracking method will allow for the software of the project to follow the exact position of the ultrasound probe. Since the position is known of each ultrasound image at a certain time, the series of 2D ultrasound images can be reconstructed into a 3D image. The integration of the 3D tracking will enable a program to properly align 2D images with the proper spacing. When completed, the user will be able to move the object around on a computer screen, and the physician will better understand where to administer the anesthesia. Although this project, as stands, is not marketable due to the patents on VectorVision, is something can be altered to create an original product, the potential for marketing this is nearly limitless. If the average hospital desires such an advanced product such as VectorVision, and are willing to pay such a high price for the technology, imagine how they would react learning similar technology is on the market for under $1000.