PII S (98)00043-X

Transcription

1 PII S (98)00043-X Ultrasound in Med. & Biol., Vol. 24, No. 9, pp , 1998 Copyright 1998 World Federation for Ultrasound in Medicine & Biology Printed in the USA. All rights reserved /98/$ see front matter Review THREE-DIMENSIONAL ULTRASOUND IMAGING THOMAS R. NELSON and DOLORES H. PRETORIUS Divisions of Physics and Ultrasound, 0610, Department of Radiology, University of California San Diego, La Jolla, CA (Received 17 October 1997; in final form 10 March 1998) Abstract The objective of this article is to provide scientists, engineers and clinicians with an up-to date overview on the current state of development in the area of three-dimensional ultrasound (3-DUS) and to serve as a reference for individuals who wish to learn more about 3-DUS imaging. The sections will review the state of the art with respect to 3-DUS imaging, methods of data acquisition, analysis and display approaches. Clinical sections summarize patient research study results to date with discussion of applications by organ system. The basic algorithms and approaches to visualization of 3-D and 4-D ultrasound data are reviewed, including issues related to interactivity and user interfaces. The implications of recent developments for future ultrasound imaging/visualization systems are considered. Ultimately, an improved understanding of ultrasound data offered by 3-DUS may make it easier for primary care physicians to understand complex patient anatomy. Tertiary care physicians specializing in ultrasound can further enhance the quality of patient care by using high-speed networks to review volume ultrasound data at specialization centers. Access to volume data and expertise at specialization centers affords more sophisticated analysis and review, further augmenting patient diagnosis and treatment World Federation for Ultrasound in Medicine & Biology. Key Words: Ultrasound, 3-D, Volume imaging, Visualization. Address correspondence to: Thomas R. Nelson, Ph.D., University of California, San Diego, Division of Physics, 0610, Department of Radiology, La Jolla, CA USA. tnelson@ucsd.edu INTRODUCTION AND OVERVIEW OF 3-D ULTRASOUND IMAGING The inherent flexibility of ultrasound imaging, its moderate cost, and advantages that include real-time imaging, physiologic measurement, use of nonionizing radiation and no known bioeffects, give ultrasound a vital role in the diagnostic process and important advantages compared to magnetic resonance imaging (MRI) and computerized tomography (CT). Ultrasound imaging is in routine use in nearly all hospitals and many physician s offices and clinics. Over the past few years, real-time 2-D ultrasound (2-DUS) imaging has made tremendous progress in obtaining important diagnostic information from patients in a rapid, noninvasive manner and has benefitted from significant improvements in image quality and visualization clarity (Fig. 1). Much of this progress has been derived from utilizing information present in the ultrasound signal, such as backscatter and Doppler shift, to extract useful physiological and tissue structure information. Because ultrasound equipment has benefitted from increasingly sophisticated computer technology, systems integration has ensured better image quality, data acquisition, analysis and display. Advances in technology, particularly high speed computing and storage hardware, have further expanded the possibilities for maximizing patient diagnostic information. Although 2-DUS makes it possible for physicians to make important contributions to patient management, there are occasions when it is difficult to develop a 3-D impression of the patient anatomy. The typical approach to overcome this problem is to scan repeatedly through the region-of interest (ROI) to clarify the exact spatial relationships. Under some conditions, this process can be time-consuming and tedious, particularly with curved structures, when there is a subtle lesion in an organ, when a mass distorts the normal anatomy, and when there are tortuous vessels or structures not commonly seen. Complex cases often make it difficult for even specialists to understand 3-D anatomy based on 2-D images because abnormalities may be difficult to demonstrate with 2-D sonography because of the particular planes that must be imaged to develop the entire 3-D impression. Advanced technology has permitted 3-D imaging 1243

2 1244 Ultrasound in Medicine and Biology Volume 24, Number 9, 1998 Fig. 1. Traditional 2-D ultrasound visualization produces planar tomographic slices through the object of interest; in this case, a fetal head and face seen in profile on the left. Volume visualization methods provide a means of displaying the entire structure in a single image, significantly improving comprehension of spatial relationships as is seen in the fetal face on the right where the eyes, nose and lips are readily visible. methods to be applied to diagnostic ultrasound by presenting the entire volume in a single image. 3-DUS has sparked interest both in the academic community and commercial industry, offering new opportunities in patient visualization (Baba 1997; Fenster and Downey 1996; Kossoff 1995; Pretorius and Nelson 1991; Rankin et al. 1993). Currently, most 3-DUS visualization is performed using graphic workstations to provide interactive performance. Integration of views obtained over a region of a patient with 3-DUS may permit better visualization in some situations and allow for a more accurate diagnosis to be made (Pretorius et al. 1995; Baba et al. 1997; Baba and Jurkovic 1997). In many situations, 3-DUS is important in demonstrating normalcy and reassuring patients. Clinical scanners such as the Voluson 530D (Kretz Technik, Zipf, Austria) are increasingly integrating workstation performance in the scanner to provide volume visualization at the patient bedside. 3-DUS also offers the physician the capability to image the patient after they have left the scanning suite and reevaluate the diagnosis with experts across networks at remote locations. Ultimately, the success of 3-DUS methods will depend on providing performance that equals or exceeds that of 2-D sonography, including real-time capability and interactivity. Although the eventual role for 3-DUS has yet to be determined, there is little doubt that its impact will be broad and substantial. In the near future, 3-DUS will be a routine part of patient diagnosis and management. This paper provides an overview of some Fig. 2. A block diagram of a 3-D ultrasound imaging system. Image data are collected in synchronization with position data and stored in a memory system. Subsequently, position and image data are combined to build a volume that is processed, rendered and displayed on the graphics display. of the basic concepts behind 3-D ultrasound, and describes areas of clinical application. ACQUISITION METHODS Acquisition of a volume data set depends on obtaining echo data from throughout the volume of interest. The general features of a volume sonographic data acquisition system are shown in Fig. 2. Numerous efforts have focused on the development of 3-D imaging techniques using ultrasound s positioning flexibility and data-acquisition speed (Baba et al. 1989; Brinkley et al. 1982; Fenster and Downey 1996; Ganapathy and Kaufman 1992; Geiser et al. 1982a, 1982b; Ghosh et al. 1982; Greenleaf et al. 1993; Kirbach and Whittingham 1994; King et al. 1993; Nelson and Pretorius 1992, 1997; Rankin et al. 1993; Robinson 1972). Much of this effort has focused on integrating transducer position information with the grey-scale sonogram. Multidimensional transducer technology will also benefit volumetric data acquisition and display (Davidsen et al. 1994; Jian-Yu and Greenleaf 1994; Pearson and Pasierski 1991; Shattuck and von Ramm 1982; Smith et al. 1992; Snyder et al. 1986; Turnbull and Foster 1992; von Ramm and Smith 1990; von Ramm et al. 1991, 1994). Most current methods are based on methods similar to conventional 2-D sonography; that is, they acquire a series of image planes from throughout the volume. The challenges, and major differences, come from the specific method used to locate the position of the slice within the volume. The performance requirements for a position-measuring system used to co-locate images are quite exacting, and are becoming more so with improvements in

3 3-D ultrasound imaging T. R. NELSON and D. H. PRETORIUS 1245 ultrasound scanners. Currently, the position-locating system should be able to locate a 2-DUS image to an accuracy of better than 0.5 mm and have an angular accuracy of better than 0.5 with respect to the volume being scanned. Because many clinical scanners provide submillimeter resolution, as frequencies increase and provide an improvement in spatial resolution, further improvements in position location or alternate methods will become essential to produce high-quality volume data. The various approaches employed for 2-DUS image location in a volume will be discussed in following sections. The amount of data stored depends on the duration of the acquisition and the number of images per second that are acquired. For example, a static scan of the liver may only require 5 15 s of data at 10 frames/s, while a cardiac examination may require s of data at 30 frames/s. If color Doppler data is also acquired, additional storage may be required. Systems that image color Doppler data also must project both color and grey-scale data into the volume or separate the color from greyscale data, depending on the visualization task. Image plane acquisition In general, current volume sonographic imaging systems are based on commercially available 1-D or annular transducer arrays whose position is accurately monitored by a position-sensing device. Position data may be obtained from stepping motors in the scan head, a translation or rotation device or a position sensor that may be electromagnetic, acoustic or optical (Detmer et al. 1994; Hernandez et al. 1996a, 1996b; King et al. 1990; Kossoff et al. 1995; Moskalik et al. 1995; Raab et al. 1979). During acquisition, images and position data are stored in a computer for subsequent reprojection into a volume data set. Depending on the type of acquisition used, the acquisition slices may be in the pattern of a wedge, a series of parallel slices, a rotation around a central axis, such as from an endocavitary probe, or in arbitrary orientations (Fig. 3), as are considered in the following paragraphs. Integrated position sensor and transducer arrays One approach to acquiring position data integrates the position-sensing system within the transducer housing. Integrated volume transducers tend to be relatively larger than standard transducers, but they eliminate most of the issues related to external position sensors with respect to calibration and accuracy. Volume transducers acquire a volume as a series of slices at slightly different orientations. After each slice, the transducer plane is moved, usually by some type of servo or stepping motor, to the next location. As a result, the relative angle between slices is exactly known, eliminating distortion in Fig. 3. Different types of volumetric ultrasound data acquisition. In each case, the probe is moved, either by a motor drive or the sonographer with position data generated. Free-hand scanning requires a separate position sensor to encode the location of the transducer. Wedge, linear and rotational scanning generally determine position from the motor drive system used to change the array position. the resultant scan. Integrated volume transducers have been produced for both transabdominal and intracavitary probes. Integrated transducers are generally integrated with the scanner so that volume data are immediately available after the volume is acquired, without subsequent reprojection or processing. Such systems make it possible to proceed directly from acquisition to display and rendering. Integrated volume transducers have a relatively small field-of view (FOV) that, although not posing a significant limitation for imaging small structures such as the heart or fetus, may limit for large organ imaging. External transducer fixation motorized drive devices A second approach to acquiring position data is to mount the conventional transducer to an external device that holds the transducer firmly, offering precise movement during scanning, and can operate in rotational, wedge and linear modes. The device is then held in a fixed position and a motor on the device moves the transducer in a controlled and well-defined fashion to sweep out a volume. The motion may be either linear, rotational or in an arc. Such devices have been used successfully for scanning of the carotids (Pretorius et al. 1992; Downey et al. 1995a), endocavitary applications such as the prostate (Downey and Fenster 1995a), and in imaging the heart using a multiplane transesophageal transducer (Ghosh et al. 1982; McCann et al. 1987;

4 1246 Ultrasound in Medicine and Biology Volume 24, Number 9, 1998 Roelandt et al. 1994a). External devices are capable of high accuracy in locating the position of the transducer relative to other scanned planes. The advantages of positional accuracy and the relative convenience of the mechanical apparatus to hold and move the transducer are balanced, however, by the constraints imposed by a rigid mechanical device that may complicate scanning of some structures, depending on the organ being imaged. Externally attached position sensing devices A third approach to position sensing uses a position sensor device attached to the 2-D scanning transducer that simultaneously determines the position and orientation of the transducer with image plane acquisition. A variety of position-sensing devices have been used to obtain position data, including articulated arms, electromagnetic (EM) field transmitter/receivers, acoustic methods using spark gaps and microphones, optical methods based on light emitting diodes (LEDs) and laser rangefinders. Although each method has advantages, currently electromagnetic position sensing is one of the most widely used techniques because it permits scanning using similar methods to routine 2-DUS studies. EM methods use a small sensor ( 1.0 cm) that can be mounted on the transducer and are accurate, although they are susceptible to distortion of the local EM field by metallic objects that can produce positional measurement inaccuracy (Detmer et al. 1994; Nelson and Pretorius 1992). After the position-sensing system has been attached to the transducer, data acquisition is essentially the same as with conventional real-time scanning. The sonographer makes a sweep over the patient area of interest, acquiring slice data into an external workstation. Typically, systems that use this approach acquire image data via a video digitizer link rather than direct digital data. Eventually, digital data will be acquired with scaling information directly, which will greatly improve the quality of volume data. Position-sensing systems can be used with standard and endocavitary transducers. Recently, there has been some development with position sensors suitable for use with intravascular transducers, although they are still investigational devices (Biosense, Inc., Miami, FL). Two-dimensional transducer arrays A fourth approach to acquiring volume data is to use multidimensional (2-D) arrays. Although the ultimate expectation is that 2-D arrays will replace integrated mechanical scanning transducers and other position-sensing systems, 2-D arrays are still a developing technology. Thus far, 3-DUS systems utilizing 2-D transducers have been used to obtain simultaneous planes from the volume, rather than acquiring the entire volume. Prototype 2-D array transducers have been built that Fig. 4. A block diagram of the process used to build a volume from individual scan planes. Individual pixels are scaled and combined with any offset values from the positioner. Rotation of the scaled pixel to the proper orientation positions the pixel into the proper part of the volume where the voxel at that position is updated by the pixel value. produce satisfactory images (von Ramm, 1991; von Ramm et al. 1994; Pearson and Pasierski 1991; Smith et al. 1991; Turnbull and Foster 1991). A significant issue to be resolved is the data bandwidth required for realtime volume acquisition. Although multiple parallel channels can be used, issues of interference and computing/storage bandwidth will need to be resolved. Fundamental physics limits the overall update rate that pulseecho methods can achieve, consistent with sampling the entire volume. Sparse array approaches have yielded promising results while reducing bandwidth requirements. 2-D array transducers are relatively small (less than 2 cm square) and, as a result, their FOV also may be relatively small. This does not pose a significant limitation for imaging the heart and other small structures, but may be a limitation for large organ imaging. VOLUME CREATION After a series of 2-DUS images are acquired, the volume is created by placing each pixel of each image at the proper location in the volume. The particular location an image is determined from the position data acquired with each image during the 2-DUS scan. Although this is

5 3-D ultrasound imaging T. R. NELSON and D. H. PRETORIUS 1247 not a conceptually difficult process (Fig. 4), the complexity depends on how the slice data are acquired and whether the position of the slice is determined from an integrated position locating system or uses a system external to the 2-DUS scanner. In systems using an integrated transducer positionlocating system, all the required scaling and calibration data are present within the scanner, which yields a highquality volume reconstruction, often as soon as the scan is completed. Systems that use some type of external position-locating system require a calibration process at the initial setup to determine proper data scaling. Also, some additional processing may be required after image acquisition to establish the size and limits of the volume scanned. These systems may or may not be capable of providing a volume immediately after scanning, depending on the performance of the workstation used. The size of the volume scanned depends on the geometry of the system used. For example, integrated transducer/position-locating systems generally provide a FOV determined by the scan parameters, but limited to what the transducer can image from a fixed location. Free-hand systems can cover a large FOV because the volume is based on images acquired during a sweep of the transducer. In theory, there is no limit to the FOV possible, subject to patient motion and data-storage limitations. Integrated mechanical stepping systems may provide a more limited FOV that also may make it difficult to image an entire organ in a single volume. Incorporation of scaling data in all systems permits measurement of distance, area and volume. PHYSIOLOGICAL SYNCHRONIZATION Imaging cardiac dynamics or blood flow as a function of time in the cardiac cycle requires some method to synchronize the data to the appropriate time of the cardiac cycle. Generally, additional data are obtained by using the electrocardiogram (ECG) signal from patient electrodes to provide a trigger signal to synchronize with the data acquisition. Either an image at a specified point in the cardiac cycle can be obtained or the trigger signal can be used to separate all the acquired images into the appropriate part of the cardiac cycle, such as may be found on cardiac 3-DUS systems (Tom Tec Imaging Systems, Boulder, CO) (Fig. 5). It also may be necessary to eliminate ectopic or irregular heart beats from the study analysis. An alternative approach is to perform some type of analysis on the acquired data to extract information about the motion in the image field, such as due to the periodic contraction of the heart. One approach uses temporal Fourier analysis of the cardiac motion to identify the fundamental frequency of the heart motion and then uses the phase information from the Fig. 5. Images of a fetal and adult heart aortic valve in the open and closed positions. ECG gating is used in the adult patient to synchronize motion with the cardiac cycle. Fetal heart synchronization is based on Fourier analysis of the cardiac motion because no ECG is available. The differences in image quality reflect the order of magnitude difference in valve sizes between the adult and fetal hearts. (Adult images courtesy of Tomtec Inc., Boulder CO.) Fourier transform to identify the location of each beat within the acquired images (Nelson 1995, 1996) (Fig. 6A D). This method does not require electrical connection to the patient and utilizes the acquired data to determine the periodic behavior; it is a retrospective method compared to real-time during acquisition. Other approaches use M-mode (Deng et al. 1996) and Doppler (Kwon et al. 1996) methods to determine the heart rate. In general, synchronization of 3-DUS to physiologic triggers is demanding from a data storage, analysis and display point of view and is not widely available at this time on most commercial systems. VOLUME VISUALIZATION ALGORITHMS AND DISPLAY METHODS The promise of 3-D ultrasound lies in making available to the physician the capability to completely visualize and understand patient anatomy and function, even after the patient has left the clinic and even across networks in collaboration with specialists. Although technological advances continue to improve all areas of 3-D ultrasound, one of the most critical areas of development is in the area of volume visualization (Coatrieux et al. 1990; McCormick et al. 1987; Watt and Watt 1992; Wolff 1992a, b, 1993). Interactive visualization methods that clearly show the anatomy of interest and permit interactive review of patient data will be essential for complete clinical acceptance. This section will review approaches to 3-DUS visualization. Although ultrasound volume data are well suited to

6 1248 Ultrasound in Medicine and Biology Volume 24, Number 9, 1998 Fig. 6. A method of determining the cardiac cycle timing using temporal Fourier analysis of the periodic cardiac motion. The upper left image shows a region-of interest located over the fetal heart. The upper right image is a magnitude display of the temporal Fourier transform. The lower left image is a plot of the amplitude of the temporally summed values of the temporal Fourier transform, showing the location of the fundamental cardiac beat frequency. The upper curve is a profile through the transform data. The lower right image is a time slice through the 2-DUS acquisition in the region of the heart showing the heart motion (similar to the previously discussed M-mode), with the Fourier-based cardiac cycle synchronization shown as vertical lines corresponding to end-diastole. visualization (Nelson 1993), the optimal method for physicians to review and interpret patient data has yet to be determined, in part because acoustic data do not represent density, and cannot be classified like CT and MRI because tissues exhibit similar acoustic properties and have relatively lower signal-to noise characteristics than other types of medical image data. Instead, ultrasound signal intensity provides a differential measure of how the acoustic impedance changes as sound passes through tissues. Image intensity increases at the interface between tissues of two different impedance values. It is possible to transform ultrasound volume data into another parameter domain that more clearly differentiates structures in the data, such as using Doppler-shifted data from regions of moving blood displayed as color-coded images merged with grey-scale image data to differentiate vascular from soft tissue structures; however, segmentation of 3-DUS data remains a challenging goal, in general. The quality of the volume data plays an important part in determining the overall quality of ultrasound visualization. One factor affecting image quality is speckle, which, in ultrasound images often exceeds the specular echo intensity. As a result, traditional boundary extraction and segmentation algorithms are difficult to implement. The signal quality of the volume data can be Fig. 7. Images from a liver volume acquisition showing three orthogonal planes through the liver. The first vertical set of images is the original acquisition, with gaps where no data were acquired. The center vertical set of images is the same data after application of a nearest-neighbor augmentation algorithm to estimate the value of the missing data. The right vertical set of images is following application of a 3-point cubic (3 3 3) median filter to the data. The median filter reduces speckle without significantly reducing resolution of the data. improved by image compounding, which occurs when pixels in multiple planes are reprojected through the same voxel. Image compounding reduces speckle intensity compared to 2-D ultrasound and improves segmentation performance (Bashford and von Ramm 1995; Trahey et al. 1986). However, precise image registration is necessary for accurate image compounding, which presumes that 1. The patient does not move, 2. accurate system calibration has occurred, and 3. the velocity of sound is constant. Because these criteria can be difficult to meet in clinical settings, due in part to displacement of patient tissues resulting from transducer motion when scanning across tissues, compounding has had limited success in volume imaging up to this time. 3-DUS visualization algorithms usually treat volume data as an array of voxels. Although adjacent 2-DUS images are generally closely spaced to each other, there may be gaps in the resulting volume due to undersampling, particularly for wedge, rotational or other nonparallel acquisition patterns. As a result, some form of interpolation becomes necessary to fill in any gaps in the volume between acquired images. This step is important because gaps in the volume are visibly distracting, can introduce rendering artifacts and can make interpretation of volume data more difficult. Finally, depending on the visualization requirements, 3-D median or Gaussian filters may be used to improve the signalto noise ratio and reduce speckle prior to application of visualization algorithms (Nelson and Elvins 1993; Pratt

7 3-D ultrasound imaging T. R. NELSON and D. H. PRETORIUS 1249 Fig. 8. Diagram of different methods of obtaining slice data from volume data. The first image shows two single planes extracted from the volume at arbitrary orientations and positions. The middle image shows application of clipping planes to produce a subcube in the volume. The last image shows the application of volume-rendering methods to reveal internal structure of the object. The upper image is inverted minimumintensity projection; the lower image is minimum-intensity projection. 1991; Russ 1992). These steps are shown in Fig. 7. Generally, the same processing is applied to the entire volume uniformly. Although having a regular grid with identical elements has advantages, many medical imaging techniques use a rectilinear grid, where the voxels are axis-aligned rectangular prisms, rather than cubes, due to nonisotropic image resolution. Because the rendering process often needs to resample the volume between grid points, the voxel approach assumes that the area around a grid point has the same value as the grid point. This approach has the advantage of making no assumptions about the behavior of data between grid points (i.e., only known data values are used for generating an image). A further difficulty with 3-DUS visualization is that the volume is generally a dense object, that is, one where the majority of voxels contain nonzero, although not necessarily clinically relevant, information. This makes it difficult to define a clearly identifiable threshold to separate adjacent structures, particularly because different tissues often have similar intensities, with only the interface being visible. Thus far, an important visualization requirement has been that the viewing station be interactive, to assist the operator to optimize viewing. Three-dimensional ultrasound visualization (Nelson and Elvins 1993) generally used three basic approaches (Kaufman et al. 1994; Kaufman 1996; Watt and Watt 1992; Wolff 1992a,b, 1993): slice projection, surfacefitting and volume rendering, discussed in the following sections (Fig. 8). Slice projection Slicing methods are one of several interactive techniques available for physician review of patient data (Fenster and Downey 1996; Nelson and Elvins 1993; Jones and Chen 1995). Extraction of a planar image of Fig. 9. Volume data for a 35-week fetal scan using the Kretz Technique Combison 530 scanner. Three orthogonal slices are extracted simultaneously. The lower right image is a volumerendered image of the fetal face, with a line showing the level of the axial plane in the rendered image. Combination of slices with volume-rendered data offers a good method of identifying the specific location of a slice within a volume. The slices in this volume were reoriented prior to display to put them into a standard anatomic orientation, which also assists comprehension of the data. arbitrary orientation at a particular location in a 3-DUS data set utilizes standard coordinate transformations and rotations. This is the most computationally straightforward approach to review data throughout the volume, requiring minimal processing with isotropic data. Interactive display of planar slices offers the physician retrospective evaluation of anatomy, particularly viewing of arbitrary planes perpendicular to the primary examination axis and other orientations not possible during data acquisition. Multiple slices displayed simultaneously can be particularly valuable to assist in understanding patient anatomy (Fig. 9). Typically, slicing methods are fully interactive, replicating the scanner operational feel. Multiplane displays are often combined with rendered images to assist in localization of slice plane position. Currently, these review approaches are the most common methods used to review 3-DUS data. Surface fitting Surface-fit algorithms typically fit some type of planar surface primitive (e.g., polygons, patches) to values defined during the segmenting process (Ekoule et al. 1991; Kim and Jeong 1996; Levoy 1988, 1990b; Lorensen and Cline 1987; Sakas et al. 1995; Sander and Zucker 1986; Schroeder and Lorensen 1996; Udupa et al. 1991). The surface-fit approaches include contour-con-

8 1250 Ultrasound in Medicine and Biology Volume 24, Number 9, 1998 Fig. 10. A schematic of the process of surface rendering a volume of data for a fetal specimen. The 2-DUS scan is projected into a volume data set, as has already been described. After determination of the appropriate threshold, a polygonal isosurface mesh is generated using standard surface-rendering methods. A low-resolution mesh is shown in the upper right image. Determination of the optimal threshold is an interactive process of adjustment and review of results. The final highresolution polygonal mesh may have greater than 250,000 polygons to preserve the fine detail of the data. A volumerendered ray-traced image from the same data is shown in the bottom center image. The lower right image is a photograph of the original specimen showing the high fidelity possible with both types of rendering. necting, marching cubes, marching tetrahedra, dividing cubes, and others. Once the surface is defined, interactive display is typically faster than volume rendering methods because fewer data points are used; surface-fit methods only traverse the volume once to extract surfaces, compared to reevaluating the entire volume in volume rendering. After extracting the surfaces, rendering hardware and standard rendering methods can be used to quickly render the surface primitives each time the user changes a viewing or lighting parameter. Changing the surface-fit threshold value is time consuming because it requires that all of the voxels be revisited to extract a new set of surface primitives. The general approach is shown in Fig. 10 (Nelson 1992). Surface-fit methods can suffer from occasional false positive and negative surface pieces, and incorrect handling of small features and branches in the data (see Fig. 10; the low resolution mesh). Artifacts can be a serious concern in medicine because they could be incorrectly interpreted by physicians as features in the data. Misinterpretations of artifacts may be avoided by viewing rendered data in combination with planar data, rather than in isolation. Although surface fitting provides a good means for visualizing spatial relationships for the entire volume in a readily comprehended manner, small features may be poorly visualized unless a significant number ( 500 K) of polygons are used (Nelson and Fig. 11. Images from a rapid surface-imaging system showing excellent fetal surface details. This system depends on the presence of sufficient amounts of amniotic fluid to produce clear views of fetal anatomy. (Images courtesy of K. Baba.) Elvins 1993; Sakas et al. 1995). Large numbers of polygons degrade display performance to the point where other volume rendering methods become superior. For these reasons, surface fitting methods are not widely used in 3-DUS visualization. Data classification and segmentation Data classification is the most difficult function in volume visualization. Data classification generally involves choosing a threshold for use with a surface-fitting algorithm or choosing color (brightness) and opacity (light attenuation) values to go with each possible range of data values for use with volume-rendering algorithms. Accurate and automatic segmentation of ultrasound data, essential for high-quality surface fitting, is a particularly difficult problem because most tissues in the data volume have similar signal characteristics that make it difficult to distinguish one tissue or organ from another based on signal intensity alone (Chen et al. 1995; Bovik 1988; Bashford and von Ramm 1995, 1996; Bamber et al. 1992; Cootes et al. 1994; Fine et al. 1991; Richard and Keen 1996; Sakas et al. 1994; Sakas and Walter 1995). Furthermore, because acoustic signal intensities vary with depth and tissue overburden, some form of image equalization is necessary to obtain a uniform image field, particularly in areas subject to acoustic shadowing. Segmentation based on signal void greatly simplifies extraction of structural features. Recent work by Baba et al. (1996, 1997) has demonstrated fetal surfaces in near real-time using simple thresholding to identify the fetal surface in the amniotic fluid (Fig. 11). Blood flow data, whether from Doppler, contrast or signal void, may be reprojected and processed in concert with or separately from 3-D tissue data (Ashton et al. 1996;

9 3-D ultrasound imaging T. R. NELSON and D. H. PRETORIUS 1251 Fig. 12. Two approaches to segmentation of vessels. The left image uses the inverted minimum-intensity projection of a liver data set to clearly show the 3-D structure of the vessels. Minimum-intensity projection is an important method for presenting echo-poor structures embedded in tissue, such as vessels, cysts, etc., requiring minimal parameter adjustment, and can be performed very rapidly. The right image segments the vessels from the grey-scale data based on the power Doppler signal. Volume rendering of the flow signal produces a clear picture of the continuity of the vessel flows within the liver. Fig. 13. Volume data from a velocity Doppler study of the carotid artery. The left image is a surface-rendered image based on a defined threshold for the color velocity data. The right image is a volume-rendered, ray-cast image of the same data. Blankenhorn et al. 1983; Bruining et al. 1995; Carson et al. 1992; Cavaye et al. 1991; Downey et al. 1995; Ehricke et al. 1994; Ferrara et al. 1996; Guo et al. 1995; Guo and Fenster 1996; Hashimoto et al. 1995; Kitney et al. 1989; Klein et al. 1992; Picot et al. 1993; Pretorius et al. 1992; Riccabona et al. 1996c; Ritchie et al. 1996; Selzer et al. 1989). Figure 12 shows a ray-cast image of liver vessels using a simple segmentation based on the power Doppler signal and signal void to visualize liver vessels. During rendering, slight changes in opacity values often have a large impact on the rendered image quality. Segmentation of blood-flow data is relatively straightforward compared to tissue and organ segmentation, where interfaces are less visible. Ongoing work is showing promising results in segmentation of the prostate (Chen et al. 1995; Strasser et al. 1996). Interactive performance greatly assists segmentation by providing feedback to optimize the image-rendering parameters. Also, using colors that approximate the color of the tissue being studied can heighten both the realism and cognition process, such as when there are stereotypical colors associated with a certain object, such as red for arteries and blue for veins (Fig. 13). Volume rendering Volume-rendering methods map voxels directly onto the screen without using geometric primitives (Cabral et al. 1995; Cohen et al. 1992; Kajiya and Von Herzen 1984; Levoy 1990a; Sabella 1988; Sarti et al. 1993; Steen and Olstad 1994; Watt and Watt 1992), but require that the entire data set be sampled each time an image is rendered or rerendered. In some situations, a low resolution pass is sometimes used to quickly create low-quality images for optimization of viewing position or parameter setting, with a high-resolution image produced immediately after values have stopped changing. The most often used volume visualization algorithm for the production of high-quality images is ray-casting. Ray-casting uses the color and opacity of each voxel along a ray passing through a volume (Drebin 1988). The trajectory of the ray is defined by the relative viewing orientation of the observer and the volume. Along a particular ray, the pixel intensity (P(r)) of the emerging ray is determined by all of the previous voxels lying along the ray path. Depending on the blending or evaluation function, the resultant value (C out ) of individual voxels (V(k)) can reflect surface or sub-surface features within the volume, depending on transparency/ opacity ( (k)) values (Fig. 14). C out C in 1 k c k k, (1) where (k) is the opacity value based on a mapping function relating opacity to the voxel brightness and/or color. When (k) 0, the voxel is transparent and when (k) 1, the voxel is opaque. c(k) is a shade value based on the local gradient or some other modifying parameter for the voxel. The specific rendering result depends on the choice of the mapping functions for (k) and c(k). The opacities, shades and colors encountered along the ray are blended to find the opacity and color of the pixel (P(r)). Each element along a ray contributes intensity and color to the final pixel as (Watt 1992): K P r k 0 K c r, k r, k 1 r, i, i k 1

10 1252 Ultrasound in Medicine and Biology Volume 24, Number 9, 1998 from the volume to highlight interfaces. Many volumerendering and surface-fit algorithms use some type of gradient shading. One method for finding the gradient is to use the Mark Hildreth operator, which is the convolution of a Gaussian with a Laplacian: V x, y, z 2 V x, y, z **G x, y, z, where: 2 V x, y, z 2 V x 2 2 V y 2 2 V 2. z Fig. 14. A diagram showing the basic concepts of ray-casting. In ray-casting, the voxel intensity is propagated forward toward the viewing plane along each ray from back to front. Each voxel (V(k)) contributes to the final image intensity based on shading (c(k)) and transparency ( (k)) values. Prototype examples of the c(k) and (k) mapping functions are shown in the small graphs. Once ( (k)) 1, full opacity has been reached and further passage of the ray into the volume does not contribute to the final image. where: r, k is the k th voxel along the r th ray and c(r, 0) c background and (r, 0) 1. No shadows or reflections are generated in raycasting and these useful visual cues must be added to optimize visual presentation (Nelson 1993). Maximum- and minimum-intensity projection (MIP) methods are one form of ray casting where only the maximum (minimum) voxel value is retained as the ray traverses the data volume. These techniques are quite simple to implement and provide good quality results for many applications (Nelson and Pretorius 1992, 1995; Pretorius and Nelson 1994). Although ray-casting is CPU-intensive, the images show the entire data set, depending on opacity and intensity values, not just a collection of thin surfaces as in surface fitting. Ray-casting can be parallelized at the pixel level because rays from all of the pixels in the image plane can be cast independently. Viewing and depth cue enhancement Most medical visualization uses orthographic views so that physicians will not see patient data warped by the perspective transformation. Under such conditions, other cues to structural features and depth, such as depth-fog and depth-brightness attenuation, shading based on gradients, and animation, can enhance comprehension of data features. Subtle surface features in ultrasound volume data can be enhanced by using gradient information The gradient can be used to approximate the normal to an imaginary surface passing through the voxel location. Most standard graphics shading models can be applied in shading elements after an approximate normal is known. Ambient, diffuse, and sometimes specular lighting components are used in the volume-visualization shading process. As in other computer graphics applications, shading is an important factor in creating understandable volume images of anatomy. Exponential depth shading is a useful technique to enhance the perspective of depth in a volume rendered image. In this approach, often used with MIP techniques, the intensity of the voxel is a function of the voxel depth in the volume is reduced by an amount based on an exponential function (Nelson and Pretorius 1995) (Fig. 15). K without depth shading: P r max k 0 V r, k K with depth shading: P r max (V(r, k)e (k) ), k 0 where: (k) is the attenuation factor at the k th location along the r th ray. Depth shading reduces visual aliasing during rotation when near and far points of the volume cross. Figures 16 (Pretorius et al. 1995), 17 (Pretorius et al. 1998), and 18 demonstrate some examples of volume rendered data sets using depth shading and different types of rendering algorithms. Animation Animation sequences, such as rotation and gated cine-loop review, greatly assist volume visualization. Without the animation display of rendered volumetric images offered by real-time processing or precalculation, the physician often has a difficult time extracting 3-D information from 2-D displays. Motion can be viewed at normal, accelerated or reduced speed to enhance com-

11 3-D ultrasound imaging T. R. NELSON and D. H. PRETORIUS 1253 Fig. 15. Images showing maximum-intensity projection comparing the effect of depth shading. All images are produced from the same data set. The two images on the left show front (0 ) and back (180 ) views of the 22-week fetal spine. Structures nearest the viewer are emphasized through use of an exponential weighting function. In comparison, the two righthand images are produced without a weighting function. The image appears the same from both orientations (0 and 180 ), which results in visual aliasing during rotating when the rotating object appears to change direction (or orientation) spontaneously. Objects close to the back appear to be moving in the opposite direction from that of objects close to the front. With both near and far objects having the same intensity, the viewer is unable to determine the true position and location. Depth shading eliminates this difficulty. prehension. Furthermore, analysis of dynamic function can increase the diagnostic value of many studies (Schwartz et al. 1994) (Fig. 19). Interactive volume-rendering methods make it possible to follow structural curvature that cannot be viewed in any planar orientation. Rotation of anatomy to a standard presentation facilitates identification of subtle anatomic landmarks by clarifying the relative location of the overlapping structures. Interactive stereo viewing incorporating motion cues makes it possible to separate structures, identify the continuity of structures and enhance anatomic visualization (Adelson and Hansen 1995). Volume editing tools Physicians often need to visualize a 3-D structure located within a data volume that may require editing to remove undesirable echoes and see the important anatomy. Segmentation of ultrasound data is possible when there are very different acoustic interfaces, such as extracting vessels imaged with Doppler methods. Unfortunately, when adjacent organs have similar acoustic properties, segmentation of ultrasound data can be difficult. The Voluson 530D (Kretz Technik, Zipf, Austria) system provides editing boxes that can be positioned around the object of interest to exclude obscuring structures. The boxes can be combined with a threshold control to further eliminate unwanted echoes. Identifying the boundary of an organ or structure in 3-D ultrasound data for morphometric analysis, however, also may benefit from the use of stereo displays and 3-D input devices. Fig. 16. Images of a fetal face showing the effect of different types of rendering. Several different methods are used to optimize display of facial details: maximum-intensity projection; x-ray projection; surface rendering; light rendering. The upper left images show maximum intensity and x-ray projection images demonstrating internal structures such as bones. X-ray projection sums all the voxels along a given ray and is of limited value for most ultrasound data. Surface-rendering methods are based on selection of a threshold/gradient to highlight a specific acoustic feature. The next images show a gradation from surface-rendered image using a combination thresholdgradient method that clearly shows surfaces, but sometimes produces a stone-hard surface, to light that demonstrates the primary surface features with a semitransparent surface painted by the original grey values of the volume, preserving high detail and including original grey-scale values on the 3-D surface. Another means of removing unwanted echoes is to use interactive editing tools that function in a volumerendered or virtual-reality environment. In this situation, the physician uses an electronic scalpel to dissect away tissues that are not part of the organ of interest (Nelson et al. 1995a) (Fig. 20). The dissection process is greatly assisted by using volume-rendered data, compared to single-plane data, particularly if some type of stereoscopic display can be used to create the impression of working in a 3-D environment. Under these conditions, it is possible to rapidly remove nonimportant tissue signals and isolate the structure of interest. Interactive, userdirected erosion techniques, where the user removes layers of 3-D data with the resulting volume immediately rendered so that the user can analyze and adjust the amount of data removed, can be helpful in improving interactivity. If the user has only affected a small area of the volume, then the image is adaptively rendered to save CPU time. Allowing users to separate structures of interest from surrounding diagnostically less important data enhances previously difficult-to see anatomical features.

12 1254 Ultrasound in Medicine and Biology Volume 24, Number 9, D Ultrasound Volume Rendered Fig. 17. Volume rendering of power Doppler data of the placenta and umbilical cord. The left image is a single slice from the 2-DUS acquisition. The right image is a volume-rendered image of the vascular anatomy. The grey-scale signal has been removed. The rendering is a modified maximum-intensity method with depth coding. QUANTITATIVE ANALYSIS METHODS An important part of patient imaging is to provide data of sufficient quality to measure the length, area or volume of organs and follow temporal changes. Although visual assessment is valuable, quantitative data provide a more accurate basis for decision making and comparison against previous studies or reference data bases. 3-DUS data are ideally suited to volume measurement and results have been reported in a number of areas (Ariet et al. 1984; Elliott et al. 1996; Favre et al. 1993, 1995; Gilja et al. 1994, 1995; Hodges et al. 1994; Hughes et al. 1996; King et al. 1991; Moritz et al. 1980, 1993; Nelson and Pretorius 1993a; Riccabona et al. 1995, 1996b; Siu et al. 1993; Sivan et al. 1997; Terris and 2D Ultrasound Volume Rendered Fig. 18. Volume rendering of power Doppler data of a polycystic kidney. The left image is a single slice from the 2-DUS acquisition; the signal voids represent the cysts in the kidney. The right image is a volume-rendered image of the vascular anatomy and the cysts. The cyst signal void is rendered using a technique similar to that for the vessels shown in Fig. 13. The rendering is a modified maximum-intensity method with depth coding. Note that the relative position of the cysts and vessels is clearly shown in the volume-rendered image. Fig. 19. Imaging of the fetal heart. Volume-rendered images of the cardiac chambers and vessels. The signal void in the original acquisition has been extracted to show only the blood signal. Chamber and vessel rendering uses a modified maximum-intensity method with depth coding. The upper panel shows volume-rendered images from several orientations. The lower panel contains a series of volume-rendered images at different points in the cardiac cycle. Note the tricuspid valve (TV), which is clearly seen in each image. The TV is closed in the first left-hand image and proceeds to open during ventricular diastole in the next few frames, as blood flows from the atria into the ventricle. Toward the end, the TV is closed once again as blood is ejected into the pulmonary artery (PA). Interactive display of cardiac dynamics greatly assists comprehension of cardiac anatomy and function. Stamey 1991; Weiss et al. 1983). Because ultrasound images are tomographic in nature, ready measurement of length within the image has always been possible. The overall accuracy of the measurement depends on the frequency of the transducer, field-of view and the limiting resolution of the scanner. Accuracy is generally possible to a few percent with measurement precision of 0.1 mm (Riccabona et al. 1996b). Volume data permit distance measurements across different slices within the volume, improving overall accuracy by eliminating foreshortening effects found with 2-DUS that occur if the structure does not lie in the plane of the image. Area measurement is also easily accomplished using a free-form closed curve or polygon (e.g., an ellipse whose shape, size and orientation are easily modified) method that permits the operator to directly measure the area of an organ or vessel without difficulty. Historically, multiple regions from different planes could be used to estimate volumes; however, direct measurement of volumes is not possible with 2-D methods because the scanner typically does not provide a means of connecting adjacent tomographic slices. There are algorithms based on assumptions of ellipsoidal geometry that are used to estimate organ volume (Brinkley et al. 1982, 1984; Chan 1993). Most volume measurements made using conventional 2-D ultrasound methods generally are accu-

13 3-D ultrasound imaging T. R. NELSON and D. H. PRETORIUS 1255 Fig. 20. Volume editing of an 18-week fetal study with an electronic scapel. The original volume data contain the uterus, the fetus and the amniotic fluid. Volume-rendering methods cannot readily differentiate what is important and what is superfluous. Removal of unimportant objects is readily accomplished by editing either the individual slices or the volume directly. The electronic scalpel is designed to remove a few voxels between the viewer and the object of interest by direct application to the volume-rendered data. Alternatively, a single slice can be edited, as shown in the upper left-hand image slice with the contour drawn around the fetus. After the contour has been defined, that contour is applied to the entire volume in that orientation and, similarly, for the two other orthogonal slices. Direct editing of the volume also is used to refine the extraction of the fetus (or object of interest). Using volume-editing methods makes it possible to completely and accurately extract the fetus from the surrounding tissue in less than 30 s compared to 5 to 10 min for conventional single-slice editing. Furthermore, volume-editing tools make it possible to correct any errors directly on the volume. Editing is conducted from any orientation permitting rapid optimization of feature extraction. Fig. 21. Measurements of organ volume have historically utilized approximations based on ellipsoidal or spherical geometries. As such, volume measurements for irregular organs often have errors of 20% or larger. Utilizing slice masking under operator control or automatic border finding algorithms, each plane of a volume data set is masked and the regions integrated to yield the volume measurement. 3-DUS volume measurements can have errors less than 5%. rate to within 5% if the organs are regularly shaped (e.g., spherical), but are only accurate to within 20% when irregularly shaped (Riccabona et al. 1996b). Volume measurement is accomplished by masking, using either individual plane masks or a volume-interactive tool to limit the volume region-of interest to only the object of interest. After the object is masked, the voxels are summed and the matrix voxel scaling factors applied to determine the volume (Fig. 21). This approach permits measurement of distance and volume for regular, irregular and disconnected objects with an accuracy of better than 5% for regular and irregular objects and in vivo organs, relatively independently of the object size over several orders of magnitude (Riccabona 1995, 1996). For small objects, in which machine resolution is a significant factor, errors will increase accordingly. The absolute error will vary with the volume of the object and the size of the voxel with the result that a larger fieldof view results in larger voxels for the same matrix size with a corresponding increase in the absolute error (Riccabona et al. 1996b). In general, the improved measurement accuracy afforded by volume sonographic methods makes possible accurate quantitative measurement of heart chambers, vessel dimensions and organ volumes. In vitro volume measurement studies have demonstrated accurate volume measurements (Gilja et al. 1994; Hodges et al. 1994; Riccabona et al. 1995, 1996b, c). In vivo volume measurements also show good results in a variety of organ systems, including the thyroid (Chanoine et al. 1991). Brunner et al. (1995) showed an increased accuracy in follicular volume measurements. Chang et al. (1997) have shown good volume measurement results for fetal heart volumes across a range of gestational ages. Liess et al. (1994) demonstrated volume measurement errors of approximately 6% in liver lesions. Riccabona (1996b) showed good accuracy between voided urine volumes and measured bladder volumes pre-void. Prostate volume measurements also have been shown to be accurate (Elliott et al. 1996; Sehgal et al. 1994). Hughes et al. (1996) measured a variety of organ volumes with an error of 2 6%. VOLUME DATA USER INTERFACES Clinical user acceptance of 3-DUS capability will depend, to a large extent, on the type of user interface provided. Many current techniques for user-data interaction are often awkward, nonintuitive and unsuited for use in a clinical setting, requiring significant effort and commitment to learn. To facilitate usage in a clinical environment, it is important to have an intuitive, easy-to use interface with a rapid learning curve that facilitates phy-

14 1256 Ultrasound in Medicine and Biology Volume 24, Number 9, 1998 sician operation of a system. As a result, human factor ergonomics considerations will become increasingly important with 3-DUS equipment to facilitate rapid learning and intuitive interaction with the scanner and volume data by the operator. The key factors critical to rapid learning and ready clinical user acceptance in 3-DUS are: 1. A user interface with a clearly labelled set of functions appropriate to the task at hand; 2. rapid response to user commands permitting interactive review, adjustment and manipulation of volume data; 3. intuitive interactivity devices (e.g., trackballs, touch screens); and 4. functions that permit rapid extraction of clinically relevant data (i.e., measurements, segmentation, visualization). This section is intended to discuss some of these important factors. Task-based user interface functions A key area for 3-DUS acceptance is to use a menu system that does not provide too many choices to the operator, but only those choices relevant to the particular task at hand. Current ultrasound scanner user interfaces often provide all the possible choices simultaneously, even if not relevant to the task, requiring the operator to remember which options (or key combinations) are required to utilize a particular feature. An alternative approach uses simplified controls with reconfigurable option panels to match the function choices to the task. The user interfaces on the Elegra (Siemens Ultrasound, Issaquah, WA), the Sonos2500 (Hewlett-Packard, Newton, MA) and the Voluson 530D (Kretz Technik, Zipf, Austria) are examples of simple-to-learn interfaces that incorporate reconfigurable displays coupled with touch screens. Rapid response to user commands After the desired operation is identified and selected, the system must respond quickly to permit interactive review, adjustment and manipulation of volume data. Interactive review of volume data is essential to clinical use, with high-performance computing systems becoming more cost-effective, greatly enhancing interactivity. In addition to orienting volume data to the optimal position, it also is important to adjust rendering and display parameters to emphasize the structure of interest. This may include changing from a maximum intensity projection to a ray cast projection and adjusting the window/level controls. In other situations, it may be necessary to segment the volume data to visualize the vessels that might require thresholding or adjusting signal levels while viewing the images. In other cases, measurement of length, area and volume might require removing part of the volume interactively while reviewing the remaining structure to verify the accuracy of the removal. In each of these situations, the system must rapidly provide feedback to the operator to assist in optimizing the process. Segmentation is a particularly important part of the process. Segmentation is required to isolate structures of interest from surrounding tissues for visualization or measurement. Because this is often a tedious and time-consuming process, tools that enhance this process are essential. These tools may include thresholding, texture analysis or region growing. They also might include 3-D display of the result using stereoscopic displays to permit depth information to assist in volume operations, such as the electronic scalpel discussed earlier. Certainly, alignment of two volume data sets (such as soft tissue and blood flow) from the same patient to create a hybrid image may be more readily performed with stereo displays and fast alignment algorithms. Intuitive interactivity devices Review of patient data must include the flexibility to rotate, scale and view objects from perspectives that optimize visualization of the anatomy of interest. Incorporation of touch screens, knobs, trackballs and 3-D positioning systems simplifies operator control of operations, especially if the response is immediate. Touch screen systems do not rely on keyboard or mouse input devices, are intuitive and provide physicians with an easy-to use and learn interface to the computer system that can greatly improve interactivity. With this approach, learning time to use the system can be significantly reduced. Stereo displays can be used by physicians to integrate displays and interactive devices so that they do not have to use a computer keyboard; rather, they interact with data volumes in 3-D space via a stereo display, in some cases, using head-mounted display systems. Rapid extraction of clinically relevant data One of the most important factors affecting the evaluation of 3-DUS data in a clinically useful system is to provide clinicians with an interactive means of reviewing patient data and extracting the diagnostically important information from the study, as part of observing and evaluating patient anatomy (Brady et al. 1995; Kaufman et al. 1993; Robb and Barillot 1988, 1989). User interfaces should incorporate interactive review of data to permit optimization of viewing orientation and data presentation (Fuchs et al. 1989). Volume-data display that incorporates both slices through the volume and rendered images from the volume similar to that shown in Fig. 9 use the volume-rendered image to provide important navigational landmarks to identify the location of the slices. When coupled with use of a standard viewing orientation, localization on the critical anatomic feature is greatly facilitated. Volume-rendering methods

15 3-D ultrasound imaging T. R. NELSON and D. H. PRETORIUS 1257 can enhance evaluation of volume data in a readily comprehensible, unambiguous manner by presenting information from the entire volume to the physician in a single view. CLINICAL APPLICATIONS Early clinical experience with 3-DUS suffered from the limitations of available technology. Nonetheless, early work clearly demonstrated the potential of using ultrasound for volume imaging (Brinkley et al. 1982; Blankenhorn et al. 1983; Greenleaf 1982; Matsumoto et al. 1981; Robinson 1972). The explosive growth of 3-DUS over the past few years is readily appreciated by reviewing the number of published articles, which reflects the recent availability of improved clinical 3-DUS imaging equipment. Although the areas of clinical applicability for 3-DUS methods continue to expand, the earliest work was in the field of cardiology, with more recent efforts focused on obstetric, vascular and gynecological areas. Identification of critical clinical areas in which 3-DUS contributes to patient management is a central focus of much research in 3-DUS occurring world-wide. The following sections will review areas of clinical application under investigation that show promise. Obstetrics The fetus is a challenging and difficult object to evaluate (Crane et al. 1994), partially due to its size and movement, and one in which ultrasound and, more recently, 3-DUS have shown significant progress. Early work demonstrated the feasibility of 3-D fetal imaging (Baba et al. 1989; Kuo et al. 1992) and, as equipment has become more sophisticated, the quality and detail possible with 3-DUS methods have become impressive. Initial reports have shown that 3-DUS can improve diagnostic assessment (Merz et al. 1995a, b; Hamper et al. 1994; Pretorius et al. 1995a). Early reports focused on demonstrating fetal anatomy, such as the fetal surface (Nelson and Pretorius 1992) and the fetal spine (Nelson and Pretorius 1995; Steiner et al. 1995). The largest series of congenital anomalies studied with both 2-DUS and 3-DUS was performed by Merz et al. (1995a) who studied 204 patients with anomalies and found that, compared to 2-DUS, 3-DUS was advantageous in demonstrating fetal defects in 62%, was equivalent in 36% and was disadvantageous in 2%. 3-DUS was more advantageous than 2-DUS for the following reasons: 1. Orthogonal planes could be viewed simultaneously; 2. the volume could be stored and reviewed later by the same examiner or another person without image degradation; 3. rendered and transparent images could be displayed; and 4. scanning time was reduced. One additional feature that is of significance to obstetrical scanning is that data can be rotated into a standard anatomic orientation so that fetal structures are easily recognized (e.g., the face can be viewed looking directly at the examiner rather than on the side) (Pretorius and Nelson 1995; Merz 1995a). In many cases, the value of 3-DUS methods is to demonstrate normal development and reassure the parents that all is well. Although various display algorithms can be used to accentuate certain fetal features, the most impressive results have been in the area of visualizing the normal fetal face and congenital facial anomalies (Devonald et al. 1995; Lee et al. 1995; Merz et al. 1995, 1997; Pretorius et al. 1995; Pretorius and Nelson 1995b; Raga et al. 1996; VanWymersch et al. 1996; Warren et al. 1995) (Fig. 16). Rendering of the fetal face provides an easily recognized view of the face to laypersons, as well as physicians, facilitating understanding of developmental anomalies such as cleft lip/palate. Planar imaging is also very important in evaluating the face, in identifying the exact plane for palate (Pretorius and Nelson 1995a) and in evaluating the facial profile (Merz et al. 1997). To date, most anomalies detected on 3-DUS that were not seen on 2-DUS imaging involve the fetal face (e.g., cleft lip, micrognathia, flat profile). Clinicians often find that a combination of rendered images and slices enhances their ability to localize subtle anatomic details (Mueller et al. 1996) (Fig. 9). Some structures, such as the spine, can be best viewed using rendering methods such as maximum-intensity projection (MIP) to demonstrate the entire structure, even though it is curved and does not lie in a single plane (Nelson et al. 1996; Johnson et al. 1997; Riccabona et al. 1996a). 3-DUS is valuable in assessing the level of neural tube involvement because orthogonal planes can be viewed simultaneously and, thus, an individual vertebra can be identified on longitudinal and transverse planes at the same time. Johnson et al. (1997) showed that 3-DUS was more accurate than 2-DUS in identifying the level of neural tube involvement in 3 of 5 cases. Other central-nervous system anomalies in which 3-DUS has added additional information include encephaloceles and anenecephaly. As equipment has improved, the specificity of the examination has increased to include visualization of other fetal anomalies, such as stomach development (Yoshizato et al. 1995; Nagata et al. 1994), club feet (Merz et al. 1995), genitalia and skeletal dysplasias (Steiner et al. 1994a). Volume estimates using 3-DUS, including fetal weight using limb circumference (Favre et al. 1993, 1995), fetal heart volumes (Chang et al. 1997), and gestational sac volume (Steiner 1994), are further improving the type of diagnostic information available. Evaluation of the fetal heart is an exciting area of

16 1258 Ultrasound in Medicine and Biology Volume 24, Number 9, 1998 research, although most work is still occurring in research laboratories due to the more complex data acquisition needed to provide dynamic information (Nelson et al. 1996) that requires cardiac gating. Static imaging of the fetal heart shows promise in evaluating the great vessels and some nonmoving cardiac structures (Deng et al. 1996; Zosmer et al. 1996). Imaging of the embryo is an important area of research that provides much opportunity for evaluation of growth and development. Although initial work has been encouraging (Lees 1992), recent equipment-performance improvements have greatly expanded our ability to obtain high-quality images of the developing embryo (Steiner 1994b) (Fig. 22). With improved imaging capability, interior structures such as the brain cavities can be imaged (Blaas et al. 1995). In the future, this area may permit us to study growth and development in new and exciting ways. Gynecology Applications of 3-DUS to gynecology also are increasing, with one of the most important contributions of 3-DUS to gynecology to date being the characterization of uterine anomalies (Athanasiou et al. 1997; Balen et al. 1993; Feichtinger 1993; Jurkovic et al. 1995; Steiner et al. 1994b; Raga et al. 1996; Weinraub et al. 1996). Congenital anomalies of the uterus can be displayed with 3-DUS by viewing the transverse plane through the uterus, which cannot be done with conventional 2-DUS, allowing for differentiation between anomalies, such as septate uterus from bicornuate uterus (Fig. 23). This has important clinical implications regarding surgery for patients with infertility. Another encouraging report suggests that 3-DUS methods provide more reproducible and accurate measurements of endometrial volume (Kyei-Menash et al. 1996) compared to 2-DUS, allowing for more accurate differentiation of cancer compared to benign diseases, such as polyps and hyperplasia (Gruboeck et al. 1996). The ability to obtain C-plane images through the uterus also permits direct visualization of intrauterine devices to verify placement (Bonilla-Musoles et al. 1996). Saline contrast is further expanding the role of 3-DUS in providing results comparable to hystersalpinography and 2-DUS methods (Ayida et al. 1996). Several investigators have shown that sonohysterography combined with 3-D methods can assist in evaluating uterine fibroids, polyps and adhesions (Balen et al. 1993; Weinraub et al. 1996). Adnexal applications of 3-DUS are also being identified. Bonilla-Musoles et al. (1995) suggest that ovarian masses can be readily imaged with 3-DUS improving diagnostic results by evaluating papillary projections, characterizing cystic walls, identifying the extent of capsular infiltrations of tumors and calculating ovarian and Fig. 22. Images of embryos [9.7-week (Dr. Pretorius), 12-week (Dr. Bonilla-Musolles) and 10.5-week (Dr. Benoit)] demonstrating the degree of anatomic detail possible with 3-DUS. Early development is an area that could gain significantly from 3-DUS techniques. (Images courtesy of Drs. Benoit and Bonilla-Musolles.) follicular volume (Brunner et al. 1995). 3-DUS is also useful in evaluating infertility patients in assessing ovarian enlargement from multiple follicles and lutein cyst (Feichtinger 1993), accurately measuring follicles and guiding interventional procedures, such as follicle aspiration (Kyei-Menash et al. 1996), and evaluating for tubal patency with contrast agents (Steiner et al. 1994b). Fig. 23. C-plane view of the uterus showing excellent anatomic depiction. Both a normal and a septate uterus are shown. This view is not possible with surface or endo-vaginal scanning methods and demonstrates the value of 3-DUS methods to obtain important diagnostic views not available with conventional ultrasound techniques.

17 3-D ultrasound imaging T. R. NELSON and D. H. PRETORIUS 1259 Cardiology The complex anatomy and dynamics of the heart make it a challenging organ to image. Some of the earliest work in 3-DUS was in the area of cardiology (Dekker et al. 1974; Fine 1988; Geiser et al. 1982a; Greenleaf 1982; Matsumoto et al. 1981; Meritz et al. 1983; Nixon et al. 1983; Stickels and Wann 1984; Sawada et al. 1983). Three- and four-dimensional imaging of the heart has been an active area of investigation and clinical evaluation for 3-DUS, particularly as improved equipment performance and advanced catheters have become available (Belohlavek et al. 1992, 1993a,b; Fulton et al. 1994; Greenleaf 1993; Levine et al. 1992; McCann et al. 1987, 1988; Ofili and Nanda 1994; Pandian et al. 1992; Salustri and Roelandt 1995; Salustri et al. 1995; Seward et al. 1995; Tardif et al. 1995; Vogel et al. 1995a). In addition to imaging, functional measurements regarding blood flow and ejection fractions provide quantitative information essential to completely understand the heart (Ariet et al. 1984; Nikravesh et al. 1984). 3-DUS methods of cardiac imaging have been shown to provide improved visualization of the heart atria and septum (Belohlavek et al. 1993b), overall cardiac anatomy (Binder et al. 1996; Flachskampf 1995; Salustri and Roelandt 1995), and congenital heart disease (Seward et al. 1995; Salustri et al. 1995a). 3-DUS images of cardiac structures and function offer the possibility of visualizing regurgitant jets using color/power Doppler (Delabays et al. 1995; Belohlavek et al. 1994; Fox and Gardiner 1988; Shiota et al. 1996) and 3-DUS methods also provide a means to evaluate valve function and anatomy (Kupferwasser et al. 1996; Levine et al. 1989; Nanda et al. 1994). Evaluation of anatomic defects, such as atriaseptal defects (Marx et al. 1995) and ventricular septal defects (Rivera et al. 1994), provides valuable information to assist in surgical planning, including simulating surgical approaches prior to operating (Vogel et al. 1995a). 3-DUS cardiac methods utilize both transthoracic (Salustri et al. 1995; Maehle et al. 1994) and transesophagel (Nanda et al. 1994; Roelandt et al. 1994; Ross et al. 1993) methods to obtain their information. Pediatric patients have shown good results with both techniques (Vogel and Losch 1994; Fulton et al. 1994). 3-DUS methods also can provide accurate measurements of cardiac chamber (Buck and Erbel 1995; Coppini 1995; Martin and Bashein 1989; Martin et al. 1990; Vogel et al. 1995a) and lumen (Matar et al. 1994) volumes. Rendering methods also can improve comprehension of dynamic cardiovascular anatomy (Cao et al. 1994; Nelson et al. 1996). Although early efforts have focused on the adult and pediatric patient, fetal cardiac imaging remains an important and challenging area (Crane et al. 1994). Conventional imaging equipment can produce static volume images of the heart that have shown some promise in imaging the great vessels and relatively nonmoving areas (Zosmer et al. 1996), although critical areas of cardiac anatomy are generally poorly visualized due to motion (Mertz et al. 1995) and small size (Meyer-Wittkopf et al. 1996). Chang et al. (1997) showed that accurate fetal heart volumes could be made with static imaging methods and demonstrated volume changes through gestation. Nelson et al. (1996) demonstrated gated fetal heart images throughout the cardiac cycle using nonecg motion Fourier analysis-based gating (Figs. 5, 19). They also showed images of each of the cardiac chambers throughout the cardiac cycle. Sklansky et al. (1997) showed that gated fetal heart images can demonstrate fetal cardiac anatomy more consistently than 2-D methods, with less dependence on the orientation of the acquisition. It also is possible to produce gated cardiac images using M- mode (Deng et al. 1996) and Doppler (Kwon et al. 1996) methods to determine the fetal heart rate. Eventually, clinical 3-DUS systems will provide the necessary memory, computational power and analysis tools to obtain high-quality gated fetal cardiac images. Musculoskeletal Musculoskeletal imaging using ultrasound has proven to be a challenging area. The vast differences in bone, cartilage and soft tissue acoustic impedances make it difficult to obtain clear views of the spatial relationships between bone, cartilage, tendon and muscles. Despite this, 3-DUS has the potential to improve visualization of joint anatomy by providing an integrated view of the overall spatial relationships. Early applications of 3-DUS to musculoskeletal anatomy have evaluated the pediatric hip. These studies have shown that a better view of anatomic relationships is possible (Bohm and Niethard 1994), although it is too early to determine whether 3-DUS is suitable for screening. 3-DUS does provide an improved view of the femoral head and acetabulum for evaluation of hip dysplasia (Gerscovich et al. 1994). 3-DUS also has been shown to be useful for calculating the volume of joint effusions and Baker cysts (Kellner et al. 1994). Evaluation of tendons and joint motion also are possible with improving technology. Psychological impact One of the greatest potentials of 3-DUS is to enhance visualization and comprehension of patient anatomy. Such capability permits a clearer impression of spatial relationships, which makes sharing information with nonspecialist physicians and patients more successful. Certainly, visualization of the fetal face has greatly improved comprehension of developmental anomalies (Merz et al. 1997; Pretorius et al. 1995), helping patients

18 1260 Ultrasound in Medicine and Biology Volume 24, Number 9, 1998 understand the management options. The clear visualization of the fetal face provided by 3-DUS also can have a very important role in improving maternal-fetal bonding (Maier et al. 1997), in understanding the severity of congenital anomalies and in assisting parents understand the management options. Dermatology Although 3-DUS methods have been predominately applied using standard clinical imaging equipment, recent work has extended 3-D methods to higher frequency imaging in the areas of dermatology and ophthalmology. Studies in the range of 20 MHz have shown promise in imaging of the superficial layers of the skin for assessment of skin lesions (Fornage et al. 1993; Stiller et al. 1993). Such methods have been shown to be valuable for obtaining reproducible measurements of skin tumor volumes, which can be useful for assessing responses to therapy (Stucker et al. 1995). Opthamology Imaging of the eye and surrounding structures requires high spatial resolution. Ultrasound ophthamologic imaging can use frequencies from standard clinical probes (10 MHz). 3-DUS methods permit viewing of vascular anatomy based on color and power Doppler methods (Downey and Fenster 1995a; Downey et al., 1996). Better results are obtained at the higher frequencies (up to the range of 100 MHz), which permit detailed imaging of the structural elements of the eye. 3-DUS methods at 50 MHz have been shown to be potentially useful for imaging the anterior segment (Silverman et al. 1995). 3-DUS methods at MHz have been shown to be valuable in demonstrating retinal detachment, assessing intraocular tumor volume (Jensen 1992) and providing detailed images to assist in planning the therapeutic approach (Coleman et al. 1995). As high-frequency imaging equipment improves and provides better capability for 3-DUS imaging, a more complete picture of eye anatomy will be possible, including visualization of blood flow and vascularity. Surgical applications Specialized catheters for intracavitary imaging (Feichtinger 1993; Hünerbein et al. 1996; Wang et al. 1994) and the capability for guiding interventional procedures with biopsy guidance (Downey et al. 1995a; State et al. 1996) are expanding the applications of volume imaging. Volume information from 3-DUS data offers the potential to provide more accurate localization of tumors, size estimation of tumors and guidance for therapy. Intraoperative guidance is a growing area of interest and importance (Kavic 1996; Fuchs et al. 1996; Vogel et al. 1995b; Strasser et al. 1995). 3-DUS needle guidance for stent placement (Cavaye and White 1993), cryosurgery (Downey et al. 1995a), radioactive seed implant placement and biopsy also has been shown to provide accurate localization and to be useful for identifying recurrence sites. Motion studies Evaluation of motion as a 4-D data set has been studied extensively in cardiology, as discussed above. Other motion applications also show promise. Monitoring of the shape of the tongue during speech or sustained sound production can be used to gain new insight to speech pathologies (Watkin and Rubin 1989; Stone 1990). Musculoskeletal imaging of joint motion is another area of interest. Also, imaging of blood-flow dynamics may ultimately prove to be a valuable means of understanding plaque formation and assessing the impact of flow restrictions. Breast Breast imaging is an important area of clinical application and research. Although 2-D ultrasound has been shown to be valuable for differentiating some aspects of benign/malignant disease (Tohno et al. 1994), 3-DUS imaging of the breast may assist this area significantly. Initial work by Rotten et al. (1991) showed the feasibility of the technique and reported that surface evaluation of breast masses with 3-DUS is more complete than 2-DUS, in that the entire surfaces of masses could be evaluated. Carson et al. (1992) demonstrated differences in the vascularity of breast tissues and masses. Additional efforts at correcting refraction and motion artifacts have the potential to improve resolution image quality (Moskalik et al. 1995), but additional work is needed to fully realize the potential of 3-DUS. 3-DUS provides improved evaluation of focal infiltrates, intracystic structures and multifocal disease plus reevaluation of data after the patient has left the clinic (Blohmer et al. 1996). 3-DUS also can be useful for assisting needle localization and guidance for biopsy purposes (Rotten et al. 1991). Urology 3-DUS methods have shown promise in imaging of the prostate (Downey et al. 1995b), both for monitoring gland pathology and responses to therapy (Richard et al. 1993; Richard and Keen 1996). Although clear delineation of the prostatic border remains difficult, 3-DUS methods show promise in identification and characterization of prostatic zones (Chen et al. 1995; Hendrix et al. 1990; Strasser et al. 1996), as well as more accurately estimating the entire prostate volume. The volume of the prostate can be measured with better than 92% accuracy using 3-DUS methods (Tong et al. 1996). Volume im-

19 3-D ultrasound imaging T. R. NELSON and D. H. PRETORIUS 1261 aging of the prostate offers guidance for interstitial laser therapy that is precise and reproducible (Strasser et al. 1995). Similar findings have been found for interstitial seed implants, which are fast and accurate (Stock et al. 1995). Imaging of the urethra and sphincter is improved with 3-DUS methods, which can provide guidance for stent design and placement and accurately measure the urethral cross-sectional area (Koelbl and Hanzal 1995; Ng, KJ et al. 1994; Tong et al. 1996). Gastrointestinal 3-DUS has been shown to provide accurate measurements of gall bladder volume and emptying (Pauletzki et al. 1996) and improved visualization of the biliary tree (Fine et al. 1991). 3-DUS provides improved evaluation of hepatic vessels and ducts in hepatobiliary disease compared to 2-D methods (Wagner et al. 1994). Liess et al. (1994) have shown that liver lesions can be more accurately measured with 3-DUS than with CT scanning or 2-DUS and that changes in size can be more readily identified on serial studies. Evaluation of diffuse liver disease, however, remains problematic. 3-DUS improves evaluation of the gastric antrum and volume estimation in patients with dyspepsia (Gilja et al. 1996). Vascular Imaging of blood flow and vessel anatomy is one of the most important areas of US imaging. 3-DUS methods show much promise to provide clear pictures of vascular anatomy and blood-flow dynamics, particularly with the advent of contrast materials and harmonic imaging (Burns et al. 1994, 1997; Forsberg et al. 1994; Goldberg et al. 1994). Evaluations of blood flow and vessel anatomy fall into two broad categories that overlap to some extent: First, external imaging using conventional imaging equipment and methods; second, intravascular imaging using specialized catheters. External imaging of blood flow and vessel anatomy using conventional imaging equipment has shown promise (Blankenhorn et al. 1983; Selzer et al. 1989; Kitney et al. 1989; Pretorius and Nelson 1991). These studies demonstrated the potential to image the carotid vessels. More recent studies have shown the applications of power Doppler to vascular imaging (Downey et al. 1995a), including the kidney and placenta (Pretorius et al. 1998; Ritchie et al. 1996). Laboratory results have shown that it is possible to accurately measure flow restrictions using power Doppler (Guo et al. 1995; Guo and Fenster 1996). 3-DUS methods have demonstrated the potential for transabdominal assessment of arterial plaque (Webber et al. 1995; Delcker and Diener 1994; Weissman et al. 1995), but additional work is need to fully assess these methods. 3-DUS methods also have shown promise for imaging of intracranial vascular anatomy (Lyden and Nelson 1997), particularly in combination with ultrasound contrast materials (Delcker and Turowski 1997). Intravascular imaging of blood flow and lumen structure using 3-DUS methods has shown great promise for imaging vessels, including the coronary arteries (Dhawale et al. 1994; DeMan et al. 1994; Roelandt et al. 1994b), despite their invasive nature (Cavaye et al. 1993; Chandrasekaran et al. 1994; Delcker and Diener 1994; Franceschi et al. 1992; Ng et al. 1994a; Pasterkamp et al. 1995; Rosenfield et al. 1991; von Birgelen et al. 1996). Intravascular methods utilize higher frequencies that permit higher resolution that can enhance visualization of flow dynamics (Chandran et al. 1996), vessel and graft patency (Ennis et al. 1993), and stent deployment (Gil et al. 1996; Mintz et al. 1993) but, thus far, technical issues regarding position localization (often a steady pullthrough method) have limited geometric accuracy. SUMMARY Three-dimensional ultrasound imaging is a new, exciting technology that allows physicians to use ultrasound to view anatomy and pathology as a volume, thereby enhancing comprehension of patient anatomy. Ongoing developments in computers and technology now permit acquisition, analysis and display of volume data in seconds, facilitating many opportunities for rapid diagnosis and interventional techniques. Both commercial and academic interest in 3-DUS is mounting and continued advancements and improved understanding are expected in the near future. Thus far, 3-DUS has demonstrated several advantages compared to 2-DUS. First, volume data can be viewed using a standard anatomic orientation with planar images to obtain simultaneous display of coronal, sagittal and axial planes, in addition to orientations that are difficult or impossible to obtain with conventional 2-DUS due to anatomical constraints (e.g., coronal plane of the uterus). Second, rendering of the entire volume allows the continuity of curved structures, such as liver vessels, the fetal spine or fetal face, to be viewed in a single image. Additionally, by rotating the volume interactively, it is possible to view the structure from multiple orientations or perspectives that enhance understanding of patient anatomy. Third, more accurate measurement of organ volume and irregularly shaped objects may be readily obtained. Fourth, volume data may be used to guide interventional procedures, providing accurate identification of needle/catheter placement. Fifth, volume data may be archived and subsequently evaluated for further critical review or teaching purposes on site or via the Internet after the patient has left the clinic. An important part of viewing 3-DUS data is the

20 1262 Ultrasound in Medicine and Biology Volume 24, Number 9, 1998 ability to review patient data interactively. The flexibility to rotate, scale and view objects from perspectives that optimize visualization of the anatomy of interest is critical to clinical acceptance, as is physician involvement in optimizing and enhancing visualization tools and in the ongoing evaluation visualization techniques. There is a continuing need for development of intuitive user interfaces and semiautomatic data-classification and segmentation tools, so physicians can quickly learn and use volume visualization systems. Minimizing artifacts from volume-visualization algorithms is a matter of diagnostic, ethical and legal importance, where lingering flaws could produce images that lead to an incorrect medical diagnosis. Errors in images could have serious ramifications in surgical procedures that rely on multidimensional data. A standard means for validating algorithms and performance is essential to future visualization efforts. Future volume-visualization systems also will need to fuse volume data from different modalities for the same patient. Optimization of ultrasound visualization depends on several factors. First, the overall image quality of the images is dependent on scanner setup and patient habitus. The distance between the structure and the transducer may lead to one side of the anatomy being imaged more clearly than the other side. Second, acquisition of 2-DUS images may be better in one plane than others due to thickness of the US scan plane. Third, rendered image quality is affected by the anatomic orientation and beam pathway. Signal dropout from shadowing due to overlying structures can significantly obscure structural detail. Compounding data obtained from different orientations can minimize the adverse effects of signal dropout, although elastic deformation of tissues due to pressure from the transducer during scanning can distort and complicate realignment of scans. Volume visualization quality is highly dependent on 2-D image quality; poor images result in poor volume data. If the patient moves during the scan, then the scan must be repeated or some form of correction for motion must be made. Importantly, faster affordable hardware is needed to shorten the time between acquisition and display. Some equipment is now becoming available for which the volume data are acquired directly into the rendering engine, permitting direct volume visualization immediately upon completion of the image acquisition, which will greatly enhance clinical acceptance. As experience with 3-DUS increases, continued refinement of analysis and visualization software will further assist making a more accurate diagnosis. Optimization of 3-DUS viewing is essential to achieve optimal clinical utility. To date, the primary viewing approaches have been: planar slices, surface fitting and volume rendering. Planar slices provide a straightforward method of display for interactive review, closely resembling clinical scanning procedures. Planar slices extracted from volume data can provide projections that may not be available during patient scanning and do not generally require intermediate processing or filtering of ultrasound image data, which further minimizes delay time to viewing. Surface fitting provides for rapid evaluation of the overall surface features of the object, but is sensitive to noise in the data; the bumpy or erratic surface significantly distorts features used in arriving at the correct diagnosis, which potentially limits use in ultrasound imaging, except in specialized applications such as vascular imaging. Volume rendering methods can produce high-quality images that are relatively tolerant of noise in the ultrasound data. Filtering can improve results, as long as it does not obscure fine detail. Careful selection of transparency values is essential to produce an accurate rendition of the structure to permit viewing surface and subsurface features that can help in establishing spatial relationships. For specific applications, maximum intensity methods give a clear view of structures, such as bones of the skeleton, although they are often best applied to selected regions of the volume. Stereoscopic viewing systems that let physicians utilize their binocular interpretative system to clarify structural relationships (Adelson and Hansen 1995; Herman et al. 1995; Hernandez et al. 1995, 1996b; Howry et al. 1956; Lateiner and Rubio 1994; Martin et al. 1995) can enhance identification of small structures with greater confidence in less time. Physician involvement in optimizing and enhancing these visualization tools is essential as part of the ongoing optimization of visualization techniques. As newer 3-DUS imaging systems become available to acquire volume data directly into the ultrasound scanner, physicians and sonographers will be able to immediately examine, visualize and interpret the patient s anatomy online. Regardless of which viewing technique is used, one key benefit of 3-DUS will be that, after the patient has been scanned, the original acquisition data may be analyzed for the entire region or magnified subregions without the need to rescan the patient. Clinical applications of 3-DUS visualization are increasing and ongoing research is evaluating those clinical applications likely to have the greatest clinical impact on patient diagnosis and management. To date, fetal, cardiac and gynecological areas have received the most attention, with other clinical areas receiving increasing interest including imaging of vascular anatomy, prostate volume measurement and assessment of seed placement, guidance of interventional needles and catheters, neonatal head evaluation, and evaluation of breast mass vascularity. In the fetus, improved visualization of fetal

21 3-D ultrasound imaging T. R. NELSON and D. H. PRETORIUS 1263 features, including the face, spine and limbs, has improved anomaly identification and been immediately appealing for clearly sharing development information with the family and colleagues. Cardiac applications have provided a surgeon s eye view using transesophageal and transthoracic techniques to assess valve geometry and motion and plan operative procedures. Gynecological applications include evaluating congenital anomalies of the uterus, endometrial cancer, adnexal masses, intrauterine device position and follicular cysts. Applications in other specialities such as urology, gastroenterology, dermatology and opthamology are expanding the range of applications for 3-DUS methods. 3-DUS imaging also permits obtaining images that may not be obtainable using conventional 2-DUS methods due to limitations in patient position or anatomy. Future developments Although clinical application of 3-DUS is growing, further developments will broaden the adoption of 3-DUS imaging worldwide. However, researchers must address specialized problems before widespread clinical use can occur including: 1. Improving overall scanning system performance (including elevational focus, compounding and shadowing compensation) and developing smaller scanners and transducers; 2. improving image and position acquisition methods for 3-DUS (e.g., freehand image-based scanning and intraluminal and endocavitary probe systems); 3. improving user interfaces, making it easier to evaluate volume data and understand patient anatomy; 4. improving algorithms, particularly automated algorithms, for segmentation, data classification and biometry; 5. improving physiological gating for both the cardiac and respiratory cycles, eliminating motion artifacts and opening a window to dynamic processes; 6. integrating 3-DUS imaging and ultrasound contrast agents to enhance visualization of vascular anatomy; 7. producing faster volume acquisition, analysis and interactive displays, ultimately leading to 8. realtime volume imaging. Future 3-DUS systems will display images of anatomy and organs in an intuitively straightforward manner, will allow physicians to feel as if they are holding a model of the organ in their hands, allowing them to see the organ or fetus as it actually is (Bajura et al. 1992; Fuchs et al. 1996; State et al. 1994) (Fig. 24). Dynamic motion will be reviewed by slowing down the images to assess fine movement; the valves or the walls of the heart will be able to be artificially stopped to make measurements or to compare specific images, either volume or conventional planar images. Blood-flow visualization through the cardiac cycle will be 3-D rather than 2-D. The 3-DUS system of the future will not treat sonographic data as a series of 2-D images viewed in Fig. 24. Future volume imaging in ultrasound, and other modalities, will offer real-time interactive review and manipulation of patient data. Instead of viewing patient data on a computer or scanner console, the sonographer or physician will directly view the internal patient anatomy and organs in an intuitively straightforward manner, which will allow physicians to feel as if they are holding a model of the organ in their hands, allowing them to see the organ or fetus as it actually is. Direct viewing of internal anatomy would facilitate more rapid comprehension of the patient s condition and provide improved feedback for interventional procedures. either real-time or statically, but as a volume with which the clinician rapidly interacts as though exploring internal patient anatomy directly. The technology to accomplish this is available today, and will benefit from continued performance increases and cost reductions, ultimately providing volume sonographic imaging systems as standard equipment at costs comparable to conventional systems available today. Ultimately, 3-DUS technology will provide a central integrating focus in ultrasound imaging. 3-DUS equipment will provide more conventional 2-DUS imaging capability, but seamlessly expand imaging to include volume acquisition and display as necessary to obtain the diagnosis. Focused application of 3-DUS methods will expand the areas in which ultrasound is the predominate imaging modality beyond those areas in which ultrasound currently is the method of choice. Through rapid transmission of volume data to specialists at distant locations, patient care will benefit from improved diagnosis and treatment. Acknowledgements The authors appreciate the assistance and support of Kretz Medison, Siemens Ultrasound, General Electric Medical Systems and Acuson Corporation. REFERENCES Adelson SJ, Hansen CD. Fast stereoscopic images with ray-traced volume rendering. Proceedings of 1994 Symposium on Volume Visualization. 1995;3 9, No Ariet M, Geiser EA, Lupkiewicz SM, Conetta DA, Conti CR. Evaluation of a three-dimensional reconstruction to compute left ventricular volume and mass. Am J Cardiol 1984;54: