Using CAVASS as the Basis for Imaging Applications

Transcription

1 Using CAVASS as the Basis for Imaging Applications George Grevera *ab, Jayaram Udupa b, Dewey Odhner b a Department of Mathematics and Computer Science, Saint Joseph s University, 5600 City Avenue, Philadelphia, PA b Medical Image Processing Group (MIPG), Department of Radiology, University of Pennsylvania, 423 Guardian Drive, 4th Floor Blockley Hall, Philadelphia, PA *ggrevera@sju.edu; phone ; fax ; ABSTRACT CAVASS (Computer Aided Visualization and Analysis Software System) is an open-source software system that is being developed by the Medical Image Processing Group (MIPG) at the University of Pennsylvania. It includes extremely efficient (and often parallel) implementations of the most commonly used image display, manipulation, and processing operations. CAVASS seamlessly interfaces with ITK and provides a user interface for it as well. It can easily interface with a PACS by directly reading and writing DICOM images and can also read and write other common image formats as well. We describe the general software architecture of CAVASS so one may quickly use it as the basis for one s own applications. Keywords: Visualization, DICOM, rendering, segmentation, registration, image processing. 1. INTRODUCTION CAVASS (Computer Aided Visualization and Analysis Software System) [1] is an open-source software system that is being developed by the Medical Image Processing Group (MIPG) at the University of Pennsylvania. Our hope is that the CAVASS application suite will not only be used as it currently exists, but new applications will be written within the CAVASS framework as well. CAVASS is based on MIPG s experience with its predecessor, 3DVIEWNIX [2] (and its progenitors, DISPLAY [3], DISPLAY82 [4], 3D83, [5], and 3D98 [6]), which was developed at MIPG as well. It includes extremely efficient (and often parallel) implementations of the most commonly used image display, manipulation, and processing operations. MIPG is also a key contributor to the ITK (Insight ToolKit) project [7] from its inception. ITK provides a wealth of image processing, segmentation, and registration methods but (purposely) lacks a user interface. CAVASS optionally and seamlessly interfaces with ITK and provides a graphical user interface for it as well. CAVASS can also easily interface with a PACS by directly reading and writing DICOM [8] image data (see Figure 1). CAVASS can also read and write other common 2D image formats such as TIFF [9], GIF [10], JPEG [11], VTK [12,13], PNG [14], STL (Stereo Lithography, a common CAD/CAM format) [15], and the NETPBM format family [16] as well. Previously, we described and evaluated the image processing [17] and visualization [18] capabilities of CAVASS. We now describe the general software architecture of CAVASS so one may quickly use it as the basis for one s own applications. CAVASS is an open-source software system that is written in portable C++ which executes on many different operating systems (Windows, Linux, Unix, and Mac) while employing a single source code base. The source code is well documented and many examples that can be used as the basis for new, additional applications are provided. Based on our group s experience with the development of 3DVIEWNIX (and its progenitors, DISPLAY, DISPLAY82, 3D83, and 3D98), extremely efficient and often parallel implementations [17,19] of key operations have been implemented. In addition to the standard image file formats mentioned above, CAVASS can also directly read and write its native file format, IM0 [20,21], which is an n-dimensional extension to the DICOM format that was provided by 3DVIEWNIX. Tools for common image filtering operations such as Gaussian, median, diffusive, and morphological filtering are available as well. Additionally, CAVASS provides segmentation tools such as threshold, live-wire [22], and fuzzy connectedness [23]. Surfaces may be created using these tools and subsequently rendered and manipulated using the

2 shell [24] and t-shell [25] rendering (see Figure 2) and manipulation capabilities of CAVASS. In addition to 3D surface and volume rendering, CAVASS also provides a number of slice display modes such as montage (for the simultaneous display of slices from multiple data sets), overlay (to superimpose slices from two data sets), and cine/cycle (for the sideby-side display of slices from multiple data sets). CAVASS also provides efficient implementations of rigid and affine registration (based on correlation and mutual information). An efficient, parallel implementation of deformable registration will be provided as well. Figure 1. Example of DICOM input via CAVASS. For those operations that are not provided directly by CAVASS, CAVASS also provides a mechanism to optionally integrate with ITK by providing ITK with a user interface (see Figure 3). Examples are currently provided for ITK s binary dilate, binary erode, binary median, binary morphology, binary threshold, binomial blur, Canny edge detection [26], gray dilate, gray erode, mean, median, and voting filter operations. CAVASS also provides code to enable ITK to read and write the native CAVASS file format (IM0) directly. CAVASS can export data in the VTK format as well. Instructional material in the form of tutorials, recipes (sequences of operations to accomplish common tasks), and source code documentation is integrated within the CAVASS framework as well (see Figure 4). 2. MATERIALS & METHODS 2.1 The CAVASS build environment As mentioned previously, CAVASS is written in portable (i.e., without any operating system specific function calls) C++. For operating system independence with regard to a graphical user interface, CAVASS employs the freely available, open source, and object-oriented wxwidgets [27,28] C++ user interface library. Using this library, one may write applications for Windows, Linux, Unix, and the Mac operating systems that share a single source code base (i.e., operating system specific code is not only unnecessary but is to be avoided for the sake of portability). As compared with other portable user interface libraries such as FLTK [29] and Java s Swing [30], wxwidgets maintains the platform s native user interface look-and-feel. To automatically generate build environments for the various operating systems and compilers/ides (which itself can become onerous), CAVASS employs the cmake utility [31,32] as does ITK. In addition to generating operating system

3 specific build environments (makefiles for Unix, Linux, and Mac, and Microsoft Visual C++ Solution files for Windows), the use of cmake also allows the user to specify whether or not code should be generated that interfaces with ITK. In that way, CAVASS may optionally provide an interface for ITK and be extended by it but CAVASS is not dependent upon ITK and does not require it to be available. For the implementation of parallel algorithms, CAVASS optionally uses the Message Passing Interface (MPI) [33-35], a portable C++ programming library. Similar to ITK, CAVASS does not require MPI to be available. One specifies via cmake whether or not one wishes to build a version of CAVASS that provides implementations of parallel algorithms (or not). Figure 2. t-shell surface rendering (entirely in software) via CAVASS. For source code control, CVS (Concurrent Versions System) [36,37] is employed to maintain the source code repository. CVS (and source code control systems in general) allows users to check source code out of the repository, modify the source code, and then check in their changes. Additionally, CVS also maintains a history of what changes were made, when the changes took place, and what individual made the changes. The user may also add additional information about the specific changes in the form of comments or annotation as well. CVS is relatively unique among source code control systems because changes may be made concurrently by many different users at the same time to the same source code file. CVS then attempts to merge these changes. If successful, the repository is updated; if unsuccessful, the user must resolve those changes that are determined to be mutually exclusive changes before the repository can be updated. To generate documentation based on the CAVASS source code, we employ the doxygen [38] system (as does ITK). Doxygen is similar to JavaDoc [39] in its approach to the creation of documentation but is not restricted to the Java programming language. Doxygen and JavaDoc function by parsing the C++ or Java source code as does a compiler but rather than ignoring comments and processing other source code statements, doxygen parses the comments and for the most part ignores other statements. Based on the information in the comments, doxygen generates documentation in the form of HTML that can be viewed by a web browser or by wxwidgets directly (Figure 4). This allows the documentation to be directly integrated into and displayed by the CAVASS application. This documentation is extensive. It includes documentation for every class, method including parameters and return values, and class data members. Graphics are also automatically produced that indicate the class inheritance hierarchy as well. This hierarchy may be traversed by the user by simply clicking on different areas of the graphic corresponding to a specific class in the inheritance hierarchy.

4 To test CAVASS, we employ 3 data sets of increasing size that are used to test all of the algorithms. The regular size data set consists of MR brain image data that is 256 x 256 x 46 voxels and 6 MB in size. The large data set is a CT of the thorax and is 512 x 512 x 459 voxels (241 MB). The third data set, super, is CT data from the Visible Woman and is 1023 x 1023 x 417 voxels (873 MB). Figure 3. Example of a CAVASS user interface for ITK s Canny edge detection implementation. 2.2 CAVASS data I/O classes With regard to the input and output of image data, CAVASS provides the CavassData class as a single methodology for reading and writing all standard image formats such as DICOM, JPEG, TIFF, and the native CAVASS IM0 format (an n-dimensional generalization of the DICOM format that was employed by 3DVIEWNIX). In its basic form, the IM0 format is a DICOM-like format that contains all of the Patient-Study-Series-Image information in a single file for an entire series. This format is very useful for a number of reasons. First, redundant Patient-Study-Series information is stored only once for the entire file rather than being stored repeatedly in each separate DICOM image data file. Furthermore, each slice is stored in the IM0 file according to its physical location with regard to the scanner coordinate system. Therefore, the sequence of 2D images in a 3D data set is determined only once when the IM0 file is created from the individual DICOM data according to the actual physical location of the slice (rather than relying on more ad hoc methods such as DICOM file naming conventions which may not be reliable). The actual physical location of each slice is recorded in the IM0 data file as well. An additional feature of the IM0 format is that an entire series may be moved easily from system to system because the single file contains the entire series (one cannot fail to copy one of the slices in a series). In more advanced forms of the IM0 file format, it may represent n-dimensional, vector-valued image data such as MRI T2 and PD data in a single file as well. The CAVASS user interface provides a module to create an IM0 file from individual DICOM files. Conversely, CAVASS also provides a tool to create individual DICOM files from an IM0 file as well. The CavassData class can only be used to read in the image header or it may be used to read in the image header and the entire data set (but it cannot be used to read in only part of the data such as a single slice). To read in a slice or a set of contiguous slices (referred to as a chunk), two subclasses of CavassData are provided: SliceData and ChunkData, respectively. With these classes, CAVASS provides access to data sets that far exceed the RAM memory capabilities of the computer on which it executes. Support for gray and color data (with various numbers of bits) is provided as well for both integer and floating point types. Furthermore, data need not always be associated with a file but may consist of data in memory only as well. Since visualization is at the heart of CAVASS, the CavassData class also provides a LUT (lookup table) for basic level-window (window width and center) contrast manipulation.

5 Figure 4. Integration of tutorial information and documentation within the CAVASS application. 2.3 Architecture of a CAVASS module CAVASS application-specific windows are referred to as frames (from wxwidgets nomenclature) and share a common menu structure for consistency across all application modules. CAVASS divides each frame into a standard menu bar, an upper canvas for drawing and displaying graphical information, a lower control area for user interface items such as buttons and text boxes, and a status bar at the bottom that contains informational messages and indicates the current functions associated with the (up to) three mouse buttons. User interface (not mouse) buttons that persist throughout a particular application module always appear to the right of the control area. Text boxes, buttons, and other items that are not always present for a particular application module appear towards the left of the control area. The control area may be hidden by the user at any time (and be made to reappear, of course) so the entire window frame may be used for the graphical display of image data. This format was based on our experience with 3DVIEWNIX. All CAVASS frames inherit from the C++ MainFrame class which provides much of the default behavior of CAVASS such as the standardized menu bar, print, copy to the clipboard, and the ability to hide and display the control area. Similarly, all CAVASS canvasses inherit from MainCanvas. To encourage reuse and consistent appearance and behavior of controls, CAVASS also provides a number of standard controls such as GrayMapControls, CineControls, etc. In addition to the source code provided for all of the CAVASS application modules, a simple example that displays and manipulates the contrast of slice data is provided (ExampleFrame and ExampleCanvas) as well. With regard to event handling, wxwidgets supports and implements the Windows-style event callback mechanism. This is very efficient and fine for most user interaction. This is in contrast with the X-Windows system which supports and implements the event queue mechanism. This approach is much more flexible for intensive user interaction. This flexibility is especially needed when there are possible delays due to computation time (e.g., 3D rendering). For tasks such as rendering, we implemented an X-Windows style event queue within CAVASS using only the wxwidgets callback mechanism. We create a separate main thread of execution that immediately responds to events in an event queue (of our own creation). Simultaneously, we also perform compute intensive tasks which run at lower priorities in other threads. The main thread continues to respond to events via the callback mechanism which intelligently queues the events for execution by the other threads because it runs at a higher priority than the compute intensive tasks. Currently, only surface and volume rendering require this mechanism. 2.4 Interface with ITK As mentioned previously, CAVASS optionally integrates with and provides a user interface for ITK (Figure 3). Rather than linking with the ITK libraries directly, CAVASS provides ITK with two classes that may be used by ITK to read and write CAVASS IM0 files (IM0VolumeReader and IM0VolumeWriter, respectively). A separate ITK program that reads in one or more IM0 files and outputs one or more IM0 files is then created to execute the specific ITK operation. Specific processing options or parameters may be passed as command line arguments to these programs as well. Once these programs with their corresponding inputs, parameters, and outputs have been defined, CAVASS provides a user interface for ITK as follows. First, CAVASS displays a slice from each of the input data sets. The user then selects the specific ITK operation from a drop down menu. Each ITK operation may have its own set of specific processing parameters. An individual parameter may either be an integer or a real number (with corresponding minimum, default, and maximum values) or a specific string from a set of possible strings (also with a corresponding default). Given a

6 particular ITK operation, the corresponding parameters appear in the user interface with sliders for integers and real number or drop down menus for strings. Processing then occurs as follows. Once input(s) and parameter(s) have been specified, the user may either press a button called Filter or Save. The Save button causes the ITK operation to be performed on the entire input data set(s). These operations may be time consuming so a progress dialog box is presented to the user to provide visual feedback. Once the operation has completed, CAVASS displays a slice from the result that corresponds to the currently displayed slice from the input. The user may navigate through these slices to view the result. For time consuming operations that require a great deal of experimentation with parameter settings, the Filter button is provided. Given a displayed input slice and a particular operation and corresponding parameter settings, the Filter button causes the operation to occur only on the single input slice. This occurs much faster than processing the entire volume so the resulting slice appears quickly. This process can be repeated using different parameter settings until optimal ones are chosen by the user. Then the Save button may be used to process the entire volume with the optimal parameter settings. Figure 3 illustrates the user interface provided by CAVASS for ITK s Canny edge detection filter operation and its corresponding four processing parameters (variance and maximum error used in Gaussian smoothing of the input image, and threshold level and outside value (any values below the threshold level parameter will be replaced by the outside value parameter)). CAVASS currently provides a user interface for the following ITK filters (and their corresponding parameters): binary dilate, binary erode, binary median, binary morphology, binary threshold, binomial blur, Canny edge detection, gray dilate, gray erode, mean, median, and voting. All of the above are implemented in CAVASS using a table that can be easily extended to include additional ITK operations. Figure 5. CAVASS dialog that allows one to configure the cluster for parallel processing and test the configuration with a simple test. 2.5 Parallel architecture To implement parallel algorithms, the CAVASS development team considered two competing yet mature standards: (1) MPI/OpenMPI (Message Passing Interface), and (2) OpenMP (Open specifications for Multi Processing) [40]. MPI is freely available for Windows, Linux, and Unix. The LAM MPI [41] implementation of MPI is part of the base Linux install and the MPICH2 [42] implementation is freely available for Windows. MPI employs a COW (cluster of workstations) model. It leverages the existing hardware/computers that one already has so it is easily expandable and relatively inexpensive. Figure 5 demonstrates the ease at which a COW can be defined using CAVASS by simply lising the names of the systems that participate in the cluster and the number of processes that should be assigned to each system. An optional, inexpensive network upgrade (a 1 Gbit or 10 Gbit switch and compatible network cards for each system) dramatically increases the performance of the COW at minimal cost. MPI can also efficiently employ multicore, shared memory systems and mix them in a heterogeneous environment with other non multi-core systems on the network. The only restriction is that the systems on the network that participate in the COW must be homogeneous with regard to operating system and hardware (all x86 Windows or all x86 Linux or all Sun UltraSPARC Unix, etc.). The other parallel standard, OpenMP, requires the purchase of expensive multiprocessor systems and specialized compilers and employs a multi-threaded, shared memory parallelism model. Because of the minimal cost and widespread availability, we choose MPI and its COW model. In CAVASS we implement parallel operations using a master-slave or manager-worker model. The system upon which the CAVASS user interface executes is the master. It sends data and

7 work tasks to each of the slave systems which perform the computation in parallel. The master divides the input image data into chunks (a contiguous set of slices, either image or object structure data) and assigns each chunk to one of the slave processors. The master then collects the results from the slave systems and displays the results for the user. In general, parallel CAVA operations can be divided into the following three groups. Type 1 proceeds in a chunk-bychunk manner, i.e., each chunk is accessed only once. An example of this type of operation is slice interpolation (see Figure 6). Type 2 is similar to Type 1 but a significant further operation is needed to combine the results. An example of type 2 is 3D rendering. Type 3 also operates in a chunk-by-chunk, but each chunk may have to be accessed more than once. An example of this type would be algorithms that employ graph traversal. CAVASS parallelizes all three groups of operations when necessary. Figure 6. Parallel interpolation as implemented in CAVASS. 3. RESULTS We have demonstrated that one may develop a wide variety of medical imaging applications within the CAVASS framework including segmentation, registration, manipulation, and visualization. For those features that are not already provided by CAVASS, it may be easily extended by the addition of new modules into the existing framework. CAVASS may also be employed to provide a user interface for ITK modules as well. The CAVASS user interface is based upon our extensive experience with 3DVIEWNIX and its progenitors and provides a consistent, easy-to-use interface for 3D medical imaging applications. We believe that CAVASS possesses many key features that, taken together, make it truly unique. It is the first open source software system with all of the following: (i) extremely efficient implementations of the most frequently used algorithms are provided, (ii) it uses a single code base but provides support for many different operating systems, (iii) parallel, cross-platform implementations of key operations are available, (iv) it interfaces with and provides a user interface for ITK, and, (v) it is dedicated specifically to the processing, manipulation, analysis, and display of 3D medical imagery.

8 4. CONCLUSIONS We have described the CAVASS software development environment, user interface, and application framework in detail and have provided examples that allow users to not only use CAVASS but help them write their own application modules as well. We have also described the input and output features (including DICOM support) of CAVASS that allow one to easily handle data sets with sizes that exceed the host computer s RAM (random access memory). As ITK grows, we have also demonstrated how CAVASS can be extended to include these new features and provide an interface for them. ACKNOWLEDGEMENTS The authors gratefully acknowledge support for this work from DHHS grant EB REFERENCES [1] Grevera, G., Udupa, J., Odhner, D., Zhuge, Y., Souza, A., Iwanaga, T., and Mishra, S., "CAVASS: a Computer Assisted Visualization and Analysis Software System. " J. Digital Imaging 20(S1): , (2007). [2] Udupa, J.K., Odhner, D., Samarasekera, S., Goncalves, R., lyer, K., Venugopal, K., and Furuie, S.: "3DVIEWNIX: an open, transportable, multidimensional, multimodality, multiparametric imaging software system," in SPIE Proceedings 2164:58-73, (1994). [3] Udupa, J.K.: "DISPLAY - A system of programs for two- and three-dimensional display of medical objects from CT data," Technical Report MIPG41, Medical Image Processing Group, Department of Computer Science, SUNY/Buffalo, Buffalo, New York, (1980). [4] Udupa, J.K.: "DISPLAY82 - A system of programs for the display of 3D information in CT data," Technical Report MIPG67, Medical Image Processing Group, Department of Radiology, University of Pennsylvania, Philadelphia, April (1983). [5] Chen, L.S., Herman, G.T., Meyer, C.R., Reynolds, R.A., and Udupa, J.K.: "3D83 - An easy-to-use software package for three-dimensional display from computed tomograms," Proceedings of IEEE Computer Society International Symposium on Medical Images and Icons, Arlington, Virginia, pp , (1984). [6] Udupa, J.K., Herman, G.T., Margasahayam, P.S., Chen, L.S., and Meyer, C.R.: "3D98: A turnkey system for the display and analysis of 3D medical objects," SPIE Proceedings 671: , (1986). [7] [8] National Electrical Manufactures Association (NEMA): [Digital Imaging and Communication in Medicine (DICOM)] Part 1: Introduction and Overview, NEMA, Washington, DC, (1993). [9] partners.adobe.com/public/developer/en/tiff/tiff6.pdf. [10] [11] [12] [13] [14] [15] en.wikipedia.org/wiki/stl_(file_format). [16] netpbm.sourceforge.net/. [17] Udupa, J.K., Grevera, G., Odhner, O., Zhuge, Y., Souza, A., Mishra, S., and Iwanaga, T., "CAVASS: Computer Assisted Visualization and Analysis Software System - Image Processing Aspects." Proc. SPIE Medical Imaging, San Diego, CA, (2007). [18] Grevera, G., Udupa, J., Odhner, O., Zhuge, Y., Souza, A., Iwanaga, T., and Mishra, S., "CAVASS: Computer Assisted Visualization and Analysis Software System - Visualization Aspects." Proc. SPIE Medical Imaging, San Diego, CA, (2007). [19] Udupa, J.K., Grevera, G.J., Odhner, D., Zhuge, Y., Souza, A., Mishra, S., and Iwanaga, T.: "Fast Parallel Image Processing on a Cluster of Workstations for Analyzing Very Large Medical Image Data Sets.: A scientific presentation at the RSNA (Radiological Society of North America) 2007 annual meeting, Chicago, IL, (2007).

9 [20] Udupa, J.K., Hung, H.M., Odhner, D., and Goncalves, R.: "Multidimensional data format specification: A generalization of the American College of Radiology National Electric Manufacturers Association Standards," Journal of Digital Imaging 5(l):26-45, (1992). [21] Udupa, J.K., Hung, H.M., Odhner, D., and Goncalves, R.: "The 3DVIEWNIX Software System, Data Format Specification: A Multidimensional Extension to the ACR-NEMA Standards," Version 1.0, Technical Report MIPG177, Medical Image processing Group, Department of Radiology, University of Pennsylvania, Philadelphia, January (1991). [22] Falcao, A., Udupa, J.K., Samarasekera, S., Sharma, S., Hirsch, B.E., Lotufo, R., User-steered image segmentation paradigms: Live wire and live lane, Graphical Models and Image Processing, 60(4): , (1998). [23] Udupa, J.K., Samarasekera, S., Venugopal, K.P., and Grevera, G.: "Fuzzy Objects and Their Boundaries." Proc. SPIE Proceedings Visualization in Biomedical Computing 2359:50-58, Rochester, MN, (1994). [24] Udupa, J.K., and Odhner, D.: "Shell rendering," IEEE Computer Graphics and Applications 13(6):58-67, (1993). [25] Grevera, G.J., and Udupa, J.K.: "T-Shell Rendering and Manipulation." Proc. SPIE Medical Imaging 5744:22-33, San Diego, CA, (2005). [26.] Canny, J., A Computational Approach To Edge Detection, IEEE Trans. Pattern Analysis and Machine Intelligence, 8: , (1986). [27] [28] Smart, J., Hock, K., and Csomor, S.: [Cross-Platform GUI Programming with wxwidgets], Prentice Hall, (2005). [29] [30] Geary, D.: [Graphic Java 2, Volume 2, Swing]. 3rd Edition, Prentice Hall, (1999). [31] [32] Martin, K., and Hoffman, B.: [Mastering CMake], 4th ed., Kitware, (2008). [33] [34] [35] Pacheco, P.: [Parallel Programming With MPI], Morgan Kaufmann, (1996). [36] [37] Vesperman, J.: [Essential CVS], 2nd ed., O Reilly Media, (2006). [38] [39] java.sun.com/j2se/javadoc/. [40] [41] [42]