Modeling the Human Brain Using Serial Cryosections from the Visible Human Project Data Set

Transcription

1 Modeling the Human Brain Using Serial Cryosections from the Visible Human Project Data Set Kevin M. Brennan The brain is an extremely complex organ, and accurate digital three-dimensional models are scarce. To better visualize the structures of the brain, the author built a polygonal model for viewing on the C-Wall Virtual Reality System at the Virtual Reality in Medicine Lab at the University of Illinois at Chicago. Benefits of Virtual Reality (VR) include stereovision, viewer-centered perspective, wide angles of view, opportunities for remote collaboration, and the ability for users to interact with the models. The Visible Human Project Female Dataset from the National Library of Medicine was used as the raw data set for this project. A neuroanatomist was consulted throughout the project to ensure that the most accurate representation was achieved. Introduction We devised a technique for creating a polygonal model based on the Female Virtual Human Dataset. The resulting model can be viewed in Virtual Reality at the Virtual Reality in Medicine Lab (VRMedLab) at the University of Illinois at Chicago (UIC). The technique, developed to create a more accurate model, began with 255 serial cryosections of the Visible Human Project Female Dataset. Working with a neuroanatomist, I hand-segmented the cryosections and the resulting images were filtered using a box-averaging algorithm. The filtered images were imported into a series of 3D visualization programs to create the final 3D polygonal model. The model was then texture-mapped based on photographs of live tissue. Finally the brain was modified for utilization in a Virtual Reality system for interactive stereoviewing (see animation in the online version). Process Segmentation of Raw Data The National Library of Medicine s (NLM) Visible Human Project Female Dataset set was the basis for the creation of the model. I selected the female data set over its male counterpart for its higher degree of detail (the male data set slices are 1.0mm in Figure 1. Raw data (cryosection 238) from the Visible Human Project Female Data Set. thickness, the female.33mm). Each cryosection is 2048 pixels by 1216 pixels in size. Figure 1 shows a representative slice from the dataset). Each slice took fifteen to thirty-five minutes to process (excluding review time with the neuronatomist, and subsequent modifications to the selections). To keep the project within time constraints, every other cryosection was used. In order to use the desired programs further in the process, the image slices used were changed from full color to grayscale. Each slice was imported into Adobe Photoshop at maximum resolution in TIF format and saved with no compression. In Photoshop, a layer of black was painted over the imported image to describe the extent of the brain tissue. Any areas that E17

2 Figure 3. (left) Traced cryosection (238). The neural tissue is now white, mimicking radiological data Figure 4. (right) Cryosection (238). The traced cryosection has now been run through the VMP at a sample size of 5. Figure 2. Traced cryosection (238). Neural tissue is in black. were not neural tissue or composed of negative space (sulci, ventricles, vessel lumina, etc.) were removed from the black representation of the section. This resulted in the neural tissue alone being represented by black (Figure 2). The resulting images were reviewed for accuracy by Dr. Conrad Anderson, a neuroanatomist at the University of Illinois, Chicago (UIC). Dr. Anderson reviewed, suggested modifications, and ultimately approved each segmented slice. In order to improve the contrast of the traced sections, and enable Mimics (Materialise) to provide a better result, negatives of these images were created. The result was brain tissue represented by white on a black background (Figure 3). The white on black representation of the neural tissue is similar to the radiological data that Mimics uses to create a 3D surface from cross-sectional data. Box-Averaging Filter After all the slices were prepared in white, a three-dimensional box-averaging filter algorithm was applied to the images (Figure 4). The algorithm was written by Zhuming Ai in C++ and runs on a Linux machine. The box-averaging filter works as a Volume Manipulation Program (VMP). This VMP takes an individual pixel in 3D space and changes its value based on the resulting average of the pixels in a sampling cube of a defined size. This effectively smooths the image (reducing noise) and helps to reduce artifacts. The sampling cube sizes tested were three, five, seven, and nine. Figure 5. Mimics interface with test sample of the cryosections loaded. Constructing the Polygonal Model The filtered white on black images (simulating radiological data) were imported for each sample size (three, five, seven, and nine), and a polygonal model of the brain was created in Mimics (Figure 5). This resulted in five brains, one created without using the VMP and the four sample sizes. Based on a review of the resulting models the cerebellum was built at a sample size of three, and the cerebrum was built at a sample size of five. These sampling levels provided the best representation of the topography with the fewest artifacts and most complete surfaces. The resulting polygonal model was then exported into Geomagics (Raindrop) (Figure 6). Geomagics was used to reduce the number of polygons in the model, while preserving detail and diminishing any additional irregularities and artifacts of the segmentation process (Figures 7). The model was then imported E18

3 Figure 6. Initial import of the model from Mimics into Geomagics. Figure 7. Detail showing density of the mesh. into 3ds Max (Autodesk), all vertices were selected, and the weld function was used at a tolerance of This removed any extraneous, repetitive, or overlapping vertices. Sufficient landmarks were captured on the cerebrum. The resulting cerebellum was used as an outline for placement and volume. Modeling the Cerebellum The cerebellum model that was created had a discontinuous surface with the small variations in its surface not captured (Figure 8). The problem was addressed by hand-modeling the cerebellum in 3ds Max. The cerebellum created from the data set was used as a guide in an effort to preserve landmarks, volume, and proportions. The resulting cerebellum had a continuous outer surface that allowed texture mapping and the addition of some major anatomical landmarks (processes that were not possible with the discontinuous surface that characterized the original geometry). Texture Mapping of Cerebrum and Cerebellum Due to the lack of detail captured in the cerebellum, I decided to use textures to illustrate the major landmarks. The mesh was unwrapped to obtain UVW mapping coordinates. Unwrapping the model produces a two-dimensional representation of the geometry that served as a guide for the creation of the texture maps. Shrink-wrap UVW coordinates were used on the cerebellum and the pole was placed under the cerebrum to hide and minimize stretching. Major landmarks of the cerebellum (including the vermis, horizontal fissure, primary fissure, and posterior lunate fissure) were placed on the model using displacement and bump mapping. The locations of the aforementioned landmarks were derived mainly from details preserved in the model directly from the data set, and from some preserved specimens. The color (diffuse property in 3ds Max) of the material was applied last. The general texture of the cerebrum and cerebellum were informed by reference photographs. The cerebrum texture coordinates were established through use of a UVW box texture projection. The texture for the cerebrum was edited to be tileable, so that it was repeated seamlessly over the surface of the geometry (Figure 9). Discussion The five-pixel box-averaging filter was the most successful in achieving a smooth and consistent result while retaining detail on the cerebrum. More jagged and discontinuous surfaces resulted at smaller sampling volumes. Capturing detail from the cerebellum was significantly more difficult because its physical features are far more detailed. The final model was comprised of a cerebrum and brain stem (filtered with a five pixel sampling cube) and a cerebellum (box-modeled based on a model built from data filtered with a three cubic pixel sample size). Building the model without using a filter was also successful. As expected, there was a degree of stair stepping between the slices. This stair stepping was the result of the differences in the slices, using every other slice, and not having the subsequent slices averaged and blended together. This effect was somewhat diminished by using the smooth function in Geomagics. UVW mapping coordinates were applied to the model to allow texturing of the slices and external surfaces of the model. The cerebellum was texture mapped to fit the anatomy. Preparation for Virtual Reality Final output included VRML (Virtual Reality Modeling Language) for viewing on the web, still images (JPEG, PNG, and TIFF), 3D models (OBJ, STL), and animations depicting the data E19

4 Figure 8. (left) Orthographic views of the model after Geomagics. Figure 9. (right) Resulting surfaces after creating the new cerebellum and texturing. Figure 10. Final model being manipulated on the C-Wall in the VRMedLab. used to create the final model. The two most dynamic outputs, VRML and the 3D polygonal models, offer unique visualization experiences. The VRML format enables the user to rotate and orient the brain in any desired position. It does not require the purchase of any special software. For presentation in Virtual Reality, specific hardware and software are required. The hardware and software (described below), available at the Virtual Reality in Medicine Lab (VRMedLab) at UIC, were used to interactively display the 3D brain model (Figures 10, 11). Due to limitations of the file formats accepted by the VR program, some retexturing of the model was Figure 11. Dr. Anderson showing a lateral view of the model in VR. necessary. Some parameters, including displacement and bump, are not supported. The C-Wall (Configurable-Wall) Virtual Reality (VR) System was created using graphics and networking libraries developed at the Electronic Visualization Lab and VR MedLab at UIC. Components of the C-Wall VR system include a dual processor Linux P.C., two commodity projectors with circular polarizing filters, an 8-foot by 6-foot rear projection screen, lightweight polarized glasses, and a hand-held radio remote control. I consider this the most valuable visualization of this model. Virtual Reality (VR) allows for stereo viewing of the model, viewer-centered perspective, control of the model s orientation, and large angles of view. E20

5 In order to capture the additional detail, more polygons would be needed. This could exacerbate another limitation of the current model: its high polygon count (2.5 million). However, as computer processors become more advanced and memory capacity becomes less expensive and more accessible, this becomes less of an issue for future projects. Currently, to address the high polygon count, three different models at varying polygon densities have been generated. Upgrades to the software used in this project, as well as entirely new programs, have made significant strides in streamlining the process described above and offering alternative techniques. Figure 12. Anaglyph 3D image (superior view) created for use with red and cyan glasses. The VR environment enables an appreciation of complex 3D structures from any angle. This unrestrained point of view and the ability to view structures of the brain in stereo is valuable, and several anaglyph images have been prepared for use in a lecture hall setting for neuroanatomy classes at UIC (Figure 12). In the C-Wall VR System the brain model can be interactively explored by one to twenty users locally, and the application can be shared by participants using VR systems anywhere in the networked world. The application provides the opportunity for remote collaboration, teaching, and the sharing of anatomical expertise. Conclusions Utilizing the expertise of anatomists and medical illustrators, hand segmentation of serial section data images can produce a high resolution, accurate, polygonal model of the brain. This model has been realized in a variety of media including still images, animations, VRML, and Virtual Reality. Still images, anaglyphs, animations, and VRML models can be used in large audience presentations and on the web. The use of networked interactive VR technology provides improved visualization, capabilities for remote learning, and opportunities for collaboration. As previously stated, the completed model has limitations. A cerebellum with a continuous polygonal surface was not obtained using the captured data. The discontinuous model was used as a template for the final cerebellum. A contributing factor was capturing the data from every other cryosection. In future projects, using every consecutive slice and testing other averaging algorithms could result in finer detail. There are also issues with areas of the cerebrum, most notably the left lateral sulcus. The definition and topology of this key landmark are not as clear as desired. Acknowledgements Mary Rasmussen, founder and director of the VRMedLab, aided in the creation of the model as it progressed through the various programs. Dr. Conwell Anderson, Associate Professor in the Department of Anatomy and Cell Biology, helped ensure the accuracy of the hand segmentation, provided a review of the final models, and offered insight into the various uses for the model and future improvements. Ray Evenhouse helped with Mimics and offered input on creating the polygonal model from the hand segmented data. Greg Blew helped provide techniques to realize a clear and concise way to describe the relationship of the cryosections to the resulting 3d model. Zhuming Ai developed the box-averaging filter. Bei Jin, working in the VRMedLab, imported and created an interface for interacting with the model. I would also like to thank Maite Suarez-Rivas for her help and motivation. Author Kevin Brennan received B.A. in Biology from Illinois Wesleyan University in 2003 and completed his Master of Science in Biomedical Visualization at the University of Illinois at Chicago in May At UIC he focused on animation and building models from cross sectional data. He worked at Argosy Publishing Visible Body for more than six years as a medical animator and later as the Senior Content Developer for the Visible Body. He is currently a Clinical Assistant Professor at UIC in the Biomedical Visualization Program. kevinmbrennan@gmail.com E21