Photorealistic Volume
Rendered Anatomical Atlases and
Interactive Virtual
Dissections
of The Dissectable
Human(TM)
John P. Kerr, Michael Sellberg,
Peter Ratiu,
Darren Knapp, and Christina Caon
Engineering Animation, Inc. 2321
North Loop Drive, Ames, IA 50010
Abstract
Anatomical atlases have historically consisted of artistic illustrations based on observations from multiple cadaver dissections. More recently, photographic atlases of cadaver dissections have gained favor due to their improved anatomical accuracy and detail [1,2]. The digitized cryosection data from the National Library of Medicine's Visible Human project has enabled a new approach to anatomical atlases. Much as photographic atlases added a higher degree of accuracy and information, the Visible Human data sets have provided a new level of dimensional and relational information to be realized. These detailed, digitized cryosectioned images provide the means, through three-dimensional (3D) visualization techniques [3], to create a unique, interactive approach to anatomical information and education. In this paper, photorealistic volume renderings of system-based and region-based 3D anatomy from the Visible Human Male and Female cryosection data sets are presented.
When using a digitized voxel cadaver, the in situ form of a structure is maintained, as shown in Plate 1 . In this image the 3D orientation of the large intestine is shown in its natural position, relative to the skeleton, with transparency effects added to the skeletal structures to allow clearer visualization of the organ. Due to the invasive nature of cadaver dissections, such positioning cannot be readily observed. Likewise, Plate 2 shows the in situ relationship of the internal and external portal system to the kidneys and spleen. Here, again, such relationships cannot be observed in traditional dissections. While the full scientific research potential of the digitized cadavers is still being discovered, the educational potential is already being realized.
Volume rendered images of the Visible Human Male and Female have been used to create two forms of new anatomical products; college and professional print atlases, and system-based and region-based interactive CD-ROMs. These atlases, produce by Engineering Animation, Inc. (EAI) and published by Mosby Year Book, provide unique, photorealistic, views of anatomical systems and structures, and their in situ relationship to other systems and structures. The two print atlases provide labeled 3D views of anatomical structures segmented from the full-body data voxels, Plate 3 . Since they are produced from a digital data set, these images provide information which cannot be produced from cadaver dissections, and cannot be accurately represented by illustrative atlases, Plate 2 . On the other hand, the CD-ROM atlases from The Dissectable Human™ series, enable user-interaction with a variety of 3D rotations and virtual dissections of the digitized cadaver. With the CD-ROMs another novel educational approach to teaching anatomy has been created. The student and professional can observe structures and organs in their in situ orientation rotated 360 degrees about a central axis. In addition, many spatial relationships are represented via interactive peel-aways of surrounding structures or of the target structure itself, Plate 4 .
The accuracy and realism of the custom volume visualization techniques employed are well suited for the production of these unique print atlases and CD-ROMs. However, the volume rendering routines, image manipulation tools, and segmentation methods [3] that have been developed to create these atlases have many other technological and computational anatomy applications. As an example, the segmented methods used to isolate individual structures in all the anatomical systems were created such that surface geometries of the structures could be automatically created. In addition, EAI has developed techniques to accurately map actual textures from the Visible Human cryosection data onto the surfaces. The result is a geometric anatomy database with actual textures that can be used in a variety of virtual reality and surgical simulation applications. Hence, while the present applications of the Visible Human Project data are important and of great educational benefit, they undoubtedly are paving the way for some extraordinary applications in the near future.
1. Introduction
Anatomical atlases have historically consisted of artistic illustrations based on observations from multiple cadaver dissections. More recently, photographic atlases of cadaver dissections have gained favor due to their improved anatomical accuracy and detail [1,2]. The National Library of Medicine's Visible Human project and technology developed by Engineering Animation, Inc. (EAI) have taken the study of anatomy to a new dimension, and have enabled a new form of three-dimensional (3D) anatomical atlas to be created. The Visible Human project provides, for the first time, single whole-body specimens of digitized anatomical cryosection data. These detailed, digitized cryosection images provide the means, through three-dimensional (3D) visualization techniques [3], to create a unique, interactive approach to anatomical information and education.
By developing automatic and semi-automatic segmentation methods, and unique ray-cast, lighting routines, EAI has been able to accurately volume render photorealistic, 3D views of structures from all ten systems of the body. With the added dimension of motion, the anatomy can be interactively displayed, rotated, and dissected on The Dissectable Human(TM) CD-ROM atlas. This type of user-definable interactive atlas differs greatly from atlases done in the past, since they have typically consisted of artistic illustrations or photographs from cadaver dissections. Photographic atlases do offer a higher degree of resolution but studying the 3D anatomy of a virtual cadaver offers the accessibility to many different views and in situ combinations of anatomy that cannot be produced via traditional cadaver dissection. Interacting directly with organs and anatomical structures enables the user to study them in relation to their surroundings and provides a unique educational tool which certainly can contribute to a more comprehensive and detailed understanding of anatomy.
These photorealistic volume rendered atlases are just the beginning of many new and innovative realms of medical education and scientific research that are being realized from the Visible Human data. Presently, real-time interaction with volume data of this magnitude is not possible. However, real-time interaction with 3D surface models created from the cryosection data can be done today on virtually any computer platform. By utilizing the segmentation information created for the volume rendered atlases, EAI has been able to produce a complete anatomy database from the Visible Human male data. In addition, we are presently generating a photorealistic texture database for the surface models based on the volume rendered images. The consequence of which is, a true interactive and complete anatomy database of dimensionally accurate size and shape, and of visually accurate appearance. Accuracy and realism must be at the forefront of any medical visualization endeavor. This database will enable the next steps toward visually convincing virtual reality environments, where educators and medical professionals can interact with realistic anatomy in variety of applications ranging from gross anatomical and physiological study to surgical planning and training [4].
2. Methods
Image segmentation routines provide a means through which a computer can distinguish between various anatomical structures of the body [3]. Ray tracing routines provide the method by which a computer can produce multiple 2-D volume views of a 3-D voxel array [5]. By applying image segmentation techniques and ray tracing routines to the Visible Human digitized cadaver data it is possible for the in situ form of anatomical structures and systems to be maintained. In Plate 1 the 3D orientation of the large intestine is shown in its natural position relative to the skeleton, transparency effects are added to the skeletal structures to allow clearer visualization of the organ. Accurate segmentation of structures has made it possible for us to produce photorealistic volume rendered stills and animations of select combinations of the human anatomy. EAI has developed and implemented three types of segmentation routines specificly to identify anatomical structures in the Visible Human cryosection data sets. These segmentation methods are; automated threshold segmentation, semi-automated NURBS curve fitting, and multimodal registration and segmentation.
A. Segmentation
Automated threshold segmentation is used on structures which have uniquely identifiable features, such as shape, color, and edge. To date we have been able to automatically segment the bone, cartilage, muscular system, subcutaneous fat, and the superficial vessels. When we manually define a region-of-interest for the computer, additional local automatic segmentation of structures is possible. Furthermore, if we provide the computer with hints to the structure we want to segment, by isolating it on one axial image our routines have been able to accurate delineate even more structures from the voxel data set. The segmentation routines employed involve multi-variable thresholding based on boundary characteristics [6]. The multi-variables are the red, green, and blue channels of the cryosection images. Histograms for each color channel are calculated and thresholds for particular structures are identified and applied to separate tissues.
For structures that cannot be fully separated from the rest of the image, semi-automated segmentation methods are used. Semi-automation refers to manual segmentation that is produced by drawing a non-uniform rational B-spline (NURBS) curve around a structure that is to be segmented, Plate 5. NURBS curve are defined by our anatomists by placing interpolation points around a structure. The curve interpolates through these points, essentially outlining the structure being segmeneted. This greatly reduces the time it takes to manually segment, a structure that may consist of tens-of-thousands of pixels which may only requires only the placement of 10 to 20 interpolation points around a structure for it to be segmented with the NURBS curve. Once a structure has been segmented on one slice it can be copied to the neighboring slice where minimal edits are required to segment the same structure. The NURBS segmentation tools are a in-house part of EAI’s commercial 3D CAD geometry manipulation software, VisModel(TM), Plate 5. In addition, VisModel(TM) allows labels to be attached to structures and substructures which makes it possible for the volume rendering routines to differentiate between structures and substructures that may have similar color tones. For example, in the muscular system each individual muscle has been identified and labeled in the segmentation data. This makes it possible for a particular muscle or a group of muscles to be rendered separately or in association with surrounding structures such as the skeletal system, Plate 5.
Multimodal registration is another means by which segmentation can be accomplished. By using the frozen computed tomography (CT) data other structures in the Visible Human cryosections have been identified. Using histogram specification [7] the hard tissue structures, specifically bone, have been separated from all soft tissue structures in the CT data set. Furthermore, the individual skeletal structures have been identified and labeled within the segmentation images. This has enabled automated segmentation of of the bony structures, by name, using single-variable thresholding in the CT slices. The segmented structures have subsequently been registered to the axial cryosection data. The necessity of registration is due to the differences in resolution between the CT data and cryosection data and because five separate scans were required to produce a full body set of CT images. These factors have made registration difficult with the Visible Human data set since the cryosection data voxel and CT data voxel are rotationally offset in the x-, y-, and z-planes, Plate 6. Transform matrices to register CT segmentation data to the cryosection data have been calculated using a homologous feature-pair matching technique [8].
B. Volume Rendering
The main algorithm components of volume rendering are ray tracing the volumetric data set, and the application of light ray properties to the scan rays. Light ray properties, such as shadowing ,Plate 4, highlighting, Plate 7, and transparency, Plate 8, are applied to the 2D multi-angular volume rendered images which greatly enhances the 3-D effects. These optical features are controlled by using light interaction equations for absorbtion, emission, and scatter [9] and are applied to the rays being projected through the digitized voxel cavader. As can be seen in Plate 9, there is a considerable difference in the quality of the image when the light equations are applied to the ray-casted image as opposed to when they are absent The first image in Plate 9 shows the full musculoskeletal system ray traced assuming a flood light model. This images looks very flat and gives virtually no 3D feel to the anatomical structures. In the second image an optical shading equation has been applied to simulate the addition of a light source at a 45 degree angle. The shaded image is qualitatively more three-dimensional than the flood lighted image, and is considerably more informative since the image now has depth, and the orientation of the anatomical structures is more apparent.
The cryosection images can be scanned at incremental angles over 360 degrees. By combining these views the anatomical structure can be given motion, Movie 1, which improves the 3D quality of the volume rendered images. Since the scanning of cyrosections is in the axial plane, the images produced will be around the z-axis of the data voxel. It is posssible to resample the data voxel in the sagittal, coronal or an oblique plane to generates views around the x-axis, y-axis or any angle relative to the data voxel. The combination of view angle manipulation and optical properties make the in situ relationships better visualized and understood.
With all data sets there are resolution limitations that may result in aliasing of the images. Aliasing occurs because the volume data set is a discrete cuboidal volume. Therefore, it cannot be resampled in oblique planes without loosing information. Antialiasing methods help minimize the amount of information that is lost in the resampling of the discrete data set. Many antialiasing techniques exist to reduce the effects of aliasing. Several of these can add significant computational time to renderings, since they use oversampling [10] or stochastic modeling [11]. In the images shown here, Plate 10, we have used a customized gradient-based interpolation method [12]. The first image was volume rendered without antialiasing, and contouring and striping effects are very apparent. The second image was rendered using our antialiasing routines. Although aliasing patterns can still be seen, they are greatly reduced.
3. Discussion
Anatomical atlases have traditionally consisted of artistic illustrations. More recently photographic atlases of cadaver dissections have gained favor due to their anatomical accuracy and detail. Much as photographic atlases have added a new degree of accuracy and information, so too does the Visible Human data voxel enable a new level of dimensional information and educational value to anatomical visualization to be attained. Since volume renderings are produced from a digital data set, the images provide information which cannot be produced from cadaver dissections, and cannot be accurately represented by illustrative atlases.
EAI has worked with Mosby Year Book to publish the volume rendered images of the Visible Human Male and Female. These volume rendered images have been used to create two forms of new anatomical products; college and professional print atlases, and a system-based interactive CD-ROM. These learning tools are unique because they provide photorealistic views of anatomical systems and structures, and their in situ relationship to other systems and structures in the body. The two print atlases work through the body by its systems. They feature labeled 3D views of individual structures and organs, Plate 11, actual combinations of structures and their spatial association, which have never before been seen in many cases, Plate 2, and virtually dissected views of a variety of regional anatomy, Plate 4. 3D volume renderings of anatomy provides a unique learning tool, by showing organs and their in situ relationships a better understanding of gross anatomy can be achieved. On the other hand, the CD-ROM atlases from The Dissectable Human(TM) series, enable user-interaction with a variety of 3D rotations and virtual dissections of the digitized cadaver,Plate 8 . With the CD-ROM another novel educational approach to teaching anatomy has been created. The student and professional can observe structures and organs in their natural orientation rotated 360 degrees about a central axis. In addition, many spatial relationships are represented via interactive peel-aways of surrounding structures or of the target structure itself,Plate 12.
Photorealistic volume rendering using our unique lighting methods have enabled the production of the first three-dimensional anatomical print atlases and interactive CD-ROM atlas, based on the Visible Human male cryosection data. Given the size of the data set, true interactive manipulation of this 3D voxel is not yet possible. Specialized hardware for real-time interaction with volume data continues to be developed, but remains limited by the size of the volume data sets. Consequently, direct 3D interaction and manipulation of anatomy is only possible via surface rendering, rather than volume rendering. However, the hybridization of the photorealistic appearance of volume rendering with the real-time speed of surface rendering can provide the next step toward "true" interactive anatomy and subsequently, realistic surgical simulation and the development of other virtual reality (VR) environments.
Accuracy and realism are the cornerstone of any anatomical visualization application. Surface models of anatomical structures and organs provide a framework on which direct user-interactive applications with anatomy can readily be achieved. Such models are the basis for developing a medical VR environment. The NURBS segmentation techniques we have developed support both the volume rendering applications as well as allow for the automatic generation of surface models [3]. Within themselves, these models provide accurate dimensional representation of anatomy, but not a realistic visual representation of anatomy. However, EAI has developed surface mapping techniques by which the actual surface information can be stripped out of the cryosection data set, or volume rendered and remapped to create accurate and realistic texture maps, Movie 2. In turn, this surface model database, with texture maps can and will be used in a wide variety of "virtual human" applications, including surgical simulation [4].
In addition to creating visual realism in our anatomy database for VR environments, we have also been working toward modeling the realistic interaction and manipulation of the surface geometries within this environment [13]. For surgical simulation applications, the environment is the anatomical surface database and texture database. The realism in an anatomy environment involves modeling functional aspects and deformation characteristics of structures. EAI is presently pursuing this realm of VR, through biomechanical modeling of anatomical entities [4]. Having handled the problem of the realistic "look" of the anatomy database, the biomechanical simulation will enable a realistic "feel" to a VR environment. As a result, the biomechanical properties of the anatomical structures in the database will provide the mechanism for interfacing the VR environment to the real world.
Once the "look" and "feel" of the VR environment is complete, what remains to be developed are the mechanisms through which the simulator user can interact with the VR environment [14]. These interfaces require the development of many additional features such as force-feedback and haptic feedback in order to maintain realistic interaction with the environment [15]. This primarily means tying in the functionality of the interface devices with the biomechanical properties of the environment. To date, EAI has only done cursory work with direct feedback interface devices. However, we have developed virtual bronchoscope and hysteroscope simulators, that provide fly-through, collision detection, and deformation characteristics with endoscopic devices in a VR environment. Our next step is to integrate other VR tools such as scopes and probes with force feedback, and gloves with haptic feedback to the virtual anatomy database with the realistic textures and biomechanical properties that we have developed.
4. Conclusions
The accuracy and realism of the custom volume visualization techniques employed are well suited for the production of these unique print atlases and CD-ROMs. However, the volume rendering routines, image manipulation tools, and segmentation methods [3] that have been developed to create the anatomical atlases have many other technological and computational anatomy applications. As an example, the segmented methods used to isolate individual structures in all the anatomical systems were created such that surface geometries of the structures could be automatically created. In addition, EAI has developed techniques to accurately map actual textures from the Visible Human cryosection data onto the surfaces. Likewise, we have began to model the biomechanical characteristics of structure geometries and incorporate them into our anatomy database. The result is a geometric anatomy database with actual textures and physical properties that can be used in a variety of virtual reality and surgical simulation applications. Hence, while the present applications of the Visible Human Project data are important and of great educational benefit, they undoubtedly are paving the way for some extraordinary applications in the near future.
[1] Gosling, J, et al., Human Anatomy, Gower Medical Publishing:London, 1991.
[2] Han, M., and C. Kim, Sectional Human Anatomy, Igaku-Shoin:New York, 1995.
[3] Kerr, J., P. Ratiu, and M. Sellberg,
"Volume Rendering of Visible Human Data for an Anatomical Virtual
Environment" Health Care in the Information Age. (eds.) S.
Weghorst, H. Sieburg, and K. Morgan,
IOS Press:Washington,
D.C.
1996, pp. 352-370.
[4] Sellberg, S, et al. 1994. "An automated virtual environment for computer-aided instruction and biomechanical analysis" Interactive Technology and the New Paradigm for Healthcare. IOS Press: Washington, DC.
[5] Samet, H. and Webber, R. 1988. "Hierarchial data structures and algorithms for computer graphics" IEEE Computer Graphics and Applications. 8(6): 59-75.
[6] Liu, J. and Yang, Y. 1994. "Multi-resolution color image segmentation" IEEE Trans. on Pattern Analysis and Machine Intelligence, 16(7): 689-700.
[7] Gonzalez, R. and Woods, R. 1993. Digital Image Processing, Addison-Wesley: Reading, MA.
[8] Meyer, D. et al. 1995. "Simultaneous usage of homologous points, lines, and planes for optimal, 3-D, linear registration of multimodality imaging data" IEEE Trans. on Medical Imaging. 14(1): 1-11.
[9] Max, N. 1995. "Optical models for direct volume rendering" IEEE Trans. on Visualization and Computer Graphics, 1(2): 99-108.
[10] Mitchell, D. 1987. "Generating antialiased images at low sample densities" Computer Graphics. 21(4): 65-69.
[11] Dippe, M. and Wold, E. 1985. "Antialiasing through stochastic sampling" SIGGRAPH '85. 19(3): 69-78.
[12] Bright, S. and Laffin, D. 1986. "Shading of solid voxel models" Computer Graphics Forum, 5(2): 131-138.
[13] Baraff, D. 1995. "Interactive Simulation of Solid Rigid Bodies" IEEE Computer Graphics and Applications, 15(3): pp. 63-75.
[14] Stytz, M.R. 1996. "Distributed Virtual Environments" IEEE Computer Graphics and Applications, 16(3): pp. 19-31.
[15] Hirose, M., K. Hirota. 1995. "Providing Force Feedback in Virtual Environments" IEEE Computer Graphics and Applications, 15(5): pp. 22-30.
