P. Gingins, P. Kalra, P. Beylot, N. Magnenat Thalmann
MiraLab, CUI, University of Geneva
J. Fasel
Department of Morphology, CMU, University of Geneva
In this paper, we present our work on building a comprehensive human model using the Visible Human Dataset (VHD) for the European project CHARM (Comprehensive Human Animation Resource Model). In contrast to the unstructured and static data provided by the VHD, the CHARM project aims at providing a structured and dynamic model of the human body. Such model includes modeling of the geometry of important organs, their inter-relationships, and their mechanical behavior. Though our present work is concentrated to the modeling of the left upper limb, the methodology and tools can be applied to the modeling of any other part or the whole body.
We also outline the benefits as well as the difficulties encountered using the VHD, in our case the Visible Male Dataset.
The Visible Human Dataset (VHD) provides complete visual insight of the entire human body. However, it is totally unstructured and static. Its primary uses are therefore limited to visualization of the contained data, for example 2D and/or 3D digital atlas. Some efforts have already been made in using the data for volume rendering, visualization, and navigation [6, 8, 10, 11].
In contrast, CHARM (Comprehensive Human Animation Resource Model), a project involving different teams throughout Europe, aims at developing a model of the human body, which is both structured and dynamic. It is structured in the sense that bones, muscles and skin are individually identified with their surfaces reconstructed. Furthermore, their inter-relationships and mechanical attributes are added, forming what we call a topological model [2]. The model is dynamic as each reconstructed organ may be moved and deformed keeping its mechanical relationships with others. Mechanical simulation can be performed, including bone motion, as well as muscles and skin deformation, thus providing a realistic behavior of the human body.
The model is restricted to the left upper limb from the wrist to the shoulder. The upper limb was chosen for its great importance for many human activities, and for its mechanical complexity. The shoulder joint is considered as one of the most complex articulations in a human body. However, the methodology and techniques can be applied to any other body part or the whole body.
We describe here our work using the VHD for the CHARM model. We also outline the benefits as well as the difficulties encountered using the Visible Male dataset. We have used both the frozen CT and anatomical cryosections from the Visible Male as a basis for building this model.
Figure 1 gives the overview of the whole process for building a topological model using medical images. It shows the modeling pipe line from acquisition of images to the construction of the model. The medical images acquired from sources like MRI, CT, or cryosections of VHD are made readable, and then preprocessed and filtered if necessary to accentuate the contained information. Then interpretation of images is done for the organs of interest, where the useful regions are labeled and identified. The identification involves definition of contours of the elements on the image slice. These contours may be refined to fit to the cross-section of the organ of interest by using matching techniques. A 3D surface is obtained by joining the 2D contours. The 3D surfaces of bones are further used for adding topological information for other elements like muscles, joints, etc. A data structure representing structural and topological information is provided to build the topological model for a given body part. Information for mechanical and physical attributes for an element can also be added.
The overall modeling process can be basically decomposed in two major parts: reconstruction and topological modeling. In the next two sections we present the details for the two parts.
Figure 1: Modeling overview.
Voxel-display representation though provides useful and sophisticated visualization, offers little capability for treatment of the data in a manner consistent with their physical properties. This is due to the lack of geometry. Surface models offer geometrical structure in the representation [1]. The first phase of our work is to identify the spatial extents of organs or elements of interest, and to provide their surface models. These are the bones, the muscles, the skin and the deep fascia.
The reconstruction process may be divided into four steps: preprocessing, labeling, 3D reconstruction, and postprocessing.
Preprocessing consists of image filtering, cropping and resolution reduction so that the data can fit into the main memory. The huge size of the VHD cryosection images forces us to make a trade off between selected volume and data resolution. An organ can nonetheless extend across different working image volumes. Thus, working with different subregions or volumes of the complete dataset implies taking care of the consistency of the units. A working stack is described with its physical size and position relative to the original dataset. The definition of contours and reconstructions also adhere to these units.
The labeling involves identifying and delineating each organ of interest in the cross-sectional images. Our experienced has revealed that interpretation of cross sectional images of the human body in particular of muscles and fascia requires specialized anatomical knowledge. We have concentrated on four basic elements for the analysis of images of Visible Male data for the left upper limb. These are: the bones, the muscles, the deep fascia and the skin. Figure 2 illustrates these elements on a cryosection of the Visible Male dataset. For bones, we use primarily CT images due to their high contrast for bones. The anatomists perform the identification of other organs on the cryosections. Despite the high resolution of cryosections, distinction between adjacent muscles may rely on texture, fiber orientation, or even sometimes on pure anatomical knowledge. In our context the term deep fascia refers to the sheath of dense connective tissue surrounding the muscle. Thus, we perform segmentation of the deep fascia by tracing the surface of the muscle. In the absence of underlying musculature, some indirect criteria such as epifascial veins are used. The segmentation of outer skin can be performed using the CT or even the cryosections as the embedding medium is in clear contrast to the color of the skin.
Figure 2: Cryosection illustrating the elements for the present study (Note: identification of fascia is not obvious).
We have developed a labeling tool (Figure 3) for this task which provides interactive facilities for identification of contours. Snakes [5] enable semi-automatic delineation when contrast is high enough. Other features of our labeling tool include contours copying between different slices, 3D visualization of reconstructed organs together with the slices for immediate visual feedback, other axis visualization and contouring.
Figure 3: Labeling tool.
After the identification and labeling of contour, the successive cross sectional contours of the organ of interest are joined using the program nuages [4]. This yields a 3D surface of the anatomical object of interest.
Then we correct reconstructions made from the cryosections from misalignments present in the original Visible Male data. Finally, we assemble the reconstructions from the different image modalities. The transformation from cryosections space to CT space is obtained using corresponding features in the two modalities and Singular Value Decomposition (SVD) is employed for matching. As already reported by others [6], a single affine transform proved to be sufficient for our needs.
Figure 4 gives the overview of the tasks of reconstruction and the postprocessing.
Figure 4: Reconstruction and postprocessing.
Figure 5 illustrates the reconstructed elements of the left upper limb using the VHD. It includes all the vertebrae, ribs, the bones of the shoulder and the forearm, major muscles of the shoulder and the elbow, the skin and the deep fascia for the elbow area. The deep fascia is used for computing local thickness of the skin for further modeling.
Figure 5: Reconstruction results.
A static surface model with no notion of constraints, no perception of dynamics, and no knowledge of muscle attachments would restrict its utility merely to visualization. In our topological model, we provide these properties and the functional relationships between different elements. The intention is to build working models with capabilities for medically acquired and bio-medically valid musculo-skeletal systems [3,7]. In our context, topological information is primarily determined by the mechanical model based on the mechanical properties of the real anatomical structures. The definition of the mechanical model involves the specification of kinematics of the joints, the dynamic parameters like mass and inertia matrix of the bones, and the lines of action and force generating parameters of muscles [9].
We have developed an object-oriented database and a corresponding library for its creation, editing, and access. The hierarchy of its classes is shown in Figure 6. This database provides models for bones, mechanical joints between them, ligaments, skin, and muscles.
Figure 6: Class tree.
Muscles include attachments and actions lines. Attachments are the area where muscles are bound or connected to the bones. Together with the origin and insertion attachments classically considered by anatomy, we model also the areas on bones which constrain the mechanical action of some specific muscles. These are called the guides. Action lines model the lines along which link spindles act. Each action line is a polyline getting from an origin to an insertion attachment, possibly through any number of guides. Tendons in our case are considered as being part of the muscle. Figure 7 demonstrates the definition of action lines and the respective attachments.
Figure 7: Action lines and attachments.
Particular mechanical parameters are also part of the database, such as muscle activation and physiological cross-sectional area (PCSA).
Joints are also modeled to specify the mechanical dependency or constraint between bones. Figure 8 shows how a joint is defined in terms of the transformations between the two bones. The frames Mj1 and Mj2 specified in the local coordinates of the respective bones define a joint. The dependency between the two bones is defined by the transformation matrix Mj. Three types of joints are modeled: Ball and Socket joint having three degrees of freedom , Hinge joint with one degree of freedom , and Ellipsoidal contact joint with 5 degrees of freedom. A distinction is made between the joints for the left and the right side, adhering to the convention in medical use.
Figure 8: Definition of joint.
A 3D interactive tool (topological modeler tm, shown in Figure 9), is provided to build the data base with structural, topological and mechanical information. The users can specify the kinematics parameters (angular limits) of joints, the dynamic parameters like inertia matrix of the bones, and the lines of action and force generating parameters of muscles. The modeler provides extensive 3D visualization and direct interaction and manipulation. The problem of assembling the skeleton when the elements (bones) are scattered is also considered. However, VHD does not need this assembling as the data for the complete body is consistent and obtained for a single particular position.
Figure 9: Topological modeler tm.
Figure 10 shows actions lines defined for the Biceps muscle. The three types of joints constructed in tm are shown in Figure 11. tm with its user friendly interface and direct 3D manipulation makes the task of modeling easy and understandable by experts of other fields (medical doctors and biomechanical engineers).
Figure 10: Action lines for the Biceps muscle.
Figure 11: Joint defined in tm: Ball & socket, hinge, and ellipsoidal contact.
tm is used for creating a complete topological and mechanical model of the human left shoulder and forearm. Figure 12 shows the constructed model with the joints, the attachments and the action lines. The design of the model itself is described in [9].
Figure 12: Topological model for the left upper limb.
The VHD has proven to be of great value for our purpose owing to: the high resolution, clearer distinction between organs compared to other non-invasive modalities, complete coverage of the body, consistency due to a single position, and availability in several modalities. It has provided a good reference (standard) model for building our data base.
Even if the VHD is of a remarkable high quality, it has some minor imperfections: missing or damaged slices, and misalignments depicted in Figure 13 and Table 1 (not present in the Visible Female Dataset). In addition, CT images have lower resolution and are incomplete(e.g., part of elbow is missing). This may be taken into account for future similar data acquisitions. The matching of the different modalities should also be planned as early as possible in the process of acquisition, by providing some frame of reference easily usable. Better identification and extraction of elements may then be achieved. CT enables us to easily separate between bones on one side and ligaments, tendons, and fat matter on the other side.
Figure 13: Frontal view showing misalignments.
Table 1: Estimation of misalignments.
We have also encountered some difficulties inherent to its size: the huge data requires a large amount of disk space or tapes resulting in long processing time for even simple tasks. Different levels of resolution were prepared, satisfying the trade off between the volume size and the image resolution.
Beyond static visualization of the interior of the human body, medical imaging may be used for modeling dynamics, physical process simulation, analysis of movements, and validation of movements. Such a comprehensive model can be applied to surgery simulation and planning, orthopedic analysis, and medical education, both elementary and professional. We have developed the necessary tools with a convivial user interface to obtain a complete model of the shoulder and forearm. The VHD has proven to be very useful in that process.
This work is supported by ESPRIT project 9036 CHARM. We are grateful to other partners of the project: Swiss Federal Institute of Technology - Lausanne, Ecole des Mines de Nantes, University de las Islas Baleares, University of Karlsruhe and Technical University of Lisbon.
1. HH Baker (1990), Surface Modeling with Medical Imagery, in 3D Imaging in Medicine (Eds KH Hoehne et al.) pp. 277-288, NATO ASI Series, Vol. F60, Springer Verlag.
2. P Beylot, P Gingins, P Kalra, N Magnenat Thalmann, W Maurel, D Thalmann, J Fasel (1996), 3D Interactive Topological Modeling using Visible Human Dataset, Proc. Eurographics '96, Computer Graphics Forum, Vol. 15, No. 3, pp. C-34-C-44.
3. SL Delp, JP Loan (1995), A Graphics-Based Software System to Develop and Analyze Models of Musculoskeletal Structures, Comput. Biol. Med., Vol. 25, No. 1, pp. 21-34.
4. B Geiger, Three Dimensional Modeling of Human Organs and its Application to Diagnosis and Surgical Planning, Ph.D. Thesis, INRIA, France, 1993.
5. M Kass, A Witkin, D Terzopoulos, Snakes: Active Contour Models, International Journal of Computer Vision, 1987; 1(4):321-331.
6. J Kerr, P Ratiu, M Sellberg (1995), Volume Rendering of Visible Human Data for an Anatomical Virtual Environment, h ttp://www.eai.com/interactive/dhuman/paper/paper.HTM.
7. E Kreighbaum, KM Barthels (1990), Biomechanics: A Qualitative Approach for Studying Human Movement, Third Edition, Macmillan, New York.
8. W Lorensen (1995), Marching Through the Visible Man, Pro. IEEE Visualization '95, pp. 368-373, (also on the web http://www. crd.ge.com/esl/cgsp/projects/vm/).
9. W Maurel, D Thalmann, P Hoffmeyer, P Beylot, P Gingins, P Kalra, N Magnenat Thalmann, A Biomechanical Musculoskeletal Model of Human Upper Limb for Dynamic Simulation, Proc. Eurographics Workshop on Computer Animation and Simulation '96, Springer Verlag, pp. 121-136.
10. D Sims (1996), Putting the Visible Human to Work, IEEE Computer Graphics & Applications, Jan 1996, pp. 14-15.
11. U Tiede, T Schiemann T, KH Hoehne (1996), Visualizing the Visible Human, IEEE Computer Graphics & Applications, Jan 1996, pp. 7-9.
Prof Nadia Magnenat Thalmann
MiraLab, CUI, University of Geneva
24 rue Général Dufour
CH-1211 Geneva 4 (Switzerland)
e-mail:
thalmann@cui.unige.ch
URL: http://miralabwww.unige.ch/