Skip to content

SDSC Homepage SDSC Homepage San Diego Supercomputer Center Contact EnVision NPACI: A Leading Edge Site San Diego Supercomputer Center
  A Geosciences Network or Understanding the Whole Earth

A Science Environment for Ecological Knowledge

Hot Commodities

How to Run an Antimatter Generator

Modeling the Matrix

Morphin’ Lizards!
SDSC Homepage
SDSC Researcher Likes Augmented Reality View
NPACI Releases Updated Cluster Software
NSF Announces Cyberinfrastructure Initiative
Fran Berman Appointed to UCSD Endowed Chair
Phil Andrews and Jay Boisseau Elected to NPACI Executive Committee
Primate Resource Online at SDSC
SDSC Announces New IBM Data-Oriented Supercomputer

by Merry Maisel

Osteoporosis is a disease characterized by low bone mass and structural deterioration of bone tissue, leading to bone fragility. Left untreated, osteoporosis can progress silently until a bone breaks. Sufferers have an increased susceptibility to fractures of the hip, spine, and wrist, according to the National Osteoporosis Foundation.
Osteoporosis is a major public health threat for an estimated 44 million Americans.

• Of the 10 million individuals estimated to have osteoporosis, eight million are women.

• About 34 million Americans, or 55 percent of the people 50 years of age and older, have low bone mass, which puts them at increased risk of developing osteoporosis and related fractures.

• Significant risk has been reported in people of all ethnic backgrounds.

• While osteoporosis primarily affects postmenopausal women because hormonal changes accelerate bone resorption, it can strike women or men at any age.
The estimated national direct expenditures (hospitals and nursing homes) for osteoporotic and associated fractures were $17 billion in 2001 ($47 million per day), and the cost is rising.

To biomechanical engineers, bones are exquisitely complex structures, more breathtaking than Gothic cathedrals or the world’s longest bridges. While bone shapes seem strange and irregular, it is their internal structure that fascinates and that many scientists are attempting to model computationally. "From a micromechanical point of view, bone architecture is wonderfully various, responding to compression and tension in a nonlinear fashion, and presenting very subtle challenges to the modeler," said Tony Keaveny, a professor of mechanical engineering and bioengineering who directs the Orthopaedic Biomechanics Laboratory at UC Berkeley.

Keaveny and his group have been using the resources of the National Partnership for Advanced Computational Infrastructure (NPACI) to develop accurate, three-dimensional, finite-element models of human bones. Their models are uniquely efficient, and their work will aid researchers, clinicians, and physicians seeking ways to prevent and treat a panoply of bone diseases. Among these, the most serious in economic and social terms is osteoporosis.


Bones have been affected by many diseases throughout human history, and orthopaedic medical practices are first recorded in ancient Egypt. Prehistoric human fossil bones show that their bearers suffered from diseases still seen today.

A misconception about bones is that, once grown, they become inert or "dead" tissue. Actually, bones are living organs. Bone cells are of three types. Osteoblasts come from the bone marrow and form new bone. Some osteoblasts turn into osteocytes after new bone is formed and become surrounded by the new bone tissue. These trapped osteocytes send out long branches that connect to other osteocytes and are thought to sense fluid stresses or cracks in the bone and help direct where the third type of bone cell, the osteoclasts, will dissolve bone.

Like all other cells in the human body, bone cells are born, live, and die, and the whole collection of cells in our bones participates in growth and degeneration.

"These complex physiological facts demand a change in the usual paradigm of biomechanical engineering research," said Keaveny. "In place of testing the assumption that one or another engineering metric–density, porosity, elasticity–may govern or explain bone failure mechanisms, we look instead at the micromechanical behavior of the tissue as a whole and model it to derive appropriate metrics as they emerge from our simulations. Scientists have long recognized that bone density alone is not the sole indicator of risk for fracture, or even bone strength itself, but the multiple factors involved in both health and disease need to be examined on the microscopic level." Since risk prediction depends on the simultaneous interactions of these factors, the group’s long-term research goal is to develop techniques that allow precise individual predictions–and precise individual interventions–to prevent bone failures.

The group studies the two main types of bone tissue: cortical and trabecular. Cortical bone is the more compact tissue that forms the outer shell of bones; trabecular bone is the porous, spongy tissue found within the spine, hips, knees and other joints. Trabecular bone is the primary load-bearing biological tissue in these key spots. It has an intricate and irregular structure with high porosity–up to 90 percent in healthy vertebral bone, for example, and more in osteoporosis.

In Keaveny’s lab at Berkeley, the structural properties of trabecular bone are measured on standard, rectangular or cylindrical samples (about three to five cubic centimeters). These are subjected to pulling or stretching (tensile stress), compression, or a combination of stresses applied along various axes through the samples. But such measurements conducted on small samples cannot reflect the effects of a tensile stress or compressive loading transmitted from afar through the length of an entire bone.

"Our computational approach makes contact with this body of measurement because we model the micromechanics of small samples like those we use in the lab," said Keaveny "But as computers have become more powerful, we are reaching beyond the limitations of small samples and discovering ways to scale up from the micromechanical sample. We must generalize and extend our results to the scale of the entire bone and model the continuum behavior of entire bones under large-scale deformation."


The modeling method used by Keaveny and his group is finite-element analysis. A "finite element" is a regular or irregular polygon or a polyhedron, a solid bounded by polygons. A model structure is typically built from many thousands or millions of these. Structural properties can be assigned and forces applied to edges and vertices. Finite-element methods are preferred to modeling on a regular grid by practitioners of structural engineering, because the methods can deal better with the highly irregular geometries of artifacts or structures. Where a grid point might only receive and transmit information from its eight nearest-neighbor points, a finite-element edge or vertex can receive and transmit information from and to an arbitrary number of neighbors, depending on the geometry of the element and the system.

The finite-element model is built on information derived directly from the laboratory analyses of bone tissue. Until recently, the group was using special microtome slicing and staining techniques, then scanning thin sections with a microscope and computationally reconstructing the three-dimensional object using tiny cubic volume elements. But advances over the past few years in the resolution of imaging methods now allow the group to derive their models directly from micro-computed tomographic scans of the samples. Eventually, they hope that additional computing power will allow them to use the method on whole-bone scans. "At the sample sizes we are able to compute today, our models are high-precision structural representations of bone," Keaveny said.

"Some of our models contain as many as 57 million finite elements," said Harun Bayraktar, a mechanical engineering graduate student whose recent research in the group is the basis for his doctoral dissertation. "The large number of voxels allows us to capture the very complex geometries of trabecular bone, but it immediately requires large-scale computational resources to calculate the response of the model to applied stresses."

Bayraktar and the Keaveny group, which includes Panayiotis Papadopoulos, a professor of mechanical engineering, have worked with a Berkeley alumnus, collaborator Mark Adams of Sandia National Laboratories at Livermore, CA, to make the finite-element models computationally tractable. Adams concentrates on methods for improving the performance of finite-element calculations on parallel architectures. "The scale and complexity of these micro-finite element bone calculations is unprecedented on distributed-memory computers, and thus it pushes common methods for both the finite-element parallelization and the linear solver past their limits," said Adams.

The implementation used on the IBM Blue Horizon supercomputer at the San Diego Supercomputer Center (SDSC) combines a string of codes. "We use a custom code, Bobcat, to translate the digital images obtained from micro-computed tomography into a finite-element mesh," Adams said. "Our parallel finite-element code, called Athena, partitions this finite-element mesh in parallel and constructs a local finite-element problem on each processor. Athena uses ParMetis from the University of Minnesota to construct high-quality partitions that minimize the amount of interprocessor communication required in the linear solution process. The processor-localized finite-element problem is solved by our serial finite-element analysis program, FEAP."

Adams explained that the most computationally expensive aspect of a finite-element simulation is the solution of a linear algebra problem (Ax = b), where b is a vector of applied forces, A is the "stiffness matrix," and x is a vector of displacements that need to be computed with the linear solver (FEAP). FEAP constructs the stiffness matrices and the output is given to Adams’s parallel algebraic multigrid solver, called Prometheus.

The large scale of problem to be solved in accurate finite-element models of bone tissue requires highly optimal solution methods, and Adams said, "Multigrid methods are the most optimal. The multigrid method used by Prometheus in this work, smoothed aggregation, uses the principles of graph theory to aggregate vertices in the finite-element graph. Each aggregate becomes one vertex in a new, coarser graph (known as a ‘grid’ for historical reasons). This process is applied recursively by Prometheus to construct a succession of coarse-grid representations of the fine-grid problem to be solved." Prometheus was built on ParMetis and PETSc from Argonne National Laboratory. The combination of codes gives the bone modelers enormous power to solve their problems despite the well-known difficulties associated with speeding up finite-element codes.


"As an example, we did a scaled speedup study of one finite-element model of a vertebral body," Bayraktar reports. "Even though the problem had more than 107 million degrees of freedom, we were able to solve it in less than eight minutes on 140 nodes of Blue Horizon." The speed of the method let them test the relative effects of two kinds of nonlinearity in the stress problem: nonlinear response caused by the nonlinear structure of trabecular bone on the microscale, and the nonlinear response of an entire sample of bone to a large geometrical deformation.

"The model results show much greater sensitivity than linear models to the direction and kind of stress loading applied to a bone," Bayraktar said. Using samples from four different sites, he found that the stresses under tension were greater and those under compression lesser than the same stresses calculated in linear models. "The combination of material and geometric nonlinearity in the models enabled them to exhibit more realistic behavior," Keaveny said, "and that will be the key to consistent prediction of bone behavior under various degrees of stress."

The group presented its results at the 2003 American Society of Mechanical Engineers Summer Bioengineering Conference in June 2003, and will present them a month later at the U.S. Association for Computational Mechanics conference. "Our results are consistent with those of a group in the Netherlands that is also employing micro-finite-element modeling, which is encouraging," Keaveny said. "But our techniques enable us to perform the computations much more efficiently and thus will enable us to scale up to the whole-bone level more accurately."

In addition to the high-resolution visualizations made from the modeling runs, the group has produced a number of animations. "We’re hoping to put together a good demonstration of all this at the Supercomputing conference in Phoenix in November," said Bayraktar. "We have submitted a paper to the conference."

"We haven’t included all the subtleties of the complex bone tissue, but we’ve come a lot closer to realism, and we’re looking forward to further improvements in both the methods and the computational platforms," said Keaveny. Using whole-bone models subjected to stresses like those encountered in accidents or disease, doctors will eventually be able to plan operations or prescribe drugs to rebuild healthy bone of the correct composition. "We’d like to be able to do this on demand in the clinical setting and prevent the enormous social costs of bone failure by intervening earlier and more effectively," Keaveny said. "That will take a much greater amount of computing power."

Merry Maisel is a senior science writer at the San Diego Supercomputer Center.