Capturing intraoperative deformations: research experience at Brigham and Womens hospital

www.elsevier.com/locate/media

Medical Image Analysis 9 (2005) 145–162

Capturing intraoperative deformations: research experienceat Brigham and Women�s hospital

Simon K. Warfield *, Steven J. Haker, Ion-Florin Talos, Corey A. Kemper,Neil Weisenfeld, Andrea U.J. Mewes, Daniel Goldberg-Zimring, Kelly H. Zou,Carl-Fredrik Westin, William M. Wells, Clare M.C. Tempany, Alexandra Golby,

Peter M. Black, Ferenc A. Jolesz, Ron Kikinis

Department of Radiology, Brigham and Women�s Hospital, Harvard Medical School, 75 Francis Street, Boston, MA 02115, USA

Department of Neurosurgery, Brigham and Women�s Hospital, Harvard Medical School, 75 Francis Street, Boston, MA 02115, USA

Available online 30 December 2004

Abstract

During neurosurgical procedures the objective of the neurosurgeon is to achieve the resection of as much diseased tissue as pos-

sible while achieving the preservation of healthy brain tissue. The restricted capacity of the conventional operating room to enable

the surgeon to visualize critical healthy brain structures and tumor margin has lead, over the past decade, to the development of

sophisticated intraoperative imaging techniques to enhance visualization. However, both rigid motion due to patient placement

and nonrigid deformations occurring as a consequence of the surgical intervention disrupt the correspondence between preoperative

data used to plan surgery and the intraoperative configuration of the patient�s brain. Similar challenges are faced in other interven-

tional therapies, such as in cryoablation of the liver, or biopsy of the prostate. We have developed algorithms to model the motion of

key anatomical structures and system implementations that enable us to estimate the deformation of the critical anatomy from

sequences of volumetric images and to prepare updated fused visualizations of preoperative and intraoperative images at a rate com-

patible with surgical decision making. This paper reviews the experience at Brigham and Women�s Hospital through the process of

developing and applying novel algorithms for capturing intraoperative deformations in support of image guided therapy.

� 2004 Elsevier B.V. All rights reserved.

1. Introduction

Resection of a brain tumor requires careful analysis

of the location of the tumor and surrounding healthy

structures in order to assess the functional consequences

of tumor resection. Accurate delineation of the tumor

margin and assessment of the structural and functionalanatomy along the surgical trajectory and in the vicinity

of the tumor is important to minimize the risk of neuro-

logical dysfunction. Direct visualization is limited by the

often similar visual appearance of healthy and diseased

1361-8415/$ - see front matter � 2004 Elsevier B.V. All rights reserved.

doi:10.1016/j.media.2004.11.005

* Corresponding author. Tel.: +1 617 732 7090; fax: +1 617 582

6033.

E-mail address: [email protected] (S.K. Warfield).

tissue, and by obscuration of deeper structures by sur-

face structures. Eloquent regions of white matter and

gray matter may not be recognizable in this way. While

total surgical resection of diseased tissue is the objective,

and is believed to correlate with positive patient out-

comes, the restricted capacity to interpret the patient

anatomy places limits on what can be achieved (Jolesz,1997).

Neurosurgical practice has been dramatically ad-

vanced by the development over the past decade of

intraoperative imaging devices. Neurosurgical proce-

dures can now be carried out in advanced operating the-

aters equipped with real-time on-demand imaging. The

availability of such imaging devices together with tech-

niques that improve the contrast between healthy and

mailto:[email protected]

146 S.K. Warfield et al. / Medical Image Analysis 9 (2005) 145–162

diseased tissue has enabled the development of advanced

minimally invasive procedures. Augmented visualiza-

tions that fuse data from several imaging devices to-

gether with intraoperative tracking of tools has

dramatically improved the precision of surgical proce-

dures (Jolesz, 1997; Kettenbach et al., 2000; Silvermanet al., 1995; Tempany et al., 2003; Black et al., 1997;

D�Amico et al., 2000, 2001; Hata et al., 2001).

The earliest engineering efforts in this domain focused

upon providing enhanced image acquisition, navigation

and display techniques (Nabavi et al., 2001). The con-

straints of the operating room place limitations upon

the time and nature of imaging modalities and proto-

cols, and typically intraoperative imaging has lowerSNR and spatial resolution than conventional imaging

techniques. The augmentation of intraoperative imaging

data with preoperatively acquired functional and struc-

tural data holds out the promise of improved surgical

navigation, reduced risk, and improved intraoperative

decision making. By exploiting preoperative imaging

modalities, such as PET, CT and/or MRI (amongst oth-

ers), the richness of the intraoperative data is signifi-cantly increased.

1.1. Intraoperative nonrigid registration

Intraoperative changes in the shape of the target

anatomy impose a stringent requirement upon naviga-

tion systems. In order to capture such shape changes it

is often necessary to make use of nonrigid registrationtechniques, which are characterized by a capacity to esti-

mate transformations that model not just affine param-

eters (global translation, rotation, scale and shear) but

also local nonrigid deformations. This typically requires

higher order transformation models with increased

numbers of parameters, and is usually more computa-

tionally expensive.

Several research groups have actively investigated anumber of strategies, reviewed below, to achieve such

data fusion in a manner suitable for intraoperative

application. Typically these use imaging data to indicate

the geometry of the registration problem. Some of these

approaches attempt to model the underlying biome-

chanics. Imaging data can also be used to provide target

information, and the goal then becomes to compute a

transformation aligning initial and target imaging data.Different approaches exploit different characteristics of

the imaging data to estimate the quality of alignment

and use different transformation models. Due to noise

and ambiguity in correspondence detection, the trans-

formation estimation can be ill-posed, with no unique

solution, and then further constraints or regularization

of the estimation problem is necessary. It is important

to make a distinction between the use of a biomechani-cal model to simulate, for example, actual brain defor-

mations, and the use of a physically motivated

regularization term in a general purpose nonrigid regis-

tration algorithm – although these may be superficially

similar, conceptually the use of a biomechanical mate-

rial model in these cases is motivated by very different

considerations.

In the domain of physically motivated image match-ing models, there is a tradeoff between the realism of the

simulated biomechanics and the time required to solve

the model. A method for fast simulation of surgery

was investigated (Bro-Nielsen, 1997) and achieved high

speed by using a model with surface nodes derived via

condensation from a volumetric finite element model.

This was developed with the goal of achieving rapid vi-

sual feedback to an operator, rather than insisting uponfully realistic simulation of the biomechanics. This

tradeoff enables such a simulator system to achieve vi-

deo frame rate speeds and is appropriate for computer

graphics oriented visualization tasks. In the context of

surgery on a specific patient it is natural to prefer to

maximize the accuracy of registration for that patient

rather than to minimize the time to achieve a registra-

tion. Nevertheless, the registration result must be com-puted at a rate compatible with surgical decision

making, and achieving this may necessitate some degree

of simplification in complexity of the model being

solved, and/or the use of high performance parallel com-

puting methods.

A more sophisticated biomechanical model, restricted

to two spatial dimensions, utilized the finite element

method with elements at the size of the pixels, togetherwith interactive input of the correspondence between

the images to be aligned (Hagemann et al., 1999). The

successful integration of fluid and elastic material mod-

els was also demonstrated (Hagemann et al., 2002).

Practical application during therapies requires a capac-

ity to deal with three-dimensional data and automatic

determination of correspondences.

Another two-dimensional model (Edwards et al.,1997, 1998) used three tissue compartments to model

heterogeneous tissue properties for tracking deforma-

tions intraoperatively. A fast solution was obtained by

using a simplified material model. Using a Sun Micro-

systems Inc. SPARC 20 workstation, the multi-grid

implementation on two-dimensional images consisting

of 128 · 128 pixels was reported to require 120–180

min to converge to a solution.A real-time intraoperative brain deformation capture

system was developed for epilepsy surgery, where brain

shift occurs relatively slowly (Skrinjar and Duncan,

1999; Skrinjar et al., 2001). Displacements of the surface

of the brain were to be estimated via stereo cameras.

Again, a simplified material model (Kelvin solid model)

was adopted in order to allow a solution for the defor-

mation could be obtained sufficiently quickly. Thenumerical model of 1521 brick elements, 2088 nodes

and 11,733 connections was solved on a Silicon Graph-

S.K. Warfield et al. / Medical Image Analysis 9 (2005) 145–162 147

ics Inc. Octane R10000 with one 250 MHz CPU and re-

quired ‘‘typically less than 10 min’’. A comparison of

two biomechanical models utilizing limited exposed sur-

face data was presented in (Skrinjar et al., 2002).

Miga, Paulsen and collaborators (Miga et al.,

1999a,b, 2000a,b,c, 2001; Paulsen et al., 1999; Robertset al., 1998, 2001, 1999) have developed a sophisticated

model of brain tissue undergoing surgery, incorporating

simulations of forces associated with tumor tissue, and

simulations of retraction and resection forces. Careful

validation experiments indicate their model is able to

closely match observed deformations (Platenik et al.,

2002). They indicate further improvements in accuracy

will be possible by incorporating sparse data from inex-pensive intraoperative imaging devices. This work has

demonstrated that computer aided updating of preoper-

ative brain images can restore close correspondence be-

tween the preoperative data and the intraoperative

configuration of the subject. Miller and colleagues

(Miller and Chinzei, 1997, 2000, 2002; Miller et al.,

2000; Chinzei and Miller, 2001) have developed a

sophisticated brain tissue material model, incorporatingnonlinear stress–strain and strong stress–strain rate

dependence, and have carried out a number of experi-

ments with soft tissue to measure true material parame-

ters suitable for simulating soft tissue deformation of the

brain, kidney and liver. Challenges in applying sophisti-

cated modeling during neurosurgery remain. Robust

and precise determination of patient-specific boundary

conditions and surgeon induced forces (such as retractorpressure) is a difficult task. The time required to create

and solve a sophisticated model also needs to be care-

fully considered in the context of intraoperative

application.

1.2. Nonrigid registration algorithms for intraoperative

MRI

At our institution a number of research oriented clin-

ical applications of image guided therapy have been

investigated (Jolesz, 1997; Silverman et al., 1995; Wong

et al., 1998; Albert et al., 2003; D�Amico et al., 2001;

Black et al., 1997). Intraoperative magnetic resonance

imaging has been developed to enable ready visualiza-

tion of intraprocedural changes in the configuration of

the patient, and to enable improved surgical navigation,monitoring and targeting.

This provides an ideal testbed from which to examine

intraoperative nonrigid registration strategies and to

evaluate the robustness of different algorithms to noise

characteristics, to intraoperative contrast changes, and

to the sparsity of the data available for intraoperative

updating. In particular this allows a carefully staged

algorithm research effort. Since intraoperative MRI pro-vides high spatial resolution, considerable flexibility in

mechanisms for achieving contrast, and good signal to

noise ratio, algorithms for intraoperative deformation

capture may first be more easily developed to suit this

high quality imaging data. Later the key lessons learned

can be used as the basis for strategies dealing with spar-

ser, noisier or less flexible contrast mechanisms and

imaging modalities.Our work in this area over the past several years has

been both at the fundamental level of new algorithm

development and in application of our technology to

specific clinical challenges (Iosifescu et al., 1995, 1997,

2001; Warfield et al., 1995a, 1999, 2000a, 2001, 2002a;

Hata, 1998; Kaus et al., 1999a, 2000, 1999b; Ferrant

et al., 2000a, 1999, 2000b,c; Hata et al., 1999; Kikinis

et al., 1999; Ruiz-Alzola et al., 2000; Nabavi et al.,2001; Chinzei et al., 2003).

1.2.1. Early experiments in nonrigid registration for

medical applications

Our early efforts in nonrigid registration were fo-

cused upon leveraging the technology developed by

Joachim Dengler (Dengler, 1986; Dengler et al., 1988;

Dengler and Schmidt, 1988, 1990). Dengler and collab-orators worked for several years to implement fast

nonrigid registration, inspired initially by such work

as that of Broit (1981) and the developments in optical

flow calculations (Horn and Schunck, 1981; Nagel and

Enkelmann, 1984, 1986). Dengler explored several

strategies for computing meaningful features from

images to maximize matching quality (Dengler and

Schmidt, 1990), and explored tradeoffs of elastic mem-brane models including models allowing ‘‘tearing’’ (dis-

continuities) of the membrane (Dengler and Schmidt,

1988) as well as methods for fast regularization of

the two-dimensional dense-field nonrigid registration

problem through a very fast numerical scheme opti-

mized for this problem (Schmidt and Dengler, 1989).

Dengler developed a multi-resolution pyramid imple-

mentation which used a nested multi-resolution similar-ity search to enable accurate matching of anatomical

features with different scales (Dengler and Schmidt,

1988). He also developed a solution to the problem

of lack of symmetry between target and source images

in the conventional intensity difference similarity met-

ric, a suggestion that was later applied by Bajcsy and

Kovacic in their seminal work (Bajcsy and Kovacic,

1989, see p. 14). This topic has recently seen renewedinterest (Cachier and Rey, 2000; Christensen and John-

son, 2001; Johnson and Christensen, 2002; Magnotta

et al., 2003; Nielsen et al., 2002). Dengler�s publicationsprimarily dealt with two-dimensional image matching

due to the computational capabilities of the hardware

available to him at that time, but he did do an imple-

mentation that ran in three dimensions and that was

used for matching medical imaging data.Dengler�s experiments over that period demon-

strated significant value in transformations of the raw


imaging data before carrying out the matching process.

In particular, he described advantages in using a pseu-

do-logarithmic transformation (Dengler and Schmidt,

1990). Following his analysis of the influence of noise

and contrast properties intrinsic to the scanner upon

the performance of such nonrigid registration algo-rithms, he found significant value in matching tissue

classifications, rather than the original data itself.

These ideas have influenced the way in which we ap-

plied nonrigid registration in a number of clinical

applications.

1.2.2. Nonrigid registration of conventional MRI data

We have pursued the application and validation ofnonrigid registration techniques to segmented data to

solve several clinical challenges. One of the critical

developments we achieved was the close integration of

nonrigid registration of a template of normal anatomy

to individual patients, not for the purpose of simply pro-

jecting boundaries from the atlas to the subject, but to

alter and refine new segmentations of the patient data

(Warfield et al., 1998a,b,c, 2000a). The creation of aprincipled mechanism for injecting information from

an atlas transformed via nonrigid registration to update

a statistical classification, and the use of that improved

statistical classification to iteratively improve the non-

rigid registration accuracy, has enabled several impor-

tant clinical applications (Warfield et al., 1995a,b,

1997, 1998a,b,c, 1999, 2000a, 2003; Wei et al., 2002;

Kaus et al., 1999a,b, 2001; Inder et al., 1999; Iosifescuet al., 1997).

Dengler�s (Dengler and Schmidt, 1988) implementa-

tion made several approximations in order to increase

its speed. Midway through the 1990s it became clear that

the computational capacity had arrived to enable us to

reassess some of the assumptions which were made.

We created a full three-dimensional nonrigid registra-

tion implementation, using mean square intensity differ-ence in local regions as the similarity metric, and

constrained by a linear elastic material in 1999 (Ferrant

et al., 1999). The nonrigid registration problem was for-

mulated in the same way as Dengler�s original formula-

tion, as a functional balancing a similarity metric and a

regularization term, and solved using calculus of varia-

tions, and a variational optimization. Unlike Dengler�simplementation, this new approach did not make anysimplifying assumptions, beyond a linearization of the

similarity metric, in order to accelerate convergence. In

practice, the method was quite successful in clinical

applications where an assumption of constant image

intensities for corresponding structures held true. In or-

der to investigate applications where an identity rela-

tionship between signal intensities did not exist,

Nobuhiko Hata investigated, over the course his Ph.D.thesis, a similar formulation but using a mutual infor-

mation measure in place of mean square intensity differ-

ences. Hata also carried out extensive comparisons to

conventional optical flow methods (Hata et al., 1998,

2000; Kikinis et al., 2000).

An important application driving development of im-

proved nonrigid registration algorithms has been aug-

mented reality visualization for real-time surgicalmonitoring, navigation and treatment. Examples of clin-

ical applications include MRI-guided prostate biopsy

and brachytherapy (Bharatha et al., 2001), MRI-guided

cryoablation of abdominal (liver, kidney) lesions (Butz,

2000; Butz et al., 2000; Warfield et al., 2000a), and neu-

rosurgery (Warfield et al., 2000a,b,c,d, 2002a, 2003;

Kaus et al., 2001; Ferrant et al., 2001, 2002).

We sought algorithmic approaches which could over-come the inherent data quality limitations (intensity var-

iation due to contrast agent uptake, scanner

inhomogeneity, noise) of intraoperative image acquisi-

tions and the restriction on time-to-result for intraoper-

ative nonrigid registration.

We were successful in developing a real-time biome-

chanical simulation of brain deformation which has

been run during clinical cases and presented new datato the neurosurgeon to enhance intraoperative decision

making (Warfield et al., 2002a). The approach leverages

imaging data to provide correspondence estimates and

this enables the utilization of a relatively sophisticated

biomechanical model.

Our most recent work has built upon our earlier ef-

forts and explorations in nonrigid registration for seg-

mentation, preoperative planning and enhancedvisualization in support of image guided surgery and

has been described previously (Warfield et al.,

1998a,b,c, 2000a, 2001; Hata et al., 1998, 1999, 2000;

Ferrant et al., 1999, 2000a,b,c, 2001; Kaus et al., 2000,

2001; Ruiz-Alzola et al., 2000; Rexilius, 2001; Bharatha

et al., 2001; Rexilius et al., 2001; Guimond et al., 2002;

Tei et al., 2001).

1.2.3. Nonrigid registration of diffusion tensor images

The area of nonrigid registration of diffusion tensor

MRI is still emerging, with our own recent work (Sierra,

2001; Ruiz-Alzola et al., 2000, 2002; Guimond

et al., 2002) being amongst the first to directly exploit

the specific characteristics of diffusion tensor MRI in

order to appropriately re-orient tensors undergoing

transformation. Our initial work in combiningstructural MRI and DTI as complementary sources

of data for driving nonrigid registration has been

promising.

The local orientation of white matter fiber tracts lo-

cally alters the mechanical properties of brain tissue in

an anisotropic manner. This can be modeled and

exploited to improve the accuracy of alignment of intra-

operative brain MRI using preoperative DT-MRI tocontrol the anisotropic material characteristics (Kem-

per, 2003).


2. Methods

This section provides an overview of our previously

published intraoperative image analysis strategy. Analy-

sis of both preoperative and intraoperative imaging data

is carried out using image segmentation and registrationtechniques. We have developed an approach for utilizing

preoperative segmentations to aid in the rapid segmen-

tation of intraoperative data. The strategy we used will

be illustrated below in the context of neurosurgery and

prostate biopsy. Preoperative segmentations are also

used to create fused visualizations of imaging data, often

using several different modalities, and to facilitate pre-

operative planning of the surgical trajectory. Intraoper-ative segmentation can be used for quantitative

monitoring of therapy progression. One such example

is the comparison of region of cryoablation of a liver tu-

mor as it proceeds with a planned target region. Regis-

tration of images is used to visualize preoperative

images in the coordinate system of the patient intraoper-

atively. Our registration procedure relies upon two

steps: first an affine transform is estimated, and then abiomechanical simulation of deformation is carried

out. The biomechanical simulation allows us to infer a

volumetric deformation from boundary conditions de-

fined by surface matching of key boundaries.

2.1. Preoperative data acquisition, image segmentation

and fusion

Preoperative data analysis typically occurs well be-

fore an interventional therapy, and sufficient time exists

for extensive, time consuming and presumably more

accurate data analysis to be carried out. In contrast,

intraoperative data must be analyzed at a rate compati-

ble with decision making during the procedure.

A number of different segmentation algorithms and

tools are suitable for preoperative image analysis,depending upon the characteristics of the imaging device

and anatomical region under consideration. Interactive

segmentation techniques (Gering et al., 2001), semi-

automatic methods (Kikinis et al., 1992; Yezzi et al.,

1997) and automated approaches (Warfield et al.,

2000a,b,c,d; Kaus et al., 2000) each have a role. Typi-

cally, the most appropriate technique to achieve accu-

rate segmentation of the structures underconsideration is chosen.

For neurosurgical cases, we seek to process DT-MRI

acquisitions to enable the display of white matter fiber

tracts (Westin et al., 2002; O�Donnell et al., 2002), and

the localization of functional cortex is achieved with

fMRI activation analysis (Tsai et al., 1999; Fisher

et al., 2001). Recently we described the feasibility of

combining preoperative functional MRI and DT-MRIto enhance intraoperative guidance for neurosurgery

(Talos et al., 2003).

2.2. Capturing intraoperative deformations during

neurosurgery

In this section we describe the image analysis pipeline

we have used to enable the capture of intraoperative

deformations during neurosurgery (Warfield et al.,2002a,b,c,d).

2.2.1. Intraoperative image processing

Acquiring new images over the course of an inter-

ventional therapy such as neurosurgery enables im-

proved monitoring and evaluation of progress. We

use a series of image processing algorithms to capture

intraoperative changes during neurosurgery, consistingof segmentation, rigid registration, active surface

matching, solution of a biomechanical model for the

volumetric deformation implied by the surface corre-

spondences, warping of preoperative data into the

intraoperative configuration and visualization of the

fused data in the coordinate system of the intraopera-

tive data.

2.2.2. Segmentation of intraoperative volumetric images

We have experimented with a number of algorithms

with the goal of achieving accurate, robust and rapid

segmentations of intraoperative imaging data (Warfield

et al., 2000a, 2002a; Yezzi et al., 2000). For a method

to be acceptable during a procedure, the surgeon needs

to be able to rely upon the operation of the method,

and to have confidence in the results that are achieved.For this reason, interactive methods that empower the

user with a large degree of control have been seen as

desirable, and we have used such methods (Yezzi

et al., 2000; Gering et al., 2001). Fully automatic seg-

mentation algorithms, or algorithms requiring merely

interactive initialization are preferable provided they

are sufficiently robust and have acceptable accuracy

and precision. We have developed and improved suchmethods and used them during interventional proce-

dures (Warfield et al., 1998a,b,c, 2000a,b,c,d, 2002a).

Adoption of any automated intraoperative image

segmentation method requires careful validation of

the performance characteristics of such a method in

the face of the normal and pathological variability of

anatomy to which the algorithm will be applied. Once

such methods are sufficiently validated, it may be pos-sible to rely upon them to achieve quantitative moni-

toring of intraoperative therapy and real-time

verification that the planned surgical trajectory is being

achieved.

Validation studies that include the assessment of

accuracy and precision require a reference standard

against which methods may be compared. An ideal ref-

erence standard for image segmentation would beknown to high accuracy and would reflect the charac-

teristics of segmentation problems encountered in


practice. There is a tradeoff between the accuracy with

which the reference standard may be known and the

degree to which the reference standard reflects segmen-

tation problems encountered in practice, that is be-

tween the accuracy and the realism of the reference

standard. The accuracy of segmentations of medicalimages has been difficult to quantify in the absence

of an accepted reference standard for clinical imaging

data.

We have developed a new estimation algorithm to

enable the assessment of the accuracy and precision

of image segmentation methods (Warfield et al.,

2002a,b,c,d, 2004). We refer to this algorithm as simul-

taneous truth and performance level estimation (STA-PLE). The algorithm uses a collection of

segmentations of the same image in order to estimate

simultaneously both the ‘‘hidden’’ true segmentation

(reference standard) and the performance level of each

segmentation generator (either human rater or segmen-

tation algorithm). Performance is characterized by sen-

sitivity and specificity parameters, and predictive values

(posterior probabilities) may also be calculated (War-field et al., 2004). We have validated this algorithm

by comparison to digital phantoms and demonstrated

its application to assessing human rater and algorithm

performance in some clinical applications of segmenta-

tion. We have evaluated validation metrics as com-

pared to this estimated ground truth (Zou et al.,

2003, 2004b). The estimated true segmentation identi-

fied by this algorithm is a reference standard thatmay be used for selecting between different segmenta-

tion algorithms, or for selecting parameters to fine tune

performance (Warfield et al., 2004).

2.2.3. Unstructured mesh creation and surface

representation

Following segmentation of preoperative MRI of the

brain, we construct an explicit representation of the sur-face of the brain and ventricles, and a volumetric tetra-

hedral mesh throughout the brain. We repeat the

segmentation and mesh construction for intraoperative

MRI of the brain. In order to be used in practice, the

mesh generation technique needs to be sufficiently rapid

that it can be used during neurosurgery, but it also needs

to generate a numerically satisfactory mesh and closely

match the true patient geometry. In order to achievethese constraints we implemented a tetrahedral mesh

generator specifically suited for segmentations of volu-

metric data. The algorithm is in essence the volumetric

counterpart of a marching tetrahedral surface genera-

tion algorithm (Ferrant et al., 1999, 2000a,b,c, 2002;

Schroeder et al., 1996; Geiger, 1993). This approach en-

ables us to easily associate inhomogeneous biomechani-

cal parameters throughout the relevant anatomy sincethe segmentation and tetrahedral mesh nodes have the

same coordinate system.

2.2.4. Rigid registration of preoperative volumetric

images to intraoperative volumetric images

For several years we have used a rapid and robust

parallel registration algorithm to achieve rigid registra-

tion (Warfield et al., 1998a,b,c). This allows us to correct

for rotation and translation differences between preoper-ative and intraoperative scans, or between subsequent

intraoperative scans. The algorithm identifies an optimal

set of transform parameters that maximizes the spatial

overlap of segmented structures in the two data sets

being aligned. On a Sun Microsystems SunFire 6800

symmetric multi-processor using 12 750MHz CPUs the

algorithm requires about 45 s to converge on typical

brain MRI suitable for neurosurgery. On a Dell Preci-sion 650n with dual 3.0 GHz Intel Xeon CPUs running

Linux, this registration requires approximately 95 s.

2.2.5. Volumetric biomechanical simulation of brain

deformation

Over the course of a neurosurgical procedure the

geometry of the brain is altered under the influence of

mechanical factors such as drainage of cerebrospinalfluid, retraction and resection of brain tissue, and a

number of physiological changes which can lead to

changes in the hydration of the tissue and changes in

blood pressure and oxygenation of the brain. When

the surgeon wants to evaluate the progress of the re-

moval of tumor, or to image the new geometry of the

patient�s brain, a new volumetric MRI is acquired. We

can then estimate a volumetric deformation field thatwarps the previous data into the new configuration of

the brain by solving a biomechanical model that simu-

lates the true brain tissue deformation. Since we don�thave access to good estimates of the true intraoperative

pressures and forces acting upon and throughout the

brain, we instead use imaging data to establish bound-

ary conditions. The strategy we use is to estimate a vol-

umetric deformation field that has the same deformationat the surface as that which we obtain via active surface

matching between the preoperative and intraoperative

data, but we use a biomechanical model of the proper-

ties of tissue throughout the brain to estimate a volumet-

ric deformation field. Critical to computing the

volumetric deformation field during surgery is a rapid

and sufficiently accurate biomechanical model.

2.2.6. Estimating correspondences of key surfaces

In the past we have used an active surface algorithm

which has been described previously (Ferrant et al.,

2002). The essence of the strategy is to identify the sur-

face of the ventricles and brain explicitly in the data to

be warped. Forces are then applied in an iterative fash-

ion to drive the surfaces (constrained with an elastic

membrane energy) towards the target (Ferrant et al.,2001). The forces are derived from a function of the im-

age intensity gradients so as to be minimized when the


surfaces match the edges in the target volume. To im-

prove the robustness and rate of convergence, prior

information regarding the expected gray level and gradi-

ents of the object being matched is used (Ferrant et al.,

2002). In recent work in matching preoperative MRI of

the prostate to intraoperative MRI of the prostate, wehave used a conformal mapping strategy to establish

surface correspondences as described further below.

2.2.7. Biomechanical simulation of volumetric brain

deformation

The most straightforward model is to treat the brain

tissue as an homogeneous linear elastic material. The

deformation energy of an elastic body, without any ini-tial stress or strain, subject to externally applied forces,

can be described by the following model (Zienkiewicz

and Taylor, 1994):

E ¼ 1

2

ZXrTedXþ

ZXFTudX; ð1Þ

where F = F(x,y,z) is a vector representing forces ap-

plied externally to the elastic body, u = u(x) = u(x,y,z)

is the displacement vector field we wish to estimate, Xis the domain of the elastic body described by a tetrahe-

dral mesh, r is the stress vector and e is the strain vector.

The stress vector is linked to the strain vector by the

constitutive equations of the material, and for our model

we have

r ¼ ðrx; ry ; rz; sxy ; syz; sxzÞT ¼ De;

where D is the elasticity matrix characterizing the mate-

rial�s properties. Finally, the strain is related to displace-

ment by the assumption that e = LTu, where L is a linear

differential operator.The tetrahedral mesh over the volume defines the set

of finite elements used to discretize the above model.

The continuous displacement field u everywhere within

a mesh element e is defined as a function of the displace-

ments at the surrounding mesh vertices uei :

uðxÞ ¼X4

i¼1

Nei ðxÞuei ;

where Nei ðxÞ are the basis functions over the interior of

the element, are zero outside the element and which

we take to be affine. Hence the interpolating function

of node i of tetrahedral element e is defined as

Nei ðxÞ ¼

1

6V e ðaei þ bei xþ cei y þ dei zÞ:

The determination of the volume of the tetrahedron Ve

and the interpolation coefficients is described in (Zie-

nkiewicz and Taylor, 1994, pp. 91–92).

The volumetric deformation field u(x) is found by

solving for the displacement field that minimizes the en-ergy described by Eq. (1). We define the matrix

Bei ¼ LNe

i at each node of each element, and since the

energy of the volume is the sum of the energy of each

element. We set

o

oueiEðue1; . . . ; ue4Þ ¼ 0; i ¼ 1; . . . ; 4

and find the following condition for a minimum:

ZX

X4

j¼1

BeTi DBe

juej dX ¼ �

ZXFNe

i ; i ¼ 1; . . . 4 8e: ð2Þ

This can be written as a linear system of equations that

can be solved for the unknown displacements at each

node i given the forces acting on the boundary nodes:

Ku ¼ �F: ð3Þ

The displacements on the boundary nodes are fixed to

match those generated by the surface correspondence

estimates. This is done by setting all elements on rows

of K corresponding to boundary elements to zero,

except for the diagonal which is set to 1. The corre-

sponding right hand side element is set equal to the

specified displacement value. In this way, solving

Eq. (3) for the unknown displacements will producea deformation field over the entire mesh, and the pre-

scribed displacements at the boundary nodes will be

preserved.

2.2.7.1. A locally anisotropic white matter material model.

We have recently extended this model to account for the

inhomogeneous material characteristics of heteroge-

neous white matter of the brain (Kemper, 2003). Thebrain tissue is modeled here locally as a transversely

anisotropic linear elastic material, in which the fiber

direction has one set of material parameters and the

cross fiber direction has another. At each tetrahedron

in the previously described mesh, the local coordinate

system aligned with the fiber direction and the local elas-

ticity parameters must be defined to calculate the elastic-

ity matrix D. We do this by assigning a diffusion tensorto each tetrahedron in the volumetric mesh and calculat-

ing its eigenvectors and eigenvalues. The major eigen-

vector corresponds to the principal fiber direction, and

the other two eigenvectors correspond to the plane per-

pendicular to the fiber. The eigenvalues represent the rel-

ative amount of diffusion in each direction.

The elasticity matrix for a transversely isotropic

material requires five independent parameters. Thecross-fiber stiffness is approximately 2· greater than

the fiber stiffness for isotropic brain tissue (Prange and

Margulies, 2002; Aimedieu et al., 2001). However, not

all brain tissue is anisotropic. Fractional anisotropy, de-

scribed in the equation below, is calculated from the

eigenvalues of the diffusion tensor and provides a quan-

titative measure of the degree of diffusion anisotropy of

the tissue.


FA ¼ 1ffiffiffi2

p

ffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiðk1 � k2Þ2 þ ðk2 � k3Þ2 þ ðk3 � k1Þ2

qffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffik21 þ k22 þ k23

q :

When fractional anisotropy is zero, the region is isotro-

pic. We infer that this indicates the material characteris-

tics should also be isotropic and so Young�s moduli

should be the same in all three directions. When frac-

tional anisotropy is one, the white matter region is com-

pletely anisotropic, and we infer that this indicates

Young�s modulus in the cross-fiber direction should begreater than Young�s modulus in the fiber direction.

Therefore, we simply calculate the Young�s modulus in

the cross-fiber direction as a linear function of the frac-

tional anisotropy and maximum stiffness ratio (a) andleave Young�s modulus in the fiber direction at its de-

fault value.

Ep ¼ ð1þ ða� 1ÞFAÞEf :

Poisson�s ratios are assumed to be equal in all threedirections because the compressibility of the tissue is

not expected to change. The shear moduli are then cal-

culated from Young�s moduli and Poisson�s ratio.Once the local elasticity matrix has been assembled, it

is rotated according to the transformation matrix from

the local coordinate system, defined by orientation of

the fibers, to the global coordinate system. The transfor-

mation, based on the eigenvectors of the diffusion ten-sor, yields the rotated stiffness matrix:

D0 ¼ TDT T:

2.2.7.2. Rapid solution of the system of equations. In or-

der for this strategy to obtain a volumetric deformation

field to be practical during surgery, it must be possible to

obtain the solution at a rate compatible with surgical

decision making. We have investigated solving thesystem of equations on a set of parallel architectures

(Warfield et al., 2000a), including high end symmetric

multi-processor (SMP) architectures, clusters of SMPs,

and loosely coupled commodity clusters including a

Beowulf cluster (Sterling et al., 1995; Anderson et al.,

1995). We were able to demonstrate that the above model

and the necessary image acquisition and image segmen-

tation, registration and augmented visualization can becarried during neurosurgery (Warfield et al., 2002a).

Our approach to parallelization involved dividing the

K matrix into equally sized domains and distributing the

assembly and computation on the rows of these domains

across processors. We solve the volumetric brain defor-

mation simulation using the linear equation solver

implemented in the portable, extensible toolkit for scien-

tific computation (PETSc) (Balay et al., 1997, 2000). Weuse the generalized minimum residual (GMRES) solver

(Freund et al., 1992) with block Jacobi preconditioning.

During neurosurgery, the system of equations was

solved on a Sun Microsystems SunFire 6800 SMP using

12 750MHz UltraSPARC-III CPUs and 12 GB of

RAM. This hardware platform provided sufficient com-

putational capacity to execute the intraoperative image

processing during neurosurgery. We have recently

experimented with both an inexpensive workstation, aDell Precision 650n with dual 3.0 GHz Intel Xeon CPUs

running Linux, and a cluster of such workstations con-

nected by 100 Mbps Fast Ethernet. This has enabled ex-

tremely rapid solution of the typical system of equations

on inexpensive and widely available hardware, which

holds out the possibility of widespread deployment of

this technology. For example, running on a single 3.0

GHz Intel Xeon CPU, for a mesh of 54,997 vertices,the time to assemble the system of equations was

approximately 9.5 s and the time to solve this system

was approximately 5.2 s.

2.2.8. Enhanced visualization during therapy

Following the computation of the volumetric trans-

formation, preoperative data sets may be warped into

the current configuration of the subject. We have usedthis strategy to display, for example, preoperative func-

tional data and segmentations of brain surface, tumor

and ventricles matched to the intraoperative acquisition.

The enhanced visualization is presented to the therapy

team carrying out the operation in two ways: a Sun

Microsystems workstation with hardware acceleration

of triangle rendering and lighting is used to rapidly ren-

der the fused data and this screen is viewed by those out-side the open-magnet operating room, and LCD

displays attached on each side of the magnet display

the scene to those inside the operation room. This en-

ables the surgeon to visualize simultaneously the visual

appearance of the surface of the patient�s brain through

the craniotomy, the volumetric data acquired by the

scanner, and representations of critical healthy and dis-

eased anatomy and functional regions as determinedfrom both preoperative and intraoperative imaging data

(Warfield et al., 2002a). This enhances the surgeon�scapacity to perceive the region of the tumor, the area

to be resected and the structures that are to be preserved

by creating a low cognitive load representation of the

brain morphology, white matter structure and function-

ally significant regions.

2.3. Capturing intraoperative deformation for

MR-guided prostate biopsy

One key application of our method is in MR-based

prostate biopsy and therapy. We have found that signif-

icant prostate shape changes occur between pre-opera-

tive 1.5 T endorectal coil imaging, in which the patient

is supine, and intraoperative 0.5 T interventional MRimaging, during which the patient is in the lithotomy po-

sition (Hirose et al., 2002). This shape change is likely


the result of changes in patient position and rectal filling

necessitated by the procedures. We have quantitatively

characterized this shape change from interactive mea-

surements of displacements of landmarks (Hirose

et al., 2002). One would expect nonrigid deformations

to occur during other procedures, such as trans-rectalultrasound guided biopsy.

Images acquired with an endo-rectal coil have a sig-

nificant intensity inhomogeneity characteristic. We have

developed software for correcting these image intensity

artifacts using a method first proposed in (Viola and

Wells, 1997; Mangin, 2000).This method is attractive be-

cause it relies solely on intrinsic properties of the ac-

quired images and is computationally efficient. Thismethod uses the entropy of the image histogram as a

metric for assessing the intensity artifact, and computes

a spatially varying intensity correction field. The theory

is that different types of tissue, when imaged correctly,

have a fairly narrow intensity profile and form a peak

in the histogram. Intensity artifacts, such as the one seen

here, serve to spread out this peak as the same tissue

maps to a wider range of intensities over different partsof the image. We have recently extended this algorithm

to enable the automatic normalization of the signal

intensity distribution of one subject to match that of an-

other (Weisenfeld and Warfield, 2004).

In summary, the method for prostate alignment in-

volves the following steps: (1) A three-dimensional model

of the entire prostate, composed of tetrahedra, is created

from intensity-corrected segmented pre-operative 1.5 TMR images. (2) The boundary surface of the capsule is

extracted from this tetrahedral mesh and is registered

using a conformal mapping technique (Angenent et al.,

1999) to a corresponding capsule surface obtained from

intraoperative images. (3) The surface point matches

from step 2 are used as boundary conditions when solv-

ing the finite element-based system of equations de-

scribed in Section 4. The volumetric deformation fieldfrom the previous step is used to interpolate pre-opera-

tive imaging data.

More recently, we have improved the speed and

robustness of the algorithm by replacing the active sur-

face algorithm, originally used to register the prostate

capsule surface, with a direct approach based on our

work in the theory of conformal mapping (Angenent

et al., 1999). In this newer method the surface of theprostate capsule, as given by the pre-operative imaging,

is modeled as a thin elastic sheet which conforms to the

altered shape of the capsule as given by the intraopera-

tive imaging. Regardless of the degree of shape deforma-

tion, or changes from convexity to concavity, the

method yields a one to one mapping of the pre-operative

capsule onto the intraoperative capsule surface. Further,

the method has several inherent advantages over ourearlier methods that make it more suitable for use in

the operating room. In particular, the core of the algo-

rithm requires only the solution of a pair of sparse linear

systems of equations, and thus can be made to run

quickly enough to be practical. Indeed, this is the sense

in which the method is direct, as the matching from pre-

operative surface to intraoperative surface does not re-

quire the calculation of intermediate sets of points inspace. The method also lends itself well to multi-proces-

sor parallelization, and standard parallel versions of lin-

ear solvers can be used to reduce the solution time

significantly.

3. Results

In this section illustrative results of our algorithm for

the capture of intraoperative deformations are pre-

sented. The quality of segmentation of intraoperative

data is crucial to our registration and visualization strat-

egy, and so validation results illustrative of our previ-

ously reported studies (Ferrant et al., 2002; Bharatha

et al., 2001) are shown.

3.1. Preoperative visualization for neurosurgical planning

An illustration of preoperative data fusion and visu-

alization is shown in Fig. 1. The axial MR image shows

the region of a tumor, and on the basis of the appear-

ance of the solid circular region of abnormal signal

intensity associated with the tumor, it could be inferred

(probably incorrectly) that it is safe to remove this tu-mor in its entirety. Upon comparing this with the visu-

alization on the right that shows functional MRI

activation of a motor task in red, motor related fiber

tracts in yellow, lateral ventricles in blue, and the region

of the tumor in green, we can see that it is possible to

trace fiber tracts from the brain stem up through the tu-

mor to the motor cortex. Such information may alter the

perceived risks versus benefits assessment involved in to-tal resection of the tumor. For example, the information

of Fig. 1 was used during the patient�s surgery. The sur-geon and the radiologist together discussed the tumor,

surrounding anatomy and white matter fibers. The

majority of the tumor was removed and a small medial

remnant was left in place as removal would have dis-

rupted the motor fibers. The patient had no post-opera-

tive neurological deficits.

3.2. Visualization of preoperative data matched to the

intraoperative images for image guided neurosurgery

Fig. 2 illustrates the projection of preoperative imag-

ing data into the intraoperative configuration of the pa-

tient. The image of Fig. 2(f) is a three-dimensional

visualization and the other images show two-dimensionalimages through a three-dimensional volumetric projec-

tion of preoperative MRI onto intraoperative MRI.

Fig. 2. The two-dimensional images illustrate changes in the brain between preoperative MRI shown in (a) and intraoperative MRI following

resection of the region of the tumor (b), together with the warped (c) and warped, clipped preoperative MRI (d). The preoperative data is clipped in

the region of the resection cavity, which is identified automatically from the intraoperative MRI. The difference between the intraoperative MRI and

the warped, clipped preoperative MRI is shown in (e), and illustrates that a close match is obtained. Image (f) displays preoperative data including

functional MRI (red), DT-MRI fiber tracts (pink), vessels (dark blue), lateral ventricles (light blue) and tumor, warped into the geometry of the

patient during surgery as indicated by intraoperative magnetic resonance imaging.

Fig. 1. (a) Axial MR image. (b) Multi-modality visualization. The axial MR image shows the region of a tumor, and on the basis of the appearance of

the solid circular region of abnormal signal intensity associated with the tumor, it could be inferred that it is safe to remove this tumor in its entirety.

Upon comparing this with the visualization on the right that shows functional MRI activation of a motor task in red, motor related fiber tracts in

yellow, lateral ventricles in blue, and the region of the tumor in green, we can see that it is possible to trace fiber tracts from the brain stem up through the

tumor to the motor cortex. Such information may alter the perceived risks versus benefits assessment involved in total resection of the tumor.


Such a three-dimensional visualization allows ready

assessment of the three-dimensional relationships be-

tween the tumor, healthy white matter and gray matter

structures, functionally active regions and critical fiber

tracts. The augmentation of the intraoperative imaging

data with the rich data available preoperatively, in the

same coordinate system, along with the ability to visual-

ize the three-dimensional relationships as well as two-

dimensional cross-sections significantly enhances the

ease with which critical intraoperative surgical decisions

may be made.

We have previously compared interactively selected

neuroanatomical landmarks in preoperative and intra-

operative images exhibiting significant brain deforma-

Table 1

Summary of landmark matching accuracy

Time pt. Surface

l ± r (mm)

Surface max.

(mm)

Central l ± r(mm)

Central max.

(mm)

Tumor l ± r(mm)

Tumor max.

(mm)

Overall l ± r(mm)

Overall max.

(mm)

1 to 2 1.9 ± 1.5 7.2 2.2 ± 1.1 4.7 2.1 ± 1.0 4.2 2.1 ± 1.4 7.2

1 to 2 reg. 0.8 ± 0.4 1.3 1.1 ± 0.7 2.0 1.6 ± 0.5 2.0 1.0 ± 0.6 2.0

2 to 3 1.8 ± 1.1 3.8 2.0 ± 1.4 4.5 1.8 ± 1.6 4.7 1.8 ± 1.2 4.7

2 to 3 reg. 0.8 ± 0.6 2.0 1.0 ± 0.6 2.8 0.8 ± 0.8 2.0 0.8 ± 0.6 2.8

3 to 4 1.7 ± 1.3 6.4 1.4 ± 0.9 3.0 1.7 ± 1.3 3.8 1.4 ± 0.9 6.4

3 to 4 reg. 0.7 ± 0.6 2.6 0.8 ± 0.6 1.8 1.7 ± 1.0 3.3 0.9 ± 0.6 3.3

4 to 5 1.1 ± 0.5 2.7 0.9 ± 0.7 2.7 3.2 ± 3.1 8.7 1.4 ± 1.5 8.7

4 to 5 reg. 0.6 ± 0.7 2.6 0.7 ± 0.5 1.3 1.9 ± 1.1 3.7 0.8 ± 0.9 3.7

Mean 1.6 ± 1.3 7.2 1.6 ± 1.2 4.7 2.3 ± 2.1 8.7 1.7 ± 1.3 8.7

Mean reg. 0.7 ± 0.6 2.6 0.9 ± 0.6 2.8 1.6 ± 0.9 3.7 0.9 ± 0.7 3.7

The maximum, mean and SD of landmark position differences between five acquisitions, before and after nonrigid registration, are reported.


tion (Ferrant et al., 2002) with the projected position of

the landmarks after nonrigid registration. We evaluated

landmark position differences before registration and

after registration, for a sequence of five intraoperative

images acquired over the course of a single surgery.

We evaluated position differences for a set of landmarks

nearby the surfaces used for volumetric deformation

estimate, more central landmarks located away fromthese surfaces, and landmarks nearby the tumor. The

overall landmark position error after registration, for

400 landmarks, was 0.9 ± 0.7 mm (mean ± SD). This

compares well with the voxel resolution of

0.9375 · 0.9375 · 2.5 mm3. These results are summa-

rized in more detail in Table 1.

3.3. Visualization of prostate MRI: preoperative data

matched to the intraoperative configuration

The capture of intraoperative deformation enables

preoperative imaging data to be used to better localize

potential targets (regions likely to be cancer) during

intraoperative brachytherapy or biopsy of the prostate.

Ultimately we expect this will improve outcomes for pa-

tients since improving intraoperative targeting enablesus to improve the technical efficacy of the procedure.

3.3.1. Signal intensity adjustment

An important first step in analyzing high resolution

prostate MRI acquired with an endo-rectal coil is the

elimination of signal intensity variations due to the sen-

sitivity profile of the coil. This is illustrated in Fig. 3.

The application of this signal intensity correction in-creases the robustness of following automated rigid reg-

istration, nonrigid registration and segmentation

procedures.

3.3.2. Alignment of preoperative and intraoperative

MRI of the prostate

We have investigated the accuracy of matching intra-

operative prostate alignment with a nonrigid registra-tion scheme and found that spatial overlap following

alignment is dramatically improved (Bharatha et al.,

2001; Zou et al., 2004a).

We have performed (Bharatha et al., 2001) a study

to evaluate our finite element model-based nonrigid

registration system. After creating and validating a

dataset of manually segmented glands from images ob-

tained in 10 MR-guided brachytherapy cases, we con-

ducted a set of experiments to assess our hypothesisthat the proposed registration system can significantly

improve the quality-of-matching of total gland (TG),

central gland (CG), and peripheral zone (PZ) over ri-

gid registration alone. The results showed that the

method provided a statistically significant improve-

ment in the quality of registration. Specifically, it

raised the Dice similarity coefficient (Dice, 1945; Zou

et al., 2004a; Zijdenbos et al., 1994), a measure of vol-ume agreement ranging from 0.0 (no agreement) to 1.0

(exact agreement) from pre-matched coefficients of

0.81, 0.78 and 0.59 for TG, CG and PZ, respectively,

to 0.94, 0.86 and 0.76. The volumes of CG and PZ re-

mained constant before and after the registration, indi-

cating that the method maintained the biomechanical

topology of the prostate.

The two model parameters, Young�s elastic modulus(E) and Poisson�s ratio (m), describing the elastic proper-

ties of the prostate peripheral zone and central gland,

were estimated as follows. Using a starting value of

E = 3 kPa and m = 0.4 (appropriate for brain paren-

chyma (Ferrant et al., 2001)), trial registrations were

made iteratively on a single case and the results in-

spected at different values of E and m. It became appar-

ent from initial runs that the observable deformation ofthe CG and PZ varied considerably, with the PZ experi-

encing greater deformation. By separately varying the

values of the parameters for the CG and PZ, a set of val-

ues was obtained [CG: E = 30 kPa and m = 0.2; PZ:

E = 3 kPa and m = 0.4] which produced the most consis-

tent results (on visual inspection), and was held fixed for

all trials (Bharatha et al., 2001).

Fig. 4 illustrates the use of conformal mapping tomatch preoperative and intraoperative prostate surfaces,

Fig. 3. On the left an axial T2-weighted image of the prostate shows the endo-rectal coil near-field effect as glowing bright regions surrounding the

anterior wall of the rectum. In the middle is shown the result of our intensity correction method. In both of these images, a horizontal line near the

anterior rectal wall shows location of pixels for which intensities are plotted on the right. The graph, plotted on a log scale, shows the dramatic

reduction in tissue intensity inhomogeneity.


followed by solution of an inhomogeneous linear elasticmaterial model. Mathematical details of this approach

are described in (Angenent et al., 1999; Haker et al.,

2000). A practical advantage of this approach is its

speed; our volumetric and surface warping techniques

are based on solving linear systems of equations.

Fig. 5 is an example of deformable registration of

prostate MR images. On the left, a T2-weighted axial

image taken intraoperatively in a 0.5 T interventionalscanner. Inside the prostate, benign prostate hyperpla-

sia (BPH), peripheral zone (PZ) and Foley catheter

(FOL) are labeled. Outside the prostate, neuro-vascu-

lar bundle (NVB) and Plexiglas rectal obturator (OB)

are labeled. On the right, outside the manually delin-

eated prostate capsule boundary (CAPS), the same im-

age as the left. Inside the capsule, registered imagedata from a 1.5 T T2-weighted axial scan taken pre-

operatively, and corrected for intensity inhomogeneity

resulting from the use of an endo-rectal coil. Note

increased conspicuity of BPH and prostate sub-

structures. No Foley catheter was present during pre-

operative scanning. Other data, such as pre-operative

spectroscopy, diffusion weighted images, and compos-

ite multi-parametric images (Chan et al., 2003), canbe aligned as well.

3.4. Validation of segmentation of intraoperative MRI

The ability to use imaging data to quantitatively as-

sess tumor resection or ablation post-operatively, and

Fig. 6. Illustration of variability of rater segmentation and simulta-

neous true segmentation and performance level estimation (STAPLE).

Fig. 4. Four paired views of prostate surface registration. In (a) coronal view (from anterior toward posterior) of pre-operative prostate capsule

surface (left) and intraoperative capsule surface (right). The color scheme shows matching points. In (b)–(d), other views of the same surfaces. In

(d) especially, the degree of prostate deformation is evident. Our registration method models the surface of the capsule as a thin elastic sheet, and

produces a mapping that registers the two surfaces in a way that is one to one and onto.

Fig. 5. Fused visualization of preoperative and intraoperative MRI of

the prostate.


to interoperatively monitor surgical progression, is ulti-

mately limited by the accuracy and precision of image

segmentation. In addition, the confidence we may havein visualizations of preoperative and intraoperative data

and in registration based upon surfaces extracted via

segmentation must be carefully assessed.

Fig. 6 illustrates the use of STAPLE (Warfield et al.,

2002a, 2004) to assess variability in repeated segmenta-

tions. In this example, five segmentations of a region

of cryoablation of a tumor were obtained by medical

students prior to, and then again following, a trainingsession with a radiologist. The frequency of selection

of each voxel in five repeated manual segmentations is

color coded in the left most column, the top image illus-

trating significant variability before training, and the

lower left image indicating much improved consistency

following training. On the right column, a probabilistic

estimate of the region of cryoablation derived by STA-

PLE is shown, on the top using the less consistent seg-mentations from before training, and on the right

bottom, using the more consistent segmentations follow-

ing training.

In both cases the estimated true segmentation re-gion accords well with the region of cryoablation. A

test of the change in estimated rater sensitivities indi-

cated a significant improvement following the training

session. This provides confidence that accurate and

precise assessments of tumor ablation coverage from

imaging data can be made by appropriately trained

raters. Fig. 6 illustrates that STAPLE can detect

changes in performance despite the absence of anexternal reference standard, and that, both before

and after training, the estimated true segmentation is

quite similar, despite the significant difference in rater

accuracy and precision.


4. Conclusion and future challenges

The challenges of image guided therapy impose sig-

nificant constraints on practical algorithms that enable

enhanced intraoperative navigation by capturing intra-

operative deformations. Augmented reality visualiza-tions need to be achieved at a rate compatible with

intraoperative decision making, and so fast solutions

are important. At the same time, such algorithms must

deal with the range of normal and pathological anatomy

encountered in therapeutic procedures, and so must be

robust to variations in anatomy. In addition, the use

of contrast agents and multiple imaging modalities im-

ply that such algorithms must also be robust to contrastchanges, to imaging system noise and to the sparsity of

available imaging updates. The accuracy of the overall

system, and component algorithms for registration, seg-

mentation and visualization must achieve a sufficiently

high level that clinical users have confidence in the appli-

cation of the technology.

We have described how we use a system of algo-

rithms, tied closely to the requirements of the preoper-ative and intraoperative imaging modalities, to

robustly and accurately capture intraoperative defor-

mations. This has enabled the construction of en-

hanced visualizations of intraoperative imaging data,

utilizing a range of preoperative imaging data to in-

crease the ease with which critical normal and patho-

logical anatomical and functional regions can be

rapidly identified. We have illustrated the techniqueswith imaging data from patients undergoing prostate

biopsy and neurosurgery.

Although the complexity of the image analysis we

carry out is high, we can currently achieve extremely

close matches between data acquired preoperatively

and data acquired intraoperatively. There are still open

problems and directions for further research. For exam-

ple, we currently do not model in detail the process oftumor resection during neurosurgery. It may be valuable

to include some form of tissue resection model for both

preoperative planning and for intraoperative monitor-

ing. Similar techniques may also allow improved model-

ing of discontinuous motion of organs of the abdomen.

Our approach relies upon sufficient intraoperative

imaging data to accurately assign correspondences be-

tween the preoperative and the intraoperative imagingdata. It may, in the future, be possible to carry out suf-

ficient intraoperative measurements to characterize clo-

sely the boundary conditions, such as brain/skull

interaction, CSF pressure, hydration state and other

physiological parameters, to enable prediction of brain

deformation ahead in time.

Furthermore, the extension of our algorithms to

other intraoperative imaging modalities holds out thepotential for wider implementation of such intraopera-

tive image analysis. One of the goals of our work with

intraoperative MRI has been to learn the techniques

necessary to bring preoperative MRI information into

the operating room, and to establish a precise corre-

spondence between the preoperative and intraoperative

configuration of the patient. Intraoperative MRI is an

excellent testbed for this, because it provides for flexibleimage contrast and excellent spatial resolution. How-

ever, our methods have been developed with a view to

the data available from other intraoperative imaging

modalities. For example, in external beam therapy of

the prostate an intraoperative CT image may be used,

and while CT provides useful information about the to-

tal gland, it is not as sensitive as MRI to internal bound-

aries of the prostate such as that between the peripheralzone and central gland. Our matching scheme relies

upon intraoperative identification of the total gland

boundary, not of the peripheral zone/central gland

boundary, and hence our method is also applicable in

this setting. Similarly, intraoperative ultrasound is more

widely available for neurosurgery than intraoperative

MRI. The surface of the brain and ventricles may be

readily identified in intraoperative ultrasound, but otherstructures are difficult to identify. Our method of align-

ment relies upon identification of only structures which

may also be identified from intraoperative ultrasound.

Non-invasive or minimally invasive surgeries reduce

the recovery time for patients, but have more limited

intraoperative visualization. The use of minimally inva-

sive ablation technologies, such as focused ultrasound or

microwave radiation, require enhanced intraoperativenavigation, and may expand the range of therapeutic

procedures that benefit from intraoperative capture of

deformations.

Acknowledgements

This investigation was supported in part by NIHGrants R21 MH67054, R01 LM007861, P41 RR13218,

P01 CA67165, R01 AG19513 and R33 CA99015 and

by a research grant from CIMIT.

References

Aimedieu, P., Grebe, R., Idy-Peretti, I., 2001. Study of brain white

matter anisotropy. In: Proceedings of the 23rd Annual Interna-

tional Conference of the IEEE.

Albert, M., Tempany, C.M., Schultz, D., Chen, M.H., Cormack, R.A.,

Kumar, S., Hurwitz, M.D., Beard, C., Tuncali, K., O�Leary, M.,

Topulos, G.P., Valentine, K., Lopes, L., Kanan, A., Kacher, D.,

Rosato, J., Kooy, H., Jolesz, F., Carr-Locke, D.L., Richie, J.P.,

D�Amico, A.V., 2003. Late genitourinary and gastrointestinal

toxicity after magnetic resonance image-guided prostate brachy-

therapy with or without neoadjuvant external beam radiation

therapy. Cancer 98 (5), 949–954.

Anderson, T.E., Culler, D.E., Patterson, D.A., 1995. A case for NOW

(networks of workstations). IEEE Micro. 15 (1), 54–64.


Angenent, S., Haker, S., Tannenbaum, A., Kikinis, R., 1999. On the

Laplace–Beltrami operator and brain surface flattening. IEEE

Trans. Med. Imaging 18 (8), 700–711.

Bajcsy, R., Kovacic, S., 1989. Multiresolution elastic matching.

Computer Vision Graph. Image Process. 46, 1–21.

Balay, S., Gropp, W.D., McInnes, L.C., Smith, B.F., 1997. Efficient

management of parallelism in object oriented numerical software

libraries. In: Arge, E., Bruaset, A.M., Langtangen, H.P. (Eds.),

Modern Software Tools in Scientific Computing. Birkhauser,

Basel.

Balay, S., Gropp, W.D., McInnes, L.C., Smith, B.F., 2000. PETSc 2.0

Users Manual. Argonne National Laboratory.

Bharatha, A., Hirose, M., Hata, N., Warfield, S.K., Ferrant, M., Zou,

K.H., Suarez-Santana, E., Ruiz-Alzola, J., D�Amico, A., Cormack,

R.A., Kikinis, R., Jolesz, F.A., Tempany, C.M., 2001. Evaluation

of three-dimensional finite element-based deformable registration

of pre- and intraoperative prostate imaging. Med. Phys. 28 (12),

2551–2560.

Black, P.M., Moriarty, T., Alexander 3rd, E., Stieg, P., Woodard, E.J.,

Gleason, P.L., Martin, C.H., Kikinis, R., Schwartz, R.B., Jolesz,

F.A., 1997. Development and implementation of intraoperative

magnetic resonance imaging and its neurosurgical applications.

Neurosurgery 41 (4), 831–842, discussion 842–845.

Broit, C., 1981. Optimal registration of deformed images. Ph.D.,

University of Pennsylvania.

Bro-Nielsen, M., 1997. Fast finite elements for surgery simulation.

Stud. Health Technol. Inform. 39, 395–400.

Butz, T., 2000. Pre- and intra-operative optimization of cryotherapy.

Masters, Swiss Federal Institute of Technology.

Butz, T., Warfield, S.K., Tuncali, K., Silverman, S.G., Sonnenberg,

E.v., Jolesz, F.A., Kikinis, R., 2000. Pre- and intra-operative

planning and simulation of percutaneous tumor ablation. In:

Proceedings of the MICCAI 2000: Third International Conference

on Medical Robotics, Imaging and Computer-assisted Surgery, 11–

14 October, Pittsburgh, PA, USA.

Cachier, P., Rey, D., 2000. Symmetrization of the non-rigid registra-

tion problem using inversion-invariant energies: application to

multiple sclerosis. In: MICCAI 2000. Springer, Berlin.

Chan, I., Wells 3rd, W., Mulkern, R.V., Haker, S., Zhang, J., Zou,

K.H., Maier, S.E., Tempany, C.M., 2003. Detection of prostate

cancer by integration of line-scan diffusion, T2-mapping and T2-

weighted magnetic resonance imaging; a multichannel statistical

classifier. Med. Phys. 30 (9), 2390–2398.

Chinzei, K., Miller, K., 2001. Towards MRI guided surgical manip-

ulator. Med. Sci. Monit. 7 (1), 153–163.

Chinzei, K., Warfield, S.K., Hata, N., Tempany, C.M.C., Jolesz, F.A.,

Kikinis, R., 2003. Planning, simulation and assistance with

intraoperative MRI. Min. Invas. Ther. Allied Technol. 12 (1–2),

59–64.

Christensen, G.E., Johnson, H.J., 2001. Consistent image registration.

IEEE Trans. Med. Imaging 20 (7), 568–582.

D�Amico, A., Cormack, R., Kumar, S., Tempany, C.M., 2000. Real-

time magnetic resonance imaging-guided brachytherapy in the

treatment of selected patients with clinically localized prostate

cancer. J. Endourol. 14 (4), 367–370.

D�Amico, A.V., Cormack, R.A., Tempany, C.M., 2001. MRI-guided

diagnosis and treatment of prostate cancer. N. Engl. J. Med. 344

(10), 776–777.

Dengler, J., 1986. Local motion estimation with the dynamic pyramid.

In: Proceedings of the Eight International Conference on Pattern

Recognition.

Dengler, J., Schmidt, M., 1988. The dynamic pyramid – a model for

motion analysis with controlled continuity. Int. J. Pattern Recogn.

Artif. Intell. 2 (2), 275–286.

Dengler, J., Schmidt, M., 1990. The pseudo-logarithmic transforma-

tion for robust displacement estimation. In: Grosskopf, R.E. (Ed.),

DAGM-Symposium. Springer, Berlin.

Dengler, J., Bertsch, H., Desaga, J.F., Schmidt, M., 1988. New trends

of image analysis in the medical field. Methods Inf. Med. 27 (2),

53–57.

Dice, L.R., 1945. Measures of the amount of ecologic association

between species. Ecology 26 (3), 297–302.

Edwards, P.J., Hill, D.L.G., Little, J.A., Hawkes, D.J., 1997. Defor-

mation for image guided interventions using a three component

tissue model. In: Proceedings of IPMI�97.Edwards, P.J., Hill, D.L., Little, J.A., Hawkes, D.J., 1998. A three-

component deformation model for image-guided surgery. Med.

Image Anal. 2 (4), 355–367.

Ferrant, M., Warfield, S.K., Guttmann, C.R.G., Mulkern, R.V.,

Jolesz, F.A., Kikinis, R., 1999. 3D Image matching using a finite

element based elastic deformation model. In: Proceedings of

MICCAI 99: Second International Conference on Medical Image

Computing and Computer-assisted Intervention, 19–22 September,

Cambridge, England.

Ferrant, M., Nabavi, A., Macq, B., Warfield, S.K., 2000a. Deformable

modeling for characterizing biomedical shape changes. In: Pro-

ceedings of DGCI2000: Discrete Geometry for Computer Imagery,

13–15 December, Uppsala, Sweden.

Ferrant, M., Warfield, S.K., Nabavi, A., Macq, B., Kikinis, R., 2000b.

Registration of 3D intraoperative mr images of the brain using a

finite element biomechanical model. In: Proceedings of MICCAI

2000: Third International Conference on Medical Robotics,

Imaging and Computer-assisted Surgery, 11–14 October, Pitts-

burgh, USA. A.M.D. a. S. Delp (Ed.), Springer, Heidelberg,

Germany.

Ferrant, M., Warfield, S.K., Nabavi, A., Macq, B., Kikinis, R., 2000c.

Registration of 3D intraoperative MR images of the brain using a

finite element biomechanical model. In: Proceedings of MICCAI

2000: Third International Conference on Medical Robotics,

Imaging and Computer-assisted Surgery, 11–14 October, Pitts-

burgh, PA, USA.

Ferrant, M., Nabavi, A., Macq, B., Jolesz, F.A., Kikinis, R., Warfield,

S.K., 2001. Registration of 3-D intraoperative MR images of the

brain using a finite-element biomechanical model. IEEE Trans.

Med. Imaging 20 (12), 1384–1397.

Ferrant, M., Nabavi, A., Macq, B., Black, P.M., Jolesz, F.A., Kikinis,

R., Warfield, S.K., 2002. Serial registration of intraoperative MR

images of the brain. Med. Image Anal. 6 (4), 337–359.

Fisher, J., Cosman, E., Wible, C., Wells, W., 2001. Adaptive entropy

rates for fMRI time-series analysis. In: Proceedings of the Fourth

International Conference on Medical Image Computing and

Computer-assisted Intervention, 14–17 October, Utrecht, The

Netherlands.

Freund, R.W., Golub, G.H., Nachtigal, N.M., 1992. Iterative solution

of linear systems. Acta Numer., 1–44.

Geiger, B., 1993. Three Dimensional Modeling of Human Organs and

its Application to Diagnosis and Surgical Planning. INRIA,

Sophia-Antipoles.

Gering, D.T., Nabavi, A., Kikinis, R., Hata, N., O�Donnell, L.J.,

Grimson, W.E., Jolesz, F.A., Black, P.M., Wells 3rd, W.M., 2001.

An integrated visualization system for surgical planning and

guidance using image fusion and an open MR. J. Magn. Reson.

Imaging 13 (6), 967–975.

Guimond, A., Guttmann, C.R.G., Warfield, S.K., Westin, C.-F., 2002.

Deformable registration of DT-MRI data based on transformation

invariant tensor characteristics. In: Proceedings of the Interna-

tional Symposium on Biomedical Imaging, Washington, DC.

Hagemann, A., Rohr, K., Stiehl, H.S., Spetzger, U., Gilsbach, J.M.,

1999. Biomechanical modeling of the human head for physically

based, nonrigid image registration. IEEE Trans. Med. Imaging 18

(10), 875–884.

Hagemann, A., Rohr, K., Stiehl, H.S., 2002. Coupling of fluid and

elastic models for biomechanical simulations of brain deformations

using FEM. Med. Image Anal. 6 (4), 375–388.


Haker, S., Angenent, S., Tannenbaum, A., Kikinis, R., Sapiro, G.,

Halle, M., 2000. Conformal surface parameterization for texture

mapping. IEEE Trans. Visual. Comput. Graph. 6 (2), 181–189.

Hata, N., 1998. Rigid and deformable medical image registration for

image-guided surgery. Ph.D., University of Tokyo, Tokyo, Japan.

Hata, N., Dohi, T., Warfield, S.K., Wells, W., Kikinis, R., Jolesz,

F.A., 1998. Multimodality deformable registration of pre- and

intraoperative images for MRI-guided brain surgery. In: Proceed-

ings of the MICCAI 98: First International Conference on Medical

Image Computing and Computer-assisted Intervention, 11–13

October, 1998, Boston, USA. W.M. Wells, A. Colchester, S. Delp.

(Ed.), Springer, Heidelberg, Germany.

Hata, N., Nabavi, A., Warfield, S., Wells, W.M., Kikinis, R., Jolesz,

F.A., 1999. A volumetric optical flow method for measurement of

brain deformation from intraoperative magnetic resonance images.

In: Proceedings of the MICCAI �99: Second International Confer-

ence on Medical Image Computing and Computer-assisted Inter-

vention, 19–22 September, Cambridge, England.

Hata, N., Nabavi, A., Wells 3rd, W.M., Warfield, S.K., Kikinis, R.,

Black, P.M., Jolesz, F.A., 2000. Three-dimensional optical flow

method for measurement of volumetric brain deformation from

intraoperative MR images. J. Comput. Assist. Tomogr. 24 (4), 531–

538.

Hata, N., Jinzaki, M., Kacher, D., Cormak, R., Gering, D., Nabavi,

A., Silverman, S.G., D�Amico, A.V., Kikinis, R., Jolesz, F.A.,

Tempany, C.M., 2001. MR imaging-guided prostate biopsy with

surgical navigation software: device validation and feasibility.

Radiology 220 (1), 263–268.

Hirose, M., Bharatha, A., Hata, N., Zou, K.H., Warfield, S.K.,

Cormack, R.A., D�Amico, A., Kikinis, R., Jolesz, F.A., Tempany,

C.M., 2002. Quantitative MR imaging assessment of prostate gland

deformation before and during MR imaging-guided brachyther-

apy. Acad. Radiol. 9 (8), 906–912.

Horn, B.K.P., Schunck, B.G., 1981. Determining optical flow. Artif.

Intell. 17, 185–203.

Inder, T.E., Huppi, P.S., Warfield, S., Kikinis, R., Zientara, G.P.,

Barnes, P.D., Jolesz, F., Volpe, J.J., 1999. Periventricular white

matter injury in the premature infant is followed by reduced

cerebral cortical gray matter volume at term. Ann. Neurol. 46 (5),

755–760.

Iosifescu, D., Shenton, M.E., Kikinis, R., Warfield, S.K., Dengler, J.,

McCarley, R.W., 1995. Elastically matching an MR imaging brain

atlas onto a new MR image of the brain. In: Proceedings of

RSNA�95.Iosifescu, D.V., Shenton, M.E., Warfield, S.K., Kikinis, R., Dengler,

J., Jolesz, F.A., McCarley, R.W., 1997. An automated registration

algorithm for measuring MRI subcortical brain structures. Neu-

roimage 6 (1), 13–25.

Johnson, H.J., Christensen, G.E., 2002. Consistent landmark and

intensity-based image registration. IEEE Trans. Med. Imaging 21

(5), 450–461.

Jolesz, F.A., 1997. 1996 RSNA Eugene P. Pendergrass new horizons

lecture. Image-guided procedures and the operating room of the

future. Radiology 204 (3), 601–612.

Kaus, M., Warfield, S.K., Jolesz, F.A., Kikinis, R., 1999a. Adaptive

template moderated brain tumor segmentation in MRI. In:

Bildverarbeitung fuer die Medizin.

Kaus, M.R., Warfield, S.K., Nabavi, A., Chatzidakis, E., Black, P.M.,

Jolesz, F.A., Kikinis, R., 1999b. Segmentation of MRI of menin-

giomas and low grade gliomas. In: Proceedings of MICCAI 99:

Second International Conference on Medical Image Computing

and Computer-assisted Intervention, 19–22 September, Cambridge,

England.

Kaus, M.R., Nabavi, A., Mamisch, C.T., Wells, W.M., Jolesz, F.A.,

Kikinis, R., Warfield, S.K., 2000. Simulation of corticospinal tract

displacement in patients with brain tumors. In: Proceedings of

MICCAI 2000: Third International Conference on Medical

Robotics, Imaging and Computer-assisted Surgery, 11–14 October,

Pittsburgh, PA, USA.

Kaus, M.R., Warfield, S.K., Nabavi, A., Black, P.M., Jolesz, F.A.,

Kikinis, R., 2001. Automated segmentation of MR images of brain

tumors. Radiology 218 (2), 586–591.

Kemper, C.A., 2003. Incorporation of diffusion tensor MRI in non-

rigid registration for image-guided neurosurgery. Master�s, Artifi-

cial Intelligence Laboratory, Massachusetts Institute of Technol-

ogy, Boston.

Kettenbach, J., Kacher, D.F., Koskinen, S.K., Silverman, S.G.,

Nabavi, A., Gering, D., Tempany, C.M., Schwartz, R.B., Kikinis,

R., Black, P.M., Jolesz, F.A., 2000. Interventional and intraoper-

ative magnetic resonance imaging. Annu. Rev. Biomed. Eng. 2,

661–690.

Kikinis, R., Shenton, M.E., Gerig, G., Martin, J., Anderson, M.,

Metcalf, D., Guttmann, C.R., McCarley, R.W., Lorensen, W.,

Cline, H., et al., 1992. Routine quantitative analysis of brain and

cerebrospinal fluid spaces with MR imaging. J. Magn. Reson.

Imaging 2 (6), 619–629.

Kikinis, R., Guttmann, C.R., Metcalf, D., Wells 3rd, W.M., Ettinger,

G.J., Weiner, H.L., Jolesz, F.A., 1999. Quantitative follow-up of

patients with multiple sclerosis using MRI: technical aspects. J.

Magn. Reson. Imaging 9 (4), 519–530.

Kikinis, R., Mehta, N.R., Nabavi, A., Chatzidakis, E., Warfield, S.,

Gering, D., Weisenfeld, N., Pergolizzi, R.S., Schwarz, R.B., Hata,

N., Wells, W., Grimson, E., Black, P.M., Jolesz, F.A., 2000.

Intraoperative visualization. In: Mazziotta, J.C., Toga, A.W.,

Frackowiak, R.S.J. (Eds.), Brain Mapping: The Disorders. Aca-

demic Press, San Diego, USA.

Magnotta, V.A., Bockholt, H.J., Johnson, H.J., Christensen, G.E.,

Andreasen, N.C., 2003. Subcortical, cerebellar, and magnetic

resonance based consistent brain image registration. Neuroimage

19 (2 Pt 1), 233–245.

Mangin, J.F., 2000. Entropy minimization for automatic correction of

intensity nonuniformity. In: Proceedings of the IEEEWorkshop on

Mathematical Methods in Biomedical Image Analysis, 11–12 June,

Hilton Head Island, SC, USA.

Miga, M.I., Paulsen, K.D., Lemery, J.M., Eisner, S.D., Hartov, A.,

Kennedy, F.E., Roberts, D.W., 1999a. Model-updated image

guidance: initial clinical experiences with gravity-induced brain

deformation. IEEE Trans. Med. Imaging 18 (10), 866–874.

Miga, M.I., Roberts, D.W., Hartov, A., Eisner, S., Lemery, J.,

Kennedy, F.E., Paulsen, K.D., 1999b. Updated neuroimaging

using intraoperative brain modeling and sparse data. Stereotact.

Funct. Neurosurg. 72 (2–4), 103–106.

Miga, M.I., Paulsen, K.D., Hoopes, P.J., Kennedy, F.E., Hartov, A.,

Roberts, D.W., 2000a. In vivo modeling of interstitial pressure in

the brain under surgical load using finite elements. J. Biomech.

Eng. 122 (4), 354–363.

Miga, M.I., Paulsen, K.D., Hoopes, P.J., Kennedy Jr., F.E., Hartov,

A., Roberts, D.W., 2000b. In vivo quantification of a homogeneous

brain deformation model for updating preoperative images during

surgery. IEEE Trans. Biomed. Eng. 47 (2), 266–273.

Miga, M.I., Paulsen, K.D., Kennedy, F.E., Hoopes, P.J., Hartov, A.,

Roberts, D.W., 2000c. In vivo analysis of heterogeneous brain

deformation computations for model-updated image guidance.

Comput. Methods Biomech. Biomed. Eng. 3 (2), 129–146.

Miga, M.I., Roberts, D.W., Kennedy, F.E., Platenik, L.A., Hartov,

A., Lunn, K.E., Paulsen, K.D., 2001. Modeling of retraction and

resection for intraoperative updating of images. Neurosurgery 49

(1), 75–84, discussion 84–85.

Miller, K., Chinzei, K., 1997. Constitutive modelling of brain tissue:

experiment and theory. J. Biomech. 30 (11–12), 1115–1121.

Miller, K., Chinzei, K., 2000. New UWA robot – possible application

to robotic surgery. Biomed. Sci. Instrum. 36, 135–140.

Miller, K., Chinzei, K., 2002. Mechanical properties of brain tissue in

tension. J. Biomech. 35 (4), 483–490.


Miller, K., Chinzei, K., Orssengo, G., Bednarz, P., 2000. Mechanical

properties of brain tissue in-vivo: experiment and computer

simulation. J. Biomech. 33 (11), 1369–1376.

Nabavi, A., Black, P.M., Gering, D.T., Westin, C.F., Mehta, V.,

Pergolizzi Jr., R.S., Ferrant, M., Warfield, S.K., Hata, N.,

Schwartz, R.B., Wells 3rd, W.M., Kikinis, R., Jolesz, F.A., 2001.

Serial intraoperative magnetic resonance imaging of brain shift.

Neurosurgery 48 (4), 787–797, discussion 797–798.

Nagel, H.H., Enkelmann, W., 1984. Towards the estimation of

displacement vector fields by ‘‘oriented smoothness’’ constraints.

In: Proceedings of the 7th International Conference on Pattern

Recognition.

Nagel, H.H., Enkelmann, W., 1986. An investigation of smoothness

constraints for the estimation of displacement vector fields from

image sequences. IEEE Trans. Pattern Anal. Mach. Intell. 8, 565–

593.

Nielsen, M., Johansen, P., Jackson, A.D., Lautrup, B., 2002. Brownian

warps: a least committed prior for non-rigid registration. In: Dohi,

T., Kikinis, R. (Eds.), MICCAI 2002. Springer, Berlin.

O�Donnell, L., Haker, S., Westin, C.-F., 2002. New approaches to

estimation of white matter connectivity in diffusion tensor MRI:

elliptic PDEs and geodesics in a tensor-warped space. In:

Proceedings of MICCAI�02: Fifth International Conference on

Medical Image Computing and Computer-assisted Intervention,

25–28 September, Tokyo, Japan.

Paulsen, K.D., Miga, M.I., Kennedy, F.E., Hoopes, P.J., Hartov, A.,

Roberts, D.W., 1999. A computational model for tracking

subsurface tissue deformation during stereotactic neurosurgery.

IEEE Trans. Biomed. Eng. 46 (2), 213–225.

Platenik, L.A., Miga, M.I., Roberts, D.W., Lunn, K.E., Kennedy,

F.E., Hartov, A., Paulsen, K.D., 2002. In vivo quantification of

retraction deformation modeling for updated image-guidance

during neurosurgery. IEEE Trans. Biomed. Eng. 49 (8), 823–835.

Prange, M.T., Margulies, S.S., 2002. Regional, directional, and age-

dependent properties of the brain undergoing large deformation. J.

Biomech. Eng. 124 (2), 244–252.

Rexilius, J., 2001. Physics-based nonrigid registration for medical

image analysis. Master, Medical University of Luebeck, Luebeck,

Germany.

Rexilius, J., Warfield, S.K., Guttmann, C.R.G., Wei, X., Benson, R.,

Wolfson, L., Shenton, M., Handels, H., Kikinis, R., 2001. A novel

nonrigid registration algorithm and applications. In: Proceedings

of MICCAI �01: Fourth International Conference on Medical

Image Computing and Computer-assisted Intervention, 14–17

October, Utrecht, The Netherlands.

Roberts, D.W., Hartov, A., Kennedy, F.E., Miga, M.I., Paulsen,

K.D., 1998. Intraoperative brain shift and deformation: a quan-

titative analysis of cortical displacement in 28 cases. Neurosurgery

43 (4), 749–758, discussion 758–760.

Roberts, D.W., Miga, M.I., Hartov, A., Eisner, S., Lemery, J.M.,

Kennedy, F.E., Paulsen, K.D., 1999. Intraoperatively updated

neuroimaging using brain modeling and sparse data. Neurosurgery

45 (5), 1199–1206, discussion 1206–1207.

Roberts, D.W., Lunn, K., Sun, H., Hartov, A., Miga, M., Kennedy,

F., Paulsen, K., 2001. Intra-operative image updating. Stereotact.

Funct. Neurosurg. 76 (3–4), 148–150.

Ruiz-Alzola, J., Westin, C.-F., Warfield, S.K., Nabavi, A., Kikinis, R.,

2000. Nonrigid registration of 3D scalar, vector and tensor medical

data. In: Proceedings of MICCAI 2000: Third International

Conference on Medical Robotics, Imaging and Computer-assisted

Surgery, 11–14 October, Pittsburgh, PA, USA.

Ruiz-Alzola, J., Westin, C.F., Warfield, S.K., Alberola, C., Maier, S.,

Kikinis, R., 2002. Nonrigid registration of 3D tensor medical data.

Med. Image Anal. 6 (2), 143–161.

Schmidt, M., Dengler, J., 1989. Adapting multi-grid methods to the

class of elliptic partial differential equation appearing in the

estimation of displacement vector fields. In: Cantoni, V., Creutz-

berg, H., Levialdi, S., Wolf, H. (Eds.), Recent Issues in Pattern

Analysis and Recognition, Lecture Notes in Computer Science 399.

Springer, Heidelberg.

Schroeder, W., Martin, K., Lorensen, B., 1996. The visualization

toolkit, an object-oriented approach to 3D graphics 1996. http://

public.kitware.com/VTK/.

Sierra, R., 2001. Nonrigid registration of diffusion tensor images. M.S.,

Swiss Federal Institute of Technology Zurich, Switzerland.

Silverman, S.G., Collick, B.D., Figueira, M.R., Khorasani, R.,

Adams, D.F., Newman, R.W., Topulos, G.P., Jolesz, F.A., 1995.

Interactive MR-guided biopsy in an open-configuration MR

imaging system. Radiology 197 (1), 175–181.

Skrinjar, O., Duncan, J.S., 1999. Real time 3D brain shift compen-

sation. In: Proceedings of IPMI.

Skrinjar, O., Studholme, C., Nabavi, A., Duncan, J.S., 2001. Steps

toward a stereo-camera-guided biomechanical model for brain shift

compensation. In: Proceedings of the International Conference of

Information Processing in Medical Imaging.

Skrinjar, O., Nabavi, A., Duncan, J., 2002. Model-driven brain shift

compensation. Med. Image Anal. 6 (4), 361–373.

Sterling, T., Becker, D.J., Savarese, D., Dorband, J.E., Ranawak,

U.A., Packer, C.V., 1995. Beowulf: a parallel workstation for

scientific computation. In: Proceedings of the International Con-

ference on Parallel Processing, Oconomowoc, WI.

Talos, F., O�Donnell, L., Westin, C.F., Warfield, S.K., Wells III,

W.M., Yoo, S.-S., Panych, L., Golby, A., Mamata, H., Maier,

S.E., Ratiu, P., Guttmann, C.R.G., Black, P.M., Jolesz, F.A.,

Kikinis, R., 2003. Diffusion tensor and functional MRI fusion with

anatomical MRI for image-guided neurosurgery. Presented at

MICCAI 2003: Sixth International Conference on Medical Image

Computing and Computer-assisted Intervention.

Tei, A., Talos, F., Bharatha, A., Ferrant, M., Black, P.M., Jolesz,

F.A., Kikinis, R., Warfield, S.K., 2001. Tracking volumetric brain

deformation during image guided neurosurgery. In: Proceedings of

VISIM: Information Retrieval and Exploration in Large Medical

Image Collections, in conjunction with MICCAI 2001: the Fourth

International Conference on Medical Image Computing and

Computer-assisted Intervention, 14–17 October, 2001, Utrecht,

The Netherlands. Springer, Heidelberg, Germany.

Tempany, C.M., Stewart, E.A., McDannold, N., Quade, B.J., Jolesz,

F.A., Hynynen, K., 2003. MR imaging-guided focused ultrasound

surgery of uterine leiomyomas: a feasibility study. Radiology 226

(3), 897–905.

Tsai, A., Fisher, J., Wible, C., Wells, W., Kim, J., Willsky, A., 1999.

Analysis of fMRI data using mutual information. In: Proceedings

of the Second International Conference on Medical Image Com-

puting and Computer-assisted Intervention, 19–22 September,

Cambridge, UK.

Viola, P., Wells, W.M., 1997. Alignment by maximization of mutual

information. Int. J. Comput. Vis. 24 (2), 137–154.

Warfield, S.K., 1997. Segmentation of magnetic resonance images of

the brain. Ph.D., The University of New South Wales, Sydney,

Australia.

Warfield, S., Dengler, J., Zaers, J., Guttmann, C.R., Wells 3rd,

W.M., Ettinger, G.J., Hiller, J., Kikinis, R., 1995a. Automatic

identification of gray matter structures from MRI to improve the

segmentation of white matter lesions. J. Image Guid. Surg. 1 (6),

326–338.

Warfield, S., Dengler, J., Zaers, J., Guttmann, C.R.G., III, W.M.W.,

Ettinger, G.J., Hiller, J., Kikinis, R., 1995b. Automatic identifica-

tion of grey matter structures from MRI to improve the segmen-

tation of white matter lesions. In: Proceedings of MRCAS�95Second International Symposium on Medical Robotics and Com-

puter-assisted Surgery, 4–7 November, 1995, Baltimore, USA.

Warfield, S.K., Jolesz, F.A., Kikinis, R., 1998a. A high performance

computing approach to the registration of medical imaging data.

Parallel Comput. 24 (9–10), 1345–1368.

http://public.kitware.com/VTK/

http://public.kitware.com/VTK/


Warfield, S.K., Jolesz, F.A., Kikinis, R., 1998b. Real-time image

segmentation for image-guided surgery. In: Proceedings of SC

1998: High Performance Networking and Computing Conference,

7–13 November, Orlando, FL, USA.

Warfield, S.K., Kaus, M., Jolesz, F.A., Kikinis, R., 1998c. Adaptive

template moderated spatially varying statistical classification. In:

Proceedings of MICCAI 98: First International Conference on

Medical Image Computing and Computer-assisted Intervention,

11–13 October, Boston, MA, USA.

Warfield, S.K., Robatino, A., Dengler, J., Jolesz, F.A., Kikinis, R.,

1999. Nonlinear registration and template driven segmentation. In:

Toga, A.W. (Ed.), Brain Warping. Academic Press, San Diego,

USA.

Warfield, S.K., Ferrant, M., Gallez, X., Nabavi, A., Jolesz, F.A.,

Kikinis, R., 2000a. Real-time biomechanical simulation of volu-

metric brain deformation for image guided neurosurgery. In:

Proceedings of SC 2000: High Performance Networking and

Computing Conference, 4–10 November, 2000, Dallas, USA.

Warfield, S.K., Kaus, M., Jolesz, F.A., Kikinis, R., 2000b. Adaptive,

template moderated, spatially varying statistical classification.

Med. Image Anal. 4 (1), 43–55.

Warfield, S.K., Mulkern, R.V., Winalski, C.S., Jolesz, F.A., Kikinis,

R., 2000c. An image processing strategy for the quantification and

visualization of exercise-induced muscle MRI signal enhancement.

J. Magn. Reson. Imaging 11 (5), 525–531.

Warfield, S.K., Nabavi, A., Butz, T., Tuncali, K., Silverman, S.G.,

Black, P.M., Jolesz, F.A., Kikinis, R., 2000d. Intraoperative

segmentation and nonrigid registration for image guided therapy.

In: Proceedings of MICCAI 2000: Third International Conference

on Medical Robotics, Imaging and Computer-assisted Surgery, 11–

14 October, Pittsburgh, PA, USA.

Warfield, S.K., Rexilius, J., Huppi, P.S., Inder, T.E., Miller, E.G.,

Wells, W.M., Zientara, G.P., Jolesz, F.A., Kikinis, R., 2001. A

binary entropy measure to assess nonrigid registration algorithms.

In: Proceedings of MICCAI 2001: Fourth International Confer-

ence on Medical Image Computing and Computer-assisted Inter-

vention, 14–17 October, Utrecht, The Netherlands.

Warfield, S.K., Guimond, A., Roche, A., Bharatha, A., Tei, A., Talos,

F., Rexilius, J., Ruiz-Alzola, J., Westin, C.-F., Haker, S., Ange-

nent, S., Tannenbaum, A., Jolesz, F.A., Kikinis, R., 2002a.

Advanced nonrigid registration algorithms for image fusion. In:

Toga, A.W. (Ed.), Brain Mapping: The Methods. Academic Press,

San Diego, USA.

Warfield, S.K., Talos, F., Tei, A., Bharatha, A., Nabavi, A., Ferrant,

M., McL. Black, P., Jolesz, F.A., Kikinis, R., 2002b. Real-time

registration of volumetric brain MRI by biomechanical simulation

of deformation during image guided neurosurgery. Comput. Visual

Sci. 5, 3–11.

Warfield, S.K., Zou, K.H., Kaus, M.R., Wells, W.M., III, 2002c.

Simultaneous validation of image segmentation and assessment of

expert quality. In: Proceedings of the International Symposium on

Biomedical Imaging, Washington, DC.

Warfield, S.K., Zou, K.H., Wells III, W.M., 2002d. Validation of

image segmentation and expert quality with an expectation-

maximization algorithm. In: MICCAI 2002, Fifth International

Conference on Medical Image Computing and Computer-assisted

Intervention. Springer, Heidelberg, Germany.

Warfield, S.K., Talos, F., Kemper, C.A., O�Donnell, L., Westin, C.-F.,

Wells, W.M., Black, P.M., Jolesz, F.A., Kikinis, R., 2003.

Capturing brain deformation. In: Proceedings of International

Symposium on Surgical Simulation and Soft Tissue Modeling,

Juan-les-Pins, France.

Warfield, S.K., Zou, K.H., Wells, W.M., 2004. Simultaneous truth and

performance level estimation (STAPLE) an algorithm for the

validation of image segmentation. IEEE Trans. Med. Imag. 23 (7),

903–921.

Wei, X., Warfield, S.K., Zou, K.H., Wu, Y., Li, X., Guimond, A.,

Mugler 3rd, J.P., Benson, R.R., Wolfson, L., Weiner, H.L.,

Guttmann, C.R., 2002. Quantitative analysis of MRI signal

abnormalities of brain white matter with high reproducibility and

accuracy. J. Magn. Reson. Imaging 15 (2), 203–209.

Weisenfeld, N.I., Warfield, S.K., 2004. Normalization of joint image-

intensity statistics in MRI using the Kullback–Leibler divergence.

In: International Symposium on Biomedical Imaging, Arlington,

VA.

Westin, C.F., Maier, S.E., Mamata, H., Nabavi, A., Jolesz, F.A.,

Kikinis, R., 2002. Processing and visualization for diffusion tensor

MRI. Med. Image Anal. 6 (2), 93–108.

Wong, T.Z., Silverman, S.G., Fielding, J.R., Tempany, C.M., Hyny-

nen, K., Jolesz, F.A., 1998. Open-configuration MR imaging,

intervention, and surgery of the urinary tract. Urol. Clin. North

Am. 25 (1), 113–122.

Yezzi, A., Kichenesamy, S., Olver, P., Tannenbaum, A., 1997.

Geometric active contours for segmentation of medical imagery.

IEEE Trans. Med. Imaging 16, 199–209.

Yezzi, A., Tsai, A., Willsky, A., 2000. Medical image segmentation via

coupled curve evolution equations with global constraints. In:

Proceedings of the Mathematical Methods in Biomedical Image

Analysis, New York.

Zienkiewicz, O.C., Taylor, R.L., 1994. The Finite Element Method:

Basic Formulation and Linear Problems, fourth ed. McGraw Hill,

New York.

Zijdenbos, A.P., Dawant, B.M., Margolin, R.A., Palmer, A.C.,

1994. Morphometric analysis of white matter lesions in MR

images: method and validation. IEEE Trans. Med. Imaging 13,

716–724.

Zou, K.H., Warfield, S.K., Fielding, J.R., Tempany, C.M., William

3rd, M.W., Kaus, M.R., Jolesz, F.A., Kikinis, R., 2003. Statistical

validation based on parametric receiver operating characteristic

analysis of continuous classification data. Acad. Radiol. 10 (12),

1359–1368.

Zou, K.H., Warfield, S.K., Bharatha, A., Tempany, C.M., Kaus,

M.R., Haker, S.J., Wells III, W.M., Jolesz, F.A., Kikinis, R.,

2004a. Statistical validation of image segmentation quality based

on a spatial overlap index. Acad. Radiol. 11 (2), 178–189.

Zou, K.H., Wells III, W.M., Kikinis, R., Warfield, S.K., 2004b. Three

validation metrics for automated probabilistic image segmentation

of brain tumors. Statist. Med. 23 (8), 1259–1282.

Capturing intraoperative deformations: research experience at Brigham and Womens hospital

Documents

Transcript of Capturing intraoperative deformations: research experience at Brigham and Womens hospital