AdvancedNonrigidRegistrationAlgorithmsfor
ImageFusion
SimonK. Warfield AlexandreGuimond Alexis Roche
Aditya Bharatha Alida Tei Florin Talos JanRexilius
JuanRuiz-Alzola Carl-FredrikWestin StevenHaker
SigurdAngenent Allen Tannenbaum FerencA. Jolesz
RonKikinis
1
Intr oduction
Medical imagesarebroughtinto spatialcorrespondence,or aligned, by theuseof registration
algorithms. Nonrigid registrationrefersto the setof techniquesthat allow the alignmentof
datasetsthat aremismatchedin a nonrigid, or nonuniformmanner. Suchmisalignmentscan
resultfrom physicaldeformationprocesses,or canbea resultof morphologicalvariability. For
example,physicaldeformationin thebraincanoccurduring neurosurgeryasa resultof such
factorsasswelling,cerebrospinalfluid (CSF)loss,hemorrhageandtheinterventionitself. Non-
rigid deformationis alsocharacteristicof theorgansandsoft tissuesof theabdomenandpelvis.1This chapterappearedin Brain Mapping: The Methods,SecondEdition, aschapter24 on pages661–690,
publishedby AcademicPressof SanDiego,USA in 2002.
In addition,nonrigidmorphologicaldifferencescanarisewhencomparisonsaremadeamong
imagedatasetsacquiredfrom different individuals. Thesechangescanbe a resultof normal
anatomicalvariability or theproductof pathologicalprocesses.Becausethegrossstructureof
the brain is essentiallysimilar amonghumans(andeven amongrelatedspecies),the factors
describedabovetendto producelocal nonrigidshapedifferences.
Nonrigidbrainregistrationtechniqueshavenumerousapplications.They havebeenusedto
alignscansof differentbrains,permittingthecharacterizationof normalandpathologicalmor-
phologicalvariation(brainmapping).They have alsobeenusedto align anatomicaltemplates
with specificdatasets,thusfacilitatingsegmentation(i.e. segmentationby registration).More
recently, thesetechniqueshavebeenusedto capturechangeswhichoccurduringneurosurgery.
With theongoingdevelopmentof robustalgorithmsandadvancedhardwareplatforms,further
applicationsin surgical visualizationandenhancedfunctionalimageanalysisareinevitable.
Oneexciting applicationof nonrigidregistrationalgorithmsis in theautomaticregistration
of multimodalimagedata.Rigid registrationof multimodaldatahasbeengreatlyfacilitatedby
the framework provided by mutual information(MI). However, MI-basedstrategiesto effec-
tively capturelargenonrigidshapedifferencesarestill beingexplored. An alternateapproach
is to normalizemultimodality imagesandthusreducetheproblemto a monomodalitymatch.
In thefirst section,wepresentanonrigidregistrationmethodwhichusesanintensitytransform
which allows a singleintensityin onemodalityto bemappedonto(up to) two intensities.The
methodis iterative, combiningin eachiterationan intensitycorrectionanda geometrictrans-
form using intensity-similaritycriteria. The methodis appliedin two caseswith promising
results.
In thenext section,we turn our attentionto theissueof imageregistrationandfusiondur-
ing neurosurgery. It is commonto desireto align preoperative datawith imagesof thepatient
acquiredduringneurosurgery. It is now widely acknowledgedthatduringneurosurgicalopera-
2
tions,nonrigidchangesin theshapeof thebrainoccurasa resultof theinterventionitself and
dueto reactive physiologicalchanges.Thesedeformations(“brain shift”) make it difficult to
relatepreoperative imagedatato theintraoperativeanatomyof thepatient.Sincepreoperative
imagingis not subjectto thesametime constraintsandlimitations in tissuecontrastselection
methodsasintraoperative imaging,a majorgoalhasbeento developrobustnonrigidregistra-
tion algorithmsfor matchingof preoperative imagedataonto intraoperative imagedata. We
presentourbiomechanicalmodelingalgorithmwhichcancapturenonrigiddeformationsbased
on surfacechangesandinfer volumetricdeformationusinga finite elementdiscretization.We
alsodescribeour early prospective experienceusing the methodduring neurosurgical cases,
andprovideexamplesof theenhancedvisualizationswhichareproduced.
In thethird section,webuild uponthethemeof physics-basedmodelsby presentinganovel
inhomogeneouselasticitymodelwhichusesalocalsimilarity measureto obtainaninitial sparse
estimateof thedeformationfield. Themethodincludesautomaticfeaturepoint extractionus-
ing a nonlineardiffusion filter. Correspondencedetectionis achieved by maximizinga local
normalizedcross-correlation.The sparseestimatesof the deformationfield calculatedat the
featurepointsarethenintroducedasexternalforces,restrictingtheregistrationprocesssothat
thedeformationfield is fixedat thosepoints.An advantageof themethodis thatfeaturepoints
andcorrespondencesareestablishedautomatically. Thusneithersegmentationnor themanual
identificationof correspondencesis required.
In thefourth sectionwe discussregistrationof Dif fusionTensorMRI dataandintroducea
framework for nonrigid registrationof tensordata(including thespecialcaseof vectordata).
The approachis basedon a multiresolutiontemplatematchingschemefollowed by interpo-
lation of the sparsedisplacementfield usinga Kriging interpolator. After warping the data,
the tensorsare locally realignedbasedon information from the deformationgradientof the
displacement.
3
In the fifth section,we presenta novel methodfor producingareapreservingsurfacede-
formations,andmore generalmasspreservingareaandvolumedeformations,basedon the
minimizationof a functionalof Monge–Kantorovich type. Thetheoryis basedaroundtheop-
timal masstransportproblemof minimizing thecostof redistributingacertainamountof mass
betweentwo distributionsgivena priori. Herethecostis a functionof thedistanceeachbit of
materialis moved,weightedby its mass.Theproblemof optimaltransportis classicalandhas
appearedin econometrics,fluid dynamics,automaticcontrol,transportation,statisticalphysics,
shapeoptimization,expert systems,andmeteorology. We show how the resultinglow-order
differentialequationsmaybeusedfor imageregistration.
The challengeof nonrigid registrationremainsoneof the outstandingopenproblemsin
medicalimageanalysis.New algorithmdevelopments,often targetedtowardspecificclinical
applications,havehelpedto identify furtherunsolvedissues.Thischapterprovidesanoverview
of thenonrigidregistrationalgorithmsbeingpursuedtodayattheSurgicalPlanningLaboratory.
4
1 Inter -Modality and Multi-Contrast Images
1.1 Intr oduction
Automatic registrationtechniquesof brain imageshave beendevelopedfollowing two main
trends: 1) registrationof multimodal imagesusing low to intermediatedegreetransforma-
tions (lessthan a few hundredparameters),and2) registrationof monomodalimagesusing
high-dimensionalvolumetricmaps(elasticor fluid deformationswith hundredsof thousands
of parameters).Thefirst category mainly addressesthe fusionof complementaryinformation
obtainedfrom different imaging modalities. The secondcategory’s predominantpurposeis
the evaluationof either the anatomicalevolution processpresentin a particularsubjector of
anatomicalvariationsbetweendifferentsubjects.Despitepromisingearlywork suchas(Hata,
1998),densetransformationfield multimodalregistrationhas,sofar, remainedrelatively unex-
plored.
Researchonmultimodalregistrationculminatedwith theconceptof mutualinformation(MI)
(Viola andWells, 1995;Wells et al., 1996b;Hataet al., 1996;Viola andWells, 1997;Maes
et al., 1997),leadingto a new classof rigid/affine registrationalgorithms.In this framework,
the registrationof two imagesis performedby maximizingtheir MI with respectto thetrans-
formationspace.A significantreasonfor thesuccessof MI asasimilarity measureresidesin its
generality, asit doesnotuseany prior informationabouttherelationshipbetweentheintensities
of the images.For instance,MI doesnot assumea linear relationshipasis typically thecase
in standardopticalflow techniques.Also, unlikesomeearlierapproaches,MI doesnot require
theidentificationof correspondingfeaturesin theimagesto beregistered.
Significantwork hasbeendonein establishingthe applicability of MI for nonrigid regis-
tration (Gaenset al., 1998;Maintz et al., 1998;Meyer et al., 1999;Likar andPernus,2000;
Hellier andBarillot, 2000;Rueckert et al., 2000;Hermosilloet al., 2001). Someauthorshave
5
further improvedtherobustnessof theapproachby modifying theoriginal MI measure,either
by including someprior information on the joint intensitydistribution (Maintz et al., 1998;
Likar andPernus,2000),or by usinghigher-orderdefinitionsof MI which incorporatespatial
information(Rueckert et al., 2000).
Ourapproachdescribedherestemsfrom theobservationthatanumberof multimodalrigid
registrationproblemscanbesolvedin practiceusingothersimilarity measuresthanMI, oneof
which is thecorrelationratio(CR)(Rocheetal.,1998).TheCRis muchmoreconstrainedthan
MI asit assumesa functional,thoughnon-linear, relationshipbetweentheimageintensities.In
otherwords,it assumesthatoneimagecouldbemadesimilar to theotherby asimpleintensity
remapping.Thus,theCR methodamountsto anadaptiveestimationstrategy whereoneimage
is alternatelycorrectedin intensityandin geometryto progressively matchtheother.
For mostcombinationsof medicalimages,thefunctionaldependenceassumptionis gener-
ally valid for amajorityof anatomicalstructures,but not for all of them.Althoughthisproblem
doesnot turn out to be critical in a rigid/affine registrationcontext, we observe that it may
seriouslyhampertheestimationof ahigh-dimensionaltransformation.Weproposehereanex-
tensionof thefunctionaldependencemodel,which we call thebifunctionalmodel,to achieve
betterintensitycorrections.While thebifunctionalmodelis morerealisticthanthefunctional
one,it remainsstronglyconstrainedandthusenablesa goodconditioningof the multimodal
non-rigidregistrationproblem.
1.2 Method
The registrationalgorithmdescribedhereis iterative andeachiterationconsistsof two parts.
The first part transformsthe intensitiesof anatomicalstructuresof a sourceimageS so that
they matchtheintensitiesof thecorrespondingstructuresof a target imageT. Thesecondpart
is concernedwith theregistrationof S (after intensitytransformation)with T usinganoptical
6
flow algorithm.
1.2.1 Intensity Transformation
Theintensitycorrectionprocessstartsby definingthesetC of intensitypairsfrom correspond-
ing voxelsof T andS. Hence,thesetC is definedas
C Sx T x ;1 x N (1)
whereN is thenumberof voxelsin theimages.Sx andT
x correspondto theintensityvalue
of thexth voxel of SandT, respectively, whenadoptingthecustomaryconventionof consider-
ing imagesasone-dimensionalarrays.Weshallnow show how to performintensitycorrection
if wecanassumethatasingleintensityvaluein Shaseither1) exactlyonecorrespondinginten-
sity valuein T (monofunctionaldependence)or 2) at leastoneandat mosttwo corresponding
intensityvaluesin T (bifunctionaldependence).
Monofunctional DependenceAssumption Our goal is to characterizethe mappingfrom
voxel intensitiesin S to thosein T, knowing that someelementsof C areerroneous,i.e. that
would not be presentin C if S andT were perfectlymatched. Let us assumeherethat the
intensityin T is a functionof theintensityin Scorruptedwith anadditivestationaryGaussian
whitenoiseη,
Tx f
Sx η
x (2)
where f is an unknown function to be estimated. This is exactly the model employed by
Rocheet al. (Rocheet al., 2000) which leadsto the correlationratio as the measureto be
maximizedfor registration.In thatapproach,for agiventransformation,oneseeksthefunction
thatbestdescribesT in termsof S. In a maximumlikelihoodcontext, this function is actually
a leastsquares(LS) fit of T in termsof S.
7
Themajordifferencebetweenour respective problemsis thatwe seeka high-dimensional
geometricaltransformation.As opposedto affine registrationwherethetransformationis gov-
ernedby the majority of goodmatches,elasticdeformationsmay be computedusingmainly
local information(i.e. gradients,local averages,etc.).Hence,wecannotexpectgooddisplace-
mentsin onestructureto correctfor badonesin another;wehaveto makecertaineachvoxel is
movedproperlyat eachiteration.For this,sincethegeometricaltransformationis foundusing
intensitysimilarity, themostpreciseintensitytransformationis required.Consequently, instead
of performinga standardLS regression,we have optedfor a robust linearregressionestimator
whichwill removeoutlyingelementsof C duringtheestimationof theintensitytransformation.
To estimatef weusetheleasttrimmedsquares(LTS)methodfollowedby abinaryreweighted
leastsquares(RLS) estimation(RousseeuwandLeroy, 1987). The combinationof thesetwo
methodsprovidesaveryrobustregressiontechniquewith outlierdetection,while ensuringthat
amaximumof pertinentvoxel pairsaretakeninto account.
Dif ferenttypesof functionscanbeusedto model f . In (Guimondetal.,2001)wemadeuse
of polynomialfunctions.TheintensitycorrespondencesbetweenT andS is thendefinedas:
Tx θ0 θ1S
x θ2S
x 2 θpS
x p (3)
whereθ θ0 θp needsto be estimatedand p is the degreeof the polynomial function.
This model is adequateto register imagesthat have a vastrangeof intensities;the restricted
polynomialdegreeimposesintensityspaceconstraintson thecorrespondences,mappingsimi-
lar intensitiesin S to similar intensitiesin T.
In the casethat S is a labeledimage,neighboringintensitiesin S will usuallycorrespond
to differentstructures.Hencethe intensityspaceconstraintis no longer required. f is then
modeledasa piecewiseconstantfunction,suchthateachlabelof S is mappedto theLTS/RLS
estimateof intensitiescorrespondingto thatlabelin T.
8
Bifunctional DependenceAssumption Functionaldependenceasexpressedin (2) andin (3)
impliesthattwostructureshaving similarintensityrangesin Sshouldalsohavesimilar intensity
rangesin T. With somecombinationsof images,this is a crudeapproximation.For example,
CSFandbonesgenerallygivesimilarintensityvaluesin T1-weightedimages,while they appear
with very distinct valuesin PD-weightedscans. Conversely, CSF and gray matterare well
contrastedin T1-weightedimages,while they correspondto similar intensitiesin PD weighted
scans.
To circumvent this difficulty, we have developeda strategy thatenablesthemappingof an
intensityvaluein S to not only one,but two possibleintensityvaluesin T. This methodis a
naturalextensionof theprevioussection.Insteadof computinga singlefunctionthatmapsthe
intensitiesof Sto thoseof T, two functionsareestimatedandthemappingbecomesaweighted
sumof thesetwo functions.
We startwith theassumptionthat if a point hasanintensitys in S, thecorrespondingpoint
in T hasanintensityt that is normallydistributedaroundtwo possiblevaluesdependingon s,
f1s and f2
s . In statisticalterms,this meansthat given s, t is drawn from a mixture of
Gaussiandistribution,
Pt s π1
s N
f1s σ2 π2
s N
f2s σ2 (4)
whereπ1
s andπ2
s 1 π1
s aremixing proportionsthatdependon the intensityin the
sourceimage,andσ2 representsthevarianceof thenoisein thetarget image.Consistentwith
the functional relationship,we will restrict ourselves to polynomial intensity functions, i.e.
f1s θ0 θ1s θ2s2 θpsp, and f2
s ψ0 ψ1s ψ2s2 ψpsp.
An intuitive way to interpretthis modelingis to statethat for any voxel, thereis a binary
“selector”variableε 1 2 thatwould tell us,if it wasobserved,which of thetwo functions
f1 or f2 actuallyservesto maps to t. Without knowledgeof ε, thebestintensitycorrectionto
9
applyto S(in theminimumvariancesense)is aweightedsumof thetwo functions,
fs t P
ε 1 s t f1
s P
ε 2 s t f2
s (5)
in which theweightscorrespondto theprobabilitythatthepointbemappedaccordingto either
thefirst or thesecondfunction.To estimatethefunctions,weemploy asequentialstrategy that
performstwo successive LTS/RLSregressionsasin themonofunctionalcase.Detailson how
theotherparametersaredeterminedcanbefoundin (Guimondet al., 2001).
1.2.2 GeometricalTransformation
Having completedthe intensity transformationstage,we endup with an intensitycorrected
versionof the sourceimage,which will be denotedS . In the monofunctionalcaseS x fSx andin thebifunctionalcaseS x f
Sx T x . We mayassumethatS is roughly
of thesamemodality asT in thesensethat correspondinganatomicalstructureshave similar
intensitiesin S and T. The geometricaltransformationproblemmay then be treatedin a
monomodalregistrationcontext.
Many algorithmshave beendevelopedthatdeformonebrain so its shapematchesthatof
another(Maintz andViergever, 1998; Toga,1999). The procedureusedherewasinfluenced
by a varietyof opticalflow methods,primarily thedemonsalgorithm(Thirion, 1995;Thirion,
1998).At agiveniterationn, eachvoxel x of T is displacedaccordingto avectorvn
x soasto
matchits correspondinganatomicallocationin Sn. Weusethefollowing scheme:
vn 1
x Gσ vn Sn hn
x T
x!
∇Sn hn
x ! 2 Sn hn
x" T
x 2∇Sn hn
x#$ (6)
whereGσ is a 3D Gaussianfilter with isotropic varianceσ2, denotesthe convolution, denotesthecomposition,∇ is thegradientoperatorandthe transformationhn
x is relatedto
10
thedisplacementby hn
x x vn
x . As is commonwith registrationmethods,wealsomake
useof multilevel techniquesto accelerateconvergence.Detailsaboutthenumberof levelsand
iterationsaswell asfilter implementationissuesareaddressedin Section1.3. We show here
how ourmethodcanberelatedto threeotherregistrationmethods:theminimizationof thesum
of squareddifference(SSD)criterion;opticalflow; and,thedemonsalgorithm.
1.2.3 Relation to SSDMinimization
In theSSDminimizationframework, onesearchesfor thetransformationh thatminimizesthe
sumof squareddifferencesbetweenthe transformedsourceimageandthe target image. The
SSDis thendefinedas:
SSDh 1
2
N
∑x% 1
S hx T
x 2 (7)
Theminimizationof (7) maybeperformedusinga gradientdescentalgorithm. By differ-
entiatingtheabove equation,we get for a givenx: ∇SSDh S h
x& T
x ∇S h
x .
Thus,thegradientdescentconsistsof aniterativeschemeof theform:
vn 1 vn α Sn hn
x" T
x ∇Sn hn
x (8)
whereα is thesteplength. If we setα to a constantvalue,this methodcorrespondsto a first
ordergradientdescentalgorithm.Comparing(8) to (6), weseethatourmethodsets
α 1!∇S hn x ! 2 T
x" S hn
x 2 (9)
andappliesa Gaussianfilter to provide a smoothdisplacementfield. Cachieret al. (Cachier
et al., 1999;Pennecet al., 1999)have shown that using(9) closelyrelates(6) with a second
ordergradientdescentof theSSDcriterion,in whicheachiterationn setshn 1 to theminimum
of the SSD quadraticapproximationat hn. We refer the readerto thesearticlesfor a more
11
technicaldiscussionon this subject.
1.2.4 Relation to Optical Flow
T andSareconsideredassuccessivetimesamplesof animagesequencerepresentedby Ix t ,
wherex x1 x2 x3 is a voxel position in the imageand t is time. The displacementsare
computedby constrainingthebrightnessof brainstructuresto beconstantin timesothat
dIx t
dt 0 (10)
It is well known that(10) is not sufficient to provide a uniquedisplacementfor eachvoxel. In
fact,this constraintleadsto
fx ∂I
x t ' ∂t!
∇xIx t ! 2∇xI
x t (11)
which is the componentof thedisplacementin thedirectionof the brightnessgradient(Horn
andSchunck,1981).
Otherconstraintsneedto beaddedto (10) to obtainthedisplacementcomponentsin other
directions. Many methodshave beenproposedto fulfill this purposeandthusregularizethe
resultingvector field (Barron et al., 1994). One that can be computedvery efficiently was
proposedby Thirion (Thirion,1998)in hisdescriptionof thedemonsregistrationmethod,using
a completegrid of demons.It consistsof smoothingeachdimensionof the vectorfield with
a Gaussianfilter Gσ. He also proposedto add ∂Ix t ' ∂t 2 to the denominatorof (11) for
numericalstability when∇xIx t is closeto zero,a term which servesthe samepurposeas
α2 in the original optical flow formulationof Horn andSchunck(Horn andSchunck,1981).
As is presentedby Bro-NielsenandGramkow (Bro-NielsenandGramkow, 1996),this kind of
regularizationapproximatesa linearelasticitytransformationmodel.
12
With this in mind, thedisplacementthatmapsa voxel positionin T to its positionin S is
foundusinganiterativemethod,
vn 1
x Gσ vn ∂I
x t ' ∂t!
∇xIx t ! 2 ∂I
x t ' ∂t 2∇xI
x t # (12)
Spatialderivativesmay be computedin several ways (Horn andSchunck,1981; Brandt,
1997; Simoncelli,1994). We have observed from practicalexperiencethat our methodper-
formsbestwhenthey arecomputedfrom theresampledsourceimageof thecurrentiteration.
As shown in Section1.2.3,this is in agreementwith theSSDminimization. Temporalderiva-
tivesare obtainedby subtractingthe target imagesfrom the resampledsourceimageof the
currentiteration.Theseconsiderationsrelate(12) to (6). Thereadershouldnotethatthemajor
differencebetweenthis methodandotheroptical flow strategiesis that regularizationis per-
formedafter thecalculationof thedisplacementsin thegradientdirectioninsteadof usingan
explicit regularizationtermin aminimizationframework.
1.2.5 Relation to the DemonsAlgorithm
Our algorithm is actually a small variation of the demonsmethod(Thirion, 1995; Thirion,
1998)usingacompletegrid of demons,itself closelyrelatedto opticalflow asdescribedin the
previoussection.Thedemonsalgorithmfindsthedisplacementsusingthefollowing formula:
vn 1
x Gσ vn S hn
x T
x!
∇Tx ! 2 S hn
x" T
x 2∇T
x # (13)
In comparing(13) and(6), it is apparentthat theonly differencebetweenour formulationand
thedemonmethodis thatderivativesarecomputedontheresampledsourceimageof thecurrent
iteration. This modificationwasperformedfollowing theobservationson theminimizationof
theSSDcriterion.
13
1.3 Resultsand Discussion
In the following sectionwe presentregistrationresultsinvolving imagesobtainedfrom mul-
tiple modalities.First, we show a typical examplewheremonofunctionaldependencecanbe
assumed:theregistrationof anatlas(Collinset al., 1998b)with aT1-weightedMR image.We
next presentanexamplewherebifunctionaldependencemaybeassumed:theregistrationof a
PD-weightedimagewith thesameT1-weightedimage.
All of the imagesusedin this sectionhave a resolutionof 1 ( 1 ( 1mm3 andrespectthe
neurologicalconvention,i.e. oncoronalslices,thepatient’s left is on theleft sideof theimage.
Before registration,imagesareaffinely registeredusing the correlationratio method(Roche
et al., 1998).
The multilevel processwasperformedat threeresolutionlevels, namely4mm, 2mm and
1mm per voxel. Displacementfields at one level are initialized from the result of the pre-
vious level. The initial displacementfield v0 is set to zero. 128 iterationsareperformedat
4mm/voxel, 32 at 2mm/voxel and8 at 1mm/voxel. Thesearetwice the numberof iterations
usedfor registrationof monomodalimagesusingtheconventionaldemonsalgorithm. We be-
lieve that makinguseof a betterstoppingcriterion, suchasthe differenceof the SSDvalues
betweeniterations,would probablyimprove theresultsshown below. This aspectis presently
underinvestigation.TheGaussianfilter Gσ usedto smooththedisplacementfield hasa stan-
darddeviation of 1 voxel regardlessof theresolution.This modelsstrongerconstraintson the
deformationfield at thebeginningof theregistrationprocessto correctfor grossdisplacements,
andweaker constraintsneartheendwhenfine displacementsaresought.Theresamplingpro-
cessmakesuseof trilinear interpolation,exceptin thecaseof theatlaswherenearest-neighbor
interpolationis used.
Computationtime to obtainthe following resultsis around60 minuteson a 450MHz PC
with 500MB of RAM (10 minutesat 4mm,20minutesat 2mmand30 minutesat 1mm).Most
14
of this time ( ) 85%)is devotedto theintensitycorrectionpart,which hasnot beenoptimized
in this first versionof our program.Theother15%is takenby thestandardregistrationcode
which is stableandwell optimized.
1.3.1 Monofunctional Dependence
We presentheretheresultof registeringtheatlaswith a T1-weightedimage.This is a typical
exampleof monofunctionaldependencebetweentheintensitiesof theimagesto register:since
theatlascanbeusedto generaterealisticMR images,it is safeto assumea functionaldepen-
dencebetweenthe intensityof theatlasandthoseof theT1-weightedimage. Also, sincethe
sourceimageS is a labeledimage,thefunction f is modeledasa piecewiseconstantfunction.
In this case,eachintensitylevel (10 in all) correspondsto a region from which to estimatethe
constantfunction.
The resultof registrationis presentedin Figure1. The first image(Figure1a) shows one
slice of the atlas. The secondone(Figure1b) is the correspondingslice of the T1-weighted
image. The third andfourth images(Figures1c and1d) presentthe resultof registeringthe
atlaswith theT1-weightedimageusingour algorithm. Figure1c shows theresultwithout the
intensity transformation;we have simply appliedto the atlasthe geometricaltransformation
resultingfrom the registrationprocedure.Figure1d shows the imageresultingfrom the reg-
istrationprocess.It hasthesameshapeastheT1-weightedimage(Figure1b) andintensities
havebeentransformedusingtheintensitycorrection.To facilitatethevisualassessmentof reg-
istrationaccuracy, contoursobtainedusingaCanny-Dericheedgedetector(ontheT1-weighted
image)havebeenoverlaidovereachimagein Figure1.
Figure1eshowsthejoint histogramof intensitiesafterregistration.Valueshavebeencom-
pressedlogarithmically and normalizedas is depictedin the color scale. The histogramis
color-codedandrangesfrom darkredrepresentinghigh point densitiesto light yellow depict-
15
ing low point densities.Thevaluesof thepiecewiseconstantfunction f areoverlaidaswhite
dots.
1.3.2 Bifunctional Dependence
Whenregisteringimagesfrom differentmodalities,monofunctionaldependencemaynot nec-
essarilybe assumed.We presentedin Section1.2.1suchan example: the registrationof PD
andT1-weightedimages.The mainproblemin this caseis that theCSF/GMintensityof the
PD imageneedsto bemappedto two differentintensitiesin theT1-weightedscan.
To solve this problem,we appliedthe methoddescribedin Section1.2.1 to register PD
andT1-weightedimageswheretwo polynomial functionsof degree12 areestimated.This
polynomialdegreewassetarbitrarily to a relatively high valueto allow significantintensity
transformations.
As shown in Figure1f-j, the CSF, which is white in the PD-weightedimage(Figure1f)
andblack in theT1-weightedimage(Figure1g), is well registered.Also, the intensitytrans-
formationis adequate(Figure1i). Thesamecommentsapplyto theGM, which is white in the
PD-weightedimage(Figure1f) andgrayin theT1-weightedimage(Figure1g).
Figure1j presentsthejoint histogramof thetwo imagesafterregistration.Thefunctions f1
and f2 foundduringtheregistrationprocessaresuperimposed,theblueline correspondsto f1
andthegreenoneto f2. Theline width for agivenintensitys is proportionalto thevalueof the
correspondingπε
s .
As canbeobservedin Figure1j, thepolynomialfunctions f1 and f2 fit well with thehigh
densityclustersof thejoint histogram.In particular, we seethat theCSF/GMintensityvalues
from the PD-weightedimage(with valuesaround220) get mappedto two different intensity
valuesin theT1-weightedscan:75and45. Themappingto 75 representstheGM (redpolyno-
mial) while themappingto 45 representsCSF(bluepolynomial).
16
Note that in theregistrationof theatlaswith theT1-weightedimageandthePD- with the
T1- weightedimage,we selectedas the sourceimagethe one which hasthe bestconstrast
betweenstructures.This is simplybecauseouralgorithmpermitsmany structuresof thesource
imageto be mappedto a single intensity. But a single intensity in the sourceimagecanbe
mappedto at most two intensitiesin the target image. Hence,it is alwaysbetterto usethe
imagewith thegreaternumberof visiblestructuresasthesourceimage.
1.4 Conclusion
Wehavepresentedanoriginalmethodto performnon-rigidregistrationof multimodalimages.
This iterativealgorithmis composedof two steps:theintensitytransformationandthegeomet-
rical transformation.Two intensitytransformationmodelsweredescribedwhichassumeeither
monofunctionalor bifunctionaldependencebetweenthe intensityvaluesin the imagesbeing
matched.Both of thesemodelsarebuilt usingrobustestimatorsto enablepreciseandaccurate
transformationsolutions.
17
0
0.5
1
50
100
150
200
250
Bac
kgro
und
CS
FG
ray
Mat
ter
Whi
te M
atte
rFa
t
Mus
cle
/ Ski
nS
kin
Sku
llG
lial M
atte
rC
onne
ctiv
e
0
0.5
1
0 75 150 225
50
100
150
200
250
T1
Inte
nsiti
esAtlas
PD Intensities
T1
Inte
nsiti
es
a
c
g
b
d
i
j
f
h
e
Figure1: Resultsof 3D registration.Registrationof Atlas with T1-weightedimage: (a)Atlas(Source).(b)T1-weightedimage(Target).(c) Atlaswithout intensitycorrection,afterregistrationwith T1. (d) Atlaswith intensitycorrection,afterregistrationwith T1. (e)Thejoint histogramof theatlasandT1-weightedimageafterregistration;valuesrangefrom dark red representinghigh point densitiesto light yellow depictinglow point densities;thewhite dotscorrespondto the intensitytransformationfoundby registeringtheatlaswith theT1-weightedimageand assumingmonofunctionaldependence(piecewise constantfunction). Registration of PD-weighted withT1-weightedimage: (f) PD-weightedimage(Source).(g) T1-weightedimage(Target). (h) PD-weightedimagewithout intensity correction,after registrationwith T1-weightedimage. (i) PD-weightedimagewith intensitycorrectionafter registrationwith T1-weightedimage. (j) The joint histogramof PD-weightedimageand T1-weightedimagem,after registration;theblue line correspondsto f1 andthegreenoneto f2; the line width for agivenintensityvalues in T1 correspondsto thevalueof thecorrespondingπε * s+ . ContourswereobtainedusingaCanny-Dericheedgedetectoron thetargets(b andg) andoverlaidon theotherimagesto betterassessthequalityof registration.Thejoint histogramsvalueshave beencompressedlogarithmicallyandnormalizedasis depictedin thecolorscale. 18
2 Image Fusion During Neurosurgery with a Biomechanical
Model of Brain Deformation
Intr oduction
A critical goalof neurosurgery is to accuratelylocate,accessandremove intracraniallesions
withoutdamaginghealthybraintissue.Theoverridinggoalis to preserveneurologicalfunction.
This requirestheprecisedelineationof thefunctionalanatomyandmorphologyof thepatient’s
brain,aswell aslesionmargins. Thesimilar visualappearanceof healthyanddiseasedbrain
tissue(e.g.aswith infiltrating tumors)andtheinability of thesurgeonto seecritical structures
beneaththe brain surfacecanposedifficultiesduring the operation.Somecritical structures,
suchaswhitematterfibertracts,maynotbevisibleatall. Moreover, thedifficulty in perceiving
lesion(e.g.tumor)boundariesmakescompleteresectionextremelydifficult (Jolesz,1997).
Over thelastdecade,advancesin image-guidedneurosurgery(IGNS) techniqueshavecon-
tributedto thegrowth of minimally-invasive neurosurgery. Theseproceduresmustbecarried
out in operatingroomswhicharespecially-equippedwith imagingsystems.Thesesystemsare
usedto acquireimagesintraoperatively, asnecessitatedby the procedure.The improved vi-
sualizationof deepstructuresandtheimprovedcontrastbetweenthelesionandhealthytissue
(dependingon themodality)allow thesurgeonto planandexecutetheprocedurewith greater
precision.
IGNS haslargely beena visualization-driventask. In thepast,it hadnot beenpossibleto
makequantitativeassessmentsof intraoperative imagingdata,andinsteadphysiciansreliedon
qualitative judgments.In orderto createa rich visualizationenvironmentwhichmaximizesthe
informationavailableto thesurgeon,previouswork hasbeenconcernedwith imageacquisition,
registrationanddisplay. Algorithm developmentfor computer-aidedIGNS hasfocussedon
improving imagingquality andspeed.Anothermajor focushasbeento developsophisticated
19
multimodality imageregistrationandfusion techniques,to enablefusion of preoperative and
intraoperative images.However, clinical experiencewith IGNSinvolving deepbrainstructures
hasrevealedthelimitationsof existing rigid registrationapproaches.Thismotivatesthesearch
for nonrigid techniquesthatcanrapidly andfaithfully capturethemorphologicalchangesthat
occurduringsurgery. In thefuture,theuseof computer-aidedsurgicalplanningwill includenot
only threedimensional(3D), modelsbut alsoinformationfrom multiple imagingmodalities,
registeredinto thepatient’sreferenceframe.Intraoperativeimagingandnavigationwill thusbe
fully integrated.
Variousimagingmodalitieshavebeenusedfor imageguidance.Theseinclude,amongoth-
ers,digital subtractionangiography(DSA), computedtomography(CT), ultrasound(US),and
intraoperative magneticresonanceimaging (IMRI). IMRI representsa significantadvantage
over othermodalitiesbecauseof its high spatialresolutionandsuperiorsoft tissuecontrast.
However, even the most advancedintraoperative imaging systemscannotprovide the same
imageresolutionor tissuecontrastselectionfeaturesaspreoperative imagingsystems.More-
over, intraoperative imagingsystemsareby necessitylimited in theamountof time available
for imaging. Multimodality registrationcanallow preoperative datathat cannotbe acquired
intraoperatively [suchasnuclearmedicinescans(SPECT/PET),functionalMRI (fMRI), MR
angiography(MRA), etc.] to bevisualizedalongsideintraoperativedata.
2.1 Nonrigid Registration for IGNS
During neurosurgical operations,changesoccurin theanatomicalpositionof brainstructures
andadjacentlesions.The leakageof cerebrospinalfluid (CSF)afteropeningthedura,hyper-
ventilation,theadministrationof anestheticandosmoticagents,andretractionandresectionof
tissueall contribute to shifting of thebrainparenchyma.This makesinformationbasedupon
preoperatively acquiredimagesunreliable. The lossof correspondencebetweenpre- andin-
20
traoperative imagesincreasessubstantiallyasthe procedurecontinues.Thesechangesin the
shapeof the brainhave beenwidely recognizedasnonrigiddeformationscalled“brain shift”
(see(Nabavi et al., 2001)).
Suitableapproachesto capturetheseshapechangesandto createintegratedvisualizationsof
preoperativedatain theconfigurationof thedeformedbrainarecurrentlyin activedevelopment.
Previouswork aimedatcapturingbraindeformationsfor neurosurgerycanbegroupedinto two
categories. In the first categorey arethoseapproachesthat usesomeform of biomechanical
model (recentexamplesinclude (Hagemannet al., 1999; Skrinjar andDuncan,1999; Miga
et al., 1999; Skrinjar et al., 2001; Ferrantet al., 2000b)). In the secondcategory are those
approachesthat usephenomenologicalmethods,relying upon imagerelatedcriteria (recent
examplesinclude(Hill et al., 1998;Hata,1998;Ferrantet al., 1999b;Hataet al., 2000)).
Purelyimage-basedmatchingmaybe ableto achieve a visually pleasingalignment,once
issuesof noiseandintensityartifactareaccountedfor. However, in our work on intraoperative
matchingwe favor physics-basedmodelswhich ultimately may be expandedto incorporate
importantmaterialproperties(suchasinhomogeneity, anisotropy) of thebrain,oncetheseare
determined.However, evenamongphysics-basedmodels,thereexist aspectrumof approaches,
usuallyinvolving a trade-off betweenphysicalplausibilityandspeed.
A fastsurgery simulationmethodhasbeendescribedin (Bro-Nielsen,1996). Here,high
computationalspeedswere obtainedby converting a volumetric finite elementmodel into a
modelwith only surfacenodes.Thegoalof thiswork wasto achieveveryhighgraphicsspeeds
consistentwith interactive computationanddisplay. This is achievedat thecostof simulation
accuracy. This typeof modelis bestsuitedto computergraphics-orienteddisplay, wherehigh
frameratesareneeded.
A sophisticatedfinite elementbasedbiomechanicalmodel for two-dimensionalbrain de-
formationsimulationhasbeenproposedin (Hagemannet al., 1999). In this work, correspon-
21
denceswereestablishedby manualinteraction.Theelementsof thefinite elementmodelwere
the pixels of the two dimensionalimage. The manualdeterminationof correspondencescan
betimeconsuming,andis subjectto humanerror. Moreover, whenmethodsaregeneralizedto
threedimensions,thenumberof pointswhich mustbeidentifiedcanbevery large. Dueto the
realitiesof clinical practice,two-dimensionalresultsarenot practical. A (threedimensional)
voxelwisediscretizationapproach,while theoreticallypossible,is extremelyexpensive from a
computationalstandpoint(evenconsideringa parallelimplementation)becausethenumberof
voxels in a typical intraoperative MRI datasetleadingto a largenumberof equationsto solve
(256x256x60= 3,932,160voxels, which correspondsto 11,796,480displacementsto deter-
mine).Downsamplingcanleadto fewervoxels,but leadsto a lossof detail.
Edwardset al. (Edwardset al., 1997)presenteda two dimensionalthreecomponentmodel
for tracking intraoperative deformation. This work useda simplified materialmodel. How-
ever, the initial 2D multigrid implementationrequired120–180minuteswhenrun on a Sun
MicrosystemsSparc20,whichmaylimit its feasibility for routineuse.
Skrinjar et al. (Skrinjar andDuncan,1999)have presenteda very interestingsystemfor
capturingreal-timeintraoperative brain shift during epilepsysurgery. In this context, brain
shift occursslowly. A very simplifiedhomogeneousbrain tissuematerialmodelwasadopted.
Following the descriptionof surfacebasedtrackingfrom intraoperative MRI driving a linear
elasticbiomechanicalmodel in (Ferrantet al., 2000b),Skrinjar et al. presenteda new imple-
mentation(Skrinjaret al., 2001)of their systemusinga linearelasticmodelandsurfacebased
trackingfrom IMRI with thegoalof eventuallyusingstereoscopiccamerasto obtainintraoper-
ativesurfacedataandhenceto captureintraoperativebraindeformation.
Paulsenet al. (Paulsenet al., 1999)andMiga et al. (Miga et al., 1999;Miga et al., 2001)
havedevelopedasophisticatedfinite elementmodelto simulatebraindeformation.Theirmodel
is interestingbecauseit incorporatessimulationsof forcesassociatedwith tumortissue,aswell
22
asthoseresultingfrom retractionandresection.A limitation of the existing approachis that
thepreoperativesegmentationandtetrahedralfinite elementmeshgenerationcurrentlyrequire
aroundfive hoursof operatortime. Nevertheless,this approachholdspromisein actuallypre-
dicting braindeformation.
Thereal-timeneedsof surgerydictatethatany algorithmusedfor prospectiveimagematch-
ing must rapidly, reliably andaccuratelycapturenonrigid shapechangesin the brain which
occurduringsurgery. Ourapproachis to constructanunstructuredgrid representingthegeom-
etry of thekey structuresin theimagedataset.This techniqueallowsusto usea finite element
discretizationthatfaithfully modelskey characteristicsin importantregionswhile reducingthe
numberof equationsto solveby usingmeshelementsthatspanmultiplevoxelsin otherregions.
Thealgorithmallowstheprojectionof preoperative imagesontointraoperativeimages,thereby
allowing thefusionof imagesfrom multiplemodalitiesandspanningdifferentcontrastmecha-
nisms.Wehaveusedparallelhardware,parallelalgorithmdesignandefficient implementations
to achieve rapidexecutiontimescompatiblewith neurosurgery.
2.2 Method
Figure2 is anoverview, illustratingtheimageanalysisstepsusedduring intraoperative image
registration. The critical imageprocessingtasksinclude segmentation(the identificationof
anatomicalstructures)andregistration.Segmentationdataareusedbothfor preoperativeplan-
ning,andto createintraoperativesegmentations.Thesegmentationdataareusedto calculatean
initial affine transformation(rotation,translation,scaling)which rigidly registersthe images,
thusinitializing thedatafor nonrigidmatchingusingour biomechanicalsimulation.Usingthe
biomechanicalmodel,thevolumetricdeformationis inferredthrougha mechanicalsimulation
with boundaryconditionsestablishedvia surfacematching. This sophisticateddeformation
modelcanbesolvedduringneurosurgery, providing enhancedintraoperativevisualization.
23
Image Analysis During Image Guided Neurosurgery:
Tissue Segmentation
InitialRegistration
BiomechanicalSimulation
Preoperative data
Intraoperative MRI
SurfaceMatching
Visualization
Neurosurgeryprogresses
Figure2: Schematicillustratingimageanalysistaskscarriedoutduringneurosurgery.
2.2.1 Preoperative Data Acquisition and Processing
Thetime availablefor imageprocessingduringsurgery is extremelylimited comparedto that
availablepreoperatively. Consequently, preoperativedataacquisitioncanbemorecomprehen-
sive,andmoreextensiveimageanalysis(for examplesegmentation)canbeperformed.
A variety of manual(Gering et al., 1999), semi-automated(Kikinis et al., 1992; Yezzi
et al., 2000)andautomated(Warfield et al., 2000a;Kauset al., 1999;Warfieldet al., 2000b)
segmentationapproachesareavailable. We selectthe mostaccurate,robust approachbased
uponthepreoperative dataandtheparticularcritical structures.For thematchingexperiments
which will bedescribedbelow, we have usedananatomicalatlas,althoughotherpreoperative
datasuchasmagneticresonanceangiographyor diffusiontensorimagescouldultimatelyalso
be used. The atlaswasconstructedfrom a high resolutionscanof a singlepatient,in which
over 200 structuresweresegmented(Kikinis et al., 1996)usinga combinationof automated
and interactive techniques.During surgery, we are especiallyinterestedin the corticospinal
tract, a region of white matterwhich canbe difficult or impossibleto directly observe with
24
conventionalMRI, andwhichmustbepreserved.Wehavepreviouslyshownthatwecanproject
thecorticospinaltractfrom theatlasontopatientscansfor preoperativesurgicalplanning(Kaus
et al., 2000).
2.2.2 Intraoperati ve ImageProcessing
Intraoperative imageprocessingconsistsof: 1.) acquiringoneor moreintraoperativevolumet-
ric datasets;2.) constructinga segmentationof the intraoperative acquisition;3.) computing
anaffine registrationof thepreoperativedataontothenew acquisition;4.) identifying thecor-
respondencesbetweenkey surfacesof the preoperative andintraoperative data;5.) solving a
biomechanicalmodelto infer a volumetricdeformationfield; 6.) applyingthedeformationto
thepreoperativedataandconstructinga new visualizationmerging critical structuresfrom the
preoperativedatawith theintraoperativedata.
Segmentationof Intraoperati ve Volumetric Images
In theexperimentsconductedbelow, a rapidsegmentationof thebrainandventricleswasob-
tainedusing a binary curvaturedriven evolution algorithm (Yezzi et al., 2000). The region
identifiedasbrain or ventriclewastheninteractively correctedto remove misclassifiedtissue
usingthesoftwaredescribedby Geringet al. (Geringet al., 2001). This approachallows the
surgeonto inspectandinteractively edit thesegmentationdata,increasingits accuracy.
Alternatively, wehaveexperimentedwith automatedintraoperativesegmentation(Warfield
et al., 1998b;Warfield et al., 2000a)utilizing tissueclassificationin a multi-channelfeature
spaceusinga modelof expectedanatomyasan initial template.Automatedapproacheswill
likely bepreferableoncethey havebeenvalidated.
25
Unstructured MeshGenerationand SurfaceRepresentation
Wehave implementedameshgeneratorwhich is optimizedfor usewith biomedicalstructures,
building uponpreviously describedtechniques(Schroederet al., 1996;Geiger, 1993). During
meshgeneration,weextractanexplicit representationof thesurfaceof thebrainandventricles
basedon thepreoperative segmentation.We alsocreatea volumetricunstructuredmeshusing
a multiresolutionversionof themarchingtetrahedraalgorithm. Themesherbeginsby subdi-
viding theimageinto cubes,which arethendividedinto 5 tetrahedrausinganalternatingsplit
patternwhich preventsdiagonalcrossingson thesharedfaces.Themeshis iteratively refined
in theregion of complex boundaries,andthena marchingtetrahedra-like approachis applied
to this multiresolutionmesh.For eachcell of thefinal mesh,the labelvalueof eachvertex is
checked,andif different,thetetrahedronis dividedalongtheedgehaving differentnodelabels.
A detaileddescriptioncanbefoundin (Ferrantet al., 1999b;Ferrantet al., 2000a).
The meshingprocessis extremely robust, allowing us to generatetetrahedralmeshesof
thebrainandventriclesfrom rapidsegmentationsof eachvolumetricintraoperativeacquisition
carriedout during the surgery. This facilitatesintraoperative matchingfrom onevolumetric
acquisitionto thenext.
Affine Registration of Preoperative to Intraoperati ve Image Datasets
For affine registration(rotation, translation,scaling),we usea fast parallel implementation
of a robust algorithm which is baseduponaligning segmentedimagedata,usinga rapidly-
converging multiresolutionsearchstrategy (Warfield et al., 1998a). Applying the resulting
transform,segmentationsandgreyscaledatafrom thepreoperativeandintraoperativescansare
rigidly registered.
26
Volumetric BiomechanicalSimulation of Brain Deformation
During the procedure,the brain undergoesnonrigid shapechangesfor the reasonsdescribed
above. During IGNS thesurgeonis ableto acquirea new volumetricMRI whenhewishesto
review the currentconfigurationof the entirebrain. A volumetricdeformationfield relating
earlieracquisitionsto this new scanis computedby first matchingsurfacesfrom the earlier
acquisitionto the currentacquisition,and then calculatingthe volumetric displacementsby
using the surfacedisplacementsas boundaryconditions. The critical conceptis that forces
areappliedto thevolumetricmodelthatwill producethesamesurfacedisplacementsaswere
obtainedby the surfacematching. The biomechanicalmodelcanthenbe usedto computea
volumetricdeformationmap.
Establishing SurfaceCorr espondences Thesurfacesof thebrainandlateralventriclesare
iteratively deformedusinga dual active surfacealgorithm. Image-derived forcesareapplied
iteratively to anelasticmembranesurfacemodelof theearlyscan,therebydeformingit soasto
matchtheboundaryof thecurrentacquisition.Thederivedforcesarea decreasingfunctionof
theimageintensitygradients,soasto beminimizedat theedgesof objectsin thevolume.We
have includedprior knowledgeabouttheexpectedgraylevel andgradientsof theobjectsbeing
matchedto increasethe convergencerateof theprocess.This algorithmis fully describedin
(Ferrantet al., 1999a).
BiomechanicalSimulation of Volumetric Brain Deformation Wetreatthebrainasahomo-
geneouslinearlyelasticmaterial.Thedeformationenergy of anelasticbodyΩ, underno initial
stressesor strains,andsubjectto externallyappliedforces,canbedescribedby the following
model(Zienkiewicz andTaylor, 1994):
Eu 1
2 , Ωσ - ε dΩ , Ω
u - F dΩ (14)
27
wherethe variablesaregiven in termsof the stressvector, σ, the strainvector, ε, the forces
F Fx y z appliedto theelasticbody(forcesperunit volume,surfaceforcesor forcescon-
centratedat the nodesof the mesh)andu ux y z v x y z w x y z - , the displacement
vectorfield we wish to compute.Sincewe areusinga linearelasticityframework, we assume
smalldeformations.Hencethestrainvectorε is givenby
ε ∂u∂x
∂v∂y
∂w∂z
∂u∂y
∂v∂x
∂v∂z
∂w∂y
∂w∂x
∂u∂z# - (15)
which canbe written as ε L u whereL is a linear operator. The elastomechanicalrelation
betweenstressesandstrainscanbeexpressedby thegeneralizedHooke’s law as
σ σx σy σz τxy τyz τzx - D ε (16)
Assumingisotropicmaterialpropertiesfor eachpoint,we obtaina symmetricelasticitymatrix
D in theform
D E1 ν 1 2ν
.//////////////01 ν ν ν 0 0 0
ν 1 ν ν 0 0 0
ν ν 1 ν 0 0 0
0 0 0 1 1 2ν2 0 0
0 0 0 0 1 1 2ν2 0
0 0 0 0 0 1 1 2ν2
24333333333333335(17)
with physicalparametersE (Young’s modulus)andν (Poisson’s ratio). See(Zienkiewicz and
Taylor, 1994)for thefull details.
For thediscretization,we usethefinite elementmethodappliedover thevolumetricimage
domainsothatthetotal potentialenergy canbewritten asa sumof potentialenergiesfor each
28
element:Eu ∑Nnodes
e% 1 Eeue . Themeshis composedof tetrahedralelementsandthuseach
elementis definedby four meshnodes.Thecontinuousdisplacementfield u everywherewithin
elemente of the meshis definedasa function of the displacementat the element’s nodesuei
weightedby theelement’s interpolatingfunctionsNei
x ,
ux Nnodes
∑i % 1
I Nei
x ue
i (18)
Linearinterpolatingfunctionsareusedto definethedisplacementfield insideeachelement.
Theinterpolatingfunctionof nodei of tetrahedralelemente is definedas
Nei
x 1
6Ve 6 aei be
i x cei y de
i z7 (19)
Thecomputationof thevolumeof theelementVe andtheinterpolationcoefficientsaredetailed
in (Zienkiewicz andTaylor, 1994,pages91–92).
The volumetric deformationof the brain is found by solving for the displacementfield
that minimizesthe deformationenergy describedby Equation(14). For our finitite element
approachthis is describedby
δEu M
∑e% 1
δEeue 0 (20)
where
δEeue Nnodes
∑i % 1
∂∂ue
iEe
ue δui
e Nnodes
∑i % 1
∂∂ve
iEe
ue δvi
e Nnodes
∑i % 1
∂∂we
iEe
ue δwi
e (21)
Sinceδuie δvi
e andδwie areindependent,definingmatrix Be
Bei Nnodes
i % 1 with Bei LNe
i for
29
everynodei of eachelemente, yieldsin thefollowing equation:
0 , ΩBe - DBeuedΩ , Ω
Ne - FedΩ (22)
with the elementstiffnessmatrix Ke 98Ω Be - DBe dΩ. An assemblyof the equationsfor
all elementsfinally leadsto a global linear systemof equations,which canbe solved for the
displacementsresultingfrom theforcesappliedto thebody:
K u F (23)
The displacementsat the boundarysurfacenodesare fixed to matchthosegeneratedby
theactive surfacemodel. Let :u be thevectorrepresentingthe displacementto be imposedat
theboundarynodes.Theelementsof the rows of thestiffnessmatrix K correspondingto the
nodesfor which a displacementis to be imposedaresetto zeroandthediagonalelementsof
theserows areset to one. The force vectorF is set to equalthe displacementvector for the
boundarynodes:F :u (Zienkiewicz andTaylor, 1994). In this way solvingEquation(23) for
theunknown displacementswill producea deformationfield over theentirevolumetricmesh
thatmatchestheprescribeddisplacementsat theboundarysurfaces.
Hardwareand Implementation
Thevolumetricdeformationof thebrainis computedby solvingfor thedisplacementfield that
minimizestheenergy describedby Equation(14),afterfixing thedisplacementsat thesurface
to matchthosegeneratedby theactivesurfacemodel.
Threevariables,representingthe x, y andz displacements,mustbe determinedfor each
elementof thefinite elementmesh.Eachvariablegivesriseto onerow andonecolumnin the
global K matrix. The rows of thematrix aredividedequallyamongsttheCPUsavailablefor
30
computationandtheglobalmatrix is assembledin parallel.EachCPUassemblesthelocal Ke
matrix for eachelementin its subdomain.AlthougheachCPUhasanequalnumberof rows to
process,becausetheconnectivity of themeshis irregular, someCPUsmaydo morework than
others.
Following matrix assembly, the boundaryconditionsdeterminedby the surfacematching
areapplied.TheglobalK matrix is adjustedsuchthat rows associatedwith variablesthatare
determinedconsistof asinglenon-zeroentryof unit magnitudeon thediagonal.
The volumetric biomechanicalbrain model systemof equations(and the active surface
membranemodel equations)are solved using the Portable,ExtensibleToolkit for Scientific
Computation(PETSc)package(Balayet al., 1997;Balayet al., 2000a)usingtheGeneralized
Minimal Residual(GMRES)solver with block Jacobipreconditioning.During neurosurgery,
the systemof equationswassolved on a SunMicrosystemsSunFire6800symmetricmulti-
processormachinewith 12750MHzUltraSPARC-III (8MB Ecache)CPUsand12GB of RAM.
This architecturegivesussufficient computecapacityto executetheintraoperative imagepro-
cessingprospectively duringneurosurgery.
Intraoperati ve Visualization
Oncethevolumetricdeformationfield hasbeencomputed,it canbeappliedto earlierdatato
warp it into the currentconfigurationof the patientanatomy. The imagingdatacanthenbe
displayedby texture mappingonto flat planesto facilitatecomparisonswith currentintraop-
erative dataaswell asprior scans. Triangle modelsof segmentedstructures(i.e. basedon
registeredvolumetric data)can be usedto display surfacerenderingsof critical anatomical
structures,overlaid on intraoperative imagedata. This allows readyappreciationof the 3D
anatomyof thesesegmentedstructurestogetherwith the imagingdatain the form of planes
passingthroughor over the3D trianglemodels(Geringet al., 2001). This augmentsthesur-
31
Figure3: Open-configurationmagneticresonancescannerduringneurosurgery.
geon’sability to seecritical structureswhichmustbepreserved(suchasthecorticospinaltract)
andto betterappreciatethelesionandits relationshipto otherstructures.
Figure3 shows theopen-configurationmagneticresonancescanneroptimizedfor imaging
duringsurgical procedures(Jolesz,1997;Black et al., 1997). The imagewe constructedwas
presentedon theLCD andincreasedthe informationavailableto thesurgeonastheoperation
progressed.
2.3 Resultsand Discussion
The imageregistrationstrategy describedherehasbeenappliedprospectively during several
neurosurgical cases.We presenthereillustrative resultswhich demonstratethe ability of our
algorithmto captureintraoperativebraindeformations.
Theenhancementprovidedby intraoperativenonrigidregistrationto thesurgical visualiza-
tion environmentis shown by ourmatchingthecorticospinaltractof apreoperatively prepared
anatomicalatlasto the initial andsubsequentintraoperative scansof a subject.This matching
32
Figure4: This figureshows thecorticospinaltractfrom our anatomicalatlasin blue,projectedinto theshapeof thebrainof thesubjectshown in Figure3.
wascarriedoutprospectively duringtheneurosurgery, demonstratingthepracticalvalueof the
approachandits ability to meetthe real-timeconstraintsof surgery. We have alsoconducted
parallelscalingexperimentswhich have yieldedvery encouragingresults. The entire image
analysisprocesscanbe completedin lessthan10 minutes,which hasbeenadequateto dis-
play theinformationto thesurgeon.Interestingly, themostcomputationallyintensive task(the
biomechanicalsimulation)hasalsobeenoptimizedthe mostandis now the fasteststep. We
anticipatethatsegmentationtechniquesrequiringlessuserinteractionwill resultin significant
improvementsin speed.
BiomechanicalSimulation of Volumetric Brain Deformation
Figure4 shows thecorticospinaltractfrom ouranatomicalatlasprojectedinto theshapeof the
brainof thesubject.Thisvisualizationhelpsthesurgeonto betterappreciatethe3D relationship
of this essentialstructureto the lesionandotherregionsof the brain. The corticospinaltract
cannotbereadilyobservedin IMRI acquisitions.
33
(a) A single slice from anearly3D IMRI scan.
(b) Thecorrespondingsliceina later 3D IMRI scan,show-ing significantbrain shift hasoccurred.
(c) The matchedslice of thefirst volume after simulationof thebraindeformation.
(d) Visualizationof the mag-nitude of the deformationfield computedin matchingimage(a) to image(b).
Figure5: Two dimensionalslicesthroughthree-dimensionaldata,showing the matchof thesimulateddeformationof the initial brain onto the actualdeformedbrain. The quality of thematchis significantlybetterthancanbeobtainedthroughrigid registrationalone.
34
Figure 5 is a typical caseillustrating the amountof brain deformationwhich can occur
duringneurosurgery, aswell astheeffectivenessof our algorithmin capturingthis shift during
neurosurgery. As shown, the quality of the matchis significantlybetterthancanbe obtained
throughrigid registrationalone.
Our earlyexperiencehasshown thatour intraoperative biomechanicalsimulationof brain
deformationis a robustandreliablemethodfor capturingthechangesin brainshapethatoccur
during neurosurgery. The registrationalgorithmrequiresno userinteractionandthe parallel
implementationis sufficiently fastto beusedintraoperatively. Weintendto incorporatepatient-
specificpreoperative datain placeof theanatomicalatlasto increasethesurgical valueof the
intraoperativeupdates.
As we refineour approach,we expectto appreciateperformancegainsbasedon moreau-
tomatedsegmentationmethods,andfurtheroptimizedparallelimplementationswhich address
loadimbalances.Improvementsin theaccuracy of thematchcouldresultfrom a moresophis-
ticatedmodelof the materialpropertiesof the brain (suchasmoreaccuratemodelingof the
cerebralfalx andthe lateralventricles).SophisticatedMR imagingmethodssuchasdiffusion
tensorMRI now enablethe preoperative imagingof inhomogeneousanistropicwhite matter
structure,which could be incorporatedinto the materialmodel. Ultimately, the predictionof
braindeformation,asopposedto thecaptureof observeddeformationdescribedhere,will most
likely requirea nonlinearmaterialmodeltogetherwith extensive monitoringof physiological
data.Suchpredictioncouldbeusedto indicatewhennew intraoperative imagingis necessary
to appropriatelyupdateboththesimulationmodelandthesurgeon’sunderstandingof thebrain
shape.
35
2.4 Conclusion
Nonrigid changesin brainmorphologyoccurduringneurosurgeryandlimit theusefulnessof
preoperative imagingfor intra-treatmentplanningandsurgical navigation. Intraoperativenon-
rigid registrationcanaddsignificantlyto the valueof intraoperative imaging. It providesfor
quantitativemonitoringof therapy application,includingtheability to make quantitativecom-
parisonswith a preoperatively-definedtreatmentplanandenablespreoperative imagedatato
bealignedwith thecurrentconfigurationof thebrainof thepatient.We have shown thateven
a relatively complex biomechanicalmodelcanbe initialized andsolvedduring neurosurgery,
providing enhancedsurgical visualization. Ultimately, suchapproachesmay provide a truly
integrated,multimodalityenvironmentfor surgicalnavigationandplanning.
36
3 Physics-BasedRegularization with an Empirical Model of
Anatomical Variability
An importantissuein nonrigidregistrationfor computer-assistedneurologyandneurosurgery
is thegenerationof deformationfieldsthatreflectthetransformationof animagein a realistic
waywith respectto thegivenanatomy. Dueto lackof imagestructure,noise,intensityartifacts,
computationalcomplexity anda restrictedtime frame(e.g. during surgery), it is not feasible
to measuredirectly the deformationoccuringat eachvoxel. This leadsto estimatesof the
deformationfield only at sparselocationswhich have to beinterpolatedthroughouttheimage.
Recently, physics-basedelasticandviscousfluid modelsfor nonrigidregistrationhave be-
comepopular(BajcsyandKovacic,1989),sincethey have thepotentialto constraintheunder-
lying deformationin a plausiblemanner. However viscousfluid models(Lesteret al., 1999;
WangandStaib,2000)have to bechosencarefully, sincethey allow largedeformations.This
is notalwayssuitablefor medicalapplicationsconcerningthebrain.Furthermore,viscousfluid
modelsdrivenby alignmentof similar grayvaluesmayallow anatomicallyincorrectmatches
of differentbut adjacentstructuresthroughthe samemechanismby which large-deformation
matchesarepermitted.For example,onegyrusmayflow from thesourcebrain to matchtwo
or moredifferentgyri in a targetbrain,producingananatomicallyincorrectmatch.
In termsof physics-basedelasticmodels,recentwork has(Davatzikos,1997;Ferrantetal.,
2000b)proposedanactivesurfacealgorithmcomputedat theboundaryof a regardedstructure
as an initial estimateof the deformationfield which was then introducedinto a volumetric
elasticmodelto infer thedeformationinsideandoutsidethesurface.A drawbackof thismethod
is thatalthoughit hasbeenshown to beaccuratecloseto theobject’s boundary, away from the
boundariesthesolutioncouldbelessaccurate.Thework by (WangandStaib,2000)represents
animprovementin thatstatisticalshapeinformation(basedon a setof imageswith manually-
37
identified boundarypoints) was includedas an additionalmatchingcriterion. Even though
suchmethodsarepromisingfor specificbrainstructures,a robust3D shaperepresentationof
thewholebrainstill remainsdifficult to achieve.
In (Collins,1994)anothernonrigidregistrationalgorithmwasproposed,basedon anitera-
tiverefinementof a local similarity measureusingasimplex optimization.In thisapproachthe
deformationfield wasconstrainedonly by smoothingaftercorrespondenceestimation,andthus
canonly beaccuratefor specificregionsof thebrain.To achievebetterresults,themethodwas
improvedby introducingvariousgyri andsulci of the brain asgeometriclandmarks(Collins
et al., 1998a).
In order to obtainrealisticdeformations,we proposeherea physics-basedelasticmodel.
Themethoddoesnot requirea segmentationanddoesnot have thedrawbackthat initial esti-
matesof thedeformationareonly generatedfor theboundaryof aconsideredstructure.Instead,
theseestimatesarecalculatedbasedon a templatematchingapproachwith a local similarity
measure.Furthermorewe have incorporateda modelfor inhomogeneouselasticitiesinto our
algorithm.Thediscretizationof theunderlyingequationis doneby a finite elementtechnique,
which hasbecomea popularmethodfor medicalimagingapplications(e.g. see(Bro-Nielsen,
1998)and(Ferrantet al., 2000b)).
3.1 Method
The processof registrationcanbe describedasan optimizationproblemthat minimizesthe
deformationenergy betweena templateanda referenceimage. Assumingthat both images
representthe samephysicalobject, the deformationthat aligns them is thereforerelatedto
the theoremof minimum potentialenergy. The ideaof our registrationprocesscannow be
describedasfollows: basedonasetof pointsextractedoutof animageasdescribedin (3.2),an
initial sparseestimateof thedeformationfield is foundby a local normalizedcross-correlation
38
(3.3). In a next stepnonrigid registrationis performedusingan elasticmodel(3.4) which is
constrainedby thesparseestimatescomputedin thepreviousstep.
3.2 FeaturePoint extraction
Let Ω denotethedomainof a volumeS: Ω ;=< with voxel positionsx x y z - x > Ω. In
a first stepa setof featurepointsis extractedout of thereferenceimage.For thatpurposewe
calculatethe gradientmagnitudeout of blurredimageintensities. In orderto obtainsuitable
featurepointsfor aninitial sparseestimateof thedeformationfield, only voxel higherthantwo
standarddeviationsabovethemeanof themagnitudeof thegradientareusedfor thecorrespon-
dencedetection(3.3).Figure6 showsthisprocessfor onesliceof aMagneticResonance(MR)
scanof thebrain.
To overcomethepooredge-preservingpropertiesof linearlow-passfilters,weuseanonlin-
eardiffusionfilter. A filteredversionp of volumeScanbedescribedasasolutionof thepartial
differentialequation(PDE):
∂t p div ? g ∇pσ 2 ∇p@ (24)
with Neumannboundaryconditionsandthe original imageas initial state(Weickert, 1997).
The diffusion function g : <A;B< is usedto reducethe noisesensivity andthusdependson
themagnitudeof thegradientof smoothedimageintensities,computedby convolving p with
a Gaussiankernelof standarddeviation σ. The ideaof the diffusion function is to stop the
filtering processat regionswith high gradients,(i.e. at edgesin an image),andto provide a
valuecloseto zerothere.In our method,we usea diffusionfunctionproposedby Weickert in
(Weickert,1997):
gx2 CDE DF 1 for x2 0
1 exp 1 CG
xH λ I 8 for x2 J 0 (25)
The parameterλ separatesregions of low contrastfrom thoseof high contrast. For values
39
greaterthanλ, thefiltering is reduced,while for valueslessthanor equalto λ strongersmooth-
ing is applied.For theconstantC, Weickert proposesC 3 31448which givesvisually good
resultsandgivestheflux fx x K g x2 theexpectedbehavior (i.e. f is increasingfor values x L λ anddecreasingfor values x J λ). As ana priori determinationof λ is very difficult,
the contrastparameterwassetinteractively for eachvolumein our approach.Furthermorea
paralleladditive operatorsplitting (AOS) schemewasusedfor computationalefficiency. See
(Weickert et al., 1998)for details.
(a) (b) (c)
Figure6: Illustrationof featurepointextraction.For abettervisualimpressiononly adetailof the imageis shown. (a) Sliceof anMR scan;(b) Sliceafterapplyinga nonlineardiffusionfilter; (c) Magnitudeof thegradientof theblurredimageafterthresholding.
3.3 Corr espondencedetection
After extractingfeaturepoints,thecorrespondencesbetweenthe referenceimageR andtem-
plateimageT is computedfor thesepoints.A commonway to minimizethedistancebetween
regionsof two volumesconsistsof finding theglobaloptimumof a function which measures
their similarity. This canbe expressedasa costfunction M : < n ;N< which is optimizedfor
a voxel x betweentwo regionsof R andT in termsof a giventransformationO ϑ. Thesearch
spaceis restrictedby asetof parametervectorsϑ >P< n.
40
Our approachusesthelocal normalizedcross-correlation(NCC)
M ϑ ∑k QSR G x I fR k "K f
T TO 1 1
ϑ
k U
∑k QSR G x I f 2R k VK ∑k QSR G x I f 2
T TO 1 1
ϑ
k XW x > Ω (26)
which is maximizedat a givenvoxel by a bruteforcesearch.Thereforewe assumea window
of sizew ( w ( w aroundavoxel x in thereferenceimage,andcomputethemaximalNCCby
shifting a window of similar sizein the templateimage. In Equation(26), this window is de-
scribedby alocalneighborhoodof avoxelx definedas Y x Z x w y w z w - x
w y w z w - . The searchspacein our methodis restrictedto translationsbecauseother
transformationslike rotationsor scalingwould be of highercomputationalcomplexity. Fur-
thermoretheNCCis only computedfor voxelswith highgradientmagnitudescalculatedoutof
blurredimageintensities,asdescribedin section(3.2). For a betterperformancefor largedata
setstheoptimizationis solvedin parallel.
3.4 Inter polation fr om sparsedisplacementestimates
Thesparsedeformationestimatesobtainedatthefeaturepointscomputedby alocalnormalized
cross-correlation,arenow introducedasexternalforcesinto anelasticmodel.Weuseasimilar
energy termasdescribedin Section2.2.2usingthefinite elementmethodfor thediscretization.
Henceweseekthedeformationu thatminimizesEquation(14)– repeatedherefor convenience
Eu 1
2 , Ωσ - ε dΩ , Ω
u - F dΩ Theunderlyingideais againto restricttheregistrationprocesssothattheresultingdeformation
field is a priori fixedby theestimatesat thesepoints.
For a volume of 256 ( 256 ( 124 voxels, the linear systemof equationswe obtain has
approximately532000unknowns,which is alsosolvedin parallelwith thePortableExtensible
41
Toolkit for ScientificComputation(PETSc)package(Balayet al., 2000a;Balayet al., 2000b;
Balayet al., 1997).Theexecutiontime for thewholeregistrationprocessis usuallyaboutfive
minuteson aclusterwith 12CPUs(seeSection2.2.2for details).
3.4.1 Inferring empirically observed anatomical variability
In orderto describethe mechanicalbehavior of tissueundergoinga deformation,the relation
betweenstressandstrain is expressedby an elasticitymatrix D, generatedduring the matrix
assembly. For isotropicmaterialtwo parametersareneeded:Young’smodulusE asa measure
of stiffnessandPoisson’s ratio ν asameasureof incompressibility.
Typically elasticityparametershave beensetarbitrarily andhomogeneously(Bajcsyand
Kovacic,1989;Ferrantet al., 2000b)which is only a roughapproximationof the underlying
tissue. RecentlyLesteret al. (Lesteret al., 1999)appliedan inhomogeneousviscousfluid
modelto brainandneckregistration.Manualsegmentationsof thebonewereusedasaregionof
high stiffness.Davatzikoset al. (Davatzikos,1997)appliedinhomogeneitiesto brainwarping
settingtheelasticityparametersof thebrainfour timeshigherthantheir valuein theventricles.
Our approachdiffers in that inhomogeneouselasticityparametersarederivedfrom anem-
pirical estimateof anatomicalvariability, sothateachdiscreteelementcanobtainits own ma-
terial propertiesduringthematrix assembly. We useda setof 154MR scansof thebrain,first
segmentedinto white matter, grey matter, cerebrospinalfluid (CSF)andbackgroundusingan
EM-basedstatisticalclassificationalgorithm(Wells et al., 1996a).In thenext step,theheadof
eachscanwasalignedto an arbitrarily selectedscanout of this database,usingglobal affine
transformations(Warfieldet al., 1998a)andour nonrigidregistrationmethod.Figure7 shows
theresultfor thetissueclassesafternonrigidregistration,averagedover all scans.In orderto
generatea model for inhomogeneouselasticities,we usean entropy measurefor eachvoxel.
42
(a) (b) (c) (d)
Figure 7: Imagefrom theaveragedvolumefor differenttissueclassesafternonrigid regis-tration.Dark regionsimply a slightoverlapping.(a)Background;(b) CSF;(c) graymatter;(d)whitematter.
Thereforewedefinethejoint voxelwiseentropy as
hs1 s2 s3 s4 4
∑i % 1
psi ln p si (27)
whereeachsi representsthesumover all scansfor oneof the four differentsegmentedtissue
classesat a certainvoxel. Accordingto theseresults,the elasticityparametersarecomputed
for every voxel. We choosea linearmappingfor thecomputedjoint voxelwiseentropy of the
identifiedbraintissueswherethePoissonratio ν wasscaledin therangeof ν > 0 1 0 4 while
Young’s elasticitymodulusE hada rangeof E > 2kPa 10kPa . Thebackgroundwassetto a
low stiffness(E 1kPa) andincompressibilityparameter(ν 0 05). Figure8 showsasliceof
thecomputedmodelandtheassociatedvaluesfor ν.
(a) (b)
Figure 8: Model of empiricallyobservedvariability. (a) Sliceout of themodelaftervoxel-wiseentropy computation.Darkregionsimply a low entropy value;(b) Computedincompress-ibility parameter(Poissonratio ν) for eachvoxel of thesameslice. Dark regionsimply a lowvaluefor ν.
43
3.5 Illustration of nonrigid registration with homogeneousand inhomo-
geneouselasticities
In orderto demonstratethe behavior of our deformationmodelwith homogeneousandinho-
mogeneouselasticities,the algorithm was appliedto register 159 MR scansof the brain of
youngadults. Eachscanwasfirst globally registeredto an arbitrarily-chosendatasetby an
affine transformation(Warfieldetal., 1998a).Thenonrigidregistrationwith homogeneousand
inhomogeneouselasticitieswasthenappliedto the aligneddata. Figure9 shows the results
of the matchingprocessafter averagingover all scans.Becausewe areperformingregistra-
tion amongdifferentsubjects,a globalaffine transformationnormallywill not beableto align
referenceandtemplateimageproperly. This leadsto a blurredaverageimage(Figure9 (b)).
Thealignmentfor theelasticmodelis shown in Figure9 (c) for homogeneousandin Figure9
(d) for inhomogeneouselasticities.In thecaseof homogeneouselasticitieswe useE 3kPa
for theYoung’s elasticitymodulus,andν 0 4 for thePoissonratio, asusedby Ferrantet al.
(Ferrantet al., 2000b).
An analysisof the summedsquareddifferencesshowed an improvementof 2% usingin-
homogeneouselasticities.This rathersmalleffect is dueto thesettingof featurepointsin our
experiments.As canbeseenin Figure8, largeregionsof whitematteronly havea smallrange
of anatomicalvariability. In other words, the large numberof fixed deformationestimates
constrainsthe interpolationdoneby the elasticmodel. Furtherresearchwill investigatenew
approximationschemesto addressthis.
44
(a) (b)
(c) (d)
Figure9: Resultsof globalaffineandnonrigidregistrationappliedto 159subjects.(a)Sliceof referencevolume; (b)-(d) Resultafter registrationandaveragingover all scansusing: (b)globalaffine registration;(c) nonrigidregistrationwith homogeneouselasticities;(d) nonrigidregistrationwith inhomogeneouselasticities.
45
4 Registration of Diffusion TensorImages
4.1 Intr oduction
A largeamountof researchhasbeendoneover thelasttwo decadeson theregistrationof med-
ical imagesprovidedby differentimagingmodalities,resultingin aproliferationof algorithms
with asolid theoreticalbackground.Non-scalarimagingmodalitiesareemergingin Radiology.
For examplePhaseContrastAngiographyMRI (PCA-MRI) (Dumoulinet al., 1989)provides
adescriptionof speedanddirectionof bloodflow, andDiffusionTensorMRI (DT-MRI) (LeBi-
hanetal.,1986;Basseretal.,1994;Pierpaolietal.,1996)providesdiffusiontensorsdescribing
local mobility of watermoleculesin tissue. The increasingclinical relevanceof suchimage
modalitieshaspromptedresearchfocusedon registrationmethodssupportingthem.
Althoughthetheorythatwill bepresentedin this chapteris generalandvalid for any data
dimensionsandarbitrarytensordata(including thespecialcaseof vectors)thedriving exam-
ple throughoutthis sectionwill beregistrationof DT-MRI data.DT-MRI is a relatively recent
MR imagingmodalityusedfor relatingimageintensitiesto therelativemobility of endogenous
tissuewatermolecules.In DT-MRI, a tensordescribinglocal waterdiffusionis calculatedfor
eachvoxel from measurementsof diffusion in several directions. To measurediffusion, the
Stejskal-Tannerimagingsequenceis used(StejskalandTanner, 1965).Thissequenceusestwo
stronggradientpulses,symmetricallypositionedarounda 180[ refocusingpulse,allowing for
controlleddiffusion weighting. DT-MRI hasshown its clinical value in early assessmentof
brain ischemiaandstroke by showing the decreasedability of the affectedtissuesto diffuse
water (Hajnal andBydder, 1997; Provenzaleand Sorensen,1999). SinceMRI methods,in
general,obtainamacroscopicmeasureof amicroscopicquantity(whichnecessarilyentailsin-
travoxel averaging),thevoxel dimensionsinfluencethemeasureddiffusiontensorat any given
locationin thebrain. Factorsaffectingtheshapeof theapparentdiffusiontensor(shapeof the
46
diffusionellipsoid)in thewhite matterincludethedensityof fibers,thedegreeof myelination,
theaveragefiberdiameterandthedirectionalsimilarity of thefibersin thevoxel. Thedirection
of maximumdiffusivity is describedby theeigenvectorcorrespondingto thelargesteigenvalue.
This is descriptive of theorientationof white matterfiber tractsin thecentralnervoussystem.
This is duetherestricteddiffusioncausedby thepresenceof a tightly packedsheathof myelin
surroundingthe axons(Basseret al., 1994; Peledet al., 1998). Somepostprocessingalgo-
rithms suitedto DT-MRI have arisenover the last years. For example,Westinet al. (Westin
et al., 2001)describesanisotropy analysisandfiltering of DT-MRI data,andRuiz et al. (Ruiz-
Alzola et al., 2001a)describesan approachto point landmarkdetectionin tensordata. The
ability of visualizingandautomaticallytracingwhite mattertractsis expectedto play a major
role in basicneurosciences,in theunderstandingof neurologicaldisorders(especiallythoseas-
sociatedwith white matterdemyelination),agingandbraindevelopment(Pouponet al., 1998;
Pouponet al., 1999;Weinsteinetal., 1999;Westinet al., 2001).
Theapproachpresentedherestemsfrom our work presentedin (Ruiz-Alzolaet al., 2000;
Ruiz-Alzolaet al., 2001a;Ruiz-Alzolaet al., 2001b)andit is basedon templatematchingby
locally optimizingasimilarity function(Sect.4.3).A localstructuredetectorfor generictensor
fields (Sect.4.4) allows us to constrainthe matchingto highly structuredareas. In order to
obtaina densedeformationfield, thesparseestimatesfrom thetemplatematchingareinterpo-
lated.Thewholeapproachis embeddedin amultiresolutionschemeusingaGaussianpyramid
in orderto dealwith moderatedeformationsanddecreasethe influenceof falseoptima. We
alsopresent(Sect.4.5)someillustrativeresultscarriedouton syntheticandclinical data.
4.2 Registration of DT-MRI Data
In addition to our own work (Ruiz-Alzola et al., 2000; Ruiz-Alzola et al., 2001a),previous
work in diffusiontensorregistrationincludestheeffortsof Alexanderandcoworkers (Alexan-
47
deret al., 1999;AlexanderandGee,2000). They extendthemultiresolutionelasticmatching
paradigmin (BajcsyandKovacic,1989;GeeandBajcsy, 1999)to tensordata. Tensorreori-
entationis not includedin the regularizationterm,but tensorsarereorientedin eachiteration
accordingto theestimateddisplacementfield. Severalstrategiesto estimatethetensorreorien-
tationfrom thedisplacementfield arealsoinvestigated.
We statetheproblemof registrationasa mappingof a referenceanatomy, depictedby the
signalSr
x , to a deformedone, representedby the signalSd
x . Equation(28) describesa
modelto characterizetherelationshipbetweenbothsignals,whereD modelsthedeformation
appliedto thereferencesignal,andbothH andthenoisev modeltheinter-scandifferences.
Sd
x H D Sr
x ;x v
x (28)
ThedeformationD representsa space-variantshift systemand,hence,its responseto a signal
Sx is D S x S
x d
x , whered
x is adisplacementfield. With regardto thedifferences
betweenthesystemsgeneratingtheimages(signals), weconsiderH to beanon-memory, pos-
sibly space-variant,systemdependingonasethx
h1
x hp
x t of unknown parameters
andthenoiseto bespatiallywhiteandwith zeromean.With thesesimplificationsanddefining
SHr
x H Sr
x ;x , themodel(28) reducesto:
Sd
x SH
r
x d
x v
x (29)
Thegoalof registrationis to find thedisplacementfield dx thatmakesthebestmatchbetween
Sr
x andSd
x accordingto (29).
48
4.3 TemplateMatching
Severalschemescanbeusedto estimatethedisplacementfield in (29). Whenthereis noa pri-
ori probabilisticinformationaboutthesignalandnoisecharacterization,aLeast-Squares(Moon
andStirling, 2000)approachis a naturalchoice.For this, all that is requiredis a suitabledefi-
nition of aninnerproductand,thereafter, aninducednorm. Notethatscalar, vectorandtensor
fieldsareapplicationsof arealdomainontoEuclideanvectorspacesandthisallowsusto define
the innerproductbetweenfieldsby meansof the integral over thewholedomainof the inner
productsbetweentheir values. Let us considerthe functionalset \ f : D "; V where
D is a real domainandV is an Euclideanspace.Thenan innerproductcanbedefinedon \as ] f1 f2 J A8
D wx ] f1
x f2 x J dx, wherew
x is a weightingfunction for the inner
product. Note that the innerproductin the left-handsideis definedbetweenfieldsandin the
right-handside,insidetheintegral, is definedbetweenvaluesof thefield.
Theleastsquaresestimatoris obtainedby minimizing a costfunction(30) thatconsistsof
thesquarednormof theestimationerror.
C ^ d x ;h x !Sd
x SH
r
x d
x ! 2 (30)
The dependency on the unknown parametershx can be removed by estimatingthem us-
ing constrainedleast-squaresschemes. For example, if the parametersare assumedto be
constantall over the spatialdomain,a least-squaresestimationcan be obtained,hdx
hSd
x Sr
x d
x , andsubstitutedin C ^ to obtaina new costfunction (31) thatonly de-
pendson d (see(Ruiz-Alzolaetal., 2001b)for furtherdetails)
Cdx C ^ d x ; h d x (31)
Theoptimizationof Cdx in orderto obtainthedisplacementfieldd
x is adauntingtaskthat
49
requiresadditionalconstraintsto make it feasible.TemplateMatching tradesoff accuracy and
computationalburdento approximatea solutionfor this optimizationproblem. It essentially
consistsof defininga templatefrom theneighborhoodof every point of thedeformeddataset.
Eachof thesetemplatesis thencomparedor matchedagainsttheneighborhoodsof tentatively
correspondentpoints in the referencedatasetand a similarity measureis obtainedfor each
of them. The tentative point whoseneighborhoodprovidesthe biggestsimilarity is selected
ascorrespondingto the currentpoint in the deformeddatasetandthe displacementbetween
both points is obtained. Thereis a fundamentaltrade-off to be consideredin the designof
the neighborhoods:they must be non-local,and hencelarge in size, in termsof the Sd
x
space-frequenciesto avoid the ill-posednessarising from the lack of discriminantstructure
(apertureproblem(Poggioet al., 1985)),andthey mustbe local, andhencesmall in size, in
termsof the unknown displacementfield spatial-frequenciesto guaranteethe validity of the
local deformationmodel.Adaptive templateswith differentsizesandweightscanhelpto deal
with this problem.
Let Tx x0 bea window functioncenteredin a genericpoint x0 in thedeformeddataset
anddesignedfollowing thepreviousremarks.Thetemplatematchingassumptionstransform(29)
into (32), thatholdsfor everypoint x0 in thedeformeddataset.
Tx x0 Sd
x T
x x0 SH
r
x d
x v
x (32)
Equation(32)hasanintuitiveinterpretation:any neighborhoodin thedeformeddatasetaround
a point x0, definedby the window function Tx x0 , correspondsto a neighborhoodin the
referencedatasetdefinedby thewindow functionTx x0 d
x which hasbeenwarpedby
thedeformationfield. Templatematchingassumesthatamodelis chosenfor thedisplacement
field andfor theparametersof thetransformationhx in a neighborhoodof thepoint x0 to be
registered.For examplethe deformationfield modelmay constrainthe templatejust to shift
50
alongthecoordinateaxes(translation),or to undergo rigid motionshenceallowing alsorota-
tionsor mayevenallow stretchandtwist. In any casethemodelfor thelocaldeformationmust
besuchthat it dependsonly on a few parameters,in orderto make thesearchcomputationally
feasible.With respectto theparameters,thecommonchoiceis to assumethemconstantin the
neighborhood.
The templatematchingsplitsa complex globaloptimizationproblem,i.e. coupledsearch-
ing for all thedisplacements,into many simplelocal ones,i.e. searchingindependentlyfor the
displacementof eachpoint usingtemplatematchingin eachcase.For example,for thecom-
moncasewherethedisplacementfield is assumedto beconstantinsidethetemplate,thecost
function(30) reducesto a setof costfunctions
C ^ d x0 ;h x !Tx x0 _? Sd
x SH
r
x d
x0 @ ! 2 (33)
wherex0 refersto every point in the deformeddataset. One of the main characteristicsof
templatematchingis theabsenceof any globalregularizationthatconstrainsthelocalvariability
of the estimateddeformationfield. While this preventsgetting trappedin falseoptima that
are far from the absoluteoptimum, as global optimizationmethodsare proneto, noisecan
producehigh frequency artifactson theestimateddeformation.Hencea further refinementof
thesolutionmaybe advisabledependingon theapplication,eitherpostfilteringtheestimated
deformationor usingit asaninitial solutionfor aglobaloptimizationscheme.
4.3.1 Similarity Functions
A Similarity Function is a monotonicfunction of the cost (30), SFdx F C
d , which
leavesthelocationsof theoptimaunchangedandremainsinvariantwith respectto theunknown
parameters.The local natureof the templatematchingmethodmakesit necessaryto definea
similarity functionSFdx0 for every point in thedeformeddatasetwhich is to bematched
51
ontothereferenceone,i.e., themonotonicfunctionis appliedto (33). In this sectiontheleast-
squaresmethodreferredto above is usedto obtainsuitablelocal similarity functionsfor the
templatematchingof generictensorfields.
Let usfirst considerthatH is theidentitymappingandthatthedisplacementfield is constant
insidethetemplate.Direct useof (33) leadsto
SFSSD
dx0 !
Tx x0 Sd
x Sr
x d
x0 ! 2 (34)
thatcorrespondsto thewell-known Sumof SquaredDifferencessimilarity function.Extending
it by usinginnerproductsandassumingthat!Tx x0 Sr
x d
x0 ! 2 is almostconstantfor
all possibledx0 leadsto analternativesimilarity functionthatcorrespondsto theCorrelation
measure.
SFC
dx0 ] T2
x x0 Sd
x Sr
x d
x0 J (35)
Let us now considerthat H is a space-invariantaffine transformationof the intensity. In this
case
Tx x0 Sd
x aT
x x0 Sr
x d
x0 bT
x x0 1 x v
x (36)
where1x refersto theonetensorfunction(all thecomponentsareequalto oneeverywhere).
Thecost(33) turnsout to be
C ^ d x0 ;a b !Tx x0 Sd
x " aSr
x d
x0 " b1
x ! 2 (37)
A similarity function invariantto a andb canbeobtainedby replacingthesecoefficientswith
their least-squaresestimationandminimizing theresultingcost.Detailscanbefoundin (Ruiz-
52
Alzola et al., 2001b). The resultingsimilarity function is the absolutevalueof a generalized
versionof thewell-known correlationcoefficient
SFρ ` d ` x0 aba cedddddf
s g 1hth2 i sj t k tl
s g 1hth2 i sj t k t
l j p ` d ` x0 aba g 1hth2 i p ` d ` x0 aba j t k tl
p ` d ` x0 aba g 1hth2 i p ` d ` x0 aba j t k t
lm ddddd (38)
where
s Tx x0 Sd
x (39)
p Tx x0 Sr
x d
x0 (40)
t Tx x0 1 x (41)
Theapplicationof theequationsaboverequiresaproperdefinitionof theinnerproduct
] S1
Kn S2
Kn J , DS1i1 o o o in x S2i1 o o o in x dx (42)
andits inducednorm !S Kp ! 2 , D
Si1 qrqrq in x Si1 qrqsq in x dx (43)
Weassumethatthetensorsarecartesian(definedwith respectto anorthonormalbasis)andwe
areusingtheEinsteinnotationfor sums(any repetitionof anindex entailsasummingover this
index). Notethatany implementationrelieson sampleddataandthereforetheintegralsabove
becomesums.
4.3.2 WarpedVectorsand Tensors
Vectorand tensordataare linked to the body underinspectionand, thereafter, any warping
of thesupportingtissuewill leadto a consequentwarpingor reorientationof thesedata. The
53
warpingof thedomaincanbeexpressedby thetransformation
x Tx tu x t d
x tu (44)
wherex standsfor pointsin thereferencedatasetandx t for pointsin thedeformedone.More-
over, the transformationis assumedto be differentiableandhencethe neighborhoodsof the
correspondentpointsx andx t arerelatedthrough
dx v ∇ Tx tuTw dx tx (45)
wherethe deformationgradient ∇ Tx t canbe easily recognizedasthe Jacobianmatrix
Jx t of thetransformationT
x t v ∇ T
x t w y J
x t δT
x t
δx t (46)
Equation(45)simplystatesthat,asfarasthetransformationis differentiable,a linearmapping
relatesthe local neighborhoodsof bothpoints. For finite sizeneighborhoods,thedeformation
gradientcorrespondsto a linearapproximation,asaTaylor’sexpansionclearlyshows
x ∆x z Tx t ∆x t T
x t 1
1!δT
x t
δx t ∆x t (47)
∆x z δTx t
δx t ∆x t (48)
In this work it will be assumedthat the linear approximationis valid sincethe function data,
vectorsor tensors,arerelatedto infinitesimalpropertiesof thetissue.Consequently, twovectors
v andv t arelocally relatedas
v Jx t| v t (49)
54
(50)
Sincetwo secondorderdiffusiontensorsP andPt canbeconsideredasassociatedto quadratic
forms,they arerelatedby
Pt Jtx tu PJ
x tp (51)
Equation(51) providesa theoreticalway to estimatethealterationof diffusiontensorsdue
to a deformationfield. Neverthelessit is not clear that DT-MRI dataactually are modified
accordingto this modelspeciallyin areasof high anisotropy, i.e. thewhite matterfiber tracts,
wherethesedataaremostrelevant.Theideahereis thattheshapeof thediffusiontensorshould
be preserved throughthe transformationandhenceit mustonly be reorientedasan effect of
local rotation and shear. This essentiallymeansthat the deformationfield only affects the
directionalpropertiesof diffusionbut not its strengthalongits principalaxes.For example,in
a referenceframeintrinsic to a fiber tractdiffusionshouldremaininvariantwith respectto the
deformation.Thishasmotivatedasearchfor tensortransformationsthatmaintaintheshapeand
includeboththeeffect of local rotationandshear. Early experimentson this topic have begun
(Sierra,2001). An ad-hocsolutionto this problemis to scaletheresultingtensorafter (51) is
appliedsoas,for example,preserve theellipsoidvolumeor normalizethe largesteigenvalue.
Anotherpossibilityis to modify thedeformationgradientsoasto avoid undesirableeffectssuch
asthescaling(Alexanderetal., 1999).Neverthelessmuchresearchis still neededto clarify the
appropriatetensortransformationto beused.
A mathematicaltool to dealwith this problemis the Polar DecompositionTheorem (see
for example(Segel, 1987) from the theoryof non-linearelasticity). It allows us to dealnot
only with infinitesimalbut alsowith finite deformations.Thetheoremstatesthat for any non-
singularsquarematrix, suchasthe DeformationGradientJx t ), thereareuniquesymmetric
55
positive definitematricesUx t andV
x t andalsoa uniqueorthonormalmatrix R
x t such
that
Jx t R
x t U
x t Vx t R
x t (52)
This leadsto importantgeometricinterpretationsof the geometricmapping. For example,
noticethat a sphereis first stretchedby the mappingin the directionsof the eigenvectorsof
Ux t andthenrotatedby R
x t . Thereafter, a transformationsuchthatlocally R
x t I is said
to beaPureStrain atx t while if Ux t V
x t I it is saidto beaRigidRotationat thatpoint.
As mentionedabove(Sect.4.3),thematchingapproachto dataregistrationreliesonamodel
for the local displacementfield inside the template. In order to perform the matching,the
vectorsor tensorsinsidethetemplatemustbereorientedaccordingto thehypothesizedmodel.
Note that if a simple model suchas just shifting the templatealong the coordinateaxes is
adopted,i.e. assuminga constantdisplacementfield for all the points inside, the vectorsor
tensorswould not be reoriented.Similarly, if the model is rigid motion no stretchingof the
vectorsor tensorsshouldbe considered.From a practicalpoint of view, no reorientationis
performedduringmatchingandthereforea constantdisplacementfield is assumedinsidethe
template.This is not a limitation aslong asthelocal rotationis smallandin fact it is accepted
in conventionaltemplatematchingof scalardata.Thereorientationis thencalculatedoncethe
displacementfield –andits gradient–hasbeenestimated.
4.4 Structure Measures
Matchingmustbeconstrainedto areaswith localhighdiscriminantstructure.Dependingonthe
dataset,this approachwill leadto very sparsecontrolpointsto estimatethedeformationfield.
The applicability of the methodultimately dependson the characteristicsof the deformation
field – it beingobvious that if the deformationfield hasa large spatialvariability, the sparse
56
displacementfield estimatedfrom thedetectedpointswill suffer from spectralaliasing.Incre-
mentingthe samplingdensityby acceptinglow structurepoints is possibledependingon the
noisecharacteristicsof thedata,sinceit is unacceptableto allow noiseto provide thediscrimi-
nantinformationthatdrivesthetemplatematching.Whenit is not possibleto provide enough
pointsaccordingto thefrequency propertiesof thedeformationfield, it might benecessaryto
resortto regularizedschemes,suchaselasticmodels,thatusethewholedataset.Alternatively,
in someapplications,anadditionalchannelof datais customarilyprovided.Thisis thecase,for
example,in DT-MRI usingEPI sequenceswhereadditionalT2-weightedimagesareprovided.
Thereforeit is possibleto estimatedifferentsparsedisplacementfields from T2 andDT-MRI
andcombinethemin orderto estimatethewholedisplacementfield, providing constraintsboth
from structuralT2 imagesandfrom thediffusiontensors(thewhitematterfiber tracts).
In orderto identify theareasof highstructure,weuseameasureof cornerness(Ruiz-Alzola
et al., 2000;Ruiz-Alzolaetal., 2001a;Ruiz-Alzolaet al., 2001b)which generalizesthelocally
averagedouterproduct(i.e. thecorrelationmatrix)of thegradientfield (Rohr, 1997)frequently
usedin thescalarcase.
4.5 Results
In this sectionresultson both syntheticand real dataare presented.The real imaging data
were acquiredon a GE Signa1.5 TeslaHorizon Echospeed5.6 scanner, using a Line Scan
Diffusiontechnique,with thefollowing parameters:no averaging,effective TR=2.4s, TE=65
ms,bhigh 750s' mm2, fov=22cm,voxel size4 8 ( 1 6 ( 1 5 mm3, 6 kHz readoutbandwidth,
acquisitionmatrix128 ( 128.
Figure (10.a) shows the clustersof points that have beendetectedas highly structured
in a DT-MRI sagittal imageoverlayedon the correspondingT2-weightedMRI imageand
Fig. (10.b) shows theseclustersin the portion correspondingto the highlightedsquare.Re-
57
call thatthematchingis performedusingtheseclusters,not theisolatedlandmarks.Noticethat
thediagonalcomponentsof thetensorfield provide strongerandmorestructuredsignalsthan
theoff-diagonalonesandhow thestructuredetectorfindsthethin structuresin theseimages.It
mustberecalledthatthecooperationof all thecomponentsis whatprovidesthisresult.In order
to obtainthe clusters,we have normalizedthe tensorfield componentsto fit into the interval 1 1 (weaker componentsdo not reachtheextremavalues).Theestimationsof thegradient
andthegeneralizedcorrelationmatriceshave beenmadeusing3 ( 3 neighborhoods.Thedif-
ficulty of presentingillustrative resultsfrom volumedatausing2D figureshasmotivatedusto
reportthis experimentusingasingleDT-MRI slice(thetensorsin it are3D). Nevertheless,the
methodis essentiallyN-dimensionalandit canbedirectlyappliedto volumesof datausingthe
sameparameters,justaddingonemoredimensionin thedefinitionof theneighborhoods.
a) b)
Figure10: a)High structureclustersoverlayedonT2W MRI b) Detailof theclustersinsidethesquare
In orderto assesstheoverall performanceof our nonrigid registrationmethod,Fig. (11.a)
shows a sagittalMRI slice of the corpuscallosumthat is deformedby a syntheticGaussian
58
field as depictedin Fig. (11.b). In order to estimatethe deformation,a Gaussianpyramid
decompositionis obtained,performingthetemplatematchingon structuredareasin eachlevel
andinterpolatingusingKriging (Ruiz-Alzolaetal.,2000;Ruiz-Alzolaetal.,2001b).Fig. (11.c)
shows theresultof reconstructingtheoriginal imagewith thedeformationfield estimatedwith
our approach,usingtheKriging estimatorwith anexponentialvariogram.
a) b) c)
Figure 11: Syntheticdeformation(a) original MRI (b) deformed(c) reconstructedoriginalusingestimateddeformationfield
Figure(12.a)shows a sliceof a DT-MRI datasetof thecorpuscallosumwheretheprinci-
pal eigenvectordirectionshave beenrepresentedusinga color codingrangingfrom blue (in-
planeprojection)to red (orthogonalto plane)(Peledet al., 1998). The whole approachhas
beenappliedto warp this datasetinto anothercorrespondingto a different individual, shown
in Fig. (12.b),usingthreelevelsof a Gaussianpyramid,andanexponentialvariogramfor the
Kriging interpolatorthat is limited to take into accountthe 8 closestsamples.Figure(12.c)
showsa T2W zoomedversionof theright handsideof theformer, correspondingto theposte-
rior corpuscallosumandtheestimateddeformationfield.
59
a) b) c)
Figure12: DT-MRI interpatientwarping. a, b) DT-MRI of different individuals. c) zoomedT2W of theposteriorcorpusof a)andestimateddeformation
4.6 Conclusions
We have describeda framework for non-rigidregistrationof scalar, vectorandtensormedical
data.Theapproachis local,sinceit is basedontemplate-matching,andresortsto amultiresolu-
tion implementationusinga Gaussianpyramidin orderto provide a coarse-to-fineapproxima-
tion to thesolution.This allows themethodto handlemoderatedeformationsandavoidsfalse
local solutions.Themethoddoesnot assumeany globala priori regularization,andtherefore
avoidsthecomputationalburdenassociatedwith thoseapproaches.
60
5 The Monge–Kantorovich Problemand ImageRegistration
In this section,we presenta methodfor producingareapreservingsurfacedeformations,and
moregeneralmasspreservingareaandvolumedeformations,basedon theminimizationof a
functionalof Monge–Kantorovich type.Thetheoryis basedon theproblemof minimizing the
costof redistributing a certainamountof massbetweentwo distributionsgivena priori. Here
thecostis a functionof thedistanceeachbit of materialis moved,weightedby its mass.We
show how theresultinglow-orderdifferentialequationsmaybeusedfor registration.
5.1 The Monge–Kantorovich Problem
Here we presenta methodfor imagewarping andelasticregistrationbasedon the classical
problemof optimalmasstransport.Themasstransportproblemwasfirst formulatedby Gaspar
Mongein 1781,andconcernedfinding theoptimalway, in thesenseof minimal transportation
cost,of moving a pile of soil from onesite to another. This problemwasgivena modernfor-
mulationin thework of Kantorovich (Kantorovich, 1948),andsois now known astheMonge–
Kantorovich problem. This type of problemhasappearedin econometrics,fluid dynamics,
automaticcontrol, transportation,statisticalphysics,shapeoptimization,expert systems,and
meteorology(Rachev andRuschendorf,1998).
Themethodwe introducein this sectionis designedfor elasticregistration,andis basedon
anoptimizationproblembuilt aroundtheL2 Monge–Kantorovich distancetakenasasimilarity
measure.Theconstraintthatweputon thetransformationsconsideredis thatthey obey amass
preservationproperty. Wewill assumethatarigid (non-elastic)registrationprocesshasalready
beenappliedbeforeapplyingour scheme.
Our methodis basedon differentialequationsandin this sensemaybethoughtof asbeing
in the sameclasswith optical flow andelasticdeformationmodelapproachesto registration.
See(Hataetal.,2000),(Davatzikos,1997)and(Lesteretal.,1999)for representativeexamples
61
of thesemethods.Ourmethodalsohasastrongconnectionto computationalfluid dynamics,an
areawhichhaspreviouslybeensuccessfullyappliedto brainregistrationproblems(Christensen
et al., 1996).
Our method,however, hasa numberof distinguishingcharacteristics.It is parameterfree.
It utilizesall of thegrayscaledatain bothimages,andplacesthetwo imageson equalfooting.
It is thussymmetrical,theoptimalmappingfrom imageA to imageB beingthe inverseof the
optimalmappingfrom B to A It doesnotrequirethatlandmarksbespecified.Theminimizerof
thedistancefunctionalinvolvedis unique;thereareno otherlocal minimizers.Thefunctional
at the heartof the methodis suchthat the correspondingdifferentialequationsgoverningits
minimizationareof low order. Finally, it is specificallydesignedto take into accountchanges
in densitythatresultfrom changesin areaor volume.
We believe that this type of elasticwarpingmethodologyis quite naturalin the medical
context wheredensitycanbea key measureof similarity. It alsooccursin functionalimaging,
whereonemaywantto comparethedegreeof activity in variousfeaturesdeformingover time,
andobtaina correspondingelasticregistrationmap. A specialcaseof this problemoccursin
any applicationwherevolumeor areapreservingmappingsareconsidered.
5.1.1 Formulation of the Problem
We now give a modernformulationof the Monge–Kantorovich problem. We assumewe are
given, a priori, two subdomainsΩ0 and Ω1 of Rd, with smoothboundaries,and a pair of
positivedensityfunctions,µ0 andµ1, definedon Ω0 andΩ1 respectively. We assume8 Ω0µ0
8Ω1
µ1 sothatthesametotalmassis associatedwith Ω0 andΩ1 We considerdiffeomorphisms
u from Ω0 to Ω1 whichmaponedensityto theotherin thesensethat
µ0 Du µ1 u (53)
62
which we will call the masspreservation(MP) property, andwrite u > MP Equation(53) is
calledtheJacobianequation.Here Du denotesthedeterminantof theJacobianmapDu, and denotescompositionof functions.In particular, Equation(53) impliesthatif asmallregionin
Ω0 is mappedto a largerregion in Ω1 thentheremustbea correspondingdecreasein density
in orderfor themassto bepreserved.
Theremaybemany suchmappings,andwewantto pick outanoptimalonein somesense.
Accordingly, wedefinethesquaredL2 Monge–Kantorovich distanceasfollows:
d22
µ0 µ1 inf
u > MP , !ux x
! 2µ0
x dx (54)
An optimalMP mapis a mapwhich minimizesthis integral while satisfyingtheconstraint
(53). The Monge–Kantorovich functional(54) is seento placea penaltyon the distancethe
mapu moveseachbit of material,weightedby thematerial’s mass.A fundamentaltheoretical
result (Brenier, 1991;GangboandMcCann,1996), is that thereis a uniqueoptimal u > MP
transportingµ0 to µ1, andthat this u is characterizedasthe gradientof a convex function wi.e., u ∇w This theorytranslatesinto a practicaladvantage,sinceit meansthat thereareno
non-globalminimato stall our solutionprocess.
5.1.2 Computing The Transport Map
Therehave beena numberof algorithmsconsideredfor computinganoptimal transportmap.
Forexample,methodshavebeenproposedbasedonlinearprogramming(Rachev andRuschendorf,
1998),andon Lagrangianmechanicscloselyrelatedto ideasfrom the studyof fluid dynam-
ics (BenamouandBrenier, 2000). An interestinggeometricmethodhasbeenformulatedby
CullenandPurser(CullenandPurser, 1984).Here,we follow closelythework in (Haker and
Tannenbaum,2001a;HakerandTannenbaum,2001b).
Let u : Ω0 ; Ω1 beaninitial mappingwith themasspreserving(MP) property. Inspiredby
63
(Brenier, 1991;Gangbo,1994),weconsiderthefamily of MP mappingsof theform u u s1 1
ass variesoverMP mappingsfrom Ω0 to itself, andtry find ans which yieldsa u without any
curl, thatis, suchthat u ∇w. Oncesuchans is found,we will have theMonge–Kantorovich
mappingu We will alsohave u u s ∇w s known asthepolar factorizationof u with
respectto µ0 (Brenier, 1991).
5.1.3 Removing the Curl
Ourmethodassumesthatwehavefoundandinitial MP mappingu Thiscanbedonefor general
domainsusinga methodof Moser(Moser, 1965;DacorognaandMoser, 1990),or for simpler
domainsusinga type of histogramspecification.Oncean initial MP u is found, we needto
apply the processwhich will remove its curl. It is easyto show that the compositionof two
masspreserving(MP) mappingsis an MP mapping,andthe inverseof an MP mappingis an
MP mapping.Thus,sinceu is anMP mapping,we have thatu u s1 1 is anMP mappingif
µ0 Ds µ0 s (55)
In particular, whenµ0 is constant,thisequationrequiresthatsbeareaor volumepreserving.
Next, we will assumethat s is a function of time, and determinewhat st shouldbe to
decreasetheL2 Monge–Kantorovich functional. This will give usanevolution equationfor s
andin turn anequationfor ut aswell, the latterbeingthemostimportantfor implementation.
By differentiatingu s u with respectto time,we find
ut Dust s1 1 (56)
Dif ferentiating(55)with respectto timeyields
divµ0st s1 1 0 (57)
64
from whichweseethatst andut shouldhave thefollowing forms:
st 1
µ0ζ # s ut
1µ0
Duζ (58)
for somevectorfield ζ onΩ0 with divζ 0 and ζ ~n 0 on∂Ω0 Here ~n denotesthenormal
to theboundaryof Ω0. This lastconditionensuresthats remainsa mappingfrom Ω0 to itself,
by preventingtheflow of s givenby st 6 1
µ0ζ 7 s from crossingtheboundaryof Ω0 This
alsomeansthattherangeof u u s1 1 is alwaysuΩ0 Ω1
Considernow theproblemof minimizing theMonge–Kantorovich functional:
M , u x x 2µ0
x dx (59)
Takingthederivativewith respectto time,andusingtheHelmholtzdecompositionu ∇w χ
with divχ 0 wefind from (58) that
12
Mt , u ζ , χ ζ (60)
Thus,in orderto decreaseM, we cantake ζ χ with correspondingformulas(58) for st and
ut providedthatwe have divχ 0 and χ ~n 0 on ∂Ω0 Thusit remainsto show thatwe
candecomposeu asu ∇w χ for sucha χ 5.1.4 Gradient Descent:Rd
Welet w beasolutionof theNeumann-typeboundaryproblem
∆w divu ∇wT~n u ~n on∂Ω0 (61)
andsetχ u ∇w It is theneasilyseenthatχ satisfiesthenecessaryrequirements.
65
Thus,by (58),wehave thefollowing evolutionequationfor u:
ut 1
µ0Du ? u ∇∆ 1 1div
u@_ (62)
Thisis afirstordernon-localschemefor ut if wecount∆ 1 1 asminus2 derivatives.Notethat
thisflow is consistentwith respectto theMonge–Kantorovich theoryin thefollowingsense.If u
is optimal,thenit is givenasu ∇w in whichcaseu ∇∆ 1 1divu ∇w ∇∆ 1 1div
∇w 0
sothatby (62), ut 0
5.1.5 Gradient Descent:R2
Thesituationis somewhatsimplerin theR2 case,dueto thefact thata divergencefreevector
field χ canin generalbe written asχ ∇ h for somescalarfunction h where represents
rotationby 90deg, so that ∇ h hy hx In this case,we solve Laplace’s equationwith a
Dirichlet boundarycondition,andderive theevolutionequation
ut 1
µ0Du∇ ∆ 1 1div
u (63)
5.1.6 Generalizations
WenotethatwecandefineageneralizedMonge–Kantorovich functionalas
M , Φ u i µ0 (64)
whereΦ : Rd ; R is a positive strictly convex C1 cost function, and i is the identity map
ix x In particular, the L2 Monge–Kantorovich problemdescribedabove, correspondsto
66
thecostfunctionΦx u x u 2 If wedefine
Ψ : ∇Φ u i (65)
thenanalysissimilar to thataboveshowsthatMt mustbeof theform
Mt , Ψ ζ (66)
whereasbefore
ζ µ0st s1 1 (67)
is adivergence-freevectorfield. Thisanalysisyieldsanevolutionequationof theform
ut 1
µ0Du
Ψ ∇∆ 1 1divΨ (68)
whereit is understoodthattheLaplacianis invertedwith respectto appropriateboundarycon-
ditions.
Further, a purelylocal flow equationfor theminimizationof theMonge-Kantorovich func-
tionalmaybeobtainedby setting
ζ ∇divΨ ∆Ψ (69)
It is straightforwardto checkthatin this casedivζ 0 and
Mt 1
2 , curlΨ 2 0 (70)
67
Thecorrespondingsecondorderlocal evolutionequationfor u is
ut 1
µ0Du
∇divΨ ∆Ψ (71)
and(70)showsthatat optimality wemusthavecurlΨ 0 soΨ ∇w for somefunctionw5.1.7 Defining the Warping Map
Typically in elasticregistration,onewantsto seeanexplicit warpingwhich smoothlydeforms
oneimageinto theother. Thiscaneasilybedoneusingthesolutionof theMonge–Kantorovich
problem.Thus,weassumenow thatwehaveappliedourgradientdescentprocessasdescribed
aboveandthatit hasconvergedto theMonge–Kantorovich optimalmappingu It is shown in
(BenamouandBrenier, 2000)thattheflow Xx t definedby
Xx t x t
ux" x (72)
is thesolutionto acloselyrelatedminimizationproblemin fluid mechanicsandprovidesappro-
priatejustificationfor using(72) to defineourcontinuouswarpingmapX betweenthedensities
µ0 andµ1. See(McCann,1997)for applicationsandadetailedanalysisof thepropertiesof this
displacementinterpolation.
5.2 Implementation and Examples
We notethateventhoughour non-localmethodrequiresthat theLaplacianbeinvertedduring
eachiteration, the problemhasbeensetup specificallyto allow for the useof standardfast
numericalsolverswhich useFFT-typemethodsandoperateon rectangulargrids (Presset al.,
1992).
We illustrate our methodswith a pair of examples. In Figure 13 we show a brain de-
68
formation sequence.One slice eachfrom two MR datasets,acquiredat the Brigham and
Women’s hospital,wereused. The first datasetwaspre-operative, the secondwasacquired
duringsurgery, aftercraniotomyandopeningof thedura.Both werepre-processedto remove
theskull. TheMonge–Kantorvichmappingwasfoundusingtheevolution equation(62) with
intensityvaluesasdensities,scaledslightly sothat thesumof theintensitieswasthesamefor
bothimages.This processtook roughly10 minuteson a singleprocessorSunUltra 10. A full
3D volumetricdatasetcantake severalhoursto process.Thedisplacementinterpolation(72)
togetherwith (53) for theintensitieswasthenusedto find thecontinuousdeformationthrough
time. The first image,in the upperleft, shows a planaraxial sliceat time t 0 00. Thebot-
tom right is an axial slice at time t 1 00 Together, theseimagesrepresentthe input to our
algorithm.Theupperright andlower left imagesrepresentthecomputedinterpolationat time
t 0 33andt 0 66respectively.
The secondexampleshows an applicationof our methodto surfacewarping. Figure14
shows a portion of the white mattersurfaceobtainedby segmentingan MRI scan. We cut
the surfaceend to end andflattenedit into the planeusing a conformalmappingtechnique
(Angenentet al., 1999a;Angenentet al., 1999b),asshown in the left of Figure15. It is well
known thatasurfaceof non-zeroGaussiancurvaturecannotbeflattenedby any meanswithout
somedistortion.Theconformalmappingis anattemptto preservetheappearanceof thesurface
throughthepreservationof angles.Theconformalmappingis a “similarity in thesmall” and
sofeatureson thesurfaceappearsimilar in theflattenedrepresentation,up to a scalingfactor.
Further, the conformalflatteningmapscanbe calculatedsimply by solving systemsof linear
equations.For a triangulatedsurfaceof a few hundredthousandtriangles,this takesonly a few
minuteson a singleprocessorcomputer. Parallelizationcanbeachievedusingfreely available
numericallinearalgebrasoftware.
However, in someapplicationsit is desirableto beableto preserve areasinsteadof angles,
69
so that the sizesof surfacestructuresare accuratelyrepresentedin the plane. The Monge–
Kantorovich approachallowsusto find suchanarea-correctflattening.Theideahereis thatthe
conformalflatteningshouldbealteredby moving pointsaroundaslittle aspossible.Oncewe
haveconformallyflattenedthesurface,wedefineadensityµ0 tobetheJacobianof theinverseof
theflatteningmap,andsetµ1 to a constant.TheMonge–Kantorovich optimalmappingis then
area-correctingby (53). Theresultingmaptook just a few minutesto calculate.Detail of the
conformalsurfaceflatteningandtheareacorrectedflatteningareshown in Figure15. Although
correctedfor area,surfacestructuresarestill clearly discernible. The curl-freenatureof the
Monge–Kantorovich mappingavoids distortioneffectsoften associatedwith areapreserving
maps.
5.3 Conclusions
In thissection,wepresentedanaturalmethodfor imageregistrationandsurfacewarpingbased
on the classicalproblemof optimal masstransportation. Although appliedhere to the L2
Monge–Kantorovich problem,themethodusedto enforcethemasspreservationconstraintis
general,asshown in Section5.1.6,andwill haveotherapplications.For example,any weighted
linearcombinationof theMonge–Kantorovich functionalanda standardL2 energy functional
or othermatchingfunctionalcanbeused.Theseideasareacurrentareaof research.
70
Figure 13: A brain deformationsequence.The upper left (time t 0 00) and lower right(t 1 00)aretheinput into thealgorithm.Theupperright (t 0 33) andlower left (t 0 67)representtheinterpolationusingtheMonge–Kantorovich displacementmapandenforcingthepreservationof mass.
71
Acknowledgments The authorsthankWilliam M. Wells III for helpful commentsanddis-
cussionduringthepreparationof this chapter.
This work wasfundedin partby theSpanishGovernment(Ministerio deEducacion y Cul-
ture)with a visiting researchfellowship (FPUPRI1999-0175)(JRA), theEuropeanCommis-
sionandtheSpanishGovernment(CICYT), with thejoint researchgrant1FD97-0881-C02-01,
aNew ConceptAwardfrom theCenterfor Integrationof MedicineandInnovativeTechnology
(SKW), andNIH grantsP41RR13218,P01CA67165,R01CA86879andR01RR11747.
References
Alexander, D. andGee,J. (2000). ElasticMatchingof Dif fusion TensorImages. Computer
VisionandImageUnderstanding, 77(2):233–250.
Alexander, D., Gee,J., andBajcsy, R. (1999). Strategies for datareorientationduring non-
rigid transformationsof diffusiontensorimages.In Taylor, C. andColchester, A., editors,
LectureNotesin ComputerScience. MedicalImageComputingandComputerAssistedIn-
tervention- MICCAI ’98, volume1496,pages463–472,Cambridge,UK. Springer-Verlag.
Angenent,S.,Haker, S.,Tannenbaum,A., andKikinis, R. (1999a).Laplace-beltramioperator
andbrainsurfaceflattening.IEEE Trans.on MedicalImaging, 18:700–711.
Angenent,S.,Haker, S.,Tannenbaum,A., andKikinis, R. (1999b). On areapreservingmaps
of minimal distortion. In Djaferis,T. andSchick,I., editors,SystemTheory: Modeling,
Analysis,andControl, pages275–287.Kluwer, Holland.
Bajcsy, R. andKovacic,S.(1989).Multiresolutionelasticmatching.ComputerVision,Graph-
ics,andImageProcessing, 46(1):1–21.
73
Balay, S., Gropp,W. D., McInnes,L. C., andSmith, B. F. (1997). Efficient managementof
parallelismin objectorientednumericalsoftwarelibraries. In Arge,E., Bruaset,A. M.,
and Langtangen,H. P., editors,Modern Software Tools in ScientificComputing, pages
163–202.BirkhauserPress.
Balay, S.,Gropp,W. D., McInnes,L. C., andSmith,B. F. (2000a).PETSc2.0 usersmanual.
TechnicalReportANL-95/11 - Revision2.0.28,ArgonneNationalLaboratory.
Balay, S., Gropp, W. D., McInnes, L. C., and Smith, B. F. (2000b). PETSchomepage.
http://www.mcs.anl.gov/petsc.
Barron,J. L., Fleet,D. J., andBeauchemin,S. S. (1994). Performanceof optical flow tech-
niques.InternationalJournalof ComputerVision, 12(1):43–77.
Basser, P., Mattiello, J., and LeBihan,D. (1994). Estimationof the effective self-diffusion
tensorfrom theNMR spin-echo.Journalof MagneticResonanceB, 103:247–254.
Benamou,J.-D. and Brenier, Y. (2000). A computationalfluid mechanicssolution to the
monge–kantorovich masstransferproblem.NumerischeMathematik, 84:375–393.
Black, P. M., Moriarty, T., Alexander, E., Stieg, P., Woodard,E. J., Gleason,P. L., Martin,
C. H., Kikinis, R., Schwartz, R. B., and Jolesz,F. A. (1997). The Developmentand
Implementationof IntraoperativeMRI andits Neurosurgical Applications.Neurosurgery,
41(4):831–842.
Brandt,J. W. (1997). Improvedaccuracy in gradient-basedoptical flow estimation. Interna-
tional Journalof ComputerVision, 25(1):5–22.
Brenier, Y. (1991).Polarfactorizationandmonotonerearrangementof vector-valuedfunctions.
Com.PureAppl.Math., 64:375–417.
74
Bro-Nielsen,M. (1996). Surgery SimulationUsing FastFinite Elements.In Visualizationin
BiomedicalComputing, pages529–534.
Bro-Nielsen,M. (1998). Finite elementmodelingin medicalVR. Proceedingsof the IEEE
SpecialIssueonVirtual & AugmentedRealityin Medicine, 86(3):490–503.
Bro-Nielsen,M. andGramkow, C. (1996).Fastfluid registrationof medicalimages.In Hohne,
K. H. andKikinis, R., editors,Proceedingsof the 4th InternationalConferenceVisuali-
sationin BiomedicalComputing(VBC’96,heldin Hamburg, Germany, September22-25,
1996), volume 1131 of Lecture Notesin ComputerScience, pages267–276.Springer-
Verlag.
Cachier, P., Pennec,X., andAyache,N. (1999). Fastnonrigid matchingby gradientdescent:
Study and improvementsof the “demons” algorithm. TechnicalReport3706, Institut
National de Rechercheen Informatiqueet en Automatique. http://www.inria.fr/rrrt/rr-
3706.html.
Christensen,G. E., Rabbit,R. D., andMiller, M. I. (1996). Deformabletemplatesusinglarge
deformationkinematics.IEEE Trans.on MedicalImag., 5:1435–1447.
Collins, D. (1994). 3D Model-basedsegmentationof individual brain structuresfor magnetic
resonanceimagingdata. PhDthesis,MontrealNeurologicalInstitute.
Collins, D., Goualher, G. L., andEvans,A. (1998a). Non-linearCerebralRegistrationwith
SulcalConstraints.In MICCAI 1998, pages974–984,Cambridge,MA, USA.
Collins, D. L., Zijdenbos,A. P., Kollokian, V., Sled,J. G., Kabani,N. J., Holmes,C. J., and
Evans,A. C. (1998b).Designandconstructionof a realisticdigital brainphantom.IEEE
Transactionsin MedicalImaging, 17(3):463–468.
75
Cullen,M. andPurser, R. (1984).An extendedlagrangiantheoryof semigeostrophicfrontoge-
nesis.J. Atmos.Sci., 41:1477–1497.
Dacorogna,B. andMoser, J. (1990). On a partialdifferentialequationinvolving the jacobian
determinant.Ann.Inst.H. PoincareAnal.NonLineaire, 7:1–26.
Davatzikos,C. (1997). SpatialTransformationandRegistrationof Brain ImagesUsingElasti-
cally DeformableModels.Comp.Vis.andImageUnderstanding, SpecialIssueonMedical
Imaging, 66(2):207–222.
Dumoulin,C.,Souza,S.,Walker, M., andWagle,W. (1989).Three-dimensionalphasecontrast
angiography. MagneticResonancein Medicine, 9:139–149.
Edwards,P. J.,Hill, D. L. G., Little, J. A., andHawkes,D. J. (1997). Deformationfor Image
GuidedInterventionsUsinga ThreeComponentTissueModel. In IPMI’97, pages218–
231.
Ferrant,M., Cuisenaire,O.,andMacq,B. (1999a).Multi-ObjectSegmentationof BrainStruc-
turesin 3D MRI Using a ComputerizedAtlas. In SPIE Medical Imaging ’99, volume
3661-2,pages986–995.
Ferrant,M., Nabavi, A., Macq, B., andWarfield, S. K. (2000a). DeformableModeling for
CharacterizingBiomedicalShapeChanges.In Borgefors,G., Nystrom, I., andSannitidi
Baja,G., editors,DGCI2000: DiscreteGeometryfor ComputerImagery; 2000Dec 13–
15; Uppsala,Sweden, volume1953of LectureNotesin ComputerScience, pages235–248,
Heidelberg, Germany. Springer.
Ferrant,M., Warfield,S.K., Guttmann,C. R. G., Mulkern,R. V., Jolesz,F. A., andKikinis, R.
(1999b). 3D ImageMatchingUsinga Finite ElementBasedElasticDeformationModel.
In Taylor, C.andColchester, A., editors,MICCAI 99: SecondInternationalConferenceon
76
Medical Image Computingand Computer-AssistedIntervention;1999Sep19–22;Cam-
bridge, England, pages202–209,Heidelberg, Germany. Springer-Verlag.
Ferrant,M., Warfield,S.K., Nabavi, A., Macq,B., andKikinis, R. (2000b).Registrationof 3D
Intraoperative MR Imagesof theBrain Usinga Finite ElementBiomechanicalModel. In
DiGioia, A. M. andDelp, S., editors,MICCAI 2000: Third InternationalConferenceon
MedicalRobotics,ImagingAndComputerAssistedSurgery; 2000Oct11–14;Pittsburgh,
USA, pages19–28,Heidelberg, Germany. Springer-Verlag.
Gaens,T., Maes,F., Vandermeulen,D., andSuetens,P. (1998). Non-rigid Multimodal Image
RegistrationUsing Mutual Information. In Proceedingsof the First InternationalCon-
ferenceon Medical Image ComputingandComputer-AssistedIntervention(MICCAI’98,
Conferenceheldin Cambridge(MA),USA,October11–13,1998), volume1496of Lecture
Notesin ComputerScience, pages1099–1106.SpringerVerlag.
Gangbo,W. (1994).An elementaryproof of thepolarfactorizationof vector-valuedfunctions.
Arch. RationalMechanicsAnal., 128:381–399.
Gangbo,W. andMcCann,R. (1996). The geometryof optimal transportation.Acta Math.,
177:113–161.
Gee,J.andBajcsy, R. (1999).Brain Warping, chapterElasticMatching:ContinuumMechani-
calandProbabilisticAnalysis,pages193–198.AcademicPress.
Geiger, B. (1993).Threedimensionalmodelingof humanorgansandits applicationto diagno-
sisandsurgical planning.TechnicalReport2105,INRIA.
Gering,D., Nabavi, A., Kikinis, R., Grimson,W., Hata,N., Everett,P., Jolesz,F., andWells,
W. (1999).An IntegratedVisualizationSystemfor Surgical PlanningandGuidanceusing
ImageFusionand InterventionalImaging. In MICCAI 99: Proceedingsof the Second
77
InternationalConferenceon Medical Image ComputingandComputerAssistedInterven-
tion, pages809–819.SpringerVerlag.
Gering,D., Nabavi, A., Kikinis, R., Hata,N., O’Donnell, L., Grimson,W., Jolesz,F., Black,
P., andWells III, W. (2001). An integratedvisualizationsystemfor surgical planningand
guidanceusingimagefusionandanopenMR. J MagnResonImaging, 13(6):967–975.
Guimond,A., Roche,A., Ayache,N., andMeunier, J. (2001). Three-dimensionalmultimodal
brainwarpingusingthedemonsalgorithmandadaptiveintensitycorrections.IEEETrans-
actionsin MedicalImaging, 20(1):58–69.
Hagemann,A., Rohr, K., Stiel, H., Spetzger, U., and Gilsbach,J. (1999). Biomechanical
modelingof the humanheadfor physically based,non-rigid imageregistration. IEEE
TransactionsOn MedicalImaging, 18(10):875–884.
Hajnal,J. andBydder, G. (1997). AdvancedMR imaging techniques, chapterClinical usesof
diffusionweightedimaging.Martin Dunitz, ltd.
Haker, S. andTannenbaum,A. (2001a). Optimal transportandimageregistration. submitted
for publication,IEEE Trans.ImageProcessing.
Haker, S. andTannenbaum,A. (2001b). Optimal transportandimageregistration. In IEEE
workshopon variational andlevel setmethodsin computervision,ICCV2001.
Hata,N. (1998). Rigid anddeformablemedicalimage registration for image-guidedsurgery.
PhDthesis,Universityof Tokyo.
Hata,N., Nabavi, A., Wells,W. M., Warfield,S.K., Kikinis, R., Black,P. M., andJolesz,F. A.
(2000). Three-DimensionalOptical Flow Methodfor Measurementof VolumetricBrain
Deformationfrom IntraoperativeMR Images.J ComputAssistTomogr, 24(4):531–538.
78
Hata, N., Wells, W. M., Halle, M., Nakajima,S., Viola, P., Kikinis, R., and Jolesz,F. A.
(1996). ImageGuidedMicroscopicSurgery SystemUsing Mutual-InformationBased
Registration.In Visualizationin BiomedicalComputing, pages307–316.
Hellier, P. andBarillot, C. (2000). Multimodal non-rigidwarpingfor correctionof distortions
in functionalMRI. In Third InternationalConferenceon Medical Image ComputingAnd
Computer-AssistedIntervention(MICCAI’00), volume1935of LectureNotesin Computer
Science, pages512–520,Pittsburgh.SpringerVerlag.
Hermosillo,G., Chefd’Hotel,C., andFaugeras,O. (2001). A VariationalApproachto Multi-
Modal ImageMatching. TechnicalReport4117,Institut NationaldeRechercheenInfor-
matiqueet enAutomatique.
Hill, D., Maurer, C., Maciunas,R., Barwise,J., Fitzpatrick,J., andWang,M. (1998). Mea-
surementof intraoperative brainsurfacedeformationundera craniotomy. Neurosurgery,
43:514–526.
Horn, B. K. P. andSchunck,B. G. (1981). Determiningoptical flow. Artificial Intelligence,
17:185–203.
Jolesz,F. (1997).Image-guidedProceduresandtheOperatingRoomof theFuture.Radiology,
204:601–612.
Kantorovich, L. V. (1948).On aproblemof monge.UspekhiMat. Nauk., 3:225–226.
Kaus,M. R.,Nabavi, A., Mamisch,C.T., Wells,W. M., Jolesz,F. A., Kikinis, R.,andWarfield,
S. K. (2000). Simulationof CorticospinalTractDisplacementin Patientswith Brain Tu-
mors. In DiGioia, A. M. andDelp, S., editors,MICCAI 2000: Third InternationalCon-
ferenceon MedicalRobotics,Imaging AndComputerAssistedSurgery; 2000Oct 11–14;
Pittsburgh,USA, pages9–18,Heidelberg, Germany. Springer-Verlag.
79
Kaus,M. R.,Warfield,S.K., Nabavi, A., Chatzidakis,E.,Black,P. M., Jolesz,F. A., andKiki-
nis, R. (1999). Segmentationof MRI of meningiomasandlow gradegliomas. In Taylor,
C. andColchester, A., editors,MICCAI 99: SecondInternationalConferenceon Medi-
cal Image ComputingandComputer-AssistedIntervention;1999Sep19–22;Cambridge,
England, pages1–10,Heidelberg, Germany. Springer-Verlag.
Kikinis, R., Shenton,M. E., Gerig, G., Martin, J., Anderson,M., Metcalf, D., Guttmann,C.
R. G., McCarley, R. W., Lorenson,W. E., Cline,H., andJolesz,F. (1992).RoutineQuan-
titative Analysisof Brain andCerebrospinalFluid Spaceswith MR Imaging. Journal of
MagneticResonanceImaging, 2:619–629.
Kikinis, R., Shenton,M. E., Iosifescu,D. V., McCarley, R. W., Saiviroonporn,P., Hokama,
H. H., Robatino,A., Metcalf,D., Wible, C. G.,Portas,C. M., Donnino,R. M., andJolesz,
F. A. (1996).A Digital BrainAtlasfor SurgicalPlanning,ModelDrivenSegmentation,and
Teaching.IEEETransactionsonVisualizationandComputerGraphics, 2(3):232–241.
LeBihan,D., Breton,E.,Lallemand,D., Grenier, P., Cabanis,E.,andLaval-Jeantet,M. (1986).
MR imagingof intravoxel incoherentmotions:applicationsto diffusionandperfusionin
neurologicdisorders.Radiology, 161:401–407.
Lester, H., Arridge, S., Jansons,K., Lemieux,L., Hajnal, J., andOatridge,A. (1999). Non-
linear Registrationwith the Variable Viscosity fluid Algorithm. In IPMI 1999, pages
238–251.
Likar, B. andPernus,F. (2000). A HierarchicalApproachto ElasticRegistrationBasedon
Mutual Information. ImageandVisionComputing, 19(1-2):33–44.
Maes,F., Collignon,A., Vandermeulen,D., Marchal,G.,andSuetens,P. (1997).Multimodality
80
imageregistrationby maximizationof mutualinformation.IEEETransactionsin Medical
Imaging, 16(2):187–198.
Maintz,J.B. A., Meijering,E. H. W., andViergever, M. A. (1998).Generalmultimodalelastic
registrationbasedon mutual information. In Hanson,K. M., editor, Medical Imaging
1998: Image Processing(MI’98, Conferenceheld in SanDiego (CA),USA,Febuary 23–
26 1998), volume 3338 of SPIE Proceedings, pages144–154,Bellingham,WA, USA.
InternationSocietyfor OpticalEngineering.
Maintz,J.B. A. andViergever, M. A. (1998).A survey of medicalimageregistration.Medical
ImageAnalysis, 2(1):1–36.
McCann,R. J. (1997).A convexity principlefor interactinggases.Adv. Math., 128:153–179.
Meyer, C. R.,Boes,J.L., Kim, B., andBland,P. H. (1999).ProbabilisticBrainAtlasConstruc-
tion: Thin-PlateSplineWarpingvia Maximizationof Mutual Information. In Proceed-
ingsof theSecondInternationalConferenceonMedicalImageComputingandComputer-
AssistedIntervention(MICCAI’99, Conferenceheld in Cambridge, England,September
19–22,1999), volume1679of Lecture Notesin ComputerScience, pages631–637,Cam-
bridge,UK. SpringerVerlag.
Miga, M., Paulsen,K., Lemery, J.,Hartov, A., andRoberts,D. (1999).In vivo quantificationof
ahomogeneousbraindeformationmodelfor updatingpreoperativeimagesduringsurgery.
IEEE TransactionsOn MedicalImaging, 47(2):266–273.
Miga, M. I., Roberts,D. W., Kennedy, F. E., Platenik,L. A., Hartov, A., Lunn, K. E., and
Paulsen,K. D. (2001).Modelingof RetractionandResectionfor IntraoperativeUpdating
of Images.Neurosurgery, 49:75–85.
81
Moon, T. K. and Stirling, W. C. (2000). Mathematicalmethodsand algorithmsfor signal
processing. Prentice-Hall,NJ,USA.
Moser, J. (1965). On thevolumeelementson a manifold. Trans.Amer. Math.Soc., 120:286–
294.
Nabavi, A., Black, P., Gering,D., Westin,C., Mehta,V., Jr, R. P., Ferrant,M., Warfield, S.,
Hata,N., Schwartz,R.,3rd,W. W., Kikinis, R.,andJolesz,F. (2001).Serialintraoperative
magneticresonanceimagingof brainshift. Neurosurgery, 48(4):787–97.
Paulsen,K., Miga, M., Kennedy, F., Hoopes,P., Hartov, A., andRoberts,D. (1999). A Com-
putationalModel for TrackingSubsurfaceTissueDeformationDuring StereotacticNeu-
rosurgery. IEEE TransactionsOn MedicalImaging, 47(2):213–225.
Peled,S., Gudbjartsson,H., Westin, C.-F., Kikinis, R., andJolesz,F. A. (1998). Magnetic
resonanceimagingshows orientationandasymmetryof white matterfiber tracts. Brain
Research, 780:27–33.
Pennec,X., Cachier, P., andAyache,N. (1999). Understandingthe“demon’s algorithm”: 3D
non-rigid registrationby gradientdescent.In ?, editor, Proceedingsof theSecondInter-
national Conferenceon Medical Image Computingand Computer-AssistedIntervention
(MICCAI’99,Conferenceheldin Cambridge, England,September19–22,1999), volume?
of Lecture Notesin ComputerScience, page? Springer-Verlag.
Pierpaoli,C., Jezzard,P., Basser, P., Barnett,A., andChiro,G. D. (1996).Dif fusiontensorMR
imagingof thehumanbrain. Radiology, 201(3):637–648.
Poggio,T., Torre, V., andKoch, C. (1985). Computationalvision andregularizationtheory.
Nature, 317:314–319.
82
Poupon,C., Clark, C., Frouin,V., LeBihan,D., Bloch, I., andMangin,J.-F. (1999). Inferring
the brain connectivity from MR diffusion tensordata. In Wells, W. M., Colchester, A.,
andDelp, S.,editors,Lecture Notesin ComputerScience. Medical image computingand
computer-assistedintervention- MICCAI ’98, volume1679,pages453–462,Cambridge,
UK. Springer-Verlag.
Poupon,C., Mangin, J.-F., Frouin, V., Regis, J., Poupon,F., Pachot-Clouard,M., LeBihan,
D., andBloch, I. (1998). Regularizationof MR diffusion tensormapsfor trackingbrain
white matterbundles. In Wells, W. M., Colchester, A., and Delp, S., editors,Lecture
Notesin ComputerScience. Medicalimagecomputingandcomputer-assistedintervention
- MICCAI ’98, volume1496,pages489–498,Cambridge,MA, USA. Springer-Verlag.
Press,W., Teukolsky, S.,Vetterling,W., andFlannery, B. (1992). Numericalrecipesin C: The
art of scientificcomputing. CambridgeUniversityPress.
Provenzale,J.andSorensen,A. (1999). Dif fusion-weightedMR imagingin acutestroke: the-
oretical considerationsand clinical applications. AmericanJournal of Roentgenology,
173:1459–1467.
Rachev, S.andRuschendorf,L. (1998).MassTransportationProblems,VolsI andII . Springer,
New York.
Roche,A., Malandain,G., andAyache,N. (2000). Unifying maximumlikelihoodapproaches
in medicalimageregistration. InternationalJournal of Imaging SystemsandTechnology,
11:71–80.
Roche,A., Malandain,G., Pennec,X., and Ayache,N. (1998). The correlationratio as a
new similarity measurefor multimodal imageregistration. In Wells, W. M., Colchester,
A., andDelp, S., editors,Proceedingsof the First InternationalConferenceon Medical
83
Image ComputingandComputer-AssistedIntervention(MICCAI’98, Conferenceheld in
Cambridge(MA),USA,October11–13,1998), volume1496of LectureNotesin Computer
Science, pages1115–1124.Springer-Verlag.http://www.inria.fr/rrrt/rr-3378.html.
Rohr, K. (1997).Dif ferentialoperatorsfor detectingpoint landmarks.ImageandVision Com-
puting, 15:219–233.
Rousseeuw, P. J. andLeroy, A. M. (1987). Robust RegressionandOutlier Detection. Wiley
seriesin probabilityandmathematicalstatistics.JohnWiley & Sons.
Rueckert, D., Clarkson,M., Hill, D., and Hawkes, D. (2000). Non-rigid registrationusing
higher-ordermutualinformation.In MedicalImaging, volume3979of SPIEProceedings,
pages438–447,SanDiego. InternationSocietyfor OpticalEngineering.
Ruiz-Alzola,J.,Kikinis, R., andWestin,C.-F. (2001a).Detectionof point landmarksin multi-
dimensionaltensordata.SignalProcessing, 81(10):InPress.
Ruiz-Alzola,J.,Westin,C.-F., Warfield,S.K., Alberola,C.,Maier, S.,andKikinis, R. (2001b).
Nonrigid Registrationof 3D TensorMedicalData.MedicalImageAnalysis. In Press.
Ruiz-Alzola,J., Westin,C.-F., Warfield,S. K., Nabavi, A., andKikinis, R. (2000). Nonrigid
registrationof 3d scalar, vectorandtensormedicaldata. In Delp, S. L., DiGioia, A. M.,
andJaramaz,B., editors,Lecturenotesin computerscience. Medicalimagecomputingand
computer-assistedintervention.MICCAI ’00, volume1935,pages541–550,Pittsburgh,
PA, USA. Springer-Verlag.
Schroeder, W., Martin, K., andLorensen,B. (1996). TheVisualizationToolkit: An Object-
OrientedApproach to 3D Graphics. PrenticeHall PTR,New Jersey.
Segel,L. (1987).Mathematicsappliedto continuummechanics. Dover, New York, USA.
84
Sierra,R. (2001). Nonrigid Registrationof Dif fusionTensorImages.Master’s thesis,Zurich,
Switzerland:SwissFederalInstituteof Technology.
Simoncelli,E. P. (1994). Designof multi-dimensionalderivativefilters. In InternationalCon-
ferenceon ImageProcessing, Austin,USA. IEEE.
Skrinjar, O. andDuncan,J. S. (1999). Real time 3D brain shift compensation.In IPMI’99,
pages641–649.
Skrinjar, O.,Studholme,C.,Nabavi, A., andDuncan,J.(2001).StepsTowardaStereo-Camera-
GuidedBiomechanicalModel for Brain Shift Compensation.In Proceedingsof Interna-
tional Conferenceof InformationProcessingin MedicalImaging, pages183–189.
Stejskal,E. andTanner, J. (1965).SpinDiffusionMeasurements:SpinEchoesin thePresence
of aTime-dependentField Gradient.Journalof ChemicalPhysics, 42:288–292.
Thirion, J.-P. (1995). Fastnon-rigidmatchingof 3D medicalimages.TechnicalReport2547,
Institut National de Rechercheen Informatiqueet en Automatique,Sophia-Antipolis.
http://www.inria.fr/rrrt/rr-2547.html.
Thirion, J.-P. (1998). Imagematchingas a diffusion process:an analogywith Maxwell’s
demons.MedicalImageAnalysis, 2(3):243–260.
Toga,A. W. (1999).Brain Warping. AcademicPress.
Viola, P. andWells,W. M. (1995).Alignmentby maximizationof mutualinformation.In Fifth
Intl. Conf. ComputerVision (ICCV), pages16–23.IEEEPress.
Viola, P. andWells, W. M. (1997). Alignmentby maximizationof mutualinformation. Inter-
nationalJournal of ComputerVision, 24(2):137–154.
85
Wang,Y. andStaib,L. (2000). Physicalmodel-basednon-rigidregistrationincorporatingsta-
tistical shapeinformation.MedicalImageAnalysis, 4:7–20.
Warfield,S.K., Jolesz,F., andKikinis, R. (1998a).A High PerformanceComputingApproach
to theRegistrationof MedicalImagingData.Parallel Computing, 24(9–10):1345–1368.
Warfield, S. K., Jolesz,F. A., andKikinis, R. (1998b). Real-TimeImageSegmentationfor
Image-GuidedSurgery. In SC1998: High PerformanceNetworkingandComputingCon-
ference;1998Nov 7–13;Orlando,USA, pages1114:1–14.IEEE.
Warfield,S. K., Kaus,M., Jolesz,F. A., andKikinis, R. (2000a).Adaptive,TemplateModer-
ated,SpatiallyVaryingStatisticalClassification.MedImageAnal, 4(1):43–55.
Warfield, S. K., Mulkern, R. V., Winalski, C. S., Jolesz,F. A., andKikinis, R. (2000b). An
ImageProcessingStrategy for the QuantificationandVisualizationof ExerciseInduced
MuscleMRI SignalEnhancement.J MagnResonImaging, 11(5):525–531.
Weickert,J. (1997).Anisotropicdiffusionin imageprocessing. TeubnerVerlag,Stuttgart.
Weickert,J.,terHaarRomeny, B., andViergever, M. (1998).Efficientandreliableschemesfor
nonlineardiffusionfiltering. IEEETrans.on ImageProcessing, 7:398–410.
Weinstein,D. M., Kindlmann,G. L., andLundberg, E. C. (1999). Tensorlines:Advection-
diffusionbasedpropagationthroughdiffusion tensorfields. In Ebert,D., Gross,M., and
Hamann,B., editors,IEEEVisualization’99, pages249–254,SanFrancisco.
Wells, W., Kikinis, R., Grimson,W., andJolesz,F. (1996a). Adaptive segmentationof MRI
data.IEEE TransactionsonMedicalImaging, 15:429–442.
Wells,W. M., Viola,P., Atsumi,H., Nakajima,S.,andKikinis, R. (1996b).Multi-modalvolume
registrationby maximizationof mutualinformation.MedicalImageAnalysis, 1(1):35–51.
86
Westin, C.-F., Maier, S. E., Mamata,H., Nabavi, A., Jolesz,F. A., andKikinis, R. (2001).
ProcessingandVisualizationof Dif fusionTensorMRI. MedicalImageAnalysis. In Press.
Yezzi,A., Tsai,A., andWillsky, A. (2000).Medicalimagesegmentationviacoupledcurveevo-
lution equationswith globalconstraints.In MathematicalMethodsin BiomedicalImage
Analysis, pages12–19.
Zienkiewicz, O. C. andTaylor, R. L. (1994). TheFinite ElementMethod: BasicFormulation
andLinear Problems. McGraw Hill BookCo.,New York, 4th edition.
87