Real-Time Volume Deformations
9
EUROGRAPHICS 2001 / A. Chalmers and T.-M. Rhyne (Guest Editors) Volume 20 (2001), Number 3 Real-Time Volume Deformations Rüdiger Westermann and Christof Rezk-Salama Scientific Visualization and Imaging Group, University of Technology Aachen Computer Graphics Group, University of Erlangen Abstract Real-time free-form deformation tools are primarily based on surface or particle representations to allow for in- teractive modification and fast rendering of complex models. The efficient handling of volumetric representations, however, is still a challenge and has not yet been addressed sufficiently. Volumetric models, on the other hand, form an important class of representation in many applications. In this paper we present a novel approach to the real-time deformation of scalar volume data sets taking advantage of hardware supported 3D texture mapping. In a prototype implementation a modeling environment has been designed that allows for interactive manipulation of arbitrary parts of volumetric objects. In this way, any desired shape can be modeled and used subsequently in various applications. The underlying algorithms have wide applicability and can be exploited effectively for volume morphing and medical data processing. 1. Introduction and related work Over the last couple of years a number of algorithms have been developed to allow for interactive free-form deforma- tions of complex models 124183 . On the basis of these algo- rithms many tools have been designed that make free-form deformations easy to apply and thus enable efficient and flex- ible manipulation of the available models. The integration of these tools into commercial software products has lead to powerful modeling systems that allow the non-expert to efficiently design and modify any desired shape. The vast majority of these approaches, however, are re- stricted to the handling of polygonal models, but they do not consider volumetric objects although the underlying defor- mation deforms a region of three dimensional space. Objects in real world, on the other hand, are often solid and the in- terior has to be considered as well. Prominent examples are gaseous volumetric objects or fluids like clouds, fire and wa- ter, semi-transparent polygonal objects that are filled with voluminous material, or scalar volume data sets as they arise from measurements (e.g., CT, MRI imaging), computations (e.g., CFD, environmental science), and in industrial model- ing or arts. Despite the fact that todays modeling tools exclusively support polygonal representations we expect the same func- tionality to be available for volumetric representations in the near future. With the rapid progress that is currently made in the development of hardware accelerated volume render- ing algorithms 2 14 11 , the emphasis will be shifting towards advanced algorithms for volume editing and manipulation as they have been developed in surface graphics in the past. Our vision is that interactive deformations of volumetric models will become a key feature in modern modeling tools exhibit- ing the same functionality as it is already available for polyg- onal models. In this paper we address the problem of enabling inter- active, i.e., high speed, volume deformations since standard approaches to volumetric modeling using deformations are generally too slow to provide the modeling environment re- quired for productive work. Targeted user groups include modelers, artists, computational scientists, as well as indus- trial designers among many others. By means of the pre- sented method arbitrary local and global deformations can be applied to volumetric objects of reasonable size as de- picted in Figure 1. Available models can be manipulated and modified but also completely new objects might be created in an artistic manner. The nature of the proposed solution and their benefits are the following: we propose to use 3D texture mapping hard- ware coupled with backward distortion of 3D texture coor- dinates to achieve the desired deformation/modeling effects. c The Eurographics Association and Blackwell Publishers 2001. Published by Blackwell Publishers, 108 Cowley Road, Oxford OX4 1JF, UKand 350 Main Street, Malden, MA 02148, USA.
Transcript of Real-Time Volume Deformations
EUROGRAPHICS2001/ A. ChalmersandT.-M. Rhyne(GuestEditors)
Volume20 (2001), Number3
Real-Time Volume Deformations
RüdigerWestermannandChristofRezk-Salama
ScientificVisualization andImagingGroup,Universityof TechnologyAachenComputerGraphicsGroup,Universityof Erlangen
Abstract
Real-timefree-formdeformationtoolsare primarily basedon surfaceor particle representationsto allow for in-teractivemodificationandfastrenderingof complex models.Theefficienthandlingof volumetricrepresentations,however, is still a challenge and hasnot yet beenaddressedsufficiently. Volumetricmodels,on the other hand,form an importantclassof representationin manyapplications.In this paperwepresenta novel approach to thereal-timedeformationof scalarvolumedatasetstakingadvantage of hardware supported3D texture mapping. Ina prototypeimplementationa modelingenvironmenthasbeendesignedthat allowsfor interactivemanipulationof arbitrary parts of volumetricobjects.In this way, any desired shapecan be modeledand usedsubsequentlyin variousapplications.Theunderlyingalgorithmshavewide applicability and can be exploitedeffectivelyforvolumemorphingandmedicaldataprocessing.
1. Introduction and related work
Over the last coupleof yearsa numberof algorithmshavebeendevelopedto allow for interactive free-formdeforma-tionsof complex models12� 4 � 1� 8� 3. Onthebasisof thesealgo-rithmsmany toolshave beendesignedthat make free-formdeformationseasytoapplyandthusenableefficientandflex-ible manipulationof the available models.The integrationof thesetools into commercialsoftware productshasleadto powerful modelingsystemsthat allow the non-expert toefficiently designandmodify any desiredshape.
The vastmajority of theseapproaches,however, are re-strictedto thehandlingof polygonalmodels,but they donotconsidervolumetricobjectsalthoughthe underlyingdefor-mationdeformsaregionof threedimensionalspace.Objectsin realworld, on theotherhand,areoften solid andthe in-terior hasto beconsideredaswell. Prominentexamplesaregaseousvolumetricobjectsor fluidslikeclouds,fire andwa-ter, semi-transparentpolygonalobjectsthat are filled withvoluminousmaterial,or scalarvolumedatasetsasthey arisefrom measurements(e.g.,CT, MRI imaging),computations(e.g.,CFD,environmentalscience),andin industrialmodel-ing or arts.
Despitethe fact that todaysmodeling tools exclusivelysupportpolygonalrepresentationswe expectthesamefunc-tionality to beavailablefor volumetricrepresentationsin the
nearfuture.With the rapid progressthat is currentlymadein thedevelopmentof hardwareacceleratedvolumerender-ing algorithms2 � 14� 11, theemphasiswill beshifting towardsadvancedalgorithmsfor volumeeditingandmanipulationasthey havebeendevelopedin surfacegraphicsin thepast.Ourvision is that interactive deformationsof volumetricmodelswill becomeakey featurein modernmodelingtoolsexhibit-ing thesamefunctionalityasit isalreadyavailablefor polyg-onalmodels.
In this paperwe addressthe problemof enablinginter-active, i.e.,high speed,volumedeformationssincestandardapproachesto volumetricmodelingusingdeformationsaregenerallytoo slow to provide themodelingenvironmentre-quired for productive work. Targeteduser groupsincludemodelers,artists,computationalscientists,aswell asindus-trial designersamongmany others.By meansof the pre-sentedmethodarbitrarylocal andglobal deformationscanbe appliedto volumetric objectsof reasonablesize as de-pictedin Figure1.Availablemodelscanbemanipulatedandmodifiedbut alsocompletelynew objectsmight be createdin anartisticmanner.
Thenatureof theproposedsolutionandtheir benefitsarethe following: we proposeto use3D texturemappinghard-warecoupledwith backward distortionof 3D texture coor-dinatesto achieve thedesireddeformation/modelingeffects.
c�
TheEurographicsAssociationandBlackwellPublishers2001.Publishedby BlackwellPublishers,108Cowley Road,Oxford OX4 1JF, UK and350Main Street,Malden,MA02148,USA.
WestermannandRezk-Salama/ Real-TimeVolumeDeformations
The primary benefitis speed,assuminggraphicshardwarethatsupports3D texturemapping,suchasthatavailableonSGIIR/Octane,HPVisualWorkstation,theATI RadeonandIntense3DWildcat graphicsaccelerators.A key benefitofour approachis that the volumeitself is not deformed,butrather the mappinginto the volume. As we will show intheremainderof thispaper, thisallows usto selectarbitrarydeformationsby simply changingthemappingsduringren-deringwithout thatwe have to explicitly deformtheobjectitself. Consequently, no spacevarying re-sampling,whichmaycompromisetheoriginaldata,is required.
Figure 1: Localandglobaldeformationsareappliedto vol-umetricobjectsin real-time. A prototypemodelingenviron-mentallowsfor theefficientdesignandmanipulationof suchobjectson thecomputer.
Thenovelty of the approachis twofold: We leverage3Dtexture mappinghardware to do non-linearusercontrolleddeformationsof volumes,going significantly beyond pre-vious approacheswhich didn’t take advantageof hardwareacceleration7 or only allowedaffinedeformationsof theen-tire volumecubeor sometesselationinto a small numberof primitivesasproposedin 5� 13. We do so with a particu-lar usageparadigmin mind, which greatly facilitatessim-ple and intuitive control over the more intricate deforma-tions we provide an algorithmicbasisfor. In this context,we developeda prototypedeformationtool thatcanbeeas-ily extendedto more sophisticatedinteractionparadigms,e.g.mechanismsthatallow for free-handmanipulationslikesculptingor carvingin virtual realityenvironments.
Theremainderof thispaperis organizedasfollows.First,weintroducethebasicideaof separatingshapefrom appear-ancenecessaryto provide basisfor theefficient handlingofvolumetricobjects.We thenproposea renderingalgorithmfor volumetricmodelstakingadvantageof hardwareaccel-erated3D texturemapping.Finally, we demonstratehow toperformarbitraryreal-timedeformationsof volumetricrep-resentationsusingbackward distortionsof 3D texture coor-dinates.We concludethepaperwith ananalysisof thedif-ferentmodulesthathave beendevelopedin thiswork.
2. Object representation
We aim to constructa volumetricrepresentationthatallowstheuserto separatelyspecifytheshapeof theobjectanditsappearance.Thereforewe proceedin the samemannerasin surfacegraphics,whereshapeis usuallydefinedby a ge-ometric representation,while appearance,besidesotherat-tributes,is determinedby the2D texture that is assignedtotheobject.Throughouttheremainderof thispaperwewill befollowing this paradigm,but appearancewill now bespeci-fied in termsof a spacefilling 3D texture.
2.1. Shape modeling
Theadvantagesof a strict separationof shapefrom appear-ancehave alreadybeenexploited in thedesignof theVolu-mizer volumerenderingAPI 13. Here,the underlyingcon-ceptis to geometricallyrepresenttheobjectby an unstruc-turedgrid, causingatesselationusingvolumetricprimitives,i.e. tetrahedra,prismsetc.A 3D texturemapdefinestheap-pearanceof theobject,andby assigninga 3D texture coor-dinateto eachgrid point elementscannow bedisplayedbymeansof hardwareaccelerated3D texturemapping.
In general,however, this concepthas turned out to beratherimpractibledue to the following reason:as soonasmany primitives areneededto accuratelymodel the shapeof theobjecttherenderingperformanceslowsdown consid-erably. In particularthis is due to the overheadthat hastobespentin orderto computefor eachelementthesectionalpolygonsneededto renderthe elementon a slice-by-slicebasis.Ontheotherhand,whenever theshapeof theobjectisgoing to be modifiedthe 3D tesselationhasto be modifiedaswell. Particularlyif non-lineardeformationsareappliedtotheobjectprimitiveshaveto besplit andnew primitiveshaveto beinserted.For any objectthatexhibitsareasonablecom-plexity grid generationanddisplaycannotbeperformedin-teractively thusmakingtheapproachlesssuitedfor thekindof problemaddressedin thiswork.
Instead,we follow a differentpath:the shapeof thevol-umetricobjectis solelydefinedby thesurfaceenclosingtheobject.Without loss of generalitywe assumethat the sur-faceis representedby a trianglemesh.As a matterof fact,during the modelingphasethe interior of the object is ex-clusively definedin termsof appearance.We will show thata geometricalrepresentationof the interior is not necessaryin general,andthatdeformationscanbeappliedmuchmoreefficiently duringtherenderingphase.
2.2. Appearance modeling
In the currentapproach,the appearanceof any volumetricobjectis modeledby meansof a3D texturemapthatentirelydefinestheinteriorof thespecifiedshape.A simpleinterfaceallows for theassignmentof appearanceto shape.Oncethegeometryhasbeenmodeledvertex positionsareautomati-cally scaledto therangeof � 0 � 1� andtheobjectis displayed
c�
TheEurographicsAssociationandBlackwellPublishers2001.
WestermannandRezk-Salama/ Real-TimeVolumeDeformations
within thetexturedunit cubeasillustratedon theleft imagein Figure2. Sincevertex positionsnow directly correspondto the relative positionof verticesin the unit cubethey areissuedas3D texturecoordinates.Thusthepartof thetexturethatis coveredby theobjectis uniquelydetermined.
Figure 2: A simpleinterfaceallowsoneto arbitrarily posi-tion the geometrythat definesthe objectsshapewithin thetexture cube. On theright, thevolumetricobjectis renderedusingour approach asoutlinedbelow.
Theusercanarbitrarilyscale,translateor rotatetheobjectwithin thetexturecubein orderto determinethepartof thetexturethatshouldbeusedto definetheobjectsappearance.Texturecoordinatesareupdatedcorrespondinglyandfinallydeterminethat part of the texture that shouldbe enclosedby the object. In a differentwindow the volumetricobjectis renderedsimultaneouslythusallowing theuserto inspectthechangesassoonasthey areissued.
2.3. Rendering volumetric objects via 3D textures
Volumerenderingvia 3D texturemapshasbecomeapower-ful tool to interactively displayandthusanalyzescalardatafields2. Interpretingvolumerenderingasthere-samplingofadiscrete3D texturemaponappropriatelyorientedsocalledcutting geometrieslike planesor spheresallows oneto ex-ploit hardwaresupportedtextureinterpolationandper-pixelblendingto simulatetheappearanceof semi-transparentme-dia.
Most commonlythetexture is re-sampledon planespar-allel to the viewing planein back-to-frontorder. The datato bestoredin the texture mapis assumedto bedefinedona Cartesiangrid, andconsequentlyonly the sectionalpoly-gonsbetweeneachplaneandthe boundingbox of the gridhave to becomputedasillustratedin Figure3. Thetexturedpolygonsarethenrenderedinto theframebufferandblendedwith thepreviousresults.
Weproceedby recognizingthatthealgorithmalreadyre-liesonthestrictseparationof shapefrom appearanceaspro-posed.Theshapeof theobjectsis determinedby thepolyg-
Figure 3: Volumerenderingby3D texture slicing.
onal representationof the boundingbox while its appear-anceis givenby thetexturemap.Now theadvantageof theproposedmodelrepresentationimmediatelybecomesclear:thebasicvolumerenderingalgorithmvia 3D texturescanbeperformedusingany desiredshapeby clipping every planewith thepolygonalrepresentation.No matterwhetherthis isa simpleboundingbox or a morecomplex geometry, onceauniquedatastructurehasbeendefinedthatstoresthegeom-etry thealgorithmproceedsin exactly thesameway.
2.4. Clipping and Tesselation
In order to efficiently computethe sectionalpolygonsbe-tweeneachcuttingplaneandthetrianglemeshourapproachproceedssimilarly to theoneproposedin 15. We exploit anactive edgelist datastructurethatconsistsof a vertex list, atrianglelist, anedgelist andanactive edgelist. In thevertexlist eachvertex positionis storedasa3D coordinateaccom-paniedby a 3D texture coordinate.In the trianglelist eachelementis representedby threereferencesinto thevertex listandinto theedgelist, respectively. Elementsin theedgeliststorelinks to the verticesdefiningthat edgeaswell astwoadditionalreferencesinto thetrianglelist thatspecifythetwofacesadjacentto thatedge.
Once an arbitrary intersectionpoint betweena cuttingplaneandanedgeof thetrianglemeshhasbeendetermined,the edgebaseddata structureallows one to quickly findthe next elementandthe edgesthathave to be checked forfurther intersectionswith the currentplane.A simplewalkacrossthemeshis performedby successively following thereferencesuntil anelementis hit that hasalreadybeenvis-ited.As theobjectis consideredto beclosedweendupwithaclosedbut maybeconcave polygonthathasto berendered(seeFigure 4). It is worth sayingthat at eachintersectionpoint theappropriatetexturecoordinatehasto becomputedby linearinterpolationaswell.
In additionto the aforementioneddatastructureswe uti-lize anactiveedgedatastructurein orderto avoid processingtheentireedgelist to find any edgethatintersectsthecurrentplane.Therefore,for eachnew view 3D spaceis partitionedinto slabsof constantwidth thatareorientedparallelto theview plane.For eachedgethefirst andthelastslabit over-lapsaredetermined,andfor both of thema pointerto thisedgeis stored.Precedingfrom backto front wedynamically
c�
TheEurographicsAssociationandBlackwellPublishers2001.
WestermannandRezk-Salama/ Real-TimeVolumeDeformations
updatetheactive edgelist thatkeepsreferencesto all edgesthatmayhave anintersectionwith thecurrentplane.When-everanew slabis enteredtheactive edgelist is updatedwithrespectto thereferencesthatarestoredfor this slab.
Dependingon theobjects’geometricrepresentationmul-tiple sectionalcontoursmay arise in eachcutting plane.In order to avoid processingelementsrepeatedlythey aretaggedonce they have been intersectedwith the currentplane.Then the entire set of contoursis retrieved by suc-cessively checkingthoseentriesin the active edgelist thathave not yet beentagged.In addition,every contourhastobeclassifiedin termsof whetherit is enclosedby any otherone.In thiscasewehave to takecarethatonly theregionbe-tweentheoutercontourandtheinnercontoursis rendered.
We proceedby recognizingthat the generatedsectionalcontoursaremost likely to be concave andthuscannotberenderedusingthe coreOpenGLrenderingprimitives.Weattack this problem by appropriatelytesselatingthosere-gionsin thecurrentslicingplanethatcover theobjects’inte-rior. Thereforewe exploit theOpenGLutility library whichprovidesadditionalfunctionalityfor theautomaticcomputa-tion of trapezoidaldecompositionsof concave polygonsin-cludingmultiple holes.
Figure 4: Differentsectionalcontours and thecorrespond-ing tesselationsare shown.On the left a sphere wasglob-ally squeezedand locally stretched.On theright theobjectwaslocally squeezedat threedifferentlocationsyieldingoneouterandthreeinnercontours thathavebeentesselatedap-propriately.
Eachtime a contourandarbitrarily many inner contourshavebeencomputedthey arehandledto thetesselatorwhichconstructsasetof trianglesthatcanbedirectlyretrievedandrenderedthroughOpenGL.In Figure 4 the computedtes-selationsfor two differentcontoursareshown. In casethatmultiple contourshave to be rendereda simple in-out testyieldstheappropriatesetsto betessellated.
3. Object deformation
Thedifficulties thatarisewhendeformationsareappliedtovolumetricobjectsthatarerenderedvia 3D texturesaretwo-fold. First,free-formdeformationtoolsusuallydon’t supportthe possibility to easily retrieve the changein the positionof an arbitrarypoint in the interior of the object.Second,
it is not obvious how to renderthedeformedobjectvia 3Dtextures.At first glancethis involves updatingthe texturemapwith respectto theapplieddeformations.This,however,canhardlybedonein real-time.
In order to accountfor both problemswe methodicallyandalgorithmicallysplit thedeformationprocedureinto twodistinctparts:themodificationof shapeandthedeformationof theobjects’interior.
3.1. Shape deformation
In our prototypeimplementationshapemanipulationsareissuedby deformingthe trianglemeshusingany availablefree-form deformationmethod.Vertex positionsare trans-formed in 3D space,but texture coordinatesremain un-changedthuspropagatingthe deformationinto the interiorof theobject.
Unfortunatelythis approachleadsto inconsistentresultsbecausethe tesselationof the sectionalpolygonsand thusthegenerationof texturecoordinatesin theinteriorof theob-jectby linearinterpolationis view dependentasillustratedinFigure5. As a matterof fact,distortionsof theinterior thatresult from object deformationscannotbe dealt with cor-rectly. However, aswill bedescribedin thenext section,ourproposeddeformationschemeallows for the integrationofthisparticularkind of deformationsstraightforwardly.
Figure 5: Thesamedeformedshapeis renderedfromdiffer-entviews.Dueto thechosentesselationdeformationsmightnot beappliedconsistently.
3.2. Appearance deformation
In orderto simulatedeformationsof theobjectsinterior wedevelopedanapproachthatis well suitedfor volumetricob-jectsdisplayedby meansof 3D texture maps.In particularthe usersuppliestwo parametersin order to specifya de-formation:a point � 0 � 1� 3 texture spacein the interior oftheobjectandadisplacementvectorthatindicatesthedirec-tion and the distancethis point shouldbe moved into this
c
TheEurographicsAssociationandBlackwellPublishers2001.
WestermannandRezk-Salama/ Real-TimeVolumeDeformations
direction. A third parameterthat is hard-codedin the de-formationtool determinesthe extent of the deformationasoutlinedin Figure6. Thedisplacementvectorandtheextentof thedisplacemententirelydefinea so-calleddisplacementvolumewith an underlyingcoordinatesystem.Every pointcontainedin this volumewill bedisplacedinto thespecifieddirection.ThedisplacementvalueV � u � v� w � for any point iscomputedasatensorproductof 1D displacementfunctions,B, asfollows:
V � u � v� w ��� Bu � u��� Bv � v��� Bw � w�
Bu and Bv are quadraticB-Splinescenteredat the dis-placedpositionwith a supportthat coversthe extent of thedeformation.In the lower third of the deformationvolumeBw is definedsimilarly. In theupperpart,however, it is de-finedasaquadraticB-Splinewith twice thesupport.By thisparticularchoicewe mimic elasticmaterialthat is squeezeddue to the deformationinto a certaindirection. FunctionsBu � u��� Bv � v� andBw � w � arepre-computedandstoredin anappropriatedatastructurein orderto minimize the compu-tationaloverheadnecessaryto retrieve thedisplacementval-ues.Sinceateachboundaryfaceof thedeformationvolumethe displacementvaluesarezeroa continuousrangein theinterior is guaranteedevenif deformationsareapplied.Fig-ure6 shows a sectionaldrawing of thedeformationvolumeandexemplifiesthedistributionof displacementvalues.
� � � � � � �
� � � � � � �
�
� � � � � ! " # " $ %� � & � $ %� " ' & ( # " �
� � � � � ! " # " $ %
)� � � � � ! " # " $ % ! & & ( � � $ % " � * � % " #+ ,
, " ! % & (
" - % " $ %
Figure 6: Theleft imageshowsthedisplacementvolumeandtheunderlyingcoordinatesystem.In thenext image a sliceout of thedisplacementvolumeis drawn.Displacementval-uesare linearly mappedto intensity.
Fromthepositionandorientationof thedisplacementvol-umethe � 0 � 1� 3 texture coordinateof any point in the dis-placementvolumecanbedirectly computed.Thedeforma-tion is then applied by shifting the coordinateabout thenegative displacementvalue along the deformationdirec-tion thusperforminga backward distortionof texture coor-dinates.This approachis quite similar to the oneproposedin 7, wheresocalledraydeflectorswhereusedto distort thedirectionof rayspassingthroughthesupportof thesedeflec-tors.
Figure 7: Ontheleft, thelocally deformedvolumeis shown.On theright, thewireframerepresentationof tesselationsineach deformationvolumeis rendered.
3.2.1. Rendering
Prior to processingthe currentslicing planewe first checkfor any possibleintersectionof this planewith any of theactivedeformationvolumes.If sotheslicingplaneis clippedagainsttheboundingbox of thedeformationvolume.Local� u � v� w� coordinatesarecomputedateachintersectionpointby linearinterpolationof coordinatesissuedatthecornersoftheboundingbox. Basedon thepositionof the intersectionpoints,on the otherhand,we candirectly computetexturecoordinateswith respectto the3D texturemapto which thedeformationis applied.
We proceedby converting the convex sectionalpolygoninto anumberof triangleswhicharerefinedfurtheron.Eachtriangleis subdividedinto four new trianglesby connectingtheedgemidpoints.Eachnew vertex getsassignedtheinter-polatedcoordinateswith respectto thelocaldeformationco-ordinatesystemandtheunderlyingtexturespace.Theproce-dureis stoppedif a userdefinedsubdivision depthhasbeenreachedor if the sizeof thegeneratedtrianglesfalls belowa thresholdthat is specifiedwith respectto pixel size.Lo-cal deformationcoordinatesareusedto index into thearraystoringthe pre-computeddeformationvalues.Thesevaluesarethenusedto displacetexturecoordinatesinto theinversedirectionof the deformation.In this way non-lineardefor-mationscan be modeledby an adaptive piece-wiselinearapproximation.
Oneof the nice featuresof this approachis that the ac-curacy of theapproximationcanbeinteractively controlled.During movements,for instance,trianglesarenot going tobesubdivided,but in astill pictureamoreaccuraterepresen-tationshouldbepreferred.Notehowever, thatevenwithouttheadaptive tesselationthedeformationwill alreadybedis-playeddueto thegenerationof clip contourswith theregionin which thedeformationtakesplace.
Now that the generalmethodfor simulatingappearancedeformationshasbeensetup, we have to considerthe ren-deringof suchdeformations.Let usshift emphasisto thefactthat for eachcuttingplanemultiple sectionalcontoursmaybe generated.Someof themarisefrom clipping the slicingplanewith the trianglemeshthat definesthe object,others
c.
TheEurographicsAssociationandBlackwellPublishers2001.
WestermannandRezk-Salama/ Real-TimeVolumeDeformations
resultfrom clippingtheplanewith thedeformationvolumes.Finally, however, all tesselatedcontourshave to berenderedin theright order.
Figure 8: On theleft each contouris tesselatedseparately.On theright tesselationis performedagainsttheoutercon-tours andtheinnercontours.
From Figure8 we observe that by separatelytesselatingevery contourwe endup with many overlappingtriangles.Renderingthesetrianglesleadsto wrong resultsandin ad-dition many fragmentsaregoing to be rasterizedtwice. Inorderto overcomethis problemwe employ thetesselatortoconstructasetof trianglesthatcoverstheregionin eachslicethat is not affectedby the deformation.Thesectionalpoly-gonsbetweenthecuttingplaneandthedeformationvolumesareissuedasinnercontoursandwe tesselatethesecontoursagainstthe crosssectionof the trianglemeshasillustrated.Now the set of trianglesexactly covers the interior of theobjectandcanbedirectly renderedin any order.
Let us concludethis sectionby mentioningthat the pre-sentedapproachcanbeusedto accountfor shapedeforma-tionsaswell. All we needto know is theregion in 3D spacethatis affectedby thedeformationof thetrianglemesh.Eachtrianglethat is suppliedby thetesselatoris recursively sub-dividedif it hasacrosssectionwith thedeformationvolume.Thedeformationof themeshis propagatedinto theobjectsinteriorby meansof thedisplacementof texturecoordinateswith respectto thedisplacementvalues.
3.3. Deformed iso-surfaces
The 3D texture basedvolume renderingtechniqueas de-scribedin Section2 canalsobeusedto displaylightediso-surfaces14. This is doneby re-sampling3D gradientmapsthat storethe pre-scaledgradientsandthe scalardatasam-plesin aRGBandα texture,respectively.
Thecommonprocedureemployed in ray tracingfor iso-surfacerendering,wherethe ray is traceduntil the first in-tersectionwith the surfaceis found,canbe simulatedeffi-cientlyby meansof OpenGLper-fragmentoperations.Com-bining alpha-anddepth-testduring re-samplingguaranteesthatonly thosetexturesamplesclosestto theviewpoint andabove/below a user-defined thresholdare drawn into theframebuffer. Per-pixel diffuselighting is accomplishedbymultiplying the RGBα components,which now store the
gradientvector, with acolormatrix 9 asavailableonSGI IRandOctanesystems.This matrix hasto be initialized prop-erly to performscaling,modelview rotationand the scalarproductcalculationwith thelight sourcedirectionvector.
input registersRGB A
slice (i +1) 3D texture
intensitygradient
color of diffuse light
direction of light const color 2
color of ambient light
primary color
second color
generalcombiner 0
A
B
C
D
dotproduct
A ● B
output registerfinalcombiner
RGB A
fragment
A
B
C
A B +(1−A) C
+ D
D
E F
ZERO
Figure 9: The register combinersetupfor non-polygonalshadediso-surfaceswith 3D texturesis illustrated.
As proposedin 11 this approachcanbeimprovedconsid-erablyby exploiting the functionalityof currentPC graph-ics hardware, i.e. the Nvidia GeForce family GPUs. Onthesechipsprogrammableper-fragmentarithmeticis avail-ableduring rasterization6, that allows to computecomplexcombinationsbetweenthecolor of incomingfragmentsandtexture samplesin a single renderingpass.The hardwareis capableof simultaneouslyperforming component-wiseproductsand dot-productcalculationsbetweenthe frag-ment color, multiple texture samplesor user-definedcon-stantRGB values.Since3D texturesarenot yet supportedby NVidia’s GeForcearchitecture,trilinear interpolationisperformedby meansof multi-textures.However, with hard-waresupportfor 3D textures,which will becomeavailableon the next generationGPUs,non-polygonalshadediso-surfacescanberenderedin asinglepassasoutlinedin figure9. Thematerialgradientis storedin theRGB portionof the3D texture and in one of the generalcombinerstagesthedotproductbetweenthegradientandthedirectionof light iscomputed.In thefinal combinertheresultis multipliedwiththe color of the diffuselight sourceandanambientlight isadded.
Usingtheseapproachesfor thedisplayof deformednon-polygonaliso-surfaces,however, the resultsbecomeincor-rect as long as the original gradientscomputedfrom thenon-deformedvolumeareusedfor illumination. In ordertocircumvent this problemwe borrow an idea that was firstproposedin 10 for renderingbump-mappedsurfacesandex-ploited further on for the gradient-lessrenderingof shadediso-surfacesvia 3D textures in 14/ 13. The diffuse lightingcomponentCdiff canbesimulatedby thedirectionalderiva-tive of thescalarmaterialX with respectto thedirectionoflight 0L. This is doneby computingforward differencesto-wardsthelight sourcein thescalarfield:
Cdiff 1 ∂X∂L 2 X 3 0p0 4�5 X 3 0p0 687:9 0L 4
c;
TheEurographicsAssociationandBlackwellPublishers2001.
WestermannandRezk-Salama/ Real-TimeVolumeDeformations
The procedurecan be accomplishedby a two-passren-dering approach.Eachslice is renderedtwice into an in-termediatebuffer, but thelatter time texturecoordinatesareslightly shiftedtowardthelight sourceandthefragmentval-uesare subtractedfrom the previous results.The forwarddifferencesarethencopiedinto thecolor buffer (seeFigure12).
The samealgorithm can be performedmuchmore effi-cientlyusing3Dmulti-texturesandper-fragmentarithmeticsasproposed.At eachvertex two texture coordinatesareis-suedthat mapinto the samescalar3D texture.The secondcoordinate,however, is shiftedtowardthelight sourceposi-tion. Figure9 givesan outline of an implementationusingregistercombiners.In thegeneralcombinertexturesamplesaresubtractedfrom eachotherand the result is multipliedwith thecolorof thelight source.Ambientlight canalsobeaddedat thefinal combinerstage.Sinceforwarddifferencesarenow computedbasedon thedeformeddata,thelightingcalculationnow accountsfor thedeformationof gradientsaswell.
ı
input registersRGB A
slice (i +1) 3D texture
color of diffuse light
direction of light const color 2
color of ambient light
primary color
second color
generalcombiner 0 output registerfinal
combiner
RGB A
fragment
A
B
C
A B +(1−A) C
+ D
D
E F
ZERO
slice (i +1) 3D texture (shifted) INVERT
A
B
C
D
sum
A + B
Figure 10: Theregister combinersetupfor non-polygonalshadediso-surfacesapproximatingdirectional derivativestowardsthelight source.
4. Analysis
In this sectionwe analyzethemainmodulesandfeaturesofour system.All testswererun on a SGI OctaneV8 systemequippedwith one R12000,400 MHz processor, 128 MBtexturememoryand256MB mainmemory.
In general,theperformancestronglydependsonthecom-plexity of thegeometryusedto definetheshapeof the ob-ject and on the numberof issueddeformationsas well astheir support.In practicalapplications,however, we noticedthat it sufficesto useonly coarsegeometriesand to applythe deformationsto the interior of the objectratherthantoits shape.Our experimentshave shown that objects- with-out deformations- consistingof approximately4-6K trian-glescanbeclippedagainst200cuttingplanesandtesselatedwhile still achieving frameratesof 10-15fps.This timealsoincludesthetimeneededto updatetheedgebaseddatastruc-tureandto computetheintersectionpointsandtheappropri-atetexturecoordinates.Althoughthis numberis far beyond
thenumberof trianglesonewould typically selectit showsthat even very complex shapescan be chosenwithout de-gradingtheperformancesignificantly.
In casethatdeformationsareappliedtheoverheadthat isintroducedmight be considerably. Even thoughonly an in-significantamountof time is neededto computethe crosssectionsof thedeformationvolumesandtheappropriatede-compositioninto triangles,the tesselationof theoutercon-touragainstmultiple innercontoursandtherecursivesubdi-visioneventuallyslow downtheperformancetosomeextent.
The differencecanbe observed very preciselyin Figure12wherethesamedatasetwasrenderedusingdifferentsub-divisiondepths.Theenclosingtrianglemeshconsistsof 112triangles.The imageon the left wasgeneratedwith 8 fps,while ontheright wecouldonly achieve 3 fps.This is duetothe increasingamountof trianglesto begeneratedandren-dered(from 6K to 96K) andthe simultaneouslyincreasingamountof numericaloperationsto beperformedin ordertointerpolatevertex andtexturecoordinates.Notehoweverthattherenderingtimedoesnotdependlinearlyonthenumberofrenderedtriangles.In thisparticularexampleapproximately90%of thetimeneededto rendertheleft imagewasusedbythetesselator. Sincewe only performthesubdivisionwithineachdeformationvolumethis amountremainsconstantfortherenderingof theright image.
Renderingthe enginedatasetshown in Figure1, on theother hand,was performedwith 2 fps in contrastto 8 fpswithout deformations.The polygoncountraisedfrom 16Kto 156K. In this examplean insignificantamountof timewasneededfor tesselationbecauseonly oneglobaldeforma-tion wasapplied.But againwerecognizethatahugeportionof theoverall time is neededfor othertasks,i.e. 3D texturemapping.
Finally let us note that in all our examplesperformancecouldbe improved furtheron by consideringthepixel-areabasedstoppingcriterionfor recursive subdivision. This hasnot yetbeenintegratedinto thisapproach.
5. Conclusion and future work
In this paper we have emphasizeda novel approachtoachieve real-timevolumedeformationsvia forward distor-tion of 3D texture coordinatesthroughstandardAPIs likeOpenGL.Onecontribution hereis that we strictly separatethe deformationof shapefrom the deformationof appear-ance.We do this in a more rigorousmanneras proposedin the OpenGLVolumizerAPI. In particularwe reducetheproblemof modelingand renderingthe adaptively refinedinterior of a volumetricobjectto a 2D problemwithin eachslicingplane.Althoughwe expecttheoverall numberof tri-anglesgeneratedandrenderedby ourapproachto besimilarto the numberthat is generatedbasedon a tetrahedraliza-tion webelieve thatourapproachsimplifiesthecomputation
c<
TheEurographicsAssociationandBlackwellPublishers2001.
WestermannandRezk-Salama/ Real-TimeVolumeDeformations
and allows for arbitraryon-the-fly refinementsand coarsi-fications.In particularthis allows us to adaptively considerdeformationsduringtherenderingphasethusdecouplingthecomplexity of therenderingalgorithmfrom objectcomplex-ity.
Our resultshave shown that thepresentedmethodis sig-nificantly faster than other methodspreviously proposedwithout introducingany imagedegradations.The usercanflexibly selectthe desiredaccuracy with which non-lineardeformationsshouldbe approximatedin the interior of theobject.Sincethedeformedvolumetricobjectis re-sampledduringrenderingit canbereadslice-by-slicefrom thecolorbuffer andrestoredin a 3D texturemap.In this way there-sult canbe usedfurther on in any otherapplication.How-ever, someaspectof ourapproachhave to beevaluatedmorecarefully:= Multiple overlappingdeformationshave not yet beenad-
dressed.In thiscontext wearethinkingaboutstrategiestoefficiently mergemultiple deformationvolumes.= So far, non-polygonalshadediso-surfacescan only berenderedusingtheproposedtwo-passapproachthatsim-ulatesdiffuseillumination.This is dueto the reasonthathardwaresupported3D texturesarenot yet availableontheNvidia GeForce2 chipset.= Our approachhasto beaccompaniedby a moresophisti-catedfree-formdeformationtool allowing for theintuitivedesignof arbitraryshapes.Wearecurrentlytrying to inte-grateour algorithminto a virtual realityenvironmentthatenablesfree-handmodelingof volumetricobjects.
Nevertheless,we areconvincedthat the ideaswe presentedwill beinfluentialfor futuredevelopments:= We have proven that real-timevolumedeformationsare
nolongeradreamandcanbeperformedoncurrentgraph-icsarchitectures.= We have demonstratedthatobjectmodelingasit wasex-clusively existing in surfacesgraphicsuntil now canalsobeappliedto volumetricobjects.= We areconvincedthat the outlinedalgorithmwill be in-fluential for medical applicationslike volume registra-tion. Oncea matchingfunctionbetweentwo objectshasbeencomputedit canbe expressedin termsof displace-mentvolumes.Theiterativeprocessthatconsecutively re-samplesthedeformedvolume,re-computesthematchingfunction andthe resultingdeformationvolumesyields apowerful tool for volumeregistrationandclassification.
References
1. D. Bechmann.SpaceDeformationModelsSurvey. ComputersandGraphics, 1994.
2. B. Cabral,N. Cam,andJ.Foran.AcceleratedVolumeRender-ing andTomographicReconstructionUsingTextureMappingHardware.In ProceedingsACM Symposiumon VolumeVisu-alization94, pages91–98,1994.
3. C. ChuaandU. Neumann.Hardware-AcceleratedFree-FormDeformations.In ProceedingsSIGGRAPH/EUROGRAPHICSWorkshoponGraphicsHardware 2000, pages33–39,2000.
4. S.Coquillart. ExtendedFree-Form Deformations:aSculptur-ing Tool for 3D GeometricModeling. In ComputerGraphics(SIGGRAPH90Proceedings), pages187–196,1990.
5. S. Fang,S. Rajagopalan,S. Huang,and R. Raghavan. De-formableVolumeRenderingby 3D TextureMappingandOc-treeEncoding.In Visualization ’96, 1996.
6. D. Kirk. FromMultitextureto RegisterCombinersto Per-PixelShading.http://www.nvidia.com/Developer.
7. Y. KurzionandR. Yagel. SpaceDeformationwith HardwareAssistance.IEEETransactionsonVisualization andGraphics,18(4):571–586, 1997.
8. R. MaxCrackenand K. Joy. Free-Form DeformationswithLatticesof Arbitrary Topology. In ComputerGraphics(SIG-GRAPH96Proceedings), pages181–188,1996.
9. T. McReynolds.Tutorial onProgrammingwith OpenGL:Ad-vancedRendering.In SIGGRAPH96, 1996.
10. M. Peercy, J. Airy, andB. Cabral. Efficient BumpMappingHardware.ComputerGraphics,Proc. SIGGRAPH’97, pages303–307,July1997.
11. C. Rezk-Salama,K. Engel,M. Bauer, G. Greiner, and Ertl.T. Interactive Volume Renderingon StandardPC Graph-ics HardwareUsing Multi-TexturesAnd Multi-StageRaster-ization. In SIGGraph/Eurographics Workshopon GraphicsHardware, 2000.
12. T. Sederberg andS. Parry. Free-Form Deformationof SolidGeometricModels. In ComputerGraphics(SIGGRAPH86Proceedings), pages151–160,1986.
13. SiliconGraphicsInc. TheVolumizerAPI.
14. R. WestermannandT. Ertl. Efficiently usingGraphicsHard-warein VolumeRenderingApplications. In ComputerGraph-ics (SIGGRAPH98Proceedings), pages291–294,1998.
15. R Westermann,O. Sommer, andT. Ertl. DecouplingPolygonRenderingfrom GeometryusingRasterizationHardware. InProceedings10th Eurographics Workshopon Rendering99,pages81–89,1999.
c>
TheEurographicsAssociationandBlackwellPublishers2001.
Figure 11: A differentapplicationof our proposedalgorithmis demonstrated:a coarsegeometryis usedto extract thebrainstructure froma 3D MRI dataset.Arbitrary clippinggeometriescanbeexploitedto interactivelysegmentmedicaldatasets.
Figure 12: On theleft, trianglesusedto renderthedeformationswere subdividedonly once. On theright, 4 subdivisionstepswereperformed.
Figure 13: Bothimagesshowthedeformedheaddataset.Ontheleft it is directlyrenderedandontheright diffuseilluminationis simulatedby forward differencing.
