A model-based method for the reconstruction of total knee replacement kinematics

IEEE TRANSACTIONS ON MEDICAL IMAGING, VOL. 18, NO. 10, OCTOBER 1999 981

A Model-Based Method for the Reconstructionof Total Knee Replacement KinematicsSilvia Zuffi, Alberto Leardini,* Fabio Catani, Silvia Fantozzi, and Angelo Cappello

Abstract— A better knowledge of the kinematics behaviorof total knee replacement (TKR) during activity still remainsa crucial issue to validate innovative prosthesis designs anddifferent surgical strategies. Tools for more accurate measure-ment of in vivo kinematics of knee prosthesis components aretherefore fundamental to improve the clinical outcome of kneereplacement. In the present study, a novel model-based methodfor the estimation of the three-dimensional (3-D) position andorientation (pose) of both the femoral and tibial knee prosthe-sis components during activity is presented. The knowledge ofthe 3-D geometry of the components and a single plane projectionview in a fluoroscopic image are sufficient to reconstruct theabsolute and relative pose of the components in space. Thetechnique is based on the best alignment of the component designswith the corresponding projection on the image plane. The imagegeneration process is modeled and an iterative procedure localizesthe spatial pose of the object by minimizing the Euclideandistance of the projection rays from the object surface. Com-puter simulation and static/dynamic in vitro tests using real kneeprosthesis show that the accuracy with which relative orientationand position of the components can be estimated is better than1.5� and 1.5 mm, respectively.In vivo tests demonstrate that themethod is well suited for kinematics analysis on TKR patientsand that good quality images can be obtained with a carefullypositioning of the fluoroscope and an appropriate dosage. Withrespect to previously adopted template matching techniques,the present method overcomes the complete segmentation ofthe components on the projected image and also features thesimultaneous evaluation of all the six degrees of freedom (DOF)of the object. The expected small difference between successiveposes inin vivo sequences strongly reduces the frequency of falseposes and both the operator and computation time.

Index Terms—Model-based pose estimation, single-plane fluo-roscopy, 3-D kinematics, total knee prosthesis.

I. INTRODUCTION

A. Motivation

I N total knee replacement (TKR) surgery, tibial and femoralarticular surfaces are replaced with metallic components

separated by a polyethylene insert. An example of the ge-ometry of the metallic components is given in Fig. 1. The

Manuscript received February 2, 1999; revised July 23, 1999. This workwas supported in part by Exactech Inc. The Associate Editors responsible forcoordinating the review of this paper and recommending its publication wereM. Viergever and W. Niessen.Asterisk indicates corresponding author.

S. Zuffi, *A. Leardini, F. Catani, and S. Fantozzi are with the MovementAnalysis Laboratory, Istituti Ortopedici Rizzoli, Bologna, Italy. S. Fantozziis also with the Dipartimento di Elettronica, Informatica e Sistemistica,Universita di Bologna, Italy.

A. Cappello is with the Dipartimento di Elettronica, Informatica e Sis-temistica, Universita di Bologna, Italy.

Publisher Item Identifier S 0278-0062(99)09628-7.

substitution of bone surfaces is necessary for arthritic kneeswhere the articular cartilage is damaged. As the cartilagenormally acts like a cushion to reduce the friction betweenjoint surfaces, the erosion causes a painful and progressivedegeneration of uncovered bone ends. The number of TKRhas been increasing in the last years. In 1995, 216 000 TKR’swere performed in the United States [1]. However, this figureshould be seen together with the 18 000 revisions performedduring the same period, which means an average failurerate of more than 8%. This is related to the longevity ofimplants, often shorter than designer’s expectations. Moreover,more than one hundred different prosthesis designs are nowavailable, and the choice is not only for size and geome-try, but also involves more critical issues such as cruciateretaining/substituting, uni/tricompartmental and mobile/fixedbearing prosthesis. Implant failure is mainly due to wear ofthe polyethylene insert, associated with an excessive stress atthe artificial joint interface, as consequence of an abnormalknee kinematics. Recent studies [2], [3] have pointed out thatTKR kinematics can be different from intact knee and evenfrom prosthesis design expectations. Tools for the accuratemeasurement ofin vivo kinematics of prosthesis componentsare therefore fundamental to compare the kinematics patternsassociated with different designs, to test new designs andsurgical strategies, and to collect valuable information fordevising new types of implants which could better replicatenormal knee function. Accuracy of the tools in the range ofa few millimeters and a few degrees is sufficient to detectabnormal and potentially dangerous joint kinematics patterns.

B. Previous Approaches

Many different techniques for knee kinematics analysishave been proposed in the past. Noninvasive methods simplyassume the position of several knee anatomical landmarksto be coincident with external position sensors (reflectingmarkers, electromagnetic sensors, etc.), fixed to the skin incorrespondence to relevant bony prominences. These methodsare limited by the large artifact introduced by the motion ofthe skin with respect to the underlying bone. Recent papersfrom some of the authors have shown that the errors involvedcan be as large as few centimeters for joint translations andseveral degrees for rotations [4], [5]. The inaccuracies involvedin traditional gait analysis do not allow the assessment ofthe effectiveness of innovative joint prosthesis features andsurgical strategies and, therefore, do not enable us to improveprosthesis design and clinical outcomes. In this new context,

0278–0062/99$10.00 1999 IEEE

982 IEEE TRANSACTIONS ON MEDICAL IMAGING, VOL. 18, NO. 10, OCTOBER 1999

Fig. 1. Femoral (top) and tibial (bottom) components of the knee prosthesisused in the validation of the method (Optetrack CR-cemented, Exactech Inc.).Relevant reference coordinate systems are shown. Origins were taken onpoints easy to locate in both the 3-D digitization and the 3-D CAD model.

the goal to be achieved in orientation and position accuracyis, respectively, a few millimeters and a few degrees.

Recently, radiographic images have been used in computer-assisted surgery [6], [7] and to more accurately evaluatemotion in replaced human joints. For TKR kinematics anal-ysis, video-fluoroscopy has been successfully used [2]–[3],[8]–[12]. In this widely used diagnostic technique, the incidentX-ray beam strikes an image intensifier, producing an imagethat is converted into an electronic signal by a televisioncamera. This signal can be visualized on a monitor, recordedon a tape, or even collected on a computer using framegrabber boards. With respect to traditional radiography, areduced X-ray dose is sufficient to form the image which,however, shows a lower quality in terms of resolution anddistortion. Video-fluoroscopy is well suited for kinematicsanalysis of human joints because a sequence of X-ray imagescan be collected during the execution of a specific task. Oneach image, the projection of prosthesis components on theimage plane can be identified. Typical fluoroscopic imagesfrom TKR patients are shown in Figs. 2 and 6. Single-planefluoroscopy is being investigated for TKR kinematics in orderto reduce costs of the analysis, practical problems in thepositioning of the foci, and, particularly, radiation to thepatients.

Human knee movement during daily living activity (e.g.,walking, stair climbing/descending, sitting/rising from a chair,etc.) involves mainly translations and rotations in the sagittalplane, but significant out-of-plane rotations occur as well. InTKR kinematics analysis using video-fluoroscopy, the problemis to estimate for each two-dimensional (2-D) fluoroscopicimage the three-dimensional (3-D) position and orientation(pose) of both the tibial and femoral component. The problem

Fig. 2. Sketch of the model for image generation process: projection linesfrom the contour points must be tangent to the object model when the modelis in the right pose. In the frame, a typical video-fluoroscopy image takenfrom an in vivo examination in a TKR patient.

can therefore be formulated as the estimation of the full sixdegrees-of-freedom (DOF) pose of an object with a well-known geometry from a single projection view.

Previous works have been presented in the field of kine-matics analysis using video-fluoroscopy. Banks and Hodge[2], [8]–[10] use a template matching technique for TKRpose estimation. The authors developed a library of contoursof each prosthesis component, rendered at several differentposes. The contours extracted from every X-ray image arecompared with each element of the library to find the bestmatch. To reduce the size of the library, only out-of-planerotations are taken into account for template matching. Eachtemplate in the library is obtained by rendering the objectmodel at different out-of-plane rotations with steps of 1. Atransformation is then applied to each template in order tonormalize for size and in-plane position and orientation. Inthe matching process, the collected image is first normalizedand then compared with every template in the library to findthe best match. Rotation and translations on the image andtranslation in the direction perpendicular to the image planeare then computed. Fourier descriptors are used to representlibrary templates and projection contours. Each contour pixelis considered as a complex value. If the contour is continuouslytraced, it could be seen as a periodic complex function, and thediscrete Fourier transform (DFT) is applied. The DFT requiresa closed contour, so a requisite for the application of thismethod is a complete segmentation of the X-ray image.

Hoff et al. [11], [12] and Stiehlet al. [3] use a similarapproach. The difference can be identified in the representa-tion of templates and collected image projection, and in thematching process. Areas are used instead of contours: librarytemplates are binary images with a fixed number of pixels andtherefore with the same area. The normalization is performedreducing the image projection to the fixed number of pixels oflibrary templates. In this method, a complete segmentation ofthe prosthesis components is necessary as well.

ZUFFI et al.: ADAPTIVE SEGMENTATION ALGORITHM 983

C. Background

The problem of 3-D model-based pose estimation from 2-Dprojections has been largely faced in computer vision. Krieg-man and Ponce [13] proposed a method based on eliminationtheory. They represented objects as a collection of parametricsurface patches and built equations of model projection con-tours parameterized by the model’s position and orientation.The pose estimation is then formulated as a fitting problembetween computed contours and real data. Due to the complexformulation of the problem, the application of this methodis restricted to very simple objects, modeled by only a fewpatches.

Lowe [14] presented a method suitable for objects thatcan be represented with simple geometric patches. He usednonlinear least squares to align projections with the extremecontour of the object. Unfortunately, TKR components aresmooth objects and a representation with planar patches wouldrequire too many elements.

In an extensive study, Lavall´ee et al. [7], [15] proposedan algorithm for the 3-D pose estimation of smooth free-form objects from a general number of planar video images,and tested the application of the general technique to twoX-ray images in the field of computer-assisted surgery. Theobject pose results from the alignment of the object modelwith all image projections. The alignment is based on theminimization of the distance between model and projectionrays. Distances are precomputed in an octree distance mapfor faster pose estimation. Because object models may be nottoo accurate, as typically obtained from 3-D segmentation onmedical images, two image projections are advised to reachthe necessary accuracy in the estimation of the object pose.

D. Present Study

Because in videofluoroscopic examination of TKR patientsduring activity some occlusion between prosthesis componentsfrequently occurs, a model-based pose estimation method forthe kinematics analysis of TKR, which should overcomethe limitation of the complete segmentation of components,typical of every previous template matching technique [2]–[3],[8]–[12], is proposed here. The aim was to investigate thesuitability of the algorithm proposed by Lavallee and Szeliski[15] in TKR kinematics analysis using single-plane fluoro-scopic images. The goal was to achieve the necessary degree ofaccuracy of the overall technique, even in a simplified versionof the algorithm proposed. The present method also featuresthe simultaneous evaluation of all the six DOF of the kneeprosthesis components.

II. M ETHODS

A. Overview

In TKR 3-D kinematics analysis using single plane video-fluoroscopy the perfect knowledge of the geometry of the twoprosthesis components is necessary together with the relevantprojection on the image plane. When a nonsymmetric objectis imaged by a nonorthogonal camera, a unique projection isproduced for each 3-D pose of the object. The assumption of

Fig. 3. Perspective projection camera model and reference coordinate sys-tems.

our work is that a single image, together with the knowledgeof the 3-D geometry, is sufficient for the complete six-DOF estimation of the object. Although femoral and tibialcomponents are symmetric with respect to the sagittal planein most of the prosthesis designs, false poses due to symmetrycan be rejected by the expected smoothness of the knee jointmotion.

The pose estimation of prosthesis components from a singleview can be obtained by aligning the 3-D object model in orderto obtain a corresponding projection as observed in the X-rayimage. The alignment between model and image is intendedto be the condition that replicates the image formation. Inthe image formation process, X rays pass through the objectto generate image’s projection points. For contour points, Xrays must be tangent to the component. The camera modeldefines projection lines that analytically represent X rays. Forpose estimation, the object model must be aligned with theprojection, that is, projection lines coming from contour pointsmust be tangent to the model surface (Fig. 2). A projectionline is tangent to the model when the minimum distance fromthe surface is zero. Using distances, the definition of a costfunction which represents the alignment is straightforward andthe pose estimation is formulated in a nonlinear minimizationproblem which is solved by an iterative procedure.

B. Projection Model and Calibration

In video-fluoroscopy, X rays are emitted from a source,passing through the object and then striking the input phos-phor. The light produced by the phosphor screen is thenconverted into an image by a television system. X raysare attenuated depending on the density of materials, sohigh-density metallic TKR components appear very dark ona fluoroscopic image. A perspective projection model canrepresent the fluoroscope. In perspective projection model, apinhole camera forms the image. X rays are considered straightlines emitted by a point source of uniform radiation in alldirections (Figs. 2 and 3).

The projection model defines a function that maps 3-Dpoints in the camera-embedded fixed coordinate system,

to 2-D points in the image coordinate system,

(1)


where are the 3-D spatial coordinates and the2-D plane coordinates (Fig. 3). The inverse functiondefines, for each image point, a straight 3-D line, ,joining to the camera focus. The position of the camerafocus with respect to the image plane is fixed and calibratedat the beginning of each experimental session.

C. Distortion Correction

Common fluoroscope images exhibit a significant geometricdistortion that occurs in the image formation chain [16].Distortion is mainly due to the curvature of the screen ofphosphors in the image intensifier and often appears as apin cushion deformation. The television-video digitizer alsoproduces further geometric distortions. For image distortioncorrection, a spatial warping technique is applied [17]. Agrid of evenly spaced metallic markers is fixed to the imageintensifier and an image is acquired. Positions of markerson the distorted image are detected. Transformation betweendetected and known positions of markers is then used tocorrect every collected image in a square-by-square way. Theposition of the center of each marker in the corrected imageis computed by the position of markers on the plate. For eachsquare, coefficients of a second-order polynomial transform[17] between coordinates of markers in the corrected imageand pixel coordinates of markers in the distorted image arethen computed. The polynomial transformation is then appliedto each pixel of the square. Only distortion produced by thearray of phosphors and by lenses that form the image is takeninto account. Deviation from the perspective projection modeldue to X-ray scatter and beam-hardening phenomenon is notconsidered here.

D. Image Acquisition and Processing

The video output of a fluoroscope is connected to a framegrabber board for image acquisition. Images are first correctedfor geometric distortion. To extract the prosthesis componentcontours from the collected image, a Canny’s edge detectoris applied [18]. The edge detector detects contour pointsaccording to grey scale values. In X-ray images, pixel valuesdepend on the density of the structures the rays pass throughbefore reaching the image plane and on the X-ray doses.Contour points are not detected where the high density of othermaterials (bone or bone-cement) close to metallic componentsproduces pixels with value not sufficiently different from thoseof the components. As for the pose estimation method aclosed contour of the components is not required, a completesegmentation is not performed, and only contour points auto-matically extracted with the edge detector are considered here.In the present implementation, spurious edges are detected andmanually erased by the operator.

E. Pose Estimation

As previously stated, the key point in the present methodis to look at the tangent condition between projection linesand model surface. When distance between line and modelis defined by the minimum Euclidean distance between eachpoint of the line and each point of the surface, projection rays

are tangent to the object when their distance to the model iszero. The tangent condition therefore corresponds, in practice,to the minimum distance condition.

Surface points are defined in a third reference system,, embedded with the model. To evaluate distance

values, the transformation that maps points in tomust be applied to ray points.

Let us indicate

(2)

respectively, the pose vector, the orientation matrix, and theposition vector of with respect to . The Euler

- - convention is used for the to transform [19].The inverse transformation to be applied is

(3)

where is the position inand . The distance betweenthe ray of the contour point and the model surface isdefined as

(4)

where negative values apply when the ray point isinternal to the surface. According to (4), the distance of aray that crosses the surface is therefore negative.

For a faster computation of the distance between projectionrays and model, Euclidean distance maps are precomputedand stored. A distance map represents an object, assigning toeach point of the volume around and internal to the object thecorresponding distance from the surface. Using distance maps,the distance of a point belonging to a projection ray from themodel surface can be computed by a trilinear interpolation ofthe eight map points closest to the point. The ray-to-modeldistance can therefore easily be found as the minimum valueof the distance for a set of points that belong to the projectionray. Distance maps are represented by 3-D arrays whose valuesare the vector distances from the closest point of the object.To build distance maps, array values are initialized to zero forsurface points and to an upper bound value for all the otherpoints. Then the algorithm for distance transform computingexplained in [19] is applied. To set negative values for pointsthat belong to the object, all values are set negative first, thena positive value is assigned to a point of the map border andpropagated to all neighbor points that are different from zero.The resolution of the distance map used in this study was0.5 mm. For the femoral component used in the present study,with map side about 70 mm long, this resolution results in astorage size of about 9 Mb.

The Euclidean distance transform was first presented byDanielsson [20] to compute distance maps with no significanterrors. Extensions of the algorithm to 3-D space have beenproposed by Mohr and Bajcsy [21] and Ragnemalm [22]. Thealgorithm applied in this work is similar to that suggested byKozinskaet al. [19] and derived originally by Borgefors [23].


The computer-aided design (CAD) model is first convertedinto a set of triangular patches, then intersections between eachelementary surface and the 3-D map are found as

(5)

where are points that define the triangular patch,is a point that belongs to the triangular patch, andis a distance map value.

The 3-D alignment condition for the object is obtained byminimizing a cost function parameterized by the pose vector,

, and defined as the sum of the squared ray-to-surfacedistances

SSD (6)

where is the number of contour points.The procedure iterates until the difference between two

successive values of the cost function is smaller than a certainthreshold. When the minimum is reached, the pose vector isthe estimate of the object pose. The minimization is carried outby applying the Levenberg–Marquardt nonlinear least squaresmethod [24]. The method requires computing the Hessianmatrix, based on the first partial derivative of the function(4) with respect to the parameters to be estimated

(7)

where is the point of the projection raythat verifies the minimum distance condition in (4). Thepose estimate is the result of an iterative procedure for theminimization of a function here represented by the sum ofthe squared distances between the 3-D object and projectionrays associated to contour points. For the application of thealgorithm an initial guess of the object pose is necessary. Whenthe minimum of the cost function is reached, the pose vector

provides the position and orientation of the objectwith respect to the camera.

The uncertainty of the DOF is estimated at the end ofthe minimization procedure. The Cramer–Rao lower boundfor the parameter covariance matrix, Cov, is computedby inverting the final Hessian matrix. Parameter standarddeviations (std) are then computed as the squared roots ofthe diagonal elements.

III. A CCURACY TESTS

A number of tests were designed and performed to assessthe robustness of the algorithm and the reliability of theproposed method. Simulation tests on synthetic images werefirst performed to assess error-free performance of the algo-rithm and to allow the convergence analysis of the estimationprocess. Relevant results can demonstrate the suitability of thealgorithm for the present investigation and can initially showwhether or not the obtained level of accuracy is enough for the

(a) (b)

(c) (d)

Fig. 4. Typical steps of the iterative pose estimation process for the femoralcomponent from computer simulated test. The previously generated com-ponent contour (solid curves) is superimposed to the CAD model renderedaccording to the current value of the pose vector (dark templates). (a) Initialpose. (b) After three iterations. (c) After seven iterations. (d) After teniterations.

TABLE ITREND OF CONVERGENCE OF THEESTIMATION PROCESS. FOR THE EIGHT STEPS,

RMSD OF THE PROJECTIONRAYS AND THE RESIDUAL POSE DEVIATION IN

TERMS OF ROTATIONS (#x;y;z) AND TRANSLATIONS (Tx;y;z) ARE REPORTED

purpose of TKR kinematics reconstruction. Second, static anddynamic in vitro tests performed on images collected froma real prosthesis in known relative position would involvefurther experimental problems related to image collection anddistortion, image quality, contour overlapping, etc. Finally, anin vivo knee kinematics analysis on a test patient, althoughnot comparable with known patterns, would introduce all thepractical problems expected during the application of the tech-nique in routine clinical settings (image quality, componentocclusion, presence of radio-opaque tissues such as bone-cement and soft tissues, field of view, etc.), and can thereforeshow how the proposed technique can cope with all theseproblems.

The fluoroscopic system used for the experimental tests wasMobilgil—hF (Lecco, Gilardoni, Italy). All the programs werewritten using the Borland C Builder compiler and run onan Intel Pentium II computer, 300 MHz, 64Mb RAM, underWindows NT.


TABLE IIRESULTS FROM A SERIES OFSIMULATED POSITIONS FOR THEFEMORAL COMPONENT. ROTATIONS (#x;y;z) AND TRANSLATIONS (Tx;y;z) ARE REPORTED FOR THE

REFERENCEKNOWN POSE OF THECOMPONENT, FOR THE INITIAL POSE DEVIATION, AND FOR THE RESIDUAL POSE DEVIATION AT THE END OF THE ESTIMATION

PROCESS. THESE LATTER ARE GIVEN TOGETHER WITH THE VALUES OF STANDARD DEVIATION OF THE ESTIMATE. LAST THREE COLUMNS REPORT, RESPECTIVELY,

THE NUMBER OF STEPS (Ns), TIME IN SECONDS (Time) AND THE FINAL RMSD BETWEEN THE PROJECTIONRAYS AND THE CAD MODEL

Fig. 5. Images from thein vitro validation test. Femoral and tibial components are held relatively fixed with bone cement. Images were obtained by varyingthe position and orientation of the entire prosthesis with respect to the fluoroscope.

A. Computer Simulated Tests

A first set of tests was performed by computer simulation.CAD models of femur and tibia components (Optetrak CR-cemented, Exactech Inc.) were rendered at known positionsusing the perspective projection model. Test contours wereobtained by applying rotations and translations to the CADmodel in the range of expected joint motion during activity.No additional distortion or noise was applied to the generatedcontours. The only source of error was, therefore, the res-olution of the distance map. An initial guess for the objectpose estimation process was also selected. Typical imagestaken during the iterative process for the pose estimationof the femoral component are shown in Fig. 4. In orderto explore the convergence domain, a preliminary analysisof nine physiologically significant reference poses (

; ) was conducted.In the worst case, which turned out to be , ,

, the final convergence to the global minimum wasalways obtained with maximal initial pose deviations of8for and and 15 for . The reference values of

and the initial pose deviations of didnot influence the convergence properties. A typical trend ofconvergence is provided in Table I. For each step, the rootmean square distance, RMSD (SSD/N) , between theprojection rays and the CAD model, and the residual distancebetween the relevant estimated and known final pose in termsof rotations and translations are given. Aseries of simulated positions belonging to the convergence

domain was analyzed. The results for the femoral componentare given in Table II. In the first six columns, the referenceknown values of rotation ( in degrees) and translation( in millimeters) imposed to the femoral componentCAD model are reported. The following six columns reportthe deviations from the reference pose imposed as an initialguess in the estimation process. Rotation and translation valueswere randomized for both reference and initial pose deviations.The residual pose deviation for both rotations and translationsare then given, together with the values of standard deviationof the estimates. The last three columns report, respectively,the number of steps, computation time and the final RMSDbetween the projection rays and the CAD model. A similaranalysis was performed for the tibial component. Relevantresults are not reported because they are very similar to thosefor the femoral component.

The maximum residual deviation was found to be 0.4for all rotations, 0.4 mm on the and directions, andapproximately 2 mm on the direction for translations.Fortunately, displacement of the knee prosthesis componentsalong the mediolateral axis is unlikely, and the mainclinical interest is in the anteroposterior and vertical

relative displacement of the components. Typical valuesof the RMSD (about 0.1 mm) were smaller than distancemap resolution (0.5 mm), demonstrating the reliability ofthe algorithm in noise-free conditions. Except for , theuncertainties (std) were of the same order of magnitude as theresidual pose deviations. This demonstrates that the algorithmis able to provide nonbiased estimates. Tables I and II show


TABLE IIIRESULTS FROM THEin vitro POSE ESTIMATION TEST. THE SIX COMPONENTS OF

THE FEMORAL TO TIBIAL RELATIVE POSE VECTOR ARE REPORTED

FOR EACH TEST IMAGE TOGETHER WITH CORRESPONDINGSTANDARD

DEVIATION OF THE ESTIMATES. CORRESPONDINGREFERENCE

VALUES (MEAN � STD FROM THE THREE REPETITIONS)OBTAINED FROM 3-D DIGITIZATION ARE GIVEN IN THE LAST ROW

that final convergence to the global minimum was alwaysobtained in less than 20 steps. Very small residual posedeviations were obtained however after the first two to threesteps

B. In Vitro Tests

In static in vitro tests, the two components of a kneeprosthesis (Optetrak CR-cemented, Exactech Inc.) were fixedin an unknown relative pose using bone cement. Fluoroscopicimages of the entire prosthesis were taken in four differentpositions and orientations with respect to the X-ray source(Fig. 5).

From each collected image, the absolute pose of the twocomponents was estimated using the method described. Therelative pose of the femoral with respect to the tibial com-ponent was then calculated using the Euler- - convention.The experimental accuracy of the method was assessed bycomparing these estimates with position measurements fromthe same prosthesis obtained using an accurate 3-D digitizer(FaroArm, FARO Technologies, Inc.). The digitizer is able tocollect point coordinates in a 3-D space using a dot probewith an accuracy of about 0.2 mm. The tibial componentwas held fixed to a workbench and the coordinates of anumber of selected well-defined points were collected onboth the component surfaces. Three repetitions of the dig-itization process were carried out from the same operator.From four characteristic points a reference coordinate systemwith anatomically oriented axes was defined for both the tibialand femoral component (see Fig. 1). The origins of the twocoordinate systems were taken on easy-to-locate points amongthe digitized set. Results for the relative pose vector of thecomponents from both 3-D matching estimation and reference3-D digitization are reported in Table III. The six componentsof the femoral to tibial relative pose vector are reported foreach test image. Values of standard deviation for the relativepose estimates are also given. The corresponding referencevalues (mean std over the three repetitions) obtained fromthe 3-D digitization are given in the last row for comparison.Solutions were obtained in less then 60 s using about 400contour points.

Fig. 6. Image from anin vivo analysis on a test patient during passive flexionwith the detected contour of the components.

Except for Test 4, relative orientation and position errors aresmaller than 1.5and 1.5 mm, respectively. The slightly worseresults for Test 4 are justified by the large overlap betweenthe two components, as shown in the corresponding image inFig. 5, causing obscuring of large parts of both the silhouettes.

A dynamic in vitro test was also performed. The twocomponents of the knee prosthesis were attached to a plex-iglas hinge structure and a flexion-extension movement wasperformed. A time sequence of 15 images was collected bymeans of a video recorder and then digitized by a framegrabber card (IMASCAN Prism, Imagraph) at 1.25 Hz. Thefile size was about 300 Kb for each image. For the firstimage, the initial guess of the estimation process was manuallydefined to be not too far apart. For each of the follow-ing frames, the initial guess pose was taken as the finalestimated pose of the previous frame. From the estimatedorientations of the reference coordinate systems associated tothe femoral and tibial components, the clinically meaningfulangles of abduction–adduction, internal–external rotation, andflexion–extension of the knee components were calculatedas the sequence of Euler elementary rotations about the-

- axes, respectively. The obtained range of motion was1.3 , 1.4 , and 37 respectively. The undesired out-of-planerotations are accounted for image quality and also for thepossible not perfect alignment between the coordinate systemsof the prosthesis components and the axis of rotation of thehinge. For both the femoral and tibial components, the finalconvergence to the global minimum was always obtained inless than 56 iterations for each frame, which corresponds toapproximately 90 s.

C. In vivo Tests

The relative pose of the two components of the OptetrakCR-cemented knee prosthesis was also reconstructed in anin vivo test using images taken from a test patient during aknee passive flexion-extension trial. The patient was seated onhis thigh and asked to keep knee muscles relaxed as muchas possible. The distal shank was raised and lowered by an


TABLE IVRESULTS FROMin vivo ANALYSIS: FEMORAL COMPONENT. FOR EACH OF THE 11 IMAGES, THE OBTAINED FINAL VALUES OF ORIENTATION (#x;y;z ; IN DEGREES) AND

POSITION (Tx;y;z; IN MILLIMETRES), TOGETHER WITH THE NUMBER OF STEPS (Steps), COMPUTATION TIME (Time) AND THE RMSD BETWEEN THE

PROJECTIONRAYS AND THE CAD MODEL ARE REPORTED. VALUES OF STANDARD DEVIATION FOR THE ESTIMATES ARE ALSO REPORTED

operator moving the knee in the sagittal plane within its rangeof motion. Out-of-plane rotations, however, were expected asguided by the articular surfaces and the soft tissues at the knee.A time sequence of 11 images was collected and recordedas in the dynamicin vitro experiment at 5 Hz. Fig. 6 showsthe noisiest of the images collected with the detected contourof the components. The file size was about 300 Kb for eachimage. For the first image, the initial guess of the estimationprocess was manually defined to be not too far apart. For eachof the following frames, the initial guess pose was taken asthe final estimated pose of the previous frame. For each of the11 images, the obtained final values of orientation (in degrees) and position ( in millimeters), togetherwith the number of steps, computation time, and the finalRMSD between the projection rays and the CAD model arereported for the femoral and tibial components respectively inTables IV and V. Values of standard deviation for the DOFestimates are also given.

Values of Euler elementary rotations about- - axes,calculated as in the dynamicin vitro test, are plotted in Fig. 7.

For both the femoral and tibial components, the final conver-gence to the global minimum was always obtained in less than80 iterations, which correspond to about 2 min. Whereas thestd of the estimates are very similar to those in simulation tests,the RMSD is a little larger because of the artifacts introducedby the distorsion and quality of the images and, possibly,because of the difference between CAD design and actualprosthesis components, as later dealt with in Section IV. Thesevalues are encouraging, however, because they are obtainedin conditions typical for the final application and still well

Fig. 7. Values of flexion–extension, abduction–adduction and inter-nal–external rotation of the knee components during a passive flexion trialfrom a TKR patient for the 11 frames analyzed.

comparable to the resolution of the distance map. Althoughthe real TKR kinematics was not known, the sequence ofrelative poses plotted in Fig. 7 shows a smooth and reasonablephysiologic pattern of motion. In the context of the domain ofconvergence, it is also important to point out that the maximumdistance between two successive orientations is here less that10 in flexion–extension.

IV. DISCUSSION

A model-based method for the kinematics analysis of TKRusing single-plane video-fluoroscopy has been presented. The


TABLE VRESULTS FROM in vivo ANALYSIS: TIBIAL COMPONENT. FOR EACH OF THE 11 IMAGES, THE OBTAINED FINAL VALUES OF ORIENTATION (#x;y;z ; IN DEGREES) AND

POSITION (Tx;y;z; IN MILLIMETRES), TOGETHER WITH THE NUMBER OF STEPS (Steps), COMPUTATION TIME (Time) AND THE RMSD BETWEEN THE

PROJECTIONRAYS AND THE CAD MODEL ARE REPORTED. VALUES OF STANDARD DEVIATION FOR THE ESTIMATES ARE ALSO REPORTED

method is based on the modeling of the fluoroscopic im-age generation process and on the knowledge of prosthesiscomponent CAD models.

With respect to the pose estimation method based on dis-tance minimization from two stereo images proposed byLavallee and Szeliski [15], the present method, dealing withsingle-plane images, is more sensitive to false local minimaof the cost function. An initial guess for the iterative poseestimation process not too far from the actual pose is thereforerequired. However, the present method is well suited for TKRkinematics analysis because the initial guess for each framein a sequence can be taken as the estimated final pose of theprevious frame. The distance between two successive posesis usually not too large for the tasks to be analyzed withthese techniques (gait, stair climbing, passive flexion, etc.), asshown in Fig. 7. Moreover, the convergence domain analysisconfirmed that the global minimum is systematically reachedas it was obtained even in the worst case.

The slightly different performances of the method betweencomputer simulation andin vitro tests can be justified by theinaccuracies in contour extraction and by the representationof the imaging process with the perspective projection model.Contour extraction in low-quality fluoroscopic images can beimproved by an appropriate dosage of the X-ray beam, inorder to obtain a better contrast between prosthesis componentsilhouette and image background. The representation of theimage generation process can also be improved by a betterestimation of the geometry of the projection model. Theauthors, however, are satisfied with the results in Table III.Only in Test 4, were the estimates of and foundfarther apart from the corresponding actual values than anyother test image and larger than project specifications, but thiswas justified by the large overlap between the two components.This configuration, however, is unlikely in real images takenfrom in vivo TKR kinematics analysis.

The problem of finding a complete segmentation of theobjects on X-ray images, as requested by previous templatematching techniques, has been overcome by designing a newtechnique in which a closed contour is no longer mandatory.The present method can cope with partial small occlusions andwith badly defined parts of the contour due to the presence ofbone cement and biological tissue, as frequently occurs ininvivo examinations in TKR patients. In Fig. 8, an example ofthe pose estimation of the tibial component with incompletecontours from thein vitro analysis is given. The relevantspurious edges can be easily detected and manually erasedby an operator, without affecting too much the final accuracyof the pose estimate (see results of thein vitro tests). Anexhaustive analysis on the effect of incomplete contours andon criteria for automatic rejection of spurious edges has notbeen performed in the present work, but it will be matter offuture investigations.

With respect to previous template matching methods forTKR kinematics, the absence of manual contour extractionand interpolation of overlapped 2-D contours reduces theamount of time spent by the operator in routine analyses.Moreover, the pose estimation process is expected to be lesstime-consuming because for the alignment error minimizationthe direction toward the convergence point is well defined.The present approach for pose estimation in each singleframe in a routine fluoroscopic examination strongly reducesthe frequency of false poses. In previous template matchingtechniques, the knowledge of the previous frame pose wasnot taken into account and the matching process in thetemplate library was therefore not optimized. Moreover, inthese methods the estimated pose values for each singleframe can eventually be used to reject false poses consideringthe overall smoothness of knee joint motion. In the presentmethod, the smoothness of motion is considered in each frame,as the relevant final solution is obtained starting from the


(a) (b) (c)

Fig. 8. Tibial component pose estimation with incomplete contours. (a) The rendered tibial CAD model and an incomplete contour at the beginningof the pose estimation. (b) At the end of the alignment process. (c) The points of minimum distance from the projection rays in the final pose arealso indicated on the CAD model.

previous pose as the initial guess, thus reducing both falsepose errors and computation time.

Global resolution of the presented method is most affectedby the pose-estimation procedure. In this context, one of themost critical point is the mismatch between the geometry of theprosthesis components and the corresponding CAD models. Inthe manufacturing process, the surfaces for articulation areusually obtained with a very high level of reproducibility,whereas the shaping of the surfaces to be in contact with boneand soft tissue (pegs, stems, lateral edges, etc.) is performedmuch less carefully. The difference between CAD modelsand real prosthesis components in these latter areas can beup to 1 mm. The present method is able to exactly detectthe series of points on the component surface that generatethe silhouette contour on the projected image. The methodis therefore suitable to implement criteria for taking intoaccount the manufacturing inaccuracies. A predefined weightcould be associated to every area of the surface accordingto manufacturing design and the process could estimate thefinal pose accordingly. Although this features deserves aspecifically addressed study in the future, the concept ofsurface weighting makes the present 3-D matching techniqueeven more relevant.

The pose estimation accuracy also depends on the resolutionof the distance map. Unlike the Lavallee and Szeliski [15]study, a fixed resolution of 0.5 mm was set in the present study.This simplification, together with the single-plane projectionview, definitely makes the present method less accurate. How-ever, when comparing the results of our tests with the originalspecification for the present application (few millimeters andfew degrees), the authors were satisfied with the object poseaccuracy obtained even in this very simplified version of thealgorithm. On the other hand, the novel simplified algorithmenables a better computational efficiency, which makes themethod suitable for personal-computer-based analysis, evenin routine clinical settings. In the near future, inclusion offurther valuable features of the original algorithm will betested in relation to the associated possible computationaldrawbacks.

Limitations of the proposed method can be therefore sum-marized in the use of a single-plane view of the objects under

analysis, in the required close pose as initial guess, in theuse of a fixed resolution distance map, and in the likelydifference between CAD models and physical geometry ofthe objects. However, computer simulated,in vitro and invivo tests showed that the accuracy of the method compareswell with previously reported techniques on TKR kinematics,all based on template matching [8], [11]. With respect tothese techniques, the present iterative method overcomes thecomplete segmentation of the components on the projectedimage, features the simultaneous evaluation of all the sixdegrees-of-freedom of the object pose, reduces frequency offalse pose errors, and, finally, minimizes both the operator andcomputation time. The use of a 3-D matching technique witha cost function based on rays-object distance also makes themethod more robust to possible outliers.

The present work is a modification of the algorithm pro-posed by Lavallee and Szeliski [15]. In that work, a gen-eralized projection was proposed although tested on twoprojections. The present authors aimed to test the performancesof a simplified version of that algorithm to the challenging fieldof TKR kinematics from fluoroscopic images, to see whether astable and accurate pose solution could be obtained with kneeprosthesis components even with single and noisy projectionimages. The present work did omit some important features ofthe Lavallee and Szeliski’s algorithm, for which final precisionand accuracy in object pose estimation should be affected.However, the goal of position and orientation accuracy in therange necessary forin vivo biomechanics investigations onknee replacement seems to be achieved, whereas computationhas been improved.

In vivo tests on patients have demonstrated that when thefluoroscope is positioned carefully according to the specifictask to be analyzed and when an appropriate dosage is set,good quality images can be obtained. Moreover, computersimulation and static/dynamicin vitro tests using real kneeprosthesis components in known and typical relative positionshave shown that the accuracy with which component orienta-tions and positions are estimated is better than 1.5and 1.5mm, respectively. Because these values are within the initialspecification for this application, the method is well suited for3-D TKR kinematics analysis of any design.


ACKNOWLEDGMENT

The authors would like to thank C. Carcasio, Istituti Or-topedici Rizzoli, for the support in the manufacturing of thecalibration and testing devices.

REFERENCES

[1] American Academy of Orthopaedic Surgeons, National Center forHealth Statistics, National Hospital Discharge Surveys, 1995.

[2] S. A. Banks, G. D. Markovich, and W. A. Hodge, “The mechanics ofknee replacements during gait,”Amer. J. Knee Surg., vol. 10, no. 4,Fall 1997.

[3] J. B. Stiehl, D. A. Dennis, R. D. Komistek, and P. A. Keblish, “In vivokinematic analysis of a mobile bearing total knee prosthesis,”ClinicalOrthopaedics Related Research, no. 345, pp. 60–66, 1997.

[4] A. Cappello, A. Cappozzo, P. F. LaPalombara, L. Lucchetti, and A.Leardini “Multiple anatomical landmark calibration for optimal bonepose estimation,”Human Movement Sci., vol. 16, pp. 259–274, 1997.

[5] A. Cappozzo, F. Catani, A. Leardini, M. G. Benedetti, and U. DellaCroce, “Position and orientation in space of bones during movement:Experimental artifacts,”Clinical Biomech., vol. 11, no. 2, pp. 90–100,1996.

[6] A. Gueziec, P. Kazanzides, B. Williamson, and R. H. Taylor, “Anatomy-based registration of CT-scan and intraoperative X-ray images forguiding a surgical robot,”IEEE Trans Med. Imag., vol. 17, pp. 715–728,Oct. 1998.

[7] A. Hamadeh, S. Lavallee, and P. Cinquin, “Automated 3-dimensionalcomputed tomographic and fluoroscopic image registration,”ComputerAided Surg., vol. 3, no. 1, pp. 11–19, 1998.

[8] S. A. Banks, “Model based 3-D kinematic estimation from 2-D per-spective silhouettes: Application with total knee prosthesis,” Ph.D.dissertation, Massachusetts Inst. Technol., Cambridge, MA, 1992.

[9] S. A. Banks and W. A. Hodge, “Accurate measurement of three-dimensional knee replacement kinematics using single-plane fluo-roscopy,” IEEE Trans. Biomed. Eng., vol. 43, pp. 638–648, June1996.

[10] S. A. Banks, G. D. Markovich, and W. A. Hodge, “In-vivo kinematics ofcruciate-retaining and -substituting knee arthroplasties,”J. Arthroplasty,vol. 12, no. 3, pp. 297–304, 1997.

[11] W. A. Hoff, R. D. Komistek, and D. A. Dennis, “Three-dimensionaldetermination of femoral-tibial contact position underin-vivo conditionsusing fluoroscopy,”Clinical Biomech., vol. 13, pp. 455–472, 1998.

[12] W. A. Hoff, R. D. Komistek, D. A. Denniset al., “Pose estimation ofartificial knee implants in fluoroscopy images using a template matchingtechnique,” inProc. 3rd IEEE Workshop Applications Computer Vision,Dec. 2–4, 1996, Sarasota, FL, pp. 181–186.

[13] D. J. Kriegman and J. Ponce, “On recognizing and positioning curved3-D objects from image contours,”IEEE Trans. Pattern Anal. MachineIntell., vol. 12, pp. 1127–1137, Dec. 1990.

[14] D. G. Lowe, “Fitting parametrized three-dimensional models to images,”IEEE Trans. Pattern Anal. Machine Intell., vol. 13, pp. 441–450, May1991.

[15] S. Lavallee and R. Szeliski, “Recovering the position and orientation offree-form objects from image contours using 3D distance maps,”IEEETrans. Pattern Anal. Machine Intell., vol. 17, pp. 378–390, Apr. 1995.

[16] J. M. Boone, J. A. Seibert, W. A. Barret, and E. A. Blood, “Analysis andcorrection of imperfections in the image intensifier-TV-digitizer imagingchain,” Med. Phys., vol. 18, no. 2, pp. 236–242, Mar./Apr. 1991.

[17] W. Pratt,Digital Image Processing, 2nd ed. New York: Wiley, 1991.[18] J. Canny, “A computational approach to edge detection,”IEEE Trans.

Pattern Anal. Machine Intell., vol. 8, pp. 679–698, Nov. 1986.[19] D. Kozinska, O. J. Tretiak, J. Nissanov, and C. Ozturk, “Multidimen-

sional alignment using the Euclidean distance transform,”Graphic.Models Image Processing, vol. 59, no. 6, pp. 373–387, 1997.

[20] P. E. Danielsson, “Euclidean distance mapping,”Comp. Graph. ImageProcessing, vol. 14, pp. 227–248, 1980.

[21] R. Mohr and R. Bajcsy, “Packing volumes by spheres,”IEEE Trans.Pattern Anal. Machine Intell., vol. 5, pp. 111–116, Jan. 1993.

[22] I. Ragnemalm, “The Euclidean distance transform in arbitrary dimen-sions,” Pattern Recognit. Lett., vol. 14, pp. 883–888, 1993.

[23] G. Borgefors, “Distance transformations in arbitrary dimensions,”Com-put. Vision Graph. Image Processing, vol. 27, pp. 321–345, 1984.

[24] D. G. Luenberger,Linear and Nonlinear Programming. Reading, MA:Addison Wesley, 1984.

A model-based method for the reconstruction of total knee replacement kinematics

Documents

Transcript of A model-based method for the reconstruction of total knee replacement kinematics