Reproducibility of Proximal Isovelocity Surface Area, Vena Contracta, and Regurgitant Jet Area for...

9
Reproducibility of Proximal Isovelocity Surface Area, Vena Contracta, and Regurgitant Jet Area for Assessment of Mitral Regurgitation Severity Simon Biner, MD,*† Asim Rafique, MD,† Farhad Rafii, MD,‡ Kirsten Tolstrup, MD,† Omid Noorani, MS,§ Takahiro Shiota, MD,† Swaminatha Gurudevan, MD,† Robert J. Siegel, MD† Tel Aviv, Israel; and Los Angeles, California OBJECTIVES The aim of this study was to evaluate the interobserver agreement of proximal isovelocity surface area (PISA) and vena contracta (VC) for differentiating severe from nonsevere mitral regurgitation (MR). BACKGROUND Recommendation for MR evaluation stresses the importance of VC width and effective regurgitant orifice area by PISA measurements. Reliable and accurate assessment of MR is important for clinical decision making regarding corrective surgery. We hypothesize that color Doppler-based quantitative measurements for classifying MR as severe versus nonsevere may be particularly susceptible to interobserver agreement. METHODS The PISA and VC measurements of 16 patients with MR were interpreted by 18 echocardiologists from 11 academic institutions. In addition, we obtained quantitative assessment of MR based on color flow Doppler jet area. RESULTS The overall interobserver agreement for grading MR as severe or nonsevere using qualitative and quantitative parameters was similar and suboptimal: 0.32 (95% confidence interval [CI]: 0.1 to 0.52) for jet area– based MR grade, 0.28 (95% CI: 0.11 to 0.45) for VC measurements, and 0.37 (95% CI: 0.16 to 0.58) for PISA measurements. Significant univariate predictors of substantial interobserver agreement for: 1) jet area– based MR grade was functional etiology (p 0.039); 2) VC was central MR (p 0.013) and identifiable effective regurgitant orifice (p 0.049); and 3) PISA was presence of a central MR jet (p 0.003), fixed proximal flow convergence (p 0.025), and functional etiology (p 0.049). Significant multivariate predictors of raw interobserver agreement 80% included: 1) for VC, identifiable effective regurgitant orifice (p 0.035); and 2) for PISA, central regurgitant jet (p 0.02). CONCLUSIONS The VC and PISA measurements for distinction of severe versus nonsevere MR are only modestly reliable and associated with suboptimal interobserver agreement. The presence of an identifiable effective regurgitant orifice improves reproducibility of VC and a central regurgitant jet predicts substantial agreement among multiple observers of PISA assessment. (J Am Coll Cardiol Img 2010;3:235– 43) © 2010 by the American College of Cardiology Foundation From the *Division of Cardiology, Tel Aviv Sourasky Medical Center, Tel Aviv, Israel; †Division of Cardiology, Cedars-Sinai Medical Center; ‡Department of Internal Medicine, Kaiser Permanente; and §Department of Biomathematics, University of California, Los Angeles, Los Angeles, California. Manuscript received May 14, 2009; revised manuscript received September 22, 2009, accepted September 28, 2009. JACC: CARDIOVASCULAR IMAGING VOL. 3, NO. 3, 2010 © 2010 BY THE AMERICAN COLLEGE OF CARDIOLOGY FOUNDATION ISSN 1936-878X/10/$36.00 PUBLISHED BY ELSEVIER INC. DOI:10.1016/j.jcmg.2009.09.029

Transcript of Reproducibility of Proximal Isovelocity Surface Area, Vena Contracta, and Regurgitant Jet Area for...

J A C C : C A R D I O V A S C U L A R I M A G I N G V O L . 3 , N O . 3 , 2 0 1 0

© 2 0 1 0 B Y T H E A M E R I C A N C O L L E G E O F C A R D I O L O G Y F O U N D A T I O N I S S N 1 9 3 6 - 8 7 8 X / 1 0 / $ 3 6 . 0 0

P U B L I S H E D B Y E L S E V I E R I N C . D O I : 1 0 . 1 0 1 6 / j . j c m g . 2 0 0 9 . 0 9 . 0 2 9

Reproducibility of Proximal Isovelocity SurfaceArea, Vena Contracta, and Regurgitant Jet Areafor Assessment of Mitral Regurgitation Severity

Simon Biner, MD,*† Asim Rafique, MD,† Farhad Rafii, MD,‡ Kirsten Tolstrup, MD,†Omid Noorani, MS,§ Takahiro Shiota, MD,† Swaminatha Gurudevan, MD,†Robert J. Siegel, MD†

Tel Aviv, Israel; and Los Angeles, California

O B J E C T I V E S The aim of this study was to evaluate the interobserver agreement of proximal

isovelocity surface area (PISA) and vena contracta (VC) for differentiating severe from nonsevere mitral

regurgitation (MR).

B A C K G R O U N D Recommendation for MR evaluation stresses the importance of VC width and

effective regurgitant orifice area by PISA measurements. Reliable and accurate assessment of MR is

important for clinical decision making regarding corrective surgery. We hypothesize that color

Doppler-based quantitative measurements for classifying MR as severe versus nonsevere may be

particularly susceptible to interobserver agreement.

M E T H O D S The PISA and VC measurements of 16 patients with MR were interpreted by 18

echocardiologists from 11 academic institutions. In addition, we obtained quantitative assessment of MR

based on color flow Doppler jet area.

R E S U L T S The overall interobserver agreement for grading MR as severe or nonsevere using

qualitative and quantitative parameters was similar and suboptimal: 0.32 (95% confidence interval [CI]:

0.1 to 0.52) for jet area–based MR grade, 0.28 (95% CI: 0.11 to 0.45) for VC measurements, and 0.37 (95%

CI: 0.16 to 0.58) for PISA measurements. Significant univariate predictors of substantial interobserver

agreement for: 1) jet area–based MR grade was functional etiology (p � 0.039); 2) VC was central MR

(p � 0.013) and identifiable effective regurgitant orifice (p � 0.049); and 3) PISA was presence of a

central MR jet (p � 0.003), fixed proximal flow convergence (p � 0.025), and functional etiology

(p � 0.049). Significant multivariate predictors of raw interobserver agreement �80% included: 1) for VC,

identifiable effective regurgitant orifice (p � 0.035); and 2) for PISA, central regurgitant jet (p � 0.02).

C O N C L U S I O N S The VC and PISA measurements for distinction of severe versus nonsevere MR are

only modestly reliable and associated with suboptimal interobserver agreement. The presence of an

identifiable effective regurgitant orifice improves reproducibility of VC and a central regurgitant jet

predicts substantial agreement among multiple observers of PISA assessment. (J Am Coll Cardiol Img

2010;3:235–43) © 2010 by the American College of Cardiology Foundation

From the *Division of Cardiology, Tel Aviv Sourasky Medical Center, Tel Aviv, Israel; †Division of Cardiology, Cedars-SinaiMedical Center; ‡Department of Internal Medicine, Kaiser Permanente; and §Department of Biomathematics, University ofCalifornia, Los Angeles, Los Angeles, California.

Manuscript received May 14, 2009; revised manuscript received September 22, 2009, accepted September 28, 2009.

Arvosfm

aa

eadmd

2casqccc

imftjh

M

Pae

1MModccmCsI(ecb

A

A

C

E

o

M

P

s

VC � vena contracta

J A C C : C A R D I O V A S C U L A R I M A G I N G , V O L . 3 , N O . 3 , 2 0 1 0

M A R C H 2 0 1 0 : 2 3 5 – 4 3

Biner et al.

Assessment of Mitral Regurgitation Severity

236

merican Society of Echocardiography guide-line recommendations for the evaluation ofmitral regurgitation (MR) severity include

assessment of color flow Doppler (CFD)egurgitant jet area and quantification of MR byena contracta (VC) and by effective regurgitantrifice area (EROA) using proximal isovelocityurface area (PISA) (1,2). A reproducible methodor assessment of MR severity is important for theanagement of patients with MR.Visualization of the regurgitant jet area in the left

trium provides a fast assessment of the shape, size,nd direction of the regurgitant jet and a qualitative

See page 244

valuation of its severity, whereas assessment of VCnd PISA requires meticulous attention to technicaletails both at image acquisition and during actualeasurements (Figs. 1A and 1B) (2–5). Echocar-

iographic research laboratories have demonstratedthat PISA and VC are accurate for MRassessment and have good interobserveragreement (3,6–11). However, the etiol-ogy of MR, the mitral valve morphology,and the regurgitant jet characteristics fur-ther complicate the assessment of MR(2,5,6). In addition, the dynamic nature ofMR and pansystolic changes in VC andproximal flow convergence rate mayalso lead to a considerable temporal vari-ation of both the VC width and the

-dimensional PISA radius throughout systole,ausing uncertainty among echocardiogram readersbout which particular frame to choose for mea-urement (7). Therefore, we hypothesized thatuantitative measurements of PISA and VC forlassifying MR as severe versus nonsevere will haveonsiderable interobserver agreement among clini-al echocardiologists.

The primary aim of this study was to evaluate thenterobserver agreement of quantitative MR assess-

ent by PISA and VC for differentiating severerom nonsevere MR. In addition, we also evaluatedhe reproducibility of qualitative CFD-based MRet area as it is frequently used in clinical settings toelp identify severity of MR.

E T H O D S

atients. Interobserver agreement of jet area–basedssessment of MR severity, PISA, and VC was

t

valuated using transthoracic echocardiograms of p

6 consecutive patients referred to Cedars Sinaiedical Center for possible surgical correction ofR. All patients had a history of effort intolerance

r were asymptomatic with left ventricular systolicysfunction. The focus of the assessment was thelassification of MR into severe versus nonsevereategories to identify appropriate candidates foritral valve surgery. The Cedars Sinai Medicalenter Institutional Review Board approved this

tudy.nterpreters. Eighteen cardiologists with mean 16range 2 to 40) years of post-cardiology fellowshipxperience as echocardiographic specialists (echo-ardiologists), practicing at 11 different university-ased institutions in the U.S., Japan, and Israel were

Figure 1. Measurement of MR PISA Radius and VC Width

(A) A magnified 4-chamber view was used to measure proximalisovelocity surface area (PISA) radius. The Nyquist limit wasadjusted in each case to provide optimal visualization of themitral regurgitation (MR) proximal flow convergence. The PISAradius is measured between the aliasing line of the isovelocitysurface and the level of mitral regurgitant orifice. (B) A magni-fied parasternal long axis view was used to measure vena con-tracta (VC) width. To obtain an optimal VC, the transducer wassometimes angulated out of the standard imaging plane so asto identify simultaneously the 3 components including: the areaof proximal flow acceleration, the VC, and the downstreamexpansion of the mitral regurgitation jet. The VC width (effectiveregurgitant orifice) is the dimension of the neck between proxi-mal flow convergence and flow expansion in receiving chamber.

B B R E V I A T I O N S

N D A C R O N YM S

FD � color flow Doppler

ROA � effective regurgitan

rifice area

R � mitral regurgitation

ISA � proximal isovelocity

urface area

rovided secure Web-based access to relevant echo-

csblaiEwsltta5c4l((ThtymwS

aTovVSieqqtdwri

wvaDDsME

w�a

crlwa

r

J A C C : C A R D I O V A S C U L A R I M A G I N G , V O L . 3 , N O . 3 , 2 0 1 0

M A R C H 2 0 1 0 : 2 3 5 – 4 3

Biner et al.

Assessment of Mitral Regurgitation Severity

237

ardiographic images. The influence of institutionaltandards on interobserver agreement was evaluatedy comparing the agreement of the 6 echocardio-ogists practicing in a single institution to thegreement of the echocardiologists from multiplenstitutions.chocardiography. A single trained sonographerith clinical trial research experience performed all

tudies. A narrow color flow sector width and theeast depth were chosen to maximize image resolu-ion. The CFD images of the mitral valve regurgi-ant jet were acquired in the parasternal long axis,pical 4- and 2-chamber views at a Nyquist limit of0 to 60 cm/s (2). In addition, proximal flowonvergence was recorded using a magnified-chamber view, with baseline shift of the Nyquistimit to optimize visualization of flow convergence2) (Figs. 2A and 2B). All echocardiogram clipsmoving images) were available on the Website.here were no still frame images. The reader,owever, was able to play the images, stop framehem, magnify them, perform frame-by-frame anal-sis, as well as take measurements using calibrationarkings and caliper. All imaging were performedith the iE33 ultrasound system (Phillips Medicalystems, Andover, Massachusetts).As shown in Figures 3A and 3B, VC was

cquired on a magnified parasternal long axis view.o image the VC, the transducer was angulated outf the standard imaging planes in an attempt toisualize the area of proximal flow acceleration, theC, and the downstream expansion of the jet (2).tudy parameters. Participants were provided CFDmages and were asked to rate the following param-ters: 1) MR grade from 1 to 4 based on visualualitative assessment of the jet area; 2) manualuantitative measurement of VC; and 3) quantita-ive measurement of PISA radius. If the echocar-iologist found a parameter to be uninterpretable, itas noted. According to reported values of PISA

adius, the EROA was calculated using the follow-ng formula:

EROA � 2� � RPISA2 � Valiasing ⁄ Vmax

here RPISA is PISA radius (cm), Valiasing is aliasingelocity of the proximal flow convergence (cm/s),nd Vmax is maximal velocity of continuous waveoppler MR signal (cm/s).efinitions. The MR was graded as severe or non-evere for each echocardiogram-Doppler parameter.

R was considered severe if VC was �7 mm, if

ROA was �40 mm2, or the jet size–based grade t

as 3 or 4; MR was considered nonsevere if VC was6.9 mm, EROA was �39 mm2, or jet size–based

ssessment graded it as 1 or 2.The MR jet was considered eccentric if it was in

lose contact with the mitral leaflet behind theegurgitant orifice, and impinged to the medial orateral wall of the left atrium, whereas central jetsere initially directed into the center of the left

trium (2,3).For each patient, temporal variability of PISA

adius and VC width were assessed by measuring

Figure 2. Limitation of PISA Radius for Assessment of MRSeverity

(A) Clear definition of the aliasing line of the proximal isoveloc-ity surface area (PISA) is critical for accurate assessment of mitralregurgitation (MR) severity. As shown in this apical 4-chamberview, the aliasing line of the isovelocity surface and the regurgi-tant orifice are well visualized, therefore reducing the potentialoverall interobserver variability in the assessment of MR severity.(B) In this apical 4-chamber view, the MR isovelocity surface isreadily identifiable; however, precise location of the regurgitantorifice is difficult to judge. This makes the assessment of proxi-mal isovelocity surface area radius less reliable and increases theinterobserver variability in the assessment of MR severity.

hese parameters frame by frame throughout sys-

tstsc

mca

taim

db

tiV

aactm(5dSfCmal

oFcrO(rmka1imStpa

kwv

sup�mr

J A C C : C A R D I O V A S C U L A R I M A G I N G , V O L . 3 , N O . 3 , 2 0 1 0

M A R C H 2 0 1 0 : 2 3 5 – 4 3

Biner et al.

Assessment of Mitral Regurgitation Severity

238

ole. We classified PISA and VC as having sub-tantial pansystolic variation if there was greaterhan a 30% difference between the greatest andmallest values of any 2 different frames of the sameardiac cycle.

We classified PISA as nonspherical if the maxi-al and minimal radius of the proximal flow

onvergence demonstrated a greater than 30% vari-tion within 1 cardiac cycle.

We used a cutoff of 30% to define substantialemporal variability of PISA radius and VC widths well as PISA sphericity to account for a 7% to 8%ntraobserver variability of the small-scale measure-

Figure 3. Limitation of VC Width for Assessment of MRSeverity

(A) As shown in this parasternal long axis view, the transitionfrom proximal flow conversion (arrow) to the vena contracta(VC) of mitral regurgitant (MR) jet is visualized facilitating identi-fication of the effective regurgitant orifice. A clearly defined VCneck is essential for reliable assessment of MR severity. (B) Inthis parasternal long axis view, the transition from the proximalflow conversion to the VC is not well visualized. Consequently,the identification of the effective MR orifice is challenging. Noclear VC neck is seen; therefore, this introduces more potentialinterobserver variability in the assessment of MR severity.

ents in our laboratory. In addition, a �30% a

ifference in these parameters was readily detectabley visual assessment.Figure 3 demonstrates identification of the effec-

ive regurgitant orifice, which was defined as visiblef the transition of proximal flow convergence to theC was clearly identifiable.Overall, raw interobserver agreement (percent

greement) of �80% was classified as substantialgreement as this agreement rate was consideredlinically adequate by all investigators. Values be-ween 60% and 79% were classified as fair agree-ent and values below 60% as poor agreement

notably least possible interobserver agreement is0%, which practically is equivalent to maximalisagreement).tatistical analysis. Statistical analysis was per-ormed with the SPSS 13.0 software (SPSS Inc.,hicago, Illinois). Continuous data are presented asean � SD. Categorical data are presented as

bsolute number or percentages. The significanceevel was set at p � 0.05.

Interobserver agreement was assessed using theverall raw agreement (percent agreement) and theleiss kappa statistic (12). Fleiss multirater kappaoefficient is an agreement rate adjusted for chanceate of agreement, and it was calculated using thenline Kappa Calculator developed by Randolph

13). For this purpose, we used Randolph’s multi-ater variation of Brennan and Prediger’s (14) free-arginal kappa. Raw agreement and multirater

appa were calculated for 3 methods (VC, PISA,nd jet area–based MR severity) in 16 patients by8 observers and were calculated for each of thendividual patients to determine the range of agree-

ent within each method across all participants.ubsequently, cumulative raw agreement and mul-irate kappa were calculated for 3 study methods inatients with substantial raw agreement (�80%)nd those with suboptimal raw agreement (�80%).

Mean and 95% confidence intervals of multiraterappa were derived for 3 study methods and t testas used for comparison of multirater kappa amongarious study methods.

The MR characteristics thought to be clinicallyignificant were dichotomized. Fisher exact test wassed to compare the significance of these predictivearameters on raw agreement of �80% versus80%. Significant variables were included in theultivariate logistic regression analysis to predict

aw agreement �80%.A p value �0.05 was considered significant for all

nalyses.

R

BgsytcmpaMr2bttn(Pa2i(

rmaoam8Ianpw[M

1(

d

J A C C : C A R D I O V A S C U L A R I M A G I N G , V O L . 3 , N O . 3 , 2 0 1 0

M A R C H 2 0 1 0 : 2 3 5 – 4 3

Biner et al.

Assessment of Mitral Regurgitation Severity

239

E S U L T S

aseline characteristics. Clinical and echocardio-raphic characteristics of 16 patients studied areummarized in Table 1: mean age was 69 � 12ears and 56% were men. Eight (50%) had func-ional MR (6 with ischemic and 2 with dilatedardiomyopathy); 8 cases were degenerative (5 withitral valve prolapse and 3 with flail leaflet). In 6

atients with functional MR, the jet was central. Inll 8 degenerative MR patients and 2 functional

R cases, the jet was eccentric. For both the PISAadius as well as for VC width 7 (5 degenerative andfunctional MR) cases demonstrated �30% frame-y-frame variability, whereas in the rest of the cases,he variability in these parameters was not substan-ial. The effective regurgitant orifice in the paraster-al long axis view was visualized in 7 other casesFig. 2A).revalence of severe MR. MR severity based on jetrea was deemed interpretable by readers in 273 of88 (95%) instances, whereas VC and PISA werenterpreted in 206 of 288 (72%) and 207 of 28872%), respectively (p � 0.001).

Table 2 shows the proportion of observers whoated MR as severe according to various assessmentethods in each patient. Using the parameter of jet

rea, echocardiologists graded MR as severe in 169f 273 (62%) instances: using VC, MR was gradeds severe in 74 of 206 (36%); and using PISA,easurements were consistent with severe MR in

5 of 207 (41%) times (p � 0.001).nterobserver agreement. Overall interobservergreement on the classification of MR as severe oronsevere was only fair for each of the studyarameters. Raw agreement and kappa coefficientere: 75 � 16% and 0.32 (95% confidence interval

CI]: 0.12 to 0.52), respectively, for jet area–based

Table 1. Baseline Clinical and EchocardiographicCharacteristics

Patient, n 16

Age, yrs 69 � 12

Sex, M/F 9/7

Etiology, degenerative/functional 8/8

MR jet, eccentric/central 10/6

PISA spherical, yes/no 10/6

�30% variation of PISA radius, yes/no 7/9

�30% variation of VC width, yes/no 7/9

Effective MR orifice identifiable, yes/no 7/9

MR � mitral regurgitation; PISA � proximal isovelocity surface area; VC �vena contracta.

R grade, 0.28 (95% CI: 0.11 to 0.45) and 75 �

5% for VC measurements, and 78 � 15% and 0.3795% CI: 0.16 to 0.58) for PISA measurements.

Figure 4 shows the absolute and percentageistribution of substantial (raw agreement �80%),

0%

10%

20%

30%

40%

50%

60%

70%

80%

90%

100%

Jet Area Vena Contracta EROA

4(25%)

5(31%)

7(44%)

5(31%)

4(25%)

7(44%)

2(12%)

8(50%)

Poor Fair Substantial

Figure 4. Distribution of Overall Raw Interobserver AgreementAssessment of MR Severity

Substantial (raw agreement �80%), fair (raw agreement 60% to 79%poor (raw agreement �60%) interobserver agreements are shown fstudy parameters for 18 observers. Out of 16 patients, substantial ament was achieved in 44% of cases for mitral regurgitation (MR) jet44% of patients for vena contracta method, and only 38% for effecregurgitant orifice area (EROA) based on proximal isovelocity surfac

Table 2. The Rate of Classification of MR as Severe by 18Observers on 3 Study Parameters

VC (%) PISA (%) Jet Size (%)

Patient 1 7 0 39

Patient 2 43 29 67

Patient 3 14 21 22

Patient 4 29 0 78

Patient 5 57 57 100

Patient 6 7 7 56

Patient 7 7 29 56

Patient 8 14 7 22

Patient 9 36 43 39

Patient 10 36 79 61

Patient 11 57 79 94

Patient 12 36 64 56

Patient 13 7 14 17

Patient 14 86 100 100

Patient 15 57 64 83

Patient 16 79 64 94

Overall 36 41 62

Abbreviations as in Table 1.

6(38%)

for

), andor 3gree-area,

tivee area

radius.

fat(t0f

k0m(sr

oica3stwvE�vtoPdMaemvt

awja

among int

EROA � effective regurgit

J A C C : C A R D I O V A S C U L A R I M A G I N G , V O L . 3 , N O . 3 , 2 0 1 0

M A R C H 2 0 1 0 : 2 3 5 – 4 3

Biner et al.

Assessment of Mitral Regurgitation Severity

240

air (raw agreement 60% to 79%), and poor (rawgreement �60%) agreement for 3 study parame-ers. Raw agreement �80% was observed only in 744%) patients for jet area–based assessment. Inhis subgroup, cumulative kappa coefficient was.72 (95% CI: 0.68 to 0.76). Substantial agreementor VC was achieved also in 7 (44%) cases, whereas

Jet Area

44%

56%

Vena Contracta

44% 44%

EROA

56%

50%

Multi Center Single Center

Distribution of Substantial Interobserver Agreement Amongologists Practicing in a Single Center and Multiple Centers

ardiologists practicing at 11 different institutions in the U.S.,Israel assessed mitral regurgitation (MR) severity using 3 meth-atients. In addition, interobserver agreement for assessment ofy was compared among the 6 echocardiologists practicing withinstitution to the agreement of the echocardiologists from multiple. There was a similar, suboptimal rate of substantial interobserver(raw agreement �80%) for all 3 color flow Doppler parameterserpreters from both single center and multicenter interpreters.

nd Multivariate Analysis of Morphological and Etiologic PredictorA

Suboptimal Agreement(Raw Agreement <80%)

Jet Area

al 50/50

/functional 67/33

VC

al 89/11

ntifiable, yes/no 22/78

/functional 67/33

ation of VC width, yes/no 78/22

EROA by PISA

al 90/10

70/30

/functional 70/30

ation of PISA radius, yes/no 80/20

ant orifice area; other abbreviations as in Table 1.

appa in this subgroup was 0.61 (95% CI: 0.58 to.64). For PISA measurement, substantial agree-ent was present in 6 (38%) patients, kappa 0.85

95% CI: 0.84 to 0.86). The difference was notignificant among the 3 study parameters. In theest of the cases, the agreement was fair or poor.

As shown in Figure 5, separate analysis of inter-bserver agreement among interpreters practicingn a single institution and those from multipleenters demonstrated that the proportion of rawgreement �80% was insignificantly different for all

study parameters, including jet area: 56% foringle institution versus 44% for multiple institu-ions (p � 0.72). For VC, raw agreement �80%as same for physicians practicing in single centerersus multiple centers at 44% (p � 1.0), and forROA measurement by PISA, raw agreement80% between the 2 groups of observers was 56%

ersus 50% of the patients (p � 0.99). Similarlyhere was no significant difference in the proportionf fair or poor agreements for all 3 study methods.redictors of substantial interobserver agreement. Toefine the source of heterogeneous agreement onR severity, etiologic and echocardiographic vari-

bles that may limit the accuracy of assessment forach study method were included in univariate andultivariate analysis. Table 3 shows the list of the

ariables evaluated for each study method as well asheir value to predict raw agreement �80%.

The only significant univariate predictor of rawgreement �80% for the jet area–based methodas functional etiology (p � 0.039). An eccentric

et had an insignificant influence on interobservergreement. Multivariate analysis did not identify

Substantial Interobserver Agreement for

ubstantial Agreementaw Agreement >80%)

Univariatep Value

Multivariatep Value

17/83 0.074 NS

0/100 0.021 NS

29/71 0.013 NS

71/29 0.049 0.035

29/71 NS NS

33/67 NS NS

17/83 0.0003 0.02

50/50 0.424 NS

17/83 0.039 NS

20/80 0.025 NS

0%

10%

20%

30%

40%

50%

60%

Figure 5.Echocardi

Eighteen cJapan, andods in 16 pMR severita single ininstitutionsagreement

Table 3. Univariate a s ofJet Area, VC, and PIS

S(R

MR jet, eccentric/centr

Etiology, degenerative

MR jet, eccentric/centr

Effective MR orifice ide

Etiology, degenerative

�30% pansystolic vari

MR jet, eccentric/centr

PISA spherical, yes/no

Etiology, degenerative

�30% pansystolic vari

a(

pcetis�

oc(vmpi(

D

Tiefoqy0qnmpdFaitii(

afldmdgimpi

(imamf

iltmgRERCflgwcc(prwsph2ppupHt

lstabiovMgMmeflgP

J A C C : C A R D I O V A S C U L A R I M A G I N G , V O L . 3 , N O . 3 , 2 0 1 0

M A R C H 2 0 1 0 : 2 3 5 – 4 3

Biner et al.

Assessment of Mitral Regurgitation Severity

241

ny significant predictor of raw agreement �80%Table 3).

In univariate analysis, the statistically significantredictors for raw agreement �80% for VC wereentral MR (p � 0.013) as well as an identifiableffective regurgitant orifice (p � 0.049). Whereas inhe multivariate analysis, only the measure of andentifiable effective regurgitant orifice showed aignificant predictive power of raw agreement80% (p � 0.035) (Table 3).For the PISA radius measurement, the presence

f a central MR jet (p � 0.003), fixed proximal flowonvergence (p � 0.025), and functional etiologyp � 0.049) were statistically significant univariateariables for prediction of raw agreement �80%. Byultivariate analysis, central regurgitant jet mor-

hology was the only significant predictor of rawnterobserver agreement �80% (p � 0.02)Table 3).

I S C U S S I O N

his is the first multicenter study to evaluate thenterobserver agreement of the quantitative param-ters of VC width and PISA to differentiate severerom nonsevere MR. We found that classificationf MR as severe as opposed to nonsevere using theuantitative CFD parameters of VC and PISAielded only fair interobserver agreement (kappa:.28 to 0.37). The interobserver agreement forualitative assessment for identifying severe fromonsevere MR was similar to the quantitativeethods (kappa: 0.32). Our study group was com-

osed of clinically experienced, practicing echocar-iologists from 11 different academic institutions.urthermore, we found that the interobservergreement among echocardiologists practicing andnstructing within the same institution was similaro the multicenter interobserver agreement andnferior to previously reported studies from singlenstitutions validating the use of PISA and VC3,6–11).

The VC width and EROA calculated by PISAre both affected by valve morphology and colorow jet characteristics. Incorporation of echocar-iographic and etiologic variables in univariate andultivariate analysis revealed some of the sources of

isagreement in our study. Various echocardio-raphic parameters were found to be significantn predicting good agreement for each study

ethod in the univariate analysis. However, theresence of a central regurgitant jet was the only

ndependent discriminator between suboptimal P

raw agreement �80%) and substantial reproduc-bility (raw agreement �80%) for PISA radius

easurement. In addition, the identification ofn effective regurgitant orifice was the singleultivariate predictor of raw agreement �80%

or VC measurement.Depending on whether observers use standard-

zed measurements in an echocardiographic coreaboratory, in a rigorously controlled research set-ing, or whether the MR grade is assessed byultiple enrolling sites with multiple echocardiolo-

ists, the rate of agreement is markedly different.eliability of MR classification was reported in theVEREST I (Endovascular Valve Edge-to-Edgeepair Study) trial. Two qualitative parameters—FD jet characteristics and pulmonary vein

ow—and 4 quantitative parameters—VC width, re-urgitant volume, regurgitant fraction, and EROA—ere evaluated. Parameters were integrated to form

omposite MR grade. Individual parameters andomposite MR severity were found to be reproducible15). However, considerable disagreement was re-orted in MR severity grading among echocardiogrameaders in the Acorn Clinical Trial (16). All studiesere interpreted to show significant MR at enrolling

ites, but the core laboratory found that 41% ofatients did not have significant MR: 7.4 % of patientsad no detectable MR, 10.6% had mild MR, and3.3% had moderate MR. Thomas et al. (17) pro-osed an MR index of 6 different indicators of MR: jetenetration into the left atrium, PISA radius, contin-ous wave jet intensity, pulmonary artery pressure,ulmonary venous flow pattern, and left atrial size.owever, there was still a considerable overlap be-

ween moderate and severe MR (17).There are several potential explanations for the

imited reproducibility for the assessment of MReverity. Qualitative assessment of mitral regurgita-ion based on jet area varies on any given still framend may thus be mischaracterized due to the frame-y-frame change in jet size. Whereas the eye rapidlyntegrates the change in jet area over time, this typef visual estimation can be highly subjective andaries among observers’ assessment. The etiology of

R, the mitral valve morphology, and the regur-itant jet characteristics can further compound theR assessment (4,5). Sources of error in theeasurement of PISA include the presence of an

ccentric jet, nonhemispheric geometry of proximalow convergence, imprecise identification of regur-itant orifice (Fig. 2), and dynamic changes ofISA radius throughout systole (6,7). As with

ISA, inherent characteristics of MR can influence

tTdtas

2gcsir0dTAActmwtsbrcsrM(caSptssec3

wvoimsttoipdaChciapcpospb

C

Trtsajimda

R8

R

J A C C : C A R D I O V A S C U L A R I M A G I N G , V O L . 3 , N O . 3 , 2 0 1 0

M A R C H 2 0 1 0 : 2 3 5 – 4 3

Biner et al.

Assessment of Mitral Regurgitation Severity

242

he ability to accurately determine the VC width.he limitations of VC include an eccentric MR jet,ynamic VC, and inadequate visualization of any ofhe 3 components of VC (Fig. 3): proximal flowcceleration, the VC, and the downstream expan-ion of the jet (3,4).

The American Society of Echocardiography003 guidelines for the evaluation of valvular regur-itation emphasize an integrated approach for thelassification of MR severity (2). However, in aubsequent large retrospective study in which sever-ty of MR was quantified by Doppler echocardiog-aphy, 198 patients with an EROA greater than.40 cm2 had a 4% per year risk of cardiac deathuring a mean follow-up period of 2.7 years (18).hese findings largely contributed to the recentmerican College of Cardiology/American Heartssociation 2008 guideline (1) recommendation to

onsider mitral valve surgery for asymptomatic pa-ients with severe MR. Given the current recom-endations for surgery on asymptomatic patientsith severe MR, it is crucial to accurately differen-

iate severe from nonsevere MR. However, ourtudy demonstrates that not only qualitative CFDut also quantitative CFD parameters have limitedeproducibility and thus may be suboptimal in thelinical setting. We believe that the findings of ourtudy, which identifies the potential difficulties ineliability to differentiate severe from nonsevere

R, support the concept of Gaasch and Meyer19), namely “to question the diagnosis of severehronic MR when little or no left ventricular or lefttrial enlargement is found.”tudy limitations. The relatively small number ofatients studied (n � 16) is a limitation. However,here were 18 observers and consequently eachtudy parameter was assessed over 200 times. Smallample size may result in overfitted models. How-ver, it will be practically difficult to have 18ardiologists evaluate MR in a bigger sample size bydifferent methods, although we do realize that it

develop guidelines for the manage- diogr 2003;16:777–

ere attempting to differentiate severe from nonse-ere MR, and we have not assessed entire spectrumf MR severity. Had patients with mild MR beenncluded in this study, the interobserver agreement

ay have been better. The absence of a true goldtandard for MR assessment is a potential limita-ion for any study as it precludes the evaluation ofhe accuracy of the interpretations. In the absencef such a gold standard, substantial variation existsn the classification of MR as severe in the sameatient depending on the method used. Theseifferences limit raw agreement rate maximums ton extent that readers may not appreciate.linical implications. As color Doppler flow methodsave limited interobserver agreement, the classifi-ation of MR as severe and clinical decision makingn patients with MR should incorporate the use ofdditional less equivocal and more reproduciblearameters such as mitral inflow pattern, MR jetontinuous Doppler profile, mitral inflow pattern,ulmonary venous flow, and mitral valve morphol-gy, as well as left ventricular and left atrial dimen-ions and left ventricular systolic function andulmonary artery systolic pressure as recommendedy the American Society of Echocardiography (2).

O N C L U S I O N S

he presence of a central MR jet improves theeproducibility of EROA assessment by PISA andhe ability to identify the effective regurgitant orificeignificantly enhances interobserver agreement of VCssessment. The VC, PISA, and CFD regurgitantet–based assessments of MR grade have substantialnterobserver agreement. Use of the VC and PISA

ethod in routine clinical practice, and for clinicalecision making based solely on EROA calculation insymptomatic MR patients may be problematic.

eprint requests and correspondence: Dr. Robert J. Siegel,700 Beverly Boulevard, Room 5623, Los Angeles, Cal-

would be ideal to have a bigger sample size. We ifornia 90048-1804. E-mail: [email protected].

E F E R E N C E S

1. Bonow RO, Carabello BA, Chatter-jee K, et al. 2008 focused updateincorporated into the ACC/AHA2006 guidelines for the managementof patients with valvular heart dis-ease: a report of the American Col-lege of Cardiology/American HeartAssociation Task Force on PracticeGuidelines (Writing Committee to

ment of patients with valvular heartdisease). J Am Coll Cardiol 2008;52:e1–142.

2. Zoghbi WA, Enriquez-Sarano M,Foster E, et al., on behalf of AmericanSociety of Echocardiography. Recom-mendations for evaluation of the se-verity of native valvular regurgitationwith two-dimensional and Dopplerechocardiography. J Am Soc Echocar-

802.

3. Hall SA, Brickner ME, Willett DL,Irani WN, Afridi I, Grayburn PA.Assessment of mitral regurgitation se-verity by Doppler color flow mappingof the vena contracta. Circulation1997;95:636–42.

4. Roberts BJ, Grayburn PA. Color flowimaging of vena contracta in mitralregurgitation: technical consider-ations. J Am Soc Echocardiogr 2003;

16:1002–6.

1

1

1

1

Kr

J A C C : C A R D I O V A S C U L A R I M A G I N G , V O L . 3 , N O . 3 , 2 0 1 0

M A R C H 2 0 1 0 : 2 3 5 – 4 3

Biner et al.

Assessment of Mitral Regurgitation Severity

243

5. Simpson IA, Shiota T, Gharib M,Sahn DJ. Current status of flow con-vergence for clinical applications: is ita leaning tower of “PISA”? J Am CollCardiol 1996;27:504–9.

6. Schwammenthal E, Chen C, BenningF, Block M, Breithardt G, LevineRA. Dynamics of mitral regurgitantflow and orifice area. Physiologic ap-plication of the proximal flow conver-gence method: clinical data and exper-imental testing. Circulation 1994;90:307–22.

7. Buck T, Plicht B, Kahlert P, SchenkIM, Hunold P, Erbel R. Effect ofdynamic flow rate and orifice area onmitral regurgitant stroke volume quan-tification using the proximal isovelocitysurface area method. J Am Coll Cardiol2008;52:767–78.

8. Bargiggia GS, Tronconi L, Sahn DJ,et al. A new method for quantitationof mitral regurgitation based on colorflow Doppler imaging of flow conver-gence proximal to regurgitant orifice.Circulation 1991;84:1481–9.

9. Rivera JM, Vandervoort PM, ThoreauDH, Levine RA, Weyman AE,Thomas JD. Quantification of mitral

regurgitation with the proximal flow

convergence method: a clinical study.Am Heart J 1992;124:1289–96.

10. Enriquez-Sarano M, Seward JB, Bai-ley KR, Tajik AJ. Effective regurgitantorifice area: a noninvasive Dopplerdevelopment of an old hemodynamicconcept. J Am Coll Cardiol 1994;23:443–51.

11. Fehske W, Omran H, Manz M,Köhler J, Hagendorff A, Lüderitz B.Color-coded Doppler imaging of thevena contracta as a basis for quantifi-cation of pure mitral regurgitation.Am J Cardiol 1994;73:268–74.

12. Fleiss, JL. Statistical Methods forRates and Proportions. 3rd edition.Hoboken, NJ: John Wiley & Sons,2003.

13. Randolph JJ. Free-marginal multiraterkappa: an alternative to Fleiss’ fixed-marginal multirater kappa. Paper pre-sented at: Joensuu University Learn-ing and Instruction Symposium 2005;October 14 and 15, 2005; Joensuu,Finland.

14. Brennan RL, Prediger DJ. Coefficientkappa: some uses, misuses, and alter-natives. Educ Psychol Meas 1981;41:687–99.

15. Foster E, Wasserman HS, Gray W, et

al. Quantitative assessment of severity e

of mitral regurgitation by serial echo-cardiography in a multicenter clinicaltrial of percutaneous mitral valverepair. Am J Cardiol 2007;100:1577–83.

6. Acker MA, Bolling S, Shemin R, etal. Mitral valve surgery in heart failure:insights from the Acorn ClinicalTrial. J Thorac Cardiovasc Surg 2006;132:568–77.

7. Thomas L, Foster E, Hoffman JI,Schiller NB. The mitral regurgitationindex: an echocardiographic guide toseverity. J Am Coll Cardiol 1999;33:2016–22.

8. Enriquez-Sarano M, Avierinos JF,Messika-Zeitoun D, et al. Quantita-tive determinants of the outcome ofasymptomatic mitral regurgitation.N Engl J Med 2005;352:875–83.

9. Gaasch WG, Meyer TE. Left ventric-ular response to mitral regurgitation:implications for management. Circu-lation 2008;118:1041–6.

ey Words: mitral valve yegurgitation y

chocardiography.