Ultrasound in Med. & Biol., Vol. 34, No. 5, pp. 718–725, 2008Copyright © 2008 World Federation for Ultrasound in Medicine & Biology

Printed in the USA. All rights reserved0301-5629/08/$–see front matter

doi:10.1016/j.ultrasmedbio.2007.11.007

● Original Contribution

INTEROBSERVER AGREEMENT IN DIAGNOSING LIVERINVOLVEMENT IN HEREDITARY HEMORRHAGIC TELANGIECTASIA

BY DOPPLER ULTRASOUND

ELISABETTA BUSCARINI,* MICHAEL GEBEL,† KENNETH OCRAN,‡ GUIDO MANFREDI,*GIOVANNA DEL VECCHIO BLANCO,*§ RUMEN STEFANOV,¶� CARLA OLIVIERI,#

CESARE DANESINO,# and ALESSANDRO ZAMBELLI**Gastroenterology Department, Maggiore Hospital, Crema, Italy; †Gastroenterology, Hepatology and Endocrinology

Department, Medical School Hannover, Hannover, Germany; ‡Gastroenterology, Hepatology and EndocrinologyDepartment, University of Berlin (Campus Charité Mitte), Berlin, Germany; §Gastroenterology Department, Tor

Vergata University, Rome, Italy; ¶Department of Social Medicine and Health Management, Medical University ofPlovdiv, Plovdiv, Bulgaria; �Statistics, Mario Negri Institute for Pharmacological Research, Clinical Research Center

for Rare Diseases “Aldo e Cele Daccò,” Bergamo, Italy; and #Genetic Institute, University of Pavia, Pavia, Italy

(Received 8 March 2007, revised 20 October 2007, in final form 12 November 2007)

Abstract—The purpose was to evaluate interobserver agreement on Doppler ultrasonographic (US) diagnosis ofliver vascular malformations (VMs) in hereditary hemorrhagic teleangiectasia (HHT) and on their severitygrading. During the interobserver agreement study, three observers with at least seven years of specificexperience using Doppler US for the diagnosis of liver VMs, judged about the presence/absence of liver VMs andtheir severity on a set of images and videoclips. Interobserver agreement was estimated with � statistics.One-hundred ten cases were reviewed during interobserver study (80 cases with liver VMs, 30 without). Verygood � values of the interobserver agreement were found for all pairs concerning the distinction betweenpresence and absence of hepatic VMs. All observers demonstrated excellent sensitivity and specificity inidentifying hepatic VMs, with their respective areas under the curve ranging from 0.97 to 0.99. Interobserveragreement among the three investigators in staging the hepatic VMs in HHT patients was moderate (Kendall’scoefficient of concordance � 0.26). Study results indicate that Doppler US diagnosis of liver VMs in HHT has ahigh degree of agreement among ultrasonographers; a moderate agreement was found regarding severitystaging. (E-mail: ebuscarini@rim.it) © 2008 World Federation for Ultrasound in Medicine & Biology.

Key Words: Hereditary hemorrhagic telangiectasia, Doppler ultrasonography, Interobserver agreement, Liver,

Vascular malformations.

INTRODUCTION

Hereditary hemorrhagic telangiectasia (HHT), or Rendu-Osler-Weber disease, is an autosomal dominant dyspla-sia characterized by widespread cutaneous, mucosal andvisceral telangiectases with a frequency that has beenestimated as one to two cases out of 10,000 subjects(Bideau et al. 1989; Dakeishi et al. 2002). Telangiectasiais the elementary lesion of HHT, arising from the dilationof a postcapillary venule that fuses directly with anarteriole, bypassing the capillary system. Clinical presen-tation in affected people varies greatly, depending on the

Address correspondence to: Elisabetta Buscarini, Gastroenterol-

ogy Department, Maggiore Hospital, Largo Dossena 2, 26013 Crema,Italy. E-mail: ebuscarini@rim.it

718

number, type and location of telangiectases or vascularmalformations (VMs) with potential morbidity and mor-tality; clinical criteria for diagnosis were revised recentlyby a panel of experts (Shovlin et al. 2000).

Liver involvement in HHT is defined by the pres-ence of VMs potentially involving all hepatic vessels. Atits earliest stage, vascular derangement appears as mi-croscopic telangiectases scattered throughout the liverparenchyma. The evolution of a vascular lesion involvesprogressive enlargement and creation of multiple directarteriovenous shunts, represented by three different typesof left-to-right shunt: hepatic artery to hepatic vein, he-patic artery to portal vein and portal vein to hepatic vein;it is also possible to find veno-venous shunts that have nohemodynamic significance. The increased hepatic blood

flow through arterio-venous fistulas, even when these are

Diagnosing liver involvement in telangiectasia by Doppler US ● E. BUSCARINI et al. 719

microscopic, entails hepatic artery dilation that can bedetected even in its initial stages; in more severe stages,the hepatic artery and its branches show complexchanges, becoming tortuous and tangled (Buscarini et al.1994, 2004). The increase in arteriovenous shunts duringthe evolution of vascular derangement further results inprogressive hepatic and/or portal veins dilation with flowabnormalities.

The diagnosis of liver involvement in HHT can bemade by sensitive methods such as abdominal Dopplerultrasonography (US) (Naganuma et al. 1995; Caselitz etal. 2003) or abdominal computed tomography (CT) (Ra-vard et al. 2004). Doppler US has been proposed as ascreening technique because of its accuracy in detectingliver VMs (up to 100% accurate with the parameter ofhepatic artery diameter [Caselitz et al. 2003]), its nonin-vasiveness, availability, repeatability and low cost (Bus-carini et al. 1997; Caselitz et al. 2003; Naganuma et al.1995); these characteristics of Doppler US imaging offera clear advantage over CT, which implies the use ofionizing radiations and a contrast medium with potentialadverse effects.

In cases of HHT with liver VMs, the diameter of thehepatic artery is dilated without overlap among HHTpatients with liver involvement and patients with HHTwithout liver involvement, patients with cirrhosis orhealthy subjects; hepatic artery diameter thus serves as ahighly sensitive diagnostic parameter for hepatic VMs inHHT (Naganuma et al. 1995; Buscarini et al. 1997, 2004;Caselitz et al. 2003; Ocran et al. 2004). The intrahepaticdilated artery branches appear as tubular structures par-allel to portal branches. Pulsed and color Doppler USallow rapid analysis of the flow pattern of hepatic VMs,including direction, turbulence and velocity. Spectralanalysis shows high-velocity flow (even aliased or tur-bulent) in the hepatic artery and its branches, with a highdiastolic phase as a result of low parenchymal resis-tances. Hepatic artery to portal vein shunts cause pulsa-tility of the portal flow, possibly with phasic or contin-uous reversal. On the other hand, hepatic artery to he-patic vein shunts cause significant changes in theDoppler waveform of the hepatic vein only in severestages of the disease.

Because Doppler US plays a pivotal role in thediagnosis of hepatic VMs, observer agreement on theparameters for identifying VMs is crucial, all the more sobecause Doppler US is an operator-dependent imagingtechnique. Studies of interobserver variability in theDoppler US evaluation of liver hemodynamics (Consen-sus conference 1990; Zoli et al. 1986; Sabbà et al. 1990;Tissot et al. 1995) show that agreement about Dopplerparameters is excellent for qualitative criteria but onlypoor to good for quantitative ones; a correlation exists

between agreement and operator experience.

This study focused on evaluating interobserveragreement on Doppler US diagnosis of liver VMs inHHT (primary aim) and on their severity grading (sec-ondary aim).

MATERIALS AND METHODS

Three observers (A, B and C) with at least 20 years’experience using Doppler US participated in the study;more specifically, operator A had 15 years of practicewith Doppler US on HHT patients, B had 10 years and Chad seven years before the study, during which they hadexamined 115, 60 and 40 cases with liver VMs, respec-tively. On the basis of statistical estimation of samplesize, each operator was instructed to prepare for the studyat least 25 sets of images (color and black-and-whiteprints) of livers with vascular malformations of differentdegrees of severity as defined in a previously proposedclassification (Table 1) (Buscarini et al. 2004) and atleast 10 sets of images of normal livers (i.e., withoutvascular malformations and from non-HHT subjects).Negative cases had to include the same proportion ofmales and females and were age matched with positivecases (all subjects had to be older than 18 y). Bothpositive and negative cases had been directly observedby study operators, and each operator received permis-sion to use liver images for study purpose either fromHHT patients or from negative subjects. US images hadto be produced by equipment with comparable technicalspecification, which was high-end Doppler US apparatus(Aloka SSD 5000, Aloka Alpha10, Aloka, Co.Ltd., To-kyo, Japan; Sonoline Elegra Advanced, Siemens MedicalSystems Inc., Issaquah, WA, USA; ATL HDI 5000,Advanced Technology Laboratories, Bothell, WA,USA). Confirmation of hepatic VMs was required with atleast another imaging technique (either contrast-en-hanced magnetic resonance imaging or helical CT, orceliac angiography [Bernard et al. 1993; Buscarini et al.1997; Caselitz et al. 2003]) and with clinical/laboratoryfindings for normal livers.

Methods for Doppler US study of liver vessels andVMs were as described previously (Buscarini et al. 1997,2004; Caselitz et al. 2003). Every set of images (obtainedby Doppler and B mode) for both negative and positivecases included eight images referring to:

1. proper hepatic artery diameter2. hepatic artery peak flow velocity (PFV)3. proper hepatic artery resistivity index (RI)4. portal vein diameter (trunk, midportion, long-axis

scan of the vessel)5. portal vein mean velocity (MV)6. hepatic vein diameter7. hepatic vein PFV (diastolic peak)

8. peripheral hypervascularization

ply a

720 Ultrasound in Medicine and Biology Volume 34, Number 5, 2008

Fewer images might suffice if they depicted clear-cut liver conditions. In addition, each participant had tobring to the training session three to 10 videos illustrat-ing the same parameters listed here. Videos were pref-erable for training purposes and photographs for theinterobserver study because the latter facilitated the eval-uation of a sample of at least 95 cases.

Two independent operators initially collected allavailable videos and photographs and prepared two sam-ples: a set of videos for training the examiners on the firstday and a test set consisting of the remaining videos andimages that were blinded, mixed and then presented in arandom playlist to the three observers.

The first day of the study was devoted to revising,discussing and establishing consensus on Doppler USparameters for liver VM diagnosis and staging. Accordwas reached on using four stages for interobserver agree-ment evaluation: early, moderate, severe and decompen-sated, corresponding to stages 0�/1, 2, 3 and 4 of theBuscarini classification (Buscarini et al. 2004). This wasfollowed by a joint training session on Doppler US videoclips to verify consensus on different parameters andmaximize agreement.

The interobserver agreement study proper tookplace the following day. After examining photographsand videoclips, the three observers made a final judgmentabout the presence/absence of liver VMs and their se-verity. They were allowed as much time as they neededto review the images, and they then filled out a forminquiring about (i) the presence/absence of liver VMsand (ii) their degree of severity. The forms were col-lected immediately upon completion. Care was taken toassure that examiners made no visual contact with oneanother or discussed or commented on cases during the

Table 1. Grading of sev

VM grading

0� ● HA diameter �5 � 6 mm, and/or● PFV �80 cm/sec, and/or● RI �0.55, and/or● Peripheral hepatic hypervascularization

1 ● HA dilatation, only extrahepatic �6 mm, and● PFV �80 cm/sec, and/or● RI �0.55

2 ● HA dilatation, extra and intrahepatic (“double● PFV �80 cm/sec● possibly associated with moderate flow abnor

3 ● complex changes in hepatic artery and its bramoderate dilatation of hepatic and/or portal ve

4 Decompensation of arteriovenous shunt such as:● marked dilatation of hepatic and/or portal ve● marked flow abnormalities in both arteries a

Veno-venous shunts may be found as well and do not necessarily impeak flow velocity.

assessment period. Videos were blinded to the examin-

ers, because there was no information on the screen thatmight enable them to identify patients.

Statistical analysisAgreement among the three observers was evalu-

ated as (i) agreement between all possible pairs of ratersand (ii) agreement of each operator vs. the standard ofreference: the original confirmed diagnosis of each caseincluded in the study, as described here, was used as thestandard of reference for the statistic evaluation of theoperator’s diagnostic performance in discriminating be-tween healthy and diseased patients. Agreement wasmeasured by means of � statistics, calculating the overall� with its standard error and significance level, andtaking into account the probability of chance agreement.A value of 1 indicates perfect agreement between ratingsand 0 indicates no agreement. The commonly acceptedscale was used on which values above 0.80 stand for verygood agreement, 0.61 to 0.80 good agreement, 0.41 to0.60 moderate agreement, 0.21 to 0.40 fair agreementand 0.20 or less poor agreement (Altman 1991). Prelim-inary estimates of sample size had shown that at least 95cases were needed for a � of 0.85 to be statisticallydifferent from a � of 0.60 at p � 0.05. For the powercalculation, an a priori assumption of about 70% positivecases was made (Donner and Eliasziw 1992). The ob-server’s diagnostic performance in detecting healthyfrom diseased patients (discrimination between presence/absence of liver VMs) was assessed additionally bydetermining diagnostic specificity (Sp), sensitivity (Se),accuracy (Acc) and positive (PPV) and negative (NPV)predictive values. An analysis of receiver operating char-acteristics (ROC) was carried out for each observer andhis/her respective AUC (areas under the curves) esti-

f hepatic VMs in HHT

l” aspect) and

f hepatic and/or portal veinsortuous and tangled) with marked flow abnormalities associated with/or abnormality of hepatic and/or portal vein flow

(s)

VM upgrading. HA � hepatic artery; RI � Resistivity Index; PFV �

erity o

channe

mality onches (tins and

innd vein

mated. Statistical analyses, carried out with WINPEPI

Diagnosing liver involvement in telangiectasia by Doppler US ● E. BUSCARINI et al. 721

(PEPI-for-Windows, Joe Abramson, London, UK) soft-ware and SPSS (ver. 13, SPSS Inc., Chicago, IL, USA),included descriptive statistics, percentage distributions(� standard error of the proportion [SEP]) and Kendall’scoefficient as a nonparametric measure of the associationbetween ordinal variables.

RESULTS

One hundred and ten cases were collected for theinterobserver study (94 image sets, 16 videos). Operatorsprovided 30 cases negative for liver VMs and 80 positivecases (confirmed by another imaging test), with the fol-lowing grading assigned according to the classificationof Buscarini et al. (2004): 10 with grade 0�/1 (early)(Fig. 1); 21 with grade 2 (moderate) (Fig. 2); 25 withgrade 3 (severe) (Fig. 3); and 24 with grade 4 (decom-pensated) (Fig. 4). Cases included in the study werehomogeneous with respect to age, sex and diagnosis. Thebasic characteristics of these patients are presented on

Fig. 1. Early stage VMs. (a, b) Grade 0�: peripheral hyptortuous small arterial branches. (c) Grade 1: a slightly dliver (L). ct � celiac trunk.

ervascularization (arrow) detected by energy Doppler study asilated (6.6 mm) hepatic artery (HA) below the left lobe of the

Table 2.

Fig. 2. Grade 2 VMs (moderate) (energy Doppler US study)intrahepatic artery branches are dilated but yet with straightcourse, parallel to portal vein, thus giving a “double channel”

aspect (arrows).

722 Ultrasound in Medicine and Biology Volume 34, Number 5, 2008

The � values of the interobserver agreement con-cerning the distinction between presence and absence ofhepatic VMs are reported in Table 3, with high � valuesfor all pairs and statistically significant in two pairs. Allobservers demonstrated excellent sensitivity and speci-ficity in identifying hepatic VMs, with their respectiveAUC ranging from 0.97 to 0.99 (Table 4); altogether 10false-negative diagnoses (6/10 in cases with early VMs)and no false-positive diagnoses were made.

Interobserver agreement among all pairs of raters instaging the hepatic VMs in HHT patients was moderate;the � coefficient for all pairs varied from 0.64 to 0.46(any value statistically significant) (Table 5); the Ken-dall’s coefficient of concordance among all observerswas fair and the lowest levels of agreement were foundabout earlier stages of hepatic VMs (Table 6).

DISCUSSION

Doppler US has the unique ability of providingaccurate and noninvasive evaluations of liver vessels bycombining qualitative (presence, direction and flow pro-file) and semiquantitative or quantitative (vascular im-pedance, area, flow velocity and volume) information.However, accurate Doppler US assessment of hepaticvessels requires expertise, along with high-end, sensitiveequipment and a measure of interobserver agreement.

In particular, the role of Doppler US as first-lineimaging for the diagnosis of liver VMs in the generalHHT population (Buscarini et al. 2006) calls for inter-observer agreement of results reported to date with thistechnique.

The outcome of our controlled interobserver study

Fig. 3. Grade 3 VMs (severe). At this stage, more complexchanges of hepatic artery branches (arrows), which are tortuousand tangled, can be demonstrated with color Doppler studyaround the portal vein (pv).

showed a very good interobserver agreement on diagnos-

ing the presence/absence of liver VMs and high diagnos-tic accuracy for all operators. These data confirm thegreat sensitivity and specificity of Doppler US for he-patic VM detection previously reported (Naganuma et al.1995; Buscarini et al. 1997, 2004; Caselitz et al. 2003;Ocran et al. 2004). The strong interobserver agreementprobably reflects the fact that VM diagnosis relies mainlyon qualitative Doppler US parameters, which are morelikely to be found and agreed upon (Consensus confer-ence 1990; Zoli et al. 1986), with corresponding differ-ences between pathologic photographs and photographsof normal examinations that can be likely noticed. Highdetection rate by Doppler US and agreement on its di-agnostic findings assume great clinical importance inso-far as diagnosis of hepatic involvement may constituteone of the criteria necessary to establish a definite orlikely diagnosis of HHT (Shovlin et al. 2000); hepaticVMs in HHT, although generally asymptomatic, can lead

Fig. 4. Grade 4 VMs (decompensated). Doppler US signs ofdecompensation of arteriovenous shunt can be found. (a) ColorDoppler study shows a markedly dilated right hepatic vein(RHV) surrounded by multiple tortuous arterial branches (ar-rows); an arteriovenous anastomosis is also visible (arrow-head). (b) Color Doppler study shows that the left branch of theportal vein (PV) is dilated, with reverted flow, and surroundedby tortuous hepatic artery branches (arrows). US examinationreveals ascites (A).

to debilitating and life-threatening complications in a

Diagnosing liver involvement in telangiectasia by Doppler US ● E. BUSCARINI et al. 723

minority of cases (Garcia-Tsao et al. 2000; Buscarini etal. 2005). A diagnosis of hepatic VMs may lead to timelytherapeutic intervention in the event of complications.

The design of our study approximated the condi-tions of imaging screening of HHT families amongwhom the proportion of liver VMs ranges from 32% to74% (Buscarini et al. 2004; Ocran et al. 2004; StabileIanora et al. 2004); although better diagnostic perfor-mance is to be expected in such a population with highpretest probability, the interobserver agreement aboutDoppler US parameters with good sensitivity and spec-ificity (Caselitz et al. 2003) enhances their applicabilityfor diagnosing liver VMs to unselected patients as well.

Our study showed fair interobserver agreement onthe grading of liver VMs. Staging liver VMs by Doppler

Table 2. Basic characteristics

Patientcharacteristics

Male

n %

Age (y), mean � SD 42 47.12 � 14Diagnosis

Negative for liver VMs 15 35.71Positive–early 5 11.90Positive–moderate 9 21.43Positive–severe 7 16.67Positive–decompensated 6 14.29

SEP � standard error of the proportion.

Table 3. Kappa coefficient of agreement for all possible pairsof raters in discriminating between presence/absence of liver

VMs

Operators � p

A vs. B 0.91 0.042B vs. C 0.85 0.054A vs. C 0.93 0.037

P value is significance vs. kappa 0.6; statistical significance for p �0.05.

Table 4. Observers’ performance in discrim

Measure

A

Value SE p

Areas under the curve (AUC) 0.99 0.01 �0.0Sensitivity (95% CI) 0.97 (0.81–1.00)Specificity (95% CI) 1.00 (0.94–1.00)Accuracy (95% CI) 0.99 (0.94–1.00)Positive predictive value (95% CI) 1.00 (0.86–1.00)Negative predictive value (95% CI) 0.99 (0.92–1.00)

SE � standard error. P refers to the comparison of the respective AUC vs. thediagnostic performance “by chance.”

US involves a combination of qualitative and quantita-tive parameters. This task can be demanding and requiresconsiderable experience; differentiation can be easier formore advanced stages for which greater agreement wasfound. The level of agreement about stages of hepaticVMs, even if not statistically significant, was better forthe pair of observers (A and B) with greater experiencein using Doppler US on HHT patients; our findings thussuggest that HHT families should be investigated andmonitored by highly experienced and specialized ultra-sonographers.

With adequate training, Doppler US can be used toadequately depict liver VMs from the very earliest stagesof their evolution to end-stage vascular derangement(Caselitz et al. 2003; Buscarini el al. 2004; Ocran et al.2004). Recent advances in Doppler US technology haveconsistently facilitated the detection of even subtle signsof liver VMs; indeed, a first Doppler US classification ofseverity proposed in 1994 (Buscarini et al. 1994) haslately been modified to better describe the spectrum ofDoppler US findings in hepatic VMs in HHT (Buscariniet al. 2004). A controlled clinical trial (Caselitz et al.2003) of 25 patients with HHT and liver VMs diagnosedon the basis of CT scan and/or angiography, as well as oftwo control groups—cirrhotic patients and healthy sub-jects—confirmed the high sensitivity and specificity of

ients in our study population

Female

pn % SEP

68 50.04 � 15.37 �0.05

15 22.06 5.03 �0.055 7.35 3.17

12 17.65 4.6218 26.47 5.3518 26.47 5.35

g between presence/absence of liver VMs

B C

Value SE p Value SE p

0.97 0.02 �0.001 0.98 0.02 �0.0010.86 (0.69–0.95) 0.88 (0.72–0.96)1.00 (0.94–1.00) 1.00 (0.94–1.00)0.95 (0.89–0.98) 0.96 (0.90–0.99)1.00 (0.86–1.00) 1.00 (0.86–1.00)0.94 (0.85–0.98) 0.95 (0.87–0.98)

of pat

SEP

.86

7.395.006.335.755.40

inatin

01

diagonal line on ROC curve, i.e., AUC � 0.500; AUC � 0.500 means

724 Ultrasound in Medicine and Biology Volume 34, Number 5, 2008

sonographic parameters proposed previously (Buscariniet al. 1997; Naganuma et al. 1995). Based on the patho-physiology of liver VMs, microscopic parenchymal VMsnot yet entailing hepatic artery dilation might be consid-ered a stage preceding grade 1 and defined as stage 0�.In such cases, the focus should be on minor Doppler USsigns of hepatic VMs, such as peripheral hypervascular-ization and/or increasing flow velocity and decreasing RIin the hepatic artery. Grade 1 may also be difficult todiagnose, requiring a differential diagnosis from othercauses of hepatic artery enlargement (such as cirrhosisand hepatic diseases associated with alcoholism or vas-cular tumors). Yet, adequate experience with early stagesof liver VMs can be obtained only in the context ofscreening programs for HHT families. Obviously, accu-rate and reproducible staging of liver VMs can have animpact on patient management: with correct staging byDoppler US, patients can receive appropriate adviceabout follow-up schedules, as well as further tests de-pending on the severity of liver involvement and itshemodynamic impact, which increases with stages en-tailing either high output cardiac failure or portal hyper-tension or ischemic cholangitis; according to VM stage,appropriate therapy, which is generally reserved tosymptomatic patients, is recommended (Buscarini et al.2006). Furthermore, the natural course of the disease canbe more accurately defined in terms of liver VM staging.

Table 5. Kappa coefficient of agreement regarding stages ofhepatic VMs

Operators � p

Kappa coefficient of agreement for all possible pairs of ratersA vs. B 0.64 0.053B vs. C 0.53 0.055A vs. C 0.46 0.053

P value is significance vs. kappa 0.60; statistical significance for p �0.05.

Table 6. Kendall’s coefficient of concordance among all 3raters for different stages of hepatic VMs

Diagnosis n Kendall chi-square p

Negative 30 1.00 — —Positive early 10 0.09 2.67 0.4459Positive moderate 21 0.36 22.78 � 0.0001Positive severe 25 0.43 31.88 � 0.0001Positive severedecompensated 24 0.54 38.57 � 0.0001Total 110 0.26 87.02 < 0.0001

Kendall’s coefficient of concordance varies between 0 (no agree-ment) and 1 (perfect agreement). Chi-square tests the null hypothesis,

that there is no concordance; if p � 0.05, there is no concordance; forp � 0.05, there is a concordance.

The present study had some limitations. Interob-server agreement was established by the study of photo-graphs or video clips, and it can be argued that they areless suitable than direct performance of US examinationfor determining technique reproducibility. Actually,methodology of the present study is very useful in theinitial part of the assessment of the reproducibility eitherof a given technique or of a given application of atechnique, in which the main issue is to establish acommon imaging interpretation; this methodology hasbeen applied usefully in studies assessing the reproduc-ibility of US (Tissot et al. 1995) and endosonography indifferent clinical contexts (Wallace et al. 2001; Gress etal. 2001; Fusaroli et al. 2002). In addition, it is generallyaccepted that liver vessels, both normal and abnormal,can be judged adequately on the basis of images, all themore so because the observers participating in this studyhad selected pictures that best illustrated the salientDoppler US findings for each case; moreover, becausethe study dealt with a rare disease, photographs andvideo clips were judged optimal to allow the evaluationof an adequate sample. The study results underscorerather two key issues in the clinical use of US: (i) asystematic and standardized documentation and analysisof Doppler US examinations is critical and needed (inHHT, but it could be generalized); and (ii) the jointreview and training session on diagnostic criteria of liverVMs that preceded the study itself probably served toimprove interobserver agreement. However, this studyprocedure confirmed the essential role of cooperativetraining (Zoli et al. 1996) in minimizing interobservervariability in the detection of liver VMs by Doppler US.Actually, a high agreement in the analysis of photo-graphs of experienced ultrasonographers is a precondi-tion for a reasonable use of diagnostic US, all the moreso because in some settings (i.e., United States) thewhole US diagnosis depends on it, because radiologistsdo not directly perform the examination.

In conclusion, the results of our study indicated thatDoppler US diagnosis of liver involvement in HHT hasa high degree of interobserver agreement among ultra-sonographers. The good interobserver agreement en-hances the clinical utility of Doppler US for the detectionand follow-up of liver VMs in HHT.

Acknowledgements—This work was supported by a grant of the ItalianHHT Foundation for Hereditary Hemorrhagic Telangiectasia “OnildeCarini.”

REFERENCES

Altman DG. Practical statistics for medical research. London: Chap-man and Hall, 1991.

Bernard G, Mion F, Henry L, Plauchu H, Paliard P. Hepatic involve-

ment in hereditary hemorrhagic telangiectasia: Clinical, radiologi-

Diagnosing liver involvement in telangiectasia by Doppler US ● E. BUSCARINI et al. 725

cal, and hemodynamic studies of 11 cases. Gastroenterology1993;105:482–487.

Bideau A, Plauchu H, Brunet G, Robert JM. Etude épidémiologique dela maladie de Rendu Osler en France: Répartition géographique etprévalence. Population 1989;1:9–28.

Buscarini E, Buscarini L, Civardi G, Arruzzoli S, Bossalini G, Pi-antanida M. Hepatic vascular malformations in hereditary hemor-rhagic telangiectasia: Imaging findings. AJR Am J Roentgenol1994;163:1105–1110.

Buscarini E, Buscarini L, Danesino C, Piantanida M, Civardi G, Qua-retti P, Rossi S, Di Stasi M, Silva M. Hepatic vascular malforma-tions in hereditary hemorrhagic telangiectasia—Doppler sono-graphic screening in a large family. J Hepatol 1997;26:111–118.

Buscarini E, Danesino C, Olivieri C, Lupinacci G, De Grazia F,Reduzzi L, Blotta P, Gazzaniga P, Pagella F, Grosso M, Pongigli-one G, Buscarini L, Plauchu H, Zambelli. A. Doppler ultrasono-graphic grading of hepatic vascular malformations in hereditaryhemorrhagic telangiectasia. results of extensive screening Ultras-chall Med 2004;25:348–355.

Buscarini E, Danesino C, Olivieri C, Lupinacci G, Zambelli A. Liverinvolvement in hereditary haemorrhagic telangiectasia or Rendu-Osler-Weber disease. Dig Liver Dis 2005;37:635–645.

Buscarini E, Plauchu H, Garcia Tsao G, White RI, Sabbà C, Miller F,Saurin JC, Pelage JP, Lesca G, Marion MJ, Perna A, Faughnan ME.Liver involvement in hereditary hemorrhagic teleangiectasia(HHT): Consensus recommendations. Liver Int 2006;26:1–7.

Caselitz M, Bahr MJ, Bleck JS, Chavan A, Manns MP, Wagner S,Gebel M. Sonographic criteria for the diagnosis of hepatic involve-ment in hereditary hemorrhagic telangiectasia (HHT). Hepatology2003;37:1139–1146.

Consensus conference: The value of Doppler US in the study of hepatichemodynamics. J Hepatol 1990;10:353–355.

Dakeishi M, Shioya T, Wada Y, Shindo T, Otaka K, Manabe M,Nozaki J, Inoue S, Koizumi A. Genetic epidemiology of hereditaryhemorrhagic telangiectasia in a local community in the northernpart of Japan. Hum Mutat 2002;19:140–148.

Donner A, Eliasziw M. A goodness-of-fit approach to inference pro-cedures for the kappa statistic: Confidence interval construction,significance testing and sample size determination. Statistics inMedicine 1992;11:1511–1519.

Fusaroli P, Buscarini E, Peyre S, Federici T, Parente F, De Angelis A,Bonanno G, Meroni E, Napolitano V, Pisani A, Sottili S, ToglianiT, Caletti G. Interobserver agreement in staging gastric malt lym-phoma by endoscopic ultrasonography. Gastrointest Endosc 2002;

55:662–668.

Garcia-Tsao G, Korzenik J, Young L, Henderson KJ, Jain D, Byrd B,Pollak JS, White RI Jr. Liver disease in patients with hereditaryhemorrhagic telangiectasia. N Engl J Med 2000;343:931–936.

Gress F, Schmitt C, Savides T, Faigel DO, Catalano M, Wassef W,Roubein L, Nickl N, Ciaccia D, Bhutani M, Hoffman B, Affronti D.Interobserver agreement for EUS in the evaluation and diagnosis ofsubmucosal masses. Gastro Endosc 2001;53:71–76.

Naganuma H, Ishida H, Niizawa M, Igarashi K, Shioya T, MasamuneO. Hepatic involvement in Osler Weber Rendu disease: Findings onpulsed and color Doppler sonography. AJR Am J Roentgenol1995;165:1421–1425.

Ocran K, Rickes S, Heukamp I, Wermke W. Sonographic findings inhepatic involvement of hereditary hemorrhagic telangiectasia. Ul-trashall Med 2004;25:191–194.

Ravard G, Soyer P, Boudiaf M, Terem C, Abitbol M, Yeh JF, BrouardR, Hamzi L, Rymer R. Hepatic involvement in hereditary hemor-rhagic telangiectasia: Helical computed tomography features in 24consecutive patients. J Comput Assist Tomogr 2004;28:488–495.

Sabbà C, Weltin GG, Cicchetti DV, Ferraioli G, Taylor KJ, NakamuraT, Moriyasu F, Groszmann RJ. Observer variability in echo-Dop-pler measurements of portal flow in cirrhotic patients and normalvolunteers. Gastroenterology 1990;98:1603–1611.

Shovlin CL, Guttmacher AE, Buscarini E, Faughnan M, Hyland RH,Westermann CJJ, Kjeldsen AD, Plauchu H. Diagnostic criteria forhereditary hemorrhagic telangiectasia (Rendu-Osler-Weber Sin-drome). Am J Med Genet 2000;91:66–67.

Stabile Ianora AA, Memeo M, Sabbà C, Cirulli A, Rotondo A, An-gelelli G. Hereditary hemorrhagic telangiectasia: Multi-detectorrow helical CT assessment of hepatic involvement. Radiology2004;230:350–359.

Tissot O, Aubé C, Namour A, Blanc F, Dauver A, Calès P. Interob-server concordance of echographic semiology in liver cirrhosis.Gastroenterol Clin Biol 1995;19:291–296.

Wallace MB, Hawes RH, Durkalski V, Chak A, Mallery S, CatalanoMF, Wiersema MJ, Bhutani MS, Ciaccia D, Kochman ML, GressFG, Van Else A, Hoffman BJ. The reliability of EUS for thediagnosis of chronic pancreatitis: Interobserver agreement amongexperience endosonographers. Gastrointest Endosc 2001;53:294–299.

Zoli M, Marchesini G, Cordiani MR, Pisi P, Brunori A, Trono A, PisiE. Echo-Doppler measurement of splanchnic blood flow in controland cirrhotic subjects. JCU 1986;14:429–435.

Zoli M, Merkel C, Sabba C, Sacerdoti D, Gaiani S, Ferraioli G, BolondiL. Interobserver and inter-equipment variability of echo-Dopplersonographic evaluation of the superior mesenteric artery. J Ultra-

sound Med 1996;15:99–106.