Copyright © Lippincott Williams & Wilkins. Unauthorized reproduction of this article is prohibited.

Development and validation of a nomogram based on clinicalfactors and standard laboratory tests for prediction ofclinically significant liver fibrosis in chronic hepatitis C virusinfectionElisabetta Sagrinia, Ilaria Ardoinoc, Giuseppe Maranoc, Alice Gianstefania,Alessandra Orlandinie, Giada Sebastianif, Gabriele Donatig,Alessandro Cucchettib, Guido Pelosie, Carlo Ferrarie, Alfredo Albertif,Elia Biganzolic,d, Fabio Piscagliaa and Luigi Bolondia

Objectives Staging liver fibrosis in chronic viral hepatitis

C (HCV) patients is essential for prompting surveillance

and treatment. The aim of this study was to develop a

nomogram, on the basis of simple clinical and laboratory

variables, to predict three clinically significant stages of

fibrosis (nil–mild, moderate, advanced/cirrhosis), using

histology as reference, and to compare its performance

with that of FibroTest, a widely used noninvasive fibrosis

score.

Materials and methods Nomograms are graphical

representations of a mathematical formula, used as

predictive tools. The study retrospectively recruited 406

HCV patients undergoing liver biopsy. Nomogram was

developed in a training set of 252 patients and tested

in a validation set of 154 patients. Histology was staged

according to the Metavir system. Fibrosis stages were

subgrouped as follows: advanced fibrosis/cirrhosis

(F3/F4, 24%), nil–mild (F0/F1, 36%), and moderate (F2,

40%). Age at biopsy, aspartate aminotransferase,

c-glutamyl transpeptidase, albumin, platelet count, and

prothrombin activity formed the basis for the so-called

Fibro-Nomogram, which, in one graphical representation,

estimates probability for different stages of fibrosis.

Results Areas under the receiver-operating characteristic

curves for advanced fibrosis/cirrhosis were similar for

training (0.86) and validation sets (0.87). For nil–mild

fibrosis, area under the receiver-operating characteristics

were 0.81 and 0.79. Compared with FibroTest,

Fibro-Nomogram performed slightly better at predicting

severe fibrosis (F3/F4) with positive likelihood ratio (LR +)

5.07 (95% confidence interval 3.08–8.37) versus LR + 3.82

(95% confidence interval 2.56–5.71) for FibroTest. For

nil–mild fibrosis, the two tests showed limited but

comparable performances.

Conclusion In HCV patients, Fibro-Nomogram, an

inexpensive and readily available predictive tool, could

enable clinicians to interpret patients’ profile, concurrently

stratifying patients into three clinically relevant probability

categories with good overall performance. Eur J

Gastroenterol Hepatol 00:000–000 �c 2013 Wolters Kluwer

Health | Lippincott Williams & Wilkins.

European Journal of Gastroenterology & Hepatology 2013, 00:000–000

Keywords: chronic viral hepatitis C, clinical decision making, liver fibrosisprediction, nomogram

aDepartment of Digestive Diseases and Internal Medicine, Division of InternalMedicine, bDivision of Surgery and Liver Transplantation, S. Orsola-MalpighiUniversity and General Hospital, Bologna, cDepartment of Clinical Sciencesand Community Health, University of Milan, dUnit of Medical Statistics, Biometryand Bioinformatics, Fondazione IRCCS National Cancer Institute Milan, Milan,eUnit of Infectious Diseases and Hepatology, University Hospital, Parma,fVenetian Institute of Molecular Medicine (VIMM), University of Padova, Padovaand gDivision of Internal Medicine, Ospedale di Rimini, Rimini, Italy

Correspondence to Elisabetta Sagrini, MD, Department of Digestive Diseasesand Internal Medicine, Division of Internal Medicine, S. Orsola-Malpighi Universityand General Hospital, Via Albertoni 15, 40138 Bologna, ItalyTel: + 39 051 6362582/ + 39 051 6362542; fax: + 39 051 6362725;e-mail: elisabetta.sagrini@gmail.com

Received 2 February 2013 Accepted 4 June 2013

IntroductionAmong patients with chronic hepatitis C virus (HCV)

infection, which is a leading cause of chronic liver disease,

the prevalence of bridging fibrosis/cirrhosis depends on

the population considered and the presence of cofactors,

ranging from 10% in those with persistently normal

aminotransferase levels up to 27–35% in patients with

persistently high aminotransferase levels [1–4].

Accurate staging of disease is crucial for predicting the

natural history of an individual patient’s disease and the

need for antiviral therapy, expensive and with relevant

side effects. The extent of liver fibrosis is a predictor of

treatment response [5–7], whereas the higher the stage

of fibrosis, the higher the risk of developing complica-

tions: hepatocellular carcinoma, portal hypertension, and

liver failure [8].

Recently, several new noninvasive liver fibrosis markers

have been developed, led by the desire to overcome the

limits of biopsy (complications, sampling errors, hardly

repeatable in the midterm).

Original article 1

0954-691X �c 2013 Wolters Kluwer Health | Lippincott Williams & Wilkins DOI: 10.1097/MEG.0b013e328363e29d

Copyright © Lippincott Williams & Wilkins. Unauthorized reproduction of this article is prohibited.

According to the latest EASL guidelines for the manage-

ment of HCV infection [4], noninvasive serum markers

can be used for the detection of significant liver fibrosis

(MetavirZ 2), taking into consideration that a combina-

tion of serum markers plus transient elastography [9] can

improve the accuracy of prediction. Liver biopsy is still

needed to resolve uncertainty and is considered as the

reference method to assess the grade of inflammation and

stage of fibrosis.

Most studies on noninvasive predictors of disease stage in

chronic hepatitis C grouped the patients grossly into

either nil–mild fibrosis versus moderate–severe fibrosis or

cirrhosis, or distinguished severe fibrosis/cirrhosis from all

other stages [10–13]. Severe fibrosis identifies high-risk

patients in whom an antiviral therapy and periodic follow-

up are highly recommended, whereas lower stages of

fibrosis describe lower-risk patients, in whom a ‘wait and

see’ policy could be considered, according to the patients’

age and desire.

This categorization into two risk groups includes a highly

heterogeneous population, thus leaving a large proportion

of patients undetermined.

A classification into three groups (nil–mild vs. moderate

vs. severe fibrosis/cirrhosis), with narrowed categories,

would leave a lower proportion of patients unclassified.

The primary aim of this study was to develop a predictive

model, consisting of routinely available laboratory tests, to

predict the above-mentioned subgroups of fibrosis, nil–

mild versus moderate versus severe fibrosis/cirrhosis, in

HCV-infected patients, using liver histology as reference.

The predictive model was validated on an external cohort

and the results compared with those of FibroTest, a

widely used and validated noninvasive score of liver

fibrosis [12].

The predictive model involved the development of a

nomogram to support clinicians in the assessment of the

probability of their patients having no–mild, moderate, or

advanced fibrosis, directly at the bedside, without the

need for complex formulae.

Model-based nomograms are graphical devices, often

favored because of their ease of use, that allow bedside

prediction with ‘paper and ruler’. We are not aware of any

studies that sought to develop a nomogram for fibrosis

prediction in the setting of HCV infection.

Materials and methodsStudy population and data collection

Four hundred and forty four patients with chronic HCV

infection who underwent liver biopsy between 1998 and

2006 were collected consecutively at three Medical Units

of Northern Italy, while being evaluated for antiviral

treatment. The inclusion criteria of this retrospective

study were chronic HCV infection, confirmed by positive

anti-HCV antibodies and HCV-RNA PCR assays in

serum, and compensated stage of disease (Child–Pugh

A5). Patients with clear signs of decompensated disease

such as hepatic encephalopathy and esophageal varices, or

insufficient liver tissue for staging fibrosis in bioptic

specimen, according to accepted recommendations (ade-

quate biopsy fragments should have a length > 1.5 cm and

contain Z 10 portal spaces), or incomplete laboratory

tests, were excluded from the study. Coexisting liver

diseases (alcohol, hepatitis B virus infection, primary

biliary cirrhosis, autoimmune hepatitis, hemochromato-

sis) were reasonably excluded by history and laboratory

data.

The total population comprised 230 patients recruited

from Parma Hospital, 40 patients from Rimini Hospital,

and 174 patients from Padova Medical Center. The study

was approved by the Institutional Review Boards of the

participating centers.

The patients were divided into two sets: patients from

Parma and Rimini constituted the training set, including

252 patients after exclusion of 13 patients from Parma

and 5 from Rimini because of incomplete data. The

validation set included 154 of 174 patients from Padova.

Two patients were excluded because of incomplete data;

18 patients were excluded as some covariates presented

values beyond the range allowed in the training set to

avoid extrapolation in the assessment of the prediction

tool [14] (Fig. 1). Patients from Padova were part of a

previous study for whom FibroTest results were available.

Therefore, data emerging from the present study could

be immediately assessed against the FibroTest [12],

calculated from the patented formula on the basis of

g-glutamyl transpeptidase (GGT), haptoglobin, a2-

macroglobulin, bilirubin, and apolipoprotein A1 (http://www.biopredictive.com). The use of a validation strategy in

the development of the predictive tool, resorting to an

external group, would reinforce the results obtained. The

development of the predictive model was followed by the

creation of a nomogram (called Fibro-Nomogram).

The analysis of the accuracy of the FibroTest in the

present series and the comparison with our predictive

model were carried out in a second independent step

after building the model on the training set and testing

on the validation set.

Sex and age at biopsy were recorded. Laboratory data

were collected within 1 month of biopsy. Requested

data were aspartate aminotransferase (AST), alanine

aminotransferase (ALT), platelet count, GGT, alkaline

phosphatase, prothrombin time, total serum protein,

percentage of albumin, total bilirubin, total cholesterol,

creatinine, white blood cells, and hemoglobin.

An ultrasound-guided liver biopsy was obtained for each

patient. Liver biopsy specimens were assessed in

each center at the time of biopsy blinded to clinical

2 European Journal of Gastroenterology & Hepatology 2013, Vol 00 No 00

Copyright © Lippincott Williams & Wilkins. Unauthorized reproduction of this article is prohibited.

and serological data. Histology, considered the reference

standard, was staged according to the Metavir classifica-

tion [15] on a five-point scale: no fibrosis (F0), portal

fibrosis without septae (F1), fibrosis with few septae

(F2), severe fibrosis without (F3), or with cirrhosis (F4).

For the purpose of the study, fibrosis stages were

subgrouped according to prognostic significance as

follows: advanced fibrosis/cirrhosis (stage 3–4), moderate

fibrosis (stage 2), and nil–mild fibrosis (stages 0–1).

Statistical analysis

The primary goal was the detection of severe, moderate,

nil/mild fibrosis (F3–F4, F2, F0–F1, respectively); for this

purpose, a multinomial model [16] was developed.

Multinomial models enable extension of logistic regres-

sion analysis to simultaneous and consistent (i.e. under

the natural constraint that outcome probabilities add up

to one) estimation of the probabilities of a categorical

outcome. The diagnostic accuracy of model predictions

was assessed both for the training and for the validation

set; the indexes of accuracy were compared in the

validation set with those obtained for the FibroTest score.

The model results were then translated into a nomogram

(Fibro-Nomogram) according to an original procedure

according to Biesheuvel et al. [17] to facilitate its current

use for the calculation of outcome probabilities. The

analysis was carried out using software R (http://www.R-project.org) [18], with ‘nnet’ and Design ‘rms’ packages

added.

The response variable was the classification of fibrosis

stages into three categories: F0–F1, F2, and F3–F4, as

assessed on the basis of histologic findings. As the focus

was on identifying nil/mild (F0/F1) and severe fibrosis

(F3/F4), moderate fibrosis (F2) was chosen as the

reference class. The independent predictors included in

the model were age at biopsy, AST, GGT, percentage of

albumin on electrophoresis, platelet count, and pro-

thrombin time. When carrying out multivariate analysis,

no automatic procedures of selection of variables were

used; predictors were included on the basis of the clinical

experience of the investigators and their evidence in the

literature in the prediction of liver fibrosis [3,12,19,20].

Possible nonlinear effects were assessed, for all variables,

using restricted cubic splines with three knots [21].

Nonlinear effects were excluded; with the only exception

of platelet count, a spline transformation was used with a

constant effect for the highest observed values

(261� 109/l). Moreover, the natural logarithm was pre-

ferred for AST and GGT, given their skewed distribution.

The diagnostic performance of the model was measured

as sensitivity, specificity, and likelihood ratio (LR).

Predictive accuracy was assessed by calculating the area

under the receiver-operating characteristics (ROC) curve

for each outcome probability, both in the training and

in the validation set.

In the validation set, the predicted stages of fibrosis

obtained from the FibroTest score were categorized in the

same way as the outcomes considered in the present

Fig. 1

Basaltraining set

270 patients

Finaltraining

set252 patients

Developmentof nomogram

−13 patientsdue to missing values

−5 patients

Comparison withFibroTest

Testing ofnomogram

Finalvalidation set154 patients

Padova: 174 patients

− 2 patients dueto missing values

− 18 patients because ofvalues beyond the rangeallowed in the trainingset

Basal studypopulation

444 patients

Rimini:40 patients

Parma:230 patients

+

Study flow chart.

Nomogram for prediction of fibrosis in chronic HCV infection Sagrini et al. 3

Copyright © Lippincott Williams & Wilkins. Unauthorized reproduction of this article is prohibited.

work. The 95% confidence intervals (CIs) for the area

under the ROC curve were based on the DeLong

method [22].

Fibro-Nomogram

Nomogram is a two-dimensional diagram designed to

compute a mathematical function graphically. Nomo-

grams constitute attractive user-friendly interfaces used

to calculate predictions of clinical outcomes, without

resorting to complex formulas or web access.

We present an original diagram for calculating predictions

of polytomous outcomes. The diagram calculates the

probability of each outcome category. Specifically, prob-

abilities of severe fibrosis (F3–F4), or no–mild fibrosis

(F0–F1) are first obtained, and then the probability of F2

can be calculated as their complement to one. The

prediction diagram is divided into two parts: the two

nomograms at the top (Fig. 2a and b) compute the values

of the two distinct model predictors (i.e. linear combina-

tions of the independent variables included in the

model), associated with a specific patient’s covariate

profile. The axis length of covariates is proportional to the

prognostic impact of variables. The values obtained are

translated into the diagram at the bottom to obtain the

probability of the outcome of interest (Fig. 2c).

How to use Fibro-Nomogram

The nomogram is used by first locating a patient’s

position on each predictor variable scale (Fig. 2a and b).

Each scale position has corresponding prognostic points

(top axis). In the top panels, the values of the linear

predictors (LPs) for severe fibrosis (LP.Sev) and no or

mild fibrosis are computed (LP.Mild): the values of each

predictor are located in the corresponding axes, and then

a line straight upward to the points axis determines how

many points the patient receives for each predictor. The

sum of the points obtained for each predictor is placed on

the total point axis. A line straight down to the bottom

axis will find the value of the LP. The curves depicted in

the diagram (Fig. 2c) represent the probability of the

outcome of interest for different values of LP. For severe

fibrosis, the labels LP* and LP indicate LP.Sev and

LP.Mild, respectively; for no–mild fibrosis, LP* corre-

sponds to LP.Mild and LP to LP.Sev.

To facilitate understanding of nomogram use, please see

the example described in Fig. 2.

ResultsPatient characteristics at the time of biopsy and fibrosis

stages are reported in Table 1.

The nomogram (Fibro-Nomogram) derived from the

multinomial model enabling the calculation of the three

outcome probabilities, (a) nil–mild, (b) moderate, or (c)

advanced fibrosis/cirrhosis, is shown in Fig. 2. In the

Fibro-Nomogram represented in Fig. 2a and b, because

of the negligible influence of some variables in predicting

specific outcomes, not all the variables included in the

multinomial model are graphically represented. The

Fibro-Nomogram actually provides adjusted probabilities

for reference values (i.e. the median) of the predictors

not shown, which closely approximate the unadjusted

probabilities both in the training and in the testing

populations.

The ROC curves for each outcome probability shown

in Figs 3–5 are explained further below.

Prediction of severe fibrosis

From a clinical standpoint, severe fibrosis (F3, F4)

preludes a stage at which complications of chronic liver

disease may already occur. On this basis, we considered it

important to have a predictive approach able to confirm

with accuracy this condition to take clinical decisions

(antiviral treatment, ultrasound surveillance, endoscopic

screening for esophageal varices). When this outcome is

considered, the statistical model results showed a good

discrimination performance with an area under the ROC

curve of 0.86 (CI 0.80–0.92) in the training set. The same

value (0.87 CI 0.80–0.93) was maintained in the

validation set, showing an intrinsic robust performance

of the predictive model in terms of outcome discrimina-

tion.

Choosing the point on the ROC curve corresponding to

the best cutoff (the black dot superimposed on the ROC

curve in Fig. 3), the model predicted severe fibrosis with

a sensitivity of 72% and a specificity of 86% in the training

set. In the validation set, the same cutoff yielded

a sensitivity of 68% and a specificity of 87%.

Prediction of nil–mild fibrosis

From a clinical standpoint, the other situation which the

authors considered relevant for clinical management was

to know whether a patient with chronic HCV has no or

mild fibrosis. At this stage, no complications of the

disease are expected and treatment is not compelling, to

be decided according to the patients’ age and desire. In

this case, our model enabled correct identification of

patients with nil–mild fibrosis with an area under the

ROC curve of 0.81 (CI 0.75–0.86) for the training set and

0.79 (CI 0.71–0.86) for the validation set (Fig. 5).

Choosing the point on the ROC curve corresponding to

the best cutoff (the black dot superimposed on the ROC

curve in Fig. 5) in the training set, the sensitivity and

specificity for nil–mild fibrosis were 77% and 70%,

respectively. In the validation set, the same cutoff

yielded a sensitivity of 82% and a specificity of 60%.

Prediction of moderate fibrosis

The main limitation of the proposed model is its

weakness in the correct identification of patients with

moderate fibrosis against the two extreme situations:

4 European Journal of Gastroenterology & Hepatology 2013, Vol 00 No 00

Copyright © Lippincott Williams & Wilkins. Unauthorized reproduction of this article is prohibited.

Fig. 2

Points0

75

320

50

60

340

0 50 100

0.025 0.1

0.1 0.2 0.5 1 2 4 6 13.50.05

0.25 0.5 1 2 3 7

150 200 250 300 350

160 80 50 30 20 10 7 5

65 70 75 80 85 90 95 105 115 125

60 70 80 90 100 120 140 160 180 250

260 200 160 120 80 60 50 40 30 20

65 55 45 35 25

10 20 30 40 50 60 70 80 90 100 0

20

20

200

70

0 20

0.025

0.05 0.1 0.2 0.5 1 2 4 9.75 38.75 97

0.1 0.25 0.5 1 2 5 10 20 50

40 60 80 100 120 140 160 180 200 220 240

66 62 58 54 50 46 42

170 150 130120110100 90 80 70 60 50

30 50 80 120 200 320

30 40 50 60 70

10 20 30 40 50 60 70 80 90 100(a) (b)

Age

AST

Platelet

Points

Age

AST

Platelet

Albumin%

Total points

Linear predictor

LP.severe

Total points

Linear predictor

LP.mild

1.0(c)LP = 0LP = 1

LP = 5

LP = 15

LP = 16LP = 18

LP = 20

LP = 25

LP = 30

LP = 35

LP = 40LP = 45

LP = 50

LP = 60

LP = 70LP = 80

LP = 90LP = 100

0.9

0.8

0.7

0.6

0.5

Pre

dict

ed p

roba

bilit

y

0.4

0.3

0.2

0.1

0.0

0 10 20 30LP∗

40 50 60

Prothrombin time

γ-Glutamyltranspeptidase

(a–c) Fibro-Nomogram for the prediction of Metavir fibrosis stages. The following steps are a guide on how to calculate outcome probabilities. Here,a case of a chronic hepatitis C virus-infected patient, a 52-year-old man, aspartate aminotransferase (AST) 150 IU/l, platelet count 158�103/ml,prothrombin time 80%, g-glutamyl transpeptidase 250 IU/l, and serum albumin 55%, is presented. He contributes 22 points for his age both in (a)and (b); this is determined by drawing a vertical line between the point of value 52 on the age axis and the points scale above. The point values for allclinical predictor variables are determined in the same manner and summed up to a total value, which is plotted on the total point axis (third from thebottom). Then, a vertical line is drawn from the total point axis straight to the linear predictor (LP) axis. He is thus assigned 147 total points(respectively 22 + 28 + 70 + 23 + 4) for predicting mild fibrosis. His total points correspond to almost 0.2 read on the LP.Mild axis. Following thesame instructions, total points of 127 (22 + 40 + 25 + 40) are obtained on the nomogram in (b) for the prediction of severe fibrosis, whichcorresponds to a LP.Sev of about 3. Let us translate the results in (c). For the prediction of severe fibrosis, let the labels LP* and LP indicate LP.Sevand LP.Mild, respectively. For the prediction of no–mild fibrosis, let LP* be LP.Mild and LP be LP.Sev. Now locate LP* (= LP.Sev of 3) on the abscissaand draw a line straight upward to find the intersection with the curve corresponding to LP value (= LP.Mild of 0.2). We obtain a probability of severefibrosis between the first two curves of high, which is about 0.58 projected on the Y-axis. The second probability of mild fibrosis can be read locatingLP.Mild on the X-axis very near the origin and is about 0.12, which is a rather negligible value. The probability of moderate fibrosis will be 0.3[(1) – (0.58) – (0.12)]. The diagram should not appear confusing when in (c) very low values are considered on the abscissa: prediction generallyresults in practically null probability. Consistent with Fibro-Nomogram outcome, the above patient had a fibrosis stage F3 on liver biopsy.

Nomogram for prediction of fibrosis in chronic HCV infection Sagrini et al. 5

Copyright © Lippincott Williams & Wilkins. Unauthorized reproduction of this article is prohibited.

Table 1 Clinical and laboratory features of the training and the validation seta

Training set Validation set

Parma (n = 217) Rimini (n = 35) Total (n = 252) Padova (n = 154)

Stage [n (%)]0 0 (0.00) 9 (25.71) 9 (3.57) 5 (3.25)1 82 (37.79) 10 (28.57) 92 (36.51) 40 (25.97)2 79 (36.41) 8 (22.86) 87 (34.52) 75 (48.70)3 40 (18.43) 6 (17.14) 46 (18.25) 17 (11.04)4 16 (7.37) 2 (5.71) 18 (7.14) 17 (11.04)

Sex (male) [n (%)] 117 (53.92) 28 (80.00) 145 (57.54) 85 (55.19)Other characteristics [mean (SD)]

Age at biopsy (years) 51.34 (11.62) 41.80 (9.21) 50.01 (11.78) 48.32 (12.15)AST (UI/l) 81 (61.9) 64.5 (49.8) 78.7 (60.5) 67 (42.2)ALT (UI/l) 128.2 (103) 99.2 (84.7) 124.1 (101.1) 106.4 (77.5)GGT (UI/l) 52.6 (44.3) 75.8 (63.7) 55.8 (47.8) 56.5 (53.1)Alkaline phosphatase (UI/l) 77 (38.1) 201.7 (78.1) 94.3 (62.8) 79.9 (28.2)Total bilirubin (mg/dl) 0.92 (0.36) 0.85 (0.25) 0.91 (0.35) 0.70 (0.30)Total protein (g/dl) 7.27 (0.62) 7.57 (0.49) 7.31 (0.62) 7.62 (0.50)Percentage of albumin 58.4 (4.1) 54.9 (4.6) 57.9 (4.3) 55.1 (3.9)Platelet count (103/ml) 194 (65) 179 (49) 192 (63) 215 (62)Prothrombin time (%) 98.8 (12.9) 88.5 (10.1) 97.3 (13.0) 90.9 (11.3)Total cholesterol (mg/dl) 173.2 (38) 168.9 (26.3) 172.6 (36.6) 172.3 (33)

ALT, alanine aminotransferase; AST, aspartate aminotransferase; GGT, g-glutamyl transpeptidase.aData are presented as total counts and percentages or as means and SDs.

Fig. 3

1−specificity

Severe fibrosis/cirrhosis

(F3 − F4)

LR+

LR−

AUC

Specificity

Sensitivity

Training TestingFibroTest

(Testing)

80.00% (71.96 − 86.18)

3.82 (2.56 − 5.71)

0.29 (0.16 − 0.54)

0.867 (0.803 − 0.931)

0.37 (0.23 − 0.61)

5.07 (3.08 − 8.37)

86.67% (79.44 − 91.62)

67.65% (50.84 − 80.87)71.88% (59.87 − 81.41)

85.64% (79.91 − 89.94)

5.00 (3.44 − 7.29)

0.33 (0.22 − 0.49)

0.859 (0.803 − 0.915)

76.47% (60.00 − 87.56)

1−specificity

1.0 F3 − F4

Training

0.8

0.6

0.4Sen

sitiv

ity

0.2

0.0

0.0 0.2 0.4 0.6 0.8 1.0

1.0 F3 − F4

Validation

0.8

0.6

0.4

0.2

0.0

0.0 0.2 0.4 0.6 0.8 1.0

Cut-off: 0.3068Sensitivity: 0.7188Specificity: 0.8564AUC: 0.8590 AUC: 0.8672

Accuracy of Fibro-Nomogram in predicting severe fibrosis stages (F3–F4). The empty circle in the validation set graph represents FibroTestperformance on the same subgroup of patients. Black dots along the receiver-operating characteristic (ROC) curve represent the ‘best’ cutoff(the point closest to the left corner of the curve) for which sensitivity, specificity, positive, and negative likelihood ratios (LRs) can be calculated.

6 European Journal of Gastroenterology & Hepatology 2013, Vol 00 No 00

Copyright © Lippincott Williams & Wilkins. Unauthorized reproduction of this article is prohibited.

actually, in such a case, the area under the ROC curve was

0.70 (CI 0.63–0.76), with a 68% sensitivity and a 67%

specificity in the training set, whereas the area under the

ROC curve for the validation set was 0.59 (CI 0.50–0.68),

with a sensitivity and a specificity of 51 and 56%,

respectively (Fig. 4).

Comparison of performance of the newly constructed

model with that of FibroTest

The Fibro-Nomogram allows evaluation of how likely the

fibrosis stage is according to specific covariate patterns.

To facilitate the comparison of our predictive model with

the FibroTest, the sensitivity and specificity for the

outcomes considered were calculated. Results were

combined with the ROC curves obtained from the

validation series (the empty circle in the second panel

of Figs 3–5). The points representing FibroTest were

superimposed with the ROC curve, showing substantially

comparable results between the two models in predicting

no–mild fibrosis (F0/F1) and in confirming severe fibrosis

(F3/F4). The FibroTest performed slightly better at

predicting intermediate fibrosis. Different choices are

available to identify a ‘best cutoff ’ on the ROC curve:

a typical option is to consider the point closest to the left

corner of the curve. When this cutoff is considered, the

LR + and LR – can be calculated to improve the

comparison with the FibroTest.

Albeit neither of the test shows optimal performance

(and their use cannot be recommended instead of the

biopsy), the main result suggests that the novel Fibro-

Nomogram, with a LR + 5.07 (95% CI 3.08–8.37) in the

prediction of F3/F4 fibrosis versus LR + 3.82 (95% CI

2.56–5.71) for FibroTest, can strengthen the confirmation

of severe fibrosis, with a possible improvement over

FibroTest performances despite wide CIs.

In terms of the diagnosis of the absence of fibrosis (i.e. for

the exclusion of fibrosis), Fibro-Nomogram (LR – 0.30,

Fig. 4

1−specificity

Training TestingFibroTest

(testing)

Moderate fibrosis

(F2)

Sensitivity 67.82% (57.43 − 76.70)

55.70% (44.73 − 66.13)

1.41 (0.83 − 1.58)

0.89 (0.65 − 1.20)

0.592 (0.501 − 0.682)

50.67% (39.60 − 61.67)

2.23 (1.41 − 3.53)

0.66 (0.52 − 0.85)

78.48% (68.21 − 86.11)

48.00% (37.07 − 59.13)

2.03 (1.57 − 2.64)

0.48 (0.35 − 0.67)

0.695 (0.629 − 0.762)

66.67% (59.17 − 73.41)

LR+

LR−

AUC

Specificity

1−specificity

1.0 F2

TrainingS

ensi

tivity

Validation

0.8

0.6

0.4

0.2

0.0

0.0 0.2 0.4 0.6 0.8 1.0

1.0 F2

0.8

0.6

0.4

0.2

0.0

0.0 0.2 0.4 0.6 0.8 1.0

Cut-off: 0.3838Sensitivity: 0.6782Specificity: 0.6667AUC: 0.6954 AUC: 0.5917

Accuracy of Fibro-Nomogram in predicting moderate fibrosis stage (F2). Empty circle in the validation set graph represents FibroTest performance onthe same subgroup of patients. Black dots along the receiver-operating characteristic (ROC) curve represent the ‘best’ cut-off (the point closest tothe left corner of the curve) for which sensitivity, specificity, positive, and negative likelihood ratios (LRs) can be calculated.

Nomogram for prediction of fibrosis in chronic HCV infection Sagrini et al. 7

Copyright © Lippincott Williams & Wilkins. Unauthorized reproduction of this article is prohibited.

95% CI 0.16–0.57) and FibroTest (LR – 0.48, 95% CI

0.33–0.71) showed limited but comparable performances.

DiscussionSeveral attempts have been made to overcome the need

for liver biopsy in the assessment of liver fibrosis. The

limitations of liver biopsy are invasiveness, morbidity,

costs, sampling errors, and interintraobserver variations,

producing around 20% false-negative cases for cirrhosis in

comparison with the real gold standard (whole liver) and

overlapping of intermediate stages [23]. The error in liver

biopsy itself makes it impossible to achieve an accuracy of

100% for any invasive test. As Mehta et al. [24] have

shown, an area under the ROC curve of more than 0.90

cannot be achieved even for a perfect marker, even in

a most favorable scenario.

Literature reports a distinction between patented and

nonpatented biochemical/clinical markers or morphologi-

cal methods such as elastography (ultrasound-based or

magnetic resonance elastography) or acoustic radiation

force imaging [9,25,26] to noninvasively assess liver

fibrosis [12,27,28]. Patented ones have been approved

by French authorities for first assessment of fibrosis in

HCV patients [29]. Patented methods are expensive, but

show better performances, in particular the FibroTest and

fibrometer, which incorporate direct markers of fibro-

sis [30].

Almost all available noninvasive tests have reported good

diagnostic accuracy, with area under the ROC around 0.80

for the prediction of significant fibrosis [31,32], but no

test accurate enough to reliably predict the single stage of

fibrosis. Simpler tests, such as APRI or Forns’ [10,11],

which provide cut-off values, have high accuracy when

results are at one end of the test spectrum, but leave a

proportion of patients undetermined, because of the gray

zone between proposed cutoffs. Most studies grouped

the patients essentially into nil–mild fibrosis versus

significant fibrosis/cirrhosis or distinguished severe fibrosis/

Fig. 5

Training TestingFibroTest(testing)

Nil−mild fibrosis

(F0 − F1)

Sensitivity 77.23% (68.14 − 84.32) 82.22% (68.67 − 90.71) 62.22% (47.63 − 74.89)

78.90% (70.32 − 85.51)

2.95 (1.95 − 4.47)

0.48 (0.33 − 0.71)

59.63% (50.25 − 68.36)

2.04 (1.54 − 2.69)

0.30 (0.16 − 0.57)

0.789 (0.714 − 0.864)

70.20% (62.48 − 76.92)

2.59 (1.98 − 3.39)

0.32 (0.22 − 0.47)

0.805 (0.751 − 0.858)

LR+

LR−

AUC

Specificity

1−specificity1−specificity

1.0

TrainingS

ensi

tivity

Validation

F0–F1F0 − F1

0.8

0.6

0.4

0.2

0.0

0.0 0.2 0.4 0.6 0.8 1.0

1.0

0.8

0.6

0.4

0.2

0.0

0.0 0.2 0.4 0.6 0.8 1.0

Cut-off: 0.4144Sensitivity: 0.7822Specificity: 0.7020AUC: 0.8045 AUC: 0.7890

Accuracy of Fibro-Nomogram in predicting the nil–mild fibrosis stage (F0–F1). Empty circle in the validation set graph represents FibroTestperformance on the same subgroup of patients. Black dots along the receiver-operating characteristic (ROC) curve represent the ‘best’ cut-off(the point closest to the left corner of the curve) for which sensitivity, specificity, positive, and negative likelihood ratios (LRs) can be calculated.

8 European Journal of Gastroenterology & Hepatology 2013, Vol 00 No 00

Copyright © Lippincott Williams & Wilkins. Unauthorized reproduction of this article is prohibited.

cirrhosis from all others. Indeed, noninvasive markers have

been shown to be able to accurately predict these two

extreme stages of fibrosis, for whom the need for therapy

and follow-up are quite established, as for example,

transient elastography and biomarkers such as APRI and

FibroTest perform similarly well in predicting severe fibrosis

in chronic HCV patients [33], whereas transient elasto-

graphy performs better in predicting cirrhosis than lesser

degree of fibrosis [34], with a reported area under the ROC

0.94–0.95 [35].

In our opinion, the use of a simple classification of

patients as with or without significant fibrosis (FZ 2) is

too broad for any accurate prognostic information (having

F2 or rather F4 implies a completely different prognosis

over the midterm either in the overall survival and

response to treatment). Similarly, having or not having

cirrhosis is also too broad as this may include either nil

fibrosis or severe fibrosis.

From a practical and clinical point of view, grouping

together patients with moderate (F2) and severe fibrosis

(F3) or cirrhosis (F4) does not help much, as the

prognosis and management may be different in F2 and

F3/F4 patients. At variance, if a test is aimed only at

excluding severe fibrosis, thus classifying patients into

one group including F0, F1, and F2 fibrosis, the clinical

benefit is again limited.

Therefore, we attempted to create a model with more

accurate prognostic information in terms of narrower

categories. On this basis, the study population was

evaluated not just simply as patients with or without

significant fibrosis (Metavir FZ 2) or cirrhosis, but by

subgrouping patients into three categories, F0–F1, F2,

and F3–F4, and developed a model to predict these three

clinical endpoints.

The model obtained an area under the ROC curve for

advanced fibrosis (FZ 3) equal to 0.86 and 0.87 in the

training and the validation set, respectively, and 0.81 and

0.79 in the prediction of nil–mild fibrosis (F0–F1),

corresponding to an intrinsic robust performance in terms

of outcome discrimination, especially helpful in con-

firmation of the extreme types of fibrosis stages.

One limitation of the model appears to be a poor

performance in correctly predicting the presence of

moderate fibrosis (F2). Among the variables entered in

the model, none seems to specifically characterize an

intermediate stage of the disease.

Indeed, this is a main limitation not only of our approach

but of almost all noninvasive tools to assess disease stages.

Namely, a considerable overlap between contiguous

stages of intermediate fibrosis occurs and any tool is

poorly able to discriminate the exact stage. Moreover, this

is not only a limitation of noninvasive tools but of biopsy

itself, given the small specimen often obtainable by

biopsy not well representative of the entire liver.

Furthermore, it is important to outline that the nomo-

gram is not a pure and absolute classification rule: it

provides the likelihood that a patient, given his/her

biomarker profile, belongs to a distinct outcome category.

Another purpose of the present work was to introduce

nomograms, original alternative tools to current noninvasive

fibrosis tests, available at the bedside. Nomograms are user-

friendly interfaces providing a measure of evidence of the

outcome of interest; they may be helpful for physicians in

validating their own judgments. These predictive tools are

being used in several medical fields such as oncology, in

which they are mainly used to predict outcomes or to stage

cancers [36]. They allow immediate assessment of the

impact of each prognostic factor for the specific outcomes,

which can be read on the length of single axis, thus

providing a sensible advantage over the so-called black-box

approaches. The interpretation of predictions is left to

practitioners also on the basis of patient profiles.

Here, an original nomogram (Fibro-Nomogram) was

developed for first-line evaluation of liver fibrosis. It

was based on clinical and laboratory data, routinely

available in patients affected by chronic HCV infection,

without any additional cost.

Comparisons between the results of Fibro-Nomogram and

FibroTest, on the validation set, were performed in the

second part of the study.

FibroTest has been validated widely in the HCV

population, with good performances globally, and is a

patented test for the diagnosis of liver fibrosis in adult

patients with untreated HCV. In the original paper,

Fibrotest obtained high positive and negative predictive

values for the diagnosis of clinically significant fibrosis

(FZ 2) and of cirrhosis or severe fibrosis (F3–F4).

On the validation set, both Fibro-Nomogram and

FibroTest showed substantially concordant results in the

prediction of extreme degrees of fibrosis. To facilitate the

comparison, as the prevalence of liver fibrosis stages is a

factor of variability in assessing the diagnostic value of a

fibrosis marker [37], attention has been focused on the

results with LRs. LRs can be less influenced by the

differences in the prevalence of diagnosis with respect to

predictive values (LR around 1 contribute scarce

significance to the post-test probability, LR around 2–5

mild increase, LR between 5 and 10 moderate increase,

and LR >10 large increase. For negative LR, less than the

unity, the smaller the LR, the greater the reduction in

post-test probability in excluding the diagnosis) [38]. In

the prediction of severe fibrosis, the model performed

slightly better than FibroTest, considering the higher

positive LR, although with a wide CI. Thus, our model

allowed to estimate simultaneously the probability

Nomogram for prediction of fibrosis in chronic HCV infection Sagrini et al. 9

Copyright © Lippincott Williams & Wilkins. Unauthorized reproduction of this article is prohibited.

of each outcome category (under the natural constraint,

these probabilities sum up to 1). If it is likely that a

patient does not have severe fibrosis, at the same time, it

is possible to derive the probability that he/she has

moderate or nil–mild fibrosis. If the latter probability is

high, a noninvasive procedure may be considered

sufficient. Vice versa, biopsy might be recommended for

a more precise assessment for patients initially predicted

to have no nil–mild fibrosis or advanced fibrosis, which

represent cases of uncertainty, when we concurrently

obtain from the nomogram similar probability values for

all three clinical outcomes. There is widespread con-

sensus that the use of a noninvasive test (serum

biomarkers or elastography) would be helpful if evidence

for cirrhosis or minimal to no fibrosis is indicated by these

tests. Should the results of noninvasive testing be

indeterminate, in our case when the ‘probability pattern’

is uncertain, then more prediction tools such as liver

biopsy would be relevant for stage confirmation.

Our predictive model was constructed on the basis of

simple laboratory values. We chose variables whose

association with the outcome of interest is biologically

plausible; age [3], GGT [12], AST [19], platelet

count [20], and serum albumin, routine biochemical

markers of liver disease, fulfilled this criterion as all of

them are known predictors of liver fibrosis.

The retrospective nature of this study and the collection

of data from different medical units did not allow for the

possibility to control for a series of confounding variables

that could have influenced the performance of the model.

A limitation could be the choice of variables, especially

AST and GGT, whose results may differ from day to day

because of transient fluctuations in hepatitis activity.

Fibro-Nomogram is a novel approach that could be used to

stratify patients with chronic HCV infection into three

clinically relevant probability categories. It showed good

overall performance, validated in an external cohort of

patients, especially in the confirmation of both extreme

stages of fibrosis. It was found to be comparable with

FibroTest results, but inexpensive and readily available.

These former issues should also be considered for countries

in which FibroTest or more expensive imaging is not

available. Moreover, the model could concurrently predict

the likelihood of different stages of fibrosis, at the same

time, at the bedside. We underline that nomograms are not

decision rules and like almost all proposed scores are

unlikely to entirely replace liver biopsy. From a clinical

standpoint, in a multimodality approach, the application of a

nomogram could help clinicians interpret patients’ profile.

AcknowledgementsThe analysis of data has been possible thanks to free

collaboration with ‘Unita di Statistica Medica, Biometria

e Bioinformatica – Fondazione IRCCS Istituto Nazionale

Tumori’ (Milan).

Elisabetta Sagrini, Alice Gianstefani, and Alessandra

Orlandini participated in collecting data and conception

and design of the study. Fabio Piscaglia, Elisabetta

Sagrini, and Alessandro Cucchetti participated in the

conception and design of the study. Elisabetta Sagrini,

Fabio Piscaglia, Ilaria Ardoino, Giuseppe Marano, and Elia

Biganzoli participated in drafting of the article. Ilaria

Ardoino, Giuseppe Marano, and Elia Biganzoli analyzed

data and carried out statistical analysis. Gabriele Donati,

Alessandro Cucchetti, Guido Pelosi, Alessandra Orlandini,

Carlo Ferrari, Alfredo Alberti, and Giada Sebastiani

provided pool of patients. All authors read and approved

the final manuscript.

Conflicts of interest

There are no conflicts of interest.

References1 Benvegnu L, Gios M, Boccato S, Alberti A. Natural history of compensated

viral cirrhosis: a prospective study on the incidence and hierarchy of majorcomplications. Gut 2004; 53:744–749.

2 Ikeda Y, Shimada M, Hasegawa H, Gion T, Kajiyama K, Shirabe K, et al.Prognosis of hepatocellular carcinoma with diabetes mellitus after hepaticresection. Hepatology 1998; 27:1567–1571.

3 Poynard T, Bedossa P, Opolon P. Natural history of liver fibrosis progressionin patients with chronic hepatitis C. The OBSVIRC, METAVIR, CLINIVIR,and DOSVIRC groups. Lancet 1997; 349:825–832.

4 Craxı A, Pawlotsky JM, Wedemeyer H, Bjoro K, Flisiak R, Forns X, et al. EASLClinical Practice Guidelines: management of hepatitis C virus infection.J Hepatol 2011; 55:245–264.

5 Davis GL, Lau JY. Factors predictive of a beneficial response to therapyof hepatitis C. Hepatology 1997; 26:122S–127S.

6 Marcellin P, Asselah T, Boyer N. Fibrosis and disease progression inhepatitis C. Hepatology 2002; 36:S47–S56.

7 Strader DB, Wright T, Thomas DL, Seeff LB. Diagnosis, management, andtreatment of hepatitis C. Hepatology 2004; 39:1147–1171.

8 Yano M, Kumada H, Kage M, Ikeda M, Shimamatsu K, Inoue O, et al. Thelong-term pathological evolution of chronic hepatitis C. Hepatology 1996;23:1334–1340.

9 Friedrich-Rust M, Ong MF, Martens S, Sarrazin C, Bojunga J, Zezem S, et al.Performance of transient elastography for the staging of liver fibrosis:a meta-analysis. Gastroenterology 2008; 134:960–974.

10 Wai CT, Greenson JK, Fontana RJ, Kalbfleisch JD, Marrero JA,Conjeevaram HS, et al. A simple noninvasive index can predict bothsignificant fibrosis and cirrhosis in patients with chronic hepatitis C.Hepatology 2003; 38:518–526.

11 Forns X, Ampurdanes S, Llovet JM, Aponte J, Quinto L, Martinez-Bauer E,et al. Identification of chronic hepatitis C patients without hepatic fibrosis bya simple predictive model. Hepatology 2002; 36:986–992.

12 Imbert-Bismut F, Ratziu V, Pieroni L, Charlotte F, Benhamou Y, Poynard T.Biochemical markers of liver fibrosis in patients with hepatitis C virusinfection: a prospective study. Lancet 2001; 357:1069–1075.

13 Vallet-Pichard A, Mallet V, Nalpas B, Verkarre V, Nalpas A, Dhalluin-Venier V,et al. FIB-4: an inexpensive and accurate marker of fibrosis in HCV infection.Comparison with liver biopsy and FibroTest. Hepatology 2007; 46:32–36.

14 Miller ME, Hui SL. Validation techniques for logistic regression model. StatMed 1991; 10:1213–1226.

15 Bedossa P, Poynard T. An algorithm for the grading of activity in chronichepatitis C. The METAVIR Cooperative Study Group. Hepatology 1996;24:289–293.

16 Venables WN, Ripley BD. Modern applied statistics with S. 4th ed. NewYork: Springer; 2002.

17 Biesheuvel CJ, Vergouwe Y, Steyerberg EW, Grobbee DE, Moons KG.Polytomous logistic regression analysis could be applied more often indiagnostic research. J Clin Epidemiol 2008; 61:125–134.

18 R Development Core Team. R: a language and environment for statisticalcomputing. Vienna, Austria: R Foundation for Statistical Computing; 2009.

19 Okuda M, Li K, Beard MR, Showalter LA, Scholle F, Lemon SM,Weinman SA. Mitochondrial injury, oxidative stress, and antioxidant gene

10 European Journal of Gastroenterology & Hepatology 2013, Vol 00 No 00

Copyright © Lippincott Williams & Wilkins. Unauthorized reproduction of this article is prohibited.

expression are induced by hepatitis C virus core protein. Gastroenterology2002; 122:366–375.

20 Adinolfi LE, Giordano MG, Andreana A, Tripodi MF, Utili R, Cesaro G, et al.Hepatic fibrosis plays a central role in the pathogenesis ofthrombocytopenia in patients with chronic viral hepatitis. Br J Haematol2001; 113:590–595.

21 Harrell FE Jr. Regression modelling strategies: with application to linearmodels, logistic regression, and survival analysis. New York: Springer; 2001.

22 Pepe M. The statistical evaluation of medical tests for classification andprediction. Oxford: Oxford University Press; 2003.

23 Bedossa P, Dargere D, Paradis V. Sampling variability of liver fibrosis inchronic hepatitis C. Hepatology 2003; 38:1449–1457.

24 Mehta SH, Lau B, Afdhal NH, Thomas DL. Exceeding the limits of liverhistology markers. J Hepatol 2009; 50:36–41.

25 Huwart L, Sempoux C, Vicaut E, Salameh N, Annet L, Danse E, et al.Magnetic resonance elastography for the noninvasive staging of liverfibrosis. Gastroenterology 2008; 135:32–40.

26 Piscaglia F, Salvatore V, Di Donato R, D’Onofrio M, Gualandi S, Gallotti A,et al. Accuracy of VirtualTouch Acoustic Radiation Force Impulse (ARFI)imaging for the diagnosis of cirrhosis during liver ultrasonography.Ultraschall Med 2011; 32:167–175.

27 Cales P, Oberti F, Michalak S, Hubert-Fouchard I, Rousselet MC, Konate A,et al. A novel panel of blood markers to assess the degree of liver fibrosis.Hepatology 2005; 42:1373–1381.

28 Adams LA, Bulsara M, Rossi E, DeBoer B, Speers D, George J, et al.Hepascore: an accurate validated predictor of liver fibrosis in chronichepatitis C infection. Clin Chem 2005; 51:1867–1873.

29 Haute Autorite de Sante. Non invasive methods for the evolution of hepaticfibrosis/cirrhosis: an update. Available at: http://www.has-sante.fr[Accessed 2 December 2012]; 2008.

30 Leroy V, Halfon P, Bacq Y, Boursier J, Rousselet MC, Bourliere M, et al.Diagnostic accuracy, reproducibility and robustness of fibrosis blood tests inchronic hepatitis C: a meta-analysis with individual data. Clin Biochem2008; 41:1368–1376.

31 Leroy V, Hilleret MN, Sturm N, Trocme C, Renversez JC, Faure P, et al.Prospective comparison of six non-invasive scores for the diagnosis of liverfibrosis in chronic hepatitis C. J Hepatol 2007; 46:775–782.

32 Parkes J, Guha IN, Roderick P, Rosenberg W. Performance of serummarker panels for liver fibrosis in chronic hepatitis C. J Hepatol 2006;44:462–474.

33 Castera L, Vergniol J, Foucher J, Le Bail B, Chanteloup E, Haaser M, et al.Prospective comparison of transient elastography, FibroTest, APRI, and liverbiopsy for the assessment of fibrosis in chronic hepatitis C.Gastroenterology 2005; 128:343–350.

34 Shaheen AA, Wan AF, Myers RP. FibroTest and FibroScan for the predictionof hepatitis C-related fibrosis: a systematic review of diagnostic testaccuracy. Am J Gastroenterol 2007; 102:2589–2600.

35 Friedrich-Rust M, Ong MF, Martens S, Sarrazin C, Bojunga J, Zeuzem S,et al. Performance of transient elastography for the staging of liver fibrosis:a meta-analysis. Gastroenterology 2008; 134:960–974.

36 Kattan MW, Yu C, Salomon L, Vora K, Touijer K, Guillonneau B. Developmentand validation of preoperative nomogram for disease recurrence within 5years after laparoscopic radical prostatectomy for prostate cancer. Urology2011; 77:396–401.

37 Poynard T, Halfon P, Castera L, Munteanu M, Imbert-Bismut F, Ratziu V, et al.Standardization of ROC curve areas for diagnostic evaluation of liver fibrosismarkers based on prevalences of fibrosis stages. Clin Chem 2007;53:1615–1622.

38 Grimes DA, Schulz KF. Refining clinical diagnosis with likelihood ratios.Lancet 2005; 365:1500–1505.

Nomogram for prediction of fibrosis in chronic HCV infection Sagrini et al. 11