Validation of a new scoring system for the assessment of clinical trial research of oral mucositis...

Validation of a New Scoring System for theAssessment of Clinical Trial Research of OralMucositis Induced by Radiation or Chemotherapy

Stephen T. Sonis, D.M.D., D.M.Sc.1

June P. Eilers, R.N., Ph.D.2

Joel B. Epstein, D.M.D., M.S.D.3

Francis G. LeVeque, D.D.S.4

William H. Liggett, Jr., D.M.D., D.M.Sc., M.D.5

Mary T. Mulagha, M.D.6†

Douglas E. Peterson, D.M.D., Ph.D.7

Ann H. Rose, Ph.D.8

Mark M. Schubert, D.D.S., M.S.D.9

Frederik K. Spijkervet, D.D.S., Ph.D.10

Janet P. Wittes, Ph.D.11

for the Mucositis Study Group

1 Division of Dentistry, Brigham and Women’s Hos-pital, Boston, Massachusetts.

2 University of Nebraska Medical Center, Omaha,Nebraska.

3 Vancouver General Hospital and the British Co-lumbia Cancer Agency, Vancouver, British Colum-bia, Canada.

4 Harper Hospital, Detroit, Michigan.

5 Johns Hopkins Oncology Center, Baltimore,Maryland.

6 Novartis Pharmaceuticals, Inc., East Hanover,New Jersey.

7 Bone Marrow Transplant Program and Depart-ment of Radiation Oncology, University of Connect-icut Health Center, School of Dental Medicine,Farmington, Connecticut.

8 OSI Pharmaceuticals, Inc., Washington, DC.

9 Fred Hutchinson Cancer Center and the Univer-sity of Washington School of Dentistry, Seattle,Washington.

10 University Hospital Groningen, Groningen, TheNetherlands.

11 Statistics Collaborative, Inc., Washington, DC.

Presented in part at the Annual Meeting of theDutch Association of Oral and Maxillofacial Sur-gery, November 1998.

Supported by Novartis Pharmaceuticals, Inc. andOSI Pharmaceuticals, Inc.

† Deceased.

The authors thank the following individuals whocontributed to the design of the scale: W. Kohn, C.McGarigle, E. Peters, S. Porter, H. Stoetter, G.Vogelsang, and D. Weissman.

The following are members of the Mucositis StudyGroup: J. Costa, J. Geier, A. Mekala, and S. Woo(Division of Dentistry, Brigham and Women’s Hos-pital, Boston, MA); A. Berger, C. Clarke, J. Kohtz, S.Kruse, and L. Wills (University of Nebraska MedicalCenter, Omaha, NE); A. Ransier, C. Sawyer (Van-couver General Hospital and the British ColumbiaCancer Agency, Vancouver, British Columbia, Can-ada); K. Damato, J. D’Ambrosio, R. Dowsett, R.Lalla, K. Nuki, C. Tafas, and P. Tutschka (BoneMarrow Transplant Program and Department ofRadiation Oncology, University of Connecticut

Health Center, School of Dental Medicine, Farm-ington, CT); R. Dansey and B. Klein (Harper Hos-pital, Detroit, MI); V. Almonte (Johns Hopkins On-cology Center, Baltimore, MD); M. Lloid (FredHutchinson Cancer Center and the University ofWashington School of Dentistry, Seattle, WA); E.Devries (University Hospital Groningen, Groningen,The Netherlands); H. Wysowskj (Novartis Pharma-ceuticals, Inc., East Hanover, NJ); and S. Jauffret(Statistics Collaborative, Inc., Washington, DC).

Dr. Liggett’s current address: 793 EKU Bypass,Richmond, KY 40475.

Address for reprints: Stephen T. Sonis, D.M.D.,D.M.Sc., Division of Dentistry, Brigham and Wom-en’s Hospital, 75 Francis Street, Boston, MA02115.

Received August 12, 1998; revision received Jan-uary 7, 1999; accepted January 19, 1999.

BACKGROUND. An impediment to mucositis research has been the lack of an

accepted, validated scoring system. The objective of this study was to design, test,

and validate a new scoring system for mucositis that can be used easily, is

reproducible, and provides an accurate system for research applications.

METHODS. A panel of experts, convened to design an objective, simple, and repro-

ducible assessment tool to evaluate mucositis with specific application to multi-

center clinical trials, developed a scale that measured objective and subjective

indicators of mucositis. Nine centers participated in the study’s validation. Paired

investigators at each center evaluated patients receiving chemotherapy or head

and neck radiation. Objective measures of mucositis evaluated ulceration/

pseudomembrane formation and erythema. Subjective outcomes of mouth pain,

ability to swallow, and function were measured. Analgesia use for mouth sensitivity

was recorded.

RESULTS. One hundred eight chemotherapy and 56 radiation therapy patients were

evaluated. Seventy-eight percent of chemotherapy patients and 64% of radiation

therapy patients had clinically significant mucositis. Cumulative daily mucositis

scores demonstrated a high correlation among observers. Using area under the

curve analysis, it was found that for chemotherapy patients, the highest correla-

tions (correlation coefficient . 0.92) occurred for the scores that selected the three

highest daily values over the course of mucositis assessment. High interobserver

correlations were noted for patients receiving radiation therapy. Objective mucosi-

tis scores demonstrated strong correlation with symptoms.

2103

© 1999 American Cancer Society

CONCLUSIONS. The scoring system evaluated was easily used, showed high inter-

observer reproducibility, was responsive over time, and measured those elements

deemed to be associated with mucositis. The use of concomitant symptomatic

measurements appeared to be unnecessary. Cancer 1999;85:2103–13.

© 1999 American Cancer Society.

KEYWORDS: mucositis, chemotherapy, radiation therapy, assessment.

W ith the availability of cytokine therapy to reducethe duration and severity of hematologic toxici-

ties associated with cancer chemotherapy, the impor-tance of oral mucositis as a significant side effect hasincreased dramatically. Not only are lesions caused bymucositis of such severity as to require routine paren-teral analgesic intervention, but they often result incancer therapy dose limitations.1-4 In addition, theconcomitant presence of breaks in the oral mucosa inan environment rich in microflora has resulted in thewell reported relation between the mouth and bacte-remias and sepsis among myelosuppressed patients.5

Taken as a whole, these factors also impact on hospitalstay and outcome.6 Chemotherapy produces an esti-mated overall incidence of 40%. Mucositis is evenmore frequent among patients undergoing radiationtherapy for tumors of the head and neck, affectingapproximately 80% of patients. Currently, there is noeffective intervention to prevent or treat mucositis.

We believe the dramatic increase in the number ofclinical studies regarding mucositis has occurred inthe past few years for three major reasons. First, bonemarrow-stimulating cytokine therapy has had a majoreffect on reducing granulocytopenia as a long termconsequence of cytoreductive therapy. Likewise, inter-leukin-11 for the treatment of chemotherapy-inducedthrombocytopenia may result in the effective controlof this side effect. The net result has been to increasethe significance of mucositis as a toxicity. Second,because of an increased understanding of the biologyof mucositis,7 investigators have identified severaltherapeutic opportunities to which biotechnology andpharmaceutical companies are responding with theapplication of innovative agents that are of potentialtherapeutic benefit.8 Finally, oncologists and radio-therapists aggressively are pursuing new agents, com-bination therapies, and dose escalations that are nowlimited by the development of mucositis.1-4

A major impediment to interventional and epide-miologic mucositis research has been the lack of anaccepted, validated, and objective scoring system formucositis.9 Instead, individuals and groups have de-veloped many scoring systems, often with differentobjectives. Review of current scoring systems suggeststhree major reasons for their development. The widelyused World Health Organization (WHO) and National

Cancer Institute (NCI) systems were developed to de-scribe toxicities associated with a particular chemo-therapeutic agent or regimen.10 Both scales combineobjective signs of mucositis (erythema and ulcer for-mation) with subjective and functional outcomes(pain and ability to eat). Although the developers ofthese scales intended the scorer to consider lesionspainful if analgesia masked the pain, in practice manyscorers ignore this differentiation, most likely resultingin underreporting and underscoring of mucositis.9

Oncology nurses have developed scoring systemsfor the assessment of mucositis and for patient man-agement.11,12 Many of these scales have a holisticquality; they include elements that are not definedtraditionally as being related directly to mucositis. Forexample, in addition to evaluation of the integrity ofthe oral mucosa, these scales often include functionaland subjective outcomes such as speech quality,avoidance of spicy foods, and swallowing as well as lipand mucosal dryness, infection, bleeding, and clean-liness.

Some research groups have attempted to de-velop mucositis scoring systems that are applicableas research tools.13,14 These systems have attemptedeither to eliminate subjective findings completely orto evaluate them independent of objective findingsand then integrate them into a comprehensivescore. As has been the case with some managementscales, several of the research scales have called forthe evaluation of individual sites in the mouth toproduce a single score.

Despite these attempts, mucositis researchershave not reached a consensus regarding an easilyused, accurate, and reproducible scoring system de-signed specifically for investigative applications in thisarea. Our objective was to design, test, and validate ascoring system for mucositis that met this purpose.

MATERIALS AND METHODSA panel of experts was convened to design an objec-tive, simple, and reproducible assessment tool to eval-uate oral mucositis with specific application to multi-center clinical trials. Attendees included a cross-section of health professionals (nurses, dentalhygienists, physicians, and dentists) who had pub-lished peer-reviewed research articles in the fields of

2104 CANCER May 15, 1999 / Volume 85 / Number 10

mucositis, oncology, and pain management, statisti-cians, and representatives of the pharmaceutical andbiotechnology industries. The attendees agreed on amucositis assessment scale that measured a consen-sus of indicators of mucositis severity. Primary indi-cators were the degrees of ulceration/pseudomem-brane and mucosal erythema measured in specificsites in the mouth (Table 1). Secondary indicatorsincluded oral pain, swallowing, and the ability to eatas assessed by the patient.

To describe the psychometric properties of thescale and to evaluate face validity, content validity,

reliability, responsiveness, and interpretability, thefollowing criteria were established as measurementoutcomes to determine the properties of the scale:

1. Face validity. Did the scale measure what expertsperceive as mucositis?

2. Content validity. Did the indicators sample the en-tire domain of mucositis? Did the scale miss anyimportant aspects of mucositis?

3 Reliability. Was there interobserver reproducibility?4. Responsiveness. Did the scale detect clinically im-

portant changes in mucositis?

TABLE 1Sample Data Collection Form Indicating the Parameters and Sites for the Objective Scoring Used forChemotherapy Patients

Chemotherapy

Patient ID Patient Initials

/ 1 /site regiment pt. # first middle last

Date: / / Time (24 hour clock): :mm dd yy

Patient is today: Inpatient Outpatient(circle)

Investigator 1

Location Ulceration/pseudomembrane* (circle) Erythema** (circle)

Upper lip 0 1 2 3 0 1 2Lower lip 0 1 2 3 0 1 2Right cheek 0 1 2 3 0 1 2Left cheek 0 1 2 3 0 1 2Right ventral and lateral tongue 0 1 2 3 0 1 2Left ventral and lateral tongue 0 1 2 3 0 1 2Floor of mouth 0 1 2 3 0 1 2Soft palate/fauces 0 1 2 3 0 1 2Hard palate 0 1 2 3 0 1 2

*Ulceration/Pseudomembrane:0 5 no lesion1 5 , 1 cm2

2 5 1 cm2–3 cm2

3 5 . 3 cm2

**Erythema:0 5 none1 5 not severe2 5 severe

NCI gradeWBC (m3)ANC (m3)

Presence of infection: Yes No If yes, circle: local non-oral systemic(circle)

Investigator’s Signature:

NCI: National Cancer Institute; WBC: leukocyte; ANC: absolute neutrophil count.

The same data were collected for patients receiving radiation therapy, although absolute neutrophil count was omitted.

New Mucositis Scoring System/Sonis et al. 2105

5. Interpretability. Did a particular score correlatewith the clinical condition of the patient?

In addition, user evaluation in terms of the ease withwhich observers used the scale was to be ascertained.

Study DesignNine centers participated in this study. Because theextent, characteristics, and time course of mucositisdiffer in patients undergoing chemotherapy and radi-ation therapy, the study was designed to have ade-quate statistical power to validate the scale in the twogroups of patients separately. Inclusion criteria wereestablished for both groups of patients. To be eligiblefor inclusion, chemotherapy patients must have re-ceived a drug regimen in which at least 50% of patientsdemonstrating Grade 3 or Grade 4 mucositis as de-fined by the WHO scale of the NCI’s Common ToxicityIndex (CTI) and have been inpatients during the che-motherapy treatment period. Patients receiving radi-ation therapy were required to have been scheduled toreceive a minimum of 50 grays. Patients of both gen-ders were eligible for study provided they were age $

12 years. Informed consent was obtained from allstudy participants.

Patients were distributed among the study sites; atmost, 25 evaluable patients for each arm could beenrolled at a single site. To assure a reasonable distri-bution of patients, we required that at least four sitesenroll a minimum of eight evaluable patients for thechemotherapy arm and at least three sites enroll aminimum of seven patients for the radiation therapyarm before enrollment was stopped.

At each site two trained investigators evaluatedeach patient for clinical manifestations of oral mucosi-tis. Investigators were required to attend a study ori-entation meeting prior to participating in the project.Although all investigators had extensive experience inthe oral care and assessment of patients, their educa-tional backgrounds varied. They included dentists, on-cology nurses, dental hygienists, and research assis-tants. At the time of each patient’s enrollment, oneindividual was designated as Investigator 1 and theother as Investigator 2. This pair of individuals re-mained constant throughout the evaluation period ofa specific patient; however, the roles could changefrom patient to patient. In addition to clinical scoring,Investigator 1 maintained responsibility for obtainingpatient assessments of pain and function.

Chemotherapy patients were evaluated begin-ning on the first day of treatment and continuing for28 calendar days. Investigator 1 evaluated the pa-tient daily (except on weekends and holidays) forthe first 12 days, after which time the patient was

evaluated each Monday, Wednesday, and Friday.Investigator 2 evaluated the patient on alternatedays beginning on the first day of chemotherapy forthe first week and then on Mondays, Wednesdays,and Fridays thereafter.

Patients receiving radiation therapy were evalu-ated differently to accommodate their dosing sched-ules. Patients were examined by Investigator 1 on theirfirst day of radiation therapy and then regularly until 2weeks after the last dose of radiation. Both investiga-tors evaluated patients on alternate days (exceptweekends and holidays) on days on which the patientreceived treatment.

Investigators were instructed to examine each pa-tient individually, but no longer than 4 hours apart.After obtaining the patient’s subjective assessment, aclinical examination was performed using a halogenlight source (Centauri Headlight Systems, Tempe, AZ).Investigator 1 obtained an NCI-CTI score on each dayof evaluation. Investigators were instructed not tocompare scores. Scoring sheets were placed in indi-vidual envelopes and sealed.

Each day, before investigators assessed oral mu-cositis, patients received a data collection form con-taining 2 visual analogue scales (VAS) comprised of a100-mm line with descriptors of severity at each end(Table 2). On the top and bottom lines, patients wereasked to place a mark corresponding to the degree oforal pain and ability to swallow, respectively, at thetime of assessment. After the completion of the VASscales, patients were asked to complete a short ques-tionnaire concerning their ability to eat (Table 2).

Daily analgesic use for oral pain was recorded forall patients as were leukocyte and absolute neutrophilcounts for chemotherapy patients. Analgesic level wasdefined by one of four levels: topical, nonsteroidalantiinflammatory drugs or equivalent, oral narcotics,or parenteral narcotics.

Data ManagementData collection sheets were mailed to a central site forprocessing. Data was double-entered using MicrosoftAccess (Microsoft Corporation, Redmond, WA) andthen imported into SAS software (SAS Institute, Cary,NC) to a second site for statistical analysis. An audit ofthe SAS database was performed independently. Aseries of SAS diagnostic tests were performed to de-scribe the distribution of variables. Out-of-range orquestionable values were confirmed against originaldata forms. All analyses were performed using PC SASVersion 6.12 or S-Plus for Windows, Version 4 (SASInstitute).

If on a given day an observer recorded data forsome, but not all, sites in the mouth, the analytic plan


specified that the missing observation was zero, as-suming that ulceration and erythema were absent.When an entire set of nine observations was missing,all observations were considered to be missing, notzero. During the course of the study, the investigatorsrealized that many of the missing sites were unmea-sured because the mouth was too painful to allowassessment. To represent the temporal course of mu-cositis accurately, the calculation of scores assumes

that a missing value for a site is simply missing; thus,the scores implicitly assign the missing values thesame value as the average of measured sites.

Four scores were analyzed:

1. The mean mucositis score was equal to (Sui/nu ) 1(Sei/ne), in which the summation was over the non-missing values. nu and ne were the number of lo-cations of nonmissing measurements of ulceration

TABLE 2Sample of the Collection Form for Subjective Data Regarding Mouth Pain and Swallowing

Radiation

Patient ID Patient initials

/ 2 /site regiment pt. # first middle last

Patient Diary Date: / / Time (24 hour clock): :mm dd yy

Mouth pain

Please indicate by a vertical line on the scale line below how severe the pain in your mouth is NOW

No pain Most severe pain

Impact on swallowing:

Please indicate by a vertical line on the scale below how well you can swallow.

No trouble in swallowing Cannot swallow anything at all (even saliva)

Please indicate how well you can swallow foods or liquids by checking below:

Function:

Normal

Only soft, solid foods

Only liquids

No foods or liquids

For the investigator: If patient did not fill in diary, please explain why.

I have reviewed this form: / /Investigator’s Signature mm dd yy

Patients completed this form on each evaluation day after standardized instructions from Investigator 1. Patients indicated their ability to function based on their eating ability in the diary at the bottom of the page.

Identical forms were completed by patients receiving chemotherapy or radiation therapy.


and erythema, respectively. This score ranged from0 –5.

2. The weighted mean mucositis score was equal to2.5[Suu/3nu) 1 (Sei/2ne)]. This score, which rangedfrom 0 –5, reweighed the previous score to equalizethe influence of ulceration and erythema.

3. The extent of mucositis score (the number of siteswith either ulceration 5 3 or erythema 5 2) rangedfrom 0 –9.

4. The worst site score was equal to the maximumerythema plus the maximum ulceration across allsites and ranged from 0 –5.

Several other scores were considered, but they hadvery little variability and hence would not be useful inclinical trials.

At the conclusion of data collection, investigatorswere asked to assess scale utility and to provide spe-cific comments with respect to certain aspects of scaleuse. Investigators also were given the opportunity toprovide other, nondirected comments.

RESULTSOne hundred eight chemotherapy patients and 56 ra-diation therapy patients were evaluated. Chemother-apy patients were distributed over all but one of thestudy sites. Four sites contributed radiotherapy pa-tients; 2 of these sites contributed 82% of the patients.The characteristics of patients in each cohort are de-scribed in Table 3. Patients receiving chemotherapytended to be younger than patients being treated withradiation therapy (44.8 years vs. 58.8 years). Not sur-prisingly, the gender ratios for each form of therapywere opposite: the male:female ratio was 39:61 forchemotherapy compared with 66:34 for radiation ther-apy.

The overwhelming majority of patients in the che-motherapy cohort were bone marrow transplant re-cipients (102 of 108); slightly more received allogeneictransplants (n 5 48) than autologous transplants (n 542). Among patients receiving radiation therapy, themandible and lower face were the most targeted ports.

More than two investigators participated at eachclinical site. The professional training of the investi-gators varied: 39% were dentists, 18% were nurses,28% were research dental hygienists, 10% were physi-cians, and 5% were research assistants. The averageamount of investigator experience in assessing theoral mucosa of patients receiving antineoplastic ther-apy was 9.6 years.

For the purposes of this study, clinically relevantmucositis was defined as oral mucositis having anyone of the following properties: 1) NCI-CTI Grade of atleast 2; 2) pain scale score, as measured by the patient

diary 100-point VAS for pain, of at least 50; 3) swal-lowing difficulty scale, as measured by the patientdiary 100-point VAS for swallowing, of at least 5; or 4)food intake limited by oral mucositis to liquids only.

Of all patients studied, 82% demonstrated someevidence of mucositis; 78% of chemotherapy patientsand 64% of radiation therapy patients had clinicallyrelevant mucositis.

Interobserver reproducibility was assessed atthree different levels: first, the score for a particularsite in the mouth on a particular day; second, thecumulative daily score for all sites; and third, cumu-lative scores over the course of mucositis. The twoinvestigators generally reported similar measurementsof individual oral sites (Tables 4 and 5). Although ahigh percentage of measurements were identical,some discordance was noted. The majority of discrep-ancies reflected a single unit scoring difference.

The evaluation of cumulative daily scores showeda high correlation of scores between the two investi-gators. Although the correlation between investigatorsscoring chemotherapy-induced mucositis was high, itwas lower than that for investigators scoring radia-tion-induced mucositis for each of the four outcomesconsidered.

To compare the two investigators with respectto assessments of the duration and severity of mu-

TABLE 3Baseline Characteristics of the Study Cohort

CharacteristicChemotherapy(N 5 108)

Radiation therapy(N 5 56)

Age (yrs)Mean 44.8 58.8Standard deviation 11.5 10.3Median 46 58Percentiles (25th, 75th) (36, 52) (52, 66)Min, Max (15, 77) (35, 82)

Gender (%)Male 39 66Female 61 34

Race (%)White 94 86Black 4 5Asian 2 4Other 1 5

Patient status at entryInpatient 94 13Outpatient 6 88

Reported mucositis history (%)a

Yes 30 —No 60 —Missing 10 —

Min: minimum; Max: maximum.a Mucositis history was not collected for radiation therapy patients.


TABLE 4Individual Measurements for Patients Receiving Chemotherapy by Investigator, in Which the Number of Matched Scores for Each Severity LevelWas Compared for Sites of Erythema and Ulceration

Erythema sites

Investigator 1

Investigator 2

None Not severe Severe Missing Total

None 5176 591 47 51 5865Not severe 678 719 195 15 1607Severe 66 214 602 12 894Missing 25 10 10 4 49Total 5945 1534 854 82 8415

Ulceration sites

Investigator 1

Investigator 2

No lesion < 1 cm2 1–3 cm2 > 3 cm2 Missing Total

No lesion 6950 189 48 14 44 7245# 1 cm2 290 181 57 17 6 551. 1–, 3 cm2 81 79 91 77 1 329$ 3 cm2 32 37 86 91 0 246Missing 26 5 5 3 5 44Total 7379 491 287 202 56 8415

TABLE 5Individual Measurements for Patients Receiving Radiation Therapy by Investigator, in Which the Number of Matched Scores for Each SeverityLevel was Compared for Sites of Erythema and Ulceration

Erythema sites

Investigator 1

Investigator 2

None Not severe Severe Missing Total

None 3818 284 38 26 4166Not severe 341 924 204 3 1472Severe 77 289 1280 8 1654Missing 9 2 3 74 88Total 4245 1499 1525 111 7380

Ulceration sites

Investigator 1

Investigator 2

No lesion < 1 cm2 1–3 cm2 > 3 cm2 Missing Total

No lesion 5774 112 20 17 16 5939# 1 cm2 177 373 69 23 2 644. 1–, 3 cm2 32 59 249 44 4 388$ 3 cm2 19 16 58 208 2 303Missing 37 2 1 2 64 106Total 6039 562 397 294 88 7380


cositis, we calculated the area under the curve de-scribing the time course of oral mucositis averagedover all chemotherapy and radiation therapy pa-tients. Only measurements made by both investiga-tors on the same day were included. All daily scoresand summaries across time demonstrated high cor-relations. For the chemotherapy patients, the high-est correlations (correlation coefficient [r] . 0.92)occurred for the scores that selected the three high-est values over the course of mucositis assessment.The areas under the curve for the entire course ofthe study (r . 0.80) and the maximum scores (r .0.84) of individual sites also showed high interob-server correlation (Tables 6 and 7).

Objective mucositis scores demonstrated strongcorrelations with symptoms associated with the con-dition. Thus mucositis scores were correlated highlywith NCI scores, pain, swallowing, and ability to eat(Table 8). This relation was present for the mean mu-cositis score, the weighted mean mucositis score, theworst site score, and the extent of mucositis score. Ofthe subjective findings measured, swallowing ap-peared to be most affected by analgesic use.

The data suggest that the scale was effective intracking temporal changes in mucositis. Plots gener-ally showed a close tracking of the mucositis scoreswith expected scores over time and with the observedpain, swallowing, and NCI scores. In general, the plotsfor the chemotherapy patients appeared to parallel

each other more closely than did the plots for theradiation therapy patients.

All examiners reported that the scale was easy touse. Completion of examination and grading took be-tween 1–5 minutes.

DISCUSSIONThe quest for an effective intervention for mucositisinduced by chemotherapy or radiation therapy neces-sitates the use of an assessment tool that measurestargeted outcomes accurately. Endpoints for existingscales, designed for toxicity assessment or patientmanagement, are not specific enough to meet re-search needs. The complexity of available scales forresearch use has hampered their routine adoption.

The scale tested in the current study was designedto provide a simple, quantitative, and accurate mech-anism for clinical mucositis assessment for use pri-marily as a research tool. Ease of use by investigatorsand tolerability by patients are two requirements of aneffective mucositis scale. Investigators reported thatthe scale was used easily; evaluations were completedin , 5 minutes. However, the repetitive examinationsproved to be difficult for patients with severe mucosi-tis and may have affected the second examiner’s abil-ity to inspect all the designated sites. This doubleexamination procedure would not be routine in typi-cal clinical trials but was required to ascertain inter-observer variability. In the current study, restricted

TABLE 6Interobserver Reproducibility for Chemotherapy Patients for Three Highest Values, Area under the Curve, and Maximum Score

Score r m b b-1 1 r-1 p 95% CI

Three highest values (N 5 108)Mean mucositis 0.98 20.04 0.96 0.06 0.74 [0.65, 0.82]Weighted mean mucositis score 0.98 20.04 0.96 0.06 0.74 [0.65, 0.82]Worst site 0.93 0.18 0.89 0.17 0.48 [0.38, 0.58]Extent of mucositis 0.96 20.04 0.94 0.10 0.53 [0.43, 0.62]

Area under the curveEntire course (N 5 108)

Mean mucositis 0.84 0.10 0.77 0.38 0.68 [0.58, 0.76]Weighted mean mucositis score 0.85 0.10 0.78 0.37 0.68 [0.58, 0.76]Worst site 0.85 0.20 0.78 0.37 0.57 [0.48, 0.67]Extent of mucositis 0.81 0.09 0.76 0.43 0.65 [0.55, 0.74]

Clinically relevant intervalEntire course (N 5 72)Mean mucositis 0.80 0.23 0.78 0.42 0.33 [0.38, 0.62]Weighted mean mucositis score 0.80 0.23 0.79 0.41 0.32 [0.37, 0.61]Worst site 0.76 0.51 0.74 0.50 0.23 [0.24, 0.47]Extent of mucositis 0.72 0.25 0.74 0.53 0.23 [0.24, 0.47]

Maximum (N 5 108)Mean mucositis 0.85 0.15 0.89 0.26 0.44 [0.35, 0.54]Weighted mean mucositis score 0.86 0.16 0.89 0.25 0.44 [0.34, 0.53]

r: correlation coefficient between measurements for the two investigators; m: intercept of the estimated regression line between investigators; b: slope of he estimated regression line between investigators; p:

proportion of [score Inv. 1 - score Inv. 2] # 0.25; 95% CI: 95% exact confidence interval for the proportion of scores within 0.25.


access to certain painful areas in a patient’s mouthmay have resulted in underscoring by the second in-vestigator.

The scale’s interobserver reproducibility wasnoted at each of the three levels assessed. In general,the two investigators reported similar measurementsfor individual oral sites. A high percentage of thesewere identical, although some discordance was noted.The majority of discrepancies reflected a difference ofa single unit. Discordance of more than one unit waslikely due to simple measurement error or the lack ofsuccinct site definition, which many have led to op-posite scoring of contiguous areas. For example, if

severe erythema was present on the labial mucosa andextended to the buccal mucosa, one investigator mayhave scored the labial mucosa as severe but under-scored the buccal mucosa whereas the second inves-tigator did the opposite. The establishment of specificsite definitions may eliminate this problem, althoughit did not appear to be significant enough to effectinterobserver reproducibility markedly.

The correlation between investigators also wasexcellent for daily mucositis scores. This trend wastrue for all mucositis scores (mean, weighted mean,worst site, or extent of severe mucositis). Eliminationof scores of 0 did not lower this correlation markedly.

TABLE 7Interobserver Reproducibility for Radiation Therapy Patients for Three Highest Values, Area under the Curve, and Maximum Score

Score r m b b-1 1 r-1 p 95% CI

Three highest values (N 5 53)Mean mucositis 0.99 20.03 1.00 0.01 0.94 [0.84, 0.99]Weighted mean mucositis score 1.00 20.02 1.00 0.01 0.96 [0.87, 1.00]Worst site 0.97 20.01 1.01 0.03 0.64 [0.50, 0.77]Extent of mucositis 0.98 20.09 1.00 0.03 0.51 [0.37, 0.65]

Area under the curveEntire course (N 5 53)

Mean mucositis 0.96 0.00 0.93 0.10 0.85 [0.72, 0.93]Weighted mean mucositis score 0.96 0.02 0.92 0.12 0.87 [0.75, 0.95]Worst site 0.96 0.02 0.95 0.09 0.70 [0.56, 0.82]Extent of mucositis 0.93 0.05 0.84 0.23 0.77 [0.64, 0.88]

Clinically relevant intervalEntire course (N 5 36)Mean mucositis 0.95 0.04 0.90 0.15 0.51 [0.58, 0.88]Weighted mean mucositis score 0.96 0.02 0.92 0.12 0.55 [0.64, 0.92]Worst site 0.96 0.24 0.91 0.13 0.51 [0.58, 0.88]Extent of mucositis 0.95 0.06 0.92 0.13 0.53 [0.61, 0.90]

Maximum (N 5 53)Mean mucositis 0.93 0.10 0.97 0.09 0.62 [0.48, 0.75]Weighted mean mucositis score 0.92 0.15 0.97 0.11 0.58 [0.44, 0.72]

r: correlation coefficient between measurements for the two investigators; m: intercept of the estimated regression line between investigators; b: slope of the estimated regression line between investigators; p:

proportion of [score Inv. 1 - score Inv. 2] # 0.25; 95% CI: 95% exact confidence interval for the proportion of scores within 0.25.

TABLE 8P Values from Analysis of Variance for the Four Candidate Scores: Relation on a Day-to-Day Basis to Pain, Swallowing, and NCI Score

Source

Mean mucositis Weighted mean Worst site Extent

Chemo Radio Chemo Radio Chemo Radio Chemo Radio

NCI , 0.001 0.002 , 0.001 0.002 , 0.001 0.008 , 0.001 0.014NCI 3 analgesic 0.087 0.041 0.18 0.057 0.001 0.50 0.042 0.002Pain , 0.001 0.089 , 0.001 0.0191 , 0.001 0.44 , 0.001 0.30Pain 3

analgesic , 0.001 0.005 0.005 0.012 0.37 0.35 0.006 0.38Swallowing , 0.001 0.010 , 0.001 0.010 , 0.001 0.59 0.001 0.041Swallowing 3

analgesic , 0.001 , 0.001 , 0.001 , 0.010 , 0.001 0.073 , 0.001 0.003

NCI: National Cancer Institute; chemo: chemotherapy; radio: radiation therapy.


The discordance between observers was higher forchemotherapy patients than for those receiving radi-ation therapy. At least two possible explanations areconsistent with this observation. First, the majority ofradiation therapy patients were assessed at two studylocations whereas chemotherapy patients were dis-tributed more evenly among six study centers. Thusfewer investigators scored the radiation therapy pa-tients. Another possible reason is that the mucosalresponse to radiation therapy tends to be more pre-dictable and defined than the response to chemother-apy; hence, scoring of the former is easier.

The accurate evaluation of temporal changes inclinical status often is important in assessing treat-ment outcome. Investigators showed high correlationin scores across time. The highest correlation (r .0.92) occurred for those scores that selected the threehighest values over the course of the measurement.Both the area under the curve (r . 0.80) and maxi-mum score analyses (r . 0.84) also showed high in-terobserver correlation. Interobserver reproducibilitydemonstrated acceptable correlations for each of thethree levels studied. All correlations were highly sig-nificant (P , 0.001).

The current study data demonstrate that the vari-ables measured correlated with the occurrence of mu-cositis as defined by the symptomatic parameters ofpain and swallowing as assessed by VAS analysis. Al-though scoring of objective changes was expected tobe correlated highly with the VAS scales, some pa-tients had discordant subjective scale and clinicalscale scores. The most likely explanation for discor-dance with the pain scale was the increased use ofanalgesics among patients with significant oral pain.Discordance of the scores derived from the swallowingand eating scales may have been due to the inability ofthe new scale to measure esophageal mucositis orlesions inferior to the oral pharynx.

Each form of mucositis score evaluated (mean,weighted mean, worst site, and extent of mucositis)showed highly statistically significant associations(P , 0.001) with pain, swallowing, and NCI score inchemotherapy patients. As noted earlier, analgesic useaffected the relation of these variables to the scalescore. It is interesting to note that these associationswere less strongly significant for the radiation therapypatients. These observations are of importance in sup-porting the hypothesis that a research tool measuringmucosal integrity and health that is dependent on ormodified by symptomatic outcomes could result inmisleading conclusions relative to the efficacy of aparticular pharmacologic agent in patients who useanalgesics to control oral symptoms.

The data suggest that the new scoring system ef-

fectively measures changes in mucosal health overtime. Of the measurements used, worst site and extentof severe mucositis appeared to be more responsive tochange than mean mucositis score. The fact that foreach day a number of intraoral sites were scored aszero and thus dampened the mean may be an expla-nation for this observation. This pattern appeared inboth chemotherapy and radiation therapy patients.Temporal plots of clinical changes associated withmucositis and measures of symptoms were well cor-related. In fact, in the chemotherapy patients the newscale score and the NCI scores virtually were super-imposable; although there appeared to be some in-consistencies among the radiation therapy patients,there nevertheless were similar scoring trends.

The new scale scores and the NCI score wereclosely related. This was in some ways surprising giventhe starting hypothesis that the accuracy of the NCIscoring system had the inherent weakness of beingmodified by analgesic symptom relief in the face ofsubstantive mucosal disruption. Two elements in thecurrent study may explain the extraordinary concor-dance of the NCI and new scale score. First, the studyinvestigators were highly experienced and directed inthe evaluation of the oral cavity. In addition, the in-tensity and thoroughness of the oral examinationmost likely was greater than is performed typically.Consequently, the accuracy of the reported NCI scoremay have been greater than is noted routinely. Sec-ond, the sequence of scoring also may have influencedthe NCI score because NCI scoring was performedafter the examination of the nine study sites dictatedby the new scoring system.

The mean and weighted mean had similar prop-erties. The mean, which is simpler to interpret, mea-sured the average level of mucositis. The maximummucositis score was less reproducible than the others.The extent of severe mucositis focused on the mostsevere sites.

The data as a whole indicate that both the meanmucositis score and the extent of severe mucositisscore, calculated over time either as the area under thecurve or as the average of the three highest values,produced scores that were reproducible and respon-sive to change. The score with the best statistical prop-erties was the average of the three highest measure-ments of the extent of severe mucositis; however, itwas not sensitive to low grades of oral mucositis. Thisfinding would not appear to impact on the scale’sutility as applied to clinical trials because, generally,efficacy of a study drug is measured by changes inclinically significant mucositis. The necessity to in-clude all sites in an evaluation was substantiated byour inability to identify one or more specific sites that


could serve as indicators for the remainder of the oralcavity.

We believe that the mucositis scale presented canserve as an effective research tool for studies evaluat-ing changes in the oral mucosa in response to chemo-therapy or radiation therapy. The scale is easy to use.It showed high interobserver reproducibility, was re-sponsive over time, and measured those elementsdeemed to be associated with mucositis. The use ofconcomitant symptomatic measurements by VAS andquestionnaires is likely to be unnecessary becausethere was a strong correlation between objective find-ings and pain, swallowing, and eating.

REFERENCES1. Denham JW, Abbott RL. Concurrent cisplatin, infusional

fluorouracil, and conventionally fractionated radiation ther-apy in head and neck cancer: dose-limiting mucosal toxicity.J Clin Oncol 1991;9:453– 63.

2. Vaughan WP, Kris E, Vose J, Bierman PJ, Gwilt P, ArmitageJO. Phase I/II study incorporating intravenous hydroxyureainto high-dose chemotherapy for patients with primary re-fractory or relapsed and refractory intermediate-grade andhigh-grade lymphoma. J Clin Oncol 1995;13:1089 –95.

3. Uziely B, Jeffers S, Isacson R, Kutsch K, Wei-Tsao D, Ye-hoshua Z, et al. Liposomal doxorubicin: antitumor activityand unique toxicities during two complementary phase Istudies. J Clin Oncol 1995;13:1777– 85.

4. Seiter K, Kemeny N, Martin D, Schneider A, Williams L,Colofiore J, et al. Uridine allows dose escalation of 5-flu-orouracil when given with N-phosphonacetyl-L-asparate,methotrexate, and leucovorin. Cancer 1993;71:1875– 85.

5. Eting LS, Bodey GP, Keefe BH. Septicemia and shock syn-drome due to viridans streptococci: a case-control study ofpredisposing factors. Clin Infect Dis 1992;14:1201–7.

6. Reuscher TJ, Sodeifi A, Scrivani SJ, Kaban LB, Sonis ST. Theimpact of mucositis on a-hemolytic streptococcal infectioninpatients undergoing autologous bone marrow transplan-tation for hematologic malignancies. Cancer 1998;82:2275–81.

7. Sonis ST. Mucositis as a biological process: a new hypoth-esis for the development of chemotherapy-induced stoma-totoxicity. Eur J Cancer 1998;34:39 – 43.

8. Sonis ST, Haley JD. Pharmacological attenuation of chemo-therapy-induced mucositis. Exp Opin Invest Drugs 1996;5:1155– 62.

9. Dodd MJ, Facione NC, Dibble SL, MacPhail L. Comparisonof methods to determine the prevalence and nature of oralmucositis. Cancer Pract 1996;4:312– 8.

10. World Health Organization. Handbook for reporting resultsof cancer treatment. Geneva: World Health Organization,1979:15–22.

11. Eilers J, Berger A, Petersen M. Development, testing andapplication of the oral assessment guide. Oncol Nurs Forum1988;15:325–30.

12. Dyck S, Brett K, Davies B. Development of a staging systemfor chemotherapy-induced stomatitis. Western Consortiumfor Cancer Nursing Research. Cancer Nurs 1991;14:6 –12.

13. Schubert MM, Williams BE, Lloid ME, Donaldson G, ChapkoMK. Clinical assessment scale for the rating of oral mucosalchanges associated with bone marrow transplantation. De-velopment of an oral mucosal index. Cancer 1992;69:2469 –77.

14. Spijkervet FK, van Saene HK, Panders AK, Vermey A, MehtaDM. Scoring irradiation mucositis in head and neck cancerpatients. J Oral Pathol Med 1989;18:167–71.


Validation of a new scoring system for the assessment of clinical trial research of oral mucositis...

Documents

Transcript of Validation of a new scoring system for the assessment of clinical trial research of oral mucositis...