http://ajs.sagepub.com/Medicine

The American Journal of Sports

http://ajs.sagepub.com/content/40/7/1611The online version of this article can be found at:

 DOI: 10.1177/0363546512446591

2012 40: 1611 originally published online May 10, 2012Am J Sports MedPieter Bas de Witte, Jan Ferdinand Henseler, Jochem Nagels, Thea P.M. Vliet Vlieland and Rob G.H.H. Nelissen

Reliability and Responsiveness Validation StudyThe Western Ontario Rotator Cuff Index in Rotator Cuff Disease Patients : A Comprehensive

  

Published by:

http://www.sagepublications.com

On behalf of: 

  American Orthopaedic Society for Sports Medicine

can be found at:The American Journal of Sports MedicineAdditional services and information for    

  http://ajs.sagepub.com/cgi/alertsEmail Alerts:

 

http://ajs.sagepub.com/subscriptionsSubscriptions:  

http://www.sagepub.com/journalsReprints.navReprints:  

http://www.sagepub.com/journalsPermissions.navPermissions:  

What is This? 

- May 10, 2012OnlineFirst Version of Record  

- Jul 2, 2012Version of Record >>

at Universiteit Leiden \ LUMC on April 28, 2013ajs.sagepub.comDownloaded from

The Western Ontario Rotator Cuff Indexin Rotator Cuff Disease Patients

A Comprehensive Reliability and ResponsivenessValidation Study

Pieter Bas de Witte,*y MD, BSc, Jan Ferdinand Henseler,y MD, Jochem Nagels,y MD,Thea P.M. Vliet Vlieland,y MD, PhD, and Rob G.H.H. Nelissen,y MD, PhDInvestigation performed at the Department of Orthopaedics,Leiden University Medical Center, Leiden, the Netherlands

Background: The Western Ontario Rotator Cuff Index (WORC) is an increasingly applied condition-specific outcome measure forrotator cuff (RC) conditions. However, in most WORC validation studies, only a limited number of psychometric properties arestudied in indistinct patient groups.

Purpose: To assess psychometric properties of the WORC according to the Scientific Advisory Committee quality criteria forhealth questionnaires in 3 patient groups with distinct RC conditions.

Study Design: Cohort study (diagnosis); Level of evidence, 2.

Methods: The WORC (range, 0-100; 21 items, 5 domains) was administered twice (T1, T2) in 92 patients (35 RC tears, 35calcific tendinitis, 22 impingement). Additionally, the Constant score (CS) and the Disabilities of the Arm, Shoulder andHand score (DASH) were recorded. Calcific tendinitis patients were reassessed 6 weeks after treatment with needling andlavage or a subacromial injection with corticosteroids (T3). We assessed floor and ceiling effects, internal consistency,test-retest reliability, precision, construct validity, minimally detectable change, and responsiveness in the diagnostic sub-groups and the total group.

Results: Mean age was 55.0 6 8.7 years, and 49 of 92 (53%) patients were female. Mean baseline WORC was 46.8 6 20.4, CSwas 63.9 6 15.4, and DASH was 40.9 6 18.6. Significant differences were found for the CS and DASH between RC tear patients(severe symptoms) and the other patients, but not for the WORC. There were no floor and ceiling effects. Internal consistency washigh: the Cronbach alpha coefficient was .95. The intraclass correlation coefficient of .89 and standard error of measurement of6.9 indicated high reproducibility. Pearson correlations of the WORC with the CS and DASH were .56 and –.65, respectively (bothP \ .001). At T3, total WORC improved significantly (mean change, 18.8; 95% confidence interval, 11.3-26.2). Correlations of theWORC change scores with CS and DASH changes were .61 and –.84, respectively (both P \ .001). Effect size was 0.96, witha standardized response mean of 0.91, indicating good responsiveness.

Conclusion: Applied to a variety of RC patients, the WORC had high internal consistency, moderate to good construct validity,high test-retest reliability, and good responsiveness. These findings support the use of the WORC as a condition-specific self-reported outcome measure in RC patients, but its validity in patients with severe symptoms needs further investigation.

Keywords: rotator cuff; WORC; quality of life; reliability; responsiveness; validity; shoulder; questionnaire

Shoulder problems, rotator cuff conditions in particular,are common musculoskeletal disorders with a high socio-economic effect. The incidence of shoulder complaints ingeneral practice is 22 per 1000 patients per year.45 Rotatorcuff conditions cover over 44% to 65% of these shouldercomplaints,28 with subacromial impingement syndrome,rotator cuff tendon tears, and calcific tendinitis as itsmost frequently diagnosed forms. Young athletic peopleand active members of society are often affected.8,38

Despite high incidence rates and the ensuing high numberof ongoing rotator cuff research projects worldwide, thereare currently few validated outcome measures focusing

*Address correspondence to Pieter Bas de Witte, MD, Department ofOrthopaedics, Leiden University Medical Center, Postzone J11R, Postbus9600, 2300 RC Leiden, the Netherlands (e-mail: p.b.de_witte@lumc.nl).

yDepartment of Orthopaedics, Leiden University Medical Center,Leiden, the Netherlands.

One or more of the authors has declared the following potential con-flict of interest or source of funding: This study is part of a larger projectfunded by ZonMw, the Netherlands organization for health research anddevelopment (NOW) (grant number 40-00703-98-8564) and the DutchArthritis Association (grant number 09-1-303).

The American Journal of Sports Medicine, Vol. 40, No. 7DOI: 10.1177/0363546512446591� 2012 The Author(s)

1611 at Universiteit Leiden \ LUMC on April 28, 2013ajs.sagepub.comDownloaded from

on rotator cuff injuries. For accurate patient assessment,however, it is advisable to combine a general health out-come measure, a general regional outcome measure, anda condition-specific outcome measure.52

In 2003, Kirkley et al26 introduced the English languageversion of the Western Ontario Rotator Cuff Index(WORC): a condition-specific self-reported instrument toassess the quality of life (QOL) of patients with shouldercomplaints as a consequence of rotator cuff disease. It com-prises 21 visual analog scale (VAS) items in 5 domains:physical symptoms, sports/recreation, work, lifestyle, andemotions. All items represent QOL aspects that can partic-ularly be influenced by rotator cuff injury. The domains arebased on the World Health Organization definition ofhealth.

The WORC is an increasingly applied outcome measurefor rotator cuff conditionsz and has been translated and val-idated in several languages, including Dutch, Portuguese,Norwegian, Persian, Turkish, and German.15,18,19,23,32,39,49

However, most studies describing the psychometric proper-ties of either the original or translated WORC versions havesome limitations: patient groups are small, not well defined,or include only patients with the same rotator cuff conditionand similar symptoms. Furthermore, in most validationstudies, only a limited set of psychometric properties istaken into account. Hence, the goal of this study was to eval-uate a comprehensive combination of the most relevant psy-chometric properties of the WORC, according to theproposed Scientific Advisory Committee quality criteria forpsychometric properties of health questionnaires,31,46 bycomparing outcome scores at several follow-up moments ina heterogeneous but strictly defined group of patients witha broad spectrum of rotator cuff conditions of varying sever-ity, including subacromial impingement syndrome, rotatorcuff tendon tears, and calcific tendinitis.

MATERIALS AND METHODS

Study Design

From April 2010, all consecutive patients referred by pri-mary health care providers for the treatment of shoulderpain with arm abduction were assessed for participationin 1 of 3 rotator cuff disease research projects, dependingon their underlying diagnosis after usual care investiga-tions. Study 1 (Trial registry no. NTR1545) was a cross-sectional study on muscle activation patterns in patientswith a full-thickness rotator cuff tear versus healthy con-trols.13 Study 2 (NTR2283) was a cross-sectional study onthe etiological mechanisms of subacromial impingementsyndrome with the use of questionnaires on shoulder func-tion, radiographs, magnetic resonance imaging (MRI), andbiomechanical methods.12 Study 3 (NTR2282) was anintervention study on the effectiveness of ultrasound-guided needle puncture, aspiration, lavage, and a subacro-mial injection with corticosteroids in patients with rotatorcuff calcific tendinitis. In all 3 studies, the WORC,

Disabilities of the Arm, Shoulder and Hand score(DASH),24 and Constant score (CS)9 were used in combina-tion with various other study-specific outcome measures.Other than with regard to the current WORC assessments,the 3 studies were independent, and there was no overlapin included patients between the studies. Data of allpatients in the 3 research projects until July 2011 wereincluded in the current WORC study. The Medical EthicsCommittee approved all 3 study protocols, and writteninformed consent was obtained from all participants.

Patients

Each of the 3 rotator cuff condition research projects hadits specific inclusion and exclusion criteria, partially over-lapping, with the general selection of patients based onusual care history taking, physical examination, and stan-dard shoulder radiographs (anteroposterior in both exter-nal and internal rotation).12,13 For all studies, patientshad one or more of the following criteria present, apartfrom a positive Neer impingement test result, a positiveHawkins test result, and diffuse unilateral shoulder painfor .3 months: pain during activities with arm abduction,extension, and/or internal rotation (eg, closing the door,putting on jacket, overhead activities); pain at night orincapable of lying on the shoulder; diffuse pain at palpationof the greater tuberosity; disturbed scapulohumeralrhythm; classic painful arc; positive Yocum test finding53;and positive full or empty can test finding.

For all 3 studies, patients were excluded in cases ofinsufficient language skills or no informed consent, anyform of inflammatory arthritis of the shoulder, glenohum-eral or acromioclavicular osteoarthritis, a history of surgi-cal interventions of the affected shoulder, clinical signs ofcervical radiculopathy, glenohumeral instability, and fro-zen shoulder syndrome (\90� of passive abduction andexternal rotation).

With respect to study-specific inclusion and exclusioncriteria, patients had to be aged 18 to 75 years for study1, 35 to 65 years for study 2, and 18 to 65 years for study3. Moreover, patients in study groups 1 and 2 had anMRI-arthrogram for usual care diagnostic purposes, whichwas also used for assessing eligibility criteria of the con-cerning studies. In study 1, all patients had a symptomaticfull-thickness rotator cuff tear that was nonresponsive tononoperative treatment. Patients with an intact rotatorcuff or a partial tear could be included in study 2. Further-more, for both studies 1 and 2, patients were excluded incase of calcific tendinitis or alternative diagnoses onMRI, including intra-articular and bony lesions (Hill-Sachs, [old] fractures, tumors), labrum abnormalities, cap-sular or ligamentous tears/avulsions, superior labral tearsfrom anterior to posterior (SLAP lesions), pulley lesions,biceps tendinitis or tears, os acromiale, cartilage lesions,and bony cysts. As calcific tendinitis can be demonstratedon standard radiographs, it can be distinguished frommost other causes of shoulder pain without MRI. There-fore, in study 3, patients underwent radiographs andultrasound-guided evaluation of the shoulder and werezReferences 1, 2, 5-7, 10, 14, 17, 20, 22, 29, 34, 36, 43, 50.

1612 de Witte et al The American Journal of Sports Medicine

at Universiteit Leiden \ LUMC on April 28, 2013ajs.sagepub.comDownloaded from

excluded in cases of other pathological abnormalities,including cuff tendon tears and biceps tendinitis.

Assessments

Patients were assessed at 3 time points: T1 (within 2 weeksbefore the scheduled outpatient visit), T2 (at the scheduledoutpatient visit), and T3 (6 weeks after treatment; only forpatients included in study 3). At T1, the WORC and DASHwere administered. These were sent to patients by regularmail, 7 to 14 days before the scheduled outpatient visit.Patients were requested, over the telephone and by regularmail, to complete the questionnaires at least 3 days beforethe visit. Patients received written instructions to the ques-tionnaires and did not receive any help with filling them out.At T2, patient characteristics (age, gender, arm dominance,affected arm, and duration of symptoms) were recorded, andclinical measures (including the CS) were obtained at theoutpatient clinic of the Department of Orthopaedics by theinvestigating researcher. Moreover, the WORC was againadministered at T2 for test-retest evaluation.

In the calcific tendinitis study, the WORC, DASH, andCS were recorded once more at 6 weeks after treatment:the WORC and DASH were sent by regular mail beforethe scheduled outpatient clinic visit. The CS was recordedat the outpatient clinic by the investigating researcher.These T3 data were used for responsiveness evaluation.

Western Ontario Rotator Cuff Index. The WORC isa self-reported disease-specific QOL measure, comprising21 items in 5 domains: physical symptoms (6 items), sportsand recreation (4 items), work (4 items), lifestyle (4 items),and emotions (3 items).26 Each item is scored on a 0- to10-cm VAS (the higher the rating, the higher the negativeeffect on QOL), summing up to a minimum total score of0 and a maximum total score of 2100 (worst possible). Ina more clinical comprehensible format, the maximum scorecan be expressed as a percentage score by subtracting thetotal score from 2100 and dividing by 2100, leading to totaloutcomes ranging from 0 (worst possible) to 100 (best possi-ble).26 In case of one missing value in a domain, the domainscore can be calculated using the average of the other itemsin the domain. In case of more than 2 missing items ina domain, the concerning WORC questionnaire is consideredincomplete and must be excluded from analyses.

In this study, we used the Dutch translation ofthe WORC. Permission was granted from both thedevelopers of the original WORC questionnaire and thetranslators.22,26,49

Constant Score. The CS is filled out by the physicianand combines objective physical examination tests andsubjective patient assessments.9 Thirty-five points areallocated for patient-reported items on pain (15 points)and activities of daily living (20 points), 40 points are avail-able for 4 physical examination items focused on (painless)range of motion, and 25 points are available for abductionstrength evaluation. Consequently, the total maximumscore is 100 points. Arm strength was measured usinga handheld dynamometer (MicroFET 2, Biometrics,Almere, the Netherlands).

Disabilities of the Arm, Shoulder and Hand Score. TheDASH is a self-reported questionnaire to measure disabilityand symptoms in patients with any or several musculoskel-etal disorders of the upper arm.24 The score contains a totalof 30 items; 21 items relate to physical function, 5 to clinicalsymptoms, and 4 to social and work-related activities. Eachitem is scored on a 5-point scale, ranging from no difficulty(1 point) to inability (5 points). The total score can be calcu-lated with a formula by the designers and ranges from0 (best score) to 100 (worst score).

Statistical Analysis

Normally distributed values were expressed using meanvalues and standard deviations, and skewed data wereexpressed using medians and ranges. The baseline sociode-mographics and clinical characteristics of patients in the 3diagnostic study groups were compared with 1-way analy-ses of variance or x2 tests where appropriate.

Floor and ceiling effects of the total WORC score, WORCdomain scores, and individual WORC items at T1 wereassessed by calculating the proportion of participants scor-ing the minimal or maximal scores, relative to the totalnumber of participants. For maximal scores, we applied per-centage scores of 90% to 100%, and similarly, we used 0% to10% percentage scores for the minimal scores. A percentagevalue of .15% of the participant scoring maximal or mini-mal scores was considered a relevant floor or ceiling effect.

Internal consistency of the items comprising the totalWORC score and the WORC domain scores was examinedby computing Cronbach alpha (a) coefficients at T1. Thiscoefficient assesses whether items within each domain orwithin the total WORC produce similar/correlating scores,contributing to and correlating with the domain and totalWORC scores, respectively.10 The Cronbach a rangesfrom .0 for poor correlation to 1.0 for best correlation. How-ever, high values are not necessarily desirable, as thismight indicate redundancy of questionnaire items.

Reliability was determined by comparing the test-retestWORC scores (T1, T2) by means of paired t tests or Wil-coxon signed-rank tests in case of nonparametric data.An instrument is considered reliable if it gives similar out-comes over time for each participant, provided that thereare no changes in the measured items over time. Addition-ally, the intraclass correlation coefficient (ICC) wasapplied. The ICC ranges from 0 to 1.00, with .00 to .39for poor, .40 to .59 for fair, .60 to .74 for good, and .75 to1.00 for excellent reliability. In this study, a 2-way random-effects model for agreement ICC (2,1) for single measure reli-ability was used for each domain.44 This model includespotential systematic differences in its analyses. Furthermore,the precision of the WORC was expressed in a standard errorof measurement (SEM), which can be estimated by the fol-lowing formula (equation 1),

SEM ¼ SD3ffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffi

1� ICCp

; ð1Þ

where SD is the pooled standard deviation of the test andretest measurements of all participants. The SEM gives an

Vol. 40, No. 7, 2012 WORC in Rotator Cuff Disease Patients 1613

at Universiteit Leiden \ LUMC on April 28, 2013ajs.sagepub.comDownloaded from

absolute measure of reliability within participants. In con-trast, the ICC gives a relative measure of reliability withinparticipants and depends on the population it is calculatedfrom.47 Furthermore, the SEM can be used to determinethe minimally detectable change (MDC),11 also reportedas the minimum difference to be considered real47 or small-est real difference4 (equation 2):

MDC ¼ SEM31:963ffiffiffi

2p

: ð2Þ

Validity of the WORC was assessed using construct val-idity, as there is no gold standard for a patient’s status ofrotator cuff–related QOL. Pearson correlation coefficients(or Spearman rank correlation coefficients in case of non-parametric data) were computed between total and domainscores of the WORC at T1 and the DASH (T1) and the CS(T2). In this study, positive correlations were defined as r =0 to .25 as poor, r = .25 to .5 as fair to moderate, r = .5 to .75as moderate to good, and r � .75 as good to excellent corre-lation. Negative correlations were defined in a similar way.The calculation of the association of the WORC and CS wasrepeated using the T2 WORC.

Lastly, responsiveness of the WORC was assessed bycomparing WORC scores before and 6 weeks after treat-ment in the calcific tendinitis group. The magnitudes ofthe changes in total and domain WORC scores wereexpressed as effect size (mean test-retest difference, dividedby the SD of the test mean) and standardized responsemean (SRM) (mean test-retest difference, divided by theSD of the mean change in score).25,30 Both outcomes canbe interpreted as follows: 0.2 is small, 0.5 is moderate,and 0.8 or higher is a large effect. Additionally, the correla-tion coefficients between changes over time in the WORCwith respect to changes in the CS and DASH were calcu-lated. Furthermore, the proportion of patients with the pre-viously defined minimal clinically important difference ofthe WORC (MCID, 11.7 percentage points)27 was assessed.

All analyses were performed for the total group as wellas for each of the 3 study groups separately. For all tests, Pvalues \.05 were considered statistically significant. Anal-yses were processed using SPSS 16.0 software (SPSS Inc,Chicago, Illinois).

RESULTS

Patient Characteristics

Of 94 patients included in the 3 projects until July 2011, 2were excluded from the calcific tendinitis study group afterfilling out the first WORC because of cancellation of thetreatment due to contraindications, as judged by the treat-ing orthopaedic surgeon. In addition, 1 patient in the cal-cific tendinitis study group did not complete the thirdassessment 6 weeks after treatment.

Of all included participants, 7 patients left 1 item unan-swered in one of the WORC. Two patients left more than 1

item unanswered. Of these 2 patients, the correspondingWORC scores were not included in the analyses. Tenpatients did not complete either the first (n = 4) or the sec-ond WORC. Their results were not included in the repro-ducibility analyses. In case of a missing T1 WORC, dataof the T2 WORC were used as baseline WORC data forcomparison with CS, DASH, and T3 scores.

The final study group comprised 92 patients with a meanage of 55.0 years: 35 with a rotator cuff tear (study 1), 22with subacromial impingement (study 2), and 35 with cal-cific tendinitis (study 3) (Table 1). On average, rotator cufftear patients were significantly older than patients in theother 2 diagnostic groups. Cuff tear patients had a signifi-cantly lower CS and higher DASH compared with impinge-ment and calcific tendinitis patients. There were nostatistically significant differences for the CS and DASHbetween the latter study groups.

Psychometric Properties of the BaselineWORC Questionnaire

The mean total WORC score at T1 was 1112.2 6 428.3(range, 208.5-1859.2). Expressed in percentage scores,the mean total WORC was 46.8 6 20.4 (range, 10.0-90.0).The WORC total and domain scores were normally distrib-uted in all groups and the total group. The mean totalWORC score was significantly lower in group 3 comparedwith group 2. In contrast to the differences between cufftear patients and impingement and calcific tendinitispatients in the DASH and CS, there was no statisticallysignificant difference between cuff tear patients and theother study groups for the WORC (Table 1).

Floor and Ceiling Effects. For all items, the domainscores, and the total scores, less than 15% of the patientsobtained the maximum or minimum score, implying therewere no floor and ceiling effects of the WORC. In the phys-ical symptoms domain, 0 patients scored between 0 and 10,and 3 patients (3.1%) had scores between 90 and 100. Inthe sports domain, 5 patients (5.2%) obtained minimalscores, and 2 (2.1%) obtained the maximum scores. Forthe work domain, there were 7 patients (7.2%) in the 0-to-10 score range and 1 (1%) in the maximal score range.In the lifestyle domain, 4 (4.1%) patients had the lowestscores, and 4 had scores in the 90-to-100 range. Lastly, inthe emotions domain, there were 6 (6.2%) and 13 (13.4%)patients in the minimal and maximal score ranges, respec-tively. For the total WORC score, both score ranges con-tained 1 patient (1%).

Internal Consistency. Internal consistency, as calcu-lated with the Cronbach a, was high for all WORC domainscores. Internal consistency for all items with respect tothe total WORC was high as well, with a Cronbach a of.95 for the total WORC (Table 2). For the 3 diagnosticgroups, Cronbach a coefficients of the total WORC scoreswere .96, .91, and .95 for rotator cuff tear, impingement,and calcific tendinitis patients, respectively.

Test-Retest Reliability. Table 3 shows that there were nostatistically significant differences between the test and

1614 de Witte et al The American Journal of Sports Medicine

at Universiteit Leiden \ LUMC on April 28, 2013ajs.sagepub.comDownloaded from

retest WORC total score and domain scores, except for thework domain in the total group, where the WORC scorewas higher at T2 (mean change, 3.0; 95% confidence inter-val, 0.1-6.0). Pearson correlation coefficient between thetest and retest scores was .90 (P \ .001) for the totalWORC score. Expressed in ICCs, test-retest reliability ofthe WORC domain scores ranged from .81 (P \ .0001) to.89 (P \ .0001) and was .89 (P \ .0001) for the totalWORC score. Corresponding SEM was 6.9 for the totalWORC score. Consequently, the 95% confidence intervalof a patient’s true score can be estimated by the following:observed score 6 1.96 3 6.9 = observed score 6 13.5. TheMDC (equation 2) was 19.1. For the 3 study groups, ICCswere .94 (P \ .0001), .82 (P \ .0001), and .84 (P \ .0001)for rotator cuff tear, impingement, and calcific tendinitispatients, respectively.

Validity. Table 2 shows Pearson correlation coefficientsbetween the WORC, CS, and DASH. Except for the emo-tions domain in impingement patients, there were signifi-cant correlations between the domain and total WORCscores with the CS and DASH.

Pearson correlations of the total WORC were .56 (P \.001) for the CS and –.65 (P \ .0001) for the DASH, ashigh percentage WORC scores mean less symptoms, wherehigh DASH scores indicate worse symptoms. Correlationswith the DASH were highest for impingement and calcifictendinitis patients: –.77 (P \ .001) and –.82 (P \ .001),respectively, in contrast to –.49 (P \ .05) in cuff tearpatients (Table 2). Correlation of the WORC at T2 insteadof the WORC at T1 with the CS (T2) showed a correlationof similar magnitude: .63 (P \ .0001).

Responsiveness (Sensitivity to Change). Table 4 showsthat, on average, the mean WORC total and domain scores,the CS, and the DASH improved significantly 6 weeks aftertreatment for calcific tendinitis. Sixteen patients (47%) hadan improvement larger than the MCID: 11.7 percentage

points, as reported by Kirkley et al.27 With a value of 18.8,mean WORC improvement was higher than the MCID.

Overall, the effect size and SRM of the WORC total anddomain scores indicated good responsiveness. Except forthe WORC sports and emotions domains, the effect sizeand SRM of the WORC were larger than those of theDASH and in the same range (effect size) or slightly lower(SRM) than those of the CS.

Table 5 shows significant and moderate to good correla-tions between changes over time of the WORC total anddomain scores and the CS and DASH. For the CS, changesin time correlated best with changes in the WORC lifestyle,physical symptoms, and sports domains. Changes in theDASH correlated moderate to good to changes in allWORC domains.

DISCUSSION

The results of our study show that the WORC is a reliable,valid, and responsive measure of health-related QOL inpatients with rotator cuff lesions of various origins. TheWORC total and domain scores correlate moderate togood with the CS and DASH.

With respect to internal consistency, the Cronbach a of.95 observed in the present study is in line with the resultsof previous studies reporting a Cronbach a of .93 for theoriginal translation and ranging from .92 to .97 for trans-lated versions.19,23,32,39,48 This implies that the domainsand items within the WORC contribute to and correlatewith each other and the total WORC score.10

We found good test-retest reliability, with test-retestcorrelation coefficients of the total WORC score rangingfrom .82 to .94 in the 3 diagnostic groups. This is in concor-dance with Huber et al,23 who reported a test-retest corre-lation coefficient of .96 using the German WORC in 21

TABLE 1Demographic and Baseline Characteristics of 92 Included Patients With a Rotator Cuff Tear (Study Group 1),

Subacromial Impingement Syndrome (Study Group 2), or Calcific Tendinitis (Study Group 3)a

Rotator Cuff Tear(n = 35)

Impingement(n = 22)

Calcific Tendinitis(n = 35)

Differences Between StudyGroups (P Value)

All Patients(n = 92)

Age, y 60.6 6 8.6 51.1 6 6.5 52.4 6 7.6 \.001b,c,d 55.0 6 8.7Gender, n (%)

Male 19 (54) 9 (41) 15 (43) .52 43 (47)Female 16 (46) 13 (59) 20 (57) 49 (53)

Duration of symptoms, mo NA 33.1 6 58.7 42.5 6 41.6 .56 39.6 6 47.0Dominant side affected, n (%) 16 (40) 12 (55) 20 (57) .78 32 (56)Constant score 53.4 6 14.8 74.7 6 10.7 67.3 6 12.3 \.001b,d 63.9 6 15.4DASH 50.9 6 18.4 28.7 6 13.5 38.6 6 16.3 \.001b,d 40.9 6 18.6WORC

Total score 1159.3 6 461.9 906.1 6 334.0 1194.7 6 414.7 .03c 1112.2 6 428.3Percentage score 44.3 6 22.0 56.8 6 15.8 43.0 6 19.7 .03c 46.8 6 20.4

aValues are expressed as mean 6 standard deviation unless otherwise indicated. NA, not applicable; DASH, Disabilities of the Arm,Shoulder and Hand score; WORC, Western Ontario Rotator Cuff Index.

bStatistically significant difference between study groups 1 and 2.cStatistically significant difference between study groups 2 and 3.dStatistically significant difference between study groups 1 and 3.

Vol. 40, No. 7, 2012 WORC in Rotator Cuff Disease Patients 1615

at Universiteit Leiden \ LUMC on April 28, 2013ajs.sagepub.comDownloaded from

patients. Intraclass correlations (ICC = .89 for the totalWORC) were comparable with those in the literature,with .96 for the WORC designers in 55 patients26 and

values ranging from .88 to .98 for translated ver-sions.18,19,23,32,39 To the best of our knowledge, there isonly one study reporting the SEM of the WORC: Lopes

TABLE 2Internal and Construct Validity of the WORC in All Patients (n = 92)a

Correlations With Constant Score and DASH

Constant Score DASH

Cronbach a RC SIS Calc All Patients RC SIS Calc All Patients

Total WORC (n = 92) .95 .59b .65c .55c .56b 2.49c 2.77b 2.82b 2.65b

Physical symptoms .85 .51c .61c .49c .50b 2.41c 2.58c 2.73b 2.56b

Sports/recreation .76 .52c .50c .51c .50b 2.41c 2.76b 2.72b 2.57c

Work .84 .47c .56c .55c .51b 2.51c 2.73b 2.80b 2.65b

Lifestyle .82 .66b .60c .45c .52b 2.51c 2.37 2.66b 2.52b

Emotions .86 .53c .20 .43c .49b 2.48c 2.69b 2.72b 2.63b

aCronbach a for internal validity and Pearson correlation coefficients of the WORC with the DASH and Constant score. DASH: 0 is best,and 100 is worst. WORC and Constant score: 0 is worst, and 100 is best. WORC, Western Ontario Rotator Cuff Index; DASH, Disabilities ofthe Arm, Shoulder and Hand score; RC, rotator cuff tendon tear (n = 35); SIS, subacromial impingement syndrome (n = 25); Calc, calcifictendinitis (n = 35).

bP \ .001.cP \ .05.

TABLE 3Reproducibility of the WORCa

Baseline Retest Mean Difference (95% CI) P Value Pearson r ICC (95% CI) SEM

Total WORC (n = 83) 46.3 6 20.2 47.9 6 21.3 1.6 (–0.5 to 3.7) .13 .90b .89 (.84 to .93) 6.9Physical symptoms 52.2 6 21.6 54.3 6 22.5 2.0 (–0.4 to 4.4) .10 .88b .87 (.81 to .92) 8.0Sports/recreation 43.3 6 21.6 44.6 6 21.6 1.3 (–1.7 to 4.3) .41 .81b .81 (.72 to .87) 9.4Work 37.3 6 22.6 40.4 6 23.8 3.0 (0.1 to 6.0) .04 .83b .83 (.74 to .88) 9.6Lifestyle 44.6 6 24.2 44.6 6 24.2 0.4 (–2.3 to 3.1) .78 .86b .86 (.80 to .91) 9.1Emotions 58.0 6 28.3 59.3 6 28.3 1.2 (–2.4 to 4.9) .50 .83b .83 (.75 to .89) 11.7

aValues are expressed as mean 6 standard deviation unless otherwise indicated. Test-retest reproducibility was assessed by comparingthe differences between test and retest examinations within 3 to 14 days, expressed in percentage scores, with paired t test and Pearsoncorrelations. Reliability was estimated with ICCs. WORC, Western Ontario Rotator Cuff Index; CI, confidence interval; ICC, intraclass cor-relation coefficient; SEM, standard error of measurement.

bP \ .001.

TABLE 4Baseline and 6-Week Follow-up Data in 34 Patients Treated With Ultrasound-Guided Needling and Lavage

or a Subacromial Injection With Corticosteroids in the Calcific Tendinitis Group (Study Group 3)a

Baseline 6-Week Follow-up Change (95% CI) P Value ES SRM

Constant score (n = 34) 67.1 6 12.4 78.2 6 16.8 11.1 (7.8 to 14.4) \.001 20.89 21.16DASH (n = 34) 38.6 6 16.5 28.6 6 22.9 29.9 (215.1 to 24.8) \.001 0.61 0.68Total WORC (n = 34) 42.9 6 19.6 61.7 6 26.7 18.8 (11.3 to 26.2) \.001 20.96 20.91

Physical symptoms 48.3 6 21.1 67.3 6 24.3 19.0 (11.4 to 26.7) \.001 20.90 20.86Sports/recreation 41.6 6 19.6 56.9 6 27.7 15.3 (7.2 to 23.4) .001 20.78 20.67Work 34.9 6 21.1 56.8 6 29.4 21.9 (13.9 to 29.9) \.001 21.04 20.97Lifestyle 39.0 6 23.7 61.9 6 31.9 22.9 (14.4 to 31.5) \.001 20.97 20.96Emotions 54.6 6 29.2 67.1 6 29.0 12.5 (4.6 to 20.4) .003 20.44 20.56

aValues are expressed as mean 6 standard deviation unless otherwise indicated. For ES and SRM: 0.2 is small, 0.5 is moderate, and 0.8 orhigher is large effect (idem for negative values). CI, confidence interval; ES, effect size; SRM, standardized response mean; DASH, Disabil-ities of the Arm, Shoulder and Hand score; WORC, Western Ontario Rotator Cuff Index.

1616 de Witte et al The American Journal of Sports Medicine

at Universiteit Leiden \ LUMC on April 28, 2013ajs.sagepub.comDownloaded from

et al32 reported mean SEMs ranging from 3.0 to 5.2, whichis of the same order as the 6.9 we observed.

Concerning construct validity, the WORC correlatedmoderate to good with the DASH (–.65) and CS (.56). Thelatter predominantly contains objective items and lacksemotional or lifestyle factors that are included in theWORC. Others have reported similar correlations of theWORC with the SF-36, American Shoulder and Elbow Sur-geons (ASES), or University of California, Los Angeles(UCLA) scores, often superior to correlations with the CS.§

Moderate to good correlation coefficients, as found for theWORC, are desirable, as high or excellent correlation coeffi-cients with other scores would mean the WORC is not ofadditional value to existing measures. To a further extent,this makes combining the WORC with other outcome meas-ures, including the CS, applicable and even advisable. How-ever, the WORC appeared less discriminative than theDASH and CS in case of severe symptoms: differenceswere found for the CS and DASH between cuff tear patients(severe symptoms on average) and the impingement andcalcific tendinitis groups, but not for the WORC.

In our study, the WORC was also compared with the CSand DASH to assess its responsiveness after treatment.Evaluating change score correlations, our results were mod-erate to good: –.84 for the DASH and .61 for the CS. Holtbyand Razmjou22 published higher correlations for changescores: .77 for the CS and .85 for the ASES in a group of50 surgically treated impingement and cuff tear patients.Possibly, the WORC correlates better with change scoresof, for example, the CS, ASES, and DASH in case of rigorousinterventions in patients with severe symptoms, that is,when large changes in scores over time can be expected.This can be the case with surgical treatment of rotatorcuff problems, as shown by Holtby and Razmjou.22 How-ever, in the current study, interventions were less invasive:patients were treated with an injection or ultrasound-guided needling and lavage for calcific tendinitis.

A limited number of studies assessed the responsive-ness of the WORC expressed in SRM or effect

size.15,16,35,41 Reported SRMs of the total WORC scorerange between 0.8 and 2.0, often comparable or superiorto other scores, including the CS, DASH, and OxfordShoulder Scale, but are mostly based on subgroups ofpatients who are defined as ‘‘responsive to treatment.’’ Inour study, mean SRM was 0.91 for patients responsiveand nonresponsive 6 weeks after the treatment of calcifictendinitis, better than the DASH and in the same rangeas the CS. Data on effect size of the WORC are scarce.An effect size of 0.92 for the Brazilian WORC has beenreported, comparable to the 0.96 in our study.15 In bothstudies, effect size of the WORC was superior to the DASH.

Hence, compared with published validation studies ofboth original and translated WORC versions, our resultsare similar.jj However, our study provides some new andimportant information. First, with regard to psychometricproperties of the WORC, in most publications, only internalconsistency measures, test-retest assessments, and/orresponsiveness are reported. In our study, a comprehensivecombination of the most relevant measurement properties isassessed in one population, as advised by, for example, theScientific Advisory Committee of the Medical OutcomesTrust and Terwee et al.31,46 Second, patient groups inmany publications are quite homogeneous, small, orselected with unclear eligibility criteria.{ In contrast, westudied a consecutive patient group composed of patientswith 1 of 3 diagnoses and with a broad spectrum of severityof symptoms, using strict inclusion and exclusion criteriaand advanced imaging technologies. Therefore, this studyassessed a comprehensive combination of relevant psycho-metric outcome measures in a large and heterogeneousbut well-defined cohort, indicating high external validityof our results. Third, we found potential discriminativeproblems of the WORC in patients with severe symptoms(in the lower score ranges in patients with cuff tears). Tothe best of our knowledge, this has not been reported earlierand has to be taken into account when using the WORC insimilar patients.

TABLE 5Longitudinal Responsivenessa

Correlations of Changes (0-6 Weeks)

Constant Score DASH

Pearson P Value Pearson P Value

Total WORC (n = 34) .61 \.001 2.84 \.001Physical symptoms .55 .001 2.65 \.001Sports/recreation .60 \.001 2.77 \.001Work .52 .002 2.78 \.001Lifestyle .66 \.001 2.77 \.001Emotions .38 .03 2.72 \.001

aCorrelations of changes in WORC scores with changes in the Constant score and DASH, from baseline to 6 weeks after treatment in thecalcific tendinitis group. DASH, Disabilities of the Arm, Shoulder and Hand score; WORC, Western Ontario Rotator Cuff Index.

§References 19, 21, 23, 26, 32, 33, 39, 41.

jjReferences 16, 18, 19, 21, 23, 32, 33, 35, 39, 41, 48.

{References 16, 18, 19, 21, 23, 32, 39, 48.

Vol. 40, No. 7, 2012 WORC in Rotator Cuff Disease Patients 1617

at Universiteit Leiden \ LUMC on April 28, 2013ajs.sagepub.comDownloaded from

Over the past 2 decades, there has been an increasingemphasis on the use of self-reported outcome measures inorthopaedic practice, including the WORC.3 Conventionalmusculoskeletal instruments are mainly based on objectivequantities that do not necessarily correlate with outcomesthat are most relevant to patients, such as activities of dailylife, mental health, or other QOL aspects. For broad andaccurate patient assessment, it is advisable to combineobjective and self-reported general health scores, generalregional (eg, shoulder) measures, and condition-specificmeasures.51,52 The results of the current study and previouspublications demonstrate that the WORC, one of few avail-able condition-specific health-related QOL measures for therotator cuff, can be used in the assessment of cuff patients.

There are some limitations of our study that need to betaken into account when interpreting our results. First,the investigating researcher was not blinded to the diag-nostic study groups and treatment, leading to potentialbias toward improvement with regard to the CS. Second,within the study period, there was only an interventionand WORC assessment of longitudinal responsiveness inthe calcific tendinitis patients. It is a possibility, however,that results for longitudinal responsiveness in the calcifictendinitis group cannot necessarily be extrapolated toother rotator cuff conditions. Third, all patients in thisstudy were referred to our medical center for treatmentby general practitioners. Therefore, this patient groupmight not be representative of all patients with rotatorcuff conditions. Fourth, the test-retest time interval wasrelatively short: near 3 to 7 days in many patients. Yet,earlier validation studies for the WORC reported a sub-stantial number of included patients to be considered clin-ically unstable over a 2-week period,18,32 and even a 2- to 3-day period for WORC test-retest assessments is not uncom-mon.19,39 Fifth, for practical reasons, the DASH (T1) andCS (T2) were not administered at the same occasion,whereas both were compared with the baseline (T1)WORC. Given the insignificant T1-T2 differences of theWORC and high T1-T2 correlation coefficients, we believethis does not influence our results. In addition, comparingthe T2 WORC to the CS (T2) gave a similar correlationcoefficient. Lastly, we used the Dutch version of theWORC, and therefore, it cannot be guaranteed that ourresults are generalizable to the original or other translatedversions. However, the Dutch translation we used wasmade by another independent institution using interna-tional translation guidelines.49 Furthermore, the resultsobtained in each of the diagnostic groups are comparablewith previously (but separately) published averagescores,18 SDs,7,18,37,40,42 Cronbach as,18,48 correlationswith the CS21,40,48 and DASH,26,48 and ICCs18,26 of theoriginal WORC in patient groups with similar diagnoses.Therefore, we think this study can be considered to bethe first to extensively assess the validity of the WORCaccording to the Scientific Advisory Committee guidelinesin a strictly defined patient group covering a broad spec-trum of rotator cuff conditions. Nevertheless, repeatingthis comprehensive combination of analyses for the origi-nal WORC in a similarly broad spectrum of rotator cuffcondition patients is recommendable.

In conclusion, our results suggest the WORC is applica-ble in research and clinical practice as a self-reported,disease-specific health-related QOL outcome measure forrotator cuff patients. It is advisable to use this disease-specific measure in combination with a regional and a gen-eral health outcome measure. The WORC is potentially lessdiscriminative between patients with severe complaintscompared with other outcome measures. This needs to bestudied further and has to be taken into account when usingthe WORC in assessing patients with severe symptoms.

ACKNOWLEDGMENT

The authors acknowledge Suzanne Wiertsema (Depart-ment of Rehabilitation Medicine, VU University MedicalCenter, Amsterdam, the Netherlands) for her permissionto use the Dutch translation of the WORC in the currentvalidation study and Sharon Griffin (Fowler KennedySport Medicine Clinic, University of Western Ontario, Lon-don, Ontario, Canada), one of the developers of the originalWORC, for her permission to study the WORC and her val-idation of the article.

REFERENCES

1. Alvarez CM, Litchfield R, Jackowski D, Griffin S, Kirkley A. A prospec-

tive, double-blind, randomized clinical trial comparing subacromial

injection of betamethasone and xylocaine to xylocaine alone in

chronic rotator cuff tendinosis. Am J Sports Med. 2005;33:255-262.

2. Balyk R, Luciak-Corea C, Otto D, Baysal D, Beaupre L. Do outcomes

differ after rotator cuff repair for patients receiving workers’ compen-

sation? Clin Orthop Relat Res. 2008;466:3025-3033.

3. Beaton DE, Schemitsch E. Measures of health-related quality of life

and physical function. Clin Orthop Relat Res. 2003;413:90-105.

4. Beckerman H, Roebroeck ME, Lankhorst GJ, Becher JG, Bezemer

PD, Verbeek AL. Smallest real difference, a link between reproduc-

ibility and responsiveness. Qual Life Res. 2001;10:571-578.

5. Bernhardsson S, Klintberg IH, Wendt GK. Evaluation of an exercise

concept focusing on eccentric strength training of the rotator cuff

for patients with subacromial impingement syndrome. Clin Rehabil.

2011;25:69-78.

6. Brander VA, Gomberawalla A, Chambers M, Bowen M, Nuber G. Effi-

cacy and safety of hylan G-F 20 for symptomatic glenohumeral oste-

oarthritis: a prospective, pilot study. PM R. 2010;2:259-267.

7. Burks RT, Crim J, Brown N, Fink B, Greis PE. A prospective random-

ized clinical trial comparing arthroscopic single- and double-row

rotator cuff repair: magnetic resonance imaging and early clinical

evaluation. Am J Sports Med. 2009;37:674-682.

8. Cohen SB, Towers JD, Bradley JP. Rotator cuff contusions of the

shoulder in professional football players: epidemiology and magnetic

resonance imaging findings. Am J Sports Med. 2007;35:442-447.

9. Constant CR, Gerber C, Emery RJ, Sojbjerg JO, Gohlke F, Boileau P.

A review of the Constant score: modifications and guidelines for its

use. J Shoulder Elbow Surg. 2008;17:355-361.

10. Cronbach LJ. Coefficient alpha and the internal structure of tests.

Psychometrika. 1951;16:297-334.

11. de Vet HC, Terwee CB, Ostelo RW, Beckerman H, Knol DL, Bouter

LM. Minimal changes in health status questionnaires: distinction

between minimally detectable change and minimally important

change. Health Qual Life Outcomes. 2006;4:54.

12. de Witte PB, Nagels J, van Arkel ER, Visser CPJ, Nelissen RGHH, de

Groot JH. Study protocol subacromial impingement syndrome: the

identification of pathophysiologic mechanisms (SISTIM). BMC Mus-

culoskelet Disord. 2011;12:282.

1618 de Witte et al The American Journal of Sports Medicine

at Universiteit Leiden \ LUMC on April 28, 2013ajs.sagepub.comDownloaded from

13. de Witte PB, van der Zwaal P, Visch W, et al. Arm ADductor activation

with arm ABduction in healthy subjects and rotator cuff tear patients:

design of a new measuring instrument. [published online ahead of print

February 15, 2012]. Hum Mov Sci. http://dx.doi.org/10.1016/

j.humov.2011.08.007.

14. Desmeules F, Minville L, Riederer B, Cote CH, Fremont P. Acromio-

humeral distance variation measured by ultrasonography and its

association with the outcome of rehabilitation for shoulder impinge-

ment syndrome. Clin J Sport Med. 2004;14:197-205.

15. Diniz Lopes A, Ciconelli RM, Carrera EF, Griffin S, Faloppa F,

Baldy dos Reis F. Comparison of the responsiveness of the Brazilian

version of the Western Ontario Rotator Cuff Index (WORC) with

DASH, UCLA and SF-36 in patients with rotator cuff disorders. Clin

Exp Rheumatol. 2009;27:758-764.

16. Ekeberg OM, Bautz-Holter E, Keller A, Tveita EK, Juel NG, Brox JI. A

questionnaire found disease-specific WORC index is not more

responsive than SPADI and OSS in rotator cuff disease. J Clin Epide-

miol. 2010;63:575-584.

17. Ekeberg OM, Bautz-Holter E, Tveita EK, Juel NG, Kvalheim S, Brox JI.

Subacromial ultrasound guided or systemic steroid injection for rotator

cuff disease: randomised double blind study. BMJ. 2009;338:a3112.

18. Ekeberg OM, Bautz-Holter E, Tveita EK, Keller A, Juel NG, Brox JI.

Agreement, reliability and validity in 3 shoulder questionnaires in

patients with rotator cuff disease. BMC Musculoskelet Disord.

2008;9:68.

19. El O, Bircan C, Gulbahar S, et al. The reliability and validity of the

Turkish version of the Western Ontario Rotator Cuff Index. Rheumatol

Int. 2006;26:1101-1108.

20. Fallatah S, Dervin GF, Brunet JA, Conway AF, Hrushowy H. Func-

tional outcome after proximal humeral fractures treated with hemiar-

throplasty. Can J Surg. 2008;51:361-365.

21. Holtby R, Razmjou H. Impact of work-related compensation claims

on surgical outcome of patients with rotator cuff related pathologies:

a matched case-control study. J Shoulder Elbow Surg. 2010;19:452-

460.

22. Holtby R, Razmjou H. Measurement properties of the Western

Ontario rotator cuff outcome measure: a preliminary report. J Shoul-

der Elbow Surg. 2005;14:506-510.

23. Huber W, Hofstaetter JG, Hanslik-Schnabel B, Posch M, Wurnig C.

[Translation and psychometric testing of the Western Ontario Rotator

Cuff Index (WORC) for use in Germany]. Z Orthop Ihre Grenzgeb.

2005;143:453-460.

24. Hudak PL, Amadio PC, Bombardier C. Development of an upper

extremity outcome measure: the DASH (Disabilities of the Arm,

Shoulder and Hand) [corrected]. The Upper Extremity Collaborative

Group (UECG). Am J Ind Med. 1996;29:602-608.

25. Kazis LE, Anderson JJ, Meenan RF. Effect sizes for interpreting

changes in health status. Med Care. 1989;27(3 Suppl):S178-S189.

26. Kirkley A, Alvarez C, Griffin S. The development and evaluation of

a disease-specific quality-of-life questionnaire for disorders of the

rotator cuff: the Western Ontario Rotator Cuff Index. Clin J Sport

Med. 2003;13:84-92.

27. Kirkley A, Griffin S, Dainty K. Scoring systems for the functional

assessment of the shoulder. Arthroscopy. 2003;19:1109-1120.

28. Koester MC, George MS, Kuhn JE. Shoulder impingement syn-

drome. Am J Med. 2005;118:452-455.

29. Lapner PC, Su Y, Simon D, El-Fatori S, Lopez-Vidriero E. Does the

upward migration index predict function and quality of life in arthro-

scopic rotator cuff repair? Clin Orthop Relat Res. 2010;468:3063-3069.

30. Liang MH, Fossel AH, Larson MG. Comparisons of five health status

instruments for orthopedic evaluation. Med Care. 1990;28:632-642.

31. Lohr KN. Assessing health status and quality-of-life instruments:

attributes and review criteria. Qual Life Res. 2002;11:193-205.

32. Lopes AD, Ciconelli RM, Carrera EF, Griffin S, Faloppa F, Dos Reis

FB. Validity and reliability of the Western Ontario Rotator Cuff Index

(WORC) for use in Brazil. Clin J Sport Med. 2008;18:266-272.

33. Lopes AD, Vilar e Furtado, Silva CA, Yi LC, Malfatti CA, Araujo SA.

Comparison of self-report and interview administration methods

based on the Brazilian versions of the Western Ontario Rotator Cuff

Index and Disabilities of the Arm, Shoulder and Hand Questionnaire

in patients with rotator cuff disorders. Clinics (Sao Paulo).

2009;64:121-125.

34. Lorbach O, Kusma M, Pape D, Kohn D, Dienst M. Influence of

deposit stage and failed ESWT on the surgical results of arthroscopic

treatment of calcifying tendonitis of the shoulder. Knee Surg Sports

Traumatol Arthrosc. 2008;16:516-521.

35. Macdermid JC, Drosdowech D, Faber K. Responsiveness of self-

report scales in patients recovering from rotator cuff surgery. J

Shoulder Elbow Surg. 2006;15:407-414.

36. Macdermid JC, Holtby R, Razmjou H, Bryant D. All-arthroscopic ver-

sus mini-open repair of small or moderate-sized rotator cuff tears:

a protocol for a randomized trial [NCT00128076]. BMC Musculoske-

let Disord. 2006;7:25.

37. MacDonald P, McRae S, Leiter J, Mascarenhas R, Lapner P. Arthro-

scopic rotator cuff repair with and without acromioplasty in the treat-

ment of full-thickness rotator cuff tears: a multicenter, randomized

controlled trial. J Bone Joint Surg Am. 2011;93:1953-1960.

38. Matava MJ, Purcell DB, Rudzki JR. Partial-thickness rotator cuff

tears. Am J Sports Med. 2005;33:1405-1417.

39. Mousavi SJ, Hadian MR, Abedi M, Montazeri A. Translation and val-

idation study of the Persian version of the Western Ontario Rotator

Cuff Index. Clin Rheumatol. 2009;28:293-299.

40. Razmjou H, Bean A, Macdermid JC, van Osnabrugge V, Travers N,

Holtby R. Convergent validity of the Constant-Murley outcome mea-

sure in patients with rotator cuff disease. Physiother Can.

2008;60:72-79.

41. Razmjou H, Bean A, van Osnabrugge V, MacDermid JC, Holtby R.

Cross-sectional and longitudinal construct validity of two rotator

cuff disease-specific outcome measures. BMC Musculoskelet Dis-

ord. 2006;7:26.

42. Razmjou H, Davis AM, Jaglal SB, Holtby R, Richards RR. Cross-sec-

tional analysis of baseline differences of candidates for rotator cuff

surgery: a sex and gender perspective. BMC Musculoskelet Disord.

2009;10:26.

43. Razmjou H, Lincoln S, Axelrod T, Holtby R. Factors contributing to

failure of rotator cuff surgery in persons with work-related injuries.

Physiother Can. 2008;60:125-133.

44. Shrout PE, Fleiss JL. Intraclass correlations: uses in assessing rater

reliability. Psychol Bull. 1979;86:420-428.

45. Sobel JS, Winters JC. Shoulder Complaints in General Practice:

Diagnosis and Treatment. Amsterdam: Meditekst; 1996.

46. Terwee CB, Bot SD, de Boer MR, et al. Quality criteria were pro-

posed for measurement properties of health status questionnaires.

J Clin Epidemiol. 2007;60:34-42.

47. Weir JP. Quantifying test-retest reliability using the intraclass correla-

tion coefficient and the SEM. J Strength Cond Res. 2005;19:231-240.

48. Wessel J, Razmjou H, Mewa Y, Holtby R. The factor validity of the

Western Ontario Rotator Cuff Index. BMC Musculoskelet Disord.

2005;6:22.

49. Wiertsema SH, Rietberg MB, Hekman KMC, Schothorst M, Steult-

jens MPM, Dekker J. Reproducibility of the Dutch version of the

Western Ontario Rotator Cuff Index (WORC). J Shoulder Elbow

Surg. In press.

50. Wong I, Burns J, Snyder S. Arthroscopic GraftJacket repair of rotator

cuff tears. J Shoulder Elbow Surg. 2010;19(2 Suppl):104-109.

51. Wright JG. Outcomes research: what to measure. World J Surg.

1999;23:1224-1226.

52. Wright RW, Baumgarten KM. Shoulder outcomes measures. J Am

Acad Orthop Surg. 2010;18:436-444.

53. Yocum LA. Assessing the shoulder: history, physical examination,

differential diagnosis, and special tests used. Clin Sports Med.

1983;2:281-289.

For reprints and permission queries, please visit SAGE’s Web site at http://www.sagepub.com/journalsPermissions.nav

Vol. 40, No. 7, 2012 WORC in Rotator Cuff Disease Patients 1619

at Universiteit Leiden \ LUMC on April 28, 2013ajs.sagepub.comDownloaded from