More outcomes than trials: a call for consistent data collection across stroke rehabilitation trials

More outcomes than trials: a call for consistent data collection across strokerehabilitation trials

M. Ali1*, C. English2,3, J. Bernhardt2, K. S. Sunnerhagen4, and M. Brady1,on behalf of the VISTA-Rehab Collaboration†

Stroke survivors experience complex combinations of impair-ments, activity limitations, and participation restrictions. Theessential components of stroke rehabilitation remain elusive.Determining efficacy in randomized controlled trials (RCTs) ischallenging; there is no commonly agreed primary outcomemeasure for rehabilitation trials. Clinical guidelines depend onproof of efficacy in RCTs and meta-analyses. However, diversetrial aims, differing methods, inconsistent data collection, anduse of multiple assessment tools hinder comparability acrosstrials. Consistent data collection in acute stroke trials has facili-tated meta-analyses to inform trial design and clinical practice.With few exceptions, inconsistent data collection has hinderedsimilar progress in stroke rehabilitation research. There is anurgent need for the routine collection of a core dataset ofcommon variables in rehabilitation trials. The European StrokeOrganisation Outcomes Working Group, the National Insti-tutes of Neurological Disorders and Stroke Common DataElements project, and the Collaborative Stroke Audit andResearch project have called for consistency in data collectionin stroke trials. Standardizing data collection can decreasestudy start up times, facilitate data sharing, and inform clinicalguidelines. Although achieving consensus on which outcomemeasures to use in stroke rehabilitation trials is a considerabletask, perhaps a feasible starting point is to achieve consistencyin the collection of data on demography, stroke severity, andstroke onset to inclusion times. Longer term goals couldinclude the development of a consensus process to establishthe core dataset. This should be endorsed by researchers,funders, and journal editors in order to facilitate sustainablechange.Key words: common data, outcomes, rehabilitation, standardization,stroke

Introduction

Stroke survivors experience unique combinations of impair-

ments, activity limitations, and participation restrictions

(Table 1), which add to the complex challenge of stroke rehabili-

tation. Factors such as the rehabilitation setting, time since stroke,

and coexisting impairments further contribute to this complexity.

Rehabilitation is multidisciplinary, bringing together medical

consultants, stroke nurses, clinical support workers or auxiliary

nursing staff, physiotherapists, and occupational and speech and

language therapists. The multidisciplinary team may also include

dieticians, podiatrists, orthoptists, orthotists, social workers, and

neuropsychologists. Together they aim to reduce the impact of

stroke on activities of daily living, maximize recovery, restitution

and participation, minimize the impact of any changes in ability,

and prevent avoidable complications (1). The essential compo-

nents of stroke rehabilitation remain elusive. Although the major-

ity of stroke patients receive both physiotherapy and occupational

therapy, consensus on the optimum treatment regimen is lacking

(2).

Randomized controlled trials (RCTs) and meta-analyses form

the evidence base for clinical practice. However, due to the chal-

lenging nature of poststroke sequelae, RCTs in stroke rehabili-

tation are particularly complex. Meaningful recovery varies

between patients: what is considered as a good outcome by one

stroke survivor may not be similarly rated by another. The use of

a wide range of discipline-specific outcome measures coupled

with the relatively small population sizes and limited scope of

trials severely limits our ability to compare results between trials

and to conduct meta-analyses. Nevertheless, there is a good indi-

cation that RCTs of many rehabilitation interventions are feasible

and improve the evidence base for rehabilitation care (3,4). Meta-

analyses of rehabilitation trial data have helped to drive practice

change as evidenced in work by the Stroke Unit Trialists’ Collabo-

ration (2), early supported discharge (5) and in community reha-

bilitation (6). The recognition of a need for high-quality evidence

of efficacy has been accompanied by an increase in the number

of stroke rehabilitation trials. Within the field of physiotherapy

alone, the number of trials conducted since 1982 has grown expo-

nentially (7). However, the translation of findings into clinical

practice has been disproportionately low in comparison with the

number of completed trials.

A multitude of outcome measures

Determining the efficacy of rehabilitation interventions in meta-

analyses is problematic as there is no commonly agreed primary

outcome measure; multiple assessment tools exist to describe

similar impairments. For example, in a recent review, 129

outcome measures were recorded for trials of interventions to

improve upper limb function (8). Similarly, a review of speech

and language interventions for aphasia found 100 outcome mea-

sures were recorded across 39 trials (9). Examination of stroke

trials published between 2001 and 2006 revealed the routine use

of up to 47 different functional outcome measures (10) and a

similar review of trials conducted between 2000 and 2011

Correspondence: Myzoon Ali*, NMAHP Research Unit, GlasgowCaledonian University, Cowcaddens Rd, Glasgow G4 0BA, UK.E-mail: [email protected], Midwifery and Allied Health Professions Research Unit,Glasgow Caledonian University, Glasgow, UK2Stroke Division, Florey Neuroscience Institutes, Austin Health,Melbourne, Vic., Australia3International Centre for Allied Health Evidence, University of SouthAustralia, Adelaide, SA, Australia4Section for Clinical Neuroscience and Rehabilitation, University ofGothenburg, Gothenburg, Sweden

Conflict of interest: None declared.

DOI: 10.1111/j.1747-4949.2012.00973.x

Review

© 2012 The Authors.International Journal of Stroke © 2012 World Stroke Organization

18 Vol 8, January 2013, 18–24

highlighted 300 different assessments for cognitive and mood

measures, across 408 studies (11).

The feasibility and practicability of consistent data collection

has been exemplified in acute stroke research. The requirement

for Clinical Trials of Investigational Medicinal Products in the

acute setting to include functional outcome measures led to

recommendations for preferred use of the modified Rankin Scale

(mRS). Training and certification followed to ensure consistency

of scoring between assessors. Consistency in data collection has

also been aided by the fact that acute stroke trials have published

guidance on outcome data collection (12), typically involve inter-

ventions administered within a brief therapeutic window, record

standardized physiological measures, and typically only follow up

patients up to three-month poststroke. In spite of different indi-

vidual trial aims, acute stroke trials show a remarkable level of

consistency in the types of data collected. This is evidenced in the

acute section of the Virtual International Stroke Trials Archive

(VISTA-Acute). This clinical trials resource was established to

facilitate novel exploratory analyses of secondary data to inform

trial design (13). Of 29 acute stroke trials within the archive, all 29

recorded patient demographic details, medical history, and

Barthel Index (BI), 19 recorded mRS, 15 recorded the National

Institutes of Health Stroke Scales, and 11 recorded the Scandina-

vian Stroke Scale. This uniformity in data collection has facilitated

meta-analyses of trial data to quantify treatment effects (14–18)

and secondary analyses to generate hypotheses and pilot trial

Table 1 Selected issues faced by individual stroke survivors

Impairment Activity limitation Participation restriction Possible interventions

• Pain• Loss of range of movement in

limbs• Spasticity• Loss of muscle strength in the

limbs, trunk, face• Loss of sensation in limbs,

trunk, face• Aphasia• Dysarthria• Dyspraxia of movement• Difficulties with bowel/bladder

control & continence• Impaired memory• Impaired executive function• Dysphagia

• Difficultiesstanding/balancing/walking

• Unable to feed self, wash,dress

• Unable to reach and graspand manipulate objects withaffected arm

• Unable to eat normalconsistency food

• Unable to express self-verbally• Difficulty remembering people,

routines, and requests• Difficulty problem solving• Difficulty reading• Difficulty concentrating

• Unable to perform activities ofdaily livings such as bankingand shopping

• Unable to drive• Limited ability to fulfil usual

family roles• Limited ability to work or

volunteer• Limited ability to participate in

usual leisure activities

• Goal-oriented teaminterventions

• Physiotherapy• Speech and language therapy• Occupational therapy• Early rehabilitation• Functional strength training• Cognitive behavioral therapy• Nutritional supplements• Dressing practice• Ankle foot orthosis• External focus feedback• Treadmill training

Fig. 1 (a) Data collection in trials within VISTA-Rehab.

ReviewM. Ali et al.


Vol 8, January 2013, 18–24 19

Fig. 1 (a) (Continued).

Review M. Ali et al.


20 Vol 8, January 2013, 18–24

Fig. 1 (b) Range of patient measures in VISTA-Rehab. VISTA, Virtual International Stroke Trials Archive.

ReviewM. Ali et al.


Vol 8, January 2013, 18–24 21

design issues (19). Although there are some instruments in fairly

widespread use, such as the Functional Independence Measure

(FIM Uniform Data System for Medical Rehabilitation)™, which

is a standard measurement tool in some countries (20), such

consensus and consistency in the collection of outcome measures

has not been evident in stroke rehabilitation research.

Many different impairment and outcome measures exist in

stroke rehabilitation trials. To date, the rehabilitation section of

the VISTA-Rehab (21) has amassed data from 38 trials, involving

10 224 patients and reporting outcomes using at least 44 different

measures. Despite the wealth of data collected, there is little

overlap in the outcome measures recorded (Fig. 1a, b). Of the

current VISTA-Rehab contributions, 25/38 trials recorded the BI,

14 trials recorded the Extended Activities of Daily Living Scale,

8/38 recorded data on the European Quality of Life Score, 9/38

recorded mRS, and 7/38 included data on the Rivermead Mobility

Index. Inconsistency in data collection also extends to baseline

measures of stroke severity and demographics that play an impor-

tant role in facilitating data comparability. As recovery from

stroke depends on a range of factors such as age, gender, type and

severity of stroke, location and size of lesion, adverse events, and

comorbidities (22–24), the inclusion of these data are essential in

order to correctly interpret analyses. Within VISTA-Rehab, only

6/38 trials recorded the type of stroke experienced by the patient,

6/38 described medical history variables, 13/38 described the time

from stroke onset to intervention, 17/38 described a measure of

initial impairment/activity limitation at baseline, and 25/38 trials

recorded the cerebral hemisphere affected by stroke. Despite the

partial overlap across some baseline and outcome variables,

analyses of data from VISTA-Rehab are further limited because of

variation in the times from index stroke to enrollment, a paucity

of data on initial stroke severity and confounding variables such

as demography, cognition, prestroke presentation, and medical

history. Diverse trial aims, differing methods, and the multiplicity

of assessment tools hinders data comparability across trials, even

within a similarly impaired patient population. All of this renders

secondary and meta-analyses of these data problematic, especially

when further complicated by different time points for follow-up

data collection and questionable consistency in training and

administration of assessment tools.

Why are there so many different outcomemeasures in current use?

Multidisciplinary interventions aim to maximize the individual’s

functional recovery but do so using discipline specific inter-

ventions, severity measures, and assessment tools and utilize indi-

vidual time frames for intervention and follow-up. In the inter-

national field, this may be further complicated by language issues

particularly in the field of communication impairment where the

very structure of the language being assessed and rehabilitated

may differ between sites. The complexity of impairments, activity

limitations, and participation restrictions faced by a stroke survi-

vor make this difficult to measure. This results in equally complex

outcome assessments that can have limitations affecting reliability

and validity.

Historically, new assessment tools were developed in response

to a need to quantify impairments where existing scales lacked

validity, did not adequately capture impairments targeted by the

intervention, functionally relevant tasks or participation compo-

nents. For example, the Fugl Meyer Assessment (FMA) was devel-

oped because contemporary measures of limb impairments

lacked quantitative numerical assessment properties (25) and

standardized measurements of posture and motor performance

(25,26). Since the development of some of the earlier assessment

tools, our understanding of the clinimetric properties and the

rigor required to develop new tools have expanded considerably.

Some stroke assessment tools may fail to meet current standards,

but their long-standing, frequent application and the subsequent

wealth of available data ensure their continued use. Improved

understanding of assessment tools has also resulted in informed

selection for use in RCTs. For example, the BI is widely known to

have a ceiling effect in stroke patients (27), and the FMA is simi-

larly limited in those with mild motor impairment. Therefore,

some have postulated that use of the FMA in combination with

the Chedoke-McMaster Disability Inventory may address the

limitation of a ceiling effect in those with mild motor impairment

(26). The need to compensate for the limitations of one assess-

ment tool by using additional assessments has contributed to the

vast array of measures in current use.

The impact of inconsistent data collection

Meta-analyses and design of future trials are hindered if existing

sources of evidence utilize different patient populations, enroll

patients within disparate poststroke time points, use different

assessment tools, and if data collection lacks consistency. As clini-

cal practice is driven by evidence from RCTs and meta-analyses

of trial data, the issue of comparability across different trials is

of utmost importance. Furthermore, the collation of data from

different trials and subsequent standardization of outcome mea-

sures by calculating the standard mean difference for use in meta-

analyses can lead to bias, which may also influence findings in

systematic reviews (28).

A call for consensus on a core dataset

The need for consistency in data collection and appropriate selec-

tion of outcome measures is a theme that has been repeated

by many investigators. Numerous initiatives have individually

sought to produce recommendations for standardized data

collection. From an acute stroke perspective, the European Stroke

Organisation Outcomes Working Group (12) recommended the

use of the mRS as a primary outcome measure in acute stroke

trials. This consensus was reached through exhaustive review of

available evidence and examination of scale properties.

In the United States, the National Institutes of Neurological

Disorders and Stroke Common Data Elements project (29)

employed a combination of rigorous review and consultation

including a working group of experts in cerebrovascular disease

trials, epidemiology, and biostatistics. They recommended 980

common data elements for a range of fields including biospeci-



22 Vol 8, January 2013, 18–24

mens and biomarkers, hospital course and acute therapies,

imaging, laboratory tests and vital signs, long-term therapies,

medical history and prior health status, outcomes and end-points,

stroke presentation, and stroke types and subtypes.

Within the rehabilitation settings of some countries, compari-

sons between patient outcomes across different locations has

been facilitated by the standardized use of the FIM as a measure

of functional status and disability (20). However, use of this

outcome measure has yet to be formally adopted on a global scale.

In the United Kingdom, rehabilitation-specific recommendations

have been investigated in the Collaborative Stroke Audit and

Research (COSTAR) initiative (30). This project sought to

develop a standard method of classifying, measuring, and timing

rehabilitation interventions, to promote collaboration among

centers and across disciplines, to establish a set of recommended

outcome measures for stroke, and to identify areas where new

scales need to be developed. COSTAR highlighted the need for

collection of a core dataset of common variables, including case

mix indicators and outcome measures in order to facilitate data

comparability. However, in spite of the efforts of these initiatives,

there still remains a need for greater consistency in current data

collection practices, reporting of a core dataset and standardized

outcomes across stroke rehabilitation trials. The American Physi-

cal Therapy Association’s Neurology Section generated recom-

mendations for use of 54 physiotherapy outcome measures for

people with stroke (31). This is still an exceedingly high number

of outcomes and is not conducive to effective analyses. This high-

lights the need to refine and recommend a smaller core set of

variables for collection.

Recommendations of appropriate outcome measures should

be based on methods that synthesize high-quality evidence of

reliability and validity. Consensus meetings between clinicians,

researchers, and those directly affected by stroke could help to

identify recommended assessment measures.

In a climate of scarce resources, it is vital that design and

delivery of rehabilitation interventions are informed by a robust

evidence base, ideally across several trials. Future efforts should

also focus on those interventions that are most likely to have

efficacy. This may be achievable through meta-analyses and use of

pooled data to generate and test hypotheses. In order to maximize

the value, utility, and comparability of these data, there is an

urgent need for standardization of data collection. In addition,

there is a need to increase awareness of using ordinal scales that

requires a different statistical approach than analysis of continu-

ous data and for valid translations of instruments into other

languages, taking into account different cultural influences.

Previous initiatives have identified the need for collection of a

core dataset of common variables, and in the acute stroke research

setting, rigorous protocols have been implemented to identify

what these common data elements should be. The next steps

should involve similarly rigorous identification and consistent

implementation of data collection in stroke rehabilitation

research. Standardizing data elements can decrease study startup

times, facilitate data sharing, and promote informed clinical

guidelines (29). Although there is scope to utilize assessment tools

specific to the needs of the individual trial, there is a greater

need for clinicians and researchers to use consistent measurement

tools to ensure comparability of data in future meta-analyses and

secondary analyses. Rehabilitation deals inherently with change.

Therefore, stroke rehabilitation trials need to employ robust

outcome measures that can detect clinically meaningful changes

in the patient population. Although achieving consensus on

which outcome measures to use in stroke rehabilitation trials is a

considerable task, perhaps a feasible starting point is to have

consistency across all rehabilitation trials in the collection of data

on demography, stroke severity, and stroke onset to inclusion

times. Longer term goals could include the development of a

consensus process to establish the core dataset, which would be

endorsed by leading researchers, funders, and journal editors.

Summary

Stroke rehabilitation is challenging; this is reflected in the

complex nature of rehabilitation trials. Issues such as the lack of

data comparability, the multitude of trial or impairment-specific

outcome measures, inconsistent collection of data, and relatively

small sizes of rehabilitation trials hinder meta-analyses to inform

clinical practice. These challenges can be overcome, and road

maps have been put forward by numerous initiatives. Achieving

consensus on the selection of outcome measures may take time;

however, a starting point would be to consistently collect data on

patient demography, initial stroke severity, and time since stroke.

Consensus on which outcome measures to collect will require

broad agreement between clinicians, researchers, patient groups,

funders, and journal editors in order to be sustainable.

References1 Royal College of Physicians. Rehabilitation medicine. 2012. Avail-

able at http://www.rcplondon.ac.uk/specialty/rehabilitation-medicine(accessed 7 February 2012).

2 Govan L, Weir CJ, Langhorne P, for the Stroke Unit Trialists’ Collabo-ration. Organized inpatient (Stroke Unit) care for stroke. CochraneDatabase Syst Rev 2007; (4):CD000197.

3 English C, Hillier S. Circuit class therapy for improving mobility afterstroke: a systematic review. J Rehabil Med 2011; 43:565–71.

4 Middleton S, McElduff P, Ward J et al. Implementation of evidence-based treatment protocols to manage fever, hyperglycaemia, and swal-lowing dysfunction in acute stroke (QASC): a cluster randomisedcontrolled trial. Lancet 2011; 378:1699–706.

5 Langhorne P, Taylor G, Murray G et al. Early supported dischargeservices for stroke patients:a meta-analysis of individual patients’ data.Lancet 2005; 365:501–06.

6 Legg L, Langhorne P, Drummond AE, Gladman JR, Logan PA, WalkerMF. Rehabilitation therapy services for stroke patients living at home:systematic review of randomised controlled trials. Lancet 2004; 363:356.

7 Moseley AM, Herbert RD, Sherrington C, Maher CG. Evidencefor physiotherapy practice: a survey of the physiotherapy evidencedatabase (PEDro). Aust J Physiother 2002; 48:43–9.

8 Foley N, Teasell R, Jutai J, Bhogal S, Kruger E. Upper extremityinterventions. 2011. Available at http://www.ebrsr.com/uploads/Module-10_upper-extremity.pdf (accessed 28 September 2011).

9 Brady MC, Kelly H, Enderby P. Speech and language therapy foraphasia follow stroke. Cochrane Database Syst Rev 2010; (5)CD000425.DOI: 10 1002/14651858 CD000425 pub2 2012.

10 Quinn TJ, Dawson J, Walters MR, Lees KR. Functional outcomemeasures in contemporary stroke trials. Int J Stroke 2009; 4:200–5.

ReviewM. Ali et al.


Vol 8, January 2013, 18–24 23

11 Lees R, Fearon P, Harrison J, Broomfield NM, Quinn TJ. Cognitiveand mood assessment in stroke research – focussed review of contem-porary studies. Stroke 2012; 43:1678–80.

12 Lees KR, Bath PMW, Schellinger PD et al. Contemporary outcomemeasures in acute stroke research: choice of primary outcomemeasure. Stroke 2012; 43:1163–70.

13 Ali M, Bath P, Brady M et al. Development, expansion and use of astroke clinical trials resource for novel exploratory analyses. Int J Stroke2012; 7:133–8.

14 Vahedi K, Hofmeijer J, Juttler E et al. Early decompressive surgery inmalignant infarction of the middle cerebral artery: a pooled analysis ofthree randomised controlled trials. Lancet Neurol 2007; 6:215–22.

15 Greer DM, Funk SE, Reaven NL, Ouzounelli M, Uman GC. Impact offever on outcome in patients with stroke and neurologic injury acomprehensive meta-analysis. Stroke 2008; 39:3029.

16 Hart RG, Benavente O, McBride R, Pearce LA. Antithrombotic therapyto prevent stroke in patients with atrial fibrillation: a meta analysis.Ann Intern Med 1999; 131:492–501.

17 Lees KR, Bluhmki E, von Kummer R et al. Time to treatment withintravenous alteplase and outcome in stroke: an updated pooledanalysis of ECASS, ATLANTIS, NINDS, and EPITHET trials. Lancet2010; 375:1695–703.

18 Hacke W, Donnan G, Fieschi C et al. Association of outcome withearly stroke treatment: pooled analysis of ATLANTIS, ECASS, andNINDS rt-PA stroke trials. Lancet 2004; 363:768–74.

19 VISTA publications. 2010. Available at http://www.vista.gla.ac.uk/vista_publicationsaspx

20 Uniform Data System for Medical Rehabilitation. 2012. Available athttp://www.udsmr.org/WebModules/FIM/Fim_Aboutaspx (accessed29 June 2012).

21 Ali M, Ashburn A, Bowen A et al. VISTA-Rehab: a resource for strokerehabilitation trials. Int J Stroke 2010; 5:447–52.

22 Kwakkel G, Wagenaar RC, Kollen BJ, Lankhorst GJ. Predicting dis-ability in stroke. A critical review of the literature. Age Ageing 1996;25:479–89.

23 Giaquinto S, Buzzelli S, Di Francesco L et al. On the prognosis ofoutcome after stroke. Acta Neurol Scand 1999; 100:202–8.

24 Counsell C, Dennis M. Systematic review of prognostic models inpatients with acute stroke. Cerebrovasc Dis 2001; 12:159–70.

25 Fugl-Meyer AR, Jaasko L, Leyman I, Olsson S, Steglind S. Thepost-stroke hemiplegic patient. Scand J Rehabil Med 1975; 7:13–31.

26 Gladstone D, Danells CJ, Black SE. The Fugl-Meyer assessmentof motor recovery after stroke: critical review of its measurementproperties. Neurorehabil Neural Repair 2002; 16:232–40.

27 Weimar C, Kurth T, Kraywinkel K et al. Assessment of functioning anddisability after Ischaemic stroke. Stroke 2002; 33:2053–9.

28 Tendal B, Nuesch E, Higgins JPT, Juni P, Gotzsche PC. Multiplicity ofdata in trial reports and the reliability of meta-analyses: empiricalstudy. BMJ 2011; 343:d4829. doi:10.1136/bmj.d4829

29 Saver JL, Warach S, Janis S et al. Standardizing the structure of strokeclinical and epidemiologic research data the National Institute ofNeurological Disorders and Stroke (NINDS) stroke common dataelement (CDE) project. Stroke 2012; 43:967–73.

30 Collaborative stroke audit and research (COSTAR). 2012. Availableat http://webarchive.nationalarchives.gov.uk/+/www.dh.gov.uk/en/Aboutus/Researchanddevelopment/AtoZ/Cardiovasculardiseaseandstroke/DH_4002086 (accessed 3 February 2012).

31 StrokEDGE documents. 2012. Available at http://www.neuropt.org/files/Final_EDGE_Recommendations_Spread_Sheet.pdf (accessed 3February 2012).

Supporting Information

Additional Supporting Information may be found in the online

version of this article:

Appendix: VISTA-Rehab Steering Committee.



24 Vol 8, January 2013, 18–24

More outcomes than trials: a call for consistent data collection across stroke rehabilitation trials

Documents

Transcript of More outcomes than trials: a call for consistent data collection across stroke rehabilitation trials