Nephrol Dial Transplant (2015) 0: 1–10doi: 10.1093/ndt/gfv209

NDT Perspectives

How to routinely collect data on patient-reportedoutcome and experience measures in renal registriesin Europe: an expert consensus meeting

Kate Breckenridge1, Hillary L. Bekker2, Elizabeth Gibbons3, Sabine N. van der Veer4,5, Denise Abbott6,

Serge Briançon7, Ron Cullen1, Liliana Garneata8, Kitty J. Jager9, Kjersti Lønning10, Wendy Metcalfe11,12,

Rachael L. Morton13,14, Fliss E.M. Murtagh15, Karl Prutz16,17, Susan Robertson18, Ivan Rychlik19,

Steffan Schon16,20, Linda Sharp21, Elodie Speyer22,23, Francesca Tentori24 and Fergus J. Caskey1,25

1UK Renal Registry, Southmead Hospital, Bristol, UK, 2Leeds Institute of Health Sciences, School of Medicine, University of Leeds, Leeds, UK,3Health Services Research Unit, Nuffield Department of Population Health, University of Oxford, Oxford, UK, 4European Renal Best Practice,

Methods Support Team, University Hospital Ghent, Ghent, Belgium, 5Health eResearch Centre, Farr Institute for Health Informatics Research,

University of Manchester, Manchester, UK, 6The National Kidney Federation, Shireoaks, Worksop, UK, 7CHU de Nancy, Epidémiologie et

évaluation cliniques, InsermCIC 1433, Nancy, France, 8‘Dr Carol Davila’ Teaching Hospital of Nephrology, Bucharest, Romania, 9European Renal

Association–European Dialysis and Transplant Association Registry, Department ofMedical Informatics, AcademicMedical Center, University of

Amsterdam, Amsterdam, The Netherlands, 10Section of Nephrology, Department of Organ Transplantation, Oslo University Hospital

Rikshospitalet, Oslo, Norway, 11Renal Unit, Royal Infirmary of Edinburgh, Edinburgh, UK, 12The Scottish Renal Registry, Glasgow, UK, 13School

of Public Health, The University of Sydney, Sydney, Australia, 14Health Economics Research Centre, Nuffield Department of Population Health,

University of Oxford, Headington, UK, 15Department of Palliative Care, Policy & Rehabilitation, King’s College London, Cicely Saunders Institute,

London, UK, 16Swedish Renal Registry, Jönköping, Sweden, 17Department of Internal Medicine, Hospital of Helsingborg, Helsingborg, Sweden,18Dumfries and Galloway Royal Infirmary, Dumfries, UK, 192nd Department of Medicine, 3rd Faculty of Medicine, Charles University, Prague,

Czech Republic, 20Diaverum Renal Services Group, Lund, Sweden, 21National Cancer Registry Ireland, Cork, Ireland, 22CHU Nancy, Pôle QSP2,

Epidémiologie et Evaluation Cliniques, Nancy, France, 23Université de Lorraine, Université Paris Descartes, Nancy, France, 24Arbor Research

Collaborative for Health, Ann Arbor, MI, USA and 25School of Social and Community Medicine, University of Bristol, Bristol, UK

Correspondence and offprint requests to: Fergus J. Caskey; E-mail: fjcaskey@doctors.org.uk

ABSTRACT

Despite the potential for patient-reported outcome measures(PROMs) and experience measures (PREMs) to enhance under-standing of patient experiences and outcomes they have not, todate, been widely incorporated into renal registry datasets. Thisreport summarizes the main points learned from an ERA-EDTAQUEST-funded consensus meeting on how to routinely collectPROMs and PREMs in renal registries in Europe. In preparation

for the meeting, we surveyed all European renal registries to es-tablish current or planned efforts to collect PROMs/PREMs. Asystematic review of the literature was performed. Publicationsreporting barriers and/or facilitators to PROMs/PREMs collec-tion by registries were identified and a narrative synthesis under-taken. A group of renal registry representatives, PROMs/PREMsexperts and patient representatives then met to (i) share any ex-perience renal registries in Europe have in this area; (ii) establishhow patient-reported data might be collected by understandinghow registries currently collect routine data and how patient-re-ported data is collected in other settings; (iii) harmonize the fu-ture collection of patient-reported data by renal registries in© The Author 2015. Published by Oxford University Press on behalf of ERA-

EDTA. This is an Open Access article distributed under the terms of the CreativeCommons Attribution Non-Commercial License (http://creativecommons.org/licenses/by-nc/4.0/), which permits non-commercial re-use, distribution, andreproduction in any medium, provided the original work is properly cited. Forcommercial re-use, please contact journals.permissions@oup.com

1

NDT Advance Access published May 16, 2015 at U

niversity of Leeds on M

ay 18, 2015http://ndt.oxfordjournals.org/

Dow

nloaded from

Europe by agreeing upon preferred instruments and (iv) to iden-tify the barriers to routine collection of patient-reported data inrenal registries in Europe. In total, 23 of the 45 European renalregistries responded to the survey. Two reported experience incollecting PROMs and three stated that they were actively explor-ing ways to do so. The systematic review identified 157 potential-ly relevant articles of which 9 met the inclusion criteria and wereanalysed for barriers and facilitators to routine PROM/PREMcollection. Thirteen themes were identified and mapped to athree-stage framework around establishing the need, setting upand maintaining the routine collection of PROMs/PREMs. Atthe consensus meeting some PROMs instruments were agreedfor routine renal registry collection (the generic SF-12, the dis-ease-specific KDQOL™-36 and EQ-5D-5L to be able to derivequality-adjusted life years), but furtherworkwas felt to be neededbefore recommending PREMs. Routinely collecting PROMs andPREMs in renal registries is important if we are to better under-stand what matters to patients but it is likely to be challenging;close international collaboration will be beneficial.

Keywords: patient-reported measures, quality indicators,registry

INTRODUCTION

Established renal failure is a chronic disease with significant as-sociatedmorbidity. Although it affects a small proportion of thegeneral population (0.06–0.12% in countries across Europe in2012) (Table A.4.3 in [1]), the quality of life of those affectedis markedly lower than for those with most other chronic con-ditions and cancers [2]. Standard dialysis provides the equiva-lent of only 10% of kidney function so many patients arechronically tired, depressed and suffer pain; and many ofthese symptoms go unrecognized [3].

The success of treatments for renal failure has been historic-ally assessed using measures considered important by doctors,such as phosphate level or urea clearance. Although instru-ments measuring the patient’s perspective have been availablefor decades, their incorporation into routine clinical practicehas until recently been slow. A number of patient-reportedmeasures exist:

• Patient-reported outcome measures (PROMs) include anymetric assessing health, illness or health care benefits fromthe patient’s perspective; in general they take the form of aquestionnaire and more specifically a quality of life or symp-tom questionnaire. In routine clinical practice, PROMs havethe potential to highlight relevant symptoms and changes insymptoms, promote patient engagement in their treatment[4] and improve patient outcomes [5]. Summarizing PROMresults across individual patients, for example at the level oftreatments or hospitals, they could be used to inform a pa-tient’s choice of treatment or assess quality of care acrossdifferent hospitals [6, 7]. The importance of PROMs as endpoints in clinical trials is also increasingly being recognized [8].

• Patient-reported experience measures (PREMs) capture in-formation about the healthcare experience as perceived by

the patient. They can refer to a variety of issues, rangingfrom cleanliness of facilities to information provision, andfrom timeliness of transport to family members0 access tohealth professionals [9]. In routine clinical practice, PREMscan be used to improve quality in clinical services [10].

• Measures reflecting aspects of patient involvement in theirhealth care, including patient activation [11] and informedand shared decision-making [12].

Dialysis and transplantation are in the almost unique pos-ition of having an existing infrastructure of regional, nationaland international registries for collecting and reporting infor-mation on all patients receiving treatment. Quality of care, asmeasured against nationally and internationally agreed stan-dards, can be publicly reported and compared between centresand between countries. These instruments were usually devel-oped formeasurement at the patient level, however, and cautionwill need to be exercised when comparing differences betweencentres or countries until more is known about their perform-ance at these levels. To be able to report symptom burden andquality of life-adjusted survival alongside laboratory measuressuch as haemoglobin, calcium, phosphate and dialysis dosewould be a major step forward in reporting what is importantto patients. While some components of quality of life have beenshown to be modifiable (e.g. the physical component summaryscore of the SF-36 in the Frequent Haemodialysis NetworkShort Daily Trial [13]), others reflect the broader social con-struct in which the patient functions which so could prove tobe beyond the influence of the health care provider.

Part of this infrastructure is the ERA-EDTA Registry, whichcollects data on renal replacement therapy (RRT) via the nationaland regional renal registries in Europe: individual patient data isavailable from 31 national and regional registries in 17 countriesand aggregated data from a further 14 national registries [1]. Thedata items collected vary by country but can include demograph-ics, primary renal disease, RRT treatment history, date and causeof death, comorbidities, details of physical examination (e.g.weight, blood pressure), laboratory measurements (e.g. haemo-globin, albumin) and details of certain therapies given. At pre-sent, none of the national and regional registries reportpatient-reported outcome/experience measures to the ERA-EDTA Registry, but it is not certain what data are collectedlocally. This paper reports the results of a survey to capture exist-ing experiences of European renal registries in PROMandPREMcollection and a literature review of the facilitators and barriers toregistries routinely collecting PROMs and PREMs. It thenpresents the discussions and conclusions of an international con-sensus meeting, funded by the QUEST initiative of theERA-EDTA, aimed at promoting and harmonizing the routinecollection of patient-reported data by European renal registries.

SURVEY OF EUROPEAN RENAL REGISTRIES

Prior to the consensus meeting, the organizers contacted all 45European renal registries (using the ERA-EDTA Registry emailcontact list) asking for responses to the following questions:

NDTPERSPECTIV

ES

2 K. Breckenridge et al.

at University of L

eeds on May 18, 2015

http://ndt.oxfordjournals.org/D

ownloaded from

(i) Does your registry currently collect any quality of life orpatient satisfaction/experience measures?

(ii) If so, what measures do you use and how are these datacollected?

(iii) If not, have you tried to collect these measures in the pastand/or do you have plans to collect these measures in thefuture?

The survey did not ask about patient involvement question-naires, patient activation or shared decision making, as thesehad not been included in the original application for funding.Responses were received from 23 out of 45 registries. Only tworegistries (Austria and France) reported experience in the collec-tion of PROMs/PREMs and a further three registries reportedthat they were actively exploring this possibility (Norway,Romania and Sweden). From these responses, there was noobvious consensus on the instruments or methods to use inthe collection of PROMs and PREMs, although some barriersto the process were identified. These included low responserates, legal constraints and the burden on staff and patients.

LITERATURE REVIEW AND NARRATIVESYNTHESIS

A systematic review of the literature was conducted with theprimary aim of identifying facilitators and barriers to the rou-tine collection of patient-reported measures by chronic diseaseregistries. Medline and EMBASE databases were searched from1981 to 2013 to identify papers reporting on the routine collec-tion of patient-reported outcome, experience or quality of lifedata by chronic disease registries, including both renal andnon-renal registries. Full details of the search terms are avail-able in the Supplementary data. Abstracts of papers identifiedin the search were reviewed and full text versions obtainedof those which appeared potentially relevant. Two reviewers(K.B. and F.C.) independently screened the full-text papersfor relevancy, and—from the relevant papers—systematicallyidentified any recurring themes across studies.

The search yielded 762 hits, of which 157 papers were deemedto be potentially relevant at the title and abstract screening stage

based on pre-defined criteria; 9 papers were finally selected[14–22] and included in the narrative synthesis conductedaccording to guidelines provided by Popay et al. [23].

All nine of the selected papers were read by two independentresearchers (K.B. and F.C.) and the two most relevant papersused to conduct a preliminary synthesis [20, 21]. Relationshipsbetween the data were then explored using all nine studies todevelop a broad conceptual model in order to provide an appro-priate framework for further exploration of relationships in thedata (Figure 1). This framework comprised three stages: estab-lishing the need for, setting-up and maintaining routinePROM/PREM data collection. In total, 13 themes were identi-fied and agreed by two independent researchers (K.B. and F.C.)across the three stages of the conceptual model, each compris-ing a number of subthemes (Table 1). For each theme that re-sulted from the thematic analysis, we identified facilitators andbarriers to implementation as reported in the selected papers,which we then translated into a series of recommendations(listed in Table 1).

Establishing the need for PROM/PREM data in registrieswas largely broken down into two main themes: recogniz-ing the importance of PROMs and agreeing that, with allthe other data they collect, registries provide a good back-drop for the collection of patient-reported data.

Several of the themes and subthemes were identified asimportant in setting up PROM/PREM collection by regis-tries. Methodological issues associated with PROMs wereidentified, and recommendations developed based on theneed for PROMs expertise, national and internationalsupport, stakeholder involvement (including patient andpublic involvement) and international standardization.As well as these themes, some specific issues relating toresources and practical aspects were identified.

More exclusive tomaintaining a PROM/PREMprogrammewere themes related to ensuring useful, high-impact outputand the process of maintaining trust in the data. Severalthemes crossed the ‘set-up’ and ‘maintenance’ parts of themodel, including subthemes under design and evaluation,and technological/information governance.

F IGURE 1 : Framework for narrative synthesis.

NDTPERSPECTIV

ES

R e p o r t i n g P R O M s a n d P R E M s t o r e g i s t r i e s 3

at University of L

eeds on May 18, 2015

http://ndt.oxfordjournals.org/D

ownloaded from

Table 1. Summary of subthemes identified by the narrative synthesis process, organized by theme

Establishing need (1) Recognition of the importance of PROMs across all stakeholders:• Highlight the potential clinical outcomes of PROMs

• Target evidence at all stakeholders

• Highlight recommendations from national/international organizations

• Identify research supporting the need for PROMs

• Capitalize on the increasing recognition of PROMs in other diseases

• Utilize the power of patient organizations to lobby policy makers

(2) Agreement that registries are a good way to collect PROMs:• Registry-based collection may overcome some of the methodological limitations (e.g. larger samples, multiple time points,

ability to account for interactions)

• Registries can be used as a sampling frame to target specific subgroups

• Provide evidence that registries are a cost-effective way to collect these data

• Highlight ability to describe equity of access to treatment/quality assurance

• Highlight ability to describe characteristics of responders/non-responders

Set up (3) PROM methodological issues• Recognize methodological issues and how these will affect design and interpretation

• Recognize the importance of using properly translated and validated instruments for a population

• Consider undertaking a feasibility study when selecting the instruments

• Consider where the control data for measures came from

(4) PROM methodological expertise• Involve experts at all stages of project design and development

• Provide staff with training in collecting, analysing and interpreting PROMs data

(5) National and international support• Mandate or incentivise the collection of PROM data

• Obtain financial support from respected national and international organizations

• Coordinate expertise and infrastructure at a national level

• Understand laws and permissions in different countries with respect to PROMs

(6) Patient and public involvement• Generate interest among the public and patients

• Involve patients in objective setting and the design of data collection and reporting

• Involve an international umbrella organization of patient associations where possible

(7) International standardization• Agree on internationally standardized systems, definitions, data architecture and timings for data collection

• Aim for an internationally agreed core data set to enable international data pooling

• Design a system that can be easily adopted by subsequent countries wanting to join

(8) Stakeholder involvement in objective setting• Involve all stakeholders in objective setting and design

• Aim to reach consensus of key objectives at/before the design stage

• Define relationships and responsibilities at the beginning

• Avoid having too many stakeholders

(9) Practical considerations including resources• Obtain sufficient funding for staff, equipment etc

• Minimize the burden of administration at the clinic level

• Provide clear guidelines and training for staff administering PROMs

• Work out the most cost-effective way of collecting PROMs

• Consider using a PROM registry that is not disease specific so resources can be shared

• Set realistic timescales

(10) Design and ongoing evaluation/modification• Understand current data collection status of existing registries

• Offer a range of modes of completion for questionnaires, including paper and electronic.

• Align the data collection to objectives set by stakeholders

• Ensure design has capacity for flexibility as the project develops

• Set criteria for evaluation of project in advance

• Aim to minimize the burden to patients

• Consider dividing up the project into a number of work packages, each with its own lead

NDTPERSPECTIV

ES

4 K. Breckenridge et al.

at University of L

eeds on May 18, 2015

http://ndt.oxfordjournals.org/D

ownloaded from

A number of these themes, such as the need for stakeholderinvolvement, PROMs expertise and international standardiza-tion, were used when planning the consensus meeting. The re-sults of the review were also presented to the delegates duringthe preparatory session of the consensus meeting.

CONSENSUS MEETING

In June 2014, experts from across Europe participated in a con-sensus meeting in Bristol, UK. Invited to the meeting were par-ticipants representing all European renal registries with aninterest in collecting PROMs and PREMs routinely; expertson PROMs, PREMs and shared decision making; a representa-tive from the National Cancer Registry Ireland with experiencein routine PROMs collection among cancer patients, and;patient and carer representatives from a large UK kidney patientcharity the National Kidney Federation. The objectives of thismeeting were to (i) share any experience renal registries in Eur-ope already have in this area; (ii) establish how patient-reporteddata might be collected by understanding how registries cur-rently collect routine data and how patient-reported data is col-lected in other settings; (iii) harmonize the future collection ofpatient-reported data by renal registries in Europe by agreeingupon preferred instruments and (iv) identify the barriers toroutine collection of patient-reported data in renal registriesin Europe.

The opening sessions of the meeting comprised the presen-tation of the results of the narrative synthesis followed by a

series of presentations aimed at addressing the first two objec-tives. Representatives from four of the five renal registries withPROMs/PREMs experience (France [24], Norway, Romaniaand Sweden) and the National Cancer Registry Ireland sharedtheir experience of PROMs/PREMs data collection. The direct-or of the ERA-EDTA Registry then described the current stateof routine data collection in renal registries in Europe; this wasfollowed by presentations from three experts in PROMs,PREMs and shared decision making. Summaries of the threemethodological presentations follow.

EXPERT REPORTS

Patient-reported outcome measures—Elizabeth Gibbons,patient-reported outcome measurement group,University of Oxford, UK

PROMs capture patients’ perceptions of their health andquality of life, and vary according to factors such as methodof completion (e.g. paper-based or electronic) and content(ranging from general items tomore disease-specific symptoms).PROMs are useful for research purposes, clinical monitoring andmore recent applications include service improvement and na-tional benchmarking (e.g. the National PROMs programme inEngland for elective procedures). In addition to this establishedprogramme, several PROMs pilots are in progress in the UK,looking at long-term conditions (LTCs) in primary care, de-pression in secondary care, cardiac revascularization and skincancer.

Maintenance (11) Technological and information governance issues• Ensure lack of familiarity with technology does not limit participation

• Ensure technology has the capacity for flexibility over time

• Consider issues of data security and information governance

• Establish and address the legal and ethical constraints of the country/state

• Develop a coordinated IT infrastructure

• Consider availability of technology across different participating countries

• Maintain database so that patient information is up to date

• Obtain legal advice on the data sharing agreements that may be necessary

• Get the technology working before rolling it out-slow technology can be a barrier

(12) Useful high-impact output• Maintain interest by maximizing published output in a range of formats

• Target different stakeholder group with outcomes of PROMs work

• Present patient-level data in a readily understandable format

• Highlight the direct benefit to patients from participation

• Use ongoing nature of data to ensure frequent analysis and dissemination

• Data should aim to help improve patient care

• Make data as freely available as possible within the constraints of confidentiality

(13) Maintaining trust/faith in the data• Ensure methodological rigour to maintain trust

• Be aware of the sensitivity of centres to publication of data that reflects poorly on their performance

• Ensure objectives and evaluation are transparent and set by stakeholders and not any group for example with a vested interest

• Be aware that PROMs viewed more positively if presented as a care management tool

• Report characteristics of responders and non-responders NDTPERSPECTIV

ES

R e p o r t i n g P R O M s a n d P R E M s t o r e g i s t r i e s 5

at University of L

eeds on May 18, 2015

http://ndt.oxfordjournals.org/D

ownloaded from

Several challenges have been identified through such pilotstudies including poor response rates and concerns aboutbenchmarking and performance management. However bene-fits include the value of individual patient monitoring, plus thepotential for service improvement.

Selection of a PROM should be informed by a systematic re-view of the literature reporting psychometric properties andconsideration of the practicalities of data collection. In 2009the PROM group, University of Oxford, was commissionedby the Department of Health in England to review PROMsfor chronic kidney disease and established renal failure [25]. Re-commendations based on the strength of evidence included theSF-36 [26], EQ-5D-5L [27] and KDQOL™-36 [28] measures.

When selecting a PROM, consideration needs to be given tothe purpose of measurement and practicalities of data collec-tion. Complexity of scoring may outweigh the benefits of preci-sion whereas short versions of instruments with narrow focusand simple indices may not provide breadth of information.Data linkage and clinical information systems can support col-lection and feedback of the data but may be complex to developand maintain.

Patient-reported experience measures—Dr Sabine vander Veer, Amsterdam Medical Center, the Netherlands

Patient experience has long since been acknowledged as animportant dimension of quality of care [29]. Whereas patientsatisfaction is the perceived discrepancy between the expectedand experienced quality of care, the construct of patient experi-ence attempts to exclude the former element from the equation[30]. However, when translating these two constructs into ac-tual measurement instruments, the distinction between themoften becomes less clear.

During the consensus meeting, eight instruments were dis-cussed that were either available in English or on which an

English publication was available in PubMed [31–38]. Most in-cluded items on a broad spectrum of care delivery aspects [31,32, 35, 36, 38]; others focussed on capturing experiences withspecific aspects like education, or with treatment in general[32, 33, 36]. The number of items varied widely between instru-ments, ranging from eleven in the Renal Treatment SatisfactionQuestionnaire [33] to over 60 in the Scottish Renal PatientExperience Survey [36]. Only two instruments were applicableto any type of RRT [33, 37], while others were designed for one[32, 34] or more dialysis modalities [31, 35, 36, 38].

Almost all instruments were developed with input from pa-tients: for example, focus groups or interviews which identifiedrelevant aspects of care [31–33, 35, 38] or cognitive testing of apreliminary version of the instrument [32, 33, 38]. In mostcases, developers evaluated the instruments’ internal consist-ency [33–35, 38, 39]. Some also assessed construct validity byexploring the correlation between patient experience as mea-sured by the instrument and global assessments of satisfaction[34, 38], clinical performance [31] or health-related quality oflife [34]. Outside a development context, wide scale use has onlybeen reported for the CHOICE and the CAHPS questionnaire[40, 41].

Shared decision-making—Dr Hilary Bekker, Universityof Leeds, UK

Measures used to assess shared decision making are predom-inantly self-report questionnaires designed to evaluate a decisionsupport intervention’s effectiveness [12, 42–45] and/or screen fordecisional outcomes within usual care [46–51]. The substantialnumber of measures available reflects the complexity of these in-terventions in terms of their impact on different people withinthe process of delivering care and components needed to en-hance people’s active reasoning and engagement with others[52] (Figure 2).

F IGURE 2 : A framework representing informed, evidence-based and shared decisions with people and their healthcare roles.

NDTPERSPECTIV

ES

6 K. Breckenridge et al.

at University of L

eeds on May 18, 2015

http://ndt.oxfordjournals.org/D

ownloaded from

Decision support intervention types include:

• Patient Decision Aids [12, 45] enabling people to makeinformed decisions (point 1, Figure 2) between options byconsideration of accurate information about all optionsand their consequences without bias, evaluation of thisinformation with their values and making a decision ontrade-offs between evaluations [12, 53, 54]. There are severalpatient-reported informed decision outcome measures [37,38, 46–50, 55–59]. Alternatively, proxy outcomes maybe used to assess an aid’s impact by capturing people’sknowledge, risk perception, values, involvement, activation,usefulness intervention, value-choice consistency and/ordecision quality [12, 45, 60–62].

• Professional Decision Support [63] enabling professionals tomake evidence-based choices (point 2, Figure 2) by using thebest evidence available, in consultationwith the patient, to de-cide upon the option which best suits that patient [64].

• Shared Decision-Making Support [65–68] within patient-professional consultations enabling the process of choosinghealthcare collaboratively (point 3, Figure 2) by exchanginginformation, preferences and values about treatments, expli-cit reasoning about choices and agreeing a choice and im-plementation plan. Some measures assess patient-reportedshared decision making outcomes [48, 69, 70]. Proxy out-comes assess an aid’s impact on the professional (e.g. pro-vided option information, elicited values, awareness ofpatient experience, etc.), [71–73] the patient (e.g. askedquestions, provided values, awareness of professional view-point, etc.) [61, 62, 74–76] and/or the concordance betweenpatient-professional factors (e.g. SDM-Q-9; decisional con-flict) [77–79].

When informed [56, 57, 59] and shared [60] decision out-comes are used in renal services, findings suggest that theyare useful service-quality indicators [49, 50, 77, 80]. However,they may respond differently from application in other con-texts, as they are not designed for decisions taking place acrossmultiple consultations, health professionals and services, withdelayed implementation and chronically ill, elderly and/oroften frail patients [80].

REACHING A CONSENSUS

The second part of the meeting addressed the objective of har-monizing the routine collection of patient-reported data byrenal registries in Europe by agreeing on preferred instrumentsand discussing some of the practicalities of this process. Atten-dees were divided into four groups and asked to discuss whichof the PROM/PREM instruments on a list should be employed,how often they should be administered and in what format. Dueto time constraints and as they had not been included in the ori-ginal application for funding it was decided to exclude themeasurement of Shared Decision Making from the consensusdiscussions. Groups were chaired by four of the attendees,who later reported back to the whole meeting. Discussions

were also audio-recorded. Participants were also asked to voteon which PROMs/PREMs they thought would be most appro-priate based on the evidence outlined throughout the course ofthe meeting.

(i) PROM instruments. The overwhelming consensus wasthat any PROMs measurement programme should aimto support improvements in the quality of care for pa-tients. In terms of the measures adopted, there was agree-ment that the programme should aim to include bothgeneric and disease-specific measures if possible,whilst minimizing the overall length of questionnairesadministered. Of all the instruments discussed, theKDQOL™-36 seemed to be preferred by delegates as itoffers both generic and disease-specific outcomes. Ofthe generic instruments, the SF-12 [81] was themost pre-ferred. The importance of capturing patient symptomswas recognized, especially if a generic health-relatedquality of life instrument was being used, but no pre-ferred symptom burden instrument was agreed on.There was not complete agreement on whether apreference-based measure should be included, but con-sensus suggested that it would be useful to have a meas-ure that would allow health economic evaluationsprovided that the length was not prohibitive. TheEQ-5D-5L was the preferred instrument for this purposebut it was recognized that SF-6D [82], which could alsoprovide the utility data necessary for calculating qualityadjusted life years in health economic evaluations,could be derived from longer SF instruments (Table 2)[82, 83].

(ii) PREM instruments. Delegates were much less familiarwith the various PREM instruments available. Therefore,although the strengths and weaknesses of the various in-struments had been presented to them earlier in themeeting, there was a broad consensus that more workwas needed to recommend specific PREMs for broaderuse.

(iii) Patient groups to be covered. There was a clear consensusthat the aim should be to include all patients on RRT inany PROMs/PREMs programme, and that data shouldbe collected on at least an annual basis. The possibility

Table 2. Summary of recommendations from consensus discussions forroutine renal registry PROM collection

Consensus

Which instrumentsGeneric SF-12Preference-based EQ-5D-5LKidney specific KDQOL™-36

Practical issuesWho? All patients on RRTWhen? At least annually

Preferably not during dialysisHow? Unassisted self-report

No clear preference for paper/webOther issues Consider ethics/consent/data protection

Conduct initial pilot study

NDTPERSPECTIV

ES

R e p o r t i n g P R O M s a n d P R E M s t o r e g i s t r i e s 7

at University of L

eeds on May 18, 2015

http://ndt.oxfordjournals.org/D

ownloaded from

of extending coverage to pre-dialysis patients in the fu-ture, when these patients are captured by registries, wasdiscussed. The importance of making patients aware ofhow the data were being used was emphasized if high re-sponse rates were to be achieved from a broad range ofpatients. Several issues relating to the timing of data col-lection were discussed, and the general view was that al-though it may be useful to collect data at specific timepoints (e.g. in relation to commencement of RRT), thismay not be feasible. As regards timings, the patient rep-resentative suggested that patients may not wish to com-plete PROMs/PREMs questionnaires whilst attending fordialysis and PROM experts raised concerns that re-sponses given while on dialysis may be sensitive to thecurrent dialysis experience rather than ‘usual’ health-related quality of life. No clear preference for paper orweb-based reporting emerged, but the group felt thatdata should be collected via unassisted self-reporting.Other issues that featured prominently in discussionswere the need to consider the legal, data protection andconsent constraints of participating countries, and thepossibility of a pilot study to assess feasibility.

There are limitations to this report. There is a scarcity of ro-bust data on measuring and reporting PROMs and PREMs inkidney patients. Further, as mentioned above, the instrumentsavailable were generally developed for measurement at the in-dividual patient level, rather than the centre or national level. Insomeways these limitations reflect the attempt to reach consen-sus across European renal registries before instruments andprocesses become established.

SUMMARY

From the survey of European renal registry representatives anddiscussions with those attending the meeting there seems to bea widespread acceptance of the need to extend renal registrydata collection to capture patient-reported outcomes and ex-perience. It is recognized that this represents a considerablechallenge for renal registries and services if all the potential ben-efits from the information gathered—such as focussing consul-tations on what matters to patients, having real-life informationon quality of life for decision-making, incorporating patientmeasures into the quality assurance of renal units, researchand improving services—are to be realized. Establishing a dia-logue between registries around PROMs/PREMs and agreeingon some preferred measures at this stage will hopefully enablethe sharing of expertize and experience and stimulate the col-lection, and facilitate the collection and comparison of patients’outcomes and experiences across Europe in the future.

SUPPLEMENTARY DATA

Supplementary data are available online at http://ndt.oxfordjournals.org.

ACKNOWLEDGEMENT

This work was funded by the ERA-EDTA Quest initiative.

CONFLICT OF INTEREST STATEMENT

None declared.

REFERENCES

1. ERA-EDTA Registry. ERA-EDTA Registry 2014 Annual Report.Amsterdam: Department of Medical Informatics, Academic MedicalCentre, 2014

2. Mittal SK, Ahern L, Flaster E et al. Self-assessed physical and mental func-tion of haemodialysis patients. Nephrol Dial Transplant 2001; 16:1387–1394

3. Murtagh FE, Addington-Hall J, Higginson IJ. The prevalence of symptomsin end-stage renal disease: a systematic review. Adv Chronic Kidney Dis2007; 14: 82–99

4. Greenhalgh J. The applications of PROs in clinical practice: what are they,do they work, and why? Qual Life Res 2009; 18: 115–123

5. Etkind SN, Daveson BA, Kwok W et al. Capture, transfer, and feedback ofpatient-centered outcomes data in palliative care populations: does it makea difference? a systematic review. J Pain Symptom Manage 2015; 49:611–624

6. Appleby J. Patient reported outcome measures: how are we feeling today?BMJ 2012; 344: d8191

7. Black N. Patient reported outcome measures could help transform health-care. BMJ 2013; 346: f167

8. Calvert M, Blazeby J, Altman DG et al. Reporting of patient-reported out-comes in randomized trials: the CONSORT PRO extension. JAMA 2013;309: 814–822

9. Crow R, Gage H, Hampson S et al. The measurement of satisfaction withhealthcare: implications for practice from a systematic review of the litera-ture. Health Technol Assess 2002; 6: 1–244

10. Haugum M, Danielsen K, Iversen HH et al. The use of data from nationaland other large-scale user experience surveys in local quality work: a sys-tematic review. Int J Qual Health Care 2014; 26: 592–605

11. Hibbard JH, Stockard J, Mahoney ER et al. Development of the PatientActivation Measure (PAM): conceptualizing and measuring activation inpatients and consumers. Health Ser Res 2004; 39(4 Pt 1): 1005–1026

12. BekkerH, Thornton JG, Airey CM et al. InformedDecisionMaking: anAn-notated Bibliography and Systematic Review. Basingstoke, UK: HealthTechnology Assessment Number 3: NHS R&D, 1999

13. Group FHNT, Chertow GM, Levin NW et al. In-center hemodialysis sixtimes per week versus three times per week. New Engl J Med 2010; 363:2287–2300

14. Daly BJ, Douglas SL, Foley H et al. Psychosocial registry for persons withcancer: a method of facilitating quality of life and symptom research. Psy-chooncology 2007; 16: 358–364

15. Pugliatti M, Eskic D, Mikolcic T et al. Assess, compare and enhance the sta-tus of Persons with Multiple Sclerosis (MS) in Europe: a European Registerfor MS. Acta Neurol Scand 2012; 195: 24–30

16. Ghatnekar O, Eriksson M, Glader EL. Mapping health outcome measuresfrom a stroke registry to EQ-5D weights. Health Qual Life Outcomes 2013;11: 34

17. Norlin JM, Steen CK, Persson U et al. Analysis of three outcome measuresin moderate to severe psoriasis: a registry-based study of 2450 patients. Br JDermatol 2012; 166: 797–802

18. Paulsen A, Pedersen AB, Overgaard S et al. Feasibility of 4 patient-reportedoutcome measures in a registry setting. Acta Orthop 2012; 83: 321–327

19. van de Poll-Franse LV, Horevoorts N, van Eenbergen M et al. The PatientReported Outcomes Following Initial treatment and Long term Evaluationof Survivorship registry: scope, rationale and design of an infrastructure for

NDTPERSPECTIV

ES

8 K. Breckenridge et al.

at University of L

eeds on May 18, 2015

http://ndt.oxfordjournals.org/D

ownloaded from

the study of physical and psychosocial outcomes in cancer survivorship co-horts. Eur J Cancer 2011; 47: 2188–2194

20. Thong MS, Mols F, Stein KD et al. Population-based cancer registries forquality-of-life research: a work-in-progress resource for survivorship stud-ies? Cancer 2013; 119(Suppl 11): 2109–2123

21. Kennedy L, Craig AM. Global registries for measuring pharmacoeconomicand quality-of-life outcomes: focus on design and data collection, analysisand interpretation. Review 128 refs. PharmacoEconomics 2004; 22: 551–568

22. Boyce MB, Browne JP, Greenhalgh J. The experiences of professionals withusing information from patient-reported outcomemeasures to improve thequality of healthcare: a systematic review of qualitative research. BMJ QualSaf 2014; 23: 508–518

23. Popay J, Roberts H, Sowden A et al. Guidance on the Conduct of NarrativeSynthesis in Systematic Reviews. A Product from the ESRC Methods Pro-gramme. Lancaster: Lancaster University, 2006

24. Gentile S, Beauger D, Speyer E et al. Factors associated with health-relatedquality of life in renal transplant recipients: results of a national survey inFrance. Health Qual Life Outcomes 2013; 11: 88

25. Gibbons E, Fitzpatrick R. A Structured Review of Patient-Reported Out-come Measures for People with Chronic Kidney Disease. Oxford: Depart-ment of Public Health, University of Oxford, 2009

26. The Short Form-36 version 2 Health Survey. https://www.optum.com/optum-outcomes/what-we-do/health-surveys/sf-36v2-health-survey.html(3 May 2015, date last accessed)

27. The EQ-5D-5L. http://www.euroqol.org/eq-5d-products/eq-5d-5l.html(3 May 2015, date last accessed)

28. The Kidney Disease Quality of Life-36. http://www.rand.org/content/dam/rand/www/external/health/surveys_tools/kdqol/kdqol36.pdf (3 May 2015,date last accessed)

29. Donabedian A. Evaluating the quality of medical care. Milbank Mem FundQ 1966; 44(Suppl): 166–206

30. Sitzia J, Wood N. Patient satisfaction: a review of issues and concepts. SocSci Med 1997; 45: 1829–1843

31. Rubin HR, Jenckes MW, Fink NE et al. Patient’s view of dialysis care: de-velopment of a taxonomy and rating of importance of different aspects ofcare. CHOICE study. Choices for Healthy Outcomes in Caring for ESRD.Am J Kidney Dis 1997; 30: 793–801

32. US Agency for Healthcare Research and Quality (AHRQ) Centers forMedicare & Medicaid Services (CMS). CAHPS In-Center HemodialysisSurvey English version. Available from. https://www.cahps.ahrq.gov/surveys-guidance/ich/, 2006 (3 May 2015, date last accessed)

33. Barendse SM, Speight J, Bradley C. The Renal Treatment Satisfaction Ques-tionnaire (RTSQ): a measure of satisfaction with treatment for chronic kid-ney failure. Am J Kidney Dis 2005; 45: 572–579

34. Kirchgessner J, Perera-Chang M, Klinkner G et al. Satisfaction with care inperitoneal dialysis patients. Kidney Int 2006; 70: 1325–1331

35. Wasserfallen J-B, Moinat M, Halabi G et al. Satisfaction of patients onchronic haemodialysis and peritoneal dialysis. Swiss Med Wkly 2006;136: 210–217

36. NHS Quality Improvement Scotland. Scottish Renal Patient ExperienceSurvey. Dialysis report. Your service—your views, 2010

37. Fadem SZ, Walker DR, Abbott G et al. Satisfaction with renal replacementtherapy and education: the American Association of Kidney Patients sur-vey. Clin J Am Soc Nephrol 2011; 6: 605–612

38. Van der Veer SN, Jager KJ, Visserman E et al. Development and validation ofthe consumer quality index instrument tomeasure the experience and prior-ity of chronic dialysis patients. Nephrol Dial Transplant 2012; 27: 3284–3291

39. Wood R, Paoli CJ, Hays RD et al. Evaluation of the consumer assessment ofhealthcare providers and systems in-center hemodialysis survey. Clin J AmSoc Nephrol 2014; 9: 1099–1108

40. Palmer SC, de Berardis G, Craig JC et al. Patient satisfaction with in-centrehaemodialysis care: an international survey. BMJ Open 2014; 4: e005020

41. Paddison CM, Elliott MN, Haviland AM et al. Experiences of care amongMedicare beneficiaries with ESRD: Medicare Consumer Assessment ofHealthcare Providers and Systems (CAHPS) survey results. Am J KidneyDis 2013; 61: 440–449

42. Sepucha KR, Borkhoff CM, Lally J et al. Establishing the effectiveness of pa-tient decision aids: key constructs and measurement instruments. BMCMed Inform Decis Mak 2013; 13(Suppl 2): S12

43. Scholl I, Koelewijn-van Loon M, Sepucha K et al. Measurement of shareddecision making—a review of instruments. Zeitschrift fur Evidenz, Fortbil-dung und Qualitat im Gesundheitswesen 2011; 105: 313–324

44. Patel D, on behalf of The Evidence Centre. Helping People Share DecisionMaking. Helping People Share Decision Making, 2012

45. Stacey D, Bennett CL, Barry MJ et al. Decision aids for people facing healthtreatment or screening decisions, 2011

46. Legare F, Kearing S, Clay K et al. Are you SURE?: assessing patient decisionalconflict with a 4-item screening test. Can FamPhysician 2010; 56: e308–e314

47. Ferron Parayre A, Labrecque M, Rousseau M et al. Validation of SURE, afour-item clinical checklist for detecting decisional conflict in patients.MedDecis Making 2014; 34: 54–62

48. Bekker HL, Nye A, Walker E et al. SHARED: Patient Experience of SharedDecision Making Measure. 2012. https://www.gem-measures.org/public/MeasureDetail.aspx?mid=1517&cat=2 (3 May 2015, date last accessed)

49. Nye A, on behalf of the NHS Advancing Quality Alliance. Your Health,Your Decision. A report from the creating a receptive culture shared deci-sion making project. Right Care, NHS, 2013. http://arma.uk.net/wp-content/uploads/2013/05/Your-Health-Your-Decision-Evaluation-Report.pdf (3 May 2015, date last accessed)

50. Durand M-A, Bekker HL, Casula A et al. Measuring Shared Decision-Making in Chronic Kidney Care. Service Evaluation Report. The UKRenal Registry in collaboration with the NHS Institute, 2013

51. King E, Taylor J, Williams R et al. TheMAGIC Programme: Evaluation. AnIndependent Evaluation of theMAGIC (MakingGoodDecisions in Collab-oration) Improvement Programme. London, UK: The Health Foundation,2013

52. Craig P, Dieppe P, Macintyre S et al. Developing and evaluating complexinterventions: the new Medical Research Council guidance. BMJ 2008;337: a1655

53. Bekker HL. The loss of reason in patient decision aid research: do checklistsdamage the quality of informed choice interventions? Patient Educ Couns2010; 78: 357–364

54. Bekker HL, Winterbottom AE, Butow P et al. Do personal stories make pa-tient decision aids more effective? A critical review of theory and evidence.BMC Med Inform Decis Mak 2013; 13(Suppl 2): S9

55. Brehaut JC, O’Connor AM, Wood TJ et al. Validation of a decision regretscale. Med Decis Making 2003; 23: 281–292

56. O’Connor AM. Validation of a Decisional Conflict Scale. Med Decis Mak-ing 2010; 15: 25–30

57. Graham I, O’Connor AM. Preparation for Decision Making Scale—Usermanual, 2010

58. Holmes-Rovner M, Kroll J, Schmitt N et al. Patient satisfaction with healthcare decisions: the satisfaction with decision scale. MedDecis Making 1996;16: 58–64

59. O’Connor AM. User Manual—Stage of Decision Making, 2003. http://decisionaid.ohri.ca/docs/develop/User_Manuals/UM_Stage_Decision_making.pdf (3 May 2015, date last accessed)

60. Bekker HL, Legare F, Stacey D et al. Is anxiety a suitable measure of decisionaid effectiveness: a systematic review? Patient Educ Couns 2003; 50: 255–262

61. Mavis B, Holmes Rovner M, Jorgenson S et al. Patient participation in clin-ical encounters: a systematic review to identify self-report measures. HealthExpect 2014; doi: 10.1111/hex.12186. [Epub ahead of print]

62. Sepucha KR, Scholl I. Measuring shared decision making: a review of con-structs, measures, and opportunities for cardiovascular care. Circ Cardio-vasc Qual Outcomes 2014; 7: 620–626

63. Hunink M, Glasziou P, Siegel J et al. Decision Making in Health and Medi-cine: Integrating Evidence andValues. Cambridge, UK: CambridgeUniver-sity Press, 2001

64. Gray JAM. Evidence-based Healthcare: How to Make Health Policy andManagement Decisions. London, UK: London; 1997

65. Charles C, Gafni A, Whelan T. Shared decision-making in the medical en-counter: what does it mean? (or it takes at least two to tango). Soc Sc Med1997; 44: 681–692

66. Coulter A. Partnerships with patients: the pros and cons of shared clinicaldecision-making. J Health Serv Res Policy 1997; 2: 112–121

67. Stacey D, Legare F, Pouliot S et al. Shared decisionmakingmodels to informan interprofessional perspective on decision making: a theory analysis. Pa-tient Educ Couns 2010; 80: 164–172

NDTPERSPECTIV

ES

R e p o r t i n g P R O M s a n d P R E M s t o r e g i s t r i e s 9

at University of L

eeds on May 18, 2015

http://ndt.oxfordjournals.org/D

ownloaded from

68. Makoul G, Clayman ML. An integrative model of shared decision makingin medical encounters. Patient Educ Couns 2006; 60: 301–312

69. Lerman CE, Brody DS, Caputo GC et al. Patients’ perceived involvement incare scale: relationship to attitudes about illness andmedical care. J Gen In-tern Med 1990; 5: 29–33

70. Clayman ML, Makoul G, Harper MM et al. Development of a shared deci-sion making coding system for analysis of patient-healthcare provider en-counters. Patient Educ Couns 2012; 88: 367–372

71. Elwyn G, Barr PJ, Grande SW et al. Developing CollaboRATE: a fast andfrugal patient-reported measure of shared decision making in clinical en-counters. Patient Educ Couns 2013; 93: 102–107

72. Elwyn G, Hutchings H, Edwards A et al. The OPTION scale: measuring theextent that clinicians involve patients in decision-making tasks. Health Ex-pect 2005; 8: 34–42

73. Kriston L, Scholl I, Holzel L et al. The 9-item Shared Decision MakingQuestionnaire (SDM-Q-9). Development and psychometric properties ina primary care sample. Patient Educ Couns 2010; 80: 94–99

74. De Silva D, on behalf of The Evidence Centre. Helping Measure Person-Centred Care. London, UK: The Health Foundation, 2014

75. Graham C, MacCormick S. Overarching questions for patient sur-veys: development report for the Care Quality Commission (CQC).National Patient Survey Co-ordination Centre. Picker Institute Eur-ope, 2012

76. Hibbard J, Gilburt H. Supporting People to Manage Their Health:An Introduction to Patient Activation. London, UK: The Kings Fund, 2014

77. FishM, Coyne E, Ferraro A. The utility of the decisional conflict tool in a pre-dialysis clinical (Abstract). In: British Renal Conference. Manchester, 2013

78. Kasper J, Heesen C, Kopke S et al. Patients’ and observers’ perceptions ofinvolvement differ. Validation study on inter-relating measures for shareddecision making. PloS one 2011; 6: e26255

79. Legare F, Leblanc A, Robitaille H et al. The decisional conflict scale: movingfrom the individual to the dyad level. Zeitschrift fur Evidenz, Fortbildungund Qualitat im Gesundheitswesen 2012; 106: 247–252

80. Winterbottom AE, Gavaruzzi T, Mooney A et al. Patient Acceptability of theYorkshire Dialysis Decision Aid (YoDDA) disseminated within predialysisservices: a non-randomised controlled study. 2014; (submitted for publication)

81. The Short Form-12 version 2 Health Survey. https://www.optum.com/optum-outcomes/what-we-do/health-surveys/sf-12v2-health-survey.html(3 May 2015, date last accessed)

82. Brazier J, Roberts J, Deverill M. The estimation of a preference-basedmeas-ure of health from the SF-36. J Health Econ 2002; 21: 271–292

83. Brazier JE, Roberts J. The estimation of a preference-based measure ofhealth from the SF-12. Med Care 2004; 42: 851–859

Received for publication: 31.1.2015; Accepted in revised form: 13.4.2015

NDTPERSPECTIV

ES

10 K. Breckenridge et al.

at University of L

eeds on May 18, 2015

http://ndt.oxfordjournals.org/D

ownloaded from