Guideline uptake is influenced by 6 implementability domains for creating and communicating...

Accepted Manuscript

Guideline uptake is influenced by 6 implementability domains for creating andcommunicating guidelines: A Realist Review

Monika Kastner, Onil Bhattacharyya, Leigh Hayden, Julie Makarski, Elizabeth Estey,Lisa Durocher, Ananda Chatterjee, Laure Perrier, Ian D. Graham, Sharon E. Straus,Merrick Zwarenstein, Melissa Brouwers

PII: S0895-4356(15)00007-4

DOI: 10.1016/j.jclinepi.2014.12.013

Reference: JCE 8787

To appear in: Journal of Clinical Epidemiology

Received Date: 7 March 2014

Revised Date: 21 November 2014

Accepted Date: 8 December 2014

Please cite this article as: Kastner M, Bhattacharyya O, Hayden L, Makarski J, Estey E, DurocherL, Chatterjee A, Perrier L, Graham ID, Straus SE, Zwarenstein M, Brouwers M, Guideline uptake isinfluenced by 6 implementability domains for creating and communicating guidelines: A Realist Review,Journal of Clinical Epidemiology (2015), doi: 10.1016/j.jclinepi.2014.12.013.

This is a PDF file of an unedited manuscript that has been accepted for publication. As a service toour customers we are providing this early version of the manuscript. The manuscript will undergocopyediting, typesetting, and review of the resulting proof before it is published in its final form. Pleasenote that during the production process errors may be discovered which could affect the content, and alllegal disclaimers that apply to the journal pertain.

ACCEPTED MANUSCRIPT

Guideline uptake is influenced by 6 implementability domains for creating

and communicating guidelines: A Realist Review

Monika Kastner

1*, Onil Bhattacharyya

1,2, Leigh Hayden

1, Julie Makarski

3, Elizabeth Estey

4, Lisa

Durocher3, Ananda Chatterjee

1, Laure Perrier

1,5, Ian D Graham

6, Sharon E Straus

1,2,7, Merrick

Zwarenstein8, Melissa Brouwers

1Li Ka Shing Knowledge Institute (LKSKI) of St. Michael's Hospital, Toronto, Ontario, Canada

2Keenan Research Centre of LKSKI of St. Michael’s Hospital, Toronto, Ontario, Canada

3Department of Oncology, McMaster University, & Escarpment Cancer Research Institute,

Hamilton, Ontario, Canada 3Strategic Policy, Planning & Initiatives, Health Services, Region of Peel, Toronto, Ontario,

Canada 5Continuing Education and Professional Development, Faculty of Medicine, University of

Toronto, Toronto, Ontario, Canada 6Department of Epidemiology and Community Medicine, University of Ottawa, & Ottawa

Hospital Research Institute, Ottawa, Ontario, Canada 7Faculty of Medicine, University of Toronto, Toronto, Ontario, Canada

8Department of Family Medicine, Schulich School of Medicine and Dentistry, Western

University, London, Ontario, Canada

*Correspondence to:

Monika Kastner

Li Ka Shing Knowledge Institute of St. Michael’s Hospital

209 Victoria Street, Room 722

Toronto, ON, M5B 1W8

Phone: 416-864-6060, x77367

Email:monika.kastner@utoronto.ca

Funding

This study was funded by Knowledge Translation Canada through a Canadian Institutes of

Health Research (CIHR)/Canadian Foundation for Innovation (CFI) grants, and a CIHR

AGREE-A3 grant.

ACCEPTED MANUSCRIPT

ABSTRACT

Objective: To identify factors associated with the implementability of clinical practice

guidelines (CPGs), and to determine what characteristics improve their uptake.

Study design and setting: We conducted a Realist Review, which involved searching multiple

sources (e.g., databases, experts) to determine what about guideline implementability works, for

whom, and under what circumstances. Two sets of reviewers independently screened abstracts

and extracted data from 278 included studies. Analysis involved the development of a codebook

of definitions, validation of data, and development of hierarchical narratives to explain guideline

implementability.

Results: We found that guideline implementability is associated with two broad goals in

guideline development: 1) creation of guideline content, which involves addressing the domains

of Stakeholder involvement in CPGs, Evidence synthesis, Considered judgment (e.g., clinical

applicability), and Implementation feasibility; and 2) the effective communication of this content,

which involves domains related to fine-tuning the CPG’s Message (using simple, clear and

persuasive language) and Format.

Conclusions: Our work represents a comprehensive and interdisciplinary effort toward better

understanding which attributes of guidelines have the potential to improve uptake in clinical

practice. We also created codebooks and narratives of key concepts, which can be used to create

tools for developing better guidelines to promote better care.

Keywords: Guideline implementability; Guideline adherence; Clinical practice guideline;

Systematic Review; Realist review

ACCEPTED MANUSCRIPT

What is new?

Key finding

• We identified six domains of guideline implementability

hypothesized to affect uptake of recommendations within 2

broad categories: 1) the “creation” of guideline content, which

involves addressing Stakeholder involvement in guidelines,

Evidence synthesis, Considered judgment, and Implementation

feasibility; and 2) the “communication” of this content by fine-

tuning the guideline’s Message and Format

What this

adds to what

is known

• Building on the work of others, this is the first systematic

review to investigate guideline implementability from a

comprehensive and multi-disciplinary perspective (i.e.,

psychology, management, and human factors engineering)

• Our analysis moves beyond the medical and implementation

science literatures (which tend to focus on the creation of

content), and incorporates other disciplinary content, which

expands understanding of the relevant factors, particularly those

related to the communication of content

What is the

implication,

what should

change now?

• Guidelines summarize clinical evidence in order to inform

clinicians’ decision making, but how they are developed and

written influences how often they are used

• Our work represents an important, comprehensive and

interdisciplinary effort toward better understanding of which

attributes of guidelines have the potential to improve uptake in

clinical practice

• We created narratives of key concepts, which can be used to

develop tools to build better guidelines and promote better care

ACCEPTED MANUSCRIPT

INTRODUCTION

Recent efforts in the clinical practice guideline (CPG) research enterprise have focused

on identifying factors that can be targeted to increase the uptake of recommendations to improve

patient outcomes and strengthen delivery systems. These efforts have included investigations

into extrinsic (changing the practice setting to facilitate recommendation use) and intrinsic

approaches (changing the guideline itself) to improving CPGs. Both approaches are needed, but

given the costs and context dependence of extrinsic approaches, investigation of intrinsic

approaches may lead to solutions at minimal cost and that may be more broadly applicable and

feasible. Shiffman et al. have referred to ‘implementability’ as the perceived characteristics of

guidelines that predict the relative ease of their implementation1. Gagliardi et al. developed a

framework of guideline implementability, which was tested with 20 different CPGs, and found

that elements related to guideline format (e.g., guidelines that provide summary versions) and

content (e.g., clinical considerations to individualize recommendations) provided the best

opportunities to modify CPGs for improved uptake2. Grol et al. found that guidelines that are

compatible with existing norms among the target group for implementation and those that do not

demand too much change to existing routines, extra resources, or acquisition of new knowledge

and skills were used more3. Michie et al. suggest that clarity, specificity of behavioural

instructions, and specific plans are important to get physicians to follow guidelines but these

factors have largely been overlooked4,5

Together, these contributions have served as an excellent foundation and have enabled a

dialogue within the health services research community about guideline implementability.

However, in developing this concept and understanding it further, the field has mostly focused

their efforts on the medical literature. In doing so, it has largely ignored relevant paradigms from

ACCEPTED MANUSCRIPT

other disciplines. For example, in the areas of social, cognitive and health psychology, there have

been decades of research developing theory and models to explain behavior change, persuasion,

motivation, and communication styles6-8.

There has been much discussion around improving the rigor of guidelines9, and

consequently, there are a number of tools to increase how the evidence supporting guideline

recommendations is synthesized, analyzed, and presented. These include AGREE II10

GRADE11

, GLIA1, ADAPTE

12 and CAN-IMPLEMENT

13. However, these tools are mostly

informed by the medical literature1,2

, and target methodological and reporting concerns.

Currently, no resources take a comprehensive view of all factors relevant to guideline

implementability and investigate this from other disciplines focused on changing human

behavior, such as psychology, marketing, design and human-factors engineering. To better

understand the concept of implementability and the relationship between characteristics of

guidelines and their uptake by clinicians (who represent a primary end users of CPGs), our

primary objective was to identify factors associated with the implemenatbility of CPGs and

recommendations through a comprehensive and multidisciplinary perspective. Our secondary

objective was to determine what characteristics are posited to improve uptake by whom, and

under what circumstances.

METHODS

We conducted a Realist Review14

, which is an explicitly theory-driven approach to the

synthesis of evidence as it seeks to interrogate the underlying mechanisms of the programs or

interventions being studied14

. We report our methods and findings according to the RAMESES

(Realist And Meta-narrative Evidence Synthesis: Evolving Standards) criteria for the publication

ACCEPTED MANUSCRIPT

standards of realist reviews16

, and include a flow diagram of our methods for increased clarity

(Figure 1).

[Figure 1 about here]

Search strategy

Consistent with methodological standards, we used a multiple search strategy that

consisted of 5 iterative stages of searching (see Figure 2): Stage 1: We consulted guideline

development and knowledge translation (KT) experts to identify a set of core articles in guideline

implementability supplemented with a background search by an information specialist in

MEDLINE and EMBASE (up to the year 2010) using the search terms: “implement*, “clinical

practice guidelines” and “knowledge translation”. Our candidate theories (i.e., reasons for poor

implementation of guidelines) were identified through consultation with guideline and KT

experts: i) Guidelines are not used in part because of specific perceived guideline characteristics

affecting uptake by clinicians (e.g., too complex or too difficult to follow); ii) There are tradeoffs

between various guideline attributes that facilitate or hinder uptake (e.g., increasing specificity

may reduce applicability). Stage 2: We sought the expertise of seven content experts known

amongst our team in Guideline development and KT (n = 3), Psychology (n = 2), Management

(n = 1), and Human factors engineering (n = 1) to suggest seminal articles or to provide

direction to where to look for articles in these disciplines (a method akin to snowball sampling in

qualitative research). Stage 3: We expanded our search by looking for the related articles of our

core Medicine citations identified in Stage 1 (n = 76). Reviewers independently selected a sub-

set of priority articles (highly cited or written by known researchers in guideline

implementability; n = 22), which were entered into the PubMed ‘related-articles’ interface

resulting in 1677 citations. Two reviewers scanned this yield independently using our inclusion

ACCEPTED MANUSCRIPT

criteria of which 131 were deemed potentially relevant. Stage 4: We searched the reference lists

of articles from stages 1 and 2 to determine if we had reached theoretical saturation. Duplicate

review of a random subset of citations did not identify new concepts or understandings, so we

did not pursue this further. Stage 5: We continuously sought out other potentially relevant

articles including the grey literature (e.g., unpublished documents and websites such as the

Guideline International Network). During all stages, we conducted saturation testing iteratively,

by asking at regular intervals of searching whether the latest sample of literature has added

anything new to our understanding and whether further searching is likely to add new

knowledge14

[Figure 2 about here]

Article selection process

Two sets of reviewers independently screened abstracts to look for our candidate theories

and to test for relevance. We included all article types written in English, and excluded opinion-

driven reports (editorials, commentaries, and letters) unless authored by an individual identified

by our team as an “expert” in the field of guideline implementability. Our unit of analysis was

not dependent on study design because relevant data could exist in any section of the article (e.g.,

Abstract; Introduction; Discussion), so we did not assess study quality14.

Data extraction

We developed a standardized data extraction form that was pilot tested independently by

two reviewers using five articles. Data was extracted by two primary reviewers and audited by

another two reviewers with disagreements resolved through group consensus on study

characteristics, discipline (e.g., medicine, psychology), guideline attribute name and definition,

ACCEPTED MANUSCRIPT

attribute operationalization (how the attribute functions within the discipline or context), attribute

relationship with guideline uptake, and any potential tradeoffs.

Data organization

We identified 1736 guideline attributes, which were initially sorted by the same name or

name root (e.g., valid/validity). Two groups of reviewers took the same list of sorted attributes

and independently clustered them into logical categories to determine how attributes are related

to one another. We then combined groups of similar attributes (including their synonyms and

antonyms) that conceptually “fit” within a larger theme, and created a label and description for

each category. Table 1 describes the operationalization of this process. We compared the

categorizations between the two reviewer groups for agreement by documenting “agreed” and

“divergent” classifications. This resulted in deriving a common set of 27 categories (and their

attributes) grouped into 5 broad domains that we conceptualized as associated with guideline

uptake: Language, Format, Rigor of development, Feasibility, and Decision-making. We

considered this conceptualization as our “raw data” for analysis.

[Table 1 about here]

Analysis

Data analysis involved a 3-level process where data were further interrogated and refined

with each level of synthesis: Level 1: Development of a codebook: We developed (in duplicate)

a codebook of definitions for each of the five identified domains and attributes to determine

recurrent patterns of outcomes (i.e., what makes guidelines implementable) and their associated

mechanisms and contexts. We included attributes that were deemed “modifiable” by guideline

developers (e.g., the actionability of recommendations vs. the self-efficacy of the guideline end-

user); documented its definition, operationalization, and the discipline and context they were

ACCEPTED MANUSCRIPT

studied; and noted any reported positive or negative relationships with guideline uptake and any

tradeoffs if existed. We subsequently looked at the data for consistency or divergence of

outcomes as a function of discipline. The various disciplines tended to focus on unique sets of

attributes; the data did not lend to a formal cross-discipline analysis; Level 2: Validation of data:

We determined which attributes were most appropriate for which domain/sub-domain to identify

the hierarchy of groupings and important relationships. We tested the validity of this

organization by administering an online survey to nine experts in KT and guideline development

known amongst our team. They were asked to review the organization, and to rename, rearrange

and condense as they saw fit. Accordingly, we collapsed and renamed some

attributes/categories/domains (e.g., the Language domain was renamed Message), and we added

a 6th domain (Stakeholder involvement) – see Table 2; Level 3: Hierarchical explanatory

narratives: We further expanded the knowledge derived from previous levels of analysis by

building up an explanation for what intrinsic factors about a guidelines promote (facilitator

attributes) or impede (barrier attributes) guideline use and uptake, under what circumstances this

happens (an explanation of situations/contexts/settings), the mechanism by which it happens (its

operationalization), and the outcomes and consequences of this. We did this by deriving

explanations for each of the broad domains and sub-domains from a summation of their smaller

parts.

RESULTS

Of 2550 potentially relevant articles identified, 350 articles were screened in full text and

278 articles contributed to the analysis from 170 journals across 7 disciplines: medicine (n =

188), psychology (n = 30), management/marketing/business (n = 23), human factors engineering

ACCEPTED MANUSCRIPT

(n = 17), information technology/computer science (n = 10), graphic design (n = 7), and

sociology (n = 3). The three most common study designs were narrative reviews (n = 110),

qualitative studies (n = 37) and, observational studies (n = 32). See Figure 3 for the flow of

article selection.

Using our hierarchical narrative approach, we built an explanation for each of the six

domains (comprising 27 attributes) showing that guideline implementability is associated with

two broad goals during the guideline development process: the Creation of guideline content (4

domains) and the Communication of this content (2 domains) – See Table 3 and Appendix A.

I. Creation of content in guidelines (see Table 4 and Appendix B). This category includes 4

domains:

1) Stakeholder involvement: Findings showed that the credibility of a guideline affects its

likelihood of uptake. The key determinants of credibility are the range of stakeholders

involved17

, disclosure of any conflicts of interest and funding sources19-22

. The literature suggests

that addressing stakeholder involvement entails ensuring the appropriate composition and

relevant and unbiased expertise of the guideline development group, documenting the views and

preferences of the target population (patients, public), and defining target end users10,23; 2)

Evidence synthesis: Synthesizing the appropriate evidence is a complex task for guideline

developers, but appears to be the most developed aspect of guideline development in the

literature, primarily originating from Medicine. Rigorous evidence synthesis is emphasized

across medicine, nursing, health policy, and the approaches reflect a common understanding of

how to conduct this process. The literature reflects the attributes deemed necessary to enhance

ACCEPTED MANUSCRIPT

guideline validity and reproducibility and comprises the consistent reporting of the elements that

need to be included in guidelines20,24-26, the execution of these elements26-28, and the currency of

guidelines29,30; 3) Considered judgment: The literature suggests that guideline developers need to

supplement evidence-based formulation of recommendations with considered judgment. This

involves the consideration of the evidence in light of other factors such the complex trade-offs

between competing benefits and harms and risks of different options for managing the disease or

condition, clinical applicability and contexts, the values and preferences of those for whom the

recommendations are intended (providers, patients, and developers), organizational needs, and

costs10,23,31-35

; 4) Implementation Feasibility: Feasibility involves the local applicability of

guidelines36, the consideration of resource constraints to make them more implementable10,37,38,

and the influence of the degree of novelty or familiarity of the interventions1,20. There is

sometimes a tension between feasibility and impact. A recommendation may be highly feasible

but unlikely to make much of a difference (it supports the status quo)39

. In contrast, a

recommendation may require far-reaching changes, and have enormous potential impact, but

have low feasibility10,27

II. The Communication of content to guideline end-users (Table 4 and Appendix B): This

category includes our final 2 domains: 5) Message: Studies suggest that to optimize the

messaging of a guideline and in turn its uptake, the language used in recommendations has to be

simple, clear, and persuasive. Literature spanned across all disciplines, with the psychology

literature focusing on how to reduce cognitive load and increase understanding and retention; the

management literature focusing on crafting convincing and salient arguments. It is also

suggested that the level of complexity of guidelines and recommendations is inversely

ACCEPTED MANUSCRIPT

proportional to its adoption40-45

and compliance46,47

. The bodies of literature did not contradict,

but rather focus on different aspects of messaging; 6) Format: A number of formatting aspects of

the guideline were identified from the literature, which allows the various elements of guidelines

to become more explicit38

, promoting their use in practice2,49-52

. These include having: i) multiple

versions of guidelines. This refers to the evolution of guidelines, which typically progress from a

research-based tool to a short version for clinical use, and to a lay-language version for

patients53

; ii) specific components within guidelines (e.g., purpose, methods, recommendations);

and iii) the presentation (appearance) of guidelines, which involves layout, structure, and

information visualization. Some of the common design principles for scientific communication

have an empirical foundation, but many are derived from best practices and user preferences.

Since most formatting principles are based on cognitive processes, they are likely to be

generalizable across disciplines and contexts.

DISCUSSION

Our synthesis of 278 articles from a multidisciplinary body of literature identified six

domains of guideline implementability that affect uptake of recommendations within 2 broad

categories (the creation and communication of guideline content). Building on the work of

Shiffman and Gagliardi, we investigated guideline implementability from a broad perspective.

Shiffman et al developed GLIA by searching the literature for guideline attributes in 5 key

reports describing impact of factors on successful implementation and by examining 3

instruments for the appraisal of guideline quality1. Gagliardi et al focused on 18 healthcare

studies and created an implementability framework comprised of 22 elements organized into 8

ACCEPTED MANUSCRIPT

domains (adaptability, usability, validity, applicability, communicability, accommodation,

implementation, and evaluation)2. We drew on a wide range of disciplines to further enhance this

work and involved appropriate experts in the review process. Our analysis moves beyond the

medical and implementation science literatures (which tend to focus on aspects of the Creation

of content), and incorporates other disciplinary content, which expands understanding related to

the Communication of content. Literature from human factors engineering adds information and

direction on how developers can structure guidelines to meet the specific needs of end-users and

mirror their work processes and approach to care. The business/management literature adds

insights and direction regarding the importance of persuasive and clear messaging while the

design literature outlines specific design principles, which are intended to improve the usability

and attractiveness of products. Literature from cognitive psychology alerts to the limitations of

information processing and provides some explicit strategies for developers to ease cognitive

We achieved rigor in our review through transparent documentation of the

methodological approach15

as well as for our search strategy, study selection, analysis and

interpretation, including any deviations from the original method. We also provided a detailed

flow diagram for our study methods (Figure 1) and article selection process (Figure 2). We also

adopted a consensus-based approach to clarify difficult concepts, where smaller groups of

investigators discussed their assumptions related to the meaning of a concept, sought

clarification from discipline-specific experts if needed, and then reached consensus within the

broader study team on the applicability of such concepts in the context of guideline

implementability.

ACCEPTED MANUSCRIPT

Our review suggests a few areas for further study. Guideline end-users commonly

complain about their length, and we know that simplifying language and content can improve

information processing and retention and improve perceived ease of use (and thus potentially

intention to use), so simplifying guidelines appears to be a worthy endeavor. However, we do

not know the threshold of simplification that could result in misinterpretation of guideline

recommendations. Determining what type of content ought to be simplified and in what ways

requires additional investigation to provide developers with guidance regarding a balanced

approach to creating and communicating guideline content. In addition, several issues of how to

operationalize the concept of Considered judgment remains unclear. An approach to help

developers facilitate the process is needed. Without practical procedures, it is likely that this step

will be conducted in a cursory fashion, or not at all.

Our work has some limitations. We chose a realist approach to develop theories about

what attributes of guidelines contribute to their implementability, for whom, and under what

circumstances. However, the literature was rarely explicit about context and circumstances of

guideline development and uptake, so it became difficult to develop and interrogate such theories

and to explore Context-Mechanism-Outcome relationships. Additionally, we had anticipated

exploring the similarities and differences between how the literatures understood guideline

implementability, but found few contradictions in the literature. Studies tended to focus on the

problems inherent in guidelines from different perspectives: usability and usefulness (human

factors engineering), information processing and retention (cognitive psychology), document

construction (design), consumer relations (management), and clinical relevance and importance

(medicine). As an alternative, we developed a conceptual framework to synthesize the data,

highlighting contextual issues through the domain narratives, positing relationships rather than

ACCEPTED MANUSCRIPT

proving them. Although we sought the expertise of content experts in Psychology, Management

and Human factors engineering, we may have missed other relevant articles in these disciplines

as we did not systematically search these beyond our expert-identified and snowball sampling

techniques. To offset this risk, we performed a “related-articles” search of “core” articles.

Additionally, literature from Medicine may be over-represented as the bulk of our core articles

were identified from this discipline. However, given that guidelines are most heavily discussed

in the health/medicine literature, we thought this was an acceptable risk of bias. There may be

relevant concepts from other disciplines we didn’t include, and there are areas that are not well

covered within the disciplines we studied. However, we incorporated a rigorous, systematic, and

transparent methodology and consulted experts at several key points during the analysis as a

check for external validity.

Conclusions and recommendations

Guidelines summarize clinical evidence to inform clinicians’ decision making, but how

they are developed and written influences how they are used. Our work represents a

comprehensive and interdisciplinary effort toward better understanding which attributes of

guidelines have the potential to improve uptake in clinical practice. However each of these will

need to be empirically tested to identify the best “package” of attributes. We also created

narratives of key concepts, which can be used to develop tools to determine their impact on

building better guidelines aimed at increasing their uptake and promoting better care.

Acknowledgements

We thank the following individuals for providing content expertise: Jamie Brehaut (Psychology);

Jeremy Grimshaw (Knowledge Translation); Stephen Hanna, Holger Schunemann, Francoise

ACCEPTED MANUSCRIPT

Cluzeau (Guideline development); Dilip Soman (Management); Mark Chignell (Human Factors

Engineering); Hyun Joo Lee (Design).

Ethical approval:

No ethics approval was required for this study.

ACCEPTED MANUSCRIPT

REFERENCES

1. Shiffman R DJ, Brandt C, Essaihi A, Hsiao A, Michel G, O’Connell R. . The Guideline

Implementability Appraisal (GLIA): development of an instrument to identify obstacles to

guideline implementation. BMC Med Info Dec Mak. 2005;5:23.

2. Gagliardi A, Brouwers MC, Palda VA, Lemieux-Charles L, Grimshaw JM. How can we

improve guideline use? A conceptual framework of implementability. Implementation

Science. 2011;6:26.

3. Grol R DJ, Thomas S, Veld C, Rutten G, Mokink H. Attributes of clinical guidelines that

influence use of guidelines in general practice: Observational study. BMJ. 1998;317:858-

4. Michie S, Johnston M. Changing clinical behaviour by making guidelines specific. BMJ.

02/07 2004;328(7435):343-345.

5. Michie S LK. Words that matter: increasing the implementation of clinical guidelines. Qual

Saf Health Care. 2005;14:367-370.

6. Rogers E. Diffusion of Innovations. New York: Free Press; 1995.

7. Rey G BF. The Expertise Reversal Effect: Cognitive Load and Motivational Explanations.

Journal of Experimental Psychology: Applied. 2011;17(1):33-48.

8. Wang A, Dowding T. Effects of visual priming on improving web disclosure to investors.

The Journal of Behavioral Finance. 2010;11(1):11-20.

9. Tricoci P AJ, Kramer JM, Califf RM, Smith SC Jr. . Scientific evidence underlying the

ACC/AHA clinical practice guidelines. JAMA. 2009;301(8):831-841.

10. Brouwers MC KM, Browman GP, Burgers JS, Cluzeau F, Feder G, et al. AGREE II:

Advancing guideline development, reporting and evaluation in health care. CMAJ.

2010;182(18):E839-842.

11. Atkins D, Best D, Briss PA, Eccles M, Falck-Ytter Y, Flottorp S, Guyatt GH, Harbour RT,

Haugh MC, Henry D, Hill S, Jaeschke R, Leng G, Liberati A, Magrini N, Mason J,

Middleton P, Mrukowicz J, O'Connell D, Oxman AD, Phillips B, Schünemann HJ, Edejer T,

Varonen H, Vist GE, Williams JW Jr, Zaza S; GRADE Working Group. Grading quality of

evidence and strength of recommendations. BMJ. 2004;328:1490.

12. Fervers B BJ, Haugh MC, Latreille J, Milka-Cabanne N, Paquet L, et al. Adaptation of

clinical guidelines: literature review and proposition for a framework and procedure. Int J

Qual Health Care. 2006;18(3):167-176.

13. Harrison MB GI, van den Hoek J, Dogherty EJ, Carley ME, Angus V. Guideline adaptation

and implementation planning: a prospective observational study. Implementation Science.

2013;8:49.

14. Pawson R GT, Harvey G, Walshe K. Realist review – a new method of systematic review

designed for complex policy interventions. . J Health Serv Res Policy. 2005;19(Suppl

1):S21-S34.

15. Kastner M EE, Perrier L, Graham ID, Grimshaw J, Straus SE, Zwarenstein M,

Bhattacharyya O. Understanding the relationship between the perceived characteristics of

clinical practice guidelines and their uptake: protocol for a realist review. Implementation

Science. 2011;6:69.

16. Wong G GT, Westhorp G, Buickingham J, Pawson R. RAMESES publication standards:

Realist synthesis. BMC Medicine. 2013;11:21.

ACCEPTED MANUSCRIPT

17. Solberg LI, Brekke ML, Fazio CJ, et al. Lessons from experienced guideline implementers:

attend to many factors and use multiple strategies. The Joint Commission journal on quality

improvement. 2000;26(4):171-188.

18. Muhlhauser I. From authority recommendations to fact-sheets--a future for guidelines.

Diabetologia. 2010;53(11):2285-2288.

19. The ADAPTE Process: Resource Toolkit for Guideline Adaptation. Version 2.0. 2009.

Available at: http://www.g-i-n.net/document-store/working-groups-

documents/adaptation/adapte-resource-toolkit-guideline-adaptation-2-0.pdf . Accessed on

September 7, 2014.

20. Rosenfeld RM SR. Clinical practice guideline development manual: A quality driven

approach for translating evidence into action. Otolaryngology Head Neck Surg. 2009;140(6

Suppl 1):S1-S43.

21. American Academy of Pediatrics Steering Committee on Quality Ia, Management.

Classifying recommendations for clinical practice guidelines. Pediatrics. 2004;114(3):874-

22. Tricoci P, Allen JM, Kramer JM, Califf RM, Smith SC, Jr. Scientific Evidence Underlying

the ACC/AHA Clinical Practice Guidelines. JAMA: The Journal of the American Medical

Association. 2009;301(8):831-841.

23. Horvath AR, Kis E, Dobos E. Guidelines for the use of biomarkers: principles, processes

and practical consideration. Scandinavian Journal of Clinical and Laboratory Investigations.

2010;70(1):109-116.

24. Grimshaw JM RI. Achieving health gain through clinical guidelines II: Ensuring guidelines

change medical practice. Quality in Health Care. 1994;3:45-52

25. Wolf M BD, Marbella AM, Casanova JE. . US Family Physicians' Experiences with Practice

Guidelines. Fam Med. 1998;30(2):117-121.

26. 27Hayward RSA WM, Tunis SR, Bass EB, Guyatt G. Users' guide to the medical literature:

VII. How to use clinical practice guidelines. Are the recommendations valid? JAMA

1995;274(7):570-574.

27. McAlister FA vDS, Padwal RS, Johnson JA, Majumdar SR. How evidence-based are the

recommendations in evidence-based guidelines? PLoS Medicine. 2007;4(8):e250.

28. Grimshaw JM, Russell IT. Effect of clinical guidelines on medical practice: a systematic

review of rigorous evaluations. Lancet. 11/27 1993;342(8883):1317-1322.

29. Institute of Medicine (IOM). Clinical Practice Guidelines We can Trust. Washington, DC:

The National Academies Press. Brief report available at:

http://www.iom.edu/Reports/2011/Clinical-Practice-Guidelines-We-Can-Trust.aspx.

Accessed on September 7, 2014.

30. Chou R. Using evidence in pain practice. Part I: Assessing quality of systematic reviews and

clinical practice guidelines. Pain Medicine. 2008;9(5):518-530.

31. Watine J FB, Nagy E, Onody R, Oosterhuis W, Bunting PS, Charet JC, Horvath AR.

Conflict between guideline methodologic quality and recommendation validity: A potential

problem for practitioners. Clinical Chemistry. 2006;52(1):65-72.

32. Oosterhuis WP BD, Watine J, Sandberg S, Horvath AR. Evidence-based guidelines in

laboratory medicine: Principles and methods. Clinical Chemistry. 2004;50(5):806-818.

33. Verkerk K vVH, Severens JL, Hendriks EJM, Burgers JS. Considered judgment in evidence-

based guideline development. International Journal for Quality in Health Care 2006;18(5):

365-369.

ACCEPTED MANUSCRIPT

34. Bousquet J, Schunemann HJ, Zuberbier T, et al. Development and implementation of

guidelines in allergic rhinitis - an ARIA-GA2LEN paper. Allergy. 2010;65(10):1212-1221.

35. Schunemann HJ, Fretheim A, Oxman A. Improving the use of research evidence in

guideline development: 10. Integrating values and consumer involvement. Health Research

Policy and Systems. 2006;4(22).

36. Fervers B, Carretier J, Bataillard A. Clinical practice guidelines. Journal of Visceral

Surgery. 2010;147(6):e341-e349.

37. Keeley PW. Clinical guidelines. Palliative Medicine. 2003;17(4):368-374.

38. Schunemann HJ, Oxman A, Brozek J, et al. Grading quality of evidence and strength of

recommendations for diagnostic tests and strategies. BMJ. 2008;336(7653):1106-1110.

39. Redelmeier DA, Ferris LE, Tu JV, Hux JE, Schull MJ. Problems for clinical judgement:

introducing cognitive psychology as one more basic science. Canadian Medical Association

Journal. 2001;164(3):358-360.

40. Dahm P YL, Gallucci M, Simone G, Schunemann HJ. How to use a clinical practice

guideline. The Journal of Urology. 2009;181:472-479.

41. Scott SD, Plotnikoff RC, Karunamuni N, Bize R, Rodgers W. Factors influencing the

adoption of an innovation: an examination of the uptake of the Canadian Heart Health Kit

(HHK). Implementation Science. 2008;3(41).

42. Tornatzky LG, Klein KJ. Innovation Characteristics and Innovation Adoption-

Implementation: A Meta-Analysis of Findings. IEEE Transactions on Engineering

Management. 1982;EM-29(1):28-43.

43. Thompson RL, Higgins CA, Howell JM. Personal Computing: Toward a Conceptual Model

of Utilization. MIS Quarterly. 1991;15(1):125-143.

44. Ball K. Surgical smoke evacuation guidelines: compliance among perioperative nurses.

AORN Journal. 2010;92(2):e1-e23.

45. Cabana MD, Flores G. The role of clinical practice guidelines in enhancing quality and

reducing racial/ethnic disparities in paediatrics. Paediatric Respiratory Review.

2002;3(1):52-58.

46. Rashidian A EM, Russell I. Falling on stony ground? A qualitative study of impelemtation

of clinical guidlines' prescribing recommendations in primary care. Health Policy.

2008;85:148-161.

47. Gurses AP, Marsteller JA, Ozok AA, Xiao Y, Owens S, Pronovost PJ. Using an

interdisciplinary approach to identify factors that affect clinicians' complicance with

evidence-based guidelines. Critical Care Medicine. 2010;38(8 Suppl):S282-S291.

48. Shaneyfelt TM, Mayo-Smith MF, Rothwangl J. Are guidelines following guidelines? The

methodological quality of clinical practice guidelines in the peer-reviewed medical

literature. JAMA, the journal of the American Medical Association. 1999;281(20):1900-

49. Carlsen B GC, Pope C. Thou shalt versus thou shalt not: A meta-synthesis of GPs' attitudes

to clinical practice guidelines. British Journal of General Practice. 2007;57:971-978.

50. Tong A. Clinical guidelines: can they be effective? Nursing Times. 2001;97(9):III-IV.

51. Grimshaw JM, Russell IT. Effect of clinical guidelines on medical practice: a systematic

review of rigorous evaluations. Obstetrical & Gynecological Survey. 1994;49(7):469.

52. Stone TT, Schweikhart SB, Mantese A, Sonnad SS. Guideline attribute and implementation

preferences among physicians in multiple health systems. Quality Management in Health

Care. 2005;14(3):177-187.

ACCEPTED MANUSCRIPT

53. Harris JS. Development, use, and evaluation of clinical practice guidelines. Journal of

Occupational and Environmental Medicine. 1997;39(1):23-34.

ACCEPTED MANUSCRIPT

Table 1: Process that we used to categorize attributes using the Message domain as an example

Goal Steps Example

Organize, group,

appropriately

label similar or

“like” attributes

1. Group attributes that are antonyms • Complex/Simple

2. Group attributes that are synonyms • Unclear/Confusing

3. Group attributes with the same root • Specific/Specificity

• Validity/Valid

4. Sort database by attribute

Categorize

attributes into

logical domains

5. Are there commonalities among

attributes?

6. Is there a central theme or focus

among groups of attributes?

The following attributes can be grouped

into a category called “Clarity”

• Unambiguous

• Precise

• Specific

Go through each

domain to

determine sense

and fit of

attributes

7. Do the attributes belong within the

same cluster?

8. Can they be collapsed?

9. Use attribute definitions to make

these decisions

The following categories can be

collapsed:

• ”Complexity” with “Information

overload”

• “Actionability” with “Wording”

Develop a

definition for

domains

10. Based on their included attributes

and definitions, define and label the

cluster

The MESSAGE domain can be defined

as: The clarity, simplicity and

persuasiveness of the guideline

language

ACCEPTED MANUSCRIPT

Table 2: Final list of attribute categories across 6 domains of guideline implementability

Category (N = 16) Major attributes Domain (N = 6)

Credibility of guideline

development group

Credibility Stakeholder

involvement

Disclosure of conflict

of interest

Conflict of interest; Transparency; Funding

sources; Editorial independence

Reporting of what is

needed

Scope; Patient preferences; Cost and resource

requirements; Outcomes data; Harms and

benefits

Evidence

synthesis

Execution of what is

needed

Evidence-based; Valid and reliable; Transparent

Updating of guidelines Updating; Currency

Clinical applicability Clinical relevance; Appropriateness of patient

population; Considered implementation

Considered

judgment

Values Guideline developer values;

Professional/provider values (clinical judgment,

clinical freedom); Patient values (acceptability,

patient preferences);

Local applicability Adaptation; Application tools Implementation

feasibility Resource constraints Availability of resources; Economic outcomes

Novelty Compatibility; Requires new knowledge and

skills

Simple Information overload; Complexity Message

Clear Actionability (specificity, ambiguity); Effective

writing

Persuasive Framing; Relative advantage

Multiple versions End-users; Versions (flat, dynamic); Document

Format

Components Components to include in guidelines (e.g.,

purpose, target audience, methods)

Presentation Layout of full document (placement, length);

Structure within sections (match the system to

the real world, sequential bundling); Information

visualization (information display [e.g.,

algorithms, pictures], information context [e.g.,

framing, vividness])

ACCEPTED MANUSCRIPT

Table 3: Summary of hierarchical narratives to describe guideline implementability

GUIDELINE IMPLEMENTABILITY: To facilitate uptake, the process of guideline development has two broad aims: the creation of content and the communication of that content. The four domains of content creation are stakeholder involvement, evidence synthesis, considered judgment in formulating recommendations, and feasibility. The two domains of content communication relate to fine-tuning the message itself and its format.

Creation of Content: The four interrelated domains of content creation are stakeholder involvement (including credibility of the developers and disclosure of conflicts of interest), evidence synthesis (specifying what evidence is needed and how and when it is synthesized), considered judgment (including clinical applicability and values), and feasibility (local applicability, resource constraints, and novelty). These domains may be considered non-sequentially and iteratively.

Communication of content: Communication of guidelines entails fine-tuning both the message of the recommendations (through use of simple, clear, and persuasive language) and their format (through representation in multiple versions, inclusion of specific components, and effective layout and structure).

STAKEHOLDER INVOLVEMENT

EVIDENCE SYNTHESIS

CONSIDERED JUDGMENT IMPLEMENTATION FEASIBILITY

MESSAGE FORMAT

The guideline development group should have appropriate composition; its members should have relevant, unbiased expertise and suitable credibility, and potential conflicts of interest should be disclosed. The target population of end users (patients, the public) should be clearly defined, and their views and preferences considered. Ensuring stakeholder involvement during guideline development facilitates uptake.

To enhance guideline validity and reproducibility, the necessary evidence must be specified, the method of synthesis clearly defined (ideally evidence-based, valid, reliable, and transparent), and the timing of sequential syntheses appropriate (balancing timeliness of the guideline with stability over time).

The guideline development group must supplement evidentiary factors (quality, quantity, and consistency) with considered judgment, making complex trade-offs between the competing benefits and harms, side effects, and risks of various options for managing the disease or condition. They must also consider clinical applicability (whether the guideline responds to variability among patients) and the values and preferences of patients, developers, and care providers (i.e., the relative worth or importance of a health state and consequences such as benefits, harms, and costs of a decision).

Feasibility reflects local applicability (i.e., strategies for adapting recommendations to local conditions), consideration of resource constraints (availability of resources and other economic implications), and the influence of novelty of (or familiarity with) the guidelines, where novelty refers to the degree to which the recommendations propose behaviours considered unconventional by clinicians or patients). Feasible implementation of guidelines allow for flexibility in individual clinical decisions, are in agreement with users’ opinions and skills, and are suitable for routine use in intended settings.

Guideline messages should use simple, clear, and persuasive language. Simplicity can be achieved by limiting the number of elements, the number of steps within each recommendation, or the number of conditional factors influencing performance (to prevent the quantity of information from exceeding available cognitive capacity). Clarity is enhanced by using specific, unambiguous language and by applying a direct writing style, with active voice, suitable punctuation, short sentences, and bullet lists to convey series of points, and without awkward breaks, abbreviations, hyphenation, redundancy, or unnecessary jargon. Finally, the guideline messages should be clinically convincing and should be framed in terms of potential gains, to convey their relative advantage over any previous approach.

Guidelines may be formatted in multiple versions (e.g., research-based, information-gathering, analytical tool; briefer guide for clinical education; short version for point-of-care clinical use; lay-language version for patients). Formatting involves determining which components to include in various versions (e.g., scope and purpose, target audience, guideline development panel, update plan, and implementation considerations), their presentation (e.g., proper placement of visual elements and document length) and structure (i.e., matching the system to the real world and bundling); and the overall visualization of information.

ACCEPTED MANUSCRIPT

Table 4: Facilitators and barriers for domains of guideline implementability*

Domain and examples of facilitators and barriers Actions for guideline developers

STAKEHOLDER INVOLVEMENT

Facilitators:

• wide range of stakeholders

• disclosure of conflicts of interest and funding sources

Barriers

• industry contributions

• recommendations based on expert opinion alone

• select unbiased participants with relevant

expertise

• declare competing interests and funding

sources

• document views and preferences of target

population

EVIDENCE SYNTHESIS

Facilitators: • consistent reporting of elements (e.g., phases of illness, resource requirements,

outcomes data, harms and benefits)

• identification of recommendations based on expert judgment or consensus

• maintenance of currency in relation to new evidence

Barrier: excessive frequency of revision

• report type and quality of evidence

• report methods of synthesis

• set timelines for guideline review

CONSIDERED JUDGMENT

Facilitators:

• perceived clinical relevance and appropriateness for patient population

• indicators of relative strength of recommendations

• definition of values that influenced recommendations

Barriers:

• lack of fit between clinicians’ experiences and recommendations

• lack of applicability to all or “typical” patients.

• use of low-quality, weak, or conflicting evidence

• state population(s) to which statements

• disclose appropriateness or applicability of

• clearly state value judgments

• document process for managing dissent

IMPLEMENTATION FEASIBILITY

Facilitators:

• limit on recommendations that require large investment of time or resources

• availability of data regarding resource requirements and cost-effectiveness

• minimization of change required for users and systems

Barriers:

• requirement for new knowledge or skills

• unconventional changes

• inconsistency of changes with existing values, needs, and experiences of adopters

• formulate recommendations in terms of

measurable criteria and targets for quality

improvement

• identify costs and resource requirements

• specify competencies, training, and

technical specifications required

• include economic data

MESSAGE

Facilitators

• actionability

• crisp and persuasive messages

• effective use of language

Barriers

• underlying evidence that is contradictory or complex

• use of evidence that is still evolving or evidence not commonly observed in practice

• ambiguity and vagueness

• poor framing of the guideline

• use conditional statements

• use specific, concrete statements

• justify any deliberate vagueness

• use short sentences with proper punctuation

• keep related information together

• frame recommendations in terms of gains

• focus on errors of omission (not doing the

right thing) rather than errors of commission

(doing the wrong thing)

FORMAT

Facilitators:

• use of multiple formats or alternate versions

• inclusion of components known to be important to implementation

• appropriate placement of graphics and text

• appropriate structure (high-level categorization of recommendations)

• information visualization to shift cognitive load to perceptual system through

graphics and animation (e.g., matching system to the real world, bundling of related

recommendations)

• use of words for procedural information, logical conditions, and abstract concepts

• use of images for spatial structures, location, and detail

• tailor guidelines to intended end users

• highlight key evidence-based features

having the most significance for patient care

• highlight key recommendations with links to

more extensive explications

• choose the most appropriate graphic

according to the type of information to be

conveyed

• use colour appropriately

*Please see Web Appendix for more detailed list of facilitators, barriers and suggested actions for guideline developers with

references.

ACCEPTED MANUSCRIPT

Figure 1: Flow of Realist Review Methods

Extraction of data 1736 guideline attributes were extracted in

duplicate from 278 included articles

Organization of data 2 sets of investigators (Group 1; Group 2)

independently clustered 1736 attributes into logical categories

Realist Review Methods

Auditing process

• Primary reviewers’ data extractions audited by second reviewer

• Disagreements resolved through consensus

Category agreement

• Compared “agreed” and “divergent” classifications

Data Extraction

Group 1 = 33 categories

Group 1 = 28 categories

Derived a common set of categories through group

discussion (N = 27) across 5 domains

LEVEL 2: Expert validation of Codebook and domain categorization

Collapsed into 16 categories across 6 domains

Validation with 9 experts

• Identify flaws in categorization

• Sense and fit of categories, attributes and their labels and definitions

Data Organization

LEVEL 1: Codebook of definitions Determine attributes of guideline uptake, their

operationalization, and relationship with uptake

Data Analysis

LEVEL 3: Hierarchical narratives Determined what works for whom, under what

circumstances and why

Search strategy An iterative, multiple search strategy that

consisted of 5 non-linear stages of searching

Article selection Two sets of reviewers (6 in total) independently

screened articles using inclusion criteria

Search strategy

Article selection

Stage 1: Core articles

Stage 2: Expert identified

Stage 3: PubMed Related articles

Stage 5: Other

Stage 4: Bibliography

Abstract level: N = 2044

Full text level: N = 350

ACCEPTED MANUSCRIPT

Figure 2: Search strategy

ACCEPTED MANUSCRIPT

Figure 3: Flow of study selection

Guideline uptake is influenced by 6 implementability domains for creating and communicating...

Documents

Transcript of Guideline uptake is influenced by 6 implementability domains for creating and communicating...

Cspcons: A Communicating Sequential Prolog with Constraints

Approaches to Turkish Foreign Policy: A Critical Realist Analysis

The Neoidealist Moment in International Studies? Realist ...

A Realist Explanation of Long Run Development Interventions ...

Physician Frustration in Communicating with Patients

Communicating Gender in Context

a realist review (Master's Thesis in Accounting) - Jultika

Organizing and Communicating - e-PG Pathshala

How to be critical and realist about economics

MICROBIOLOGICALLY INFLUENCED CORROSION OF MILD STEEL IN

Lecture: Realist Theories of International Relations, 2014-15

The 'Relational Subject' According to a Critical Realist ...

Realist(ic) Evaluation Tools for OST Programs - QTurn

Communicating the Resilience and Corporate Social ... - MDPI

communicating-in-business.pdf - WordPress.com

The Role of Narrative in Communicating Science

Communicating the Corporate Brand Internally - DIVA

A realist view on the treatment of identical particles

Delegitimizing Al-Qaeda: A Jihad-Realist Approach

Communicating and migratable interactive multimedia documents