5

Pract. Dev. Health Care 6(1) 5–24, 2007Copyright © 2007 John Wiley & Sons, Ltd DOI: 10.1002/pdh

Research and review

A realist synthesis of evidence relating to practice development: Methodology and methods

Brendan McCormack—Professor of Nursing Research, Institute of Nursing Research, University of Ulster and Director of Nursing Research & Practice Development, Royal Hospitals Trust, Belfast, UK; Adjunct Professor of Nurs-ing, Monash University, Melbourne, and Visiting Fellow, Faculty of Nursing, University of Sydney, Australia

Jayne Wright—Research Associate, Institute of Nursing Research, University of Ulster, UK

Belinda Dewar—Independent Research ConsultantGill Harvey—Senior Lecturer in Healthcare and Public Sector Management,

Manchester Business School, University of Manchester, UKKay Ballantine—Sub-Librarian, Faculty of Life and Health Sciences, University

of Ulster, UK

Background

Practice development (PD) is a term that has been used to describe particular ap-proaches to supporting changes in health care (predominantly nursing) for over 20 years. A number of studies have been published that have sought to clarify what is meant by the term and establish the relationships between its component parts (Garbett and McCormack, 2002; Manley and McCormack, 2003; McCormack et al., 1999; Unsworth, 2000). However, there remains a need to provide a full and critical in-depth review of the wide range of published and unpublished literature in this area. Those working within PD draw on a range of theoretical and practical infl uences from a range of disciplines (management, education, organizational development and so on).

Practice Development in Health CarePract. Dev. Health Care 6(1) 5–24, 2007Published online in Wiley InterScience(www.interscience.wiley.com) DOI: 10.1002/pdh.210

McCormack et al.6

Pract. Dev. Health Care 6(1) 5–24, 2007Copyright © 2007 John Wiley & Sons, Ltd DOI: 10.1002/pdh

There is also therefore a need to understand the relationships between perspectives, approaches and methods used under the heading of PD, and those employed in other forms of development activity within health services, both nationally and internationally.

In February 2005, NHS Education for Scotland (NES) and NHS Quality Im-provement Scotland (NHS QIS) invited tenders for a literature review examining the evidence around PD. The study commissioners noted the increasing amount of litera-ture in the area, characterized by the emergence of a range of theoretical frameworks and conceptual analyses under what had initially been a relatively ill-defi ned umbrella term for a range of change management and knowledge use strategies (Garbett and McCormack, 2002; Manley and McCormack, 2003; McCormack et al., 1999; Unsworth, 2000).

This is the fi rst in a series of four papers which present each part of the study. The focus of the fi rst paper is the methodology and methods utilized. The second paper focuses on the fi ndings from the literature review. The third paper presents the out-comes from telephone interviews with key experts in the fi eld of PD and a synthesis of the fi ndings. Finally, the fourth paper presents the recommendations arising from the study.

Purpose

The purpose of this review was to identify approaches to PD and examine critically the evidence base that supports them, drawing on both empirical data and expert opinion. The expected outcomes were:

• Comparative analyses of approaches to PD

• Indicators to guide the choice of approaches that match particular circumstances

• Indicators that may lead to the development of a Scotland-wide model of PD for health care organizations

The study was designed in two phases:Phase 1 focused on reviewing the published PD literature using the review methods of realist synthesis, as outlined by Pawson and colleagues (2004). The phase was opera-tionalized through three stages of work, from explicitly agreeing the focus of the study, the identifi cation of contributory theories to PD that would shape the review questions (the review was structured around 13 areas of theory, refl ecting four broad theoretical perspectives of PD), the development of a set of bespoke review forms, searching for literature to answer these questions and, fi nally, conducting the systematic review of the literature. A total of 169 papers were selected for review following various stages of refi nement of the search strategy. Of these papers, 71 explicitly used PD as a study methodology or studied the experience of involvement in PD; 30 were scholarly reviews of PD literature; six were concept analyses; 29 papers were studies in which PD was

Methodology and methods 7

Pract. Dev. Health Care 6(1) 5–24, 2007Copyright © 2007 John Wiley & Sons, Ltd DOI: 10.1002/pdh

implicit to the work and 33 were empirical research studies that related to PD but did not focus on PD processes or outcomes.

Phase 2 was informed by the outcomes of Phase 1, and was undertaken in two stages: stage 1 included the review of the grey PD literature using the same review processes as Phase 1, and stage 2 involved conducting telephone interviews with key informants internationally. A total of 41 items of grey literature* reviewed, along with four books that were widely referenced in the literature because they add to an understanding of PD methodology. A total of 47 interviews were undertaken with key informants representing strategic, organizational, unit and academic roles in the UK, Republic of Ireland, Sweden, The Netherlands, New Zealand, Australia, Canada and the USA. The interview schedule was derived from the initial themes derived from the literature review.

Methodology

[Practice development] comprising theories, involves the actions of people, consists of a chain of steps or processes that interact and are rarely linear, is embedded in social (healthcare) systems, is prone to modifi cation and it usually exists in open systems that change through learning.

The study methodology is based on a recently developed framework derived from realistic evaluation (Pawson and Tilley, 1997). Faced with the challenge of evaluating ‘messy’ data collected in unpredictable practice contexts, evaluation researchers in health care have increasingly begun to turn to realistic evaluation as a means of linking together mechanisms and their outcomes in relation to the contexts in which they oc-cur (Redfern et al., 2003; Greenhalgh et al., 2004; Tolson, 1999; Wilson and McCormack, 2006).

A systematic review of evidence is a process of secondary research that identifi es studies relevant to a particular topic, appraises the quality of these studies according to predetermined criteria and synthesizes their results using a scientifi c methodology. Conventional systematic reviews tend to impose a strict hierarchy of evidence focused on questions of effectiveness, and address very narrowly focused questions that rarely refl ect the complexity of the context in which interventions are operationalized. There-fore, reviews may not exist or be possible for more complex service delivery or policy issues. As a consequence, the review fi ndings may have limited clinical applicability.

*Grey literature, also called ‘Non-conventional literature’, comprises scientifi c and technical re-ports, patent documents, conference papers, internal reports, government documents, newsletters, factsheets and theses, which are not readily available through commercial channels (adapted from a defi nition from http://en.wikipedia.org/wiki/Grey-literature) [last accessed 09/02/07].

McCormack et al.8

Pract. Dev. Health Care 6(1) 5–24, 2007Copyright © 2007 John Wiley & Sons, Ltd DOI: 10.1002/pdh

The positivist underpinnings of systematic reviews suggest that generalizable fi ndings can only be derived from evidence that has a strong relationship between cause and effect – that is, that the same effect is produced with regularity. Regularity is therefore only likely to occur under specifi c circumstances – that is, in a closed system (Wilson and McCormack, 2006). Such closed systems fail to recognize the complexity of the natural world, a world in which the same mechanism can produce different outcomes dependent on the context.

The key question that researchers working within a realistic perspective ask is, how do certain causal mechanisms (e.g. a workshop), operating in particular circum-stances (e.g. a particular PD project), create certain changes (outcomes)? This empirical characteristic of realistic research means that researchers are primarily concerned with analysing the causal mechanisms within the environment, any interactions that occur between mechanisms, and the power of mechanisms to generate outcomes (Outhwaite, 1987). They observe that, when it comes to the delivery of complex programmes and services, the ‘same’ intervention never gets implemented in an identical manner. Even if it did, the recipe for success in one setting might not be transferable to a different context. Thus, realists attempt to understand complex social interactions/interventions. According to Pawson et al. (2004) and Sridharan et al. (2006), complex social interven-tions comprise theories, involve the actions of people, consist of a chain of steps or processes that interact and are rarely linear, are embedded in social systems, are prone to modifi cation and exist in open systems that change through learning.

Pawson and colleagues (Pawson et al., 2004; Pawson, 2006; Pawson and Boaz, 2004) have developed a model for synthesizing plural forms of evidence that are gener-ated through the complex interactions between processes (mechanisms). The model, called ‘realist synthesis’, draws on the methodological logic of realistic evaluation and applies it to the synthesis of plural forms of evidence. A realist synthesis follows similar stages to more familiar forms of systematic review (Table 1) with some particular features that distinguish it. Notably, a realist synthesis derives its focus from a negotiation be-tween the study commissioners and the researchers. Realist synthesis has been dev eloped in response to the limitations of systematic reviews outlined earlier. A realist review carries exactly the same objective, the refi nement of theories, rather than an attempt to make claims about the absolute preferability of one alternative over another.

This approach is relevant to a systematic review of evidence pertaining to PD, as the latter can be seen to embrace the features of complex social interventions – it comprises theories, involves the actions of people, consists of a chain of steps or pro-cesses that interact and are rarely linear, is embedded in social (health care) systems, is prone to modifi cation and it usually exists in open systems that change through learning.

Finding and articulating the programme theories

Pawson et al. (2004) set out the following steps for a review using this realist synthesis approach.

Methodology and methods 9

Pract. Dev. Health Care 6(1) 5–24, 2007Copyright © 2007 John Wiley & Sons, Ltd DOI: 10.1002/pdh

Ph

ase

1

Table 1. The process of realist synthesis (adapted from Pawson et al., 2004)

‘Traditional’ Stages in a Activities undertaken at each stage

stages in realist synthesis

systematic study

review

Defi ne the scope 1.1 Identify the For example:

of the review question • What is the nature and content

of the intervention?

• What are the circumstances or

context for its use?

• What are the policy intentions or

objectives?

• What are the nature and form of

its outcomes or impacts?

• Undertake exploratory searches

to inform discussion with review

commissioners/decision makers

1.2 Clarify the For example:

purpose(s) of • Theory integrity – does the

the review intervention work as predicted?

• Theory adjudication – which

theories about the intervention

seem to fi t best?

• Comparison – how does the

intervention work in different

settings, for different groups?

• Reality testing – how does the

policy intent of the intervention

translate into practice?

1.3 Find and • Search for relevant ‘programme

articulate the theories’

programme • Draw up ‘long list’ of

theories programme theories

• Group, categorize or synthesize

theories

• Design an evaluative framework

to be ‘populated’ with evidence

McCormack et al.10

Pract. Dev. Health Care 6(1) 5–24, 2007Copyright © 2007 John Wiley & Sons, Ltd DOI: 10.1002/pdh

Table 1. Continued

‘Traditional’ Stages in a Activities undertaken at each stage

stages in realist synthesis

systematic study

review

Search for and 2.1 Search for • Decide and defi ne purposive

appraise the the evidence sampling strategy

evidence • Defi ne search sources, terms

and methods to be used

(including cited reference

searching)

2.2 Appraise the • Test relevance – does the

evidence research address the theory

under test?

• Test rigour – does the research

support the conclusions drawn

from it by the researchers or the

reviewers?

Extract and 3.1 Extract the • Develop data extraction forms

synthesize results or templates

fi ndings • Extract data to populate the

evaluative framework with

evidence

3.2 Synthesize • Compare and contrast fi ndings

fi ndings from different studies

• Use fi ndings from studies to

address purpose(s) of review

• Seek both confi rmatory and

contradictory fi ndings

• Refi ne programme theories in

the light of evidence

3.3 Consultation • Identify expert cohort from

on and literature (citation and volume

refi nement of of publications) and relevant

fi ndings networks

• Telephone interviews

* NB THIS STAGE HAS BEEN

DESIGNED SPECIFICALLY FOR THIS

STUDY

Ph

ase 2

Ph

ase

1

Methodology and methods 11

Pract. Dev. Health Care 6(1) 5–24, 2007Copyright © 2007 John Wiley & Sons, Ltd DOI: 10.1002/pdh

Table 1. Continued

‘Traditional’ Stages in a Activities undertaken at each stage

stages in realist synthesis

systematic study

review

4. Draw conclusions and make • Involve commissioners/decision

recommendations makers in review of fi ndings

• Draft and test out

recommendations and

conclusions based on fi ndings

with key stakeholders

• Disseminate review with

fi ndings, conclusions and

recommendations

Ph

ase 2

• Concept mining: extraction of a theory from the existing literature

• Theory formalization: codifi cation of the theory into a set of explanatory propositions (or model)

• Evidence synthesis: revision and development of that model to explain the complex pattern of success and failure found in the empirical evidence.

Phase 1 of the study addressed the fi rst two of these steps (concept mining and theory formalization). Phase 2 of the study continued the process of theory formalization through the analysis of grey literature and interviews with ‘experts’, as well as engaging in ‘evidence synthesis’. Developing the focus of the study and the theories to be exam-ined is an important aspect of a realist synthesis study (Pawson et al., 2004), as it provides the structure for examining a diverse body of information.

For this study, the structure was developed from:

• The steering group’s ideas and questions

• The project team’s constructions

• Conceptualizations of PD found in the literature (Garbett and McCormack, 2002; Manley and McCormack, 2003; McCormack et al., 1999; Rycroft-Malone et al., 2002; Unsworth, 2000).

The challenge of developing the framework for a realist synthesis is to fi nd the level of abstraction that allows the reviewers both to stand back from the mass of detail and variation in the data set, and also meet the purpose of the review as required by the commissioner. To this end, the framework is structured around the following four ideas (Figure 1):

McCormack et al.12

Pract. Dev. Health Care 6(1) 5–24, 2007Copyright © 2007 John Wiley & Sons, Ltd DOI: 10.1002/pdh

• Who? – looking at dimensions around the people who are the focus of PD at the level of the individual, team, organization (e.g. the people who become developed) and so on – so it’s ‘who’ in a specifi c sense (e.g. practitioners, users) as well as in a collective sense (e.g. teams). This idea includes the concepts of leadership, culture and context.

• By whom? – looking at the dimensions involved in the people doing the devel-oping; for example, looking at their orientation to the people that they are working with (insider/outsider etc.)

• What/why? – looking at the impulse to engage in PD work, working with ideas around inductive and deductive rationales and the role of policy initiatives

• How? – looking at the actual mechanisms involved, facilitation styles, theoreti-cal orientations, knowledge utilization, involvement of users.

These ideas need to be related to the outcomes that result (Figure 1), where the analysis in each quadrant looks to understand relevant outcomes in relation to the programme theories being scrutinized – for example, in the ‘How’ quadrant, action learning as part of an emancipatory strategy is meant to produce outcomes x and y, and whether or not these outcomes were produced and under what circumstances. Tables 2, 3, 4 and 5 illustrate the themes and elements dimensions of; who, by whom, what/why and how.

An initial explanatory model

The next stage in realist synthesis is putting the conceptual fragments together into a theoretical model. Pawson et al. (2004) suggest that it is the model that does the work of explanatory synthesis, as it identifi es the primary foci of inquiry from the variety of studies that make up the evidence base. In addition, the outcomes derived from the use of the model pull together and shape the advice offered to the review commissioners.

Figure 1. Model for conceptualizing practice development programme.

Methodology and methods 13

Pract. Dev. Health Care 6(1) 5–24, 2007Copyright © 2007 John Wiley & Sons, Ltd DOI: 10.1002/pdh

Table 2. The dimensions of study of the ‘who’ of practice development

Theme Elements

1.1 Level of involvement • Practitioners only, unidisciplinary

• Practitioners only, multidisciplinary

• Multiple levels of involvement (e.g. managers,

external agencies, service users/patients)

1.2 Scale of activity • Small scale (e.g. single clinical unit)

• Medium scale (e.g. service)

• Large scale (e.g. organizational)

1.3 Context in which practice • Weak

development takes place • Strong

1.4 Culture in which practice • Weak

development takes place • Strong

1.5 Leadership in environment in which • Weak

practice development takes place • Strong

Table 3. The dimensions of study of the ‘by whom’ of practice development

2.1 Location of practice developer • Clinically based

• Trust based

• Externally based – consultancy

• Externally based – higher education

2.2 Access by practice developer • Invited (entry at initiation of individuals/teams)

• Negotiated (entry in cooperation with

individuals/teams)

• Imposed (entry without engagement/willingness

of individuals/teams)

2.3 Theoretical orientation of the practice • ‘Working on’ – working with

developer(s) deductively derived ideas, practitioners seen as

recipients, practice developer as ‘expert’

• ‘Working with’ – uses inductive approaches to

facilitation, practitioners seen as collaborators

McCormack et al.14

Pract. Dev. Health Care 6(1) 5–24, 2007Copyright © 2007 John Wiley & Sons, Ltd DOI: 10.1002/pdh

Table 4. The dimensions of study of the ‘what/why’ of practice development

3.1 Initiation of practice development activity • Locally (e.g. practice setting) initiated –

practice-based issue(s)

• Locally initiated – response to strategic/policy

issues

• Organizationally initiated – response to

organizational issues (e.g. quality of care)

• Organisationally initiated – response to

strategic/policy issues

3.2 Focus of activity • Practice improvement

+ Learning for practitioners (e.g. new knowledge)

+ Change in attitudes and ways of perceiving

practice (e.g. empowerment)

Table 5. The dimensions of study of the ‘how’ of practice development

4.1 How learning happens • Training model – information shared through traditional

educational approaches focusing on issues specifi c to the project

being undertaken

• Refl ective model – learning needs and knowledge generated

through refl ective approaches

4.2 How change happens • Technical – agenda controlled by those who hold power,

underpinned by one-way transfer of knowledge, focus on

practice competence, low focus on understanding of culture and

context, restricted consultation of stakeholders including service

users

• Emancipatory – high focus on raising awareness of impact of

culture and context on practice, collaborative approaches to

agenda for and direction of change, emphasis on wide

stakeholder collaboration including service users

4.3 How knowledge is used • Technical – deductive approach to

and generated knowledge use – low emphasis on developing new knowledge

and theory, evaluation based on technical interests (e.g.

numerical evidence) assuming correlation between independent

and dependent variables regardless of context

• Emancipatory – use of both deductive and inductive knowledge –

emphasis on developing new knowledge and theory from wide

range of stakeholders including service users, evaluation based

on practical interests, incorporating a consideration of the

impact of context and culture

Methodology and methods 15

Pract. Dev. Health Care 6(1) 5–24, 2007Copyright © 2007 John Wiley & Sons, Ltd DOI: 10.1002/pdh

The formulation of the ‘quadrant model’ provides the basis of this, inasmuch as it postulates four components that interact to produce PD outcomes. This model can be expanded (Figure 2) to incorporate the arguments above as issues to be explored empirically.

The model suggests four theoretical areas with 13 areas of focus. Within each of these areas, the model suggests themes for exploration as follows:

Theory area 1 – Properties of the people and context in practice development

a. What is the impact of the extent of involvement of different stakeholders on the outcomes of PD?

b. What is the impact of the scale of a study on the outcomes of PD?c. How do contextual factors in the study setting have an impact on the outcomes

of PD?d. How do cultural factors in the study setting have an impact on the outcomes

of PD?e. How do styles of leadership in the study setting have an impact on the outcomes

of PD?

WHO? BY

WHO

M?

WH

AT/

WH

Y?HO

W?

Level of involvement

Context

Culture

Scale of activity

Lead

ership

Lo

cation

Access

Theoretical orientation

Starting pointFocusof activity

Change

Knowledge

Learn

ing

Figure 2. Explanatory model for practice development.

McCormack et al.16

Pract. Dev. Health Care 6(1) 5–24, 2007Copyright © 2007 John Wiley & Sons, Ltd DOI: 10.1002/pdh

Theory area 2 – Properties of the people involved in developing practice

f. How does the location of a practice developer have an impact on the outcomes of PD?

g. How do the means by which the practice developer gains access to the practice environment have an impact on the outcomes of PD?

h. How do the methodological positions taken by practice developers have an impact on the outcomes of PD?

Theory area 3 – Issues surrounding the initiation and carrying out of practice development

i How do factors involved in the initiation of PD have an impact on its outcomes?

j. What are the foci of PD activity and how do they have an impact on its outcomes?

Theory area 4 – Approaches for the use of knowledge, bringing about change and supporting learning in practice development

k. How do approaches taken to support learning within PD have an impact on outcomes?

l. How do approaches taken to bringing about change within PD have an impact on outcomes?

m. What forms of knowledge use and knowledge generation are used in PD and what are the consequences for the outcomes?

Finding and appraising the published evidence

Identifying the review questions

[. . .] identifying practice development literature is problematic because it is not a thesaurus search term in any of the established databases.

The fi rst stages of a realist synthesis study consist of a clarifi cation of the intended ‘product’; what is it that the study commissioners want to know, and to what ends? An exercise at the initial meeting of the steering group helped the group to develop indi-vidual and collective questions about PD. These have, in turn, informed the identifi ca-tion of relevant programme theories for scrutiny.

Methodology and methods 17

Pract. Dev. Health Care 6(1) 5–24, 2007Copyright © 2007 John Wiley & Sons, Ltd DOI: 10.1002/pdh

Searching the literature

As has been reported elsewhere (Garbett and McCormack, 2002), identifying PD lit-erature is problematic because it is not a thesaurus search term in any of the established databases. We therefore needed to develop a theoretically consistent approach to searching the databases. The approach needed also to refl ect the concerns of the research methodology being used – realist synthesis (Pawson et al., 2004), which argues that the literature needs to be scrutinized to identify ‘programme theories’. We identifi ed two possible options:

Search using practice and development as key words – We have experienced the consequences of this in previous studies (Garbett and McCormack, 2002), where data-bases search the words ‘practice’ and ‘development’ separately, as well as in conjunction. As a result, we then have to scrutinize a large number of papers and exclude a large number.

Map commonly used subject headings onto concepts emerging from commonly used defi nitions of practice development – We envisaged this as consisting of matching con-ceptual ideas from defi nitions of PD with subject headings; for example, matching the concept of ‘improvement’ with the subject headings ‘*Quality Improvement’, ‘*Quality Assurance/mt [Methods]’. To this end, we conducted a preliminary exercise, working with a dedicated librarian.

Looking at the second approach as an alternative seemed unlikely to be more effective than the previous approach that we have used before. By taking sources that advance a theoretical framework for PD (Garbett and McCormack, 2002; Kitson and Currie, 1996; Mallett et al., 1997; Manley and McCormack, 2003; Unsworth, 2000), we were able to account for each paper selected, in terms of the features of PD that it portrayed (Table 6).

The librarian colleague used the above concepts as a checklist to guide the selection of papers found using the terms ‘practice’ and ‘development’ across a range of databases. Using the concepts helped us to increase the rigour and specifi city of the selection of papers for in-depth study.

The possible drawback of this approach is that it assumed the quality of the previous studies and ran the risk of ‘closing down’ the possible range of programme theories that would be identifi ed. Nonetheless, it is argued that we needed to look for instances of the term ‘practice development’ being used as a descriptor for specifi c ways of working, as a precursor to looking in greater depth at the programme theories that are described in PD literature. The search included the databases listed in Table 7.

Merging these fi ndings into one database provided a list of 376 references, to which were added 14 papers that we were aware of but that were not found in the searches (for example, Manley, 2000a, b; Bellman, 2003), producing a total of 390 pa-pers. However, examination of the list showed that entries in different databases may use different conventions, resulting in duplication. In this case, 38 papers were entered twice and a further 183 papers were excluded because they were descriptive papers

McCormack et al.18

Pract. Dev. Health Care 6(1) 5–24, 2007Copyright © 2007 John Wiley & Sons, Ltd DOI: 10.1002/pdh

Table 6. Concepts in defi nitions of practice development

Concept Source(s)

Process of improvement (including a focus Garbett and McCormack, 2002; Mallett et al., 1997;

on quality) Manley and McCormack, 2003; Unsworth, 2000

Patient-centred care Garbett and McCormack, 2002; Mallett et al., 1997;

Manley and McCormack, 2003; Unsworth, 2000

Facilitation Garbett and McCormack, 2002; Manley and

McCormack, 2003

Professional development Garbett and McCormack, 2002; Mallett et al., 1997;

Manley and McCormack, 2003

Transformation Garbett and McCormack, 2002; Manley and

McCormack, 2003

Systematic and rigorous Garbett and McCormack, 2002; Kitson and Currie, 1996;

Manley and McCormack, 2003

Emancipation Garbett and McCormack, 2002; Manley and

McCormack, 2003

Service users’ perspectives Garbett and McCormack, 2002; Unsworth, 2000

Effectiveness (including implementation of Garbett and McCormack, 2002; Kitson and Currie, 1996;

plural forms of evidence) Mallett et al., 1997; Unsworth, 2000

Managed change Garbett and McCormack, 2002; Kitson and Currie, 1996;

Unsworth, 2000

Acting on the culture and context of care Garbett and McCormack, 2002; Manley and

McCormack, 2003

(editorials or news stories), or did not meet criteria as PD papers. The remaining 169 papers formed the basis of the fi rst phase review. The classifi cation of these papers is shown in Table 8.

Finding and appraising the grey literature

In order to access the grey literature, an e-mail was sent to members of networks that the researchers are associated with (Table 9). Thirty-seven items of grey literature were received, and in addition four books were reviewed that are widely referenced in the

Methodology and methods 19

Pract. Dev. Health Care 6(1) 5–24, 2007Copyright © 2007 John Wiley & Sons, Ltd DOI: 10.1002/pdh

Table 7. Databases searched

Database searched (papers fi ltered using the table above) References identifi ed

British Nursing Index 63

Cinahl 203

First Search 52

Medline 33

National Electronic Library for Health (NeLH) 0

Psycinfo 10

Social Science Citation Index 11

AMED 0

HMIC 0

Bandolier (www.jr2.ox.ac.uk/bandolier/) 4

Table 8. Classifi cation of papers found

Category Number of papers

1. Explicitly use practice development as a study methodology or study the 71

experience of involvement in practice development

2. Scholarly reviews of practice development literature 30

3. Concept analyses 6

4. Studies in which practice development approaches are implicit (for example 29

using facilitative approaches to change)

5. Papers based on empirical research, but not containing evidence about practice 33

development processes or outcomes

literature because they add to an understanding of PD methodology (Bellman, 2003; Bryar and Griffi ths, 2003; McCormack et al., 2004; Page et al., 1998), resulting in a review of 41 items of grey literature and books.

The literature was read by members of staff of the Nursing Development Team of the Royal Hospitals, Belfast, UK, and the lead researcher. Each staff member read at least two items of the grey literature and completed a data extraction sheet for each item. Reviewers raised questions and issues on the data extraction sheets, and these were clarifi ed and responded to by the lead researcher.

Telephone interviews

This interview stage took the form of a round of telephone interviews with key informants in the fi eld of PD. Informants were selected via the networks in which the researchers are involved and from prominent authors identifi ed in the earlier stages from the UK, the Netherlands, Australia, New Zealand and Canada.

McCormack et al.20

Pract. Dev. Health Care 6(1) 5–24, 2007Copyright © 2007 John Wiley & Sons, Ltd DOI: 10.1002/pdh

A snowball sampling technique was used (Burns and Grove, 1997). This sam-pling approach works on the principle of ‘social networks’. When a researcher has identifi ed a few participants who meet the inclusion criteria, their assistance is sought in getting in touch with others who have similar characteristics. An e-mail was sent to a list of 65 people derived from the identifi ed networks, asking for their participation in the interview and the identifi cation of further participants who:

• Had engaged in PD work as participants or as project facilitators/coordinators

• Were key stakeholders in commissioning/funding/managing practice develop-ment work

• Had a strategic role in developing practice

• Had successfully completed PD activities

A total of 72 potential interviewees were identifi ed. Based on the timeframe available for interviews and discussion with the project steering group, it was agreed that a sample of 40–50 interviews would be acceptable in order to get a wide range of views on the initial themes emerging from Phase 1 of the study.

Table 9. International networks

• International Practice Development Collaboration: Members in the UK, Australia and the

Netherlands

• The Foundation of Nursing Studies

• Developing Practice Network: Members throughout the UK

• The Professional and Practice Development Nurses’ Forum – Scotland

• Knowledge Utilisation Collaboration: Members in the UK, Australia, New Zealand, Canada, USA,

Holland and Sweden

• International Residential Practice Development School Collaboration: Members in the UK,

Australia, the Netherlands and New Zealand. A network of 30 facilitators and more than 500

participants is available to the research team Academic Partnerships: Monash University,

Melbourne, Australia; James Cook University, Townsville, Australia; Victoria University, Wellington,

New Zealand; Auckland University, New Zealand; University of Melbourne, Australia; Southern

Health Network, Victoria Australia; The Alfred Hospital, Melbourne, Australia; The Children’s

Hospital at Westmead, NSW, Australia; Northern Sydney Central Coast Health, NSW, Australia;

University of Northumbria, Newcastle, UK; Bournemouth University, UK.

• The Virtual Institute [VI]: A virtual institute of research and practice development. Lead

organisation is the Royal College of Nursing, UK.

• The Virtual Practice Development College, Glasgow Caledonian University, UK

Methodology and methods 21

Pract. Dev. Health Care 6(1) 5–24, 2007Copyright © 2007 John Wiley & Sons, Ltd DOI: 10.1002/pdh

Sampling procedure

The population (n = 72) were grouped according to the role that the person had in PD into one of four groups – organizational, strategic, academic or unit, with the breakdown shown in Table 10.

From this population, interviewees were selected according to the following criteria:

• 50% of people from groups with >10 potential participants

• 100% of people from groups with <10 potential participants

• Geographical spread

• Spread across the identifi ed networks (Table 9).

Using these criteria, a sample was selected, with the breakdown shown in Table 11.

Interview management process

Each selected interviewee was e-mailed, and details of the project provided. The sample was divided among the three interviewers, so that each interviewer had a sample from each role group. The interview schedule was e-mailed to interviewees in advance of the interview, so that they had some degree of familiarity with the process and had some time to think about their answers. Interviews were tape recorded using a ‘Phonopart Telephone Conversation Recorder [model number TL1076]’.

Table 10. Practice development roles of population

Role Number

Organizational 45

Strategic 9

Unit 4

Academic 14

Total 72

Table 11. Sample of participants selected for interview

Role Number

Organizational 25

Strategic 9

Unit 4

Academic 9

Total 47

McCormack et al.22

Pract. Dev. Health Care 6(1) 5–24, 2007Copyright © 2007 John Wiley & Sons, Ltd DOI: 10.1002/pdh

Interview schedule

An interview schedule was devised following consultation with the steering group. The schedule was designed to refl ect themes from the Phase 1 report. In addition, following discussion of the Phase 1 report with the steering group, other issues were raised that were considered to be of importance but were not explicit in the literature. A fi nal in-terview schedule was developed that refl ected these combined perspectives.

Data extraction, analysis and synthesis

Data analysis followed an eight-step process (Figure 3). The data from the individual data extraction sheets for the published literature were extracted and copied onto ‘the-ory synthesis forms’ for each of the four theory areas. The data consisted of direct quotes, researchers’ commentaries and impressions. These data were grouped according to the particular emphasis in the data and the researchers’ impressions of the specifi c meanings in the data. Each theory synthesis form was read and re-read in order to gain overall impressions of the data, and rough notes were made. The data from each theory synthesis form were then themed. In some cases, the original papers were revisited in order to clarify meanings and fi nalize themes. This stage of the work completed stage 3.2 of the realistic synthesis process and Phase 1 of the overall study. A draft report was formulated and presented to the project steering group for discussion, clarifi cation and challenge. Some ‘gaps’ in the literature were identifi ed (for example, the lack of

1. Extraction of data from individual data extraction sheets and inputting to theory synthesis forms

2. Identifying themes from data in each theory area

3. Draft report to the project steering group for discussion, clarification, challengeand identification of gaps

4. Input grey literature data from individual data extraction sheets to theory synthesis forms

5. Identifying/ clarifying themes from data in each theory area

6. Writing narrative (findings)

7. Consideration of themes from telephoneinterviews against initial literature themes

8. Final themes and sub-themes – data synthesis

Figure 3. Data extraction, analysis and synthesis.

Methodology and methods 23

Pract. Dev. Health Care 6(1) 5–24, 2007Copyright © 2007 John Wiley & Sons, Ltd DOI: 10.1002/pdh

accounts regarding the funding of PD projects), and these were highlighted as important considerations in the review of the grey literature and in telephone interviews.

The grey literature data were then fed into the ‘theory synthesis forms’ for each of the theory areas in order to form a complete data set. Few themes changed signifi -cantly, but instead the grey literature either strengthened or weakened initial themes. The themes were constructed into a narrative, and these narratives formed the struc-ture of a report.

The fi nal stage of analysis consisted of an analysis of the telephone interviews. The interview data were themed under the questions on the interview schedule. Key quotes and comments were highlighted and these were fed into the discussion of the data, in order to highlight particular issues, confi rm themes from the literature verifi ed or contradict the strength of claims made in the literature analysis, and identify novel issues and themes. This fi nal stage resulted in the identifi cation of four overarching themes and nine sub-themes that synthesized all of the data (discussed in Paper 3).

Conclusion

This fi rst paper has presented the background to the study and the methodology uti-lized. The paper has illustrated the value of realistic synthesis as a method for studying complex interventions in response to the perceived limitations of traditional systematic review methodology, which, it is argued, follows a highly specifi ed and intentionally infl exible methodology, with the aim of assuring high reliability. A realist review, by contrast, follows a more heterogeneous and iterative process, which is less amenable to prescription but which needs to be equally rigorous. Thus, the audit trail of the review shows how decisions were made, evidence sought, sifted and assessed, and fi ndings ac-cumulated and synthesized.

ReferencesBellman L (2003). Nurse-led Change and Development in Clinical Practice. London: Whurr Publishers

Ltd.Bryar RM, Griffi ths JM (2003). Practice Development in Community Nursing: Principles and Processes.

London: Arnold.Burns N, Grove SK (1997). The Practice of Nursing Research: Conduct, Critique and Utilization. London:

WB Saunders Company.Garbett R, McCormack B (2002). A concept analysis of practice development. NT Research 7(2):

87–100.Greenhalgh T, Robert G, Bate P, Kyriakidou O, MacFarlane F, Peacock R (2004). How to Spread Good

Ideas: A Systematic Review of the Literature on Diffusion, Dissemination and Sustainability of Innovation in Health Service Delivery and Organisation. Report for the National Co-ordinating Centre for NHS Service Delivery and Organisation R&D (NCCSDO), University College London.

Kitson A, Currie L (1996). Clinical practice development and research activities in four district health authorities. Journal of Clinical Nursing 5(1): 41–51.

Mallett J, Cathmoir D, Hughes P, Whitby E (1997). Forging new roles . . . professional and practice development. Nursing Times 93(18): 38–9.

McCormack et al.24

Pract. Dev. Health Care 6(1) 5–24, 2007Copyright © 2007 John Wiley & Sons, Ltd DOI: 10.1002/pdh

Manley K (2000a). Organisational culture and consultant nurse outcomes: Part 1 – Organisational culture (including commentary by Scholes J). First published in Nursing Standard 14: 34–8. Nursing in Critical Care 5(4): 179–86.

Manley K (2000b). Organisational culture and consultant nurse outcomes: Part 2 – Consultant nurse outcomes (including commentary by Scholes J). Nursing in Critical Care 5(5): 240–8.

Manley K, McCormack B (2003). Practice development: Purpose, methodology, facilitation and evaluation. Nursing in Critical Care 8(1): 22–9.

McCormack B, Manley K, Garbett R (2004). Practice Development in Nursing. Oxford: Blackwell.McCormack B, Manley K, Kitson A, Titchen A, Harvey G (1999). Towards practice development – A

vision in reality or a reality without vision? Journal of Nursing Management 7(5): 255–64.Outhwaite W (1987). New Philosophies of Social Science: Realism, Hermeneutics and Critical Theory.

Basingstoke: Macmillan.Page S, Allopp D, Casley S (1998). The Practice Development Unit; An experiment in multidisciplinary

innovation. Whurr Publishers Ltd, London.Pawson R (2006). Evidence-Based Policy: A Realist Perspective. London: Sage.Pawson R, Boaz A (2004). Methods Briefi ng 3: Evidence-based Policy, Theory-based Synthesis, Practice-based

Reviews. Centre for Census and Survey Research, University of Manchester, Manchester.Pawson R, Greenhalgh T, Harvey G, Walshe K (2004). Realist Synthesis: An Introduction. RMP Methods

Paper 2/2004. Centre for Census and Survey Research, University of Manchester, Manchester.Pawson R, Tilley N (1997). Realistic Evaluation. London: Sage Publication.Redfern S, Christian S, Norman I (2003). Evaluating change in health care practice: Lessons from three

studies. Journal of Evaluation in Clinical Practice 9(2): 239–49.Rycroft-Malone J, Kitson A, Harvey G et al. (2002): Ingredients for change: Revisiting a conceptual frame-

work. Quality and Safety in Health Care 11(2): 174–80.Sridharan S, Platt S, Hume M (2006). How to Evaluate Complex Public Health Interventions – Evaluation

Methods. Keynote address, Royal College of Physicians of Edinburgh, February 2006, Edinburgh.Tolson D (1999). Practice innovation: A methodological maze. Journal of Advanced Nursing 30(2):

381–90.Unsworth J (2000). Practice development: A concept analysis. Journal of Nursing Management 8(6):

317–26.Wilson V, McCormack B (2006). Critical realism as emancipatory action: The case for realistic evaluation

in practice development. Nursing Philosophy 7(1): 45–57.

Address correspondence to Professor Brendan McCormack, Nursing & Midwifery Development Centre, 3rd Floor, Bostock House, Royal Hospitals Trust, Grosvenor Road, Belfast, BT4 3BX, UK. Tel: 02890 635332. E-mail: brendan.mccormack@royalhospitals.n-i.nhs.uk