ORIGINAL ARTICLE

Exploring Knowledge Exchange at the Research–Policy–PracticeInterface in Children’s Behavioral Health Services

Laurel K. Leslie • Susan Maciolek •

Kathleen Biebel • Gifty Debordes-Jackson •

Joanne Nicholson

� Springer Science+Business Media New York 2014

Abstract This case study explored core components of

knowledge exchange among researchers, policymakers,

and practitioners within the context of the Rosie D. versus

Romney class action lawsuit in Massachusetts and the

development and implementation of its remedial plan. We

identified three distinct, sequential knowledge exchange

episodes with different purposes, stakeholders, and

knowledge exchanged, as decision-making moved from

Federal Medicaid policy to state Medicaid program stan-

dards and to community-level practice. The knowledge

exchanged included research regarding Wraparound, a key

component of the remedial plan, as well as contextual

information critical for implementation (e.g., Federal

Medicaid policy, managed care requirements, community

organizations’ characteristics).

Keywords Research use � Policy � Intermediary �Knowledge exchange �Wraparound � Children’s behavioral

health

Introduction

The last decade has witnessed the publication of major

reports identifying significant gaps in mental health ser-

vices provided in the US (Committee on Quality of

Healthcare in America 2001; Institute of Medicine (US)

2006; New Freedom Commission on Mental Health 2003).

Despite an expanding body of promising or evidence-based

mental health practices, uptake into usual care settings

remains limited (Weisz and Addis 2006). Federal agencies,

health insurance companies, and accreditation bodies have

increasingly called for the adoption of evidence-informed

practices in mental health (Hogan 2003).

Calls to bridge the gap between ‘‘what is known’’ and

‘‘what is done’’ have led to a small but growing body of

literature on how researchers interface with policymakers

and practitioners. Early models of this interface focused on

the autonomous production and passive transfer of research

by researchers to practitioners and policymakers, describ-

ing linear, rational processes whereby research was con-

ducted by a research ‘‘producer,’’ usually located within an

academic center, and then ‘‘pushed’’ to a ready and waiting

research ‘‘user’’ in the policy or practice arena (Lavis et al.

2003). Challenges faced both by researchers attempting to

communicate their findings to policymakers and practitio-

ners, and by policymakers and practitioners attempting to

access research, prompted the recognition and promotion

of intermediaries who act as translators of research findings

to potential users (Lavis et al. 2003).

More recently, knowledge translation or knowledge

exchange frameworks have been promoted as a potentially

useful for examining the interactions among researchers,

policymakers, and practitioners. There is considerable

variability in the terminology used to describe these

frameworks, both within health-related research (e.g.,

L. K. Leslie (&)

Tufts Medical Center/ Floating Hospital for Children,

800 Washington Street, #345, Boston, MA 02111, USA

e-mail: lleslie@tuftsmedicalcenter.org

S. Maciolek

Policy and Management Consultant, Boston, MA, USA

K. Biebel � G. Debordes-Jackson

University of Massachusetts Medical School, Worcester, MA,

USA

J. Nicholson

Geisel School of Medicine at Dartmouth, Hanover, NH, USA

123

Adm Policy Ment Health

DOI 10.1007/s10488-014-0535-7

knowledge translation, linkage and exchange, knowledge

to action) and in other disciplines (e.g., technology transfer,

knowledge management, change management). Despite

this lack of consistent terminology, the World Health

Organization has adopted knowledge translation as a

guiding principle and defined it as ‘‘the synthesis,

exchange, and application of knowledge by relevant

stakeholders to accelerate the benefits of global and local

innovation in strengthening health systems and improving

people’s health’’ (World Health Organization 2006).

While multiple knowledge translation or exchange

(hereafter, ‘‘exchange’’) frameworks have been proposed

across publications (Estabrooks et al. 2008; Hogan 2003;

Mitton et al. 2007; Pentland et al. 2011), several themes are

common. First, these frameworks posit that researchers,

practitioners, and policymakers represent different stake-

holder groups with distinct cultures and priorities, all of

whom have knowledge and expertise that is needed by the

other. This addresses what Tseng describes as the risk of

‘‘privileging researchers’ perspectives and relegating

practice professionals to the receiving end of research and

dissemination efforts’’ (Tseng 2012) by highlighting the

multi-directional exchange of knowledge. Second, explicit

among these frameworks is the concept that the knowledge

being exchanged does not consist only of the available

research on an evidence-based practice but also includes

other forms of knowing, such as experiential knowledge

about the context within which a policy or practice might

be embedded (Graham et al. 2006). Third, principles from

participatory research processes are incorporated, high-

lighting the role of relationships and trust among

researchers, policymakers, and practitioners (Gagnon 2011;

Ko et al. 2005). Last, knowledge translation and exchange

frameworks acknowledge the complexities of the socio-

political context in which practice and policy decision-

making occurs (Gibbons 2008).

However, research regarding knowledge exchange

remains limited. Mitton and colleagues’ review of this

literature found that only 20 % of studies reported any real-

world application and characterized the majority of papers

as ‘‘opinion pieces, reviews, or surveys of stakeholders’’

(Mitton et al. 2007). In addition, most papers have origi-

nated in Canada or Europe, with only a limited number

reflecting decision-making in public systems in other

countries with different political structures and processes

for providing mental health care, such as the US.

The study described below addresses these limitations by

exploring the application of knowledge exchange within the

context of a major public mental health policy initiative in

the US. Reforms in the Commonwealth of Massachusetts’

children’s behavioral health system offer a unique and

timely opportunity to examine the application of knowledge

translation and exchange to children’s mental health policy,

programs, and practice. Specifically, we employed a case

study design to ask the following questions: (1) what

knowledge was exchanged, in what setting(s), and for what

purpose? (2) What role did researchers, policymakers, and

practitioners play in the knowledge exchange? (3) What

policy, program, and practice decisions and subsequent

actions resulted from the exchange? (4) What can this case

study add to the range of knowledge exchange frameworks

examining the research–policy–practice interface? For

these analyses, we defined knowledge as inclusive of pub-

lished research evidence as well as other types of knowl-

edge reflecting the context of the policy initiative and the

experiences of stakeholders (Asen et al. 2011).

The Context: The Massachusetts Children’s Behavioral

Health System

In 2001, a class action lawsuit, Rosie D. v. Romney, was

brought by plaintiffs to the Federal Court system under the

early periodic screening diagnosis and treatment (EPSDT)

provisions of the Medicaid Act. The federal EPSDT pro-

visions entitle Medicaid-eligible individuals under the age

of 21 to screening, diagnosis, and treatment that is medi-

cally necessary to correct or ameliorate physical and

mental health conditions. Plaintiffs sought to compel

Massachusetts to provide access to and coordination of

home-based behavioral health services to enable children

with severe emotional disturbance identified under EPSDT

to receive treatment and support in their homes and home

communities. In 2006, the Federal Court found Massa-

chusetts in violation of EPSDT, a remedial plan was

established, and an independent Court Monitor was

appointed to monitor the state’s implementation of the

plan.

A key provision of the Rosie D. remedial plan was

intensive care coordination using Wraparound, a team-

based planning process to provide family-driven, intensive

care coordination for children involved with public child-

and family-serving systems (e.g., mental health, child

welfare, juvenile justice, special education) (Burchard et al.

2002; Rossman 2002). While there is some debate as to

whether the available evidence supports Wraparound’s

designation as ‘‘promising’’ or ‘‘evidence-based,’’ (Suter

and Bruns 2009) its documented success in increasing

community-based care in lieu of institutional care has led

to its implementation both within the US as well as in other

countries, its mandated use in several Federally supported

programs, and its inclusion in the Surgeon General’s

reports on Children’s Mental Health and on Youth Vio-

lence (Center for Mental Health Services (US), National

Center for Injury Prevention and Control, National Institute

of Mental Health (US), and Office of the Surgeon General

(US) 2001).

Adm Policy Ment Health

123

Wraparound is a care planning practice in which formal

services and informal supports are coordinated across

systems. It is not a direct service treatment intervention

and, thus, differs from the single treatment designs that are

often designated as evidence-based practices. Wraparound

builds on system of care principles and prioritizes family

voice and choice in care planning and delivery. Manualized

education and training resources are available from the

National Wraparound Initiative (http://nwi.pdx.edu/) as

well as from national purveyors who provide training and

technical assistance to organizations seeking to implement

Wraparound.

As part of the implementation of the Rosie D. remedy

plan, 24 community-based organizations in Massachusetts

were selected in 2009 through a competitive procurement

process to provide Wraparound. The 24 organizations

provide services through 32 Community Service Agencies

(CSAs) (reflecting 29 geographical service areas and 3

special populations). CSA contracts were established and

administered through the Medicaid Program’s five man-

aged care entities (MCEs). The state also selected, through

a competitive procurement, a national Wraparound pur-

veyor to provide training and coaching to CSA staff. The

MCEs established technical assistance teams to provide

consultation to the CSAs and the Wraparound purveyor on

the implementation of Wraparound in the context of the

state’s Medicaid managed behavioral health care structures

and processes.

The planning for and implementation of the Rosie D.

remedial plan permitted an in-depth explanatory case study

to inform theory regarding knowledge exchange at the

researcher-policymaker-practitioner interface. Conduct of

our study occurred concurrent with community imple-

mentation, after the Rosie D. trial ended and the remedial

plan was developed. Our primary interest was in knowl-

edge exchange at the community level. However, under-

standing the socio-political context established by the court

ruling, the remedial plan, and the Medicaid standards was

essential for framing decisions made at the community

level. Consequently, study methodology required relying

on retrospective reports; some potential key informants

also were not available. Nevertheless, we gained important

insights about the nature and extent of knowledge

exchange across decision-making episodes across multiple

levels.

Methods

Because the theoretical understanding of knowledge

exchange is at a formative stage, we employed qualitative

methods (Denzin and Lincoln 2005). Qualitative methods

are particularly appropriate when the questions at hand

relate to dynamic processes, and are useful in laying the

groundwork for further theory development (Weiss 1998).

We employed a nested case study design, appropriate to

theory generation (Eisenhardt 1989) and useful in under-

standing state health reforms (Brownson et al. 2009).

The study was conducted across three linked and over-

lapping Phases. Phase 1 included the collection and ana-

lysis of public documents regarding the Rosie D. court

decision and the development and implementation of the

remedial plan. Phase 2 consisted of one-on-one and group

interviews with state-level key informants engaged in

policy and program decisions related to the Rosie D. court

case and the remedial plan, including state agency repre-

sentatives and consultants, the MCE technical assistance

teams, and the Wraparound purveyor. Phase 3 consisted of

one-on-one and group interviews and document analysis

during site visits with four provider agencies serving as

CSAs.

The document review and interview protocols were guided

by concepts drawn from the use of research evidence literature

as a priori specification of constructs (Eisenhardt 1989).

Specifically, we drew on Gibbons’ knowledge exchange

framework (Gibbons 2008) and Tseng and Seidman’s social

settings framework to guide purposeful sampling decisions

(i.e., the selection of state-level key informants and CSA sites)

(Tseng and Seidman 2007). Data were obtained from multiple

sources and stakeholders with divergent points of view and

experiences with the systems change initiative to allow for the

triangulation of data (Brownson et al. 2009; Creswell and

Plano Clark 2007). Study procedures were approved by

Institutional Review Boards at all three academic institutions

affiliated with the study.

Phase 1. Setting the Context: Document Procurement

and Review

Documents procured were publicly available, accessible

through state agency or other websites, and included Court

documents, progress reports prepared by the Massachusetts

Executive Office of Health and Human Services (EOHHS),

Medicaid program and procurement documents, reports

prepared by the Court Monitor, and presentations to

stakeholders. In addition, state-level informants, inter-

viewed in Phase 2, nominated documents. At least two

research team members initially reviewed the 87 docu-

ments for: (1) Specific descriptions that permitted delin-

eation of the timeline of the Rosie D. lawsuit and the

development and implementation of the proposed remedial

plan; and (2) any text regarding knowledge that might have

been exchanged among stakeholders as well as any

guidelines or parameters put into place that informed pol-

icy and practice decisions. Eighteen documents were

selected for final review by the entire research team.

Adm Policy Ment Health

123

Phase 2. State-Level Policy and Program: Key

Informant Interviews

A purposeful sampling strategy was used to recruit study

participants representing the plaintiffs’ attorneys, the

defendants for the state, court-oversight officials, state

agency administrators, parent advocates, Medicaid MCE

technical assistance teams, and the national Wraparound

purveyor, as well as other individuals whom key infor-

mants recommended as offering important perspectives to

the study.

Potential participants were sent an email informing them

of the study and inviting their participation. All invitees

agreed to participate. Two study investigators conducted

each of the face-to-face interviews between December

2010 and February 2012 in a location convenient to par-

ticipants; the interviews lasted 1–2 h. If several individuals

from an organization were identified and requested to be

interviewed together, these individuals were interviewed

using standardized group interview facilitation techniques

(Frey and Fontana 1991; Kitzinger 1995).

Prior to the interviews, informed consent was obtained

and participants completed a brief background survey (e.g.,

training and background, years of experience with the

agency or organization, years of experience in various

professional roles, participation in statewide professional

activities, and role in the development and implementation

of the Rosie D. remedial plan). Investigators employed a

semi-structured interview protocol informed by the

knowledge translation and exchange literature, knowledge

gained from the document review, and investigators’ pre-

vious experiences as public agency staff or clinicians or in

mid-career fellowship placements with public agencies or

research settings. The protocol also contained a printed

timeline, extending from 2006 to 2012; informants were

asked to frame their responses within the context of the

timeline. All individual interviews (n = 8) and group

interviews (2 groups; n = 13) were digitally recorded,

transcribed by a professional transcriptionist, and de-

identified to maintain study participants’ privacy. Investi-

gator pairs alternated interviews, allowing for the objective

perspectives of research team members not assigned to

particular interviews (Eisenhardt 1989). Investigators met

for peer debriefing following interviews to review their

experiences and impressions, and explore any lack of

clarity in meaning.

Phase 3. Practice Considerations: Community Service

Agency (CSA) Key Informant Interviews

A purposeful sampling strategy was used to recruit four

provider agencies serving as CSAs for inclusion in the

study. Investigators emailed a description of the study, an

invitation to participate, and a link to an on-line survey to

the Executive Directors of the 21 community-based orga-

nizations providing CSA services in 29 service areas across

Massachusetts. (The decision was made not to include the 3

organizations providing services to specialty populations

(e.g., deaf and hearing impaired), given that their purview

was statewide rather than geographic-specific). The pro-

vider agency survey included questions about their site

locations, size, organization of staff and resources, service

array, clients served, and budget.

Twelve agency surveys were returned to the study team

reflecting 17 CSAs. Visual inspection of survey data (e.g.,

geographic location; urban, suburban or rural; previous

experience with care coordination; and range of experience

in the children’s mental health experience) regarding

responding agencies as compared with non-respondents

suggested the respondents provided an adequate represen-

tation of the larger pool of CSA agencies. A quantitative

ranking system was established to characterize agencies

across three domains hypothesized by Tseng and Seidman

(2007) to be relevant to processes in agency settings: (1)

social organization (i.e., the complexity of overall agency

organizational structure and financing); (2) physical orga-

nization (i.e., the location of service delivery site(s) for the

remedial plan services); and (3) economic/temporal orga-

nization (i.e., infrastructure support for training, quality

improvement, and/or research) (Tseng and Seidman 2007).

Four provider agencies were identified for the study, rep-

resenting a range of complexity in each domain. All invited

sites agreed to participate.

Study team members mailed agency Executive Directors

or their assistants, if preferred, to schedule site visits with

administrators, managers, and direct service staff nomi-

nated by Executive Directors as key players in imple-

menting the Rosie D. remedial plan at the agency level.

Prior to site visits, research staff compiled agency profiles

based on responses from the online survey and the CSAs’

original application to the state, their agency’s organiza-

tional chart, their most recent annual report, and other

relevant program materials. Site visits were scheduled over

2–3 days and took place between April 2011 and May

2012, at times and in locations convenient for agency staff.

CSA profiles were reviewed and confirmed by CSA par-

ticipants during site visit. A semi-structured interview

protocol was developed to explore the agency’s role in

Rosie D. activities and services using the same printed

timeline as in Phase 2. Other questions addressed the

decision to serve as a CSA, barriers and facilitators to the

incorporation of Wraparound into the agency’s organiza-

tion and activities, and the exchange of information (e.g.,

research, contextual factors) during implementation. All

individual interviews (n = 14) and group interviews (9

groups; n = 38) were digitally recorded, professionally

Adm Policy Ment Health

123

transcribed, and de-identified to maintain study partici-

pants’ confidentiality.

Analysis

All responses from individual surveys were manually

entered into database software (Microsoft Office Excel

2007) and imported into a statistical software package

(IBM SPSS 20.0) for the calculation of all descriptive data.

Quantitative data obtained in provider agency surveys were

analyzed using the software component of the online sur-

vey application to summarize descriptive information.

All transcriptions of qualitative interview data were

checked against digital audio files for accuracy. Investigators

used a summative analysis approach to the qualitative data as

outlined by Rapport (2010). Members of the research team

began reviewing narrative data as it became available, as

earlier phases of the study informed later phases in response to

knowledge gained (Weiss 1998). The team read and re-read

the raw material, generating memos regarding impressions,

emerging concepts and relationships, and reviewing findings

that elaborated or corroborated a priori themes of research use

previously defined in the literature (Miles and Huberman

1994; Palinkas et al. 2009). Visual displays and tables were

developed to manage and present data, including timelines,

flow charts, and matrices of themes and relationships (Ei-

senhardt 1989). All study documents and transcripts were

uploaded into Dedoose, a web-based software program that

facilitates mixed methods data management (Dedoose 2012),

for further coding, elaboration, and specification of knowl-

edge exchange concepts and relationships. Agency case study

data were prepared following a template to facilitate the

integration of data across agencies (Lincoln and Guba 1985).

Results

Participants

Twenty-one state-level key informants (i.e., state agency

administrators, legal staff, court, the Wraparound purveyor,

MCEs) and 55 community-level key informants (i.e., CSA

administrators and staff) were interviewed (see Table 1).

The majority were white, female, and had a college degree or

higher. CSA participants were younger, reported less prior

experience with research, and were relatively new additions

to their current agency compared to state-level participants.

Primary Themes

Several primary themes were identified regarding knowl-

edge exchange during the development and implementa-

tion of the Rosie D. remedial plan. First, three distinct

episodes of activities were identified that occurred at dif-

ferent levels of policy and practice and with different

purposes: (1) the development of the Rosie D. remedial

plan through court processes at the Federal policy level;

Table 1 Study participants

Characteristics* State-level policy and

program participants

N = 21 (%)

CSA

participants

N = 55 (%)

Gender

Female 81 85

Male 19 15

Race/ethnicity

White or Caucasian 100 92

Native Hawaiian or other

Pacific Islander

– 4

African-American – –

Hispanic/Latina(o) – 11

Puerto Rican – 9

Other – 2

Chose not to answer – 4

Age

22–34 5 31

35–44 38 31

45–54 14 25

55–64 43 11

65? – 2

Education

High school graduate/GED – 9

Partial college credit – 5

Associate’s degree – 7

Bachelor’s degree 10 15

Master’s degree 67 60

Professional degree 5 –

Doctorate degree 19 4

Prior experience with research

Yes 62 40

No 33 55

NA 5 5

Type of agency affiliated with

State agency 76 –

Provider agency – 100

NA/other 24 –

Number of years with agency

1–3 years 33 65

4–7 years 24 20

8–11 years 5 7

12 or more years 29 7

NA 10 –

* Race/ethnicity does not sum to 100 % as Hispanic/Latino was asked

as a separate question; other characteristics approximate to 100 % due

to rounding conventions

Adm Policy Ment Health

123

(2) the establishment of Medicaid program standards for

remedy services and a CSA request for proposals (RFP) at the

state program level; and (3) the CSAs’ implementation of

Wraparound at the community practice level. The knowledge

exchanged thus moved sequentially from the Federal level to

the state level and then to the local community level. (see

Fig. 1). The first episode may be better characterized as

knowledge transfer, bounded as it was by the Court’s rules of

engagement, with attorneys presenting witnesses and evi-

dence to the judge for his consideration. The second and third

episodes exemplified knowledge exchange as an interactive

process. Second, while some participants transitioned from

one episode to another, episodes were notable for markedly

different participants as the level (Federal, state, community)

and purpose (policy, program, practice) of the knowledge

exchange shifted. In addition, at the same time that knowledge

exchange narrowed in scope from federal to local, the numbers

and diversity of participants increased. Third, each episode

resulted in a particular decision made by different decision-

makers that informed the next actions to be taken. Different

knowledge was exchanged to inform these decisions, with

research playing different roles in each episode. Documents

from one episode served to inform the actions taken in the

subsequent episode (see Table 2). Below we describe each

episode in more detail, highlighting these three themes.

Episodes

Episode 1: Federal Court Policy Decisions Regarding

Need and Scope of Remedial Plan

The first episode consisted of a series of formal exchanges

typical of a court proceeding, with the Judge serving as the

arbiter of the evidence presented by witnesses identified by

attorneys for the plaintiffs and the state. The first phase of

Episode 1 occurred during a 6-week trial in May and June

2005, during which the parties presented 45 witnesses and

introduced over 500 exhibits (Center for Public Repre-

sentation 2006). The Judge heard testimony from a range of

researchers, clinicians, program administrators, family

members, and fiscal and other experts. This quote from one

attorney describes the types of experts who testified and the

knowledge they presented to the Judge:

One expert team was essentially the [pause] the story

of Wraparound, the story of home-based services, so

that was a very, very heavily data-based presenta-

tion… they’re the ones who are gonna describe the

model and also demonstrate the effectiveness of the

model compared to what existed. A second group of

experts who we called the ‘programmatic experts’

who were people who came and observed, they

analyzed information about Massachusetts mental

health services for kids… and then spoke about the

extent to which Massachusetts does or doesn’t offer

Wraparound services. And then we had a third group

of experts who were fiscal, financial, right, and

Medicaid data experts… And they were there to

speak to, what is Massachusetts spending, and what

does this alternative cost, and make a number of

projections about that. And then we had a fourth

group of experts who were, who we called ‘client

experts’.

With respect to the types of knowledge presented during

this initial episode, the range varied substantially. Federal

reports and research studies provided some context for the

Scope of Decision

Parameters

Episode #1: Policy Episode #3: PracticeEpisode #2: Program

Federal Medicaid EPSDT Law and Regulations

State Medicaid Program Standards

Wraparound Research Evidence

DecisionMakers &

Intermediaries(in italics)

Judge

Community Service AgenciesManage Care Entities Wraparound Purveyor

Dates 2005-2006 2007-2008 2009+

Plaintiff’s Attorneys

Court Monitor

State’s Attorneys State Administrators

Medicaid Program

Consultant

Fig. 1 Research use over time

in the Massachusetts children’s

behavioral health initiative

Adm Policy Ment Health

123

scope of the problem regarding rates of children’s behav-

ioral health needs and possible intervention models.

Research studies cited during the trial included papers

published in the professional literature and evaluation

results from demonstration projects funded by the Sub-

stance Abuse Mental Health Services Administration

(SAMHSA). Additional program evaluation data were also

shared, including reports of programs implemented in other

states (Center for Public Representation 2006). Families of

plaintiff children shared narratives of their experiences

regarding receiving services in Massachusetts.

As a result of this non-jury trial, the Federal Court ruled

that the Massachusetts Medicaid program was in violation

of Federal Medicaid EPSDT requirements. Subsequent to

this ruling, the Judge ordered a remedial plan that included

intensive care coordination, using the Wraparound model,

as a key component. The following quote from the Judge’s

Memorandum of Decision on January 26, 2006 reflects the

impact of the knowledge exchanged during the Court

proceedings on the Judge’s formulation of his decision:

The evidence showed, time and again, that the

Commonwealth’s efforts to comply with these mini-

mum EPSDT assessment and service coordination

requirements were woefully inadequate, with detri-

mental consequences for thousands of vulnerable

children … Defendants’ [the state’s] failure to pro-

vide adequate assessments, service coordination, and

home-based supportive services for Medicaid-eligible

children with serious emotional disturbances was

glaring from the evidence and at times shocking in its

consequences (Rosie D. v. Romney 2006).

The Judge ordered the plaintiffs and the state to confer

and negotiate a joint remedial plan. As the plaintiffs

developed their proposed remedial plan, they ‘‘incorpo-

rated lots of specifics about what the system would look

like, including infrastructure, not just services, but also

incorporating oversight, data collection, documentation,

monitoring of all that information and services.’’ Central to

their plan development was identifying ‘‘programs that

work, or… programs that have a track record.’’ For

example, the plaintiffs gathered information from grants

funded by the Robert Wood Johnson Foundation and

SAMSHA as well as from other states’ initiatives in Mil-

waukee, Indianapolis, and New York. In developing their

proposed remedial plan, the state listened to parent advo-

cates, looked at research, and examined other states’ pro-

grams. The practical consideration of what might work,

given the sociopolitical context in Massachusetts, was a

high priority: ‘‘New Jersey was a model we looked at a lot,

‘cause we had to bring in, we always had to be mindful of

the Medicaid platform.’’

At the end of the imposed deadline, the two parties were

unable to reach agreement and presented separate propos-

als to the Judge. On July 16, 2007, the Judge issued a final

Judgment that essentially adopted the state’s proposed

remedial plan. Interestingly, despite acknowledged ten-

sions during the court proceedings and the development of

the remedial plan, the decision to include Wraparound in

Table 2 Characteristics of three episodes

Episode #1: Federal policy Episode #2: State Medicaid program standards Episode #3: Community-level CSA

practice

Dates 2005–2006 2007–2008 2009 to date

Decision to be made Was Massachusetts in violation

of Federal EPSDT

requirements?

How would the remedial plan be

operationalized in Massachusetts Medicaid

program standards?

How would CSAs implement

Wraparound within Medicaid

managed care?

Decision-maker(s) Federal Judge State administrators with oversight from Court

and input from plaintiffs’ attorneys

CSA administrators and line staff

Decision parameter Federal State Community

Decision/action Memorandum of decision

Judgment and remedial plan

Medicaid Program standards developed

CSA request for proposals developed

Ongoing, iterative decisions drive

day-to-day Wraparound

implementation

Sources of

information for

decision-making

Researchers

Clinical experts

Wraparound state pilot

managers

Plaintiff families

Medicaid managers

Family advocacy groups

Program experts

CSAs

Managed care entities

Wraparound purveyor

State administrators

Intermediaries Plaintiff and state attorneys Plaintiff’s attorneys

Consultant

Managed care entities

Wraparound purveyor

Adm Policy Ment Health

123

the plan was facilitated by consensus, not dispute, between

the plaintiffs and the state. One state manager said, ‘‘I think

everybody wanted Wraparound. I think we wanted it, they

wanted it.’’

Episode 2: Translating the Remedy Provisions into State

Medicaid Managed Care Program Standards

The second episode focused on translating the provisions of

the Court’s Judgment into action regarding the state

Medicaid program. Episode 2 differed from Episode 1 in

that the decision as to how the remedial plan would be

operationalized was negotiated between the plaintiff

attorneys and the state administrators directly and not in the

Federal Court. While the plaintiffs’ attorneys continued to

be actively involved, state leadership shifted from attorneys

in the state Attorney General’s office to administrators in

the Massachusetts EOHHS and the state Medicaid pro-

gram. The Judge also appointed a Court Monitor to rep-

resent him (see Table 1.)

The negotiations provided opportunity for the exchange

of additional knowledge, some of which reflected research

but most of which related to the managed care and regu-

latory context specific to Massachusetts. This contextual

knowledge was essential for the development of the pro-

gram standards regarding the implementation of Wrap-

around within the Massachusetts Medicaid managed care

system. Participants during this episode included repre-

sentatives from other states that had made similar changes

in their Medicaid program, national experts, clinicians, and

parent advocates. One state-level key informant summa-

rized the negotiations as follows:

All of the services were developed through negotia-

tion with the attorneys for the plaintiffs, so every

definition was developed in negotiation, the service

descriptions that providers used, the program model,

the operations manuals and the medical necessity

criteria were all a negotiation between the attorneys

for the plaintiffs and the state… And, so, they also,

you know, they were attorneys and so their expertise

was on the legal matters. They had to use a group of

external consultants to advise them. We were using

external kind of consultants to advise us. And then

our Court Monitor was using consultants that she

would bring into talk to both of us. So, it was all of

that that was coming together to, to develop, kind of,

what the service descriptions looked at.

This episode was characterized by intense time pres-

sures that affected the knowledge exchange process. The

Judge had established tight implementation timeframes,

requiring intensive work planning. One state manager

described this episode: ‘‘we had to get a product to the

attorneys to respond to in three weeks, so that we could

negotiate it for two, so we could get it to the MCOs to roll

out, so they could go live with providers in three months,

so providers could have, you know, six to eight weeks to

get ready.’’ When asked to describe the specific role of

research in the process, one informant stated: ‘‘research is

always in the mix, but where research falls in terms of, is it

the first, second, third, or fourth kind of point in the deci-

sion-making process, in that type of process does vary.’’

This informant clarified: ‘‘it’s not that, again, research

didn’t play a role, ‘cause it did. I’m wondering if it

would’ve played a different role if it wasn’t in the context

of a lawsuit… it’s just different…the time pressure…was

always the number one decider.’’

Managing knowledge exchange among the parties in a

manner that produced complex documents reflecting deci-

sions made in a short timeframe was facilitated by a local

consultant with a highly regarded national reputation and

extensive expertise in Medicaid regulations. The consultant

functioned as an intermediary, demonstrating the necessary

skills to effectively work with attorneys as well as the

ability to generate realistic options for consideration by

state program managers. The consultant was not vested

with the decision-making authority of an institution (as was

the Judge) but rather exercised influence based on the trust

accorded him by the parties. One state-level informant

commented: ‘‘You know, I think that’s what was critical,

was I think [consultant] was trusted by both sides… You

know, that’s what mattered. You know, the parties are

never, never, ever gonna trust each other… a trustworthy

conduit of information…. that was absolutely essential.’’

The consultant thus functioned as an intermediary in the

exchange between the plaintiffs and the state, building on

both his trusted status as a neutral party as well as his

expertise in the knowledge at the heart of the lawsuit and

remedial plan (i.e., Medicaid regulations and operations).

A number of meetings during this episode were in fact held

in his office, creating a neutral space in which to negotiate

critical decisions during this episode.

Episode 3: Implementing Wraparound Practice Through

Community Service Agencies (CSAs)

The third episode consisted of numerous ongoing and iter-

ative exchanges to address the two major challenges the state

faced: (1) to train rapidly a sufficient workforce of intensive

care coordinators and family partners able to deliver Wrap-

around; and (2) to implement Wraparound within the context

of the state Medicaid program standards and managed care

reimbursement strategies. This episode also involved a sig-

nificantly different and diverse set of stakeholders and

organizations than in the first two episodes. Stakeholders

continued to include state administrators but a number of

Adm Policy Ment Health

123

new parties were added including the national Wraparound

purveyor staff, MCE technical assistance teams, CSA man-

agers, supervisors and, ultimately, frontline workers. The

Court Monitor continued to function in providing review of

implementation processes.

Wraparound requires a skilled workforce of trained care

coordinators and family partners, whose job descriptions

differ significantly from traditional clinical roles. In order

to provide the knowledge to community organizations to

move program standards regarding Wraparound into

practice, the state initially provided immediate training by

a local (Massachusetts) community-based organization that

had used Wraparound in a SAMHSA funded program

outside the context of Medicaid managed care. The state

subsequently established a 2-year contract with a national

purveyor to provide intensive training and coaching.

Training methods in Wraparound include ‘‘four-day

didactic trainings and then there were coaches’ trainings

and then there’s individual coaching’’ as well as ‘‘site

based coaches’’ to address implementation with specific

children and families seen at community organizations.

The state also required the Medicaid MCEs (MCEs) to

‘‘have experts within the plans who knew the services, who

could kind of make sure that their internal processes were

gonna support the service.’’

Unlike the first two episodes that produced decisions

documented in writing, the CSAs had to make numerous

decisions on a daily basis. The ad-hoc nature of these

decisions, at times, led to conflicting advice from the

Wraparound purveyor and the MCEs. As one member of an

MCE technical assistance team observed: ‘‘First and fore-

most there’s an inherent rub between Wraparound and

Medicaid, so having to have the [pause] you know, having

to understand Wraparound so that you can try and make it fit

as best as possible into Medicaid is an ongoing challenge.’’

A state administrator tied implementation to the fit between

research and the managed care environment in Massachu-

setts, describing the goal to ‘‘build upon the research that

was available at the time about… Wraparound,’’ but high-

lighting the unique challenge of context: ‘‘… with the

exception of say, a New Jersey, who is using a ASO or some

other examples of that, you didn’t have this kind of inter-

section with five different managed care companies by

which you had to navigate.’’ Knowledge exchange thus

needed to focus not only on the specifics of Wraparound but

also on the context in which it was being embedded.

How knowledge was exchanged among the Wraparound

purveyor, the MCEs, and the CSAs evolved over time. In

the beginning, the Wraparound purveyor and the MCE

technical assistance teams each consulted separately with

the CSAs. After a period of some frustration among all

parties due to this arrangement, the state started acting as a

mediator between the MCEs and Wraparound purveyor,

arranging direct conversations via conference calls and

other mechanisms to facilitate knowledge exchange. One

MCE technical assistance team member described the

conference calls as: ‘‘…one way that I think that we’ve

worked well with the, the [Wraparound] coaches to be able

to have … I think direct contact with the coaches worked

well.’’ Relationships developed over time, as the Wrap-

around purveyor and the MCE technical assistance teams

worked together with the common purpose of supporting

the effective practice of Wraparound by the CSAs within

the Medicaid program. The knowledge exchanged across

these groups within the context of CSA decision-making

informed the perspectives of each of the groups. As one

member of the Wraparound purveyor’s team commented,

‘‘And I think that’s one of the things that’s made it smooth

is, you know, I can make this sound like we’ve gotten the

MCEs to change. Well, we changed together.’’

While knowledge exchange among the CSAs, MCEs,

and Wraparound purveyor helped facilitate problem-solv-

ing around Wraparound implementation, many of the CSA

administrators and staff interviewed described challenges

experienced at the interface between Wraparound and

managed care. CSA administrators struggled with billing

and reimbursement issues: ‘‘Nobody really helps you as

much with how to do Wrap in the managed care world. You,

know? Like how to manage your billing and time.’’ Simi-

larly, a consistent theme identified by CSA staff members

was the conflict between core Wraparound principles of

‘‘family voice, family choice’’ and Medicaid managed care

reimbursement requirements. For example, CSA staff

commented that MCE requirements to complete a family

assessment within 10 days conflicted with Wraparound’s

commitment to meaningfully engage families on their own

terms and within families’ timeframes. This was particu-

larly challenging for staff working with referred families

who were not necessarily invested in Wraparound and were

active clients with the Department of Children and Families

(the state child welfare/protective services agency). Despite

training, a manual, and coaching and knowledge exchange

occurring with the MCEs and Wraparound purveyor, CSAs

felt additional interaction among the CSAs to exchange

their experienced-derived knowledge with each other would

be helpful: ‘‘We don’t get to… sit down at a table and say,

‘how did you do that?’… ‘how did you figure that out?’ I

think that would be a, a much bigger help, you know?’’

Discussion

This case study explored the theoretical constructs of

knowledge exchange in one state as policy, program, and

practice decisions were made regarding the need for sys-

tems change (Episode 1) and the subsequent

Adm Policy Ment Health

123

implementation of Wraparound within a Medicaid managed

care platform in response to a Federal court case (Episodes

2–3). Our findings affirm that the translation and application

of research findings in policy and practice requires rela-

tionships among researchers, intermediaries, policymakers,

and practitioners that support the mutual, multi-directional

exchange of a broad spectrum of knowledge (Mendel et al.

2008; Tseng 2012). Essential knowledge exchanged in

these episodes included research but also knowledge

regarding context (e.g., socio-political and regulatory). It

should be noted that, within our broader conceptualization

of knowledge exchange, we define exchanges that involved

intermediaries representing the research (i.e., Wraparound)

and context (e.g., the MCEs) in Episode 3 as knowledge

exchange. Some might argue that the absence of research-

ers, even when represented by intermediaries, is not con-

sistent with other published definitions of knowledge

exchange. We posit that our inclusive definition of other

types of knowledge and the in-depth examination in this

case study allowed us to uncover important themes

regarding knowledge exchange frameworks.

In our review of the literature, few knowledge exchange

frameworks explicitly recognize that knowledge exchange

may occur within distinct episodes over time, each with

different purposes and decisions to be made, requiring the

exchange of different types of knowledge, and involving

multiple and shifting stakeholders representing research as

well as policy, program, and practice. The complexity

inherent within the inter-connected decision-making epi-

sodes in this case study of mental health policy allows

further elaboration of previous frameworks as discussed

below.

Shifting Diversity of Stakeholders and Knowledge

Exchanged Over Time

While knowledge exchange frameworks have identified the

value of the perspectives and contributions of multiple and

diverse participants, this case study highlights the different

levels (i.e., federal, state, community) within and across

which knowledge was exchanged in order to make and

implement mental health policy decisions over time.

Rather than more static models of ongoing relationships

between research producers and research users, relation-

ships in this case study consisted of multi-directional

exchanges among a shifting panoply of research experts;

decision-makers at the policy, program, and practice levels;

and intermediaries.

As the overarching goal of improving mental health

services for children in Massachusetts covered by Medicaid

falls under the purview of Federal, state, and community-

based entities, decisions and actions at the Federal, state,

and community level were necessary; no one level was

sufficient by itself to change publicly funded mental health

services for children. The Federal Court’s responsibility

was to determine if the state had failed to comply with

Federal EPSDT requirements and, once finding so, to

identify the specific remedial services that were needed,

employing research where the Judge deemed appropriate.

State administrators were required to establish new state

Medicaid program standards that would ensure compliance

with the remedial plan, including Wraparound, and to do so

in the context of the Massachusetts Medicaid Program’s

managed care arrangements. They supplemented research

knowledge provided by national experts with their own

professional experience as well as that of administrators in

other states that had made similar changes in their Med-

icaid programs and of parent advocates. The community-

based provider agencies that serve as CSAs were respon-

sible for determining how to implement Wraparound using

the resources available in their community and within

Medicaid rules and rates. They relied on their own exper-

tise and experience in delivering services to their com-

munities as well as knowledge provided by the

Wraparound purveyor and by the MCE technical assistance

teams. The CSAs also acknowledged the need for sharing

information across CSAs with respect to operationalizing

the core principles and practices of Wraparound within the

context of the Massachusetts Medicaid program stan-

dards and a managed care platform.

Similarly, stakeholders representing ‘‘research produc-

ers’’ changed over time as the nature of the decisions being

made changed. In Episode 1, researchers provided data

regarding the rates of mental health needs of children and

the unmet need of children in Massachusetts; researchers,

program developers, and evaluators also shared evidence

for potential effective interventions to address these needs.

In contrast, in Episode 3, trained purveyors of the Wrap-

around model provided technical assistance and problem

solving regarding implementing Wraparound in a manner

consistent with its evidence base. Some knowledge

exchange models call for the active engagement of end-

users in the prioritization of research questions and the

development of study designs, adopting a participatory

research approach where researchers and end-users work

together from the creation of research to its implementation

(Graham et al. 2006). Our findings suggest that the sheer

number and diversity of stakeholders engaged in moving

research first to policy and then to practice may require

other creative approaches for exchanging knowledge that

accommodate this potential shifting set of stakeholders.

Intermediaries Translate Contexts as well as Research

The research literature identifies the critical role of inter-

mediaries in translating research to potential policy and

Adm Policy Ment Health

123

practice users. Our case study revealed an important vari-

ation of this function. In this case study, intermediaries

were also needed to translate the complex context (i.e.,

policy, regulatory, and fiscal environment) in which the

research (in this case study, Wraparound) was to be

embedded. In Episode 2, this function was filled by a

trusted consultant who used his own expertise and

knowledge about the Massachusetts Medicaid program and

other states’ Medicaid programs to facilitate knowledge

exchange among state administrators, plaintiff attorneys

and key stakeholders. In Episode 3, the Wraparound pur-

veyor functioned as an intermediary by sharing information

about Wraparound. The MCEs also functioned as inter-

mediaries regarding the Massachusetts Medicaid managed

care parameters. Without creating a shared knowledge

basis that included both research and context, it would have

been difficult to develop program standards and opera-

tionalize those standards for CSA line staff.

Decision-Making Parameters Narrow and Participants

Broaden over Time

Over the course of the three episodes, decision-making

parameters narrowed in scope, moving from the Federal

level to the state level and then to the local community level.

Decisions made during each episode occurred within the

context of parameters established by decisions made in the

previous episode. At the same time that the parameters nar-

rowed in scope, the numbers of participants with different

organizational affiliations participating in decision-making

increased. (see Fig. 1.) The decisions in Episode 1 were

made by the Judge, the sole arbiter and decision-maker,

using evidence presented in the Court by attorneys bringing

in a variety of individuals to testify and share ‘‘knowledge,’’

broadly defined. Following the Judge’s rulings, decision-

making responsibility shifted to a small group of state

administrators meeting in the neutral space of the trusted

intermediary to make detailed changes to the Medicaid

program specifications, with the input of the plaintiffs’

attorneys and the Court Monitor. In the third episode,

knowledge was exchanged across a number of players rep-

resenting the state, the MCEs, the Wraparound purveyor, and

CSA administrators and staff. These exchanges occurred in

multiple locations where state and regional trainings and on-

site consultations occurred. Consequently, the setting and

mechanisms for fostering knowledge exchange changed.

Public Documents Promote Coherence Across Episodes

Given the shifts in level (Federal, state, community) and

the broadening numbers of stakeholders engaged in

knowledge exchange, coherence from one episode to the

next was critical for ensuring that the desired goal, the

implementation of Wraparound, was achieved. Consistency

in some participants across episodes promoted some

coherence. For example, the Court Monitor and the plain-

tiffs’ attorneys continued to be involved in Episode 2 to

ensure that decisions were made within the parameters

established by the remedial plan in Episode 1. However,

while there was some overlap of participants, most par-

ticipants were not consistent across the episodes requiring

some other vehicle for transmitting information.

Instead, public documents served an essential function

to maintaining coherence. Both Episode 2 and 3 were

guided by written documents produced through decisions

made during previous episodes. These documents estab-

lished a public record, readily available on websites, that

allowed participants as well as outside observers to track

the basis of decisions from policy to program to practice.

Study Limitations

The context of a Federal class action lawsuit against a public

agency might have affected the type and nature of knowl-

edge exchange that occurred. A courtroom has unique rules

of engagement and standards of evidence that bound the

potential for knowledge exchange. Some might suggest that

uni-directional knowledge transfer from witnesses to the

judge would be a more accurate characterization of the trial

process than knowledge exchange, as knowledge was pri-

marily transferred to the Judge in Episode 1, with knowl-

edge exchange occurring in Episodes 2 and 3. Our findings

in this regard are limited by the fact that we did not observe

the trial or interview the judge or a diverse set of witnesses,

including researchers describing the high rates of mental

health needs among children or potential interventions. The

use of research evidence in the courts should be directly

examined because class action lawsuits are not uncommon

with respect to service provision under Medicaid, public

mental health, or child welfare/child protective services.

Indeed, one review from 1995 to 2005 identified 35 class

action lawsuits initiated to improve children’s public ser-

vices during that decade (Kosanovich and Joseph 2005).

However, class action lawsuits are not the only impetus for

policy change. The US Congress, state legislatures, and city

councils enact policy, all of which must be translated into

programs and practice in order to have the intended impact

on children, families, and communities.

We also recognize the limitations of our case study

approach. First, these three episodes occurred over an

extended time period; hence, the interviews conducted for

Episodes 1 and 2 were retrospective in nature while those

for Episode 3 took place as Wraparound was being

implemented. In addition, we did not interview the original

researchers who developed Wraparound, but instead

interviewed a national Wraparound purveyor. Last, case

Adm Policy Ment Health

123

studies are inherently biased in their examination of any

process as they are rooted in a particular context. Addi-

tional research should be conducted to confirm our findings

regarding multiple, interconnected episodes over time in

other contexts.

Conclusions

This case study explored common themes across knowl-

edge exchange frameworks within the context of public

mental health policy in one state. Practically, with respect

to the delivery of mental health services, our paper high-

lights the complexity of moving research to policy and

practice and calls for heightened attention to this com-

plexity. With respect to the delivery of evidence-informed

mental health services, translation and application of

Wraparound occurred in distinct, sequential episodes as

research moved to policy and subsequently practice. Our

findings affirm the value of employing a knowledge

exchange framework to examine the interface between

research, policy, and practice. Findings also suggest that

attention must be paid to the changing individuals, orga-

nizations, priorities, and informational needs to inform

decisions and subsequent action across episodes. In addi-

tion, explicit acknowledgement of intermediaries translat-

ing research, policy, and practice information and the skill

sets needed to serve as trusted intermediaries deserves

further exploration. Future research should investigate

specific strategies for managing transitions across episodes

and promoting effective exchanges across domains (e.g.,

Federal, state, community) and stakeholders.

Acknowledgments This research was conducted as part of the

‘‘Using Evidence to Improve Medicaid Mental Health Services for

Massachusetts Children and Youth Study’’ funded by the William T.

Grant Foundation. We thank Tully Saunders, Alexandra Caro, and

Thomas Mackie for their editorial assistance and review. We grate-

fully acknowledge the decision-makers who generously shared their

time and expertise with the research team, all of whom demonstrated

a noteable commitment to the well-being of the children of

Massachusetts.

References

Asen, R., Gurke, D., Solomon, R., Conners, P., & Gumm, E. (2011).

‘‘The research says’’: Definitions and uses of a key policy term in

federal law and local school board deliberations. Argumentation

and Advocacy, 47, 195–213.

Brownson, R. C., Chriqui, J. F., & Stamatakis, K. A. (2009).

Understanding evidence-based public health policy. American

Journal of Public Health, 99(9), 1576–1583. doi:10.2105/AJPH.

2008.156224.

Burchard, J. D., Bruns, E. J., & Burchard, S. N. (2002). The

Wraparound approach. In B. J. Burns & K. Hoagwood (Eds.),

Community treatment for youth: Evidence-based interventions

for severe emotional and behavioral disorders (1st ed.,

pp. 69–90). New York, NY: Oxford University Press.

Center for Mental Health Services (US), National Center for Injury

Prevention and Control, National Institute of Mental Health

(US), and Office of the Surgeon General (US). (2001). Youth

violence: A report of the surgeon general. Rockville, MD: Office

of the Surgeon Genera. Retrieved from http://www.ncbi.nlm.nih.

gov/pubmed/20669522. Accessed 14 Oct 2013.

Center for Public Representation. (2006). Summary of the Court’s

Decision in Rosie D. V. Romney. Retrieved from http://rosied.

org/Default.aspx?pageId=526903. Accessed 14 Oct 2013.

Committee on Quality of Healthcare in America, and Institute of

Medicine (US). (2001). Crossing the quality chasm: A new

health system for the 21st century. Health San Francisco (pp.

1–358). National Academy Press. Retrieved from http://

webcache.googleusercontent.com/search?q=cache:WYXMC

zaAmCMJ:www.nap.edu/html/quality_chasm/reportbrief.

pdf. Accessed 15 Oct 2013.

Creswell, J. W., & Plano Clark, V. L. (2007). Designing and

conducting mixed methods research. In N. Edition (Ed.), Applied

linguistics (Vol. 2) (p. 457). Thousand Oaks: Sage. doi:10.1111/j.

1753-6405.2007.00096.x.

Dedoose. (2012). Los Angeles, CA: SocioCultural Research Consul-

tants, LLC.

Denzin, N. K., & Lincoln, Y. S. (Eds.). (2005). The SAGE handbook

of qualitative research (Vol. 3). Sage. Retrieved from http://

www.loc.gov/catdir/toc/ecip053/2004026085.html. Accessed 15

Oct 2013.

Eisenhardt, K. M. (1989). Building theories from case study research.

Academy of Management Review, 14(4), 532.

Estabrooks, C. A., Derksen, L., Winther, C., Lavis, J. N., Scott, S. D.,

Wallin, L., et al. (2008). The intellectual structure and substance

of the knowledge utilization field: A longitudinal author co-

citation analysis, 1945 to 2004. Implementation Science, 3, 49.

Frey, J. H., & Fontana, A. (1991). The group interview in social

research. The Social Science Journal, 28(2), 175–187.

Gagnon, M. L. (2011). Moving knowledge to action through

dissemination and exchange. Journal of Clinical Epidemiology,

64(1), 25–31.

Gibbons, M. (2008). Why is knowledge translation important?

Grounding the conversation. Focus: Technical Brief, 21.

Retrieved from http://www.uofaweb.ualberta.ca/kusp/pdfs/Cal

garyCIHR41.pdf. Accessed 16 Oct 2013.

Graham, I. D., Logan, J., Harrison, M. B., Straus, S. E., Tetroe, J. M.,

Caswell, W., et al. (2006). Lost in knowledge translation: Time

for a map? The Journal of Continuing Education in the Health

Professions, 26(1), 13–24.

Hogan, M. F. (2003). The President’s New Freedom Commission:

recommendations to transform mental health care in America.

Psychiatric Services, 54(11), 1467–1474.

Institute of Medicine (US). (2006). Improving the quality of health

care for mental and substance-use conditions. Washington, DC:

National Academies Press.

Kitzinger, J. (1995). Introducing focus groups. BMJ, 311, 299–302.

Ko, D.-G., Kirsch, L., & King, W. (2005). Antecedents of knowledge

transfer from consultants to clients in enterprise system imple-

mentations. Management Information Systems Quarterly, 29(1),

59–85.

Kosanovich, A., & Joseph, R. M. (2005). Child welfare consent

decrees: Analysis of thirty-five Court actions from 1995 to 2005,

(October). Retrieved from http://www.cwla.org/advocacy/con

sentdecrees.pdf?pagewanted=all. Accessed 16 Oct 2013.

Lavis, J. N., Robertson, D., Woodside, J. M., Mcleod, C. B., &

Abelson, J. (2003). How can research organizations more

effectively transfer research knowledge to decision makers?

Milbank Quarterly, 81(2), 221–248.

Adm Policy Ment Health

123

Lincoln, Y. W., & Guba, E. G. (1985). Naturalistic inquiry. Newbury

Park, CA: Sage.

Mendel, P., Meredith, L. S., Schoenbaum, M., Sherbourne, C. D., &

Wells, K. B. (2008). Interventions in organizational and

community context: A framework for building evidence on

dissemination and implementation in health services research.

Administration and Policy in Mental Health, 35(1–2), 21–37.

doi:10.1007/s10488-007-0144-9.

Miles, M. B., & Huberman, A. M. (1994). Qualitative data analysis:

An expanded sourcebook. Beverly Hills, CA: Sage.

Mitton, C., Adair, C. E., McKenzie, E., Patten, S. B., & Waye Perry,

B. (2007). Knowledge transfer and exchange: Review and

synthesis of the literature. Milbank Quarterly, 85(4), 729–768.

New Freedom Commission on Mental Health. (2003). Achieving the

promise: Transforming mental health care in America. Final report.

DHHS Pub. No. SMA-03-3832. Rockville, MD. Retrieved from

http://govinfo.library.unt.edu/mentalhealthcommission/reports/

FinalReport/downloads/FinalReport.pdf. Accessed 14 Oct 2013.

Palinkas, L. A., Aarons, G. A., Chorpita, B. F., Hoagwood, K.,

Landsverk, J., & Weisz, J. R. (2009). Cultural exchange and the

implementation of evidence-based practices: Two case studies.

Research on Social Work Practice, 19(5), 602–612. doi:10.1177/

1049731509335529.

Pentland, D., Forsyth, K., Maciver, D., Walsh, M., Murray, R., Irvine,

L., et al. (2011). Key characteristics of knowledge transfer and

exchange in healthcare: Integrative literature review. Journal of

Advanced Nursing, 67(7), 1408–1425.

Rapport, F. (2010). Summative analysis: A qualitative method for

social science and health research. International Journal of

Qualitative Methods, 9(3), 270–290.

Rosie D. v. Romney, 410 F.Supp.2d 18. (2006). Retrieved from http://

www.clearinghouse.net/chDocs/public/MH-MA-0005-0012.

pdf. Accessed 19 Sept 2013.

Rossman, S. (2002). Services integration: Strengthening offenders and

families, while promoting community health and safety. In: From

prison to home: The effect of incarceration and reentry on children,

families, and communities. The Urban Institute. Retrieved from

http://www.urban.org/publications/410625.html. Accessed 15 Oct

2013.

Suter, J. C., & Bruns, E. J. (2009). Effectiveness of the wraparound

process for children with emotional and behavioral disorders: A

meta-analysis. Clinical Child and Family Psychology Review,

12(4), 336–351.

Tseng, V. (2012). Social policy report: The uses of research in policy

and practice (Vol. 26). New York: Society for Research in Child

Development.

Tseng, V., & Seidman, E. (2007). A systems framework for

understanding social settings. American Journal of Community

Psychology, 39(3–4), 217–228.

Weiss, C. H. (1998). Evaluation (2nd ed.). Upper Saddle River, NJ:

Prentice-Hall Inc.

Weisz, J. R., & Addis, M. E. (2006). The research-practice tango and

other choreographic challenges: Using and testing evidence-based

psychotherapies in clinical care settings. In C. D. Goodheart, A.

E. Kazdin, & R. J. Sternberg (Eds.), Evidence-based psychother-

apy: Where practice and research meet (1st ed., pp. 179–206).

Washington, DC: American Psychological Association.

World Health Organization. (2006). Bridging the ‘‘know–do’’ gap

meeting on knowledge translation in global health (p. 17).

Geneva, Switzerland. Retrieved from http://www.who.int/kms/

WHO_EIP_KMS_2006_2.pdf. Accessed 16 Oct 2013.

Adm Policy Ment Health

123