Collecting Post-Discharge

Outcome Data:Using Data to Inform Practice

Improvement and Results from the BBI Feasibility Pilot

Presented By:

Gary Blau: Child, Adolescent and Family Branch, SAMHSA (MD)Madge “Pat” Mosby: BBI Consultant (MD)Bob Lieberman: BBI Consultant; Lieberman Group, Grants Pass (OR)

July 25, 2018BBI Pre-Institute, University of Maryland Training Institutes

BBI MEASUREMENT PRODUCTSBBI Documents:1. BBI: Instructional Guide – Strategies for Completion of the Building Bridges Initiative 2. Self-Assessment Tool (SAT) – also available in Spanish3. BBI SAT for Youth and Families – also available in Spanish4. BBI SAT for Residential and Community Staff and Advocates 5. BBI SAT Glossary – also available in Spanish6. BBI: Outcome and Performance Measures for Residential: Building Consensus on Residential

Measures for Outcomes and Performance Measures 7. BBI Post-Residential Outcomes Pilot Survey

Webinars: 1. BBI Web-based Training: BBI: Successful Strategies for Tracking Long-term Outcomes (1 CEU)2. Best Practices in Using Data to Inform Practice and Tracking Long-term Outcomes for

Residential Programs and their Community CounterpartsArticle: Levison-Johnson, J., Kohomban, J.C., Blau, G., Caldwell, C., Dougherty, R., & Warder. R. (Spring, 2012). Keep Your Eyes on the Prize: Defining and Tracking What’s Important in Residential Care. Teaching Family Association Newsletter, 1(38), 1-8.Book: Blau, G., Caldwell, B., & Lieberman, R.E. (2014). Residential interventions for children, adolescents and families: A best practice guide. New York, NY: Routledge.

• Efforts are occurring to demonstrate effectiveness of residential interventions, but only recently have the efforts to assess post-discharge outcomes taken a broad national focus.

• Two uses for post-discharge data:

correlate with process and experience indicators for quality improvement purposes;

assess impact of the overall residential intervention.

• A variety of concerns, at varying levels, about feasibility and utility of collecting and accurately using post-discharge data.

Discussion:Identify concerns you can imagine or

have heard.

Context for the BBI Pilot

BBI Pilot Study

• A pilot study was designed to assess the feasibility of collecting post-departure follow-up data. This focused on addressing a variety of concerns regarding the feasibility and utility of doing so.

• Practitioners, researchers, family members, and policy makers collaborated in the development of a survey of post-discharge function in five domains: living situation, school/education, risk factors, community engagement and support, and access to services

• This pilot was not about obtaining actual outcome data – it was a test of the process of collecting data with the tool

• Residential providers in three states were recruited to follow a set protocol in collecting follow-up data

What are families looking for with data measures and post-

discharge outcomes for themselves and their children?

Family Voice/Recommendations

BBI Pilot Findings: Successes and Barriers

Successes• Providers were able access contact information from existing

records• Contact data was fairly up-to-date and few refusals to participate• Caregivers seemed to be comfortable answering questions• Caregivers provided positive feedback on survey

Barriers• Some caregivers were difficult to reach• Some providers did not allocate sufficient resources• Some providers had much smaller response rates

Discussion:

How might you address the perceived and actual barriers in your state or community?

• Strong organizational commitment to collecting follow-up data

• Adequate staff resources

• Flexible call times

• Additional possibilities An incentive for caregivers to participate Stratification designs to yield representative responses Response/non-response analysis to test for biased

responses

BBI Pilot Findings: Resources Needed

• Post-discharge data is one component in quantifying the long-term

impact of residential interventions

• Measure organizational capacity and readiness: this is needed to

identify resources and barriers to successful provider engagement

and implementation

• Use a methodologically sound longitudinal design to separate out

the impact of context on functional outcomes

Discussion point:

Immediate thoughts about next steps?

BBI Next Steps: Full-Scale Effort to Measurethe Impact of Residential Interventions

• It is possible to obtain follow-up data

• Accurate contact information is essential, but this typically already exists in organizational information systems

• The BBI survey is available and offers a uniform tool for collecting post-discharge data (click here).

• Correlate aggregated post-discharge data with the degree of family-driven, youth-guided, culturally and linguistically competent, trauma-informed, and/or transformational type practice strategies provided.

In the Meantime: Using Data to Inform

Practice Improvement

CHILDREN’S VILLAGE/NY• Identified an increase in post-discharge problems correlated to less work with

families during the residential intervention;• Combined MST with residential for families with multiple and complex

challengesDAMAR/IN• Changed business model to serving families within 45 miles so could increase

work with families• Practice model now includes all staff disciplines working in homes and

communities (i.e. schools) BOYS TOWN/NE• Years of post-discharge data inform ongoing practice improvementsYOUTH VILLAGES/TN• Put a primary focus on working with families and communities • Responded to data by shifting away from points and levels to trauma-informed

practices

Residential Program Examples of Using Post-

Discharge Data to Inform Practice Improvement

What do you want attendees to consider and take back with them as they

improve their focus on using data to inform practice improvement – both

during the residential intervention and post-residential intervention?

Family Voice/Recommendations

Wherever you are in

measuring and collecting data,

you can jump in and use it

to improve practice.

THE JOURNEY TO OUTCOMES

Next steps with measuring outcomes

and using data to inform practice

in your organization, community and/or state.

SMALL GROUP ACTIVITY

Madge “Pat” Mosby: BBI Consultant

MMosby@mcfof.org

Robert E. Lieberman, M.A., LPC

lieberbob@yahoo.com

Gary M. Blau, Ph.D.

Gary.Blau@samhsa.hhs.gov

CONTACT INFORMATION

Taking BBI back to your Program, Community and

State

• End of day activity • Customized action plan