Treasury Guest Lecture: Barry Duncan - Presentation Slide

heartandsoulofchange.com; pcoms.com February 2016

betteroutcomesnow.com 1

The Partners for Change Outcome Management System

Barry Duncan, Barry Duncan, Psy.DPsy.D. . heartandsoulofchange.comheartandsoulofchange.comppcoms.comcoms.com betteroutcomesnow.combetteroutcomesnow.com

Evidence Based Practice One Client at a Time

What’s Up: PCOMSResearch, Predictors, Implications

What’s Up: PCOMSResearch, Predictors, Implications

1. Research-proven strategy to address the problems of psychosocial services

2. Incorporates known predictors of ultimate outcome

3. Implications for Policy

PCOMS t GlPCOMS at a Glance

Psychosocial ServicesThe Good…

Study after study, Study after study, and studies of and studies of t di h tht di h thstudies show the studies show the

average treated average treated client is better off client is better off than 80% of the than 80% of the untreated sample.untreated sample.

Psychosocial ServicesThe Bad…

•• Drop out rates Drop out rates average average 47%47%

•• Providers Providers vary… a lot vary… a lot

TDCRP: top third psychiatrists giving placebo bested bottom third giving

Provider DifferencesProvider DifferencesIncredible Variation Among ProvidersIncredible Variation Among Providers

Provider DifferencesProvider DifferencesIncredible Variation Among ProvidersIncredible Variation Among Providers

g gmeds; clients of best counselors improve 50% more & dropped out 50% less; meds useful for clients of more effective, not for less-effective McKay, K., Imel, Z., & Wampold, B. (2006).

Psychiatrist effects in the psychopharmalogicaltreatment of depression. Journal of Depressive Disorders, 92, 287-290.



Reported a combined reliable and clinically

More Bad: Public Behavioral Health Less than a Third Benefit

ysignificant change rate in everyday clinical practice just 29.1%. Over70% not helped Hansen, N., Lambert, M., Forman, E. (2002). The

psychotherapy dose-response effect and its implications for treatment delivery services. Clinical Psychology: Science and Practice, 9, 329-343.

And the UglyProviders Don’t Know

• 20-70% range

• Graded their effectiveness, A+ to F—67% said A or better; none rated belowbetter; none rated below average; Another study: 85% of their clients improve & 90% saw themselves as above the 75th percentile.

•• Providers Providers don’t know how don’t know how effective they areeffective they are

Sapyta, J., Riemer, M., & Bickman, L. Feedback to clinicians: Theory, research, and practice. Journal of Clinical Psychology: In Session, 61, 145-153Walfish, S., McAlister, B., O’Donnell, P., & Lambert, M. (2012). An investigation of self-assessment bias in mental health providers. Psychotherapy Reports, 110(2), 1-6. doi: 10.2466/02.07.17

No MoreHead in the Sand

• Despite overall efficacy, dropouts a problem, not everyone benefits, providers vary significantly and don’t have a clue about effectiveness, and outcomes in many settings are dismal.

Problems Spurred Interest inQuality Improvement Strategies

• Primary Approach Has BeenBeen Transporting Evidence Based Treatments

Problems withTransporting EBTs

• Dodo Bird VerdictVerdict

• Cost and Practicality

“Everybody has won and “Everybody has won and all all must have prizes.”must have prizes.”

Cost and PracticalityDoesn’t Make Cents

• Cost of Implementing EBT; VA—20 million

• Practicality Given Staff Turnover, Fidelity

• Implementing an EBT for Every Diagnoses…Over 150 EBTs and Over 400 Diagnoses



No MoreHead in the Sand

• The Primary Quality yImprovement Strategy Has Big Problems.

To The Rescue, Another QI StrategyContinuous Client Feedback: PCOMS

The The ORSORS The The SRSSRSDownload free working copies at: Download free working copies at:

ppcoms.comcoms.com or or heartandsoulofchange.comheartandsoulofchange.com

Improving or NotNot Rocket Science

• PCOMS simply identifies clients who are not responding so that the lack of progress can beof progress can be addressed in a proactive way that keeps clients engaged while helpers collaboratively seek new directions…but does it improve outcomes?

• Feedback v TAU; Both persons reliable or sig. h 50 5%

Becoming BetterBecoming BetterAnkerAnker, , DuncanDuncan, & , & SparksSparks (2009)(2009)

Becoming BetterBecoming BetterAnkerAnker, , DuncanDuncan, & , & SparksSparks (2009)(2009)

change—50.5% v. 22.6%; ES: .50; 4 xs# of clin. sig. change

• FU: TAU-34.2% v. 18.4% Feedback sep./divorce rate

Anker, M., Duncan, B., & Sparks, J. (2009). Using client feedback to improve couple therapy outcomes: A randomized clinical trial in a naturalistic setting. Journal of Consulting and Clinical Psychology,77(4), 693-704.

Reese, Norsworthy, & Rowlands (2009)Reese, Toland, Slone, & Norsworthy, 2010

• N=148: Feedback group doubled controls (10.4 vs. 5.1 pts); ES: .48

Reese, R., Norsworthy, L., & Rowlands, S. (2009). Does a continuous feedback model improve psychotherapy outcomes? Psychotherapy,46, 418-431.

• Like Norway study, clients, regardless of risk status, benefit from continuous feedback

• 2010 study is replication of Norway Trial

Reese, R., Toland, M., Slone, N., & Norsworthy, L. (2010). Effect of client feedback on couple psychotherapy outcomes. Psychotherapy, 47, 616-630.

A SAMHSA EBPBut Different

PCOMS is a-theoretical & therefore additive totherefore additive to any therapeutic orientation, including other EBPs; PCOMS applies to clients of all diagnostic categories



Meta-analysis by Lambert & Shimokawa (2011)of PCOMS (the ORS and SRS)

Those in feedback group had 3.5 higher odds of experiencing reliable change

Those in feedback group had less than half the odds of experiencing deterioration

Feedback attained .48 ES

Lambert, M. J., & Shimokawa, K. (2011). Collecting client feedback. Psychotherapy, 48, 72-79.

Group, Substance Abuse, & PCOMSSchuman, Slone, Reese, & Duncan, 2014

RCT of 263 returning Afghanistan and Iraq Vets and Soldiers about to be Deployed: improvedDeployed: improved outcomes on the ORS, more reliable and clinically significant change, superior clinician and Commander ratings; and reduced drop outs

Schuman, D., Slone, N., Reese, R.J. & Duncan, B. (2014). Efficacy of Client Feedback in Group Psychotherapy with Soldiers Referred for Substance Abuse Treatment . Psychotherapy Research.

Cooper, Stewart, Sparks, & Bunting, 2013

• Cohort study of 288 7-11 yr olds; caretaker-228; teacher-249; 11

l 28 h lcounselors; 28 schools

• 2 fold advantage over children not using feedback as measured on the SDQ

Cooper, M., Stewart, D., Sparks, J., & Bunting, L. (2013). School-based counseling using systematic feedback: A cohort study evaluating outcomes and predictors of change. Psychotherapy Research, 23, 474–488.

Regarding Provider VariabilityFeedback Improves Outcomes

• Norway: 9 of 10 got better outcomes

• Feedback raised effectiveness of the lower ones to their more successful colleagues.

• Provider in low effectiveness group became BEST w/feedback!

PCOMS Addresses the ProblemsRecaptures At Risk Clients

• Feedback tailors services based on response, provides an early warningprovides an early warning system to prevents drop-outs & negative outcomes, & solves provider variability—feedback improves performance as demonstrated in 5 RCTs

But What About the Real World?Benchmarking Studies

• Compares results of RCTs to outcomes in real world. Real world

h h d tmuch harder to secure good outcomes

• Comparable results would suggest a viable method of quality improvement



Southwest Behavioral HealthPublic Behavioral Health

Achieved outcomes comparable to RCTs of depression

SBHS, Multicultural PBH CenterWith a $70 Million Budget

pand feedback. N of 5186; 957 with substance abuseHow? PCOMS!

Reese, R. J., Duncan, B. L., Bohanske, R. T., Owen, J. J., & Minami, T. (2014). Benchmarking outcomes in a public behavioral health setting: Feedback as a quality improvement strategy. Journal of Consulting andClinical Psychology, 82, 731–742.

5 RCTs, the largest benchmarking study ever conducted in public behavioral health, & a cohort study h h h CO S d li i h

Delivers

have shown that PCOMS delivers with youth & adults, in individual, couple, & group therapy, with both mental health & substance abuse problems, & with the impoverished and disenfranchised.

••Client’sClient’s rating of the rating of the alliancealliance the best predictor of the best predictor of engagement and outcome.engagement and outcome.

PCOMS Incorporates the PredictorsEarly Change and the Alliance

••Client’sClient’s subjective subjective experience of change early experience of change early in the process the best in the process the best predictor of success for any predictor of success for any particular pairing.particular pairing.

Baldwin, S., Berkeljon, A., Atkins, D., Olsen, J., & Nielsen, S. (2009). Rates of change in naturalistic psychotherapy. Journal of Consulting and Clinical Psychology, 77(2), 203-211.

Some clients do take longer, but the mythology never dies

N=4676; 77% attended 8 or less, and 91% 12 or less

Note that even for the clients who take longer, change starts early…just is flatter

Conclusions: Early change is an important factor for the prediction of short- and long-term outcome.

Clients who report little or no progress early on will likely show no improvement over the entire course of service or will end up

The Research about Early Change Is a Gift

The Research about Early Change Is a Gift

course of service, or will end up on the drop-out list—early change predicts engagement and a good outcome at termination. Provides a tangible way for us to identify folks who are not responding so that we can chart a new course.

Sudden Epiphany? NOTSudden Epiphany? NOT

Myth: Clients flat-line & then spike. Some clients do take longer but change starts right away

Myth: Clients flat-line & then spike. Some clients do take longer but change starts right awaystarts right away. So the question remains: When do you start getting worried when clients are not responding? I vote for sooner rather than later.

starts right away. So the question remains: When do you start getting worried when clients are not responding? I vote for sooner rather than later.



•Client’s rating of the rating of the alliancealliance the best predictor of the best predictor of engagement and outcome.engagement and outcome.

Starting With Starting With the Predictorsthe PredictorsChange and the Alliance

•Client’s subjective subjective experience of change early experience of change early in the process the best in the process the best predictor of success for any predictor of success for any particular pairing.particular pairing.

The AllianceBordin’sBordin’s Classic DefinitionClassic Definition

The Alliance:

•• Relational Bond Relational Bond A tA t ll

Duncan, B. (2014). On Becoming a Better Therapist, 2nd ed. Washington, DC: APA

•• Agreement on Agreement on goals goals • Agreement on tasksAgreement on tasks

Up to Seven Times the the Impact of Impact of Model/Technique; Model/Technique; Accounts Accounts for Most of for Most of Provider Provider VarianceVariance

36-50%

The Alliance: The Alliance: Over Over 1000 Research FindingsResearch Findings

Quality of the alliance more potent predictor of outcome than orientation, ,experience, or professional discipline

Same across modalities and orientations; client populations

The Session Rating Scale Measuring the AllianceThe Session Rating Scale Measuring the Alliance

••Give at the end Give at the end of session;of session;

••Each line 10 cm Each line 10 cm in length;in length;

•Reliable, valid, feasible

••Score in cm to Score in cm to the nearest mm;the nearest mm;

••Discuss with Discuss with client anytime client anytime total score falls total score falls belowbelow 3636

40.0%40.0%

30 0%30 0%

Factors Accounting for Factors Accounting for Successful Outcome

ClientClient/Life/Life

30.0%30.0%

15.0%15.0%15.0%15.0%

Lambert, M. J. (2013). The efficacy and effectiveness of psychotherapy. In M. J. Lambert (Ed.), Bergin and Garfield’s handbook of psychotherapy and behavior change (6th ed., pp. 169–218). Hoboken, NJ: Wiley.

AllianceAlliance

HopeHope/Expectancy/Expectancy

Models/TechniquesModels/Techniques



Client/Life FactorsBottom Line

• Engage clients & rally their

“The quality of the participation [emerges] as & rally their

resources to the cause.

[ g ]the most important determinant of outcome."

Orlinsky, D. E., Rønnestad, M. H., & Willutzki, U. (2004). Fifty years of process -outcome research: In M. J. Lambert (Ed.), Bergin and Garfield’s handbook of psychotherapy and behavior change (5th ed., pp. 307-390). New York: Wiley.

Until lions have their Until lions have their historians, tales of historians, tales of

Clients Are the Lions of ChangeAccount for Majority of Outcome

,,hunting will always hunting will always glorify the hunter.glorify the hunter.

African Proverb

ClientsThe Heart of Change

Client outcome feedback makes consumers the hi t i f th ihistorians of their own change

Partnering w/clients to monitor outcome engages the most potent factor of change

The AllianceThe Soul of Change

The AllianceThe Soul of Change

Alliance feedback enables a fit between client expectations,

f d ipreferences, and services

Does not leave the alliance to chance—applying over 1000 studies showing the relationship of the alliance to positive outcomes

Client Feedback Engages Consumers in a Partnership

Client Feedback Engages Consumers in a Partnership

Overlaps with & affects all factors—tie that binds

Soliciting feedback is a living process that engages

Overlaps with & affects all factors—tie that binds

Soliciting feedback is a living process that engagesliving, process that engages clients in monitoring outcome, heightens hope, fits client preferences, maximizes therapist-client fit, and is itself a core feature of change.

living, process that engages clients in monitoring outcome, heightens hope, fits client preferences, maximizes therapist-client fit, and is itself a core feature of change.

So What Does All This Mean in Terms of

P li ?

So What Does All This Mean in Terms of

P li ?

Barry,

Policy?What Can the Data Do for

Us?

Policy?What Can the Data Do for

Us?



Reliable ChangeReliable Change: Exceeds : Exceeds maturation, chance, and maturation, chance, and measurement error: 6 on measurement error: 6 on

The ORS: Reliable and Clinically Significant Change

the ORS; the ORS; Clinically Clinically Significant ChangeSignificant Change: A : A change of 6 plus crossing change of 6 plus crossing the clinical cut off of 25, a the clinical cut off of 25, a score typical of those not in score typical of those not in serviceservice

Using DataAt the Individual Level

Becoming BetterLinking Outcome to Process

••See clients See clients moremorefrequently when the frequently when the slope of change is slope of change is steep.steep.steep.steep.

••Begin to space the Begin to space the visits as the rate of visits as the rate of change lessens.change lessens.

••See clients as long See clients as long as there is as there is meaningful meaningful change.change.

Love Your DataKey Progress Indicators

Love Your DataKey Progress Indicators

1. Average Change: Is it above 6?

1. Average Change: Is it above 6?Is it above 6?

2. % reaching ETR RCSC

3. Effect Size

Is it above 6?2. % reaching ETR

RCSC3. Effect Size



Stepping UpReturn on Investment

Stepping UpReturn on Investment

People are afraid the data will be used against them often called P4P or pay for performance (punish for

People are afraid the data will be used against them often called P4P or pay for performance (punish forperformance (punish for performance

Better to encourage participation in data management as a continuous quality improvement strategy

performance (punish for performance

Better to encourage participation in data management as a continuous quality improvement strategy

Arbitrary Performance StandardsWill Bring Out the Dark Side

Arbitrary Performance StandardsWill Bring Out the Dark Side

Client welfare invoked as an explanation—But even the best don’t benefit up to a third of their clients.

Client welfare invoked as an explanation—But even the best don’t benefit up to a third of their clients.to a third of their clients. Will promote cheating and ruin the benefits

Instead encourage to identify non-responding clients and develop programming to address needs

to a third of their clients. Will promote cheating and ruin the benefits

Instead encourage to identify non-responding clients and develop programming to address needs

Raise the Bar of Effectiveness and Efficiency

Raise the Bar of Effectiveness and Efficiency

Client feedback improves both effectiveness and efficiency…engages clients

Data collection

Client feedback improves both effectiveness and efficiency…engages clients

Data collection Data collection demonstrates both POV/ROI and serves as a CQI strategy

Reward for participation, QI, and creative programming to address non-responders

Data collection demonstrates both POV/ROI and serves as a CQI strategy

Reward for participation, QI, and creative programming to address non-responders

Treasury Guest Lecture: Barry Duncan - Presentation Slide

Documents

Transcript of Treasury Guest Lecture: Barry Duncan - Presentation Slide