Dynamic risk among Canadian psychiatric inpatients: Validity and reliability of the HARM-20V3 and...

Dynamic risk among Canadian forensic psychiatric inpatients: Validity and reliability of the HCR-20:V3 and the HARM

Alana N. Cook, Gary Chaimowitz, Mini Mamak, & Heather Moulden

Violence Risk & Psychiatry

  17-50% of inpatient psychiatry patients have a history of violence (Choe et al., 2008)

  Physical injury

  Psychological trauma

  Increased stigma

  Limited access to services for perpetrators

  Financial burden to social, criminal justice, & mental health services (Hodgins et al., 2007)

What do we do at St. Joes?   Forensic Service

  Monthly risk assessments for all forensic patients   HARM (Chaimowitz & Mamak, 2011)

  Comprehensive violence risk assessment (~yearly)   HCR-20, version 3 (Douglas et al., 2013) & other tools

  Management plans   Monitoring, Supervision, (Multidisciplinary) Treatment

  Communication of risk   Weekly meetings

  Summary of risk factors, risk judgment, formulation/scenarios, & management plan on file

Level Incident Description 9 Critical Incident – Possible Life

and Death –Possible Police Call

Serious violent assault or sexual assault. The victim requires medical attention. Police could be summoned

8 Violent Unprovoked Assault

Impulsive interpersonal assault in which no apparent precursors are identifiable

7 Violent Assault Aggression involves physical contact with another person

6 Push/Shove Clearly aggressive push or shove

5 Destruction of Property Aggression directed at property

4 Improper Physical Contact

Behaviour not an assault but physical contact inappropriate

3 Intimidating, Threatening, Personal Space Violated

Patient’s body language and words are threatening in nature

2 Intimidating, Raised Voice Patient is verbally intimidating, possibly yelling and possibly using profanities

1 Rude, Argumentative Patient is being rude, argumentative, and possibly challenging staff authority

No Intervention Physical Intervention Verbal Intervention N P V

Evaluating risk assessment tools

Professional Utility

• Is this tool useful for practice?

Reliability • Can this tool be applied consistently across different professionals?

Do similar tools lead professionals to the same opinion?

Validity • Is this tool helpful at predicting who will or will not be violent? At

identifying critical needs for the client for treatment or management?

Legal • Is the use of this tool fair in legal decision making? Is it logical to use

this tool to draw opinions of risk?

HARM

HCR-20V3

HARM (Chaimowitz & Mamak, 2011)

  Originally developed for use in forensic inpatient psychiatric settings

  Goals: improve team discussions about risk, to educate clinical team members about risk factors, and to direct attention to risk management strategies

  4 historical and 10 dynamic risk factors

  Clinicians to assign risk categories for violence and manage the risk for violence in the coming days and weeks

  Embedded in the HARM, the Aggressive Incidents Scale (AIS)

  defines nine levels of aggression and provides a standardized method of recording violent incidents

HARM: Research support   Utility

  User indicate improved team meetings

  Assisted in educating staff about variables and issues related to risk

  AIS Reliability

  Forensic psychiatric staff (nurses, occupational therapists, social workers, recreation staff, psychiatrists; ICC1 = 0.93) already familiar with the AIS tool

  Inexperienced or novice staff members (forensic service legal counsel, intake coordinator, psychiatric fellows and students) who had no prior experience with the AIS (ICC1 = 0.92).

  Current papers first evaluations of the validity of the HARM &AIS

HCR-20V3 (Douglas, Hart, Webster, & Belfrage, 2013)

  Newest version of the HCR-20

  Assess general violence risk, include violence in forensic psychiatric settings

  Applied in seven systematic and standardized steps

  10 historical and 10 dynamic risk factors

  Summary risk judgements about violence used to inform management strategies to mitigate or prevent future violence from occurring

HCR-20V3: Research Support

  Reliability and validity of Version 2 well-established (Douglas et al., 2014)

  Version 3 is significantly correlated with Version 2 of the HCR-20 (Douglas & Belfrage, 2014; Douglas et al., 2013; Strub, Douglas, & Nicholls, 2014)

  Growing support for the utility, interrater reliability, and predicative validity of the HCR-20V3 across various samples and settings (Doyle, et al., 2014; Douglas & Belgrage, 2014; Kötter, et al., 2014; Strub, Douglas, & Nicholls, 2014; de Vogel, et al., 2014).

See special issue on HCR-20V3

Concurrent Validity HARM/HCR-20V3

Predictive Validity of the HARM

Predictive Validity of the HCR-20V3

Need for empirical evaluations of the HARM & HCR-20V3

Method

Our sample

N = 39 Participants 85% male

General (n = 26) & Secure

(n = 13) units

8% Unfit and 92% NCRMD

92% Psychotic Symptoms

61% secondary diagnosis of

substance use

72% White Mean age of 40 years

Mean 9 previous

admissions

Mean length of stay >4 years

67% violent index offence

Mean 2 past violent offences

(range 0,11)

Procedure   HARM prospectively rated monthly by clinical team

  HCR-20V3 prospectively rated monthly from file by 5 trained raters

  Outcomes recorded using the AIS & MOAS

A1 A2

• FU2

A3

• FU3

A4

• FU4

A5

• FU5

A6

• FU6

HARM

SRR

Immediate (Days) with professional support

Short-term (Weeks) with professional support

Immediate (Days) without professional support

Short-term (Days) without professional support

Numerical Scores

4 Historical (0,1): range 0,4

10 Dynamic (-1,0,1): range -10,10

HARM Total: range -10,14

HCR-20V3

SRR

Risk for Violence/Case Priority

Serious Physical Harm

Imminent Violence



Numerical Scores

10 Historical (0,1,2): range 0,20

5 Clinical + 5 Risk = CR/Dynamic(0,1,2): range 0,20

HCR-20V3 Total: range 0,40

1-m

on

th

3-m

on

th+

TRA

JEC

TORY

IRR Summary Risk Ratings, N =11

Conclusory Opinions ICC1 Interpretation (Chichetti & Sparrow, 1981)

1 month: Risk for Violence/Case priority

.80 Excellent

3 month: Risk of violence/Case priority

.80 Excellent

ICC = Intraclass correlations

Outcomes

AIS & MOAS

Intimidation Threats Physical

MOAS

Narrow

Broad

Overview of the Data

0

10

20

30

40

50

60

70

A1 A2 A3 A4 A5 A6

% Low

Moderate

High

Proportion of HARM SRR Overtime: Immediate (Days) With Support

0

10

20

30

40

50

60

70

A1 A2 A3 A4 A5 A6

% Low

Moderate

High

Proportion of HARM SRR Overtime: Short-Term (Weeks) With Support

0

0.5

1

1.5

2

2.5

3

A1 A2 A3 A4 A5 A6

Historical

Dynamic

Total

HARM Numerical Scores Overtime

Proportion of HCR SRR Overtime: Future Violence/Case Priority (1-month)

0

10

20

30

40

50

60

A1 A2 A3 A4 A5 A6

% Low

Moderate

High

0

10

20

30

40

50

60

A1 A2 A3 A4 A5 A6

% Low

Moderate

High

Proportion of HCR SRR Overtime: Future Violence/Case Priority (3-month)

HCR Numerical Scores Overtime

0

5

10

15

20

25

30

A1 A2 A3 A4 A5 A6

Me

an

Sco

re

H

C+R

Total

Proportion of Any Violence

0%

20%

40%

60%

80%

100%

None

Any Violence

Proportion of Physical Violence

0%

20%

40%

60%

80%

100%

None

Physical Violence

HARM

Concurrent Validity

Hamilton Anatomy of Risk Management: Forensic Version (HARM-FV)

  Tool developed to assist with risk assessment and risk prediction at a clinical team level

  Focus on assessing change in dynamic risk factors and Risk Management.

HARM/HCR-20V3 SRR Correlations

Pearson’s r correlations for HARM and HCR 20 v3 SRRs HARM Immediate (days) with Professional Support

HARM Short-term (weeks) with Professional Support

HCR-20V3 1-month Risk for Violence/Case Priority .58** .62**

HCR-20V3 1-month Risk for Serious Physical Harm .50** .53**

HCR-20V3 1-month Imminent Risk for Violence .55** .55**

HCR-20V3 3-month Risk for Violence/Case Priority .54** .56**

HCR-20V3 3-month Risk for Serious Physical Harm .50** .52**

Note: N = 234; ** = p < 0.01; SRR = Summary Risk Ratings; SRR for the HARM and HCR-20V3 rated as high, moderate, low

HARM/HCR-20V3 Numerical Correlations

Pearson’s r correlations for HARM and HCR 20 v3 Total Scores HARM Historical HARM Dynamic HARM Total

HCR-20V3 Historical 0.35**

HCR-20V3 Dynamic 0.32**

HCR-20V3 Total .27**

Note: N = 234; ** = p < 0.01; HARM total score = sum of historical and dynamic risk factors, range -10 to 14; HCR-20V3 total score = sum of historical, clinical, and risk management factors, range 0 to 40.

Agreement

Kappa Coefficients for HARM and HCR 20V3 SRRs HARM Immediate (days) with Professional Support

HARM Short-term (weeks) with Professional Support

HCR-20V3 1-month Case Priority .42** .46**

HCR-20V3 3-month Case Priority .32** .40**

Note: N = 234; ** = p < 0.0005; SRR = Summary Risk Ratings; SRR for the HARM and HCR-20V3 rated as high, moderate, low

Agreement Across Risk Ratings

Taken Together

  Our results align with what others have found: moderate to good association between tools   Kropp et. al (2011): SAM and PCL:SV

  Beggs & Grace (2010): VRS: SO and Static 99

  Warren et. al. (2003): HCR-20 and PCL-R

Agreement: AIS and MOAS

  AIS scores were significantly and highly correlated with scores on MOAS

(r=0.92, p<0.01)

  Substantial Agreement found between AIS and MOAS when looking at whether an act of violence occurred in a follow up period

(K=.79, p<.0005)

HARM

Predictive Validity

Predictive Validity

  Much debate about how best to assess discrimination accuracy (2013 special issue Behavioral Sciences and the Law)

  We adopted two methods: Logistic regression and ROC   LR provides some insight into the relative

contribution of risk ratings to the prediction, as well as accuracy estimates (classification tables)

  ROC, and the AUC is typical method of evaluation and thus reported for comparative purposes

Predictive Validity

  SPJ and actuarial show comparable predictive validity (SPJ incrementally better)

  Average predictive validity ratings for inpatient violence risk:   HCR-20 (C score): AUCs .68 - .72/ (SRR): .75 -.92

  BVC: AUCs .77 - .83

  DASA: AUCs .76 - .83

(Chu, Daffern & Ogloff, 2013; Wilson et al., 2013)

Any violence HARM

Regressions: Any Violence Full 6mth FU

Trajectory Risk Rating B OR (95% CI)

Immediate With Support - -

Without Support 1.44* 4.22 (1.03,17.4)

Short-term With Support - -

Without Support 1.63 5.11 (.89,29.29)

Totals Historical - -

Dynamic - -

Combined Total - -

Regressions: Any Violence (SRR)

1 2 3 4 5 6

Trajectory Risk B OR B OR B OR B OR B OR B OR

Immediate

With Support

- - - - - - 1.53* 4.62 - - - -

Without Support

- - - - - - 2.08 7.97 2.32* 10.17 - -

Short- term

With Support

- - - - - - 1.58* 4.57 1.96* 7.10 - -

Without Support

- - - - - - - - - - 2.14 8.05

Totals Historical - - - - - - - - - - - -

Dynamic - - - - - - - - .40* 1.49 .25 1.28

Total - - - - - - - - 2.9 1.33 .27 1.31

HARM SRR AUCs: Any Violence

Trajectory Risk Full 6mth FU AUC (CI)

Immediate

With Support -

Without Support .72* (.53,.91)

Short-term

With Support -

Without Support .64 (.43,.86)

Follow-up Period AUC (CI)

Trajectory 1 2 3 4 5 6

Immediate

With Support

- - - .75* (.57,.93) - -

Without Support

.71(.54-.89) - - .67 (.49,.86) .72*(.54,.89) .64(.43,.85)

Short-term

With Support

- - - .75* (.57,.93) .78*(.62,.94) -

Without Support

- - - - .68(.54,.86) -

Totals Historical - - - - - -

Dynamic - - - - - .72 (.49,.95)

Total - - - - - .76 (.49,1.00)

HARM SRR AUCs: Any Violence

Longitudinal Logit Regressions: HARM Ratings & Any Violence

1 month FU


Short-Term With Support .63 (p = .09) 1.88 (.92, 3.87)

Without Support 1.53* 4.64 (1.07,20.04)

Totals Dynamic .06 (p = .41) 1.06 (.92,1.21)

Combined Total .06 (p = .34) 1.07 (.93,1.22)

*p < .05; **p < .01

Physical HARM

Regressions: Physical Violence Full 6mth FU


Immediate With Support - -

Without Support - -

Short-term With Support - -

Without Support - -

Totals Historical - -

Dynamic - -

Combined Total - -

Regressions: Physical Violence 1 2 3 4 5 6

Trajectory Risk B OR B OR B OR B OR B OR B OR

Immediate

With Support

- - - - - - - - 2.41* 11.15 (1.03,

121.07) - -

Without Support

- - - - - - - - - - - -

Short- term

With Support

- - - - - - - - 2.48* 11.97 (1.03,

139.52) - -

Without Support

- - - - - - - - - - - -

Totals Historical - - - - - - - - - - - -

Dynamic - - - - - - - - - - - -

Total - - - - - - - - - - - -

HARM AUCs: Physical Violence


Immediate

With Support -

Without Support -

Short-term

With Support -

Without Support -



Immediate

With Support

- - - - .86*

(.67-1.00) -

Without Support

- - - - - -

Short-term

With Support

- - - - .85*

(.64-1.00) -

Without Support

- - - - - -

Totals Historical - - - - - -

Dynamic - - - - .64

(.26-1.00) -

Total - - - - .57

(.19-.96) -

HARM AUCs: Physical Violence

Longitudinal Logit Regressions: HARM Ratings and Physical Violence

1 month FU


Immediate With Support .93 (p = .08) 2.53 (.90,7.13)

Without Support - -

Totals Historical -1.05* .35 (.13,.93)

Dynamic .20 (p = .08) 1.23 (.98,1.54)

Combined Total .16 (p =.17) 1.17 (.93,1.47)

*p < .05; **p < .01

Taken Together

  HARM SRR account for some significant variance in the prediction of any and physical violence toward the end of the follow-up period

  HARM SRR demonstrated good predictive accuracy

  Support for use of risk judgments rather than total scores (as by HARM design and implementation)

  We’d like to see improved reliability in our monthly predictions

HCR-20V3

Predictive Validity

Any violence HCR-20V3

Regressions: Any Violence Full 6mth FU

Trajectory Risk B OR (CI)

1-month


- -

Serious Physical Harm - -

Imminent Violence - -

3-month


- -

Serious Physical Harm 2.20* 9.03 (1.09,74.90)

H Scale .237* 1.27 (1.00,1.60)

C + R Scale - -

Total Score .270* 1.31 (1.06,1.60)

Regressions: Any Violence (SRR)

1 2 3 4 5 6

Trajectory

Risk B OR B OR B OR B OR B OR B OR

1-month


- - - - - - 1.23* 3.43 .84 2.85 1.51* 4.50


- - - - .88 2.42 - - - - - -

Imminent Violence

- - - - .94 2.56 2.91* 6.28 - - - -

3-month


- - - - - - 1.90* 6.16 - - - -


- - - - 1.00 2.72 1.19 - - - - -

1 2 3 4 5 6

B OR B OR B OR B OR B OR B OR

H Scale - - - - - - - - - - - -

C + R Scale

- - - - - - - - - - - -

Total Score - - - - - - - - - - .19* 1.21

Regressions: Any Violence (Numerical Score)

HCR SRR AUCs: Any Violence


1-month


-

Serious Physical Harm -

Imminent Violence -

3-month


-

Serious Physical Harm .74* (.57, .90)

* p < .05

HCR Numerical Score AUCs: Any Violence

Full 6mth FU AUC (CI)

H Scale .67 (.45,.88)

C + R Scale -

Total Score .81* (.67,.94)



1-month


- - - .68

(.48,.88) .67

(.49,.86) .71*

(.53,.89)


- - .58

(.33,.83) - - -

Imminent Violence

- - .62

(.37,.87) .75*

(.56,.93) - -

3-month


- - - .77*

(.61,.92) - -


- - .60

(.34,.86) .68

(.49,.88) - -

HCR SRR AUCs: Any Violence

HCR Numerical Score AUCs: Any Violence

Follow-Up Period AUC (CI)

1 2 3 4 5 6

H Scale - - - - - -

C + R Scale - - - - - -

Total Score - - - - - .73* (.54,.92)

Longitudinal Logit Regressions: HCR-20V3 Ratings & Any Violence

1 month FU


1-month


.30 (p = .31) 1.35 (.76,2.41)


.33 (p = .68) 1.14 (.60,2.17)

Imminent Violence .42 (p = .22) 1.52 (.78,2.97)

H Scale .18 (p =.08) 1.20 (.98,1.47)

C + R Scale .12* 1.12 (1.02,1.23)

Total Score .15** 1.16 (1.06,1.27)

*p < .05; **p < .01

Physical HCR-20V3

Regressions: Physical Violence (SRR & Numerical Score)

Full 6mth FU


1-month


1.23 3.4 (.56,20.74)


2.50* 12.20 (1.23,121.23)

Imminent Violence - -

3-month


- -

Serious Physical Harm 2.70* 15.35 (1.37,172.67)

H Scale - -

C + R Scale - -

Total Score - -

Regressions: Physical Violence (SRR)

1 2 3 4 5 6

Trajectory

Risk B OR B OR B OR B OR B OR B OR

1-month


- - - - - - - - - - - -


- - - - - - - - - - - -

Imminent Violence

- - - - - - - - - - - -

3-month


- - - - - - - - - - - -


2.09* 8..08 - - - - - - - - - -

1 2 3 4 5 6

B OR B OR B OR B OR B OR B OR

H Scale - - - - - - - - - - - -

C + R Scale - - - - - - - - - - - -

Total Score - - - - 0.20* 1.23 - - - - - -

Regressions: Physical Violence (Numerical Scores)

HCR SRR AUCs: Physical Violence


1-month


.69 (.28,1.00)

Serious Physical Harm .89* (.75,1.00)

Imminent Violence -

3-month


-

Serious Physical Harm .91* (.77,1.00)

HCR Numerical Score AUCs: Physical Violence

Full 6mth FU AUC (CI)

H Scale -

C + R Scale -

Total Score -



1-month


- - - - .90*

(.79,1.00) -


- - 1.00*

(1.00,1.00) - .98*

(.94,1.00) -

Imminent Violence

- - .97*

(.91,1.00) - 1.00*

(1.00,1.00) -

3-month


- - - - .92*

(.82,1.00) -


.93* .(81,1.00) -

.98* (.94,1.00) -

.98* (.94,1.00) -

HCR SRR AUCs: Physical Violence

HCR Numerical Score AUCs: Physical Violence

Follow-Up Period AUC (CI)

1 2 3 4 5 6

H Scale - - - - - -

C + R Scale - - - - - -

Total Score - - .70

(.26,1.00) - - -

Longitudinal Logit Regressions: HCR-20V3 Ratings Physical Violence

1 month FU


1-month


1.39** 4.03 (1.63,9.95)


1.68** 5.37 (2.57,11.25)

Imminent Violence 1.73** 5.23 (2.67,11.85)

H Scale .03 (p = .82) 1.03 (.80,1.32)

C + R Scale .27** 1.31 (1.09,1.58)

Total Score .18* 1.19 (1.03,1.38)

*p < .05; **p < .01

Taken Together: HCR

Any Violence • Support for use of risk judgments rather than total scores

(as by HCR-20V3 design and implementation) • Longitudinal results indicate Dynamic Factors and Total

score as predictive, but overall no consistent results

Physical Violence • SSR of serious physical harm predictive over 6-month FU • Support for use of risk judgments rather than total scores

(as by HCR-20V3 design and implementation) • Longitudinal results indicate SRR and Dynamic Factors as

predictive for physical violence

Conclusions

• Moderate support for HARM/HCR-20V3 • Strong support for the AIS/OAS

Concurrent Validity

• Variable predictive validity across both measures • SRR for both tools performing better than numerical scores

Predictive Validity

• Prediction of change scores • Role of management in mediating violence risk

Future Directions

Contact Information

Alana N. Cook [email protected]

Mini Mamak [email protected]

Gary Chaimowitz [email protected]

Heather Moulden [email protected]

Dynamic risk among Canadian psychiatric inpatients: Validity and reliability of the HARM-20V3 and...

Documents

Transcript of Dynamic risk among Canadian psychiatric inpatients: Validity and reliability of the HARM-20V3 and...