Psychological Assessment1998, Vol. 10, No. 4, 435-443

Copyrighl 1998 by the American Psychological AMOCiadon, Inc.1040-3590/98/J3.00

Psychometric Properties of the MacArthur CompetenceAssessment Tool-Criminal Adjudication

Randy K. Otto and Norman G. PoythressUniversity of South Florida

Robert A. NicholsonUniversity of Tulsa

John F. EdensUniversity of South Florida

John Monahan, Richard J. Bonnie,Steven K. Hoge, and Marlene Eisenberg

University of Virginia

This article describes the development of a new clinical instrument for use in assessments of adult

criminal defendants' competence to proceed to adjudication, the MacArthur Competence Assessment

Tool-Criminal Adjudication (MacCAT-CA). The MacCAT-CA was derived from a more comprehen-

sive research instrument (MacArthur Structured Assessment of Competencies of Criminal Defendants;

Hoge, Bonnie, Poythress, Monahan, & Eisenberg, 1997) on the basis of considerations efface validity

for use in legal contexts, psychometric analyses, and advice from mental health experts who reviewed

an earlier prototype. This article presents the results from an National Institute of Mental Health-

sponsored validation study that investigated the psychometric properties of the MacCAT-CA.

At least 25,000 criminal defendants are referred annually for

evaluation of their competence to participate in legal proceed-

ings (Steadman & Hartstone, 1983). Although a number of

measures have been specifically designed to assess defendants'

capacities in this area (e.g., Competence Screening Test [CST],

Lipsitt, Lelos, & McGarry, 1971; Georgia Court Competency

Test [GCCT], Wildman et al., 1980; CADCOMP, Barnard et

al., 1991; Competency Assessment Instrument [CAI], Labora-

tory of Community Psychiatry, 1974; Interdisciplinary Fitness

Interview [IFI], Golding, Roesch, & Schreiber, 1984; Fitness

to Stand Trial Interview, Menzies, Webster, Roesch, Jensen, &

Randy K. Otto, Norman G. Poythress, and John F. Edens, Department

of Mental Health Law and Policy, University of South Florida; Robert

A. Nicholson, Department of Psychology, University of Tulsa; John

Monahan and Richard J. Bonnie, School of Law, University of Virginia;

Steven K. Hoge, School of Law and School of Medicine, University of

Virginia; Marlene Eisenberg, Institute of Law, Psychiatry, and PublicPolicy, University of Virginia.

John F. Edens is now at the Department of Psychology, Sam Houston

State University.

This research was supported by the MacArthur Foundation Research

Network on Mental Health and the Law, and by Grant RO1MH54517-

O1A1 from the National Institute of Mental Health. We are grateful to

Tom Feucht-Haviar for his assistance on a pilot test of this measure.

Norman G. Poythress, John Monahan, Richard J. Bonnie, and Steven

K. Hoge are authors of the MacArthur Competence Assessment Tool-

Criminal Adjudication (MacCAT-CA) published by Psychological As-

sessment Resources. Randy K. Otto, Norman G. Poythress, Robert A.

Nicholson, John F. Edens, John Monahan, Richard J. Bonnie, and Steven

K. Hoge are authors of the MacCAT-CA manual, also published byPsychological Assessment Resources.

Correspondence concerning this article should be addressed to Randy

K. Otto, Department of Mental Health Law and Policy, Florida Mental

Health Institute, University of South Florida, 13301 Bruce B. DownsBoulevard, Tampa, Florida 33612-3899. Electronic mail may be sent to

orto@hal.fmhi.usf.edu.

Eaves, 1984), each exhibits various limitations (Grisso, 1991;

Melton, Petrila, Poythress, & Slobogin, 1997; Nicholson, 1992).

Briefly, some measures were designed to serve primarily as

screening instruments (e.g., CST and GCCT) and appear mainly

to be collections of items having various legal content but little

or no underlying conceptual structure. Others (e.g., CAI and IFI)

lack standardized administration and criterion-based scoring,

relying instead on the judgment of the clinician to determine,

on a case-by-case basis, which questions to pose to the defendant

and how to rate the responses received. None is designed to

provide quantitative indexes of discrete competence-related abil-

ities, nor have interpretative norms based on large, national

samples been developed for any of these measures.

Given the pivotal role of a defendant's competence, one of the

major research initiatives of the MacArthur Foundation Research

Network on Mental Health and the Law was the development

of a standardized research instrument for evaluating criminal

defendants' psycholegal abilities related to competence to pro-

ceed to adjudication. The MacArthur Structured Assessment

of Competencies of Criminal Defendants (MacSAC-CD; Hoge,

Bonnie, Poythress, Monahan, & Eisenberg, 1997) was based on

Bonnie's (1992, 1993) comprehensive theory of legal compe-

tence, and it contains measures of discrete competence-related

abilities. Items in the MacSAC-CD were modeled after those

created by Grisso, Appelbaum, Mulvey, and Fletcher (1995) for

the assessment of competence to consent to treatment (see, more

generally, Grisso & Appelbaum, 1998). Each measure in the

MacSAC-CD involves standardized administration and crite-

rion-based scoring, thus reducing much of the discretionary

(and potentially idiosyncratic) scoring that plagues existing

measures.

Hoge et al. (1997) recently described the MacSAC-CD

and presented the primary findings from a field study that in-

volved both incompetent and competent defendants as research

participants. The results of this field study indicated that the

435

436 OTTO ET AL.

MacSAC-CD had satisfactory psychometric properties, con-

struct validity, and potential classification utility for research

applications. Significant differences in the expected direction

between mean scores of hospitalized incompetent (HI), jail

unscreened (JU), and jail treated (JT; i.e., jail inmates who

were receiving mental health services but whose competence

was not raised as an issue) participants were obtained on each

measure. Within a subset of the HI group, the measures were

sensitive to changes in the patients' or defendants' clinical condi-

tion; reassessment of those patients or defendants who had been

deemed clinically restored to competence revealed that scores

on the MacSAC-CD increased, reflecting gains in competence-

related abilities. Although the primary field study used only

male defendants (W = 366), a second study involving 106

female defendants produced similarly encouraging findings con-

cerning the validity of the MacSAC-CD (Poythress, Hoge, et

al., 1998).

Having demonstrated the potential utility of the MacSAC-CD

for the assessment of competence-related abilities, the network's

next priority was to develop a clinically portable measure for

actual use in clinical practice. The field study revealed that

administration time for the MacSAC-CD (approximately 1.5 to

2 hours) exceeded considerably that required by other currently

available measures; further, the need to reduce the length of the

measure was indicated by some duplication in scoring proce-

dures and redundancy across separate measures (see Hoge et

al., 1997). In light of these considerations, and keeping in mind

the importance of face validity for application in the legal con-

text, the MacSAC-CD, an instrument comprising 7 measures

containing 47 items, was streamlined to one comprising 3 mea-

sures containing 22 items. Decisions about item retention were

informed by an examination of (a) the item-total correlations

for each item with its original MacSAC-CD measure, and (b)

the impact on coefficient alpha for the MacSAC-CD measure if

the item was removed.

The resulting 22-item clinical instrument, the MacArthur

Competence Assessment Tool-Criminal Adjudication (Mac-

CAT-CA; Poythress, Nicholson, et al., in press) evaluates three

discrete competence-related abilities: understanding (the ability

to understand general information related to the law and adjudi-

catory proceedings), reasoning (the ability to discern the poten-

tial legal relevance of information, and capacity to reason about

specific choices that confront a defendant in the course of adju-

dication), and appreciation (rational awareness of the meaning

and consequences of the proceedings in one's own case). These

abilities are assessed through three separate measures on the

MacCAT-CA. The content of both the Understanding (eight

items) and Reasoning (eight items) measures refers to a hypo-

thetical legal scenario in which a defendant is charged with

aggravated assault. Questions comprising the Appreciation mea-

sure (six items) refer specifically to the examinee's own case.

Items are scored on a 0-2 scale, with higher scores indicative

of higher levels of capacity. The specific legal content of each

item is summarized in the Appendix. The MacCAT-CA was

reviewed by experienced forensic clinicians for ease of adminis-

tration and was pilot tested to determine administration time,

which ranges from 25 to 55 minutes. Because the MacCAT-CA

was not normed on persons with IQ estimates below 60 (see

below) it is not recommended for use with this population.

From 1996 to 1998, we conducted a multistate study, funded

by the National Institute of Mental Health, to establish the clini-

cal utility of the MacCAT-CA. In this article, we report the

findings regarding the psychometric properties and construct

validity of the instrument. Our clinical protocol, comprising the

MacCAT-CA and other measures relevant to the investigation of

the validity of the MacCAT-CA, was administered to 729 pre-

trial, fekmy defendants in eight states.

The selection of states as potential sites for data collection

was informed by two considerations. First, state systems for

providing forensic evaluation services vary considerably in their

organizational structure. For example, the typology of forensic

service delivery systems proposed by Grisso, Cocozza, Stead-

man, Fisher, and Greer (1994) consisted of five system types

(i.e., traditional inpatient, modified-traditional, community-

based, private practitioner, and mixed evaluation systems) as

well as a ' 'not classified'' category. Second, states differ in the

presence or absence of specific training requirements for foren-

sic examiners and in the availability of state-sponsored training

programs designed to meet those training requirements (Farkas,

DeLeon, & Newman, 1997). Both of these variables-—organiza-

tional structure for forensic service delivery and the existence

of state-sponsored forensic training—could influence the

thresholds for determining incompetence that are applied across

states. Therefore, gathering data from each system type and

from states with and without forensic examiner training was

necessary to inform judgments about the generalizability of

findings on the basis of the MacCAT-CA. To provide representa-

tive national data, we selected individual states that represented

each of the six categories identified by Grisso et al. (1994). In

addition, where possible, within types of service delivery sys-

tems, states with and without forensic examiner training were

selected.

Method

Eligibility Criteria ami Recruitment Procedures

Data were collected- from three groups of felony defendants: (a)

defendants admitted to forensic psychiatric units after being adjudicated

incompetent to proceed (HI); (b) defendants in jail who were receiving

treatment for mental health problems but who were presumed competent

(JT); and (c) randomly selected jail inmates who were presumed com-

petent (JU). Participants were recruited from six states representing

categories in the Grisso et al. (1994) typology: Washington (tradi-

tional); Louisiana (private practitioner); Oklahoma (community-

based); South Carolina (modified-traditional); Wisconsin (mixed); and

Utah (not classified). According to the survey by Farkas et al. (1997),

none of these six states reported the availability of extensive, state-

sponsored forensic evaluator training programs. Therefore, two addi-

tional states (Michigan [modified-traditional] and Alabama [commu-

nity-based]) that provide systematic training for forensic examiners also

were included.

Both male and female felony defendants were eligible for participation

in the study; participants were paid $10 for their time. Potential partici-

pants had to be English-speaking, between the ages of 18 and 65, without

a diagnosis suggesting organicity, and they had to have a prorated Wechs-

ler Adult Intelligence Scale-Revised (WAIS-R; Wechsler, 1981) Full

Scale IQ of 60 or greater. For the HI sample, participants were recruited

within 14 days of hospital admission to minimize the effects of treatment

on their performance on the research measures. All but 4(1%) partici-

pants were assessed within 10 days of admission.

Potential HI participants were identified as they entered the forensic

hospital for competence restoration treatment; essentially all eligible

MAcCAT-CA 437

participants were approached by research assistants. Jail inmate partici-

pants were recruited from lists of defendants provided by cooperating

public defender offices in each state. Jail inmates receiving mental health

services (potential JT participants) were identified by the research assis-

tant with the assistance of jail mental health staff and randomly selected

for invitation to participate in the study. The remaining inmates were

randomly selected for participation as members of the JU group.

Potential participants were approached by a research assistant and

informed about the study using informed consent procedures approved

by a university institutional review board. Those expressing an interest

in participating were screened by the research assistant with respect to

their competence to consent to the research project and their cognitive

functioning (see below). After participants completed the research pro-

tocol, $10 was placed in their institutional accounts. Refusal rates for

the HI, JT, and JU samples were 22%, 9%, and 9%, respectively. Whereas

the 22% refusal rate is higher than desirable, it is lower than is often

found in studies of acutely mentally ill people. For example, in studies

of competence to consent in civil contexts (consent to voluntary hospital-

ization; consent to treatment), Appelbaum, Mirkin, and Bateman (1981)

and Grisso, Appelbaum, and Hill-Fbtouhi (1997) reported refusal rates

of 33% and 43%, respectively.

Measures

Measures gathered on each participant consisted of historical and

demographic information, estimated IQ, measures of current adjustment

and psychopathology, the MacCAT-CA, and a treating clinician's global

estimate of competence to proceed (HI participants only). Measures

were administered in the order listed below.

Estimate of cognitive-intellectual functioning. To estimate partici-

pants' cognitive abilities and screen out potential participants with mark-

edly limited abilities, we administered to all potential participants the

Information and Picture Completion subtests of the WAIS-R. Perfor-

mance on these two scales is highly correlated with WAIS-R Rill Scale

IQ scores (r = .86 and r = .92, respectively; Kaufman, Ishikuma, &

Kaufman-Packer, 1991). Because cognitive functioning can be impaired

by underlying mental disorder, this score is best considered as an index

of current functioning rather than a measure of baseline intellectual

capacity. Thirty-six potential HI participants were excluded because they

obtained WAIS-R Rill Scale IQ estimates of below 60, whereas 8 poten-

tial JT participants and 3 potential JU participants were excluded on

these grounds.

Background interview data. Background information on the partici-

pants was gathered through a brief structured interview. Information

obtained included history of psychiatric treatment, arrest history, educa-

tional level, and highest socioeconomic status achieved.

Brief Psychiatric Rating Scale-Anchored (SPSS). Scored after the

administration of a 15-minute interview, the BPRS (Overall & Gorham,

1962) provides a reliable and valid estimate of the presence and severity

of psychopathology. We used the anchored version (Woerner, Man-

nuzza, & Kane, 1988) in which the severity of 18 symptoms of mental

disorder (e.g., depression and orientation) are rated on a 7-point, an-

chored Likert scale. Ratings on the 18 items are summed to provide a

global measure of current psychopathology, and four subscale scores

that provide indices of psychoticism, depression, emotional withdrawal,

and hostility can be calculated (Overall & Porterfield, 1963).

MMPI-2 Psychoticism scale. The 25-item Psychoticism scale was

derived from the MMPI-2 item pool. Based on a five-factor model of

personality especially relevant for pathological populations (Harkness &

McNulty, 1994), the Psychoticism scale measures the gross degree of

correspondence between an individual's internal models of reality and

the external social and physical world (Harkness, McNulty, & Ben-

Porath, 1995). Estimates of internal consistency (coefficient alpha) rang-

ing from .70 to .84 have been reported across diverse nonclinical and

clinical samples. The scale also yields appropriate mean differences

across these samples, as well as expected patterns of correlations with

Table 1

Demographics by Group

Variable

AgeMSD

SexMale (%)Female (%)

EthnicityWhite (%)Non-White (%)

Education

MSD

SESM

SD

JU

31.339.18

93.96.1

35.065.0

11.261.98

56.1010.96

JT

32.469.35

88.811.2

50.2

49.8

11.352.51

56.2512.45

HI

36.0110.66

89.410.6

49.850.2

11.44

2.68

57.9413.13

Note. JU = unscreened jail inmates; JT = jail inmates receiving mentalhealth services; HI = hospitalized incompetent defendants; SES = socio-economic status.

measures of theoretically relevant and divergent constructs. This measure

was added to the research protocol after data collection already had

been initiated; nevertheless, data were available for 89% (n = 647) of

the participants.

MacCAT-CA. The MacCAT-CA, the clinical instrument derived

from the MacSAC-CD research instrument, was administered to all

participants.

Chan data. Relevant background information on the participants

was gathered, in part, by way of institutional chart review. A review of

the hospital chart or jail records provided information on participants'

age, race, admission date, and current psychiatric status including diag-

nosis, treatment status, and medications.

Clinical judgment regarding competence. After HI participants

completed the research protocol, a hospital mental health professional

familiar with the participant's mental state and adjustment offered a

global judgment of the participant's competence to proceed using an

anchored 6-point Likert scale (1 = clearly incompetent, 6 = clearly

competent}.

Results

Demographic Data

Table 1 provides descriptive information about the hospital-

ized and jailed defendant samples recruited for this study (for

ffl, n = 283; for JT, n = 249; for JU, n = 197).' Participants

were predominantly male (90%), and gender was distributed

similarly across the HI, JT, and JU samples, x2(2, N = 729)

= 3.90, p = .14. Although there was a balance of White and

non-White participants in the study, they were somewhat dispro-

portionately distributed across the three samples, X2(2, N =

729) = 12.99, p < .002. HI participants as a group were signifi-

cantly older than participants in the JU sample, f(724) = 25.01,p < .001, and the JT sample, f(724) = 16.40, p < .001.

The mean years of education reported by participants was

11.36 (SD = 2.45), and level of education did not vary signifi-

1 Different degrees of freedom for the analyses that follow reflect

missing data on these variables for a small number of participants.

438 OTTO ET AL.

Table 2

Mental Status Characteristics by Group

JU JT HI

Variable M SD M SD M SD Univariate F df

Estimated WAIS-R Full Scale IQ 85.23 11.99 85.49 14.61 83.21 14.05 2.17 2,726

MMPI-2 Psychoticism 63.32 14.13 70.48 18.40 74.68 19.64 20.93* 6,646BPRS total 29.16 6.99 35.95 9.07 38.80 10.04 68.26* 2,726BPRS Psychoticism 3.41 1.16 4.% 2.61 7.55 3.72 131.96* 2,722BPRS Depression 8.70 3.96 10.76 4.55 8.20 3.99 24.48* 2,722BPRS Hostility 5.00 2.26 6.15 2.69 6.74 3.09 23.53* 2,722BPRS Emotional Withdrawal 3.91 1.50 4.62 2.33 5.40 2.95 22.27* 2,722

Note. JU = unscreened jail inmates; JT = jail inmates receiving mental health services; HI = hospitalized

incompetent defendants; WAIS-R = Wechsler Adult Intelligence Scale-Revised; MMPI-2 = MinnesotaMultiphasic Personality Inventory-2; BPRS = Brief Psychiatric Rating Scale-Anchored.

"p < .001.

cantly across groups. Using Hollingshead and Redlich's (1958)

two-factor index of social position, participants identified the

highest level of employment ever held. The mean scores (see

Table 1) equate with Social Class 4 (e.g., clerical and sales

workers, technicians). The group means did not differ signifi-

cantly in terms of highest socioeconomic status attained by

participants.

The participants in the various samples reported similar

amounts and types of involvement with the criminal justice

system. The mean number of estimated misdemeanor and felony

arrests for the total sample was 5.24 (SD = 9.52) and 1.91 (SD

= 2.79), respectively.2 There were no significant differences

between the groups in terms of their estimated number of felony

arrests, F(2,721) = .35, p = .71, or misdemeanor arrests, F(2,

714) = 1.11, p = .33.

Clinical Measures

Group characteristics on measures of psychopathology and

cognitive functioning are summarized in Table 2. The three

samples did not differ in terms of their estimated current cogni-

tive-intellectual functioning. There was a significant group ef-

fect as measured by the MMPI-2 Psychoticism scale, with the

HI sample demonstrating greater levels of psychopathology than

both the JT, r(646) = 2.55, p = .011, and JU, t(646) = 6.46,

p < .001, samples. Similarly, participants in the HI sample

obtained higher scores on the BPRS total score than the JT and

JU samples, rs(726) = 3.65 and 11.58, respectively, as well as

higher scores on the BPRS Psychoticism, fs(722) = 10.49 and

15.74, Hostility, is(722) = 2.61 and 6.85, and Emotional With-

drawal, B(722) = 3.69 and 6.61, subscales, respectively (all

ps < .01). Similar to findings in the MacSAC-CD field study

(Hoge et al., 1997), HI participants were less depressed than

JT participants, f(722) = 6.75, p < .001, but did not differfrom JU participants, r(722) = 1.30, p = .19.

MacCAT-CA Performance and Comparisons

Internal consistency. Cronbach's alpha and the mean and

range of interitem correlations for each of the MacCAT-CA

measures are provided in Table 3. These values serve as indices

of scale reliability and item homogeneity, respectively (Fiske,

1971). The alphas ranged from .81 (Reasoning) to .85 (Under-

standing) to .88 (Appreciation), indicating good internal consis-

tency for these measures. These estimates are comparable to

those obtained using the MacSAC-CD research measure (Hoge

et al., 1997). The mean interitem correlations were .36, .42, and

.54 for Reasoning, Understanding, and Appreciation, respec-

tively, indicating appropriate homogeneity of item content for

all three measures. Although the Appreciation measure has the

fewest items, its relatively greater item homogeneity enabled it

to produce the highest estimate of scale reliability.

Interrater reliability. Tb evaluate interrater reliability, we

drew a sample of 48 protocols from the database (2 HI, 2 JT,

and 2 JU from each state). Scoring assigned by the original

research assistant who completed the protocol was removed,

and 42 protocols were mailed to each research assistant for

rescoring.3 One protocol was not returned; hence, the interrater

reliability analysis was based on 47 total cases scored by each

of the eight research assistants. Three analyses of variance

(ANCTVAs) were conducted on the total scores from the Under-

standing, Reasoning, and Appreciation measures, respectively,

to compute an intraclass correlation for each measure. In each

of the ANOVks, rater was treated as a random variable (see

Shrout & Fleiss, 1979, Model 2). As several authors have em-

phasized, the intraclass correlation is superior to the traditional

product-moment correlation as an index of reliability (e.g., see

Bartko & Carpenter, 1976). Interscorer reliability for the three

measures as estimated by this procedure ranged from very good

to excellent, with intraclass R = .75 for Appreciation, .85 for

Reasoning, and .90 for Understanding (see Table 4). Although

explicit scoring guidelines are provided for each item on each

measure, the scoring criteria are more rigorous for the Under-

standing items, and less so for the Appreciation measure (where

a judgment about the plausibility of a defendant's "reasons" is

required). The pattern of intraclass correlations for the three

2 One participant who reported 1,000 misdemeanor arrests was de-

leted from this analysis.3 Research assistants did not rescore the six protocols they originally

completed that were subsequently mailed to the remaining research assis-

tants. Therefore, only 42 new protocols were mailed to them.

MACCAT-CA 439

Table 3

Internal Consistency ofMacArthur Competence Assessment

Tool— Criminal Adjudication Measures

Interitem correlation

Measure M Range

UnderstandingReasoningAppreciation

.85

.81

.88

.42

.36

.54

.31 -.53

.21 -.53

.4S-.62

measures is consistent with the degree of rigor in the scoring

guidelines.

Correlations Between MacCAT-CA Scales and Clinical

Measures

Support for the construct validity of the MacCAT-CA is found

in the pattern of correlations between MacCAT-CA measures

and select clinical variables. Ideally, measures of competence-

related abilities should correlate positively with estimated IQ,

but they should correlate negatively with measures of psychopa-

thology, particularly with measures of psychoticism. As revealed

in Table 5, this is the pattern of relationships obtained in the

present study, providing evidence of convergent validity for the

MacCAT-CA.

Consistent with the findings of Hoge et al. (1997) with the

research instrument (MacSAC-CD), the MacCAT-CA measures

correlate negatively with disturbed thinking as measured by the

Table 4

Interrater Reliability ofMacArthur Competence Assessment

Tool-Criminal Adjudication Measures

Measure/Item no. ICC

Understanding1234567g

Reasoning9

10111213141516

Appreciation171819202122

.90

.82

.88

.80

.88

.82

.92

.90

.86

.85

.93

.75

.87

.34

.90

.74

.72

.45

.75

.73

.66

.57

.70

.53

.42

Table 5

Correlations Between MacArthur Competence Assessment

Tool-Criminal Adjudication and Clinical Measures

Variable Understanding Reasoning Appreciation

Estimated WAIS-RFull Scale IQ

MMPI-2 PsychoticismBPRS TotalBPRS Psychoticism

BPRS DepressionBPRS HostilityBPRS Emotional

WithdrawalClinician Ratings of

Competence'

.41-.33-.23-.40

.18-.01

-.34

.36

.34-.31-.29-.48

.18-.10

-.27

.42

.14-.21

-.36-.52

.18-.28

-.19

.49

Note. WAIS-R = Wechsler Adult Intelligence Scale-Revised; MMPI-2 = Minnesota Multiphasic Personality Inventory-2; BPRS = BriefPsychiatric Rating Scale-Anchored, p < .001 for all correlations exceptHostility-Understanding correlation (p = .71) and Hostility-Reasoningcorrelation (p = .007)." Correlations for clinician ratings are based only on data from the 283

hospitalized incompetent participants.

BPRS Psychoticism subscale and MMPI-2 Psychoticism scale,

and with psychopathology generally as measured by the BPRS

total score. Present cognitive ability, as measured by perfor-

mance on select subtests of the WMS-R, correlated positively

with performance on the MacCAT-CA. Similar to results ob-

tained with the research version of the MacCAT-CA (the Mac-

SAC-CD; Hoge et al., 1997), presumed competent jail defen-

dants showed higher levels of depression as measured by the

BPRS than hospitalized incompetent defendants.

Global ratings of the HI participants' competence to proceed

were offered by forensic clinicians who were knowledgeable

about their clinical conditions. As expected, clinicians' ratings

of competence were moderately correlated with performance on

the MacCAT-CA (Understanding, r = .36; Reasoning, r = .42;

Appreciation, r = .49) presumably due, in part, to the restriction

in range (i.e., only hospitalized incompetent defendants received

such clinician ratings). These results provide evidence of con-

current validity for the MacCAT-CA.

Sample differences. Table 6 provides the mean and the stan-

dard deviation of each sample for each of the three MacCAT-CA

psycholegal ability measures: Understanding, Reasoning, and

Appreciation. These means differed significantly in a multivari-

ate analysis of covariance (MANCOV\) in which type of pre-

trial forensic evaluation service delivery system and mandated

training requirements were entered as covariates, multivariate

F(18, 2,160) = 9.79, p < .001." For each measure, planned

orthogonal comparisons revealed that the HI sample scored sig-

nificantly lower (i.e., were more impaired regarding their cam-

Note. ICC = intraclass correlation.

' Although these two variables are important to assess the generaliz-

ability of MacCAT-CA scores, they are potential confounds that might

obscure within-state group differences (HI vs. JT and HI vs. JU) across

states with differing organizational structures and training requirements.

It should be noted, however, that not including these covariates in this

analysis resulted in essentially similar results, with defendants in the HI

group obtaining significantly lower scores than those in the JT and JU

groups across all three MacCAT-CA measures.

440 OTTO ET AL.

Tible 6

MacArthur Competence Assessment Tool-Criminal Adjudication Descriptive Statistics by Group and

Multivariate Analysis of Covariance (MANCOVA) Results

JU(n = 197)

Measure

UnderstandingReasoning

Appreciation

M

12.5013.27

11.44

SD

3.082.641.01

IT(n = 249)

M

12.56

12.9011.02

SD

3.252.901.63

HI(n = 283)

M

9.119.337.89

SD

4.194.314.01

HI(R)(n = 90)

M

10.7811.149.87

SD

3.41

3.753.00

HI(C)(n = 170)

M

8.14

8.236.58

SD

4.174.234.08

Univariate F'

(2, 720)

79.1295.92

121.07

Univariate F"

(3,696)

73.3588.27

129.08

Note. Clinician ratings of competence were unavailable for 23 hospitalized incompetent (M) defendants, who were not included in the secondMANCOVA. JU = unscreened jaU inmates, JT = jail inmates receiving mental health services; HI(C) = clinically affirmed incompetent ffl defendants;ffl(R) = residual HI defendants. All F values significant at p < .001.• Univariate F for MANCOVA comparing JU, JT, and HI samples. " Univariate F for MANCOVA comparing JU, JT, HI(R), and HI(C) samples.

petence-related abilities) than the JU sample [Understanding,

((720) = 11.37; Reasoning, ((720) = 12.72; Appreciation,

((720) = 13.74, allps < .001]. The planned comparisons con-

trasting the means from the HI and JT samples provide a more

stringent test of group discrimination. As expected, and lending

support for the claim that the MacCAT-CA can discriminate

between competent and incompetent mentally ill defendants, the

HI group mean was significantly lower than the JT group mean

for each MacCAT-CA measure [Understanding, ((720) = 10.04;

Reasoning, ((720) = 10.66; Appreciation, /(720) = 12.84, all

ps < .001].

This MANOTvA was repeated after separating the HI sample

into two groups on the basis of clinicians' ratings of compe-

tence. The HI(C) sample comprised those HI participants who

were clinically confirmed as incompetent on the basis of inde-

pendent ratings provided by a treatment-team clinician at the

time that the research protocol was administered. The HI(R)

sample comprised the residual HI participants whose clinician

ratings reflected a clinical impression that the individuals were

at least marginally competent at the time of protocol administra-

tion. A significant multivariate effect was obtained, F(18,

2,088) = 9.82, p < .001, for this analysis. Planned comparisons

indicated that the HI(C) sample obtained lower mean scores

on each of the MacCAT-CA measures when compared to the

HI(R) group, rs(696) = 6.05, 6.98, and 9.93 for Understanding,

Reasoning, and Appreciation, respectively; all ps < .001. Fur-

• thermore, despite obtaining higher scores than the HI(C) sam-

ple, the HI(R) sample obtained lower mean scores on each of

the MacCAT-CA measures when compared with both the JU

sample, (s(696) = 5.19, 5.53, and 4.70 for Understanding, Rea-

soning, and Appreciation, respectively; all ps < .001, and the

JT sample, rs(696) - 3.88, 3.66, and 3.62 for Understanding,

Reasoning, and Appreciation, respectively; all ps < .001, sug-

gesting greater impairment in competence-related abilities (see

Table 6).

We note that we did not include race as a covariate in these

analyses, although race does have a statistically significant bivar-

iate correlation with Understanding (—.15) and Reasoning

(—.09).' Partial correlations, controlling for cognitive function-

ing (i.e., prorated IQ scores), indicated nonsignificant correla-

tions between race and MacCAT-CA performance. Thus, we

attribute the difference in MacCAT-CA scores between White

and non-White participants to differences in current cognitive

functioning.

The preceding analyses documented the statistical signifi-

cance of the differences between groups of participants on the

MacCAT-CA measures. Table 7 provides information regarding

the magnitude of those between-group differences. The measure

of effect size used to estimate group differences was Cohen's

(1977) d, which is defined as follows:

d = -

where Xi and X2 are the means of the two groups being com-

pared, and s is the pooled within-group standard deviation. Thus,

Cohen's d expresses the difference between means relative to

within-group variation. As the first two columns of Table 7 show,

comparisons involving all hospitalized incompetent defendants

yielded effect sizes of approximately 1 SD across the three

MacCAT-CA measures. Further, as can be seen in the third and

fourth columns of the table, substantially larger effect sizes were

obtained from comparisons involving the clinically affirmed

incompetent defendants. In every case, the between-group dif-

ferences exceeded 1 SD, ranging from almost 1.25 to more than

1.5 SDs. In contrast, more moderate effects were observed in

comparisons involving incompetent defendants who were not

clinically affirmed as such. The latter effect sizes clustered

around one half of a standard deviation.

Discussion

We have described the derivation of a new clinical tool for

use in the assessment of criminal defendants' adjudicative com-

petence, the MacCAT-CA. We also have presented research

findings concerning its psychometric properties and potential

clinical utility in evaluating psycholegal abilities relevant tocompetence determinations. These results suggest that the Mac-

CAT-CA measures of Understanding, Reasoning, and Apprecia-

tion have good interrater reliability (intraclass Rs ranging from

.75 to .90) and strong internal consistency (as > .80, mean

5 It is noted, however, that race only accounted for a small percentageof the variance associated with the Understanding (2%) and Reasoning(1%) scores.

MACCAT-CA 441

Table 7

Effect Sizes (Cohen's d)for Comparisons Between Samples of Jailed Defendants

and Hospitalized Incompetent Defendants

Comparison

Measure JU-ffl JT-HI JU-HI(C) JT-HI(C) JU-Hl(R) JT-HI(R)

UnderstandingReasoningAppreciation

.901.061.13

.91

.961.00

1.201.451.69

1.211.331.54

.54

.70

.84

.54

.56

.55

Note. JU = unscreened jail inmates; JT = jail inmates receiving mental health services; HI = hospitalizedincompetent defendants; HI(C) = clinically affirmed incompetent HI defendants; HI(R) = residual HIdefendants.

interitem rs ranging from .36 to .54), as well as evidence of

construct validity (expected patterns of correlation with mea-

sures of cognitive ability, psychopathology, and clinical judg-

ments of degree of impaired competence).

The psychometric properties of the MacCAT-CA scales ob-

served in this study compare favorably with those reported for

other measures of adjudicative competence, such as the 22-item

CST (Lipsitt et al., 1971) and the 21-item GCCT-Mississippi

State Hospital revision (GCCT-MSH; Nicholson, Briggs, &

Robertson, 1988). Although the individual MacCAT-CA scales

contain fewer items, they produced estimates of internal consis-

tency reliability that are similar to those reported for the other

competency measures. In addition, differences between compe-

tent and incompetent defendants on the MacCAT-CA scales are

comparable to those obtained with the CST and GCCT-MSH.

Reanalysis of data reported in a quantitative review comparing

competent and incompetent defendants (Nicholson & Kugler,

1991) yielded a mean effect size of 1.05 SDs across six studies

(n = 429) involving the CST and a mean effect size of 1.20

SDs across four studies (n = 539) involving the GCCT. In the

present study, effect sizes for the MacCAT-CA scales ranged

from almost 1 SD for comparisons with all incompetent defen-

dants to more than 1.5 SDs for comparisons with clinically

affirmed incompetent defendants. Interesting to note, the Under-

standing scale, which covers information similar to that assessed

by other knowledge-based competency measures (e.g., the char-

acteristics of criminal prosecution and defense, the nature of

criminal charges, the consequences of conviction), tended to

yield smaller effects than either the Reasoning or Appreciation

scales, which tap domains of competence-related abilities not

represented on other measures.

The MacCAT-CA will be available to the field after develop-

ment of clinical interpretive norms, currently in progress. Norms

will be derived using this eight-state sample of 729 defendants,

providing data about the distribution of MacCAT-CA perfor-

mance among competent and incompetent criminal defendants.

These norms will serve as guideposts to inform individualized

judgments regarding the competence of defendants, more spe-

cifically, defendants' abilities to understand and reason about

legally relevant information, and to appreciate the significance

of such information in the context of their own legal situation.

Although the MacCAT-CA is useful in evaluating criminal

defendants whose competence has been questioned, it is not

offered as a measure of legal competence per se. That is, we

do not anticipate the derivation of a single optimal cutoff score

for the empirical classification of a defendant as competent

or incompetent. Other psycholegal abilities are relevant to the

ultimate determination of competence (e.g., memory for specific

events related to one's own case; coherency of speech) and a

court may consider other nonpsychological factors (e.g., legal

complexity of the case) in making the dichotomous judgment

about "competence." As a result, no single measure, no matter

how nuanced, can encompass all of the •potentially relevant fac-

tors. Additionally, the MacCAT-CA includes no scales designed

to assess the examinee's test-taking set. Some defendants may

be motivated to appear to lack capacity when in fact, they do

not. As is the case with any forensic evaluation, the examiner

must consider the possibility of exaggerated and fabricated

deficits or psychopathology. When such deception is suspected,

it is necessary to use other measures designed to assess the

defendant's test-taking set.

Nevertheless, the MacCAT-CA, through its measurement of

understanding, reasoning, and appreciation capacities, offers a

more systematic and differentiated analysis of the psycholegal

abilities relevant to adjudicative competence than do existing

measures. Grounded in a comprehensive theory of adjudicative

competence, the MacCAT-CA offers the additional features of

standardized administration and criterion-related scoring that

are missing from other interview-based measures. Used as part

of a comprehensive evaluation that includes a relevant history

(e.g., psychiatric history), current mental-status evaluation, and

inquiry about case-specific memories, the MacCAT-CA should

enhance the thoroughness and quality of clinical evaluations of

adjudicative competence.

References

Appelbaum, P. S., Mirkin, S. A., & Bateman. A. L. (1981). Empirical

assessment of competency to consent to psychiatric hospitalization.

American Journal of Psychiatry, I3S, 1170-1176.

Barnard, G., Thompson, J. W., Freeman, W. C., Robbins, L., Gies, D.,

& HanKns, G. C. (1991). Competency to stand trial: Description

and initial evaluation of a new computer-assisted assessment tool

(CADCOMP). Bulletin of the American Academy of Psychiatry and

Law, 19, 367-381.

Bartko, J. J., & Carpenter, W. T. (1976). On the methods and theory of

reliability. Journal of Nervous and Mental Disease, 163, 307-317.

Bonnie, R. J. (1992). The competence of criminal defendants: A theoret-

ical reformulation. Behavioral Sciences and the Law, W, 291-316.

Bonnie, R. J. (1993). The competence of criminal defendants: Beyond

Dusky and Drope. University of Miami Law Review, 47, 291-316.

442 OTTO ET AL.

Cohen, J. (1977). Statistical power analysis for the behavioral sciences(Rev. ed.). New York: Academic Press.

Farkas, G. M., DeLeon, P. H., & Newman, R. (1997). Sanity examinercertification: An evolving national agenda. Professional Psychology:Research and Practice, 28, 73-76.

Fiske, D. W. (1971). Measuring the concepts of personality. Chicago:Aldine.

Golding, S. L., Roesch, R., & Schreiber, J. (1984). Assessment andconceptualization of competency to stand trial: Preliminary data onthe Interdisciplinary Fitness Interview. Law and Human Behavior, 9,321-334.

Grisso, T. (1991). Clinical assessment for legal decision making: Re-search recommendations. In S. A. Shah & B. D. Sales (Eds.), Lawand mental health: Major developments and research needs. (DHHSPublication No. ADM 91-1875, pp. 49-80). Rockville, MD: NationalInstitute of Mental Health.

Grisso, T., & Appelbaum, P. S. (1998). Assessing competence to consentto treatment. New York: Oxford University Press.

Grisso, X, Appelbaum, P. S., & HUl-Fotouhi, C. (1997). The MacCAT-TA clinical tool to assess patients' capacities to make treatment deci-sions. Psychiatric Services, 48, 1415-1419.

Grisso, X, Appelbaum, P. S., Mulvey, E. P., & Fletcher, K. (1995).The MacArthur treatment competence study. II: Measures of abilities

related to competence to£onsent to treatment. Law and Human Behav-ior, 19, 127-148.

Grisso, X, Cocozza, J., Steadman, H., Fisher, W., & Greer, A. (1994).The organization of pretrial forensic evaluations services: A nationalprofile. Law and Human Behavior, 18, 377-394.

Harkness, A. R., & McNulty, J. L. (1994). The personality psychopa-thology five (PSY-5): Issues from the pages of a diagnostic manualinstead of a dictionary. In S. Strack & M. Lorr (Eds.), Differentiatingnormal and abnormal personality. New %rk: Springer.

Harkness, A. R., McNulty, J. L., & Ben-Porath, Y. S. (1995). The per-sonality psychopathology five (PSY-5): Constructs and MMPI-2scales. Psychological Assessment, 7, 104-114.

Hoge, S. K., Bonnie, R. J., Poythress, N. G., Monahan, J., & Eisenberg,M. (1997). The MacArthur Adjudicative Competence Study: Devel-opment and validation of a research instrument. Law and HumanBehavior, 21, 141-179.

Hollingshead, A. R., & Redlich, F. C. (1958). Social class and mentalillness. New York: Guilford Press.

Kaufman, A. S., Ishikuma, X, & Kaufman-Packer, J. L. (1991). Amaz-ingly short forms of the WMS-R. Journal of Psychoeducational As-sessment, 9, 4-15.

Laboratory of Community Psychiatry (1974). Competency to stand trial

and mental illness. (DHEW Publication No. ADM 74-103). Rock-ville, MD: Department of Health, Education, & Welfare.

Lipsitt, P., Lelos, D., & McGarry, A. L. (1971). Competency for trial:A screening instrument. American Journal of Psychiatry, 128, 105-109.

Melton, G. B., Petrila, J., Poythress, N., & Slobogin, C. (1997). Psycho-logical evaluations for the courts: A handbook for mental healthprofessionals and lawyers (2nd ed.). New Tfork: Guilford Press.

Menzies, R. J., Webster, C. D., Roesch, R., Jensen, F., & Eaves, D.(1984). The Fitness Interview Test: A semi-structured instrument forassessing competency to stand trial. Medicine and Law, 3, 151-162.

Nicholson, R. (1992, August). Defining and assessing criminal compe-tencies. Paper presented at the 100th Annual Convention of the Ameri-can Psychological Association, Washington, DC.

Nicholson, R. A., Briggs, S. R., & Robertson, H. C. (1988). Instrumentsfor assessing competency to stand trial: How do they work? Profes-sional Psychology: Research and Practice, 19, 383-394.

Nicholson, R. A., & Kugler, K. E. (1991). Competent and incompetentcriminal defendants: A quantitative review of comparative research.Psychological Bulletin, 109, 355-370.

Overall, J., & Gorham, D. (1962). The Brief Psychiatric Rating Scale.Psychological Reports, 10, 799-812.

Overall, J., & Porterfield, J. (1963). Power vector method of factoranalysis. Psychometrika, 28, 415-422.

Poythress, N. G., Hoge, S. K.. Bonnie, R. J., Monahan, J., Eisenberg,M., & Feucht-Haviar, T. (1998). The competence related abilities ofwomen criminal defendants. Journal of the American Academy ofPsychiatry and Law, 26, 215-222.

Poythress, N. G., Nicholson, R., Otto, R. K., Edens, J. F., Bonnie, R. J.,Monahan, J., & Hoge, S. K. (in press). Manual for the MacArthurCompetence Assessment Tool- Criminal Adjudication (MacCAT-CA).Odessa, FL: Psychological Assessment Resources.

Shrout, P. E., & Fleiss, J. L. (1979). Intraclass correlations: Uses inassessing rater reliability. Psychological Bulletin, 86, 420-428.

Steadman, H. J., & Hartstone, E. (1983). Defendants incompetent tostand trial. In J. Monahan & H. J. Steadman (Eds.), Mentally disor-dered offenders: Perspectives from law and social science. New M>rk:Plenum.

Wechsler, D. (1981). The Wechsler Adult Intelligence Scale-Revised:Manual. San Antonio, TX: Psychological Corporation.

Wildman, R., Batchelor, E., Thompson, L., Nelson, F., Moore, J., Pat-terson, M., & DeLaosa, M. (1980). The Georgia Court CompetencyTest: An attempt to develop a rapid quantitative measure for fitnessfor trial. Unpublished manuscript, Forensic Services Division, CentralState Hospital, Milledgeville, Georgia.

Woerner, M. G., Mannuzza, S., & Kane, J. M. (1988). Anchoring theBPRS: An aid to improved reliability. Psychopharmacology Bulletin,24, 112-118.

MAcCAT-CA 443

Appendix

Legal Content of the MacCAT-CA Measures

A. Legal Content of the Understanding Measure1. Understanding the roles of defense attorney and prosecutor.2. Understanding both the act and mental elements of a serious

offense.3. Understanding the elements of a less serious offense.4. Understanding the role of a jury.5. Understanding the responsibilities of a judge at a jury trial.6. Understanding sentencing as a function of the severity of the

offense.7. Understanding the process of a guilty plea.8. Understanding the rights waived in pleading guilty.

B. Legal Content of the Reasoning Measure9. Reasoning about evidence suggesting self-defense.

10. Reasoning about evidence related to criminal intent.11. Reasoning about evidence of provocation.12. Reasoning about motivation for one's behavior.13. Reasoning about the potential impact of alcohol on one's

behavior.14. Capacity to identify information that might inform the decision

to plead guilty versus plead not guilty.

15. Capacity to identify both potential costs and potential benefits ofa legal decision (e.g., pleading guilty).

16. Capacity to compare one legal option (e.g., accepting a pleabargain) with another legal option (e.g., going to trial) in termsof advantages and disadvantages.

C. Legal Content of the Appreciation Measure17. Plausibility of defendant's beliefs about the likelihood of being

treated fairly by the legal system.18. Plausibility of defendant's beliefs about likelihood of being

helped by his/her lawyer.19. Plausibility of defendant's beliefs about whether to disclose case

information to his/her attorney.20. Plausibility of defendant's beliefs about likelihood of being found

guilty.21. Plausibility of defendant's beliefs about likelihood of being pun-

ished if found guilty.22. Plausibility of defendant's beliefs about whether to accept a plea

bargain.

Received April 23, 1998Revision received June 21, 1998

Accepted July 30, 1998 •

750 First Street, NE - Washington, DC 20002-424220̂ ?J36- S5

American Psychological Associat750 First Street, NEWashington, EC Z0002-424Z

75IM=ir5t Street, NE - Washington, DC 20002-4;

l ^ ^ « ^ a a l ^ ' ^ o d a i t : ^ W ^ d a : - > a s . ""™

n Psychological Associate ISO First Street, KB

shington, DC 20002-4242

nstruetlons to Publisher

of bonOi, mongagBi. or other HtuitUn of ttw p

eolation mgsi Be puDKOaa: It rnt