Memory and Attention Profiles in Pediatric Traumatic Brain Injury

Daniel N. Allen 1,*, Brian D. Leany 1, Nicholas S. Thaler 1, Chad Cross 2, Griffin P. Sutton 1,Joan Mayfield 3

1Department of Psychology, University of Nevada, Las Vegas, NV, USA2Department of Environmental and Occupational Health, Epidemiology and Biostatistics Unit, University of Nevada, Las Vegas, NV, USA

3Our Children’s House at Baylor, Dallas, TX, USA

*Corresponding author at: Neuropsychology Research Program, Department of Psychology, University of Nevada Las Vegas, 4505 Maryland Parkway,

Las Vegas, NV 89154, USA. Tel.: +1-702-895-1379; fax: +1-702-895-0195.

E-mail address: daniel.allen@unlv.edu (D.N. Allen).

Accepted 16 June 2010

Abstract

Traumatic brain injury (TBI) causes heterogeneous patterns of neurocognitive deficits. In an attempt to identify homogenous subgroups

within this heterogeneity, cluster analysis was used to examine memory and attention abilities as measured by the Test of Memory and

Learning (TOMAL) in 300 children, 150 with TBI and 150 matched nonbrain injured controls (standardization sample [SS]). Significant

differences were present between the TBI and the SS groups on all TOMAL subscale and index scores, with the TBI groups performing

approximately 1.3 SD below the SS. Factor analysis of the TOMAL indicated six factors that assessed various aspects of verbal and nonverbal

learning and memory, as well as attention/concentration. Cluster analyses of TOMAL factor scores indicated that a four-cluster solution was

optimal for the SS group, and a five-cluster solution for the TBI group. For the TBI clusters, differences were present for clinical, achieve-

ment, neurocognitive, and behavioral variables, providing some support for the validity of the cluster solution. These findings suggest that

TBI results in unique patterns of neurocognitive impairment that are not accounted for by individual differences in test performance com-

monly observed in normal populations. Additionally, neurocognitive profiles identified using cluster analysis may prove useful for identifying

homogeneous subgroups of children with TBI that are differentiated by a number of important clinical, cognitive, and behavioral variables

associated with treatment and outcomes.

Keywords: Attention; Childhood brain insult; Head injury; Traumatic brain; Injury; Learning and Memory

Introduction

Behavioral disturbances and neurocognitive deficits in attention, learning, and memory commonly occur following trau-

matic brain injury (TBI; Babikian & Asarnow, 2009; Roman et al., 1998; Schwartz, et al., 2003; Yeates et al., 2005).

These neurocognitive deficits are associated with a number of important outcomes (Gil, 2003; Rassovsky et al., 2006).

However, both the severity and patterns of neurocognitive impairment are heterogeneous. Heterogeneity of these neurocogni-

tive and behavioral deficiencies are not reflected in more general approaches to TBI severity classification based on, for

example, open and closed head injury, length of coma or post-traumatic amnesia, and persisting neurological signs (Reitan

& Wolfson, 1993), and it may be that neurocognitive tests could provide more meaningful classification by identifying sub-

groups of children with TBI, who differ not only in the patterns of neurocognitive deficits, but also exhibit differences in

other domains, such as behavioral abnormalities, academic achievement, and functional outcomes. Cluster analysis has

been effectively used to identify subgroups of patients who exhibit differing patterns of neurocognitive impairment in a

variety of psychiatric and neurological disorders, including TBI (Allen et al., 2000; Crawford, Garthwaite, & Johnson,

1997; Demery, Pedraza, & Hanlon, 2002; Wiegner & Donders, 1999). Studies of adults with TBI indicate the existence of

discrete TBI subgroups that differ with regard to both pattern and severity of neurocognitive deficits (Chan, Hoosain, Lee,

Fan, & Fong, 2003; Crosson, Greene, Roth, Farr, & Adams, 1990; Goldstein, Allen, & Caponigro, 2010; Malec, Machulda,

Archives of Clinical Neuropsychology 25 (2010) 618–633

# The Author 2010. Published by Oxford University Press. All rights reserved. For permissions, please e-mail: journals.permissions@oxfordjournals.org.

doi:10.1093/arclin/acq051 Advance Access publication on 28 July 2010

at UN

LV

University L

ibraries on September 27, 2012

http://acn.oxfordjournals.org/D

ownloaded from

& Smigielski, 1993; Millis & Ricker, 1994; van der Heijden & Donders, 2003). Fewer studies have examined neurocognitive

heterogeneity in children with TBI, but provide evidence for similar heterogeneity as observed in adults, when the California

Verbal Learning Test for Children and the Wechsler Intelligence Scale for Children are examined (Donders & Warschausky,

1997; Mottram & Donders, 2006).

Although providing substantive support for the presence of neurocognitive heterogeneity in TBI, the results of these studies

are limited in a number of ways, including in some cases, the use of small sample size which limits generalizability, the use of

IQ tests to develop clusters, because IQ tests have limited sensitivity to brain damage (Malec et al., 1993), and the relative

absence of support for the “external validity” of cluster solutions through, for example, examination of cluster differences

on neurocognitive, achievement, or behavioral variables that were not included in the cluster analysis. With regard to this

latter point, associations between cluster membership and post-injury disability in TBI (Malec et al. 1993), as well as differ-

ences in neurocognitive abilities and affective symptomotology in HIV infected individuals (Murji et al., 2003), provide some

evidence that neurocognitive subtypes may have utility in predicting post-injury disability, psychiatric status, and possibly

other outcomes in pediatric TBI. A further limitation is that there has not been a direct comparison between clusters

derived in TBI and nonbrain injured samples in order to establish whether TBI results in different profiles, even though the

importance of such comparisons has been noted for many years (Crosson et al., 1990). Separate cluster analysis of TBI and

standardization sample (SS) data for measures such as the CVLT and Wechsler Intelligence scales (Donders, 1996, 1999,

2008; Donders, Zhu, & Tulsky, 2001) provide support for such differences, but direct comparisons between the groups are

made difficult due to differences among the TBI and normal groups in sample size, demographic variables, as well as other

considerations. Additionally, there has been a tendency for cluster analytic studies to focus on a single ability (e.g., verbal

memory), which constrains the identification of clusters that might reflect meaningful subtypes within TBI, because TBI

may result in a variety of neurocognitive deficits. Finally, in some studies, cluster analysis is used to investigate the psycho-

metric properties of specific tests by, for example, determining base rates of score profiles for a particular instrument in stan-

dardization samples. Although these studies provide valuable psychometric information about specific tests, they do not

address broader questions, such as whether there are common neurocognitive profiles in children with TBI that identify

TBI subgroups/subtypes, as established by external validity indicators, which in turn may predict important outcomes.

Cluster analysis can be used as a taxometric procedure to address such questions in children with TBI, in order to develop

a classification system that may prove useful in understanding not only the common types of neurocognitive profiles that

result from TBI, but also cognitive, clinical, and behavioral abnormalities that characterize each neurocognitive subgroup.

Such classification may, in turn, prove useful for predicting prognosis and outcomes, prescribing treatment, and directing reha-

bilitation. On balance, the existing studies do provide a strong base from which to hypothesize that meaningful subgroups can

be identified in pediatric TBI using neuropsychological tests, even though the validity of these neurocognitive subgroups has

not been examined extensively.

The purpose of the current study was to further investigate neurocognitive heterogeneity in pediatric TBI. We were particu-

larly interested in providing a broader assessment of cognitive abilities than accomplished by using tests of a single ability and

the Test of Memory and Learning (TOMAL; Reynolds & Bigler, 1994a, 1994b) for this purpose. The TOMAL includes tests of

verbal and nonverbal memory, immediate and delayed recall, as well attention/concentration. Because attention and memory

deficits are common sequelae of childhood TBI (Babikian & Asarnow, 2009), and memory deficits are predictive of edu-

cational needs and persists even after extended periods of recovery (Miller & Donders, 2003; Yeates et al., 2002), the

TOMAL should be an excellent tool for examination of TBI subtypes. A second goal was to determine whether differing pat-

terns of learning, memory, and attention/concentration would be evident between our sample of children with TBI and a normal

control sample. A third goal was to determine if the clusters identified in the TBI group differed from each other on important

clinical, cognitive, and behavioral variables, in a manner that might be anticipated based on prior research, as such differences

would provide further support for cluster validity.

With regard to number of clusters, based on prior cluster analyses, we hypothesized that cluster analysis of the TOMAL in

children with TBI would result in at least four clusters, two that were differentiated by the level of performance (average/low

average and impaired) and two additional clusters differentiated by relative strengths in either verbal or nonverbal memory

abilities. Also, because the TOMAL has an Attention/Concentration index (ACI), and given the presence of attention deficits

in TBI, a fifth cluster might also emerge that exhibited selective impairment in this domain. For the normal comparison sample

selected from the TOMAL SS, four clusters were predicted that would be differentiated primarily by the level of performance

(superior, high average, average, low average).

With regard to the external validity of the TBI clusters, comparison among the clusters on demographic, clinical, achieve-

ment, IQ, neuropsychological, and behavioral variables were made in order to determine if the clusters identified in the TBI

group differed in a manner that was consistent with prior research. We did not predict differences between the clusters on

demographic variables, although it was hypothesized that the most severely impaired cluster would have lower Glasgow

D.N. Allen et al. / Archives of Clinical Neuropsychology 25 (2010) 618–633 619

at UN

LV

University L

ibraries on September 27, 2012

http://acn.oxfordjournals.org/D

ownloaded from

Coma Scale (GCS) scores, a shorter time interval between injury and assessment (Mottram & Donders, 2006), and a younger

age at injury compared with the average cluster. We hypothesized younger age at injury in the impaired cluster because longi-

tudinal studies suggest that earlier age of injury is associated with poorer outcomes including slower development of attention,

motor, and verbal and spatial skills (Anderson, Catroppa, Morse, Haritou, & Rosenfeld, 2000; Anderson, Catroppa, Rosenfeld,

Haritou, & Morse, 2000; Anderson & Moore, 1995; Anderson, Morse, Catroppa, Haritou, & Rosenfeld, 2004; Kriel, Krach, &

Panser, 1989; Thompson et al., 1994). Significant differences between the clusters were also hypothesized for intellectual vari-

ables (Mottram & Donders, 2006), with perceptual organization abilities and speed of information processing demonstrating

the largest differences among the clusters, and associated with overall levels of neurocognitive impairment. Neurocognitive

variables assessing attention, perceptual organization, executive, language, and motor performance were predicted to

exhibit differential patterns of impairment that were consistent with domain- and modality-specific patterns identified in the

clusters. For example, individuals exhibiting selective deficits in verbal memory were predicted to also exhibit relative impair-

ment on tests of verbal and language abilities, whereas nonverbal memory impairment was expected to exhibit associations

with visuospatial and perceptual processing deficits (Allen, Thaler, Donohue, & Mayfield, 2010). We have reported prelimi-

nary evidence supporting generalized decrements across achievement test scores in children with TBI (Stolberg, Terranova,

Jones, Allen, & Mayfield, 2009) and thus anticipated overall level of performance differences between clusters, with the

most impaired cluster exhibiting the lowest achievement test scores, although differential patterns of impairment among the

clusters may also be present. Finally, given that difficulties with social behavioral regulation are common following child

TBI (Dennis, Guger, Roncadin, Barnes, & Schachar, 2001; Fay et al., 2009; Hawley, 2003; McKinlay, Grace, Horwood,

Fergusson, & MacFarlane, 2009; Yeates et al., 2004) and that these problems are associated with severity of TBI (e.g.,

Dennis et al., 2001), we predicted that severity of impairment as indicated by cluster membership would be associated with

attention, hyperactivity, and conduct problems in school and at home, as well as increased levels of self-reported psychological

distress.

Materials and Methods

Participants

Participants included 300 children. Of these, 150 had sustained TBI and were selected from a consecutive series of cases

referred for neuropsychological evaluation (TBI group). Definitive evidence of brain damage resulting from TBI was estab-

lished in all cases using appropriate neuroimaging, laboratory, and examinational findings. GCS scores were available for

101 of the participants and indicated that on average, injuries were severe in nature (median ¼ 7). Participants were included

if they had a primary diagnosis of TBI, had been administered the TOMAL as part of a comprehensive neuropsychological

evaluation, and did not have pre-injury neurological or neurodevelopmental disorders. Of the 150 participants selected,

56.3% were Caucasian, 23.4% were African American, 17.2% were Hispanic, 0.8% were Asian American, and 2.3% were

of other ethnicities. The sample was 52.1% male with an average age of 11.7 years (SD ¼ 3.7; range ¼ 5–19). They were

assessed an average of 6.9 months (SD ¼ 3.1; range ¼ 3–21) following injury. Ten (6%) participants had open head injuries,

whereas the others had closed head injuries. The remaining 150 children (SS group) were selected from the TOMAL standar-

dization smaple. SS cases were matched individually to the TBI participants on age and sex. When more than one member of

the SS group matched a TBI participant on age and sex, random selection was used to choose the SS case. For 17 TBI cases,

matches were not available, so these cases were randomly selected from the remaining SS cases. Participants in the SS group

were on average 11.2 years old (SD ¼ 3.1; range ¼ 5–18) and 47.4% were male.

Measures

Test of Memory and Learning (Reynolds & Bigler, 1994a). The TOMAL was developed for children ages 5–19 years and is

composed of 14 subtests, 10 that are core and 4 that are supplementary. Core subtests include Memory for Stories, Facial

Memory, Word Selective Reminding, Visual Selective Reminding, Object Recall, Abstract Visual Memory, Digits

Forward, Visual Sequential Memory, Paired Recall, and Memory for Location. Supplementary subtests include Letters

Forward, Digits Backward, Letters Backward, and Manual Imitation. Subtest scores are used to calculate a number of core

index scores, including a Verbal Memory, Nonverbal Memory, Composite Memory, Delayed Recall, and Attention/

Concentration Indexes. Index scores are standard scores with means of 100 and standard deviations of 15.

Demographic and clinical information. In addition to demographic variables, severity of injury was measured by the GSC

(Teasdale & Jennett, 1974). The GCS was administered either by first responders at the scene of the accident or in the

620 D.N. Allen et al. / Archives of Clinical Neuropsychology 25 (2010) 618–633

at UN

LV

University L

ibraries on September 27, 2012

http://acn.oxfordjournals.org/D

ownloaded from

emergency room when the child was transported to the hospital. The time interval between injury and assessment was used as

an indicator of recovery, with a longer interval indicating more recovery.

Intellectual, achievement, and neurocognitive measures

Wechsler Intelligence Scales (WIS). The WISs were used to assess intelligence. Because children were tested over a period

of more than 7 years and were of different ages, some were administered the WISC-III (n ¼ 116) or WISC-IV (n ¼ 10), and

others the Wechsler Adult Intelligence Scale-Revised (n ¼ 9), or Wechsler Adult Intelligence Scale-Third Edition (n ¼ 7).

Given that versions of the Wechsler Scales share many common subtests, and subtests are designed to measure similar abilities

across age groups, data were combined across the various versions. We analyzed individual subtests that were strong measures

of their representative index scores (Reynolds & Ford, 1994), including Vocabulary (Verbal Comprehension Index), Block

Design (Perceptual Organization Index), Digit Span (Working Memory Index), and Digit Symbol/Coding (Processing

Speed Index). We also examined group differences on Full-Scale IQ.

The Woodcock–Johnson Psychoeducational Battery Tests of Achievement (Woodcock & Johnson, 1989). Academic

achievement was evaluated using the Revised or Third version of the Woodcock–Johnson Psychoeducational Battery Tests

of Achievement. Broad Reading and Broad Math cluster scores were selected for analysis because these were completed by

most participants and reflect two of the major components assessed by the tests.

Grooved Peg Board Test (Tiffin, 1948). The Grooved Peg Board Test is a widely used measure of motor speed and dexter-

ity. Scores included in the analyses were the time taken to place all of the pegs with the dominant and nondominant hands.

Beery–Buktenica Developmental Test of Visual-Motor Integration (Fifth Edition). The Beery–Buktenica Developmental

Test of Visual-Motor Integration (Beery & Beery, 2004) is a paper and pencil test of visual perception and visual-motor inte-

gration that takes approximately 10–15 min to administer and is comprised of 30 figures of increasing complexity that the test

subject is instructed to copy.

Trail Making Test Parts A and B Intermediate Version (Reitan & Wolfson, 1993). The Trail Making Test parts A and B were

used to assess psychomotor speed, visual search and sequencing, and mental flexibility (Part B).

Conners’ Continuous Performance Test II (Conners & MHS Staff, 2000). The Conners’ Continuous Performance Test II

(CPT-II; Conners & MHS Staff, 2000) is a computer-administered task comprised of a series of letters presented intermittently

on the computer screen, with time intervals of varying lengths occurring between the letters. Test subjects were asked to click

the mouse when a letter appeared that was not the letter “X.”

The Oral and Written Language Scales (Carrow-Woolfolk, 1996). The Oral and Written Language Scale (OWLS) is an

individually administered assessment of receptive and expressive language for ages 3 through 21 years. The OWLS is com-

prised of three co-normed scales, including Listening Comprehension (LC), Oral Expression, and Written Expression. The

LC scale was used in this study as a measure of receptive language.

The Behavior Assessment System for Children (Reynolds & Kamphaus, 1992). The Behavior Assessment System for

Children (BASC; Reynolds & Kamphaus, 1992) Parent Rating Scale (PRS), Teacher Rating Scale (TRS), and Self-Report

of Personality (SRP) scales were used to assess adaptive and clinical (maladaptive) behaviors. Analyses focused on TRS,

PRS, and SRP composite scores, but given the literature suggesting the presence of attention deficits, impulsivity, conduct pro-

blems, and learning difficulties following TBI, we examined these domains as assessed by individual BASC subscales.

Procedure and data analysis. Tests were administered according to standardized procedures by a pediatric neuropsychologist

or doctoral level technicians who were supervised by the pediatric neuropsychologist. The TOMAL was routinely administered

to all children, unless there was a reason not to administer the test (referral question, age of child, etc.). The BASC was com-

pleted by either the parent (PRS), child with TBI (SRP), or teacher (TRS).

To establish the sensitivity of the TOMAL to TBI, SS and TBI between group comparisons were made using multivariate

analysis of variance (MANOVA) for the TOMAL subtest and Index scores, with post hoc univariate analyses used to examine

specific subtest and composite scores differences between the groups when overall MANOVAs were significant. Principal com-

ponent analysis (PCA) with varimax rotation was then used to identify groupings of TOMAL subtests to be used in the cluster

analysis. PCA was conducted because prior investigations of the TOMAL have produced mixed results with regard to the

D.N. Allen et al. / Archives of Clinical Neuropsychology 25 (2010) 618–633 621

at UN

LV

University L

ibraries on September 27, 2012

http://acn.oxfordjournals.org/D

ownloaded from

number of factors (compare Alexander & Mayfield, 2005; Reynolds & Bigler, 1994b, 1996; and Thaler, Allen, McMurray, &

Mayfield, 2010). The Kaiser–Guttman criterion was used to select the number of factors. Factor scores rather than TOMAL

Index scores were used in the cluster analysis, because index scores are not independent of each other, as some TOMAL subt-

ests contribute to more than one index score. Factor scores are also more reliable than individual subtest scores. Factor scores

were calculated by deriving the mean of the standard scores for each factor’s respective subtests (e.g., Allen, Strauss, Kemtes,

& Goldstein, 2007).

Cluster analysis was then used to examine neurocognitive heterogeneity in the SS and TBI groups. We analyzed the data

with four- and five-cluster solutions to determine which solution was optimal. TBI and SS groups were analyzed separately to

determine optimal cluster solutions for each group. Ward’s method of cluster analysis was used because it is consistent with the

cluster analytic methodology of previous studies (Curtiss, Vanderploeg, Spencer, & Salazar, 2001; Mottram & Donders, 2006),

it produces results that are consistent with other agglomerative clustering methods, and is less affected by outliers compared

with those methods (Morris, Blashfield, & Satz, 1981). We used squared Euclidian distance as the dissimilarity measure in the

analyses because it provides a direct, physical measure of distance in Euclidean space (Everitt, Landau, & Leese, 2001) and it is

sensitive to pattern and level of performance differences among individuals. To identify the appropriate number of clusters, we

followed the approach suggested by Aldenderfer and Blashfield (1984). Stability of the cluster solutions derived using Ward’s

method were examined using K-means iterative partitioning cluster analysis. For K-means, the number of clusters and initial

cluster centers were specified based on Ward’s method results. K-means derives clusters by iteratively adding cases to the group

that has the mean value closest to the unassigned data point (Hair, Black, Babin, Anderson, & Tatham, 1992). Cohen’s kappa

was then used to assess the extent to which the K-means and Ward’s method solutions agreed. On the basis of these various

methods, along with consideration of the a priori hypotheses regarding number and type of clusters, the most appropriate sol-

utions were identified for the SS and the TBI groups.

External validity of the optimal TBI cluster solution was then examined by comparisons on variables that were not included

in the cluster analysis, but were theoretically and clinically relevant to TBI, including demographic, clinical, IQ, achievement,

neurocognitive, and behavioral variables. In cases where interaction effects were hypothesized (e.g., IQ and neurocognitive

variables), mixed-model ANOVAs were used to examine cluster differences, in which cluster membership served as a between-

subjects variable and test scores as a within-subjects variable. When no interaction effects were predicted, chi-squared, uni-

variate ANOVAs, or multivariate ANOVAs were used (as appropriate) to examine cluster differences.

Results

There were no significant differences between the TBI and the SS groups on age or sex, x2(1) ¼ 0.66, p ¼ .42, indicating

that the group selection and matching procedure was successful. MANOVAs indicated significant differences between the SS

and the TBI sample for TOMAL subtest scores, F(68, 528) ¼ 5.59, p , .001, h2p, and Index scores, F(68, 528) ¼ 12.4, p ,

.001, h2p = .31. Following the Bonferonni correction for multiple comparisons, significant differences were present between

the TBI and the SS groups on all of the TOMAL scores. The SS group’s performance on the TOMAL subtest and index

scores approximated the SS mean, and the TBI group performed approximately 1.3 SD below the SS group on most

TOMAL scores, suggesting that the TOMAL was sensitive to TBI (detailed results of these analyses may be obtained from

the corresponding author).

Principal Component Analysis

PCA conducted for the SS group indicated six factors based on the Kaiser–Guttman criteria. This solution approached

simple structure, with most TOMAL subtests exhibiting primary loadings greater than 0.70, and few cross loadings greater

than 0.40. The six factors were interpretable and assessed attention/concentration, and verbal and nonverbal memory processes,

with distinctions between the verbal and the nonverbal factors based to some extent on task requirements. For the TBI group,

the Kaiser–Guttman criteria indicated four factors. However, the four-factor solution did not approach simple structure and

some factors that were not interpretable. When a six-factor solution was specified for the TBI group, a solution comparable

to the SS group was obtained that approached simple structure. A final PCA conducted for the combined samples generally

improved the six-factor solution (Table 1) and is consistent with the original conceptualization of the TOMAL as a

measure of verbal and nonverbal memory processes, and attention/concentration (Reynolds & Bigler, 1994a, 1996). On the

basis of these PCA results, six-factor scores were derived and subsequently used in the cluster analyses (separate PCA

results for the TBI and the SS groups may be obtained from the corresponding author).

622 D.N. Allen et al. / Archives of Clinical Neuropsychology 25 (2010) 618–633

at UN

LV

University L

ibraries on September 27, 2012

http://acn.oxfordjournals.org/D

ownloaded from

Cluster Analyses for TBI and SS Groups

Cluster analysis of the TOMAL subtests was accomplished specifying four- and five-cluster solutions for the TBI and the SS

groups. For both groups, a preliminary graphical inspection of the cluster solutions based on the factor scores was not particu-

larly informative because the number of factors made interpretation difficult, so Index score profiles were examined. Factor

score profiles were largely consistent with the Index score profiles, although Index score profiles were easier to interpret

and allowed for examination of profiles as they would occur in clinical settings (i.e., clinicians will not typically interpret

factor scores).

Examination of index score profiles for the SS cluster solutions indicated that the clusters were differentiated primarily by

the level of performance. The four-cluster solution consisted of a low average (C1), average (C2), and two above average clus-

ters, one that obtained scores slightly above 100 (C3), and another with superior performance on the ACI (mean ¼ 120.5) as

well as elevated scores on the Verbal and Nonverbal Memory Indexes (C4; Fig. 1, Frame A). The five-cluster solution further

divided the average cluster (C2) into two clusters, which did not markedly differ from each other. Thus, preliminary inspection

of the cluster solutions indicated that the four-cluster solution was the better of the two solutions for the SS group.

For the TBI group, clusters were differentiated by both level and pattern of performance. The four-cluster solution consisted

of a low-scoring cluster that exhibited a relative increase on the Delayed Recall Index (DRI) and ACI, a middle cluster exhibit-

ing a relatively lower score on the VMI, an average cluster that exhibited its highest performance on the Nonverbal Memory

Index, and an average cluster that obtained its lowest Index scores on the ACI and NMI. As seen in Fig. 1, Frame B, the

five-cluster solution included an Impaired cluster, that obtained Index scores ranging from 56.7 to 66.7, and obtained the

highest scores on the DRI and ACI, with relatively lower scores on the VMI and NMI. There was also a Verbal cluster that

obtained its lowest score on the VMI (70.8), with NMI, DRI, and ACI scores approximating 80.0. A Visual cluster is also appar-

ent that exhibits the lowest performance on the NMI (65.6) with scores on the VMI, DRI, and ACI of approximately 80.0. Two

additional clusters exhibit performance across all Indexes in the low average to average range, with the first cluster obtaining

average scores on all indexes (Average cluster), and the second exhibiting relative decreases on the ACI and possibly the NMI

(Attention cluster). Thus, preliminary inspection of the four- and five-cluster solutions in the TBI group suggest that the four-

and five-cluster solutions provide clear indication of level and pattern of performance differences, with this being the case

Table 1. Factor structure of the TOMAL combined SS and TBI groups (N ¼ 300)

TOMAL Factors

1 2 3 4 5 6

LF 0.83 0.12 0.13 0.15 0.16 0.10

DF 0.82 0.06 0.18 0.09 0.06 0.14

LB 0.76 0.32 0.07 0.11 0.00 0.17

DB 0.75 0.23 0.08 0.13 0.06 0.12

VSM 0.48 0.11 0.28 0.35 0.22 0.04

WSR 0.28 0.73 0.21 0.19 0.15 0.12

PR 0.16 0.74 0.23 0.02 0.04 0.24

OR 0.38 0.49 0.31 0.26 0.27 0.21

WSRD 0.14 0.78 0.10 0.26 0.12 -0.05

MFSD 0.19 0.27 0.86 0.15 0.16 0.08

MFS 0.19 0.22 0.90 0.07 0.07 0.16

VSRD 0.11 0.22 0.11 0.88 0.06 0.15

VSR 0.37 0.20 0.10 0.76 0.07 0.20

FM 0.19 0.18 0.26 0.21 0.72 0.10

FMD 0.05 0.08 0.01 20.03 0.88 0.03

AVM 0.24 0.12 0.31 0.19 0.31 0.60

MFL 0.21 0.14 0.06 0.16 20.02 0.85

Eigenvalue 7.1 1.5 1.2 1.1 1.0 0.8

% Variance 41.6 8.9 7.0 6.4 5.8 4.7

Chronbach’s a 0.86 0.82 0.92 0.74 0.62 0.56

Notes: TOMAL ¼ Test of Learning and Memory; SS ¼ Standardization sample group; TBI ¼ Traumatic brain injury group; LF ¼ Letters Forward; DF ¼

Digits Forward; LB ¼ Letters Backwards; DB ¼ Digits Backwards; VSM ¼ Visual Sequential Memory; WSR ¼Word Selective Reminding; PR ¼ Paired

Recall; OR ¼ Object Recall; WSRD ¼Word Selective Reminding Delayed; MFS ¼Memory for Stories; MFSD ¼Memory for Stories Delayed; VSRD ¼

Visual Selective Reminding Delayed; VSR ¼ Visual Selective Reminding; FM ¼ Facial Memory; FMD ¼ Facial Memory Delayed; AVM ¼ Abstract

Visual Memory; MFL ¼Memory for Location.

D.N. Allen et al. / Archives of Clinical Neuropsychology 25 (2010) 618–633 623

at UN

LV

University L

ibraries on September 27, 2012

http://acn.oxfordjournals.org/D

ownloaded from

particularly for the five-cluster solution. The five-cluster solution also appears to be the most theoretically and clinically rel-

evant of the solutions, based on the a priori hypothesis.

Stability of the four- and five-cluster solutions examined using the K-means iterative partitioning method indicated that the

Cohen’s kappas for the SS group were 0.86 and 0.77, respectively, and 0.78 and 0.87, respectively, for the TBI group. Kappas

at or above a 0.80 level are considered to indicate excellent agreement (Landis & Koch, 1977) and indicate that four-cluster SS

solution and five-cluster TBI solution are stable and have a high-level agreement when derived using Ward’s and K-means

methods of cluster analysis. Therefore, the five-cluster solution appeared optimal for TBI, and the four-cluster solution

appeared optimal for SS.

TOMAL Index score profiles for the four-cluster SS solution and five-cluster TBI solutions are presented in Fig. 1. In the

figure, the Verbal Memory, Nonverbal Memory, and Attention/Concentration Indexes are presented first, followed by the

Composite and Delayed Recall Memory Indexes, as these latter indexes combined verbal, nonverbal, and attention/concen-

tration subscale scores, and so are not useful in determining group differences across discrete abilities. Comparison of the

SS and TBI cluster solutions indicated not only overall differences in the level of performance, with the SS clusters generally

obtaining higher scores than the TBI clusters, but also that the TBI clusters were differentiated by both the level and the pattern

of performance, whereas the SS clusters tended to be differentiated by the level of performance, with pattern differences in the

SS group mostly determined by DRI scores. In contrast, for the TBI group, there were verbal and nonverbal memory discre-

pancies among the clusters, as well as a cluster that exhibited relative impairment in the attention domain (figures plotting the

factor scores for all cluster solutions may be obtained from the corresponding author).

Given that the five-cluster solution for the TBI sample was most consistent with the hypothesis, in combination with its

increased stability and theoretical interest, we further evaluated the external validity of the five-cluster solution in the TBI

group. Descriptive statistics for the TBI five-cluster solution on the TOMAL subtest and Index scores are presented in

Table 2 (tables containing descriptive statistics for the TOMAL subtests for all TBI and the SS cluster solutions may be

obtained from the corresponding author).

External Validity of the TBI Clusters

Demographic and clinical differences among the TBI clusters. Descriptive statistics for the demographic and clinical variables

are presented in Table 2. Chi-square analyses indicated no significant differences among the five clusters with regard to gender,

x2(4) ¼ 2.60, p ¼ .63, open or closed TBI, x2(4) ¼ 1.79, p ¼ .78, type of accident that caused the TBI, x2(28) ¼ 0.29, p ¼ .40,

Fig. 1. Cluster profiles of TOMAL Indexes for the standardization sample group four-cluster solution (Frame A), and TBI group five-cluster solution

(Frame B). Frame A: SS group; Frame B: TBI group. VMI ¼ Verbal Memory Index; NMI ¼ Non-verbal Memory Index; ACI ¼ Attention/Concentration

Index; CMI ¼ Composite Memory Index; DRI ¼ Delayed Memory Index.

624 D.N. Allen et al. / Archives of Clinical Neuropsychology 25 (2010) 618–633

at UN

LV

University L

ibraries on September 27, 2012

http://acn.oxfordjournals.org/D

ownloaded from

or whether the children were appropriately restrained in the case of motor vehicle accidents, x2(4) ¼ 4.74, p ¼ .32. One-way

ANOVAs indicated that the clusters did not differ significantly with regard to age, F(4,145) ¼ 1.66, p ¼ .16, age at injury,

F(4,145) ¼ 1.70, p ¼ .15, or time elapsed between TBI and assessment, F(4,145) ¼ 0.88, p ¼ .48. However, analysis of

ranked GCS scores indicated significant differences among the groups, F(4,96) ¼ 2.5, p , .05. Post hoc comparisons indicated

that the Attention cluster had significantly higher GCS scores than the Verbal cluster, although comparable but nonsignificant

differences were also present for the Impaired and Visual clusters.

IQ, achievement, and neuropsychological differences among clusters. Descriptive statistics for IQ, achievement, and neurop-

sychological variables are presented in Table 3, as are ANOVA results and post hoc comparisons examining differences among

the clusters. For Full-Scale IQ, ANOVA indicated a significant difference between the clusters (p , .001). The Average and

Attention clusters did not significantly differ from each other, but obtained significantly higher scores than the Verbal and

Visual clusters, which in turn performed significantly better than the Impaired cluster. Mixed-model ANOVA examining

potential differences on subtests selected from the four Wechsler Index scores indicated significant effects for Cluster,

F(4,134) ¼ 35.64, p , .001, h2p = .515, and for Wechsler subtest, F(3,402) ¼ 8.53, p , .001, h2

p = .060, as well as a signifi-

cant Cluster by Wechsler subtest interaction effect, F(12,402) ¼ 2.42, p ¼ .005, h2p = .067. The interaction appears to be

caused primarily by decrements in performance on the Digit Symbol/Coding (DC) subtest for the Average cluster, as well

Table 2. Demographic, Clinical, and TOMAL subtest and Index scores for the five-cluster solution in the TBI group

Variables Clusters

Average (n ¼ 38) Attention (n ¼ 28) Visual (n ¼ 14) Verbal (n ¼ 54) Impaired (n ¼ 16)

Mean SD Mean SD Mean SD Mean SD Mean SD

Age (years) 12.9 3.6 11.3 3.5 11.1 4.0 12.1 2.9 11.4 3.2

Age at Injury (years) 12.8 3.6 11.2 3.4 10.9 3.9 11.8 3.0 11.3 3.2

Time Elapsed (months) 7.0 2.5 6.3 2.4 7.0 3.5 8.2 5.0 6.9 2.2

GCS 7.5 3.2 8.5 3.5 6.1 2.4 6.2 2.8 6.0 2.1

Gender (% men) 55.3 53.6 61.1 42.9 68.8

TBI Type (% closed) 92.6 97.4 92.9 93.8 89.3

TOMAL Subtests

MFS 8.9 2.7 11.6 1.6 9.2 1.5 6.1 2.0 5.4 2.2

WSR 10.2 2.4 10.2 2.6 6.7 2.5 6.5 2.9 3.6 2.4

OR 8.5 3.3 7.7 2.1 5.1 2.0 4.2 2.3 1.9 1.7

DF 7.8 3.0 7.0 2.2 7.0 2.2 6.4 2.5 4.3 1.2

PR 10.4 2.4 9.3 3.2 7.7 3.1 5.5 3.2 3.4 3.0

LF 8.1 3.0 7.0 1.8 6.3 2.2 6.5 2.6 4.2 2.2

DB 10.0 2.4 8.5 1.8 8.3 1.6 7.5 1.7 5.8 1.8

LB 9.7 2.8 8.5 1.6 7.6 1.3 7.1 2.3 5.3 2.0

FM 9.7 2.7 8.5 2.7 6.4 2.7 6.5 2.4 5.5 2.4

VSR 8.4 3.1 6.9 2.3 3.9 1.9 6.7 2.9 2.7 1.4

AVM 10.5 2.7 8.3 2.2 3.4 2.0 6.9 2.6 3.3 2.6

VSM 9.9 2.6 9.4 1.8 8.0 2.8 7.4 2.6 5.6 1.7

MFL 12.1 3.4 7.5 3.0 3.3 2.0 6.8 3.4 4.1 3.1

MFSD 7.6 2.8 10.3 1.5 7.9 1.7 4.6 2.5 3.2 1.8

FMD 10.3 1.7 9.0 2.6 7.9 2.9 8.5 2.5 7.8 1.5

WSRD 10.0 2.2 9.4 1.8 7.4 2.3 6.6 2.5 3.9 1.7

VSRD 9.9 1.4 9.1 2.1 6.4 1.3 8.2 2.2 5.7 1.7

TOMAL Indexes

VMI 94.0 11.7 94.0 8.3 80.4 7.9 70.8 8.3 56.7 7.7

NMI 100.7 8.6 86.9 6.8 65.6 8.4 78.6 8.5 60.6 7.4

CMI 97.4 8.4 90.2 5.3 71.9 8.1 73.6 6.6 56.9 5.0

DRI 96.2 8.8 96.2 8.1 82.6 9.7 79.7 8.5 67.7 5.5

ACI 92.6 13.5 84.5 7.9 82.8 10.2 79.3 11.8 66.5 5.8

Notes: TOMAL ¼ Test of Learning and Memory; TBI ¼ Traumatic brain injury group; GCS ¼ Glasgow Coma Scale; MFS ¼Memory for Stories; WSR ¼

Word Selective Reminding; OR ¼ Object Recall; DF ¼ Digits Forward; PR ¼ Paired Recall; LF ¼ Letters Forward; DB ¼ Digits Backwards; LB ¼ Letters

Backwards; FM ¼ Facial Memory; VSR ¼ Visual Selective Reminding; AVM ¼ Abstract Visual Memory; VSM ¼ Visual Sequential Memory; MFL ¼

Memory for Location; MFSD ¼Memory for Stories Delayed; FMD ¼ Facial Memory Delayed; WSRD ¼Word Selective Reminding Delayed; VSRD ¼

Visual Selective Reminding Delayed; VMI ¼ Verbal Memory Index; NMI ¼ Non-Verbal Memory Index; CMI ¼ Composite Memory Index; DRI ¼

Delayed Recall Index; ACI ¼ Attention/Concentration Index.

D.N. Allen et al. / Archives of Clinical Neuropsychology 25 (2010) 618–633 625

at UN

LV

University L

ibraries on September 27, 2012

http://acn.oxfordjournals.org/D

ownloaded from

Table 3. Cluster differences on the Wechsler IQ, the Woodcock–Johnson, and neurocognitive test performances

Clusters Total F-value p-value Tukey h2p

Average Attention Verbal Visual Impaired

M SD M SD M SD M SD M SD M SD

Intelligence

n 37 27 47 13 15 139

VO 9.0 3.4 9.4 2.4 6.5 2.6 5.9 3.2 4.7 2.1 7.4 3.2 12.0 .000 AtAv, VrVsI .259

BD 9.8 2.7 9.0 3.0 7.8 3.1 5.0 2.8 3.0 2.0 7.8 3.5 20.0 .000 AvAtVr, VsI .361

DS 10.4 3.1 8.8 2.0 7.3 2.5 7.9 2.5 5.1 1.4 8.2 3.0 15.3 .000 AvAt, AtVrVs, I .312

DC 8.0 2.8 8.7 3.3 5.3 2.9 6.2 3.1 3.1 2.2 6.5 3.4 13.6 .000 AtAv, AvVs, VsVr, VrI .289

IQ (n ¼ 146) 93.1 13.0 93.9 10.8 78.5 10.7 76.6 10.3 63.9 8.3 83.4 14.7 28.5 .000 AtN, VrVs, I .447

Achievement

n 37 26 42 12 15 132

Math 102.6 16.5 101.6 10.9 87.8 13.7 88.1 11.8 72.3 18.5 92.9 17.4 15.9 .000 AvAt, VsVr, I .334

Reading 98.2 13.4 101.4 9.0 87.1 14.3 87.6 12.6 75.3 17.6 91.7 15.6 12.5 .000 AtAv, AvVsVr, I .283

Attention

n 31 25 42 13 14 125

Commissions 47.9 7.2 50.4 11.4 49.4 11.0 50.4 8.3 50.2 10.7 49.4 9.9 0.3 .88 – .010

Hit Rate 51.3 9.4 50.4 15.8 50.1 19.8 48.4 16.2 45.1 21.6 49.7 16.7 0.4 .83 – .013

Hit Rate SE 56.5 10.0 57.5 9.8 65.3 13.9 57.1 10.7 72.3 15.5 61.5 13.2 6.2 .000 AvVsAtVr, VrI .172

Variability 54.2 10.2 56.0 9.3 60.1 13.3 53.0 9.4 65.6 12.9 57.7 11.9 3.6 .009 VsAvAtVr, AtVrI .106

Detectability 51.3 7.6 52.5 9.5 53.2 9.7 52.4 10.1 58.0 10.2 53.0 9.3 1.3 .26 – .043

Perseveration 56.6 16.1 53.7 11.9 64.6 24.5 53.7 12.5 70.8 20.2 60.0 19.5 3.1 .02 AtVsAvVr, AvVrI .092

Language

n 36 27 50 14 14 141

OWLS 100.3 17.0 95.3 10.9 85.3 12.2 86.1 13.6 77.9 15.6 90.4 15.6 10.3 .000 AvAt, AtVsVr, VsVrI

Visuopercept.

n 34 27 50 13 15 139

BVMI 90.5 12.8 88.9 10.5 87.9 13.7 79.0 11.9 78.5 13.5 86.9 13.2 3.8 .006 AvAtVr, AtVrVsI

Motor

n 37 26 44 12 13 132

GPBD 74.1 15.0 76.2 30.0 89.3 35.0 86.0 27.7 96.3 51.6 83.1 32.1 1.9 .11 – .057

GPBND 80.9 22.8 83.2 26.3 99.2 46.1 114.2 55.5 131.3 74.3 96.9 46.4 4.4 .002 AtAvVrVs, VrVsI .123

Executive

n 28 18 29 11 10 96

TMA 25.1 9.6 31.2 15.2 34.6 15.8 31.1 12.2 55.0 55.3 32.9 22.7 3.6 .009 AvVsAtVr, VrI .137

TMB 46.2 17.6 61.9 24.0 79.0 40.9 79.1 37.9 150.1 110.6 73.7 53.6 10.0 .000 AvAtVrVs, I .305

Notes: IQ ¼Wechsler Full-Scale IQ; VO ¼Wechsler Vocabulary; BD ¼Wechsler Block Design; DS ¼Wechsler Digit Span; DC ¼Wechsler Digit Symbol/Coding; Reading ¼Woodcock Johnson

Broad Reading; Math ¼Woodcock Johnson Broad Math; CPT ¼ Continuous Performance Test; Commissions ¼ CPT Commissions; Hit Rate ¼ CPT Hit Rate; Hit Rates SE ¼ CPT Hit Rate

Standard Error; Variability ¼ CPT Variability of the Standard Errors; Detectability ¼ CPT Detectability; Perseverations ¼ CPT Perseverations; OWLS ¼ Oral and Written Language Scale;

BVMI ¼ Beery–Buktenica Developmental Test of Visual-Motor Integration-5th Edition; GPBD ¼ Grooved Peg Board dominant hand; GPBND ¼ Grooved Peg Board nondominant hand; TMA ¼

Trail Making Test Part A; TMB ¼ Trail Making Test Part B; Av ¼ Average Cluster; At ¼ Attention Cluster; Vr ¼ Verbal Cluster; Vs ¼ Visual Cluster; I ¼ Impaired Cluster.

626

D.N

.A

llenet

al.

/A

rchives

of

Clin

ical

Neu

ropsych

olo

gy

25

(2010)

618–633

at UNLV University Libraries on September 27, 2012 http://acn.oxfordjournals.org/ Downloaded from

as diminished performance on the Vocabulary (VO) and Block Design (BD) subtests by the Visual Cluster. Significant differ-

ences among the clusters were present for each subtest, with post hoc analysis indicating a general pattern in which the Average

and Attention clusters performed significantly better than the other three clusters (Table 3). Consistent with modality-specific

effects, the Normal cluster performed significantly better than the Verbal and Visual clusters on the Vocabulary and Block

Design subtests, respectively.

For the Woodcock–Johnson achievement test data, mixed-model ANOVA indicated a significant effect for Cluster,

F(4,127) ¼ 17.50, p , .001, h2p = .355, but not for WJ Composite Score, F(1,127) ¼ 0.08, p ¼ .78, h2

p = .001, or for the

Cluster by WJ Composite Score interaction effect, F(4,127) ¼ 1.19, p ¼ .32, h2p = .036. Post hoc comparisons indicated

the level of performance differences with the Normal and Attention clusters obtaining higher scores than the Visual and

Verbal clusters, who in turn had higher scores than the Impaired cluster (Table 3).

For the attention domain, CPT variables were entered into the mixed-model ANOVA, and results indicated a significant

main effect for Cluster, F(4,120) ¼ 2.83, p ¼ .03, h2p = .086, and for CPT score, F(5,600) ¼ 22.40, p , .001, h2

p = .157,

as well as a significant Cluster by CPT score interaction effect, F(20,600) ¼ 2.22, p ¼ .002, h2p = .069. Expected differences

were present between groups on a number of CPT measures, with the Impaired cluster typically performing worse than the

others, with significant differences among the clusters for CPT Standard Error, Variability, and Perseverations (Table 3).

Significant differences were also present between the clusters on the visuomotor (BVMI) and language (OWLS) domains.

Post hoc analyses indicated that for language, the Average cluster exhibited the best performance but did not differ from the

Attention cluster, whereas the Impaired cluster exhibited the worst performance, but did not differ from the Verbal or Visual

clusters. For visuomotor integration, the Average cluster again exhibited the best performance, although it did not differ from

the Attention or Verbal cluster, while the Impaired cluster exhibited the worst performance, but did not differ from the Visual,

Verbal, or Attention clusters.

For the motor domain, results for the Grooved Peg Board dominant and nondominant hands indicated that there were sig-

nificant main effects for Cluster, F(4,127) ¼ 3.74, p ¼ .007, h2p = .105, and Hand, F(1,127) ¼ 30.95, p , .001, h2

p = .196,

and a significant Cluster by Hand interaction effect, F(4,127) ¼ 2.83, p ¼ .028, h2p = .082. The dominant hand was faster

than the nondominant hand in all groups, although a larger than expected discrepancy is present for the Impaired cluster,

with the nondominant hand being slower than expected. Significant differences among the clusters were present for only

the nondominant hand, and post hoc analyses indicated that the Normal and the Attention clusters performed similar to

each other and significantly better than the Visual and Verbal clusters, who in turn performed better than the Impaired cluster.

Finally, for the executive function domain, mixed-model ANOVA indicated a significant effect for Trail, F(1,91) ¼ 8.9,

p , .001, h2p = .082, Cluster, F(1,91) ¼ 141.7, p , .001, h2

p = .609, and a significant Trail by Cluster interaction effect,

F(1,91) ¼ 8.5, p , .001, h2p = .271. As can be seen from Table 3, the interaction effect was caused primarily by the

larger than expected discrepancy between Parts A and B for the Impaired cluster compared with the others. Post hoc analy-

sis indicated that in general, the Impaired cluster differed from the other clusters on Parts A and B, although the other

clusters did not differ from each other.

Behavioral differences among the TBI clusters. Descriptive statistics for the BASC subscales are presented in Table 4.

Separate MANOVAs were conducted for the Parent, Teacher, and Self-report ratings, where cluster membership served as

the between-subjects variable and BASC scores were the dependent variables. Additionally, scale and composite scores

were analyzed separately given the overlap between composite and scale scores. MANOVA for the PRSs indicated an

overall significant effect, F(12,387) ¼ 2.95, p ¼ .01, h2p = .064, although the MANOVA for the composite scores was not sig-

nificant, F(16,528) ¼ 1.41, p ¼ .13, h2p = .041. The largest effect was for the Hyperactivity and Conduct scales. Post hoc ana-

lyses indicated the Impaired cluster exhibited more behavioral problems than the other clusters, who did not differ from each

other.

MANOVA for the Teacher scale ratings indicated an overall significant effect, F(16,156) ¼ 3.48, p , .001, h2p = .263, as

did the MANOVA for the Teacher composite scores, F(20,152) ¼ 2.95, p , .001, h2p = .279. Significant group differences

were present among the clusters on all of the scale scores, and all but one of the composite scores (internalizing problems),

with the Impaired cluster again exhibiting more behavioral disturbances than the other clusters.

MANOVA for the Self-Report composite scores indicated no significant differences among the clusters, F(12,318) ¼ 1.74,

p ¼ .06, h2p = .062.

Discussion

The current study provides a number of findings relevant to neurocognitive functioning in children with TBI, particularly

with regard to heterogeneity. Most notably, results support the notion that TBI produces marked heterogeneity in

D.N. Allen et al. / Archives of Clinical Neuropsychology 25 (2010) 618–633 627

at UN

LV

University L

ibraries on September 27, 2012

http://acn.oxfordjournals.org/D

ownloaded from

neurocognitive function, so that there is no profile that best typifies these children and adolescents, although there do appear to

be subgroups within this diverse population that might be characterized by performance profiles from neuropsychological tests

that assess memory and attention/concentration abilities. Evidence supporting the validity of these subgroups was provided not

only by comparisons on clinical, neuropsychological, and behavioral variables, but also through comparison with clusters

identified in a matched normal sample. With regard to the latter point, the need for direct comparisons of TBI and normal

cluster solutions has long been suggested (Crosson et al., 1990), in order to determine if there are profiles unique to TBI

that represent more than normal or expected variation in test performance among individuals, and thus would be attributable

to brain injury. Results indicated that the TBI clusters differed in patterns of performance which was accounted for largely by

discrepancies in verbal and nonverbal memory abilities, or a relative decrease in attention/concentration. There was also evi-

dence for a group that exhibited severe impairment, as well as another that exhibited average performance. These TBI clusters

stood in contrast to the four clusters identified in the SS group, which were differentiated primarily by level of performance,

with pattern of performance differences accounted for primarily by the tendency toward reduced variability in the SS group on

the DRI compared with the TBI group (SD’s ¼ 8.8 and 12.9, respectively; Table 1). Thus, the current findings support the con-

tention that TBI results in patterns of memory and attention deficits that vary markedly from those observed in normal popu-

lations and these patterns are not easily accounted for by intra-individual variation in test performance that is often observed in

nonbrain injured populations. Differences in profile and number of clusters are also consistent with prior studies that have

undertaken separate TBI and SS analyses on, for example, the Wechsler Intelligence Scales (e.g., compare Donders &

Warschausky, 1997, and Donders, 1996).

Table 4. Parent, teacher, and self-report behavioral reports of the five-cluster solution

BASC Scores Clusters Total F-value p-value Tukey h2p

Average Attention Visual Verbal Impaired

Mean SD Mean SD Mean SD Mean SD Mean SD Mean SD

Parent Rating

Subscales

n 37 26 14 42 15 134

Hyperactivity 53.5 16.2 52.4 13.4 60.4 18.9 56.7 16.0 63.9 19.7 56.2 16.5 1.7 .16 – .050

Attention 58.4 12.8 55.0 13.2 61.9 11.7 59.0 10.7 72.4 9.6 59.9 12.6 5.7 .000 AtAvVrVs, I .149

Conduct 52.8 15.5 51.2 15.1 57.7 18.7 51.1 13.6 64.8 22.5 53.8 16.5 2.4 .05 VrAtAvVs, VsI .071

Composites

n 37 25 14 47 14 137

Externalizing 55.0 17.1 52.7 15.0 60.1 20.2 54.6 14.6 66.4 21.7 56.1 17.0 1.9 .11 – .055

Internalizing 53.9 12.0 51.9 10.9 55.9 17.2 55.1 14.2 50.6 11.5 53.8 13.1 0.5 .72 – .016

Behavioral 56.1 15.1 53.7 13.5 60.2 20.1 57.9 16.3 64.4 16.6 57.6 16.0 1.2 .32 – .035

Adaptive Skills 44.7 11.9 45.2 13.9 44.0 9.5 44.0 11.1 41.8 7.0 44.2 11.3 0.2 .93 – .007

Teacher Rating

Subscales

n 7 8 6 17 6 44

Hyperactivity 50.1 12.8 45.8 8.8 58.5 13.0 48.1 7.8 63.2 11.6 51.5 11.4 3.8 .01 AtVrAv, VsI .283

Attention 50.3 7.0 50.9 7.2 58.5 11.3 58.2 9.9 75.0 4.1 58.0 11.3 8.4 .000 AvAtVrVs, I .464

Conduct 47.7 7.3 46.4 2.8 44.5 2.3 45.7 4.8 66.0 16.7 48.8 10.0 9.5 .000 VsVrAtAv, I .493

Learning 47.0 4.6 50.5 10.5 58.2 8.3 56.5 8.9 79.2 7.7 57.2 12.6 13.8 .000 AvAtVsVr, I .587

Composites

n 7 8 6 17 6 44

Externalizing 49.0 10.6 45.6 6.0 50.5 6.3 46.4 5.7 64.7 13.6 49.7 9.9 6.4 .000 AtVrVsAv, I .395

Internalizing 48.4 8.6 50.0 12.2 63.5 12.0 51.0 7.3 51.3 9.7 52.2 10.2 2.6 .052 – .209

School 48.6 5.4 50.6 8.3 58.5 9.7 57.7 9.5 78.3 5.8 57.9 12.1 12.6 .000 AvAtVsVr, I .564

Behavioral 49.7 11.6 48.0 9.5 59.5 12.3 50.9 6.3 62.8 6.4 53.0 9.9 3.8 .01 AtAvVr, I .281

Adaptive Skills 50.7 8.9 49.8 10.4 43.8 8.7 47.1 10.0 31.2 6.2 45.5 10.8 4.7 .004 AtAvVrVs, I .323

Self-Report

Composites

n 33 23 11 33 12 112

School 47.2 9.5 49.2 9.7 54.6 10.7 50.4 11.0 57.3 12.0 50.4 10.7 1.3 .26 – .048

Clinical 48.8 12.0 48.4 7.7 50.7 10.4 48.5 7.9 46.4 7.4 48.5 9.3 0.2 .93 – .011

Personal 50.9 11.0 51.5 7.8 49.0 10.6 46.3 11.0 46.1 7.6 49.0 10.2 1.5 .21 – .054

Emotional 48.0 11.1 48.5 8.7 51.9 12.3 51.1 9.1 49.3 6.8 49.5 9.7 0.6 .64 – .023

Notes: Av ¼ Average Cluster; At ¼ Attention Cluster; Vr ¼ Verbal Cluster; Vs ¼ Visual Cluster; I ¼ Impaired Cluster.

628 D.N. Allen et al. / Archives of Clinical Neuropsychology 25 (2010) 618–633

at UN

LV

University L

ibraries on September 27, 2012

http://acn.oxfordjournals.org/D

ownloaded from

Although the number of clusters and profile differences are strongly influenced by the measures that are entered into the

cluster analysis (Malec et al., 1993), there is some consistency across studies of adults and children with moderate to

severe TBI which may support the utility of neuropsychological tests in classifying injury severity. For example, most

studies identify average and impaired clusters. Other studies that have examined verbal memory abilities find that some chil-

dren with TBI exhibit deficits in various aspects of memory functioning and others do not. Studies of attention also identify

clusters of children with selective deficits in some aspects of attention. In this study, nonverbal memory tests were included and

a group was also identified with selective deficits in this area. However, one objection to the results of cluster analytic studies is

that the replicability and thus generalizability of cluster solutions are not typically assessed. One study that compared cluster

solutions derived from two different TBI samples that were assessed using the same tests provided strong support for the stab-

ility and generalizability of neurocognitive clusters identified across different TBI groups (Curtiss et al., 2001). Another study

that investigated the stability of clusters derived using different tests (abstraction vs. IQ) in 221 patients with schizophrenia

identified four clusters for each set of tests, and there was a significant degree of association between the solutions, although

this association was weak (Goldstein, Allen, & Seaton, 1998). Thus, although there is support for the stability of clusters across

populations and possibly across tests, test selection remains a significant determining factor in the number and profile of the

obtained clusters, which highlights the need for selection of tests that are directly relevant to the types of cognitive deficits

exhibited by the population under investigation. In the current study, measures of memory and attention/concentration were

selected for cluster analysis because deficits in these areas are commonly reported following TBI.

With regard to the specific clusters, the Average cluster obtained TOMAL Index scores ranging from 94.0 to 100.7. We do

not refer to the Average cluster as “normal” even though their performance is in the average range, because it is likely that some

reduction in neurocognitive abilities resulting from TBI occurred in these children, and so they should not be considered

normal. Donders and Warschausky (1997) found that their highest performing WISC-III TBI cluster had a higher socioeco-

nomic background, which may indicate that while still average in their performance, some decline from premobid levels of

functioning had occurred. A similar observation has been made for some individuals with schizophrenia who perform in

the average range on neuropsychological tests (Palmer et al. 1997), but nevertheless exhibit subtle neurocognitive deficits con-

sistent with the presence of schizophrenia (Allen, Goldstein, & Warnick, 2003). These patients have been referred to as “neu-

ropsychologically normal” or alternatively “high cognitive functioning,” akin to terminology used to describe autism (e.g.,

Goldstein et al., 2008). These high cognitive functioning patients with schizophrenia are of some interest since preserved neu-

rocognitive function is associated with better outcomes, and less severe impairment is associated with decreased neuroana-

tomic abnormalities (Allen et al., 2000; Wexler et al., 2009). The Average cluster in this study may also exhibit more

subtle signs of brain damage, as indicated by the relative decrease in processing speed inferred by relatively poorer perform-

ance on the Digit Symbol/Coding subtest compared to the other WISC subtests (Allen et al., 2010; Donders & Janke, 2008).

Whether the nomenclature of “high functioning” applied to schizophrenia and autism would also be a useful descriptor in clas-

sifying childhood TBI awaits further investigation, although it is apparent that more severe deficits have been linked to poorer

outcomes in TBI, in this study and in others.

The Impaired cluster obtained TOMAL Index scores ranging between 56.7 and 67.7, and consistently performed more

poorly than the other clusters on many of the IQ, neuropsychological, and achievement variables, and also exhibited more

severe behavioral disturbances and poorer adaptive functioning than the other clusters. There were no marked differential pat-

terns of impairment across the neuropsychological domains assessed as external validity variables, which is consistent with the

overall picture of severe impairment. This impairment was apparent not only on neurocognitive measures but also measures of

achievement, which are typically considered less susceptible to decline following TBI and other acquired brain disorders, and

can be used as an indicator of premorbid function in some cases. However, the current findings suggest that for this impaired

cluster performance on achievement tests is also diminished, which is consistent with a preliminary report of 170 children with

TBI, who performed significantly worse on many of the WJ III scales compared to the SS mean (Stolberg et al., 2009).

Elevations in parent ratings of attention and conduct problems were also apparent. Similarly, teacher ratings indicated increased

problems with hyperactivity, attention, conduct and learning, as well as increased externalizing, school and behavioral pro-

blems, and decreased adaptive skills. Thus, the Impaired cluster identified in this study and exhibits disturbances across a

broad range of neurocognitive, achievement, and behavioral variables and is also expected to require the most intensive treat-

ment and rehabilitative services during recovery.

There was also a cluster that obtained its lowest score on the Attention Concentration Index, although it also exhibited a

decrease in performance on the TOMAL Nonverbal Memory Index. This cluster was similar to the Average cluster in

many respects, including performance on the IQ and achievement tests, as well as on tests of motor speed and executive func-

tion. Expected decrements in performance on tests of attention, particularly the CPT and Wechsler Digit Span, were not appar-

ent. Also, expected deficits in attention on the behavioral ratings were not apparent, which may reflect a more general lack of

association between neuropsychological and behavioral measures of attention (Dennis et al., 2001; Naglieri, Goldstein,

D.N. Allen et al. / Archives of Clinical Neuropsychology 25 (2010) 618–633 629

at UN

LV

University L

ibraries on September 27, 2012

http://acn.oxfordjournals.org/D

ownloaded from

Delauder, & Schwebach, 2005; Young & Gudjonsson, 2005). It did appear that compared with the Average cluster, the

Attention cluster exhibited somewhat poorer performance on tests of language and visuoperceptual abilities. This later

finding may be associated with the relative decrease in performance on the TOMAL Nonverbal Memory Index, which has

previously been shown to have significant associations with the Perceptual Reasoning Index from the WISC-IV (Allen

et al., 2010), suggesting its reliance on visuoperceptual and visual reasoning processes. However, given the lack of associated

decrements in attention on neuropsychological and behavioral measures, and the decrease in the TOMAL Nonverbal Memory

index, this cluster should not be viewed as a “pure” attention cluster, and its external validity is questionable.

Verbal and Nonverbal clusters were also identified, with the Verbal Cluster obtaining index scores ranging from 70.8

(Verbal Memory Index) to 78.6, and the Visual cluster obtaining index scores ranging from 65.6 (Nonverbal Memory

Index) to 82.8. Verbal memory tests have been the most extensively examined in cluster analytic studies in children with

TBI, and the current results provide further evidence for a cluster that is differentiated by this ability. Measures of nonverbal

learning and memory have not been included in prior cluster analytic studies of children with TBI, but based on the current

results there also appears to be a subgroup of children with TBI who are characterized by relatively poor performance on

measures of this domain. With regard to external validity, the Verbal and Visual clusters typically performed worse than

the Average and Attention clusters and better than the Impaired cluster on the neurocognitive variables. In contrasting the

Verbal and Visual clusters on the IQ variables, the Verbal cluster exhibits the expected dissociation between Vocabulary

and Block Design, although this pattern in less pronounced in the Visual cluster. However, the Verbal cluster did not differ

from the Visual cluster on the OWLS, which is a measure of listening comprehension. Consistent with neurocognitive data,

the Visual cluster scored lower on the perceptual organization domain, without any distinct discrepancies in other areas.

Behaviorally, few differences were present, although the Visual cluster did exhibit elevations on BASC parent ratings of

Conduct and teacher ratings of Hyperactivity, similar to those observed in the Impaired cluster. These differences between

the Visual and Verbal clusters provide preliminary support for their external validity, although some differences were not

as robust as might have expected.

With regard to external validity of the TBI clusters on clinical and demographic variables, similar to prior studies, we found

that only severity of coma significantly differentiated clusters, and clusters were not differentiated on gender, ethnicity, cause of

injury, injury-testing interval, or age at testing (Crosson et al., 1990; Donders & Warschausky, 1997). Contrary to hypotheses, age

at injury and time elapsed from injury to assessment did not differ among clusters, which appears inconsistent with longitudinal

studies indicating a developmental influence on how TBI effects outcome (e.g., Verger et al., 2001). Few studies have investigated

associations between age or developmental level at the time of TBI and outcome, but existing longitudinal studies suggest that

earlier age of injury is associated with poorer outcomes including slower development of attention, motor, and verbal and spatial

skills (Anderson, Catroppa, Morse, et al., 2000, 2004; Anderson & Moore, 1995; Duval et al. 2008; Kriel et al., 1989; Thompson

et al., 1994). Failure to find age differences among our clusters was likely a result of the relatively brief time between injury and

assessment compared with other studies (e.g., Hawley, 2004; Millis et al., 2001). Whether clusters identified in this study will be

useful in predicting longitudinal outcomes awaits future research that incorporates longer time periods between injury and assess-

ment. However, one longitudinal study of neurocognitive, adaptive, behavioral, and academic functioning of children with mod-

erate to severe TBI who were classified according to the longitudinal patterns of outcomes, found that those with severe TBI had

an increased likelihood of experiencing poorer outcomes, functional deficits, and behavioral deterioration an average of 4 years

after injury (Fay et al., 2009). It also remains to be seen whether injury severity classified using neurocognitive tests such as the

TOMAL would provide better prediction of long-term outcomes than severity classification based on the GCS or other similar

techniques, or whether comparable variability in course and outcome would be noted.

To our knowledge, this is the first study to examine differences in behavior among neurocognitive clusters in children and

adolescents with TBI using a well-standardized multimethod behavioral assessment, despite the observation that behavioral

disturbances are common following TBI (McKinlay et al., 2009; Yeates et al., 2004). The results do not generally support

a profile of behavioral disturbance that is differentiated based on membership in neurocognitive clusters in most cases.

Rather, severity of neurocognitive impairment was more generally associated with disturbances in behavior and poorer adap-

tation, with the most severely impaired neurocognitive cluster exhibiting clinical elevations on a number of the BASC subscale

and composite scores. Interestingly, there were differences between the type of behavioral report, with teacher reports consist-

ently demonstrating the most differences among clusters. Although the findings are limited by a relatively small number of

teacher ratings (n ¼ 44), results are consistent with a recent preliminary study of children with TBI using the BASC-2 that

found increased teacher endorsement of externalizing and school-related problems more frequently than internalizing pro-

blems, with the greatest elevations on the hyperactivity, attention, and learning problems subscales (Thaler, Terranova,

Reynolds, Mayfield, & Allen 2010). Also, although behavioral problems following pediatric TBI are typically evaluated via

parent reports (Yeates & Taylor, 2006), with less research available for teacher ratings, there do appear to be inconsistencies

between parent and teacher ratings (Gross, Fogg, Garvey, & Julion, 2004; Lee, Elliott, & Barbour, 1994; McAuley, 2000;

630 D.N. Allen et al. / Archives of Clinical Neuropsychology 25 (2010) 618–633

at UN

LV

University L

ibraries on September 27, 2012

http://acn.oxfordjournals.org/D

ownloaded from

McConaughy, Stanger, & Achenbach, 1992). These inconsistencies are likely influenced by contextual factors relating to

setting (e.g., hyperactivity may cause less disruption in the home than in the classroom), attribution of problem behaviors

to internalizing or externalizing factors (Max et al., 1998), or by differences between teachers and parents referent comparisons

to normally developing children. Because of these differences, teachers may be more accurate in rating hyperactivity, whereas

parents are most reliable in rating conduct disorders, oppositional behavior, and internalizing behaviors (Loeber, Green, &

Lahey, 1990; Yeates & Taylor, 2006). Thus, the suggestion that teacher reports provide unique insights to childhood behavioral

disorders (Randazzo, Landsverk, & Ganger, 2003; Thaler, Terranova, et al., 2010) is consistent with the current findings and

highlights the importance of supplementing parental reports with those of teachers when evaluating behavioral disturbances in

children with TBI.

The current study has a number of limitations including that some clusters contained a limited number of participants, which

may have decreased statistical power to detect cluster differences on some external validity variables. Also, because this was a

study of a convenience sample referred for clinical evaluation, there was incomplete data for some tests (e.g., BASC TRS),

which may limit generalizability of findings. In a prospective investigation, alternative measures may have been selected

and some domains would have been more thoroughly evaluated (e.g., executive function). Additionally, children with mild

TBI were not included. Their inclusion may have produced a high average cluster that would contain those children with

above average abilities premorbidly, who did not experience a substantial decrease in neurocognitive abilities as a result of

TBI. Finally, while a revised version of the TOMAL has been recently published (TOMAL-2; Reynolds & Voress, 2007),

similar patterns of performance are anticipated on the TOMAL-2, because no new items or subtests were introduced, and

the child SS remained largely the same. Future studies may attempt to determine stability of this cluster solution across differ-

ent populations of children with TBI, as well as whether neurocognitive classifications of TBI, such as the one reported here,

will be more useful in predicting longer-term outcomes in comparison to currently existing and commonly used classification

methods such as the GCS. However, the current results do suggest that deficits in memory and attention abilities, particularly

for patients who exhibit severe impairment in these areas, are associated with disturbances across a wider range of neurocog-

nitive and behavioral domains, and so may prove to be clinically useful in predicting outcomes, anticipating adjustment pro-

blems, and planning educational and rehabilitative interventions designed to remediate deficits and improve functional

outcomes following TBI in children.

Conflict of Interest

None declared.

Acknowledgements

Standardization Sample (SS) data was from the Test of Memory and Learning (TOMAL) standardization sample. Copyright

1994 by Pro-Ed. Used with permission. All rights reserved. We thank the TOMAL publisher for allowing access to the stan-

dardization data for this study.

References

Aldenderfer, M., & Blashfield, R. (1984). Cluster analysis. Newbury Park: Sage Publications.

Alexander, A. I., & Mayfield, J. (2005). Latent factor structure of the Test of Memory and Learning in a pediatric traumatic brain injured sample: Support for a

general memory construct. Archives of Clinical Neuropsychology, 20, 587–598.

Allen, D. N., Goldstein, G., & Warnick, E. (2003). A consideration of neuropsychologically normal schizophrenia. Journal of the International

Neuropsychological Society, 9, 56–63.

Allen, D. N., Seaton, B. E., Goldstein, G., Sanders, R. D., Gurklis, J. A., Jr, Peters, J. L., et al. (2000). Neuroanatomic differences among cognitive and symptom

subtypes of schizophrenia. Journal of Nervous and Mental Disease, 188, 381–384.

Allen, D. N., Strauss, G. P., Kemtes, K. A., & Goldstein, G. (2007). Hemispheric contributions to nonverbal abstract reasoning and problem solving.

Neuropsychology, 21, 713–720.

Allen, D. N., Thaler, N. S., Donohue, B., & Mayfield, J. (2010). WISC-IV profiles in children with traumatic brain injury: Similarities to and differences from

the WISC-III. Psychological Assessment, 22 (1), 57–64.

Anderson, V., Catroppa, C., Morse, S., Haritou, F., & Rosenfeld, J. (2000). Recovery of intellectual ability following TBI in childhood: Impact of injury sever-

ity and age at injury. Pediatric Neurosurgery, 32, 282–290.

Anderson, V., & Moore, C. (1995). Age at injury as a predictor of outcome following pediatric head injury: A longitudinal perspective. Child Neuropsychology,

1, 187–202.

Anderson, V. A., Catroppa, C., Rosenfeld, J., Haritou, F., & Morse, S. A. (2000). Recovery of memory function following traumatic brain injury in pre-school

children. Brain Injury, 14, 679–692.

D.N. Allen et al. / Archives of Clinical Neuropsychology 25 (2010) 618–633 631

at UN

LV

University L

ibraries on September 27, 2012

http://acn.oxfordjournals.org/D

ownloaded from

Anderson, V. A., Morse, S. A., Catroppa, C., Haritou, F., & Rosenfeld, J. V. (2004). Thirty month outcome from early childhood head injury: A prospective

analysis of neurobehavioural recovery. Brain: A Journal of Neurology, 127, 2608–2620.

Babikian, T., & Asarnow, R. (2009). Neurocognitive outcomes and recovery after pediatric TBI: Meta-analytic review of the literature. Neuropsychology, 23,

283–296.

Beery, K. E., & Beery, N. A. (2004). The Beery–Buktenica Developmental Test of Visual-Motor Integration: Administration, scoring, and teaching manual

(5th ed.). Minneapolis, MN: NCS Pearson.

Carrow-Woolfolk, E. (1996). Oral and Written Language Scales. Circle Pines, MN: American Guidance Service.

Chan, R. C. K., Hoosain, R., Lee, T. M. C., Fan, Y. W., & Fong, D. (2003). Are there sub-types of attentional deficits in patients with persisting post-concussive

symptoms? A cluster analytical study. Brain Injury, 17, 131–148.

Conners, C. K., & MHS Staff (Eds.). (2000). Conners’ Continuous Performance Test II: Computer program for windows technical guide and software manual.

North Tonwanda, NY: Multi-Health Systems.

Crawford, J. R., Garthwaite, P. H., & Johnson, D. A. (1997). WAIS-R subtest pattern clusters in closed-head-injured and healthy samples. Clinical

Neuropsychologist, 11, 249–257.

Crosson, B., Greene, R. L., Roth, D. L., Farr, S. P., & Adams, R. L. (1990). WAIS-R pattern clusters after blunt-head injury. The Clinical Neuropsychologist, 4,

253–262.

Curtiss, G., Vanderploeg, R. D., Spencer, J., & Salazar, A. M. (2001). Patterns of verbal learning and memory in traumatic brain injury. Journal of the

International Neuropsychological Society, 7, 574–585.

Demery, J. A., Pedraza, O., & Hanlon, R. E. (2002). Differential Profiles of Verbal Learning in Traumatic Brain Injury. Journal of Clinical and Experimental

Neuropsychology, 24 (6), 818–827.

Dennis, M., Guger, S., Roncadin, C., Barnes, M., & Schachar, R. (2001). Attentional-inhibitory control and social-behavioral regulation after childhood closed

head injury: Do biological, developmental, and recovery variables predict outcome? Journal of the International Neuropsychological Society, 7, 683–692.

Donders, J. (1996). Cluster subtypes in the WISC-III standardization sample: Analysis of factor index scores. Psychological Assessment, 8, 312–318.

Donders, J. (1999). Cluster subtypes in the standardization sample of the California Verbal Learning Test—Children’s version. Developmental

Neuropsychology, 16, 163–175.

Donders, J. (2008). Subtypes of learning and memory on the California Verbal Learning Test—Second Edition (CVLT-II) in the standardization sample.

Journal of Clinical and Experimental Neuropsychology, 30 (7), 741–748.

Donders, J., & Janke, K. (2008). Criterion validity of the Wechsler Intelligence Scale for Children—Fourth Edition after pediatric traumatic brain injury.

Journal of the International Neuropsychological Society, 14, 651–655.

Donders, J., & Warschausky, S. (1997). WISC-III factor index score patterns after traumatic head injury in children. Child Neuropsychology, 3, 71–78.

Donders, J., Zhu, J., & Tulsky, D. (2001). Factor index score patterns in the WAIS-III standardization sample. Assessment, 8, 193–203.

Duval, J., Braun, C. M. J., Montour-Proulx, I., Daigneault, S., Rouleau, I., & Begin, J. (2008). Brain lesions and IQ: Recovery versus decline depends on age of

onset. Journal of Child Neurology, 23, 663–668.

Everitt, B. S., Landau, S., & Leese, M. (2001). Cluster analysis (4th ed.). London: Arnold.

Fay, T. B., Yeates, K. O., Wade, S. L., Drotar, D., Stancin, T., & Taylor, H. G. (2009). Predicting longitudinal patterns of functional deficits in children with

traumatic brain injury. Neuropsychology, 23, 271–282.

Gil, A. M. (2003). Neurocognitive outcomes following pediatric brain injury: A developmental approach. Journal of School Psychology, 41, 337–353.

Goldstein, G., Allen, D. N., & Caponigro, J. M. (2010). A retrospective study of heterogeneity in neurocognitive profiles associated with traumatic brain injury.

Brain Injury, 24 (4), 625–635.

Goldstein, G., Allen, D. N., Minshew, N. J., Williams, D. L., Volkmar, F., Klin, A., et al. (2008). Structure of intelligence in children and adults with high

functioning autism. Neuropsychology, 22, 301–311.

Goldstein, G., Allen, D. N., & Seaton, B. E. (1998). A comparison of clustering solutions for cognitive heterogeneity in schizophrenia. Journal of the

International Neuropsychological Society, 4 (4), 353–362.

Gross, D., Fogg, L., Garvey, C., & Julion, W. (2004). Behavior problems in young children: An analysis of cross-informant agreements and disagreements.

Research in Nursing and Health, 27, 413–425.

Hair, J. F., Black, B., Babin, B., Anderson, R. E., & Tatham, R. (1992). Multivariate data analysis with reading (6th ed.). New Jersey: Prentice Hall.

Hawley, C. A. (2003). Reported problems and their resolution following mild, moderate and severe traumatic brain injury amongst children and adolescents in

the UK. Brain Injury, 17, 105–129.

Hawley, C. A. (2004). Behaviour and school performance after brain injury. Brain Injury, 18, 645–659.

Kriel, R. L., Krach, L. E., & Panser, L. A. (1989). Closed head injury: Comparison of children younger and older than 6 years of age. Pediatric Neurology, 5 (5),

296–300.

Landis, J. R., & Koch, G. G. (1977). The measurement of observer agreement for categorical data. Biometrics, 33 (1), 159–174.

Lee, S. W., Elliott, J., & Barbour, J. D. (1994). A comparison of cross-informant behavior ratings in school-based diagnosis. Behavioral Disorders, 19 (2),

87–97.

Loeber, R., Green, S. M., & Lahey, B. B. (1990). Mental health professionals’ perception of the utility of children, mothers, and teachers as informants on

childhood psychopathology. Journal of Clinical Child Psychology, 19 (2), 136–143.

Malec, J. F., Machulda, M. M., & Smigielski, J. S. (1993). Cluster analysis of neuropsychological test results among patients with traumatic brain injury (TBI):

Implications for a model of TBI-related disability. Clinical Neuropsychologist, 7, 48–58.

Max, J., Koele, S., Smith, W., Sato, Y., Lindgren, S., Robin, D., et al. (1998). Psychiatric disorders in children and adolescents after severe traumatic brain

injury: A controlled study. Journal of the American Academy of Child and Adolescent Psychiatry, 37 (8), 832–840.

McAuley, C. (2000). Children’s adjustment over time in foster care: Cross-informant agreement, stability and placement disruption. British Journal of Social

Work, 30, 91–107.

McConaughy, S. H., Stanger, C., & Achenbach, T. M. (1992). Three-year course of behavioral/emotional problems in a national sample of 4- to 16-year-olds:

I. Agreement among inform-ants. Journal of the American Academy of Child and Adolescent Psychiatry, 31, 932–940.

632 D.N. Allen et al. / Archives of Clinical Neuropsychology 25 (2010) 618–633

at UN

LV

University L

ibraries on September 27, 2012

http://acn.oxfordjournals.org/D

ownloaded from

McKinlay, A., Grace, R., Horwood, J., Fergusson, D., & MacFarlane, M. (2009). Adolescent psychiatric symptoms following preschool childhood mild trau-

matic brain injury: Evidence from a birth cohort. The Journal of Head Trauma Rehabilitation, 24, 221–227.

Miller, L. J., & Donders, J. (2003). Prediction of educational outcome after pediatric traumatic brain injury. Rehabilitation Psychology, 48, 237–241.

Millis, S. R., & Ricker, J. H. (1994). Verbal learning patterns in moderate and severe traumatic brain injury. Journal of Clinical and Experimental

Neuropsychology, 16, 498–507.

Millis, S. R., Rosenthal, M., Novack, T. A., Sherer, M., Nick, T. G., Kreutzer, J. S., et al. (2001). Long-term neuropsychological outcome after traumatic brain

injury. The Journal of Head Trauma Rehabilitation, 16, 343–355.

Morris, R., Blashfield, R. K., & Satz, P. (1981). Neuropsychology and cluster analysis: Potentials and problems. Journal of Clinical Neuropsychology, 3,

79–99.

Mottram, L., & Donders, J. (2006). Cluster subtypes on the California Verbal Learning Test—Children’s version after pediatric traumatic brain injury.

Developmental Neuropsychology, 30, 865–883.

Murji, S., Rourke, S. B., Donders, J., Carter, S. L., Shore, D., & Rourke, B. P. (2003). Theoretically derived CVLT subtypes in HIV-1 infection: Internal and

external validation. Journal of the International Neuropsychological Society, 9, 1–16.

Naglieri, J. A., Goldstein, S., Delauder, B. Y., & Schwebach, A. (2005). Relationships between the WISC-III and the Cognitive Assessment System with

Conners’ rating scales and continuous performance tests. Archives of Clinical Neuropsychology, 20, 385–401.

Palmer, B. W., Heaton, R. K., Paulsen, J. S., Kuck, J., Braff, D., Harris, M. J., et al. (1997). Is it possible to be schizophrenic yet neuropsychologically normal?

Neuropsychology, 11, 437–446.

Randazzo, K., Landsverk, J., & Ganger, W. (2003). Three informants’ reports of child behavior: Parents, teachers, and foster parents. Journal of the American

Academy of Child and Adolescent Psychiatry, 42, 1343–1350.

Rassovsky, Y., Satz, P., Alfano, M. S., Light, R. K., Zaucha, K., McArthur, D. L., et al. (2006). Functional outcome of TBI I: Neuropsychological, emotional,

and behavioral mediators. Journal of Clinical and Experimental Neuropsychology, 28, 567–580.

Reitan, R. M., & Wolfson, D. (1993). The Halstead-Reitan Neuropsychological Test Battery: Theory and clinical interpretation (2nd ed.). Tucson, AZ:

Neuropsychology.

Reynolds, C. R., & Bigler, E. D. (1994a). Test of Memory and Learning. Austin, TX: Pro-Ed.

Reynolds, C. R., & Bigler, E. D. (1994b). Test of Memory and Learning: Examiner’s manual. Austin, TX: Pro-Ed.

Reynolds, C. R., & Bigler, E. D. (1996). Factor structure, factor indexes, and other useful statistics for interpretation of the Test of Memory and Learning

(TOMAL). Archives of Clinical Neuropsychology, 11, 29–43.

Reynolds, C. R., & Ford, L. (1994). Comparative Three-Factor Solutions of the WISC-III and WISC-R at 11 Age Levels Between 6-l/2 and 16-l/2 Years.

Archives of Clinical Neuropsychology, 9, 553–570.

Reynolds, C. R., & Kamphaus, R. W. (1992). Behavior Assessment System for Children. Circle Pines, MN: American Guidance Service.

Reynolds, C. R., & Voress, J. K. (2007). Test of Memory and Learning (2nd ed.). Austin, TX: Pro-Ed.

Roman, M. J., Delis, D. C., Willerman, L., Magulac, M., Demadura, T. L., de la Pena, J. L., et al. (1998). Impact of pediatric traumatic brain injury on com-

ponents of verbal memory. Journal of Clinical and Experimental Neuropsychology, 20, 245–258.

Schwartz, L., Taylor, H. G., Drotar, D., Yeates, K. O., Wade, S. L., & Stancin, T. (2003). Long-term behavior problems following pediatric traumatic brain

injury: Prevalence, predictors, and correlates. Journal of Pediatric Psychology, 28, 251–263.

Stolberg, P., Terranova, J., Jones, P. W., Allen, D. N., & Mayfield, J. (2009). Performance on the Woodcock–Johnson Test of Achievement of individuals with

traumatic brain injury [Abstract]. Archives of Clinical Neuropsychology, 24 (5), 476.

Teasdale, G., & Jennett, B. (1974). Assessment of coma and impaired consciousness: A practical scale. Lancet, 2 (7872), 81–84.

Thaler, N. S., Allen, D. N., McMurray, J., & Mayfield, J. (2010). Sensitivity of the Test of Memory and Learning to attention and memory deficits in children

with ADHD. Clinical Neuropsychologist, 24 (2), 246–264.

Thaler, N. S., Terranova, J., Reynolds, C. R., Mayfield, J., & Allen, D. N. (2010). Teacher reported behavioral disturbances in children with TBI: An

Examination of the BASC-2. Presented at the American College of Professional Neuropsychology Annual Meeting, Feb. 26–28, Las Vegas, NV.

Thompson, N. M., Francis, D. J., Stuebing, K. K., Fletcher, J. M., Ewing-Cobbs, L., Miner, M. E., et al. (1994). Motor, visual-spatial, and somatosensory skills

after closed head injury in children and adolescents: A study of change. Neuropsychology, 8, 333–342.

Tiffin, J. (1948). Purdue Pegboard Test. Lafayette, IN: Lafayette Instrument Company.

van der Heijden, P., & Donders, J. (2003). WAIS-III factor index score patterns after traumatic brain injury. Assessment, 10, 115–122.

Verger, K., Junque, C., Levin, H. S., Jurado, M. A., Perez-Gomez, M., Bartres-Faz, D., et al. (2001). Correlation of atrophy measures on MRI with neurop-

sychological sequelae in children and adolescents with traumatic brain injury. Brain Injury, 15, 211–221.

Wexler, B. E., Zhu, H., Bell, M. D., Nicholls, S. S., Fulbright, R. K., Gore, J. C., et al. (2009). Neuropsychological near normality and brain structure abnorm-

ality in schizophrenia. The American Journal of Psychiatry, 166, 189–195.

Wiegner, S., & Donders, J. (1999). Performance on the California Verbal Learning Test after traumatic brain injury. Journal of Clinical and Experimental

Neuropsychology, 21, 159–170.

Woodcock, R. W., & Johnson, M. B. (1989). Woodcock–Johnson psycho-educational battery – revised. Chicago: Riverside Publishing.

Yeates, K. O., Armstrong, K., Janusz, J., Taylor, H. G., Wade, S., Stancin, T., et al. (2005). Long-term attention problems in children with traumatic brain

injury. Journal of the American Academy of Child and Adolescent Psychiatry, 44 (6), 574–584.

Yeates, K. O., Swift, E., Taylor, H. G., Wade, S. L., Drotar, D., Stancin, T., et al. (2004). Short- and long-term social outcomes following pediatric traumatic

brain injury. Journal of the International Neuropsychological Society, 10, 412–426.

Yeates, K. O., & Taylor, H. G. (2006). Behavior problems in school and their educational correlates among children with traumatic brain injury. Exceptionality,

14, 141–154.

Yeates, K. O., Taylor, H. G., Wade, S. L., Drotar, D., Stancin, T., & Minich, N. (2002). A prospective study of short- and long-term neuropsychological out-

comes after traumatic brain injury in children. Neuropsychology, 16, 514–523.

Young, S., & Gudjonsson, G. (2005). Neuropsychological correlates of the YAQ-S and YAQ-I self- and informant-reported ADHD symptomatology, emotion-

al and social problems and delinquent behaviour. British Journal of Clinical Psychology, 44, 47–57.

D.N. Allen et al. / Archives of Clinical Neuropsychology 25 (2010) 618–633 633

at UN

LV

University L

ibraries on September 27, 2012

http://acn.oxfordjournals.org/D

ownloaded from