Post on 05-May-2023
Memory and Attention Profiles in Pediatric Traumatic Brain Injury
Daniel N. Allen 1,*, Brian D. Leany 1, Nicholas S. Thaler 1, Chad Cross 2, Griffin P. Sutton 1,Joan Mayfield 3
1Department of Psychology, University of Nevada, Las Vegas, NV, USA2Department of Environmental and Occupational Health, Epidemiology and Biostatistics Unit, University of Nevada, Las Vegas, NV, USA
3Our Children’s House at Baylor, Dallas, TX, USA
*Corresponding author at: Neuropsychology Research Program, Department of Psychology, University of Nevada Las Vegas, 4505 Maryland Parkway,
Las Vegas, NV 89154, USA. Tel.: +1-702-895-1379; fax: +1-702-895-0195.
E-mail address: daniel.allen@unlv.edu (D.N. Allen).
Accepted 16 June 2010
Abstract
Traumatic brain injury (TBI) causes heterogeneous patterns of neurocognitive deficits. In an attempt to identify homogenous subgroups
within this heterogeneity, cluster analysis was used to examine memory and attention abilities as measured by the Test of Memory and
Learning (TOMAL) in 300 children, 150 with TBI and 150 matched nonbrain injured controls (standardization sample [SS]). Significant
differences were present between the TBI and the SS groups on all TOMAL subscale and index scores, with the TBI groups performing
approximately 1.3 SD below the SS. Factor analysis of the TOMAL indicated six factors that assessed various aspects of verbal and nonverbal
learning and memory, as well as attention/concentration. Cluster analyses of TOMAL factor scores indicated that a four-cluster solution was
optimal for the SS group, and a five-cluster solution for the TBI group. For the TBI clusters, differences were present for clinical, achieve-
ment, neurocognitive, and behavioral variables, providing some support for the validity of the cluster solution. These findings suggest that
TBI results in unique patterns of neurocognitive impairment that are not accounted for by individual differences in test performance com-
monly observed in normal populations. Additionally, neurocognitive profiles identified using cluster analysis may prove useful for identifying
homogeneous subgroups of children with TBI that are differentiated by a number of important clinical, cognitive, and behavioral variables
associated with treatment and outcomes.
Keywords: Attention; Childhood brain insult; Head injury; Traumatic brain; Injury; Learning and Memory
Introduction
Behavioral disturbances and neurocognitive deficits in attention, learning, and memory commonly occur following trau-
matic brain injury (TBI; Babikian & Asarnow, 2009; Roman et al., 1998; Schwartz, et al., 2003; Yeates et al., 2005).
These neurocognitive deficits are associated with a number of important outcomes (Gil, 2003; Rassovsky et al., 2006).
However, both the severity and patterns of neurocognitive impairment are heterogeneous. Heterogeneity of these neurocogni-
tive and behavioral deficiencies are not reflected in more general approaches to TBI severity classification based on, for
example, open and closed head injury, length of coma or post-traumatic amnesia, and persisting neurological signs (Reitan
& Wolfson, 1993), and it may be that neurocognitive tests could provide more meaningful classification by identifying sub-
groups of children with TBI, who differ not only in the patterns of neurocognitive deficits, but also exhibit differences in
other domains, such as behavioral abnormalities, academic achievement, and functional outcomes. Cluster analysis has
been effectively used to identify subgroups of patients who exhibit differing patterns of neurocognitive impairment in a
variety of psychiatric and neurological disorders, including TBI (Allen et al., 2000; Crawford, Garthwaite, & Johnson,
1997; Demery, Pedraza, & Hanlon, 2002; Wiegner & Donders, 1999). Studies of adults with TBI indicate the existence of
discrete TBI subgroups that differ with regard to both pattern and severity of neurocognitive deficits (Chan, Hoosain, Lee,
Fan, & Fong, 2003; Crosson, Greene, Roth, Farr, & Adams, 1990; Goldstein, Allen, & Caponigro, 2010; Malec, Machulda,
Archives of Clinical Neuropsychology 25 (2010) 618–633
# The Author 2010. Published by Oxford University Press. All rights reserved. For permissions, please e-mail: journals.permissions@oxfordjournals.org.
doi:10.1093/arclin/acq051 Advance Access publication on 28 July 2010
at UN
LV
University L
ibraries on September 27, 2012
http://acn.oxfordjournals.org/D
ownloaded from
& Smigielski, 1993; Millis & Ricker, 1994; van der Heijden & Donders, 2003). Fewer studies have examined neurocognitive
heterogeneity in children with TBI, but provide evidence for similar heterogeneity as observed in adults, when the California
Verbal Learning Test for Children and the Wechsler Intelligence Scale for Children are examined (Donders & Warschausky,
1997; Mottram & Donders, 2006).
Although providing substantive support for the presence of neurocognitive heterogeneity in TBI, the results of these studies
are limited in a number of ways, including in some cases, the use of small sample size which limits generalizability, the use of
IQ tests to develop clusters, because IQ tests have limited sensitivity to brain damage (Malec et al., 1993), and the relative
absence of support for the “external validity” of cluster solutions through, for example, examination of cluster differences
on neurocognitive, achievement, or behavioral variables that were not included in the cluster analysis. With regard to this
latter point, associations between cluster membership and post-injury disability in TBI (Malec et al. 1993), as well as differ-
ences in neurocognitive abilities and affective symptomotology in HIV infected individuals (Murji et al., 2003), provide some
evidence that neurocognitive subtypes may have utility in predicting post-injury disability, psychiatric status, and possibly
other outcomes in pediatric TBI. A further limitation is that there has not been a direct comparison between clusters
derived in TBI and nonbrain injured samples in order to establish whether TBI results in different profiles, even though the
importance of such comparisons has been noted for many years (Crosson et al., 1990). Separate cluster analysis of TBI and
standardization sample (SS) data for measures such as the CVLT and Wechsler Intelligence scales (Donders, 1996, 1999,
2008; Donders, Zhu, & Tulsky, 2001) provide support for such differences, but direct comparisons between the groups are
made difficult due to differences among the TBI and normal groups in sample size, demographic variables, as well as other
considerations. Additionally, there has been a tendency for cluster analytic studies to focus on a single ability (e.g., verbal
memory), which constrains the identification of clusters that might reflect meaningful subtypes within TBI, because TBI
may result in a variety of neurocognitive deficits. Finally, in some studies, cluster analysis is used to investigate the psycho-
metric properties of specific tests by, for example, determining base rates of score profiles for a particular instrument in stan-
dardization samples. Although these studies provide valuable psychometric information about specific tests, they do not
address broader questions, such as whether there are common neurocognitive profiles in children with TBI that identify
TBI subgroups/subtypes, as established by external validity indicators, which in turn may predict important outcomes.
Cluster analysis can be used as a taxometric procedure to address such questions in children with TBI, in order to develop
a classification system that may prove useful in understanding not only the common types of neurocognitive profiles that
result from TBI, but also cognitive, clinical, and behavioral abnormalities that characterize each neurocognitive subgroup.
Such classification may, in turn, prove useful for predicting prognosis and outcomes, prescribing treatment, and directing reha-
bilitation. On balance, the existing studies do provide a strong base from which to hypothesize that meaningful subgroups can
be identified in pediatric TBI using neuropsychological tests, even though the validity of these neurocognitive subgroups has
not been examined extensively.
The purpose of the current study was to further investigate neurocognitive heterogeneity in pediatric TBI. We were particu-
larly interested in providing a broader assessment of cognitive abilities than accomplished by using tests of a single ability and
the Test of Memory and Learning (TOMAL; Reynolds & Bigler, 1994a, 1994b) for this purpose. The TOMAL includes tests of
verbal and nonverbal memory, immediate and delayed recall, as well attention/concentration. Because attention and memory
deficits are common sequelae of childhood TBI (Babikian & Asarnow, 2009), and memory deficits are predictive of edu-
cational needs and persists even after extended periods of recovery (Miller & Donders, 2003; Yeates et al., 2002), the
TOMAL should be an excellent tool for examination of TBI subtypes. A second goal was to determine whether differing pat-
terns of learning, memory, and attention/concentration would be evident between our sample of children with TBI and a normal
control sample. A third goal was to determine if the clusters identified in the TBI group differed from each other on important
clinical, cognitive, and behavioral variables, in a manner that might be anticipated based on prior research, as such differences
would provide further support for cluster validity.
With regard to number of clusters, based on prior cluster analyses, we hypothesized that cluster analysis of the TOMAL in
children with TBI would result in at least four clusters, two that were differentiated by the level of performance (average/low
average and impaired) and two additional clusters differentiated by relative strengths in either verbal or nonverbal memory
abilities. Also, because the TOMAL has an Attention/Concentration index (ACI), and given the presence of attention deficits
in TBI, a fifth cluster might also emerge that exhibited selective impairment in this domain. For the normal comparison sample
selected from the TOMAL SS, four clusters were predicted that would be differentiated primarily by the level of performance
(superior, high average, average, low average).
With regard to the external validity of the TBI clusters, comparison among the clusters on demographic, clinical, achieve-
ment, IQ, neuropsychological, and behavioral variables were made in order to determine if the clusters identified in the TBI
group differed in a manner that was consistent with prior research. We did not predict differences between the clusters on
demographic variables, although it was hypothesized that the most severely impaired cluster would have lower Glasgow
D.N. Allen et al. / Archives of Clinical Neuropsychology 25 (2010) 618–633 619
at UN
LV
University L
ibraries on September 27, 2012
http://acn.oxfordjournals.org/D
ownloaded from
Coma Scale (GCS) scores, a shorter time interval between injury and assessment (Mottram & Donders, 2006), and a younger
age at injury compared with the average cluster. We hypothesized younger age at injury in the impaired cluster because longi-
tudinal studies suggest that earlier age of injury is associated with poorer outcomes including slower development of attention,
motor, and verbal and spatial skills (Anderson, Catroppa, Morse, Haritou, & Rosenfeld, 2000; Anderson, Catroppa, Rosenfeld,
Haritou, & Morse, 2000; Anderson & Moore, 1995; Anderson, Morse, Catroppa, Haritou, & Rosenfeld, 2004; Kriel, Krach, &
Panser, 1989; Thompson et al., 1994). Significant differences between the clusters were also hypothesized for intellectual vari-
ables (Mottram & Donders, 2006), with perceptual organization abilities and speed of information processing demonstrating
the largest differences among the clusters, and associated with overall levels of neurocognitive impairment. Neurocognitive
variables assessing attention, perceptual organization, executive, language, and motor performance were predicted to
exhibit differential patterns of impairment that were consistent with domain- and modality-specific patterns identified in the
clusters. For example, individuals exhibiting selective deficits in verbal memory were predicted to also exhibit relative impair-
ment on tests of verbal and language abilities, whereas nonverbal memory impairment was expected to exhibit associations
with visuospatial and perceptual processing deficits (Allen, Thaler, Donohue, & Mayfield, 2010). We have reported prelimi-
nary evidence supporting generalized decrements across achievement test scores in children with TBI (Stolberg, Terranova,
Jones, Allen, & Mayfield, 2009) and thus anticipated overall level of performance differences between clusters, with the
most impaired cluster exhibiting the lowest achievement test scores, although differential patterns of impairment among the
clusters may also be present. Finally, given that difficulties with social behavioral regulation are common following child
TBI (Dennis, Guger, Roncadin, Barnes, & Schachar, 2001; Fay et al., 2009; Hawley, 2003; McKinlay, Grace, Horwood,
Fergusson, & MacFarlane, 2009; Yeates et al., 2004) and that these problems are associated with severity of TBI (e.g.,
Dennis et al., 2001), we predicted that severity of impairment as indicated by cluster membership would be associated with
attention, hyperactivity, and conduct problems in school and at home, as well as increased levels of self-reported psychological
distress.
Materials and Methods
Participants
Participants included 300 children. Of these, 150 had sustained TBI and were selected from a consecutive series of cases
referred for neuropsychological evaluation (TBI group). Definitive evidence of brain damage resulting from TBI was estab-
lished in all cases using appropriate neuroimaging, laboratory, and examinational findings. GCS scores were available for
101 of the participants and indicated that on average, injuries were severe in nature (median ¼ 7). Participants were included
if they had a primary diagnosis of TBI, had been administered the TOMAL as part of a comprehensive neuropsychological
evaluation, and did not have pre-injury neurological or neurodevelopmental disorders. Of the 150 participants selected,
56.3% were Caucasian, 23.4% were African American, 17.2% were Hispanic, 0.8% were Asian American, and 2.3% were
of other ethnicities. The sample was 52.1% male with an average age of 11.7 years (SD ¼ 3.7; range ¼ 5–19). They were
assessed an average of 6.9 months (SD ¼ 3.1; range ¼ 3–21) following injury. Ten (6%) participants had open head injuries,
whereas the others had closed head injuries. The remaining 150 children (SS group) were selected from the TOMAL standar-
dization smaple. SS cases were matched individually to the TBI participants on age and sex. When more than one member of
the SS group matched a TBI participant on age and sex, random selection was used to choose the SS case. For 17 TBI cases,
matches were not available, so these cases were randomly selected from the remaining SS cases. Participants in the SS group
were on average 11.2 years old (SD ¼ 3.1; range ¼ 5–18) and 47.4% were male.
Measures
Test of Memory and Learning (Reynolds & Bigler, 1994a). The TOMAL was developed for children ages 5–19 years and is
composed of 14 subtests, 10 that are core and 4 that are supplementary. Core subtests include Memory for Stories, Facial
Memory, Word Selective Reminding, Visual Selective Reminding, Object Recall, Abstract Visual Memory, Digits
Forward, Visual Sequential Memory, Paired Recall, and Memory for Location. Supplementary subtests include Letters
Forward, Digits Backward, Letters Backward, and Manual Imitation. Subtest scores are used to calculate a number of core
index scores, including a Verbal Memory, Nonverbal Memory, Composite Memory, Delayed Recall, and Attention/
Concentration Indexes. Index scores are standard scores with means of 100 and standard deviations of 15.
Demographic and clinical information. In addition to demographic variables, severity of injury was measured by the GSC
(Teasdale & Jennett, 1974). The GCS was administered either by first responders at the scene of the accident or in the
620 D.N. Allen et al. / Archives of Clinical Neuropsychology 25 (2010) 618–633
at UN
LV
University L
ibraries on September 27, 2012
http://acn.oxfordjournals.org/D
ownloaded from
emergency room when the child was transported to the hospital. The time interval between injury and assessment was used as
an indicator of recovery, with a longer interval indicating more recovery.
Intellectual, achievement, and neurocognitive measures
Wechsler Intelligence Scales (WIS). The WISs were used to assess intelligence. Because children were tested over a period
of more than 7 years and were of different ages, some were administered the WISC-III (n ¼ 116) or WISC-IV (n ¼ 10), and
others the Wechsler Adult Intelligence Scale-Revised (n ¼ 9), or Wechsler Adult Intelligence Scale-Third Edition (n ¼ 7).
Given that versions of the Wechsler Scales share many common subtests, and subtests are designed to measure similar abilities
across age groups, data were combined across the various versions. We analyzed individual subtests that were strong measures
of their representative index scores (Reynolds & Ford, 1994), including Vocabulary (Verbal Comprehension Index), Block
Design (Perceptual Organization Index), Digit Span (Working Memory Index), and Digit Symbol/Coding (Processing
Speed Index). We also examined group differences on Full-Scale IQ.
The Woodcock–Johnson Psychoeducational Battery Tests of Achievement (Woodcock & Johnson, 1989). Academic
achievement was evaluated using the Revised or Third version of the Woodcock–Johnson Psychoeducational Battery Tests
of Achievement. Broad Reading and Broad Math cluster scores were selected for analysis because these were completed by
most participants and reflect two of the major components assessed by the tests.
Grooved Peg Board Test (Tiffin, 1948). The Grooved Peg Board Test is a widely used measure of motor speed and dexter-
ity. Scores included in the analyses were the time taken to place all of the pegs with the dominant and nondominant hands.
Beery–Buktenica Developmental Test of Visual-Motor Integration (Fifth Edition). The Beery–Buktenica Developmental
Test of Visual-Motor Integration (Beery & Beery, 2004) is a paper and pencil test of visual perception and visual-motor inte-
gration that takes approximately 10–15 min to administer and is comprised of 30 figures of increasing complexity that the test
subject is instructed to copy.
Trail Making Test Parts A and B Intermediate Version (Reitan & Wolfson, 1993). The Trail Making Test parts A and B were
used to assess psychomotor speed, visual search and sequencing, and mental flexibility (Part B).
Conners’ Continuous Performance Test II (Conners & MHS Staff, 2000). The Conners’ Continuous Performance Test II
(CPT-II; Conners & MHS Staff, 2000) is a computer-administered task comprised of a series of letters presented intermittently
on the computer screen, with time intervals of varying lengths occurring between the letters. Test subjects were asked to click
the mouse when a letter appeared that was not the letter “X.”
The Oral and Written Language Scales (Carrow-Woolfolk, 1996). The Oral and Written Language Scale (OWLS) is an
individually administered assessment of receptive and expressive language for ages 3 through 21 years. The OWLS is com-
prised of three co-normed scales, including Listening Comprehension (LC), Oral Expression, and Written Expression. The
LC scale was used in this study as a measure of receptive language.
The Behavior Assessment System for Children (Reynolds & Kamphaus, 1992). The Behavior Assessment System for
Children (BASC; Reynolds & Kamphaus, 1992) Parent Rating Scale (PRS), Teacher Rating Scale (TRS), and Self-Report
of Personality (SRP) scales were used to assess adaptive and clinical (maladaptive) behaviors. Analyses focused on TRS,
PRS, and SRP composite scores, but given the literature suggesting the presence of attention deficits, impulsivity, conduct pro-
blems, and learning difficulties following TBI, we examined these domains as assessed by individual BASC subscales.
Procedure and data analysis. Tests were administered according to standardized procedures by a pediatric neuropsychologist
or doctoral level technicians who were supervised by the pediatric neuropsychologist. The TOMAL was routinely administered
to all children, unless there was a reason not to administer the test (referral question, age of child, etc.). The BASC was com-
pleted by either the parent (PRS), child with TBI (SRP), or teacher (TRS).
To establish the sensitivity of the TOMAL to TBI, SS and TBI between group comparisons were made using multivariate
analysis of variance (MANOVA) for the TOMAL subtest and Index scores, with post hoc univariate analyses used to examine
specific subtest and composite scores differences between the groups when overall MANOVAs were significant. Principal com-
ponent analysis (PCA) with varimax rotation was then used to identify groupings of TOMAL subtests to be used in the cluster
analysis. PCA was conducted because prior investigations of the TOMAL have produced mixed results with regard to the
D.N. Allen et al. / Archives of Clinical Neuropsychology 25 (2010) 618–633 621
at UN
LV
University L
ibraries on September 27, 2012
http://acn.oxfordjournals.org/D
ownloaded from
number of factors (compare Alexander & Mayfield, 2005; Reynolds & Bigler, 1994b, 1996; and Thaler, Allen, McMurray, &
Mayfield, 2010). The Kaiser–Guttman criterion was used to select the number of factors. Factor scores rather than TOMAL
Index scores were used in the cluster analysis, because index scores are not independent of each other, as some TOMAL subt-
ests contribute to more than one index score. Factor scores are also more reliable than individual subtest scores. Factor scores
were calculated by deriving the mean of the standard scores for each factor’s respective subtests (e.g., Allen, Strauss, Kemtes,
& Goldstein, 2007).
Cluster analysis was then used to examine neurocognitive heterogeneity in the SS and TBI groups. We analyzed the data
with four- and five-cluster solutions to determine which solution was optimal. TBI and SS groups were analyzed separately to
determine optimal cluster solutions for each group. Ward’s method of cluster analysis was used because it is consistent with the
cluster analytic methodology of previous studies (Curtiss, Vanderploeg, Spencer, & Salazar, 2001; Mottram & Donders, 2006),
it produces results that are consistent with other agglomerative clustering methods, and is less affected by outliers compared
with those methods (Morris, Blashfield, & Satz, 1981). We used squared Euclidian distance as the dissimilarity measure in the
analyses because it provides a direct, physical measure of distance in Euclidean space (Everitt, Landau, & Leese, 2001) and it is
sensitive to pattern and level of performance differences among individuals. To identify the appropriate number of clusters, we
followed the approach suggested by Aldenderfer and Blashfield (1984). Stability of the cluster solutions derived using Ward’s
method were examined using K-means iterative partitioning cluster analysis. For K-means, the number of clusters and initial
cluster centers were specified based on Ward’s method results. K-means derives clusters by iteratively adding cases to the group
that has the mean value closest to the unassigned data point (Hair, Black, Babin, Anderson, & Tatham, 1992). Cohen’s kappa
was then used to assess the extent to which the K-means and Ward’s method solutions agreed. On the basis of these various
methods, along with consideration of the a priori hypotheses regarding number and type of clusters, the most appropriate sol-
utions were identified for the SS and the TBI groups.
External validity of the optimal TBI cluster solution was then examined by comparisons on variables that were not included
in the cluster analysis, but were theoretically and clinically relevant to TBI, including demographic, clinical, IQ, achievement,
neurocognitive, and behavioral variables. In cases where interaction effects were hypothesized (e.g., IQ and neurocognitive
variables), mixed-model ANOVAs were used to examine cluster differences, in which cluster membership served as a between-
subjects variable and test scores as a within-subjects variable. When no interaction effects were predicted, chi-squared, uni-
variate ANOVAs, or multivariate ANOVAs were used (as appropriate) to examine cluster differences.
Results
There were no significant differences between the TBI and the SS groups on age or sex, x2(1) ¼ 0.66, p ¼ .42, indicating
that the group selection and matching procedure was successful. MANOVAs indicated significant differences between the SS
and the TBI sample for TOMAL subtest scores, F(68, 528) ¼ 5.59, p , .001, h2p, and Index scores, F(68, 528) ¼ 12.4, p ,
.001, h2p = .31. Following the Bonferonni correction for multiple comparisons, significant differences were present between
the TBI and the SS groups on all of the TOMAL scores. The SS group’s performance on the TOMAL subtest and index
scores approximated the SS mean, and the TBI group performed approximately 1.3 SD below the SS group on most
TOMAL scores, suggesting that the TOMAL was sensitive to TBI (detailed results of these analyses may be obtained from
the corresponding author).
Principal Component Analysis
PCA conducted for the SS group indicated six factors based on the Kaiser–Guttman criteria. This solution approached
simple structure, with most TOMAL subtests exhibiting primary loadings greater than 0.70, and few cross loadings greater
than 0.40. The six factors were interpretable and assessed attention/concentration, and verbal and nonverbal memory processes,
with distinctions between the verbal and the nonverbal factors based to some extent on task requirements. For the TBI group,
the Kaiser–Guttman criteria indicated four factors. However, the four-factor solution did not approach simple structure and
some factors that were not interpretable. When a six-factor solution was specified for the TBI group, a solution comparable
to the SS group was obtained that approached simple structure. A final PCA conducted for the combined samples generally
improved the six-factor solution (Table 1) and is consistent with the original conceptualization of the TOMAL as a
measure of verbal and nonverbal memory processes, and attention/concentration (Reynolds & Bigler, 1994a, 1996). On the
basis of these PCA results, six-factor scores were derived and subsequently used in the cluster analyses (separate PCA
results for the TBI and the SS groups may be obtained from the corresponding author).
622 D.N. Allen et al. / Archives of Clinical Neuropsychology 25 (2010) 618–633
at UN
LV
University L
ibraries on September 27, 2012
http://acn.oxfordjournals.org/D
ownloaded from
Cluster Analyses for TBI and SS Groups
Cluster analysis of the TOMAL subtests was accomplished specifying four- and five-cluster solutions for the TBI and the SS
groups. For both groups, a preliminary graphical inspection of the cluster solutions based on the factor scores was not particu-
larly informative because the number of factors made interpretation difficult, so Index score profiles were examined. Factor
score profiles were largely consistent with the Index score profiles, although Index score profiles were easier to interpret
and allowed for examination of profiles as they would occur in clinical settings (i.e., clinicians will not typically interpret
factor scores).
Examination of index score profiles for the SS cluster solutions indicated that the clusters were differentiated primarily by
the level of performance. The four-cluster solution consisted of a low average (C1), average (C2), and two above average clus-
ters, one that obtained scores slightly above 100 (C3), and another with superior performance on the ACI (mean ¼ 120.5) as
well as elevated scores on the Verbal and Nonverbal Memory Indexes (C4; Fig. 1, Frame A). The five-cluster solution further
divided the average cluster (C2) into two clusters, which did not markedly differ from each other. Thus, preliminary inspection
of the cluster solutions indicated that the four-cluster solution was the better of the two solutions for the SS group.
For the TBI group, clusters were differentiated by both level and pattern of performance. The four-cluster solution consisted
of a low-scoring cluster that exhibited a relative increase on the Delayed Recall Index (DRI) and ACI, a middle cluster exhibit-
ing a relatively lower score on the VMI, an average cluster that exhibited its highest performance on the Nonverbal Memory
Index, and an average cluster that obtained its lowest Index scores on the ACI and NMI. As seen in Fig. 1, Frame B, the
five-cluster solution included an Impaired cluster, that obtained Index scores ranging from 56.7 to 66.7, and obtained the
highest scores on the DRI and ACI, with relatively lower scores on the VMI and NMI. There was also a Verbal cluster that
obtained its lowest score on the VMI (70.8), with NMI, DRI, and ACI scores approximating 80.0. A Visual cluster is also appar-
ent that exhibits the lowest performance on the NMI (65.6) with scores on the VMI, DRI, and ACI of approximately 80.0. Two
additional clusters exhibit performance across all Indexes in the low average to average range, with the first cluster obtaining
average scores on all indexes (Average cluster), and the second exhibiting relative decreases on the ACI and possibly the NMI
(Attention cluster). Thus, preliminary inspection of the four- and five-cluster solutions in the TBI group suggest that the four-
and five-cluster solutions provide clear indication of level and pattern of performance differences, with this being the case
Table 1. Factor structure of the TOMAL combined SS and TBI groups (N ¼ 300)
TOMAL Factors
1 2 3 4 5 6
LF 0.83 0.12 0.13 0.15 0.16 0.10
DF 0.82 0.06 0.18 0.09 0.06 0.14
LB 0.76 0.32 0.07 0.11 0.00 0.17
DB 0.75 0.23 0.08 0.13 0.06 0.12
VSM 0.48 0.11 0.28 0.35 0.22 0.04
WSR 0.28 0.73 0.21 0.19 0.15 0.12
PR 0.16 0.74 0.23 0.02 0.04 0.24
OR 0.38 0.49 0.31 0.26 0.27 0.21
WSRD 0.14 0.78 0.10 0.26 0.12 -0.05
MFSD 0.19 0.27 0.86 0.15 0.16 0.08
MFS 0.19 0.22 0.90 0.07 0.07 0.16
VSRD 0.11 0.22 0.11 0.88 0.06 0.15
VSR 0.37 0.20 0.10 0.76 0.07 0.20
FM 0.19 0.18 0.26 0.21 0.72 0.10
FMD 0.05 0.08 0.01 20.03 0.88 0.03
AVM 0.24 0.12 0.31 0.19 0.31 0.60
MFL 0.21 0.14 0.06 0.16 20.02 0.85
Eigenvalue 7.1 1.5 1.2 1.1 1.0 0.8
% Variance 41.6 8.9 7.0 6.4 5.8 4.7
Chronbach’s a 0.86 0.82 0.92 0.74 0.62 0.56
Notes: TOMAL ¼ Test of Learning and Memory; SS ¼ Standardization sample group; TBI ¼ Traumatic brain injury group; LF ¼ Letters Forward; DF ¼
Digits Forward; LB ¼ Letters Backwards; DB ¼ Digits Backwards; VSM ¼ Visual Sequential Memory; WSR ¼Word Selective Reminding; PR ¼ Paired
Recall; OR ¼ Object Recall; WSRD ¼Word Selective Reminding Delayed; MFS ¼Memory for Stories; MFSD ¼Memory for Stories Delayed; VSRD ¼
Visual Selective Reminding Delayed; VSR ¼ Visual Selective Reminding; FM ¼ Facial Memory; FMD ¼ Facial Memory Delayed; AVM ¼ Abstract
Visual Memory; MFL ¼Memory for Location.
D.N. Allen et al. / Archives of Clinical Neuropsychology 25 (2010) 618–633 623
at UN
LV
University L
ibraries on September 27, 2012
http://acn.oxfordjournals.org/D
ownloaded from
particularly for the five-cluster solution. The five-cluster solution also appears to be the most theoretically and clinically rel-
evant of the solutions, based on the a priori hypothesis.
Stability of the four- and five-cluster solutions examined using the K-means iterative partitioning method indicated that the
Cohen’s kappas for the SS group were 0.86 and 0.77, respectively, and 0.78 and 0.87, respectively, for the TBI group. Kappas
at or above a 0.80 level are considered to indicate excellent agreement (Landis & Koch, 1977) and indicate that four-cluster SS
solution and five-cluster TBI solution are stable and have a high-level agreement when derived using Ward’s and K-means
methods of cluster analysis. Therefore, the five-cluster solution appeared optimal for TBI, and the four-cluster solution
appeared optimal for SS.
TOMAL Index score profiles for the four-cluster SS solution and five-cluster TBI solutions are presented in Fig. 1. In the
figure, the Verbal Memory, Nonverbal Memory, and Attention/Concentration Indexes are presented first, followed by the
Composite and Delayed Recall Memory Indexes, as these latter indexes combined verbal, nonverbal, and attention/concen-
tration subscale scores, and so are not useful in determining group differences across discrete abilities. Comparison of the
SS and TBI cluster solutions indicated not only overall differences in the level of performance, with the SS clusters generally
obtaining higher scores than the TBI clusters, but also that the TBI clusters were differentiated by both the level and the pattern
of performance, whereas the SS clusters tended to be differentiated by the level of performance, with pattern differences in the
SS group mostly determined by DRI scores. In contrast, for the TBI group, there were verbal and nonverbal memory discre-
pancies among the clusters, as well as a cluster that exhibited relative impairment in the attention domain (figures plotting the
factor scores for all cluster solutions may be obtained from the corresponding author).
Given that the five-cluster solution for the TBI sample was most consistent with the hypothesis, in combination with its
increased stability and theoretical interest, we further evaluated the external validity of the five-cluster solution in the TBI
group. Descriptive statistics for the TBI five-cluster solution on the TOMAL subtest and Index scores are presented in
Table 2 (tables containing descriptive statistics for the TOMAL subtests for all TBI and the SS cluster solutions may be
obtained from the corresponding author).
External Validity of the TBI Clusters
Demographic and clinical differences among the TBI clusters. Descriptive statistics for the demographic and clinical variables
are presented in Table 2. Chi-square analyses indicated no significant differences among the five clusters with regard to gender,
x2(4) ¼ 2.60, p ¼ .63, open or closed TBI, x2(4) ¼ 1.79, p ¼ .78, type of accident that caused the TBI, x2(28) ¼ 0.29, p ¼ .40,
Fig. 1. Cluster profiles of TOMAL Indexes for the standardization sample group four-cluster solution (Frame A), and TBI group five-cluster solution
(Frame B). Frame A: SS group; Frame B: TBI group. VMI ¼ Verbal Memory Index; NMI ¼ Non-verbal Memory Index; ACI ¼ Attention/Concentration
Index; CMI ¼ Composite Memory Index; DRI ¼ Delayed Memory Index.
624 D.N. Allen et al. / Archives of Clinical Neuropsychology 25 (2010) 618–633
at UN
LV
University L
ibraries on September 27, 2012
http://acn.oxfordjournals.org/D
ownloaded from
or whether the children were appropriately restrained in the case of motor vehicle accidents, x2(4) ¼ 4.74, p ¼ .32. One-way
ANOVAs indicated that the clusters did not differ significantly with regard to age, F(4,145) ¼ 1.66, p ¼ .16, age at injury,
F(4,145) ¼ 1.70, p ¼ .15, or time elapsed between TBI and assessment, F(4,145) ¼ 0.88, p ¼ .48. However, analysis of
ranked GCS scores indicated significant differences among the groups, F(4,96) ¼ 2.5, p , .05. Post hoc comparisons indicated
that the Attention cluster had significantly higher GCS scores than the Verbal cluster, although comparable but nonsignificant
differences were also present for the Impaired and Visual clusters.
IQ, achievement, and neuropsychological differences among clusters. Descriptive statistics for IQ, achievement, and neurop-
sychological variables are presented in Table 3, as are ANOVA results and post hoc comparisons examining differences among
the clusters. For Full-Scale IQ, ANOVA indicated a significant difference between the clusters (p , .001). The Average and
Attention clusters did not significantly differ from each other, but obtained significantly higher scores than the Verbal and
Visual clusters, which in turn performed significantly better than the Impaired cluster. Mixed-model ANOVA examining
potential differences on subtests selected from the four Wechsler Index scores indicated significant effects for Cluster,
F(4,134) ¼ 35.64, p , .001, h2p = .515, and for Wechsler subtest, F(3,402) ¼ 8.53, p , .001, h2
p = .060, as well as a signifi-
cant Cluster by Wechsler subtest interaction effect, F(12,402) ¼ 2.42, p ¼ .005, h2p = .067. The interaction appears to be
caused primarily by decrements in performance on the Digit Symbol/Coding (DC) subtest for the Average cluster, as well
Table 2. Demographic, Clinical, and TOMAL subtest and Index scores for the five-cluster solution in the TBI group
Variables Clusters
Average (n ¼ 38) Attention (n ¼ 28) Visual (n ¼ 14) Verbal (n ¼ 54) Impaired (n ¼ 16)
Mean SD Mean SD Mean SD Mean SD Mean SD
Age (years) 12.9 3.6 11.3 3.5 11.1 4.0 12.1 2.9 11.4 3.2
Age at Injury (years) 12.8 3.6 11.2 3.4 10.9 3.9 11.8 3.0 11.3 3.2
Time Elapsed (months) 7.0 2.5 6.3 2.4 7.0 3.5 8.2 5.0 6.9 2.2
GCS 7.5 3.2 8.5 3.5 6.1 2.4 6.2 2.8 6.0 2.1
Gender (% men) 55.3 53.6 61.1 42.9 68.8
TBI Type (% closed) 92.6 97.4 92.9 93.8 89.3
TOMAL Subtests
MFS 8.9 2.7 11.6 1.6 9.2 1.5 6.1 2.0 5.4 2.2
WSR 10.2 2.4 10.2 2.6 6.7 2.5 6.5 2.9 3.6 2.4
OR 8.5 3.3 7.7 2.1 5.1 2.0 4.2 2.3 1.9 1.7
DF 7.8 3.0 7.0 2.2 7.0 2.2 6.4 2.5 4.3 1.2
PR 10.4 2.4 9.3 3.2 7.7 3.1 5.5 3.2 3.4 3.0
LF 8.1 3.0 7.0 1.8 6.3 2.2 6.5 2.6 4.2 2.2
DB 10.0 2.4 8.5 1.8 8.3 1.6 7.5 1.7 5.8 1.8
LB 9.7 2.8 8.5 1.6 7.6 1.3 7.1 2.3 5.3 2.0
FM 9.7 2.7 8.5 2.7 6.4 2.7 6.5 2.4 5.5 2.4
VSR 8.4 3.1 6.9 2.3 3.9 1.9 6.7 2.9 2.7 1.4
AVM 10.5 2.7 8.3 2.2 3.4 2.0 6.9 2.6 3.3 2.6
VSM 9.9 2.6 9.4 1.8 8.0 2.8 7.4 2.6 5.6 1.7
MFL 12.1 3.4 7.5 3.0 3.3 2.0 6.8 3.4 4.1 3.1
MFSD 7.6 2.8 10.3 1.5 7.9 1.7 4.6 2.5 3.2 1.8
FMD 10.3 1.7 9.0 2.6 7.9 2.9 8.5 2.5 7.8 1.5
WSRD 10.0 2.2 9.4 1.8 7.4 2.3 6.6 2.5 3.9 1.7
VSRD 9.9 1.4 9.1 2.1 6.4 1.3 8.2 2.2 5.7 1.7
TOMAL Indexes
VMI 94.0 11.7 94.0 8.3 80.4 7.9 70.8 8.3 56.7 7.7
NMI 100.7 8.6 86.9 6.8 65.6 8.4 78.6 8.5 60.6 7.4
CMI 97.4 8.4 90.2 5.3 71.9 8.1 73.6 6.6 56.9 5.0
DRI 96.2 8.8 96.2 8.1 82.6 9.7 79.7 8.5 67.7 5.5
ACI 92.6 13.5 84.5 7.9 82.8 10.2 79.3 11.8 66.5 5.8
Notes: TOMAL ¼ Test of Learning and Memory; TBI ¼ Traumatic brain injury group; GCS ¼ Glasgow Coma Scale; MFS ¼Memory for Stories; WSR ¼
Word Selective Reminding; OR ¼ Object Recall; DF ¼ Digits Forward; PR ¼ Paired Recall; LF ¼ Letters Forward; DB ¼ Digits Backwards; LB ¼ Letters
Backwards; FM ¼ Facial Memory; VSR ¼ Visual Selective Reminding; AVM ¼ Abstract Visual Memory; VSM ¼ Visual Sequential Memory; MFL ¼
Memory for Location; MFSD ¼Memory for Stories Delayed; FMD ¼ Facial Memory Delayed; WSRD ¼Word Selective Reminding Delayed; VSRD ¼
Visual Selective Reminding Delayed; VMI ¼ Verbal Memory Index; NMI ¼ Non-Verbal Memory Index; CMI ¼ Composite Memory Index; DRI ¼
Delayed Recall Index; ACI ¼ Attention/Concentration Index.
D.N. Allen et al. / Archives of Clinical Neuropsychology 25 (2010) 618–633 625
at UN
LV
University L
ibraries on September 27, 2012
http://acn.oxfordjournals.org/D
ownloaded from
Table 3. Cluster differences on the Wechsler IQ, the Woodcock–Johnson, and neurocognitive test performances
Clusters Total F-value p-value Tukey h2p
Average Attention Verbal Visual Impaired
M SD M SD M SD M SD M SD M SD
Intelligence
n 37 27 47 13 15 139
VO 9.0 3.4 9.4 2.4 6.5 2.6 5.9 3.2 4.7 2.1 7.4 3.2 12.0 .000 AtAv, VrVsI .259
BD 9.8 2.7 9.0 3.0 7.8 3.1 5.0 2.8 3.0 2.0 7.8 3.5 20.0 .000 AvAtVr, VsI .361
DS 10.4 3.1 8.8 2.0 7.3 2.5 7.9 2.5 5.1 1.4 8.2 3.0 15.3 .000 AvAt, AtVrVs, I .312
DC 8.0 2.8 8.7 3.3 5.3 2.9 6.2 3.1 3.1 2.2 6.5 3.4 13.6 .000 AtAv, AvVs, VsVr, VrI .289
IQ (n ¼ 146) 93.1 13.0 93.9 10.8 78.5 10.7 76.6 10.3 63.9 8.3 83.4 14.7 28.5 .000 AtN, VrVs, I .447
Achievement
n 37 26 42 12 15 132
Math 102.6 16.5 101.6 10.9 87.8 13.7 88.1 11.8 72.3 18.5 92.9 17.4 15.9 .000 AvAt, VsVr, I .334
Reading 98.2 13.4 101.4 9.0 87.1 14.3 87.6 12.6 75.3 17.6 91.7 15.6 12.5 .000 AtAv, AvVsVr, I .283
Attention
n 31 25 42 13 14 125
Commissions 47.9 7.2 50.4 11.4 49.4 11.0 50.4 8.3 50.2 10.7 49.4 9.9 0.3 .88 – .010
Hit Rate 51.3 9.4 50.4 15.8 50.1 19.8 48.4 16.2 45.1 21.6 49.7 16.7 0.4 .83 – .013
Hit Rate SE 56.5 10.0 57.5 9.8 65.3 13.9 57.1 10.7 72.3 15.5 61.5 13.2 6.2 .000 AvVsAtVr, VrI .172
Variability 54.2 10.2 56.0 9.3 60.1 13.3 53.0 9.4 65.6 12.9 57.7 11.9 3.6 .009 VsAvAtVr, AtVrI .106
Detectability 51.3 7.6 52.5 9.5 53.2 9.7 52.4 10.1 58.0 10.2 53.0 9.3 1.3 .26 – .043
Perseveration 56.6 16.1 53.7 11.9 64.6 24.5 53.7 12.5 70.8 20.2 60.0 19.5 3.1 .02 AtVsAvVr, AvVrI .092
Language
n 36 27 50 14 14 141
OWLS 100.3 17.0 95.3 10.9 85.3 12.2 86.1 13.6 77.9 15.6 90.4 15.6 10.3 .000 AvAt, AtVsVr, VsVrI
Visuopercept.
n 34 27 50 13 15 139
BVMI 90.5 12.8 88.9 10.5 87.9 13.7 79.0 11.9 78.5 13.5 86.9 13.2 3.8 .006 AvAtVr, AtVrVsI
Motor
n 37 26 44 12 13 132
GPBD 74.1 15.0 76.2 30.0 89.3 35.0 86.0 27.7 96.3 51.6 83.1 32.1 1.9 .11 – .057
GPBND 80.9 22.8 83.2 26.3 99.2 46.1 114.2 55.5 131.3 74.3 96.9 46.4 4.4 .002 AtAvVrVs, VrVsI .123
Executive
n 28 18 29 11 10 96
TMA 25.1 9.6 31.2 15.2 34.6 15.8 31.1 12.2 55.0 55.3 32.9 22.7 3.6 .009 AvVsAtVr, VrI .137
TMB 46.2 17.6 61.9 24.0 79.0 40.9 79.1 37.9 150.1 110.6 73.7 53.6 10.0 .000 AvAtVrVs, I .305
Notes: IQ ¼Wechsler Full-Scale IQ; VO ¼Wechsler Vocabulary; BD ¼Wechsler Block Design; DS ¼Wechsler Digit Span; DC ¼Wechsler Digit Symbol/Coding; Reading ¼Woodcock Johnson
Broad Reading; Math ¼Woodcock Johnson Broad Math; CPT ¼ Continuous Performance Test; Commissions ¼ CPT Commissions; Hit Rate ¼ CPT Hit Rate; Hit Rates SE ¼ CPT Hit Rate
Standard Error; Variability ¼ CPT Variability of the Standard Errors; Detectability ¼ CPT Detectability; Perseverations ¼ CPT Perseverations; OWLS ¼ Oral and Written Language Scale;
BVMI ¼ Beery–Buktenica Developmental Test of Visual-Motor Integration-5th Edition; GPBD ¼ Grooved Peg Board dominant hand; GPBND ¼ Grooved Peg Board nondominant hand; TMA ¼
Trail Making Test Part A; TMB ¼ Trail Making Test Part B; Av ¼ Average Cluster; At ¼ Attention Cluster; Vr ¼ Verbal Cluster; Vs ¼ Visual Cluster; I ¼ Impaired Cluster.
626
D.N
.A
llenet
al.
/A
rchives
of
Clin
ical
Neu
ropsych
olo
gy
25
(2010)
618–633
at UNLV University Libraries on September 27, 2012 http://acn.oxfordjournals.org/ Downloaded from
as diminished performance on the Vocabulary (VO) and Block Design (BD) subtests by the Visual Cluster. Significant differ-
ences among the clusters were present for each subtest, with post hoc analysis indicating a general pattern in which the Average
and Attention clusters performed significantly better than the other three clusters (Table 3). Consistent with modality-specific
effects, the Normal cluster performed significantly better than the Verbal and Visual clusters on the Vocabulary and Block
Design subtests, respectively.
For the Woodcock–Johnson achievement test data, mixed-model ANOVA indicated a significant effect for Cluster,
F(4,127) ¼ 17.50, p , .001, h2p = .355, but not for WJ Composite Score, F(1,127) ¼ 0.08, p ¼ .78, h2
p = .001, or for the
Cluster by WJ Composite Score interaction effect, F(4,127) ¼ 1.19, p ¼ .32, h2p = .036. Post hoc comparisons indicated
the level of performance differences with the Normal and Attention clusters obtaining higher scores than the Visual and
Verbal clusters, who in turn had higher scores than the Impaired cluster (Table 3).
For the attention domain, CPT variables were entered into the mixed-model ANOVA, and results indicated a significant
main effect for Cluster, F(4,120) ¼ 2.83, p ¼ .03, h2p = .086, and for CPT score, F(5,600) ¼ 22.40, p , .001, h2
p = .157,
as well as a significant Cluster by CPT score interaction effect, F(20,600) ¼ 2.22, p ¼ .002, h2p = .069. Expected differences
were present between groups on a number of CPT measures, with the Impaired cluster typically performing worse than the
others, with significant differences among the clusters for CPT Standard Error, Variability, and Perseverations (Table 3).
Significant differences were also present between the clusters on the visuomotor (BVMI) and language (OWLS) domains.
Post hoc analyses indicated that for language, the Average cluster exhibited the best performance but did not differ from the
Attention cluster, whereas the Impaired cluster exhibited the worst performance, but did not differ from the Verbal or Visual
clusters. For visuomotor integration, the Average cluster again exhibited the best performance, although it did not differ from
the Attention or Verbal cluster, while the Impaired cluster exhibited the worst performance, but did not differ from the Visual,
Verbal, or Attention clusters.
For the motor domain, results for the Grooved Peg Board dominant and nondominant hands indicated that there were sig-
nificant main effects for Cluster, F(4,127) ¼ 3.74, p ¼ .007, h2p = .105, and Hand, F(1,127) ¼ 30.95, p , .001, h2
p = .196,
and a significant Cluster by Hand interaction effect, F(4,127) ¼ 2.83, p ¼ .028, h2p = .082. The dominant hand was faster
than the nondominant hand in all groups, although a larger than expected discrepancy is present for the Impaired cluster,
with the nondominant hand being slower than expected. Significant differences among the clusters were present for only
the nondominant hand, and post hoc analyses indicated that the Normal and the Attention clusters performed similar to
each other and significantly better than the Visual and Verbal clusters, who in turn performed better than the Impaired cluster.
Finally, for the executive function domain, mixed-model ANOVA indicated a significant effect for Trail, F(1,91) ¼ 8.9,
p , .001, h2p = .082, Cluster, F(1,91) ¼ 141.7, p , .001, h2
p = .609, and a significant Trail by Cluster interaction effect,
F(1,91) ¼ 8.5, p , .001, h2p = .271. As can be seen from Table 3, the interaction effect was caused primarily by the
larger than expected discrepancy between Parts A and B for the Impaired cluster compared with the others. Post hoc analy-
sis indicated that in general, the Impaired cluster differed from the other clusters on Parts A and B, although the other
clusters did not differ from each other.
Behavioral differences among the TBI clusters. Descriptive statistics for the BASC subscales are presented in Table 4.
Separate MANOVAs were conducted for the Parent, Teacher, and Self-report ratings, where cluster membership served as
the between-subjects variable and BASC scores were the dependent variables. Additionally, scale and composite scores
were analyzed separately given the overlap between composite and scale scores. MANOVA for the PRSs indicated an
overall significant effect, F(12,387) ¼ 2.95, p ¼ .01, h2p = .064, although the MANOVA for the composite scores was not sig-
nificant, F(16,528) ¼ 1.41, p ¼ .13, h2p = .041. The largest effect was for the Hyperactivity and Conduct scales. Post hoc ana-
lyses indicated the Impaired cluster exhibited more behavioral problems than the other clusters, who did not differ from each
other.
MANOVA for the Teacher scale ratings indicated an overall significant effect, F(16,156) ¼ 3.48, p , .001, h2p = .263, as
did the MANOVA for the Teacher composite scores, F(20,152) ¼ 2.95, p , .001, h2p = .279. Significant group differences
were present among the clusters on all of the scale scores, and all but one of the composite scores (internalizing problems),
with the Impaired cluster again exhibiting more behavioral disturbances than the other clusters.
MANOVA for the Self-Report composite scores indicated no significant differences among the clusters, F(12,318) ¼ 1.74,
p ¼ .06, h2p = .062.
Discussion
The current study provides a number of findings relevant to neurocognitive functioning in children with TBI, particularly
with regard to heterogeneity. Most notably, results support the notion that TBI produces marked heterogeneity in
D.N. Allen et al. / Archives of Clinical Neuropsychology 25 (2010) 618–633 627
at UN
LV
University L
ibraries on September 27, 2012
http://acn.oxfordjournals.org/D
ownloaded from
neurocognitive function, so that there is no profile that best typifies these children and adolescents, although there do appear to
be subgroups within this diverse population that might be characterized by performance profiles from neuropsychological tests
that assess memory and attention/concentration abilities. Evidence supporting the validity of these subgroups was provided not
only by comparisons on clinical, neuropsychological, and behavioral variables, but also through comparison with clusters
identified in a matched normal sample. With regard to the latter point, the need for direct comparisons of TBI and normal
cluster solutions has long been suggested (Crosson et al., 1990), in order to determine if there are profiles unique to TBI
that represent more than normal or expected variation in test performance among individuals, and thus would be attributable
to brain injury. Results indicated that the TBI clusters differed in patterns of performance which was accounted for largely by
discrepancies in verbal and nonverbal memory abilities, or a relative decrease in attention/concentration. There was also evi-
dence for a group that exhibited severe impairment, as well as another that exhibited average performance. These TBI clusters
stood in contrast to the four clusters identified in the SS group, which were differentiated primarily by level of performance,
with pattern of performance differences accounted for primarily by the tendency toward reduced variability in the SS group on
the DRI compared with the TBI group (SD’s ¼ 8.8 and 12.9, respectively; Table 1). Thus, the current findings support the con-
tention that TBI results in patterns of memory and attention deficits that vary markedly from those observed in normal popu-
lations and these patterns are not easily accounted for by intra-individual variation in test performance that is often observed in
nonbrain injured populations. Differences in profile and number of clusters are also consistent with prior studies that have
undertaken separate TBI and SS analyses on, for example, the Wechsler Intelligence Scales (e.g., compare Donders &
Warschausky, 1997, and Donders, 1996).
Table 4. Parent, teacher, and self-report behavioral reports of the five-cluster solution
BASC Scores Clusters Total F-value p-value Tukey h2p
Average Attention Visual Verbal Impaired
Mean SD Mean SD Mean SD Mean SD Mean SD Mean SD
Parent Rating
Subscales
n 37 26 14 42 15 134
Hyperactivity 53.5 16.2 52.4 13.4 60.4 18.9 56.7 16.0 63.9 19.7 56.2 16.5 1.7 .16 – .050
Attention 58.4 12.8 55.0 13.2 61.9 11.7 59.0 10.7 72.4 9.6 59.9 12.6 5.7 .000 AtAvVrVs, I .149
Conduct 52.8 15.5 51.2 15.1 57.7 18.7 51.1 13.6 64.8 22.5 53.8 16.5 2.4 .05 VrAtAvVs, VsI .071
Composites
n 37 25 14 47 14 137
Externalizing 55.0 17.1 52.7 15.0 60.1 20.2 54.6 14.6 66.4 21.7 56.1 17.0 1.9 .11 – .055
Internalizing 53.9 12.0 51.9 10.9 55.9 17.2 55.1 14.2 50.6 11.5 53.8 13.1 0.5 .72 – .016
Behavioral 56.1 15.1 53.7 13.5 60.2 20.1 57.9 16.3 64.4 16.6 57.6 16.0 1.2 .32 – .035
Adaptive Skills 44.7 11.9 45.2 13.9 44.0 9.5 44.0 11.1 41.8 7.0 44.2 11.3 0.2 .93 – .007
Teacher Rating
Subscales
n 7 8 6 17 6 44
Hyperactivity 50.1 12.8 45.8 8.8 58.5 13.0 48.1 7.8 63.2 11.6 51.5 11.4 3.8 .01 AtVrAv, VsI .283
Attention 50.3 7.0 50.9 7.2 58.5 11.3 58.2 9.9 75.0 4.1 58.0 11.3 8.4 .000 AvAtVrVs, I .464
Conduct 47.7 7.3 46.4 2.8 44.5 2.3 45.7 4.8 66.0 16.7 48.8 10.0 9.5 .000 VsVrAtAv, I .493
Learning 47.0 4.6 50.5 10.5 58.2 8.3 56.5 8.9 79.2 7.7 57.2 12.6 13.8 .000 AvAtVsVr, I .587
Composites
n 7 8 6 17 6 44
Externalizing 49.0 10.6 45.6 6.0 50.5 6.3 46.4 5.7 64.7 13.6 49.7 9.9 6.4 .000 AtVrVsAv, I .395
Internalizing 48.4 8.6 50.0 12.2 63.5 12.0 51.0 7.3 51.3 9.7 52.2 10.2 2.6 .052 – .209
School 48.6 5.4 50.6 8.3 58.5 9.7 57.7 9.5 78.3 5.8 57.9 12.1 12.6 .000 AvAtVsVr, I .564
Behavioral 49.7 11.6 48.0 9.5 59.5 12.3 50.9 6.3 62.8 6.4 53.0 9.9 3.8 .01 AtAvVr, I .281
Adaptive Skills 50.7 8.9 49.8 10.4 43.8 8.7 47.1 10.0 31.2 6.2 45.5 10.8 4.7 .004 AtAvVrVs, I .323
Self-Report
Composites
n 33 23 11 33 12 112
School 47.2 9.5 49.2 9.7 54.6 10.7 50.4 11.0 57.3 12.0 50.4 10.7 1.3 .26 – .048
Clinical 48.8 12.0 48.4 7.7 50.7 10.4 48.5 7.9 46.4 7.4 48.5 9.3 0.2 .93 – .011
Personal 50.9 11.0 51.5 7.8 49.0 10.6 46.3 11.0 46.1 7.6 49.0 10.2 1.5 .21 – .054
Emotional 48.0 11.1 48.5 8.7 51.9 12.3 51.1 9.1 49.3 6.8 49.5 9.7 0.6 .64 – .023
Notes: Av ¼ Average Cluster; At ¼ Attention Cluster; Vr ¼ Verbal Cluster; Vs ¼ Visual Cluster; I ¼ Impaired Cluster.
628 D.N. Allen et al. / Archives of Clinical Neuropsychology 25 (2010) 618–633
at UN
LV
University L
ibraries on September 27, 2012
http://acn.oxfordjournals.org/D
ownloaded from
Although the number of clusters and profile differences are strongly influenced by the measures that are entered into the
cluster analysis (Malec et al., 1993), there is some consistency across studies of adults and children with moderate to
severe TBI which may support the utility of neuropsychological tests in classifying injury severity. For example, most
studies identify average and impaired clusters. Other studies that have examined verbal memory abilities find that some chil-
dren with TBI exhibit deficits in various aspects of memory functioning and others do not. Studies of attention also identify
clusters of children with selective deficits in some aspects of attention. In this study, nonverbal memory tests were included and
a group was also identified with selective deficits in this area. However, one objection to the results of cluster analytic studies is
that the replicability and thus generalizability of cluster solutions are not typically assessed. One study that compared cluster
solutions derived from two different TBI samples that were assessed using the same tests provided strong support for the stab-
ility and generalizability of neurocognitive clusters identified across different TBI groups (Curtiss et al., 2001). Another study
that investigated the stability of clusters derived using different tests (abstraction vs. IQ) in 221 patients with schizophrenia
identified four clusters for each set of tests, and there was a significant degree of association between the solutions, although
this association was weak (Goldstein, Allen, & Seaton, 1998). Thus, although there is support for the stability of clusters across
populations and possibly across tests, test selection remains a significant determining factor in the number and profile of the
obtained clusters, which highlights the need for selection of tests that are directly relevant to the types of cognitive deficits
exhibited by the population under investigation. In the current study, measures of memory and attention/concentration were
selected for cluster analysis because deficits in these areas are commonly reported following TBI.
With regard to the specific clusters, the Average cluster obtained TOMAL Index scores ranging from 94.0 to 100.7. We do
not refer to the Average cluster as “normal” even though their performance is in the average range, because it is likely that some
reduction in neurocognitive abilities resulting from TBI occurred in these children, and so they should not be considered
normal. Donders and Warschausky (1997) found that their highest performing WISC-III TBI cluster had a higher socioeco-
nomic background, which may indicate that while still average in their performance, some decline from premobid levels of
functioning had occurred. A similar observation has been made for some individuals with schizophrenia who perform in
the average range on neuropsychological tests (Palmer et al. 1997), but nevertheless exhibit subtle neurocognitive deficits con-
sistent with the presence of schizophrenia (Allen, Goldstein, & Warnick, 2003). These patients have been referred to as “neu-
ropsychologically normal” or alternatively “high cognitive functioning,” akin to terminology used to describe autism (e.g.,
Goldstein et al., 2008). These high cognitive functioning patients with schizophrenia are of some interest since preserved neu-
rocognitive function is associated with better outcomes, and less severe impairment is associated with decreased neuroana-
tomic abnormalities (Allen et al., 2000; Wexler et al., 2009). The Average cluster in this study may also exhibit more
subtle signs of brain damage, as indicated by the relative decrease in processing speed inferred by relatively poorer perform-
ance on the Digit Symbol/Coding subtest compared to the other WISC subtests (Allen et al., 2010; Donders & Janke, 2008).
Whether the nomenclature of “high functioning” applied to schizophrenia and autism would also be a useful descriptor in clas-
sifying childhood TBI awaits further investigation, although it is apparent that more severe deficits have been linked to poorer
outcomes in TBI, in this study and in others.
The Impaired cluster obtained TOMAL Index scores ranging between 56.7 and 67.7, and consistently performed more
poorly than the other clusters on many of the IQ, neuropsychological, and achievement variables, and also exhibited more
severe behavioral disturbances and poorer adaptive functioning than the other clusters. There were no marked differential pat-
terns of impairment across the neuropsychological domains assessed as external validity variables, which is consistent with the
overall picture of severe impairment. This impairment was apparent not only on neurocognitive measures but also measures of
achievement, which are typically considered less susceptible to decline following TBI and other acquired brain disorders, and
can be used as an indicator of premorbid function in some cases. However, the current findings suggest that for this impaired
cluster performance on achievement tests is also diminished, which is consistent with a preliminary report of 170 children with
TBI, who performed significantly worse on many of the WJ III scales compared to the SS mean (Stolberg et al., 2009).
Elevations in parent ratings of attention and conduct problems were also apparent. Similarly, teacher ratings indicated increased
problems with hyperactivity, attention, conduct and learning, as well as increased externalizing, school and behavioral pro-
blems, and decreased adaptive skills. Thus, the Impaired cluster identified in this study and exhibits disturbances across a
broad range of neurocognitive, achievement, and behavioral variables and is also expected to require the most intensive treat-
ment and rehabilitative services during recovery.
There was also a cluster that obtained its lowest score on the Attention Concentration Index, although it also exhibited a
decrease in performance on the TOMAL Nonverbal Memory Index. This cluster was similar to the Average cluster in
many respects, including performance on the IQ and achievement tests, as well as on tests of motor speed and executive func-
tion. Expected decrements in performance on tests of attention, particularly the CPT and Wechsler Digit Span, were not appar-
ent. Also, expected deficits in attention on the behavioral ratings were not apparent, which may reflect a more general lack of
association between neuropsychological and behavioral measures of attention (Dennis et al., 2001; Naglieri, Goldstein,
D.N. Allen et al. / Archives of Clinical Neuropsychology 25 (2010) 618–633 629
at UN
LV
University L
ibraries on September 27, 2012
http://acn.oxfordjournals.org/D
ownloaded from
Delauder, & Schwebach, 2005; Young & Gudjonsson, 2005). It did appear that compared with the Average cluster, the
Attention cluster exhibited somewhat poorer performance on tests of language and visuoperceptual abilities. This later
finding may be associated with the relative decrease in performance on the TOMAL Nonverbal Memory Index, which has
previously been shown to have significant associations with the Perceptual Reasoning Index from the WISC-IV (Allen
et al., 2010), suggesting its reliance on visuoperceptual and visual reasoning processes. However, given the lack of associated
decrements in attention on neuropsychological and behavioral measures, and the decrease in the TOMAL Nonverbal Memory
index, this cluster should not be viewed as a “pure” attention cluster, and its external validity is questionable.
Verbal and Nonverbal clusters were also identified, with the Verbal Cluster obtaining index scores ranging from 70.8
(Verbal Memory Index) to 78.6, and the Visual cluster obtaining index scores ranging from 65.6 (Nonverbal Memory
Index) to 82.8. Verbal memory tests have been the most extensively examined in cluster analytic studies in children with
TBI, and the current results provide further evidence for a cluster that is differentiated by this ability. Measures of nonverbal
learning and memory have not been included in prior cluster analytic studies of children with TBI, but based on the current
results there also appears to be a subgroup of children with TBI who are characterized by relatively poor performance on
measures of this domain. With regard to external validity, the Verbal and Visual clusters typically performed worse than
the Average and Attention clusters and better than the Impaired cluster on the neurocognitive variables. In contrasting the
Verbal and Visual clusters on the IQ variables, the Verbal cluster exhibits the expected dissociation between Vocabulary
and Block Design, although this pattern in less pronounced in the Visual cluster. However, the Verbal cluster did not differ
from the Visual cluster on the OWLS, which is a measure of listening comprehension. Consistent with neurocognitive data,
the Visual cluster scored lower on the perceptual organization domain, without any distinct discrepancies in other areas.
Behaviorally, few differences were present, although the Visual cluster did exhibit elevations on BASC parent ratings of
Conduct and teacher ratings of Hyperactivity, similar to those observed in the Impaired cluster. These differences between
the Visual and Verbal clusters provide preliminary support for their external validity, although some differences were not
as robust as might have expected.
With regard to external validity of the TBI clusters on clinical and demographic variables, similar to prior studies, we found
that only severity of coma significantly differentiated clusters, and clusters were not differentiated on gender, ethnicity, cause of
injury, injury-testing interval, or age at testing (Crosson et al., 1990; Donders & Warschausky, 1997). Contrary to hypotheses, age
at injury and time elapsed from injury to assessment did not differ among clusters, which appears inconsistent with longitudinal
studies indicating a developmental influence on how TBI effects outcome (e.g., Verger et al., 2001). Few studies have investigated
associations between age or developmental level at the time of TBI and outcome, but existing longitudinal studies suggest that
earlier age of injury is associated with poorer outcomes including slower development of attention, motor, and verbal and spatial
skills (Anderson, Catroppa, Morse, et al., 2000, 2004; Anderson & Moore, 1995; Duval et al. 2008; Kriel et al., 1989; Thompson
et al., 1994). Failure to find age differences among our clusters was likely a result of the relatively brief time between injury and
assessment compared with other studies (e.g., Hawley, 2004; Millis et al., 2001). Whether clusters identified in this study will be
useful in predicting longitudinal outcomes awaits future research that incorporates longer time periods between injury and assess-
ment. However, one longitudinal study of neurocognitive, adaptive, behavioral, and academic functioning of children with mod-
erate to severe TBI who were classified according to the longitudinal patterns of outcomes, found that those with severe TBI had
an increased likelihood of experiencing poorer outcomes, functional deficits, and behavioral deterioration an average of 4 years
after injury (Fay et al., 2009). It also remains to be seen whether injury severity classified using neurocognitive tests such as the
TOMAL would provide better prediction of long-term outcomes than severity classification based on the GCS or other similar
techniques, or whether comparable variability in course and outcome would be noted.
To our knowledge, this is the first study to examine differences in behavior among neurocognitive clusters in children and
adolescents with TBI using a well-standardized multimethod behavioral assessment, despite the observation that behavioral
disturbances are common following TBI (McKinlay et al., 2009; Yeates et al., 2004). The results do not generally support
a profile of behavioral disturbance that is differentiated based on membership in neurocognitive clusters in most cases.
Rather, severity of neurocognitive impairment was more generally associated with disturbances in behavior and poorer adap-
tation, with the most severely impaired neurocognitive cluster exhibiting clinical elevations on a number of the BASC subscale
and composite scores. Interestingly, there were differences between the type of behavioral report, with teacher reports consist-
ently demonstrating the most differences among clusters. Although the findings are limited by a relatively small number of
teacher ratings (n ¼ 44), results are consistent with a recent preliminary study of children with TBI using the BASC-2 that
found increased teacher endorsement of externalizing and school-related problems more frequently than internalizing pro-
blems, with the greatest elevations on the hyperactivity, attention, and learning problems subscales (Thaler, Terranova,
Reynolds, Mayfield, & Allen 2010). Also, although behavioral problems following pediatric TBI are typically evaluated via
parent reports (Yeates & Taylor, 2006), with less research available for teacher ratings, there do appear to be inconsistencies
between parent and teacher ratings (Gross, Fogg, Garvey, & Julion, 2004; Lee, Elliott, & Barbour, 1994; McAuley, 2000;
630 D.N. Allen et al. / Archives of Clinical Neuropsychology 25 (2010) 618–633
at UN
LV
University L
ibraries on September 27, 2012
http://acn.oxfordjournals.org/D
ownloaded from
McConaughy, Stanger, & Achenbach, 1992). These inconsistencies are likely influenced by contextual factors relating to
setting (e.g., hyperactivity may cause less disruption in the home than in the classroom), attribution of problem behaviors
to internalizing or externalizing factors (Max et al., 1998), or by differences between teachers and parents referent comparisons
to normally developing children. Because of these differences, teachers may be more accurate in rating hyperactivity, whereas
parents are most reliable in rating conduct disorders, oppositional behavior, and internalizing behaviors (Loeber, Green, &
Lahey, 1990; Yeates & Taylor, 2006). Thus, the suggestion that teacher reports provide unique insights to childhood behavioral
disorders (Randazzo, Landsverk, & Ganger, 2003; Thaler, Terranova, et al., 2010) is consistent with the current findings and
highlights the importance of supplementing parental reports with those of teachers when evaluating behavioral disturbances in
children with TBI.
The current study has a number of limitations including that some clusters contained a limited number of participants, which
may have decreased statistical power to detect cluster differences on some external validity variables. Also, because this was a
study of a convenience sample referred for clinical evaluation, there was incomplete data for some tests (e.g., BASC TRS),
which may limit generalizability of findings. In a prospective investigation, alternative measures may have been selected
and some domains would have been more thoroughly evaluated (e.g., executive function). Additionally, children with mild
TBI were not included. Their inclusion may have produced a high average cluster that would contain those children with
above average abilities premorbidly, who did not experience a substantial decrease in neurocognitive abilities as a result of
TBI. Finally, while a revised version of the TOMAL has been recently published (TOMAL-2; Reynolds & Voress, 2007),
similar patterns of performance are anticipated on the TOMAL-2, because no new items or subtests were introduced, and
the child SS remained largely the same. Future studies may attempt to determine stability of this cluster solution across differ-
ent populations of children with TBI, as well as whether neurocognitive classifications of TBI, such as the one reported here,
will be more useful in predicting longer-term outcomes in comparison to currently existing and commonly used classification
methods such as the GCS. However, the current results do suggest that deficits in memory and attention abilities, particularly
for patients who exhibit severe impairment in these areas, are associated with disturbances across a wider range of neurocog-
nitive and behavioral domains, and so may prove to be clinically useful in predicting outcomes, anticipating adjustment pro-
blems, and planning educational and rehabilitative interventions designed to remediate deficits and improve functional
outcomes following TBI in children.
Conflict of Interest
None declared.
Acknowledgements
Standardization Sample (SS) data was from the Test of Memory and Learning (TOMAL) standardization sample. Copyright
1994 by Pro-Ed. Used with permission. All rights reserved. We thank the TOMAL publisher for allowing access to the stan-
dardization data for this study.
References
Aldenderfer, M., & Blashfield, R. (1984). Cluster analysis. Newbury Park: Sage Publications.
Alexander, A. I., & Mayfield, J. (2005). Latent factor structure of the Test of Memory and Learning in a pediatric traumatic brain injured sample: Support for a
general memory construct. Archives of Clinical Neuropsychology, 20, 587–598.
Allen, D. N., Goldstein, G., & Warnick, E. (2003). A consideration of neuropsychologically normal schizophrenia. Journal of the International
Neuropsychological Society, 9, 56–63.
Allen, D. N., Seaton, B. E., Goldstein, G., Sanders, R. D., Gurklis, J. A., Jr, Peters, J. L., et al. (2000). Neuroanatomic differences among cognitive and symptom
subtypes of schizophrenia. Journal of Nervous and Mental Disease, 188, 381–384.
Allen, D. N., Strauss, G. P., Kemtes, K. A., & Goldstein, G. (2007). Hemispheric contributions to nonverbal abstract reasoning and problem solving.
Neuropsychology, 21, 713–720.
Allen, D. N., Thaler, N. S., Donohue, B., & Mayfield, J. (2010). WISC-IV profiles in children with traumatic brain injury: Similarities to and differences from
the WISC-III. Psychological Assessment, 22 (1), 57–64.
Anderson, V., Catroppa, C., Morse, S., Haritou, F., & Rosenfeld, J. (2000). Recovery of intellectual ability following TBI in childhood: Impact of injury sever-
ity and age at injury. Pediatric Neurosurgery, 32, 282–290.
Anderson, V., & Moore, C. (1995). Age at injury as a predictor of outcome following pediatric head injury: A longitudinal perspective. Child Neuropsychology,
1, 187–202.
Anderson, V. A., Catroppa, C., Rosenfeld, J., Haritou, F., & Morse, S. A. (2000). Recovery of memory function following traumatic brain injury in pre-school
children. Brain Injury, 14, 679–692.
D.N. Allen et al. / Archives of Clinical Neuropsychology 25 (2010) 618–633 631
at UN
LV
University L
ibraries on September 27, 2012
http://acn.oxfordjournals.org/D
ownloaded from
Anderson, V. A., Morse, S. A., Catroppa, C., Haritou, F., & Rosenfeld, J. V. (2004). Thirty month outcome from early childhood head injury: A prospective
analysis of neurobehavioural recovery. Brain: A Journal of Neurology, 127, 2608–2620.
Babikian, T., & Asarnow, R. (2009). Neurocognitive outcomes and recovery after pediatric TBI: Meta-analytic review of the literature. Neuropsychology, 23,
283–296.
Beery, K. E., & Beery, N. A. (2004). The Beery–Buktenica Developmental Test of Visual-Motor Integration: Administration, scoring, and teaching manual
(5th ed.). Minneapolis, MN: NCS Pearson.
Carrow-Woolfolk, E. (1996). Oral and Written Language Scales. Circle Pines, MN: American Guidance Service.
Chan, R. C. K., Hoosain, R., Lee, T. M. C., Fan, Y. W., & Fong, D. (2003). Are there sub-types of attentional deficits in patients with persisting post-concussive
symptoms? A cluster analytical study. Brain Injury, 17, 131–148.
Conners, C. K., & MHS Staff (Eds.). (2000). Conners’ Continuous Performance Test II: Computer program for windows technical guide and software manual.
North Tonwanda, NY: Multi-Health Systems.
Crawford, J. R., Garthwaite, P. H., & Johnson, D. A. (1997). WAIS-R subtest pattern clusters in closed-head-injured and healthy samples. Clinical
Neuropsychologist, 11, 249–257.
Crosson, B., Greene, R. L., Roth, D. L., Farr, S. P., & Adams, R. L. (1990). WAIS-R pattern clusters after blunt-head injury. The Clinical Neuropsychologist, 4,
253–262.
Curtiss, G., Vanderploeg, R. D., Spencer, J., & Salazar, A. M. (2001). Patterns of verbal learning and memory in traumatic brain injury. Journal of the
International Neuropsychological Society, 7, 574–585.
Demery, J. A., Pedraza, O., & Hanlon, R. E. (2002). Differential Profiles of Verbal Learning in Traumatic Brain Injury. Journal of Clinical and Experimental
Neuropsychology, 24 (6), 818–827.
Dennis, M., Guger, S., Roncadin, C., Barnes, M., & Schachar, R. (2001). Attentional-inhibitory control and social-behavioral regulation after childhood closed
head injury: Do biological, developmental, and recovery variables predict outcome? Journal of the International Neuropsychological Society, 7, 683–692.
Donders, J. (1996). Cluster subtypes in the WISC-III standardization sample: Analysis of factor index scores. Psychological Assessment, 8, 312–318.
Donders, J. (1999). Cluster subtypes in the standardization sample of the California Verbal Learning Test—Children’s version. Developmental
Neuropsychology, 16, 163–175.
Donders, J. (2008). Subtypes of learning and memory on the California Verbal Learning Test—Second Edition (CVLT-II) in the standardization sample.
Journal of Clinical and Experimental Neuropsychology, 30 (7), 741–748.
Donders, J., & Janke, K. (2008). Criterion validity of the Wechsler Intelligence Scale for Children—Fourth Edition after pediatric traumatic brain injury.
Journal of the International Neuropsychological Society, 14, 651–655.
Donders, J., & Warschausky, S. (1997). WISC-III factor index score patterns after traumatic head injury in children. Child Neuropsychology, 3, 71–78.
Donders, J., Zhu, J., & Tulsky, D. (2001). Factor index score patterns in the WAIS-III standardization sample. Assessment, 8, 193–203.
Duval, J., Braun, C. M. J., Montour-Proulx, I., Daigneault, S., Rouleau, I., & Begin, J. (2008). Brain lesions and IQ: Recovery versus decline depends on age of
onset. Journal of Child Neurology, 23, 663–668.
Everitt, B. S., Landau, S., & Leese, M. (2001). Cluster analysis (4th ed.). London: Arnold.
Fay, T. B., Yeates, K. O., Wade, S. L., Drotar, D., Stancin, T., & Taylor, H. G. (2009). Predicting longitudinal patterns of functional deficits in children with
traumatic brain injury. Neuropsychology, 23, 271–282.
Gil, A. M. (2003). Neurocognitive outcomes following pediatric brain injury: A developmental approach. Journal of School Psychology, 41, 337–353.
Goldstein, G., Allen, D. N., & Caponigro, J. M. (2010). A retrospective study of heterogeneity in neurocognitive profiles associated with traumatic brain injury.
Brain Injury, 24 (4), 625–635.
Goldstein, G., Allen, D. N., Minshew, N. J., Williams, D. L., Volkmar, F., Klin, A., et al. (2008). Structure of intelligence in children and adults with high
functioning autism. Neuropsychology, 22, 301–311.
Goldstein, G., Allen, D. N., & Seaton, B. E. (1998). A comparison of clustering solutions for cognitive heterogeneity in schizophrenia. Journal of the
International Neuropsychological Society, 4 (4), 353–362.
Gross, D., Fogg, L., Garvey, C., & Julion, W. (2004). Behavior problems in young children: An analysis of cross-informant agreements and disagreements.
Research in Nursing and Health, 27, 413–425.
Hair, J. F., Black, B., Babin, B., Anderson, R. E., & Tatham, R. (1992). Multivariate data analysis with reading (6th ed.). New Jersey: Prentice Hall.
Hawley, C. A. (2003). Reported problems and their resolution following mild, moderate and severe traumatic brain injury amongst children and adolescents in
the UK. Brain Injury, 17, 105–129.
Hawley, C. A. (2004). Behaviour and school performance after brain injury. Brain Injury, 18, 645–659.
Kriel, R. L., Krach, L. E., & Panser, L. A. (1989). Closed head injury: Comparison of children younger and older than 6 years of age. Pediatric Neurology, 5 (5),
296–300.
Landis, J. R., & Koch, G. G. (1977). The measurement of observer agreement for categorical data. Biometrics, 33 (1), 159–174.
Lee, S. W., Elliott, J., & Barbour, J. D. (1994). A comparison of cross-informant behavior ratings in school-based diagnosis. Behavioral Disorders, 19 (2),
87–97.
Loeber, R., Green, S. M., & Lahey, B. B. (1990). Mental health professionals’ perception of the utility of children, mothers, and teachers as informants on
childhood psychopathology. Journal of Clinical Child Psychology, 19 (2), 136–143.
Malec, J. F., Machulda, M. M., & Smigielski, J. S. (1993). Cluster analysis of neuropsychological test results among patients with traumatic brain injury (TBI):
Implications for a model of TBI-related disability. Clinical Neuropsychologist, 7, 48–58.
Max, J., Koele, S., Smith, W., Sato, Y., Lindgren, S., Robin, D., et al. (1998). Psychiatric disorders in children and adolescents after severe traumatic brain
injury: A controlled study. Journal of the American Academy of Child and Adolescent Psychiatry, 37 (8), 832–840.
McAuley, C. (2000). Children’s adjustment over time in foster care: Cross-informant agreement, stability and placement disruption. British Journal of Social
Work, 30, 91–107.
McConaughy, S. H., Stanger, C., & Achenbach, T. M. (1992). Three-year course of behavioral/emotional problems in a national sample of 4- to 16-year-olds:
I. Agreement among inform-ants. Journal of the American Academy of Child and Adolescent Psychiatry, 31, 932–940.
632 D.N. Allen et al. / Archives of Clinical Neuropsychology 25 (2010) 618–633
at UN
LV
University L
ibraries on September 27, 2012
http://acn.oxfordjournals.org/D
ownloaded from
McKinlay, A., Grace, R., Horwood, J., Fergusson, D., & MacFarlane, M. (2009). Adolescent psychiatric symptoms following preschool childhood mild trau-
matic brain injury: Evidence from a birth cohort. The Journal of Head Trauma Rehabilitation, 24, 221–227.
Miller, L. J., & Donders, J. (2003). Prediction of educational outcome after pediatric traumatic brain injury. Rehabilitation Psychology, 48, 237–241.
Millis, S. R., & Ricker, J. H. (1994). Verbal learning patterns in moderate and severe traumatic brain injury. Journal of Clinical and Experimental
Neuropsychology, 16, 498–507.
Millis, S. R., Rosenthal, M., Novack, T. A., Sherer, M., Nick, T. G., Kreutzer, J. S., et al. (2001). Long-term neuropsychological outcome after traumatic brain
injury. The Journal of Head Trauma Rehabilitation, 16, 343–355.
Morris, R., Blashfield, R. K., & Satz, P. (1981). Neuropsychology and cluster analysis: Potentials and problems. Journal of Clinical Neuropsychology, 3,
79–99.
Mottram, L., & Donders, J. (2006). Cluster subtypes on the California Verbal Learning Test—Children’s version after pediatric traumatic brain injury.
Developmental Neuropsychology, 30, 865–883.
Murji, S., Rourke, S. B., Donders, J., Carter, S. L., Shore, D., & Rourke, B. P. (2003). Theoretically derived CVLT subtypes in HIV-1 infection: Internal and
external validation. Journal of the International Neuropsychological Society, 9, 1–16.
Naglieri, J. A., Goldstein, S., Delauder, B. Y., & Schwebach, A. (2005). Relationships between the WISC-III and the Cognitive Assessment System with
Conners’ rating scales and continuous performance tests. Archives of Clinical Neuropsychology, 20, 385–401.
Palmer, B. W., Heaton, R. K., Paulsen, J. S., Kuck, J., Braff, D., Harris, M. J., et al. (1997). Is it possible to be schizophrenic yet neuropsychologically normal?
Neuropsychology, 11, 437–446.
Randazzo, K., Landsverk, J., & Ganger, W. (2003). Three informants’ reports of child behavior: Parents, teachers, and foster parents. Journal of the American
Academy of Child and Adolescent Psychiatry, 42, 1343–1350.
Rassovsky, Y., Satz, P., Alfano, M. S., Light, R. K., Zaucha, K., McArthur, D. L., et al. (2006). Functional outcome of TBI I: Neuropsychological, emotional,
and behavioral mediators. Journal of Clinical and Experimental Neuropsychology, 28, 567–580.
Reitan, R. M., & Wolfson, D. (1993). The Halstead-Reitan Neuropsychological Test Battery: Theory and clinical interpretation (2nd ed.). Tucson, AZ:
Neuropsychology.
Reynolds, C. R., & Bigler, E. D. (1994a). Test of Memory and Learning. Austin, TX: Pro-Ed.
Reynolds, C. R., & Bigler, E. D. (1994b). Test of Memory and Learning: Examiner’s manual. Austin, TX: Pro-Ed.
Reynolds, C. R., & Bigler, E. D. (1996). Factor structure, factor indexes, and other useful statistics for interpretation of the Test of Memory and Learning
(TOMAL). Archives of Clinical Neuropsychology, 11, 29–43.
Reynolds, C. R., & Ford, L. (1994). Comparative Three-Factor Solutions of the WISC-III and WISC-R at 11 Age Levels Between 6-l/2 and 16-l/2 Years.
Archives of Clinical Neuropsychology, 9, 553–570.
Reynolds, C. R., & Kamphaus, R. W. (1992). Behavior Assessment System for Children. Circle Pines, MN: American Guidance Service.
Reynolds, C. R., & Voress, J. K. (2007). Test of Memory and Learning (2nd ed.). Austin, TX: Pro-Ed.
Roman, M. J., Delis, D. C., Willerman, L., Magulac, M., Demadura, T. L., de la Pena, J. L., et al. (1998). Impact of pediatric traumatic brain injury on com-
ponents of verbal memory. Journal of Clinical and Experimental Neuropsychology, 20, 245–258.
Schwartz, L., Taylor, H. G., Drotar, D., Yeates, K. O., Wade, S. L., & Stancin, T. (2003). Long-term behavior problems following pediatric traumatic brain
injury: Prevalence, predictors, and correlates. Journal of Pediatric Psychology, 28, 251–263.
Stolberg, P., Terranova, J., Jones, P. W., Allen, D. N., & Mayfield, J. (2009). Performance on the Woodcock–Johnson Test of Achievement of individuals with
traumatic brain injury [Abstract]. Archives of Clinical Neuropsychology, 24 (5), 476.
Teasdale, G., & Jennett, B. (1974). Assessment of coma and impaired consciousness: A practical scale. Lancet, 2 (7872), 81–84.
Thaler, N. S., Allen, D. N., McMurray, J., & Mayfield, J. (2010). Sensitivity of the Test of Memory and Learning to attention and memory deficits in children
with ADHD. Clinical Neuropsychologist, 24 (2), 246–264.
Thaler, N. S., Terranova, J., Reynolds, C. R., Mayfield, J., & Allen, D. N. (2010). Teacher reported behavioral disturbances in children with TBI: An
Examination of the BASC-2. Presented at the American College of Professional Neuropsychology Annual Meeting, Feb. 26–28, Las Vegas, NV.
Thompson, N. M., Francis, D. J., Stuebing, K. K., Fletcher, J. M., Ewing-Cobbs, L., Miner, M. E., et al. (1994). Motor, visual-spatial, and somatosensory skills
after closed head injury in children and adolescents: A study of change. Neuropsychology, 8, 333–342.
Tiffin, J. (1948). Purdue Pegboard Test. Lafayette, IN: Lafayette Instrument Company.
van der Heijden, P., & Donders, J. (2003). WAIS-III factor index score patterns after traumatic brain injury. Assessment, 10, 115–122.
Verger, K., Junque, C., Levin, H. S., Jurado, M. A., Perez-Gomez, M., Bartres-Faz, D., et al. (2001). Correlation of atrophy measures on MRI with neurop-
sychological sequelae in children and adolescents with traumatic brain injury. Brain Injury, 15, 211–221.
Wexler, B. E., Zhu, H., Bell, M. D., Nicholls, S. S., Fulbright, R. K., Gore, J. C., et al. (2009). Neuropsychological near normality and brain structure abnorm-
ality in schizophrenia. The American Journal of Psychiatry, 166, 189–195.
Wiegner, S., & Donders, J. (1999). Performance on the California Verbal Learning Test after traumatic brain injury. Journal of Clinical and Experimental
Neuropsychology, 21, 159–170.
Woodcock, R. W., & Johnson, M. B. (1989). Woodcock–Johnson psycho-educational battery – revised. Chicago: Riverside Publishing.
Yeates, K. O., Armstrong, K., Janusz, J., Taylor, H. G., Wade, S., Stancin, T., et al. (2005). Long-term attention problems in children with traumatic brain
injury. Journal of the American Academy of Child and Adolescent Psychiatry, 44 (6), 574–584.
Yeates, K. O., Swift, E., Taylor, H. G., Wade, S. L., Drotar, D., Stancin, T., et al. (2004). Short- and long-term social outcomes following pediatric traumatic
brain injury. Journal of the International Neuropsychological Society, 10, 412–426.
Yeates, K. O., & Taylor, H. G. (2006). Behavior problems in school and their educational correlates among children with traumatic brain injury. Exceptionality,
14, 141–154.
Yeates, K. O., Taylor, H. G., Wade, S. L., Drotar, D., Stancin, T., & Minich, N. (2002). A prospective study of short- and long-term neuropsychological out-
comes after traumatic brain injury in children. Neuropsychology, 16, 514–523.
Young, S., & Gudjonsson, G. (2005). Neuropsychological correlates of the YAQ-S and YAQ-I self- and informant-reported ADHD symptomatology, emotion-
al and social problems and delinquent behaviour. British Journal of Clinical Psychology, 44, 47–57.
D.N. Allen et al. / Archives of Clinical Neuropsychology 25 (2010) 618–633 633
at UN
LV
University L
ibraries on September 27, 2012
http://acn.oxfordjournals.org/D
ownloaded from