Less Unique Variance Than Meets the Eye: Overlap Among TraditionalNeuropsychological Dimensions in Schizophrenia

Dwight Dickinson1,2,3 and James M. Gold2,3

2VISN 5 Mental Illness Research, Education and Clinical Center,Baltimore, MD; 3Department of Psychiatry, University of Mary-land School of Medicine, Baltimore, MD

Themagnitude of the overlap among dimensions of neuropsy-chological testperformance inschizophreniahasbeenthesub-ject of perennial controversy. This issue has taken on renewedimportance with the recent focus on cognition as a treatmenttarget in schizophrenia. A substantial body of factor analyticliterature indicates that dimensions are separable in schizo-phrenia. However, this literature is generally uninformativeas to whether the separable dimensions are independent,weakly correlated, or strongly correlated. Factor analyseshave often used methods (ie, principal components analysiswith orthogonal rotation) that preclude this determination,and correlations among factor-based domain compositesand underlying measures have been reported infrequently inthese studies. Current meta-analyses of reported ‘‘between-dimension’’ correlations for individual neuropsychologicalmeasures and for cognitive domain composite variables indi-cate that cognition variables in schizophrenia are correlated,on average, at a ‘‘medium’’ level of r 5 0.37 for individualmeasures from different cognitive dimensions and r 5 0.45fordomaincomposites.Because these aremeanbivariate cor-relations, the multiple correlation of an individual measurewith all the other measures in a cognitive battery is likely tobe higher. Measure reliabilities of 0.80 or less also implygreater commonality among traditional neuropsychologicalmeasures. In short, there are underappreciated constraintson the amount of reliable cognitive performance variance intraditional neuropsychological test batteries that is free tovary independently.Theabilityof suchbatteries toreveal cog-nitivedomain–specific treatmenteffects inschizophreniamaybe much more limited than is generally assumed.

Key words: schizophrenia/cognition/correlation/neuropsychological assessment/clinical trials

Introduction

The Measurement and Treatment Research to ImproveCognition in Schizophrenia (MATRICS) initiative hascatalyzed the identification and testing of promising newcognitive treatments, which could lead to importantadvances in schizophrenia therapeutics. The current spot-lighton cognition in schizophreniacreates an opportunity,and a reason, to revisit an issue that has long nagged theschizophrenia research community: the degree to whichthe different neuropsychological performance domainsin schizophrenia (and underlying measures) are indepen-dent of one another, weakly correlated, or substantiallyoverlapping. A growing body of factor analytic literaturesupports a multifactor structure of neuropsychologicaltest performance in schizophrenia.1,2 Some in the fieldhave argued that these factors, and the separate groupsof traditional neuropsychological variables used to mea-sure them, are sufficiently independent of one anotherto be suitable for assaying discrete neural systems (eg,Eganet al3). Others havehighlighted the multidimensionalnature of most neuropsychological tests and the empiricalrelationships among cognitive domains and underlyingmeasures and have questioned the directness of factorconnections to discrete neural substrates.4,5 This discus-sion continues. For example, an important part of theMATRICS initiative was the development of a batteryof neuropsychological tests that could provide a set of con-sistent outcome measures for clinical trials of cognition-enhancing drugs for schizophrenia.6 The broad sampleof participating experts was unambiguous that ‘‘reliablecoverageofthemajorseparablecognitivedeficits inschizo-phrenia’’ would be an ‘‘essential’’ feature of the battery.

Ideally, separable in this context would refer to cognitivedeficits with distinct causes or neural substrates that maytherefore respond to different types of pharmacologicalinterventions. However, at a minimum, separable should re-fer to cognitive deficits that are distinguishable at the statis-tical or analytic level, such as in the results of factor analyticstudies (Nuechterlein et al7(p30)).

At the same time, this requirement ‘‘raised a number ofpoints of consideration primarily concerning the abilityto differentiate distinct cognitive factors. Many expertsargued that it is important to assess effects on separable

1To whom correspondence should be addressed; VISN 5 MentalIllness Research, Education and Clinical Center, 10 North GreeneStreet, Suite 6a (BT/MIRECC), Baltimore, MD 21201; tel: 410-605-7817, fax: 410-605-7739, e-mail: Dwight.Dickinson@va.gov.

Schizophrenia Bulletin vol. 34 no. 3 pp. 423–434, 2008doi:10.1093/schbul/sbm092Advance Access publication on August 16, 2007

Published by Oxford University Press 2007.

423

cognitive constructs but that we are not able to separatethem as clearly as we would like’’ (Kern et al8(p16), seeCarter9).

Numerous trials of cognition-enhancing agents forschizophrenia are planned or under way. Some of thesetrials will test pharmacological agents that are thought tohave discrete neural targets and novel mechanisms of ac-tion. However, it is unclear that the most commonly usedassessment approach—ie, batteries of mainly traditionalneuropsychological instruments—will be able to discrim-inate domain-specific cognitive treatment effects. Therehave been reports in the literature of somewhat specificeffects of certain agents on particular cognitive measuresor performance domains. Many of these reports are fromacute challenge studies rather than clinical trials (eg,using amphetamine,10,11 glucose,12,13 or nicotine14–16).Some have used narrowly targeted batteries 11,14,16

and/or nontraditional measures.10 Meanwhile, in schizo-phrenia clinical trials that have reported cognitive bene-fits using standard neuropsychological batteries (mostly,but not exclusively, antipsychotic trials), these effectshave been broadly generalized across domains of cogni-tive performance,17–21 small in magnitude,22,23 or both.24

Thus, there is still insufficient information for investiga-tors and sponsors of clinical trials to know whether typ-ical batteries of neuropsychological tests can reliablyidentify treatment effects that are specific to particularbrain systems or mechanisms.

Below, we first address an important conceptual issueconcerning the factor analytic literature that is related tothe debate about the magnitude of the overlap amongcognitive dimensions and measures in schizophrenia.We then present meta-analyses of the schizophrenia lit-erature from the past 10 years reporting correlationsamong individual and composite measures of neuropsy-chological performance. The analyses summarize corre-lation findings for traditional neuropsychologicalvariables that are the same as or similar to those includedin the MATRICS battery. This battery was developedthrough a broad-based consensus process, has been ap-proved for use in schizophrenia clinical trials by theNational Mental Health Advisory Council6 of the Na-tional Institute of Mental Health and is intended to bethe default neuropsychological test battery for this pur-pose. Although the debate over the magnitude of inter-relationships among cognitive domains and measureslong predates the MATRICS, we use the example ofthis battery for illustrative purposes because of the im-portant role it is expected to have in trials of cogni-tion-enhancing agents in the future.

Factor Analysis and the Independence of CognitiveDimensions

The field has looked to the schizophrenia factor analyticliterature to identify separable dimensions of cognitive

performance.7 Obtained factors differ somewhat amongstudies depending on the batteries used. Nevertheless, al-most all studies have found that cognitive performance inschizophrenia, as measured by traditional neuropsycho-logical tests, is characterized by multiple factors. It wasthis literature that guided the MATRICS in identifyingcognitive dimensions for representation in the consensusneuropsychological battery.7 The main conclusion of thatsection of the MATRICS report was that:

the evidence, across all available studies, favored sufficientseparation of the selected cognitive dimensions to includethem as different cognitive dimensions in a battery intendedto examine treatment effects in clinical trials (Nuechterleinet al7(p36)).

It bears emphasis, though, that limitations of the under-lying literature precluded conclusions about the magni-tude of overlap among these cognitive dimensions.Most of the studies reviewed used principal componentsanalysis (PCA) accompanied by orthogonal rotationmethods (eg, Varimax).7 Technically speaking, PCAyields components, not factors. The distinction is nottrivial, although it is often overlooked. PCA segmentsall the variance in a dataset into components, includingvariance that might be caused by some set of underlying,common latent factors and variance that is unique to in-dividual measures.25 In contrast, the various methods offactor analysis (eg, principal axis, maximum likelihood)only segment the variance attributable to the commonfactors of the dataset. PCA is perfectly acceptablewhen the goal is simple data reduction. However, thenonseparation of unique measure variance in PCA meansthat it is technically inaccurate and potentially misleadingto describe the components as ‘‘factors’’ or ‘‘dimensions’’representing underlying latent constructs (eg, attention/working memory, executive functioning).25

Another overlooked point is that orthogonal rotationmethods do not establish the independence of cognitivedimensions. Rotation of factors is undertaken to improvethe resolution and interpretability of the derived factors.As the name suggests, orthogonal rotations (such as Var-imax) yield independent, orthogonal factors. They havea number of desirable mathematical properties and arewidely used.26 However, orthogonal rotation does notdiscover independent, uncorrelated dimensions in a data-set. Rather, this type of rotation groups and weights in-dividual variables subject to the requirement that theresulting factors be orthogonal, whether or not this isconsistent with the actual structure of the data. Findingsfrom a recent schizophrenia clinical trial27 illustrate thispossible difference between relationships among orthog-onal factors and relationships among observed variables.There, baseline cognitive data were subjected to PCA,and 3 orthogonal factors were derived. Yet, when com-posite cluster scores were constructed on the basis of theorthogonal components, they nevertheless showed high

D. Dickinson & J. M. Gold

424

intercorrelations (r = 0.44–0.63). The fact that the firstunrotated component in that analysis accounted for45% of the total variance was taken to indicate ‘‘the pres-ence of a large, reliable general factor’’ and led the inves-tigators to use a single global cognitive composite asthe primary cognitive outcome variable (Greenet al27(p975)). Thus, while orthogonal rotations necessarilyresult in factors (or components) that are mathematicallyindependent, such solutions may not always provide thebest model of the actual interrelationships in the data. In-deed, where other methods have been used in research onschizophrenia cognitive performance, derived factorstend to show strong correlations.2,28

The main points of this discussion of factor analyticmethodology are as follows. The factor analytic literatureprovides strong support for the conclusion that there aremultiple separable cognitive factors in schizophrenia.However, the existence of a good-fitting, multiple factorsolution does not imply weak associations among factors,measures loading on different factors, or the latent cog-nitive dimensions that the factors represent. As noted,it is entirely possible to derive a good-fitting multiplefactor solution in which the different derived factorsare strongly correlated.2,28 Composite scores based onsuch factors may show strong correlations, as well.1 Inshort, ‘‘factors,’’ ‘‘dimensions,’’ and ‘‘domains’’ may beseparable in statistical terms while still showing substan-tial intercorrelations. Thus, the schizophrenia factoranalytic literature leaves largely unresolved the extentof the overlap among neuropsychological tests and fac-tor-based cognitive composites. And it remains unclearwhether cognitive batteries composed of traditional neu-ropsychological measures will be able to identify domain-specific effects in upcoming clinical trials. The followingmeta-analyses of correlations among individual measuresand among domain composite scores were undertaken toprovide additional information about this issue.

Meta-analysis of Neuropsychological Correlations inSchizophrenia

Study Selection

The MATRICS Consensus Cognitive Battery was cre-ated through a painstaking and broad-based consensusprocess including the academic community, the NationalInstitutes of Health, the U.S. Food and Drug Adminis-tration (FDA), and the pharmaceutical industry.6 Be-cause of this, it is fair to think of the battery asreflecting the accumulated experience of the entire fieldin using traditional neuropsychological measures to effi-ciently assess cognitive performance and treatment-related change in schizophrenia. Given this pedigree,and the likely use of the battery in upcoming schizophre-nia clinical trials, we used it as a starting point for ouranalysis. The MATRICS battery consists mainly of

well-known measures that have been time-tested inschizophrenia research (eg, the Hopkins Verbal LearningTest [HVLT] and Trail Making Test) or updated ana-logues of such measures (eg, Brief Assessment of Cogni-tion in Schizophrenia Symbol Digit Coding). Nine of thecognitive tests included in the MATRICS battery address6 much researched dimensions of performance in schizo-phrenia: speed of processing, attention/vigilance, work-ing memory, verbal learning and memory, visuallearning and memory, and reasoning and problem solv-ing. An additional test addresses ‘‘social cognition.’’ Al-though the empirical base for this dimension is morelimited than it is for the first 6, it was included in recog-nition of fast-growing interest in social cognition impair-ment in this illness.7 The final MATRICS battery isreflected in table 1.

Examination of bivariate correlations reported in theliterature directly addresses the magnitude of the overlapamong cognitive measures and dimensions in schizophre-nia. We assembled reports of correlations among individ-ual neuropsychological variables from schizophreniasamples that are the same as those included in theMATRICS battery, close analogues of those measures(eg, other list learning tasks in addition to the HVLT),or widely used measures that tap the same dimensionsspecified by the MATRICS (eg, Wechsler Logical Mem-ory for verbal learning, and the Wisconsin Card SortingTest for reasoning and problem solving). (Specific cita-tions for the measures referred to in this article are avail-able in standard neuropsychological reference works29

and cited studies.) In view of the limited literatureaddressing social cognition and underlying measures,we left this dimension out of the analysis. Our searchwas restricted to the past 10 years because all the individ-ual MATRICS battery measures, or close analogues,were issued and in general use during this period. Twoapproaches were used to locate articles for the analysis.First, articles were identified through a series of searchesof the PubMed and PsycINFO electronic databases withcombinations of key words schizophreni*, correlat* cog-niti*, neuropsychologi*, memory, attention, executive, andprocessing speed. Second, we searched the reference listsfrom these articles to identify additional publications.

The following criteria were used to select studies forreview. (1) The study must have reported correlationsamong neuropsychological measures and/or domaincomposites the same as or similar to those representedin the MATRICS cognitive battery in a sample (or sam-ples) of individuals with schizophrenia or schizoaffectivedisorder (correlations for social cognition measures andcomposites were not sought). (2) Current analyses target‘‘between-dimension’’ associations, so we searched forstudies that reported correlations between measuresfrom the different cognitive dimensions defined by theMATRICS and excluded studies if they only reported‘‘within-dimension’’ correlations. (3) The study must

425

Schizophrenia Neuropsychological Dimensions

have been published between 1997 and June 2006. (4) Thestudy must have based diagnoses on contemporary diag-nostic criteria (eg, Diagnostic and Statistical Manual ofMental Disorders, Revised Third Edition; Diagnosticand Statistical Manual of Mental Disorders, Fourth Edi-tion; International Classification of Diseases, Ninth Revi-sion, or later). (5) Results must have been reported withsufficient detail to allow calculation of effect sizes. (6) Fi-nally, the study must have been reported in English. Morethan 500 studies were identified through the overlappingdatabase searches. Initially, abstracts and titles werereviewed to eliminate studies that obviously failed tomeet criteria (eg, no correlations were reported).Thirty-three articles reporting schizophrenia correlationson relevant neuropsychological measures were retainedfor examination in greater detail. Of these, 2 includedonly ‘‘within-dimension’’ and no ‘‘between-dimension’’correlations. Ten studies reported correlations of relevantvariables or analogues only with other variables that areoutside of our current focus (eg, verbal IQ, motor speed).Twenty-one studies met inclusion criteria for the meta-analysis. Of these, 12 studies reported multiple, be-tween-dimension, bivariate correlations between individ-ual MATRICS battery variables or analogues.28,30–40

Seven studies reported bivariate correlations betweencomposite scores that matched or were similar to certainMATRICS-designated cognitive dimensions.1,27,41–45

Two studies reported both individual measure and com-posite correlations.46,47

Data Extraction and Analysis

We conducted 2 parallel analyses, 1 for bivariate corre-lations between individual measures and 1 for bivariatecorrelations between composite measures. All analyseswere performed using the Comprehensive Meta-Analysissoftware package.48 For each study, we entered the spec-ified correlations and sample sizes into the database.These values were averaged study-by-study using appro-priate transformations and a random-effects model,rather than the more commonly employed fixed-effectsmodel.49,50 This conservative estimation method yieldsmore generalizable parameter estimates than would bederived using a fixed-effects model, most clearly evidentin larger estimated standard errors and confidence inter-vals (CIs).49 Study-by-study mean correlations wereweighted by sample size and combined, again using a ran-dom-effects model, to obtain a grand mean correlation.We recorded information about a number of potentialmoderator variables, including age, education, treatmentsetting, and chronicity of the patient samples. We usedmeta-regression (ie, regressing study mean correlationsagainst study moderator variable values) to test the influ-ence of the moderator variables.

Results

Main meta-analysis results are presented in table 2 forindividual variable correlations and in table 3 for corre-lations between composite variables. Across 14 studies of

Table 1. Cognitive Dimensions and Tests in MATRICS Consensus Cognitive Battery

Speed ofprocessing

Brief Assessment of Cognition in Schizophrenia: SymbolDigit Coding

Timed paper-and-pencil test; respondent uses a key towrite digits that correspond to nonsense symbols

Category fluency: animal naming Oral test in which respondent names as many animals asshe/he can in 1 min

Trail Making Test: Part A Timed paper-and-pencil test; respondent draws a linethrough consecutively numbered circles

Attention/vigilance

Continuous Performance Test—Identical Pairs Computer-administered measure of sustained attention;respondent presses a response button to consecutivematching numbers

Working memory(nonverbal)

Wechsler Memory Scale-III: Spatial Span Board with 10 irregularly spaced cubes; respondent tapscubes in same (or reverse) sequence

(verbal) Letter-Number Span Orally administered test requires mental reordering ofstrings of number and letters

Verbal learning Hopkins Verbal Learning Test—Revised Orally administered test using a list of 12 words; 3learning trials; tests recall of words

Visual learning Brief Visuospatial Memory Test—Revised Requires reproduction of 6 geometric figures frommemory

Reasoning andproblem solving

Neuropsychological Assessment Battery: Mazes Seven timed paper-and-pencil mazes

Social cognition Mayer-Salovey-Caruso Emotional Intelligence Test:Managing Emotions

Paper-and-pencil multiple-choice test that assesses howpeople manage their emotions

Note: MATRICS, Measurement and Treatment Research to Improve Cognition in Schizophrenia. Specific citations for theneuropsychological measures referred to in this article are available in standard neuropsychological reference works29 and in theMATRICS Consensus Cognitive Battery manual6.

426

D. Dickinson & J. M. Gold

Table 2. Summary of Bivariate Correlations Reported Between Individual Neuropsychological Measures in Different MATRICS Cognitive Dimensions

StudySample Size andCharacteristics

MATRICS Measures(or analogues) forWhich BivariateCorrelations Are Reported

Reported ‘‘Within-Dimension’’Correlations That Are ExcludedFrom Analysis (SpecificMATRICS Dimension)

Number ofIncluded‘‘Between-Dimension’’Correlations

Mean of ‘‘Between-Dimension’’ Correlations[95% CI] (Full Range)

Allen et al30 169 male inpatients; age36.2 (7.9), education 12.3(1.9), duration 12.7 (7.3)

WAIS-R measures: Digit Span,Arithmetic, Block Design, DigitSymbol

Arithmetic/Digit Span (WM) 5 .32 [.25–39] (.26–.43)

Chen et al31 204 (122 male)inpatients; age 40.4(12.2), education 8 (3.4)

Logical Memory (immediate), DigitSpan, Category Fluency, VisualReproduction, PerseverativeErrors (WCST)

None 10 .24 [.18–.30] (.07–.37)

Dickinsonet al28

120 (91 male)outpatients; age 39.3(8.4), education ;12,duration ;18

WAIS-III measures: Block Design,Matrix Reasoning, Arithmetic,Digit Span, Letter-NumberSequencing, Digit Symbol, SymbolSearch

Block Design/Matrix Reasoning(RP), Arithmetic/Digit Span &Digit Span/Letter-Number &Arithmetic/Letter-Number (WM),Digit Symbol/Symbol Search (PS)

16 .49 [.46–.53] (.31–.61)

Friis et al46 219 (125 male) firstepisode; age 27.9 (9.4),education 12.1 (2.7)

CVLT (immediate), PerseverativeErrors (WCST), Letter Fluency,Trails A and B (combined),Identical Pairs CPT

Letter Fluency/Trail Making (PS) 13 .31 [.27–.34] (.21–.45)

Gold et al32 36 inpatients; age 34.4(7.8), education 12.8(1.6)

Correlations of Letter-Number Spanand Perseverative Errors (WCST)with Trails B, Digit Span,Gordon’s CPT, and Letter Fluency

Digit Span/Letter-Number (WM),Perseverative Errors/Trails B (RP)

7 .44 [.33–.54] (.16–.56)

Holthausenet al33

118 (87 male) first orsecond episode; age 23.3(5.3), duration < 1

CVLT (immediate), Rey ComplexFigure Memory, CategoryFluency, Stroop (color and wordconditions combined), Gordon’sCPT

Category Fluency/Stroop color andword (PS)

9 .30 [.24–.36] (.15–.44)

Keefe et al47 1123 to 1332 (74% male)outpatients; age 40.6(11.1), education 12.1(2.2), duration 14.4(10.7)

Category and Letter Fluency(combined), Digit Symbol, WCSTvariables (combined), WISCMazes, HVLT, VisuospatialWorking Memory variables(combined), Letter-NumberSequencing, Identical Pairs CPTvariables (combined)

Category and Letter Fluency/DigitSymbol (PS), WCST/WISC Mazes(RP), Visuospatial WorkingMemory/Letter-Number (WM)

25 .40 [.37–.44] (.26–.56)

Keefe et al34 ;140 (79% male)inpatients andoutpatients; age 34.7(11.3), education 12.0(2.3)

Brief Assessment of Cognition inSchizophrenia subtest correlations:Verbal List Learning, DigitSequencing (span), Category andLetter Fluency (combined),Symbol Digit Coding, Tower ofLondon

Category and Letter Fluency/SymbolDigit (PS)

9 .33 [.24–.41] (.08–.50)

42

7

Schizo

phren

iaNeuropsychologica

lDim

ensio

ns

Table 2. Continued

StudySample Size andCharacteristics

MATRICS Measures(or analogues) forWhich BivariateCorrelations Are Reported

Reported ‘‘Within-Dimension’’Correlations That Are ExcludedFrom Analysis (SpecificMATRICS Dimension)

Number ofIncluded‘‘Between-Dimension’’Correlations

Mean of ‘‘Between-Dimension’’ Correlations[95% CI] (Full Range)

Kurtz et al35 32 (23 male) outpatients;age 39.9 (9.0), education11.9 (2.0), duration 18.9(10.7)

Correlations of Perseverative Errors(WCST) and Penn ConditionalExclusion Test with LetterFluency, Trails A, Trails B, DigitSymbol, HVLT (immediate)

Perseverative Errors/Penn CET andPerseverative Errors/Trails B andPenn CET/Trails B (RP)

8 .40 [.28–.50] (.22–.61)

Mahurinet al36

53 (45 male) inpatients;age 33.4 (10.1),education 11.2 (2.5),duration 12.8 (7.5)

Letter Fluency, Verbal Series CPT,Trails B, Symbol Digit, HVLT(immediate), Digit Span, WCST,Stroop (color-word)

Letter Fluency/Symbol Digit (PS),Trails B/WCST and Trails B/Stroop and WCST/Stroop (RP)

24 .43 [.38–.47] (.17–.59)

Morrenset al37

30 (22 male) inpatients;age 27.5 (7.0)

Correlations of Symbol Digit withRey AVLT, WCST, Identical PairsCPT, Letter-Number Sequencing,Trails A and B

Symbol Digit/Trails A (PS) 5 .30 [.18–.41] (.07–.42)

Silver et al38 27 male inpatients; age38 (13.5), education 11.2(3.0), duration 12.7(10.5)

Correlations of WM tasks, DigitSpan (backward), and DotWorking Memory Test with PennCPT, Benton Visual RetentionTest, Visual Object Learning Test,Penn Face Memory, PennAbstraction, Inhibition andWorking Memory Test

None 10 .46 [.29–.61] (.09–.81)

Stratta et al39 30 (21 male) inpatients;age 36.9 (8.5), education12.8 (3.6), duration 14.9(8.8)

Visual-Spatial Working MemoryTest, Digit Span (backward), DigitSymbol, Perseverative Errors(WCST)

Visual-Spatial Working Memory/Digits backward (WM)

4 .24 [.01–.46] (.01–.46)

Strauss andSummerfelt40

27 patients Dot Working Memory Test, Letter-Number Sequencing, SpatialWorking Memory Test,Perseverative Errors (WCST),HVLT (immediate)

Dot Test/Letter-Number and DotTest/Spatial Working Memory andLetter-Number/Spatial WorkingMemory (WM)

7 .27 [.12–.40] (.06–.42)

Sample-weighted grand mean (14 studies, ;2405 patients) [95% CI] .37 [.33–.40] z = 18.73 P < .001

Note: Specific citations for the neuropsychological measures referred to in this article are available in standard neuropsychological reference works29 and in the specificstudies cited. Cognitive dimensions specified by the MATRICS are abbreviated as follows: . MATRICS, Measurement and Treatment Research to Improve Cognition inSchizophrenia; CI, confidence interval; WAIS-R, Wechsler Adult Intelligence Scale-Revised; WM, working memory; WCST, Wisconsin Card Sorting Test; WAIS-III,Wechsler Adult Intelligence Scale-Third Edition; RP, reasoning and problem solving; PS, processing speed; CVLT, California Verbal Learning Test; CPT, continuousperformance test; WISC, Wechsler Intelligence Scale for Children; HVLT=Hopkins Verbal Learning Test; AVLT=Auditory Verbal Learning Test.

42

8

D.Dick

inson&

J.M.Gold

Table 3. Summary of Bivariate Correlations Reported Between Cognitive Composite Scores Representing Different MATRICS Cognitive Dimensions

StudySample Size andCharacteristics

Description of Cognitive Composite Scores for WhichBivariate Correlations Are Reported

Number ofIncludedCorrelations

Mean of Correlations[95% CI] (Full Range)

Binderet al41

40 (20 male) first episode;age ;31.5

Composites based on a priori variable groupings for VerbalLearning—Logical Memory, Paired Associates, and CVLTvariables; Visual Learning—Visual Reproduction variables;Working Memory—Digit Span and Reading Span;Processing Speed—Trails, Digit Symbol, and Stroopvariables

6 .35 [.21–.47] (.12–.54)

Dickinsonet al1

148 (97 male) outpatients;age 33.5 (7.5), education13.2 (2.2), duration ;11

Composites based on a priori variable groupings for VerbalLearning—CVLT and Logical Memory variables; VisualLearning—Visual Reproduction variables; ProcessingSpeed—Symbol Cancellation, Trails A, Category Fluency;Executive Function—Trails B, WCST variables, Digit Span

6 .61 [.56–.66] (.50–.69)

Friis et al46 219 (125 male) first episode;age 27.9 (9.4), education12.1 (2.7)

Factor analysis–based composites for VerbalLearning—CVLT variables; Working Memory—LetterFluency, Digit Span Distractibility variables, CPT hits;Executive Function—WCST variables

3 .34 [.27–.40] (.31–.37)

Greenet al27

62 (57 male) outpatients;age ;43.3 (;8.5),education ;12.7 (;1.2),duration ;25 (;5.5)

Factor analysis–based composites for Attention/Perception—CPT variables and Span of Apprehension;Memory and Fluency—CVLT, Fluency, and Digit SpanDistractibility variables; Executive Function—WCSTvariables

3 .54a [.33–.70] (.44–.63)

Gur et al41 53 (34 male) outpatients;age 34.1 (11.1), education13.4 (2.3), duration 10.4(9.5)

Composites of traditional measures correlated with compositesof computerized battery measures for Verbal Learning:Traditional—Logical Memory and CVLT variables andComputerized—Penn Word Memory Test variables;Executive: Traditional—WCST variables andComputerized—Computerized Ravens, Penn InhibitionTest, Penn Abstraction, Inhibition, and Working MemoryTest

2 .54 [.40–.66] (.48–.60)

Hill et al43 45 (28 male) first episode;age 26.1 (8.1), education13.7 (3.3)

Composites based on a priori variable groupings for VerbalLearning—CVLT variables; Visual Learning—VisualReproduction variables; Executive Function—WCST andStroop variables, Trails B, and Letter Fluency;Attention—Trails A, Digit Symbol, Digit Span,Cancellation variables

6 .52 [.38–.64] (.35–.71)

Keefe et al47 1123 to 1332 (74% male)outpatients; age 40.6(11.1), education 12.1(2.2), duration 14.4 (10.7)

Composites based on a priori variable groupings for VerbalLearning—HVLT variables; Processing Speed—LetterFluency, Category Fluency, Digit Symbol, GroovedPegboard; Reasoning—WCST variables, WISC Mazes;Working Memory—Visuospatial Working Memory, Letter-Number Sequencing; Vigilance—Identical Pairs CPT

10 .50a [.47–.53] (.36–.62)

42

9

Schizo

phren

iaNeuropsychologica

lDim

ensio

ns

schizophrenia samples and 2405 patients, the mean be-tween-dimension correlation reported between individualneuropsychological variables of the sort included in theMATRICS battery was r = 0.37, with study means rang-ing from r = 0.24 to r = 0.49. Summarizing informationfrom 9 studies and 1860 patients, the mean correlation ofcognitive composite variables was higher, r = 0.45, witha wider range of study means, from r = 0.16 to r = 0.61.Cohen’s convention is that correlations of approximately0.1, 0.3, and 0.5 are ‘‘small,’’ ‘‘medium,’’ and ‘‘large’’ ef-fect sizes, respectively.51 Thus, the grand mean correla-tions derived here for individual measures and domaincomposites fall between Cohen’s medium and large effectsize designations.

While r = .37 represents a combined correlation acrossall the analyzed studies, it is likely that the mean between-dimension correlation for measures specific to theMATRICS cognitive battery would be higher. This con-clusion follows from an analysis of the published datafrom the largest study included in the current review,which reports baseline cognitive findings from the Clin-ical Antipsychotic Trials of Intervention Effectiveness(CATIE).47 Of all the studies included in this review,the neuropsychological battery used in CATIE has themost specific measure overlap with the MATRICS bat-tery (as well as the largest subject sample), employing 6measures that are the same as or similar to measures spec-ified for the MATRICS battery. These 6 measures yielded14 between-dimension correlations out of the 25 between-dimension correlations averaged and reported in table 2.Combining just these correlations, in the same manner asin the foregoing analyses, yields a mean correlation ofr = 0.45 (95% CI = 0.41–0.49). This compares toa mean correlation of r = 0.34 (95% CI = 0.30–0.38)for the CATIE battery correlations outside theMATRICS battery subset.

To be clear, the derived correlations address the tradi-tional neuropsychological measures and domains exem-plified by the MATRICS cognitive battery. Given limitedschizophrenia research on the MATRICS social cogni-tion measure, its relations with more traditional measureswere not included in these analyses. Thus, the points raisedin the following paragraphs may not apply to this dimen-sion of cognitive performance (but see Sergi et al52).

Meta-regression of mean study correlations on age wasnot significant, either for individual neuropsychologicalmeasures or for cognitive composites. Moderator vari-able information for education, treatment setting, andduration of illness was insufficient (ie, either missing ornonnormally distributed) to allow meta-regression onthese variables.

Discussion

There are 2 main conclusions of this review. First, theschizophrenia factor analytic literature provides strongT

able3.

Co

nti

nu

ed

Stu

dy

Sa

mp

leS

ize

an

dC

ha

ract

eris

tics

Des

crip

tio

no

fC

og

nit

ive

Co

mp

osi

teS

core

sfo

rW

hic

hB

iva

ria

teC

orr

ela

tio

ns

Are

Rep

ort

ed

Nu

mb

ero

fIn

clu

ded

Co

rrel

ati

on

sM

ean

of

Co

rrel

ati

on

s[9

5%

CI]

(Fu

llR

an

ge)

Vel

lig

an

eta

l44

40

(34

ma

le)

pa

tien

tste

sted

on

ho

spit

al

rele

ase

;a

ge

35

.7(9

.7),

edu

cati

on

11

.4(2

.5),

du

rati

on;

12

Co

mp

osi

tes

ba

sed

on

ap

rio

riv

ari

ab

leg

rou

pin

gs

for

Ver

ba

lM

emo

ry—

HV

LT

va

ria

ble

sa

nd

Dig

itS

pa

n;

Ex

ecu

tiv

eF

un

ctio

n—

WC

ST

an

dC

om

ple

xR

eact

ion

Tim

ev

ari

ab

les;

Su

sta

ined

Att

enti

on

—Id

enti

cal

Pa

irs

CP

Tv

ari

ab

les;

Vis

ua

lM

emo

ry—

Rey

Co

mp

lex

Fig

ure

Mem

ory

6.3

9[.

28

–.5

0]

(.2

4–

.58

)

Vil

lalt

a-G

ilet

al4

51

13

(77

ma

le)

ou

tpa

tien

ts;

ag

e4

1.6

(12

.8),

du

rati

on

18

.9(1

1.2

)

Co

mp

osi

tes

ba

sed

on

ap

rio

riv

ari

ab

leg

rou

pin

gs

for

att

enti

on

—C

on

ner

sC

PT

va

ria

ble

s,T

rail

sA

,D

igit

Sp

an

,a

nd

Dig

itS

ym

bo

l;A

bst

ract

ion

an

dF

lex

ibil

ity

—W

CS

Ta

nd

Str

oo

pv

ari

ab

les,

Tra

ils

B;

Ver

ba

lL

earn

ing

—S

pa

nis

hV

erb

al

Lea

rnin

gT

est,

‘‘O

per

ati

ve

Mem

ory

’’—

Ver

bal

Flu

ency

va

ria

ble

s,T

rail

sB

6.1

6[.

08

–.2

3]

(.0

1–

.3)

Sa

mp

le-w

eig

hte

dg

ran

dm

ean

(9st

ud

ies,;

18

60

pa

tien

ts)

[95

%C

I].4

5[.

35

–.5

4]z=

8.2

P<

.00

1

Note:

Sp

ecif

icci

tati

on

sfo

rth

en

euro

psy

cho

log

ica

lm

easu

res

refe

rred

toin

this

art

icle

are

av

ail

ab

lein

sta

nd

ard

neu

rop

sych

olo

gic

al

refe

ren

cew

ork

s29

an

din

the

spec

ific

stu

die

sci

ted

.M

AT

RIC

S,

Mea

sure

men

ta

nd

Tre

atm

ent

Res

earc

hto

Imp

rov

eC

og

nit

ion

inS

chiz

op

hre

nia

;C

I,co

nfi

den

cein

terv

al;

CV

LT

,C

ali

forn

iaV

erb

al

Lea

rnin

gT

est;

CP

T,

con

tin

uo

us

per

form

an

cete

st;

WC

ST

,W

isco

nsi

nC

ard

So

rtin

gT

est;

HV

LT=

Ho

pk

ins

Ver

ba

lL

earn

ing

Tes

t.aT

hes

est

ud

ies

rep

ort

edth

era

ng

eo

fd

om

ain

com

po

site

corr

ela

tio

ns,

rath

erth

an

each

biv

ari

ate

corr

ela

tio

n(G

reen

eta

l27

(p9

75

) ,K

eefe

eta

l47

(p2

03

9) ).

Mea

nco

rrel

ati

on

sw

ere

calc

ula

ted

fro

mth

ese

ran

ge

end

po

ints

.

430

D. Dickinson & J. M. Gold

evidence that dimensions of cognitive performance inschizophrenia are separable but does not reveal the de-gree of separateness among those dimensions or amongthe neuropsychological measures used to measure them.Many of the studies in this literature have used PCA withorthogonal rotational methods that make independenceamong factors a precondition of the factor analytic solu-tion. Such solutions may not optimally identify true ‘‘fac-tors’’ or accurately reflect variable interrelationships inthe underlying data. Second, the current meta-analysesof reported ‘‘between-dimension’’ correlations for indi-vidual cognitive measures and for cognitive domain com-posite variables indicate that, in schizophrenia, pairs ofcognitive variables from different domains correlate sig-nificantly. The basis for the first conclusion was pre-sented earlier, so this discussion focuses on the secondconclusion and its implications.

We assembled studies of schizophrenia patients thatreported bivariate correlations between individual neuro-psychological measures and between cognitive domaincomposites. Given the consensus development of theMATRICS cognitive battery and its likely prominencein schizophrenia research in coming years, we took thisbattery as a starting point. We focused on correlationsreported for measures and domains that are the sameas or similar to those specified for the MATRICS cogni-tive battery. In order to highlight the magnitude of rela-tionships across cognitive dimension boundaries, weexamined only ‘‘between-dimension’’ correlations (eg,between an individual verbal list learning measure,such as the HVLT, and an individual working memorymeasure, such as Letter-Number Sequencing) and ex-cluded ‘‘within-dimension’’ correlations (eg, between 2processing speed measures, such as Digit Symbol andTrails A). Our meta-analysis averaged relevant correla-tions reported in the literature, first within studies andthen between studies, to derive representative grandmean correlations. Results across numerous studiesand schizophrenia subjects indicated that the typical ‘‘be-tween-dimension’’ correlation reported in the literaturebetween traditional neuropsychological measures is r =0.37. Adding weight to this finding, ‘‘between-dimension’’correlations among individual measures in a MATRICSbattery subset from the CATIE study were higher (meanr = 0.45). The typical correlation between domain com-posites is comparable (r = 0.45).

For reasons discussed subsequently, we believe that bi-variate correlations of this magnitude place an importantconstraint on the ability of traditional neuropsychologicalbatteries to signal independent, domain-specific treatmenteffects. Likely objections to this conclusion include the fol-lowing: (1) that these correlations are not large from thestandpoint of the squared correlation (ie, "r2"), leavingmuch of performance on 1 neuropsychological measurefree to vary independently of performance on other meas-ures, and (2) that even large correlations among neuropsy-

chological measures at baseline do not precludeindependent dimensions of change with treatment.

Regarding the first objection, we believe that squaredcorrelation calculations understate the implications ofthe derived correlations for 3 principal reasons. First,our position is that squared correlations are simply thewrong metric by which to evaluate the likelihood that in-dividual neuropsychological measures can signal uniquetreatment effects in the context of other such measures.The squared correlation is relevant in regression as ‘‘theamount of variance in 1 variable that is predictable fromknowledge of the other variable’’ (Jensen53(p198); see alsoCohen J and Cohen P54). In contrast, the correlation co-efficient directly characterizes the nature and degree ofrelation between 2 variables because ‘‘the proportionof their total variance that [two variables] have in com-mon’’ (Jensen53(p198). In this sense, it is analogous to the‘‘genetic correlation,’’ which directly indexes (ie, in itsunsquared form) the proportion of genetic influenceon 2 variables that is common to both.55 As Jensen statesfurther,

when we wish to express the proportion of the total variancein one variable that is predictable from another variable, weuse r2. When we speak of the proportion of the total vari-ance that the two variables have in common, we use r(Jensen53(p255)).

Second, it bears emphasis that the average correlationvalues derived in these meta-analyses are bivariate corre-lations. In other words, a given variable from a typicalbattery of traditional neuropsychological tests is likelyto have an observed correlation approximatingr = 0.37 with each of the ‘‘other domain’’ variables inthe battery. (Correlations with other variables withina domain will be higher, by definition.) Ideally, we wouldlike to know the multiple correlation of the individualvariable with the entire group of ‘‘other domain’’ varia-bles (ie, excluding ‘‘within-domain’’ correlations). Thisinformation is too seldom available for meta-analysis.However, a specific example, also from the CATIE re-port, illustrates the point. In that study, the mean corre-lation of the HVLT with individual neuropsychologicalmeasures from other domains was r = 0.40 (95% CI =0.32–0.46; range = 0.27–0.53). At the same time, the cor-relation of the HVLT with a composite of all the ‘‘otherdomain’’ CATIE neuropsychological measures (exclud-ing the HVLT) was r = 0.55 (Keefe et al47(p2039)). In gen-eral, with such neuropsychological measures, it is fair toassume that multiple correlations (even excluding within-domain correlations) will increase relative to the mean bi-variate correlation as the number of tests in a batteryincreases. Consequently, the ability of a given test batteryvariable to signal a unique effect, above and beyond theentire group of ‘‘other domain’’ test battery variables,would be less than indicated by the mean bivariateassociation.

431

Schizophrenia Neuropsychological Dimensions

Third, even multiple correlations would leave a misim-pression about the amount of potentially unique variancein the current context—ie, where the focus is on correla-tions between psychological measures known to have ad-equate, but not exceptional, reliability. Reliabilities set anupper limit on correlations between measures and are thebasis for the ‘‘correction for attenuation due to unreli-ability’’ from classical psychometric theory: the ‘‘disatte-nuated’’ correlation equals the observed correlationdivided by the square root of the product of the measures’reliabilities.26,56 Reported reliabilities for individual andcomposite cognitive measures in schizophrenia are sel-dom above (and often below) the 0.8 level.34,57–61 Apply-ing the disattenuation correction to the correlationsderived in this meta-analysis, using a generous reliabilityestimate of 0.80, yields an estimated bivariate correlationof r = 0.46 between individual measures from differentcognitive dimensions in schizophrenia (corrected up-wards from r = 0.37) and an estimated correlation ofr = 0.56 between typical domain composite scores (cor-rected from r = 0.45). An estimated correlation ofr = 0.56 would likewise result if the correction were ap-plied to correlations between variables within theMATRICS cognitive battery subset of measures reportedin the CATIE baseline data. Obviously, using as a startingpoint, the multiple correlation values reported in theCATIE study (Keefe et al47(p2039)), or lower reliabilityvalues, would result in considerably higher disattenuatedmultiple correlations. Of course, these adjustments arehypothetical. However, application of the disattenua-tion formula illustrates that, given typical reliabilitiesand bivariate correlations for traditional neuropsycho-logical measures in schizophrenia, considerably lessreliable cognitive performance variance is likely to befree to vary independently of other measures than itmight first appear.

A second likely objection to the significance of the typ-ical between-dimension correlations reported here is thateven strong baseline correlations do not necessarily pre-clude independent change in cognitive performance withtreatment. It is true that relatively specific cognitive treat-ment effects have sometimes been reported with pharma-cological manipulations. It is also true that most of thesereports have been based on narrow or specialized assess-ments in the context of acute challenge studies.10,11,13–16

The relevance of these studies for extended pharmacolog-ical treatment trials is uncertain. In actual clinical trialsreporting cognitive benefits for schizophrenia patients,and using typical neuropsychological batteries, theseeffects tend to be broadly generalized.17,19–24,62 Further,2 recent studies have used factor analytic techniques toexamine the factor structure of cognitive change with an-tipsychotic treatment in schizophrenia. Analyses of datafrom the CAFE clinical trial (Comparison of Atypicalsin First Episode schizophrenia), which included manytraditional neuropsychological measures, found that a

single, generalized factor characterized both baselineneuropsychological performance and change in perfor-mance after treatment.63 In a similar factor analysis ofneuropsychological change scores, Harvey et al64 identi-fied 2 broad dimensions of neuropsychological change(‘‘verbal/learning’’ and ‘‘attention’’). The points raisedin this article leave ample room for debate. However,in combination, they raise the possibility that morethan half of the reliable overall variance in a typicalbattery of traditional neuropsychological measures, per-haps substantially more, is common variance (see alsoDickinson et al65). This hypothesis would help to explainwhy, at least from a psychometric standpoint, clear inde-pendent signals have been rare in schizophrenia clinicaltrials that have used batteries of traditional neuropsycho-logical tasks to measure cognitive outcome.

In conclusion, current evidence suggests that cognitivedimensions assayed by typical neuropsychological testbatteries, and the underlying measures, share substantialcommon variance. The lack of a widely accepted cogni-tive test battery was a clear obstacle to progress in testingand securing FDA approval for cognition-enhancingagents for schizophrenia.6 The MATRICS cognitive bat-tery, developed with the broad support of the schizophre-nia research community, provided a starting point inaddressing this obstacle. However, while such traditionalneuropsychological test batteries are well suited to doc-ument broad changes in overall cognitive performance,they remain somewhat dubious tools for unravelingthe complex neurobiology of schizophrenia and are un-likely to allow precise testing of novel pharmacologicalagents targeting specific brain mechanisms.5,9

The concerns raised here are well known. Indeed, theMATRICS emphasized the field’s need for new measure-ment technologies more tightly linked with discrete neu-ral systems in its recommendations to the FDA.66 TheCNTRICS project (Cognitive Neuroscience TreatmentResearch to Improve Cognition in Schizophrenia), a re-cently-funded extension of the MATRICS, will playa central role in this regard, adapting methods and con-ceptual frameworks developed in the basic cognitive neu-roscience literature for use in clinical research. Ideally,these more molecular cognitive measures will comple-ment the molar clinical measurement approaches adop-ted and validated as part of the MATRICS effort.Even assuming such advances, however, confident reso-lution of the debate about the extent of the overlapamong cognitive dimensions in schizophrenia ultimatelywill depend on improved understanding of the neurobi-ological and genetic underpinnings of cognition andpharmacological response in this illness.

References

1. Dickinson D, Ragland JD, Calkins ME, Gold JM, Gur RC.A comparison of cognitive structure in schizophrenia patients

432

D. Dickinson & J. M. Gold

and healthy controls using confirmatory factor analysis.Schizophr Res. 2006;85:20–29.

2. Gladsjo JA, McAdams LA, Palmer BW, Moore DJ, JesteDV, Heaton R. A six-factor model of cognition in schizophre-nia and related psychotic disorders: relationships withclinical symptoms and functional capacity. Schizophr Bull.2004;30:739–754.

3. Egan MF, Goldberg TE, Gscheidle T, et al. Relative risk forcognitive impairments in siblings of patients with schizophre-nia. Biol Psychiatry. 2001;50:98–107.

4. Keefe RS. The contribution of neuropsychology to psychia-try. Am J Psychiatry. 1995;152:6–15.

5. MacDonald AW, 3rd, Carter CS. Cognitive experimental ap-proaches to investigating impaired cognition in schizophrenia:a paradigm shift. J Clin Exp Neuropsychol. 2002;24:873–882.

6. Nuechterlein KH, Green MF. MATRICS Consensus Cogni-tive Battery. Los Angeles, Calif: MATRICS Assessment,Inc; 2006.

7. Nuechterlein KH, Barch DM, Gold JM, Goldberg TE, GreenMF, Heaton RK. Identification of separable cognitive factorsin schizophrenia. Schizophr Res. 2004;72:29–39.

8. Kern RS, Green MF, Nuechterlein KH, Deng BH. NIMH-MATRICS survey on assessment of neurocognition in schizo-phrenia. Schizophr Res. 2004;72:11–19.

9. Carter CS. Applying new approaches from cognitive neuro-science to enhance drug development for the treatment of im-paired cognition in schizophrenia. Schizophr Bull. 2005;31:810–815.

10. Barch DM, Carter CS. Amphetamine improves cognitivefunction in medicated individuals with schizophrenia and inhealthy volunteers. Schizophr Res. 2005;77:43–58.

11. Siegel BV, Jr., Trestman RL, O’Flaithbheartaigh S, et al. D-amphetamine challenge effects on Wisconsin Card Sort Test.Performance in schizotypal personality disorder. SchizophrRes. 1996;20:29–32.

12. Newcomer JW, Craft S, Fucetola R, et al. Glucose-inducedincrease in memory performance in patients with schizophre-nia. Schizophr Bull. 1999;25:321–335.

13. Stone WS, Seidman LJ, Wojcik JD, Green AI. Glucose effectson cognition in schizophrenia. Schizophr Res. 2003;62:93–103.

14. Barr RS, Culhane MA, Jubelt LE, et al. The effects of trans-dermal nicotine on cognition in nonsmokers with schizophre-nia and nonpsychiatric controls. Neuropsychopharmacology,advance online publication, April 18, 2007. doi: 10.1038/sj.npp.1301423.

15. Harris JG, Kongs S, Allensworth D, et al. Effects of nicotineon cognitive deficits in schizophrenia. Neuropsychopharma-cology. 2004;29:1378–1385.

16. Smith RC, Warner-Cohen J, Matute M, et al. Effects ofnicotine nasal spray on cognitive function in schizophrenia.Neuropsychopharmacology. 2006;31:637–643.

17. Harvey PD, Green MF, McGurk SR, Meltzer HY. Changesin cognitive functioning with risperidone and olanzapinetreatment: a large-scale, double-blind, randomized study.Psychopharmacology (Berl). 2003;169:404–411.

18. Keefe RS, Seidman LJ, Christensen BK, et al. Long-termneurocognitive effects of olanzapine or low-dose haloperidolin first-episode psychosis. Biol Psychiatry. 2006;59:97–105.

19. Purdon SE, Jones BD, Stip E, et al. Neuropsychologicalchange in early phase schizophrenia during 12 months oftreatment with olanzapine, risperidone, or haloperidol. TheCanadian Collaborative Group for research in schizophrenia.Arch Gen Psychiatry. 2000;57:249–258.

20. Velligan DI, Prihoda TJ, Sui D, Ritch JL, Maples N, Miller AL.The effectiveness of quetiapine versus conventional antipsy-chotics in improving cognitive and functional outcomes in stan-dard treatment settings. J Clin Psychiatry. 2003;64:524–531.

21. Weickert TW, Goldberg TE, Marenco S, Bigelow LB, EganMF, Weinberger DR. Comparison of cognitive performancesduring a placebo period and an atypical antipsychotic treat-ment period in schizophrenia: critical examination of con-founds. Neuropsychopharmacology. 2003;28:1491–1500.

22. Buchanan RW, Conley RR, Dickinson D, Nelson MW, GoldJM, McMahon RP. Galantamine for the treatment of cogni-tive impairments in patients with schizophrenia [abstract].Schizophr Bull. 2007;33:424.

23. Turner DC, Clark L, Pomarol-Clotet E, McKenna P,Robbins TW, Sahakian BJ. Modafinil improves cognitionand attentional set shifting in patients with chronic schizo-phrenia. Neuropsychopharmacology. 2004;29:1363–1373.

24. Olincy A, Harris JG, Johnson LL, et al. Proof-of-concept tri-al of an alpha7 nicotinic agonist in schizophrenia. Arch GenPsychiatry. 2006;63:630–638.

25. Costello AB, Osborne JW. Best practices in exploratory fac-tor analysis: four recommendations for getting the mostfrom your analysis. Pract Assess Res Eval. 2005;10:1–9.

26. Nunnally JC. Psychometric Theory. 2nd ed. New York, NY:McGraw-Hill; 1978.

27. Green MF, Marder SR, Glynn SM, et al. The neurocognitiveeffects of low-dose haloperidol: a two-year comparison withrisperidone. Biol Psychiatry. 2002;51:972–978.

28. Dickinson D, Iannone VN, Gold JM. Factor structure of theWechsler Adult Intelligence Scale-III in schizophrenia. As-sessment. 2002;9:171–180.

29. Lezak MD. Neuropsychological Assessment. 3rd ed. NewYork, NY: Oxford University Press; 1995.

30. Allen DN, Huegel SG, Seaton BE, Goldstein G, Gurklis JA,Jr., van Kammen DP. Confirmatory factor analysis of theWAIS-R in patients with schizophrenia. Schizophr Res.1998;34:87–94.

31. Chen EY, Lam LC, Chen RY, Nguyen DG, Chan CK,Wilkins AJ. Neuropsychological correlates of sustained atten-tion in schizophrenia. Schizophr Res. 1997;24:299–310.

32. Gold JM, Carpenter C, Randolph C, Goldberg TE,Weinberger DR. Auditory working memory and WisconsinCard Sorting Test performance in schizophrenia. Arch GenPsychiatry. 1997;54:159–165.

33. Holthausen EA, Wiersma D, Sitskoorn MM, DingemansPM, Schene AH, van den Bosch RJ. Long-term memory def-icits in schizophrenia: primary or secondary dysfunction?Neuropsychology. 2003;17:539–547.

34. Keefe RS, Goldberg TE, Harvey PD, Gold JM, Poe MP,Coughenour L. The Brief Assessment of Cognition in Schizo-phrenia: reliability, sensitivity, and comparison with a stan-dard neurocognitive battery. Schizophr Res. 2004;68:283–297.

35. Kurtz MM, Wexler BE, Bell MD. The Penn Conditional Ex-clusion Test (PCET): relationship to the Wisconsin CardSorting Test and work function in patients with schizophre-nia. Schizophr Res. 2004;68:95–102.

36. Mahurin RK, Velligan DI, Miller AL. Executive-frontal lobecognitive dysfunction in schizophrenia: a symptom subtypeanalysis. Psychiatry Res. 1998;79:139–149.

37. Morrens M, Hulstijn W, Van Hecke J, Peuskens J, Sabbe BG.Sensorimotor and cognitive slowing in schizophrenia as mea-sured by the Symbol Digit Substitution Test. J Psychiatr Res.2006;40:200–206.

433

Schizophrenia Neuropsychological Dimensions

38. Silver H, Feldman P, Bilker W, Gur RC. Working memorydeficit as a core neuropsychological dysfunction in schizo-phrenia. Am J Psychiatry. 2003;160:1809–1816.

39. Stratta P, Daneluzzo E, Prosperini P, Bustini M, Mattei P,Rossi A. Is Wisconsin Card Sorting Test performance relatedto ‘working memory’ capacity? Schizophr Res. 1997;27:11–19.

40. Strauss ME, Summerfelt A. The neuropsychological study ofschizophrenia: a methodological perspective. In: LenzenwegerMF, Hooley JM, editors. Principles of Experimental Psychol-ogy: Essays in Honor of Brendan A. Maher. Washington, DC:American Psychological Association; 2003:119–134.

41. Binder J, Albus M, Hubmann W, et al. Neuropsychologicalimpairment and psychopathology in first-episode schizo-phrenic patients related to the early course of illness. EurArch Psychiatry Clin Neurosci. 1998;248:70–77.

42. Gur RC, Ragland JD, Moberg PJ, et al. Computerized neuro-cognitive scanning: II. The profile of schizophrenia. Neuro-psychopharmacology. 2001;25:777–788.

43. Hill SK, Schuepbach D, Herbener ES, Keshavan MS,Sweeney JA. Pretreatment and longitudinal studies of neuro-psychological deficits in antipsychotic-naive patients withschizophrenia. Schizophr Res. 2004;68:49–63.

44. Velligan DI, Bow-Thomas CC, Mahurin RK, Miller AL,Halgunseth LC. Do specific neurocognitive deficits predictspecific domains of community function in schizophrenia?J Nerv Ment Dis. 2000;188:518–524.

45. Villalta-Gil V, Vilaplana M, Ochoa S, et al. Neurocognitiveperformance and negative symptoms: are they equal inexplaining disability in schizophrenia outpatients? SchizophrRes. 2006;87:246–253.

46. Friis S, Sundet K, Rund BR, Vaglum P, McGlashan TH.Neurocognitive dimensions characterising patients with first-episode psychosis. Br J Psychiatry Suppl. 2002;43:s85–s90.

47. Keefe RS, Bilder RM, Harvey PD, et al. Baseline neurocog-nitive deficits in the CATIE schizophrenia trial. Neuropsycho-pharmacology. 2006;31:2033–2046.

48. Borenstein M, Hedges LV, Higgins J, Rothstein H. Compre-hensive Meta-analysis. Englewood, NJ: Biostat; 2000.

49. Raudenbush SW. Random effects models. In: Cooper H,Hedges LV, eds. The Handbook of Research Synthesis. NewYork, NY: Sage Publications; 1994:301–321.

50. Shaddish WR, Haddock CK. Combining estimates of effectsize. In: Cooper H, Hedges LV, eds. The Handbook of Re-search Synthesis. New York, NY: Sage Publications;1994:261–285.

51. Cohen JD. Statistical Power for the Behavioral Sciences. Hill-sdale, NJ: Lawrence Earlbaum; 1988.

52. Sergi MJ, Rassovsky Y, Widmark C, et al. Social cognition inschizophrenia: relationships with neurocognition and nega-tive symptoms. Schizophr Res. 2007;90:316–324.

53. Jensen AR. Bias in Mental Testing. New York, NY: FreePress; 1980.

54. Cohen J, Cohen P. Applied Multiple Regression/CorrelationAnalysis for the Behavioral Sciences. Hillsdale, NJ: LawrenceErlbaum; 1983.

55. Pennington BF, Filipek PA, Lefly D, et al. A twin MRI studyof size variations in human brain. J Cogn Neurosci.2000;12:223–232.

56. Schmidt FL, Hunter JE. Measurement error in psychologicalresearch: lessons from 26 research scenarios. Psychol Meth-ods. 1996;1:199–223.

57. Cutler NR, Veroff AE, Frackiewicz EJ, Welke TL, KurtzNM, Sramek JJ. Assessing the neuropsychological profile ofstable schizophrenic outpatients. J Neuropsychiatry Clin Neu-rosci. 1996;8:423–428.

58. Harvey PD, Palmer BW, Heaton RK, Mohamed S, KennedyJ, Brickman A. Stability of cognitive performance in olderpatients with schizophrenia: an 8-week test-retest study. AmJ Psychiatry. 2005;162:110–117.

59. Heaton RK, Gladsjo JA, Palmer BW, Kuck J, Marcotte TD,Jeste DV. Stability and course of neuropsychological deficitsin schizophrenia. Arch Gen Psychiatry. 2001;58:24–32.

60. Velligan DI, DiCocco M, Bow-Thomas CC, et al. A brief cog-nitive assessment for use with schizophrenia patients in com-munity clinics. Schizophr Res. 2004;71:273–283.

61. Wilk CM, Gold JM, Bartko JJ, et al. Test-retest stability ofthe repeatable battery for the assessment of neuropsycholog-ical status in schizophrenia. Am J Psychiatry. 2002;159:838–844.

62. Keefe RS, Seidman LJ, Christensen BK, et al. Comparativeeffect of atypical and conventional antipsychotic drugs onneurocognition in first-episode psychosis: a randomized, dou-ble-blind trial of olanzapine versus low doses of haloperidol.Am J Psychiatry. 2004;161:985–995.

63. Hill SK, Sweeney JA, Hamer RA, et al. Efficiency of theBACS and CATIE neuropsychological batteries in assessingcognition and antipsychotic related change in cognition dur-ing the CAFE clinical trial [abstract]. Schizophr Bull.2007;33:561.

64. Harvey PD, Green MF, Bowie C, Loebel A. The dimensionsof clinical and cognitive change in schizophrenia: evidence forindependence of improvements. Psychopharmacology (Berl).2006;187:356–363.

65. Dickinson D, Iannone VN, Wilk CM, Gold JM. General andspecific cognitive deficits in schizophrenia. Biol Psychiatry.2004;55:826–833.

66. MATRICS. Recommendations to the U.S. Food and DrugAdministration, Division of Neuropharmacological Drug Prod-ucts, for Evaluation of Efficacy of Pharmacological Treatmentsfor Cognition in Schizophrenia. Los Angeles, Calif: Universityof California Los Angeles; 2005.

434

D. Dickinson & J. M. Gold