Cognitive flexibility in autism spectrum disorder:Explaining the inconsistencies?

Lien Van Eylen a,b,c,d,*, Bart Boets b,d, Jean Steyaert b,d,f, Kris Evers b,c,d,Johan Wagemans c,d, Ilse Noens a,d,e

a Parenting and Special Education Research Group, Katholieke Universiteit Leuven (K.U. Leuven), BelgiumbDepartment of Child Psychiatry, UPC-K.U. Leuven, Belgiumc Laboratory of Experimental Psychology, K.U. Leuven, Belgiumd Leuven Autism Research (LAuRes), K.U. Leuven, Belgiume Psychiatric and Neurodevelopmental Genetics Unit, Massachusetts General Hospital, Boston, USAfDepartment of Clinical Genetics, University Hospital Maastricht, The Netherlands

1. Introduction

Autism spectrum disorders (ASDs) are early onset neurodevelopmental disorders characterized by a co-occurrence of

impairments in social reciprocity and communication, combined with restricted and repetitive patterns of interests and

activities (American Psychiatric Association [APA], 2000). These restricted and repetitive behaviors and interests have been

proposed to be associatedwith executive dysfunctions in individuals with ASD (Boyd, McBee, Holtzclaw, Baranek, & Bodfish,

2009; Happe & Ronald, 2008; Hill, 2004; Turner, 1997). Executive functioning (EF) involves goal-oriented planning and

regulation of thoughts and actions (Denckla, 1996). It is an umbrella term for several higher-order cognitive functions (Hill,

2004). Pennington and Ozonoff (1996) have outlined six EF domains: inhibition, working memory, contextual memory,

planning, fluency (or generativity), and cognitive flexibility (or set-shifting). Of all these EF domains, cognitive flexibility has

been most clearly related to repetitive behaviors in ASD (Lopez, Lincoln, Ozonoff, & Lai, 2005).

Research in Autism Spectrum Disorders 5 (2011) 1390–1401

A R T I C L E I N F O

Article history:

Received 23 January 2011

Accepted 25 January 2011

Available online 20 February 2011

Keywords:

Autism spectrum disorders

Cognitive flexibility

Task-switching

Wisconsin Card Sorting Task

Disengagement

Task instructions

A B S T R A C T

The Wisconsin Card Sorting Task (WCST) is the only cognitive flexibility task that has

consistently shown deficits in individuals with an autism spectrum disorder (ASD). As this

is the only task characterized by limited explicit task instructions and a high degree of

disengagement required to perform the switch, we hypothesized that cognitive flexibility

deficits of individuals with ASD might only become apparent in situations fulfilling these

requirements. However, theWCST involves various additional cognitive processes besides

switching, making it uncertain whether difficulties are indeed due to cognitive flexibility

impairments. The aim of this study was to investigate whether individuals with ASD show

cognitive flexibility impairments on amore controlled task-switching variant of theWCST,

still fulfilling both requirements. We therefore developed such a task and administered it

to 40 high-functioning children with ASD and 40 age- and IQ- matched typically

developing controls. As predicted, individuals with ASD made more perseveration errors

and had a significantly higher switch cost than typically developing controls, but they

performed equally well on the control measures.

� 2011 Elsevier Ltd. All rights reserved.

* Corresponding author at: Department of Child Psychiatry, University Hospital of Leuven, Herestraat 49 – Box 7003, B-3000 Leuven, Belgium.

Tel.: +32 (0)16/34 22 45.

E-mail address: Lien.VanEylen@ped.kuleuven.be (L. Van Eylen).

Contents lists available at ScienceDirect

Research in Autism Spectrum Disorders

Journal homepage: ht tp : / /ees .e lsev ier .com/RASD/defaul t .asp

1750-9467/$ – see front matter � 2011 Elsevier Ltd. All rights reserved.

doi:10.1016/j.rasd.2011.01.025

Cognitive flexibility refers to ‘‘the ability to shift to different thoughts or actions depending on situational demands’’

(Geurts, Corbett, & Solomon, 2009, p. 74). Although it has been shown that flexibility deficits in ASD do occur and that they

are related to repetitive behaviors in ASD (South, Ozonoff, &McMahon, 2007; Yerys et al., 2009), there aremany inconsistent

findings. Studies investigating cognitive flexibility in natural settings bymeans of the Behavior Rating Inventory of Executive

Function (BRIEF) have shown that people with ASD have problems with flexibility in daily life (Gioia, Isquith, Kenworthy, &

Barton, 2002; Mackinlay, Charman, & Karmiloff-Smith, 2006). These flexibility deficits have also been related to the

repetitive behaviors typical for ASD (Boyd et al., 2009). However, studies measuring cognitive flexibility in a clinical or

research setting have yielded more inconsistent findings.

Table 1 provides an overview of the most commonly used cognitive flexibility tasks in autism research, indicating the

number of studies showing deficient versus intact performance in ASD. In a recent review, Geurts et al. (2009) make a

distinction between three types of cognitive flexibility tasks: (a) traditional clinical neuropsychological measures, (b) a

hybrid neuropsychological/experimental paradigm (i.e., the intra-dimensional/extra-dimensional shift task [ID/ED] of the

Cambridge Automated Neuropsychological Test and Battery – CANTAB), and (c) experimental task-switching paradigms.

Firstly, the clinical neuropsychological measures are mostly paper-and-pencil tasks, without control stages measuring

possible confounding variables. Concerning these measures, all studies using the Wisconsin Card Sorting Task (WCST) have

consistently reported deficits in individuals with ASD, whereas the majority of studies using other neuropsychological

measures (e.g., the trail making test) reported intact performance. Secondly, the ID/ED shift task is a computerized task that

contains several stages measuring possible confounds. Studies using this task also yielded inconsistent findings, with the

majority of studies showing intact performance in individuals with ASD (as measured by the number of errors on the ED-

shift trials). Thirdly, the experimental task-switching paradigms provide a more controlled measure of cognitive flexibility

by allowing the calculation of a switch cost, that is the difference in response time between maintain trials (the sorting rule

stays the same) and switch trials (the sorting rule changes) (Geurts et al., 2009). So far, only two behavioral studies have

applied such a switch cost paradigm to investigate cognitive flexibility in ASD and both of them reported intact performance

in individuals with ASD.

Taken together, the literature on cognitive flexibility in ASD shows two types of inconsistencies. First, there are

inconsistencies within a certain measure: studies using the same task can yield different findings. These inconsistencies

might be due to differences in participant characteristics like age, IQ, and co-occurring disorders (Geurts et al., 2009; Happe,

Booth, Charlton, & Hughes, 2006; Hill, 2004; Russo et al., 2007). Second, there are inconsistencies betweenmeasures. Briefly

stated, based on measures of everyday behavior and findings from the WCST, there is clear evidence for impairments in

cognitive flexibility in individuals with ASD, whereas studies using other cognitive flexibility tasks generally fail to reveal

these impairments. This apparent discrepancy between the obvious inflexibility in natural settings and the inconsistent

findings on tasks used in clinical or research settings has been referred to as ‘‘the paradox of cognitive flexibility in ASD’’

(Geurts et al., 2009).

In this article, we propose and test an explanation for the inconsistent findings in the ASD literature on theWCST and the

other cognitive flexibility tasks used in clinical or research settings.

One explanation entails that deficits on the WCST are not inherently due to cognitive inflexibility, but result from

problems with other aspects of the task (like the high social demands, high workingmemory and generativity load), that are

controlled for in other cognitive flexibility measures. This might indeed explain why individuals with ASD show deficits on

the WCST and intact performance on more controlled experimental task switching paradigms. However, it does not explain

the intact performance on the other clinical cognitive flexibility tasks that also impose high social demands, and/or high

workingmemory and generativity loads (e.g., themodifiedWCST and the Playing Cards Test from the Behavioral Assessment

of the Dysexecutive Syndrome [BADS]; see Table 1). Furthermore, postulating that individuals with ASD have no cognitive

flexibility deficits clearly contradicts with the obvious inflexibility they show in daily life.

Given this clear daily life inflexibility of individuals with ASD, we suggest that they do have problems with cognitive

flexibility, but that these deficits only become apparent under specific conditions. When closely comparing the WCST with

the other cognitive flexibility tasks, it appears that this task differs from the others in at least one of two factors thatmight be

crucial to elicit the cognitive flexibility deficits in individualswith ASD: (a) the degree of explicitly provided task instructions

and (b) the amount of disengagement required to perform the switch.

Cognitive flexibility tasks can be categorized according to the degree of explicitly provided task instructions by applying a

five level taxonomy (see Table 1). At the lowest level (0), no indications are given about the rules that should be applied, nor

that a rule switch will occur. At level 1, participants are instructed before the task that a rule switch will occur. At level 2,

participants are instructed that a rule switchwill occur and they get an explicit warning during the task indicatingwhen they

have to switch. At level 3, participants are instructed before the task what rules should be applied and when to switch to

another rule. At the highest level (4), a cue is shown on each trial, explicitly indicatingwhich rule should be applied. Based on

this classification, the WCST appears to be the only task with the lowest degree of explicitly provided task instructions. On

the contrary, the experimental task-switching paradigms are situated on the highest degree, and all other cognitive

flexibility tasks somewhere in between. Our concept of ‘degree of explicitly provided task instructions’ is comparable with

that of ‘degree of rule constraints’ (Ciesielski & Harris, 1997) and ‘degree of open-endedness’ (White, Burgess, & Hill, 2009).

Although these other concepts are broader, we could say that higher degrees of explicitly provided task instructions

correspond with higher degrees of rule constraints and lower degrees of open-endedness. Ciesielski and Harris (1997) have

already shown that the lower the degree of rule constraints in flexibility tasks, the higher the impairments for individuals

L. Van Eylen et al. / Research in Autism Spectrum Disorders 5 (2011) 1390–1401 1391

Table 1

Overview of cognitive flexibility tasks used in autism research, partially adapted from Geurts et al. (2009).

Task Description Main dependent

measures

ASD versus

TD (# studies

showing deficient

versus intact

performance)

Degree of

explicitly

provided task

instructions

Disengagement

required?

Wisconsin

Card Sorting

Task (WCST)

A sorting task in which participants have to determine how to sort cards on the

basis of unspecified categories (color, form and number). The sorting rules have

to be inferred based on the given feedback. Without notice, the sorting rule

changes after 10 consecutive correct responses and the participant has to

disengage from the previous sorting rule in order to discover the new rule

- # or % perseverative

answers

- # or % perseverative

errors

- # categories found

Deficits:

- Standard WCST (9:0)a

- Computerized

WCST (3:0)b

0 Yes

Modified WCST Similar to the WCST, but now participants are explicitly warned each time when

the rule changes (after six consecutive correct responses).

# perseverative errors No deficits (0:1)c 2 Yes

Trail Making

Test (TMT)

Timed task consisting of two parts. In part A, a series of numbers has to be

connected in ascending order. In part B, a series of numbers and letters has to be

connected in ascending order while alternating between numbers and letters

- Time B

- Time B� Time A

- Ratio time B/Time A

Inconsistent (2:2)d 3 Yes

D-KEFS TMT This task consists of five conditions that assess visual scanning, number

sequencing, letter sequencing, number–letter switching and motor speed. The

number–letter switching task requires to switch between connecting numbers

and letters in ascending order

Completion time

switch condition

No deficits (0:1)e 3 Yes

BADS Playing

Cards Test

Playing cards are turned over, one at a time. In the first part, children are

instructed to say ‘yes’ to red cards and ‘no’ to black cards. In the second part, the

rule is changed, and children have to indicate whether the card has the same

color as the preceding one

# errors in the

second part

No deficits (0:2)f 3 Yes

D-KEFS

Color-Word

This task consists of four conditions assessing naming of color patches, reading

of color-words printed in black ink, inhibition and inhibition combined with

switching. In the inhibition condition, participants have to perform the

traditional Stroop task, i.e., inhibiting word reading in order to name the

dissonant ink colors in which the words are printed. In the inhibition/switching

condition, half of the color-words are encased in a box. Participants have to

name the dissonant ink color except for the boxed words, in which case they

must switch and read the word itself

- Completion time

switch condition

- # errors switch

condition

Inconsistent (1:1)g 3 Yes

CANTAB ID/

ED Task

On each trial two stimuli are presented and the participant has to learn which

stimulus is correct based on the provided feedback. Nine stages have to be

completed assessing: (1) simple discrimination between two pink shapes; (2)

simple reversal, using the same stimuli but with contingencies reversed; (3)

compound discrimination separate: the same shape remains correct, but now

the stimuli consist of one of two shapes combined with one of two non-

overlapping white lines; (4) compound discrimination superimposed: same as

the previous stage, but now the lines are superimposed on the shapes

(=compound stimuli); (5) compound reversal: using the same stimuli as in 4,

but the other shape is correct; (6) intra-dimensional (ID) shift: new compound

stimuli are presented and one of the new shapes is correct; (7) intra-

dimensional reversal: using the same stimuli as in 6, but now the other shape is

correct; (8) extra-dimensional (ED) shift: new compound stimuli are presented

and now a particular line is correct; (9) extra-dimensional reversal: same

stimuli as in 8, but now the other line becomes correct

- # trials to criterion

at the ED-shift stage

- # errors to criterion

at the ED-shift stage

Inconsistent (1:6)h 1 Reduced

disengagement

L.VanEylen

etal./R

esearch

inAutism

Spectru

mDiso

rders

5(2011)1390–1401

1392

Dimension-Change

Card Sort task

A computerized sorting task in which stimuli have to be sorted by either color or

shape. On each trial a cue is provided, explicitly indicating the sorting rule. The

task consists of six blocks in which the sorting rule is either color or shape, and

one mixed block in which the sorting rule changes

- Difference in # errors

on mixed versus

single rule blocks

- Difference in reaction

time (RT) on mixed

versus single rule blocks

No deficits (0:1)i 4 Yes

Matching Task Stimuli have to be sorted on color or shape. Before starting a new block of trials,

a cue explicitly indicates the sorting rule. A distinction is made between switch

blocks (in which the sorting rule differs from the one in the previous block) and

repetition blocks (in which the sorting rule is the same as in the previous block)

Switch cost: difference

in reaction time and

error rate on the first

trial of a switch versus

repetition block

No deficits (0:1)j 4 Yes

Note: Only behavioral studies comprising at least 15 individuals (IQ> 70) per group were included. The tasks in the white rows are clinical neuropsychological measures, the task in the light grey row is the hybrid

neuropsychological/experimental measure, and the tasks in the dark grey rows are experimental task-switching paradigms.a Geurts, Verte, Oosterlaan, Roeyers, and Sergeant (2004), Goldstein, Johnson, andMinshew (2001), Griebling et al. (2010), Lopez et al. (2005), Minshew, Meyer, and Goldstein (2002), Pellicano, Maybery, Durkin,

and Maley (2006), Pellicano (2007) and Verte, Geurts, Roeyers, Oosterlaan, and Sergeant (2005, 2006).b Robinson, Goddard, Dritschel, Wisley, and Howlin (2009), Tsuchiya, Oki, Yahara, and Fujieda (2005) and Winsler, Abar, Feder, Schunn, and Rubio (2007).c Hill and Bird (2006).d Intact performance: Corbett, Constantine, Hendren, Rocke, and Ozonoff (2009) and Hill & Bird (2006). Deficient performance: Goldstein et al. (2001) and Minshew et al. (2002). Remark: Both studies showing a

deficient performance in ASD only looked at the performance on part B, without controlling for group differences on part A.e Lopez et al. (2005).f Hill and Bird (2006) and White et al. (2009).g Intact performance: Lopez et al. (2005). Deficient performance: Corbett et al. (2009).h Intact performance: Corbett et al. (2009), Goldberg et al. (2005), Happe et al. (2006), Landa and Goldberg (2005), Sinzig, Morsch, Bruning, Schmidt, and Lehmkuhl (2008) and Yerys et al. (2009). Deficient

performance: Ozonoff et al. (2004).i Dichter et al. (2010).j Poljac et al. (2010).

L.VanEylen

etal./R

esearch

inAutism

Spectru

mDiso

rders

5(2011)1390–1401

1393

with ASD compared to typically developing controls. The other factor that may account for the between task inconsistencies

in the ASD literature is the amount of disengagement required to perform the rule switch. Typically, switching to another rule

requires both disengagement from (or inhibition of) the previously correct stimulus–response association and activation of

the required one (Monsell, 2003; Smith, Taylor, Brammer, & Rubia, 2004). However, in experimental settings the amount of

disengagement required to perform a switch can be reduced or even eliminated simply by not showing the previously

correct stimulus. This is what happens in the ED-shift of the ID/ED task (see Table 1 for a description of the task). In this stage

individuals have to learn a new stimulus–response association (i.e., a new rule), but since the previously correct stimulus is

no longer displayed, they do not have to disengage from it. There is evidence that individuals with ASD have problems with

disengagement of attention (Casey, Gordon, Mannheim, & Rumsey, 1993; Hughes & Russell, 1993; Kawakubo et al., 2007;

Landry & Bryson, 2004). Therefore, reducing the amount of disengagement necessary to perform a switch may facilitate the

task for individuals with ASD, hence reducing the differences in performance between individuals with ASD and typically

developing controls.

To summarize, as an explanation for the inconsistent findings between theWCST and the other cognitive flexibility tasks

we hypothesize that individuals with ASD do show impairments in cognitive flexibility, but that these impairments might

only become apparent in situations characterized by the lowest degree of explicitly provided task instructions and a high

amount of disengagement required to perform the switch.

So far, the WCST is the only task fulfilling both requirements. However, this task requires various additional cognitive

processes besides switching,making it uncertainwhether difficulties are due to cognitive flexibility impairments. The aim of

this study is to investigate whether individuals with ASD show cognitive flexibility impairments on a more controlled, but

equally open-ended task-switching variant of the WCST. We therefore developed a task-switching paradigm with the

following characteristics: (a) the task has the lowest degree of explicitly provided task instructions (�WCST); (b) the task

requires a high amount of disengagement to perform the switch (�WCST); but (c) the influence of confounding variables on

task performance is minimized on the one hand by reducing social demands, workingmemory and generativity load, and on

the other hand by providing a within-subject calculation of the switch cost. This version of the WCST is an adaptation of a

task used to investigate brain correlates of cognitive flexibility in typically developing participants (Watson, Azizian, &

Squires, 2006). In the present study, the taskwas administered to a group of high-functioning childrenwith ASD and age- and

IQ-matched typically developing controls. Consistentwith our hypothesis, we predict that childrenwith ASDwill showmore

perseveration errors and a higher switch cost on this task compared to typically developing controls.

2. Methods

2.1. Participants

The autism spectrum disorder (ASD) group comprised 40 children (36 boys and 4 girls). They all received a formal

diagnosis of ASD made by a multidisciplinary team according to DSM-IV-TR criteria (APA, 2000). Twenty-one of them were

recruited through the Flemish Autism Association. The 19 other children participated in a larger family study of the Leuven

Autism Research (LAuRes) consortium. Their diagnosis was additionally confirmed with the Developmental, Dimensional

and Diagnostic Interview (3di; Skuse et al., 2004). Both ASD groups did not differ significantly from one another on IQ, nor on

test performance (perseveration errors and switch cost; data not shown). To increase power they were collapsed into an

omnibus ASD group. None of the ASD children had a known neurological or genetic disorder.

The control group comprised 40 typically developing (TD) children (36 boys and 4 girls), recruited through schools,

personal contacts and advertisements. None of these children presented any neurological or psychiatric disorders or had a

first degree family member with a developmental, learning or neurological disorder. None of these children was on

medication. This information was gathered during a telephone conversation with one of the parents.

We included children and adolescents from 8 to 18 years old, with a verbal, performance, and total IQ score above 70.

Intelligence was assessed with a shortened version of the Wechsler Intelligence Scale for Children, Third Edition (WISC-III;

Wechsler, 1992), or with a shortened version of the Wechsler Adult Intelligence Scale, Third Edition (WAIS-III; Wechsler,

1997) for participants above 17 years old. This shortened version consisted of four subtests: Vocabulary, Similarities, Picture

Completion and Block Design (Sattler, 2001). The participants were group-wise matched on the basis of gender,

chronological age, verbal IQ, performance IQ and full-scale IQ (see Table 2).

Table 2

Characteristics of the participating groups.

Characteristics ASD group (n = 40) TD group (n = 40) t(78) p

Mean SD Mean SD

Age 11.33 2.18 11.13 2.22 0.41 0.69

VIQ 106.68 14.82 109.65 12.14 0.98 0.33

PIQ 104.25 14.77 103.88 14.09 0.12 0.91

FSIQ 105.45 12.34 106.76 9.04 0.54 0.59

L. Van Eylen et al. / Research in Autism Spectrum Disorders 5 (2011) 1390–14011394

2.2. Materials: Wisconsin Card Sorting Task With Controlled Task Switching (WCST-WCTS)

For this task, children sat approximately 57 cm from a 17-in. LCD computer screen on which the stimuli were displayed.

On each trial, three cards arranged in a pyramid form were simultaneously presented on the screen (see Fig. 1). Each card

contains a figure with a specific shape (circle, triangle, heart, cross, star, square, lightning or flower) and color (blue, green,

yellow, red, grey, pink, black or orange). The card at the top is the reference card, and the cards at the bottom are choice cards.

One of the two choice cards has the same shape as the reference card, but has a different color. The other choice card has the

same color as the reference card, but has a different shape. Children had to match the reference card with the correct choice

card, based on either color or shape. Only two sorting rules from the original WCST were implemented to minimize the

workingmemory demands (Watson et al., 2006). On two successive trials the cards had different colors and different shapes.

The sorting stimulus (consisting of the three cards) remained on the screen until participants made a valid response.

Responses were made by pressing the left or right button of a 2-button-response box. Both accuracy and reaction time were

recorded. For half of the trials the correct choice card was the left one and required a left button press, and for the other half

the correct choice card was the right one and required a right button press. After responding, visual and auditory feedback

was given during 600ms (the word ‘‘correct’’ displayed on a green background accompanied by a high tone versus the word

‘‘incorrect’’ on a red background accompanied by a low tone). Stimulus presentation and response registration were

controlled by Affect 4.0 (Spruyt, Clarysse, Vansteenwegen, Baeyens, & Hermans, 2010).

At the beginning of the task participants received the following instructions: ‘‘In this task three cards are shown. One card

appears at the top and two cards appear at the bottom, one left and one right. You have to indicate whether the upper card

matches the left or right card at the bottom, by pressing the left or right response button as soon as possible. The computer

will then indicatewhether your choicewas correct or incorrect. You have to try to correctly sort asmany cards as possible, as

fast as possible. Is that clear?’’. The examiner was not allowed to give any indication that the sorting principles involve color

or shape, or that therewould be a switch from one category to the other. After instructions, a practice blockwas completed to

ensure that the participant understood the instructions. In this practice block cards had to be sorted according to shape and

after eight consecutive correct responses this blockwas completed. After the practice block, the participants were presented

with five runs, each comprising five blocks. In each block a particular sorting rule had to be applied. Randomly after seven,

eight or nine consecutive correct answers the sorting rule changed and a new block began. Accordingly, the sorting rule

changed four times in each run and 20 times throughout the whole task. After each run a short break was provided to avoid

fatigue. Failure to achieve the criterion of 7–9 consecutive correct responses within 50 trials resulted in discontinuation of

the task.

Themain parameters of interest were themean number of perseveration errors and the switch cost. A perseveration error

is defined as the repetition of an incorrect answer. To account for task discontinuation, we controlled for the number of

completed blocks by calculating the mean number of perseveration errors per block. The switch cost is calculated as the

difference between the mean reaction time on switch trials and the mean reaction time on maintain trials. A switch trial is

[()TD$FIG]

Fig. 1. Illustration of the protocol of the Wisconsin Card Sorting Task With Controlled Task Switching (WCST-WCTS).

L. Van Eylen et al. / Research in Autism Spectrum Disorders 5 (2011) 1390–1401 1395

defined as the first correct trial of a block, following an incorrect trial and followed by four consecutive correct answers. A

maintain trial is defined as the fourth correct trial after a switch trial. For each block, except the first block of each run, one

switch and one maintain trial were calculated (see Fig. 1).

Additionally, the number of practice trials required to complete the practice block was registered as it provides a general

indication of the ability to learn a rule from feedback. After the task, participants were also asked which sorting rules they

had to apply, to check whether the correct rules (color and shape) were used.

Finally, a number of other parameters were also derived because they are also measured by other commonly used

flexibility tasks, like the ID/ED task and the WCST: (a) the number of blocks completed; (b) the mean number of trials per

block: themean number of trials necessary to attain the criterion of 7–9 consecutive correct responses; (c) themean number

of failures tomaintain set: the number of times the participantmakes an error after five consecutive correct answers, divided

by the number of blocks completed; and (d) themean number of errors: the sum of themean number of perseveration errors

and the mean number of failures to maintain set.

2.3. Procedure

Informed consentwas obtained from the parents and also from the participants older than 16 years. All participants were

tested individually in a quiet room either at the University Hospital ‘Gasthuisberg’ in Leuven, or at school. Testing took place

within the context of a larger study consisting of two 2-h sessions. The order of the taskswas counterbalanced to avoid order-

effects. In addition, computerized tasks were altered with other task formats, to provide enough variation. Furthermore,

breaks were provided to avoid fatigue. After testing, participants could choose a reward for their participation (e.g., a film

ticket, a comic book, marbles, a ball, etc.).

This studywas approved by theMedical Ethical Committee of the University Hospitals Leuven and the Ethical Committee

of the Faculty of Psychology and Educational Sciences of the Katholieke Universiteit Leuven, Belgium.

2.4. Data analysis

For the reaction time data, all values smaller than 100ms or larger than 5000ms were removed. Afterwards, data were

inspected for outliers on the main dependent variables (mean perseveration errors and switch cost). A participant was

considered an outlier if his/her standard score was �2.5 above or below the mean on at least one of these variables. This

yielded one outlier per group for the mean perseveration errors and three outliers per group for the switch cost. All the

analyses were performed including and excluding these outliers. Both ways of analyzing yielded similar results, although

group differences were more pronounced when the outliers were excluded. Here we only report the analysis on the full

sample as it is more conservative and representative.

The assumptions for parametric testingwere checked and appeared to be violated for all dependentmeasures. To obtain

more normally distributed data, the reaction time measures were log-transformed prior to analysis. For the other

measures, non-parametric Mann–Whitney U tests were applied. A significance level of p< 0.05 (two-sided) was adopted

for all analyses and Cohen’s d effect sizes were calculated for between-subject comparisons. An effect size ranging from0.2

to 0.3 is considered small, values around 0.5 are medium and values of 0.8 or above are considered large effects (Cohen,

1988).

3. Results

The main cognitive flexibility measures were the mean number of perseveration errors per block and the switch cost.

The mean number of perseveration errors was higher for the ASD group than for the TD group (see Fig. 2). In spite of the

considerable effect size (d = 0.53), this difference was only marginally significant (Mann–Whitney U = 1792; p = 0.09). In

addition to the conservativeway of testing this effect, this lack of significance is probably due to the large variance in the ASD

group, which was significantly larger than the variance in the TD group (Levene’s test for homogeneity: F(1,78) = 4.91;

p = 0.03).

To evaluate the switch cost, a 2� 2 (trial type� group) repeated measure analysis of variance (ANOVA) was performed,

with reaction time (RT) as the dependent measure, trial type (switch versus maintain trials) as the within-subject factor and

group as the between-subject factor (see Fig. 3). This analysis revealed a significantmain effect of trial type (F(1,78) = 362.33,

p< 0.001, with slower reaction times on switch than onmaintain trials), nomain effect of group (F(1,78) = 2.86, p = 0.09), and

a significant trial type� group interaction (F(1,78) = 4.90, p = 0.03). Follow-up contrasts revealed no significant group

difference on themaintain trials (t(78) = 1.27, p = 0.21, d = 0.12), but a highly significant group difference on the switch trials

(t(78) = 4.40, p< 0.001, d = 0.52). In addition to this repeated measure analysis, the switch cost was calculated for each

individual (i.e., the mean reaction time on switch trials minus the mean reaction time on maintain trials) and both groups

were compared on this measure (see Fig. 4). A t-test revealed that the ASD group presented a significantly higher switch cost

than the TD group (t(78) = 2.64, p = 0.01, d = 0.60).

Both groups were also compared on the additional measures but no significant group differences were found (see Table 3).

Furthermore, after the task all participants confirmed that they understood the task requirements, i.e., that they had to

switch between sorting according to color and sorting according to shape.

L. Van Eylen et al. / Research in Autism Spectrum Disorders 5 (2011) 1390–14011396

4. Discussion

The present study investigated the performance of children with ASD and typically developing controls on an

experimental task-switching variant of the WCST, which controls for confounding variables, requires a high amount of

disengagement to perform the switch, and has the lowest degree of explicitly provided task instructions. (According to our

taxonomy outlined in Section 1 this means that there are no indications that a switch will occur, nor about the rules that

should be applied.) Consistent with the task-switching literature, all participants responded slower on switch than on

maintain trials, showing that this task induced a switch cost (Monsell, 2003). When comparing the performance of the ASD

and the TD groups, the data revealed that children with ASD tended to make more perseveration errors and had a higher

[()TD$FIG]

0

0.5

1

1.5

2

2.5

3

TDASD

Me

an

pe

rse

ve

ratio

n

err

or

Fig. 2.Meannumber of perseveration errors per block for childrenwith an autism spectrumdisorder (ASD) and typically developing controls (TD). Error bars

depict 1 SE of the mean.[()TD$FIG]

0

500

1000

1500

2000

ASD TD

Mean R

T

Maintain

Switch

Fig. 3. Mean reaction time as a function of group (ASD versus TD) and trial type (switch versus maintain trials). Error bars depict 1 SE of the mean.[()TD$FIG]

0

200

400

600

800

ASD TD

Me

an

sw

itch

co

st

Fig. 4. Mean switch cost of the ASD and the TD group. Error bars depict 1 SE of the mean.

Table 3

Group comparison for the additional measures.

Measure ASD group (n = 40)

Mean (SD)

TD group (n = 40)

Mean (SD)

Mann–Whitney U p (two-sided) Cohen’s d

Number of practice trials 12.58 (8.37) 10.35 (3.83) 1773.5 0.12 0.34

Number of blocks completed 19.33 (8.98) 21.83 (6.99) 1521 0.22 0.31

Trials per block 17.48 (6.79) 15.15 (4.63) 1743 0.24 0.40

Mean maintain failures per block 0.31 (0.23) 0.36 (0.32) 1561.5 0.58 0.22

Total errors 2.26 (2.73) 1.23 (1.33) 1756.5 0.18 0.47

L. Van Eylen et al. / Research in Autism Spectrum Disorders 5 (2011) 1390–1401 1397

switch cost compared to typically developing controls. This is consistent with our predictions and indicates cognitive

flexibility impairments in children with ASD. In addition, there was no group difference in the number of practice trials

needed to complete the practice block, implying that both groups understood the task instructions and were equally able to

learn a rule from feedback. Furthermore, when asked which sorting rules they had to apply, all participants correctly

indicated that they had to switch between sorting according to color and sorting according to shape, showing that they used

the correct sorting rules. Individuals with ASD neither had problems to maintain a certain rule. This is evidenced by

comparablemean numbers of failures tomaintain set between both groups. In addition, the reaction time data revealed that

both groups responded equally fast on maintain trials. However, children with ASD responded significantly slower on the

switch trials. All these findings indicate that individuals with ASD had specific difficulties with switching, but not with other

aspects of the task.

Although themean number of perseveration errors was considerably higher for the ASD group compared to the TD group

(and the effect size was fairly large), this difference was not significant. A closer inspection of the results indicates that this

might be due to the large variance in the ASD group, which was significantly larger than that of the TD group. This group

difference in variance might be due to floor effects in the control group or to the larger heterogeneity in the ASD group.

Some cognitive flexibility tasks, like the ID/ED task and theWCST, use other indicators of cognitive flexibility: the number

of blocks completed, the number of trials per block and the number of total errors.When comparing the performance of both

groups on these measures, no group differences were found. Since group differences were found on switch cost measures,

this might indicate that the other measures are not sensitive enough to detect cognitive flexibility deficits. For example, the

number of total errors is the sum of the perseveration errors and the failures to maintain set. Our data indicate that the ASD

groupmademore perseveration errors, but slightly less failures tomaintain set compared to the TD group. So in this case, the

group difference for the number of total errors is smaller, and thus less sensitive to detect differential performance, than the

number of perseveration errors.

In this study, we administered a task-switching paradigmwith the lowest degree of explicitly provided task instructions

and found flexibility impairments in childrenwith ASD. The only other cognitive flexibility task with this degree of explicitly

provided task instructions is theWCST, which also reveals flexibility deficits in individuals with ASD (see Table 1). However,

all other task-switching paradigms currently used in ASD research have high degrees of explicitly provided task instructions

and do not reveal cognitive flexibility impairments in individuals with ASD. This corroborates our hypothesis that the degree

of explicitly provided task instructions might be crucial to elucidate cognitive flexibility impairments in ASD. In line with

this, it has been demonstrated that tasks with low degrees of ‘rule constraints’ (Ciesielski & Harris, 1997) or more open-

ended tasks (White et al., 2009) provoke the most severe impairments in individuals with ASD.

The degree of explicitly provided task instructions can also be related to different degrees of internally versus externally

controlled shifting. Based on a factor analysis, it has been shown that two separate groups of cognitive flexibility tasks can be

discerned: tasks requiring self-directed or internal control over shifting and tasks reflecting external control over shifting

(Teunisse, Cools, van Spaendonck, Aerts, & Berger, 2001). Tasks with low degrees of explicitly provided rule constraints rely

more on internally controlled processing because of the implicit shifting rules and categories, whereas the explicitly

presented shifting rules elicit more externally controlled shifting behavior (Teunisse et al., 2001). In linewith this, Gioia et al.

(2002) mention that in tests tapping executive functions in an explicit way (thus with high degrees of explicitly provided

task instructions) the examiner provides the necessary structure and organization, thereby serving as the participant’s

external executive control and relieving the demands on executive functions (including cognitive flexibility). They further

state that, as a result, an individual with significant executive dysfunctions can often perform adequately onwell-structured

tests (Gioia et al., 2002). This is evinced by a lesion study demonstrating that performance on theWCST of patients with focal

frontal lesions improvedwhen theywere informed about the sorting rules and received awarning during the task indicating

that the rule would change on the following trial (without mentioning the actual sorting criterion) (Stuss et al., 2000).

Accordingly Stuss et al. (2000) demonstrated that explicitly providing structured verbal instructions makes a test less

sensitive to frontal lesions. Based on these findings, it is plausible that individuals with ASD do present cognitive flexibility

impairments, but that they are able to compensate for their deficits on tasks with high degrees of explicitly provided task

instructions. Support for this interpretation is provided by an fMRI study comparing performance of individuals with ASD

and TD controls on a switch task with a cue presented before each trial, explicitly indicating which rule should be applied

(Schmitz et al., 2006). Both groups had similar behavioral results, but individuals with ASD showed increased parietal lob

activation. According to Schmitz et al. (2006, p. 14), this ‘‘might reflect a compensatory mechanism for dysfunctional frontal

brain regions’’. The idea that offering structure might reduce cognitive flexibility deficits has important implications as it

underscores the usefulness of intervention strategies for ASD that emphasize the explicit provision of structure and explicit

step-by-step instructions (Klin & Volkmar, 1995).

Cognitive flexibility deficits of individuals with ASD on more open-ended tasks might be due to difficulties with internal,

more abstract regulation of performance or problems with understanding the implicit task demands (Larson et al., 2010;

White et al., 2009). These deficits can also be linked with performancemonitoring difficulties. Performancemonitoring (also

referred to as response monitoring) involves the process of evaluating the consequences of behavior and making

adjustments to optimize outcomes (Thakkar et al., 2008). This appears to be problematic for individuals with ASD (South,

Larson, Krauskopf, & Clawson, 2010; Thakkar et al., 2008). Difficulties with performance monitoring in ASD have been

proposed to result from structural and functional abnormalities in the anterior cingulate cortex (ACC), which were also

related to higher ratings of repetitive behavior (Thakkar et al., 2008). Given that individuals with ASD perform adequately on

L. Van Eylen et al. / Research in Autism Spectrum Disorders 5 (2011) 1390–14011398

explicitly cued switching tasks but fail on themore implicit switching tasks, where they have to interpret their own errors as

a cue for initiating the switch, we hypothesize that a core aspect of the cognitive flexibility impairments in ASDmight rely on

deficient monitoring, interpreting and adjusting their own behavior. Further research is needed to investigate the links

between cognitive flexibility deficits, problems with performance monitoring, and repetitive behavior in individuals with

ASD.

Concerning the issue of disengagement required to perform a switch, there is evidence that individuals with ASD have

problems with attentional disengagement (Casey et al., 1993; Hughes & Russell, 1993; Kawakubo et al., 2007; Landry &

Bryson, 2004). However our study does not allow for disentangling the relative importance of disengagement versus explicit

task instructions for revealing cognitive flexibility deficits in ASD. It is also unclear whether tasks need to have both the

lowest degree of explicitly provided rule constraints as well as high amounts of disengagement in order to elicit a cognitive

flexibility deficit in ASD. This study only demonstrates that childrenwith ASDwith IQ scores above 70, are impaired on a task

switching variant of the WCST with the lowest degree of explicitly provided task instructions and high amounts of

disengagement required to perform the switch. Although there is additional evidence supporting our hypothesis that both

the degree of explicitly provided task instructions and the amount of disengagement might be crucial to elucidate cognitive

flexibility impairments in ASD (see Section 1), further research is needed to establish this more firmly. This can be done, for

example, by directly comparing performance on tasks with low versus high degrees of explicitly provided task instructions

and low versus high degrees of disengagement required to perform the switch in a 2� 2 factorial within-subjects design. An

additional question that could be addressed in further research is whether our findings can be generalized to individuals

with ASD with a different age and/or IQ range. An important remark in this regard is that individuals with ASD are

characterized by a large heterogeneity both in the ASD phenotype (Wing, 1997) and in neurocognitive characteristics.

Therefore, it might be possible that several more homogeneous subgroups exist, each with a specific cognitive flexibility

profile.

A last remark concerns the ecological validity of the cognitive flexibility measures used in experimental studies. In daily

life, flexibly switching to different thoughts or actions is mostly triggered implicitly by events indicating that alternative

thoughts or actions may be more appropriate given the situation. In addition, in order to effectively perform a switch the

individual mostly has to disengage attention from the current thoughts or actions. Although all cognitive flexibility tasks

greatly reduce the characteristics of daily life situations, we do suggest that more open-ended and implicit tasks (compared

to highly constrained and explicit tasks), as well as tasks requiring high amounts of disengagement, more closely resemble

everyday situations and are therefore more ecologically valid. This is supported by the finding that measures of everyday

behavior (e.g., the BRIEF, Gioia et al., 2002; Mackinlay et al., 2006) as well as the most open-ended and implicit flexibility

tasks that require high amounts of disengagement (the WCST and our task) reveal cognitive flexibility impairments in ASD,

while all other cognitive flexibility tasks fail to consistently show these deficits (see also Table 1). Additional research might

further investigate the ecological validity of these tasks by calculating the correlation between task performance and the

cognitive flexibility score of the BRIEF.

In summary, the present study investigated the performance of childrenwith ASD and typically developing controls on an

experimental task-switching variant of the WCST, which controls for confounding variables, requires a high amount of

disengagement to perform the switch, and has the lowest degree of explicitly provided task instructions. As predicted,

individuals with ASD made more perseveration errors and had a higher switch cost than typically developing controls, but

they performed equally well on the control measures. These findings, along with other observations, indicate that

individuals with ASD do have cognitive flexibility impairments, but that these impairments might only be revealed under

conditions with a low degree of explicitly provided task instructions and a high amount of disengagement required to

perform the switch. However, further research is needed to fully establish this claim.

Acknowledgements

This study was funded by a doctoral and postdoctoral fellowship from the Fund for Scientific Research (FWO Flanders) to

Lien Van Eylen and Bart Boets, respectively, a grant from the Research Council of the K.U. Leuven (IDO/08/013) to Jean

Steyaert, Johan Wagemans, and Ilse Noens, and from the Methusalem program (METH/08/02) to Johan Wagemans. The

authors thank all children who participated, Jeroen Clarysse for his help with programming the task in Affect 4.0, and the

following master students from the faculty of Psychology and Educational Sciences for their assistance in data collection:

Jolien Hoskens, Carola Damiaans, Ioanna Kitsinis, Mieke Boes, Eline De Proft, and Chiara Boodts.

References

American Psychiatric Association. (2000). Diagnostic and statistical manual of mental disorders (4th ed.). Washington, DC: Author.Boyd, B. A., McBee, M., Holtzclaw, T., Baranek, G. T., & Bodfish, J. W. (2009). Relationships among repetitive behaviors, sensory features, and executive functions in

high functioning autism. Research in Autism Spectrum Disorders, 3, 959–966.Casey, B. J., Gordon, C. T., Mannheim, G. B., & Rumsey, J. M. (1993). Dysfunctional attention in autistic savants. Journal of Clinical and Experimental Neuropsychology,

15, 933.Ciesielski, K., & Harris, R. J. (1997). Factors related to performance failure on executive tasks in autism. Child Neuropsychology, 3, 1–12.Cohen, J. (1988). Statistical power analysis for the behavioral sciences (2nd ed.). Hillsdale, NJ: Lawrence Erlbaum Associates.Corbett, B. A., Constantine, L. J., Hendren, R., Rocke, D., & Ozonoff, S. (2009). Examining executive functioning in childrenwith autism spectrum disorder, attention

deficit hyperactivity disorder and typical development. Psychiatry Research, 166, 210–222.

L. Van Eylen et al. / Research in Autism Spectrum Disorders 5 (2011) 1390–1401 1399

Denckla, M. B. (1996). A theory and model of executive function: A neuropsychological perspective. In G. R. Lyon & N. A. Krasnegor (Eds.), Attention, memory, andexecutive function (pp. 263–277). Baltimore, MD: Paul H. Brookes.

Dichter, G. S., Radonovich, K. J., Turner-Brown, L. M., Lam, K. S. L., Holtzclaw, T. N., & Bodfish, J. W. (2010). Performance of children with autism spectrum disorderson the Dimension-Change Card Sort task. Journal of Autism and Developmental Disorders, 40, 448–456.

Geurts, H. M., Verte, S., Oosterlaan, J., Roeyers, H., & Sergeant, J. A. (2004). How specific are executive functioning deficits in attention deficit hyperactivity disorderand autism? Journal of Child Psychology & Psychiatry, 45, 836–854.

Geurts, H. M., Corbett, B., & Solomon, M. (2009). The paradox of cognitive flexibility in autism. Trends in Cognitive Sciences, 13, 74–82.Gioia, G. A., Isquith, P. K., Kenworthy, L., & Barton, R. M. (2002). Profiles of everyday executive function in acquired and development disorders. Child

Neuropsychology, 8, 121–137.Goldberg, M., Mostofsky, S. H., Cutting, L. E., Mahone, E. M., Astor, B. C., Denckla, M. B., et al. (2005). Subtle executive impairment in children with autism and

children with ADHD. Journal of Autism and Developmental Disorders, 35, 279–293.Goldstein, G., Johnson, C. R., & Minshew, N. J. (2001). Attentional processes in autism. Journal of Autism and Developmental Disorders, 31, 433.Griebling, J., Minshew, N., Bodner, K., Libove, R., Bansal, R., Konasale, P., et al. (2010). Dorsolateral prefrontal cortex magnetic resonance imaging measurements

and cognitive performance in autism. Journal of Child Neurology, 25, 856–863.Happe, F., & Ronald, A. (2008). The fractionable autism triad: A review of evidence from behavioural, genetic, cognitive and neural research. Neuropsychology

Review, 18, 287–304.Happe, F., Booth, R., Charlton, R., & Hughes, C. (2006). Executive function deficits in autism spectrum disorders and attention-deficit/hyperactivity disorder:

Examining profiles across domains and ages. Brain and Cognition, 61, 25–39.Hill, E. L. (2004). Evaluating the theory of executive dysfunction in autism. Developmental Review, 24, 189–233.Hill, E. L., & Bird, C. M. (2006). Executive processes in Asperger syndrome: Patterns of performance in a multiple case series. Neuropsychologia, 44, 2822–2835.Hughes, C., & Russell, J. (1993). Autistic children’s difficulty with mental disengagement from an object: Its implications for theories of autism. Developmental

Psychology, 29, 498–510.Kawakubo, Y., Kasai, K., Okazaki, S., Hosokawa-Kakurai, M., Watanabe, K., Kuwabara, H., et al. (2007). Electrophysiological abnormalities of spatial attention in

adults with autism during the gap overlap task. Clinical Neurophysiology, 118, 1464–1471.Klin, A., & Volkmar, F. (1995). Asperger’s syndrome: Guidelines for treatment and intervention. NewHaven, Connecticut: Learning Disabilities Association of America.Landa, R. J., & Goldberg, M. C. (2005). Language, social, and executive functions in high functioning autism: A continuum of performance. Journal of Autism and

Developmental Disorders, 35, 557–573.Landry, R, & Bryson, S. E. (2004). Impaired disengagement of attention in young children with autism. Journal of Child Psychology and Psychiatry, 45, 1115–1122.Larson, M. J., South, M., Krauskopf, E., Clawson, A., & Crowley, M. J. (2010). Feedback and reward processing in high-functioning autism. Psychiatry Research, in

press, doi:10.1016/j.psychres.2010.11.006.Lopez, B. R., Lincoln, A. J., Ozonoff, S., & Lai, Z. (2005). Examining the relationship between executive functions and restricted, repetitive symptoms of autistic

disorder. Journal of Autism and Developmental Disorders, 35, 445–460.Mackinlay, R, Charman, T., & Karmiloff-Smith, A. (2006). High functioning children with autism spectrum disorder: A novel test of multitasking. Brain and

Cognition, 61, 14–24.Minshew, N. J., Meyer, J., & Goldstein, G. (2002). Abstract reasoning in autism: A disassociation between concept formation and concept identification.

Neuropsychology, 16, 327–334.Monsell, S. (2003). Task switching. Trends in Cognitive Sciences, 7, 134–140.Ozonoff, S., Cook, I., Coon, H., Dawson, G., Joseph, R. M., Klin, A., et al. (2004). Performance on Cambridge neuropsychological test automated battery subtests

sensitive to frontal lobe function in people with autistic disorder: Evidence from the collaborative programs of excellence in autism network. Journal of Autismand Developmental Disorders, 34, 139–150.

Pellicano, E. (2007). Links between theory of mind and executive function in young children with autism: Clues to developmental primacy. Developmental

Psychology, 43, 974–990.Pellicano, E., Maybery, M., Durkin, K., & Maley, A. (2006). Multiple cognitive capabilities/deficits in children with an autism spectrum disorder: Weak central

coherence and its relationship to theory of mind and executive control. Development and Psychopathology, 18, 77–98.Pennington, B. F., & Ozonoff, S. (1996). Executive functions and developmental psychopathology. Journal of Child Psychology and Psychiatry and Allied Disciplines, 37,

51–87.Poljac, E., Simon, S., Ringlever, L., Kalcik, D., Groen, W. B., Buitelaar, J. K., et al. (2010). Impaired task switching performance in children with dyslexia but not in

children with autism. The Quarterly Journal of Experimental Psychology, 63, 401.Robinson, S., Goddard, L., Dritschel, B., Wisley, M., & Howlin, P. (2009). Executive functions in children with autism spectrum disorders. Brain and Cognition, 71,

362–368.Russo, N., Flanagan, T., Iarocci, G., Berringer, D., Zelazo, P. D., & Burack, J. A. (2007). Deconstructing executive deficits among persons with autism: Implications for

cognitive neuroscience. Brain and Cognition, 65, 77–86.Sattler, J. M. (2001). Assessment of children: Cognitive applications (4th ed.). San Diego, CA: Jerome M Sattler Publisher Inc.Schmitz, N., Rubia, K., Daly, E., Smith, A., Williams, S., & Murphy, D. G. (2006). Neural correlates of executive function in autistic spectrum disorders. Biological

Psychiatry, 59, 7–16.Sinzig, J., Morsch, D., Bruning, N., Schmidt, M. H., & Lehmkuhl, G. (2008). Inhibition, flexibility, working memory and planning in autism spectrum disorders with

and without comorbid ADHD-symptoms. Child and Adolescent Psychiatry and Mental Health 2 Retrieved from http://www.capmh.com/content/2/1/4.

Skuse, D., Warrington, R., Bishop, D., Chowdhury, U., Lau, J., Mandy, W., et al. (2004). The developmental, dimensional and diagnostic interview (3di): A novelcomputerized assessment for autism spectrum disorders. Journal of the American Academy of Child and Adolescent Psychiatry, 43, 548–558.

Smith, A. B, Taylor, E., Brammer, M., & Rubia, K. (2004). Neural correlates of switching set as measured in fast, event-related functional magnetic resonanceimaging. Human Brain Mapping, 21, 247–256.

South, M., Ozonoff, S., & McMahon, W. M. (2007). The relationship between executive functioning, central coherence, and repetitive behaviors in the high-functioning autism spectrum. Autism, 11, 437–451.

South, M., Larson, M. J., Krauskopf, E., & Clawson, A. (2010). Error processing in high-functioning autism spectrum disorders. Biological Psychology, 85, 242–251.Spruyt, A., Clarysse, J., Vansteenwegen, D., Baeyens, F., & Hermans, D. (2010). Affect 4.0. Experimental Psychology, 57, 36–45.Stuss, D. T., Levine, B., Alexander, M. P., Hong, J., Palumbo, C., Hamer, L., et al. (2000). Wisconsin Card Sorting Test performance in patients with focal frontal and

posterior brain damage: Effects of lesion location and test structure on separable cognitive processes. Neuropsychologia, 38, 388–402.Teunisse, J., Cools, A. R., van Spaendonck, K. P., Aerts, F. H., & Berger, H. J. (2001). Cognitive styles in high-functioning adolescents with autistic disorder. Journal of

Autism and Developmental Disorders, 31, 55–66.Thakkar, K. N., Polli, F. E., Joseph, R. M., Tuch, D. S., Hadjikhani, N., Barton, J. J., et al. (2008). Response monitoring, repetitive behaviour and anterior cingulate

abnormalities in autism spectrum disorders (ASD). Brain, 131, 2464–2478.Tsuchiya, E., Oki, J., Yahara, N., & Fujieda, K. (2005). Computerized version of theWisconsin card sorting test in children with high-functioning autistic disorder or

attention-deficit/hyperactivity disorder. Brain and Development, 27, 233–236.Turner, M. (1997). Towards an executive dysfunction account of repetitive behavior in autism. In J. Russell (Ed.), Autism as an executive disorder. New York, US:

Oxford University Press.Verte, S., Geurts, H. M., Roeyers, H., Oosterlaan, J., & Sergeant, J. A. (2005). Executive functioning in children with autism and Tourette syndrome. Development and

Psychopathology, 17, 415–445.Verte, S., Geurts, H.M., Roeyers, H., Oosterlaan, J., & Sergeant, J. A. (2006). Executive functioning in childrenwith an autism spectrumdisorder: Canwe differentiate

within the spectrum? Journal of Autism and Developmental Disorders, 36, 351–372.

L. Van Eylen et al. / Research in Autism Spectrum Disorders 5 (2011) 1390–14011400

Watson, T. D., Azizian, A., & Squires, N. K. (2006). Event-related potential correlates of extradimensional and intradimensional set-shifts in a modified WisconsinCard Sorting Test. Brain Research, 1092, 138–151.

Wechsler, D. (1992). Wechsler intelligence scale for children (3rd ed.). London: The Psychological Corporation.Wechsler, D. (1997). Wechsler adult intelligence scale (3rd ed.). San Antonia, TX: The Psychological Corporation.White, S. J., Burgess, P. W., & Hill, E. L. (2009). Impairments on open-ended executive function tests in autism. Autism Research, 2, 138–147.Wing, L. (1997). The autism spectrum. The Lancet, 350, 1761–1766.Winsler, A., Abar, B., Feder, M., Schunn, C., & Rubio, D. (2007). Private speech and executive functioning among high-functioning children with autistic spectrum

disorders. Journal of Autism and Developmental Disorders, 37, 1617–1635.Yerys, B. E.,Wallace, G. L., Harrison, B., Celano,M. J., Giedd, J. N., & Kenworthy, L. E. (2009). Set-shifting in childrenwith autism spectrumdisorders. Autism, 13, 523–

538.

L. Van Eylen et al. / Research in Autism Spectrum Disorders 5 (2011) 1390–1401 1401