Mu wave suppression during the perception of meaningless syllables: EEG evidence of motor...

N

ME

1

2

S3

D4

5

a6

7

A8

R9

R10

A11

A12

13

K14

M15

S16

E17

S18

S19

120

21

w22

s23

s24

&25

S26

t27

i28

t29

C30

v31

M32

M33

t34

u35

L36

T37

d38

0d

PR

OO

F

ARTICLE IN PRESSG ModelSY 3299 1–6

Neuropsychologia xxx (2009) xxx–xxx

Contents lists available at ScienceDirect

Neuropsychologia

journa l homepage: www.e lsev ier .com/ locate /neuropsychologia

u wave suppression during the perception of meaningless syllables:EG evidence of motor recruitment

tephen Crawcour, Andrew Bowers, Ashley Harkrider, Tim Saltuklaroglu ∗

epartment of Audiology and Speech Language Pathology, University of Tennessee, 553 S. Stadium Hall, Knoxville, TN 37996-0740, United States

r t i c l e i n f o

rticle history:eceived 11 December 2008eceived in revised form 16 April 2009ccepted 3 May 2009vailable online xxx

eywords:irror neuron

peech perceptionlectroencephalography

a b s t r a c t

Motor involvement in speech perception has been recently studied using a variety of techniques. In thecurrent study, EEG measurements from Cz, C3 and C4 electrodes were used to examine the relative powerof the mu rhythm (i.e., 8–13 Hz) in response to various audio-visual speech and non-speech stimuli, assuppression of these rhythms is considered an index of ‘mirror neuron’ (i.e., motor) activity. Fourteenadult native English speaking females watched and listened to nine audio-video stimuli clips assembledfrom three different auditory stimuli (speech, noise, and pure tone) combined with three different videostimuli (speech, noise, and kaleidoscope—made from scrambling an image from the visual speech). Rel-ative to the noise–noise (baseline condition), all visual speech conditions resulted in significant levels ofsuppression, a finding that is consistent with previous reports of mirror activity to visual speech and mu

D yllables

ensorysuppression to ‘biological’ stimuli. None of the non-speech conditions or conditions in which speech waspresented via audition only resulted in any significant suppression of the mu rhythm in this population.Thus, visual speech perception appears to be more closely associated with motor activity than acousticspeech perception. It is postulated that in this study, the processing demands incurred by the task wereinsufficient for inducing significant mu suppression via acoustic speech only. The findings are discussedin theoretical contexts of speech perception and the mirror system. We suggest that this technique may

-inva

39

40

41

42

43

44

45

46

47

48

49

50

51

52

53

54

OR

RE

CTE

offer a cost-efficient, non

. Introduction

The processes underlying human speech perception have beenidely examined and debated over the last six decades or so. By

ome accounts, speech is perceived as a function of its acoustic con-tituents and their impact on the auditory system (Klatt, 1979; KuhlMiller, 1975; Massaro & Cohen, 1990; Ohala, 1996; Stevens, 1981;

ussman, 1989). Additionally, these acoustic theories generally holdhat speech perception and production are distinct processes. Crit-cisms of these perspectives were propagated by studies showinghe lack of acoustic invariance in similar speech percepts (Liberman,ooper, Shankweiler, & Studdert-Kennedy, 1967), the influence ofisual stimuli on speech percepts (e.g., McGurk effect; McGurk &acDonald, 1976), the phenomenon of categorical perception (e.g.,ann & Liberman, 1983), and the limited temporal resolution of

he auditory system for processing rapidly changing acoustic stim-

UN

C

Please cite this article in press as: Crawcour, S., et al. Mu wave suppressof motor recruitment. Neuropsychologia (2009), doi:10.1016/j.neurops

li (Liberman, 1957; Liberman, Delattre, & Cooper, 1952). As such,iberman and Mattingly (1985) proposed an alternative viewpoint.hey suggested that speech is perceptually coded as a sequence ofynamic ‘gestures’, that are representative of the manner in which

∗ Corresponding author.E-mail address: [email protected] (T. Saltuklaroglu).

55

56

57

028-3932/$ – see front matter © 2009 Published by Elsevier Ltd.oi:10.1016/j.neuropsychologia.2009.05.001

sive technique for measuring motor activity during speech perception.© 2009 Published by Elsevier Ltd.

they are produced and co-articulated in the vocal tract. It is thesegestures that are thought to form the invariants for both perceptionand production, linking the two processes and allowing them tooperate efficiently and effectively, together as one, in a specializedlinguistic manner. As such, under this ‘motor theory’, the dynamicarchitecture of the mechanism employed to produce speech playsan essential role in its perception.

Conceptually, motor theory appeared to answer many questionsregarding the nature of speech, yet one of its downfalls was a rel-ative dearth of physiological evidence for the neural connectivitybetween speech perception and production. The discovery of mir-ror neurons in the ventral premotor cortex (area F5) of the macaquemonkey (Di Pellegrino, Fadiga, Fogassi, Gallese, & Rizzolatti, 1992;Rizzolatti, Fadiga, Gallese, & Fogassi, 1996) provides compellingevidence for motor involvement in sensory processes and there-fore, a central linking of perception and production. Neurons in thismotor region, which is considered to be a homolog of Broca’s areain humans (Rizzolatti & Craighero, 2004), were found to fire bothwhen monkeys performed or observed goal directed actions (e.g.,

ion during the perception of meaningless syllables: EEG evidenceychologia.2009.05.001

grasping). The location and firing patterns of these ‘mirror neurons’ 58

helped support the notion of motor involvement in speech per- 59

ception (Liberman & Whalen, 2000) and theoretical perspectives 60

of human communication evolving from this observation/action 61

matching system. This neural matching system is thought to 62

dx.doi.org/10.1016/j.neuropsychologia.2009.05.001

http://www.sciencedirect.com/science/journal/00283932

http://www.elsevier.com/locate/neuropsychologia

mailto:[email protected]


E

INN

2 sycho

c63

g64

(65

t66

r67

w68

o69

g70

a71

i72

b73

74

s75

t76

b77

i78

t79

e80

s81

M82

n83

I84

l85

v86

a87

a88

r89

t90

(91

i92

c93

f94

295

(96

i97

s98

T99

o100

m101

m102

103

i104

p105

v106

t107

t108

p109

n110

S111

i112

v113

d114

t115

y116

i117

s118

t119

S120

f121

o122

t123

p124

125

p126

p127

e128

129

130

131

132

133

134

135

136

137

138

139

140

141

142

143

144

145

146

147

148

149

150

151

152

153

154

155

156

157

158

159

160

161

162

163

164

165

166

167

168

169

170

171

172

173

174

175

176

177

178

179

180

181

182

183

184

185

186

187

188

189

NC

OR

RE

CT

ARTICLEG ModelSY 3299 1–6

S. Crawcour et al. / Neurop

reate a biological link between senders and receivers of gesturaloals, which may have served in the evolution of communicationRizzolatti & Arbib, 1998). These notions were further bolstered byhe discovery of a subclass of mirror neurons that fired not only inesponse to seeing an action, but also to hearing sounds associatedith a specific action such as paper ripping (Kohler et al., 2002)

r peanut breaking (Keysers et al., 2003). Hence, the authors sug-ested that mirror neurons code the intended goal of an action inn abstract amodal manner rather than a specific action itself, elic-ting strong parallels to the nature of speech ‘gestures’ as describedy motor theorists.

A growing body of research suggests that in humans, the mirrorystem may be involved in action recognition, imitation, empa-hy and theory of mind. Its role in speech perception also haseen scrutinized using various measures. In the auditory modal-

ty, transcranial magnetic stimulation (TMS) has been used to showhat listening to lingual speech sounds could evoke stronger motorvoked potentials (MEPs) in the tongue relative to non-speechounds (Fadiga, Craighero, Buccino, & Rizzolatti, 2002) and strongerEPs in lip muscles when listening to speech while watching white

oise (Watkins & Paus, 2004; Watkins, Strafella, & Paus, 2003).n addition, using functional magnetic resonance imaging (fMRI),istening to meaningless speech has been found to bilaterally acti-ate portions of the ventral premotor cortex (though not Broca’srea), portions of the motor cortex, and the supplementary motorrea (Wilson, Saygin, Sereno, & Iacoboni, 2004) and motor corticalegions in a somatotopic manner (Pulvermüller et al., 2006) rela-ive to non-speech stimuli. In the visual modality, Nishitani and Hari2002) used magnetoecepahlography (MEG) to reveal that observ-ng still pictures of lips could activate Broca’s area and the motorortex. Similar bilateral motor activation patterns have also beenound to silent speech lip movements using fMRI (Campbell et al.,001) and stilled speech (Calvert & Campbell, 2003). Buccino et al.2004) also used fMRI to discover significantly higher activationsn portions of the left inferior frontal gyrus in response to viewingpeech reading and lip smacking, but not to viewing a dog barking.hey suggested that activation of one’s own motor system via actionbservation occurs when the action in question is part of one’s ownotor repertoire, again suggesting a biological underpinning for theirror system.Speech can be perceived unimodally via either audition or vision

n isolation, or bimodally (audio-visually). Though the studies aboverovide evidence that speech perception through either audition orision can activate the human mirror system, they did not examinehe relative strength of each modality for inducing mirror activa-ion or their relative strengths compared to audio-visual speecherception. Because the mirror system is thought to have close con-ections to the somatosensory (SI) system, Möttönen, Järveläinen,ams, and Hari (2004) used MEG to examine how viewing and hear-ng speech modulated activity in the left SI mouth cortex. Whereasiewing speech induced significant SI modulation, hearing speechid not. Similarly Sundara, Namasivayam, and Chen (2001) foundhat visual and audio-visual presentations of the syllable/ba/bothielded significant increases in MEP amplitudes, whereas the MEPncrease produced from auditory perception alone did not reachignificance. Activation levels of motor areas during speech percep-ion have also been examined using fMRI. Skipper, Nusbaum, andmall (2005) found that audio-visual speech activated the inferiorrontal gyrus and premotor cortex to a greater extent than audior visual speech alone. They also found that the activation level ofhe premotor cortex was modulated by a number of phonemes that
U

articipants could visually identify.It has been suggested that the visually perceived gestures may

lay the stronger role in activating the motor system during speecherception (Skipper, von Wassenhove, Nusbaum, & Small, 2007),specially in conditions in which auditory speech is absent or

D P

RO

OF

PRESSlogia xxx (2009) xxx–xxx

degraded (Callan et al., 2003). In addition, it is not clear how brainregions involved in speech production are differentially activatedduring various perception tasks. For example, it has been sug-gested that motor recruitment might be influenced by the degreeof linguistic processing necessary to a task (Callan, Jones, Callan, &Akahane-Yamada, 2004; Ojanen et al., 2005; Wilson & Iacoboni,2006). Thus, the numerous measures of motor recruitment inspeech perception and the variety of stimuli employed, combinedwith the diversity of findings, make it difficult to reconcile the dis-crepancies within the current body of research and explain theextent to which motor recruitment may be necessary in speechperception. As such, further investigation in this area is warranted.

Electroencephalography (EEG) has been suggested as a promis-ing, cost-efficient and non-invasive means of indirectly examiningthe mirror neuron activity in humans. In particular, measurementsof oscillation amplitudes in the mu frequencies (8–13 Hz) mea-sured across the sensorimotor cortices acquired via surface levelelectrodes are thought to provide a valid index of mirror activ-ity (Altschuler, Vankov, Wang, Ramachandran, & Pineda, 1997).Mu ‘rhythms’ are influenced by both motor activity and atten-tion (see Pineda, 2005 for full review). When a person is at rest,amplitudes of waves in this band are highest because sensorimotorneurons responsible for generating these waves fire synchronously.Conversely, when a person performs an action, the pattern of fir-ing is asynchronized, resulting in suppression of the mu waveand smaller amplitudes. However, a number of studies also havefound that mu waves are suppressed when normal adults observehuman hand movements (Muthukumaraswamy & Johnson, 2004;Muthukumaraswamy, Johnson, & McNair, 2004; Oberman et al.,2005; Virji-Babul et al., 2008) and implied point-light human bio-logical animation (Saygin, Wilson, Hagler, Bates, & Sereno, 2004;Ulloa & Pineda, 2007), and even when participants imagine biolog-ical motion (Pineda, Allison, & Vankov, 2000).

Because mu suppression can occur in these passive observa-tion/imagination conditions in the absence of motor activity, thelevel of suppression is thought to provide an index of mirror neuronactivity. When employing these paradigms, recordings from Cz, C3and C4 are thought to be indirect measures of cortical activity in thesupplementary motor areas and left and right sensorimotor cortices(S1-M1; Babiloni et al., 1999), respectively. Hence, EEG recordingsfrom these electrodes are considered to be measuring “downstreammodulation of sensorimotor areas by mirror neurons” (Oberman etal., 2005, p. 191). As the recordings are made from the scalp, it isdifficult to map the sources of suppression to cortical landmarks.However, Nishitani and Hari (2000), in a study using MEG, foundthat the sources of mirror activity may be further ‘upstream’ inthe primary motor cortex and in the inferior frontal cortex (e.g.,BA 44). As these regions are often activated in speech perceptiontasks, it seems plausible that EEG recordings of mu rhythms at elec-trode sites Cz, C3 and C4 might also be suppressed when speech isperceived.

Though Muthukumaraswamy, Johnson, Gaetz, and Cheyne(2004) examined mu suppression to oro-facial movements (i.e.,teeth-baring, blowing), to our knowledge, EEG has not yet beenused to examine differential levels of mirror neuron activity tothe perception of speech and non-speech stimuli. Previous stud-ies that have identified selective mirror neuron functioning inresponse to observed dynamic biological stimuli have employedvisual noise and ‘non-biological’ conditions as bases for compari-son. Oberman et al. (2005) used a bouncing ball, whereas Ulloa andPineda (2007) used scrambled versions of their point-light biolog-


ical animations. As speech can be conveyed through both auditory 190

and visual channels, in order to differentiate the effects of speech 191

from noise and non-biological stimuli, it seems logical in the cur- 192

rent study to employ all three types of stimulus (noise, speech, 193

and non-biological) conditions for both input modalities. Based 194


D

INN

sycho

u195

p196

p197

a198

t199

s200

e201

o202

t203

s204

2205

2206

207

S208

o209

b210

s211

fi212

i213

n214

i215

t216

U217

2218

219

(220

v221

v222

i223

a224

t225

c226

r227

o228

u229

n230

o231

r232

w233

M234

s235

a236

237

238

239

240

241

242

243

244

245

246

247

248

249

250

251

252

253

254

255

256

257

258

259

260

261

262

263

264

265

266

267

268

269

270

271

272

273

274

275



pon previous findings of motor activation as a function of speecherception, it is hypothesized that conditions in which speech isresented visually will result in the highest levels of mirror neuronctivation and therefore, the most robust mu suppression relativeo when noise is presented through both modalities. When non-peech stimulus conditions are presented, mu suppression is notxpected. These data will also provide some preliminary findingsf how auditory speech may suppress mu waves and contribute tohe body of research investigating how auditory and visual speechignals interact in motor recruitment.

. Methods

.1. Participants

Fourteen right-handed female adults aged 20–51 (mean age = 28.03 years;D = 8.24 years) who were native English speakers and had no diagnosed historyf communicative, cognitive or attentional disorder. A female cohort was chosenecause of the recent finding of stronger mu suppression in response to biologicaltimuli in females than in males (Cheng et al., 2008). As, to our knowledge this is therst study examining mu suppression to speech, it seemed appropriate to first exam-

ne the gender in which the responses might be strongest. In addition, in order toegate any potential ‘processing’ differences that second language acquisition may

ncur, native English speakers were chosen because an English speaker producedhe speech stimuli. Prior to the experiment, informed consent (approved by Theniversity of Tennessee Institutional Review Board) was obtained for all participants.

.2. Stimuli and experimental conditions

The stimuli consisted of audio-visual presentations of (1) meaningless speech,2) noise and (3) a second set of non-biological stimuli. Audio, video and audio-isual speech stimuli were constructed from recordings (using a Sony DCRHC30ideo camera) of the mouth (Fig. 1) of a native English speaking adult male produc-ng a continuous stream of co-articulated meaningless CV syllables (e.g., /da/, /bi/,nd /ga/). A male voice was chosen for the stimuli as males have lower fundamen-al frequencies and their voices are richer in harmonic content. The audio-visuallips were edited using Apple iMovie (version 5.0.2), which allowed for the sepa-ation of the audio and video tracks where necessary. The noise stimuli consistedf auditory and visual white noise. In the same vein that Ulloa and Pineda (2007)sed scrambled point-light stimuli as analogs of their point-light stimuli, the visualon-biological analog of speech movement was constructed by taking a still frame

UN

CO

RR

EC

TE


f the mouth recordings and converting it into a kaleidoscope with five symmet-ical portions of the mouth centered at 72◦ to each other using software from theebsite (http://www.krazydad.com/kaleido). Video recordings were made (Screenovie Recorder 2.6) as undulating kaleidoscopic motion was applied in a random

ymmetrical and concentric manner (Fig. 2). The auditory non-biological analog was1000 Hz pure tone. To help ensure uniform intensity levels, all audio stimuli were

Fig. 1. Three examples of three still frames extracted from the dyna

Fig. 2. Three examples of three still frames extracted from the dynami

PR

OO

F

PRESSlogia xxx (2009) xxx–xxx 3

normalized using Adobe Audition (version 1.5) software. Hence, 90 s audio-visualstimuli clips were constructed for nine different audio-visual conditions: (1) NN:noise (auditory)–noise (visual), (2) NS: noise–speech, (3) NK: noise–kaleidoscope,(4) SN: speech–noise, (5) SS: speech–speech, (6) SK: speech–kaleidoscope, (7) TN:tone–noise, (8) TS: tone–speech, and (9) TK: tone–kaleidoscope. The speech stimulicontained 61–69 syllables. Each clip was recorded onto a DVD using Apple iDVD(version 5.0.1).

2.3. Procedure

The experiment was conducted in an electronically and magnetically shielded,double-walled, sound-treated booth. Participants were seated in a comfortablereclining armchair with their heads and necks well supported. After the electrodeswere placed, they were instructed to sit quietly with their eyes open and attend tothe audio and visual stimuli. The stimuli were played from the DVD with a PhillipsHDMI DVD player (model DVP5892). The visual output was displayed on a 32-in.Phillips LCD monitor (model 32PFL5332D). The audio output was routed througha Mackie DFX-6 mixer and was delivered binaurally to participants’ ears at theirpreferred intensity level using Ear Tone model ER-1-14A insert earphones. In orderto help ensure that participants were paying attention, they were asked to silentlycount the number of times they heard or saw a syllable initiated by the /b/ phoneme.All experimental conditions were presented to participants twice in separate ran-dom sequences (i.e., two blocks). Experimental conditions were separated by 30 sbreaks, during which participants were asked to report the number of /b/ initiatedsyllables they were able to recognize from the previous condition. All participantsreported the hearing or seeing /b/ initiated syllables in the speech conditions and noparticipant reported hearing or seeing speech in any of the non-speech conditions.

2.4. EEG data acquisition and analysis

Thirteen electrodes were used to acquire EEG data based on the international10–20 method of electrode placement (Jasper, 1958) using a 32-channel, unlinked,sintered NeuroScan Quik Cap. Non-inverting electrodes included Cz, C3, C4, Pz, P3,P4, Fz, F3, F4, O1, O2, M1, and M2. The inverting electrode was placed on the nasionand the ground electrode was at Fpz. The electro-oculogram (EOG) was recorded byelectrodes placed on the left superior orbit and the left interior orbit. The impedancesof all electrodes were measured at 30 Hz before, during, and after testing and werenever greater than 10 k�.

EEG was collected and analyzed using Compumedics NeuroScan Scan 4.3.3software and Synamps 2 system. Data were obtained for approximately 180 s percondition (90 s for each of two runs), filtered (0.15–100 Hz), and digitized via a 24-bitanalog-to-digital converter at a sampling rate of 500 Hz. It has been reported thatEEG in the 8–13 Hz frequency band recorded from electrodes in the region of the


occipital cortex are confounded by states of expectancy and awareness (Klimesch, 276

Doppelmayr, Russegger, Pachinger, & Schwaiger, 1998). In addition, there are shared 277

frequencies between the mu rhythm and posterior alpha bands and the activity in 278

the posterior alpha bands may be stronger than the mu rhythms. For this reason, 279

researchers like Oberman et al. (2005) have suggested that EEG obtained from Cz, 280

C3, and C4 might be confounded by posterior activity. However, this factor is present 281

mic visual speech stimuli. The original stimuli were in color.

c visual kaleidoscopic stimuli. The original stimuli were in color.


http://www.krazydad.com/kaleido

INN

4 sycho

a282

w283

f284

r285

w286

t287

i288

w289

a290

e291

292

i293

c294

m295

s296

O297

n298

a299

3300

301

s302

t303

H304

o305

r306

a307

a308

m309

w310

r311

312

p313

e314

e315

m316

a317

c318

Faattn(co

319

320

321

322

323

324

325

326

327

328

329

330

331

332

333

334

335

336

337

338

339

340

341

342

343

344

345

346



cross conditions as they all they involve visual stimuli presentation to participantsith their eyes open and hence does not affect any condition to a greater extent. To

urther combat this, the first and last 10 s of each of the two blocks of 90 s data cor-esponding to one experimental condition was removed. The remaining 70 s of dataere used in the analysis. As data for each condition was collected in two blocks, a

otal 140 s of data were collected per condition. Eye blink and eye movements weredentified in the EOG channel and EEG at all other channels during these intervalsas removed. Then, the integrated power in the 8–13 Hz range was calculated usingFast Fourier Transformation (1024 points) on the data, which were divided into 2 spochs. A cosine window was used to minimize artifacts due to data splicing.

Mu suppression was calculated as the ratio of the power during the eight exper-mental conditions (NS, NT, KS, SS, SN, ST, KN, and KT) relative to the power in the NNondition, which was considered the baseline. Data are presented as ratios to mini-ize individual variability in absolute mu power that may result from differences in

calp thickness or electrode impedance, as opposed to mirror neuron activity (e.g.,berman et al., 2005). A log transformation was applied to each ratio because of theon-normal distribution that ratio data yield. Thus, negative and positive log ratiosre indicative of mu suppression and enhancement, respectively.

. Results

Suppression of mu rhythms relative to baseline were mea-ured from Cz, C3 and C4 as these are the sites coinciding withhe mu wave source and most robust measurements (Pineda &echt, 2009). Because alpha bands were not suppressed in anyther electrode site, we can safely rule out the possibility thatecordings from these sites were influenced by posterior alphactivity. Mu suppression across three electrode sites (Cz, C3, and C4)nd eight conditions were analyzed using a two-factor repeated-easures ANOVA. Greenhouse-Geisser corrections were appliedhere sphericity assumptions were violated and Bonferroni cor-

ections were applied to multiple post hoc comparisons.A significant main effect was found for condition [F(7,91) = 6.367,

= 0.001]. However, no significant main effect was found for the

UN

CO

RR

EC

TE


lectrode site [F(2,26) = 0.105, p = 0.901] or the interaction betweenlectrode site and condition, [F(14,182) = 0.992, p = 0.463]. Hence,u suppression data were collapsed across the three electrodes and

re displayed as function of experimental condition in Fig. 3. Theollapsed data were used in post hoc t-tests that examined how

ig. 3. Mu suppression across experimental condition consisting of combinations ofuditory and visual stimuli. Labels on the X-axis represent the following conditions:udio speech and visual speech (SS), audio noise and visual speech (NS), audio pureone and visual speech (TS), audio speech and visual white noise (SN), audio pureone and visual white noise (TN), audio speech and visual kaleidoscope (SK), audiooise and visual kaleidoscope (NK), and audio pure tone and visual kaleidoscopeTK). Bars represent mean log ratio of mu power relative to baseline (NN condition)ollapsed across Cz, C3 and C4 electrodes. Error bars represent one standard errorf the mean.

347

348

349

350

351

352

353

354

355

356

357

358

359

360

361

362

363

364

365

366

367

368

369

370

371

372

373

374

375

376

D P

RO

OF


each condition suppressed the mu wave relative to the baseline(Oberman et al., 2005; Oberman, Ramachandran, & Pineda, 2008).The conditions in which visual speech were presented were theonly ones to induce significant and robust levels of mu suppres-sion [SS: t(13) = −4.223, p = .008; NS: t(13) = −3.904, p = .016; TS:t(13) = −4.760, p < .001].

4. Discussion

In accord with our first hypothesis, significant suppression ofthe mu wave was found across electrodes Cz, C3 and C4 when par-ticipants visually perceived streams of meaningless co-articulatedspeech gestures (i.e., viewed continuous lip movements). All threevisual speech conditions tested (SS, NS, and TS) produced simi-lar levels of suppression, suggesting that visual speech perceptionrecruited the motor system irrespective of the auditory stim-ulus with which it was paired. These findings are consistentboth with studies showing similar patterns of mu suppression toobserved human movements (Muthukumaraswamy, Johnson, &McNair, 2004; Oberman et al., 2005; Virji-Babul et al., 2008) andwith studies using MEG or fMRI showing bilateral motor activity tovisual speech perception (Calvert & Campbell, 2003; Campbell et al.,2001; Nishitani & Hari, 2002; Skipper et al., 2005). Furthermore, inaccord with our second hypothesis, the non-speech conditions (TN,TK, and NK) failed to produce any significant levels of mu suppres-sion, most likely, because in these conditions, participants did notperceive stimuli containing arrays of movements existing withintheir own motor repertoire (Buccino et al., 2001).

These findings also appear to be consistent with those ofMöttönen et al. (2004) and Sundara et al. (2001) who foundevidence of motor activation in response to visual but not audi-tory speech perception. As such, they support notions that motorrecruitment for speech is often mediated primarily by visual sys-tems (Skipper et al., 2007) and that when visual speech informationis presented, perception of the gesture is supported by an internalsimulation of the observed articulatory movements (Callan et al.,2003). This motoric supplementation which appears to be affordedby the presence of visual speech may contribute to previous find-ings showing that the addition of congruent visual cues to auditoryspeech is associated with increases in speech intelligibility (Sumby& Pollack, 1954), improved speech detection thresholds (Grant &Seitz, 2000), and accelerated neural processing (Van Wassenhove,Grant, & Poeppel, 2005).

Current findings may contrast with those that have found motorinvolvement during auditory-only speech perception tasks (e.g.,Fadiga et al., 2002; Watkins et al., 2003; Watkins & Paus, 2004;Wilson et al., 2004). Skipper et al. (2005) suggest that any briefmotor activation that occurs during auditory speech perceptionmay not be observed in methods that employ temporal averag-ing. Perhaps however, a closer examination of acoustic speechsignals may help foster a better understanding of their inconsis-tent motor activation patterns. Acoustic speech signals are richlyencoded and contain cues indicative of the gesture (e.g., formantstructures, glottal pulsing, onset spectra) from every level of thevocal tract. In fact, the redundancy of information pointing tothe intended gesture allows that the percept be recovered evenwhen considerable portions of the acoustic signal are removed. Anextreme case of this can be found when humans perceive linguis-tic information from sine wave analogs of speech that are devoidof traditional acoustic cues (Remez, Rubin, Pisoni, & Carrell, 1981).


One perspective derived from cognitive psychology that might con- 377

tribute to explaining this substantial human capacity is the view 378

of speech perception as an ‘embodied’ process (Aziz-Zadeh & Ivry, 379

2009; Galantucci, Fowler, & Turvey, 2006; Skipper et al., 2005; 380

Wilson, 2002). From this standpoint, the manner in which speech 381


D

INN

sycho

i382

C383

t384

p385

s386

p387

S388

s389

i390

s391

p392

393

v394

B395

c396

s397

&398

s399

a400

i401

n402

v403

i404

l405

O406

r407

t408

a409

r410

a411

i412

a413

v414

s415

f416

i417

t418

t419

420

n421

t422

i423

C424

m425

i426

m427

a428

b429

t430

i431

p432

t433

g434

f435

d436

f437

a438

w439

N440

r441

m442

s443

a444

b445

446

p447

448

449

450

451

452

453

454

455

456

457

458

459

460

461

462

463

464

465

466

467

468

469

470

471

472

473

Q1 474

475

476

477

478

479

480

481

482

483

484

485

486

487

488

489

490

491

492

493

494

495

496

497

498

499

500

501

502

503

504

505

506

507

508

509

510

511

512

513

NC

OR

RE

CTE



s perceived is experientially shaped through multiple interactions.onnections between the acoustics of speech and the motoric ges-ural configurations responsible for producing those acoustics (i.e.,arity) become strengthened over time. It seems logical that theseensorimotor connections are strengthened most during the earlyeriods of language acquisition and development (Kalinowski &altuklaroglu, 2003). Therefore, it may be plausible that once aufficient phonemic repertoire is acquired, motor involvement dur-ng basic speech perception becomes less essential; yet the motorystem may remain an available referent during more demandinghonemic or linguistic processing tasks.

The relationship between processing demands and motor acti-ation may be further considered when examining the role ofroca’s area in speech perception. Though Broca’s area is classicallyonsidered a key speech production area, it is also considered byome to mediate functioning of the mirror system (Hari, Levanen,Raij, 2000; Nishitani & Hari, 2002) and play a strong role in multi-

ensory processing. During passive speech perception tasks, Broca’srea has been found to be more active in conditions that requirencreased amounts of processing such as when an auditory sig-al is degraded or absent (Callan et al., 2003), when auditory andisual signals are not matched (Ojanen et al., 2005), when listen-ng to non-native phonemes (Wilson & Iacoboni, 2006) and whenistening to sentences paired with incongruent gestures (Willems,zyurek, & Hagoort, 2007). Hence if the suppression of the mu

hythm is influenced by activity in Broca’s area, it may be postulatedhat in the current study, the processing demands incurred by theuditory-only speech conditions were insufficient for necessitatingecruitment of the motor system and therefore, did not result inny significant suppression of the mu rhythm. That is, the mean-ngless acoustic speech signal was produced by an English speakernd was presented clearly and audibly. Its combination with eitherisual noise or the kaleidoscopic stimuli most likely did not tax thepeech processing demands. In addition, the participants were allemale native English speakers, free from speech, language or hear-ng disorders, and recruited from a university setting, suggestinghat they had little difficulty interpreting the phonemic informationhat was presented under these conditions.

Skipper et al. (2005) suggest that rather than being static inature, the mirror system may be a dynamic entity with variablehresholds for engagement across stimuli, both within and betweenndividuals and dependent on the nature and goals of a given task.onsidering this dynamic nature and the possibility that speechechanisms within individuals may be tied to their embodiment

n world experiences, it is not surprising that the original tenets ofotor theory have been recently re-examined (e.g., Galantucci et

l., 2006; Lotto, Hickok, & Holt, 2009). The current paradigm maye appropriate for shedding further light on questions surroundinghe role of motor involvement in speech perception. Future stud-es examining mu suppression to speech may consider increasingrocessing demands, perhaps by adding different types of noise tohe speech signal (e.g., Callan et al., 2003), using non-native lin-uistic forms (e.g., Wilson & Iacoboni, 2006) or employing speechrom disordered populations such as those who stutter or haveysarthric speech. Another application may be found in the dif-erent populations that may be tested, Much remains to be learnedbout the speech processing in disordered populations such thoseith autism, specific language disorders and aphasia. In addition,yström (2008) recently used high density EEG to find similar mir-

or responses in infants and adults in response to goal directedovements and as such, EEG may be appropriate for examining
U

peech perception mechanisms in children and comparing them todults, especially if theories about embodied processing prove toe valid.

To our knowledge, this is the first study to examine mu sup-ression during auditory and visual speech perception. Though we

PR

OO

F

PRESSlogia xxx (2009) xxx–xxx 5

believe the findings to be of importance, certain limitations shouldbe recognized in interpreting the findings. First, though the murhythm is thought to emanate from sensorimotor areas, the lim-ited spatial resolution of surface EEG recordings does not allowaccurate site mapping of the neural activity. As such, future inves-tigations continue to require that motor activity measured by musuppression is indexed as a function of the task and relative to otherconditions under investigation. Second, this study used an exclu-sively female cohort. Though females were chosen because of theirhigher sensitivity in recording mu suppression, these findings needto be replicated in males. Third, though the visual non-biologic ana-log was dynamic (i.e., the kaleidoscope), the non-biological analogto auditory speech (i.e., the pure tone) was not. In future stud-ies, to better understand the differential abilities of speech versusnon-speech acoustic signals to induce mirror activity, using moredynamic non-biological acoustic stimuli as controls may be advan-tageous.

These preliminary findings showed mu suppression, consideredevidence of cortical motor activation, to visually perceived streamsof meaningless syllables, irrespective of acoustic pairing. Thoughsuppression to acoustic only speech stimuli was not found herein,we are cautiously optimistic that it might be found in future stud-ies using this technique. As such, indexing mu suppression acrossaudio-visual tasks appears to be a non-invasive, cost-efficient tech-nique that may continue to provide clues to the many unansweredquestions about the nature of human speech perception.

Uncited reference

Lahav, Saltzman, and Schlaug (2007).

References

Altschuler, E. L., Vankov, A., Wang, V., Ramachandran, V. S., & Pineda, J. A. (1997).Person see, person do cells. Abstracts-Social Neuroscience, 23, 1848.

Aziz-Zadeh, L., & Ivry, R. B. (2009). The human mirror neuron system and embodiedrepresentations. Advances in Medicine and Biology, 629, 355–376.

Babiloni, C., Carducci, F., Cincotti, F., Rossini, P. M., Neuper, C., Pfurtscheller, et al.(1999). Human movement related potentials vs desynchronization of EEG alpharhythm: A high resolution EEG study. NeuroImage, 10, 658–665.

Buccino, G., Binkofski, F., Fink, G. R., Fadiga, L., Fogassi, L., Gallese, et al. (2001). Actionobservation activates premotor and parietal areas in a somatotopic manner: AnfMRI study. The European Journal of Neuroscience, 13, 400–404.

Buccino, G., Lui, F., Canessa, N., Patteri, I., Lagravinese, G., Benuzzi, F., et al. (2004).Neural circuits involved in the recognition of actions performed by noncon-specifics: An fMRI study. Journal of Cognitive Neuroscience, 16, 114–126.

Callan, D. E., Jones, J. A., Callan, A. M., & Akahane-Yamada, R. (2004). Phonetic per-ceptual identification by native- and second-language speakers differentiallyactivates brain regions involved with acoustic phonetic processing and thoseinvolved with articulatory-auditory/orosensory internal models. NeuroImage,22, 1182–1194.

Callan, D. E., Jones, J. A., Munhall, K. G., Callan, A. M., Kroos, C., & Vatikiotis-Bateson, E.(2003). Neural processes underlying perceptual enhancement by visual speechgestures. NeuroReport, 14, 2213–2218.

Campbell, R., MacSweeney, M., Surguladze, S., Calvert, G. A., McGuire, P. K., Brammer,M. J., et al. (2001). Cortical substrates for the perception of face actions: An fMRIstudy of the specificity of activation for seen speech and for meaningless lower-face acts (gurning). Cognitive Brain Research, 12, 233–243.

Calvert, G. A., & Campbell, R. (2003). Reading speech from still and moving faces: Theneural substrates of visible speech. Journal of Cognitive Neuroscience, 15, 57–70.

Cheng, Y., Lee, P.-L., Yang, Ch-Y., Lin, Ch-P., Hung, D., & Decety, J. (2008). Genderdifferences in the mu-rhythm of the human mirror-neuron system. PLoS ONE, 3,1–7.

Di Pellegrino, G., Fadiga, L., Fogassi, L., Gallese, V., & Rizzolatti, G. (1992). Understand-ing motor events: A neurophysiological study. Experimental Brain Research, 91,176–180.

Fadiga, L., Craighero, L., Buccino, G., & Rizzolatti, G. (2002). Speech listening specifi-cally modulates the excitability of tongue muscles: A TMS study. European Journalof Neuroscience, 15, 399–402.

Galantucci, B., Fowler, C. A., & Turvey, M. T. (2006). The motor theory of speech


perception reviewed. Psychonomic Bulletin & Review, 13, 361–377. 514

Grant, K. W., & Seitz, P. F. (2000). The use of visible speech cues for improving auditory 515

detection of spoken sentences. Journal of the Acoustical Society of America, 108, 516

1197–1208. 517

Hari, R., Levanen, S., & Raij, T. (2000). Timing of human cortical functions during 518

cognition: Role of MEG. Trends in Cognitive Sciences, 4, 455–462. 519


E

INN

6 sycho

J520

521

522

K523

524

525

K526

527

528

K529

530

K531

532

533

K534

535

K536

537

L538

539

540

L541

542

L543

544

L545

546

547

L548

L549

550

L551

552

M553

554

M555

556

M557

558

M559

560

M561

562

563

M564

565

M566

567

568

N569

570

N571

572

N573

574

O575

576

577

O578

579

580

581

O582

583

584

585

586

587

588

589

590

591

592

593

594

595

596

597

598

599

600

601

602

603

604

605

606

607

608

609

610

611

612

613

614

615

616

617

Q2 618

619

620

621

622

623

624

625

626

627

628

629

630

631

632

633

634

635

636

637

638

639

640
EC
T



asper, H. H. (1958). Report on the committee on methods of clinical examinationin electroencephalography. The ten-twenty system. Electroencephalography andClinical Neurophysiology, 10(Suppl.), 371–375.

alinowski, J., & Saltuklaroglu, T. (2003). Choral speech: The amelioration of stutter-ing via imitation and the mirror neuronal system. Neuroscience and BiobehavioralReviews, 27, 339–347.

eysers, C., Kohler, E., Umilta, M. A., Nanetti, L., Fogassi, L., & Gallese, V. (2003).Audiovisual mirror neurons and action recognition. Experimental Brain Research,153, 628–636.

latt, D. H. (1979). Speech perception: A model of acoustic-phonetic analysis andlexical access. Journal of Phonetics, 7, 279–285.

limesch, W., Doppelmayr, M., Russegger, H., Pachinger, T., & Schwaiger, J. (1998).Induced alpha band power changes in the human EEG and attention. Neuro-science Letters, 244, 73–76.

ohler, E., Keysers, Ch., Umilta, M. A., Fogassi, L., Gallese, V., & Rizzolatti, G. (2002).Hearing sounds, understanding action. Science, 27, 846–848.

uhl, P. K., & Miller, J. D. (1975). Speech perception by the chinchilla: Voiced-voicelessdistinction in alveolar plosive consonants. Science, 190, 69–72.

ahav, A., Saltzman, E., & Schlaug, G. (2007). Action representation of sound:Audiomotor recognition network while listening to newly acquired actions. TheJournal of Neuroscience, 27, 308–314.

iberman, A. M. (1957). Some results of research on speech perception. Journal of theAcoustical Society of America, 29, 117–123.

iberman, A., Cooper, F., Shankweiler, D., & Studdert-Kennedy, M. (1967). Perceptionof speech code. Psychological Reviews, 74, 431–461.

iberman, A. M., Delattre, P., & Cooper, F. S. (1952). The role of selected stimulus-variables in the perception of the unvoiced stop consonants. American Journal ofPsychology, 65, 497–516.

iberman, A., & Mattingly, I. (1985). The motor theory revised. Cognition, 21, 1–36.iberman, A. M., & Whalen, D. H. (2000). On the relation of speech to language. Trends

in Cognitive Sciences, 4, 187–196.otto, A. J., Hickok, G. S., & Holt, L. L. (2009). Reflections on mirror neurons and speech

perception. Trends in Cognitive Sciences, 13, 110–114.ann, V. A., & Liberman, A. M. (1983). Some differences between phonetic and

auditory modes of perception. Cognition, 14, 211–235.assaro, D. W., & Cohen, M. M. (1990). Perception of synthesized audible and visible

speech. Psychological Science, 1, 55–63.cGurk, H., & MacDonald, J. (1976). Hearing lips and seeing voices. Nature, 264,

746–748.öttönen, R., Järveläinen, J., Sams, M., & Hari, R. (2004). Viewing speech modulates

activity in the left SI mouth cortex. NeuroImage, 24, 731–737.uthukumaraswamy, S. D., Johnson, B. W., Gaetz, W. C., & Cheyne, D. O. (2004).

Modulation of neuromagnetic oscillatory activity during the observation of oro-facial movements. Neurology and Clinical Neurophysiology, 2, 1–4.

uthukumaraswamy, S. D., & Johnson, B. W. (2004). Changes in rolandic mu rhythmduring observation of a precision grip. Psychophysiology, 41, 152–156.

uthukumaraswamy, S. D., Johnson, B. W., & McNair, N. A. (2004). Mu rhythm mod-ulation during observation of an object-directed grasp. Cognitive Brain Research,19, 195–201.

ishitani, N., & Hari, R. (2000). Temporal dynamics of cortical representation foraction. PNAS, 97, 913–918.

ishitani, N., & Hari, R. (2002). Viewing lip forms: Cortical dynamics. Neuron, 36,1211–1220.

yström, P. (2008). The infant mirror neuron system studied with high density EEG.Social Neuroscience, 3, 334–347.

berman, L. M., Hubbard, E. M., McCleery, J. P., Altschuler, E. L., Ramachandran, V.S., & Pineda, J. A. (2005). EEG evidence for mirror neuron dysfunction in autismspectrum disorders. Cognitive Brain Research, 24, 190–198.

UN

CO

RR


berman, L. M., Ramachandran, V. S., & Pineda, J. A. (2008). Modulation of musuppression in children with autism spectrum disorders in response to famil-iar or unfamiliar stimuli: The mirror neuron hypothesis. Neuropsychologia, 46,1558–1565.

hala, J. (1996). Speech perception is hearing sounds, not tongues. Journal of theAcoustical Society of America, 99, 1718–1725.

D P

RO

OF


Ojanen, V., Möttönen, R., Pekkola, J., Jääskeläinen, I. P., Joensuu, R., Autti, T., etal. (2005). Processing of audiovisual speech in Broca’s area. NeuroImage, 25,333–338.

Pineda, J. A. (2005). The functional significance of mu rhythms: Translating “seeing”and “hearing” into “doing”. Brain Research Reviews, 50, 57–68.

Pineda, J. A., Allison, B. Z., & Vankov, A. (2000). The effects of self-movement, obser-vation, and imagination on mu-rhythms and readiness potentials: Toward abrain-computer interface. IEEE Transactions on Neural Systems and RehabilitationEngineering, 8, 219–222.

Pineda, J. A., & Hecht, E. (2009). Mirroring and mu rhythm involvement in socialcognition: Are there dissociable subcomponents of theory of mind? BiologicalPsychology, 80, 306–314.

Pulvermüller, F., Huss, M., Kherif, F., del Prado Martin, F. M., Hauk, O., & Shtyrov, Y.(2006). Motor cortex maps articulatory features of speech sounds. PNAS, 103,7865–7870.

Remez, R. E., Rubin, P. E., Pisoni, D. B., & Carrell, T. D. (1981). Speech perceptionwithout traditional speech cues. Science, 212, 947–950.

Rizzolatti, G., & Arbib, M. A. (1998). Language within our grasp. Trends in Neuroscience,21, 188–194.

Rizzolatti, G., & Craighero, L. (2004). The mirror neuron system. Annual Review ofNeuroscience, 27, 169–192.

Rizzolatti, G., Fadiga, L., Gallese, V., & Fogassi, L. (1996). Premotor cortex and therecognition of motor action. Cognitive Brain Research, 3, 131–141.

Saygin, A. P., Wilson, S. M., Hagler, D. J., Jr., Bates, E., & Sereno, M. (2004). Point-lightbiological motion perception activates human premotor cortex. The Journal ofNeuroscience, 24, 6181–6188.

Skipper, J. I., Nusbaum, H. C., & Small, S. L. (2005). Listening to talking faces: Motorcortical activation during speech perception. NeuroImage, 25, 76–89.

Skipper, J. I., von Wassenhove, V., Nusbaum, H. C., & Small, S. L. (2007). Hearing lipsand seeing voices: How cortical areas supporting speech production mediateaudiovisual speech perception. Cerebral Cortex, 17, 2387–2399.

Stevens, K. N. (1981). Constraints imposed by the auditory system on the propertiesused to classify speech sounds: Evidence from phonology, acoustics, and psy-choacoustics. In T. Myers, J. Laver, & J. Anderson (Eds.), The cognitive representationof speech (pp. 61–74). Amsterdam, New York: Elsevier North-Holland.

Sumby, W. H., & Pollack, L. (1954). Visual contribution of speech intelligibility innoise. Journal of the Acoustical Society of America, 26, 125–212.

Sundara, M., Namasivayam, A. K., & Chen, R. (2001). Observation-execution matchingsystem for speech: A magnetic stimulation study. Neuroreport, 12, 1341–1344.

Sussman, H. (1989). Neural coding of relational invariance in speech: Human lan-guage analogs to the barn owl. Psychological Review, 96, 631–642.

Ulloa, E. R., & Pineda, J. A. (2007). Recognition of point-light biological motion: Murhythms and mirror neuron activity. Behavioral Brain Research, 183, 188–194.

Van Wassenhove, V., Grant, K. W., & Poeppel, D. (2005). Visual speech speeds up theneural processing of auditory speech. PNAS, 102, 1181–1186.

Virji-Babul, N., Moiseeva, A., Cheunga, T., Weeks, D., Cheynec, D., & Ribary, U. (2008).Changes in mu rhythm during action observation and execution in adults withDown syndrome: Implications for action representation. Neuroscience Letters,436, 177–180.

Watkins, K. E., & Paus, T. (2004). Modulation of motor excitability duringspeech perception: The of Broca’s area. Journal of Cognitive Neuroscience, 16,978–987.

Watkins, K. E., Strafella, A. P., & Paus, T. (2003). Seeing and hearing speech excites themotor system involved in speech production. Neuropsychologia, 41, 989–994.

Willems, R. M., Ozyurek, A., & Hagoort, P. (2007). When language meets action: Theneural integration of gesture and speech. Cerebral Cortex, 17, 2322–2333.

Wilson, M. (2002). Six views of embodied cognition. Psychonomic Bulletin & Review,9, 625–636.


Wilson, S. M., & Iacoboni, M. (2006). Neural responses to non-native phonemes vary- 641

ing in producibility: Evidence for the sensorimotor nature of speech perception. 642

NeuroImage, 33, 316–325. 643

Wilson, S. M., Saygin, A. P., Sereno, M. I., & Iacoboni, M. (2004). Listening to speech 644

activates motor areas involved in speech production. Nature Neuroscience, 7, 645

701–702. 646


Mu wave suppression during the perception of meaningless syllables: EEG evidence of motor...

Documents

Transcript of Mu wave suppression during the perception of meaningless syllables: EEG evidence of motor...