Its not only in the eyes

International Journal of Psychophysiology xxx (2013) xxx–xxx

INTPSY-10644; No of Pages 8

Contents lists available at SciVerse ScienceDirect

International Journal of Psychophysiology

j ourna l homepage: www.e lsev ie r .com/ locate / i jpsycho

It's not only in the eyes: Nonlinear relationship between face orientation and N170amplitude irrespective of eye presence

Mikołaj Magnuski ⁎, Mateusz GolaDepartment of Psychology, University of Social Sciences and Humanities, Warsaw, Poland

⁎ Corresponding author. Tel.: +48 693 412 337.E-mail address: [email protected] (M. Mag

0167-8760/$ – see front matter © 2013 Elsevier B.V. Allhttp://dx.doi.org/10.1016/j.ijpsycho.2013.04.016

Please cite this article as: Magnuski, M., Golairrespective of eye presence, International Jo

a b s t r a c t
a r t i c l e i n f o
Article history:Received 30 January 2013Received in revised form 13 April 2013Accepted 20 April 2013Available online xxxx

Keywords:N170Face inversion effectFace processingEvent Related PotentialsEEG

We have investigated the interplay between face orientation, eye presence, and N170 amplitude by recordingEvent Related Potentials. To clarify previous reports of nonlinearity in N170 amplitude changes along rotationangle changes, we adopted Itier et al.'s model (Itier et al., 2007) which links N170 face inversion effects withthe presence of eyes. Comparison of N170 amplitude and latency for five stimulus categories (Faces-with-eyes,Faces-without-eyes, Eyes, Cars-with-lights, Cars-without-lights) in five different rotations (0, 45, 90, 135, 180)resulted inmixed conclusions. The main findings of this study are as follows: (1) a strong nonlinear relationshipbetween N170 and angle of rotation that is specific to faces, distinguishing face from car category evenwhen nosignificant differenceswere observedbetween these categories for upright and inverted orientations; and (2) thenonlinear relationship betweenN170 and angle of rotation does not depend on eye presence.Wealso propose analternative model according to which N170 amplitude consists of two related aspects of face processing: (A) in-compatibility (relative distance of the stimulus pattern from experience-based hypothetical prototype) and(B) integration (degree to which stimulus is integrated into holistic representation), with the former affectingthe latter. Moreover, we suggest two possible neural events underlying these two aspects of face processing:neural population size activated by the stimulus, and synchronization within this population.

© 2013 Elsevier B.V. All rights reserved.

1. Introduction

Humans are unquestionably experts in face processing: faces arean advantageous stimulus category in terms of memorization andrecognition. However, one of the drawbacks of expertise with uprightfaces is the impairment of processing when faces are presented in aninverted orientation. This effect is much larger for faces than for anyother visual object (Yin, 1969). Although this face inversion effect(FIE) has been studied extensively, its causes have yet to be fullyexplained.

Researchers have previously used intermediate degrees of rotationas an investigative tool to provide a more detailed perspective on theFIE (Murray et al., 2000; Stürzel and Spillmann, 2000; Rossion andBoremanse, 2008; Jemel et al., 2009; Jacques and Rossion, 2007).Studies using Thatcherized faces—faces with mirror-inverted eyes andmouth that look grotesque when presented upright but almost normalin an inverted orientation (Thompson, 1980)—and awide range of rota-tion degrees suggest nonlinearity in the FIE. At an angle of approximate-ly 90–120° the perception of grotesqueness of Thatcherized facesintensifies (Murray et al., 2000), with subjects reporting a ‘qualitativeswitch’ in perception within that angle range (Stürzel and Spillmann,2000; see also Lewis, 2001).

nuski).

rights reserved.

, M., It's not only in the eyes:urnal of Psychophysiology (2

The dominant view in explaining the abovementioned effects as-sumes that faces are processed in a much more configural or holisticmanner than other stimuli (Tanaka and Farah, 1993; Maurer et al.,2002; Rossion, 2009; but see Sekuler et al., 2004). The idea ofconfigural processing stresses the importance of processing secondorder relations, which are relations between facial features such asthe distances between the eyes, nose, and mouth (see Maurer et al.,2002 for a review). Holistic processing refers to a related processbut emphasizes its integrated, gestalt-like nature; this has been dem-onstrated, for example, in composite illusion (Young et al., 1987) andthe part-whole effect (Goffaux, 2008). Indeed, there is evidence forboth: impaired configural processing by face inversion and greater re-liance on local features (Leder et al., 2001), as well as impaired holis-tic processing of inverted faces (Rossion and Boremanse, 2008; VanBelle et al., 2010). Although Maurer et al. (2002) argue for the func-tional separability of these aspects of face processing, it is possiblethat they are closely related such that second-order processing relieson holistic processing (Rossion, 2009).

The N170-Event Related Potential (ERP) component, shown to bespecifically enhanced by presentation of face stimuli (Bentin et al.,1996), is also sensitive to the main experimental methods of testingconfigural/holistic processing: face inversion (Rossion et al., 1999),thatcherization (Milivojevic et al., 2003; Carbon et al., 2005), andcomposite illusion (Jacques and Rossion, 2009). The effects are never-theless quite paradoxical: although N170 is face-selective and widelyrecognized as a fingerprint of structural encoding (Eimer, 2000; Sagiv

Nonlinear relationship between face orientation and N170 amplitude013), http://dx.doi.org/10.1016/j.ijpsycho.2013.04.016

http://dx.doi.org/10.1016/j.ijpsycho.2013.04.016

mailto:[email protected]


http://www.sciencedirect.com/science/journal/01678760


2 M. Magnuski, M. Gola / International Journal of Psychophysiology xxx (2013) xxx–xxx

and Bentin, 2001; Latinus and Taylor, 2006; Harris and Nakayama,2008; Eimer et al., 2011), its amplitude is actually higher in responseto stimuli that cause impaired configural processing. Most prominently,N170 amplitude is higher and its latency is longer for inverted facesthan for upright faces (Rossion et al., 1999). As shown in studies exam-ining composite illusion (Letourneau and Mitchell, 2008; Jacques andRossion, 2009), N170 is also stronger for misaligned face stimuli.Upright thatcherized faces, on the other hand, elicit higher N170 ampli-tude than upright regular faces. However, this effect is reversed(inverted thatcherized faces give rise to lower N170: Carbon et al.,2005) or disappears (Milivojevic et al., 2003) for the inverted orienta-tion. N170 amplitude is also related to a decrease in behavioral accuracyand an increase in reaction time for FIE tasks (Jacques and Rossion,2007), which would suggest that an increased N170 amplitude reflectsthe additional costs of processing inverted faces.

Several accounts explaining N170 FIE have been proposed, mostof which can be classified as quantitative or qualitative approaches.According to the first family of models (Rossion et al., 1999;Watanabe et al., 2003; Marzi and Viggiano, 2007) stronger N170 am-plitude for inverted faces stems from higher demands on processingsuch stimuli. This in turn causes face-specific neural populations tobe more active. The second group of models (Itier et al., 2007;Sadeh and Yovel, 2010) emphasizes the involvement of additionalneural resources in inverted face processing. For example, invertedfaces may be perceived as such non-typical stimuli that their process-ing triggers the supplementary recruitment of neural populations notspecialized in processing faces, which in turn causes a qualitativeswitch in face processing.

The pattern of changes in N170 amplitude accompanying increas-ing face rotation can be informative in addressing these models, asnonlinearity in such a pattern may suggest that qualitative changesare taking place (e.g. a sudden recruitment of additional neural pop-ulations). Concurrently, N170 FIE has also been investigated byadding intermediate angles of rotation, which allows for N170 ampli-tude and latency changes to be observed as a function of rotationangle (Jemel et al., 2009; Jacques and Rossion, 2007). These studiesare compatible with the aforementioned nonlinearity observed inbehavioral studies. Jacques and Rossion (2007), although primarilyinterested in the correlation between ERP values and subject perfor-mance for rotated stimuli, showed nonlinearity in the relationship be-tween N170 amplitude and picture rotation. Their results indicatethat N170 amplitude increases with rotation angles between 30 and90°, but remains stable past that range until 180° (inverted orienta-tion). Similarly, Jemel et al. (2009) showed an approximately linearincrease in N170 amplitude with face rotation up to 90°. This increasewas followed by a relative lack of amplitude changes for further rota-tional degrees (with a similar effect for latency). The authors suggestthat this nonlinearity could support qualitative models but not deci-sively, especially as the source analysis they performed did not un-cover any changes in the N170 source location along rotationalsteps — a prediction which follows the qualitative assumption of ad-ditional recruitment.

Oneway to resolve this confusionwould be to directly test a specificqualitativemodel using the rotational approach. One influential qualita-tive model (Itier et al., 2007) assumes that higher N170 amplitude forinverted faces is caused by additional recruitment of eye-selective neu-rons that remain silent when a face is presented in the upright position.Such additional eye-specific activity can be triggered due to weakerconfigural processing of inverted faces and greater reliance on local fea-tures or, as Itier et al. (2007) suggest, due to diminished inhibition ofthese neurons because of disturbed face context.

We chose Itier et al.'s model as a test case for nonlinearity effectsprimarily due to its specificity with respect to the neural processesengaged in N170 FIE and the properties of face stimuli (eye presencevs. inversion) that invoke these neural changes. Because of thesecharacteristics, Itier et al.'s model is both attractive and relatively

Please cite this article as: Magnuski, M., Gola, M., It's not only in the eyes:irrespective of eye presence, International Journal of Psychophysiology (2

easy to test. Our reasoning was that if eye presence is responsible forN170 FIE, then the nonlinear results observed by Jemel et al. (2009)should be present only if rotated faces have eyes (or at least morestrongly in this case). In fact, Itier et al. (2007) showed that there is noamplitude difference between upright and inverted eyeless faces. It istherefore possible that this type of face presentation will result in littleor no variation in N170 amplitude throughout rotation steps.

In order to address these hypotheses we have investigated theinterplay between stimulus orientation, eye presence, and N170amplitude and latency by recording Event Related Potentials for fivestimulus categories (Faces-with-eyes, Faces-without-eyes, Eyes, Cars-with-lights, Cars-without-lights) in five different rotations (0, 45, 90,135, 180). On the basis of Itier et al.'s model, we expected to find strongnonlinear N170 amplitude changes with rotation for the Faces-with-eyes category only and significant differences between Faces-with-eyes and Faces-without-eyes categories starting at least as earlyas 135° of rotation.

2. Subjects and methods

Fifteen young and healthy subjects (8 males; age range18-25 years, M = 22,3; SD = 1,1) participated in this study. All par-ticipants had normal or corrected to normal vision, and each receivedcandy as compensation for their participation.

32 different gray-scale photographs were used in procedure prep-aration: 16 neutral emotional state faces (50% male) and 16 cars. Faceimages were taken from the Warsaw Set of Emotional Facial Expres-sion Pictures (Olszanowski et al., 2008) and car images originatefrom a study by Benjamin Dering (Dering et al., 2009). These mate-rials were acquired and used with the authors' consent.

To verify our hypotheses, we prepared five categories of stimulibasing on the 32 described images: 1. Faces; 2. Faces-without-eyes;3. Eyes-only; 4. Cars; and 5. Cars-without-lights (see Fig. 1a). Stimulifrom each category were prepared in five angles of clock-wise rota-tion: 0°, 45°, 90°, 135°, and 180° (see Fig. 1b). All images werecropped so that the eye region/headlights were exactly in the centerof the stimulus. The pivot point was exactly between the eyes at theheight of the pupils for faces, and between the headlights for cars.The eye-only stimuli included the eyes, eyebrows, and the region be-tween the eyes. All pictures were presented centrally on a gray back-ground. The total number of all stimuli was 400 (5 categories × 5angles × 16 different photographs).

All 400 stimuli were randomly presented in four blocks containing100 trials, starting with a fixation cross (450–750 ms) followed bystimulus presentation (250 ms). The inter stimulus interval (ISI)lasted for 550 ms (see Fig. 1c). In order to sustain attention butavoid biasing it towards any individual stimulus category, eachblock contained 20 additional trials requiring the assessment ofdisplayed line orientation. Participants had to answer if a lineappearing on-screen was horizontal or vertical. During all other trialsthe task was simply to observe the displayed stimuli. Blocks wereseparated with self-regulated breaks (no longer than 10 min).

The EEG signal was acquired using a Biopac MP150 amplifier withstandard 10–20 Electro-cap system (21 electrodes). Hardware bandpass filter was set from 0.1 to 100 Hz and data was recorded with200 Hz sampling rate at 20 k amplification. Linked earlobe clipswere used as a reference, and impedance of electrodes was keptbelow 5 kΩ. Subjects sat in a sound-proof experimental room infront of a computer LCD monitor which was 80 cm away fromthem. ACQKnowledge 4.1 software was used for signal acquisition,and the experimental procedure was conducted using E-Prime 2.

3. Analysis

For data analysis, EEGLAB 8.0.3.5 (Delorme and Makeig, 2004),Matlab 2007b, and SPSS 17 software were used. One of the fifteen



0o45o

90o

135o

180o

+

550 msISI

Fixation450-750 ms

Stimulus250 ms

1

2

3

4

5

a) Stimulus category

b) Angles of rotation

c) Trial design

Fig. 1. a) Five categories of stimuli used in the experiment: 1. Faces; 2. Faces-without-eyes; 3. Eyes-only; 4. Cars; and 5. Cars-without-lights; b) Five angles of clock-wise rotation: 0°,45°, 90°, 135°, 180°; c) each trial started with fixation cross (450–750 ms) and was followed by stimulus presentation (250 ms), ISI duration = 550 ms.

3M. Magnuski, M. Gola / International Journal of Psychophysiology xxx (2013) xxx–xxx

participants was excluded from further analysis because of the lowbehavioral results accuracy (70%). Trials contaminated with artifacts(including eye-movement and blinking) were removed for all four-teen remaining subjects. These contaminated trials amounted to atmost 15%, depending on the particular subject. Three separate algo-rithms from EEGLAB software were used to identify the artifacts.These algorithms searched for abnormal trends (with criterion of50 μV/s for minimal slope of the signal), abnormal distribution ofdata (trials with kurtosis in excess of four standard deviations), ab-normal spectra, and findings with abnormal values (with >±60 μVcriterion). Finally, we visually inspected all remaining trials for finalcorrections.

All accepted trials were averaged into 650 ms sweeps (−200 to450 ms) according to the five stimulus categories and five rotationangles. N170 mean amplitude was measured using an automatedprocedure at the T6 (P8) electrode, where it was maximal. Peak am-plitudes and latencies were taken at maximum negativity using a ±50 ms window centered at 170 ms after stimulus onset (windowlimits: 120 and 220 ms). Normalized mean amplitudes were calculat-ed (McCarthy and Wood, 1985) for each subject. Repeated-measuresanalyses of variance (ANOVAs) were conducted for N170 amplitudeand latency comparing all categories of stimuli and rotation angles.Greenhouse–Geisser adjusted degrees of freedom were used forANOVA when applicable. Multiple comparisons used Bonferroni cor-rections. All planned comparisons (within-category inversion effects)used paired sample t-tests (two-tailed).

4. Results

The assessment of line orientation (horizontal or vertical) in 20trials, which controlled for sustained attention, was almost perfect(95–100%) in all fourteen subjects. There was no significant differ-ence (p > 0.8) between controlling trials preceded by images offaces and cars.

For verification of our predictions, peaks in theN170 range extractedfrom grand mean ERP waveforms were compared between the fivestimulus categories (1. Faces; 2. Faces-without eyes; 3. Eyes-only;4. Cars; and 5. Cars-without-lights) and five rotation angles (0°, 45°,90°, 135°, 180°), according to N170 amplitude and latency (Fig. 2).


4.1. Main effect of the stimuli type

The main effect of stimulus type (averaged across the 5 rotation an-gles) was found to be significant (F(4, 52) = 19.14; p b 0.001; eta2 =0.596). There were no significant statistical differences between any ofthe three face-like stimuli categories (Faces, Faces-without-eyes, andEyes-only). N170 amplitude for the two remaining categories (Carsand Cars-without-lights) was significantly smaller (p b 0.001) thanfor all other stimuli. These two car categories do not show significantdifferences (p > 0.8) (Fig. 3a, top).

Latencies for Faces and Faces-without-eyes were significantlyshorter (p b 0.01) than latencies for Cars and Cars-without-lights.There were no differences within faces and cars categories (p > 0.9).Stimuli from the Eyes category had the longest latency, and differed sig-nificantly from all other categories of stimuli (p ≤ 0.001with respect toFaces, Faces-without-eyes, Cars-without-lights, and p b 0.05 for Cars).

These results show that removing the eyes from stimuli depictinga face (i.e. Faces-without-eyes) does not significantly affect N170 la-tency nor amplitude. Similarly, removing the lights from stimulidepicting cars (i.e. Cars-without-lights) does not change N170 ampli-tude or latency (Fig. 3a).

4.2. Inversion effect

In order to checkwhether the inversion of stimulus affects N170 am-plitude, ANOVA for stimulus type (5) × stimulus orientation (2: upright,inverted) was conducted. The analysis showed a significant main effectof stimulus orientation (F(1, 13) = 10.01; p b 0.01; eta2 = 0.439).Paired comparisons analyses showed an inversion effect (strongeramplitude for inverted stimulus) not only for Faces-with-eyes category(p b 0.05), but also for Cars (t(13) = 2.78, p b 0.05), and Cars-without-lights (t(13) = 4.32, p b 0.01) (see Fig. 3b). Inversion effectsfor the categories Eyes and Faces-without-eyes were not significant.

To inspect latency effects, stimulus type (5) × stimulus orientation(2: upright, inverted) ANOVA was conducted, showing a significantmain effect of stimulus orientation (F(1, 13) = 7.45; p b 0.05; eta2 =0.364) and a significant main effect of stimulus type (F(2.09, 27.10) =19.79; p b 0.001; eta2 = 0.604). The greatest increase in latency wasobserved for Faces-with-eyes category (t(13) = −3.419; p b 0.01),



-10

-8

-6

-4

-2

0 0 0 0 0

Faces with eyes Faces without eyes Eyes Cars Cars without lights

150

160

170

180

190

-10

-8

-6

-4

-2

150

160

170

180

190

-10

-8

-6

-4

-2

150

160

170

180

190

-10

-8

-6

-4

-2

150

160

170

180

190

-10

-8

-6

-4

-2

150

160

170

180

1900 45 90 135 180

0 45 90 135 180

0 45 90 135 180

0 45 90 135 180

0 45 90 135 180

0 45 90 135 180

0 45 90 135 180

0 45 90 135 180

0 45 90 135 180

0 45 90 135 180

Late

ncy

(ms)

Am

plitu

de (

µV)

Fig. 2. N170 amplitudes (a) and latencies (b) at electrode P8 (T6) for each of the five stimulus categories (1. Faces; 2. Faces-without-eyes; 3. Eyes-only; 4. Cars; and5. Cars-without-lights) and five rotation angles (0°, 45°, 90°, 135°, 180°).


while Faces-without-eyes displayed only a trend (t(13) = -1.914). La-tency increase was also significant for Eyes (t(13) = 3.4; p b 0.01),but not for any Car category.

4.3. Stimulus rotation

Another ANOVA analysis (stimulus rotation (5) × stimulus type(5)) revealed a significant main effect of rotation (F(2.04, 26.5) =15.74; p b 0.005; eta2 = 0.353) and a significant interaction effectof rotation and stimulus type (F(16, 208) = 2.11; p b 0.001;eta2 = 0.273) in N170 amplitude.

140

150

160

170

180

190

Am

plitu

de (

µV)

Late

ncy

(ms)

-8

-7

-6

-5

-4

-3

-2

-1

0

-8

-7

-6

-5

-4

-3

-2

-1

0

140

150

160

170

180

190

faces eyes cars

Am

plitu

de (

µV)

Late

ncy

(ms)

a) b)

faceswithout eyes

carswithout lights

Fig. 3. a) N170 amplitude and latency mean scores (averaged across the five rotation ang(bottom) at electrode P8 for inversion effects — comparison of 0° and 180° for each stimul


A separate ANOVA was conducted for all Face categories (angle ofrotation (5) × with/without eyes (2)), showing an even stronger ef-fect for angle of rotation (F(1.5684, 20.389) = 12.4258; p b 0.001)with no significant effect of eye presence (p = 0.99344) and a trendfor eye presence ∗ angle of rotation interaction (F(4.000, 52.000) =2.541; p = 0.051).

4.4. Nonlinear relationship between angle of rotation and N170amplitude is specific to faces and independent of eye presence

To test for nonlinearity of N170 amplitude for rotation dynamicsof faces, polynomial ANOVA contrasts were applied to both Face

faces eyes carsupright

inverted

faceswithout eyes

carswithout lights

les) for all stimulus types used in the study. b) N170 amplitudes (top) and latenciesi category. The error bars represent standard error.




categories (Faces-with-eyes and Faces-without-eyes) separately. Bothcategories displayed significant quadratic contrasts (Faces-with-eyes:F(1, 13) = 67.808; p b 0.00001; Faces-without-eyes: F(1, 13) =26.615; p b 0.001), with no other contrast reaching significanceapart from fourth-order for Faces-without-eyes (F(1, 13) = 16.963;p b 0.05). The effect was similar when contrast analysis was appliedto an averaged Face category (both Face categories averaged foreach step of rotation, F(1, 13) = 105,75; p b 0.001; eta2 = 0.891).No other stimulus category showed a significant square contrast(p > 0.6). These results show that the nonlinear (quadratic) change ofN170 amplitude with rotation is specific to faces (Fig. 4) and does notdepend on eye presence (Fig. 5).

Paired comparisons for all Cars and all Faces categories showedsignificant differences for angles: 45° (t(13) = −3.73; p b 0.01),90° (t(13) = −11.97; p b 0.001), and 135° (t(13) = −7.98;p b 0.01). Although previous analysis of the inversion effect (stimulustype (5) × stimulus orientation (2) ANOVA; Fig. 3) suggested that themean amplitudes of Car, Car-without-lights, and Face are similar,extending the data set by intermediate rotation angles allows us todistinguish face categories from car categories as their N170 dynam-ics in rotation are considerably different (Fig. 4). This confirms theimportance of processes influencing the N170 component dynamicsthat appear between the extremes of the stimulus orientation variable(upright and inverted).

Differences in mean latency occurring between stimuli categoriesshowed a main effect of stimulus type (F(2.08, 26.97) = 32.67,p b 0.001; eta2 = 0.72), significant main effect of rotation (F(1.92,24.88) = 10.29, p b 0.005; eta2 = 0.442), and a significant interac-tion effect (F(6.42, 83.40) = 2.05, p b 0.05; eta2 = 0.136).

5. Discussion

We have investigated the interplay between stimulus (face or car)orientation, eye presence, andN170 amplitude and latency by recordingEvent Related Potentials for five stimulus categories (Faces-with-eyes,Faces-without-eyes, Eyes, Cars-with-lights, Cars-without-lights) infive different rotations (0, 45, 90, 135, 180; see Fig. 2 for general results).

Main findings of this study were as follows: (1) the strongnonlinear relationship between N170 and angle of rotation is specificto faces, distinguishing face from car category even if no significant

-10

-9

-8

-7

-6

-5

-4

-3

-2

-1

00 45 90 135 180

faces eyeless faces

cars cars without lights

Am

plitu

de (

µV)

Angle of rotation (in degrees)

Fig. 4. Comparison of N170 amplitudes at electrode P8 for faces (Faces andFaces-without-eyes; black lines) and cars (Cars and Cars-without-lights; gray lines)at five rotation angles (0°, 45°, 90°, 135°, 180°).


differences are observed between these categories for upright andinverted orientations; and (2) the nonlinear relationship betweenN170 and angle of rotation does not depend on eye presence.

5.1. Nonlinear relationship between face orientation and N170amplitude is face-specific

Although we did not find significant amplitude differences be-tween Faces and Cars categories for both basic orientations (uprightand inverted), we observed significant differences for all intermediateangles of rotation. This finding is important because it corresponds tobasic results of studies casting doubt on N170 face-selectivity(Thierry et al., 2007; Dering et al., 2009, 2011), and at the sametime provides evidence for N170 face-selectivity. It has been sug-gested, for example, that N170 face-selectivity is a methodologicalartifact arising from lower inter-stimulus variance (i.e. higher homo-geneity) within the face stimuli category (Thierry et al., 2007). Whilethis claim is extensively discussed and refuted by other authors(Rossion and Jacques, 2008; Bentin et al., 2007; Ganis et al., 2012),the influence of within-category variance on N170 amplitude remainsan open question. Previous results show that cropped faces producesignificantly higher N170 amplitude (Dering et al., 2011), whichcould explain the N170 face-specificity present in studies usingcropped faces and uncropped control stimuli. In our study, we useduncropped stimuli and took care so that both cars and faces createda homogenous group— all faces as well as all cars were photographedfrom the same frontal angle. This may potentially explain why wefailed to observe differences in N170 amplitude between cars andfaces in upright as well as inverted positions. However our resultsdo not point to N170 being non-face-specific because the overallN170 rotational dynamics are considerably different for car and facecategories (see Fig. 4).

The increase in amplitude for inverted car stimuli likely stemsfrom their higher resemblance to faces in that orientation (we used‘front-facing’ cars only). Inverted cars bring to mind cartoon facesrather than realistic faces (see Fig. 1b); however, as schematic faces(Latinus and Taylor, 2006) or face-looking objects (Hadjikhani et al.,2009) give rise to higher N170 amplitude comparable to normalfaces, this possibility should be taken into consideration.

5.2. Nonlinear relationship between face orientation and N170amplitude is independent of eye presence

Our results are in conflict withwhat onemight expect if eye-specificactivity was the dominant force behind N170 FIE, as Itier et al. (2007)suggest. First of all, we did not find significant differences betweeninverted Faces-with-eyes and inverted Faces-without-eyes, as Itier etal. did. In fact, the only orientation with significant differences betweenthese categories was the upright position (Faces-without-eyes >Faces-with-eyes), which again is in conflict with Itier et al.'s findings.While these discrepancies for upright and inverted orientations mayin some part be due to methodological differences (additional rotationsteps, different behavioral tasks, possible differences in stimuli used,seeing same faces with and without eyes multiple times for differentorientations), it does not seem plausible that the general lack ofeye-presence effects in our data is purely due to these differences.After all, if Itier's model was correct, the eye-presence effect should berobust and seen for at least some angles of rotation. However, we didnot observe such changes for any of the intermediate angles (45, 90,135) and inverted orientation, while differences for upright orientationare in an opposite direction to Itier et al.'s model (see Fig. 5).

While our results do not support Itier et al.'s model, a definitiveresolution of these discrepancies should come from intracranial re-cordings. Recent findings by Rosburg et al. (2010) show that the re-gion speculated by Itier et al. to be the locus of eye-selective cellsand thus taking part in N170 FIE, the superior temporal sulcus



0

2

4

6

8

200 004003002001001

Faces without eyes

***

-10

-8

-6

-4

-2

-10

-8

-6

-4

-2

0

2

4

6

8

200 004003002001001

Faces with eyes

*ns.

ns.

* ns.

*****

- significant - not significant * - p < 0.05 - p < 0.01 - p < 0.001** ***

µV µV

smsm

Upright

Rotated by 90 degrees

Inverted

----

Fig. 5. ERPs at electrode P8 for inversion effects — comparison of Faces and Faces-without-eyes at 0°, 90° and 180° of rotation.


(STS), is not responsible for the effect because it is not sensitive tostatic stimuli. Moreover, apart from showing that N170 FIE can betracked down to the lateral occipital cortex (LOC), Rosburg et al.showed that face-selective but also house-selective sites within theLOC were more active during presentation of inverted than uprightfaces. Such observations, although in line with a qualitative approachto N170 FIE, are at odds with Itier et al.'s model. However, they do notdisprove it directly because eye-selective activity was not measured.

5.3. Nonlinear N170 rotational dynamics

That being said, the main point of this study was not to assess Itieret al.'s model validity, but to investigate the dependency of N170amplitude on face orientation at intermediate steps of rotation. Itieret al.'s model was thought of as a potentially useful tool to clarifyand distinguish qualitative and quantitative effects in N170 rotationdynamics.

Although our findings did not follow expectations derived from Itieret al.'s model, they do not rule out a qualitative interpretation. The ob-served nonlinear N170 amplitude changes could suggest a ‘qualitativeswitch’ occurring alongside rotation. Interestingly, the highest ampli-tude of N170 for both Face categories is approximately 90° of rotation,which is significantly higher than the next rotational step of 135°. Thisis similar to previous behavioral studies using Thatcher Illusions,which showed that the highest decay of holistic processing occurswithin the 90–120° range (Rossion and Boremanse, 2008; Stürzel andSpillmann, 2000) and that subject's reported occurrence of a perceptualswitch in the same range. We may therefore suspect that the observedpeak of N170 amplitude at 90° and its subsequent decrease reflects thisdeficit in holistic processing.

On the other hand, we cannot rule out a quantitative interpreta-tion on the basis of our results. It is possible that N170 amplitude re-flects difficulty of processing, and that for some reason faces rotatedby approximately 90° are more difficult to process. One such reasoncould be that in this rotation range facial features that are normallyarranged horizontally are arranged vertically (and vice versa), whileboth upright and fully inverted faces share the same vertical and hor-izontal organization and symmetry of features. There is evidence thatperception of horizontal and vertical relations is not equally impairedby face rotation (Goffaux and Rossion, 2006), but no behavioral re-sults support the notion that face processing is more difficult at 90°of rotation than both upright and inverted orientations.


While it is not obvious why N170 amplitude, interpreted ‘quantita-tively’ or ‘qualitatively’ alone, should behave in the manner observedin our results, there is the possibility of a hybrid interpretation whereboth ‘qualitative’ and ‘quantitative’ changes take place with rotation.We outline such a hybrid model in the following section.

5.4. Alternative model of N170 changes with face rotation

We propose a hybrid model according to which N170 amplitudeconsists of two related aspects of face processing: (A) incompatibil-ity (relative distance of the stimulus pattern from experience-basedhypothetical prototype) and (B) integration (degree to whichstimulus is integrated into holistic representation), with the formeraffecting the latter. Moreover, we suggest possible neural eventsunderlying these two aspects of face processing: neural populationsize activated by the stimulus, and synchronization within thispopulation.

Incompatibility (distance to prototype) increases with relativedistance from typical orientations of previously experienced faces.Faces tilted by 90° or more from the upright position are rarely ex-perienced and thus incompatible. Incompatibility should increasewith stimulus rotation (from upright to inverted), with the steepestslope occurring as the stimulus approaches an orientation withinthe 90–120° range.

We suggest that this component of the proposed model is relatedto the size of the neural population which is activated by stimuluspresentation. More precisely, we refer to the ‘sharpening’ model ofpriming and perceptual learning (Desimone, 1996; Wiggs andMartin, 1998; Grill-Spector et al., 2006), which states that well-learned stimuli have corresponding specialized, sharp representa-tions incorporating neurons (or neural sets) that over time haveproven to be most relevant and effective in stimulus encoding. Thismeans that the overall size of the stimulus-coding population shoulddecrease with learning, leading to sparse, specialized representations(Freedman et al., 2006; Sigala and Logothetis, 2002; Baker et al.,2002). We propose that once such selectivity has been established,atypical or degraded stimuli from the learned class activate largerneural populations because they contain features (or manipulationsof features) not present in the original stimuli set to which neural re-sponses sharpened. There is evidence for such an effect. For example,Rainer et al. (2004) showed that more neurons respond to degradedlearned stimuli than to learned coherent stimuli or degraded novelstimuli. Such an incompatibility effect could be the main mechanism




responsible for increasing N170 amplitude with rotation in the 0–90°range (in our results as well as in studies of Jemel et al., 2009; Jacquesand Rossion, 2007), or higher N170 amplitude to faces of unnaturalcolor (Minami et al., 2011; Nakajimaa et al., 2012), monkey faces(De Haan et al., 2002), Thatcherized faces (Milivojevic et al., 2003;Carbon et al., 2005), and disrupted human faces (Cassia et al.,2006). Moreover, previously reported lower N170 amplitude in re-sponse to familiar compared to unfamiliar faces (Jemel et al., 2003;Marzi and Viggiano, 2007) is also consistent with the sharpeningeffect.

Integration (representing the stimulus as a whole), the secondpart of the model, is related to the level of holistic face processing.This type of processing has been shown to drop most drasticallywithin the 90–120° orientation range. We relate this aspect of themodel to the integrity and effectiveness of the representation—theinfluence it has on further processing (in this case it is the effective-ness of holistically representing a face). Activity evoked by a stimu-lus in the relevant region may be wide-spread, but with relativelylow effectiveness (especially compared to the size of the activatedpopulation). One mechanism with the potential for increasingeffectiveness of a representation is the degree of oscillatory synchro-ny between units that compose it (Fries, 2005; Singer, 1999). Robust,overlearned representations should not only be sparse, but alsowell-synchronized. Strong oscillatory synchronization of the repre-senting population leads to its stability as well as conveying infor-mation more effectively because the information package is sent asone, in the same phase (Fries, 2005). This increases its influence onfurther processing stages and should specifically promote holisticprocessing.

Our model emphasizes interaction between incompatibility andintegration: sharp representations should promote synchronization,while representations without clear-cut borders and with activationspilling over large and sometimes conflicting neural populationsshould hinder synchronization. Therefore, rising incompatibility hasa negative impact on integration, which is an effect relevant to faceinversion and especially in the context of our results. As the face stim-ulus is rotated further away from upright orientation it becomesmoreincompatible, and thus activates wider neural populations. At thesame time, the size of the activated population starts to weaken itsoverall synchronization and as a result disturbs the holistic process-ing of a face. Such an interaction should lead to a stronger signal atearly steps of rotation (45°) when the dominant force influencingN170 amplitude is the size of the activated population, but shouldlater be balanced or diminished by counteracting de-synchronization

0%

20%

40%

60%

80%

100%

120%

0

0.5

1

1.5

2

2.5

3

3.5

0° 45° 90° 135° 180°

po

pu

lati

on

siz

e

angle of rotation

IncompatibilityIntegration

Fig. 6. Graphical representation of interaction between incompatibility and integration procferent angles of face rotation.


of the activated population (135° and further; see Fig. 6 for a simple de-piction of this idea).

The proposed interaction between incompatibility and integra-tion incorporates quantitative changes (growing incompatibility,decreasing integration) that may have qualitative effects: asincompatibility grows, the neural populations activated by theface stimulus should include more neuronal groups which are lessface-specific, possibly culminating in recruitment of object-specificpopulations as suggested by some fMRI studies (Aguirre et al.,1999; Haxby et al., 1999) and intracortical recordings (Rosburg etal., 2010). Inclusion of such conflicting populations and decreasingsynchronization between them could lead tomodified stimulus pro-cessing by way of, for example, significant contribution fromless-face-specific populations.

6. Conclusions

We have investigated the interplay between face orientation, eyepresence, and N170 amplitude by recording Event Related Potentials.To clarify previous reports of nonlinearity in N170 amplitude changesalong rotation, we adopted Itier et al.'s model (2007) of N170 face in-version effect.

The main findings of this study are the following: (1) the strongnonlinear relationship between N170 and angle of rotation is specificto faces, distinguishing face from car category even if no significantdifferences are observed between these categories for upright andinverted orientations; (2) the nonlinear relationship between N170and angle of rotation does not depend on eye presence.

We also proposed a hybrid model according to which N170amplitude consists of two related aspects of face processing: (A)incompatibility (relative distance of the stimulus pattern fromexperience-based hypothetical prototype) and (B) integration (de-gree to which stimulus is integrated into holistic representation),with the former affecting the latter. Moreover, we suggest possibleneural events underlying these two aspects of face processing: neu-ral population size activated by the stimulus, and synchronizationwithin this population.

Acknowledgment

Wewould like to thank Jan Kaseja for his invaluable help with EEGrecording and data analysis as well as his uplifting enthusiasm.

0° 45° 90° 135° 180°

% s

ynch

ron

ized

Incompatibility * Integration[pouplationsize] * [% of population synchronized]

esses assumed by our model and the impact it should have on N170 amplitude at dif-




References

Aguirre, G.K., Singh, A., D'Esposito, M., 1999. Stimulus inversion and the responses offace and object-sensitive cortical areas. NeuroReport 10, 189–194.

Baker, C.I., Behrmann, M., Olson, C.R., 2002. Impact of learning on representation ofparts and wholes in monkey inferotemporal cortex. Nature Neuroscience 5 (11),1210–1216.

Bentin, S., Allison, T., Puce, A., Perez, E., McCrathy, G., 1996. Electrophysiological studiesof face perception in humans. Journal of Cognitive Neuroscience 8, 551–565.

Bentin, S., Taylor, M.J., Rousselet, G.A., Itier, R.J., Caldara, R., Schyns, P.G., Jacques, C.,Rossion, B., 2007. Controlling interstimulus perceptual variance does not abolishN170 face sensitivity. Nature Neuroscience 10, 801–802.

Carbon, C.C., Schweinberger, S.R., Kaufmann, M.J., Leder, H., 2005. The Thatcher illusionseen by the brain: an event-related brain potentials study. Brain Research. CognitiveBrain Research 24, 544–555.

Cassia, V.M., Kuefner, D., Westerlund, A., Nelson, C.A., 2006. Modulation of face-sensitive event-related potentials by canonical and distorted human faces: therole of vertical symmetry and up–down featural arrangement. Journal of CognitiveNeuroscience 18, 1343–1358.

De Haan, M., Pascalis, O., Johnson, M.J., 2002. Specialization of neural mechanisms un-derlying face recognition in human infants. Journal of Cognitive Neuroscience 14,199–209.

Delorme, A., Makeig, S., 2004. EEGLAB: an open source toolbox for analysis of single-trial EEG dynamics. Journal of Neuroscience Methods 134, 9–21.

Dering, B., Martin, C.D., Thierry, G., 2009. Is the N170 peak of visual event-related brainpotentials car-selective? NeuroReport 20, 902–906.

Dering, B., Martin, C.D., Moro, S., Pegna, A.J., Thierry, G., 2011. Face-sensitive processesone hundred milliseconds after picture onset. Frontiers in Human Neuroscience.http://dx.doi.org/10.3389/fnhum.2011.00093.

Desimone, R., 1996. Neural mechanisms for visual memory and their role in attention.Proceedings of the National Academy of Sciences of the United States of America93, 13494–13499.

Eimer, M., 2000. The face-specific N170 component reflects late stages in the structuralencoding of faces. NeuroReport 11, 2319–2324.

Eimer, M., Gosling, A., Nicholas, S., Kiss, M., 2011. The N170 component and its links toconfigural face processing: a rapid neural adaptation study. Brain Research 1376,76–87.

Freedman, D.J., Riesenhuber, M., Poggio, T., Miller, E.K., 2006. Experience-dependentsharpening of visual shape selectivity in inferior temporal cortex. Cerebral Cortex16, 1631–1644.

Fries, P., 2005. A mechanism for cognitive dynamics: neuronal communication throughneuronal coherence. Trends in Cognitive Sciences 9, 474–480.

Ganis, G., Smith, D., Schendan, H.E., 2012. The N170, not the P1, indexes the earliesttime for categorical perception of faces, regardless of interstimulus variance.NeuroImage 62, 1563–1574.

Goffaux, V., 2008. The horizontal and vertical relations in upright faces are transmittedby different spatial frequency ranges. Acta Psychologica 128, 119–126.

Goffaux, V., Rossion, B., 2006. Face inversion disproportionately impairs the perceptionof vertical but not horizontal relations between features. Journal of ExperimentalPsychology. Human Perception and Performance 33, 995–1002.

Grill-Spector, K., Henson, R., Martin, A., 2006. Repetition and the brain: neural modelsof stimulus-specific effects. Trends in Cognitive Sciences 10, 14–23.

Hadjikhani, N., Kveraga, K., Naik, P., Ahlfors, S.P., 2009. Early (N170) activation of face-specific cortex by face-like objects. Neuroreport 20 (4), 403.

Harris, A.M., Nakayama, K., 2008. Rapid adaptation of the M170 response: importanceof face parts. Cerebral Cortex 18, 467–476.

Haxby, J.V., Ungerleider, L.G., Clark, V.P., Schouten, J.L., Hoffman, E.A., Martin, A., 1999.The effect of face inversion on activity in human neural systems for face and objectperception. Neuron 22, 189–199.

Itier, R.J., Alain, C., Sedore, K., McIntosh, A.R., 2007. Early face processing specificity: it'sin the eyes! Journal of Cognitive Neuroscience 19 (11), 1815–1826.

Jacques, C., Rossion, B., 2007. Early electrophysiological responses tomultiple face orienta-tions correlate with individual discrimination performance in humans. NeuroImage36 (3), 863–876.

Jacques, C., Rossion, B., 2009. The initial representation of individual faces in the rightoccipito-temporal cortex is holistic: electrophysiological evidence from the com-posite face illusion. Journal of Vision 9 (6).

Jemel, B., Pisani, M., Calabria, M., Crommelinck, M., Bruyer, R., 2003. Is the N170 forfaces cognitively penetrable? Evidence from repetition priming of Mooney facesof familiar and unfamiliar persons. Brain Research. Cognitive Brain Research 17,431–446.

Jemel, B., Coutya, J., Langer, C., Roy, S., 2009. From upright to upside-down presentation: aspatio-temporal ERP study of the parametric effect of rotation on face and house pro-cessing. BMC Neuroscience. http://dx.doi.org/10.1186/1471-2202-10-100.

Latinus, M., Taylor, M.J., 2006. Face processing stages: impact of difficulty and separa-tion of effects. Brain Research 1123, 179–187.


Leder, H., Candrian, G., Huber, O., Bruce, V., 2001. Configural features in the context ofupright and inverted faces. Perception 30, 73–83.

Letourneau, S.M., Mitchell, T.V., 2008. Behavioral and ERP measures of holistic face pro-cessing in a composite task. Brain and Cognition 67 (2), 234–245.

Lewis, M.B., 2001. The lady's not for turning: rotation of the Thatcher illusion. Perception30, 769–774.

Marzi, T., Viggiano, M.P., 2007. Interplay between familiarity and orientation in faceprocessing: an ERP study. International Journal of Psychophysiology 65 (3),182–192.

Maurer, D., Le Grand, R., Maurer, D., 2002. The many faces of configural processing.Trends in Cognitive Sciences 6 (6), 255–260.

McCarthy, G., Wood, C.C., 1985. Scalp distributions of event-related potentials: an am-biguity associated with analysis of variance models. Electroencephalography andClinical Neurophysiology/Evoked Potentials Section 62 (3), 203–208.

Milivojevic, B., Clapp, W.C., Johnson, B.W., Corballis, M.C., 2003. Turn that frown upsidedown: ERP effects of thatcherization of misorientated faces. Psychophysiology 40,967–978.

Minami, T., Goto, K., Kitazaki, M., Nakauchi, S., 2011. Effects of color information on faceprocessing using event-related potentials and gamma oscillations. Neuroscience176, 265–273.

Murray, J.E., Yong, E., Rhodes, G., 2000. Revisiting the perception of upside-down faces.Psychological Science 11 (6), 492–496.

Nakajimaa, K., Minamib, T., Nakauchia, S., 2012. The face-selective N170 component ismodulated by facial color. Neuropsychology 50, 2499–2505.

Olszanowski, M., Pochwatko, G., Kukliński, K., Ścibor-Rylski, M., Ohme, R., 2008.Warsaw set of emotional facial expression pictures — validation study. Presenta-tion on: EAESP General Meeting, Opatija, Croatia (See: http://emotional-face.org).

Rainer, G., Lee, H., Logothetis, N.K., 2004. The effect of learning on the function of mon-key extrastriate visual cortex. PLoS Biology 2 (2), E44. http://dx.doi.org/10.1371/journal.pbio.0020044.

Rosburg, T., Ludowig, E., Dümpelmann, M., Alba-Ferrara, L., Urbach, H., Elger, C.E., 2010.The effect of face inversion on intracranial and scalp recordings of event-relatedpotentials. Psychophysiology 47, 147–157.

Rossion, B., 2009. Distinguishing the cause and consequence of face inversion: the per-ceptual field hypothesis. Acta Psychologica 132, 300–312.

Rossion, B., Boremanse, A., 2008. Nonlinear relationship between holistic processing ofindividual faces and picture-plane rotation: evidence from the face composite illu-sion. Journal of Vision 8, 1–13.

Rossion, B., Jacques, C., 2008. Does physical interstimulus variance account for earlyelectrophysiological face sensitive responses in the human brain? Ten lessons onthe N170. Neuroimage 39 (4), 1959–1979.

Rossion, B., Delvenne, J.F., Debatisse, D., Goffaux, V., Bruyer, R., Crommelinck, M., Guerit,J.M., 1999. Spatio-temporal localization of the face inversion effect: an event relat-ed potentials study. Biological Psychology 50, 173–189.

Sadeh, B., Yovel, G., 2010. Why is the N170 enhanced for inverted faces? An ERP com-petition experiment. NeuroImage 53, 782–789.

Sagiv, N., Bentin, S., 2001. Structural encoding of human and schematic faces: holisticand part-based processes. Journal of Cognitive Neuroscience 13, 937–951.

Sekuler, A.B., Gaspar, C.M., Gold, J.M., Bennett, P.J., 2004. Inversion leads to quantitative,not qualitative, changes in face processing. Current Biology 14 (5), 391–396.

Sigala, N., Logothetis, N.K., 2002. Visual categorization shapes feature selectivity in theprimate temporal cortex. Nature 415, 318–320.

Singer, W., 1999. Neuronal synchrony: a versatile code for the definition of relations?Neuron 24, 49–65.

Stürzel, F., Spillmann, L., 2000. Thatcher illusion: dependence on angle of rotation.Perception 29 (8), 937–942.

Tanaka, J.W., Farah, M.J., 1993. Parts and wholes in face recognition. Quarterly Journalof Experimental Psychology. A, Human Experimental Psychology 46 (2), 225–245.

Thierry, G., Martin, C.D., Downing, P., Pegna, A.J., 2007. Controlling for interstimulusperceptual variance abolishes N170 face selectivity. Nature Neuroscience 10,505–511.

Thompson, P., 1980. Margaret Thatcher — a new illusion. Perception 9, 483–484.Van Belle, G., De Graef, P., Verfaillie, K., Rossion, B., Lefevre, P., 2010. Face inversion im-

pairs holistic perception: Evidence from gaze-contingent stimulation. Journal ofVision 10 (5).

Watanabe, S., Kakigi, R., Puce, A., 2003. The spatiotemporal dynamics of the face inver-sion effect: a magneto- and electro-encephalographic study. Neuroscience 116,879–895.

Wiggs, C.L., Martin, A., 1998. Properties and mechanisms of perceptual priming. CurrentOpinion in Neurobiology 8, 227–233.

Yin, R.K., 1969. Looking at upside-down faces. Journal of Experimental Psychology 81,141–145.

Young, A.W., Hellawell, D., Hay, D.C., 1987. Configurational information in face percep-tion. Perception 16, 747–759.


http://dx.doi.org/10.3389/fnhum.2011.00093

http://dx.doi.org/10.1186/1471-2202-10-100

http://emotional-face.org

http://dx.doi.org/10.1371/journal.pbio.0020044

http://dx.doi.org/10.1371/journal.pbio.0020044


Its not only in the eyes

Documents

Transcript of Its not only in the eyes