arX

iv:0

904.

4217

v1 [

nlin

.CD

] 27

Apr

200

9

Noname manuscript No.(will be inserted by the editor)

Sami El Boustani and Alain Destexhe∗

Does brain activity stem from high-dimensional chaotic dynamics ?Evidence from the human electroencephalogram, cat cerebralcortex and artificial neuronal networks

Received: date / Revised: date

Abstract Nonlinear time series analyses have suggested thatthe human electroencephalogram (EEG) may share statisti-cal and dynamical properties with chaotic systems. Duringslow-wave sleep or pathological states like epilepsy, corre-lation dimension measurements display low values, while inawake and attentive subjects, there is not such low dimen-sionality, and the EEG is more similar to a stochastic vari-able. We briefly review these results and contrast them withrecordings in cat cerebral cortex, as well as with theoreti-cal models. In awake or sleeping cats, recordings with mi-croelectrodes inserted in cortex show that global variablessuch as local field potentials (local EEG) are similar to thehuman EEG. However, in both cases, neuronal dischargesare highly irregular and exponentially distributed, similarto Poisson stochastic processes. To attempt reconcile theseresults, we investigate models of randomly-connected net-works of integrate-and-fire neurons, and also contrast global(averaged) variables, with neuronal activity. The networkdisplays different states, such as “synchronous regular” (SR)or “asynchronous irregular” (AI) states. In SR states, theglobal variables display coherent behavior with low dimen-sionality, while in AI states, the global activity is high-dim-ensionally chaotic with exponentially distributed neuronaldischarges, similar to awake cats. Scale-dependent Lyapunovexponents andε-entropies show that the seemingly stochas-tic nature at small scales (neurons) can coexist with morecoherent behavior at larger scales (averages). Thus, we sug-gest that brain activity obeys similar scheme, with seem-ingly stochastic dynamics at small scales (neurons), whilelarge scales (EEG) display more coherent behavior or high-dimensional chaos.

Unite de Neurosciences Integratives et Computationnelles (UNIC),CNRS, Gif-sur-Yvette, France∗: Corresponding author at the following address:UNIC, Bat 33, CNRS, 1 Avenue de la Terrasse, 91198 Gif sur Yvette,France.E-mail: Destexhe@unic.cnrs-gif.fr

1 Introduction

A number of methods from nonlinear dynamical systemshave been applied to the different states of the human EEG.Early studies have calculated correlation dimensions fromEEG and reported evidence for low-dimensional chaos forslow-wave sleep [Babloyantz et al., 1985; Mayer-Kress etal., 1988], as well as for pathological states such as epilepsy[Babloyantz & Destexhe, 1986; Frank et al., 1990] or theterminal state of Creutzfeldt-Jakob disease [Destexhe et al.,1988]. These findings have been confirmed by numerousstudies [reviewed in Destexhe, 1992; Elbert et al., 1994; Korn& Faure, 2003]. Interestingly, in these studies, the EEG dy-namics during wakefulness or REM sleep did not show ev-idence for low-dimensional dynamics. These results werealso corroborated with other measurements such as Lyapunovexponents, entropies, power spectral densities, autocorrela-tions and symbolic dynamics [Destexhe, 1992].

The existence of low-dimensional chaotic dynamics insystems with such large degrees of freedom is surprising.Chaotic dynamics in extended coupled systems have beenstudied extensively for the last decades and is still a matterof intense investigation. In particular, for network of sim-plified neurons, as the spin-glass models, mean-field theo-ries have proven the existence of stable chaotic attractors[Sompolinsky et al., 1988; van Vreeswijk & Sompolinsky,1996; van Vreeswijk & Sompolinsky, 1998]. These systemshave been shown to exhibit chaotic persistence regarding pa-rameter changes [Albers et al., 2006], a property that makeschaotic dynamics more common than exceptional [Sprott,2008]. From a computational point of view, chaotic behaviouror nearly chaotic regime (edge of chaos) can be optimal forinformation processing [Bertschinger & Natschlager, 2004;Legenstein & Maass, 2007]. In network models of spikingneurons, chaotic regimes have also been studied [Cessac,2008; Cessac & Vieville, 2008] and can be present only astransients during which the system is numerically indistin-guishable from a usual chaotic attractor. However the life-time of these transient periods is known to increase exponen-tially with network size [Crutchfield & Kaneko, 1988; Tel &

2

Lai, 2008] making them more relevant in practice than thereal stable attractor for large enough networks.

In this paper, we intend to study a specific spiking neuronmodel which displays these properties and yield biophysicalbehaviour relevant to understand EEG data. Classical non-linear tools as the correlation dimension or the Lyapunovexponents have given some insight on macroscopic quanti-ties as the EEG or the overall activity of numerical mod-els but are often criticized because of their limitation forlow-dimensional dynamics [Kantz & Schreiber, 2004]. Inorder to distinguish between microscopic dynamics and col-lective behaviour, we borrow recent tools developed in thecontext of high-dimensional systems and which offer anal-ysis at different scales. In particular, finite-size Lyapunovexponent [Aurell et al., 1997] as well asε−entropy [Gas-pard & Wang, 1993] provide a scale-dependent descriptionof spiking neuron networks and could detect whether or notlow-dimensional dynamics prevail at a macroscopic scales.These tools will be applied to EEG data as well as numericalmodels.

2 Methods

In this section we briefly describe the analytical and numer-ical tools which will be used to probe the nonlinear natureof the EEG recordings or the numerical simulations. Most ofthese analysis are extensively used in the literature. Further-more, we used the TISEAN toolbox [Hegger et al., 1999] toperform most of our nonlinear analysis. We also give infor-mation about the numerical model.

2.1 Phase space reconstruction

We used the method of time-delayed vectors of the time se-ries, which yields reconstructed attractors topologically equiv-alent to the original attractor of the system [Takens, 1981;Sauer et al., 1991]. We chose a fixed delay parameter de-termined by the first minimum of the mutual information[Fraser & Swinney, 1986]. The embedding dimension waschosen such that any self-similar asymptotic behavior satu-rates beyond this dimension, indicating a successful attractorreconstruction.

2.2 Correlation dimension

The correlation dimension was measured from the correla-tion integral [Grassberger & Procaccia, 1983] which is esti-mated by using

C(ε) =1

N2

N

∑i, j=1

Θ (ε −||x(i)−x( j)||) (1)

wherex(i) ∈ IR is the m-dimensional delay vector andε isa threshold distance which reflects the scale under consider-ation. Moreover we used a Theiler window [Theiler, 1990]

according to the temporal correlation of each time series toavoid spurious estimation. If the correlation integral mani-fests a power-law behavior over a range of scale which sat-urates with increasing embedding dimension, the correla-tion dimension is simply the corresponding power-law co-efficient.

2.3 Recurrence plots

This tool provides an intuitive visualization of recurrencesin trajectories on the attractor [Eckmann et al., 1987]. Therecurrence matrix is defined by

RP(i, j) = Θ (ε −||x(i)−x( j)||) (2)

where the matrix entry is equal to 1 if the trajectoryx at i isin theε-neighbor of itself atj and 0 otherwise. Stereotypicpatterns in the recurrence plot can indicate the existence ofperiodic behavior, low-dimensional chaos or stochastic-likedynamics. In particular, diagonal lines are typical of periodicdynamics whereas clouds of dots are produced by a stochas-tic component. For an extensive review on recurrence plot,the reader should refer to [Marwan et al., 2006].

2.4 Finite-size Lyapunov andε-entropy

Scale-dependent nonlinear analysis have been greatly stud-ied the last decade and have come in handy to distinguishdeterministic (possibly chaotic) dynamics from stochasticnoise. In this paper, we will mainly consider two quanti-ties : the Finite-Size Lyapunov Exponent (FSLE) and theε-entropy. The former has been introduced in the context ofdeveloped turbulence [Aurell et al., 1997] and has proven tobe more suited for a broad range of system which dynam-ics can exhibit low-dimensional chaotic behavior only onlarge-scale [Shibata & Kaneko,1998; Cencini et al., 1999;Gao et al., 2006]. Roughly speaking, if we want to quantifythe sensitivity to initial conditions on large scales, it isnec-essary to consider perturbations which are not infinitesimal.Therefore, the FSLE can be defined as follows

λ(δ ) =1

< Tr(δ ) >

⟨

ln

(

∆r

δ

)⟩

(3)

where for a perturbation between two initial trajectoriesδ ,Tr is the minimal time required for those trajectories to beseparated by a distance∆r greater than or equal toδ r wherer = 2 is usually taken. The brackets in Eq. 3 denotes aver-ages on the attractor for many realizations. Theε-entropy isa generalization of the Kolmogorov-Sinai entropy rate [Gas-pard & Wang, 1993] which is defined for a finite scaleε andtime delayτ by

h(ε,τ) = limm→∞

hm(ε,τ) (4)

=1τ

limm→∞

1m

Hm(ε,τ) (5)

3

with

hm(ε,τ) =1τ(Hm+1(ε,τ)−Hm(ε,τ) (6)

whereHm(ε,τ) is the entropy estimated with box partitionof the phase space for box size given byε on the attractorreconstructed with a time delayτ and an embedding dimen-sionm. This quantity exhibits a plateau on particular scalesif a deterministic low-dimensional dynamics occurs at thesescales. It can thus be used to describe large scale dynamicsindependently of the small scale noisy behavior or can beused to distinguish noise from chaos in some cases [Cencini,2000].

2.5 Avalanche analysis

To identify the presence of scale invariance, typical of self-organized critical states [Jensen, 1998], we used an “avalancheanalysis” methods [Beggs and Plenz, 2003]. This methodconsists of detecting “avalanches” as clusters of contiguousevents separated by silences, by binning the system’s activ-ity in time windows (1 ms to 16 ms were used). Cluster ofevents were defined from the spike times among the ensem-ble of simultaneously recorded neurons. The scale invari-ance was determined from the distribution of avalanche size,calculated as the total number of events [Beggs and Plenz,2005].

2.6 Network of Integrate-and-Fire neurons

Networks of ”integrate-and-fire” neurons were simulated ac-cording to models and parameters published previously [Vo-gels & Abbott, 2005; El Boustani et al., 2007]. The networkconsisted of 5,000 neurons, which were separated into twopopulations of excitatory and inhibitory neurons, forming80% and 20% of the neurons, respectively. All neurons wereconnected randomly using a connection probability of 2%.

The membrane equation of celli was given by:

CmdVi

dt= −gL(Vi −EL) + Si(t) + Gi(t) , (7)

whereCm = 1 µF/cm2 is the specific capacitance,Vi is themembrane potential,gL = 5×10−5 S/cm2 is the leak conduc-tance density andEL = -60 mV is the leak reversal potential.Together with a cell area of 20,000µm2, these parametersgive a resting membrane time constant of 20 ms and an inputresistance at rest of 100 MΩ . The functionSi(t) representsthe spiking mechanism intrinsic to celli andGi(t) stands forthe total synaptic current of celli. Note that in this model,excitatory and inhibitory neurons have the same properties.

In addition to passive membrane properties, IF neuronshad a firing threshold of -50 mV. Once the Vm reaches thresh-old, a spike is emitted and the membrane potential is reset to-60 mV and remains at that value for a refractory period of5 ms. This model was inspired from a previous publication

reporting self-sustained irregular states [Vogels and Abbott,2005].

Synaptic interactions were conductance-based, accord-ing to the following equation for neuroni:

Gi(t) = − ∑j

g ji(t)(Vi−E j) , (8)

whereVi is the membrane potential of neuroni, g ji(t) is thesynaptic conductance of the synapse connecting neuronj toneuroni, andE j is the reversal potential of that synapse.E jwas 0 mV for excitatory synapses, or -80 mV for inhibitorysynapses.

Synaptic interactions were implemented as follows: whena spike occurred in neuronj, the synaptic conductanceg jiwas instantaneously incremented by a quantum value (qe =6 nS andqi = 67 nS for excitatory and inhibitory synapses,respectively) and decayed exponentially with a time constantof τe = 5 ms andτi = 10 ms for excitation and inhibition, re-spectively.

3 Evidence for chaotic dynamics in EEG activity

Human EEG recordings during different brain states are il-lustrated in Fig. 1, along with a 2-dimensional representa-tion of the phase portrait obtained from each signal. Dur-ing wakefulness (eyes open) and REM sleep, the dynamicsis characterized by low-amplitude and very irregular EEGactivity, while during deep sleep, the EEG displays slowwaves (“delta waves”) of large amplitude. Oscillatory dy-namics with a frequency around 10 Hz is seen in the occipi-tal region when the eyes are closed (“alpha rhythm”). Patho-logical states, such as epilepsy or comatous states, displaylarge amplitude oscillations, which are strikingly regular.

A first evidence for chaotic dynamics is that EEG dy-namics display a prominent sensitivity to initial conditions.This sensitivity is illustrated in Fig. 2 for the alpha rhythm(awake eyes closed) and slow-wave sleep (stage IV). A setof close initial conditions is defined by choosing a neigh-boring points in phase space, and this set of points is fol-lowed in time. The divergence of the trajectories emanatingfrom each initial condition is evident from the illustration ofFig. 2, and is actually exponential. This exponential diver-gence betrays the presence of at least one positive Lyapunovexponent. A more quantitative investigation using numericalmethods [Wolf et al., 1985] reveals the presence of positiveLyapunov exponents for all EEG states investigated [Des-texhe, 1992].

EEG dynamics also displays other characteristics of chaoticdynamical systems, such as broad-band power spectra (notshown) and fractal attractor dimensions. This latter pointwas investigated by a number of laboratories, and is sum-marized in Fig. 3. Correlation integralsC(r) are calculatedfrom the reconstructed phase portraits for different embed-ding dimensions (Fig. 3A-B). The correlation dimensiondis obtained by estimating the scaling ofC(r) by using loga-rithmic representations, in which the slope directly givesan

4

Fig. 1 Electroencephalogram signals and phase portraits during differ-ent brain states in humans.Left: 5 seconds of EEG activity in different brain states (same ampli-tude scale). Right: 2-dim phase portrait of each signal. Modified from[Destexhe, 1992].

Fig. 2 Illustration of the sensitivity to initial conditions in EEG dy-namics.A cluster of neighboring points in phase space (leftmost panels) is fol-lowed in time and is shown on the same phase portrait after 200ms,400 ms and 3 seconds (from left to right; same data as in Fig. 1). A,Awake eyes closed; B. Sleep stage IV. Modified from [Destexhe, 1992].

estimate ofd. The calculation ofd as a function of the em-bedding dimension (Fig. 3C) saturates to a constant valuesfor some states (such as deep sleep or pathologies), but notfor others. This is the case for the EEG during wakefulness,which dimensiond does not saturate, which is a sign of highdimensionality.

The correlation dimensions obtained for different brainstates are represented in Fig. 4 as a function of the mean

Fig. 3 Correlation dimension calculated from different brain states inhumans.A. Correlation integralsC(r) calculated from sleep stage IV. B. Cor-relation integrals calculated from Awake eyes open. C. Correlationdimension as a function of embedding dimension for different brainstates. Symbols: * = Awake eyes open, + = REM sleep, squares = sleepstage 2, circles = sleep stage 4. Modified from [Destexhe, 1992].

amplitude of the EEG signal. This representation reveals a“hierarchy” of dimensionalities for the EEG. The arousedstates, such as wakefulness and REM sleep, are character-ized by high dimension and no sign of slow-wave activ-ity. As the brain drifts towards sleep, the dimensionality de-creases and attains its lowest level during the deepest phaseof sleep, in which the EEG is dominated by large-amplitudeslow waves. A further decrease is seen during pathologies,which are also dominated by slow-wave activity.

Fig. 4 Dimension – amplitude representation for different brain states.The correlation dimension of the EEG is shown as a function oftheamplitude range (maximal amplitude deflection calculated over 1 sec-ond periods. Modified from [Destexhe, 1992].

5

4 Evidence for stochastic dynamics in brain activity

The above results are consistent with the idea that awakebrain activity may be associated with high-dimensional dy-namics, perhaps analogous to a stochastic system. To fur-ther investigate this aspect, we have examined data fromanimal experiments in which both microscopic (cells) andmacroscopic (EEG) activities can be recorded. The corre-spondence between these variables is shown in Fig. 5 for catcerebral cortex during wakefulness and slow-wave sleep.

Fig. 5 Distributed firing activity and local field potentials in catcortexduring wake and sleep states.Recordings were done using eight bipolar tungsten electrodes in catparietal cortex (data from [Destexhe et al., 1999]). The irregular firingactivity of 8 multi-units is shown at the same time as the LFP recordedin electrode 1. During wakefulness, the activity is sustained and irreg-ular (see magnification below). During slow-wave sleep (SWS), theactivity is similar as wakefulness, except that “pauses” offiring occurin all cells, and in relation to the slow waves (one example isshown ingray in the bottom graph). The boxes in the top graphs are shown inbottom graphs at 20 times higher temporal resolution.

Those electrical measurements were made using tung-sten microelectrodes directly inserted in cortical gray matter[Destexhe et al., 1999]. This recording system enables theextraction of two signals: a global signal, similar to the EEG,which is called “local field potential” (LFP) and reflects theaveraged electrical activity of a large population of neurons.In addition, single neurons can be distinguished and can beextracted. These signals are shown and compared in Fig. 5.During wakefulness, the LFP shows low-amplitude irregu-lar activity, while neuronal discharges seem random. Slow-wave sleep is characterized by dominant delta waves in theLFP, as in human sleep stage IV. The occurrence of slow

waves is correlated with a concerted pause in the firing ofthe neurons [Destexhe et al., 1999].

Analyzing the spike discharge of single neurons revealedthat the interspike-intervals (ISI) are exponentially distributed,in a manner indistinguishable from a Poisson stochastic pro-cess (Fig. 6A). Performing an avalanche analysis revealedthat the distribution of avalanche size from the neuronal dis-charges was also exponential (Fig. 6B). The same scalingcould be explained by uncorrelated Poisson processes, as ifthe neurons discharged randomly and independently. Thisanalysis was reproduced from a previous study (Bedard etal., 2006).

Fig. 6 Analysis of neuronal activity in awake cat cerebral cortex.A. Interspike interval (ISI) distributions computed from extracellularlyrecorded neurons in wakefulness (natural logarithms). A Poisson pro-cess with the same average rate is shown for comparison (blue; curvedisplaced upwards for clarity). The inset shows a log-log represen-tation. The red curves indicate the theoretical value for Poisson pro-cesses. B. Avalanche analysis of extracellular recordingsin the awakecat (natural logarithms). The same analysis was performed on surro-gate data (blue; Poisson processes). The inset shows the same data inlog-log representation. Modified from Bedard et al., 2006.

Thus, these data and analysis show that the dynamics ofneuronal activity in the awake cerebral cortex is similar tostochastic processes. The statistics of the ISI distributions,as well as the collective dynamics (“avalanches”) cannot bedistinguished from a Poisson stochastic processes.

5 Models of irregular dynamics in neuronal networks

We next consider if this type of dynamics can be found intheoretical models of neuronal networks. We consider ran-domly-connected networks of excitatory and inhibitory pointneurons, in which firing activity is described by the “integrate-and-fire” model, while synaptic interactions are conductance-based. As shown in previous publications (Vogels and Ab-bott, 2005; El Boustani et al., 2007), this model can displaystates of activity consistent with recordings in awake neocor-tex, as shown in Fig. 7. The network can display differenttypes of states, such as “synchronous regular” (SR) states,or “asynchronous irregular” (AI), both of which are illus-trated in Fig. 7. In SR states, the activity is oscillatory andsynchronized between neurons, while in AI states, neuronsare desynchronized and fire irregularly, similar to recordingsin awake cats (compare with cells in Fig. 5). The averagedactivity of the network is coherent and of high amplitude

6

in SR states, but is very noisy and of low amplitude in AIstates, similar to the EEG or LFP activity seen in wakeful-ness (compare with EEGs in Fig. 1 and LFPs in Fig. 5).

Inhibitory Conductance ∆ginh

(nS)

Exc

itato

ry C

ondu

ctan

ce ∆

g exc (

nS)

Firing Rate

1 20 40 60 801

3

5

7

9

0

50

100

150

200

Exc

itato

ry C

ondu

ctan

ce ∆

g exc (

nS)

ISI CV

1 20 40 60 801

3

5

7

9

0

0.5

1

1.5

2

100

200

300

400

500

Neu

ron

id

SR state

0 20 40 60 80 1000

200

400

600

Time (ms)

Act

ivity

(H

z)

100

200

300

400

500AI state

0 20 40 60 80 1000

20

40

60

Time (ms)

SR

AI

SR

AI

Fig. 7 Different network states in randomly-connected networks ofspiking neurons.Top: state diagrams representing the average firing rate (left) and thefiring irregularity (ISI CV, right) of a randomly-connectednetworkof 5,000 integrate-and-fire neurons with a sparse connectivity of 2%.The phase diagram is drawn according to the excitatory and inhibitoryquantal conductance. For increasing inhibition the dynamics undergo atransition from synchronous firing among the neuron and regular firingto a state where the synchrony has substantially decreased and neuronsfire irregularly. The black rectangle indicates a transition from the SRto the AI regime. We selected this region for the rest of the study (seenext figures). Bottom: the two different states, “Synchronous Regular”(SR, left) and “Asynchronous Irregular” (AI, right). The correspondingregions in the state diagram are indicated on top. For both panels, theupper part show a raster plot of a 500 neurons taken randomly fromthe network. Each dot is a spike emitted by the correspondingneuronacross time. In the lower part, the mean firing rate computed among thewhole network with a time bin of 0.1 ms. The asynchronous activity isreflected through the less fluctuating mean activity.

The time series of these averaged network activities (bot-tom graphs in Fig. 7) were analyzed similarly to the EEG inSection 3. We were particularly interested in the transitionregion between AI and SR states where the firing looses itscoherence. This trajectory in the phase diagram is shown asa black rectangle in the upper panels of Fig. 7 and is charac-terized by a abrupt drop of activity concomitant with an in-creasing irregular firing. Similar results can be obtained byreducing the excitatory synaptic strength instead. However,to avoid a loss of stability, we preferred to study the tran-sition by increasing the inhibitory synaptic strength. Phaseportraits of SR and AI states, as well as the correspondingrecurrence plots, are illustrated in Fig. 8. As expected, SRstates (leftmost panel in Fig. 8) display a limit cycle phaseportrait, while AI states (rightmost panel) appear as a dense

unstructured attractor. When the dynamics is dominated byinhibition the limit cycle is rapidly blurred by small-scalefluctuations which become ubiquitous for AI states. How-ever, even though the phase portrait does not display anystructure, the recurrence plot still exhibit a strong determin-istic component (diagonal line) which is completely lost fora state well beyond the boundary region (noisy recurrenceplot).

Fig. 8 Recurrence plot and phase portraits of different network states.From left to right, different step of the transition from theSR state(leftmost) to AI state (rightmost). All states differed by the value ofthe inhibitory conductance∆ginh, as indicated. The top graphs indicatethe recurrence plot. The diagonals indicate the existence of a domi-nant periodic component. As the slaved degrees of freedom are un-leashed with increasing inhibition, the recurrence plot isblurred by thestochastic-like component emerging at larger scales. Thisis the mani-festation of the transition to a high-dimensional dynamics. For networkstates in the middle of the AI region, there is almost no visible recur-rency and the recurrence plot looks like that of a stochasticprocess.The bottom graphs show bi-dimensional phase space reconstructionof the corresponding state. We can clearly see the ”noisy” limit cyclewhich progressively degenerates into a dense high-dimensional attrac-tor confirming what is shown by the recurrence plot.

To quantify this progressive release of degree of free-dom, we estimated the correlation dimension on the over-all activity for those different states. Fig. 9 shows the com-puted dimension for three different embedding dimensions.The network activity displays a low-dimension dynamics asexpected for SR state whereas the dimension suddenly in-creases with increasing inhibition conductance. Moreover,the corresponding values don’t saturate with increasing em-bedding dimension suggesting a high-dimensional attractor.Therefore, beyond the dotted black line, the correlation di-mension is not a relevant measure anymore.

We also analyzed the spiking activity of the model us-ing the same statistical tools as for experimental data. Thedistribution of ISIs during AI states was largely dominatedby an exponential component (Fig. 10A, very similar to ex-perimental data (compare with Fig. 6A). Analyzing popula-tion activity, through “avalanche analysis”, displayed expo-nential distributions (Fig. 10B), also similar to experimentaldata (compare with Fig. 6B). Power-law scaling was alsopresent for a limited range (see Fig. 10B, inset). Similar re-sults were obtained for different network states (not shown).

One can ask what would be the effect of a more spe-cific connectivity architecture on these results. In previouswork, we have shown that macroscopic properties in these

7

21 22 23 24 25 26 27 281

2

3

4

5

6

7

8

9

10

11

12Correlation dimension for different network regimes

∆ginh

(nS)

Cor

rela

tion

dim

ensi

on

Embeddingdimension

Dim = 12

Dim = 16

Dim = 20

Fig. 9 Transition to a high-dimensional attractor in the AI regime.The correlation dimension is plotted according to the network statefrom the SR to the AI regime. Saturation of the scaling regionin thecorrelation integral is shown by illustrating the estimated correlationdimension for three embedding dimensions. Below the dottedline, themeasure is considered to saturate correctly whereas above this line,the measure can not be trusted anymore. In particular for lowinhibi-tion conductances, there are severe shifts with increasingembeddingdimension. The dotted line is given by the disappearance of the plateauin theε-entropy of Fig. 12.

networks are conserved for more local connectivity, as longas the the connexions remain sparse [El Boustani & Des-texhe, 2009]. Therefore, the phase diagram for topologicalnetworks owns similar structures as those displayed in Fig.7.However, in the limit of “first-neighbor” connectivity, corre-lations between neurons become significantly stronger andthe resulting neuronal dynamics are more regular and farfrom biological observations. Thus, even though more lo-cal architectures can result in more correlated activity, thesecorrelations have to remain small in order to preserve thenetwork stability and the rich repertoire of regimes. Our re-sults are thus general for locally-connected networks as longas the sparseness is respected.

Fig. 10 Analysis of the network dynamics in a model of asynchronousirregular states.A. ISI distributions during the AI state in a randomly-connected net-work of 16,000 integrate-and-fire neurons. ISI distributions are expo-nential (noisy trace), as predicted by a Poisson process (red line; in-set: log-log representation using natural logarithms). B.Absence ofavalanche dynamics in this model (same description as Fig. 6B). Thered lines indicates regions of exponential scaling; the redline in insetindicates a region with power-law scaling. Modified from El Boustaniet al., 2007.

6 How to reconcile these data ?

The above data show that in both human, cat and models, thedynamics can show clear signs of coherence and low dimen-sionality at the level of macroscopic measurements (EEG,LFPs, averages), while microscopically, neuronal dynamicsare highly irregular and resemble stochastic processes. Inanattempt to reconcile these observations, we rely on recentlyintroduced generalization of classical nonlinear tools. In par-ticular, Finite-Size Lyapunov Exponent [Aurell et al., 1997;Shibata & Kaneko, 1998; Cencini et al., 1999; Gao et al.,2006] and theε-entropy [Cencini et al., 2000] have provento be valuable measures to probe system which display dif-ferent behaviors at different scales. In the present context,where low-dimensional dynamics can take place in top of ahighly irregular neuronal behavior, those measures appearasthe most natural.

They were applied to the human EEG, as shown in Fig. 11.The FSLE in the upper panel neither possess a plateau norbehave according to a power-law except in the small-scalelimit where the stochastic-like component is dominant. ForCreutzfeldt-Jacob disease as well as for epileptic seizure,however, it seems that a small region displays a almost scale-free behavior which would indicate a low-dimensional chaoticdynamics. Following [Shibata & Kaneko, 1998], it seemsthat the cortical activity lives in a high-dimensional attrac-tor which could be chaotic. Therefore, the FSLE does notreally help to untangle the different scales dynamics here.However, when resorting to theε-entropy, we get a differ-ent picture. Fig. 11B clearly shows that most of these cor-tical states manifest a large plateau on large-scale. Theseplateau is a signature of low-dimensional dynamics [Cenciniet al., 2000] whereas the small-scale power-law behavior isthe signature of stochastic-like dynamics produced by high-dimensional attractor. In accordance with Fig. 3, the dynam-ics during awake eyes open and REM sleep do not own aplateau, and their attractor dimension is too high to be distin-guished from noise. However, it should be noted that thosedifferent dynamics are produced by the same system hencebeing mainly modulated by endogenic factors. In light ofthis result, we can conclude that there is no contradictionbetween the experimental data acquired at the macroscopiclevel (EEG) and the at the microscopic level (Spiking Activ-ity). For large network of coupled units, averaged quantitiescan display low-dimensional structured dynamics while be-ing seemingly stochastic at a smaller scale.

We also evaluated the same quantities from the aver-aged activities of the numerical model. In Fig. 12A, we seeas before that the FSLE does not exhibit any scale-free re-gion except for the small-scale limit. Thus we can not relyon this measure to distinguish small- and large- scale be-havior. In contrast, theε-entropy in Fig. 12B yields a largeand distinct plateau in the SR regime. The disappearance ofthis plateau have been used as a criteria to draw the dot-ted black line in Fig. 9. For inhibitory conductances largerthan∆ginh ≃ 23.5 (nS), theε-entropy slowly converges to itssmall-scale stochastic-like behavior. We thus recover a very

8

Fig. 11 Scale-dependent Lyapunov exponent and Epsilon-entropy fordifferent brain states.The scale-dependent Lyapunov exponent (A) andε-entropy (B) werecalculated from the same EEG states as shown in Fig. 1. The FSLE(A) does not show any plateau and if the inverse FSLE is plottedin a semilog scale on the x-axis, no low-dimensional chaoticbehav-ior can be detected for most states (data not shown). Following [Shi-bata & Kaneko,1998] example for general coupled maps, the macro-scopic activity seems to exhibit high-dimensional chaoticdynamics.For small scalesε → 0, a scall-free region is found with a power-law exponent around 0.791 (black dotted line) as expected from themicroscopic stochastic-like dynamics [Cencini et al., 2000; Gao etal., 2006]. Because of the absence of a large scale-free region in thesemilog region, we can not define a macroscopic Lyapunov exponentfrom this measure. However, for the Creutzfeldt-Jacob disease and theepileptic seizure, the FSLE have a free-scale region on a broad rangewhich is different from the other cortical states. This could indicatea low-dimensional chaotic dynamic even though it is not reflected inthe semilog scale. Indeed, theε-entropy (B) manifests a clear plateau(red dotted line) for the epileptic seizure, the Creutzfeldt-Jakob dis-ease, sleep stage 2 and 4 and the alpha waves. These plateau indi-cates the existence of a low-dimensional attractor on the correspondingscales [Cencini et al., 2000] which can sometimes be easily visualizedwith an embedding procedure (see Fig. 1). For REM and awake states,this plateau disappears leaving the dynamics in a stochastic-like (high-dimension) state at all scales. The small-scale behavior isidentical tothe FSLE (green dotted line).

comparable behaviour to the EEG data where pathologicalstates or deep sleep produce structured activity at the EEGlevel and awake state or REM are comparable to the dynam-ics of AI regime, as suggested previously [van Vreeswijk &Sompolinsky, 2006; van Vreeswijk & Sompolinsky, 2008;Vogels & Abbott, 2005; El Boustani et al., 2007; Kumar etal., 2008].

Fig. 12 Scale-dependent Lyapunov exponent and Epsilon-entropy fordifferent network states.The Finite-Size Lyapunov Exponent (A) andε-entropy (B) were cal-culated from the same network states as shown in Fig. 7. The FSLEbehaves almost identically for every network state. There is no evi-dence for a low-dimensional chaotic dynamics which would beindi-cated by a scale-free region in intermediate scales. The power-law atsmall-scale owns a coefficient around 0.418 (black dotted line) charac-terizing the stochastic-like dynamic (high-dimensional deterministic)at those scales. Even though no evidence for low-dimension chaos canbe found from the FSLE, theε-entropy (B) has a large plateau (red dot-ted line) for state lying in the SR region (low inhibitory conductance).This plateau is shortened when the network dynamics is driven towardthe AI regime where eventually the dynamics is indistinguishable fromstochastic process. The small-scale behavior is identicalto the FSLE(green dotted line).

7 Discussion

In this paper, we have briefly reviewed correlation dimen-sion analyses of human EEG, which revealed a hierarchy ofbrain states, where the dimensionality varies approximatelyinversely to the level of arousal (Fig. 4). In particular, awakeand attentive subjects display high dimensionalities, whiledeep sleep of pathological states show evidence for low di-mensionalities. These low dimensions are difficult to recon-cile with the fact that these signals emanate from the activityof millions (if not billions) of neurons. In cat cerebral cor-tex, the cellular activity during wake and sleep are highlyirregular, and exponentially distributed like stochastic(Pois-son) processes, a feature which is also difficult to reconcilewith low dimensionalities at the EEG level, even though LFPdata do not manifest low-dimensional dynamics (data notshown). We performed similar analyses on computationalmodels, which also display these apparently coherent activ-ities at the level of large-scale averages, while the micro-

9

scopic activity is highly irregular. In particular SR states candisplay coherent behavior at large scales, while AI states donot show evidence for coherence, similar to recordings inawake cats and humans.

The nature of the dynamics exhibited by models is as-similable to high-dimensional chaos. It has been shown thatAI states in such models shut down after some time, and arethus transient in nature [Vogels & Abbott, 2005; El Bous-tani et al., 2007; Kumar et al., 2008; El Boustani & Des-texhe, 2009]. This lifetime has been estimated to increaseexponentially with the network size [Kumar et al., 2008;El Boustani & Destexhe, 2009], and can reach considerabletimes (beyond any reasonable simulation time; [Kumar etal., 2008]). Moreover, other recent studies [Cessac, 2008;Cessac & Vieville, 2008] have obtained analytical resultsonsimilar models where “transient chaotic-like regimes” werefound. More precisely, these regimes are periodic, but witha period which also grows exponentially with network size.These results are reminiscent of the non-attractive chaoticmanifolds extensively discussed in the literature [Crutch-field & Kaneko, 1988; Dhamala et al., 2001; Dhamala &Lai, 2002; Tel & Lai, 2008]. Hence, even though the chaoticnature of the dynamics is not inherent of the underlying sys-tem, the network spends a long enough period trapped inthis transient dynamics indistinguishable from chaos fromanumerical point of view.

To characterize the dynamics at different scales, we es-timated quantities such as FSLE orε-entropy, which clearlyshow that microscopic scales (neurons) tend to be very high-dimensional and complex, in many ways similar to “noise”,while more coherent behavior can be present at large scales.This analysis is consistent with the recently proposed con-cept of “macroscopic chaos”, where a very high-dimensionalmicroscopic dynamics coexists with low dimensionality atthe macroscopic level. In this paper, the chosen numericalmodel is known to be purely deterministic and can displayhighly irregular spiking patterns close to stochastic processesor noise. From a nonlinear analysis point of view, most ofthis dynamics is indistinguishable from a random process.However, as soon as a large scale behaviour emerges, theε-entropy can keep track of it and still manifests the small-scale irregular fluctuations.

We conclude that numerical models of recurrent neu-ronal networks, with conductance-based integrate-and-fireneurons, can be assimilated to high-dimensional chaotic sys-tems, and are in many ways similar to the EEG. Moreover,even though they exhibit limit-cycle regimes for several pa-rameter sets, this collective dynamics is built on top of irreg-ular and high-dimensional neuronal activity which is onlyapparent at small-scales. Interestingly, this scheme remindsfluid dynamics, where a seemingly random microscopic dy-namics may also coexist with more coherent behavior atlarge scales. This is the case for example close to the transi-tion to turbulence, where fluids can show evidence for low-dimensional chaos [Brandstater et al., 1983]. More devel-oped turbulence, however, does not show such evidence, pre-sumably because a large number of degrees of freedom have

been excited and the high-dimensional dynamics is presentat all scales. It is possible that similar considerations applyto brain dynamics, especially when considering the recentdebate about the nature of the ongoing activity of visual pri-mary cortex (V1). Recordings made using voltage-sensitivedyes imaging (VSDI) in this cortical area have suggestedthat spontaneous activity consists of a seemingly-random re-play of sensory-evoked orientation maps of activity in anes-thetized [Kenet et al., 2003], but not in awake animals [Omeret al., 2008]. These observations have raised the question ofwhether the cortical activity of V1 could be described by asingle-state high-dimensional attractor or a low-dimensionalmultistable attractor [Goldberg et al., 2004]. It is likelythatat the VSDI scale, both scenarios are possible at the sametime but at different scales, which would be consistent withthe present results.

Acknowledgments

Research supported by the CNRS, ANR and the EuropeanCommunity (FACETS grant FP6 15879). More details areavailable athttp://cns.iaf.cnrs-gif.fr

References

Albers, D.J., Sprott, J.C. & Crutchfield, J.P. [2006] ”Persis-tent chaos in high dimensions,” Phys. Rev. E74:057201

Aurell, E., Boffetta, G., Crisanti, A., Paladin, G. & Vulpi-ani, A. [1997] ”Predictability in the large : an extensionof the concept of Lyapunov exponent,” J. Phys. A30(1),1-26

Babloyantz, A. & Destexhe, A. [1986] ”Low dimensionalchaos in an instance of epileptic seizure”, Proc. Natl.Acad. Sc. USA83, 3513-3517.

Babloyantz, A., Nicolis, C. & Salazar, M. [1985] ”Evi-dence for chaotic dynamics of brain activity during thesleep cycle”, Phys. Lett. A111, 152-156.

Bedard, C., Kroger, H. & Destexhe, A. [2006] ”Does the 1/ffrequency scaling of brain signals reflect self-organizedcritical states?”, Phys. Rev. Lett.97(11) : 118102

Beggs, J. & Plenz, D. [2003] Neuronal avalanches in neo-cortical circuits. J. Neurosci.23, 11167-11177.

Bertschinger, N. & Natschlager, T. [2004] ”Real-Time Com-putation at the Edge of Chaos in Recurrent Neural Net-works,” Neural Comp.16, 1413-1436

Brandstater, A., Swift, J., Swinney, H.L., Wolf, A., Farmer,J.D., Jen, E. & Crutchfield, J.P. [1983] ”Low-dimensionalchaos in a hydrodynamic system”. Phys. Rev. Lett.51:1442-1445.

Cencini, M., Falcioni, M., Vergni, D. & Vulpiani A. [1999]”Macroscopic chaos in globally coupled maps,” PhysicaD 130, 58-72

Cencini, M., Falcioni, M., Olbrich, E., Kantz, H. & Vulpi-ani A. [2000] ”Chaos or noise: Difficulties of a distinc-tion,” Physica D130, 58-72

10

Cessac, B. [2008] ”A discrete time neural network modelwith spiking neurons; Rigorous results on the sponta-neous dynamics,” J. Math. Biol.56, 311-345

Cessac, B. & Vieville, T. [2008] ”On Dynamics of Integrate-and-Fire Neural Networks with Conductance Based Synapses,”Front. Comput. Neurosci.2(2)

Crutchfield, J.P. & Kaneko, K. [1988] ”Are attractors rele-vant to turbulence?,” Phys. Rev. Lett.60(26), 2715-2718

Destexhe, A. [1992]Nonlinear Dynamics of the Rhythmi-cal Activity of the Brain (in French), Doctoral Disser-tation (Universite Libre de Bruxelles, Brussels, 1992).http://cns.iaf.cnrs-gif.fr/alain thesis.html

Destexhe, A., Contreras, D. & Steriade, M. [1999] ”Spa-tiotemporal analysis of local field potentials and unit dis-charges in cat cerebral cortex during natural wake andsleep states”, J. Neurosci.19, 4595-4608.

Destexhe, A., Sepulchre, J.A. & Babloyantz, A. [1988] ”Acomparative study of the experimental quantification ofdeterministic Chaos”. Phys. Lett. A132: 101-106.

Dhamala, M., Lai, Y.-C. & Holt, R.D. [2001] ”How of-ten are chaotic transients in spatially extended ecologicalsystems?,” Phys. Lett. A280, 297-302

Dhamala, M. & Lai, Y.-C. [2002] ”The natural measure ofnonattracting chaotic sets and its representation by un-stable periodic orbits,” Int. J. Bif. Chaos12(12), 2991-3005

Eckmann, J.P., Kamphorst, S.O. & Ruelle, D. [1987] ”Re-currence Plots of Dynamical Systems,” Europhys. Lett5, 973977.

Elbert, T., Ray, W.J., Kowalik, Z.J., Skinner, J.E., Graf K.E.& Birbaumer, N. [1994] ”Chaos and physiology: deter-ministic chaos in excitable cell assemblies”, Physiol. Rev.74: 1-47.

El Boustani, S., Pospischil, M., Rudolph-Lilith, M. & Des-texhe, A. [2007] ”Activated cortical states: experiments,analyses and models”, J. Physiol. Paris101: 99-109.

El Boustani, S. & Destexhe, A. [2009] ”A master equationformalism for macroscopic modeling of asynchronousirregular activity states,” Neural Comp.21: 46-100

Frank, G.W., Lookman, T., Nerenberg, M.A.H., Essex, C.& Lemieux, J. [1990] ”Chaotic time series analysis ofepileptic seizures,” Physica46D: 427-438.

Fraser, A.M. & Swinney H.L. [1986] ”Independent coor-dinates for strange attractors from mutual information,”Phys. Rev. A33, 1134-1140

Gaspard, P. & Wang, X.J. [1993] ”Noise, chaos, and(τ,ε)-entropy per unit time,” Phys. Rep.235(6), 321-373.

Goldberg, J.A., Rokni, U., & Sompolinsky, H. [2004] ”Pat-terns of ongoing activity and the functional architectureof the primary visual cortex,” Neuron42, 489-500

Grassberger, P. & Procaccia, I. [1983] ”Measuring the stran-geness of strange attractors,” Physica D9, 189208

Gao, J.B., Hu, J., Tung, W.W. & Cao Y.H. [2006] ”Distin-guishing chaos from noise by scale-dependent Lyapunovexponent,” Phys. Rev. E74:066204

Hegger, R., Kantz, H. & Schreiber, T. [1999] ”Practicalimplementation of nonlinear time series methods: TheTISEAN package,” Chaos9(413)

Jensen, H.J. [1998]Self-Organized Criticality. EmergentComplex Behavior in Physical and Biological Systems.(Cambridge University Press, Cambridge UK).

Kantz, H. & Shreiber, T. [2004] ”Nonlinear time series anal-ysis,” Cambridge University Press

Kenet, T., Bibitchkov, D., Tsodyks, M., Grinvald, A. &Arieli, A. ”Spontaneously emerging cortical representa-tions of visual attributes,” Nature425, 954-956

Korn, H. & Faure, P. [2003] ”Is there chaos in the brain? II.Experimental evidence and related models”, C.R. Biol.326, 787-840.

Kumar, A., Schrader, S., Aertsen, A. & Rotter, S. [2008]”The High-Conductance State of Cortical Networks”, Neu-ral Comput.20, 1-43

Legenstein, R. & Maass, W. [2007] ”Edge of chaos and pre-diction of computational performance for neural circuitmodels,” Neural Net.20, 323-334

Marwan, N., Romano, M.C., Thiel, M. & Kurths, J. [2006]”Recurrence plots for the analysis of complex systems”Physics Rep.438, 237:329

Mayer-Kress, G., Yates, F.E., Benton, L., Keidel, M., Tirsh,W., Poppl, S.J. & Geist, K. [1988] ”Dimension analy-sis of nonlinear oscillations in brain, heart and muscle”.Math. Biosci.90: 155-182.

Sauer, T., Yorke, J.A. & Casdagli, M. [1991] ”Embedol-ogy,” J. Stat. Phys.65, 579-616

Shibata, T. & Kaneko, K. [1998] ”Collective chaos,” Phys.Rev. Lett.81(9):4116

van Vreeswijk, C.A. & Sompolinsky, H. [1996] ”Chaosin neuronal networks with balanced excitatory and in-hibitory activity,” Science274, 1724-1726

van Vreeswijk, C.A. & Sompolinsky, H. [1998] ”Chaoticbalanced state in a model of cortical circuits,” NeuralComp.10,1321-1372

Sompolinsky, H., Crisanti, A. & Sommers, H.J [1988] ”Chaosin random neural networks,” Phys. Rev. Lett.61, 259-262

Sprott, J.C. [2008] ”Chaotic dynamics on large networks,”Chaos18:023135

Takens, F. [1981] ”Detecting strange attractors in turbu-lence,” Lecture notes in mathematics, Vol.898. Dynami-cal systems and turbulence, Springer, Berlin

Theiler, J. [1990] ”Estimating fractal dimension,” J. Opt.Soc. Amer. A7, 1055

Tel, T. & Lai, Y.-C. [2008] ”Chaotic transients in spatiallyextended systems,” Physics Rep.460, 245-275

Vogels, T.P. & Abbott, L.F. [2005] ”Signal propagation andlogic gating in networks of integrate-and-fire neurons”,J. Neurosci.25, 10786-10795.

Wolf, A., Swift, J.B., Swinney, H.L. & Vastano, J.A. [1985]”Determining Lyapunov exponents from a time series”,Physica D16, 285-317.