A working memory model for serial order that storesinformation in the intrinsic excitability properties of neurons

Eduardo Conde-Sousa & Paulo Aguiar

Received: 19 June 2012 /Revised: 30 January 2013 /Accepted: 8 February 2013 /Published online: 29 March 2013# Springer Science+Business Media New York 2013

Abstract Models for temporary information storage in neu-ronal populations are dominated by mechanisms directlydependent on synaptic plasticity. There are neverthelessother mechanisms available that are well suited for creatingshort-term memories. Here we present a model for workingmemory which relies on the modulation of the intrinsicexcitability properties of neurons, instead of synaptic plas-ticity, to retain novel information for periods of seconds tominutes. We show that it is possible to effectively use thismechanism to store the serial order in a sequence of patternsof activity. For this we introduce a functional class ofneurons, named gate interneurons, which can store informa-tion in their membrane dynamics and can literally act asgates routing the flow of activations in the principal neuronspopulation. The presented model exhibits properties whichare in close agreement with experimental results in workingmemory. Namely, the recall process plays an important rolein stabilizing and prolonging the memory trace. This meansthat the stored information is correctly maintained as long asit is being used. Moreover, the working memory model isadequate for storing completely new information, in time

windows compatible with the notion of “one-shot” learning(hundreds of milliseconds).

Keywords Working memory . Serial order . Intrinsicexcitability modulation . Delay period . Matchenhancement . Short-term memory

1 Introduction

Findings by Eric Kandel, Tim Bliss and Terje Lomo (Blissand Lomo 1973; Kandel 2001), have led to the widely heldview that memories are stored as modifications of synapticstrength. Since this outstanding work, many mathematicalmodels have successfully used synaptic plasticity, in variousforms, as the basis for mechanisms related to learning,memory and development in neural circuits (Abbott andNelson 2000). But synaptic plasticity is not, by far, the onlyadaptive processes available on the nervous system with thesuitable properties for memory formation. Parallel studieshave demonstrated information storage based on non-synaptic processes, with the modulation of membrane con-ductances being one of the most important among them(Mozzachiodi and Byrne 2010). A combination of experi-mental and theoretical studies has shown that neurons withcomplex temporal dynamics can provide short-term memo-ry properties supported only by their intrinsic neuronalmechanisms (Marder et al. 1996). In other words, the mod-ulation of membrane conductances, giving rise to changes inneuronal excitability, can be used to temporarily storeinformation.

The ideas of membrane conductances modulation as amechanism for information storage have naturally perfused

Action Editor: A. Borst

E. Conde-Sousa : P. AguiarFaculdade de Ciências, Universidade do Porto, Rua do CampoAlegre, 687,4169-007, Porto, Portugal

E. Conde-Sousa : P. Aguiar (*)Centro de Matemática da Universidade do Porto, Rua do CampoAlegre, 687,4169-007, Porto, Portugale-mail: pauloaguiar@fc.up.pt

J Comput Neurosci (2013) 35:187–199DOI 10.1007/s10827-013-0447-7

to the mathematical models of working memory (Durstewitzet al. 2000; Fall and Rinzel 2006; Fransen et al. 2002, 2006).Working memory (WM) is a well-established type of dis-tributed short-term memory (Baddeley 2003) which, amongother things, has the capability of being activated with asingle exposure to the information pattern. Specific featuresassociated with the short-term modulation of intrinsic excit-ability, most notably the time-scales, are in accordance withthe requirements of a working memory system. The fewtheoretical studies addressing this connection analyze a spe-cific set of biologically plausible membrane currents thatcan act synergistically to prolong excitability states, and focuson the behavior of single neurons or small networks of neu-rons (Fall and Rinzel 2006; Fransen et al. 2002, 2006).

Here we present a working memory model which usesmodulation of intrinsic neuronal excitability (MINE) fortemporarily storing multiple novel activity patterns and theserial order between them. Previous models for sequencelearning/recall relied mostly on recurrent networks and re-quired specific synaptic plasticity rules (Botvinick and Plaut2006; White et al. 2004). The network architectures havetaken advantage of auto-associative recurrent networks(Gibson and Robinson 1992), networks with transmissiondelays (Herz et al. 1991) or modular functional units(Dehaene et al. 1987). Nevertheless, ubiquitous to thesenetworks for sequence learning is the use of synapsesto store the information. Our model shows that MINEproperties can effectively support the temporary storageof completely new information, on time scales which arecompatible with WM. Furthermore, MINE can producememory formation mechanisms which are simultaneouslyrobust to noise and capable of storing information inexposure time windows of less than 100 milliseconds(compatible with “one-shot” learning). All synapses inour model are static, in the sense that they do not exhibitany form of plasticity.

A key element in the model we propose is the existenceof a specific class of excitatory interneurons which we call“gate” interneurons. These gate interneurons are able tostore information regarding the patterns of activity, and theirorder, in their membranes intrinsic excitability. While shar-ing the use of a calcium activated nonspecific cation current(ICAN) and the absence of synaptic plasticity, the dynamicsof the proposed model and the existence of the gate in-terneurons distinguishes it from previous models capableof one-shot short-term buffering (Jensen et al. 1996; Koeneand Hasselmo 2007, 2008; Lisman and Idiart 1995).

2 Material and methods

The dynamics of the neurons were mathematically describedusing conductance based models following the Hodgkin-

Huxley formalism (Hodgkin and Huxley 1952), in order toachieve a significant biophysical detail. All simulationswere carried out in the NEURON simulation environment(Hines and Carnevale 1997). The model implementationwas written to take advantage of NEURON’s capability torun in parallel hardware. A dual quad-core Xeon 5355(2.66 GHz) workstation with 32 GB RAM was used forthe numerical simulations. The model implementation inNEURON is available through ModelDB (Crasto et al.2007) under accession number 147461.

2.1 Single cell models

Neuronal morphology is not relevant for the WM model wepropose here and, as such, point-wise neurons are consid-ered. In terms of NEURON implementation, each neuron ishowever modeled by a single cylindrical compartment (sin-gle segment) with length L=20 μm and diameter diam=20 μm. The total surface area of the compartment is1,257 μm2. This value can be taken into account to convertarea dependent quantities to absolute values and vice-versa.

Three classes of neurons are considered: i) principalneurons (P) responsible for receiving and transmitting infor-mation to other brain areas; ii) inhibitory interneurons (I)responsible for controlling the network activity level; andiii) excitatory gate interneurons (G) responsible for storingpattern information and pattern order through MINE mech-anisms. Together with leakage current and synaptic currents,all neuron classes have two standard ion currents: the tran-sient sodium and the delayed rectifier potassium currents.The gate interneurons have however two additional ioncurrents: a high threshold calcium current ICaL which isactivated by strong membrane potential depolarization, anda nonspecific cation current ICAN which activation increaseswhen the intracellular calcium concentration rises.

The time evolution of the membrane potential is de-scribed by the equation:

Cm V⋅ ¼ −IL−I ion−I syn

where Cm=1μF/cm2 is the membrane capacitance; V repre-

sents the membrane potential; IL=gL×(V−EL) is the leakagecurrent, gL is the maximal leak conductance and EL is thereversal potential for the leakage current; Iion is the sum ofall the ionic currents on each neuron; and Isyn is the sum ofall synaptic currents impinging on the neuron.

For inhibitory interneurons and principal neurons Iion =INa + IKdr, and for gate interneurons Iion = INa + IKdr + ICaL +ICAN.

A temperature of 22 °C was assumed in all simulationsand all the ionic current dynamics were taken fromSenseLab (Crasto et al. 2007).

188 J Comput Neurosci (2013) 35:187–199

2.1.1 Synaptic current

The synaptic current is modeled as the sum all inputsynaptic currents to each neuron. The model uses threedifferent types of synapses: NMDA, AMPA and GABAB.In all types, the synaptic conductance profile is modeledby a dual exponential function with parameters τrise andτdecay (Table 1). In NMDA synapses the conductance isfurther multiplied by a factor representing the magnesiumblock (Jahr and Stevens 1990). From the total NMDAsynaptic current impinging on the gate interneurons, afraction of 10 % is in the form of calcium current(Burnashev et al. 1995) which contributes to modifica-tions in the intracellular calcium concentration.

Inhibition relies on GABAB synapses with a slowtime course in the order of 200 ms, a value in accor-dance with experimental data (Otis et al. 1993). Thisslow time course, as soon will become clear, is impor-tant to promote competition between individual patternsin the stored sequence: the slow time course ensuresthat each pattern in the sequence is not erroneouslyreactivated. The time scale of the of the GABAB inhi-bition sets an upper bound for the duration of the entiresequence recall and has a direct impact on the maxi-mum number of patterns which can be stored in asequence. A value of 200 ms is chosen to enable thestorage of sequences with 5–6 patterns.

2.1.2 Sodium and potassium currents

The transient sodium INa and the delayed rectifier potassiumIKdr currents are modeled according to the canonicalHodgkin-Huxley kinetics with modifications proposed by(Traub and Miles 1991) to model hippocampal pyramidalcells (parameters in Table 2).

INa ¼ gNa � m3 � h� V−ENað ÞIKdr ¼ gKdr � n4 � V−EKdrð Þ

The kinetic equations for gating variables are given by

x⋅ ¼ −

x−x∞ Vð Þτx Vð Þ

where

x∞ ¼ αx

αx þ βx

and

τ x ¼ 1

αx þ βx

for x ∈{m, n, h}. The activation and inactivation gate func-tions are:

αm ¼ 0:32� 13− V−V traubð Þð Þexp

13− V−V traubð Þ4

� �−1

βm ¼ 0:28� V−V traubð Þ−40ð Þexp

V−V traubð Þ−405

� �−1

αh ¼ 0:128� exp12− V−V traubð Þ

18

� �

βh ¼4

1þ exp35− V−V traubð Þ

5

� �

αn ¼ 0:032� 15− V−V traubð Þð Þexp

15− V−V traubð Þ5

� �−1

βn ¼ 0:5exp10− V−V traubð Þ

40

� �

2.1.3 High-threshold calcium current

The high-threshold calcium current ICaL is modeledaccording to the equation (McCormick and Huguenard1992):

ICaL ¼ pCaL � m2 � GHK V ; cai; caoð Þ

where pCaL is the Ca2+ membrane permeability, cai and cao

are, respectively, the intracellular and extracellular calciumconcentration, and GHK is the Goldman-Hodgkin-Katzequation. The kinetic equation for activation variable is:

m⋅ ¼ −

m−m∞ Vð Þτm Vð Þ

Where

m∞ ¼ 1

1þ expV þ 10

−10

� �

Table 1 Synaptic reversal potential and conductance profileparameters

Synapse EX [mV] τrise [ms] τdecay [ms]

AMPA 0 0.1 5

NMDA 0 5 70

GABAB −85 10 200

J Comput Neurosci (2013) 35:187–199 189

and

τm ¼ 1

αm þ βm

αm ¼ 1:6

1þ exp −0:072� V−5ð Þð Þβm ¼ 0:02� 1:31−V

1−expV−1:315:36

� �

2.1.4 Calcium dependent nonspecific cation current

The calcium dependent nonspecific cation current ICAN istaken from (Bal and McCormick 1993) with modificationsintroduced in (Destexhe et al. 1994) and is described by theequation:

ICAN ¼ gCAN � m2 � V−ECANð Þ

For our version of the model we modified the middlepoint of activation function to 0.5×10−3mM (parametersin Table 2). The introduction of these specific calciumand calcium-dependent currents follows the results frommodeling studies that have demonstrated the effective-ness of these currents in producing temporary modula-tions in neuronal activity (Aguiar et al. 2010; Fall andRinzel 2006).

2.1.5 Intracellular calcium dynamics

The intracellular calcium concentration (cai) is modeledtaking into account the calcium entry due to currentICaL and NMDA synapses, and a fast removal processwhich is due to buffering and active pumps (Destexheet al. 1993). We use depth=0.1 μm, caiτ=1 ms, cai∞=5×10−5mM.

2.2 Network architecture

The network connectivity plays an important role in theWM model for serial order. The three neuronal populationsare connected according to the following architecture: i)inhibitory interneurons establish bidirectional connectionswith the nearest principal neuron; ii) gate interneurons arearranged in clusters also establishing bidirectional connec-tions with a principal neuron; iii) principal neurons also sendlong-range connections to gate interneurons outside thelocal cluster. This architecture configuration places the gateinterneurons in a pivotal position regarding the flow ofinformation in the principal population. An example of asmall network with 3 principal neurons is shown in Fig. 1a,and a schematic representation of the network architecture ispresented in Fig. 1b.

The synaptic peak conductances (efficacies) for each typeof connection in the network are summarized in Table 3.Some synaptic efficacies have to be scaled according to thepattern size, more precisely to the expected number of activegates associated with each principal neuron. In excitatorysynapses with both NMDA and AMPA receptors, the rela-tive contribution of both synaptic currents is subject to the

Table 2 Parameters for ion currents. The notation gX refers to themaximal conductance of current X, EX refers to the reversal potentialand pCaL is the calcium membrane permeability. The reversal potentialfor calcium is calculated according to the Goldman-Hodgkin-Katz(GHK) equation, since the intracellular calcium concentration (cai) isnot constant

Ioncurrent

Parameters Principalneurons

Inhibitoryinterneurons

Gateinterneurons

IL EL (mV) −65 −65 −65

gL (mS/cm2) 4.5×10−2 4.5×10−2 3.9×10−2

INa ENa (mV) 50 50 50

gNa (mS/cm2)100 80 80

Vtraub (mV) −50 −50 −60

IKdr EKdr (mV) −70 −70 −70

gKdr (mS/cm2)20 20 20

Vtraub (mV) −50 −50 −60

ICaL ECa – – GHK

pCaL (cm/s)3×10−5

ICAN ECAN – – 0

gCAN (mS/cm2)0.01

Fig. 1 Network architecture. a Model consisting of 3 principal neu-rons (center), defining 3 clusters with one inhibitory interneuron (top)attached to each principal neuron, and 6 gate interneurons divided ingroups of two (bottom). b Schematic feed-forward representation of thetopographic connections between each population: each interneuronconnects to a principal neuron which in turn establish a large number oflocal connections with the gates local cluster and, in addition, sendsisolated long-range connections to gates outside the local cluster; eachgates cluster sends convergent connections back to the principal neuronwhich closes the circuit with the local interneuron. Notice that in theschematic representation, the two P layers correspond to the samepopulation P (the same for the I layers)

190 J Comput Neurosci (2013) 35:187–199

following constraint: at −72 mV the postsynaptic responseamplitude of both AMPA and NMDA currents is the same(Alonso et al. 1990). To introduce variability in the synap-ses, each individual synaptic peak conductance is drawnfrom a uniform distribution with mean value taken fromTable 3 and with a variability of ±2.5 %. The synaptic peakconductances of the connections from gate interneurons toprincipal neurons are normalized according to the networksize. The normalization factor K reflects the expected num-ber of active gate interneurons connected to each principalneuron. The synaptic conduction delays for all synapses arefixed at 1 ms.

3 Results

For clarity purposes, the results are separated into topics.First the intrinsic properties of the gate interneurons arepresented together with their role in the working memorymodel. Secondly the network behavior is addressed, wherethe emergence of the working memory system is analyzed.Constraints regarding the required properties for efficientstorage and recall are also discussed, followed by informa-tion on the robustness of the qualitative behavior of themodel to changes in parameter space. Finally, issues regard-ing scaling are addressed and a modified version of theshort-term memory model for serial order, with less storagecapacity but better scaling properties, is presented.

3.1 Gate interneuron firing properties

Calcium activated nonspecific cation (CAN) currents playan important role in the dynamics of neurons in the prefron-tal cortex and in the entorhinal cortex (Bal and McCormick1993; Haj-Dahmane and Andrade 1999; Klink and Alonso1997), two regions involved in working memory and se-quence learning mechanisms (Baddeley 2003; Hahn et al.2012; Suh et al. 2011). In our model, the presence ofcalcium dependent cation current with slow dynamics al-lows gate interneurons to have quantitatively different

responses to currents of different duration. When the neuronis at its resting potential for a long period, the CAN currentis negligible. With the first action potential (AP), elicited byan external current, there is an inward current of calcium dueto ICaL, increasing the intracellular calcium concentration(cai). This leads to a partial activation of the CAN currentwhich in turn originates a subthreshold after-depolarization(ADP). If the external current leads to more than one spikeat a firing rate above 11 Hz, the inter spike interval is smallerthan the recovery time of the ICAN activation variable (mICAN ).This implies that at the end of each spike, the variablemICAN ishigher than it was at the end of the previous spike. If thenumber of spikes caused by the external current is small, theCAN current will be only partially activated leading to a smallmembrane depolarization (see Figs. 2a1, b1 and 3a2). How-ever, if there is a sufficient number of spikes in a short periodof time (ifmICAN goes beyond 0.71), the activation of the CANcurrent will generate a depolarization capable of reactivatingthe sodium current leading to a new AP, and starting the cycleagain. This mechanism is transiently maintained without theneed for an external current (Fig. 2a2, b2). This dynamics canbe further assessed in phase space. The generation of an actionpotential is associated with a cycle in the membranepotential (sodium activation followed by sodium inacti-vation and potassium activation) and a rise in intracel-lular calcium concentration cai. As a result of thecombined effect of these factors there is an increase inICAN activation. The trajectories in phase space havethree main regions (Fig. 2b1, b2): i) an initial stagewith current injection where each spike produces acycle in membrane potential and in cai, followed by aprogressive increase in the ICAN activation variable; ii) ifthe stimulation is sufficient (Fig. 2a2, b2) the synergis-tic action of the membrane currents is able to sustainmembrane activity in the absence of external stimulationfor a period of a few seconds; iii) the last region re-flects the loss of stability which results from the pro-gressive deactivation of ICAN.

It is important to notice that this is not cellular bistability:the period of self-sustained spiking is finite and the neuron

Table 3 Synaptic mean peak conductances, in μS, between the threepopulation classes in the network. Gin represents the gate interneuronsinside the principal neuron cluster, while Gout represents the gateinterneurons outside the cluster. The presented excitatory synaptic peakconductances results from the combined sum of NMDA and AMPA

conductances. The AMPA peak conductance corresponds to a fractionof 0.16 of the total conductances and the NMDA peak conductance tothe remaining fraction of 0.84, in accordance to Alonso et al. (1990)(see text)

Post-synaptic neuron

I P Gin Gout

Pre-synaptic neuron I – 1.2×10−3 – –

P 1.5×10−3 – 1.3×10−4 5.0×10−4

G – 1.1×10−3×K – –

J Comput Neurosci (2013) 35:187–199 191

inevitably, and spontaneously, converges to the resting statein the absence of additional stimulation. This behavior isinstead associated with the generation of plateau potentialsand may be seen as a transiently stable depolarized state.

The partial activation of the CAN current produced byshort stimulations produces an additional behavior: the am-plitude of the external current needed to prolong the gener-ation of a sequence of APs is smaller than the amplitudeneeded to generate the initial APs. This happens becauseafter each new AP, the accumulating CAN current depolar-izes the membrane and less external current is required forreaching threshold. So if two currents are introduced se-quentially into the neuron, it is possible for the second tobe weaker than the first in order to give rise to prolongedactivity of elicited AP’s (Fig. 3). On the other hand, if thesame two currents are injected on the reverse order, the first(smaller) current only weakly depolarizes the membrane,leading to negligible activation of the CAN current, provid-ing almost no contribution to the membrane responses uponthe arrival of the second current. This asymmetry plays animportant role in our model for serial order since a neuronwith this property can respond differently to whether theweak stimulation arrives first or not.

Consider two currents of 250 ms duration and amplitudesof 1 pA (representing weak synaptic connections) and 3 pA

(representing strong synaptic connections). In panels a2 andb2 of Fig. 2 it can be seen that if these two currents impingeon neuron at the same time (4 pA current injection for250 ms) the neuron undergoes a state transition that allowsit to sustain the activity for roughly 4 s without the aid ofany external current. On the other hand, if these two signalsimpinge on the neuron sequentially, two qualitatively dif-ferent responses are produced. If the weaker current arrivesfirst (Fig. 3, top panels), during the corresponding excitationperiod the membrane only depolarizes slightly which leadsto negligible entrance of calcium which in turn leads tonegligible activation of the CAN current. Therefore, whenthe stronger current arrives, a reduced number of APs areelicited and the neuron is unable to sustain the activity afterthe end of the excitation. However, if the signals arrive inthe reverse order (Fig. 3, bottom panels), a different re-sponse takes place. With the strongest current first, theneuron fires 3 APs which is not enough to sustain theactivity by itself. However, the CAN current is partiallyactivated and causes a depolarization close to the thresholdlevel. When the second (weaker) current arrive, with thehelp of depolarization induced by the CAN current, a fewmore APs are elicited allowing the neuron to sustain theactivity for some seconds after the end of the externalexcitation.

3.2 Network model

With the asymmetry property previously described it ispossible for the gate interneurons to encode a sequencewithout requiring any synaptic efficacy change over time

Fig. 2 Phase space analysis. A current of 4 pA is injected into a gateinterneuron for two different durations: (a1 and b1) 130 ms and (a2and b2) 250 ms. On panels A1 and A2 the three traces are (from top tobottom): membrane potential, CAN current and injected current. Ac-tion potentials are truncated at -45 mV. Panels b1 and b2 show 3Dphase plots corresponding to the two different current injections. The3D phase plots show the evolution of the variables for membranepotential, CAN current activation variable and intracellular calciumconcentration. If enough stimulation is provided (a2 and b2) thesynergistic action of the membrane currents is able to sustain mem-brane activity in the absence of external stimulation, for a period ofroughly 4 s

Fig. 3 Relevance of the input stimuli order for the temporary storagemechanism. Two currents with 250 ms duration are injected sequen-tially into the neuron, one of 1 pA and another of 3 pA. Depending onthe order of the injected currents (a1, b1), the neuron responds verydifferently (a2, b2 respectively). Action potentials are truncated at−40 mV

192 J Comput Neurosci (2013) 35:187–199

and trials. Consider a small circuit with direct connectionsbetween two principal neurons A and B. A key element inthis model is to replace these direct connections with gateinterneurons, with their ability to self-sustain activity, asmediators in the connections between principal neurons(Fig. 4). With the appropriate network architecture, it ispossible to differentiate between AB serial activation andBA serial activation. If during a learning phase neuron A isactivated followed by neuron B, only gate interneuron ABwill be excited to the level of self-sustaining its activity forsome seconds. It is important to notice that this frameworkcan be extended to the case where A and B represent notindependent isolated neurons but rather synchronous ensem-bles of neurons.

These properties of the model are first explored for asmall scale network consisting of 3 independent principalneurons, 3 inhibitory interneurons and 6 gate interneurons,as illustrated in Fig. 1a. Synapses are static and described bythe parameters presented in Tables 1 and 3. The simulationprotocol mimics an external stimulation where two of theprincipal neurons are activated sequentially (P1 and P2),leaving the remaining principal neuron without externalstimulation. The behavior of the network model is shownin Fig. 5.

An external stimulus, originated from another brain re-gion, generates four APs at a firing rate of 17 Hz in neuronP1. After the end of the excitation period of P1, and with adelay close to 60 ms, P2 is subject to a similar externalstimulation. The sequence of firing P1→P2 changes theactivity state of specific gate interneurons. The interneuronresponsible for coding the sequence P1→P2 (G12) receives astrong synaptic current from P1 and after the end of thiscurrent a weaker current from P2 arrives, leading to a

consistent excitation and consequently to a self-sustainedactivity for a period of a few seconds. On the other hand, thegate interneuron responsible for coding the sequence P2→P1(G21) receives the same two currents but in reverse order, soit only fires a small amount of times which is not sufficientto sustain the activity after the excitation period. Both gateinterneurons coding P1→P3 (G13) and P2→P3 (G23) receiveonly the strong current, generate only two APs and are notable to retain the activity.

After the storage period, which in the above mentionedexample lasted approximately 400 ms in total, the gateinterneurons retain the information with a firing rate closeto 12 Hz for a delay period which can be in the order ofseconds. The recall process is initiated after reactivation ofP1. When P1 is reactivated, G12 shows a match enhancementand its firing rate rises to more than 20 Hz producing aselective excitation to P2 which leads to its depolarizationand consequently to the generation of a small number ofAPs (the number depends on the activation of the inhibitoryinterneuron I2 which inhibits P2).

When larger networks are considered, each element in thememory sequence is no longer represented by a singleneuron but instead by a group of neurons firing at the sametime (not necessarily synchronized – a jitter on the order oftens of milliseconds is allowed). Every pattern in the se-quence is composed by the same number of principal neu-rons and the synaptic efficacies do not depend on thenumber of patterns, but rather on each pattern size. Themodel can consequently store sequences with different num-ber of patterns, from one trial to the next, without anyparameter change. Recall that this model does not havesynapses linking principal neurons to each other: principal

Fig. 4 By allowing gate interneurons (AB and BA) to mediate theconnections between principal neurons (A and B) it becomes pos-sible, without any temporal modification in the synaptic efficacies,to store the serial order in the activation of the principal neurons.All synapses are static: strong synapses are represented by w+ andweak synapses by w-

Fig. 5 Working memory in a small network. In this example, thesequence P1→P2 is stored. The sequential activation of the principalneurons P1 and P2 excite consistently the gate interneuron responsiblefor coding the sequence P1→P2 (G12), while exciting less the gateinterneurons coding sequences P2→P1, P2→P3 and P1→P3. The stor-age interval is marked with a shaded bar. Only gate interneuron G12

self-sustains the activity during a delay period. After the storageperiod, every time P1 fires there is a match enhancement in G12 leadingto the activation of P2

J Comput Neurosci (2013) 35:187–199 193

neuron Pi projects to principal neuron Pj if there is a gateinterneuron encoding the sequence i→j. It is worth men-tioning that a recurrent network in the principal neuronspopulation would further facilitate intra-pattern associations,but it is not required for the proper functional behavior ofthe model.

In larger networks it is not necessary to have an individualgate interneuron for each ordered pair of principal neurons. Inthis situation, principal neurons randomly establish long-range connections with gate interneurons outside the localcluster according to a predefined fraction – named gate sam-pling fraction (γ). This fraction can be below 1.0 and repre-sents the probability that, given an ordered pair of principalneurons, there is a gate interneuron establishing the connec-tion between them. It is important to acknowledge that gateinterneurons literally act as activity dependent gates routingthe flow of activations in the principal neurons population.

Consider now a network with 100 principal neurons, witha high connectivity level at the gate interneurons (reflectedin the gate sampling fraction γ=0.95), leading to a totalnumber of neurons close to 10,000. These values are chosento be well-matched with the properties of cortical columns.The number of neurons in a cortical minicolumn is in theorder of 100 with an extremely high connectivity level(Buxhoeveden and Casanova 2002); hypercolumns aggre-gate several minicolumns (again in the order of 100) to formanother modular structure. Collectively, the highlyconnected hypercolumns combine on the order of 10,000neurons. We hypothesize that our working memory modelfor serial order is appropriate at this scale, where eachprincipal neurons can be connected with a group of neighborexcitatory interneurons (in the order of 100) – the gateinterneurons cluster – establishing the proposed networkstructure supporting one-shot short-term information stor-age. It is important to recall, that in our model, each princi-pal neuron can represent a small ensemble of synchronizedprincipal neurons. In terms of cortical columns this meansthat a principal neuron in the model may represent a smallgroup of synchronized pyramidal cells.

To assess the performance of the model, a sequence of 5activity patterns is stored in this network with 100 principalunits, each pattern being represented by 17 P neurons.Evoked activity is induced in the neuronal sub-populationsengaged in encoding the serial memory through generationof 4 spikes at a frequency of 17 Hz. Each sub-population issequentially activated with a delay of 60 ms (measuredbetween the end of one pattern and the beginning of thenext). A jitter taken from a uniform distribution with 20 msrange is further added to the onset of the evoked activity ineach neuron. Neurons that did not participate in the memorystorage were subject to spontaneous stochastic activity withan average firing rate of 0.3 Hz. The results are shown inFig. 6.

A very interesting property that emerges at the networklevel is that the stored information is stable while it is beingused: every reactivation of the sequence helps stabilizing thegate interneurons activity; but if the stored information isnot used (recalled) for a period of more than a few secondsthe gate interneurons become silent and reset.

Simulations show that the model can store and recallsuccessfully sequences with up to 6 patterns. If a seventhpattern is inserted, the recall process is corrupted: eachpattern retains its individuality but the information of orderis partially lost.

3.3 Model’s time constraints

Temporal features are crucial elements on this modeland it is important to quantify the robustness of thedynamics to the properties of the time windows associ-ated with storage and recall. Important aspects includethe amount of time needed to store a sequence ofpatterns, the time interval between the storage of onepattern and the next one, the total amount of timeduring which the model can hold the information (if itis not being actively used), and mainly, the time re-quired for the model to reset the information when it isnot needed anymore. These points are now addressed.

Given the parameterization of the model, to activate thegate interneurons responsible for the inter-pattern codifica-tion is necessary that the principal neurons fire four spikes atfiring rates between 16 and 20 Hz and the neurons in thenext pattern fire another four spikes in the same frequencyrange. If the interval between the last spike in one constel-lation and the first spike in the next is too short, gate in-terneurons assume that all the neurons belong to one uniquepattern and the sequence will be lost. On the other hand, if

Fig. 6 Working memory for serial order. In the storage phase thenetwork stores the sequence of activity patterns in the gate interneu-rons, after a single exposure to the stimuli. In the recall phase, afteractivation of the first activity pattern, the network is capable of acti-vating the remaining patterns in the appropriate order. Storage isaccomplished exclusively through the modulation of intrinsic mem-brane excitability properties

194 J Comput Neurosci (2013) 35:187–199

the interval between the last spike in one constellation andthe first spike in the next is too long, when the weakercurrent arrives to the gate interneurons it is no longer ableto elicit the remaining APs required for the activation to beconsidered as consistent. Depending on the firing rate of theactivation, the storage of each pattern in the sequence takesroughly between 150 and 190 ms, while the interval be-tween two consecutive patterns is within 40 and 100 ms.

In order to determine the holding time and resetting time,some tests were performed. The first test involves the stor-age of a sequence of three constellations, constellation 1consisting of neurons P0 to P4, constellation 2 consisting ofneurons P5 to P9, and constellation 3 consisting of neuronsP10 to P14. After the storage period, the reactivations of thefirst pattern were delayed from one trial to the next in 0.5 s.The first reactivation occurs after 2 s of the storage, thesecond reactivation after 2.5 s of the first and so on. It ispossible to notice that the system can sustain the activity byitself for a period above 4.5 s (Fig. 7a). In terms of resettingtime, the system must remain unstimulated for a periodclose to 6 s before it completely forgets the previous infor-mation and be ready to store a new sequence (Fig. 7b).

The 6 s interval reflects the time needed to naturallydestabilize the gates self-sustained activity and deactivatethe currents involved in the intrinsic excitability modulation.This interval can nevertheless be shortened if the action ofinhibitory synapses is considered. The self-sustained activ-ity can be interrupted with strong inhibitory synapses (or

with the combined action of synapses with smaller conduc-tances).These synapses have been modeled by alpha func-tions with 10 ms time constant, 2 × 10−2 μS peakconductance, and reversal potential of −75 mV (Fig. 7c, d).So, in the presence of inhibitory control, after roughly 1 s, thesystem can be reset and allowed to store completely newsequences. In the example shown in Fig. 7, a new sequencewas constructed by reactivating the same patterns by thereverse order. However, any other sequence of patterns wouldbe possible, even sequences comprising a different number ofpatterns, or sequences where each individual pattern wasformed by neurons that in the previous sequence belongedto different patterns.

Notice that the magnitude of the inhibition required forthe model’s reset is not too small (which would render thestorage unreliable) nor too big (requiring very strong inhi-bition, for very long time to reset the system). This balancein the inhibition magnitude is not always present in othermodels where, in some cases, strong and long-lasting inhi-bition is necessary to reset the system (see for exampleFransen et al. 2002).

3.4 Robustness in parameter space

Globally the dynamics of the full model are stable to smallchanges in the parameters. As expected, the parameters cap-turing the highest variability in the main features of the modelare the ones associated with the CAN current. Changing thetime constant and/or the kinetics of the CAN current producesmodifications in the total time that gate interneurons are ableto self-sustain activity. However it should be pointed that thegate interneurons electrophysiological model, despite its sim-plicity in the number of currents, is rich in compensatorymechanisms. Larger changes in the CAN current can becompensated, for example, with changes in the CaL current.We therefore argue that, while larger changes in isolatedparameters can severely affect the performance of the model,if these changes are counteracted by other modifications underthe assumption of regulatory homeostatic mechanisms(Grashow et al. 2010; Prinz et al. 2004), the model is robustat maintaining the same qualitative results. In other words, theparameter space of this model contains large regions capableof producing the same qualitative results. For example, thesame qualitative results are obtained after an increase of100 % in gCAN if this change is compensated with a combineddecrease of 38 % in pCaL and 4.5 % in gL.

3.5 Model scaling

The size of this model (total number of neurons) growsquadratically with the size of the principal neurons popula-tion. Let NP be the size of the population P and NG thenumber of gate interneurons in each local cluster. If γ is the

Fig. 7 Storage and recall temporal constraints. a The system is able toretain the information up to the point where the reactivation intervalbecomes larger than 4.5 s. b Roughly 6 s are required to reset thesystem and allow it to store a different sequence. c If inhibitorysynapses block the activity in all gate interneurons, the system canreset in less time. d The activity on the gate interneurons encodes theserial order information. Here is shown the membrane potential ofthree gate interneurons belonging to the cluster of principal neuron#6 (part of the 2nd pattern): the sequential activation of neurons inpattern 1 and in pattern 2 (1→2); the co-activation of two neurons inpattern 2 (2→2); and the sequential activation of neurons in pattern 3and in pattern 2 (3→2)

J Comput Neurosci (2013) 35:187–199 195

gate sampling fraction, each cluster contains γ(NP – 1) = NG

gate interneurons, and the total size of the model is NP × (2 +NG) = γNP

2 + (2 – γ)NP. For large P populations and highgate sampling fractions there is therefore a large discrepancybetween the sizes of P population and G population. Toensure a small difference between the orders of magnitudeof P and G population sizes, while preserving high gatesampling fractions, the size of the principal populationshould fall within the scale of a couple hundred independentunits. This is the scale compatible with cortical columns.

When the P population size is large, it is possible toreduce the gate sampling fraction. This reduction, whileretaining a stable memory system behavior, can beattained as long as the following bounds are held: theinput current to each principal neuron provided by theavailable gate interneurons must be i) strong enough tomake sure the expected evoked responses are producedand ii) weak enough to avoid spurious activations. Giv-en these bounds it is possible to assess, for a specificprincipal population size, the minimal number of gateinterneurons per local cluster to ensure the proper func-tional behavior of the memory system. In other words,what is the minimal gate sampling fraction that guaran-tees correct storage/recall. Gate sampling connectionsare assumed to be purely random (i.e. no topographicconnections).

Consider the principal neuron Pi, which is engaged in oneof the sequence patterns (other than the first). If there are NG

gate interneurons projecting to Pi, after the storage period,only some will be active: the ones encoding the activitypattern where Pi is engaged and the ones encoding theactivity of the previous pattern in the sequence. In total, ifeach pattern is composed of M neurons, there are 2×M-1active gate interneurons projecting synapses to Pi, if thegates sampling fraction is 1. Each of these gate interneuronsare firing continuously at a rate of about 12 Hz, exceptduring the period in which the pattern is reactivated, wherethe rates exceed 20 Hz. So, in order to avoid that Pireactivates due to its gate interneurons inputs (as they fireat 12 Hz), but force Pi to reactivate when its gate interneu-rons increase their firing rates (slightly above 20 Hz), thesum of all synaptic inputs arriving on a short time window(membrane’s time constant) must be within a controledinterval. For our model parameters, this interval is betweento 1.02×10−3 and 1.22×10−3 μS. This is the core constrainsetting the memory system performance and capacity. Fix-ing the pattern size M in the beginning, the number of gateinterneurons engaged in encoding the patterns sequencefollows the hypergeometric distribution

X≡HyperGeometric Np−1; 2�M−1;NG

� �

with mean value

E X½ � ¼ 2�M−1ð Þ � NG

Np−1:

Therefore, to obtain a functional and robust workingmemory system, the synaptic strengths between gate inter-neurons and principal neurons must be weighted by E[X] inorder to obtain a total synaptic input that falls within the pre-established interval. For γ<1, and as a result of the randomconnectivity, there is some variability in the number of inputeach principal neuron receives from active gate interneu-rons. While in larger P populations these fluctuations areless relevant, for the scale at which this model is intended –cortical columns, where the number of independent princi-pal units is on the order of 100 – the gate sampling fractionshould be above 0.9. This is accordance with cortical col-umns size and connectivity (Buxhoeveden and Casanova2002). This working memory model for serial order doesnot scale well for larger numbers of independent principalunits because the network architecture is prepared to storeevery possible pattern (with a fixed number of active units).This provides high flexibility in encoding novel informa-tion, but leads to a high discrepancy between the sizes of Pand G populations when the size of the former grows be-yond a couple hundred independent units. There is conse-quently a balance between the flexibility to store moreinformation (encoded in the complexity of the individualpatterns of activity) and the discrepancy between the sizes ofP and G populations.

The gate interneurons framework can nevertheless beextended to large networks, with better scaling properties,if the ability to encode novel information is reduced.

3.6 Modified model with linear scaling properties

A linear relation between total principal neurons and gateinterneurons can be attained in a slightly modified architec-ture, but also relying on our gate interneurons hypothesis tocreate a short-term memory system capable of storing order.This modified architecture is less powerful in encodinginformation, but is adequate to larger principal populationswhere each independent principal unit is associated with asmall number of excitatory interneurons (the gate interneu-rons cluster).

Consider, as before, a population of principal neurons Pconnected with a population of gate interneurons G, butwhere each gate interneuron cluster is composed by a smallnumber n of gates, independent of the size of population P.On the limit, n can be 1. Each gate interneuron receives Cc

long-range random connections from the principal popula-tion (Cc >> 1), and establishes a connection with the localprincipal neuron. An external population controls directlythe activity on the gate interneurons. As before, synaptic

196 J Comput Neurosci (2013) 35:187–199

efficacies are parameterized so that activity can only betransferred between two principal neurons through a gateinterneuron if that gate interneuron is activated.

Depending on the specific pattern of activity tempo-rarily stored in the gate interneurons population,completely different sequences of activations will beformed in the principal population. The gate interneu-rons population is therefore used to temporarily storerouting information, and drive different sequences ofactivations on the principal population. Different con-stellations of active gates mean different sequences. InFig. 8 we present the results of a simulation with 20 Pneurons and 20 G neurons. This routing model onlydiffers from the previous WM model for serial orderin the above mentioned modifications in the connectionswith the gate interneuron cluster, and in the fact that theinformation is stored directly on the G population (asopposed as in the P population in the WM model).

In this modified architecture, the positive feed-backwhich may lead to increasing levels of activity is solvedwith an inhibitory activity level control mechanism.

While not part of this modified model, complementaryplastic mechanisms can be used to modify the connectionsof the network, selecting and shaping the activation se-quences in the P population which are associated withspecific active constellations in the G population.

4 Concluding remarks

We have shown that the modulation of the intrinsic excit-ability properties of neurons can be effectively used forproducing short-term memory systems capable of storingsequences of patterns. The membrane dynamics in our neu-ronal models are biologically plausible and use a reducednumber of currents (two in the principal and inhibitory in-terneurons, four in the gate interneurons population).

Importantly, cells fitting the properties required by ourgate interneurons hypothesis can be found in areas relatedwith working memory. This is the case of spiny stellate cellswhich are excitatory interneurons involved in recurrent ex-citation in neocortical circuits (Feldmeyer et al. 1999;Thomson et al. 2002). The features of the network architec-ture assumed in this model are thus attuned with what can befound in cortical networks. Moreover it is worth mentioningthat gate-like excitatory interneurons can also be found inthe hippocampal formation, a structure which is similarlyinvolved in sequence learning and short-term memorymechanisms (Suh et al. 2011): bi-directional connectivitybetween principal neurons and excitatory interneurons arereported between granule cells and mossy cells in the den-tate gyrus (Jackson and Scharfman 1996). The existence ofsynergetic currents capable of prolonging self-sustained ac-tivity in these excitatory interneurons (e.g. CAN and CaLcurrents) remains however a hypothesis of our model (butsee Nagy et al. 2012).

Our working memory model capable of storing se-quences has important properties that are in agreement withknown features of short-term memory for serial order. First,information can be stored upon a single exposure to thestimuli (stimulation period of hundreds of milliseconds),compatible with the notion of one-shot memory. Second,information can be retained for periods of many seconds andfinally, memory reactivation/usage extends memory reten-tion. The fact that gate interneurons are not bistable hasimportant consequences, namely gate activity can be resetwith short-duration inhibitory currents with biophysicallyreasonable amplitudes. The capability of our model to en-code temporal sequences in the background activity of gateinterneurons after a short (transient) exposure to the stimuli,is in agreement with recent results regarding memory reten-tion in brain tissue in vitro (Hyde and Strowbridge 2012).

A key element in our model is to establish a pathwaybetween principal neurons which is mediated not by

Fig. 8 Gate interneurons can temporarily store different sequences ofactivations in the principal population. a Activation of the same prin-cipal neuron (#12, middle line second column) leads to differentactivation sequences (black or gray) depending on the constellationof gate interneurons that are presently active (black or gray, respec-tively). b, c Membrane potential of the P population neurons driven bythe two different patterns of activity temporarily stored in the Gpopulation (black, gray). The first neuron in both sequences (neuron#12) is externally activated

J Comput Neurosci (2013) 35:187–199 197

synapses but by a specific class of neurons - the gate in-terneurons - with the ability to control the flow of activa-tions (routing) in the principal neurons population. Theinformation temporarily stored in the gates specifies whichsub-populations will be activated and in which order. Thecombination of gate interneuron and the assumed architec-ture allows a versatile serial order storage-recall mechanism.Considering two units A and B, the model shows that anyconfiguration involving these two units is equally feasible: i)A followed by B; ii) B followed by A; iii) A and B co-activated; and iv) all other configurations where A and B areindependent or do not activate sequentially.

With the gate interneurons we are, in some sense, replac-ing synaptic storage for neuronal storage. It should be notedthat storing information in neuronal intrinsic membranedynamics:

– provides a robust storage mechanism (synapses in thecentral nervous system are often highly probabilistic);

– makes information available in a non-local manner -changing a synapse only modulates a target neuronwhile a single gate interneuron can be used to modulateseveral neurons (gate interneurons can “broadcast” thestored information to more targets)

– works reliably, and is less affected by noise, on timescales of a few hundred milliseconds, compatible withfast storage mechanisms (one-shot learning)

– provides continuous access to the stored information (inthe form of firing activity), while a synapse needs to beactivated to readout the stored information

We therefore argue that, while penalizing scalability,there are many contexts and scales where storing informa-tion in interneuron’s intrinsic membrane dynamics, insteadof in synapses, can lead to important functional advantages.

Acknowledgments Research funded by the European Regional De-velopment Fund through the programme COMPETE and by the Por-tuguese Government through the FCT - Fundação para a Ciência e aTecnologia under the project PEst-C/MAT/UI0144/2011. EduardoConde-Sousa was supported by the grant SFRH/BD/65633/2009 fromFCT, co-financed by European Social Fund under the program POPHof the National Strategic Reference Framework.

References

Abbott, L. F., & Nelson, S. B. (2000). Synaptic plasticity: taming thebeast. Nature Neuroscience, 3(Suppl), 1178–1183.

Aguiar, P., Sousa, M., & Lima, D. (2010). NMDA channels togetherwith L-type calcium currents and calcium-activated nonspecificcationic currents are sufficient to generate windup in WDR neu-rons. Journal of Neurophysiology, 104, 1155–1166.

Alonso, A., de Curtis, M., & Llinas, R. (1990). Postsynaptic Hebbianand non-Hebbian long-term potentiation of synaptic efficacy in

the entorhinal cortex in slices and in the isolated adult guinea pigbrain. Proceedings of the National Academy of Sciences of theUnited States of America, 87, 9280–9284.

Baddeley, A. (2003). Working memory: looking back and lookingforward. Nature Reviews Neuroscience, 4, 829–839.

Bal, T., & McCormick, D. A. (1993). Mechanisms of oscillatoryactivity in guinea-pig nucleus reticularis thalami in vitro: a mam-malian pacemaker. The Journal of Physiology, 468, 669–691.

Bliss, T. V., & Lomo, T. (1973). Long-lasting potentiation of synaptictransmission in the dentate area of the anaesthetized rabbit fol-lowing stimulation of the perforant path. The Journal of Physiol-ogy, 232, 331–356.

Botvinick, M. M., & Plaut, D. C. (2006). Short-term memory for serialorder: a recurrent neural network model. Psychology Review, 113,201–233.

Burnashev, N., Zhou, Z., Neher, E., & Sakmann, B. (1995). Fractionalcalcium currents through recombinant GluR channels of theNMDA, AMPA and kainate receptor subtypes. The Journal ofPhysiology, 485(Pt 2), 403–418.

Buxhoeveden, D. P., & Casanova, M. F. (2002). The minicolumnhypothesis in neuroscience. Brain, 125, 935–951.

Crasto, C. J., Marenco, L. N., Liu, N., Morse, T. M., Cheung, K. H.,Lai, P. C., et al. (2007). SenseLab: new developments in dissem-inating neuroscience information. Briefings in Bioinformatics, 8,150–162.

Dehaene, S., Changeux, J. P., & Nadal, J. P. (1987). Neural networksthat learn temporal sequences by selection. Proceedings of theNational Academy of Sciences of the United States of America,84, 2727–2731.

Destexhe, A., Babloyantz, A., & Sejnowski, T. J. (1993). Ionic mech-anisms for intrinsic slow oscillations in thalamic relay neurons.Biophysical Journal, 65, 1538–1552.

Destexhe, A., Contreras, D., Sejnowski, T. J., & Steriade, M. (1994). Amodel of spindle rhythmicity in the isolated thalamic reticularnucleus. Journal of Neurophysiology, 72, 803–818.

Durstewitz, D., Seamans, J. K., & Sejnowski, T. J. (2000).Neurocomputational models of working memory. Nature Neuro-science, 3(Suppl), 1184–1191.

Fall, C. P., & Rinzel, J. (2006). An intracellular Ca2+ subsystem as abiologically plausible source of intrinsic conditional bistability ina network model of working memory. Journal of ComputationalNeuroscience, 20, 97–107.

Feldmeyer, D., Egger, V., Lubke, J., & Sakmann, B. (1999). Reliablesynaptic connections between pairs of excitatory layer 4 neuroneswithin a single ‘barrel’ of developing rat somatosensory cortex.The Journal of Physiology, 521(Pt 1), 169–190.

Fransen, E., Alonso, A. A., & Hasselmo, M. E. (2002). Simulations ofthe role of the muscarinic-activated calcium-sensitive nonspecificcation current INCM in entorhinal neuronal activity during de-layed matching tasks. Journal of Neuroscience, 22, 1081–1097.

Fransen, E., Tahvildari, B., Egorov, A. V., Hasselmo, M. E., & Alonso,A. A. (2006). Mechanism of graded persistent cellular activity ofentorhinal cortex layer v neurons. Neuron, 49, 735–746.

Gibson,W. G., & Robinson, J. (1992). Statistical analysis of the dynamicsof a sparse associative memory. Neural Networks, 5, 645–661.

Grashow, R., Brookings, T., & Marder, E. (2010). Compensation forvariable intrinsic neuronal excitability by circuit-synaptic interac-tions. Journal of Neuroscience, 30, 9145–9156.

Hahn, T. T., McFarland, J. M., Berberich, S., Sakmann, B., & Mehta,M. R. (2012). Spontaneous persistent activity in entorhinal cortexmodulates cortico-hippocampal interaction in vivo. Nature Neu-roscience, 15, 1531–1538.

Haj-Dahmane, S., & Andrade, R. (1999). Muscarinic receptors regulatetwo different calcium-dependent non-selective cation currents inrat prefrontal cortex. European Journal of Neuroscience, 11,1973–1980.

198 J Comput Neurosci (2013) 35:187–199

Herz, A. V., Li, Z., & van Hemmen, J. L. (1991). Statistical mechanicsof temporal association in neural networks with transmission de-lays. Physical Review Letters, 66, 1370–1373.

Hines, M. L., & Carnevale, N. T. (1997). The NEURON simulationenvironment. Neural Computation, 9, 1179–1209.

Hodgkin, A. L., & Huxley, A. F. (1952). A quantitative description ofmembrane current and its application to conduction and excitationin nerve. The Journal of Physiology, 117, 500–544.

Hyde, R. A., & Strowbridge, B. W. (2012). Mnemonic representationsof transient stimuli and temporal sequences in the rodent hippo-campus in vitro. Nature Neuroscience, 15, 1430–1438.

Jackson, M. B., & Scharfman, H. E. (1996). Positive feedback fromhilar mossy cells to granule cells in the dentate gyrus revealed byvoltage-sensitive dye and microelectrode recording. Journal ofNeurophysiology, 76, 601–616.

Jahr, C. E., & Stevens, C. F. (1990). Voltage dependence of NMDA-activated macroscopic conductances predicted by single-channelkinetics. Journal of Neuroscience, 10, 3178–3182.

Jensen, O., Idiart, M. A., & Lisman, J. E. (1996). Physiologicallyrealistic formation of autoassociative memory in networks withtheta/gamma oscillations: role of fast NMDA channels. Learningand Memory, 3, 243–256.

Kandel, E. R. (2001). The molecular biology of memory storage: adialogue between genes and synapses. Science, 294, 1030–1038.

Klink, R., & Alonso, A. (1997). Morphological characteristics of layerII projection neurons in the rat medial entorhinal cortex. Hippo-campus, 7, 571–583.

Koene, R. A., & Hasselmo, M. E. (2007). First-in-first-out item re-placement in a model of short-term memory based on persistentspiking. Cerebral Cortex, 17, 1766–1781.

Koene, R. A., & Hasselmo, M. E. (2008). Consequences of parameterdifferences in a model of short-term persistent spiking buffersprovided by pyramidal cells in entorhinal cortex. Brain Research,1202, 54–67.

Lisman, J. E., & Idiart, M. A. (1995). Storage of 7 +/− 2 short-termmemories in oscillatory subcycles. Science, 267, 1512–1515.

Marder, E., Abbott, L. F., Turrigiano, G. G., Liu, Z., &Golowasch, J. (1996). Memory from the dynamics of intrin-sic membrane currents. Proceedings of the National Academyof Sciences of the United States of America, 93, 13481–13486.

McCormick, D. A., & Huguenard, J. R. (1992). A model of theelectrophysiological properties of thalamocortical relay neurons.Journal of Neurophysiology, 68, 1384–1400.

Mozzachiodi, R., & Byrne, J. H. (2010). More than synaptic plasticity:role of nonsynaptic plasticity in learning and memory. Trends inNeurosciences, 33, 17–26.

Nagy, G. A., Botond, G., Borhegyi, Z., Plummer, N. W., Freund, T. F.,& Hajos, N. (2012). DAG-sensitive and Ca(2+) permeableTRPC6 channels are expressed in dentate granule cells and in-terneurons in the hippocampal formation. Hippocampus.

Otis, T. S., De Koninck, Y., & Mody, I. (1993). Characterization ofsynaptically elicited GABAB responses using patch-clamp re-cordings in rat hippocampal slices. The Journal of Physiology,463, 391–407.

Prinz, A. A., Bucher, D., & Marder, E. (2004). Similar network activityfrom disparate circuit parameters. Nature Neuroscience, 7, 1345–1352.

Suh, J., Rivest, A. J., Nakashiba, T., Tominaga, T., & Tonegawa, S.(2011). Entorhinal cortex layer III input to the hippocampus iscrucial for temporal association memory. Science, 334, 1415–1420.

Thomson, A. M., West, D. C., Wang, Y., & Bannister, A. P. (2002).Synaptic connections and small circuits involving excitatory andinhibitory neurons in layers 2–5 of adult rat and cat neocortex:triple intracellular recordings and biocytin labelling in vitro. Ce-rebral Cortex, 12, 936–953.

Traub, R. D., & Miles, R. (1991). Neuronal networks of the hippo-campus. Cambridge: CUP.

White, O. L., Lee, D. D., & Sompolinsky, H. (2004). Short-termmemory in orthogonal neural networks. Physical Review Letters,92, 148102.

J Comput Neurosci (2013) 35:187–199 199