Abstract. Stimulus representation is a functional inter-pretation of early sensory cortices. Early sensory corticesare subject to stimulus-induced modi®cations. Commonmodels for stimulus-induced learning within topographicrepresentations are based on the stimuli's spatial structureand probability distribution. Furthermore, we argue thataverage temporal stimulus distances re¯ect the stimuli'srelatedness. As topographic representations re¯ect thestimuli's relatedness, the temporal structure of incomingstimuli is important for the learning in cortical maps.Motivated by recent neurobiological ®ndings, we presentan approach of cortical self-organization that addition-ally takes temporal stimulus aspects into account. Theproposed model transforms average interstimulus inter-vals into representational distances. Thereby, neuraltopography is related to stimulus dynamics. This o�ersa new time-based interpretation of cortical maps. Ourapproach is based on a wave-like spread of corticalactivity. Interactions between dynamics and feedforwardactivations lead to shifts of neural activity. The psycho-physical saltation phenomenon may represent an ana-logue to the shifts proposed here. With regard to corticalplasticity, we o�er an explanation for neurobiological®ndings that other models cannot explain. Moreover, wepredict cortical reorganizations under new experimental,spatiotemporal conditions. With regard to psychophys-ics, we relate the saltation phenomenon to dynamics andinteraction in early sensory cortices and predict furthere�ects in the perception of spatiotemporal stimuli.

1 Introduction

A functional interpretation of early sensory cortices isstimulus representation (Marshall et al. 1937; Merzenichet al. 1978). Stimuli are grouped to form representa-tional units that are topographically represented in so-

called cortical maps. The homunculus of the primarysomatosensory cortex, retinotopy and orientation mapsof the primary visual cortex, and frequency maps of theprimary auditory cortex constitute well-known and well-studied examples of such topographic representations.

An increasing number of experiments reveal theplasticity of such maps (Jenkins et al. 1990; Allard et al.1991; Garraghty and Kaas 1992). From a theoreticalpoint of view, neural maps seem to be the result of self-organizing processes (von der Malsburg 1973), leadingto an equilibrium that is based on the statistics of in-coming stimuli (Kohonen 1982; Obermayer et al. 1992;Joublin et al. 1996).

Furthermore, cortical maps may be understood in adimension reduction framework (Durbin and Mitchison1990). Certain neighborhood relations are preserved bythese mappings of high-dimensional stimulus spacesonto the cortex. As a result of this preservation, localcomputations in stimulus space can also be performedlocally in the cortex. In this context, the essential ques-tion arises according to which criteria neighborhoodrelations are mapped onto the cortex.

Common models that simulate the formation andalteration of cortical maps consider only the spatialstructure and the probability distribution of the stimuli(von der Malsburg 1973; Willshaw and von der Mals-burg 1976; Kohonen 1995). In these models stimuli aredrawn randomly, i.e. their temporal context is neglected.The resulting topographic structures depend on thecoding of the stimuli and receptive ®elds (RFs) (Erwinet al. 1995; Mayer et al. 1998; Wiemer et al. 1999).

Contrarily, we regard stimulus dynamics, i.e. thetemporal course of incoming stimuli, to be of funda-mental importance for the formation, alteration, andinterpretation of cortical representations. Our point ofview is based on the following arguments.

1. Recent neurobiological experiments concerningplasticity in the primary somatosensory cortex of adultmonkeys demonstrate that synchronous stimuli are in-tegrated (i.e. represented at one cortical location),whereas asynchronous stimuli [200±300 ms interstimu-lus interval (ISI)] are cortically segregated (i.e. repre-

Correspondence to: J. Wiemere-mail: jan.wiemer@neuroinformatik.ruhr-uni-bochum.de

Biol. Cybern. 82, 173±187 (2000)

Learning cortical topography from spatiotemporal stimuli

J. Wiemer, F. Spengler, F. Joublin, P. Stagge, S. Wacquant

Institut fuÈ r Neuroinformatik, Ruhr-UniversitaÈ t Bochum, D-44780 Bochum, Germany

Received: 17 March 1999 /Accepted in revised form: 10 August 1999

sented at distant cortical locations; Wang et al. 1995;Spengler et al. 1996; Spengler et al. 1999). Here, the ISIplays a key role in guiding the processes of cortical re-organization (cf. Sect. 2). Conventional models of self-organization (e.g. Kohonen 1995) cannot explain theseresults (see Sect. 6.1 and Wiemer et al. 1998a,b).

2. Psychophysical experiments demonstrate system-atic mislocalizations of localized somatosensory, visual,and auditory stimuli when their temporal distances lie inthe range of up to several hundred milliseconds (Geldard1982; Kilgard and Merzenich 1995; Cholewiak 1999).This phenomenon, called saltation, illustrates the con-tinuous transformation of temporal distances intoperceived spatial distances (cf. Sect. 5). The perceivedspatial distance between two stimuli is composed of theirreal spatial distance and an ISI-dependent variationthereof. Small (large) ISIs lead to small (large) perceiveddistances.

3. An important aspect, which determines corticalrepresentations, is the correlation between sensory andcortical dynamics (Garraghty and Kaas 1992), i.e.Hebbian learning (Brown et al. 1990; Weinberger 1996).However, sensory and cortical dynamics are not tem-porally separable, as assumed by previous models (vonder Malsburg 1973; Kohonen 1995). Neural responses inthe primary somatosensory and auditory cortex of themonkey for example, do not only depend on the cur-rently applied stimulus but are additionally in¯uencedby earlier stimulations. Depending on the spatiotempo-ral (spectro-temporal) relation, such responses are in-hibited or enhanced, even for ISIs up to 300 ms(Gardner and Costanzo 1980; Brosch et al. 1998).

4. Natural sensory stimulation develops continuouslyin time. Therefore, the temporal proximity of the in-coming stimuli yields a metric that re¯ects the stimuli'srelatedness and functional similarity. We argue thatcortical representations could be formed and altered inaccordance with such a time-based metric. Thereby,stimuli that follow each other closely in time can be re-garded as belonging together and should be representedtopographically close to each other (Wiemer et al. 1998a).

We argue that the cortical reorganization observed bySpengler et al. (1996, 1999) and also by Wang et al.(1995) is the result of a self-organizing process based onspatiotemporal stimuli (Buonomano and Merzenich1995). The experimental ®ndings and their interpretationare presented in Sect. 2. A model of self-organizationthat explains the observed cortical segregation of exper-imental stimuli is proposed in Sect. 3 and Sect. 4. Ittransforms temporal distances of the applied stimuli intospatial distances of the corresponding neural represen-tations. These time-to-space transformations areachieved by a fundamental wave-like neural dynamics.This dynamics leads to shifts of neural stimulus responsesthat depend on the stimuli's spatiotemporal context.

We speculate that such shifts contribute to the psy-chophysical saltation phenomenon. Our approach con-stitutes a framework in which this illusion can beexplained from dynamics and interactions in primarysensory cortices. This is elaborated in Sect. 5. Our viewis in accordance with the experimental ®nding that

neural responses in the primary somatosensory and au-ditory cortex depend on stimuli going back up to severalhundred milliseconds.

Furthermore, we predict features of cortical reorga-nization and psychophysical percepts under new exper-imental, spatiotemporal conditions (Sect. 4.3 andSect. 5.2, respectively).

With regard to cortical plasticity, our approach raisesthe following questions. To what extent does the for-mation of cortical topography rely on the temporalstructure of the incoming stimuli? Does the transfor-mation of temporal stimulus distances into cortical to-pography constitute a general principle of cortical self-organization? Furthermore, may this principle lead to abetter understanding of the structure of early as well asof higher cortical areas, e.g. of the inferotemporal cortex(Tanaka et al. 1991; Wang et al. 1996)?

Our approach is discussed in Sect. 6. We emphasizefar-reaching implications (time-based interpretation ofcortical topography, in¯uence on cortical reorganiza-tion, psychophysical saltation), indicate biologicalplausibility, and suggest extensions (more general stimuliand neural responses, learning of lateral connections).Finally, we summarize and present an outlook in Sect. 7.

2 Neurobiological experiment

Many experiments on cortical plasticity can be under-stood in the context of self-organization based onstationary neural stimulus responses and Hebbianlearning. In this section, we present neurobiological®ndings that motivate an extension of the self-organiz-ing processes to include the stimuli's temporal structureas well.

2.1 Experimental paradigm

Spengler and colleagues (1996, 1999) trained monkeyson a tactile two-stimuli discrimination task for 5±7months. In daily sessions of several hours, two vibratingbars (sinusoidal vibrations at 75 Hz, 80 ms duration and100 lm amplitude) were applied to several ®ngersegments. One of the bars stimulated synchronouslythe distal segments of the second, third, and fourth®nger (denoted `bar A'), the other stimulated synchro-nously the proximal, middle, and partly the distalsegment of the third ®nger (`bar B'). The two experi-mental stimuli overlapped slightly on the distal segmentof the third ®nger (see the sketch of the experimental set-up in Fig. 1a). In the task, the two stimuli werealternately applied with an ISI of 300 ms serving asbackground stimuli. As target stimuli the monkeys hadto detect two consecutive stimuli of either bar A or bar Band retract their hand from the stimulator. The handretraction was counted as (1) a `hit' when it occurredwithin the given reaction time window, (2) a `miss' whenit was beyond the end of the reaction time window, and(3) a `false positive' when it preceded the presentation ofthe target stimuli (Fig. 1a). Hits were rewarded with

174

food pellets, misses and false positives were followed bycage light time out.

2.2 Experimental results

The resulting hand representations in the primarysomatosensory cortex (layer IV, area 3b) of the controland experimental hemisphere are shown in Fig. 1b andc, respectively. They were obtained by electrophysiolog-ical mapping in the anesthetized monkey after successfulcompletion of the training (Wang et al. 1995; Spengleret al. 1996, 1999).

The hand representation of the control hemisphereseems to be unaltered by experimental training. Fig-ure 1b shows an orderly progression of cutaneous RFlocations across lateral-to-medial (digit 1 to digit 5) andcaudal-to-rostral (proximal to distal) surfaces of thehand digits. Borders between cortical representations ofadjacent ®ngers and ®nger segments are distinct. In ad-dition to the representation of glabrous digital skinsurfaces, small islands of RFs located on the handdorsum are scattered in the map. The representation ofdorsal digits is fragmented and incomplete as in naivemonkeys (Merzenich et al. 1987).

In order to characterize the observed cortical reor-ganizations, we introduce the following terms: (1) rep-resentational distance is the spatial distance betweencortical activities induced by two stimuli measured incortical coordinates or in `parameter space' as, e.g. de-®ned in population coding (Jancke et al. 1996); (2) in-tegration is the fusion of di�erent stimuli into onerepresentation, or the reduction of their representational

distance; (3) segregation is the process of increasingrepresentational distance.

In contrast to the control hemisphere, the hand rep-resentation of the experimental hemisphere reveals ex-tensive stimulus-speci®c reorganization.

1. Integration. In addition to single segment represen-tation, some neurons have attained RFs extending overthose ®nger segments that were synchronously stimulatedin the experiment. The corresponding cortical regionswhere such neurons were found are marked as `bar A' ±(`bar B') speci®c multiple segment RFs in Fig. 1c. Con-cerning bar B, RFs extend over multiple segments of one®nger (i.e. digit 3), concerning bar A, RFs even extendover multiple segments of di�erent ®ngers.

2. Segregation. Although the experimental stimulioverlap spatially on the distal segment of the third ®n-ger, the corresponding cortical representations are notadjacent to one another but segregated into di�erentregions of the hand representation. In the example inFig. 1c, the segregation is revealed by a dorsal inputband, i.e. a cortical region where the neurons' RFs arelocated on the back of the hand. The dorsal input bandlies between the experimental stimuli's representations.It may result from unmasking of non-dominant dorsala�erents in the glabrous hand surface representation (seeSchroeder et al. 1997).

In another experiment, Wang et al. (1995) used asimilar paradigm consisting of two parallel tactile bars,leading to similar results. In order to evaluate the originof the neuronal changes, Wang et al. (1995) analyzedventro-posterior thalamus response maps. They revealedno equivalent reorganization. Therefore, the represen-tational plasticity appears to be of cortical origin.

Fig. 1a±c. Neurobiological experiment. Monkeys were trained to discriminate two spatially overlapping tactile stimuli (`bar A', `bar B') that werealternately applied and temporally separated (interstimulus interval=300 ms) (a). The monkeys' task was to detect two consecutive stimuli ofeither bar A or bar B. After training, the animals' hand representations of the primary somatosensory cortex (layer IV, area 3b) were mappedelectrophysiologically. A usual topographic representation with exclusively single segment receptive ®elds (RFs) is found on the controlhemisphere (b). The experimental hemisphere reveals stimulus speci®c reorganizations (c). Synchronously stimulated ®nger segments areintegrated while representations of the two experimental stimuli are segregated into di�erent regions of the hand representation. The maps shownin b and c are reconstructed from a total of 230 and 231 microelectrode penetration sites, respectively

175

3 The model

A possible approach to interpret the presented experi-mental ®ndings lies in the assumption that knowledgeabout the trained task can in¯uence the topographicstructure of the primary somatosensory cortex. Top-down processes could evoke task-speci®c changes ofcortical activities. Changed neural activities and corre-lation-based learning could lead to the observed topo-graphic reorganizations. However, little is known aboutthe top-down in¯uence which higher cortical processesexert on primary cortical areas.

On the other hand, the generalization of stimulus-induced learning from purely spatial stimuli to spatio-temporal stimuli is a natural step. Therefore, we addressthe question whether the above-stated experimental®ndings can be explained by self-organizing processesthat are based on spatiotemporal stimuli.

Integration of synchronous stimuli can be understoodas a direct consequence of Hebbian learning. The cor-tical segregation phenomenon is not as easily explained.It requires additional mechanisms, although not neces-sarily non-Hebbian learning. Our explanation in thissection and in Sect. 4 is based on Hebbian learning andcortical dynamics.

3.1 Spatiotemporal stimuli

We believe that the dynamics of incoming stimuli re¯ectsthe stimuli's relatedness with regard to their functional

meaning, and that stimulus dynamics is thereforeimportant for the learning of topography. Accordingly,we model the ¯ux of incoming stimuli as a sequence ofstimuli,

s1; s2; s3; . . . ; sn; sn�1; . . . ; �1�in which the temporal proximity of consecutive stimuli isexpressed in a corresponding sequence of ISIs,

isi1; isi2; . . . ; isin; . . . : �2�The presentation time tn of a stimulus sn is given by

tn �Xnÿ1n0�1

isin0 �t1 � 0� : �3�

We remark that this leads to an inhomogenous discret-ization of time.

The stimuli's spatial structure is high-dimensionallycoded; stimuli are activity patterns over an array ofsensors (see Erwin et al. 1995; Riesenhuber et al. 1996;Wiemer et al. 1999).

3.2 Network architecture

The model consists of three two-dimensional layers(Fig. 2). The ®rst layer is an array of N s

1 � N s2 sensors (`s'

for sensors). High-dimensionally coded stimuli sn areapplied to this layer at discrete times tn. The second layerconsists of N s

1 � N s2 neural units and possesses additional

Fig. 2. Network architecture and generation of layer-3response. The model consists of three two-dimensionallayers: a sensory array to which arbitrary shaped stimulican be applied, a second layer that temporally integratesthe activity of the sensory array, and a third layerconsisting of units with plastic connections that bringforth topography. Every sensory unit (i; j) is topograph-ically connected to the corresponding unit (i; j) of thesecond layer; the weights of these connections are ®xedand set to unity. Each layer-3 unit (k; l) receives inputvia connection weights wkl;ij from all layer-2 units. Inthis framework, a layer-3 response is generated asfollows. At time tn the stimulus sA has evoked a layer-3response cA. An interstimulus interval isin later adi�erent stimulus sB is applied and leads to a layer-2activity that consists of a strong sB component, inaddition to a more or less attenuated sA component.Due to neural dynamics (not shown in this ®gure, seeFig. 3) and interaction, the resulting feedforwardactivation is sharpened (c0B) and shifted to form alayer-3 activity cB that depends on the applied ISI isin

176

dynamic properties. Every unit �i; j� of this layer isconnected to the corresponding sensor �i; j� via ®xedconnections that are set to unity. Each of the N c

1 � N c2

(`c' for cortical) neural units of the third layer isconnected to all units of the second layer. The corre-sponding connection matrix W�tn� consists of elementswkl;ij�tn� that determine how strongly layer-2 neurons�i; j� can activate layer-3 neurons �k; l� via feedforwardconnections at time tn. We denote the weight vector wklthat speci®es the selectivity of neuron �k; l� as the RF ofthat neuron. The weight vectors are learned during theprocess of self-organization and determine the resultingtopographic structure.

RFs and stimuli are coded in the same way. Usinghigh-dimensional coding, we can interpret elements ofthe connection matrix biologically as e�ective strengthsof connections between sensors and cortical neurons. Inaddition, we can model stimulus-induced changes in thespatial RF shape that are experimentally observed(multiple segment RFs, cf. Sect. 2.2). Finally, high-di-mensional coding will facilitate the transfer of our modelto other ®elds of signal processing, e.g. the processing ofvisual and auditory stimuli.

3.3 Neural activity and dynamics

Activities in layers 2 and 3 are subject to dynamics. Attimes of stimulus presentation tn, the sensory activitys�tn� is fed into layer 2. Between two stimulus presenta-tions at times tn and tn�1, layer-2 activity p�tn� decays(Fig. 2). This type of dynamics leads to a time-depen-dent averaging over successive incoming stimuli.

As a consequence of the dynamics, the layer-2 activityp�tn � isin� approaches continuously the superposition ofthe presented stimuli, s�tn� � s�tn�1�, when the ISIisin � tn�1 ÿ tn is reduced to zero. The functional rele-vance of this layer's dynamics lies in an interpolationbetween successive stimuli (Wallis and BuÈ ltho� 1999).Thereby, stimuli that follow each other closely in time are

associated. In the framework of Hebbian learning, thiscan lead to the formation of invariant object represen-tation (see Edelman and Weinshall 1991; Wallis 1996).

Activity is fed from layer 2 to layer 3 at the times ofstimulus presentation. As usual, we obtain the feedfor-ward activation of layer 3, cff�tn�, by multiplying theconnection matrix W�tn� with the layer-2 activity p�tn�:cff�tn� �W�tn� � p�tn� : �4�The connection strengths W�tn� result from a learningprocess after nÿ 1 adaptation steps at times t1; . . ., tnÿ1(cf. Sect. 4).

The layer-3 activity c�tn� builds up from the currentfeedforward activation cff�tn�, and the activity state ofthis layer, that has evolved out of the earlier activityc�tnÿ1� from time tnÿ1 to tn. The evolution of layer-3 ac-tivity between two stimulations is of wave-like type; ex-citation propagates into its neural surround. Thisdynamics may result from local interactions, e.g. betweenexcitatory and inhibitory neurons (Wilson and Cowan1973). The dynamics is fundamental in the sense of ageneral principle of locality; information can only besubmitted with ®nite speed. In other words, ``e�ectspropagate from point to neighboring point'' (Haag 1993).

In the case of a monomodal and rotation symmetriclayer-3 activity, an `elementary wave' propagates in twodimensions as shown in Fig. 3. Only idealized responsesof this type will be used in our simulations (cf. Sect. 4).Assuming linear superposition, the dynamics corre-sponding to general activity patterns can be reduced tothe superposition of elementary waves. As an analogue,one may think of water waves that a raindrop generateswhen it falls into a puddle.1

The functional relevance of layer-3 dynamics lies inthe transformation of temporal coding into spatial

Fig. 3. Neural dynamics causing a shift ofactivity. Two stimuli sA, sB are consecutivelyapplied with a given interstimulus interval (ISI)cf. Fig. 2. At time tn � isin, the layer-3 activitycA�tn�, generated by stimulus sA�tn� at time tn,has propagated into its surround. The stimulussB�tn � isin� (applied at that time) generates afeedforward activation that is sharpened byrecurrent inhibition (denoted c0B) and shiftedtowards the prevailing wave front c0A to formthe corresponding layer-3 response cB�tn � isin�(see main text for details)

1Water waves can reveal non-linear dynamics; they do not gen-erally obey Huygens's principle of linear superposition (Mehaute1976). We refer to water waves as a vivid example to illustrate thesimplicity of this type of dynamics

177

coding. In combination with feedforward activation, thewave re¯ects a spatiotemporal stimulus structure. Itdeforms feedforward activations and leads to layer-3activities that incorporate spatiotemporal stimulus cor-relations.

3.4 Shifts of activation

A layer-3 response c�tn� is obtained by assuming thefollowing principle of interaction: the feedforwardactivation cff�tn� is sharpened by local recurrentinhibition (von der Malsburg 1973; Wilson and Cowan1973; Amari 1980) and shifted towards the currentwave front of this layer's dynamics (Fig. 3). In a neural®eld model, and also in biological systems, these shiftvectors can result from an asymmetry in neuralexcitability due to the dynamical state of the layer.The lengths of these shifts depend on the distancesbetween the wave front and the center of feedforwardactivation. These distances result from the spatialdistance between stimulus representations, the wavevelocity, and the ISI used. The shift may also dependexplicitly on the ISI re¯ecting the wave's time-depen-dent attenuation. An example for these interactions willbe presented in Sect. 4.1.

The interaction described above leads to two oppos-ing situations that we denote integration and segrega-tion. When the ISI between the two stimuli sA � s�tnÿ1�,sB � s�tn� is small, the wave induced by sA is still locatednear its initial response. In this case, a shift of feedfor-ward activation towards the wave front implies a de-crease in representational distance, i.e. integration(Fig. 4a). On the other hand, when the ISI between thetwo stimuli sA, sB is large, the wave has already passedthe cortical location of maximal feedforward activationinduced by sB (Fig. 4b). Therefore, a shift towards thewave front leads to an increase of the representationaldistance, i.e. segregation.

3.5 Adaptation of weights

In our model, topography is learned by the adaptationof the e�ective synaptic weights wkl;ij from the second to

the third layer. We assume Hebbian learning, i.e. weightsare adapted as a function of the correlation betweenpresynaptic and postsynaptic activity. Accordingly, weapply normalized Hebbian learning with layer-2 activityas the presynaptic component and layer-3 activity as thepostsynaptic component. In addition, a second learningrule is applied in order to incorporate forgetting. Theprocess is called homosynaptic depression: presynapticactivity in the absence of above-threshold postsynapticactivity causes synaptic depression (Brown et al. 1990;see also Sect. 4). Figure 5 illustrates the combined e�ectof postsynaptic shifts and homosynaptic depressionleading to a decay of synaptic weights.

4 Numerical experiment

The essential ideas of our approach were presented inthe previous section. We now put this ansatz intoconcrete terms (Sect. 4.1). Our ®rst application is the`ontogenetic' formation of topographic structure(Sect. 4.2). We then apply our model to the experiment

Fig. 4a,b. ISI dependence of integration and segregation. The layer-3 response to a stimulus sB is shifted according to its current state of dynamics.For small ISIs, the wave evoked by an earlier stimulus sA is located near its initial response. A shift of activation towards the wave front leads tointegration (a). In the case of large ISIs, the wave has already passed the location of maximal sB-induced feedforward activation. Therefore, a shifttowards the wave front means segregation (b)

Fig. 5. Illustration of unlearning. Presynaptic activity in the absence of(above-threshold) postsynaptic activity causes synaptic depression(`homosynaptic depression'). A shift of layer-3 activity exposes acortical region that is highly selective for the applied layer-2 activitypattern. Especially in this region, large synaptic weights are reducedleading to the development of new neural selectivities

178

by Spengler and colleagues (Sect. 4.3). Finally, bychanging the spatiotemporal structure of the experimen-tal stimuli, we are led to predictions of corticalreorganization under new experimental conditions.

4.1 A simple form of the proposed model

In our simulations, each of the three two-dimensionallayers possesses N � N units (N � 30). In the following,layers 1, 2, and 3 are named sensory, precortical, andcortical layer, respectively. However, layer 2 may alsorepresent an early cortical area, e.g. with regard to therepresentation of high-dimensional visual stimuli(Sect. 6.3).

As sensory activity patterns, we apply high-dimensionally coded Gauss-shaped stimuli s�tn� �sij�tn�� �

i;j2f1;:::;Ng,

sij�tn� � expÿ �iÿ in�22r2

s1

� �jÿ jn�22r2

s2

!; �5�

centered at y�tn� � �in; jn�T with widths rs1, rs2. At anytime tn, the last two stimuli s�tnÿ1�, s�tn� are temporallyintegrated to form the precortical activity

p�tn� � s�tn� � �1ÿ isinÿ1=sI�H�sI ÿ isinÿ1�s�tnÿ1� ; �6�with ISI isinÿ1 � tn ÿ tnÿ1, decay sI , and Heavisidefunction H(�) (H�x� � 0 for x � 0, H�x� � 1 for x > 0;sI � 4 in our simulations).

We assume cortical responses c�tn� � ckl�tn� to be ofGauss-shaped form

ckl�tn� � expÿ �k ÿ kn�2 � �lÿ ln�22r2

c

!; �7�

centered at locations x�tn� � �kn; ln�T with width rc.During the time interval �tnÿ1; tn�, the earlier responsec�tnÿ1� to the stimulation at time tnÿ1 propagates as aradially localized and symmetric wave with constantvelocity v � 1 into its surround (Fig. 3).2 At time tn, thelocation of the cortical response x�tn� to the stimuluss�tn� is computed by adding a shift D�tn� to the center ofcortical feedforward activation xff�tn� � �kffn ; lffn �T :x�tn� � xff�tn� � D�tn� : �8�We de®ne the center of cortical feedforward activationxff�tn� as the average position using feedforward activa-tions cffkl�tn� as weighting factors (see Eq. 4) and takingonly high activations cffkl�tn� > hff �maxkl cffkl�tn� intoaccount (hff � 0:8). The shift D�tn� points from xff�tn�towards the cortical wave front along the shortestconnecting path [endpoint xw�tn�]. The length of thispath determines the magnitude of the shift according toa non-linear function:

D�tn� � f jjxw�tn� ÿ xff�tn�jj� �

: �9�

We introduce two parameters that determine thisfunction; the maximum length of the shift, j, and itsdecay constant for large distances, mj. The shift length isproportional to the distance between wave and feedfor-ward activation for distances that are smaller than j.For larger distances, it decreases exponentially:

f �x� � x : 0 � x � jj exp ÿ�xÿ j�=�mjj�� � : x > j .

��10�

We choose j � 3 and mj � 5.Having computed presynaptic and postsynaptic ac-

tivity, p�tn� and c�tn�, we now change the synapticstrengths from the precortical to the cortical layer usingthe following learning rules: if a neuron �k; l� of layer 3responds strongly to a given input p�tn�, i.e. ckl�tn� � hc(postsynaptic modi®cation threshold: hc � 0:3), then itssynaptic weights are adjusted towards this input:

w0kl;ij�tn�1� � wkl;ij�tn� � ackl�tn�pij�tn� : �11�This learning rule is strictly Hebbian; the change ofsynaptic weights is proportional to presynaptic andpostsynaptic activity. On the other hand, if a corticalneuron responds too weakly, ckl�tn� < hc, its selectivityfor the current input is reduced:

w0kl;ij�tn�1� � wkl;ij�tn� 1ÿ apij�tn�� �

: �12�This applies to the neuron �k; l�'s connections �kl; ij�that ful®ll wkl;ij�tn� > hw �maxijfwkl;ij�tn�g (we choosehw � 0:75). Presynaptic activity causes synaptic depres-sion if it is not accompanied by above-thresholdpostsynaptic activity. A learning rule which incorporatesunlearning was earlier introduced by Cooper et al.(1979) in order to explain the e�ect of visual experienceon the speci®city of cortical neurons. This concept waselaborated by Bienenstock et al. (1982) who employed avariable postsynaptic modi®cation threshold that adaptsaccording to averaged postsynaptic activity (Bear et al.1987). In the following, it will su�ce to apply a ®xedpostsynaptic modi®cation threshold.

Each learning step is completed by multiplicativeweight normalization:

wkl;ij�tn�1� �w0kl;ij�tn�1�P

i0j0 w0kl;i0j0 �tn�1�

; �13�

(von der Malsburg 1973). Thereby, the total a�erentsynaptic strength towards each cortical neuron is keptconstant during learning:X

ij

wkl;ij�tn� � 1 for all tn : �14�

The proposed learning algorithm is outlined in Fig. 6.

4.2 Simulation of `ontogenesis'

In the present subsection, we apply our model toidealized `natural' spatiotemporal stimuli. This leads toa topographic map that is in equilibrium with the pool

2 The spatial scale is de®ned by the number of neurons, the choicev � 1 introduces a time scale

179

of stimuli used. The results will serve as a starting pointfor the simulations of post-ontogenetic plasticity.

We have argued that the relatedness of sensory ac-tivity patterns is re¯ected by their temporal nearness. Inthe simplest case of monomodal stimuli, this relationreduces to a correlation between position and time.Here, we choose Gauss-shaped stimuli of ®xed width(rs1 � rs2 � 2) and at random (uniformly distributed)positions yn � �in; jn�T 2 �rs1 � 1;N ÿ rs1�2. The ISIbetween two consecutive stimuli sn at yn and sn�1 at yn�1is set proportional to their spatial distance

isin � 1

vsjjyn ÿ yn�1jj : �15�

This proportionality describes approximately the aver-age spatiotemporal correlations of natural somato-sensory stimuli. It expresses the continuous developmentof local stimulations. The proportionality factor 1=vs(written as an inverse velocity) is selected to corres-pond to the previously ®xed sizes of the cortical layerand the stimuli's position range fvs � �N ÿ 1� ÿ 2rs1� �=�N ÿ 1� � 0:86g.

In order to obtain global topographic order fromrandomly initialized weights, the learning rate, a, andthe width of cortical response, rc, should decreasemonotonicly during `ontogenesis'.3 We choose

a�n� � ai�af

ai�n=nf 1 : n � nf 1

af : n > nf 1 .

��16�

with a�1� � ai � 1, initial learning rate; a�n � nf 1� �af � 0:002, ®nal learning rate (Fig. 7b; Ritter et al.1990; Kohonen 1995).4 The duration of `ontogenesis' isset to nf 2 � 5� 104 (nf1 � 0:9 nf 2). The width of corticalresponse rc is reduced analogously; ai and af aresubstituted by ri � 15�� N=2� and rf � 1:2, respective-ly. The choice of parameters is in accordance withKohonen (1990); see also (Wiemer et al. 1999).

Figure 7a illustrates the formation of topography; atthree di�erent time steps (n � 5� 103, 3� 104, 5� 104),the cortical lattice is projected into stimulus space. Eachneuron is represented by its RF center; centers belongingto neighboring units are connected by straight lines.Starting with randomly initialized synaptic weights(n � 1), localized RFs are learned and arranged in to-pographic order, leading to a state of equilibrium. Thesynaptic weights, wkl;ij�nf 2�, and ®nal parameters a � af ,rc � rf of the resulting network will be used in thefollowing simulations of post-ontogenetic plasticity.

4.3 Simulation of post-ontogenetic plasticity

In the experiment by Spengler et al. (1996), two classesof tactile stimuli have to be distinguished; the laboratoryanimals received experimental stimuli during dailytraining sessions and natural stimuli between twosessions. Accordingly, we subdivide the series of stimuliapplied in our simulations into alternating subseries ofexperimental and natural stimuli (nexp � 16, nnat � 4).The two tactile bars A and B used as experimentalstimuli (Fig. 1) are approximated by slightly overlap-ping, elongated Gaussian functions with half-widths rA

si,rB

si, i � 1; 2 (rAs1 � rB

s2 � 5, rAs2 � rB

s1 � 2) located at yA,yB [Fig. 9a: yA � �15; 11�; yB � �15; 19�]. In accordancewith our argumentation, we choose the ISI of theexperimental stimuli, isiA;B, to be large compared to thestimuli's representational distance, i.e. we choose

isiA;B > dAB�0�; �v � 1� : �17�The initial (post-ontogenetic) representational distancebetween the centers of feedforward activations, xff;A�0�and xff;B�0�, is given by dAB�0� � jjxff;A�0� ÿ xff;B�0�jj. Theactivations are induced by the two stimuli A, B before thelearning experiment (n is set to zero after `ontogenesis',isiA;B � 16). A subseries of `natural' stimuli is construct-ed as in Sect. 4.2, with the temporal distances againbeing proportional to the spatial distances.

In order to simulate the experiment, we apply theresulting series of stimuli to our model until a state ofequilibrium is reached with regard to the representa-tional distance of the experimental stimuli (Fig. 10). Thenumerically obtained cortical map reveals characteristicfeatures that are neurobiologically observed.

1. Integration. Neural units have attained increased RFsthat coincide with the extent of the experimentalstimuli (Fig. 8). This result is a direct consequence ofHebbian learning.

2. Segregation. The representational distance betweenthe two experimental stimuli is increased (Fig. 9).This result occurs as a consequence of wave-likecortical dynamics and Hebbian learning.

Due to the unlearning by homosynaptic depression(Eq. 12), cortical neurons located between the experi-mental stimulus representations have attained less pro-nounced tuning curves. Former dominant synapticweights are weakened and non-dominant connections

Fig. 6. Sketch of the learning algorithm

3 The question to what extent biological topographic structuresare learned by the correlation of neural activity cannot be answeredin general. Therefore, we assume random initialization as the `worstcase'. Global order can alternatively be achieved by `polaritymarkers' (Willshaw and von der Malsburg 1976)4 The learning rate a does not vanish but is kept at a small level af

re¯ecting post-ontogenetic plasticity

180

strengthened. We consider this phenomenon to beanalogous to the formation of a dorsal input band ob-served experimentally (Sect. 2 and Wang et al. 1995).

The neurobiological experiment reveals cortical re-organization at only one ®xed time chosen by the ex-perimentalist for the electrophysiological mappingprocedure. In the numerical experiment, we can addi-tionally analyze the temporal course of the reorganiza-tion; we observe a continuous shift of corticalrepresentations, saturating at a representational distancedAB�n � 6:4� 104� � v � isiA;B (Fig. 10a). Moreover, weare in a position to continue the experiment in theframework of our simulations by applying exclusively`natural' stimuli. Figure 9d shows the resulting revers-ibility, i.e. the system regains its initial state of equilib-rium (apart from ¯uctuations). Its temporal course isincluded in Fig. 10 (n > 6:4� 104).

In our simulations, the experimental ISI constitutesthe decisive quantity that di�erentiates between inte-

gration and segregation (Fig. 4). In order to demon-strate its e�ect, we rerun the numerical experimentdescribed above with a di�erent `small' experimental ISI,isi0A;B < dAB�0� (isi0A;B � 4). In this case, the experimentalstimuli's representational distance is consecutively re-duced during the learning process, saturating atdAB�n � 105� � v � isi0A;B (Fig. 9d, 10b). This cortical in-tegration illustrates our prediction that temporal stim-ulus distances may be transferred into representationaldistances to form cortical topography.

5 Psychophysics

We have introduced a model of cortical self-organiza-tion as a possible explanation for the neurobiological®ndings presented in Sect. 2. It leads to shifts ofcortical activation that depend on the spatiotemporalconditions of the incoming stimuli. In this section, wesummarize psychophysical results that support suchcortical processes.

5.1 The saltation phenomenon

Spatially separated stimuli applied to the skin aresystematically mislocalized in perception when theirISI lies in the range of up to several hundred millisec-onds. This phenomenon is called saltation (Geldard andSherrick 1972). It illustrates the transformation oftemporal stimulus distances into perceived spatial dis-tances.

In a simple paradigm, Geldard and colleagues (1972)applied three stimuli of equal intensity to two loci of theskin, e.g. 5 cm or 10 cm apart on the back of the fore-arm (Fig. 11). The ®rst stimulus s1 was given at the locusl1 to assure the readiness of the experimental subjects.About 800 ms later, the second stimulus s2 was appliedto the same locus l1. An ISI isi2 later, the third stimuluss3 was applied to the second locus l2. The subjects' taskwas to judge the position of the second stimulus. Thisestimated position varied systematically with the ISI isi2between the second and third stimulus; small (large) ISIslead to large (small) perceived shifts of the second

Fig. 7a,b. Simulation of `ontogenesis'. Each neuron is represented by its receptive ®eld (RF) center: centers belonging to neighboring units areconnected by straight lines. Starting with randomly initialized synaptic weights (n � 1), localized RFs are learned and arranged in topographicorder leading to a state of equilibrium (a). The temporal course of learning rate a is shown in (b)

Fig. 8. Reorganization of RFs. Radially symmetric RFs resultfrom `ontogenesis' as we show exemplarily for two neurons (leftcolumn, a). Simulating the experiment by Spengler et al. (1996) weobserve stimulus induced shifts of RFs (cortical segregation) as well aschanges in RF size and symmetry (feedforward integration, rightcolumn)

181

stimulus towards the third stimulus. The perceivedspatial distance between the two stimuli was found to bea monotonicaly increasing function of their ISI.

We remark that the saltation phenomenon is clearlyseparable from apparent motion. In saltation, a mislo-calized stimulus gives a concisely localized impression.Its position is determined by the ISI used. In contrast,apparent motion consists of ``a somewhat broadly lo-calized, continuous, unbroken sweep'' (Geldard 1982).

Geldard and colleagues analyzed the saltation phe-nomenon extensively by the described paradigm, whichthey called ``reduced rabbit''. They could demonstratethat saltation is not generated in the periphery but that itis a phenomenon of the central nervous system (Geldard1982). Saltatory jumps penetrate, for example, anesthe-tized skin areas and they do not cross the body's mid-line, i.e. they re¯ect the brain's functional architecture inearly sensory processing. Furthermore, saltation seemsto be a universal phenomenon of sensory processing;visual and auditory analogs are described in Geldardand Sherrick (1974) and Hari (1995). Saltatory jumpscan also be generated by monocular stimuli presented tothe two eyes at slightly di�erent positions in the visual®eld (Geldard 1976).

Geldard's ``reduced rabbit'' was symmetrized byKilgard and Merzenich (1995). They introduced a fourthstimulus s4 at the second locus (Fig. 11b). While theperceived spatial distances between second and thirdstimuli were in accordance with Geldard's ®ndings, na-

Fig. 9a±c. Topographic reorganization. Top row Lattice of neural units in stimulus space. Bottom row: Cortical activation induced by experimentalstimuli shown in superposed form. Starting from a state of equilibrium (n � 0) that represents the adult cortex before training (a), experimentalstimuli induce cortical segregation (b) or integration (d) depending on the experimental ISI. The application of exclusively `natural' stimuli leadsback to the initial state; we present the reversibility of cortical segregation (c). An asymmetry in the shift of the experimental stimuli'srepresentations results from the stimuli's di�erent orientation (see also Fig. 10)

Fig. 10a,b. Temporal course of reorganization. We present thetemporal course of cortical position (left column) and representationaldistance (right column) for `large' experimental ISI (segregation, a) and`small' ISI (integration, b). Cortical position and representationaldistance saturate according to the stimuli's spatiotemporal correla-tions (n � 6:4� 104 and n � 105, respectively). Thereafter, only`natural' stimuli are applied demonstrating reversibility. We remarkthat in our simulations segregation progresses faster than integration.This ®nding depends crucially on the interaction function f expressingthe ISI-dependent shift of cortical activation [see (6)]

182

ive subjects generally observed shifts of both of thesestimuli, the second and the third stimuli being shiftedtowards each other. Moreover, Kilgard and Merzenich(1995) found that the centers of the perceived stimulidepend on attention or expectation.

Thus, the perception of two localized stimuli seems tobe decomposed into two components. One process de-termines the distance between the stimuli and is unaf-fected by attention or expectation. Accordingly, weassume this process to re¯ect processing in early (pri-mary) cortical areas. A second process determines thestimuli's center of mass; it is a�ected by attention orexpectation and probably involves higher cortical areas.

Cholewiak (1999) analyzed interactions in the per-ception of spatiotemporally localized stimuli by a sym-metric two-stimuli paradigm. He found that theperceived spatial distance dperc�s1; s2� between the twostimuli s1, s2 is composed of their true spatial distancedtrue�s1; s2� and an ISI-dependent variation D(ISI) thereof

dperc�s1; s2� � dtrue�s1; s2� � D�ISI� ; �18�(Fig. 11c). These results ®t well to the ISI-dependentshifts of cortical activations that we introduced in ourmodel of cortical plasticity in order to simulate theneurobiological ®ndings of Sect. 2.

5.2 Application of our approach

Although the saltation phenomenon has been known formany years, there is still no systematic theoreticalapproach towards its understanding. Here, we proposeto transfer the model of cortical self-organizationpresented above to the perception of spatiotemporalstimuli. This o�ers a framework in which systematic

time-dependent shifts of cortical responses result fromcortical dynamics and interaction. Accordingly, wepredict the following psychophysical percepts.

1. Saltation should also occur for other stimulus pa-rameters (besides sensory spatial coordinates) that aretopologically represented in early cortices (e.g. fre-quency in audition, orientation in vision).

2. Saltation should extent to segregation, i.e. to increasedperceived distances for `large' ISIs, where `large' isrelative to the stimuli's representational distance. Thecorresponding time range (ISIs of about 200±400 ms)has not been systematically analyzed so far. Accordingto our model, the extent of such segregating shiftsmight be smaller than those observed for small inte-grating ISIs. This asymmetry could result from a time-dependent attenuation of cortical excitation.

6 Discussion

6.1 Importance of temporal stimulus structurefor cortical topography

We argue that the temporal order and proximity of theincoming stimuli re¯ect the stimuli's relatedness andfunctional similarity. As topographic representationsre¯ect the stimuli's relatedness, these temporal stimulusaspects are important for learning in cortical maps. Inthe case of high-dimensional visual stimuli, this idea iselaborated in Sect. 6.3.

The very goal of sensory processing is to provide abasis for the generation of adequate behavior. An or-ganism must react to changes in its environment ac-cording to its needs. Thereby, the time scales ofperception and behavior are linked to each other. There

Fig. 11a±c. Experimental paradigms for psychophysical saltation. Geldard (1976, 1982) introduced the `reduced rabbit paradigm' consisting ofthree stimuli at two loci: a ®rst stimulus s1 at locus l1 to assure readiness, a second stimulus s2 about 800 ms later also at l1, and a third stimulus s3an ISI isi2 2 [0, 300] ms later at the second locus l2 (a). Experienced experimental subjects perceived the locus of s2 to be shifted towards theattractant s3. Kilgard and Merzenich (1995) symmetrized the `reduced rabbit paradigm' by the introduction of a fourth stimulus at the secondlocus (b). Naive subjects reported a mislocalization of both stimuli s2, s3. The stimuli's center of mass was in¯uenced by attention or expectation.A reduced symmetric paradigm consisting of only two stimuli was used by Cholewiak (1999, c). Here, experimental subjects were only asked tojudge the distance between stimuli (or the spatial extent of stimulation if only one stimulus was felt), not their loci. For symmetry reasons, weassume in c that both stimuli are shifted towards each other

183

is no bene®t, for example, of highly time-resolved per-ception if the assembled information cannot be trans-ferred into behavior.

In e�cient stimulus representations, not all possiblestimuli are di�erentiated. Instead, stimuli are groupedinto behaviorally relevant classes. We argue that corticalareas could use the temporal proximity of the stimuli toform e�cient representations. Stimuli that follow eachother very closely in time should be represented together(i.e. in one class); their di�erentiation is behaviorallyirrelevant. Stimuli that typically possess behaviorallyrelevant temporal distances to one another should bedi�erentiated. Average ISIs may serve as essential cri-teria that guide the corresponding integration and seg-regation processes.

Previous models of topographic self-organization arebased on the spatial structure and probability distribu-tion of the stimuli (von der Malsburg 1973; Kohonen1995). At discrete times, stimuli are presented to theneural network and neural properties are successivelyadapted according to some learning rule. Typically, thestimuli are chosen randomly according to their proba-bility distribution and independent of earlier stimuluschoices. In this framework, there are only the two ex-tremes of synchronous and asynchronous stimulus pre-sentation. Any two consecutive stimuli are temporallyseparated, i.e. they do not interact in the generation ofnetwork responses. The e�ect of an incoming stimulusonly depends on the actual weights (and the learningrate) and not on the activations due to former stimuli.

In the experiment of Spengler and colleagues (1996),stimuli are applied that are adjacent in space (spatialoverlap on the third ®nger) but not in time. Why is thespatial neighborhood of the experimental stimuli nottransformed into a cortical neighborhood? We arguethat the observed cortical segregation of the experi-mental stimuli results from their temporal separation (byan ISI of 300 ms). A self-organizing process that is re-stricted to spatial stimulus patterns cannot segregate thecorresponding representations. Therefore, we presentedan approach to self-organization that extracts topogra-phy from spatiotemporal stimuli. The model transformsaverage temporal stimulus distances into representa-tional distances. Our simulations of the neurobiologicalexperiment support this view.

6.2 Time-based interpretation of cortical maps

The presented model of stimulus-induced learningexhibits ISI-dependent representational distances. Thisconstitutes an interpolation of neurobiological ®ndings.In other words, we assume that the ISI dependence ofrepresentational distances is not con®ned to a meredi�erentiation of synchronous and asynchronous stim-uli, but that this dependence extends to the whole rangefrom zero to several hundred milliseconds.

Our ansatz o�ers a time-based interpretation of wellknown cortical topographies. First, hand representa-tions in cortical area 3b of monkeys typically do notre¯ect the proportions of the sensory dimensions. Given

the topography of the cortical hand representation(Merzenich et al. 1978; Jenkins et al. 1990; Recanzoneet al. 1992), we notice a compression in the rostro-cau-dal direction (along represented ®ngers) relative to anexpansion in the medio-lateral direction (across thedi�erent represented ®ngers). According to our model,this distortion may re¯ect the fact that the average ISIbetween stimuli on adjacent segments of the same ®ngeris smaller than the average ISI between stimuli on ad-jacent segments of di�erent ®ngers.

Second, the roughly logarithmic structure of the re-tinotopic projection onto the primary visual cortex ofmonkeys may correspond to spatiotemporal correlationsin the retinal ¯ow ®elds; the decrease in cortical mag-ni®cation from foveal to peripheral visual coordinatescorrelates with an increase of retinal velocity due to self-motion (Schwartz 1980; Lappe and Rauschecker 1995).

6.3 Representation of high-dimensional stimuli

The Gaussian functions used in our simulations asexperimental stimuli are su�cient to illustrate our ideasand to generate simple topographic maps. However, webelieve that our approach would also be highly valuablefor the representation of high-dimensional stimuli. TheEuclidean distances of high-dimensional stimuli do notnecessarily re¯ect their relatedness. One may think ofdi�erences in retinal projections generated by thetranslation and rotation of three-dimensional objects.

Under natural conditions of stimulation, the lattergeometrical transformations result from self and objectmotion. This leads to di�erent views of single objectsthat are linked by temporal proximity. The resultingtemporal proximity could be used by visual systems, notonly to learn invariances in the selectivities of single cells(Edelman and Weinshall 1991; Wallis 1996), but also totopographically represent di�erent single object viewsnext to each other, i.e. to represent stimuli with similarfunctional meaning spatially similarly. We see the ad-vantage of such a topographic coding scheme in its ro-bustness; interpolation and extrapolation of stimuli canbe performed in a functional meaningful space.

Neurons in the anterior inferotemporal cortex (IT) ofmonkeys respond selectively to moderately complex vi-sual stimuli and cluster in columnar regions (Tanaka et al.1991). In addition, the selectivities of these neurons arehighly plastic, they seem to constitute a visual associativelong-term memory (Miyashita 1988). Furthermore, opti-cal imaging analysis of the functional organization in ITsuggests a ``continuousmapping of related features over aregion around 1 mm in size'' (Wang et al. 1996). There-fore, we hypothesize that the inferotemporal cortex mayrepresent topographically typical three-dimensionalviews according to a time-based metric.

6.4 Mapping stimulus frequency onto the cortex

So far, we have demonstrated how self-organizingprocesses can transfer temporal stimulus distances into

184

representational distances. However, it is known fromneurobiological experiments that the probability distri-bution of the stimuli is also mapped onto corticaltopography; more frequently applied stimuli tend tocapture larger cortical areas (Recanzone et al. 1992;Elbert et al. 1995; Pantev et al. 1998). The latterphenomenon can be easily incorporated in our ap-proach. If we consider plastic intracortical connectionsthat are adapted according to Hebbian learning, intra-cortical connections can be strengthened by propagatingwaves. Accordingly, more frequently stimulated corticalregions develop stronger intracortical connections, re-sulting in faster propagation and correspondingly in-creased representational areas.

6.5 Maladaptive plasticity

Our approach may be relevant in the context of mal-adaptive plasticity. The phenomena phantom limb pain,focal dystonia, and dyslexia, for example, may result frompathological changes in cortical topology (Merzenichet al. 1993; Flor et al. 1995; Byl et al. 1996). Therapyincluding systematic alterations of the temporal structureof sensory stimuli may prevent further topographicdeterioration and/or may be bene®cial to re-establishtopographic representations in e�ected primary cortices.

6.6 Biological plausibility of wave-like dynamics

From a theoretical perspective, the assumed wave-likedynamics is of a fundamental type (Sect. 3.3). Weremark that the assumed wave-like dynamic does nothave to be radially symmetric in single-trial analysis inorder to exert its in¯uence on the learning process. Itmay be superimposed by ¯uctuations that are random ordue to varying uncontrolled conditions of stimulation.However, the dynamics should be observable at least onaverage over several trials.

There is some experimental evidence that wave-likedynamics occurs in biological neural networks. It maybe realized in di�erent forms: horizontal propagation ofactivity (Tanifuji et al. 1994; Prechtl et al. 1997; Bring-uier et al. 1999), di�usion of a volatile substancethrough cortical tissue (e.g. nitric oxide; Krekelberg andTaylor 19965), the propagation and dynamic interactionof chemical substances (e.g. calcium; Garaschuk et al.1998) or of non-inactivating natrium currents (Taylor1993).

In the case of neural activity waves, the assumed typeof dynamics was; for example observed by Tanifuji et al.(1994), who studied the propagation of excitation inslices of rat visual cortex. Prechtl and colleagues (1997)analyzed neural responses in the visual cortex of unan-esthetized turtles. Their single-trial analysis reveals thecoupling of spatial and temporal aspects of stimulus-

evoked activity, e.g. propagating wave fronts ofdepolarization and hyperpolarization. More recently,Bringuier et al. (1999) conducted in vivo intracellularrecordings to analyze subthreshold responses evoked bystimuli outside the classical RF. They observed a linearrelation between the latency of postsynaptically evokeddepolarization and the stimulus eccentricity to the min-imal discharge ®eld. Their ®ndings suggest a radial waveof activity spreading at a constant speed of about100 mm/s over a radius of more than 10 mm.6

6.7 Saltation

Our ansatz o�ers a framework in which psychophysicalsaltation can be partly attributed to interactions anddynamics in early sensory cortices. The psychophysicalsaltation phenomenon transforms temporal stimulusdistances into perceived spatial distances. It seems tobe a general feature of sensory cortical processing as itoccurs in several modalities (Geldard 1982; somatosen-sory, visual, and auditory; see Sect. 5.1). The phenom-enon may prove to be analogous to the ISI-dependentshifts of cortical responses introduced in our model ofself-organization. Assuming that shifted cortical stimu-lus responses result in ISI-dependent cortical reorgani-zations and lead to saltation, we predict that saltationextends functionally to additional topographically rep-resented parameters and temporally (for `large' ISIs) tosegregation.

We remark that the present experimental evidence forsaltatory jumps is restricted to interpolation; illusorylocalized stimuli vary between the two sensorily stimu-lated loci. The prediction of segregation implies theprediction of extrapolation; for speci®c spatiotemporalconditions, stimuli should also be perceivable outside thestimulated region (Sect. 5.2).

6.8 Neural ®eld dynamics

In our simulations, we have focused on the time scale oflong-term learning related to cortical reorganization.Stimulus responses were computed according to somebasic rules. They were not generated in a dynamic wayresulting from explicitly simulated interactions betweenfeedforward activation and intracortical inhibition andexcitation. In addition, stimulus responses were restrict-ed to monomodal functions.

The presented approach is exclusively based on pro-cesses that act locally in cortical coordinates (or in aparameter space). Therefore, time-dependent neural®elds, as proposed by Wilson and Cowan (1973), o�er aframework for re®nement leading to more realistic cor-tical stimulus responses and also to the assumed type of

5 NO di�uses at a speed of 2 mm/s, but seems to be spatiallylimited to volumes of tissue of about 10 lm (Montague and Sej-nowki 1994)

6 With regard to the neurobiological ®ndings presented in Sect. 2,we do not know whether the reorganizational process has alreadysaturated or reached anatomical limits. Therefore, we can onlyestimate a lower bound of a few millimeters per second for thecortical velocity

185

fundamental dynamics. They can be applied to achievenumerical results that allow a quantitative comparisonwith psychophysical and neurobiological data (Janckeet al. 1996; SchoÈ ner et al. 1997).

7 Summary and outlook

We have presented an approach to the learning ofcortical topography from stimulus dynamics, therebystressing the importance of temporal stimulus structure.The resulting model was applied to the simulation of aneurobiological experiment. We have demonstrated howspatiotemporal correlations can lead to topography anddeduced quantitative neurobiological and qualitativepsychophysical predictions.

Expressed in more general terms, our work puts for-ward the following question: To what extent do bio-logical neural systems make use of the temporalstructure of incoming signals to built up and alter theirrepresentation? We believe that this issue is essential forbiological neural systems in order to speed up signalprocessing and increase its robustness, i.e. to provide asound basis for the generation of adequate behavior. Webelieve that experimental tests of our predictions mayconstitute a further step towards a better understandingof cortical plasticity and cortical stimulus representa-tion.

Acknowledgements. We thank T. Burwick for stimulating discus-sions. We are also grateful to B. Sendho� for his comments onearlier drafts of this manuscript. The work was supported by grantDFG, SFB 509.

References

Allard T, Clark S, Jenkins W, Merzenich M (1991) Reorganiza-tion of somatosensory area 3b representation in adult owlmonkeys after digital syndaktyly. J Neurophysiol 66:1048±1058

Amari S (1980) Topographic organization of nerve ®elds. BullMath Biol 42:339±364

Bear M, Cooper L, Ebener F (1987) A physiological basis for atheory of synapse modi®cation. Science 237:42±48

Bienenstock E, Cooper L, Munro P (1982) Theory for the devel-opment of neuron selectivity: orientation speci®city and bin-ocular interaction in visual cortex. J Neurosci 2:32±48

Bringuier V, Chavane F, Glaeser L, Fregnac Y (1999) Horizontalpropagation of visual activity in the synaptic integration ®eld ofarea 17 neurons. Science 283:695±699

Brosch M, Schulz A, Budinger E, Scheich H (1998) Enhancementof neuronal responses in sequences of pure tones in macaqueauditory cortex. Soc Neurosc Abstr 24:400

Brown T, Kairiss E, Keenan C (1990) Hebbian synapses: bio-physical mechanisms and algorithms. Annu Rev Neurosci13:475±511

Buonomano D, Merzenich M (1995) Temporal informationtransformed into a spatial code by a neural network with re-alistic properties. Science 267: 1028±1030

Byl N, Merzenich M, Jenkins W (1996) A primate genesis model offocal dystonia and repetitive strain injury [...]. Neurology47:508±520

Cholewiak R (1999) The perception of tactile distance: in¯uences ofbody site, space and time. Perception, in press

Cooper L, Liberman F, Oja E (1979) A theory for the acquisitionand loss of neuron speci®city in visual cortex. Biol Cybern33:9±28

Durbin R, Mitchison G (1990) A dimension reduction frameworkfor understanding cortical maps. Nature 343:644±647

Edelman S, Weinshall D (1991) A self-organizing multipleviewrepresentation of 3d objects. Biol Cybern 64:209±219

Elbert T, Pantev C, Weinbruch C, Rockstroh B, Taub E (1995)Increased cortical representation of the ®ngers of the left handin string players. Science 270:305±307

Erwin E, Obermayer K, Schulten K (1995) Models of orientationand ocular dominance columns in the visual cortex: a criticalcomparison. Neural Comput 7:425±468

Flor H, Elbert T, Knecht S, Wienbruch C, Pantev C, Birbaumer N,Larbig W, Taub E (1995) Phantom-limb pain as a perceptualcorrelate of cortical reorganization following arm amputation.Nature 375:482±484

Garaschuk O, Hanse E, Konnerth A (1998) Developmental pro®leand synaptic origin of early network oscillations in the CA1region of rat neonatal hippocampus. J Physiol (Lond.)507.1:219±236

Gardner E, Costanzo R (1980) Temporal integration of multiple-point stimuli in primary somatosensory cortical receptive ®eldsof alert monkeys. J Neurophysiol 43:444±468

Garraghty P, Kass J (1992) Dynamic features of sensory and motormaps. Curr Opin Neurobiol 2:522±527

Geldard F (1976) The saltatory e�ect in vision. Sensory Processes1: 77±86

Geldard F (1982) Saltation in somethesis. Psychol Bull 92:136±175Geldard F, Sherrick C (1972) The cutaneous ``rabbit'': a perceptual

illusion. Science 178:178±179Geldard F, Sherrick C (1974) Princeton cutaneous research project.

Technical Report 24, Princeton UniversityHaag R (1993) Local quantum physics, 2nd edn, Springer, Berlin

Heidelberg New YorkHari R (1995) Illusory directional hearing in humans. Neurosci

Lett 189: 29±30Jancke D, Akhavan A, Erlhagen W, Giese M, Steinhage A,

SchoÈ ner G, Dinse H (1996) Population coding in cat visualcortex reveals nonlinear interactions as predicted by a neural®eld model. ICANN'96, Proceedings. Springer, Berlin Heidel-berg New York pp 641±648

Jenkins W, Merzenich M, Ochs M, Allard T, Guic-Robles E (1990)Functional reorganization of primary somatosensory cortex inadult owl monkeys after behaviorally controlled tactile stimu-lation. J Neurophysiol 63:82±104

Joublin F, Spengler F, Wacquant S, Dinse H (1996) A columnarmodel of somatosensory reorganizational plasticity based onhebbian andnon-hebbian learning rules. BiolCybern 74:275±286

Kilgard M, Merzenich M (1995) Anticipated stimuli across skin.Nature 373:663

Kohonen T (1982) Self-organized formation of topologically cor-rect feature maps. Biol Cybern 43:59±69

Kohonen T (1990) The self-organizing map. Proc IEEE 73:1464±1480

Kohonen T (1995) Self-organizing maps. Springer, Berlin Heidel-berg New York

Krekelberg E, Taylor J (1996) Nitric oxide in cortical map for-mation. J Chem Neuroanal 10:191±196

Lappe M, Rauschecker J (1995) Motion anisotropes and headingdetection. Biol Cybern 72:261±277

Malsburg C von der (1973) Self-organization of orientation sensi-tive cells in the striata cortex. Biol Cybern 14:85±100

Marshall W, Woolsey C, Bard P (1937) Cortical representation oftactile sensibility as indicated by cortical potentials. Science85:388±390

Mayer N, Hermann M, Bauer H-U, Geisel T (1998) A corticalinterpretation of assoms. Int ICANN'98, Proc Springer, BerlinHeidelberg New York, pp 361±366

Mehaute BL (1976) An introduction to hydrodynamics and waterwaves. Springer, Berlin Heidelberg New York

186

Merzenich M, Kaas J, Suhr M, Lin C (1978) Double representationof the body surface within cytoarchitectonic areas 3b and 1 in``si'' in the owl monkey (aotus trivirgatus). J Comp Neurol181:41±74

Merzenich M, Nelson R, Kaas J, Stryker M, Jenkins W, Zook J,Cynader M, Schoppmann A (1987) Variability in hand surfacerepresentations in areas 3b and 1 in adult owl and squirrelmonkeys. J Comp Neurol 258:281±296

Merzenich M, Schreiner C, Jenkins W, Wang X (1993) Neuralmechanisms underlying temporal integration, segmentationand input sequence representation: some implications for theorigin of learning disabilities. Ann NY Acad Sci 628:1±22

Miyashita Y (1988) Neuronal correlate of visual associative long-term memory in the primate temporal cortex. Nature 335:817±820

Montague P, Sejnowki T (1994) The predictive brain: Temporalcoincedence and temporal order in synaptic learning mecha-nisms. Learning Memory 1: 1±33

Obermayer K, Blasdel G, Schulten K (1992) Statistical-mechanicalanalysis of self-organization and pattern formation during thedevelopment of visual maps. Phys Rev A 45:7568±7589

Pantev C, Oostenveld R, Engelien A, Ross B, Roberts L, Hoke M(1998) Increased auditory cortical representation in musicians.Nature 392: 811±814

Prechtl J, Cohen L, Pesaran B, Mitra P, Kleinfeld D (1997) Visualstimuli induce waves of electrical activity in turtle cortex. ProcNatl Acad Sci 94:7621±7626

Recanzone G, Mersenich M, Jenkins W, Grajski K, Dinse H (1992)Topographic reorganization of hand representation in corticalarea 3b of owl monkeys trained in a frequency-discriminationtask. J Neurophysiol 67:1031±1066

Riesenhuber M, Bauer H-U, Geisel T (1996) Analyzing phasetransitions in high-dimensional self-organizing maps. Biol Cy-bern 75:397±407

Ritter H, Martinetz T, Schulten K (1990) Neuronale Netze.Addison-Wesley, Reading, Mass

SchoÈ ner G, Kopecx K, Erlhagen W (1997) The dynamic neural®eld theory of motor programming: arm and eye movement.In: Morasso P, Sanguinet V (eds) Self-organization, computa-tional maps, and motor control. Elsevier, Amsterdam pp 271±310

Schroeder C, Seto S, Garraghty P (1997) Emergence of radial nervedominance in median nerve cortex after median nervetransection in an adult squirrel monkey. J Neurophysiol77:522±526

Schwartz E (1980) Computational anatomy and functional archi-tecture of striate cortex: a spatial mapping approach to per-ceptual coding. Vision Res 20:645±669

Spengler F, Hilger T, Wang X, Merzenich M (1996) Learning in-duced formation of cortical populations involved in tactileobject recognition. Soc Neurosci Abstr 22:105

Spengler F, Hilger T, Wang X, Merzenich M (1999) Cortical plas-ticity underlying tactile object recognition learning

Tanaka K, Saito H, Fukada Y, Moriya M (1991) Coding visualimages of objects in the inferotemporal cortex of the macaquemonkey. J Neurophysiol 66:170±189

Tanifuji M, Sugiyama T, Murase K (1994) Horizontal propagationof excitation in rat visual cortical slices revealed by opticalimaging. Science 266:1057±1059

Taylor C (1993) Na� currents that fail to inactivate. Trends Neu-rosci 16:455±460

Wallis G (1996) Using spatio-temporal correlations to learn in-variant object recognition. Neural Netw 9:1513±1519

Wallis G, BuÈ ltho� H (1999) Learning to recognize objects. TrendsCogn Sci 3:22±31

Wang G, Tanaka K, Tanifuji M (1996) Optical imaging of func-tional organization in the monkey inferotemporal cortex. Sci-ence 272:1665±1668

Wang X, Merzenich M, Sameshima K, Jenkins W (1995) Remod-elling of hand representation in adult cortex determined bytiming of tactile stimulation. Nature 378:71±75

Weinberger SCN (1996) Evidence for the hebbian hypothesis inexperience-dependent physiological plasticity of neocortex: acritical review. Brain Res Rev 22:191±228

Wiemer J, Burwick T, von Seelen W (1999) Self-organizing mapsfor visual feature representation based on natural binocularstimuli Biol Cybern, in press

Wiemer J, Spengler F, Joublin F, Stagge P, Wacquant S (1998a) Amodel of cortical plasticity: integration and segregation basedon temporal input patterns. ICANN'98, Proc Springer, BerlinHeidelberg New York, pp 367±372

Wiemer J, Spengler F, Joublin F, Stagge P, Wacquant S, Leon-hardt R (1998b) A model of cortical plasticity based on tem-poral input sequences. Soc Neurosci Abstr 24:637

Willshaw D, von der Malsburg C (1976) How patterned neuralconnections can be set up by self-organization. Proc R SocLond B 194:431±445

Wilson H, Cowan J (1973) A mathematical theory of the functionaldynamics of cortical and thalamic nervous tissue. Biol Cybern13:55±80

187