Simulation of the complexHäussler-von der Malsburg Dynamics

Dr. P. Nonnenmann

April 22,2008

Contents

Introduction

Neuro- biological Foundations

1.1 The central Vision track

1.2 Retinotopy

1.3 Goals of mathematical modelling and simulation

1.4 Results

mathematical model

The dynamical system

The associated global attractor

Stationary points

On the existence of stationary points

A conjecture based on algebraic geometry

Convolution

3.1 The discrete lD-Fourier Transform

3.2 The discrete lD-convolutions

3.3 Discrete 2D-convolution

3.4 The associated C-Operators

Stability via Linearization

4.1 Linearization

The

2.t

2.2

.)2

t1

j)(

7

7

B

o

o

10

10

t2

13

15

16

18

18

19

20

2L

24

24

254.2 Stability of stationary points

Numerical simulation

5.1 Computational details of the simulation

5.2 A combination of cyclic and linear convolution

5.3 Linear versus cyclic dynamics

5.4 Cyclic dynamics with different starting points

5.5 A different C-operator: the Sobel-fil1,er .

5.6 Sensitive hehavior with respect to u,0

5.7 Eigenralues of the linearized system

Conclusion

26

26

27

28

30

35

35

36

39

Chapter 1

Neuro-biological Foundations

This Chapter introduces briefly the anatomy of the visual system and the problemof self-organization in question.

1.1 The central Vision track

The level of the receptors lies on the opposite side of the retina, opposite to wherethe light comes in (rrinside'r). Between the receptors and the rroutside", we have thelevel of the horizontal cells, the bipolar cells, the amacrines and the gan-glion cells. The level ofthe receptors consists of2 types ofphoto receptors: 120*106

Stäbchen and 6 x 106 Zapfen. The Stäbchen are mainly important during timesof low light intensity, whereas the Zapfen are responsible -for color daylight vision.In the receptor cell there is a molecule called 11-cis-retinal. After absorption of aphoton, the 11-cis-retinal transforms into an isomer, called all-trans-retinal. Aftera geometrical change in the relative positions of the molecules, the early receptorpotential is created.

This signal is being transduced to the mentioned cells and finally to the ganglioncells. The visual information is being transferrpd to the brain via the nervus opti-cus, which is built out of the axons of the ganglion cells. The nervi optici unite tothe chiasma opticum, where the fibers stemming from the nasal part of the retinacross to the opposite side. The fibers from the temporal part of the retina stay on

the same side and connect to the first central processing centers of the vision track:the corpus geniculatum laterale (CGL) and the tectum opticum. Onlyabout 101,100 of the fibers end at the tectum, the others at the CGL.The CGL axons t'run" as a fiber called radiatio optica to the primary visualcortex (area l7). From there, connections exist to the secundary (area 18) andternary(area 19) visual cortex; see [ff] .

1.2 Retinotopy

The pa,per ILl anall'zes a probieur of biological rnorp]rogenesis ]tv self-organizatiril:the formation of so-r:alled retinotopic prrojections. During ont,ogenesisi axons ol neu-rons of the retina start to grow to a region of the midclle brain: the tect,um. In thebeginrring, t,he synaptic connections are rather ranckrm. In the aclult, neighbour-ing cells o{ the retjna project orrto neighbouring cells of the tectum: retinotopir:projection.

Assuuring there are two chains of ,A/ neurols in both retina antl tectum, we call thestrength of the connection between retina cell r ancl tectuur cell f the weight ur" € IFR .

The weights form arr (l/,ly')-rnatrix (u.,1.) and the two projectiorrs are represented byuratrices which have nonzero entries only in the main- and antldiagonal ( definitionis given later ), respectively.

A mechanism ca,pable of generating retinotopy ha^s been described in [2, 31. It usescooperativity and competition, see Fig.1.

Fig.1

Retins

--+++--EaUüF

Modification of weigüts. Elevated weights

iocrease by cooperxivity (*)- Due to competitiorcorresponding columns aod rorrs (-).

whicl lie closely together iq

this growth is compensated

tle conaection-matrix

by a decrease in the

Adiacent, rieiglrts arnl-,lify tirernst-.lves through cot,r1-relaLiLrtr (*), ancl ilre being inhib-i1,ed through comlret,ition trv the corresp,r.,nding rou,s and colun-rns (-) .

1.3 Goals of mathematical modelling and simula-tion

A main feature is the formulation of cooperativity and comlretitivity in mathematicalterms, the main object of investigation is to explain the dynamical betravior of theweights and to detect general structures.

By anal),tic reasoning several properties of the dynamical systeur modelling the prob-

lern aborre can be detected: continuous dependenc5., stationary points, boundedness

of orbits, attractors, . . . . By simulations some questions ma)' be checked concerningthe long-term-behavior of the system. For ttie theory of dl,n3pi.ol systems, see eg.

[16,17,181.

L.4 Results

In this thesis we cornpäre tlrr: cyclic convoh tion, r'l here there are r:ilcles of r;ells. rvhich

is biologically unrealistic, u.itli the biologicalll, more plausible linear cornolution (In

[1] the cyrtlir: convolution ri,as used.). Linear convolution means ihat the boundarl-

celis are being treated in tlie follou,ing u,ay : tectuur artd retina a,re being continuecl

b1' artificial cells lvith u,eight zero and convolution is done on this artificial fielcl of

c:ells.

Since for linear convolution some anai-vtic steps (unknown stationaly points, invari-ance of eigenspaces, .. . ) can not be done, a synergetic analysis seems no longer

possible: Therefore we use numerical methods lbr analyzing the dynautics ancl the

stabilit.y o[ tlre solutions.

Chapter 2

The mathematical model

Here we formulate the model of retinotopy by non-linear, high dimensional ordinarydifferential equations, fo[owing [11.

2.1 The dynamical system

As mentioned above, we assume ihat both retina and tectum are 1-dimensional

chains, each consisting of trf cells. Hence we consider the real (,n[, -n/)-matrix ttt :(u1,.) of weights between cells t, r . The form of the equations describing the tempo-ral change u' of w is given as follows:

, t,T :1,..., N,

(2. 1)

wherea)0and

w'r,,:F1,(w') :: f,,,(w) - l,l(2N)ur, (ä f4,(w)n j Ortrl)

ft,,(r)::a t tt-4,C6,,(w) ,

N

Cr,,(r),: » k(t - i,t' * i)rui,i .

,i,j:|

t, ,\

(2 3)

(2.1) is a system of first order of cubic equations in N2 variables. ft,,(*) is thegrowth rate of u.r, consisting of the growth rate a of the incoming retina axons toih. t""tr* and the cooperation term Q,,(u). The latter is the sum of products

k(t*i,r-i)utti

where k(t * i,r - j) describes the amplification of synapse (t, r) by synapse (2, j).

{2.4)

10

The C-operator has been written as a general convolution: we will distinguish be-tween the linear and cyclic convolution (see Chapter 3). The kernel k is normalizedas follows:

N

k(i,i)) 0,t,r : 1,....N, I,k(i,3) : 1.i:|,.i:\

(2.5)

This property is assumed throughout the thesis without further mention. For mostof the considerations the kernel k is derived from a standard Gaussian:

k(r,ü :: erp(-(t;' + y'1112"'17

Usually we put a:0.6.

Since the righthand-side F of the differential equation (2.1) is a system of polyno-mials in the variables arrr it is differentiable and so it is locally Lipschitz continuous.Hence there exists a uniquely determined local solution for a given initial value 'u.,0:

ru'(s) : F(Tr,'(s)) , -s € i0, 7) , tu(0) : ,n0 ,uthet'e T > A; (2.6)

see [r91.To obtain a global solution it is sufficient to show that such a solution is boundedin the domain [0, f) of definitiorr. A first step in this direction is to prove tha,t thehypercube

K7,1 :: l0,l/] x .. . x [0,.n/]

is an invariant subset.

Lemma 2.L Let w0 e K1s . Th,en, the a,,ssoc'iated solutic,n w erists for all tr,mes andut(t') e K1s for all t> 0.

Proof:As we knou,, rve have a uniquely determined local solution

ur : 10, T) * R.N'N .

Suppose utu(st) : 0 for som€ s1 € [0,7). Then by uniqueness reasons tli.(s) : 0

foralls)sr.Suppose wt (sr): N for sorl€ s1 ) 0 . Then

w|, : f*(*) - lf 2u6lSu, * SuTl,A/

Su, : 1/N» ft,(w) > llN(a*a1,,.C1,,(ru))+r/,nrf a

11

i.:1

where we have estimated only ,S2., ,Sq is estimated courpletely analogous.

.|,,(st) : a * tuy,,(s1)Ct,,{*) - lf2u6lSu, * Sull

-Lf 2w6(s1X(N * t)lNc')

-112(N-1)a<0,

This sho'ws that the local solution stays in the hypercube K,nr. By a rvell knc,rvn

reasoning, the solution is deflned for all times and stays in the h.vpercube K1r. I

2.2 The associated global attractor

Here we define the irnportant notion of an attractor and determine the attractorexplicitly for the system in question. Notice that the dynamical systern (2.1) is onlymeaningfulforbiologicalreasonsrvhenu,0 € RI'',: {A : (ar)lau ) 0 for a\li,,i *:1,...,1/).

\\re clefine the flow induced by the solution ur starting at tuo by:

O(f . uo) :: tr,(l), 1 ) 0. tr,o e Rf'N .

Definition 2.2 A subset A c R{'N i,s called, an attractor i.ff1. A i,s closed a'nd bou'nded

2. A i,s 'irtuariant, that i,s 0(R+, A) C A3. A i,s attract'ing, that i,s

1U eU(A)1U cVlty )0Vu6 e VVt>ty@t(t)e tJ')

Theorem 2.3 The hypercube Ky is a global attractor.

Proof:Clearly, I{,nr is bounded and closed, hence compact.Set

[.I :: [0, (N+e)]N,N, V ::[0, (l/*2e)]N,N, € ] 0.

We have to shorv that each orbit starting in I/ ends up h U . As in the proof ofLemma 2.5, we can rewrite our equation (2.1) as

w'r, : o,(i - utt)*w6c6(w)(l*w*lN) - (tl2N)wnllrru,c*(r) +luactl@))i.*t j*r

12

Assume:Then

*'rr(t)

Therefore

rvhich shows thats>*efl'L "

There exists t,r with w,(s) > l[+e for all s € I c ]R and ut1,(s) I IY *2e.

cy(1 - N -.) + (N + z.)iI k(t - i,,r - j)(r/ + 2e)l(1 - (r/ + dlN)xl

a(1 -N*.) +l/(l/+2e)2(-e):: A,[10

IL TL

u(1) -u(0): I ,'G)ds<AI I tds<MtJo Jo

'u(s) < l/ + e for all s Z *e/l[. Therefore, wb(s) e [/ for allI

10

Figure 2.1: The orbit ends in K1.

Figure 2.1 is an illustration of Theorem 2.3. Three components of the trajectory areshorvn starting in u(0) : 61[ e IR5'5 ending in a stationary point (from the numericalpoint of view) rvhich is approximately equal to the diagonal matrix with entries 4.7in the diagonal.

2.3 Stationary points

Here we present some results concerning stationarl, points.

Definition 2.4 A matrir P e RN,N i,s called a stat'ionary poi,nt of the matri,r func-ti,on F i.fJ

F(P) : o.

13

Consider the property

c(r) : n.

This property is satisfied when we use cyclic convolution as one can see very easily;see also [1].

Lemma 2.5 Sup,tose that the property C(il) : fl zs satisfi,ed. Then ln the casea : 0 we haue only one h,omogeneous real stat'ionary po,int, namely the matrir nuhich'is a matrir w,ith entrtes 1.

Proof:After some arithmetics, 1ve can rervrite system (2.1) as

wl, : a(l - wt,) * w 6 C 7, (ut) (1 - u: r, I N) * (t I 2 N )w 6" [I rr. C,, (r) + I u q C 71 (w)] .

i*,(2 7)

Setting lug: 0,[., n, € C, u,e get:

F (*) * : ll'l*4; r!'i

- )Ylfr;:).'

Therefore o : 1 or a: *J4, u,hich are trvo imaginary zeros.

Remark 2.6 The reformulatr,on of th,e rigltt-hand .side r,rt, (2.1) suggest t,wo types ofcand'idates of stat'ionary potnts P : (pni) € IRN,N;

. pt5 x 1 euerywhere (P x n)

. p4 x N for seuero,l i, j (P approrimately a, loose Ttermutat,ion matru,r)

Following [10], rve define:

Definition 2.7 A loose Ttermutati,on, matriris one that conta,ins at most a single 1i,n each row a'nd column, and 0's euerywhere else.

Theorem 2.8 Suppose that the proyterty C6): { z's satisJied,. If a : A, for artylctose permutatr,on matrh P of si,ze,Ay' x -Ay', W :: NP i,s a stationary Tto,int of system(2 1).

Proof:See [101. IThus the number of stationary points is at least exponential in N, in the case a - 0.In general, the assertion of Theorem 2.8 does not hold when a > 0.

I

t4

2.4 On the existence of stationary points

As rve have seen in Section 2.1 the sign of the value of the righthand side F ondif{'erent boundaries of the hypercube K1,. is different:

F(...,0,...)r.>0, F'(...,N,...)r<9. (28)

This suggests that F should have a zero P in the hypercube, that is F(P) : 0due to reasoning as in the mean value theorem. But the mean value theorem isin a straightfonvard manner applicable in a one dimensional situation only. Fortu-nately there is a generalization to the higher dimensional case which is knorvn as

the Poincare-\,[iranda Theoreur.; see [20].

Definition 2.9 Let 1" :: [0,1]' e R" ,k { n and

li,: {r e I" I x1, :0}, If ;: {r : rr : 1}

. called the k-th oTtytos'i,te faces.

Theorem 2.1O (Poincare-Miranda ) Let

.f : I" '---* R", -f : (fr,. ., f")

be a continuous ?nl,p. Ass'ume for all k I n

fnUi) < o, /e(r/) > o

. Ttten there erists

reln: f(t):g

Theorem 2.LL There erists a stat'ionary poi,nt P in the dynam,ical system (2.7) forthe l'inear cose.

ProoftApply the Theorem in [20]. f

Unfortunately, Theorem 2.11 gives no new information when the property C([) : g

is satisfied ,since we know already the stationary point I. But in the linear case,

which is in the focus of this thesis, we have now a stationary point by analyticalreasoning.

15

2.5 A conjecture based on algebraic geometry

In this section rve forurulate a conjecture concerning the stationary points seen froma totally different subject: algebraic geometry.

Definition 2.12

a) An algebraic system of equat'ions ot.rer lR is a system of the Jorrn

fo(Xr,..., Xn) :0 ,'i:7,. -.,m, (2 e)

where /, e RlXr,...,,Y,] are polynomials.

b) A suhset 1/ c CN is called an ffine R - uariety i,ff there i,s a ,systern (2.9).such that V i,s tl'r,e set of all soluti,ons of (2.9) .

Definition 2.L3 Let X be a set and Toyt be a set of subsets of X, such that1. $,X e Top,2. if (At)ier 'is any family Ai e Top, then f,,r, Ai € Top,3. if A1, Az e Top, thert' A1U A2 e Top .

The sets A e Top are called closed sets'in X, Top a totrtology and the pa,ir (X,To,p)a topolog'ical space.

Remark 2.L4 A R-uariety V can be endowed u:i,th the Zariski toytology, r'enderingit a letpologtcal spnce.

Definition 2.15 Let X be a topologi,cal suace.

Th,e Krull rli,ntensi,on dimX of X i,s the supremum of all lengths n of atl cha,ins

Xo C Xr C "' C Xn wi,th Xt. * X*t,'i:0,...tzn-r

of closed, i,rreduci,ble subsets Xi . Also let dirn0 :: *1 .

(See [12] for the definition of "irreduciblerr.)

Remark 2.L6 Although this notion 'is uery general, ,it reduces ,in spec,ial cases toord,'inary not'ions of dimensr,on (eg. dimension of a mani,folcl ).

Definition 2.17 Let A be a commutatiue ring.An i,deal I c A'is a subgroup of A, such that r[ C I for all r e A.

16

First, let N:2. Thenu,eassumefort,r:1,2, Ft,(a,u) :6, F beingtherigthhand-side of our equation. Let V be the corresponding affine varietl,. Let also

A :: IRlo, 1rtt, ..,ut22), I :: (Ftr, .., Frr) C A,

lvhere I is the ideal in Ä generated b1,the fl; . Then we use (see [12]);

Theorem 2.18 (The Krull dimension) W" haue the equ'iau,lence ot' the followi,ngassertions:

(a) di:,nV : 1.

(b) For all P e AlXr,...,)(r], P hom,ogeneous, ulith P([,rr,...,Frr):0 impli,es

P e IAlXr,...,&] .

Conjecture 1:

dimT:1

\,Iost prolrably, it is possible to proof (b) in the Theorern, due to the structure ofthe polynomial in cluestion.

dim I,' : 1 means, that the stationary points lbrm an algebraic curve, if cr is consid-ered as a rariable.If a is a constanl,, then the same conjecture shorvs dim v.- : 0. B), delinition of theKrrrll dimension, there exist m stationary solutions. nt, ) 7 .

L7

Chapter 3

Convolution

This chapter gives some defrnitions and results concerning the operation "convo-lution", mainly because we did not find the discrete 1D, 2D linear convolutionproperly defined in the literature. So it should serve as an overview; see [21]. Herewe use the notation which is comrnon in image processing.

3.1 The discrete lD-Fourier Transform

Letn€N,n)1.\\reset' a.:: exp(2tri,f rt)

and call wn tlte primitive n-th root of unity. Clearly, r,r,e have a*. : L

The map^: C'-*--, C", rvhere

n.- |

4,- » r'nkrx,J : 0,.... n - 1,

k-0

is called lD-discrete Fourier Tlansform (DFT,,). The map-: C, -* C", rvith

n*l

\ :: (tln)\a;ikz1,, j :0,...,n * 7,

k:0

is called inverse discrete Fourier tansform (DFT;l). as it is rvell knorvn,

l', C" -----* C'are linear and bijective maps? with -: -1 .

Obviously, the implementation of the discrete Fourier Transform leads also to animplementation of the inverse discrete Fourier Transform. As it is knoln, the com-putational effort to evaluate the discrete Fourier transform is O(n) when one usesthe idea of "divide and conquer" (fast Fourier transform). The importance of thediscrete Fourier transform consists in the fact that it is the main tool to evaluateconvohrtions.

1B

3.2 The discrete lD-convolutions

Convolution is a binary operation on vectors or matrices.

Definition 3.1- (Linear convolution) Let a:: (as,..,an-r),b:: (b0,..,Öo-r) €C. Define

min{L,n-l)

(o * b); :: I a*bt-r,, I : 0, .., 2n - 2.

k:nraz(0,1-n*1)

a + b is called the linear convolutioa oJ a,b.

Note that o * ö is an (2n - 1)-tuple over C.

Notation:Hadamard product: Let z, y tie complex n-tuples.Then r O y is the cornponent'rviseproducl vecLor.

Padding: Let r € C', then (rl0;) e qn+l ir the vector rvith I zeros at the end.

linear convolutiou can be computed by using the discrete Fourier transform:

Theorem 3.2 Let a,b e C. Then

(a x bl01) : D FT; lD FT2*(a10,) o D FT2"(b10")).

Definition 3.3 (Cyclic convolution) Let a,b e C. We set

n-1

(a8b)7,: I a1,bg--x1 mod n, l:0,. ..,fr-1,,t:0

and ue call a & b the cyclic convolution of a,b.

Theorem 3.4 Let a.b € C". Then

a & b : D FT;i|D FT,(a) o D FT,(b)1 .

The following lemma connects cyclic convolution and linear convolution.

Lemma 3.5 Let a,b e C. Then

(a * bl01) : (a10") I (bI0").

19

For a vector J :: (rr,.., ,n-r) e C', we define

/ *n rn-L ,,\lr.t,,\J"- '

xn*2 ''n /

The rnatrix Z(r) is called the circulant matrix of r.

Lemma 3.6 Let a,b e C. Then

a8b7 bBa: Z(h)a-Z(a)b.

Theorem 3.7 Let h e C". ZUt) has e,illenuecto,rs

and e'i,genualues

g".....!Jn't , a- -- (1.oT....,rf("-")

n-l

40,.., )r-r I Är, : Lhr rf^ .

k:0

3.3 Discrete 2D-convolution

Definition 3.8 Let f :(f(i,,j)) €C",,,h:(h{i,j))e C'".his al*ernet.

(f 'r h)(i, i) ::

(f a h)(t,, j) ::

f (k,l)h(i - k, j - l),i, j :0, .. ., 2n - 2 ;

-0,"'rfr-1'

These are the 2D discrete linear and cyclic convolution, respectively.

Remark 3.9 These can be written as a product of 1D conuolutions:Let a,,b e C". Remember the Gauss kernel

h(, , y) : erp[- (r2 + a2) I Qo2 )l : h' (r)h' (y) where h' (u) :: erp(-u2 I Qo2))

Let a o b :- (or bt)*, e c,,,n, k,l : o, ..,n * 1

min(i,tr-7') müt(j.n-L)

I f tft't)h(i - Ä',i -k:mat (0,i. - n* 1) L:mat (0,i * n* r)n.*1

I /(*, t)h((i, - k) -oa ,,(i - /) *oa n),i,ik,I:O

20

be the dyadic product of the uectors a,b. I'hen

n-1 n-l

(a I h'). (b e ä') : [I "o h'(i. -,t)], II bt h'U * t)lj

A:0 l:0: ll,@*b)h(i' - k, j - t))u,i : (a o b) I /ü .

KI

On the left side is the dyadic product of 1D corvolutions, the riglrt side is a 2Dconvolution.

3.4 The associated C-Operators

Now u,e are prepared to define the C-operator by 2D-convolution which u,e warrt touse in s5-stern(2.1):

Linear convolution: C(tu) :: k * ut

Cyclic convolutior; (l(ur) :: k g w.

Here k : (ki) € lRn'n is a kernel; see Section 2.1.

Using Theorem 3.7 (circulant matrix), we can calculate the eigenmatrices and eigen-values of the cyclic C-operator:

Lemma 3.7O We hu'ue:

C(y* . a") : \* )'n(y* o y")

fnr all mjn :0, ...,ly' - 1 .

Proof :

C(g*.y") : (y*oy")&k: (a^ A k,). (y" A k,)

: (Z(k') y*) c (Z(k') y"): (A* U*) . (\" A"): )_ Ä, (y*.g")

2t

rFor readabilitl,, su1 rn: n:7

t : (l *;,,'11L nir')N-1 N-1

t:0

where a,r is the n-thsynergetics in [1].

: L rr,, erp(N-r2r i.(p + l))P"L

and

y' . yt : (rr ,')r,,erplt{-L2r i(k + t)l

root of unity. These are exactly the expressions obtained by

Lemma 3.7L Suppose tlta,t 1 is an e,igenualu,e of C . Ttten

lrl s t'

Proof:Letubeaneigenvector: C)('u)-^tu,ul0.Letlrr,,l:ma-x{lu;3llz, j:1,...,,^,r}.Then lu1,,l ) 0 and rve have

lrllr,l : lc*(r)l

I » k(t*i,r*i)urili:L,i:t

» k(t * i,r * j)lu*|,i:\,i:l

» k(t-i.r -.7) :1.i.:l,j:l

IEigenvalues of the cyclic C-operator

Eigenvalues of the linear rnap k E, . : ]R.N,N -

IRN,N, k is the shifted Gauss kernel(the maximum is the entry k(0,0) .

We shorv the four largest eigenvalues for different N. The calculations are done withMar:Las.

N:5?i: 1.000 4.762 A.762 0.762

22

l/:9

.^/ : 15

7i : 1.000 0.9183 0.9183 0.9183

7i : 1.000 0.9697 0.9697 0.9697

Eigenvalues of the linear C-operator

Eigenvalues of the map c * . : IR.N,N ---_+ IR.N,N, c is the centered Gauss kernel.Again, rve show the four largest eigenr.alues for clifferent l/ , calculated rvith M.qr-Lns. The other eigenvalues are close, too"

l/:5. ö-i : 0.9092 0.7896 0.7896 0.6857

N:gdr : 0.9661 0.9176 0.9176 0.8716

A/: 15

di : 0.9866 0.9668 0.9668 0.9,175

The results above suggest

Conjecture 2

Let 7N : (rfl,".,7nry,) € l-(N), the vector space of all bounded, real sequences.Then

JIL('N-dN):o'rvhich rve deline as follows:

ve >01,41 € NV,A/ > MVi:7,...,N2 lryr_ä,rl <..(This could be called uniform discrete convergence .) It would be formidableto prove this result. The main difficulty consists iu the fact that the operator C isnot symmetric in general. Therefore the powerful tool of Raleigh-quotients cannotbe applied.

All this holds,for o : 0.6 in the Gauss kernel. Let o :0.1 , so that the Gauss kernelis very narrow. Then for all N, ri we have

Ji:6i:1.0

23

Chapter 4

Stability via Linearization

4.L Linearization

Now we want to calculate the jacobian Dtr(P) of ttre righthand side f in (2.1) ',vhoseeigenvalues may be used to decide whether a stationarl, point is I'stable'r.

Notation:Z :: X e Y is the pointu,ise product of X, Y.

Let F(P) : 0 . We linearize F in the neighborhood of the stationary point P. Asin [17] . lVe use

B:IRN'N_-IRN'N,.^/,^/

Br,(W) :: (112N)(Lr7 + » utj),t,r' :7,. . . ,N, tf W : (.0) .

i:L .i:rUsing this mapping the righthand-side in (2.1) can be fbrmulated in a compact form:

F(P) : aI" + P o C(p) * p e B@n+ pe C(p)) : 0.

LetV€IRN,N.Then

F(v + P): an+ (v+ p) o (cv + cp) * (v +p) o (ar + B((v + p) e (cv + cp))): v o cP + P a cv - (y + P) o B(an+v a cv

+vocP+Pocv+PacP): V oCP + P oCV -V o B(C{L+ p aCp) * p o B(V aCp + p aCV)

.1 I

Cancelling terms due to F(P) : 0 and neglecting terms which are not linear in Vu,e obtain for the jacobian of F in P

DF(P)(v): V aCP + P oCV -V oB(a1 + p oCp)-pe B(V a p + p aCV): v a cP + P ct cv - av *v o B(P a cP) - p a B(v ecp) - p a B(p a cv)

4.2 Stability of stationary points

LetB,(r) :: {u e R.N,N : lr - ul ( r}, r } fl,

rvhere l.l is any matrix norm.

Definition 4.1 The stationary po,int P is called stable i,ff

Ve > Old > OVr € B6(P)(O(t,r) e B,(P) for att r > 0),

where Q is the flow a,ssociated to tlte sohtt'ion with starti,ng poi,nt r . The stat,ionar.ypoint P i,s callerl, o,ttractiue i,ff

ld > OVr € B,,(PX,!*o(l,r) * P) .

Tlr'e stat'ionary poi.nt P ds called asymptoticully stable iff P i,s stable and, attract,iue.

A characterization of the property "asymptotically stable,, is given in

Theorem 4.2 Let P be a stationary point. Then follow,ing assertions are equ,iua-lent:a) P is asymptotically stabl,e.b) P i,s attract,iue.c) Re(Ä) < 0 for all ei,genualues ), of DF(P) .

Proof: See [17]. IA property which we need in the discussion of the simulation results is given in

Definition 4.3 A stat'ionary point P i,s called hyperboli,c i,ff i.t has nonzero, non-'imag'inary, po s,itiu e and neg at,iu e e'ig enu alues on,ly.

25

Chapter 5

Numerical simulation

5.1 Computational details of the simulation

All computations are performed in N,fATLAB.

Our equation (2.1) together rvith an initial condition is an initial value problern(r\-P):

u' : F(tu), u(0) : uoi (5.1)

l/ is the size of the weight matrix ur.

Let tu be a global sohil,ion. For ttre numerical approximation of this solution \re use

the forrvarcl Euler rnethod with step size ä, :

,!n*1 - w" + h,F(tun) , w' : u(0) . (5 2)

Thus, the approximation of the solution are given by the sequence of values u0 , u1 . . .

Notice that each ruk may be considered as matrix in IRN,N.

In the follorving, M is the number of iterations. Therefore tr,If is r.alue of theapproximation of the solution at the "fina} time" T :: Mh.

The linear convolution is reaiized b1, using packages in NIe:rLas. Cyclic convolutionis programmed straight{bnvardll, according to the definition.

We shall use pixels for the entries of a matrix trr: the brighter the pixel, the largerthe entry of ir.

diag stands for the rnain, diagx for the diagonal going from left bottom to righttop.

26

Frorn the above, a rather large number of parameters steers each simulation:

l/ size of the dynamical systeut

a size of the grolvth rate

h stepsize in the Euler method

l,t number -of approximating valnes

i,'.:,; ;""*::il:ffi:o "width" of the gaussian

?uo initial value

q constant in the homotopy

5.2 A combination of cyclic and linear convolution

For combining the cyclic C-operator C" and the linear one C7, u,e introduce a ho-

motopy h :

Encl :: End(RN'N), h: I x Bn62 ---- End

(q,C.,Cr) r_--* 0-dC"*qCt,u,here I :: 10,11 and End are the endomorphisms of IRN'N .

Consider the dynamics fbrq-0: cyclic case;q:1: linear case, N:5, u,here C. is the cyclic and C7 the linearconvolution.

Conjecture 3:

that is, only for g : 0 the system shows the cyclic behavior.If q is only slightly greater than zero, the system switches abruptly into the linearmode. That is suggested by the follo'iving series of test-calculations:

h: l}-a, h,[ : 106,a : 0.1 .

q: l}-a,\0-7,e: 10*10,r1(o) : r * -- > *M a: diag-(4.75)

(the "zeros'r outside the diag- are about 0.05).

Note horv close rve are to 0, yet still linear behaviour.

q:10"2,tu(o) :1- -* > tuL{ xn

Finally the cyclic case; of coursel numerically 10-12 is equal to zero.

27

14

Figure 5.1: The weight matrixleft:linear,right:cyclic, a : 0.95

246a101214

wM as a pixel pattern at the end of an iteration.

5.3 Linear versus cyclic dynamics

Starting point u(0) : diag(1g-'), h : lA-2, M - 10n, I/ : 1b.These will be the parameters, unless otherwise specified.

linear case:

a : 0.95.

f o.os o.e5 o.eb o.eb \^t I o.cs 1 .01 I .o l o.es Ia -l:

1.01 ". : l.ot I

\ o.cs o.e5 o.e5 o.si IFor a comparison betu,een the linear and cyclic case) see Figure 5.1.

a : 0.93.

For the linear case and tr(O) : diag(10*2), we see an abrupt change in the matrixwM for

a : 0.93, 0.923,0.92

For this transition, see Figure 5.2.

l-,- )1!Lr :( äi3;?'

a : 0.5.

0.737713.8602

0

2B

Lin,ear case:

24§8101214

re 5.2: linear: The transition r

2468101214

Figure 5.2: The transition of wM for a : 0.g3 (left) and a : 0.g28 (right)

2 4 6 8 10 12 1t

Figure 5.3'. wtut for a : 0.92

29

Cyclic case:

f n.amt o.13oe o \,.rn, :l o'lsos 1.3'8s74 o

I"-l:::l\0 0 ... rs.rrzn)

Tiny differences occur at the boundaries, because the trvo cases differ mainly there(C-operators!).

a : 0.3, 0.2,0.1

\\'e see the diagonal matrix, for both cases, the diagonal numbers slightly increasingup to 14.78 i

wM x diag(14.78,.. .,14.78) ;

see Figure 5.4.

Figure 5.4:

A typical orbit is shown in Figure 5.5 where a : 0.1 is chosen anclzuy2(t),wz,lt),,wz,z(t)are being plotted.

The values (1.5,7, 1.5) indicate a hyperbolic point.

5.4 Cyclic dynamics with different starting points

We simulate the cyclic dynamics, but now with starting point u(0) :: (Aj|) whereA'ü:0 for all i,, j *t,r and Ali: N. All these initial values are loose peräutationmatrices.

Let N :5,a:0.1 and tir(O; : Äi,1. Then

,rAr : diag(4.7, .. ,4.7) .

2466101214

1: Linear or cyclic, shown toM

30

2

1.5

1

1.5

0la

Figure 5.5: A typical orbit.Linear or cyclic, a : 0.1., plotted al',vayswn(t) , wzz(t) , wz(t')

That demonstrates the strong self-organization capabilities of the system: if rve

have one non-zero entry only in the diagonal, the system moves to the diagonal.N{oreover, the final values wLt ate always permutation matrices.

Let N:5,d:0.1 and'u(O) :41,2. Then uM is the matrix shown in Figure ??.Let N : 5, o : 0.1 and tl(O) : 41,3 . Then tlM is the matrix shown in Figure ??and the orbit in Figure ??.

There is a sharp h;,,perbolic point at iteration step n : 47000 rvhere ,4r1 : 100000.Ttre corresponding matrix is plotted in Figure 5.9.

Let N:5,o:0.1 and tl(0):,42'1. Then wM is the matrix shown in Figure 5.10.

Let,A/ :5,a:0.1 and to(O) :43,3. Then wM is depicted in Figure 5.11 .

Finally, let ly':5,o:0.1 and u,,(O) : A4,4.We show the orbit in Figure 5.12 .

There is indication for an hyperbolic point at about (2.2,2.2,2.2).

Other initial matrices of the form At' show similar results.

Before reaching the endpoint, all orbits rivisitil an hyperbolic point:lrere tu12 : 1lU21 : rD22 :2.2 at iteration step n : 7500.

31

1

0.5

1

'1.5

2

2.5

3

-]

Figure 5.6: tr(0) : A1,2, this the final matrix tuM.

1 1_5 2 2.5 3 3.5 4 4.5 5

Figure 5.7: w(0): A1,3, this the final matrix u.'M

2

2.5

J

4

4.5

5

32

1 1.5 2 2.5 3 3.5 4 4.5 5 5.5

0.8

0.6

0,4

0.2

02.5

Figure 5.8: Here is the orbit with u(0) : A1'3

Figure 5.9: This is the matrix associated to the poiut n:47000 in the last orbit.

0.5

1

2

?.5

3

3.5

4

4.5

Ä

5.50

,,Jd

0.5

1

1.5

2

2.5

J

3.5

4

4.5

5

5.50

0.5

1

1.5

2

2.5

3

3.5

1 1.5 2 2.s 3 3.5 4 4.5 5

Figure 5.10: T.u(O) : A2,1, shown trM.

1 1.5 2 2.5 3 3.5 4 4.5

Figure 5.t1: .u(0) -- A',t,shown urrl

34

?.5

2

1.5

1

0.5

05

Figure 5.12: The orbit rvith w(0): tr+'t.

5.5 A different C-operator: the Sobel-filter

For a change, rve finish u,ith an un-biological kernel, called Sobel filter, u,hich is

usecl in irnage processing for enhancing edges:

/t 2 1\sn:{o o ol

\-r *2 -t)We choose ly' : 5. Therefore the matrix So is enlarged by filling up 'lvith zeros insuch a way that So is located in ttre center.

Consider ur(0): diag(10-2)o:0.1. The resultiug matrix wM is drawn in Figure5.13:

The rate of convergence is lorver than rvith the Gauss filter for the same other pa-

rameters. uLM is stable,r,vith four complex eigenvalues; notice that the Sobel-filtercontains negative entries.

5.6 Sensitive behavior with respect to trO

One aspect of complex systems is the sensitivity with respect to the initial matrix.This is being illustrated by the nexL two examples. Set -lü:5,a:0.1 .

Linear case,

tuo:f +10-2 I

35

0.5

1

1.5

2

?.5

3

4

4.5

5

5.5 rAA

The orbit is shown inThe cyclic case gives

Now we try a slightlyLinear,

Figure 5.13: The Sobel-filter, shownlr

'tD"'

.M: diag(4.75)

5.14.the same resuit.

different perturbation of 1[.

tuo:1[+i0-2

with orbit 5.15 .

The cyclic case gives a broken di,ag" .

5.7 Eigenvalues of the

The eigenvalues of the linearization inthe stationarl, point is stable.

,nM : diag.(4.75)

So tiny differences produce rroppositerr results.

linearized system

a stationary point are used to decide whether

36

1 1.5 2 2.5 3 3.5 4 4.5

10

Figure 5.14: First perturbation of I.Note that there is hardly any hyperbolic point.

Figure 5.15:point.

Second perturbation of X.Note that there is a probable hyperbolic

1.4

1.2

.t

0.8

0.6

0.4

0.2

02.5

00

J/

Cyclic case :

Let N : 5,0 : 0.1, and tu(0) : I.The eigenvalues Ä; of the linearization DF(wM), utilI :1[ are gir,-en in the follou,ingtable:

0.48 0.48 0.48 0.48 0.200.20 0.20 0.20 0.20 0.200.20 0.20 0.06 0.06 0.060.06 -0.22 *0"22 -0.22 -0.22

-0.40 -0.40 *0.40 -0.40 -1.i0Let y, 'i: 1,...,1/2, be the eigenvalues of the C-operator.

Result according to the simulation:1. All eigenvalues have multiplicity 4 or B (except 1).2. \\,'e have Äl - h - a (except at the boundary of the matrix )These properties are in agreement wittr [1]. 'Ihe point u,M seems to be hyperbolic .

Linear case :

Let l/ :5,a: 0.1, and ur(O) : I.Tlre eigenvalues ); of the linearization DF(ru^r'), lDM : diag-(,X.76) arer gir,en in thefollorving table:

-1.18 -1.19 -1.23 -1.27 -1.2{)**+***+********* * *-2.30 *2.37

Result of the simulation:1. All eigenvalues have algebraic multiplicity 1., \. J ^.a. /\z 7- lt u.

3. The point torr seems to be stable.

stable, too.

3B

Conclusion

We have seen that the complex self-organization properties of the Häussler-vonder Malsburg equations are retained when using linear convolution instead of theoriginal cyclic convolution - which is biologically more realistic. From many startingpoints the dynamics reaches one of the 2 diagonal weight matrices. The differencelies in the algebraic multiplicities of the linearized eigenvalues. This phenomenonfits exactly in the theory of circulant matrices, which describe the cyclic convolution.

Moreover the dynamics is complex in the sense that tiny changes in the initialvalues can produce very different final statioaary states.

Another result is that the biological hypercube K1,, is an attractor for the system.

Also we proved the existence of a stationary solution for the linear convolution,through the Poincare-Miranda theorem.

We gave also a conjecture about the eigenvalues of the C-operators as l[ goes tooo. An interesting future investigation woüld be the introduction of a model whichdescribes the system when N goes to oo.

39

?

Bibliography

[1] C. von der Malsburg A. F. Häussler. Development of retinotopic projections:an analytic treatment. J. Theoret. Neurob'ia., 2:47-73, 1983.

[2] D.J. Willshaw C. von der Malsburg. How to label nerve cells so that they caninterconnect in an ordered fashion. Proc. Natl. Acad. Sci,. USA,74:5L76*5L78,1977.

[3] D.J. Willshaw C. von der Malsburg. Differential equations for the developmentof topological nerve fibre projections. SIAM-AMS Proc.,13:39-47, 1981.

t4l C. von der Nlalsburg. The correlation theory of brain function. MPI Jor Bio-physi,cal Chemi,stry, Internal Report B 1-2: 1-80, 1981.

[5] C.von der Malsburg J.Zht. Maplets for correspondence-based object recogni-tion. N eural N etw orks, 1 7(B-9) : 131 1*1326, 200 4.

[6] C.von der Malsburg J. Zfui. Associative memory of connectivity patterns.N eurocomputi,ng, 69 :1305-1308, 2006.

[7] H. Haken. Synerget'ics,Introd,uct'ion. Springer,Berlin, 1978.

[Bl H. Haken. Generalized ginsburg-landau equations for phase transition-like phe-nomena in lasers,non-linear optics, hydrodynamics and chemical reactions. Z.Physi,k B, 21:105*174, 7975.

[9] H. Haken. Higher order corrections to generalized ginsburg-landau equationsof non-equilibrium systems. Z. Physi,k 8,22:69*72,1975.

[10] J. Zhu. Analysis of map formation in visual perception. FIAS, pages 1-14,2007.

[11] M. Guessmann. Selbstorganisation zwischen Mannigfaltigkeiten euklidischerund nichteuklidischer Geometrie durch Kooperation und'Kompetition. D'isser-tat'ion, Uni, Stuttgart, 20A6.

[12] E.Kunz. Ei,nführu.ng i,n die algebraische Geometrie. Vieweg, Braunschweig,1997.

[13] F. Lorenz. L'ineare Algebra L BI Verlag, Mannheim, 1984.

[14] S. Lang. Algebra. Addison-Wesley, Reading, 1984.

I

40

[lS] G.Tnervs. Ph,ysiologie des Menschen. Springer, Berlin, 1977.

[fO1 iV. de 1\Ielo. The Geometry of Dynam,ical Systezns. Springer, Berlin, 1982.

[17] J. Baumeister. Differerrtialgleichungen. Script, 2006.

[18] Teschl. I)ynamisc]re Systeme. Scri'pt, 20A5.

[19] W. \&'alter. Gewöhnl'iche Di'fferenti,alglei,chungen. Springer, Berlin, 2000.

[20] W. Kulpa. The Poincare-1\{iranda theorem. The american mathemat'i,c:al

monthly, 104(6): 5a5-550, 1997.

[21] J. Baumeister. Itrverse Probleme. Scri4tt,2007-

4t