Single-variable reaction systems: Deterministic and stochastic models

This article appeared in a journal published by Elsevier. The attachedcopy is furnished to the author for internal non-commercial researchand education use, including for instruction at the authors institution

and sharing with colleagues.

Other uses, including reproduction and distribution, or selling orlicensing copies, or posting to personal, institutional or third party

websites are prohibited.

In most cases authors are permitted to post their version of thearticle (e.g. in Word or Tex form) to their personal website orinstitutional repository. Authors requiring further information

regarding Elsevier’s archiving and manuscript policies areencouraged to visit:

http://www.elsevier.com/copyright

http://www.elsevier.com/copyright

Author's personal copy

Single-variable reaction systems: Deterministic and stochastic models

M.N. Steijaert a, A.M.L. Liekens b, D. Bošnacki a, P.A.J. Hilbers a, H.M.M. ten Eikelder a,*

a Department of Biomedical Engineering, Eindhoven University of Technology, P.O. Box 513, 5600 MB Eindhoven, The Netherlandsb VIB Department of Molecular Genetics, University of Antwerp, Universiteitsplein 1, B-2610 Wilrijk, Belgium

a r t i c l e i n f o

Article history:Received 26 March 2010Received in revised form 23 June 2010Accepted 29 June 2010Available online 14 July 2010

Keywords:Chemical master equationReaction kineticsBistabilityQuasi-stationarityPotential function

a b s t r a c t

Biochemical reaction networks are often described by deterministic models based on macroscopic rateequations. However, for small numbers of molecules, intrinsic noise can play a significant role and sto-chastic methods may thus be required. In this work, we analyze the differences and similarities betweena class of macroscopic deterministic models and corresponding mesoscopic stochastic models. We deriveexpressions that provide a clear and intuitive view upon the behavior of the stochastic model. In partic-ular, these expressions show the dependence of both the dynamics and the stationary distribution of thestochastic model on the number of molecules in the system. As expected, most properties of the stochas-tic model correspond well with those in the deterministic model if the number of molecules is largeenough. However, for some properties, both models are inconsistent, even if the number of moleculesin the stochastic model tends to infinity. Throughout this paper, we use a bistable autophosphorylationcycle as a running example. For such a bistable system, we give an explicit proof that the rate of conver-gence to the stationary distribution (or the second eigenvalue of the transition matrix) depends exponen-tially on the number of molecules.

� 2010 Elsevier Inc. All rights reserved.

1. Introduction

In the past decades, new biochemical techniques have led to alarge increase in knowledge about biology at the molecular level.Simultaneously, a growing awareness has emerged that reduction-ism alone is insufficient for unraveling the complex interactionswithin biochemical networks. This awareness has led to the riseof the holistic field of systems biology, in which wet-lab experi-ments are combined with (multiscale) computational modeling.Traditionally, most of the systems investigated in this field are de-scribed by deterministic models based on ordinary differentialequations (ODEs). In addition, there is a growing interest in sto-chastic modeling techniques [1,2]. The advantage of those stochas-tic techniques is that they explicitly take into account the intrinsicnoise that is present in real-life biochemical networks.

In this paper, we focus on the relations between stochastic anddeterministic models for a certain class of biochemical systems.More specifically, we consider systems consisting of a number ofsimilar molecules that can each be in two ‘configurations’. Supposethat the reversible interconversion between those configurations isdefined by a macroscopic deterministic model based on kineticrate laws. In principle, such a deterministic model describes thebehavior of a reaction system for very large numbers of molecules.

For a finite number of molecules, we can also introduce a meso-scopic1 stochastic Markov model. The parameters for such a modelcan be derived from the deterministic model. It is generally assumedand for some elementary cases even proven that the expectedbehavior of the stochastic model corresponds with the dynamics ofthe deterministic model if the number of molecules in the describedvolume is large enough [3]. For many smaller systems, a determinis-tic model also yields a reasonable approximation, which can be ana-lyzed more efficiently than the full stochastic system [4]. However,there are also systems that display significantly different behaviorif the number of molecules is relatively small [5].

For some systems it is not trivial to explain how the expectedbehavior of the stochastic model relates to that of the deterministicmodel. This paper studies the relation between the stochastic anddeterministic models for a specific class of reaction systems. As arunning example, we study a bistable reaction system. More pre-cisely, we use both a deterministic and a stochastic model to de-scribe a highly idealized reaction module consisting of aphosphorylation reaction, a de-phosphorylation reaction and atrans-autophosphorylation reaction. This running example, furtherreferred to as ‘autophosphorylation cycle’, illustrates the paradox-ical combination of bistability and stochasticity. After all, the long-term behavior of a stochastic model of a chemical reaction system

0025-5564/$ - see front matter � 2010 Elsevier Inc. All rights reserved.doi:10.1016/j.mbs.2010.06.006

* Corresponding author. Tel.: +31 40 247 5153.E-mail address: [email protected] (H.M.M. ten Eikelder).

1 Following [10], we use the term mesoscopic to refer to the stochastic descriptionof a system in terms of macroscopic variables.

Mathematical Biosciences 227 (2010) 105–116

Contents lists available at ScienceDirect

Mathematical Biosciences

journal homepage: www.elsevier .com/locate /mbs


is in general independent of its initial state, which intuitively con-flicts with the required bistability.

In Section 3, we derive a potential function that provides aframework for the analysis of the stochastic model for both smalland larger numbers of molecules. This function is used to describethe stationary distribution of the stochastic model and the ex-pected transition times (see Section 4). The expected times of tran-sitions between individual states are also related to the rate atwhich the probability distribution converges to the stationary dis-tribution. This convergence clearly depends on the eigenvalues ofthe transition matrix of the stochastic model. We prove in Section 5that under certain conditions, the transition matrix has an eigen-value which converges to zero exponentially fast with an increas-ing number of molecules. In each of Sections 2–5, we firstintroduce the generic theory and subsequently exemplify thisusing the autophosphorylation cycle introduced in Section 2. A dis-cussion of both the generic theory and the autophosphorylation cy-cle follows in Section 6. In this section, we also discuss theresemblances and differences between our work and some classicand more recent research papers in the field of statistical physics.

2. Two models

2.1. Model definitions

In this paper, we focus on systems in which each molecule canbe in two ‘configurations’. This type of system occurs in many dif-ferent biological processes. For instance, many signaling proteinscan be modified by reversible post-translational modificationssuch as phosphorylation or methylation [6]. Other examples in-clude molecules that can switch between different conformationsor localizations within a cell.

The generic system consists of a total of N molecules, which canbe in either of the configurations X0 and X1, and the overallreactions

X0 ¢ X1

To allow a comparison between systems with various values of N,we define the proportion x of the molecules that is in the X1 config-uration. This proportion can be derived from the number of mole-cules (denoted with #) or from the concentrations (in squarebrackets):

x ¼ #X1

#X0 þ#X1¼ #X1

N¼ ½X1�½X0� þ ½X1�

:

As mentioned before, we will compare two representations of thisgeneric system.

The first representation of this system is the ‘deterministicmodel’. In this representation, we use a system of ODEs to describethe time evolution of x. The rate at which x changes due to thereaction X1 ? X0 is given by a real, smooth, non-negative functiona(x); the rate of the opposite reaction is defined by a real, smooth,non-negative function b(x). We further assume a(x) > 0 for all0 6 x < 1, a(1) = 0, b(x) > 0 for all 0 < x 6 1 and b(0) = 0. The dynam-ics of the deterministic model are given by

dxdt¼ aðxÞ � bðxÞ: ð1Þ

The second representation of the system described above is the ‘sto-chastic model’. In this representation, we consider all possible‘microstates’ (further ‘states’), each of which corresponds with aninteger number of molecules in configuration X1. Hence, a modelwith N molecules has N + 1 possible states, which are numberedaccording to the number of X1 molecules (see Fig. 1). We allow onlyone reaction to occur at a time; hence, from each state only direct

neighbor states can be reached in one reaction step. Due to this‘one-step-at-a-time property’ and the Markovian properties of themodel, the model is in fact a birth-and-death process [7] in whichthe forward and backward propensities ak and bk, have the role ofbirth and death rate, respectively.

One of the obvious differences between both models is that N is aparameter of the stochastic model but not an explicit part of thedeterministic model. Consequently, the expected behavior of thestochastic model changes with growing N, while the deterministicmodel is independent of N. To allow a useful comparison, the sto-chastic models with different values of N must all have the same to-tal concentration as is used in the deterministic model. This meansthat the volumes of the stochastic models scale linearly with N.

The propensities of state change ak and bk in the stochasticmodel can be related to the deterministic production rates a(x)and b(x) as follows:

ak ¼ NakN

� �; ð2Þ

bk ¼ NbkN

� �: ð3Þ

Using those expressions, we ensure that both models act on thesame time scale. The relations in (2) and (3) can be used for all uni-molecular and pseudo-unimolecular reactions (i.e., reactions inwhich the total number of molecules does not change). In thisway, assumptions about the deterministic model are transferredto the stochastic model. As a result, elementary steps that are hid-den in the deterministic model remain hidden in the stochasticmodel. For the Michaelis–Menten reaction, this is discussed in [8].

Let z(t) be the probability distribution vector, which containsthe probabilities zk(t) that the system is in a state k 2 {0, . . .,N} ata time t. The dynamics of the state probabilities in the stochasticmodel are described by the Chemical Master Equation [9]:

dzdt¼M � z; ð4Þ

where the tridiagonal matrix M = (mij) (with i, j = 0,1, . . .,N), is given by

mij ¼

aj if j ¼ i� 1;bj if j ¼ iþ 1;�ai � bi if j ¼ i;

0 otherwise:

8>>><>>>:

The stationary distribution for such a birth-and-death process fol-lows directly from its (stochastic) detailed balance property [10]and is given by:

Zk ¼Qk�1

i¼0ai

biþ1PNj¼0

Qj�1i¼0

aibiþ1

� � : ð5Þ

2.2. Trans-autophosphorylation

Throughout this paper we exemplify our methods with thebistable reaction system given by the following reactions:

Xþ S !k1 ;j XP þ S; ð6Þ

Xþ XP !k2 ;j 2XP; ð7Þ

XP þ U !k3 ;j Xþ U: ð8Þ

0 1 2 N-1 N

N-1

N2

10

1β β β

α α α

Fig. 1. The N + 1 states of the stochastic model.

106 M.N. Steijaert et al. / Mathematical Biosciences 227 (2010) 105–116


This system consist of proteins X that can be phosphorylated into XP

by a kinase S. In addition, protein molecules X are also subject to aninter-molecular (or trans-) autophosphorylation reaction in whichphosphorylated kinase molecules XP phosphorylate other moleculesX. Finally, the phosphatase U de-phosphorylates XP. We further re-fer to this system as the ‘autophosphorylation cycle’.

For this specific case study, x ¼ ½XP �½X�þ½XP �

¼ #XPN represents the pro-

portion phosphorylated protein. We assume Michaelis–Mentenkinetics with identical normalized Michaelis constants j for allthree reactions, catalytic rate constants k1 and k2 for the phosphor-ylation reactions (6) and (7), respectively, and the maximum veloc-ity k3 for the de-phosphorylation reaction (8) with a constantphosphatase concentration [U]. Hence, the reaction rates for thedeterministic model are given by

aðxÞ ¼ k1ð1� xÞsjþ 1� x

þ k2xð1� xÞjþ 1� x

;

bðxÞ ¼ k3xjþ x

:

For reasons of simplicity we use dimensionless parameters. We fur-ther use k1 = k2 = k3 = 1 and j ¼ 1

20, although similar results can beobtained for a broader range of parameters. Note that the onlyremaining parameter is the ‘input signal’ s, which represents the to-tal amount of kinase S.

For both models of the autophosphorylation cycle, we can easilyinfer the long-term behavior. The deterministic model has one ormore steady states that can either be stable or unstable. As shownin Fig. 2(a), the number of steady states depends on s. For input sig-nals s 6 r2 � 0.651 the system has a stable steady state x = c1(s)with a low proportion of phosphorylated molecules, while fors P r1 � 0.373 there is a stable state x = c3(s) with a high propor-tion of phosphorylated molecules. The most interesting parameterregion is r1 6 s 6 r2, where both stable steady states c1(s) and c3(s)exist, separated by an unstable steady state x = c2(s). If s is variedwithin this ‘bistable range’, the model displays hysteresis. In theremainder of this paper, we omit the argument s and write c1 in-stead of c1(s), etc.

For the stochastic model, a first indication of the long-termbehavior is given by the stationary distribution Z, which dependson both the input parameter s and the number of molecules N. Typ-ically, this distribution Z is bimodal for s within the bistable range(see for instance Fig. 2(b)) and unimodal for all other values of s. Inanalogy with the c1, c3 and c2 for the deterministic model, we referto the left local maximum, the right local maximum and the inter-mediate minimum of the bimodal distribution as G1, G3 and G2,respectively. The maxima of the unimodal distributions are re-ferred to as G1 for s < r1 and G3 for s > r2. We are primarily inter-ested in the behavior of this reaction system in the bistable

range. In Section 3.2, we discuss how the corresponding bimodalstationary distributions depend on both s and N.

3. Asymptotic form of the stationary distribution

3.1. The kinetic potential F

In this section, we rewrite the stationary distribution in such away that its behavior for N ?1 can easily be inferred. It is easilyseen from (5) that the ratio of the stationary probabilities of statesk and ‘ (with k 6 ‘ < N) equals

Z‘Zk¼Y‘�1

i¼k

ai

biþ1¼ ak

a‘

Y‘i¼kþ1

ai

bi¼

a kN

� �a ‘

N

� � Y‘i¼kþ1

a iN

� �b i

N

� � : ð9Þ

We first rewrite this expression such that its behavior for large Nbecomes more clear. Define F and Qk,‘ by

FðxÞ ¼ �Z x

0ln

aðyÞbðyÞ

� �dy; ð10Þ

Qk;‘ ¼Y‘

i¼kþ1

a iN

� �b i

N

� � : ð11Þ

Note that F does not depend on the number of molecules N. In fact Fhas the role of a ‘potential function’. As it is purely derived from thekinetic description of the reaction system, this function is furthercalled the ‘kinetic potential’. A further discussion about the use ofthis and related potential functions in the literature is given inSection 6.3.

Trivially

1N

ln Qk;‘ ¼1N

X‘i¼kþ1

lna i

N

� �b i

N

� � !

¼ 1N

X‘i¼kþ1

ciN

� �; ð12Þ

with the function c defined by

cðxÞ ¼ lnaðxÞbðxÞ

� �: ð13Þ

The right-hand side of (12) can be seen as a Riemann sum, i.e., as anapproximation of an integral. The rectangle rule states that for suf-ficiently smooth functions cZ b

acðxÞdx ¼ h

Xm

i¼1

cðaþ ihÞ þ Dða; b;hÞ;

in which h ¼ b�am is the step size and the error D(a,b,h) has the

asymptotic behavior

Dða; b;hÞ ¼ h2ðcðaÞ � cðbÞÞ þ Oðh2Þ;

Fig. 2. (a) The steady states of the deterministic model of the autophosphorylation cycle as a function of s. Solid lines denote stable steady states; the dotted line denotes theunstable steady state. (b) The stationary distribution of the stochastic model for r1 < s = 0.55 < r2 and N = 100.

M.N. Steijaert et al. / Mathematical Biosciences 227 (2010) 105–116 107


see for instance [11]. Using this relation we can rewrite (12) as

1N

ln Q k;‘ ¼Z ‘

kcðxÞdx� 1

2Nc k� �� c ‘� ��

þO N�2� �

¼ �F ‘� �þ F k� �� 1

2Nc k� �� c ‘� ��

þO N�2� �

; ð14Þ

where ‘ � ‘=N and k � k=N. Eliminating c with (13) finally results in

Q k;‘ ¼ eNðF kð Þ�F ‘ð ÞÞ

ffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffia ‘� �

b k� �

b ‘� �

a k� �

vuuut 1þOðN�1Þ� �

: ð15Þ

The relations (14) and (15) only hold for 0 < k, ‘ < N. In other cases,(14) does not hold since either c k

� �or c ‘

� �is undefined. Substitu-

tion of (15) in (9) gives

Z‘Zk¼ eNðF kð Þ�F ‘ð ÞÞ

ffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffia k� �

b k� �

a ‘� �

b ‘� �

vuuut 1þOðN�1Þ� �

: ð16Þ

Eq. (16) expresses the ratio between the stationary probabilities ofstates k and ‘ for a stochastic system with a given value of N. Nowwe want to know how such a ratio changes for increasing N. Sincewe want to compare those ratios for similar macroscopic situations,we let k and ‘ depend on N in such a way that k and ‘ tend to con-stant values ~k and ~‘ for large enough N. More precisely, we considerthe behavior of the right-hand side of (16) for the case

k � kN! ~k and ‘ � ‘

N! ~‘ if N !1:

Using (16) we see that the behavior of Z‘/Zk is determined by thesign of the difference F ~k

� �� F ~‘� �

. If this difference is negative,the quotient Z‘/Zk will increase exponentially with N; if this differ-ence is positive, the quotient will decrease exponentially with(increasing) N. Only if F ~‘

� �¼ F ~k

� �, then Z‘/Zk will tend to a finite

constant value if N increases. This implies that, for sufficiently largeN, the maxima of the stationary distribution correspond with theminima of F and the minima of the stationary distribution corre-spond with maxima of F. Clearly, the kinetic potential F determinesthe behavior of the system for large values of N.

For the deterministic system, it is easily verified that dFðxðtÞÞdt 6 0

for solutions x(t) of (1) and that stable/unstable steady states of(1) correspond with internal minima/maxima of F. This means thatF can be used to construct a Lyapunov function for (1).

Above we have shown that (for large enough N) the locations ofthe minima/maxima of the stationary distribution correspond withthe locations of the maxima/minima of F. As discussed above, theunstable/stable steady states of the deterministic model also corre-spond with the maxima/minima of F. Consequently, if N is large en-ough, the locations of the maxima/minima of the stationarydistribution of the stochastic model correspond with the stable/unstable states of the deterministic model.

3.2. Asymptotic behavior of the autophosphorylation cycle

Now we apply the methods from Section 3.1 on the autophos-phorylation cycle. Fig. 3 shows the kinetic potential F for inputs = 0.52. Typically, within the bistable range, this function hastwo local minima. For all s in the bistable range, the function Fhas two local minima in c1 and c3, separated by a local maximumin c2. This also means that for large enough N the locations of theextrema G1, G2 and G3 of the stationary distribution correspondwith c1, c2 and c3, respectively.

The relative order of the two maxima of the stationary distribu-tion in G1 and G3 can also easily be found. For s < r0 � 0.5297 theglobal minimum of F is in c1, while for s > r0 the global minimum

of F is in c3. Now consider the case s < r0 (as shown in Fig. 3). Inthat case, F(c1) < F(c3) and thus (16) implies that the global maxi-mum of the stationary distribution is in G1, if N is sufficiently large.Moreover, the quotient of the probabilities in G3 and in G1 tends tozero exponentially fast in N. Hence, although the local maximum ofthe stationary distribution in G3 exists for all N, this maximum be-comes exponentially small compared to the dominant maximumin G1. In fact, for large N and s < r0, most probability in the station-ary distribution is concentrated around G1. This means that, fort ?1, the system will almost always be near c1. Similarly, if theinput signal s > r0, the dominant maximum of the stationary distri-bution will be in G3. In this case, the system will almost always benear c3 for large t. Only for s = r0 both minima of F have the samevalue and both local maxima of the stationary distribution havenon-vanishing (but not necessarily equal) probabilities. The pre-dicted dependence of the stationary distribution on s and N can in-deed be observed in Fig. 4. To allow a better comparison of variousvalues of N, this figure shows N Zk as a function of the proportionphosphorylated proteins k/N.

For s – r0, the previous considerations imply that the stationarydistribution is ‘hardly bimodal’ if N is large. This contrasts with thedeterministic model, which has two stable steady states for all s inthe interval (r1,r2). However, note that we are considering the sta-tionary distribution, i.e., the distribution for t ?1. The systemmay still exhibit bistable behavior for large (finite) times. In thenext section, we show that the time needed before the systemloses its bistability increases exponentially with N.

4. Expected transition times

4.1. Single-step and multi-step times

It is often interesting to see how fast one state in the stochasticmodel is reached from another state. Consider for example theaforementioned autophosphorylation cycle with s is in the bistablerange. The stochastic model predicts that, in the long run, the sys-tem can most likely be found in one of the maxima of the station-ary distribution. However, the stationary distribution provides noinformation about the time required to reach one of those maximafrom a given initial state. In this section, we show how the kineticpotential F can be used to calculate the expected time T k!‘ for atransition from state k to state ‘.

Let us first focus on the expected transition times betweenneighboring states. The expected time for a transition from statek to state k + 1 is given by

T k!kþ1 ¼Xk

i¼0

1ai

Yk

j¼iþ1

bj

aj; ð17Þ

see for instance [7,10]. This ‘single-step time’ T k!kþ1 takes into ac-count all possible paths from k to k + 1, including paths that visit

–0.040

–0.02

0

0.02

1x

F (x

)

Fig. 3. The kinetic potential F for input s = 0.52. Since 0.52 < r0, the global minimumof F is in c1.



states ‘ < k. We first analyze how T k!kþ1 changes if N tends to infin-ity. Rewriting (17) in terms of the functions a(x) and b(x) yields

T k!kþ1 ¼Xk

i¼0

1Na i

N

� � Yk

j¼iþ1

b jN

� �a j

N

� � ¼Xk

i¼0

1Na i

N

� �Q i;k

;

in which Qi,k is defined by (11). We would like to use the expressionfor Qi,k given in (15). However, that expression does not hold fori = 0. Therefore, for 0 < k < N, we separate the summation as follows:

T k!kþ1 ¼b 1

N

� �Nað0Þa 1

N

� �Q1;kþXk

i¼1

1Na i

N

� �Q i;k

:

Since b 1N

� �¼ OðN�1Þ, the first term is OðN�1Þ times the second term.

Hence we can write

T k!kþ1 ¼Xk

i¼1

1Na i

N

� �Q i;k

!1þOðN�1Þ� �

: ð18Þ

Using (15), we can rewrite (18) as

T k!kþ1 ¼Xk

i¼1

1Na i

N

� � eN F kNð Þ�F i

Nð Þð Þffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffia i

N

� �b k

N

� �b i

N

� �a k

N

� �s

1þOðN�1Þ� �

: ð19Þ

We now study the behavior of T k!kþ1 in the limit N !1; k! ~kfor various points ~k. From (19) follows that

T k!kþ1 ¼ OðN�1Þif FðxÞ > F ~k

� �for all x < ~k;

T k!kþ1 > C ecN ðfor positive constants C and cÞif FðxÞ < F ~k

� �for any x < ~k:

The first case will be further referred to as ‘non-exponential single-step times’, while the second will be referred to as ‘exponentialsingle-step times’. A similar dependence on F can be found forsingle-step times T k!k�1 in the opposite direction.

The reason for the occurrence of exponential single-step timesis that a path from state k to k + 1 may include ‘detours’ via statesm with m < k. Both exponential and non-exponential single-steptimes include this kind of detours, for which the probability de-creases with N while the expected time increases with N. The dif-ference between the two types of single-step times is that for thenon-exponential type the decreasing probability of detours cancelsout their growing expected times, while for the exponential typethe expected times of some rare paths increase faster in N thantheir probabilities decrease.

For the non-exponential single-step times, a more precise anal-ysis yields that

T k!kþ1 ¼1

Nða ~k� �� b ~k� �Þþ OðN�2Þ for N !1:

Note that this behavior corresponds with that of the deterministicmodel, which moves from any y ¼ ~k with a(y) > b(y) toyþ Dy ¼ ~kþ 1

N in approximately a time 1NðaðyÞ�bðyÞÞ.

Next, we consider the expected time for a transition from state kto a state ‘ with ‘ > k, which is given by

T k!‘ ¼X‘�1

i¼k

T i!iþ1: ð20Þ

The combination of this expression with (19) yields an expression ofthe expected transition time (‘multi-step time’) between states kand ‘ (with ‘ > k) in terms of the kinetic potential F. The multi-steptime T k!‘ for a transition with approximately constant k and ‘, canbe found by summingOðNÞ single-step times. If all those single-steptimes are of the non-exponential type, we find T k!‘ ¼ Oð1Þ. How-ever, if one or more of those single-step times are of the exponentialtype, we find T k!‘ > C ecN (with positive constants C and c). Hence,the way multi-step times depend on N is determined by the shapeof F on the interval 0; ~‘

.

4.2. Transition times in the autophosphorylation cycle

As discussed above, the shape of the kinetic potential F deter-mines how single-step and multi-step times depend on N. Wenow apply this knowledge to the autophosphorylation cycle.Fig. 5 shows F for the autophosphorylation cycle with s = 0.55. Inthis figure, five regions (I–V) can defined, which (as discussed be-low) determine the behavior of both the stochastic and determin-istic model. Since F has a similar shape for all s in (r0,r2), theregions I–V can be defined for each s in this interval. This is shownin Fig. 6. For all s in (r0,r2), both models show the same qualitativebehavior in the same regions. Similar, but mirrored, results can beobtained for all s in (r1,r0).

The region to which ~k belongs determines the asymptoticbehavior of T k!kþ1. For ~k in regions I and IV, there are no x < ~k,for which FðxÞ < F ~k

� �. Therefore, all T k!kþ1 are of the non-expo-

nential type. However, for each ~k in regions II, III and V, there

15

10

5

0 0.5 1 0 0.5

k / Nk / N

(a) s = 0.52 (b) s = 0.52

N Z

k

N Z

k

1

2

4

6

8

10N = 100

N = 200

N = 300

Fig. 4. The scaled stationary distributions NZk for s = 0.52 and 0.55, and various N as a function of the proportion phosphorylated proteins k/N.

–0.060 0.5 1*

–0.04

–0.02

0

x

F (x

)

I II III IV V

Fig. 5. Function F for s = 0.55 with regions I–V. The symbol ‘*’ denotes the value of x(with c2 < x < c3) for which F(x) = F(c1).



always is an x < ~k, for which FðxÞ < F ~k� �

. Hence, in those regions,all single-step times T k!kþ1 are of the exponential type. Remarkably,region III exemplifies that even single-step times in the direction ofincreasing stationary probability can grow exponentially with N.

Obviously, the shape of F also determines how single-step timesin the opposite direction depend on N. For ~k in region V, there is nox > ~k, for which FðxÞ < F ~k

� �and T k!k�1 is of the non-exponential

type. For ~k in any of the other regions, there is always a x > ~k withFðxÞ < F ~k

� �. Consequently, all corresponding single-step times

T k!k�1 are of the exponential type.We now compare the single-step times of the stochastic model

with the behavior of the deterministic model. Table 1 shows foreach region the types of single-step times in both directions andthe sign of the derivative of the deterministic model. Recall thatthe deterministic model moves from any y ¼ ~k with a(y) > b(y) toyþ Dy ¼ ~kþ 1

N in a time OðN�1Þ; analogously, it moves from anyy ¼ ~k with b(y) > a(y) to y� Dy ¼ ~k� 1

N in a time OðN�1Þ.In regions I, IV and V the deterministic model moves in the

direction for which the single-step time in the stochastic modelis non-exponential. On average, the stochastic model makes a stepof size 1/N in this direction in a time OðN�1Þ. This is in good agree-ment with the deterministic model, which moves over the samedistance in a time OðN�1Þ. Contrastingly, for regions II and III, sin-gle-step times in both directions are of the exponential type. Forboth of these regions the deterministic model moves in the direc-tion of the local minimum of F; again, the deterministic modelmoves over an interval of size 1/N in that direction in a timeOðN�1Þ. This marks an inconsistency between the stochastic andthe deterministic model.

For both regions II and III it can be shown that the occurrence ofexponential single-step times is due to rare detours that pass thelocal maximum in F. It is possible to rewrite (17) in such a way thatsingle-step times can be expressed in terms of groups of possiblepaths from k to k + 1. In particular, we are interested in paths thatvisit some state ‘ (with ‘ < k), but not states ‘ � 1. We refer to theprobability for such a path as Pð‘Þk!kþ1. The expected time to go from

state k to state ‘ (without visiting state k + 1) and then to state k + 1(without visiting state ‘ � 1) is further referred to as T ð‘Þk!kþ1. Thesingle-step time from k to k + 1 can now be expressed as follows:

T k!kþ1 ¼Xk

‘¼0

Pð‘Þk!kþ1Tð‘Þk!kþ1:

As shown in Fig. 7 for the case k = N/2, this separation according to‘path diameter’ clearly uncovers the occurrence of the aforemen-tioned rare paths. With increasing N, the probability of visitingstates with ‘� c2 decreases (see Fig. 7(a)), while the correspondingexpected time increases (see Fig. 7(b)). The net effect is an (approx-imately exponential) increase in the product of the probability andexpected times for some ‘� c2 (see Fig. 7(c) and the detailed viewin Fig. 7(d)). By omitting all paths with ‘ < c2 from the calculation, anon-exponential single-step time is obtained which approximatesthe time of the deterministic model.

By applying the expression (20) for multi-step times, we findthat T k!‘ ¼ Oð1Þ, if both ~k and ~‘ are in the same region withnon-exponential single-step times. All other multi-step times con-tain exponential single-step times and therefore grow exponen-tially fast with N. Fig. 8(a) shows the expected times fortransitions between the local maxima G1 and G3 of the stationarydistribution for s = 0.55. A path from one local maximum to theother includes a large number of exponential single-step times inthe direction of decreasing stationary probability. Logically, the ex-pected time for such a path grows exponentially in N. However, asdiscussed before, even expected times of single steps in the direc-tion of increasing stationary probability may grow exponentially inN. As a result, also multi-step times in such a direction may begrow exponentially with N. In Fig. 8(b) such an exponential depen-dence is shown for transitions from (G1 + G2)/2 to G1 and from(G2 + G3)/2 to state G3.

5. Eigenvalue analysis

5.1. Eigenvalues of M

In the previous section, we have seen that, although every statecan be reached from every other state, the expected time for such atransition can be extremely long and may even grow exponentiallywith N. This means that the probability vector z(t) may convergevery slowly to the stationary distribution. In this section, we showhow the eigenvalues of M provide more insight into this behaviorand its dependence on N.

The Perron–Frobenius theorem [12] (applied to the matrixI + cM for a suitable c) yields that there is one simple eigenvaluek0 = 0 with a stochastic eigenvector v0. Furthermore, Gershgorin’scircle theorem [13,14] states that each of the eigenvalues of M isin at least one of the discs with center �ai � bi and radius ai + bi

(with i = 0,1, . . .,N). Consequently, all eigenvalues have a non-posi-tive real part.

It is easily verified2 that M is similar to the symmetric tridiagonalmatrix V = (vij) (with i, j = 0,1, . . .,N), with elements

v ij ¼

ffiffiffiffiffiffiffiffiajbi

pif j ¼ i� 1;ffiffiffiffiffiffiffiffi

aibj

pif j ¼ iþ 1;

�ðai þ biÞ if j ¼ i;

0 otherwise:

8>>><>>>:

ð21Þ

0 1s

x

0

0.5

1

0

V

IV

III

II

I

Fig. 6. Steady states of the deterministic model (see also Fig. 2) plotted togetherwith the curves that separate the ‘exponential’ and ‘non-exponential’ regions in thestochastic model (dotted lines).

Table 1Behavior of the stochastic and the deterministic model in the regions I–V. ‘exp’denotes exponential single-step times; ‘non-exp’ denotes non-exponential single-steptimes.

Region I II III IV V

T k!kþ1 non-exp exp exp non-exp expT k!k�1 exp exp exp exp non-expDeterministic model dx

dt > 0 dxdt < 0 dx

dt > 0 dxdt > 0 dx

dt < 0

2 When the determinant of a tridiagonal matrix A is expressed in terms of itselements Ai,j, the off-diagonal elements only appear in products of the formAk,k�1Ak�1,k. For the tridiagonal matrices M and V these products are identical. Mand V also have the same elements on their diagonals. As a result, the determinantsdet (M � qI) and det (V � qI) for any q are identical. This means that M and V havethe same eigenvalues.



Hence the eigenvalues of V and M are real. In summary this meansthat there is one eigenvalue k0 = 0; all other eigenvalues k1, . . .,kN

are negative and real. We will order the eigenvalues such thatjk0j < jk1j 6 jk2j 6 � � � 6 jkNj. Moreover, V has an orthonormal set ofN + 1 eigenvectors. The corresponding eigenvectors of M will bewritten as v0, . . .,vN. The (stochastic) eigenvector with eigenvaluek0 = 0 is the stationary distribution: v0 = Z.

We now study the behavior of the solution of the Master equa-tion (4), with a stochastic vector z(0) as initial value at t = 0. Theinitial vector can be written as

zð0Þ ¼XN

i¼0

civi:

Since z(0) and v0 = Z are stochastic vectors and all other eigenvec-tors of M have a zero element sum, the coefficient c0 = 1. This leadsto the solution

zðtÞ ¼ ZþXN

i¼1

civi expðkitÞ: ð22Þ

Clearly, the eigenvalues k1, . . .,kN determine how fast z(t) convergesto Z. In particular k1 is important for the time required for this con-vergence and how this time depends on N. Since k1(<0) is the sec-ond largest eigenvalue of the symmetric matrix V, its value can befound from the Rayleigh quotient

k1 ¼maxy?w

yT � V � yyT � y

� �; ð23Þ

where w is the eigenvector of V with eigenvalue k0 = 0. It is easilyverified that w is related to the corresponding eigenvector Z of Mby wi ¼

ffiffiffiffiZip

. By using an appropriate vector y perpendicular to w(23) gives a lower bound for k1:

yT � V � yyT � y 6 k1 < 0: ð24Þ

5.2. Eigenvalue analysis of a bistable system

The construction of an appropriate vector y to use in (24) de-pends on the specific system that is considered. We will considerthe case of a bistable system, i.e., a system with a kinetic potentialF with two local minima. Hence for sufficiently large N, the station-ary distribution Z has two local maxima. This holds for the consid-ered autophosphorylation cycle if the parameter s satisfiesr1 < s < r2. For this network the first two eigenvectors of M areshown in Fig. 9. From that figure we conclude that the correspond-ing eigenvector v1 for M looks like an adapted form of v0 = Z inwhich the first half is multiplied by a positive factor and the secondhalf is multiplied by a negative factor. In this way, an approxima-tion for v1 can be constructed. However, in view of the Rayleighquotient (23) we are more interested in an approximation y ofthe eigenvector of V with eigenvalue k1. Therefore, we use a similarapproach to construct an approximate eigenvector y. In otherwords, we use two parts of w (the eigenvector of V with eigenvaluek0 = 0), multiplied by suitable coefficients, to construct the approx-imate eigenvector y (with eigenvalue k1).

0 0.5 0.005 0.01010–3

100

103

10–6

10–3

100

103

0.500.50

100

101

102

103

10-10

10-5

100

N = 100

N = 200

N = 300

(a) (b)

(c) (d)

/N/N

/N/N

Fig. 7. Separation of T k!kþ1 into the contributions of paths by diameter for k = 0.5N and s = 0.55.

0 500 1000 0 500 1000

N N

1020

1010

100

1020

1010

100

(a) (b)

Fig. 8. (a) Expected transition times from G1 to G3 (solid line) and from G3 to G1 (dashed line) as a function of N for s = 0.55. (b) Expected transition times from (G1 + G2)/2 to G1

(solid line) and from (G3 + G2)/2 to G3 (dashed line) as a function of N for s = 0.55.



Recall that V has an eigenvector w corresponding to eigenvaluek0 = 0. This vector w has elements wi ¼

ffiffiffiffiZip

. We will use this vectorto construct parts of the vector y, which is used as an approxima-tion for the eigenvector corresponding to k1.

Let r be an index with 0 6 r < N. Define

yL ¼ ðw0; . . . ;wr ;0; . . . ;0ÞT ¼ffiffiffiffiffiZ0

p; . . . ;

ffiffiffiffiffiZr

p;0; . . . ;0

� �T

yR ¼ ð0; . . . ;0;wrþ1; . . . ;wNÞT ¼ 0; . . . ; 0;ffiffiffiffiffiffiffiffiffiZrþ1

p; . . . ;

ffiffiffiffiffiffiZN

p� �T:

We now construct a vector y by

y ¼ ð1�xÞyL �xyR

¼ ð1�xÞffiffiffiffiffiZ0

p; . . . ; ð1�xÞ

ffiffiffiffiffiZr

p;�x

ffiffiffiffiffiffiffiffiffiZrþ1

p; . . . ;�x

ffiffiffiffiffiffiZN

p� �T;

ð25Þ

with a constant 0 6x 6 1. The approximate eigenvector y has to beperpendicular to w, hence the condition

wT � y ¼ ð1�xÞXr

i¼0

w2i �x

XN

i¼rþ1

w2i ¼ 0

must hold. UsingPN

i¼0w2i ¼ 1, we obtain

x ¼Xr

i¼0

w2i ¼

Xr

i¼0

Zi: ð26Þ

We now compute numerator and denominator of the Rayleigh quo-tient in (24) for the vector y constructed in this way. We first con-sider the numerator. The multiplication V � y yields a vector withelements qi. Since each row in V only has non-zero entries on thediagonal and on the sub-and supra-diagonal positions, we find:

qi ¼ v i;i�1yi�1 þ v i;iyi þ v i;iþ1yiþ1:

Combining (25) and (21) with the equalities Zi�1ai�1 = Zibi and Zi+1-

bi+1 = Ziai (obtained from the stochastic detailed balance propertydiscussed in [10]), we find the following:

qi ¼

0 if 0 6 i < r;

�ffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiaibiþ1Ziþ1

p¼ �ai

ffiffiffiffiZip¼ �ar

ffiffiffiffiffiZrp

if i ¼ r;ffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiai�1biZi�1

p¼ bi

ffiffiffiffiZip¼ brþ1

ffiffiffiffiffiffiffiffiffiZrþ1

pif i ¼ r þ 1;

0 if r þ 1 < i 6 N:

8>>>>>><>>>>>>:

For the numerator of the Rayleigh quotient we thus obtain

yT � V � y ¼ yrqr þ yrþ1qrþ1 ¼ �arZr:

The denominator of the Rayleigh quotient in (24) is given by

yT � y ¼XN

i¼0

y2i ¼ ð1�xÞ2

Xr

i¼0

Zi þx2XN

i¼rþ1

Zi:

Using (26), this can be written as

yT � y ¼ ð1�xÞ2xþx2ð1�xÞ ¼ xð1�xÞ:

Combining the numerator and denominator we obtain the follow-ing lower bound for k1:

�arZr

xð1�xÞ 6 k1 < 0: ð27Þ

Note that this bound holds for any value of r. Moreover, besides Zr,also x depends on r, see (26).

We now use the fact that we consider a system with a bimodalstationary distribution Z, with local maxima in G1 and G3 and anintermediate local minimum in G2. We select r = G2 such that r

N is(approximately) the position of the intermediate maximum c2 ofthe function F. For this choice we compute Zr and x. Now x is the to-tal probability in the left part of the distribution, i.e., up to r. That partis centered around G1 and approximately Gaussian. In Appendix A isshown that the sum over all probabilities Z‘ with ‘ 6 G2 can beapproximated by:

x ¼XG2

‘¼0

Z‘ � ZG1

ffiffiffiffiffiffiffiffiffiffiffiffiffi2pN

F 00ðc1Þ

s: ð28Þ

The right part of the distribution is centered around the point G3.Using the same method as the one leading to (28), we obtain

1�x � ZG3


F 00ðc3Þ

s: ð29Þ

Substitution of (28) and (29) in (27) gives

�aG2 ZG2

ffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiF 00ðc1ÞF

00ðc3Þp

2pNZG1 ZG3

6 k1 < 0: ð30Þ

This bound for k1 still contains the stationary probabilities in thepoints G1, G2 and G3. We assume for the moment that the stationaryprobabilities Zk have their global maximum in k = G3 (as it is thecase for the autophosphorylation cycle with s in (r0,r2)). We nowrewrite the bound to find its asymptotic dependence on N. Firstly,we know aG2=N � aðc2Þ. Secondly, since Zk has its global maximumat G3, we know from Appendix A that

ZG3 �

ffiffiffiffiffiffiffiffiffiffiffiffiffiF 00ðc3Þ2pN

s:

Thirdly, using (16) we find that the quotient of ZG2 and ZG1 satisfies:

ZG2

ZG1

¼ O eNðFðc1Þ�Fðc2ÞÞ� �

:

Combining the above, we rewrite the lower bound in (30) as:

�aG2 ZG2

ffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiF 00ðc1ÞF

00ðc3Þp

2pNZG1 ZG3

¼ �O ac2

ffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiF 00ðc1ÞN

2p

seNðFðc1Þ�Fðc2ÞÞ

0@

1A:

Hence, we obtain a lower bound for k1 with asymptotic behavior forN ?1 given by

�OffiffiffiffiNp

eNðFðc1Þ�Fðc2ÞÞ� �

6 k1 < 0: ð31Þ

Of course a similar bound can be derived if not G3 but G1 is the glo-bal maximum of the stationary distribution. This results in (31)with c1 replaced by c3. In both cases, k1 tends to zero exponentiallyfast if N ?1. As a result, the convergence of the solution of (22) to

0 0.5 10 0.5 1–1

–0.5

0

0.5

1

–1

–0.5

0

0.5

1

k/Nk/N

0 1

Fig. 9. Eigenvectors of the transition matrix M for the autophosphorylation cyclewith s = 0.55 and N = 100. The figures show the normalized elements of eigenvec-tors v0 and v1 for the first two eigenvalues k0 and k1.



the stable stationary Z (which depends on a time constant 1/k1) cantake extremely long for a large number of molecules N.

Note that, up to (27), the argument above holds for any functionF and any point r. However, the selection of r to correspond withthe minimum c2 of F, that lies between two maxima in c1 and c3,is of course only possible if F has this behavior.

5.3. Eigenvalue analysis for the autophosphorylation cycle

For the autophosphorylation cycle, the dependence of eigen-values k1 and k2 on N is shown in Fig. 10. This figure shows that k1 in-deed tends exponentially to zero if N ?1, while k2 converges to anegative constant value. This has a number of implications for thedynamics of the system. As k2, . . .,kN are negative and do not vanishwith increasing N, the corresponding terms c2v2exp(k2t), . . .,cNvNexp(kNt) in (22) decrease exponentially in t, for each N. However,k1 tends to zero exponentially fast if N ?1. As a result, the corre-sponding term in (22) decreases with a time constant 1/k1 thatgrows exponentially in N.

Roughly, the dynamics for a constant input s can be regarded asthree separate phases: (a) the initial phase, in which the terms cor-responding to k2, . . .,kN decrease exponentially over time; (b) the‘quasi-stationary’ phase, in which the system can be approximatedby the terms corresponding to k0 and k1; and (c) the stationaryphase, which occurs if the term c1v1exp(k1t) has also vanished. Incontrast with phase (c), both phases (a) and (b) depend on the ini-tial conditions. The length of phase (b) depends on both the initialdistribution z(0) and eigenvalue k1 and grows exponentially with N.

Fig. 9 shows the elements of eigenvectors v0 and v1 for the firsttwo eigenvalues of M. Recall that v0 corresponds with the station-ary distribution Z. All quasi-stationary distributions depend onboth v0 and v1 and are of the form v0 + uv1. For a given initial con-dition z(0), the system will converge fast to v0 + uv1, with udepending on z(0). On a much longer timescale, u will tend to zero,eventually resulting in the stationary solution.

6. Conclusions and discussion

This paper shows that a potential function can be a useful toolfor the comparison of stochastic and deterministic models ofchemical reaction systems. The kinetic potential used in this paperprovides both intuitive and precise information about the way thebehavior of the stochastic model depends on the number of mole-cules N. This can be used to study the convergence (and in somerespects the lack of convergence) of the behavior of this model tothat of the corresponding deterministic model with increasing N.As discussed in Section 6.3, potential functions have been used be-fore to analyze the stationary distribution and expected times ofstochastic models. Nevertheless, our work provides several impor-tant novelties.

Previously, potential functions have been used because theyprovide information about the stationary distribution and expectedtimes. We show that our kinetic potential provides additional infor-mation in ways we have not encountered in the literature before. Inbistable systems, the kinetic potential can be used to give an expli-cit proof that the second eigenvalue of the transition matrix M de-pends on N in an exponential fashion. The kinetic potential alsoallows to distinguish between regions with characteristic behavior.For the autophosphorylation cycle, five regions can be identifiedwith characteristic behavior of both the stochastic and determinis-tic model (as summarized in Fig. 5 and Table 1).

Our approach provides a generic and self-contained frameworkfor a biologically interesting class of systems. In this approach, weparticularly focus on the dependence of the stochastic model on Nand its relation with the deterministic model. The analysis ispurely based on the deterministic rate equations, without furtherreference to thermodynamic notions. The approach discussed inthis paper is only suitable for single-variable systems, but can begeneralized to a larger class of systems.

6.1. Autophosphorylation cycle

Throughout this paper, we have exemplified our approach usingan idealized autophosphorylation cycle. The relative simplicity ofthis autophosphorylation cycle and the chosen Michaelis–Mentenkinetics allows us to analytically study both the deterministicand the stochastic dynamics. Analysis of this relatively simple sys-tem provides insight into its behavior to an extent that is generallyimpossible for more complex systems. We do not claim thatMichaelis–Menten kinetics provide a realistic description for areal-life network with this topology. In fact, although a phosphor-ylation cycle without autophosphorylation can realistically bemodeled with Michaelis–Menten kinetics [15], it is known that thisgenerally is not the case for more complex networks derived fromsuch a cycle [16]. Nevertheless, a detailed analysis of the validity ofMichaelis–Menten kinetics in this particular case is outside thescope of this paper.

In some respects, the behavior of the deterministic model re-flects that of the stochastic model for very large N. For instance,the local maxima of the stationary distribution tend to the stablesteady states of the deterministic model for N ?1. On the otherhand, the deterministic model only represents the expected behav-ior of the stochastic model during the initial and quasi-stationaryphase. Indeed, the length of the quasi-stationary phase growsexponentially in N. Hence, in the limit N ?1 the subsequent sta-tionary phase is never reached.

For transition times between different proportions of phosphor-ylated proteins the comparison is more complicated. If a transitionis impossible in the deterministic model, this is reflected in the sto-chastic model by an expected time that grows exponentially in Nfor N ?1. If a transition is possible in the deterministic model,the corresponding expected time in the stochastic model can eithertend to the transition time in the deterministic model or – remark-ably – grow exponential for N ?1, depending on the shape of F.

The exponential dependency of expected single-step times on Nis explained by the separation of those times into probabilities andtimes by path diameter (see Fig. 7). In the deterministic model, theinfluence of extremely long paths is ignored, since these paths oc-cur rarely for large N. However, in this way the deterministic mod-el also ignores the enormous influence of such paths in cases inwhich they do occur. In real-life biochemical reaction networks,N is often relatively small and these paths may still have a signifi-cant role. When modeling such systems with a deterministic mod-el, one should be aware of the existence of those paths and the factthat they are neglected in the model.

200 400 600 800

N

-100

-10-5

-10-10

Eig

en

va

lue

Fig. 10. Eigenvalues k1 (solid line) and k2 (dashed line) of M as a function of N fors = 0.55.



6.2. Bistability in real-life networks

Bistability is a common feature of biochemical reaction net-works. It can be found in the various types of networks that arepresent in a cell. For instance autoregulatory protein-DNA feedbackloops in gene regulatory networks cause bistability [17]. Also insignaling networks [18] and in networks that regulate cellular pro-cesses, bistable subnetworks (or ‘motifs’) can be identified. Theautophosphorylation cycle that is used as a running example inthis paper, is an idealized representation of one of the smallestphosphorylation modules that can theoretically yield bistability[19]. We observed that 143 out of 292 kinases in the Human Pro-tein Reference Database [20] show autophosphorylation.3

A well-know example of bistability in intracellular signalingand regulation is the protein Ca2+/calmodulin-dependent proteinkinase II (CaMKII) [21,22], which is involved in synaptic plasticity(i.e., the cellular process that is required for the storage of memoryin the brain). Although the exact mechanism is much more compli-cated, the CaMKII network shares some characteristics with ourautophosphorylation cycle. CaMKII proteins form rings on whicheach protein can be phosphorylated and dephosphorylated. Inaddition, phosphorylated CaMKII proteins can autophosphorylateother CaMKII proteins. This results in two switch states: one witha large and one with a small proportion of phosphorylated pro-teins. The highly phosphorylated switch state can be stable for along time and induce other processes which potentiate the synap-tic connection between two neurons.

Another example is the eukaryotic cell cycle [23]. This cycle isdivided in distinct phases that each contain processes involved inthe duplication of cell contents and division of the cell. Transitionsbetween those phases require robust and irreversible switchingevents, since unwanted phase transitions can be harmful. Alsothese switching events are due to bistable motifs in the reactionnetwork (see for instance [24]).

The CaMKII system and the regulation of the cell cycle regulationhave clearly different functions but similar requirements. The bista-ble modules in those networks need to switch fast between twostates at the correct moment, while they should avoid switchingat other moments. As those modules are relatively complex, theycannot straightforwardly be analyzed with our single-variable ap-proach. We can, however, gain some qualitative understanding ofsuch modules from our autophosphorylation cycle. Recall that theshape of the kinetic potential F provides information about boththe deterministic and the stochastic description of this network.Hence, manipulation of the shape of F affects the properties of thesystem for both large and small numbers of molecules.

In the case of the autophosphorylation cycle, the shape of F canbe changed by the input kinase concentration s. For instance, atemporary increase in s can change the kinetic potential from a bi-modal to a unimodal function and by that increase the probabili-ties for states with large proportions of phosphorylated proteins.A more complex reaction system may involve more sophisticatedmechanisms to reshape the kinetic potential and by that manipu-late the properties of the switch. For instance, deeper minima inthe kinetic potential may lead to a more stable switch for smallernumbers of molecules.

6.3. Related work

The kinetic potential F has been used before to describe extinc-tion times for birth-and-death processes [25–27]. However, it is

different from the potential functions generally used to describestochastic reaction systems [10,28]. The difference between thetwo types of potential functions is that the first type is definedon a macroscopic scale and the latter on a mesoscopic scale. Theadvantage of the first type is that it does not require reformulationwhen the size of the system (i.e., the number of molecules N) ischanged. This makes it more suitable for the applications we areinterested in.

For classic mass-action systems with known microscopicparameters the kinetic potential F is linearly related to the Gibbsfree energy. However, this is generally not the case if the meso-scopic model is derived from a macroscopic model. The advantageof using F instead of the Gibbs free energy is that the former doesnot require information about thermodynamic properties such astemperature. In many deterministic models of biochemical reac-tion networks such properties are not explicitly taken into account.Moreover, F can also straightforwardly be applied to non-elemen-tary reaction kinetics.

The quasi-stationary behavior of the autophosphorylation cycleis reminiscent of Keizer’s paradox described by [29]. In that sys-tem, there is one reaction species of interest, which is involved intwo reactions: an autocatalysis reaction and a first-order degrada-tion reaction. The deterministic model has two stable steadystates: total extinction (i.e., zero molecules) and a non-zero valuefor which degradation and autocatalysis are balanced. The stochas-tic model has one true steady state (total extinction) and a quasi-stationary distribution. In analogy with our autophosphorylationcycle, the quasi-stationary distribution in the system of [29] van-ishes after exponentially long times. Numerical results discussedin [29] indicate that the second eigenvalue of the transition matrixM tends to zero in an exponential fashion. An important novelty inthis paper is that we provide an actual proof that this behavior isindeed exponential.

In the field of statistical physics, other bistable reaction systemshave been studied with interesting results that partially overlapwith ours, e.g. [30,31]. There are, however, some clear differences.In statistical physics the dependence of the stationary distributionon the number of molecules N is usually studied by describing thedynamics of the stochastic system with a Fokker–Planck equation[32] and then studying the stationary solutions of this equation.Note, however, that the Fokker–Planck equation is only an approx-imation that is obtained by omitting higher order terms. In our ap-proach we start with the exact stationary distribution (5) andmanipulate that to obtain the exact expression (9) for the quotientof two probabilities. This leads to the asymptotic behavior as givenin (16).

A well-known theoretical reaction system that can show bista-bility is the so-called Schögl model, introduced by [30]. In a re-cent paper [33], the relations between the stochastic anddeterministic model of this reaction system are discussed. Thispaper also describes the differences between mathematical andchemical detailed balance. Many results in this paper can be re-lated to our results. The authors discuss these subjects from athermodynamic point of view, while our focus is purely kinetic.Therefore, both papers provide complementary insights. Note thatthis paper also observes the exponential growth of the secondeigenvalue of the transition matrix, which is confirmed by our ex-plicit proof.

Another recent paper, [27], shows that also for the Schögl model,there are exponentially long transition times, which can easily befound from the shape of the potential function. This is in goodagreement with our findings. In the discussion section of [27], theauthors mention some open questions regarding the most probableand most dominant paths between two states. The separation ofpaths according to path diameter (discussed in Section 4.2)provides the information required to answer those questions.

3 In the Human Protein Reference Database (Release 8, downloaded from http://www.hprd.org), there are 292 proteins that catalyze a phosphorylation reaction. For143 of those proteins, there is at least one reaction in which the same protein is boththe enzyme and the substrate of a phosphorylation reaction.



Moreover, Fig. 7 provides a clear example for the fact that the mostprobable path is not always the most dominant path.

For many chemical reaction systems, it is practically impossibleto study the stochastic dynamics analytically. In most of thosecases, the simulation of individual trajectories with the exact sto-chastic simulation algorithm introduced by Gillespie [34] providesa useful tool to obtain insight into the behavior of the system.However, one should keep in mind that a solid stochastic analysisin some cases may require an extremely large number of simula-tions. For example, in the autophosphorylation cycle the expectedtime for a transition from one given state to another may dependheavily on rare paths. A reasonable estimation of this expectedtime requires enough simulations to sufficiently sample all thoserare paths. For increasingly large values of N, the probability ofthose paths (but not necessarily their influence) decreases expo-nentially, leading to an exponential increase in the number of re-quired simulations.

Acknowledgments

This work is supported by the European Commission throughthe Evolving Cell Signaling Networks in Silico (ESIGNET) projectof the Sixth Framework Programme.

Appendix A. Stationary distributions near local maxima

The relation (16) allows an easy computation of the form of thestationary distribution near a local maximum. Let k be a local max-imum of the distribution Z and let ‘ a nearby state for which wewant to express the probability. If k and N are large, the maximumof the distribution in k corresponds with a minimum of the func-tion F in ~k � k ¼ k

N. For points ‘ not too far away from k, we canapproximate the difference F k

� �� F ‘� �

by a second order Taylorseries:

F k� �� F ‘� ��

F 00 k� �ð‘� kÞ2

2¼ �

F 00 k� �ð‘� kÞ2

2N2 :

The first-order term in this Taylor series vanishes since F 0 k� �¼ 0.

Substitution in (16) gives

Z‘ � Zk exp�F 00 k

� �ð‘� kÞ2

2N

0@

1A

ffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffia k� �

b k� �

a ‘� �

b ‘� �

vuuut ;

with ‘ ¼ ‘=N. The exponential term gives rise to a normal distribu-tion around k, with a variance r2 ¼ N=F 00 k

� �. The square root term

equals 1 for ‘ = k. Moreover, the influence of this term diminisheswith increasing N, hence we can omit this term for large values ofN. This leads to

Z‘ � Zk exp�F 00 k

� �ð‘� kÞ2

2N

0@

1A: ð32Þ

Now we assume the maximum at k is the only maximum in the sta-tionary distribution. Hence, all probabilities Z‘ will be very small,except the Z‘ with ‘ near k. In that situation, the sum of all probabil-ities Z‘ can be computed by using the Gaussian approximation (32).Instead of summing over all ‘, it is sufficient to sum over all ‘ withj‘ � kj 6 d, where d is a few times the standard deviation r. Thisyields

XN

‘¼0

Z‘ �Xkþd

‘¼k�d

Z‘ � Zk

Z kþd

y¼k�dexp

�y2

2NF 00 k� ��

dy � Zk


F 00 k� �

vuut :

Since the sum of all probabilities in the distribution must be one, weconclude that in this situation the maximum of the probability dis-tribution is approximately given by

Zk �

ffiffiffiffiffiffiffiffiffiffiffiffiffiF 00 k� �

2pN

vuut: ð33Þ

If F has two or more local minima, then the probability distributionZ will also have two or more local maxima for large enough N. It iseasily verified that, in that situation, (33) is still an upper bound forthe local maximum of the probability distribution in k.

References

[1] T. Turner, S. Schnell, K. Burrage, Stochastic approaches for modelling in vivoreactions, Comput. Biol. Chem. 28 (3) (2004) 165.

[2] D.J. Wilkinson, Stochastic modelling for quantitative description ofheterogeneous biological systems, Nat. Rev. Genet. 10 (22) (2009) 122.

[3] T.G. Kurtz, The relationship between stochastic and deterministic models forchemical reactions, J. Chem. Phys. 57 (7) (1972) 2976.

[4] O. Wolkenhauer, M. Ullah, W. Kolch, K.H. Cho, Modeling and simulation ofintracellular dynamics: choosing an appropriate framework, IEEE Trans.Nanobiosci. 3 (3) (2004) 200.

[5] M.N. Artyomov, J. Das, M. Kardar, A.K. Chakraborty, Purely stochastic binarydecisions in cell signaling models without underlying deterministicbistabilities, Proc. Natl. Acad. Sci. USA 104 (48) (2007) 18958.

[6] M. Beckerman, Molecular and Cellular Signaling, Springer Science, New York,2005.

[7] L.J.S. Allen, An Introduction to Stochastic Processes with Applications toBiology, Pearson Education, Upper Saddle River, NJ, 2003.

[8] Y. Cao, D.T. Gillespie, L.R. Petzold, Accelerated stochastic simulation of the stiffenzyme-substrate reaction, J. Chem. Phys. 123 (2005) 144917.

[9] D.A. McQuarrie, Stochastic approach to chemical kinetics, J. Appl. Probab. 4 (3)(1967) 413.

[10] N.G. Van Kampen, Stochastic Processes in Physics and Chemistry, North-Holland, Amsterdam, 1992.

[11] J. Stoer, Einführung in die Numerische Mathematik I, second ed., Springer,Berlin, 1976.

[12] A. Berman, R.J. Plemmons, Nonnegative Matrices in the Mathematical Sciences,Academic Press, New York, 1979.

[13] S.A. Gershgorin, Über die Abgrenzung der Eigenwerte einer Matrix, Izv. Ahad.Nauk SSSR Ser. Fiz.-Mat. 6 (1931) 749.

[14] P.D. Lax, Linear Algebra and Its Applications, second ed., Wiley, New York,2007.

[15] A. Goldbeter, D.E. Koshland, An amplified sensitivity arising from covalentmodification in biological systems, Proc. Natl. Acad. Sci. USA 78 (11) (1981)6840.

[16] A.C. Ventura, J.A. Sepulchre, S.D. Merajver, A hidden feedback in signalingcascades is revealed, PLoS Comput. Biol. 4 (3) (2008) e1000041.

[17] F.J. Isaacs, J. Hasty, C. Cantor, J.J. Collins, Prediction and measurement of anautoregulatory genetic module, Proc. Natl. Acad. Sci. USA 100 (13) (2003)7714.

[18] K. Lai, M.J. Robertson, D.V. Schaffer, The sonic hedgehog signaling system as abistable genetic switch, Biophys. J. 86 (5) (2004) 2748.

[19] J.E. Lisman, A mechanism for memory storage insensitive to molecularturnover: a bistable autophosphorylating kinase, Proc. Natl. Acad. Sci. USA82 (1985) 3055.

[20] T.S. Keshava Prasad, R. Goel, K. Kandasamy, et al., Human Protein ReferenceDatabase-2009 update, Nucleic Acids Res. 37 (2009) D767.

[21] M. Lynch, Long-term potentiation and memory, Physiol. Rev. 84 (1) (2004) 87.[22] V. Lucic, G. Greif, M. Kennedy, Detailed state model of CaMKII activation and

autophosphorylation, Eur. Biophys. J.: EBJ 38 (1) (2008) 83.[23] A. Csikász-Nagy, D. Battogtokh, K. Chen, B. Novák, J. Tyson, Analysis of a

generic model of eukaryotic cell-cycle regulation, Biophys. J. 90 (12) (2006)4361.

[24] B.M. Slepchenko, M. Terasaki, Bio-switches: what makes them robust?, CurrOpin. Genet. Dev. 14 (4) (2004) 428.

[25] P. Hanggi, H. Grabert, P. Talkner, H. Thomas, Bistable systems: master equationversus Fokker–Planck modeling, Phys. Rev. A 29 (1) (1984) 371.

[26] C.R. Doering, K.V. Sargsyan, L.M. Sander, Extinction times for birth–death processes: exact results, continuum asymptotics, and the failureof the Fokker–Planck approximation, Multiscale Model. Simul. 3 (2) (2005)283.

[27] C.R. Doering, K.V. Sargsyan, L.M. Sander, E. Vanden-Eijnden, Asymptotics ofrare events in birth–death processes bypassing the exact solutions, J. Phys.:Condens. Matter 19 (2007) 065145.

[28] D. Gillespie, Markov Processes: An Introduction for Physical Scientists,Academic Press, New York, 1992.

[29] M. Vellela, H. Qian, A quasi-stationary analysis of a stochastic chemicalreaction: Keizer’s paradox, Bull. Math. Biol. 69 (5) (2007) 1727.



[30] F. Schlögl, Chemical reaction models for non-equilibrium phase transitions, Z.Phys. 253 (1972) 147.

[31] H.K. Janssen, Stochastisches Reaktionsmodell für einen Nichtgleichsgewichts-Phasenübergang, Z. Phys. 270 (1974) 67.

[32] H. Risken, The Fokker–Planck Equation: Methods of Solution and Applications,Springer, Berlin, 1989.

[33] M. Vellela, H. Qian, Stochastic dynamics and non-equilibrium thermodynamicsof a bistable chemical system: the Schlögl model revisited, J. R. Soc. Interface 6(39) (2009) 925.

[34] D.T. Gillespie, A general method for numerically simulating the stochastic timeevolution of coupled chemical reactions, J. Comput. Phys. 22 (4) (1976) 403.


Single-variable reaction systems: Deterministic and stochastic models

Documents

Transcript of Single-variable reaction systems: Deterministic and stochastic models