Approximate analysis of probabilistic processes: logic, simulation and games

Josee Desharnais∗, Francois Laviolette∗

Dep. d’informatique et de genie logicielUniversite LavalQuebec Canada

{Josee.Desharnais,Francois.Laviolette}@ift.ulaval.ca

Mathieu TracolLRI

Universite Paris-SudFrance

tracol@lri.fr

Abstract

We tackle the problem of non robustness of simulationand bisimulation when dealing with probabilistic processes.It is important to ignore tiny deviations in probabilities be-cause these often come from experience or estimations. Afew approaches have been proposed to treat this issue, forexample metrics to quantify the non bisimilarity (or close-ness) of processes. Relaxing the definition of simulationand bisimulation is another avenue which we follow. Wedefine a new semantics to a known simple logic for prob-abilistic processes and show that it characterises a notionof ε-simulation. We also define two-players games that cor-respond to these notions: the existence of a winning strat-egy for one of the players determines ε-(bi)simulation. Ofcourse, for all the notions defined, letting ε = 0 gives backthe usual notions of logical equivalence, simulation andbisimulation. However, in contrast to what happens whenε = 0, two-way ε-simulation for ε > 0 is not equal to ε-bisimulation. Next we give a polynomial time algorithmto compute a naturally derived metric: distance betweenstates s and t is defined as the smallest ε such that s andt are ε equivalent. Finally we show that most of these no-tions can be extended to deal with probabilistic systems thatallow non-determinism as well.

1 Introduction

In program verification, the goal is typically to checkautomatically whether a system conforms to its pre-established specification. For non-probabilistic systems,one usually expects equivalence between the two, and mostof the time this equivalence is chosen to be bisimulation. Inthe case of probabilistic systems it has been observed [14]that the comparison between processes should not be basedon equivalences. One reason is that the binary information

∗Research supported by CRSNG

given by bisimulation is not precise enough since it iden-tifies close processes as non bisimilar as well as would bevery different processes. Another reason is that most of thetime the stochastic information comes from observations, orfrom theoretical estimations. In this context, it is more use-ful to have a notion of approximate equivalence or distance.Distances or metrics have been defined for LMPs [9, 22]and some work has been done to estimate bisimulation witha certain degree of confidence [13, 11]. Relaxing the def-inition of simulation and bisimulation is another avenue,which we follow, to tackle the problem of non-robustnessof bisimulation.

We propose ε-bisimulation and ε-simulation to relax theusual notions. We give logical and game characterisationsfor those two notions.

The notion of ε-bisimulation will define a certain ballaround an LMP. Simulation is a non-symmetric version ofbisimulation. Roughly speaking, a process simulates an-other one if it can be observed that it can do at least what theother can do. We define a new semantics to a known simplelogic for probabilistic processes and prove the characterisa-tion, which is based on the following relation: we say thata state logically simulates another one if it ε-satisfies everyformula that the other one satisfies. Of course, when lettingε = 0, we retrieve the usual notions of logical equivalence,simulation and bisimulation [8].

Games semantics have applications in linear logic, pro-gramming languages and model-checking [1]. One impor-tant contribution of this paper is such a two players gamefor probabilistic systems. In the game which characterisesε-simulation, players will have to play the whole game ontheir assigned system, whereas in the game characterisingε-bisimulation, the Adversary will have the possibility toforce the Prover to trade places with him. Such a possibilityof changing sides is intuitively coherent with the followingresult: ε-bisimulation needs negation for its logical charac-terisation. One important consequence of this observationis that ε-bisimulation is not equal to two-way ε-simulationfor ε > 0, which is in contrast to what happens when ε = 0.

We also introduce a fixpoint logic, which can be seenas a fragment of the µ-calculus extended to the context ofprobabilistic systems. We generalize the notion of approx-imate satisfiability to this logic as well. It differs from thequantitative µ-calculus of [6, 5] in that it is not a real logic,and is adapted to the kind of relaxations we introduce.

The definition and characterisation of approximatebisimulation and simulation mentioned above are very nat-ural. Moreover, they are tractable. We present an algorithmfor computing the maximal corresponding relations on Sof ε-simulation and ε-bisimulation that runs in polynomialtime in the size of the considered system.

A natural (pseudo-)distance comes out of ε-bisimulation:two states s, t are at distance less than or equal to ε if theyare ε-bisimilar. This distance contrasts with most of theones defined in the past [9, 21, 13, 6], in that it does nothave any discount factor: it can differentiate states whichhave different behaviors even only in the long term. Such aproperty is interesting for the study of systems that are notsupposed to stop ever. The only other known distance thatdoes not discount the future [9] is intractable. Indeed, theproblem of evaluating the distance between two states hasrecently been proven decidable by Van Breugel et al. [20]but the algorithm has a double exponential complexity. Ouralgorithm is polynomial.

We generalise this distance to the non-deterministic case,getting a very natural and practicable distance relation. In[10] and [6] an attempt is done in this direction, but the com-plexity of the procedure is not detailed. The computabledistances on probabilistic systems defined in the past, es-sentially in [21, 13] are based on the computation of a fixedpoint which can be done in polynomial time only when adiscount factor is introduced, which weakens the effects oflong term behaviours.

The plan of the paper is as follows. Next section con-tains background, known definitions and results on LMPs.In Section 3, we define a loose semantics for the usual sim-ple logic introduced Larsen and Skou [15] and collect a fewresults about it. Our main contribution is in Section 4. Wedefine three notions of ε-simulation and show that they co-incide: a logical one, a structural one, and another one de-pending on the existence of a winning strategy in a two-players game. At the end of this section we introduce ourfixpoint logic. In Section 5 we give an algorithm to com-pute ε-simulation efficiently and the corresponding distanceis presented in Section 6. Finally, Section 7 is dedicated to adiscussion on the generalisation of this work to probabilisticautomata. Missing proofs can be found in appendix.

2 Preliminaries

A Labelled Markov Process is a probabilistic transitionsystem whose state space can be continuous. However, in

this paper, we restrict to countable state space systems. Wewill use in that case the natural mesure space (S,Σ), whereΣ = P(S).A sub-probability measure on S is a map µ :Σ → [0, 1] that is countably additive and gives 0 measureto the empty set. If X ∈ P(S) and R ⊆ S × S, we defineR(X) = {y ∈ S|∃x ∈ X|xRy}. A set A ∈ Σ is R-closedif R(A) ⊆ A.

2.1 Labelled Markov Processes

A Labelled Markov Process (LMP) is a model of astochastic system that reacts to actions taken by its envi-ronment. The system is in a state at any moment and, givenan action (label) performed by the environment, it can jumpto another state following a probability distribution [7].

Definition 1 (LMP). S = (S,Σ, h : L × S × Σ →[0; 1], i) is a Labelled Markov Process (LMP) if (S,Σ) isa measurable space, i is a distinguished state called theinitial state, and for all a ∈ L and s ∈ S, h(a, s, A)is a sub-probability measure as a function of A.

Since S is countable we consider Σ = 2S without spec-ification in the definition of the system. h(a, s,A), that wewrite ha(s,A), is a measure of the likelihood that the LMPwill jump to a state in A given that it was at s and had to re-act to action a. Note that a condition of Σ-measurability ofh(a, s, A) as a function of smust be imposed when workingwith uncountable processes.

LMPs differ from standard Markov chains in that theydepend on an auxiliary setL of actions and in that the transi-tions are only asked to be sub-probability measures insteadof probability measures. Sub-probabilities are to model thepossibility of blocking. This is of particular interest whenstudying simulation since a transition that sums up to lessthan one can be simulated by a transition with higher prob-ability. With transitions defined as total probability mea-sures (like in most studied models such as fully probabilisticprocesses and probabilistic automata [17, 3]) the facts thatprobabilities sum up to 1 and that every transition must bematched make simulation for probabilistic processes veryclose to simulation for non-probabilistic processes.

2.2 The simple logic L

Logics are powerful and well established tools to de-scribe properties that processes must satisfy. We recall thesimple logic L introduced by Larsen and Skou [15].

Definition 2 (Logic L). The syntax of L is the following:

L : θ ::= > | θ1 ∧ θ2 | θ1 ∨ θ2 | 〈a〉δ θ, with δ ∈ [0; 1].

The satisfaction relation |=⊆ S×L is defined by structuralinduction on the formulas of L as usual for >, ∧, ∨ and as

2

follows for the modal formula:s |= 〈a〉δ θ iff ha(s, [[θ]]) > δ, where [[θ]] = {s |= θ}.Disjunction is not necessary to obtain logical characteri-

sation of bisimulation but it is necessary for simulation [8].Apart from the presence of disjunction, the strict or non-strict inequality in the semantics of 〈a〉δ θ is also a varyingparameter in related papers. The original logic was intro-duced with a non-strict inequality. The choice depends onthe use of the logic. We may also add negation to this logic,giving the logic L¬, which is more expressive but have thesame discriminating power. Indeed, there is no formula ofLthat is equivalent to, for example, ¬>. However, two statessatisfy the same formulas of L¬ if and only if they satisfythe same formulas of L.

Simulation and bisimulation. Intuitively, a state simu-lates another one if it can be observed that it can do at leastwhat the other can do. Of course, this can be formalised inmany ways. In the following section we will give three dif-ferent views of ε-simulation: through a structural definitionof LMPs, through a logic and through a game. For the usualnotion of simulation, only the first two have been defined.

The following definition of simulation is a structural oneand hence is based on the existence of a particular relationon the states of an LMP.

Definition 3. [8] Let S = (S,Σ, h, i) be an LMP. A bi-nary relation R on S is a simulation if whenever sRt, wehave that for all a ∈ L and every R-closed set A ∈ Σ,ha(s,A) ≤ ha(t, A). We say that s is simulated by t if sRtfor some simulation relation R.

Simulation can be lifted straightforwardly from states toLMPs through initial states.

The following characterisation shows that the logic isrich enough to represent simulation and conversely that sim-ulation respects the logic. It includes a notion of logicalsimulation.

Theorem 1 ([8]). In an LMP S such that (S,Σ) is analytic,in particular when S is countable, if s, t ∈ S, then s issimulated by t iff for all formulas θ ∈ L we have s |= θ ⇒t |= θ.

Bisimulation is a “symmetric” simulation.

Definition 4 ([8]). Let S = (S, σ, h, i) an LMP. An equiva-lence relation R on S is a bisimulation if whenever sRt,we have that for all a ∈ L and every R-closed set A,ha(s,A) = ha(t, A). We say that s and t are bisimilarif sRt for some bisimulation relation R.

Bisimulation also has a logical characterisation.

Theorem 2 ([7]). Let S be an LMP, and s, t ∈ S. Then sand t are bisimilar iff for all formulas θ of the logic L (withor without ∨) we have s |= θ ⇔ t |= θ.

A direct consequence of the fact that the same logicL can characterise bisimulation and simulation is that forLMPs, bisimulation is equal to two-way simulation. Thatis, states s and t are bisimilar if and only if s simulates tand t simulates s. When one adds external non determinismto probabilistic processes (as in probabilistic automata [18])this is no longer the case. The situation then imitates whatis encountered in the non probabilistic setting.

For the remaining of the paper we fix a countable LMPS = (S,Σ, h : L × S × Σ → [0; 1], i); we will use freelythe notation s, t for states of S .

3 Approximate satisfiability: ε-logic

We now introduce a variation of the logic L as our firststep towards relaxing the notions above. The syntax is thesame, only the semantics changes to permit a relaxation thatcan also be used for strengthening the logic. The satisfac-tion relation is defined relatively to a real parameter ε, typ-ically in [−1; 1]. For ε > 0, ε-satisfying a formula willbe easier than satisfying the formula in the usual sense,whereas if ε < 0, ε-satisfying a formula will be more diffi-cult. We make the definition precise:

Definition 5. Let ε ∈ [−1, 1]. The syntax of the logicL¬,ε isthe same as the syntax of L¬. The approximate satisfactionrelation |=ε⊆ S×L¬,ε is defined by structural induction onthe formulas of L¬,ε as follows:

• s |=ε > for each state s• s |=ε ¬θ iff s 6|=−ε θ• s |=ε θ1 ∧ θ2 iff s |=ε θ1 and s |=ε θ2.• s |=ε 〈a〉δ θ iff ha(s, [[θ]]ε) > δ − ε,

where [[θ]]ε = {s ∈ S|s |=ε θ}.

If θ ∈ L¬,ε, we say that s ε-satisfies θ when s |=ε θ.

We denote by Lε the logic without negation.If ε > 0, the semantics requires that in order for s to ε-

satisfy 〈a〉δ θ, it only suffices that with the a-labelled transi-tion from s, we can jump in [[θ]]ε, a slightly bigger set thanthe usual one [[θ]], with a probability δ − ε slightly smallerthan the usual one δ. On the contrary, if ε < 0, we demandthat in order for s to ε-satisfy 〈a〉δ θ, it must be that withthe a-labelled transition from s, we can jump in a slightlysmaller set [[θ]]ε than the usual one [[θ]], with a probabilityδ − ε slightly bigger than the usual one. The case of nega-tion is natural: if ε > 0, then s |=ε ¬θ means s is closeto satisfy ¬θ. We say that this corresponds to the fact thats does not strongly satisfy θ. Accordingly, if ε < 0, thens |=ε ¬θ means s makes a little bit more than satisfying ¬θ.This correspond to the fact that s is not close to satisfying θ.

We will see that this new semantics diminishes the ex-pressive power of the logic. This could be considered a

3

killing argument, but one must recall that we want to usethe new semantics to build approximate notions of simula-tion and bisimulation. Here is how the formulas of L¬,ε andL¬ compare.

Definition 6. Let ε ∈ [−1; 1]. By structural induction onthe formulas of L¬, we construct for each θ ∈ L¬,ε anassociated formula θε ∈ L¬: >ε = >; (θ1∧θ2)ε = (θ1)ε∧(θ2)ε; (〈a〉δ θ)ε = 〈a〉δ−ε θε; (¬θ)ε = ¬(θ−ε).

Here we use the fact that 〈a〉λ θ is still a valid formula,even if λ < 0 or λ > 1, which gives in turn that (θε)−ε = θ.Clearly the transformation is additive: if θ ∈ L¬, ε, ε′ ∈[−1; 1], then (θε)ε′ is the formula θε+ε′ .

Example 1. If θ = 〈a〉.5 (¬〈a〉.2>), then θε =〈a〉.5−ε (¬〈a〉.2+ε>).

Proposition 1. Let θ ∈ L¬ and ε ∈ [−1; 1]. Then [[θ]]ε =[[θε]], where [[θ]] = {s ∈ S|s |= θ}.

The proof is an easy structural induction on the formulas.This proves thatL¬ is more expressive thanL¬,ε. Never-

theless, it will make the correspondence between simulationand logic smoother. Indeed, it calls naturally the followingapproximate simulation: a state s ∈ S will be ε-simulatedby a state t ∈ S if t can match the transitions that s canmake, up to ε.

Definition 7 (Logical ε-simulation.). Let ε ∈ [0; 1] ands, t ∈ S. We will say that t Lε-simulates s, written s ≺Lε t,if for all formula θ ∈ L we have :

s |= θ ⇒ t |=ε θ.

Clearly ≺Lε is not a transitive relation. However, usingthe fact that s |=ε θ ⇔ s |= θε, it can be shown easily that ifs, t, u ∈ S are such that s ≺Lε1 t and t ≺Lε2 u, then s ≺Lε1+ε2u. Of course we cannot use the logic with negation to dealwith simulation.

4 ε-simulation and ε-bisimulation

In what follows our aim is to give two characterisationsof approximate simulation. The most important notion isa generalisation of the structural definition of simulation.Following Definition 3, we define relations of ε-simulation.Next we give a definition in terms of a two players game,similarly as what is done in the context of concurrent sys-tems. We will show that these three notions coincide.

Definition 8 (Relations of ε-simulation). A relation R ⊆S × S is a relation of ε-simulation if whenever sRt, then∀a ∈ L,X ∈ Σ,

ha(t,R(X)) ≥ ha(s,X)− ε.

We say that s is ε-simulated by t, written s ≺ε t, if sRt forsome relation of ε-simulation R on S .

As for usual simulation [3], for a countable state spacemodel, ∼ε is the largest relation of ε-simulation and it canbe computed as a greatest fixed point. Let F : 2S×S →2S×S associate to any relation R ⊆ S × S the relationF (R) ⊆ S × S such that sF (R)t iff for all a ∈ L andall set E ⊆ S, we have ha(t,Rε,n(E)) ≥ ha(s,E) − ε.Then ≺ε will be the greatest fixpoint of F in 2S×S .

Example 2. Let ε > 0. Consider the following systems:

S : s1 sb, 12oo a,.9

zz T : t1 tb, 12−εoo a,.9+ε

zz

For any ε′ > ε we have s ≺ε′ t and t ≺ε′ s.These systems are typical of the analysis that we pro-

pose. Indeed, starting from the statement that a model ofa system cannot be exact, we are interested in comparingmodels, but with a certain degree of tolerance. More pre-cisely, s can be seen as a model of the real system t (andconversely): we would like to consider them as equivalentif we can tolerate an error of ε.

We now show that the two notions of approximate simu-lation that we have defined so far, logical Lε-simulation andstructural ε-simulation, are in fact equivalent.

Theorem 3. s ≺Lε t iff s ≺ε t.Games for ε-simulation. We introduce a new notion of

approximate ε-simulation in terms of game and show that itis also equivalent to ε-simulation ≺ε.Definition 9 (Game S, s, t). Two players, Prover (P),whose goal is to prove that s is simulated by t, and Adver-sary (A), are playing the following game on S. The gameconsists in the repetition of the following steps:

• Step 1: A chooses a label a and a set E ⊆ S.

• Step 2: P must choose a set F ⊆ S, such thatha(t, F ) ≥ ha(s,E)− ε.

• Step 3: A chooses t′ ∈ F from which P will have toplay in the next sequence of steps.

• Step 4: P chooses s′ ∈ E from which A will have toplay in the next sequence of steps.

If one player gets stuck at some point, for instance if Acannot choose any transition from s, or if P cannot finda relevant set F during Step 2, he looses. If the game con-tinues without ever stopping, Prover wins. We say that A(resp. P) has a winning strategy if there exists a sequenceof choices for the steps 1 and 3 (resp. 2 and 4), respondingto the choices of P (resp. A), such that anyhow being theresponses of P (resp. of A), such a sequence of moves leadsto a victory of A (resp. of P).

We write s ≺Gε t if P has a winning strategy.

4

This game seems quite simple but it is rich enough tocharacterise ε-simulation.

Theorem 4. s ≺Gε t iff s ≺ε t iff s ≺Lε t.Proof. Only the first equivalence is left to prove. We sketchthe proof. ( ⇒) is a direct consequence of the fact that ≺Gεis a relation of ε-simulation.

(⇐): given R a relation of ε-simulation on S such thatsRt, we construct a winning strategy for P. For this we pointout that at Step 2, P can choose the set F = R(E) suchthat ha(t, F ) ≥ ha(s, E) − ε. But then whatever being thechoice of t′ ∈ F at Step 3, since F ⊆ R(E), there willalways exist s′ ∈ E such that s′Rt′. With this choice weare back to the initial situation, and we see that P cannotloose, which means that he has a winning strategy.

Now that we have defined the logic Lε and ε-simulation,we are ready to face the question of approximate equiva-lence. Of course, it will not be an equivalence. We cannotask for our approximate equivalence relation to be transi-tive: two states s, t with quite different behaviours couldbe linked by a long sequence close states. It is quite nat-ural, however, to define ε-bisimulation as a symmetric ε-simulation.

Definition 10 (Relation of ε-bisimulation). A relation R ⊆S×S is a relation of ε-bisimulation if it is symmetric and arelation of ε-simulation. Two states s and t are ε-bisimilar,written s ∼ε t, if there exists a relation of ε-bisimulation Ron S such that sRt.

In Example 2 we have s ∼ε′ t for all ε′ > ε.Note that the symmetry of the relation could be replaced

by the requirement that the inverse of R be a relation ofε-simulation as well as R.

This definition looks similar to the one introduced byGiacalone et al. [14] but it is more faithful to probabilis-tic notions by allowing to regroup states when matching atransition. Indeed, Giacalone et al. define the ε-matching oftransition probabilities through states instead of sets. Thisis key because matching states impose that the transitionstructure of the two processes be equal. This definition co-incides with the usual definition of bisimulation when ε = 0and is natural when we think in terms of games.

As for ε-simulation, ∼ε can be computed iteratively forcountable state space model as a greatest fixpoint: ∼ε isnothing more than the greatest symmetric fixpoint of the re-lation transformer F on 2R×S .

We now continue the parallel construction: we present atwo-players game that will give another definition for an ap-proximate bisimulation. We will show that this new notioncoincides with ε-bisimulation.

Definition 11 (Game S, x, y for ε-bisimulation). Two play-ers, Prover (P) and Adversary (A), are playing the following

game on S. The game consists in a sequence of steps whichare repeated:

• Step 0: A chooses a state s ∈ {x, y}. A will play on swhereas P will play on t ∈ {x, y} \ {s}.

• Step 1: A chooses a label a and a set E ⊆ S.

• Step 2: P chooses a set F ⊆ S, such that ha(t, F ) ≥ha(s,E)− ε.

• Step 3: A chooses a state x′ in E or F .

• Step 4: P chooses a state y′ in the set not chosen by A.This way we get one state in E and one state in F .

The winning conditions are the same as for the game for ε-simulation. We write s ∼Gε t if P has a winning strategy inthe game S, s, t.

We show that the two previous notions of approximatebisimulation coincide.

Theorem 5. P has a winning strategy in the game S, s, t iffthere exists a relation of ε-bisimulation R on S × S suchthat sRt. That is s ∼Gε t iff s ∼ε t.

Proof. (⇒) As in Theorem 4, we show that ∼Gε is a re-lation of ε-bisimulation, which is a mere reformulation ofthe game rules. The fact that ∼Gε is a symmetric relationcomes directly from the definition of the game: we allow Ato choose his starting state at each round. If P has a winningstrategy, then we will have s ∼Gε t as well as t ∼Gε s.(⇒) This direction is also the same as for Theorem 4, thefact that R is symmetric corresponding here again to thechoice of A for his start point.

The next step would be to show that a logical notionof approximate equivalence, ∼Lε , and the game notion ∼Gεor the structural notion ∼ε coincide. However somethingsubtle happens, and we have to differentiate the case whenε = 0 from the other cases. Indeed, we already know byTheorem 2 that the logical notion of bisimulation ∼L0 andthe structural notion of bisimulation ∼0 coincide. Some-thing fascinating happens when ε > 0: the situation thenimitates what is encountered for non deterministic bisimu-lation. Two-way simulation and bisimulation disagree, andhence, negation is necessary in the logic to characterise ε-bisimulation for ε > 0.

Proposition 2. For ε ∈]0; 1], two-way ε-simulation isweaker than ε-bisimulation. That is, there exists a finiteLMP with two states s, t such that s ≺ε t and t ≺ε s, buts 6∼ε t.

Proof. We give a particular example with ε = 1/8. Clearlythis example could be modified to handle any ε ∈]0; 1].

5

sa,1/8

~~}}}}

}}}}

a,3/8

²²

a,4/8

&&NNNNNNNNNNNNN ta,2/8

xxqqqqqqqqqqqqq

a,4/8

²²

a,2/8

ÃÃ@@@

@@@@

@

s1 s2

b,1

²²

x

b,1

²²

t2

b,1

²²

t1

s4 s3c,1oo y t3

c,1 // t4

Then s ≺1/8 t and t ≺1/8 s, since x is 1/8-simulated byboth s1 and s2. On another hand, s 6∼1/8 t because we have{s ∈ S|x ∼1/8 s

}= {x}. But ha(s, {x}) = 4/8, which is

strictly greater than ha(t, {x}) + 1/8 = 3/8.

We now show that L¬,ε characterises ε-bisimulation.

Definition 12 (Logical ε-equivalence ∼L¬ε ). Let ε ≥ 0.Then s and t are said to be ε-equivalent for L¬, writtens ∼L¬ε t, iff s ≺L¬ε t. As usual, this means s |= θ ⇒ t |=ε θfor any θ ∈ L¬.

It is easy to see that this defines a symmetric relation.

Theorem 6. Let∼ε be the largest relation of ε-bisimulationon S. Then we have: s ∼ε t iff s ∼L¬ε t.

Proof. We sketch the proof. For ⇒, we suppose s ∼ε t andwe show by induction on the structure of the formulas ofL¬,ε that for all θ ∈ L¬,ε we have s |= θ ⇒ t |=ε θ. Theinteresting cases are θ = 〈a〉δ φ and θ = ¬φ.

Suppose θ = 〈a〉δ φ. Then ha(s, [[φ]]) > δ. By hy-pothesis s ∼ε t, hence if we take Y = ∼ε ([[φ]]) we getha(t, Y ) ≥ ha(s, [[φ]]) − ε. But for all x ∈ ∼ε([[φ]]), thereexists y ∈ [[φ]] such that x ∼ε y, hence, by induction hy-pothesis, for all x ∈ ∼ε ([[φ]]) we have x |=ε φ. This pre-cisely shows that t |=ε θ.

Suppose θ = ¬φ. We have to show that t |=ε ¬φ, i.e.that t |= (¬φ)ε, i.e. that t |= ¬φ−ε, i.e. that t 6|= φ−ε. Butt |= φ−ε would imply by induction that s |= φ−ε+ε, i.e. thats |= φ which is a contradiction.

For ⇐, we just have to show that ∼L¬ε is a relation of ε-bisimulation. We can follow the proof of Theorem 3, usingthe fact that ∼L¬ε is a symmetric relation.

The presence of negation in the logic reflects the fact thatwe allow Adversary to change sides during the game. Wegive a formula that distinguishes the two states s and t ofProposition 2’s example:

φ = 〈a〉.49 〈b〉.9 ¬(〈d〉.5>).

Then s |= φ, but t 6|=1/8 φ. This formula is in L¬\L.

A fixpoint logic for LMPs A fixpoint logic allows todeal with long term and global properties, which may beinteresting in the study of probabilistic processes. Manygeneralisations have been proposed ([9, 6, 5, 16]). Ours

differs from the quantitative µ-calculus of [6, 5] in that it isnot a real logic, and is adapted to the kind of relaxations weintroduce. We restrict to finite LMPs but the results can beeasily generalised to countable state space systems.

Definition 13 (The fixpoint logic Lµ).Syntax: The syntax of Lµ is defined with respect to a set Vof calculus variables. It is defined as follows:

Lµ : φ ::= θ | 〈a〉δ φ | ¬φ | φ1 ∧ φ2 | φ1 ∨ φ2 | 〈a〉1 Z |Z | µZ.φ | νZ.φ.

For θ any formula of L¬, a ∈ L, δ ∈ [0; 1], and calcu-lus variables Z ∈ V . As usual in µ-calculus, in formulasµZ.φ and νZ.φwe require that all occurrences of the boundvariable Z in φ occur in the scope of an even number ofoccurrences of ¬. A formula φ is closed if every calculusvariable Z in φ occurs in the scope of a quantifier µZ orνZ. For simplicity, we will do the following abuse of nota-tion, denoting by qµ the set of closed formulas of qµ. Weask that the formula φ in any sub-formula 〈a〉δ φ is a closedformula. The fact that we cannot have 〈a〉δ before a non-closed formula φ, except if δ = 1, will weaken this logic inan interesting way.Substitution: As for the µ-calculus, a variable valuationξ : V → 2S is a function that maps every variables Z ∈ Vto a set of states. We write ξ[Z 7→ X] for the valuation thatagrees with ξ on all variables, except that Z is mapped toX . Given a valuation ξ and a formula φ ∈ Lµ, the seman-tics of Lµ gives us a subset [[φ]]ξ of S:

[[θ]]ξ = [[θ]] for all θ ∈ L¬. [[Z]]ξ = ξ(Z)[[µZ.φ]]ξ =

⋂{S′ ⊆ S|[[φ]]ξ[Z 7→S′] ⊆ S′}[[νZ.φ]]ξ =

⋃{S′ ⊆ S|[[φ]]ξ[Z 7→S′] ⊇ S′}.

In this definition, [[µZ.φ]]ξ [[νZ.φ]]ξ are respectively the low-est and the greatest fixpoint of the following (monotonic)function:

g : P(S) → P(S), S′ 7→ [[φ]]ξ[Z 7→S′].

As usual the fixpoints are well defined by the continuity andmonotonicity of all operators and can be computed by Pi-card iteration.

Example 3. Let φ ∈ L. Consider the formula ψ1 =µZ.(〈a〉1 Z ∨ φ). Then [[ψ1]] is the set of s ∈ S from whichthe system can do action a with probability 1 until it reachesa state satisfying φ.

Now we would like to extend our relaxation of the sat-isfiability of formulas of L¬ to formulas of Lµ. The pointof our definition will be that we relax the transition prob-abilities only outside fixpoint operators: we do not changethe 1 in the expression µZ.(〈a〉1 Z), but we relax the othervalues. We give a short example to illustrate the definitionof the relaxed formula φε, given φ ∈ Lµ.

6

Example 4. Let φ = 〈c〉.5 (µZ.(〈a〉1 Z ∨ ¬〈b〉.2>)). Letε ∈ [−1; 1]. Then φε = 〈c〉.5−ε (µZ.(〈a〉1 Z∨¬〈b〉.2+ε>)).

Even if the logic Lµ is more expressive, it has the samediscriminating power as L. However, ∼µε has a discriminat-ing power between ∼ and ∼ε.Proposition 3. ∼⊂∼µε ⊂∼ε.

The following example will prove the strictness of thesecond inclusion. It exhibits systems that could be consid-ered as being related in the same way as do the ones ofExample 2. However, they are the key to a very importantdifference between ∼µε and ∼ε. One could say a weaknessof∼ε. However we will argue that this case is very peculiar.

Example 5. In the following processes, it could be interest-ing, from a probabilistic point of view, to see that s and thave different behaviors in the long term.

S : s1 sa,εoo a,1−ε

zz T : t a,1yy

Indeed s “leaks”: a run on the system S will eventuallyalways end in state s1 from which it cannot move. Howevera run on the system T will always continue to pass throughstate t. We know that in that case we have s ∼ε′ t, for anyε′ > ε. The fixpoint logic will differentiate the two: for anyε′ ≥ 0 we have s 6∼µε′ t. Consider the followingLµ formula:φ = νZ.(〈a〉1 Z). We construct [[φ]]S and [[φ]]T . We writeXi for the increasing sequence of Picard iteratives for S .We have: X0 = {s, s1}, X1 = [[〈a〉1 Z]]Z={s,s1} = {s},X2 = [[〈a〉1 Z]]Z={s} = ∅. Hence [[φ]]S = ∅, but [[φ]]T ={t}. As well, for any ε′ > 0, [[φ]]Sε′ = ∅, and [[φ]]Tε′ = {t}.This proves that s 6∼µε t.

This result is coherent with the fact that for the distancesdc of Desharnais et al., we have limc→1 d

c(s, t) = 1.

One could argue that this example shows that ∼µε is bet-ter than ∼ε. However, we believe that this undesirable be-havior of ∼ε will only happen in extreme cases like thisone. When we consider models involving cycles, more pre-cisely one of which is taken with probability exactly 1 andthe other with a closed to 1 probability, we are confrontedto a 0/1 result. Indeed, in the limit, the first cycle will betaken infinitely often with probability 1 whereas the otherwill fail with probability 1. We believe that claiming value0 or 1 is a strong commitment when studying probabilis-tic systems. For one thing, this cannot be obtained throughexperimentation.

As a conclusion we can say that the introduction of fix-point formulas in the logic L has made possible to differen-tiate more states, and the difference is made on variations ofprobabilities of long term runs.

Example 6. This is a tiny variation of Example 2. Let

S : s1 sb, 12oo a,1

zz T : t1 tb, 12−εoo a,1

zz

These systems of Example 2 are ∼µε -close for any ε′ > ε.

5 Decision procedures for ε-simulation and ε-bisimulation

So far we have introduced new notions of approximatesimulation and equivalence. In what follows we present ageneralisation of the algorithms introduced by Baier [2] todeal with our approximate notion of simulation. We makea link between the characterisation of the simulation rela-tions in terms of right-closed sets, and the characterisationin terms of maximum flows in suitable networks.

Given ε ≥ 0, one could be interested in computingefficiently the largest relation of ε-simulation, that is ≺ε.Recall that in the finite case, ≺ε can be computed as thegreatest fixpoint of the relation transformer F , that is ≺ε=⋂n∈NRε,n, where theRn,ε are defined inductively on n as:

• Rε,0 = S × S.

• sRε,n+1t iff for all a ∈ L and all set E ⊆ S,ha(t,Rε,n(E)) ≥ ha(s,E)− ε.

Similarly as is done by Baier [2], and Segala andLynch [18] for ≺, one can compute ≺ε efficiently as fol-lows. Since |Rε,0| = |S|2 and since ≺ε= Rε,n wheneverRε,n = Rε,n+1, we have that ≺ε= Rε,|S|2 and therefore itcan be computed in |S|2. The following important theoremwill allow us to determine efficiently if, roughly speaking, adistribution ν is greater than another distribution µ, up to ε.

Theorem 7. Let µ, ν be two distributions on a finite statespace S, and R ⊆ S × S be a binary relation on S. Thenthe following are equivalent:

• For all set E ⊆ S we have ν(R(E)) ≥ µ(E)− ε.

• The maximum flow in the network N (µ, ν,R) isgreater than or equal to µ(S)− ε.

Here the network N (µ, ν,R) is a flow network of sizeO(|S|), that we define in the appendix.

Proof. The proof uses methods of network theory.

We can now give an algorithm to compute≺ε iteratively.

Algorithm 1. Input: A finite state LMP S = (S,L, h),ε ∈ [0; 1].Output: The relation ≺ε on S × S.method:Let R0 = S × S.For j = 0 to |S|2 do begin:Rj+1 = Rj .For all (s, t) ∈ Rj do begin:

For all a ∈ L do begin:

7

If the maximum flow of the networkN (ha(s,−), ha(t,−),Rj) is strictly smallerthan ha(s, S)− ε, then Rj = Rj − {s, t}.

endend

endreturn R|S|2 .

Using an algorithm that computes the maximum flowof the networksN (ha(s,−), ha(t,−),Rj) in timeO(|S|3)(for instance [19]), we get an algorithm for computing ≺εon an LMP S in time O(|S|7 · |L|).

Now, if we want to compute ∼ε, it will probably not bepossible to use methods of partitions refinement [2, 4]. In-deed these methods, which construct the bisimulation rela-tion∼ on an LMP more efficiently than the simulation rela-tion (O(n3) opposed to O(n7)), rely fundamentally on thefact that ∼ is an equivalence relation, what ∼ε is not in thegeneral case if ε > 0. Still, since∼ε is the greatest symmet-ric fixpoint of F it can be computed as ∼ε=

⋂n∈NRε,n,

where the Rn,ε are defined inductively on n as follows:

• Rε,0 = S × S.• If s, t ∈ S, then we have sRε,n+1t iff for all a ∈ L

and all set E ⊆ S we have:

– ha(t,R(E)) ≥ ha(s,E)− ε.– ha(s,R(E)) ≥ ha(t, E)− ε.

As for ≺ε, ∼ε can clearly be computed as Rε,|S|2 if Sis finite. As for ≺ε, using maximum flow algorithms, weget an algorithm for computing ∼ε on an LMP S in timeO(n7).

6 A distance between states using ∼ε

A main issue concerning models of probabilistic systemsis the problem of defining a relevant distance between statesof a probabilistic model [9, 21, 13]. Most works are basedon a distance that can be defined as the maximal differenceof valuation of functions associated to formulas for the twoconsidered states. The idea is: the longest is a distinguish-ing formula, the closest should be the states. An importantcharacteristic of these metrics is that differences are accu-mulated while time evolves. This can be countered with theuse of a discount factor which lowers the effects of longterms events, making the distances lie mostly on short timeevolution of the system. Unfortunately, this approach has anundesirable side effect, that every difference become negli-gible after a certain horizon of time. Such a characteris-tic might be reasonable in the case of the study of “non-permanent” systems, that is, for instance systems which aresupposed to stop working after a while. But this is a weak-ness in the case of the study of permanent systems, for in-stance for the study of the long term behavior of a protocolbetween two persons, which is not supposed to stop ever.

We now propose a pseudo-distance between states of aprobabilistic system from the notions we presented. Thisdistance does not accumulate differences through evolutionof time and still does not suffer from a finite horizon visionas do the discounted distances.

We define d on S × S as follows:

d : S × S → [0; 1](s, t) 7→ inf {ε ∈ [0; 1] | s ∼ε t} .

The function d is a pseudo-distance function on the states ofthe considered LMP: the triangular inequality comes fromthe fact that if ε1, ε2 ∈ [0; 1], then s ∼ε1 u and u ∼ε2 timplies s ∼ε1+ε2 t.Theorem 8. The kernel of d, i.e. the set of pairs at zerodistance, is ∼0, the relation of bisimulation on S .

Since we have produced in the last section an algorithmto compute relation ∼ε on a finite LMP S in time O(|S|7),we can in a straightforward way, using a dichotomic ap-proach, compute the distance d(s, t) between two states ofS up to precision δ ∈ [0; 1] in time O(|S|7 · ln(δ)).

The distance d is a new distance, incomparable with thedistances presented in [9, 21, 13]. We give two examples toshow this fact. In the following, dc, c ∈ [0; 1[ is the distancewith discount factor c that is studied in the cited papers.

Example 7. In the following system, d(s, t) ≤ 0.1 sinces ∼.1 t, but for all c ∈ [0; 1] we have dc(s, t) ≥ (cn −cn0.95n). Hence if n is large enough and c is close enoughto 1, then dc(s, t) can be as close as possible to 1.

x1 ... xn

sa,0.95

//

a,0.05>>~~~~~~~~s1 ... sn−1

a,0.95//

a,0.05<<xxxxxxxxsn

ta,1

// t1 ... tn−1a,1

// tn

Example 8. In the next example for all c ∈ [0; 1[ wehave dc(s, t) ≤ cn, but d(s, t) ≥ .9 since s 6∼.9 t.s

a,1// s1 ...

a,1// sn−1

a,0.05// sn

ta,1

// t1 ...a,1

// tn−1a,1

// tn

Example 5 also underlines the fact that anyhow we definea distance on a probabilistic system, we will always have tochoose between giving more weight to close behaviours orto long term behaviours, since the two are not compatible.With the distance d we have decided to keep the possibilityof distinguishing between states which behave differentlyonly in the far future. Since we also want not to differen-tiate much between states which behave closely in the near

8

future, it is not possible for the same distance to differenti-ate strongly between states which behave almost the samefor a long time. As we have seen, using our fixpoint logiccan be a solution to enrich our distance.

The last examples underline important differences be-tween the two distances:

• The distances dc, c ∈ [0; 1[ differentiate states thathave different short range evolutions, whereas distanced can still separate states whose behaviors diverge onlyin the long range, as in Example 5.

• The distance d may be unable to differentiate stronglytwo states which have a slightly different behavior, butfor a long time.

As a conclusion we can say that these two kinds of distancesare not comparable. We consider that the properties of d aredesirable. The first one is important because we want tomeasure differences for a long horizon. For instance forsystems which are not supposed to stop evoluting ever. Thesecond one may be desirable because probabilities are esti-mates and therefore, small errors should be ignored. Also,we may want not to accumulate small errors along the run.However, as presented in example 5, in some case we wantto accumulate such error. A way to handle this problemfor our distance is to consider its generalisation to the fix-point logic we introduced. We confer to the appendix fora discussion on this topic. Another important property ofthis distance is that it can be easily generalised to other for-malisms, as is discussed in the next section.

7 Non deterministic processes

Probabilistic automata, such as presented by Segala [18],can be seen as a generalisation of LMP in that they allownon-deterministic probabilistic transitions: several proba-bilistic transitions with the same label can leave a state.

Definition 14 (Probabilistic Automata [18]). A probabilis-tic automata is a tuple (S, s0, L,D), where S is a set ofstates, L is a set of actions, s0 is a particular state calledthe initial state, andD ⊆ S×L×Disc(S) is the transitionrelation, where Disc(S) is the set of probabilistic distribu-tions on S.

We write s a→ µ instead of (s, a, µ) for a transition inD. We require that the system be finitely branching: fromevery state s ∈ S there exists a finite number of transitionss

a→ µ, with a ∈ L. Informally, the outgoing transitionss

a→ µ represent the non-deterministic alternatives in thestate s. A transition s a→ µ asserts that in state s the systemcan perform an action a that will lead with probability µ(t)to a state t ∈ S.

No practical distance has been constructed yet on thestate space of such models. The distances presented in[6] and [10] are highly non-constructive, and the ones de-fined in [9, 13, 21], apply only on purely probabilistic sys-tems. We will briefly show how our distance constructionfor LMPs can be generalised in that case as well. We firstpoint out the small changes that have to be done to the struc-tural definitions of ε-simulation and ε-bisimulation.

Definition 15 (ε-simulation on a probabilistic automata). IfA is a probabilistic automata and R ⊆ S × S is a relation,then R will be a relation of ε-simulation if:

∀s, t ∈ S sRt⇒ ∀s a→ µ, there exists a transition t a→ νsuch that ∀X ⊆ S we have ν(R(X)) ≥ µ(X)− ε.

ε-bisimulation is the symmetric notion associated to ε-simulation, as for LMPs.

We leave the logical characterisation of ε-simulation andε-bisimulation for future work. We present a game charac-terisation of these notions. The game steps for ε-simulationare now:

• Step 1: A chooses a transition s a→ µ and a set E ⊆ S.

• Step 2: P chooses a transition t a→ ν from t with labela and a set F ⊆ S such that ν(F ) ≥ µ(E)− ε.

• Step 3: A chooses t′ ∈ F from which P will have toplay in the next sequence of steps.

• Step 4: P chooses s′ ∈ E from which A will have toplay in the next sequence of steps.

This game is adapted for bisimulation in the same way as forLMPs. The proof of Theorem 5 can be modified to deal withnon-determinism. The only subtle point in this translationis the fact that we use the ”finite branching criterion” at theend of the proof.

As for LMPs, we have an algorithm which works in poly-nomial time in the size of the system to compute the rela-tion ∼ε on a probabilistic automaton A, given ε > 0. Oncemore, we can, using a dichotomic approach, compute thedistance d(s, t) between two states of a probabilistic au-tomatonA up to precision δ ∈ [0; 1] in time O(|S|7 · ln(δ)).

8 Conclusion

The main contributions of this paper are relaxed notionsof bisimulation and simulation that have nice characterisa-tions using logics, games and structural definitions. The no-tion of ε-bisimulation also gives rise to a new notion of anefficiently computable distance between systems that per-mits to measure differences for a long horizon without accu-mulating differences. As we explained, we consider that thelatter property is key in the probabilistic framework wheremodels involve probabilities are obtained from estimations

9

or experiments. Moreover, a long horizon is preferable forreactive systems that are expected to last. The fact that ourdistance is computable in polynomial time is an importantachievement of this paper, since the other known long hori-zon distance, d1, has a double exponential complexity [20].

We prove these results in the context of deterministicprobabilistic systems, but we also show how they can beextended to handle non determinism. Future work will pro-vide a logical characterisation in this setting.

We believe that something interesting is hiding behindthe fact that two-way ε-simulation is not ε-bisimulation forε > 0, contrarily to the situation when ε = 0. Here again, asin Example 5, we observe the analysis at the bounds mustbe done carefully.

The point of view we introduced may allow to build moreefficient states-clustering techniques for large state-spacesystems. Defining ε-simulation relations on systems withcontinuous state spaces is also an issue; of course, it wouldbe theoretically interesting, but it could also simplify thestudy of such systems, using a quotient machinery on thestate space with the relation ∼ε. As we have seen, studyingglobal or long term properties through an approximate logicis worth full. A complement for this approach, to tackle itscomplexity, would be to generalise parity games, whose res-olution is equivalent to µ-calculus model-checking [12], toour context and to our fixpoint logic.

References

[1] S. Abramsky and R. Jagadeesan. Games and full com-pleteness for multiplicative linear logic. J. Symbolic Logic,59(2):543–574, 1994.

[2] C. Baier. Polynomial time algorithms for testing probabilis-tic bisimulation and simulation. In Proceedings of the 8thInternational Conference on Computer Aided Verification(CAV’96), number 1102 in Lecture Notes in Computer Sci-ence, pages 38–49, 1996.

[3] C. Baier and M. Kwiatkowska. Domain equations for prob-abilistic processes. Mathematical. Structures in Comp. Sci.,10(6):665–717, 2000.

[4] S. Cattani and R. Segala. Decision algorithms for prob-abilistic bisimulation. In CONCUR ’02: Proceedings ofthe 13th International Conference on Concurrency Theory,pages 371–385, London, UK, 2002. Springer-Verlag.

[5] L. de Alfaro and R. Majumdar. Quantitative solution ofomega-regular games380872. In STOC ’01: Proceedings ofthe thirty-third annual ACM symposium on Theory of com-puting, pages 675–683, New York, NY, USA, 2001. ACM.

[6] L. de Alfaro, R. Majumdar, V. Raman, and M. Stoelinga.Game relations and metrics. In LICS ’07: Proceedings ofthe 22nd Annual IEEE Symposium on Logic in ComputerScience, pages 99–108, Washington, DC, USA, 2007. IEEEComputer Society.

[7] J. Desharnais, A. Edalat, and P. Panangaden. Bisimulationfor labeled Markov processes. Information and Computa-tion, 179(2):163–193, December 2002.

[8] J. Desharnais, V. Gupta, R. Jagadeesan, and P. Panangaden.Approximating labeled Markov processes. Information andComputation, 184(1):160–200, July 2003.

[9] J. Desharnais, V. Gupta, R. Jagadeesan, and P. Panangaden.Metrics for labelled Markov processes. Theor. Comput. Sci.,318(3):323–354, 2004.

[10] J. Desharnais, R. Jagadeesan, V. Gupta, and P. Panangaden.The metric analogue of weak bisimulation for probabilisticprocesses. In Proceedings of the 17th Annual IEEE Sympo-sium on Logic in Computer Science (LICS02), pages 413–422, Copenhagen, Denmark, July 2002. IEEE Computer So-ciety.

[11] J. Desharnais, F. Laviolette, and S. Zhioua. Testing proba-bilistic equivalence through reinforcement learning. In Pro-ceedings of the 26th Conference on Foundations of SoftwareTechnology and Theoretical Computer Science, FSTTCS2006, 2006.

[12] E. A. Emerson, C. S. Jutla, and A. P. Sistla. On model-checking for fragments of -calculus. In CAV, pages 385–396,1993.

[13] N. Ferns, P. Panangaden, and D. Precup. Metrics for Markovdecision processes with infinite state spaces. In Proceedingsof the 21th Annual Conference on Uncertainty in ArtificialIntelligence (UAI-05), page 201, Arlington, Virginia, 2005.AUAI Press.

[14] A. Giacalone, C.-C. Jou, and S. A. Smolka. Algebraic rea-soning for probabilistic concurrent systems. In Proceedingsof the Working Conference on Programming Concepts andMethods, IFIP TC2, 1990.

[15] K. G. Larsen and A. Skou. Bisimulation through probablistictesting. Information and Computation, 94:1–28, 1991.

[16] A. McIver and C. Morgan. Games, probability, and thequantitative µ-calculus qmµ. In LPAR ’02: Proceedings ofthe 9th International Conference on Logic for Programming,Artificial Intelligence, and Reasoning, pages 292–310, Lon-don, UK, 2002. Springer-Verlag.

[17] A. Parma and R. Segala. Logical characterisation of bisim-ulations for discrete probabilistic systems. In Proceedingsof Fossacs 07, volume 4423 of Lecture Notes In ComputerScience, pages 287–301, Braga, Portugal, 2007. Springer-Verlag.

[18] R. Segala and N. Lynch. Probabilistic simulations for proba-bilistic processes. In B. Jonsson and J. Parrow, editors, Pro-ceedings of CONCUR94, number 836 in Lecture Notes InComputer Science, pages 481–496. Springer-Verlag, 1994.

[19] M. P. K. V. Malhotra and S. Maheshwari. An O(|v|3) algo-rithm for finding maximum flows in networks. InformationProcessing Letters, 7:277–8, 1978.

[20] F. van Breugel, B. Sharma, and J. Worrell. Approximating abehavioural pseudometric without discount for probabilisticsystems. In H. Seidl, editor, FoSSaCS, volume 4423 of Lec-ture Notes in Computer Science, pages 123–137. Springer,2007.

[21] F. van Breugel and J. Worrell. An algorithm for quantitativeverification of probabilistic transition systems. In CONCUR’01: Proceedings of the 12th International Conference onConcurrency Theory, pages 336–350, London, UK, 2001.Springer-Verlag.

[22] F. van Breugel and J. Worrell. A behavioural pseudometricfor probabilistic transition systems, 2004.

10

A The relation≺ε is the greatest fixpoint of F

Definition 16 (The relation Rε). Rε is the relation definedon S × S by Rε =

⋂n≥0Rε,n, where the relations Rε,n

are defined inductively on n as follows:

• Rε,0 = S × S.

• sRε,n+1t iff for all a ∈ L and all set E ⊆ S we have:

– ha(t,Rε,n(E)) ≥ ha(s,E)− ε

Lemma 1. Rε is a relation of ε-simulation.

Proof. • Let s, t ∈ S be such that sRεt. Let a ∈ L,and X ⊆ S. We have to show that ha(t,Rε(X)) ≥ha(s,X)− ε.sRεt means that for all n ∈ N we have sRε,nt. Henceby an easy induction on nwe can see that for all n ≥ 0,ha(t,Rε,n(X)) ≥ ha(s,X)− ε.Next we can see by induction that the Rε,n forma decreasing sequence of relations. Thus theRε,n(X) form a decreasing sequence of sets. Bycontinuity of ha(t,−), we have ha(t,Rε(X)) =ha(t,

⋂n≥0Rε,n(X)) = infn≥0 ha(t,Rε,n), hence

ha(s,Rε(X)) ≥ ha(s,X)− ε.

We can prove now the result of this section:

Lemma 2. Rε =∼ε.Proof. We just have to show that if R is a relation of ε-simulation of S, then R ⊆ Rε. For this we fix R a relationof ε-simulation on S and we show by induction on n ∈ Nthat for all n ∈ N, R ⊆ Rε,n.

• The case n = 0 is clear since Rε,0 = S × S.

• Let n ∈ N and suppose R ⊆ Rε,n. Let s, t ∈S such that sRt. Let X ⊆ S, and a ∈ L.Then ha(t, R(X)) ≥ ha(s,X) − ε. But by induc-tion R(X) ⊆ Rε,n(X), hence ha(t,Rε,n(X)) ≥ha(s,X)− ε. This proves sRε,n+1t, i.e. R ⊆ Rε,n+1.

B Proof of Theorem 3

Theorem 3: Let s, t be two states of an LMP S with count-able state space. Then s ≺Lε t iff s ≺ε t.Proof. ⇐. Let R be a relation of ε-simulation. We proveby structural induction on formulas that for all φ ∈ Lε,R([[φ]]) ⊆ [[φ]]ε which will prove the result. The base case istrivial, and since R(−) distributes over unions, the case ofdisjunction is also trivial. For conjunction, the result comes

from the fact that for any two setsR(A∩B) ⊆ R(A)∩R(B)and from the following: for any two formulas θ1 and θ2,[[θ1 ∧ θ2]]ε = [[θ1]]ε ∩ [[θ2]]ε. Let use assume the claim forφ and prove it for 〈a〉q φ. Let t ∈ R([[〈a〉q φ]]). Then thereis some s |= 〈a〉q φ such that sRt. Since R is a simula-tion, and letting X = [[φ]] there is some Y ⊆ R([[φ]]) suchthat ha(t, Y ) > ha(s, [[φ]]) − ε. By induction hypothesis,R([[φ]]) ⊆ [[φ]]ε, and we obtain ha(t, [[φ]]ε) > ha(s, [[φ]])−ε.This implies that t ∈ [[〈a〉q φ]]ε as wanted.⇒. For the other direction we prove that≺Lε is a relation

of ε-simulation. Suppose that s ≺Lε t. Let a ∈ L, andE ⊆ S. We want to show that:

ha(t,Lε(E)) ≥ ha(s,E)− ε.

Where Lε(E) := {y ∈ S|∃e ∈ E|e ≺Lε y}.Let F = Lε(E) and λ < ha(s,E). Since S is countable,

there exists E′ ⊆ E finite such that ha(s,E′) > λ. Notethat E′ ⊆ Lε(E), and E′ = ∪e′∈E′ ∩e′|=ψ [[ψ]]ε. Indeed, itis easy to check that ∩e′|=ψ[[ψ]]ε is exactly the set of statesthat Lε-simulate e′. Clearly we can consider that the param-eters in the formulas of L are taken in Q, and hence we canenumerate the formulas of Lε as (ψi)i∈N.

Since s ≺Lε t, it implies that for all formula ψ ∈ Lε,ha(t, [[ψ]]ε) ≥ ha(s, [[ψ]]) − ε. In particular, we havefor all k ∈ N we have ha(t, [[∨e′∈E′ ∧e′|=ψi,i<k ψi]]ε) ≥ha(s, [[∨e′∈E′ ∧e′|=ψi,i<k ψi]]) − ε. By continuity ofha(t,−) and ha(s,−), and since the sets are decreasing to∪e′∈E′ ∩e′|=ψ [[ψ]]ε and ∪e′∈E′ ∩e′|=ψ [[ψ]] as k increases, itfollows that

ha(t,∪e′∈E′ ∩e′|=ψ [[ψ]]ε) ≥ ha(s,∪e′∈E′ ∩e′|=ψ [[ψ]])− ε.

We can now prove that ha(t, F ) ≥ λ− ε.

ha(t, F ) = ha(t,Lε(E))≥ ha(t,Lε(E′))= ha(t,∪e′∈E′ ∩e′|=ψ [[ψ]]ε)≥ ha(s,∪e′∈E′ ∩e′|=ψ [[ψ]])− ε≥ ha(s, E′)− ε≥ λ− ε

.

Since λ was arbitrary, we indeed have that ha(t, F ) ≥ha(s,E)− ε.

The preceding proof uses crucially the countability of thestate space. An extension to uncountable state spaces couldhave to go through non trivial domain theory machinery aswas done in [8].

C Proof of Theorem 4

We prove the following lemma which is used in the proofof theorem 4.

Lemma 3. ≺Gε is a relation of ε-simulation.

11

Proof. This is a mere reformulation of the game rules. Lets, t ∈ S be such that s ≺Gε t. Let E ⊆ S and a ∈ L.Wehave to show that there exists a set F ⊆≺Gε (E) such thatha(t, F ) ≥ ha(s,E) − ε. But P has a winning strategy.Hence P can find a “winning” F at Step 3, that is an F suchthat ha(t, F ) ≥ ha(s,E) − ε and such that for any choicet′ that A makes in Step 3, P will be able to find s′ ∈ F suchthat he has a winning strategy for the game S, s′, t′. Thismeans that P can find a set F ⊆ S such that ha(t, F ) ≥ha(s,E) − ε and for all t′ ∈ F , there exists s′ ∈ E suchthat s′ ≺Gε t′, i.e. F ⊆≺Gε (E). Hence such a set F isconvenient for the fact that ≺Gε is a relation of ε-simulation.

D Lemma for Theorem 6

Lemma 4. ∼L¬ε is a symmetric relation.

Proof. Of course this will come from the presence of thenegation in the logic. Let s, t ∈ S, such that s ∼L¬ε t.Suppose t |= θ. We show that s |=ε θ, i.e. that s |= θε. Ifnot, s |= ¬θε. Hence since s ∼L¬ε t we would have t |=ε

¬θε, i.e. t |= ¬θε−ε, i.e. t |= ¬θ, which is a contradiction.

E Network theory. Proof of Theorem 7

In order to prove Theorem 7 we need to introduce someconcepts of flow network theory.

A network is a tuple N = (V,E,⊥,>, c) where (V,E)is a finite directed graph in which every edge (u, v) ∈ E hasa non-negative, real-valued capacity c(u, v). If (u, v) 6∈ Ewe assume c(u, v) = 0. We distinguish two vertices: asource ⊥ and a sink >.

For v ∈ V let in(v) be the set of incoming edges to nodev, and out(v) the set of outgoing edges from node v.

A flow function is a real function f : V × V → R withthe two following properties for all nodes u and v:

• Capacity constraints: 0 ≤ f(u, v) ≤ c(u, v). The flowalong an edge cannot exceed its capacity.

• Flow conservation: for each node v ∈ V − {⊥,>},∑e∈in(v) f(e) =

∑e∈out(v) f(e)

The flow F(f) of f is given by

F(f) =∑e∈out(⊥) f(e)−∑

e∈in(>) f(e).

A cut of a network is a split of the nodes in V into twosets S and T , such that ⊥∈ S and > ∈ T . The capacity ofa cut (S, T ) is defined as:

c(S, T ) =∑u∈S,t∈T |(s,t)∈E c(s, t).

We begin by a presentation of a flow network that hasbeen used in the past few years to compute efficiently sev-eral simulation relations on the state space of a probabilisticsystem. (cf Baier, Hermanns, Kwiatkowska...). Our notionof simulation is formulated in a different way than theirs.We speak about sets and ”right-closed” sets, whereas theyuse the notion of weight functions between two distribu-tions. As a consequence our characterisation of simulationin terms of maximum flow in a network is more tricky.

We present here the flow network introduced byBaier [2]. and show that our notion of simulation can fitin that context and generalise her notion of simulation.

Definition 17 (The network N (µ, ν,R)). Let S be a finiteset, R a subset of R × S, and µ, ν two distributions on S.Let S′ = {t′|t ∈ S}, where t′ are pairwise distinct ”new”states (i.e. t′ 6∈ S). We choose new elements ⊥ and > notcontained in S∪S′,⊥6= >. Then the networkN (µ, ν,R) =(V,E,⊥,>, c) is defined as follows:

• V = S ∪ S′ ∪ {⊥,>}.

• E = {(s, t′)|(s, t) ∈ R} ∪ {(⊥, s)|s ∈ S} ∪{(t′,>)|t ∈ S}.

• The capacity function c is given by: c(⊥, s) = µ(s),c(t′,>) = ν(t), and c(s, t′) = 1 for all s, t ∈ S.

For the main theorem we will need some notions aboutnetworks we present now. For the following we fix a finiteset S, a relation R on S × S, two distribution µ, ν on S,and the associated network N = N (µ, ν,R).

We are interested in the following theorem, which gen-eralises the result of [2].

Theorem 9. Let S be a finite state space of size k, R ⊆S × S a relation, and µ, ν ∈ SubD(S). Then the followingare equivalent:

1. The maximum flow in N (µ, ν,R) is greater or equalthan µ(S)− ε.

2. For all E ⊆ S we have ν(R(E)) ≥ µ(E)− ε.

Proof.

1 ⇒ 2. Suppose the maximum flow in N (µ, ν,R) isgreater or equal than µ(S) − ε, i.e. that µ(S) − ε ≤∑s∈S f(>, s) (1). Let E ⊆ S. Let f be a function of

maximal flow in N (µ, ν,R).We have:

∑s∈S f(>, s) ≤ µ(S − E) +

∑s∈E f(>, s),

and∑s∈E f(>, s) =

∑s∈E

∑t∈R(E) f(s, t′) ≤∑

t∈R(E) ν(t′).

12

Hence

∑s∈S f(>, s) ≤ µ(S)− µ(E) +

∑t∈R(E) ν(t

′),

With (1), we get indeed µ(E)− ε ≤ ν(R(E)).

2 ⇒ 1. We use the powerfull Max-flow Min-cut theorem toprove this way. The Max-flow Min-cut theorem shows thatin our context, we just have to prove that the minimal cuton the network N (µ, ν,R) is no less than µ(S)− ε. First anotation: we write {s1, ..., sn} for the set S, and the nodesof the network are: {>,⊥} ∪ {s1, ..., sn} ∪ {s′1, ..., s′n}. If(X,Y ) is a cut on N (µ, ν,R), we will say that s, t ∈ Ssatisfy (1) for the cut (X,Y ) if:

s ∈ X , t′ ∈ Y , and sRt. (1)

Let (X, y) be a cut on N (µ, ν,R). We want to show thatits capacity is no less than µ(S) − ε. Our first remark isthat we can suppose that there are no s, t ∈ S satisfying(1) for (X,Y ). Indeed, suppose (X,Y ) contains such s, t.then define X ′ = X ∪ {t}, and Y ′ = Y \{t′}. (X ′, Y ′) isof course a cut on N (µ, ν,R). Moreover, the capacity of(X ′, Y ′) is smaller than the capacity of (X,Y ). Indeed wehave c((X,Y )) = c((X ′, Y ′)) + c(s, t′) − c(t′,>), that isc((X,Y )) = c((X ′, Y ′)) + 1 − ν(t). Since ν(t) ≤ 1 forall t ∈ S, we get the result. Now, there exist stricly lesscouple s, t in number which verify (1) for (X ′, Y ′) thanfor (X,Y ). If we keep going this way, we construct a cut(X ′′, Y ′′) onN (µ, ν,R) such that no couple s, t in S satisfy(1) for (X ′′, Y ′′), and whose capacity is lower or equal thatthe capacity of (X,Y ).

Now let (X,Y ) is a cut onN (µ, ν,R) such that no s, t ∈S satisfying (1) for (X,Y ). Let X1 = X ∩ {s1, ..., sn},X2 = X∩{s′1, ..., s′n}, Y1 = Y ∩{s1, ..., sn}, and Y2 = Y ∩{s1, ..., sn}. By definition of a cut, {s1, ..., sn} = X1 ∪ Y1

and {s′1, ..., s′n} = X2 ∪ Y2. Now, the fact that no s, t ∈ Ssatisfy (1) for (X,Y ) allows us to simplify the expressionof the capacity of this cut. Indeed we get:

c((X,Y )) =∑s∈Y1

c(⊥, s) +∑t′∈X2

c(t′,>).

For all t′ ∈ X2, c(t′,>) = ν(t) and c(⊥, s) = µ(s), wheret is naturally the state of S associated to t′. (For simplicity,in the future if K ⊆ {s′1, ..., s′n}, we write simply ν(K) orµ(K) for

∑t′∈K ν(t) or

∑t′∈K ν(t) repectively). Finally

we get c((X,Y )) = µ(Y1) + ν(X2). Another consequenceof the fact that no s, t ∈ S satisfy (1) for (X,Y ) is thatR(X1) ⊆ X2. This and the fact that {s1, ..., sn} = X1∪Y1

implies c((X,Y )) ≥ µ({s1, ..., sn} − X1) + ν(R(X1)).Now, by hypothesis, for all E ⊆ S we have ν(R(E)) ≥µ(E)− ε. Finally: c((X,Y )) ≥ µ({s1, ..., sn})−µ(X1)+µ(X1)− ε, i.e. c((X,Y )) ≥ µ(S)− ε.

F Expressiveness of the fixpoint logic Lµ

We begin by a precise definition for the approximationof the fixpoint logic. As for L, to each formula φ ∈ Lµ andε ∈ [−1; 1] we associate a new formula φε. We can definedirectly the associated approximate semantics |=ε⊆ S × Sas follows: if φ ∈ Lµ and s ∈ S, s |=ε φ iff s |= φε.

Definition 18. Let ε ∈ [−1; 1]. To each formula φ ∈ Lµ weassociate a new formula φε. The construction is defined bystructural induction on the formulas of Lµ as follows:

• If φ ∈ L, φ = ¬ψ, φ = 〈a〉δ ψ with ψ closed, φ =ψ1 ∧ ψ2 or φ = ψ1 ∨ ψ2, φε is constructed the sameway as for the logic L. (i.e. for instance (〈a〉.5 ψ)ε =〈a〉.5−ε ψε).

• If φ = 〈a〉1 Z, then φε = φ.

• If φ = µZ.ψ then φε = µZ.ψε.

• If φ = νZ.ψ then φε = νZ.ψε.

The important fact is that we do not relax the satisfia-bility condition before a fixpoint variable. This way, weimpose that the paths generated when considering the satis-faction of a global or long term formula are all taken withprobability 1.

Now the proof of Proposition 3:

• The logic Lµ has the same differentiating power as L.

• ∼⊆∼µε⊆∼ε and the inclusions may be strict. That is∼µε has a differentiating power between ∼ and ∼ε.

Proof. • The fact that L and Lµ have the same differen-tiation power comes easily from the fact that the sys-tem we consider have finite state space. Hence thefixpoint sets may be computed in a finite number ofPicard iterations. If we have a formula of Lµ whichdifferentiate two states s, t, we can unroll the fixpointcalculation to get a formula of finite depth, i.e. a for-mula of L, which differentiates these two states

• Clearly the last point implies ∼⊆∼µ0 . And sinceclearly ∼µε , seen as a function mapping ε > 0 to asubset of S ×S, is a growing function, we get ∼⊆∼µε .Finally, since L is a subset of Lµ, we get∼µε⊆∼ε. Thestrictness of the inclusions will follow from the exam-ples we consider next.

G Algorithm for computing∼ε in polynomialtime

Algorithm 2. Input: A finite state LMP S = (S,L, h),ε ∈ [0; 1].

13

Output: The relation ∼ε on S × S.method:Let R0 = S × S.For j = 0 to |S|2 do begin:Rj+1 = Rj .For all (s, t) ∈ Rj do begin:

For all a ∈ L do begin:If the maximum flow of the networkN (ha(s,−), ha(t,−),Rj) is strictly smallerthan ha(s, S) − ε, or if the maximum flowof the network N (ha(t,−), ha(s,−),Rj)is strictly smaller than ha(t, S) − ε thenRj = Rj − {s, t}.

endend end

return R|S|2 .

Using the algorithm that computes the maximum flow ofthe networks N (ha(s,−), ha(t,−),Rj) in time O(|S|3),we get an algorithm which computes ∼ε on S × S in timeO(|S|7 · |L|).

H Theorem 8: the kernel of d is ∼

In this section S is a countable LMP. We give a proof ofTheorem 8: For s, t ∈ S, we have s ∼ t iff d(s, t) = 0, i.e.iff for all ε > 0 we have s ∼ε t.Proof. The fact that ∼ is a subset of the kernel of d is clear.We use the logical characterisation of ∼ε: we show by in-duction on the structure of the formulas that for any φ ∈ Lthe following are equivalent:

1. s |= φ⇒ t |= φ.

2. For all ε > 0, s |= φ⇒ t |=ε φ.

By symmetry, this will give that for all φ ∈ L, s |= φ ⇔t |= φ iff for all ε > 0 s |= φ⇒ t |=ε φ and t |= φ⇒ s |=ε

φ. Hence, for all φ ∈ L, this will prove that:

(∀ε > 0 s ∼ε t) ⇒ (s |= φ⇔ t |= φ)

which implies s ∼ t if d(s, t) = 0.Now 1 ⇒ 2 is clear. We prove 2 ⇒ 1 by induction on

the structure of the formulas

• The case φ = > is clear.

• The cases φ = ¬ψ, φ = ψ1 ∧ ψ2, φ = ψ1 ∨ ψ2 aresimple as well.

• The interesting case is this one: Suppose φ = 〈a〉δ ψ.Let λ < ha(s, [[ψ]]). Suppose 2 and s |= φ. By hypoth-esis, for any ε > 0 we have ha(t, [[φε]]) ≥ λ−ε. But byinduction

⋂n≥1[[φ1/n]] = [[φ]], hence by continuity of

the operator ha(t,−), we have ha(t,⋂n≥1[[φ1/n]]) =

ha(t, [[psi]]) and ha(t,⋂n≥1[[φ1/n]]) ≥ λ. We get

ha(t, [[ψ]]) ≥ λ. Since this is true for any λ <ha(s, [[ψ]]), this proves ha(t, [[ψ]]) ≥ ha(s, [[ψ]]), hencet |= φ.

I Algorithm for computing ∼ε on a proba-bilistic automata in polynomial time

Algorithm 3. Input: A finite state probabilistic automataA = (S,L, h), ε ∈ [0; 1].Output: The relation ∼ε on S × S.method:Let R0 = S × S.For j = 0 to |S|2 do begin:Rj+1 = Rj .For all (s, t) ∈ Rj do begin:

For all a ∈ L, and s a→ µ ∈ D do begin:If for all t a→ ν ∈ D the maximum flow ofthe networkN (µ, ν,Rj) is strictly smaller thanµ(S)− ε then Rj = Rj − {s, t}.

endFor all a ∈ L, and t a→ ν ∈ D do begin:

If for all s a→ µ ∈ D the maximum flow ofthe networkN (ν, µ,Rj) is strictly smaller thanν(S)− ε then Rj = Rj − {s, t}.

endend end

return R|S|2 .

Using the algorithm that computes the maximum flowof the networks N (µ, ν,Rj) in time O(|S|3), we get analgorithm which computes∼ε on S×S in timeO(|S|7 ·|L|·m2), where m is maxCard{s a→ µ ∈ D|s ∈ S, a ∈ L}.Usually we have that m is a constant independent of n.

14