Minimax Games for Stochastic Systems Subject

to Relative Entropy Uncertainty:

Applications to SDE’ s on Hilbert Spaces

N.U. AHMED1 and C.D. CHARALAMBOUS2

Abstract-In this paper we consider minimax games for stochastic uncertain systems

with the pay-off being a nonlinear functional of the uncertain measure where the

uncertainty is measured in terms of relative entropy between the uncertain and the

nominal measure. The maximizing player is the uncertain measure, while the mini-

mizer is the control which induces a nominal measure. Existence and uniqueness of

minimax solutions are derived on suitable spaces of measures. Several examples are

presented illustrating the results. Subsequently, the results are also applied to con-

trolled Stochastic Differential Equations (SDE’s) on Hilbert Spaces. Based on infinite

dimensional extension of Girsanov’s measure transformation, martingale solutions are

used in establishing existence and uniqueness of minimax strategies. Moreover, some

basic properties of the relative entropy of measures on infinite dimensional spaces are

presented and then applied to uncertain systems described by a Stochastic Differen-

tial inclusion on Hilbert space. The worst case measure representing the maximizing

player (adversary) is found and it is shown that it is described by an evolution equa-

tion on the space of measures on the original state space.

1N.U. Ahmed is with the School of Information Technology and Engineering and De-partment of Mathematics University of Ottawa, Ottawa, Ontario, CANADA.E-mail: ahmed@site.uottawa.ca

2C.D. Charalambous is with the Department of Electrical and Computer Engineering,University of Cyprus, Nicosia, CYPRUS. Also with the School of Information Technologyand Engineering, University of Ottawa, Ottawa, Ontario, CANADA.E-mail: chadcha@ucy.ac.cyThis work is supported by the European Commission under the project ICCCSYSTEMSand by the University of Cyprus under an internal medium size research project.

1

1 Motivation

Minimax games are important when dealing with an uncertain system which may

consist of a whole family of (mathematically) well defined systems. In the signal

processing and control community, the designer is the estimator or the control player

which wishes to minimize the pay-off challenged (presented) by the nature. This is

modeled as a functional of measures induced by the uncertain system. Specifically,

robustness problems in control theory can be formulated using this framework. In the

past several papers have appeared dealing with minimax games [11, 12, 14]. In [15],

the stochastic analog is addressed, in which the maximizing players are described by

measures. Moreover, in [16, 17], discrete-time minimax problems are also introduced

in which the maximizing measure is parameterized by random processes describing the

uncertainty of the system. In [18, 19], an alternative approach is put forward in which

the uncertain measure for general nonlinear partially observable uncertain systems is

computed. These models as well as the results presented are directly or indirectly

related to H∞ problems [6], dissipation inequalities [7], and risk-sensitive problems

[8-19]. An explanation of the various connections can be found in [20]. Uncertain

systems and Variational inequalities can be described by differential inclusions in

a unified framework covering both deterministic and stochastic systems. Optimal

control and nonlinear filtering of such systems are considered in several papers [21-

25].

This paper is fundamentally different from the above described references in both the

theory and applications considered. The theoretical development is concerned with

minimax games in which the pay-off is a nonlinear functional of the uncertain measure.

The maximizing player belongs to a set described by relative entropy between the

uncertain and the nominal measures, while the control affects the nominal measure.

Existence and uniqueness of the minimax strategies are presented on the space of

2

measures defined on Polish spaces. The results are then applied to controlled Stochas-

tic Differential equations (SDE’s) and inclusions defined on Hilbert Spaces. Infinite

dimensional extension of Girsanov’s measure transformation is used in establishing

existence and uniqueness of minimax strategies. The worst case measure is found and

it is shown that it is governed by an evolution equation on the space of countably ad-

ditive probability measures on Hilbert Space. To the best of the authors knowledge,

the treatment found here appears to be the first one dealing with minimax games

subject to relative entropy constraints for SDE’s and inclusions on Hilbert spaces.

2 Introduction

Let X be a Polish space, in particular, a complete separable metric space and let

BX the sigma algebra of Borel sets generated by the metric topology. Let M1(X)

denote the space of countably additive regular probability measures defined on BX .We assume throughout the paper that M1(X) is furnished with the weak topology.

It is well known that the weak topology is metrizable with respect to the Prohorov

metric which makes it a complete metric space. We shall denote this metric by dP .

Thus, a net µαw−→ µo if and only if dP (µα, µo) −→ o. Since the weak topology and

the metric topology are equivalent we note the following facts. A set Γ ⊂ M1(X) is

compact with respect to the weak topology, if and only if, it is compact with respect

to the metric topology dP . A functional g : M1(X) −→ R is weakly continuous, if

and only if, it is continuous with respect to the Prohorov metric dP .

Let ν, µ ∈ M1(X) and suppose ν is absolutely continuous with respect to the measure

µ, which is denoted by ν ≺ µ. Then, the Radon-Nikodym derivative of ν with respect

to µ exists and this is given by an h ∈ L1(X,µ) such that

dν = hdµ.

We note that, given that ν ≺ µ, the RND is unique, that is, there is only one

3

h ∈ L1(X,µ) that satisfies the above identity. Indeed, if both h and f satisfy the

above identity, then ∫Xϕ(x)h(x) − f(x) dµ = 0

for all ϕ ∈ L∞(X,µ). This is impossible unless h = f µ a.e. Identifying h with f that

differ from h only on a set of µ measure zero, we have the uniqueness. If further,

h log h ∈ L1(X,µ), then the entropy of ν relative to the measure µ, known as relative

entropy, is defined and it is given by

H(ν|µ) ≡∫Xh log hdµ =

∫X(dν/dµ) log(dν/dµ)dµ

=∫Xlog(dν/dµ)dν. (2.1)

The basic problem we wish to study can be stated as follows.

Basic Problem: Let η : M1(X) −→ [−∞,∞), be an extended real valued continu-

ous function. Let M0 ⊂ M1(X) denote the set of measures induced by a controlled

stochastic system under the assumptions that the system is perfectly known (system

operators and other parameters and coefficients etc., are all perfectly known). In

other words, M0 denotes the set of attainable measures on X associated with an

unambiguously defined (perfectly known) controlled stochastic system. However, in

the real world situation, the system parameters are not entirely known to the con-

troller. This introduces some degree of uncertainty in the system dynamics around

the nominal controlled system and hence the set of measures induced by the true

system may differ from those represented by the set M0. For each fixed r > 0 and

µ ∈ M0 ⊂ M1(X), define the set

Ar(µ) ≡ ν ∈ M1(X) : H(ν|µ)) ≤ r. (2.2)

This set represents the set of true measures induced by the actual physical system

given that the nominal controlled system induces the measure µ. This is the set of

uncertainty about the system measured in terms of relative entropy, relative to the

4

measure µ. Thus, the set of measures induced by the uncertain controlled system is

given by

Mr ≡⋃

(Ar(µ)), µ ∈ M0

.

Clearly M0 ⊂ Mr for all r ≥ 0 and limr↓0 Mr = M0. Since the relative entropy is

nonnegative, the parameter r determines the degree of uncertainty. Thus the larger

the r is, the greater is the uncertainty.

The problem is to find a µo ∈ M0 that minimizes the functional

Fr(µ) ≡ supη(ν); ν ∈ Ar(µ) (2.3)

over the set M0. In other words, we wish to find a solution to the min-max problem

(Pis) : infµ∈M0

supη(ν) : ν ∈ Ar(µ) (2.4)

In [15] a specific linear functional η(ν) is considered and results related to Large De-

viations Theory are derived. Here, η(ν) is a general nonlinear functional on M1(X).

3 Existence of the Min-Max Solution

In the following section we investigate the question of existence of solution to the

basic problem (Pis).

Lemma 3.1 Let η : M1(X) −→ [−∞,+∞) be an upper semi continuous function

and strictly concave. Then, for every µ ∈ M0 ⊂ M1(X), there exists a unique

νo ∈ Ar(µ) at which η attains its supremum.

Proof. Without loss of generality, we may assume that η(ν) ≡ −∞. If so there is

nothing to prove. It is well known that, for each µ ∈ M1(X), Ar(µ) is a closed convex

5

subset of M1(X) and compact with respect to the metric topology dP [1,Lemma 1.4.3,

pp. 37]. Let

supη(ν), ν ∈ Ar(µ) ≡ mo.

We show that there exists a νo ∈ Ar(µ) at which the sup is attained. Let να ∈ Ar(µ)

be a maximizing net so that

limαη(να) = supη(ν) : ν ∈ Ar(µ) = mo. (3.5)

Since Ar(µ) is compact, there exists a subnet of the net να, relabeled as the original

net, and a νo ∈ Ar(µ) so that να −→ νo in the Prohorov topology. It follows from

upper semi continuity of η and (3.5) that

mo = limαη(να) ≤ lim sup

αη(να) ≤ η(νo). (3.6)

Since νo ∈ Ar(µ), it is apparent that η(νo) ≤ mo. It follows from this and the above

inequality that η(νo) = mo. Thus η attains its supremum on Ar(µ). Since η is strictly

concave and Ar(µ) is convex, it is clear that the maximizer is unique. Indeed, suppose

there is another element µo ∈ Ar(µ) at which η attains its maximum. By convexity of

Ar(µ), we have (1/2)νo+(1/2)µo ∈ Ar(µ), and since η is a strictly concave functional,

we have

η((1/2)νo + (1/2)µo) > (1/2)η(νo) + (1/2)η(µo) = mo. (3.7)

Since mo is the maximum on Ar(µ), this is impossible. This contradiction completes

the proof. •

Remark 3.2 It is clear from the above result that under the given assumptions, η

attains its supremum at a unique point in the set Ar(µ).

The following Lemma is used to established existence of the minimizing player.

6

Lemma 3.3 Suppose the assumptions of Lemma 2.1 hold. Then the graph G(Ar) of

the multifunction Ar is sequentially closed and the function Fr as defined by (2.3) is

lower semi continuous.

Proof. By definition the function Fr is given by

Fr(µ) ≡ supη(ν); ν ∈ Ar(µ).

By virtue of the previous lemma, for every µ ∈ M0, there exists a unique ν ∈ Ar(µ)

so that

Fr(µ) = η(ν) ≡ η(ν(µ)),

that is, the maximizer ν is uniquely determined by µ. Thus Fr is a well defined single

valued functional. First we prove that the graph of the multifunction µ −→ Ar(µ)

given by

G(Ar) ≡ (ν, µ) ∈ M1(X) ×M1(X) : ν ∈ Ar(µ)

is closed. Suppose (θn, βn) ∈ G(Ar) and θndP−→ θo, βn

dP−→ βo. We must show that

(θo, βo) ∈ G(Ar). Since θn ∈ Ar(βn), it follows from the definition of the multifunction

Ar that we have

H(θn|βn) ≤ r, for all n ∈ N.

Thus, it suffices to show that H(θo|βo) ≤ r. It is known that, by virtue of the Donsker-

Varadhan variational formula, the map (ν, µ) −→ H(ν|µ) is convex and lower semi-

continuous on its domain of definition. Thus

H(θo|βo) ≤ lim infH(θn|βn) ≤ r,

and hence the graph of Ar is sequentially closed, that is, θo ∈ Ar(βo). Now we show

that Fr is lower semi continuous. Let µo ∈ M1(X) and suppose µndP−→ µo. We show

that, along a subsequence if necessary, Fr(µo) ≤ lim inf Fr(µ

n). For the fixed positive

number r and given µo ∈ M1(X), consider the set

Ar(µo) ≡ ν ∈ M1(X) : H(ν|µo) ≤ r. (3.8)

7

Take any ε > 0 and consider the family of sets

Ar+ε(µn) ≡ ν ∈ M1(X) : H(ν|µn) ≤ r + ε, ε > 0. (3.9)

Since µndp−→ µo, for every ε > 0 there exists an nε ∈ N such that

Ar(µo) ⊂ Ar+ε(µ

n) ∀ n ≥ nε. (3.10)

By our assumption η is upper semi continuous and strictly concave. Since the above

sets are convex and compact (in the Prohorov topology), it follows from Lemma 3.1

that η attains unique supremum on each of the above sets and they are determined

by µo and µn, ε alone. Thus, by our definition of Fr given by the expression (2.3),

we have

Fr(µo) = supη(ν), ν ∈ Ar(µ

o) (3.11)

Fr+ε(µn) ≡ supη(ν), ν ∈ Ar+ε(µ

n). (3.12)

Clearly, it follows from the inclusion (3.10) and the identities (3.11), (3.12) that

Fr(µo) ≤ Fr+ε(µ

n), ∀ n ≥ nε. (3.13)

Hence

Fr(µo) ≤ lim inf

n→∞ Fr+ε(µn), ε > 0. (3.14)

Since this holds for any ε > 0, we have

Fr(µo) ≤ lim inf

n→∞ Fr(µn). (3.15)

Thus µ −→ Fr(µ) is lower semi continuous at µo. Since µo is arbitrary, this implies

that Fr is lower semi continuous on M1(X). •

Combining these results we prove the existence of solution of the problem (Pis).

Theorem 3.4 Consider the problem (Pis) and suppose that η : M1(X) −→ [−∞,+∞)

is upper semi continuous and strictly concave and the set M0 is compact with respect

to the Prohorov topology. Then the problem (Pis) has a solution.

8

Proof. The proof follows from Lemma 3.3. Indeed, by virtue of Lemma 3.3, the map

µ −→ Fr(µ) is lower semi continuous and by assumption M0 is compact. Hence Fr

attains its minimum on M0. •.

So far we have considered inf-sup problem. In certain situations the problem may be

reversed and we are required to solve the sup-inf problem

(Psi) : supµ∈M0

infη(ν) : ν ∈ Ar(µ).

This is solved in the following theorem.

Theorem 3.5 Consider the problem (Psi) and suppose η : M1(X) −→ (−∞,∞] is

lower semi continuous and the set M0 is compact in the Prohorov topology. Then the

problem (Psi) has a solution.

Proof. Define the function

Gr(µ) ≡ infη(ν) : ν ∈ Ar(µ).

Since η is lower semi continuous and bounded away from −∞ and Ar(µ) is compact,

this is a well defined real valued function on M1(X). As in Lemma 3.3, one can verify

that for any (dP ∼= weakly) convergent sequence µndP−→ µo in M1(X), we have

Gr(µo) ≥ lim sup

nGr(µ

n).

Thus µ −→ Gr(µ) is upper semi continuous. Since M0 is compact, Gr attains its

supremum on M0. This completes the proof. •

A Special Case. Let g : Rn −→ R be a real valued function and ϕi, i = 1, 2, · · ·n ∈BM(X), a family of bounded Borel measurable functions defined on X. Define the

9

functional

ηg(ν) ≡ g(ν(ϕ1), ν(ϕ2), · · · , ν(ϕn)) (3.16)

where ν(ϕ) ≡ ∫X ϕ(x)ν(dx), ϕ ∈ BM(X). The cost functional is given by

Fg(µ) ≡ supηg(ν), ν ∈ Ar(µ). (3.17)

The problem is to find a µ ∈ M0 that imparts a minimum to the functional Fg.

As a corollary of Theorem 3.4 we have the following result.

Corollary 3.6 Let g : Rn −→ [−∞,+∞) be upper semi continuous and strictly

concave, ϕi1≤i≤n an arbitrary family from BM(X), and M0 a compact subset of

M1(X). Then there exists a µ∗ ∈ M0 at which the functional Fg, given by (3.17),

attains its minimum.

Remark 3.7 It is important to point out that the concavity assumption imposed on

the functional η guarantees uniqueness of the maximizer. But this is not necessary if

we are only interested in the question of existence of a solution to the inf-sup problem

(Pis) as described by the expression (2.4). In this case all the preceding results remain

valid without the uniqueness. To clarify this further, take any element µ ∈ M1(X)

and define

Cr(µ) ≡ν ∈ Ar(µ) : η(ν) = supη(ϑ), ϑ ∈ Ar(µ)

.

This is the set of maximizers of η on the set Ar(µ). It follows from the upper semi

continuity of η that this set is closed and hence, being a closed subset of a compact set,

is compact. Clearly, the functional Fr(µ), as given by the expression (2.3), satisfies

the obvious identity,

Fr(µ) = η(Cr(µ)).

Thus the lack of concavity does not affect the functional Fr and hence the results on

existence.

10

3.1 Examples

We present some examples illustrating the preceding results.

(E1)(Target seeking Problem): Let µd ∈ M1(X) be a desired measure, and

suppose we wish to find a µ ∈ M0 (the attainable set) that approximates µd as closely

as possible. Clearly the obvious choice of the objective functional is the Prohorov

metric giving

η(ν) ≡ dP (µd, ν).

Since dP is a metric, the functional η as defined is continuous. If one disregards

the uncertainty, one would be willing to minimize this functional on the attainable

set of the perfect system given by M0. By our assumption this set is compact and

since η is continuous it attains its minimum on it. Let µo ∈ M0 be a minimizer.

Then η(µo) ≤ η(µ), µ ∈ M0. But since the system is uncertain, the actual law (true

measure) in force may be any element from Ar(µo) and one may face the worst

situation,

η(ν∗) ≡ supη(ν), ν ∈ Ar(µo) = Fr(µ

o).

Clearly η(ν∗) ≥ η(µo) with H(ν∗|µo) ≤ r. So instead of choosing µo, we may choose

one that minimizes the maximum distance. That is, we choose an element µ∗ ∈ M0

so that

Fr(µ∗) = infFr(µ), µ ∈ M0

where

Fr(µ) ≡ supη(ν) ≡ dP (µd, ν) : ν ∈ Ar(µ).

Clearly

Fr(µ∗) ≤ Fr(µ

o) = η(ν∗).

Thus, we conclude that the inf-sup strategy is closer to the desired goal than the

strategy based on the deterministic dynamics with attainable set M0.

11

(E2)(Evasion Problem): Let Γk, 1 ≤ k ≤ n ⊂ BX be a family of disjoint sets

and suppose we wish to find a measure from the attainable set M0 that has minimum

concentration of mass on these sets. This represents an evasion problem. But since

the system is uncertain, we have a sort of games problem, game against natural

uncertainty. Thus we may formulate the problem as follows. Let g : Rn+ −→ R+ ≡

[0,∞) be any bounded continuous function, increasing in all its arguments. Define

the functional η as follows,

η(ν) ≡ g(ν(Γ1), · · · , ν(Γn))≡ g(ν(χΓ1), · · · , ν(χΓn)),

where χΓ denotes the indicator function of the set Γ ∈ BX . Disregarding uncertainty,

and noting that η is continuous, and M0 compact, there exists a µo ∈ M0 at which

η attains its minimum, η(µo) ≤ η(µ), µ ∈ M0. Considering the uncertainty at µo we

have Fr(µo) ≥ η(µo) where the function Fr is as defined before. Since Fr is lower semi

continuous and M0 is compact, letting µ∗ ∈ M0 denote the point at which Fr attains

its infimum, we conclude that Fr(µ∗) ≤ Fr(µ

o). In other words the inf-sup strategy

exists and is better than pure deterministic strategy which ignores the presence of

uncertainty.

In example (E2), if we assume that the sets Γk, 1 ≤ k ≤ n are closed subsets of

X, then for the existence of inf-sup strategy, it is sufficient if g is merely upper semi

continuous.

(E3)(Hitting Problem) Let O1,O2, · · · ,On be a family of open subsets of X

and g : Rn −→ (−∞,∞] an increasing function of all its arguments. Consider the

functional ηg given by

ηg(ν) ≡ g(ν(O1), · · · , ν(On)).

The objective is to find a ν from the attainable set of measures that maximizes the

12

concentration of mass on the given sets. This is equivalent to hitting the target with

maximum probability. Again due to the presence of uncertainty, we must maximize

the minimum concentration giving rise to the sup-inf problem

supµ∈M0

infηg(ν), ν ∈ Ar(µ).

By virtue of Theorem 3.5, it suffices to verify that the functional ηg, as defined above,

is lower semi continuous on M1(X) in the Prohorov topology. Suppose νkw−→ νo,

then, since Oi are open sets, we have

lim infk→∞

νk(Oi) ≥ νo(Oi), i = 1, 2, · · · , n.

Thus, g being an increasing function of it’s arguments, we have

lim infk→∞

g(νk(O1), · · · , νk(On)) ≥ g(lim infkνk(O1), · · · , lim inf

kνk(On))

≥ g(νo(O1), · · ·νo(On)).

Hence

lim infk→∞

ηg(νk) ≥ ηg(ν

o)

and so ηg is lower semi continuous on M1(X). Since for each µ ∈ M1(X), Ar(µ) is

compact, ηg attains its infimum on it determined by µ alone. Thus the function

Gr(µ) ≡ infηg(ν), ν ∈ Ar(µ)

is well defined. Now using similar arguments as in Lemma 3.3, we can verify that for

any convergent sequence µn with µndP−→ µo, we have

Gr(µo) ≥ lim sup

n→∞Gr(µ

n).

Thus Gr is upper semi continuous in the Prohorov topology and so attains it maximum

on M0. Hence the (sup-inf) problem as stated above has a solution.

13

4 Uncertain Stochastic Differential Equations on

Hilbert Space

In this section we wish to apply the results of the previous section to a class of

stochastic control problems in infinite dimensional spaces. This is a considerable gen-

eralization of the results found in [15-19]. Consider the controlled stochastic system

dx = Axdt+ fo(x, u)dt+B(x)dW, x(0) = x0, t ∈ I ≡ [0, T ], (4.18)

on a separable Hilbert space H, with A being the infinitesimal generator of a C0

semigroup S(t), t ≥ 0, on H , f0 : H × U −→ H and B : H −→ L(H) suitable

maps and W a cylindrical Brownian motion adapted to the filtered probability space

(Ω,F ,Ft ↑, P ). It is assumed that all the parameters A, f0, B, x0 defining the system

are perfectly known. Very often the system parameters, possibly based on noisy

estimates, are not exactly known. In this case the true physical system may be

described by a perturbed version of (4.18) as follows,

dx = Axdt+ fo(x, u)dt+ f(x)dt+B(x)dW, t ∈ I ≡ [0, T ],

x(0) = x0, f(x) ∈ F (x), x ∈ H, (4.19)

where f is any measurable selection of a Borel measurable multifunction F represent-

ing the uncertainty in the nominal drift f0. Note that F may absorb any bounded

perturbation of the unbounded operator A and those associated with the nominal

drift. We identify the system (4.18) as the nominal system and system (4.19) as the

true physical system with parametric uncertainty as described above.

In this section we consider a control problem for the system (3.19) which is described

by parametric uncertainty. Such problems arize in robust control problems on infinite

dimensional spaces.

Admissible Controls: Let U be a closed subset of another Polish space and Uad

14

the class of admissible controls taking values from U. More detailed information on

the structure of the class of admissible controls will be presented later.

Existence of Solutions, Properties of Attainable Measures: Standard assump-

tions imposed on f0, B guaranteeing the existence of mild solutions are as stated

below [3] . There exist constants K,L > 0, such that for all v ∈ U , f0 and B satisfy

the following Lipschitz and growth properties:

‖ f0(x, v) ‖2 + ‖ B(x) ‖2H.S≤ K(1+ ‖ x ‖2), (4.20)

‖ f0(x, v) − f0(y, v) ‖2 + ‖ B(x) − B(y) ‖2H.S≤ L ‖ x− y ‖2, (4.21)

for all x, y ∈ H where ‖ B(x) ‖H.S denotes the Hilbert-Schmidt norm of the operator

B(x). Under the above assumptions, it is well known [3] that for each control u ∈Uad the (nominal) system (4.18) has a unique Ft-adapted mild solution x with the

properties

(p1): supt∈I E ‖ x(t) ‖2H<∞ and (p2): x ∈ C(I,H) P − a.s.

In case controls are based on state feedback, the Lipschitz property imposed on the

drift f0 is rather restrictive and may be lost. Thus it is necessary to generalize this.

This can be done by accepting martingale solution (weak solution).

Lemma 4.1 Consider the system

dx = Axdt+B(x)dW, x(0) = x0, (4.22)

and suppose A generates a C0-semigroup S(t), t ≥ 0, on H, B satisfy the assumptions

(4.20) and (4.21) and that E ‖ x0 ‖2< ∞. Then (4.22) has a unique mild solution

with sample paths x ∈ C(I,H) P -a.s satisfying Esupt∈I ‖ x(t) ‖2H <∞.

Proof See Da Prato- Zabczyk [ 3, Chapter 7].

15

Lemma 4.2 Consider the control system

dx = Axdt+ f0(x, u)dt+B(x)dW, x(0) = x0. (4.23)

Suppose A and B satisfy the assumptions of Lemma 4.1 and further there exists a

finite positive number γ such that

sup‖ B−1(x)f0(x, v) ‖H , (x, v) ∈ H × U ≤ γ (4.24)

and f0 satisfy the growth assumption (4.20), and that E ‖ x0 ‖2< ∞. Then for any

given control u ∈ Uad, the system (4.23) has a martingale solution uniquely determined

by the solution of (4.22) and the control u ∈ Uad.

Proof Under the given assumption (4.24) it follows from Theorem 10.14 (see also

Proposition 10.17) [3] that the process

W (t) ≡W (t) −∫ t

0B−1(x(s))f0(x(s), us) ds (4.25)

evaluated along the mild solution of (4.22) is a cylindrical Brownian motion on an

extended (Skrokhod extension) probability space (Ω, F , Ft ↑, P ). Thus, on this prob-

ability space the mild solution of (4.22) is in fact the martingale solution of (4.23)

driven by the Browniam motion W . By Girsanov theorem [3, Da Prato-Zabczyk,

Theorems 10.14, 10.18, pp.290], the probability measure induced by the controlled

process governed by (4.23) is absolutely continuous with respect to that induced by

the process governed by (4.22). More precisely, let X ≡ C(I,H) denote the space of

continuous functions on I with values in H. Furnished with the standard sup norm

topology this is a Banach space. Let BX denote the Borel algebra of subsets of the

set X making (X,BX) a measurable space. This is the canonical sample space, a

Borel measurable space (X,BX), on which we may define different measures. Let µ0

denote the measure induced by the system (4.22) on BX and µu the one induced by

the system (4.23). Then by virtue of Girsanov theorem as mentioned above, we have

µu ≺ µ0 and the RND is given by

(dµu/dµ0) = exp∫ T

0(B−1f0, dW ) − (1/2)

∫ T

0‖ B−1f0 ‖2

H ds≡ Λ(u). (4.26)

16

This means that the martingale (or weak) solution of (4.23), described in terms of

the probability measure it induces on the path space X, is uniquely determined by

the measure induced by (4.22) and the control u ∈ Uad through the relation

dµu = Λ(u)dµ0. (4.27)

This completes the proof. •

Remark 4.3 The assumption (4.24) is satisfied if f0 satisfies the growth condition

(4.20) and the operator B(x) is surjective satisfying

‖ B(x)z ‖H≥ c(1+ ‖ x ‖q) ‖ z ‖H , q ≥ 1, (4.28)

for a constant c > 0.

Theorem 4.4 Consider the control system (4.23) and suppose the assumptions of

Lemma 4.2 hold, and the set Drnd ≡ Λ(u), u ∈ Uad is a closed bounded subset of

L1(X,µ0) satisfying

limµ0(Γ)→0

∫Γ

Λ(u)dµ0 = 0 uniformly with respect u ∈ Uad.

Then the set of attainable measures M0, induced by the control system (4.23), is

compact in the Prohorov topology.

Proof. The proof is a direct consequence of the celebrated Dunford-Pettis theorem

[4, pp.93]. By virtue of our assumptions, the set Drnd is a closed, bounded, uniformly

integrable subset of L1(X,µ0) and so by Dunford-Pettis theorem it is a weakly se-

quentially compact subset of L1(X,µ0). Hence, the attainable measures described

by

M0 ≡ µ ∈M1(X) : µ(K) =∫Kgdµ0, g ∈ Drnd, K ∈ BX

is a weakly compact subset of M1(X). Since H is a separable Hilbert space, C(I,H)

is a separable metric space with respect to the supnorm topology and hence a Polish

17

space. Thus M1(X) is a Polish space and the weak topology on M1(X) is equivalent

to the Prohorov metric topology. Hence M0 is compact in the Prohorov topology.

This completes our proof. •

Remark 4.5 Sufficient conditions guaranteeing the assumed properties of the set

Drnd are (i): u −→ f0(x, u) is continuous from U to H, (ii): Uad is a compact

metric space, and (iii): sup(x,u)∈H×U ‖ B−1(x)f0(x, u) ‖< ∞. This later condition

follows easily from the properties (4.20) and (4.28).

Perturbed System. The uncertain system described by (4.19) is essentially a

stochastic differential inclusion

dx ∈ Axdt+ f0(x, u)dt+ F (x)dt+B(x)dW, x(0) = x0, t ∈ I. (4.29)

We assume throughout that F : H −→ 2H \ ∅ is a measurable multifunction in the

sense that for every Γ ∈ BH the set x ∈ H : F (x) ∩ Γ = ∅ ∈ BH .

Let c(H) ⊂ 2H \ ∅ denote the class of nonempty closed subsets of H . The following

result relates the uncertain system (4.29) to the nominal system (4.23) in terms of

relative entropy (information distance).

Theorem 4.6 Consider the system (4.29) and suppose A, f0, B satisfy the assump-

tions of Lemma 4.2. Further, suppose that F is a measurable multifunction mapping

H to c(H) and there exists a finite positive number β such that

sup‖ B−1(x)h ‖H , h ∈ F (x)

≤ β, ∀ x ∈ H. (4.30)

Then for each fixed u ∈ Uad, the system has a nonempty set of martingale solutions

denoted by MF ,u ⊂ M1(X) and that each member ν of MF ,u is absolutely continuous

with respect to the corresponding martingale solution µu of the nominal (or perfect)

system (4.23). Further, there exists a finite positive number r such that

H(ν|µu) ≤ r, ∀ ν ∈ MF ,u u ∈ Uad.

18

Proof. Since H is a separable Hilbert space, it is also a Polish space with respect to

its standard topology. Clearly (H,BH) is a measurable space and by our assumption

F : H −→ c(H) is measurable. Thus it follows from standard selection theorem [ see

2, Theorem 2.1, pp.154] that F has a nonempty set of measurable selections which

we may denote by SF . Let f ∈ SF and consider the system

dx = Axdt+ f0(x, u)dt+ f(x)dt+B(x)dW, x(0) = x0, t ∈ I. (4.31)

For any u ∈ Uad, let µu ∈ M1(X) be the measure corresponding to the martingale

solution of (4.23). Then following identical arguments as in the proof of Lemma 4.2,

one can verify that ρ given by

ρ ≡ Exp∫

I(B−1f , dW ) − (1/2)

∫I‖ B−1f ‖2

H dt

(4.32)

is the RND of the measure νu ∈ M1(X), associated with the martingale solution of

(4.31), with respect to the measure µu, associated with the martingale solution of

(4.23). Computing the entropy of νu relative to µu, we find that

H(νu|µu) = (1/2)∫Xϕ(ξ)dνu(ξ) (4.33)

where

ϕ(x) ≡∫I‖ B−1(x(t))f (x(t)) ‖2

H dt , x ∈ X ≡ C(I,H).

By our assumption (4.30), it follows from (4.33) that

H(νu|µu) ≤ (1/2)β2T ≡ r. (4.34)

Since (4.30) holds for all selections f ∈ SF , and every admissible control u, we con-

clude that H(νu|µu) ≤ r for all u ∈ Uad. •

Pay-Off Functional. Let (U, d) be a compact metric space and Bρ(H,U) denote the

class of Borel measurable functions from H to U furnished with the metric topology

ρ(f, g) ≡ supx∈H

d(f(x), g(x)), f, g ∈ Bρ(H,U).

19

Let Uo denote the class of measurable functions on I with values in the metric space

Bρ(H,U). We may introduce a metric on it as follows.

γ(u, v) ≡ λt ∈ I, ρ(ut, vt) = 0,

where λ denotes the Lebesgue measure on I. With respect to this topology, (U0, γ) is a

complete metric space. Let Uad ⊂ U0 denote the class of admissible controls assumed

to be compact with respect to the γ topology. Thus, this class of controls represent

a class of compact Markovian feedback controls.

Now consider the classical control problem (Pc) : Find a control uo ∈ Uad at which

the following inf-sup is attained

Jo ≡ infu∈Uad

supν∈MF ,u

Eν

(∫I (t, x, u)dt+ Ψ(x(T ))

), (4.35)

where Eν(·) denotes integration with respect to the measure ν, a martingale solu-

tion of (4.29) corresponding to a given Markovian control u and a given measurable

selection f ∈ SF . Defining

ψu(x) ≡∫I (t, x(t), u(t, x(t)))dt+ Ψ(x(T )),

our problem can be compactly presented as follows: Find u ∈ Uad that minimizes the

functional Jo(u) given by

Jo(u) ≡ supν∈MF ,u

∫Xψu(x)dν(x). (4.36)

In other words,

Jo ≡ infu∈Uad

Jo(u). (4.37)

Clearly this is a particular case of the general problem studied in Section 3. In view of

the discussions of Section 3, this problem can be formulated as the original problem

studied in Section 2, expression (2.4). For this problem one has to choose η,M0, and

the multifunction Ar as given below:

η(ν) ≡∫Xψu(x)dν(x), ν ∈ MF ,u

M0 ≡ µ ∈ M1(X) : dµ = gdµo, g ∈ Drnd (4.38)

Ar(µ) ≡ ν ∈ M1(X) : H(ν|µ) ≤ (1/2)β2T ≡ r, µ ∈ M0.

20

The min-max problem (Pc) has a solution as stated in the following corollary.

Corollary 4.7 Consider the control problem (Pc) for the uncertain system described

by the Stochastic Differential Inclusion) (SDI) (4.29) and suppose the assmptions

of Theorem 4.4 and 4.6 hold and that and Ψ are bounded away from −∞. Let

η,M0,Ar be as described by (4.38). Then the problem (Pc) has a min-max strategy

uo ∈ Uad.

Proof. The proof is similar to that of Theorem 3.4. The important difference here is

to note that the functional η must be maximized on MF ,u ⊂ Ar(µu) instead of Ar(µ

u).

Since F is closed valued, the set of measurable selections denoted by SF is closed.

That is, the limit of any (uniformly) convergent sequence of measurable selections

is a measurable selection of F . This implies that the set MF ,u is a closed subset of

the compact set Ar(µu), and hence itself compact (in the Prohorov topology). If the

functional Jo given by

Jo(u) ≡ supη(ν), ν ∈ MF ,u

is identically +∞ for every u ∈ Uad, then there is nothing to prove. So we may assume

that there is a nonempty set Us ⊂ Uad on which the supremum is finite. Without loss

of generality we may assume that the whole admissible set has this property. Then

for any u ∈ Uad, η is a bounded linear and hence continuous functional and it attains

its supremum on MF ,u. Thus we may conclude that the functional Jo(u) is a well

defined real valued functional. Hence the functional Fr given by

Fr(µu) = Jo(u)

is well defined. Since M0 is compact and Fr is lower semi continuous, there exists a

µo ∈ M0 at which Fr attains its minimum and hence there exists a control uo ∈ Uadat which Jo(·) attains its minimum. This completes our proof. •

21

5 Construction of the Maximizing Measure

In general a non parametric model may cover a wider class of uncertain stochastic

control systems provided the uncertainty is measured in terms of Prohorov metric

topology. But for a fairly large class of physical systems, relative entropy (though it is

not a metric) provides a good and mathematically convenient measure of uncertainty.

This is the measure we have use throughout the paper. Let X denote a Polish space,

BX the Borel algebra of subsets of the set X, and M1(X) the space of probability

measures defined on BX . This is the state space. Let Uad denote the class of admissible

controls. The perfect system can be described by the single valued map,

Φ : Uad −→ M1(X),

that assigns a unique element µ ∈ M1(X) corresponding to each choice of control

policy u giving Φ(u) = µ. Clearly, the attainable measures, denoted by the set

M0 ≡ Φ(u), u ∈ Uad,

describes the power of the control system in the sense of its capacity to produce

a wider class of probability measures (possibly with different support properties)

guaranteeing broader freedom of choice and control. The model for the uncertain

system may be chosen as follows: for any choice of µ ∈ M0 and any real number

r > 0, define the set valued map as before by

Ar(µ) ≡ ν ∈ M1(X) : H(ν|µ) ≤ r.

Clearly this set is always non empty, since it contains µ itself. Then the uncertain

system is governed by the multi valued map

Φr(u) = Ar(Φ(u)) = Ar(µu),

with range given by

Mr ≡⋃Φr(u), u ∈ Uad =

⋃Ar(µ);µ ∈ M0.

22

The min-max problems studied in Section 3 directly applies to this later class of

uncertain systems.

The Worst Case Measure. This section is concerned with reformulating con-

strained optimization problems as unconstrained ones. For the uncertain system

described in Section 4, with the optimization problem (4.35) or equivalently (4.36),

(4.37), this is not easy. This is because the set MF ,u ⊂ Ar(µu) may not have simple

geometric and topological properties (such as convexity, compactness etc.) as does

the set Ar(µu). However, for nonparametric uncertainty as discussed in the introduc-

tion of this section, this is relatively easy. In other words the optimization problem

(4.35) or its equivalent (4.36), (4.37) is relaxed to the following simpler problem:

Find u ∈ Uad that minimizes the following functional.

Jo(u) ≡ sup∫Xψu(x)dν(x), ν ∈ Ar(µ

u)

≡ inf∫X−ψu(x)dν(x), ν ∈ Ar(µ

u). (5.39)

First we consider the problem (5.39). Using duality theory (Lagrange functionals),

we show that this constrained problem is equivalent to an unconstrained problem.

Throughout this section, for convenience of notation, we write µu for Φ(u) and con-

sider the number r > 0, describing Ar(µu), fixed. For an arbitrary but fixed u ∈ Uad

or equivalently the associated martingale solution µu, define the Lagrangian

L(u; s, ν) ≡ Eν(ψu) − s(H(ν|µu) − r), s ∈ R0 ≡ [0,∞), u ∈ Uad, ν ∈ M1(X), (5.40)

and suppose there exist a ν∗s ∈ M1(X) and s∗ ≥ 0 such that

L(u; s, ν∗s ) = supL(u; s, ν), ν ∈ M1(X), (5.41)

and

ϕ(u, s∗)) ≡ infs≥0

L(u; s, ν∗s ). (5.42)

23

Lemma 5.1 Suppose that for each u ∈ Uad, ψu ∈ L1(X, ν) for all ν ∈ Ar(µu). Then

the problem (5.39) is equivalent to the unconstrained problem

Jo(u) = infs≥0

supν∈M1(X)

L(u, s, ν). (5.43)

Proof The proof follows directly from [5, Theorem 1, pp.224] once we identify the

relevant functions f,G introduced there, as

f(ν) ≡ Eν(−ψu), and G(ν) ≡(H(ν|µu) − r

)

and the relevant function spaces denoted by Ξ,Z as follows. Let M(X) denote the

vector space of bounded countably additive signed measures on the Borel field of

subsets of the Polish space X. Furnished with the total variation norm, this is a

Banach space. Choose Ξ = M(X) and for the ordered vector space Z take Z ≡ R

the real line with the standard topology and natural ordering so that it is a ordered

vector space. For the constraint set we take Ω ≡ Ar(µu). Clearly f, as defined above,

is a convex functional and, since the set Ar(µu) is a convex subset of M1(X), it is also

a convex subset of M(X) and so the constraint set Ω is a convex subset of M(X).

Since

ν −→ H(ν|µu)

is convex, the function G(ν), as defined above is a convex functional. Further note

thatG(µu) = −r ≤ 0. Thus all the assumptions of [5, Theorem 1, pp.224] are satisfied.

Hence the conclusion follows. •

Theorem 5.2 Suppose the assumptions of Lemma 4.1 hold. Then

Jo(u) = ϕ(u, s∗) ≡ infs≥0

L(u, s, ν∗s ).

Proof. Follows directly from the above Lemma. •

24

Next, we present the expression of the worst case measure and some of the properties

of the dual functional.

Theorem 5.3 Let u ∈ Uad and µu ∈ M1(X) the measure induced by the unperturbed

system. Suppose that for all s > 0, ψu/s and exp(ψu/s) ∈ L1(X,µu). Then the

following statements hold:

(S1): The dual functional L(u, s, ν∗s ) given by (5.41) is related to the cumulant gen-

erating function of ψu with respect to µu through the following relation

L(u, s, ν∗s ) = s sup(1/s)

∫ψu(x)dν −H(ν|µu), ν ∈ M1(X)

+ sr (5.44)

= s log(∫

Xexpψu(x)/s)dµu

)+ sr = sCµu(1/s) + sr, (5.45)

where

Cµu(θ) ≡ log∫Xeθψu(x)dµu, θ ∈ R.

(S2): The measure ν∗s that maximizes the expression within the parenthesis of equa-

tion (5.44) is given by

dν∗s =exp(ψu(x)/s)dµ

u

∫X exp(ψu(x)/s)dµ

u, (5.46)

and further we have

∫Xψu(x)dν

∗s (x) = s log

∫Xexpψu(x)/sdµu + sH(ν∗s |µu), s ∈ (0,∞). (5.47)

(S3): There exists a unique s∗ ∈ (0,∞) at which ϕ(u, s) ≡ L(u, s, ν∗s ) attains its

infimum.

(S4): The measure ν∗s∗ that maximizes the functional (5.39) belongs to the boundary

of the constraint set, that is, ν∗s∗ ∈ ∂Ar(µu).

25

Proof. (S1): By virtue of Legendre-Fenchel duality,

supν∈M1(X)

∫X

(ψu/s)dν −H(ν|µu)

= log(∫

Xexp(ψu/s)dµ

u). (5.48)

Hence (5.45) follows from (5.44) and (5.48). The second identity is merely the def-

inition of cumulant generating function. (S2): This follows from direct verification.

Substituting ν∗s given by (5.46) in the expression within the parenthesis of equation

(5.44) we have

∫X

(ψu/s)dν∗s −H(ν∗s |µu) = log

∫X

exp(ψu/s)dµu. (5.49)

Thus it follows from (5.48) of (S1) that ν∗s , given by (5.46), is the maximizer of

L(u, s, ν). Identity (5.47) is an obvious consequence of this result. (S3): By definition

ϕ(u, s) = L(u, s, ν∗s ). Hence using ν∗s given by (5.46) in this expression we have

ϕ(u, s) = L(u, s, ν∗s ) = s log(∫

Xexp(ψu/s)dµ

u)

+ sr. (5.50)

For fixed u, this is a C1 function in the variable s ∈ (0,∞]. By direct computation

we find that

dϕ(u, s)/ds = log∫X

exp(ψu/s)dµu −

∫X(ψu/s) exp(ψu/s)dµ

u

∫X exp(ψu/s)dµu

+ r. (5.51)

By equating this to zero we determine s∗ that minimizes (or maximizes) ϕ(u, s).

Define the function h by

h(θ) ≡∫X(θψu) exp(θψu)dµ

u

∫X exp(θψu)dµu

− log∫X

exp(θψu)dµu, θ ∈ [0,∞). (5.52)

By equating (5.51) to zero, it follows from (5.52) that finding s that minimizes ϕ(u, s)

is equivalent to finding a θ∗ ∈ (0,∞) at which h satisfies the following equation

h(θ) = r. (5.53)

It is clear that h(0) = 0; we show that h is a monotonically increasing function over

the set R0 ≡ [0,∞). By taking the derivative of h with respect to θ followed by some

elementary manipulations we find

h′(θ) = θ

∫X

(ψu)2dλθ −

(∫Xψudλθ

)2(5.54)

26

where λθ ≡ ν∗1/θ. Note that λ0 = µu and that the expression within the parenthesis

of equation (5.54) is strictly positive for all θ ∈ [0,∞). It is zero only when ψu is

a constant ( a trivial case) . Hence h is monotonically increasing and so, for any

finite positive number r, there exists a unique θ∗ such that h(θ∗) = r implying that

equation (4.53) has a unique solution. Hence (S3) follows upon taking s∗ = (1/θ∗).

(S4): Using the measure ν∗s∗ or equivalently ν∗1/θ∗ in the expression for its entropy

relative to the measure µu, we have

H(ν∗1/θ∗|µ) =∫X

log(dν∗1/θ∗dµu

)dν∗1/θ∗ (5.55)

=∫X

(θ∗ψu) exp(θ∗ψu)dµu∫X exp(θ∗ψu)dµu

− log(∫X

exp(θ∗ψu)dµu). (5.56)

Clearly this last expression coincides with the expression (5.52) evaluated at θ∗. Hence

H(ν∗1/θ∗|µu) = h(θ∗) = r (5.57)

proving that ν∗1/θ∗ ∈ ∂Ar(µu). This completes the proof. •

Evolution of the Worst Case Measure. The worst case measure (5.46) is defined

on the path space C(I,H) ≡ X. It will be desirable to introduce another equivalent

problem to (5.39), in which the nominal and uncertain measures are defined on H .

This is done as follows. Substituting ν∗s into the unconstraint Lagrangian, we found

as seen in Theorem 5.3, (S1) that

L(u; s, ν∗s ) = sr + s log ∫

Xe

1sψu(x)dµu

. (5.58)

Next, define the measure valued process πut ; 0 ≤ t ≤ T on H by

πut (φ) = Eµuφ(x(t)) exp

(1

s

∫ t

0 (τ, x(τ), u(τ))dτ

)=

∫Hφ(z)dπut (z), (5.59)

for t ∈ I ≡ [0, T ]. Also, introduce the operator L(u) associated with the nominal

system (4.18). This is given by

(L(u)φ)(t, x) = (A∗Dφ(x), x)H + (Dφ(x), fu(t, x))H + (1/2)Tr(D2φBB∗)(x),

27

for t > 0, x ∈ D(A) ⊂ H. Then one can verify that for any φ ∈ D(L(u)), πut (φ), 0 ≤t ≤ T satisfies the following identity

πut (φ) = π0(φ) +∫ t

0πuτ (L(u(τ))φ)(τ)dτ +

1

s

∫ t

0πuτ ( (u(τ))φ)(τ)dτ, (5.60)

where π0 ∈ M1(H) is the law of the initial state. Since this identity holds for all

φ ∈ D(L(u)), πu is the weak solution of the Forward Kolmogorov equation

dπut = L∗(u)πut dt+B∗uπ

ut dt, π0 = p0, (5.61)

where B∗u is the adjoint of the operator Bu given by Buφ ≡ (1/s) uφ which is a

multiplication operator with (1/s) u acting as a potential.

Using the definition of the measure-valued process πut (·); 0 ≤ t ≤ T, given by (5.59),

we deduce

∫Xe

1sψu(x)dµu =

∫Xe

1sΨ(x(T ))+ 1

s

∫ T

0(t,x(t),u(t))dtdµu = πuT (e

1sΨ). (5.62)

Substituting (5.62) into (5.58) we have

L(u; s, ν∗s ) = sr + s log ∫

Xe

1sψu(x)dµu

= sr + s log

∫He

1sΨ(z)dπuT (z)

. (5.63)

Notice that the resulting Langrangian is expressed in terms of the measure πuT (·) ∈M1(H), instead of the measure µu ∈ M1(X). Now using Legender-Fenchel duality,

the last term of (5.63) can be written as

s log ∫

He

1sΨ(z)dπuT (z)

= supχ∈M1(H)

∫H

Ψ(z)dχ(z) − sH(χ|πuT ). (5.64)

Moreover, as seen before, the measure χu,∗s that maximizes the above expression is

given by

dχu,∗s (z) =exp1

sΨ(z)dπuT (z)∫

H exp1sΨ(z)dπuT (z).

(5.65)

28

Thus, the original problem is transformed to an equivalent one on H rather than X,

and we have the following identity.

L(u; s, ν∗s ) = supν∈M1(X)

∫Xψu(x)dν − s

(H(ν|µu) − r

)

= sr + s log ∫

Xe

1sψu(x)dµu

= sr + s log ∫

He

1sΨ(z)dπuT (z)

= supχ∈M1(H)

∫H

Ψ(z)dχ(z) − s(H(χ|πuT ) − r

). (5.66)

It is important to note that the radius of uncertainty r used here may be different

from the one used in the original path space X.

Hence, we have proved the following theorem.

Theorem 5.4 Define the sphere of uncertainty by

Br(πuT ) =χ ∈ M1(H) : H(χ|πuT ) ≤ r

.

Then Jo(u) defined by (5.39) is equivalent to the transformed optimization problem

Jotran(u) = sup ∫

HΨ(z)dχ(z), χ ∈ Br(πuT )

(5.67)

where the nominal measure is πuT ∈ M1(H) obtained from the weak solution of equa-

tion (5.61). Moreover, the worst case measure, that is, the measure at which (5.67)

is realized, is given by (5.65).

Furthermore, the statements of Theorem 5.3 (S1), (S2) hold with the obvious changes

ψu → Ψ, µu → πu, νu → χu, X → H. In addition, when = 0 statements of Theo-

rem 5.3 (S3), (S4) also hold, with πu given by (5.59) with = 0.

Proof. The equivalence of the two problems follows from the fact that L(u; s, ν∗s )

given by the first equality in (5.66) is the Lagrangian associated with Jo(u) defined

by (5.39), while the last equality in (5.66) is the Lagrangian associated with (5.67).

The worst case measure is given by (5.65). The last statement holds following the

derivation of Theorem 5.3.

29

Remark 5.5 For complete solution of the problem we convert (5.67) into an uncon-

straint problem using the last two identities in (5.66) to obtain the cost functional

Jotrans(u) and then minimize this functional subject to measure dynamics (5.61). This

can be formulated as a terminal control problem as follows. Note that

Jotrans(u) = infs>0

sr + s log

∫He

1sΨ(z)dπuT (z)

≡ inf

s>0Φ(πuT ).

Thus, the problem is

Φ(πuT ) −→ infu∈Uad

infs>0

subject to the dynamic constraint

dπut = L∗(u)πut dt+B∗uπ

ut dt, π0 = p0.

Moreover, when = 0, then the problem is

Φ(πuT ) −→ infu∈Uad

infs>0

subject to the dynamic constraint

dπut = L∗(u)πut dt, π0 = p0,

and since the solution πut (·); 0 ≤ t ≤ T is independent of s, then s∗ corresponds to

the boundary constraint H(χu,∗|πuT )|s=s∗ = r, while the infimum over u ∈ Uad is found

independently.

We leave this problem for future research.

6 Conclusion

This paper derives several results for minimax games of stochastic uncertain sys-

tems, when the pay-off is a nonlinear functional of the uncertain measure, and the

uncertainty is measured in terms of relative entropy between the uncertain and the

nominal measure. Existence and uniqueness of minimax solutions are derived on

suitable spaces of measures. Subsequently, the results are also applied to controlled

Stochastic Differential Equations (SDE’s) on Hilbert Spaces. Based on infinite dimen-

sional extension of Girsanov’s measure transformation, martingale solutions are used

30

in establishing existence and uniqueness of minimax strategies. The worst case mea-

sure representing the maximizing player is found and it is shown that it is described

by an evolution equation on the space of measures on the original state space.

7 References

[1]. P.Dupuis and R.S.Ellis, (1997), A Weak Convergence Approach to the Theory of

Large Deviations, John Wiley.

[2]. S.Hu and N.S.Papageorgiou,(1997), Handbook of Multivalued Analysis, Vol 1:

Theory, Kluwer Academic Publishers, DOrdrecht/Boston/London.

[3]. G.Da Prato and J. Zabczyk, Stochastic Equations in Infinite Dimensions,(1992)

Encyclopedia of Mathematics and its Applications, Cambridge University Press, (1992).

[4].J.Diestel,(1984), Sequences and Series in Banach Spaces, Springer-Verlag New

York,Inc

[5]. D. G. Luenberger, (1969), Optimization by Vector Space Methods, John Wiley

[6] G. Zames, “Feedback and optimal sensitivity: Model reference transformations,

multiplicative seminorms, and approximate inverses,” IEEE Transactions on Auto-

matic Control, vol. 26, no. 2, pp. 301–320, 1981.

[7] J. Willems, “Dissipative dynamical systems part I: General theory,” Arch. Rational

Mech. Anal., vol. 45, pp. 321–351, 1972.

[8] D. Jacobson, “Optimal stochastic linear systems with exponential performance

criteria and their relation to deterministic differential games,” IEEE Transactions on

Automatic Control, vol. 18, no. 2, pp. 124–131, 1973.

31

[9] A. Bensoussan and J. H. van Schuppen, “Optimal control of partially observable

stochastic systems with an exponential-of-integral performance index,” SIAM Journal

on Control and Optimization, vol. 23, no. 4, pp. 599–613, 1985.

[10] P. Whittle, “A risk-sensitive maximum principle,” Systems and Control Letters,

vol. 15, pp. 183–192, 1990.

[11] W. H. Fleming and W. M. McEneaney, “Risk-sensitive control and differential

games, pp.185-197,” in Stochastic Theory and Adaptive Control, eds., T.E. Duncan

and B. Pasik-Duncan, (New York), Springer-Verlag 1992.

[12] M. James, J. Baras, and R. Elliott, “Risk-sensitive control and dynamic games for

partially observed discrete-time nonlinear systems,” IEEE Transaction on Automatic

Control, vol. 39, no. 4, pp. 780–792, 1994.

[13] C. Charalambous and J. Hibey, “Minimum principle for partially observable non-

linear risk-sensitive control problems using measure-valued decompositions,” Stochas-

tics and Stochastics Reports, vol. 57, 1996.

[14] C. Charalambous, “The role of informations state and adjoint in relating nonlin-

ear output feedback risk-sensitive control and dynamic games,” IEEE Transactions

on Automatic Control, vol. 42, no. 8, pp. 1163–1170, 1997.

[15] P. D. Pra, L. Meneghini, and W. Runggaldier, “Some connections between

stochastic control and dynamic games.”Mathematics of Control, Signals, and Sys-

tems , vol. 9, pp. 303–326, 1996.

[16] V. A. Ugrinovskii and I. R. Petersen, “Finite Horizon Minimax Optimal Control

of Stochastic Partially Observed Time Varying Systems,” Mathematics of Control,

Signals and Systems, vol. 12, pp. 1–23, 1999.

32

[17] I. R. Petersen and M. R. James and P. Dupuis, “Minimax Optimal Control

of Stochastic Uncertain Systems with Relative Constraints,” IEEE Transactions on

Automatic Control, vol. 45, no. 3, pp. 398–412, 2000.

[18] C.D. Charalambous and F. Rezaei and S. Djouadi, ”Optimization of Stochastic

Uncertain Systems: Large Deviations and Robustness,” in Proceedings of the 42nd

IEEE Conference on Decision and Control, Hawaii, U.S.A., December 9-12, Vol.4,

pages 4249-4253, 2003.

[19] C.D. Charalambous and F. Rezaei, ”Characterization of the Optimal Disturbance

Attenuation for Nonlinear Stochastic Uncertain Systems,” in Proceedings of the 42nd

IEEE Conference on Decision and Control, Hawaii, U.S.A., December 9-12, Vol.4,

pages 4260-4264, 2003.

[20] C.D. Charalambous, F. Rezaei and A. Kyprianou, ”Relations Between Infor-

mation Theory, Robustness and Statistical Mechanics of Stochastic Systems,” in

Proceedings of the 43nd IEEE Conference on Decision and Control, pp.3479-3484,

Bahamas, December 14-17, 2004.

[21] N.U.Ahmed, Existence of Solutions of Nonlinear Stochastic Differential Inclu-

sions, Proc. world Congress of Nonlinear Analysis’ 92, (Ed. V Lakshmikantham),

(1992),1699-1712.

[22] N.U.Ahmed, Optimal Relaxed Controls for Nonlinear Infinite Dimensional Stochas-

tic Differential Inclusions, in Optimal Control of Differential Equations, Lec. Notes

in Pure and applied Mathematics, Marcel Dekker Inc, Vol.160,(1994), 1-19.

[23] N.U.Ahmed, Optimal Control of Impulsive Stochastic Evolution Inclusions, Dis-

cussiones Mathematicae, Differential Inclusions, Control and Optimization, 22(2002),155-

184.

33

[24] N.U.Ahmed, Optimal Relaxed Controls for Infinite Dimensional Stochastic Sys-

tems of Zakai Type, SIAM J. Control and Optim. 34 (5), (1996), 1592-1615.

[25] N.U.Ahmed and X.Xiang, Nonlinear Uncertain Systems and Necessary Condi-

tions of Optimality, SIAM J. Control and Optim. 35(5), (1997), 1755-1772.

34