SOP TRANSACTIONS ON APPLIED MATHEMATICS Applications of Dynamic Programming to Generalized Solutions...

SOP TRANSACTIONS ON APPLIED MATHEMATICSISSN(Print): 2373-8472 ISSN(Online): 2373-8480

Volume 1, Number 2, July 2014

SOP TRANSACTIONS ON APPLIED MATHEMATICS

Applications of Dynamic Programming toGeneralized Solutions for Hamilton –Jacobi Equations with State ConstraintsLiubov Shagalova*Department of dynamical systems, Krasovskii Institute of Mathematics and Mechanics, Ural Branch of the Russian Academy ofSciences, Ekaterinbutg, Russia*Corresponding author: [email protected]

Abstract:Cauchy problem is considered for a Hamilton - Jacobi equation with state constrains arising inmolecular biology. This problem has no classical solution, and known concepts of generalized(minimax and/or viscosity) solution are not applicable to it. We introduce a new concept of acontinuous generalized solution and propose a method for constructing this solution by solvingan auxiliary optimal control problem.

Keywords:dynamic programming, generalized solution, Hamilton – Jacobi equation, method of characteris-tics, minimax solutions, optimal control, state constrains, value function, viscosity solutions

1. INTRODUCTION

A Cauchy problem for a Hamilton - Jacobi equation that arises in molecular biology in the Crow -Kimura model of molecular evolution is considered in the article. The Hamiltonian of the equation isnonlinear and depends on the state and momentum variables. In addition to the initial condition the stateconstraints are given also – the equation is considered in a strip equal to the Cartesian product of the timehalf-line and an interval in the state space on which a boundary function is defined at the initial moment.Given boundary function is also nonlinear.

The problem under consideration has no classical solution. Furthermore, methods developed forproblems without state constraints are not applicable to this problem.

Solution of the problem should be understood in the generalized sense. However, known concepts ofthe generalized solutions are inapplicable to this problem. In particular, the well-known theorems onthe existence of viscosity solutions [1–3] require the coercivity of the Hamiltonian, and this conditionis broken in our problem. And minimax solutions [4] were not considered for problems with stateconstraints.

In paper [5], using the tools of nonsmooth analysis, we introduced the concept of a continuousgeneralized solution to such an initial value problem for the Hamilton - Jacobi equation in rectangular

70

Applications of Dynamic Programming to Generalized Solutions for Hamilton – Jacobi Equations with StateConstraints

section of the strip on time interval [0;T ]. The moment T > 0 is chosen so that all characteristics of theoriginal Hamilton – Jacobi equation starting from the initial manifold were extendable to it.

The generalized continuous solution is introduced on the basis of minimax (and/or viscosity) approachusing inequalities for sub- and superdifferentials.

The Hamiltonian of the problem is concave in the momentum variable (i.e., the equation preserves theBellman type) in the strip defined by the state constraints. This allows us to set an auxiliary optimal controlproblem with target set consisting of the given initial manifold and the boundary of state constraints. Theinitial function is extended smoothly on the boundary of the state constraints.

The auxiliary optimal control problem has a value function. The properties of this value functionare investigated. It is shown that the continuous generalized solution of the original Cauchy problemfor Hamilton – Jacobi equation with state constraints in the considered closed bounded domain can beconstructed using the value function. Therefore we can say that the method of dynamic programming [6]is applied to construct a generalized solution.

It has been shown [5] that the generalized solution is not unique, because it depends on the specificsmooth extension of the initial condition to the boundary of the state constraints. In this paper, we considersuch an extension of the initial conditions, when the momentum variable has constant values on the upperand lower boundary of the strip defined by state constraints. Thus the initial Cauchy problem reduces tothe Dirichlet problem for which the continuous minimax (and/or viscosity) solution exists and is unique.

Our analysis is based on the results obtained in the mathematical theory of optimal control [7, 8] and inthe theory of generalized (minimax [4] and viscosity [1, 2]) solutions to the Hamilton - Jacobi equationsand employs the generalized method of characteristics [9].

2. HAMILTON–JACOBI EQUATION IN CROW–KIMURA MODEL

In article [10] for the Crow – Kimura model of molecular evolution the following Hamilton – Jacobiequation was obtained.

∂u/∂ t +H(x,∂u/∂x) = 0, (1)

whereH(x, p) =� f (x)+1� 1+ x

2e2p � 1� x

2e�2p. (2)

Function f (·) in (2) is called fitness. Equation (1) is considered for t � 0,�1 x 1. It is also assumedthat function u0 : R ! R is given such that the following initial condition should be satisfied.

u(0,x) = u0(x), x 2 [�1;1]. (3)

In [10] behavior of the maximum of the solution u(·) for the problem (1)-(3) have been investigatedin the case of quadratic fitness f (x) = x2 for the initial function of the form u0(x) =�a(x� x0)2, a > 0.This analysis was based on physical interpretations.

71


2.1 Investigation of Characteristic System

One of the main methods for investigation of the Cauchy problem for first order PDEs and findingclassical solutions of such problems is the method of characteristics (see, e.g. [11]). This method reducesthe integration of PDEs to the integration of the characteristic system of ODEs.

The characteristic system for equation (1) has the form

x = Hp(x, p) =�(1+ x)e2p +(1� x)e�2p,

p =�Hx(x, p) = f 0(x)+(e2p � e�2p)/2, (4)

u = pHp(x, p)�H(x, p) = px+q.

Here Hx(x, p) = ∂H(x, p)/∂x, Hp(x, p) = ∂H(x, p)/∂ p, f 0(x) = ∂ f (x)/∂x.The system (4) is considered with initial conditions agreed with (3)

x(0,y) = y, p(0,y) = u00(y),z(0,y) = u0(y), y 2 [�1;1] (5)

Further we assume that the initial function u0(·) is continuously differentiable and fitness f (·) is a twicecontinuously differentiable function. Under these assumptions, there exist constants M1 and M2 such that

M1 f (x) M2, x 2 [�1;1]. (6)

Let y 2 [�1;1] is given. Denote p(0) = p(0,y). Let for i = 1,2

Ki =12

ln✓

�Mi +q

M2i +1

◆

, Ci =|e2p(0) +Mi �

q

M2i +1|

e2p(0) +Mi +q

M2i +1

. (7)

One can prove that the inequality K1 � K2 is valid.

Let T ⇤ = T ⇤(y) be such a moment that on the interval [0,T ⇤) the solution of the Cauchy problem (4)-(5)exists and is noncontinuable to the right. The next estimates depending on p(0) are valid for T ⇤.

Theorem 2.1.a) If p(0)> K1 then T ⇤ < �1

2p

M21+1

lnC1, 0 <C1 < 1, and p(t)!+• as t ! T ⇤.

b) If p(0)< K2 then T ⇤ < 12p

M22+1

ln

�M2+p

M22+1

C2

⇣

M2+p

M22+1

⌘

!

, C2 > 0, and p(t)!�• as t ! T ⇤.

c) If K2 p(0) K1 then T ⇤ � min

(

12p

M21+1

ln

�M1+p

M21+1

C1

⇣

M1+p

M21+1

⌘

!

, �12p

M22+1

lnC2

�

, C1 > 0, 0 <C2 < 1

Proof of Theorem 2.1 is based on the analysis of the following two equations

p = M1 +(e2p � e�2p)/2, (8)

p = M2 +(e2p � e�2p)/2. (9)72


The solutions of separable differential equations (8) and (9) are respectively minorant and majorant forthe component p(·) of the characteristic system (4). The proof of theorem 1 is technical in its nature andwe omit it.

DefineT⇤ = min

y2[�1,1]T ⇤(y).

Suppose 0 < T < T⇤ is given. Below we consider problem (1)-(3) in the closed bounded region

PT = [0;T ]⇥ [�1;1].

Also we use the notationPT = (0;T )⇥ (�1;1).

The solutions(x(·,y), p(·,y),z(·,y)) : [0,T ]! R⇥R (10)

to system (4), which depend on the parameter y, are called the characteristics of equation (1).

If the conditionsx(t,y) = x, ∂x(t,y)/∂y 6= 0, y 2 [�1;1]

hold at a point (t,x) 2 PT , then the graphs of the state components x(·,y) of the characteristics (10) donot intersect at (t,x) 2 PT and the set of parameters

Y (t,x) = {y 2 [�1,1] |x(t,y) = x}= {y(t,x)}

is a one-element set. Then, to construct a classical solution to the Hamilton – Jacobi equation (1) at thepoint (t,x) 2 PT , we apply Cauchy’s method of characteristics according to which the classical solutioncan be represented as

u(t,x) = z(t,y(t,x)), (t,x) 2 PT .

Since the Hamiltonian H(x, p) (2) is nonlinear in this problem, the characteristics (10) are disjoint onlyin a sufficiently small neighborhood of the initial conditions t = 0, u(0,x) = u0(x), x 2 [�1;1], howeversmooth the fitness function f (·) and the initial function u0(·) may be. Outside this neighborhood, there isno classical solution. Moreover, as it follows from the lemma 1 below, there exist points in PT throughwhich no state projection of the characteristics (4)-(5) passes. Let

Oe(t⇤,x⇤) = {(x,y) 2 PT | |x� x⇤|+ |t � t⇤| e}.

Lemma 2.1.(Please see the reference [5])

Let 0 < t⇤ < T , x⇤ 2 {�1;1}. Then there exists e > 0 such that, for any y 2 [�1;1] the graph of the componentx(·,y) of the characteristic has no common points with the neighborhood Oe (t⇤,x⇤), i.e. {(t,x(t,y)) | 0 < t < T}\Oe (t⇤,x⇤) =?.

Figure 1 and Figure 2 represent the behavior of projections of characteristics onto (t,x) - plane.

Thus, using the method of characteristics, we can construct a solution to problem (1)-(3) only locally.This problem has no global classical solution; therefore, we should treat a solution in a generalized sense.

73


Figure 1. Projections of characteristics onto the (t,x) - plane. Initial function u0(x) = �1/4(x + 0.4)2, fitnessf (x) = 1/8(x+0.4)2.

Figure 2. Projections of characteristics onto the (t,x) - plane. Initial function u0(x) = �1/8x3 + 1/6x2, fitnessf (x) = 1/8(x�0.3)2.

3. CONTINUOUS GENERALIZED SOLUTION OF THE PROBLEM WITHSTATE CONSTRAINTS

In the theory of Hamilton - Jacobi equations various concepts of generalized solutions have beenconsidered (see, e.g. [12]). M.G. Crandall and P.-L. Lions have introduced viscosity solutions [1]. Theequivalent concept of the minimax solution was independently developed by A.I. Subbotin [4].

In [5] a new definition (see Definition 1 below) of the continuous generalized solution of the problem(1)-(3) on compact set PT = [0,T ]⇥ [�1,1] has been introduced. This definition is based on the minimaxand viscosity approaches and uses the following concepts of nonsmooth analysis [3, 13].

Let W be a set in R2. Denoted by W the closure of this set. Symbol C(W ) will denote the class offunctions continuous on the set W .

Let u(·) 2C(W ) and (t,x) 2W . The subdifferential of the function u(·) at (t,x) is the set

D�u(t,x) = {(a,s) 2 R2 | liminf(t,y)!(t,x)

(t,y)2W

h

�

u(t,y)�u(t,x)�a(t � t)� s(y� x)�

⇥

⇥ (|t � t|+ |y� x|)�1i

� 0}. (11)

Remark 3.1.Note that at the boundary points of the set W , the subdifferential D�u(t,x), if nonempty, is an unbounded set.Indeed, let a point (t⇤,x⇤) belong to the boundary of the set W , (a,s) 2 D�u(t⇤,x⇤), and a vector (n1,n2) be anoutward normal to the set W at the point (t⇤,x⇤). Then the definition (11) of the subdifferential immediately implies

74


the inclusion(a+ kn1,s+ kn2) 2 D�u(t⇤,x⇤) (12)

for any positive number k.

The superdifferential of the function u(·) at (t,x) is the set

D+u(t,x) = {(a,s) 2 R2 | limsup(t,y)!(t,x)(t,y)2W

h

�

u(t,y)�u(t,x)�a(t � t)� s(y� x)�

⇥

(|t � t|+ |y� x|)�1i

0}. (13)

Denote by Dif(u) the set of points at which the function u(·) 2C(W ) is differentiable. For a given setM, its convex hull [14] is denoted by coM. Define the set

∂u(t,x) = con

(a,s)�

�a = limi!•

∂u(ti,xi)

∂ t,s = lim

i!•

∂u(ti,xi)

∂x;(ti,xi)! (t,x)

as i ! •, (ti,xi) 2W \Dif(u)o

. (14)

Note that when the function u(·) is locally Lipschitz, the set (14) coincides with the Clarke subdifferen-tial [13].

Definition 3.1.A generalized solution to problem (1)–(3) in the domain PT is a continuous function u(·) satisfying the initialcondition (3) and the conditions

a+H(x,s) 08(a,s) 2 D+u(t,x),8(t,x) 2 PT , (15)

a+H(x,s)� 0, 8(a,s) 2 D�u(t,x),8(t,x) 2 PT , (16)

a+H(x,s)� 0, 8(a,s) 2 D�u(t,x)\∂u(t,x),8(t,x) 2 GT , (17)

where GT = {(t,x)|0 < t < T,x = 1}[{(t,x)|0 < t < T, x =�1}.

Remark 3.2.In Definition 1 the generalized solution is required to satisfy inequality (17) for the subdifferential at the points ofthe boundary GT . This is associated with the concavity of the Hamiltonian (2) in the variable p in the domain PT .

Note that in contrast to Definition 1, the notion of generalized viscosity solution for equation (1) on the set W , asintroduced in [3], requires that this solution to problem (1)–(3) in the domain W = PT should satisfy inequality (17)at the boundary points (t,x) 2 GT for all (a,s) 2 D�u(t,x).

In [3], the authors obtained conditions ensuring existence and uniqueness theorems for a generalized viscositysolution. One of the main conditions under which the existence of a generalized viscosity solution has been provedis the coercivity of the Hamiltonian:

H(x, p)!+• as |p|! • . (18)

The Hamiltonian defined by relation (2) does not satisfy condition (18), for example, at x = 1 and x = �1;therefore, the known existence theorems for a generalized viscosity solution in the sense of [3] cannot be applied toproblem (1)-(3).

75


Moreover, the notion of generalized viscosity solution is inapplicable to the initial value problem (1)–(3) definedon a compact setPT if D�u(t⇤,x⇤) 6= ? at some point (t⇤,x⇤) 2 GT , because inequality (17) does not hold in thiscase. Indeed, let 0 t⇤ T and, for definiteness, x⇤ = 1. Then the unit vector (0,1) defines the direction of anoutward normal to the set PT at (t⇤,x⇤). Let (a,s) 2 D�u(t⇤,x⇤). Then, according to Remark 1, inclusion (12) isvalid:

(a,s+ k) 2 D�u(t⇤,x⇤) 8k � 0,

and for condition (17) to hold it is necessary that

a+H(1,s+ k) = a� f (x)+1� e2(s+k) � 0 8k � 0,

which is obviously not the case. Therefore, we introduce the intersection of subdifferentials (see condition (17)) intoDefinition 1 of a continuous generalized solution to the initial value problem (1)–(3) on the compact set PT .

4. AUXILIARY OPTIMAL CONTROL PROBLEM

Because the moment T was chosen by such a way that characteristics (4) are continuable on the interval[0,T ], it follows that components p(·,y) are bounded on this interval. Let P ⇢ R be such a bounded closedinterval that

p(t,y) 2 P for all t 2 [0,T ], y 2 [�1,1]. (19)

Let y(·) be some continuously differentiable function in R2 such that y(t,x) =

8

>

<

>

:

u0(x), t = 0, x 2 [�1;1];u0(�1)�H (�1,u0(�1)) t, t � 0, x =�1;u0(1)�H (1,u0(1)) t, t � 0, x = 1,

(20)

∂y(t,x)∂x

=

8

>

<

>

:

u00(x), t = 0, x 2 (�1;1);u00(�1), t � 0, x =�1;u00(1), t � 0, x = 1,

(21)

∂y(t,x)∂ t

=

8

>

<

>

:

�H (x,u00(x)) , t = 0, x 2 (�1;1);�H (t,u00(�1)) , t � 0, x =�1;�H (t,u00(1)) , t � 0, x = 1.

(22)

Consider an optimal control problem whose solution will be used for constructing a generalized solutionto problem (1)-(3).

4.1 Statement of an Auxiliary Optimal Control Problem

In the domain PT , using the reverse time t = T � t we consider the problem of optimal control (OCP)of a system whose dynamics are described by the differential equation

dydt

=�Hp(y, p) = (1+ y)e2p � (1� y)e�2p, p 2 P. (23)76


The values of the control parameter p are chosen from the set P, which is defined in accordance with(19). If the initial state y(t0) = y0, (t0,y0) 2 PT of the system is specified, then admissible controls aregiven by measurable functions p(·) : [t0,T ]! P. Denote the set of all admissible controls by P[t0,T ].

Letj(t,y) = y(T � t,y), (t,y) 2 [0,T ]⇥R (24)

On the motions of system (23) define the cost functional

I(t0,y0)(p(·)) =Z t]

t0

⇥

p(t)Hp(y(t), p(t)) � H(y(t), p(t))⇤

dt + j(t],y(t])), (25)

where y(·) = y(·;t0,y0, p(·)) : [t0,T ]! [�1,1] is a trajectory of system (23) emanating from the initialpoint (t0,y0) and corresponding to the control p(·) 2 P[t0,T ], and t] = t](t0,y0, p(·)) is the instant atwhich this trajectory reaches the target set GT for the first time:

GT = {(t,y)|0 t T,y = 1}[{(t,y)|0 t T, y =�1}[{(t,y)|t = T,�1 y 1},t] = min

t2[t0,T ]{t |y(t,t0,y0, p(·)) 2 GT}.

The OCP consists in controlling system (23) in a way that provides an optimal result (value) V (t0,y0),which is defined by the relation

V (t0,y0) = supp(·)2P[t0 ,T ]

I(t0,y0)(p(·)), (26)

for the initial point (t0,y0) 2 PT .

4.2 Hamiltonian

The Hamiltonian H of the OCP has the form

H(y,s) = maxp2P

[�sHp(y, p)+ pHp(y, p)�H(y, p)]. (27)

Let us show that the relationH(y,s) =�H(y,s) (28)

holds for the Hamiltonian H(·) of the OCP and the Hamiltonian H(·) (2) of equation (1) in the domain(�1;1)⇥P 3 (y,s).

Transforming the expression to be maximized in (27), we obtain

(�sHp(y, p)+ pHp(y, p)+ (H(y,s)�H(y, p)))�H(y,s).

It follows from (2) that the Hamiltonian H(y,s) is concave in s for y 2 (�1,1); therefore, we have

H(y,s)�H(y, p) (s� p)Hp(y, p).

This implies that�sHp(y, p)+ pHp(y, p)�H(y, p)+H(y,s) 0.

Therefore, if s 2 P, then the maximum in (27) is attained on the element p = s, and equality (28) is proved.77


4.3 Optimality Conditions in the OCP

Denote by P0[t0,T ]

the set of all absolutely continuous functions p(·) : [t0,T ] ! P. It is clear thatP0[t0,T ]

⇢ P[t0,T ]. Define the set

∂yj(t,y) = con

s�

�

�

�

s = limi!•

∂u(ti,yi)

∂y; (ti,yi) ! (t,y) as i ! •, (ti,yi) 2 PT \ Dif(j)

o

.

Applying the Pontryagin maximum principle [7, 8] to the OCP and taking into account the compactnessof the set P, relation (28), the continuous differentiability of the function Hp(·), and the properties of thefunction j(·), we can prove the following statement.

Theorem 4.1.For any initial point (t0,y0) 2 PT , the optimal result V (t0,y0) (26) in the OCP is attained on the set P0

[t0,T ], i.e.

V (t0,y0) = maxp(·)2P0

[t0 ,T ]

I(t0,x0)(p(·)). (29)

An absolutely continuous function p0(·) that maximizes (29), satisfies the equation

p0(t) = Hx(y0(t), p0(t)) (30)

for almost every t 2�

t0,t](t0,y0, p0(·))�

and the transversality condition

p(t]) = ∂j(t],y0(t]))/∂y, (31)

where y0(·) is the corresponding trajectory of the control system (23).

5. VALUE FUNCTION OF OCP

Statements provided in this section, describe structure and properties of the value function of OCP.

5.1 Representative formula for Value Function

In [5] the following statement have been proved.

Theorem 5.1.The value function in the OCP can be represented as

V (t,y) = maxa2A

w(t,y,a) for all(t,y) 2 PT , (32)

where the parameter a takes values in a metric compact set A.

The function w(·) : PT ⇥A ! R is continuous, and the functions

PT ! R : (t,y)! w(t,y,a), a 2 A

have partial derivatives for almost all (t,y)2 PT , and these derivatives are uniformly bounded by a constant L > 0and continuous at the points (t,y) 2 PT for a 2 A0(t,y) = {a 2 A| w(t,y,a) =V (t,y)}.

78


5.2 Properties of the Value Function

It follows from theorem 5.1 that function V (·) is Lipschitz continuous. According to the next theorem,the value function V (·) is subdifferentiable at the interior points of the domain PT .

Theorem 5.2.The relation

? 6= D�V (t,y) = co⇢✓

∂w(t,y,a)

∂t,

∂w(t,y,a)

∂y

◆

�

�

�

a 2 A0(t,y)�

(33)

holds for all points (t,y) 2 PT = (0,T )⇥ (�1,1).

Proof. Let (t,x) 2 (0,T )⇥ (�1,1). Let us prove that D�V (t,x) 6= /0 and the following inclusion is valid.

co⇢✓

∂w(t,x,a)

∂ t,

∂w(t,x,a)

∂x

◆

�

�a 2 A0(t,x)�

⇢ D�V (t,x). (34)

Let a 2 A0(t,x), (t,y) 2 Oe(t,x), where e is small enough. Then, using theorem 4.1, obtain

V (t,y)�V (t,x)� w(t,y,a)�w(t,x,a) =∂w(t,x,a)

∂ t(t � t)+

∂w(t,x,a)

∂x(y� x)

+o(|t � t)+ |y� x|),

It follows from the above inequality and from the definition of the subdefferential that

✓

∂w(t,x,a)

∂ t,

∂w(t,x,a)

∂x

◆

2 D�V (t,x) 6= /0. (35)

The set D�V (t,x) is convex, so from (35) we have (34).

Now prove the inclusion

D�V (t,x)⇢ co⇢✓

∂w(t,x,a)

∂ t,

∂w(t,x,a)

∂x

◆

|a 2 A0(t,x)�

. (36)

From theorem 3 we have that function V (·) is Lipschitz continuous. Since D�V (t,x) 6= /0, the subdiffer-ential D�V (t,x) coincides with the Clarke subdifferential ∂CV (t,x). In this case

∂CV (t,x) = co{(a,s)�

�

�

�

a = limi!•

∂V (ti,xi)

∂ t, s = lim

i!•

∂V (ti,xi)

∂x;(ti,xi)! (t,x) i ! •, (ti,xi) /2 W}.

(37)

Here W is the set of points where the function V (·) is not differentiable. According to Rademacher’stheorem W is a null-measure set.

It follows from theorem 3 that at the points of differentiability of the function V (·) we have

∂V (ti,xi)

∂ t=

∂w(ti,xi,ai)

∂ t,

∂V (ti,xi)

∂x=

w(ti,xi,ai)

∂x, ai 2 A0(ti,xi). (38)

79


The map (t,x) ! A0(t,x) is compact-valued and upper semicontinuous. So, if ai 2 A0(ti,xi) and(ti,xi)! (t,x), then ai ! a⇤ 2 A0(t,x), and from (37), (38) it follows

∂CV (t,x) = co⇢✓

∂w(t,x,a)

∂ t,

∂w(t,x,a)

∂x

◆

|a 2 A0(t,x)�

. (39)

From the equality D�V (t,x) = ∂CV (t,x) and (39) we have (36).

Lemma 5.1.Let (t,y) 2 PT and (a,s) 2 D�V (t,y). Then

a�H(y,s) 0. (40)

Proof. It was noted in the proof of the theorem 4 that D�V (t,x) = ∂CV (t,x) and (37) holds.

It is known that at the points where it is differentiable the function V (·) of optimal result satisfies theBellman equation [6]

∂V (t,x)/∂ t �H(x,∂V (t,x)/∂x) = 0. (41)

Let

D⇤V (t,x) = {(a,s)�

�

�

�

a = limi!•

ai, s = limi!•

si;

si =∂V (ti,xi)

∂ t, ai = H(xi,si),(ti,xi)! (t,x) i ! •, (ti,xi) /2 W}. (42)

Here W is the set of points where the function V (·) is not differentiable.

Taking into account (41), we obtain from (37) the representation

D�V (t,x) = coD⇤V (t,x). (43)

By virtue of Caratheodory’s theorem (see, e.g. [15]) any element (a,s) of the closed convex setD�V (t,x) represented as a linear combination

a =3

Âi=1

liai, s =3

Âi=1

lisi,3

Âi=1

li = 1, li � 0, (44)

where (ai,si) 2 D⇤V (t,x).

Thus, by convexity of the function �H(x, ·) we have from (44) and (42)

a�H(x,s) =3

Âi=1

liai �H(x,3

Âi=1

lisi)3

Âi=1

li[ai �H(x,si)] = 0. (45)

80


Figure 3. Characteristics forming the generalized solution for the case u0(x) =�0.05cos(2px), f (x) =�0.01x4.

Figure 4. Characteristics forming the generalized solution for the case u0(x) = 0.2(x�0.3)2, f (x) = 18 (x�0.4)2.

The following statement describes the behavior of the mapping (t,y)! D�V (t,y) near the set GT .

Lemma 5.2.Let (t⇤,y⇤) 2 GT and a sequence of points (ti,yi) 2 PT converge to (t⇤,y⇤) as i ! •. Let (ai,si) 2 D�V (ti,yi),

i = 1,•. Then

(ai,si)!✓

H✓

y⇤,∂j(t⇤,y⇤)

∂y

◆

,∂j(t⇤,y⇤)

∂y

◆

as i ! •.

Proof of Lemma 5.2 can be carried out according to the same scheme as the proof of similar statementcontained in [5] for the value function of optimal control problem with the other (different from given byrelations (20)-(22), (24)) terminal function corresponding to the different continuous extension of initialconditions of the Cauchy problem (1)-(3) to the boundary of state constraints.

6. SOLUTION OF THE ORIGINAL PROBLEM

The following statement is true, establishing the relationship between the value function V (·) in OCPand the generalized in the sense of Definition 1 solution of the problem (1)-(3).

81


Theorem 6.1.The function

u(t,x) =V (T � t,x), t 2 [0;T ], x 2 [0,1] (46)

is a continuous generalized solution to problem (1)–(3) in the domain PT .

Proof. Since the value function V (·) satisfies the equality V (T,x) = u0(x), �1 x 1, it followsfrom (46) that u(0,x) = u0(x), �1 x 1, i.e. the function u(·) satisfies the initial condition (3).

Let us show that u(·) is a viscosity solution of the equation (1) on the set PT . Let (t,x)2 (0,T )⇥(�1,1).Let us show that inequalities 15)–(16) are satisfied.

It follows from (46) and Theorem 5.2 that

? 6= D�u(t,x) =�

(�a,s)|(a,s) 2 D�V (t,x)

. (47)

So, applying the Lemma 3, we obtain

a+H(x,s)� 0, (a,s) 2 D�u(t,x).

The inequality (16) is proved.

It follows from (47) that superdifferential D+u(t,x) is nonempty only in the case when (t,x) isdifferentiability point for the function u(·). In this case

D+u(t,x) = (∂u(t,x)

∂ t,

∂u(t,x)∂x

).

It is known [6] that at points of differentiability the function V (·) of optimal result for OCP satisfies theBellman equation

∂V (t,x)/∂ t �H(x,∂V (t,x)/∂x) = 0. (48)

Taking into account (48), we obtain from (46)

∂u(t,x)∂ t

+H✓

x,∂u(t,x)

∂x

◆

= 0.

The inequality (15) is proved. Thus, the function u(·) is a viscosity solution of the equation (1) on the setPT .

The inequality (17) for the points of the set GT follows from Lemma 3.

Thus, the function u(·) defined by relations (46), is a generalized (in sence of Definition 1) solution ofthe Cauchy problem (1)–(3) in the domain PT .

Figure 3 and Figure 4 represent characteristics forming the generalized solutions of the problem(1)-(3).

ACKNOWLEDGMENTS

This work was supported by the Russian Foundation for Basic Research (project no. 14-01-00168), andPresidium of the Russian Academy of Sciences under the program ”Fundamental problems of nonlineardynamics in the mathematical and physical sciences.”

82


References

[1] M. G. Crandall and P.-L. Lions, “Viscosity solutions of Hamilton-Jacobi equations,” Transactions ofthe American Mathematical Society, vol. 277, no. 1, pp. 1–42, 1983.

[2] M. G. Crandall and R. Newcomb, “Viscosity solutions of Hamilton-Jacobi equations at the boundary,”Proceedings of the American Mathematical Society, vol. 94, no. 2, pp. 283–290, 1985.

[3] I. Capuzzo-Dolcetta and P.-L. Lions, “Hamilton-Jacobi equations with state constraints,” Transac-tions of the American Mathematical Society, vol. 318, no. 2, pp. 643–683, 1990.

[4] A. I. Subbotin, Generalized solutions of first order PDEs: The Dynamical Optimization Perspective.Boston: Birkhauser, 1995.

[5] N. N. Subbotina and L. G. Shagalova, “On a solution to the Cauchy problem for the Hamilton-Jacobiequation with state constraints,” Trudy Instituta Matematiki i Mekhaniki UrO RAN, vol. 17, no. 2,pp. 191–208, 2011.

[6] R. Bellman, Dynamic Programming. Princeton, NJ: Princeton Univ. Press, 1957.[7] L. S. Pontryagin, V. G. Boltyanskii, R. V. Gamkrelidze, and E. F. Mishchenko, The Mathematical

Theory of Optimal Processes. Oxford: Pergamon Press, 1964.[8] A. A. Milyutin, A. V. Dmitruk, and N. P. Osmolovskii, Os- molovskii, Maximum Principle in Optimal

Control. Moscow: Tsentr Prikl. Issled., Mekh.- Mat. Fak., Mosk. Gos. Univ., 2004.[9] N. N. Subbotina, “The method of characteristics for HamiltonJacobi equations and applications to

dynamical optimization,” Journal of Mathematical Sciences, vol. 20, no. 3, pp. 2955–3091, 2004.[10] D. B. Saakian, O. Rozanova, and A. Akmetzhanov, “Dynamics of the Eigen and the Crow-Kimura

models for molecular evolution,” Physical Review E, vol. 78, no. 4, 2008. 041908.[11] R. Courant and D. Hilbert, “Methods of mathematical physics,” New York: Interscience, vol. 2,

pp. 115–128, 1962.[12] S. N. Kruzhkov, “Generalized solutions of nonlinear equations of the first order with several variables.

I,” Matematicheskii Sbornik, vol. 112, no. 3, pp. 394–415, 1966.[13] F. H. Clarke, Optimization and nonsmooth analysis. New York: John Wiley & Sons, Inc., 1983.[14] R. T. Rockafellar, “Convex analysis, volume 28 of princeton mathematics series,” 1970.[15] J. Warga, Optimal control of differential and functional equations. Academic Press, 1972.

83

About This Journal AM is an open access journal published by Scientific Online Publishing. This journal focus on the following scopes (but not limited to): ¾ Algebraic Topology ¾ Approximation Theory ¾ Category Theory, Homological Algebra ¾ Coding Theory ¾ Combinatorics

¾ Cryptography ¾ Cryptology, Geometry ¾ Difference and Functional Equations ¾ Discrete Mathematics ¾ Dynamical Systems and Ergodic Theory ¾ Field Theory and Polynomials ¾ Fluid Dynamics ¾ Fourier Analysis ¾ Functional Analysis ¾ Functions of a Complex Variable ¾ Fuzzy Mathematics ¾ General Algebraic Systems ¾ Group Ring Theory ¾ Group Theory and Generalizations ¾ Heat Transfer ¾ Image Processing, Signal Processing and

Tomography

¾ Information Sciences ¾ Integral Equations ¾ Lattices, Algebraic Structures ¾ Linear and Multilinear Algebra, Matrix Theory ¾ Mathematical Biology and Other Natural

Sciences ¾ Mathematical Economics and Financial

Mathematics ¾ Mathematical Physics ¾ Measure Theory and Integration ¾ Neutrosophic Mathematics ¾ Number Theory ¾ Numerical Analysis ¾ Operations Research, Optimization ¾ Operator Theory ¾ Ordinary and Partial Differential Equations ¾ Potential Theory ¾ Real Functions ¾ Rings and Algebras ¾ Topological Groups ¾ Wavelets and Wavelet Transforms

Welcome to submit your original manuscripts to us. For more information, please visit our website: http://www.scipublish.com/journals/AM/

You can click the bellows to follow us: � Facebook: https://www.facebook.com/scipublish � Twitter: https://twitter.com/scionlinepub � LinkedIn: https://www.linkedin.com/company/scientific-online-publishing-usa � Google+: https://google.com/+ScipublishSOP

http://www.scipublish.com/journals/AM/

https://www.facebook.com/scipublish

https://twitter.com/scionlinepub

https://www.linkedin.com/company/scientific-online-publishing-usa

https://google.com/+ScipublishSOP

SOP welcomes authors to contribute their research outcomes under the following rules: ¾ Although glad to publish all original and new research achievements, SOP can’t bear any

misbehavior: plagiarism, forgery or manipulation of experimental data. ¾ As an international publisher, SOP highly values different cultures and adopts cautious attitude

towards religion, politics, race, war and ethics. ¾ SOP helps to propagate scientific results but shares no responsibility of any legal risks or harmful

effects caused by article along with the authors. ¾ SOP maintains the strictest peer review, but holds a neutral attitude for all the published articles. ¾ SOP is an open platform, waiting for senior experts serving on the editorial boards to advance the

progress of research together.

SOP TRANSACTIONS ON APPLIED MATHEMATICS Applications of Dynamic Programming to Generalized Solutions...

Documents

Transcript of SOP TRANSACTIONS ON APPLIED MATHEMATICS Applications of Dynamic Programming to Generalized Solutions...