An extended descent framework for variational inequalities

JOURNAL OF OPTIMIZATION THEORY AND APPLICATIONS: Vol. 80, No. 2, FEBRUARY 1994

An Extended Descent Framework for Variational Inequalities

D. L. Z~u 2 AND P. MARCOTTE 3

Communicated by F. Giannessi

Abstract. In this paper, we develop a very general descent framework for solving asymmetric, monotone variational inequalities, We introduce two classes of differentiable merit functions and the associated global convergence frameworks which include, as special instances, the projection, Newton, quasi-Newton, linear Jacobi, and nonlinear methods. The generic algorithm is very flexible and consequently well suited for exploiting any particular structure of the problem.

Key Words. Mathematical programming, variational inequalities, descent methods.

1. Introduction

The aim of this paper is to present an extended framework of globally convergent methods for solving monotone variational inequality problems (VIP). For a good survey of solution methods for solving VIP, the reader is referred to Harker and Pang (Ref. t). Several unifying algorithmic frameworks have already been proposed in the literature. In particular, Dafermos (Ref. 2) has presented a class of approximation methods whose iterates contract to the unique solution of a strongly monotone VIP. However, it has recently been shown by Zanni (Ref. 3) that the estimate of the contraction constant was often close to unity. Independently, Cohen (Ref. 4) has presented related iterative methods based upon the auxiliary

~This research was supported by the National Science and Engineering Research Council of Canada, Grant A5789, and by the Department of National Defence of Canada, Grant FUHBP.

Zlnvited Researcher, Center for Research on Transportation, Universit6 de Montrral, Montrral, Qurbec, Canada.

3Professor, Drpartement d'Informatique et de Recherche Oprrationnelle, Universit6 de Mon- trral, Montrral, Qurbec, Canada.

349 0022-3239/94/0200-0349507.00/0 �9 1994 Plenum Publishing Corporation

350 JOTA: VOL. 80, NO. 2, FEBRUARY 1994

problem principle. More recently, authors have proposed descent algorithms for VIP that proceed by minimizing surrogate objectives (gap functions). A linesearch strategy allows these algorithms to converge globally from any starting point [Marcotte (Ref. 5), Marcotte and Dussault (Ref. 6-7), Fukushima (Ref. 8)]. In particular, it has been shown by Marcotte and Dussault (Ref. 7) that Newton's direction is gap-decreasing, thus providing a global convergence framework for Newton's method applied to VIP. In this paper, we study a very general gap function that provides an equivalent differentiable optimization formulation for VIP. It includes, as special cases, the projective gap function of Fukushima (Ref. 8) as well as extensions thereof proposed by Wu, Florian, and Marcotte (Ref. 9) and Taji et al. (Ref. 10).

The paper will be organized as follows. Basic notations are introduced in Section 2. Section 3 addresses a general descent formulation for VIP. Section 4 defines the auxiliary problem which will be used in the definition of the general framework (Section 5) and its two realizations (Sections 6 and 7). A conclusion (Section 8) will close the paper.

2. Notations and Basic Definitions

Let V be a closed, nonempty, convex subset of R n, and let F be a mapping from V into R n. We say that F is monotone on V if

(F(v) - F ( u ) , v - u ) > O , Vv, u~V; (1)

it is strictly monotone on V if the above inequality is strict for v # u; it is strongly monotone on V with modulus # if there exists a positive constant tt such that

(F(v) -F(u),v-u)>_~llv -ulJ 2, Vv, u~V, (2)

where II II denotes the Euclidean norm in Rn; and it is strongly pseudomonotone on V with modulus/~, / /> 0, if for every u, v ~ V we have that

(F(u), u - v ) >_ 0 implies (F(v), v - u ) >_ fl [I v - u 112. (3)

We say that F is coercive with respect to v if there exists a vector v~ V such that

lim ( F ( v ) , v - v ~ 1 7 6 (4) w v,M-~ oo

Every strongly monotone mapping is strongly pseudomonotone and coercive. Furthermore, i f f is coercive and monotone, the solution set to VIP is convex and compact.

JOTA: VOL. 80, NO. 2, FEBRUARY 1994 351

If F is differentiable on V, then F is monotone [respectively, strongly monotone] on V if and only if the Jacobian matrix VF(v) is positive semidefinite [respectively, positive definite] on V. The mapping F is Lipschitz continuous on V, with Lipschitz constant L < 0% if

I[F(u)-F(v)[] <Lllu-vll, Vu, v V. (5)

Finally, a function f : V ~ R is said to be strongly convex on V with modulus a (a > 0) if, for all u, v ~ V and as[0, 1], we have

f(o~u +(1 -~)v) <_~f(u) +(1 - ~ ) f ( v ) -a[0~(1 -~) /2] [[u - v [[2. (6)

If furthermore f is continuously differentiable on V, then f is strongly convex on V if and only if its gradient is strongly monotone on V.

Now we consider the variational inequality problem: Find v*e V such that

(VIP) ( F ( v * ) , v - v * } > _ O , V v e V .

Definition 2.1. A gap function for VIP is a nonnegative function g defined on V such that g(v) = 0 ~ v is a solution to VIP, i.e., the solution set of VIP coincides with the global solutions of the mathematical programming problem minwvg(v ) .

Gap functions have been introduced to monitor the convergence of sequences to a solution of a variational inequality. The standard nondiffer- entiable gap function, defined as

g(v) = max (F(v) , v - u ) , u E V

has been shown by Marcotte and Dussault (Ref. 6) to be closely related to the duality gap associated with VIP.

3. Extended Differentiable Gap Function Formulation

Let f~(u, v): V x V ~ R be nonnegative, continuously differentiable on V x V, strongly convex on V with respect to u for all v e V and satisfy

f~(v, v) = 0 and V, f~(v, v) = 0, for all v e V.

Define

h(u, v) = (V(v), v - u ) - s v), (7)

g(v) = max h(u, v) = h(H(v), v), (8) u~ g

where the maximizer H(v) is unambiguously defined, since h is the sum of a linear and a strongly convex term.


Theorem 3.1. g is a gap function.

Proof. Since h(v, v) = 0, it is clear that g(v) > 0 for every v ~ V. Now, assume that v solves VIP. Then,

h(u, v) = (F(v ) , v - u ) - f~(u, v) <_ O,

which implies that

g(v) = O.

Conversely, assume that g(v) = 0. This implies that

h(u, v) < O, for all u ~ V,

and that v is a solution to the optimization problem max,,~ v h(u, v). Thus, v satisfies the variational inequality

(Vuh(v , v), u - v ) _< O, Vu ~ V,

i.e.,

Hence,

since

( F ( v ) , u - v ) + (Vu a ( v , v), u - v ) _> O, Vu e V.

(F(v), u - v ) > O, Vu ~ V,

( V . f~(u, v), u - v ) = O.

Consequently, v solves VIP. []

Theorem 3.2. If the mapping F is continuous on V, then the function g defined by (8) is also continuous. Furthermore, if F is continuously differentiable, then g is also continuously differentiable and its gradient is given by

Vg(v) = F(v) - (VF(v))r(n(v) - v) - V , f l ( n ( v ) , v), (9)

where H(v) is the unique solution of the maximization problem appearing on the right-hand side of (8).

Proof. The first part of the statement follows from the continuity of the max operator. If F is continuously differentiable, it follows, from the uniqueness of the maximizer H(v) of (8), that g is differentiable with gradient

Vg(v) = Vvh(H(v) , v)

= F(v ) - ( V F ( v ) , t4(v) - ~ ) - Vo a(H(v), v). []


Theorem 3.3. Let v* be a solution to VIP. I f F is strongly pseudomono tone with modulus # on V and if the gradient o f f~ with respect to u is Lipschitz cont inuous with constant La on V, then there exists a positive constant a such that

g(v) >_~l[v - v*l[2, V v ~ V . (10)

Proof . Since F is strongly pseudomono tone with modulus # on V and

( F ( v * ) , v - v * > > _ O , V v e V ,

we have

< v ( v ) , v - v * > >_~l lv-~*l l =, v o ~ v . (11)

Set

v t = v + t ( v * - v ) , for t e [0 , 1].

Since f~ is convex in u and its gradient with respect to u is Lipschitz cont inuous, we get

fl(v,, v) -f~(v, v) _ (Vu ~(v,, v), v, - v > = (Vu f~(v,, v) - Vu~(V, v), v, - v}

_ _ _ L o l l s , - vii 2

Then, we have

g(v) >_ (F(v ) , v - v t > - ~~(vt, v)

= (F(v) , v - vt ) - (f~(v,, v) - f~(v, v))

> t<F(v) , v - v*> - Z~ I1~, - ~112

>_ t (F (v ) , v - - v * > - z.t=llv -~*11 =

> ( t# -- t2Lo)I1~ - *11:

= ~214Ln - L a ( t - / ~ / 2 L o ) 2] Iio - v*ll =

In particular, if we let

t = min{1, l t /2Ln} ,

we obtain

g(v) > ~[I v -- ~*11 =,

where

~p -- La ,

= [ # 2 / 4 L ~ '

which proves the result.

if # > 2La,

if # _< 2Ln,

[]


4. Auxiliary Variational Inequality

Let F(u, v): V • V ~ R" be a mapping (the auxiliary mapping), continuous and strongly monotone with respect to u, and continuous with respect to v. Let us consider the auxiliary variational inequality problem: Find w ~ V such that

(AVIP(v)) (F(w, v) - F ( v , v) + r ( v ) , u - w ) > 0 , Vu~V. (12)

In view of the strong monotonicity of F(u, v) with respect to u e V, this auxiliary variational inequality has a unique solution, which we denote W(v).

Proposition 4.1. The mapping W: V--, V is continuous on V. Further- more, v is solution of VIP if and only if it is a fixed point of W, i.e., v -- W(v).

Proof. The first half of the proposition follows from Ref. 1, Theorem 5.4. To prove the second half, we first suppose that v = W(v). This yields

(F(v), u - ~) >_ 0, v u ~ v,

and v solves VIP. Conversely, suppose that v solves VIP. Then,

( F ( 0 , W(v) - v ) >__ 0,

and it follows from (12) that

(F(W(v), v) - F(v, v) + F(v), v - W(v) ) 2 0 .

Adding the two preceding inequalities, one obtains

( r ( w ( o , v) - r ( . , v), v - w ( v ) ) > o,

and we conclude, from the strong monotonicity of F with respect to u, that the equality v = W(v) must hold. []

5. Extended Framework for Solving VIP

Proposition 4.1 suggests a fixed-point algorithm for solving VIP. For given vke V, consider the auxiliary variational inequality problem: Find v k+ le V such that

(AVIP(vk)) (r(v~+',Vk)--r(Vk, Vk)+F(vk),U--Vk+')>_O, V u e V , (13)

i.e., vk+1= W(vk). In our approach, we will instead use

d k = W(v k) - v k (14)

as a descent direction.


The strong monotonicity of F with respect to u ensures that the auxiliary problem always has a unique solution W(vk). For specific choices of F, AVIP(v k) will be easier to solve than the original VIP. In this paper, we will present a very general decent framework for solving monotone variational inequalities, which incorporates a line search strategy of Armijo type. If v k = W(vk), we obtain the solution to VIP. Otherwise, the vector d k = W(v k) - v k is a descent direction for g at v k under conditions to be considered later. Let us first state the generic algorithm.

Generic Algorithm A1.

(i) Select a feasible vector v ~ a tolerance factor e, and parameters 7, t , a in the open interval (0, 1). Set k to zero.

(ii) At step k of the algorithm, determine the search direction d k by solving the auxiliary variational inequality problem AVIP (vk); see formulas (13) and (14). If g ( v k + d k) <Tg(vk), set vk+l= vk+ dg. Otherwise, select the smallest integer m such that

g(v k) - g ( v k + flmdk) >__ --aflm(Vg(vk), dk) , set ~k = tim

and v k+ 1 = v k + ~kdk.

This step follows the Armijo linesearch rule.

(iii) Stop if II v § - v ll < , . Otherwise, increase k by one and return to Step (ii).

The gap function g monitors the convergence of the iterates to a solution of VIP. Through suitable choices of the auxiliary function f~, alternative gap functions can be derived. Each combination of f~ and F will generate a different descent algorithm. The function [1 should be chosen such that the gap function g and its gradient Vg are easily computed at every point of V. In the following sections, we introduce two types of gap functions and the associated convergence frameworks.

6. First Class of Gap Functions

Let

f~(u, v) = (1/2)(u - v)rG(u - v),

where G is any n x n symmetric positive-definite matrix. Define

hi(u, v) = (F(v), v - u ) - ( 1 / 2 ) ( u - v ) r G ( u - v ) (15)


and the associated gap function

gl (v) = max hi (u , v), (16) UE V

which coincides with the gap function considered by Fukushima (Ref. 8). Let H(v) be the unique solution of the maximization problem

max~ hi (u, v), which is equivalent to the problem

min Ilu - (~ - G-'F(v))[12- (17) u ~ V

We have that

H(v) = Projv,6(v - G - I F ( v ) ) ,

the projection of the point v - G-~F(v ) on the set V with respect to the matrix norm [l" I[o" By construction of H(v) , we have

( F ( v ) + G(H(v ) - v), v - H(v) ) >>. O.

Thus,

(F (v ) , v - H ( v ) ) > I I H ( 0 - v I1~,

and we obtain

Vgl (v) = F(v) - (VF(v) - G ) r ( H ( v ) - v).

This leads to the following procedure.

(18)

(19)

Assumption 6.1. Let the mapping F(u, v) be continuous differentiable in u and satisfy

lira r(u, 0 - r e ( 0 II < ~, Vu, v E V, (20)

for some positive constant 6.

Proposition 6.1. Let F be continuously differentiable and strongly monotone with modulus/~ on V. Let F(u, v) be continuously differentiable and strongly monotone with respect to u (with modulus 2) and continuous

Convergence conditions for the preceding algorithm will use the following assumption.

Genetic Algorithm A2. This algorithm differs from Generic Algorithm A1 in that the gap function gl is used as the objective in place of the gap function g. Clearly, this algorithmic scheme is subsumed by that of Generic Algorithm A1.


with respect to v on V. Let G be an n x n symmetric positive-definite matrix with minimum eigenvalue 2rain(G). Moreover, assume that one of the following conditions is satisfied:

(a) Assumption 6.1 is met, with 5 < 2 and 6 <2in t , (G)< I1~11 < 22 -- 6.

(b) Assumption 6.1 is met, with 6 < # and 6 -<~in(G) -< IIGII < 2# - 6.

(c) F(u, v) = V r ( v ) ( u - v) (modified Newton method) and ]IG [] < 4~. (d) F(u, v) = G(u - v) (projection method).

Then, the vector d k = W(v k) - - v k satisfies the inequality

( V g , ( v k ) , d k ) < - -4 IId~ll ~, (21)

for some r /> 0, i.e., d k is a feasible descent direction for gl at v k, whenever v k is not a solution to VIP.

Proof.

(a) For simplicity of presentation, we temporarily omit the super- script k. Since d = W(v) - v , it follows from (19) that

(Vg,(v), d ) = (F(v), W(v) - v ) - (VF(v) T(W(v) - v), n (v ) - v )

+ ( W ( v ) - v, C ( H ( v ) - v ) )

= - ( r ( W ( v ) , v ) - r ( v , v) + F(v), H(v) - W(v))

+ ( F ( v ) , H ( v ) - v ) + [IS(v) - v 112

+ ( r ( W ( v ) , v) - r (v , v) - r E ( v ) ~ ( W ( v ) - v), H ( v ) - W ( v ) )

- ( d , V F ( v ) T d ) - (1/2)II w(v) - H(v)II~

- (1/2) I l u ( v ) - v II~ + (1/2) (d, Gd). (22)

Since W(v) is a solution to AVIP(v), the first two lines in (22) are nonpositive; see (18). Let us rewrite the third term T as

T = (F(W(v), v) -- F(v, v) -- V F ( v ) r ( W ( v ) - v), H(v ) - W ( v ) )

= (F(W(v), v) - F(v, v) - V F ( v ) r ( W ( v ) - v), v - W ( v ) )

+ ( r ( W ( v ) , v) - r (v , v) - V F ( v ) ~ ( W ( v ) - v), H ( v ) - v )

= T, + r : .

We have

T, _ - 2 Ilall 2 + <d, VF(v) d )


and

(fo ) T2 = [V. F(v + t ( W ( v ) - v), v) - VF(v)] r ( W ( v ) - v) dt , H ( v ) - v

< [Vu F(v + t ( W ( v ) - v), v) - VF(v)] r(W(v) - v) d t �9 I lH(v) - ~ II

j" -- I l tvu r ( , + t ( W ( v ) - v ) , v) - - V F ( v ) l r ( W ( v ) - v)II d t " IIH(o) - ~ 11

0

_< (1/2)~[ II w(v) - ~ II 2 + IIH(~) - ~112]

_< (1]2)~ l[ W ( v ) - v II 2 + (1/2)~(~min(G)) -x ]lU(v ) - v II~.

Regrouping the terms we obtain

(Vgl(v), d ) < - [ 2 - (1/2)(5 + Hall)] [[d[I 2

- (1]2)( 1 - 6(2mi,(G)) -1) [[H(v) - v I I~.

Let

~/= 2 - (112)(6 + ]]G[]).

According to our assumptions, we have that q is positive and 1 - 5(2mi.(G))-1 is nonnegative. Thus,

<Vgl(v), d> < - ~ Ildl[ 2.

( b ) In this case, we estimate T directly f rom (22),

T = (V(W(v), v) - F(v, v) - V r ( v ) r ( W ( v ) - v), H ( v ) - W ( v ) )

( ) = [V, F(v + t ( W ( v ) - v), v) - VF(v)] 7"(W(v) - - v) dt , H ( v ) - W ( v ) i

"1 dt <_ [ V u r ( v + t ( W ( v ) - v ) , v ) - V F ( v ) ] T ( W ( v ) - v ) - [[H(v) - W(v) [[

"1

<_ ][[V ,r (v+t (W(v) -v ) , v) - VF(v)]~(W(v) - v ) [ ] dr. ][H(v) -- W(v)[[ 0

_< (1/2)6[ [I W ( v ) - v [[2 + []H(v) - W(v)[[:]

__ (1/2)5 l[ W ( v ) - v[I 2 + (1]2)5(2mi,(G)) -1 [IH(v) - W(v)112.

Thus, we have

( V g l ( V ) , d ) < - [ # - ( 1 / 2 ) ( 6 + [[G[[)] []dl[ 2

- ( I/2)( 1 - 6(2mi.(G)) -1)[[H(v) - W(v)][2 .

JOTA: VOL 80, NO. 2, FEBRUARY 1994

Let

= ~ - (112)(~ + IIGD

Our assumptions imply that ~/is positive and

1 - - ~(2min(G))- I ~ 0.

Therefore,

( V g , ( v ) , d ) < - ~ Ildll =

(c) In this case, we have T = 0 and

(Vg, (v), d) = - ( d , VF(v)d) - (1/2)II w(v) - n(~)I1~

- (1[2)IlH(v) - v II~ + ( 1/2)(d, Gd)

< - ( d , Vr(v)rd) + (1]4) (d, Gd)

-< - ( ~ - HG 1]/4)]ldl[ =.

Let

n = . -]IGl[/4,

which is positive. Then,

(Vgl (V), d) < - - . Ildll =

(d) In this case, W(v) = H(v) and T = 0. From (22), we have

(Vg, (v), d) = - (d, VF(v)d) - (1/2) IIH(v) - v II~ + (1/2) (d, Gd)

= - (a, VF(v)d)

<_ -.lidll 2,

and with r/=/~, we have

(Vg,(v), d ) < - ~ Ildll =

359

Theorem 6.1. (Global Convergence). Let F, F(u, v), and G be given as in Proposition 6.1. Then, for any starting point v% V, the sequence {v *} converges to the unique solution of VIP.

Proof. Since v k and 1) k -4;- d k both belong to V and since 0 < ~k < 1, it follows from the convexity of V that the sequence {v k} is feasible. From Proposition 6.1 and the linesearch rule, the sequence {gl(v~)} is strictly decreasing. From Theorem 3.3, the sequence {v k} is bounded, and hence has at least one accumulation point. Let the sequence {vk}, k sK , be any convergent subsequence of {vk}, and let ~e V be its limit point.

[]

360 JOTA: VOL 80, NO. 2, FEBRUARY 1994

If

gl (v k + d k) < ~gl (ok), k eK, ~, < 1,

holds infinitely, there exists a subsequence {ok}, k e K ' = K, such that

lira gl(v k) = 0 , keK ' .

Since g~ is continuous, g l ( ~ ) = 0 . Moreover, from Theorem 3.1, 6 is a solution of VIP.

Next, we consider the case were

g, (o k + d k) _< ~g, (v k)

holds for only finitely many values of the index k, i.e., there exists k-~ such that

gl (Ok "at- ark) > ]~gl (ok), k > k].

Since dk= W(v k) - -v k and W is continuous, the sequence {d k} must converge to some a v = W(v-) - ~. To prove that ~ is a solution of VIP, it will be sufficient to show that a7 = 0. Assume the contrary. Then, there exists E > 0 and 22 such that IIdU -> ~ for all values of k larger than ~ . It then follows from (21) that

(Vgl (ok), d k) < - rl~ 2, Vk ~K, k >_ 22, (23)

from which we obtain

(Vg,(O), a~ < -r/~ 2 <0 . (24)

On the other hand, it follows from the linesearch rule that \

gl (Ok) -- gl ( ok + 1) > -- O.~t k (Vg 1 (ok), d k) > 0 (25)

and

g,(v k) --gl(Vk +(~k/fl)d k) < --a(~k/fl)(Vgl(Vk),dk), (26)

for all k~K, k >>-El. Inequality (25) implies that the nonnegative sequence {gl(vk)} is monotonically decreasing and therefore converges. Then, we have

lira ~k (Vgi(vO, dk) = O. k e K

Hence from (23), we get

lim ~t k = 0. k ~ K

Dividing both sides of (26) by ctk/fl and taking the limit, we obtain

--(Vg,(v-), aT) < - a (Vg , (~ ) , t~ ,


and since tr < 1,

(Vgl(~), a> _> 0,

which contradicts (24). Consequently, we must have aT= 0, i.e., ~ is a solution of VIP. Moreover, the strong monotonicity of F ensures that VIP has a unique solution, and we can conclude that the entire sequence {v k} converges to the unique solution of VIP. []

Remark 6.1. In each of the following situations, Assumption 6.1 is satisfied with F(u, v) = A(v)(u - v):

(i) A(v) = VF(v); (ii) A(v) ,~ VF(v) and HA(v) - VF(v)II -< 6, t$ > 0; (iii) A(v) = (1]2)[VF(v) + VF(v) r] and

II 1]2[Vt(v) - VF(v) r] II -- ~, ~ > 0; (iv) A(v) = D(v), the diagonal of VF(v), and

[ I V F ( v ) - O(v)II -< ~, ~ > 0;

~L(v) +D(v) / to and nD(v)(l/o~ - 1) + U(v)II _<~, (v) A(v)

= [U(v) + D(v)/~o and [IO(v)(l/o~ - 1) + L(v)II-< ~, where L(v) [respectively, U(v)] denotes the lower triangular [respectively, upper triangular] part of VF(v), and co r 2). Here, (i) refers to Newton's method, (ii) refers to the quasi-Newton method, (iii) refers to the symmetric Newton method, (iv) refers to the linear Jacobi method, and (v) refers to the SOR, linearized Gauss-Seidel method (co = 1).

If one sets

r(u, v) = VF(v)(u - v),

the auxiliary variational inequality of Algorithm A1 becomes

(W(vk)(w-v'9 + F(v%u-w) >_O, Vu~V,

which corresponds to the modified Newton method of Taji ct al. (Ref. 10), for which we give the weaker convergence condition

IIGll < 4#.

Such a condition, which is not required in the gap-decreasing algorithm of Marcottc (Ref. 5), is essential here. Indeed, if G were arbitrary, the algorithm would not necessarily yield a descent direction for the gap function, as may be observed on the following counterexample.

Example 6.1. Let

, c = {v [ v, _ 0, v2 _ 0}.


We note that F is strongly monotone with modulus

/~ = 3 -(1/2)x//i-7.

This problem has the solution v* = (0, 4.8) r. Take

(7 = I/e, ~ > O.

At v ~ (10, 0) r, we have

= ~(0, 0) r, i f e > 0.2, H'(v~ [(10 - 50e, 0) r, otherwise.

From (19), if e < 0.2, we have

Vgl (v ~ = (50~, 6 + 150~) r,

(v* - v ~ Vgl (v~ = 220g + 28.8 > 0.

If g > 0.2, we have

Vgl (v ~ = (60 - (10let), 36) r,

(v* - v ~ Vgl(v~ = -427 .2 + 100/e.

Only for ct > 0.234 is (v* - v ~ Vgl(x~ negative, yielding a descent direction of gl at v ~ Our lower bound on e is 0.266, while Taji et al.'s is 0.533.

Remark 6.2. Under some regularity assumption, it can be proven that there exists an index k: such that

Ctk = 1, for all k > E,

thus ensuring a quadratic rate of convergence; see Ref. 7.

Corollary 6.1. Let F be as in Theorem 6.1, and let VF(v) have positive diagonal elements. Let

VF(v) = D(v) + C(v),

D(v) and C(v) are the diagonal and off-diagonal parts of VF(v), respectively. If

IfC(v)H < ~ and 6 (~min(D(1))),

choose a matrix G such that

_ ;l~in(G ) < IIGII < 22m~n(D(v)) - ~ .

Then, for any starting point v~ V, the sequence {v k} generated by the modified linear Jacobi method is well defined and converges to the unique solution of VIP.

JOTA: VOL. 80, NO, 2, FEBRUARY 1994 363

Proof. Set

F(u, v) = D(v)(u - v),

which is strongly monotone with respect to u with modulus 2min(D(v)). Since

C(v) = VF(v) - D(v) = VF(v) - Vu r(u, v)

and

lie(v) II ~ a, with 6 < 2min(D(v)),

the conclusion of Theorem 6.1 holds. []

Corollary 6.2. Let F be as in Theorem 6.1. Let A(v) and S(v) denote the symmetric and skew-symmetric parts of the positive-definite matrix VF(v), respectively. If IIs(o II-< a and t3 < , , choose the matrix G such that

a _< ;~,o(a) _< tlatl < 2 . +a .

Then, for any starting point v% V, the sequence {v k} generated by the modified symmetric Newton method is well defined and converges to the unique solution of VIP.

Proof. As in the proof of Corollary 6.1,

r ( u , v) = a ( v ) ( u - v)

is strongly monotone uniformly in u, with modulus 2 = , . Since

s ( v ) = VE(v) - A(v) = VF(v ) - V . r ( u , v)

and

IIs(v) ll ~ a, a < . ,

the conclusion of Theorem 6.1 holds. []

We note that the conditions of Corollaries 6.1 and 6.2 are sufficient conditions for local convergence of the linear Jacobi and symmetric New- ton methods (Harker and Pang, Ref. 1). Thus, there exists an integer ~- such that we can switch to the true iterative method from the k-th iteration on. In real applications, one would resort to trial and error to determine ~.


7. Second Class of Gap Functions

In this section, f~ assumes the general form introduced in Section 3, and the auxiliary mapping is chosen as

r'(u, v) = Vu ~(u, v).

We write

hz(u, v) = (F (v ) , v - u ) - ~ (u , v), (27)

g2(v) = max h2(u, v) = (F (v ) , v - W ( v ) ) - f~(W(v) , v), (28) UE V

where W(v) is the unique solution of the auxiliary variational inequality AVIP(v). From (9), we have

Vg2(v) = F(v ) + ( V F ( v ) ) r ( v - W(v) ) - Vv f~(W(v), v). (29)

Generic Algorithm A3.

(i) Select a feasible vector v ~ a tolerance factor E, and parameters ~, fl, a in the open interval (0, 1) Set k to zero.

(ii) At step k of the algorithm, determine the search direction d k by solving the auxiliary variational inequality problem

(AVIP(ve)) ( V u f ~ ( W ( v k ) , v k) + F ( v k ) , u - W ( v k ) ) > 0 , Vu E V,

and let d k = W ( v k) - v k. If g 2 ( v k + d k) < vg2(vk), set v k+l = V k + d k. Otherwise, select the smallest integer m such that

g2(t~ k) -g2(v k q- flmdk) > --tTflm(Vg2(vk), dk),

and set v k+ 1 = v k + flmdk"

This step follows from Armijo's linescarch rule.

(iii) Stop if ]Iv T M - vkll < ~. Otherwise, increase k by one and return to Step (ii).

Assumption 7.1. The function f~(u, v) is continuously differentiable on V x V and satisfies (Vu f~(u, v) + Vv f~(u, v), u - v ) > O, u v ~ V.

Proposition 7.1. If F is strongly monotone with modulus # on V, fl satisfies Assumption 7.1, and v g is not a solution to VIP, then dk = W ( v k) - - v k satisfies

(Vg2(vk) , d k ) < --~/[IdYll 2, (30)

for some ~/> O, i.e., d k is a feasible decent direction for g2 at v k.


Proof. Let us temporarily drop the iteration index k. Since d = W(o) - o , it follows from (29) that

(Vg2(v), d ) = (F(v), W(v) - v ) - (VF(v)r(W(v) - v), W(v) - v )

- ( v o t~ (w(v) , v), w(v) - v )

= ( v ~ a ( w ( v ) , v) + F(v) , W(v) - v )

- (Vo ta(W(v), v) + Vu ~ ( w ( o ) , v), W(v) - v )

- (d, VF(v )d ) . (31)

Since W(v) is a solution of AVIP(v), the first term of (31) is nonpositive. From Assumption 7.1, the second term of (31) is also nonpositive. Also, the strong monotonicity of F implies

(a, V F ( v ) d ) >_ ~ Ildll =

Taking r /= #, the result follows. []

Theorem 7.1. Global Convergence Theorem. Let F be as in Proposi- tion 7.1. Then, for any starting point v~ V, the sequence {v k} converges to the unique solution of VIP.

Proof. It is similar to the proof of Theorem 6.1. []

If

f~u , v) = (1 /2 ) (u - v ) r / ~ ( v ) ( u - v),

B(v) is symmetric and VB(v)(u - v) is positive definite for all u, v e V, then Assumption 7.1 is satisfied. This is obviously true if B(v) = B, where B is a positive-definite matrix (usual projection method), and we recover the framework of Wu, Florian, and Marcotte (Ref. 9).

8. Conclusions

In this paper, we presented a very general decent framework for solving VIP and analyzed its relationship with existing algorithms. It is clear that the choice of a gap function should be made in accordance with that of the underlying algorithm. As an example, it is advantageous to use the projective function jointly with the projection algorithm, and so on. Let us also mention that decomposition can be obtained from our generic algorithm. For instance, if R" = R nl x R n2 . . . . . x R ~r Vg c R% i = 1 , . . . , N, and V = V~ x V2 x . . . . . x VN, a decomposition can be achieved by adequately choosing the mappings F and ft. Also, coupled with the


techniques of Zhu and Marcotte (Ref. 1 I), our method can deal with the following variational inequality: Find v* such that

(F(v*), v - v*) +f~ (v) - f ~ (v*) > O, Vv ~ V, (32)

where f~ is a convex and continuously differentiable function from R" into R. We can also use the partial technique of Zhu and Marcotte (Ref. 11) to solve variational inequalities where F assumes the form

r(u, v) = (V l(u), F 2 (v)) r,

where VF1 is symmetric and VFz is skew-symmetric. This will be discussed in a subsequent paper.

References

1. HARKER, P. T., and PANG, J. S., Finite-Dimensional Variational Inequality and Nonlinear Complementarity Problems: A Survey of Theory, Algorithms, and Applications, Mathematical Programming, Vol. 48, pp. 161-220, 1990.

2. DAFERMOS, S., An Iterative Scheme for Variational Inequalities, Mathematical Programming, Vol. 26, pp. 40-47, 1983.

3. ZANNt, L., On the Convergence Rate of Two Projection Methods for Variational Inequalities in R", Calcolo, 1992.

4. COHEN, G., Auxiliary Problem Principle Extended to Variational Inequalities, Journal of Optimization Theory and Applications, Vol. 59, pp. 325-333, 1988.

5. MARCO'rrE, P., A New Algorithm for Solving Variational Inequalities, with Application to the Traffic Assignment Problem, Mathematical Programming, Vol. 33, pp. 339-351, 1985.

6. MARCOTTE, P., and DUSSAULT, J. P., A Sequential Linear Programming Algorithm for Solving Monotone Variational Inequalities, SIAM Journal on Control and Optimization, Vol. 27, pp. 1260-1278, 1989.

7. MARCO'rrE, P., and DUSSAULT, J. P., A Note on a Globally Convergent Newton Method for Solving Monotone Variational Inequalities, Operations Research Letters, Vol. 6, pp. 35-42, 1987.

8. FUKUSHIMA, M., Equivalent Differentiable Optimization Problems and Descent Methods for Asymmetric Variational Inequality Problems, Mathematical Pro- gramming, Vol. 53, pp. 99-110, 1992.

9. Wu, J. H., FLORIAN, M., and MARCOTTE, P., A General Descent Framework for the Monotone Variational Inequality Problem, Mathematical Programming, Vol. 61, pp. 281-300, 1993.

10. TAJI, K., FLrKUSHIMA, M., and IBARAKI, T., A Globally Convergent Newton Method for Solving Monotone Variational Inequalities, Mathematical Program- ming, Vol. 58, pp. 369-383, 1993.

11. Zrtu, D. L., and MARCOTrE, P., Auxiliary Problem Principle and Descent Methods, Preprint, University of Montreal, 1991.

An extended descent framework for variational inequalities

Documents

Transcript of An extended descent framework for variational inequalities