International Journal of Computer MathematicsVol. 00, No. 00, Month 2005, 1–9

Iterative method with modification of the right-hand sidevector for non-linear complementarity problems

NATAŠA KREJIC, ZORANA LUŽANIN* and SANJA RAPAJIC

Department of Mathematics and Informatics, University of Novi Sad, Trg Dositeja Obradovica 4,21000 Novi Sad, Serbia and Montenegro

(Received 6 October 2004; in final form 13 October 2004)

Non-linear complementarity problems (NCPs) are often solved by iterative methods based on a gen-eralization of the classical Newton method and its modifications for smooth equations. We consider amethod with modification of the right-hand side vector for a reformulation of the semi-smooth equationarising from the NCP and prove local convergence of the proposed method. Some numerical resultsare presented.

Keywords: Non-linear complementarity problems; Semi-smooth systems; Modified Newton method

AMS Subject Classifications: 65H10; 90C33

C.R. Category: G.1.3

1. Introduction

For a given continuously differentiable mapping F : Rn → Rn, where F(x) = (F1(x), . . . ,

Fn(x)), Fi : Rn → R, i = 1, . . . , n, the non-linear complementarity problem (NCP) consistsof finding a vector x = (x1, . . . , xn) ∈ Rn such that

x ≥ 0, F (x) ≥ 0, xTF(x) = 0. (1)

This has a large number of important applications in physics, economics and engineering.There are several ways of reformulating NCPs as non-linear systems [1–4]. The resultingsystem of non-linear equations is non-smooth, or more precisely, semi-smooth. A large classof iterative methods based on the Newton method and its generalizations has been developedfor systems of this type in recent years [1–3, 5, 6]. These methods deal with semi-smoothnessby using some kind of generalized Jacobian.

The main purpose of this paper is to present a generalization of a method with a modificationof the right-hand side vector (MRV) which will be used for solving NCP. The method for thesmooth case was introduced in [7]. The basic idea in the smooth case was acceleration of

*Corresponding author. Email: zorana@im.ns.ac.yu

International Journal of Computer MathematicsISSN 0020-7160 print/ISSN 1029-0265 online © 2005 Taylor & Francis Group Ltd

http://www.tandf.co.uk/journalsDOI: 10.1080/00207160500168508

Techset Composition Ltd, Salisbury gcom116833.TeX Page#: 9 Printed: 20/6/2005

2 N. Krejic et al.

the fixed Newton method by a relaxation parameter and modification of the right-hand sidevector, leading to a low linear algebra cost. The smooth case algorithm for the non-linearsystem G(x) = 0 [7], where G: Rn → Rn, is given below.

ALGORITHM: MRV method Let x0 ∈ Rn and the non-singular matrix A ∈ Rn,n be given.For k = 0, 1, 2, . . .

• Step 1. Solve

Ask = (αkH(xk) − I )G(xk) (2)

where H(xk) = G′(xk) − A, I is the identity matrix and αk is a real parameter.• Step 2. Define xk+1 = xk + sk.

Every iteration requires calculation H(xk), using the Jacobian at the current iteration andsolving the system of linear equations (2).

The initial approximation A of the Jacobian is fixed throughout iterative process, thusrequiring a solution of linear systems with the same matrix in all iterations, as in the fixedNewton method. Convergence results for A = G′(x0) and the special choice of αk are givenin [7]. Introduction of a relaxation parameter not only enhances convergence but can alsogenerate a method which is similar to the Newton method.

In this paper we propose an MRV method for NCP based on a semi-smooth reformulationof the NCP and we prove local convergence of the method.

The paper is organized as follows. We present the basic properties of the semi-smoothreformulation of NCP and the semi-smooth functions in section 2. The algorithm is formulatedin section 3. Some appropriate choices of relaxation parameters and the convergence propertiesof the algorithm are described and discussed. Section 4 reports the results from a computationalexperiment using the methods studied here and compares them with Newton and fixed Newtonmethods. Section 5 contains some final remarks.

2. Preliminaries

First, we give some notation. The ith row of M ∈ Rn,n is denoted by [M]i , and ei, i = 1, . . . , n,represents the canonical base for Rn.

Problem (1) can be reformulated as a non-smooth system of non-linear equations

G (x) = 0 (3)

with

G (x) = min {x, F (x)} (4)

where min is taken componentwise, see [8]. It is easy to see that x∗ is a solution of (3) if andonly if it solves the NCP. If there exists i ∈ {1, 2, . . . , n} such that xi = Fi(x), the function G

may be non-smooth at x, in which case the classical results for smooth problems cannot beapplied. We extend the results from the smooth case, defining an MRV method for NCP, andprove its local q-linear convergence.

The mapping G given by (4) is locally Lipschitzian and hence, by Rademacher’s theorem,G is differentiable almost anywhere [9].

The concept of ∂B(x) and ∂b(x) is introduced in [10, 11].

Non-linear complementarity problems 3

DEFINITION 1 Let G: Rn → Rn be a locally Lipschitzian function and let DG denote the setwhere G is differentiable. For x ∈ Rn the set given by

∂BG (x) ={

limxk→x

G′ (xk) : xk ∈ DG

}

is called the B-subdifferential of G at x.

Clarke’s generalized Jacobian [9] is

∂G (x) = conv ∂BG (x) .

DEFINITION 2 Let G: Rn → Rn be locally Lipschitzian at x ∈ Rn. We say that G is semi-smooth at x if

limV ∈∂G(x+ty ′),

y′→y,t→0

Vy ′

exists for every y ∈ Rn.

The mapping G: Rn → Rn defined by (4) is semi-smooth in the sense of definition 2. It wasproved in [12] that a function from Rn to Rn is semi-smooth if and only if all its componentfunctions are semi-smooth.

We define the set

∂bG (x) = ∂BG1 (x) × ∂BG2 (x) × · · · × ∂BGn (x)

for G(x) = (G1(x), G2(x), . . . , Gn(x)), Gi : RnarrowR, i = 1, . . . , n.

The lemmas and theorem given below will be used to prove the convergence of the newmethod which will be proposed in section 3.

LEMMA 1[13] Assume that G:RnarrowRn is locally Lipschitzian function. If all V ∈ ∂bG(x)

are non-singular, then there exists a positive constant β such that ‖V −1‖ ≤ β for any V ∈∂bG(x). Furthermore, there exists a neighborhood N(x) of x such that for any y ∈ N(x), allW ∈ ∂bG(y) are non-singular and satisfy ‖W−1‖ ≤ 10/β9.

LEMMA 2 [14] If G: Rn → Rn is locally Lipschitz continuous and semi-smooth at x ∈ Rn,then

limh→0, V ∈∂G(x+h)

‖G (x + h) − G (x) − V h‖‖h‖ = 0.

Lemma 2 and the semi-smoothness of G imply that

‖G (x + h) − G (x) − V h‖ = o (‖h‖) (5)

for any V ∈ ∂bG(x + h) and h sufficiently small if ‖ · ‖ is the infinity norm. Throughout therest of the paper ‖ · ‖ is used to denote the infinity norm.

Extension of the smooth inexact Newton method (IN) for solving smooth equations tosolving non-smooth equations is introduced in [15] with local convergence results.

THEOREM 1 [15] Let G: RnarrowRn be a locally Lipschtzian function. Assume that G issemi-smooth at x∗, where G(x∗) = 0, and all matrices V∗ ∈ ∂bG(x∗) are non-singular and{ηk} is a sequence of positive numbers. Then there exists a neighborhood of x∗ and η > 0 suchthat, if {ηk} is bounded by η, the non-smooth inexact Newton method is q-linearly convergentto x∗ in its neighborhood.

4 N. Krejic et al.

3. The algorithm and convergence analysis

We now define a new method, the MRV method, for NCP. Let x0 be an initial approximation,A ∈ Rn,n and λ ∈ {0, 1}. The matrix B is defined by

[B]i =

ei if x0i < Fi

(x0

)[A]i if x0

i > Fi

(x0

)λei + (1 − λ) [A]i if x0

i = Fi

(x0

) , i = 1, 2, . . . , n. (6)

With each x ∈ Rn we associate a set of matrices {V (x)} such that

[V (x)]i =

ei if xi < Fi (x)[F ′ (x)

]i

if xi > Fi (x)

λei + (1 − λ)[F ′ (x)

]i

if xi = Fi (x)

, i = 1, 2, . . . , n (7)

where λ ∈ {0, 1}. It follows from the definition of ∂bG(x) that V (x) ∈ ∂bG(x).

ALGORITHM: MRV method for NCP Let x0 ∈ Rn, A ∈ Rn,n be given, and matrix B definedby (6) be non-singular.

For k = 0, 1, 2, . . .

• Step 1. Solve

Bsk = (αkH(xk) − I )G(xk) (8)

where

H(xk) = V (xk) − B. (9)

V (xk) is given by (7) and αk is a real parameter.• Step 2. Define xk+1 = xk + sk.

As in the smooth case, each iteration requires the calculation of the right hand side vectorin (8) and the solution of the system of linear equations with the same matrix B. The linearalgebra cost for solving system (8) is again O(n2) flops in the general case, assuming thatV (xk) is available. Roughly speaking, if A is an approximation of F ′ at the solution of theNCP, matrix B is an imitation of the generalized Jacobian of G.

We will prove local convergence of the new method under the following assumptions.

A1. There exists x∗ ∈ Rn which is a solution of the NCP problem (1), i.e. G(x∗) = 0,A2. All elements of ∂bG(x∗) are non-singular.

For NCP the assumption A2 is equivalent to the b-regularity assumption [16].The solution x∗ is degenerate if

β(x∗) = {

i: i ∈ {1, 2, . . . , n} , x∗i = Fi

(x∗) = 0

} �= ∅.

Otherwise it is non-degenerate.We associate with x∗ ∈ Rn the set of matrices {V∗ = V (x∗)} such that

[V∗]i =

ei if x∗i < Fi (x

∗)[F ′ (x∗)

]i

if x∗i > Fi (x

∗)λei + (1 − λ)

[F ′ (x∗)

]i

if x∗i = Fi (x

∗), i = 1, 2, . . . , n (10)

where λ ∈ {0, 1}. From the definition of ∂bG(x) we have V∗ ∈ ∂bG(x∗).

Non-linear complementarity problems 5

The algorithm uses the relaxation parameter αk which influences the approximation of theinverse generalized Jacobian (B−1 + αk[I − B−1V (xk)]). One possibility is to take αk = α

during the whole process. The special case αk = α = 0 would lead to some kind of generalizedfixed Newton method. Although some numerical tests show reasonably good behaviour of thealgorithm with αk = α it is impossible to say anything definite about the choice of α in thegeneral case.

Another possibility considered in the smooth case [7] is to determine the parameter αk suchthat the new iteration becomes as close as possible in ‖ · ‖2 to the Newton iteration. This choiceof αk is called the optimal parameter because it is a solution of the optimization problem

αoptk := arg minα∈R‖L(xk + sk)‖2

2 (11)

with the MRV correction

sk = −B−1(I − αkH(xk))G(xk)

and the linear model

L(xk + d) = G(xk) + V (xk)d.

The solution of the minimization problem (2) is the optimal parameter

αoptk = 〈v, w + t〉

〈w + t, w + t〉 (12)

where

M = H(xk)B−1, v = MG(xk), w = H(xk)G(xk), t = Mw. (13)

In order to calculate this, two linear systems have to be solved, both with the same matrix B

Bv1 = G(xk)

Bt1 = w

followed by

v = H(xk)v1, t = H(xk)t1, w = H(xk)G(xk).

Let us now prove the local convergence of the MRV method with the optimal parameter forNCP. We shall consider it as a non-smooth inexact Newton method which is a generalizationof the classical IN method for the smooth case.

THEOREM 2 Suppose that the basic assumptions are satisfied and matrix B defined by (6) isnon-singular. There exist ε and δ such that if ‖x0 − x∗‖ ≤ ε and ‖A − F ′(x∗)‖ ≤ δ then thesequence {xk} generated by the MRV method with optimal parameter α

optk for NCP is well

defined and converges q-linearly to x∗.

Proof The mapping F is continuous, and so there exists ε1 > 0 such that for x ∈ N1(x∗) =

{x ∈ Rn: ‖x − x∗‖ ≤ ε1} and i = 1, 2, . . . , n we have

x∗i < Fi

(x∗) =⇒ xi < Fi (x)

x∗i > Fi

(x∗) =⇒ xi > Fi(x).

(14)

By the assumption, matrix B defined by (6) is non-singular for λ ∈ {0, 1}. For the chosenconstant λ ∈ {0, 1}, we associate V (x) with each x ∈ N1(x

∗) by (7) and the matrix V∗ is

6 N. Krejic et al.

associated with x∗ by (10), Therefore it follows that

[V (x) − V∗]i ∈ {0,[F ′ (x) − F ′ (x∗)]

i}. (15)

Since all V∗ ∈ ∂bG(x∗) are non-singular, lemma 1 implies that there exists M0 > 0 such that‖V −1∗ ‖ ≤ M0. We choose δ < 1/M0. For x0 ∈ N1(x

∗) implications (14) hold. Therefore fromthe definition of B and V∗ we have

[V∗ − B]i ∈ {0,[F ′ (x∗) − A

]i}.

Since ‖A − F ′(x∗)‖ ≤ δ it follows that

‖V∗ − B‖ ≤ δ. (16)

It follows from the perturbation lemma 17 and the inequality

‖V −1∗ (B − V∗)‖ ≤ ‖V −1

∗ ‖‖B − V∗‖ ≤ M0δ < 1

that there exists M1 > 0 such that

‖B−1‖ ≤ M1. (17)

Let δ > 0 be sufficiently small so that

√nM1(δ + δ) ≤ η < 1. (18)

Then there exists ε2 ∈ (0, ε1] such that for

x ∈ N2(x∗) = {x ∈ Rn: ‖x − x∗‖ ≤ ε2} ⊆ N1(x

∗)

‖V (x) − V∗‖ ≤ δ (19)

holds. Inequality (19) follows from (15) and the fact that F ′ and its components are continuous.We choose ε ≤ ε2 and suppose that ‖x0 − x∗‖ ≤ ε. We define the functions

s(x, α) = −B−1(I − αH(x))G(x)

and

p(x, α) = ‖G(x) + V (x)s(x, α)‖2

for

x ∈ N(x∗) = {x ∈ Rn : ‖x − x∗‖ ≤ ε} ⊆ N2(x∗) ⊆ N1(x

∗)

where V (x) is the matrix associated with x for the chosen λ. Since

αopt = argminα∈R‖G(x) + V (x)s(x, α)‖22

it follows that

p(x, αopt) ≤ p(x, 0).

Non-linear complementarity problems 7

By (16)–(19) we have

p(x, 0) = ‖G(x) + V (x)s(x, 0)‖2

≤ √n‖G(x) + V (x)s(x, 0)‖

≤ √n‖G(x) − V (x)B−1G(x)‖

≤ √n‖I − V (x)B−1‖‖G(x)‖

≤ √n‖B − V (x)‖‖B−1‖‖G(x)‖

≤ √n(‖B − V∗‖ + ‖V∗ − V (x)‖)‖B−1‖‖G(x)‖

≤ √n(δ + δ)M1‖G(x)‖

≤ η‖G(x)‖.Considering the MRV method for NCP as a non-smooth inexact Newton method with

‖rk‖ = p(xk, αoptk )

we have

‖rk‖ = p(xk, αoptk ) ≤ p(xk, 0) ≤ η‖G(xk)‖

for xk ∈ N(x∗), and so the statement follows from theorem 1. �

4. Numerical results

In this section we report the results obtained for two complementarity problems consideredin [13]. The convergence criteria are

||xk − x∗|| ≤ 10−4||xk|| + 10−4 and ||G(xk)|| ≤ 10−4.

Divergence is declared if ||G(xk)|| ≥ 102 and we declare slow convergence if the number ofiterations is >20. Both cases are marked with an asterisk in the tables. Q1

Problem 1 [13]Function F : R4 → R4 is given by

F1(x) = 3x21 + 2x1x2 + 2x2

2 + x3 + 3x4 − 6

F2(x) = 2x21 + x1 + x2

2 + 3x3 + 2x4 − 2

F3(x) = 3x21 + x1x2 + 2x2

2 + 2x3 + 3x4 − 1

F4(x) = x21 + 3x2

2 + 2x3 + 3x4 − 3

Since β(x∗) = ∅ this problem is non-degenerate with solution

x∗ = (√

6/2, 0, 0, 0.5), F (x∗) = (0, 2 + √6/2, 5, 0).

Using the starting approximations x0 and A = F ′(x0) we tested the generalized Newtonmethod (N), the MRV method for NCP with either optimal parameter (MRVopt) or constantparameter (MRVconst), and the generalized fixed Newton method (FN) (α = 0).

8 N. Krejic et al.

Table 1. Problem 1.

x0 N FN MRVconsta MRVopt

(1, 0, 0, 0) 3 7 (3, −0.13) 3(1, 0, 1, 0) 3 7 (3, −0.13) 3(1, 0, 0, 1) 3 7 (3, −0.13) 3(1, 0.2, 0.5, 1) 4 16 (16, −0.2) 7(1, 0, 1, −1) 4 19 (17, −0.1) 4(1.5, −0.5, 4.5, −1) 4 ∗ (10, −0.2) 6(1.1, −0.1, 3.1, −0.1) 4 16 (16, −0.2) 7(0.85, 0.2, 0.5, 1) 4 ∗ (17, −0.1) 9

∗The first number is the number of iterations and the second is the value ofparameter α.

Problem 2 [13] Function F : R4 → R4 is given by

F1(x) = 3x21 + 2x1x2 + 2x2

2 + x3 + 3x4 − 6

F2(x) = 2x21 + x1 + x2

2 + 10x3 + 2x4 − 2

F3(x) = 3x21 + x1x2 + 2x2

2 + 2x3 + 9x4 − 9

F4(x) = x21 + 3x2

2 + 2x3 + 3x4 − 3.

This is a degenerate complementarity problem with the following two solutions

x∗D = (

√6/2, 0, 0, 0.5), F (x∗

D) = (0, 2 + √6/2, 0, 0)T

and

x∗ND = (1, 0, 3, 0), F (x∗

ND) = (0, 31, 0, 4).

Since β(x∗D) = {3}, x∗

D is a degenerate solution.It is easy to check that V (x∗

D) and V (x∗ND) are non-singular.

Tables 1 and 2 show that the new method with the optimal parameter behaves quite wellcompared with the generalized Newton method. As expected, the MRV method for NCP withthe constant parameter performed worse than the method with the optimal parameter, althoughits real potential is yet to be tested on problems of larger dimensions. The generalized fixedNewton method (MRV method with α = 0) performed worst in these two examples, thusjustifying the proposed modification of the right-hand side vector.

Table 2. Problem 2.

x0 Na FN MRVconstb MRVopt

(1, 0, 0, 0) (3, x∗D) (4, −0.1, x∗

D) (3, x∗D)

(1, 0, 1, 0) (3, x∗D) (4, −0.1, x∗

D) (3, x∗D)

(1, 0, 0, 1) (3, x∗D) (3, −0.13, x∗

D) (3, x∗D)

(1, 0.2, 0.5, 1) (4, x∗D) (8, −0.15, x∗

ND) (5, x∗ND)

(1, 0, 1, −1) (3, x∗D) (4, −0.1, x∗

D) (3, x∗D)

(1.5, −0.5, 4.5, −1) (3, x∗ND) (7, x∗

ND) (5, −0.1, x∗ND) (3, x∗

ND)

(1.1, −0.1, 3.1, −0.1) (3, x∗ND) (4, x∗

ND) (4, −0.15, x∗ND) (3, x∗

ND)(0.85, 0.2, 0.5, 1) (4, x∗

D) (12, x∗ND) (9, −0.1, x∗

ND) (7, x∗ND)

aThe number is the number of iterations and the symbol is the solution to which the methodconverges.bThe first number is the number of iterations, the second number is the value of parameter α andthe symbol is the solution to which the method converges.

Non-linear complementarity problems 9

5. Final remarks

We have presented one generalization of a Newton-like method with a modification of theright-hand side vector for NCPs. The method is derived using the semi-smooth reformula-tion with the classical Minty function and the generalized Jacobian from ∂b. Local q-linearconvergence is proved in the infinity norm.

The main idea of this method in the smooth case is to modify the right-hand side vector inthe chord equation so that the fixed Newton method (or chord method) becomes more like theNewton method. We have shown how to apply the same idea to non-smooth systems of non-linear equations arising from NCPs. The relaxation parameter and the generalized Jacobian atcurrent iteration have been used to modify the right-hand side vector.

The behaviour of the new method is influenced by the relaxation parameter. The simplestcase with a constant parameter is one possibility, and another is the optimal parameter. Theparameter analysed here in detail is the one that makes the iterations as similar as possible tothe generalized Newton iterations.

Finally, we performed some numerical experiments. The test examples are dense systems ofsmall dimensions. It is shown that in these cases we obtain fairly good results. Further inves-tigation is planned in order to obtain more reliable numerical results and broader comparisonwith other methods in the generalized Newton-like family.

References[1] De Luca, T., Facchinei, F. and Kanzow, C., 2000, A theoretical and numerical comparison of some semismooth

algorithms for complementarity problems. Computational Optimization and Applications, 16, 173–205.[2] Lopes, V.L.R., Martinez, J.M. and Perez, R., 1999, On the local convergence of quasi-Newton methods for

nonlinear complementarity problems. Applied Numerical Mathematics, 30, 3–22.[3] Pieraccini, S., Gasparo, M.G. and Pasquali, A., 2003, Global Newton-type methods and semismooth

reformulations for NCP. Applied Numerical Mathematics, 44, 367–84.[4] Qi, L. andYang,Y., 2002, NCP functions applied to Lagrangian globalization for the nonlinear complementarity

problem. Journal of Global Optimization, 24, 261–83.[5] De Luca, T., Facchinei, F. and Kanzow, C., 1996, A semismooth equation approach to the solution of nonlinear

complementarity problems. Mathematical Programming, 75, 407–39.[6] Dingguo, P. and Weiwen, T., 2002, Globally convergent inexact generalized Newton’s methods for nonsmooth

equations. Journal of Computational and Applied Mathematics, 138, 37–49.[7] Krejic, N. and Luzanin, Z., 2002, Newton-like method with modification of the right-hand-side vector.

Mathematics of Computation, 71, 237–50.[8] Pang, J.S. and Qi, L., 1993, Nonsmooth equations: motivation and algorithms. SIAM Journal on Optimization,

3, 443–65.[9] Clarke, F.H., 1990, Optimization and Nonsmooth Analysis (Philadelphia, PA: SIAM).

[10] Qi, L., 1993, Convergence analysis of some algorithms for solving nonsmooth equations. Mathematics ofOperations Research, 18, 227–44.

[11] Qi, L., 1996, C-differentiability, C-differential operators and generalized Newton methods. Technical Report,School of Mathematics, University of New South Wales, Sydney, Australia.

[12] Qi, L. and Sun, J., 1993, A nonsmooth version of Newton’s method. Mathematical Programming, 58, 353–67.[13] Sun, D. and Han, J., 1997, Newton and quasi-Newton methods for a class of nonsmooth equations and related

problems. SIAM Journal on Optimization, 7, 463–80.[14] Facchieni, F. and Kanzow, C., 1997, A nonsmooth inexact Newton method for the solution of large-scale

nonlinear complementarity problems. Mathematical Programming, 76, 493–512.[15] Martinez, J.M. and Qi, L., 1995, Inexact Newton methods for solving nonsmooth equations. Journal of

Computational and Applied Mathematics, 60, 127–45.[16] Pang, J.S. and Gabriel, S.A., 1993, NE/SQP: a robust algorithm for nonlinear complementarity problem.

Mathematical Programming, 60, 295–337.[17] Dennis, J.E., Jr and Schnabel, R.B., 1983, Numerical Methods for Unconstrained Optimization and Nonlinear

Equations (Englewood Cliffs, NJ: Prentice-Hall).

Journal …International Journal of Computer Mathematics

Article ID …GCOM116833

TO: CORRESPONDING AUTHOR

AUTHOR QUERIES - TO BE ANSWERED BY THE AUTHOR

The following queries have arisen during the typesetting of your manuscript. Please answer

the queries.

This article has been subject to some rewriting. Please check carefully to ensure that the meaning has not been changed

Reference numbering has been altered so that references are cited in numerical order. Please check carefully

Please check that journal titles have been correctly expanded

Q1 Section 4, para. 1 Please confirm rewording of last sentence: "Both cases are marked with an asterisk in the tables."

Production Editorial Department, Taylor & Francis Ltd.

4 Park Square, Milton Park, Abingdon OX14 4RN

Telephone: +44 (0) 1235 828600 Facsimile: +44 (0) 1235 829000