A long step barrier method forconvex quadratic programming

Report 90-53

K.M. AnstreicherD. den Hertog

C. RoosT. Terlaky

Technische Universiteit DelftDelft University of Technology

Faculteit der Technische Wiskunde en InformaticaFaculty of Technical Mathematics and Informatics

ISSN 0922-5641

Copyright c 1990 by the Faculty of Technical Mathematics and Informatics, Delft, TheNetherlands.No part of this Journal may be reproduced in any form, by print, photoprint, microfilm,or any other means without permission from the Faculty of Technical Mathematics andInformatics, Delft University of Technology, The Netherlands.

Copies of these reports may be obtained from the bureau of the Faculty of TechnicalMathematics and Informatics, Julianalaan 132, 2628 BL Delft, phone+3115784568.A selection of these reports is available in PostScript form at the Faculty’s anonymousftp-site. They are located in the directory /pub/publications/tech-reports at ftp.twi.tudelft.nl

DELFT UNIVERSITY OF TECHNOLOGYREPORT 90{53A LONG STEP BARRIER METHOD FORCONVEX QUADRATIC PROGRAMMINGK. M. Anstreicher, D. den Hertog,C. Roos, T. Terlaky

ISSN 0922{5641Reports of the Faculty of Technical Mathematics and Informatics 90{53Delft 1990i

K.M. Anstreicher:Department of Operations Research,Yale University,84 Trumbull St.,New Haven, CT 06520,U.S.A.D. den Hertog, C. Roos and T. Terlaky,Faculty of Technical Mathematics and Informatics, Delft University of Technology, P.O. Box 5031,2600 GA Delft, The Netherlands.This work is completed with the support of a research grant from SHELL.The fourth author is on leave from the E�otv�os University, Budapest, and partially supported byOTKA No. 2116.Copyright c 1990 by Faculty of Technical Mathematics and Infor-matics, Delft, The Netherlands.No part of this Journal may be reproduced in any form, by print,photoprint, micro�lm or any other means without written permis-sion from Faculty of Technical Mathematics and Informatics, DelftUniversity of Technology, The Netherlands. 1

AbstractIn this paper we propose a long{step logarithmic barrier function method for convex quadraticprogramming with linear equality constraints. After a reduction of the barrier parameter, a seriesof long steps along projected Newton directions are taken until the iterate is in the vicinity of thecenter associated with the current value of the barrier parameter. We prove that the total numberof iterations is O(pnL) or O(nL), dependent on how the barrier parameter is updated.Key Words: convex quadratic programming, interior point method, logarithmic barrier function,polynomial algorithm.

1 IntroductionKarmarkar's [14] invention of the projective method for linear programming has given rise to activeresearch in interior point algorithms. At this moment, the variants can roughly be categorized intofour classes: projective, a�ne scaling, path{following and potential reduction methods.Researchers have also extended interior point methods to other problems, including convex quadraticprogramming (QP). Kapoor and Vaidya [12], [13] and Ye and Tse [31] extended Karmarkar's pro-jective method for QP. They proved that their method requires O(nL) iterations, where L is thesize of the problem. Dikin [5] already proved convergence for the a�ne scaling method applied toQP. Ye [29] further analysed this method.Monteiro and Adler [21] developed a primal{dual small{step path{following method for QP. Ye[28], Ben Daya and Shetty [1], and Goldfarb and Liu [8] proposed small{step logarithmic barriermethods for QP. All these methods require O(pnL) iterations. Renegar and Shub [26] simpli�edand uni�ed the complexity analysis for these methods. Monteiro et al. [22] proposed a small{steppath{following method for QP in which only the primal{dual a�ne scaling direction is used.Nesterov and Nemirovsky [24] proposed small{step barrier methods for smooth convex program-ming problems. In Chapter 5 of their monograph they also work out long{step barrier methodsfor convex quadratic programming problems with linear inequality constraints. Jarre [10], [11]and Mehrota and Sun [18], [19] analysed a small{step path{following method for quadraticallyconstrained QP and smooth convex problems, based on Huard's method of centers. Den Hertog etal. [2] extended this method to a long{step method. In [3] they also proposed a long{step methodfor smooth convex problems, based on the logarithmic barrier function approach.It is well{known that QP can be formulated as a Linear Complementarity Problem (LCP). So,methods for solving LCP's are indirect methods for QP. However, the transformation from QP toLCP may make these methods ine�cient for QP, due to the increase in the size of the matricesinvolved. Many interior point methods for solving LCP have appeared in the literature: e.g. [16],[17], [20], [30]. Kojima et al. [15] gave a uni�ed approach for interior point methods for LCP.In this paper we propose a long{step logarithmic barrier method for QP with linear equalityconstraints. For a suitable proximity criterion, we show that the Newton process quadraticallyconverges, which makes it possible to simplify the analysis signi�cantly compared with [2], [3]and [24]. It may be noted that the latter papers deal with inequality constraints, and hencecannot be applied directly to our problem with equality constraints. Moreover, it will appear thatthe iteration bound derived in this paper is much better than the one obtained by Nesterov andNemirovsky [24].Our method is an extension of Den Hertog et al.'s [4] method for linear programming. Steps alongprojected Newton directions are taken, until the iterate is su�ciently close to the current center.After that, the barrier parameter is reduced and the process is repeated. We will use the null{spaceformulation for the projected Newton direction. It is shown that the total number of iterations isO(pnL) or O(nL), dependent on how the barrier parameter is updated.The paper is organized as follows. In Section 2 we prove some properties of the central path. InSection 3 proximity criterions are introduced, and some properties of nearly centered points areproved. The quadratic convergence property will appear to be a good expedient for proving theother properties. Then, in Section 4 we describe our algorithm and in Section 5 we will derive anupper bound for the total number of iterations. Finally, in Section 6 we end up with some remarks.1

Notation. As far as notations are concerned, e shall denote the vector of all ones and I theidentity matrix. Given an n-dimensional vector x we denote by X the n�n diagonal matrix whosediagonal entries are the coordinates xj of x; xT is the transpose of the vector x and the samenotation holds for matrices. Finally kxk denotes the l2 norm.2 Properties of the central pathWe consider the convex quadratic programming problem in the following standard form:(QP ) 8><>: min q(x) = cTx+ 12xTQxAx = b; x � 0:Here Q is a symmetric, positive semi{de�nite n�n matrix, A is an m�n matrix, b and c are m�and n� dimensional vectors respectively; the n�dimensional vector x is the variable in which theminimization is done. The dual formulation to (QP ) is:(QD) 8><>: maxd(x; y) = bTy � 12xTQxATy � Qx+ s = c; s � 0;where y is an m�dimensional vector.It is well{known that for all x and y that are feasible for (QP ) and (QD) we haved(x; y) � z� � q(x);where z� denotes the optimal objective value for (QP ). Optimality holds if and only if the com-plementary slackness relation xTs = 0 is satis�ed.Without loss of generality we assume that all the coe�cients are integer. We shall denote by Lthe length of the input data of (QP ).We make the standard assumption that the feasible set of (QP ) is bounded and has a nonemptyrelative interior. In order to simplify the analysis we shall also assume that A has full rank, thoughthis assumption is not essential.We consider the logarithmic barrier functionf(x; �) := cTx+ 12xTQx� � nXj=1 ln xj ; (1)where � is a positive parameter. The �rst and second order derivatives of f arerf(x; �) = c+ Qx� �X�1e;r2f(x; �) = 1�Q+X�2:2

Consequently, f is strictly convex on the relative interior of the feasible set. It also takes in�nitevalues on the boundary of the feasible set. Thus it achieves a minimum value at a unique point.The necessary and su�cient �rst order optimality conditions for this point are:AT y �Qx+ s = c; s � 0; (2)Ax = b; x � 0; (3)Xs = �e; (4)where y and s are m� and n�dimensional vectors respectively.Let us denote the unique solution of this system by (x(�); y(�); s(�)). Then the primal and dualcentral path is de�ned as the solution set x(�) and y(�) respectively, for � > 0. It is easy to seethat the duality gap in (x(�); y(�); s(�)) satis�esx(�)T s(�) = n�: (5)Hence, if � ! 0 then x(�) and (x(�); y(�)) will converge to optimal primal and dual solutionsrespectively.The following lemma states that the primal objective decreases along the primal path and thedual objective increases along the dual path. The �rst part is a classical result of Fiacco andMcCormick [6]. The second part also follows from this result, since (x(�); y(�)) is the minimizer ofthe (convex) dual logarithmic barrier function. Our proof is completely di�erent from the classicalone.Lemma 1 The objective q(x(�)) of the primal problem (QP ) is monotonically decreasing and theobjective d(x(�); y(�)) of the dual problem (QD) is monotonically increasing if � decreases.Proof:To prove the �rst part of the lemma it su�ces to show thatq(x(�))0 = cTx0 + xTQx0 � 0; (6)where the prime denotes the derivative with respect to �. Using that x(�) and y(�) satisfy (2){(4)and taking derivatives with respect to � we obtainATy0 �Qx0 + s0 = 0; (7)Ax0 = 0; (8)Xs0 + Sx0 = e: (9)Now, using (2) and (8), we havecTx0 = (s�Qx+ AT y)Tx0 = sTx0 � xTQx0:Using (9), (4), (7) and (8) respectively, this results intocTx0 + xTQx0 = sTx0 = eTSx0 = (Sx0 +Xs0)TSx0= (x0)TS2x0 + �(x0)T s0= (x0)TS2x0 + �(x0)TQx0 � 0:3

Thus, the �rst part of the lemma follows because of (6).To prove the second part of the lemma it su�ces to show thatd(x(�); y(�))0 = bTy0 � (x0)TQx � 0: (10)Multiplying (9) by AS�1 we obtainAS�1Xs0 +Ax0 = AS�1e;which reduces to AX2s0 = b. Now, taking the inner product with y0 results intobTy0 = (ATy0)TX2s0= (x0)TQX2s0 � (s0)TX2s0= (x0)TQX(e� Sx0)� (s0)TX2s0:Consequently, we have bTy0 � (x0)TQx = ��(x0)TQx0 � (s0)TX2s0 � 0:Together with (10), this proves the second part of the lemma. 23 Properties near the central pathTo measure the distance to the central path of non{centered points, we will introduce three mea-sures. The �rst is analogous to Roos and Vial's [27] appealing measure for linear programming�(x; �) := miny kX� (c+Qx� ATy)� ek: (11)Loosely speaking, �(x; �) measures the deviation from optimality condition (4). The unique so-lution of the minimization problem in the de�nition of �(x; �) is denoted by y(x; �) and thecorresponding slack variable by s(x; �) (i.e. s(x; �) = c+Qx�AT y(x; �)). It can easily be veri�edthat x = x(�)() �(x; �) = 0() y(x; �) = y(�):In the sequel of this paper we will deal with the scaled version of (QP ):(QP ) 8><>: min q(x) = cTx+ 12xTQxAx = b; x � 0;where A = AX , c = Xc, Q = XQX , and x is the current iterate. Hence, a step starting at e in(QP ) corresponds to a step starting at x in (QP ). The scaled versions of rf(x; �) and r2f(x; �)will be denoted by g = g(x; �) and H = H(x; �) respectively.In the algorithm, described in the following section, we will do steps along projected Newtondirections with respect to f . Gill et al. [7] give two alternative (equivalent) forms for this direction:4

� the range{space form p(x; �) = �H�1(I � AT (AH�1AT )�1AH�1)g; (12)� the null{space form p(x; �) = �Z(ZTHZ)�1ZT g; (13)where Z is an n� (n �m) matrix, with independent columns, such that AZ = 0.The second and third measure for the distance to the central path are kp(x; �)k and kp(x; �)kH(x;�),where the latter is de�ned by kp(x; �)k2H(x;�) = p(x; �)TH(x; �)p(x; �). We note that becauseH(x; �) is positive de�nite, k:kH(x;�) de�nes a norm.In the sequel of the paper we will sometimes write � and p instead of �(x; �) and p(x; �) forbriefness' sake. We remark that although all of the three measures will be used in the analysis,only kpkH will be used in the algorithm.We will work with the null{space form for p, because it facilitates the analysis very much. In theanalysis we will also assume that ZTZ = I , hence Z is orthonormal. In this case we have thefollowing well{known properties:Property 1. ZZT is the projection onto the null{space of A;Property 2. kZ�k = k�k for any �;Property 3. kZTxk � kxk for any x, with equality if x is in the null{space of A.The following lemma shows that there is a close relationship between the three measures.Lemma 2 For given x and �, kpk2 � kpk2H = �pTg � �2.Proof:Using the null{space form (13) it follows that ZZT g = �ZZTHZZTp. Property 1 and the factthat Ap = 0 imply ZZT p = p. Consequently, we may write�pTg = �pTZZT g = (ZZT p)THZZTp = pTHp = pT (I + 1�Q)p � kpk2:This proves the �rst inequality of the lemma.Using the de�nition of p it follows that�gTp = gTZ �I + 1�ZTQZ��1 ZT g� kZT gk2 k�I + 1�ZTQZ��1 k� kZT gk2;5

where the last inequality follows because the eigenvalues of �I + 1�ZTQZ��1 are all less than orequal to 1. Moreover, ZT g = ZT c+Qe� � e!= ZT c+Qe� ATy� � e! ;where the last equality holds for any y, because AZ = 0. Putting y equal to y(x; �), it followsfrom Property 3 that kZT gk � �. Hence it follows that �gTp � �2. This proves the lemma. 2Now we will prove some fundamental lemmas for nearly centered points. The following lemmashows quadratic convergence in the vicinity of the central path for all of the three measures.Lemma 3 If kp(x; �)k < 1 then x� = x+Xp(x; �) is a strictly feasible point for (QP ). Moreover,kp(x�; �)k � kp(x�; �)kH(x�;�) � �(x�; �) � kp(x; �)k2 � kp(x; �)k2H(x;�) � �(x; �)2:Proof:It is easy to see that Ax� = b. Moreover, x� = X(e + p) > 0, because kpk < 1. This proves the�rst part of the lemma.Using the de�nition of �(x�; �) we have�(x�; �) = miny kX(I + P )� �c+ QX(e+ p)�AT y�� ek= miny k(I + P )� �c+Q(e+ p)� ATy�� ek; (14)where �p = Z(I + 1�ZTQZ)�1ZT c+ Qe� � e! :So Q(e+ p) = Qe+ QZ(I + 1�ZTQZ)�1ZT e � c+Qe� ! :Next consider the projection of Q(e+ p) onto the null-space of A (using Property 1):ZZTQ(e+ p) = ZZTQe+ ZZTQZ(I + 1�ZTQZ)�1ZT e � c+Qe� != Z "ZTQe+ ��(I + 1�ZTQZ)� �I� (I + 1�ZTQZ)�1ZT e� c+ Qe� !#= Z "ZTQe+ �ZT e� ZTQe � ZT c� �(I + 1�ZTQZ)�1ZT e� c+ Qe� !#= ZZT (�e� c)� �p: 6

Note that there are certainly m{dimensional vectors u and v such thatQ(e+ p) = ZZTQ(e+ p) +ATu;and (�e � c) = ZZT (�e� c) + ATv:Setting y = u� v we then obtainQ(e+ p)� ATy = (�e� c)� �p: (15)Using this y in (14), it follows that�(x�; �) � k(I + P )(e� p)� ek = kPpk � kpk2:This proves the middle inequality of the lemma. The rest follows immediately from Lemma 2. 2Lemma 4 If kpkH < 1 then f(x; �)� f(x(�); �) � kpk2H1� kpk2H :Proof:The barrier function f is convex for �xed �, whencef(x; �)� f(x+Xp; �) � �gTp = kpk2H ; (16)where the equality follows from Lemma 2. Now, let x0 := x and let x0; x1; x2; � � � denote thesequence of points obtained by repeating Newton steps, starting at x0. Then we may write, usingLemma 3 f(x; �)� f(x(�); �) = 1Xi=0 �f(xi; �)� f(xi+1; �)�� 1Xi=0 kpk2i+1H� kpk2H1� kpk2H : 2Lemma 5 If kpkH < 1; thenjq(x)� q(x(�))j � kpkH(1 + kpkH)1� kpkH �pn:7

Proof:Since q(x) is convex, we haverq(x)TXp � q(x+Xp)� q(x) � rq(x+Xp)TXp: (17)For the left{hand side expression we can derive the following lower bound, using that rq(x) =c+Qx = �X�1g + �X�1erq(x)TXp = �gTp+ �eT p � ��kpk2H � �kpkHpn; (18)where the last inequality follows from Lemma 2. Now, using (15), we derive an upper bound forthe right{hand side expression in (17):rq(x+Xp)TXp = cTp+ pTQ(e+ p) = cT p+ pT (AT y + �e � c� �p)= �eT p� �kpk � �eT p � �kpkHpn: (19)Consequently, substitution of (18) and (19) into (17) yieldsjq(x)� q(x+Xp)j � kpkH(1 + kpkH)�pn:Again, let x0 := x and let x0; x1; x2; � � � denote the sequence of points obtained by repeating Newtonsteps, starting at x0. Then we may write, using Lemma 3jq(x)� q(x(�))j = ����� 1Xi=0 �q(xi)� q(xi+1)������� 1Xi=0 ���q(xi)� q(xi+1)���� 1Xi=0 kpk2Hi(1 + kpk2Hi)�pn� (1 + kpkH)�pn 1Xi=0 kpk2Hi� kpkH(1 + kpkH)1� kpkH �pn: 24 The AlgorithmIn our long{step algorithm a linesearch along the Newton direction is done until the iterate issu�ciently close to the current center. After that, the barrier parameter is reduced, and theprocess starts again.In the next section we will show that a linesearch along the Newton direction reduces the barrierfunction value by a constant if kpkH � 12 . This enables us to derive an upper bound for the numberof steps between reductions of �. 8

If kpkH < 12 we can give bounds on f(x; �)� f(x(�); �), jq(x)� q(x(�))j and cTx� z� by Lemmas4 and 5.AlgorithmInput:�0 is the initial barrier value, �0 � 2O(L);t is an accuracy parameter, t = O(L);� is the reduction parameter, 0 < � < 1;x0 is a given interior feasible point such that kp(x0; �0)kH(x0;�0) � 12 ;beginx := x0; � := �0;while � > 2�t dobegin (outer step)while kpkH � 12 dobegin (inner step)~� := argmin�>0 ff(x+ �Xp; �) : x+ �Xp > 0gx := x+ ~�Xpend (inner step)� := (1� �)�;end (outer step)end.For �nding the initial point that satis�es the input assumptions of the algorithm we refer thereader to e.g. Ye [28]. Later on the centering assumption kp(x0; �0)kH(x0;�0) � 12 will be relaxed.5 Convergence analysis of the AlgorithmIn this section we will derive upper bounds for the total number of outer and inner iterations.Theorem 1 After at most K = O(L� ) outer iterations, the algorithm ends up with a primalsolution such that q(x)� z� � 2�O(L).Proof:We can derive an upper bound for the gap q(x)� z� after K outer iterations from (5) and Lemma5: q(x)� z� = q(x(�K))� z� + q(x)� q(x(�K)) � �K(n+ 32pn);where �K = (1� �)K�0. This means that q(x)� z� � 2�O(L) certainly holds if(1� �)K�0(n+ 32pn) � 2�O(L):Taking logarithms we require K � O(L) + ln(n+ 32pn) + ln �0� ln(1� �) :9

Since we have assumed that �0 � 2O(L), and since � � � ln(1��), this certainly holds if K = O(L� ).2We note, that this �nal primal solution can be rounded to an optimal solution for (QP ) in O(n3)arithmetic operations. (See e.g. [25].)The following lemma is needed to derive an upper bound for the number of inner iterations ineach outer iteration. It states that a su�cient decrease in the value of the barrier function can beobtained by taking a step along the Newton direction.Lemma 6 Let � := (1 + kpkH)�1. Then�f := f(x; �)� f(x+ �Xp; �) � kpkH � ln(1 + kpkH):Proof:We write down the Taylor expansion for f :f(x+ �Xp; �) = f(x; �) + �gTp+ 12�2pTHp+ nXk=3 tk ;where tk denotes the k�order term in the Taylor expansion.Since tk = (��)kk nXi=1 pki ;we �nd, by using Lemma 2,jtk j � �kk nXi=1 jpijk � �kk nXi=1 jpij2! k2 � �kk kpkk � �kk kpkkH :Using Lemma 2, we have for the linear and quadratic term in the Taylor series�pTg + 12�2pTHp = (12�2 � �)kpk2H :So we �nd f(x+ �Xp; �) � f(x; �) + (12�2 � �)kpk2H + nXk=3 �kk kpkkH� f(x; �)� �kpk2H � ln(1� �kpkH)� �kpkH :Hence �f � �(kpk2H + kpkH) + ln(1� �kpkH): (20)The right hand side is maximal if � = � = (1 + kpkH)�1. Substitution of this value �nally gives�f � kpkH � ln(1 + kpkH):This proves the lemma. 210

Theorem 2 Each outer iteration requires at mostN = 11�1� � ��n + 3pn�+ 113inner iterations.Proof:This proof is a generalization of Gonzaga's [9] proof for the linear case. Let us consider the (k+1)'stouter iteration. The starting point is then (xk; �k), with kp(xk; �k�1)kH(xk;�k�1) < 12 . LetN denotethe number of inner iterations. For each inner iteration we know, according to Lemma 6, that thedecrease in the potential function value is at least�f � 12 � ln(1 + 12) > 111 :Following the N inner iterations, we have xk+1 with kp(xk+1; �k)kH(xk+1;�k) < 12 . So, we havef(xk+1; �k) � f(xk; �k)� 111N:Equivalently, 111N � f(xk; �k)� f(xk+1; �k): (21)Now we will derive an upper for the right hand side. The de�nition of f(x; �); x > 0 implies thatf(x; �k) = f(x; �k�1) + q(x)�k � q(x)�k�1= f(x; �k�1) + q(x)�k�1 � 11� � � 1�= f(x; �k�1) + �1� � q(x)�k�1 :Using this we obtainf(xk; �k)� f(xk+1; �k) = f(xk; �k�1)� f(xk+1; �k�1) + �1� � 1�k�1 (q(xk)� q(xk+1)): (22)Because xk and xk+1 are approximately centered with respect to x(�k�1) and x(�k) respectively,using Lemma 5 for the �rst and Lemma 1 for the second inequality we �ndq(xk)� q(xk+1) � q(x(�k�1)) + 32�k�1pn� q(x(�k)) + 32�kpn= q(x(�k�1))� q(x(�k)) + 32(2� �)�k�1pn� (q(x(�k�1))� d(x(�k�1); y(�k�1)))� (q(x(�k))� d(x(�k); y(�k))) + 3�k�1pn= �k�1n� �kn+ 3�k�1pn= �k�1 ��n + 3pn� :11

Secondly, using Lemma 4 (with kpkH = 12), and the fact that x(�k�1) minimizes f(x(�k�1); �k�1),we obtain f(xk ; �k�1)� f(xk+1; �k�1) = f(xk; �k�1)� f(x(�k�1); �k�1)+f(x(�k�1); �k�1)� f(xk+1; �k�1)� f(xk; �k�1)� f(x(�k�1); �k�1)� 13 :Hence, substitution of the last two inequalities into (22) yieldsf(xk ; �k)� f(xk+1; �k) � �1� � ��n + 3pn�+ 13 :Substitution of this inequality into (21) yields the lemma. 2Combining Theorem 1 and 2, the total number of iterations turns out to be given by� 111� � ��n + 3pn�+ 113� �O(L): (23)This makes clear that� if we take � = O( 1pn ) then O(pnL) iterations are needed;� if we take � = O(1) then O(nL) iterations are needed.6 Concluding remarks6.1 Computing the Newton directionEven though our analysis is based on the null{space form for the Newton direction, in practiceeither the null{space or the row{space form can be used. It is obvious that the null{space form ismore e�cient when the number of linear constraints is relatively large compared to the number ofvariables. The row{space form is e�cient when the number of linear constraints is small comparedto the number of variables.In our analysis we assumed that Z is orthonormal. However, the search direction doesn't changeif Z is any basis for the nullspace of A. So, in practice we don't have to do all the work to �nd anorthonormal Z on each iteration. For example, if a Z is found such that AZ = 0, then Z = X�1Zsatis�es AXZ = 0. We refer the reader to Gill et al. [7] for the numerical aspects.6.2 Obtaining dual feasible solutionsAt the end of each sequence of inner iterations we have a primal feasible x such that kpkH � 1. Thefollowing lemma shows that a dual feasible solution can be obtained by performing an additionalfull Newton step, and projection. 12

Lemma 7 Let x� = x+Xp(x; �). If kp(x; �)kH(x;�) � 1 then � := �(x�; �) � 1 and y := y(x�; �)is dual feasible. Moreover, the duality gap satis�es�(n � �pn) � q(x�)� d(x�; y) � �(n + �pn):Proof:By Lemma 3 we have �(x�; �) � kp(x; �)k2H(x;�) � 1. By the de�nition of s(x; �) = c + Qx �AT y(x; �) we have �(x�; �) = kX�s(x�; �)� � ek � 1:This implies s(x�; �) � 0, so y(x�; �) is dual feasible. Moreover,�����(x�)Ts(x�; �)� � n����� = ����eT (X�s(x; �)� � e)���� � kekkX�s(x�; �)� � ek = �pn:Consequently, using that (x�)T s(x�; �) = q(x�)� d(x�; y)�(n � �pn) � q(x�)� d(x�; y) � �(n + �pn): 26.3 Small{step path{following methodsSmall{step path{following methods start at a nearly centered iterate and after the parameter isreduced by a small factor, a unit Newton step is taken. The reduction parameter is su�cientlysmall, such that the new iterate is again nearly centered with respect to the the new center. Small{step barrier methods for convex quadratic programming have been given by Ye [28], Goldfarb andLiu [8], and Ben Daya and Shetty [1]. The following lemma shows that if � is small, then we obtainsuch a small{step path{following method.Lemma 8 Let x� := x + Xp(x; �) and �� := (1 � �)�, where � = 110pn . If �(x; �) � 12 then�(x�; ��) � 12.Proof:Due to the de�nition of our measure we have�(x; ��) = kXs(x; ��)�� � ek� kXs(x; �)�� � ek= k 11� � (Xs(x; �)� � e) + ( 11� � � 1)ek� 11� � (�(x; �) + �pn)� 11� 110 (12 + 110)= 23 : 13

Now we can apply the quadratic convergence result (Lemma 3)�(x�; ��) � �(x; ��)2 � 49 < 12 : 26.4 Relaxing the initial centering conditionThe initial centering condition kp(x0; �0)kH(x0;�0) � 12 can be relaxed tof(x0; �0)� f(x(�0); �0) � O(pnL)if � = O( 1pnL) is used, and to f(x0; �0)� f(x(�0); �0) � O(nL)if � = O(1) is used. This holds because of Lemma 6. For the last case the assumption is equivalentwith the assumption x0j � 2�L for all j. This can easily be veri�ed. Since x(�0) is primal feasible,it can be written as a convex combination of basic feasible solutions. The coordinates xj of eachbasic feasible solution satisfy xj � 2L. Moreover, q(x0)� q(x(�0)) � 2O(L). Thereforef(x0; �0)� f(x(�0); �0) = q(x0)� q(x(�0))�0 � nXj=1 ln x0j + nXj=1 ln xj(�0) � O(nL):6.5 Results for the LP caseIt is worthwile to look at the results for the LP case, for which Q = 0. In this case the projectedNewton direction (13) reduces to p = �ZZT g, which coincides with the scaled projected gradientdirection. It is easy to verify that the three measures kpk, kpkH and � are exactly the same. (i.e.in Lemma 2 equalities hold instead of inequalities). Hence, the resulting algorithm and results arethe same as in [4].6.6 Barrier methods for LCPIt is well{known (see e.g. Murty [23]) that the Linear Complementarity Problem (LCP)(LCP ) 8><>: y =Mx+ q; x; y � 0xTy = 0;is completely equivalent with the following quadratic programming problem(QP ) 8><>: min xT yy =Mx+ q; x; y � 0:IfM is positive semi{de�nite, then this problem is a equivalent to a convex quadratic programmingproblem, since xTy = 12xTQx+ qTx, where Q =M +MT . Consequently, the algorithm proposedin this paper can also be applied to positive semi{de�nite LCP's.14

6.7 Comparison with Nesterov and Nemirovsky's resultsIn Chapter 5 of their monograph [24], Nesterov and Nemirovsky analysed long{step barrier methodsfor QP with linear inequality constraints. Their analysis is totally di�erent from ours: it is notbased on changes in the barrier function value, for example. On one hand their analysis is moregeneral, but on the other hand it is also very complicated. From their analysis it can be extractedthat the total number of iterations is at most O((1� + n4�7)L lnn). Note that the iteration bound(23) is better than this one. Our bound is much better if we deal with real long{step algorithms(i.e. � is large). For example:� If we take � = O( 1pn) then Nesterov and Nemirovsky require O(pnL lnn) iterations. Thedi�erence with our O(pnL) iteration bound is not so signi�cant in this case.� If we take � = O(1) then Nesterov and Nemirovsky require O(n4L lnn) iterations. Thisbound is much worse than our O(nL) iteration bound.References[1] Ben Daya, M., Shetty, C.M. (1988), Polynomial Barrier Function Algorithms for ConvexQuadratic Programming, Research Report, No. J 88{5, School of Industrial and SystemsEngineering, Georgia Institute of Technology, Atlanta.[2] Den Hertog, D., Roos, C., Terlaky, T. (1990), A Potential Reduction Method for a Classof Smooth Convex Programming Problems, Report No. 90{01, Faculty of Mathematics andInformatics/Computer Science, Delft University of Technology, Delft, Holland.[3] Den Hertog, D., Roos, C., Terlaky, T. (1990), On the Classical Logarithmic Barrier Method fora Class of Smooth Convex Programming Problems, Report No. 90{28, Faculty of Mathematicsand Informatics/Computer Science, Delft University of Technology, Delft, Holland.[4] Den Hertog, D., Roos, C., Vial, J.-Ph. (1990), An O(pn) Complexity Reduction for LongStep Path{following Methods, Revised version of Report No. 89{85, Faculty of Mathematicsand Informatics/Computer Science, Delft University of Technology, Delft, Holland.[5] Dikin, I.I. (1967), Iterative Solution of Problems of Linear and Quadratic Programming,Doklady Akademiia Nauk SSSR 174, 747{748.[6] Fiacco, A. V. and McCormick, G. P. (1968), Nonlinear Programming, Sequential Uncon-strained Minimization Techniques, Wiley and Sons, New York.[7] Gill, P.E., Murray, W. and Wright, M.H. (1988), Practical Optimization, Seventh Printing,Academic Press, Inc., San Diego, California.[8] Goldfarb, D. and Liu, S. (1988), An O(n3L) Primal Interior Point Algorithm for ConvexQuadratic Programming, Manuscript, Department of Industrial Engineering and OperationsResearch, Columbia University, New York, NY.[9] Gonzaga, C.C. (1989), Large{Steps Path{Following Methods for Linear Programming: BarrierFunction Method, Report ES{210/89, Department of Systems Engineering and ComputerSciences, COPPE{Federal University of Rio de Janeiro, Rio de Janeiro, Brasil.15

[10] Jarre, F. (1987), On the Convergence of the Method of Analytic Centers when applied toConvex Quadratic Programs, Report No. 35, Schwerpuntkprogramm AnwendungsbezogeneOptimierung und Steuerung, Institut f�ur Angewandte Mathematik und Statistik, Universit�atW�urzburg, W�urzburg, West{Germany.[11] Jarre, F. (1989), The Method of Analytic Centers for Smooth Convex Programs, Dissertation,Institut f�ur Angewandte Mathematik und Statistik, Universit�at W�urzburg, W�urzburg, West{Germany.[12] Kapoor, S. and Vaidya, P. (1986), Fast Algorithms for Convex Quadratic Programming andMulticommodity Flows, Proc. 18th Annual ACM Symp. Theory Comput. 147{159.[13] Kapoor, S. and Vaidya, P.M. (1988), An Extension of Karmarkar's Interior Point Methodto Convex Quadratic Programming, Technical Report, Department of Computer Science,University of Illinois at Urbana{Champaign, Urbana.[14] Karmarkar, N. (1984), A New Polynomial{Time Algorithm for Linear Programming, Combi-natorica 4, 373{395.[15] Kojima, M., Megiddo, N., Noma, T., Yoshise, A. (1990), A Uni�ed Approach to Interior PointAlgorithms for Linear Complementarity Problems, Research Report RJ 7493 (70008), IBMAlmaden Research Division, San Jose, California.[16] Kojima, M., Mizuno, S., Yoshise, A. (1987), A Polynomial Time Algorithm for a Class ofLinear Complementarity Problems, Research Report, Department of Information Sciences,Tokyo Institute of Technology, Tokyo, Japan.[17] Kojima, M., Mizuno, S., Yoshise, A. (1988), An O(pnL) Iteration Potential Reduction Al-gorithm for Linear Complementarity Problems, Research Report, Department of InformationSciences, Tokyo Institute of Technology, Tokyo, Japan.[18] Mehrota, S. and Sun, J. (1988), On Computing the Centers of a Convex Quadratically Con-strained Set, Technical Report 88{11, Department of IE/MS, Northwestern University, Evas-ton IL.[19] Mehrota, S. and Sun, J. (1988), An Interior Point Algorithm for Solving Smooth Convex Pro-grams Based on Newton's Method, Technical Report 88{08, Department of IE/MS, North-western University, Evaston IL.[20] Mizuno, S. (1990), An O(n3L) Algorithm Using a Sequence for a Linear ComplementarityProblem, Journal of the Operations Research Society of Japan Vol. 33, No. 1.[21] Monteiro, R.D.C., Adler, I. (1987), Interior Path Following Primal{Dual Algorithms, PartII: Convex Quadratic Programming, Department of Industrial Engineering and OperationsResearch, University of California, Berkeley, California.[22] Monteiro, R.D.C., Adler, I., Resende, M.G.C. (1988), A Polynomial{Time Primal{Dual A�neScaling Algorithm for Linear and Convex Quadratic Programming and its Power Series Ex-tension, Report ESRC 88{8, Department of Industrial Engineering and Operations Research,University of California, Berkeley, California.16

[23] Murty, K.G. (1988), Linear Complementarity, Linear and Nonlinear Programming, Helder-mann Verlag Berlin, Germany.[24] Nesterov, Y.E., and Nemirovsky, A.S. (1989), Self{Concordant Functions and PolynomialTime Methods in Convex Programming, Moscow.[25] Papadimitriou, C.R., and Steiglitz, K. (1982), Combinatorial Optimization: Algorithms andComplexity, Prentice{Hall, Englewood Cli�s, New Jersey.[26] Renegar, J., and Shub, M. (1988) Simpli�ed Complexity Analysis for Newton LP Methods,Technical Report No. 807, School of Operations Research and Industrial Engineering, CornellUniversity, Ithaca, New York.[27] Roos, C. and Vial, J.{Ph. (1988), A Polynomial Method of Approximate Centers for LinearProgramming, to appear in Mathematical Programming.[28] Ye, Y. (1987), Further Development on the Interior Algorithm for Convex Quadratic Pro-gramming, Engeneering{Economic Systems Department, Stanford University, Stanford.[29] Ye, Y. (1987), Interior Algorithms for Linear, Quadratic and Linearly Constrained Con-vex Programming, Ph.D. Dissertation, Engeneering{Economic Systems Department, StanfordUniversity, Stanford.[30] Ye, Y. and Pardalos, P. (1989), A Class of Linear Complementarity Problems Solvable inPolynomial Time, Department of Management Sciences, The University of Iowa, Iowa City,Iowa.[31] Ye, Y. and Tse, E. (1986), A Polynomial{time Algorithm for Convex Quadratic Programming,Manuscript, Engeneering{Economic Systems Department, Stanford University, Stanford.

17