ISS

N 0

249-

6399

ap por t de r ech er ch e

INSTITUT NATIONAL DE RECHERCHE EN INFORMATIQUE ET EN AUTOMATIQUE

Multimodularity, Convexity and OptimizationProperties

Eitan ALTMAN Bruno GAUJAL and Arie HORDIJK

N˚ 3181

Juin, 1997

THÈME 1

Multimodularity, Convexity and OptimizationPropertiesEitan ALTMAN * Bruno GAUJAL ** and Arie HORDIJK ***Thème 1 � Réseaux et systèmesProjet Mistral, SloopRapport de recherche n�3181 � Juin, 1997 � 25 pagesAbstract: We investigate in this paper the properties of multimodular functions. In doingso we give alternative proofs for properties already established by Hajek, and we extend hisresults. In particular, we show the relation between convexity and multimodularity, whichallows us to restrict the study of multimodular functions to convex subsets of Zm. We thenobtain general optimization results for average costs related to a sequence of multimodularfunctions. In particular, we establish lower bounds, and show that the expected averageproblem is optimized by using balanced sequences. We �nally illustrate the usefulness ofthis theory in admission control into a D/D/1 queue with �xed batch arrivals, with nostate information. We show that the balanced policy minimizes the average queue lengthfor the case of an in�nite queue, but not for the case of a �nite queue. When further addinga constraint on the losses, it is shown that a balanced policy is also optimal for the case of�nite queue.Key-words: Multimodular functions, convexity, balanced sequences, admission controlinto a queue. (Résumé : tsvp)* INRIA, BP 93, 2004 Route des Lucioles, 06902 Sophia Antipolis Cedex, France. E-mail: alt-man@sophia.inria.fr. URL:http://www.inria.fr:80/mistral/personnel/Eitan.Altman/me.html** INRIA, BP 93, 2004 Route des Lucioles, 06902 Sophia Antipolis Cedex, France. E-mail: gau-jal@sophia.inria.fr. Bruno Gaujal is a member of a common project between CNRS, UNSA and INRIA.*** Dept. of Mathematics and Computer Science, Leiden University, P.O.Box 9512, 2300RA Leiden, TheNetherlands. E-mail: hordijk@wi.leidenuniv.nl. The research of Arie Hordijk was done while he was onsabbatical leave at INRIA, Sophia-Antipolis; it has been partially supported by the Ministère Français del'Éducation Nationale et de l'Enseignement Supérieur et de la Recherche.Unité de recherche INRIA Sophia Antipolis

Multimodularité, convexité et propriétés d'optimisationRésumé : Dans cet article, nous nous intéressons aux propriétés des fonctions multimo-dulaires. Ce faisant, nous montrons de nouvelles preuves des propriétés établies par Hajeket nous étendons certains de ses résultats. En particulier, nous montrons la relation quiexiste entre multimodularité et convexité qui nous permet d'étudier la multimodularité surdes parties convexes de Zm. Ensuite, nous obtenons des résultats généraux d'optimisationpour le coût moyen d'une suite de fonctions multimodulaires. En particulier, on exhibe desbornes inférieures et on montre qu'elles sont atteintes pour les suites équilibrées. Finale-ment, on illustre l'utilité de cette théorie pour le contrôle d'admission dans une �le D/D/1avec des arrivées par paquets de taille �xe, sans information. On montre que la politiqueéquilibrée minimise la longueur moyenne de la �le pour une capacité in�nie mais pas dansle cas d'une �le à capacité �nie. Dans ce cas, Si l'on ajoute une contrainte sur les pertes,on montre aussi qu'une politique équilibrée est optimale.Mots-clé : Multimodularité, convexité, contrôle d'admission, suites équilibrées.

Multimodularity, Convexity and Optimization Properties 31 IntroductionThe multimodularity property of functions was much investigated in the context of queueingsystems. There are several cases in that �eld in which this property was exploited to solvestochastic control. Optimal admission control under no queue information was studiedby Hajek [7]. The precise problem was to admit customers to a single queue, under theconstraint that the long run fraction of customers admitted be at least p. The optimalityof a policy based on a balanced sequence of admission actions was obtained in [7] for thenumber of customers in a one-server queue with exponential service and a renewal arrivalprocess.Another application of multimodular functions is in the control of queues with full stateinformation. Weber and Stidham [8] (and later Glasserman and Yao [4]) obtained monotoneproperties of the optimal control policies as a function of the state, in a problem of controlof service rates in a system of m queues in tandem. The methodology was strongly basedon the multi-modularity properties of the immediate costs and the cost-to-go functions.The purpose of this paper is to study the properties of multimodular functions, as a toolfor further investigating the control of queueing systems. We provide alternative proofs forproperties already established by Hajek. We further show the relation between convexityand multimodularity, which allows us to restrict the study of multimodular functions toconvex subsets of Zm.We then develop basic optimization tools for average costs, related to a sequence ofmultimodular functions. In particular, we establish lower bounds which are achieved bybalanced sequences.We �nally illustrate the usefulness of this theory in admission control into a queue; wecite some results for the G/G/1 queue, and provide a detailed analysis of the D/D/1 queuewith �xed batch arrivals, with no state information. We show, for the latter, that the policywhich is de�ned through a balanced sequence minimizes the average queue length for thecase of an in�nite queue, but not for the case of a �nite bu�er. However, when furtherrestricting to those policies for which no losses occur, we obtain again the optimality ofbalanced policies. To conclude that example, we study also the case where it is possible toadmit a part of an arriving batch.In follow-up papers, we shall make use of the theoretical results of this paper in orderto study more general admission and service control problems in dynamic systems that canbe described using the max-plus algebra, with general inter-arrival and service times.2 Properties of multimodular functionsWe present in this section a short overview and extension of Hajek's theory of multimodu-lar functions. We begin by presenting the de�nition of multimodularity, and some generalproperties (Subsection 2.1). We then present in Subsection 2.2 the relation between mul-timodularity and convexity. The properties presented in Subsection 2.2 are those neededin the following sections on optimization and control.RR n�3181

4 Eitan ALTMAN Bruno GAUJAL and Arie HORDIJKLet ei 2 Nm , i = 1; :::; m denote the vector having all entries zero except for a 1 in itsith entry. De�ne di = ei�1 � ei, i = 1; :::; m (for an integer i taking values between 0 andm, we understand throughout i� 1 = m for i = 0).Let F = f�e1; d2; :::; dm; emg. De�ne G = fei;�ei; di;�di; i = 0; 1; :::; mg.De�nition 2.1 (Hajek). A function f on Zm is multimodular with respect to F if for allx 2 Zm, v; w 2 F , v 6= w,f(x+ v) + f(x+ w) > f(x) + f(x+ v + w): (1)Unless otherwise stated, we shall say that f is multimodular if it is multimodular withrespect to F .2.1 General propertiesFor a function g de�ned on Zm, de�ne�ig(x) = �eig(x) = g(x+ ei)� g(x); �di = �i�1g ��ig:We further de�ne ��ei = ��ei , ��di = ��di . Note that�dig(x) = g(x+ei+di)�g(x+ei).It is easy to check thatLemma 2.1. �v is a linear function for any v 2 G. For all v; w 2 G, �v�w = �w�v.Lemma 2.2. f is multimodular if and only if�v�wf 6 0 (2)for all v; w 2 F , w 6= v.Proof. Consider �rst w = di; v = dj (v 6= w). Then�v�wf(x) = (�i�1 ��i)(�j�1 ��j)f(x)= (�i�1 ��i)(f(x+ ej�1)� f(x+ ej))= f(x + ej�1 + ei�1)� f(x+ ej�1 + ei)� f(x + ej + ei�1) + f(x+ ej + ei)= f(z + dj + di)� f(z + dj)� f(z + di) + f(z) (3)where z 4= x+ ej + ei.Let v = ej; w = �ei. Then�v�wf(x) = ��j�if(x)= ��j(f(x+ ei)� f(x))= �f(x + ei + ej) + f(x+ ei) + f(x+ ej)� f(x)= �f(z + ej) + f(z) + f(z + [�ei] + ej)� f(z + [�ei]) (4)INRIA

Multimodularity, Convexity and Optimization Properties 5where z 4= x+ ei.Let v = ei; w = dj (j 6= i+ 1).�v�wf(x) = �i(�j�1 ��j)f(x)= �i(f(x+ ej�1)� f(x + ej))= f(x+ ej�1 + ei)� f(x+ ej�1)� f(x + ej + ei) + f(x+ ej)= f(z + dj + ei)� f(z + dj)� f(z + ei) + f(z) (5)where z = x+ ej.Let v = �ei; w = dj (j 6= i).�v�wf(x) = ��i(�j�1 ��j)f(x)= ��i(f(x+ ej�1)� f(x+ ej))= �f(x + ej�1 + ei) + f(x + ej�1) + f(x+ ej + ei)� f(x+ ej)= �f(z + dj) + f(z + dj + [�ei]) + f(z) + f(z + [�ei]) (6)where z = x+ ej + ei.The above equations easily imply the Lemma.Lemma 2.3. (i) If f is multimodular then(i) For all i; j, �i�jf > 0; (7)and(ii) For all i; j, �j�jf > �i�jf: (8)Proof. (i) Without loss of generality, assume that i 6 j. Then�i�jf = ���e1 � iXk=2 �dk! mXl=j+1�dl +�em! fThe proof of (i) is established by applying Lemma 2.2.For i < j we have�j�jf = (�j ��i)�jf +�i�jf > (�j ��i)�jf= � jXk=i+1�dk! mXl=j+1�dl +�em! fand (ii) is established by applying Lemma 2.2. For i > j we have�j = �i � m�1Xk=i+1�dk ��m ���e1 � jXk=2 �k:RR n�3181

6 Eitan ALTMAN Bruno GAUJAL and Arie HORDIJKHence �j�jf = �j(�j ��i)f +�j�if > �j(�j ��i)f= ���e1 � jXk=2 �dk! � mXk=i+1�dk ��m ���e1 � jXk=2 �k! f:Again, (ii) is established by applying Lemma 2.2.Remark 2.1. The converse of the above Lemma holds for the 2-dimensional case: F =f�e1; d2; e2g. Indeed, assume that (7) and (8) hold. Then ��e1�e2 6 0 due to (7);��e1�d2 = ��e1(�e1 ��e2) 6 0, and �e2�d2 = �e2(�e1 ��e2) 6 0 due (8). Hence f ismultimodular by Lemma 2.2.Lemma 2.4. If f is multimodular then for all i = 1; :::; m,(i) �ei�dif > 0,(ii) �ei�djf 6 0, j < i and �ei�djf > 0, j > i,(iii) �d1�dif 6 0, i 6= 1,(iv) �di�dif > 0.(v) �di�djf 6 0 for i 6= j, i 6= 1, j 6= 1.Proof. By taking i = j � 1 in Lemma 2.3 (ii), we obtain (i).(ii) For j < i, �ei�djf = mXk=i �dk +�em!�djf:For j > i, �ei�djf = ���e1 � iXk=2 �dk!�djf:For both cases, the proof is established by applying Lemma 2.2.(iii) �d1�dif = (�em +��e1)�dif:The proof is established by applying Lemma 2.2.(iv) �di�dif = �di �Xj 6=i �dj! fFor i 6= 1, the proof is established by applying Lemma 2.2. For i = 1 it follows from part(iii) of this Lemma. INRIA

Multimodularity, Convexity and Optimization Properties 72.2 Multimodularity and convexityIn the space Rm , m+ 1 extreme points in Zm form a simplex.A simplex consisting of the extreme pointsfx0; :::; xmg of Zm is called an atom (de�nedin [7] �3) if and only if for some ordering of the subset and for some permutation fi1; � � � ; imgof (0; 1; :::; m), x1 = x0 + gi1x2 = x1 + gi2:::xm = xm�1 + gimx0 = xm + gi0 (9)where f0; :::; fm are the elements of F .Next we present a characterization of an atom [7], which is essential for the optimizationresult that we obtain in the following sections. Denote by bxc the largest integer smallerthan or equal to x. Then the following trivially holdsZ 10 bx + �cd� = x: (10)Given z 2 Rm , � 2 R, de�ne the vector uz(�) in Zm:uzi (�) = b� + z1 + :::+ zic � b� + z1 + :::+ zi�1c;i = 1; :::; m. Then by (10), Z 10 uz(�)d� = z: (11)Since Z 10 b� + z1 + :::+ zicd� = z1 + ::: + zi;Z 10 b� + z1 + :::+ zi�1cd� = z1 + ::: + zi�1:uz(�) is periodic in � with period 1, and piecewize constant with at most m+ 1 jumpsper period. Thus, the set fuz(�) : 0 6 � 6 1g contains at most m + 1 vectors, all integervalued. The next Lemma follows from [7]:Lemma 2.5. A point z is contained in an atom called S(z) if and only if the extremepoints of S(z) contain fuz(�) : 0 6 � 6 1g. A point z is in the interior of an atom S(z) ifand only if the extreme points of S(z) equal fuz(�) : 0 6 � 6 1g.RR n�3181

8 Eitan ALTMAN Bruno GAUJAL and Arie HORDIJKEach point z 2 Rm is contained is some atom S(z). It can thus be expressed as aconvex combination of the extreme points of S(z).For any function f on Zm, we de�ne the corresponding function ef on Rm as follows.It agrees with f on Zm, and its value on an arbitrary point in z 2 Rm is obtained as thecorresponding linear interpolation of the values of f on the extreme points of the atomS(z).Theorem 2.1. f is multimodular if and only if ef is convex.Proof. �only if�:We check convexity at a point z; it is established by showing that at point z, at anydirection d, the right derivative is greater that or equal to the left derivative. It obviouslysu�ces to check at points that are on the boundary of an atom, since, by de�nition, ef islinear in the interior of atoms. Hence, we �rst assume that the point z is on the interior ofa face (of dimension m� 1) which is common between two adjacent atoms. Without lossof generality, assume that the atoms (de�ned below by their extreme points) areA = A(x0; x1; :::; xm) and A = A(x0; x�1; :::; xm):where xi satisfy (9) and x�1 = x0 + gi2 ; x2 = x�0 + gi1 :Case 1: gi1 = �e1 = (�1; 0; :::; 0), gi2 = em = (0; 0; :::; 1).X1 Commonface

�e1 ememX0X�1X2 z of dimension m� 1�e1Figure 1: Checking convexity at a point z, Case 1.Decompose direction d in its projection djj over the common face and in the directiond? perpendicular to that face. In the direction djj, the left and right derivatives are equal.Note that the hyper-plane through x1; x0; x2; x�1 is perpendicular to the common face. Inthe direction d?, the right derivative is a constant c times ef(x�1)� ef(z). The left derivative

INRIA

Multimodularity, Convexity and Optimization Properties 9is c( ef(z)� ef(x1)). Omitting the constant c, we get for the di�erence( ef(x�1)� ef(z))� ( ef(z)� ef(x1))= 12�(f(x�1)� f(x0))� (f(x2)� f(x1))� (12)+12�(f(x1)� f(x0))� (f(x2)� f(x�1))� (13)The fact that both (12) as well as (13) are nonnegative follows by applying (1) with x = x0.Then, (12) > 0 since 8<: x�1 = x0 + emx2 = x0 � e1 + emx1 = x0 � e1(13) > 0 since 8<: x1 = x0 � e1x2 = x0 � e1 + emx�1 = x0 + emCase 2: gi1 = em and gi2 = �e1. It is handled as Case 1.Case 3: gi1 = d2 = (1;�1; 0; :::; 0), gi2 = �e1. We thus set x = x0, and x�1 = x0 � e1; x2 =x0 � e1 + d2; x1 = x0 + d2. In this case, x2 � x1 is perpendicular to the common face.We decompose again d to the projection on the common face djj, and to the direction d?perpendicular to the face. As in Case 1, it su�ces to consider the direction d?. The rightderivative in this direction is f(x�1)� f(x0), and the left derivative is f(x2)� f(x1) (bothup to a multiplicative constant).The di�erence between the right and left derivatives is indeed nonnegative: f(x�1) �f(x0)� (f(x2)� f(x1)) > 0. This is obtained again by applying (1).X0 d2Common faceX�1

of dimension m� 1�e1

X2 �e1 X1d2Figure 2: Checking convexity at a point on the common face, Case 3Case 4: gi1 = d2 = (1;�1; 0; :::; 0), gi2 = d3. In this case, x�1 � x1 is perpendicular to thecommon face, and the analysis is as for Case 1.All other cases, in which z is in the interior of a face (of dimension m � 1), commonto two adjacent atoms, are similar to one of those considered above. It now remains toRR n�3181

10 Eitan ALTMAN Bruno GAUJAL and Arie HORDIJKconsider the case where z lies on a subspace of A\A of dimension at most m�2. We notethat the point z can be expressed as the limit of a sequence zn of points that are on theboundary between two atoms which have a face (of dimension m � 1) in common. Sincef is continuous (this follows from the linear interpolation), and since the limit of convexfunctions is also convex, the convexity at the point z is established as well.�if�: Consider an arbitrary point x0 and any two distinct elements gi; gj in F . We have toshow that f(x0) + f(x2)� f(x1)� f(x�1) 6 0; (14)where x1 4= x0 + gi, x�1 4= x0 + gj, x2 4= x1 + gj = x�1 + gi.De�ne z 4= 0:5(x1 + x�1) = 0:5(x0 + x2) and consider the line segment x1 ! z ! x�1.The left derivative (l.d.) and right derivative (r.d.) in z are given byl:d: = ef(12(x1 + x�1))� ef(x1) = 12f(x0) + 12f(x2)� f(x1);r:d: = ef(x�1)� ef(12(x0 + x2)) = f(x�1)� 12f(x0)� 12f(x2):Since ef is convex, r:d:� l:d: is non-positive, and hence (14) holds.Remark 2.2. (Restriction of Multimodularity to a convex set)It is clear from the proof that we can restrict the domain of f in Theorem 2.1. Indeed, let Abe a convex set which is a union of a set of atoms. The equivalence of the multimodularityof f and the convexity of ef still holds if we restrict the function f to A, and restrict thede�nition of multimodularity to directions that lead to points in A. In other words, f ismultimodular in A if the following holds. If x0; x0+ gi; x0+ gj; x0+ gi+ gj are all elementsof A then f(x0) + f(x0 + gi + gj)� f(x0 + gi)� f(x0 + gj) 6 0:Next we consider the integer convexity properties of a function f . A function f is saidto be integer convex if the following holds. For vectors x and d in Zm, we havef(x+ d)� f(x) > f(x)� f(x� d):Theorem 2.2. Let f be multimodular. Then it is integer convex.Proof. De�ne �+d (x):= the right derivative of ef at x in the direction d and ��d (x):= theleft derivative of ef at x in the direction d. Since ef is convex (Theorem 2.1) then�+d (x) > ��d (x): (15)INRIA

Multimodularity, Convexity and Optimization Properties 11Since ef(y) = f(y) at the integer points, and since ef is convex, we have�+d (x) 6 f(x+ d)� f(x)jdj ; ��d (x) > f(x)� f(x� d)jdj :This, together with (15) imply the integer convexity of f .The converse of the above theorem is not true:Counter-example 2.1. Consider the convex function f : Nm ! R given by f(x) =maxi=1;:::;m xi. It is integer convex since it is the maximum of convex (linear) functions.However, it is not multimodular. Indeed, consider m = 2, x = (i+ 1; i) for some integer i.Then 2i+ 2 = f(x� e1 + em) + f(x) > f(x� e1) + f(x+ e2) = 2i+ 1:Hence f is not multimodular.Theorem 2.3. Let f be multimodular, and let h : R ! R be convex nondecreasing. As-sume that f(x+di) 6 f(x); i = 2; :::; m, and f(x�em) 6 f(x). Then h(f) is multimodular.Proof. Let g 2 F , g 6= di. Since f is multimodular, we have:f(x+ di)� f(x) > f(x + di + g)� f(x+ g) =: �: (16)Hence,h(f(x+ di + g))� h(f(x + g)) = h(f(x + di) + �)� h(f(x+ di)) 6 h(f(x) + �)� h(f(x))= h(f(x) + f(x+ di + g)� f(x+ g))� h(f(x))6 h(f(x + di))� h(f(x)):The �rst inequality follows from the convexity of h, and since f(x + di) 6 f(x). Thesecond inequality follows from the fact that h it is nondecreasing, and the fact that by(16), f(x+ di) > f(x) + f(x+ di + g)� f(x + g).The same argument holds for em replacing di, which establishes the proof.3 The optimality of regular policies for a single criterionIn this section, we will use multimodularity to optimize a cost function based on a sequenceof functions which will represent a quantity of interest in a given model, such as workloadin a queue for example (see � 6 for a more precise instance of the problem).Consider a sequence of nonnegative functions fk : Nk ! R that satisfy the followingassumptions:� <1> fk is multimodular.RR n�3181

12 Eitan ALTMAN Bruno GAUJAL and Arie HORDIJK� <2> fk(a1; :::; ak) > fk�1(a2; :::; ak); 8K > 1;For a given sequence fakg, we de�ne the cost g(a) asg(a) = limN!1 1N NXn=1 fn(a1; :::; an):Let p and � be two positive reals. We de�ne the balanced sequence fapk(�)g with rate pand initial phase � as, apk(�) = bkp + �c � b(k � 1)p+ �c; (17)where bxc is the largest integer smaller than or equal to x. Note that the set fapk(�); 0 6� 6 1g are extreme points of an atom containing the point (p; p; :::; p).The aim of this section will be to prove that this sequence minimizes the function g,provided that some conditions (including <1> and <2>) above hold. This sequence wasused by Hajek in [7], and we use several properties of the balanced sequence establishedtherein. To establish the main optimization results, we need the following technical Lemma.Lemma 3.1. If fk satis�es assumption <2>, then the function efk satis�es assumption<2> for positive real numbers.Proof. Let z = (z1; � � � ; zk) 2 Rk+ . This point belongs to an atom S(z) made by theextreme points x0; x1; � � � ; xk. The numbering of the extreme points of the atom is chosensuch that according to the base Fk = (�ek1; dk1; � � � ; dkk�1; ekk), x1 = x0 � ek1. The otherindices are arbitrary. This implies that x0j = x1j for all j > 2. If we call P the projectionof Rk+ onto Rk�1+ along the �rst coordinate,P (dkj ) = P (dk�1j ) if 1 < j < kP (dk1) = �ek�11P (ek1) = 0P (ekk) = ek�1kP (xi) = (xi2; � � � ; xik)Theses equalities imply that P (x0) = P (x1) and P (x1); � � � ; P (xk) form an atom inRk�1+ , using the de�nition of an atom. Also, P(z) belongs to this atom, and if(z1; z2; � � � ; zk) = 1� kXi=1 �i! x0 + �1x1 + � � �+ �kxk;then (z2; � � � ; zk) = P (z1; z2; � � � ; zk)= 1� kXi=1 �i!P (x0) + �1P (x1) + � � �+ �kP (xk)= 1� kXi=2 �i!P (x1) + � � �+ �kP (xk): INRIA

Multimodularity, Convexity and Optimization Properties 13Now,efk(z1; z2; � � � ; zk) = 1� kXi=1 �i! fk(x0) + �1fk(x1) + � � �+ �kfk(xk)> 1� kXi=1 �i! fk�1(P (x0)) + �1fk�1(P (x1)) + � � �+ �kfk�1(P (xk))= 1� kXi=2 �i! fk�1(P (x1)) + � � �+ �kfk�1(P (xk))= efk�1(z2; � � � ; zk):Theorem 3.1. Under assumptions <1> and <2>, let � be a random variable, uniformlydistributed in [0; 1], and denote the expectation w.r.t. � by E�. ThenlimN!1E�fN(ap1(�); :::; apn(�)) = limN!1 efN(p; p; :::; p): (18)Proof We have for all n, E�fn(ap1(�); :::; apn(�)) = efn(p; :::; p): (19)(This follows (11), from Lemma 2.5, and fact that efn is a�ne on each atom, and agreeswith fn for the extreme points of the atom.) Since efN (p; p; :::; p) is increasing in N byLemma 3.1, the limit in N exists (it is possibly in�nite).We call the sequence fap(�)g the randomized balanced policy.Theorem 3.2. Under assumptions <1> and <2>, for every � 2 [0; 1],limN!1 1N NXn=1 fn(ap1(�); :::; apn(�)) 6 limN!1 efN(p; p; :::; p): (20)Proof. De�ne fm(�; p) 4= fm(ap1(�); :::; apm(�)):fm is periodic (in �) with period 1. De�nef 0m(�; p) 4= fm(ap�m+1(�); :::; ap0(�)):Then we have f 0m(�0; p) = fm(�; p) where �0 = � �mp; (21)RR n�3181

14 Eitan ALTMAN Bruno GAUJAL and Arie HORDIJKIndeed, f 0m(�0; p) = fm(ap�m+1(�0); :::; ap0(�0))= fm(ap�m+1(� +mp); :::; ap0(� +mp)) = fm(�; p);where the last equality follows from the fact that ap�m+k(�+mp) = apk(�), k = 1; :::; m. f 0mis again periodic w.r.t. �, with period 1, and is increasing in m so that the following limitexists (possibly in�nite): f 01(�; p) 4= limm!1 f 0m(�; p):Moreover, we have that E�f 0m(�; p) = efm(p; :::; p), where � be a random variable, uni-formly distributed in [0; 1] (this follows (11), from Lemma 2.5, and fact that efn is a�ne oneach atom, and agrees with fn for the extreme points of the atom). Hence,E�f 01(�; p) = limN!1 efN(p; p; :::; p): (22)Consider now the balanced sequence for �xed �. Then1N NXm=1 fm(ap1(�); :::; apm(�))6 1N NXm=1 fN(ap�(N�m)(�); :::; ap0(�); :::; apm(�))6 1N NXm=1 f 01(�mp + �; p):The last inequality follows from assumption <2> for the functions fk, as well as an argu-ment similar to the one used in (21).If p is irrational, applying the ergodic theorem of Weyl and Von Neumann ([5]), wehave limN!1 1N NXm=1 f 01(�mp + �; p) = E�f 01(�; p):From Equation (22), we have E�f 01(�; p) = limN!1 efN(p; p; :::; p): This implies that ifp is irrational, limN!1 1N NXm=1 f 01(�mp + �; p) = limN!1 efN(p; p; :::; p) (23)If p is rational, then p = q=d were q and d are relatively prime and d > 1. This impliesthat the sequence (ap�(N�m)(�); :::; ap0(�); :::; apm(�)) is constant if � mod 1 2 [j=d; (j+1)=d],INRIA

Multimodularity, Convexity and Optimization Properties 15for all j. Therefore, f 0m(�; p) is also constant on these intervals and by passage to the limit,f 01(�; p) is constant on these intervals. Now, note that Frac(� � mp) 2 [j=d; (j + 1)=d]for exactly one value of m out of d consecutive values of m because q are d are relativelyprime.Now, we havelimN!1 1N NXm=1 f 01(�mp + �; p) = 1d d�1Xm=1 f 01(m=d; p) = E�f 01(�; p):Equation (22) concludes this case as well.3.1 Lower BoundsIn this subsection, we establish lower bounds for the discounted cost for all sequences fakg.This then serves for obtaining a lower bound on the average cost. Here, we use the followingassumption for the functions fk.� <3> For any sequence fakg 9 a sequence fbkg such that8k;m; k > m; fk(b1; � � � ; bk�m; a1; :::; am) = fm(a1; :::; am).We use the notions de�ned in the previous sections.Let us �x the sequence fakg, as well as some arbitrary integer, N . We de�ne p� 4=(1� �)P1k=1 �k�1ak.Now, using assumptions <1> through <3>, we have1Xn=1(1� �)�n�1fn(a1; a2:::; an)> NXn=1(1� �)�n�1fN(b1; � � � bN�n; a1; a2:::; an) + 1Xn=N+1(1� �)�n�1fN (an�N+1; a2:::; an)> efN b1 N�1Xn=1(1� �)�n�1 + �Np�; b2 N�2Xn=1(1� �)�n�1 + �N�1p�; � � � ; p�!= B(N;�; p�); (24)whereB(N;�; p) 4= efN b1 N�1Xn=1(1� �)�n�1 + �Np; b2 N�2Xn=1(1� �)�n�1 + �N�1p; � � � ; p! (25)Note that B is de�ned for a �xed sequence fakg. Also note that B(N;�; p) is lowersemi-continuous in � for 0 6 � < 1 and in p.RR n�3181

16 Eitan ALTMAN Bruno GAUJAL and Arie HORDIJKUsing Lemma 7.1 in the Appendix, we �nd,limm!1 1m mXn=1 fn(a1; � � � ; an) > lim�"1(1� �) 1Xn�1 �n�1fn(a1; � � � ; an)> lim�"1 B(N;�; p�)> infq2LB(N; 1; q); (26)where L is the set of all limit points of p� as � " 1.3.2 Optimality of the Balanced SequencesTheorem 3.3. Under assumptions <1>,<2> and <3>, and given some p 2 [0; 1], andany � 2 [0; 1], if the functions fk(a1; � � � ; ak) are increasing in all ai, then the balancedsequence ap(�) minimizes the average cost g(a) over all sequences that satisfy the constraint:limN!1 1N NXn=1 an > p:Proof.We denote by p 4= limN!1 1N NXn=1 anBy using Lemma 7.1 in the Appendix,p 6 p 6 lim�"1 p� = inffq; q 2 Lg:If the functions ffkg are increasing, then B is increasing in p, therefore,g(a) > infq2LB(N; 1; q) > B(N; 1; p); (27)by Equation (26). Note that for any given p, by de�nition ofB, B(N; 1; p) = efN(p; p; � � � ; p).If we let N go to in�nity, we getg(a) = limm!1 1m mXn�1 fn(a1; � � � ; an) > limN!1 efN(p; p; p): (28)Theorem 3.2 shows that limN!1 efN(p; p; p) > g(ap(�)). Thus g(a) > g(ap(�)).When the functions fk are decreasing, we have the analogous result.INRIA

Multimodularity, Convexity and Optimization Properties 17Theorem 3.4. Under assumptions <1>,<2> and <3>, and given some p 2 [0; 1], andany � 2 [0; 1], if the functions fk(a1; � � � ; ak) are decreasing in all ai, then the balancedsequence ap(�) minimizes the average cost g(a) over all sequences that satisfy the constraint:limN!1 1N NXn=1 an 6 p:Proof. The proof is similar to the previous one, using the fact that ifp 4= limN!1 1N NXn=1 an;then p > p > lim�"1 p� = supfq; q 2 Lg:4 The optimality of balanced policies for multiple crite-riaIn this section, we establish general conditions under which the balanced policy is optimalwhen the cost function depends on multiple criteria. This has applications for routingcontrol in several queues rather than admission control in one queue as in the previoussection.Now, we study the following general optimization problem. Consider K sequences offunctions f (i)n ; i = 1; :::; K. Each set of functions f (i)n will satisfy assumptions <1>,<2>and <3>, as in Section 3.A policy is a sequence a = (a1; a2; :::), where an is a vector taking values in f0; 1gK.We consider the additional constraint that for every integer j, only one of the componentsof aj may be di�erent than 0. A policy satisfying this constraint is called feasible.Let h be a convex increasing function from RK to R. De�neg(a) 4= limN!1 1N NXn=1 h(f 1n(a); :::; fKn (a));Following notations introduced in Section 3, we get a bound called Bi(N;�; pi) forsequence i. Here, we denote byBi(�; pi) 4= supN Bi(N;�; pi);RR n�3181

18 Eitan ALTMAN Bruno GAUJAL and Arie HORDIJKand Bi(pi) 4= sup�61 Bi(�; pi):Note that by convexity of f (i)n , Bi(pi) is continuous from below.Our objective is to minimize g(a) (with no constraints on the asymptotic fractions).Theorem 4.1. Assume that for all i, the functions f (i)n satisfy assumptions <1>,<2> and<3>. The following lower bound holds for all policies:g(a) > infp1+:::+pK=1 h(B1(p1); :::; BK(pK)):Proof. Due to Lemma 7.1 in the Appendix, Jensen's inequality and Equation 24, we havelimN!1 1N NXn=1 h(f 1n; :::; fKn )> lim�!1(1� �) 1Xn=1 �n�1h(f 1n; :::; fKn )> lim�!1 h (1� �) 1Xn=1 �n�1f 1n; :::; (1� �) 1Xn=1 �n�1fKn !> lim�!1 h (B1(�; pa1(�)); :::; BK(�; paK(�))) (29)where pai (�) 4= (1� �) 1Xk=1 �k�1aik: (30)We note that PKi=1 pai (�) = 1. Hence, one may choose a sequence �n " 1 such that thefollowing limits exist: limn!1 pai (�n) = pi; i = 1; :::; K (31)and PKi=1 pi = 1. From the continuity of Bi(�; pi) in p and � we get from (29)g(a) > h(B1(1; p1); :::; ; BK(1; pK)) (32)> infp1+:::+pK=1 h(B1(1; p1); :::; BK(1; pK)):Note that there exists some p� that achieves the in�muminfp1+:::+pK=1 h(B1(1; p1); :::; BK(1; pK)); INRIA

Multimodularity, Convexity and Optimization Properties 19since h(B1(1; p1); :::; BK(1; pK)) is continuous in p = (p1; � � � ; pK).Consider the policy ap�(�) given byap�k;i(�) = bkp�i + �ic � b(k � 1)p�i + �ic: (33)There are some p� for which the condition of feasibility of the policy ap�(�) is satis�ed,that is, there exists some � = (�1; :::; �K), such that the policy ap�(�) given in (33) is feasible.These p� are called balanced and are more exhaustively studied in [3] and references therein.Theorem 4.2. Assume that h is linear nondecreasing and that p� is balanced. Then ap�(�)is optimal for the average cost, i.e. it minimizes g(a) over all feasible policies.Proof. The proof follows directly from Theorem 3.2 together with Theorem 4.1.The balance condition on p� is still not completely characterized, however, we canmention two simple cases for which p� is balanced. i.e. for which there exist some �1; :::; �K ,such that ap�(�) is feasible.� P1: K = 2.� P2: K criteria with symmetric costs (h(x) =Pi xi and all f i are equal).Corollary 4.1. (i) Consider problem P1. There exists some p such that the balanced policyis optimal for any initial phase �.(ii) Consider problem P2. By symmetry, the balanced policy with p = 1=K is optimal forany initial phase �.Next, we restrict again to the case of a single objective (K = 1), and show that theresults of the previous section can be extended. More precisely, we show that balancedpolicy is optimal in a stronger sense.Corollary 4.2. Given some p 2 [0; 1], and any � 2 [0; 1], balanced policy ap(�) minimizesthe average cost g(a) over all policies that satisfy the constraint:lim�!1 p1(�) > p1: (34)where pa1(�) is de�ned in (30).Note that the constraint limN!1 1N PNn=1 an > p1 (in Theorem 3.3) implies (34), due toLemma 7.1 in the Appendix. Therefore the minimization in Theorem 3.3 is over a subclassof the set of policies on which minimization is performed in Corollary 4.2. Thus, Corollary4.2 implies that a policy a that satis�es (34) does not perform better than Hajek's policy(with p = p1) even if limN!1 1N PNn=1 an < p1.Proof of Corollary 4.2: Choose an arbitrary policy a that satis�es (34). Choose asubsequence �n " 1 such that limn!1 pa1(�n) = p1. The proof now follows by combiningTheorem 3.2 with (32).RR n�3181

20 Eitan ALTMAN Bruno GAUJAL and Arie HORDIJK5 Admission control in a queueIn this section we give a typical application of the multimodularity theory by looking ata G/G/1 queue with batch arrivals and an admission control a that must accept a givenproportion p of the arriving customers. A more detailed analysis of this system can befound in [2]. The model is illustrated in Figure 3.a lostarrivals ./G/1 queueFigure 3: Admission control in a ./G/1 queue.Let fTigi2N be the sequence of arrival times, with the convention that T1 = 0, the queuebeing empty at time 0. The admission control is de�ned through an arrival sequence. Thearrival sequence is a sequence of integer numbers, a = (a1; a2; � � � ; aN ; � � � ), where ai givesthe number of customers admitted to the queue at time Ti. We also denote by f�kg thesequence of service times in the server.We denote by Wk(a1; � � � ; ak) the workload in the queue at time Tk under the admissioncontrol a. Here, Wk will play the role of the functions fk.The following result is proved in [2]. (The special case of D/D/1 queue is analyzed indetails in the next section).Theorem 5.1. The function E�;TWk(a1; � � � ; ak) (where E�;T denotes the expectation w.r.t.the service times and the inter-arrival times) has the following properties:� E�;TWk(a1; � � � ; ak) is multimodular.� E�;TWk(a1; � � � ; ak) > E�;TWm(ak�m+1; � � � ; ak) >, for k > m.� E�;TWk(a1; � � � ; ak) = E�;TWm(0; � � � ; 0; a1; � � � ; ak), for k < m.� E�;TWk(a1; � � � ; ak) is increasing in ai.The expectation of (any nondecreasing convex function h of) the workload satis�esconditions <1>,<2> and <3>. The general theorem 3.3 applies and the balanced admis-sion policy ap(�) with rate p minimizes the Cezaro limitg(a) = limN!1 1N NXn=1E�;TWn(a1; :::; an);among all policies with rate at least p. INRIA

Multimodularity, Convexity and Optimization Properties 21This example can be generalized. In [2], the traveling time of a customer in an arbitrarynetwork of queues which forms an event graph is shown to be multimodular with respectto the admission sequence. The optimality of the balanced sequence in this case is provedin [2]. More general applications of these results can also be found in a forthcoming paper[3].6 Applications in high-speed telecommunication systemsIn this section we present another illustration of the theorems that are given in sections 3and 4 that we fully develop. Here, we consider a simple model composed of a controlledD/D/1 queue with service times �n = � and inter-arrival times �n = � all deterministic.Assume that the available actions are 0 (corresponding to rejecting an arriving customer)and 1 (corresponding to acceptance of an arriving customer).The type of problem we consider is typical in high speed telecommunications networks,and in particular, to the ATM (Asynchronous Transfer Mode). The latter has been chosenby the standardization committee ITU-T [1] as the main standard for integration of servicesin broadband networks. In order to handle e�ciently a large variety of applications, such asvoice, data, video and �le transfer, cells of �xed size are used, giving rise to our model thatuses �xed service times. Fixed inter-arrival times are typical for isochronous applications(voice, video) and also for large �le transfer.Two important measures of quality of services in ATM networks are loss probabilities(CLR - Cell Loss Ratios) and delays. According to the ATM standard [1], when a CBR(Constant Bit Rate) session is established, the network should provide a guarantee thatthese two measures are bounded by given constants. Since the available sources are limitedand, moreover, might be shared with other applications, a typical objective of the networkis to minimize the delay of the CBR session while meeting the constraint on the lossprobabilities. Losses might be due either to over�ow, or to deliberate packet discardingby the network (e.g. to allow the resources to be available for other applications). Theproblem can be formulated in our framework as one of discarding cells so as to minimizethe average queue size (i.e. the workload in the system) which is known to be proportionalto the average sojourn time (due to Little's law), subject to a lower bound p on the averagecell discarding rate.We now describe the state evolution of the system. If xn denotes the amount of workloadin the system immediately after the nth arrival that occurs after time 0, and the systemis initially empty (at time 0), thenxn+1 = max(xn � �; 0) + an�:The solution of this recursion is given by the expansion of the Lindley equation:xn+1 = fn(a1; :::; an) = max(0; n�1Xk=j(aj� � �); j = 1; :::; n� 1)+ an�: (35)RR n�3181

22 Eitan ALTMAN Bruno GAUJAL and Arie HORDIJKWe show by a simple inductive argument that fm is indeed multimodular for all integersm over Zm. The function x0(a) 4= 0, a 2 f0; 1gN is clearly multimodular, as well as thefunction x1(a) = a1�. Assume that xn = fn(a) is multimodular. xn+1(a) is a convexincreasing function of fn, and is therefore multimodular by Theorem 2.3. Note that theassumptions f(x+ di) 6 f(x); i = 2; :::; N , and f(x� eN ) 6 f(x) indeed hold since a > 0.In particular, the assumption f(x + di) 6 f(x); i = 2; :::; N , follows from the fact thatxi+2 = max(0; ai� � �; xi � 2� + ai�) + ai+1�:Our goal is to obtain a policy a� that minimizes an expected average cost related tothe amount of work in the system at arrival epochs. The cost to be minimized is thusg(a) 4= limN!1 1N NXn=1 fn(a1; :::; an);subject to the constraint: limN!1 1N NXn=1 an > p�:Consider �rst the case of a queue with in�nite capacity. Then, it follows from Theorem3.3 that a balanced policy (with arbitrary �) is optimal. The assumptions of the Theoremindeed hold:� fn (in (35)) is indeed monotone increasing in ai;� Property <3> (in Subsection 3.1) holds by choosing bk = 0, sincefk(a1; :::; ak) = fm(0; :::; 0| {z }m�k ; a1; :::; ak); k < m; (36)� By combining (36) with the �rst monotonicity property, we getfk�1(a2; :::; ak) = fk(0; a2; :::; ak) 6 fk(a1; :::; ak);which establishes Property <2> (in the beginning of Section 3).Consider now a queue with a �nite storage capacity for the workload, i.e. the workloadat the queue at each time instant is bounded by C. When the queue is full, the over�owworkload is lost. Hajek's policy need not be optimal anymore, as the following exampleshows.Counter-example 6.1. (Non optimality of a balanced policy)Let � = 1, � = 100, C = 100, p� = 0:01. Assume that the cost to be minimized is theaverage queue length. Hajek's policy achieves an average queue length of 50:5 for any �.INRIA

Multimodularity, Convexity and Optimization Properties 23Consider now the periodic policy of period 200 that accepts 2 consecutive customers andrejects all following ones. After the second acceptance, the amount of work in the systemis 100 due to the limit on the queue capacity, and there is loss of workload (of 99 units).The average queue length is 25.75. Thus the new policy achieves half the queue length asthe previous one.Although the balanced policy in the above counter example results in a larger queue, ithas the advantage over the other policy of not creating losses. As we now show, a balancedpolicy is optimal if we restrict to policies with the further constraint that no losses areallowed. Thus, consider the class of policies that satisfy the constraint:xn(a1; :::; an) 6 Cwhere xn is given by (35).Since xn is multimodular for all n, the setf(a1; :::; an) : efn(a1; :::; an) 6 Cgis convex for all n. Using now the remark 2.2 and Theorem 3.3, we conclude that a balancedpolicy is again optimal.In the above admission control we considered only the possibility of accepting or re-jecting the whole arriving batch (of 100). In practice, arriving batches may correspond tocells originating from di�erent sources, and it is often possible to reject only a part of thebatch.Assume, thus, that the available actions are a 2 f0; 1; :::; Ng, where a = i meansaccepting i(100=N) units of workload. Assume that the batch size of 100 is an integermultiple of N . We can thus split an arrival batch and accept only a fraction of it; moreprecisely, we can either reject it, or accept 1=Nth of the batch, or 2=Nth, etc.... Thesmallest unit of batch which we can accept (i.e. N�1 is called a mini batch.Consider now the balanced policy a�[N ] that is given in (17) corresponding to p = p�N .In other words, instead of considering a target fraction p of the whole batch to be accepted,which is smaller than (or equal to) 1, the new target corresponds to the average number ofmini batches to be accepted, and can be any real number between 0 and N (in particular,p = N will correspond to accepting N mini-batches, i.e. the whole original batch).We may repeat the above calculation and show that this policy is optimal for the casesof (i) the in�nite queue and (ii) the bounded queue, restricting to policies that do notgenerate losses. Moreover, for both cases, this policy is better than the one that consistsof accepting or rejecting the whole batch according to the policy a� de�ned above, sincea� is a feasible policy in our new problem, for which a�[N ] is optimal (Theorem 3.3).In order to illustrate the last point, consider N = 10. A balanced policy corresponds toacceptance of a mini-batch of 10 units, once every 10 time slots. The average queue lengthobtained by that policy is 5.5, i.e. about ten times less than the one obtained when thewhole batch was to be accepted or rejected.RR n�3181

24 Eitan ALTMAN Bruno GAUJAL and Arie HORDIJK7 AppendixThe following Lemma is often used in applications of optimal control (or games) with anaverage cost criteria (see e.g. [6]) yet it is not easy to �nd its proof in the literature in theformat in which it has been applied.Lemma 7.1. Consider a sequence an of real numbers all having the same sign. Thenlimn!1 1N NXn=1 an > lim�!1(1� �) 1Xk=1 �k�1ak (37)> lim�!1(1� �) 1Xk=1 �k�1ak > limn!1 1N NXn=1 an 4= p (38)Proof. Note that 11� � 1Xk=1 �k�1ak = 1Xk=1 kXl=1 al!�k�1 (39)1(1� �)2 = 1Xk=1 k�k�1: (40)Hence (1� �) 1Xk=1 �k�1ak � p = (1� �)2 1Xk=1 1k kXl=1 al � p! k�k�1: (41)For any � > 0, choose N� such that 1N NXn=1 an > p� �for all N > N�. Then the right-hand side of (41) is bounded below by(1� �)2 N��1Xk=1 1k kXl=1 al � p! k�k�1 � � 1Xk=N� k�k�1!> (1� �)2 " N� max16k6N� ����� kXl=1 al � kp�����!� �(1� �)�2# > �2�for � su�ciently close to 1. This establishes (38). (37) is obtained similarly.INRIA

Multimodularity, Convexity and Optimization Properties 25References[1] ITU-T Recommendation I371, �Tra�c control and congestion control in B-ISDN�,Perth, Nov. 1995[2] E. Altman, B. Gaujal and A. Hordijk, �Admission Control in Stochastic EventGraph�, unpublished, 1997.[3] E. Altman, B. Gaujal and A. Hordijk, �Balanced Sequences and Optimal Routing�,unpublished, 1997.[4] P. Glasserman and D.D. Yao, �Monotone Optimal Control of Permutable GSMPs�,Math. of Oper. Res. 19 pp. 449�476, 1994.[5] Y.G. Sinai, �Introduction to Ergodic Theory� Princeton University Press Princeton,N.J., 1976.[6] R. Sznajder and J. A. Filar, �Some comments on a theorem of Hardy and Little-wood�, J. of Optimization Theory and Appl. 75, No. 1, 1992, pp. 201 - 208, 1982.[7] B. Hajek, �Extremal splitting of point processes", Mathematics of Operations Re-search, 10, pp. 543-556, 1985.[8] R. R. Weber and Stidham, Jr., S., �Optimal Control of Service Rates in Networksof Queues�, Advances of Applied Probability 19, pp. 202�218, 1987.

RR n�3181

Unité de recherche INRIA Lorraine, Technopôle de Nancy-Brabois, Campus scientifique,615 rue du Jardin Botanique, BP 101, 54600 VILLERS LÈS NANCY

Unité de recherche INRIA Rennes, Irisa, Campus universitaire de Beaulieu, 35042 RENNES CedexUnité de recherche INRIA Rhône-Alpes, 655, avenue de l’Europe, 38330 MONTBONNOT ST MARTIN

Unité de recherche INRIA Rocquencourt, Domaine de Voluceau, Rocquencourt, BP 105, 78153 LE CHESNAY CedexUnité de recherche INRIA Sophia Antipolis, 2004 route des Lucioles, BP 93, 06902 SOPHIA ANTIPOLIS Cedex

ÉditeurINRIA, Domaine de Voluceau, Rocquencourt, BP 105, 78153 LE CHESNAY Cedex (France)

ISSN 0249-6399