Adaptive Algorithms for Feedback-Based Flow Control in High Speed, Wide-Area ATM Networks

IEEE JOURNAL ON SELECTED AREAS IN COMMUNICATIONS, VOL. 13, NO. 7, SEPTEMBER 1995 1267

Adaptive Algorithms for Feedback-Based Flow Control in High-speed, Wide-Area ATM Networks

Flavio Bonomi, Debasis Mitra, Fellow, ZEEE, and Judith B. Seery

Abstract-This paper gives a class of flow control algorithms for the adaptive allocation of bandwidths to virtual connections (VC) in high-speed, wide-area ATM networks. The feedback rate to the source from the network is parsimonious, with each feedback bit indicating whether the buffer at a distant switch is above or below a threshold. The service discipline at the switch is First- Come-First-Served. The important goal of adaptability aims to make all of the network bandwidth available to the active VC’s, even though the number of such VC’s is variable over a given range. Each VC has two parameters, one giving its minimum guaranteed bandwidth and the other is the weight for determining its share of the uncommitted bandwidth. Judicious selection of these parameters defines distinctive services, such as Best Effort and Best Effort with Minimum Bandwidth. We derive design rules for selecting the parameters of the algorithms such that the appropriate guarantees and fairness properties are exhibited in the dynamical behavior. The systematic use of “damping” in right proportion with “gain” is shown to be a powerful device for stabilizing behavior and achieving fairness. Our analyses are based on a simple analytic fluid model composed of a system of first-order delay-differential equations, which reflect the propagation delay across the network. Extensive simulations examine the following: (1) fairness, especially to start-up VC’s; (2) oscillations; (3) transient behavior, such as the rate of equalization from ditrerent initial conditions; (4) disparate bandwidth allocations; (5) multiple paths with diverse propagation delays; (6) adaptability and robustness with respect to parameters; and (7) interoperabdity of different algorithms.

I. INTRODUCTION EEDBACK-BASED flow control in wide-area network- F ing suffers from the unavoidable fact that the time con-

stants associated with the adaptive processes are of the order of the large propagation delays, which, when coupled with high speeds, translates to a major limitation. On the other hand, there is a certain broad class of bursty application processes such that, when the demand for bandwidth arises, it persists for time periods which are comparable to the aforementioned time constants or longer. For users of such processes, feedback- based flow control offers value. Also, besides the direct goal of allocating bandwidth, feedback algorithms are in the related business of providing “backward information propagation,” i.e., communicating allocations from the network to the user. Without such information, the user cannot fully exploit the bandwidth allocated to it, no matter how generous, and how efficient and fair the process. These and other reasons help to

Manuscript received September 30, 1994; revised April 1, 1995. F. Bonomi is with the ATM Platform Organization, AT&T Network

D. Mitra and J. B. Seery are with AT&T Bell Laboratories, Murray Hill,

IEEE Log Number 9413100.

Systems, Red Bank, NJ 07701 USA.

NJ 07974 USA.

sources

Fig. 1. Network model.

explain the wide interest and the intensity of recent proceed- ings of the ATM Forum dealing with the Available Bit Rate service category and the role of feedback to support it. The interested reader may consult [3] and references therein.

Thls paper gives a class of flow control algorithms, which, while quite general by design, is primarily intended for the environment where feedback to the user from the network is parsimonious and in the form of single bits. In the base case the rate of the feedback stream is approximately uniform with the interarrival time between bits a parameter, which may range typically from about 0.1 to 1 .O times the round-trip propagation delay, T . We envisage the algorithms being applied to virtual connections (VC) in wide-area networks, perhaps spanning the continental USA, and hence, propagation delays may be large. The feedback information conveyed by each bit is simply whether the instantaneous queue length at the distant switch is above a threshold or not. Thus the feedback is explicit [9]. See [20] and [8] for important early work on adaptive, feadback-based algorithms.

Fig. 1 is a sketch of the network model considered here. To keep matters simple, we consider only a single switch in the paths of the various VC’s. The figure shows various propagation delays for the VC’s; hence, the sources may be located at various geographical sites. The figure shows p and QT to be, respectively, the service rate and the queue threshold at the distant node. The packet or cell size is assumed to be fixed, not randomly distributed. Variability in this study is due solely to the dynamics of the adaptation and the turning on and off of the VC’s.

Our main contributions are as follows. We formalize certain desirable properties of bandwidth allocations which deal with fairness and guarantees on minimum bandwidth, and identify dynamical behavior which converges to give such properties. We give a unified framework for designing a broad class of dynamical feedback algorithms which satisfy the aforementioned

0733-8716/95$04.00 0 1995 IEEE

~

I268 IEEE JOURNAL ON SELECTED AREAS IN COMMUNICATIONS, VOL 13, NO 7, SEPTEMBER 1995

properties. We show that the systematic use of “damping” is a powerful device for stabilizing behavior and for achieving fairness; when damping is used in the right proportion with “gain”, then the disadvantages of damping, namely, reduced dynamic range and sluggish transient behavior, may be effectively nul- lified. Finally, we demonstrate through simulations that, when properly designed, the algorithms demonstrate adaptability and performance, which is fair, stable and responsive.

Each VC has two nonnegative parameters associated with its bandwidth allocation, v3 and cj, for virtual connection j. The rate parameter vj is the minimum bandwidth and o3 is the positive weight given to the VC in determining its share of the uncommitted bandwidth. Various services with distinctive characteristics may be designed by appropriate choice of these parameters. Best Effort and Best Effort with Minimum Bandwidth are examples of such services given in Section IV.

The algorithms given in this paper are for VC’s which share a common buffer at the network node and the service discipline at the node is First-Come-First-Served. (In separate work we have examined various weighted round robin disciplines, where fairness is more easily obtained.) An important reason for considering the shared memory-FCFS implementation is its simplicity and low cost. Similarly, in today’s technology, processing capacity is at a premium, and parsimonious feedback together with a relatively low rate of updates, is highly desirable for its low processing requirements.

The analyses in Sections I11 and IV are based on an analytic fluid model which approximates the algorithms. The analytic results provide guidelines for the design of parameters of the algorithms. The results show that there are two distinct operating regimes, Unsaturated and Saturated. In steady state, the former has unutilized network bandwidth, oscillation-free flow and empty buffers. In the latter regime all bandwidth is utilized, and both flow and queue behavior oscillates around p. and QT, respectively. An important attribute of the algorithms is that as the degree of saturatedness increases, as happens with increasing number of VC’s, the amplitude and energy of the oscillations increase gracefully. That is, there is a very broad range of parameters in the Saturated regime for which performance is acceptable in terms of high utilization and low delay. This robustness is at the core of the adaptability of the algorithms. That is, when the number of virtual circuits J is allowed to vary over a broad range and parameters of ongoing connections are not rescaled with J, performance goals continue to be satisfied.

An important goal is to make all of the network bandwidth available to the active VC’s, with the individual allocations commensurate with their allocation parameters. Our simulation studies which are reported in Section V make the assumption that when a VC turns on it acts like an infinite data source, i.e., with unbounded data to send. Hence, even when there is only a small number of VC’s on, the utilized fraction of the

bandwidth is close to unity, and this is also true at the other extreme when the maximum number of permissible VC’s are on simultaneously.

We briefly review some prior work. The algorithms proposed and designed in [ 151-[ 171, [6] were based on packet-by- packet adaptation and driven by response time measurements. In both these respects the present work is different. Elwalid [5] has developed a fundamental theory, based on delay- differential equations, of response-time based adaptations in the presence of large propagation delays. He shows how damping and gain parameters may be selected to optimize transient behavior. Reference [ 181 first gave the basic symmetric algorithm discussed later in this paper, and also the design rules for the algorithm.

We list some key issues which are examined in this paper, as well as in various other prior studies.

Fairness in short and long term behavior. This is spe- cially important for a start-up VC, which has difficulty in obtaining its share of the bandwidth from existing VC’s. Large amplitude oscillations. Zhang [23] has shown that they exist in certain algorithms. Fendick et al., [6] give a beautiful treatment in which they prove that such oscillations are controlled to acceptably small magnitudes in a class of algorithms. Mukherjee and Strikwerda’s analysis [ 191 illuminates the possibility of large amplitude oscillations in the JRJ algorithm. Transient behavior, i.e., the quality of the responsiveness in a dynamic environment. Mishra and Kanakia [14], Keshav [l 11, Bolot and Shankar [2], Mackrucki [ 131, Benmohamed and Meerkov [l], Hluchyj and Yin [7], Kanakia et al. [ 101 and Keshav [ 121 provide studies in which algorithms are proposed and investigated with the focus on transient behavior. Arbitrary bandwidth allocation to VC’s. Various propagation delays. Zhang [23] has shown that there is typically a tendency to discriminate against VC’s on longer paths. Robustness with respect to parameters.

11. ALGORITHMS

this section we begin by giving fairly general asynchronous versions of the algorithms for rate control. Next, the algorithm is specialized to some specific implementations and to the synchronous version, which is followed by some simple enhancements. Finally, the counterpart for windows’ control is described.

A. Rate Control

In the description in (1) (below) R3(t) denotes the rate allocation to the jth VC just after an update at time t. The algorithm is asynchronous, and hence, the VC’s which have their rates updated simultaneously at t form an indeterminate

BONOMI er al : ADAPTIVE ALGORITHMS FOR FEEDBACK-BASED FLOW CONTROL 1269

set. Let U ( t ) denote this set. The number of elements in U ( t ) may range from 1 to J , where J is the total number of VC’s. In (1) U,@) E { -1, l} is the feedback received at the source at time t just prior to the update in the rate allocation, from the distant switch. An update may be triggered by various events, including the expiration of a timer, a counter reaching a predetermined value, or the receipt of feedback information. As indicated in Fig. 1 it is assumed that the propagation time between source and distant switch for VCj is 7 j / 2 in each direction. This assumption is particularly justified in wide-area networks, where the propagation delay far exceeds other components of the transport delay. Some results on the robustness of our rate control algorithms with respect to this assumption will be presented in Section V. The feedback bit u j ( t ) is +1 or -1 depending upon whether the queue at the distant switch at time (t - 7-j/2), Q(t - 7,/2) < Q T , or 2 QT, respectively. The rate parameter vj in (1) is the user’s expectation of minimum bandwidth. Also, yj(u) is the damping function and a,(u) is the gain function, which are nonnegative design parameters of the algorithm and the subject of discussions later in the paper. We only consider the following special forms of these functions

Particular attention is given to two algorithms, which are specified by the following choice of parameters.

Jacobson-Ramakrishnan-Jain (JRJ) [8], [20]

Mitra-Seery (MS) [ 181

(3)

(4)

Thus, the JRJ algorithm is asymmetric in the expansionary and contractive phases, in contrast to the symmetry of the MS algorithm, which is uniformly damped.

In (1) d j ( t ) is the “delay,” or time interval, between the prior update of the rate of VCj and the update at time t.

The asynchronous nature of the algorithm in (1) allows substantial generality in implementations of feedback control. By specializing (1) various specific updating and congestion detection schemes may be accommodated. For instance, ATM Resource Management (RM) cells may be used to carry back congestion information to the source. These RM cells may be transmitted periodically, with the period determined by the geographical length of the virtual connection. Another scheme, which too is described by (l), is where a RM cell is transmitted

for every N data cells transmitted. In the latter scheme the overhead traffic is kept to a fixed proportion of the useful data traffic, and has the attraction of scalability in the framework of many small virtual connections. The reader may consult [3] for further information on these and other schemes.

The asynchronous model in (1) may be specialized to the following synchronous model, which is itself widely applicable and also helpful as an intermediate between (1) and the fluid models of succeeding sections. Let A, denote the fixed interval between updates of the rate for VCj, and abbreviate R, (.A,) to R,(n), n = 0 ,1 , . . . . Then

R,(n) = R,(n - 1) - r,(.,(n))A,[R,(n - 1) - U31

+a, (U3 (.))A, U , (n ) ( 5 )

where u,(nA,) has also been abbreviated to u3(n).

B. Floor and Ceiling

and “Ceiling.” First, define the function sat, (.) thus We modify (1) to incorporate two new mechanisms, “Floor”

= x , if u3 5x5 RT1 - Rcell - , if x > RYil

sat,(z) = u j , if x < uj (6)

where RTi1 is a parameter of VCj. The modification to (1) is shown in (7) at the bottom of this page.

The motivation for Floor is that it guarantees that at all points in time the rate allocated to VCj is not less than v,. It’s main effect is during start-up. As will be seen later, even without Floor the algorithms dynamically converge to satisfy this rate requirement.

The motivations for Ceiling are different. One such motivation arises when the design is for possibly many VC’s, but only a few VC’s are active. Ceiling restricts the rates of these active VC’s, so that the surge in the buffer at the distant node is mitigated when several VC’s turn active at about the same time.

C. Window Control

The window for a VC restricts the number of packets or cells which have been transmitted but not acknowledged as having been received by the distant switch. Windows and rates are closely related. Our simulation studies have covered both. A major difference in our algorithms is that we have an “ideal window” and an “actual window”, which is necessitated by the restrictions that apply to the actual window. For instance, when it is desired that the window contract, it may only do so one packet at a time by skipping a packet transmission when an acknowledgment is received. The ideal window, like rates, is not subject to such restrictions. The ideal window is adapted similarly to rates, while the actual window attempts to track the ideal window.

1270

I

IEEE JOURNAL ON SELECTED AREAS IN COMMUNICATIONS, VOL. 13, NO 7, SEFJTEMBER 1995

Let IC:')@) denote the ideal window of VCj at time t. Similar to (l), (8) is shown at the bottom of this page. Here KFin is related to the minimum rate vj; the specific relation is clarified in the next section. The functions y3 ( p ) and u3 ( p ) are restricted as in (2).

Let Kj( t ) denote the actual window of VCj at time t. As mentioned above, a decrement of the actual window is necessarily of size 1. While increments may be larger, the description in (9) (at the bottom of this page) is for the case where increments are also of size 1.

111. MODELS AND ANALYSIS FOR A SINGLE CONNECTION

In this section, we concentrate on the case of just one virtual connection ( J = l), and prepare for the case of multiple connections, which is treated in the next section. A fluid model is introduced to approximate the algorithm described in Section I1 and certain basic properties of the model are established. In this section we suppress the virtual connection index j.

A. Model

The model for a single VC is

(10) r+[q5(t) - V] + A+u(t), if u(t) > 0 r-[4(t) - V] + A-u(t) , if u(t) < 0

and u(t) = sgn[QT - Q(t - 7/2)]. In (1 l), [XI+ = max(z,O), and I?+, r-, A+, A- are nonnegative parameters. Here 4(t) is the flow rate at the source transmitter and represents the fluid approximation to the local (in time) throughput of packets or cells. The classical queueing theoretic steady state throughput is approximated by the long-term average: limt,m s,' $(s)ds. In (1 1) Q(t ) is an approximation to the queue in packets at the distant switch, and QT denotes the threshold. The nonlinear dynamics in (1 1) reflect the fact that the buffer content is nonnegative, i.e., Q(t ) 2 0. A similar restriction also applies to the rate 4(t) , but this is not modeled. The reason is that +( t) is typically large in our applications and the "floor" of 4(t) plays an insignificant role. This contrasts with our objective of keeping buffer contents small.

The correspondence with the description of the algorithm in (1) is obtained thus: 4(t) M R(t), and

Hence, the parameters of the models in (1) and (10) have simple correspondences

r + M y + , r- M Y - , A + % : + , A- M U - . (13)

The fluid model (lo)-( 11) may also be used to approximate the adaptive window algorithm, but the correspondences are somewhat different. The amount of fluid in the virtual connection at time t corresponding to a window K(')( t ) is approximately T d ( t ) . Proceeding as before, we let

d K(')( t ) - K(')(t - d ( t ) ) d ( t )

T - $ w M

and, matching coefficients of (8) and (lo), we obtain the following correspondences

r+ M ?+, r- M y ; ~ + M u + / ~ ,

A- M u - / r ; v M Kmin/7. (14)

In particular, the important ratios are A+/I'+ = S(uf/rf)

Our design guidelines are first derived for the fluid models and then translated to the algorithms. Hence, a noteworthy point is that the design of algorithms for rate adaptation (see (13)) is independent of the propagation delay 7, while the algorithm for window adaptation (see (14)) depends on 7.

Equation (10) for the JRJ and MS algorithms has the following forms

d dt

and A - / r - = $(u-/y-).

(15) JRJ: -4(t) = Au(t) , if u(t) > 0

= -r[4(t) - V I , if u(t) < 0

B. Regimes

The rate parameter v in (1) and (10) is the user's expectation of minimum bandwidth. For this guarantee to hold it is necessary that admission control ensures

V F PP (17)

where p is the long-term channel utilization. In our investiga- tions we have found p to be close to unity, more specifically in the range (0.80, 0.98).

Consider now the existence of stationary solutions, i.e., solutions of (10) and (11) with $4(t) = &Q(t) = 0. The only possibilities are

4 = v + A+/I'+, and 4 = v - A - / r - (18)

BONOMI er al.: ADAPnVE ALGORITHMS FOR FEEDBACK-BASED FLOW CONTROL 1271

corresponding, respectively, to U G +1 and U = -1. For the latter to hold, Q > QT and 4 > p, and since p > v from (17), it follows that 4 > v, which contradicts the expression in (18). Hence, we may rule out U -1. The former stationary solution in (18) exists provided (v+ A + / r + ) < p. Hence, we

and hence, the approach to All? from above is exponentially fast.

We now give a basic boundedness result which holds for systems, possibly asymmetric, which are ''uniformly damped," i.e.

introduce the following Dejnition: If (v + A + / r + ) < p then the system is

Unsaturated, and if (v + A + / r + ) > p then the system is Saturated.

Proposition 3.1: (1) If the system is Unsaturated then there exists a stationary solution 4 = Y + A + / r + , Q

(2) If the system is Saturated then no stationary solution exists. 0

Note that for the JRJ algorithm the system is always Saturated, since I'f = 0. The following summarizes the dynamical behavior of Unsaturated systems.

0.

Proposition 3.2: Suppose the system is Unsaturated. 1) For any t l , if 4(tl) 5 v + A + / r + , then 4(t) 5

v + A + / P for all t 2 tl . If $( to) > v + A + / r + , then there exists tl (tl > t o ) such that @(t) decreases monotonically in [to, t l ) and 4(tl) = v + A + / r + .

2) There exists t2 such that for all t 2 t2

Q ( t ) < Q T , 4(t) --f (v+ + A + / r + ) and

at the exponential rate r+. Proof: (1) If $( t ) > v + A + / P then from (10)

d z$(t ) 5 max[-r+$(t) + A + , -r-$(t) - A - ] < 0 (19)

where

$( t ) 4 f $ ( t ) - v. (20)

(2) Say f$(t) 5 v + A + / P for all t 2 t l . Since the system is Unsaturated

4(t - 7 / 2 ) - p < 0 ( t 2 tl + 7 / 2 )

and consequently, from (1 I ) , for all t 2 tl + 7-/2

d < 0, if Q ( t ) > 0 d t Q ( t ) = 0, if Q ( t ) = 0. -

Hence, after (tl + 7/2) the buffer content decreases monotonically till the buffer is empty and thereafter it remains empty. Hence, in particular, there exists tasuch that for all t 2 t2

Q ( t ) < QT and ~ ( t ) E 1.

Hence, from (lo), for all t 2 t 2

f$ ( t ) = [ v + A + / r + ] [4(t2) - - A + / r + ] . (22)

Since r+ > 0 for Unsaturated systems, the result follows. 0 Observe that the queue threshold QT does not have a role

in determining the steady state values of Q ( t ) and 4(t) . We remark that for the symmetric MS algorithm it is

possible to sharpen the statements made above. For instance, it follows from (19) that for this case

rM min(r+ , r - ) > 0. (24)

Note that this result holds regardless of whether the system is Saturated or Unsaturated.

Proposition 3.3: Suppose the system is uniformly damped and let AM A max(A+,A-) . Then for any t o and t > t o

I4(t) - V I F AM/rM + e-r"(t-to)[l$(to) - V I - A M / ~ M ] . (25)

Proof: With $(t) defined in (20) and g\$(t)\ defined as the right derivative [4]

d+ d --I?Nt)l dt = [s~n$(t)l$(t)

[sgn$(t)][-r+$(t) + A + ] , if u(t) > 0 [ sgn$( t ) ] [ - r -$( t ) - A - ] , if u(t) < 0 -I'+l$(t)l + A+[sgn$(t)] , if u(t) > 0 -r-I$(t)l - A-[sgn$(t)] , if u(t) < 0

= I = {

(26) I -rMI$(t)l + A M .

Hence, for t > t o

I$(t)l 5 AM/rM + e-r"(t-to)[($(to)( - A M / ~ M ] . (27)

0

C. Periodic Solutions

We proceed to the Saturated regime where it is known from Proposition 3.1 that no stationary solutions exist. We show (see Fig. 2) by construction that periodic solutions (limit cycles) exist in this regime. To keep it simple, the construction is for the MS algorithm with v = 0. The technique may be extended to cover other cases, albeit at the cost of introducing various tedious details. This construction is related to one in [6], with the main difference being the presence of damping here; also, [6] considered various nonlinearities. See also [ 191 for construction of limit cycles for the JRJ algorithm.

Start by postulating a point t o such that for t E [ to - 7/2, to]

Note that the postulated f$( t ) is a solution of the equations

We will show that at a later point, t 3

4 ( t 3 - s ) = 4(to - s ) , and

(0 I s I 7 / 2 ) . Q(t3 - S ) = Q(to - S ) (30)

This will establish the existence of a periodic solution to (IO)-( 11). There are three phases present in the limit cycle.

1272 IEEE JOURNAL ON SELECTED AREAS IN COMMUNICATIONS, VOL. 13, NO. 7, SEPTEMBER 1995

0

I I and (40) are identical.

-I 1

Q(t - 7/2) < QT (t E Phase 1). (31) 0 = 4 2 [ ( 5 + e) + ( p + e) '1 (41) Phase 1 terminates at ti where

4 = p + A7 + J2QTA (42) &(ti - ~ / 2 ) = Q T . (32)

Hence, from (10) and (1 l), u(t) = 1 for t E Phase 1. Hence

1 1'2 -4(t) d = -I?d(t) + A (t E Phase 1). (33) 6 = p - AT - AT + Jm)' - . dt

Now, since the system is Saturated it follows from (32) that (43)

Iv. MULTIPLE CONNECTIONS: FAIRNESS AND DESIGN 4(to) > p, and hence

d 4(t) > pl --4(t) > 0 (t E Phase 1). (34) This section breaks from the preceding section in focusing

on multiple connections and the Saturated operating regime. Fairness is a dominating theme with multiple connections and most of this section is concerned with, first, defining and then achieving, through design of algorithms, various forms

Hence d rlt

= 4(' - 7/2) - p > ( t E Phase '). (35) -I

of fair behavior. The Saturated operating regime is the one of choice, even at the cost of oscillatory system behavior, since capacity is more completely utilized. Finally, the topic of interoperability, i.e., the co-existence of diverse algorithms,

The time tl , see (32), is calculated from (33) and (35). As shown in Fig. 2, Q ( t l ) overshoots QT.

Phase 2 terminates at t 2 where

Q ( t 2 - 7/21 = QT (36) is investigated.

A. Fairness and

QT < Q(t - 7/2) (t E Phase 2). (37) Each VC has two associated parameters for purposes of

Hence, the governing equations for Phase 2 are bandwidth allocation, (vj,cj) for VCj. Where 4 j is the bandwidth realized by VCj, it is desired that

(38) and

(44)

4 ' I - v ' r Note that dqh/dt < 0. Hence, Q ( t ) is concave and reaches a maximum value Q at time denoted by t;. Also, 4(tz) < p.

' j - vj = U , for all pairs ( j , j ' ) . (45)

Note that { cj} are relative weights and hence, are not required to sum to unity. It is tacitly understood that admission control

* j aj I

In Phase 3

4(t - 7/2) < p and Q(t - 7/2) < QT, (39)

BONOMI er al.: ADAPTIVE ALGORITHMS FOR FEEDBACK-BASED FLOW CONTROL 1273

is administered to enforce the condition Then

< PP (46) j

where p is the network bandwidth, and p is the utilization. Admission control is not necessary if uj = 0 for all j.

Service classes of particular interest are, 1) Best Effort: uj = 0, ai = a > 0. 2) Best Effort with Minimum Bandwidth: uj > 0, aj =

Other services may similarly be defined by appropriate selection of the parameters uj and aj. Typically service will be mixed, i.e., service classes will coexist with each class comprising of several VC’s.

a > 0.

B. Model

We extend the model in (10) and (1 1) in the natural way. Let (I?;, AT, r; , A;) be parameters associated with VCj in this model. For j = 1 , 2 , . . . , J

-rT[&(t) - vj] + Afuj(t) , if u j ( t ) > 0 --r;[4j(t) - u j ] + A;uj(t), if u j ( t ) < 0

and u,(t) = sgn[QT - Q(t - 7-,/2)]. For the JRJ algorithm r; = A; = 0 for all 3 ; let r3 = r; and A, = A;. For the symmetric MS algorithm I?; = r; = I?, and

The system is defined to be Saturated or Unsaturated depending upon whether E,(., + AT/I‘T) is greater or less than p. respectively. The implication of the regime on the existence of stationary solutions, which is stated for a single VC in Proposition 3.1, also holds for multiple connections. Proposition 3.2 extends naturally and states that the dynamical behavior is such that the stationary solution is approached from all initial conditions if the system is Unsaturated. Finally, dynamical behavior is oscillatory if the system is Saturated, but bounded, as stated in Proposition 3.3.

In the Unsaturated regime the unutilized bandwidth is [p - E, ( v, + A; / r ; ) ] > 0. This is a strong incentive to operate in the Saturated regime. Also, it is typically inevitable that systems will sometimes operate in the Saturated regime. This is because, in long time scales, the number of virtual circuits, J , will vary substantially and, even if the design is Unsaturated for small J , it will typically be Saturated at the other extreme of large J.

3 ’ AT = A- A

C. Pointwise Fairing

We show here that with proper design, the divergences from fair bandwidth allocation in all our algorithms vanish monotonically, even exponentially in certain cases. Define

-rfej(t) + Afu(t), if ~ ( t ) > 0 (50) d t -I’;dj(t) + Aju( t ) , if u(t) < 0

where u(t) = sgn[QT - Q(t)]. Note that with this change of variables the forcing function u(t) is common to all j in (50). Also

We propose the following design rule: for j = 1,2, . . . , J

(52)

where A+, A-, r+, r- are nonnegative constants common to all VC’s.

AT = A+a, r: = A- = A - ~ , r- = r-

From (51) and (52), for any pair j , j ’

Define Vjjt (t) to be the “divergence from fairness,”

(55) d d t -vjjf(t) = - C ( t ) V j j / ( t )

where

c(t> 2 r+&,o + n+)<,,. (56)

Note that c ( t ) 2 0, for all t. Now, from (55) , for any to < t

Vjj, ( t ) = Vjjf ( t o ) exp [ - lot c(s )ds] . (57)

If the system is Unsaturated, then V,,t(t) + 0 as t + 03,

since, as previously noted, &(t) -+ U, + A;/J?,f = v, + a,A+/r+. If the system is Saturated, then the expansionary and contractive phases recur infinitely often and consequently (see 56), st’, c ( s )ds + 03 as t -+ CO. Hence, from (57), in this case too, V,,t(t) -+ 0 as t + 00.

This property, wherein the divergences from fair bandwidth allocation vanish monotonically, is what we mean by “pointwise fairing.”

Thus far the focus has been on the comparative behavior of VC’s. Insight into the aggregate behavior is obtained by summing (50) over all virtual circuits. Let

J

O ( t ) 2 e,@). (58) ,=1

I


Then, from the design rule (52)

-r+O(t) + A+(& aj)u( t ) , if ~ ( t ) > 0 -r-O(t) + A-(C, aj)u(t>, if u(t) < 0.

(59)

That is, O(t) behaves like the bandwidth of a single VC for which the algorithm parameters are I?+, A + ( C j a j ) , r-, A - ( C j a j ) . The analysis of Section I11 may be brought to bear on this quantity.

Recall that our prior results in this section have shown that O j ( t ) + ajC(t), for all j , where C ( t ) is a common function. This fact gives

The following is a summary of our results, which hold regardless of whether the system is Saturated or Unsaturated.

Proposition 4.1: Let the parameters of the algorithm be chosen by the design rule (52), and let Vjjt(t) in (54) be a measure of the divergence from fair allocation for any two VC's, j and j ' .

1) For all initial conditions, Vjjl(t) is monotonic, nonin- creasing and approaches 0 as t -+ 00.

2) Let O(t) in (58) denote the aggregate bandwidth. Equa- tion (60) gives the asymptotic behaviors of bandwidths of individual VC's in terms of the aggregate. O(t) is obtained as the solution of the model for a single VC. 0

Let us now examine in somewhat greater detail the implications of the design rule on the JRJ and MS algorithms. For the JRJ algorithm, from (56)

0, if u(t) > 0 I?, if ~ ( t ) < 0. c ( t ) =

That is, the divergence from fairness remains constant during the expansionary phases and decreases exponentially during the contractive phases.

In contrast, for the symmetric MS algorithm, for all t

and consequently

i.e., the approach to fairness is exponentially fast with rate r. This is an important and attractive feature of the algorithm.

D. Averaging Based on Periodicity

The results of this subsection apply only to the MS algorithm in the Saturated regime. Specifically, symmetry in the general form of the algorithm is used in an essential manner; also, the results here depend on the periodicity of {&(t)} and Q(t ) , which are the observed forms of behavior of the Saturated system. The benefit is a relaxation of the design rules in (52), which is of considerable convenience in the design for VC's with very different geographical lengths and updating frequencies. However, we give up exponential pointwise convergence to fair bandwidth allocation for "averaged fairness,"

i.e., the fair allocation of the time-averaged bandwidths of the VC's.

d

Specializing (47) to the MS algorithm, we obtain

- + ~ t ) d t = -rj+j(t) + A ~ U ~ ( ~ ) ( j = 1 , 2 , . . . , J ) (64)

where + j ( t ) solutions of (64) with period T, i.e., for all j and t

4 j ( t ) - u j . Consider continuous periodic

+,(t) = +,(t + T ) , and U,@) = u,(t + T ) . (65)

Integrating (64) over the interval Is, s + T] and noting that JSs,"" &$,(t)dt = 0, we obtain

s+T s+T

+j( t )d t = (A,/r,) 1 y,(t)dt. (66)

Define the time averages s+T

(4,) ; 1 4, ( t )dt , (4 = * - ls+T U , ( t )d t . (67)

Then from (66), observe that the second integral is independent of j .

which is an important result.

condition for any pair ( j , j ' ) Let fair behavior in the averaged sense refer to the following

(69) ( 4 3 ) - U3 - (43 ' ) - U,' -- g.? g, I

From (68) we see that fairness in the averaged sense is achieved by the MS algorithm provided the following design rule is followed

A, / rJ = (constant)a, ( j = 1 , 2 , . . . , J ) (70)

where A, 2 0, r, 2 0. This is a considerable relaxation over the design rule in (52). Of course, if (52) is satisfied, then so is (70).

We note in passing that if the system is Unsaturated then such a design rule gives the desired behavior in (69), provided ( 4 J ) is interpreted as the stationary value of $J~, for each 3 .

A necessary condition for periodic behavior is that the system is Saturated. In this case

j

Combining (68) and (71)

Hence, if admission control is administered, i.e., then ( U ) > 0, and consequently, from (68)

uj < pp,

(4j) > V j (73)

i.e., the soft version of the guarantee on minimum bandwidths is satisfied.

BONOMI et al.. ADAF'TIVE ALGORITHMS FOR FEEDBACK-BASED FLOW CONTROL 1275

E. Averaging Based on S m l l Amplirude Oscillations

When we consider algorithms, like the SRJ algorithm, which behave asymmetrically in its expansionary and contractive phases, the arguments in the preceding subsection leading to averaging based on just periodicity break down, as we shall see. To cope with this situation we make the more restrictive assumption of small amplitude oscillations to arrive at equations for the time averages of rates. These equations also apply to the symmetric MS algorithm, and it is reassuring that there is agreement between the result thus obtained and the prior result based on periodicity. The assumption made here states that the oscillations in the rates have high frequency, and hence may equivalently be viewed as a form of time scale separation.

From (47)

where $j (t) = q5j (t) - uj. Assume, as in (65), that a periodic solution of period T exists. Let

IUJ(,)<0dt (75) Ls+T s+T

T + P L luJ(t)>odt , T - ' and let p denote the fraction of time that the feedback signal is positive, i.e.

T + p = - T + + T - '

Integrating (74) over [s, s + T ] gives

rf

(77)

since, from periodicity, Jss+T &$j( t )d t = 0. Let

1 s+T

($;) ' T+ J + j ( t ) L , ( t ) > o d ~ , a n d

1 s+T

($7) ' T- w ) L J ( t ) < o d t . (78)

From (77) and (78)

p[rf($;) - AT] + ( 1 -p)[I';($3r) + A i ] = 0. (79)

In the asymmetric case this line of reasoning does not take us any further.

It is instructive at this point to examine the JRJ algorithm for a single VC.

Typical solutions are sketched in Fig. 3. Solving (go), we get

$ m a = $min + AT1 $,in = $m,e-rT2.

Hence

t

(82)

The oscillations are of small amplitude, i.e., ($max - $min)/$max << 1, if and only if rT2 is small.

We proceed by returning to (79), which was obtained from periodicity, and make the additional assumption of small amplitude oscillations. Then, see (78),

$max - $min - - - e - r T z .

$ m a

($7) = My). (83)

Let (q3) denote the (approximately) common quantity in (83). Substituting in (79) we obtain for the general algorithm

PA,' - ( 1 - p ) A i ( j = 1 , 2 , . . . , J ) . (84)

pr; + (1 - p)r; (IlJ =

In the special case of the JRJ algorithm (84) yields

and for the MS algorithm

(86)

On noting that ( 2 p - 1) = ( U ) , we see that (86) is equivalent to (68). This is reassuring because (68) was obtained from substantially weaker assumptions.

We may make use of (85) to design the JRJ algorithm for fair behavior in the averaged sense, which is described by (69). The design rule is simply (70).

Note that since the right-hand quantity in (85) is positive, it follows immediately that

A ($3) = (2P - 112 .

r3

($3) = ( 4 3 ) - V3 > 0 (87)

for all j , i.e., the soft version of the guarantee on minimum bandwidth is satisfied by the JRJ algorithm.

We end this subsection by reminding the reader that the results of this subsection are based on the assumption of small amplitude oscillations, which limits the applicability of the results. However, further inferences are drawn from this model in the following subsections and checked with simulations. The results are reported later. It suffices to mention here that the results support the usefulness of the analysis.

1276 IEEE JOURNAL ON SELECTED AREAS IN COMMUNICATIONS, VOL. 13, NO. I , SEPTEMBER 1995

F. Discussion of Design Procedures

Here we recapitulate and discuss briefly the implications of the preceding analyses on design and operating procedures. Our approach is to do our primary design for the continuous-time fluid models and subsequently translate back to the parameters of the actual algorithm in (1) and (5). The advantage is that the fluid models are more tractable and provide a uniform framework. The disadvantage is that the translation can introduce undesirable artifacts when the time intervals between updates in the algorithm are excessively lengthy. However, the latter should be avoided in any case. The translation is effected by (13) for rates and by (14) for windows.

An important feature of the algorithms discussed here is the extraordinary robustness of performance over a broad range of parameters corresponding to various degrees of saturatedness. This feature will be amply exhibited in the simulation results to be presented in the next section. To be specific, let us consider design for pointwise fairing and refer to (59), which gives the governing equation for the aggregate bandwidth e(t) . Consider P, A+, I?-, and A- to be fixed, while J and therefore (E,”=, 03) are varied. As the latter quantity, which, say, is denoted by C J , is increased, the system becomes increasingly Saturated with increasing energy in the fluctuations of O(t) and Q(t), the buffer content. However, the growth in the fluctuations is surprisingly slow and over an order of magnitude change in C J the realized bandwidth remains close to the network bandwidth, p.

The aforementioned feature is, of course, central to the adaptability of the algorithm, by which we mean robust behavior over a wide range of the number of VC’s, J . This is essential since J and CJ vary with time, and when new

that of VC’s in group 2. This should not be surprising since we have seen (recall (61) and (62), for instance) that at a detailed level the algorithms share bandwidth differently. The implication, however, is that interoperability of algorithms is very limited.

Here we show that interoperability of the JRJ and MS algorithms can be designed for certain limited frameworks. Suppose that VC’s 1 ,2 , . . . , Nl comprise group 1 and VC’s (NI + l)? . . . , ( N I + N2) comprise group 2. Further suppose that the goal is to provide Best Effort service, i.e., vj E 0 and C J ~ = CJ for 1 5 j 5 ( N I + N2). Accordingly let

= Al/rl, j = 1 , 2 , . . . , N~ j = (Nl + 1), ( N I + a ) , . . . , (Nl + N 2 ) .

(88)

A3/r3 =

Now, from (85) and (86), we require

(89)

and, from (71)

For given N I and N2, we may solve (89) and (90) for (Al/rl)/(AZ/rz). Let

denote the fair share of bandwidth for each VC. Then

A1 - A 2 / r 2 - 4 * - - I‘l ’* A2/I‘2 + 4* .

connections are set up or old connections tom down, the other connections’ parameters are not modified. Such adaptability may be thought of as single scale adaptability. The robustness discussed above breaks down when C J is varied over several

Hence, if the above design rule is followed then in steady state all VC’s will receive a fair share of bandwidth. Observe that as expected, in this design

orders of magnitude. To cope with this situation, i.e., to extract multiple scale adaptability, it is necessary to rescale parameters. For example, we may consider scales in which 1 5 J 5 20 and 21 5 J 5 200, and the dwell time in each scale to be fairly long. In this case, we propose that the parameters of all VC’s are rescaled at the time of a change of scale.

G. Interoperability

So far we have covered systems in which all VC’s operate according to a common algorithm. What if some VC’s, say in group 1, operate according to the JRJ algorithm and other VC’s, say in group 2, operate according to the MS algorithm? Our prior analysis allows us to predict steady state behavior in such a configuration. First, all VC’s in each group behave fairly, in accordance to the design rule employed. However, by contrasting (85) and (86) it is clear that representative behavior of the two groups will be different. For instance, if all VC’s have a common value of A J / r 3 , then since {p/(l - p)} > (2p - 1) for all p in (O,l), it follows that the bandwidth obtained by VC’s in group 1 will be more than

(93)

Simulation results confirm that the algorithm design in (92) gives fair behavior.

V. SIMULATIONS

Extensive simulations of the algorithms in Section I1 were done in order to illustrate, qualitatively and quantitatively, some of the issues discussed above. The simulations used PANACEA [21], a software package for the performance of queueing networks. The adaptive controls, which are nonstandard in the queueing network framework, were implemented by exploiting the “user-escape’’ mechanism designed for nonstandard applications [22]. The simulated entities are ATM cells.

In this presentation we limit our consideration to: Rate-based flow control. Window-based implementations have also been simulated, but are not described here. Infinite data sources, i.e., once it turns on, a VC continues indefinitely to generate traffic at the controlled rate.

BONOMI er al.: ADAPTIVE ALGORITHMS FOR FEEDBACK-BASED FLOW CONTROL

rA=.OOE

40-

m-

0

1277

VCL

-aucq I I I I

. . . . . . . . . . . . . .

MS

1: 0 0.5 I 1.5 2 2.5 3 3.5 4 4.5 5 5.5 6 6.5 7 1.5 8

--- I MS

MS 140-

im-

- 100- - 0 PP 80- sg

ii 40-

m- 0 1-y I I I I I L

0 0.5 I 1.5 2 2.5 3 3.5 4 45 5 5 5 6 65 7 7 5 Tiny (sec)

(b) Fig. 5. MIS, T~ = 10 ms, A, = 5 ms, U, = 1.0, ( j = 1,2), and QT = 6.5 cells. MS algorithm.

Various gains and damping constants with ratio a/? held fixed at 108 MWs and v = 34 M/s. VCl turns on at time 0 and VC2 at 2.5 s. f l = 155

Periodic rate updating, following the synchronous model of (5 ) in Section II. The updating interval ranges from 0.57- to 2.07. Single point of congestion, or queue, characterized by the FCFS service discipline; buffer overflows are not modeled. Fixed packet size of 53 bytes, the standard ATM cell size, and transmission rates of either 155 Mws or 45 Mws. In

most cases the round-trip propagation delay is 10 ms, and in one case it is 100 ms.

The issues explored in the following subsections include: (1) the general behavior of the algorithms as a function of their key parameters (e.g., gain, damping, queue threshold, round- trip delays, rate updating intervals); (2) fairness for multiple virtual connections; (3) interoperability; and (4) adaptability as a function of the number of active virtual connections.


0 I I I I I I 1 1 1 0 05 I I5 2 25 3 35 4 45 5 55 6 65 7 75 8

Time (sec)

(a)

a i A i = a 2 A 2 = 0 19Mbps. YI Al=y2 A T = O l l l 160

120-

100-

80-

60-

40-

20 -

0

120-l A I 100-

80-

60-

40-

20 -

0

0 0.5 1 1.5 2 2.5 3 3.5 4 4.5 5 5.5 6 6.5 7 75 8 Time (sec)

(b)

Fig. 6. Mb/s, rJ = 10 ms, AJ = 5 ms, uJ = 1.0, ( j = 1 , 2 ) , and QT = 6.5 cells. JRJ algorithm.

Various gains and damping constants with ratio a/- , held fixed at 108 Mb/s and U = 34 Mb/s. VCI turns on at time 0 and VC2 at 2.5 s. p = 155

0 05 1 15 2 25 3 35 4 45 5 55 6 65 7 15 8 Time (sec)

Fig. 7. Mbls, . AJ = 0.013, U] = 13 Mbk, for 3 = 1.2. MS algorithm.

Different propagation delays. VCI turns on at time 0 and VC2 at 2.5 s. p = 45 Mb/s, A3 = 5 ms, cJ = 1.0, and QT = 4.5ceIls. a3 AJ = 0.32

A. Effect of Gain, Damping, and Other Relevant Parameters

ln this section we investigate the effect of the gain parameter, a, and the damping constant, 7, on both the steady state and the transient behavior of the source rates.

Fig. 4 considers a single VC ( J = 1) in which the Source rate is adapted by the MS algorithm. The damping constant y is fixed while the gain a is varied. It is easy to identify cases

in the Unsaturated and the Saturated regimes. Note that rates only are plotted; queues are not shown. The overall conclusion is that it is desirable to operate in the Saturated regime, where the rise time is smaller, the steady state throughput is maximal and the oscillations are modest. Even in the extreme example of the top curve, where a / y + v z 3 . 7 ~ , the behavior is acceptable.

BONOMI er al.: ADAITIVE ALGORITHMS FOR FEEDBACK-BASED FLOW CONTROL 1219

30 r-7 MS

. . I . . . . . . . . . . . , . . 0 0.5 1 1.5 2 2.5 3 3.5 4 4.5 5 5.5 6 6.5 7 7.5 8

Time (sec)

(a)

RI - - R I

I I

10 I , l 1 1 1 l l I I I l I I I o 0 5 I 1 5 2 2 5 3 3 5 4 4 5 5 5 5 6 6 5 7 7 5 8

Time (sec)

(b)

Fig. 8. QT = 4.5 cells. V I = vz = 13 Mb/s. a l l y l = a2/12. MS algorithm.

Different damping constants. VC1 turns on at time 1 and VC2 at 2.5 s. p = 45 Mb/s, r, = 10 ms, A, = 5ms, cJ = 1.0 (3 = 1,2), and

A major issue investigated in Figs. 5 and 6 is whether the algorithms are fair; i.e., whether a second VC, which turns on when the first VC is already using most or all of the network bandwidth, succeeds in ultimately obtaining its share of the bandwidth, assumed here to be half. In Figs. 5 and 6, J = 1 for t < 2.5 s and J = 2 for t 2 2.5 s; as before, p = 155 Mbls and U z p/4.

The MS algorithm is considered in Fig. 5. The parameters conform to the design rule for pointwise fairing in Section IV-c

yj = y; aj = a j a , 0 1 = a2 = 1 ( j = 1,2).

In Fig. 5

2

(uj /yj + u j ) = 142 Mb/s for t < 2.5 s

= 284 Mb/s for t 2 2.5 s. j = 1

Since p = 155 Mbls it follows that the network is Unsaturated for t < 2.5 s, and Saturated for t 2 2.5 s. Observe in the figure that for t < 2.5 s, the approach to steady state is nonoscillatory

and the queue is essentially empty. After t = 2.5 s, a spike in the queue occurs as the second VC turns on and the first one has not yet given up sufficient bandwidth. Also, for t > 2.5 s, the queue is no longer empty and oscillations in the rates and in the queues exist, but are not excessive. The results in Fig. 5 indicate that of the two sets of parameters used, the one in (a), which is less cautious, is also the more desirable since the initial ramp-up to steady state is faster, equalization occurs earlier and the incremental cost in increased oscillations is modest.

Now consider the behavior of the JRJ algorithm shown in Fig. 6. We see similar behavior in the rates and queues. By varying the gain and damping parameters, the transient behavior, amplitude of oscillations and time to convergence in the rates, as well as queue size, can be controlled. Two extreme examples are shown. In (b) the first VC has not yet reached it steady state limit before the second turns on. A major difference between the two algorithms is illustrated in Figs. 5 and 6. As stated earlier, the JRJ algorithm always operates in the Saturated regime. This fact explains the large initial oscillations in rate and spike in the queue in Fig. 6(a), which was designed to roughly correspond to Fig. 5(a) in steady state. In order to make the two algorithms operate more


a , A , = O 8 6 M b p s . ~ , A , = 0 0 8 . o , = l 0 . ~ = 1 . 2 . 3 . 4 MSalgonthm

MS I20

loo

80

60

40

20

0 0 O S I 1 5 2 2 5 3 3 5 4 4 5 5 5 5 6 6 5 1 7 5 8

T m a (sec)

(a)

al A,=O43 Mbps. yl A,= 024, o l=l 0, j=1 .2 ,3 ,4 JRJalgonthm

I-$ I20

60

40

20

0 0 0 5 I 1 5 2 2 5 3 3 5 4 4 5 5 5 5 6 6 5 7 7 5 8

Time (sec)

(b)

Fig. 9. and QT = 6 . j cells. VI = v3 = 22 Mb/s, v2 = v4 = 0.

Best Effort with Minimum Bandwidth. VCl and VC2 turn on at time 0, VC3 and VC4 at 3.5 s. p = 155 Mb/s, r3 = 10 ms, A3 = 5 ms,

160

7 120 MS

P 1 1 l I I I I l I l I 1 1 I

0 0.5 I 1.5 2 2.5 3 3.5 4 4.5 5 5 5 6 6.5 7 7.5 8 Time (sec)

Fig. 10. cells. u j . AJ = 0.34 Mb/s, yJ . AJ = 0.005,

Another set of service parameters. VCl and VC2 trun on at time 0, VC3 and VC4 at 3.5 s. p = 155Mb/s, = 10 ms, AI = 5 ms, and QT = 6.5 = 1 , 2 , 3 , 4 . v1 = v3 = 22 Mb/s, u1 = 0 3 = 2.5: vz = v4 = 4 .4 Mb/s, 02 = 0 4 = 1.0. MS algorithm.

similarly, in the remainder of the simulations we have imposed a ceiling R"$\ on the rates for the JRJ algorithm, so that for each VC

We have conducted simulations to assess the sensitivity of the algorithms with respect to the queue threshold QT

and the updating interval A. In contrast to Fig. 5, where the threshold QT = 6.5 cells, we simulated a system with identical parameters, except that QT = 60.5 cells. The queue in the latter case is larger, which increases the system response time, but otherwise the behaviors are quite similar.

Similarly we have simulated systems in which the updating interval is four times larger than in Fig. 5. Although the rates

BONOMI et al. ADAPTIVE ALGORITHMS FOR FEEDBACK-BASED FLOW CONTROL

Rates for 8 of the 24 VCs

30-

25 -

I ISVCS I 24VCs 6VCs I I I2VCS ! I

I

- a m E f 15

10

5

I I I I I 0 0.5 I 1.5 2 2.5 3 3.5 4 4.5 5 5.5 6 6.5 7 1.5 8

Time (sec)

(a)

Aggregate rate and queue

1281

0 0.5 I 1.5 2 2.5 3 3.5 4 4.5 5 5.5 6 6.5 7 7.5 E Time (sec)

(b)

Fig. 11.

Mb/s, -,] . Aj = 0.008. v3 = 4 Mb/s, R F = 23 Mb/s, oJ = 0.6, j = 1 , 2 , . . . ,24 . MS algorithm.

Many VC’s. Groups of 6 VC’s turn on at 0, 2, 4, and 6 s. p = 155Mb/s, T~ = 10 ms, Aj = 5 ms, and QT = 6.5 cells. a j . Aj = 0.86

exhibit greater variability and the queues are larger, qualitative behaviors are similar.

B. Fairness Issues

In this subsection we provide experimental evidence sup- porting our analysis of fairness of Sections IV-A-IV-E. Let us first consider cases where the design rule of (52) is applied with aJ v and J = 2. The pointwise fairing to be expected in this case is exhibited in Fig. 5 and also in Fig. 7.

In Fig. 5 all the parameters are the same for the two VC’s. In particular the feedback round-trip delay rJ is the same, with = 10 ms. The rates allocated to the two VC’s tend to become identical and the curves overlap perfectly. The same parameters are used in Fig. 7, except for the round-trip delays, which are now 71 = 10 ms and 72 = 100 ms; the MS algorithm is used. The responses exhibit pointwise fairing; the rates tend to overlap perfectly when shifted by ~ 1 / 2 and 72/2, respectively (recall (49)).

Let us next consider evidence of averaged fairness. In Fig. 8 the MS algorithm is used, and the less restrictive design rule of (70) is adopted, with a3 = 1 for j = 1,2, and a l ly l = a2/y2 = 25Mb/s. From inspection of the rates and their difference we note that although no pointwise fairing

1, vJ

occurs, the curves oscillate around a common average value, but with different amplitudes. We also obtain similar behavior from the JRJ algorithm.

We have also simulated cases where the round-trip delays are modeled as the sum of a constant component and an exponentially distributed stochastic component. Periodic behavior can not be identified in the responses, but the rates tend to be fair.

We return to the case of constant round-trip delays, and investigate the characteristics of services defined by the choice of parameters vJ and aJ (see (45)). Fig. 9(a) and (b) show implementations of “Best Effort with Minimum Bandwidth” by the MS and JRJ algorithms for J = 4, different minimum bandwidths and equal access to uncommitted bandwidth. Fig. 10 illustrates behavior for another choice of parameters vJ and a3, which is consistant with a more sophisticated service. As the reader may verify, the observed dynamical behaviors converge to conform to the service specifications.

C. Adaptability and Interoperability

In order to assess the robustness of our control algorithms with respect to the number of VC’s, J , competing for bandwidth at a single node, we observed the impact of increasing

1282 IEEE JOURNAL ON SELECTED AREAS IN COMMUNICATIONS, VOL. 13. NO. 7, SEPTEMBER 1995

Std Deviation Utilization of

Queue Length Bandwidth (%)

Mean Queue Length

S J = J of Network

1 0 . 4 2 0.499 47 2 1 0.943 0.230 94.3 J I 6.75 I 1 1 3 7 I 96

40 1 181.01 1 231.16 68.3 100 1 1262.1 1 1161 72.1

J on both their transient and steady state behavior. Recall (59) , which shows that for the design in (52) , the behavior of the aggregate bandwidth depends on J via the gain factor

Fig. 1 1 presents the transient behavior of the MS algorithm as new groups of six VC’s turn on at 0, 2, 4, and 6 s, respectively. In the interval up to 2 s the active VC’s do not reach the Saturated regime; after 2 s the system is Saturated, and the degree of saturation increases at 4 and 6 s. The rates for a subset of the VC’s are shown in Fig. 1 I(a), while the aggregate rate and the queue length are shown in Fig. 1 I(b). Note the relatively slow growth in the fluctuations as J increases. A selection of results of steady state behavior of the aggregate bandwidth is presented in Table I. The results are obtained by studying the behavior of a single connection under the MS algorithm, when the parameters y and n are fixed, while the factor E,,, chosen to be J for simplicity, is in the range 1-100. Note that the Saturated regime is reached when C J = J > 2. Larger queues and a degradation of efficiency is noticeable when J 2 20. Rescaling of the parameters of the algorithm may be necessary for larger values of .J, as discussed in Section IV-F.

As discussed in Section IV-G, a limited degree of interoperability can be achieved among different algorithms in the class considered in this paper. We have simulated systems with four VC’s in which two used the MS algorithm and the remaining two used the JRJ algorithm, and where the service is Best Effort with Minimum Bandwidth. The parameters of the algorithms are selected following (92). The responses indeed exhibit fair behavior commensurate with the service definition.

c.J = E,.,.

ACKNOWLEDGMENT

The authors gratefully acknowledge the benefit of several years of collaboration on the topic of this paper with their colleagues A. Elwalid, K. Fendick, D. Peck, and V. Saksena. Special thanks are due to A. Weiss, whom we consulted frequently and who always gave generously of his time and advice.

REFERENCES

[ 1 1 L. Benmohamed and S. M. Meerkov, “Feedback control of congestion in packet switching networks: The case of a single congested node,” IEEU4C.M Trunc. .\emm-kiti,q. \ol. I . no. 6. pp. 693-708. 1993.

J.-C. Bolot and A. U. Shankar, “Dynamic behavior of rate-based flow control mechanisms,“ ACM SIGCOMM, Comp. Commun. Rev., vol. 30, pp. 3 5 4 9 , 1990. F. Bonomi and K. Fendick, “The rate-based flow control framework for the available bit rate ATM service,” IEEE Network Mag., vol. 9, no. 2, pp. 25-39, 1995. W. A. Coppel, Stability und Asymptotic Behavior of Differential Equa- tions. A. 1. Elwalid, “Adaptive rate-based congestion control for high speed wide-area networks: stability and optimal design,” preprint, 1993. K. W. Fendick, M. A. Rodrigues, and A. Weiss, “Analysis of rate-based feedback control strategy for long haul data transport,” Perform. Eval., vol. 16, pp. 67-84, 1992. M. Hluchyj and N. Yin, “On closed-loop rate control for ATM networks,” in Proc. IEEE INFOCOM 94, pp. 99-108. V. Jacobson, “Congestion avoidance and control,” in Proc. ACM SIG- COMM’88, pp. 3 14-329. R. Jain, “Myths about congestion management in high-speed networks,” Digital Equipment Corp., Rep. DEC-TR-726, 1990. H. Kanakia, P. Mishra, and A. Reibman, “An adaptive congestion control scheme for real-time packet video transport,” in Proc. ACM SIGCOMM, 1993. S. Keshav, ”A control theoretic approach to flow control,” in Proc. ACM SIGCOMM. 1991. -, “Packet-pair flow control,” preprint, 1994. B. A. Mackrucki, “On the performance of submitting excess traffic to ATM networks,” in Pmc. IEEE GLOBECOM ‘91, 1991, pp. 281-288. P. Mishra and H. Kanakia, “A hop by hop rate based congestion control scheme,’’ in Proc. ACM SIGCOMM, 1992. D. Mitra, “Asymptotically optimal design of congestion control for high speed data networks.” IEEE Trans. Comm., vol. 40, no. 2, pp. 301-3 1 1, 1492. D. Mitra and J . B. Seery, “Dynamic adaptive windows for high speed data networks: Theory and simulations,” in Proc. ACM SIGCOMM, 1990, pp. 20-30. -, “Dynamic adaptive windows for high speed data networks with multiple paths and propagation delays,” Comput. Networks, ISDN Sysr., vol. 25, pp. 663-679, 1993. -, “New adaptive algorithms for windows and rates in high speed, wide area networks based on periodic, 1-bit, explicit feedback,” unpub. rep., Jan. 1993. A. Mukherjee and J. C. Strikwerda, “Analysis of dynamic congestion control protocolr-A Fokker-Planck approximation,” in Proc. ACM SIGCOMM, 1991, pp. 159-169. K. K. Ramakrishnan and R. Jain, “A binary feedback scheme for congestion avoidance in computer networks with a connectionless network layer,’’ in Proc. ACM SIGCOMM ‘88, pp. 303-313. K. G. Ramakrishnan and D. Mitra, “PANACEA: An integrated set of tools for performance analysis,” in Modeling Techniques and Tools .for Cornpurer Etduotions, R. Puigjaner and D. Potier, Eds. New York: Plenum, 1988, pp. 2 5 4 0 . J. B. Seery, “Customizing PANACEAISimulation,” AT&T Bell Labs Memorandum, 1988. L. Zhang, “A new architecture for packet switching network protocols,” Ph.D. Thesis, MIT, Lab. for Comp. Sci., Cambridge, MA 02139, 1989. -, “Virtual clock: A new traffic control algorithm for packet switching network.” in Proc. ACM SIGCOMM, 1990, pp. 8-18.

Boston, MA: D. C. Heath, 1965.

Flavio Bonomi was born in Buenoq Aires, Ar- gentina, in 1954 He received the Laurea from Pavia University, Pavia, Italy, in 1978, and the M E and Ph D degrees from Cornell University, Ithaca, NY, in 1981 and 1485, respectively, a11 in electrial engineering

From 1985 to 1991, he was with the Department of Performance Analysis at AT&T Bell Laborato- ries, Holmdel, NJ, where he worked on telecommu- nications systems dnd performdnce modeling and analysis in the area of computer systems Since

1941, he has been with the ATM Platform Organization of AT&T Network Systems, Red Bank, NJ, where he is a Distinguished Member of the Technical Staff His interests and responsibilities within both research and development on ATM systems include switch architecture design, performance analysis, design of traffic management schemes. and network engineering

BONOMI er d.: ADAITIVE ALGORlTHMS FOR FEEDBACK-BASED FLOW CONTROL 1283

Judith B. Seery received the B.A. degree in mathematics from the College of St. Elizabeth, Convent Station, NJ, in 1%8, and the M.S. degree in mathematics from New York University, New York, in 1972.

She joined AT&T Bell Laboratories in 1968 and is presently a Member of the Technical Staff in the Mathematical Sciences Research Center. Her current research interests include the modeling and simulation of queueing networks representing com- munication and computing systems.

Ms. Seery is a member of the Mathematics Association of America, the Society for Industrial and Applied Mathematics, and the Association for Women in Mathematics.

Adaptive Algorithms for Feedback-Based Flow Control in High Speed, Wide-Area ATM Networks

Documents

Transcript of Adaptive Algorithms for Feedback-Based Flow Control in High Speed, Wide-Area ATM Networks