Optimizing the Throughput of Data-Drivenbased Streaming in Heterogeneous

Overlay Network

Meng Zhang1, Chunxiao Chen1, Yongqiang Xiong2,Qian Zhang3, and Shiqiang Yang1

1 Dept. of Computer Sci. & Tech., Tsinghua Univ., Beijing 100084, China,{zhangmeng00,chencx05}@mails.tsinghua.edu.cn, yangshq@tsinghua.edu.cn

2 Microsoft Research Asia, Beijing 100080, China,Yongqiang.Xiong@microsoft.com

3 Dept. of Computer Sci., Hong Kong Univ. of Sci. and Tech., Hong Kong, China,qianzh@cs.ust.hk

Abstract. Recently, much attention has been paid on data-driven (orswarm-like) based live streaming systems due to its rapid growth in de-ployment over Internet. In such systems, nodes randomly select theirneighbors to form an unstructured overlay mesh (gossip-style overlayconstruction) and then each node requests desired data blocks from itsneighbors (block scheduling). To improve the performance, most of ex-isting works focus on the gossip-style overlay construction issue; howeverfew concentrate on optimizing the block scheduling for improving thethroughput of a constructed overlay, especially in heterogeneous envi-ronment. In this paper, we propose a scheme to optimize the throughputof data-driven streaming systems in heterogeneous overlay network. Wefirst model the block scheduling problem as a classical min-cost flow prob-lem and thereby derive a global optimal solution. Based on this idea, wethen propose DONLE - a fully distributed asynchronous scheduling al-gorithm. Simulation results verify that DONLE is superior to a numberof conventional strategies.

1 Introduction

As the most promising alternative to IP multicast, overlay multicast especiallymulticast through peer-to-peer (P2P) network has attracted a lot of attentionduring the past decade. One of the most important applications of overlay mul-ticast is to stream live media content to a huge population of end users throughInternet, also known as peer-to-peer streaming.

A lot of measurement studies in P2P overlay networks reveal that the bot-tleneck bandwidth between the end hosts exhibits extremely heterogeneity. Todeal with heterogeneity in streaming multicast applications, numerous solutionshas been proposed for both IP multicast [1] and overlay multicast [2, 3]. Theirbasic way is to encode the source video into multiple layers, and each receiversubscribes an appropriate number of layers due to its bandwidth capacity.

2 Meng ZHANG et al.

Recently, a new category of overlay streaming multicast protocols called data-driven protocols (or swarm-like protocols) [4–7] targeting non-interactive stream-ing multicast applications has been proposed. Unlike conventional tree-basedapproaches, in data-driven protocol, each node randomly finds some nodes asits neighbors so that an unstructured network is formed. This step is usuallycalled gossip-style overlay construction (or membership management). The nextstep named block scheduling is also intuitive: the live media content is dividedinto blocks (or segments, packets) and every node announces what blocks ithas to its neighbors. Then each node explicitly requests the blocks of inter-est from its neighbors according to their announcement. Actually, it is similarto Bit-Torrent protocol [8]. Some systematical studies (such as [9]) show thatdata-driven approach is better than tree-based approach under many conditionsespecially in high churn rate of clients. Meanwhile, data-driven based stream-ing systems are also emerging and rapidly deployed over Internet [4, 10, 11] inthe past two years. Given the significance of data-driven streaming protocol, itis important to study how to improve the throughput of this category of pro-tocols especially under heterogeneous network. Most of existing works in P2Pstreaming with layered coding [2, 3] use stream level scheduling method. How-ever, the scheduling in data-driven protocol is more fine-grained because it needsa block level scheduling. This leads to the challenge: how does a node decide tofetch which block of which layer from which neighbor node under heterogeneousbandwidth constraints.

In our previous work [12], we have studied how to do optimal scheduling inhomogeneous environment. In this paper, we propose DONLE, a Data-drivenOverlay Network algorithm using LayEred coding to handle the heterogeneity.We first state the basic block scheduling problem, then model the problem as aclassical min-cost flow problem and give a global optimal block scheduling solu-tion. After that, we propose a fully distributed algorithm - DONLE, doing localoptimal block scheduling at each node. Simulation results show that DONLE issuperior to a number of recent proposed scheduling strategies under the sameoverlay topology. The remainder of this paper is organized as follows. In Section2 we briefly review the related work. In Section 3, we state the block schedul-ing problem in detail and formulate the problem. Next, in Section 4, we modelthis scheduling problem as an equivalent min-cost flow problem and derive theglobal optimal scheduling algorithm. Section 5 presents the proposed distributedasynchronous algorithm DONLE. The performance of DONLE is evaluated inSection 6. We conclude this paper in Section 7.

2 Related Work

Actually, there are a wealth of research efforts towards improving the overlaymulticast throughput. Early researchers in this area mainly focus on how to con-struct single or multiple application layer tree(s). LION [3] employs a stream-level multi-path based method to improve the throughput of overlay networkusing network coding. Recently, a new category of overlay multicast protocols

Lecture Notes in Computer Science 3

- data-driven (or swarm-like) protocols are proposed [4–7]. In theses protocols,PALS [7] is an adaptive streaming mechanism from multiple senders to a singlereceiver using layered coding, which is actually a swarm-like (or data-driven)protocol. PALS mainly focuses on coping with the network dynamics such asbandwidth variations and sender participation. PALS evaluates its performanceunder the scenario of streaming from multiple senders to a single receiver verydetailedly. Yet it does not involve the performance of the data-driven proto-col under an overlay mesh and does not aim to improve the throughput ofdata-driven streaming. Besides, many recent works have been done to improvethe gossip-style overlay construction for various purposes [13, 14]. However, fewworks address how to maximize the throughput of data-driven streaming in aconstructed heterogeneous overlay mesh.

3 Block Scheduling: Problem Statement And Formulation

In this section, we first intuitively explain what we optimize in data-drivenstreaming. Then we formulate this problem. Our basic approach is comprehen-sive. We define a priority for every desired block of each node due to the blockimportance, such as block layer, and its rarity. Our goal is to maximize the av-erage priority sum of all streaming blocks that are delivered to each node in onerequest period under heterogeneous bandwidth constraints.

3.1 Block Scheduling Problem

The idea of DON based streaming system is similar to Bit-Torrent protocol [8].In our protocol, each node will independently find its neighbors in the overlay sothat an unstructured random overlay mesh will be formed. The media streamingis encoded with layered coding, and every layer is divided into blocks with thesame size, each of which has a unique sequence number. Every node has a slidingwindow which contains all the up-to-date blocks on the node and goes forwardcontinuously at the speed of streaming rate. We call the front part of the slidingwindow exchanging window. The blocks in the exchanging window are the onesbefore the playback deadline, and only these blocks will be requested if theyare not received. The unavailable blocks beyond playback deadline will be nomore requested. Every node periodically pushes all its neighbors a bit vectorcalled buffer map in which each bit represents the availability of a block in itssliding window to announce what blocks it holds. Due to the announcement ofthe neighbors, each node will periodically send requests to its neighbors for thedesired blocks in its exchanging window. We call the time between two requestsa request period (or period for short, typically 1∼6 sec). Each node will decidefrom which neighbor to ask for which blocks at the beginning of each requestperiod. When a block does not arrive after its request is issued for a while andis still in the exchanging window, it is requested in the following period again.In layered video coding, video is encoded into a base layer and several enhancedlayers and a higher layer can only be decoded if all lower layers are available,

4 Meng ZHANG et al.

Table 1. Notations

Notation Description

N Set of all nodes in the overlay except the source node 0L Number of encoded layersrl, l = 1, · · · , L The cumulative rate from layer 1 to layer l, blocks per secondIi, Oi, i = 0, · · · , |N | The inbound and outbound bandwidth capacity of node iEik, i, k = 0, · · · , |N | The maximum end-to-end bandwidth from node i to khij ∈ {0, 1} “hij = 1” denotes node i holds block j; “hij = 0”, otherwiseNBRi Set of neighbors of node iτ The request periodπi

j The priority of block j for node iWT The exchanging windows size scaled by timeCi The current clock time at node idi

j Play out deadline of block j at node iDi Set of all desired blocks in the exchanging window of node i

namely the block dependency. So in our algorithm, the blocks in lower layeralways have higher priority than the ones in the upper layer.

3.2 Problem Formulation

In this section, we will give the formulation of the block scheduling problem (BSPfor short). As aforementioned, we try to maximize the average priority sum ofall streaming blocks that are delivered to each node in one request period un-der heterogeneous bandwidth constraints. We consider two types of bandwidthconstraints, namely access bottlenecks (inbound and outbound bandwidth ca-pacity) and non-access bottleneck bandwidth (maximum end-to-end availablebandwidth). As all the blocks have the same size, we use blocks per secondto represent the amount of inbound, outbound, and end-to-end available band-width. Table 1 summarizes the notations in the rest of this paper.

Block Priority Definition We give each block a priority due to its im-portance for a specified node. Two key factors that have impact on the blockimportance are considered here: the layer factor and the rarity factor. As inlayered coding, the upper layer can be decoded only if the lower layers are avail-able, we should ensure that the blocks of lower layer have higher priority. Besides,many previous works such as [15] demonstrate that requesting the block withrarest holders first brings more diversity to the system and help the block spreadmore rapidly. The following is the priority value of block j for node i:

πij = βΠR

( ∑

k∈NBRi

hkj

)+ (1− β)θΠL(λj), where β = (di

j − Ci)/WT (1)

We let both function ΠR and ΠL monotonously decreasing. Function ΠL

satisfies ΠL(λj) À ΠL(λk) when λj < λk, for any block j and k so as toguarantee the layer dependency requirement. Parameter 0 ≤ β ≤ 1 representsthe current position block j in the exchanging window. We let θ have relatively

Lecture Notes in Computer Science 5

large value. Although our block priority definition is a simple linear combinationof the two factors, it can guarantee the following key requirements: a) when alower-layer block is in near the playback deadline (β = 0), it has much higherpriority than any other upper-layer blocks (large value of θ); b) a block withfewer holders has higher priority than the one with more holders in the samelayer and the same position in exchanging window.

Formulation We formulate the block scheduling problem. We define thedecision variable xi

kj to denote whether node i ∈ N should request block j ∈ Di

from its neighbor k ∈ NBRi:

xikj =

{1, node i should request packet j from neighbor k0, otherwise (2)

Our target is to maximize the average priority sum of blocks that each nodecan receive with heterogenous bandwidth constraints:

max1|N |

∑

i∈N

∑

j∈Di

∑

k∈NBRi

πijhkjx

ikj

s.t.(a)

∑

k∈NBRi

xikj ≤ 1,∀i ∈ N, j ∈ Di (b)

∑

j∈Di

∑

k∈NBRi

xikj ≤ τIi,∀i ∈ N

(c)∑

i∈NBRk

∑

j∈Di

xikj ≤ τOk,∀k ∈ N (d)

∑

j∈Di

xikj ≤ τEki,∀i ∈ N, k ∈ NBRi

(e) xikj ∈ {0, 1},∀i ∈ N, k ∈ NBRi, j ∈ Di

(3)The formulation is a comprehensive integer linear programming, and we call

this optimization problem global block scheduling problem (or global BSP forshort). Constraint a) ensures no duplicate blocks are requested. Constraints b)and c) guarantee the blocks numbers that are downloaded from node i anduploaded to node k do not exceed the inbound and outbound bandwidth limita-tion respectively. Furthermore, constraint d) ensures that the number of blockstransmitted from node k to node i is under the constraint of end-to-end availablebandwidth. Finally, constraint e) indicates that it is an integer programming.

4 Modeling and Global Optimal Solution

In this section, we will show that the global BSP (3) can be transformed into anequivalent minimum cost flow problem that can be solved in polynomial time.We call solving such a min-cost flow problem a global optimal solution. The min-cost flow problem is introduced in [16]. By double scaling algorithm [16], the timecomplexity for min-cost flow problem is bounded with O(nm(log log U) log(nC)),where n and m are the number of vertices and arcs while U and C is the largestmagnitude of arc capacity and cost respectively.

Fig. 1(a) and Fig. 1(b) show a sample of global BSP with four nodes andits min-cost flow modeling respectively. In Fig. 1(b), the two numbers close to

6 Meng ZHANG et al.

(a) A global block schedulingproblem

(b) Model as a min cost flow problem

Fig. 1. An example of the equivalent MCFP

an arc represent the capacity and per unit flow cost of the arc. Rather thandescribe the general model formally, we merely describe the model ingredientsfor these figures. In data-driven streaming, we decompose a node into its threeroles: a send, a receiver and a neighbor. We model each sender k as a vertexsk, each receiver i as a vertex ri, and each neighbor k of node i as a vertexnik. Further, we model a desired block j for node i as a vertex bij . Besides weadd two virtual vertices: a source vertex s and a sink vertex t. The decisionvariables for this problem are whether to request block j from neighbor k ofnode i which we represent by an arc from vertex nik to vertex bij if block jis a desired by node i. These arcs are capacitated by 1 and their per unit flowcost is 0. And we insert arc from vertex nik to bij to indicate that neighbor kof node i holds block j. To avoid duplicate blocks, we add arc capacitated by 1from bij to ri and set the per unit flow cost as the priority of block j for node imultiplied a constant −1/|N |. To satisfy the outbound bandwidth constraint ofnode k, we add arc between vertex s and vertex sk whose capacity is τOk. Andfor the maximum end-to-end available bandwidth from neighbor k to node i, weinsert arc from vertex sk to nik with capacity τEik. Finally, to incorporate theinbound bandwidth constraint of node i, we introduce arc between ri and t withcapacity τIi. To guarantee maximum number of blocks are delivered, we insertuncapacitated arc from vertex t to s that has a negative per unit flow cost withlarge absolute value. Finally we have the conclusion: The min-cost flow problemwould yield the optimal solution of the global BSP. We omit the proof here.

5 Heuristic Distributed Algorithm - DONLE

In this section, based on the basic idea of the global optimal solution, we presentthe heuristic practical algorithm - DONLE which is fully distributed and asyn-

Lecture Notes in Computer Science 7

chronous. In DONLE, each node will decide from which neighbor to fetch whichblocks at the beginning of its request period. As the request period is relativelyshort (such as 2 seconds), our scheduling algorithm should make decision asrapidly as possible. So in our heuristic distributed algorithm, we just do a lo-cal optimal block scheduling on each node based on the current knowledge ofthe block availability among the neighbors. The local optimal block schedulingcan also be modeled as a min-cost flow problem. As shown in Fig. 1(b), thesub min-cost flow problem in the each rectangle is just the local optimal blockscheduling.

However, one problem to do local scheduling is that each node does notknow the optimal flow amount on arcs (sk, nki) (≤ Ok). In other words, weshould estimate the proper upper-bound of the bandwidth from each neighbor.For simplicity, here we use a purely heuristic way for each node to estimate themaximum rate at which each neighbor can send blocks. Our approach is to usethe historical traffic from each neighbor to do this. More formally, let Qki denotethe estimated maximum rate at which neighbor k ∈ NBRi can deliver to nodei. Of course, Qki should not exceed Ok. We let g

(p)ki denote the total number

of blocks received by node i from neighbor k in the pth period. In each requestinterval, we use the average traffic received by node i in the previous P periods toestimate Qki in the (p+1)th period: Qki = γ ·( ∑p

ω=p−P+1 g(ω)ki

)/Pτ . Parameter

γ(> 1) is a constant called aggressive coefficient. Then we can do a local optimalblock scheduling formulated as below and solve it by its equivalent min-cost flowproblem in polynomial time. We call it a local BSP. Our distributed algorithmis heuristic and we examine its performance and the gap between DONLE andthe global optimal solution by simulation in Section 6.

max∑

j∈Di

∑

k∈NBRi

P ijhkjx

ikj (4)

s.t.(a)

∑

k∈NBRi

xikj ≤ 1,∀j ∈ Di, (b)

∑

j∈Di

∑

k∈NBRi

xikj ≤ τIi,∀i ∈ N

(c)∑

j∈Di

xikj ≤ τQki,∀i ∈ N, k ∈ NBRi, (d) xi

kj ∈ {0, 1},∀k ∈ NBRi, j ∈ Di

6 Performance Evaluation

In this section, we compare DONLE to other existing block scheduling strategies,and also examine the gap between DONLE and the global optimal solution.Three conventional strategies are compared here:

– Random Strategy: each node will assign each desired block randomly to aneighbor which holds that block. Chainsaw [5] uses this simple strategy. Weexamine how this method works in layered data-driven streaming.

8 Meng ZHANG et al.

(a) Conservative (b) Aggressive (c) Tradeoff

Fig. 2. Three round robin strategies

– Local Rarest First (LRF) Strategy: As Section 3 depicted, a block that hasthe minimum owners among the neighbors will be requested first. DONet [4]adopts this strategy. We also introduce this method into layered data-drivenstreaming and compare it with ours.

– Round Robin (RR) Strategy: All the desired packets will be assigned to oneneighbor in a prescribed order in a round-robin way. If the block is only avail-able at one sender, it is assigned to that sender. Otherwise, it is assigned toa sender that has the maximum surplus available bandwidth. In Fig 2, weintroduce three conventional block ordering schemes used in the literature.Fig. 2(a) shows the conservative block ordering: it always requests blocks oflower layers first. On the contrary, aggressive block ordering scheme requestsblocks of all layers with lowest sequence number (or time stamp) preemp-tively as illustrated in Fig. 2(b). Fig. 2(c) uses a zigzag ordering (slope=1)which is a tradeoff between the two extreme schemes.

To evaluate the performance, we define delivery ratio of a layer to repre-sent the number of different blocks that arrive at each node before the playbackdeadline over the total number of blocks encoded in that layer. Since the totalnumber of blocks in a layer is a constant that relies on the encoding and packeti-zation, the average delivery ratio among all nodes can represent the throughputof the overlay. We compare DONLE and global optimal solution to the follow-ing five strategies: random, LRF, RR-conservative, RR-aggressive, RR-tradeoff.To ensure fair comparison, all the approaches have the same physical networkand end-host participants in each scenario. Each curve in all the plots is anaverage over 10 simulation runs. We encode the video into 10 layers, and eachlayer has a rate of 50Kbps. To evaluate the quality of a specified layer, we av-erage the delivery ratio of that layer over all nodes that can achieve the layerdue to their inbound bandwidth. We use 500 nodes in the overlay and set therequest period to 2 seconds. We set the node access bandwidth is asymmetric:the inbound bandwidth evenly distributes across 15Kbps to 1Mbps; while theoutbound bandwidth of each node is randomly selected between half and onetime of its inbound bandwidth. We set the outbound bandwidth of the sourcenode to 2Mbps. Previous study [17] has shown that there is a sweet range ofneighbor count or peer degree (roughly between 6 to 14) where the deliveredquality to the majority of peers is high. Therefore, in our simulation, each noderandomly selects 14 other nodes as its neighbors. We set the exchanging window

Lecture Notes in Computer Science 9

(a) The bandwidth bottleneck is only at last mile. (b) The bottleneck is not only at last mile.

Fig. 3. Average delivery ratio at each layer

to 10 seconds so as to avoid large delay and set the sliding window to 1 minuteaiming to increase the opportunity of serving more neighbors.

As shown in Fig. 3(a), we compare the global optimal solution and DONLEto five other strategies. In this figure the bottlenecks are configured to be onlyat the last mile. We note that the global optimal solution has the best perfor-mance, and the delivery ratio in all layers is nearly 1. This demonstrates that thegenerated topologies have sufficient capacity to support all the nodes to receiveall layers that they can achieve. The performance of DONLE is also fairly good.Most of the delivery ratio in lower layers has nearly 1 and most in higher layersis also above 0.9. However, though the RR-conservative method has perfect de-livery ratio in layer 1 to 4, the quality has a cliff drop from layer 5. This meansall the users can enjoy the video of 4 layers very smoothly, yet few nodes canreceive data beyond the 4th layer even if their inbound bandwidth is sufficient tosupport higher quality. This is because requesting lower layers first leads to badblock diversity among nodes. In contrast, the curve of the RR-aggressive methodis flat. We note that most nodes can not watch even the base layer, althoughmore blocks of higher layers are propagated, since this method does not considerthe layer dependency. RR-tradeoff methods leverage the previous two methods.Here we use zigzag ordering with slope of 1/10 in RR-tradeoff. We found thatthe LRF strategy has more deliver ratio than round-robin schemes. Meanwhile,the random strategy has the poorest performance. As shown in Fig. 3(a), ourdistributed method DONLE outperforms other strategies much with a gain of10%∼80%. Nevertheless, there is still about 12% gap between the global opti-mal solution and DONLE. In Fig. 3(b), we investigate the performance of thesemethods when the bottleneck is not only at last mile. In this figure, we let themaximum end-to-end available bandwidth distribute across 10Kbps 150Kbps.All the other configurations are not changed. The delivery ratio of all methodsdegrades compared to the results when bottleneck is only at last mile. The per-formance from the best to the poorest in turn is still DONLE, LRF, round-robinschemes, and random. It is observed that the rarity factor has significant im-

10 Meng ZHANG et al.

pact on the throughput improvement in data-driven streaming. Therefore LRFstrategy has better performance than round-robin and random strategies. Fur-ther, DONLE not only considers the rarity factor, but also does a local optimalscheduling that utilize the local bandwidth capacity as sufficient as possible asexplained intuitively in Section 3. Hence DONLE outperforms other strategies.

7 Conclusion and Future Work

To improve the throughput of data-driven streaming in heterogeneous network,we propose a global optimal solution and a distributed algorithm - DONLE.Our simulation results show that our proposed algorithm DONLE is superiorto a number of conventional strategies. For future work, we will study how tomaximize the blocks delivered over a horizon of several periods, taking intoaccount the inter-dependence between the periods. We are also planning to domore experiments on examining the parameter sensitivities in our algorithm.

References

1. McCanne, S., Jacobson, V., Vetterli, M.: Receiver-driven layered multicast. In:ACM SIGCOMM 1996. (1996)

2. Cui, Y., Nahrstedt, K.: Layered peer-to-peer streaming. In: NOSSDAV. (2003)3. Zhao, J., Yang, F., Zhang, Q., Zhang, Z., Zhang, F.: Lion: Layered overlay multicast

with network coding. IEEE Trans. on Multimedia (2007) Accepted to be published.4. Zhang, X., Liu, J., Li, B., Yum, T.S.P.: Coolstreaming/donet: A data-driven over-

lay network for efficent media streaming. In: IEEE INFOCOM 2005. (2005)5. Pai, V., Kumar, K., et al: Chainsaw: Eliminating trees from overlay multicast. In:

IEEE INFOCOM 2005, Conell, US (2005)6. Zhang, M., Zhao, L., Tang, Y., Luo, J., Yang, S.: Large-scale live media streaming

over peer-to-peer networks through global internet. In: ACM workshop on Ad-vances in peer-to-peer multimedia streaming (P2PMMS), Singapore (2005) 21–28

7. Agarwal, V., Rejaie, R.: Adaptive multi-source streaming in heterogeneous peer-to-peer networks. In: SPIE/ACM MMCN 2005, San Jose, CA, USA (2005)

8. Cohen, B.: Bittorrent website: http://bitconjuer.com. (2006)9. Silverston, T., Fourmaux, O.: Source vs data-driven approach for live p2p stream-

ing. In: IEEE International Conference on Networking 2006, Mauritius (2006)10. GridMedia: http://www.gridmedia.com.cn/. (2006)11. PPLive: http://www.pplive.com/. (2006)12. Zhang, M., Xiong, Y., Zhang, Q., Yang, S.: On the optimal scheduling for media

streaming in data-driven overlay networks. In: IEEE GLOBECOM. (2006)13. Venkataraman, V., Francis, P.: On heterogeneous overlay construction and random

node selection in unstructured p2p networks. (In: IEEE INFOCOM 2006)14. Jiang, J., Nahrstedt, K.: Randpeer: Membership management for qos sensitive

peer-to-peer applications. (In: IEEE INFOCOM 2006)15. Bhrarmbe, A.R., Herley, C., Padmanabhan, V.N.: Analyzing and improving a

bittorrent network’s performance mechanisms. (In: IEEE INFOCOM 2006)16. Ahuja, R.K., Magnanti, T.L., Orlin, J.B.: Network Flows: Theory, Algorithms,

and Applications. (Prentice Hall)17. Magharei, N., Rejaie, R.: Understanding mesh based peer-to-peer streaming. In:

ACM NOSSDAV 2006, Newport, Rhode Island, USA (2006)