978-1-4244-1694-3/08/$25.00 ©2008 IEEE

Resource Allocation in a Client/Server Hybrid Networkfor Virtual World Environments

Luis Diego Briceno1, Howard Jay Siegel1,2, Anthony A. Maciejewski1,Ye Hong1, Brad Lock1, Mohammad Nayeem Teli2, Fadi Wedyan2,

Charles Panaccione2, and Chen Zhang2

Colorado State University1Department of Electrical & Computer Engineering

2Department of Computer ScienceFort Collins, CO 80523

{LDBricen, HJ, AAM, YHong, Bradley.Lock, Mohammad.Teli,Fadi.Wedyan, Charles.Panaccione, Chen.Zhang}@colostate.edu

Abstract

The creation of a virtual world environment (VWE)has significant costs, such as maintenance of serverrooms, server administration, and customer service. Theinitial development cost is not the only factor that needsto be considered; factors such as the popularity of aVWE and unexpected technical problems during and af-ter the launch can affect the final cost and success ofa VWE. The capacity of servers in a client/server VWEis hard to scale and cannot adjust quickly to peaks indemand while maintaining the required response time.To handle these peaks in demand, we propose to employusers’ computers as secondary servers. The introduc-tion of users’ computers as secondary servers allowsthe performance of the VWE to support an increase inusers. In this study, we develop and implement five staticheuristics to implement a secondary server scheme thatreduces the time taken to compute the state of the VWE.The number of heterogeneous secondary servers, con-version of a player to a secondary server, and assign-ment of players to secondary servers are determined bythe heuristics implemented in this study. A lower boundof the performance is derived to evaluate the results ofthe heuristics.

1 Introduction

Heterogeneous distributed and parallel systems mayoperate in an environment where certain system perfor-mance features degrade due to unpredictable circum-stances, such as sudden machine failures, higher than

This research was supported by the NSF under grant CNS-0615170and by the Colorado State University George T. Abell Endowment.

expected load, or inaccuracies in the estimated systemparameters. The environment considered in this researchis a virtual world environment (VWE) (e.g., a massivemultiplayer online gaming (MMOG) environment). In aVWE, each user controls an avatar (an image that repre-sents and is manipulated by a user) in a virtual world andinteracts with other users. In general, most VWEs usea client/server architecture to control the virtual gameworld. The client/server architecture has some disad-vantages: the initial procurement of servers is expen-sive, server administration is required, customer serviceis necessary, and the architecture is hard to scale basedon demand. In addition to the initial development cost,other factors such as the popularity of a VWE, and un-expected technical problems during and after the launchcan also affect the final cost and success of a VWE [24].

This study focuses on scaling the system based ondemand. Consider an environment where each of Nusers produces a data packet that needs to be processed.There is a main server (MS) that controls the state ofthe virtual world. If the performance falls below ac-ceptable standards, the MS can off-load calculations tosecondary servers (SSs). An SS is a user’s computerthat is converted into a server to avoid degradation in theperformance of the VWE.

The performance of the heterogeneous system usedto simulate the game world must not degrade beyond ac-ceptable parameters even if the VWE is oversubscribed.The heuristics must convert users to SSs and assign theremaining users to an existing SS or the MS. This prob-lem is similar to the assignment of tasks to machines(e.g., [3, 6, 25, 27]) with SS and the MS as machinesand the remaining users as tasks.

A session in the VWE is assumed to last for an ex-tended period of time, with a small break between ses-

sions [20]. During these sessions it is assumed that nousers enter or leave the game (i.e., the users in a gamingsession do not change). These simplifying assumptionsmake a static resource allocation heuristic viable [1].

In this study, five heuristics for determining the num-ber of SSs, which users are converted to SSs, and howusers are distributed among the SSs and the MS areproposed. The heuristics were used to minimize the timeit takes to process the world update requests from allusers. A mathematical lower bound is derived to evalu-ate the performance of the heuristics in this VWE.

This paper is organized as follows. Section 2 pro-vides the problem statement and describes the perfor-mance metrics. In Section 3, we focus on the five pro-posed heuristics. Section 4 discusses the lower bound onour performance metrics. Our results for the five heuris-tics are shown in Section 5. In Section 6, we providethe related work and in Section 7 we present our conclu-sions.

2 Problem Statement

2.1 Problem Description

In this study, we will consider an MMOG environ-ment where the performance of a user is sensitive to la-tency [2]. The purpose of this research is to maintain anacceptable system performance (despite the MS usedto maintain the MMOG environment being oversub-scribed) without increasing the processing power of theMS. The proposed solution is to convert users to SSsthat assist the MS in computation. In the client/serversolution shown in Figure 1(a), all users connect to theMS, therefore the MS is the only machine performingcomputation. In the SS solution shown in Figure 1(b),the MS and SSs perform computation and the MS re-solves conflicts among users and SSs connected to it.

The allocation of users as SSs has similar security re-quirements as distributed servers and peer-to-peer basedMMOG systems. The issue of security in distributedservers and peer-to-peer MMOG environments are stud-ied in [4] and will not be discussed here because we con-sider it to be a separate research problem.

The following simplifying assumptions are madeabout the communication model in this system. Thecommunication time from node (user computer, SS, orMS) A to node B is the same as the communicationtime from node B to node A; however, the communi-cation time between different pairs of nodes will vary.The communication times among the users, SSs, andthe MS do not change during a session. These timesare independent of the number of users connected to theSS or MS. These simplifying assumptions are used toreduce the complexity of the simulations.

Without loss of generality, the level of activity in theVWE of all the users is considered identical (i.e., thefrequency of interaction with the VWE is the same for

all players); therefore, the computational load is basedon the number of users (i.e., they have the same compu-tational needs). To model the computation times of theMS and SSs we need to consider how the computationgrows with the increase in the number of users. Let nα

be the number of users connected to secondary serverα (SSα) and µα be a constant for SSα (this constantrepresents the heterogeneity in the computing power ofthe users’ computers, and each user has a different con-stant). In [15], latency in a MMOG environment showsa “weak exponential” increase with an increase in play-ers. This study uses these observations to create a modelfor the MMOG environment. The computation time foran SS (Compα) can be calculated as:

Compα = µα · (nα)2. (1)

Let nsecondary be the total number of users connectedto all the SSs, nnss be the number of SSs, nmain bethe number of users connected to MS, and b and c arecomputational constants of the MS. The computationtime of the MS (CompMS) is:

CompMS = c · nsecondary + b · (nmain + nnss)2. (2)

2.2 Objective Function andPerformance Metric

Let RTx represent the Response Time (RT) of apacket (representing an action) sent by the computer ofuser x (Ux) to the MS (possibly through an SS) andreturning to Ux (with the corresponding consequence ofthat action). Let Comm(A,B) be the communicationtime between node A and node B. The equation used tocalculate RTx if Ux is connected directly to the MS is:

RTx = Comm(Ux,MS) + CompMS

+ Comm(MS, Ux);RTx = 2 · Comm(Ux,MS) + CompMS . (3)

If a user is connected to an SSα then the equation is:

RTx = Comm(Ux, SSα) + Compα

+ Comm(SSα,MS) + CompMS

+ Comm(MS, SSα) + Comm(SSα, Ux);RTx = 2 · Comm(Ux, SSα) + Compα

+ 2 · Comm(SSα,MS) + CompMS . (4)

If Ux is SSα then Equation 4 is used withComm(Ux, SSα) = Comm(SSα, Ux) = 0.

The performance metric of a resource allocation inthis environment is:

RTmax = max∀Ux

(RTx). (5)

The goal of the heuristics is to minimize RTmax. Thevalue of RTmax will be influenced by three elements:

2

1

Simple Diagram of MMOG

structure

Px: Player x U1

U2

U3

U4

U5

U6

UN–1

UN

MS

…

(a)

2

Simple Diagram of proposed

MMOG structure

U1

SSα/U2

U3

U4

U5

SSβ/U6

UN–1

UN

MS

…

(b)

Figure 1. (a) Client/server architecture versus (b) Secondary Server architecture.

the number of SSs, which users’ computers are con-verted to SSs, and the assignment of non-SS users toan SS or to the MS. If a user is connected to an SSthen this user cannot have any users connected to it.

3 Resource Allocation Heuristics

3.1 Overview

All heuristics were limited to a maximum executiontime of 10 minutes. Five heuristics for determining anallocation of resources are presented in this section. Inthis context, resource allocation implies assigning a userin one of three ways: (1) attaching it directly to the MSwithout making it an SS (although it can become one),(2) attaching it to the MS and making it an SS, or (3)attaching it to an existing SS. An unassigned user is onethat has not been assigned yet.

3.2 Dual Iterative Minimization

In Dual Iterative Minimization (DIM), a solution isrepresented in the form of an vector whose ith elementindicates the way user i is connected to the MS, eitherdirectly or the SS to which it is connected.. A potentialhost (PH) is a user that is not connected to the MSthrough an SS; i.e., either it is attached directly to theMS (possibly as an SS) or is unassigned at this point.This heuristic considers assigning an unassigned user toall PHs or the MS and picks the PH that provides theminimum RTx. If this PH is not already an SS then itis converted to one. The pseudo-code is shown in Figure2. A post-optimization procedure is run on the solutionfrom the DIM heuristic. This post-optimization is shownin Figure 3.

3.3 Tabu Search

Tabu Search [12] enhances the performance of a localsearch method by storing the previously visited areas in

3

(1) Mark all users as unassigned.

(2) For each unassigned user (u) in a fixed arbitrary order.

(a) Define minRT as the RT if u is connected directly to the MS.

(b) Among all PHs, find the PH that minimizes RT of u connected to the MSthrough PH (RTu→PH→MS) .

(i) If RTu→PH→MS is less than minRT then attach u to PH , and convert PH to an SSif it is not one yet.

(ii) Else, attach u directly to the MS.

(c) Mark user u as assigned.

(3) Output final solution.

Algorithm 1: DIM pseudo code

(1) Start from a complete mapping (generated randomly).

(2) Perform short hops (to do local search in a neighborhood) for a maximum number of iterations (itermax) exper-imentally set to 300.

(3) The set of secondary servers that represent the neighborhood of the final solution from (2) is added as an entry tothe tabu list of size 20 (oldest entry in the tabu list is replaced).

(4) Perform a long hop by randomly generating a complete mapping that has a set of secondary servers that is notidentical to a set in the tabu list.

(5) Steps (2) to (4) are executed until the available time for mapping expires (the time limit was 10 minutes).

(6) The best known mapping is output as the solution (once the available time expires).

Algorithm 2: Procedure for Tabu Search

Figure 2. DIM pseudo code.

(1) Mark all users as unassigned.

(2) For each unassigned user (u) in a fixed arbitrary order.

(a) Define minRT as the RT if u is connected directly to the MS.

(b) Among all PHs, find the PH that minimizes RT of u connected to the MSthrough PH (RTu→PH→MS) .

(i) If RTu→PH→MS is less than minRT then attach u to PH , and convert PH to an SSif it is not already one.

(ii) Else, attach u directly to the MS.

(c) Mark user u as assigned.

(3) Output final solution.

Algorithm 1: DIM pseudo code

3.4 Discrete Particle Swarm Optimiza-tion

Discrete particle swarm optimization (DPSO) isbased on the original particle swarm optimization[KeE95]. In [KeE95], the authors developed the algo-rithm based on the flocking behavior of animals suchas birds or fish. Instead of organisms, they called theobjects flying through the air “particles.” The DPSO im-plemented in this study is based on the discrete versionof particle swarm optimization in [PuM06].

For the DPSO, there are P particles (determined em-pirically), and each particle has N dimensions. The lo-cation of the ith particle in the jth dimension (Xij) in-dicates whether the jth user is an SS or a non-SS in theith particle. In general, the position of the particle willrepresent the set of SS used by the MRT heuristic to de-termine the full mapping. Vij is the velocity of the ith

particle in the jth dimension (Vij ∈ (−∞,∞)). To con-vert this velocity to a position a sigmoid function (s(Vij)= 1/

(1 + e−Vij

)) is used (this function returns a value

from s(Vij) ∈ [0, 1]). Each particle i has a best personal(Pij for j : 1 ≤ j ≤ N ) and the swarm has a best global(Gj for j : 1 ≤ j ≤ N ).

We define w as the inertia coefficient that slows thevelocity of the particle over time, pw as the attractionto the personal best solution, and gw as the attractionof the particle to the global best solution. These threeconstants are determined empirically. The procedure isshown in Figure 3.

3.5 Ant Colony Optimization

Ant colony optimization (ACO) is a population basedapproach that mimics the path finding behavior of ants[MaC96]. In an ant colony, ants will wander randomlylooking for food. Once food is found the ant returnsto the colony while laying a pheromone trail. Otherants are more likely to follow this trail to the food thusstrengthing the pheromone trail. In the ACO heuristic,each ant represents a full solution and ants receive pos-itive feedback about past solutions through pheromonetrails.

Figure 2. DIM pseudo code.

the search space using a tabu list so they are not revis-ited. To make the size of the tabu list reasonable, onlythe last n (set empirically to 20) visited neighborhoodsare saved [12]. The solution was represented by vectorsas in the DIM heuristic.

Local moves (or short hops) explore the neighbor-hood of the current solution, searching for the local min-imum. All the moves that we use in the Tabu Search areconsidered greedy in the sense that we accept a neigh-boring solution if it has a smaller RTmax (better objec-tive function value); however, applying greedy movesmay cause the Tabu Search to reach a local minimumthat it cannot escape. The global move (or long hop) isused to escape local minima by producing a random so-lution with a new set of SSs that is not in the tabu list.The procedure for Tabu Search is shown in Figure 4 andthe procedure for the short hops is shown in Figure 5.

3.4 Minimum Return Time (MRT)

The Minimum Return Time (MRT) heuristic is basedon the Minimum Completion Time (MCT) heuristic [14,22]. This heuristic is used by the discrete particle swarmoptimization heuristic to generate a solution from a setof users that are connected directly to the MS.

Directly connected users (DCUs) are users that areconnected directly to the MS. In MRT, the users that arenot DCUs are assigned in a fixed arbitrary order to theDCU that gives each user its minimum RT. The pseudo-code for implementing the MRT is shown in Figure 6.

3.5 Discrete Particle SwarmOptimization

Discrete particle swarm optimization (DPSO) isbased on the original particle swarm optimization [18].In [18], the authors developed the algorithm based onthe flocking behavior of animals such as birds or fish. In-stead of organisms, they called the objects flying through

the air “particles.” The DPSO implemented in this studyis based on the discrete version of particle swarm opti-mization in [23].

For the DPSO, there are P (determined empirically)particles, and each particle has N dimensions, whereeach dimension corresponds to a user. For the ith par-ticle, the value in the jth dimension (Xij) indicateswhether the jth user is a DCU or a non-DCU (0 if theuser is not a DCU and 1 if the user is a DCU). A particlehas a set of DCUs that will be used by the MRT heuristicto determine the resource allocation. This representationwill cause the search space to be a hypercube.

The velocity of a particle is used to explore other ar-eas of the hypercube. Vij is the velocity of the ith par-ticle in the jth dimension (Vij ∈ (−∞,∞)). To con-vert this velocity to a position a sigmoid function (s(Vij)= 1/

(1 + e−Vij

)) is used (this function returns a value

from 0 to 1). At each iteration of the DPSO, the positionof particle i may be changed for all i; however, the bestpersonal solution (P ∗

i ), is updated only when particle ifinds a better solution. Thus, each particle i has a bestpersonal solution (P ∗

i , an N dimensional vector). Theswarm has a best global solution (G∗, an N dimensionalvector). G∗ is equal to the P ∗

i with the lowest RTmax

over all i. The particles will be attracted to their bestpersonal solution and to the best global solution. Thisattraction allows exploration around the best known so-lutions. The notation P ∗

ij represents the value (0 or 1) of

the jth user for the best personal solution of particle i,and G∗

j represents the value of the jth user for the bestglobal solution.

We define w as the inertia coefficient that slows thevelocity of the particle over time, pw as the attraction tothe best personal solution, and gw as the attraction of theparticle to the best global solution. These three constantsare determined empirically. The particles are allowed tomove for itermax iterations. The procedure is shown inFigure 7.

4

(1) RTbest is equal to the value of RTmax of the solution from the DIM heuristic.

(2) For each user (Ux) connected to an SS.

(a) Connect Ux to the MS and connect the user with RTmax to Ux.

(b) Find the RTmax of this configuration.

(c) If RTmax < RTbest then save this solution as the best solution.

(2) For each user (Ux) connected to an SS.

(a) Swap Ux with the user with RTmax.

(b) Find the RTmax of this configuration.

(c) If RTmax < RTbest then save this solution as the best solution otherwise undo the swap.

(3) For each SS (SSx).

(a) Connect the user with RTmax to SSx.

(b) Find the RTmax of this configuration.

(c) If RTmax < RTbest then save this solution as the best solution otherwise undo the change.

(4) Output best solution.

Algorithm 1: Procedure for the DIM post-optimization.

is the RTmax of the kth ant. Let ∆τkij(t) represent how

assigning user i to server j influences the update to thepheromone value. If the kth ant assigns user i to serverj then

∆τkij(t) =

1RTmax(s[k])

,

otherwise, user i was not assigned to server j and

∆τkij(t) = 0.

The value of ∆τij(t + 1) represents how attractive theassignment of user i to server j was among all the ants initeration t and it is calculated using the following equa-tion:

∆τij(t + 1) =number of ants∑

k=1

∆τkij(t) .

Let τij(t) be the pheromone value associated withassigning user i on server j in iteration t, and ρ is theevaporation coefficient. The update equation for thepheromone is:

τij(t + 1) = (1− ρ) · τij(t) + ρ ·∆τij(t + 1) .

For s[k], the probability of assigning player i toserver j is based on the desirability of this assignment(η(s[k − 1], i, j)). This desirability is calculated usingthe following equation:

η(s[k − 1], i, j) =1

RTmax(s[k])−RTmax(s[k − 1]).

The equation that calculates the probability of user ibeing assigned to server j (pre(s[k − 1], i, j)) is influ-enced by α and β. These variables can be modified to

improve the performance of the ACO, for our implemen-tation the best values were determined empirically to beα = 7 and β = 9. The probability is calculated with thefollowing equation:

pre(s[k−1], i, j) =τij(t)

α · η(s[k − 1], i, j)β∑r∈Sets

τir(t)α · η(s[k − 1], i, r)β

.

A random variable (numrand) is generated with a uni-form distribution (U(0,1)). Using numrand, the chosenserver (sc) is the server that satisfies the following equa-tion:

sc∑s=1

pre(s[k − 1], i, s) ≤ numrand , (6)

numrand <

sc+1∑s=1

pre(s[k − 1], i, s) . (7)

The ACO procedure is shown in Figure 8.

3.7 Min-Min RT

The Min-Min RT heuristic is based on the con-cept of Min-Min heuristic [13]. The Min-Min heuris-tic is widely used in the area of resource allocation[7, 10, 11, 13, 15, 16, 18, 25]. It takes a given set ofpotential SSs and assigns remaining users to a potentialSSs or to the MS. Recall that it possible for a potentialSS to not have any users connected to it. The proce-dure to implement the Min-Min RT is shown in Figure9. This heuristic is used by the RT Iterative Minimiza-tion and the Genetic Algorithm.

Figure 3. DIM post-optimization pseudo code.

(1) Start from a complete resource allocation (generated randomly).

(2) Perform short hops (to do local search in a neighborhood) for a maximum number of iterations (itermax) exper-imentally set to 300.

(3) The set of secondary servers that represent the neighborhood of the final solution from (2) is added as an entry tothe tabu list of size 20 (oldest entry in the tabu list is replaced).

(4) Perform a long hop by randomly generating a complete resource allocation that has a set of secondary serversthat is not identical to a set in the tabu list.

(5) Steps (2) to (4) are executed until the available time for resource allocation expires (the time limit was 10 minutes).

(6) The best known resource allocation is output as the solution (once the available time expires).

Algorithm 1: Procedure for Tabu Search

(1) Given a predetermined set of SSs, all users that are not in the set of SS are marked as unassigned.

(2) For each unassigned user, the server (main or secondary) that gives the minimum RT is determined (first mini-mum).

(3) The best paired user/server (i.e., with smallest RT) among all the pairs generated in (2) is selected (secondminimum).

(4) The user in the best pair selected in (3) is then assigned to its paired server.

(5) Steps (2) through (4) are repeated until all tasks are assigned.

Algorithm 2: Procedure for Min-Min RT

(1) An initial population of 100 (determined empirically) chromosomes is generated.

(2) For each chromosome, the RTmax is calculated based on the full resource allocation determined with the Min-Min RT heuristic.

(3) While there are less than 1000 iterations without improvement or 10 minutes have not elapsed.

(a) 50 pairs of parents are selected using the linear bias function.

(b) From the 50 pairs of parents, 100 offspring are generated using 2 point crossover.

(c) For each offspring there is a 1% probability (determined empirically) of mutating each field in the chromo-some.

(d) A new population is created with the offspring. If the elite chromosome (the best chromosome found so far)is not in the new population then remove the worst offspring and insert the elite chromosome in its place.

(4) The output is the elite chromosome.

Algorithm 3: Procedure for the GA

Figure 4. Procedure for Tabu Search.

3.6 Min-Min RT

The Min-Min RT heuristic is based on the con-cept of Min-Min heuristic [14]. The Min-Min heuris-tic is widely used in the area of resource allocation[7, 10, 11, 14, 16, 17, 19, 27]. It takes a given set ofDCUs and assigns remaining users to a DCU or to theMS. The procedure to implement the Min-Min RT isshown in Figure 8. This heuristic is used by the RT Iter-ative Minimization and the Genetic Algorithm.

3.7 Genetic Algorithm

The genetic algorithm (GA) is a heuristic based onthe process of evolution [13]. It uses a chromosome torepresent a solution; this solution can be “mixed” with

other solutions and “mutated” to produce new solutions.In this study, the chromosome represents which play-

ers are DCUs. To convert this representation to a re-source allocation, we use the Min-Min RT heuristic. Theprocedure for the chosen variation of a GA is shown inFigure 9. The selection process for this heuristic is doneusing the linear bias function [26].

3.8 RT Iterative Minimization

This greedy heuristic consists of two phases. In phase1, we iteratively adjust the number of DCUs (by addingor removing) to minimize RTmax. In phase 2, we swapthe DCUs with users to minimize RTmax. This heuristicalso uses the Min-Min RT heuristic to convert a set ofDCUs to a resource allocation. The procedure for RT

5

(1) Set the number of executed short hops sh to 0

(2) Do:

(a) Increase sh by 1 and determine the user with RTmax (denoted URTmax). If URTmax is the only oneconnected to a given SS then go to (3) (to avoid changes in the neighborhood).

(b) For each server Sα (MS and all SSs) in a fixed arbitrary order, if assigning URTmax to Sα decreases theRTmax, then assign URTmax to Sα.

(3) While sh < itermax ( = 300) and a better assignment is found in (2b) then go to (2).

(4) Do:

(a) Increase sh by 1 and randomly select a user connected to the MS (UMS).

(b) For each secondary server SSα (only SSs) in a fixed arbitrary order, if assigning UMS to SSα decreasesthe RTmax, then assign UMS to SSα.

(5) While sh < itermax ( = 300) and a better assignment is found in (4b) then go to (4).

(6) Do:

(a) Increase sh by one and determine URTmax.

(b) For each server Sα (MS and all SSs) in a fixed arbitrary order, if assigning URTmax to Sα decreases theRTmax then assign URTmax to Sα. If the assignment of URTmax to Sα leaves an SS without any usersconnected it, then it is removed from the set of secondary servers.

(7) While sh < itermax ( = 300) and a better assignment is found in (6b) then go to (6).

Algorithm 1: Short hop moves for Tabu Search.

as the Chair and on the Program Committee for numer-ous other conferences. An up-to-date vita is available atwww.engr.colostate.edu/∼aam.Ye Hong is a Ph.D. student in Electrical and ComputerEngineering at Colorado State University. He receivedhis Master degree in Computer Science from TsinghuaUniversity in 2006, and his Bachelor degree from Ts-inghua University in 2002. His current research inter-ests include parallel and distributed computing, hetero-geneous computing, robust computer systems, and per-formance evaluation.Bradley L. Lock received his B.S. in Computer Engi-neering from Colorado State University in May 2001.He is pursuing a M.S. degree in Electrical Engineeringat Colorado State University. He currently works for In-tel Corporation as a VLSI design engineer.Mohammad Nayeem received his B.E. degree in Elec-tronics and Communications Engineering from Re-gional Engineering College, University of Kashmir in2000 and M.S. in Computer Science from ColoradoState University in 2007. He worked in Wipro InfotechIndia from 2000 to 2003 as Customer Services Engineerand as software engineer in Hewlett-Packard from 2003to 2005. He is currently a Computer Science Ph.D. stu-dent at Colorado State University. His research is fo-cused on machine learning, scheduling, and optimiza-tion using artificial intelligence techinques.Fadi Wedyan received his B.S. in Computer Science

from Yarmouk University in 1995 and M.S. in Com-puter Science from Al-albayt University in 1999. Heworked as a Systems developer from 1996 to 1999 atthe Free Zones corp., Jordan and as an Instructor at Jor-dan University of Science and Technology from 2000 to2005. He is currently a Computer Science Ph.D. studentat Colorado State University. His research interests arescheduling and optimization.Christopher Klumph is an undergraduate senior stu-dent at Colorado State University seeking a B.S. degreein Computer Engineering. His senior design project fo-cuses on research in heterogeneous computing.Kody K. Willman is attending Colorado State Univer-sity to receive her B.S. degree in Electrical and Com-puter Engineering and a minor in Mathematics. She isinterested in pursuing research in heterogeneous com-puting systems, network engineering and maintenance,and mobility and robotic systems.

Figure 5. Short hop moves for Tabu Search.

(1) A list of users that are not DCUs is generated in a fixed arbitrary order.

(2) The first user in this list is assigned to the DCU that gives it the minimum RT .

(3) The user assigned in (2) is removed from the list.

(4) The computation time for the DCU where the user was assigned is updated.

(5) Steps (2) through (4) are repeated until all users have been assigned.

Algorithm 1: Procedure for MRT.

(1) Initialize an array of P particles by N dimensions randomly with 0 or 1 (a value of 0 indicates a user is not aDCU and 1 indicates the user is a DCU).

(2) Determine RTmax using the MRT heuristic.

(3) Initialize the global and best personal solutions (P ∗i s) using the particles in (1), and set the best global solution to

the best P ∗i over all i.

(4) For (i = 1 to P ) do

(a) For (j = 1 to N ) do

(i) Generate 3 different random variables (R1,R2, and R3) each with a uniform distribution [0, 1] ∈ <(ii) Vij = w · Vij + pw ·R1 · (P ∗

ij −Xij) + gw ·R2 · (G∗j −Xij)

(iii) If Vij is less than the minimum allowed velocity (Vmin) then Vij = Vmin.(iv) If Vij is greater than the maximum allowed velocity (Vmax) then Vij = Vmax.(v) If (R3 < S(Vij)) then Xij = 1, otherwise Xij = 0.

(b) Determine RTmax using the MRT heuristic.

(5) Set the best personal solutions (P ∗i ) using the particles in (4),and set the best global solution to the best P ∗

i overall i.

(6) Repeat (4) until the number of iterations is equal to itermax.

Algorithm 2: Procedure for DPSO.

5

Figure 6. Procedure for MRT.

Iterative Minimization is shown in Figures 10 (Phase 1)and 11 (Phase 2).

4 Lower Bound

The primary purpose of deriving a mathematicallower bound was to evaluate the experimental results ofour proposed heuristics. The bound has two componentsthat can be calculated independently. The first compo-nent finds the minimum possible computation time ofthe MS and SSs (by performing an exhaustive searchof all possible computation times). This componenthas three assumptions: (a) communication times are ig-nored, (b) all users have the same computational con-stant (→ µmin = min∀Ux

µx), and (c) users connectedto SSs are evenly distributed among SSs. Compo-

nent (a) reduces the complexity because heterogeneityin communication is removed from the problem, (b)removes the heterogeneity in computing power of theSSs, and (c) minimizes the maximum computation timeamong SSs.

The second component is the lower bound on thecommunication time. This bound is calculated by find-ing the minimum time each user requires to connect tothe MS (either connected directly to the MS or throughanother user), and then finding the maximum of thesetimes. For the calculation of this second component, thecomputation time is ignored.

Let n be the total number of users that are connectedto the MS, nnss plus nmain. The lower bound (LB) isgiven as:

6

(1) A list of users that are not DCUs is generated in a fixed arbitrary order.

(2) The first user in this list is assigned to the DCU that gives it the minimum RT .

(3) The user assigned in (2) is removed from the list.

(4) The computation time for the DCU where the user was assigned is updated.

(5) Steps (2) through (4) are repeated until all users have been assigned.

Algorithm 1: Procedure for MRT.

(1) Initialize an array of P particles by N dimensions randomly with 0 or 1 (a value of 0 indicates a user is not aDCU and 1 indicates the user is a DCU).

(2) Determine RTmax using the MRT heuristic.

(3) Initialize the global and best personal solutions (P ∗i s) using the particles in (1), and set the best global solution to

the best P ∗i over all i.

(4) For (i = 1 to P ) do

(a) For (j = 1 to N ) do

(i) Generate 3 different random variables (R1,R2, and R3) each with a uniform distribution [0, 1] ∈ <(ii) Vij = w · Vij + pw ·R1 · (P ∗

ij −Xij) + gw ·R2 · (G∗j −Xij)

(iii) If Vij is less than the minimum allowed velocity (Vmin) then Vij = Vmin.(iv) If Vij is greater than the maximum allowed velocity (Vmax) then Vij = Vmax.(v) If (R3 < S(Vij)) then Xij = 1, otherwise Xij = 0.

(b) Determine RTmax using the MRT heuristic.

(5) Set the best personal solutions (P ∗i ) using the particles in (4),and set the best global solution to the best P ∗

i overall i.

(6) Repeat (4) until the number of iterations is equal to itermax.

Algorithm 2: Procedure for DPSO.

5

Figure 7. Procedure for DPSO.

(1) Given a predetermined set of DCUs, all users that are not in the set of DCUs are marked as unassigned.

(2) For each unassigned user, the server (main or DCU) that gives the minimum RT is determined (first minimum).

(3) The best paired user/server (i.e., with smallest RT) among all the pairs generated in (2) is selected (secondminimum).

(4) The user in the best pair selected in (3) is then assigned to its paired server.

(5) Steps (2) through (4) are repeated until all tasks are assigned.

Algorithm 3: Procedure for Min-Min RT

(1) An initial population of 100 (determined empirically) chromosomes is generated.

(2) For each chromosome, the RTmax is calculated based on the full resource allocationdetermined with the Min-Min RT heuristic.

(3) While there are less than 1000 iterations without improvement in the best solutionor 10 minutes have not elapsed.

(a) 50 pairs of parents are selected using the linear bias function.

(b) From the 50 pairs of parents, 100 offspring are generated using 2 point crossover.

(c) For each offspring there is a 1% probability (determined empirically) of mutatingeach field in the chromosome.

(d) A new population is created with the offspring. If the elite chromosome (the best chromosome found so far)is not in the new population then remove the worst offspring and insert the elite chromosome in its place.

(4) The output is the elite chromosome.

Algorithm 4: Procedure for the GA

7

Figure 8. Procedure for Min-Min RT.

(1) Given a predetermined set of DCUs, all users that are not in the set of DCUs are marked as unassigned.

(2) For each unassigned user, the server (main or DCU) that gives the minimum RT is determined (first minimum).

(3) The best paired user/server (i.e., with smallest RT) among all the pairs generated in (2) is selected (secondminimum).

(4) The user in the best pair selected in (3) is then assigned to its paired server.

(5) Steps (2) through (4) are repeated until all tasks are assigned.

Algorithm 3: Procedure for Min-Min RT

(1) An initial population of 100 (determined empirically) chromosomes is generated.

(2) For each chromosome, the RTmax is calculated based on the full resource allocationdetermined with the Min-Min RT heuristic.

(3) While there are less than 1000 iterations without improvement in the best solutionor 10 minutes have not elapsed.

(a) 50 pairs of parents are selected using the linear bias function.

(b) From the 50 pairs of parents, 100 offspring are generated using 2 point crossover.

(c) For each offspring there is a 1% probability (determined empirically) of mutatingeach field in the chromosome.

(d) A new population is created with the offspring. If the elite chromosome (the best chromosome found so far)is not in the new population then remove the worst offspring and insert the elite chromosome in its place.

(4) The output is the elite chromosome.

Algorithm 4: Procedure for the GA

7

Figure 9. Procedure for the GA.

7

Phase 1:

(1) Randomly pick k (a random number between 0 to N ) users to form the initial set of DCUs (SetSS),with ρSS ∈ SetSS .

(2) Create a full resource allocation using Min-Min RT heuristic with SetSS .

(3) Find the user with RTmax. If the user is in SetSS with no players connected to it thenremove it from SetSS and go to (2).

(4) Set RTref = RTmax.

(5) For each ρSS ∈ SetSS . Find the RTmax using the Min-Min RT heuristic with SetSS − {ρSS}.This will create |SetSS | values of RTmax and for each a corresponding set of secondary servers.

(6) Find the smallest RTmax value and the set of DCUs that gives the smallest RTmax from 5, and store them asRTremove and Setremove, respectively.

(7) For each non-SS user (ρnonSS /∈ SetSS). Find the RTmax using the Min-Min RT heuristic with SetSS ∪{ρnonSS}.This will create N − |SetSS | values of RTmax and for each a corresponding set of secondary servers.

(8) Find the smallest RTmax value and the set of SSs that gives the smallest RTmax from 7, and store them as RTadd

and Setadd, respectively.

(9) If RTremove < RTadd and RTremove < RTref , then SetSS = Setremove.

(10) If RTadd ≤ RTremove and RTadd < RTref , then SetSS = Setadd.

note If RTadd > RTref and RTremove > RTref , then no improvement was found.

(11) Steps (3) through (10) are repeated until 1000 iterations have passed or no improvement is found.

Algorithm 1: RT Iterative Minimization: phase 1

13

Figure 10. RT Iterative Minimization: Phase 1.

8

Phase 2:

(1) Determine the set of DCUs in the resource allocation (denoted SetSS) from the resource allocation produced inPhase 1.

(2) Create a full resource allocation using the Min-Min RT heuristic with SetSS .

(3) Find the user with RTmax (denoted URTmax). If URTmax is in SetSS with no players connected to it thenremove it from SetSS and go to (2).

(4) Set RTref to RTmax.

(5) If URTmax is connected to a secondary server (SSmax), then

(a) For each ρnonSS /∈ SetSS , find the RTmax using the Min-Min RT heuristic with SetSS ∪ {ρnonSS} −{SSmax}.This will create N − |SetSS | values of RTmax and for each a corresponding set of secondary servers.

(b) Find the smallest RTmax value and the set of DCUs that gives the smallest RTmax from 5(a), and storethem as RTswap and Setswap, respectively.

(c) If RTswap < RTref , then SetSS = Setswap; else, no improvement is obtained.

(6) If URTmax is connected to the MS, then

(a) For each ρnonSS , find the RTmax if ρnonSS is converted to an SS and URTmaxis attached to it. This willcreate N − |SetSS | values of RTmax and for each a corresponding ρnonSS .

(b) Find the smallest RTmax value and the ρnonSS that gives the smallest RTmax (now called ρtoswap)from 6(a).

(c) For each ρSS , find the RTmax using the Min-Min RT heuristic with SetSS ∪ {ρtoswap} − {ρSS}.This will create |SetSS | values of RTmax and for each a corresponding set of secondary servers.

(d) Find the smallest RTmax value and the set of DCUs that gives the smallest RTmax from 6(c), and storethem as RTSSswap and SetSSswap, respectively.

(e) If RTSSswap < RTref , then SetSS = SetSSswap; else, no improvement is obtained.

(7) Steps (4) through (6) are repeated until 1000 iterations have passed or no improvement is found.

(8) To get the final solution, use Min-Min RT heuristic with SetSS .

Algorithm 1: RT Iterative Minimization: phase 2.

13

Figure 11. RT Iterative Minimization: Phase 2.

9

fcomp(n, nnss) = µmin ·⌈

N − n

nnss

⌉2

+ c · (N − n) + b · n2 ; (6)

fcomm(Ux, Uy) = 2 · Comm(Ux, Uy)+ 2 · Comm(Uy,MS) ; (7)

The case where Ux = Uy is considered to account forthe case when Ux is connected to the MS.

LB = min1≤n≤N

(min

0≤nnss≤n(fcomp(n, nnss))

)+ max

Ux∈all users

(min

Uy∈all users(fcomm(Ux, Uy))

). (8)

Proof. The proof will be divided into two parts. Thefirst part will be to prove that the computational bound isminimum and the second part will be to prove the com-municational minimum.

The first part of the bound does an exhaustive evalu-ation of all possible configurations for nnss and n. Thiswill give us all the possible computations times. It willmove n from 1 (only one user connected to the MS) toN (all users connected to the MS). For each of thesevalues of n it will attempt all possible configurations ofnnss ≤ n. It is important to note that nnss = 0 is aninvalid configuration unless n = N (i.e., the only sce-nario where we do not have SSs is when all users areconnected directly to the MS), and in this case we con-sider (N − N)/(0) = 0. Because we are consideringall the possible configurations it is not possible to get asmaller computation time.

The second part of the bound finds the smallest com-munication time for each user, then it finds the max-imum among these times. This method does an ex-haustive search of the possible communication times(through an SS or directly connected to the MS).Therefore, the user with the maximum communicationtime cannot have a smaller communication time inde-pendently of the configuration.

5 Results

The simulation had 200 users interacting in theMMOG environment. The constants for these simula-tions were b = 0.03 and c = 0.06 (the values for theseconstants were set to approximate realistic values forlatencies in an MMOG environment). For this study,100 scenarios were created with varying communicationtimes and µα for each user. Because the RTmax of theoptimal solution can be intractable to compute, a lowerbound was used to compare the performance of the re-sults.

Figure 12 shows the results averaged over the 100scenarios. All the results are shown with a 95% con-fidence interval. From Figure 12, we can observe thatthe Tabu Search had the worst performance comparedto all of the other heuristics. One problem with theTabu Search is that the long hop can result in a verypoor solution. This was improved by seeding the re-sult of the DIM into the Tabu Search. When the DIM isused in the Tabu Search (Seeded Tabu), it outperformsall other heuristics. The DIM, DSPO (1500 iterations us-ing a population of 600 particles with pw and gw equalto 2, and w equal to 1), and GA heuristics performedrelatively well compared to all the other heuristics (ap-proximately 20 time units above the LB). The RT forthe Seeded Tabu was about 13 time units more than themathematical lower bound.

If all users were connected to the MS then theRTmax would be approximately 1200 time units (2002 ·b + max∀Ux 2 · Comm(Ux,MS) ≈ 1200). The use ofthe secondary server based approach in our simulationsleads to an improvement of an order of magnitude (i.e.,83 time units versus 1200 time units).

6 Related Work

Various MMOG architectures are reported in the lit-erature (e.g. client/server [9], peer-to-peer [5, 15, 20],mirrored server [8]). Each architecture has its ownadvantages, for example the client/server and mirroredserver allows the company that develops the MMOG en-vironment to maintain a tight control of the game state;however, there is a significant monetary cost associatedwith maintaining a large-scale MMOG environment. Ina peer-to-peer architecture, because of the absence of acentralized game state controller; no peer has full con-trol over the game state making it difficult to assert aconsistent VWE. The advantage of using a peer-to-peerarchitecture is that there is no single point of failure andthe MMOG environment can be maintained without asignificant monetary cost.

Maintaining a seamless interactive experience for theusers is an important factor in MMOG, because an in-crease in latency within the system can lead to deterio-ration in the gaming experience [2, 9]. In [15], the au-thors show that the latency follows a “weak exponentialincrease” as the number of users grows in system. Ourstudy focuses on latency as a critical performance pa-rameter that must be maintained and uses the results in[15] to model the relationship between latency and thenumber of users.

This study proposes a hybrid client/server architec-ture to combine the best elements of both the centralizedclient/server and peer-to-peer architectures. Our workis most similar to [21]. In [21], a distributed systemuses intermediate servers (analogous to our definition ofsecondary servers) to reduce the communication latencyto the central server. The main difference between our

10

90.32

104.12

88.8492.19

83.1

89.2

71.7296

0

20

40

60

80

100

120

DIM Tabu Search GA RT Iterative

Minimization

Seeded Tabu DSPO LB

heuristics

RT

max (

tim

e u

nit

s)

Figure 12. Results for heuristics with the computational parameters of the MS set to: b = 0.03and c = 0.06. The values are averaged over 100 scenarios, and the error bars show the 95%confidence intervals

studies is that in [21] the intermediate servers are pre-defined and do not participate as users in the VWE. Ourwork is different from [8, 9] because it considers con-verting users to secondary servers. This work is alsodifferent to [5, 15, 20] because it has a “non-peer” cen-tralized server. The use of the centralized server in thehybrid approach may have a single point of failure, how-ever it allows the game developer to control the MMOGenvironment and uses peers to reduce the computationof the main server.

7 Conclusions

This study evaluated an oversubscribed VWE thatemploys a group of users to do parts of the calculationsrequired to operate a VWE. The main objective of thisstudy was to develop heuristics to evaluate the perfor-mance of a secondary server based VWE.

In this study, we showed that the secondary serversadd capacity to the main server. This extra capacitycan be used to maintain an acceptable performance evenwhen the VWE is oversubscribed. The heuristics wereable to improve the performance the MS would be ableto have if all users were directly connected to the it bymore than an order of magnitude. The LB was calcu-lated to compare the performance of the heuristics to amathematical bound on performance. The LB indicatesthat the solutions for the GA and Seeded Tabu Searchare close to the LB (by about 17 and 12 time units re-

spectively). It would be interesting to seed the GA withthe solution from the DIM heuristic to see if the solutionof the GA improves.

A possible expansion of this study is to improve themodel by removing the simplifying assumptions (e.g.,the constant communication times, same time to the aserver and back). This study also assumed that users arewilling to become an SS. This problem could also bereformulated using game theory to consider the behaviorof selfish and/or cooperative users.

Acknowledgments: The authors would like to thankSamee Khan for his valuable contributions.

References

[1] S. Ali, T. D. Braun, H. J. Siegel, A. A. Maciejewski,N. Beck, L. Boloni, M. Maheswaran, A. I. Reuther, J. P.Robertson, M. D. Theys, and B. Yao. Characterizing re-source allocation heuristics for heterogeneous comput-ing systems. In Advances in Computers Volume 63: Par-allel, Distributed, and Pervasive Computing, pages 91–128, 2005.

[2] G. J. Armitage. An experimental estimation of latencysensitivity in multiplayer quake 3. In 11th IEEE Interna-tional Conference on Networks (ICON ’03), pages 137–141, Sept. 2003.

[3] L. Barbulescu, L. D. Whitley, and A. E. Howe. Leap be-fore you look: An effective strategy in an oversubscribedscheduling problem. In 19th National Conference on Ar-tificial Intelligence, pages 143–148, July 2004.

11

[4] N. E. Baughman and B. N. Levine. Cheat-proof playoutfor centralized and distributed online games. In IEEEConference on Computer Communications (INFOCOM’01), pages 104–113, Mar. 2001.

[5] A. Bharambe, J. Pang, and S. Seshan. Colyseus: A dis-tributed architecture for online multiplayer games. In3rd Symposium on Networked Systems Design and Im-plementation, pages 155–168, 2006.

[6] T. D. Braun, H. J. Siegel, N. Beck, L. Boloni, R. F. Fre-und, D. Hensgen, M. Maheswaran, A. I. Reuther, J. P.Robertson, M. D. Theys, and B. Yao. A comparisonof eleven static heuristics for mapping a class of inde-pendent tasks onto heterogeneous distributed computingsystems. Journal of Parallel and Distributed Computing,61(6):810–837, June 2001.

[7] L. D. Briceno, M. Oltikar, H. J. Siegel, and A. A. Ma-ciejewski. Study of an iterative technique to minimizecompletion times on non-makespan machines. In Inter-national Heterogeneity in Computing Workshop (HCW’07), page 138, Mar. 2007.

[8] E. Cronin, A. R. Kurc, B. Filstrup, and S. Jamin. Anefficient synchronization mechanism for mirrored gamearchitectures. Multimedia Tools Applications, 23(1):7–30, 2004.

[9] G. Deen, M. Hammer, J. Bethencourt, I. Eiron,J. Thomas, and J. H. Kaufman. Running quake ii ona grid. IBM Systems Journal, 45(1):21–44, 2006.

[10] Q. Ding and G. Chen. A benefit function mappingheuristic for a class of meta-tasks in grid environments.In 1st International Symposium on Cluster Computingand the Grid (CCGRID ’01), page 654, May 2001.

[11] S. Ghanbari and M. R. Meybodi. On-line mapping al-gorithms in highly heterogeneous computational grids:A learning automata approach. In International Con-ference on Information and Knowledge Technology (IKT’05), May 2005.

[12] A. Hertz and D. de Werra. The tabu search metaheurstic:How we used it. Annals of Mathematics and ArtificialIntelligence, 1(1–4):111–121, Sept. 1990.

[13] J. Holland. Adaptation in Natural and Artificial Systems:An Introductory Analysis with Applications to Biology,Control, and Artificial Intelligence, First Edition. TheUniversity of Michigan, 1975.

[14] O. H. Ibarra and C. E. Kim. Heuristic algorithms forscheduling independent tasks on non-identical proces-sors. Journal of the ACM, 24(2):280–289, Apr. 1977.

[15] T. Iimura, H. Hazeyama, and Y. Kadobayashi. Zonedfederation of game servers: a peer-to-peer approach toscalable multi-player online games. In 3rd ACM SIG-COMM workshop on Network and System Support forGames, pages 116–120, Aug. 2004.

[16] Z. Jinquan, N. Lina, and J. Changjun. A heuristicscheduling strategy for independent tasks on grid. InEighth International Conference on High-PerformanceComputing in Asia-Pacific Region ’05, page 6, Nov.2005.

[17] K. Kaya, B. Ucar, and C. Aykanat. Heuristics forscheduling file-sharing tasks on heterogeneous systemswith distributed repositories. Journal of Parallel andDistributed Computing, 67(3):271–285, Mar. 2007.

[18] J. Kennedy and R. Eberhart. Particle swarm optimiza-tion. In IEEE International Conference on Neural Net-works, pages 1942–1948, Nov. 1995.

[19] J.-K. Kim, H. J. Siegel, A. A. Maciejewski, andR. Eigenmann. Dynamic mapping in energy constrainedheterogeneous computing systems. In IEEE Interna-tional Parallel and Distributed Processing Symposium(IPDPS ’05), Apr. 2005.

[20] B. Knutsson, H. Lu, W. Xu, and B. Hopkins. Peer-to-peer support for massively multiplayer games. In IEEEConference on Computer Communications (INFOCOM’04), pages 96–107, Mar. 2004.

[21] K.-W. Lee, B.-J. Ko, and S. Calo. Adaptive server selec-tion for large scale interactive online games. ComputerNetworks, 49(1):84–102, Sept. 2005.

[22] M. Maheswaran, S. Ali, H. J. Siegel, D. Hensgen, andR. F. Freund. Dynamic mapping of a class of indepen-dent tasks onto heterogeneous computing systems. Jour-nal of Parallel and Distributed Computing, 59(2):107–121, Nov. 1999.

[23] J. Pugh and A. Martinoli. Discrete multi-valued particleswarm optimization. In IEEE Swarm Intelligence Sym-posium ’06, pages 103–110, May 2006.

[24] A. Shaikh, S. Sahu, M.-C. Rosu, M. Shea, and D. Saha.On demand platform for online games. IBM SystemsJournal, 45(1):7–20, 2006.

[25] V. Shestak, E. K. P. Chong, H. J. Siegel, A. A. Ma-ciejewski, L. Benmohamed, I.-J. Wang, and R. Daley.A hybrid branch-and-bound and evolutionary approachfor allocating strings of applications to heterogeneousdistributed computing systems. Journal of Parallel andDistributed Computing. accepted, to appear.

[26] D. Whitley. The genitor algorithm and selective pres-sure: Why rank based allocation of reproductive trials isbest. In 3rd International Conference on Genetic Algo-rithms, pages 116–121, June 1989.

[27] M. Wu and W. Shu. Segmented min-min: A static map-ping algorithm for meta-tasks on heterogeneous com-puting systems. In 9th IEEE Heterogeneous ComputingWorkshop (HCW ’00), pages 375–385, Mar. 2000.

Biographies

Luis D. Briceno is currently pursuing his Ph.D. degreein Electrical and Computer Engineering at ColoradoState University. He obtained his B.S. degree in elec-trical and electronic engineering from the Universityof Costa Rica. His research interests include heteroge-neous parallel and distributed computing.Howard Jay Siegel was appointed the Abell EndowedChair Distinguished Professor of Electrical and Com-puter Engineering at Colorado State University (CSU)in 2001, where he is also a Professor of ComputerScience. He is the Director of the CSU Information Sci-ence and Technology Center (ISTeC), a university-wideorganization for promoting, facilitating, and enhancingCSU’s research, education, and outreach activitiespertaining to the design and innovative application ofcomputer, communication, and information systems.From 1976 to 2001, he was a professor at PurdueUniversity. Prof. Siegel is a Fellow of the IEEE anda Fellow of the ACM. He received a B.S. degree inelectrical engineering and a B.S. degree in management

12

from the Massachusetts Institute of Technology (MIT),and the M.A., M.S.E., and Ph.D. degrees from theDepartment of Electrical Engineering and ComputerScience at Princeton University. He has co-authoredover 350 technical papers. His research interests includerobust computing systems, resource allocation in com-puting systems, heterogeneous parallel and distributedcomputing and communications, parallel algorithms,and parallel machine interconnection networks. Hewas a Coeditor-in-Chief of the Journal of Paralleland Distributed Computing, and was on the EditorialBoards of both the IEEE Transactions on Parallel andDistributed Systems and the IEEE Transactions on Com-puters. He was Program Chair/Co-Chair of three majorinternational conferences, General Chair/Co-Chair ofseven international conferences, and Chair/Co-Chairof five workshops. He is a member of the Eta KappaNu electrical engineering honor society, the SigmaXi science honor society, and the Upsilon Pi Epsiloncomputing sciences honor society. He has been aninternational keynote speaker and tutorial lecturer, andhas consulted for industry and government. For moreinformation, please see www.engr.colostate.edu/∼hj.Anthony A. Maciejewski received the B.S.E.E,M.S., and Ph.D. degrees in Electrical Engineeringin 1982, 1984, and 1987, respectively, all from TheOhio State University. From 1988 to 2001, he wasa Professor of Electrical and Computer Engineeringat Purdue University. In 2001, he joined ColoradoState University where he is currently the Head of theDepartment of Electrical and Computer Engineering.He is a fellow of IEEE. An up-to-date vita is availableat www.engr.colostate.edu/∼aam.Ye Hong is a Ph.D. student in Electrical and ComputerEngineering at Colorado State University. He receivedhis Masters degree in Computer Science from TsinghuaUniversity in 2006, and his Bachelors degree fromTsinghua University in 2002. His current researchinterests include parallel and distributed computing,heterogeneous computing, robust computer systems,and performance evaluation.Bradley L. Lock received his B.S. in Computer Engi-neering from Colorado State University in May 2001.He is pursuing an M.S. degree in Electrical Engineeringat Colorado State University. He currently works forIntel Corporation as a VLSI design engineer.Mohammad Nayeem received his B.E. degree inElectronics and Communications Engineering fromRegional Engineering College, University of Kashmir in2000 and his M.S. in Computer Science from ColoradoState University in 2007. He worked in Wipro InfotechIndia from 2000 to 2003 as a Customer ServicesEngineer and as a software engineer in Hewlett-Packardfrom 2003 to 2005. He is currently a Computer SciencePh.D. student at Colorado State University. His researchis focused on machine learning, scheduling, and opti-mization using artificial intelligence techinques.

Fadi Wedyan received his B.S. in Computer Sciencefrom Yarmouk University in 1995 and his M.S. inComputer Science from Al-albayt University in 1999.He worked as a Systems developer from 1996 to 1999at the Free Zones Corp., Jordan and as an Instructorat the Jordan University of Science and Technologyfrom 2000 to 2005. He is currently a Computer SciencePh.D. student at Colorado State University. His researchinterests are scheduling and optimization.

13