An Autonomic Open Marketplace for Inter-CloudService Management

Haydn Mearns, John Leaney, Artem Parakhine, John DebenhamCentre for Quantum Computation and Intelligent Systems

University of Technology, Sydney,Email: {haydn, johnl, aparakhi, debenham@it.uts.edu.au}

Dominique VerchereAlcatel-Lucent Bell Labs,

Email: Dominique.Verchere@alcatel-lucent.com

Abstract—The rise of utility in cloud computing and telecom-munications has introduced greater complexity in the provi-sioning and performance management of remote services. Wepropose extended management strategies for this complexity.Our overall aim is for the management to accept responsibilityfor the complex service in an open marketplace. Responsibilityis, firstly, defined by aiming to cover the totality of moderncomplex services, managing both the connectivity and virtualinfrastructure. Secondly, responsibility is further defined asmanaging risk and resilience in the provisioning and operation ofthe complex service. With these aims, we are working towards abundled service provider agent architecture, which can negotiateon the open service market. This approach aims to also optimisethe utilisation of the providers infrastructure while reducing therisk of failure to users through total service management. Wepresent the specification, design and simulation of the bundledservice agents in a marketplace environment.

Index Terms—Cloud Computing; Telecommunications;Agents; Open Market

I. INTRODUCTION

The relatively recent introduction of Cloud Computinghas opened up a wealth of possibilities in terms of datamanagement, application processing and scalability. Amazon’sEC2 public clouds allow service providers to rapidly increaseand decrease the availability of their service, with relativelylow overhead. The obvious advantages of this technology hasseen the Cloud Computing environment rapidly expand, withalready established public clouds such as Amazon offeringincreased services, space and availability, and telecommuni-cation companies starting to leverage their infrastructure toprovide hybrid cloud models. It is not hard to imagine that itis just a short step to the existence of an environment wherethe number of cloud environments allows real competition inthe market place.

A. Motivation

The growth in cloud environments has encouraged someresearchers to consider the issues involved in working withmultiple cloud or service providers. Researchers are lookingin the direction of marketplace interaction to provide bothincreased cloud utilisation and reduced cost. Amazon currentlyprovides a beginning to this market place with their Spotpricing for batch processing.

The reservoir project [1] argues that two main issues ofthe current cloud environment is a lack of scalability in single

cloud providers, and the lack of interoperability between them.We would like to add a further issue to those consideredby the reservoir group, which is responsibility for the entirecomplex service. For responsible coverage of any secure videoprocessing application run on a public cloud there also needssome secure quality assured bandwidth between the userand the application and its components, the management ofwhich is just as important as the management of the virtualmachine running in the cloud. Our motivation is twofold;firstly to give coverage of the total service sought in the cloud,and, secondly to take responsibility for the running of thatservice. Responsible coverage of the total service means thatthe cloud service is to be seen as the collection of all theuser’s applications and computing services, together with theirrelated communications services. Operational responsibilityfor the service is thus to be responsible for the applicationsand the communications. It is further defined as providingrisk management for the services (to price fairly and manageeffectively), to negotiate, provision and schedule, and, toensure a resilient service (in which poor performance and totalfailure are managed). This work builds on our bundled servicenetwork management in [2]

II. RELATED WORK

A. Service Management in Inter-Clouds

The reservoir federated cloud [1] is an architecture for thecombination of various cloud providers through their ServiceManifest SLA’s and service management agents into onefederated cloud, in order to manage the service across theproviders. While this system could manage the failure ofproviders in the federation, the management would come ata cost, as the federated cloud requires that its service manageragents run continuously across the infrastructure providers,even when not being utilised. The cloudbus project [3] alsoproposes an architecture for the interaction with multiple cloudproviders, through a market orientated cloud exchange, thatwould allow services to be negotiated through SLA’s to in-crease the scalability and performance of provisioned services.However, this version of the federation of cloud providersdoes not consider the necessity of guaranteeing the serviceconnection between the cloud providers and coordinating thebandwidth interaction.

B. Technical Integration

Researchers are already approaching the technical issuesin managing the entire bundle of services, providing routingas a service [4] and of moving virtual machines over widearea networks without losing service [5]. On a more basiclevel, the Amazon cloud and current research toolkits suchas cloudbus [6] already allow replication and on demandscalability in their applications, and accommodate multiplecurrent virtual machine architectures (such as VMware andXen).

C. Auctions and Negotiation For Cloud Services

Work has been done on auctions and negotiation mech-anisms for service level agreements with service providers.There are currently multiple types of auctions available forthese interactions, from single item auctions such as English,Dutch, and Vickery to Combinatorial auctions, which focuson more complex valuations across multiple resources, andadapting and ensuring QoS requirements in cloud computingSLA’s [7]. There have been various approaches to creatingan open-market for clouds and grids, such as SORMA [8],cloudbus [3], and GridWay [9] with the current proposalsare utilising various combinatorial auction strategies such asContinuous Double Auction [10], Zero-intelligence plus andQ-Strategy. There is currently however, no consensus on themost appropriate strategy for the market, and the pricing strat-egy is only one part of issues surrounding the implementationof a global open market. Further issues include a commonstructure for the negotiated SLA’s language. Previous workin our research group [11] has built an ontology for SLAnegotiation, which covers the information required for the lowlevel provisioning of complex services by agents.

D. Agents and Cloud Computing and Telecommunications

The inherent distribution of a telecommunications networkand the multiple services that they provide, corresponds wellwith a multi-agent systems’ ability to cooperate towards mul-tiple goals [12]. All of the previously mentioned systems forservice management in the inter-cloud, such as the reservoirproject, cloudbus, and SORMA utilise agents implicitly withindividual Service Manager agents interacting co-operativelytowards a Service Manifest in the case of reservoir, Cloudbrokers interacting with the cloud co-ordinator in the case ofcloudbus, and bidders and sellers in the case of SORMA.

SERA [13], is one system that explicitly utilises a multi-agent system in the integration of customer jobs and resourcemanagement. The distributed SERA architecture instantiatesboth Job agents, whose responsibilities include, getting re-sources, scheduling, stopping, suspending and running jobs,and the Resource agents, whose responsibilities include mon-itoring scheduled and running jobs, registering resources andlimited recovery. This limited recovery is defined as triggeringappropriate policies through the agents rule engine, policiesthat must be defined statically at design time.

III. DEVELOPMENT OF A SERVICE MODEL

A. Overall Model Description

We are working towards a bundled service agent architec-ture, which can negotiate on an open market, and which willeventually lead to optimisation of the utilisation of the cloudand telecommunication providers networks, while reducing therisk of service failure from the customers point of view. Figure1 shows the entity relationship for the providers describedbelow.

Fig. 1. Entity Relationship DiagramUsers: make bundled requests via their user agents.User Agent: (UA) negotiate with bundled service providers toobtain services at the required quality and price.Bundled Service (BS): as described by the TeleManagementforum [14], and extended to include cloud services, is acollection of multiple services offered with incentives, thatinclude extra organisational issues.Bundled Service Provider (BSP): a service provider whichprovides bundled services to users, and negotiates with (anumber of) single service providers including application andconnectivity to provide the user’s service, at the requiredquality.Bundled Service Agent (BSA): is a an agent assigned toa User request, and to the subsequent management of thebundled service.Single Service (SS): The contracted SLA with a single serviceprovider. For the purpose of this project we define the serviceprimitives to be; Remote Application (RA), Remote VirtualMachine (RM), Remote Virtual Storage (RS), Video Confer-ence (VC), Audio Conference (AC), and Network Connectivity(NC). Examples in [11].Single Service Provider (SSP): is a provider of single ser-vices. They may have many single services available.Single Service Agent (SSA): An agent in control of the singleservice for the selected service time period.Single Resource(SR): At this stage we are assuming that

an SR maps to a resource of the primitive single servicetype. Note: Within each service type primitive operationaldecisions are made with regards to response times, bandwidth,and processing capabilities. This variability will give riseto varying QoS, which can only be judged by measuredperformance.Single Resource Agent (SRA): An agent for a single resource.The agent keeps track of the resources allocated for the singleservices at the particular service times.

B. System Operation and Architecture

Fig. 2. The Architecture.Figure 2 shows the overall architecture for bundled services

in an open marketplace. Users will have agents which willnegotiate in a User Bundled Service Provider environment, tonegotiate on a best match for the user requirements. Bundledservice agents will negotiate with single service agents in aBundled Service Provider- Single Service Provider environ-ment, to negotiate on a best match for the bundled serviceagent’s requirements. The network and cloud service providerdata mining will supply data to the bundled service providersto manage risk of failure. The cloud and network are modelledas a collection of single service resources.

C. Bundled Service Provider

The Bundled Service Providers (BSP) shown in Figure 2,are responsible for negotiating the use of the single serviceproviders, and managing the failure of single services in thebundle. In general the BSP wishes to book reliable singleservices from the market, arrange their dependencies andmonitor their progress. Depending on the user’s requirementsin the service definition, individual modules to control the

Fig. 3. Creation Sequence Diagram.

provisioning of each single service are added to the bundledservice agent’s portfolio. The SIMS module is used for judgingrisk, and long term monitoring of performance.

Fig. 4. Management of Failure Sequence Diagram.Figure 3 shows the bundled service provider sequence

diagram for acceptance of a new contract. The bundled serviceprovider creates a bundled service agent to manage the bundledservice. The bundled service agent then assigns single servicemodules to each service. These single service modules thennegotiate on the market with the single service providers toobtain a quote for the service. The BSA’s buying strategyis determined by the customers expectation, the serviceshistorical performance, the services future requests availabilityand the services reliability and cost.The operational data isprovided by the SIMS(see below). The BSA then ranks therest of the available SSA’s for the each of the BSA’s singleservice controls for re-negotiation in case of failure. The BSAinteracts with a SSA through a series of status messages, withthese status messages representing the monitoring informationof the SSA on the service resources performance. Theseservices messages have been classified to four states, green,yellow orange and red, indicating whether the service is fine,degrading, in danger of failing, or failing. The four states of

Fig. 5. Simplified Management BBN with no evidence.

Run, Reserve, Offload and Failover represents the control flowof the BSA for interaction with the single service providers fora single service, which encapsulate the BSA’s desire Run theservice, Reserve resources on a secondary provider, offload onhigh intermittent failure and Failover to the new provider onfailure.

Figure 4 is a sequence diagram of failure and followingis a description of the interactions on failure. In order tomanage failure from the bundled services point of view, theBSA utilizes both renegotiation of, and, the lowering of qualityrequirements for the service. Renegotiation swaps providersupon the receipt of performance information from the singleservice providers. If the services’ performance level is too low,or that the information received from the agent indicates thatthe service has failed, then the bundled agent firsts instructsthe SSP to drop the required level of service temporarily (tomaintain some service functionality), renegotiates and reservesthe service with another provider for the remainder of theservices’ time, starts offloading the service to the secondaryprovider and finally fails over to the secondary provider. Forthe required quality level of the service to be maintained, theBSA again utilises the SIMS shown below to determine theprobable risk of engaging a particular single service providerfor service recovery. Then the BSA negotiates a new contractwith the chosen SSA for the remaining time period.

D. Single Service Provider Agents

Each SSA in the model has access to one Service Re-source, and, is responsible for provisioning and schedulingthe resource across the Service Resource. The SSA sets theprice of access which is determined by the resource costand the providers total utilisation. Currently the SSA agentsimplements a congestion pricing strategy which increases thecost of the service during periods of high Resource utilisation.This strategy of congestion pricing encourages bundled serviceagents to utilise other service providers during periods ofhigh demand, ensuring that the risk to the performance ofthe already provisioned services is minimised.

The single service agent interacts with the bundled serviceagent via status messages which inform the BSA of the

service resource status. The information gathered for the statusmessages is related to the current utilisation of the singleresource as well as the delay which occurs on the usagemessages.

We have modelled single service agents to simulate failureand send internal monitoring messages on the current servicesperformance. This allows the model to simulate the basicfunctionality of re-provisioning failed services with anotherprovider for the remainder of the contracted time.

E. Single Service Resource(s)

All resources are modelled as abstract resources, dividedinto four components of differing QoS. The componentsare labelled, as High, Medium, Low, and Best Effort andhave appropriate queues sizes for network connectivity andvirtual machine/application processing, with delays relating totheir individual priorities. A resource pool for resource unitsrepresents the percentage of utilisation in the cloud or net-work infrastructure. Greater utilisation of the Cloud/Networkresource results in poorer overall performance affecting lowerquality services first.

F. Service Information Management System

At the highest level of abstraction, any management decisionfacing a bundled service agent (BSA), such as choosing thecourse of action in case of a noticeable drop in performance,or, providing a quote for a specific bundle request from theclient, must take into account a multitude of variables drawnfrom a number of distinct information sources. The non-deterministic nature of the management problem facing theBSA imposes some requirements on the method of adminis-tration. Firstly, the agent must be able to identify the types ofsituations that may lead to catastrophic failures and possessknowledge of contingencies aimed at dealing with emergingcrises. Secondly, to identify the situations before they emerge,the agent must be able to collect network data relevant to theservices involved. Thirdly, the agent must be able to trade-offquality metrics such as availability and performance againstthe cost metrics so as not to over-optimise the bundle withrespect to customer’s technical requirement, since this course

of action may hurt its overall aim of maximising potentialbenefits.

Bayesian Belief Networks [15] are the core of the SIMS.Figure 5 depicts the abstract, starting network (for all agents)which takes into account the BSA’s position with respect to itscurrent and future customers as well as its currently contractedservices and attempts to convert a set of variables carriedby this context into one of the three decisions: do nothing,contract a new service and obtain a new customer. Specifically,the nodes can be described as falling into one of 3 groups:

• Variable - these nodes are representative of variation inthe BSA’a operational environment.

1) Customer Expectation: estimates of performancebased on customer information.

2) Historical Service Performance: aggregation of datafrom service’s performance characteristics.

3) Future Bundle Requests: BSA’s existing commit-ments.

• Analytical: nodes of this type represent BSA’s estimationof possible risk exposure.

1) Risk Of Spot Contract Failure: shows exposure topossible loss of income from a single contract whichaffects no other customers.

2) Risk Of Multiple Contract Failure: shows exposureto loss revenue from a suite of contracts which maypotentially result in a significant loss.

• Actionable: provide the BSA with estimation of necessityfor certain changes in the portfolio.

1) BSA Service Action: actions that affect one servicewithin the bundle.

2) BSA Bundle Action: action that affects the composi-tion of the bundle, in this case the BSA may decideto expand the composition of available resources.

3) BSA Market Action: represents BSA’s behaviour inthe market.

IV. MODEL DESCRIPTION

The bundled service agent simulation is required to model:the processes of quoting, risk evaluation, single service nego-tiation (including re-negotiation on failure), and service com-pletion and failure. Individually, the modelling requirementsare:

• Process modelling.• Constraints on state transition.• Goal based modelling, with explicit failure and abort

states.A state based model using Statecharts [16], was created to

simulate the negotiation and provisioning of bundled servicesin an open market. The model follows the architecture de-scribed in Section III-B and contains Bundled Service Provideragents (BSA), Single Service Provider agents (SSA), ServiceResource agents (SR) and Users across a market environment.The model is a high level view of the system, focussing onthe agents influence on service provisioning, and management,specifically dealing with resilience and recovery.

Fig. 6. Graphical representation of the case study.

V. CASE STUDY

An industrial case study is used to examine the effectivenessof the total control afforded by a bundled service manager. Wehave based the case study on a on-line travel company withwhich we have worked. The aim of the case study is to improveperformance, by increasing quality of service, and, applicationperformance, via a responsible, bundled service approach.

The on-line travel company’s website contains informationand bookings on tours, accommodation, attractions and travel,alone or in any combination. Overall the sheer size of thecontent available in this travel agency makes accessing thedata from a central location result in poor performance withhigh latency and poor usability. In order to improve on theperformance of the site the company wishes to split the func-tional logic of the site from the advertising and content, placingthe content with different geographical cloud providers, andleaving the site logic in one central location. A graphicaldescription is shown in Figure 6.

In order for this type of setup to be effective in the past,websites used static geographical locations, paying content de-livery companies a flat fee subscription for use of their contentdata centres. The amount of content served and the bandwidthavailable to a particular website is statically pre-arranged andgenerally unchangeable without significant undesirable leadtime. Further the connectivity between the central functionalwebsite and the content servers is almost always best effort,resulting in little or no control over the different applicationsections interaction.

With the use of the bundled service provider agent however,the three services of website logic, content server and thenetwork connectivity could be dynamically served in periodsof high usage in particular regions, with content servers, rep-resented by remote virtual machines, placed in different cloudenvironments based entirely on their particular geographicallocations. As the bundle includes QoS guaranteed tunnelsbetween the various content servers and the central location,latency caused by poor connectivity or high network utilisationwould be avoided. Further as the content servers are only setup in locations as they are needed, there would be certainlycost benefits over the traditional flat subscription rates.

Further as performance information about the serviceproviders is stored by the bundled service agents SIMS,

TABLE IUSER TABLE FOR BEST FIT SCENARIO

Name Type Start Time End Time From To Quality Resilience Min Qual PossibleA NC 9:30 10:30 Australia France High High Low CompulsoryB RM 9:30 10:30 Australia France High High Low CompulsoryC NC 10:00 11:00 Australia USA High High Low CompulsoryD RM 10:00 11:00 Australia USA High High Low CompulsoryE NC 12:00 1:00 Australia France High High Low CompulsoryF RM 12:00 1:00 Australia France High High Low CompulsoryG NC 1:00 1:10 Australia UK Low Low Low Compulsory

Fig. 7. Best Fit Consumption Results, Utilisation of Service Resource over Time

the bundled service provider can take responsibility for theperformance of the application, through initially simple meth-ods of choosing the most reliable providers, and reservingand offloading on secondary providers in the face of poorperformance.

VI. SIMULATION

We have used simulation to explore the complexity ofdealing with interacting agents and resources, and to evaluatethe required performance criteria of resilience and scalability.To test the performance of the architecture the model was runthrough a series of simulations. The simulation creates a seriesof contracts that the bundled service providers attempts tofulfil. All contracts consist of 5 randomly generated services,the combination of which is taken from the outline of thecase study. It uses varying requested cloud service types suchas virtual machines, applications including the guaranteednetwork connectivity between them. It also varies levels ofquality, and runs for a randomly generated time. The resultsof the simulation was then examined to see whether:

• The bundled service agent takes appropriate steps in thecase of poor service performance, up to and includingrecovery from failure.

• The bundled service agent performs best fit consumption.• The bundled service providers and agents, effectively

utilise the network resources assigned to them and arerelatively scalable.

• And that the Bundled service providers are cost effective,in that they maintain their cost margin across multipleresources.

A. Best fit Consumption

To test the best fit consumption, the simulation of randomlygenerated contracts was limited to one contract and the fol-lowing scenario to test the systems response. The case studywas the basis for the scenario which describes the need fora content server based in a virtual machine to be run in a

cloud environment based in the geographic location of Europe(France) accompanied by a dedicated network back to themain site in Australia for one hour. Half an hour later thisis also needed in the USA. This geographical need is thenrepeated for France one hour later. The log of the unusualusages are then sent to a tracking company with headquartersin UK, for historical and trending analysis. This trace is theuser specification for this request, is displayed in Table ??.

Figure 7 shows the result of this simulation, The graphsshow the individual percentage of utilisation of the BSA’s onfour of the five single service resources (with the fifth beingblank). The results shows that The BSA’s utilise four SSP’sto provide services rather than two, avoiding congestion onthe individual services and reducing the risk to providing thebundles.

B. Effective Utilisation

The case study scenario was randomised as described in thesimulation introduction and run continuously with multiplebundled service providers, against a five network resourceswith a high generation of simultaneous contracts (over twenty).The purpose of this simulation is to test evaluate the bottle-necks on dynamic service provisioning.

Figure 8 shows the utilization of the cloud and connectivityresources frequently goes above 80 percent, which has been setas a value at which poor performance in the network resourcesis likely to cause failures in the bundled service contracts.Running the simulation with the same high number of BundledService Providers but low simultaneous contract generation,resulted in easily manageable resource utilization.

C. Recovery from Failure

To validate the recovery on failure behaviour, the behaviourof one of the bundled service agents single service controlwas examined in detail. The service run in this simulation issimple network connectivity from Australia to France, run for1 hour and 10 minutes at a requested low quality rate.

Fig. 8. Percentage of Resource Utilisation over Time

The bundled service quote is specified as $942 and isthe result of the bundled service agent determining the bestprovider for the service based on the quality requirementadded to a static profit margin for the service (currently 10%).The actual cost of the service is the result of the combinedcosts of all service providers that were engaged, includingthose engaged on the failure of the primarily chosen providerresulting in a price of $863. This price shows that the failure ofthe primary service(s) increased the overall cost to the bundledservice provider, reducing the profit for that service slightly.

The percentage of messages that each single serviceprovider sent to the bundled service agent was recorded,before the probability of failure, as described by the internalmonitoring message sent by the SIMS, was deemed too high tocontinue. The SIM system currently determines the probabilityof failure from the load on the network resources as well as thenumber of messages in the network resources queue. With areporting message sent every minute the number of messagesrecorded for provider 1 was 60, and for provider 2 was 20.This indicates that the bundled service agent started to offload(duplicate) the service with the reserved single service providertowards the end of the service and failed over to the newservice for the last ten minutes.

D. Cost Effectiveness

The following scenario demonstrates the cost effectivenessof the bundled service providers. It describes a simulationbased on the case study with five agents accepting contractsover a simulated period of 20 hours, setting quotations forthe contracts and calculating the cost to the BSP. The BSP’squotes have a standard 10% addition to cover the cost of re-provisioning if necessary.

Figure 9 shows the results of the simulation for the fiveBSPs. BSP’s 0,1,2,3 show that the (notional) 10% margin hasbeen maintained, indicating a steady profit for those providers,whereas BSP 4’s budget and cost are level, indicating thatproblems with the service has reduced the profit margin for

its contracts, however not to the point of losing money.

VII. DISCUSSION

Advantages for the user: The design of the bundled serviceagent returns the ownership and control of the services to theuser. The bundled service agent accepts the responsibilities forthe service in a combined environment of telecommunicationsand hybrid public/private clouds, where it is often difficult forthe purported user of the service to have any control over theperformance of that service.Advantages for independent implementation: The use ofa market place for interaction between the bundled serviceagents and single service agents, allows heterogeneous imple-mentation by the single service providers with uniform accessfor the bundled service providers.Issues in negotiation The market paradigm requires rapidrenegotiation of service level agreements on poor performance.However early results of the SIMS project indicates that thereis generally some early warning of failure, enough for the timerequired by electronic negotiation.Issues in performance information The bundled serviceagents require performance information from the serviceprovider. This information would have to be detailed enoughto be useful, yet not infringe on any of the single servicessecurity. If the bundled service providers do not receiveinformation from the single service providers they should beable to build their own performance inference network in theSIMS subsystem.Issues in scheduling and service capacity: For quotationand negotiation of future services the single service provider’smanagement system also needs to contain some ability toforecast the systems utilisation and capacity for schedulingthe provisioned services.

VIII. CONCLUSION

This paper has presented the aims, architecture, conceptsand ideas behind providing a bundled service agent for use

Fig. 9. BudgetCost vs Actual Cost over Time

on in open market. The foundation of the motivation for thebundled service agent is the wish to explore the requirementsfor cost effective and efficient total service management,through increasing the utilisation of the individual serviceproviders infrastructure. Further it is argued in this paperthat dynamically created services are insufficient without thecontrol provided by a bundled service agent that is prepared toaccept the responsibility for the dynamically created services,performing the coordination of any composite service andguaranteeing their delivery.

Towards this goal the paper has presented and validated: adescriptive architecture with the ability to utilise componentsof mining and negotiation, and, a method and approach tojudge the risk of providing the bundled service.

This work is seen as an important first step in matchingthe customer’s requirements, of resilience and economy, ondynamic cloud based services.

Future work will focus on other aspects of the architectureincluding dealing with scalability issues resulting from thelimitation of resources.

In accepting the responsibility for the service, and providinga guarantee that the services contracted will be delivered andmanaged from end to end, the bundled service provider addsresilience (for the customer) to the services which has beenmissing from current contracted services. Further, the use of anopen market place for the negotiation of these services allowsa degree of flexibility in service choice which is lacking intoday’s industry.

IX. ACKNOWLEDGEMENTS

This work was partly funded by Alcatel-Lucent Bell Labsand a research grant from the Australian Research Council,The authors would like to thank the respective bodies, whichhave provided a scholarship for the first author and supportfor the project. The authors would also like to thank theUTS Centre for Quantum Computation and Intelligent Systems(QCIS) for their support.

REFERENCES

[1] B. Rochwerger, D. Breitgand, E. Levy, A. Galis, K. Nagin, I. M.Llorente, R. Montero, Y. Wolfsthal, E. Elmroth, J. Caceres, M. Ben-Yehuda, W. Emmerich, and F. Galan, “The reservoir model and archi-tecture for open federated cloud computing,” IBM Journal of Researchand Development, vol. 53, no. 4, pp. 4:1 –4:11, july 2009.

[2] H. Mearns, J. Leaney, A. Parakhine, J. Debenham, and D. Verchere, “Anautonomic open marketplace for service management and resilience,” inNetwork and Service Management (CNSM), 7th international conferenceon, 2011.

[3] R. Buyya, R. Ranjan, and R. Calheiros, “Intercloud: Utility-orientedfederation of cloud computing environments for scaling of applicationservices,” in Algorithms and Architectures for Parallel Processing, ser.Lecture Notes in Computer Science, C.-H. Hsu, L. Yang, J. Park, andS.-S. Yeo, Eds. Springer Berlin / Heidelberg, vol. 6081, pp. 13–31.

[4] “A survey of network virtualization,” Computer Networks, vol. 54, no. 5,pp. 862 – 876, 2010.

[5] F. Hao, T. V. Lakshman, S. Mukherjee, and H. Song, “Enhancingdynamic cloud-based services using network virtualization,” in Proceed-ings of the 1st ACM workshop on Virtualized infrastructure systems andarchitectures, ser. VISA ’09, 2009.

[6] R. Buyya, S. Pandey, and C. Vecchiola, “Cloudbus toolkit for market-oriented cloud computing,” in Cloud Computing, ser. Lecture Notes inComputer Science, M. Jaatun, G. Zhao, and C. Rong, Eds. SpringerBerlin / Heidelberg, vol. 5931, pp. 24–44.

[7] V. Stantchev and C. Schropfer, “Negotiating and enforcing qos andslas in grid and cloud computing,” in Advances in Grid and PervasiveComputing, ser. Lecture Notes in Computer Science, N. Abdennadherand D. Petcu, Eds. Springer Berlin / Heidelberg, vol. 5529, pp. 25–35.

[8] J. Nimis, A. Anandasivam, N. Borissov, G. Smith, D. Neumann,N. Wirstrom, E. Rosenberg, and M. Villa, “Sorma - business cases forand open grid market: Concept and implementation,” in Grid Economicsand Business Models, ser. Lecture Notes in Computer Science. SpringerBerlin / Heidelberg, 2008, vol. 5206, pp. 173–184.

[9] A. Rubio-Montero, E. Huedo, R. Montero, and I. Llorente, “Manage-ment of virtual machines on globus grids using gridway,” Parallel andDistributed Processing Symposium, International, vol. 0, p. 358, 2007.

[10] S. Garg, S. Venugopal, and R. Buyya, “A meta-scheduler with auctionbased resource allocation for global grids,” in Parallel and DistributedSystems, 2008. ICPADS ’08. 14th IEEE International Conference on,dec 2008, pp. 187–194.

[11] L. Green, “Service level negotiation in a heterogeneous telecommuni-cations environment,” in Proceeding International Conference on Com-puting, Communications and Control Technologies (CCCT04), Austin,TX, USA, August 2004.

[12] M. Wooldridge, An Introduction to MultiAgent Systems, 2nd ed. WileyPublishing, 2009.

[13] J. Ejarque, R. Sirvent, and R. Badia, “A multi-agent approach forsemantic resource allocation,” in Cloud Computing Technology andScience (CloudCom), 2010 IEEE Second International Conference on,30 2010-dec. 3 2010, pp. 335 –342.

[14] T. Forum, “Bundled service definition,”http://www.tmforum.org/BundledServices/7221/home.html, retrieved on11-04-2011.

[15] A. Bashar, G. Parr, S. I. McClean, B. W. Scotney, and D. Nauck,“Knowledge discovery using bayesian network framework for intelli-gent telecommunication network management,” in Knowledge Science,Engineering and Management, 2010, pp. 518–529.

[16] D. Harel, “Statecharts: A visual formalism for complex systems,”Science of computer programming, vol. 8, no. 3, pp. 231–274, 1987.