The AutoI approach for the orchestration of autonomic networks

Ann. Telecommun. (2011) 66:243–255DOI 10.1007/s12243-010-0187-x

The AutoI approach for the orchestrationof autonomic networks

Daniel Fernandes Macedo · Zeinab Movahedi · Javier Rubio-Loyola ·Antonio Astorga · Giannis Koumoutsos · Guy Pujolle

Received: 1 February 2010 / Accepted: 28 May 2010 / Published online: 17 June 2010© Institut Télécom and Springer-Verlag 2010

Abstract Existing services require assurable end-to-end quality of service, security and reliabilityconstraints. Therefore, the networks involved in thetransport of the data must cooperate to satisfy thoseconstraints. In a next generation Internet, each of thosenetworks may be managed by different entities. Fur-thermore, their policies and service level agreements(SLAs) will differ, as well as the autonomic manage-ment systems controlling them. In this context, we inthe Autonomic Internet (AutoI) consortium proposethe Orchestration Plane (OP), which promotes theinteraction among different Autonomic ManagementSystems (AMSs). The OP mediates the communicationand negotiation among AMSs, ensuring that their SLAs

D. F. Macedo (B) · Z. Movahedi · G. PujolleLaboratoire d’Informatique Paris VI-Paris Universitas,104 Avenue du Président Kennedy, 75016, Paris, Francee-mail: [email protected], [email protected]

Z. Movahedie-mail: [email protected]

G. Pujollee-mail: [email protected]

J. Rubio-LoyolaCINVESTAV Tamaulipas,Victoria, Mexicoe-mail: [email protected]

A. AstorgaUniversitat Politècnica de Catalunya,Barcelona, Spaine-mail: [email protected]

G. KoumoutsosUniversity of Patras,Patras, Greecee-mail: [email protected]

and policies meet the requirements needed for theprovisioning of the services. It also simplifies the fed-eration of domains and the distribution of new servicesin virtualised network environments.

Keywords Network management · Autonomicnetworking · Management orchestration ·Next-generation internet

1 Introduction

Autonomic management systems, as initially describedin the IBM manifesto [9], have been defined for a singlecomputing entity. In networking, devices are ownedand managed by different operators, and as a conse-quence several different management systems will runat the same time. Due to the existence of several man-agement standards, protocols and vendors, managinga network is much more complex than managing asingle system. Thus, it is not practical to devise a singleautonomic control loop (ACL) that autonomically per-forms all the network management functions (definedin the TMN standards as the FCAPS acronym—fault,configuration, accounting, performance and security).Thus, we need to define one or more ACLs for oneor more management functions in order to simplify thedesign of each control loop.

Furthermore, the operation of the network man-agement system will depend on the interaction of allthose control loops, which must ensure, among otherkey aspects, that the network operates within normalparameters set by the business goals of the operators.Also, the decisions of one control loop may sometimesgo against the objectives of another. As an example, an

244 Ann. Telecommun. (2011) 66:243–255

autonomic security component may use a heavier en-cryption scheme to improve the security of the network,however this encryption scheme may require too muchprocessing and bandwidth, reducing the throughput toa level below the performance dictated on the servicelevel agreement (SLA) of a self-configuration controlloop.

In order to provide end-to-end quality of service(QoS) in the Internet, several sub-networks havingdifferent managers (or belonging to different adminis-trative domains) must cooperate. This requires that theprotocols as well as the configuration of the domains(i.e. security policies, QoS and SLAs) are compatible.If they are not, a re-negotiation and re-configurationprocess is required.

In the Autonomic Internet (AutoI) project [1, 5], weintroduced a new management component to deal withthe cooperation of autonomic management systems.This distributed component, which in the AutoI archi-tecture is represented by the Orchestration Plane (OP),enables the cooperation of a number of ACLs, ensuringthat their decisions are aligned with the requirements ofthe end-users. This guided cooperation, or orchestra-tion, guarantees that the overall optimisation goals ofeach autonomic component and control protocols arealigned with the goals and SLAs defined for the entirenetwork. Orchestration also allows administrative man-agement domains run by different operators or admin-istrators to automatically adjust their configuration toaccommodate the federation of networks.

The OP deals with the meta-management of Au-tonomic Management Systems (AMSs), that is, thedeployment and re-configuration of autonomic man-agement control loops in order to allow their interop-eration. This is achieved based on a set of high-levelgoals, defined for each of the managed network do-mains that form the orchestrated network. The OPensures the interoperation of management systems,even though those systems use different set of high-level goals and management standards. This processmay be accomplished through the negotiation of newSLAs and policies, the deactivation of conflicting man-agement systems followed by the activation of othermanagement systems, or the migration of such systemsor parts of them within the orchestrated network. Theentire orchestration process is governed by orchestra-tion policies, which dictate what are the compromisesthat each of the managed domains are willing to makefor the sake of interoperability.

The objective of this article is to describe the overallarchitecture of the OP, its functions and associatedpolicies. Readers willing to probe further on the im-

plementation details are encouraged to read the ar-ticles referenced throughout the text. This article isorganised as follows. Section 2 presents the relatedwork. Section 3 presents the AutoI project architecture.Section 4 describes the orchestration plane concept andits architecture. Section 5 presents the orchestrationpolicies. Section 6 presents a use case that highlightsthe benefits of Orchestration in the AutoI architecture.Section 7 concludes the article.

2 Related work

Due to the importance of inter-domain network co-operation on network management, several works andstandards have already been proposed. Among them,the COPS standard is the most important one, dealingwith the exchange of network policies [3]. COPS is acentralised architecture consisting of a policy decisionpoint (PDP) and a set of policy enforcement points(PEP). This enables centralised QoS policy injectionand management on the PDP (on updates pushed tothe agents). Also, it provides dynamic load-dependentadjustments and traffic control based on the data fromthe PEPs (requests to the server). COPS-SLS is anextension of the COPS protocol to negotiate SLSseither between customers and networks or betweennetworks [14]. Though COPS-SLS enables the inter-operation of heterogeneous networks, its heavyweightnature and its reliance on a central entity reduce itsapplicability on autonomic networks.

Recently, with the advent of autonomic networking,the automatic cooperation of networks gained moreimportance. Studwell discusses the need of orchestra-tion in autonomic networking, focusing on the role ofstandards for the orchestration of self-managing sys-tems [19]. He proposes the coordination of standardsfor the advancement of self-management standards.

Clark proposed a new concept, called the knowledgeplane, which gathers information from the networkand uses it to autonomously re-configure its nodes [4].Clark recommends using techniques derived from ar-tificial intelligence and cognitive sciences to supportthe uncertainties and the complexity of building sucha plane upon the current networks. Compared with ourproposition of the orchestration plane, the knowledgeplane of Clark could more likely be seen as a junctionof the management plane, an information base and asort of elementary orchestration plane capable to act onimperfect and conflicting situations. However, Clark’splane does not handle the orchestration of heteroge-neous management domains and control loops.

Ann. Telecommun. (2011) 66:243–255 245

The Inference Plane proposed in the FOCALE au-tonomic architecture aims to provide autonomicity andto enable the mediation and coordination across het-erogeneous domains [18]. The inference plane definesan universal lexicon that uses a model-based translationfunction to translate this lexicon into different vendor-specific languages and programming models. The lex-icon is a combination of information, data models andontologies. While in the FOCALE architecture, a singleplane performs management and orchestration tasks,we separate these two aspects in order to reduce com-plexity. Similar to the FOCALE architecture, we alsorecommend techniques for mapping between differentdata models and ontology translation to allow differentmanagement systems to communicate and cooperate.

4D is a new architectural model for the Internet,where tasks are divided into four planes: decision, dis-semination, discovery and data [10]. In 4D, the dataplane acts based on the configurations received by thedecision plane. Decisions are based on the informationfetched by the Discovery plane, which constructs a viewof the network resources. Next, the decisions are sentto the Data plane using the Dissemination plane. Themain advantage of such architecture is the centralisa-tion of decisions into one single plane, removing theproblems of multiple layers dealing with similar issues.While our orchestration plane enables the negotiationand federation of different management domains, 4Ddoes not deal with the fact that multiple network man-agement entities may exist, once each domain will beoperated by a different organization.

3 The AutoI architecture

The Autonomic Internet project [1, 5] is a STREPproject financed by the European Union. Its objectiveis to propose a new management architecture for theautonomic management of virtual services and virtualnetwork elements (VNEs). In such a network, theVNEs and services can be created, destroyed, deployedand migrated autonomically.

The functionality proposed by the AUTOI andspecifically performed within the orchestration com-ponents is something totally new, as it introduces andautomates the machine-based communication betweenAutonomous entities that manage the future inter-net. This functionality is nowadays performed networkmanagers and administrators, with no machine-basedinference to aid the process. In order to achieve this vi-sion, the AutoI project employs five functional planes,the OSKMV (orchestration, services, knowledge, man-

Resource Virtualization

Orchestration Plane

Knowledge Plane

Management Plane

Internet

Programmable Networks

Information Models& Ontologies

PolicyModels

Knowledge

Knowledge

Knowledge

ControlAlgorithms

Knowledge Management

Self-Management

IP NetworksWireless Networks

Other Networks

Sensor Networks3GPP Networks

ResourceManagement

Fig. 1 The AutoI architecture

agement and virtualisation) planes, as shown in Fig. 1.Those planes are described below.

– The Virtualisation Plane (VP) virtualises physicalresources, allowing the migration and on-the-fly re-configuration of network resources [2]. It abstractsall the virtualisation issues away from other compo-nents of the architecture.

– The Management Plane (MP) deals with the main-tenance and creation of individual control loops [1].Those loops are realised by Autonomic Manage-ment Systems (AMSs), which perform the MAPE(Monitor, Analyse, Plan and Execute) [9] func-tions. Each AMS represents an administrativeand/or organisational boundary, called AMS do-main, which manages a set of devices or sub-networks using both a common set of directives (i.epolicies and SLAs) among peer AMSs, and theirinternal directives. AMSs can also manage services.

– The Service Enablers Plane (SP) is responsible forservice discovery, deployment and composition [1].It employs virtual resources to set up new services,such as a VPN, a file sharing service or a multimediatransport service, among others.

– The Knowledge Plane (KP) implements a distrib-uted information service, providing all the planeswith their required information [12, 15]. It timelydisseminates information for the other planes, anddetermines the three ‘W’s of information manage-ment: Which information is needed, from Who andWhen. It also provides inference engines in orderto derive knowledge from the management infor-mation and context stored in the KP.

– The Orchestration Plane deals with the orchestra-tion of multiple domains as well as the interaction

246 Ann. Telecommun. (2011) 66:243–255

of different AMSs and services [13]. While theSP and the MP deal with the management of asingle service or control loop, respectively, theorchestration plane mediates and guides the in-teraction among several AMSs. Since the AMSsare autonomic entities, the orchestration planeacts solely as a mediator, detecting and manag-ing conflicts, determining SLAs that satisfy allinvolved parties and determining the need for (re-)deployment and re-location of AMSs and services.This information is modelled using a set of ontolo-gies, specified in a common information model [8].

Throughout this document the term high-level poli-cies or business objectives are particularly linked tohigh-level aspects of management and control over agiven system. This is, by no means, aligned to eco-nomical aspects such as pricing strategies. Policies inthe AutoI architecture follow a policy continuum [6].In the AutoI Policy Continuum, policies are refinedfrom high-level policies into lower level policies that,through a model-based translator, can be applied to anyinstance of equipment. In AutoI, the following levels ofpolicies are defined:

– Orchestration level policies control the negotiation,distribution and federation tasks of the orchestra-tion plane.

– System level policies are related to the AMSs. Theyare used to define the configuration and operationof the AMSs.

– Component level policies manage the virtual com-ponents defined in the AutoI architecture. They areapplied at the VNE level.

– Instance level policies are embedded in the physicaldevices, which use them to make their own localmanagement decisions.

The orchestration plane deals mostly with orchestra-tion level policies. Within orchestration policies, nego-tiation, federation, distribution and governance policiesare defined, as we will present in Section 5. The OPdeals with system level policies when it mediates anegotiation between AMSs, In this case, it ensures thatthe system level policies produced by the AMSs arealigned with the orchestration policies.

It is worth mentioning that both the Distributed Or-chestration Components (DOCs) and the AMSs workwith the help of High-level Policies, business objectives,or other means to drive their management and controlactivities. In both cases, AUTOI advocates for a policycontinuum [6] that translates high-level directives intosystem level policies that are ultimately enforced intheir respective entities as mentioned earlier. The pol-

icy continuum specialises a policy refinement process[17] with which high-level goals are iteratively refinedinto lower level goals, and that are eventually translatedinto system-level policies. Refinement patterns allowthe systematic derivation of goals into refined lower-level goals, and eventually the parameterization of sys-tem policies that would achieve the high-level policiesor business objectives. We advocate for an approach inwhich new refinement patterns can be updated by theDOC and AMS administrative parties in cases when nosystematic translation is possible. In these cases, newrefinement patterns can be provided at any level ofthe hierarchical composition of goal graphs. It is worthmentioning that a fully automatic refinement processwithout intervention of administrative parties is verydifficult if not impossible to achieve. In addition, asDOCs and AMSs exhibit self-governance properties,their refinement patterns and policies with which theymanage and control their resources, may be different.

4 The orchestration plane

The purpose of the Orchestration Plane is to governand integrate the Behaviours of the network in re-sponse to changing context and in accordance with ap-plicable high-level goals and policies, ensuring integrityof the Future Internet management operations. TheOrchestration Plane can be seen as a control frame-work into which any number of components can beplugged into or out in order to achieve the requiredfunctionality.

The OP would also supervise the optimisation andthe distribution of knowledge within the KPs of theinvolved administrative management domains to en-sure that the required knowledge is available in theproper place at the proper time. This implies that theOP may use either local knowledge for real time con-trol, as well as a more global knowledge to managelong-term processes, perform planning and inferences.The OP acts as control workflow for the AMSs of theorchestration domain, ensuring bootstrapping, initiali-sation, dynamic re-configuration, adaptation and con-textualisation, optimisation, organisation and closingdown of AMSs. The OP enables the following orches-tration functions, which will be described in details inSection 4.2: the federation of orchestration domains,the negotiation of policies, the distribution of man-agement tasks among AMSs, the monitoring of AMSBehaviour (governance tasks) and the management ofthe system view of all the AutoI components.

The OP is made up of one or more DOCs, which con-trols a single orchestration domain composed of several

Ann. Telecommun. (2011) 66:243–255 247

AMSs. DOCs facilitate the cooperation among AMSsof an orchestration domain. DOCs also cooperate witheach other in order to ensure end-to-end QoS. EachAMS controlled by the DOC represents a set of virtualentities, which manage a set of virtual devices, sub-networks, or networks using a common set of policiesand knowledge. The set of virtual resources managedby each AMS are non-overlapping. The architecture ofthe DOC is shown in Fig. 2.

DOCs use the knowledge plane to store and dis-seminate the information required for their operation.The information can be decoupled into two parts, orviews, according to their relevance to a given DOC.The Intra-System View concerns information requiredto orchestrate the services within the orchestrationdomain, while the Inter-System View deals with theorchestration of several orchestration domains. TheIntra-System View contains information that enablesDOCs to become aware of the particular situation thatthey are now in; the Inter-System View provides similarinformation for collaborating DOCs.

This section describes each of the components of theDOCs. We present the Dynamic Planner as well as adomain-specific language used for describing orchestra-tion tasks. Next, we present the main Behaviours thatmake up the DOC. Finally, we describe the mechanismsemployed in the AutoI architecture to ensure the relia-bility of the orchestration components.

4.1 The dynamic planner

The Dynamic Planner is the central entity of the DOCs.It is responsible for dispatching Behaviours, which willtake care of specific orchestration tasks. The DP isa generic execution engine of orchestration tasks (or

Dynamic Planner

Orchestration

AMS

AMS Behavior

AMS

AMS Behavior

AMS

AMS Behavior

AMS

AMS Behavior

Po

licie

s

Near real-time control

Behaviors


Behaviors


Behaviors

Core BehaviorsSystem

views

Systemviews

Systemviews

Fig. 2 The components of the DOC

meta-management of self-governing management sys-tems). Furthermore, the DP distributes policies andSLAs to the AMSs. This is performed at the deploy-ment of new AMSs, or whenever the orchestrationpolicies change.

To reduce resource consumption of the networkingcomponents, the DP is an event-based component. Itrelies on the monitoring facilities of the knowledgeplane, as well as those of the governance Behaviours, totrigger notifications of important event changes. Eventsmay be triggered from within the network (i.e. a conflicthas happened), or from the outside (i.e. the operatordefines a new set of goals for the OP). The followingtypes of events exist in the OP:

– Parameter changes: A change occurred in a set ofvariables, which have values within a certain range.One such event could be the delay of a link becom-ing too high, which may require the renegotiationof the SLAs or the redeployment of an AMS.

– Conflicts: A conflict within two AMSs or Behav-iours has been detected. Those conflicts will usuallycome from the governance Behaviours.

– Federation requests: The operator, or an AMS, hasrequested the federation of two or more domains.

– Separation of a federation request: The operator, oran AMS, has requested the separation of a domaininto two smaller domains.

– Distribution requests: The operator, or an AMS, hasrequested the deployment of a new AMS.

– Request to close down a component: The operator,or an AMS, has requested the closing down of anAMS.

– New goals and high-level policies: The operator hasredefined the goals, policies or user requirementsthat must be satisfied by the DOC.

These events trigger workflows, which are describedin a domain-specific language (DSL) [8]. The lan-guage uses an orchestration vocabulary to simplify theprogramming of workflows. In order to simplify thelanguage and allow for a fast implementation of the in-terpreter, the vocabulary of this DSL is quite small andthe DP relies on the implementation of specific Behav-iours to perform the bulk of the orchestration tasks. Forexample, if there is a need to solve a specific problem,the DP will identify that such a problem occurred bymonitoring certain variables within a set of pre-definedboundaries, and then it will trigger a Behaviour that issuitable for that situation. The language is composed ofthe following elements:

– Conditional execution blocks: if/then/else con-structs allows for actions being taken only if certain

248 Ann. Telecommun. (2011) 66:243–255

conditions are met. This allows, for example, tostartup different Behaviours based on the state ofthe monitored parameters of the NEs, or to definea default operation if an unknown condition occurs.

– Loops, counters, timers and timeouts: used to cre-ate iterations and timed steps in the orchestrationprocesses. Timeouts can be used to cancel tasks ifthey do not complete within a certain allowed time,improving the stability of the architecture.

– Parallelisation block: allow the orchestration planeto start up several Behaviours at the same time,allowing orchestration tasks to run concurrently.

– Startup and closedown of Behaviours: Workflowsare allowed to startup and closedown as many Be-haviours as necessary. Coupled with the conditionalconstructs, workflows may use different Behavioursbased on the condition of the VNEs.

– Events: federation, distribution, conflicts and para-meters changes may trigger events. These eventscan be associated to specific workflows, allowingthe DP to respond differently to each event.

– Workflow-related: workflows are allowed to startupother workflows, based on a set of conditions. Thisfeature allows workflows to be smaller and simplerto understand, since long workflows can be brokendown into smaller ones. Each workflow has anassociated name, which is used to identify them.

Figure 3 illustrates a workflow for the federationof IPv4-IPv6 networks. This operation will require thenegotiation of certain parameters, as well as the deploy-ment of a tunnel. To allow this process to completewithin acceptable time limits, the workflow definesthat the two networks being federated must reach an

wflow FederateHeterogeneous;Federate:

If N1.IPversion == 4 && N2.IPversion == 6 {timeout 300 {

trials = 0;do {

startup IPNegotiation;call IPNegotiation.triggerNegotiation();trials = trials + 1;

} until trials == 10;if trials == 10 {

loadwflow FederationHeterogeneousFailed; }

else { startup IPv6Tunnel (N1, N2);

}closedown IPNegotiation;trigger onchange N1.IPversion > 4, FederateHomogeneous;

} otherwise { loadwflow FederationHeterogeneousFailed;

} else {

loadwflow FederateHomogeneous; }

Fig. 3 An example orchestration workflow

agreement after ten negotiation rounds or five minutes.Further, if the IPv4 network is upgraded to IPv6, theFederateHomogeneous workflow is triggered in orderto re-configure the network.

4.2 Behaviours

Behaviours describe a specific orchestration task to beperformed by the DOC, such as the negotiation of high-level policies, the distribution of tasks, the creation ordestruction of services and virtual routers and so on.

The life-cycle and operation of Behaviours are con-trolled by the DP. Behaviours may interact with eachother when necessary. For example, a federation Be-haviour may interact with a QoS Behaviour if thedesired QoS could not be met when two networks arejoined. We distinguish two types of Behaviours:

Knowledge-Related Behaviours They supervise thecollection and dissemination of orchestration-relatedknowledge needed for Intra and Inter-domain Behav-iours. They define the information to be collected, itsperiodicity and from where it must be retrieved. ThoseBehaviours are specific to each service, and the wholeset of these Behaviours supervises the storage of infor-mation in the Knowledge Plane.

Core Behaviours They deal with the integration oftwo or more orchestration domains, as well as thesmooth operation of the resulting orchestration do-main. This is performed by four key Behaviours, whichare described below.

4.2.1 Distribution behaviour

The distribution Behaviour provides communicationand control services that enable management tasks tobe split into parts that run on multiple AMSs withinan Orchestration Plane. The distribution Behaviour,thus, controls the deployment of AMSs and theircomponents.

This process begins with one AMS or the operatorindicating that a new service or AMS must be deployed.The distribution Behaviour may also be triggered whenAMS’s policies change following a negotiation process.The DP then starts up the distribution Behaviour, indi-cating the high level goals that must be respected duringthis specific deployment. The Behaviour, in turn, startsa new deployment iteration, deriving the set of policiesthat will be used as well as the AMSs that should be de-ployed. Once the policies and the AMSs to be deployedare defined, the distribution Behaviour triggers the SP,which deploys the components. The SP, in turn, willindicate to the Behaviour if it is possible to deploy an

Ann. Telecommun. (2011) 66:243–255 249

AMS that fulfils the QoS restrictions demanded by theDOC. If the service cannot be deployed, the Behavioursignals the DP that the deployment is not possible. Asa consequence, the DP triggers a re-negotiation of thedistribution parameters. After the parameters are re-negotiated, the distribution retries the deployment bynotifying to the SP the new parameters of the compo-nents that must be deployed.

After the SP performs the deployment, the AMSsnotify the DOC if the policies and SLAs that have beenprovided are acceptable. If the AMSs reject them, theDP should decide if the distribution must be cancelled,or if a new attempt must be performed. If a new distrib-ution round is needed, the DP deletes the componentsdeployed in the previous iteration and then retries thenew deployment. This time, a new set of policies andrequirements, defined either by a negotiation processor explicitly after an operator’s request, is used in theprocess.

4.2.2 Negotiation behaviour

The negotiation Behaviour enables the orchestrationdomains and AMSs to negotiate their business objec-tives in order to define a common set of goals that canbe maintained across the federation. Since the DOCshave the advantage of a more holistic view of thenetwork, DOCs mediate the negotiation process, actingas trusted third parties or service brokers. Negotiationsare triggered for two reasons: First, following conflictsproduced during the federation process. Second, whenAMSs change their policies or SLAs autonomously,leading to an incompatibility with the orchestrationpolicies and SLAs. We have proposed two differentnegotiation approaches in the AutoI project: one basedon coalition formation [16] and another based on bar-gaining [7].

Figure 4 shows an example of the operation ofthe negotiation Behaviour. The Behaviour operates in

Fig. 4 FSM representation of the negotiation behaviour

iterations. In each iteration, the Behaviour contactsthe negotiating entities, sending them the system levelpolicies and SLAs that have been proposed by theother conflicting entities (setOtherPolicies()). Thoseentities must propose a new set of policies that they arewilling to deploy (checkNegotiationResult()). Duringthis process, those entities may communicate amongthem if needed. Once they reach a decision, the newsystem level policies are forwarded to the negotiationBehaviour (AnalyseProposedPolicies()), to check if theagreed policies are aligned with the overall orchestra-tion policies. If not, the negotiation Behaviour rejectsthe policies and starts another negotiation round. If oneiteration fails, each entity will used the experience ofthe past iteration to propose a different set of policies.the next try will be done using the experience of pastiteration. For example, the DOC can modify its set ofpolicies, making them less restrictive, in order to avoidthe problem raised previously.

To avoid an endless negotiation, the negotiationBehaviour detects if one or more entities are not willingto compromise. For example, if one AMS alwaysreturns the same set of system level policies, this mayindicate that it is not willing to change its configurationfor the sake of the others. The negotiation may also belimited in the number of possible iterations, the amountof time spent on iterations, or the total amount of timespent negotiating. If the process fails, the negotiationBehaviour communicates this to the DP (Negotiation-Failed()). The DP may propose a set of measures tosolve the conflict: (1) shut down some of the compo-nents involved in the negotiation and start another ver-sion of those components; (2) change the orchestrationpolicies or (3) declare failure, leaving the configurationin its last stable state and notifying the operator.

4.2.3 Federation behaviour

The federation Behaviour supervises the entire federa-tion process of a set of orchestrated domains willing tobe combined into a larger domain guided by commonhigh level goals, while maintaining local autonomy. Afederation may be triggered by a request of an operatoror when a new service is created, requiring two domainsto come together.

Since each domain may have different SLAs andpolicies, a federation attempt may trigger a negotiationof the orchestration and system level policies. This maylead even to the re-deployment of services in the casethat the new set of policies are not compatible withsome of the deployed services. A distribution Behav-iour may also be triggered to assist deployment ofAMSs and services considering new high-level policies.

250 Ann. Telecommun. (2011) 66:243–255

Figure 5 shows the operation of the FederationBehaviour and its interaction with other Behaviours.When the DP triggers the federation Behaviour on thedomain A, it sends a Federation Request message tothe DP of domain B. This request contains the poli-cies over which domain A desires to federate with thedomain B. This set contains negotiable as well as non-negotiable policies. When the DP of domain B receivesthe Federation Request, the negotiation Behaviour islaunched in order to negotiate the federation policies.If the negotiation process succeeds, the DP of DomainB sends a Federation Confirm message to the Fed-eration Behaviour of domain A. Upon receiving thisconfirmation, the latter confirms the federation and theFederation process is started. This process may dependon the two networks being federated, and may includeactions such as deploying a VPN, setting up virtualrouter parameters, among others. If the negotiationBehaviour rejects the negotiation, the DP of DomainB sends a Reject message to Federation Behaviour ofDomain A.

4.2.4 Governance behaviour

In the AutoI architecture, AMSs are self-governingelements. Hence, they may decide to change their poli-cies, SLAs and user requirements on the fly, withoutcooperation or intervention of the DOCs. The DOCsmust hence monitor the actions of the AMSs to ensurethat they are aligned with the high-level orchestrationgoals of each orchestration domain and also to preventinstability. The Governance Behaviour performs thismonitoring. Each instance of the governance Behav-iour monitors one AMS. In a nutshell, the functions ofthe governance Behaviour are as follows:

– Monitoring the actions of the AMSs: the DOCswatch the configuration and policies of the AMSs,always verifying if this configuration is still alignedwith the goals set at the orchestration level.

AMS (Domain A) AMS (Domain B)DOC (Domain A) DOC (Domain B)

Federation

FederationRequest(Negotiable &

non-negotiable policies)

Derivation of Negotiable & Non-negotiable policies

Negotiation Behavior

Federation confirmed

Distribution BehaviorDistribution Behavior

Fig. 5 Diagram of the federation behavior

– Enforcement of policies and SLAs def ined by theDOCs: the governance Behaviour checks for mis-behaviours, caused by AMSs changing their sys-tem level policies. Once they happen, the DOCswill take measures to notify the AMS of its non-compliance. Those requests take the form of func-tion calls, requiring the AMS to use a certain setof orchestration policies or a configuration that haspreviously been agreed.

– Trigger for federation, negotiation and distributiontasks upon non-compliance: the governance Be-haviour detects that an action from the part ofan AMS is conflicting with the objectives of theorchestration plane, allowing the DOCs to start upthe proper counter-measures. The DOC must findnew ways to ensure the smooth operation of thenetwork, such as the renegotiation of the systemlevel policies of one or more AMSs, the replace-ment of certain AMSs by another implementation,or the need to merge/split or migrate the network.

4.3 DOC reliability

The DOC is implemented using a modular architecture.The core of this architecture is the Dynamic Planner,which is kept as simple as possible in order to increasethe degree of fault tolerance of the DOCs. Since the DPcontrols the execution of Behaviours, it is able to detectif a Behaviour is failing. In this case, the DP may re-start failed Behaviours or trigger the activation of otherBehaviours when the problem persists.

Another issue that may lead to failures in the orches-tration process is inconsistencies in the orchestrationpolicies. In this case, the DOC cannot act on its own be-cause it lacks the knowledge of which policies are moreimportant (e.g. an operator may wish to maximise profitat all costs, while another may prefer network reliabilityeven if this leads to an initial period of revenue lossin an effort to attract more customers). Thus, in suchsituations the DOC sends an alert to the administrator,which must propose a new set of policies. Meanwhile,the DOC may revert to a known set of sane poli-cies, such as the previous set of non-conflicting policies.

Finally, failures can occur in the network elementrunning the DOC. Since AMS are self-governing, theyalso have its own ACL. As a consequence, the AMSsare able to detect misbehaviours or failures in theoperation of the DOCs of their domain. If an AMSdetects a failure in one of the DOCs, it may triggerthe redeployment of the entire DOC or its migrationto another device on the network.

Further, it is possible to delegate the responsibilitiesto another DOC if there is a failure in a certain DOC.

Ann. Telecommun. (2011) 66:243–255 251

This can be achieved due to several reasons. First,the orchestration components were designed usinga knowledge centric approach: declarative rule pro-gramming and ontologies were used to make ourKnowledge Base (KB) portable and interoperable aswell as independent of programming languages andcommunication technologies. Second, the orchestrationarchitecture is distributed, having the ability to keep therequired knowledge in more than one place. This allowsthe orchestration domain to demand the deployment ofanother DOC in a sane networking element based onthe information stored in the KB (rules and ontologies).

5 Orchestration policies

The AutoI orchestration plane is policy-based, usingpolicies for determining the operation of the DP andthat of the Behaviours. There are two types of or-chestration policies: Dynamic Planner policies definethe parameters of the DP, controlling the operationof the OP as a whole. Meanwhile, Behaviour-relatedpolicies control each of the specific orchestration tasksof the OP.

5.1 Dynamic planner orchestration policies

The DP uses policies to set limits on the execution ofthe workflows defined by the operator. Those policiesdefine the maximum amount of resources (e.g. CPU,memory, time, number of running workflows) usedfor each of the workflows. Orchestration policies alsodefine the actions taken by the DP when a policy failsor when a certain orchestration event is not capturedby any workflow. The DP policies also provide meansto improve the evolvability of the installed workflowsand policies. By means of default policies, operators areable to identify situations that were not foreseen at thedesign of workflows and policies. Further, policies areone of the tools to ensure the stability of the DP. TheDP employs the following types of policies:

Resource limiting policies resource limitation policiesat the DP level assure the stability of the system. Theyare put in place to terminate ill-behaved workflows,which use too many resources or take too much time tocomplete. Those policies are defined on a per-workflowbasis, as well as a generic policy for all the workflows.In case a certain workflow does not have an associatedresource-limiting policy, the default limiting policy isused if it exists.

“Multi-tasking” policies they limit the amount ofmulti-tasking of each workflow, i.e. constraining the

amount of workflows that may be triggered by eachworkflow. A DP-wide policy can also be defined, inorder to specify the maximum amount of workflowsrunning at each DP. Excess workflows may be put inan execution queue. Similar to the resource limitingpolicies, multi-tasking policies are defined to improvethe stability of the DP platform. They may catch ill-behaved workflows or design flaws in the writing of theworkflows.

Default action policies those policies define actions toperform when a workflow fails, or when no workflowsare attached to a certain event. This approach is similarto the default action of switch clauses in programminglanguages. Default action policies might be used, forexample, to communicate to the operator that the OPfound a situation that was not contemplated by the in-stalled workflows. As an example, when an ill-behavedworkflow overflows its allocated resources, the opera-tor could be notified, receiving the name of the work-flow as well as some measurements related to the stateof the network and the state of AMSs and Behaviours.

5.2 Behaviour policies

Behaviour policies control the federation, distribution,negotiation and governance aspects of the DOCs. Be-haviours have their own policies, as follows:

Distribution policies Policies in the distribution Be-haviour define the components to be deployed for agiven AMS and their capabilities. Capability policiesdescribe, in a somewhat abstract way, the functionsthat the deployed AMSs must have. They may definethe type of autonomic management that it should per-form (i.e. self-configuration, self-optimization), as wellas the service that it will manage (multimedia, Web,telephony)1.

Further, distribution policies define the QoS con-straints and SLAs of the AMSs. Some of those pa-rameters may be negotiable, while others may not.QoS and SLA-related policies will guide the AMSs inthe deployment of the specific components requiredfor the service being distributed. The QoS and SLApolicies are derived from the high level goals definedby the operators of the orchestration domains and ofthe federation that the DOC belongs to. The policiesdefine ranges of values for different QoS parameters,

1Each AMS is responsible for only one service, in order tosimplify the operation of the AMSs. This allows the AMSs tofocus on the autonomic management of the services, while theOP deals with the issues arising from the interaction of services.

252 Ann. Telecommun. (2011) 66:243–255

and if those parameters are mutable or not. This allowsthe AMSs to redefine the QoS constraints of specificcomponents of the service as well as the entire service.

Negotiation policies The negotiation Behaviour usespolicies to represent the parameters of the negotiationprocess, as well as the ranges of AMS and serviceparameters being negotiated. They define the non-negotiable terms of the SLA, that is, the requirementsthat one of the AMSs or the DOC are not willing tocompromise on. Negotiable parameters, on the otherhand, use policies to define a set of ranges or allowedvalues. Those policies can be used to describe thecapabilities of the AMSs and DOCs, as well as whatresources and service levels each of the components arewilling to provide.

Finally, negotiation policies may define properties ofthe negotiation process, setting values for the parame-ters of the negotiation approaches. Such policies coulddefine the maximum amount of negotiation iterations,the maximum time per iteration, among others. Themost important use of those policies is to define limitson the negotiation process, i.e. to improve the scala-bility and stability of the OP. Those policies may alsodefine goals that must be achieved at each negotiationround, and actions if those goals are not met. For exam-ple, an AMS should provide a different, more flexibleset of constraints at each round. Another policy couldstate that, if an AMS is inflexible after a certain numberof rounds, the negotiation process is stopped.

Governance policies they define a set of operationalparameters for the DOCs, as well as the actions thatshould be taken for certain violations. Governance poli-cies are derived from the high-level goals and user re-quirements of the services. They resemble events, beingdefined in the following form: if a set of parameterslies outside the defined ranges, then do the specifiedactions. The monitored parameters may comprise prop-erties of the AMS, user-facing services or virtual links.For example, if a certain service meets a significantdegradation in the QoS, the governance Behaviour maydecide to renegotiate the SLAs of the self-configurationAMS managing this service.

Federation policies they define which AMSs shouldparticipate in the federation process, and how theycommunicate and negotiate. The list of involved AMSsis based on the purpose of the federation and the spe-cific requirements of the federation. Further, policiesare also used to define the set of negotiable and non-negotiable QoS parameters and SLAs that must drivethe federation. Based on these policies, self-governedAMSs check if the high-level federation policies are not

in contradiction with their own non-negotiable policies.If there is a conflict, the AMS rejects the federation,as there is no possibility of negotiation. In contrast, ifthere is a disagreement in negotiable policies, the fed-eration process continues after one or more negotiationrounds.

6 Use case

This section presents an use-case of the AutoI architec-ture exercising the functions of the DOCs. More specif-ically, we use the DOCs to support the deploymenta new service. This deployment is triggered by a mo-bile client roaming among different wireless networks.Section 6.1 describes the scenario of the use-case. Next,Section 6.2 shows how this scenario would be realisedusing the AutoI architecture. Finally, Section 6.3 brieflycomments on the implementation details of the negoti-ation phase of the scenario.

6.1 Scenario description

Consider an application service that provides largeamounts of multimedia files which are stored on ge-ographically distributed servers. Application servicesare provisioned to home and corporate clients withcertain contracted quality levels, each with concreterequirements. For timely-effective service provision,the client downloads content from the closest server,which in turn would get the information from the serverthat stores the information. Servers provide a kind ofP2P overlay network [11] with communication channelshaving large capacity to download information directlyfrom the server that stores it. Clients are not associatedto any permanent server and can use different typesof terminals. Further, clients can be fixed or mobile.The latter ones may pass through locations with variousaccess systems and technologies, namely areas withaccess points for local area network IEEE 802.11 (Wi-Fi), wide area network fixed and mobile (IEEE 802.16and IEEE 802.16e, respectively) and regional area net-works (i.e. IEEE 802.22—WRAN). In order to copewith the requirements of the clients, especially security,the system establishes a virtual private network (VPN)between the server with stored information and theclient terminal as long as it is appropriate and possible,e.g. different encryption processes may be coordinatedalong a path and sections that could not provide guar-anteed security should be avoided. A simplified rep-resentation of this use case is depicted in Fig. 6 forwhich relevant orchestration operations are describedthereafter.

Ann. Telecommun. (2011) 66:243–255 253

Fig. 6 Use caserepresentation

Consider the case where the client in Fig. 6 enjoysa VoD service at Location A and that moves towardsa location in which two access controllers can connecthim (Location A ∩ B in Fig. 6). The AMS Wirelessshould decide to which access controller the client willbe connected taking into account a number of factslike client resource demands, its profile, access pointload, etc. Let us consider that the result of this decisionis to have the client connected to access controller B.The AMS Fixed that is sensible to this context changereacts accordingly, setting up VPN2 in Fig. 6 to enablethe transmission of the packet stream from the contentserver to the access controller to which the client is nowconnected to. Also as VPN1 is no longer needed, it isended to release resources.

6.2 Scenario workflow

Before application services are actually configured, theDOC registers all available management domains, eachspecialised with an AMS. At this stage, the AMSs arenot active, they are registered to further participatein the potential provision of services and hence toeventually negotiate and federate. Application servicedeployments can be started on demand by the clientcontacting an administrative domain, or contacting theuser interface of the DOC. In any case, negotiationrounds among the registered AMSs are coordinated bythe DOC with the aim to define what services are provi-sioned by which AMSs. The DOC provides the meansto facilitate negotiations as AMSs can belong to severaldifferent operators and can exchange information indifferent formats, mechanisms or protocols.

Negotiation requests consist of an SLA specifying anagreement between a service provider and a customer.The SLA is communicated by the client to the AMSthe client is attached to. The AMS then delegates therequest to a DOC if it is not capable of providing the

requested SLA on its own. In our case, the AMS isnot hosting the requested service, which is Video onDemand (VoD), and thus forwards the request to afederated DOC.

DOC knowledge requirements The DOC has its ownKnowledge Base (KB) for keeping the knowledge re-quired in the orchestration process (intra-system view).This knowledge has to do with:

1. An up-to-date view of the network topology andexact knowledge of the conditions inside the net-work regarding interconnected AMSs.

2. Rules and policies to analyze the request and locatealternatives for realizing it with the minimum cost.

3. The ability to break down the request to two ormore parts.

4. The ability to deploy or discover AMSs and under-stand and use their communication interfaces.

The DOC uses the orchestration-related knowledgementioned above to break down the request to two ormore parts, each destined for the appropriate AMS inorder to deploy the requested service. In this process,which will define the final request from the DOC tothe AMSs, the high level goals and policies are incorpo-rated to draw the final decision on what each AMS willbe asked to do. After defining what each AMS shoulddo we enter the negotiation process, where the AMSsexchange proposals in order to reach an agreementbased on the guidelines the DOC has provided. TheDOC takes care of the following negotiation issues:

1. The time within which an agreement must bereached. Time is considered important due to theneed to provide a quick answer to the requestingclient.

2. The negotiating agents capabilities in relation tothe resources available depending on the exact en-vironment they are deployed into.

254 Ann. Telecommun. (2011) 66:243–255

3. The selected protocol for the negotiation.4. The semantics in the messages and proposals being

exchanged.5. The decision-making logic.

The negotiation ends with an agreement among theAMSs, consisting of a new SLA. Thus, the AMSs en-force this SLA using their level of knowledge and thepolicy continuum for translating and configuring virtualor non-virtual resources.

The DOC activates the AMSs that ended up withsuccessful and appropriate negotiations and providesthem with concrete high-level directives of the serviceprovision to each of them. The two AMSs shown inFig. 6 (AMS Fixed and AMS Wireless) have beenselected after a negotiation process among a numberof potential providers, as they offered the most appro-priate AMS services for the provision of the requestedapplication service.

It is worth mentioning that the configuration andmaintenance of application services passes through aneffective and systematic refinement process [17] of thehigh-level directives that the DOC provides to theAMSs. AMSs should internally derive the system levelpolicies (or other means) with which they will controltheir resources to provide the compromised QoS. Inour use case, system level policies should define whatcontext variables an AMS should subscribe to, andthe actions that should be taken when such contextvariables change. The AMS Fixed for example shouldsubscribe to context variables that help in determiningto which edge router it will deliver the content (e.g. aninterface in Router A in Fig. 6) and the characteristicsof the VPN.

6.3 DOC implementation

The implementation of the Dynamic Planner used aninference engine that exploits the DOCs knowledgebase and reasons for the next step to be taken. Knowl-edge is appropriately formalised as deductive logic pro-grams with the use of JESS rules. Deontic and ECArules were mostly used. Deontic policies were used dueto their advantage in allowing multiple systems to coor-dinate, for example, in exchanging information (autho-risation, prohibition) or for giving orders (obligation,dispensation). ECA policies are reaction-based rulesso that a DOC can react to events and requests. Goalbased policies were also introduced for the high-levelpolicies of the DOC. Utility Function-based policies,on the other hand, were used to answer our need fora quick decision-making process, as it was the case forthe negotiation between AMSs. Rules were loaded and

executed into the JESS inference engine, which outputsdecisions to be enforced by the Java engine.

7 Conclusions

This article presented the concept of the Orchestra-tion Plane as well as its architecture within the AutoIEU project. The need for an OP in next-generationnetworks arises from the deployment of several au-tonomic management systems, which must cooperateto achieve an acceptable end-to-end QoS. Since eachautonomic system will have its own policies and SLAs,those systems will need to compromise in order to meetthe user’s demands. This process is facilitated by theorchestration plane, which intermediates any commu-nication among AMSs and oversees their operation.The OP has four main tasks: the distribution of AMSs,the federation of domains, the negotiation of SLAs andgoals among AMSs and the monitoring of AMSs, dueto their self-governance capabilities.

The AutoI orchestration plane is composed ofDOCs, which are distributed components deployed onthe network. DOCs are a generic framework for theexecution of workflows, which dictate how each of theorchestration tasks are performed. They use a genericscheduler, the Dynamic Planner, to start/stop tailor-made components specific to each task (the Behav-iours). DOCs use policies to customise the operationof the Dynamic Planner as well as the Behaviours.This organization provides an extensible yet simplearchitecture, where the functionalities of the OP can beextended or modified at run-time.

We demonstrate the use of such an architecture inthe context of user mobility, where services must beredeployed on demand. This requires the negotiationof new SLAs among the AMSs, which is coordinatedby the DOCs.

Acknowledgements This article has been realised in the contextof the Autonomic Internet (AutoI) EU research project. AutoIis an FR7 partially financed EU project for the period 2008–2009. It is backed by a consortium led by Hitachi Europe andcomposed by partners from Waterford Institute of Technology(Ireland), University College London (UK), Polytechnic Univer-sity of Catalunia (Spain), INRIA (France), University of Passau(Germany), University Paris VI (France), UCOPIA (France),University of Patras (Greece), Gingko Networks (France).

References

1. Bassi A, Denazis S, Fahy C, Serrano M, Serrat J (2007) Au-tonomic internet: a perspective for future internet servicesbased on autonomic principles. In: 2nd IEEE international

Ann. Telecommun. (2011) 66:243–255 255

workshop on modelling autonomic communications environ-ments (MACE)

2. Berl A, Fischer A, de Meer H (2009) Using system virtualiza-tion to create virtualized networks. In: GI/ITG workshop onoverlay and network virtualization (NVWS)

3. Boyle J, Cohen R, Herzog S, Rajan R, Sastry A (2000)RFC2748: the COPS (Common Open Policy Service)protocol

4. Clark DD, Partridge C, Ramming JC, Wroclawski JT (2003)A knowledge plane for the internet. In: Proceedings ofthe conference on applications, technologies, architectures,and protocols for computer communications (SIGCOMM),pp 3–10

5. The AutoI consortium. AutoI—autonomic internet project.Available at: http://www.ist-autoi.eu/. Accessed 12 June 2010

6. Davy S, Barrett K, Balasubramaniam S, van der Meer S,Jennings B, Strassner J (2006) Policy-based architecture toenable autonomic communications—a position paper. In: 3rdIEEE consumer communications and networking conference(CCNC), pp 590–594

7. Pujolle G (ed) (2009) Deliverable D2.2: orchestration planeand interfaces. Technical Report D2.2, Autonomic Internet(AutoI) project

8. Fahy C, Davy S, Boudjemil Z, van der Meer S, Loyola JR,Serrat J, Strassner J, Berl A, de Meer H, Macedo DF (2008)Towards an information model that supports service-aware,self-managing virtual resources. In: 3rd IEEE internationalworkshop on modelling autonomic communications environ-ments (MACE), pp 102–107

9. Ganek AG, Corbi TA (2003) The dawning of the autonomiccomputing era. IBM Syst J 42(1):5–18

10. Greenberg A, Hjalmtysson G, Maltz DA, Myers A, RexfordJ, Xie G, Yan H, Zhan J, Zhang H (2005) A clean slate 4Dapproach to network control and management. SIGCOMMComput Commun Rev 35(5):41–54

11. Lua EK, Crowcroft J, Pias M, Sharma R, Lim S (2005)A survey and comparison of peer-to-peer overlay networkschemes. IEEE Communications Survey & Tutorials 7(2):72–93

12. Mamatas L, Clayman S, Charalambides M, Galis A, PavlouG (2010) Towards an information management overlay forthe future internet. In: IEEE/IFIP network operations andmanagement symposium (NOMS)

13. Movahedi Z, Abid M, Macedo DF, Pujolle G (2009) A policy-based orchestration plane for the autonomic managementof future networks. In: 6th international workshop on nextgeneration networking middleware (NGNM)

14. Nguyen TMT, Boukhatem N, Doudane YG, Pujolle G (2002)COPS-SLS: a service level negotiation protocol for the inter-net. IEEE Commun Mag 40(5):158–165

15. Pentikousis K, Galis A, Agüero R (2009) Information man-agement and sharing for ambient multiaccess networks.In: IEEE global information infrastructure symposium(GIIS)

16. Rubio-Loyola J, Mérida-Campos C, Willmott S, Astorga A,Serrat J, Galis A (2009) Service coalitions for future internetservices. In: IEEE international conference on communica-tions (ICC)

17. Rubio-Loyola J, Serrat J, Charalambides M, Flegkas P,Pavlou G (2006) A methodological approach toward therefinement problem in policy-based management systems.IEEE Commun Mag 44(10):60–68

18. Strassner J, Foghlu MO, Donnelly W, Agoulmine N (2007)Beyond the knowledge plane: an inference plane to supportthe next generation internet. In: First international globalinformation infrastructure symposium (GIIS), pp 112–119

19. Studwell TW (2003) Orchestrating self-managing systems forautonomic computing: the role of standards. In: IFIP/IEEEinternational workshop on distributed systems: operationsand management (DSOM)

http://www.ist-autoi.eu/

The AutoI approach for the orchestration of autonomic networks

Documents

Transcript of The AutoI approach for the orchestration of autonomic networks