An Efficient Framework for Running Applications on Clusters, Grids, and Clouds

An Efficient Framework for RunningApplications on Clusters, Grids and Clouds

Brian Amedro, Francoise Baude, Denis Caromel, Christian Delbe, Imen Filali,Fabrice Huet, Elton Mathias, and Oleg Smirnov

Abstract Since the appearance of the distributed computing technology, there hasbeen a significant effort in designing and building the infrastructure needed to tacklethe challenges raised by complex scientific applications, which require massivecomputational resources. This increases the awareness to harness the power and theflexibility of Clouds, which have emerged recently as an alternative to data-centersor private clusters. We describe in this chapter an efficient high level Grid and Cloudframework which allows a smooth transition from clusters and Grids to Clouds. Themain lever is the ability to move application infrastructure specific information awayfrom the code and manage them in a deployment file. An application can thus easilyrun on a cluster, a grid or a cloud or any mixture of them without modification.

1 Introduction

Traditionally, HPC relied on supercomputers, clusters, or more recently computinggrids. With the rising concept of cloud computing and effective technical solutions,questions like “is cloud computing ready for HPC”, or “does a computing cloudconstitute a relevant reservoir of resources for parallel computing” are around. Thischapter gives some concrete answers to such kind of concerns. Offering a suitablemiddleware and associated programming environment to HPC users willing to takeadvantage of cloud computing is also a concern we address in this chapter. One nat-ural solution is to extend a grid computing middleware in a way it becomes ableto harness cloud computing resources. A consequence is that we end up with amiddleware that is able to unify resource acquisition and usage of Grid and Cloudresources. This middeware was specially designed to cope with HPC computationand communication requirements, but its usage is not restricted to this kind of ap-plication.

INRIA – I3S – CNRS – UNSA, France, e-mail: [email protected]

1

2 Authors Suppressed Due to Excessive Length

This chapter explores in details the relevance of using hybrid Grid/Cloud en-vironments and the technical challenges that such mixing raise at the middlewarelevel. In particular, this chapter provides and analyses performance results we ob-tained on Amazon EC2 running some of the NAS parallel benchmarks. Then, weprovide some insight into two complementary and relevant concepts: Cloud burst-ing and seeding: (1) cloud bursting is relevant whenever the amount of availableresources on a cluster or grid is not sufficient to face a required level of computingpower, and so must be augmented with some Cloud resources, be it in a static andanticipated way, or dynamically in an on-demand basis; (2) cloud seeding is relevantwhenever some specific kinds of resources are not available within the computingcloud and so, must be acquired from outside.

The remainder of this chapter is organized as follows: Section 2, presents someexisting cloud frameworks and our position in relation to them. Section 3 givessome benchmarks and proposes solutions to deploy applications in a Cloud, throughthe proposed framework. Section 4 details the application deployment in a unifiedenvironment mixing Grids, Cluster and Clouds. Section 5 deals with the deploymentprocess under cloud bursting and cloud seeding scenarios. Section 6 concludes thischapter and outlines future directions.

2 Related Work

2.1 General view of Cloud computing frameworks

Cloud services are mainly divided into three services delivery models: SaaS (Soft-ware as a Service, e.g. Google Mail), PaaS (Platform as a Service, e.g. Google Ap-pEngine) and IaaS (Infrastructure as a Service, e.g. Amazon EC2). Since the workpresented in this chapter is strongly related to Infrastructure as a Service model, weonly focus in this section on this category of service. IaaS providers aim to offerresources to users as pay-as-you-go manner. A key provider of such a service isAmazon through its Elastic Cloud Computing (EC2) and Simple Storage Service(S3).

Some services or tools have been proposed to ease the use of Clouds or en-hance their functionalities. enStratus[6] provides a set of tools for managing Cloudinfrastructure and handling the ”confidence” questions about moving an applica-tion into a Cloud. The user does not need to change a line of code for enStratus.Scalr[11] is a hosting environment for Amazon EC2. It provides services such asload balancing, fault tolerance and self-scaling. Vertebra [12] is a Cloud computingframework for the orchestration of complex processes. It takes into considerationsecurity, fault tolerance, and portability aspects. The OpenNebula Toolkit [2] is avirtual infrastructure engine that allows a dynamic deployment and reallocation ofvirtual machines. It leverages existing virtualization platforms to come up with anew virtualization layer between the service and the physical infrastructure. It sup-

An Efficient Framework for Running Applications on Clusters, Grids and Clouds 3

ports private, public and hybrid Cloud deployment model. Nimbus [1] is a set ofopen source tools that provide an IaaS Cloud computing solution. It allows usersto lease remote resources by deploying virtual machines on those resources. Usingthe Nimbus Cloud, requested resources can be dynamically adjusted as function ofthe application needs.

2.2 Cloud computing middleware

In order to run applications on a Cloud, one needs a flexible middleware which easesthe development and the deployment process. GridGain [8] provides a middlewarethat aims to develop and run applications on both public or private Clouds withoutany changes in the application code. It is also possible to write dedicated applica-tions based on the map/reduce programming model. Although GridGain providesmechanism to seamlessly deploy applications on a Grid or a Cloud, it does not sup-port the deployment of the infrastructure itself. It does, however, provides protocolsto discover running GridGain nodes and organize them into topologies (Local Grid,Global Grid, ...) to run application on only a subset of all nodes.

Elastic Grid [7] infrastructure provides dynamic allocation, deployment andmanagement of Java applications through the Cloud. It also offers a Cloud virtual-ization layer that abstracts specific Cloud computing provider technology to isolateapplications from specific implementations.

3 Deploying Applications in the Cloud

For the rest of our study, we will focus on the Amazon EC2 Web Service. Initially,we seek to determine the performance that can be expected. Then, we propose so-lutions to facilitate the deployment of applications in this context and to enablethe usage of hybrid Grid/Cloud environments. To reach this goal, we will use theProActive Parallel Suite[3]. This framework is composed by three parts: 1.ProActiveProgramming offers a Java API for parallel and distributed computing, 2.ProActiveResource Manager gathers heterogeneous computing resources (parallel machines,clouds, grids, etc...) into an unified access mechanism (further details are given inSection 5.1, and 3.ProActive Scheduler runs any kind of tasks (native, Java, Matlab,etc...) on a set of nodes acquired by the resource manager.

3.1 Benchmarking the cloud

In order to asses the worthiness of using Amazon EC2 cloud as an HPC platform,we have deployed a series of benchmarks well known in the world of HPC, the MPI


NAS Parallel Benchmarks. We have launched them on four different architectures,described in Tab.1: a private cluster and three types of Amazon EC2 instances. Inorder to provide consistent and predictable amount of CPU capacity, Amazon de-scribes the CPU capacity of its instances in terms of EC2 Compute Unit and claimthat it is the equivalent CPU capacity of a 1.0-1.2 GHz 2007 Opteron.

Private ClusterProcessors 2 QuadCore Opteron 2356 (2.3GHz) / 64-bitsMemory 32 GBHard drive 2*73 GB SAS 15000 rpmI/O Performance Gigabit Ethernet

Small High-CPU Medium High-CPU XLargeEC2 Compute Units 1 /32-bits 5 (2*2.5) / 32-bits 20 (8*2.5) / 64-bitsMemory 1.7 GB 1.7 GB 7 GBHard drive 160 GB 350 GB 1690 GBI/O Performance Moderate Moderate High

Table 1 Deployment architectures

S M XL Private0

20

40

60

80

100

120

thro

ughput

(MB

)

Throughput

0.0

0.2

0.4

0.6

0.8

1.0

late

ncy

(m

s)

Latency

Fig. 1 I/O performance comparison between a private cluster and EC2 instances

The chart presented on Fig.1 shows three stages. The Small and the Medium in-stances share the same throughput and latency, describing the Moderate EC2 I/Operformance, while the XLarge instance reflect the High EC2 I/O performance.Compared to the Gigabit Ethernet connectivity provided by our own private clus-ter, there is a large gap, especially for latency.

Figure 2 shows the performance (mflops) of three of the NAS Parallel Bench-marks on each architecture, varying the number of processes. Results averaging 10runs and their variation does not exceed 6%. Up to 32 processes, we run one processper machine, then, we increase the number of processes per machine.

• EP is an embarrassingly parallel problem which involves almost no communica-tion between processes. It is a strong test for pure computational speed. This test


8 16 32 2*32 4*32 8*32 8 16 32 2*32 4*32 8 16 32 2*32 4*32EP - Class C CG - Class C FT - Class C

0

1000

2000

3000

4000

5000

6000

mflo

ps

EC2 SmallEC2 High-CPU MediumEC2 High-CPU XLargePrivate Cluster

Fig. 2 NAS Parallel Benchmarks

clearly shows the speed difference between all the instances. In terms of speed,the XL instance is roughly equivalent to our private architecture (8 cores at 2.3GHz), whereas the medium instance runs at the same frequency, but with onlytwo cores. Similarly, we see that the small instance is about 2.5-3 times slowerthan the others.

• CG computes a conjugate gradient involving a large number of small messages,and is a strong test for communication performance. It confirms the results shownin Fig.1. The Amazon EC2 instances performance are in fact well below what weget with our private cluster.

• FT is a Fourier transformation benchmark. It is a test for both, computation andcommunication speed involving large data transfers. With such a problem, thegap between our private architecture with medium and XL instances narrows.

As shown by the previous experiments, EC2 does not offer good performancefor communication intensive applications, compared to a local cluster. However,CPU intensive applications do not present significant performance hit. When deal-ing with a complex application mixing communications and computations, it mightbe interesting to have a part on a Cloud and another on a cluster, depending onthe application characteristics and the possibility to decompose the application inseparate parts. This however makes deploying such application more complex.

We now present different mechanisms that simplify the execution of applicationsover heterogeneous environments.


3.2 The ProActive GCM Deployment

The ProActive middleware provides an abstract descriptor-based deployment modeland framework [4], giving users the capability to deploy an application on differentplatforms without changing the source code. The idea behind the ProActive GCMDeployment is to perform the discovery of resources, creation of remote processesand data handling externally to the application, separating completely the businesscode and deployment. In addition to these activities, the definition of the deploymentcan also encompass security, tunneling of communications, fault tolerance and sup-port of portable file transfer operations.

The whole deployment process (Fig. 3) and environment configuration is definedby means of XML descriptors which depict the application requirements and de-ployment process. The deployment of ProActive/GCM applications depends on twotypes of descriptors:

• GCM Application Descriptors (GCMA): the GCMA descriptors define applications-related properties, such as localization of libraries, file transfer, application pa-rameters and non-functional services (logging, security, checkpoint and fault tol-erance). GCMA descriptors expose the resulting physical environment as a logi-cal network of virtual nodes (VNs) which are used by applications as an abstractrepresentation of computing nodes. The GCMA also defines one or multiple re-source providers;

• GCM Deployment Descriptors (GCMD): the GCMD descriptors define accessprotocols to reach the resources (e.g. SSH, RSH, GSISSH, etc.), acquisition pro-tocols and tools which are required to access to resources (e.g. Amazon EC2,PBS, LSF, Sun Grid Engine, OAR, etc.), creation protocols which have a relationon how to launch processes (e.g. SSH, OAR, gLite, Globus) and communicationprotocols (e.g. RMI, RMISSH, HTTP, SOAP, etc).

The advantages of this model are clear: if on one side users want to add a newresource provider (e.g. a private cluster, production grid or cloud), the applicationcode does not change and a single line is enough to add the resource provider tothe application descriptor (GCMA). On the other side, the definition of the deploy-ment process happens just once for each resource and can be reused for differentapplications.

3.3 Technical Solutions for the Deployment over HeterogeneousInfrastructures

In the best possible scenario, all the machines involved in one computation are exter-nally accessible through a public IP without any network restriction. In practice, thisrarely occurs and machines are usually isolated by firewall and NAT. So, we mustexplore more sophisticated strategies to make possible the communication amongprocesses located in different domains.


Application Descriptor (GCMA)

Application/ADL

Application Definition

Application Properties Resources Requirements Resources Providers

Access Protocols Resource Acquisition Protocols

Creation Protocols

Infrastructure Definition

Deployment Descriptor (GCMD)

VN

Fig. 3 GCM Descriptor-based deployment model

3.3.1 Virtual Private Network (VPN)

A Virtual Private Network (VPN) is an overlay network built on top of an existinglarger one. It is often installed to provide a secure extension of a private networkinto an insecure environment such as the Internet. Thus, communication betweennodes are tunneled through the underlying network, bypassing firewalls.

In order to expose a private network to the Internet to allow some external ma-chines to connect into, a VPN Gateway , which will be the entry point of the networkmust be configured. This gateway will be part of the private network, but also hasaccess to the Internet. Then, each client machine who wishes to join the VPN willconnect to the gateway. Regarding firewalls, client and gateway must be defilteredfor both input and output VPN traffic.

A VPN can thus offer a way to add external resources to an IaaS, or add IaaSresources to a private infrastructure.

3.3.2 Amazon Virtual Private Cloud (VPC)

Amazon Virtual Private Cloud (VPC) service provides a private subnetwork withinthe Amazon Cloud. All EC2 nodes, composing this Amazon VPC, are isolated andcan only be reached through a VPN connection from a private network. It allows aseamless extension of an existing infrastructure and EC2 resources can be managedas private ones.

This service allows to extend existing security and management policies of aprivate IT infrastructure to the VPC. By doing so, it allows applications to be seam-lessly executed in multi-domain environments.


3.3.3 Message Forwarding and Tunneling

The ProActive middleware also offers a solution to address network restrictions likefirewalls and NAT, which is built-in and lighweight, based on SSH. This solutionalso provides a seamless integration of forwarding and tunneling, but at the appli-cation level (i.e. no need to configure routing at the OS and network levels). It onlyrequires a configuration of entry points of each involved domain and used commu-nication protocols, like ssh.

Figure 4 depicts a scenario where a single application runs over a set of nodesdistributed in Amazon EC2 Cloud and Grid50001. In this scenario, all the nodes lo-cated in Amazon EC2 cloud offer inbound and outbound communication, but nodeslocated on Grid5000 are isolated from the external network. ProActive, however,enables the usage of these resources as if every node would be accessible by everyother node by forwarding incoming and outgoing messages through the Grid5000gateway.

EC2 computing instances

Grid5000gateway

Computing nodes

INTERNET

FirewallGrid5000Amazon EC2

RMISSH / HTTP / SOAPCommunications

RMI Communications

Fig. 4 Tunneling and forwarding communications on an heterogeneous Cloud-Grid environment

In a more protected environment, nodes might be isolated in both sides. TheProActive built-in tunneling/forwarding can be configured through a double for-warding mechanism to handle such a situation. In any case, applications remainunchanged and the execution in different scenarios only requires the modificationof configuration files associated to the ProActive Runtime. The communication pro-cess may involve a multi-protocol approach.

3.4 Conclusion and Motivation for Mixing

We have seen that the deployment of an application in a heterogeneous environmentcan be simplified with tools such as the ProActive GCM Deployment. Technicalsolutions such as VPN and SSH tunneling are used to manage the characteristics ofa network while securing connections. We have also seen that, an IaaS as Amazon

1 Grid5000 is a French national Grid distributed over 9 sites for a total of about 5000 cores


EC2, offers a range of instances with features and performance that can match userneeds.

Thus, we can consider to set up different usage strategies to mix resources withdifferent goals. Strategies could be geared towards a full transition phase towardsfull cloud outsourcing computing, an optimization of costs by fitting the choice ofcomputing resources to the needs of the application, or by temporarily extending aninfrastructure to meet a special need.

4 Moving HPC Applications from Grids to Clouds

It is not yet clear how much of impact Clouds will have on HPC in the future. In fact,it is very unlikely that highly optimized clusters will be replaced by Cloud resources,in a near future. Considering that most enterprises and public institutes already countwith in-house HPC resources, which could provide processing power at lower costs,the notion of private Clouds or a mix between Clouds and cluster/Grids resourcesseems more cost-effective to solve the problem of resources providing.

Scientific applications require sophisticated middleware, because they usuallypresent complex multi-point interactions and strong processing and network re-quirements, which directly impact performance. Porting such kind of applicationsto heterogeneous environments increase the importance of middleware support.

In this section we present a versatile GCM/ProActive-based lightweight frame-work that supports distributed and parallel scientific applications so that porting oflegacy applications is possible and easy to any kind of distributed computing envi-ronment or even a mixture of them. To illustrate this, we also present performanceresults obtained with a scientific PDE-based application in different contexts, in-cluding an experimental Grid, a public Cloud and the mixture of these infrastruc-tures.

4.1 HPC on Heterogeneous Multi-Domain Platforms

From the development point of view, the usage of resources spread across multi-domain platforms as a single infrastructure requires an integrated middleware. Suchmiddleware should provide users with clear abstractions to develop applications thatcould be easily adapted to be deployed on different resource providers, despite ofdifferent underlying resources characteristics.

In the next section we present in more details a component-based integratingmiddleware which emphasizes a clear separation between application developmentand the execution platform. This middleware eases the transition from clusters togrids and Clouds by providing seamless deployment and multi-protocol point-to-point and multi-point communication in multi-domain environments.


4.2 The Hierarchical SPMD concept and Multi-level Partitioningof Numerical Meshes

The traditional way of designing domain decomposition based simulations is toadopt an SPMD technique combining mesh partitioning and the message passingprogramming model. The hierarchical SPMD is an evolution of the traditional flatSPMD parallel programming paradigm towards a heterogeneous hierarchical ap-proach. The hierarchical SPMD concept consists in assigning hierarchical identifiersto processes and treating collective communications in a topology-aware manner.

Heterogeneity in network and resources is a challenging issue for domain de-composition based scientific applications. The main reason comes from the fact thatthese applications rely upon a bulk synchronous iterative approach and applicationsloop at the pace of the slowest process. The hierarchical network topology and com-puting power heterogeneity must, therefore, be considered in the mesh partitioningand communication process.

We propose a multi-level partitioning approach to balance load among processorsand optimize the communication process. The multi-level partitioner is capable oftaking into account resources characteristics (CPU power, amount of memory) andtopology to partition a global mesh in a way that each process presents an equiva-lent processing time, yet minimizing communication through slower links [9]. Thedifferent levels defining the physical hierarchy are mapped into the communicationprocess, which is configured depending on the effective location of communicatingprocesses and the available communication protocols. The runtime also takes topol-ogy into account to stage the communication operations so that communication overslower networks (e.g. Internet) are avoided.

4.3 The GCM/ProActive-based Lightweight Framework

The GCM/ProActive-based lightweight framework takes the form of a component-based infrastructure that offers support to multi-protocol communication. This in-frastructure is composed accordingly to the hierarchy of resources and gives theapplications a view of a unique global computing infrastructure, despite of the lo-calization and access restrictions of resources.

Figure 5 shows an example of such composition, which reflects a single globalapplication deployed upon a resources set deployed onto two separate but inter-connected administrative domains. On the left, we run a standalone MPI applica-tion running on a Cloud (e.g. a set of Amazon EC2 instances) and on the rightanother standalone MPI application running over a multi-cluster based grid (e.g. theGrid5000). Each of the MPI processes is wrapped by a GCM primitive componentwhich is connected to the external router component representing the next level upin the infrastructure. Due to the hierarchical composition and the routing interfacesassociated to higher levels, all the nodes are logically connected, even if indirectly,


to every other in the multi-domain, and so the independent MPI executions are cou-pled to form a single parallel application along the Hierarchical SPMD concept.

Router 1Router 0

p1:0 p1:2

p1:1p0:0

p0:1

MxN Interface

Proxy Component Router Component

Fig. 5 Typical GCM/ProActive-Based Multidomain Runtime Support for HPC

Collective communications take profit of the topology, enabling them to bestaged and parallelized. Besides and whenever possible, for optimization purposes,we can create on-demand direct bindings to perform point-to-point communications,thus bypassing the hierarchy.

4.4 Performance Evaluation

We evaluate the component-based framework and the hierarchical SPMD modelthrough a non-trivial simulation of electromagnetic wave propagation in three-spacedimensions. This simulation is based on a finite element method working on arbi-trarily unstructured tetrahedral meshes for solving a system of Maxwell equations.

From the computational point of view, the execution is characterized by two typesof operations: purely local operations on tetrahedra for computing integral valuesand a calculation involving neighbor subdomains which involves a gather-compute-scatter sequence. Formulations are described in more details in [5].

In [9], we highlighted performance improvements for applications developedalong the Hierarchical SPMD concept compared to pure “flat” MPI implementa-tions including grid-aware ones (as Grid-MPI). The experimental results we presenthere focus on the comparison among three scenarios: a multi-cluster grid, the Ama-zon EC2 Cloud and a multi-domain environment which couple both setups.

The experiments we present here were conducted in one cluster (grelon locatedin Nancy, France) of the Grid5000 testbed and the Amazon EC2 platform with twoinstance sizes: Small instances and High-CPU Extra Large Instances. Grid5000 re-sources present Dual Core Intel Xeon 5110 (1.6GHz) with 2GB of memory andGigabit ethernet interconnection. Small Amazon EC2 instances represent 1 virtualcompute unit with 1GB of memory and High-CPU Extra Large 20 compute units


(8 virtual cores with 2.5 EC2 Compute Units each) with 7GB of memory. The soft-ware involved on these experiments are Java Sun SDK v1.6.0 07, ProActive v3.91and GridMPI v2.1.1.

4 8 16 32 64Number of Nodes

0

5000

10000

15000

20000

25000

MFl

ops/

s

Average MFlops/s Performance

EC2 Small

EC2 Small + G5K

EC2 XLarge + G5K

EC2 XLarge

G5K

4 8 16 32 64Number of Nodes

0

200

400

600

800

1000

1200

1400

1600

1800

Tim

e(s

)

Execution Time

EC2 Small

EC2 Small + G5K

EC2 XLarge + G5K

EC2 XLarge

G5K

Fig. 6 Performance over Grid5000, Amazon EC2 and resource mix

Figure 6 presents the overall execution times and MFlops/s obtained in the dif-ferent scenarios. The application being network and CPU intensive, both CPU andnetwork affect the overall performance. In average, small Amazon EC2 instancespresent a performance four times smaller than the one using standard cluster ofGrid5000. High-CPU Extra Large instances present a better CPU performance thanGrid5000 machines, but a slower network interconnection which results in a com-parable global performance. A mix of Grid5000 resources and Small Amazon EC2does not perform well compared to single-site execution over Grid5000 because,even with the balance of load by the partitioner, processes running on EC2 pre-sented a worse performance, which impacted the overall application performance.Using both Grid5000 resources and Extra Large EC2 instances has proved to bemore advantageous, presenting, in average, only 15% of overhead for such inter-domain execution compared to the average of the best single domain ones, mainlybecause of high-latency communication and message tunneling, but this overheadcould be further softened because of the possibility to add extra resources to/fromthe Grid/Cloud.

From a cost-performance point of view, previous performance evaluations ofAmazon EC2 [10] showed that MFlops obtained per dollar spent decreases exponen-tially with increasing computing cores and correspondingly, the cost for solving alinear system increases exponentially with the problem size. Our results indicate thesame when using Cloud resources exclusively. Mixing resources, however, seems tobe more feasible since a trade-off between performance and cost can be reached bythe inclusion of in-house resources in computation.


5 Dynamic Mixing of Clusters, Grids and Clouds

As we have seen, mixing Cloud and private resources can provide performance closeto those of a larger private cluster. However, doing it in a static way can lead to awaste of resources if an application does not need the computing power during itscomplete lifetime. We will now present a tool which enables the dynamic use ofCloud resources.

5.1 The ProActive Resource Manager

The ProActive Resource Manager is a software for resource aggregation across thenetwork, itself developed as a ProActive application. It delivers compute units repre-sented by ProActive nodes (Java Virtual Machines running the ProActive Runtime)to a scheduler which is in charge of handling a task flow and distribute tasks onaccessible resources. Thanks to the deployment framework presented in the Section3.2, it can retrieve computing nodes using different standards such as SSH, LSF,OAR, gLite, EC2, etc. Its main functions are:

• Deployment, acquisition and release of ProActive nodes to/from an underlyinginfrastructure.

• Supplying ProActive Scheduler with nodes for tasks executions, based on re-quired properties.

• Maintaining and monitoring the list of resources and managing their states.

Resource Manager is aimed to abstract the nature of a dynamic infrastructureand simplify effective utilization of multiple computing resources, enabling theirexploitation from different providers within a single application. In order to achievethis goal the Resource Manager is split into multiple components.

The core is responsible for handling all requests from clients and delegating themto other components. Once the request of getting new nodes for computation is re-ceived, the core redirects it to a ”selection manager”. This component finds appro-priate nodes in a pool of available nodes based on criteria provided by clients, suchas a specific architecture or a specific library.

The pool of nodes is formed by one or several node aggregators. Each aggregator(node source) is in charge of nodes acquisition, deployment and monitoring from adedicated infrastructure. It also has a policy defining conditions and rules of acquir-ing/releasing nodes. For example, a time slot policy will acquire nodes only for aspecified period of time.

All platforms supported by GCMD are automatically supported by the ProActiveResource Manager. When exploiting an existing infrastructure the Resource Man-ager takes into account the fact that it could be utilized by local users and providesfair resource utilization. For instance Microsoft Compute Cluster Server has its ownscheduler and the ProActive deployment process has to go through it instead of


contacting cluster nodes directly. This behavior makes possible the coexistence ofProActive Resource Manager with others without breaking their integrity.

As we mentioned before the node source is a combination of infrastructure and apolicy representing a set of rules driving the deployment/releasing process. Amongseveral such predefined policies, two have to be mentioned. The first one addressa common scenario when resources are available for a limited time. The other oneis balancing policy - the policy which holds the number of nodes depending onuser needs. One of such balancing policy implemented in the Proactive ResourceManager, which acquires new nodes dynamically when the scheduler is overloadedand releases them as soon as there is no more activity in the scheduler.

Using node sources as building blocks helps to describe all resources at your dis-posal and the way they are used. Pluggable and extensible policies and infrastruc-tures make possible to define any kind of dynamic resources aggregation scenarios.One of such scenario is Cloud bursting.

5.2 Cloud bursting : managing spike demand

Companies or research institutes can have a private cluster or use Grids to performtheir daily computations. However, the provisioning of these resources is often donebased on average usage, for cost management reasons. When a sudden increase incomputation arises, it is possible to offload some of them to a Cloud. This is oftenreferred to as Cloud bursting.

Amazon EC2

Local network

EC2 computing instances

ProActiveScheduler &

Resource Manager

Local computing nodesINTERNET

HTTP Connections

RMI Connections

Firewalls

Fig. 7 Example of a Cloud bursting scenario

The Figure 7 illustrates a Cloud bursting scenario. In our example, we have anexisting local network which is composed of a ProActive Scheduler with a ResourceManager. This resource manager handles computing resources such as desktop ma-chines or clusters. In our figure, these are referenced as local computing nodes.


A common kind of application for a scheduler is a bag of independent tasks (nocommunication between the tasks). The scheduler will retrieve through the resourcemanager a set of free computing nodes to run pending tasks. This local network isprotected from Internet with a firewall which filters connections.

When the scheduler experiences an uncommon load, the resource manager canacquire new computing nodes from the Amazon EC2 Cloud. This decision is basedon a scheduling loading policy, which takes into account the current load of thescheduler and the Service Level Agreement provided. These parameters are directlyset in the resource manager administrator interface. However, when offloading tasksto the Cloud, we have to pay attention to the boot delay which implies a waitingtime of few minutes between a node request and its availability for the scheduler.

The Resource Manager is capable to bypass firewalls and private networks byany of the approaches presented in the Section 3.3.3.

5.3 Cloud seeding : dealing with heterogeneous hardware andprivate data

In some cases, a distributed application, composed of dependent tasks, can performa vast majority of its tasks on a Cloud, while running some of them on a local infras-tructure in order to have access to computing resources with a specific configurationsuch as a GPU equipment. This type of requirement can also happen in cases wheresome tasks use a software protected by a license, use some private data or algorithmswhich should never get out to an untrusted environment as a Cloud.

Cloud Seeding aims at providing a solution to address such problems by extend-ing the capabilities of a Cloud with specific external resources.

Amazon EC2 network Customer network

Seed subnet

Special nodes

VPN GatewayINTERNET

EC2 compute instances

ProActiveScheduler &

Resource Manager

VPN Connection

Firewalls

Fig. 8 Example of a simple Cloud seeding scenario

Figure 8 shows a simple Cloud seeding example. In this scenario, most parts ofthe application is hosted in the Cloud. The ProActive Scheduler and the ResourceManager are hosted on an Amazon EC2 instance, as well as the computing nodeswhich are used by the scheduler. However, some tasks need a particular resourceconfiguration to be executed, such as a GPU processor. The resource manager can


handle, in addition to the Amazon EC2 nodes, a set of special nodes from the cus-tomer network gathered in a seed subnet.

As we saw in the section 3.3, multiple technical solutions can be used to buildsuch configuration. In our example, we used a VPN based solution. To enable thescheduler and the resource manager to communicate with these special nodes, wegather them in a seed subnet which hosts a VPN gateway and connect the schedulerand the resource manager to this VPN gateway. However, this type of configurationdoes not allow Amazon EC2 instances to communicate directly with these specialnodes. If we want to permit such communication, one solution is for each AmazonEC2 instance to create a VPN connection with the VPN gateway. Another solution isto build an Amazon VPC, as described in Section 3.3.2, to connect the seed and theVPC subnets together, thus creating a virtual network authorizing communicationbetween any nodes in this network.

6 Conclusion

In this paper we have evaluated the benefits of Cloud computing for scientific ap-plications. Although the performance can be similar to a dedicated cluster for com-putational intensive code, it drops when running communication intensive one. Thisobservation motivates the need for mixing Cloud and other traditional computingplatforms. Hybrid platforms requires mechanisms adapted to gather resources, de-ploy applications and ensure efficient communications.

Although low-level solution such as VPN may be useful, these solutions are lim-ited because they hide the heterogeneity and prevent the adaptation of the deploy-ment and applications to the environment. Using ProActive and the ProActive/GCMdeployment allows to specify how resources are acquired and how the communi-cation should be performed (through simple protocols or message tunneling andforwarding).

We have shown that these mechanisms are powerful enough to build a modularGrid/Cloud middleware to support scientific domain-decomposition applications,and illustrate it through the adaptation of a complex communication and networkintensive HPC application to efficiently run over a mix of Cloud resources and ded-icated ones, without much of overhead. Finally, we have also shown how ProActiveResource Manager enables to dynamically mix Cloud and Grid platforms, allowingboth Cloud bursting and Cloud seeding within a single framework. These mecha-nisms also offer a solution to smoothly migrate applications from clusters and Gridsto Clouds.

Acknowledgements Experiments presented in this paper were carried out using the Grid’5000experimental testbed, being developed under the INRIA ALADDIN development action with sup-port from CNRS, RENATER and several French Universities as well as other funding bodies (seehttps://www.grid5000.fr). The authors would also thank Amazon Web Services and the PacaGridCPER for providing computing resources.


References

1. Nimbus toolkit, http://workspace.globus.org/.2. Opennebula project, http://www.opennebula.org/.3. Proactive parallel suite, http://proactive.inria.fr.4. F. Baude, D. Caromel, L. Mestre, F. Huet, and J. Vayssiere. Interactive and descriptor-based

deployment of object-oriented grid applications. Proceedings of the 11th IEEE InternationalSymposium on High Performance Distributed Computing, 2002.

5. M. Bernacki, S. Lanteri, and S. Piperno. Time-domain parallel simulation of heterogeneouswave propagation on unstructured grids using explicit non-diffusive, discontinuous Galerkinmethods. J. Comp. Acoustics, 14(1):57–81, 2006.

6. enStratus. The enstratus framework for amazon ec2, http://www.enstratus.com/.7. E. Grid. Elastic grid, http://www.elastic-grid.com/.8. GridGain. Grid grain: The open cloud platform, http://www.gridgain.com/.9. E. Mathias, V. Cave, S. Lanteri, and F. Baude. Grid-enabling spmd applications through

hierarchical partitioning and a component-based runtime. In Euro-Par ’09: Proceedings ofthe 15th International Euro-Par Conference on Parallel Processing, pages 691–703, Berlin,Heidelberg, 2009. Springer-Verlag.

10. J. Napper and P. Bientinesi. Can cloud computing reach the top500? In UCHPC-MAW ’09:Proceedings of the combined workshops on UnConventional high performance computingworkshop plus memory access workshop, pages 17–20, New York, NY, USA, 2009. ACM.

11. Scalr. The scalr framework for cloud computing, http://www.scalr.net/.12. Vertebra. Easy to manage framework for orchestrating complex processes in the cloud,

http://www.engineyard.com/vertebra.

An Efficient Framework for Running Applications on Clusters, Grids, and Clouds

Documents

Transcript of An Efficient Framework for Running Applications on Clusters, Grids, and Clouds