Perspectives on Grid ComputingUwe Schwiegelshohn 1 Rosa M. Badia 21 Marian Bubak 6 Marco Danelutto 7

Schahram Dustdar 8 Fabrizio Gagliardi 9 Alfred Geiger 10 Ladislav Hluchy 11

Dieter Kranzlmüller 12 Erwin Laure 13 Thierry Priol 14 Alexander Reinefeld 2

Michael Resch 18 Andreas Reuter 15 Otto Rienhoff 16 Thomas Rüter 20 Peter Sloot 17

Domenico Talia 3 Klaus Ullmann 19 Ramin Yahyapour 4 Gabriele von Voigt 5

Abstract

Grid computing has been the subject of many large national and international IT projects in the past. However,it seems that the general expectations in Grids have not come true. Instead in the view of many observers, the Gridconcept is on the way to be replaced by Cloud computing and other as-a-Service approaches. In this paper, we tryto analyze the current situation and to describe potential obstacles for the future development of Grids. Althoughpointing out shortcomings of the past we state that the concept of Grids is still valid and includes new developmentslike Cloud computing and virtualization. Further, we believe that future applications need this concept and that thisconcept in turn needs more research for its reliable, efficient and easy to use implementation.

Key words: Grid computing, Grid applications, Grid middleware, Provisioning of Grid services

1 TU Dortmund University, corresponding author2 Zuse-Institute Berlin3 ICAR-CNR and University of Calabria4 TU Dortmund University5 Leibniz Universität Hannover6 AGH University of Science and Technology Krakow PL,and Universiteit van Amsterdam NL7 University of Pisa8 Technical University of Vienna9 Microsoft Research10T-Systems SfR11Slovak Academy of Sciences Bratislava12Ludwig-Maximilians-Universität München13Royal Institute of Technology (KTH) Stockholm14 INRIA Rennes15EML Research and TU Kaiserslautern16Georg-August-University Göttingen17University of Amsterdam18HPC Center Stuttgart (HLRS)19DFN, Germany20 IBM, Germany21CSIC and BSC, Barcelona

1. Introduction

For more than ten years Grid computing has beenpromoted as the global computing infrastructure ofthe future [25,6], and even Jeremy Rifkin consideredit as one of sources of impact of scientific and tech-nological changes on the economy and society [39].From a user’s perspective, this claim is based on theobservation that large scale simulations and usageof large data bases are spreading fast to many disci-plines, from natural sciences via engineering even tothe humanities. Therefore, the use of large IT sys-tems has reached areas whose members have tradi-tionally little experience in administrating and man-aging these systems. To the users in these areas,the concept of a computing infrastructure similar tothe electrical power infrastructure is particularly ap-pealing. From an operator point of view, Grid com-puting is of interest as few large compute centers areoperating more efficiently than many smaller dis-

Preprint submitted to Elsevier October 24, 2009

Dagstuhl Seminar Proceedings 09082 Perspectives Workshop: The Future of Grid Computing http://drops.dagstuhl.de/opus/volltexte/2009/2225

tributed installations providing the same amount intotal hardware. The IT infrastructure shares thisproperty with the generation of electrical power thatis mainly based on large power plants in addition tosome smaller distributed power producers exploit-ing wind and solar energy.

This potential of Grid computing has convincedmany countries to provide significant investmentsinto the generation of a Grid infrastructure duringthe past years, like, for instance, the D-Grid ini-tiative in Germany [44,27], Grid’5000 in France [9],DAS in The Netherlands [5], PL-Grid in Poland[36], NAREGI in Japan [34] or Open Science Gridin USA [32]. Undoubtedly, some specific variantsof Grid computing, like Enterprise Grids [19], havereached production status. The same is true for largescale Grids spanning many different administrationdomains in communities that require huge amountsof data and use a hierarchical storage concept, likethe IT infrastructure for particle physics [30,7]. Thenumber of users is still small compared to the fore-casts often publicly announced in project descrip-tions and vision publications.

Grid is not only an infrastructure, first of all itis a paradigm of sharing, selecting, and aggregatingof large scale geographically distributed and (pos-sibly) virtualised resources (people, software, data,computers, experiment equipments) depending onavailability, capability, cost, QoS requirements forsolving large-scale problems by a new type of users:virtual organizations [25].

The authors of this paper feel that it is time toreflect on the foundations of Grid computing in or-der to determine the most promising directions ofGrid research. As Grid computing touches many ar-eas of computer science there is a large number ofcomputer scientists who have addressed many facetsof Grids and produced many more or less Grid re-lated results. As in the past, this uncoordinated typeof research is likely to produce good results in thelong run as most niches are eventually covered byexperts in the corresponding fields. However, it maytake a long time to achieve a solution for a complexsystem like an efficiently working production Gridwith many interacting services. Therefore, the au-thors met in a Dagstuhl perspectives workshop inFebruary 2009 to discuss the current situation andthe likely future evolution of Grid computing. Tothis end, we took into account the view of partici-pants from academia and industry and tried to iden-tify open issues and possible research directions.

The paper is organized as follows: Section 2 pro-

vides a comparison between the evolution of the In-ternet and Grids as Grids are sometimes regardedas the Future Generation Internet and the Internetis most likely the biggest IT success story in recenthistory. Section 3 presents current and future ap-plications that are most likely to benefit from Gridcomputing. Next, Section 4 discusses the most suit-able organization of a Grid infrastructure, while Sec-tion 5 looks closer on the middleware stack and in-teroperability issues. The potential of Grids for thecooperation between industry and academia is ad-dressed in Section 6. Section 7 considers provisioningaspects by academic service and resource providers.Finally, we conclude with a summary and outlookin Section 8.

2. Lessons to Be Learned from the Internet

There are several reasons for selecting the nameFuture Generation Internet for the Grid computing.On the one hand, Grid technology requires the avail-ability of networks. On the other hand, the Inter-net has become synonymous for information shar-ing while the sharing of data, resources, and servicesis one of the pillars of Grids. This connection sug-gests a strong relationship between the Internet andGrids. Moreover, Internet technology has becomeindispensable for academic research and for manycommercial activities. Grid protagonists expect asimilar development for Grid technology. Hence, bylooking at the historical development of the Internettechnology, we may be able to identify key develop-ments with respect to technology and organization.But it is also important to determine possible dif-ferences between the Internet and Grids in order toprevent wrong expectations. Altogether such a com-parison may help us to forecast the future develop-ment of Grid computing.

2.1. Initial Situation

The initial situations differ significantly: The In-ternet started nearly from scratch while Grid tech-nology is based on substantial experiences in dis-tributed computing. Due to the lack of previous ex-periences, some design decisions during the devel-opment of the Internet technology proved to be un-suitable and had to be overcome. In such cases itwas always important to replace the older technol-ogy step-by-step with new more suitable develop-ments and not to wait for the perfect solution. De-

2

spite the difference in the starting situations, we feelthat a similar approach is useful for developing Gridtechnology as well as old distributed computing so-lutions may not always be appropriate for the Grid.To this end, we first need to identify the key proper-ties and requirements of Grid technology and theirfuture potentials. Then it may be appropriate toenhance existing technologies for networks and dis-tributed computing with components implementingthese properties and requirements.

2.2. Social Influences

Most users enthusiastically accepted Internettechnology as part of the newly evolving IT tech-nology although there were some reservations intraditional IT centers. Therefore, these users triedto overcome initial problems instead of dismissingthe technology. Possibly due to the different initialsituation, no similar enthusiasm can be observedfor Grid computing. Therefore, it is important toconvince both the Grid providers and Grid users ofthe usefulness of Grid technology by giving themexamples of applications where Grid technologyhelps them to solve their problems.

2.3. Potential for Solution

Very soon after its start, many scientists wel-comed the Internet as a new technology with a hugepotential for many different scientific disciplines.Later the web repeated the same process on an evenlarger scale. Although right at the start, these po-tentials were only identified by so called visionariesand often looked unrealistic when being formulated,most of them sooner or later became reality. We be-lieve that these visions are an important guidelinefor each new technology. Therefore, it is necessaryto formulate visions related to applications usingGrid technology and its usefulness in order to definea goal for Grid development and research.

2.4. Infrastructure

One of the key success factors of the Internet is itsstable infrastructure that provides a reliable base fornumerous Internet services today. This infrastruc-ture provides a vehicle for service delivery which,from a user’s point of view, is reliable, easy to accessand through appropriate market developments rela-tively cheap. As Grid technology is also supposed to

perform an intermediate delivery function for appli-cations the Grid infrastructure must also meet thedemands reliable, easy to access and relativelycheap. Unfortunately, the Grid infrastructures oftoday do not yet satisfy these requirements. There-fore, Grid research and development face the primechallenge of reaching this goal in our view.

2.5. The Use of Standards

Internet technology is based on (quasi-)standardson all technical levels. For the lower (physical) level,like the Ethernet, standardized specifications havecleared the way to mass products that are both re-liable and cheap. On higher levels, standards arenecessary to achieve interoperability with the helpof professionally produced and maintained middle-ware. Although Grid middleware is likely to be morecomplex than Internet middleware, the observationof standards will be necessary as well. Hence, we be-lieve that it is important to leverage the standardiza-tion process for Grid subsystems and use standardsin future Grid applications in the most flexible way.

2.6. Scaling

While it has been demonstrated that Internettechnology scales with respect to the number ofusers, available bandwidth and others, the actualusage numbers cannot support the same claim forGrid technology. Therefore, it is crucial to furtheraddress this issue especially regarding importantproperties of the infrastructure, like stability andreliability for users.

2.7. Sharing

The Internet is a shared resource. This sharing isexecuted on the technical level and usually invisibleto a user in modern networks. Although the Gridstarted with a resource sharing paradigm it turnedout that boundlessness of this feature is not desiredby everyone: Most Grid users only want to sharetheir resources, like data and compute power, witha well defined and restricted set of users. We be-lieve that it is important to modify the old sharingparadigm to concepts that support well-defined col-laborations on all scales.

3

3. Future Applications Using Grids

The acceptance of the Internet is based on themultitude of applications using the Internet. Simi-larly, any successful development of the Grid on alarge (economic) scale requires key applications thatbenefit from the Grid. We believe that these appli-cations will be related to modeling and simulationof technical systems and real world processes.

Modeling of these systems and processes for sci-entific purposes started in the 1960-ties with rathersimple mathematical tools. During the next twodecades, this approach became known under thelabel cybernetics. Methodological shortcomings,limited computer resources and underestimation ofcomplexity were the key reasons for the failure ofthe cybernetics approach at that time. However,the idea to model complex systems revived with theadvent of High Performance Computing tools andadvanced IT resources. Meanwhile, after 10 yearsof experience in combining natural sciences andmodeling problems with the help of IT, it becomesclearer how things will progress and what we canexpect from the future.

Recent advances in experimental techniques suchas detectors, sensors, and scanners have opened upnew windows into physical and biological processeson many levels of detail. The resulting data ex-plosion requires sophisticated techniques, like gridcomputing and collaborative virtual laboratories,to register, transport, store, manipulate, and sharethe data. The complete cascade from the individualcomponents to the fully integrated multi-sciencesystems crosses many orders of magnitude in tem-poral and spatial scales. The challenge is to studynot only the fundamental processes on all theseseparate scales, but also their mutual couplingthrough the scales in the overall system, and theresulting emergent properties. These complex sys-tems display endless signatures of order, disorder,self-organization and self-annihilation. Understand-ing, quantifying and handling this complexity isone of the biggest scientific challenges of our time[47,49,28] requiring more advanced, collaborativeproblem solving environments based on workflowsystems [54] and virtual laboratories [10].

A prototypical example comes from biomedicine,where we have data from virtually all levels betweenmolecule and man and yet we have no models wherewe can study these processes as a whole. It is a realcomplex system: from a biological cell, made of thou-

sands of different molecules that work together, tobillions of cells that build our tissue, organs and im-mune system, to our society, six billion unique inter-acting individuals. The complete cascade from thegenome, proteome, metabolome, physiome to healthconstitutes multi-scale, multi-science systems, andcrosses many orders of magnitude in temporal andspatial scales [48].

Another example comes from flood prediction:sensor data from dikes and up-to-date river flow pat-terns combined with information on the state of thelocks and bridges need to be integrated with satel-lite data and weather models to predict floodingand give decision support to local governments. Forthis we need to build time critical numerical mod-els that can simulate what-if scenarios, where hy-draulic, flow and weather prediction models are in-tegrated.

The sheer complexity and range of spatial andtemporal scales defies any existing numerical modeland computational capacity. The only way out isby combining data on all levels of detail with, forinstance, large scale particle-based, stochastic andcontinuous models; an open research area. Thechallenges include understanding how one can re-construct multi-level systems and their dynamicsthrough computational simulation within virtuallaboratories that connect models to massive sets ofheterogeneous and often incomplete data. Concep-tual, theoretical and methodological foundationsare necessary in understanding these multi-scaleprocesses, dynamic networks, and the associatedpredictability limits of such large scale computersimulations.

3.1. Computational Science

The modeling world is governed by parallel com-puting and networking of resources: It is not a co-incidence that the fastest computer for some yearswas called Earth Simulator [42]. Now, newer mod-els include an increasing amount of data from dif-ferent sources, type and quality. Sensors provide uswith a multitude of data from deep space to hu-man implants and an important research topic issharing of data in virtual organizations [43,40]. Themanagement of data quality, the ontologies of thedescriptors, and the handling of errors in the datahave emerged to important prerequisites for validresearch processes using these data. The longtermstorage of data sources and their availability are the

4

mathematics

physiscs

chemistrybiomedicine

earth sciences(multiscale)

economy

social sciences

increasing impact on society and the individuum

1960s 2020stime

com

plex

ity

Figure 1. Schematic representation of the anticipated devel-opment of simulation techniques in society.

precondition for transparency of and trust in thethese processes. A prime example of this researchprocess is the combination of analysis in the biomed-ical sciences with phenotype data including biosig-nals, see Fig. 1. This approach began a few years agoand will need extensive IT resources that can be pro-vided by Grid computing during the next decades.

The additional inclusion of environmental data isan important next step. However, it will substan-tially add to the complexity of the model and istherefore not likely to start in the immediate future.Eventually the simulation will not any more rep-resent a mere statistical knowledge but provide anindividually related knowledge posing all data pro-tection and ethical questions already anticipated inthe literature at the end of the first phase of scien-tific modeling. If we advance further in complexitywe will try to model more and more factors of ourenvironment and try to predict the future by calcu-lating virtual worlds. However, validation of resultswill become increasingly difficult as we will have tocontrol a very large amount of variables.

An example where computational eScience pro-vides new ways to understand complex systemscomes from the European ViroLab project [52],where novel distributed computing techniques arecombined with novel computational methods inorder to provide medical doctors with a decisionsupport system for drug ranking in [46,45].

3.2. The Virtual World in the Grid

It is always difficult to predict a key future appli-cation well in advance. But we know that such an

application must be of vital importance to a largepart of the world population in order to play a dom-inant role in a networked world. Looking at the webapplications of today, we observe that social net-works like Facebook 22 [1] or MySpace 23 [51] areparticularly appealing because of the aggregated in-formation provided by links which build complexnetwork structures that reveal previously unknownrelationships between nodes representing people ororganizations. Other popular services, like Earth 24 ,mainly exploit one dimension: they serve a singlepurpose such as finding a restaurant or the short-est route from A to B. Their usefulness increasesby adding further spatial and temporal informationfrom sensors resulting in a multidimensional dataspace that allows us to address various real worldproblems. For instance, geographic coordinates frommobile phones may allow to trace user or vehiclemovements and thereby to draw maps of social be-havior and contacts or to predict traffic jams. An-other striking example is the recent tracing of flue ac-tivities by analyzing the keywords typed into searchengines by users.

This example shows the value of linking seeminglyunrelated spatial or temporal data. In general, thebenefit of data bases increases exponentially withthe amount and dimensionality of the available data.Based on these data, sophisticated algorithms [29]will provide various services, up to a one-to-one map-ping of real world phenomena into a virtual worldenvironment. However, this is only possible with anIT infrastructure that has the ability to gather datafrom various sources, to analyze that data in realtime and to deliver the results on demand. It is fur-ther worth noticing that Grid computing alreadystarted to be applied in art and humanities [8].

3.3. Impact on Society

The Grid infrastructure and the Grid paradigmare the best candidates to support a new approach toscientific investigations known as Open Science andScience 2.0. As science is becoming more and morea social process, the new, web-based social tools arebeing used to facilitate and accelerate the scientificdiscovery processs [53]. In Open Science the data, re-sults, methods and software tools are freely availableenabling massively distributed collaboration. myEx-

22http://www.facebook.com23http://www.myspace.com24http://earth.google.com

5

virtual world"world in the grid" real world

mathematical model impact on real world

?

spatio/temporalsensors & monitors

Figure 2. Schematic representation of a socially controlledfeed back circle.

periment (http://myexperiment.org) [41], originallydeveloped to support workflow applications on Gridsystems is considered as an example of Science 2.0tools.

We need to be aware that these applications mayalso create new problems: many instruments devel-oped for scientific studies can also be used to in-terfere with our everyday world - as harmless helpto solve professional and private problems but alsoas potential threat to the pillars of human rights.The development therefore needs to be pursued ina socially controlled feed back circle as presented inFig. 2.

There, the circles represent states of complexityin modeling. The more powerful the simulation ap-proaches become the more necessary it is to coun-terbalance potential misuse of the tools by societalcontrol and transparency of validity and reliabil-ity. We expect this task to be much more demand-ing than the attempt to keep transparency on thetrustfulness of Internet sites. On the positive side,we expect that the novel way in which Grid-like in-frastructures allow for the integration of a plethoraof data coming from databases sensors and instru-ments, with a multitude of computational models,will open up new ways to decision support in an in-creasingly complex world.

4. Layer Structure of Grids

The Grid will only succeed on a broad scale if itenables a straight forward execution of the key ap-plications discussed in Section 3. Unfortunately, to-day’s Grid middleware stacks do not generally sup-port an easy transfer of these applications to theGrid as they are not able to achieve sufficient trans-parency. In Grids, transparency mainly denotes thehiding of the heterogeneity of resources and servicesfrom the user. Unfortunately, the attempt to address

the heterogeneity of resources and applications ina broad way has produced large and complex mid-dleware stacks that often exhibit a high operationalcomplexity as well. This complexity then leads tothe described lack of transparency. There are twomain approaches to address this problem: On theone hand, we can strive for less complex middle-ware with reduced functionality, see Section 5. Onthe other hand, we can try to handle this complexityby enforcing strict guide lines during developmentand operation. One way of establishing these guidelines is the use of architectures that inherently sup-port them, like layer structures that are found innetwork protocols and other software architectures.In order to determine whether layer structures aresuitable for Grids as well, we examine their advan-tages and disadvantages with respect to the futuredevelopment of the Grid.

4.1. Reasons for Layer Structures

In general, layered structures– restrict the number of control flows,– do not support the use of functionalities across

layers,– guarantee the highest performance,– enforce the definition of stable interfaces,– support an intuitive mapping of responsibilities

to layer functions.A pure layer structure only allows a single flow

of control, that is, each object passes through eachlayer exactly once in order to reach the exit of the ar-chitecture. During such a pass, the layer may modifythe object, for instance, by adding or removing at-tributes to the object, or leave the object unchanged.This concept supports tracing and therefore an easydetection and localization of errors while in an ar-chitecture that allows general control flows typicallyexpressed with the help of a directed graph, an erro-neous control flow may also be the cause of an error.

However, in a pure layer structure each layer onlycommunicates via interfaces with its neighboringlayers. Therefore, components of a layer can only usethe functionality defined within the layer while thedefinition of components that are accessed acrossdifferent layers is not allowed. This may lead to cum-bersome code and duplication of functionalities withpotential coherence problems. Therefore, many ex-isting software architectures use some form of layerstructure but also allow cross sectional functional-ity (like e.g. CORBA and its services). However, the

6

flow of control is not unique anymore.As the control flow passes through all layers in

a pure layer structure some processes suffer perfor-mance losses, if the process does not require the ser-vices provided by a layer. Nevertheless, the controlflow of the process must enter and leave this layer.In order to avoid some of these performance lossesit is common to implement bypassing in real layerstructures, see, for instance, the TCP/IP protocolstack.

In any software system, interfaces must be care-fully maintained. In general we can distinguishbetween published and internal unpublished inter-faces. It may be appropriate to modify componentsof a system to improve, for instance, performanceor manageability. Such modification may includeunpublished interfaces. The group controlling thespecific implementation of this part of the architec-ture carries the responsibility for the modification.However, the modification of a published inter-face can possibly affect components produced andmaintained by other institutions. It may also indi-rectly influence the whole system if, for instance,old attributes are no longer provided. As such mod-ification may hence have severe consequences, theseinterfaces are often considered to be fixed pointsof a system. As a large number of these fix pointsreduces the flexibility of a system with respect toimprovement and adaptation to new requirements,we usually try to minimize them. However, fewpublished interfaces lead to large sets of compo-nents and prevent the modularity of the system.During the design of a software system, it is there-fore very important to find an appropriate compro-mise between these conflicting demands and selectan appropriate number of published interfaces. Alayer structure makes this selection explicit as itautomatically distinguishes between published in-terfaces connecting neighboring layers and internalunpublished interfaces between components of thesame layer.

In large systems, it is very difficult to find design-ers and administrators that overlook the whole sys-tem. Typically, teams are formed with different re-sponsibilities according to their abilities. In a layerstructure, the interaction of each team is mainly tar-geted towards the teams responsible for the neigh-boring layers. This approach is beneficial for devel-opment and administration, particularly, if the dif-ferent teams do not belong to the same organizationas the interaction across organizational boundariesis often impeded by conflicting policies. Note that

these properties apply to both technical and orga-nizational layer structures.

4.2. Layers and Grids

In general, Grids abstract physical resources andIT-services such that the allocation of compute jobsto resources and services as well as the execution ofthese jobs are supposed to be primarily automaticprocesses. Due to the heterogeneity of resources anapplications, access rights, number of resources, andother properties of Grids, these processes are rathercomplex even without considering additional userrequests regarding specific interactions with the re-sources during the processes. Functions of these pro-cesses may share some middleware components andservices resulting in cyclic control flows. Therefore,an efficient implementation of a Grid middlewarewith a pure layer structure seems to be impossible.But, there are often natural precedence constraintsfor many of these management functions. These con-straints naturally lead to a partial order suggestingsome form of layer structure in parts of the archi-tecture, see the following example:

In a simplified form of Grid job submission, thefirst layer may be responsible for authentication asstated above, that is, it adds credentials to the Gridobject representing a job request and then forwards itto the next layer that handles VO membership infor-mation. The following layer may look at the charac-teristics of the data request in this object and initiatea data query before the resource coordination in thefourth layer would steer the request to the appropri-ate Grid computing resources.

Remember that contrary to web services, Grid ob-jects are generally state full and capable of carryingattributes allowing the objects to be forwarded andexecuted on different entities if necessary. Moreover,it is possible to speed up the process of this exampleby parallelization. But, as such parallelization willresult in a significant increase of complexity it is ap-propriate to only address such methods for perfor-mance enhancing once the system is mature.

Sometimes the requirement of only a single in-terface between two layers is considered artificialand disadvantageous if a layer contains many com-ponents. But it can also be beneficial with respectto upgrade and modification flexibility to define asingle interface that is observed by all componentsin the layer even if this approach results in addi-tional complexity. Consider a Grid comprising dif-

7

ferent machines with different operating systems.There the use of virtualization techniques may al-low the user to provide a single job package thatruns on all machines although the performance maybe reduced. Of course, it is the responsibility of theresource provider to support an appropriate virtualmachine on his resource. We consider this virtualmachine as some form of interface. Of course, anypathological case simply combining several disjointinterfaces into a single one should be avoided.

As already mentioned, a layer structure is not re-stricted to technical issues but may be even more im-portant in the organizational domain as in Grid de-velopment and operation, there are people involvedwith different background and different abilities:

(i) Users know the details of the specific problemthey want to solve and at least how to applytheir appropriate software tools.

(ii) Software community administrators know howto manage the mentioned software tools andsupport them with a run time environmentand means to access data and compute re-sources.

(iii) Grid operation administrators know how tohandle and maintain basic Grid services.

(iv) Resources administrators or service develop-ers know the details and requirements of theirresources or services.

Due to the heterogeneity of resources, servicesand applications, people involved in Grid comput-ing usually belong to different organizations or loca-tions. Therefore, interaction between the membersof these groups may be sometimes tedious and errorprone. A layer structure can facilitate the interac-tion and communication within a Grid if it considersthese different organizations and locations.

An example of a form of organizational layerstructure is given in Fig. 3: The top layer containsthe users’ applications that are supported by ser-vice layer which provides mechanisms that may beunique for a given group of users. As these mecha-nisms cannot be shared it makes little sense to shiftthem to the next more general layer. The next layersincludes all services that are common to all Grids orat least to a majority of Grids. In this way synergiesbetween different Grids and different resources canbe exploited. The bottom layer contains resources.

Note that this structure is a simple blueprint fora Grid while a real Grid infrastructure needs someextensions. For instance, there may be community-specific services that form layers within a commu-nity. These services may also include specific re-

sources that are accessed with the help of specialinterfaces not controlled by the third layer of ourstructure. Nevertheless, we believe that such a layerapproach is a well suited starting point for designingGrid organizations.

4.3. Challenges of Layered Grid Architectures

We have already observed that layers also havedisadvantages. Therefore, we next address the chal-lenges that arise when implementing a layered Gridarchitecture. As already mentioned, a mandatorylayer structure may reduce the performance aseach layer must be traversed although some layersdo not contribute anything to the process in thiscase. If this detrimental effect is restricted to themanagement, like in the job submission process, itmay be easier to accept than if it influences the ac-tual job execution, as in the virtualization case. Inorder to avoid such a performance loss, we may usedifferent layer structures for management and exe-cutions tasks.

Moreover, a layered Grid architecture may sup-port bypassing. However, when selecting a bypassthe user himself is responsible for the functionalityof his approach. For instance, a user may target hisapplication to a specific operating system instead ofusing virtualization. But this may result in a failureof job execution in case of a configuration mistakeor at least reduce the number of available resourcesfor this job and there influence manageability.

But, bypassing layers may have the already men-tioned negative impacts on failure localization:When each layer adds attributes to a Grid objectwe can determine a trace for an object with anerroneous result by looking at all attributes of theobject. This approach requires that all objects aretagged by each layer even if only default values areused. Tracing is also a feature that is beneficial forthe management of Service Level Agreements:If, for instance, each object receives an uniquestamp from each layer containing time informationand an identification key, a Grid management com-ponent can extract these attributes and derive per-formance information for the whole process chainin the Grid. If a job execution involves several ser-vice level agreements with different providers it isthus possible to determine and locate violations. Itis therefore possible that the infrastructure or someproviders may prohibit bypassing due to ensuretracing and failure handling.

8

Application layer* portals* applications clients

Users

Service layer* VO services* allocation management* service orchestration

Middleware layer (gLite, Unicore, ...)* accounting, authorisation, accounting* security* monitoring* information services

Hardware layer* servers (computers) * storage

Figure 3. Prototypical Grid layers organization.

Services and components must be assigned to lay-ers in a layer structure. This is even true for servicesor components that are used by different layers. Atleast for reasons of responsibility they must be as-signed to an organizational layer. This assignmentof components and functionality to layers gen-erally influences the performance of the architec-ture. Further, it is difficult to reverse a decision oncemade. Also components are subject to functionaland non functional requirements. Non functional re-quirements are quite often discovered during thecourse of a project that has the task to define thedetails of the appropriate Grid architecture. Theytypically establish a relationship between organiza-tional and technical layers. Examples for these nonfunctional requirements are:– Flexibility to adapt to different virtual organiza-

tions– Restrictions like access to programs and data– Monitoring and steering of functions or modules– Presentation of data belonging to specific virtual

organizationsTo determine the necessary interfaces in this

model, the application of Use Cases is often a suit-

able approach. Use cases help to analyze in whichorder functions must be executed. At the beginning,this analysis may result in the introduction of newintermediate layers to close gaps in a first simple ar-chitecture and its operational model. However, oncea higher degree of maturity is reached, there may bemore powerful components that span across layers.Then some interfaces may become obsolete result-ing in a modification of the original layer structure.

5. Role of Middleware

Grid middleware is the software layer that con-nects the different parts of a Grid system and thusgenerates a useful environment. As in distributedsystems, the middleware should bridge any trans-parency gaps due to the heterogeneity of Grid re-sources and services.

Middleware for Grid systems may be build basedupon the experience from distributed systems, usingthe following parts [37]:– Communicating processes where we exploit ex-

change of messages (socket, MPI, PVM), sup-ported by APIs,

9

– Remote procedure calls, based on transfer of con-trol from one process to another one, with inter-face definition, supported by programming lan-guage entity (a function),

– Objects with the Distributed Object Oriented Ar-chitecture, which enables a transparent access toobjects within a distributed system, and it is sup-ported by a programming language entity (an ob-ject),

– Components using the Component Oriented Ar-chitecture, with design by composition, configu-ration and deployment, supported with program-ming language entities (extensions of objects),

– Services with the Service Oriented Architecture,which is stateless without any guarantee aboutthe delivery of messages. This approach does notmap to a programming language entity.As a consequence, there are currently multiple

versions of Grid middleware also called middlewarestacks, like gLite [31], Globus Toolkit [24], UNI-CORE [21], ARC [18], and others. These stacks gen-erate a new type of heterogeneity as they providedifferent functionalities and serve different applica-tion domains. Moreover, none of them is generallyaccepted by all user communities. As they do notyet support common interfaces the European Gridinitiative (EGI) intends to initiate such a standard-ization effort. But in their present state, we con-sider these middleware stacks to be still in the stateof experimental software, that tries to solve prob-lems with a best effort approach. For instance, thepresent implementation of failure management doesnot satisfy the common requirements to software ofdistributed systems. It is necessary to develop newmodels and methodologies to handle failures with-out exposing too much of them to the users. We be-lieve that the current state and diversity of Grid mid-dleware implementations are a result of past fund-ing approaches, lack of sufficient basic research, andpartial use of technologies that were not yet matureenough. In order to improve the situation, we needa new approach to design Future Grid middleware.

5.1. Back to the applications

During the last ten years, different implementa-tions of Grid middleware have been tested with var-ious applications. We should use these experiences,analyze these applications and determine how wecan cluster them according to the functionalitiesthey require from a middleware system. This ap-

proach will help to redesign existing Grid middle-ware or perhaps even develop a new Grid middle-ware for a given cluster of applications if necessary.The public expectation of extending the scope ofGrid technologies to a wide spectrum of applicationshas undoubtedly contributed to the present com-plexity of Grid middleware as it has forced devel-opers to integrate new functionalities into existingsoftware systems. This has led to a loss of structurewithin and between the layers. For the future, webelieve that it is necessary to develop a minimal setof Grid middleware components that are based oncommonalities between different applications func-tionalities and still provide a sufficient foundationto support these applications.

In practice, the first step involves the definition ofthe requirements to Grid middleware and its func-tionalities based on the needs of resource providersand Grid users. This definition also includes select-ing appropriate layers for the operation of the Gridmiddleware. The agreed functionality must be splitinto pieces, which are then implemented individuallyfollowing common software engineering principles.This should be supported by standardization effortsto provide well-defined interfaces between the dif-ferent components and especially layers of the Gridarchitecture. There are already a few examples forsuccessful standards, see [38,23,2]. Of course, Gridapplications will evolve forcing Grid middleware todo the same. Therefore, any roadmap of Future Gridmiddleware functionality should allow changing re-quirements over the course of time.

5.2. Experimental and Production Middleware

In the present situation, we must face at the sametime the needs of users for production quality Gridmiddleware and the lack of maturity in existing Gridmiddleware. Therefore, it is necessary to split thedevelopment process of a Future Grid middlewareinto two parts: a research part where computer scien-tists seek the next breakthrough in technology withthe help of prototype implementations, and an engi-neering part that ensures production quality of theGrid middleware used for Grid applications. Unfor-tunately, most funding efforts have been directed to-wards development of production Grids. The miss-ing support for the research part often transformedGrid systems that were supposed to be of productionquality into prototype implementations resulting inuser disappointment. Note that it is merely impos-

10

sible to carry out experiments on a production Gridmiddleware itself without generating threats to thereliable execution of user applications. But these ex-periments are necessary to improve the maturity ofGrid middleware. Only few initiatives have been car-ried out so far to overcome this situation, like DAS inthe Netherlands or the French Grid’5000. As manyresearch topics still need to be addressed, we believethat the research part should be supported concur-rently to the production effort. A close collaborationbetween researchers and engineers is thus requiredto identify problems when operating a productioninfrastructure and to generate advanced solutionsthat have been intensively and rigorously tested us-ing experimental large scale Grid platforms.

5.3. Middleware Interoperability

To describe the present situation of Grid middle-ware, we extend the comparison between the Inter-net and Grids from Section 2: Anyone is capable ofaddressing, but not always accessing for obvious se-curity reasons, any physical nodes or software ser-vices available on the Internet. This is one of the rea-sons for the success of the Internet. It is achieved bya single distributed DNS service that is offered to theoperating systems and middlewares. Now consideran alternative Internet with several distinct and in-compatible DNS systems controlled by various or-ganizations that manage their own set of routers.It is highly unlikely that this alternative approachwould have produced the same success. But unfor-tunately, this scenario of the alternative Internet re-sembles the current Grid where several Grid middle-ware stacks co-exist today managing their own in-frastructures and resources. These stacks constituteisolated silos instead of providing a global pool ofaddressable resources. There is still a lack of inter-operability between existing stacks although severalattempts have been made to investigate this issuein several EU-funded projects like UNIGRID, GRIPor DataTAG [16]. Grid interoperability was also thesubject of an intense collaboration between Euro-pean and US teams managing Grid infrastructuressuch as the Open Science Grid 25 and EGEE 26 . Butmost of these projects only consider interoperabilitybetween no more than two existing Grid middlewarestacks instead of addressing it in a more global way.

25http://www.opensciencegrid.org26http://www.eu-egee.org/

5.3.1. Types of interoperabilityIn general, interoperability is the ability of a sys-

tem to cooperate with or to exploit functions or partsof another system belonging to a different organi-zation. More particular, the Grid gurus website 27

states:Interoperability is the native ability of Grids andGrid technologies to interact directly via commonopen standards.

Other definitions of Grid interoperability have beengiven in [22]. Nevertheless, in a complex system likethe Grid, we believe that it is necessary to distin-guish different types of interoperability. To this end,we have identified three areas that can possibly en-hance effectiveness and reliability of Grid use:– Data interoperability It is widely acknowl-

edged that data plays a key role in Grid appli-cations. But many scientific and business appli-cations are presently facing a common challenge:managing the availability of large distributeddata. On the one hand, detectors, laboratoryinstruments, sensors, and media are repeatedlyproducing amounts of data that exceed the ca-pacities of the available local data storage andcomputing systems. On the other hand, manyGrid applications need to access and share multi-ple data sources that are widely distributed [35].Therefore, data integration and interoperabilityis a basic issue to be solved for supporting a widespread use of Grids.

Due to the variety and heterogeneity of bothapplications and data-related resources, the mid-dleware must support higher-level models andservices that assist users in exploiting several datarepositories within an application. Frequentlythese data repositories have been been createdby different organizations that often reserve theright to make local, independent decisions abouttheir best data management paradigm. Even ifthese organizations do not want to implementinteroperating software Grid users still are likelyto demand that another Grid middleware layerenables them to use existing data, possibly in situto avoid huge and unnecessary data transfers. Inorder to support this data interoperability, com-mon data formats as well as a common metadataformat and structure must be defined.

– Service interoperability While basic Grid ser-vices must be implemented on each Grid this is

27http://gridgurus.typepad.com/grid_gurus/2008/06/grid-interopera.html

11

not necessary true for application services. Nev-ertheless, an application may also be interested ininvoking a higher level service that is only avail-able in another Grid using a different middleware.This service interoperability mainly uses two basicmechanisms: standard interface description possi-bly with proper metadata carrying associated nonfunctional information, and reflection, supportinginspection of the interfaces and of the informationassociated to these interfaces.

– Security and authentication interoperabil-ity The agreement on a common, shared securityinfrastructure is actually a condition sine quanon for any interoperability in Grids. Interoper-ability simply cannot be conceived without anagreement concerning the security procedures.This agreement must preserve all the disjointsecuring and authentication mechanisms of thecooperating Grids.

5.3.2. The need for interoperabilityAlthough there seems to be a consensus forming

in the Grid community about the demand for inter-operability and the types of interoperability, thereare still doubts concerning the real need to fully im-plement Grid interoperability in a secure, reliableand effective way:– Presently, only the high energy physics commu-

nity has already demonstrated the ability to bene-fit from Grid interoperability to execute large setsof independent and highly demanding tasks. Butthere is no evidence of either other communitiesor specific applications urgently requiring Grid in-teroperability. It is true that some applicationsuse services and data from different Grids. Butthis is likely to be implemented at the applicationlevel rather than requiring actual Grid interop-erability [4]. Moreover, even the physics commu-nity seems to be interested in Grid interoperabil-ity mainly as a viable solution to integrate Gridsbuilt at different times, within different local andnon local communities and possibly with different(versions of) Grid middleware to be able to use allthe resources of the different Grids as well as toaccess, combine and compute the data gatheredin these different Grids. This is a more restrictedform of Grid interoperability than the one we ad-dressed above.

– Is it really necessary to implement Grid inter-operability to achieve a joint exploitation of thedata and resources of distinct Grids or can we

adopt some more primitive and lightweight solu-tions such as installing in the target Grid a virtualmachine with a remote execution service allowingusers to submit computations on the Grid nodesas tasks embedded in such a virtual machine? Thelatter case may be simpler to achieve than the for-mer one provided (local) data access is supportedthrough proper interfaces and abstractions.

It is commonly recognized that access to distributed,structured data, possibly owned and/or provided bydifferent organizations is a key point for many futureapplications and requires some kind of Grid interop-erability. But it is not clear whether this distributedand structured data processing should be supportedon the Grid middleware or on the application levelusing the proper mechanisms and services providedby the individual Grid. We should particularly takeinto account that more than twenty years of researcheffort in the business area led to standards that rep-resent metadata and model the rules governing ac-cess to the data. These standards are widely inde-pendent of the application frameworks that are usedto program the business application. Therefore, theyrepresent a viable solution to address data interop-erability.

However, the current situation regarding Grid in-teroperability may resemble the situation existingat the very beginning of the mobile phone era. Atthat time it was quite common that mobile phonesoperated by some provider could not easily commu-nicate with mobile devices within another network.But very soon users started to insist on more inter-operability leading to the present situation with ev-ery mobile phone being able to communicate withany other phone regardless of the provider. We be-lieve that a similar pressure may motivate the devel-opment of real Grid interoperability if, for instance,a new programming model allows programmers todevelop different kinds of massively parallel appli-cations in a seamless and effortless way.

5.3.3. The road to interoperabilityCurrently, the size and the functionality of the ex-

isting middleware stacks may be the main hurdle toachieve interoperability between them. Therefore,it may be necessary to reduce the functionality ofthese stacks. We believe that there are several com-ponents of current Grid middleware stacks that canbe considered for promotion outside of the actualGrid middleware such that the corresponding poli-cies can be accessed by all interacting Grids. Simi-

12

larly, the DNS is something separate from the Inter-net protocol layers but actually all the actors in theInternet scenario agreed to it.

In our view, the components of the following listare promising candidates for such promotion:– VO management,– accounting and billing,– resource and job monitoring.– security, in a first step limited to certificate man-

agement.For instance, in interoperating Grids, the account-ing and billing systems should be run in each indi-vidual Grid by exploiting the primitive accountingand billing facilities of them. These primitives pro-vide the data necessary to implement global, inter-grid accounting and billing activities.

In addition, the migration of some basic mech-anisms in Grid middleware to operating systemsmight be helpful. Promising candidates for this mi-gration are in particular those mechanisms that areconsidered necessary and universal, and whose im-plementation is not subject to further research. Asan example, the XtreemOS project [13] suggests tomigrate authentication mechanisms to the Linux op-erating system level. However, if we combine bothapproaches the Grid middleware may become a verythin layer. But we must realize the different reasonsfor these migrations:– Mechanisms are migrated to the operating system

level mainly for efficiency reasons and to reducesecurity pitfalls while moving up policies into ex-ternal inter-grid entities supports interoperability.

– Transferring a mechanism to the operating sys-tems level requires a general consensus by manu-facturers or communities developing the operat-ing systems while only an agreement by the peopleactually using Grid interoperability is necessaryfor the policy movement.Nevertheless, we believe that both approaches still

leave enough space to implement in the Grid mid-dleware those mechanisms and policies that are nei-ther fully general nor very basic to be completelyindependent of the Grid implementation at hand.In Fig. 4, a layered abstraction of Grid middlewareand infrastructure is presented that actually demon-strates how services are moved into Grid indepen-dent, high level interoperability services components(shift up) while other basic mechanisms are migratedto the OS level (shift down).

For such a Future Grid middleware, we need toidentify those (and only those) standards (or partsof the Grid middleware) that support Grid interop-

erability in order to efficiently implement it. In turn,this subset of middleware mechanisms will repre-sent the minimal API to Grid middleware neededto (inter)operate a Grid with another one of distinctform. This subset can be considered as a kind ofRISC Grid middleware while the remaining servicesand functionalities of the middleware are regardedas features that help providing a better abstractionand a better implementation of that Grid.

Overall this process allows identifying the primi-tive or base features of Grid middleware (RISC APIin Fig. 4) and thus it will help defining a layeredabstraction of the Grid middleware such as the onepresented in Fig. 4. Such a layered abstraction ofGrid middleware and infrastructure will allow todevelop implementations of Grid middleware muchbetter than the monolithic implementations avail-able nowadays. It will also support later modifica-tions of the middleware as addressed in Section 4.

6. Cooperation between Academia andIndustry

The Internet has become indispensable for thecommercial and the academic world. It is anotherreason for the success of the Internet that it did notremain confined to the academic world. A similarsituation applies to the Grid technology as severalaspects of it are commonly applied in industry:– Many large companies have implemented enter-

prise Grids.– An increasing number of companies engage in

Cloud computing.– Software providers offer different software systems

(middleware, tools, and software-as-a-service)that play an important role in Grids.There are also various projects in which compa-

nies offer Grid services to academic users or use Gridservices from academic providers. However in gen-eral, the academic and the industrial Grid world arestill separated: Academic users rely on academic sys-tems while industry mainly sees computing as anin-house activity. But the situation is changing. Al-though changes in industry are perhaps more rapidthan those in the public sector, the Grid discus-sion has also unleashed a wave of change in thefield of public research computing services. On theone hand, the Grid may become an efficient vehiclefor industry to provide academia with new services.Particularly, the Grid may support a new economyof scale to enable service provisioning by industry

13

Services:

Hardware(processing, networking, storage)

Operating System

Virtualization

Mechanisms:

Policies

Application

Candidates to be moved to OS(provided manufacturers agree)

Interoperability services (VO management, accounting,

billing job and resource monitoring, certificate management, ...

Candidates to be promoted outside grid mw(to enforce/simplify/support grid interoperability)

GridInfrastructure

GridMiddleware

commongrid specific

complex, grid specificbasic

Experiments, Portal, ...

Virtual laboratory services

Computational services

Data services

Grid infrastructure

RISC API

Figure 4. Layered abstractions in Grid Infrastructure and Grid Middleware. Migration of common services and and basicmechanisms (right). Mapping of a currently used Grid assets (Virolab, http://www.virolab.org) (left)

in fields that have so far been considered exclusivelyacademic, like high performance computing. On theother hand, the Grid may allow academic centers toexport their services beyond their traditional aca-demic users into the market.

Both worlds are presently facing difficulties dueto the strong and increasing demand for IT re-sources and services. As compute power includingthe necessary data services becomes a commodity,IT managers increasingly subject it to a make-or-buy-analysis. At the same time, academia is underpressure to justify the high expenses for computepower and to find alternative ways of funding. Bothsides may gain from bringing academic and indus-trial resources and needs together.

To analyze the potential of such a collaborationin the Grid domain we can look at some existing andwell established collaborations. In the following, wepresent one example: In 1995, the German State ofBaden-Württemberg, the University of Stuttgart,and the car manufacturers Porsche and Daimlerjointly founded the company hww. It was the pur-pose of hww to bundle resources and demands ofthe academic and industrial partners and to providethe necessary legal and organizational frameworkto leverage expertise and funding. While initiallythe company was mainly concerned with the op-eration of systems it gradually shifted its focus toorganizational issues thus transforming the original

aggregation of computing systems into a produc-tion Grid. Once operational issues were resolved,flexible usage, service level agreements, and ease ofaccess became important. At the time of writing,hww distributes about 50 millions CPU hours peryear and is developing a concept that allows forservice provisioning in the field of simulation anddevelopment cycles.

However, hww is a one-of-a-kind project that wasable to grow in a very special environment and canonly serve as a prototype. It is an open questionwhether Grids can play a key role in bringing for-ward a new general concept of collaboration betweenacademia and industry that is based on mutual ser-vice exchange.

In general, collaboration between public researchand industry is driven by a number of factors. In or-der to determine the prospects of such a collabora-tion, we need to answer the following key questions:– Which present or future needs of industry can be

satisfied efficiently by public resources?– What are the benefits and the impact of Cloud

computing for public research?– Which collaboration model allows public research

and industry to mutually use services?– Which main problems must be overcome to make

such a collaboration work?

14

6.1. Overflow Capacity Argument

It has been stated frequently that industry re-quires substantial computing power that signifi-cantly varies over time, like the server capacity ofonline shops. To solve this problem, industry cansatisfy its basic needs with in-house compute re-sources while external resources may be used ondemand. This scheme works well if the peak demandof different industries is well distributed. Unfortu-nately, this is not always the case, see the increasedaccess to online shop servers before Christmas.Academia may be able to help if– industry is at all interested in outsourcing com-

puting activities and– academia can easily postpone its own computing

needs to provide external access to its systems.Looking at the proliferation of simulation in en-

gineering, it is obvious that there is a true need forsimulation in a large group of industries. Even forsmall and medium sized companies, simulation hasbecome a standard tool. Further, more and morelarge companies are outsourcing development tasksand request that simulation be used to verify designparameters. Therefore, it can be expected that com-panies of all size may have an interest in using Gridresources. However, we do believe that for small andmedium sized companies, Grids may not be predom-inantly used to provide overflow capacity but as anenabling technology that allows these companies tocompete with large enterprises without having to in-vest heavily in computing infrastructure.

In the moment, mainly large companies rely onGrid solutions. But it is the short term and projectoriented planning modus in such companies thatbrings in Grid solutions while the problem of over-flow capacity is not considered to be so important.Departments in large companies increasingly oper-ate as profit centers on the basis of individual in-ternal projects. The time frame typically is in therange of 6-12 months. This tendency is expected tocontinue. The short project period and the require-ments of a profit center force these departments tolook for new solutions. The Grid may be a solutionthat allows operating with reduced fixed costs whilestill being able to provide the necessary level of per-formance. In this scenario, the main benefits of theGrid will be flexibility, availability and cost reduc-tion.

Of course, the same situation may also arise insmall and medium sized companies. But here bad

network connectivity and lack of technical skills of-ten inhibit the use of Grid computing. While net-work connectivity is likely to improve in the nearfuture, the technical skill problem might remain asan inhibiting factor.

6.2. The New Challenge of Cloud Computing

Cloud computing is a new concept that hasevolved recently from the Grid world [11,3]. As sev-eral companies have started to offer Cloud services,academia and funding organizations alike considerCloud computing to be interesting as it may im-prove flexibility and reduce fixed costs [15].

Both flexibility and cost reduction have becomesubstantial issues in the academic world as pub-lic research is under similar constraints as industry.Hence, Cloud computing may become attractive foracademic in-house computing at a wide scale if aca-demic users can actually benefit from the availabil-ity of Cloud computing. From the point of view ofresearchers the key aspect will be the general avail-ability of the services. Therefore, end users might behappy with having a specified and guaranteed levelof service at their finger tips within a standardizedenvironment at reasonable cost and not insist on lo-cally provided service by their compute center [17].

6.3. The Co-operation Argument

As we have identified potential gains in usage ofindustrial resources by academia and vice versa theissue of co-operation comes up. Co-operation be-tween academia and industry in general is a constantmatter of both political and academic debate. Is-sues of ethics and organization are discussed as wellas the question of know-how transfer and educationfor the job. When it comes to service provisioningco-operation seems to be easier to achieve. But itis not clear which co-operation model provides themost benefits. As we have seen above, the very for-mal co-operation of jointly founding a company canbe very successful. However, as there is some over-head to this solution, other approaches may be bet-ter in some situations.

In the future, we expect to see a variety of forms inco-operation between public research and industryin the field of Grid computing. Project driven loseco-operations will co-exist with formal agreementsand will complement a Grid service market whichprovides basic services as a commodity.

15

The decision for a specific type of co-operation willbe driven by the needs of the participating organi-zations. For short term co-operation and for highlyfocussed and specialized initiatives, a project orga-nization may be the best concept. For a long termco-operation involving basic Grid services, it maybe best to apply a simple customer-vendor relationwith all the commercial formalities that come withit.

6.4. The Technical Argument

If the interest in co-operation is going to grow asexpected, we must discuss realization issues. Gridcomputing working groups usually focus on techni-cal issues. It is certainly true, that several technicalissues must be solved before academia and industrycan share resources. But beyond these technical is-sues there are other problems that might be equallyimportant for the support of such co-operation.

As with any co-operation we must overcome dif-ferences in culture and methodologies that we findin public research on the one hand and in indus-try on the other hand. Although this is a commonproblem its impact must not be underestimated. Ina customer-vendor relation, these differences will beeasier to overcome. However, when we enter intopublic-private partnerships management, both sideswill have to be careful about these issues.

For instance, security has so far been consideredas a technical problem. However, a deeper analy-sis shows that it is the problem of compliance thatmust be considered here. While public research isgenerally aiming at perfect security (if this is everpossible), a focus of industry is on identifying thecause of security breaches. In more detail, public re-search puts all efforts in technically avoiding anysuch breaches while industry is also interested inidentifying the person responsible for the breach.

Finally, we can also identify a technical problemthat may prove to be a key issue in Grid businessmodels: the management of licenses. It is not thepure cycle selling that makes the future Grid attrac-tive but the full simulation service. A key role will beassigned to independent software vendors. Throughtheir license policies they will be able to drive thefuture development. However, as of today, the Gridis often considered as a menace to existing licensingmodels. Future strategies must be more flexible andneed to create a win-win situation for both Indepen-dent Software Vendors (ISV) and users.

7. Role of Academic Resource and ServiceProviders

Almost each academic institution features its lo-cal compute center. It used to be the task of thiscompute center to provide the researchers of this in-stitution with IT resources and services. Researchersonly needed remote access to special resources likesupercomputers. While these compute centers willstill be providers for all basic IT-services such as net-working, e-Mail, or others, we expect that many sci-entific services will be offered by external providersif the local center is not able to match their qual-ity or costs. However, it is not clear whether spe-cific research services can be as easily outsourced asstandard computing services. For example, there isa need for support in case of optimizations or paral-lelization of codes for dedicated machines, which isnot likely to be offered by a generic service provider.We believe that academic service providers will con-tinue to offer Grid services mainly for three reasons:– Research requirements in a specific field– Specific demands of users asking for scientific ser-

vices– Generalization of services or interfaces for several

communities at the same timeIt is also not clear whether the present model of

mainly industry based Cloud computing services canbe extended to satisfy the academic requirementsspecified above, see also Section 6.

Once the needs for Grid services are specified, wemust analyze how many sites are needed in the Grid.For instance, the world’s largest production Grid in-frastructure Enabling Grids for E-science (EGEE)connects 267 sites worldwide today. There is a dan-ger that such a large number of sites may result in in-efficiencies, as each site no matter how small requiresits own operation, maintenance and infrastructureservices. Yet, concentrating Grid resources into toofew or even single locations may reduce competitionand hamper further development. Therefore, it is farfrom optimal as well.

For each Grid, we may identify different con-straints which can be of political or technical nature.They may address the needs for distributed locationof data, or the ability to deal with different user re-quirements, or the needs for different heterogeneousresources. As such, it is a delicate issue to choose theright number of sites within a Grid infrastructure.

It is also clear, that Grid computing is not re-stricted to elementary resources like computing cy-

16

cles or storage capacity. It further includes collabo-ration, abstraction, and other aspects that must beconsidered accordingly. As a result, the current situ-ation in Europe is quite diverse, ranging from coun-tries with large Grids for research, like Grid5000 inFrance, to countries where each center owns its Grid-enabled cluster without sharing its services with oth-ers.

These regional differences must also be consideredwhen thinking about sustainability. The Grid hasthe potential to create a market place where Gridservices can be offered to many more users thanthose within the limits of one institution. This in-troduces many opportunities due to the economyof scale, enabling Grids to achieve sustainability byutilizing the forces of the market once a certain sizeis achieved.

However, Grid services are currently provided forfree. To use the forces of the market, some formof payment for these services must be introduced.To this end, we must take different options into ac-count, from paying for the whole package, througha pay-per-use model, or just paying a portion of thefee with some mechanism of subsidizing. Yet, pastexperiences with other IT services have shown thatasking for payment from research/scientific commu-nities may lead to reductions or even a total stopin usage with potentially bad consequences for re-search. Therefore, an appropriate strategy shouldoffer useful services with different rates for differentlevels or qualities of service and different usage pat-terns. The Grid idea as such could be a vehicle tomake the market for services work.

8. Summary and Conclusions

During the last years, the Grid became an essen-tial technology platform in many application fields.However, the number of users is not dramatically in-creasing. In comparison to the revolutionary adop-tion of the Internet, Grids show a slower and evo-lutionary proliferation. Presently, there is no indi-cation of a rapid adoption of Grids into the massmarket.

Grid as a paradigm of sharing, selecting, and ag-gregating of large scale distributed virtualised re-sources is still an inspiration for challenging investi-gations even if they are labeled Cloud or sky com-puting.

The future of Grids will depend on key appli-cations that require the dynamic use of large-scale

compute and data resource. The trend towards avirtual representation of the real world may pro-vide such an application scenario. Further, the smartcombination of online data from sensor networksand arbitrary archives on the one hand and com-puting facilities on the other hand will provide novelservices. The benefit of these services may not berestricted to scientific fields like particle physics orclimate research, but also reach into industrial andsocietal domains.

A new paradigm of scientific investigations, calledsystem-level science, whose main feature is the in-tegration of diverse sources of knowledge about theconstituent parts of a complex system with the goalof obtaining an understanding of the system’s prop-erties as a whole [26] requires a Grid infrastructure.Such applications are compute- and data-intensive,are developed using many programming languages,and used in dynamic scenarios which involve vari-ous levels of coupling and composition types such asparallel or workflow processing. These applicationsenforce an evolution from simple computing infras-tructures supporting batch jobs to more advancedsoftware systems which provide high-level services.Problems such as access to computation, deploy-ment and application management remain a chal-lenge, due to some inherent features of Grid systems.

The once foreseen global and open Grid infras-tructure did not emerge yet. So far, a general needfor interoperable Grids has not been demonstrated.But more specific forms of interoperability are of in-terests and can be achieved on application and mid-dleware levels [14,50,33]. The realization of theseforms of interoperability requires a mature and reli-able middleware. Unfortunately, current Grid mid-dlewares implementations do not only fail to inter-operate seamlessly but they are also too complexto allow quick appropriate modifications. For thesake of reducing this complexity, it can be expectedthat some of the necessary Grid functionality canbe moved to different layers. Security and data inte-gration are key requirements which could be movedfrom the middleware to the operating system, as inthe XtreemOS project. Other functions, like meta-scheduling and brokering, can be moved up to thecommunity or even to the application levels. To solvethese middleware related problems, it is not suffi-cient to invest more funds in the generation of newproductions Grids but also to support more basicresearch in the Grid domain.

The integration of dedicated service and resourceproviders is a key property of the Grid concept. How-

17

ever before achieving these goals, many legal andadministrative questions must be solved. Moreover,it is likely that some but not all Grid services foracademia can be outsourced to reduce costs. Par-ticularly, specific research aspects will only be ad-dressed by academic service providers as the criticalnumber of users may be too small to generate suf-ficient interest in the commercial world. But theseacademic providers must focus on these specific ser-vices while possibly leaving off-the-shelf services toexternal providers. In general, efficiency issues willgain more attention and lead to typical make or buydecisions regarding the services which are locally of-fered or not.

We live in a complex world where we are over-loaded by information and constantly need to makedecisions, taking into account endless sources of in-formation. This is true in day to day life, but alsoin scientific practice. Information from apparatus,observations, databases, literature, experiments etcneeds to be gathered, registered, combined andmade useful. The good news is that we can generatemore and more information, the bad news is thatwe often have no idea as to how to integrate andcombine these bits and pieces of information suchthat they are manageable and make sense, such thatwe can distill knowledge from it and build decisionsupon it.

It is the Grid idea that can assist us here. Com-plete service offerings including hardware resourceswill provide the necessary added value to users. Par-ticularly, efficient data integration in large-scale dis-tributed systems will play a major role in Grid de-velopment. The use of Cloud computing and virtu-alization techniques for machines, storage and net-works may generate new ways to provide and to uti-lize resources. It would be a big mistake to considerthese approaches as a competitor to Grids as theyare actually part of the Grid concept and must beintegrated with other components of Grid technol-ogy. But to achieve this integration many researchand organizational issues must be solved.

We should not waste our time in redefining termsor key technologies: clusters, Grids, Clouds... whatis in a name? Ian Foster recently 28 quoted MironLivney saying: "I was doing Cloud computing waybefore people called it Grid computing", referring tothe ground breaking Condor technology [12,20]. Itis the Grid scientific paradigm that counts!.

28 Ian Fosters presentation at WorldComp 2009, Las Vegas,July 13th, 2009

The authors hope that this paper will revive andstimulate discussion on achievements and failuresof more that 10 years of research under the Gridcomputing umbrella.Acknowledgments. This work is the outcome ofa Dagstuhl Perspectives Workshop on the Future ofGrid Computing (Seminar 09082) which took placefrom February 15 to February 20, 2009. The authorswould like to express their gratitude for the gener-ous support by the Dagstuhl staff and everyone whocontributed to the success of the seminar.

References

[1] Alessandro Acquisti and Ralph Gross. Imaginedcommunities: Awareness, information sharing, andprivacy on the facebook. Lecture Notes in ComputerScience, 4258:36–58, 2006.

[2] Ali Anjomshoaa, Fred Brisard, Michel Drescher, DonalFellows, An Ly, Stephen McGough, Darren Pulsipher,and Andreas Savva. Job Submission DescriptionLanguage (JSDL) Specification. Technical Report GFD-R.056, Open Grid Forum (OGF), 2005.

[3] Michael Armbrust, Armando Fox, Rean Griffith,Anthony D. Joseph, Randy H. Katz, Andrew Konwinski,Gunho Lee, David A. Patterson, Ariel Rabkin, IonStoica, and Matei Zaharia. Above the clouds: A berkeleyview of cloud computing. Technical Report EECS-2009-28, EECS at UC Berkeley, 2009.

[4] Matthias Assel and Bettina Krammer. TowardsInnovative Healthcare Grid Solutions: ViroLab - AVirtual Laboratory for Infectious Diseases. In InProceedings of the German e-Science Conference 2007,Baden-Baden, Germany, 2007.

[5] Henri Bal, Raoul Bhoedjang, Rutger Hofman, CerielJacobs, Thilo Kielmann, Jason Maassen, Rob VanNieuwpoort, John Romein, Luc Renambot, Tim Rül,Ronald Veldema, Kees Verstoep, Aline Baggio, GercoBallintijn, Ihor Kuz, Guillaume Pierre, Maarten VanSteen, Andy Tanenbaum, Gerben Doornbos, DesmondGermans, Hans Spoelder, Evert jan Baerends, Stan VanGisbergen, Hamideh Afsermanesh, Dick Van Albada,Adam Belloum, David Dubbeldam, Zeger Hendrikse,Bob Hertzberger, Alfons Hoekstra, Kamil Iskra, DronaK, Dennis Koelma, Frank Van Der Linden, BennoOvereinder, Peter Sloot, Piero Spinnato, Dick Epema,Arjan Van Gemund, Pieter Jonker, Andrei Radulescu,Cees Van Reeuwijk, Henk Sips, Peter Knijnenburg,Michael Lew, Floris Sluiter, and Lex Wolters.The distributed ASCI supercomputer project. TheInternational Journal of Supercomputer Applicationsand High Performance Computing, 34:76–96, 2000.

[6] Henri Bal, Cees de Latt, Seif Haridi, Keith Jefferyad Jesus Labarta, Domenico Laforenza, PeterMaccallum, Joan Massó, Ludek Matyska, ThierryPriol, Alexander Reinefeld, Andreas Reuter,Michel Rigiguidel, Dave Snelling, Martin vanSteen, and Roman Tirler. Next generation

18

grid(s) - european grid research 2005 - 2010.ftp://ftp.cordis.europa.eu/pub/ist/docs/ngg_eg_final.pdf,June 2003.

[7] Ian Bird, Bob Jones, and Kerk F. Kee. Theorganization and management of grid infrastructures.IEEE Computer, 42(1):36–46, 2009.

[8] Tobias Blanke, Mark Hedges, and Stuart Dunn. Artsand humanities e-science–current practices and futurechallenges. Future Generation Computer Systems,25(4):474 – 480, 2009.

[9] Raphaël Bolze, Franck Cappello, Eddy Caron, MichelDaydé, Frédéric Desprez, Emmanuel Jeannot, YvonJégou, Stephane Lantéri, Julien Leduc, Noredine Melab,Guillaume Mornet, Raymond Namyst, Pascale Primet,Benjamin Quetier, Olivier Richard, El-Ghazali Talbi,and Touche Iréa. Grid’5000: a large scale and highlyreconfigurable experimental grid testbed. InternationalJournal of High Performance Computing Applications,20(4):481–494, November 2006.

[10] Marian Bubak, Tomasz Gubala, Maciej Malawski,Bartosz Balis, Wlodzimierz Funika, Tomasz Bartynski,Eryk Ciepiela, Daniel Harezlak, Marek Kasztelnik,Joanna Kocot, Dariusz Krol, Piotr Nowakowski, MichalPelczar, Jakub Wach, Matthias Assel, and AlfredoTirado-Ramos. Virtual laboratory for collaborativeapplications. In Mario Cannataro, editor, Handbookof Research on Computational Grid Technologies forLife Sciences,Biomedicine and Healthcare, InformationScience Reference, chapter XXVII. IGI Global, 2009.ISBN: 978-1-60566-374-6.

[11] Rajkumar Buyya, Chee Shin Yeo, Srikumar Venugopal,and James Broberg a nd Ivona Brandic. Cloudcomputing and emerging it platforms: Vision, hype, andreality for del ivering computing as the 5th utility. FutureGeneration Computer Systems, 25(6):599 – 616, 2009.

[12] Condor project website, 2009. http://www.cs.wisc.edu/condor/.

[13] MassimoCoppola, Yvon Jégou, Brian Matthews, Christine Morin,Luis Pablo Prieto, Óscar David Sánchez, Erica Y. Yang,and Haiyan Yu. Virtual organization support within agrid-wide operating system. IEEE Internet Computing,12(2):20–28, 2008.

[14] Peter V. Coveney, Giovanni Giupponi, Shantenu Jha,Steven Manos, Jon MacLaren, Stephen M. Pickles,Radhika Saksena, Thomas Soddemann, James Suter,Mary-Ann Thyveetil, and Stefan J. Zasada. Largescale computational science on federated internationalgrids: The role of switched optical networks. FutureGeneration Computer Systems, 26(1):99 – 110, 2010.

[15] Marios D. Dikaiakos, Dimitrios Katsaros, Pankaj Mehra,George Pallis, and Athena Vakali. Cloud computing:Distributed internet computing for it and scientificresearch. IEEE Internet Computing, 13(5):10–13, 2009.

[16] Flavia Donno, Vincenzo Ciaschini, David Rebatto,Luca Vaccarossa, and Marco Verlato. The worldgridtransatlantic testbed: a successful example of gridinteroperability across eu and u.s. domains. In CHEP2003, La Jolla - CA, USA, March 24-28 2003.

[17] Jorge Ejarque, Marc de Paloland Íñigo Goiri, FerranJulià, Jordi Guitart, Rosa M. Badia, and JordiTorres. SLA-driven semantically-enhanced dynamic

resource allocator for vir tualized service providers. InProceedings of the e-Science conference, 2008.

[18] Mattias Ellert, MichaelGrønager, Aleksandr Konstantinov, Balázs Kónya,Jonas Lindemann, Ilja Livenson, Jakob LanggaardNielsen, Marko Niinimäki, Oxana Smirnova, and AndersWäänänen. Advanced resource connector middlewarefor lightweight computational grids. Future GenerationComputer Systems, 23:219–240, 2007.

[19] Grid Enterprise Service Forum, 2009. http://www.opengroup.org/ges/.

[20] Dick H. J. Epema, Miron Livny, René van Dantzig,Xander Evers, and Jim Pruyne. A worldwide flockof condors: Load sharing among workstation clusters.Future Generation Computer Systems, 12(1):53 – 65,1996. Resource Management in Distributed Systems.

[21] Dieter Erwin. UNICORE–a Grid computingenvironment. Concurrency and Computation: Practiceand Experience, 14(13-15):1395–1410, 2002.

[22] Laurence Field and Markus Schulz. Gridinteroperability: the interoperations cookbook. Journalof Physics: Conference Series, 119(1):012001 (8pp),2008.

[23] Ian Foster, Andrew Grimshaw, Peter Lane, WilliamLee, Mark Morgan, Steven Newhouse, Stephen Pickles,Darren Pulsipher, Christopher Smith, and MarvinTheimer. Open grid services architecture basic executionservice specification 1.0. Technical Report GFD-R.108,Open Grid Forum (OGF), 2008.

[24] Ian Foster and Carl Kesselman. Globus: Ametacomputing infrastructure toolkit. InternationalJournal of Supercomputing Applications, 11(2):115–128,1997.

[25] Ian Foster and Carl Kesselman. The Grid 2:Blueprint for a New Computing Infrastructure (TheMorgan Kaufmann Series in Computer Architecture andDesign). Morgan Kaufmann, November 2003.

[26] Ian Foster and Karl Kesselman. Scaling system-level science: Scientific exploration and IT implications.Computer, 39(11):31–39, 2006.

[27] Wolgang Gentzsch and Alexander Reinefeld. Specialsection on D-Grid. Future Generation ComputerSystems, 25:266–267, 2009.

[28] Alfons G. Hoekstra, Simon F. Portegies Zwart, MarianBubak, and Peter M.A. Sloot. Towards distributedpetascale computing. In David Bader, editor, PetascaleComputing: Algorithms and Applications, chapter 8,pages 147–164. Chapman and Hall, 2008. ISBN9781584889090.

[29] David Jensen and Jennifer Neville. Correlation andsampling in relational data mining. In n Proceedingsof the 33rd Symposium on the Interface of ComputingScience and Statistics, 2001.

[30] Bob Jones. An Overview of the EGEE Project, volume3664 of Lecture Notes in Computer Science, pages 1–8.Springer-Verlag, August 2005.

[31] Erwin Laure, Steve M. Fisher, ťAkohs Frohner, ClaudioGrandi, Peter Kunszt, Ales Krenek, Ole Mulmo, FabrizioPacini, Francesco Prelz, John White, Maite Barroso,Predrag Buncic, Frederic Hemmer, Alberto Di Meglio,and Åke Edlund. Programming the Grid with gLite.

19

Computational Methods in Science and Technology,12(1):33–45, 2006.

[32] Miron Livny, Ruth Pordes, Knet Blackburn, and PaulAvery. Open science grid. annual report to doe.Technical report, Open Science Grid, December 2008.

[33] Maciej Malawski, Tomasz Bartynski, and MarianBubak. Invocation of operations from script-based gridapplications. Future Generation Computer Systems,26(1):138 – 146, 2010.

[34] NAREGI - National Research Grid Initiative, 2007.http://www.naregi.org/i.

[35] María A. Nieto-Santisteban, Jim Gray, Alexander S.Szalay, James Annis, Aniruddha R. Thakar, and WilliamO’Mullane. When database systems meet the grid. InCIDR, pages 154–161, 2005.

[36] Polish infrastructure for supporting computationalscience in the european research space, 2009.http://www.plgrid.pl/en.

[37] Thierry Priol. Objects, components, services for gridmiddleware, 2005. The keynote lecture at European GridConference: Advances in Grid Computing - EGC 2005,Amsterdam, The Netherlands, February 14-16, 2005.

[38] Hrabri Rajic, Roger Brobst, Waiman Chan, Fritz Ferstl,Jeff Gardiner, Andreas Haas, Bill Nitzberg, HrabriRajic, and John Tollefsrud. Distributed resourcemanagement application api (drmaa) specification 1.0.Technical Report FD-R-P.022, Open Grid Forum(OGF), 2004.

[39] Jeremy Rifkin. The European Dream: How Europe’sVision of the Future is Quietly Eclipsing the AmericanDream. Peguin, 2004.

[40] Matei Ripeanu, Munindar P. Singh, and Sudharshan S.Vazhkudai. Virtual organizations. IEEE InternetComputing, 12(2):10–12, 2008.

[41] David De Roure, Carole Goble, and RobertStevens. Designing the myexperiment virtual researchenvironment for the social sharing of workflows. In ThirdIEEE International Conference on e-Science and GridComputing, 2007, Bangalore, India; 10-13 December2007, pages 603–610, 2007.

[42] Tetsuya Sato. The earth simulator: Roles and impacts.Parallel Computing, 30(12):1279–1286, 2004.

[43] Tobias Scholl, Bernhard Bauer, Benjamin Gufler,Richard Kuntschke, Angelika Reiser, and AlfonsKemper. Scalable community-driven data sharing in e-science grids. Future Generation Computer Systems,25(3):290 – 300, 2009.

[44] Uwe Schwiegelshohn. The structure of D-Grid. InMarian Bubak, Michał Turata, and Kazimierz Wiatr,editors, Proceedings of the Cracow Grid Worlshop 2008,pages 3–10, 2009.

[45] Peter M.A. Sloot, Alexander V. Boukhanovsky,W. Keulen, Alfredo Tirado-Ramos, and Charles A.B.Boucher. Decision support in individualized e-health.Journal of Clinical Monitoring and Computing, 19(4-5):1287–1307, 2005.

[46] Peter M.A. Sloot, Peter V. Coveney, Gokhan Ertaylan,Viktor Mueller, Charles A.B. Boucher, and MarianBubak. HIV Decision Support: From Molecule toMan. Philosophical Transactions of the Royal Society,367(1898):2691–2703, 2009.

[47] Peter M.A. Sloot, Daan Frenkel, Henk A. Van derVorst, Antoine van Kampen, Henri E. Bal, Paul

Klint, Robert M. Mattheij, Jack van Wijka, JoopSchaye, Huib Jan van Langevelde, Rob H. Bisseling,Berend Smit, E. Valenteyn, Henk J. Sips, Jos B.T.M.Roerdink, and Koen G. Langedoen. Computational e-science: Studying complex systems in silico. a nationalcoordinated initiative. white paper. Technical report,Informatics Institute, Universiteit van Amsterdam,February 2007.

[48] Peter M.A. Sloot, Alfredo Tirado-Ramos, Ilkay Altintas,Marian Bubak, and Charles A.B. Boucher. Decisionsupport in individualized e-health. IEEE Computer,39(11):40–46, 2006.

[49] Peter M.A. Sloot, Geert Dick van Albada, MarianBubak, and Anne Trefethen. 2nd IEEE InternationalConference on e-Science and Grid Computing. IEEE,2006. ISBN 0-7695-2734-5; CD.

[50] Larry Smarr, Maxine Brown, and Cees de Laat. Specialsection: Optiplanet – the optiputer global collaboratory.Future Generation Computer Systems, 25(2):109 – 113,2009.

[51] Mike Thelwall. Social networks, gender, and friending:An analysis of myspace member profiles. Journal of theAmerican Society for Information Science and Technology, 59(8):1321–1330, 2008.

[52] A virtual laboratory for infectious diseases, 2006-2009.http://www.virolab.org.

[53] Mitchell Waldrop. Science 2.0: Great new tool, or greatrisk?, January 9 2008. http://www.sciam.com/article.cfm?id=science-2-point-0-great-new-tool-or-great-risk.

[54] Zhiming Zhao, Adam Belloum, and Marian Bubak.Special section on workflow systems and applicationsin e-science. Future Generation Computer Systems,25(5):525 – 527, 2009.

20