Extending Grid-based Workflow Tools with Patterns/Operators

Cecilia Gomes1, Omer F. Rana2 and Jose Cunha1

CITI Center, Universidade Nova de Lisboa, Portugal

School of Computer Science/Welsh e-Science Center, Cardiff University, UK

Abstract

Many Grid applications involve combining compu-tational and data access components into complexworkflows. A distinction is generally made be-tween mechanisms to compose components (referredto as build time functions) and subsequently mech-anisms to execute these components on distributedresources (referred to as run time functions). Anapproach to supporting such build and run timefunctions using specialist patterns and operators ispresented. “Structural” patterns may be treatedas meta-components within a workflow system, andused within the composition process. Subsequently,such components may be scheduled for execution us-ing “Behavioural” patterns via the enactment pro-cess. Application examples to demonstrate how suchpatterns – and subsequently operators – may be usedare presented. Their implementation within the Tri-ana Problem Solving Environment is also described.

1 Introduction and Motivation

The importance of workflow in Grid computing ap-plications has been recognised by a number of re-searchers [18, 1]. Generally such workflow involvesthe orchestration of services which are hosted ondifferent resources, and often in different adminis-trative domains. According to [2], Grid workflowmanagement could be characterised into build timefunctions and run time functions. Build time func-tions relate to defining and modelling the workflowtasks/services and their dependencies. A variety ofapproaches exist for such build time specification –

ranging from Directed Acyclic Graphs (DAGs) tolanguage-based formalisms. On the other hand, runtime functions relate to the management of workflowexecution/enactment and interactions with Grid re-sources. Examples of this approach include the useof constraint checking mechanisms [24]. This separa-tion is useful to allow a given build time function tobe mapped to multiple run time functions and viceversa. The run time function is built over an existingGrid middleware system (such as the Globus toolkitor UNICORE).

Existing approaches to workflow construction(build time functions) involve connecting multiplecomponents according to their dependencies. Thegranularity of each component can vary from being acomplete application, a library component to a sub-routine. Given such a composition, it is possible todevelop some general patterns identifying how com-ponents within a workflow may be combined. Hence,it is possible to identify a catalogue of such commonpatterns, as undertaken in [19], in the context of busi-ness services. Such patterns form the basis for struc-turing a particular application, and could also be in-cluded within a workflow system and instantiated toparticular executable components at run time. A sec-ond approach to build time functions involves the dy-namic construction of a workflow from application-level descriptions of a given data product. In thisinstance, given a data product, a planning engine isused for selecting appropriate components to achievethe required data product. The approach can also beextended to a run time function, requiring the plan-ner to also discover suitable computational resourcesto execute these components. Such an approach is

1

used to first create an “abstract” workflow – identi-fying suitable components based on the descriptionof their capabilities, and combining these togetherusing a pre-defined plan library. The abstract work-flow is then mapped into a “concrete” workflow byassociating each component with an executable task(identifying location of input/output files, configura-tion parameters, and access rights) [3].

The use of planning approaches in composing WebServices has also been considered by other authors,such as with OWL-S [21] and recently WSMO [22].Such approaches involve the use of a rich semantic an-notation with each service, and a process graph thatrepresents relationships between services. OWL-S,for instance, makes use of a “service profile” (defin-ing the I/O model of the service), a “service model”(identifying how the service works), and a “ser-vice grounding” (identifying how the service shouldbe executed). A process model (atomic, simple orcomposite) is then used to combine services, usingflow-control constructs such as sequence, iterate,repeat-until etc. An alternative approach involvesthe use of planning as model checking. In this ap-proach an executor, given a goal to achieve, identifiesall sets of possible states that can be valid given thegoal state and the current input data. These statesare then pruned based on other knowledge (such as“beliefs”) available to the executor. A key featureof this approach is that the executor can only par-tially observe the domain within which compositionis taking place – therefore imprecise knowledge of thedomain is assumed. However, it is necessary to builda model of the domain – involving a detailed under-standing of the starting state and a description of allthe interaction protocols involved. This requirementrenders this approach difficult to utilize in the contextof a distributed Grid system.

1.1 Extending Workflow Tools

Our approach lies between a user-defined workflow(both build and run time), and an automated work-flow using planning/semantic approaches identifiedabove. We believe the semantic approaches aretoo complex for many existing Grid applications, asthey require application users to provide annotations

to their existing components. Furthermore, find-ing equivalence between semantically similar compo-nent descriptions is a non-trivial undertaking. Onthe other hand, model checking approaches requirea detailed description of protocols involved withina particular domain and could lead to a high com-putational complexity to identify the set of all validstates – in the context of one or more protocols. Wetherefore propose an approach that makes use of de-sign patterns at the compositional level (referred toas “structural patterns”) and at the behaviour/runtime level (referred to as “behavioural patterns”).These patterns can then be manipulated using script-ing tools through the use of “structural” and “be-havioural operators”. Hence we: (1) extend the ca-pabilities of existing workflow systems with supportfor Patterns and Operators, and (2) map the “Be-havioural” patterns and operators to a resource man-agement system. In this way we address both buildtime and run time functions, and provide an optionmidway between a full semantic annotation of com-ponents for automated composition and a totally userdefined composition using a graphical tool.

Patterns allow the abstraction of common inter-actions between components, thereby enabling reuseof interactions that have proven useful in similar do-mains. A user may build an application in a struc-tured fashion by selecting the most appropriate set ofpatterns, and combine them according to pre-definedoperator semantics. Users may define new patternsand add these as standard components to a tool li-brary for use by others. Patterns and Operatorsalso provide additional capability that is not easilyrepresentable via visual components. Our approachtreats patterns as first class entities but differs fromother work [5, 6] in that the user may explicitly de-fine structural constraints between components, sep-arately from the behavioural constraints. Our ap-proach is somewhat similar to that of van der Aalstet al. [17] – although they do not make a distinctionbetween structural and behavioural patterns. Theirwork does not have a notion of “operators” as ameans to manipulate such patterns. They also focuson Petri net models of their patterns, whereas ourconcern is to link patterns with particular resourcemanagers and composition tools. Furthermore, the

2

approach we adopt enables a workflow designer toidentify constraints on the enactment process. Themapping of operators to a resource manager is there-fore an aspect not considered by other Grid-basedworkflow systems. The approach presented here isprimarily aimed at computational scientists and de-velopers, who have some understanding of the com-putational needs of their application domain. A sci-entist should be aware about the likely co-ordinationand interaction types between components of theapplication (such as a database or numeric solveretc). The structural and behavioural patterns pre-sented here will enable such scientists and develop-ers to utilise common usage scenarios within a do-main (either the use of particular components, suchas database systems, or interactions between compo-nents, such as the use of streaming).

The paper is structured as follows: section 2 in-troduces our concept of Patterns and Operators, andsection 3 demonstrates how these are implemented inthe Triana system [7]). The mapping of behaviouralpatterns and operators to the DRMAA API [8] isthen provided in section 5.

2 Structuring Workflow withPatterns and Operators

A workflow pattern encodes a commonly recurringtheme in component composition. A pattern is there-fore defined in an application independent manner,and particularly useful for configuring and specify-ing systems that are composed of independent sub-domains. Patterns are aimed at capturing somegeneric attributes of a system – which may be furtherrefined (eventually) to lead to an implementation.These are important requirements for Grid comput-ing applications, which generally need to operate indynamic environments. When using patterns, Gridapplication developers may deploy previously gener-ated pattern templates as an initial step, and thenrefine these based on our operators. The use of pat-tern operators is also particularly important to dealwith dynamicity, because they provide the capabil-ity to modify a pattern at run time. Furthermore,

pattern operators may be applied in an ordered com-bination – and may be shared between users. Thepresented structural and behavioural operators maybe implemented using a number of different scriptinglanguages (such as Python or Perl) – and thereforethe specified semantics are not restricted to our Javaimplementation. A brief overview of our pattern tem-plates and operators is provided here, details can befound in [10, 11]. These references also contain se-mantics of the patterns and operators discussed here,along with Petri net models of some of these opera-tors.

Alternative related work has been undertaken bythe parallel computing community, and is based onthe use of algorithmic skeletons. The predominantmotivation behind this has been the need to over-come the difficulty of constructing parallel programs– by capturing common algorithmic forms which maysubsequently be used as components for building par-allel programs [27], [28]. Such skeletons are expectedto provide parameterizable abstractions that may becomposed – generally using a functional program-ming language. A skeleton is expected to be transpar-ent to an application user (and may come with a pre-packaged implementation). Skeletons are viewed for-mally as polymorphic, higher-order functions – whichmay be repeatedly applied to achieve various trans-formations (on data structures such as lists). Her-rmann and Lengauer [26] outline the use of a pro-gramming language “Higher-order Divide and Con-quer” (HDC) based on a subset of the functional pro-gramming language Haskell. They suggest that theuse of a powerful type system in functional languagesmake them more suitable than other paradigms. Al-though useful for specifying programs in a concisesyntax, we believe such approaches are limited in thecontext of Grid environments. This is primarily dueto the absence of tools available in such languagesfor connecting to Grid middleware, such as Globusor UNICORE. Although skeletons based approachesdo provide a useful prototyping tool for analysis. Ouruse of “operators” borrows from the use of transfor-mation techniques in skeleton based approaches, al-beit our focus is on the use of object-oriented tech-niques. Furthermore, our design patterns and opera-tors are aimed at supporting workflow-based systems,

3

and not focused on use within a particular program-ming environment.Structural Pattern Templates encode componentconnectivity, representing topologies like a ring, astar or a pipeline, or design patterns like Facade,Proxy or Adapter [14]. The possibility of encodingthese structural constraints allows, for example, therepresentation of common software architectures inhigh-performance computing applications. For exam-ple, the pipeline pattern may be used in a signal pro-cessing application where the first stage may consistof a signal generator service producing data to a setof intermediate stages for filtering. Frequently, thelast stage consists of a visualization service for ob-serving results. Similarly, the proxy pattern, allowsthe local presence of an entity’s surrogate, allowingaccess to the remote entity.Behavioural Pattern Templates capture recur-ring themes in component interactions, and definethe temporal and the (control and data) flow depen-dencies between the components. Generally, theseapplications involve distribution of code from a mas-ter, the replication of a code segment (such as withina loop), or parameter sweeps over one or more indi-cies. Behavioural patterns define the temporal (con-trol and data) flow dependencies between the com-ponents. The separation of ‘structure’ from ‘be-haviour’ allows the selection of the most adequatecombinations for a particular application. We pro-vide several behavioural patterns such as Master-Slave, Client-Server, Streaming, Peer-to-Peer, Mo-bile Agents/Itinerary, Remote Evaluation, Code-on-Demand, Contract, Observer/Publish-Subscriber,Parameter sweep, Service Adapter, and so on. Forexample, the Service Adapter pattern attaches ad-ditional properties or behaviours to an existing ap-plication to enable it to be invoked as a service [15].The Master-Slave pattern, in turn, can be mapped tomany parallel programming libraries, and representsthe division of a task into multiple (usually indepen-dent) sub-units – and shares some similarities withthe Client-Server pattern – although the control flowin the latter is more complex.Structural Operators support the composition ofstructural patterns, without modifying the structuralconstraints imposed on the pattern. These operators

ProxyProxy

ProxyReal

Subject

Result pattern

RealSubject

Proxy

Pattern

RealSubject

Proxy a Proxy b

Result pattern

Increase( proxyPT, 2 ) Extend( proxyPT, element )

Figure 1: The increase and extend structural opera-tors.

provide a user with a simple and flexible way to re-fine structural patterns. There are several structuraloperators such as increase, decrease, extend, reduce,rename, replace, replicate, embed, etc. For examplein figure 1 it is possible to observe the result of apply-ing the increase and extend operators to the Proxypattern. The semantics of these operators can befound in [10].Behavioural Operators are applied over the struc-tural operator templates combined with the be-havioural patterns after instantiating the templateswith specific runnable components. Behavioural op-erators act upon pattern instances for execution con-trol and reconfiguration purposes. Behavioural oper-ators include: Start (starts the execution of a specificpattern instance), Stop (stops the execution of a pat-tern instance saving its current state), Resume (re-sumes the execution of a2 pattern instance from thepoint where it was stopped), Terminate (terminatesthe execution of a specific pattern instance), Restart(allows the periodic execution of a pattern instance),Limit (limits the execution of a specific pattern in-stance to a certain amount of time; when the timeexpires the execution is terminated), Repeat (allowsthe repetition of the execution of a specific pattern acertain number of times), etc. Both structural opera-tors and behavioural operators can be combined intoscripts which may be later reused in similar applica-tions. When using Patterns and Operators, a userwould develop their application in the following way:

1. Provide a structure definition by selection ofstructural patterns and if necessary their refine-ment through structural operators;

2. Provide a behaviour definition (i.e. definition

4

of data and control flows) by selection of be-havioural patterns.

3. Manipulate actual data flows between compo-nents to support component execution by the useof behavioural operators.

4. If necessary, provide any dynamic reconfigura-tion of the workflow by behavioural and struc-tural operators.

We implement these patterns and operators withinthe Triana Problem Solving Environment and theDRMAA API. Currently, we do not provide any sup-port to a workflow developer about which structuralor behavioural pattern/operator would be most suit-able in a given context. This decision is left purely forthe developer. Consequently, the same workflow out-come can be achieved in multiple possible ways. Wetherefore do not provide any “methodology” for theselection of patterns or operators. A summary of thepatterns and operators that are currently supportedcan be found in Table 1.

3 Implementation over the Tri-ana Workflow Environment

The Triana Problem Solving Environment providesa composition environment to allow a set of compo-nents to be combined and subsequently a mechanismfor distribution of components [16]. The compositionenvironment is developed so that it can be used in-dividually, allowing the composition environment tobe used alongside a variety of different execution en-gines. An XML-based task graph is generated fromthe composition tool, and supports bindings for dis-tributing components using Web Services or Peer-2-Peer technologies (based on JXTA). Triana also re-quires the existence of a Triana execution environ-ment to exist on each node that is to host a Trianaservice. This is also a significant difference from ex-isting portal technologies [1]; existing systems assumethe presence of a hosting environment on resources.

Our prototype extends Triana [12], and allows de-velopers to utilise a collection of pre-defined patternsfrom a library. Triana comes with components (called

Figure 2: The Triana Graphical User Interface show-ing a ring and a star structural pattern

units) for signal processing, mathematical calcula-tions, audio and image processing, etc, and provides awizard for the creation of new components, which canthen be added to the toolbox. Structural Patterns ap-pear as standard components that can be combinedwith other patterns or executable units. Triana pro-vides both a composition editor, and a deploymentmechanism to support this. The Pattern library pro-vided within Triana treats patterns as “group units”(i.e. units made up of others). Each element withinsuch group units is a “dummy” component (or a placeholder) and can subsequently be instantiated with ex-ecutables from the Triana toolbox. Hence, structuralpattern templates are collections of dummy compo-nents that can be instantiated with other structuralpattern templates or with executables. Every pat-tern has an associated ‘pattern controller/executor’which enables refinement of structural patterns viaoperators, and allows enactment of the selected Be-havioural patterns and operators.

A Galaxy simulation application with Triana is il-lustrated in figure 3. The Galaxy formation examplemay be represented by a star pattern template, where

5

Patterns OperatorsStructural Pipeline, Star, Rename, Replace, Increase,

Ring, Bus Decrease, Extend, Reduce,Adapter,Proxy, Facade Replicate, Embed, Group/Aggregate

Behavioural Master-Slave, Streaming, IsEqual, IsRecursive,Client-Server, Peer-2-Peer, IsDisjoint, IsSubset, IsSuperset,Mobile Agents/Itinerary, IsComposite, IsInComposite, IsCompatible,

Remote Evaluation, IsOwner, Owner, OwnerGroup,Code-on-Demand, Contract, AssignActivity, RemoveActivity,Observer/Subscribe-Publish, Start, Terminate, Stop, Log, Resume,

Parameter Sweep Restart, Limit, Repeat, Steer,ChangeDependencies.Synchronise,

ChangeDependencies.ChangeDataFlow,ChangeDependencies.ChangeControlFlow,

ChangeDependencies.ChangeSharedDataDependencies

Table 1: Pattern Templates and Operator Summary

Figure 3: Final configuration: image processing inthe “Galaxy Formation example”

the nucleus contains the actions necessary to generateand control the animation execution, and the satel-lites represent image processing and analysis actions.Both the actions at the nucleus and at the satellitesare supported by pipeline templates. Figure 4 showshow to embed a pipeline into the nucleus of the star(called DummyUnit).

Figure 4: Combining a pipeline pattern template witha star pattern template

4 Application Usage

The requirement of distributed data and computa-tional resources has been demonstrated in a numberof astrophysics applications, such as in the Astro-Grid [23] and GEO-LIGO [25] projects. Applicationsin these projects require high-performance comput-ing support for undertaking data analysis on variousdata types (text, images) and involve large quantitiesof time-based data. Data capture and analysis ad-ditionally involves scientists and instruments whichare geographically distributed – requiring a workflow

6

engine to coordinate execution across different sites.We demonstrate through two application scenariosthe use of our patterns and operators within Triana.

4.1 GEO-LIGO Data Analysis and Vi-sualisation

The GEO-LIGO project, in particular, is related togravitational wave experiments [13] where data cap-tured from laser interferometers (such as GEO600,LIGO and VIRGO) needs to be accessed and anal-ysed. Similarly, galaxy and star formation usingsmoothed particle hydrodynamics generates largedata files containing snapshots of an evolving sys-tem stored in 16 dimensions. Typically, a simplisticsimulation would consist of around a million particlesand may have a raw data frame sizes of 60 Mbytes,with an overall data set size of the order of 6 GBytes.The dimensions describe particle positions, velocities,and masses, type of particle, and a smoothed particlehydrodynamic radius of influence. After calculation,each snapshot is entirely independent of the othersallowing distribution over the Grid for independentdata processing and graphic generation.

Figure 5 shows a simple example where a wavedetector is producing data to be analysed and dis-played by several services – allowing multiple scien-tists to compare the results of analysis. The exampleillustrates two visualisation services, and a transfor-mation and visualisation pipeline. To configure thisapplication example, a user first identifies the rele-vant structural patterns: a star pattern with fourelements is created to represent the connections be-tween the Wave Detector service (figure 5) and thetransformation and visualisation services. In turn, tosupport the Transformation and Visualisation ser-vice (figure 5) the user creates a pipeline patternwith three elements. To obtain the right number ofcomponent place holders in both pattern templates(PTs), the user may apply the Increase() or the De-crease() operators. Subsequently, the user combinesboth pattern templates by embedding the pipelinepattern into one of the star pattern element. Finally,suitable components are instantiated into these ele-ments.

This example is implemented in Triana by mod-

(type B)Service

Visualisation

Service(type A)

Visualisation

Transformationand

ServiceVisualisation

TransformationFirst

TransformationSecond Visualisation

Wave Detector

Figure 5: Analysing gravitational wave data

elling the output of the gravitational wave detectorby a component which generates a wave with param-eterisable amplitude, type (sawtooth, sinusoid) andfrequency. The first configuration step is the creationof the two required pattern template: a Star and aPipeline. In order to create a Star PT, the user dragsand drops the DrawStar unit from Triana’s Patternstoolbox and initialises it (figure 6). In this case, itis necessary to increase the number of elements (fig-ure 7). For the creation of the Pipeline PT, the userselects the DrawPipeline unit and repeats the process.

Figure 6: Initialisation of a Star pattern template

Two PTs with Dummy Units representing compo-nent place holders are added in the workflow com-position tool. The DummyUnit may be instantiatedto a structural pattern template, or to an executable

7

Figure 7: Addition of one satellite to a Star PT

component from the Triana toolbox. The next config-uration step is to structure the two templates so thatthe Transformation and Visualisation service is con-nected to the Wave Detector service. The Embed op-erator is used on the Star PT to associate the PipelinePT with one of the DummyUnits. Finally, the userinstantiates the pattern templates with the necessaryexecutable components from the Triana toolbox. Anexample can be seen in figure 8.

Figure 8: Instantiation of a Unit.

Figure 9 shows the final configuration after all thetemplate slots have been instantiated. The gravita-tional wave detector (figure 5) is represented in this

example by the Wave unit. Two graphical display-ing units for rendering input signals are selected torepresent the visualisation services: the SGTGrapherand the Histogrammer. The selected transformationservices for instantiating the first two pipeline stagesare the Gaussian unit (which adds noise to the datagenerated by the Wave) and the FFT unit (whichperforms a Fast Fourier transform).

Figure 9: Final configuration.

4.1.1 Execution and Configuration Scripts

The data analysis described above may also be auto-mated through a script – which demonstrates a morepowerful use of the behavioural operators. For in-stance, where a scientific instrument is constantlyproducing data, it would be useful to restart thedata analysis application periodically to automati-cally analyse this data. To achieve this, we developa simulation script that includes the “Restart” Be-havioural operator, resulting in re-launching of theexecution every 20000 milliseconds. The restartingoperation can then be aborted at any time by callingthe “Terminate” Behavioural operator. The script iscreated with a text editor in declarative style, andconsists of references to the behavioural and struc-tural operators – applied on instances of patternswithin the Triana workflow. Each line in the scriptis used to launch a separate activity to manipulatethe workflow, and the script interpreter blocks un-

8

til a given activity completes. Elements such asInstantiate are used to map executable componentinstances to DummyUnits within a pattern template.It is also possible for the script to only partially spec-ify the contents of DummyUnits, allowing a user tomanually associate the remaining DummyUnits us-ing a graphical editor. The interpreter of the scriptis referred to as the “Pattern Controller” and workalongside the workflow enactor in Triana.

1: Initialize

2: Increase 1

3: Create Pipeline TransfVisSrv

4: RunStructuralScript TransfVisSrv

5: Instantiate DummyUnit

/toolboxes/SignalProc/Injection/Gaussian.xml

5: Instantiate DummyUnit1

/toolboxes/SignalProc/Algorithms/FFT.xml

6: Instantiate DummyUnit2

/toolboxes/SignalProc/Output/SGTGrapher.xml

7: EndStructuralScript

8: Embed TransfVisSrv DummyUnit1

9: Instantiate DummyUnit

/toolboxes/SignalProc/Input/Wave.xml

10:Instantiate DummyUnit2

/toolboxes/SignalProc/Output/Histogrammer.xml

11:Instantiate

DummyUnit3

/toolboxes/SignalProc/Output/SGTGrapher.xml

12:Restart

20000

The script is run by the pattern controller of a StarPT which performs the following steps (the numberreferences relate to the lines in the script above): a)creates the Star (1); b) adds one satellite to the nu-cleus (2); c) creates a Pipeline PT (named Trans-fVisSrv – line 3) and instantiates all its slots (calledDummyUnit(i) – lines 4-7); d) embeds the Transfor-mation and Visualisation service (TransfVisSrv) intothe first satellite (DummyUnit1 – line 8); e) instanti-ates the rest of the empty slots of the template (lines9-11); and, apply the behavioural operator Restart,in order to execute the instantiated Star every 20000milliseconds (12). The behaviour relies on the useof the Streaming Behavioural pattern supported bydefault.

Figure 10: Regular data production by the gravita-tional wave detection service

Figure 11: Producing different waves every 10 sec-onds

4.1.2 Production Use

The analysis of gravitational waves data example de-scribed in sub-section 4.1.1 does not take into account

9

the regular production of data by the wave detector.To simulate such a situation, we describe a config-uration (figure 10) where the Count tool producesdifferent values, at each execution, to the frequencyparameter of the Wave tool. Frequency starts at avalue of 100Hz, and it is increased at each executionby 100Hz upto a maximum of 4000Hz. Consequently,the Wave tool produces different waves which can bevisualised in the SGTGrapher tool.

Figure 12: Two different waves produced at two con-secutive execution steps

In the same way as described in section 4.1.1, theRestart Behavioural operator can be applied to thesimulation in order to generate a sequence of differentwaves at a fixed time period (10 seconds). Figure 12shows two consecutive snapshots of the SGTGraphertool.

Figure 13: Selection of the Terminate Behaviouraloperator

The automatic re-execution can be stopped at any

Figure 14: The debug window showing the executionof the Terminate Behavioural operator

time by applying the Terminate Behavioural operator(figures 13 and 14). The Count tool remembers theintermediate value for the frequency parameter of thelast execution. Therefore, the user may, for example,repeat the execution a certain number of times, byrestarting from the previous saved frequency value.

Figure 15 (on page 15) shows the selection of theRepeat Behavioural operator for repeatedly launchingthe execution of the simulation, by a certain numberof times (in this case, 10 times). In this way, the usercan see the result after each consecutive iterations.The debug window in the figure shows that the Repeatoperator was repeatedly called.

The second simulation involving frequent data pro-duction could also be achieved by modifying the con-figuration in figure 9 – containing a pipeline PT asthe nucleus of the star PT – connecting the Countand Wave components. Modification in this examplecould be achieved through the use of structural oper-ators, for instance, the Increase structural operatorcould be used to extend the pipeline PT, thereby pro-viding another DummyUnit to host the Count com-ponent. An alternative would be to use the Replaceoperator to swap the existing pipeline PT with an-other one containing two DummyUnits instead of one.

10

4.2 Database Access

As in astrophysics, other scientific experiments in En-vironmental and Genetic Sciences, Nuclear Physics,or Earth/Ocean Surface Topography also require thedistributed storage of data across different organisa-tions, and their subsequent manipulation by manyusers. One common characteristic of such applica-tions is database enquiry, where data may be spreadover several databases. Such databases may be pop-ulated through simulation engines or directly via sci-entific instruments producing data in real time. Wedemonstrate how the Facade PT may be used to ac-cess several databases within a workflow, where thedatabases may either be replicated, or store differ-ent types of data that needs to be aggregated. TheFacade PT therefore provides a uniform interface tosend a query to these databases. Behaviour associ-ated with such a pattern may involve the redirectionof queries to a particular database (based on contentor request type), or based on Quality of Service is-sues, such as response time.

Figure 16: Connecting the Facade pattern to two sub-workflows

Figure 16 shows the Pipeline Structural patternconnecting the client application (Requester) to theFacade Structural pattern in Triana. The latterredirects requests to two subsystems already instan-tiated with two structural patterns: Pipeline andPipeline1. Both pipelines configure possible as-

sociations of databases to data analysis/processingtools. In terms of behaviour, a simple version ofthe Client/Server Behavioural Pattern represents thedata and control flows between the Requester (client)and the Facade (server): the server analyses the re-quests and redirects them to the subsystems which, inturn, produce data. Additionally, the Master/Slaverepresents another eligible Behavioural patterns torepresent the data and control flows between the Fa-cade and the two sub-workflows.

Figure 17: Internal Structure of Pipeline sub-workflows connected to Facade Pattern

The sub-workflow represented by the Pipeline pat-tern in figure 17 contains, as its first element, the DB-Explore component to query a database using SQL [?]requests. The output is processed by the Make-Curve and displayed with the GraceGrapher compo-nent. The Pipeline1 sub-workflow provides access toa different or a replicated database combined withanother visualisation tool (Histogrammer) for out-put data analysis. Data and control flows in bothpipelines is provided by the Streaming Behaviouralpattern.

4.2.1 Real Time Analysis

To access real time data, instead of that in a pre-populated database, a user can re-configure the op-eration in section 4.2 to interact with a Real TimeEngine (RTE). To achieve this, the Stop Behavioural

11

pattern is applied to the Pipeline Structural patterninvolved, followed by an application of the ExtendStructural Operator to the Facade Structural pattern– to redirect requests to a RTE in addition to thedatabase. As a result, the existing Facade becomesa sub-workflow of the new Facade pattern, and thepipeline containing the RTE becomes the other sub-workflow. In this way, behaviour associated with theoutmost Facade may redirect requests to the inner-most Facade pattern or to the RTE, or both. TheClient/Server Behavioural Pattern is used to definethe data and control flows between the outmost Fa-cade and its own sub-workflows. Figure 18 illustratesthe resulting configuration within Triana.

Figure 18: Re-direction to a RTE

5 Mapping to the DRMAAAPI

Execution management associated with the be-havioural patterns and operators needs to be sup-ported via an enactment engine. To achieve this,we map our behavioural operators using a resourcemanagement API. Hence, behavioural patterns areimplemented over the run-time system used to exe-cute the components. There is no visual representa-

tion of these, as they are provided as a collection ofscripts that need to be configured by a user priorto execution. Current implementation work is fo-cused on the mapping of behavioural patterns overcurrently available APIs, such as Java CoG Kit [9]and DRMAA [8]. In this section, we describe howstructural and behavioural operators are mapped tothe DRMAA API – which can be used with the SunGrid Engine. DRMAA provides a generalised API toexecute jobs over Distributed Resource ManagementSystems (DRMSs). It includes common operationson jobs like termination or suspension. A job is arunning application on a DRMS and it is identifiedby a job id attribute that is passed back by the DRMSupon job submission. This attribute is used by thefunctions that support job control and monitoring.DRMAA API uses an interface definition language(with IN, OUT and INOUT parameters), and alsoprovides support for handling errors (via error codes).

Figure 19 shows the steps to configure and exe-cute an application based on the pipeline structuralpattern combined with the dataflow behavioural pat-tern. In step 1, a user defines a pipeline patterntemplate with three elements, and in step 2 addsan extra element to the pattern template. In step3 the dataflow behavioural pattern to be applied toall the elements of the pattern template is selected.An entity at the pattern level is defined, the patterncontroller/executor, responsible for enforcing the se-lected behavioural pattern at each element. In step4 all component place-holders are instantiated withcomponents (Applications) that may represent a unitin Triana or a group of units organized in a work-flow. Step 5 represents the application of the be-havioural operators to a pattern instance. The op-erators are supported by functions in the DRMAAAPI that manage the execution of the Applicationsby a resource manager. The execution of each Appli-cation is supported by a job (running executable) inthe resource manager.

Figure 19 shows how behavioural operators actupon pattern instances – essentially pattern tem-plates combined with some behavioural pattern andinstantiated with executable applications. Applica-tion execution using DRMAA require the definitionof attributes like the application’s name, its initial

12

Pattern Template

Structural Pattern

(structural composition)

1−

4− Instantiation of the

pattern elements

(component place−holders)

with executable applications

(App1, ..., App4)

Increase(P,1)

job4job3job2job1

Pattern Instance

Start, Stop, Resume, Terminate,

Limit, Restart,... 5−

Functions of a Distributed Resource Manager

Pattern controller

Pattern Instance

App1 App2 App3 App4

Pattern controller

Pattern Template

Pattern controller

2−

3−

Application of a

structural pattern

Application of a

Behavioural pattern

(increase)

−− dataflow model:

control dependencies

data flow dependencies

enforce the behavioural pattern

supported at each pattern element

by the wrapper

Creation of jobs that

support the execution

of the applications at

a resource manager.

Application of the

behavioural operators

to the pattern instance.

The "pattern controller" will

Figure 19: The necessary steps to configure and exe-cute an application using patterns and pattern opera-tors. Please read the figure starting from the bottom.

input parameters, the necessary remote environmentthat has to be set up for the application to run,and so forth. These attributes are used to explic-itly configure the task to be run via a particularresource manager. Although DRMAA has the no-tion of sessions, only one session can be active at atime. A single DRMAA session for all the opera-tors is assumed. Hence drmaa init and drmaa exitroutines are called, respectively, after the pattern in-stance is created and in the end of the script pro-

gram. As an example, we show how a pipelinepattern can be mapped to DRMAA: Elementpattern elements[MAX ELEMS] – contains the Ele-ments that compose a specific pattern instance. Sim-ilarly, job identifiers[MAX ELEMS] represents theidentifiers returned by the drmaa run job routine forjobs created to support pattern elements. The or-der of the activities is preserved. DRMAA variablesfrequently used: INOUT jt is a job template (opaquehandle), and INOUT drmaa context error buf con-tains a context-sensitive error upon failed return.

Start Operator – to initiate execution of PipelineElements.

for(int index = Pipeline.pattern_elements.length -1

; index >= 0;

index --) {

/*launch all activities in the pipeline*/

int ret = drmaa_allocate_job_template(

jt, drmaa_context_error_buf );

process_error( ret, drmaa_context_error_buf );

define_attributes(

jt, Pipeline.pattern_elements[index] );

/* Pipeline.startTime defines the time at which all

elements in the pipeline instance

should start running. */

ret = drmaa_set_attribute( jt, drmaa_start_time,

Pipeline.startTime,

drmaa_context_error_buf);

process_error( ret, drmaa_context_error_buf );

/* run job */

ret = drmaa_run_job(

job_id, jt, drmaa_context_error_buf );

process_error( ret, drmaa_context_error_buf ); }

Repeat Operator – in this instance a single opera-tor is used to re-execute an entire pattern instance acertain number of times ( “n” in the code).

for(int count = 0;

count < n; count++) {

Start( Pipeline );

/* wait for all the jobs that compose

the pipeline to terminate */

drmaa_synchronize(

Pipeline.job_identifiers, timeout, 0,

drmaa_context_error_buf );

/*timeout is bigger than all

job execution times */ }

13

6 Conclusion and Future Work

The extension of a Problem Solving Environment(Triana) with Patterns and Operators is described.Composition is achieved using a pattern extendedgraphical interface provided with Triana – whereasexecution is managed by mapping Operators to theDRMAA API. We believe a Pattern based approachis particularly useful for reuse of component librariesand for mapping applications to a range of differentexecution environments. The DRMAA API was se-lected because of the significant focus it has receivedwithin the Grid community – and the availability ofcommercial resource management systems (such asGrid Engine from Sun Microsystems) that make useof it. We are also investigating alternatives to DR-MAA (such as Java CoG) [9] – primarily as currentversions of DRMAA are aimed at executing batchjobs. With the emerging focus on Web Services in theGrid community, the DRMAA API has also laggedbehind other equivalent developments (such as theJava CoG kit).

Patterns and operators provide a useful extensionto existing workflow engines, as they enable the cap-ture of common software usage styles across differ-ent application communities. The pipeline and starstructural patterns, for instance, are commonly foundin scientific applications (such as integrating a datasource with a mesh generator, followed by a visu-aliser). Describing such compositions in a more for-mal way (as we have attempted to do here), willenable practisioners to identify common software li-braries and tools. This is particularly important assoftware that performs similar functionality is avail-able from a variety of different vendors. Providingthe right balance between tools that require users topossess programming skills, and those that are basedon a visual interface is difficult to achieve. By com-bining the visual interface of Triana with more ad-vanced patterns and operators, we are attempting toenhance the functionality offered through (a varietyof) existing workflow tools. Full usage of these ideasby the applications community is still a future aimfor us.

Patterns and operators also provide an importantmiddle ground between mechanisms to automatically

composed workflows – using planning approaches,and workflows that are constructed manually by auser. The planning approach requires an applicationdeveloper to semantically annotate components anddevelop complex models that represent possible in-teractions between such components. Often this isa challenging task and hard to achieve in practise.On the other hand, allowing a user to construct aworkflow manually ignores the number of commoninterconnection structures that existing within scien-tific computing applications. Using a combinationof both structural and behavioural operators, we areable to address both build and run time characteris-tics associated with workflow management.

Acknowledgement: We would like to thank IanWang and Matthew Shields of the Triana Group inCardiff for their help with Triana internals.

References[1] M. Li and M. A. Baker, “A Review of Grid Portal Tech-

nology”, in Book, “Grid Computing: Software Envi-ronment and Tools” (ed: Jose Cunha and O.F.Rana),Springer Verlag, 2006

[2] J. Yu and R. Buyya, “A Taxonomy of Workflow Man-agement Systems for Grid Computing”, Journal of GridComputing, Vol:3, pp 171-200, 2006.

[3] J. Blythe, E. Deelman, Y. Gil, C. Kesselman, A. Agarwal,G. Mehta and K. Vahi, “The Role of Planning in GridComputing”, 13th Int. Conf. on Automated Planning andScheduling (ICAPS), Trento, Italy, 2003.

[4] G. Fox, D. Gannon and M. Thomas, “A

Summary of Grid Computing Environments”,

Concurrency and Computation: Practice and

Experience (Special Issue), 2003. Available

at:http://communitygrids.iu.edu/cglpubs.htm

[5] B. Wydaeghe, W. Vanderperren, “Visual Composi-tion Using Composition Patterns”, Proc. Tools 2001,Santa Barbara, USA, July 2001.

[6] ObjectVenture, The ObjectAssembler VisualDevelopment Environment, Java Sys.-Con.Journal, June 1, 2003. Article available at:http://java.sys-con.com/read/37562.htm. Lastaccessed: July 2006.

[7] The GridLab project. See Web site at:http://www.gridlab.org/. Last accessed: Jan-uary 2004.

14

[8] Habri Rajic, Roger Brobst et al., “Distributed Re-source Management Application API Specification1.0”. Global Grid Forum DRMAA Working Group.See Web site at: http://drmaa.org/wiki/. Last ac-cessed: July 2006.

[9] Gregor von Laszewski, Ian Foster, Jarek Gawor,and Peter Lane, ”A Java Commodity Grid Kit,”Concurrency and Computation: Practice and Ex-perience, vol. 13, no. 8-9, pp. 643-662, 2001,http:/www.cogkits.org/.

[10] M.C.Gomes, O.F.Rana, J.C.Cunha “Pattern Opera-tors for Grid Environments”, Scientific ProgrammingJournal, Volume 11, Number 3, 2003, IOS Press.

[11] M.C.Gomes, J.C.Cunha, O.F.Rana, “A Pattern-based Software Engineering Tool for Grid Envi-ronments”, Concurrent Information Processing andComputing proceedings, NATO Advanced ResearchWorkshop, Sinaia, Romania, June 2003, IOS Press.

[12] I. Taylor et al., “Triana”(http://www.trianacode.org/). Triana is theworkflow engine for the EU GridLab project(http://www.gridlab.org/). Last Visited: January2004.

[13] The GEO600 project. See Web site at:http://www.geo600.uni-hannover.de/. Last ac-cessed: July 2006.

[14] E. Gamma, R. Helm, R. Johnson, J. Vlissides, “De-sign Patterns: Elements of Reusable Object-OrientedSoftware”, Addison-Wesley, 1994.

[15] O. F. Rana, D. W. Walker, “Service Design Pat-terns for Computational Grids”, in “Patterns andSkeletons for Parallel and Distributed Computing”,F. Rabhi and S. Gorlatch(Eds), Springer, 2002.

[16] I. J. Taylor, M. S. Shields, I. Wang and O. F. Rana,“Triana Applications within Grid Computing andPeer to Peer Environments”, Journal of Grid Com-puting, Vol:1, No:2, pp 199-217, 2003.

[17] W.M.P. van der Aalst, A.H.M. ter Hofstede, B. Kie-puszewski, and A.P. Barros. “Workflow Patterns”.Distributed and Parallel Databases, 14(3), pages 5-51, July 2003.

[18] A. Slominki, D. Gannon and G. Fox, “Introductionto Workflows and Use of Workflows in Grids and GridPortals”, Global Grid Forum 9, presented in “GridComputing Environments” session, Chicago, Octo-ber 2004.

[19] N. Russell, W.M.P. van der Aalst, A.H.M. ter Hof-stede, D. Edmond, “Workflow Resource Patterns:Identification, Representation and Tool Support”,Proceedings of CAiSE, pp 216–232, 2005.

[20] M. Pistore, F. Barbon, P. Bertoli, D. Shaparauand P. Traverso, “Planning and Monitoring WebService Composition”, Workshop on Planning andScheduling for Web and Grid Services, held alongside14th International Conference on Automated Plan-ning and Scheduling (ICAPS 2004), Whistler, BritishColumbia, Canada, June 3-7 2004.

[21] N. Milanovic and M. Malek, “Current Solutions forWeb Service Composition”, IEEE Internet Comput-ing, Vol:8, Issue 6, pp 51-59, Nov.-Dec. 2004.

[22] D. Roman, M. Dimitrov and M. Stolberg,“D14v0.1. Choreography in WSMO”, DERIWorking Draft 17 July 2004. Available at:http://www.wsmo.org/2004/d14/v0.1/20040717/.Last accessed: July 2006.

[23] “AstroGrid and the Virtual Observatory” Project.Details at: http://www.astrogrid.org/. Last ac-cessed: July 2006.

[24] J. Chen and Y. Yang, “Multiple States based Tem-poral Consistency for Dynamic Verification of Fixed-time Constraints in Grid Workflow Systems”, Con-currency and Computation: Practice and Experi-ence, 2006.

[25] “Laser Interferometer GravitationalWave Observatory” Project. Details at:http://www.ligo.caltech.edu/. Last accessed:July 2006.

[26] C. A. Herrmann and C. Lengauer, “TransformingRapid Prototypes to Efficient Parallel Programs”,book chapter in “Patterns and Skeletons for Paralleland Distributed Computing”, (Fethi A. Rabhi andSergei Gorlatch (Eds)), Springer Verlag, 2002.

[27] Murray I. Cole and Andrea Zavanella, “Coordinat-ing Heterogeneous Parallel Systems with Skeletonsand Activity Graphs”, Journal of Systems Integra-tion, 10(2), pp 127–143, 2001.

[28] S. Gorlatch, “Extracting and implementing list ho-momorphisms in parallel program development”, Sci-ence of Computer Programming, 33(1), pp 1–27,1998.

15

Figure 15: Applying the Repeat Behaviour Pattern for launching the execution ten times

16