Online Testing Framework for Web Services

Tien-Dung Cao1, Patrick Felix1, Richard Castanet1 and Ismail Berrada2

1LaBRI - CNRS - UMR 5800, University of Bordeaux 1351 cours de la liberation, 33405 Talence cedex, France.

Email: {cao,felix,castanet}@labri.fr2L3I, La Rochelle University, 17042 La Rochelle, France.

Email: ismail.berrada@univ-lr.fr

Abstract—Testing conceptually consists of three activities:test case generation, test case execution and verdict assignment.Using online testing, test cases are generated and simultane-ously executed (i.e. the complete test scenario is built during testexecution). This paper presents a framework that automaticallygenerates and executes tests “online” for conformance testingof a composite of Web services described in BPEL. Theproposed framework considers unit testing and it is based ona timed modeling of BPEL specification, a distributed testingarchitecture and an online testing algorithm that generates,executes and assigns verdicts to every generated state in thetest case.

Keywords-Composite of Web Services, BPEL, Test Genera-tion, Timed Extended Finite State Machine, Online Testing,Conformance Testing.

I. INTRODUCTION

Conformance testing is a commonly used activity toimprove the system reliability. It is a very time-consumingprocess that requires a great deal of expert knowledge ofthe systems being tested. The use of formal specificationsprovides a support for automating this process.

The ways to test a system can be classified into twobasic groups. The most natural way, namely the activetesting approach, consists of carrying out the test derivationfrom specifications. Test cases are then executed againstthe system under test and verdicts are assigned. Anotherpossibility is the passive testing approach. The absenceof controls and observations allow only the validation oftraces, and thus this approach checks that a trace of animplementation is a valid execution of the specification.

In general, Web services testing is a black-box test-ing where test cases are designed based on the interfacespecification of the Web services under test, not on itsimplementation. In this case, the internal logic does not needto be known, whether it’s coded in Java, C# or BPEL [1].On the contrary, business process unit (a composite of Webservices) testing can be classified in two kinds:• White-box approach. As BPEL is an executable lan-

guage, the BPEL description of a composite of Webservices is considered as the source code of the compo-sition. It can be executed by any BPEL engine (Active

BPEL, Oracle...). Test cases are then designed basedon the internal logic of the service under test.

• Gray box approach (we call gray-box to differencewith black-box testing of Web services because weknow that interactions exist between a service undertest and its partners). In this approach, a composite ofWeb service can be coded in a different language fromthe specification, for instance, a BPEL specification iscoded as a Java program (even BPEL). An implementa-tion of the composite Web service is then tested withoutany information of its internal structure. Test cases arethen generated only from the specification (WSDL [2]and BPEL).

Testing conceptually consists of three activities: test casegeneration, test case execution and verdict assignment. Us-ing online testing, test cases are generated and simultane-ously executed [24]. In this paper, we focus on model-based unit testing [3] of a composite of Web services givenby BPEL specifications. We consider conformance testingusing the gray-box approach . We present a framework thatautomatically generates and executes tests “online” (i.e. thecomplete test scenario is built during test execution) for acomposite of Web services described in BPEL. In order tomodel the BPEL behavior with the timing constraints anddata variables, the BPEL specification is transformed intothe Timed Extended Finite State Machines (TEFSM) model.This formalism is closely related to timed automata [19]and permits to carry out timing constraints, clocks, stateinvariants on clocks and data variables. From this model, wepropose an online testing algorithm to automatically generatetest cases, and simultaneously execute them to issue verdictsbased on distributed testing architecture that is also proposedin this framework.

The rest of paper is organized as follows. Section II re-views some previous works on testing of a composite of Webservices. In section III, we give the definition of TEFSMthat is used to model BPEL process, and some relatednotations. Section IV presents an overview of BPEL con-cepts. Section V describes the relationship between BPELconcepts and TEFSM. Section VI presents a framework that

includes a timed modeling of BPEL specification, a testarchitecture, and an online testing algorithm. A prototypetool is describled in the section VII. Finally, section VIIIconcludes the paper and presents future works.

II. RELATED WORKS

In the last years, several techniques and tools have beendeveloped to test Web services. Various approaches forservice composition testing were analyzed by [3] includingunit testing, integration testing, black-box testing and white-box testing of choreographies and orchestrations.• Testing BPEL descriptions. Jose Garcia-Fanjul et

al [7] use the SPIN model checker to generate testcases specification for compositions given in BPEL. Inorder to systematically derive test suites, the transitioncoverage criterion is considered. Yongyan Zheng andPaul Krause [8, 10] model each BPEL activity by anautomaton (also referred as Web Service Automaton).These automata are then transformed into Promela, theinput format of the SPIN model checker. [11] uses onemore time the SPIN model checker to verify BPELspecification. However, the authors do not transformBPEL directly into Promela as in [7]. BPEL will betranslated to guard conditions which are transformedto Promela. In all of these methods, test cases aregenerated from counterexamples given by the SPINmodel checker. Transforming BPEL into IntermediateFormat Language (IF) is presented in [15] and [16].Timed test cases are generated using TestGen-IF tool[30]. [28, 29] present also the frameworks for white-boxtesting . However, the authors do not consider automatictest case generation [16].

• Testing WSDL descriptions. [5] present a test casesgeneration approach based on WSDL descriptions. R.Heckel and L. Mariani [21] also present a test casesgeneration approach based on the definition of rulesfor each service. S. Salva and A.Rollet [22] propose amethod that is able to check the following aspects: op-eration existence, exception management, and sessionmanagement. Some of open source tools for testingWSDL descriptions of Web services were developedsuch as soapUI [32] and TestMaker [33]. In [4], theauthors propose a framework that extends UDDI (Uni-versal Description, Discovery and Integration) registryrole from the current one of a passive service direc-tory, to sort of an accredited testing organism, whichvalidates service behaviour before actually registeringit.

Regarding online testing framework, [24, 25] present theT-Uppaal tool for model based testing of embedded real-timesystems. T-Uppaal automatically generates and executestests from the state machine model of the implementationunder test (IUT) and assumes that environment specifies therequired and allowed observable (real time) behaviors of the

IUT. Finally, in [25] and [27], the authors introduce the TorXtool which is based on a timed extention of the ioco testingtheory. These works consider the input/output actions andtiming constraints, but do not consider the data (the valueof messages). However, we cannot use these frameworksfor testing of a composite of web services because theydo not support the SOAP messages: its input format (timedautomata, Promela, LOTOS ...) do not support data variableor its data type is very poor. Moreover, they can not alsosimulate a partner as a web services.

III. PRELIMINARIES

BPEL specification can be described by means of formalmodels such as TEFSM [14, 19] (Timed Extended FiniteState Machine). In this section, we introduce the TEFSMmodel and some related definitions.

Clocks and Constraints: A clock is a variable that allowsto record the passage of time. It can be set to a certain valueand inspected at any moment to see how much time haspassed. Clocks increase at the same rate, they are rangedover IR+, and the only assignments allowed are clock resetsin the form c:=0. For a set C of clocks, and a set V ofvariables, the set of clock constraints Φ(C) is defined by thegrammar: Φ := Φ1|Φ2|Φ1 ∧ Φ2,Φ1 := c ≤ m,Φ2 := n ≤ cwhere c is a clock of C, and (n, m) are two natural numbers.P(V) is a set of linear inequalities on V, (c0, c1, ..., cn )(resp. (v0, v1, ..., vm)) will be denoted by ~c (resp. ~v).

Definition 1: (TEFSM): A TEFSM M is a tuple, M = (S,s0, V, E, C, Inv, T) where:• S = {s0, s1, ..., sn}, is a finite set of states;• s0 ∈ S is an initial state;• V is a finite set of data variables, D|V |V is the data

variable domain of V;• E is a finite set of events. E is partitioned into:

– Input event ?a where: the form ?pl.pt.op.msg: thereception of the message (msg) for the operator(op) of the portTyte pt from the partner (pl),represents a format of a;

– Output event !b where: the form !pl.pt.op.msg: theemission of the message (msg) for the operator (op)of the portTyte pt to the partner (pl), represents aformat of b;

– τ is the internal event.• C is a finite set of clocks including a global clock gc

(never reset);• Inv: S 7→ Φ(C) is a mapping that assigns a time

invariant to states;• T ⊆ S × E × P (V ) ∨ Φ(C)× 2C × µ× S is a set of

transitions where:– P (~v)&φ(~c): are guard conditions on data variables

and clocks;– µ(~v): Data variable update function where µ :V 7−→ D

|V |V ;

– X ⊆ 2C : Set of clocks to be reset;• M is a deterministic machine;A transition t = (s < e, [g], {f ; c} > s′) ∈ T represents

an edge from state s to state s′ on event e. g is a set ofconstraints over clocks and data variables, f is a set of dataupdate function, and c is a set of clocks to be reset.

Definition 2: (Partial of TEFSM): Let M be a TEFSM.The partial machine of M is defined by PM = (S, sin, Sout,V, E, C, Inv, T) where: (S, sin, V, E, C, Inv, T) is a TEFSMand Sout ⊂ S.

A partial of TEFSM is a TEFSM with an input state sin(representing the entering state of the partial machine andwhich replaces the initial state s0) and a set of output states,Sout (representing the exit state of the partial machine).

IV. BPEL CONCEPTS

BPEL [1] provides constructs to describe complex busi-ness processes that can interact synchronously or asyn-chronously with their partners (each partner is a Web service:basic or composite). Like any programming language, BPELhas the basic activities as basic commands to interact withits partner (receive, reply, invoke) or internal interaction(wait, exit, empty, throw, assign), and typical control struc-tures including sequence, switch, while, etc. In addition,BPEL uses the flow construct to provide concurrency andsynchronization. Synchronization and concurrency amongactivities are provided by means of links. Each link canhave a source activity and a target activity. Furthermore,a transition condition, which is a boolean expression, isassociated with each link and is evaluated (true or false)when the source activity terminates. Each activity of a flowmay be exist a join condition, which consists of incominglinks of the activity combined by boolean operators. Onlywhen all the values of its incoming links are defined (true)and its join condition evaluates to true, an activity is enabledand can start. Otherwise if the join condition evaluates tofalse, the activity will not be executed. A BPEL processalways starts with the process element (i.e the root of theBPEL document). It is composed of the following children:partnerLinks that declares the partners, variables, activitiesand the optional children: faultHandlers, eventHandlers, cor-relationSets. These children are concurrent. A scope has theproperties as a sub-functions of any programming languageto execute a sub-process. BPEL uses a correlationSetproperty as a sessionId of the partners upon the value ofdata variables. The figure 1 shows an example of BPEL withthree partners that is described in figure 8.

V. RELATIONSHIP BETWEEN BPEL CONCEPTS ANDTEFSM

In our framework, we only model the activities of a BPELprocess because we are only interested in the input/outputmessages, the timing constraints of these messages and the

Figure 1. A BPEL example

conditions specification of input variables. An internal faultwill be assigned a fail verdict. We use a stop variable forthe machine of process to terminate (assign to true) the restactivities if the termination is activated by an exit activity orthe throw activity. We use only a local clock c and a globalclock gc. The scope activity will be modeled as a process.The property correlationSet will be not considered becausewe focus on single session testing. Most of activities aremodeled as [14].

A. Messages

A BPEL variable is always connected to a message froma WSDL description of partners. In BPEL, a Web service (apartner) that is involved in the process is always modeledas a porType pt (i.e. abstract group of operations (noted op)supported by a service). These operations are executed viaa partnerlink (noted by pl). In our formalism, for instance,the input message ?pl.pt.op.v denotes the reception of themessage op(v) (constructed from the operation op and theBPEL variable v) via the channel pl.

B. Basic activities

A basic activity can be one of the following: receive, reply,invoke, assign, wait, empty, exit, throw. Each basic activityis described by a partial machine ( denotes τ ).Receive Activity: <receive partnerLink=pl portType=pt op-eration=op variable=msg>

PM = ({sin, sout}, sin , {sout}, {v, stop},{?pl.pt.op.msg}, {c}, {(sin, true), (sout, true)}, {t1})• t1=(sin,<?pl.pt.op.msg,[stop=false],{v=msg;c}>,sout)

Reply Activity: <reply partnerLink=pl portType=pt opera-tion=op variable=msg>

PM = ({sin, sout}, sin , {sout}, {stop}, {!pl.pt.op.msg},{c}, {(sin, true), (sout, true)}, {t1})• t1=(sin,<!pl.pt.op.msg,[stop=false],{c}>,sout)

Assign Activity: <assign> <from> v2 </from> <to> v1

</to> ... </assign>PM = ({sin, sout}, sin , {sout}, {v1,v2,...,vn,stop}, ∅, {c},

{(sin, true), (sout, true)}, {t1})• t1=(sin,< ,[stop=false],{v1=v2,...;c}>,sout)

Wait Activity: <wait (for=d | until=dl)>.• <wait for=d>: PM = ({sin, sout}, sin , {sout}, {stop},{ }, {c}, {(sin,c≤d),(sout, true)}, {t1})

– t1=(sin,< ,[c=d & stop=false],{c} >,sout)• <wait until=dl>: PM = ({sin, sout},sin , {sout},{stop}, { }, {gc}, {(sin,gc≤dl), (sout, true)}, {t1})

– t1=(sin,< ,[gc=dl & stop=false],∅ >,sout)Throw Activity: <throw faultName = fault faultVariable =data/>

PM = ({sin, sout}, sin, {sout}, {stop}, { }, ∅,{(sin, true), (sout, true)}, {t1})• t1=(sin,< ,[],{stop=true,fault=data}>,sout)

Exit Activity: <exit/>PM = ({sin, sout}, sin , {sout}, {stop}, { }, ∅,

{(sin, true), (sout, true)}, {t1})• t1=(sin,< ,[],{stop=true}>,sout)

Invoke Activity: <invoke partnerLink=pl portType=pt oper-ation=op inputVariable=msg in outputVariable=msg out>

PM = ({sin, s1, sout}, sin , {sout}, {v in, v out, stop},{!pl.pt.op.msg in, ?pl.pt.op.msg out}, {c}, {(sin, true),(s1, true) (sout, true)}, {t1, t2})• t1=(sin, <!pl.pt.op.msg in, [stop=false], {c}>, s1)• t2=(s1, <?pl.pt.op.msg out, [stop=false], {v out =

msg out; c}>, sout)Empty Activity: <empty/>

PM = ({sin, sout}, sin , {sout}, {stop}, ∅, {c},{(sin, true), (sout, true)}, {t1})• t1=(sin,< ,[stop=false],{c}>,sout)

C. Structured activities

Structural activities are the sequence, while, switch, flow,pick, repeatUntil, if and scope. They take some partialmachines PMi,i∈[0,n] and combine them to have a newpartial machine.

The partial machines (set of states and transitions) ofstructural activities (sequence, while, switch and pick) areshown in Fig 2. In partial machine of sequence, we renamethe sout of PMi with sin of PMi+1. The set of variables,events, clocks are:

V = {v0, .., vm} ∪ V0 ∪ ... ∪ Vn,

E = {e0, .., el} ∪ E0 ∪ ... ∪ En,

where v0, .., vm is the new variables that uses to control thestructure (resp e0, .., el).

Figure 2. Modeling structural activities

In our framework, the repeatUntil activity will be modeledas a while activity. The conditional behavior if will bealso modeled as a switch activity, and the eventHandleractivity will be modelled as a pick activity. The flow activityallows describing one or more concurrent activities [1]. Itspecifies the parallel execution of flow corresponding topartial TEFSMs. The partial machine of a flow finishes whenall of its sub-partial machine finish. So when a sub-partialmachine finishes, it changes into a temporary state to waitthe rest of sub-partial machine finish before moving intosout. We choice this temporary state is sin of flow’s partialmachine. This is easy to transform from a flow activity toa sequence activity because the test framework can work asa sequence upon time order while it receives the parallelrequests. We use a boolean variable for each sub-partialmachine to examine the termination of each machine. Theinitial value of these variables is false. The links defined inthe flow activity allows to enforce precedence between theseactivities, i.e. it allows synchronization. We add a transition

to enforce precedence between these partial machines. Thepartial machine of a flow activity and link variables areshown in Fig 3.

Figure 3. Modeling Flow activity and Link variables

D. Limitations

Our framework has the following limitations: the at-tributes joinCondition, supressJoinFailure of the flow activ-ity are not treated. An activity with correlation property willbe modeled by adding a variable status of properties as in[15]. In many cases (not all), may be we do not use thecorrelation property because we test only single session.

VI. ONLINE TESTING FRAMEWORK

A. Conformance Relation

In order to capture the notion of conformance betweenimplementations under test (IUT) of a Web service and itsspecification, we will use a conformance relation. Beforedoing this, we need some additional notations:• The action transition (discrete transition), denoted bys

a→ s′ such that sa[g]{u}−→ s′ ∈ T , indicates that

if the guard (on variables and clocks) g is true, thenthe automaton fires the transition by executing theinput/output action a, changing the current values of thedata variables by executing all assignments, changingthe current values of the clocks and timers by executingall time setting/resetting, updating the buffers contentsof the system by consuming the first signal required byinput actions and by appending all signals required byoutput actions, where u is a set of update functions,and moving into the next state s′.

• The timestamps transition (timing transition), denotedby s

c=d→ s′ where c is a local clock (or sgc=dl→ s′

where gc is a global clock), indicates that TEFSM willmove to state s′ when local clock c reaches a timeduration d (or global clock gc reaches a deadline (dl)),for example, the transition of the wait activity.

For a ∈ E, we write s a→, iff ∃s′ ∈ S such that s a→ s′.We write s

a1,...,an−→ s′ iff ∃s1, s2, ..., sn−1 ∈ S such thatsa1→ s1

a2→ s2...sn−1an→ s′. We write s a⇒, iff ∃s′, s′′, s′′′ ∈

S such that sτ,...,τ−→ s′′

a→ s′′′τ,...,τ−→ s′. We define Γ = (E ×

IR+) as the set of observable actions (actions and delays) andΓτ = (Eτ × IR+) as the set of observable actions includinginternal actions (i.e. Eτ = E ∪{τ}). A timed sequence σ ∈Seq(Γ) is composed of actions a and non-negative reals d

such that: s(a,d)→ s′⊕d, where d is a timing delay. Let Γ′ ⊆ Γ

and σ ∈ Seq(Γ) be a timed sequence. πΓ′(σ) denotes theprojection of σ to Γ′ obtained by deleting from σ all actionsnot present in Γ′. The observable timed traces of an IUT isdefined by: Trace(IUT ) = {πΓ(σ)|σ ∈ Seq(Γτ ) ∧ s0

σ⇒}.Definition 3: (Conformance Relation): An implementa-

tion of Web services under test (denotes IUTI ) is conformto its specification (denotes IUTS) ⇐⇒ ∀Trace(IUTI),∃Trace(IUTS) such that Trace(IUTI) ⊆ Trace(IUTS).

An implementation of Web services under test is conformto its specification, Iff for each timed trace that are generatedby the implementation, we must find at least a timed tracebelonging to its specification.

B. Test architecture

Usually, to perform unit testing of a composite Webservice, it is necessary to simulate its partners. This is dueto following reasons [29]:

1) In unit testing phase, some of the dependent partnerservices are still under development.

2) Some partner services are developed and governed byother enterprises. Sometimes it is impossible to obtainthe partner services’ source code and related modules,and set up the running environment for the testing.

3) Even we have had some partner services ready for use,simulated ones are sometimes still preferred becausethey could generate more interaction scenarios withless effort.

4) Another case, we cannot test directly some partnerservices that are ready for use because it generatesthe data perturbation. For example, we cannot senda soap message to the bank service to create a newaccount while we test our service.

It is straightforward to derive a test architecture that de-scribes the test environment consisting of simulated services.A Web service under test (SUT) and its partners (includingalso client) communicate by exchanging input and outputmessages. When the SUT is being tested, the tester playsthe role of the environment (i.e. its partner services).

In our work, we consider the tester as a Web servicebecause the partners of SUT are also Web services. A Webservice tester contains one controller component and a set oftest execution components that represent the partner servicesof SUT. The controller will generate test cases (sequencesof input/output message and timing delays) and send/receiveinput output message to/from each test execution component.Each test execution component may receive a message fromthe controller and sends it to SUT, or receives a messagefrom SUT and forward it to the controller. The controlleranalyses the result and assigns a verdict to the test caseexecution. We assume here that SUT has N partner servicesand one client. The test architecture is shown in figure 4.

Figure 4. The test architecture

C. Online testing algorithm

In offline testing, test cases are generated from the modelwhere the complete test scenarios and verdicts are com-puted before execution. In contrast, online testing [24–27]combines test generation and execution: only a single testprimitive is generated from the specification at a time, andexecuted on the SUT. An observed test run is a timedtrace consisting of an alternating sequence of input/outputmessages and timing delays.

The tester can perform the basic actions: either send aninput to the SUT, to do a delay for a time synchronizationwith SUT, or wait for an output for a duration d. If thetester observes an output at time d′ ≤ d, then it checks

whether this is legal according to the specification. If not,a timeout fault will be raised as a fail verdict. In thecase of flow activity, may be several outputs occur for aduration d. So that, we use a queue (Q) to save all theseoutputs. The tester will wait an output by checking if thisqueue is empty. The variable values will be updated bythe data from input/output message. Note that, in a givenstate, the guard condition of a transition has one of threevalues: true, false or undefined when the variables valueson guard are undefined. If the invariant of the state s isc ≤ m (c is a local clock), then the value of the functioninvariant(s) will be m (i.e. invariant(s) = m). The onlinetesting algorithm for a composite of Web services is shownin the figures 5 and 6. This algorithm will assign a failverdict for the current trace if exists an output or delaythat do not belong to any observable timed traces of itsspecification (i.e. ∃Trace(IUT I),∀Trace(IUT S) suchthat Trace(IUT I) /∈ Trace(IUT S)). A pass verdictwill be assigned iff a timed trace arrives a final state ofits specification.

function getNextAction(State s) // list of transitionsBEGINresult := ∅; // transitions listqueue := {t ∈ T |t.src = s ∧ t.guard! = false};WHILE queue 6= ∅ DOtrans := queue.pop()T := T \ {trans};IF trans is an input output transition or a

timestamps transition THENresult.add(trans);

ELSEtemp := {t ∈ T |t.src = trans.target

∧t.guard! = false};queue.add(temp);

ODreturn rerult;

END

Figure 6. Test generation and execution (part 2/2)

D. Testing framework design

In this section, we present an example of the frameworkimplementation design. We have implemented our frame-work on a local machine (i.e. the test execution componentsand the controller component are installed on the samemachine). The architecture is shown in Fig 7. The frameworkconsists of six main elements:

1) compiler: loads and compiles input format (TEFSM);2) analyzer: analyze the WSDLs to extract informations

of partner services about: operator, port, synchronous,asynchronous etc;

Online Testing AlgorithmInput: - M = (S, so, V, E,C, Inv, T ) //specification of the SUT

- Test execution number : N .- Time to wait for an output message from the SUT: tmax; //network timeout

Output: Timed test cases suite and its corresponding verdicts.BEGIN

state := s0; //current statecounter := 0;tcList := ∅; // a test case listtc := ∅; // a test caseQ := ∅; // a queue to keep all SOAP message that are forwarded by test execution componentsWHILE counter ≤ N DO

- acList := getNextAction(state); //list of transitions, figure 6IF acList = ∅ THEN //arrives the final state

IF tc 6= ∅ THEN //founds a test case- tcList.add(tc);- tc.empty();

- state := s0 //restart (search a new test case)- counter + +;

ELSEIF itList = acList.inputTrans() ∪ acList.timedTrans() 6= ∅ THEN

- randomly choose t ∈ itList;IF t ∈ acList.inputTrans() THEN

- i msg := generate data(t.action); //generate data for input message- randomly delay n times units based on guard condition of t on clock- send to test execution(i msg); // send an input message to SUT- update variable(i msg); // update variables with the input message value- tc.add(i msg, n)

ELSE // t ∈ acList.timedTrans() (i.e. timestamps transition)- delay d times units- tc.add(delay = d) //add delay into test case

- state := t.target // moving into new stateELSE //waiting an output message from the SUT

- d := invariant(state) = true?tmax : invariant(state)- sleep for d time units or wake up if Q is not empty at d′ ≤ dIF (o msg = Q.pop()) is not null THEN

//find a transition t such that t.action = o msg from current state;IF search(o msg, state) = true THEN

- update variable(o msg);- state := t.target // moving into new state- tc.add(o msg, d′)

ELSE- exit(); //current test case is fails

ELSE- exit(); //current test case is fails (clock constraints)

ODEND

Figure 5. Test generation and execution (part 1/2)

3) test execution that sends and receives the SOAP mes-sages to/from the SUT;

4) data generation from the WSDLs and XML Schema;5) data update function library; and6) the controller managing the test execution components

and the data generation.

Figure 7. Online testing framework architecture

In our framework, we use the TEFSM as an in-put specification. Hence, we develop a prototype (calledBPEL2TEFSM) using rules of section V to transform theBPEL description into a TEFSM specification.

1) Test execution component: Each test execution compo-nent represents a partner service (i.e. simulating partner ser-vice). However, we do not know its internal structure, so, wewill generate automatically its output messages (of partnerservices) based on the message type of WSDL specification.Constraints on variable will be applied on these outputmessages to satisfy a test case selection. In our framework,the role of each test case execution is very simple. Each testexecution component receives the SOAP message from thecontroller and sends it to SUT by setting SOAP messageinto the queue Q, and it receives a SOAP message fromSUT and forward it to the controller. All of test executioncomponents have the same role, but we created one testexecution component for each partner service because it hasa different address. These test execution components arecreated by the controller based on the partner link numberof BPEL specification.

2) Data generation: We reuse the code of SoapUI [32]to generate SOAP format. The data for each field of SOAPmessage is randomly generated or use a default value. Thisdepends on the configuration file. This component will be

controlled by the controller within the generate data()function of figure 5.

3) Controller: The controller implements the online test-ing algorithm of figures 5 and 6. As the tester is a webservice (asynchronous), we need a WSDL specificationfor the tester. In this WSDL specification, we define twofunctions:• start(TEFSM M, File[]wsdl, int N, int tmax)

to receive a test request from the client with fourparameters: a TEFSM M that describles the behaviourof the web services, a wsdl file list of partners, testexecution number N , and network timeout, tmax, forsynchronous action;

• finished() to return the result that is a list of test caseand its verdict;

VII. A PROTOTYPE TOOL

A. WSOTF tool

In addition to the theoretical framework we have devel-oped a prototype tool, called WSOTF (Web Services, OnlineTesting Framework), that helps in the automation of ourtesting approach. In particular, the algorithm presented inthis paper (fig. 5 and 6) are fully implemented in the tool.This tool is implemented by Java. We use a http server asthe web services to handle the output message of SUT. Itsinput format is a xml file that describles a TEFSM. Thisfile also indicates the location of wsdls. This tool supportscurrently SOAP binding type ”document” and the type ofinternal varibales that use to control the behaviour are int,boolean.

B. An example of online testing result

In this section, we illustrate our framework using theLoan Web Service (fig. 8). This process receives an inputfrom the client. If this input is less than 10, it invokes thesynchronous Assessment Service and receives a risk result.In the case, this risk is low, so it sends a yes response yes tothe client. Otherwise (i.e. input≥10 or risk!=low), it invokesthe asynchronous Approval Service by sending a request anduses a BPEL pick activity for one of the following cases: (1)to receive an asynchronous response from the partner serviceand send this response to client; (2) to send a timeout faultto client if there is not response from the partner serviceafter a duration (e.g. 60 seconds).

The figure 9 introduces the TEFSM of the Loan WebService, where c is a local clock. To simple, we do not usestop variable in this example. We assume here that:

1) all types of messages are integers;2) the value of the input messages (i.e. input msg, risk,

response) and timing delay for each input message arerandomly generated;

3) the test execution number is five (i.e. N=5).The algorithm will finish either when there is a fault or the

test execution number reaches the limit. We have a following

Figure 8. The Loan Web Service

timed test case list with the format (message name (value),interval time between two actions) (Fig 10):

VIII. CONCLUSIONS

To address the problem of Web services testing, this paperproposes an online testing framework for a composite ofWeb services described in BPEL. We mainly covered fourtopics: modeling BPEL specification by a TEFSM, a testarchitecture, an online testing algorithm which generates andexecutes test cases, and an example of testing frameworkdesign. We focus on unit testing of an implementation of acomposite of Web services, based on gray-box approach andconformance testing. In addition to the theoretical frame-work we have developed a prototype tool, called WSOTF.

These are some of limitations in our framework. Severaltest cases cannot be selected because the algorithm randomlyselects the test cases. In the future, we will memorize thisselection to improve the random selection of next time.Moreover, it is limited by the time execution number. In thecase of flow activity, if the service invokes many actions onparallel and it validates the timing constraints. May be thesetiming constraints are not validated because our frameworkworks (with a flow activity) as a sequence.

In the future works, we will extend TEFSM with aset of state properties (SP ): S → {on, off , null} isa mapping that assigns a property to states. A state hasthe property on represents sin of flow activity (resp. offrepresents sout). And next, we improve the algorithm toprocess simultaneously the activities of a flow activity. When

Figure 9. TEFSM of the Loan Web Service

1. (?input(8), 0)→ (!invoke1 msg out(8), 2) →(?risk(1), 1)→ (!invoke2 msg out(8), 3)→(?res msg(1), 30)→ (!output(1), 2) =⇒ PASS

2. (?input(1), 0)→ (!invoke1 msg out(1), 2)→ (?risk(0), 2)→ (!output(1), 2) =⇒ PASS

3. (?input(5), 0)→ (!invoke1 msg out(5), 5)→ (?risk(1), 0)→ (!invoke2 msg out(5), 0)→ (delay = 60)→ (!output(−1), 1) =⇒ PASS

4. (?input(10), 0)→ (!invoke2 msg out(10), 2)→ (?res msg(4), 20)→ (!output(4), 2) =⇒ PASS

5. (?input(15), 0)→ (!invoke2 msg out(15), 2)→ (?res msg(7), 10)→ (!output(7), 1) =⇒ PASS

Figure 10. Test Result

a state has the property be on, all of next actions willbe simultaneously processed instead of randomly select anaction and only process it.

ACKNOWLEDGMENT

This Research is supported by the French Na-tional Agency of Research within the WebMov Projecthttp://webmov.lri.fr

REFERENCES

[1] OASIS. Web Services Business Process Execution Lan-guage (BPEL) Version 2.0, April 2007. http://docs.oasis-open.org/wsbpel/2.0/wsbpel-v2.0.html

[2] Web Services Description Language 1.1.http://www.w3.org/TR/wsdl

[3] A. Bucchiarone, H. Melgratti, and F. Severoni, “Testing Ser-vice Composition”, In Proceedings of ASSE07, Mar del Plata,Argentina, August 2007.

[4] A. Bertolino, A. Polini, “The Audition Framework for TestingWeb Services Interoperability”, Proc of the 31st EUROMICROConference on Software Engineering and Advanced Applica-tions, 2005.

[5] X. Bai, W. Dong, W.T. Tsai, Y. Chen, “WSDL-Based Auto-matic Test Case Generation for Web Services Testing”, Proc ofthe IEEE International Workshop on Service-Oriented SystemEngineering (SOSE), pp 207 - 212, Beijing October 2005.

[6] Arthur Gill, “Introduction to the Theory of Finite-State Ma-chines”, Published by McGraw-Hill Book Co.., New York,1962.

[7] Jose Garcia-Fanjul, Javier Tuya, Claudio de la Riva, “Gen-erating Test Cases Specifications for BPEL Compositions ofWeb Services Using SPIN”, International Workshop on WebServices Modeling and Testing. WS-MaTe 2006.

[8] Y. Zheng, P. Krause, “Automata Semantics and Analysis ofBPEL”, International Conference on Digital Ecosystems andtechnologies. DEST 2007.

[9] Y. Zheng, J. Zhou, P. Krause, “A Model Checking based TestCase Generation Framework for Web Services”, InternationalConference on Information Technology. ITNG 2007.

[10] Y. Zheng, J. Zhou, P. Krause, “An Automatic Test CaseGeneration Framework for Web Services”, JOURNAL OFSOFTWARE, VOL. 2, NO. 3, SEPTEMBER 2007.

[11] X. Fu T. Bultan J. Su, “Analysis of Interacting BPEL WebServices”, International Conference on World Wide Web. May17 - 22, 2004, New York, USA.

[12] A. Wombacher, P. Fankhauser, and E. Neuhold, “Transform-ing bpel into annotated deterministic Finite state automata forservice discovery” Procs of ICWS04, 2004.

[13] R. Kazhamiakin, P. Pandya, and M. Pistore, “Timed modelingand analysis in web service compositions”, The First Interna-tional Conference on Availability, Reliability and Security, vol.Volume 0, pp. 840 846, 2006.

[14] M. Lallali, F. Zaidi, A. Cavalli, “Timed modeling of web ser-vices composition for automatic testing”, The 3rd ACM/IEEEInternational conference on Signal-Image technologie andinternet-Based Systems (SITIS’2007), China 16 - 19 december2007.

[15] M. Lallali, F. Zaidi, A. Cavalli, “Transforming BPEL intoIntermediate Format Language for Web Services CompositionTesting”, The 4th IEEE International Conference on NextGeneration Web Services Practices, October, 2008

[16] M. Lallali, F. Zaidi, A. Cavalli, Iksoon Hwang, “AutomaticTimed Test Case Generation for Web Services Composition”,Sixth European Conference on Web Services. Dublin, Ireland,Nov 12 - 14, 2008.

[17] T. D. Cao, P. Felix, R. Castanet, I. Berrada, “Testing WebServices Composition using the TGSE Tool”, IEEE 3rd Inter-national Workshop on Web Services Testing (WS-Testing 2009),July 7, 2009, Los Angeles, CA, USA.

[18] C. Bartolini, A. Bertolino, E. Marchetti, A. Polini, “WS-TAXI: a WSDL-based testing tool for Web Services”, In-ternational Conference on Software Testing Verification andValidation, April 1 - 4, 2009, Denver, Colorado - USA.

[19] R. Alur, D. L. Dill, “A Theory of Timed Automata”, Theoryof Computer Science .vol 126, no 2, pp 183 - 235 , 1994.

[20] ChangSup Keum, Sungwon Kang, In-Young Ko JongmoonBaik and Young-Il Choi “Generating Test Cases for WebServices Using Extended Finite State Machine”, TestCom2006, New York, NY, USA, May 16 - 18, 2006

[21] R. Heckel and L. Mariani, “Automatic conformance testingof web services”, Fundamental Approaches to Software Engi-neering, pp. 34 - 48, LNCS 3442, 2 - 10 April, 2005.

[22] S. Salva and A. Rollet, “Automatic web service testingfrom WSDL descriptions”, 8th IEEE International Conferenceon Innovative Internet Community Systems, Schoelcher, Mar-tinique, June 16-18, 2008.

[23] H. Huang, W. T. Tsai, R. Paul, Y. Chen, “Automated ModelChecking and Testing for Composite Web Services”, EighthIEEE International Symposium on Object-Oriented Real-TimeDistributed Computing (ISORC’05), pp.300-307, 2005.

[24] M. Mikucionis, K. G. Larsen, B. Nielsen, “T-UPPAAL:Online Model-based Testing of Real-time Systems”, 19th IEEEInternational Conference on Automated Software Engineering,pp 396 - 397. Linz, Austria, Sept 24, 2004.

[25] K. G. Larsen, M. Mikucionis, B. Nielsen, “Online Testing ofReal-time Systems Using UPPAAL”, Formal Approaches toTesting of Software. Linz, Austria. Sept 21, 2004

[26] G. J. Tretmans and H. Brinksma “TorX: Automated Model-Based Testing”, First European Conference on Model-DrivenSoftware Engineering, Nuremberg, Germany, Dec 11 - 12,2003.

[27] H. Bohnenkamp and A. Belinfante, “Timed Testing withTorX”, Formal Methods 2005, LNCS 3582, pp. 173 - 188,2005.

[28] P. Mayer, “Design and Implementation of a Framework forTesting BPEL Compositions”, Master thesis, Leibniz Univer-sity, Hannover, Germany, Sep 2006.

[29] Z. Li, W. Sun, Z.B. Jiang, X. Zhang, “BPEL4WS UnitTesting: Framework and Implementation”, Proc of the IEEEInternational Conference on Web Service (ICWS’05), pp 103- 110, 2005.

[30] A. Cavalli, Edgardo Montes De Oca, W. Mallouli, M. Lal-lali, “Two Complementary Tools for the Formal Testing ofDistributed Systems with Time Constraints”, 12th IEEE Inter-national Symposium on Distributed Simulation and Real TimeApplications, Canada, Oct 27 - 29, 2008.

[31] BPELUnit - The Open Source Unit Testing Framework forBPEL. http://www.se.uni-hannover.de/forschung/soa /bpelunit/

[32] EVIWARE. soapUI. http://www.eviware.com/[33] PushToTest. TestMaker. http://www.pushtotest.com/