Extending EFSMs to Specify and Test Timed Systems with Action Durations and Timeouts

Extending EFSMs to Specify and Test TimedSystems with Action Duration and Time-Outs

Mercedes G. Merayo, Manuel Nunez, and Ismael Rodrıguez

Abstract—In this paper, we introduce a timed extension of the extended finite state machines model. On one hand, we consider that

(output) actions take time to be performed. This time may depend on several factors, such as the value of variables. On the other hand,

our formalism allows us to specify time-outs. In addition to presenting our language, we develop a testing theory. First, we define

10 timed conformance relations and relate them. Second, we introduce a notion of timed test and define how to apply tests to

implementations. Finally, we give an algorithm to derive sound and complete test suites with respect to the implementation relations

presented in the paper. This paper represents an extended and improved version of [1].

Index Terms—Testing and debugging, formal methods, specification techniques, specifying and verifying and reasoning about

programs.

Ç

1 INTRODUCTION

REAL-TIME systems are hardware and software systemsthat are subject to timing constraints, that is, the time

that the system can spend for producing a response has adeadline. These systems can be found in several branches ofthe industrial world, for example, in aeronautics andmedicine. It would be unacceptable that a system providinga medical treatment, for example, radiotherapy, does notconsider critical situations, allowing that an extra dose ofradiation would be applied to a patient. It would bedesirable that the system could react to these situationsautonomously, stopping the treatment if the system doesnot stop before a bounded amount of time. Anotherpractical example can be found in security in publictransport. Usually, trains decrease speed uniformly if thedriver does not push a button with a preestablishedfrequency. This behavior avoids having the train cause anaccident if the driver loses consciousness. Thus, it isnecessary to assure the correctness of real-time systemsforces to apply a methodology that considers time require-ments. In most models, temporal requirements usually referto the time that a system consumes for performingoperations. However, another kind of time constraint canaffect systems: time-outs. A time-out is a specified periodthat will be allowed to elapse in a system waiting for aninteraction with it; if the period ends and no action has beenproduced, the state of the system changes and its reactionsto the actions that are received from the environment maybe different.

Formal methods allow the analysis of systems andreasoning about them with mathematical precision andrigor. It is widely recognized that formal methods andtesting are complementary techniques [2], [3], [4], [5], [6], [7]

since they help to check the correctness of systems andprovide a framework for testing. The use of formal methodsfor describing a specification provides a more precise andconsistent starting point for the testing process. Hence, it isnecessary to provide languages to formally specify thesesystems. Even though there exist a myriad of timedextensions of classical frameworks [8], [9], [10], [11], [12],[13], [14], [15], [16], most of them specialize in only one ofthe previous variants: Time is either associated with actionsor associated with delays/time-outs. Our formalism basedon extended finite state machines allows to us specify, in anatural way, both time aspects.

The theoretical framework is complicated by twoadditional features. First, we consider that variables mayinfluence the timing aspects of the specification. Thus, theexecution of an action may take different time values if thevalue of the variables change. Second, we do not imposeany restriction on the deterministic behavior of ourmachines. This implies, again, that the same sequence ofactions may take different time values to be executed.

We assume that both specifications and implementationscan be modeled by the previous formalism and propose aformal testing methodology to systematically test a systemwith respect to a specification. Testing a system requiresusing a specific notion of correctness. When we deal withreal-time systems, it is necessary to establish two differentlevels of conformance: functional and timed conformance.Regarding functional conformance, we have to considerthat the sequences of inputs/outputs produced by thesystem under test must be allowed in the specification, thatis, the implementation does not invent anything for thoseinputs that are specified in the specification. In addition, wehave to take into account the possible time-outs. Forexample, a sequence of inputs/outputs could be acceptedonly after different time-outs have been triggered.

Regarding temporal performance requirements, ourtesting methodology will take into account only theconsumed time while performing actions, not the time thatthe system waits for a reaction from the environment. Thelatter solely affects which actions are allowed/forbidden.

IEEE TRANSACTIONS ON COMPUTERS, VOL. 57, NO. 6, JUNE 2008 835

. The authors are with the Departamento de Sistemas Informaticos yComputacion, Universidad Complutense de Madrid, Ciudad Universitarias/n Madrid, Madrid 28040, Spain.E-mail: [email protected], {mn, isrodrig}@sip.ucm.es.

Manuscript received 24 Feb. 2007; revised 8 Nov. 2007; accepted 12 Nov.2007; published online 9 Jan. 2008.Recommended for acceptance by D. Gizopoulos.For information on obtaining reprints of this article, please send e-mail to:[email protected], and reference IEEECS Log Number TC-0070-0207.Digital Object Identifier no. 10.1109/TC.2008.15.

0018-9340/08/$25.00 � 2008 IEEE Published by the IEEE Computer Society

We propose several timed conformance relations accord-ing to the interpretation of good implementation for a givenspecification. Let us emphasize that testing the temporalbehavior of a system affected by nondeterminism requiresfacing some issues that are not considered by other testingapproaches. For instance, we may require that, among all ofthe time values the implementation may need to perform atask, one of them is smaller than the one corresponding tothe specification for this task. Hence, if the application of atest to the implementation takes a long time, this does notnecessarily mean that it cannot do it faster.

With respect to the application of tests to implementa-tions, the abovementioned nondeterministic temporal be-havior of specifications and/or implementations requiresthat tests work in a specific manner. For instance, if weapply a test and we observe that the implementation takesless time than the one required by the specification, thenthis single application of the test allows us to know that theimplementation may be faster than the specification but notthat it must be so.

In terms of related work, our language is based onextended finite state machines (EFSM), which have beenextensively used by the formal testing community. Thispaper continues the work in [17]. The main advantage withrespect to this previous work is that we can now expresstime-outs, we remove all of the restrictions regardingnondeterminism of the machines, and we consider moreconformance relations. Our way to deal with time iscompletely different from that in timed automata [13]. Aswe said before, we can associate time with the performanceof actions, whereas time-outs can be easily represented.These features not only improve the modularity of models,but they are also suitable for clearly identifying systemrequirements and responsibilities in a testing methodology.Our model implies the existence of a unique clock to controlhow time passes. This fact can make our model apparentlyless expressive than timed automata, where different clockscan be used. For instance, it is not direct to express timerelations between different parts of the machines (forexample, to specify that a transition takes two time unitsmore than the previously performed transition). However,these situations can be easily simulated by using additionalvariables (we are working within an EFSM model wherevariables associated with different domains can be used) torecord relevant time values. However, it is worth mention-ing that our notion of time-out is related to havinginvariants associated with states of a timed automata, asdescribed in [18]: Once the value of a clock variable exceedsthe invariant, the system produces a prescribed output andenters a new state. Among those models based on timedautomata, the ones most related to ours are those whereurgency has been added to the performance of transitions(for example, [19], [20], and [21]).

Regarding testing of temporal requirements, there existseveral proposals (for example, [22], [23], [24], [25], [26]), butmost of them are based on timed automata. Moreover, inthese approaches, tests are independent and the diagnosis ofa test does not depend on other tests. By considering thattests are interrelated, we can relate nondeterminism andtemporal requirements, as well as define and applyseveral conformance relations where nondeterminism isexplicitly considered.

Due to the intrinsic difficulty behind testing timedsystems, different approaches have been studied, each ofthem falling into one or more of the following categories:1) Only some behaviors, out of those that are relevant forthe correctness of the implementation, are tested (see, forexample, [27], [28]). In these cases, methods to choose thosetests that seem to have a higher capability to find errors areproposed, though they are usually heuristic or are based onrestricting the behavior to be tested to some specific testpurposes. 2) A complete finite test suite is derived from thespecification, that is, if all tests in the finite suite are passed,then the implementation is correct (see, for example, [29],[18]). Usually, the finiteness of this suite requires introdu-cing strong assumptions about the implementation, both todeal with functional requirements (for example, to assumethat the maximal number of states in the implementation isknown) and timed requirements (for example, urgency ofoutputs and discretization of time). In general, the applic-ability of the derived test suite is not feasible because thenumber of derived tests is astronomic. 3) A complete infinitetest suite is extracted from the specification (see, forexample, [30], [31], [32], [26]). In particular, a test derivationalgorithm is defined in such a way that, for all implementa-tion behaviors that must be tested before granting thecorrectness, a suitable test for checking this behavior isadded by the algorithm after executing it some finite time.In this sense, such an algorithm is complete (that is, itprovides full fault coverage with respect to the consideredtesting relation) in the limit. The methods are interestingbecause, on one hand, weaker assumptions are required inthese methodologies and, on the other hand, a method tofind and construct any required test is needed if we want toselect some of these tests according to some criteria. That is,methods that are exhaustive in the limit are the basis forother nonexhaustive but more practical methods. Themethodology presented in this paper fits into the 3) categoryand, consequently, its aim is to provide test suites that arecomplete in the limit, while, in turn, no strong assumptionsare required (for example, about the number of states of theimplementation).

The rest of the paper is structured as follows: In Section 2,we introduce our model. In Section 3, we give our timedconformance relations and provide several examples to showthe differences among them. In Section 4, we show how testsare defined and applied to implementations. In Section 5, wepresent an algorithm to derive sound and complete test suiteswith respect to the implementation relations that wepresented in Section 3. Finally, in Section 6, we present ourconclusions and some directions for future work.

2 A TIMED EXTENSION OF THE EFSM MODEL

In this section, we introduce a new notion of timed extendedfinite state machine. As we have indicated in the introductionof this paper, we will add new features so that the timedbehavior of a system can be properly specified. On onehand, we consider that output actions take time to beexecuted. These time values will not depend only on thecorresponding action to be performed and the current stateof the machine. Actually, we will also consider that thesetime values take into account the current value of thevariables. For instance, by using this approach, we cansimulate that the speed with which a task is performed

836 IEEE TRANSACTIONS ON COMPUTERS, VOL. 57, NO. 6, JUNE 2008

depends on the available resources. These resources, moreproperly, the amount of each resource owned by thecorresponding process, can be encoded as a tuple ofvariables in our model. This will be done in the forthcomingDefinition 2 by considering a domain D, defined as theCartesian product of different data domains to representvalues. Along this line, [32] presents such an approach todealing with resources since the process algebra PAMR [32],useful to specify systems where resources play a funda-mental role, can be encoded into a notion of timed EFSM. Onthe other hand, we will also consider that the machine canevolve by raising time-outs. Intuitively, if, after a givenamount of time and depending on the current state, we donot receive any input action, then the machine will changeits current state. In the rest of the paper, we will use thefollowing notation:

Definition 1. A tuple of elements ða1; a2 . . . ; anÞ will be denotedby �a. We say that a ¼ ½a1; a2Þ is an interval if a1; a2 2 IN anda1 � a2. A tuple of intervals ðp1; . . . ; pnÞ will be denoted by �p.Let �t ¼ ðt1; . . . ; tnÞ, �t0 ¼ ðt01; . . . ; t0nÞ, and �q ¼ ðq1; . . . ; qnÞ. Wewrite

. �t 2 �q if, for all 1 � j � n, we have tj 2 qj,

. �t ¼ �t0 if, for all 1 � j � n, we have tj ¼ t0j, and

. �t � �t0 if, for all 1 � j � n, we have tj � t0j.In addition, we introduce the following notations:

.P

�t denotes the sum of the elements belonging to thetuple �t, that is,

Pnj¼1 tj.

. �ið�tÞ, for all 1 � i � n, denotes the value ti.

Definition 2. Let D1; . . . ; Dm be sets of values and let usconsider D ¼ D1 �D2 � � � � �Dm. We have that D is thedomain where variables take values. We denote by Time thedomain to define time values. A TEFSM in the following is atuple M ¼ ðS; I; O; TO; Tr; sin; �yÞ, where S is a finite set ofstates, I is the set of input actions, O is the set of outputactions, TO : S�!S � ðTime [1Þ is the time-out func-tion, Tr is the set of transitions, sin is the initial state, and�y 2 D is the tuple of initial values of the variables.

A transition is a tuple ðs; s0; i; o; Q; Z; CÞ, where s, s0 2 Sare the initial and final state of the transition, i 2 I and o 2 Oare the input and output action associated with the transition,

Q : D�!Bool is a predicate on the set of variables, Z :

D�!D is a transformation over the variables, and C :

D�!Time is the time that the transition needs to be

completed.

A configuration in M is a pair ðs; �xÞ, where s 2 S is the

current state and �x 2 D is the tuple containing the current

values of the variables.

We say that M is input enabled if, for all configurations

ðs; �xÞ and input i 2 I, there exist s0, o, Q, Z, and C such that

Qð�xÞ holds and ðs; s0; i; o; Q; Z;CÞ 2 Tr.Given a configuration ðs; �xÞ, a transition ðs; s0; i; o; Q; Z;CÞ

denotes that, if the input i is received and Qð�xÞ holds, thenthe output o will be produced after Cð�xÞ units of time andthe configuration will be ðs0; Zð�xÞÞ. In this paper, weconsider that time is discrete. In particular, we willsometimes enumerate the elements of Time simply as 0,1, 2, and so on. Finally, let us remark that, even though we

have variables, we do not consider input or outputparameters in our transitions.

For each state s 2 S, the application of the time-outfunction TOðsÞ returns a pair ðs0; tÞ, indicating the time thatthe machine can remain at the state s waiting for an inputaction and the state to which the machine evolves if noinput is received on time. We assume that TOðsÞ ¼ ðs0; tÞimplies s 6¼ s0, that is, time-outs always produce a change ofthe state. We indicate the absence of a time-out in a givenstate by setting the corresponding time value to 1.

Let us comment on our notion of input enabling. In untimedmodels based on EFSMs, a machine is considered to be inputenabled if it can accept an input at any point in time. In thecase of timed systems, this notion has to be convenientlyadapted. We consider that the machine has to accept anyinput once it is placed in a state, that is, it is not performing atransition (time-out transitions are instantaneous). Infor-mally, we can see this notion of input enabling as a userinteracting with a system through a console. When theprompt symbol appears, the user can write certain commandsand the system must react in some way (performing acomputation, giving an error message, etc.). When the systemis occupied performing an operation, the prompt symboldisappears and keyword interactions are ignored.

Finally, let us remark that, in this paper, we are notconsidering a framework where communicating machinescan be defined. Thus, we assume that inputs are providedby the environment and outputs are sent to the environ-ment. However, it is not difficult to extend the currentframework to deal with communications between differentmachines by adapting the framework presented in [32] tothe timed model presented in this paper.

Definition 3. Let M ¼ ðS; I; O; TO; Tr; sin; �yÞ be a TEFSM andc0 ¼ ðs0; �x0Þ be a configuration ofM. A tuple ðs0; s; i=o; t; to; �vÞis a step of M for the configuration c0 if there existk � 0 states s1; . . . ; sk 2 S such that, for all 1 � j � k,we have TOðsj�1Þ ¼ ðsj; tjÞ and there exists a transitionðsk; s; i; o;Q; Z; CÞ 2 Tr such that Zð�x0Þ ¼ �v,

t ¼Xkj¼1

tj;Xkj¼1

tj þ �2ðTOðskÞ" !

;

to ¼ Cð�x0Þ, and Qð�x0Þ holds.We say that ðt1=i1=to1=o1; . . . ; tr=ir=tor=orÞ is a

timed evolution of M if there exist r steps of Mðsin; s1; i1=o1; t1; to1; �y1Þ; . . . ; ðsr�1; sr; ir=or; tr; tor; �yrÞ for theconfigurations ðsin; �yÞ; . . . ; ðsr�1; �yr�1Þ, respectively. We de-note by TEvolðMÞ the set of timed evolutions of M. Inaddition, we say that ðt1=i1=o1; . . . ; tr=ir=orÞ is a functionalevolution of M. We denote by FEvolðMÞ the set of functionalevolutions of M.

Intuitively, a step is a transition preceded by zero or moretime-outs. The interval t indicates the time values where theinput action could be received. A functional evolution is asequence of inputs/outputs corresponding to the transitionsof a chain of steps. The first of these steps begins with theinitial configuration of the machine. These steps include thetime interval, indicated by the different intervals tj, wheneach input ij could be accepted. As we will explain later whenwe introduce our functional implementation relation, seeDefinition 5, functional evolutions need to include time

MERAYO ET AL.: EXTENDING EFSMS TO SPECIFY AND TEST TIMED SYSTEMS WITH ACTION DURATION AND TIME-OUTS 837

information. Specifically, they must contain informationrelated to the triggered time-outs. This is due to the factthat time-outs influence the different input/output se-quences that TEFSMs can perform. This information isencoded into the intervals where input actions can beperformed.

In addition to the previous information, timed evolutionspresent the time consumed to execute each output afterreceiving each input in each step of the evolution.

Example 1. Let us consider the TEFSM depicted in Fig. 1. Wesuppose that the variables of the TEFSM are given by atuple �x 2 IR4

þ and we denote by xi the ith component of�x. Let us assume that, initially, �x ¼ ð1; 2; 2; 1Þ. Next, wegive some of the steps that the machine can generate. Forexample, ðs1; s2; i1=o1; ½0; 3Þ; 3; ð2; 2; 2; 0ÞÞ represents thetransition t12 when no time-outs precede it. The input i1can be accepted before three time units pass (this isindicated by the interval ½0; 3Þ). In addition, o1 takesC1ðð1; 2; 2; 1ÞÞ ¼ 3 time units to be performed and thenew values of �x are Z1ðð1; 2; 2; 1ÞÞ ¼ ð2; 2; 2; 0Þ. Thesecond one, ðs1; s4; i1=o2; ½3; 7Þ; 4; ð1; 3; 1; 1ÞÞ, is builtfrom the time-out transition associated to the state s1

and the transition t34. The step represents that if, afterthree time units no input is received, then the time-outtransition associated with that state will be triggeredand the state will change to s3. After this, the machinecan accept the input i1 before four time units pass, thatis, the time-out assigned to the state s3. Therefore,during the time interval ½3; 7Þ, if the machine receivesan input i1, it will emit an output o2 and the state willchange to s4. Similarly, we can obtain the stepðs1; s1; i1=o2; ½7;1Þ; 1; ð1; 3; 1; 1ÞÞ using the time-out tran-sitions corresponding to s1 and s3 and the transition t61.All of the steps presented in this example correspond tothe initial configuration ðs1; ð1; 2; 2; 1ÞÞ.

Now, we present an example of a timed evolutionbuilt from two steps and assume that s1 is the initialstate: ð½7;1Þ=i1=1=o2; ½3; 7Þ=i1=5=o2Þ. The configurationthat has been considered for the first step is againðs1; ð1; 2; 2; 1ÞÞ. The configuration that corresponds to thesecond step is the one obtained after the first step hasbeen performed, that is, ðs1; ð1; 3; 1; 1ÞÞ.

In the following definition, we introduce the concept ofinstanced evolution. Intuitively, instanced evolutions areconstructed from evolutions by instantiating to a concretevalue each time-out, given by an interval, of the evolution.

Definition 4. Let M ¼ ðS; I; O; TO; Tr; sin; �yÞ be a TEFSM ande ¼ ðt1=i1=to1=o1; . . . ; tr=ir=tor=orÞ be a timed evolution ofM. We say that the tuple ðt1=i1=to1=o1; . . . ; tr=ir=tor=orÞ isan instanced timed evolution of e if for all 1 � j � r, wehave tj 2 tj. In addition, we say that the tupleðt1=i1=o1; . . . ; tr=ir=orÞ is an instanced functional evolu-tion of e.

We denote by InsTEvolðMÞ the set of instanced timedevolutions of M and we denote by InsFEvolðMÞ the set ofinstanced functional evolutions of M.

We will sometimes refer to the tuple

ðt1=i1=to1=o1; . . . ; tr=ir=tor=orÞ 2 InsTEvolðMÞ

as ð�t; �; �toÞ, where t ¼ ðt1; . . . ; trÞ, � ¼ ði1=o1; . . . ; ir=orÞ, andto ¼ ðto1; . . . ; torÞ. Similarly, we will also refer to instancedfunctional evolutions as ð�t; �Þ.Example 2. If we consider the timed evolutionð½7;1Þ=i1=1=o2; ½3; 7Þ=i1=5=o2Þ shown in the previousexample, we have that the tuples ð8; =i1=1=o2; 5=i1=5=o2Þand ð12; =i1=1=o2; 3=i1=5=o2Þ are two of its instancedtimed evolutions.

3 (TIMED) IMPLEMENTATION RELATIONS

In this section, we introduce our implementation relations.Intuitively, an implementation is related to a specification ifand only if the implementation is correct with respect to thespecification. However, in our context, the notion ofcorrectness has several possible definitions. In fact, in thecase of timed systems, a good implementation is even lessprecise than in untimed systems. For example, one mayconsider that a system I is a good implementation of aspecification S if I takes the same time to perform its tasksas the specification S, while another could consider that theimplementation has to be always/sometimes faster. Thus,for each of these considerations, we will define a differentimplementation relation.

Even though there are different perspectives to consider-ing what a good implementation is, there is an agreementon correctness if we consider only functional behavior, that is,abstracting the time that actions take to be performed.Regarding the performance of usual inputs and outputs, animplementation should not invent behaviors when inputsspecified in the specification are applied to it. In formalterms, this means that the relevant evolutions of theimplementation must be contained in those correspondingto the specification. Thus, all of our implementationrelations follow the same pattern: an implementation I


Fig. 1. Example of TEFSM.

conforms to a specification S if, for all possible evolution ofS, the outputs that the implementation I may perform aftera given input are a subset of those for the specification.

Additionally, we require that the implementation alwaycomply in a certain manner with the time-outs establishedby the specification. This is due to the fact that time-outshave to be part of this untimed relation because they caninfluence the functional behavior of a system, that is, theyinfluence the actions that a TEFSM can perform at a givenpoint of time. This is illustrated in the following example.

Example 3. Let us consider the schematic machinesdepicted in Fig. 2. These diagrams represent simplifiedTEFSMs where data is not used. We consider thefollowing notation: A transition labeled by “i=o; t”denotes that the execution of the output action o takestime t to be performed after the input i is received; atransition with a label t indicates that a time-out will beapplied at time t, that is, if, after t time units, no expectedinput is received, then the time-out is executed.

We will have that M 0 is not functionally conforming toM 00. The sequence i1=o2 which can be performed by M 0

cannot be performed by M 00. If we consider theconformance of M 00 with respect to M 0 and we onlycheck the possible sequences of inputs/outputs, M 00

would conform to M 0 due to the fact that the uniquesequence that can be performed by M 00 is i1=o1. However,this sequence is allowed by M 0 only in the case where theinput has been received after three time units. Therefore,under our conformance framework, M 00 does not con-form to M 0. We can say the same regarding theconformance of M 000 with respect to M 0. However, thisis not the case when considering the conformance of M 0

with respect to M 000. The sequences performed by M 0 areaccepted by M 000 at any time. Therefore, M 0 functionallyconforms to M 000.

In our framework, specifications will be modeled by usingtimed extended finite state machines. Let us note that weconsider black-box testing, that is, a system will be tested withrespect to a specification without having information aboutthe internal structure of the system. The only knowledgeabout the implementation under test (IUT) must be inferredfrom the outputs observed when tests interact with thesystem. As usual, we also consider that the IUT is describedby using a TEFSM having the same sets of inputs and outputsas the specification. We also assume that implementations areinput enabled. Let us note that we do not restrict the machinesto be deterministic. Thus, both implementations and speci-fications may present nondeterministic behavior. In additionto increasing the expressive power of the framework, thepresence of nondeterminism also allows us to define moreimplementation relations. These facts augment the usability

of our framework and are an important advantage withrespect to previous work [17].

First, we introduce the implementation relation conff ,where only functional aspects of the system (that is, whichoutputs are allowed/forbidden) are considered, while theperformance of the system (that is, how fast actions areexecuted) is ignored. Let us note that the time spent by asystem waiting for the environment to react has thecapability to affect the set of available outputs of thesystem. This is so because this time may trigger a change ofthe state by raising one or more time-outs. Therefore, arelation focusing on functional aspects must explicitly takeinto account the maximal time the system may stay in eachstate. This time is given by the time-out of the state. This isthe reason why we have to partially take into account timeaspects in the definition of the following functionalimplementation relation: As noted before, raising time-outsmay influence the actions available at a given point.

Definition 5. Let S and I be two TEFSMs. We say that Ifunctionally conforms to S, denoted by I conff S, if, for alle 2 FEvolðSÞ, with e ¼ ðt1=i1=o1; . . . ; tr=ir=orÞ and r � 1,we have that for all t1 2 t1; . . . ; tr 2 tr and o0r,

e0 ¼ðt1=i1=o1; . . . ; tr=ir=o0rÞ 2 InsFEvolðIÞ

+e0 2 InsFEvolðSÞ:

Let us note that, if the specification also has the propertyof an input enabled, then we may remove the condition “foreach e 2 FEvolðSÞ, with e ¼ ðtt1=i1=o1; . . . ; ttr=ir=orÞ andr � 1.”

Example 4. Let us consider the schematic machines depictedin Fig. 3. Let us note that the behavior of M1 and M2 isexactly the same regardless of the time-out presented inM1. All transitions available forM1 after taking a time-outare also available in M2 from its first state. Thus, we haveM1 conff M2. Actually, we also have M2 conff M1. Forsimilar reasons, we have M1 conff M9, M9 conff M1,M2 conff M9, and M9 conff M2.

Next, we show how the availability of outputs affectsthe functional relation conff . We have M5 6conff M11. Letus note that, if i2 is offered inM11 after executing i1=o1, thenonly o1 can be produced. However, M5 can produce thisoutput, as well as o2, which is forbidden byM11. Therefore,we do not haveM5 conff M11. If we considerM6 instead ofM5, then the same considerations apply: We haveM6 6conff M11. Moreover, we have M9 6conff M10 andM10 6conff M9. Let us note that, after one time unit passesand the time-out is raised if i1 is offered, then M9 mustanswer o1, while o2 is forbidden. However, it is the otherway around for M10.

In the following definition, we introduce our timedimplementation relations. We will distinguish two classesof conformance relations: weak and strong. The family ofweak conformance relations demands conditions over thetotal time associated to instanced timed evolutions of theimplementation with respect to the corresponding in-stanced timed evolutions of the specification. In contrast,strong conformance relations establish requests over thetime values corresponding to the performance of each


Fig. 2. Examples of functional conformance.

transition separately. For each of these approaches, wedefine five relations. Informally, the confsa and confwarelations (conforms always) hold if, for all instanced timedevolution e of the implementation, we have that if itsassociated instanced functional evolution is an instancedfunctional evolution of the specification, then e is also aninstanced timed evolution of the specification. In the confswand confww relations (conformance in the worst case), theimplementation is forced for each instanced timed evolu-tion fulfilling the previous conditions to be faster than theslowest instance of the same evolution in the specification.The confsb and confwb relations (conforms in the best case) aresimilar but take into account only the fastest instance of thespecification. Finally, the relations confssw and confwsw(sometimes worst), and confssb and confwsb (sometimes best)are similar to the previous relations, but, in each case, onlyone instance of each timed evolution of the implementationis required to be as fast as the worst/best instance in thespecification.

Definition 6. Let �to ¼ ðto1 . . . torÞ 2 Timer. Given an instanced

functional evolution

insfevol ¼ ðt1=i1=o1; . . . ; tr=ir=orÞ 2 ðTime� I �OÞr;

we denote by insfevolr�to the instanced timed evolution

ðt1=i1=to1=o1; . . . ; tr=ir=tor=orÞ. Let S and I be two TEFSMs.

The timed conformance relations are defined as follows:

. (strong always) I confsa S iff I conff S and for allinsfevol 2 InsFEvolðIÞ \ InsFEvolðSÞ we have that8 �ti:

insfevolr�ti 2 InsTEvolðIÞ+

insfevolr�ti 2 InsTEvolðSÞ:

. (strong best) I confsb S iff I conff S and for allinsfevol 2 InsFEvolðIÞ \ InsFEvolðSÞ we have that8 �ti:


8 �ts : insfevolr�ts 2 InsTEvolðSÞ¼)�ti � �tsð Þ:

. (strong worst) I confsw S iff I conff S and for allinsfevol 2 InsFEvolðIÞ \ InsFEvolðSÞ we have that8 �ti:


9 �ts : insfevolr�ts 2 InsTEvolðSÞ ^ �ti � �tsð Þ:

. (strong sometimes best) I confssb S iff I conff Sand for all insfevol 2 InsFEvolðIÞ \ InsFEvolðSÞwe have that 9 �ti such that

insfevolr�ti 2 InsTEvolðIÞ^

8 �ts : insfevolr�ts 2 InsTEvolðSÞ¼)�ti � �tsð Þ:

. (strong sometimes worst) I confssw S iff I conff Sand for all insfevol 2 InsFEvolðIÞ \ InsFEvolðSÞwe have that 9 �ti, �ts such that


insfevolr�ts 2 InsTEvolðSÞ^

�ti � �ts

0BBBB@

1CCCCA:

. (weak always) I confwa S iff I conff S and for allinsfevol 2 InsFEvolðIÞ \ InsFEvolðSÞ we have that8 �ti:


9 �ts : insfevolr�ts 2 InsTEvolðSÞ ^P

�ti¼P

�tsð Þ:

. (weak best) I confwb S iff I conff S and for allinsfevol 2 InsFEvolðIÞ \ InsFEvolðSÞ we have that8 �ti:


8 �ts : insfevolr�ts 2 InsTEvolðSÞ¼)P

�ti�P

�tsð Þ:


Fig. 3. Examples of schematic machines representing TEFSMs.

. (weak worst) I confww S iff I conff S and for allinsfevol 2 InsFEvolðIÞ \ InsFEvolðSÞ we have that8 �ti:


9 �ts : insfevolr�ts 2 InsTEvolðSÞ ^P

�ti�P

�tsð Þ:

. (weak sometimes best) I confwsb S iff I conff S andfor all insfevol 2 InsFEvolðIÞ \ InsFEvolðSÞ wehave that 9 �ti such that


8 �ts : insfevolr�ts 2 InsTEvolðSÞ¼)P

�ti�P

�tsð Þ:

. (weak sometimes worst) I confwsw S iff I conff Sand for all insfevol 2 InsFEvolðIÞ \ InsFEvolðSÞwe have that 9 �ti, �ts such that


insfevolr�ts 2 InsTEvolðSÞ^P

�ti�P

�ts

0BBBB@

1CCCCA:

There are several reasons why an implementation willnot conform to a specification. We will give severalexamples where different systems depicted in Fig. 3 areconsidered. For the sake of simplicity, we will use someadditional conformance binary operators. In the nextexamples, I conf� S denotes that all implementation rela-tions given in Definition 6 hold between I and S. If none ofthese relations holds, then we denote it by I conf�= S.Besides, I conftu S denotes that all relations except confsaand confwa hold.

Example 5. If we consider the machines M3 and M2, wehave M3 conf�= M2. This is due to the existence of differenttime values to perform output actions. Let us note that M3

may take three time units to perform the output o1 if itreceives the input i1 after three time units, ð3=i1=3=o1Þ,while M2 only needs two time units, ð3=i1=2=o1Þ. SinceM3 is, in any case, slower than M2 for these sequences ofinputs/outputs, no conformance relation where M3 is theIUT and M2 is the specification holds. On the contrary,we have M2 conftu M3: Despite the fact that M2 does nottake the same time values as M3 for each sequence, itstime is always smaller than (in the case where the inputis received after the time-out is triggered) or equal to (inthe case where the input is received before the time-out istriggered) the time values corresponding to M3.

As we have seen, reducing the time consumed by actionscan produce that a certain TEFSM conforms to another onewith respect to some of the available implementationrelations. In spite of the fact that the requirements ontime-outs are strict, sometimes having different time-outscan also produce the same effect.

Example 6. Let us consider M3 and M4. In this case, thenontimed conformance is produced by the fact that the

machines have different time-outs. Most input/output

sequences in M3 and M4 take the same time values to be

performed. However, there is an exception: the sequence

including time-out 3. In M3, we have ð3=i1=3=o1Þ, but, in

M4, we have ð3=i1=2=o1Þ. This is so because, after three

time units pass, the state does not yet change in M4.

Hence, we have M4 conftu M3, but M3 conf�= M4.

Let us consider a case where IUTs and specifications can

spend different time values in executing pairs of inputs/

outputs included in evolutions.

Example 7. We consider M7 and M8. Since they only

perform sequences of length 1, strong relations coincide

with their respective weak versions. Next, we will refer

to strong relations. Both M7 and M8 can execute i1=o1 by

spending an amount of time that the other one cannot

take. Therefore, we do not have M7 confsa M8. The worst

time values to execute i1=o1 in M7 and M8 are 4 and 3,

respectively, while the best time values are 1 and 2,

respectively. The worst time of M7 is not better than

either the worst or the best time of M8. Thus, we have

neither M7 confsw M8 nor M7 conf

sb M8. However, the

best time of M7 is better than both the worst and the best

time values of M8. Therefore, both M7 confssw M8 and

M7 confssb M8 hold. Moreover, the worst time of M8 is

better than the worst one in M7 but worst than the best of

M7. Hence, M8 confsw M7 holds, but M8 conf

sb M7 does

not. Finally, the best time in M8 is better than the worst of

M7 but not better than its best one. Thus, M8 confssw M7

holds, but M8 confssb M7 does not. All of these situations

are due to the fact that these machines have different time

requirements.

Finally, we show how temporal requirements are dealt

with by strong and weak relations.

Example 8. Let us consider M11 and M12. No strong relation

holds between these TEFSMs in any direction. The reason is

thatM11 performs i1=o1 faster thanM12, butM12 performs

the next transition, i2=o1, faster thanM11. The result is that

none of these machines is always at least as fast,

concerning transitions, as the other one. However, if we

consider complete timed evolutions, instead of their steps

separately (that is, weak relations), then some relations

hold. Let us note that M11 performs both available

sequences of inputs/outputs (i1=o1 and i1=o1; i2=o1) faster

than M12: In M11, they spend two and six time units,

respectively, while these time values are four and seven,

respectively, in M12. Therefore, all weak relations except

confwa hold: We have M11 confww M12, M11 conf

wb M12,

M11 confwsw M12, and M11 conf

wsb M12. Let us remark that

none of these relations holds if we exchange the roles of

both machines.

Theorem 1. The relations given in Definition 6 are related as

follows:


I confwsw S ( I confwsb S

* *I confwa S ) I confww S ( I confwb S

* * *I confsaS ) I confsw S ( I confsb S

+ +I confssw S ( I confssb S:

Besides, we have

I confssw S ) I confwswS

and

I confssb S ) I confwsbS:

Proof. We only need to consider the evolutions of I alsobelonging to S (for the rest of the evolutions, thepremises of the corresponding conformance relation donot hold). First, let us note that the condition aboutfunctional conformance is the same in all of thedefinitions. Therefore, we only need to take into accountthe conditions on time values appearing in the secondclause of the corresponding relations. If I confsa S, thenwe have that each timed evolution in I fulfilling theconditions given in the definition of confsa also appearsin S. Therefore, we have I confsw S. If I confsb S, theneach timed evolution of I fulfilling the conditions givenin the definition of confsb is faster than the fastest instanceof the same evolution of S. Therefore, it is also faster thanthe slowest one of S. Therefore, we conclude I confsw S.

If I confsw S, then we know that each instance of atimed evolution of I needs a time less than or equal tothe one corresponding to the slowest instance, for thesame evolution, of the specification S. In particular, thereexists an instance fulfilling the condition imposed byconfssw. Therefore, we conclude I confssw S. The samereasoning can also be used to prove that I confsb Simplies I confssb S.

If I confssb S, then we have that, for all evolution of I,there exists an instance that is faster than the fastestinstance of the same evolution in S. In particular, thisinstance is also faster than the slowest instance of S.Therefore, we conclude that I confssw S.

Regarding the implications relating weak confor-mance relations, we can argue in the same way.

Finally, it only remains to study the implicationsbetween strong and weak conformance relations. Thereasoning is equivalent. The differences between themconsist of the way temporal conditions must be fulfilled.Although strong relations demand conditions over thetime value of each transition that composes an evolution,weak relations do it over the total time of the evolution.In fact, it is straightforward to check that, if the differentconditions fulfill, for each of the transitions, then theyfulfill for the corresponding evolution containing allthose transitions. Hence, we have that, if I confsa S, thenI confwa S, if I confww S, then I confsw S and I confsb Simplies I confwb S. For the same reason, we have thatI confsswS ) I confwswS and I confssbS ) I confwsb S. tu

Let us remark that the implications inferred in the previousresult are obviously transitive. For instance, we also haveI confwa S ) I confwsw S. Let us also note that, if specificationsare deterministic, then, on one hand, the relations confwb andconfww coincide, while, on the other hand, confsb and confswalso coincide. However, these relations would still bedifferent from the confwa and confsa relations. Similarly, ifthis property holds in the implementations, then all relationsconcerning the best instanced timed evolutions of theimplementation (sometimes relations) coincide with thecorresponding relation, where all of the instanced timedevolutions are taken into account.

Lemma 1. Let I ¼ ðSI; II ; OI; TOI; TrI; sinI ; �yIÞ and S ¼ðSS; IS; OS; TOS; TrS; sinS ; �ySÞ be two TEFSMs. If there donot exist different transitions ðs; s0; i; o; Q; Z; CÞ andðs; s00; i; o; Q0; Z0; C0Þ belonging to TrS , then

I confsb S , I confsw S I confssw S , I confssb S;

I confwb S , I confww S I confwsw S , I confwsb S:

If there do not exist different transitions ðs; s1; i; o; Q; Z; CÞand ðs; s2; i; o; Q

0; Z0; C0Þ belonging to TrI , then

I confsb S , I confssb S I confsw S , I confssw S;

I confwb S , I confwsb S I confww S , I confwsw S:

The hierarchy of relations induced by Theorem 1 allowsus to compare implementations in the following way: I1 ispreferable to I2 to implement S if the former meets with S, astricter relation.

Definition 7. Let I1, I2; , and S be TEFSMs and confx and confybe timed conformance relations such that I1 confx S,I2 confy S, confx ) confy, and confy 6) confx. We saythat I1 is preferred to I2 to implement S and we denote it byI1 >S I2.

4 DEFINITION AND APPLICATION OF TESTS

In our setting, tests represent sequences of inputs applied toan IUT. Once an output is received, it must be checkedwhether it belongs to the set of expected ones or not. Inaddition to checking the functional behavior of the IUT,tests also have to detect whether wrong timed behaviorsappear. Thus, tests have to include capabilities to deal withboth ways of specifying time. On one hand, we will includetime stamps. The time values recorded from the IUT whileapplying the test will be compared with the ones expectedby the specification. Each time stamp will contain a set oftime sequences corresponding to the time values that thespecification establishes for each transition of a timedevolution. Since we do not restrict nondeterministicbehavior, we will have as many time sequences as possibletimed evolutions can exist for a given input/outputsequence. Moreover, depending on the number of inputsapplied so far, we will have different lengths for theassociated time sequences appearing in the time stamps ofthe test. For example, if we reach a pass state of the test afterapplying three inputs and receiving three outputs, then thestate will include as time stamp a sequence containing threetime values. In addition, tests will include delays beforeoffering input actions. The purpose of delays is to induce


time-outs in the tested machine. For example, let us

consider testing whether a machine conforms to M10, as

defined in Fig. 3. We can define a test inducing a delay of

zero time units in the tested machine and checking that,

after applying i1, we receive o1. Another test might apply a

delay of one time unit; afterward, we should observe o2

after applying the same input i1. Thus, we may indirectly

check whether the time-outs imposed by the specification

are reflected in the IUT by offering input actions after a

specific delay. Let us note that we cannot observe when the

IUT takes a time-out. However, it is still possible to check

the IUT behavior after different delays. Finally, let us

remark that our tests have a more complex structure than

being a sequence of inputs (and its corresponding expected

outputs). This situation also appears in other testing

frameworks (see, for example, [33], where quasi-reduction

and quasi-equivalence are studied in the framework of

nondeterministic FSMs).

Definition 8. A test is a tuple

T ¼ ðS; I; O; Tr; s0; SI; SO; SF ; SP ; C;WÞ;

where S is the set of states, I and O are disjoint sets of input

and output actions, respectively, Tr � S � ðI [OÞ � S is the

transition relation, s0 2 S is the initial state, and the sets SI ,

SO, SF , and SP constitute a partition of S, that is, these four

sets represent a division of S into nonoverlapping parts that

cover all of S. The transition relation and the sets of states

fulfill the following conditions:

. SI is the set of input states. We have that s0 2 SI . Forall input states s 2 SI , there exists a unique outgoingtransition ðs; a; s0Þ 2 Tr. For this transition, we havethat a 2 I and s0 2 SO.

. SO is the set of output states. For all output states 2 SO, we have that for all o 2 O there exists a uniquestate s0 such that ðs; o; s0Þ 2 Tr. In this case, s0 62 SO.Moreover, there do not exist i 2 I and s0 2 S such thatðs; i; s0Þ 2 Tr.

. SF and SP are the sets of fail and pass states,respectively. We say that these states are terminal.Thus, for all states s 2 SF [ SP we have that there do notexist a 2 I [O and s0 2 S such that ðs; a; s0Þ 2 Tr.

Finally, we have two timed functions. C :

SP�!S1j¼1 PðTimejÞ is a function associating time stamps

with pass states and W : SI�!Time is a function associating

delays with input states.

We say that a test T is valid if the graph induced by T is atree with root at the initial state s0. In the rest of the paper, weconsider only valid tests.

Let � ¼ i1=o1; . . . ; ir=or. We write T ¼)�sT if sT 2

SF [ SP and there exist states s12; s21; s22; . . . sr1; sr2 2 Ssuch that fðs0; i1; s12Þ; ðsr2; or; sT Þg � Tr, for all 2 � j � rwe have ðsj1; ij; sj2Þ 2 Tr and, for all 1 � j � r� 1 we haveðsj2; oj; sðjþ1Þ1Þ 2 Tr.

Let T be a test, � ¼ i1=o1; . . . ; ir=or, sT be a state of T and

t 2 Timer. We write T ¼)�

�tsT if T ¼)

�sT , t1 ¼Wðs0Þ, and

for all 1 < j � r we have tj ¼Wðsj1Þ.In Fig. 4, we show a graphical representation of some

tests. Let us remark that T ¼)�sT and its variant T ¼)

�

�tsT

imply that sT is a terminal state. Next, we define the

application of a test suite to an implementation. We say that

the test suite T is passed if, for all tests belonging to the suite,

we have that the terminal states reached by the composition

of implementation and test are pass states. Besides, we give

different timing conditions in a similar way as we did for

implementation relations.

Definition 9. Let I be a TEFSM, T be a valid test,

� ¼ i1=o1; . . . ; ir=or, sT be a state of T , t ¼ ðt1; . . . ; trÞ, and

�to ¼ ðto1; . . . ; torÞ. We write IkT ¼)�

�tsT if T ¼)

�

�tsT and

ð�t; �Þ 2 InsFEvolðIÞ. We write IkT ¼)�

�t;�tosT if IkT ¼)

�

�tsT

and ð�t; �; �toÞ 2 InsTEvolðIÞ. Let

e ¼ ð�t; �; �toÞ 2 InsTEvolðIÞ:

We define the set

Testðe; T Þ ¼ fðT; sT ÞjT 2 T ^ IkT ¼)�

�t;�tosTg:

We say that

. I passes the test suite T , denoted by passðI; T Þ, if forall tests T 2 T there do not exist �, sT , t such thatIkT ¼)

�

�tsT and sT 2 SF .

. I strongly passes the test suite T for any time ifpassðI; T Þ and for all e ¼ ð�t; �; �toÞ 2 InsTEvolðIÞand for all ðT; sT Þ 2 Testðe; T Þ we have that �to 2CðsT Þ holds.

. I strongly passes the test suite T in the best time ifpassðI; T Þ and for all e ¼ ð�t; �; �toÞ 2 InsTEvolðIÞ,for all ðT; sT Þ 2 Testðe; T Þ, and for all �tc 2 CðsT Þ wehave that �to � �tc holds.


Fig. 4. Examples of tests.

. I strongly passes the test suite T in the worst time ifpassðI; T Þ and for all e ¼ ð�t; �; �toÞ 2 InsTEvolðIÞand for all ðT; sT Þ 2 Testðe; T Þ, there exists �tc 2CðsT Þ such that �to � �tc holds.

. I strongly passes the test suite T sometimes in besttime if passðI; T Þ and there exists e ¼ ð�t; �; �toÞ 2InsTEvolðIÞ such that for all ðT; sT Þ 2 Testðe; T Þand for all �tc 2 CðsT Þ we have that �to � �tc holds.

. I strongly passes the test suite T sometimes in worsttime if passðI; T Þ and there exists e ¼ ð�t; �; �toÞ 2InsTEvolðIÞ such that for all ðT; sT Þ 2 Testðe; T Þ,there exists �tc 2 CðsT Þ such that �to � �tc holds.

. I weakly passes the test suite T for any time ifpassðI; T Þ and for all e ¼ ð�t; �; �toÞ 2 InsTEvolðIÞand for all ðT; sT Þ 2 Testðe; T Þ, we have that

P�to ¼P

�tc holds for some �tc 2 CðsT Þ.. I weakly passes the test suite T in the best time if

passðI; T Þ and for all e ¼ ð�t; �; �toÞ 2 InsTEvolðIÞ,for all ðT; sT Þ 2 Testðe; T Þ, and for all �tc 2 CðsT Þ,we have that

P�to �

P�tc holds.

. I weakly passes the test suite T in the worst time ifpassðI; T Þ and for all e ¼ ð�t; �; �toÞ 2 InsTEvolðIÞand for all ðT; sT Þ 2 Testðe; T Þ, there exists �tc 2CðsT Þ such that

P�to �

P�tc holds.

. I weakly passes the test suite T sometimes in besttime if passðI; T Þ and there exists e ¼ ð�t; �; �toÞ 2InsTEvolðIÞ such that for all ðT; sT Þ 2 Testðe; T Þand for all �tc 2 CðsT Þ, we have that

P�to �

P�tc

holds.. I weakly passes the test suite T sometimes in worst

time if passðI; T Þ and there exists e ¼ ð�t; �; �toÞ 2InsTEvolðIÞ such that for all ðT; sT Þ 2 Testðe; T Þthere exists �tc 2 CðsT Þ such that

P�to �

P�tc holds.

5 TEST DERIVATION

In this section, we present an algorithm to derive tests from

specifications. As usual, the idea underlying our algorithm

consists of traversing the specification in order to get all of

the possible input/output sequences in an appropriate way.

First, we introduce some additional notation. The following

functions will be used in the forthcoming derivation

algorithm.

Definition 10. Let M ¼ ðS; I; O; TO; Tr; sin; �yÞ be a TEFSM.

The function outðs; i; �xÞ computes the set of outputs

associated with those transitions that can be executed from s

after receiving the input i, and assuming that the value of the

variables is given by �x:

outðs; i; �xÞ ¼ o9 s0 2 S;Q;Z;C :ðs; s0; i; o; Q; Z; CÞ 2 Tr^Qð�xÞ

��8<:

9=;:

We have that afterTOðs; t; �x; �tÞ computes the state thatwill be reached in M if we start in the configuration ðs; �xÞ andt time units pass without receiving an input:

afterTOðs; t; �x; �tÞ ¼ðs; �x; �tÞ if �2ðt0Þ > t;

afterTOð�1ðt0Þ; t� �2ðt0Þ; �x; �tÞ otherwise;

�

where we are considering t0 ¼ TOðsÞ. Let us be reminded thatTOðsÞ denotes the time-out associated with the state s, that is,a pair containing the reached state after the performance of thetime-out and when the time-out will be triggered.

The function afterðs; i; o; �x; �tÞ computes all of the statesthat can be reached from a state s after receiving the input i,producing the output o, for a value of the variables �x andassuming that the time values included in �t ¼ ðt1; . . . ; tnÞdenote the different time values needed to perform previousactions. In addition, this function also returns the new value ofthe variables and the updated tuple of time values consumedsince the system started its performance:

afterðs; i; o; �x; �tÞ ¼

ðs0; �x0; �t0Þ

9 Q;Z;C :

ðs; s0; i; o; Q; Z;CÞ 2 Tr^Qð�xÞ ^ Zð�xÞ ¼ �x0 ^ Cð�xÞ ¼ t^�t0 ¼ ðt1; . . . ; tn; tÞ

��

9>>>=>>>;:

8>>><>>>:

The previously defined functions can be extended in thenatural way to deal with sets:

outðG; Þ ¼[ðs;�xÞ2G outðs; i; �xÞ;

afterTOðD; tÞ ¼ fafterTOðs; t; �x; �tÞjðs; �x; �tÞ 2 Dg;afterðD; i; oÞ ¼

[ðs;�x;�tÞ2D afterðs; i; o; �x; �tÞ:

The algorithm to derive tests from a specification is givenin Fig. 5. This algorithm is nondeterministic and itsapplication generates a single test. By considering all ofthe possible nondeterministic choices in the algorithm, weextract a full test suite from the specification. Let us remarkthat this set will be, in general, infinite. For a givenspecification M, we denote this test suite by testsðMÞ.Essentially, our algorithm consists of traversing the speci-fication M in all of the possible ways. In addition togenerating tests that take into account the outputs that canbe followed by the application of an input, the tests will alsoinclude temporal information regarding the time valuesassociated with the performance of these outputs in thespecification.

Next, we explain how the algorithm works. A set ofpending situations D keeps those triples denoting the statesthat could have been reached in the transversal of thespecification, the corresponding value of the variables, andsequences of time values that could appear in a state of thetest whose definition (that is, the construction of itsoutgoing transitions) has not been yet completed. A pairðD; sT Þ 2 Saux indicates that we did not complete the statesT of the test and that the possible situations for that state inthe specification are given by the set D. Let us remark thatD is a set of situations, instead of a single one due to thenondeterminism that can appear in the specification.

Example 9. LetM ¼ ðS; I; O; TO; Tr; sin; �yÞ be a TEFSM. Let ussuppose that we have two transitions ðs; s0; i; o; Q1; Z1; C1Þand ðs; s00; i; o; Q2; Z2; C2Þ 2 Tr. If we want to compute theevolutions ofM after performing i=o, we have to consider


both s0 and s00. Formally, for a configuration ðs; �xÞ andtaking into account that the time values consumed for eachtransition so far are given by �t ¼ ðt1; . . . ; tnÞ, we have tocompute the set afterðfðs; �x; �tÞg; i; oÞ. As we explainedbefore, the application of this function will return thedifferent situations that can be reached by the specificationafter receiving the input i and generating the output o,considering that the current configuration of the machineis ðs; �xÞ. Therefore, if Q1ð�xÞ and Q2ð�xÞ hold, we have a setwith two elements, that is, fðs0; Z1ð�xÞ; �t1Þ; ðs00; Z2ð�xÞ; �t2Þg,where �t1 ¼ ðt1; . . . ; tn; C1ð�xÞÞ and �t2 ¼ ðt1; . . . ; tn; C2ð�xÞÞ.

Let us consider the different steps of the algorithm. Theset Saux initially contains a tuple with the initial situation ofthe specification (that is, the initial state, the initial value ofvariables, and the empty tuple of time values) and the initialstate of the test. For each tuple belonging to Saux, we havetwo possibilities (under the heading inductive cases). Thefirst possibility simply indicates that the state of the testunder construction becomes a passing state (inductivecase 1 of the algorithm). If the second possibility is chosen,then it has to be checked that there exists a delay td and aninput i such that the specification can perform at least oneoutput after applying the input i after the delay td (this isformalized in the side condition associated with the secondinductive case). If this is the case, we update the time valuesby considering that this delay is applied (Step 2.b of thealgorithm). Next, we generate an input transition in the testlabeled by i and having as delay td (Steps 2.e-g of the

algorithm). Then, the whole set of the outputs is consideredto generate a new transition in the test for each of theseoutputs. If the output is not expected by the specification(Step 2.h of the algorithm), then a transition leading to afailing state is created. This could be simulated by a singlebranch in the test, labeled by else, leading to a failing state.For the expected outputs (Step 2.i of the algorithm), wecreate a transition with the corresponding output action andadd the appropriate tuple to the set Saux, that is, the newpending situations after traversing the transitions corre-sponding to the input i and each of the expected outputs.

It is important to remark that the second inductive casecan be chosen only when the set Saux becomes singleton.This restriction implies that our derived tests correspond tovalid tests, as introduced in Definition 8. Let us also notethat finite tests are constructed simply by considering a stepwhere the second inductive case is not applied.

Example 10. Next, we show how our algorithm works togenerate tests. In Fig. 4, we present some tests derivedfrom the specification presented in Fig. 1. We supposeagain that the variables of the TEFSM are given by a tuple�x 2 IR4

þ that initially �x ¼ ð1; 2; 2; 1Þ.In order to generate the test T1, a delay of four time

units is applied in Step 2.a of the algorithm; the input i1is chosen. A transition labeled by this input is generatedin the test. Next, all outputs are considered. Due to thefact that, after i1, the specification only accepts the outputo2, two transitions leading to a fail state are created for


Fig. 5. Derivation of test cases from a specification.

the outputs o1 and o3, respectively (Step 2.h of thealgorithm). Moreover, a transition for the output o1 iscreated in the test (Step 2.i of the algorithm). After this, theinput i2 is selected and for example, a delay of five timeunits is established for this input state. After this, thecorresponding transitions are created in the test for theaccepted/forbidden outputs in the specification. Finally,Step 1 of the algorithm is applied in order to conclude thegeneration of this test. Only one pass state is created whichcontains a tuple of time values corresponding to thetransitions traversed in the specification. It will be used tocompare the time values that the implementation takes toperform outputs with the ones that are presented in thespecification. The tuple ð4; 3Þ that appears in the pass stateof the central branch of T1 is extracted from the instancedtemporal evolution ð4=i1=o2=4; 5=i2=o1=3Þ of the specifica-tion that was used to create the test. The tests T1 and T2

consider the same input, i1, in the first transition. Thedifference lies in the delays that have been considered foreach of them, four and two time units, respectively. Thisfact means that, for the test T2, the output o1 is accepted.

Regarding T3, let us note that the pass state presents a

set with two tuples. This is due to the fact that, if the

input i2 is applied without considering a delay (zero

time units), the machine could trigger either the

transition s2 ��!i2=o3

s1 or the transition s2 ��!i2=o3

s5. The

machine would emit the same output o3, but the amount

of time spent in performing these transitions would be

different for each of them, four and two units of time,

respectively.

Let us comment on the finiteness of our algorithm. If wedo not impose any restriction on the implementation (forexample, a bound on the number of states), we cannotdetermine some important information such as the maximallength of the sequences that the implementation canperform. In other words, we would need a coverage criterion

to generate a finite test suite. Since we do not assume anycriteria, all we can do is to say that the derived test suite isthe (possibly infinite) suite that would allow us to provecompleteness. Obviously, one can impose restrictions suchas “generate n tests” or “generate all of the tests withm inputs,” and completeness will be obtained up to thatcoverage criterion.

The next result relates, for a specification S and animplementation I, the implementation relations and appli-cation of test suites. The nontimed aspects of our algorithmare based on the one developed for the ioco relation.Therefore, in spite of the differences, the nontimed part ofthe proof of the next result is a simple adaptation of that in[34]. Regarding temporal aspects, let us remark that theexistence of different instances of the same timed evolutionin the specification is the reason why only some tests (andfor some time values) are forced to be passed by theimplementation (for example, sometimes we only need thefastest/slowest test). Specifically, we take those tests matchingthe requirements of the specific implementation relation. Inthis sense, the result holds because the temporal conditionsrequired to conform to the specification and to pass the testsuite are, in fact, the same.

Theorem 2. Let S and I be two TEFSMs. We have that

. I confsa S iff I strongly passes testsðSÞ for any time.

. I confsw S iff I strongly passes testsðSÞ in the worsttime.

. I confsb S iff I strongly passes testsðSÞ in the besttime.

. I confssw S iff I strongly passes testsðSÞ sometimesin the worst time.

. I confssbS iff I strongly passes testsðSÞ sometimes inthe best time.

. I confwa S iff I weakly passes testsðSÞ for any time.

. I confww S iff I weakly passes testsðSÞ in the worsttime.

. I confwb S iff I weakly passes testsðSÞ in the besttime.

. I confwsw S iff I weakly passes testsðSÞ sometimes inthe worst time.

. I confwsb S iff I weakly passes testsðSÞ sometimes inthe best time.

Proof. We will prove the first result. The technique is

similar for the other ones.First, let us show that I strongly passes testsðSÞ for

any time implies I confsa S. We will use the contra-positive, that is, we will suppose that I confsa S does nothold and we will prove that I does not strongly passtestsðSÞ for any time. If I confsa S does not hold, thenwe have two possibilities:

. either I conff S does not hold or

. there exists an instanced timed evolutione ¼ ð�t; �; �toÞ 2 InsTEvolðIÞ such that ð�t; �Þ 2InsFEvolðSÞ and e 62 InsTEvolðSÞ.

Let us consider the first case, that is, we suppose thatI conff S does not hold. Then, there exist e ¼ ð�t; �Þ, with� ¼ ði1=o1; . . . ; =ir=orÞ and �t ¼ ðt1; . . . ; trÞ, and e0 ¼ ð�t; ��Þ,with �� ¼ ði1=o1; . . . ; =ir=o

0rÞ, being r � 1 and such that

e 2 InsFEvolðSÞ, e0 2 InsFEvolðIÞ, and e0 62 InsFEvolðSÞ.We will show that there exists a test

T ¼ ðS; I; O; Tr; s; SI; SO; SF ; SP ; C;WÞ 2 testsðSÞ

such that T ¼)�

�tsT , with sT 2 SP , and T ¼)

�0

�tuT , with

uT 2 SF . By constructing such a test T , we obtain IkT ¼)�0

�tuT for a fail state uT . Thus, we conclude that S does not

pass testsðSÞ. We will build this test T by applying the

algorithm given in Fig. 5. The different nondeterministic

choices underlying the algorithm will be resolved in the

following way:

1. for 1 � j � r, do

a. Apply inductive case 2 for the input action ijselecting td :¼ tj.

b. Apply inductive case 1 for all the elementsðD; sT Þ 2 Saux that are obtained by processingan output different from oj.

endfor.2. Apply the first inductive case for the last elementðD; sT Þ 2 Saux.


Let us remark that Step 1a corresponds to continuingthe construction of the test in the state reached by thetransition labeled by oj�1 (in the case of j ¼ 1, we meanthe initial state of the test). Let us also note that the spineof the constructed test is the sequence �.

Since e0 62 InsFEvolðSÞ, the last application of the

second inductive case for the output o0r must necessarily

be associated to Step 2e. Therefore, the previous algorithm

generates a test T such that T ¼)�0

�tuT with uT 2 SF . Since

e0 2 InsFEvolðIÞ, we have that IkT ¼)�0

�tuT . Given the fact

that T 2 testsðSÞ, we deduce that passðI; testsðSÞÞdoes not hold. Thus, we conclude that I does not

strongly pass testsðSÞ for any time.

Let us suppose now that I confsa S does not hold

because there exists e ¼ ð�t; �; �toÞ 2 InsTEvolðIÞ such

that ð�t; �Þ 2 InsFEvolðSÞ and e 62 InsTEvolðSÞ. Since

ð�t; �Þ 2 InsFEvolðSÞ, let us consider the test T that we

built before. We again have T ¼)�

�tsT , with sT 2 SP . The

time stamps associated with the state sT are generated by

considering all of the possible tuples of time values in

which � could be performed in S by considering the

delays contained in �t. Besides, since e 2 InsTEvolðIÞ, we

also have IkT ¼)�

�t;�tosT . Thus, if e 62 InsTEvolðSÞ, then

�to 62 CðsT Þ. We conclude I does not strongly pass

testsðSÞ for any time.

Let us show now that I confsa S implies I stronglypasses testsðSÞ for any time. Again by contrapositive,we will assume that I does not strongly pass testsðSÞfor any time and we will conclude that I confsa S doesnot hold. If I does not strongly pass testsðSÞ for anytime, then we have two possibilities:

. either passðI; testsðSÞÞ does not hold or

. there exist e ¼ ð�t; �; �toÞ 2 InsTEvolðIÞ andðT; sT Þ 2 Testðe; testsðSÞÞ such that IkT ¼)

�

�t;�tosT ,

with sT 2 SP and �to 62 CðsT Þ.

First, let us assume that I does not strongly pass

testsðSÞ for any time because passðI; testsðSÞÞ does

not hold. This means that there exists a test T 2 testsðSÞand some � ¼ ði1=o1; . . . ; ir=orÞ, sT 2 SF , and �t such that

IkT ¼)�

�tsT . Then, there exists �t such that T ¼)

�

�tsT .

According to our derivation algorithm, a branch of a

derived test leads to a fail state only if its associated

output action is not expected in the specification. Thus,

e ¼ ð�t; �Þ 62 InsFEvolðSÞ. Let us note that our algorithm

allows to create a fail state only as the result of the

application of the second inductive case. One of the

premises of this inductive case is outðSM; iÞ 6¼ ;, that is,

the specification is allowed to perform some output

actions after the reception of the corresponding input.

Thus, there exists an output action o0r and a sequence of

pairs input/output �� ¼ ði1=o1; . . . ; ir�1=or�1; ir=o0rÞ such

that e0 ¼ ð�t; ��Þ 2 InsFEvolðSÞ. Given the fact that

e 2 InsFEvolðIÞ, e 62 InsFEvolðSÞ, and e0 2 InsFEvolðSÞ,we have that I conff S does not hold. We conclude

I confsa S does not hold.

Let us suppose now that I does not pass testsðSÞ for

any time because there exist e ¼ ð�t; �; �toÞ 2 InsTEvolðIÞand T 2 Testðe; testsðSÞÞ such that IkT ¼)

�

�t;�tosT , with

sT 2 SP and �to 62 CðsT Þ. Since sT 2 SP , we deduce

ð�t; �Þ 2 InsFEvolðSÞ. Besides, by considering that�to 62 CðsT Þ, we deduce ð�t; �; �toÞ 62 InsTEvolðSÞ. Finally,

by using ð�t; �; �toÞ 2 InsTEvolðIÞ, we conclude that

I confsa S does not hold. tuAs a straightforward corollary, we have that the

dependencies between conformance relations that wepresented in Theorem 1 also hold for the correspondingtesting relations. For instance, since confa ) confw, we alsodeduce that “passes T at any time” implies “passes T in theworst time.”

6 CONCLUSIONS AND FUTURE WORK

In this paper, we have proposed a new model to specifytimed systems. In contrast with most approaches, ourformalism allows us to define time in two different ways:duration of actions and time-outs of the system. Thus, byseparating these two notions, temporal properties ofsystems are easier to specify than by using a formalismwhere only one of the possibilities is available. We have alsodeveloped a testing theory. On one hand, we have definedseveral conformance relations that take into account theinfluence of nondeterminism in the behavior of systems. Onthe other hand, we have introduced a notion of test. In orderto capture the timed behavior of the system under test, atest can both delay the execution of the implementation andrecord the time that it took to perform a given action.Finally, we have also presented an algorithm to derivesound and complete test suites with respect to theimplementation relations presented in this paper.

In terms of future work, we would like to take this workas a first step, together with [35] and [36], to define a testingtheory for systems presenting stochastic time together withtime-outs. An interesting continuation of this work, assuggested in one of the reviews, consists of dealing withfault models. In this paper and keeping the classicalapproach of ioco-like frameworks, we considered thatany fault can appear. However, it would be interesting toanalyze how the results of this paper have to be adapted todeal with restricted fault models, for example, to consideronly those machines that have a (known) bound on thenumber of states. Along this line, we also consider studyingdifferent methods for reducing the size of the generated testsuite. Finally, we would like to apply our methodology tosome real systems, particularly to the testing of hardwarecircuits. We already have a first prototype that has to beoptimized since it does not yet scale to deal with systemshaving a large number of states.

ACKNOWLEDGMENTS

The authors would like to thank the reviewers of this paperfor the careful reading. The quality of the paper has notablyincreased by considering their useful comments and


suggestions. This research was partially supported by the

Spanish MEC project WEST/FAST (TIN2006-15578-C02-01)

and the Marie Curie project MRTN-CT-2003-50512/TAROT.

REFERENCES

[1] M. Merayo, M. Nunez, and I. Rodrıguez, “Extending EFSMs toSpecify and Test Timed Systems with Action Durations andTimeouts,” Proc. 26th IFIP WG 6.1 Int’l Conf. Formal Techniques forNetworked and Distributed Systems, pp. 372-387, 2006.

[2] B. Bosik and M. Uyar, “Finite State Machine Based FormalMethods in Protocol Conformance Testing,” Computer Networksand ISDN Systems, vol. 22, pp. 7-33, 1991.

[3] D. Lee and M. Yannakakis, “Principles and Methods of TestingFinite State Machines: A Survey,” Proc. IEEE, vol. 84, no. 8,pp. 1090-1123, 1996.

[4] A. Petrenko, “Fault Model-Driven Test Derivation from FiniteState Models: Annotated Bibliography,” Proc. Fourth SummerSchool on Modeling and Verification of Parallel Processes, pp. 196-205, 2001.

[5] E. Brinksma and J. Tretmans, “Testing Transition Systems: AnAnnotated Bibliography,” Proc. Fourth Summer School on Modelingand Verification of Parallel Processes, pp. 187-195, 2001.

[6] K. El-Fakih, N. Yevtushenko, and G.v. Bochmann, “FSM-BasedIncremental Conformance Testing Methods,” IEEE Trans. SoftwareEng., vol. 30, no. 7, pp. 425-436, July 2004.

[7] I. Rodrıguez, M. Merayo, and M. Nunez, “HOT L: Hypothesesand Observations Testing Logic,” J. Logic and Algebraic Program-ming, vol. 74, no. 2, pp. 57-93, 2008.

[8] J. Sifakis, “Use of Petri Nets for Performance Evaluation,” Proc.Third Int’l Symp. Measuring, Modelling and Evaluating ComputerSystems, pp. 75-93, 1977.

[9] G. Reed and A. Roscoe, “A Timed Model for CommunicatingSequential Processes,” Theoretical Computer Science, vol. 58,pp. 249-261, 1988.

[10] W. Yi, “CCS + Time = An Interleaving Model for Real TimeSystems,” Proc. 18th Int’l Colloquium on Automata, Languages, andProgramming, pp. 217-228, 1991.

[11] X. Nicollin and J. Sifakis, “An Overview and Synthesis on TimedProcess Algebras,” Proc. Third Int’l Conf. Computer Aided Verifica-tion, pp. 376-398, 1991.

[12] J. Quemada, D.d. Frutos, and A. Azcorra, “TIC: A TimedCalculus,” Formal Aspects of Computing, vol. 5, pp. 224-252, 1993.

[13] R. Alur and D. Dill, “A Theory of Timed Automata,” TheoreticalComputer Science, vol. 126, pp. 183-235, 1994.

[14] M. Hennessy and T. Regan, “A Process Algebra for TimedSystems,” Information and Computation, vol. 117, no. 2, pp. 221-239,1995.

[15] J. Davies and S. Schneider, “A Brief History of Timed CSP,”Theoretical Computer Science, vol. 138, pp. 243-271, 1995.

[16] J. Baeten and C. Middelburg, Process Algebra with Timing, Springer,2002.

[17] M. Nunez and I. Rodrıguez, “Conformance Testing Relations forTimed Systems,” Proc. Fifth Int’l Workshop Formal Approaches toSoftware Testing, pp. 103-117, 2006.

[18] J. Springintveld, F. Vaandrager, and P. D’Argenio, “Testing TimedAutomata,” Theoretical Computer Science, vol. 254, no. 1-2, pp. 225-257, 2001.

[19] R. Barbuti and L. Tesei, “Timed Automata with Urgent Transi-tions,” Acta Informatica, vol. 40, no. 5, pp. 317-347, 2004.

[20] B. Gebremichael and F. Vaandrager, “Specifying Urgency inTimed I/O Automata,” Proc. Third IEEE Int’l Conf. Software Eng.and Formal Methods, pp. 64-73, 2005.

[21] P.-A. Hsiung, S.-W. Lin, Y.-R. Chen, C.-H. Huang, J.-J. Yeh, H.-Y.Sun, C.-S. Lin, and H.-W. Liao, “Model Checking Timed Systemswith Urgencies,” Proc. Fourth Int’l Symp. Automated Technology forVerification and Analysis, pp. 67-81, 2006.

[22] D. Clarke and I. Lee Automatic Generation of Tests for TimingConstraints from Requirements, Proc. Third Workshop Object-Oriented Real-Time Dependable Systems, pp. 199-206, 1997.

[23] T. Higashino, A. Nakata, K. Taniguchi, and A. Cavalli, “Generat-ing Test Cases for a Timed I/O Automaton Model,” Proc. 12th Int’lWorkshop Testing of Communicating Systems, pp. 197-214, 1999.

[24] A. En-Nouaary and R. Dssouli, “A Guided Method for TestingTimed Input Output Automata,” Proc. 15th Int’l Conf. TestingCommunicating Systems, pp. 211-225, 2003.

[25] M. Krichen and S. Tripakis, “Black-Box Conformance Testing forReal-Time Systems,” Proc. 11th Int’l SPIN Workshop Model Checkingof Software, pp. 109-126, 2004.

[26] L. Brandan Briones and E. Brinksma, “Testing Real-Time MultiInput-Output Systems,” Proc. Seventh Int’l Conf. Formal Eng.Methods, pp. 264-279, 2005.

[27] H. Fouchal, E. Petitjean, and S. Salva, “An User-Oriented Testingof Real Time Systems,” Proc. IEEE Workshop Real-Time EmbeddedSystems, 2001.

[28] R. Cardell-Oliver, “Conformance Tests for Real-Time Systemswith Timed Automata Specifications,” Formal Aspects of Comput-ing, vol. 12, no. 5, pp. 350-371, 2000.

[29] R. Cardell-Oliver and T. Glover, “A Practical and CompleteAlgorithm for Testing Real-Time Systems,” Proc. Fifth Int’l Symp.Formal Techniques in Real-Time and Fault-Tolerant Systems, pp. 251-260, 1998.

[30] D. Mandrioli, S. Morasca, and A. Morzenti, “Generating TestCases for Real Time Systems from Logic Specifications,” ACMTrans. Computer Systems, vol. 13, no. 4, pp. 356-398, 1995.

[31] J. Peleska and M. Siegel, “Test Automation of Safety-CriticalReactive Systems,” South African Computer J., vol. 19, pp. 53-77,1997.

[32] M. Nunez and I. Rodrıguez, “Encoding PAMR into (timed) EFSMs,”Proc. 22nd IFIP WG 6.1 Int’l Conf. Formal Techniques for Networkedand Distributed Systems, pp. 1-16, 2002.

[33] A. Petrenko and N. Yevtushenko, “Conformance Tests asChecking Experiments for Partial Nondeterministic FSM,” Proc.Fifth Int’l Workshop Formal Approaches to Software Testing, pp. 118-133, 2006.

[34] J. Tretmans, “Test Generation with Inputs, Outputs and RepetitiveQuiescence,” Software—Concepts and Tools, vol. 17, no. 3, pp. 103-120, 1996.

[35] M. Nunez and I. Rodrıguez, “Towards Testing Stochastic TimedSystems,” Proc. 23rd IFIP WG 6.1 Int’l Conf. Formal Techniques forNetworked and Distributed Systems, pp. 335-350, 2003.

[36] M. Merayo, M. Nunez, and I. Rodrıguez, “ImplementationRelations for Stochastic Finite State Machines,” Proc. ThirdEuropean Performance Eng. Workshop, pp. 123-137, 2006.

Mercedes G. Merayo is currently workingtoward the PhD degree in the ComputerSystems and Computation Department at theUniversidad Complutense de Madrid, where sheis also an assistant teacher. She has published15 papers in refereed journals and internationalvenues. Her research interests are formalmethods and probabilistic/timed/stochastic ex-tensions in formal testing.

Manuel Nunez received the PhD degree inmathematics/computer science and the MSdegree in economics from the UniversidadComplutense de Madrid, in 1996 and 2002,respectively. Since 1997, he has been anassociate professor in the Computer Systemsand Computation Department at the same uni-versity. He has published more than 80 papers inrefereed journals and international venues. Hisresearch interests include formal notions of

testing and performance evaluation, application of economic theory tocomputer science, and formal specification and analysis of e-commercesystems.

Ismael Rodrıguez received the PhD degree incomputer science from the Universidad Complu-tense de Madrid, Spain, in 2004, receiving theBest Thesis Award from the Computer ScienceSchool. Since 2007, he has been an associateprofessor in the Computer Systems and Compu-tation Department at the same university. He haspublished more than 50 papers in refereedjournals and international venues. His researchinterests include formal methods, testing techni-

ques, e-learning environments, and e-commerce.


Extending EFSMs to Specify and Test Timed Systems with Action Durations and Timeouts

Documents

Transcript of Extending EFSMs to Specify and Test Timed Systems with Action Durations and Timeouts