Conformance Testing for Statecharts

Dirk Seifert, Steffen Helke, and Thomas Santen

Technical University of Berlin,Institute for Software Engineering and Theoretical Computer Science,

Sekr. FR 5-6, Franklinstr. 28/29, D-10587 Berlin,Germany,fdseifert,helke,santeng@cs.tu-berlin.de

Abstract. We describe an approach to automatically generate test cases fromobject-oriented statecharts as they are used in the UML and supported by devel-opment tools such as I-Logics Rhapsody.Applying Tretmans’ approach to generate test cases for I/O automata from la-beled transition systems, we consider a statechart as a specification of a systemunder test. Test cases generated from such a specification can be used to drive atest harness, creating stimuli for the system under test andobserving the reactionsof the system under test to produce a verdict.This work contributes in three respects to using statecharts for specifying andverifying systems. First, it provides a semantics of the features of object-orientedstatecharts that go beyond labeled transition systems suchas asynchronous se-quential event processing, truly parallel event generation, and input-enabledness.Second, defining two variants of an implementation relationfor statecharts itmakes the notion of conformance testing for statecharts precise. Third, it in-troduces a compact data structure for representing the statechart semantics thatallows for efficient generation of test cases and easily supports both variants ofimplementation relations. A prototypical implementationcooperating with thetool Rhapsody serves as a proof of concept.

1 Introduction

Statecharts [7] are a well accepted notation for specifyingreactive systems. They areuseful in particular to specify embedded software, such as the software for controllersin automobiles, because the reactive behavior usually is the most important aspect ofsoftware in those systems.

Nowadays, as the industrial take-up of modeling languages such as the UML [12]is increasing, statecharts used in an object-oriented context become more and more im-portant. This is why there is an active field of research [6, 11, 4] on defining a semanticsof statecharts that is compatible with the object-orientedparadigm [12], and on build-ing tools such as Rhapsody [9] to support an object-orientedstyle of modeling reactivesystems with statecharts.

One of the reasons why the object-oriented paradigm is increasingly accepted fordeveloping embedded systems is their increasing complexity. Today, in an automobile,for example, dozens of embedded controllers communicatingwith each other are the

norm rather than the exception. Testing those systems is extremely costly. In this situa-tion, not only support for constructing such systems but also for testing them thoroughlyand efficiently is needed.

In this paper, we address the problem of generating test cases from object-orientedstatecharts as they are supported by Rhapsody. We base our work on Tretmans’ foun-dational research on test case generation from labeled transition systems [13]. To adapthis algorithm to the specifics of object-oriented statecharts, we address three problems.

First, we provide a semantics of object-oriented statecharts that includes a modelof observational behavior, distinguishing between input,output, and internal events.The semantics addresses the specific features of object-oriented statecharts: althoughevents may occur simultaneously, they are processed asynchronously and sequentially.Therefore, a semantic model must store not yet processed andnewly occurring eventsuntil the preceeding ones have been processed. Furthermore, like classical statecharts[7], object-oriented ones just ignore events for which no explicit transition is specifiedin an active state. This makes statecharts input enabled, asthey will never refuse toprocess an event. We discuss those semantic features and describe a formal semanticsreflecting them in Section 3.

Second, we define a meaning of conforming behavior of one statechart (the im-plementation) to another (the specification). Actually, wedefine two implementationrelations that differ in the way they interpret the “defaultbehavior” of ignoring eventsfor which no explicit treatment is specified: one implementation relation treats thatdefault behavior of the specification strictly, allowing the implementation to reducenon-determinism only; the other implementation relation treats the default behavior as“don’t care” and allows the implementation to extend behavior that the specificationdoes not explicitly prescribe. After a brief discussion of conformance testing in gen-eral, Section 4 introduces those two implementation relations for statecharts.

Third, we address the problem of exponentially growing semantical representationsfor statecharts with the previously mentioned features. Weintroduce acompact seman-tic interpretationof an object-oriented statechart as a data structure that isconsiderablysmaller than a straightforward representation of the semantics of a statechart. The com-pact semantic interpretation reflects the observable behavior of a statechart and canserve as input to an (appropriately modified) version of Tretmans’ algorithm [13]. Wedescribe the compact semantic interpretation in Section 5,and the test case generationin Section 6.

Before we present the technical details, we remind the reader of the basic features ofobject-oriented statecharts and introduce a running example in Section 2. Section 7 putsour contribution in the context of other work. Section 8 summarizes, briefly describesa prototypical implementation of the algorithm presented in Section 6, and discussespossible extensions of our work.

2 Example Specification

The statechart in Figure 1 describes the behavior of a (simplified) money changingmachine. The machine has two reservoirs for coins of 1BC and 2BC. Additionally, it has aslot to insert 5BC bills, and a button to request change. When somebody inserts a 5BC bill

��

��

��

MoneychangerChange Box1

Box20

2 2’

3 4

5

Full

Full

Empty

Empty

i1/1BCi1/1BC

i2/2BC i2/2BCi2/2BC i2/2BC

5BC but / i1but / i2

Box1 empty

Box2 empty

Box1 full

Box2 full

Fig. 1.Statechart for a Moneychanger.

into the slot and repeatedly presses the button, then the machine will output a numberof 1BC and 2BC coins up to a total of 5BC.

The statechart has a root state calledMoneychangerthat consists of three orthogo-nal substates (and-states):Change, Box1, andBox2. The statesBox1 andBox2 modelreservoirs of 1BC and 2BC coins, respectively.

A box is in the substateFull if its reservoir contains a sufficient number of coins,otherwise it is in the substateEmpty. For Boxk, an eventBoxk emptyoccurs if thenumber of coins falls below a certain limit. After service personal refills the box aneventBoxk full will occur. The default transitions, i.e. the transitions originating fromthe pseudo-state, pointing at the statesFull in Box1 andBox2 signify that initially, whenthe machine starts working, both boxes are full.

If Boxk is in stateFull, then pressing the button of the machine (eventbut) causesa transition fromFull to itself, which generates an (internal) eventik: the labelbut/ikof the transition means that the eventbut triggers the transition and, if the transitionfires, then the eventik is generated. Therefore, if both boxes are full, pressing the buttonproduces an eventbut, that is visible to both substatesBox1 andBox2 and thus producestwo (internal) eventsi1 andi2.

The third orthogonal substateChangespecifies an algorithm of changing 5BC billsdepending on the generated and processed events. InitiallyChangeis in state5. If a 5BCbill is inserted into the machine (event5BC), thenChangegoes to state0. Pressing thebutton of the machine causes the statesBoxk to generate eventsik. Changeprocessesthese events by outputting a number of 2BC and 1BC coins until it finally reaches state5and is ready for processing the next 5BC bill1.

The algorithm we present in the following takes a statechartlike the one in Figure 1and produces a number of test cases to test whether a given implementation exhibitsthe behavior modeled in the chart. A test case for Figure 1 would, for example, reject

1 Depending on the number of coins in the machine, it may actually occur thatChangedoes notreach state5. This is, of course, undesirable in practice, but helps to keep the example simple.The same holds for the defined strategies to change a 5BC bill.

an implementation outputting 1BC immediately after an input of5BC andbut, whereas itwould accept the sequence of observations [5BC,but,2BC,1BC,but,2BC].

In addition to orthogonaland-states, statecharts allow for another manner of com-posing states:xor-states to hierarchically organize states. If the statechart in Figure 1was extended by transitions beginning and ending at the stateMoneychanger, then tak-ing a transition leavingMoneychangerwould cause all substates ofMoneychangerto beleft, and taking a transition ending atMoneychangerwould cause the default substatesof Moneychangerto be entered as well.

3 UML Statecharts

In the UML, each class has an optional statechart which describes the dynamic behaviorof all of instances of the class [12]. As a consequence of integrating statecharts into theobject-oriented paradigm, UML statecharts have many extensions caused by the needof modeling communication between objects. These are, for example, several differenttypes of events modeling synchronous or asynchronous message processing, timing,and other properties of object communication. As another consequence, the behavior ofUML statecharts, in particular the way events are processedand firing transitions aredetermined, also differs from the behavior of classical statecharts [7].

In the present paper, we focus on the latter aspect of the semantics of object-orientedstatecharts and disregard the complexity added by different types of messages and thedata they may carry. Defining the set of statecharts we consider and the semantics wedefine for those statecharts, the following sections make that restriction precise. Thosedefinitions partly build on other work on defining semantics of object-oriented state-charts [6, 11, 4]. In the following sections, we particularly emphasize the semantic fea-tures of object-oriented statecharts that make test case generation hard.

3.1 Statecharts with Asynchronous Signal Events

A statechartSC is a 7-tuple as follows:SC = (�;�is; �; �; �;�;E)The set of states� is hierarchically structured. The set�is � � is the set of initial

states. The role of each state in the hierarchy is determinedby the function� : � !fsimple; xor; andg. This function partitions� into a set of simple states�ss and aset of composite states� s. The set of composite states consists of mutual exclusionstates�xor and orthogonal states�and. For each composite state in�xor, there existsa unique initial substate in�is.

The substate relation� determines the hierarchical structure of the states and de-scribes a tree structure. The expressionA�B means thatB is a substate ofA.

Labeled transitions relate the states of a statechart. The set � contains all validlabels. The transition relation� � � � ��� defines the source, label, and target ofeach transition inSC .

The setE of signal events supports asynchronous communication. It can be splitinto the mutually disjoint sets of internal (unobservable)eventsEint and observable

eventsEobs. The observable events are again distinctly classified as incoming events inEin and outgoing events inEout.The labels in� have the formtrigger/action, wheretrigger2 Ein [Eint represents

the occurrence of a single event, andaction� Eout [ Eint is a set of events generatedwhen the transition fires.

3.2 Semantics

We associate a labeled transition systemSCs to every statechartSC as defined in theprevious section. In contrast toSC , the semantic interpretation is non-hierarchical anddefined as a 4-tupel as follows:SCs = (�s; �sis; �s; �s)

A member of the set�s of semantic states is called astatusand consists of twocomponentshC;Qi. The first componentC is a set of active states ofSC , calledactivestate configuration. Such a configuration represents a tree of active states respecting thehierarchical structure that� imposes on the states ofSC. The second componentQ is acontainer of events, called anevent queue. The event queue contains the events that thestatechart has received during a run but has not processed yet. The UML specification[12] does not define the exact nature of the event queue2. In accordance with the choicemade in Rhapsody, we interpretQ as a FIFO queue in the present paper.

All initial statuses (in�sis) consist of a valid initial configuration and an empty eventqueue.

Supporting conformance testing is a major goal of the semantics we define here.Therefore, we are especially interested in the observable aspects of a statechart specifi-cation, and rely on a semantics that hides the internal behavior of SCs. This becomesobvious in the definition of the transition relation�s � �s � �s � �s that relatesstatuses of�s by labeled transitions. A label in�s is a set ofobservableevents, i.e.a subset ofEin [ Eout. BecauseEin andEout are disjoint, the label of a transitionexactly determines the events that the statechart receivesfrom the environment and theones it produces when the transition fires. The labelfg represents a transition step withunobservable, internal communication only.

The special features of object-oriented statecharts mentioned in Section 1 becomeobvious when we consider how the semantic model reflects the dynamics of a statechart,i.e. how transitions ofSC map to transitions ofSCs. The transition relation�s relatesstatuses via labeled transitions:�s � �s � �s ��s.

The semantics we describe here assumes a synchronous time model, where a globalclock triggers each step (the parallel firing of non-conflicting enabled transitions) ofthe statechart. It also assumes an open system semantics where aglobal environmentproduces a set of incoming events that the statechart accepts with each time step.

The first event in the event queue (thecurrent event) determines the set of enabledtransitions. Thetransition selection algorithm[12] identifies the maximal sets of non-conflicting transitions as subsets of the set of enabled ones. In general, there is morethan one of those sets. This is one source of non-deterministic behavior of a statechart.

2 Thus, the UML specification allows different sematic interpretations that may be adequate indifferent domains of application.

Executing the transitions of one non-empty maximal set of non-conflicting tran-sitions yields a new configuration (thenext configuration) and produces a number ofinternal and outgoing events. Thenext statusof SCs contains the next configurationand thenext queuethat results from deleting the current event from the event queueand appending the internal events that the transitions produce to the event queue. If thetransitions produce more than one internal event, those events can be appended to thequeue in any order. The same is true for the incoming events the statechart receives fromthe global environment. Therefore, the statuses consisting of the next configuration andone of the possible next queues can all be reached from the current status, which isanother source of non-determinism in a statechart: the set of the outgoing events and ofthe incoming events make up the labels of the transitions from the current status to eachof the next statuses.

If the set of non-conflicting transitions is empty, then there is no transition startingin a state of the current configuration whose trigger is the current event. In this case thestatechart ignores the current event, removing it from the queue while adding the in-coming events from the global environment to the queue. Thenthe statechart continuesto process the next event in the queue.

It depends on the application context whether or not that default behavior shouldconstrain possible implementations. In Section 4, we definetwo implementation rela-tions on statecharts that differ in the way they treat the default behavior. To make thispossible in a modular fashion, we define thecompletion�s of the transition relation�s with �s � �s, and for each status in�s that is not the source of a transition in�s, �s contains transitions labeled with any possible set of incoming events and lead-ing to a status whose configuration is identical to the current configuration and whoseevent queue is the current event queue with the current eventremoved and the incomingevents added.

4 Conformance Testing

As we have seen in Section 3.2, the semantics of a statechart can be defined as a (verylarge) labeled transition system. Therefore, it is clear that an approach for conformancetesting based on labeled transition systems is, in principle, applicable to statecharts.There are, however, two major issues in adapting such an approach to statecharts: first,an adequate conformance relation on statecharts must be defined, and, second, a com-pact representation of the semantics of a statechart must befound that makes the testcase generation algorithm practically feasible. We address the latter issue in Section 5.We summarize Tretmans’ approach to conformance testing forlabeled transition sys-tems in Section 4.1, and we motivate two variants of a conformance relation for state-charts in Section 4.2.

4.1 Conformance Testing for LTS

Tretmans [13] describes an approach for conformance testing based on labeled transi-tion systems (LTS). He classifies the labels into inputs and outputs which an LTS uses tocommunicate with its environment. The specification of a reactive system is an arbitrary

LTS. As a test hypothesis, Tretmans assumes that every implementation can be modeledas an input-enabled LTS, i.e. an LTS that never rejects an input from the environment.

Based on this hypothesis, the correctness of an implementation with respect to aspecification can be modeled formally by a conformance relation between labeled tran-sition systems. Many different conformance relations are possible, their use dependingon the application context. All definitions discussed in [13] consider the possible obser-vations a system can perform after certain traces.

For his approach to conformance testing, Tretmans uses a conformance relation thatconsiders only those traces of the implementation that are covered by the specification.An implementation correctly implements a specification if any possible observationmade of an implementation after a certain trace of the specification, can be related toan observation made of the specification by the same observer. Thus, this conformancerelation requires that an implementation does what it should do. But it does not requirethat an implementation does not do what it is not allowed to do.

Formally, an implementationi is input/output-conformwith respect to a given spec-ifications, if for all traces� of the specification the observations of the implementationare included in the observations of the specification:i conforms to s =def 8� 2 traces(s) :

observations(i after �) � observations(safter�)Based on this definition of conformance, Tretmans describesa non-deterministic

algorithm to generate test cases for conformance testing. Input to that algorithm is thesuspension automaton of a specification which is deterministic and includes transitionsmodeling time-outs.

A test case is an LTS constructed by one of the three followingsteps. First, it mayjust terminate with the verdictpass. Second, it may supply an input to the implementa-tion according to the behavior of the specification. Third, it may check the output of theimplementation against the output expected by consideringthe specification, producingthe verdictpassif the output conforms to the specification, otherwise the verdict fail.

4.2 Conformance Testing for Statecharts

A critical issue in determining an adequate conformance relation for statecharts is todecide on the role of transitions in the specification that stem from the completion�sof �s. Consider, for example, the statechart of Figure 1. If the chart is in state2’ ofChange, and the current event is5BC, then, according to the completed transition relation,the chart will stay in that configuration while the current event5BC will be deleted fromthe event queue.

Figure 2 shows a statechart that, in the same configuration, reacts differently to acurrent event of5BC: that chart will produce an eventrejectto make the moneychangerreject the additionally inserted bill and return it to the customer.

Depending on the intuitive interpretation of the default behavior represented by thecompletion, the statechart of Figure 2 may or may not be acceptable as an implementa-tion of the one of Figure 1: if the specifiers consider the default behavior as “don’t care”,

��

��

��

b1f

b1e

MoneychangerChange Box10

2 2’Full

i1 /1BCi2 /2BC i2 /2BCi2 /2BC 5BC

/reject

Fig. 2.Handling Nonspecified Input.

because they do not describe it explicitly, then Figure 2 is an acceptable implementa-tion; if, however, the specifiers do not want an implementation to change the observablebehavior of the specification in any case, then Figure 2 is notacceptable.

Using different labeled transition systems, we can captureboth variants of a con-formance relation for statecharts formally. LetSCs be a statechart with semanticsSCss = (�ss ; �siss; �ss; �ss) and completed semanticsSCss = (�ss ; �siss; �ss; �ss).Let SCi be another statechart with similar definitions ofSCsi andSCsi . We callSCiweakly conformingto SCs if SCsi conforms to SCss , and we callSCi stronglyconformingto SCs if SCsi conforms to SCss . The statechart of Figure 2 weakly –but not strongly – conforms to the one of Figure 1.

Introducing-observations, the compact semantic interpretation we present in thefollowing section supports test case generation for both notions of conformance, asdescribed in Section 6.

5 Compact Semantic Interpretation

The semantics of a statechart as outlined in Section 3.2 produces very large labeledtransition systems with much redundancy. One reason is thatthe set of events generatedin one step can be enqueued in the event queue in any possible order – producing sta-tuses (and transitions leading to them) that differ only in the order of events in the eventqueues. Another reason is that all possible incoming eventsmust be dealt with in allstatuses although most of them will just be enqueued and eventually dequeued withoutcausing any observable effect. In the present section, we describe the compact seman-tic automaton (CSA) as a data structure with much less redundancy, and we sketch analgorithm to generate a CSA from a statechart description.

5.1 Compact Semantic Automaton (CSA)

In the following, we call an incoming eventrelevantif there is at least one transitionfrom the current configuration that this event can trigger; otherwise, we call the eventnegligible.

For the example of the money changing machine, Figure 3 illustrates the relationbetween a compact semantic automaton and the correspondingsemantic interpretation.

��������

��������

��������

......

C5,��C5,

��C5,

�5BC�

C0,��C0,

��C0,

�but

�C0 ,

�i1,i2� C0 ,

�i2,i1�

C0 ,�i2�

C20 ,��C2,

��C2 ,

�i1�C20 ,

�i1�

C3,��

C3,��

C2 ,�i1,i2� C2 ,

�i2,i1�

2 � 1 1 2�2 � 1 2 1�

C2,�but

�C0 ,

�fi1,i2g�C2 ,

�fi1g�C20 ,�fi1g�

C2 ,�fi1,i2g�

C2 ,�fi1g,fi1,i2g�

C0,�fi1g, fi1,i2g�

C5,�Ein

�C0,

�Ein

�C0 ,

�Ein, fi1,i2g�

5BC/fg

but/fgbut/fg

but/fgbut/fgi2/f2BCgi2/f2BCg

i1/f1BCg

5BC/fg[Ein

but/fg[Ein

f1BCgf2BCgf2BCg f2BCgf2BCg

f5BCg

fg fgfgfg fg fgfg

fÆg fbutgfbutg

Semantic LTS

(incomplete)(incomplete)

Compact SemanticAutomaton

Interpretation ofCSA for incoming

events

Fig. 3.Compact Representation of a Section of the Semantic Automaton.

The left hand side of the figure shows a part of the semantics ofthe moneychanger, thecenter of the figure shows the corresponding CSA, and the right hand side shows aninterpretation of the CSA considering all possible incoming events.

In the initial statushC5; [℄i, the statechart is in the default states and the event queueis empty. The relevant incoming events are5BC, but, Box1 emptyandBox2 empty. Thepart shown in Figure 3 considers only what happens if the incoming events5BC andbutoccur in that order with ample time between them for the moneychanger to completelyprocess5BC first3. Therefore, there is only one transition from the initial status to astatus where5BC is the only member of the event queue. Processing that event leads toconfigurationC0 and leaves the event queue empty. Processingbut in C0, the chartsBox1 and Box2 generate the eventsi1 and i2 at the same time. Therefore, there aretwo transitions leading to statuses with event queues reflecting the possible orders ofprocessingi1 and i2. Processing those events non-deterministically leads to one of theconfigurationsC2 or C20 , and so forth.

Combining certain statuses with the same configuration, a compact semantic au-tomaton reduces redundancy. A CSA is defined as a 4-tupel as follows:SC = (� ; � is; � ; � )

3 Note that in the complete semantics, there are transitions for all incoming events starting ateach of the statuses shown in Figure 3.

� is a set of compact semantic states. A compact semantic stateis calledcompactstatusand consists of two components. The first component is a stateconfiguration likein a semantic status. The second component is a queue ofsetsof marked events. Theset of marked events contains each evente 2 Eint in two copies, a marked copyeand an unmarked copye. It also contains a distinguished event =2 E that we use tomark transitions stemming from the completion�s of the semantics (c.f. Section 3.2).A member of the set of initial compact statuses� is has an initial configuration and anempty queue.

A member of the queue represents all permutations of the internal events that aregenerated simultaneously in an atomic synchronous time step. Thus a compact statusmay represent a number of semantic statuses. The compact status

C0,

�fi1;i2g�� inFigure 3, represents the semantic stateshC0,[i1,i2]i andhC0,[i2,i1]i.

A marked event, such asi1 inC20 ,�fi1g��, represents one that may be present or not

in a member of the event queue: the queue�fi1; i2g� represents the event queues

�i1,i2�

and�i2,i1�, and after processing i2 the resulting queue either is empty or it contains only

i1.The compact statuses of aSC are related by labeled transitions. A label in�

has the formtrigger/actionobs, wheretrigger represents theprocessingof an event andactionobs is the set of outgoing events including that processingtrigger producesin one time step. Additionally, we add a special labelÆ to � that indicates possiblequiescence [13], i.e. a situation in which the system cannotproceed autonomously.

A CSA does not explicitly represent the observation of incoming events but thetriggers of transitions show when incoming events are processed. Because statechartsare input-enabled, any incoming event can be received at anytime. The right-hand sideof Figure 3 shows that explicitly representing those eventswould result in the set ofmarkedincoming eventsEin being the heads of the queues in all compact statuses –which makes it redundant.

5.2 Generation of Compact Semantic Automata

Starting at the initial statuses of a statechartSC , we generate the corresponding CSASC by calculating all successive statuses. The labels of the resulting CSA contain allobservable behavior ofSC , which makesSC an appropriate basis for calculating testcases as described in Section 6.

The functionsstepandgenerateprovide the functionality to generateSC fromSC.The functionstepdetermines the labels of transitions starting at a given configurationand event queue that are triggered by a given event, and the compact statuses they reach.

Algorithm 1 ( step : � �E ! IP(� �� ))Input to the function is the current statuss=h ;Qi, containing the current configuration and the current queueQ, and the current evente. The function next calculates all nextconfigurations and the sets of actions generated by triggering e. Output of the functionis a set of pairs containing the compact label and the next status:[n( 0; A) : next( ; e) � �e=A\Eout; h ;Qb(A�Eout)i�o

Based on the maximal sets of non-conflicting transitions (c.f. Section 3.1), the func-tion nextdetermines all possible configurations 0 reachable from by e, and the sets ofeventsA the statechart generates when going from to 0. Combining that information,stepproduces the corresponding set of compact statuses and transition labels.

Whereasstepproduces only compact statuses and transition labels that result fromprocessing a given trigger event, the functiongeneratedetermines all compact statusesreachable from a current one. This includes selecting trigger events, marking events inthe event queue, insertingÆ-transitions to indicate quiescence, producing-transitionsto represent default behavior, and, of course, processing the information provided bystep.

Algorithm 2 ( generate : � ! IP(� �� ))Input to the function is the current statuss=h ;Qi, containing the current configuration and the current queueQ. Output of the function is a set of pairs consisting of thecompact label and the next status:LET Et = trigger(enabledTrans( )) trigger eventsEr = Et \ Ein relevant incoming trigger eventsEn = Ein �Er negligible incoming trigger eventsS = Sne : En � �e=; h ;Qi�o calculate-edges

(1)

IN IF Q=[ ℄ THEN(� processEr and add aÆ-edge and the-edges�)Sne : Er � step�h ;Qi; e�o [ n�Æ; h ;Qi�o [ S (2)

ELSE IF8e :hdQ�marked?(e) THEN(� distinguish empty and nonempty head and add the-edges�)generate

� ; tl Q��[generate

� ; unmark(hdQ)btl Q�� [ S (3)

ELSE LETEri = hdQ \ Et relevant internal eventsEni = hdQ�Eri negligible internal events(4)

IN (� processEri andEr and add the-edges�)Sne : Eri [Er � step� ; �Eri�feg [ mark(Eni)�btl Q�; e�o [ S (5)

After determining the sets of relevant and negligible events (1), the functiongen-eratedistinguishes three situations. First, if the current queue is empty (2), then thestatechart can process only the relevant incoming events generated by the global envi-ronment. If the environment does not supply a relevant event, then the system cannot

continue autonomously, which is marked by aÆ-edge. Adding-edges for all neg-ligible incoming events models the default behavior of the completed semantics (c.f.Section 3.2).

Second, if the queue is not empty and all events in the head of the current queueare marked (3), then the semantic status represented by the current configuration andthat queue either contains an empty queue or a queue whose head is any of the internalevents in that set. Two recursive calls togenerateproduce the behavior for those twosituations. Again, appropriate-edges model the default behavior.

Third, if the head of the queue contains unmarked events, then all continuations forthe relevant internal (4) and incoming events need to be considered (5). The functionstepsupplies the continuations for each relevant evente. If a negligible internal event inEni was head of the current semantic queue, then it would be deleted from that queuewithout an observable effect. Therefore, the members ofEni are marked4 in the queueof the reached compact status.

6 Test Case Generation

A test case is a labeled transition system that checks whether a certain part of the observ-able behavior of an implementation under test conforms to the specification (describedby a statechart). To this end, a test case generates stimuli (in Ein) for the implementa-tion, and compares its reaction (inEout) to the one the specification prescribes.

To generate test cases based on the CSA for a statechart similar to Tretmans’ algo-rithm (c.f. Section 4.1), it remains to determine theobservabletraces of a specificationfrom the CSA and to produce test cases from these traces. The main problem here is todetermine the points in time at which the test case shall supply stimuli to the implemen-tation under test.

A CSA for a statechart completely includes the observable behavior of that chart. Togenerate test cases from a CSA, we traverse the CSA and collect all observable events.These traces are the basis for the test case generation.

The incomingevents that stimulate a certain behavior are the triggers ofthe transi-tions on a particular path in a CSA. Those transitions mark the points in an executionwhere the incoming events are processed (c.f. Figure 3). However, the CSA does not de-termine when such an event is generated: because a statechart is input-enabled and cangenerate internal events in parallel to receiving incomingevents, the triggering eventcan have been enqueued in the event queue some time before it is actually processed.

In the following, we assume that the processing order of incoming events in thecompact semantic automaton is equal to the order of the eventgeneration within theglobal environment. Thus a statechart can process an eventb after an eventa onlyif it receivesa beforeb from the environment. Because the global environment andthe implementation under test do not synchronize the generation of events, we cannotassume a specific order of incoming events and generated outgoing events. Hence, thetest case generation procedure must consider any possible order of those events, whilepreserving the order of outgoing events.

4 Here, contradicting the intuition of a “queue”, we change the head of the queue rather thendequeuing it.

����

����

����

����

... ... ... ...

C5 ,� � C5 ,

� �C5 ,

� �C5 ,

� �C0 ,

� �C0 ,

�fi1,i2g�C2 ,

�i1�

C3 ,� �

f5BCgf5BCgf5BCgf2BCgf2BCg f1BCgf1BCgfbutgfbutgfbutgfbutg f2BC,butg f1BC,butgfgfg

fgfg

fg

C5,�5BC�C5,

�5BC�C5,

�5BC�

C0,�but

�C0,

�but

�C0,

�but

�C0,

�i2 ,i1,but

�C0 ,

�i2,i1�C0 ,

�i2,i1�

C2,�i1,but

�C2,

�i1,but

�C2 ,

�i1�

C3,�but

�C3,

�but

�C3,

�but

�5BC/fgbut/fgi2/f2BCgi1/f1BCgbut/fg

=)Fig. 4.Determining Possible Generation Points of Incoming Events

To clarify this point, consider the following path of the CSAin Figure 3, which is aresponse to feeding a 5BC bill to the moneychanger and pressing the button two times:5BC/fg � but/fg � i2/f2BCg � i1/f1BCg � but/fg � : : : =) 8><>:5BC � but � but � 2BC � 1BC � : : :5BC � but � 2BC � but � 1BC � : : :5BC � but � 2BC � 1BC � but � : : :

The last three labels in the path contain the observations2BC,1BC andbut. Becausethe system produces2BC and1BC but the environment supplies the eventbut, the lattercan occur at any time before, between or after the two other events.

In general, we need to generate all possible interleavings between incoming andoutgoing events while preserving causality and the order ofevents in the respectiveclasses. Figure 4 shows how this process produces the different semantic interpretationsthat a path of the example CSA represents.

With this procedure to generate traces from a CSA, we can produce input for Tret-mans’ algorithm which generates test cases as sketched in Section 4.1. The test casesshown in Figure 5 are intended to illustrate the result of applying the complete proce-dure to the example statechart. Each test case produces stimuli for the implementationunder test, which are marked by an exclamation mark. It receives responses of the im-plementation, which are marked by a question mark. The response ?� denotes a time-outstemming from aÆ-transition in the CSA. This “observation” covers the case that theimplementation does not respond to a stimulus but waits for another one. Each termina-tion node carries a verdict of the test case, one of the labels“pass” or “fail”, dependingon whether or not the chain of stimuli and responses on a path of the test case conformsto the specification.

��

��

��

��

����

�� �� ��

��

��

��

�� �� ��

��

�� �� ��

��

��

��

�� �� ��

�� �� ��

��

��

���

���

���

���

���

���

��������

��������

������������

������������

��������

��������

��������

��������

!5BC!5BC!5BC!but!but!but

!but

!but

!but

?1BC?1BC ?1BC?1BC

?1BC?1BC?2BC

?2BC ?2BC?2BC?2BC ?2BC

?�?�?� ?�

?�?�passpass

passpass

pass

pass

failfail

failfail

failfail

failfail

fail

Fig. 5.Example Test Cases

7 Related Work

The generation of test cases from statecharts has been studied in different ways. Hong etal. [8] proposed an automatic test generation algorithm using model checking. Based onthe classical STATEMATE semantics [7], they translate statecharts into Kripke struc-tures and show how the model checker can be used to generate test cases with respectto given coverage criterion. We are not aware of work on generating test cases fromstatecharts according to a conformance test theory.

A detailed overview of the fundamental literature for formal testing can be foundin Brinksma’s and Tretmans’ annotated bibliography [2]. DeNicola and Hennessy [3]introduced a formal theory of testing on which Brinksma [1] built an approach to derivetest cases from a formal specification. This approach assumes that a testing processcan communicate synchronously with the system under test. This assumption is notadequate for statecharts. Recent work focuses on asynchronous communication [15]and is close to our approach.

To adapt Tretmans’ algorithm for UML statecharts needs a precise semantic basis.Many variants of semantics with subtle differences have been proposed for statecharts[16]. In recent years, some research has contributed to giving UML statecharts a pre-cise semantics [6, 11, 4]. Latella et al. [11] map statecharts to the intermediate format ofextended hierarchical automata and define an operational semantics on these automata.Eshuis et al. [6] give a real time semantics for UML statecharts and identify the differ-ences with STATEMATE statecharts in detail. Basically, ourcontribution to this workis to integrate a description of observable aspects in the semantics of statecharts.

The ideas of conformance testing were realized in some test tools to generate testcases automatically. An implementation of the Tretmans’ algorithm for the formal de-scription language SDL can be found in the commercial tool set ObjectGeode [10]. Thefirst larger test tool that is completely based on a formal model of conformance testingis TORX [14] and supports specifications in LOTOS and PROMELA. Grieskamp et al.[5] implement an algorithm for generating test cases from a given abstract state machine(ASM) specification.

8 Conclusions

The contribution of our work lies in refining the loose UML semantics for a subsetof object-oriented statecharts and defining a compact representation for the semanticinterpretation. To this end, we introduce a model of observational behavior and definetwo possible implementation relations for UML statecharts.

Two features make the complete semantic description of a non-trivial statechartimpracticably large: truly parallel event generation and input-enabledness. To cope withtruly parallel event generation, we must consider all permutations of the events that aregenerated simultaneously in an atomic synchronous time step. For input-enabledness,we must consider the extension of the queue in each atomic synchronous time step byany number of defined incoming events. Each feature results in a tremendously largeset of next states for which the CSA provides a compact representation.

In this paper, we describe the principle setting of our approach. We have a proto-typical implementation that collaborates with Rhapsody. The prototype consists of twoparts. Independently from Rhapsody, the first part analyzesa given statechart and gen-erates a test suite. The second part uses Rhapsody to animatethe generated test suiteand trigger a UML model. To achieve this, a Rhapsody model describes the test en-vironment and a test driver, which interpret a test suite. Tomake the algorithm moreefficient, we intend to generate the semantic interpretation “on the fly” and execute testcases as they are generated in a “step by step” fashion.

On the theoretical side, we are working on extensions of the theory of inter-objectcommunication and other kinds of events, in particular events carrying data and syn-chronous events.

References

1. E. Brinksma. A theory for the derivation of tests. InProtocol Specification, Testing andVerification, VIII. North-Holland, 1988.

2. E. Brinksma and J. Tretmans. Testing transition systems:An annotated bibliography.LectureNotes in Computer Science, 2001.

3. R. de Nicola and M. C. B. Hennessy. Testing equivalences for processes.Theoretical Com-puter Science, 1984.

4. R. Eshuis and R. Wieringa. Requirements level semantics for UML statecharts. InFor-mal Methods for Open Object-Based Distributed Systems, IV. Kluwer Academic Publishers,2000.

5. W. Grieskamp, Y. Gurevich, W. Schulte, and M. Veanes. Generating finite state machinesfrom abstract state machines.ACM International Symposium on Software Testing and Anal-ysis, 2002.

6. D. Harel and E. Gery. Executable object modeling with statecharts. In18th InternationalConference on Software Engineering. IEEE Press, 1996.

7. D. Harel and A. Naamad. The STATEMATE semantics of statecharts.ACM Transactions onSoftware Engineering and Methodology, 1996.

8. H.S. Hong, I. Lee, O. Sokolsky, and S.D. Cha. Automatic test generation from statechartsusing model checking. InWorkshop on Formal Approaches to Testing of Software. BRICSNotes Series, 2001.

9. I-Logix. Rhapsody in J, Version 3, 2001. www.ilogix.com.

10. A. Kerbrat, T. Jeron, and R. Groz. Automated test generation from SDL specifications. In9th SDL Forum. Elsevier Science, 1999.11. D. Latella, I. Majzik, and M. Massink. Towards a formal operational semantics of UML

statechart diagrams. InFormal Methods for Open Object-Based Distributed Systems,III .Kluwer Academic Publishers, 1999.

12. OMG – Object Management Group. Unified Modeling LanguageSpecification, Version 1.3,2000. www.omg.org.

13. J. Tretmans. Test generation with inputs, outputs, and repetitive quiescence. InWorkshop onTools and Algorithms for the Construction and Analysis of Systems. Springer-Verlag, 1996.

14. J. Tretmans and A. Belinfante. Automatic testing with formal methods. In7th European Int.Conference on Software Testing, Analysis & Review. EuroStar Conferences, 1999.

15. L. Verhaard, J. Tretmans, P. Kars, and E. Brinksma. On asynchronous testing. In5th Int.Workshop on Protocol Test Systems. North-Holland, 1992.

16. M. von der Beeck. A comparison of statechart variants. InFormal Techniques in RealTimeand FaultTolerant Systems. Springer-Verlag, 1994.