An Empirical Evaluation of the PLUSS Toolkit

An Empirical Evaluation of the PLUSS Toolkit

Magnus Eriksson, Henrik Morast, Jürgen Börstler and Kjell Borg

UMINF-06.31 ISSN-0348-0542

UMEÅ UNIVERSITY Department of Computing Science

SE-901 87 UMEÅ SWEDEN

An Empirical Evaluation of the PLUSS Toolkit Magnus Eriksson1, Henrik Morast2, Jürgen Börstler3 and Kjell Borg1

1 BAE Systems Hägglunds AB

SE-891 82 Örnsköldsvik Sweden

{Magnus.Eriksson, Kjell.Borg}@baesystems.se

2 Syntell AB PO Box 100 22

SE-100 55 Stockholm Sweden

[email protected]

3 Umeå University Dept. of Computing Science

SE-901 87 Umeå Sweden

[email protected]

ABSTRACT The PLUSS approach (Product Line Use case modeling for Systems and Software engineering) is a domain modeling method tailored towards the development of long lived software intensive systems, for example defense systems. The goal of the PLUSS approach is to provide means to maintain a common and complete use case model for a whole family of systems, and to provide a good overview of all variants within such a family. In this paper, we describe how the commercial requirements management tool Telelogic DOORS and the UML modeling tool IBM-Rational Rose can be extended and used for managing system family models in accordance with the PLUSS approach. We refer to these extensions as the PLUSS toolkit. An empirical study is presented where the PLUSS toolkit is applied and evaluated in the target domain. Results indicate that the toolkit allow developers to work effectively. Some minor shortcomings of the toolkit were however identified and we therefore propose improvements to increase its efficiency.

1 INTRODUCTION Software intense defense systems, for example vehicles, are developed in short series. They are always customized for specific customer needs and they are expected to have a very long life span, often 30 years or longer. For a business to be competitive in a market like this it is important to achieve high levels of reuse and effective maintenance. An interesting approach to address such issues is known as software product line development. The basic idea of this approach is to use domain knowledge to identify common parts within a family of products and to separate them from the differences between the products. The commonalties are then used to create a product platform that can be used as a common baseline for all products within the product family.

For embedded software we believe it is important that product line concepts such as domain modeling are also introduced into the systems engineering process. This due to the fact that embedded software requirements are for the most part not posed by customers or end users, but by systems engineering and the systems architecture. Due to earlier positive single system experiences with use cases, we are interested in identifying a use case driven product line approach that can be applied by both our systems- and software engineering teams. To address issues in existing product line use case modeling approaches we have developed a domain modeling approach that utilizes features models [11], use cases [10] and use case realizations [13]. We refer to this approach as the PLUSS (Product Line Use case modeling for Systems and Software engineering) approach [5]. The goal of the PLUSS approach is to provide means to maintain a common and complete use case model for a whole family of systems. Furthermore, the approach must be suitable for both systems- and software engineering, and provide a good overview of all variants within such a family model.

For software product line development to become efficient in every day practice, adequate tool support is an important factor. Unfortunately, tools for software product line development are today almost nonexistent. There are some experimental tools and very few commercial tools available. Furthermore, existing tools are not providing the functionality needed to support the PLUSS approach in a satisfying manner. The working group for “Practices and Issues Related to Product Line Tool Support” of the Fourth Product Line Practice Workshop identified a number of risks associated with using tools designed for single system development in a software product line environment [2]. The following list summarizes some of those risks:

1. There is a high risk that resources may be expended unnecessarily because of lack of adequate tool support.

2. There is a high risk that artifacts become inconsistent because tools do not enable linkage or traceability between the different types of artifacts.

3. There is a high risk that tools become difficult to maintain because of the need for local modifications to enable product line development.

Even tough we recognize and agree with these risks, we have chosen to use our existing single system CASE tool environment to support the PLUSS approach. The two main arguments for this decision were:

1. Using the existing CASE tool environment will minimize the need for training of personnel.

2. Using commercially available tools will minimize the amount of tool code that need to be maintained.

The decision has forced us to implement a number of small extensions to the tool set; we will for the reminder of this paper refer to these extensions as the PLUSS toolkit. Our strategy is however to keep these extensions as small as possible.

The main tools used to support the PLUSS approach, is the requirements management tool Telelogic DOORS [18] and the UML modeling tool IBM-Rational Rose [9]. Both of these tools are widely used and accepted in industry. The main contribution of this paper is an empirical evaluation of how well these tools, when using with the PLUSS Toolkit extensions, can be utilized to manage system family use case models according to the PLUSS approach.

The remainder of the paper is organized as follows: Section 2 provides a short introduction to the PLUSS approach. Section 3 describes how we use DOORS and Rose to manage our system family models and which extensions we have implemented to better support the approach. In section 4, we present an empirical industrial study in which the PLUSS toolkit is evaluated in the target domain. In section 5, we summarize the paper, draw conclusions and present some future work in the area.

2 THE PLUSS APPROACH – DOMAIN MODELING WITH FEATURES, USE CASES AND USE CASE REALIZATIONS

The concept of use cases was first introduced by Ivar Jacobson in the early 1990s and has today become a widely accepted and used requirement modeling technique, both in the software- and the systems engineering communities. Use case modeling is a functional decomposition technique that provides a semi formal framework for structuring requirements. A good use case should address one complete and well-defined goal that the user wants to accomplish by interacting with the system [1].

As we described in [5], it must be possible to manage variability among family members to be able to maintain a common use case model for a whole system family. Our approach for

managing variability in PLUSS is based on the work by Griss et al. on FeatuRSEB [8]. In FeatuRSEB a feature model is added to the 4+1 view model adopted by Jacobson et al. in RSEB [10]. The feature model in FeatuRSEB takes “center stage” and provides a high-level view of the domain architecture and the reusable assets in the product family. Kang et al. first proposed using feature models in 1990 as part of Feature Oriented Domain Analysis (FODA) [11]. In feature models, system features are organized into trees of AND and OR nodes that represent the commonalities and variations within a family of related systems. General features are located at the top of the tree and more refined features are located below. Originally, FODA described “Mandatory”, “Optional” and “Alternative” features that may have “requires” and “excludes” relations to other features. Mandatory features are available in all systems built within a family. Optional features represent variability within a family that may or may not be included in products. Alternative features represent an “exactly-one-out-of-many” selection that has to be made among a set of features. A “requires” relationship indicates that a feature depends on some other feature to make sense in a system. An “excludes” relationship between two features indicates that both features can not be included in the same system.

Like Griss et al. we argue that feature models are better suited for domain modeling than for example UML use case diagrams and that a feature model therefore should be used as the high level view of the product family. However, in PLUSS the primary purpose of the feature model is not to take “center stage”, but rather to be a tool for visualizing variants in our abstract product family use case model. We use the feature model as a tool for structuring and instantiating our abstract family models into concrete product use case models for each system built within the family. We also maintain use case realizations [13] and change cases [3] as part of the PLUSS system family model. We utilize use case realizations to trace variant use case behavior to the system design, and change cases to mark proposed however not yet accepted functionality in the domain

2.1 PLUSS Modeling Notations

2.1.1 PLUSS Use Case Modeling

As described in [4,5] and shown in Figure 1 and Figure 2, we have chosen to adopt an extended version of the RUP-SE “Flow of Events” notation [16] for describing use case scenarios and use case realizations. We have chosen natural language descriptions since the PLUSS approach must be applicable for both systems and software engineering. This increases the number and diversity of stakeholders interested in the models and thereby makes for example UML unsuitable for the purpose. Our natural language descriptions can however be supplemented with UML diagrams to improve understandability as needed.

As shown in Figure 1, the extension provides means to specify variants of use case scenarios in a very compact way. As shown in step ‘3c’ and step ‘5b’ of Figure 1 the PLUSS notation also provides means to specify parameters within use case descriptions. Like Mannion et al. [14], we distinguish between local parameters and global parameters. In PLUSS, the scope of a local parameter is the use case in which it resides and the scope of a global parameter is the whole domain model. Like Mannion et al. we use the symbols ‘$’ and ‘@’ respectively to denote local and global parameters.

Mandatory step

Optional step

Exactly one to beselected for amandatory step

At least one to beselected for amandatory step

Exactly one to beselected for anoptional step

At least one to be selected for anoptional step

Step Actor Action BlackboxSystem Response

BlackboxBudgeted Req.

1

23a3b3c(4)(5)a(5)b(5)c(6)(6)(6)

22

The Actor…

……………………………

……

The System…

……………………………

……

It shall…

………… @PARAM_1……… $PARAM_2…………

……

Mandatory step

Optional step

Exactly one to beselected for amandatory step

At least one to beselected for amandatory step

Exactly one to beselected for anoptional step

At least one to be selected for anoptional step



1

23a3b3c(4)(5)a(5)b(5)c(6)(6)(6)

22

The Actor…

……………………………

……

The System…

……………………………

……

It shall…

………… @PARAM_1……… $PARAM_2…………

……



1

23a3b3c(4)(5)a(5)b(5)c(6)(6)(6)

22

The Actor…

……………………………

……

The System…

……………………………

……

It shall…

………… @PARAM_1……… $PARAM_2…………

……

Figure 1: The PLUSS notation for describing variants in use case scenarios

DesignElement_1…

………………

Whitebox Action

DesignElement_2… DesignElement_3…

It shall…

………………

WhiteboxBudgeted Req.

……

1

2

3

Step

The Actor…

…

The use case ends when…

Actor Action

The System…

…

…

BlackboxSystem Response

It shall…

…

…


DesignElement_1…

………………

Whitebox Action

DesignElement_2… DesignElement_3…

It shall…

………………

WhiteboxBudgeted Req.

……

1

2

3

Step

The Actor…

…

The use case ends when…

Actor Action

The System…

…

…

BlackboxSystem Response

It shall…

…

…


Figure 2: An Example Use Case Realization in the PLUSS Notation

In PLUSS, change cases are modeled in UML use case diagrams utilizing the UML stereotype mechanism [15]. As shown in Figure 3, we stereotype use case as <<change case>> and the association relation as <<impact>> to visualize the needed entities.

Crew MemberView Video Select Video Source

<<include>>

<<change case>>View Picture-in-Picture Video

<<extend>>

<<impact>>

Crew MemberView Video Select Video Source

<<include>>

<<change case>>View Picture-in-Picture Video

<<extend>>

<<impact>>

Figure 3: An example Change case in UML

2.1.2 PLUSS Feature Modeling

Original FODA has no defined mechanism to specify the relation “at-least-one-out-of-many” [6]. We address this, according to our experience, important shortcoming by defining a new feature type called “Multiple Adaptor” feature in PLUSS. This feature type is similar to FODA’s alternative features, but instead of representing the “exactly-one-out-of-many” relationship, it captures the missing relationship. Its name follows the naming scheme proposed by Mannion et al. for the equivalent relation in their work on reusable requirements [14]. We have also chosen to rename alternative features to “Single Adaptor” features following the same naming scheme.

The feature modeling notation used in PLUSS is based on the FODA notation but it has been slightly modified to better suit our modeling needs as shown in Figure 4. As in the original notation a filled black circle represents a mandatory feature and a non-filled circle represents an optional feature. Single and multiple adaptor features are represented by the letters ‘S’ and ‘M’, respectively, surrounded by a circle. Single and multiple adaptor features are also color-coded in the PLUSS notation to ease identification in large feature graphs.

Domain

a

ac

Saac

Saaa

Saab

Saba

abaa

Sabb

b

Mbca

Mbcb

Sbc

Sbb

Mbbb

Mbba

<<excludes>>

<<requires>>

S Single AdaptorMandatory Optional M Multiple AdaptorRequires Excludes

ba

Mbcc

Domain

aa

acac

Saac

Saac

Saaa

Saaa

Saab

Saab

Saba

Saba

ababaaaa

SabbS

abb

bb

MbcaM

bcaM

bcbM

bcb

SbcSbc

SbbS

bb

MbbbM

bbbM

bbaM

bba

<<excludes>>

<<requires>>

S Single AdaptorMandatory Optional M Multiple AdaptorRequires Excludes

S Single AdaptorS Single AdaptorMandatoryMandatory OptionalOptional M Multiple AdaptorM Multiple AdaptorRequiresRequires ExcludesExcludes

baba

Mbcc

Mbcc

Figure 4: A Feature model example in the PLUSS notation

In the original FODA notation an arc connects sets of alternative features. Due to limitations in our tool support we had to remove this arc from the PLUSS notation. Instead we imposed a modeling constraint saying that only one set of single adaptor and multiple adaptor features respectively may exist under the same parent feature. This was necessary to avoid ambiguity in models regarding which set of single or multiple adaptor features a certain feature belongs to. If several sets are needed under the same parent, each set must be grouped under their own parent feature.

2.2 Relating Features and Use Cases Gomaa [7] proposed to model each feature as a use case package. We have extended this idea saying that possibly a whole set of features compose a use case package. This have the advantage of enabling us to also visualize variants within use cases specifications using the feature model.

As shown in the example in Figure 5, use cases as well as use case scenarios can be mapped to features of any type in the feature model to capture required variants among the members

of a system family. As shown in the example in Figure 6, there is also a predefined set of feature constructs that should be mapped against each type of variant in the use case scenario notation described in section 2.1.1. This means that we are actually maintaining redundant information in our notation. We do however consider this to be a minor problem since an automatic consistency check between the models could easily be implemented and we believe the extra information considerably improves the readability of the notation.

As shown in the example in Figure 6, also use case parameters are mapped to the feature model. A local parameter is mapped to a set of features on an appropriate level below the feature holding the use case owning the local parameter in the feature tree. A global parameter is mapped to a set of features on an appropriate level above the feature holding the highest level use case referencing the global parameter.

Domain

aa

Saac

Saac

Saaa

Saaa

Saab

Saab

Saba

Saba

ababaaaa

SabbS

abb

bb

MbcaM

bcaM

bcbM

bcb

SbcSbc

SbbS

bb

MbbbM

bbbM

bbaM

bba

baba

Mbcc

Mbcc

Optional Feature: abOptional Feature: ab

<<change case>>

<<impact>>

<<extend>>

Use Case: <X>Introduction<Some info...>

Main Success Scenario.Alternative Scenarios<Scenario Name 1>.<Scenario Name 2>.Exceptional Scenarios <Scenario Name>.

Use Case: <X>Introduction<Some info...>

Main Success Scenario.Alternative Scenarios<Scenario Name 1>.<Scenario Name 2>.Exceptional Scenarios <Scenario Name>.

Figure 5: An Example of the Relationship between Features, Use Cases and Use Case

Scenarios



1

23a3b3c(4)(5)a(5)b(5)c(6)(6)(6)

22

The Actor…

……………………………

……

The System…

……………………………

……

It shall…

………… @PARAM_1……… $PARAM_2…………

……

SS

SM

MM

SS

S

MM

M

S S S

S S S



1

23a3b3c(4)(5)a(5)b(5)c(6)(6)(6)

22

The Actor…

……………………………

……

The System…

……………………………

……

It shall…

………… @PARAM_1……… $PARAM_2…………

……

SS

SM

MM

SS

S

MM

M

S S SS S S

S S SS S S

Figure 6: An Example of the Relationship between Features Variant Scenario Steps in the

PLUSS notation

A meta-model for integration of features, use cases and use case realizations is presented Figure 7. It describes how use cases, scenarios and scenario steps are included by feature selections. It also shows how the included use case scenario steps prescribes a certain set of design elements via use case realizations. Variant use case behavior is thereby traced to the system design.

include

**

0..1

require

refine

*

exclude*

**

1

1

1..*

*

*

instantiate

1

include*

include

* 0..1*

realize1..*

1..*

2..*

composed-of

1..*

1..* *include

1..*

1

1..*

0..1

11

*

1..*

*

Feature

Domain Model

Parameter

Global Parameter Local Parameter

Use Case

Scenario

Step

Design Element

System Family

Use Case Realization

System

include

**

0..1

require

refine

*

exclude*

**

1

1

1..*

*

*

instantiate

1

include*

include

* 0..1*

realize1..*

1..*

2..*

composed-of

1..*

1..* *include

1..*

1

1..*

0..1

11

*

1..*

*

Feature

Domain Model

Parameter

Global Parameter Local Parameter

Use Case

Scenario

Step

Design Element

System Family

Use Case Realization

System

Figure 7: The PLUSS Meta-model in UML [5]

2.3 Product Instantiation in the PLUSS Approach Since we are maintaining a common use case model for the whole system family in PLUSS, product instantiation is basically done by adding any new requirements to the model and then using the feature model to choose among its variants. The set of included features directly corresponds to a specific set of included use cases for the product. A product use case model is then generated from the domain model by applying a filter, sorting out features not included in the current system. This will result in three types of generated reports:

• A “Use Case Model Survey” including all use cases for the current product, • “Use Case Specifications” for all use cases in the survey, and • “Use Case Realizations” for all use cases in the survey.

3 PLUSS TOOL SUPPORT We use Telelogic DOORS to manage our system family use case models according to the PLUSS meta-model and we use IBM-Rational Rose for drawing feature graphs and UML diagrams. We then generate appropriate reports from our domain model as MS Word documents that are publish to the development teams in our Software Configuration Management (SCM) system as shown in Figure 8.

Feature Model,Use Case Specifications,

Change Case Specificationsand

Use Case Realizations

DOORS

Feature Graphand

UML Diagrams

Rose

Repository ofPublished Reports

Software CM SystemMS Word Reports




DOORS




DOORS

Feature Graphand

UML Diagrams

Rose

Feature Graphand

UML Diagrams

Rose

Repository ofPublished Reports

Software CM SystemMS Word ReportsMS Word Reports Figure 8: Overview of CASE Tool Environment

3.1 Telelogic DOORS Telelogic DOORS is an object database intended for requirements management. DOORS could be described as a document oriented database since all objects are contained in modules that very much appear like documents to the user. Objects may be arranged in a hierarchic structure in modules. Typically, node objects are used as headings and leaf objects are used for data items such as requirements. Like in a document, both headings and text are usually displayed in a single column in DOORS. This column is referred to as the main column.

It is possible to define attributes on both module and object level in DOORS. An object attribute holds a value for each object in a module. Module attributes can be used for storing additional data not related to specific objects in modules, for example module status. An attribute definition specifies what kind of information an attributes can store. It is for example possible to define multi-valued enumerated attributes that may have any number of items set in an enumeration.

DOORS has a link concept that can be used for defining relationships between any two objects within the database. Links form the basis for traceability in DOORS. Rules can be set up for controlling what kinds of relationships are allowed, based on which modules objects belong to.

Object attributes can be displayed in columns within a module in the DOORS graphical user interface. Combining this possibility with DOORS support for filtering on properties of objects, a user may define views suiting different needs. These views are used when working with data and also for reporting and exporting information to other tools. Views may be saved in a module for later use.

3.2 Feature Modeling in DOORS Feature models are managed in specific DOORS modules. The basic idea is to use the standard DOORS object hierarchy to capture the “refine” relation used for building the feature tree structure. Each feature becomes a heading and leaf objects are used for capturing different kinds of information regarding each feature.

A number of attributes are defined in a feature model module:

• A Characteristics attribute is used for classifying each object in the module as: “General Information”, a “Feature”, a “Feature description”, a “Feature graph” or a “Use Case Diagram”. General information objects are typically used for introductory information in the domain model, such as use case actor descriptions etc.

• A Feature Type attribute is used for classifying features in the feature hierarchy. It can take the values “mandatory”, “optional”, “single adaptor” or “multiple adaptor”.

• A Require attribute and an Exclude attribute is used to capture the “requires” and “excludes” relation between features. A reference holding the identity of the required or excluded feature objects are used for this. These references are then shown as the names of the referenced features in the user interface using DOORS dynamic columns [19].

• A Use Case Package attribute is used when filtering out a use case model survey from the domain model. As mentioned in section 2.2, a sub-tree of the feature model typically corresponds to a use case package for a specific system within a system family. This attribute is used for marking such sub trees in the model.

• A Product Instances attribute is used to mark features as included or not by systems within a system family. This information is managed with a multi-valued enumeration attribute that can assume the names of all systems in the family as shown in the rightmost column of Figure 10. This information is then used for filtering out product specific use case models for the different systems in the family. Equivalent information can however also be captured as incoming links from higher level specifications to feature objects. Which method to use, depends on the requirements for traceability to higher level specifications in the specific project.

Figure 9: The DOORS Domain Model View (note that project specific info. is blurred)

3.3 Feature Graphs in Rose To be able to draw feature graphs, we utilized the UML stereotype mechanism [15] as follows:

• Icons for all PLUSS feature types were created. These icons were then associated with UML Classes that had been stereotyped to the different feature types in Rose.

• The UML Dependency relation was stereotyped as “require”. • The UML Association relation was stereotyped as “excludes”.

Furthermore, standard UML classes are used for visualizing domain names and standard UML association relations are used for visualizing the refinement relation in Rose. We draw these feature graphs in use case diagrams, and we have therefore also added the required feature modeling elements to the use case diagram toolbar in Rose as shown in Figure 9.

Figure 10: The Rose Feature Modeling View (note that project specific info. is blurred)

3.4 Use Case Specifications and Use Case Realizations in DOORS Each use case is managed as a separate module in the DOORS database. Introductory information is captured in a traditional DOORS document structure with headings and text in the DOORS main column. To capture use case scenarios and use case realizations in DOORS, we use one DOORS object to describe each scenario step. To be able to do this, object attributes for the “Step”, “Blackbox System Response”, “Blackbox Budgeted Req.”, “Whitebox Action” and “Whitebox Budgeted Req.” columns must be defined in DOORS use

case modules. For the “Actor Action” column, the DOORS main column is used.

The relationship between blackbox and whitebox scenario steps (realizations) is managed by making whitebox step objects children to their corresponding blackbox step object using the standard DOORS object hierarchy. This means that use case specifications and the use case realisation are actually different views of the same DOORS module.

3.5 Relating Features and Use Cases in DOORS We relate features and use cases in accordance with PLUSS meta-model shown in Figure 7, as follows in DOORS:

• A feature can be related to zero or more complete use cases. We manage this relation in DOORS by inserting a UML use case diagram showing those use cases as leaf objects in a feature model module under a feature object.

• A feature can be related to zero or more use case scenarios. We manage this relation in DOORS by creating an outgoing link from a heading object naming a scenario in a use case module, to a feature object in a feature model module using the DOORS link concept.

• A feature can be related to zero or more scenario steps. We manage this relation in DOORS by creating an outgoing link from a scenario step object in a use case module to a feature object in a feature model module, using the DOORS link concept.

As shown in Figure 6 we do not relate mandatory scenario steps to the feature model. The same is also true for mandatory use case scenarios. This is not needed since the primary purpose of the feature model is not to provide a total view of a system family, but rather to visualize variants in the family use case model.

3.6 DOORS Extensions for the PLUSS Approach DOORS eXtension Language (DXL) [19] is an integrated part of the DOORS product. DXL is a scripting language based on the C/C++ notation that can be used for customizing DOORS according to special needs. DXL supports all kinds of manipulation of the DOORS database, from simple reading and modifying data to advanced database administrative tasks.

DXL also support creation of user interfaces. Interactive, dynamic dialogue boxes can be created and it is possible to add new menus in the standard DOORS user interface. This provides a possibility to create integrated customized DOORS tools. DXL can also be used for controlling other Windows applications that provide automation interfaces. This possibility is utilised for example in the DOORS standard export to MS Word and MS Excel.

The following sections briefly describe some DXL tools that were developed as part of the PLUSS toolkit. In total, approximately 1.000 lines of low complexity DXL code was developed to extend DOORS to better support the PLUSS approach.

3.6.1 Check Feature Model

The purpose of the “Check Feature Model” tool is to help the user with a basic consistency check of feature model modules. The tool verifies that a number of rules regarding feature model modules are fulfilled; any violations of the rules are presented to the user in the graphical user interface. The tool verifies that the following set of conditions hold:

• Only heading objects are marked as features. • All features objects have a defined feature type.

• No “require” relations exist within a set of “single adaptor features”. • Only feature objects have “require” relations. • Only feature objects have “exclude” relations. • All “exclude” relations are bi-directional. • No contradictions exist regarding “exclude” relations and “mandatory features”. • No “exclude” relations exist between ancestors and descendant in the feature

hierarchy. • No contradictions exist regarding “exclude” relations and “require” relations.

3.6.2 Check Configuration

The purpose of the “Check Configuration” tool is to help the user verify that a specific product instance of the domain model does not contradict the rules of the feature model. Any violations of the rules are presented to the user in the graphical user interface. The tool verifies that the following set of conditions hold:

• All “mandatory features” with included parent features are included in the configuration.

• All ancestors to included features in the feature hierarchy are included in the configuration.

• The configuration does not violate “exclude” relations. • The configuration does not violate “require” relations. • The configuration does not violate the rules of “single adaptor features”. • The configuration does not violate the rules of “multiple adaptor features”.

3.6.3 Illustrate Feature Model Hierarchy

The purpose of the “Illustrate Feature Model Hierarchy” tool is to illustrate a portion of the feature model hierarchy in the DOORS graphical user interface. From a chosen feature object in a feature model module, a feature tree with all ancestors of the selected feature is drawn in a dialog box. The tool provides possibilities to collapse and expand the nodes of the tree. The tool also provides the user with an option to insert the illustration as a picture in a new DOORS object below the chosen feature object.

3.6.4 Create Use Case Module

The purpose of the “Create Use Case Module” tool is to provide the possibility to automatically create use case modules with the needed structure, attributes and views predefined. The tool accomplishes this by copying and renaming a predefined use case template module with the required properties. This tool helps to align the way different projects manage use cases in DOORS and it reduces the modeling effort needed.

3.6.5 Create Product Use Case Model Survey

The purpose of the “Create Product Use Case Model Survey” tool is to provide the possibility to filter out a use case model survey for a specific product instance from a feature model module. The tool accomplishes this by removing, from the selected view, those objects that does not fulfill each of the following conditions:

• If an object is marked as “General Information”, it must be either a descendant of an included “Feature” object or marked as included by the specified product instance itself.

• If the object is marked as a “Feature”, it must also be marked as included by the specified product instance and be marked as being a “Use Case Package”. The name of these features then becomes headings in the use case model hierarchy in the survey.

• If the object is marked as a “Use Case Diagram”, it must be a descendant of a “Feature” object marked as included by the specified product instance.

The resulting view can then be exported into a Use Case Model Survey Report for the selected product instance using the standard DOORS MS Word export.

3.6.6 Export Use Case

The purpose of the “Export Use Case” tool is to provide the possibility to export use case modules from DOORS to MS Word as use case specifications and use case realizations using the PLUSS notation.

The “Export Use Case” tool is based on the standard DOORS MS Word export which is also written in DXL. The basic idea of the tool is to export blackbox and whitebox scenarios as specially formatted tables and all other information as ordinary headings and text. To be able to recognize scenarios within use case modules, the tool assumes that each scenario step have a step identifier as prescribed by the PLUSS notation.

3.6.7 Import Use Case from Word

The purpose of the “Import Use Case from Word” tool is to provide the possibility to import use cases written in MS Word to DOORS and format these use case according to the DOORS use case template.

This tool utilizes the DOORS standard MS Word import and then performs some post processing to structure the module contents according to the company standard.

4 EMPIRICAL EVALUATION OF THE PLUSS TOOLKIT This empirical tool evaluation was designed as a blocked subject-project study [17]. The objective of the study was to evaluate how well the existing CASE tool environment, extended with the PLUSS toolkit, supported the PLUSS approach.

A number of response variables relevant for measuring the performance of the tool environment were identified during the study design. One important input to the identification of these measures was the risks related to using tools designed for single system development discussed in section 1.

4.1 Study Context The study was preformed with the Swedish defense contractor BAE Systems Hägglunds AB, which is a leading manufacturer of combat vehicles, all terrain vehicles and a supplier of various turret systems. BAE System Hägglunds AB develops software according to a tailored version of the IBM-Rational Unified Process (RUP) [13]. RUP is a use case driven software development process, widely used in industry.

The study included data collection from two different projects including subjects form both

systems- and software engineering teams. The first project regarded development of a turret system and the second project regarded development of a vehicle system. Both systems were considered to be of typical size and complexity for the organization. All subjects had previous experience and/or formal training in using the CASE tools utilized in the study.

4.2 Method The study involved collecting four different types of data:

• The first type of data was collected by examining documentation [17]. The resulting modeling artifacts were inspected by the research team to identify possible common modeling errors that could be related to poor tool support. The research team also examined inspection records for the resulting artifacts to identify possible common errors that were captured during inspections and reviews.

• The second type of data was collected by participant observation [17]. The research team participated in a number of modeling sessions to get first hand information regarding possible problems encountered by the teams using the tools. The research team also attended a brainstorming session where problems with existing tools and needed improvements were discussed by the developers.

• The third type of data was collected through questionnaires [12]. The main topics of the questionnaires were experiences working with the tools and descriptions of possible extensions to the tools created by developers. Questionnaires were designed to have both specific and open ended questions to also elicit unexpected types of information. Questionnaires were also used for eliciting information regarding subject attitudes towards the modeling notations used and information regarding the amount of experience of each subject using the tools. This information was later taken into account during data analysis.

• The final type of data was collected trough interviews [17]. Semi-structured interviews were performed with team members and main topics of these interviews were related to how the risks mentioned in section 1 applied to the way single system tools are used in the PLUSS approach.

The different types of data collected were first analyzed individually to find patterns and trends in the responses. The different types of data were then analyzed all together and conclusions were drawn.

4.3 Threats to Validity To minimize threats to the study’s construct validity, data was collected with several different methods that also allowed unexpected types of information to be elicited.

To minimize threats to the study’s internal validity, the case study project was staffed using the organizations normal staff-allocation procedures. Furthermore, subject attitudes regarding notations and their level of experience using the tools were taken into account during data analysis to minimize the effect of these possibly confounding factors [12].

To minimize threats to the study’s external validity, the study was conducted in the target domain of extremely long-lived software intense systems and projects were selected to be of typical size and complexity for the organization [12].

To minimize threats to the study’s conclusion validity, results were triangulated by collecting data with four different methods from several different sources. Furthermore, results were discussed with the teams to assure that their opinions were represented correctly [19].

4.4 Results

4.4.1 Documentation Examination

Document examination reviled that four types of modeling errors existed in the resulting documentation that could have been automatically detected. Instead these errors were now being captured during inspections, which is more costly. The following list summarizes these errors:

• Style violations regarding how use case scenarios should begin and end. • “Whitebox Budgeted Requirements” not derived from “Blackbox Budgeted

Requirements”. • Use case scenario step numbering errors. • Variant use case scenario steps not related to features in the feature model.

4.4.2 Participant observation

Participant observation reviled that shortcomings in the graphical user interfaces of both DOORS and Rose were sources of irritation for all teams involved in the evaluation. DOORS lack of shortcut keys, inability to remember previous selections in dialog boxes and unpredictable behavior regarding screen focus are examples of such shortcomings. Rose appeared unpredictable in its behavior regarding its auto-layout functions and also regarding the way diagrams were redrawn when new modeling elements were added to them. Besides from irritation, these problems also caused extra time to be spent on workarounds. One example of such a workaround was a Visual Basic application written by one subject that automatically filled out the DOORS MS Word export dialog with the correct information whenever it appeared on the screen. Another example was the need to manually move objects around in Rose diagrams to force redrawing of objects to remove spurious drawing artifacts.

Participant observation also reviled that synchronization between DOORS and Rose was a fairly time consuming and to some extent error prone activity. The synchronization consisted of mainly two tasks: drawing and updating a feature graph in Rose according to the underlying feature model maintained in DOORS, and inserting UML use case diagrams drawn in Rose as pictures in a feature model module maintained in DOORS. The DXL tool “Illustrate Feature Model Hierarchy” was meant to relieve the teams of the first of these tasks. Participant observation did however show that subjects preferred to draw feature graphs in Rose instead of using the tool. The reason was that the DXL tool did not implement any algorithm to minimize the width of the tree, but simply put all features on a specific level on same height in the graph. This resulted in very wide graphs that could not easily be printed on paper.

A problem pointed out during the tool support brainstorming meeting was that the DOORS MS Word use case export was not “good looking” and that it required manual “fixing” before complying with the company standard outline. This problem had however already been address by one of the subjects who created a VBA script in his MS Word use case export template that automatically formatted the DOORS output according to the company standard. This template was later also distributed to the other subjects. Another shortcoming of the tool environment pointed out during the brainstorming meeting was the inability to export whole set of use cases based on certain criteria. The teams had experienced a need to be able export for example all use case in the model, use cases that had been changed since the last export and a manually selected subset of a use case model. A DOORS DXL extension with this

functionality was later implemented and distributed to the subjects. One team also expressed a need for stronger means to keep track of the status of each use case specification during the brainstorming meeting. The team had also already implemented a simple DXL tool addressing this shortcoming. The tool extracted status information, for example if the use case was “ready for review” or “approved”, from the use case model and then exported this information to MS Excel. Excel then presented the information graphically using a macro defined by the team.

Participant observation also revealed that one subject had developed a method to automatically extract and insert UML sequence diagrams created in Rose in their use case realizations exported from DOORS. This feature was implemented by adding textual address tags to DOORS use case modules were they wanted diagrams to appear. These use case modules were then exported using a Word template including a VBA script which started Rose and used the Rose API to extract and replace the textual tags with the required pictures.

4.4.3 Questionnaires

Also questionnaires indicated that shortcomings in the graphical user interfaces of DOORS and Rose caused irritation among the subjects. A need for better integration between the tools were also expressed in questionnaires, subjects considered it to be much work to first draw diagrams in Rose and then manually insert them in DOORS.

Questionnaires did however also show that subjects considered the tools to have a number of positive characteristics. They considered the tools to be easy to use and very flexible, and that this allowed for good organization of the models. They also considered it to be positive that the existing CASE tool environment was utilized, since it made it easier to get started and reduced the need for training. Subjects also considered Rose to work well for drawing feature graphs.

As general summary of the questionnaires, subjects felt that the tool environment worked well when having access to the PLUSS toolkit. On the question what they would like to see in the “ideal” tool, they did however come up with the following suggested improvements:

• A WYSIWYG function that enable printing of “nice looking” specifications in accordance with the PLUSS notation directly from DOORS, without first having to export to MS Word.

• The possibility to draw UML diagrams directly in DOORS. • A stronger model checking function that verifies that rules of the model are not

violated.

4.4.4 Interviews

Interviews showed that subjects were satisfied with the overall performance of the tools. An initial resource investment had been required to adjust the tools to match the process. However at the time of the study, subjects felt that the tool environment enabled them to work effectively.

Interviews also showed that subjects considered it to be a risk that the tools were not integrated enough and that this might be a cause for artifacts to become inconsistent. Subject also felt that it might be confusing to people that models are versioned in DOORS and that the SCM system is only used for publishing reports. Subjects did however feel that these issues should not cause any major problems, as long as clear instructions on how to use the tools are

available.

Furthermore, interviews showed that subjects did not consider the PLUSS toolkit to pose a major maintenance problem since the extensions are of relatively small size and low complexity. One subject did however see a risk that the organization might be afraid of moving to newer and better releases of the tools, because of the risk for problems with the extensions. Subjects furthermore considered it to be important to have DXL knowledge in-house for future maintenance issues.

5 SUMMARY AND CONCLUSIONS We have described how commercially available CASE tools, traditionally used in single system development, can be extended and utilized for managing system family models in accordance with the PLUSS approach. The described tools were applied and evaluated in an empirical study in the target domain.

The study showed that subjects considered the tools to allow them to work effectively and that the local extensions should not pose a major maintenance problem. Problems with the graphical user interfaces of both DOORS and Rose were however sources of irritation among the subjects.

Since the study indicated that the PLUSS toolkit allowed subjects to work effectively, we will focus our future work on improving the existing environment rather than replacing it. To avoid our tool extensions becoming a major maintenance issue, we do however want to continue keeping them as small as possible. Based on the study results, we have identified the following three prioritized improvements to the toolkit:

• A Use case model checker that notifies developers of the different types of common modeling errors that were identified during document examination.

• An improved version of the Illustrate Feature Model Hierarchy tool that implements an algorithm to minimize the width of feature graphs. This would likely lead to more widespread use of the tool among the developers. This would in turn remove some of the effort needed for manual synchronization between DOORS and Rose.

• An improved version of the Create Product Use Case Model Survey tool that automatically extracts UML use case diagrams from Rose. This would remove much of the effort needed for synchronization between DOORS and Rose. Another way to address this problem would be to evaluate if the commercial product DOORS Analyst [18] would resolve this issue. The DOORS Analyst product enables drawing of UML diagrams directly in DOORS

6 ACKNOWLEDGEMENTS The authors would like to thank all the people at BAE Systems Hägglunds AB who contributed to this work.

REFERENCES [1] Adolph S., Bramble P., Cockburn A., Pols A.: Patterns for effective Use Cases,

Addison-Wesley (2003)

[2] Bass L., Clements P., Donohoe P., McGregor J. Nortrop L.: Fourth Product Line Practice Workshop Report, Technical Report CMU/SEI-2000-TR-002, Software Engineering Institute, Carnegie Mellon University, Pittsburgh, PA (2000)

[3] Ecklund E., Delcambre L., Freiling M.: Change Cases - Use Cases that Identify Future

Requirements, Proceedings of OOPSLA’96, San Jose, Ca, October 6-10, (1996) 342-358.

[4] Eriksson M., Börstler J., Borg K.: Marrying Features and Use Case for Product Line Requirements Modeling of Embedded Systems, Proceedings of the Fourth Conference on Software Engineering Research and Practice in Sweden SERPS’04, Institute of Technology, UniTryck, Linköping University, Sweden (2004) 73-82

[5] Eriksson M., Börstler J., Borg K.: The PLUSS Approach – Domain Modeling with Features, Use Cases and Use Case Realizations, To appear in the proceedings of the Ninth International Software Product Line Conference, (2005)

[6] Fey D., Fajta R., Boros A.: Feature Modeling: A Meta-model to enhance Usability and Usefulness, Proceedings of the International Conference on Software Product Lines, Lecture Notes in Computer Science, Vol. 2371, Springer-Verlag, (2002) 198-216.

[7] Gooma H. Designing Software Product Lines with UML – From Use Cases to Pattern-Based Software Architectures, Addisson-Wesley (2004)

[8] Griss M., Favaro J., d’Alessandro M.: Integrating Feature Modeling with the RSEB, Proceedings of the Fifth International Conference on Software Reuse, Vancouver, BC, Canada, (1998) 76-85.

[9] IBM-Rational Software, Available at: http://www-306.ibm.com/software/rational/ (2005)

[10] Jacobson I., Griss M., Jonsson P.: Software Reuse – Architecture, Process and Organization for Business success, Addison-Wesley (1997)

[11] Kang K. Cohen S., Hess J., Novak W., Peterson A.: Feature Oriented Domain Analysis (FODA) Feasibility Study, Technical Report CMU/SEI-90-TR-021, Software Engineering Institute, Carnegie Mellon University, Pittsburgh, PA (1990)

[12] Kitchenham B., Pickard L., Pfleeger S.: Case Studies for Method and Tool Evaluation, IEEE Software, Vol. 12 Nr. 45 (1995) 52-62

[13] Kruchten P.: The Rational Unified Process - An Introduction, Second Edition, Addison-Wesley (2000)

[14] Mannion M., Lewis O., Kaindl H., Montroni G., Wheadon J.: Representing Requirements on Generic Software in an Application Family Model, Proceedings of the International Conference on Software Reuse ICSR-6 (2000) 153-196.

[15] Object Management Group: Unified Modeling Language Version 2.0, Available at: http://www.uml.org (2005)

[16] Rational Software: The Rational Unified Process for Systems Engineering Whitepaper, Ver. 1.1, Available at: http://www.rational.com/media/whitepapers/TP165.pdf, (2003)

[17] Seaman C.: Qualitative Methods in Empirical Studies of Software Engineering, IEEE Transactions on Software Engineering, July/August (1999) 557-572

[18] Telelogic AB, Available at: http://www.telelogic.com/ (2005)

[19] Telelogic AB, DXL Reference Manual, DOORS 7.1 Manual creation date: (3 May 2004)

An Empirical Evaluation of the PLUSS Toolkit

Documents

Transcript of An Empirical Evaluation of the PLUSS Toolkit