DWARM: an ontology of data warehousearchitecture reference model ?

Piotr Szwed1 and Wojciech Komnata1 and Dariusz Dymek1

AGH University of Science and Technology{wkomnata, pszwed}@agh.edu.pl eidymek@kinga.cyf-kr.edu.pl,

Abstract. This paper describes DWARM, an ontology formalizing anew data warehouse architecture reference model intended do capturecommon five architectural approaches, as well as to provide means fordescribing complex hybrid architectures that emerge due to observedevolution of business and technology. The ontology defines concepts, e.g.layers, processes, containers and property classes, as well as relations thatcan be used to construct precise architectural models. Formalization ofarchitecture description as an ontology gives an opportunity to performautomatic or semiautomatic validation and assessment.

Keywords: Data warehousing, data warehouse architecture, ontology,architecture evaluation

1 Introduction

Mature data warehousing technologies and solutions have been present on themarket for over 20 years. They are developed within companies to support var-ious analytical functions, reporting and provide accurate data for decision sys-tems. A variety of applications, business needs, technical reasons or requirementsrelated to performance, scalability, modifiability causes that the problem of se-lection of data warehouse architecture attracts many discussions and debates.

The literature reports five dominant architectural approaches: IndependentData Marts, Data Mart Bus architecture, Hub-and-spoke, Centralized and Fed-erated [1, 2]. Empirical studies of the field, e.g. [3] show that the most popularare Centralized and Hub-and-spoke recommended by Bill Inmon [4] and DataMart Bus proposed by Ralph Kimball [5].

In many practical situations a particular architecture of a data warehouseis developed as a result of evolutionary process stimulated by business needsemerging during the system lifetime. This often leads to hybrid solutions orpromotes federated architectures. In particular, the latter can be applied in suchsituations as company mergers or acquisitions, when it is usually more efficient tointegrate legacy systems with an additional middleware layer providing schemamapping and reconciliation [6], than to build a completely new infrastructure.

? This is a draft version of the paper accepted at the Beyond Databases, Archi-tectures and Structures - 11th International Conference, BDAS 2015,Ustron, Poland, May 26-29, 2015.

2

In [7, 8] we have proposed DWARM, a data warehosue architecture referencemodel that was intended to unify known architectural styles. The model dis-tinguishes eleven layers comprising processes loading and transforming data andcontainers (data stores, source systems and consumer applications). Its layers arethe following: (1) Source, (2) Extraction, (3) Temporary Staging Area (TSA), (4)Loading, (5) Central Data Repository, (6) Data Mart Feeding, (7) Data Mart,(8) Data Integration, (9) Federated Data Repository, (10) Data Delivery, and(11) Consumer Applications.

The reference model can be used to express known architectures by tailoring(removing whole layers or individual components) and multiplying instancesof selected layers. Also hybrid architectures combining basic approaches canbe expressed with DWARM, provided they respect some consistency rules. Anumber of examples can be found in [7].

The language used to specify the reference model follows the concepts ofStructured Analysis [9]. It distinguishes processes, data stores, etc. It is expres-sive enough to give an overall view of components structure. However, we foundthat for more complex tasks, as automatic architecture validation or assessment,a more formal and rigorous description is required. This was the motivation fordeveloping the DWARM ontology, which provides a taxonomy of processes andcontainers, assigns them to appropriate layers, defines feasible connections be-tween components, as well as additional properties that can be used for thearchitecture evaluation purposes.

The paper is organized as follows: Section 2 discusses related works, it isfollowed by Section 3 specifying ontology goals. Next Section 4 presents theontology content. A few partial examples of models are given in Section 5, finallySection 6 provides concluding remarks.

2 Related works

An application of ontologies to provide a systematic and formal description ofsoftware architectures was first proposed by Kruchten in [10]. The ontology dis-tinguished several types of decisions that can be applied to software architectureand its development process. Main categories included: Existence, Ban, Propertyand Executive decisions. The ontology defined also attributes, which were usedto describe decisions, including states (Idea, Tentative, Decided, Rejected, etc.).In [11] an ontology supporting Architecture Tradeoff Analysis Method (ATAM)based evaluation was proposed. The ontology specified concepts covering theATAM [12] model of architecture, quality attributes, architectural styles and de-cisions, as well as influence relations between elements of architectural style andquality attributes. The effort to structure the knowledge about architectural de-cisions, was accompanied by works aimed at a development of tools enabling theedition and graphical visualization of design decisions, often in a collaborativemode, e.g. [13].

Such tasks as software architecture assessment face the problem of gather-ing information related both to overall design, and the detailed architectural

3

decisions made [14]. An ontology supporting ATAM based assessment of sys-tems following SOA paradigm was proposed in [15, 16]. Its main goal was todefine types of components and their attributes, with an intent of using themwhile representing architecture instances as ontological models. The obtainedrich descriptions that can be then used for reasoning about quality attributes.An example of security risk evaluation, that reuses parts of formal architecturedescription can be found in [17].

The problem of data warehouse assessment was widely discussed [1, 2] andeven several guidelines of dedicated questionnaires can be found [18]. Surpris-ingly, up to our best knowledge there are no references to direct application ofontologies to describe architectural models and design decisions. A number ofpapers related to application of ontologies in a data warehouse design processcan be found, e.g. [19, 20]. They are focused, however, on important, yet quitedifferent problem of building data models based on ontologies and semantic in-tegration within data warehouses.

3 Ontology goals

The main goal of the DWARM (Data Warehouse Architecture Reference Model)ontology is to provide a formal description of data warehouse architectures com-prising concepts and relations used to define them, as well as constraints thatshould be satisfied by the components. It was also assumed that a concrete ar-chitecture instance could be defined using the ontology elements. This allows toapply common Semantic Web tools and methods, e.g. reasoners, to evaluate con-sistency of models and compliance with restrictions and data warehouse designguidelines.

Ontologies are often described as unions of two layers: terminological (TBox )and assertional (ABox ). The TBox defines concepts and types of relation includ-ing: taxonomic relations between concepts, object properties and datatype prop-erties. The ABox, in turn, gathers facts about individuals and existent relations.

While designing the DWARM ontology we assumed that the reference modelof data warehouse will be reflected as the TBox layer. It comprises two groups ofentities: core components, which correspond to elements defined in architecturalviews and classes defining component properties. Their individuals, which can betreated as constants, are also defined in the ontology. A concrete architectureinstance, including processes, data stores, source systems and analytic tools,together with assigned properties, is to be defined as an ABox layer (c.f. Fig 1).

CoreComponents

ComponentProperties

Architecture Instance (Abox)

DWARM OntologyValidation

Evaluation

Fig. 1. A concept of DWARM ontology application

4

An intended application of DWARM ontology, apart from providing precisedescription language, is to use it within a software tool allowing to document, val-idate and assess data warehouse architecture. The ontology was populated withclasses, relations and individuals identified based on literature review summa-rized in [7]. The ontology is formalized in OWL language. At present, it definesabout 100 classes and 40 relations.

4 Ontology content

The ontology is built around a foundational model that specifies core classes andrelations. It can be treated as a metamodel of the language used to describe thereference architecture. The class diagram in Fig. 2 depicts an ontology skele-ton that was further extended by subclassing main classes: Containers, Process,Data, etc. They are described in the rest of this section.

Container

Process

hasIn

put

hasO

utput

hasOutput

DataLayer

hasContainer

hasProcess

Metadata

ArchitectureModel

uses

Agent

owns&accesses

InfrastructureComponent

-uses-uses

data

-hasInput

Fig. 2. Top-level ontology classes (CoreCompnents)

4.1 Containers

Class Container is an abstraction of architecture component capable of storing,sending or receiving data. The ontology distinguishes 6 classes of containers:SourceSystem, TemporaryStagingArea: temporary data for ETL processes, Cen-tralDataRepository, DataMart, FederatedDataRepository and ConsumerApplica-tion. Containers can be assigned with such properties as stored data, subject,type (RelationalDataBase, NOSQLDataBase, PlainFile, RecordOrientedStorage,XML, VirtualContainer), database engine, interface type (SQL, WebService,Propriety), implementation technology and metadata.

5

4.2 Processes

Process represents an activity consisting in data transformation and transferbetween containers. The DWARM ontology defines 15 classes of process. Basicrelations that can be used to connect them with containers are: hasInput andhasOutput. Restrictions for valid architecture models are similar to those, definedfor Structured Analysis [9]: processes without inputs (data generators) or outputs(black holes) are not allowed.

Processes can be described using additional properties defined in the ontol-ogy: metadata, language, execution engine, trigger and data read policy. For thelater, appropriate dictionaries (classes and individuals) are defined: ProcessTrig-ger (Time, DataArrival, Manual, Process) and DataReadPolicy (Incremental,FullRefresh). Information on defined processes is summarized in Table 1.

Table 1. Processes

Process Description

Extraction A component of L02-ExtractionLayer. Reads data from a source system andstores in the TemporaryStagingArea container.

Transformation Belongs to the layer L03-TemporaryStagingAreaLayer. Responsible for datatransformation within temporary staging area.

LoadingFromTSA Part of L04-LoadingLayer. Used to model data loading. In an architecturemodel its subclasses should be used: LoadingFromTSAToCDR (loading to acentral repository) or LoadingFromTSAToDM (loading to a data mart).

ETL. The process does not belong to any of layers. It was introduced to preserve theterminological convention. The process is a composition of extraction, trans-formation and loading processes.

CDRTransformation. Data transformation within a central repository, e.g. calculation of aggregatedvalues. Constitutes an element of the layer L05-CentralDataRepositoryLayer.

DMFeeding. Part of L06-DataMartFeedingLayer. The process feeds data marts from a cen-tral repository.

DMTransformation. Data transformation for data marts, e.g. a process calculating totals for variousdata dimensions. Belongs to L07-DataMartLayer.

Integration. Process belonging to the layer L08-DataIntegrationLayer. It is allowed to readdata either from a central repository or a data mart, then it delivers it to acontainer FederatedDataRepository.

FDRTransformation A process of data transformation undergoing in federated data repository (thelayer L09-FederatedDataRepositoryLayer).

DataDelivery. An abstract superclass of three classes of processes intended to be used: DM-DataDelivery, CDRDataDelivery and FDRDataDelivery. They provide datafor applications of the top layer reading them from correspondingly: datamarts, a central repository of a federated repository. All processes belong toL10-DataDeliveryLayer.

4.3 Layers

Following the data warehouse reference model [7], the ontology defines 11 classesof layers. To increase readability their numbers were included into names. Layersare abstract elements of the model, not having tangible counterpart in an imple-mented software. They constitute an integrating and ordering element. An indi-vidual representing an architecture model is linked with layer instances. They, inturn, are linked by containsContainer and containsProcess with container andprocess instances (Fig. 3).

6

Fig. 3. Layers and their definitions

The ontology content can be browsed staring from layer classes using a soft-ware for ontology modeling and visualization, e.g. Protege tool. Fig. 3 shows atypical structure of links within the DWARM ontology. The layer L04-LoadingLayercontains the abstract process LoadingFromTSA. One of its subclasses Load-ingFromTSAToCDR describes a concrete process, whose individual may appearin a concrete data warehouse architecture instance. The figure shows proper-ties defined at the class level (output to CentralDataRepository) and propertiesinherited from superclasses.

4.4 Data

The class Data is intended to model data types, which can be assigned to con-tainers. Data cane be characterized by such properties, as: model (relational, di-mensional, object-oriented), owner, materialization, type (primary or secondary)and representation (record, file, XML, etc.)

The ontology defines three object properties for the Data class. The relationsubclass corresponds to inheritance, whereas references to association. The thirdintroduced relation: originates is intended to model dependencies between sourcedata and data resulting from their transformation (e.g. filtering, joining). Hence,it allows to trace, from where the data originate, as well as to check, whetherthe process topology corresponds to data dependencies.

4.5 Metadata

The ontology specifies 25 classes of metadata arranging them into taxonomiescorresponding to three commonly used classifications [21]: business and techni-

7

cal, front-end and back-end, as well as functional: InfrastructureMetadata, Mod-elMetadata, ProcessMetadata, QualityMetadata, ProcessMetadata and Admin-istrationMetadata. To support classification the ontology defines three classesof metadata properties: MetadataUsage, MetadataApplicationArea, Metadata-Function. They come along with predefined individuals that can be used asconstants in owl:hasValue restrictions, e.g. QualityMetadata = Metadata u(hasFunction DataAssessmentAndMetrics).

4.6 Component properties

Component properties are, beside core components, the second group of classesin DWARM ontology. Their taxonomy corresponds to basic types of compo-nents, hence, the top level properties are: ContainerProperty, ProcessProperty,DataProperty, MetadataProperty and AgentProperty.

In many cases property classes have predefined individuals that enumeratepossible architectural decisions. A designer documenting the system architecturefor a particular component may select one or a few dictionary values, e.g. for aData object they can choose from the following DataRepresentation individuals:{File, Record, Structured, Unstructured, XML}.

The classes of properties are treated as a part of ontology that is intendedfor future extensions. Documentation of architecture of a particular system mayrequire using new classes of design decisions, which are difficult to establish at thestage of ontology formalization. Based on an experience related to assessmentof software architectures with ATAM method [14] it can be concluded that acertain freedom in describing design decisions and a capability to introduce newtraits of components, e.g. referencing particular technologies, may be a key pointwhile collecting information required to perform architecture evaluation.

5 Examples of models

Fig. 4 depicts an excerpt from an ontology describing an example architectureof a data warehouse for a fictitious insurance company. The company activitiesconcentrate within two areas. The first is selling insurances in branches and bymobile agents. The second is claim handling with a central system allowing todeposit notifications by clients, as well as agents.

The diagram was prepared following the convention of OntoGraph plugin forProtege ontology editor. Elements symbolizing classes are marked with circles,whereas individuals (actual architecture elements) with diamonds. Dashed linearrows are used to show has individual relations (target elements are individualsbelonging to the classes placed at arrows starts), whereas continuous line is usedfor hasInput and hasOutput object properties. It should be noted, that arrowsreflect directions of relations and not data flows.

The class SourceSystem has three individuals: Branches, Claims and Mo-bileAgents. The system comprises a single instance of temporary data container

8

TSA, belonging to the class TemporaryStagingArea. It is fed by separate pro-cesses of Extraction type: BranchesExtr, ClaimsExtr and MobiAgExtr. For greaterclarity, data transformation processes were omitted. The next architecture ele-ment is Loading, an instance of loading process belonging to the class Load-ingFromTSAToCDR. It is linked by hasInput relation with TSA and by hasOut-put with the central data repository CDR. The system architecture includes twodata marts: SalesDM and ClaimsDM. They are fed by two processes of DMFeed-ing type: correspondingly SalesFeed and ClaimsFeed. The consumer applicationlayer comprises three elements: SalesReporting, ClaimsReporting and NewProd-uctsBenchmarking. It is fed by processes belonging to classes DMDataDeliveryor CDRDataDelivery. Processes SalesOLAP and ClaimsOLAP are instances ofDMDataDelivery. Their inputs are data marts SalesDM and ClaimsDM, whereastheir outputs are analytic applications for sales and claim handling. The toolNewProductsBenchmarking requires aggregated information related both to salesand claims, hence it is fed directly from the central repository CDR by the pro-cess CrossChk (member of CDRDataDelivery).

MobileAgentsClaimsBranches

BranchesExtr ClaimsExtr MobiAgExtr

TSA

Loading

CDR

SalesFeed ClaimsFeed

SalesDM ClaimsDM

SalesOLAP ClaimsOLAP

CrossChk

ConsumerApplication

CDRDataDelivery

DMDataDelivery

DataMart

DMFeeding

CentralDataRepository

LoadingFromTSAToCDR

TemporaryStagingArea

Extraction

SourceSystem

SalesReporting ClaimsReportingNewProducts Benchmarking

Fig. 4. An excerpt of data warehouse architecture model in a form of ontology

Fig. 5 shows another architectural view comprising containers and the data,which they store. The relation originates is used to capture data dependencies,

9

e.g. data related to AgentActivity stored in the CDR has its origins in Agent-ClaimHandling and AgentPolicySales stored in the MobileAgents source system.

MobileAgents

Claims

Branches

CDRClaimData

AgentActivity

PolicyData

Data

data

data

data data

data

data

originates

originates

originates

originates

SourceSystem CentralDataRepository

BranchesPolicySales

BramchesClaimData

ClaimSourceData

AgentClaimHandling

AgentPolicySales

Fig. 5. A partial model of containers and data

Formal description as an ontology of both data warehouse architecture andthe reference model gives an opportunity to validate their compliance. The ontol-ogy defines a number of axioms (total 403) defining constraints using universal,existential or cardinality restrictions, eg. process CDRDataDelivery hasInput ex-actly 1 CentralDataRepository and hasOutput only ConsumerApplication. Thevalidation can be achieved with dedicated software tools implementing proce-dures for consistency checking, or general purpose ontology reasoners, e.g. Pelletor FaCT++. In consequence, a data warehouse designer can be warned aboutpotential flaws in architecture design.

Fig. 6a shows an architecture model, in which the Delivery process readsdata both from the central repository CDR and the data mart DM. Such con-figuration, although possible to implement, is not recognized as valid in theDWARM reference architecture. Implementation of a process reading data fromtwo dependent sources may potentially cause problems with data freshness, dataaccess privileges and data model interpretation. According to formally definedconstraints reflecting good practices in data warehouse design, a delivery processshould read data from a single container. This is reflected by restrictions on pro-cess classes: DMDataDelivery, CDRDataDelivery and FDRDataDelivery, whichare to be linked by hasInput relation only with containers of type DataMart,CentralDataRepository and FederatedDataRepository respectively.

The reference architecture model allows for a situation, when it is necessaryto deliver data from various repositories. This can be achieved with processesbelonging to a data integration layer accompanied by a federated data repository,either in materialized or not materialized form. Fig. 6b presents a refactoredarchitecture complying the reference model. Components marked in Fig. 6 withgray color were replaced by the following elements: IntegrProc (an integrationprocess), FDR (federated data repository) and DeliveryFed (process deliveringdata from FDR).

10

CDR

DMFeed

DM

Delivery

FrontEndApp

hasInput

hasInput

hasOutput

ConsumerApplication

DataDelivery

DataMart

DMFeeding

CentralDataRepository CDR

DMFeed

DM

DeliveryFed

FrontEndApp

hasInput

hasOutput

FDR

IntergrProc

hasOutput

hasInput

hasInput

ConsumerApplication

FDRDataDelivery

FederatedDataRepository

Integration

DataMart

DMFeeding

CentralDataRepository

a b

Fig. 6. Examples of models: (a) not compliant with the reference architecture, (b)refactored to the compliant form

6 Conclusions

The described here DWARM ontology constitute a formal specification of layeredmodel of data warehouse architecture including basic components: processes,containers, data and metadata. The formal character of the OWL language usedto encode the ontology, allows to define constraints, that should be respectedwithin the architectures complying the reference model. This regards both oc-currences of elements of particular types, as well as their connections. An ad-vantage of using ontology based architecture models is the possibility to processand analyze them automatically. The ontology was designed to allow future ex-tensions. It is possible to define new component properties and enrich the modelwit additional information, e.g. related to performance (number of records, datasize, frequency of process executions).

References

1. Ariyachandra, T., Watson, H.J.: Which data warehouse architecture is most suc-cessful? Business Intelligence Journal 11(1) (2006) 4

2. Ariyachandra, T., Watson, H.: Key organizational factors in data warehouse ar-chitecture selection. Decision Support Systems 49(2) (2010) 200–212

3. Alsqour, M., Matouk, K., Owoc, M.L.: A survey of data warehouse architectures- preliminary results. In: Computer Science and Information Systems (FedCSIS),2012 Federated Conference on, IEEE (2012) 1121–1126

4. Inmon, W.: Building the Data Warehouse. Wiley (2005)5. Kimball, R., Ross, M.: The Data Warehouse Toolkit: The Definitive Guide to

Dimensional Modeling. Wiley (2013)6. Chromiak, M., Stencel, K.: A data model for heterogeneous data integration ar-

chitecture. In Kozielski, S., Mrozek, D., Kasprowski, P., Malysiak-Mrozek, B.,

11

Kostrzewa, D., eds.: Beyond Databases, Architectures, and Structures - 10th Inter-national Conference, BDAS 2014, Ustron, Poland, May 27-30, 2014. Proceedings.Volume 424 of Communications in Computer and Information Science., Springer(2014) 547–556

7. Dymek, D., Komnata, W., Kotulski, L., Szwed, P.: Data Warehouse Architectures.Reference model and formal architecture description. In polish: Architektury Hur-towni Danych. Model referencyjny i formalny opis architektury. AGH Universityof Science and Technology Press (2015)

8. Dymek, D., Komnata, W., Szwed, P.: Proposal of a new data warehouse archi-tecture reference model. In Kozielski, S., , Malysiak-Mrozek, B., Kasprowski, P.,Mrozek, D., Kostrzewa, D., eds.: Beyond Databases, Architectures, and Structures- BDAS 2015. Communications in Computer and Information Science. SpringerInternational Publishing (2015) submitted to BDAS 2015 conference

9. Yourdon, E.: Modern Structured Analysis. 2nd edn. Prentice Hall PTR, UpperSaddle River, NJ, USA (2000)

10. Kruchten, P. In: An ontology of architectural design decisions in software intensivesystems. Citeseer (2004) 54–61

11. Erfanian, A., Aliee, F.S.: An ontology-driven software architecture evaluationmethod. In: Proceedings of the 3rd international workshop on Sharing and reusingarchitectural knowledge. SHARK ’08, New York, NY, USA, ACM (2008) 79–86

12. Kazman, R., Klein, M., Clements, P.: ATAM: Method for architecture evaluation.CMUSEI 4(August) (2000) 83

13. Lee, L., Kruchten, P. In: Visualizing Software Architectural Design Decisions.Volume 5292. Springer-Verlag (2008) 359–362

14. Szwed, P., Wojnicki, I., Ernst, S., G lowacz, A.: Application of new ATAM toolsto evaluation of the dynamic map architecture. In Dziech, A., Czyzewski, A., eds.:Multimedia Communications, Services and Security. Volume 368 of Communica-tions in Computer and Information Science. Springer Berlin Heidelberg (2013)248–261

15. Szwed, P., Skrzynski, P., Rogus, G., Werewka, J.: Ontology of architectural deci-sions supporting ATAM based assessment of SOA architectures. In Ganzha, M.,Maciaszek, L.A., Paprzycki, M., eds.: Proceedings of the 2013 Federated Confer-ence on Computer Science and Information Systems, Krakow, Poland, September8-11, 2013. (2013) 287–290

16. Szwed, P., Skrzynski, P., Rogus, G., Werewka, J.: SOAROAD: an ontology ofarchitectural decisions supporting assessment of service oriented architectures. In-formatica (Slovenia) 38(1) (2014) 31–42

17. Szwed, P., Skrzynski, P.: A new lightweight method for security risk assessmentbased on Fuzzy Cognitive Maps. Applied Mathematics and Computer Science24(1) (2014) 213–225

18. Habers, F.: The DW maturity assessment questionnaire. Technical Report UU-CS-2010-021, Institute of Information and Computing Sciences, Utrecht University,3508 TC, Utrecht, The Netherlands. (September 2010)

19. Nebot, V., Berlanga, R., Perez, J., Aramburu, M., Pedersen, T.: Multidimensionalintegrated ontologies: A framework for designing semantic data warehouses. InSpaccapietra, S., Zimanyi, E., Song, I.Y., eds.: Journal on Data Semantics XIII.Volume 5530 of Lecture Notes in Computer Science. Springer Berlin Heidelberg(2009) 1–36

20. Khouri, S., Ladjel, B.: A methodology and tool for conceptual designing a datawarehouse from ontology-based sources. In: Proceedings of the ACM 13th inter-national workshop on Data warehousing and OLAP, ACM (2010) 19–24

12

21. Shankaranarayanan, G., Even, A.: The metadata enigma. Communications of theACM 49(2) (February 2006) 88–94