Cultural Heritage and the Semantic Web

Benjamins V. R.1, Contreras J.1, Blázquez M.1, Dodero J. M.3, Garcia A.2, Navas E.2, Hernandez F.2∗, Wert C.2+

1Intelligent Software Components, S.A., www.isoco.com { rbenjamins, jcontreras}@isoco.com

2Residenca de Estudiantes, www.residencia.csic.es, enavas@fundacionginer.org, {cwert, fhc}@residencia.csic.es

3Universidad Carlos III de Madrid, www.uc3m.es dodero@inf.uc3m.es

Abstract. Online cultural archives represent vast amounts of interesting and useful information. During the last decades huge amounts of literature works have been scanned to provide better access to Humanities researchers and teachers. Was the problem 20 years ago one of scarceness of information (precious originals only to consult in major libraries), today’s problem is that of information overload: many databases online and many CD collections are available; each with their own search forms and attributes. This makes it cumbersome for users to find relevant information. In this paper, we describe a case study of how Semantic Web Technologies can be used to disclose cultural heritage information in a scalable way. We present an ontology of Humanities, a semi-automatic tool for annotation, and an application to exploit the annotated content. This tool, positioned somewhere in the middle, between a basic editor and a fully automatic wrapper, helps annotators performing heavy knowledge acquisition tasks in a more efficient and secure way.

1. Introduction

Online cultural archives represent vast amounts of interesting and useful information. During the last decades huge amounts of literature works have been scanned to provide better access to Humanities researchers and teachers. Most works are scanned as images, which are available through microfiches, CDs and online databases. More recently, OCR techniques are increasingly applied to provide full text search facilities. Was the problem 20 years ago one of scarceness of information (precious originals only to consult in major libraries), today's problem is that of information overload: many databases online and many CD collections are available, each with their own search forms and attributes. As is the case with general search engines, keyword based search has its limitations. One can only search for specific words and their co-occurrence in documents. This may give acceptable results for 'normal' Web users, in the area of humanities it is not sufficient because one is above all interested in

∗ Now at: Digibis: fhc@digibis.com + Now at: Patronato General de la Universidad Alcalá www.fgua.es

2 Benjamins V. R.1, Contreras J.1, Blázquez M.1, Dodero J. M.3, Garcia A.2, Navas E.2, Hernandez F.2, Wert C.2

relations e.g. between artists, their works, the friends, their studies, who they inspired, etc. In this paper, we describe a case study of how Semantic Web Technologies can be used to disclose cultural heritage information in a scalable way. We present an ontology of Humanities, a semi-automatic tool for annotation, and a publication procedure for generating a semantic cultural portal.

The basic idea is the following: • Build an acceptable ontology of Humanities by involving professionals. • Use the ontology to semantically annotate existing cultural content. • Support the annotation process by an “intelligent” editor. • Publish the results on a website, providing functionalities for semantic

navigation, intelligent search and 3D visualization. • Provide a methodology so that other content providers can publish and

exploit their content on the Semantic Web.

1.1 The Humanities domain

Research in the area of Humanities typically aims at investigating object such as events, persons, and movements, in a historical or cultural context. Applied to art, this includes, for example, paintings (e.g. the Guernika of Picasso), movements (Impressionism or Modernism), painters (Paul Klee) and authors (Cervantes). In this application domain, relations between persons, works, locations, etc. are very important because they constitute large part of the context, which gives meaning to the object of study. Therefore, keyword-based approaches to retrieve information have serious limitations, since those approaches simply ignore the meaning and the interrelations the concepts have. The other approach used in this area relies on RDBMS, which enables a more precise retrieval. However, there are many databases available, each with its own schema. More recently, also thesauri are used to improve retrieval of information by considering synonyms, hyponyms, etc. Ontologies go one step further by allowing the expression of additional relations between domain concepts. In addition, they can play the role of meta models for integration of multiple heterogeneous sources.

1.2 Ontologies

An ontology is a shared and common understanding of some domain that can be communicated across people and computers [15, 16, 3, and 17]. Ontologies can therefore be shared and reused among different applications [12]. An ontology can be defined as a formal, explicit specification of a shared conceptualisation [15, 3]. “Conceptualisation” refers to an abstract model of some phenomenon in the world by having identified the relevant concepts of that phenomenon. “Explicit” means that the type of concepts used, and the constraints on their use are explicitly defined. “Formal” refers to the fact that the ontology should be machine-readable. “Shared” reflects the notion that an ontology captures consensual knowledge, that is, it is not private to some individual, but accepted by a group. An ontology describes the subject matter

3

using the notions of concepts, instances, relations, functions, and axioms. Concepts in the ontology are organized in taxonomies through which inheritance mechanisms can be applied. It is our experience that especially the social part for building a commonly agreed ontology is not easy [2].

In the next section, we outline the ontology of Humanities, along with the methodology followed for its construction. Section 3 presents the semi-automatic annotation tool. In Section 4, we discuss different types of exploitation of the semantic content. Finally, Section 5 presents a discussion of the results and concludes the paper.

2. An Ontology of Humanities

To build the ontology, the Competency Questions Methodology [25] has been used. Competency questions are questions that the experts require the ontology-based application to provide answers for. Answers to the questions provide concrete inputs for the classes, instances and relationships to include in the ontology. In Table 1, we illustrate some of the competency questions used, categorized according to the main concepts they have elicited. Concept Competency Question Person Who wrote ‘Cráter’?

Every member of PNV Editors of the Gaceta Literaria journal Painters that participated any exposition in Barcelona in 1923

Works Which works is about Rafael Laffón? What another name is known for work X? Where did X study? Which publications came out from the congress X?

Places and Dates

Where was X brought up? Where was placed Publisher X? Which authors were born in Madrid? When was group X dissolved?

Activities Which activities did Ramón Gómez de la Serna play? What did Ramón Basterrá play in the 20’s? Who managed publisher X? What post did X occupy?

Organizations Where did Eusa, Víctor study? Who managed institution X? Who studied in institution X? Who formed group X?

Styles List of every author qualified as post-modernist Which works are considered as ultraist? Which movements does Picasso belong to? What another name is known for group X?

Events What events did Antonio Espina participate in Madrid in 1920 and

4 Benjamins V. R.1, Contreras J.1, Blázquez M.1, Dodero J. M.3, Garcia A.2, Navas E.2, Hernandez F.2, Wert C.2

Concept Competency Question 1921 years? Every congress held in Madrid Who participated in any congress held in Seville in 1920?

Relationships Which acquaintances did Antonio Espina have with Ramón Gómez? What acquaintance did Antonio Espina have with Mexico? What acquaintance did Antonio Espina have with the group SIC? Enemies of X

Table 1. Competency Questions for the Humanities Ontology

Some of the concepts included in the ontology include: Studies, Profession, Company, Institution, Academic Organization, Person, Work, Expression, Manifestation, Graphic works, Literary work, Musical work, Event, Exposition, Group, Social Group, Movement, Subject. A complete description of the ontology can be obtained from [10]. Each concept of the ontology is described through several attributes.

An important design decision we took was that relationships between concepts are modelled as first class objects. This decisions was taken because often the relationships themselves have attributes that cannot be modelled by its involving concepts. Take for example, the relation “studied_at” between a person and an organization. This relation is qualified by a start and end date, which is not meaningful to person nor location.

The ontology has been constructed with Protégé 2000 [19]. Fig 1 shows a fragment of the ontology in Protégé 2000.

5

Fig. 1. Ontology for Cultural Domain

As starting point for the ontology, we used two types of existing ontologies: 1. General ontologies to model things like persons, organizations, events, etc.

Examples of ontologies considered include: SUO [23], Generalized Upper Model [14], WordNet [27], CyCorp [6]

2. Specific standards for the humanities domain for modelling things like works art, typical relations like ‘studied_in’, ‘inspired_by’, etc. We included IFLA [13] a standard of Federation of Library Associations and Institutions, and MARC [18] a XML standard for automatically processable bibliographical information

3. The Semi-Automatic Annotation Tool

The annotation task for the Semantic Web takes as input existing content, either structured, semi-structured or unstructured, and provides as output the same content along with a semantic annotation based on ontologies. The semantics as such are defined in ontologies. The annotations provide pointers to these ontologies.

Annotation can be performed in several manners, ranging from completely manual to tool-assisted to fully automatic. As a result of the analysis performed in [5], it turns out that the type of annotation approach to be chosen depends on the rate of structure the content exhibits. More structure allows for more automation, while maintaining the quality of the annotations.

6 Benjamins V. R.1, Contreras J.1, Blázquez M.1, Dodero J. M.3, Garcia A.2, Navas E.2, Hernandez F.2, Wert C.2

As has been the experience of several researchers and practitioners, the annotation effort is a serious barrier to its widespread use [1, 25 and 26]. Although in the area of humanities manual annotation efforts are considered as necessary and are actually performed, significant improvements are possible ranging from intelligent assistants to (semi)-automatic annotators.

Together with a detailed methodology for ontology management and population we built an intelligent annotation editor. There are considered two kinds of users for the tool: • Knowledge engineer (in charge of ontology schema management): performs

major changes on the ontology, especially on the ontology schema, evaluating the final impact on the existing instances.

• Annotator (in charge of ontology population): introduces new instances in the ontology and maintain the existing ones.

The annotation tool, a kind of intelligent editor that helps the user to perform annotation tasks for a given source text is developed as a plug-in to the Protégé 2000 tool. As shown in Fig 2, the editor allows loading the source text to be annotated (right hand side). It allows standard editing operations on the ontology and the instances as provided by Protégé 2000 (left hand side). Apart from the standard Protégé functionalities, the user can easily add new instances using a drag-and-drop facility.

7

Fig. 2. Assisted Editor for Cultural Text Annotations

For instance, we can say that Picasso is an artist by selecting "Picasso" in the source text and dragging it to the ontology concept "Person" and releasing it. This creates the instance and pops up the Protégé form for creating instances. The annotation process does not change the source text itself (i.e. by putting the ontological tags in the text), rather it creates a link from the instance to the original string in the source text that caused its creation. As such the original string is an occurrence of the instance, and there can be many occurrences of the same instance throughout the text. Thus, if we drag-and-drop the identical string "Picasso" again on "person", then rather than creating a new instance, it creates a new occurrence of the same instance, unless, of course, the user decides otherwise. We can also select a string like "inspired-by" and drag-and-drop it onto the corresponding relation in the ontology (modelled as a class). Now the editor creates in instance of this relationship and pops up the corresponding Protégé 2000 from where the user is prompted among other to complete the domain and range of the relation.

Since complete manual annotation is a tedious and error-prone work, we added improvements towards automatic annotation to the editor. Automatic (cascading) creation of instances and a recommendation facility, each of them discussed in more detail below.

8 Benjamins V. R.1, Contreras J.1, Blázquez M.1, Dodero J. M.3, Garcia A.2, Navas E.2, Hernandez F.2, Wert C.2

3.1 Annotation Rules

In complex domain as described here, when performing massive knowledge acquisition tasks, often exist typical annotation patterns to be performed. For instance, each time a new artistic work is being annotated, it makes sense to also create new instances for its expression and manifestation (as defined in the IFLA standard [13]). In IFLA "work" is defined as the idea of an art work, like that of the Guernika painting of Picasso (Spanish Civil War and the Guernika bombing). The "expression" is a painting (it could as well be expressed in a poem or movie). The "manifestation" is the actual painting that can be enjoyed in the "Reina Sophia" museum in Madrid, Spain. These kind of patterns stem from dependency relations between concepts in the domain ontology.

For these typical annotation sequences we included a rule engine that allows conditional firing of a rule set to add a new instance, check for name conflicts or consult the annotator for ambiguity resolution, among others. The rule engine is based on the open source software [11] and was connected to the cultural domain ontology using an ad-hoc programmed java proxy.

3.2 Recommendations

In order to increase the accuracy and speed of the annotation process the editor includes recommendation functionalities for the annotator. When the users asks for advice for selected words or text parts the systems first checks existing instances in the domain ontology. The check is performed using a Natural Language Processing (NLP) module that decides whether two instances could be the same. This subsystem includes simple lexical and morphological modules possibly augmented with a synonym dictionary. If the selected word or a part of the selected text is identified as a possible new occurrence of an existing instance the system asks the annotator to decide which of the following action should be performed: (i) adding a completely new instance, (ii) modifying on existing instance with a new occurrence (adding new source link) or (iii) discarding any ontology modification. The more instances the ontology contains, the better recommendation the system can offer.

Other way of advising is firing guessing rules. These rules can suggest possible concepts for the selected text. For instance if a selected word starts with capital letter, it is not the first sentence word, and it was not recognized in the morphological module, it will be designated as a proper name and the concepts “NAME” (person’s or place’s) will be suggested to the annotator.

3.3 Conflict Resolution

One of the most complex concepts in this cultural ontology in the NAME concept. Since almost all things can be named in different ways and the exploitation will stress access to the information using this attribute we took special care modelling it. Each author, place, work, etc. can possess a number of names, variable in the time line or different depending on the relation they participate in. For instance, an author can

9

write a book using one pseudonym, then use his legal name when attending an exposition and use an acronym when writing a new book with two colleges. All these names should point to the same person instance.

That is why the system offers instance duplication detection that warns the annotator of a possible existing instances using the name relations between concepts.

3.4 Search Facilities

We tested our tool by several users. It turned out that search facilities are extremely important in order for the annotator to keep track of what has already been annotated, as well as allowing him to follow different annotation strategies. Two example strategies include: i) following the text from begin to end, and ii) topic (author, work, movement) based. Especially in the latter strategy, search facilities are extremely important. The editor allows the following types of search: • Marking instance from ontology: Instances already annotated have link to text

marking their occurrences. It is very useful for the annotator to check what text was already processed.

• Search instance in ontology: This functionality is often used as part of the recommendations. The user can search for ontology instances that contain part of the text in their source.

All search functionalities take into account that almost all instances are referenced with their ‘name’ attribute. That is why when searching for occurrences the system identifies instances not only by source but also by their ‘name’ value related.

3.5 Import Facilities

Since there exists a bibliographic standard for authors and works storage defined upon the XML language called MARC [18], we built an import tool for translating it into the ontology formalism. The tool uses the rule engine and parsers to process XML input file and fill the ontology with acquired instances for persons, and artistic works. As well as the drag and drop functionalities this one also includes conflict detection for avoiding data duplication. Whenever the import tool detects possible data repetition, it postpones the decision to the end of the process and then asks the user to resolve possible actions to be taken: (i) adding new instance, (ii) adding new occurrence of an existing instance or (iii) skipping any action.

10 Benjamins V. R.1, Contreras J.1, Blázquez M.1, Dodero J. M.3, Garcia A.2, Navas E.2, Hernandez F.2, Wert C.2

4. Exploitation of the Semantic Content

4.1 A Semantic Portal for Cultural Heritage

A small web-based prototype for the cultural ontology built earlier showed us that the knowledge base as modelled by domain experts and knowledge engineers is not always a good candidate to visualize it as is. Since many relations in this domain were modelled as explicit concepts, navigation became tedious and unfriendly.

Fig. 3. Explicit visualization ontology for web publication

The main purpose for building ontologies is to provide semantic content for intelligent systems. The knowledge models are designed to offer the appropriate information to be exploited by the software. No visualization criteria are used to build an ontology and often the information is not suitable to be published as it is: • Concepts may have too many attributes • When relations are represented as independent concepts (first class objects) the

navigation becomes tedious • Concepts to be shown do not always correspond to modelled ones. Therefore we felt a need for explicit visualization rules that allow the creation of views on the cultural ontology, in order to visualize only the relevant information in a user friendly way. We introduced the concept of “visualization ontology”, which makes explicit all visualization rules and allows an easy interface management. This ontology will contain concepts and instances (publication entities) as seen on the interface by the end user, and it will retrieve the attribute values from the cultural ontology using a query. It does not duplicate the content of the original ontology, but links the content to publication entities using an ontology query language. This way one ontology that represents a particular domain can be visualized through different views.

The visualization ontology has two predefined concepts:

Ontology view

RD

QL

Architecture

Com

mand P

atternInternal format

(XML)

Architecture

Com

mand P

attern

Person

Works at

Partner

V. RichardBenjamins iSOCO

Works at

Knowledge base(RDF/RDFS)

EmployeeEnglish

Richard atiSOCO

Publication model(RDF/RDFS) Final 3D (X3D)

RDQLJava

Business LogicXSL

Transformations

11

• Publication entity: Concept that encapsulates objects as they will be published in the portal. Any concept defined in the visualization ontology will inherit from it and should define these attributes • XSL style-sheet associated to the concept that translates its instances to final

format (HTML, WAP, VoiceXML, etc.) • Query that retrieves all attribute values from the original ontology.

• Publication Slot: Each attribute that is going to appear on the web should inherit from this concept. Different facets describe how the attribute will appear on the page. • Web label: The label that will appear with the value • RDQL: reference to the query used to retrieve the attribute value • Link: When the published value should perform some action on mouse click

(link, email, button, etc...), the action is described here. Portal elements are described as children of the Publication Entity and their instances are defined according to the languages the entity will be published in (labels in English, Spanish, etc.), or the channel (whether the transformation style-sheet is going to translate into HTML, WAP, or just XML). In this case, the news library is exported in HTML format.

Back-office management is divided into two tasks: • Content management on domain ontology: adding new instances or modifying the

overall schema. • Visualization management on publication ontology: modifying how information is

shown (look and feel, layout, etc..) Both tasks are performed using Protégé 2000 editor, since both domain and publication models are defined in RDF language.

4.2 3D Visualization

We have also developed a 3D generation module that allows navigating through the ontology content. For that purposes we have implemented software that translates given domain ontology, applying user defined visualization rules into the X3D [29] standard.

The resulting scene shows instances in a three-dimensional net represented as geometrical bodies with an ad-hoc defined texture. The scene is highly interactive allowing user to move the focus position and interact with the object (clicking on them).

12 Benjamins V. R.1, Contreras J.1, Blázquez M.1, Dodero J. M.3, Garcia A.2, Navas E.2, Hernandez F.2, Wert C.2

Fig. 4. 3D navigation on domain ontology

5. Discussion and Conclusions

In this paper we presented an approach and tool of for creating a cultural "Semantic Web Island", and we showed examples of how this can be exploited for the benefit of art lovers. One should notice that the mentioned technology is generic and can be applied to any other domain with small modifications, provided there is domain ontology available. An exception to this statement should be considered: cultural domain has some strong requirements on instance names (due to pseudonyms, different typing or languages). Special attention to these particularities has to be paid when modelling humanities or cultural domains.

There are basically two types of tools for annotation (which actually are two extremes of a continuous line): • Editors to manually annotate content with terms defined in ontologies • Wrappers to automatically annotate content with ontological terms

When an editor not only allows for mark-up of content, but also provides support

(e.g. suggestions of ontology terms for particular content), then it moves up the continuum to the wrapper end. The editor we described in this paper is somewhere in the middle of this continuum.

Our next step is to improve the NLP part of the tool, including shallow parsing, to come closer to automatic ontology population. On the exploitation side, we are

13

working on extending the 3D visualization part to automatically generate thema-oriented online "cultural tours".

6. Acknowledgements

Part of this work has been funded by the European Commission in the context of the project Esperonto Services IST-2001-34373 and SWWS IST-2001-37134 and by the Spanish government in the scope of the project: ONTO-H (PROFIT, TIC): Intelligent access to digital cultural content based on an ontology for Humanities.

7. References

[1] V. R. Benjamins and D. Fensel. Editorial: Problem-solving methods. International Journal of Human-Computer Studies, 49(4):305–313, October 1998. Special issue on Problem-Solving Methods.

[2] V. R. Benjamins, D. Fensel, S. Decker, and A. Gomez-Perez. (KA)2: Building ontologies for the internet: a mid term report. International Journal of Human-Computer Studies, 51(3):687–712, 1999.

[3] W. N. Borst. Construction of Engineering Ontologies. PhD thesis, University of Twente, Enschede, 1997.

[4] CBL. Common business library, 1999. http://www.commerceone.com/solutions/default.html.

[5] Contreras et al. D31: Annotation Tools and Services, Esperonto Project: www.esperonto.net

[6] CyCorp: http://www.cyc.com/ [7] DAML. http://www.daml.org [8] S. Decker and F. Maurer. Editorial: organizational memory and knowledge management.

International Journal of Human-Computer Studies, 51(3):511–516, 1999. Special Issue on Organizational Memory and Knowledge Management.

[9] Stefan Decker, Michael Erdmann, Dieter Fensel, and Rudi Studer: Ontobroker: Ontology Based Access to Distributed and Semi-Structured Information. In R. Meersman et al. (eds.): Semantic Issues in Multimedia Systems. Proceedings of DS-8. Kluwer Academic Publisher, Boston, 1999, 351-369.

[10] Juan Manuel Dodero, Jesús Contreras, Richard Benjamins, Test Case Ontology Specification Cultural Tour. D9.2, Esperonto Project, www.esperonto.net.

[11] Drools http://drools.org/ [12] A. Farquhar, R. Fikes, and J. Rice. The ontolingua server: a tool for collaborative ontology

construction. International Journal of Human-Computer Studies, 46(6):707–728, June 1997. [13] Federation of Library Associations and Institutions: http://www.ifla.org [14] Generalized Upper Model: http://www.darmstadt.gmd.de/publish/komet/gen-

um/newUM.html [15] T. R. Gruber. A translation approach to portable ontology specifications. Knowledge

Acquisition, 5:199–220, 1993. [16] N. Guarino. Formal ontology, conceptual analysis and knowledge representation.

International Journal of Human-Computer Studies, 43(5/6):625–640, 1995. Special issue on The Role of Formal Ontology in the Information Technology.

14 Benjamins V. R.1, Contreras J.1, Blázquez M.1, Dodero J. M.3, Garcia A.2, Navas E.2, Hernandez F.2, Wert C.2

[17] G. van Heijst, A. T. Schreiber, and B. J. Wielinga. Using explicit ontologies in KBS development. International Journal of Human-Computer Studies, 46(2/3):183–292, 1997.

[18] MARC http://www.loc.gov/marc/ [19] Protete 2000 tool: http://protege.stanford.edu [20] RDF. Resource description framework, 1998. http://www.w3.org/TR/WD-rdf-syntax. [21] RDFS. Resource description framework schema, 1998. http://www.w3.org/TR/1998/WD-

rdf-schema-19980814. [22] A. Th. Schreiber, J. M. Akkermans, A. A. Anjewierden, R. de Hoog, N. R. Shadbolt, W.

Van de Velde, and B. J. Wielinga. Knowledge Engineering and Management, The CommonKADS methodology. MIT Press, 2000.

[23] SUO Standard Upper Ontology http://suo.ieee.org/ [24] W. Swartout and A. Tate. Coming to terms with ontologies. IEEE Intelligent Systems and

Their Applications, 14(1):19–19, January/February 1999. [25] M. Uschold and M. Gruninger. Ontologies: principles, methods, and applications.

Knowledge Engineering Review, 11(2):93–155, 1996. [26] D. A. Waterman F. Hayes-Roth, D. B. Lenat. Building Expert Systems. Addison Wesley,

1983. [27] WordNet: http://www.cogsci.princeton.edu/~wn/ [28] XML. Extensible markup language, 1998. http://www.w3.org/TR/PR-xml-971208. [29] X3D http://www.web3d.org/x3d.html