Supporting service composition with ontology-based UPnP AV architecture in AV environment

Abstract-The current trend in digital home technologies is

slowly moving towards creating entertainment experience where

users can enjoy any multimedia contents conveniently with

minimal manual configuration. Automated service composition

is useful in enhancing the entertainment experience through

coordination among existing services to fulfill user request.

UPnP AV architecture is well adopted for building

consumer-oriented AV products with simple installation and

configuration setting. However, not all services specified in

UPnP AV architecture is useful for automating service

composition. Thus, in this paper, we propose a methodology to

organize the information advertised by UPnP AV devices and

selectively add the information into ontology to support service

composition. We focus our research on AV home network

environment. With the ontology, complex service composition

can be performed easily so that a better entertainment

experience can be offered to the users.

I. INTRODUCTION

ith the emergence of digital entertainment product,

users enjoy variety of entertainment devices ranging

from Personal Computer (PC) to Consumer Electronic (CE).

Entertainment CE devices, such as television, music player,

radio and home theatre system, have formed an inseparable

part of a home audio visual (AV) network.

Even though several devices could offer the same service,

each may provide different capacities. For example, although

all music players provide the service to play music files, some

might have the capability to play MP3 files but some can only

play WAV files.

Moreover, a few devices can cooperate together to perform

certain task in sharing digital entertainment contents, for

instance watch DVD movie. Task can be performed through

composing two or more services from different devices. To

compose the service for playing DVD movie, service such as

play DVD movie, display visual output and produce audio

output service are required. DVD movie can be played by

DVD player; while the graphical output can be streamed to a

television; and the audio output can be streamed to home

theatre system which can provide users with a better sound

system. Undoubtedly, users will favour a self-configuring

system where the setup of new device added into the home AV

network is trivial.

Some of the popular and well developed technologies like

Manuscript received June 13, 2008.

Kelvin Choy Chuong Wen, Tan Su Wei, Wong Sau Peng and Norliza

Mohamad Zaini are with Multimedia University, Jalan Multimedia,

Cyberjaya 63100 Selangor MALAYSIA (e-mail: [email protected],

[email protected], [email protected], [email protected])

Universal Plug and Play (UPnP) [1][2], Java Intelligent

Network Infrastructure (Jini) [3], Home Audio Video

interoperability (HAVi) [4] address the device

interoperability by defining the protocol used among the

devices to communicate with each other [5]. In general, these

technologies provide certain mechanism for a device to

discover other devices and services on the same network.

More importantly, protocols are defined for a device to

control or make use of the discovered services.

UPnP specification presents the greatest promise on

interoperability among the devices since UPnP specification

is widely adopted by most of the digital entertainment

industries [6]. UPnP Audio Visual (AV) architecture, which is

under UPnP standard, is specially designed to suit audio video

environment. UPnP AV specification defines the templates

for CE devices that interact with entertainment contents.

However, based solely on the UPnP AV Architecture is

insufficient to create an automated entertainment experience.

UPnP AV specification only exposes the services that can be

used to control the device. Users still need to identify and

choose the suitable device manually to perform certain task.

Thus, automated service composition has become an

important feature in home networking since service

composition aids in automating selection of appropriate

devices. Automated service composition combines different

basic services in the network to suit the problem requirements

without the need of user involvement. It accelerates

applications development, service reuse and complex service

construction.

Ontology is one of the effective ways to represent the

information needed for the service composition. Ontology

represents a set of concepts within a domain and the

relationships between those concepts. It is used to describe

about the properties or to define the domain. For example,

ontology can be used to describe the personal information

(concept) and the ownership of the information (relationship)

about the people (domain). Ontologies are normally used in

semantic web browser and knowledge management as a form

of knowledge representation for information [7].

Currently, there are a lot of researches in ontology and

service composition. Ontology provides the classification

about the devices on the network; while service composition

describes the services required to perform a task. Therefore,

both ontology and service composition can be integrated to

provide a platform for service composition to utilize the

information classified by ontology. However, the information

gathered in the reviewed researches does not based on any

existing specification such as UPnP. UPnP service description

Supporting Service Composition with Ontology-based UPnP AV

Architecture in AV Environment

Kelvin Choy Chuong Wen, Tan Su Wei, Wong Sau Peng, Norliza Mohamad Zaini

W

Proceedings of the 2008 IEEE Conference on Innovative Technologies in Intelligent Systems and Industrial Applications Multimedia University, Cyberjaya, Malaysia, 12-13 July 2008

978-1-4244-2416-0/08/$20.00©2008 IEEE

104

[8] provides information about the services available on the

device. Some of the services provided by the service

description can indirectly reflect the capabilities provided by

the device. The services from UPnP are advertised in the

network without proper classification to support service

composition.

Thus, we propose an ontology-based structure to represent

the UPnP AV device related information on the network. In

this paper, we analyze the UPnP AV specification and identify

the services that are necessary for service composition.

Ontology-based knowledge can be used in service

composition to assist user in choosing appropriate device to

perform desired task. The information extracted from UPnP

AV devices will be organized according to the schema that we

proposed.

In the remainder of this paper, Section 2 presents a

background study of UPnP AV architecture and similar

researches. Section 3 describes our architecture’s overview

and ontology structure of our proposal. In Section 4, we

analyze UPnP AV architecture and identify the elements that

are useful to support service composition. Conclusion and

future work are discussed in Section 5.

II. BACKGROUND STUDY

UPnP AV architecture closely follows the mechanism

defined in UPnP architecture, such as automatic address

configuration, device discovery, control and eventing. The

protocols used in UPnP AV architecture include TCP/IP,

HTTP, SSDP, SOAP, GENA, and XML. The UPnP AV

architecture extends the base of UPnP architecture to target

CE devices such as televisions, VCD Players, and PCs to

interact with entertainment contents. Three main logical

entities are classified in UPnP AV architecture: control point,

media server, and media renderer. Media server handles the

entertainment content and sends the content to devices that

request the content. Media renderer receives the contents from

other devices especially from media server and renders the

content using local hardware. The control point operates as a

gateway between user and UPnP devices to manage the

operation of media server and media renderer.

There have been a number of related works [9 - 11] on

collecting and organizing information from the devices on the

network. Many existing frameworks utilize ontology structure

to achieve semantic service discovery and service integration.

Network Appliance Service Utilization Framework

(NASUF) [9] is proposed to address the challenges associated

with the wireless ad hoc networked appliances where the

functions are not known a priori. NASUF is claimed to

provide mechanism that can dynamically compose the

services through ontological information. However, the

organization of device or service information is not clearly

described.

On the other hand, the framework in [10] is designed

especially for general devices to describe the devices and

services to support service discovery. In this research, the

proposed ontology is focused on device information instead of

service information. Therefore, the framework is not suitable

for service composition.

In [11], Kaloja et al. identified the required vocabularies

and classifications for different functional domains (mobile,

Personal Computer (PC), Consumer Electronics (CE)). The

intention of the vocabularies and classification is to enable the

efficient automated execution of the integration for

heterogeneous networks in a seamless manner to address the

user requirement. Despite the completeness of the proposed

classification, the ontology structure includes extra

information that might not be necessary for achieving our goal

which particularly focused on home entertainment device

network.

In summary, current related ontology researches are not

best suit for service composition in AV environment. In this

paper, we proposed an ontology-based schema for UPnP AV

specification to support service composition. In our ontology

architecture, we refine the ontology that we proposed to a

more precise level, specifically sufficient to support an

effective service composition. The root of the proposed

ontology is based on capability information instead of device

information.

III. PROPOSED ARCHITECTURE

A. Problem Statement

Current existing technologies such as UPnP, Jini, HAVi and

others addressed the communication problem among devices

on the network. However, during device and service

discovery, the discovered information is not organized in a

structured manner so as to support service composition. With

these technologies, when more and more devices become

available in the network, it may prove to be a daunting task for

the users to decide which devices to be used for a task

involving multiple devices. In this respect, ontology can be

used to organize the capability of devices on the network to

facilitate automated selection of proper devices to be used. As

mentioned earlier, related works mainly focus on the structure

of ontology for organizing information about devices and their

services. These proposed ontology structures are not related to

any existing home networking specifications such as UPnP.

These ontology structures are not specific. Hence, we propose

an architecture utilizing ontology which organizes the UPnP

AV specification to support service composition.

B. Assumption

In our architecture, we assume that inter-device

communication is done using UPnP mechanism for discovery,

advertisement, controlling and monitoring.

C. Architecture Overview

The main objective of this architecture is to extract the

information from UPnP AV specification to support service

composition. The extracted information will be organized into

ontology for service selection, access and usage.


978-1-4244-2416-0/08/$20.00©2008 IEEE

105

Information advertised by the networked devices is

organized into device ontology and capability ontology. The

device ontology describes the device’s profile (e.g. device IP

address, device’s location and device’s usage status); while

the capability ontology describes the device’s functionalities.

The capability ontology only contains the necessary

information about the functionalities which is needed in the

service composition.

Figure 1 shows the context diagram of our architecture to

extract and organize information. In general, device will

advertise the device and service description. Advertisement

received by processing device will be processed by

information organizing component. By referring to the

ontology schema, information processor will process the

advertised information to construct a knowledgebase.

When service description advertisement is received by the

processing device, interpreter will interpret the message by

referring to the interpretation knowledgebase and determine

the capability from the service. Detail of interpretation will be

described in the following sub-section. Device description

advertised from devices and capability information extracted

from the interpreter will be organized by Information

Organizer which refers to organization structure for

organizing the information. Organized information will then

be stored in device taxonomy and capability taxonomy

respectively.

Ontology Schema

Processing Device

Information Processor

Knowledgebase

Information Organizing Component

Figure 1: Context Diagram of Proposed Architecture Model

D. Interpretation Structure and Knowledgebase

Interpretation structure is a schema that provides the

guideline for information processor to process the advertised

service description. From the service description, interpreter

can determine the capability of a device by referring to

interpretation knowledgebase. Interpretation knowledgebase

store the information about the essential actions that is useful

in service composition.

Figure 2 shows the interpretation structure which is used by

interpreter for reference. Service is needed in the

interpretation structure because the interpreter can skip the

unrelated UPnP service to enhance the interpretation speed.

Action in the interpretation structure provides the exact

functions of the device to be invoked, in which the result from

the invocation of functions will be interpreted by the process

in the interpretation structure. Process defines how the action

should be requested or interpreted to extract the capability.

The result of the interpretation will be categorized under the

capability category specified in interpretation knowledgebase.

Interpretation structure also considers the condition where

there might be certain capabilities that are specific for certain

device despite of the same model. Dependency shows either

the capability that is interpreted from service is a general

capability for the same device model or is a device specific

capability.

Figure 2: Interpretation Structure

Figure 3 shows the interpretation knowledgebase example.

From the Connection Manager service, we can get the format

that is supported by the device by calling the UPnP action of

GetProtocolInfo. Invoking this action will return information

about the protocol, network and content format of the device.

Content format can be differentiated into format supported by

audio output and visual display after the return value of

invoked action is processed by audio playback process and

video playback process. The result will be categorized into

AudioOutput and VisualDisplay category respectively.

Figure 3: Example of Interpretation Knowledgebase

E. Organization Structure

We proposed 2 parts of organization structure for

organizing device and capabilities information, namely device

structure and capability structure. Device structure organizes


978-1-4244-2416-0/08/$20.00©2008 IEEE

106

the general information about device especially the device

profile. Capability structure organizes the functionalities

which are interpreted from the device’s service description by

the interpreter.

Device structure is designed as in Figure 4. It includes the

information device description such as the device model, type,

unique identifier (UUID), preferred name, ip address,

location, status, and manufacturer information. All device

specific information is grouped under the hierarchy of UUID

to represent a physical device. Device Type and manufacture

information is clustered under the hierarchy of model name

because the device type (such as television, lighting, speaker,

etc.) and manufacturer will be same for same model of device.

Capabilities are categorized by the capabilities functionality

as capability category. Capabilities are further divided into

input and output of the processes. Device structure and

capability structure are connected together from capability in

capability structure to model name or UUID in device

structure depending on the device dependency. If the

capability are general capability which compliance to the

device of the same model, the capability structure will be

connected to model name through capability_in_model

relationship. On the other hand, the capability which is device

specific will be connected to UUID directly through

capability_in_udn relationship.

Figure 4: Capability and Device Structure

IV. UPNP AV ARCHITECTURE ANALYSIS AND DISCUSSION

UPnP AV architecture [6] provides a standard specification

for UPnP based AV devices. In order to support

interoperability among devices, media server and media

renderer exposes several services. In particular, media server

exposes services such as Content Directory, AV Transport

and Connection Manager services. These services enable

searching for the contents and ensure the smoothness in the

streaming of AV contents out to other devices. On the other

hand, Rendering Control, AV Transport, and Connection

Manager services are exposed by media renderer to control

the local hardware and ensure the streaming of AV contents

from other devices. We will analyze these services and the

useful information necessary for supporting service

composition.

A. Content Directory

Content Directory services [12] are implemented by media

server to provide information about the media resource that is

shared for other rendering device to render the AV files.

Client, especially with User Interface (UI) can discover and

list out the media resource that is accessible by media server

through Content Directory service. Audio tracks, movie clips,

pictures are some of the media resources that a media server

may be accessible.

Among all the services in Content Directory [12], the

service that is useful for service composition is Search action.

Search action is used in UPnP to search for objects in the

media server that match the search criteria. Through the

Search action, we can extract the available media resource

and group them under different categories by manipulating the

input parameter for SearchCriteria argument. For example,

we group the music track under AudioContent in our ontology

by specifying the SeachCriteria with upnp:class derivefrom

“object.item.audioItem.audioTrack”. Hence, Search function

is sufficient to gather the information we required to support


Another similar UPnP action compare to Search action is

Browse action. Browse action is used in UPnP to

incrementally browse hierarchy for the contents in media

server. However, Browse action is not required in gathering

information for service composition. Browse action is more

useful where user would like to browse through the available

media resource list through UI provided for the users.

B. AV Transport

AV Transport services [13] are implemented by both media

server and media renderer to control the flow of the audio and

video streams among the participating devices. AV Transport

also provides information about the AV capability (capability

to play, record, record quality, etc.), current transport status

(play mode, play quality, play speed, etc.) and the media status

(track number, duration, media medium, etc.).

Among all the services in AV Transport [13], the services

that are useful for service composition are Record and

GetDeviceCapabilities action. Record action is used in UPnP

to start recording at the specific instance;

GetDeviceCapabilities action is used to retrieve the

information on device capability for specific instance, for

example recording format and recording quality. With Record

action, we can directly recognize the record action as having

the Record capability. GetDeviceCapabilities action provides

the information about the recordable media (CD-RW, HDD,

etc.) in RecMedia argument. With this information, the service

composition can easily locate the suitable device to store the

media.

However, functions in transport status are more to monitor

the transportation of media resource. On the other hand,

functions in media status are more to display for the user

acknowledgement and should be managed by the UI. UI

should also handle the function to modify the media status

(play, pause, next, previous, etc.) at control point. Thus, these


978-1-4244-2416-0/08/$20.00©2008 IEEE

107

functions are not suitable to be included in service

composition.

C. Connection Manager

Connection Manager services [14] are implemented by both

media server and media renderer mainly to check the protocol

and content information of the sink (the destination) and

source resource. Connection Manager services also provide

interface to allow the control point to setup and teardown

connections between the sink and source resource.

Among all the services in Connection Manager [14], the

service that is useful for service composition is

GetProtocolInfo action. GetProtocolInfo action is used in

UPnP to check the protocol-related information, such as

protocol, network, and content format information. From

GetProtocolInfo action, we identify the content format

information from the result of GetProtocolInfo action. The

supported format information will be grouped together in

AudioOutput or VisualDisplay category in our ontology. This

information is important when the service composition need to

decide the most suitable device to present the entertainment

resource to the user.

However, the function to setup and teardown the connection

between devices is not relevant in service composition and

should be handled by control point during the preparation to

transport the resource from sink to source.

D. Rendering Control

Rendering Control services [15] are implemented by media

renderer to query or adjust the rendering attribute of the

device supported. Rendering attribute includes the display

characteristic (brightness, contrast, color gain, etc.) and audio

characteristic (volume, equalizer settings, balance, etc.). User

can retrieve the current setting of the rendering attribute and

change the rendering attribute according to the user

preference through UPnP control point. Functions in

Rendering Control should be bound to the UI provided by

control point. Moreover, Rendering Control services are

intended to display or adjust the rendering attribute but are

insignificant to influence the decision to compose a service.

Hence, Rendering Control services are not necessary for


E. Scheduled Recording

Scheduled Recording services [16] are implemented on

UPnP devices that have the ability to schedule the recording of

content especially broadcast content. Scheduled Recording

will grant the permission of UPnP control points to record the

content at desired time.

Among all the service in Scheduled Recording [16], the

service that is useful for service composition is

CreateRecordSchedule action. CreateRecordSchedule action

is used in UPnP to create a schedule to record at desired time.

The present of CreateRecordSchedule action in the service

description indicates that the device has the capability to

record the content at desired time.

UPnP

Service

UPnP

Action

Input

Argument

for the

Action

Input Value for the Argument Output

Argument

for the

Action

Interpreted

capability

Capability

Category

Content

Directory

Search Search

Criteria

upnp:class derivefrom

“object.item.audioItem.audioTrack”

Result audio

format

Audio

Content


“object.item.audioItem.audioBroadcast

”

Result audio

channel

frequency

Audio

Channel

Content


“object.container. storageVolume”

Result location of

storage

Available

Storage

Volume


“object.item.videoitem.movie”

Result video

format

Video

Content

upnp:class derivefrom “object.item.

videoitem.videoBroadcast”

Result video

channel

frequency

Video

Channel

Content

AV

Transport

Record - - - Instance

Record

Recording

Get Device

Capabilities

- - Rec Media Media Recording

Connection

Manager

Get

Protocol

Info

- - Sink Audio

Playback

Audio

Output

- - Sink Video

Playback

Video

Display

Scheduled

Recording

Create

Scheduled

Recording

- - - Scheduled

Record

Recording


978-1-4244-2416-0/08/$20.00©2008 IEEE

108

Table 1: Summary of Capabilities Extracted from UPnP AV Specification

However, the functions to browse the record schedules or

task and the actual method to create the record schedule are to

be known by the control point instead of service composition.

The procedure to record the content will be handled by the

control point once the control point receives the input from the

user. Hence, as long as CreateRecordSchedule action is

presented in the service description, it is sufficient to conclude

that the device has the capability to record the content support


F. UPnP AV Architecture Analysis Summary

From the analysis of the UPnP AV architecture, we

summarize a few capabilities that can be extracted from

existing UPnP AV specification. Table 1 shows the summary

of the capabilities that can be extracted based on the analysis

on UPnP AV specification.

With this information, it is sufficient to perform several

composite services such as search and play music, record

channel, and etc. The examples of the services that can be

performed are:

i. Play a song name “Boys” from any device.

Capability Category required:

a) Audio Content

b) Audio Output

ii. Play a movie name “Harry Portal” from any DVD

source.


a) Video Content

b) Video Display

iii. Record radio channel 15 at 4PM on March 19, 2004


a) Audio Channel Content

b) Scheduled Recording

c) Available Storage Volume

The scenario and walkthrough example on the usage of the

ontology and the system are described in [17].

V. CONCLUSION AND FUTURE WORK

With a large choice of functional capabilities from devices

added to the network, service composition is essential to

simplify the user decision in selecting among multiple

devices. In this paper, we analyze UPnP AV specification and

interpret the service information to determine the capability of

the device. From the analysis, we discover that UPnP AV

specification contains unnecessary information for service

composition. Thus, besides interpreting the service

information, we selectively store the information into

ontology-based knowledgebase for the use in service

composition.

We organize the information into ontology-based data

structure to aid in service composition. With the classification

of device-related information in our ontology structure,

system can determine the suitable devices capable to perform

certain task.

With the rapid development of technology in entertainment

devices, many new device capabilities will appear in the near

future. In future, we will be looking into the selection strategy

when there are more than one devices provide the same

capability. We’ll also concentrate on the extendibility and

flexibility of the interpretation schema. With the mechanism

for extendibility and flexibility of interpretation schema, new

devices can easily be adopted into our architecture.

REFERENCES

[1] Fout,T., “Universal Plug and Play in Windows XP”, July 2001,

http://www.microsoft.com/technet/prodtechnol/winxppro/eval

uate/upnpxp.mspx

[2] Member of UPnP Forum, “UPnP Device Architecture 1.0”,

UPnP Implementer Corporation, 2 December 2003

[3] Member of Sun MicroSystems, “Jini Architecture Overview”,

Sun Microsystems Inc., 1999.

[4] Member of HAVi Organization, “HAVi, the A/V digital

network revolution”, The HAVi Organization, 1999.

[5] Helal S., “Standards for Service Discovery and Delivery”,

Pervasive Computing, IEEE, Vol. 1, Issue 3, pg 95-100,

July-Sept. 2002.

[6] Member of UPnP Forum, “Overview of UPnP AV

Architecture”, UPnP Implementer Corporation, 7 February

2003

[7] Steffen Staab, Rudi Studer, “Overview of Ontology

Technology”, Springer, January 2004. [8] Member of UPnP Forum, “ServiceType:V Service Template

Version 1.01 For UPnP Version 1.0”, UPnP Implementer

Corporation, 16 January 2001

[9] Madjid Merabti, “Networked Appliances in Home

Entertainment”, ACM International Conference Proceeding

Series; Vol. 223, Proceedings of the 2006 international

conference on Game research and development, pg 288-293,

2006.

[10] Bandara A., Payne T. R., de Roure D. and Clemo G. (2004) “An

Ontological Framework for Semantic Description of Devices

(Poster).” In Proceedings of International Semantic Web

Conference (ISWC), Hiroshima, Japan.

[11] Kaloja J., Kantorovitch J, et. al, “The Vocabulary Ontology

Engineering for the semantic modeling of home services”,

http://www.hitech-projects.com/euprojects/amigo/publications

/ICEIS2006.pdf

[12] Member of UPnP Forum, “ContentDirectory:2 Service

Template Version 1.01 For UPnP Version 1.0”, UPnP

Implementer Corporation, 31 May 2006

[13] Member of UPnP Forum, “AVTransport:2 Service Template

Version 1.01 For UPnP Version 1.0”, UPnP Implementer

Corporation, 31 May 2006

[14] Member of UPnP Forum, “ConnectionManager:2 Service



[15] Member of UPnP Forum, “RenderingControl:2 Service



[16] Member of UPnP Forum, “ScheduledRecording:1 Service



[17] Choy C.W., Wong S.P., Tan S.W., Norliza M.Z., Ettikan K.K,

“Organization of Devices Information in Ontology

Representation to Support Service Composition”, The 3rd

International Symposium on Information Technology 2008

(ITSim2008), 2008, to be published


978-1-4244-2416-0/08/$20.00©2008 IEEE

109

Supporting service composition with ontology-based UPnP AV architecture in AV environment

Documents

Transcript of Supporting service composition with ontology-based UPnP AV architecture in AV environment