A Description-based Approach to Mashup of Web Applications, Web Services and

Mobile Phone Applications Prach CHAISATIEN, Takehiro TOKUDA

{prach, tokuda} @tt.cs.titech.ac.jp Department of Computer Science, Tokyo Institute of Technology

Meguro, Tokyo 152-8552, Japan

Abstract. Recent developments in mobile technology have enabled mobile phones to work as mobile Web servers. However, the composition of mobile phone applications and Web resources to form new mashup applications requires mobile programming knowledge ranging from how to create user interfaces, network connections and access to Web resources. Furthermore, the unique capabilities of mobile phone applications such as access to camera inputs or sensor data are often limited to local use only. To address these problems, we present a description-based approach and an Integration Model for the composition of mobile mashup applications combining Web applications, Web services and mobile phone applications (i.e., generic components). The compositions appear to require less native mobile programming knowledge. In the current work, to leverage access to these services and applications, an Interface Wrapper was used to transform generic components into mashup components. Composers were able to transform and reuse form-based query results from Web applications and integrate them with wrapped output from users’ interaction with mobile phone applications, and other Web services. The final applications can be configured to work two ways: 1) as native mobile phone applications or 2) as a Web application accessible externally via a mobile Web server application.

Keywords. Mobile phone application, mobile Web server, Web service, Web application, mobile mashup, Interface Wrapper

1. Introduction

Mobile phone applications deliver unique capabilities such as GPS location services, voice recognition and camera/image processing applications. There are some problems related to the composition of mashup applications of these components with existing Web resources. One of these problems is related to the lack of mobile programming language knowledge needed for the creation of user interfaces and control parts. Another issue is that composers need not only to know how to create a standalone mobile application, but also need additional skills to program the mobile phones to access and reuse Web resources.

To address these problems, this paper presents a description-based approach to flexibly compose mashup applications from 3 generic component categories: Web applications, Web services, and mobile phone applications. With minimum

configuration required, our approach allows composers to accomplish the following tasks in the aforementioned categories:

• Simplify and reuse form-based query results from Web applications. • Extract selected portions from Web services’ outputs. • Generate and configure Web service interfaces for mobile phone applications. In the composition procedures, first, the Integration Model is used to describe and

plan data flows of the mashup components. The Integration Model later is expanded into configurations of a Mobile Integration Description file (MID file). Then the mashup application generator uses the file to generate the actual mashup applications. In the similar a manner, composers are required to fill the control parameters of each component in the MID file. Thus a mashup application is generated according to those configurations. Lastly, composers were able to configure the final mashup application to run on the device as a mobile phone application or to be accessed externally as a Web application via the mobile Web server application.

To leverage access to each mashup component, we termed the components that transform communication interfaces between component categories “Interface Wrappers”. For instance, the Web service wrapper detailed in this study enables inter-component communication and external access to a mobile phone application using a Web service interface.

Figure 1. Overview of the mashup applications, Interface Wrappers and their relation to outputs and clients

This study’s contribution is the presentation of the model and the methodology to

reuse non-API Web resources with existent mobile phone applications to form mashup applications. The established method is to use Java classes to build and to connect components, while our method controls the data flows of existent mashup components through the utilization of configuration files and its parameters. The implementation of this study shows that our approach allows composers to flexibly reuse capabilities of sensors and peripherals controlled by mobile phone applications to integrate them with Web resources and generate new mashup applications.

The organization of the rest of this paper is as follows. Related work and research background are reviewed in Section 2. A mashup example is presented in Section 3 to demonstrate our approach. Section 4 explains the method of composing a mobile mashup application. The composition is divided into three working processes: planning

process, configuration process, and application generation process. Also Section 4, we present the Web information extraction tool used in the configuration process. In Section 5, we provide detailed mashup composition examples, then evaluate them by presenting the applications’ actual drawbacks and problems when applying the same model with other resources. In Section 6 we give a general discussion by making a comparison to the conventional approaches in terms of generation process, objectives and limitations. In Section 7, we describe this study’s future work and present our concluding remarks.

2. Background and Related Work

Research disciplines in mobile mashup are usually related to these fields of study: 1. Web page tailoring and adaptation. 2. Web information extraction and reproduction. 3. Mobile mashup languages, modeling and their applications. 4. Mobile servers and ubiquitous environments. Generally, the conventional focus in tailoring and adapting Web pages for viewing

on mobile devices gives more importance to extracting and simplifying visual outputs. DOM tree based extraction and proxy server architecture, as presented in [1] [4] and [11], are used to adapt the presentation of a Web page on mobile devices to assist navigation effectiveness. Although these methods promote minimization of information and visualization, when composing a mashup application for mobile phones they appear to support less functionality in communications and integrations over multiple working components.

Research in the field of Web information extraction emphasizes methods to correctly indicate and reproduce parts of Web applications for creating new mashup Web applications. The study in [7] proposed a Web information extraction method to generate the virtual Web service functions from Web applications at the client side. This research targeted static contents, a limitation which was later corrected in [8] by allowing dynamic Web contents created by client-side scripts. These two systems are implemented using large external Java libraries including Java Applet. In our case, a mobile device cannot handle loads from external libraries to extract and simulate entire Web pages.

In approaches to use description language based on XML, research in [13] and [15] has shown that the majority of description-based XML languages are designed to support content delivery to mobile phones and handheld devices. However, most languages target user interface design and do not facilitate integration with Web information. XISL [12], which extends interaction and input methods, requires an implementation of interpreter and the dialog manager modules. One substantial difference when compared to our approach is that we reuse interactions from existent mobile phone applications, and do not create new applications concerning users’ interaction from the description file.

A method to generate a mobile phone application using configuration files was presented in mFoundry’s Mojax [14]. This framework borrows syntaxes from JavaScript, CSS and XML. Mojax applications are compiled and run as native Java code (J2ME) on a device. Mojax also supports development of plug-ins that can access device capabilities such as location services, address book, audio and video. Our approach introduces the transformation of generic components into mashup

components. Moreover, developers are able to write the optional control parts using Web or mobile programming.

In 2002, Intel Research introduced the Personal Server [19], which wirelessly connected to the local network environment. The personal server allows an HTTP connection to access personal pages or file storage. A more specific study of component-based infrastructure can be found in [16]. This system used abstract user interface descriptions to represent software components on an embedded hardware system. Although a method to display system information and control the hardware system using a variety of clients was presented, those connections were specific to the transport layer. The use of information for integration with Web information was also not found.

The sharing of mobile services presented in [10] is a system based on websites that support user generated mobile services. Our approach instead promotes the use of mobile integration with Web content, allowing contributors from these two platforms to share their works. The mobile service system in [17] provides an extension of presence awareness to mobile users. Without the implementation of the central server system, our approach is applicable with always-on HTTP connection via the mobile Web server, which allows quick access to shared information anywhere and anytime.

Concerning development of mobile Web servers, ServersMan [18] is a mobile application targeted on major mobile platform (iPhone, Windows Mobile and Android). The application enables Web access to devices’ file storage and other parameters such as GPS latitude and longitude. The operations to access devices’ other resources, such as digital compass or accelerometer, are not yet defined. Moreover, the reuse of existent mobile application in the device is also not presented.

3. Mobile Mashup Example

We present our example to demonstrate how the description-based approach is used in composing a mashup application. In Example 5.1.1, we show the composition of a mashup application for displaying nearest Wikipedia article and local weather information according to the mobile phone’s location.

Figure 2. Mobile Mashup Example

This application is targeted for a mobile Internet device (i.e., Output, Web application, iPod Touch, Safari Browser) by compositing a location service from a mobile phone application (i.e., Component A, publisher, GPS Locator, mobile application) with Web services. With no built-in GPS hardware at the client side, component B and C can alternatively retrieve information from a location service on component A and perform queries for their Web service outputs (i.e., Component B, subscriber, Wikinear, Web service and component C, subscriber, LocalWeather, Web service). Working procedures in composing this mashup application are as follow.

1. Specify a starting component: The data flows in this mashup application begin from a component that accesses GPS parameters from a mobile application (GPSLocator). For compatibility to the next components, we first transform the parameters by applying the Web service wrapper to GPSLocator. Composers must specify the Intent parameters and Intent’s extra parameters to retrieve data from this mobile application. It is also required that composers specify the provider role to the component including provider’s ID. Composers need to specify the Web service wrapper’s JSON message as well.

2. Specify next components: The required parameters for the next components are Web services’ URL, query field names, and each field’s value. In this example both Wikinear and LocalWeather used fields named lat for latitude and lng for longitude. The components’ role must be set to subscriber and use publisher’s ID where lat and lng are referred to.

3. Specify the output component: The output component, which is in the form of Web applications, uses query results from Web services described in 2. Composers must specify the mobile Web server’s access path and the output page in the form of HTML code and refer to parameters from Web services’ output.

4. Generate the final mashup application: Composers input information in item 1 - 3 to the MID file and generate the output Web application which is placed on the mobile Web server. Users can access it using a mobile Web server host name and access path according to the configuration.

To support the compositions of mashup applications, the Integration Model is used to plan the data flows in the mashup applications.

4. Method in the Composition of Mobile Mashup Application

Our method of composing mobile mashup applications consists of planning, configuration, and generation processes. In the planning process, the Integration Model is used to outline the component’s roles, the data flows, and the format of output forms for mashup applications. In the configuration process, the Integration Model is adapted and expanded into the actual configuration of MID files. We use the Web extraction tool to aid composers in retrieving configurations for data extraction from Web applications. Later in the application generation process, the mobile mashup application generator uses MID files to generate the actual mobile mashup application. The data flows involve different components located in different locations of a large system. Therefore, we explain the current system architecture to assist in understanding data flows and how the generated mobile mashup applications are to be placed in the system. Then we present the detailed processes of generating mashup applications.

4.1. Planning Process: Integration Model

Table 1. Model representation of mashup components, roles and output forms

Category Model Representation Abbreviation

Mashup component C [Web Service (name)] C [Web Application (name)] C [Mobile Phone Application (name)]

C [WS (name)] C [WA (name)] C [MA (name)]

Role R [Publisher] R [Medium] R [Subscriber]

R [Pi] R [Mij] R [Sj]

Output form O [Web Service] O [Web Application] O [Mobile Phone Application]

O [WS] O [WA] O [MA]

A model representation of three mashup components, roles and output forms are shown in Table 1. Parameter indices indicate the publisher-subscriber relationship of component couples. As an extension to publisher (P) and subscriber (S) roles, we use medium role (M) to describe the component that is publishing a subscribed output from another component. The representation of component A in mobile mashup example 5.1.1, which is the Web service wrapper applied to mobile phone applications can be written as

C [Mobile Phone Application (GPS Locator)], O [Web Service]

We call these one-tier compositions Interface Wrappers, which are used in order to transform an output’s interface for communication between other mashup components. Table 2 contains a model representation of wrappers, their corresponding functions and sample usages.

Table 2. Model representation of Interface Wrappers, their corresponding functions and sample usages.

Case Model Representation Function Sample Usage

(a) C [Web Application] O [Web Service] = W [WS[WA (name)]]

Web content extractor functioning as a Web service

Extracts texts from query-based Web page (e.g. product search, book

reviews, game ratings)

(b) C [Mobile Phone Application] O [Web Service] = W [WS[MA (name)]]

Mobile Web Service Wrapper

Retrieves GPS coordinates from mobile

phone application via a Web Service

(c) C [Web Application] O [Mobile Phone Application] = W [MA(name)[WA(name)]]*

Mobile application functioning as a Web content extractor

Displays part of query-based Web page

(d) C [Web Service] O [Mobile Phone Application] = W [MA(name)[WS(name)]] *

Mobile application functioning as a Web service connector

Selects and displays texts from Web service’s result

using native mobile phone application

(e) C [Web Service] O [Web Application] = W [WA[WS(name)]]

Web application functioning as a Web Service connector

Searches and displays results from a Web

service

(f) C [Mobile Phone Application] O [Web Application] = W [WA[MA(name)]]

Mobile Web Application Wrapper

Searches for contact info, media or database query

on the mobile phone

*Since usage and output of mobile applications are different, application name has to be declared.

Outputs in the form of Web application (WA) or mobile applications (MA) can be used as end points in creating a mashup application. A composition that contains only one wrapper might not be enough to form a meaningful application. In Section 5, we select case (a) (b) and (c) for our implementation because of the following reasons.

• Case (a) and case (e) are considered as existent Web extraction techniques. Web service (WS) output from wrapped WA in case (a) would be more appropriate for showing complexity in creating a mashup application.

• Case (b) and (f) are similar to each other, only the output forms are different. We select case (b) to show further integration of its WS output.

• Case (c) is more complex than (d). We would like to show how information is extracted from WA in (c) while (d) contains only simple operations to use WS output.

By applying new interface syntax (b) to example 5.1.1, a new abstract model can be declared as

W [WS[MA(GPS Locator)]], R [P1], C [WS(Wikinear)], R [S1], C [WS(Weather Report)], R [S1], O [WA(iPod Touch, Desktop)]

In the next section, we describe how to adapt and expand this model as an actual configuration file.

4.2. Configuration Process: MID File and Web Extraction Tool

In the configuration process, the Integration Model is adapted and is expanded into the actual configuration of MID files. We use the Web extraction tool to aid composers in retrieving configurations for data extraction from Web applications.

4.2.1. MID File Table 3. Structure of the <project> scope in MID files.

Scope Possible Child Element Composition rule <component> (…) </component> or <wrapper> (…) </wrapper>

One or more, in any order <project>

<output> (…) </output> One or none <name> NAME </name> One <role> (…) </role> One

<component> <webapplication> (…) </webapplication> or <webservice> (…) </ webservice > or <mobileapplication> (…) </mobileapplication>

One

<name> NAME </name> One <role> (…) </role> One <webapplication> (…) </webapplication> or <webservice> (…) </ webservice > or <mobileapplication> (…) </mobileapplication>

One <wrapper>

<output> (…) </output> One

<output> <webapplication> (…) </webapplication> or <webservice> (…) </ webservice > or <mobileapplication> (…) </mobileapplication>

One

Note: Italic capital letters denote that an element’s content has to follow definition rules. Notation (…) means that element’s inner contents are in the form of tags.

A mobile integration description file or MID file is an XML description file and is used for the configuration of working components in our mashup applications. Since each component has unique configurations, we begin by explaining the structure of MID files in Table 3, then we provide a detailed description of each element.

The <project> scope of the MID file represents the mashup application according to the Integration Model. Composers are able to put as many <component> and <wrapper> inside this scope, in any order. The name of each component and wrapper (NAME) must be unique since the namespaces are used to access data between components. We then specify the category of each element using one of <webapplication>, <webservice> or <mobileapplication>. This <project> scope ends with one <output> element if the last component is not a wrapper. We do not use <output> when the last component is a wrapper since it contains its own output element. How a MID file is used is shown in Example 5.1.1 and in Figure 3. The MID file includes the Web service wrapper of GPSLocator mobile application and other two Web services. Another example of the Web Service wrapper is shown using the Amazon Web application [3] in Figure 4.

Figure 3. MID file of example 6.1.1 (simplified)

As shown in Figure 3, the followings are important control parts in a MID file: a) Subscriber ID must match the publisher ID of the component it subscribes to.

The publisher.output namespace is used to access parameters in <output> scope of the corresponding publisher.

b) Inside the <wrapper> scope of Interface Wrappers, the namespace results is used to refer to data in the <results> scope of the component.

c) The <output> element is placed under the <project> and <wrapper> scopes. In the case of project’s output, components’ names are used in front of the namespace results. The namespace results is used to access to the corresponding component’s results (i.e., Wikinear.results and LocalWeather.results).

d) In the current development version, Web applications can be generated using settings limited to the combination of <string> and <variable>. It is recommended that composers use external Web builders and integrate the working Web pages with a Web service output.

e) The parameter in <mode> under the scope of <mobileapplication> is used to determine whether the information from mobile applications is retrieved actively from a foreground process, or passively from a background process. The <mode> parameter of the output mobile application is only allowed to be set as active.

Figure 4. Part of a MID file showing the Web service wrapper of the Amazon Web Application (simplified).

In Figure 4, the following are important control parts in a MID file regarding

<webapplication> scope: f) The parameter in <mode> under the scope of <webapplication> is used to

determine whether the information to be input into the query form of the Web application is passed from other components passively (passive) or requires user input (active).

g) The parameter in <value> under the scope of <input> and <webapplication> is defined as the input of Web application components. This parameter can be set as

a namespace or a user input (userinput). A Web application’s mode has to be set to active in order to use the parameter userinput. For the next <input> element, if the resulting pages require clicks to continue to the next page, we set parameter click to the <value> scope.

h) The parameter in <value> under the scope of <result> and <webapplication> is defined as the result of Web application components. This parameter can be set as a transfer of the entire information of this node to results scope (self) or a text attribute of the node (text).

i) In Section 4.3, we show the Web extraction tool which automatically generates parameters used in the <webapplication> scope.

4.2.2. Web Extraction Tool

To aid composers in correctly specifying the desired information from the form-based Web application, we use the web extraction tool to help in generating parameters for Web application components (i.e., <webapplication> scope) in the composition of mashup application. The tool is written in JavaScript which can be run on any browser. Figure 5 shows the sample use of the Web extraction tool applied to www.amazon.com [3] to search for a product title using a barcode number. We later used this page as a Web application component to integrate with a barcode reader mobile application in Section 5.1.2. Working steps to acquire the parameters are listed below.

Figure 5. Sample use of the Web extraction tool.

1. Composers select the desired input box and perform right click. At this point

the tool will recognize ID, name and class name attribute of the input box. It also calculates the distance of two nodes between the form node to the selected input box node.

2. Next, composers input an example into the input box and submit the page (we used barcode number of a book to search for result).

3. After this step, composers highlight the desired content on the resulting page and perform right click to the element.

4. Results from step 1 are applied to the <input> element of the Web application component’s setting in the MID file. In the case that the resulting page requires clicks over links, a parameter click can be applied to <value> element and repeat step 3. Setting for multiple resulting pages can be accomplished by adding more <input> element to the configuration.

5. When composers reach the final resulting page, they can perform right clicks to more than one element to generate the corresponding parameters. These parameters will be placed in the <result> element of the Web application component’s setting in the MID file.

4.3. Generation Process: System and Mobile Mashup Application Generator

In this section, we begin by explain the current system architecture for understanding data flows and how the generated mashup applications are to be placed in the system. Then we present the detailed processes of generating mashup applications. In the last part, we describe the Web extraction tool which aid composers in data extraction from Web applications.

4.3.1. System Architecture

Figure 6. System architecture

In Figure 6, the physical appearance of the system’s hardware is an Android mobile phone which can function as a generic mobile phone capable of running mobile applications. We used i-jetty open-source Web container [9] to extend its capabilities to

function as a mobile Web server. As the categories of final mashup applications are of two kinds, describes as follows:

In the first case, the mashup applications are generated as mobile applications and are installed to the application storage section like other typical mobile applications. This case occurs when one or more components in the composition are used as an active component. Active components consists of active mobile applications (AMA), active Web applications (AWA), wrappers containing mobile application (WAMA) and wrappers containing active Web application (WAWA). This means that the mashup application used user interaction with a mobile phone application or a Web application. Therefore, an entry point on the mobile phone had to be created as an installable mobile application (using Android Package or APK files). Additional control parts to access to Web resources are stored in the Web Files section of the mobile Web server.

In the second case, the mashup applications are generated as Web applications and placed on the Web Files section of the mobile Web server. This case occurs when there is no AMA or WAMA found in the composition. Therefore, the mobile Web server can passively access all sensors’ parameters from mobile applications without any user interaction with a mobile phone application. The Web Files section stores generated Java Servlet for external access, and Java classes which enable Web server module to access information from other existent mobile applications in the same machine.

4.3.2. Mobile Mashup Application Generator

The main process of generating mashup applications is performed according to configurations in the MID file. The process follows these steps:

1. The generator notes the presence of active components in the composition. 2. From the realizations in step 1, the generator will decide whether to create a

switcher mobile application (SMA, see Figure 5) which can switch through user interfaces (UI) in mobile applications according to the number of active components found. Rules to determine the generation of SMA are as follow. 2.1. If one or more active components are found then the generator will

generate an SMA as APK file and control parts will be generated and placed in the Web Files section on the mobile Web server.

2.2. If no active component is found then an SMA will not be created. Only the control parts will be generated and placed in the Web Files section on the mobile Web server.

3. The generator then begins generating each component’s working class files which will be assigned to the corresponding section decided in step 2.

4. Additional control class files work as linkages between components: these file are assigned the Web Files section.

5. The generator then compiles these assigned class file as follows 5.1. Class files assigned to SMA are compiled into APK files. 5.2. Class files assigned to Web Files are compiled into the Web application

archive (WAR) files. The installation tool for the generated files is still being developing. However,

composers are currently able to install APK files to the mobile phone via Android debug bridge tool. WAR files can be installed to the mobile Web server module using the i-jetty user interface [9] which requires the files to be downloaded through network connection. In Figure 7 we show the relationship of each mashup component to class files and compilation files in the generation process.

Figure 7. Relationship of each component to class files and compilation files in the generation process

5. Evaluation

As there are many components composers can use in the composition of mashup applications, they appear to have endless combinations. Therefore, we show the successful composition examples at first, and then review the specific problems in applying the same method to other similar cases. In our general discussion in Section 6, we compare our composition method with the conventional method in terms of: 1) composition processes, 2) objectives and limitations of the composition.

5.1. Mashup Examples

5.1.1. Geolocation Mashup: Web Service Interface Wrapper for Sensors

In Figure 8, the Geolocation Mashup application shows how the composition aids a device which lacks sensors, and which can indirectly use parameters from mobile phone applications via Interface Wrapper. The latitude and longitude parameters from GPSLocator mobile application are then used as a part of query strings of Wikinear and LocalWeather Web service. The output of this mashup application is configured as a Web application. The type of Interface Wrapper used in this mashup application is selected from case (b) in Table 2.

Figure 8. Model representation and screen shots of Geolocation Mashup

The MID file for this mashup application (see Figure 3) contains no active

component. Therefore, this application is only generated as a WAR file and is installed in the Web Files section of the mobile Web server.

In the process of generating a mashup application, the WAR file contains 5 class files. Each generated class file performs a specific task as follows:

a) GPSLocatorMAInfoAccess.class - accesses location’s latitude and longitude. b) GPSLocatorWSOutput.class - generates the JSON message of a) c) WikinearWSOutput.class - queries and generates outputs from the Wikinear Web

service using parameters from b). d) LocalWeatherWSOutput.class - queries and generates outputs from the

LocalWeather Web service using parameters parameters from b). e) ProjectWAOutput.class - generates outputs as a Web application using

information from c) and d). The ProjectWAOutput.class in e) is a Java Servlet which other devices can access

as a webpage. One can access this page using the mobile Web server hostname and the path specified in the MID file (e.g. http://[hostname]/location/mashup-output).

5.1.2. Book Reviews Barcode Reader Mashup: Web Content Extraction

Figure 9 demonstrates the composition of a mobile phone’s user input interface with external Web resources. The mashup application aids users by shortening the interaction sequences. These sequences may conventionally require users to copy and paste one output from one section for use in another section. The type of Interface Wrapper used in this mashup application is selected from case (a) in Table 2.

Figure 9. Model representation and screen shots of Book Reviews Barcode Reader Mashup

The objective is to compose a mashup application which can translate a product’s

barcode into a product name and then search for related information via Web resources, displaying the output on the mobile phone. The following must be considered prior to the configuration process:

• Active components such as the Barcode scanner component and the Web display component.

• Web application components which can use parameters generated from the Web extraction tool (i.e., www.amazon.com [3] and www.goodreads.com [6]).

First, composers use the Web extraction tool to generate parameters for their compositions, and then begin to configure each component in the composition. Composers are required to install the barcode reader application manually and composers need to know the Intent configurations of the barcode reader application.

In the process for generating this sort of mashup application, the APK file contains 2 class files concerning the access to user interface while the WAR file contains 3 class files. Each generated class file performs a specific task as follows:

a) BarcodeReaderUIAccess.class - accesses the barcode reader interface via Intent. b) AmazonWAExtractor.class - extracts the information from www.amazon.com [3]

using parameters from a). c) AmazonWSOutput.class - generates JSON message from b). d) GoodReadsWAExtractor.class - extracts information from www.goodreads.com

[6] using parameters from c). e) ProjectMAUIAccess.class - accesses the user interface on the mobile phone to

display output from d). In intended use, after users manually install the APK file and the WAR file to the

mobile phone, user can access the barcode scanner interface from the home screen. We discuss the performance of our Web content extraction method in Section 5.2.2.

5.1.3. Japanese-English Talking Dictionary Mashup: Active Components

Figure 10. Model representation and screen shots of Japanese-English Talking Dictionary Mashup

Figure 10 demonstrates the fewest possible number of active components a mashup application can contain. The Integration Model consists of one Interface Wrapper which transforms a Web application into a mobile application. The objective of this composition is to test the active Web application component, a component which works with the active mobile application component. The type of Interface Wrapper used in this mashup application is selected from case (c) in Table 2.

In the configuration process, first composers first use the Web extraction tool to generate parameters from ALC Japanese dictionary Web application [2] (e.g. www.alc.co.jp) and then create a Web application component. This active publisher component requires a user’s input. After the query in the publisher component has finished, the mashup application starts the TextToSpeech mobile application, reading the translated words in English.

The mashup application consists of 4 class files according to case (c) of Table 2. Each generated class file performs a specific task as follows:

a) ProjectWAUIAccess.class - provides a user interface which accesses the Web application www.alc.co.jp [2] for entering querying words.

b) ALCWAExtractor.class - extracts the information from www.alc.co.jp [2] using parameters from a).

c) TextToSpeechUIAccess.class: passes the extracted information from b) to the TextToSpeech mobile application and start the application.

This mashup application contains only an APK file (as demonstrated in Figure 7 using the model W [MA*[WA*]]). After the file is manually installed, the icon on the mobile phone’s menu works as an entry point to the mashup application.

5.2. Problems regarding composition and usage, related case studies, and proposed solutions

5.2.1. Geolocation Mashup: Web Service Interface Wrapper for Sensors

There are several problems found in the composition of the Geolocation Mashup. First, the composition output as a Web application can only be configured using strings and variables – this limitation provides less flexibility compared to other Web builder tools or dynamic Web programming languages. Since the Web service wrapper’s outputs provide a location’s latitude and longitude, those outputs can be accessed as a Web service. Composers may combine this Web service wrapper’s output with their HTML, JavaScript code or other dynamic Web programming language (instead of our provided template) to generate Web application output.

The second problem occurred when we applied the Web service Interface Wrapper to the GPSLocation mobile application to passively access the location’s latitude and longitude. In random cases, when the location’s parameters were accessed through this method they were not updated and the system’s GPS module had to be restarted. When we further applied the mashup application to passively access the accelerometer of the mobile phone, the parameters were also not automatically updated. However, in the case of the accelerometer, we found that there is a background process to retrieve these parameters. To solve this problem, the mobile application has to be reprogrammed to bring the process to the active state. Then our mashup application would be able to access the parameters correctly. The other solution is that we can periodically connect to the Web service output using JavaScript. In this way the connection provides active access to the parameters.

In sum, our method to access mobile phones’ sensors’ parameters via the Web service Interface Wrapper is bound by the limitations of the wrapped mobile phone applications. To passively retrieve sensors’ parameters, composers have to carefully test the mobile phone application to see how the parameters update themselves. It is recommended that the mashup applications access the parameters through a connection via the Web service Interface Wrapper. The access also provides compatibility with typical JavaScript and HTML code.

5.2.2. Book Reviews Barcode Reader Mashup: Web Content Extraction

In the configuration process of the mashup application presented in Example 5.1.2, composers are required to use our Web extraction tool. We use the tool to specify and generate parameters to point to one result from a result list. To show more results, more parameters have to be specified which causes composers to repeatedly use the Web extraction tool to generate more parameters. In general programming, programmers can write extraction loops; however, extraction loops is not presented in our method.

Our objective for this mobile mashup application was to use one result from the search result of the product barcode number. And we found that the product’s barcode number rarely matches more than one result. In the case where many results are preferred, it is recommended that the information that is extracted from the Web application is wrapped using the Web service Interface Wrapper so that composers can apply the wrapper’s outputs with typical programming.

Concerning the performance of reusing a non-API Web application (as presented in Example 5.1.2), to display book reviews provided by www.goodreads.com [6], a navigation through 3 Web pages is needed (i.e., search page, result list page, and

reviews page). The GoodReadsWAExtractor.class requires time to finish parsing the DOM tree in each page and yields a time lag due to the network latency and hardware constraints of mobile phones. Composers may confront more time lags when working with Web pages containing many contents.

Another problem found was that the information in the final resulting page is difficult to extract. We have applied the same method to retrieve the ESRB software ratings [5] using video games’ product barcodes. The essential information to retrieve from the final resulting page is in the form of a table with text. Composers can select the entire <table> tag or specify all portions to be taken.

To conclude, our method to perform Web extractions on mobile phones using description-based configurations requires more manual work than conventional programming does. The solution is to make the data flows diverge to parse the DOM tree remotely with other Web servers. In this case, we have to redesign the system architecture to be able to connect to external Web servers to do the configured tasks. Mobile phone hardware constraints and network latency also yields time lag when performing Web extraction. It is better that the complexity of the DOM tree is reduced before the parsing process begins.

5.2.3. Japanese-English Talking Dictionary Mashup: Active Components

We use a similar method to generate parameters for a Web application component using the Web extraction tool. The difference from the other example is that this Web application component is set as active and prompts for user input before proceeding to the next step. To support this active component, we derived the display screen of the active Web application component from the Android WebView. This WebView functions similarly to the mobile phone’s default browser. The WebView is configured to suppress all user interaction with the screen except for the selected input box. Later, the user submits words for a result which is passed to the TextToSpeech application.

We found that when this active Web application component is configured to use more than one input box, the display screen provided by WebView was quite confusing - users may not be able to see and use the selected two input boxes since the boxes are located far from each other. We propose that the WebView should be able hide the visual display of unrelated items. To do this, first, the Web extraction tool has to be modified to be able to generate parameters used in the display configuration. Second, the structure of MID file has to be changed to support these parameters. This proposed method may lower the domain of error concerning user interaction in the mobile mashup application (an application contains many active components).

6. General Discussion

In this study, our approach to creating mashup applications is to plan them as an Integration Model. Our model expands into real configurations and generates a mobile mashup application according to those configurations. The main difference between conventional composition of mobile mashup application using Java mobile programming is that, instead of programming code, we attempt to reuse the existent components by applying configuration via parameters. We discuss and compare two approaches as follows.

6.1. Composition Processes

Our composition process to create mashup applications starts from the composer’s plan for what components will be used. We limit the composition domain to 3 component categories. Composition of the components requires less API and documentation when compared to conventional programming. To begin composing a mashup application, composers may simply define a minimum of two Web resources and combine them as a mashup application using the MID file.

On the other hand, to begin Java programming, composers need to study and indicate what APIs used. To create even a single Web application component, the programming tasks ranged from visual display of a Web application, network programming to retrieve information from Web resources, and creation of other controls such as navigation buttons. Our approach aimed to reduce unnecessary programming work; we use only parameters to compose mashup applications. The approach also limits the domain of error which can occur in conventional mashup composition with mobile programming language.

Alternatively, composers can create Web service wrappers to share the output of mashup applications through mobile Web servers. This also provides flexibility to composers who are skilled at Web programming by allowing them to program or reuse mobile phone applications without the knowledge of mobile programming APIs.

6.2. Objectives and Limitations

Our approach to composing a mashup application was not aiming to build new user interfaces or new logical algorithms. Our goal of reusing parts of Web resources with mobile applications includes a substantial limitation to composing mashup applications. The combination domain is only limited to the content on the Web resources and the existent mobile applications. For instance, creating a new camera application to detect and translate specific sign language into other representations cannot be accomplished with our approach. Such work still requires low-level implementation using mobile programming language.

Moreover, our approach does not provide logic or mathematic functions within data flows. In our composition method, composers are required to add more mashup components to perform logic or mathematical calculations. For instance, to calculate the distance between the user’s current position (latitude and longitude) and the closest Wikipedia article based on the article’s GPS coordinates. As shown in Example 5.1.1, composers have to work through these following tasks: search for a distance calculator Web application which uses two pairs of coordinates, generate parameters from the Web application, configure the MID file, and generate the entire mashup application again. In conventional programming, by adding only a few lines of mathematical calculation to the proper part of the program, the calculation can be performed.

The constraints of mobile phone hardware are also a significant limitation to our approach. When the system is running many processes (e.g., a mobile Web server module, the Web information extractor classes and accesses to sensors’ parameter), it tends to run slower as many background processes share memory resources. The drawback is also caused by the use of many classes connecting mashup components together. We believe that a specific system resource manager for running mashup application is needed.

7. Conclusion and Future Work

In this study we presented a description-based integration approach to aid the composition of mobile mashup applications. We used the Integration Model and Interface Wrappers in the planning process. Later we expand the model into the actual configuration for generating a mobile mashup application for placement in the mobile Web server system. Our approach allows composers to access mobile phone sensors’ parameters and to create mobile mashup applications with less programming effort, using parameters to describe the data flows. We composed three mashup applications to demonstrate how actual configuration works. The example compositions have many limitations that need to be considered and resolved in the future. Our future work includes the research area of Web extraction performance, mashup composition flexibility, and performance improvement of the mobile Web Server.

8. Reference

[1] Boonlit Adipat and Dongsong Zhang. Adaptive and Personalized Interfaces for Mobile Web, Proceedings of the 15th Annual Workshop on Information Technologies and Systems, 2005.

[2] ALC. Http://www.alc.co.jp/ [3] Amazon. Http://www.amazon.com/ [4] Nilton Bila, Troy Ronda, Iqbal Mohomed, Khai N. Truong and Eyal de Lara. PageTailor: Reusable

End-User Customization for the Mobile Web. In Proceedings of the 5th International Conference on Mobile Systems, Applications and Services, 2007.

[5] Entertainment Software Rating Board. Http://www.esrb.org/ [6] Goodreads. Http://www.goodreads.com/ [7] Junxia Guo, Hao Han and Takehiro Tokuda, A New Partial Information Extraction Method for Personal

Mashup construction, Proceedings of the 19th European - Japanese Conference on Information Modeling and Knowledge Bases, 2009.

[8] Hao Han and Takehiro Tokuda, A Method for Integration of Web Applications Based on Information Extraction, Proceedings of the 8th International Conference on Web Engineering, 2008.

[9] I-jetty. Http://code.google.com/p/i-jetty/ [10] Christian S. Jensen, Carmen Ruiz Vicente, and Rico Wind, User-Generated Content: The Case for

Mobile Services. IEEE Computer 41 (2008), 116–118. [11] Yung-Wei Kaoa, Tzu-Han Kaoa, Chi-Yang Tsaia and Shyan-Ming Yuana, A personal Web page

tailoring toolkit for mobile devices, Computer Standards & Interfaces 31 (2009), 437-453.   [12] Kouichi Katsurada, Yusaku Nakamura, Hirobumi Yamada and Tsuneo Nitta, XISL: A Language for

Describing Multimodal Interaction Scenarios, Proceedings of the 5th International Conference on Multimodal Interfaces, 2003.

[13]  Kris Luyten and Karin Coninx, An XML-based runtime user interface description language for mobile computing devices, Proceedings of the 8th International Workshop on Design, Specification, and Verification of Interactive Systems, 2001.

[14] mFoundry mojax. Http://www.mfoundry.com/platform.html [15] Nathalie Souchon and Jean Vanderdonckt, A Review of XML-compliant User Interface Description

Languages, Proceedings of 10th International Conference on Design, Specification, and Verification of Interactive Systems, 2003.

[16] Peter Rigole, Chris Vandervelpen, Kris Luyten, Karin Coninx, Yolande Berbers and Yves Vandewoude, A Component-Based Infrastructure for Pervasive User Interaction, Proceedings of Software Techniques for Embedded and Pervasive Systems, International Workshop on Software Techniques for Embedded and Pervasive Systems, 2005.

[17] Xueshan Shan, A Presence-enabled Mobile Service System for Integrating Mobile Devices with Enterprise Collaborative Environment, Proceedings of the International Workshop on Wireless Ad-hoc Networks, 2005.

[18] ServerMans. Http://serversman.com/index_en.jsp [19] Roy Want , Trevor Pering , Gunner Danneels , Muthu Kumar , Murali Sundar and John Light, The

Personal Server: Changing the Way We Think about Ubiquitous Computing, Proceedings of the 4th International Conference on Ubiquitous Computing, 2002.