Proceedings of the ACL-HLT 2011 System Demonstrations, pages 20–25,Portland, Oregon, USA, 21 June 2011. c©2011 Association for Computational Linguistics

A Mobile Touchable Application for Online Topic Graph Extraction andExploration of Web Content

Gunter Neumann and Sven SchmeierLanguage Technology Lab, DFKI GmbH

Stuhlsatzenhausweg 3, D-66123 Saarbrucken{neumann|schmeier}@dfki.de

Abstract

We present a mobile touchable application foronline topic graph extraction and explorationof web content. The system has been imple-mented for operation on an iPad. The topicgraph is constructed from N web snippetswhich are determined by a standard search en-gine. We consider the extraction of a topicgraph as a specific empirical collocation ex-traction task where collocations are extractedbetween chunks. Our measure of associationstrength is based on the pointwise mutual in-formation between chunk pairs which explic-itly takes their distance into account. An ini-tial user evaluation shows that this system isespecially helpful for finding new interestinginformation on topics about which the user hasonly a vague idea or even no idea at all.

1 Introduction

Today’s Web search is still dominated by a docu-ment perspective: a user enters one or more key-words that represent the information of interest andreceives a ranked list of documents. This technologyhas been shown to be very successful when used onan ordinary computer, because it very often deliversconcrete documents or web pages that contain theinformation the user is interested in. The followingaspects are important in this context: 1) Users basi-cally have to know what they are looking for. 2) Thedocuments serve as answers to user queries. 3) Eachdocument in the ranked list is considered indepen-dently.

If the user only has a vague idea of the informa-tion in question or just wants to explore the infor-

mation space, the current search engine paradigmdoes not provide enough assistance for these kindof searches. The user has to read through the docu-ments and then eventually reformulate the query inorder to find new information. This can be a tedioustask especially on mobile devices. Seen in this con-text, current search engines seem to be best suitedfor “one-shot search” and do not support content-oriented interaction.

In order to overcome this restricted document per-spective, and to provide a mobile device searches to“find out about something”, we want to help userswith the web content exploration process in twoways:

1. We consider a user query as a specification ofa topic that the user wants to know and learnmore about. Hence, the search result is basi-cally a graphical structure of the topic and as-sociated topics that are found.

2. The user can interactively explore this topicgraph using a simple and intuitive touchableuser interface in order to either learn moreabout the content of a topic or to interactivelyexpand a topic with newly computed relatedtopics.

In the first step, the topic graph is computed onthe fly from the a set of web snippets that has beencollected by a standard search engine using the ini-tial user query. Rather than considering each snip-pet in isolation, all snippets are collected into onedocument from which the topic graph is computed.We consider each topic as an entity, and the edges

20

between topics are considered as a kind of (hidden)relationship between the connected topics. The con-tent of a topic are the set of snippets it has been ex-tracted from, and the documents retrievable via thesnippets’ web links.

A topic graph is then displayed on a mobile de-vice (in our case an iPad) as a touch-sensitive graph.By just touching on a node, the user can either in-spect the content of a topic (i.e, the snippets or webpages) or activate the expansion of the graph throughan on the fly computation of new related topics forthe selected node.

In a second step, we provide additional back-ground knowledge on the topic which consists of ex-plicit relationships that are generated from an onlineEncyclopedia (in our case Wikipedia). The relevantbackground relation graph is also represented as atouchable graph in the same way as a topic graph.The major difference is that the edges are actuallylabeled with the specific relation that exists betweenthe nodes.

In this way the user can explore in an uniform wayboth new information nuggets and validated back-ground information nuggets interactively. Fig. 1summarizes the main components and the informa-tion flow.

Figure 1: Blueprint of the proposed system.

2 Touchable User Interface: Examples

The following screenshots show some results for thesearch query “Justin Bieber” running on the cur-

rent iPad demo–app. At the bottom of the iPadscreen, the user can select whether to perform textexploration from the Web (via button labeled “i–GNSSMM”) or via Wikipedia (touching button “i–MILREX”). The Figures 2, 3, 4, 5 show results forthe “i–GNSSMM” mode, and Fig. 6 for the “i-MILREX” mode. General settings of the iPad demo-app can easily be changed. Current settings allowe.g., language selection (so far, English and Germanare supported) or selection of the maximum numberof snippets to be retrieved for each query. The otherparameters mainly affect the display structure of thetopic graph.

Figure 2: The topic graph computed from the snippets forthe query “Justin Bieber”. The user can double touch ona node to display the associated snippets and web pages.Since a topic graph can be very large, not all nodes aredisplayed. Nodes, which can be expanded are marked bythe number of hidden immediate nodes. A single touchon such a node expands it, as shown in Fig. 3. A singletouch on a node that cannot be expanded adds its label tothe initial user query and triggers a new search with thatexpanded query.

21

Figure 3: The topic graph from Fig. 2 has been expandedby a single touch on the node labeled “selena gomez”.Double touching on that node triggers the display of as-sociated web snippets (Fig. 4) and the web pages (Fig.5).

3 Topic Graph Extraction

We consider the extraction of a topic graph as a spe-cific empirical collocation extraction task. How-ever, instead of extracting collations between words,which is still the dominating approach in collocationextraction research, e.g., (Baroni and Evert, 2008),we are extracting collocations between chunks, i.e.,word sequences. Furthermore, our measure of asso-ciation strength takes into account the distance be-tween chunks and combines it with the PMI (point-wise mutual information) approach (Turney, 2001).

The core idea is to compute a set of chunk–pair–distance elements for the N first web snip-pets returned by a search engine for the topic Q,and to compute the topic graph from these ele-ments.1 In general for two chunks, a single chunk–pair–distance element stores the distance between

1For the remainder of the paper N=1000. We are using Bing(http://www.bing.com/) for Web search.

Figure 4: The snippets that are associated with the nodelabel “selena gomez” of the topic graph from Fig. 3.In or-der to go back to the topic graph, the user simply touchesthe button labeled i-GNSSMM on the left upper corner ofthe iPad screen.

the chunks by counting the number of chunks in–between them. We distinguish elements which havethe same words in the same order, but have differentdistances. For example, (Peter, Mary, 3) is differentfrom (Peter, Mary, 5) and (Mary, Peter, 3).

We begin by creating a document S from theN -first web snippets so that each line of S con-tains a complete snippet. Each textline of S isthen tagged with Part–of–Speech using the SVM-Tagger (Gimenez and Marquez, 2004) and chun-ked in the next step. The chunker recognizes twotypes of word chains. Each chain consists of longestmatching sequences of words with the same PoSclass, namely noun chains or verb chains, wherean element of a noun chain belongs to one ofthe extended noun tags2, and elements of a verb

2Concerning the English PoS tags, “word/PoS” expressionsthat match the following regular expression are considered asextended noun tag: “/(N(N|P))|/VB(N|G)|/IN|/DT”. The En-

22

Figure 5: The web page associated with the first snippetof Fig. 4. A single touch on that snippet triggers a callto the iPad browser in order to display the correspondingweb page. The left upper corner button labeled “Snip-pets” has to be touched in order to go back to the snippetspage.

chain only contains verb tags. We finally ap-ply a kind of “phrasal head test” on each iden-tified chunk to guarantee that the right–most ele-ment only belongs to a proper noun or verb tag.For example, the chunk “a/DT british/NNP for-mula/NNP one/NN racing/VBG driver/NN from/INscotland/NNP” would be accepted as proper NPchunk, where “compelling/VBG power/NN of/IN”is not.

Performing this sort of shallow chunking is basedon the assumptions: 1) noun groups can representthe arguments of a relation, a verb group the relationitself, and 2) web snippet chunking needs highly ro-bust NL technologies. In general, chunking cruciallydepends on the quality of the embedded PoS–tagger.However, it is known that PoS–tagging performanceof even the best taggers decreases substantially when

glish Verbs are those whose PoS tag start with VB. We are us-ing the tag sets from the Penn treebank (English) and the Negratreebank (German).

Figure 6: If mode “i–MILREX” is chosen then text ex-ploration is performed based on relations computed fromthe info–boxes extracted from Wikipedia. The centralnode corresponds to the query. The outer nodes repre-sent the arguments and the inner nodes the predicate of ainfo–box relation. The center of the graph corresponds tothe search query.

applied on web pages (Giesbrecht and Evert, 2009).Web snippets are even harder to process becausethey are not necessary contiguous pieces of texts,and usually are not syntactically well-formed para-graphs due to some intentionally introduced breaks(e.g., denoted by . . . betweens text fragments). Onthe other hand, we want to benefit from PoS tag-ging during chunk recognition in order to be able toidentify, on the fly, a shallow phrase structure in websnippets with minimal efforts.

The chunk–pair–distance model is computedfrom the list of chunks. This is done by traversingthe chunks from left to right. For each chunk ci, aset is computed by considering all remaining chunksand their distance to ci, i.e., (ci, ci+1, disti(i+1)),(ci, ci+2, disti(i+2)), etc. We do this for each chunklist computed for each web snippet. The distancedistij of two chunks ci and cj is computed directlyfrom the chunk list, i.e., we do not count the position

23

of ignored words lying between two chunks.The motivation for using chunk–pair–distance

statistics is the assumption that the strength of hid-den relationships between chunks can be covered bymeans of their collocation degree and the frequencyof their relative positions in sentences extracted fromweb snippets; cf. (Figueroa and Neumann, 2006)who demonstrated the effectiveness of this hypothe-sis for web–based question answering.

Finally, we compute the frequencies of eachchunk, each chunk pair, and each chunk pair dis-tance. The set of all these frequencies establishesthe chunk–pair–distance model CPDM . It is usedfor constructing the topic graph in the final step. For-mally, a topic graph TG = (V,E, A) consists of aset V of nodes, a set E of edges, and a set A of nodeactions. Each node v ∈ V represents a chunk andis labeled with the corresponding PoS–tagged wordgroup. Node actions are used to trigger additionalprocessing, e.g., displaying the snippets, expandingthe graph etc.

The nodes and edges are computed from thechunk–pair–distance elements. Since, the numberof these elements is quite large (up to severalthousands), the elements are ranked according toa weighting scheme which takes into account thefrequency information of the chunks and their collo-cations. More precisely, the weight of a chunk–pair–distance element cpd = (ci, cj , Dij), with Di,j ={(freq1, dist1), (freq2, dist2), ..., (freqn, distn)},is computed based on PMI as follows:

PMI(cpd) = log2((p(ci, cj)/(p(ci) ∗ p(cj)))

= log2(p(ci, cj))− log2(p(ci) ∗ p(cj))

where relative frequency is used for approximatingthe probabilities p(ci) and p(cj). For log2(p(ci, cj))we took the (unsigned) polynomials of the corre-sponding Taylor series3 using (freqk, distk) in thek-th Taylor polynomial and adding them up:

PMI(cpd) = (n∑

k=1

(xk)k

k)− log2(p(ci) ∗ p(cj))

, where xk =freqk∑n

k=1 freqk

3In fact we used the polynomials of the Taylor series forln(1 + x). Note also that k is actually restricted by the numberof chunks in a snippet.

The visualized topic graph TG is then computedfrom a subset CPD′

M ⊂ CPDM using the m high-est ranked cpd for fixed ci. In other words, we re-strict the complexity of a TG by restricting the num-ber of edges connected to a node.

4 Wikipedia’s Infoboxes

In order to provide query specific backgroundknowledge we make use of Wikipedia’s infoboxes.These infoboxes contain facts and important rela-tionships related to articles. We also tested DB-pedia as a background source (Bizer et al., 2009).However, it turned out that currently it containstoo much and redundant information. For exam-ple, the Wikipedia infobox for Justin Bieber containseleven basic relations whereas DBpedia has fifty re-lations containing lots of redundancies. In our cur-rent prototype, we followed a straightforward ap-proach for extracting infobox relations: We down-loaded a snapshot of the whole English Wikipediadatabase (images excluded), extracted the infoboxesfor all articles if available and built a Lucene Indexrunning on our server. We ended up with 1.124.076infoboxes representing more than 2 million differ-ent searchable titles. The average access time isabout 0.5 seconds. Currently, we only support ex-act matches between the user’s query and an infoboxtitle in order to avoid ambiguities. We plan to ex-tend our user interface so that the user may choosedifferent options. Furthermore we need to find tech-niques to cope with undesired or redundant informa-tion (see above). This extension is not only neededfor partial matches but also when opening the sys-tem to other knowledgesources like DBpedia, new-sticker, stock information and more.

5 Evaluation

For an initial evaluation we had 20 testers: 7 camefrom our lab and 13 from non–computer science re-lated fields. 15 persons had never used an iPad be-fore. After a brief introduction to our system (andthe iPad), the testers were asked to perform threedifferent searches (using Google, i–GNSSMM andi–MILREX) by choosing the queries from a set often themes. The queries covered definition ques-tions like EEUU and NLF, questions about personslike Justin Bieber, David Beckham, Pete Best, Clark

24

Kent, and Wendy Carlos , and general themes likeBrisbane, Balancity, and Adidas. The task wasnot only to get answers on questions like “Who is. . . ” or “What is . . . ” but also to acquire knowledgeabout background facts, news, rumors (gossip) andmore interesting facts that come into mind duringthe search. Half of the testers were asked to firstuse Google and then our system in order to comparethe results and the usage on the mobile device. Wehoped to get feedback concerning the usability ofour approach compared to the well known internetsearch paradigm. The second half of the participantsused only our system. Here our research focus wasto get information on user satisfaction of the searchresults. After each task, both testers had to rate sev-eral statements on a Likert scale and a general ques-tionnaire had to be filled out after completing theentire test. Table 1 and 2 show the overall result.

Table 1: Google

#Question v.good good avg. poorresults first sight 55% 40% 15% -query answered 71% 29% - -interesting facts 33% 33% 33% -suprising facts 33% - - 66%

overall feeling 33% 50% 17% 4%

Table 2: i-GNSSMM#Question v.good good avg. poor

results first sight 43% 38% 20% -query answered 65% 20% 15% -interesting facts 62% 24% 10% 4%suprising facts 66% 15% 13% 6%

overall feeling 54% 28% 14% 4%

The results show that people in general preferthe result representation and accuracy in the Googlestyle. Especially for the general themes the presen-tation of web snippets is more convenient and moreeasy to understand. However when it comes to in-teresting and suprising facts users enjoyed exploringthe results using the topic graph. The overall feelingwas in favor of our system which might also be dueto the fact that it is new and somewhat more playful.

The replies to the final questions: How success-

ful were you from your point of view? What did youlike most/least? What could be improved? were in-formative and contained positive feedback. Usersfelt they had been successful using the system. Theyliked the paradigm of the explorative search on theiPad and preferred touching the graph instead of re-formulating their queries. The presentation of back-ground facts in i–MILREX was highly appreciated.However some users complained that the topic graphbecame confusing after expanding more than threenodes. As a result, in future versions of our system,we will automatically collapse nodes with higherdistances from the node in focus. Although all of ourtest persons make use of standard search engines,most of them can imagine to using our system atleast in combination with a search engine even ontheir own personal computers.

6 Acknowledgments

The presented work was partially supported bygrants from the German Federal Ministry of Eco-nomics and Technology (BMWi) to the DFKI The-seus projects (FKZ: 01MQ07016) TechWatch–Ordoand Alexandria4Media.

ReferencesMarco Baroni and Stefan Evert. 2008. Statistical meth-

ods for corpus exploitation. In A. Ludeling andM. Kyto (eds.), Corpus Linguistics. An InternationalHandbook, Mouton de Gruyter, Berlin.

Christian Bizer, Jens Lehmann, Georgi Kobilarov, SorenAuer, Christian Becker, Richard Cyganiak, SebastianHellmann. 2009. DBpedia - A crystallization point forthe Web of Data. Web Semantics: Science, Servicesand Agents on the World Wide Web 7 (3): 154165.

Alejandro Figueroa and Gunter Neumann. 2006. Lan-guage Independent Answer Prediction from the Web.In proceedings of the 5th FinTAL, Finland.

Eugenie Giesbrecht and Stefan Evert. 2009. Part-of-speech tagging - a solved task? An evaluation of PoStaggers for the Web as corpus. In proceedings of the5th Web as Corpus Workshop, San Sebastian, Spain.

Jesus Gimenez and Lluıs Marquez. 2004. SVMTool: Ageneral PoS tagger generator based on Support VectorMachines. In proceedings of LREC’04, Lisbon, Por-tugal.

Peter Turney. 2001. Mining the web for synonyms: PMI-IR versus LSA on TOEFL. In proceedings of the 12thECML, Freiburg, Germany.

25