Discovering User Interests from Web Browsing Behavior: An Application to Internet News Services

Proceedings of the 35th Hawaii International Conference on System Sciences - 2002

Proceedings of the 35th Annual Hawaii0-7695-1435-9/02 $17.00 © 2002 IEEE

Discovering User Interests from Web Browsing Behavior: An Application to Internet News Services

Ting-Peng Liang and Hung-Jen Lai

Department of Information Management National Sun Yat-sen University

Kaohsiung, Taiwan Email: [email protected]

Abstract Discovering user interests is a very important task for providing personalized services in electronic commerce. A popular approach is to develop customer profiles from their browsing behavior. In this paper, we present an approach that analyzes the browsing content and time to determine user interests. An empirical study using actual news provided by the China Times shows that the proposed system outperforms the traditional headline news compiled by the news editor in both objective performance indices and customer satisfaction. 1. Introduction

The rapid propagation of the Internet,

along with the evolution of information technologies, has changed the nature of many businesses. The large amount of transactional data collected from the use of information systems allows a company to better understand customer needs and to integrate the knowledge into their product design and marketing plans. For physical products (e.g., computers and televisions), mass customization and fast response to market needs become critical to remaining competitive. For digital products and services (e.g., news services and other Internet content providers, ICP), personalized services that offer the tailored content to different clients based on their interests become feasible and necessary.

In this paper, we propose an approach that builds customer profiles from their browsing behavior recorded by the computer and recommends personal services delivered on the web based on the profiles. The approach is then applied to the Internet news services to evaluate its applicability. The news recommendation system includes components for news structure analysis, customer profile analysis, and personal recommendation mechanism. An empirical study was performed to evaluate the proposed approach.

The remainder of the paper is organized as follows. First, literature related to information

1

0-7695-1435-9/02 $1 International Conference on System Sciences (HICSS-35�02)

filtering and recommendation will be reviewed briefly. This is followed by the presentation of our approach and its application in Internet news services. Section 4 presents the experimental design. Findings are shown in Section 5. Section 6 concludes the paper.

2. Literature Review

The wide spread of the Internet has created

an efficient channel for information dissemination. The information overload, however, becomes a problem. How to reduce unnecessary information and provide customized services becomes an important issue.

A few filtering mechanisms have been proposed in the past. A typical one is to ask the reader to report his interest after reading. The system can then build a profile of the reader and make recommendation accordingly. For example, Mock and Vemuri [10] presented the Intelligent News Filtering Organization System (INFOS) that asked each reader to indicate whether he liked the report. The system reorganizes the order of news based on the revealed preference. Results from a pilot test show that INFOS can effectively reduce the reader’s search load.

Another approach is behavior-based. For instance, Sakagami and Kamba [17] developed the ANATAGONOMY that learns reading preference from the browsing behavior (e.g., scroll, enlarge windows, etc.) of a user. The system has a learning engine and a scoring engine to produce personalized web news.

Information filtering and recommendation can also be performed based on feedback from others. For instance, Konstan, et al. [5] proposed a system, called GroupLens, which summarized the feedback from previous readers to allow the next reader to determine whether to read it. This is called collaborative filtering. A system proposed by Balabanovic and Shoham [1] combines content analysis and collaborative filtering. It takes into account the association between a reader and the theme of a report to identify the discrepancy between different individuals.

7.00 (c) 2002 IEEE 1


Proceedings of the 35th Annu0-7695-1435-9/02 $17.00 © 2

Rucker and Polanco [16] proposed a system that analyzes the structure of bookmarks to determine the interests of an individual. In fact, bookmarks reveal not only the person’s interest but also the way information is organized. 3. A Time-based Approach to User

Profiling The key to information filtering and

recommendation is user profiling. In general, user profiles can be obtained from self-reporting or analysis of browsing behavior. Although self-reporting may be considered more accurate in some cases, it is often tedious and difficult to deal with dynamic changes. Therefore, much research has focused on identifying user interests from the browsing data collected on-line. In this section, we present a time-based approach that determines user interests based on the time they spent viewing objects with known attributes. The underlying assumption of the method is that the more an object contains the information of interest to a user, the longer the user would view the object. Because errors may exist when the browsing time is too long or too short, we use the average reading speed and recency weight to adjust the interest level. The method can be described briefly in the following: The reader profile is represented as a combination of attributes and interest levels. The interest level of a particular attribute is determined by the previous time spending on browsing items having the attribute and may be adjusted by recency and other factors. Therefore, given an object Rj[Aij(pij)], the approach is defined as follows: Definition 1: Interest level of an object The interest level of an object is an indicator of the extent to which a user is interested in the object. The interest level is calculated by the following equation: σj(Rj) = f(Tj/Tj

*), Where: σi(Rj) is the interest level of the

object, Rj; Tj is the time spent by the user on reading

Rj; T0 <=Tj <=Tu, T0 is the lower bound for a browsing time to be considered reasonable, and Tu is the upper bound for a browsing time to be considered reasonable.

Tj* is the estimated reasonable reading

time based on previous average reading speed; f is a function that calculates the interest

level. It may be a linear or a sigmoid function. Definition 2: Interest level of an attribute The interest level of an attribute is the aggregation of the interest levels of objects that have the attribute. It is calculated as follows: σi = Σj[σj(Rj)*Aij(pij)], where σi is the interest level of attribute Ai;

2

0-7695-1435-9/02 $1al Hawaii International Conference on System Sciences (HICSS-35�02)

002 IEEE

Aij(pij) shows that the likelihood of object Rj having the attribute Ai is pij. Definition 3: Recency adjustment The recency adjustment is to give higher weights to objects accessed recently than those accessed earlier. The recency weight of an object can be calculated by the following equation: γj(Rj) = g(Dj),

Where: γi(Rj) is the recency weight of the object, Rj;

Dj is the elapse day of reading Rj; D0 <=Dj <=Du, D0 is the lower bound for a elapse day to be considered for adjustment, and Du is the upper bound for an elapse day to be considered for adjustment.

g is a function that calculates the recency weight. It may be a linear or a sigmoid function. Definition 4: Adjusted Interest level of an attribute If the interest level needs to be adjusted by the recency weight, then the interest level becomes: σi = Σj[σj(Rj)* γi(Rj)*Aij(pij)]. Definition 5: User profile The profile of a user is a combination of object attributes and their associated interest levels. It can be represented as: U([Ai(σi)], where U is a user, Ai is a set of attribute. 4. Application to Personal News

Recommendation

In this section, an application of the time-based mechanism to personal news recommendation over the Internet is described. News services are popular because the Internet provides an efficient way for news distribution. It can also be personalized at a very low cost. Therefore, it is an excellent domain for testing the method. Since each news report contains certain characters, the system needs a module to determine the attributes of the content that are of interest. Therefore, four modules are essential: structure analysis, reader profile analysis, rating for recommendation, and learning (as shown in Figure 1). 4.1 Structure analysis

The first step for personalized news service is to analyze the product, i.e., the structure of the news. The foundation of structure analysis is to identify keywords in a report and to build the keyword dictionary. Keywords in each report are identified based on the property of the words. Miller [9] developed a comprehensive keyword dictionary (called WordNet) that includes many nouns and verbs. McQuail [8] points out that keywords are those

7.00 (c) 2002 IEEE 2


Proceedings of the 35th Ann0-7695-1435-9/02 $17.00 ©

words associated with who, what, where, when, and why. Names mentioned in a report are also

important keywords [14,15,19]].

ual Ha2002 IE

Figure 1. Architecture of the News Recommendation System

News

Browsing Behavior

Structure Analysis

Reader Profile

Rating Recommended

News

Threshold Value

Learning

In our mechanism, we use nouns with a

particular emphasis on the role and issues mentioned in the report. In the following “state of the web” report, for example, the keywords marked in italics can be identified.

State of the Web: Inside.com Says 'No

Thanks' to Yahoo!

Could this be the beginning of the end of the

portal strategy as we know it?

By James J. Cramer

You might not have noticed that

Inside.com isn't on Yahoo! (Nasdaq: YHOO

- news) anymore. You may never even have

heard of Inside.com. But you have to

understand that this lone decision, by Steve Brill, the head of Inside.com, is sending

shock waves throughout the portal world.

Here's why.

When the Web first started to be

commercial, outfits like Yahoo! and America Online needed to have content to wrap

around their ads. They first tried to grow it

and pay for it. Then, an epiphany struck Bob Pittman at AOL: Content providers needed eyeballs so badly that they would pay AOL

to be there! That shift in strategy was the

death knell for almost all original content providers on the Web because if you didn't

have money, you couldn't pay, and the only

people who could pay were established players and players that tapped the public

markets. (Source: http://www.yahoo.com/)

After identifying keywords, we further

analyze the position and frequency of keywords. Major steps include:

(1) Determine whether synonyms exist, including American Online = AOL, President Clinton = Bill Clinton, Yahoo = YHOO, etc..

(2) Calculate the frequency of keywords. In the previous example, the word

3

0-7695-1435-9/02 $17waii International Conference on System Sciences (HICSS-35�02) EE

Inside.com and Yahoo appear four times and AOL appears three times.

(3) Position adjustment: Since the title and first paragraph often contain more important information in a report, the keywords that appeared in the title are multiplied by 10 and those that appeared in the first paragraph are multiplied by 3. As a result, the adjusted frequency of Inside.com is 19, while Yahoo is 17, and AOL is 3. This significantly differentiates the relative importance of different keywords in the report.

(4) Noise elimination: To simplify the structure, minimum frequency may be set to remove unimportant words. For instance, we may set a rule that keywords whose adjusted frequency is lower than 20% of the most important keyword are removed. Then, only keywords whose adjusted frequencies are higher than 4 are considered valid and will be recorded to represent the structure of the report.

(5) Conversion to ratios: All keyword frequencies are then converted into a ratio, which is the frequency of a keyword/sum(frequencies of all keywords). The structure of the report is the collection of valid keywords along with their respective frequency ratio. The structure of the above example is [Inside.com (.31), Yahoo (.28), portal (.21), web (.20)].

4.2 Analysis of Reader Profile

Based on the algorithm specified in Section 3, we can analyze the interest profile of a user in the following procedures:

(1) Calculate the average reading speed of the user: The computer keeps a record of the time a user read a report. These data are aggregated and

.00 (c) 2002 IEEE 3


Proceedings of the 35th Annual Haw0-7695-1435-9/02 $17.00 © 2002 IEE

adjusted by the length of the reports. The average reading speed is calculated by dividing the total number of words by the total reading time.

(2) Calculate the interest level: The interest level of a report is calculated by the time spent in reading it. Interest level is represented by the ratio of dividing the actual reading time by the estimated reading time, where the estimated reading time = total words in the report/average reading speed. A mapping table is built to determine the interest levels. In our system, we set a range between 3 and 250 seconds as the reasonable range for reading a news report. The incidents outside this range are considered exceptional and are assigned an interest level of 0. If the time ratio of a reading is below .25 (i.e., the actual reading time is 25% of the estimated reading time), the case is considered a fast browsing and is assigned an interest level of 1. The ratio between .25 and .75 is assigned the value of 2, between .75 and 1.25 is assigned 3, between 1.25 and 1.75 is assigned 4, and above 1.75 is assigned 5.

(3) Conduct recency adjustment: Since we can reasonably assume that reports read recently can more accurately reflect a reader’s interest, the system gives a weight of 2 to reports that were read within D1 days, 1.75 to reports read between D1 and D2 days, 1.5 to those read between D2 and D3, and 1 to those longer than D3.

(4) Calculate the adjusted interest level of a reader on a report is the product of the interest level from step (3) multiplied by the recency weight result from step (4).

(5) Build the profile: The interest profile of a user is to multiply news structure by the interest level. For example, suppose a user has read two reports that have the following structure and his interest levels of the reports A001 and A015 are 4 and 2 respectively, then the resulting interest profile is [Web (2.0), Yahoo (1.72), Inside.com (1.24), Portal (.84), Merger (.20)]. This indicates that the user is most interested in reports related to web, followed by Yahoo, Inside.com, Portal, and Merge.

[Example] the following are structures of

2

0-7695-1435-9/02 $1aii International Conference on System Sciences (HICSS-35�02) E

two reports that had been read by a user: [A001: Inside.com (.31), Yahoo (.28), Portal (.21), Web (.20)]. [A015: Web (.60), Yahoo (.30), Merger (.10)]

4.3 Rating and Recommendation Rating and recommendation determine the matching between a new report and a reader. If the matching level is higher than the threshold value, the report will be recommended to the reader. Otherwise, it is dropped. Steps for matching reports with a reader include:

(1) Determine the structure of the report. For example, we have a new report A032, whose structure is [A032: Portal (0.7), Merger (0.3)].

(2) Calculate the matching level: The matching level is calculated by aggregating the interest levels of different keywords. In the example, the matching level is 0.648 (=0.7*0.84+0.3*0.2*).

(3) Recommend news based on matching levels: A hurdle can be set to screen out reports with low matching levels. The reports whose matching levels are higher than the threshold value will be recommended. In this step, guidelines on the number of news recommended and distribution of news among different categories can be used to enhance the accuracy of recommendation.

4.4 Learning The learning module is designed to adjust various weights. It is not the focus of the paper and hence is omitted. 5. Empirical Study

In order to evaluation the news

recommendation mechanism, an experimental study was performed. The benchmarks were the regular headline approach (HLA) and the self-reported interests (SRI) approach. Prototype systems that present news by HLA, SRI, and browsing behavior analysis (BBA) approach were developed for the experiment.

7.00 (c) 2002 IEEE 4


Proceedings of the 35th Ann0-7695-1435-9/02 $17.00 ©

5.1 Experimental variables and

hypotheses The independent variable of the study is

different news recommendation mechanisms, which include BBA, SRI, and the traditional HLA. The dependent variable is the evaluation of the systems, which include objective performance measures and subjective user satisfaction. They will be described in Section 5.3.

According to the framework, we would like to examine whether news recommendation mechanisms perform better than the traditional HLA, and whether BBA outperforms SRI. The null hypotheses are:

H1: BBA and SRI perform equally well with HLA,

H2: BBA performs equally well with SRI.

5.2 Experimental Systems Three experimental websites that provide

news services were designed. The HLA system copies the regular headline news approach. That is, the home page outlines the titles of the headline news that were determined by the editor. Other news is organized into categories with the category names (such as sport, stock, etc.) shown on the homepage. The reader has to click a category name to get into the second level web pages in order to read the titles within the category. The design of the SRI and BBA systems is the same except that their home pages are customized for each individual reader. When a user logs into the SRI or BBA system, the computer identifies his reading interests from the user profile and then composes the recommended news into the homepage. Those that are not recommended remain in the second level, i.e., their category names are shown on the homepage but the reader needs to click on the category name to see news titles.

All systems have automated recording module that keeps track of the following data: (1) which news the subject click to read, (2) time spent in reading a particular news, (3) time spent in choosing a news to read (i.e., the time that the subject logs on the system but not for reading the news).

5.3 Measurement Instruments (1) Background questionnaire

The background questionnaire collects data about the subject. It includes 11 questions: (1) most interesting category of news (a total of 13 categories), (2) least interesting category of news, (3) if you have 20 minutes, would you

3

0-7695-1435-9/02 $1ual Hawaii International Conference on System Sciences (HICSS-35�02)

2002 IEEE

browse titles or read news in a news website, (4) frequency of using news websites in a week, (5) average time spent on a news website in a visit, (6) which is your most common news source (print media, TV, or web), (7) experiences in using the web, (8) motivation for using news websites, (9) importance of receiving news and information to you, (10) preference in filtering news by the system, (11) choosing news by interests or location of the title.

(2) Objective performance measures

Both objective performance and user satisfaction of the subjects were measured. Two indices are common for measuring objective performance: precision and recall [18]. Precision measures the portion of recommended news that is relevant (i.e., read by the subject) and recall measures the portion of relevant news that is recommended. To explore further detail of the performance, five indices have been developed for the experiment: acceptance rate, hit rate by the number of news, hit rate by the reading time, effective usage rate, and effective reading rate. The definition of these indices is summarized in Table 2.

Table 2. Definition of Subjective Performance Measures Measure Definition Acceptance Rate

No. of recommended and read/No. of recommended

Number Hit Rate

No. of recommended and read/Total No. of read

Time Hit Rate Total time reading recommended news/Total reading time

Effective use rate

Total time of selecting and reading news/Total time

Effective reading rate

Total time of reading/Total time of selecting and reading

The acceptance rate (AR) shows the

number of recommended reports that are read by the subject. Number hit rate (NHR) shows the portion of the report read by the subject that is among the recommended list. The time hit rate (THR) shows the portion of reading time that is spent on the recommended news. Effective use rate (EUR) shows the availability of the system, that is, the portion of time spent with the system that is actually selecting and reading news. It excludes the time for transmitting and processing news by the system. Effective reading rate (ERR) shows the portion of available time spent on reading news. The higher these indices are, the better.

(2) User Satisfaction

The instrument for measuring user

7.00 (c) 2002 IEEE 5


Proceedings of the 35th Annu0-7695-1435-9/02 $17.00 © 20

satisfaction includes four dimensions: information content, customized services, user interface, and system value. Satisfaction on information content is measured by three questions adapted from Doll [3]: (1) whether the system finds the news that the reader wants to read, (2) whether the system filters out the news that the reader does not want, and (3) whether the system captures the right category of interest to the reader.

Satisfaction on customized services is measured by three questions adapted from the personalized service portion of SERVQUAL [12]. They are: (1) whether the system provides personal attention, (2) whether the system captures my interests, and (3) whether the system provides customized services.

Satisfaction on user interface is measured by four questions adapted from Doll [3]. They are (1) whether the system is easy to use, (2) whether the system is friendly, (3) whether the interface is properly formatted, and (4) whether the presentation is clear. System value asks about whether the system is useful and is quick to find interesting news. Table 3 summarizes the measured dimensions. Finally, a question is designed to assess the overall satisfaction of the user on the system. All questions are on a 7-point Likert scale with 1 being least agreed and 7 being most agreed.

Table 3. Dimensions for Measuring System Satisfaction Information content

Customized service

User Interface

System value

- Find the wanted - Filter the unwanted - Find right category

- Personal attention - Capture interests - Customized service

- Ease of use - Friendly - Right format - Clear presentation

- Useful - Quicker

The reliability test of the questionnaire

using Cronbach’s alpha shows that the instrument is generally acceptable because most alpha values are higher than 0.6 (in Table 4).

Table 4. The Reliability Data Dimension Cronback α Information content 0.7018 Customized services 0.7714 User interface 0.8861 System value 0.6792

5.4 Experimental Design and Procedures

Since collecting browsing data needs to

have consecutive uses of a website, the subjects were asked to participate in the experiment for four days. A total of 96 volunteered subjects

wdHHdhG

vsOSatDattt

nn(enbedenchhscimc

f

t(daH

ns

n

nSas

Table 5. Distribution of News in the Experiment

4

0-7695-1435-9/02 $1al Hawaii International Conference on System Sciences (HICSS-35�02) 02 IEEE

ere recruited at the beginning. They were ivided into two groups, one of which viewed LA and SRI (Group I) and the other viewed LA and BBA (Group II). Nine of them ropped out during the experiment,. So, we ad a total of 87 effective subjects, with 43 in roup I and 44 in Group II.

Subjects in both groups were asked to iew HLA in the first three days and fill out a atisfaction questionnaire after the second day. n the fouth day, subjects in Group I viewed RI and those in Group II viewed BBA. They ll filled out questionnaires again to indicate heir satisfaction with the experimental system. ue to the difference in the recommendation

pproach, subjects in Group I needed to indicate heir interests in the report on a 1-7 scale (7 to be he most interesting) after each reading, while he subjects in Group II did not have to do so.

In order to be close to the real world, the ews adopted for the experiment was actual ews provided by China Times www.chinatimes.com.tw). During the xperimental period (June 7 – June 10, 2000), all ews available on the website of China Times efore 9:00 am were downloaded to the xperimental system, organized based on ifferent approaches, and then presented to the xperimental subjects. The average number of ews per day was 255, distributed into 13 ategories, with an average of 44 chosen as eadline news by the editors and put in the omepage in the HLA approach. Table 5 hows the distribution of reports in the 13 ategories: headline news (HDL), politics, nternational, China, finance, stock, technology,

edical, entertainment, sports, art, and omments.

The experimental procedures include the ollowing:

Day 1: the subject logged onto the website o read the description of the experiment approximately 5 minutes), filled out personal ata (5 minutes), learned the system (5 minutes), nd read the news (20 minutes) arranged by the DA approach.

Day 2: Logged onto the system, read the ews for 20 minutes, and filled out the atisfaction questionnaire.

Day 3: Logged onto the system, read the ews for 20 minutes.

Day 4: Logged onto the system, read the ews for 20 minutes that were arranged by the RI approach for Group I and by the BBA pproach for Group II, and then filled out the atisfaction questionnaire.

7.00 (c) 2002 IEEE 6


Proceedings of the 35th An0-7695-1435-9/02 $17.00 ©

HDL Polit Social Int’l China Finance Stock Tech Med Ent. Sport Art Comm Total 6/7 41 12 23 13 13 45 12 8 10 14 15 10 14 230 6/8 45 9 24 7 16 49 36 13 8 13 21 10 14 265 6/9 45 11 22 22 16 48 33 11 10 17 24 9 11 279 6/10 44 11 21 19 16 33 22 9 9 14 21 11 14 244 Avg 44 11 23 15 15 44 26 10 9 15 20 10 13 245 5.5 Experimental Results
(1) Descriptive statistics
Major data collected from the experiment includes (1) distribution of reading interests (2) motivation for reading from websites, (3) major news source, (4) reading behavior, and (5) performance of the experiment. They are illustrated in Tables 6-8. Table 6. Most Interesting News Category as Indicated by the Subject

Ent HDL Social Sport Polit Int’l Med China Finance Tech Stock Art Comm Total No. 24 13 13 12 6 6 6 2 2 2 1 0 0 87 % 27.6 14.9 14.9 13.8 6.9 6.9 6.9 2.3 2.3 2.3 1.1 0 0 100

Table 7. Motivation for Using News Websites

Free news Love web Social interaction Time killing Total No. 39 32 9 7 87 % 44.8 36.8 10.4 8 100
Table 8. Major News Sources Web TV Print media Radio Total No. 35 31 18 3 87 % 40.2 35.6 20.7 3.4 100
Table 6 shows that the most preferred category was entertainment news with a total of 27.6% of

the subjects chosing it. The least preferred news were art and comments. Table 7 shows that the major reason for the subjects to use web news was because it was free. The second reason was their love in using the web. Table 8 shows that the web was the most favorable news source for 40% of the subjects, whereas TV was the next favorable one.

Descriptive data (mean and standard deviation) of the browsing behavior are shown in Table 9. The system recorded the number of reports read by the subjects (NRR), number of news accepted by the subjects (NRA), number of news shown on the homepage (NNS), system processing time (SPT), selecting time (ST), reading time (RT), and time for rating the read news (TRN, for HDA and SRI).
Table 9. Browsing Statistics of the Subjects
HLA SRI BBA Mean St. dev. Mean St. dev. Mean St. dev.

NRR 14.02 6.32 14.53 5.99 14.34 5.21 NRA 2.16 2.49 6.33 3.51 6.27 3.55 NNS 41 0 17.77 7.42 17.61 5.97 SPT 38.46 35.52 16.88 5.72 16.30 6.57 ST 213.51 89.03 188.04 66.67 176.66 61.05 RT 597.13 108.74 651.86 80.75 707.05 60.94 TRN 50.88 20.83 43.20 22.56 0 0

The statistics indicate that the HLA system presents 41 news titles to all subjects in the homepage,

whereas SRI and BBA present an average of 17.77 and 17.61 recommended news titles in the homepage to each subject, respectively. Standard deviations among the subjects are 7.42 and 5.97 for SRI and BBA. That is, the recommendation systems are more selective than the standard headline news version. This allows the subject to spend more time on reading news (RT) and less time on selecting news (ST). The objective performance indices calculated from the browsing behavior are shown in Table 10. Table 10. Performance Indices of the Three Systems

2

0-7695-1435-9/02 $17.00 (c) 2002 IEEE 7nual Hawaii International Conference on System Sciences (HICSS-35�02) 2002 IEEE


Proceedings of the 35th A0-7695-1435-9/02 $17.00

HLA SRI BBA Mean St. dev. Mean St. dev. Mean St. dev, AR .0532 .0609 .3943 .2340 .3787 .2051 NHR .1710 .2119 .4539 .2304 .4573 .2354 THR .2115 .2449 .4469 .2432 .4622 .2803 EUR .9007 .0486 .9332 .0270 .9818 .0073 ERR .7343 .1159 .7749 .0829 .8001 .0689

nn© 2

(2) Hypothesis Testing

Since all subjects used the HLA system before using SRI or BBA, the paired t-test is used to test

the performance difference between SRI and HLA, and SRI and BBA, but independent t-test is used to test the difference between SRI and BBA. H1: SRI and BBA perform equally well with HLA

The results of the paired t-test as shown in Tables 11 and 12 indicate that SRI outperforms HLA significantly in all objective indices and user satisfaction. The results in Tables 13 and 14 indicate that BBA also outperforms HLA significantly in most objective indices (ERR is near significant) and user satisfaction (except the perceived value). Therefore, we can safely conclude that the recommendation systems outperform the traditional headline approach. The null hypothesis is
rejected.
Table 11. Results of paired t-test on Performance for SRI and HLA (df=42) Mean SRI HDL

Difference t-value Significance

AR 0.3943 0.0530 0.3416*** 8.820 0.000 NHR 0.4539 0.1710 0.2829*** 6.117 0.000 THR 0.4469 0.2115 0.2354*** 4.786 0.000 EUR 0.9332 0.9007 0.033*** 5.084 0.000 ERR 0.7749 0.7343 0.041** 2.451 0.018

Note: ** denotes p<0.05; *** denotes p<0.01
Table 12. Results of paired t-test on Satisfaction for SRI and HLA (df=42)
Mean SRI HDL


Content 5.8488 5.2558 0.5930*** 4.379 0.000 Customization 5.7558 4.6628 1.0930*** 7.474 0.000 Interface 5.6802 5.4244 0.2558*** 2.632 0.012 Value 5.9767 5.3488 0.6279*** 3.699 0.001 Overall 5.8605 5.2558 0.6047*** 6.800 0.000

Note: *** denotes p<0.01
Table 13. Results of paired t-test on Performance for BBA and HLA (df=43)
Mean BBA HDL


AR 0.3786 0.0690 0.3099*** 9.244 0.000 NHR 0.4573 0.2236 0.2337*** 3.939 0.000 THR 0.4622 0.2440 0.2183*** 3.173 0.003 EUR 0.9819 0.9729 0.009*** 5.15 0.000 ERR 0.8001 0.7711 0.029* 1.92 0.061

Note: * denotes p<0.10; *** denotes p<0.01

3

0-7695-1435-9/02 $17.00 (c) 2002 IEEE 8ual Hawaii International Conference on System Sciences (HICSS-35�02) 002 IEEE



Table 14. Results of paired t-test on Satisfaction for SRI and HLA (df=42) Mean BBA HDL


Content 5.6250 5.3068 0.3182* 1.956 0.057 Customization 5.4091 4.5682 0.8409*** 4.650 0.000 Interface 5.5739 5.3125 0.2614** 2.168 0.036 Value 5.5455 5.4773 0.0678 0.380 0.706 Overall 5.7727 5.3409 0.4318*** 3.772 0.000

Note: * denotes p<0.10; *** denotes p<0.01
H2: SRI and BBA perform equally well
Tables 15 and 16 show the performance differences between SRI and BBA, which use two different recommendation mechanisms. BBA performs better than SRI in the rate of effective use (EUR), but worse than SRI in customization and perceived value. The overall satisfaction shows no significant difference. Therefore, the null hypothesis cannot be denied. We conclude that both recommendation mechanisms are equally good.

Table 15. Results of t-test on Performance for SRI and BBA (df=85)

Mean SRI BBA


AR 0.3943 0.3787 0.016 0.332 0.741 NHR 0.4539 0.4573 0.003 0.069 0.945 THR 0.4469 0.4622 0.0153 0.272 0.786 EUR 0.9332 0.9819 0.048*** 11.608 0.000 ERR 0.7749 0.8001 0.025 1.546 0.126

Note: * denotes p<0.10; *** denotes p<0.01 Table 16. Results of paired t-test on Satisfaction for SRI and HLA (df=42)

Mean SRI BBA


Content 5.8488 5.6250 0.2238 1.506 0.136 Customization 5.7558 5.4091 0.3467** 2.004 0.048 Interface 5.6802 5.5739 0.1064 0.633 0.529 Value 5.9767 5.5455 0.2933* 1.740 0.086 Overall 5.8605 5.7727 0.087 0.377 0.512

Note: * denotes p<0.10; ** denotes p<0.05; *** denotes p<0.01
(3) Effect of individual difference
In addition to the effect of recommendation

mechanisms, we also examine the possible effect of individual difference. Two effects are found to be significant in ANOVA tests: (1) motivation for using web news has a significant impact on overall satisfaction (F=5.227, p<0.01) and (2) the preferred news source has a significant impact on the acceptance rate of recommended news (F=2.790, p<0.05).

Readers with a motivation for more social interaction (mean= 6.3333) are more satisfied with the system than all other groups. Those with motivations of web loving (5.8438) and free news (mean = 5.7436) are more satisfied than the time killing group (mean = 5.1429). These may be because members of the social interaction group enjoy reading the news for communicating with their friends. They are more selective in what news they would like to read. The members of the time killing group read the news for no major purpose and hence

0-7695-1435-9/02 $nual Hawaii International Conference on System Sciences (HICSS-35�02) 2002 IEEE

are not very selective. The subjects whose major news source

being the print media have the highest rate of acceptance (mean=0.4983), which is significantly higher than that of the TV group (mean=0.3231). This may be because web news is still presented in a format similar to print media. Therefore, the print media group is more accustomed to the web news.

6. Concluding Remarks

In this paper, we have presented a

time-based approach to analyzing user profile and its application to Internet news recommendation services. Empirical results show that the proposed approach performs as well as the self-reported interests approach and significantly better than the headline news approach. References

[1] Balabanovic, M. and Shoham, Y., “Fab:

4

17.00 (c) 2002 IEEE 9



Content-based Collaborative Recommendataion,” Communications of the ACM, 40:3, 1997, pp. 66-72.

[2] Blumler, J. G., "The Role of Theory in Uses and Gratifications Studies," Communication Research, Vol. 6, 1979, pp. 9-36.

[3] Doll, W.J. and Torkzadeh, G., “The Measurement of End-user Computing Satisfaction,” MIS Quarterly, 12:2, 1988, pp. 913-923.

[4] Katz, E., Blumler, J.G., and Gurevitch, M., "Utilization of Mass Communication by Individual," in Blumler, J.G. and Katz, E.(Eds.), The Use of Communications, Sage, Beverly Hills, CA, 1974, pp. 19-32.

[5] Konstan, J. A., Miller, B. N., Maltz, D., Herlocker, J. L., Gordon, L. R., and Riedl, J., "GroupLens: Applying Collaborative Filtering to Usenet News," Communications of the ACM, Vol. 40, No. 3, Mar-97, pp. 77-87. [6] Liang, T.P., "A Composite Approach to Inducing Knowledge for Expert Systems Design," Management Science, 38:1, pp. 1-17, 1992

[7] Lorrie, A., “The Electronic Newspaper of the Future: Rationale, Design, and Implications,”http://cecl.wustl.edu/~cs142/article/MISC/PUBLISHING/electronic-newspaper-cranor, 1992.

[8] McQuail, D., Media Performance, London: Sage Publications, 1992.

[9] Miller, G.A., “WorleNet: A Lexical Database for English,” Communications of the ACM,” 38:11, 1995, pp. 39-41.

[10] Mock and Vemuri, "Information Filtering Via Hill Climbing, Wordnet, and Index Patterns," Information Processing & Management, Vol. 33, No. 5, 1997, pp. 633-644.

[11] Muller, J. and Kamerer, D., “Reader Preference for Electronic Newspaper,” Newspaper Research Journal, 16:3, 1995.

[12] Parasuraman, A., Zeithaml, V.A. and Berry, L.L., “SERVQUAL: A Multiple-item Scale for Measuring Consumer Perceptions of Service Quality,” Journal of Retailing, 64:1, 1988, pp. 12-40.

[13] Pazzani, M.J., " A Framework for Collaborative, Content-Based and Demographic Filtering," Artificical Intelligence Reivew, http://www.ics.uci.edu/~pazzani/ Publications/ Publications.html.

[14] Rau, L., “Extracting Company Names from Text,” Sixth IEEE Conference on Artificial Intelligence Applications, Miama Beach, Florida,

2

0-7695-1435-9/02 $1nual Hawaii International Conference on System Sciences (HICSS-35�02) 2002 IEEE

1991, pp. 29-32.

[15] Rowe, N.C., “Inferring Depictions in Natural Language Captions for Efficient Access to Picture Data,” Information Processing and Management, 29:4, 1994, pp. 453-461.

[16] Rucker, J. and Polanco, M.J., “Siteseer: Personalized Navigation for Web,” Communications of the ACM, 40:3, 1997, pp. 73-76.

[17] Sakagami, H. and Kamba, T., “Learning Personal Preferences on Online Newspaper Articles from User Behaviors,” Computer Networks and ISDN Systems, Vol. 29, 1997, pp.1447-1455.

[18] Saracevic, T., and Kantor, P., "A Study in Information Seeking and Retrieving. II. User, Questions and Effectiveness," Journal of the American Society for Information Science, Vol.39, No.3, 1998, pp. 176-195.

[19] Spink, A. and Leatherbury, M., “Name Authority Files and Humanity Database Searching,” Online and CDROM Review, 18:3, 1994, pp. 143-147.

7.00 (c) 2002 IEEE 10

Discovering User Interests from Web Browsing Behavior: An Application to Internet News Services

Documents

Transcript of Discovering User Interests from Web Browsing Behavior: An Application to Internet News Services