Int. J. Intercultural Information Management, Vol. 1, No. 2, 2009 127

Copyright © 2009 Inderscience Enterprises Ltd.

International differences in web page evaluation guidelines

Thomas Mandl* Information Science, University of Hildesheim, Marienburger Platz 22, 31141 Hildesheim, Germany E-mail: mandl@uni-hildesheim.de *Corresponding author

Tatiana de la Cruz NEOMAN Bus GmbH, Marketing Planning and Intelligence (BVMS), Heinrich-Büssing-Str. 1, D-38239 Salzgitter, Germany E-mail: tatiana.delacruzobregon@de.man-mn.com

Abstract: The notion of quality of websites depends on the culture of the individual user. This article provides empirical evidence based on the analysis of existing evaluation guidelines from different countries and a questionnaire for users from two countries, Peru and Germany. Cultures are typically characterised by cultural dimensions which are correlated with the responses in the study presented. Some of the evidence is statistically significant. Developers of web pages and e-commerce sites with an international user group need to be aware of the fact that the evaluation criteria applied in different cultures may be different.

Keywords: evaluation guidelines; intercultural research; quality of information services; web design.

Reference to this paper should be made as follows: Mandl, T. and de la Cruz, T. (2009) ‘International differences in web page evaluation guidelines’, Int. J. Intercultural Information Management, Vol. 1, No. 2, pp.127–142.

Biographical notes: Thomas Mandl is an Assistant Professor for Information Science at the University of Hildesheim, Germany. His research interests include information retrieval, human-computer interaction, applications of machine learning, multi-lingual systems and international information systems. He studied at the University of Regensburg, Germany and at the University of Illinois in Champaign/Urbana, USA.

Tatiana de la Cruz studied International Information Management at the University of Hildesheim, Germany. She works for the ‘NEOMAN Bus GmbH’ Business Unit of the MAN AG, where she was initially involved in the worldwide service for bus exports. Currently, she is responsible for the strategic marketing for Latin America.

128 T. Mandl and T. de la Cruz

1 Quality of web pages

On the web, no central quality assurance is implemented and as a consequence the quality of the documents is extremely heterogeneous. This fact has been lamented to a great extent: “For many topics, the world wide web contains hundreds or thousands of relevant documents of widely varying quality” (Amento, Terveen and Hill, 2000). Page et al. (1998) stress the simplicity of publishing on the web: “the simplicity of creating and publishing web pages results in a large fraction of low quality web pages that users are unlikely to read”. Web information services need to be sensitive to these differences and should favour high quality pages to present to their users. Consequently, quality evaluation becomes necessary, but what does quality mean and how can it be defined?

Quality may concern interaction as well as content and presentation. A helpful framework for discussing the quality of information products is provided by Price and Shanks (2004). Their model is formulated in analogy to the levels of language research: syntax, semiotics and pragmatics. Syntax and semiotic refer to errors on the level of the use and the meaning of symbols. Quality issues on these lower levels are detectable for users and evaluators. For the purpose of this study, the pragmatic level is of relevance. It is concerned with quality issues occurring during the use of information systems. Quality problems on the pragmatic level are more difficult to resolve.

A model of Marchand (1990) lists several approaches for quality definitions:

Transcendent: this approach assumes the existence of an objective and absolute quality which is universally valid.

User-oriented: this approach emphasises the subjectivity and puts quality into the context of the user situation.

Product-oriented: the information product and its features are the focus of this approach which assumes that quality is measurable and quantifiable to a large extent.

Production-oriented: the process of product creation and the fulfilment of specifications are the main issues for quality.

Value-oriented: a balance between positive aspects of the product and its costs is required.

Most quality definitions for the web belong to the first two approaches which form two extreme positions along a continuous spectrum. Most empirical researches with user tests on web page quality indicate that the subjective and context-dependent definitions are more appropriate (Cooke 1999; Amento, Terveen and Hill, 2000).

Automatic approaches for quality evaluation of information products are currently not very advanced. Several solutions have been proposed:

The number of hypertext-links pointing to a web page is used for the automatic assessment of its quality (link analysis; Henzinger, Motwani and Silverstein, 2002).

Advanced experimental methods for automatic quality assessment evaluate a much larger number of criteria than algorithms for link analysis (Mandl, 2006).

International differences in web page evaluation guidelines 129

Students who want to be admitted to graduate school in the USA often take a standardised test which requires the candidates to write an essay. While this essay was previously evaluated by two humans, it is now read only by one expert and additionally graded by an automated system (Miltsakaki and Kukich, 2004).

Guidelines or norms are certainly not always followed; however, they have significant influence on the creation and perception of websites. Guidelines always express cultural values and attitudes. Thus, we have assumed that guidelines for web-page creation and evaluation must be one key to research on cultural differences in web design. Findings in this area may also contribute to research on cultural differences on a global phenomenon like the internet.

In this article, we focus on criteria lists for the evaluation of websites which are mainly published within the tradition of library science. The lack of critical thinking toward the internet resources common among many users has led to the creation of many evaluation guidelines (Graham and Metaxas, 2003).

The empirical work presented here is twofold. First, we have analysed guidelines from different cultures found on the web. The results have led to the creation of a questionnaire on the importance of quality criteria which was filled out by more than 300 internet users mainly from Peru and Germany. The results have confirmed that criteria are weighted differently across cultures.

The remainder of the article is organised as follows. Section 2 provides a short literature review on the research on cultural differences in web design. Section 3 shows the results of the analysis of guidelines found on the internet. Section 4 presents the results of the subsequent study with the questionnaire.

2 Cultural differences in web page design

Like any information system, web pages and e-commerce sites need to be adapted to the language and culture of the potential user or customer. This process is called localisation (Gilham, 2004; Aykin, 2005). Many suggestions for the adaptation refer to simple facts like formats, colours or symbols (Romberg, Röse and Zühlke, 1999). Moreover, localisation also needs to consider hidden aspects of culture (Barber and Badre 1998; Sturm, 2005). Empirical studies reveal that users do not always prefer the local site which is supposedly adapted for them (Cyr et al., 2004). This shows that much more research is necessary in this area.

Understanding a particular culture and the resulting requirements for the design of information systems requires an understanding of the concept of culture and the factors that contribute to its existence (cf. del Galdo 1996; Hermeking, 2005; Choong, Plocher and Rau, 2005;).

Quantified models of culture are difficult to find. Hofstede defines culture as learned patterns of ‘thinking, feeling, and potential acting’ that form the mental programme or the ‘software of the mind’ (Hofstede, 1997). This particular ‘software’ affects our way of thinking and our learning behaviour. Cross-culturalists like Trompenaars argue that culture consists of several layers and illustrates that idea by using the metaphor of an onion: the most visible outer layers are easier to access than the hidden inner core, which is difficult to identify (Trompenaars and Hampden-Turner, 1997). Cultures are often classified in accordance with their relative positions on a number of polar scales which

130 T. Mandl and T. de la Cruz

cultural anthropology commonly calls cultural dimensions. The position of a culture on those scales is determined by the dominant value orientations. Hofstede defined four dimensions of culture (Hofstede, 1997; Hofstede and Hofstede, 2005):

1 Power distance measures the extent to which subordinates (employees, students) respond to power and authority (managers, teachers) and how they expect and accept unequal power distribution.

2 Individualism vs. collectivism: these value orientations refer to the ties among individuals in a society.

3 Uncertainty avoidance describes the extent to which individuals feel threatened by uncertain or unknown situations.

4 Masculinity vs. femininity: these two extreme values of this dimension focus on the differences between the social roles attributed to men and women and the expected behaviour of the two sexes.

Cultural dimensions like the ones defined by Hofstede have often been used in the research on cultural differences, although these dimensions have often been criticised within intercultural research. The quantitative study of Hofstede did not take differences within cultures in heterogeneous populations into consideration and was possibly influenced by a corporate culture as it was carried out among employees of one company (Jarman, 1998).

For research on the localisation of information systems, cultural dimensions have often been a starting point because they provide a plausible and quantified culture model. Marcus and Gould (2000) presented examples for web pages with differences for all cultural dimensions. However, their findings are based on a small and pre-selected set of websites.

Empirically convincing studies are difficult to set up from a methodological perspective. In common quantitative human-computer interaction studies, two versions of a user interface are presented to two user groups selected from the same culture and which are believed to be homogeneous. For comparative studies in international web design analysis, the user groups are different and their reaction to the system is being investigated. However, it is difficult for researchers to maintain the system constant. The system cannot be presented to two groups of users from different countries without being modified. The system needs to be translated and culturally adapted. For example, the investigated task may be done completely different in the two cultures. Typical user groups comprised of university students may have quite different features like social group in different societies, for instance. Hence, the system often needs to be changed significantly in order to be adequate for a real-life experiment, something which makes comparability difficult (Evers, 2002). This is a general problem encountered in intercultural research (Eckensberger and Plath, 2003) which is highly relevant for the analysis of intercultural aspects of information technology (Law and Hvannberg, 2004).

To overcome these problems, sometimes expatriates are used as test users (Sheppard and Scholtz, 1999). This may loosen up the problem of a language barrier, however, one may not forget that language competency in a second language is not comparable to the language competence in a native language. Another way to overcome the methodological problems is the use of mock-up systems instead of real-life web pages (e.g. in Sheppard and Scholtz, 1999; de Wit, Diehl and Arts, 2005; Hodemacher, Mandl and Jarman, 2005).

International differences in web page evaluation guidelines 131

The drawback lies in the artificiality of the experiment. One further method is the use of pages which are in a foreign language for all test users (e.g. Dormann and Chisalita, 2002; Schmitz, 2005).

Dormann and Chisalita (2002) tried to quantify the differences between the perception of test users from different cultures and the design of websites from other countries. Their analysis is focused on the dimension femininity vs. masculinity for university websites in Italy and Scandinavia.

So far it is not clear how cultural dimensions may contribute to research on intercultural web design. Some authors noted that the assumptions made on the basis of cultural dimensions were misleading (Griffith, 1998), others are more critical and doubt that relations can be found (Gilham, 2004).

3 Comparison of evaluation criteria

Primarily, existing guidelines from different countries were compared in order to establish if there are any differences in the quality evaluation. Many of the methodological problems for comparing web pages from different cultures empirically could be avoided by this approach. We assume that web evaluation guidelines are a condensed manifestation of cultural values.

For this study, we searched for criteria lists for the evaluation of websites which were published on the web. A comprehensive list of all guidelines can be found in de la Cruz (2003). Even considerable differences within a culture occur sometimes. Table 1 from the literature shows a comparison for guidelines from the USA. Table 1 Evaluation criteria for web pages

Rank Criteria Importance

1 Bias/purpose/objectivity 17 2 Currency 13 3 Author 12 3 Publisher 12 4 Credibility 10 4 Accuracy 10 5 Documentation 6 6 Relevance to user 5 7 Scope/coverage 4 7 Authors authority 4 7 Aesthetics/visual content 4 8 Authority 3 8 Intended audience 3 8 Clarity 3

132 T. Mandl and T. de la Cruz

Table 1 Evaluation criteria for web pages (continued)

Rank Criteria Importance

9 Appropriateness of format 2 9 Navigation 2 9 Site access and usability 2 9 Validity 2 9 Learning environment 2 9 Information structure and design 2 10 Academic credibility 1 10 Content design and technical feature 1 10 Date of creation 1 10 Link to local pages 1 10 Quality of links 1 10 Reading levels 1 10 Relation to other works 1 10 Spelling 1

Source: Radford, Barnes and Barr (2002).

The differences between evaluation criteria from different cultures are even larger than those within a culture. For our analysis, we identified 17 lists from the USA and 15 lists from German and French sites and from Spanish-speaking countries.

It is remarkable to note that there are many more lists in the USA than in other countries. This certainly due to the fact the internet plays an important role in a technologically highly developed country like the USA. However, it seems that the specific culture of the USA is favourable for the formulation of lists of evaluation criteria. The culture of the USA is highly explicit and exhibits a high level of uncertainty avoidance. Both facts contribute to the existence of guidelines. An explicit culture of low context prefers to specify most aspects of life and relies little on meaning conveyed through context. Cultures with high uncertainty avoidance also prefer to state why websites are of good quality. Nevertheless, this interpretation is speculative because it is impossible to exhaustively collect all criteria lists for the evaluation of websites.

We also need to consider that the population of the USA is ethnically highly heterogeneous and that there are likely to be differences among these groups. As stated above, the study of Hofstede (1997) did not take internal diversity into account. However, in a mix of culture, it is necessary to rely more on explicit communication because implicit cultural signals cannot be interpreted correctly by all members. As a consequence, even members from various cultural backgrounds may tend to exhibit low context behaviour in the USA.

The criteria lists from the non-English sources are summarised in Table 2. The comparison showed that the differences are mainly due to different rankings of similar criteria. Table 2 shows the criteria at the rank at which it was first mentioned in any of the lists.

International differences in web page evaluation guidelines 133

Table 2 Comparison of evaluation criteria for web pages

German sources French sources Sources from Spanish-speaking countries

1 Coverage 1 Authority 1 Authority 2 Accuracy 1 Coverage 1 Navigation 3 Navigation 2 Currency 1 Structure and design 4 Target group mentioned 2 Structure and design of

information 2 Currency

4 Currency 2 Objectivity 2 Contact 5 Helpful links 2 Accuracy 2 Zugang 5 Objectivity 3 Navigation 3 Accuracy 5 Authority 4 Understandability 3 Coverage 5 Structure and design of

information 4 Target group mentioned 3 Multimediality

(Videos, Music, etc.) 5 Aesthetic design 5 Cost of information 3 aesthetic Design 6 Usability 5 Origin and language of content 4 Service (help) 6 Contact (Chat room, etc.) 6 Advertisement separated from

content 4 Entertainment

6 Uniqueness 6 Local search 4 Interactivity 6 Peer review

(expert revision) 6 Peer review

(expert revision) 4 Objectivity

6 Understandability 6 Contact 5 Language of content 6 Clarity of presentation 6 Access 6 Local search 6 Access 6 Aesthetic design 7 Usability 6 Fast download 6 Uniqueness 7 Understandability 6 Meta information 7 Cost of Information 6 Service (fast response) 7 Helpful links 7 Little advertisement 7 Target group mentioned 7 Wap access 7 Uniqueness

Source: de la Cruz (2003).

The differences are quite obvious and can also be related to cultural dimensions. However, this method cannot be the only one used to verify cultural differences. In addition to the criteria lists themselves, their perception and their use need to be evaluated as well.

4 Empirical study for the internet users in Peru and Germany

The preliminary findings from the analysis of guidelines led to an empirical study which examined the attitudes of users towards the criteria and the perception of the importance of the criteria within a quality definition. Our hypothesis was that the criteria found in typical guidelines for the evaluation of websites are weighted differently by the internet users from two cultures. Further details on the experiments can be found in de la Cruz (2003).

134 T. Mandl and T. de la Cruz

4.1 Comparing Peru and Germany

The differences between the values on the cultural dimensions for both Peru and Germany can be seen in Table 3. The major differences can be observed for individualism and power distance. But also for the other two dimensions for which values are available, the differences are considerable. Table 3 Values of cultural dimensions for Germany and Peru

Cultural dimension

Power distance

Individualism vs. collectivism

Masculinity vs. femininity

Uncertainty avoidance

Long term-orientation

MD IDV MAS UVI ILO Germany 35 67 66 65 31 Peru 64 16 42 87 na Source: Hofstede and Hofstede (2005).

Both in Germany and Peru, the use of the internet is widespread. However, both the economic situation and the cultural values have led to a different participation of the population. The situation can be characterised as the digital divide in the information society and can be seen in Table 4. Although Peru is fairly advanced to other Latin US countries, it lags behind Germany in the number of the internet users (Telefónica del Perú, 2002). Table 4 Information and communication technology in Germany and Peru at the time

of the study

Gross national product per person in USD

Telephone lines per 100 citizens

Number of personal computers per

100 citizens Internet users per

100 citizens Germany 22,420 63.46 33.60 36.37 Peru 1,923 7.75 4.79 11.50 Source: Telefónica del Perú (2002).

The usage situation in Peru is characterised by the low number of users. Most internet users live in cities and are very young. The capital Lima concentrates more than 31% of all inhabitants of the country. The lack of PCs as well as the collective orientation leads to the popularity of the internet cabins where users access the internet in a semi-public setting. These cabins also fulfil a social function as a meeting point for your people. The internet cafes in Germany play a much smaller role. The demographic profile of users of the internet cabins differs from those who have access to the internet at home or at work. The percentage of female users is highest for the internet cabins (47%) compared to access at home (37%) and at work (41%) (Red Científica Peruana, 2002).

The most frequent activities are e-mail, information seeking and chatting (Telefónica del Perú, 2002). According to Hofstede, the use of a tool like the internet for communicating is typically for a feminine culture like Peru (Emerald, 2005).

4.2 Study setup

Based on the analysis presented above, a set of relevant criteria was selected to be included in the questionnaire. Three main questions with different sets of evaluation criteria were asked:

What calls your attention when looking at a website? (Likert scale).

International differences in web page evaluation guidelines 135

What features of a website are important for you? (Likert scale).

What are the most important criteria for the evaluation of a website? (rank the criteria according to their importance).

The items can be seen in the result section. The questionnaire was implemented as a dynamic internet page under a temporary Uniform Resource Locator and results were directly entered into a database. The questionnaire was translated into four languages, and four versions were presented online (English, French, Spanish and German).

Within four weeks, the questionnaire was completed by 368 individuals of whom 226 were Germans. The second largest group were 109 Peruvians. Both in Germany and Peru, the questionnaire had been advertised by mailing lists and through personal contacts. Participants from 16 other countries also filled out the questionnaire, but for no other country, more than seven people entered their data. The other countries were included only in the second part of the analysis presented in this article. In Germany, 43% of the respondents were male and in Peru, 48%. The majority was between 21 and 25 years old (43%) and the age group between 26 and 30 was the second largest group (25%). Some 14% were below 20 years old while only 13% were older than 30 and merely 1% was older than 50 years. The age distribution of the participants reflects the user patterns well.

The participants graded their experience with the internet and their information competence. Most people marked good (51%) or even very good (28%) and only 18% regarded themselves of medium competence (3rd on a five-point scale). There were remarkable differences between the self-estimate of the Peruvian and German participants which can be seen in Table 5. Table 5 Internet experience of the participants

Germany (%) Peru (%)

Very good 35 9 Good 53 47 Average 9 37 Below average 1 5 Insufficient 0 0

It is not entirely clear, whether these differences reflect the actual experience or are due to cultural differences. It is well known for Asian cultures that the self placement needs to consider the politeness of Asian participants who consider themselves less proficient as comparable members of Western cultures (Iwai, 2005). A similar effect may occur for a comparison between Peru and Germany. Peruvian culture is more oriented toward quality of life (femininity according to Hofstede), whereas Germans place more emphasis on competitiveness and achievement (masculine values).

Some 60% of the test persons were university students and 50% used the internet less than five hour a week. In the survey, 55% of the respondents were male. These numbers seem well-balanced and present the average user as well as the typical target group for many websites. The most common internet activities of the respondents were information seeking (89%) and communication via e-mail (77%). Less common were downloads (29%), e-commerce (26%) and chats (20%). Here, the results gained from the respondents were different from the general statistics presented in Section 4.1 (Telefónica

136 T. Mandl and T. de la Cruz

del Perú, 2002). Nevertheless, the Peruvians used the internet more often for communicative activities than the Germans.

The difference between Peru and Germany was very large for the activities associated with e-commerce. Only 7% of the Peruvians had ever bought or sold anything via the internet while 33% of the Germans had. That fact may be associated with the high desire for uncertainty avoidance in Peru.

4.3 Results

The results show that the criteria are evaluated differently, however, not all differences proved to be statistically significant. The following Tables 6–8 present the results from the three main questions in the questionnaire mentioned above.Table 6 Comparison of evaluation criteria for web pages (grade 1 corresponds to

highest importance, grade 5 corresponds to no importance)

What calls your attention when looking at a website?

German participants, average grade

Peruvian participants, average grade

Significant with 90%probability

Visual appearance 2.23 2.62 Content 1.41 1.66 * Reputation 2.93 2.98 Structure and organisation

1.99 2.61

Hint from a friend 2.29 3.03 Originality 2.66 2.48

Table 7 Comparison of features of web (grade 1 corresponds to highest importance, grade 5 corresponds to no importance)

What features of a website are important for you?

German participants, average grade

Peruvian participants,average grade

Significant with 90%probability

Little advertisement 2.00 2.71 Good navigation 1.38 1.73 * Ease of access to information 1.31 1.51 * Currency of information 1.37 1.55 * Aesthetic design, multi-medial design

2.46 2.05

Entertainment 4.23 2.86 Service (help, hotline, newsletter)

2.36 1.99

Fast download 1.79 1.41 * Objectivity of the information 2.04 2.61 Sponsor is clearly marked 3.57 3.15 Helpful links 2.69 2.34 Well-known organisation 3.28 3.03 Site is listed in web catalogs 2.67 2.17

International differences in web page evaluation guidelines 137

Table 8 Ranking of evaluation criteria for web pages

Most important criteria for the evaluation of websites

German participants, average rank

Peruvian participants, average rank

Good navigation 5 1 Currency of information 2 1 Clarity of design 4 5 Correctness 1 4 Understandability 6 7 Popularity 9 9 Content 3 3 Aesthetic design 8 6 Local site search 7 8

For the first two questions a test for statistical significance was conducted for each item. For none of the items, statistical significance on the usual levels of 1 and 5% could be shown. A few items marked with an asterisk proved to be significant with less than 10% error probability.

A surprising finding is that the Peruvian participants did not grade reputation higher than German users. We expected that the reputation of the author would be important because of the higher power distance in the Latin US cultures. It seems that the stronger orientation towards collectivism makes the Peruvian participants neglect the importance of the individual author. This is a good example to show that plausible conclusions based on one cultural dimension may not always be true.

The differences between the responses on German and Peruvian questionnaires in the three tables can be summarised by calculating the rank correlation coefficient between the rankings for each country. The Spearman rank correlation coefficient is shown in Table 9. Table 9 Spearman correlation coefficient for the three questions

Question Rank correlation coefficient

What calls your attention when looking at a website? 0.57 What features of a website are important for you? 0.81 Most important criteria for the evaluation of websites 0.50

Only for the website features, a strong correlation exists between the two countries. For the criteria, there is merely a weak correlation between Peruvian and German participants. The relatively strong correlation between features and culture may be due to the fact that Germany and the Latin US countries do not have the most different cultures based on the cultural dimensions according to Hofstede (1997).

4.4 Relation to cultural dimensions

From the results above, the question arises why some criteria are considered more important than others. Which cultural dimensions may be responsible for preferences of a particular culture? To analyse the relation between the answers of the participants and the values of the cultural dimensions we looked up the values of the cultural dimensions for

138 T. Mandl and T. de la Cruz

all countries which had participants in the survey and which were listed in Hofstede (1997). The values for Germany and Peru can be seen in Table 10. These values were compared to the responses in the survey by calculating the correlation values. By applying this method, we could integrate participants from countries with very few respondents. Because Peru has no value for short- vs. long-term orientation, this dimension could not be considered. Table 10 Values of cultural dimensions for the two main countries in the study

Dimension Power distance Individualism Masculinity Uncertainty avoidance

Peru 64 16 42 87 Germany 35 67 66 65

Source: Hofstede (1997).

The correlations between all responses and cultural values were calculated. Table 11 presents the correlation values with an absolute value higher than 0.25. No correlation with this strength could be found between gender and preference for any criteria. The influence of culture seems to be stronger than that of gender. Table 11 Correlation to cultural dimensions

Criteria Cultural dimension Correlation coefficient

Significant with 95% probability

(Chi-square square test)

Objectivity Uncertainty avoidance 0.28 Structure Uncertainty avoidance 0.28 Entertainment Masculinity 0.55 * Little advertisement Uncertainty avoidance 0.31 Hint from a friend Masculinity 0.33 Hint from a friend Individualism/Collectivism 0.33

These correlation values need to be carefully interpreted. It is not clear whether they represent statistically reliable values because of the scale of the compared values. However, they clearly provide directions for further research. Some of the results are plausible and have been expected. The relation between the evaluation criteria objectivity and structure and the cultural dimension uncertainty avoidance is a good example. Structure is used more intensely in cultures in which uncertainty is less tolerated.

4.5 Discussion and interpretation

The study shows that criteria for website quality evaluation are seen differently across cultures. Lists published on the internet vary significantly.

The results from the questionnaire clearly show that the participants from different countries assign different importance to the same criteria. The cultural formation of individuals seems to have a great impact on the criteria that they explicitly consider important for websites. However, the correlation between culture and features considered important and culture and preferred criteria vary.

International differences in web page evaluation guidelines 139

Some common and plausible interpretations are confirmed but several are contradicted by our empirical results like the relation between power distance and importance of the author. A correlation analysis gives hints to the reasons for some results. It points to some cultural dimensions which may be responsible for preferences for certain criteria.

The quality of websites is likely to be interpreted differently as well, based on these results. However, this needs to be empirically validated. This study shows only a relation between the importance assigned to criteria, it cannot prove that these criteria really influence the evaluation process which is holistic and where the criteria are applied subconsciously. The results indicate that values and criteria are considered to be of different importance in cultures, however, our data does not prove that this result is any difference in behaviour.

5 Outlook

Our study has clearly indicated that guidelines for the evaluation of websites differ from culture to culture. The typical criteria are arranged in substantially different rankings. The study also shows that there is no global culture of the internet and that local cultures still dominate the behaviour of web users. It is unlikely that a tool like information technology alone will change the way people think (Emerald, 2005).

Much further research is necessary. Guidelines for web design may also provide an interesting corpus for research which we intend to study. User studies, like the one by Dormann and Chisalita (2002), still play a key role in the understanding of the relation between culture and web design. Large empirical studies also need to be conducted. Recently, there has been some interest in the use of colours on the web (Eibl and Mandl, 2005; Kondratova et al., 2005), but other factors of design need to be assessed from an intercultural perspective as well. This kind of empirical research needs to be more transparent. In the future, the development of an open corpus of web pages relevant for intercultural research would be desirable.

Apart from web design, the current popularity of social software which supports the interaction between users (e.g. blogs, wikis, networking tools, sharing systems for photographs, recommendations, etc.) requires intercultural studies as well. For social computing, the social aspect is central. Because the way the people interact within a society is highly culturally dependent, different tools and functionality should be provided for each culture. For the interaction of users from different cultures, further adaptation is necessary.

References Amento, B., Terveen, L. and Hill, W. (2000) ‘Does ‘Authority’ mean quality? Predicting expert

quality ratings of web documents’, Paper presented in the Proceedings of the Annual International ACM Conference on Research and Development in Information Retrieval (SIGIR 2000), Athens (pp.296–303).

Aykin, N. (2005) Usability and Internationalization of Information Technology. Mahwah, New Jersey, NJ: Lawrence Erlbaum Associates.

140 T. Mandl and T. de la Cruz

Barber, W. and Badre, A. (1998) ‘Culturablity: the merging of culture and usability’, Paper presented in the Proceedings of the Fourth Conference on Human Factors and the Web.Available at: http://www.research.att.com/conf/hfweb/proceedings/barber.

Choong, Y-Y., Plocher, T.A. and Rau, P-L.P. (2005) ‘Cross-cultural web design’, in R. Procter and K.-P. Vu (Eds), Handbook of Human Factors in Web Design (pp.284–300). Boca Raton, Florida: CRC Press.

Cooke, A. (1999) Authoritative Guide to Evaluating Information on the Internet. New York and London: Neal-Schuman.

De la Cruz, T. (2003) Kulturelle Unterschiede bei der Bewertung von Internet-Angeboten, Master Thesis, International Information Management. University of Hildesheim. Available at: http://web1.bib.uni-hildesheim.de/edocs/2005/503985538/meta.

Cyr, D., Ilsever, J., Bonanni C. and Bowes, J. (2004) ‘Website design and culture: an empirical investigation’. V. Evers, E. del Galdo, D. Cyr, and C. Bonanni (Eds), Designing for Global Markets 6: Proc Sixth International Workshop on Internationalization of Products and Systems (IWIPS 2004) (pp.33–45).

Dormann, C. and Chisalita, C. (2002) ‘Cultural values in web site design’, Paper presented in the Proceedings of the Eleventh European Conference on Cognitive Ergonomics (ECCE 11),Catania, Italia, September 8–11.

Del Galdo, E. (1996) ‘Culture and design’, in E. Del Galdo and J. Nielsen (Eds), International User Interfaces. New York, NY: Wiley.

Eckensberger, L. and Plath, I. (2003) ‘Möglichkeiten und Grenzen des variablenorientierten Kultur-vergleichs: Von der Kulturvergleichenden Psychologie zur Kulturpsychologie’, in H. Kaelble and J. Schriewer (Eds), Vergleich und Transfer – Komparatistik in den Sozial-, Geschichts- und Kulturwissenschaften (pp.55–99). Frankfurt: Campus.

Eibl, M. and Mandl, T. (2005) ‘An empirical assessment of color use on the www’, Paper presented in the Proceedings of the 11th International Conference on Human-Computer Interaction (HCI Intl.): Las Vegas, 22–27. Juli. Mahwah, NJ; London: Lawrence Erlbaum Associates. Vol. 2 – The Management of Information: E-Business, the Web, and Mobile Computing.

Emerald (2005) ‘Spotlight on Geert Hofstede’, Emerald Now Newsletter. Available at: http://www.emeraldinsight.com/info/about_emerald/emeraldnow/archive/july2001 spotlight.jsp.

Evers, V. (2002) ‘Cross-cultural applicability of user evaluation methods: a case study amongst Japanese, North-American, English and Dutch Users’, Paper presented in the Proceedings of the ACM CHI Conference (pp.740–741).

Gilham, R. (2004) ‘Towards a strategic model of design support for localisation’, V. Evers, E. del Galdo, D. Cyr and C. Bonanni (Eds), Designing for Global Markets 6: Proc Sixth International Workshop on Internationalization of Products and Systems (IWIPS 2004) (pp.103–113).

Graham, L. and Metaxas, P. (2003) ‘Of course it’s true; i saw it on the internet!’: critical thinking in the internet era’, Communications of the ACM, Vol. 46, pp.71–75.

Griffith, T.L. (1998) ‘Cross-cultural and cognitive issues in the implementation of new technology: focus on group support systems and Bulgaria. Interacting with computers’, Special Issue: the Role of Culture in the Globalisation of Human-Computer Systems, Vol. 9, pp.431–447.

Henzinger, M., Motwani, R. and Silverstein, C. (2002) ‘Challenges in web search engines’, ACM SIGIR Forum, Vol. 36, pp.11–22.

Hermeking, M. (2005) ‘Culture and internet consumption: contributions from cross-cultural marketing and advertising research’, Journal of Computer-Mediated Communication, Vol. 11 article 10. Available at: http://jcmc.indiana.edu/vol11/issue1/hermeking.html.

Hodemacher, D., Mandl, T. and Jarman, F. (2005) ‘Kultur und web-design: ein empirischer vergleich zwischen Großbritannien und Deutschland’, in A. Auinger (Ed.), Workshop-Proceedings der 5. fachübergreifenden Konferenz Mensch und Computer, Linz, Austria. September 2005. Österreichische Computer Gesellschaft (pp.93–101).

International differences in web page evaluation guidelines 141

Hofstede, G. (1997) Culture and Organizations: Software of the Mind. London: McGraw-Hill. Hofstede, G. and Hofstede, G.J. (2005) Cultures and Organizations: Software of the Mind. 2., erw.

und überarb. Aufl. New York, NY: McGraw-Hill. Iwai, N. (2005) ‘Japanese general social surveys (2): methodological experiments in administering

the questionnaire, incentives, scales and wording’, ZA-Information, Vol. 57, pp.83–103. Jarman, F. (1998) The Perception of Asia: Japan and the West. Hildesheim: Universitätsbibliothek. Kondratova, I., Goldfarb, I., Gervais, R. and Fournier, L. (2005) ‘Culturally appropriate web

interface design: web crawler study’, Paper presented in the Proceedings of the 8th IASTED International Conference on Computer and Advanced Technology in Education (CATE 2005).Oranjestad, Aruba. ACTA Press, Anaheim/Calgary/Zurich. August 29–31 (pp.359–364).

Law, E. and Hvannberg, E. (2004) ‘Analysis of combinatorical user effect in international usability tests’, Paper presented in the Proceedings of the SIGCHI conference on Human factors in computing systems (ACM Press) Vienna, Austria (pp.9–16).

Mandl, T. (2006) ‘Implementation and evaluation of a quality based search engine’, Paper presented in the Proceedings of the 17th ACM Conference on Hypertext and Hypermedia (HT ‘06) Odense, Denmark, August 22nd–25th. ACM Press. (pp.73–84). Available at: http://doi.acm.org/10.1145/1149941.1149957.

Marchand, D. (1990) ‘Managing information quality’, in I. Wormell (Ed.), Information Quality, Definitions and Dimensions. Paper presented in the Proceedings of the NORDINFO Seminar,Copenhagen. Los Angeles, USA: Taylor Graham (pp.7–17).

Marcus, A. and Gould, E. (2000) ‘Cultural dimensions and global web user-interface design: what? So what? Now what?’ Paper presented in the Proceedings of the 6th Conference on Human Factors and the Web in Austin, Texas, 19 June.

Miltsakaki, E. and Kukich, K. (2004) ‘Evaluation of text coherence for electronic essay scoring systems’, Natural Language Engineering, Vol. 10, pp.25–55.

Schmitz, K. (2005) Kulturabhängige Unterschiede bei der Benutzung und Bewertung von Websites am Beispiel von Deutschland und Taiwan. Master Thesis, International Information Management. University of Hildesheim. Available at: http://web1.bib.uni-hildesheim.de/ edocs/2006/505460009/meta.

Sheppard, C. and Scholtz, J. (1999) ‘The effects of cultural markers on web site us’, Paper presented in the Proceedings of the 5th Conference on Human Factors and the Web, 3 June. Gaithersburg, Maryland. Available at: http://zing.ncsl.nist.gov/hfweb/proceedings/sheppard.

Sturm, C. (2005) ‘Approaches for a successful product localization’, in A. Auinger (Ed.), Workshop-Proceedings der 5. fachübergreifenden Konferenz Mensch und Computer, Linz, Austria. September 2005. Österreichische Computer Gesellschaft. Available at: http://mc.informatik.uni-hamburg.de/mc2005/download/ergebnisse/workshops/WS8_B1.pdf.

Trompenaars, F. and Hampden-Turner, C. (1997) Riding the Waves of Culture: Understanding Cultural Diversity in Business. London: Nicholas Brealey.

Page, L., Brin, S., Motwani, R. and Winograd, T. (1998) The PageRank Citation Ranking: Bringing Order to the Web. Manuscript. Available at: http://citeseer.csail.mit.edu/page98pagerank.html.

Price, R. and Shanks, G. (2004) ‘A semiotic information quality framework’, IFIP WG 8.3 International Conference on Decision Support Systems (DSS), Prato, Italien (pp.658–672).

Radford, M., Barnes, S. and Barr, L. (2002) Web Research: Selecting, Evaluating, and Citing.Boston, MA: Allyn and Bacon.

Red Científica Peruana (2002) ‘Contenidos y Servicios’ Available at: http://www.yachay.com.pe/ especiales/internet/index3.htm.

Romberg, M., Röse, K. and Zühlke, D. (1999) Global Demands of non-European Markets for the Design of User-Interfaces. MMI-Interaktiv Nr. 1, March.

142 T. Mandl and T. de la Cruz

Telefónica del Perú (2002) ‘La Sociedad de la información en el perú: presente y perspectivas 2003–2005’, Lima: Telefónica del Perú.

de Wit, F., Diehl, J. and Arts F. (2005) ‘How to design cultural appropriate web sites for knowledge transfer: understanding preferences’, in D. Day, E. del Galdo and V. Evers (Eds), Designing for Global Markets 7: Proceedings of the Seventh International Workshop on Internationalization of Products and Systems (IWIPS 2005), Amsterdam, The Netherlands 7–9 July (pp.143–158).